VoxMel offline, voice clone, 2 languages

VoicesOfflineGPU optionalVoice cloningCommercial use OK

Dubs your subtitles in English and Mandarin with VoxDance Lab's VoxMel, in each speaker's own voice cloned closely from the video and its subtitles, or in one of eight preset voices. It runs on your own computer, and the add-on, the model and the voices are all Apache-2.0.

Install it from Subtitld

Settings Add-ons VoxMel (offline, voice clone, 2 languages) Install

Subtitld downloads it, checks its SHA-256, and keeps it updated.

Source on GitHub
Languages
2
Models
1
Voices
9
Add-on download
15–19 MB
RAM, at least
2 GB

Listen

What it does

Close clones

Subtitld cuts a few seconds of each speaker from the video and sends them with the subtitles spoken in them. VoxMel clones that voice closely, in English or Mandarin.

Eight preset voices

Two women and two men for each language, for speakers you don't want to clone. VoxMel made them itself: no person's voice was recorded for them.

Quick for a cloning model

A cloned line takes two to four seconds per second of speech on a recent laptop processor.

Free for commercial work

The add-on, the audio.cpp engine, the VoxMel weights and the preset voices are all Apache-2.0.

Before you install

  • Disk: about 1.2 GB, most of it the 1.04 GB model, downloaded on the first line you dub.
  • Memory: about 2 GB free.
  • Time: on an Intel Core i7-13700T, a line in the speaker's own voice took two to four seconds per second of speech. On an Apple Silicon Mac it runs on the GPU.
  • Languages: English and Mandarin only. For other languages, VoxCPM2 clones voices in 30 and OmniVoice in 646.

Voices

In the Speakers tab, choose VoxMel as a speaker's Dubbing engine, then pick a Voice. Turn on Fit dub to subtitle duration for the speaker to stretch or squeeze each line to its subtitle: VoxMel itself has no control of pace.

9 voices in 2 languages.

  • en 4
  • zh 4

The cloned voice

With Speaker's own voice, Subtitld cuts a sample of that speaker from the project's audio, from one of their subtitles of 3 to 12 seconds, or several short ones joined, and sends the subtitle text with it. VoxMel needs both the sample and its words. The Fallback reference clip in the add-on's settings, with its transcript, is used only when no sample can be cut.

VoxMel clones best within a language. An English dub of a Mandarin speaker works well; a Mandarin dub of an English speaker may slur some words.

Clone the timbre only doesn't work with this add-on, because VoxMel can't clone without the words of the sample. Leave it off for speakers who use VoxMel.

Preset voices

Each preset is a short clip that VoxMel made without any reference recording, kept with its exact words. Every line of a speaker given a preset clones that clip, so the voice stays the same through the video. English presets suit English lines and Mandarin presets Mandarin lines.

Models

There is one model, VoxMel 0.6B with its Vocos vocoder, its weights stored in 8 bits, as the audio.cpp project converted it. It is not inside the add-on: it downloads into Subtitld's shared model cache the first time you dub, from Hugging Face at a pinned revision, and is checked. An interrupted download resumes where it stopped. Updating or removing the add-on keeps the model.

Model DownloadGood for
VoxMel 0.6B (8-bit) Default voxmel
992 MB
Every voice, in English and Mandarin

The first line also loads the model, so it takes longer than the ones after it. On an Apple Silicon Mac, the very first time, the add-on also speaks one short test line on the GPU, and uses the CPU if that doesn't work. Cancel stops a line between two steps, or stops the download.

Options

In Settings › Add-ons › VoxMel (offline, voice clone, 2 languages) › Configure.

Quality steps
Default: 32

How finely VoxMel draws each moment of speech. 32 is VoxMel's own setting; fewer is faster, with less detail.

Text guidance
Default: 3.5

How closely the speech follows the text. 3.5 is VoxMel's own setting.

Voice guidance
Default: 3

How closely a cloned voice follows the clip. 3.0 is VoxMel's own setting; higher can sound closer, or strained.

CPU threads
Default: 0

0 picks the performance cores on CPUs that also have efficiency cores. Fewer threads keep the computer more responsive while it dubs.

Fallback reference clip
Default: empty

A clip for the cloned voice when Subtitld can't cut one from the video, for example with no video open: 3 to 15 seconds of one person speaking.

Fallback reference transcript
Default: empty

Exactly what is said in the fallback clip, word for word. VoxMel clones from the clip and its words, so the fallback clip is refused without it.

Languages

English and Mandarin Chinese. VoxMel reads the language from the text.

  • en
  • zh

Platforms

System DownloadTested
Linux x86-64 18.8 MBDubbed with the real model on a real machine and on AlmaLinux 8; in CI, the installed add-on downloads the model, speaks, clones a voice and cancels a line
macOS (Apple silicon) 18.1 MBIn CI, the installed add-on downloads the model, speaks, clones a voice and cancels a line (Apple Silicon)
Windows x86-64 15.4 MBIn CI, the installed add-on downloads the model, speaks, clones a voice and cancels a line

Linux x86-64 (any distribution from 2018 on, glibc 2.28 or newer), macOS 13.3 or newer on Apple Silicon, and Windows x86-64. Each release is built on all three, and on each CI installs the add-on, downloads the model, and has it speak, clone a voice and cancel a line.

Troubleshooting

  • network_unavailableThe model couldn't be downloaded. Check your connection and try again: the download resumes.
  • disk_fullNot enough disk space for the model. It needs about 1.2 GB free.
  • oomOut of memory. Close other programs; VoxMel needs about 2 GB free.
  • bad_paramsThe line has no text, or the cloned voice got no sample or no words with it. The message says which. For a cloned voice, open the video and leave Clone the timbre only off.
  • unsupported_voiceThe project names a voice this version doesn't have. Pick the voice again in the Speakers tab.
  • internalSomething failed inside the add-on; the message has the details.

License

Add-on
Apache-2.0
with its eight preset voices
Engine
Apache-2.0
audio.cpp, with ggml (MIT)
Model
Apache-2.0
VoxMel 0.6B weights, by VoxDance Lab; Vocos vocoder MIT

Fine for commercial work: the add-on, its preset voices, the audio.cpp engine and the VoxMel weights are all Apache-2.0, and the Vocos vocoder and ggml are MIT. The dubs you make are yours. A cloned voice sounds like a real person: clone voices you have the right to use, and say that a dub is synthetic where it matters.

Versions

  • 26.11.1

    The first release: text-to-speech for Subtitld with VoxDance Lab's VoxMel 0.6B, in English and Mandarin.

    • The speaker's own voice, cloned closely from the clip Subtitld cuts from the video and its subtitles. VoxMel needs the clip's words, so "Clone the timbre only" doesn't work with this add-on. It clones best within a language: an English dub of a Mandarin speaker works well, a Mandarin dub of an English speaker may slur some words.
    • Eight preset voices, two women and two men in each language. VoxMel made them itself, with no reference recording, and every line of a speaker clones the same clip, so the voice stays the same.
    • Runs on the CPU, or on the GPU of an Apple Silicon Mac once a short test line shows that works there. A cloned line takes two to four seconds per second of speech on a recent laptop CPU.
    • The 1.04 GB model downloads on the first line, pinned and checked; about 2 GB of memory. The archives are about 17 MB, and the Linux one runs on glibc 2.28 and newer.

    The add-on, its preset voices, audio.cpp and the VoxMel weights are Apache-2.0; the Vocos vocoder, ggml, cJSON, libyaml, utf8proc and llama.cpp's tokenizer are MIT. Each archive carries THIRD_PARTY_LICENSES/, with the one change we made to audio.cpp (a step callback for cancel and progress).

Other Voices add-ons