OmniVoice offline, voice clone, 646 languages, non-commercial

VoicesOfflineGPU optionalVoice cloningNon-commercial

Dubs your subtitles in any of 646 languages with k2-fsa's OmniVoice, in each speaker's own voice cloned from the video, even into another language, or in a voice designed from a few traits. It runs on your own computer. The model is for non-commercial use only.

Install it from Subtitld

Settings Add-ons OmniVoice (offline, voice clone, 646 languages, non-commercial) Install

Subtitld downloads it, checks its SHA-256, and keeps it updated.

Source on GitHub
Languages
655
Models
1
Voices
12
Add-on download
14–17 MB
RAM, at least
2.5 GB

What it does

646 languages

From Afrikaans to Zulu, with many languages no other add-on speaks, and regional varieties of Arabic, Chinese and more.

The speaker's own voice, in any language

Subtitld cuts a few seconds of each speaker from the video and sends them with the subtitles spoken in them. OmniVoice clones that voice, and keeps it when the dub is in another language.

Or a designed voice

Ten ready-made voices, from a child to an older man, or one you put together from a few traits: gender, age, pitch, an English accent, a whisper.

Offline after the first line

The model downloads once, the first time a line needs it. After that, nothing leaves your computer.

Non-commercial model. The OmniVoice weights are under CC-BY-NC, which doesn't permit commercial use. The add-on's code is Apache-2.0, but that doesn't change the weights' terms. For commercial work, VoxCPM2 (30 languages) and ZONOS3-Mini (43 languages) also clone voices, and their weights are Apache-2.0.

Before you install

  • Disk: about 1.5 GB, most of it the 1.35 GB model, downloaded on the first line you dub.
  • Memory: about 2.5 GB free.
  • Time: on a CPU each second of speech takes several seconds to make. On an Intel Core i7-13700T, a line in the speaker's own voice took about 6 seconds per second of speech, and a designed voice's first line takes longer, while the add-on makes the voice. On an Apple Silicon Mac it runs on the GPU.

For quick dubs, Piper or Kokoro are much lighter, and Piper also covers many languages.

Voices

In the Speakers tab, choose OmniVoice as a speaker's Dubbing engine, then pick a Voice. Turn on Fit dub to subtitle duration for the speaker to stretch or squeeze each line to its subtitle.

12 voices.

The cloned voice

With Speaker's own voice, Subtitld cuts a sample of that speaker from the project's audio, from one of their subtitles of 3 to 12 seconds, or several short ones joined, and sends the subtitle text with it. OmniVoice needs both the sample and its words. The dub can be in any of its languages, whatever language the sample is in. The Fallback reference clip in the add-on's settings, with its transcript, is used only when no sample can be cut.

Clone the timbre only doesn't work with this add-on, because OmniVoice can't clone without the words of the sample. Leave it off for speakers who use OmniVoice.

Designed voices

OmniVoice makes up a new voice every time it reads a set of traits. So the add-on makes each speaker's voice once, as a short sentence, and every line of that speaker reuses it: the voice stays the same through the video, and two speakers given the same designed voice still sound like two people.

Models

There is one model, OmniVoice, with its weights stored in 8 bits, as the audio.cpp project converted it. It is not inside the add-on: it downloads into Subtitld's shared model cache the first time you dub, from Hugging Face at a pinned revision, and is checked. An interrupted download resumes where it stopped. Updating or removing the add-on keeps the model.

Model DownloadGood for
OmniVoice (8-bit) Default omnivoice
1.3 GB
Every voice, in all 646 languages

The first line also loads the model, so it takes longer than the ones after it. On an Apple Silicon Mac, the very first time, the add-on also speaks one short test line on the GPU, and uses the CPU if that doesn't work. Cancel stops a line between two steps, or stops the download.

Options

In Settings › Add-ons › OmniVoice (offline, voice clone, 646 languages, non-commercial) › Configure.

Quality steps
Default: 16

How many passes OmniVoice makes over each line. 16 is fast and keeps every word; 32, OmniVoice's own setting, takes twice as long and comes a little closer to the speaker.

Guidance
Default: 2

How closely the speech follows the text and the voice. 2.0 is OmniVoice's own setting.

CPU threads
Default: 0

0 picks the performance cores on CPUs that also have efficiency cores. Fewer threads keep the computer more responsive while it dubs.

Voice gender
Default: empty

For the voice "Voice you describe". Leave a trait at Any to let OmniVoice choose it.

Voice age
Default: empty

For the voice "Voice you describe": from child to elderly.

Voice pitch
Default: empty

For the voice "Voice you describe": from very low to very high.

Accent (English)
Default: empty

For the voice "Voice you describe": the accent it speaks English with, such as British or Indian.

Whisper
Default: off

For the voice "Voice you describe": a whispered voice.

Fallback reference clip
Default: empty

A clip for the cloned voice when Subtitld can't cut one from the video, for example with no video open: 3 to 15 seconds of one person speaking.

Fallback reference transcript
Default: empty

Exactly what is said in the fallback clip, word for word. OmniVoice clones from the clip and its words, so the fallback clip is refused without it.

Languages

OmniVoice's own list of 646 languages, including Arabic, Bengali, Chinese and Cantonese, English, French, German, Hindi, Indonesian, Japanese, Korean, Persian, Portuguese, Russian, Spanish, Swahili, Turkish, Urdu, Vietnamese, Yoruba, Zulu and hundreds of regional languages. Subtitld tells it the project's language; when it doesn't know the language, OmniVoice tells it from the text.

  • aae
  • aal
  • aao
  • ab
  • abb
  • abn
  • abr
  • abs
  • abv
  • acm
  • acw
  • acx
  • adf
  • adx
  • ady
  • aeb
  • aec
  • af
  • afb
  • afo
  • ahl
  • ahs
  • ajg
  • aju
  • ala
  • aln
  • alo
  • am
  • amu
  • an
  • anc
  • ank
  • anp
  • anw
  • aom
  • apc
  • apd
  • ar
  • arb
  • arq
  • ars
  • ary
  • arz
  • as
  • ast
  • avl
  • awo
  • ayl
  • ayp
  • az
  • ba
  • bag
  • bas
  • bax
  • bba
  • bbj
  • bbl
  • bbu
  • bce
  • bci
  • bcs
  • bcy
  • bda
  • bde
  • bdm
  • be
  • beb
  • bew
  • bfd
  • bft
  • bg
  • bgp
  • bhb
  • bhh
  • bho
  • bhp
  • bhr
  • bjj
  • bjk
  • bjn
  • bjt
  • bkh
  • bkm
  • bky
  • bmm
  • bmq
  • bn
  • bnm
  • bnn
  • bns
  • bo
  • bou
  • bqg
  • br
  • bra
  • brh
  • bri
  • brx
  • bs
  • bsh
  • bsj
  • bsk
  • btm
  • btv
  • bug
  • bum
  • buo
  • bux
  • bwr
  • bxf
  • byc
  • bys
  • byv
  • byx
  • bzc
  • bzw
  • ca
  • ccg
  • ceb
  • cen
  • cfa
  • cgg
  • chq
  • cjk
  • ckb
  • ckl
  • ckr
  • cky
  • cmn
  • cnh
  • cpy
  • cs
  • cte
  • ctl
  • cut
  • cux
  • cv
  • cy
  • da
  • dag
  • dar
  • dav
  • dbd
  • dcc
  • de
  • deg
  • dgh
  • dgo
  • dje
  • dmk
  • dml
  • dru
  • dty
  • dua
  • dv
  • dyu
  • dzg
  • ebr
  • ebu
  • ego
  • eiv
  • eko
  • ekr
  • el
  • elm
  • en
  • eo
  • es
  • esu
  • et
  • eto
  • ets
  • etu
  • eu
  • ewo
  • ext
  • eyo
  • fa
  • fan
  • fat
  • ff
  • ffm
  • fi
  • fia
  • fil
  • fip
  • fkk
  • fmp
  • fr
  • fub
  • fuc
  • fue
  • fuf
  • fuh
  • fui
  • fuq
  • fuv
  • fy
  • ga
  • gbm
  • gbr
  • gby
  • gcc
  • gdf
  • gej
  • ges
  • ggg
  • gid
  • gig
  • giz
  • gjk
  • gju
  • gl
  • glw
  • gn
  • gol
  • gom
  • gsl
  • gu
  • gui
  • gur
  • guz
  • gv
  • gwc
  • gwe
  • gwt
  • gya
  • gyz
  • ha
  • hah
  • hao
  • haw
  • haz
  • hbb
  • he
  • hem
  • hi
  • hia
  • hkk
  • hla
  • hno
  • hoj
  • hr
  • hsb
  • ht
  • hu
  • hue
  • hul
  • hux
  • hwo
  • hy
  • hz
  • ia
  • ibb
  • id
  • ida
  • idu
  • ig
  • ijc
  • ijn
  • ik
  • ikw
  • in
  • is
  • ish
  • iso
  • it
  • its
  • itw
  • itz
  • iw
  • ja
  • jal
  • jax
  • jgo
  • ji
  • jmx
  • jns
  • jqr
  • juk
  • juo
  • jv
  • ka
  • kab
  • kai
  • kaj
  • kam
  • kbd
  • kbl
  • kbt
  • kcq
  • kdh
  • kea
  • keu
  • kfe
  • kfk
  • kfp
  • khg
  • khw
  • kj
  • kjc
  • kjk
  • kk
  • kln
  • kls
  • km
  • kmr
  • kmy
  • kn
  • kna
  • knn
  • ko
  • kol
  • koo
  • kpo
  • kqo
  • ks
  • ksd
  • ksf
  • kto
  • kuh
  • kvx
  • kw
  • kwm
  • kxp
  • ky
  • kyx
  • lag
  • lb
  • lcm
  • ldb
  • lg
  • lij
  • lir
  • lkb
  • lla
  • ln
  • lnu
  • lo
  • loa
  • lrk
  • lss
  • lt
  • ltg
  • lto
  • lua
  • luo
  • lus
  • lv
  • lwg
  • mab
  • maf
  • mai
  • mau
  • max
  • mbo
  • mcf
  • mcn
  • mcx
  • mdd
  • mde
  • mdf
  • mek
  • mer
  • meu
  • mfm
  • mfn
  • mfo
  • mfv
  • mgg
  • mgi
  • mhk
  • mhr
  • mi
  • mig
  • miu
  • mk
  • mkf
  • mki
  • ml
  • mlq
  • mn
  • mne
  • mni
  • mqy
  • mr
  • mrj
  • mrr
  • mrt
  • ms
  • mse
  • msh
  • msw
  • mt
  • mtr
  • mtu
  • mtx
  • mua
  • mug
  • mui
  • mve
  • mvy
  • mxs
  • mxu
  • mxy
  • my
  • myv
  • mzl
  • nal
  • nan
  • nap
  • nb
  • nbh
  • ncf
  • nco
  • ncx
  • ndi
  • ng
  • ngi
  • nhg
  • nhi
  • nhn
  • nhq
  • nja
  • nl
  • nla
  • nlv
  • nmg
  • nmz
  • nn
  • nnh
  • no
  • noe
  • npi
  • nso
  • ny
  • nyu
  • oc
  • odk
  • odu
  • ogo
  • om
  • orc
  • oru
  • ory
  • os
  • pa
  • pbs
  • pbt
  • pbu
  • pcm
  • pes
  • pex
  • phl
  • phr
  • pip
  • piy
  • pko
  • pl
  • plk
  • plt
  • pmq
  • pms
  • pmy
  • pnb
  • poc
  • poe
  • pow
  • prq
  • ps
  • pst
  • pt
  • pua
  • pwn
  • qug
  • qum
  • qup
  • qur
  • qus
  • quv
  • qux
  • quy
  • qva
  • qvi
  • qvj
  • qvl
  • qwa
  • qws
  • qxa
  • qxp
  • qxt
  • qxu
  • qxw
  • rag
  • rm
  • ro
  • rob
  • rof
  • roo
  • rth
  • ru
  • rup
  • rw
  • sa
  • sah
  • sat
  • sau
  • say
  • sbn
  • sc
  • scl
  • scn
  • sd
  • sei
  • shu
  • si
  • sip
  • siw
  • sjr
  • sk
  • skg
  • skr
  • sl
  • sn
  • snc
  • snk
  • so
  • sol
  • sps
  • sq
  • sr
  • src
  • sro
  • ssi
  • ste
  • sua
  • sv
  • sva
  • sw
  • swh
  • szy
  • ta
  • tan
  • tar
  • tay
  • tbf
  • tcf
  • tcy
  • tdn
  • tdx
  • te
  • tg
  • tgc
  • th
  • the
  • thq
  • thr
  • thv
  • ti
  • tig
  • tio
  • tk
  • tkg
  • tkt
  • tl
  • tli
  • tlp
  • tn
  • tok
  • tpl
  • tpz
  • tqp
  • tr
  • trp
  • trq
  • trv
  • trw
  • tt
  • ttj
  • ttr
  • ttu
  • tui
  • tul
  • tuq
  • tuv
  • tuy
  • tvo
  • tvu
  • tw
  • twu
  • txs
  • txy
  • udl
  • ug
  • uk
  • uki
  • umb
  • ur
  • ush
  • uz
  • uzn
  • vai
  • var
  • ver
  • vi
  • vmc
  • vmj
  • vmm
  • vmp
  • vmz
  • vot
  • vro
  • wbl
  • wci
  • weo
  • wes
  • wja
  • wji
  • wo
  • wof
  • xh
  • xhe
  • xka
  • xmf
  • xmv
  • xmw
  • xpe
  • xti
  • xtu
  • yaq
  • yav
  • yay
  • ydd
  • ydg
  • yer
  • yes
  • yi
  • yo
  • yue
  • zga
  • zgh
  • zh
  • zoc
  • zoh
  • zor
  • zpv
  • zpy
  • zsm
  • ztg
  • ztn
  • ztp
  • zts
  • ztu
  • zu
  • zza

Platforms

System DownloadTested
Linux x86-64 17.3 MBDubbed with the real model on a real machine and on AlmaLinux 8; in CI, the installed add-on downloads the model, speaks, clones a voice and cancels a line
macOS (Apple silicon) 16.6 MBIn CI, the installed add-on downloads the model, speaks, clones a voice and cancels a line (Apple Silicon)
Windows x86-64 14.0 MBIn CI, the installed add-on downloads the model, speaks, clones a voice and cancels a line

Linux x86-64 (any distribution from 2018 on, glibc 2.28 or newer), macOS 13.3 or newer on Apple Silicon, and Windows x86-64. Each release is built on all three, and on each CI installs the add-on, downloads the model, and has it speak, clone a voice and cancel a line.

Troubleshooting

  • network_unavailableThe model couldn't be downloaded. Check your connection and try again: the download resumes.
  • disk_fullNot enough disk space for the model. It needs about 1.5 GB free.
  • oomOut of memory. Close other programs; OmniVoice needs about 2.5 GB free.
  • bad_paramsThe line has no text, the cloned voice got no sample or no words with it, or the voice you describe has no traits set. The message says which. For a cloned voice, open the video and leave Clone the timbre only off.
  • unsupported_voiceThe project names a voice this version doesn't have. Pick the voice again in the Speakers tab.
  • internalSomething failed inside the add-on; the message has the details.

License

Add-on
Apache-2.0
Engine
Apache-2.0
audio.cpp, with ggml (MIT)
Model
CC-BY-NC
OmniVoice weights, by k2-fsa; non-commercial only

The add-on and the audio.cpp engine are Apache-2.0, and ggml, which audio.cpp runs on, is MIT. The OmniVoice weights, by k2-fsa, are CC-BY-NC: what you dub with them is for non-commercial use only. A cloned voice sounds like a real person: clone voices you have the right to use, and say that a dub is synthetic where it matters.

Versions

  • 26.11.1

    The first release: text-to-speech for Subtitld with k2-fsa's OmniVoice.

    • 646 languages, from Afrikaans to Zulu, with many regional languages and varieties.
    • The speaker's own voice, cloned from the clip Subtitld cuts from the video and its subtitles, and kept when the dub is in another language. OmniVoice needs the clip's words, so "Clone the timbre only" doesn't work with this add-on.
    • Ten designed voices (from a child to an older man, a whispering woman) and a voice you describe in Configure from a few traits: gender, age, pitch, an English accent, whisper. Each speaker's voice is made once and kept the same on every line.
    • A speaker's rate sets the pace.
    • Runs on the CPU, or on the GPU of an Apple Silicon Mac once a short test line shows that works there. A cloned line takes about 6 seconds per second of speech on a recent laptop CPU.
    • The 1.35 GB model downloads on the first line, pinned and checked; about 2.5 GB of memory. The archives are about 17 MB, and the Linux one runs on glibc 2.28 and newer.

    The OmniVoice weights are CC BY-NC: what you dub with them is for non-commercial use only. The add-on and audio.cpp are Apache-2.0; ggml, cJSON, libyaml, utf8proc and llama.cpp's tokenizer are MIT. Each archive carries THIRD_PARTY_LICENSES/, with the one change we made to audio.cpp (a step callback for cancel and progress).

Other Voices add-ons