Wav2Lip (offline lip-sync)
Redraws the mouth in a talking-head video so it follows a new audio track, such as your dub. It runs on the CPU, with no PyTorch, and works offline once its two model files have downloaded. The weights are for research and non-commercial use only.
Settings Add-ons Wav2Lip (offline lip-sync) Install
Subtitld downloads it, checks its SHA-256, and keeps it updated.
- Add-on download
- 176–274 MB
What it does
In every frame it finds the largest face, redraws its mouth to match the sound and blends the patch back with a soft edge.
It needs a face in each frame. One frame where it can't find one, such as a cutaway, stops the job.
It writes a new video, H.264 with AAC sound, as long as the audio: a shorter track cuts the video, a longer one replays it from the first frame.
It reads every frame into memory before it starts, so long or high-resolution clips need a lot of RAM. Resize factor reduces it.
Not for commercial work. The Wav2Lip weights were trained on LRS2 and are licensed for research and non-commercial use only, and so are the SCRFD face-detector weights. The add-on's MIT license covers its own code, not them.
Nothing in Subtitld runs it yet. You can install it, but no command in the app sends it a video to lip-sync so far.
Models
There is nothing to choose. Two files download the first time it runs, then everything stays on your computer: wav2lip_256.onnx, the lip-sync model (about 200 MB), and scrfd_2.5g_bnkps.onnx, the face detector (about 3 MB).
A file already in the folder is used as it is. On a computer without a connection, download both yourself and put them there:
Where models are kept
Outside the add-on, so updating or uninstalling it keeps them; delete the folder to free the space.
| System | Folder |
|---|---|
| Linux | ~/.cache/subtitld/wav2lip |
| macOS | ~/.cache/subtitld/wav2lip |
| Windows | %USERPROFILE%\.cache\subtitld\wav2lip |
Options
In Settings › Add-ons › Wav2Lip (offline lip-sync) › Configure.
0 10 0 0Extra pixels added around the detected face box before lip-sync. Increase 'bottom' if the chin is cut off.
1Shrinks every frame by this factor first, and the result keeps the smaller size. 2 halves the width and height: faster, less memory, lower resolution.
0.3How sure the face detector must be before it accepts a face. Lower it if a job stops at a frame where no face was found.
Platforms
| System | Download |
|---|---|
| Linux x86-64 | 273.6 MB |
| macOS (Apple silicon) | 202.4 MB |
| Windows x86-64 | 175.9 MB |
The builds are made in CI, without running a lip-sync.
Troubleshooting
bad_paramsSomething in the input stopped it, and the message says what. Most often it's a frame with no face found: lower Face detection threshold, or use a clip where the face is always in view. Silent or very short audio stops it too.model_missingA model couldn't be downloaded from its address. The message gives the file, the folder and the address: download it yourself and put it there.network_unavailableThere was no connection while the models downloaded. Try again online; once both files are in, it never needs the network.internalAnything else, a cancelled job included. On a long or high-resolution video, raise Resize factor: the whole clip is held in memory.
License
Non-commercial use only. The add-on's code is MIT, but the Wav2Lip and SCRFD weights it downloads are for research and non-commercial use, and following their licenses is up to you.
Versions
- 0.1.0