Subtitld project

Anatomy of a file

A .usfx is a ZIP. Any archive tool opens it, renamed .zip if the tool insists.

Tears of Steel.usfx
subtitles.usf
manifest.xml
thumbnail.png
assets/speakers/Celia.png
assets/dubs/77a03e27.wav
assets/video/Tears_of_Steel.mkv
assets/waveform.npy
assets/audio/vocals.flac

The document says what happens; the assets are what it plays. A dub in subtitles.usf names its clip by its path in the package, assets/dubs/77a03e27.wav, so the package can move to another folder, disk or computer and still be whole.

The members

Names in assets/ are made of letters, digits, -, _ and .; anything else becomes _, so Tears of Steel.mkv is stored as Tears_of_Steel.mkv. Letters outside ASCII are kept, the name written in UTF-8 and flagged as such (bit 11 of the ZIP header), as APPNOTE asks.

The first bytes

Every project starts the same way: a ZIP local file header, then the name of its first member.

50 4B 03 04
14 00 00 00 08 00
4E 70 51 5D
64 CE 1E 0B
6F 3B 00 00 7F E2 00 00
0D 00 00 00
73 75 62 74 69 74 6C 65 73 2E 75 73 66

The name is never compressed, even when the member is. That makes it a signature: a ZIP whose name at byte 30 is subtitles.usf is a Subtitld project. Haiku's Tracker tells projects from other archives by exactly that rule, 0.60 ("PK\003\004") [30] ("subtitles.usf" | "manifest.xml"), ranked above the plain ZIP rule. ODF and EPUB do the same with a first member named mimetype.

A video can be larger than ZIP's original limit of 4 GiB. From 2 GiB on, Subtitld writes Zip64's extra fields and end records (APPNOTE section 4.5), as it does past 65,535 members; Subtitld, its Windows thumbnail handler and any current unzip read them.

The manifest

<?xml version="1.0" encoding="UTF-8"?>
<manifest version="1.0">
  <format>USFX</format>
  <generator>Subtitld</generator>
  <source path="/home/ana/Videos/Tears of Steel.mkv"
          basename="Tears of Steel.mkv"/>
  <voicemix volume="0.5000"/>
  <entry path="subtitles.usf" type="application/x-usf+xml"/>
  <entry path="thumbnail.png" type="image/png"/>
  <entry path="assets/speakers/Celia.png" type="image/png"/>
  <entry path="assets/dubs/77a03e27.wav" type="audio/wav"/>
</manifest>

The types are application/x-usf+xml, image/png, audio/wav, audio/flac, audio/mpeg, video/mp4, video/x-matroska, video/quicktime, video/webm, video/ogg, and application/octet-stream for the waveform and anything else.

The document

subtitles.usf is a Universal Subtitle Format file, UTF-8, as Subtitld writes it on its own, with what a project needs added in USF's own style: elements and attributes where USF has none. A USF reader that skips what it does not know reads its subtitles.

<USFSubtitles version="1.0"
              xmlns:subtitldcc="https://subtitld.cc/ns/usf/1">
  <metadata>
    <title>Tears of Steel</title>
    <author><name>Subtitld</name></author>
    <language code="eng">English</language>
    <subtitldcc:origin share-id="Xk3pQ9aZ2bT" version="3"/>
  </metadata>
  <styles>
    <style name="Celia">
      <fontstyle color="#e5484d"/>
      <dubbing engine="edge-tts" voice="en-GB-SoniaNeural"
               rate="0" pitch="0" fit_to_subtitle="true"/>
    </style>
  </styles>
  <subtitles>
    <language code="eng">English</language>
    <subtitle start="00:00:23.000" stop="00:00:24.500">
      <text speaker="Celia" style="Celia">You're a jerk, Thom.</text>
      <dubbing uid="77a03e27" engine="edge-tts"
               voice="en-GB-SoniaNeural" rate="0" pitch="0"
               path="assets/dubs/77a03e27.wav"
               start="23.0" end="24.5"/>
    </subtitle>
  </subtitles>
  <subtitles>
    <language code="por">Portuguese (Brazil)</language>
    <subtitle start="00:00:23.000" stop="00:00:24.500">
      <text speaker="Celia" style="Celia">Você é um idiota, Thom.</text>
    </subtitle>
  </subtitles>
</USFSubtitles>

What Subtitld adds to USF

In a plain .usf, speakers' pictures and dubs' audio can travel inside the XML, as base64 in <avatar> and <data>. A .usfx never does that: they are members of their own, a quarter smaller than in base64, and read without parsing the XML.

Times

A subtitle's start and stop are USF times, rounded to the nearest millisecond. A dub's times are plain seconds written in full, 18.01579117916244: the shortest decimal that reads back as the same 64-bit number, so a clip placed by dragging comes back exactly where it was.

Older Subtitld versions wrote times a millisecond early, and a start at zero as 23:59:59.999. Subtitld reads that value as zero, and leaves the millisecond as it is.

Takes and subclips

A subtitle can have several takes. The first <dubbing> is the one that plays; the others are alternates, silent until you choose one, which then moves to the front.

A take that was split or stretched keeps a list of <subclip>s, placed like clips in an audio editor:

A subclip plays from the take's start plus its offset, for end minus start seconds. Gaps are silence; overlaps are mixed. The sources are never changed, so every split and stretch can be undone, and a stretch is always remade from raw_path, never from a stretch of a stretch.

Translations

Each language has its own <subtitles> block, named by its ISO 639-2 code and, in the element, the language's name, which tells Brazil's Portuguese from Portugal's. Every block holds every subtitle at the same times; only the first holds dubs. Subtitld matches a translation to its subtitle by start time, or by the one it overlaps most, so blocks written by other tools need not line up.

Opening a project

Big projects open at once: Subtitld reads first what the editor needs, and the rest while you work.

Members are extracted into Subtitld's own folders, never beside the project: the document, faces and dubs into a cache folder named after the MD5 of the project's path. What comes in while you work streams in 1 MiB chunks to a temporary file, renamed into place when complete, so a crash halfway leaves nothing half-written. The next time the same project opens, a file already there is used again only if it is exactly the member's size; otherwise it is extracted again.

The video is found in this order:

1. a video inside the package, in assets/video/;
2. the path in <source path>, if it is still there;
3. a file named as in <source basename>, in the project's folder;
4. otherwise, Subtitld asks for it.

A file that is not a ZIP, is cut short, or fails a CRC check opens nothing: Subtitld says so and takes it off the recent list. Earlier projects open too: one without a manifest, or with its USF under another name at the root of the ZIP.

Saving a project

Save never puts the project you have at risk.

1. A snapshot. The project is copied as it is when you save, so you can keep editing while it is written. Saves, autosaves and backups take turns: one writes at a time.
2. A new file. The package is written to <project>.usfx.tmp beside the project.
3. Nothing missing. A dub whose audio is not in the cache yet, still streaming in or cleared, is copied from the file about to be replaced. A project saved without a video keeps the picture it had.
4. Checked. The new file is opened again and every member's CRC-32 verified.
5. Swapped. Only then does it replace the project, in one rename (os.replace): the project's name never holds half a project.

If any step fails, the temporary file is deleted, the project on disk is the one you had, and your edits are still open in Subtitld. While a project's dubs are still streaming in, Save waits for them.

What goes inside

Global settings, then USFX, chooses what else a project carries, for every project you save:

• Speaker portrait imagesThe speakers' faces. On.
• Timeline waveform cacheThe waveform, so a project opens on another computer without computing it again. Off.
• Original audio trackThe video's sound, as FLAC. Off.
• Processed audio (vocals + background)The sound separated into voices and the rest, as FLAC, so separation need not run again. Off.
• Original video fileThe video itself: one file with everything, as big as the video and more. Off.

Each goes in when Subtitld has one to put in: the separated sound, for one, only once separation has run. The subtitles, translations, speakers, voices and every dub are always inside. Deflate takes about a sixth off a dub's WAV; a video and FLAC hardly shrink.

Autosave and backups

• Autosave the project fileWrites your .usfx itself, as Save does, the same safe way. For a project saved as USFX.
• Automatic backupsA complete .usfx in Subtitld's backup folder, apart from your own file, named <project>_<YYYYMMDDHHMMSS>.usfx, when there are changes to keep. Past Keep at most, 20 unless you choose, the oldest are deleted.

In file managers

The thumbnail is a card in Subtitld's look: the video's frame where the project was left, the subtitle on it, and under it the twelve seconds of timeline around the playhead, the waveform and the subtitles in their speakers' colours. 4:3 at 512 × 384, so it fills a file manager's square cell better than a wide frame. Without a video, Subtitld's mark stands in for the frame.

Double-clicked, a project opens in Subtitld on all four. Both thumbnailers read only thumbnail.png, refuse one over 8 MiB, and take nothing that does not start with PNG's signature, 89 50 4E 47 0D 0A 1A 0A. The Windows one takes it only stored, as Subtitld writes it. The Linux one uses Python's standard library alone, as file managers run thumbnailers sandboxed and many at once.

Reading a .usfx yourself

Nothing in a project is Subtitld's alone. To look inside:

unzip -l "Tears of Steel.usfx"
unzip -p "Tears of Steel.usfx" manifest.xml

To read its subtitles, in every language, with Python's standard library:

import zipfile
import xml.etree.ElementTree as ET

def text_of(element):
    """A <text>'s words, <br/> as a line break."""
    parts = [element.text or '']
    for child in element:
        parts.append('\n' if child.tag == 'br' else text_of(child))
        parts.append(child.tail or '')
    return ''.join(parts)

with zipfile.ZipFile('Tears of Steel.usfx') as usfx:
    usf = ET.fromstring(usfx.read('subtitles.usf'))

for block in usf.iter('subtitles'):
    language = block.find('language')
    if language is not null:
        print('#', language.get('code'), language.text)
    for subtitle in block.iter('subtitle'):
        text = subtitle.find('text')
        print(subtitle.get('start'), subtitle.get('stop'),
              text.get('speaker'), text_of(text))

For a project from someone else, parse with defusedxml in place of xml.etree, as with any XML you did not write.

Making one

The smallest project Subtitld opens is a ZIP holding a subtitles.usf. With no manifest, Subtitld asks for the video. To make one that file managers know too, follow what Subtitld writes:

• subtitles.usf first, then manifest.xml;
• every member compressed with Deflate, but thumbnail.png, stored;
• dubs under assets/dubs/, named in the document by their path in the package;
• UTF-8 everywhere.

How it compares

USFX is built as the document packages of office suites and e-books are: a ZIP, a main document in XML, an inventory, the media beside them.

The main document is an open format in its own right: take a .usfx's subtitles.usf out and it is a USF file, a format with a public specification since 2002.

Trivia