Programmatic Video Subtitles: Word Timing, Highlighting, and Layout

Captions improve comprehension, accessibility, and silent autoplay performance, but automated subtitles need more than plain text. The renderer needs word timing, a layout strategy, and styles that remain readable across changing backgrounds.

Use word-level timing

A subtitle word contains the normalized word, display punctuation, start time, and end time.

{
  "word": "build",
  "punctuated_word": "Build",
  "start": 0.2,
  "end": 0.56
}

Word timing lets the renderer highlight the active word, reveal speech progressively, and keep visual changes synchronized with narration.

Choose a subtitle source

RenderingVideo supports three useful forms:

• An inline array for small, self-contained schemas.
• A reusable subtitle asset referenced with { "$ref": "subtitle-main" }.
• An external JSON source referenced with { "src": "https://..." }.

External endpoints should return either the word array directly or an object with a words array. The public preview flow normalizes this data before playback.

Batch or stream mode

Batch mode groups several words and highlights the currently spoken word. It works well for polished social clips where the viewer should read a short phrase at a time.

Stream mode reveals words sequentially. It creates a more immediate transcription feel and can suit demos or conversational content.

Use wordsPerBatch to control density. Short vertical videos often work with fewer words per batch than desktop explainers.

Design for readability

Start with position, maximum text width, font size, and contrast. A semi-transparent background or shadow can protect captions from busy footage. Keep lines away from mobile controls and platform overlays.

{
  "type": "subtitle",
  "start": 0,
  "duration": 8,
  "words": { "$ref": "subtitle-main" },
  "config": {
    "mode": "batch",
    "wordsPerBatch": 4,
    "position": "bottom",
    "fontSize": 60,
    "textColor": "#FFFFFF",
    "highlightColor": "#7DD3FC",
    "backgroundColor": "rgba(0,0,0,0.45)"
  }
}

Test the longest expected phrase, not only the average one. Localized text can expand significantly, and names or numbers may not break where you expect.

Separate caption motion from word highlighting

config.fadeInAnimation controls fade behavior inside subtitle batches. Clip-level animations, keyframes, and transition affect the whole subtitle container. Keeping those concepts separate prevents an entrance animation from disrupting per-word emphasis.

Build a repeatable pipeline

A scalable caption workflow is:

1. Transcribe narration into timestamped words.
2. Normalize punctuation and ensure times increase monotonically.
3. Store the data as a subtitle asset or public JSON response.
4. Apply a brand caption preset.
5. Generate a preview and inspect line wrapping on every target ratio.
6. Render only after timing and safe areas pass review.

Read the complete subtitle element reference and inspect the captioned example on the homepage.