Distributed Video Processing

When to Distribute

Long Videos

Split by time segments:

• 10-minute video → 10 x 1-minute segments
• Process segments in parallel
• Merge outputs

High Resolution

Split by spatial tiles:

• 8K frame → 16 x 2K tiles
• Process tiles in parallel
• Stitch with overlap blending

Multi-Stage Pipelines

Pipeline parallelism:

• Stage 1: Decode (CPU)
• Stage 2: Enhance (GPU 1)
• Stage 3: Face restore (GPU 2)
• Stage 4: Encode (CPU/GPU)

Stages run concurrently on different frames.

Segment-Based Parallelism

The Pattern

Video → Split by time → Parallel process → Merge

Segment 1 → GPU 1 → Output 1 ↘
Segment 2 → GPU 2 → Output 2 → Concatenate → Final
Segment 3 → GPU 3 → Output 3 ↗

Segment Boundaries

Challenge: AI models need context across boundaries.

• Split at scene changes (natural boundaries)
• Overlap segments and blend
• Use temporal smoothing models

Keyframe Alignment

Split at keyframes for clean cuts:

ffprobe -select_streams v -show_entries packet=pts_time,flags \
  -of csv video.mp4 | grep K

Tile-Based Parallelism

The Pattern

Frame → Split into tiles → Parallel process → Stitch

Tile 1 → GPU 1 → Enhanced 1 ↘
Tile 2 → GPU 2 → Enhanced 2 → Stitch → Final frame
Tile 3 → GPU 3 → Enhanced 3 ↗
Tile 4 → GPU 4 → Enhanced 4 ↗

Overlap for Blending

Tiles must overlap to avoid seams:

• Overlap: 32-64 pixels
• Blend in overlap region
• Linear or sigmoid blending

When to Use

• Frames larger than GPU memory
• 8K, 16K processing
• Memory-limited GPUs (T4 for 4K)

Coordination Challenges

Scatter-Gather

Coordinator splits work and collects results:

Coordinator:
1. Split video into N segments
2. Dispatch to N workers
3. Wait for all to complete
4. Merge outputs
5. Report completion

Handling Failures

• Worker failure: Reassign segment to another worker
• Partial failure: Retry only failed segments
• Coordinator failure: Persist state, resume from checkpoint

Load Balancing

Segments may have different complexity:

• Static scenes: fast
• Complex motion: slow

Solutions:

• Smaller segments for better balance
• Work stealing: fast workers take from slow
• Adaptive sizing based on content analysis

BetterVideo's Approach

We handle distribution transparently:

Automatic Segmentation

• Videos over 5 minutes are segmented
• Split at scene changes when possible
• Overlap and blend at boundaries

Automatic Tiling

• 4K+ frames are tiled
• Tile size based on GPU memory
• Seamless blending

What You See

• One job submission
• One status endpoint
• One output file

Distribution is an implementation detail.