Resolved

MoviePy frame rendering bottleneck limits video generation to 7-8 fps

Python
MoviePy
Performance
Video Processing
Multiprocessing
FFmpeg
Rendering
Backend

Single-threaded MoviePy `make_frame` callback causes 90% CPU underutilization (3% usage on 32-core) | 15-minute render times for 7-minute videos with no parallelization options

Issue

A video generation pipeline using Python’s MoviePy library suffered from severe performance bottlenecks during frame rendering. The system achieved only 7-8 frames per second (fps) despite hardware capable of significantly higher throughput. This resulted in 3% CPU utilization on a 32-core Intel 13900K processor and 15-minute processing times for 7-minute 1080p videos. The bottleneck stemmed from the sequential execution of the make_frame callback, which recalculated tokenization, layout, and scroll positions for every single frame without any parallelization.

Culprit

MoviePy’s architecture enforced sequential execution through its callback-based design, leading to the following inefficiencies:

  1. Single-threaded frame processing: The make_frame(t) function executed sequentially, processing one frame at a time.
  2. Shared mutable state: Scroll offsets and segment tracking required sequential updates between frames, preventing parallelization.
  3. Redundant per-frame calculations: Tokenization, syntax highlighting, and character positioning were recalculated for every frame, introducing unnecessary computational overhead.
  4. Image processing in hot paths: Image manipulation operations occurred during frame generation instead of being precomputed.

Solution

A three-phase parallel rendering architecture was implemented to address these bottlenecks:

1. Pre-calculation Phase (Sequential)

  • Tokenization and syntax highlighting: Performed a single pass to process and highlight tokens for all segments.
  • Absolute positioning: Precomputed the positions, colors, and fonts for all characters.
  • Frame metadata: Generated scroll offsets and visible character ranges for all frames in advance.

2. Parallel Rendering Phase (Multiprocessing)

  • Pure function design: Redesigned the rendering function (render_single_frame) to operate without shared state, enabling parallel execution.
  • Full CPU utilization: Leveraged all 32 cores to render frames simultaneously, achieving 104 fps (up from 7-8 fps).

3. Direct FFmpeg Integration

  • Bypassed MoviePy overhead: Wrote raw frames directly to FFmpeg’s input stream, reducing latency and enabling parallel encoding.
  • Parallel encoding: Utilized FFmpeg’s multithreaded capabilities to encode frames at 312 fps.
  • Configurable fallback: Introduced a flag to toggle between parallel and sequential rendering modes.

Performance Improvements

  • Rendering speed: Achieved 492 frames in 4.7 seconds (104 fps).
  • Encoding speed: FFmpeg encoding completed in 1.6 seconds (312 fps).
  • Overall speedup: 13x faster than the original sequential implementation, with identical output quality.

The solution eliminated the sequential bottlenecks while ensuring pixel-perfect rendering consistency.