YouTube video queue starves X video rendering for hours

Issue

The video rendering system experienced unfair resource allocation where YouTube videos monopolized GPU access via a shared VideoLock, causing X videos to wait up to 8+ hours even when only 2 X videos were pending. This occurred because YouTube’s sequential processing held the lock continuously, preventing X videos from rendering until the entire YouTube batch completed.

Culprit

The root cause was a lack of coordination between independent watcher queues (watcher_youtube.py and watcher_x.py). The system relied on a shared FIFO lock mechanism but lacked:

1. Inter-watcher communication: Each watcher processed its own queue without checking for pending videos in the other queue.
2. Yield points: YouTube’s processing loop had no mechanism to release the lock and allow X videos to proceed between batches.
3. Lock starvation: YouTube’s continuous lock acquisition prevented X videos from accessing the GPU until all YouTube videos were processed.

Solution

Interleaved Processing Architecture

A new method, _check_and_process_x_video(), was introduced to enable interleaved processing between YouTube and X videos. The solution:

1. Filesystem Polling: After processing each YouTube pair, the system checks the uploads/x/ directory for pending X videos.
2. Direct Processing: If X videos are pending, a temporary XScriptWatcher instance is created to process one X video at a time.
3. Lock Handoff: The GPU lock is released before processing X videos, ensuring fair resource allocation.
4. Controlled Alternation: Processing alternates between YouTube and X videos every 30-60 minutes, reducing X wait time from 8+ hours to under 30 minutes.

Performance Impact

• Reduced X Wait Time: From 8+ hours to <30 minutes in typical scenarios.
• Minimal Overhead: ~3ms per YouTube pair when no X videos are pending.
• Fair GPU Access: Resources are now distributed evenly between content types, preventing monopolization.

The solution maintains lock safety while introducing lightweight coordination through filesystem checks and temporary watcher instantiation.