Resolved

Disconnected automation pipeline requires manual file transfers between systems

Automation
Bash
Python
DevOps
Pipeline
Integration
Video Processing
Scripting

Three independent systems for script generation, video creation, and uploads lacked integration | Users had to manually copy files between each step, creating operational bottlenecks

Working with a Python-based automation stack (script generator, video renderer, and upload watcher) that required manual intervention between each step. The solution implements a Bash orchestrator to chain these independent systems into a single automated pipeline.

Issue

The video generation workflow required five manual steps between three independent systems: running the script generator (orchestrator_v6.py), copying the output to input.md, executing the video renderer (main.py), moving the resulting files to the uploads directory, and ensuring the upload watcher (run_watcher.py) was running. This created operational friction and prevented batch processing.

Culprit

The systems were designed as standalone components without integration points:

  1. orchestrator_v6.py output files to creation/output_v6/
  2. main.py expected input from input.md in root directory
  3. run_watcher.py monitored platform-specific subdirectories in uploads/ No automated handoff existed between these stages, requiring manual file transfers.

Solution

Created automate.sh as a Bash orchestrator with these technical implementations:

Pipeline Chaining

# Step 1: Generate script
cd creation/ && python orchestrator_v6.py --topic "$TOPIC" --url "$URL"

# Step 2: Process video
cp "creation/output_v6/script_*.md" input.md
python main.py --platform "$PLATFORM"

# Step 3: Stage for upload
TIMESTAMP=$(date +%s)
mv output.mp4 "uploads/$PLATFORM/video_${TIMESTAMP}.mp4"
mv thumbnail.jpg "uploads/$PLATFORM/video_${TIMESTAMP}_thumb.jpg"

Key Features

  1. Timestamp-based filenames: Prevents collisions during batch processing
  2. Platform routing: Dynamic directory selection via --platform flag
  3. Metadata generation: Auto-creates JSON with title/description
  4. Error handling: set -e terminates on any failure
  5. Input flexibility: Supports both URL and code snippet inputs

Architecture

[automate.sh] → [orchestrator_v6.py] → [main.py] → [uploads/platform/] → [run_watcher.py]

The solution maintains complete separation of concerns while adding the missing integration layer. The watcher remains a separate process, preserving its existing OAuth and upload logic.