Resolved

Pygments syntax highlighting shows grey variables instead of colors

Python
Syntax Highlighting
Pygments
VSCode
Video Generation
Frontend
Debugging
Theming

Python code variables render grey in video generation despite Ayu Mirage theme configuration | Pygments token mapping issues cause visual inconsistency in syntax highlighting

Issue

The automated video generation system rendered Python code with visually indistinguishable grey syntax highlighting, despite using the Ayu Mirage theme’s color palette. Critical elements such as exception classes, class names, and decorators appeared grey instead of their expected theme colors, creating a flat visual appearance. Additionally, turquoise-colored code segments incorrectly displayed as green in the final output due to a rendering bug.

Culprit

The root causes included:

  1. Incomplete token-to-color mappings: Generic variables were mapped to a grey scope for visual neutrality, while other token types lacked proper mappings or were incorrectly mapped to non-existent theme scopes.
  2. Rendering pipeline bug: The progressive coloring logic incorrectly applied markdown text coloring rules to all turquoise-colored elements, including code segments, due to the absence of language context checks.
  3. Limited comment detection: The system only recognized Python-style comments (#), failing to handle JavaScript/TypeScript/C-style comments (//), resulting in ungrouped lines in the output.

Solution

To address these issues, the system was updated with the following improvements:

Token Color Mapping Overhaul

  • Variables: Overrode the default grey mapping for variables to use turquoise, ensuring visual distinction.
  • Comprehensive Token Mappings: Added mappings for previously unmapped token types, such as exception classes, class names, and decorators, to their respective Ayu Mirage theme scopes.

Rendering Pipeline Fix

  • Language Context Checking: Introduced checks to restrict markdown text coloring rules to text-only segments, preventing incorrect application to code segments.

Extended Comment Support

  • Comment Detection: Enhanced comment detection to recognize both # and // styles, ensuring proper grouping of commented lines in the output.

Results

The improvements resulted in visually distinct syntax highlighting for all token types, including:

  • Generic variables and magic variables: turquoise.
  • Exception classes and instance variables: turquoise or salmon.
  • Class names: blue.
  • Decorators: peach.

This solution ensures that the generated videos display code with accurate and visually appealing syntax highlighting, adhering to the Ayu Mirage theme’s intended color scheme.