AI evaluator stuck in infinite GOOD rating loop with moving goalposts

An AI-driven evaluator-optimizer system for automated video script generation encountered an infinite loop of refinement. The system, which relied on Python-based components to analyze and refine technical explanations around immutable code samples, used mechanical checklists and iterative feedback to achieve EXCELLENT ratings. However, the system became stuck in a cycle where it repeatedly assigned GOOD ratings while introducing new subjective criteria in each iteration, ultimately preventing the scripts from reaching the desired EXCELLENT rating.

Issue

The evaluator-optimizer loop was stuck in an infinite refinement cycle characterized by:

1. Consistently returning a GOOD rating without ever achieving EXCELLENT
2. Introducing new subjective feedback criteria in each iteration
3. Abandoning previous feedback requirements without verification
4. Terminating after four iterations without reaching EXCELLENT

Key symptoms included:

• Iteration 2/4: GOOD - “In the overview, replace ‘This script exposes a single add tool’ with something more precise.”
• Iteration 3/4: GOOD - “Different issue about explanations—clarify the purpose of the function.”
• Iteration 4/4: GOOD - “Now complaining about repetition (a new issue!)”

Culprit

1. Subjective Rating System

The original criteria relied on vague terms like “insightful” and “educational” to define an EXCELLENT rating, such as:

EXCELLENT: explanations must be insightful and educational without repetition or bloat. This ambiguity allowed the AI to continuously invent new requirements, creating an unending loop.

2. Code vs. Text Confusion

The evaluator attempted to modify immutable code samples instead of focusing on the explanatory text. For example, it would critique code like:

def add(a, b): return a + b

with feedback like:

“This should have error handling”

When the actual issue was misalignment between the code and its explanation.

3. Missing Iteration Memory

The system lacked mechanisms to:

• Verify if previous feedback had been addressed,
• Prevent reintroducing resolved issues, or
• Graduate to an EXCELLENT rating once requirements were met.

4. Unconstrained Feedback Generation

The AI could invent unlimited new “issues,” such as:

• “Could be more concise”
• “Missing deeper insight about X”
• “Terminology could be more precise”
• “Should explain Y pattern better”

Solution

1. Mechanical Checklist Implementation

The rating system was overhauled to use explicit, objective criteria in a structured checklist format:

RATING SYSTEM - MECHANICAL CHECKLIST: AUTO-FAIL (POOR rating): □ Does the first code block match the original exactly? □ Does the overview align with the actual code content?

EXCELLENT rating (all of these must be true): □ No repetition (no phrase appears twice or more) □ Concise (no sentence can be removed without losing value) □ Insightful (explains why, not just what)

2. Code Immutability Enforcement

Explicit warnings were added to all prompts to clarify that code samples are immutable and cannot be modified. Feedback should only address the explanatory text.

3. Structured Feedback Templates

Feedback was standardized into specific formats, such as:

If the “Some repetition” criterion failed, the evaluator had to:

• Identify the exact repeated phrase (quoted)
• Specify the sections containing it
• Request removal of the repetition from those sections.

4. Iteration Consistency Rules

New evaluation constraints were introduced to prevent arbitrary criteria:

CRITICAL EVALUATION RULES:

1. If all critical requirements are met and no significant issues remain, rate EXCELLENT.
2. Do not invent new criteria beyond the predefined checklist.
3. If feedback from a previous iteration was followed and resolved, rate EXCELLENT.

5. Modified Components

The following files were updated:

1. Evaluator prompts were completely revised to include:

   • A mechanical checklist system,
   • Clear warnings about code immutability,
   • Structured output requirements,
   • Instructions to avoid inventing new criteria.

2. Generator prompts (for overview, breakdown, and closing sections) were updated to emphasize:

CRITICAL UNDERSTANDING: Only revise your explanatory text. The code is immutable. Never attempt to modify code—only improve the text.

Expected Behavior After Fix

Before (Broken):

• Iteration 1: GOOD - “Fix X”
• Iteration 2: GOOD - “Never mind, fix Y”
• Iteration 3: GOOD - “Actually, fix Z”
• Iteration 4: GOOD - “Give up”

After (Fixed):

• Iteration 1: GOOD - “Fix repetition of ‘minimal tool pattern’”
• Iteration 2: EXCELLENT - “Repetition removed, all checklist items pass”