LLM ignores feedback in multi-turn conversations, causing regeneration instead of iterative improvement

A system using language models (LLMs) for content generation relied on multi-turn conversations between a generator and an evaluator to iteratively improve outputs. Despite receiving specific feedback, the LLM repeatedly generated fundamentally different outputs across iterations, failing to incorporate the feedback and stagnating in quality.

Issue

The LLM-based generator consistently produced the same or unrelated outputs across iterations, despite receiving evaluator feedback. This resulted in stagnation (e.g., GOOD → GOOD → GOOD) rather than iterative improvement. The outputs showed no correlation between feedback content and subsequent revisions, often reintroducing new issues instead of addressing the identified problems.

Culprit

The conversation structure lacked clear instruction hierarchy, treating the original prompt and feedback as equal-level messages. The LLM defaulted to regenerating content from the original prompt (which was more detailed) instead of incorporating the incremental feedback. This ambiguity caused the model to ignore the feedback and prioritize the original task, leading to a lack of iterative refinement.

Solution

Restructured Conversation Flow

The conversation structure was redesigned to establish clear instruction priority while preserving context:

1. Instruction Hierarchy: Feedback was framed as the primary task with explicit directives, ensuring the LLM focused on revisions rather than regeneration.
2. Context Preservation: The original prompt and previous attempts were retained for reference but deprioritized.
3. Actionable Steps: Feedback was wrapped in a TASK: directive with explicit steps to revise the previous response.
4. Output Constraints: Instructions were added to ensure only the improved version was output, avoiding meta-commentary.

Key Changes:

• Feedback was explicitly labeled as the primary task with clear actionable steps.
• The original prompt and previous attempt were retained for context but deprioritized.
• Output constraints were added to focus solely on the revised version.
• Validation mechanisms were implemented to verify meaningful differences in iterations and feedback incorporation.

Validation:

• Iterations were checked to ensure meaningful revisions.
• Feedback incorporation was validated through logs and rating progression.
• Rating improvements were confirmed (e.g., FAIR → GOOD → EXCELLENT).
• No regeneration artifacts were observed.