Resolved

Documentation validation system fails with incorrect accuracy scores

Python
AI/ML
Documentation
Validation
ChromaDB
LLM
NLP
Embeddings

Validation system using ChromaDB returned unrealistic 0% accuracy when checking explanation quality | Fixed by switching from explanation-based to code-based document retrieval

Issue

The validation system failed to retrieve relevant documentation sections for LLM-generated code explanations, consistently returning 0% accuracy. The problem stemmed from searching ChromaDB using potentially incorrect explanation text instead of the trusted source code being explained.

Culprit

The original validation flow extracted claims from the explanation text and used those claims to search documentation. This approach failed because:

  1. Explanation text might contain hallucinations or errors, leading to incorrect claims.
  2. Searching documentation with incorrect claims retrieved irrelevant sections.
  3. The system then validated explanations against unrelated documentation, resulting in meaningless accuracy scores.

The core issue was relying on untrusted explanation text for the search query rather than the known-correct source code.

Solution

Architecture Redesign

The validation flow was redesigned to use the trusted source code for documentation retrieval:

  • Code-Centric Search: ChromaDB queries now use the actual code being explained, ensuring relevant documentation is retrieved.
  • Contextual Validation: The LLM receives both the code and retrieved documentation to validate the accuracy of the explanation.

Validation Process Improvements

  1. Structured Prompt: A detailed prompt was developed to guide the LLM, emphasizing:

    • The code is trusted and should not be critiqued.
    • The explanation must accurately reflect the code’s functionality, as validated against official documentation.
    • Focus on explanation accuracy while ignoring superficial code quirks like formatting.

  2. Error Handling: Removed silent fallbacks, replaced JSON parsing requirements with plain-text feedback, and made the validation model configurable via environment variables.

  3. Performance Optimizations:

    • Persistent ChromaDB storage to avoid re-indexing.
    • Configurable models for fast validation.
    • Token limits to control feedback length.

The updated solution achieves 80-95% realistic accuracy by grounding explanations in official documentation and providing feedback loops for improvement.