Resolved

LLM-generated tutorials contain undetected factual inaccuracies

Python
AI/ML
LLM
Validation
Semantic Search
ChromaDB
Documentation
Automation

Automated video tutorials make unverified claims about API behavior | Solution uses ChromaDB semantic search against official docs to validate technical accuracy

Issue

LLM-generated technical tutorials contained unverified claims about FastMCP API behavior, lacking ground truth validation. This resulted in:

  • Undetected hallucinations regarding API parameters.
  • Incorrect method signatures in code examples.
  • Missing or misrepresented configuration options.
  • No citation trail to official documentation.

Culprit

The validation approach relied solely on LLM self-assessment without external reference points, leading to:

  1. No Document Grounding: No connection to official FastMCP documentation.
  2. Semantic Gap: Claims were compared only against LLM’s parametric knowledge.
  3. No Citation Trail: Inaccuracies could not be traced to specific documentation sections.
  4. Binary Judgments: Only ‘correct/incorrect’ assessments without detailed explanations.

Solution

ChromaDB Semantic Index

  1. Document Ingestion: Indexed all .mdx files from the official FastMCP documentation into ChromaDB using the all-MiniLM-L6-v2 embedding model.
  2. Semantic Search Endpoint: Added a tool to search and retrieve relevant documentation chunks for validation purposes.

Document-Grounded Validation

  1. Claim Extraction: Used an LLM to extract technical claims from generated tutorials in JSON format.
  2. Semantic Validation Loop: For each claim, retrieved relevant documentation chunks and validated the claim against these chunks using the LLM.

Key Implementation Details

  • Ultra-Lean Architecture: 155 lines total across components.
  • Zero Configuration: Utilized ChromaDB defaults for embeddings.
  • Fail-Fast Design: No error handling for LLM JSON parsing, making errors visible.
  • Whole-File Indexing: Processed complete .mdx files without chunking.
  • Accuracy Scoring: Returned percentage accuracy and specific inaccuracies with citations.

Performance Metrics

  • Indexing: 181 documentation files indexed in approximately 20 seconds.
  • Validation: ~3 seconds per tutorial script.
  • Accuracy Improvement: Increased accuracy from 78% to 96% in test cases, with documentation citations.