Issue

The RAG backend implementation exists in two parallel forms, creating maintenance challenges:

A production-ready monolithic system with established OAuth, RBAC, and vector storage integration, currently deployed.
An untested modular refactor with advanced features like AST-aware chunking and hybrid search, but with unresolved dependencies and no operational history.

Key tensions include:

The monolithic version lacks advanced features such as AST-based chunking and hybrid search algorithms.
The modular design, while theoretically superior, introduces significant migration risks, dependency issues, and no clear path for integration with the current deployment.
Technical debt in the monolith, such as outdated validators and structural violations, contrasts with the refactor’s potential over-engineering risks.

Culprit

The two implementations evolved independently, leading to:

A working monolith with comprehensive test coverage and production deployment, but without advanced RAG features.
A modular design with promising features but no operational validation, unresolved dependencies, and no alignment with existing deployment infrastructure.

The current monolithic implementation lacks:

AST-aware chunking (e.g., using tree-sitter for syntactic parsing).
Hybrid search (combining BM25 ranking with vector similarity).
Cross-encoder reranking for improved retrieval quality.
Comprehensive load testing to validate performance under production conditions.

The monolithic codebase suffers from:

Deprecated validators requiring migration to newer versions.
Structural violations (e.g., large, unmodularized files violating separation of concerns).
Dependency resolution risks due to missing lock files.

The modular design introduces potential risks:

Excessive complexity (e.g., a 17,000-line chunking module).
Unproven hybrid search components with no evidence of production readiness.
No clear migration path between implementations, risking divergence and maintenance confusion.

An incremental strategy over 6 weeks balances risk mitigation with feature adoption:

Extract modular tools from the refactor (e.g., hybrid search components) and integrate them into the monolith.
Implement AST-aware chunking as a scaled-down proof of concept.
Add hybrid search combining traditional ranking (e.g., BM25) with vector similarity.

Expand AST chunking to support multiple languages (e.g., Python, JavaScript).
Implement reciprocal rank fusion for hybrid search.
Introduce cross-encoder reranking based on performance testing.

Achieve 80%+ test coverage by porting existing tests and adding new validation cases.
Conduct load testing to ensure the refactored components perform under production conditions.

Dependency Management: Use a structured dependency resolution tool to manage version conflicts.
Kubernetes Compatibility: Maintain existing deployment configurations while adding support for new hybrid search features.
Resilience: Implement circuit breakers to handle failures in external services (e.g., vector databases, ranking models).

Complete Rebuild: Abandoning the working monolith in favor of an untested refactor would introduce unnecessary risk and delay.
Parallel Maintenance: Running both implementations concurrently would complicate CI/CD pipelines and create confusion over the “source of truth.”

Metric	Current Status	Target
Test Coverage	~60%	80%+
Code Modularity	Monolithic	Reduced file size (<300 LOC per file)
Search Precision	0.78 (vector only)	0.85+ (hybrid)
Chunking Accuracy	0.65 (regex)	0.89 (AST-aware)
Deployment Stability	Established	Maintained with new features

This approach ensures a gradual, low-risk migration while preserving production stability and adopting advanced RAG capabilities incrementally.