Resolved

MCP RAG system architecture drift from plnamed to implemented design

Python
AI/ML
RAG
Kubernetes
DevOps
Security
rule
Backend
Architecture

Technical analysis of deviations between plnamed and actual implementation in a Kubernetes-hosted RAG system | Covers transport layer gaps, infrastructure changes, and partial RAG pipeline integration

Issue

The deployed Python-based MCP RAG system on Azure Kubernetes Service (AKS) exhibited significant architectural deviations from the original design, particularly in three critical areas:

  1. Transport Layer: The system was limited to standard input/output (stdio) communication, despite plans to support server-sent events (SSE) and HTTP protocols for broader client compatibility.
  2. Infrastructure Layer: The deployment used a single-node AKS cluster with manual scaling instead of the planned Azure Container Apps with built-in autoscaling, increasing operational complexity.
  3. RAG Pipeline: While core hybrid search functionality (combining vector and BM25 search) was implemented, advanced features like query analysis and reranking were not integrated into the default workflow, despite existing as separate components.

Additionally, the system introduced unplanned complexity in security and compliance but lacked planned monitoring solutions.

Culprit

  1. Transport Layer Limitations: The transport mechanism was hardcoded for stdio, lacking abstraction to support additional protocols like SSE or HTTP. This design choice made it difficult to extend functionality without refactoring the core system.
  2. Infrastructure Mismatch: The decision to use AKS instead of Azure Container Apps was driven by a focus on Kubernetes and GitOps workflows, but it introduced unnecessary manual scaling and operational overhead.
  3. Partial RAG Implementation: Although the core hybrid search components were functional, additional features like query analysis and reranking were not included in the default pipeline, despite being available as standalone modules.

Solution

To address these issues, the following changes were implemented:

  1. Transport Layer Expansion: A protocol-agnostic transport handler interface was introduced to support multiple communication protocols (stdio, SSE, HTTP). This allowed for backward compatibility while enabling future extensibility.
  2. Infrastructure Optimization: The AKS cluster was configured to support autoscaling, aligning with the original plan for dynamic resource allocation. This reduced operational complexity while maintaining GitOps workflows.
  3. RAG Pipeline Completion: The query analysis and reranking components were integrated into the pipeline as optional features, controlled by configurable flags. This ensured that advanced functionality could be enabled as needed without impacting performance.

These changes addressed the architectural drift while maintaining backward compatibility and improving flexibility for future enhancements. Additionally, security and compliance requirements were upheld, and monitoring was fully integrated using Prometheus.