Resolved

RAG backend infrastructure deployed but core functionality not implemented

RAG
Azure
Kubernetes
FluxCD
Python
DevOps
Backend
Debugging

Azure Kubernetes infrastructure fully operational with FluxCD and OpenAI integration | RAG tools exist as skeleton code but never deployed or tested with missing Docker images and failing Redis pod

Issue

The documentation for the RAG backend infrastructure on Azure Kubernetes Service (AKS) and FluxCD GitOps misled users into believing the system was fully operational. Despite the infrastructure being deployed and operational (38/38 resources), no actual RAG functionality was available due to missing components, including:

  1. The MCP server was not deployed, with no Docker images built or Kubernetes manifests configured.
  2. ChromaDB, the vector database, was not deployed.
  3. The Redis pod failed to start due to an ImagePullBackOff error.
  4. The tool integrations for search, indexing, and analysis were only skeleton code, with no runtime deployment or testing.
  5. FluxCD kustomizations were blocked, preventing the application from deploying.

As a result, users attempting to follow the documentation would encounter errors such as missing MCP server pods, failing Redis pods, and connection failures to ChromaDB and other tools.

Culprit

The primary causes of this issue were:

  1. Missing Deployment Pipeline: While the infrastructure was managed via FluxCD GitOps, there was no CI/CD pipeline to build and push the MCP server Docker image to Azure Container Registry (ACR).
  2. Unbuilt Components: The RAG backend code existed as class structures and method signatures but was never containerized or configured for Kubernetes deployment.
  3. Documentation Misalignment: The guides and documentation assumed a fully operational RAG backend, despite it not being deployed or tested.

Solution

To address these issues, the following steps were implemented:

  1. Unblock Redis: Corrected the image pull error by specifying the correct Redis image tag and restarting the deployment.

  2. Deploy ChromaDB: Created a StatefulSet manifest for ChromaDB to deploy the vector database in the Kubernetes cluster.

  3. Build and Deploy MCP Server: Created a Dockerfile for the MCP server, built the Docker image, and pushed it to Azure Container Registry. Kubernetes manifests were then created to deploy the MCP server in the cluster.

  4. Implement CI/CD Pipeline: Added a GitHub Actions workflow to automate the building and deployment of the MCP server Docker image.

  5. Update Documentation: Revised the documentation to clarify the current state of the system. Added a new deployment guide for the RAG backend and updated existing guides to reflect the actual operational status.

  6. Validation Steps: Implemented tests to verify the integration of RAG tools with ChromaDB and Redis, and conducted end-to-end testing to ensure the system was functioning correctly.

These changes ensured that the RAG backend was properly deployed, operational, and aligned with the documentation, providing users with a functional system as described.