Azure MCP RAG infrastructure cost optimization for $100 monthly budget

Issue

Developing a production-grade RAG infrastructure on Azure requires balancing essential security features (e.g., network isolation, secrets management, audit logging) with strict cost constraints. The primary challenges include:

1. Cost overruns when including premium security features like Azure Firewall or private endpoints.
2. Performance limitations from single-zone deployments and spot instances.
3. Operational complexity in maintaining security while using cost-saving measures like public clusters or service endpoints.

Culprit

The root causes stem from architectural trade-offs required to stay within the $100/month budget:

• Single-zone deployment (instead of multi-zone) to reduce costs but introduces availability risks.
• Spot instances for cost savings but risks preemption and downtime.
• Public AKS clusters (instead of private) to save costs but exposes API endpoints.
• Service endpoints (instead of private endpoints) to reduce costs but provide weaker network isolation.
• Standard Azure Container Registry (ACR) lacks geo-replication and customer-managed encryption.
• Basic monitoring (Log Analytics + Prometheus) instead of Azure Monitor Premium.

Solution

Cost-Optimized Architecture

A three-layer separation was implemented with clear cost boundaries:

1. Foundation Layer ($20/month)

   • Virtual Network with Network Security Groups (free).
   • Key Vault Standard with soft delete (free tier).
   • Log Analytics with basic retention.
   • Private DNS zones for core services.

2. Application Layer ($65/month)

   • AKS: Single spot instance (B2s size) for compute savings.
   • ACR Standard: For container storage with basic features.
   • Azure OpenAI: Pay-per-use embeddings for cost control.
   • On-cluster services (Redis, ChromaDB, Prometheus/Grafana) for monitoring.

Security Compromise Matrix

The following security trade-offs were accepted to stay within budget:

Security Feature	Premium Implementation	Cost-Optimized Approach	Risk Accepted
Network Isolation	Azure Firewall ($900/month)	NSGs with default deny rules	Less granular L7 filtering
Private Connectivity	Private Endpoints ($20-40/month)	Service endpoints + NSG restrictions	Public IP exposure (with restrictions)
Key Management	HSM-backed Key Vault ($1/key)	Standard Key Vault (platform keys)	No customer-managed encryption
High Availability	Multi-zone AKS ($30-50/month)	Single-zone with spot instances	Node failure = downtime
Disaster Recovery	Multi-region ($500+/month)	Single-region only	No regional failover

Operational Cost Controls

1. Automated Shutdown Scripts: Stop AKS during non-business hours to reduce costs.
2. Budget Alerts: Configured alerts at 80% of the $100/month budget threshold.
3. OpenAI Cost Monitoring: Monitor embedding usage to stay within pay-per-use limits.

Performance vs Cost Trade-offs

The following components reflect the trade-offs between cost and performance:

Component	Premium Choice	Cost-Optimized Choice	Performance Impact
AKS Nodes	3x D4s_v3 (16 vCPU, 64GB)	1x B2s spot (2 vCPU, 4GB)	8x less compute, spot interruptions
Container Registry	Premium (geo-replication)	Standard (single-region)	No geo-redundancy
Monitoring	Azure Monitor Premium	Log Analytics + Prometheus	Limited Azure-native alerts
Network Security	Azure Firewall + Private Link	NSGs + Service Endpoints	Less granular traffic filtering
OpenAI Model	Dedicated capacity (100k TPM)	Pay-per-use (1k TPM)	Rate limiting at scale

This architecture achieves a balance between cost constraints and essential RAG functionality while accepting controlled risks.