Resolving Azure VM Quota Constraints for AKS Deployment
Issue
The deployment of an AKS cluster for a RAG (Retrieval-Augmented Generation) SDK failed due to unavailable VM sizes in the Azure subscription. Specifically, the configuration required B-series VMs (burstable instances), but these were blocked for the subscription in the eastus region, causing the entire deployment to stall at the AKS creation stage.
Culprit
The subscription imposed restrictions on cost-optimized VM families, including:
- B-series VMs: Blocked for all regions, despite being the target for cost-efficient Kubernetes workloads.
- Alternative low-cost SKUs: Only high-cost VM families (e.g., DC-series, FX-series) were available, leading to a 2.5x budget overrun compared to the planned architecture.
- Pre-flight validation failure: Quota constraints were not checked before deployment, leading to an unsupported configuration.
Solution
Three approaches were implemented:
1. Immediate Workaround
Used Azure OpenAI embeddings directly without AKS to bypass VM quota constraints. This allowed the system to function while waiting for quota approval.
2. Recommended Fix
Requested a quota increase for B-series VMs in the Azure Portal. Once approved, the deployment resumed successfully.
3. Alternative Architectures
- Azure Container Instances (ACI): Deployed the application as containers without Kubernetes, avoiding VM quotas entirely.
- Multi-cloud fallback: Considered alternative cloud providers (e.g., Google Cloud) with more flexible quota policies.
Prevention Measures
- Pre-flight validation: Added scripts to check VM availability before deployment.
- Fallback configurations: Updated Terraform to support alternative VM sizes or cloud providers if quotas are unavailable.
Outcome: The issue was resolved by either requesting quota approval or deploying alternative architectures, ensuring the system met cost and performance targets.