Resolved

Kubernetes production patterns fail on single-node clusters with resource limits

Kubernetes
DevOps
AKS
Helm
GitOps
Prometheus
Resource Optimization
Infrastructure as Code

Production-grade Helm charts for GitOps, monitoring and security exceed single-node AKS capacity (2vCPU/8GB) | Learn how to right-size resource requests without sacrificing architecture quality

Issue

A single-node Azure Kubernetes Service (AKS) cluster, constrained to 2 vCPUs and 8GB RAM, struggled to deploy production-grade monitoring and security tools due to excessive resource requests from default Helm configurations. The cluster’s CPU capacity was nearly exhausted (99% allocation), blocking the deployment of core applications. Despite actual resource usage being significantly lower than requested, Kubernetes scheduling policies rejected new pods due to artificial constraints.

Culprit

The primary issue was overprovisioned default resource requests in production Helm charts, designed for multi-node clusters. These defaults assumed significantly higher resource availability than the single-node environment provided. For example, Prometheus requested 500m CPU while only requiring 40m, leading to 92% wasted capacity. Kubernetes schedules pods based on requested resources, not actual usage, creating a deadlock where new pods could not be scheduled despite 82% unused capacity.

Solution

The issue was resolved through resource right-sizing and burst capacity preservation:

  1. Resource Optimization: Resource requests for all services were significantly reduced while maintaining a 20% buffer over actual usage. This freed up substantial CPU and memory capacity, allowing all production patterns (GitOps, monitoring, security) to operate simultaneously.

  2. Burst Capacity Preservation: Limits were retained at higher values than requests to ensure performance headroom without impacting scheduling.

  3. Configuration Adjustments: Key services like Prometheus, Gatekeeper, and Redis were reconfigured to use minimal requests while maintaining operational stability. For instance, Prometheus’ CPU requests were reduced from 500m to 50m, a 90% decrease.

The changes resulted in freeing up 650m CPU, enabling the deployment of all production-grade tools on the constrained single-node cluster while maintaining full functionality, including GitOps automation, observability, and security enforcement.