Resolved

Redis pod stuck in Pending state due to insufficient CPU in AKS

Redis
Kubernetes
AKS
DevOps
FluxCD
OCI
Resource Constraints
Helm

Single-node AKS cluster at 96% CPU capacity blocks Redis deployment with 'Insufficient cpu' error | Pod remains Pending despite successful OCI chart creation and FluxCD OCIRepository readiness

The project addressed a Kubernetes scheduling issue where a Redis pod remained in a Pending state due to insufficient CPU resources on a single-node Azure Kubernetes Service (AKS) cluster. Despite successful artifact creation and FluxCD readiness, the pod could not be scheduled due to high CPU utilization (96%), resulting in a FailedScheduling error.

Issue

The Redis pod (redis-master-0) remained in a Pending state with the error 0/1 nodes are available: 1 Insufficient cpu, preventing successful deployment. The Kubernetes scheduler reported no available nodes for scheduling due to high CPU consumption, and the pod showed 0/2 containers ready. Similar issues affected other FluxCD components (image-automation-controller, image-reflector-controller) due to identical resource constraints.

Culprit

The single-node AKS cluster was operating at 96% CPU capacity (1826m/1900m requested), leaving insufficient resources for new workloads. The Redis deployment required 70m CPU (50m for Redis + 20m for exporter), but the available CPU after accounting for existing workloads and Kubernetes overhead was insufficient. The node type (Standard_DC2ads_v5, 2 vCPU) lacked headroom for:

  • Existing workloads consuming 1826m CPU.
  • Redis pod requirements (70m CPU).
  • Kubernetes system overhead (20-30m CPU).

Solution

Immediate Workarounds (Development/Test)

To address the issue temporarily, the following steps were taken:

  • Scaled down non-critical workloads to free up CPU resources, releasing approximately 150m CPU.
  • Reduced Redis resource requests by adjusting CPU and memory allocations to lower values (e.g., 25m CPU and 64Mi memory instead of 50m CPU and 128Mi memory).

Production Solution

The long-term solution involved upgrading the node size to a larger instance type with more CPU capacity. This adjustment:

  • Increased the node size from 2 vCPU to 4 vCPU (Standard_DC4ads_v5).
  • Expanded the total available CPU capacity from ~1900m to ~3900m, providing sufficient headroom for Redis and other workloads.
  • Ensured proper resource allocation for Kubernetes system overhead and new deployments.

Verification of OCI Functionality

Despite the scheduling issues, the core OCI artifact integration remained functional:

  • FluxCD successfully pulled the Redis chart from the OCI registry (docker.io/bitnami/redis:7.4.3-debian-12-r0).
  • The OCIRepository resource showed readiness (READY: True).
  • The HelmRelease generated the correct pod specification from the OCI-based chart.
  • The scheduling failures were attributed to CPU constraints rather than issues with OCI artifacts.

This approach ensured scalable Redis deployment while maintaining the integrity of the GitOps workflow and OCI-based artifact management.