Deployment fixes for vLLM, Redis, and network policy issues in K3s cluster

A K3s Kubernetes cluster encountered deployment failures for the vLLM (LLM inference server) due to missing GPU RuntimeClass, Redis connection restrictions, and incomplete network policies, disrupting the deployment of FastMCP 2.0. These issues prevented proper GPU access, Redis connectivity, and inter-service communication.

Issue

1. vLLM GPU Access Failure: vLLM pods failed to deploy due to an incompatible RuntimeClass, resulting in a FailedCreate state with the error RuntimeClass 'nvidia-cdi' not found.
2. Redis Connection Blocking: API pods were unable to connect to Redis due to its protected mode, which blocked external connections despite being in a trusted Kubernetes network.
3. Network Policy Gap: The network policy lacked an egress rule for Redis, preventing API pods from accessing Redis on port 6379, while other services like PostgreSQL and vLLM remained accessible.

Culprit

1. RuntimeClass Mismatch: The cluster’s default nvidia RuntimeClass did not match the requirement for Wayland CDI v0.5.0, which needed a specific nvidia-cdi RuntimeClass.
2. Overly Restrictive Redis Configuration: Redis was configured in protected mode, which blocked all external connections by default.
3. Incomplete Network Policy: The existing network policy did not include a rule for Redis, causing connectivity issues between API and Redis services.

Solution

GPU RuntimeClass Fix

A nvidia-cdi RuntimeClass was created to resolve the GPU access issue:

• Defined a RuntimeClass named nvidia-cdi that uses the existing nvidia handler.
• Applied the RuntimeClass to the cluster, enabling successful vLLM deployment.
• Successfully loaded the Qwen3-8B-FP8 model with 8.8 GiB VRAM usage and a load time of 83 seconds.

Redis Configuration Update

Redis was reconfigured to disable protected mode, allowing connections from trusted Kubernetes services:

• Updated Redis configuration to disable protected mode for cluster-wide connectivity.
• Applied changes and restarted the Redis deployment.

Network Policy Expansion

The network policy was expanded to include Redis connectivity:

• Added an egress rule to allow API pods to communicate with Redis on port 6379.
• Ensured API pods could access all required services, including PostgreSQL, Redis, vLLM, DNS, and S3.

Cleanup

Stale replicasets were removed to eliminate crash-looping pods and resource waste:

• Removed 6 obsolete replicasets (API, email-worker, Redis, vLLM) to ensure only healthy pods remained.
• All current pods now show a Running status with proper resource allocation.

The fixes resolved GPU access issues, Redis connectivity, and network restrictions, ensuring smooth deployment and operation of the FastMCP 2.0 stack.