Cloudflare Tunnel fails with QUIC UDP connection timeouts

Issue

Cloudflare tunnel pods experienced persistent connection failures, with logs showing “timeout: no recent network activity” and “connection refused” errors. The tunnels intermittently fell back to HTTP/2 over TCP, breaking email webhook delivery and resulting in zero successful SNS deliveries. Two key issues emerged: authentication failures due to missing tunnel credentials and protocol-level failures caused by blocked UDP traffic.

Culprit

Missing Tunnel Token

The Kubernetes secret responsible for storing the Cloudflare tunnel token contained a null value despite the token being defined in environment variables. The script generating the secret failed to properly encode the token in base64, leaving the pod without valid authentication credentials.

Blocked UDP Protocol

The network policy in place only permitted TCP traffic on port 7844, the port required for Cloudflare’s QUIC protocol. Since QUIC relies on UDP, this restriction forced the tunnels to fallback to less reliable HTTP/2 over TCP port 443, exacerbating the connectivity issues.

Solution

Authentication Fix

The issue with the missing tunnel token was resolved by updating the secret generation process to ensure proper base64 encoding of the token.

Network Policy Update

The network policy was modified to include UDP support on port 7844, enabling the QUIC protocol. This change was applied by updating the policy configuration.

Deployment Workflow

The updated network policy was deployed, followed by a restart of the tunnel deployment to load the new token and policy. Verification confirmed successful QUIC connections over UDP, restored tunnel health, and full recovery of the email webhook delivery pipeline.