Resolved

Stripe webhook integration fails to process payment events in production

Stripe
Webhooks
Python
FastAPI
Kubernetes
Redis
Payment Processing
Backend

Payment processing system lacks proper webhook endpoint for Stripe events | Solution implements secure webhook with signature verification and Redis deduplication

Issue

The payment processing system lacked a functional webhook endpoint to receive and process Stripe payment events, such as payment_intent.succeeded, payment_intent.payment_failed, and charge.refunded. This limitation resulted in:

  • No real-time capture of payment status updates.
  • Lack of automated synchronization for booking status.
  • Manual intervention required for refund processing.
  • Inability to trigger payment confirmation emails automatically.

Culprit

Three critical architectural gaps prevented the system from processing Stripe webhooks:

  1. Missing Endpoint: No /stripe-webhook route existed in the FastAPI application to receive Stripe’s POST requests.
  2. No Security Layer: Absence of signature verification made the system vulnerable to spoofed webhook requests.
  3. Duplicate Processing Risk: Without idempotency handling, Stripe’s retry mechanism could cause duplicate payment processing.

As a result, the system relied on manual dashboard checks and lacked real-time event processing capabilities.

Solution

To address these issues, a secure webhook endpoint was implemented with the following components:

Webhook Endpoint Implementation

A dedicated FastAPI endpoint was created to handle Stripe webhook events securely:

  • Signature Verification: Validates the authenticity of incoming webhook requests using a STRIPE_WEBHOOK_SECRET to prevent spoofed events.
  • Redis Deduplication: Ensures idempotency by checking Redis for previously processed events, preventing duplicate processing.
  • Event Processing: Handles different event types, such as payment success, failure, and refunds, to update system state accordingly.

Kubernetes Deployment Architecture

The deployment architecture included:

  • Cloudflare Tunnel: Secures and routes traffic to the Kubernetes ingress.
  • Kubernetes Ingress: Directs traffic to the API service with TLS termination.
  • API Pods: Two replicas with Redis connections for deduplication.
  • Secrets Management: Secure storage and injection of Stripe keys using Kubernetes Secrets.

Security Implementation

Security measures included:

  • Signature Verification: Ensures all incoming requests are authenticated using Stripe’s webhook signing mechanism.
  • Rate Limiting: Limits incoming requests to 100 per minute per IP address to prevent abuse.
  • TLS Encryption: Enforces secure communication via Cloudflare tunnel.
  • Idempotency: Uses Redis to track processed events, preventing duplicates with a 5-minute TTL.
  • Secret Management: Stores sensitive keys in environment variables and Kubernetes Secrets.

Event Processing Flow

The system now processes Stripe events in real-time:

  • Receives Webhook Events: Stripe sends payment events via HTTPS.
  • Verifies and Processes: The webhook endpoint validates and processes each event type, updating the system state.
  • Triggers Automated Actions: Payment confirmations, booking updates, and refund processing are now automated.

These changes enabled real-time payment processing, improved system reliability, and eliminated the need for manual intervention in payment-related workflows.