Back to 0fee
0fee

Webhook Delivery and Retry System

How 0fee.dev delivers webhooks with HMAC-SHA256 signatures, exponential backoff retries, and auto-disable after failures. By Juste A. Gnimavo and Claude.

Thales & Claude | March 25, 2026 11 min 0fee
webhooksretryhmacexponential-backoffreliability

When a payment completes on 0fee.dev -- whether through Stripe's checkout page, Hub2's USSD push, or PaiementPro's redirect flow -- the merchant needs to know. They cannot poll the API continuously. The solution is webhooks: HTTP POST requests sent from 0fee.dev to the merchant's server whenever a payment event occurs. But webhook delivery is harder than it appears. Merchant servers go down. Networks fail. Firewalls block requests. The webhook system must handle all of this reliably.

HMAC-SHA256 Signature Generation

Every webhook delivery is signed with HMAC-SHA256 so the receiving merchant can verify that the request genuinely came from 0fee.dev and was not tampered with in transit.

python# services/webhook_delivery.py
import hmac
import hashlib
import json
import time

def generate_webhook_signature(
    payload: dict,
    secret: str,
    timestamp: int = None,
) -> str:
    """
    Generate HMAC-SHA256 signature for a webhook payload.

    The signature covers: timestamp + "." + JSON payload.
    This prevents replay attacks (timestamp is included)
    and ensures payload integrity.

    Args:
        payload: The webhook event data
        secret: The merchant's webhook secret (whsec_...)
        timestamp: Unix timestamp (defaults to now)

    Returns:
        Signature string in format "t=timestamp,v1=hex_signature"
    """
    if timestamp is None:
        timestamp = int(time.time())

    # Serialize payload deterministically
    payload_str = json.dumps(payload, sort_keys=True, separators=(",", ":"))

    # Sign: timestamp.payload
    message = f"{timestamp}.{payload_str}"
    signature = hmac.new(
        secret.encode("utf-8"),
        message.encode("utf-8"),
        hashlib.sha256,
    ).hexdigest()

    return f"t={timestamp},v1={signature}"

The signature format t=timestamp,v1=signature follows the convention popularized by Stripe. Including the timestamp in the signed message prevents replay attacks: even if an attacker captures a valid webhook, replaying it hours later would produce a stale timestamp that the merchant can reject.

Why sort_keys=True?

JSON serialization is not deterministic by default -- the order of keys in a dictionary can vary between serialization calls. By sorting keys and using compact separators, the same payload always produces the same JSON string, which always produces the same signature. Without this, a valid webhook could fail verification simply because Python serialized the keys in a different order.

Webhook Event Types

The system generates webhooks at every significant point in the payment lifecycle:

EventTriggerTypical Timing
payment.createdPayment initiated successfullyImmediate
payment.completedProvider confirms payment receivedSeconds to minutes
payment.failedProvider reports payment failureSeconds to minutes
payment.expiredPayment timeout (no action taken)15-30 minutes
payment.cancelledMerchant or customer cancelsImmediate
refund.createdRefund initiatedImmediate
refund.completedRefund processed by providerHours to days
refund.failedRefund could not be processedHours to days
checkout.completedCheckout session finishedImmediate
checkout.expiredCheckout session timed out30 minutes

Event Payload Structure

Every webhook event follows the same structure:

json{
    "id": "evt_abc123def456",
    "type": "payment.completed",
    "created_at": "2025-12-10T14:35:08Z",
    "data": {
        "id": "txn_749b8b1afbd846eaba95",
        "app_id": "app_live_xyz789",
        "amount": 5000,
        "currency": "XOF",
        "status": "completed",
        "payment_method": "PAYIN_ORANGE_CI",
        "provider": "paiementpro",
        "customer": {
            "phone": "+2250709757296",
            "email": "[email protected]"
        },
        "metadata": {
            "order_id": "ORD-2025-001"
        },
        "created_at": "2025-12-10T14:32:08Z",
        "completed_at": "2025-12-10T14:35:08Z"
    }
}

The data field contains the full payment object at the time of the event. This means the merchant does not need to make an additional API call to get payment details -- the webhook contains everything.

Delivery Mechanism

Webhook delivery is handled by the WebhookDeliveryService, which queues events for delivery via Celery:

pythonclass WebhookDeliveryService:
    """Service for delivering webhook events to merchant endpoints."""

    @staticmethod
    async def queue_event(
        app_id: str,
        event_type: str,
        data: dict,
    ):
        """
        Queue a webhook event for delivery.

        1. Look up the merchant's webhook URL and secret
        2. Generate the event payload and signature
        3. Store the event in the database
        4. Queue delivery via Celery
        """
        with get_db() as conn:
            app = conn.execute(
                "SELECT webhook_url, webhook_secret FROM apps WHERE id = ?",
                (app_id,)
            ).fetchone()

        if not app or not app["webhook_url"]:
            return  # No webhook configured

        event_id = f"evt_{generate_id()}"
        timestamp = int(time.time())

        event_payload = {
            "id": event_id,
            "type": event_type,
            "created_at": datetime.utcnow().isoformat() + "Z",
            "data": data,
        }

        signature = generate_webhook_signature(
            event_payload,
            app["webhook_secret"],
            timestamp,
        )

        # Store the event
        with get_db() as conn:
            conn.execute(
                """
                INSERT INTO webhook_events
                    (id, app_id, event_type, payload, status, created_at)
                VALUES (?, ?, ?, ?, 'pending', ?)
                """,
                (event_id, app_id, event_type,
                 json.dumps(event_payload),
                 datetime.utcnow().isoformat())
            )

        # Queue for delivery
        from tasks.webhook_retry import deliver_webhook
        deliver_webhook.delay(event_id)

Integration with Payment Lifecycle

Webhook events are triggered at key points in the payment flow:

python# In the payment route handler
async def process_payment(data, auth):
    # ... create transaction in database ...

    # Trigger payment.created webhook
    await WebhookDeliveryService.queue_event(
        app_id=auth["app_id"],
        event_type="payment.created",
        data=payment_response,
    )

    return payment_response

# In the webhook handler (when provider sends callback) async def handle_provider_webhook(provider_id, payload, headers): # ... process provider webhook ... # ... update transaction status ... BLANK if new_status == "completed": await WebhookDeliveryService.queue_event( app_id=transaction["app_id"], event_type="payment.completed", data=updated_payment, ) elif new_status == "failed": await WebhookDeliveryService.queue_event( app_id=transaction["app_id"], event_type="payment.failed", data=updated_payment, ) ```

Exponential Backoff Retries

When a webhook delivery fails, the system retries with exponential backoff:

python# tasks/webhook_retry.py
from celery import shared_task
import httpx

# Retry schedule: 1min, 5min, 30min, 2h, 8h, 24h
RETRY_DELAYS = [60, 300, 1800, 7200, 28800, 86400]
MAX_ATTEMPTS = 6

@shared_task(
    bind=True,
    max_retries=MAX_ATTEMPTS,
    default_retry_delay=60,
)
def deliver_webhook(self, event_id: str):
    """
    Deliver a webhook event to the merchant's endpoint.
    Retries with exponential backoff on failure.
    """
    with get_db() as conn:
        event = conn.execute(
            """
            SELECT we.*, a.webhook_url, a.webhook_secret
            FROM webhook_events we
            JOIN apps a ON we.app_id = a.id
            WHERE we.id = ?
            """,
            (event_id,)
        ).fetchone()

    if not event:
        return

    if not event["webhook_url"]:
        mark_event_skipped(event_id, "No webhook URL configured")
        return

    payload = json.loads(event["payload"])
    timestamp = int(time.time())

    signature = generate_webhook_signature(
        payload, event["webhook_secret"], timestamp
    )

    attempt_number = self.request.retries + 1
    delivery_id = f"del_{generate_id()}"

    try:
        start_time = time.time()

        response = httpx.post(
            event["webhook_url"],
            json=payload,
            headers={
                "Content-Type": "application/json",
                "X-ZeroFee-Signature": signature,
                "X-ZeroFee-Event": event["event_type"],
                "X-ZeroFee-Delivery": delivery_id,
                "X-ZeroFee-Timestamp": str(timestamp),
                "User-Agent": "ZeroFee-Webhook/1.0",
            },
            timeout=30,
        )

        duration_ms = int((time.time() - start_time) * 1000)

        # Log the delivery attempt
        log_delivery_attempt(
            event_id=event_id,
            delivery_id=delivery_id,
            attempt=attempt_number,
            status_code=response.status_code,
            response_body=response.text[:1000],
            duration_ms=duration_ms,
        )

        if response.status_code < 300:
            # Success
            mark_event_delivered(event_id)
            return

        # Non-success status code -- retry
        raise Exception(
            f"Webhook returned HTTP {response.status_code}: "
            f"{response.text[:200]}"
        )

    except Exception as exc:
        # Log the failed attempt
        log_delivery_attempt(
            event_id=event_id,
            delivery_id=delivery_id,
            attempt=attempt_number,
            status_code=0,
            response_body=str(exc)[:1000],
            duration_ms=0,
            success=False,
        )

        # Calculate next retry delay
        retry_index = min(self.request.retries, len(RETRY_DELAYS) - 1)
        countdown = RETRY_DELAYS[retry_index]

        if self.request.retries < MAX_ATTEMPTS:
            self.retry(exc=exc, countdown=countdown)
        else:
            mark_event_failed(event_id)
            check_consecutive_failures(event["app_id"])

The Retry Schedule

AttemptDelay After FailureCumulative Wait
1Immediate0
21 minute1 minute
35 minutes6 minutes
430 minutes36 minutes
52 hours2h 36min
68 hours10h 36min

After the 6th failed attempt, the event is marked as permanently failed. The total retry window spans approximately 10 hours and 36 minutes. This gives merchants ample time to fix issues with their webhook endpoints (server restarts, deployments, DNS changes) while ensuring events are not retried indefinitely.

Auto-Disable After Consecutive Failures

If a merchant's webhook endpoint fails 10 consecutive times across any events, the system automatically disables webhook delivery for that application:

pythondef check_consecutive_failures(app_id: str):
    """
    Check if webhook endpoint should be auto-disabled
    after too many consecutive failures.
    """
    with get_db() as conn:
        # Count consecutive failures (no successes in between)
        recent_events = conn.execute(
            """
            SELECT status FROM webhook_events
            WHERE app_id = ?
            ORDER BY created_at DESC
            LIMIT 10
            """,
            (app_id,)
        ).fetchall()

    consecutive_failures = 0
    for event in recent_events:
        if event["status"] == "failed":
            consecutive_failures += 1
        else:
            break

    if consecutive_failures >= 10:
        # Disable webhooks for this app
        with get_db() as conn:
            conn.execute(
                """
                UPDATE apps
                SET webhook_active = 0,
                    webhook_disabled_reason = 'auto_disabled_failures',
                    webhook_disabled_at = ?
                WHERE id = ?
                """,
                (datetime.utcnow().isoformat(), app_id)
            )

        # Notify the merchant via email
        notify_webhook_disabled(app_id)

The merchant receives an email notification explaining that their webhook endpoint has been disabled due to repeated failures. They can re-enable it from the dashboard after fixing the underlying issue.

This auto-disable mechanism protects both 0fee.dev and the merchant. Without it, the system would continue queuing retries for a permanently broken endpoint, consuming resources and generating noise in delivery logs.

Delivery Logging

Every delivery attempt is logged in the webhook_deliveries table:

sqlCREATE TABLE webhook_deliveries (
    id TEXT PRIMARY KEY,
    event_id TEXT NOT NULL REFERENCES webhook_events(id),
    attempt INTEGER NOT NULL,
    status_code INTEGER,
    response_body TEXT,
    duration_ms INTEGER,
    success INTEGER DEFAULT 0,
    created_at TEXT DEFAULT CURRENT_TIMESTAMP
);

This log serves three purposes:

  1. Debugging. When a merchant reports missing webhooks, the delivery log shows exactly what happened: which status code their server returned, how long the request took, and what error message was received.
  1. Dashboard visibility. The merchant dashboard displays delivery history for each webhook event, allowing developers to see failed attempts and response details.
  1. Analytics. Aggregate delivery metrics (success rate, average latency, most common failure codes) inform platform health monitoring.

Webhook Verification Code Examples

Merchants need to verify webhook signatures on their end. The documentation provides examples in four languages:

Python

pythonimport hmac
import hashlib
import json
import time

def verify_zerofee_webhook(
    payload: bytes,
    signature_header: str,
    secret: str,
    tolerance: int = 300,  # 5 minutes
) -> bool:
    """Verify a 0fee.dev webhook signature."""
    # Parse signature header: t=timestamp,v1=signature
    parts = dict(
        part.split("=", 1) for part in signature_header.split(",")
    )
    timestamp = int(parts.get("t", "0"))
    expected_sig = parts.get("v1", "")

    # Check timestamp tolerance (prevent replay attacks)
    if abs(time.time() - timestamp) > tolerance:
        return False

    # Compute expected signature
    message = f"{timestamp}.{payload.decode('utf-8')}"
    computed = hmac.new(
        secret.encode("utf-8"),
        message.encode("utf-8"),
        hashlib.sha256,
    ).hexdigest()

    return hmac.compare_digest(computed, expected_sig)

Node.js

javascriptconst crypto = require("crypto");

function verifyZerofeeWebhook(payload, signatureHeader, secret, tolerance = 300) {
    // Parse signature header
    const parts = Object.fromEntries(
        signatureHeader.split(",").map((p) => p.split("=", 2))
    );
    const timestamp = parseInt(parts.t, 10);
    const expectedSig = parts.v1;

    // Check timestamp
    if (Math.abs(Date.now() / 1000 - timestamp) > tolerance) {
        return false;
    }

    // Compute signature
    const message = `${timestamp}.${payload}`;
    const computed = crypto
        .createHmac("sha256", secret)
        .update(message)
        .digest("hex");

    return crypto.timingSafeEqual(
        Buffer.from(computed),
        Buffer.from(expectedSig)
    );
}

Go

gopackage main

import (
    "crypto/hmac"
    "crypto/sha256"
    "encoding/hex"
    "fmt"
    "math"
    "strconv"
    "strings"
    "time"
)

func VerifyZerofeeWebhook(payload []byte, signatureHeader, secret string, tolerance int64) bool {
    parts := make(map[string]string)
    for _, part := range strings.Split(signatureHeader, ",") {
        kv := strings.SplitN(part, "=", 2)
        if len(kv) == 2 {
            parts[kv[0]] = kv[1]
        }
    }

    timestamp, _ := strconv.ParseInt(parts["t"], 10, 64)
    expectedSig := parts["v1"]

    // Check timestamp
    if math.Abs(float64(time.Now().Unix()-timestamp)) > float64(tolerance) {
        return false
    }

    // Compute signature
    message := fmt.Sprintf("%d.%s", timestamp, string(payload))
    mac := hmac.New(sha256.New, []byte(secret))
    mac.Write([]byte(message))
    computed := hex.EncodeToString(mac.Sum(nil))

    return hmac.Equal([]byte(computed), []byte(expectedSig))
}

PHP

php<?php

function verifyZerofeeWebhook(
    string $payload,
    string $signatureHeader,
    string $secret,
    int $tolerance = 300
): bool {
    // Parse signature header
    $parts = [];
    foreach (explode(',', $signatureHeader) as $part) {
        [$key, $value] = explode('=', $part, 2);
        $parts[$key] = $value;
    }

    $timestamp = (int) ($parts['t'] ?? 0);
    $expectedSig = $parts['v1'] ?? '';

    // Check timestamp
    if (abs(time() - $timestamp) > $tolerance) {
        return false;
    }

    // Compute signature
    $message = "{$timestamp}.{$payload}";
    $computed = hash_hmac('sha256', $message, $secret);

    return hash_equals($computed, $expectedSig);
}

All four examples use constant-time comparison functions (hmac.compare_digest, timingSafeEqual, hmac.Equal, hash_equals) to prevent timing attacks. This is a subtle but important security detail: if a naive string comparison is used, an attacker could determine the correct signature character by character by measuring response times.

The Webhook System as a Contract

Webhooks are not just a notification mechanism -- they are a contract between 0fee.dev and its merchants. The merchant trusts that every payment event will be delivered. The retry system with exponential backoff, the delivery logging, the auto-disable with notification, and the signature verification -- all of these exist to make that contract reliable. When a merchant integrates webhooks, they need to trust that no event will be silently lost.


This article is part of the "How We Built 0fee.dev" series. 0fee.dev is a payment orchestrator covering 53+ providers across 200+ countries, built by Juste A. GNIMAVO and Claude from Abidjan with zero human engineers. Follow the series for the complete build story.

Share this article:

Responses

Write a response
0/2000
Loading responses...

Related Articles