Skip to content

Free, Open Source, Self-Hosted WhatsApp API Gateway

docs(webhook): state how long a completed job retires its outbox row

Yudhi Armyndhariscdd3753e4956
diff --git a/docs/06-api-specification.md b/docs/06-api-specification.md
index 69356f475..550f3ac91 100644
--- a/docs/06-api-specification.md
+++ b/docs/06-api-specification.md
@@ -4742,7 +4742,7 @@ A delivery shed because the dispatch queue was full, or refused during shutdown,
**Errors:** `401` missing/invalid API key · `403` key role below ADMIN
-Queued deliveries retain their pre-hook outbox copy until the worker completes delivery or stores a terminal failure. The reconciler checks the current job before replaying, retires the row of a completed job without a replay, and keeps rows while Redis cannot be inspected. If failure persistence is unavailable after the replay budget is spent, it retries the storage handoff without another POST. This recovery boundary is best effort before the outbox row is created; receivers must still deduplicate repeated POSTs.
+Queued deliveries retain their pre-hook outbox copy until the worker completes delivery or stores a terminal failure. The reconciler checks the current job before replaying, retires the row of a completed job without a replay for as long as the queue still keeps that job (completed jobs are kept for an hour, at most the 1000 most recent), and keeps rows while Redis cannot be inspected. If failure persistence is unavailable after the replay budget is spent, it retries the storage handoff without another POST. This recovery boundary is best effort before the outbox row is created; receivers must still deduplicate repeated POSTs.
#### POST /api/webhooks/delivery-failures/redrive
@@ -7272,7 +7272,7 @@ Webhook delivery is **at-least-once**. A consumer can legitimately receive the s
- The underlying WhatsApp engine can re-fire an event for a single message.
- A failed delivery (non-2xx response, timeout, or network error) is retried.
-**Crash boundary.** For an ordinary dispatch, a successfully written outbox record retains the pre-hook event data until delivery succeeds, the subscription becomes ineligible, or the terminal failure is stored. Queued records keep this copy even after enqueue; the worker retires it only after one of those outcomes. If failure recording fails, the copy remains eligible for recovery. The bounded sweep (`WEBHOOK_RECONCILE_INTERVAL_MS`, default 60s) checks the stored job before replaying, leaves active jobs alone, retires the row of a completed job without a replay, and reuses the stored `X-OpenWA-Idempotency-Key`. If Redis cannot confirm the job state, replay waits. A missing or failed job, or a stranded direct delivery, can be replayed within `WEBHOOK_RECONCILE_MAX_ATTEMPTS`. A replay sent directly (the queue off, or its enqueue failed) that exhausts its retries stores its terminal failure row but keeps the outbox copy pending, so later sweeps replay it until that budget is spent; exhausting it retires the outbox only after terminal failure recording succeeds. Receiver deduplication remains necessary because a crash after receiver processing can cause another POST.
+**Crash boundary.** For an ordinary dispatch, a successfully written outbox record retains the pre-hook event data until delivery succeeds, the subscription becomes ineligible, or the terminal failure is stored. Queued records keep this copy even after enqueue; the worker retires it only after one of those outcomes. If failure recording fails, the copy remains eligible for recovery. The bounded sweep (`WEBHOOK_RECONCILE_INTERVAL_MS`, default 60s) checks the stored job before replaying, leaves active jobs alone, retires the row of a completed job without a replay for as long as the queue still keeps that job (completed jobs are kept for an hour, at most the 1000 most recent), and reuses the stored `X-OpenWA-Idempotency-Key`. If Redis cannot confirm the job state, replay waits. A missing or failed job, or a stranded direct delivery, can be replayed within `WEBHOOK_RECONCILE_MAX_ATTEMPTS`; a completed job already evicted from the queue reads as missing. A replay sent directly (the queue off, or its enqueue failed) that exhausts its retries stores its terminal failure row but keeps the outbox copy pending, so later sweeps replay it until that budget is spent; exhausting it retires the outbox only after terminal failure recording succeeds. Receiver deduplication remains necessary because a crash after receiver processing can cause another POST.
Outbox creation is best effort, and message persistence and outbox insertion are separate transactions. A write failure or crash between them can still lose the webhook delivery. Failure records with zero attempts are excluded from operator redrive. Capacity and shutdown rejections can still be recovered from an unsettled outbox record; shutdown during retry backoff may follow earlier POSTs. The direct dispatcher leaves its local in-flight work alone during reconciliation. Graceful shutdown drains current queue jobs for at most 10 s; direct sends, an operator redrive replay in flight, and an outbox replay in flight with its plugin hooks, get up to `WEBHOOK_SHUTDOWN_DRAIN_MS` (default 5s). That replay sends at most one more POST, bounded by `WEBHOOK_TIMEOUT`, and no further row is replayed; the next start replays what it left pending.