Skip to content

Free, Open Source, Self-Hosted WhatsApp API Gateway

docs(webhooks): document terminal delivery redrive in the runbook (#1793)

Yudhi Armyndharisfdd192cc224d
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 1c24a8591..af29b165f 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -66,6 +66,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Retain send idempotency keys after engine-stage failures, including HTTP 409.
- Reject poll votes with no matching options while preserving explicit vote clearing.
+### Documentation
+
+- The webhook runbook documents the ADMIN delivery-failure redrive and no longer says failed deliveries retry on their own.
+
### Upgrade notes (behavior changes)
- Baileys answers `400` for an animated sticker with more than 500 frames.
diff --git a/docs/11-operational-runbooks.md b/docs/11-operational-runbooks.md
index b5bd91bc2..f83b1ab6c 100644
--- a/docs/11-operational-runbooks.md
+++ b/docs/11-operational-runbooks.md
@@ -252,7 +252,7 @@ as a delivery failure rather than retaining payloads without limit.
**Prerequisites:**
-- An OPERATOR (or ADMIN) API key, and an ADMIN key for step 2
+- An OPERATOR (or ADMIN) API key, and an ADMIN key for steps 2 and 6
- Access to webhook endpoint
**Steps:**
@@ -323,8 +323,36 @@ curl -X PUT http://localhost:2785/api/sessions/{sessionId}/webhooks/{webhookId}
-H "Content-Type: application/json" \
-d '{"active": true}'
-# 6. Retry failed deliveries
-# No retry-failed API — failed deliveries auto-retry with exponential backoff (doc 06 §6.6)
+# 6. Replay deliveries that exhausted their retries
+# Automatic retry covers one delivery: up to the webhook's retryCount attempts with exponential
+# backoff (doc 06 §6.6). A step 2 row with attempts > 0 has spent those attempts; do not wait on
+# it. The outbox sweep runs only while WEBHOOK_RECONCILE_INTERVAL_MS > 0 (default 60000). It
+# sends an event again only if it was shed, refused at shutdown, or cut off by a restart or a
+# database fault before its dispatch settled, and then only until WEBHOOK_RECONCILE_MAX_ATTEMPTS
+# sweeps are spent, so such an event can show an attempts > 0 row while the sweep still holds it.
+# A row can be replayed only while step 2 lists it with "replayable": true, that is, it was
+# recorded while WEBHOOK_FAILURE_PAYLOAD_RETENTION_HOURS > 0 (with the queue on, also when the
+# event was queued) and that window has not passed. With the default 0 no event data is kept
+# and this call replays nothing; turning the setting on later does not cover rows already
+# recorded. Those events must be re-sent by their source.
+# Rows stay inside the ADMIN key's allowedSessions; sessionId, webhookId and ids only narrow
+# the batch. The webhook must be active and still subscribed to the event (re-enable it if
+# step 5C left it off). Each row gets one direct POST to the webhook's current URL with its
+# stored X-OpenWA-Idempotency-Key; a delivered row is removed, a failed one stays. If the sweep
+# still holds the event it can POST it again under the same key, so receivers must deduplicate.
+curl -X POST http://localhost:2785/api/webhooks/delivery-failures/redrive \
+ -H "X-API-Key: $ADMIN_API_KEY" \
+ -H "Content-Type: application/json" \
+ -d '{"sessionId": "{sessionId}", "webhookId": "{webhookId}", "limit": 50}'
+# Expected: {"redriven": N, "delivered": N, "enqueued": 0, "failed": 0, "skipped": 0, "remaining": 0}
+# failed > 0: the replay did not deliver, most often because the receiver still rejects it (the
+# server log names the cause); fix that before calling again. remaining > 0 alone does not mean
+# another call will succeed. skipped > 0: a webhook:before hook cancelled the replay or the webhook
+# changed during the call. A row listed in step 2 but left out of the batch is excluded: it holds
+# no event data, retention is now 0 or its window has passed, it has attempts 0, its webhook was
+# deleted, disabled or unsubscribed, or its session is outside the key's allowedSessions or the
+# body's sessionId. Body fields, limits and the full exclusion list are in doc 06
+# (POST /api/webhooks/delivery-failures/redrive).
```
**Verification:**
diff --git a/docs/21-glossary.md b/docs/21-glossary.md
index d6ecfaaa4..611083ef8 100644
--- a/docs/21-glossary.md
+++ b/docs/21-glossary.md
@@ -73,7 +73,7 @@ Web interface to manage OpenWA without using the API directly. Built with React
### Dead Letter Queue (DLQ)
-The durable record of deliveries abandoned after every retry; an outbound-webhook row with `attempts: 0` was instead shed, oversize, failed preflight or refused at shutdown, and a later successful delivery of the event removes its row. In OpenWA it is a **database table**, not a Redis queue: `webhook_delivery_failures` for outbound webhooks and `integration_delivery_failures` for plugin ingress. Used for debugging; **redrive exists for plugin ingress only** (`POST /api/integration/instances/:pluginId/:instanceId/redrive`); an outbound-webhook failure row is a record, not a replay source. The gateway replays a shed or shutdown-refused delivery from its outbox on its own; any other recovery means the provider or your own tooling re-sending the event.
+The durable record of deliveries abandoned after every retry; an outbound-webhook row with `attempts: 0` was instead shed, oversize, failed preflight or refused at shutdown, and a later successful delivery of the event removes its row. In OpenWA it is a **database table**, not a Redis queue: `webhook_delivery_failures` for outbound webhooks and `integration_delivery_failures` for plugin ingress. Both support an ADMIN redrive: plugin ingress with `POST /api/integration/instances/:pluginId/:instanceId/redrive`, and outbound webhooks with `POST /api/webhooks/delivery-failures/redrive`, which replays only terminal rows recorded while `WEBHOOK_FAILURE_PAYLOAD_RETENTION_HOURS` > 0 (default 0 keeps no event data) and still inside that window. While `WEBHOOK_RECONCILE_INTERVAL_MS` > 0 the gateway also replays a shed or shutdown-refused delivery from its outbox on its own. Neither path covers a delivery that was over the size cap or failed preflight, nor an exhausted one recorded without event data or past its retention window: the provider or your own tooling must re-send that event.
### Docker
diff --git a/src/common/docs-governance.spec.ts b/src/common/docs-governance.spec.ts
index 81a4494f2..f6f64630b 100644
--- a/src/common/docs-governance.spec.ts
+++ b/src/common/docs-governance.spec.ts
@@ -198,6 +198,29 @@ describe('governance docs match the repository', () => {
expect(risks).toMatch(/\[README disclaimer\]/);
});
+ // A delivery that spent its retries is not retried by its dispatch. Only a shed, shutdown-refused or
+ // interrupted event goes back through the outbox sweep, and the operator can replay a terminal row
+ // whose payload was kept.
+ it('points the webhook runbook and the glossary at the outbound redrive route', () => {
+ const route = '/api/webhooks/delivery-failures/redrive';
+ const staleClaim = /No retry-failed API|auto-retr|nothing retries|for plugin ingress only/i;
+ const openapi = JSON.parse(read('openapi.json')) as { paths: Record<string, { post?: unknown }> };
+ expect(openapi.paths[route]?.post).toBeDefined();
+ const runbook = between(read('docs/11-operational-runbooks.md'), '### Runbook: Webhook Delivery Failure', '\n### ');
+ expect(runbook).toContain(route);
+ expect(runbook).toContain('WEBHOOK_FAILURE_PAYLOAD_RETENTION_HOURS');
+ expect(runbook).not.toMatch(staleClaim);
+ // The sweep can still replay an attempts > 0 row, so a redrive may race it, but only while it runs.
+ expect(runbook).toContain('WEBHOOK_RECONCILE_MAX_ATTEMPTS');
+ expect(runbook).toContain('WEBHOOK_RECONCILE_INTERVAL_MS');
+ expect(runbook).toMatch(/skipped > 0/);
+ const glossary = between(read('docs/21-glossary.md'), '### Dead Letter Queue (DLQ)', '\n### ');
+ expect(glossary).toContain(route);
+ expect(glossary).toContain('WEBHOOK_FAILURE_PAYLOAD_RETENTION_HOURS');
+ expect(glossary).toContain('WEBHOOK_RECONCILE_INTERVAL_MS');
+ expect(glossary).not.toMatch(staleClaim);
+ });
+
it('keeps the README non-affiliation disclaimer', () => {
const section = between(read('README.md'), '## Disclaimer', '\n## ');
expect(section).toMatch(/not affiliated/);