Skip to content

Free, Open Source, Self-Hosted WhatsApp API Gateway

fix(infra): drop send idempotency claims of sessions a restore removes

Yudhi Armyndharis94a74e6bf26a
diff --git a/docs/05-database-design.md b/docs/05-database-design.md
index 6fa00b834..b4f6c1180 100644
--- a/docs/05-database-design.md
+++ b/docs/05-database-design.md
@@ -715,7 +715,7 @@ The data connection also owns:
- **`status_updates`** — inbound status/story broadcasts with a 24-hour TTL (`src/modules/status-store/entities/status-update.entity.ts`); unique on `(sessionId, waStatusId)`. Attached media is stored via `StorageService`, not in the row.
- **`webhook_delivery_failures`**: durable record of a webhook delivery that exhausted its retries, or was not sent (attempts 0) (`src/modules/webhook/entities/webhook-delivery-failure.entity.ts`), surfaced via the ADMIN `GET /webhooks/delivery-failures`. A shed or shutdown-refused delivery is replayed by the outbox until its replay budget runs out; an oversize payload or a preflight failure on first dispatch is retired after its failure record is stored. A later successful delivery removes the row. While `WEBHOOK_FAILURE_PAYLOAD_RETENTION_HOURS` > 0 (default 0), a terminal row (attempts > 0) also keeps the pre-hook event data in `payload` (never selected by the listing, which reports `replayable` instead) so ADMIN `POST /webhooks/delivery-failures/redrive` can replay it with the stored idempotency key; the payload is cleared once that window passes and the row stays.
- **`webhook_outbox_events`**: the outbound delivery record (`src/modules/webhook/entities/webhook-outbox-event.entity.ts`). A successfully written `pending` row holds pre-hook event data before the attempt. Enqueue changes it to `queued` without dropping the payload. For an ordinary dispatch, delivery success, subscription invalidation or a successfully recorded terminal failure retires the copy; failed terminal recording leaves it available for reconciliation. A sweep replay sent directly (the queue off, or its enqueue failed) that fails keeps the copy pending until the replay budget is spent, and a delivered redrive of the event also retires it. The reconciler checks the stored job before replaying queued work and reuses its idempotency key. Settled rows drop their payload and are pruned on age (§5.7). Message persistence and outbox insertion remain separate transactions.
-- **`send_idempotency_keys`**: one claimed `Idempotency-Key` per `(sessionId, idempotencyKey)` (unique) from the opt-in header on the message send routes (`src/modules/message/entities/send-idempotency-key.entity.ts`). The insert that wins the unique key runs the send; the row then holds `state` (`pending`, `completed` with the `responseBody` replayed to retries, or `failed` when the outcome is unknown), the `route` and a SHA-256 `requestHash` of route and body. Rows expire 24 hours after the claim (`expiresAt`) and are pruned hourly (§5.7). Not exported by `GET /infra/export-data`.
+- **`send_idempotency_keys`**: one claimed `Idempotency-Key` per `(sessionId, idempotencyKey)` (unique) from the opt-in header on the message send routes (`src/modules/message/entities/send-idempotency-key.entity.ts`). The insert that wins the unique key runs the send; the row then holds `state` (`pending`, `completed` with the `responseBody` replayed to retries, or `failed` when the outcome is unknown), the `route` and a SHA-256 `requestHash` of route and body. Rows expire 24 hours after the claim (`expiresAt`) and are pruned hourly (§5.7). Not exported by `GET /infra/export-data`; a replace-all import keeps the claims of the sessions it restores and removes the others.
- **`plugin_instances`** — one configured instance of an adapter plugin, keyed `${pluginId}:${instanceId}` (`src/modules/integration/entities/plugin-instance.entity.ts`); holds the host-minted ingress HMAC secret, masked on API reads.
- **`ingress_events`** — persist-before-ack durable row and inbound dedup oracle, unique on `(pluginId, instanceId, providerDeliveryId)` (`src/modules/integration/entities/ingress-event.entity.ts`). The full payload is retired to `NULL` once dispatch is settled, leaving a slim dedup marker.
- **`conversation_mappings`** — bidirectional WA-chat ↔ provider-conversation mapping plus handover state (`src/modules/integration/entities/conversation-mapping.entity.ts`).
diff --git a/src/modules/infra/infra-data.controller.spec.ts b/src/modules/infra/infra-data.controller.spec.ts
index 1e375f640..095947564 100644
--- a/src/modules/infra/infra-data.controller.spec.ts
+++ b/src/modules/infra/infra-data.controller.spec.ts
@@ -43,6 +43,7 @@ import { WebhookOutboxEvent } from '../webhook/entities/webhook-outbox-event.ent
import { WebhookOutboxService } from '../webhook/webhook-outbox.service';
import { IntegrationDeliveryFailure } from '../integration/entities/integration-delivery-failure.entity';
import { StatusUpdate } from '../status-store/entities/status-update.entity';
+import { SendIdempotencyKey } from '../message/entities/send-idempotency-key.entity';
import { AutomationRule } from '../automation/entities/automation-rule.entity';
import { AuditAction } from '../audit/entities/audit-log.entity';
import { ScopeBindingService } from '../integration/scope-binding.service';
@@ -85,6 +86,7 @@ describe('InfraDataController.importData round-trips export-data (no silent mess
IntegrationDeliveryFailure,
StatusUpdate,
AutomationRule,
+ SendIdempotencyKey,
],
synchronize: true,
});
@@ -159,6 +161,40 @@ describe('InfraDataController.importData round-trips export-data (no silent mess
expect((await ds.getRepository(Session).findOneByOrFail({ id: 's2' })).desiredState).toBeNull();
});
+ it('drops the send idempotency claims of a session the restore removes and keeps those of one it restores', async () => {
+ await seedSession('s1');
+ await seedSession('s2');
+ const claims = ds.getRepository(SendIdempotencyKey);
+ for (const sessionId of ['s1', 's2']) {
+ await claims.save(
+ claims.create({
+ sessionId,
+ idempotencyKey: 'order-42',
+ route: 'send-text',
+ requestHash: 'h',
+ state: 'completed',
+ responseBody: { messageId: `sent-by-${sessionId}` },
+ expiresAt: new Date(Date.now() + 3_600_000),
+ }),
+ );
+ }
+ const both = await controller.exportData();
+ const onlyS2 = {
+ tables: {
+ ...both.tables,
+ sessions: (both.tables.sessions as unknown as Array<{ id: string }>).filter(row => row.id === 's2'),
+ },
+ };
+
+ expect(await controller.importData(onlyS2 as never)).toMatchObject({ imported: true });
+ expect((await claims.find()).map(claim => claim.sessionId)).toEqual(['s2']);
+
+ // A later restore that brings s1 back must not answer a retried send with the old stored response.
+ expect(await controller.importData({ tables: both.tables })).toMatchObject({ imported: true });
+ expect(await claims.findOneBy({ sessionId: 's1' })).toBeNull();
+ expect(await claims.findOneBy({ sessionId: 's2' })).not.toBeNull();
+ });
+
it('carries LIVE claims forward by their remaining time, including a peer node’s', async () => {
await seedSession('s1');
await seedSession('s2');
diff --git a/src/modules/infra/infra-data.service.ts b/src/modules/infra/infra-data.service.ts
index 15289bb68..79f3cdba1 100644
--- a/src/modules/infra/infra-data.service.ts
+++ b/src/modules/infra/infra-data.service.ts
@@ -760,9 +760,9 @@ export class InfraDataService {
// OTHER failure (lock, I/O, aborted tx) propagate to the transaction rollback below — a blind
// `.catch(() => {})` here could otherwise silently commit a MERGED (not replaced) restore on
// SQLite, violating the endpoint's "replaces existing data" contract.
- const clearTable = async (table: string): Promise<void> => {
+ const clearTable = async (table: string, where = ''): Promise<void> => {
try {
- await queryRunner.query(`DELETE FROM ${table}`);
+ await queryRunner.query(`DELETE FROM ${table}${where}`);
} catch (err) {
if (!isMissingTableError(err)) throw err;
this.logger.debug('Skipped clearing a table that does not exist during import', { table });
@@ -935,6 +935,20 @@ export class InfraDataService {
}
}
+ // send_idempotency_keys is not archived, so it is trimmed rather than cleared. A claim of a session
+ // this restore dropped would outlive it by up to 24 h, and a later restore that brings the id back
+ // would answer a retried send with the old stored response instead of sending it. The claims of a
+ // session the restore keeps stay: a retry of a send made before the restore still replays.
+ if (warnings.length === 0) {
+ try {
+ await clearTable('send_idempotency_keys', ' WHERE "sessionId" NOT IN (SELECT "id" FROM sessions)');
+ } catch (error) {
+ warnings.push(
+ `Failed to drop the send idempotency claims of removed sessions: ${error instanceof Error ? error.message : String(error)}`,
+ );
+ }
+ }
+
// "Replace all data" must be all-or-nothing: the import already DELETEd every row, so if any
// INSERT failed we must roll back (restoring the pre-import data) rather than commit a
// half-wiped DB and report success. A partial restore reported as imported:true was how