Skip to content

Free, Open Source, Self-Hosted WhatsApp API Gateway

fix(session): preserve tombstones before message hooks

Yudhi Armyndharis179b9f1c90ea
diff --git a/docs/06-api-specification.md b/docs/06-api-specification.md
index b06f43512..28caf5763 100644
--- a/docs/06-api-specification.md
+++ b/docs/06-api-specification.md
@@ -7294,6 +7294,8 @@ These are the events OpenWA actually emits. A webhook is registered with an `eve
> **A message delivered already deleted.** whatsapp-web.js raises its message events for a message that WhatsApp Web adds already deleted for everyone, with no revoke before it (not yet seen on a live session). Its `message.sent`, and its `message.received` when it is the first delivery of the id, carry `type: 'revoked'` and an empty `body`; the `message:sent` and `message:received` plugin hooks get it with no content, and it is stored as a deleted message. The message already stored under its id, unless it is of another chat or sender, is cleared as a revoke clears it; no `message.revoked` goes out for that.
+> Before either message hook, a content delivery with an id makes one indexed lookup of its stored tombstone, so a later duplicate stays deleted across restarts. The new delivery lends no in-flight quote until the lookup finishes. If the lookup fails, the gateway logs a warning and keeps the delivery to preserve message receipt during storage failures; suppression based on a stored tombstone is unavailable for that delivery.
+
> **Large media is not inlined into webhook payloads.** A `media` blob whose decoded size exceeds `WEBHOOK_MEDIA_INLINE_MAX_BYTES` (default **1 MiB**; `0` = never inline) is replaced — before the payload is fanned out to your webhook — with the marker form `media: { mimetype, filename?, omitted: true, sizeBytes }`, the same shape the engine emits for capped inbound media. Media at or under the cap stays inline unchanged. Additionally, if the serialized body still exceeds `WEBHOOK_MAX_PAYLOAD_BYTES` (default **1 MiB**) after `webhook:before` hooks ran, any remaining inline media is shed the same way so the event is still delivered; only a payload that is over budget _without_ inline media is dropped (recorded in `GET /api/webhooks/delivery-failures`). Because shedding happens before enqueue, queued and failed BullMQ jobs in Redis never carry the blob either — failed-job retention is bounded by the queue's `removeOnComplete`/`removeOnFail` windows (1h/1000 completed, 24h/5000 failed). The two limits compound: `WEBHOOK_MAX_PAYLOAD_BYTES` is measured on the serialized JSON body after `WEBHOOK_MEDIA_INLINE_MAX_BYTES` has been applied, and base64 inflates media by a third, so at the defaults media above roughly **768 KiB** reaches your webhook as the omitted marker. To inline larger media, raise both together and keep the payload limit at least 4/3 of the inline limit plus room for the rest of the event (for example `5242880` inline with `7340032` payload). The **WebSocket** `message.received` / `message.sent` events apply only the inline cap, with the same marker: an over-cap blob is never broadcast to subscribed sockets (or, with the Redis adapter enabled, replicated across every replica's pub/sub link). On either sink, media the engine downloaded stays retrievable from [`GET /api/sessions/:sessionId/messages/:chatId/:messageId/media`](#get-apisessionssessionidmessageschatidmessageidmedia), which returns `404` when nothing was stored.
> **`event.response` reports RSVPs to WhatsApp events (Baileys only).** A WhatsApp event (the calendar-style invitation created from the attach menu, with "Going" / "Not going" / "Maybe" buttons) arrives as an ordinary `message.received` when it is posted. Each answer to it arrives as `event.response`:
diff --git a/src/engine/adapters/inbound-hold.spec.ts b/src/engine/adapters/inbound-hold.spec.ts
index 5a174b153..ccd98f9d5 100644
--- a/src/engine/adapters/inbound-hold.spec.ts
+++ b/src/engine/adapters/inbound-hold.spec.ts
@@ -1212,10 +1212,10 @@ describe('Baileys inbound messages held for storage', () => {
it('has the session hold the message when an edit reaches it, and the row takes the text', async () => {
const h = heldProjector({ unique: true, sqlite });
- const heldWhenEdited: boolean[] = [];
+ const quotedWhenEdited: boolean[] = [];
const rowsWhenEdited: number[] = [];
const onMessageEdited: EngineEventCallbacks['onMessageEdited'] = e => {
- heldWhenEdited.push(h.projector.inFlightInbound('s1', e.messageId) !== undefined);
+ quotedWhenEdited.push(h.projector.inFlightInbound('s1', e.messageId) !== undefined);
rowsWhenEdited.push(h.rows.length);
h.callbacks.onMessageEdited!(e);
};
@@ -1232,8 +1232,8 @@ describe('Baileys inbound messages held for storage', () => {
expect(h.announcedAs('message.edited')).toEqual([
expect.objectContaining({ messageId: 'M', body: 'fixed caption' }),
]);
- // PostgreSQL holds it from the arrival, before the emit, which inFlightInbound does not report yet.
- expect(heldWhenEdited).toEqual([sqlite]);
+ // The stored-message lookup has not finished, so neither database lends an unchecked quote.
+ expect(quotedWhenEdited).toEqual([false]);
// Either way the edit reaches it before its row is written, and an edit that meets the row with the same
// text is no edit to the session.
expect(rowsWhenEdited).toEqual([0]);
diff --git a/src/engine/adapters/wwebjs-message-changes.spec.ts b/src/engine/adapters/wwebjs-message-changes.spec.ts
index 07264ccf0..e32da62ce 100644
--- a/src/engine/adapters/wwebjs-message-changes.spec.ts
+++ b/src/engine/adapters/wwebjs-message-changes.spec.ts
@@ -2,7 +2,7 @@ import { EventEmitter } from 'events';
import { setFlagsFromString } from 'v8';
import { runInNewContext } from 'vm';
import { WAState, type Client } from 'whatsapp-web.js';
-import type { Repository } from 'typeorm';
+import { DataSource, type Repository } from 'typeorm';
import { WhatsAppWebJsAdapter } from './whatsapp-web-js.adapter';
import type { WwebjsLifecycle } from './wwebjs-lifecycle';
import type { WwebjsEngineHost } from './wwebjs-host';
@@ -11,7 +11,7 @@ import { EngineRegistry } from '../engine-registry.service';
import { MessageProjector } from '../../modules/session/message-projector.service';
import { deferred } from '../../modules/session/inbound-room';
import { createLogger } from '../../common/services/logger.service';
-import type { Message } from '../../modules/message/entities/message.entity';
+import { Message, MessageDirection } from '../../modules/message/entities/message.entity';
import type { Session } from '../../modules/session/entities/session.entity';
import type { EventsGateway } from '../../modules/events/events.gateway';
import type { WebhookService } from '../../modules/webhook/webhook.service';
@@ -683,14 +683,17 @@ describe.each([
});
// A send made through the API stores its row before its echo comes, so the echo loses its insert.
- it('announces the echo of a send the API stored unmarked, reading no row for it', async () => {
+ it('announces the echo of a send the API stored unmarked after checking its tombstone', async () => {
rows.push({ sessionId: 's1', waMessageId: 'S', chatId: CHAT, direction: 'outgoing', type: 'text', body: 'sent' });
client.emit('message_create', own('S', 'sent'));
await flush();
expect(events('message.sent')).toEqual([expect.objectContaining({ body: 'sent' })]);
expect(events('message.sent')[0]).not.toHaveProperty('edited');
- expect(findOne).not.toHaveBeenCalled();
+ expect(findOne).toHaveBeenCalledTimes(1);
+ expect(findOne).toHaveBeenCalledWith(
+ expect.objectContaining({ where: { sessionId: 's1', waMessageId: 'S', type: 'revoked' } }),
+ );
});
it('announces an own send edited during its media download with the mark each time it is delivered', async () => {
@@ -782,7 +785,7 @@ describe.each([
await flush();
expect(events('message.sent')).toEqual([
- expect.objectContaining({ body: 'e0', type: 'text', edited: true }),
+ expect.objectContaining({ body: '', type: 'revoked' }),
expect.objectContaining({ body: '', type: 'revoked' }),
]);
expect(events('message.sent')[1]).not.toHaveProperty('edited');
@@ -808,6 +811,207 @@ describe.each([
const deletedRow = (id: string) => ({ waMessageId: id, body: '', type: 'revoked', metadata: null });
const deletedDelivery = (id: string, fromMe = false) => ({ ...incoming(id, '', { fromMe }), type: 'revoked' });
+ it('takes an earlier queued own delivery back when a later one recognizes its stored tombstone', async () => {
+ const hook = deferred<void>();
+ hookGate = hook.promise;
+ client.emit('message', incoming('P', 'earlier'));
+ await flush();
+ client.emit('message_create', own('S', 'original text'));
+ await flush();
+ rows.push({ sessionId: 's1', chatId: CHAT, direction: 'outgoing', ...deletedRow('S') });
+ client.emit('message_create', own('S', 'stale duplicate'));
+ await flush();
+ hook.resolve();
+ await flush();
+
+ expect(events('message.sent')).toHaveLength(2);
+ for (const sent of events('message.sent'))
+ expect(sent).toEqual(expect.objectContaining({ body: '', type: 'revoked' }));
+ });
+
+ it.each([false, true])(
+ 'hooks a later content delivery of a message delivered deleted with no content, own=%s',
+ async fromMe => {
+ const event = fromMe ? 'message_create' : 'message';
+ client.emit(event, deletedDelivery('A', fromMe));
+ await flush();
+ client.emit(event, incoming('A', 'stale text', { fromMe }));
+ await flush();
+
+ expect(hooked).toEqual(['', '']);
+ expect(rows).toEqual([expect.objectContaining(deletedRow('A'))]);
+ expect(dispatchedAfterDeleted()).toEqual([]);
+ },
+ );
+
+ it.each([false, true])(
+ 'hooks a content delivery of a stored tombstone with no content after other messages, own=%s',
+ async fromMe => {
+ // These rows outlive the projector that stored them, and any bounded record of recent messages.
+ for (let n = 0; n <= 501; n++)
+ rows.push({
+ sessionId: 's1',
+ chatId: CHAT,
+ direction: fromMe ? 'outgoing' : 'incoming',
+ ...deletedRow(`A${n}`),
+ });
+ client.emit(fromMe ? 'message_create' : 'message', incoming('A0', 'stale text', { fromMe }));
+ await flush();
+
+ expect(hooked).toEqual(['']);
+ expect(rows[0]).toEqual(expect.objectContaining(deletedRow('A0')));
+ expect(contentAnnounced()).toEqual([]);
+ },
+ );
+
+ it.each([
+ ['chat', CHAT, '[email protected]', 'incoming', undefined, undefined],
+ ['direction', CHAT, CHAT, 'outgoing', undefined, undefined],
+ ])(
+ 'keeps the content of a delivery whose %s differs from the stored tombstone',
+ async (_, chatId, storedChat, direction, storedAuthor, author) => {
+ rows.push({
+ sessionId: 's1',
+ waMessageId: 'A',
+ chatId: storedChat,
+ direction,
+ author: storedAuthor,
+ type: 'revoked',
+ body: '',
+ });
+ projector.handleInboundMessage('s1', engine, {
+ id: 'A',
+ chatId,
+ from: chatId,
+ to: SELF,
+ fromMe: false,
+ isGroup: chatId.endsWith('@g.us'),
+ kind: 'individual',
+ body: 'another message',
+ type: 'text',
+ timestamp: 1_700_000_000,
+ author,
+ });
+ await flush();
+
+ expect(hooked).toEqual(['another message']);
+ expect(rows[0].body).toBe('');
+ },
+ );
+
+ it.each([false, true])('keeps a real delivery when the tombstone lookup is unavailable, own=%s', async fromMe => {
+ findOne.mockRejectedValueOnce(new Error('database unavailable'));
+ client.emit(fromMe ? 'message_create' : 'message', incoming('A', 'real message', { fromMe }));
+ await flush();
+
+ expect(hooked).toEqual(['real message']);
+ expect(rows).toEqual([expect.objectContaining({ waMessageId: 'A', body: 'real message', type: 'text' })]);
+ expect(contentAnnounced()).toHaveLength(1);
+ });
+
+ it.each([
+ [CHAT, ''],
+ ['[email protected]', 'kept'],
+ ])('checks content still being read against a deleted delivery in %s', async (chatId, expected) => {
+ const read = deferred<null>();
+ findOne.mockReturnValueOnce(read.promise);
+ const original = {
+ id: 'A',
+ chatId: CHAT,
+ from: CHAT,
+ to: SELF,
+ fromMe: false,
+ isGroup: false,
+ kind: 'individual',
+ body: 'kept',
+ type: 'text',
+ timestamp: 1_700_000_000,
+ } as IncomingMessage;
+ projector.handleInboundMessage('s1', engine, original);
+ projector.handleInboundMessage('s1', engine, {
+ ...original,
+ chatId,
+ from: chatId,
+ type: 'revoked',
+ body: '',
+ });
+ await flush();
+ read.resolve(null);
+ await flush();
+
+ expect(hooked.at(-1)).toBe(expected);
+ });
+
+ it('keeps another chat delivery separate from a deleted message still in its hook', async () => {
+ const hook = deferred<void>();
+ hookGate = hook.promise;
+ const deleted = {
+ id: 'A',
+ chatId: CHAT,
+ from: CHAT,
+ to: SELF,
+ fromMe: false,
+ isGroup: false,
+ kind: 'individual',
+ body: '',
+ type: 'revoked',
+ timestamp: 1_700_000_000,
+ } as IncomingMessage;
+ projector.handleInboundMessage('s1', engine, deleted);
+ await flush();
+ projector.handleInboundMessage('s1', engine, {
+ ...deleted,
+ chatId: '[email protected]',
+ type: 'text',
+ body: 'another message',
+ });
+ await flush();
+
+ expect(hooked).toEqual(['', 'another message']);
+ hook.resolve();
+ await flush();
+ });
+
+ it.each([false, true])('runs no hook after its engine retired during the tombstone lookup, own=%s', async fromMe => {
+ const read = deferred<null>();
+ findOne.mockReturnValueOnce(read.promise);
+ client.emit(fromMe ? 'message_create' : 'message', incoming('A', 'late message', { fromMe }));
+ await flush();
+ engines.delete('s1');
+ read.resolve(null);
+ await flush();
+
+ expect(hooked).toEqual([]);
+ expect(rows).toEqual([]);
+ expect(contentAnnounced()).toEqual([]);
+ });
+
+ it.each([
+ [false, false],
+ [false, true],
+ [true, false],
+ [true, true],
+ ])('lends no unchecked quote, then the checked body, deleted=%s own=%s', async (deleted, fromMe) => {
+ const row = { sessionId: 's1', chatId: CHAT, direction: fromMe ? 'outgoing' : 'incoming', ...deletedRow('A') };
+ if (deleted) rows.push(row);
+ const read = deferred<Row | null>();
+ findOne.mockReturnValueOnce(read.promise);
+ let quoted: unknown;
+ rewrite = data => {
+ quoted = projector.inFlightInbound('s1', 'A');
+ return data;
+ };
+ client.emit(fromMe ? 'message_create' : 'message', incoming('A', 'original text', { fromMe }));
+ await flush();
+ expect(projector.inFlightInbound('s1', 'A')).toBeUndefined();
+
+ read.resolve(deleted ? row : null);
+ await flush();
+ expect(quoted).toEqual({ chatId: CHAT, body: deleted ? '' : 'original text' });
+ });
+
it('stores an own message stored and edited, then delivered again as deleted, as deleted', async () => {
client.emit('message_create', own('S', 'orig'));
await flush();
@@ -863,7 +1067,8 @@ describe.each([
hookGate = hook.promise;
client.emit('message', incoming('P', 'earlier'));
await flush();
- failInsert = row => row.waMessageId === 'S' && row.type !== 'revoked';
+ let inserts = 0;
+ failInsert = row => row.waMessageId === 'S' && ++inserts === 1;
client.emit(fromMe ? 'message_create' : 'message', incoming('S', 'orig', { fromMe }));
await flush();
editEvent(client, 'S', 'e0', { fromMe });
@@ -1672,3 +1877,78 @@ describe('changes kept while whatsapp-web.js processes a message (adapter level)
expect(process.memoryUsage().heapUsed - before).toBeLessThan((MESSAGES * LONGEST_TEXT) / 4);
});
});
+
+describe('stored tombstones on a shared SQLite connection', () => {
+ it.each([
+ [false, false],
+ [false, true],
+ [true, false],
+ [true, true],
+ ])('waits for another transaction before trusting the row, commit=%s own=%s', async (commit, fromMe) => {
+ const db = new DataSource({ type: 'better-sqlite3', database: ':memory:', entities: [Message], synchronize: true });
+ await db.initialize();
+ const repository = db.getRepository(Message);
+ const runner = db.createQueryRunner();
+ const engines = new EngineRegistry();
+ const engine = {} as IWhatsAppEngine;
+ engines.set('s1', engine);
+ const seen: string[] = [];
+ const message: IncomingMessage = {
+ id: 'A',
+ chatId: CHAT,
+ from: fromMe ? SELF : CHAT,
+ to: fromMe ? CHAT : SELF,
+ fromMe,
+ isGroup: false,
+ kind: 'individual',
+ type: 'text',
+ body: 'original text',
+ timestamp: 1_700_000_000,
+ };
+ const projector = new MessageProjector(
+ repository,
+ { update: jest.fn().mockResolvedValue({ affected: 1 }) } as unknown as Repository<Session>,
+ engines,
+ new Proxy({}, { get: () => jest.fn() }) as unknown as EventsGateway,
+ { dispatch: jest.fn().mockResolvedValue(undefined) } as unknown as WebhookService,
+ {
+ execute: jest.fn((event: string, data: unknown) => {
+ if (event === 'message:sent' || event === 'message:received') seen.push((data as IncomingMessage).body);
+ return Promise.resolve({ continue: true, data });
+ }),
+ } as unknown as HookManager,
+ { ingest: jest.fn() } as unknown as StatusStoreService,
+ { resolveSenderPhone: jest.fn().mockResolvedValue(null) } as unknown as SessionLidResolver,
+ );
+ try {
+ await repository.insert({
+ sessionId: 's1',
+ waMessageId: 'A',
+ chatId: CHAT,
+ from: message.from,
+ to: message.to,
+ direction: fromMe ? MessageDirection.OUTGOING : MessageDirection.INCOMING,
+ type: 'text',
+ body: 'original text',
+ timestamp: message.timestamp,
+ });
+ await runner.startTransaction();
+ await runner.manager.update(Message, { waMessageId: 'A' }, { type: 'revoked', body: '' });
+ if (fromMe) projector.handleOwnSendEcho('s1', engine, message);
+ else projector.handleInboundMessage('s1', engine, message);
+ await flush();
+ expect(seen).toEqual([]);
+ if (commit) await runner.commitTransaction();
+ else await runner.rollbackTransaction();
+ for (let n = 0; n < 20; n++) await new Promise(resolve => setTimeout(resolve, 2));
+ expect(seen).toEqual([commit ? '' : 'original text']);
+ expect(await repository.findOneByOrFail({ waMessageId: 'A' })).toEqual(
+ expect.objectContaining({ type: commit ? 'revoked' : 'text', body: commit ? '' : 'original text' }),
+ );
+ } finally {
+ if (runner.isTransactionActive) await runner.rollbackTransaction();
+ await runner.release();
+ await db.destroy();
+ }
+ });
+});
diff --git a/src/modules/session/message-projector.inbound-hold.spec.ts b/src/modules/session/message-projector.inbound-hold.spec.ts
index 4ae7a5855..bc97893d1 100644
--- a/src/modules/session/message-projector.inbound-hold.spec.ts
+++ b/src/modules/session/message-projector.inbound-hold.spec.ts
@@ -990,15 +990,16 @@ describe('MessageProjector holding inbound messages (PostgreSQL)', () => {
expect(dispatched('message.received')).toEqual([]);
});
- it('lends nothing to a quote before the emit, and the message once emitted', async () => {
+ it('lends nothing to a quote before the stored-message check, and the message after it', async () => {
let release: () => void = () => undefined;
insertGate = () => new Promise(resolve => (release = resolve));
const t = admit(arrival('A', 'chat1'));
expect(projector.inFlightInbound('s1', 'A')).toBeUndefined();
projector.handleInboundMessage('s1', engine, message('A', 'chat1'), t);
- expect(projector.inFlightInbound('s1', 'A')).toEqual(expect.objectContaining({ body: 'body of A' }));
+ expect(projector.inFlightInbound('s1', 'A')).toBeUndefined();
await flush();
+ expect(projector.inFlightInbound('s1', 'A')).toEqual(expect.objectContaining({ body: 'body of A' }));
release();
await flush();
expect(projector.inFlightInbound('s1', 'A')).toBeUndefined();
diff --git a/src/modules/session/message-projector.service.spec.ts b/src/modules/session/message-projector.service.spec.ts
index a26c865ec..b1c4fc7f5 100644
--- a/src/modules/session/message-projector.service.spec.ts
+++ b/src/modules/session/message-projector.service.spec.ts
@@ -766,12 +766,13 @@ describe('MessageProjector (inbound projection)', () => {
expect(projector.inFlightInbound('other-session', 'wamid.1')).toBeUndefined();
});
- it('lends no text to a quote once the message is revoked before its row is written', () => {
+ it('lends no text to a quote once the message is revoked before its row is written', async () => {
const engine = makeEngine();
engines.set(SESSION_ID, engine);
hookManager.execute.mockImplementationOnce(() => new Promise(() => undefined));
projector.handleInboundMessage(SESSION_ID, engine, makeIncoming());
+ await settle();
projector.handleMessageRevoked(SESSION_ID, engine, { id: 'wamid.1' } as never);
expect(projector.inFlightInbound(SESSION_ID, 'wamid.1')).toEqual({ chatId: '[email protected]', body: '' });
@@ -1327,6 +1328,7 @@ describe('MessageProjector (inbound projection)', () => {
projector.handleInboundMessage(SESSION_ID, engine, makeIncoming({ id: 'A', chatId: chatA }));
projector.handleInboundMessage(SESSION_ID, engine, makeIncoming({ id: 'B', chatId: chatA }));
+ await flush();
// Both chains started: hooks still run concurrently.
expect(hookManager.execute).toHaveBeenCalledTimes(2);
@@ -1348,6 +1350,7 @@ describe('MessageProjector (inbound projection)', () => {
projector.handleInboundMessage(SESSION_ID, engine, makeIncoming({ id: 'A', chatId: chatA }));
projector.handleInboundMessage(SESSION_ID, engine, makeIncoming({ id: 'C', chatId: '[email protected]' }));
+ await flush();
release.get('C')!();
await flush();
@@ -1361,6 +1364,7 @@ describe('MessageProjector (inbound projection)', () => {
projector.handleInboundMessage(SESSION_ID, engine, makeIncoming({ id: 'A', chatId: chatA }));
projector.handleInboundMessage(SESSION_ID, engine, makeIncoming({ id: 'B', chatId: chatA }));
+ await flush();
release.get('A')!(true);
release.get('B')!();
await flush();
@@ -1376,6 +1380,7 @@ describe('MessageProjector (inbound projection)', () => {
projector.handleInboundMessage(SESSION_ID, engine, makeIncoming({ id: 'A', chatId: chatA }));
projector.handleInboundMessage(SESSION_ID, engine, makeIncoming({ id: 'B', chatId: chatA }));
+ await flush();
release.get('B')!();
await flush();
engines.set(SESSION_ID, makeEngine());
@@ -1393,6 +1398,7 @@ describe('MessageProjector (inbound projection)', () => {
projector.handleInboundMessage(SESSION_ID, engine, makeIncoming({ id: 'A', chatId: chatA }));
projector.handleInboundMessage(SESSION_ID, engine, makeIncoming({ id: 'B', chatId: chatA }));
+ await flush();
release.get('B')!();
await flush();
projector.handleMessageRevoked(SESSION_ID, engine, { id: 'B' } as never);
@@ -1429,6 +1435,7 @@ describe('MessageProjector (inbound projection)', () => {
});
if (fromMe) projector.handleOwnSendEcho(SESSION_ID, engine, b);
else projector.handleInboundMessage(SESSION_ID, engine, b);
+ await flush();
release.get('B')!();
await flush();
// The REST delete-for-me: no engine revoke event follows, so no message.revoked went out.
@@ -1456,6 +1463,7 @@ describe('MessageProjector (inbound projection)', () => {
projector.handleInboundMessage(SESSION_ID, engine, makeIncoming({ id: 'A', chatId: chatA }));
projector.handleInboundMessage(SESSION_ID, engine, makeIncoming({ id: 'B', chatId: chatA, body: 'typo' }));
+ await flush();
release.get('B')!();
await flush();
projector.applyMessageEditQueued(SESSION_ID, { messageId: 'B', body: 'fixed' } as never);
@@ -1480,6 +1488,7 @@ describe('MessageProjector (inbound projection)', () => {
projector.handleInboundMessage(SESSION_ID, engine, makeIncoming({ id: 'A', chatId: chatA }));
projector.handleOwnSendEcho(SESSION_ID, engine, makeIncoming({ id: 'S', chatId: chatA, fromMe: true }));
+ await flush();
release.get('S')!();
await flush();
expect(inserted()).toEqual([]);
@@ -1498,6 +1507,7 @@ describe('MessageProjector (inbound projection)', () => {
const release = holdHooks();
projector.handleInboundMessage(SESSION_ID, engine, makeIncoming({ id: 'A', chatId: chatA }));
projector.handleOwnSendEcho(SESSION_ID, engine, makeIncoming({ id: 'S', chatId: chatA, fromMe: true }));
+ await flush();
release.get('S')!();
await flush();
return { engine, releaseA: () => release.get('A')!() };
@@ -1690,7 +1700,10 @@ describe('MessageProjector (inbound projection)', () => {
await flush();
await expect(receipt?.settled).resolves.toBeUndefined();
- expect(messageRepository.findOne).not.toHaveBeenCalled();
+ expect(messageRepository.findOne).toHaveBeenCalledTimes(1);
+ expect(messageRepository.findOne).toHaveBeenCalledWith(
+ expect.objectContaining({ where: { sessionId: SESSION_ID, waMessageId: 'wamid.1', type: 'revoked' } }),
+ );
expect(received()).toHaveLength(1);
});
@@ -1981,7 +1994,10 @@ describe('MessageProjector (inbound projection)', () => {
await flush();
await expect(receipt?.settled).resolves.toBeUndefined();
- expect(messageRepository.findOne).not.toHaveBeenCalled();
+ expect(messageRepository.findOne).toHaveBeenCalledTimes(1);
+ expect(messageRepository.findOne).toHaveBeenCalledWith(
+ expect.objectContaining({ where: { sessionId: SESSION_ID, waMessageId: 'wamid.1', type: 'revoked' } }),
+ );
expect(received()).toHaveLength(1);
});
diff --git a/src/modules/session/message-projector.service.ts b/src/modules/session/message-projector.service.ts
index 2fa6570c7..bddbb4b9e 100644
--- a/src/modules/session/message-projector.service.ts
+++ b/src/modules/session/message-projector.service.ts
@@ -7,6 +7,7 @@ import { Session } from './entities/session.entity';
import { Message, MessageDirection, MessageStatus } from '../message/entities/message.entity';
import { EngineRegistry } from '../../engine/engine-registry.service';
import { KeyedMutationQueue } from '../../common/utils/keyed-mutation-queue';
+import { awaitNoOpenTransaction } from '../../common/utils/shared-connection';
import { SessionLidResolver } from './session-lid-resolver.service';
import { buildMessageMetadata, REVOKED_ROW_PATCH, storableWaMessageId } from './message-row.mapper';
import { MessageMutationProjector } from './message-mutation-projector';
@@ -442,7 +443,7 @@ export class MessageProjector {
// handler's rewrite.
const inFlightKey = `${id}:${message.id}`;
const inFlight = turn
- ? Object.assign(turn.entry, { quote: quoteOf(messageData), ref: turn.entry.ref ?? refOf(messageData) })
+ ? Object.assign(turn.entry, { ref: turn.entry.ref ?? refOf(messageData) })
: this.trackInFlight(inFlightKey, messageData);
this.arrive(inFlight, message);
@@ -453,7 +454,7 @@ export class MessageProjector {
// Execute hook for message received - plugins can modify or stop processing. The hook chain and
// sender resolution run concurrently across messages; the catch is attached now so a failure
// while the commit waits its turn is never an unhandled rejection.
- const prepared = this.startHookChain(id, engine, turn, messageData, () =>
+ const prepared = this.startHookChain(id, engine, turn, messageData, messageData =>
this.hookManager
.execute('message:received', messageData, {
sessionId: id,
@@ -465,7 +466,7 @@ export class MessageProjector {
},
})
.then(({ data }) =>
- this.prepareInboundMessage(id, this.messageOrEngineCopy(id, 'message:received', data, message)),
+ this.prepareInboundMessage(id, this.messageOrEngineCopy(id, 'message:received', data, messageData)),
),
).catch(onFailure);
const outcome = deferred<InboundOutcome | undefined>();
@@ -487,22 +488,58 @@ export class MessageProjector {
* Start a message's hook chain: at once without a turn. With one, once every earlier admitted
* message of the chat was emitted or dropped, so hooks start in arrival order. A message revoked
* before then skips its hooks, as its content is gone, and an edit made before then reaches them.
+ * A delivery of content checks for a stored tombstone before either hook; one that came deleted
+ * reaches them cleared. An unavailable database keeps the delivery, as the storage path does.
*/
private startHookChain(
id: string,
engine: IWhatsAppEngine,
turn: InboundTurn | undefined,
messageData: InboundMessageData,
- run: () => Promise<InboundMessageData>,
+ run: (message: InboundMessageData) => Promise<InboundMessageData>,
): Promise<InboundMessageData | null> {
- if (!turn) return run();
+ const checked = async (): Promise<InboundMessageData | null> => {
+ let data = messageData;
+ if (data.id && data.type !== 'revoked') {
+ try {
+ const connection = this.messageRepository.manager?.connection;
+ if (connection && !(await awaitNoOpenTransaction(connection))) {
+ throw new Error('Timed out waiting for the message database transaction');
+ }
+ // The unique message key bounds this read. A tombstone outlives this projector and any engine copy.
+ const row = await this.messageRepository.findOne({
+ where: { sessionId: id, waMessageId: data.id, type: 'revoked' },
+ select: { type: true, chatId: true, direction: true, author: true },
+ });
+ if (row?.type === 'revoked' && !mutationTargetsForeignRow(row, claimOf(data))) {
+ data = deletedPlaceholder(data);
+ }
+ } catch (error) {
+ // Preserve the receive path's fail-open behavior when storage is unavailable.
+ this.logger.warn('Failed to read the stored message before its hooks; keeping the delivery', {
+ sessionId: id,
+ messageId: data.id,
+ error: String(error),
+ });
+ }
+ }
+ if (!this.engines.isLive(id, engine)) return null;
+ const pending = this.inboundInFlight.get(`${id}:${data.id}`);
+ if (pending && !this.foreignToInFlight(pending, claimOf(data))) {
+ if (data.type === 'revoked') this.holdRevoked(id, pending);
+ if (pending.revoked) data = deletedPlaceholder(data);
+ pending.quote = quoteOf(data);
+ }
+ return run(data);
+ };
+ if (!turn) return checked();
const { entry } = turn;
return turn.hookGate.promise.then(() => {
if (!this.engines.isLive(id, engine)) return null;
- if (entry.revoked) return messageData;
+ if (entry.revoked && messageData.type !== 'revoked') return messageData;
// A delivery that came deleted has no text for the edit to replace.
if (entry.editedBody !== undefined && messageData.type !== 'revoked') messageData.body = entry.editedBody;
- return run();
+ return checked();
});
}
@@ -519,10 +556,11 @@ export class MessageProjector {
/** Record a message as not yet written; a re-fire of the same id keeps the changes recorded so far. */
private trackInFlight(key: string, message: InboundMessageData): InFlightEntry {
- const { ref, revoked, revokeAnnounced, editedBody, edited, ackStatus, reactions } =
+ const { quote, ref, revoked, revokeAnnounced, editedBody, edited, ackStatus, reactions } =
this.inboundInFlight.get(key) ?? {};
const inFlight: InFlightEntry = {
- quote: quoteOf(message),
+ // A new delivery lends no unchecked content while its stored-message lookup waits.
+ quote,
// The first delivery of an id says whose message it is: a repeat sent from another chat must not
// take that over from it, or a change aimed through the repeat would be judged by the wrong rule.
ref: ref ?? refOf(message),
@@ -1137,12 +1175,12 @@ export class MessageProjector {
// path, the entry follows the chain's rewrites, so a quote carries what the row will hold.
const inFlightKey = `${id}:${message.id}`;
const inFlight = turn
- ? Object.assign(turn.entry, { quote: quoteOf(messageData), ref: turn.entry.ref ?? refOf(messageData) })
+ ? Object.assign(turn.entry, { ref: turn.entry.ref ?? refOf(messageData) })
: this.trackInFlight(inFlightKey, messageData);
this.arrive(inFlight, message);
// Execute hook for message sent - plugins can modify or stop processing. Like the inbound path,
// the hook chain runs concurrently and the commit waits its turn on the chat's queue.
- const prepared = this.startHookChain(id, engine, turn, messageData, () =>
+ const prepared = this.startHookChain(id, engine, turn, messageData, messageData =>
this.hookManager
.execute('message:sent', messageData, {
sessionId: id,
@@ -1153,7 +1191,7 @@ export class MessageProjector {
return true;
},
})
- .then(({ data }) => this.messageOrEngineCopy(id, 'message:sent', data, message)),
+ .then(({ data }) => this.messageOrEngineCopy(id, 'message:sent', data, messageData)),
).catch(onFailure);
this.commitInTurn(id, message, turn, async () => {
try {