Free, Open Source, Self-Hosted WhatsApp API Gateway
TypeScript apibotgatewayself-hostedwhatsappwhatsapp-apiwhatsapp-automationwhatsapp-botwhatsapp-gatewaywhatsapp-gateway-apiwhatsapp-webwhatsapp-web-api
fix(session): announce a repeated own message with its edit mark
Yudhi Armyndharis65fdd866a820
diff --git a/docs/06-api-specification.md b/docs/06-api-specification.mdindex 9c53ce06e..710b8ac08 100644--- a/docs/06-api-specification.md+++ b/docs/06-api-specification.md@@ -7288,7 +7288,7 @@ These are the events OpenWA actually emits. A webhook is registered with an `eve> **`contact` on `message.received` is minimal unless you opt in (`WEBHOOK_CONTACT_DETAILS`).** By default the payload carries only `contact { name?, pushName? }`. Setting `WEBHOOK_CONTACT_DETAILS=true` adds `id`, `number`, `shortName`, `type`, `isMyContact`, `isWAContact`, `isBusiness`, `isEnterprise`, `verifiedName`, `verifiedLevel`, `isBlocked` and `labels`, each present only when the engine has a value for it. Every field is copied from the contact record the engine has already loaded for the message, so opting in costs no extra WhatsApp lookup and cannot add rate-limiting pressure. The same enrichment reaches the websocket event, since both are dispatched from one payload. `contact` itself is omitted entirely when the engine has no value for any of its fields, so treat the whole object as optional rather than just its members. **whatsapp-web.js only** — the Baileys path emits `contact { pushName }` when a push name is available, and never reads this flag.-> **`edited` on `message.received` and `message.sent`.** When an edit reached a message before the gateway stored it, the message is announced with the edited `body` and `edited: true`, as its stored row carries `metadata.edited` (an edit event that repeats the text sets neither). Its `message.edited` can arrive before the message itself, so a client that holds no copy of it yet reads the mark from here. The field is absent otherwise.+> **`edited` on `message.received` and `message.sent`.** When an edit reached a message before the gateway stored it, the message is announced with the edited `body` and `edited: true`, as its stored row carries `metadata.edited` (an edit event that repeats the text sets neither). Its `message.edited` can arrive before the message itself, so a client that holds no copy of it yet reads the mark from here. An own message the engine delivers again after an edit marked its stored row is announced in `message.sent` with `edited: true` too, after the `message.edited` of an edit that reached the gateway first (it remembers up to 500 marked messages per session, the oldest forgotten first, while the session has an engine). The field is absent otherwise.> **Large media is not inlined into webhook payloads.** A `media` blob whose decoded size exceeds `WEBHOOK_MEDIA_INLINE_MAX_BYTES` (default **1 MiB**; `0` = never inline) is replaced — before the payload is fanned out to your webhook — with the marker form `media: { mimetype, filename?, omitted: true, sizeBytes }`, the same shape the engine emits for capped inbound media. Media at or under the cap stays inline unchanged. Additionally, if the serialized body still exceeds `WEBHOOK_MAX_PAYLOAD_BYTES` (default **1 MiB**) after `webhook:before` hooks ran, any remaining inline media is shed the same way so the event is still delivered; only a payload that is over budget _without_ inline media is dropped (recorded in `GET /api/webhooks/delivery-failures`). Because shedding happens before enqueue, queued and failed BullMQ jobs in Redis never carry the blob either — failed-job retention is bounded by the queue's `removeOnComplete`/`removeOnFail` windows (1h/1000 completed, 24h/5000 failed). The two limits compound: `WEBHOOK_MAX_PAYLOAD_BYTES` is measured on the serialized JSON body after `WEBHOOK_MEDIA_INLINE_MAX_BYTES` has been applied, and base64 inflates media by a third, so at the defaults media above roughly **768 KiB** reaches your webhook as the omitted marker. To inline larger media, raise both together and keep the payload limit at least 4/3 of the inline limit plus room for the rest of the event (for example `5242880` inline with `7340032` payload). The **WebSocket** `message.received` / `message.sent` events apply only the inline cap, with the same marker: an over-cap blob is never broadcast to subscribed sockets (or, with the Redis adapter enabled, replicated across every replica's pub/sub link). On either sink, media the engine downloaded stays retrievable from [`GET /api/sessions/:sessionId/messages/:chatId/:messageId/media`](#get-apisessionssessionidmessageschatidmessageidmedia), which returns `404` when nothing was stored.diff --git a/src/engine/adapters/wwebjs-message-changes.spec.ts b/src/engine/adapters/wwebjs-message-changes.spec.tsindex d300397a3..b0a1d899b 100644--- a/src/engine/adapters/wwebjs-message-changes.spec.ts+++ b/src/engine/adapters/wwebjs-message-changes.spec.ts@@ -140,18 +140,26 @@ describe.each([// The bodies the message:received and message:sent hooks were given, and a gate that holds the first of them.let hooked: string[];let hookGate: Promise<void> | undefined;+ let engines: EngineRegistry;+ let engine: IWhatsAppEngine;+ let findOne: jest.Mock;+ // Holds a row write the test names until the promise it returns settles.+ let holdWrite: (patch: Row) => Promise<void> | undefined;+ let projector: MessageProjector;beforeEach(() => {process.env.INBOUND_MEDIA_CONCURRENCY = '1';rows = [];hooked = [];hookGate = undefined;+ holdWrite = () => undefined;+ findOne = jest.fn(({ where }: { where: Row }) => Promise.resolve(rows.find(r => matches(r, where)) ?? null));// The real projector over a stand-in `messages` table, wired as the database wires it: only PostgreSQL// admits a message on arrival.const repo = {manager: { connection: { options: { type } } },find: jest.fn().mockResolvedValue([]),- findOne: jest.fn(({ where }: { where: Row }) => Promise.resolve(rows.find(r => matches(r, where)) ?? null)),+ findOne,create: jest.fn((x: Row) => ({ ...x })),insert: jest.fn((row: Row) => {// As the real table's unique index refuses a second row under an id.@@ -164,16 +172,20 @@ describe.each([return Promise.resolve({ identifiers: [{ id: `pk-${String(row.waMessageId)}` }], generatedMaps: [] });}),update: jest.fn((where: Row, patch: Row) => {- const hit = rows.filter(r => matches(r, where));- for (const r of hit) Object.assign(r, patch);- return Promise.resolve({ affected: hit.length });+ const write = () => {+ const hit = rows.filter(r => matches(r, where));+ for (const r of hit) Object.assign(r, patch);+ return { affected: hit.length };+ };+ const held = holdWrite(patch);+ return held ? held.then(write) : Promise.resolve(write());}),};dispatch = jest.fn().mockResolvedValue(undefined);- const engines = new EngineRegistry();- const engine = {} as IWhatsAppEngine;+ engines = new EngineRegistry();+ engine = {} as IWhatsAppEngine;engines.set('s1', engine);- const projector = new MessageProjector(+ projector = new MessageProjector(repo as unknown as Repository<Message>,{ update: jest.fn().mockResolvedValue({ affected: 1 }) } as unknown as Repository<Session>,engines,@@ -579,6 +591,156 @@ describe.each([expect(rows).toEqual([expect.objectContaining({ waMessageId: 'A', body: 'v1', metadata: { edited: true } })]);expect(events('message.received')).toEqual([expect.objectContaining({ body: 'v1', edited: true })]);});++ // An own message delivered again loses its insert to the row stored first, which an edit may have marked since.+ const own = (id: string, body: string) => incoming(id, body, { fromMe: true });+ const sentMarks = (): unknown[] => events('message.sent').map(m => m.edited);++ it('announces an own message delivered again after its edit was stored with the mark', async () => {+ client.emit('message_create', own('S', 'orig'));+ await flush();+ editEvent(client, 'S', 'e0', { fromMe: true });+ await flush();+ expect(rows).toEqual([expect.objectContaining({ waMessageId: 'S', body: 'e0', metadata: { edited: true } })]);+ client.emit('message_create', own('S', 'e0'));+ await flush();++ expect(events('message.sent')).toEqual([+ expect.objectContaining({ body: 'orig' }),+ expect.objectContaining({ body: 'e0', edited: true }),+ ]);+ expect(events('message.sent')[0]).not.toHaveProperty('edited');+ });++ it('announces an own message delivered again while its edit is written after that edit, with the mark', async () => {+ client.emit('message_create', own('S', 'orig'));+ await flush();+ const write = deferred<void>();+ holdWrite = patch => (patch.body === 'e0' ? write.promise : undefined);+ editEvent(client, 'S', 'e0', { fromMe: true });+ client.emit('message_create', own('S', 'e0'));+ await flush();+ write.resolve();+ await flush();++ const announced = (dispatch.mock.calls as Array<[string, string, Row]>)+ .filter(c => c[1] === 'message.sent' || c[1] === 'message.edited')+ .map(c => `${c[1]} ${String(c[2].body)}${c[2].edited ? ' edited' : ''}`);+ expect(announced).toEqual(['message.sent orig', 'message.edited e0', 'message.sent e0 edited']);+ });++ // A send made through the API stores its row before its echo comes, so the echo loses its insert.+ it('announces the echo of a send the API stored unmarked, reading no row for it', async () => {+ rows.push({ sessionId: 's1', waMessageId: 'S', chatId: CHAT, direction: 'outgoing', type: 'text', body: 'sent' });+ client.emit('message_create', own('S', 'sent'));+ await flush();++ expect(events('message.sent')).toEqual([expect.objectContaining({ body: 'sent' })]);+ expect(events('message.sent')[0]).not.toHaveProperty('edited');+ expect(findOne).not.toHaveBeenCalled();+ });++ it('announces an own send edited during its media download with the mark each time it is delivered', async () => {+ const media = download();+ client.emit('message_create', incoming('S', 'sent', { page: media.page, fromMe: true }));+ await flush();+ editEvent(client, 'S', 'sent again', { media: true, fromMe: true });+ await flush();+ media.finish();+ await flush();+ const page = jest.fn().mockResolvedValue({ mimetype: 'image/png', data: 'QUJD' });+ client.emit('message_create', incoming('S', 'sent again', { page, fromMe: true }));+ await flush();++ expect(events('message.sent').map(m => [m.body, m.edited])).toEqual([+ ['sent again', true],+ ['sent again', true],+ ]);+ });++ it('announces an own message delivered again after its edit and its delete without the mark', async () => {+ client.emit('message_create', own('S', 'orig'));+ await flush();+ editEvent(client, 'S', 'e0', { fromMe: true });+ await flush();+ revokeEvent(client, 'S', { fromMe: true });+ await flush();+ expect(rows).toEqual([expect.objectContaining({ waMessageId: 'S', type: 'revoked', metadata: null })]);+ client.emit('message_create', { ...own('S', ''), type: 'revoked' });+ await flush();++ expect(sentMarks()).toEqual([undefined, undefined]);+ expect(events('message.sent')[1]).not.toHaveProperty('edited');+ });++ it('announces an own message delivered again after its delete and an API edit of it without the mark', async () => {+ client.emit('message_create', own('S', 'orig'));+ await flush();+ revokeEvent(client, 'S', { fromMe: true });+ await flush();+ await projector.recordOutboundMessageEdit('s1', 'S', 'e0', { kind: 'edit', chatId: CHAT, fromMe: true });+ client.emit('message_create', { ...own('S', ''), type: 'revoked' });+ await flush();++ expect(rows).toEqual([expect.objectContaining({ waMessageId: 'S', type: 'revoked', metadata: null })]);+ expect(sentMarks()).toEqual([undefined, undefined]);+ });++ it('announces an own message stored again after its row was removed without the mark of that row', async () => {+ client.emit('message_create', own('S', 'orig'));+ await flush();+ editEvent(client, 'S', 'e0', { fromMe: true });+ await flush();+ // The chat was cleared: its rows are removed, and the message can be delivered again.+ rows.length = 0;+ client.emit('message_create', own('S', 'e0'));+ await flush();+ client.emit('message_create', own('S', 'e0'));+ await flush();++ expect(sentMarks()).toEqual([undefined, undefined, undefined]);+ });++ it('forgets the marks of a session once it has no engine', async () => {+ client.emit('message_create', own('S', 'orig'));+ await flush();+ editEvent(client, 'S', 'e0', { fromMe: true });+ await flush();+ engines.delete('s1');+ engines.set('s1', engine);+ client.emit('message_create', own('S', 'e0'));+ await flush();++ expect(sentMarks()).toEqual([undefined, undefined]);+ });++ it('keeps the marks of a session whose engine was replaced', async () => {+ client.emit('message_create', own('S', 'orig'));+ await flush();+ editEvent(client, 'S', 'e0', { fromMe: true });+ await flush();+ engines.set('s1', {} as IWhatsAppEngine);+ engines.set('s1', engine);+ client.emit('message_create', own('S', 'e0'));+ await flush();++ expect(sentMarks()).toEqual([undefined, true]);+ });++ it('keeps the marks of 500 messages of a session, forgetting the one marked first', async () => {+ const ids = Array.from({ length: 501 }, (_, i) => `S${i}`);+ for (const id of ids) {+ rows.push({ sessionId: 's1', waMessageId: id, chatId: CHAT, direction: 'outgoing', type: 'text', body: 'orig' });+ editEvent(client, id, 'e0', { fromMe: true });+ }+ await flush();+ client.emit('message_create', own('S0', 'e0'));+ client.emit('message_create', own('S1', 'e0'));+ client.emit('message_create', own('S500', 'e0'));+ await flush();++ expect(sentMarks()).toEqual([undefined, true, true]);+ });});describe('changes kept while whatsapp-web.js processes a message (adapter level)', () => {diff --git a/src/modules/session/message-mutation-projector.ts b/src/modules/session/message-mutation-projector.tsindex 5fba72ad9..35be8747d 100644--- a/src/modules/session/message-mutation-projector.ts+++ b/src/modules/session/message-mutation-projector.ts@@ -23,6 +23,8 @@ export class MessageMutationProjector {private readonly webhookService: WebhookService,private readonly messageMutations: KeyedMutationQueue,private readonly logger: LoggerService,+ // Told of each row the marker below lands on.+ private readonly onMarkedEdited: (sessionId: string, waMessageId: string) => void = () => undefined,) {}/**@@ -165,10 +167,11 @@ export class MessageMutationProjector {/** Keep the marker alongside existing reactions, media and quote metadata. */async markMessageEdited(sessionId: string, waMessageId: string): Promise<void> {- await updateMessageMetadata(this.messageRepository, { sessionId, waMessageId }, metadata => ({+ const marked = await updateMessageMetadata(this.messageRepository, { sessionId, waMessageId }, metadata => ({...metadata,edited: true,}));+ if (marked) this.onMarkedEdited(sessionId, waMessageId);}/**diff --git a/src/modules/session/message-projector.service.ts b/src/modules/session/message-projector.service.tsindex dc965bcc3..3df1ddea0 100644--- a/src/modules/session/message-projector.service.ts+++ b/src/modules/session/message-projector.service.ts@@ -72,6 +72,9 @@ export const ACK_RECONCILE_DELAY_MS = 750;*/export const PERSIST_RETRY_DELAY_MS = 300;+/** How many of its messages marked edited a session remembers as marked (see markedEdited). */+const EDITED_MARKS_KEPT_PER_SESSION = 500;+/*** How a message insert ended: `yes` the row landed, `dup` a unique violation (`retried` says whether* it came from the retry, when the first attempt may have committed before its error), `failed` a@@ -237,6 +240,11 @@ export class MessageProjector {// On PostgreSQL the entry exists from the message's arrival (admitInbound), before its emit.private readonly inboundInFlight = new Map<string, InFlightEntry>();+ // Ids whose row was marked edited, by session, while it has an engine: an own message delivered again loses its+ // insert to that row, and is announced with the mark without reading it (see commitOwnSendEcho). Never stored; past+ // EDITED_MARKS_KEPT_PER_SESSION the oldest is forgotten, and a repeat of it goes out unmarked.+ private readonly markedEdited = new Map<string, Set<string>>();+// Reaction/edit applies, extracted to a plain collaborator. It shares this instance's// messageMutations queue, so the public enqueue path and the queued applies serialize on one chain.private readonly mutationProjector: MessageMutationProjector;@@ -268,13 +276,22 @@ export class MessageProjector {this.webhookService,this.messageMutations,this.logger,+ (sessionId, waMessageId) => this.noteMarkedEdited(sessionId, waMessageId),);this.engines.onRetire((id, engine) => {for (const t of this.tickets.get(id) ?? []) if (t.engine === engine) t.drop();this.deleteRoomIfIdle(id);+ if (!this.engines.has(id)) this.markedEdited.delete(id);});}+ private noteMarkedEdited(sessionId: string, waMessageId: string): void {+ let marks = this.markedEdited.get(sessionId);+ if (!marks) this.markedEdited.set(sessionId, (marks = new Set()));+ marks.add(waMessageId);+ if (marks.size > EDITED_MARKS_KEPT_PER_SESSION) marks.delete(marks.values().next().value!);+ }+/*** Whether the engines hold each inbound message from its arrival until its row is written. Only* PostgreSQL needs it: a downloaded payload waits in memory for its asynchronous insert, while@@ -971,6 +988,7 @@ export class MessageProjector {return;}const result = await this.messageRepository.update({ sessionId, waMessageId }, REVOKED_ROW_PATCH);+ this.markedEdited.get(sessionId)?.delete(waMessageId);if (!result?.affected || !notify) return;const row = await this.messageRepository.findOne({ where: { sessionId, waMessageId } });if (!row) return;@@ -1150,6 +1168,7 @@ export class MessageProjector {resolveFeatureFlags(this.configService).storeEphemeralMessages ||!(outgoing.ephemeralDuration && outgoing.ephemeralDuration > 0);+ let rowMarked = false;if (mayPersist) {const dbMessage = this.messageRepository.create({sessionId: id,@@ -1175,8 +1194,19 @@ export class MessageProjector {const outcome = await this.insertWithRetry(id, engine, dbMessage, 'outgoing');if (outcome.landed === 'stale') return;// The REST writer may have won either attempt; its row still takes the echo's pending changes.- if (outcome.landed === 'dup') this.applyChangesMadeInFlight(id, outgoing.id, false, claimOf(outgoing));+ if (outcome.landed === 'dup') {+ this.applyChangesMadeInFlight(id, outgoing.id, false, claimOf(outgoing));+ // That row may be older than this delivery and edited since, the edit still being written: announced once the+ // changes queued for the message have run, with the mark they left.+ rowMarked = await new Promise<boolean>(resolve =>+ this.enqueueMessageMutation(id, outgoing.id, () =>+ Promise.resolve(resolve(this.markedEdited.get(id)?.has(outgoing.id) === true)),+ ),+ );+ }if (outcome.landed === 'yes') {+ // The id had no row, so a mark remembered for it was of a row removed since.+ this.markedEdited.get(id)?.delete(outgoing.id);this.applyChangesMadeInFlight(id, outgoing.id, insertedCleared);// Fire-and-forget, mirroring onMessage: plugin providers (search etc.) see phone-// composed sends exactly like API sends.@@ -1201,9 +1231,10 @@ export class MessageProjector {const announced = this.withChangesMadeInFlight(id, finalMessage);if (!announced) return;- void this.webhookService.dispatch(id, 'message.sent', announced);+ const sent = rowMarked ? { ...announced, edited: true as const } : announced;+ void this.webhookService.dispatch(id, 'message.sent', sent);// Emit real-time event to WebSocket clients (as message.sent, not message.received)- this.eventsGateway.emitMessageSent(id, announced);+ this.eventsGateway.emitMessageSent(id, sent);}/** Engine callback body, lifted out of initializeEngine so the wiring table stays readable. */