Free, Open Source, Self-Hosted WhatsApp API Gateway
TypeScript apibotgatewayself-hostedwhatsappwhatsapp-apiwhatsapp-automationwhatsapp-botwhatsapp-gatewaywhatsapp-gateway-apiwhatsapp-webwhatsapp-web-api
fix(engine): mark a message edited and restored before it is stored
Yudhi Armyndharisfe454c522f5c
diff --git a/docs/06-api-specification.md b/docs/06-api-specification.mdindex 240a603cd..7f765eed8 100644--- a/docs/06-api-specification.md+++ b/docs/06-api-specification.md@@ -1383,7 +1383,7 @@ GET /api/sessions/:sessionId/messages?since=1789855200000&until=1789941600000&orEach `Message`: `{ id (uuid), sessionId, waMessageId (string|null), chatId, from, to, chatName (string|null; the sender's push name, or their saved contact name when no push name was reported), author (string|null; the real sender of a group, status or broadcast-list message, where from holds the group, status@broadcast or list id; on Baileys a list message the account received is filed under the sender, so from is the sender too), body (string|null), type, direction ('incoming'|'outgoing'), timestamp (number|null), metadata (object|null), mediaPath (string|null; storage key of archived media), mediaMimetype (string|null), status ('pending'|'sent'|'delivered'|'read'|'failed'), createdAt (ISO date) }`. By default, ordered by `createdAt` DESC, then by a dialect-dependent second key. The tiebreaker matters: `createdAt` is not unique (SQLite stores whole seconds, a PostgreSQL bulk write ties every row it inserts, and a history backfill carries WhatsApp's own second-resolution timestamp), and without a total order two pages of one walk can repeat a row and omit another. On SQLite the second key is `rowid`, the stored insertion sequence, so messages sharing a second come back in the order they arrived. PostgreSQL has no equivalent (`ctid` moves on every ack update), so it keeps `id`, a random uuid: the walk is equally correct there, but a same-second group is not in arrival order. Note that `offset` still addresses a position by count, so a list taking concurrent writes can shift under a walk: a message arriving mid-walk pushes every older row down one, and the next page re-serves a row the previous one already returned. Pass `after` instead to walk a live chat safely; it anchors on the last row you received, which an arriving message cannot move. The response is the raw service object (no envelope). Unlike the live `IncomingMessage` shape below, this persisted `Message` does **not** carry `kind` — re-derive the chat kind from `chatId` (see `ChatKind` / `chatKind()`) if needed. When present, `metadata` may include `media`, `quotedMessage`, `call`, `reactions`, `poll { name, options, allowMultipleAnswers }` for a poll creation message, and, for Baileys inbound business prompts, `buttons: [{ id, text }, …]` so a client (including the dashboard Chats thread) can re-render the choices after reload and tap them via [`POST .../click-button`](#post-apisessionssessionidmessagesclick-button).-For messages whose edits the gateway observes, `metadata.edited` is `true`. An edit event that repeats the text the message already carries changes nothing and does not set it. Chats displays a localized edited indicator for these messages, including after reloading. Existing messages are not retroactively marked, and a deleted message keeps its deleted placeholder instead. The `message.edited` event payload is unchanged.+For messages whose edits the gateway observes, `metadata.edited` is `true`. An edit event that repeats the text the message already carries changes nothing and does not set it, while a message edited and then edited back to its earlier text is still marked. Chats displays a localized edited indicator for these messages, including after reloading. Existing messages are not retroactively marked, and a deleted message keeps its deleted placeholder instead. The `message.edited` event payload is unchanged.> **Inline media is carried up to a budget, then omitted.** `MESSAGE_LIST_INLINE_MEDIA_BUDGET_BYTES` (8 MiB of encoded base64 by default) bounds how much inline media one response may hold across its rows. A row is not a bounded object, `limit` is clamped to `[1,100]` but each row can carry its base64 in `metadata.media.data`, so a page of media rows could otherwise reach hundreds of megabytes and fail the read outright. The budget is spent newest-first, matching the selected descending order, so a page that cannot carry everything keeps the most recent media. Past it a payload is replaced with `{ mimetype, filename?, omitted: true, sizeBytes }`, the same marker the engine emits for inbound media over `MEDIA_DOWNLOAD_MAX_BYTES`, and the bytes remain available from [`GET /messages/:chatId/:messageId/media`](#get-apisessionssessionidmessageschatidmessageidmedia). Two rules bound the edges: the newest payload is always inlined even when it alone exceeds the budget (otherwise a single large photo would be permanently unreadable through this route), and a budget of `0` means "never inline" and grants no such allowance. The knob is validated at boot, `8MiB` would parse to 8 bytes, and is forwarded by both compose files. The MCP `MessageList` tool shares this path and the same budget. A row stored under `MESSAGE_INLINE_MEDIA=archive` arrives with the marker already in place, plus `archived: true`, its bytes live only in the chat-media archive, behind the same media route (see that route for the mode).@@ -7288,7 +7288,7 @@ These are the events OpenWA actually emits. A webhook is registered with an `eve> **`contact` on `message.received` is minimal unless you opt in (`WEBHOOK_CONTACT_DETAILS`).** By default the payload carries only `contact { name?, pushName? }`. Setting `WEBHOOK_CONTACT_DETAILS=true` adds `id`, `number`, `shortName`, `type`, `isMyContact`, `isWAContact`, `isBusiness`, `isEnterprise`, `verifiedName`, `verifiedLevel`, `isBlocked` and `labels`, each present only when the engine has a value for it. Every field is copied from the contact record the engine has already loaded for the message, so opting in costs no extra WhatsApp lookup and cannot add rate-limiting pressure. The same enrichment reaches the websocket event, since both are dispatched from one payload. `contact` itself is omitted entirely when the engine has no value for any of its fields, so treat the whole object as optional rather than just its members. **whatsapp-web.js only** — the Baileys path emits `contact { pushName }` when a push name is available, and never reads this flag.-> **`edited` on `message.received` and `message.sent`.** When an edit reached a message before the gateway stored it, the message is announced with the edited `body` and `edited: true`, as its stored row carries `metadata.edited` (an edit event that repeats the text sets neither). Its `message.edited` can arrive before the message itself, so a client that holds no copy of it yet reads the mark from here. An own message the engine delivers again after an edit marked its stored row is announced in `message.sent` with `edited: true` too, after the `message.edited` of an edit that reached the gateway first (it remembers up to 500 marked messages per session, the oldest forgotten first, while the session has an engine). The field is absent otherwise.+> **`edited` on `message.received` and `message.sent`.** When an edit reached a message before the gateway stored it, the message is announced with the edited `body` and `edited: true`, as its stored row carries `metadata.edited`. This holds when a later edit restored the text it arrived with too, and the `body` is then that text; edit events that each repeat the text set neither. Its `message.edited` can arrive before the message itself, so a client that holds no copy of it yet reads the mark from here. An own message the engine delivers again after an edit marked its stored row is announced in `message.sent` with `edited: true` too, after the `message.edited` of an edit that reached the gateway first (it remembers up to 500 marked messages per session, the oldest forgotten first, while the session has an engine). The field is absent otherwise.> **Large media is not inlined into webhook payloads.** A `media` blob whose decoded size exceeds `WEBHOOK_MEDIA_INLINE_MAX_BYTES` (default **1 MiB**; `0` = never inline) is replaced — before the payload is fanned out to your webhook — with the marker form `media: { mimetype, filename?, omitted: true, sizeBytes }`, the same shape the engine emits for capped inbound media. Media at or under the cap stays inline unchanged. Additionally, if the serialized body still exceeds `WEBHOOK_MAX_PAYLOAD_BYTES` (default **1 MiB**) after `webhook:before` hooks ran, any remaining inline media is shed the same way so the event is still delivered; only a payload that is over budget _without_ inline media is dropped (recorded in `GET /api/webhooks/delivery-failures`). Because shedding happens before enqueue, queued and failed BullMQ jobs in Redis never carry the blob either — failed-job retention is bounded by the queue's `removeOnComplete`/`removeOnFail` windows (1h/1000 completed, 24h/5000 failed). The two limits compound: `WEBHOOK_MAX_PAYLOAD_BYTES` is measured on the serialized JSON body after `WEBHOOK_MEDIA_INLINE_MAX_BYTES` has been applied, and base64 inflates media by a third, so at the defaults media above roughly **768 KiB** reaches your webhook as the omitted marker. To inline larger media, raise both together and keep the payload limit at least 4/3 of the inline limit plus room for the rest of the event (for example `5242880` inline with `7340032` payload). The **WebSocket** `message.received` / `message.sent` events apply only the inline cap, with the same marker: an over-cap blob is never broadcast to subscribed sockets (or, with the Redis adapter enabled, replicated across every replica's pub/sub link). On either sink, media the engine downloaded stays retrievable from [`GET /api/sessions/:sessionId/messages/:chatId/:messageId/media`](#get-apisessionssessionidmessageschatidmessageidmedia), which returns `404` when nothing was stored.diff --git a/src/engine/adapters/baileys-events.ts b/src/engine/adapters/baileys-events.tsindex 4a91c9583..de8592224 100644--- a/src/engine/adapters/baileys-events.ts+++ b/src/engine/adapters/baileys-events.ts@@ -336,10 +336,14 @@ export class BaileysEvents {* The latest edit of a message still being processed, by id, with the key it was sent under. The* edit is announced first and finds no row or preview to change, and a repeat delivery can be stored* after the edit was applied, so the original is stored and announced with this text instead, and- * a quote or forward meanwhile carries it too (see pendingEditOf). Dropped once the message's- * processing settles, and bounded like deletedForEveryone, which wins over it.+ * a quote or forward meanwhile carries it too (see pendingEditOf). `afterOtherText` says an earlier+ * edit had another text, so one of them changed the text even if this one restores it. Dropped once+ * the message's processing settles, and bounded like deletedForEveryone, which wins over it.*/- private readonly editedWhileInFlight = new Map<string, { envelope: WAMessageKey; body: string }>();+ private readonly editedWhileInFlight = new Map<+ string,+ { envelope: WAMessageKey; body: string; afterOtherText: boolean }+ >();/*** The deliveries of an id handed to onMessage that have no outcome yet, by id: how many, and a promise@@ -773,8 +777,10 @@ export class BaileysEvents {this.host.recordMessageEdit(chatJid, edited.messageId, edited.body);const target = this.inboundInFlight.get(edited.messageId);if (target && this.mayChange(target.key, msg.key, false)) {+ const earlier = this.editedWhileInFlight.get(edited.messageId);+ const afterOtherText = !!earlier && (earlier.afterOtherText || earlier.body !== edited.body);this.editedWhileInFlight.delete(edited.messageId); // re-inserted as the newest- this.editedWhileInFlight.set(edited.messageId, { envelope: msg.key, body: edited.body });+ this.editedWhileInFlight.set(edited.messageId, { envelope: msg.key, body: edited.body, afterOtherText });if (this.editedWhileInFlight.size > BaileysEvents.DELETED_FOR_EVERYONE_LIMIT) {const [oldest] = this.editedWhileInFlight.keys();this.editedWhileInFlight.delete(oldest);@@ -888,8 +894,10 @@ export class BaileysEvents {const content = b.normalizeMessageContent(toStore.message ?? undefined);if (content) setBaileysText(content, editedBody);// The session is told of the edit too, with a ticket at once and without one right after the emit, and- // compares it with the text the message arrived with.- if (editedBody !== incoming.body) incoming.originalBody = incoming.body;+ // compares it with the text the message arrived with. It is told even when the latest edit restored that+ // text after another one: the message was edited all the same, and edits announced before the emit, past+ // the changes that wait, left no mark.+ if (editedBody !== incoming.body || edit?.afterOtherText) incoming.originalBody = incoming.body;incoming.body = editedBody;}if (opts.generation === this.storeGeneration) {diff --git a/src/engine/adapters/inbound-hold.spec.ts b/src/engine/adapters/inbound-hold.spec.tsindex ca331785c..5a174b153 100644--- a/src/engine/adapters/inbound-hold.spec.ts+++ b/src/engine/adapters/inbound-hold.spec.ts@@ -1265,6 +1265,36 @@ describe('Baileys inbound messages held for storage', () => {expect(handedOver).toEqual([{ body, originalBody }]);});+ it.each([+ ['edits that restore it', ['fixed caption', 'caption'], 'caption'],+ ['edits that restore it and then repeat it', ['fixed caption', 'caption', 'caption'], 'caption'],+ ['edits that each repeat it', ['caption', 'caption'], undefined],+ ])(+ 'hands the message over with the text it arrived with after %s only if one changed it',+ async (_, bodies, originalBody) => {+ const h = heldProjector({ unique: true, sqlite });+ const handedOver: Array<Pick<IncomingMessage, 'body' | 'originalBody'>> = [];+ const { events } = build(4, {+ ...h.callbacks,+ onMessage: (m, t) => (+ handedOver.push({ body: m.body, originalBody: m.originalBody }),+ h.callbacks.onMessage!(m, t)+ ),+ });+ const release = download();+ upsert(events, media('M', CHAT, { caption: 'caption' }));+ await flush();+ for (const [i, body] of bodies.entries()) {+ upsert(events, edit(`E${i}`, 'M', body));+ await flush();+ }+ release();+ await flush();++ expect(handedOver).toEqual([{ body: 'caption', originalBody }]);+ },+ );+it.each([['an edit', 'fixed caption', true],['an edit that repeats the text', 'caption', undefined],@@ -1637,6 +1667,94 @@ describe('Baileys inbound messages held for storage', () => {]);});+ // Its edits went out as message.edited, so it was edited, though the last one restored the text it arrived with.+ it.each([+ ['within the 64 changes that wait', false],+ ['past the 64 changes that wait', true],+ ])('marks a message edited and then restored to its text while it downloads, %s', async (_, pastBound) => {+ const h = heldProjector({ unique: true, sqlite: true });+ const { events } = build(4, h.callbacks);+ const releaseOther = download();+ const release = download();+ upsert(events, media('O', CHAT), media('M', CHAT, { caption: 'caption' }));+ await flush();+ if (pastBound) upsert(events, ...Array.from({ length: 64 }, (_x, i) => reaction(`R${i}`, 'O', `r${i}`)));+ await flush();+ upsert(events, edit('E1', 'M', 'fixed caption'));+ await flush();+ upsert(events, edit('E2', 'M', 'caption'));+ await flush();+ expect(h.announcedAs('message.edited')).toHaveLength(pastBound ? 2 : 0);+ releaseOther();+ release();+ await flush();++ const row = h.rows.find(r => r.waMessageId === 'M');+ expect(row).toEqual(expect.objectContaining({ body: 'caption' }));+ expect((row?.metadata as Row).edited).toBe(true);+ expect(h.announcedAs('message.received').filter(m => m.id === 'M')).toEqual([+ expect.objectContaining({ body: 'caption', edited: true }),+ ]);+ });++ it.each([+ ['within three media deadlines', false],+ ['three media deadlines after they began to wait', true],+ ])('marks a message edited and then restored to its text, its edits announced %s', async (_, late) => {+ process.env.MEDIA_DOWNLOAD_TIMEOUT_MS = '30';+ const h = heldProjector({ unique: true, sqlite: true });+ const store = memoryStore();+ const read = store.getStoredMessage;+ const answer = deferred<void>();+ let checked = false;+ store.getStoredMessage = (id: string) => {+ if (id !== 'M' || checked) return read(id);+ checked = true;+ return answer.promise.then(() => read(id));+ };+ const { events } = build(4, h.callbacks, store);+ upsert(events, text('M', CHAT));+ await flush();+ upsert(events, edit('E1', 'M', 'fixed'));+ await flush();+ upsert(events, edit('E2', 'M', 'text M'));+ await flush();+ if (late) await sleep(120);+ expect(h.announcedAs('message.edited')).toHaveLength(late ? 2 : 0);+ answer.resolve();+ await flush();++ expect(h.rows).toEqual([expect.objectContaining({ waMessageId: 'M', body: 'text M' })]);+ expect((h.rows[0].metadata as Row).edited).toBe(true);+ expect(h.announcedAs('message.received')).toEqual([+ expect.objectContaining({ id: 'M', body: 'text M', edited: true }),+ ]);+ });++ it('leaves a message unmarked whose edits announced at once each repeated its text', async () => {+ const h = heldProjector({ unique: true, sqlite: true });+ const { events } = build(4, h.callbacks);+ const releaseOther = download();+ const release = download();+ upsert(events, media('O', CHAT), media('M', CHAT, { caption: 'caption' }));+ await flush();+ upsert(events, ...Array.from({ length: 64 }, (_x, i) => reaction(`R${i}`, 'O', `r${i}`)));+ await flush();+ upsert(events, edit('E1', 'M', 'caption'));+ await flush();+ upsert(events, edit('E2', 'M', 'caption'));+ await flush();+ expect(h.announcedAs('message.edited')).toHaveLength(2);+ releaseOther();+ release();+ await flush();++ const row = h.rows.find(r => r.waMessageId === 'M');+ expect(row).toEqual(expect.objectContaining({ body: 'caption' }));+ expect(row?.metadata as Row).not.toHaveProperty('edited');+ expect(h.announcedAs('message.received').filter(m => m.id === 'M')[0]).not.toHaveProperty('edited');+ });+/** How many changes wait, and for how many messages. */const internals = (events: BaileysEvents) => {const state = events as unknown as { changesAwaiting: number; changesAwaitingEmit: Map<string, unknown> };diff --git a/src/engine/adapters/wwebjs-message-changes.spec.ts b/src/engine/adapters/wwebjs-message-changes.spec.tsindex 290aad89c..ec446186b 100644--- a/src/engine/adapters/wwebjs-message-changes.spec.ts+++ b/src/engine/adapters/wwebjs-message-changes.spec.ts@@ -571,6 +571,25 @@ describe.each([expect(events('message.received')).toEqual([expect.objectContaining({ body: 'v65', edited: true })]);});+ // Its edits went out as message.edited, so it was edited, though the last one restored the text it arrived with.+ it.each([+ ['within the changes kept for it', 1],+ ['past the changes kept for it', 64],+ ])('marks a message edited and then restored to its text during its contact lookup, %s', async (_, edits) => {+ const contact = deferred<null>();+ client.emit('message', incoming('A', 'orig', { contact: contact.promise }));+ await flush();+ for (let i = 1; i <= edits; i++) editEvent(client, 'A', `v${i}`);+ editEvent(client, 'A', 'orig');+ await flush();+ contact.resolve(null);+ await flush();++ expect(rows).toEqual([expect.objectContaining({ waMessageId: 'A', body: 'orig', metadata: { edited: true } })]);+ expect(events('message.received')).toEqual([expect.objectContaining({ body: 'orig', edited: true })]);+ expect(events('message.edited')).toHaveLength(edits + 1);+ });+// whatsapp-web.js builds a repeat delivery from the message as it is now, so one raised after an edit carries the// edited text, and its lookups can end before those of the first delivery.it.each([@@ -823,22 +842,41 @@ describe('changes kept while whatsapp-web.js processes a message (adapter level)const { client, arrivals } = wire();const plain = deferred<null>();const reacted = deferred<null>();- const restored = deferred<null>();+ const repeated = deferred<null>();client.emit('message', incoming('A', 'orig', { contact: plain.promise }));client.emit('message', incoming('B', 'orig', { contact: reacted.promise }));- client.emit('message', incoming('C', 'orig', { contact: restored.promise }));+ client.emit('message', incoming('C', 'orig', { contact: repeated.promise }));reactionEvent(client, 'B', '👍');- editEvent(client, 'C', 'v1');+ editEvent(client, 'C', 'orig');editEvent(client, 'C', 'orig');plain.resolve(null);reacted.resolve(null);- restored.resolve(null);+ repeated.resolve(null);await flush();expect(arrivals).toEqual([undefined, undefined, undefined]);});+ it('hands over the text a message arrived with when its edits restored it, past the changes kept too', async () => {+ const { client, bodies, arrivals } = wire();+ const kept = deferred<null>();+ const past = deferred<null>();+ client.emit('message', incoming('A', 'orig', { contact: kept.promise }));+ client.emit('message', incoming('B', 'orig', { contact: past.promise }));+ editEvent(client, 'A', 'v1');+ editEvent(client, 'A', 'orig');+ for (let i = 1; i <= PER_MESSAGE; i++) editEvent(client, 'B', `v${i}`);+ editEvent(client, 'B', 'orig');++ kept.resolve(null);+ past.resolve(null);+ await flush();++ expect(bodies).toEqual(['orig', 'orig']);+ expect(arrivals).toEqual(['orig', 'orig']);+ });+it('hands a repeat delivery that carries an edit over with the text the first delivery arrived with', async () => {const { client, bodies, arrivals } = wire();const first = deferred<null>();diff --git a/src/engine/adapters/wwebjs-message-events.ts b/src/engine/adapters/wwebjs-message-events.tsindex fe2da65f9..9e28f1def 100644--- a/src/engine/adapters/wwebjs-message-events.ts+++ b/src/engine/adapters/wwebjs-message-events.ts@@ -23,6 +23,8 @@ interface MessageInProcessing {text: string;/** The text the first delivery arrived with: a repeat is built from the message as it is now, edited. */arrived: string;+ /** An edit changed its text, though a later one may have restored the text it arrived with. */+ edited: boolean;/** A delete for everyone reached it. */deleted: boolean;changes: Array<() => void>;@@ -111,15 +113,17 @@ export function registerWwebjsMessageEvents(client: Client, host: WwebjsEngineHoif (!kept) {// Past the bound a message is not tracked, and what reaches it is delivered at once, as it always was.if (inProcessing.size >= MAX_MESSAGES_IN_PROCESSING) return NOT_KEPT;- kept = { text: message.body, arrived: message.body, deleted: false, changes: [] };+ kept = { text: message.body, arrived: message.body, edited: false, deleted: false, changes: [] };inProcessing.set(message.id, kept);}const mine = kept;return {handOver: handedOver => {if (mine.deleted) return false;- // The session is told of the edits too, and compares them with the text the message arrived with.- if (mine.arrived !== mine.text) handedOver.originalBody = mine.arrived;+ // The session is told of the edits too, and compares them with the text the message arrived with. It is told+ // even when a later edit restored that text: the message was edited all the same, and the edits delivered+ // before it, past the bound, left no mark.+ if (mine.edited) handedOver.originalBody = mine.arrived;handedOver.body = mine.text;return true;},@@ -141,7 +145,10 @@ export function registerWwebjsMessageEvents(client: Client, host: WwebjsEngineHodeliver();return;}- if (change.editedText !== undefined) kept.text = change.editedText;+ if (change.editedText !== undefined) {+ kept.text = change.editedText;+ kept.edited = true;+ }if (change.deletes) kept.deleted = true;if (kept.changes.length < MAX_CHANGES_KEPT_PER_MESSAGE) {kept.changes.push(deliver);diff --git a/src/engine/interfaces/whatsapp-engine.interface.ts b/src/engine/interfaces/whatsapp-engine.interface.tsindex 7c2753e45..8f866c6c4 100644--- a/src/engine/interfaces/whatsapp-engine.interface.ts+++ b/src/engine/interfaces/whatsapp-engine.interface.ts@@ -106,7 +106,9 @@ export interface IncomingMessage {/*** Set by an engine that applied an edit to `body` before it handed the message over: the text the message* arrived with. The edits that reached the message meanwhile reach the session too, and are compared with- * this text. The session removes the field before the message is hooked, stored or announced.+ * this text. It equals `body` when a later edit restored that text, as the message was edited all the same,+ * and is unset when every edit repeated the text. The session removes the field before the message is+ * hooked, stored or announced.*/originalBody?: string;/** User-facing chat kind of the conversation this message belongs to (derived from `chatId`). */diff --git a/src/modules/session/message-projector.service.ts b/src/modules/session/message-projector.service.tsindex 3df1ddea0..d7be5e3d2 100644--- a/src/modules/session/message-projector.service.ts+++ b/src/modules/session/message-projector.service.ts@@ -529,7 +529,8 @@ export class MessageProjector {/*** The engine handed the message over: an edit held until now is compared with the text it arrived with. One the- * engine applied itself changed that text (it names the arrival text only then), and marks the row the same way.+ * engine applied itself changed that text, though a later one may have restored it (it names the arrival text only+ * then, so it can equal the body), and marks the row the same way.*/private arrive(entry: InFlightEntry, message: IncomingMessage): void {entry.body = message.originalBody ?? message.body;