Skip to content

Free, Open Source, Self-Hosted WhatsApp API Gateway

fix(session): bound inbound media held for storage on PostgreSQL (#1854)

Yudhi Armyndharisd64721add4e3
diff --git a/.env.example b/.env.example
index d34686e85..1be46afbc 100644
--- a/.env.example
+++ b/.env.example
@@ -566,9 +566,13 @@ WEBHOOK_SSRF_PROTECT=true
# MEDIA_DOWNLOAD_MAX_BYTES, so a sender that understates it can make one download exceed that cap. On
# PostgreSQL each downloaded payload (received media, and the echo of every media send on
# whatsapp-web.js) then waits in memory as base64 (about 4/3 of its size, on both engines) until its row
-# is stored, and how many wait is not bounded: a burst faster than the database writes adds messages
-# waiting x 4/3 x size. Lowering this does not bound that; MEDIA_DOWNLOAD_MAX_BYTES caps each payload,
-# MEDIA_DOWNLOAD_ENABLED=false holds none.
+# is stored. N bounds those too: a session downloads media only into one of N slots, each held until the
+# row is stored, so it holds at most N x 4 x ceil(MEDIA_DOWNLOAD_MAX_BYTES / 3) bytes of payload
+# (279620272, about 267 MiB, at the defaults) however slow the inserts. Each payload takes a whole slot
+# whatever its size, and a chat's messages are stored in arrival order, so a text waits for the media
+# that arrived before it in its chat. Media that finds no free slot while none is freed for
+# 2 x MEDIA_DOWNLOAD_TIMEOUT_MS arrives with the omitted marker. On SQLite rows are written at once and
+# nothing waits, except behind a slow message:received or message:sent plugin hook, which is not bounded.
# INBOUND_MEDIA_CONCURRENCY=4 # default 4
# Process-wide ceiling on inbound media downloads across all sessions, on top of the per-session one.
# 0 (default) is off. When on, media that waits longer than MEDIA_DOWNLOAD_TIMEOUT_MS for a slot is
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 0e44aac92..6a011009c 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -23,6 +23,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Changed
+- On PostgreSQL, inbound media that waits `2 x MEDIA_DOWNLOAD_TIMEOUT_MS` without a free slot arrives with the omitted marker instead of being downloaded.
+- On PostgreSQL, a chat's messages are stored in arrival order, so a text waits for earlier media in its chat.
+- On PostgreSQL with whatsapp-web.js, a held message's contact and quoted-message lookups wait at most `MEDIA_DOWNLOAD_TIMEOUT_MS`.
- The JavaScript SDK release job runs `npm audit` over the SDK tree before it installs, tests and publishes.
- The JavaScript SDK CommonJS build no longer sets `moduleResolution` node10, which TypeScript 7 removed; the built output is unchanged.
- JavaScript SDK CI and release jobs install with `--engine-strict` wherever they run the tests, so an unsupported Node fails at install.
@@ -33,6 +36,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
+- Bound the inbound media a PostgreSQL session holds before storing it to `INBOUND_MEDIA_CONCURRENCY` payloads, keeping each chat's arrival order.
+- Drop a Baileys edit, revoke or reaction from another chat, or an edit or revoke from the wrong author, aimed at a message still being downloaded, on SQLite too.
- Warn at storage export when a file is larger than the per-entry import cap `STORAGE_IMPORT_MAX_BYTES`.
- Refuse a `PORT` with surrounding whitespace at boot instead of binding a port the image healthcheck cannot probe.
- Skip or cancel the Baileys version lookup of a session stopped while it connects, so no proxy dispatcher outlives it.
diff --git a/docs/11-operational-runbooks.md b/docs/11-operational-runbooks.md
index 386d54f8e..da3b4028c 100644
--- a/docs/11-operational-runbooks.md
+++ b/docs/11-operational-runbooks.md
@@ -223,11 +223,26 @@ curl -H "X-API-Key: $API_KEY" \
# C. If memory climbs during a media burst (received, or sent on whatsapp-web.js):
# on PostgreSQL each payload, received or the echo of a media send, waits in memory as base64
-# (about 4/3 of the file size) until its row is stored, with no limit on how many wait; a slow
-# message:received or message:sent plugin hook holds them the same way on either database.
-# Lower MEDIA_DOWNLOAD_MAX_BYTES to cap each one (it also caps outbound media sends), or set
-# MEDIA_DOWNLOAD_ENABLED=false to hold none. Lowering INBOUND_MEDIA_CONCURRENCY does not bound
-# it. Archiving (CHAT_MEDIA_ARCHIVE_ENABLED=true) adds its own cost on either database: each
+# (about 4/3 of the file size) until its row is stored, in one of INBOUND_MEDIA_CONCURRENCY
+# slots per session, so a session holds at most INBOUND_MEDIA_CONCURRENCY x 4 x
+# ceil(MEDIA_DOWNLOAD_MAX_BYTES / 3) bytes (about 267 MiB at the defaults) even while inserts
+# stall. Lower either setting to shrink that bound (MEDIA_DOWNLOAD_MAX_BYTES also caps outbound
+# media sends), or set MEDIA_DOWNLOAD_ENABLED=false to hold none. Media logged as "No room for
+# inbound media" arrived with the omitted marker because no slot was freed for 2 x
+# MEDIA_DOWNLOAD_TIMEOUT_MS. A slot is held until its message's commit settles: the
+# message:received or message:sent hook chain, with RESOLVE_LID_TO_PHONE=true the sender's phone
+# lookup, the insert, and on whatsapp-web.js a timed-out page download (until
+# PUPPETEER_PROTOCOL_TIMEOUT_MS fails it). To find the holder, look for long-running inserts
+# (SELECT pid, now() - query_start, state, query FROM pg_stat_activity WHERE query ILIKE
+# 'INSERT INTO "messages"%') and for "Sandboxed plugin ... timed out after 5000ms" warnings: a
+# plugin loaded from the plugins directory delays a slot by at most 5 s per hook, so such a stall
+# clears on its own. A hook registered in-process (a plugin registered programmatically) has no
+# time limit: one that never settles keeps its chat's later messages from being stored and holds
+# every slot they or its own message took, for the life of the process. A reconnect does not free
+# them, so once INBOUND_MEDIA_CONCURRENCY slots are stuck the session sheds all media; fix or
+# remove that hook and restart the process. On SQLite a slow message:received or message:sent
+# plugin hook still holds payloads with no limit on how many.
+# Archiving (CHAT_MEDIA_ARCHIVE_ENABLED=true) adds its own cost on either database: each
# message being archived holds its payload as base64 and decoded until the upload finishes (more
# under MESSAGE_INLINE_MEDIA=archive, which also reads the file back and rewrites the row), and
# nothing limits how many archive at once. Lowering CHAT_MEDIA_ARCHIVE_MAX_BYTES (25 MiB by
diff --git a/docs/12-troubleshooting-faq.md b/docs/12-troubleshooting-faq.md
index 243923c76..0d9c424ed 100644
--- a/docs/12-troubleshooting-faq.md
+++ b/docs/12-troubleshooting-faq.md
@@ -1246,12 +1246,12 @@ docker exec openwa-api curl http://host.docker.internal:8080
>
> A media burst adds roughly `INBOUND_MEDIA_CONCURRENCY` x `MEDIA_DOWNLOAD_MAX_BYTES` per session on top of that (4 x 50 MiB by default; about a third more on whatsapp-web.js, which holds each payload as base64). On whatsapp-web.js Chromium also holds each file being downloaded, encrypted and decrypted, and a download starts whenever the size the sender declares is within `MEDIA_DOWNLOAD_MAX_BYTES`, so a sender that understates it can make one download exceed that cap. Set `INBOUND_MEDIA_GLOBAL_CONCURRENCY` to cap concurrent inbound media downloads across all sessions. On whatsapp-web.js a download that times out keeps its slot until Puppeteer fails the call (`PUPPETEER_PROTOCOL_TIMEOUT_MS`, 180 s by default), since it cannot be aborted.
>
-> On PostgreSQL media messages can also wait in memory after the download: received media, and the echo of media this account sends, which is downloaded the same way (every media send on whatsapp-web.js; on Baileys, media sent from a linked phone). Each payload is held as base64, about 4/3 of the file size (roughly 67 MiB for a 50 MiB file), until its message row is stored, and nothing limits how many wait: they pile up while media arrive faster than PostgreSQL writes them. A slow `message:received` or `message:sent` plugin hook, or with `RESOLVE_LID_TO_PHONE=true` a slow phone lookup for a privacy-id sender, holds payloads the same way for as long as it runs, on SQLite too, and the chat's later messages wait behind it. How many wait depends on how long each insert takes. For example, on a local PostgreSQL 16 where a 50 MiB row took about 0.4 s to insert, 24 files of 50 MiB arriving at 8 per second left 13 waiting (about 0.9 GiB, peak RSS 1.8 GiB), and at 4 per second one insert that stalled for 8 s left 21 waiting (1.4 GiB, peak RSS 2.3 GiB); SQLite left none. Size for it as messages waiting x 4/3 x media size, and treat that as a lower bound: writing a row makes further copies of its payload, so peak RSS ran up to about twice the waiting bytes.
+> On PostgreSQL media messages can also wait in memory after the download: received media, and the echo of media this account sends, which is downloaded the same way (every media send on whatsapp-web.js; on Baileys, media sent from a linked phone). Each payload is held as base64, about 4/3 of the file size (roughly 67 MiB for a 50 MiB file), until its message row is stored. A session downloads media only into one of `INBOUND_MEDIA_CONCURRENCY` slots, and a slot stays taken until the row is stored and the engine has let go of the payload, so one session holds at most `INBOUND_MEDIA_CONCURRENCY` x 4 x ceil(`MEDIA_DOWNLOAD_MAX_BYTES` / 3) bytes of payload: 279,620,272 bytes (about 267 MiB) at the defaults, 69,905,068 with `INBOUND_MEDIA_CONCURRENCY=1`. That holds however slow the inserts are, including one that never finishes, and across a reconnect of the session; size a process for sessions x that bound. It does not count a Baileys download in progress (the decoded file, up to `MEDIA_DOWNLOAD_MAX_BYTES`, while it is encoded), the copies the PostgreSQL driver makes while it writes a row, or anything that keeps a message after its row is stored (webhook deliveries, archiving, plugins). Each payload takes a whole slot whatever its size, and a chat's messages are stored in arrival order, so a text waits for media that arrived before it in its chat while other chats keep flowing. When no slot is freed for 2 x `MEDIA_DOWNLOAD_TIMEOUT_MS` (60 s by default), media still waiting for one arrives with the omitted marker, so a stalled database costs media rather than memory. A chat message the sender deletes for everyone before its row is stored is stored cleared on both engines, and media it had not downloaded yet is not downloaded. A delete, edit or reaction from another chat, or a delete or edit from someone other than the sender, is ignored, as it is for a stored message. On SQLite this now also holds while the message is still being downloaded, where Baileys used to pass such a change on. On Baileys, a status post deleted meanwhile, or a message a history sync reports deleted, is dropped instead, as before. On whatsapp-web.js a held message also looks up its sender's contact and its quoted message before it is emitted, and its chat's later messages wait for it: each lookup waits at most `MEDIA_DOWNLOAD_TIMEOUT_MS`, and the message goes out without what did not arrive. On a local PostgreSQL 16 where a 50 MiB row took about 0.4 s to insert, 24 files of 50 MiB arriving at 8 per second held at most 4 payloads (267 MiB; without the slots 17 waited, 1.1 GiB), and a table lock that stalled every insert for 30 s held 4 for the whole stall (without the slots all 24, 1.6 GiB) and shed none, since the 60 s wait had not run out. On SQLite rows are written as messages arrive and none wait, except behind a slow `message:received` or `message:sent` plugin hook, or with `RESOLVE_LID_TO_PHONE=true` a slow phone lookup for a privacy-id sender: there those hold payloads for as long as they run, with no limit on how many, and the chat's later messages wait behind them. On PostgreSQL they run inside the slots: a slot is held until its message's commit settles, which covers the hook chain, the phone lookup, the insert and, on whatsapp-web.js, a timed-out page download (until `PUPPETEER_PROTOCOL_TIMEOUT_MS` fails it). A plugin loaded from the plugins directory delays a slot by at most 5 s per hook (`SANDBOX_HOOK_TIMEOUT_MS`), but a hook registered in-process has no time limit: one that never settles keeps its chat's later messages from being stored, and holds every slot they or its own message took, until the process restarts, since a reconnect does not free them. Once `INBOUND_MEDIA_CONCURRENCY` slots are stuck that way, every media message of the session arrives with the omitted marker. The [high memory runbook](./11-operational-runbooks.md#runbook-high-memory-usage) shows how to find the holder.
>
-> - `MEDIA_DOWNLOAD_MAX_BYTES` caps each waiting payload (larger media arrive as the omitted marker), not how many wait. It also caps outbound media sends.
+> - `MEDIA_DOWNLOAD_MAX_BYTES` caps each waiting payload (larger media arrive as the omitted marker), and with it the bound. It also caps outbound media sends.
> - `MEDIA_DOWNLOAD_ENABLED=false` holds no payload at all; every media message arrives as the omitted marker.
-> - Lowering `INBOUND_MEDIA_CONCURRENCY` does not bound it. It only slows how fast payloads reach the database, which helps while one download takes longer than one insert; a stalled insert still lets them pile up.
-> - Archiving (`CHAT_MEDIA_ARCHIVE_ENABLED=true`) adds to it on either database, in both `MESSAGE_INLINE_MEDIA` modes. Each message is archived after its row is written, with no limit on how many archive at once, and each one holds its base64 payload and the decoded bytes until the upload finishes. `MESSAGE_INLINE_MEDIA=archive` also holds the read-back file, the re-read metadata and its snapshot, and saves no memory, since the row is still written with the inline copy first. Only media up to `CHAT_MEDIA_ARCHIVE_MAX_BYTES` (25 MiB by default) are archived; larger media add nothing, so lowering it bounds the cost per message, not how many archive at once. How many overlap grows with storage latency (such as S3) and with how fast downloads complete. On SQLite, 40 files of 20 MiB at 10 per second peaked at about 0.55 GiB RSS without archiving, and at 1.4 GiB with archiving in the default inline mode when each upload took 2 s (17 at once). In the 8 per second burst above, with `CHAT_MEDIA_ARCHIVE_MAX_BYTES` raised so the 50 MiB files were archived to local disk storage, peak RSS on SQLite rose from about 1 GiB to 2.1 to 3 GiB in inline mode and to 4.2 to 5.1 GiB in archive mode (up to 20 archives at once); on PostgreSQL, whose slower inserts spread the archives out, from 1.8 GiB to 2 GiB in inline mode and to 2.7 to 2.9 GiB in archive mode.
+> - Lowering `INBOUND_MEDIA_CONCURRENCY` lowers the bound on PostgreSQL, at the cost of fewer media downloaded and stored at once. On SQLite it only limits downloads in progress.
+> - Archiving (`CHAT_MEDIA_ARCHIVE_ENABLED=true`) adds to it on either database, in both `MESSAGE_INLINE_MEDIA` modes. Each message is archived after its row is written, with no limit on how many archive at once, and each one holds its base64 payload and the decoded bytes until the upload finishes. `MESSAGE_INLINE_MEDIA=archive` also holds the read-back file, the re-read metadata and its snapshot, and saves no memory, since the row is still written with the inline copy first. Only media up to `CHAT_MEDIA_ARCHIVE_MAX_BYTES` (25 MiB by default) are archived; larger media add nothing, so lowering it bounds the cost per message, not how many archive at once. How many overlap grows with storage latency (such as S3) and with how fast downloads complete. On SQLite, 40 files of 20 MiB at 10 per second peaked at about 0.55 GiB RSS without archiving, and at 1.4 GiB with archiving in the default inline mode when each upload took 2 s (17 at once). In a burst of 24 files of 50 MiB at 8 per second, with `CHAT_MEDIA_ARCHIVE_MAX_BYTES` raised so the 50 MiB files were archived to local disk storage, peak RSS on SQLite rose from about 1 GiB to 2.1 to 3 GiB in inline mode and to 4.2 to 5.1 GiB in archive mode (up to 20 archives at once).
>
> With the queue off (`QUEUE_ENABLED=false`), every webhook delivery that is running, parked or waiting out a retry backoff also holds its event's payload, including inline media up to `WEBHOOK_MEDIA_INLINE_MAX_BYTES` (1 MiB by default) as base64. Up to `WEBHOOK_DISPATCH_CONCURRENCY` + `WEBHOOK_DISPATCH_MAX_QUEUED` deliveries (1016 by default) wait this way, so receivers that stop answering during a media burst can hold about 1.3 GiB per process at the defaults. Lower either dispatch setting or `WEBHOOK_MEDIA_INLINE_MAX_BYTES`, or enable the queue, which keeps those payloads in Redis instead.
diff --git a/docs/19-plugin-architecture.md b/docs/19-plugin-architecture.md
index 1db064263..cd0a309d7 100644
--- a/docs/19-plugin-architecture.md
+++ b/docs/19-plugin-architecture.md
@@ -526,9 +526,13 @@ The gateway still stores and emits each chat's messages in arrival order: the `m
`message:sent` chains run concurrently, but the row insert, the websocket event, `message:persisted` and
the start of webhook dispatch wait for every earlier message of the same chat. A slow handler therefore
delays that chat's later messages, by at most its 5 s hook timeout (`SANDBOX_HOOK_TIMEOUT_MS`) for a
-sandboxed plugin. Webhook dispatch itself is not ordered: its queue jobs and HTTP deliveries run
-concurrently, so a receiver that needs order should sort by the message's `data.timestamp`. That value is
-whole seconds from WhatsApp, so messages sent within the same second cannot be put back in order.
+sandboxed plugin. On PostgreSQL a chat's `message:received` and `message:sent` chains also start in
+arrival order. A handler registered in-process that never settles also holds the media slots of its
+message, so once `INBOUND_MEDIA_CONCURRENCY` slots are stuck the session's media arrives with the omitted
+marker (see [12 - Troubleshooting & FAQ](./12-troubleshooting-faq.md)). Webhook dispatch itself is not
+ordered: its queue jobs and HTTP deliveries run concurrently, so a receiver that needs order should sort
+by the message's `data.timestamp`. That value is whole seconds from WhatsApp, so messages sent within the
+same second cannot be put back in order.
## 19.6 Plugin Loader
diff --git a/src/engine/adapters/baileys-events.ts b/src/engine/adapters/baileys-events.ts
index ed25b9142..c49b4645c 100644
--- a/src/engine/adapters/baileys-events.ts
+++ b/src/engine/adapters/baileys-events.ts
@@ -5,6 +5,7 @@ import {
EngineEventCallbacks,
GroupEvent,
IncomingCallEvent,
+ InboundTicket,
ParticipantPresence,
PresenceState,
CallOutcome,
@@ -33,6 +34,7 @@ import { toUnixSeconds } from './baileys-history';
import { EngineNotReadyError } from '../../common/errors/engine-not-ready.error';
import { CallNotFoundError } from '../../common/errors/call-not-found.error';
import {
+ admitInboundSafely,
capInboundMedia,
coerceDeclaredSize,
inboundMediaMaxBytes,
@@ -158,6 +160,33 @@ export function isWhatsAppMediaSource(
}
}
+const BAILEYS_MEDIA_TYPES = new Set([
+ 'imageMessage',
+ 'videoMessage',
+ 'ptvMessage',
+ 'audioMessage',
+ 'documentMessage',
+ 'documentWithCaptionMessage',
+ 'stickerMessage',
+]);
+
+/** Whether a normalized content type carries a downloadable payload. */
+function isBaileysMediaType(contentType: string | undefined): boolean {
+ return contentType !== undefined && BAILEYS_MEDIA_TYPES.has(contentType);
+}
+
+/** The media sub-message of normalized content: its mimetype and declared size. */
+function mediaSubMessage(normalized: NonNullable<WAMessage['message']>) {
+ return (
+ normalized.imageMessage ??
+ normalized.videoMessage ??
+ normalized.ptvMessage ??
+ normalized.audioMessage ??
+ normalized.documentMessage ??
+ normalized.stickerMessage
+ );
+}
+
export interface BaileysEventsHost {
/** Live socket handle for media re-upload requests (inbound media download). */
getSocket(): WASocket;
@@ -189,6 +218,10 @@ export interface BaileysEventsHost {
consumeOwnSend(id: string | null | undefined): boolean;
/** A message this session already delivered or sent, from the persistent store; undefined without a store. */
getStoredMessage(messageId: string): Promise<WAMessage | null> | undefined;
+ /** The currently-registered admitInbound callback, if any (assigned at initialize()). */
+ getAdmitInbound?(): EngineEventCallbacks['admitInbound'];
+ /** The Baileys module once loaded (it is, before any socket exists), without waiting for it. */
+ getLoadedLib?(): typeof BaileysLib | undefined;
/** The currently-registered onMessage callback, if any (assigned at initialize()). */
getOnMessage(): EngineEventCallbacks['onMessage'];
/** The currently-registered onMessageCreate callback, if any (assigned at initialize()). */
@@ -359,8 +392,11 @@ export class BaileysEvents {
// limiter rejects only when it has been closed, since processInboundMessage handles its own
// failures (a media download that fails emits the omitted marker rather than throwing).
const generation = this.storeGeneration;
+ // Where the session holds a message until its row is written (PostgreSQL), admit it now, in
+ // arrival order: its chat's order, and a slot of the session's room when it downloads media.
+ const ticket = this.admitInbound(msg);
const processed = this.host.inboundLimiter
- .run(() => this.processInboundMessage(msg, { generation }))
+ .run(yieldSlot => this.processInboundMessage(msg, { generation, ticket, yieldSlot }))
.catch((error: unknown) => {
// Only one failure can actually land here today: the limiter closing, an orderly teardown.
// Its queue is unbounded so it never sheds, and processInboundMessage swallows its own
@@ -375,8 +411,11 @@ export class BaileysEvents {
: 'Inbound media download failed; emitting message without media',
{ msgId: msg.key?.id ?? 'unknown', ...(closed ? {} : { error: String(error) }) },
);
- return this.processInboundMessage(msg, { generation, skipMedia: true });
+ return this.processInboundMessage(msg, { generation, skipMedia: true, ticket });
});
+ // The room stays held until the processing settles too, and a message never emitted drops its turn.
+ ticket?.holdUntil(processed);
+ void processed.finally(() => ticket?.drop());
const id = msg.key.id;
if (id) {
// A repeat delivery can arrive while the first is still downloading and finish before it, so
@@ -396,6 +435,47 @@ export class BaileysEvents {
}
}
+ /** Admit a live message (see EngineEventCallbacks.admitInbound); undefined where nothing is held. */
+ private admitInbound(msg: WAMessage): InboundTicket | undefined {
+ const admit = this.host.getAdmitInbound?.();
+ if (!admit) return undefined;
+ return admitInboundSafely(
+ () => {
+ // Learned before the chat id is canonicalized, as processInboundMessage does without a ticket.
+ this.host.recordKeyLidMappings(msg.key);
+ const remoteJid = msg.key.remoteJid!;
+ return admit({
+ id: msg.key.id ?? '',
+ chatId: this.host.toNeutralJid(baileysChatJid(remoteJid, msg.key.participant, msg.key.fromMe === true)),
+ status: remoteJid === 'status@broadcast',
+ needsRoom: this.plannedMediaDownload(this.host.getLoadedLib?.(), msg),
+ });
+ },
+ error =>
+ this.host.logger.error(
+ `Could not admit inbound message ${msg.key.id ?? 'unknown'}; processing it without room for media`,
+ String(error),
+ ),
+ );
+ }
+
+ /**
+ * Whether processing `msg` downloads its media: the gates of {@link resolveInboundMedia}, read before
+ * the processing starts. True when the module is not loaded yet, which cannot happen once a socket
+ * delivers messages, so a slot is never missing for a download.
+ */
+ plannedMediaDownload(b: typeof BaileysLib | undefined, msg: WAMessage): boolean {
+ if (!b) return true;
+ const ownStatusPost = msg.key.fromMe === true && msg.key.remoteJid === 'status@broadcast';
+ const normalized = b.normalizeMessageContent(msg.message ?? undefined) ?? msg.message ?? undefined;
+ return (
+ isBaileysMediaType(b.getContentType(normalized)) &&
+ !ownStatusPost &&
+ isMediaDownloadEnabled() &&
+ coerceDeclaredSize(mediaSubMessage(normalized ?? {})?.fileLength) <= inboundMediaMaxBytes()
+ );
+ }
+
/** Diagnostic: log a contacts event's size + whether records carry names/lids (and a small sample). */
logContactEvent(
event: string,
@@ -421,7 +501,12 @@ export class BaileysEvents {
private async processInboundMessage(
msg: WAMessage,
- opts: { generation: number; skipMedia?: boolean },
+ opts: {
+ generation: number;
+ skipMedia?: boolean;
+ ticket?: InboundTicket;
+ yieldSlot?: <R>(fn: () => Promise<R>) => Promise<R>;
+ },
): Promise<void> {
try {
const b = await this.host.loadLib();
@@ -431,8 +516,8 @@ export class BaileysEvents {
const chatJid = baileysChatJid(remoteJid, msg.key.participant, msg.key.fromMe === true);
// Learn any lid->pn pair the key carries BEFORE canonicalizing ids below, so a fresh @lid
// sender resolves to its phone in this message and for later contact lookups (#362). The pairs
- // also write through to the persistent lid->phone table via addLidMappings.
- this.host.recordKeyLidMappings(msg.key);
+ // also write through to the persistent lid->phone table via addLidMappings. Admission did it already.
+ if (!opts.ticket) this.host.recordKeyLidMappings(msg.key);
// A live disappearing message (also viewOnce / documentWithCaption / edited) arrives wrapped, so the
// raw `getContentType` returns the OUTER wrapper key (e.g. 'ephemeralMessage') and downstream type/
// body/media/location detection would miss the real inner content. Normalize ONCE so the true inner
@@ -619,6 +704,8 @@ export class BaileysEvents {
const ownStatusPost = msg.key.fromMe === true && remoteJid === 'status@broadcast';
const incoming = await this.mapMessage(msg, contentType, {
skipMediaDownload: opts.skipMedia || ownStatusPost,
+ ticket: opts.ticket,
+ yieldSlot: opts.yieldSlot,
});
// Stored before it is announced: whoever hears about this message may act on it at once (a quoted
// reply, a reaction, a read receipt), and the store holds a read of an id until its write lands.
@@ -644,13 +731,15 @@ export class BaileysEvents {
}
// Its delete was announced first and found nothing to clear, so announcing the message now, or
// leaving its text as the chat preview, would publish what the sender took back. An edit announced
- // first found nothing to change either, so the message carries it here and in the preview.
- if (!deleted) {
- if (msg.key.fromMe === true) {
- this.host.getOnMessageCreate()?.(incoming);
- } else {
- this.host.getOnMessage()?.(incoming);
- }
+ // first found nothing to change either, so the message carries it here and in the preview. Where
+ // the projector recorded the delete on the message it holds, the message still goes there, as on
+ // whatsapp-web.js: it stores the row cleared and announces nothing. A delete it never saw (a
+ // status post, a history revoke) keeps the message from being emitted at all.
+ if (!deleted || opts.ticket?.revoked) {
+ const emit = msg.key.fromMe === true ? this.host.getOnMessageCreate() : this.host.getOnMessage();
+ // The ticket goes along only where one was issued, so elsewhere the call is unchanged.
+ if (opts.ticket) emit?.(incoming, opts.ticket);
+ else emit?.(incoming);
}
this.host.recordMessage(msg, incoming.type);
if (deleted) {
@@ -693,8 +782,9 @@ export class BaileysEvents {
* Whether an edit, revoke or reaction names a stored message it cannot touch: one in another chat,
* or, with `checkAuthor`, one somebody else sent. WhatsApp clients ignore such a message, but Baileys
* emits it as-is and the projector updates the stored row by id alone, so a contact who knows an id
- * could rewrite or erase that message. Fails open: with no stored original, or ids that cannot be
- * compared (see {@link differentWaIds}), the event goes through as it always has.
+ * could rewrite or erase that message. A message still being processed is checked against the key
+ * it arrived under. Fails open: with no original at all, or ids that cannot be compared (see
+ * {@link differentWaIds}), the event goes through as it always has.
*
* A reaction to a message the account sent to a broadcast list is exempt: Baileys files the
* own-device copy under the list jid (`<id>@broadcast`), while each recipient reacts from their 1:1
@@ -709,8 +799,11 @@ export class BaileysEvents {
checkAuthor: boolean,
kind?: 'reaction',
): Promise<boolean> {
+ // A message still being processed has no stored copy yet, but the projector may already hold it
+ // (PostgreSQL), so its own key is checked instead.
const original = targetId
- ? (await this.readStoredMessage(targetId, 'checking what an edit, revoke or reaction targets'))?.key
+ ? ((await this.readStoredMessage(targetId, 'checking what an edit, revoke or reaction targets'))?.key ??
+ this.inboundInFlight.get(targetId)?.key)
: undefined;
if (!original) return false;
const broadcast = kind === 'reaction' && !!original.remoteJid?.endsWith('@broadcast');
@@ -1342,16 +1435,9 @@ export class BaileysEvents {
content: NonNullable<WAMessage['message']>,
b: typeof BaileysLib,
skipMediaDownload: boolean,
+ room?: { ticket?: InboundTicket; yieldSlot?: <R>(fn: () => Promise<R>) => Promise<R> },
): Promise<IncomingMessage['media']> {
- const isMediaType =
- contentType === 'imageMessage' ||
- contentType === 'videoMessage' ||
- contentType === 'ptvMessage' ||
- contentType === 'audioMessage' ||
- contentType === 'documentMessage' ||
- contentType === 'documentWithCaptionMessage' ||
- contentType === 'stickerMessage';
- if (!isMediaType) {
+ if (!isBaileysMediaType(contentType)) {
return undefined;
}
@@ -1363,13 +1449,7 @@ export class BaileysEvents {
// Emit the omitted marker so the media field is present (webhook/n8n/dashboard contract).
// mimetype is available pre-download from the message content.
const normalizedContent = b.normalizeMessageContent(content) ?? content;
- const subMessage =
- normalizedContent.imageMessage ??
- normalizedContent.videoMessage ??
- normalizedContent.ptvMessage ??
- normalizedContent.audioMessage ??
- normalizedContent.documentMessage ??
- normalizedContent.stickerMessage;
+ const subMessage = mediaSubMessage(normalizedContent);
return {
mimetype: subMessage?.mimetype ?? '',
filename: normalizedContent.documentMessage?.fileName ?? undefined,
@@ -1381,13 +1461,7 @@ export class BaileysEvents {
// normalizeMessageContent unwraps documentWithCaptionMessage / viewOnceMessage / ephemeralMessage
// so we reach the inner media sub-message — needed BEFORE download for the declared-size pre-gate.
const normalizedContent = b.normalizeMessageContent(content) ?? content;
- const subMessage =
- normalizedContent.imageMessage ??
- normalizedContent.videoMessage ??
- normalizedContent.ptvMessage ??
- normalizedContent.audioMessage ??
- normalizedContent.documentMessage ??
- normalizedContent.stickerMessage;
+ const subMessage = mediaSubMessage(normalizedContent);
const mimetype = subMessage?.mimetype ?? '';
const filename = normalizedContent.documentMessage?.fileName ?? undefined;
const maxBytes = inboundMediaMaxBytes();
@@ -1405,6 +1479,21 @@ export class BaileysEvents {
}
try {
+ // Where payloads wait for their rows (PostgreSQL), download only into a slot of the session's
+ // room, held until the row is written. The inbound limiter slot is given up while waiting, so
+ // other chats' messages keep flowing. No room in time: the marker, and nothing downloaded.
+ const ticket = room?.ticket;
+ if (ticket) {
+ const granted = ticket.reserve();
+ const yieldSlot = room?.yieldSlot ?? (<R>(fn: () => Promise<R>) => fn());
+ if (!(granted === true || (granted !== false && (await yieldSlot(() => granted))))) {
+ this.host.logger.warn('No room for inbound media within the wait; emitting the omitted marker', {
+ msgId: msg.key.id,
+ sizeBytes: declared,
+ });
+ return { mimetype, filename, omitted: true, sizeBytes: declared };
+ }
+ }
// Stream-download with a running-total abort so a sender who understates fileLength still
// can't materialise an over-cap blob. For under-cap media this yields the identical buffer.
const buf = await this.downloadInboundMediaCapped(msg, maxBytes);
@@ -1449,7 +1538,11 @@ export class BaileysEvents {
async mapMessage(
msg: WAMessage,
contentType: string | undefined,
- opts?: { skipMediaDownload?: boolean },
+ opts?: {
+ skipMediaDownload?: boolean;
+ ticket?: InboundTicket;
+ yieldSlot?: <R>(fn: () => Promise<R>) => Promise<R>;
+ },
): Promise<IncomingMessage> {
const b = await this.host.loadLib();
const content = msg.message ?? {};
@@ -1462,7 +1555,7 @@ export class BaileysEvents {
// Body: text first, then media caption, then WhatsApp Business interactive shapes (#562).
const body = extractBaileysBody(normalized);
const location = extractBaileysLocation(normalized, contentType);
- const media = await this.resolveInboundMedia(msg, contentType, content, b, opts?.skipMediaDownload === true);
+ const media = await this.resolveInboundMedia(msg, contentType, content, b, opts?.skipMediaDownload === true, opts);
// The quote, the disappearing-messages timer, the mentions and the status styling all come from
// one region of the content — see BaileysMessageContext.
const context = extractBaileysContext(normalized);
diff --git a/src/engine/adapters/baileys-lifecycle.ts b/src/engine/adapters/baileys-lifecycle.ts
index 03f591810..a5f563c60 100644
--- a/src/engine/adapters/baileys-lifecycle.ts
+++ b/src/engine/adapters/baileys-lifecycle.ts
@@ -222,6 +222,11 @@ export class BaileysLifecycle {
});
}
+ /** The module once {@link loadLib} has loaded it, which happens before any socket exists. */
+ get loadedLib(): typeof BaileysLib | undefined {
+ return this.lib;
+ }
+
/** Lazily loaded @whiskeysockets/baileys module (ESM-only; loaded on first connect, not at boot). */
async loadLib(): Promise<typeof BaileysLib> {
return (this.lib ??= await import('@whiskeysockets/baileys'));
diff --git a/src/engine/adapters/baileys.adapter.ts b/src/engine/adapters/baileys.adapter.ts
index 1d566616a..7e18b0f0a 100644
--- a/src/engine/adapters/baileys.adapter.ts
+++ b/src/engine/adapters/baileys.adapter.ts
@@ -152,6 +152,8 @@ export class BaileysAdapter implements IWhatsAppEngine {
pendingEditOf: (messageId, target) => this.events.pendingEditOf(messageId, target),
rememberOwnSend: id => this.ownSends.remember(id),
consumeOwnSend: id => this.ownSends.consume(id),
+ getAdmitInbound: () => this.callbacks.admitInbound,
+ getLoadedLib: () => this.lifecycle.loadedLib,
getOnMessage: () => this.callbacks.onMessage,
getOnMessageCreate: () => this.callbacks.onMessageCreate,
getOnMessageRevoked: () => this.callbacks.onMessageRevoked,
diff --git a/src/engine/adapters/inbound-hold.spec.ts b/src/engine/adapters/inbound-hold.spec.ts
new file mode 100644
index 000000000..c9b6428a5
--- /dev/null
+++ b/src/engine/adapters/inbound-hold.spec.ts
@@ -0,0 +1,829 @@
+import { EventEmitter } from 'events';
+import { readFileSync, readdirSync } from 'node:fs';
+import { join } from 'node:path';
+import { WAState } from 'whatsapp-web.js';
+import type { Repository } from 'typeorm';
+import type { WAMessage, WASocket } from '@whiskeysockets/baileys';
+import { BaileysEvents, type BaileysEventsHost } from './baileys-events';
+import { WhatsAppWebJsAdapter } from './whatsapp-web-js.adapter';
+import { ConcurrencyLimiter } from '../../common/utils/concurrency-limiter';
+import { createLogger } from '../../common/services/logger.service';
+import { EngineRegistry } from '../engine-registry.service';
+import { MessageProjector } from '../../modules/session/message-projector.service';
+import { deferred } from '../../modules/session/inbound-room';
+import type { Message } from '../../modules/message/entities/message.entity';
+import type { Session } from '../../modules/session/entities/session.entity';
+import type { EventsGateway } from '../../modules/events/events.gateway';
+import type { WebhookService } from '../../modules/webhook/webhook.service';
+import type { HookManager } from '../../core/hooks';
+import type { StatusStoreService } from '../../modules/status-store/status-store.service';
+import type { SessionLidResolver } from '../../modules/session/session-lid-resolver.service';
+import type { EngineEventCallbacks, IncomingMessage, IWhatsAppEngine } from '../interfaces/whatsapp-engine.interface';
+
+/**
+ * Both engines against the real projector, as PostgreSQL wires them: each inbound message is admitted
+ * on arrival, its media is downloaded only into a slot of the session's room, held until its row is
+ * written, and its chat's order and any change that reaches it before its emit are kept.
+ */
+
+type Row = Record<string, unknown>;
+
+const flush = async (): Promise<void> => {
+ for (let i = 0; i < 8; i++) await new Promise(resolve => setImmediate(resolve));
+};
+const sleep = (ms: number): Promise<void> => new Promise(resolve => setTimeout(resolve, ms));
+
+/** The real projector over a stand-in `messages` table whose inserts a test can stall per message. */
+function heldProjector() {
+ const rows: Row[] = [];
+ const hookStarts: string[] = [];
+ const hookedBodies: Record<string, string> = {};
+ const stalls = new Map<string, Promise<void>>();
+ const repo = {
+ manager: { connection: { options: { type: 'postgres' } } },
+ find: jest.fn().mockResolvedValue([]),
+ findOne: jest.fn(({ where }: { where: Row }) =>
+ Promise.resolve(rows.find(r => r.waMessageId === where.waMessageId) ?? null),
+ ),
+ create: jest.fn((x: Row) => ({ ...x })),
+ insert: jest.fn(async (row: Row) => {
+ await stalls.get(String(row.waMessageId));
+ rows.push({ ...row });
+ return { identifiers: [{ id: `pk-${String(row.waMessageId)}` }], generatedMaps: [] };
+ }),
+ update: jest.fn((where: Row, patch: Row) => {
+ const hit = rows.filter(r => r.waMessageId === where.waMessageId);
+ for (const r of hit) Object.assign(r, patch);
+ return Promise.resolve({ affected: hit.length });
+ }),
+ };
+ const hookManager = {
+ execute: jest.fn((event: string, data: unknown) => {
+ if (event === 'message:received' || event === 'message:sent') {
+ hookStarts.push((data as IncomingMessage).id);
+ hookedBodies[(data as IncomingMessage).id] = (data as IncomingMessage).body;
+ }
+ return Promise.resolve({ continue: true, data });
+ }),
+ };
+ const webhookService = { dispatch: jest.fn().mockResolvedValue(undefined) };
+ const ingested: string[] = [];
+ const statusStore = {
+ ingest: jest.fn((_s: string, status: { id: string }) => {
+ ingested.push(status.id);
+ return Promise.resolve({ row: status, created: false });
+ }),
+ };
+ const gateway = new Proxy({}, { get: () => jest.fn() });
+ const engines = new EngineRegistry();
+ const engine = {} as IWhatsAppEngine;
+ engines.set('s1', engine);
+ const projector = new MessageProjector(
+ repo as unknown as Repository<Message>,
+ { update: jest.fn().mockResolvedValue({ affected: 1 }) } as unknown as Repository<Session>,
+ engines,
+ gateway as unknown as EventsGateway,
+ webhookService as unknown as WebhookService,
+ hookManager as unknown as HookManager,
+ statusStore as unknown as StatusStoreService,
+ { resolveSenderPhone: jest.fn().mockResolvedValue(null) } as unknown as SessionLidResolver,
+ );
+ // The same table SessionEngineEventWiring builds on PostgreSQL.
+ const callbacksFor = (e: IWhatsAppEngine): EngineEventCallbacks => ({
+ admitInbound: a => projector.admitInbound('s1', e, a),
+ onMessage: (m, t) => projector.handleInboundMessage('s1', e, m, t),
+ onMessageCreate: (m, t) => projector.handleOwnSendEcho('s1', e, m, t),
+ onMessageRevoked: m => projector.handleMessageRevoked('s1', e, m),
+ onMessageEdited: m => projector.applyMessageEditQueued('s1', m),
+ });
+ const callbacks = callbacksFor(engine);
+ const stall = (id: string): (() => void) => {
+ const gate = deferred<void>();
+ stalls.set(id, gate.promise);
+ return () => gate.resolve();
+ };
+ const announced = (): string[] =>
+ (webhookService.dispatch.mock.calls as Array<[string, string, Row]>)
+ .filter(c => c[1] === 'message.received')
+ .map(c => String(c[2].id));
+ return {
+ projector,
+ rows,
+ hookStarts,
+ hookedBodies,
+ callbacks,
+ callbacksFor,
+ engines,
+ stall,
+ announced,
+ ingested,
+ inserted: () => rows.map(r => r.waMessageId),
+ };
+}
+
+const env = { ...process.env };
+afterEach(() => {
+ process.env = { ...env };
+});
+
+describe('Baileys inbound messages held for storage', () => {
+ const downloadMediaMessage = jest.fn();
+ const unwrap = (content: Row | null | undefined): Row | undefined => {
+ let current = content ?? undefined;
+ for (let i = 0; i < 5 && current; i++) {
+ const inner = (current.ephemeralMessage ?? current.viewOnceMessage ?? current.documentWithCaptionMessage) as
+ { message?: Row } | undefined;
+ if (!inner) break;
+ current = inner.message;
+ }
+ return current;
+ };
+ const lib = {
+ normalizeMessageContent: unwrap,
+ extractMessageContent: unwrap,
+ getContentType: (content: Row | undefined) =>
+ Object.keys(content ?? {}).find(k => k === 'conversation' || k.endsWith('Message')),
+ downloadMediaMessage,
+ proto: { Message: { ProtocolMessage: { Type: { REVOKE: 0, MESSAGE_EDIT: 14 } } } },
+ } as unknown as Awaited<ReturnType<BaileysEventsHost['loadLib']>>;
+
+ const stream = (bytes = 3) => ({
+ // eslint-disable-next-line @typescript-eslint/require-await
+ async *[Symbol.asyncIterator]() {
+ yield Buffer.alloc(bytes, 1);
+ },
+ destroy: jest.fn(),
+ });
+
+ const build = (concurrency: number, callbacks?: EngineEventCallbacks) => {
+ const emitted: string[] = [];
+ const recordKeyLidMappings = jest.fn();
+ const host = {
+ getSocket: () => ({ updateMediaMessage: jest.fn() }) as unknown as WASocket,
+ getSocketOrNull: () => null,
+ logger: createLogger('InboundHoldSpec'),
+ loadLib: () => Promise.resolve(lib),
+ getLoadedLib: () => lib,
+ getFetchDispatcher: () => undefined,
+ toNeutralJid: (jid: string) => jid,
+ normalizedSelfJid: () => '[email protected]',
+ inboundLimiter: new ConcurrencyLimiter(concurrency),
+ recordKeyLidMappings,
+ recordMessage: () => undefined,
+ recordMessageEdit: () => undefined,
+ consumeOwnSend: () => false,
+ getStoredMessage: () => undefined,
+ putStoredMessage: () => undefined,
+ updateStoredMessage: () => undefined,
+ getAdmitInbound: () => callbacks?.admitInbound,
+ getOnMessage: () => (m: IncomingMessage, t?: never) => {
+ emitted.push(m.id);
+ callbacks?.onMessage?.(m, t);
+ },
+ getOnMessageCreate: () => callbacks?.onMessageCreate,
+ getOnMessageRevoked: () => callbacks?.onMessageRevoked,
+ getOnMessageEdited: () => callbacks?.onMessageEdited,
+ getOnMessageReaction: () => callbacks?.onMessageReaction,
+ getOnMessageAck: () => undefined,
+ getOnGroupEvent: () => undefined,
+ getOnCall: () => undefined,
+ getOnPresenceUpdate: () => undefined,
+ getOnCallOutcome: () => undefined,
+ } as unknown as BaileysEventsHost;
+ return { events: new BaileysEvents(host), emitted, recordKeyLidMappings };
+ };
+
+ const media = (id: string, chat: string, over: Row = {}): WAMessage => ({
+ key: { id, remoteJid: chat, fromMe: false },
+ messageTimestamp: 1_700_000_000,
+ message: { imageMessage: { mimetype: 'image/jpeg', fileLength: 3, ...over } },
+ });
+ const text = (id: string, chat: string): WAMessage => ({
+ key: { id, remoteJid: chat, fromMe: false },
+ messageTimestamp: 1_700_000_000,
+ message: { conversation: `text ${id}` },
+ });
+ const upsert = (events: BaileysEvents, ...messages: WAMessage[]) =>
+ events.handleMessagesUpsert({ messages, type: 'notify' });
+
+ beforeEach(() => {
+ downloadMediaMessage.mockReset();
+ downloadMediaMessage.mockImplementation(() => Promise.resolve(stream()));
+ });
+
+ describe.each([1, 4])('at INBOUND_MEDIA_CONCURRENCY=%i', k => {
+ beforeEach(() => {
+ process.env.INBOUND_MEDIA_CONCURRENCY = String(k);
+ });
+
+ // K payloads stored in K chats whose inserts stall: the room is full and stays full.
+ const fillRoom = async (h: ReturnType<typeof heldProjector>, events: BaileysEvents) => {
+ const holders = Array.from({ length: k }, (_, i) => `H${i}`);
+ const releases = holders.map(id => h.stall(id));
+ upsert(events, ...holders.map((id, i) => media(id, `hold${i}@s.whatsapp.net`)));
+ await flush();
+ expect(downloadMediaMessage).toHaveBeenCalledTimes(k);
+ return () => releases.forEach(release => release());
+ };
+
+ it('lets another chat text through while media waits for room, its limiter slot given up', async () => {
+ const h = heldProjector();
+ const { events, emitted } = build(k, h.callbacks);
+ await fillRoom(h, events);
+
+ // K media waiting for room would hold every limiter slot if they did not give theirs up.
+ upsert(events, ...Array.from({ length: k }, (_, i) => media(`W${i}`, `wait${i}@s.whatsapp.net`)));
+ upsert(events, text('T', '[email protected]'));
+ await flush();
+
+ expect(emitted).toContain('T');
+ expect(h.inserted()).toEqual(['T']);
+ expect(downloadMediaMessage).toHaveBeenCalledTimes(k);
+ });
+
+ it('commits a same-chat text after the media it arrived behind', async () => {
+ const h = heldProjector();
+ const { events, emitted } = build(k, h.callbacks);
+ const release = await fillRoom(h, events);
+ upsert(events, media('M', '[email protected]'), text('T', '[email protected]'));
+ await flush();
+ // The text is processed meanwhile, but its hook and row wait for the media.
+ expect(emitted).toContain('T');
+ expect(h.hookStarts).not.toContain('T');
+
+ release();
+ await flush();
+ const order = h.inserted().filter(id => id === 'M' || id === 'T');
+ expect(order).toEqual(['M', 'T']);
+ expect(h.hookStarts.indexOf('M')).toBeLessThan(h.hookStarts.indexOf('T'));
+ });
+
+ it('emits the omitted marker without downloading when no room comes free in time', async () => {
+ process.env.MEDIA_DOWNLOAD_TIMEOUT_MS = '20';
+ const h = heldProjector();
+ const seen: IncomingMessage[] = [];
+ const onMessage = h.callbacks.onMessage!;
+ const { events } = build(k, { ...h.callbacks, onMessage: (m, t) => (seen.push(m), onMessage(m, t)) });
+ await fillRoom(h, events);
+ upsert(events, media('LATE', '[email protected]', { fileLength: 7 }));
+ await sleep(80);
+ await flush();
+
+ expect(downloadMediaMessage).toHaveBeenCalledTimes(k);
+ expect(seen.find(m => m.id === 'LATE')?.media).toEqual(expect.objectContaining({ omitted: true, sizeBytes: 7 }));
+ expect(h.inserted()).toContain('LATE');
+ });
+ });
+
+ it("keeps the room of a retired engine's download until it settles, so a replacement engine's media waits", async () => {
+ process.env.INBOUND_MEDIA_CONCURRENCY = '1';
+ const h = heldProjector();
+ const old = build(1, h.callbacks);
+ const pending = deferred<ReturnType<typeof stream>>();
+ downloadMediaMessage.mockImplementationOnce(() => pending.promise);
+ upsert(old.events, media('A', '[email protected]'));
+ await flush();
+ expect(downloadMediaMessage).toHaveBeenCalledTimes(1);
+
+ // Retiring the engine drops its pending turn, so only the hold on the processing keeps the slot.
+ const fresh = {} as IWhatsAppEngine;
+ h.engines.set('s1', fresh);
+ const next = build(1, h.callbacksFor(fresh));
+ upsert(next.events, media('B', '[email protected]'));
+ await flush();
+ expect(downloadMediaMessage).toHaveBeenCalledTimes(1);
+
+ pending.resolve(stream());
+ await flush();
+ expect(downloadMediaMessage).toHaveBeenCalledTimes(2);
+ expect(h.inserted()).toEqual(['B']);
+ });
+
+ it('downloads nothing for media granted room but deleted for everyone before its download starts', async () => {
+ // Room for both; the limiter has one slot, so GONE is granted room at arrival but queued behind H.
+ process.env.INBOUND_MEDIA_CONCURRENCY = '2';
+ const h = heldProjector();
+ const { events } = build(1, h.callbacks);
+ const pending = deferred<ReturnType<typeof stream>>();
+ downloadMediaMessage.mockImplementationOnce(() => pending.promise);
+ upsert(events, media('H', '[email protected]'), media('GONE', '[email protected]'));
+ await flush();
+ // The delete is applied outside the limiter, which GONE still waits in (a delete delivered through
+ // the same limiter would queue behind it).
+ h.callbacks.onMessageRevoked!({
+ id: 'GONE',
+ revokedId: 'GONE',
+ chatId: '[email protected]',
+ type: 'revoked',
+ body: '',
+ timestamp: 1_700_000_001,
+ });
+ await flush();
+ expect(downloadMediaMessage).toHaveBeenCalledTimes(1);
+
+ pending.resolve(stream());
+ await flush();
+
+ expect(downloadMediaMessage).toHaveBeenCalledTimes(1);
+ expect(h.inserted().sort()).toEqual(['GONE', 'H']);
+ expect(h.rows.find(r => r.waMessageId === 'GONE')).toEqual(expect.objectContaining({ body: '', type: 'revoked' }));
+ expect(h.announced()).toEqual(['H']);
+ });
+
+ it('stores a message deleted for everyone while it downloaded cleared, unannounced, and gives its slot back', async () => {
+ // One slot of room; the limiter has spare slots, so the delete is processed during the download.
+ process.env.INBOUND_MEDIA_CONCURRENCY = '1';
+ const h = heldProjector();
+ const { events, emitted } = build(4, h.callbacks);
+ const pending = deferred<ReturnType<typeof stream>>();
+ downloadMediaMessage.mockImplementationOnce(() => pending.promise);
+ upsert(events, media('GONE', '[email protected]'), media('NEXT', '[email protected]'));
+ await flush();
+ upsert(events, {
+ key: { id: 'REV', remoteJid: '[email protected]', fromMe: false },
+ messageTimestamp: 1_700_000_001,
+ message: { protocolMessage: { type: 0, key: { id: 'GONE', remoteJid: '[email protected]', fromMe: false } } },
+ });
+ await flush();
+ expect(downloadMediaMessage).toHaveBeenCalledTimes(1);
+
+ pending.resolve(stream());
+ await flush();
+
+ // As on whatsapp-web.js: the row is kept, cleared, and only what was not taken back is announced.
+ expect(emitted).toEqual(['GONE', 'NEXT']);
+ expect(downloadMediaMessage).toHaveBeenCalledTimes(2);
+ expect(h.inserted()).toEqual(['GONE', 'NEXT']);
+ expect(h.rows[0]).toEqual(expect.objectContaining({ body: '', type: 'revoked' }));
+ expect(h.announced()).toEqual(['NEXT']);
+ });
+
+ it('downloads nothing for media deleted for everyone while it waits for room, and stores it cleared', async () => {
+ process.env.INBOUND_MEDIA_CONCURRENCY = '1';
+ const h = heldProjector();
+ const { events } = build(4, h.callbacks);
+ const release = h.stall('H');
+ upsert(events, media('H', '[email protected]'), media('GONE', '[email protected]'));
+ await flush();
+ expect(downloadMediaMessage).toHaveBeenCalledTimes(1);
+ upsert(events, {
+ key: { id: 'REV', remoteJid: '[email protected]', fromMe: false },
+ messageTimestamp: 1_700_000_001,
+ message: { protocolMessage: { type: 0, key: { id: 'GONE', remoteJid: '[email protected]', fromMe: false } } },
+ });
+ await flush();
+
+ expect(downloadMediaMessage).toHaveBeenCalledTimes(1);
+ expect(h.rows).toEqual([expect.objectContaining({ waMessageId: 'GONE', body: '', type: 'revoked' })]);
+ expect(h.announced()).toEqual([]);
+ release();
+ await flush();
+ expect(h.inserted()).toEqual(['GONE', 'H']);
+ });
+
+ it.each([
+ ['from another chat', { remoteJid: '[email protected]', fromMe: false }],
+ ['from the wrong author', { remoteJid: '[email protected]', fromMe: true }],
+ ])('ignores a delete and an edit %s aimed at media waiting for room', async (_, envelope) => {
+ process.env.INBOUND_MEDIA_CONCURRENCY = '1';
+ const h = heldProjector();
+ const { events } = build(4, h.callbacks);
+ const release = h.stall('H');
+ upsert(
+ events,
+ media('H', '[email protected]'),
+ media('MREV', '[email protected]', { caption: 'real caption' }),
+ media('MEDIT', '[email protected]', { caption: 'real caption 2' }),
+ );
+ await flush();
+ const target = (id: string) => ({ id, remoteJid: envelope.remoteJid, fromMe: false });
+ upsert(
+ events,
+ {
+ key: { id: 'REV', ...envelope },
+ messageTimestamp: 1_700_000_001,
+ message: { protocolMessage: { type: 0, key: target('MREV') } },
+ },
+ {
+ key: { id: 'ED', ...envelope },
+ messageTimestamp: 1_700_000_001,
+ message: { protocolMessage: { type: 14, key: target('MEDIT'), editedMessage: { conversation: 'forged' } } },
+ },
+ );
+ await flush();
+ release();
+ await flush();
+
+ const row = (id: string) => h.rows.find(r => r.waMessageId === id);
+ expect(row('MREV')).toEqual(expect.objectContaining({ body: 'real caption', type: 'image' }));
+ expect(row('MEDIT')).toEqual(expect.objectContaining({ body: 'real caption 2' }));
+ expect(h.hookedBodies.MEDIT).toBe('real caption 2');
+ expect([...h.announced()].sort()).toEqual(['H', 'MEDIT', 'MREV']);
+ expect(downloadMediaMessage).toHaveBeenCalledTimes(3);
+ });
+
+ it.each([
+ ['waiting for room (PostgreSQL)', true],
+ ['still downloading (SQLite)', false],
+ ])('drops a reaction from another chat aimed at media %s', async (_, admitted) => {
+ process.env.INBOUND_MEDIA_CONCURRENCY = '1';
+ const h = heldProjector();
+ const reactions: Array<{ messageId: string; chatId: string }> = [];
+ const onMessageReaction: EngineEventCallbacks['onMessageReaction'] = r => reactions.push(r);
+ const download = deferred<void>();
+ let release: () => void;
+ let callbacks: EngineEventCallbacks;
+ if (admitted) {
+ release = h.stall('H');
+ callbacks = { ...h.callbacks, onMessageReaction };
+ } else {
+ // No admission callback, as SQLite wires the engine: M is held only by its own download.
+ release = () => download.resolve();
+ downloadMediaMessage.mockImplementation(() => download.promise.then(() => stream()));
+ callbacks = { onMessage: jest.fn(), onMessageReaction };
+ }
+ const { events } = build(4, callbacks);
+ const chat = '[email protected]';
+ upsert(events, ...(admitted ? [media('H', '[email protected]')] : []), media('M', chat));
+ await flush();
+ const react = (id: string, remoteJid: string, text: string): WAMessage => ({
+ key: { id, remoteJid, fromMe: false },
+ messageTimestamp: 1_700_000_001,
+ message: { reactionMessage: { key: { id: 'M', remoteJid, fromMe: false }, text } },
+ });
+ upsert(events, react('R1', '[email protected]', 'forged'), react('R2', chat, 'genuine'));
+ await flush();
+ release();
+ await flush();
+
+ expect(reactions).toEqual([expect.objectContaining({ messageId: 'M', chatId: chat, reaction: 'genuine' })]);
+ });
+
+ it('neither ingests nor emits a status post deleted while it waits for room', async () => {
+ process.env.INBOUND_MEDIA_CONCURRENCY = '1';
+ const h = heldProjector();
+ const { events, emitted } = build(4, h.callbacks);
+ const release = h.stall('H');
+ const poster = { remoteJid: 'status@broadcast', participant: '[email protected]', fromMe: false };
+ upsert(events, media('H', '[email protected]'), {
+ key: { id: 'ST', ...poster },
+ messageTimestamp: 1_700_000_000,
+ message: { imageMessage: { mimetype: 'image/jpeg', fileLength: 3, caption: 'secret' } },
+ });
+ await flush();
+ // The projector keeps no in-flight entry for a status post, so it never sees this revoke.
+ upsert(events, {
+ key: { id: 'REV', ...poster },
+ messageTimestamp: 1_700_000_001,
+ message: { protocolMessage: { type: 0, key: { id: 'ST', ...poster } } },
+ });
+ await flush();
+ release();
+ await flush();
+
+ expect(emitted).toEqual(['H']);
+ expect(h.ingested).toEqual([]);
+ });
+
+ it('neither emits nor stores a held message a history revoke deleted', async () => {
+ process.env.INBOUND_MEDIA_CONCURRENCY = '1';
+ const h = heldProjector();
+ const { events, emitted } = build(4, h.callbacks);
+ const release = h.stall('H');
+ const chat = '[email protected]';
+ upsert(events, media('H', '[email protected]'), media('GONE', chat));
+ await flush();
+ await events.applyHistoryRevoke({ id: 'GONE', remoteJid: chat, fromMe: false }, { remoteJid: chat, fromMe: false });
+ release();
+ await flush();
+
+ expect(emitted).toEqual(['H']);
+ expect(h.hookStarts).toEqual(['H']);
+ expect(h.inserted()).toEqual(['H']);
+ expect(h.announced()).toEqual(['H']);
+ });
+
+ it('still processes a message, and the rest of its batch, when its admission throws', async () => {
+ const h = heldProjector();
+ let calls = 0;
+ const admitInbound: EngineEventCallbacks['admitInbound'] = a => {
+ if (++calls === 1) throw new Error('boom');
+ return h.callbacks.admitInbound!(a);
+ };
+ const seen: IncomingMessage[] = [];
+ const onMessage = h.callbacks.onMessage!;
+ const { events } = build(4, { ...h.callbacks, admitInbound, onMessage: (m, t) => (seen.push(m), onMessage(m, t)) });
+ upsert(events, media('A', '[email protected]', { fileLength: 7 }), text('B', '[email protected]'));
+ await flush();
+
+ // No room was reserved for it, so its media is not downloaded.
+ expect(seen.find(m => m.id === 'A')?.media).toEqual(expect.objectContaining({ omitted: true, sizeBytes: 7 }));
+ expect(downloadMediaMessage).not.toHaveBeenCalled();
+ expect(h.inserted().sort()).toEqual(['A', 'B']);
+ });
+
+ it.each([
+ ['an image', media('A', '[email protected]'), true],
+ [
+ 'a disappearing image',
+ {
+ ...media('A', '[email protected]'),
+ message: { ephemeralMessage: { message: { imageMessage: { fileLength: 3 } } } },
+ },
+ true,
+ ],
+ [
+ 'a view-once image',
+ {
+ ...media('A', '[email protected]'),
+ message: { viewOnceMessage: { message: { imageMessage: { fileLength: 3 } } } },
+ },
+ true,
+ ],
+ ['media declared over the cap', media('A', '[email protected]', { fileLength: 60 * 1024 * 1024 }), false],
+ [
+ 'the account own status post',
+ { ...media('A', 'status@broadcast'), key: { id: 'A', remoteJid: 'status@broadcast', fromMe: true } },
+ false,
+ ],
+ ['a protocol message', { ...text('A', '[email protected]'), message: { protocolMessage: { type: 5 } } }, false],
+ [
+ 'a reaction',
+ { ...text('A', '[email protected]'), message: { reactionMessage: { key: { id: 'X' }, text: 'ok' } } },
+ false,
+ ],
+ ['a text', text('A', '[email protected]'), false],
+ ] as Array<[string, WAMessage, boolean]>)(
+ 'plans a download for %s exactly when processing downloads it',
+ async (_, msg, planned) => {
+ const { events } = build(4);
+ expect(events.plannedMediaDownload(lib, msg)).toBe(planned);
+ upsert(events, msg);
+ await flush();
+ expect(downloadMediaMessage).toHaveBeenCalledTimes(planned ? 1 : 0);
+ },
+ );
+
+ it('plans no download, and downloads nothing, with media downloads disabled', async () => {
+ process.env.MEDIA_DOWNLOAD_ENABLED = 'false';
+ const { events } = build(4);
+ const msg = media('A', '[email protected]');
+ expect(events.plannedMediaDownload(lib, msg)).toBe(false);
+ upsert(events, msg);
+ await flush();
+ expect(downloadMediaMessage).not.toHaveBeenCalled();
+ });
+
+ it('records the key lid pairs once per message, with or without a ticket', async () => {
+ const h = heldProjector();
+ const held = build(4, h.callbacks);
+ upsert(held.events, text('A', '[email protected]'));
+ const plain = build(4);
+ upsert(plain.events, text('B', '[email protected]'));
+ await flush();
+ expect(held.recordKeyLidMappings).toHaveBeenCalledTimes(1);
+ expect(plain.recordKeyLidMappings).toHaveBeenCalledTimes(1);
+ });
+});
+
+describe('whatsapp-web.js inbound messages held for storage', () => {
+ const build = (callbacks: EngineEventCallbacks) => {
+ const adapter = new WhatsAppWebJsAdapter({ sessionId: 'hold', sessionDataPath: './data/sessions', puppeteer: {} });
+ const client = Object.assign(new EventEmitter(), {
+ info: { wid: { user: '628123' }, pushname: 'Tester' },
+ getState: jest.fn().mockResolvedValue(WAState.CONNECTED),
+ pupPage: { evaluate: jest.fn().mockResolvedValue(true) },
+ });
+ (adapter as unknown as { client: unknown }).client = client;
+ (adapter as unknown as { callbacks: unknown }).callbacks = callbacks;
+ (adapter as unknown as { setupEventHandlers: () => void }).setupEventHandlers();
+ return { adapter, client };
+ };
+ const incoming = (id: string, from: string, page?: jest.Mock, body = '') => ({
+ id: { _serialized: id },
+ from,
+ body,
+ type: page ? 'image' : 'chat',
+ timestamp: 1_700_000_000,
+ fromMe: false,
+ hasMedia: Boolean(page),
+ _data: { mimetype: 'image/png', size: 3 },
+ client: { pupPage: { evaluate: page } },
+ getContact: jest.fn().mockResolvedValue(null),
+ hasQuotedMsg: false,
+ });
+
+ beforeEach(() => {
+ process.env.INBOUND_MEDIA_CONCURRENCY = '1';
+ });
+
+ it.each(['revoke', 'edit'] as const)(
+ 'applies the %s of a text held behind a same-chat image, never announcing or hooking what was taken back',
+ async change => {
+ const h = heldProjector();
+ const { client } = build(h.callbacks);
+ const page = deferred<{ mimetype: string; data: string }>();
+ client.emit(
+ 'message',
+ incoming(
+ 'm1',
+ jest.fn(() => page.promise),
+ ),
+ );
+ client.emit('message', incoming('t2', '[email protected]', undefined, 'secret'));
+ await flush();
+ if (change === 'revoke') {
+ client.emit(
+ 'message_revoke_everyone',
+ { id: { _serialized: 'rev' }, from: '[email protected]', to: '[email protected]', fromMe: false, timestamp: 2 },
+ { id: { _serialized: 't2' } },
+ );
+ } else {
+ client.emit(
+ 'message_edit',
+ { id: { _serialized: 't2' }, from: '[email protected]', to: '[email protected]', type: 'chat', fromMe: false },
+ 'corrected',
+ );
+ }
+ await flush();
+ page.resolve({ mimetype: 'image/png', data: 'QUJD' });
+ await flush();
+
+ const t2 = h.rows.find(r => r.waMessageId === 't2');
+ expect(h.inserted()).toEqual(['m1', 't2']);
+ if (change === 'revoke') {
+ expect(t2).toEqual(expect.objectContaining({ body: '', type: 'revoked' }));
+ expect(h.announced()).toEqual(['m1']);
+ expect(h.hookStarts).toEqual(['m1']);
+ } else {
+ expect(t2).toEqual(expect.objectContaining({ body: 'corrected' }));
+ expect(h.announced()).toEqual(['m1', 't2']);
+ expect(h.hookStarts).toEqual(['m1', 't2']);
+ expect(h.hookedBodies.t2).toBe('corrected');
+ }
+ },
+ );
+
+ it('keeps the slot of a timed-out page download until the page settles', async () => {
+ process.env.MEDIA_DOWNLOAD_TIMEOUT_MS = '100';
+ // The adapter's limiter gets a spare slot, so only the room (one slot) can hold the second download.
+ process.env.INBOUND_MEDIA_CONCURRENCY = '2';
+ const h = heldProjector();
+ const { client } = build(h.callbacks);
+ process.env.INBOUND_MEDIA_CONCURRENCY = '1';
+ const page = deferred<{ mimetype: string; data: string }>();
+ const second = jest.fn().mockResolvedValue({ mimetype: 'image/png', data: 'QUJD' });
+ client.emit(
+ 'message',
+ incoming(
+ 'slow',
+ jest.fn(() => page.promise),
+ ),
+ );
+ client.emit('message', incoming('next', '[email protected]', second));
+ await sleep(150);
+ // Emitted without media at its deadline and stored, but the page still works on the payload.
+ expect(h.inserted()).toEqual(['slow']);
+ expect(second).not.toHaveBeenCalled();
+
+ page.resolve({ mimetype: 'image/png', data: 'QUJD' });
+ await flush();
+ expect(second).toHaveBeenCalledTimes(1);
+ });
+
+ it('emits the declared-only marker without a page download when no room comes free in time', async () => {
+ process.env.MEDIA_DOWNLOAD_TIMEOUT_MS = '20';
+ const h = heldProjector();
+ h.stall('held');
+ const seen: IncomingMessage[] = [];
+ const onMessage = h.callbacks.onMessage!;
+ const { client } = build({ ...h.callbacks, onMessage: (m, t) => (seen.push(m), onMessage(m, t)) });
+ const late = jest.fn();
+ client.emit(
+ 'message',
+ incoming('held', '[email protected]', jest.fn().mockResolvedValue({ mimetype: 'image/png', data: 'QUJD' })),
+ );
+ client.emit('message', incoming('late', '[email protected]', late));
+ await sleep(80);
+ await flush();
+
+ expect(late).not.toHaveBeenCalled();
+ expect(seen.find(m => m.id === 'late')?.media).toEqual(expect.objectContaining({ omitted: true, sizeBytes: 3 }));
+ });
+
+ it('downloads nothing for media revoked while it waits for room, and stores it cleared', async () => {
+ const h = heldProjector();
+ const release = h.stall('held');
+ const { client } = build(h.callbacks);
+ const page = jest.fn().mockResolvedValue({ mimetype: 'image/png', data: 'QUJD' });
+ client.emit(
+ 'message',
+ incoming('held', '[email protected]', jest.fn().mockResolvedValue({ mimetype: 'image/png', data: 'QUJD' })),
+ );
+ client.emit('message', incoming('gone', '[email protected]', page));
+ await flush();
+ client.emit(
+ 'message_revoke_everyone',
+ { id: { _serialized: 'rev' }, from: '[email protected]', to: '[email protected]', fromMe: false, timestamp: 2 },
+ { id: { _serialized: 'gone' } },
+ );
+ await flush();
+
+ expect(page).not.toHaveBeenCalled();
+ expect(h.rows).toEqual([expect.objectContaining({ waMessageId: 'gone', body: '', type: 'revoked' })]);
+ release();
+ await flush();
+ expect(h.announced()).toEqual(['held']);
+ });
+
+ it('downloads nothing for media granted room but revoked while its contact lookup is pending', async () => {
+ process.env.INBOUND_MEDIA_CONCURRENCY = '2';
+ const h = heldProjector();
+ const { client } = build(h.callbacks);
+ const contact = deferred<null>();
+ const page = jest.fn().mockResolvedValue({ mimetype: 'image/png', data: 'QUJD' });
+ client.emit('message', { ...incoming('gone', '[email protected]', page), getContact: jest.fn(() => contact.promise) });
+ await flush();
+ client.emit(
+ 'message_revoke_everyone',
+ { id: { _serialized: 'rev' }, from: '[email protected]', to: '[email protected]', fromMe: false, timestamp: 2 },
+ { id: { _serialized: 'gone' } },
+ );
+ await flush();
+ contact.resolve(null);
+ await flush();
+
+ expect(page).not.toHaveBeenCalled();
+ expect(h.rows).toEqual([expect.objectContaining({ waMessageId: 'gone', body: '', type: 'revoked' })]);
+ expect(h.announced()).toEqual([]);
+ });
+
+ it('bounds the page lookups of a held message, so its chat does not wait on a hung one', async () => {
+ process.env.MEDIA_DOWNLOAD_TIMEOUT_MS = '20';
+ const h = heldProjector();
+ const { client } = build(h.callbacks);
+ const hung = {
+ ...incoming('hung', '[email protected]'),
+ getContact: jest.fn(() => new Promise(() => undefined)),
+ hasQuotedMsg: true,
+ getQuotedMessage: jest.fn(() => new Promise(() => undefined)),
+ };
+ client.emit('message', hung);
+ client.emit('message', incoming('next', '[email protected]', undefined, 'after'));
+ await sleep(80);
+ await flush();
+
+ expect(h.inserted()).toEqual(['hung', 'next']);
+ });
+
+ it('still emits a message whose admission throws, without downloading its media', async () => {
+ const h = heldProjector();
+ const seen: IncomingMessage[] = [];
+ const onMessage = h.callbacks.onMessage!;
+ const { client } = build({
+ ...h.callbacks,
+ admitInbound: () => {
+ throw new Error('boom');
+ },
+ onMessage: (m, t) => (seen.push(m), onMessage(m, t)),
+ });
+ const page = jest.fn().mockResolvedValue({ mimetype: 'image/png', data: 'QUJD' });
+ client.emit('message', incoming('a', '[email protected]', page));
+ await flush();
+
+ expect(page).not.toHaveBeenCalled();
+ expect(seen[0]?.media).toEqual(expect.objectContaining({ omitted: true, sizeBytes: 3 }));
+ expect(h.inserted()).toEqual(['a']);
+ });
+
+ it('gives the status seed and history no ticket: they download outside the room', async () => {
+ const h = heldProjector();
+ h.stall('held');
+ const admitInbound = jest.fn(h.callbacks.admitInbound);
+ const { adapter, client } = build({ ...h.callbacks, admitInbound });
+ client.emit(
+ 'message',
+ incoming('held', '[email protected]', jest.fn().mockResolvedValue({ mimetype: 'image/png', data: 'QUJD' })),
+ );
+ await flush();
+ admitInbound.mockClear();
+
+ const page = jest.fn().mockResolvedValue({ mimetype: 'image/png', data: 'QUJD' });
+ const capped = await (
+ adapter as unknown as { capInboundMediaFor: (msg: unknown, max?: number) => Promise<unknown> }
+ ).capInboundMediaFor(incoming('seed', 'status@broadcast', page), 1000);
+ expect(capped).toEqual(expect.objectContaining({ data: 'QUJD' }));
+ expect(admitInbound).not.toHaveBeenCalled();
+
+ // Only the live message events take tickets; the seed and history callers pass none.
+ const callers = readdirSync(__dirname)
+ .filter(f => f.startsWith('wwebjs-') && f.endsWith('.ts') && !f.endsWith('.spec.ts'))
+ .filter(f => readFileSync(join(__dirname, f), 'utf8').includes('admitInbound'));
+ expect(callers).toEqual(['wwebjs-message-events.ts']);
+ });
+});
diff --git a/src/engine/adapters/inbound-media-cap.ts b/src/engine/adapters/inbound-media-cap.ts
index c22da3c1a..c7904677e 100644
--- a/src/engine/adapters/inbound-media-cap.ts
+++ b/src/engine/adapters/inbound-media-cap.ts
@@ -1,4 +1,4 @@
-import type { IncomingMessage } from '../interfaces/whatsapp-engine.interface';
+import type { InboundTicket, IncomingMessage } from '../interfaces/whatsapp-engine.interface';
import { ConcurrencyLimiter } from '../../common/utils/concurrency-limiter';
/** Default inbound media cap: 50 MiB. Shares MEDIA_DOWNLOAD_MAX_BYTES with the outbound download cap. */
@@ -197,3 +197,27 @@ export function capInboundMedia(args: {
}
return { mimetype: args.mimetype, filename: args.filename, data: args.toBase64() };
}
+
+const NO_ROOM: InboundTicket = {
+ reserve: () => false,
+ holdUntil: () => undefined,
+ drop: () => undefined,
+ revoked: false,
+};
+
+/**
+ * Admit a live message (EngineEventCallbacks.admitInbound) without letting a throw escape the event
+ * handler. A message whose admission failed still goes out, but holds no room, so its media takes the
+ * omitted marker instead of a download nothing bounds.
+ */
+export function admitInboundSafely(
+ admit: () => InboundTicket | undefined,
+ onError: (error: unknown) => void,
+): InboundTicket | undefined {
+ try {
+ return admit();
+ } catch (error) {
+ onError(error);
+ return NO_ROOM;
+ }
+}
diff --git a/src/engine/adapters/whatsapp-web-js.adapter.ts b/src/engine/adapters/whatsapp-web-js.adapter.ts
index a8ca84699..a9af2c6d8 100644
--- a/src/engine/adapters/whatsapp-web-js.adapter.ts
+++ b/src/engine/adapters/whatsapp-web-js.adapter.ts
@@ -8,6 +8,7 @@ import {
MessageResult,
MediaInput,
IncomingMessage,
+ InboundTicket,
Contact,
Group,
GroupInfo,
@@ -44,7 +45,13 @@ import { ChannelMediaNotSupportedError } from '../../common/errors/channel-media
import { WwebjsGroups } from './wwebjs-groups';
import { type WwebjsEngineHost } from './wwebjs-host';
import { registerWwebjsMessageEvents } from './wwebjs-message-events';
-import { WwebjsMessaging, declaredOnlyMedia, downloadCappedMedia, type PageMediaDownload } from './wwebjs-messaging';
+import {
+ WwebjsMessaging,
+ declaredMediaSize,
+ declaredOnlyMedia,
+ downloadCappedMedia,
+ type PageMediaDownload,
+} from './wwebjs-messaging';
import { WwebjsContacts } from './wwebjs-contacts';
import { WwebjsProfile } from './wwebjs-profile';
import { WwebjsLabels } from './wwebjs-labels';
@@ -60,7 +67,6 @@ import { WwebjsStuckAuth } from './wwebjs-stuck-auth';
import { WwebjsCalls } from './wwebjs-calls';
import {
capInboundMedia,
- coerceDeclaredSize,
inboundMediaConcurrency,
inboundMediaMaxBytes,
inboundMediaTimeoutMs,
@@ -214,7 +220,7 @@ export class WhatsAppWebJsAdapter extends EventEmitter implements IWhatsAppEngin
reportIfPageTransportError: (error, context) => this.reportIfPageTransportError(error, context),
ensureNotChannelRecipient: chatId => this.ensureNotChannelRecipient(chatId),
getNumberId: number => this.getNumberId(number),
- capInboundMediaFor: (msg, maxBytesOverride) => this.capInboundMediaFor(msg, maxBytesOverride),
+ capInboundMediaFor: (msg, maxBytesOverride, ticket) => this.capInboundMediaFor(msg, maxBytesOverride, ticket),
config: this.config,
getCallbacks: () => this.callbacks,
};
@@ -285,10 +291,13 @@ export class WhatsAppWebJsAdapter extends EventEmitter implements IWhatsAppEngin
* (2) let the page drop a decrypted payload over the cap before encoding it (downloadMediaInPage),
* and (3) run the download through the concurrency limiter for backpressure. Resolves an envelope
* whenever it has one to build: the payload, or the declared-only marker when no blob is available.
+ * With a ticket (PostgreSQL), the download waits for a slot of the session's room first, and that
+ * slot is held until the page download settles, since it cannot be aborted.
*/
private async capInboundMediaFor(
msg: Message,
maxBytesOverride?: number,
+ ticket?: InboundTicket,
): Promise<IncomingMessage['media'] | undefined> {
if (!isMediaDownloadEnabled()) {
return declaredOnlyMedia(msg);
@@ -300,8 +309,7 @@ export class WhatsAppWebJsAdapter extends EventEmitter implements IWhatsAppEngin
// An override (the status seed's STATUS_MEDIA_MAX_BYTES) only tightens the global cap: a larger one
// would let a download the cap below always drops run anyway, past the memory guard it exists for.
const maxBytes = Math.min(maxBytesOverride ?? Number.POSITIVE_INFINITY, inboundMediaMaxBytes());
- const data = (msg as unknown as { _data?: { size?: number; mimetype?: string; filename?: string } })._data;
- const declared = coerceDeclaredSize(data?.size);
+ const declared = declaredMediaSize(msg);
if (declared > maxBytes) {
this.logger.warn('Inbound media declared size exceeds the cap; skipped download', {
msgId,
@@ -310,6 +318,13 @@ export class WhatsAppWebJsAdapter extends EventEmitter implements IWhatsAppEngin
});
return declaredOnlyMedia(msg);
}
+ if (ticket) {
+ const granted = ticket.reserve();
+ if (!(granted === true || (granted !== false && (await granted)))) {
+ this.logger.warn('No room for inbound media within the wait; emitting the omitted marker', { msgId });
+ return declaredOnlyMedia(msg);
+ }
+ }
// The page download can't be aborted, so freeing the slot the moment the wall-clock deadline fires
// would admit a fresh download while the abandoned one is still materialising in heap — letting the
// number of in-flight downloads exceed inboundMediaConcurrency(). Instead, HOLD the slot until the real
@@ -362,6 +377,8 @@ export class WhatsAppWebJsAdapter extends EventEmitter implements IWhatsAppEngin
// Per-session slot first, then the process-wide one, so a session parks at most its own
// INBOUND_MEDIA_CONCURRENCY waiters on the shared gate. Both are held until the download settles.
const slotHeld = this.inboundLimiter.run(() => runUnderGlobalMediaGate(downloadInSlot));
+ // The payload can reach Node after the wait below gave up on it: keep the room until it settles.
+ ticket?.holdUntil(slotHeld);
// Defensive only, and deliberately kept. `run()` rejects on a full queue (gone — the queue is
// unbounded) or on close(), which nothing calls on this limiter; the task itself swallows both
// download outcomes. So nothing is expected here — but an unhandled rejection from a
diff --git a/src/engine/adapters/wwebjs-host.ts b/src/engine/adapters/wwebjs-host.ts
index 1df9d265f..a67e3e690 100644
--- a/src/engine/adapters/wwebjs-host.ts
+++ b/src/engine/adapters/wwebjs-host.ts
@@ -1,5 +1,9 @@
import { type Client, type Message } from 'whatsapp-web.js';
-import { type EngineEventCallbacks, type IncomingMessage } from '../interfaces/whatsapp-engine.interface';
+import {
+ type EngineEventCallbacks,
+ type InboundTicket,
+ type IncomingMessage,
+} from '../interfaces/whatsapp-engine.interface';
import { type createLogger } from '../../common/services/logger.service';
import { EngineTransportError } from '../../common/errors/engine-transport.error';
import { type WhatsAppWebJsConfig } from './whatsapp-web-js.adapter';
@@ -20,7 +24,11 @@ export interface WwebjsEngineHost {
reportIfPageTransportError(error: unknown, context: string): void;
ensureNotChannelRecipient(chatId: string): void;
getNumberId(number: string): Promise<string | null>;
- capInboundMediaFor(msg: Message, maxBytesOverride?: number): Promise<IncomingMessage['media'] | undefined>;
+ capInboundMediaFor(
+ msg: Message,
+ maxBytesOverride?: number,
+ ticket?: InboundTicket,
+ ): Promise<IncomingMessage['media'] | undefined>;
readonly config: WhatsAppWebJsConfig;
/** Live callbacks bag — read per event, since initialize() installs it after delegates are built. */
getCallbacks(): EngineEventCallbacks;
diff --git a/src/engine/adapters/wwebjs-message-events.ts b/src/engine/adapters/wwebjs-message-events.ts
index 8e58dd2bb..464e4c0a4 100644
--- a/src/engine/adapters/wwebjs-message-events.ts
+++ b/src/engine/adapters/wwebjs-message-events.ts
@@ -1,5 +1,6 @@
-import { type Client, MessageTypes } from 'whatsapp-web.js';
+import { type Client, type Message, MessageTypes } from 'whatsapp-web.js';
import {
+ type InboundTicket,
type IncomingMessage,
type RevokedMessage,
type ReactionEvent,
@@ -7,7 +8,8 @@ import {
} from '../interfaces/whatsapp-engine.interface';
import { type SerializedWid } from '../types/whatsapp-web-js.types';
import { buildEditedMessage, buildIncomingMessageBase, mapContactFields } from './message-mapper';
-import { extractWwebjsCall, wwebjsAckToDeliveryStatus } from './wwebjs-messaging';
+import { extractWwebjsCall, plannedInboundDownload, wwebjsAckToDeliveryStatus } from './wwebjs-messaging';
+import { admitInboundSafely, inboundMediaTimeoutMs, withInboundDownloadTimeout } from './inbound-media-cap';
import { type WwebjsEngineHost } from './wwebjs-host';
/**
@@ -18,73 +20,42 @@ import { type WwebjsEngineHost } from './wwebjs-host';
* latches this file never touches.
*/
export function registerWwebjsMessageEvents(client: Client, host: WwebjsEngineHost): void {
- // eslint-disable-next-line @typescript-eslint/no-misused-promises
- client.on('message', async msg => {
- try {
- const incomingMessage: IncomingMessage = buildIncomingMessageBase(msg);
-
- // Attach the sender's contact info. getContact() gives the real sender (author in groups, from
- // in 1:1); we read only its synchronous fields and never the async getters (profile pic, about),
- // which would hit WhatsApp on every message.
- try {
- const contact = await msg.getContact();
- if (contact) {
- // Off by default the payload keeps { name, pushName }; WEBHOOK_CONTACT_DETAILS opts into the
- // full set. Merge over the base so the notifyName pushName isn't lost, and skip an empty
- // result so we don't emit an empty contact object.
- const full = process.env.WEBHOOK_CONTACT_DETAILS === 'true';
- const merged = { ...incomingMessage.contact, ...mapContactFields(contact, full) };
- if (Object.keys(merged).length > 0) {
- incomingMessage.contact = merged;
- }
- }
- } catch (error) {
- host.logger.error('Error getting message contact', String(error));
- }
-
- // Handle location
- if (msg.type === MessageTypes.LOCATION && msg.location) {
- incomingMessage.location = {
- latitude: Number(msg.location.latitude),
- longitude: Number(msg.location.longitude),
- description: msg.location.description || undefined,
- address: msg.location.address || undefined,
- url: msg.location.url || undefined,
- };
- }
+ // Where the session holds a message until its row is written (PostgreSQL), each arrival is admitted
+ // synchronously, before anything awaits, so its chat's order and its room follow arrival. The
+ // ticket keeps the room until the processing settles, and is dropped when nothing was emitted.
+ const admit = (msg: Message, base: IncomingMessage, mayDownload: boolean): InboundTicket | undefined =>
+ admitInboundSafely(
+ () =>
+ host.getCallbacks().admitInbound?.({
+ id: base.id,
+ chatId: base.chatId,
+ status: Boolean(base.isStatusBroadcast),
+ needsRoom: mayDownload && plannedInboundDownload(msg),
+ }),
+ error =>
+ host.logger.error(
+ `Could not admit inbound message ${base.id}; processing it without room for media`,
+ String(error),
+ ),
+ );
+ const holdWhileProcessing = (ticket: InboundTicket | undefined, done: Promise<void>): void => {
+ ticket?.holdUntil(done);
+ void done.then(
+ () => ticket?.drop(),
+ () => ticket?.drop(),
+ );
+ };
- // Handle media
- if (msg.hasMedia) {
- try {
- const capped = await host.capInboundMediaFor(msg);
- if (capped) incomingMessage.media = capped;
- } catch (error) {
- host.logger.error('Error downloading media', String(error));
- }
- }
-
- // Handle quoted message
- if (msg.hasQuotedMsg) {
- try {
- const quoted = await msg.getQuotedMessage();
- incomingMessage.quotedMessage = {
- id: quoted.id._serialized,
- body: quoted.body,
- };
- } catch (error) {
- host.logger.error('Error getting quoted message', String(error));
- }
- }
-
- // Surface call-log detail on the live path too (getChatHistory already does this), so a missed/
- // video incoming call renders a labeled bubble instead of a generic "Call".
- const call = extractWwebjsCall(msg);
- if (call) incomingMessage.call = call;
-
- host.getCallbacks().onMessage?.(incomingMessage);
+ client.on('message', msg => {
+ let base: IncomingMessage;
+ try {
+ base = buildIncomingMessageBase(msg);
} catch (error) {
host.logger.error('Error processing incoming message', String(error));
+ return;
}
+ const ticket = admit(msg, base, true);
+ holdWhileProcessing(ticket, handleIncoming(host, msg, base, ticket));
});
client.on('message_create', msg => {
@@ -96,15 +67,18 @@ export function registerWwebjsMessageEvents(client: Client, host: WwebjsEngineHo
return;
}
- void (async () => {
+ let ticket: InboundTicket | undefined;
+ const done = (async () => {
const incomingMessage = buildIncomingMessageBase(msg);
+ // Synchronous, as the first await comes after it.
+ ticket = admit(msg, incomingMessage, !incomingMessage.isStatusBroadcast);
// Enrich with the media payload through the same capped path the incoming handler uses —
// the base builder is sync and carries none, so a phone-sent image would otherwise persist
// and render as a bare 📎 marker even though the media is downloadable right here. Not for an own
// status post: the echo consumer drops those, so the download would only hold a limiter slot.
if (msg.hasMedia && !incomingMessage.isStatusBroadcast) {
try {
- incomingMessage.media = await host.capInboundMediaFor(msg);
+ incomingMessage.media = await host.capInboundMediaFor(msg, undefined, ticket);
} catch (error) {
host.logger.warn('Own-send media download failed; emitting echo without media', {
msgId: msg.id?._serialized,
@@ -113,11 +87,14 @@ export function registerWwebjsMessageEvents(client: Client, host: WwebjsEngineHo
}
}
try {
- host.getCallbacks().onMessageCreate?.(incomingMessage);
+ // The ticket goes along only where one was issued, so elsewhere the call is unchanged.
+ if (ticket) host.getCallbacks().onMessageCreate?.(incomingMessage, ticket);
+ else host.getCallbacks().onMessageCreate?.(incomingMessage);
} catch (error) {
host.logger.error('Error processing outgoing message', String(error));
}
})();
+ holdWhileProcessing(ticket, done);
});
client.on('message_ack', (msg, ack) => {
@@ -227,3 +204,91 @@ export function registerWwebjsMessageEvents(client: Client, host: WwebjsEngineHo
}
});
}
+
+/**
+ * The `message` event body: enrich the incoming message (contact, location, media, quote, call) and
+ * emit it with its ticket. Never rejects.
+ */
+async function handleIncoming(
+ host: WwebjsEngineHost,
+ msg: Message,
+ incomingMessage: IncomingMessage,
+ ticket: InboundTicket | undefined,
+): Promise<void> {
+ // A held message (a ticket) keeps every later message of its chat waiting until it is emitted, so its
+ // page lookups get the download deadline instead of Puppeteer's protocol timeout, and go without.
+ const lookup = <T>(what: string, call: Promise<T>): Promise<T | null> =>
+ ticket
+ ? withInboundDownloadTimeout(call, inboundMediaTimeoutMs(), () =>
+ host.logger.warn(`Getting the ${what} passed MEDIA_DOWNLOAD_TIMEOUT_MS; emitting without it`, {
+ msgId: incomingMessage.id,
+ }),
+ )
+ : call;
+ try {
+ // Attach the sender's contact info. getContact() gives the real sender (author in groups, from
+ // in 1:1); we read only its synchronous fields and never the async getters (profile pic, about),
+ // which would hit WhatsApp on every message.
+ try {
+ const contact = await lookup('message contact', msg.getContact());
+ if (contact) {
+ // Off by default the payload keeps { name, pushName }; WEBHOOK_CONTACT_DETAILS opts into the
+ // full set. Merge over the base so the notifyName pushName isn't lost, and skip an empty
+ // result so we don't emit an empty contact object.
+ const full = process.env.WEBHOOK_CONTACT_DETAILS === 'true';
+ const merged = { ...incomingMessage.contact, ...mapContactFields(contact, full) };
+ if (Object.keys(merged).length > 0) {
+ incomingMessage.contact = merged;
+ }
+ }
+ } catch (error) {
+ host.logger.error('Error getting message contact', String(error));
+ }
+
+ // Handle location
+ if (msg.type === MessageTypes.LOCATION && msg.location) {
+ incomingMessage.location = {
+ latitude: Number(msg.location.latitude),
+ longitude: Number(msg.location.longitude),
+ description: msg.location.description || undefined,
+ address: msg.location.address || undefined,
+ url: msg.location.url || undefined,
+ };
+ }
+
+ // Handle media
+ if (msg.hasMedia) {
+ try {
+ const capped = await host.capInboundMediaFor(msg, undefined, ticket);
+ if (capped) incomingMessage.media = capped;
+ } catch (error) {
+ host.logger.error('Error downloading media', String(error));
+ }
+ }
+
+ // Handle quoted message
+ if (msg.hasQuotedMsg) {
+ try {
+ const quoted = await lookup('quoted message', msg.getQuotedMessage());
+ if (quoted) {
+ incomingMessage.quotedMessage = {
+ id: quoted.id._serialized,
+ body: quoted.body,
+ };
+ }
+ } catch (error) {
+ host.logger.error('Error getting quoted message', String(error));
+ }
+ }
+
+ // Surface call-log detail on the live path too (getChatHistory already does this), so a missed/
+ // video incoming call renders a labeled bubble instead of a generic "Call".
+ const call = extractWwebjsCall(msg);
+ if (call) incomingMessage.call = call;
+
+ if (ticket) host.getCallbacks().onMessage?.(incomingMessage, ticket);
+ else host.getCallbacks().onMessage?.(incomingMessage);
+ } catch (error) {
+ host.logger.error('Error processing incoming message', String(error));
+ }
+}
diff --git a/src/engine/adapters/wwebjs-messaging.ts b/src/engine/adapters/wwebjs-messaging.ts
index ff1b5dfe9..92976bc1d 100644
--- a/src/engine/adapters/wwebjs-messaging.ts
+++ b/src/engine/adapters/wwebjs-messaging.ts
@@ -23,6 +23,7 @@ import {
coerceDeclaredSize,
inboundMediaMaxBytes,
ingestMediaBudgetBytes,
+ isMediaDownloadEnabled,
} from './inbound-media-cap';
import { buildIncomingMessageBase, mapContactFields } from './message-mapper';
import { buildVCard } from './vcard';
@@ -72,6 +73,19 @@ export function declaredOnlyMedia(msg: Message): IncomingMessage['media'] {
};
}
+/** The media size the sender declared; 0 when unknown, which does not pre-gate the download. */
+export function declaredMediaSize(msg: Message): number {
+ return coerceDeclaredSize((msg as unknown as { _data?: { size?: unknown } })._data?.size);
+}
+
+/**
+ * Whether the adapter's capInboundMediaFor, with no override, downloads this message's media: its
+ * pre-gates, read as the message arrives so the download can be given room first.
+ */
+export function plannedInboundDownload(msg: Message): boolean {
+ return Boolean(msg.hasMedia) && isMediaDownloadEnabled() && declaredMediaSize(msg) <= inboundMediaMaxBytes();
+}
+
/** A media download as {@link downloadMediaInPage} returns it: `data` is empty when the payload was over the cap. */
export interface PageMediaDownload {
mimetype: string;
diff --git a/src/engine/engine-registry.service.spec.ts b/src/engine/engine-registry.service.spec.ts
index 2aab48ef3..4cb44e114 100644
--- a/src/engine/engine-registry.service.spec.ts
+++ b/src/engine/engine-registry.service.spec.ts
@@ -207,4 +207,48 @@ describe('EngineRegistry', () => {
expect(registry.activeIds().sort()).toEqual(['running', 'starting']);
});
});
+
+ describe('onRetire', () => {
+ // The projector releases what a retired engine's messages hold from here, so every path that
+ // ends an engine's life must report it, and only those.
+ const retired = (): Array<[string, unknown]> => {
+ const seen: Array<[string, unknown]> = [];
+ registry.onRetire((id, engine) => seen.push([id, (engine as unknown as { tag: string }).tag]));
+ return seen;
+ };
+
+ it('fires for delete, deleteIfLive and clear, after the map changed', () => {
+ const seen = retired();
+ let liveDuringListener: boolean | undefined;
+ const a = engineStub('a');
+ registry.set('s1', a);
+ registry.onRetire(id => (liveDuringListener = registry.has(id)));
+ registry.delete('s1');
+ registry.set('s2', engineStub('b'));
+ registry.deleteIfLive('s2', engineStub('stale'));
+ registry.deleteIfLive('s2', registry.get('s2')!);
+ registry.set('s3', engineStub('c'));
+ registry.set('s4', engineStub('d'));
+ registry.clear();
+ registry.delete('missing');
+
+ expect(seen).toEqual([
+ ['s1', 'a'],
+ ['s2', 'b'],
+ ['s3', 'c'],
+ ['s4', 'd'],
+ ]);
+ expect(liveDuringListener).toBe(false);
+ });
+
+ it('fires for a set that replaces a different engine, not for one that re-registers the same', () => {
+ const seen = retired();
+ const first = engineStub('first');
+ registry.set('s1', first);
+ registry.set('s1', first);
+ registry.set('s1', engineStub('second'));
+
+ expect(seen).toEqual([['s1', 'first']]);
+ });
+ });
});
diff --git a/src/engine/engine-registry.service.ts b/src/engine/engine-registry.service.ts
index 30f1d82e7..50200095d 100644
--- a/src/engine/engine-registry.service.ts
+++ b/src/engine/engine-registry.service.ts
@@ -43,6 +43,8 @@ export class EngineRegistry {
*/
readonly initializing = new Set<string>();
+ private readonly retireListeners: Array<(id: string, engine: IWhatsAppEngine) => void> = [];
+
// ── Map-compatible surface (used by the lifecycle owner) ──────────────
get(id: string): IWhatsAppEngine | undefined {
@@ -56,8 +58,10 @@ export class EngineRegistry {
* It stays optional only because the specs that register a bare engine stub do not care.
*/
set(id: string, engine: IWhatsAppEngine, proxyUrl?: string): void {
+ const previous = this.engines.get(id);
this.engines.set(id, engine);
this.proxies.set(id, proxyUrl);
+ if (previous && previous !== engine) this.retired(id, previous);
}
has(id: string): boolean {
@@ -65,13 +69,30 @@ export class EngineRegistry {
}
delete(id: string): boolean {
+ const previous = this.engines.get(id);
this.proxies.delete(id);
- return this.engines.delete(id);
+ const deleted = this.engines.delete(id);
+ if (previous) this.retired(id, previous);
+ return deleted;
}
clear(): void {
+ const previous = this.entries();
this.engines.clear();
this.proxies.clear();
+ for (const [id, engine] of previous) this.retired(id, engine);
+ }
+
+ /**
+ * Run `listener` each time an engine stops being the live one for its session (replaced, deleted,
+ * or cleared at shutdown), after the map changed, so `isLive` already reports it stale.
+ */
+ onRetire(listener: (id: string, engine: IWhatsAppEngine) => void): void {
+ this.retireListeners.push(listener);
+ }
+
+ private retired(id: string, engine: IWhatsAppEngine): void {
+ for (const listener of this.retireListeners) listener(id, engine);
}
get size(): number {
diff --git a/src/engine/index.ts b/src/engine/index.ts
index 4d71af89e..e4c656827 100644
--- a/src/engine/index.ts
+++ b/src/engine/index.ts
@@ -1,4 +1,4 @@
// Engine public surface for code outside src/engine: re-exported here so modules never reach into
// adapters/ internals. Intentionally narrow — add a symbol only when an outside consumer needs it.
export { BaileysStoredMessage } from './adapters/baileys-stored-message.entity';
-export { inboundMediaMaxBytes } from './adapters/inbound-media-cap';
+export { inboundMediaConcurrency, inboundMediaMaxBytes, inboundMediaTimeoutMs } from './adapters/inbound-media-cap';
diff --git a/src/engine/interfaces/whatsapp-engine.interface.ts b/src/engine/interfaces/whatsapp-engine.interface.ts
index 0010dab68..a4fd4a257 100644
--- a/src/engine/interfaces/whatsapp-engine.interface.ts
+++ b/src/engine/interfaces/whatsapp-engine.interface.ts
@@ -796,15 +796,49 @@ export interface CallOutcomeEvent {
timestamp: number;
}
+/**
+ * A message's place in its session from the moment it arrives until its row is written (see
+ * `EngineEventCallbacks.admitInbound`). The engine hands it back with the message it emits.
+ */
+export interface InboundTicket {
+ /**
+ * Room for one downloaded payload: true once granted, false when the message asked for none, or
+ * when no room came free in time (the media then goes out with the omitted marker, undownloaded).
+ * Call it right before the download, never earlier: the grant is held until the row is written.
+ */
+ reserve: () => boolean | Promise<boolean>;
+ /** Keep the room until `p` settles too: work on the payload that may outlive the emit. */
+ holdUntil: (p: Promise<unknown>) => void;
+ /** The message will not be emitted: release its turn and its room. A no-op after the emit. */
+ drop: () => void;
+ /**
+ * Whether the session recorded a delete for everyone of this message while holding it, so emitting
+ * it stores the row cleared and announces nothing. False for a revoke the session never saw.
+ */
+ readonly revoked: boolean;
+}
+
export interface EngineEventCallbacks {
onQRCode?: (qr: string) => void;
onReady?: (phone: string, pushName: string) => void;
- onMessage?: (message: IncomingMessage) => void;
+ /**
+ * Set only where a downloaded payload outlives the emit (PostgreSQL writes asynchronously). The
+ * engine calls it synchronously as each live message arrives, in arrival order, and passes the
+ * ticket to the emit, or drops it when the message is not emitted. `needsRoom` says whether the
+ * message will download media; `status` marks a status post. Undefined when the engine is stale.
+ */
+ admitInbound?: (arrival: {
+ id: string;
+ chatId: string;
+ needsRoom: boolean;
+ status: boolean;
+ }) => InboundTicket | undefined;
+ onMessage?: (message: IncomingMessage, ticket?: InboundTicket) => void;
/**
* Fired for messages the account itself created (outgoing) — including sends composed on a
* linked phone, which the `message`/`onMessage` event never delivers. Used to emit `message.sent`.
*/
- onMessageCreate?: (message: IncomingMessage) => void;
+ onMessageCreate?: (message: IncomingMessage, ticket?: InboundTicket) => void;
/**
* Fired when the delivery status of an outgoing message advances. The adapter maps its native
* delivery signal to the neutral `DeliveryStatus`, so consumers never see engine-specific codes.
diff --git a/src/modules/session/inbound-room.spec.ts b/src/modules/session/inbound-room.spec.ts
new file mode 100644
index 000000000..6e465a18b
--- /dev/null
+++ b/src/modules/session/inbound-room.spec.ts
@@ -0,0 +1,117 @@
+import { InboundRoom, deferred, roomState, type RoomState } from './inbound-room';
+
+const flush = async (): Promise<void> => {
+ for (let i = 0; i < 20; i++) await Promise.resolve();
+};
+
+describe('InboundRoom', () => {
+ const env = { ...process.env };
+
+ beforeEach(() => {
+ process.env.INBOUND_MEDIA_CONCURRENCY = '2';
+ process.env.MEDIA_DOWNLOAD_TIMEOUT_MS = '1000';
+ });
+
+ afterEach(() => {
+ process.env = { ...env };
+ jest.useRealTimers();
+ });
+
+ const requested = (room: InboundRoom, n: number): RoomState[] =>
+ Array.from({ length: n }, () => {
+ const s = roomState();
+ void room.request(s);
+ return s;
+ });
+
+ it('grants free slots synchronously and the rest in request order', async () => {
+ const room = new InboundRoom();
+ const [a, b, c, d] = requested(room, 4);
+
+ expect([a.granted, b.granted, c.granted, d.granted]).toEqual([true, true, false, false]);
+
+ b.done.resolve();
+ await flush();
+ expect([c.granted, d.granted]).toEqual([true, false]);
+
+ a.done.resolve();
+ await flush();
+ expect(d.granted).toBe(true);
+ });
+
+ it('holds a slot until done and every hold settle, including a hold pushed after the grant', async () => {
+ process.env.INBOUND_MEDIA_CONCURRENCY = '1';
+ const room = new InboundRoom();
+ const [a, b] = requested(room, 2);
+ const engine = deferred<void>();
+ const page = deferred<void>();
+ a.holds.push(engine.promise);
+
+ a.done.resolve();
+ await flush();
+ expect(b.granted).toBe(false);
+
+ // Pushed while the earlier hold still runs (a page download outliving the emit).
+ a.holds.push(page.promise);
+ engine.resolve();
+ await flush();
+ expect(b.granted).toBe(false);
+
+ page.resolve();
+ await flush();
+ expect(b.granted).toBe(true);
+ });
+
+ it('keeps a waiter waiting while the room makes progress, and gives up only once it stalls', async () => {
+ jest.useFakeTimers();
+ process.env.INBOUND_MEDIA_CONCURRENCY = '1';
+ const room = new InboundRoom();
+ const holders = requested(room, 3);
+ const waiter = roomState();
+ void room.request(waiter);
+ let answer: boolean | undefined;
+ void room.wait(waiter).then(v => (answer = v));
+
+ // The window is twice MEDIA_DOWNLOAD_TIMEOUT_MS; each release restarts it.
+ for (const holder of holders) {
+ await jest.advanceTimersByTimeAsync(1500);
+ holder.done.resolve();
+ await flush();
+ }
+ await flush();
+ expect(answer).toBe(true);
+
+ const stalled = roomState();
+ void room.request(stalled);
+ let late: boolean | undefined;
+ void room.wait(stalled).then(v => (late = v));
+ await jest.advanceTimersByTimeAsync(1999);
+ expect(late).toBeUndefined();
+ await jest.advanceTimersByTimeAsync(1);
+ expect(late).toBe(false);
+ expect(stalled.abandoned).toBe(true);
+ });
+
+ it('lets an abandoned request pass its slot straight on', async () => {
+ process.env.INBOUND_MEDIA_CONCURRENCY = '1';
+ const room = new InboundRoom();
+ const [a, b, c] = requested(room, 3);
+ b.abandoned = true;
+
+ a.done.resolve();
+ await flush();
+
+ expect(b.granted).toBe(false);
+ expect(c.granted).toBe(true);
+ expect(room.gate.activeCount).toBe(1);
+ });
+
+ it('is idle once every slot came back', async () => {
+ const room = new InboundRoom();
+ const [a] = requested(room, 1);
+ expect(room.idle).toBe(false);
+ a.done.resolve();
+ await flush();
+ expect(room.idle).toBe(true);
+ });
+});
diff --git a/src/modules/session/inbound-room.ts b/src/modules/session/inbound-room.ts
new file mode 100644
index 000000000..43ae409ce
--- /dev/null
+++ b/src/modules/session/inbound-room.ts
@@ -0,0 +1,87 @@
+import { ConcurrencyLimiter } from '../../common/utils/concurrency-limiter';
+import { inboundMediaConcurrency, inboundMediaTimeoutMs } from '../../engine';
+
+export interface Deferred<T> {
+ promise: Promise<T>;
+ resolve: (value: T) => void;
+}
+
+export function deferred<T>(): Deferred<T> {
+ let resolve!: (value: T) => void;
+ const promise = new Promise<T>(r => (resolve = r));
+ return { promise, resolve };
+}
+
+/**
+ * One message's claim on its session's {@link InboundRoom}. It holds no message, so a waiter parked
+ * in the room costs a few closures, never a payload.
+ */
+export interface RoomState {
+ granted: boolean;
+ /** Gave up before its grant: once admitted, it passes the slot straight on. */
+ abandoned: boolean;
+ grant: Deferred<boolean>;
+ /** Resolved once the message's row is written, or once it will not be emitted. */
+ done: Deferred<void>;
+ /** Work on the payload that may outlive `done` (engine processing, a page download, a status ingest). */
+ holds: Promise<unknown>[];
+}
+
+export function roomState(): RoomState {
+ return { granted: false, abandoned: false, grant: deferred(), done: deferred(), holds: [] };
+}
+
+/**
+ * Room for the downloaded payloads one session holds in memory before their rows are written:
+ * INBOUND_MEDIA_CONCURRENCY slots of one payload each, granted in arrival order. A slot is held until
+ * the row is written AND every hold settles, so a stalled insert keeps it, and the payloads held stay
+ * within the slots however long the store takes.
+ */
+export class InboundRoom {
+ readonly gate = new ConcurrencyLimiter(inboundMediaConcurrency());
+ private lastRelease = Date.now();
+
+ /** Queue `s` for a slot. Synchronous: a free slot is granted before this returns. */
+ request(s: RoomState): Promise<void> {
+ return this.gate
+ .run(async () => {
+ if (s.abandoned) return;
+ s.granted = true;
+ s.grant.resolve(true);
+ await s.done.promise;
+ // Read after `done`, and to the end of the array: a hold pushed while an earlier one runs counts too.
+ for (let i = 0; i < s.holds.length; i++) await s.holds[i];
+ })
+ .then(() => {
+ this.lastRelease = Date.now();
+ });
+ }
+
+ /**
+ * Wait for the grant. Gives up only when the room made no progress (no slot released) for twice
+ * MEDIA_DOWNLOAD_TIMEOUT_MS: a holder is legitimately busy for one download deadline plus its
+ * commit, so a burst behind a moving room keeps its media, and only a stalled room sheds.
+ */
+ wait(s: RoomState): Promise<boolean> {
+ if (s.granted) return Promise.resolve(true);
+ const window = 2 * inboundMediaTimeoutMs();
+ const asked = Date.now();
+ const arm = (delay: number): void => {
+ const timer = setTimeout(() => {
+ if (s.granted || s.abandoned) return;
+ const idle = Date.now() - Math.max(this.lastRelease, asked);
+ if (idle < window) return arm(window - idle);
+ s.abandoned = true;
+ s.grant.resolve(false);
+ }, delay);
+ timer.unref?.();
+ void s.grant.promise.then(() => clearTimeout(timer));
+ };
+ arm(window);
+ return s.grant.promise;
+ }
+
+ get idle(): boolean {
+ return this.gate.activeCount === 0 && this.gate.queuedCount === 0;
+ }
+}
diff --git a/src/modules/session/message-projector.inbound-hold.spec.ts b/src/modules/session/message-projector.inbound-hold.spec.ts
new file mode 100644
index 000000000..02107d151
--- /dev/null
+++ b/src/modules/session/message-projector.inbound-hold.spec.ts
@@ -0,0 +1,379 @@
+import type { Repository } from 'typeorm';
+import { MessageProjector } from './message-projector.service';
+import { EngineRegistry } from '../../engine/engine-registry.service';
+import { Message, MessageStatus } from '../message/entities/message.entity';
+import { Session } from './entities/session.entity';
+import { EventsGateway } from '../events/events.gateway';
+import { WebhookService } from '../webhook/webhook.service';
+import { HookManager } from '../../core/hooks';
+import { StatusStoreService } from '../status-store/status-store.service';
+import { SessionLidResolver } from './session-lid-resolver.service';
+import { deferred } from './inbound-room';
+import type {
+ IncomingMessage,
+ InboundTicket,
+ IWhatsAppEngine,
+} from '../../engine/interfaces/whatsapp-engine.interface';
+
+/**
+ * On PostgreSQL the engines hold each inbound message from its arrival until its row is written
+ * (admitInbound): the chat's hook and commit order is fixed at arrival, a payload is downloaded only
+ * into a slot of the session's room, and a revoke, edit, reaction or ack that reaches a message
+ * before its emit is kept for its row.
+ */
+
+type Row = Record<string, unknown>;
+
+const flush = async (): Promise<void> => {
+ for (let i = 0; i < 5; i++) await new Promise(resolve => setImmediate(resolve));
+};
+
+// TypeORM's FindOperator, read without importing its internals.
+const matches = (row: Row, where: Record<string, unknown>): boolean =>
+ Object.entries(where).every(([key, want]) => {
+ if (key === 'metadata') return true;
+ const op = want as { _type?: string; _value?: unknown } | null;
+ if (op && typeof op === 'object' && op._type === 'not') return row[key] !== op._value;
+ if (op && typeof op === 'object' && op._type === 'in') return (op._value as unknown[]).includes(row[key]);
+ return row[key] === want;
+ });
+
+const message = (id: string, chatId: string, over: Partial<IncomingMessage> = {}): IncomingMessage => ({
+ id,
+ chatId,
+ from: chatId,
+ body: `body of ${id}`,
+ type: 'text',
+ timestamp: 1_700_000_000,
+ fromMe: false,
+ isGroup: false,
+ kind: 'individual',
+ ...over,
+});
+
+const arrival = (id: string, chatId: string, needsRoom = false, status = false) => ({ id, chatId, needsRoom, status });
+
+describe('MessageProjector holding inbound messages (PostgreSQL)', () => {
+ const env = { ...process.env };
+ let rows: Row[];
+ let insertGate: (row: Row) => Promise<void>;
+ let repo: Record<string, unknown>;
+ let hookManager: { execute: jest.Mock };
+ let webhookService: { dispatch: jest.Mock };
+ let eventsGateway: Record<string, jest.Mock>;
+ let statusStore: { ingest: jest.Mock };
+ let engines: EngineRegistry;
+ let engine: IWhatsAppEngine;
+ let projector: MessageProjector;
+ let hookStarts: string[];
+
+ const build = (type: string): void => {
+ rows = [];
+ hookStarts = [];
+ insertGate = () => Promise.resolve();
+ repo = {
+ manager: { connection: { options: { type } } },
+ find: jest.fn().mockResolvedValue([]),
+ findOne: jest.fn(({ where }: { where: Record<string, unknown> }) =>
+ Promise.resolve(rows.find(r => matches(r, where)) ?? null),
+ ),
+ create: jest.fn((x: Row) => ({ ...x })),
+ insert: jest.fn(async (row: Row) => {
+ await insertGate(row);
+ rows.push({ ...row });
+ return { identifiers: [{ id: `pk-${String(row.waMessageId)}` }], generatedMaps: [] };
+ }),
+ update: jest.fn((where: Record<string, unknown>, patch: Row) => {
+ const hit = rows.filter(r => matches(r, where));
+ for (const r of hit) Object.assign(r, patch);
+ return Promise.resolve({ affected: hit.length });
+ }),
+ };
+ hookManager = {
+ execute: jest.fn((event: string, data: unknown) => {
+ if (event === 'message:received' || event === 'message:sent') hookStarts.push((data as IncomingMessage).id);
+ return Promise.resolve({ continue: true, data });
+ }),
+ };
+ webhookService = { dispatch: jest.fn().mockResolvedValue(undefined) };
+ eventsGateway = {
+ emitMessage: jest.fn(),
+ emitMessageSent: jest.fn(),
+ emitMessageRevoked: jest.fn(),
+ emitMessageReaction: jest.fn(),
+ emitMessageEdited: jest.fn(),
+ emitMessageAck: jest.fn(),
+ emitStatusReceived: jest.fn(),
+ };
+ statusStore = { ingest: jest.fn() };
+ engines = new EngineRegistry();
+ engine = { tag: 'one' } as unknown as IWhatsAppEngine;
+ engines.set('s1', engine);
+ projector = new MessageProjector(
+ repo as unknown as Repository<Message>,
+ { update: jest.fn().mockResolvedValue({ affected: 1 }) } as unknown as Repository<Session>,
+ engines,
+ eventsGateway as unknown as EventsGateway,
+ webhookService as unknown as WebhookService,
+ hookManager as unknown as HookManager,
+ statusStore as unknown as StatusStoreService,
+ { resolveSenderPhone: jest.fn().mockResolvedValue(null) } as unknown as SessionLidResolver,
+ );
+ };
+
+ const admit = (a: ReturnType<typeof arrival>, from = engine): InboundTicket => {
+ const ticket = projector.admitInbound('s1', from, a);
+ if (!ticket) throw new Error('not admitted');
+ return ticket;
+ };
+ const insertedIds = (): unknown[] => rows.map(r => r.waMessageId);
+ const dispatched = (event: string): Row[] =>
+ (webhookService.dispatch.mock.calls as Array<[string, string, Row]>).filter(c => c[1] === event).map(c => c[2]);
+ const persistedHooks = (): Row[] =>
+ (hookManager.execute.mock.calls as Array<[string, { message: Row }]>)
+ .filter(c => c[0] === 'message:persisted')
+ .map(c => c[1].message);
+ const room = () =>
+ (projector as unknown as { rooms: Map<string, { gate: { activeCount: number } }> }).rooms.get('s1');
+
+ beforeEach(() => {
+ process.env.INBOUND_MEDIA_CONCURRENCY = '1';
+ build('postgres');
+ });
+
+ afterEach(() => {
+ process.env = { ...env };
+ });
+
+ it('starts hooks and inserts in arrival order when a chat emits out of order, leaving other chats free', async () => {
+ const a = admit(arrival('A', 'chat1'));
+ const b = admit(arrival('B', 'chat1'));
+ const c = admit(arrival('C', 'chat1'));
+ const d = admit(arrival('D', 'chat2'));
+
+ projector.handleInboundMessage('s1', engine, message('D', 'chat2'), d);
+ projector.handleInboundMessage('s1', engine, message('C', 'chat1'), c);
+ projector.handleInboundMessage('s1', engine, message('B', 'chat1'), b);
+ await flush();
+ // The other chat is not behind chat1's first message; chat1 waits for it.
+ expect(insertedIds()).toEqual(['D']);
+ expect(hookStarts).toEqual(['D']);
+
+ projector.handleInboundMessage('s1', engine, message('A', 'chat1'), a);
+ await flush();
+
+ expect(hookStarts).toEqual(['D', 'A', 'B', 'C']);
+ expect(insertedIds()).toEqual(['D', 'A', 'B', 'C']);
+ });
+
+ it('stores a message revoked before its emit cleared, without running its hook or announcing it', async () => {
+ const t = admit(arrival('A', 'chat1', true));
+ projector.handleMessageRevoked('s1', engine, {
+ id: 'A',
+ revokedId: 'A',
+ chatId: 'chat1',
+ from: 'chat1',
+ type: 'revoked',
+ body: '',
+ timestamp: 1,
+ });
+ projector.handleInboundMessage(
+ 's1',
+ engine,
+ message('A', 'chat1', { type: 'image', body: 'secret', media: { mimetype: 'image/png', data: 'QUJD' } }),
+ t,
+ );
+ await flush();
+
+ expect(hookStarts).toEqual([]);
+ expect(rows).toEqual([expect.objectContaining({ waMessageId: 'A', body: '', type: 'revoked', metadata: null })]);
+ expect(persistedHooks()).toEqual([expect.objectContaining({ body: '', type: 'revoked', metadata: null })]);
+ expect(dispatched('message.received')).toEqual([]);
+ expect(eventsGateway.emitMessage).not.toHaveBeenCalled();
+ });
+
+ it('runs the hook, stores and announces a message edited before its emit with the edit', async () => {
+ const t = admit(arrival('A', 'chat1'));
+ projector.applyMessageEditQueued('s1', { messageId: 'A', body: 'edited' } as never);
+ projector.handleInboundMessage('s1', engine, message('A', 'chat1', { body: 'original' }), t);
+ await flush();
+
+ const calls = hookManager.execute.mock.calls as Array<[string, IncomingMessage]>;
+ const hooked = calls.find(c => c[0] === 'message:received')?.[1];
+ expect(hooked?.body).toBe('edited');
+ expect(rows[0]).toEqual(expect.objectContaining({ body: 'edited' }));
+ expect(dispatched('message.received')[0]).toEqual(expect.objectContaining({ body: 'edited' }));
+ });
+
+ it('stores a reaction and an ack that arrived before the emit once the row is written', async () => {
+ const t = admit(arrival('A', 'chat1'));
+ projector.applyReactionQueued('s1', { messageId: 'A', chatId: 'chat1', reaction: 'ok', senderId: '[email protected]' });
+ projector.handleMessageAck('s1', engine, 'A', 'read');
+ await flush();
+ projector.handleInboundMessage('s1', engine, message('A', 'chat1'), t);
+ await flush();
+
+ expect(rows[0].status).toBe(MessageStatus.READ);
+ expect((rows[0].metadata as Row).reactions).toEqual({ '[email protected]': 'ok' });
+ });
+
+ it('applies a REST delete and a REST edit made before the emit', async () => {
+ const a = admit(arrival('A', 'chat1'));
+ const b = admit(arrival('B', 'chat2'));
+ await projector.recordRevoke('s1', 'A');
+ await projector.recordOutboundMessageEdit('s1', 'B', 'rest edit');
+ projector.handleInboundMessage('s1', engine, message('A', 'chat1'), a);
+ projector.handleInboundMessage('s1', engine, message('B', 'chat2'), b);
+ await flush();
+
+ expect(rows.find(r => r.waMessageId === 'A')).toEqual(expect.objectContaining({ body: '', type: 'revoked' }));
+ expect(rows.find(r => r.waMessageId === 'B')).toEqual(expect.objectContaining({ body: 'rest edit' }));
+ // A delete for me: the message still exists for its sender, so it is announced as revoked.
+ expect(dispatched('message.received').map(m => [m.id, m.type, m.body])).toEqual([
+ ['A', 'revoked', ''],
+ ['B', 'text', 'rest edit'],
+ ]);
+ });
+
+ it('keeps a revoke recorded between two deliveries of one id for the delivery that is emitted', async () => {
+ const first = admit(arrival('A', 'chat1'));
+ const second = admit(arrival('A', 'chat1'));
+ first.drop();
+ await flush();
+ projector.handleMessageRevoked('s1', engine, { id: 'A', chatId: 'chat1' } as never);
+ projector.handleInboundMessage('s1', engine, message('A', 'chat1'), second);
+ await flush();
+
+ expect(rows).toEqual([expect.objectContaining({ waMessageId: 'A', type: 'revoked' })]);
+ expect(dispatched('message.received')).toEqual([]);
+ });
+
+ it('lends nothing to a quote before the emit, and the message once emitted', async () => {
+ let release: () => void = () => undefined;
+ insertGate = () => new Promise(resolve => (release = resolve));
+ const t = admit(arrival('A', 'chat1'));
+ expect(projector.inFlightInbound('s1', 'A')).toBeUndefined();
+
+ projector.handleInboundMessage('s1', engine, message('A', 'chat1'), t);
+ expect(projector.inFlightInbound('s1', 'A')).toEqual(expect.objectContaining({ body: 'body of A' }));
+ await flush();
+ release();
+ await flush();
+ expect(projector.inFlightInbound('s1', 'A')).toBeUndefined();
+ });
+
+ it('keeps no payload of a repeat delivery in the shared entry once that delivery committed', async () => {
+ process.env.INBOUND_MEDIA_CONCURRENCY = '2';
+ let inserts = 0;
+ insertGate = () => (++inserts === 2 ? new Promise(() => undefined) : Promise.resolve());
+ const first = admit(arrival('X', 'chat1', true));
+ const repeat = admit(arrival('X', 'chat1', true));
+ expect(await first.reserve()).toBe(true);
+ expect(await repeat.reserve()).toBe(true);
+ const image = (data: string) => message('X', 'chat1', { type: 'image', media: { mimetype: 'image/png', data } });
+ // The repeat is emitted first; the first delivery commits first and gives its slot back.
+ projector.handleInboundMessage('s1', engine, image('P2xx'), repeat);
+ projector.handleInboundMessage('s1', engine, image('P1xx'), first);
+ await flush();
+ expect(room()?.gate.activeCount).toBe(1);
+
+ // Only the repeat's own payload may still be reachable: its slot accounts for it.
+ const entries = (projector as unknown as { inboundInFlight: Map<string, unknown> }).inboundInFlight;
+ expect(JSON.stringify([...entries.values()])).not.toContain('P1xx');
+ expect(projector.inFlightInbound('s1', 'X')).toEqual({ chatId: 'chat1', body: 'body of X' });
+ });
+
+ it('keeps the slot of a payload whose insert stalls, sheds later media after the wait, and lets text flow', async () => {
+ process.env.MEDIA_DOWNLOAD_TIMEOUT_MS = '20';
+ insertGate = row => (row.waMessageId === 'M1' ? new Promise(() => undefined) : Promise.resolve());
+
+ const m1 = admit(arrival('M1', 'chat1', true));
+ expect(await m1.reserve()).toBe(true);
+ projector.handleInboundMessage('s1', engine, message('M1', 'chat1', { type: 'image' }), m1);
+
+ const m2 = admit(arrival('M2', 'chat2', true));
+ const text = admit(arrival('T', 'chat3'));
+ projector.handleInboundMessage('s1', engine, message('T', 'chat3'), text);
+ await flush();
+ expect(insertedIds()).toEqual(['T']);
+
+ const started = Date.now();
+ expect(await m2.reserve()).toBe(false);
+ expect(Date.now() - started).toBeGreaterThanOrEqual(35);
+ // The stalled payload is still referenced by its commit, so its slot is not handed back.
+ expect(room()?.gate.activeCount).toBe(1);
+
+ projector.handleInboundMessage('s1', engine, message('M2', 'chat2', { type: 'image' }), m2);
+ await flush();
+ expect(insertedIds()).toEqual(['T', 'M2']);
+ });
+
+ it('keeps the room across an engine replacement, and releases the old engine pending turns at once', async () => {
+ let release: () => void = () => undefined;
+ insertGate = row => (row.waMessageId === 'M1' ? new Promise(resolve => (release = resolve)) : Promise.resolve());
+ const m1 = admit(arrival('M1', 'chat1', true));
+ projector.handleInboundMessage('s1', engine, message('M1', 'chat1', { type: 'image' }), m1);
+ const waiting = admit(arrival('M2', 'chat2', true));
+ const pendingReserve = waiting.reserve();
+ const pendingText = admit(arrival('T1', 'chat2'));
+ await flush();
+
+ const replacement = { tag: 'two' } as unknown as IWhatsAppEngine;
+ engines.set('s1', replacement);
+ // Woken at once, not after the room wait.
+ expect(await pendingReserve).toBe(false);
+ expect(pendingText.reserve()).toBe(false);
+
+ // The replaced engine's stalled payload still counts: the new engine gets no extra slot.
+ const t2 = admit(arrival('T2', 'chat2'), replacement);
+ const m3 = admit(arrival('M3', 'chat2', true), replacement);
+ projector.handleInboundMessage('s1', replacement, message('T2', 'chat2'), t2);
+ await flush();
+ // chat2's dropped turns no longer stand in the way.
+ expect(insertedIds()).toEqual(['T2']);
+ let granted: boolean | undefined;
+ void Promise.resolve(m3.reserve()).then(v => (granted = v));
+ await flush();
+ expect(granted).toBeUndefined();
+
+ release();
+ await flush();
+ expect(granted).toBe(true);
+ });
+
+ it('holds a status post slot until its ingest settles', async () => {
+ const ingest = deferred<{ row: unknown; created: boolean }>();
+ statusStore.ingest.mockReturnValue(ingest.promise);
+ const status = admit(arrival('S1', 'status@broadcast', true, true));
+ projector.handleInboundMessage(
+ 's1',
+ engine,
+ message('S1', 'status@broadcast', { isStatusBroadcast: true, from: '[email protected]', type: 'image' }),
+ status,
+ );
+ const next = admit(arrival('M1', 'chat1', true));
+ let granted: boolean | undefined;
+ void Promise.resolve(next.reserve()).then(v => (granted = v));
+ await flush();
+ expect(statusStore.ingest).toHaveBeenCalled();
+ expect(granted).toBeUndefined();
+
+ ingest.resolve({ row: {}, created: false });
+ await flush();
+ expect(granted).toBe(true);
+ });
+
+ it('holds nothing on SQLite: no admission, and the untouched path stores and announces as before', async () => {
+ build('better-sqlite3');
+ expect(projector.holdsInbound).toBe(false);
+
+ projector.handleInboundMessage('s1', engine, message('A', 'chat1'));
+ await flush();
+
+ expect(insertedIds()).toEqual(['A']);
+ expect(dispatched('message.received')).toHaveLength(1);
+ expect((projector as unknown as { rooms: Map<string, unknown> }).rooms.size).toBe(0);
+ });
+});
diff --git a/src/modules/session/message-projector.service.ts b/src/modules/session/message-projector.service.ts
index 8b77dff97..096e0392f 100644
--- a/src/modules/session/message-projector.service.ts
+++ b/src/modules/session/message-projector.service.ts
@@ -23,10 +23,12 @@ import {
IWhatsAppEngine,
DeliveryStatus,
IncomingMessage,
+ InboundTicket,
ReactionEvent,
EditedMessage,
RevokedMessage,
} from '../../engine/interfaces/whatsapp-engine.interface';
+import { deferred, InboundRoom, roomState, type RoomState } from './inbound-room';
import { createLogger } from '../../common/services/logger.service';
import { EventsGateway } from '../events/events.gateway';
import { WebhookService } from '../webhook/webhook.service';
@@ -87,6 +89,85 @@ interface InboundPersistOutcome {
*/
type InboundMessageData = { [K in keyof IncomingMessage]: IncomingMessage[K] };
+const quoteOf = (m: Pick<IncomingMessage, 'chatId' | 'body'>) => ({ chatId: m.chatId, body: m.body });
+
+/** A message not yet written, and the changes that reached it meanwhile (see inboundInFlight). */
+interface InFlightEntry {
+ // What a quote of it may lend, as the hook chain last left it. Never the message itself: a repeat
+ // delivery shares the entry, and its payload must not outlive the slot that accounts for it.
+ // Unset while the message is admitted but not yet emitted (see admitInbound).
+ quote?: Pick<IncomingMessage, 'chatId' | 'body'>;
+ revoked?: boolean;
+ // Set when the revoke came from the engine, so message.revoked went out for it.
+ revokeAnnounced?: boolean;
+ editedBody?: string;
+ ackStatus?: MessageStatus;
+ // Latest reaction per sender; '' means the sender withdrew theirs.
+ reactions?: Record<string, string>;
+ // Tickets sharing the entry: a repeat delivery keeps what was recorded for the first one.
+ holders?: number;
+}
+
+/**
+ * A message admitted on arrival (PostgreSQL only): its place in the chat's hook and commit order, its
+ * claim on the session's room when it downloads media, and its in-flight entry, so a change that
+ * targets it before its emit is kept for its row.
+ */
+class InboundTurn implements InboundTicket {
+ state: 'pending' | 'emitted' | 'dropped' = 'pending';
+ /** The commit to run in the chat's turn, or null when the message will not be emitted. */
+ readonly outcome = deferred<(() => Promise<void>) | null>();
+ readonly emitted = deferred<void>();
+ /** Resolved once every earlier message of the chat was emitted or dropped: the hook may start. */
+ readonly hookGate = deferred<void>();
+
+ constructor(
+ readonly engine: IWhatsAppEngine,
+ readonly entry: InFlightEntry,
+ private readonly room?: InboundRoom,
+ readonly s?: RoomState,
+ ) {}
+
+ get revoked(): boolean {
+ return this.entry.revoked === true;
+ }
+
+ // A message revoked before its download needs none: its row is stored cleared.
+ reserve(): boolean | Promise<boolean> {
+ if (!this.s || !this.room || this.state === 'dropped') return false;
+ if (this.entry.revoked) return this.giveUp();
+ return this.s.granted || this.room.wait(this.s).then(granted => granted && !this.entry.revoked);
+ }
+
+ /** Leave the room's queue, if still in it: once admitted, the claim passes the slot straight on. */
+ giveUp(): false {
+ if (this.s && !this.s.granted) {
+ this.s.abandoned = true;
+ this.s.grant.resolve(false);
+ }
+ return false;
+ }
+
+ holdUntil(p: Promise<unknown>): void {
+ this.s?.holds.push(p.catch(() => undefined));
+ }
+
+ emit(job: () => Promise<void>): void {
+ if (this.state !== 'pending') return;
+ this.state = 'emitted';
+ this.emitted.resolve();
+ this.outcome.resolve(job);
+ }
+
+ drop(): void {
+ if (this.state !== 'pending') return;
+ this.state = 'dropped';
+ this.outcome.resolve(null);
+ this.emitted.resolve();
+ this.giveUp();
+ }
+}
+
@Injectable()
export class MessageProjector {
private readonly logger = createLogger('MessageProjector');
@@ -107,24 +188,26 @@ export class MessageProjector {
this.logger.error(`Unexpected failure committing a message: ${key}`, String(err));
});
+ // PostgreSQL only: a chat's `message:received` / `message:sent` chains start in arrival order, each
+ // once every earlier admitted message of the chat was emitted or dropped (see admitInbound).
+ private readonly chatEmits = new KeyedMutationQueue((key, err) => {
+ this.logger.error(`Unexpected failure ordering a message: ${key}`, String(err));
+ });
+
+ // Per session, not per engine: a replaced engine's payloads still waiting for their rows keep
+ // counting, so a reconnect cannot multiply the room.
+ private readonly rooms = new Map<string, InboundRoom>();
+
+ // Admitted messages not yet committed, by session, so retiring an engine can release its own.
+ private readonly tickets = new Map<string, Set<InboundTurn>>();
+
// Messages not yet written, by `${sessionId}:${waMessageId}`: inbound ones from the start of their
// `message:received` chain, own-send echoes from the start of their `message:sent` chain, each until
// its commit (which may wait behind earlier messages of the chat) ends. A handler that quotes the
// message reads it from here, and a revoke, edit or ack landing meanwhile finds no row to update: it
// is recorded here and applied once the row is written (see applyChangesMadeInFlight).
- private readonly inboundInFlight = new Map<
- string,
- {
- message: InboundMessageData;
- revoked?: boolean;
- // Set when the revoke came from the engine, so message.revoked went out for it.
- revokeAnnounced?: boolean;
- editedBody?: string;
- ackStatus?: MessageStatus;
- // Latest reaction per sender; '' means the sender withdrew theirs.
- reactions?: Record<string, string>;
- }
- >();
+ // On PostgreSQL the entry exists from the message's arrival (admitInbound), before its emit.
+ private readonly inboundInFlight = new Map<string, InFlightEntry>();
// Reaction/edit applies, extracted to a plain collaborator. It shares this instance's
// messageMutations queue, so the public enqueue path and the queued applies serialize on one chain.
@@ -158,13 +241,108 @@ export class MessageProjector {
this.messageMutations,
this.logger,
);
+ this.engines.onRetire((id, engine) => {
+ for (const t of this.tickets.get(id) ?? []) if (t.engine === engine) t.drop();
+ this.deleteRoomIfIdle(id);
+ });
}
- /** Engine callback body, lifted out of initializeEngine so the wiring table stays readable. */
- handleInboundMessage(id: string, engine: IWhatsAppEngine, message: IncomingMessage): void {
+ /**
+ * Whether the engines hold each inbound message from its arrival until its row is written. Only
+ * PostgreSQL needs it: a downloaded payload waits in memory for its asynchronous insert, while
+ * better-sqlite3 writes synchronously, so the backlog drains within the arrival's own task.
+ */
+ get holdsInbound(): boolean {
+ return this.messageRepository.manager?.connection?.options?.type === 'postgres';
+ }
+
+ /**
+ * Admit a message as it arrives (see EngineEventCallbacks.admitInbound), synchronously, in arrival
+ * order. The engine passes the ticket to its emit or drops it. Each admitted message of a chat
+ * starts its hook chain and is committed in this order, and changes that target it before its emit
+ * are recorded on its in-flight entry. One that downloads media asks the session's room for a slot
+ * now, granted in arrival order and held until its row is written.
+ */
+ admitInbound(
+ id: string,
+ engine: IWhatsAppEngine,
+ a: { id: string; chatId: string; needsRoom: boolean; status: boolean },
+ ): InboundTicket | undefined {
+ if (!this.engines.isLive(id, engine)) return undefined;
+ const key = a.status || !a.id ? null : `${id}:${a.id}`;
+ let entry: InFlightEntry = {};
+ if (key) {
+ entry = this.inboundInFlight.get(key) ?? entry;
+ this.inboundInFlight.set(key, entry);
+ }
+ entry.holders = (entry.holders ?? 0) + 1;
+ const room = a.needsRoom ? this.room(id) : undefined;
+ const t = new InboundTurn(engine, entry, room, room && roomState());
+ let set = this.tickets.get(id);
+ if (!set) this.tickets.set(id, (set = new Set()));
+ set.add(t);
+ const finish = (): void => {
+ if (entry.holders) entry.holders--;
+ if (key && !entry.holders && this.inboundInFlight.get(key) === entry) this.inboundInFlight.delete(key);
+ set.delete(t);
+ if (!set.size && this.tickets.get(id) === set) this.tickets.delete(id);
+ t.s?.done.resolve();
+ };
+ if (a.status) {
+ void t.outcome.promise.then(finish);
+ } else {
+ const chatKey = `${id}:${a.chatId}`;
+ this.chatEmits.enqueue(chatKey, () => {
+ t.hookGate.resolve();
+ return t.emitted.promise;
+ });
+ this.chatCommits.enqueue(chatKey, async () => {
+ try {
+ const job = await t.outcome.promise;
+ if (job) await job();
+ } finally {
+ finish();
+ }
+ });
+ }
+ if (room && t.s) {
+ void room.request(t.s).then(() => {
+ if (!this.engines.has(id)) this.deleteRoomIfIdle(id);
+ });
+ }
+ return t;
+ }
+
+ private room(id: string): InboundRoom {
+ let room = this.rooms.get(id);
+ if (!room) this.rooms.set(id, (room = new InboundRoom()));
+ return room;
+ }
+
+ private deleteRoomIfIdle(id: string): void {
+ if (this.rooms.get(id)?.idle) this.rooms.delete(id);
+ }
+
+ /**
+ * Engine callback body, lifted out of initializeEngine so the wiring table stays readable. With a
+ * ticket (PostgreSQL), the hook chain starts in the chat's arrival order and the commit takes the
+ * turn admitInbound reserved; whatever path returns early drops the ticket.
+ */
+ handleInboundMessage(id: string, engine: IWhatsAppEngine, message: IncomingMessage, ticket?: InboundTicket): void {
+ const turn = ticket instanceof InboundTurn ? ticket : undefined;
+ try {
+ this.receiveInbound(id, engine, message, turn);
+ } finally {
+ turn?.drop();
+ }
+ }
+
+ private receiveInbound(id: string, engine: IWhatsAppEngine, message: IncomingMessage, turn?: InboundTurn): void {
if (!this.engines.isLive(id, engine)) return;
if (message.isStatusBroadcast) {
- this.ingestInboundStatus(id, engine, message);
+ const ingest = this.ingestInboundStatus(id, engine, message);
+ // The status store holds the payload until the ingest settles.
+ turn?.holdUntil(ingest);
return;
}
if (this.shouldSkipEphemeralMessage(id, message)) return;
@@ -178,9 +356,12 @@ export class MessageProjector {
void this.sessionRepository.update(id, { lastActiveAt: new Date() }).catch(() => undefined);
// Convert IncomingMessage to plain object for dispatch
const messageData = { ...message };
- // Tracks the chain's current copy, so a quote taken mid-chain carries an earlier handler's rewrite.
+ // Tracks what the chain's current copy lends a quote, so one taken mid-chain carries an earlier
+ // handler's rewrite.
const inFlightKey = `${id}:${message.id}`;
- const inFlight = this.trackInFlight(inFlightKey, messageData);
+ const inFlight = turn
+ ? Object.assign(turn.entry, { quote: quoteOf(messageData) })
+ : this.trackInFlight(inFlightKey, messageData);
const onFailure = (err: unknown): null => {
this.logger.error(`onMessage handler failed for ${id}`, String(err));
@@ -189,36 +370,70 @@ export class MessageProjector {
// Execute hook for message received - plugins can modify or stop processing. The hook chain and
// sender resolution run concurrently across messages; the catch is attached now so a failure
// while the commit waits its turn is never an unhandled rejection.
- const prepared = this.hookManager
- .execute('message:received', messageData, {
- sessionId: id,
- source: 'Engine',
- accept: data => {
- if (!isMessagePayload(data)) return false;
- inFlight.message = data;
- return true;
- },
- })
- .then(({ data }) =>
- this.prepareInboundMessage(id, this.messageOrEngineCopy(id, 'message:received', data, message)),
- )
- .catch(onFailure);
- this.chatCommits.enqueue(this.chatCommitKey(id, message), async () => {
+ const prepared = this.startHookChain(id, engine, turn, messageData, () =>
+ this.hookManager
+ .execute('message:received', messageData, {
+ sessionId: id,
+ source: 'Engine',
+ accept: data => {
+ if (!isMessagePayload(data)) return false;
+ inFlight.quote = quoteOf(data);
+ return true;
+ },
+ })
+ .then(({ data }) =>
+ this.prepareInboundMessage(id, this.messageOrEngineCopy(id, 'message:received', data, message)),
+ ),
+ ).catch(onFailure);
+ this.commitInTurn(id, message, turn, async () => {
try {
const finalMessage = await prepared;
if (finalMessage) await this.commitInboundMessage(id, engine, finalMessage);
} catch (err) {
onFailure(err);
} finally {
- this.untrackInFlight(inFlightKey, inFlight);
+ if (!turn) this.untrackInFlight(inFlightKey, inFlight);
}
});
}
+ /**
+ * Start a message's hook chain: at once without a turn. With one, once every earlier admitted
+ * message of the chat was emitted or dropped, so hooks start in arrival order. A message revoked
+ * before then skips its hooks, as its content is gone, and an edit made before then reaches them.
+ */
+ private startHookChain(
+ id: string,
+ engine: IWhatsAppEngine,
+ turn: InboundTurn | undefined,
+ messageData: InboundMessageData,
+ run: () => Promise<InboundMessageData>,
+ ): Promise<InboundMessageData | null> {
+ if (!turn) return run();
+ const { entry } = turn;
+ return turn.hookGate.promise.then(() => {
+ if (!this.engines.isLive(id, engine)) return null;
+ if (entry.revoked) return messageData;
+ if (entry.editedBody !== undefined) messageData.body = entry.editedBody;
+ return run();
+ });
+ }
+
+ /** Queue a message's commit on its chat, or hand it to the turn admitInbound already queued. */
+ private commitInTurn(
+ id: string,
+ message: IncomingMessage,
+ turn: InboundTurn | undefined,
+ job: () => Promise<void>,
+ ): void {
+ if (turn) turn.emit(job);
+ else this.chatCommits.enqueue(this.chatCommitKey(id, message), job);
+ }
+
/** Record a message as not yet written; a re-fire of the same id keeps the changes recorded so far. */
private trackInFlight(key: string, message: InboundMessageData) {
const { revoked, revokeAnnounced, editedBody, ackStatus, reactions } = this.inboundInFlight.get(key) ?? {};
- const inFlight = { message, revoked, revokeAnnounced, editedBody, ackStatus, reactions };
+ const inFlight = { quote: quoteOf(message), revoked, revokeAnnounced, editedBody, ackStatus, reactions };
this.inboundInFlight.set(key, inFlight);
return inFlight;
}
@@ -246,8 +461,10 @@ export class MessageProjector {
*/
inFlightInbound(sessionId: string, waMessageId: string): Pick<IncomingMessage, 'chatId' | 'body'> | undefined {
const entry = waMessageId ? this.inboundInFlight.get(`${sessionId}:${waMessageId}`) : undefined;
+ // Admitted but not emitted yet: nobody can have heard of it, so it has nothing to lend.
+ if (!entry?.quote) return undefined;
// A revoked message lends no text, as its cleared row would not; the chatId still scopes the quote.
- return entry && (entry.revoked ? { chatId: entry.message.chatId, body: '' } : entry.message);
+ return entry.revoked ? { chatId: entry.quote.chatId, body: '' } : entry.quote;
}
/**
@@ -279,16 +496,17 @@ export class MessageProjector {
}
/** `isStatusBroadcast` arm of {@link handleInboundMessage}: ingest into the status store, not the message pipeline. */
- private ingestInboundStatus(id: string, engine: IWhatsAppEngine, message: IncomingMessage): void {
+ private ingestInboundStatus(id: string, engine: IWhatsAppEngine, message: IncomingMessage): Promise<void> {
// Status/Story posts arrive via the inbound path for some engines; ingest them into the
// status store instead of the message pipeline. Mirrors the isStatusBroadcast guard in
// onMessageCreate below: an own-send echo (fromMe) stays a plain drop — never ingested,
// never dispatched — so a status you posted never re-appears in your own webhooks/API as
// if a contact had posted it.
- if (message.fromMe) return;
+ if (message.fromMe) return Promise.resolve();
const status = buildIncomingStatus(message);
- if (status) {
- void this.statusStore
+ if (!status) return Promise.resolve();
+ return (
+ this.statusStore
.ingest(id, status)
// Dispatch only on a fresh insert: ingest also resolves with the pre-existing row
// for a duplicate delivery (or the winner's row after a lost insert race), and the
@@ -304,8 +522,8 @@ export class MessageProjector {
sessionId: id,
error: err instanceof Error ? err.message : String(err),
}),
- );
- }
+ )
+ );
}
/** Operator opt-out gate for disappearing messages: true when the message must be skipped entirely. */
@@ -406,6 +624,7 @@ export class MessageProjector {
// (the row has no FK, so a session-delete cleanup would never reap it) or dispatch for a
// session that no longer exists. Mirrors the synchronous isLiveEngine gate at entry.
if (!this.engines.isLive(id, engine)) return null;
+ const insertedCleared = this.applyHeldChanges(id, dbMessage);
// De-duplicate at the source: the engine can re-fire `message` for one inbound message
// (#464). UNIQUE(sessionId, waMessageId) makes the insert the atomic dedup oracle — a
@@ -414,7 +633,7 @@ export class MessageProjector {
// message is never dropped by a transient DB failure.
const outcome = await this.insertWithRetry(id, engine, dbMessage, 'incoming');
if (outcome.landed === 'yes') {
- this.applyChangesMadeInFlight(id, incoming.id);
+ this.applyChangesMadeInFlight(id, incoming.id, insertedCleared);
return { dbMessage, persisted: true };
}
if (outcome.landed === 'stale') return null;
@@ -427,6 +646,20 @@ export class MessageProjector {
return { dbMessage, persisted: false };
}
+ /**
+ * PostgreSQL holds a message from its arrival, so a revoke or edit often lands before its row: write
+ * the row as the change leaves it, so `message:persisted` and the media archive never see content
+ * that was taken back. A change landing during the insert is still applied after it. Returns whether
+ * the row is written cleared.
+ */
+ private applyHeldChanges(id: string, dbMessage: Message): boolean {
+ if (!this.holdsInbound) return false;
+ const entry = dbMessage.waMessageId ? this.inboundInFlight.get(`${id}:${dbMessage.waMessageId}`) : undefined;
+ if (entry?.revoked) Object.assign(dbMessage, REVOKED_ROW_PATCH);
+ else if (entry?.editedBody !== undefined) dbMessage.body = entry.editedBody;
+ return entry?.revoked === true;
+ }
+
/**
* Insert a message row under the UNIQUE(sessionId, waMessageId) dedup oracle. `insert()` (not
* `save()`) is load-bearing: a duplicate insert throws, which is the atomic dedup oracle for #464
@@ -476,9 +709,11 @@ export class MessageProjector {
* Write a revoke, edit, reaction or ack that arrived before the message's row existed onto the row
* just inserted: its own write matched no row then. Queued on the message's mutation chain, so it
* lands after any edit already queued, and a revoke wins over the rest. A change arriving after the
- * insert finds the row itself. Reactions are only stored here; their event already went out.
+ * insert finds the row itself. Reactions are only stored here; their event already went out. A row
+ * inserted cleared is cleared again, in case a change raced the insert, but `message:persisted`
+ * already carried it cleared and does not fire twice.
*/
- private applyChangesMadeInFlight(id: string, waMessageId: string): void {
+ private applyChangesMadeInFlight(id: string, waMessageId: string, insertedCleared = false): void {
const pending = waMessageId ? this.inboundInFlight.get(`${id}:${waMessageId}`) : undefined;
const ackStatus = pending?.ackStatus;
if (ackStatus) {
@@ -488,7 +723,7 @@ export class MessageProjector {
}
if (!pending?.revoked && pending?.editedBody === undefined && !pending?.reactions) return;
this.enqueueMessageMutation(id, waMessageId, async () => {
- if (pending.revoked) return this.revokeRow(id, waMessageId);
+ if (pending.revoked) return this.revokeRow(id, waMessageId, !insertedCleared);
try {
if (pending.editedBody !== undefined) {
await this.messageRepository.update({ sessionId: id, waMessageId }, { body: pending.editedBody });
@@ -521,7 +756,11 @@ export class MessageProjector {
recordRevoke(sessionId: string, waMessageId: string): Promise<void> {
// A message not written yet takes the revoke once its row lands; the UPDATE below matches nothing.
const inFlight = this.inboundInFlight.get(`${sessionId}:${waMessageId}`);
- if (inFlight) inFlight.revoked = true;
+ if (inFlight) {
+ inFlight.revoked = true;
+ // Media still waiting for room needs none now: wake it, so it is stored cleared at once.
+ for (const t of this.tickets.get(sessionId) ?? []) if (t.entry === inFlight) t.giveUp();
+ }
return new Promise(resolve =>
this.enqueueMessageMutation(sessionId, waMessageId, () =>
this.revokeRow(sessionId, waMessageId).finally(resolve),
@@ -533,12 +772,12 @@ export class MessageProjector {
* Apply {@link REVOKED_ROW_PATCH}, then hand the cleared row to `message:persisted` so a plugin index
* keyed by row id drops the content too. Must run on the message's mutation chain.
*/
- private async revokeRow(sessionId: string, waMessageId: string): Promise<void> {
+ private async revokeRow(sessionId: string, waMessageId: string, notify = true): Promise<void> {
// An undefined condition is DROPPED from the where-clause, which would clear every row of the session.
if (!waMessageId) return;
try {
const result = await this.messageRepository.update({ sessionId, waMessageId }, REVOKED_ROW_PATCH);
- if (!result?.affected) return;
+ if (!result?.affected || !notify) return;
const row = await this.messageRepository.findOne({ where: { sessionId, waMessageId } });
if (!row) return;
void this.hookManager
@@ -609,7 +848,17 @@ export class MessageProjector {
}
/** Engine callback body, lifted out of initializeEngine so the wiring table stays readable. */
- handleOwnSendEcho(id: string, engine: IWhatsAppEngine, message: IncomingMessage): void {
+ handleOwnSendEcho(id: string, engine: IWhatsAppEngine, message: IncomingMessage, ticket?: InboundTicket): void {
+ const turn = ticket instanceof InboundTurn ? ticket : undefined;
+ try {
+ this.receiveOwnSendEcho(id, engine, message, turn);
+ } finally {
+ turn?.drop();
+ }
+ }
+
+ /** {@link handleOwnSendEcho} with the ticket resolved; every early return leaves the ticket to be dropped. */
+ private receiveOwnSendEcho(id: string, engine: IWhatsAppEngine, message: IncomingMessage, turn?: InboundTurn): void {
if (!this.engines.isLive(id, engine)) return;
// `message_create` fires for every message the account creates, including sends composed on a
// linked phone — which the `message`/`onMessage` event never delivers. Incoming messages are
@@ -642,29 +891,32 @@ export class MessageProjector {
// send landing meanwhile is recorded here and applied once the row is written. Like the inbound
// path, the entry follows the chain's rewrites, so a quote carries what the row will hold.
const inFlightKey = `${id}:${message.id}`;
- const inFlight = this.trackInFlight(inFlightKey, messageData);
+ const inFlight = turn
+ ? Object.assign(turn.entry, { quote: quoteOf(messageData) })
+ : this.trackInFlight(inFlightKey, messageData);
// Execute hook for message sent - plugins can modify or stop processing. Like the inbound path,
// the hook chain runs concurrently and the commit waits its turn on the chat's queue.
- const prepared = this.hookManager
- .execute('message:sent', messageData, {
- sessionId: id,
- source: 'Engine',
- accept: data => {
- if (!isMessagePayload(data)) return false;
- inFlight.message = data;
- return true;
- },
- })
- .then(({ data }) => this.messageOrEngineCopy(id, 'message:sent', data, message))
- .catch(onFailure);
- this.chatCommits.enqueue(this.chatCommitKey(id, message), async () => {
+ const prepared = this.startHookChain(id, engine, turn, messageData, () =>
+ this.hookManager
+ .execute('message:sent', messageData, {
+ sessionId: id,
+ source: 'Engine',
+ accept: data => {
+ if (!isMessagePayload(data)) return false;
+ inFlight.quote = quoteOf(data);
+ return true;
+ },
+ })
+ .then(({ data }) => this.messageOrEngineCopy(id, 'message:sent', data, message)),
+ ).catch(onFailure);
+ this.commitInTurn(id, message, turn, async () => {
try {
const finalMessage = await prepared;
if (finalMessage) await this.commitOwnSendEcho(id, engine, finalMessage);
} catch (err) {
onFailure(err);
} finally {
- this.untrackInFlight(inFlightKey, inFlight);
+ if (!turn) this.untrackInFlight(inFlightKey, inFlight);
}
});
}
@@ -714,6 +966,7 @@ export class MessageProjector {
// awaits. Re-check liveness so a late continuation can't persist an orphan row
// (mirrors onMessage).
if (!this.engines.isLive(id, engine)) return;
+ const insertedCleared = this.applyHeldChanges(id, dbMessage);
// A unique violation means the REST send path already persisted this API-originated
// send: the dedup oracle working as intended, not an error. Any other failure (after the
// one transient retry) fails open, so a real send is never dropped on a DB fault.
@@ -722,7 +975,7 @@ export class MessageProjector {
// The REST writer may have won either attempt; its row still takes the echo's pending changes.
if (outcome.landed === 'dup') this.applyChangesMadeInFlight(id, outgoing.id);
if (outcome.landed === 'yes') {
- this.applyChangesMadeInFlight(id, outgoing.id);
+ this.applyChangesMadeInFlight(id, outgoing.id, insertedCleared);
// Fire-and-forget, mirroring onMessage: plugin providers (search etc.) see phone-
// composed sends exactly like API sends.
void this.hookManager
diff --git a/src/modules/session/session-engine-event-wiring.ts b/src/modules/session/session-engine-event-wiring.ts
index 9d28ec823..73502ad60 100644
--- a/src/modules/session/session-engine-event-wiring.ts
+++ b/src/modules/session/session-engine-event-wiring.ts
@@ -233,7 +233,11 @@ export class SessionEngineEventWiring {
}
host.handleEngineReady(id, engine, phone, pushName);
},
- onMessage: (message): void => host.messages.handleInboundMessage(id, engine, message),
+ // PostgreSQL only: holds each inbound message from arrival until its row is written.
+ ...(host.messages.holdsInbound
+ ? { admitInbound: arrival => host.messages.admitInbound(id, engine, arrival) }
+ : {}),
+ onMessage: (message, ticket): void => host.messages.handleInboundMessage(id, engine, message, ticket),
onHistoryMessages: messages => {
if (!host.isLiveEngine(id, engine)) return;
// Persist for the chat view only; no dispatch (these predate the live session).
@@ -241,7 +245,7 @@ export class SessionEngineEventWiring {
.persistHistoryMessages(id, engine, messages)
.catch(err => this.logger.error(`Failed to persist history messages for ${id}`, String(err)));
},
- onMessageCreate: (message): void => host.messages.handleOwnSendEcho(id, engine, message),
+ onMessageCreate: (message, ticket): void => host.messages.handleOwnSendEcho(id, engine, message, ticket),
onMessageAck: (messageId, status, chatId): void =>
host.messages.handleMessageAck(id, engine, messageId, status, chatId),
onMessageRevoked: (message): void => host.messages.handleMessageRevoked(id, engine, message),
diff --git a/src/modules/session/session.service.spec.ts b/src/modules/session/session.service.spec.ts
index bfa46d0d4..e287adc70 100644
--- a/src/modules/session/session.service.spec.ts
+++ b/src/modules/session/session.service.spec.ts
@@ -5414,6 +5414,23 @@ describe('SessionService', () => {
}
});
+ // better-sqlite3 writes within the arrival's own task, so nothing is held there and the engines
+ // run exactly as before; PostgreSQL inserts asynchronously and gets the arrival tickets.
+ it('offers admitInbound to the engine on PostgreSQL only', async () => {
+ const sqlite = await startAndCapture();
+ expect(sqlite).not.toHaveProperty('admitInbound');
+ await service.stop('sess-uuid-1');
+
+ (messageRepository as unknown as { manager: unknown }).manager = {
+ connection: { options: { type: 'postgres' } },
+ };
+ mockEngine.initialize.mockClear();
+ const postgres = await startAndCapture();
+ const ticket = postgres.admitInbound?.({ id: 'wa-1', chatId: '[email protected]', needsRoom: false, status: false });
+ expect(typeof ticket?.reserve).toBe('function');
+ ticket?.drop();
+ });
+
it('ignores onMessage from a superseded engine (no persist, no webhook)', async () => {
const callbacks = await startAndCapture();
enginesOf().set('sess-uuid-1', { marker: 'engine-B' });