Free, Open Source, Self-Hosted WhatsApp API Gateway
TypeScript apibotgatewayself-hostedwhatsappwhatsapp-apiwhatsapp-automationwhatsapp-botwhatsapp-gatewaywhatsapp-gateway-apiwhatsapp-webwhatsapp-web-api
feat(messages): add message-time windows
Yudhi Armyndharis6476f0986358
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex b614f6ab3..9a0d4c80f 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0### Added+- Filter stored messages by time, direction, type or message reference, with stable message-time pagination.- Expose poll choices and selection mode in message events, stored messages, history, and dashboard chats.- Expose optional last-message types in chat lists and SDKs.- Add bounded webhook failure redrive with optional payload retention.diff --git a/docs/05-database-design.md b/docs/05-database-design.mdindex 153e401b4..1d631930e 100644--- a/docs/05-database-design.md+++ b/docs/05-database-design.md@@ -915,7 +915,8 @@ src/database/migrations/ # data connection (pluggable)├── 1786950000000-ScrubNonPhoneLidMappings.ts # drops lid_mappings rows where earlier releases stored a broadcast id as the phone├── 1787000000000-AddWebhookDeliveryFailurePayload.ts # webhook_delivery_failures.payload (optional replay copy)├── 1787100000000-AddSendIdempotencyKeys.ts # send_idempotency_keys (opt-in send retry guard)-└── 1787200000000-DeduplicateTerminalWebhookFailures.ts # one terminal record per webhook and idempotency key+├── 1787200000000-DeduplicateTerminalWebhookFailures.ts # one terminal record per webhook and idempotency key+└── 1790812800000-AddMessageWindowIndexes.ts # compound timestamp indexes for message-time pagination```> [!NOTE]diff --git a/docs/06-api-specification.md b/docs/06-api-specification.mdindex bf9acc6f0..ff5d4eb17 100644--- a/docs/06-api-specification.md+++ b/docs/06-api-specification.md@@ -1317,14 +1317,32 @@ Get persisted message history for a session from the local DB (paginated, filter**Query parameters**-| Name | Type | Required | Default | Description |-| ----------- | ------- | -------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |-| chatId | string | No | — | Filter by chat ID. Matched across `@c.us` / `@s.whatsapp.net` dialects via the lid-mapping table. Required for a key restricted to selected chats. |-| from | string | No | — | Filter by sender; matches `from` or a group message's `author`. A phone also matches any lid that resolves to it. |-| limit | integer | No | 50 | Clamped to `[1,100]`; a non-finite value falls back to 50. |-| offset | integer | No | 0 | Clamped to `>=0`; a non-finite value falls back to 0. |-| after | string | No | — | Keyset cursor: the `id` of the last message of the previous page. Anchors the window to a row rather than a count, so a message arriving mid-walk cannot shift it. Takes precedence over `offset`. Unknown in this session gives `400`. |-| inlineMedia | boolean | No | true | Set `false` (or `0`) to omit every inline media payload, leaving each row's `{ omitted, sizeBytes }` marker and the media endpoint. The budget below is per response, so a paged walk pulls it afresh on every page. |+| Name | Type | Required | Default | Description |+| ----------- | ------- | -------- | --------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |+| chatId | string | No | — | Filter by chat ID. Matched across `@c.us` / `@s.whatsapp.net` dialects via the lid-mapping table. Required for a key restricted to selected chats. |+| from | string | No | — | Filter by sender; matches `from` or a group message's `author`. A phone also matches any lid that resolves to it. |+| limit | integer | No | 50 | Clamped to `[1,100]`; a non-finite value falls back to 50. |+| offset | integer | No | 0 | Clamped to `>=0`; a non-finite value falls back to 0. |+| after | string | No | — | Keyset cursor: the `id` of the last message of the previous page. Anchors the window to a row rather than a count, so a message arriving mid-walk cannot shift it. Takes precedence over `offset`. Unknown in this session gives `400`. |+| inlineMedia | boolean | No | true | Set `false` (or `0`) to omit every inline media payload, leaving each row's `{ omitted, sizeBytes }` marker and the media endpoint. The budget below is per response, so a paged walk pulls it afresh on every page. |+| messageId | string | No | (none) | Exact WhatsApp message reference within the session and optional chat filter. |+| since | number | No | (none) | Inclusive lower message-time bound, Unix epoch milliseconds. |+| until | number | No | (none) | Exclusive upper message-time bound, Unix epoch milliseconds. |+| direction | string | No | (none) | `incoming` or `outgoing`. |+| type | string | No | (none) | Exact stored type token, such as `text`, `image` or `voice`; no search text required. |+| orderBy | string | No | createdAt | Newest first by ingestion time (`createdAt`) or message time (`timestamp`). |++Time bounds accept decimal or scientific notation from 0 to 9007199254740991 milliseconds. Blank, malformed, non-finite, unsafe or reversed bounds return `400`. They compare against stored whole-second message times, with `since` inclusive and `until` exclusive. `orderBy=timestamp` uses the message UUID as its tie-breaker and excludes unknown message times. Without these options, ingestion-time ordering and existing stored rows are preserved.++With a time bound or `orderBy=timestamp`, `unknownTimestampTotal` counts rows matching all non-time filters whose message time is unknown. Those rows are excluded from `messages` and `total`. This describes stored history; exhausting a page does not prove complete remote WhatsApp history.++Keep the same filters and order when continuing with `after`. When using the new filters, the anchor must belong to the selected session, chat, sender, type, direction, message reference and time window. An unknown, deleted or mismatched anchor returns `400`. Concurrent older arrivals may appear on subsequent pages; pagination does not freeze the archive.++Example (stored messages for 2026-09-20 in Europe/Vienna):++```http+GET /api/sessions/:sessionId/messages?since=1789855200000&until=1789941600000&orderBy=timestamp&inlineMedia=false&limit=100+```**Response** `200`@@ -1355,11 +1373,11 @@ Get persisted message history for a session from the local DB (paginated, filter}```-Each `Message`: `{ id (uuid), sessionId, waMessageId (string|null), chatId, from, to, chatName (string|null; the sender's push name, or their saved contact name when no push name was reported), author (string|null; the real sender of a group, status or broadcast-list message, where from holds the group, status@broadcast or list id; on Baileys a list message the account received is filed under the sender, so from is the sender too), body (string|null), type, direction ('incoming'|'outgoing'), timestamp (number|null), metadata (object|null), mediaPath (string|null; storage key of archived media), mediaMimetype (string|null), status ('pending'|'sent'|'delivered'|'read'|'failed'), createdAt (ISO date) }`. Ordered by `createdAt` DESC, then by a dialect-dependent second key. The tiebreaker matters: `createdAt` is not unique (SQLite stores whole seconds, a PostgreSQL bulk write ties every row it inserts, and a history backfill carries WhatsApp's own second-resolution timestamp), and without a total order two pages of one walk can repeat a row and omit another. On SQLite the second key is `rowid`, the stored insertion sequence, so messages sharing a second come back in the order they arrived. PostgreSQL has no equivalent (`ctid` moves on every ack update), so it keeps `id`, a random uuid: the walk is equally correct there, but a same-second group is not in arrival order. Note that `offset` still addresses a position by count, so a list taking concurrent writes can shift under a walk: a message arriving mid-walk pushes every older row down one, and the next page re-serves a row the previous one already returned. Pass `after` instead to walk a live chat safely; it anchors on the last row you received, which an arriving message cannot move. The response is the raw service object (no envelope). Unlike the live `IncomingMessage` shape below, this persisted `Message` does **not** carry `kind` — re-derive the chat kind from `chatId` (see `ChatKind` / `chatKind()`) if needed. When present, `metadata` may include `media`, `quotedMessage`, `call`, `reactions`, and, for Baileys inbound business prompts, `buttons: [{ id, text }, …]` so a client (including the dashboard Chats thread) can re-render the choices after reload and tap them via [`POST .../click-button`](#post-apisessionssessionidmessagesclick-button).+Each `Message`: `{ id (uuid), sessionId, waMessageId (string|null), chatId, from, to, chatName (string|null; the sender's push name, or their saved contact name when no push name was reported), author (string|null; the real sender of a group, status or broadcast-list message, where from holds the group, status@broadcast or list id; on Baileys a list message the account received is filed under the sender, so from is the sender too), body (string|null), type, direction ('incoming'|'outgoing'), timestamp (number|null), metadata (object|null), mediaPath (string|null; storage key of archived media), mediaMimetype (string|null), status ('pending'|'sent'|'delivered'|'read'|'failed'), createdAt (ISO date) }`. By default, ordered by `createdAt` DESC, then by a dialect-dependent second key. The tiebreaker matters: `createdAt` is not unique (SQLite stores whole seconds, a PostgreSQL bulk write ties every row it inserts, and a history backfill carries WhatsApp's own second-resolution timestamp), and without a total order two pages of one walk can repeat a row and omit another. On SQLite the second key is `rowid`, the stored insertion sequence, so messages sharing a second come back in the order they arrived. PostgreSQL has no equivalent (`ctid` moves on every ack update), so it keeps `id`, a random uuid: the walk is equally correct there, but a same-second group is not in arrival order. Note that `offset` still addresses a position by count, so a list taking concurrent writes can shift under a walk: a message arriving mid-walk pushes every older row down one, and the next page re-serves a row the previous one already returned. Pass `after` instead to walk a live chat safely; it anchors on the last row you received, which an arriving message cannot move. The response is the raw service object (no envelope). Unlike the live `IncomingMessage` shape below, this persisted `Message` does **not** carry `kind` — re-derive the chat kind from `chatId` (see `ChatKind` / `chatKind()`) if needed. When present, `metadata` may include `media`, `quotedMessage`, `call`, `reactions`, and, for Baileys inbound business prompts, `buttons: [{ id, text }, …]` so a client (including the dashboard Chats thread) can re-render the choices after reload and tap them via [`POST .../click-button`](#post-apisessionssessionidmessagesclick-button).-> **Inline media is carried up to a budget, then omitted.** `MESSAGE_LIST_INLINE_MEDIA_BUDGET_BYTES` (8 MiB of encoded base64 by default) bounds how much inline media one response may hold across its rows. A row is not a bounded object, `limit` is clamped to `[1,100]` but each row can carry its base64 in `metadata.media.data`, so a page of media rows could otherwise reach hundreds of megabytes and fail the read outright. The budget is spent newest-first, matching the `createdAt` DESC order above, so a page that cannot carry everything keeps the most recent media. Past it a payload is replaced with `{ mimetype, filename?, omitted: true, sizeBytes }`, the same marker the engine emits for inbound media over `MEDIA_DOWNLOAD_MAX_BYTES`, and the bytes remain available from [`GET /messages/:chatId/:messageId/media`](#get-apisessionssessionidmessageschatidmessageidmedia). Two rules bound the edges: the newest payload is always inlined even when it alone exceeds the budget (otherwise a single large photo would be permanently unreadable through this route), and a budget of `0` means "never inline" and grants no such allowance. The knob is validated at boot, `8MiB` would parse to 8 bytes, and is forwarded by both compose files. The MCP `MessageList` tool shares this path and the same budget. A row stored under `MESSAGE_INLINE_MEDIA=archive` arrives with the marker already in place, plus `archived: true`, its bytes live only in the chat-media archive, behind the same media route (see that route for the mode).+> **Inline media is carried up to a budget, then omitted.** `MESSAGE_LIST_INLINE_MEDIA_BUDGET_BYTES` (8 MiB of encoded base64 by default) bounds how much inline media one response may hold across its rows. A row is not a bounded object, `limit` is clamped to `[1,100]` but each row can carry its base64 in `metadata.media.data`, so a page of media rows could otherwise reach hundreds of megabytes and fail the read outright. The budget is spent newest-first, matching the selected descending order, so a page that cannot carry everything keeps the most recent media. Past it a payload is replaced with `{ mimetype, filename?, omitted: true, sizeBytes }`, the same marker the engine emits for inbound media over `MEDIA_DOWNLOAD_MAX_BYTES`, and the bytes remain available from [`GET /messages/:chatId/:messageId/media`](#get-apisessionssessionidmessageschatidmessageidmedia). Two rules bound the edges: the newest payload is always inlined even when it alone exceeds the budget (otherwise a single large photo would be permanently unreadable through this route), and a budget of `0` means "never inline" and grants no such allowance. The knob is validated at boot, `8MiB` would parse to 8 bytes, and is forwarded by both compose files. The MCP `MessageList` tool shares this path and the same budget. A row stored under `MESSAGE_INLINE_MEDIA=archive` arrives with the marker already in place, plus `archived: true`, its bytes live only in the chat-media archive, behind the same media route (see that route for the mode).-**Errors:** `400` `after` names no message in this session · `401` missing/invalid API key · `403` a key restricted to selected chats sent no `chatId`, or one outside its allowlist+**Errors:** `400` invalid filters or an unknown or mismatched cursor · `401` missing/invalid API key · `403` a key restricted to selected chats sent no `chatId`, or one outside its allowlistA blank `after` is treated as absent, the way a blank `limit` or `offset` already is, so a clienttemplating a cursor it has not got yet keeps the unfiltered first page rather than a `400`.diff --git a/openapi.json b/openapi.jsonindex efe1d8cf3..ee6355986 100644--- a/openapi.json+++ b/openapi.json@@ -4268,6 +4268,67 @@"schema": {"type": "boolean"}+ },+ {+ "name": "since",+ "required": false,+ "in": "query",+ "description": "Inclusive message-time lower bound, Unix epoch milliseconds (not ingestion time).",+ "schema": {+ "type": "number"+ }+ },+ {+ "name": "until",+ "required": false,+ "in": "query",+ "description": "Exclusive message-time upper bound, Unix epoch milliseconds.",+ "schema": {+ "type": "number"+ }+ },+ {+ "name": "direction",+ "required": false,+ "in": "query",+ "schema": {+ "enum": [+ "incoming",+ "outgoing"+ ],+ "type": "string"+ }+ },+ {+ "name": "orderBy",+ "required": false,+ "in": "query",+ "description": "Newest first. Default createdAt preserves existing ordering; timestamp uses message time and excludes rows without a known timestamp.",+ "schema": {+ "enum": [+ "createdAt",+ "timestamp"+ ],+ "type": "string"+ }+ },+ {+ "name": "type",+ "required": false,+ "in": "query",+ "description": "Exact stored message type token, e.g. text, image, voice.",+ "schema": {+ "type": "string"+ }+ },+ {+ "name": "messageId",+ "required": false,+ "in": "query",+ "description": "Exact WhatsApp message reference; remains scoped to the session and chat.",+ "schema": {+ "type": "string"+ }}],"responses": {@@ -4282,7 +4343,7 @@}},"400": {- "description": "`after` names no message in this session. The keyset comparison would otherwise return an empty page, which reads exactly like the end of the history, so a walk resumed from a stale or foreign cursor would stop silently instead of reporting the cursor.",+ "description": "Invalid message filters or an unknown cursor. With time, direction, type or message-reference filters, the cursor must belong to the same selection.","content": {"application/json": {"schema": {@@ -19842,6 +19903,10 @@"MessageListResponseDto": {"type": "object","properties": {+ "unknownTimestampTotal": {+ "type": "number",+ "description": "Rows matching the non-time filters with unknown message time, before time bounds. Present only for time selections; excluded from their messages and total. Stored history is not proof of complete remote history."+ },"messages": {"description": "Newest first.","type": "array",diff --git a/sdk/go/client_test.go b/sdk/go/client_test.goindex a6dff3283..9e2fb4726 100644--- a/sdk/go/client_test.go+++ b/sdk/go/client_test.go@@ -175,12 +175,14 @@ func TestListSessionsQueryName(t *testing.T) {}func TestQueryEncoding(t *testing.T) {- rt := &recordTransport{status: 200, body: `{"messages":[],"total":0}`}+ rt := &recordTransport{status: 200, body: `{"messages":[],"total":0,"unknownTimestampTotal":2}`}c := newTestClient(t, rt)- _, err := c.Messages.List(context.Background(), "s1", &ListMessagesQuery{+ page, err := c.Messages.List(context.Background(), "s1", &ListMessagesQuery{ChatID: Ptr("[email protected]"),Limit: Ptr(10),+ Since: Ptr(1789855200000.5), Until: Ptr(1789941600000.0),+ Direction: Ptr("incoming"), OrderBy: Ptr("timestamp"), Type: Ptr("image"), MessageID: Ptr("M1"),})if err != nil {t.Fatalf("List: %v", err)@@ -192,6 +194,14 @@ func TestQueryEncoding(t *testing.T) {if _, ok := q["offset"]; ok {t.Fatal("nil offset should not appear in query")}+ for key, expected := range map[string]string{"since": "1789855200000.5", "until": "1789941600000", "direction": "incoming", "orderBy": "timestamp", "type": "image", "messageId": "M1"} {+ if q.Get(key) != expected {+ t.Fatalf("%s = %q, want %q", key, q.Get(key), expected)+ }+ }+ if page.UnknownTimestampTotal == nil || *page.UnknownTimestampTotal != 2 {+ t.Fatalf("unknown time count = %v", page.UnknownTimestampTotal)+ }}func TestNilQueryOmitted(t *testing.T) {diff --git a/sdk/go/types_message.go b/sdk/go/types_message.goindex 377405132..5af7e803f 100644--- a/sdk/go/types_message.go+++ b/sdk/go/types_message.go@@ -3,6 +3,7 @@ package openwaimport ("encoding/json""net/url"+ "strconv")// MessageResponse is the acknowledgement for a sent message.@@ -173,10 +174,17 @@ type EditMessageRequest struct {// ListMessagesQuery filters GET /sessions/:id/messages.type ListMessagesQuery struct {- ChatID *string- From *string- Limit *int- Offset *int+ // Inclusive lower/exclusive upper message-time bounds, Unix epoch milliseconds.+ Since *float64+ Until *float64+ Direction *string+ MessageID *string+ OrderBy *string+ Type *string+ ChatID *string+ From *string+ Limit *int+ Offset *int// After is a keyset cursor: the id of the last message of the previous page. Takes// precedence over Offset.After *string@@ -187,6 +195,16 @@ type ListMessagesQuery struct {func (q *ListMessagesQuery) values() url.Values {v := url.Values{}+ if q.Since != nil {+ v.Set("since", strconv.FormatFloat(*q.Since, 'f', -1, 64))+ }+ if q.Until != nil {+ v.Set("until", strconv.FormatFloat(*q.Until, 'f', -1, 64))+ }+ setStr(v, "direction", q.Direction)+ setStr(v, "orderBy", q.OrderBy)+ setStr(v, "messageId", q.MessageID)+ setStr(v, "type", q.Type)setStr(v, "chatId", q.ChatID)setStr(v, "from", q.From)setInt(v, "limit", q.Limit)@@ -245,8 +263,9 @@ type MessageRecord struct {// MessageListResponse is the paginated message list payload.type MessageListResponse struct {- Messages []MessageRecord `json:"messages"`- Total int `json:"total"`+ UnknownTimestampTotal *int `json:"unknownTimestampTotal,omitempty"`+ Messages []MessageRecord `json:"messages"`+ Total int `json:"total"`}// ChatHistoryMedia is the media block on a live history message.diff --git a/sdk/java/src/main/java/com/rmyndharis/openwa/model/ListMessagesQuery.java b/sdk/java/src/main/java/com/rmyndharis/openwa/model/ListMessagesQuery.javaindex 96b543ace..854be6fc1 100644--- a/sdk/java/src/main/java/com/rmyndharis/openwa/model/ListMessagesQuery.java+++ b/sdk/java/src/main/java/com/rmyndharis/openwa/model/ListMessagesQuery.java@@ -2,7 +2,12 @@ package com.rmyndharis.openwa.model;/** Query parameters for {@code GET /sessions/:id/messages}. Null fields are omitted. */public record ListMessagesQuery(- String chatId, String from, Integer limit, Integer offset, String after, Boolean inlineMedia) {+ String chatId, String from, Integer limit, Integer offset, String after, Boolean inlineMedia,+ Double since, Double until, String direction, String orderBy, String type, String messageId) {+ public ListMessagesQuery(String chatId, String from, Integer limit, Integer offset, String after, Boolean inlineMedia) {+ this(chatId, from, limit, offset, after, inlineMedia, null, null, null, null, null, null);+ }+public static Builder builder() {return new Builder();}@@ -14,6 +19,39 @@ public record ListMessagesQuery(private Integer offset;private String after;private Boolean inlineMedia;+ private Double since;+ private Double until;+ private String direction;+ private String orderBy;+ private String type;+ private String messageId;+ public Builder messageId(String v) {+ this.messageId = v;+ return this;+ }+ public Builder type(String v) {+ this.type = v;+ return this;+ }++ /** Inclusive message-time lower bound, Unix epoch milliseconds. */+ public Builder since(Double v) {+ this.since = v;+ return this;+ }+ /** Exclusive message-time upper bound, Unix epoch milliseconds. */+ public Builder until(Double v) {+ this.until = v;+ return this;+ }+ public Builder direction(String v) {+ this.direction = v;+ return this;+ }+ public Builder orderBy(String v) {+ this.orderBy = v;+ return this;+ }public Builder chatId(String v) {this.chatId = v;@@ -48,7 +86,8 @@ public record ListMessagesQuery(}public ListMessagesQuery build() {- return new ListMessagesQuery(chatId, from, limit, offset, after, inlineMedia);+ return new ListMessagesQuery(chatId, from, limit, offset, after, inlineMedia,+ since, until, direction, orderBy, type, messageId);}}}diff --git a/sdk/java/src/main/java/com/rmyndharis/openwa/model/MessageListResponse.java b/sdk/java/src/main/java/com/rmyndharis/openwa/model/MessageListResponse.javaindex a799ffe0b..6f73b0ca8 100644--- a/sdk/java/src/main/java/com/rmyndharis/openwa/model/MessageListResponse.java+++ b/sdk/java/src/main/java/com/rmyndharis/openwa/model/MessageListResponse.java@@ -3,4 +3,8 @@ package com.rmyndharis.openwa.model;import java.util.List;/** Paginated payload returned by {@code GET /sessions/:id/messages}. */-public record MessageListResponse(List<MessageRecord> messages, int total) {}+public record MessageListResponse(List<MessageRecord> messages, int total, Integer unknownTimestampTotal) {+ public MessageListResponse(List<MessageRecord> messages, int total) {+ this(messages, total, null);+ }+}diff --git a/sdk/java/src/test/java/com/rmyndharis/openwa/resources/MessagesResourceTest.java b/sdk/java/src/test/java/com/rmyndharis/openwa/resources/MessagesResourceTest.javaindex 56aa237cf..fca3e52a4 100644--- a/sdk/java/src/test/java/com/rmyndharis/openwa/resources/MessagesResourceTest.java+++ b/sdk/java/src/test/java/com/rmyndharis/openwa/resources/MessagesResourceTest.java@@ -137,11 +137,19 @@ class MessagesResourceTest {@Testvoid listHitsMessagesPathWithQuery() {- tx.respond(200, "{\"messages\":[],\"total\":0}");- client.messages.list("s", ListMessagesQuery.builder().chatId("[email protected]").limit(10).build());+ tx.respond(200, "{\"messages\":[],\"total\":0,\"unknownTimestampTotal\":2}");+ var page = client.messages.list("s", ListMessagesQuery.builder().chatId("[email protected]").limit(10)+ .since(1789855200000d).until(1789941600000d).direction("incoming")+ .orderBy("timestamp").type("image").messageId("M1").build());assertEquals(HttpMethod.GET, tx.lastRequest().method());assertTrue(tx.lastRequest().url().startsWith("http://h/api/sessions/s/messages?"));assertTrue(tx.lastRequest().url().contains("limit=10"));+ assertTrue(tx.lastRequest().url().contains("since=1.7898552E12"));+ assertTrue(tx.lastRequest().url().contains("until=1.7899416E12"));+ for (String field : List.of("direction=incoming", "orderBy=timestamp", "type=image", "messageId=M1")) {+ assertTrue(tx.lastRequest().url().contains(field));+ }+ assertEquals(Integer.valueOf(2), page.unknownTimestampTotal());}@Testdiff --git a/sdk/javascript/src/types.ts b/sdk/javascript/src/types.tsindex 8f09addf6..a7416ddf2 100644--- a/sdk/javascript/src/types.ts+++ b/sdk/javascript/src/types.ts@@ -486,6 +486,16 @@ export interface SendPollRequest {}export interface ListMessagesQuery {+ type?: MessageType;+ /** Inclusive message-time lower bound, Unix epoch milliseconds. */+ since?: number;+ /** Exclusive message-time upper bound, Unix epoch milliseconds. */+ until?: number;+ direction?: MessageDirection;+ /** Newest first; timestamp excludes rows without known message time. Default createdAt. */+ orderBy?: 'createdAt' | 'timestamp';+ /** Exact WhatsApp message reference within the selection. */+ messageId?: string;chatId?: Jid;from?: Jid;limit?: number;@@ -633,6 +643,8 @@ export interface ChatHistoryMessage {/** Paginated payload returned by `GET /sessions/:id/messages`. */export interface MessageListResponse {+ /** Unknown message-time rows in the selected scope; present for time selections only. */+ unknownTimestampTotal?: number;messages: MessageRecord[];total: number;}diff --git a/sdk/javascript/test/messages.test.ts b/sdk/javascript/test/messages.test.tsindex 8afd03b06..860f254d7 100644--- a/sdk/javascript/test/messages.test.ts+++ b/sdk/javascript/test/messages.test.ts@@ -6,6 +6,22 @@ function client(t: MockTransport): OpenWAClient {return new OpenWAClient({ baseUrl: 'http://x', apiKey: 'k', fetch: t.asFetch() });}+it('preserves message-time filters and the unknown timestamp count', async () => {+ const t = new MockTransport().on('GET', /\/messages$/, { body: { messages: [], total: 0, unknownTimestampTotal: 2 } });+ const page = await client(t).messages.list('s', {+ since: 1789855200000.5, until: 1789941600000, direction: 'incoming',+ orderBy: 'timestamp', type: 'image', messageId: 'M1',+ });+ const query = new URL(t.lastCall!.url).searchParams;+ expect(query.get('since')).toBe('1789855200000.5');+ expect(query.get('until')).toBe('1789941600000');+ expect(query.get('direction')).toBe('incoming');+ expect(query.get('orderBy')).toBe('timestamp');+ expect(query.get('type')).toBe('image');+ expect(query.get('messageId')).toBe('M1');+ expect(page.unknownTimestampTotal).toBe(2);+});+describe('MessagesResource idempotency keys', () => {const sends: Array<[string, (c: OpenWAClient, key?: string) => Promise<unknown>]> = [['send-text', (c, key) => c.messages.sendText('s', { chatId: '[email protected]', text: 'hi', mentions: ['[email protected]'] }, key)],diff --git a/sdk/php/tests/MessagesTest.php b/sdk/php/tests/MessagesTest.phpindex 1f1e94ed3..86e4e4d0d 100644--- a/sdk/php/tests/MessagesTest.php+++ b/sdk/php/tests/MessagesTest.php@@ -58,15 +58,23 @@ class MessagesTest extends TestCase$backend = (new MockBackend())->on(200, ['messages' => [['id' => 'm1', 'body' => 'hi', 'chatId' => '[email protected]']],'total' => 1,+ 'unknownTimestampTotal' => 2,]);$client = $backend->makeClient();- $page = $client->messages->list('s1', ['limit' => 10]);+ $page = $client->messages->list('s1', [+ 'limit' => 10, 'since' => 1789855200000.5, 'until' => 1789941600000,+ 'direction' => 'incoming', 'orderBy' => 'timestamp', 'type' => 'image', 'messageId' => 'M1',+ ]);$this->assertArrayHasKey('messages', $page);$this->assertIsArray($page['messages']);$this->assertSame('m1', $page['messages'][0]['id']);$this->assertSame(1, $page['total']);$this->assertStringContainsString('/messages', $backend->lastCall()['path']);$this->assertStringContainsString('limit=10', $backend->lastCall()['query']);+ foreach (['since=1789855200000.5', 'until=1789941600000', 'direction=incoming', 'orderBy=timestamp', 'type=image', 'messageId=M1'] as $field) {+ $this->assertStringContainsString($field, $backend->lastCall()['query']);+ }+ $this->assertSame(2, $page['unknownTimestampTotal']);}public static function mediaSegments(): arraydiff --git a/sdk/python/openwa/types.py b/sdk/python/openwa/types.pyindex 6a97b4a64..d5b016c5d 100644--- a/sdk/python/openwa/types.py+++ b/sdk/python/openwa/types.py@@ -486,7 +486,9 @@ class SendPollRequest(TypedDict):ListMessagesQuery = TypedDict("ListMessagesQuery",# ``after`` is a keyset cursor: the id of the last message of the previous page.- {"chatId": Jid, "from": Jid, "limit": int, "offset": int, "after": str, "inlineMedia": bool},+ {"chatId": Jid, "from": Jid, "limit": int, "offset": int, "after": str, "inlineMedia": bool,+ "since": float, "until": float, "direction": Literal["incoming", "outgoing"],+ "orderBy": Literal["createdAt", "timestamp"], "type": str, "messageId": str},total=False,)@@ -636,6 +638,7 @@ ChatHistoryMessage = TypedDict(class MessageListResponse(TypedDict):"""Paginated payload returned by ``GET /sessions/:id/messages``."""+ unknownTimestampTotal: NotRequired[int]messages: list[MessageRecord]total: intdiff --git a/sdk/python/tests/test_sdk.py b/sdk/python/tests/test_sdk.pyindex eb7fdff6a..494098e16 100644--- a/sdk/python/tests/test_sdk.py+++ b/sdk/python/tests/test_sdk.py@@ -152,11 +152,17 @@ class TestClientCore:assert "name=my-bot" in backend.last_call.urldef test_query_params_skip_none(self):- backend = MockBackend().on("GET", "/messages", body=[])- make_client(backend).messages.list("s1", {"chatId": "[email protected]", "limit": 10})+ backend = MockBackend().on("GET", "/messages", body={"messages": [], "total": 0, "unknownTimestampTotal": 2})+ result = make_client(backend).messages.list("s1", {+ "chatId": "[email protected]", "limit": 10, "since": 1789855200000.5, "until": 1789941600000,+ "direction": "incoming", "orderBy": "timestamp", "type": "image", "messageId": "M1",+ })url = backend.last_call.urlassert "chatId=a%40c.us" in urlassert "limit=10" in url+ for field in ("since=1789855200000.5", "until=1789941600000", "direction=incoming", "orderBy=timestamp", "type=image", "messageId=M1"):+ assert field in url+ assert result["unknownTimestampTotal"] == 2def test_build_url_escapes_a_literal_percent_in_a_query_value(self):# httpx before 0.27.1 sent "%41" unescaped, so the gateway decoded it to "A".diff --git a/src/database/migrations/1790812800000-AddMessageWindowIndexes.ts b/src/database/migrations/1790812800000-AddMessageWindowIndexes.tsnew file mode 100644index 000000000..da4757017--- /dev/null+++ b/src/database/migrations/1790812800000-AddMessageWindowIndexes.ts@@ -0,0 +1,23 @@+import { MigrationInterface, QueryRunner } from 'typeorm';++const indexes = [+ ['IDX_messages_session_timestamp_id', '"sessionId", "timestamp", "id"'],+ ['IDX_messages_session_chat_timestamp_id', '"sessionId", "chatId", "timestamp", "id"'],+ ['IDX_messages_session_direction_timestamp_id', '"sessionId", "direction", "timestamp", "id"'],+] as const;++/** Match the entity schema on synchronize-disabled deployments, in both supported dialects. */+export class AddMessageWindowIndexes1790812800000 implements MigrationInterface {+ name = 'AddMessageWindowIndexes1790812800000';++ async up(queryRunner: QueryRunner): Promise<void> {+ if (queryRunner.dataSource.options.type === 'postgres') await queryRunner.query('SET LOCAL statement_timeout = 0');+ for (const [name, columns] of indexes) {+ await queryRunner.query(`CREATE INDEX IF NOT EXISTS "${name}" ON "messages" (${columns})`);+ }+ }++ async down(queryRunner: QueryRunner): Promise<void> {+ for (const [name] of indexes) await queryRunner.query(`DROP INDEX IF EXISTS "${name}"`);+ }+}diff --git a/src/modules/message/dto/message-responses.dto.ts b/src/modules/message/dto/message-responses.dto.tsindex a10d306e1..a7406872b 100644--- a/src/modules/message/dto/message-responses.dto.ts+++ b/src/modules/message/dto/message-responses.dto.ts@@ -120,6 +120,11 @@ export class MessageListItemDto {}export class MessageListResponseDto {+ @ApiPropertyOptional({+ description:+ 'Rows matching the non-time filters with unknown message time, before time bounds. Present only for time selections; excluded from their messages and total. Stored history is not proof of complete remote history.',+ })+ unknownTimestampTotal?: number;@ApiProperty({ type: [MessageListItemDto], description: 'Newest first.' })messages!: MessageListItemDto[];diff --git a/src/modules/message/entities/message.entity.ts b/src/modules/message/entities/message.entity.tsindex 50ac33d62..37ed2c258 100644--- a/src/modules/message/entities/message.entity.ts+++ b/src/modules/message/entities/message.entity.ts@@ -40,6 +40,10 @@ export enum MessageStatus {// stats reads that filter on chatId without a session. The explicit name matches the migration that// creates it on synchronize-disabled deployments, so both schema paths converge on one index.@Index('IDX_messages_sessionId_chatId_createdAt', ['sessionId', 'chatId', 'createdAt'])+// Explicit id tiebreaking keeps a message-time walk stable across SQLite and PostgreSQL.+@Index('IDX_messages_session_timestamp_id', ['sessionId', 'timestamp', 'id'])+@Index('IDX_messages_session_chat_timestamp_id', ['sessionId', 'chatId', 'timestamp', 'id'])+@Index('IDX_messages_session_direction_timestamp_id', ['sessionId', 'direction', 'timestamp', 'id'])// Composite index for the ack-driven status UPDATE (scoped by sessionId + waMessageId).// Without it every ack does a full table scan of a hot table.@Index('UQ_messages_sessionId_waMessageId', ['sessionId', 'waMessageId'], { unique: true })diff --git a/src/modules/message/message-list-ordering.pg.spec.ts b/src/modules/message/message-list-ordering.pg.spec.tsindex 7958b2398..7658cbb93 100644--- a/src/modules/message/message-list-ordering.pg.spec.ts+++ b/src/modules/message/message-list-ordering.pg.spec.ts@@ -11,6 +11,7 @@ import type { HookManager } from '../../core/hooks';import type { LidMappingStoreService } from '../../engine/identity/lid-mapping-store.service';import type { SendPacingService } from './send-pacing.service';import type { MessageSendService } from './message-send.service';+import { AddMessageWindowIndexes1790812800000 } from '../../database/migrations/1790812800000-AddMessageWindowIndexes';/*** `createdAt` is not a unique sort key. On SQLite it is whole seconds; on Postgres `NOW()` is@@ -120,6 +121,114 @@ const POSTGRES_ENABLED = process.env.DATABASE_TYPE === 'postgres';expect(distinct.size).toBe(ROWS); // no row repeated, and therefore none missed});+ it('applies fractional message-time bounds to bigint timestamps', async () => {+ const sessionId = 'sess-time-bounds';+ await repository.insert([+ ...Array.from({ length: 8 }, (_, i) => ({+ id: randomUUID(),+ sessionId,+ chatId: '[email protected]',+ from: '[email protected]',+ to: '[email protected]',+ timestamp: 1000 + Math.floor(i / 2),+ type: 'image',+ direction: i % 2 ? MessageDirection.OUTGOING : MessageDirection.INCOMING,+ })),+ {+ id: randomUUID(),+ sessionId,+ chatId: '[email protected]',+ from: '[email protected]',+ to: '[email protected]',+ type: 'image',+ direction: MessageDirection.INCOMING,+ },+ {+ id: randomUUID(),+ sessionId,+ chatId: '[email protected]',+ from: '[email protected]',+ to: '[email protected]',+ type: 'text',+ direction: MessageDirection.INCOMING,+ },+ ]);+ for (const [until, timestamps] of [+ [1003000, [1002, 1001]],+ [1002000.5, [1002, 1001]],+ [1002000, [1001]],+ ] as const) {+ const page = await service.getMessages(sessionId, {+ since: 1000000.5,+ until,+ orderBy: 'timestamp',+ direction: MessageDirection.INCOMING,+ type: 'image',+ inlineMedia: false,+ });+ expect(page.total).toBe(timestamps.length);+ expect(page.unknownTimestampTotal).toBe(1);+ expect(page.messages.map(m => m.timestamp)).toEqual(timestamps);+ }+ });++ it('walks tied message times through reversible timestamp indexes', async () => {+ const sessionId = 'sess-time-walk';+ const runner = ds.createQueryRunner();+ const migration = new AddMessageWindowIndexes1790812800000();+ await migration.up(runner);+ await migration.up(runner);+ await repository.insert(+ Array.from({ length: 240 }, (_, i) => ({+ id: randomUUID(),+ sessionId,+ chatId: '[email protected]',+ from: '[email protected]',+ to: '[email protected]',+ timestamp: 1000 + Math.floor(i / 4),+ })),+ );+ const ids: string[] = [];+ let after: string | undefined;+ do {+ const page = await service.getMessages(sessionId, {+ since: 1000000,+ until: 1060000,+ orderBy: 'timestamp',+ after,+ limit: 31,+ });+ expect(page.total).toBe(240);+ ids.push(...page.messages.map(m => m.id));+ if (!after)+ await repository.insert({+ id: randomUUID(),+ sessionId,+ chatId: '[email protected]',+ from: '[email protected]',+ to: '[email protected]',+ timestamp: 1060,+ });+ if (page.messages.length < 31) break;+ after = page.messages.at(-1)!.id;+ } while (ids.length < 300);+ expect(ids).toHaveLength(240);+ expect(new Set(ids).size).toBe(240);+ const names = async (): Promise<string[]> =>+ (+ (await runner.query(+ `SELECT indexname FROM pg_indexes WHERE tablename = 'messages' AND indexname LIKE 'IDX_messages_session_%timestamp_id'`,+ )) as Array<{ indexname: string }>+ ).map(row => row.indexname);+ expect(await names()).toHaveLength(3);+ await migration.down(runner);+ expect(await names()).toEqual([]);+ expect(await repository.countBy({ sessionId })).toBe(241);+ await migration.up(runner);+ expect(await names()).toHaveLength(3);+ await runner.release();+ });+/*** A chat filter on a user id expands to several dialect candidates, which picks a SQLite-only sort* key there. PostgreSQL must keep the plain one, so the chat-filtered walk runs on the real server.diff --git a/src/modules/message/message-window.spec.ts b/src/modules/message/message-window.spec.tsnew file mode 100644index 000000000..817a70f74--- /dev/null+++ b/src/modules/message/message-window.spec.ts@@ -0,0 +1,226 @@+import 'reflect-metadata';+import { randomUUID } from 'crypto';+import { DataSource, Logger } from 'typeorm';+import { MessageService } from './message.service';+import { Message, MessageDirection } from './entities/message.entity';+import { parseMessageWindow, validateMessageWindow } from './message-window';+import { AddMessageWindowIndexes1790812800000 } from '../../database/migrations/1790812800000-AddMessageWindowIndexes';++describe('message-time selection', () => {+ let ds: DataSource;+ let service: MessageService;+ const queries: Array<{ sql: string; params?: unknown[] }> = [];+ const logger: Logger = {+ logQuery: (sql, params) => queries.push({ sql, params: params as unknown[] }),+ logQueryError: () => undefined,+ logQuerySlow: () => undefined,+ logSchemaBuild: () => undefined,+ logMigration: () => undefined,+ log: () => undefined,+ };++ beforeEach(async () => {+ ds = new DataSource({+ type: 'better-sqlite3',+ database: ':memory:',+ entities: [Message],+ synchronize: true,+ logging: ['query'],+ logger,+ });+ await ds.initialize();+ service = new MessageService(+ ds.getRepository(Message),+ {} as never,+ {} as never,+ {} as never,+ { findLidsForPhone: () => Promise.resolve([]), findPhoneForLid: () => Promise.resolve(null) } as never,+ {} as never,+ {} as never,+ );+ const rows = Array.from({ length: 240 }, (_, i) => ({+ id: randomUUID(),+ sessionId: 's1',+ chatId: '[email protected]',+ from: '[email protected]',+ to: 'me',+ waMessageId: `m${i}`,+ body: `m${i}`,+ timestamp: 1000 + Math.floor(i / 4),+ direction: i % 2 ? MessageDirection.OUTGOING : MessageDirection.INCOMING,+ // Reverse ingestion order and repeated times: message time cannot be inferred from createdAt.+ createdAt: new Date((2000 - Math.floor(i / 3)) * 1000),+ }));+ await ds.getRepository(Message).save(rows);+ await ds+ .getRepository(Message)+ .save({ sessionId: 's1', chatId: '[email protected]', from: 'x', to: 'me', body: 'unknown time' });+ queries.length = 0;+ });+ afterEach(async () => {+ await ds.destroy();+ });++ it('reads one exact reference without widening its session or chat', async () => {+ const selected = await service.getMessages('s1', { chatId: '[email protected]', messageId: 'm42', inlineMedia: false });+ expect(selected.total).toBe(1);+ expect(selected.messages[0].body).toBe('m42');+ expect((await service.getMessages('s2', { messageId: 'm42' })).total).toBe(0);+ expect((await service.getMessages('s1', { chatId: '[email protected]', messageId: 'm42' })).total).toBe(0);+ expect(() => parseMessageWindow({ messageId: '' })).toThrow();+ const foreign = await ds.getRepository(Message).findOneByOrFail({ waMessageId: 'm43' });+ await expect(service.getMessages('s1', { messageId: 'm42', after: foreign.id })).rejects.toThrow('Unknown cursor');+ });++ it('uses message time, inclusive since and exclusive until, without flooring milliseconds', async () => {+ const selected = await service.getMessages('s1', {+ since: 1000_001,+ until: 1003_000,+ orderBy: 'timestamp',+ direction: MessageDirection.INCOMING,+ });+ expect(selected.total).toBe(4);+ expect(selected.unknownTimestampTotal).toBe(0);+ expect(selected.messages.map(m => m.timestamp)).toEqual([1002, 1002, 1001, 1001]);+ expect(selected.messages.every(m => m.direction === MessageDirection.INCOMING)).toBe(true);+ });++ it('keeps zero message time distinct from a positive subsecond boundary', async () => {+ await ds.getRepository(Message).update({ waMessageId: 'm0' }, { timestamp: 0 });+ expect((await service.getMessages('s1', { messageId: 'm0', since: Number.MIN_VALUE })).total).toBe(0);+ expect((await service.getMessages('s1', { messageId: 'm0', until: Number.MIN_VALUE })).total).toBe(1);+ expect((await service.getMessages('s1', { messageId: 'm0', since: 0, until: 1 })).total).toBe(1);+ });++ it('walks more than 100 tied rows once, despite a younger arrival between pages', async () => {+ const ids: string[] = [];+ let after: string | undefined;+ do {+ const page = await service.getMessages('s1', {+ since: 1000_000,+ until: 1060_000,+ orderBy: 'timestamp',+ limit: 31,+ after,+ inlineMedia: false,+ });+ ids.push(...page.messages.map(m => m.id));+ if (!after)+ await ds+ .getRepository(Message)+ .save({ sessionId: 's1', chatId: '[email protected]', from: 'x', to: 'me', timestamp: 1060, body: 'new arrival' });+ if (page.messages.length < 31) break;+ after = page.messages.at(-1)!.id;+ } while (ids.length < 300);+ expect(ids).toHaveLength(240);+ expect(new Set(ids).size).toBe(240);+ });++ it('rejects unknown, deleted and selection-mismatched anchors instead of returning false exhaustion', async () => {+ const page = await service.getMessages('s1', { chatId: '[email protected]', orderBy: 'timestamp', limit: 1 });+ const anchor = page.messages[0];+ for (const options of [+ { chatId: '[email protected]', after: anchor.id },+ { until: 1000_000, after: anchor.id },+ { after: randomUUID() },+ ])+ await expect(service.getMessages('s1', { ...options, orderBy: 'timestamp' })).rejects.toThrow('Unknown cursor');+ await ds.getRepository(Message).delete(anchor.id);+ await expect(service.getMessages('s1', { orderBy: 'timestamp', after: anchor.id })).rejects.toThrow(+ 'Unknown cursor',+ );+ });++ it('keeps legacy ingestion order and unknown-time rows when new options are absent', async () => {+ const legacy = await service.getMessages('s1');+ expect(legacy.total).toBe(241);+ expect(legacy.messages[0].body).toBe('unknown time');+ expect((await service.getMessages('s1', { orderBy: 'timestamp' })).total).toBe(240);+ expect((await service.getMessages('s1', { orderBy: 'timestamp' })).unknownTimestampTotal).toBe(1);+ await ds.getRepository(Message).save({ sessionId: 's1', chatId: 'status@broadcast', from: 'x', to: 'me' });+ expect((await service.getMessages('s1')).total).toBe(242);+ });+ it('filters message type without requiring search text', async () => {+ const repo = ds.getRepository(Message);+ const row = (await repo.find({ take: 1 }))[0];+ await repo.update(row.id, { type: 'image', timestamp: 1001 });+ const page = await service.getMessages('s1', {+ since: 1000_000,+ until: 1060_000,+ orderBy: 'timestamp',+ type: 'image',+ });+ expect(page.total).toBe(1);+ expect(page.messages.map(m => m.type)).toEqual(['image']);+ });++ it.each([+ [{ orderBy: 'timestamp' }, 'IDX_messages_session_timestamp_id'],+ [{ orderBy: 'timestamp', chatId: '[email protected]' }, 'IDX_messages_session_chat_timestamp_id'],+ [{ orderBy: 'timestamp', direction: MessageDirection.INCOMING }, 'IDX_messages_session_direction_timestamp_id'],+ ] as const)('seeks the bounded time window through its compound index (%p)', async (options, index) => {+ await service.getMessages('s1', { ...options, since: 1000_000, until: 1060_000 });+ const page = queries.find(q => q.sql.includes('ORDER BY') && q.sql.includes('LIMIT'))!;+ const plan = await ds.query<{ detail: string }[]>(`EXPLAIN QUERY PLAN ${page.sql}`, page.params);+ expect(plan.map((r: { detail: string }) => r.detail).join(' ')).toContain(index);+ expect(plan.map((r: { detail: string }) => r.detail).join(' ')).not.toContain('TEMP B-TREE');+ });++ it('migration preserves rows when applied and reverted', async () => {+ const runner = ds.createQueryRunner();+ const migration = new AddMessageWindowIndexes1790812800000();+ await migration.up(runner);+ await migration.up(runner);+ const indexes = (await runner.query('PRAGMA index_list(messages)')) as Array<{ name: string }>;+ const names = indexes.map(r => r.name);+ expect(names.filter((n: string) => n.startsWith('IDX_messages_session_')).sort()).toEqual([+ 'IDX_messages_session_chat_timestamp_id',+ 'IDX_messages_session_direction_timestamp_id',+ 'IDX_messages_session_timestamp_id',+ ]);+ await migration.down(runner);+ expect(await ds.getRepository(Message).count()).toBe(241);+ const reverted = (await runner.query('PRAGMA index_list(messages)')) as Array<{ name: string }>;+ expect(reverted.some(row => row.name.startsWith('IDX_messages_session_'))).toBe(false);+ await migration.up(runner);+ await runner.release();+ });+});++describe('message-window HTTP validation', () => {+ it('keeps absent options absent and parses fractional millisecond bounds', () => {+ expect(parseMessageWindow({})).toEqual({});+ expect(+ parseMessageWindow({ since: '1000000.5', until: '1001000', direction: 'incoming', orderBy: 'timestamp' }),+ ).toEqual({+ since: 1000000.5,+ until: 1001000,+ direction: 'incoming',+ orderBy: 'timestamp',+ });+ });+ it('accepts scientific notation used by SDK numeric query serializers', () => {+ expect(parseMessageWindow({ since: '1.7898552E12', until: '1.7899416E12' })).toEqual({+ since: 1789855200000,+ until: 1789941600000,+ });+ });+ it.each(['', ' ', 'NaN', 'Infinity', '-1', '0x10', '123ms', '1e400', '9007199254740992'])(+ 'rejects invalid HTTP timestamps %p',+ since => {+ expect(() => parseMessageWindow({ since })).toThrow();+ },+ );+ it.each([+ { since: NaN },+ { until: Infinity },+ { since: Number.MAX_SAFE_INTEGER + 1 },+ { since: -1 },+ { since: 1, until: 1 },+ { since: 2, until: 1 },+ { direction: 'any' },+ { orderBy: 'random' },+ ])('rejects invalid internal selection %p', value => {+ expect(() => validateMessageWindow(value as never)).toThrow();+ });+});diff --git a/src/modules/message/message-window.ts b/src/modules/message/message-window.tsnew file mode 100644index 000000000..c3e291b96--- /dev/null+++ b/src/modules/message/message-window.ts@@ -0,0 +1,84 @@+import { BadRequestException } from '@nestjs/common';+import { MessageDirection } from './entities/message.entity';++export interface MessageWindow {+ /** Unix epoch milliseconds, inclusive. Selection uses WhatsApp's message time. */+ since?: number;+ /** Unix epoch milliseconds, exclusive. */+ until?: number;+ direction?: MessageDirection;+ /** Exact stored message type (e.g. text, image or voice), no full-text search required. */+ type?: string;+ /** Exact WhatsApp message reference, scoped to the same session/chat. */+ messageId?: string;+ /** Existing ingestion-time order remains the default. Both orders are newest first. */+ orderBy?: 'createdAt' | 'timestamp';+}++/** Match integer message seconds without underflowing a positive millisecond bound to zero. */+export function messageTimeBoundarySeconds(milliseconds: number): number {+ return milliseconds === 0 ? 0 : Math.max(1, Math.ceil(milliseconds / 1000));+}++export function validateMessageWindow(window: MessageWindow): void {+ for (const key of ['since', 'until'] as const) {+ const value = window[key];+ if (+ value !== undefined &&+ (typeof value !== 'number' || !Number.isFinite(value) || value < 0 || value > Number.MAX_SAFE_INTEGER)+ ) {+ throw new BadRequestException(`${key} must be Unix epoch milliseconds between 0 and ${Number.MAX_SAFE_INTEGER}`);+ }+ }+ if (window.since !== undefined && window.until !== undefined && window.since >= window.until) {+ throw new BadRequestException('since must be earlier than the exclusive until boundary');+ }+ if (window.direction !== undefined && !Object.values(MessageDirection).includes(window.direction)) {+ throw new BadRequestException('direction must be incoming or outgoing');+ }+ if (window.type !== undefined && (typeof window.type !== 'string' || !/^[a-z][a-z0-9_]{0,49}$/.test(window.type))) {+ throw new BadRequestException('type must be a non-empty message type token');+ }+ if (+ window.messageId !== undefined &&+ (typeof window.messageId !== 'string' ||+ !window.messageId.length ||+ window.messageId.length > 200 ||+ /\s/.test(window.messageId) ||+ [...window.messageId].some(char => char.charCodeAt(0) < 32 || char.charCodeAt(0) === 127))+ ) {+ throw new BadRequestException('messageId must be a non-empty WhatsApp message reference up to 200 characters');+ }+ if (window.orderBy !== undefined && !['createdAt', 'timestamp'].includes(window.orderBy)) {+ throw new BadRequestException('orderBy must be createdAt or timestamp');+ }+}++export function parseMessageWindow(query: {+ since?: string;+ until?: string;+ direction?: string;+ type?: string;+ /** Exact WhatsApp message reference, scoped to the same session/chat. */+ messageId?: string;+ orderBy?: string;+}): MessageWindow {+ const milliseconds = (key: 'since' | 'until'): number | undefined => {+ const value = query[key];+ if (value === undefined) return undefined;+ // Number('') and Number('0x10') are valid JavaScript, not valid HTTP timestamps.+ if (typeof value !== 'string' || !/^\d+(?:\.\d+)?(?:[eE][+-]?\d+)?$/.test(value))+ throw new BadRequestException(`${key} must be Unix epoch milliseconds`);+ return Number(value);+ };+ const window: MessageWindow = {+ since: milliseconds('since'),+ until: milliseconds('until'),+ direction: query.direction as MessageDirection | undefined,+ type: query.type,+ messageId: query.messageId,+ orderBy: query.orderBy as MessageWindow['orderBy'],+ };+ validateMessageWindow(window);+ return Object.fromEntries(Object.entries(window).filter(([, value]) => value !== undefined));+}diff --git a/src/modules/message/message.controller.spec.ts b/src/modules/message/message.controller.spec.tsindex 3fd848b6d..6310ff589 100644--- a/src/modules/message/message.controller.spec.ts+++ b/src/modules/message/message.controller.spec.ts@@ -13,6 +13,79 @@ import type { ApiKey } from '../auth/entities/api-key.entity';import type { SendBulkMessageDto } from './dto/bulk-message.dto';import type { Response } from 'express';+describe('MessageController - message-time selection', () => {+ const getMessages = jest.fn().mockResolvedValue({ messages: [], total: 0, unknownTimestampTotal: 0 });+ const controller = new MessageController(+ { getMessages } as unknown as MessageService,+ {} as unknown as BulkMessageService,+ new ChatScopeService(),+ );+ beforeEach(() => getMessages.mockClear());++ it('passes new HTTP options into the existing bounded list', async () => {+ await controller.getMessages(+ 's1',+ undefined,+ undefined,+ undefined,+ undefined,+ undefined,+ 'false',+ undefined,+ '1000000.5',+ '2000000',+ 'incoming',+ 'timestamp',+ );+ expect(getMessages).toHaveBeenCalledWith(+ 's1',+ expect.objectContaining({+ since: 1000000.5,+ until: 2000000,+ direction: 'incoming',+ orderBy: 'timestamp',+ inlineMedia: false,+ }),+ );+ });+ it('rejects invalid selection before accessing storage', async () => {+ await expect(+ controller.getMessages(+ 's1',+ undefined,+ undefined,+ undefined,+ undefined,+ undefined,+ undefined,+ undefined,+ '2000',+ '1000',+ ),+ ).rejects.toThrow('since');+ expect(getMessages).not.toHaveBeenCalled();+ });+ it('keeps chat-restricted reads fenced even for new filters', async () => {+ await expect(+ controller.getMessages(+ 's1',+ undefined,+ undefined,+ undefined,+ undefined,+ undefined,+ undefined,+ { allowedChats: ['[email protected]'] } as ApiKey,+ '1000',+ '2000',+ 'incoming',+ 'timestamp',+ ),+ ).rejects.toThrow();+ expect(getMessages).not.toHaveBeenCalled();+ });+});+/*** `getChatMedia` serves third-party bytes from the API origin. The route shape, the roles and the* status codes are already held by the OpenAPI snapshot and the route-fence gates; the two headersdiff --git a/src/modules/message/message.controller.ts b/src/modules/message/message.controller.tsindex eef14f4bb..4d20bb83a 100644--- a/src/modules/message/message.controller.ts+++ b/src/modules/message/message.controller.ts@@ -49,6 +49,7 @@ import {import { ChatQuotedAllowed, ChatScoped, CurrentApiKey, RequireRole } from '../auth/decorators/auth.decorators';import { type ApiKey, ApiKeyRole } from '../auth/entities/api-key.entity';import { ChatScopeService } from '../auth/chat-scope.service';+import { parseMessageWindow } from './message-window';import {CHANNEL_MEDIA_501,CUSTOM_LINK_PREVIEW_501,@@ -81,6 +82,11 @@ export class MessageController {// Fenced on the optional ?chatId=: the guard checks it when present, and requireChat refuses a// chat-restricted key that omits it, so such a key reads only its own chats' stored history.+ @ApiQuery({+ name: 'messageId',+ required: false,+ description: 'Exact WhatsApp message reference; remains scoped to the session and chat.',+ })@ChatScoped('fenced')@Get()@ApiOperation({ summary: 'Get message history for a session' })@@ -119,12 +125,37 @@ export class MessageController {description: 'Message history',type: MessageListResponseDto,})+ @ApiQuery({+ name: 'since',+ required: false,+ type: Number,+ description: 'Inclusive message-time lower bound, Unix epoch milliseconds (not ingestion time).',+ })+ @ApiQuery({+ name: 'until',+ required: false,+ type: Number,+ description: 'Exclusive message-time upper bound, Unix epoch milliseconds.',+ })+ @ApiQuery({ name: 'direction', required: false, enum: ['incoming', 'outgoing'] })+ @ApiQuery({+ name: 'type',+ required: false,+ type: String,+ description: 'Exact stored message type token, e.g. text, image, voice.',+ })+ @ApiQuery({+ name: 'orderBy',+ required: false,+ enum: ['createdAt', 'timestamp'],+ description:+ 'Newest first. Default createdAt preserves existing ordering; timestamp uses message time and excludes rows without a known timestamp.',+ })@ApiResponse({status: 400,description:- '`after` names no message in this session. The keyset comparison would otherwise return an ' +- 'empty page, which reads exactly like the end of the history, so a walk resumed from a stale ' +- 'or foreign cursor would stop silently instead of reporting the cursor.',+ 'Invalid message filters or an unknown cursor. With time, direction, type or message-reference filters, ' ++ 'the cursor must belong to the same selection.',})@ApiResponse({status: 403,@@ -141,9 +172,16 @@ export class MessageController {@Query('after') after?: string,@Query('inlineMedia') inlineMedia?: string,@CurrentApiKey() apiKey?: ApiKey,+ @Query('since') since?: string,+ @Query('until') until?: string,+ @Query('direction') direction?: string,+ @Query('orderBy') orderBy?: string,+ @Query('type') type?: string,+ @Query('messageId') messageId?: string,) {this.chatScope.requireChat(apiKey, chatId);return this.messageService.getMessages(sessionId, {+ ...parseMessageWindow({ since, until, direction, orderBy, type, messageId }),chatId,from,limit: limit ? parseInt(limit, 10) : undefined,diff --git a/src/modules/message/message.service.ts b/src/modules/message/message.service.tsindex a03e58d99..d8d717acf 100644--- a/src/modules/message/message.service.ts+++ b/src/modules/message/message.service.ts@@ -18,11 +18,12 @@ import { MessageSendService, SaveOutgoingMessageData } from './message-send.serv// Type-only: the module binds this class to PLUGIN_MESSAGE_PORT with a `useExisting` alias, which// TypeScript does not check, so `implements` is what keeps the two in step.import type { PluginMessagePort } from '../../core/plugins/plugin-host-ports';+import { MessageWindow, messageTimeBoundarySeconds, validateMessageWindow } from './message-window';// Re-exported for existing importers (bulk send shares the rendered-template cap with the send path).export { DEFAULT_TEMPLATE_RENDER_MAX_CHARS } from './message-send.service';-export interface GetMessagesOptions {+export interface GetMessagesOptions extends MessageWindow {chatId?: string;/** Filter by sender. A phone matches stored `@c.us`/`@s.whatsapp.net` ids AND any lid resolving to it. Group messages match on `author` (the real sender) as well as `from` (which holds the group JID). */from?: string;@@ -297,8 +298,10 @@ export class MessageService implements PluginMessagePort {async getMessages(sessionId: string,options: GetMessagesOptions = {},- ): Promise<{ messages: Message[]; total: number }> {+ ): Promise<{ messages: Message[]; total: number; unknownTimestampTotal?: number }> {const { chatId, from, after, inlineMedia } = options;+ validateMessageWindow(options);+ const messageTimeOrder = options.orderBy === 'timestamp';// Sanitize pagination: a non-finite limit/offset — e.g. `?limit=abc` -> NaN —// must never reach TypeORM's take()/skip(). Clamp to sane bounds; fall back to defaults.const rawLimit = options.limit;@@ -307,7 +310,7 @@ export class MessageService implements PluginMessagePort {typeof rawLimit === 'number' && Number.isFinite(rawLimit) ? Math.min(Math.max(Math.trunc(rawLimit), 1), 100) : 50;const offset = typeof rawOffset === 'number' && Number.isFinite(rawOffset) ? Math.max(Math.trunc(rawOffset), 0) : 0;- const tiebreak = this.orderTiebreak;+ const tiebreak = messageTimeOrder ? 'id' : this.orderTiebreak;// Match across dialects: a stored chatId may be `@s.whatsapp.net` (e.g. an outbound send addressed// by a raw engine id) while the caller filters by the neutral `@c.us` from the chat list - same// chat, different dialect. Resolving both sides through the table keeps them equal.@@ -318,18 +321,18 @@ export class MessageService implements PluginMessagePort {// consideration, so it seeks (sessionId, chatId, createdAt) and sorts only that chat's rows, the// plan ANALYZE would pick. A single candidate already uses that index in order, and PostgreSQL// (no unary `+` on timestamps) has statistics, so both keep the plain key.- const createdAtKey =- tiebreak === 'rowid' && chatIds !== undefined && chatIds.length > 1 ? '+message.createdAt' : 'message.createdAt';+ const orderColumn = messageTimeOrder ? 'timestamp' : 'createdAt';+ const orderKey =+ this.orderTiebreak === 'rowid' && chatIds !== undefined && chatIds.length > 1+ ? `+message.${orderColumn}`+ : `message.${orderColumn}`;const query = this.messageRepository.createQueryBuilder('message').where('message.sessionId = :sessionId', { sessionId })- .orderBy(createdAtKey, 'DESC')- // `createdAt` is not unique: SQLite stores whole seconds, Postgres NOW() is transaction-scoped- // so a bulk write ties every row, and a history backfill stamps WhatsApp's own second-resolution- // timestamp. Without a tiebreaker the tie group's order is whatever the plan produces, and- // Postgres sorts it differently between two statements, so a page walk repeats some rows and- // never returns others. See `orderTiebreak` for why the key differs by dialect.+ .orderBy(orderKey, 'DESC')+ // Message times and bulk ingestion times can tie. The second key keeps pages stable;+ // legacy ingestion order retains its dialect-specific tiebreaker..addOrderBy(`message.${tiebreak}`, 'DESC').take(limit);@@ -343,6 +346,10 @@ export class MessageService implements PluginMessagePort {if (chatIds) {query.andWhere('message.chatId IN (:...chatIds)', { chatIds });}+ if (options.direction !== undefined) {+ query.andWhere('message.direction = :direction', { direction: options.direction });+ }+ if (options.type !== undefined) query.andWhere('message.type = :messageType', { messageType: options.type });if (from) {// Resolve the filter through the lid->phone table so a phone matches not just the stored@@ -361,6 +368,24 @@ export class MessageService implements PluginMessagePort {});}+ if (options.messageId !== undefined) {+ query.andWhere('message.waMessageId = :messageId', { messageId: options.messageId });+ }+ const hasTimeSelection = messageTimeOrder || options.since !== undefined || options.until !== undefined;+ // Unknown times cannot belong to a verified time window. Count the gap within the same+ // non-time filters, before applying time bounds, without loading message bodies.+ const unknownTimestampTotal = hasTimeSelection+ ? await query.clone().andWhere('message.timestamp IS NULL').getCount()+ : undefined;+ if (messageTimeOrder) query.andWhere('message.timestamp IS NOT NULL');+ // Integer seconds preserve both boundaries with ceil; fractional bigint parameters fail on PostgreSQL.+ if (options.since !== undefined) {+ query.andWhere('message.timestamp >= :since', { since: messageTimeBoundarySeconds(options.since) });+ }+ if (options.until !== undefined) {+ query.andWhere('message.timestamp < :until', { until: messageTimeBoundarySeconds(options.until) });+ }+// A budget of 0 means "never inline" and grants no single-payload allowance, which is exactly// what an opted-out caller asks for, so the flag picks the budget rather than a second code path.const inlineMediaBudget = inlineMedia === false ? 0 : resolveMessageListInlineMediaBudgetBytes();@@ -369,6 +394,19 @@ export class MessageService implements PluginMessagePort {const total = await query.clone().getCount();if (after !== undefined) {+ // A selection cursor must name a row inside this same selection, not merely a row from the+ // session. Otherwise an unrelated chat/time/direction could silently truncate a valid walk.+ if (+ (messageTimeOrder ||+ options.since !== undefined ||+ options.until !== undefined ||+ options.direction !== undefined ||+ options.type !== undefined ||+ options.messageId !== undefined) &&+ !(await query.clone().andWhere('message.id = :cursorId', { cursorId: after }).getCount())+ ) {+ throw new BadRequestException('Unknown cursor for this message selection');+ }// The anchor's sort key is resolved INSIDE the statement. Carrying it in the cursor instead// would mean round-tripping a timestamp through JSON, and neither dialect survives that:// SQLite holds two text shapes for one instant (`datetime('now')` writes 19 chars, a stamped@@ -376,8 +414,8 @@ export class MessageService implements PluginMessagePort {// microseconds to a millisecond Date. Both mis-seek silently, which is the very failure this// cursor exists to remove.query.andWhere(- `(message.createdAt, message.${tiebreak}) < ` +- `(SELECT anchor."createdAt", anchor."${tiebreak}" FROM messages anchor ` ++ `(message.${orderColumn}, message.${tiebreak}) < ` ++ `(SELECT anchor."${orderColumn}", anchor."${tiebreak}" FROM messages anchor ` +'WHERE anchor."id" = :after AND anchor."sessionId" = :sessionId)',{ after, sessionId },);@@ -402,9 +440,13 @@ export class MessageService implements PluginMessagePort {}// The 1..100 clamp above bounds the ROW COUNT, not the response: each row carries its inline- // base64 in metadata.media.data. Spent newest-first (the query orders createdAt DESC), so the+ // base64 in metadata.media.data. Spent newest-first in the selected order, so the// most recently viewed media still arrives inline and the rest keeps its omitted marker.- return { messages: await this.readPageSpendingBudget(sessionId, page, inlineMediaBudget), total };+ return {+ messages: await this.readPageSpendingBudget(sessionId, page, inlineMediaBudget),+ total,+ ...(unknownTimestampTotal !== undefined ? { unknownTimestampTotal } : {}),+ };}/**