Skip to content

Free, Open Source, Self-Hosted WhatsApp API Gateway

feat(messages): add message-time windows

Yudhi Armyndharis6476f0986358
diff --git a/CHANGELOG.md b/CHANGELOG.md
index b614f6ab3..9a0d4c80f 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
+- Filter stored messages by time, direction, type or message reference, with stable message-time pagination.
- Expose poll choices and selection mode in message events, stored messages, history, and dashboard chats.
- Expose optional last-message types in chat lists and SDKs.
- Add bounded webhook failure redrive with optional payload retention.
diff --git a/docs/05-database-design.md b/docs/05-database-design.md
index 153e401b4..1d631930e 100644
--- a/docs/05-database-design.md
+++ b/docs/05-database-design.md
@@ -915,7 +915,8 @@ src/database/migrations/ # data connection (pluggable)
├── 1786950000000-ScrubNonPhoneLidMappings.ts # drops lid_mappings rows where earlier releases stored a broadcast id as the phone
├── 1787000000000-AddWebhookDeliveryFailurePayload.ts # webhook_delivery_failures.payload (optional replay copy)
├── 1787100000000-AddSendIdempotencyKeys.ts # send_idempotency_keys (opt-in send retry guard)
-└── 1787200000000-DeduplicateTerminalWebhookFailures.ts # one terminal record per webhook and idempotency key
+├── 1787200000000-DeduplicateTerminalWebhookFailures.ts # one terminal record per webhook and idempotency key
+└── 1790812800000-AddMessageWindowIndexes.ts # compound timestamp indexes for message-time pagination
```
> [!NOTE]
diff --git a/docs/06-api-specification.md b/docs/06-api-specification.md
index bf9acc6f0..ff5d4eb17 100644
--- a/docs/06-api-specification.md
+++ b/docs/06-api-specification.md
@@ -1317,14 +1317,32 @@ Get persisted message history for a session from the local DB (paginated, filter
**Query parameters**
-| Name | Type | Required | Default | Description |
-| ----------- | ------- | -------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| chatId | string | No | — | Filter by chat ID. Matched across `@c.us` / `@s.whatsapp.net` dialects via the lid-mapping table. Required for a key restricted to selected chats. |
-| from | string | No | — | Filter by sender; matches `from` or a group message's `author`. A phone also matches any lid that resolves to it. |
-| limit | integer | No | 50 | Clamped to `[1,100]`; a non-finite value falls back to 50. |
-| offset | integer | No | 0 | Clamped to `>=0`; a non-finite value falls back to 0. |
-| after | string | No | — | Keyset cursor: the `id` of the last message of the previous page. Anchors the window to a row rather than a count, so a message arriving mid-walk cannot shift it. Takes precedence over `offset`. Unknown in this session gives `400`. |
-| inlineMedia | boolean | No | true | Set `false` (or `0`) to omit every inline media payload, leaving each row's `{ omitted, sizeBytes }` marker and the media endpoint. The budget below is per response, so a paged walk pulls it afresh on every page. |
+| Name | Type | Required | Default | Description |
+| ----------- | ------- | -------- | --------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| chatId | string | No | — | Filter by chat ID. Matched across `@c.us` / `@s.whatsapp.net` dialects via the lid-mapping table. Required for a key restricted to selected chats. |
+| from | string | No | — | Filter by sender; matches `from` or a group message's `author`. A phone also matches any lid that resolves to it. |
+| limit | integer | No | 50 | Clamped to `[1,100]`; a non-finite value falls back to 50. |
+| offset | integer | No | 0 | Clamped to `>=0`; a non-finite value falls back to 0. |
+| after | string | No | — | Keyset cursor: the `id` of the last message of the previous page. Anchors the window to a row rather than a count, so a message arriving mid-walk cannot shift it. Takes precedence over `offset`. Unknown in this session gives `400`. |
+| inlineMedia | boolean | No | true | Set `false` (or `0`) to omit every inline media payload, leaving each row's `{ omitted, sizeBytes }` marker and the media endpoint. The budget below is per response, so a paged walk pulls it afresh on every page. |
+| messageId | string | No | (none) | Exact WhatsApp message reference within the session and optional chat filter. |
+| since | number | No | (none) | Inclusive lower message-time bound, Unix epoch milliseconds. |
+| until | number | No | (none) | Exclusive upper message-time bound, Unix epoch milliseconds. |
+| direction | string | No | (none) | `incoming` or `outgoing`. |
+| type | string | No | (none) | Exact stored type token, such as `text`, `image` or `voice`; no search text required. |
+| orderBy | string | No | createdAt | Newest first by ingestion time (`createdAt`) or message time (`timestamp`). |
+
+Time bounds accept decimal or scientific notation from 0 to 9007199254740991 milliseconds. Blank, malformed, non-finite, unsafe or reversed bounds return `400`. They compare against stored whole-second message times, with `since` inclusive and `until` exclusive. `orderBy=timestamp` uses the message UUID as its tie-breaker and excludes unknown message times. Without these options, ingestion-time ordering and existing stored rows are preserved.
+
+With a time bound or `orderBy=timestamp`, `unknownTimestampTotal` counts rows matching all non-time filters whose message time is unknown. Those rows are excluded from `messages` and `total`. This describes stored history; exhausting a page does not prove complete remote WhatsApp history.
+
+Keep the same filters and order when continuing with `after`. When using the new filters, the anchor must belong to the selected session, chat, sender, type, direction, message reference and time window. An unknown, deleted or mismatched anchor returns `400`. Concurrent older arrivals may appear on subsequent pages; pagination does not freeze the archive.
+
+Example (stored messages for 2026-09-20 in Europe/Vienna):
+
+```http
+GET /api/sessions/:sessionId/messages?since=1789855200000&until=1789941600000&orderBy=timestamp&inlineMedia=false&limit=100
+```
**Response** `200`
@@ -1355,11 +1373,11 @@ Get persisted message history for a session from the local DB (paginated, filter
}
```
-Each `Message`: `{ id (uuid), sessionId, waMessageId (string|null), chatId, from, to, chatName (string|null; the sender's push name, or their saved contact name when no push name was reported), author (string|null; the real sender of a group, status or broadcast-list message, where from holds the group, status@broadcast or list id; on Baileys a list message the account received is filed under the sender, so from is the sender too), body (string|null), type, direction ('incoming'|'outgoing'), timestamp (number|null), metadata (object|null), mediaPath (string|null; storage key of archived media), mediaMimetype (string|null), status ('pending'|'sent'|'delivered'|'read'|'failed'), createdAt (ISO date) }`. Ordered by `createdAt` DESC, then by a dialect-dependent second key. The tiebreaker matters: `createdAt` is not unique (SQLite stores whole seconds, a PostgreSQL bulk write ties every row it inserts, and a history backfill carries WhatsApp's own second-resolution timestamp), and without a total order two pages of one walk can repeat a row and omit another. On SQLite the second key is `rowid`, the stored insertion sequence, so messages sharing a second come back in the order they arrived. PostgreSQL has no equivalent (`ctid` moves on every ack update), so it keeps `id`, a random uuid: the walk is equally correct there, but a same-second group is not in arrival order. Note that `offset` still addresses a position by count, so a list taking concurrent writes can shift under a walk: a message arriving mid-walk pushes every older row down one, and the next page re-serves a row the previous one already returned. Pass `after` instead to walk a live chat safely; it anchors on the last row you received, which an arriving message cannot move. The response is the raw service object (no envelope). Unlike the live `IncomingMessage` shape below, this persisted `Message` does **not** carry `kind` — re-derive the chat kind from `chatId` (see `ChatKind` / `chatKind()`) if needed. When present, `metadata` may include `media`, `quotedMessage`, `call`, `reactions`, and, for Baileys inbound business prompts, `buttons: [{ id, text }, …]` so a client (including the dashboard Chats thread) can re-render the choices after reload and tap them via [`POST .../click-button`](#post-apisessionssessionidmessagesclick-button).
+Each `Message`: `{ id (uuid), sessionId, waMessageId (string|null), chatId, from, to, chatName (string|null; the sender's push name, or their saved contact name when no push name was reported), author (string|null; the real sender of a group, status or broadcast-list message, where from holds the group, status@broadcast or list id; on Baileys a list message the account received is filed under the sender, so from is the sender too), body (string|null), type, direction ('incoming'|'outgoing'), timestamp (number|null), metadata (object|null), mediaPath (string|null; storage key of archived media), mediaMimetype (string|null), status ('pending'|'sent'|'delivered'|'read'|'failed'), createdAt (ISO date) }`. By default, ordered by `createdAt` DESC, then by a dialect-dependent second key. The tiebreaker matters: `createdAt` is not unique (SQLite stores whole seconds, a PostgreSQL bulk write ties every row it inserts, and a history backfill carries WhatsApp's own second-resolution timestamp), and without a total order two pages of one walk can repeat a row and omit another. On SQLite the second key is `rowid`, the stored insertion sequence, so messages sharing a second come back in the order they arrived. PostgreSQL has no equivalent (`ctid` moves on every ack update), so it keeps `id`, a random uuid: the walk is equally correct there, but a same-second group is not in arrival order. Note that `offset` still addresses a position by count, so a list taking concurrent writes can shift under a walk: a message arriving mid-walk pushes every older row down one, and the next page re-serves a row the previous one already returned. Pass `after` instead to walk a live chat safely; it anchors on the last row you received, which an arriving message cannot move. The response is the raw service object (no envelope). Unlike the live `IncomingMessage` shape below, this persisted `Message` does **not** carry `kind` — re-derive the chat kind from `chatId` (see `ChatKind` / `chatKind()`) if needed. When present, `metadata` may include `media`, `quotedMessage`, `call`, `reactions`, and, for Baileys inbound business prompts, `buttons: [{ id, text }, …]` so a client (including the dashboard Chats thread) can re-render the choices after reload and tap them via [`POST .../click-button`](#post-apisessionssessionidmessagesclick-button).
-> **Inline media is carried up to a budget, then omitted.** `MESSAGE_LIST_INLINE_MEDIA_BUDGET_BYTES` (8 MiB of encoded base64 by default) bounds how much inline media one response may hold across its rows. A row is not a bounded object, `limit` is clamped to `[1,100]` but each row can carry its base64 in `metadata.media.data`, so a page of media rows could otherwise reach hundreds of megabytes and fail the read outright. The budget is spent newest-first, matching the `createdAt` DESC order above, so a page that cannot carry everything keeps the most recent media. Past it a payload is replaced with `{ mimetype, filename?, omitted: true, sizeBytes }`, the same marker the engine emits for inbound media over `MEDIA_DOWNLOAD_MAX_BYTES`, and the bytes remain available from [`GET /messages/:chatId/:messageId/media`](#get-apisessionssessionidmessageschatidmessageidmedia). Two rules bound the edges: the newest payload is always inlined even when it alone exceeds the budget (otherwise a single large photo would be permanently unreadable through this route), and a budget of `0` means "never inline" and grants no such allowance. The knob is validated at boot, `8MiB` would parse to 8 bytes, and is forwarded by both compose files. The MCP `MessageList` tool shares this path and the same budget. A row stored under `MESSAGE_INLINE_MEDIA=archive` arrives with the marker already in place, plus `archived: true`, its bytes live only in the chat-media archive, behind the same media route (see that route for the mode).
+> **Inline media is carried up to a budget, then omitted.** `MESSAGE_LIST_INLINE_MEDIA_BUDGET_BYTES` (8 MiB of encoded base64 by default) bounds how much inline media one response may hold across its rows. A row is not a bounded object, `limit` is clamped to `[1,100]` but each row can carry its base64 in `metadata.media.data`, so a page of media rows could otherwise reach hundreds of megabytes and fail the read outright. The budget is spent newest-first, matching the selected descending order, so a page that cannot carry everything keeps the most recent media. Past it a payload is replaced with `{ mimetype, filename?, omitted: true, sizeBytes }`, the same marker the engine emits for inbound media over `MEDIA_DOWNLOAD_MAX_BYTES`, and the bytes remain available from [`GET /messages/:chatId/:messageId/media`](#get-apisessionssessionidmessageschatidmessageidmedia). Two rules bound the edges: the newest payload is always inlined even when it alone exceeds the budget (otherwise a single large photo would be permanently unreadable through this route), and a budget of `0` means "never inline" and grants no such allowance. The knob is validated at boot, `8MiB` would parse to 8 bytes, and is forwarded by both compose files. The MCP `MessageList` tool shares this path and the same budget. A row stored under `MESSAGE_INLINE_MEDIA=archive` arrives with the marker already in place, plus `archived: true`, its bytes live only in the chat-media archive, behind the same media route (see that route for the mode).
-**Errors:** `400` `after` names no message in this session · `401` missing/invalid API key · `403` a key restricted to selected chats sent no `chatId`, or one outside its allowlist
+**Errors:** `400` invalid filters or an unknown or mismatched cursor · `401` missing/invalid API key · `403` a key restricted to selected chats sent no `chatId`, or one outside its allowlist
A blank `after` is treated as absent, the way a blank `limit` or `offset` already is, so a client
templating a cursor it has not got yet keeps the unfiltered first page rather than a `400`.
diff --git a/openapi.json b/openapi.json
index efe1d8cf3..ee6355986 100644
--- a/openapi.json
+++ b/openapi.json
@@ -4268,6 +4268,67 @@
"schema": {
"type": "boolean"
}
+ },
+ {
+ "name": "since",
+ "required": false,
+ "in": "query",
+ "description": "Inclusive message-time lower bound, Unix epoch milliseconds (not ingestion time).",
+ "schema": {
+ "type": "number"
+ }
+ },
+ {
+ "name": "until",
+ "required": false,
+ "in": "query",
+ "description": "Exclusive message-time upper bound, Unix epoch milliseconds.",
+ "schema": {
+ "type": "number"
+ }
+ },
+ {
+ "name": "direction",
+ "required": false,
+ "in": "query",
+ "schema": {
+ "enum": [
+ "incoming",
+ "outgoing"
+ ],
+ "type": "string"
+ }
+ },
+ {
+ "name": "orderBy",
+ "required": false,
+ "in": "query",
+ "description": "Newest first. Default createdAt preserves existing ordering; timestamp uses message time and excludes rows without a known timestamp.",
+ "schema": {
+ "enum": [
+ "createdAt",
+ "timestamp"
+ ],
+ "type": "string"
+ }
+ },
+ {
+ "name": "type",
+ "required": false,
+ "in": "query",
+ "description": "Exact stored message type token, e.g. text, image, voice.",
+ "schema": {
+ "type": "string"
+ }
+ },
+ {
+ "name": "messageId",
+ "required": false,
+ "in": "query",
+ "description": "Exact WhatsApp message reference; remains scoped to the session and chat.",
+ "schema": {
+ "type": "string"
+ }
}
],
"responses": {
@@ -4282,7 +4343,7 @@
}
},
"400": {
- "description": "`after` names no message in this session. The keyset comparison would otherwise return an empty page, which reads exactly like the end of the history, so a walk resumed from a stale or foreign cursor would stop silently instead of reporting the cursor.",
+ "description": "Invalid message filters or an unknown cursor. With time, direction, type or message-reference filters, the cursor must belong to the same selection.",
"content": {
"application/json": {
"schema": {
@@ -19842,6 +19903,10 @@
"MessageListResponseDto": {
"type": "object",
"properties": {
+ "unknownTimestampTotal": {
+ "type": "number",
+ "description": "Rows matching the non-time filters with unknown message time, before time bounds. Present only for time selections; excluded from their messages and total. Stored history is not proof of complete remote history."
+ },
"messages": {
"description": "Newest first.",
"type": "array",
diff --git a/sdk/go/client_test.go b/sdk/go/client_test.go
index a6dff3283..9e2fb4726 100644
--- a/sdk/go/client_test.go
+++ b/sdk/go/client_test.go
@@ -175,12 +175,14 @@ func TestListSessionsQueryName(t *testing.T) {
}
func TestQueryEncoding(t *testing.T) {
- rt := &recordTransport{status: 200, body: `{"messages":[],"total":0}`}
+ rt := &recordTransport{status: 200, body: `{"messages":[],"total":0,"unknownTimestampTotal":2}`}
c := newTestClient(t, rt)
- _, err := c.Messages.List(context.Background(), "s1", &ListMessagesQuery{
+ page, err := c.Messages.List(context.Background(), "s1", &ListMessagesQuery{
ChatID: Ptr("[email protected]"),
Limit: Ptr(10),
+ Since: Ptr(1789855200000.5), Until: Ptr(1789941600000.0),
+ Direction: Ptr("incoming"), OrderBy: Ptr("timestamp"), Type: Ptr("image"), MessageID: Ptr("M1"),
})
if err != nil {
t.Fatalf("List: %v", err)
@@ -192,6 +194,14 @@ func TestQueryEncoding(t *testing.T) {
if _, ok := q["offset"]; ok {
t.Fatal("nil offset should not appear in query")
}
+ for key, expected := range map[string]string{"since": "1789855200000.5", "until": "1789941600000", "direction": "incoming", "orderBy": "timestamp", "type": "image", "messageId": "M1"} {
+ if q.Get(key) != expected {
+ t.Fatalf("%s = %q, want %q", key, q.Get(key), expected)
+ }
+ }
+ if page.UnknownTimestampTotal == nil || *page.UnknownTimestampTotal != 2 {
+ t.Fatalf("unknown time count = %v", page.UnknownTimestampTotal)
+ }
}
func TestNilQueryOmitted(t *testing.T) {
diff --git a/sdk/go/types_message.go b/sdk/go/types_message.go
index 377405132..5af7e803f 100644
--- a/sdk/go/types_message.go
+++ b/sdk/go/types_message.go
@@ -3,6 +3,7 @@ package openwa
import (
"encoding/json"
"net/url"
+ "strconv"
)
// MessageResponse is the acknowledgement for a sent message.
@@ -173,10 +174,17 @@ type EditMessageRequest struct {
// ListMessagesQuery filters GET /sessions/:id/messages.
type ListMessagesQuery struct {
- ChatID *string
- From *string
- Limit *int
- Offset *int
+ // Inclusive lower/exclusive upper message-time bounds, Unix epoch milliseconds.
+ Since *float64
+ Until *float64
+ Direction *string
+ MessageID *string
+ OrderBy *string
+ Type *string
+ ChatID *string
+ From *string
+ Limit *int
+ Offset *int
// After is a keyset cursor: the id of the last message of the previous page. Takes
// precedence over Offset.
After *string
@@ -187,6 +195,16 @@ type ListMessagesQuery struct {
func (q *ListMessagesQuery) values() url.Values {
v := url.Values{}
+ if q.Since != nil {
+ v.Set("since", strconv.FormatFloat(*q.Since, 'f', -1, 64))
+ }
+ if q.Until != nil {
+ v.Set("until", strconv.FormatFloat(*q.Until, 'f', -1, 64))
+ }
+ setStr(v, "direction", q.Direction)
+ setStr(v, "orderBy", q.OrderBy)
+ setStr(v, "messageId", q.MessageID)
+ setStr(v, "type", q.Type)
setStr(v, "chatId", q.ChatID)
setStr(v, "from", q.From)
setInt(v, "limit", q.Limit)
@@ -245,8 +263,9 @@ type MessageRecord struct {
// MessageListResponse is the paginated message list payload.
type MessageListResponse struct {
- Messages []MessageRecord `json:"messages"`
- Total int `json:"total"`
+ UnknownTimestampTotal *int `json:"unknownTimestampTotal,omitempty"`
+ Messages []MessageRecord `json:"messages"`
+ Total int `json:"total"`
}
// ChatHistoryMedia is the media block on a live history message.
diff --git a/sdk/java/src/main/java/com/rmyndharis/openwa/model/ListMessagesQuery.java b/sdk/java/src/main/java/com/rmyndharis/openwa/model/ListMessagesQuery.java
index 96b543ace..854be6fc1 100644
--- a/sdk/java/src/main/java/com/rmyndharis/openwa/model/ListMessagesQuery.java
+++ b/sdk/java/src/main/java/com/rmyndharis/openwa/model/ListMessagesQuery.java
@@ -2,7 +2,12 @@ package com.rmyndharis.openwa.model;
/** Query parameters for {@code GET /sessions/:id/messages}. Null fields are omitted. */
public record ListMessagesQuery(
- String chatId, String from, Integer limit, Integer offset, String after, Boolean inlineMedia) {
+ String chatId, String from, Integer limit, Integer offset, String after, Boolean inlineMedia,
+ Double since, Double until, String direction, String orderBy, String type, String messageId) {
+ public ListMessagesQuery(String chatId, String from, Integer limit, Integer offset, String after, Boolean inlineMedia) {
+ this(chatId, from, limit, offset, after, inlineMedia, null, null, null, null, null, null);
+ }
+
public static Builder builder() {
return new Builder();
}
@@ -14,6 +19,39 @@ public record ListMessagesQuery(
private Integer offset;
private String after;
private Boolean inlineMedia;
+ private Double since;
+ private Double until;
+ private String direction;
+ private String orderBy;
+ private String type;
+ private String messageId;
+ public Builder messageId(String v) {
+ this.messageId = v;
+ return this;
+ }
+ public Builder type(String v) {
+ this.type = v;
+ return this;
+ }
+
+ /** Inclusive message-time lower bound, Unix epoch milliseconds. */
+ public Builder since(Double v) {
+ this.since = v;
+ return this;
+ }
+ /** Exclusive message-time upper bound, Unix epoch milliseconds. */
+ public Builder until(Double v) {
+ this.until = v;
+ return this;
+ }
+ public Builder direction(String v) {
+ this.direction = v;
+ return this;
+ }
+ public Builder orderBy(String v) {
+ this.orderBy = v;
+ return this;
+ }
public Builder chatId(String v) {
this.chatId = v;
@@ -48,7 +86,8 @@ public record ListMessagesQuery(
}
public ListMessagesQuery build() {
- return new ListMessagesQuery(chatId, from, limit, offset, after, inlineMedia);
+ return new ListMessagesQuery(chatId, from, limit, offset, after, inlineMedia,
+ since, until, direction, orderBy, type, messageId);
}
}
}
diff --git a/sdk/java/src/main/java/com/rmyndharis/openwa/model/MessageListResponse.java b/sdk/java/src/main/java/com/rmyndharis/openwa/model/MessageListResponse.java
index a799ffe0b..6f73b0ca8 100644
--- a/sdk/java/src/main/java/com/rmyndharis/openwa/model/MessageListResponse.java
+++ b/sdk/java/src/main/java/com/rmyndharis/openwa/model/MessageListResponse.java
@@ -3,4 +3,8 @@ package com.rmyndharis.openwa.model;
import java.util.List;
/** Paginated payload returned by {@code GET /sessions/:id/messages}. */
-public record MessageListResponse(List<MessageRecord> messages, int total) {}
+public record MessageListResponse(List<MessageRecord> messages, int total, Integer unknownTimestampTotal) {
+ public MessageListResponse(List<MessageRecord> messages, int total) {
+ this(messages, total, null);
+ }
+}
diff --git a/sdk/java/src/test/java/com/rmyndharis/openwa/resources/MessagesResourceTest.java b/sdk/java/src/test/java/com/rmyndharis/openwa/resources/MessagesResourceTest.java
index 56aa237cf..fca3e52a4 100644
--- a/sdk/java/src/test/java/com/rmyndharis/openwa/resources/MessagesResourceTest.java
+++ b/sdk/java/src/test/java/com/rmyndharis/openwa/resources/MessagesResourceTest.java
@@ -137,11 +137,19 @@ class MessagesResourceTest {
@Test
void listHitsMessagesPathWithQuery() {
- tx.respond(200, "{\"messages\":[],\"total\":0}");
- client.messages.list("s", ListMessagesQuery.builder().chatId("[email protected]").limit(10).build());
+ tx.respond(200, "{\"messages\":[],\"total\":0,\"unknownTimestampTotal\":2}");
+ var page = client.messages.list("s", ListMessagesQuery.builder().chatId("[email protected]").limit(10)
+ .since(1789855200000d).until(1789941600000d).direction("incoming")
+ .orderBy("timestamp").type("image").messageId("M1").build());
assertEquals(HttpMethod.GET, tx.lastRequest().method());
assertTrue(tx.lastRequest().url().startsWith("http://h/api/sessions/s/messages?"));
assertTrue(tx.lastRequest().url().contains("limit=10"));
+ assertTrue(tx.lastRequest().url().contains("since=1.7898552E12"));
+ assertTrue(tx.lastRequest().url().contains("until=1.7899416E12"));
+ for (String field : List.of("direction=incoming", "orderBy=timestamp", "type=image", "messageId=M1")) {
+ assertTrue(tx.lastRequest().url().contains(field));
+ }
+ assertEquals(Integer.valueOf(2), page.unknownTimestampTotal());
}
@Test
diff --git a/sdk/javascript/src/types.ts b/sdk/javascript/src/types.ts
index 8f09addf6..a7416ddf2 100644
--- a/sdk/javascript/src/types.ts
+++ b/sdk/javascript/src/types.ts
@@ -486,6 +486,16 @@ export interface SendPollRequest {
}
export interface ListMessagesQuery {
+ type?: MessageType;
+ /** Inclusive message-time lower bound, Unix epoch milliseconds. */
+ since?: number;
+ /** Exclusive message-time upper bound, Unix epoch milliseconds. */
+ until?: number;
+ direction?: MessageDirection;
+ /** Newest first; timestamp excludes rows without known message time. Default createdAt. */
+ orderBy?: 'createdAt' | 'timestamp';
+ /** Exact WhatsApp message reference within the selection. */
+ messageId?: string;
chatId?: Jid;
from?: Jid;
limit?: number;
@@ -633,6 +643,8 @@ export interface ChatHistoryMessage {
/** Paginated payload returned by `GET /sessions/:id/messages`. */
export interface MessageListResponse {
+ /** Unknown message-time rows in the selected scope; present for time selections only. */
+ unknownTimestampTotal?: number;
messages: MessageRecord[];
total: number;
}
diff --git a/sdk/javascript/test/messages.test.ts b/sdk/javascript/test/messages.test.ts
index 8afd03b06..860f254d7 100644
--- a/sdk/javascript/test/messages.test.ts
+++ b/sdk/javascript/test/messages.test.ts
@@ -6,6 +6,22 @@ function client(t: MockTransport): OpenWAClient {
return new OpenWAClient({ baseUrl: 'http://x', apiKey: 'k', fetch: t.asFetch() });
}
+it('preserves message-time filters and the unknown timestamp count', async () => {
+ const t = new MockTransport().on('GET', /\/messages$/, { body: { messages: [], total: 0, unknownTimestampTotal: 2 } });
+ const page = await client(t).messages.list('s', {
+ since: 1789855200000.5, until: 1789941600000, direction: 'incoming',
+ orderBy: 'timestamp', type: 'image', messageId: 'M1',
+ });
+ const query = new URL(t.lastCall!.url).searchParams;
+ expect(query.get('since')).toBe('1789855200000.5');
+ expect(query.get('until')).toBe('1789941600000');
+ expect(query.get('direction')).toBe('incoming');
+ expect(query.get('orderBy')).toBe('timestamp');
+ expect(query.get('type')).toBe('image');
+ expect(query.get('messageId')).toBe('M1');
+ expect(page.unknownTimestampTotal).toBe(2);
+});
+
describe('MessagesResource idempotency keys', () => {
const sends: Array<[string, (c: OpenWAClient, key?: string) => Promise<unknown>]> = [
['send-text', (c, key) => c.messages.sendText('s', { chatId: '[email protected]', text: 'hi', mentions: ['[email protected]'] }, key)],
diff --git a/sdk/php/tests/MessagesTest.php b/sdk/php/tests/MessagesTest.php
index 1f1e94ed3..86e4e4d0d 100644
--- a/sdk/php/tests/MessagesTest.php
+++ b/sdk/php/tests/MessagesTest.php
@@ -58,15 +58,23 @@ class MessagesTest extends TestCase
$backend = (new MockBackend())->on(200, [
'messages' => [['id' => 'm1', 'body' => 'hi', 'chatId' => '[email protected]']],
'total' => 1,
+ 'unknownTimestampTotal' => 2,
]);
$client = $backend->makeClient();
- $page = $client->messages->list('s1', ['limit' => 10]);
+ $page = $client->messages->list('s1', [
+ 'limit' => 10, 'since' => 1789855200000.5, 'until' => 1789941600000,
+ 'direction' => 'incoming', 'orderBy' => 'timestamp', 'type' => 'image', 'messageId' => 'M1',
+ ]);
$this->assertArrayHasKey('messages', $page);
$this->assertIsArray($page['messages']);
$this->assertSame('m1', $page['messages'][0]['id']);
$this->assertSame(1, $page['total']);
$this->assertStringContainsString('/messages', $backend->lastCall()['path']);
$this->assertStringContainsString('limit=10', $backend->lastCall()['query']);
+ foreach (['since=1789855200000.5', 'until=1789941600000', 'direction=incoming', 'orderBy=timestamp', 'type=image', 'messageId=M1'] as $field) {
+ $this->assertStringContainsString($field, $backend->lastCall()['query']);
+ }
+ $this->assertSame(2, $page['unknownTimestampTotal']);
}
public static function mediaSegments(): array
diff --git a/sdk/python/openwa/types.py b/sdk/python/openwa/types.py
index 6a97b4a64..d5b016c5d 100644
--- a/sdk/python/openwa/types.py
+++ b/sdk/python/openwa/types.py
@@ -486,7 +486,9 @@ class SendPollRequest(TypedDict):
ListMessagesQuery = TypedDict(
"ListMessagesQuery",
# ``after`` is a keyset cursor: the id of the last message of the previous page.
- {"chatId": Jid, "from": Jid, "limit": int, "offset": int, "after": str, "inlineMedia": bool},
+ {"chatId": Jid, "from": Jid, "limit": int, "offset": int, "after": str, "inlineMedia": bool,
+ "since": float, "until": float, "direction": Literal["incoming", "outgoing"],
+ "orderBy": Literal["createdAt", "timestamp"], "type": str, "messageId": str},
total=False,
)
@@ -636,6 +638,7 @@ ChatHistoryMessage = TypedDict(
class MessageListResponse(TypedDict):
"""Paginated payload returned by ``GET /sessions/:id/messages``."""
+ unknownTimestampTotal: NotRequired[int]
messages: list[MessageRecord]
total: int
diff --git a/sdk/python/tests/test_sdk.py b/sdk/python/tests/test_sdk.py
index eb7fdff6a..494098e16 100644
--- a/sdk/python/tests/test_sdk.py
+++ b/sdk/python/tests/test_sdk.py
@@ -152,11 +152,17 @@ class TestClientCore:
assert "name=my-bot" in backend.last_call.url
def test_query_params_skip_none(self):
- backend = MockBackend().on("GET", "/messages", body=[])
- make_client(backend).messages.list("s1", {"chatId": "[email protected]", "limit": 10})
+ backend = MockBackend().on("GET", "/messages", body={"messages": [], "total": 0, "unknownTimestampTotal": 2})
+ result = make_client(backend).messages.list("s1", {
+ "chatId": "[email protected]", "limit": 10, "since": 1789855200000.5, "until": 1789941600000,
+ "direction": "incoming", "orderBy": "timestamp", "type": "image", "messageId": "M1",
+ })
url = backend.last_call.url
assert "chatId=a%40c.us" in url
assert "limit=10" in url
+ for field in ("since=1789855200000.5", "until=1789941600000", "direction=incoming", "orderBy=timestamp", "type=image", "messageId=M1"):
+ assert field in url
+ assert result["unknownTimestampTotal"] == 2
def test_build_url_escapes_a_literal_percent_in_a_query_value(self):
# httpx before 0.27.1 sent "%41" unescaped, so the gateway decoded it to "A".
diff --git a/src/database/migrations/1790812800000-AddMessageWindowIndexes.ts b/src/database/migrations/1790812800000-AddMessageWindowIndexes.ts
new file mode 100644
index 000000000..da4757017
--- /dev/null
+++ b/src/database/migrations/1790812800000-AddMessageWindowIndexes.ts
@@ -0,0 +1,23 @@
+import { MigrationInterface, QueryRunner } from 'typeorm';
+
+const indexes = [
+ ['IDX_messages_session_timestamp_id', '"sessionId", "timestamp", "id"'],
+ ['IDX_messages_session_chat_timestamp_id', '"sessionId", "chatId", "timestamp", "id"'],
+ ['IDX_messages_session_direction_timestamp_id', '"sessionId", "direction", "timestamp", "id"'],
+] as const;
+
+/** Match the entity schema on synchronize-disabled deployments, in both supported dialects. */
+export class AddMessageWindowIndexes1790812800000 implements MigrationInterface {
+ name = 'AddMessageWindowIndexes1790812800000';
+
+ async up(queryRunner: QueryRunner): Promise<void> {
+ if (queryRunner.dataSource.options.type === 'postgres') await queryRunner.query('SET LOCAL statement_timeout = 0');
+ for (const [name, columns] of indexes) {
+ await queryRunner.query(`CREATE INDEX IF NOT EXISTS "${name}" ON "messages" (${columns})`);
+ }
+ }
+
+ async down(queryRunner: QueryRunner): Promise<void> {
+ for (const [name] of indexes) await queryRunner.query(`DROP INDEX IF EXISTS "${name}"`);
+ }
+}
diff --git a/src/modules/message/dto/message-responses.dto.ts b/src/modules/message/dto/message-responses.dto.ts
index a10d306e1..a7406872b 100644
--- a/src/modules/message/dto/message-responses.dto.ts
+++ b/src/modules/message/dto/message-responses.dto.ts
@@ -120,6 +120,11 @@ export class MessageListItemDto {
}
export class MessageListResponseDto {
+ @ApiPropertyOptional({
+ description:
+ 'Rows matching the non-time filters with unknown message time, before time bounds. Present only for time selections; excluded from their messages and total. Stored history is not proof of complete remote history.',
+ })
+ unknownTimestampTotal?: number;
@ApiProperty({ type: [MessageListItemDto], description: 'Newest first.' })
messages!: MessageListItemDto[];
diff --git a/src/modules/message/entities/message.entity.ts b/src/modules/message/entities/message.entity.ts
index 50ac33d62..37ed2c258 100644
--- a/src/modules/message/entities/message.entity.ts
+++ b/src/modules/message/entities/message.entity.ts
@@ -40,6 +40,10 @@ export enum MessageStatus {
// stats reads that filter on chatId without a session. The explicit name matches the migration that
// creates it on synchronize-disabled deployments, so both schema paths converge on one index.
@Index('IDX_messages_sessionId_chatId_createdAt', ['sessionId', 'chatId', 'createdAt'])
+// Explicit id tiebreaking keeps a message-time walk stable across SQLite and PostgreSQL.
+@Index('IDX_messages_session_timestamp_id', ['sessionId', 'timestamp', 'id'])
+@Index('IDX_messages_session_chat_timestamp_id', ['sessionId', 'chatId', 'timestamp', 'id'])
+@Index('IDX_messages_session_direction_timestamp_id', ['sessionId', 'direction', 'timestamp', 'id'])
// Composite index for the ack-driven status UPDATE (scoped by sessionId + waMessageId).
// Without it every ack does a full table scan of a hot table.
@Index('UQ_messages_sessionId_waMessageId', ['sessionId', 'waMessageId'], { unique: true })
diff --git a/src/modules/message/message-list-ordering.pg.spec.ts b/src/modules/message/message-list-ordering.pg.spec.ts
index 7958b2398..7658cbb93 100644
--- a/src/modules/message/message-list-ordering.pg.spec.ts
+++ b/src/modules/message/message-list-ordering.pg.spec.ts
@@ -11,6 +11,7 @@ import type { HookManager } from '../../core/hooks';
import type { LidMappingStoreService } from '../../engine/identity/lid-mapping-store.service';
import type { SendPacingService } from './send-pacing.service';
import type { MessageSendService } from './message-send.service';
+import { AddMessageWindowIndexes1790812800000 } from '../../database/migrations/1790812800000-AddMessageWindowIndexes';
/**
* `createdAt` is not a unique sort key. On SQLite it is whole seconds; on Postgres `NOW()` is
@@ -120,6 +121,114 @@ const POSTGRES_ENABLED = process.env.DATABASE_TYPE === 'postgres';
expect(distinct.size).toBe(ROWS); // no row repeated, and therefore none missed
});
+ it('applies fractional message-time bounds to bigint timestamps', async () => {
+ const sessionId = 'sess-time-bounds';
+ await repository.insert([
+ ...Array.from({ length: 8 }, (_, i) => ({
+ id: randomUUID(),
+ sessionId,
+ chatId: '[email protected]',
+ timestamp: 1000 + Math.floor(i / 2),
+ type: 'image',
+ direction: i % 2 ? MessageDirection.OUTGOING : MessageDirection.INCOMING,
+ })),
+ {
+ id: randomUUID(),
+ sessionId,
+ chatId: '[email protected]',
+ type: 'image',
+ direction: MessageDirection.INCOMING,
+ },
+ {
+ id: randomUUID(),
+ sessionId,
+ chatId: '[email protected]',
+ type: 'text',
+ direction: MessageDirection.INCOMING,
+ },
+ ]);
+ for (const [until, timestamps] of [
+ [1003000, [1002, 1001]],
+ [1002000.5, [1002, 1001]],
+ [1002000, [1001]],
+ ] as const) {
+ const page = await service.getMessages(sessionId, {
+ since: 1000000.5,
+ until,
+ orderBy: 'timestamp',
+ direction: MessageDirection.INCOMING,
+ type: 'image',
+ inlineMedia: false,
+ });
+ expect(page.total).toBe(timestamps.length);
+ expect(page.unknownTimestampTotal).toBe(1);
+ expect(page.messages.map(m => m.timestamp)).toEqual(timestamps);
+ }
+ });
+
+ it('walks tied message times through reversible timestamp indexes', async () => {
+ const sessionId = 'sess-time-walk';
+ const runner = ds.createQueryRunner();
+ const migration = new AddMessageWindowIndexes1790812800000();
+ await migration.up(runner);
+ await migration.up(runner);
+ await repository.insert(
+ Array.from({ length: 240 }, (_, i) => ({
+ id: randomUUID(),
+ sessionId,
+ chatId: '[email protected]',
+ timestamp: 1000 + Math.floor(i / 4),
+ })),
+ );
+ const ids: string[] = [];
+ let after: string | undefined;
+ do {
+ const page = await service.getMessages(sessionId, {
+ since: 1000000,
+ until: 1060000,
+ orderBy: 'timestamp',
+ after,
+ limit: 31,
+ });
+ expect(page.total).toBe(240);
+ ids.push(...page.messages.map(m => m.id));
+ if (!after)
+ await repository.insert({
+ id: randomUUID(),
+ sessionId,
+ chatId: '[email protected]',
+ timestamp: 1060,
+ });
+ if (page.messages.length < 31) break;
+ after = page.messages.at(-1)!.id;
+ } while (ids.length < 300);
+ expect(ids).toHaveLength(240);
+ expect(new Set(ids).size).toBe(240);
+ const names = async (): Promise<string[]> =>
+ (
+ (await runner.query(
+ `SELECT indexname FROM pg_indexes WHERE tablename = 'messages' AND indexname LIKE 'IDX_messages_session_%timestamp_id'`,
+ )) as Array<{ indexname: string }>
+ ).map(row => row.indexname);
+ expect(await names()).toHaveLength(3);
+ await migration.down(runner);
+ expect(await names()).toEqual([]);
+ expect(await repository.countBy({ sessionId })).toBe(241);
+ await migration.up(runner);
+ expect(await names()).toHaveLength(3);
+ await runner.release();
+ });
+
/**
* A chat filter on a user id expands to several dialect candidates, which picks a SQLite-only sort
* key there. PostgreSQL must keep the plain one, so the chat-filtered walk runs on the real server.
diff --git a/src/modules/message/message-window.spec.ts b/src/modules/message/message-window.spec.ts
new file mode 100644
index 000000000..817a70f74
--- /dev/null
+++ b/src/modules/message/message-window.spec.ts
@@ -0,0 +1,226 @@
+import 'reflect-metadata';
+import { randomUUID } from 'crypto';
+import { DataSource, Logger } from 'typeorm';
+import { MessageService } from './message.service';
+import { Message, MessageDirection } from './entities/message.entity';
+import { parseMessageWindow, validateMessageWindow } from './message-window';
+import { AddMessageWindowIndexes1790812800000 } from '../../database/migrations/1790812800000-AddMessageWindowIndexes';
+
+describe('message-time selection', () => {
+ let ds: DataSource;
+ let service: MessageService;
+ const queries: Array<{ sql: string; params?: unknown[] }> = [];
+ const logger: Logger = {
+ logQuery: (sql, params) => queries.push({ sql, params: params as unknown[] }),
+ logQueryError: () => undefined,
+ logQuerySlow: () => undefined,
+ logSchemaBuild: () => undefined,
+ logMigration: () => undefined,
+ log: () => undefined,
+ };
+
+ beforeEach(async () => {
+ ds = new DataSource({
+ type: 'better-sqlite3',
+ database: ':memory:',
+ entities: [Message],
+ synchronize: true,
+ logging: ['query'],
+ logger,
+ });
+ await ds.initialize();
+ service = new MessageService(
+ ds.getRepository(Message),
+ {} as never,
+ {} as never,
+ {} as never,
+ { findLidsForPhone: () => Promise.resolve([]), findPhoneForLid: () => Promise.resolve(null) } as never,
+ {} as never,
+ {} as never,
+ );
+ const rows = Array.from({ length: 240 }, (_, i) => ({
+ id: randomUUID(),
+ sessionId: 's1',
+ chatId: '[email protected]',
+ to: 'me',
+ waMessageId: `m${i}`,
+ body: `m${i}`,
+ timestamp: 1000 + Math.floor(i / 4),
+ direction: i % 2 ? MessageDirection.OUTGOING : MessageDirection.INCOMING,
+ // Reverse ingestion order and repeated times: message time cannot be inferred from createdAt.
+ createdAt: new Date((2000 - Math.floor(i / 3)) * 1000),
+ }));
+ await ds.getRepository(Message).save(rows);
+ await ds
+ .getRepository(Message)
+ .save({ sessionId: 's1', chatId: '[email protected]', from: 'x', to: 'me', body: 'unknown time' });
+ queries.length = 0;
+ });
+ afterEach(async () => {
+ await ds.destroy();
+ });
+
+ it('reads one exact reference without widening its session or chat', async () => {
+ const selected = await service.getMessages('s1', { chatId: '[email protected]', messageId: 'm42', inlineMedia: false });
+ expect(selected.total).toBe(1);
+ expect(selected.messages[0].body).toBe('m42');
+ expect((await service.getMessages('s2', { messageId: 'm42' })).total).toBe(0);
+ expect((await service.getMessages('s1', { chatId: '[email protected]', messageId: 'm42' })).total).toBe(0);
+ expect(() => parseMessageWindow({ messageId: '' })).toThrow();
+ const foreign = await ds.getRepository(Message).findOneByOrFail({ waMessageId: 'm43' });
+ await expect(service.getMessages('s1', { messageId: 'm42', after: foreign.id })).rejects.toThrow('Unknown cursor');
+ });
+
+ it('uses message time, inclusive since and exclusive until, without flooring milliseconds', async () => {
+ const selected = await service.getMessages('s1', {
+ since: 1000_001,
+ until: 1003_000,
+ orderBy: 'timestamp',
+ direction: MessageDirection.INCOMING,
+ });
+ expect(selected.total).toBe(4);
+ expect(selected.unknownTimestampTotal).toBe(0);
+ expect(selected.messages.map(m => m.timestamp)).toEqual([1002, 1002, 1001, 1001]);
+ expect(selected.messages.every(m => m.direction === MessageDirection.INCOMING)).toBe(true);
+ });
+
+ it('keeps zero message time distinct from a positive subsecond boundary', async () => {
+ await ds.getRepository(Message).update({ waMessageId: 'm0' }, { timestamp: 0 });
+ expect((await service.getMessages('s1', { messageId: 'm0', since: Number.MIN_VALUE })).total).toBe(0);
+ expect((await service.getMessages('s1', { messageId: 'm0', until: Number.MIN_VALUE })).total).toBe(1);
+ expect((await service.getMessages('s1', { messageId: 'm0', since: 0, until: 1 })).total).toBe(1);
+ });
+
+ it('walks more than 100 tied rows once, despite a younger arrival between pages', async () => {
+ const ids: string[] = [];
+ let after: string | undefined;
+ do {
+ const page = await service.getMessages('s1', {
+ since: 1000_000,
+ until: 1060_000,
+ orderBy: 'timestamp',
+ limit: 31,
+ after,
+ inlineMedia: false,
+ });
+ ids.push(...page.messages.map(m => m.id));
+ if (!after)
+ await ds
+ .getRepository(Message)
+ .save({ sessionId: 's1', chatId: '[email protected]', from: 'x', to: 'me', timestamp: 1060, body: 'new arrival' });
+ if (page.messages.length < 31) break;
+ after = page.messages.at(-1)!.id;
+ } while (ids.length < 300);
+ expect(ids).toHaveLength(240);
+ expect(new Set(ids).size).toBe(240);
+ });
+
+ it('rejects unknown, deleted and selection-mismatched anchors instead of returning false exhaustion', async () => {
+ const page = await service.getMessages('s1', { chatId: '[email protected]', orderBy: 'timestamp', limit: 1 });
+ const anchor = page.messages[0];
+ for (const options of [
+ { chatId: '[email protected]', after: anchor.id },
+ { until: 1000_000, after: anchor.id },
+ { after: randomUUID() },
+ ])
+ await expect(service.getMessages('s1', { ...options, orderBy: 'timestamp' })).rejects.toThrow('Unknown cursor');
+ await ds.getRepository(Message).delete(anchor.id);
+ await expect(service.getMessages('s1', { orderBy: 'timestamp', after: anchor.id })).rejects.toThrow(
+ 'Unknown cursor',
+ );
+ });
+
+ it('keeps legacy ingestion order and unknown-time rows when new options are absent', async () => {
+ const legacy = await service.getMessages('s1');
+ expect(legacy.total).toBe(241);
+ expect(legacy.messages[0].body).toBe('unknown time');
+ expect((await service.getMessages('s1', { orderBy: 'timestamp' })).total).toBe(240);
+ expect((await service.getMessages('s1', { orderBy: 'timestamp' })).unknownTimestampTotal).toBe(1);
+ await ds.getRepository(Message).save({ sessionId: 's1', chatId: 'status@broadcast', from: 'x', to: 'me' });
+ expect((await service.getMessages('s1')).total).toBe(242);
+ });
+ it('filters message type without requiring search text', async () => {
+ const repo = ds.getRepository(Message);
+ const row = (await repo.find({ take: 1 }))[0];
+ await repo.update(row.id, { type: 'image', timestamp: 1001 });
+ const page = await service.getMessages('s1', {
+ since: 1000_000,
+ until: 1060_000,
+ orderBy: 'timestamp',
+ type: 'image',
+ });
+ expect(page.total).toBe(1);
+ expect(page.messages.map(m => m.type)).toEqual(['image']);
+ });
+
+ it.each([
+ [{ orderBy: 'timestamp' }, 'IDX_messages_session_timestamp_id'],
+ [{ orderBy: 'timestamp', chatId: '[email protected]' }, 'IDX_messages_session_chat_timestamp_id'],
+ [{ orderBy: 'timestamp', direction: MessageDirection.INCOMING }, 'IDX_messages_session_direction_timestamp_id'],
+ ] as const)('seeks the bounded time window through its compound index (%p)', async (options, index) => {
+ await service.getMessages('s1', { ...options, since: 1000_000, until: 1060_000 });
+ const page = queries.find(q => q.sql.includes('ORDER BY') && q.sql.includes('LIMIT'))!;
+ const plan = await ds.query<{ detail: string }[]>(`EXPLAIN QUERY PLAN ${page.sql}`, page.params);
+ expect(plan.map((r: { detail: string }) => r.detail).join(' ')).toContain(index);
+ expect(plan.map((r: { detail: string }) => r.detail).join(' ')).not.toContain('TEMP B-TREE');
+ });
+
+ it('migration preserves rows when applied and reverted', async () => {
+ const runner = ds.createQueryRunner();
+ const migration = new AddMessageWindowIndexes1790812800000();
+ await migration.up(runner);
+ await migration.up(runner);
+ const indexes = (await runner.query('PRAGMA index_list(messages)')) as Array<{ name: string }>;
+ const names = indexes.map(r => r.name);
+ expect(names.filter((n: string) => n.startsWith('IDX_messages_session_')).sort()).toEqual([
+ 'IDX_messages_session_chat_timestamp_id',
+ 'IDX_messages_session_direction_timestamp_id',
+ 'IDX_messages_session_timestamp_id',
+ ]);
+ await migration.down(runner);
+ expect(await ds.getRepository(Message).count()).toBe(241);
+ const reverted = (await runner.query('PRAGMA index_list(messages)')) as Array<{ name: string }>;
+ expect(reverted.some(row => row.name.startsWith('IDX_messages_session_'))).toBe(false);
+ await migration.up(runner);
+ await runner.release();
+ });
+});
+
+describe('message-window HTTP validation', () => {
+ it('keeps absent options absent and parses fractional millisecond bounds', () => {
+ expect(parseMessageWindow({})).toEqual({});
+ expect(
+ parseMessageWindow({ since: '1000000.5', until: '1001000', direction: 'incoming', orderBy: 'timestamp' }),
+ ).toEqual({
+ since: 1000000.5,
+ until: 1001000,
+ direction: 'incoming',
+ orderBy: 'timestamp',
+ });
+ });
+ it('accepts scientific notation used by SDK numeric query serializers', () => {
+ expect(parseMessageWindow({ since: '1.7898552E12', until: '1.7899416E12' })).toEqual({
+ since: 1789855200000,
+ until: 1789941600000,
+ });
+ });
+ it.each(['', ' ', 'NaN', 'Infinity', '-1', '0x10', '123ms', '1e400', '9007199254740992'])(
+ 'rejects invalid HTTP timestamps %p',
+ since => {
+ expect(() => parseMessageWindow({ since })).toThrow();
+ },
+ );
+ it.each([
+ { since: NaN },
+ { until: Infinity },
+ { since: Number.MAX_SAFE_INTEGER + 1 },
+ { since: -1 },
+ { since: 1, until: 1 },
+ { since: 2, until: 1 },
+ { direction: 'any' },
+ { orderBy: 'random' },
+ ])('rejects invalid internal selection %p', value => {
+ expect(() => validateMessageWindow(value as never)).toThrow();
+ });
+});
diff --git a/src/modules/message/message-window.ts b/src/modules/message/message-window.ts
new file mode 100644
index 000000000..c3e291b96
--- /dev/null
+++ b/src/modules/message/message-window.ts
@@ -0,0 +1,84 @@
+import { BadRequestException } from '@nestjs/common';
+import { MessageDirection } from './entities/message.entity';
+
+export interface MessageWindow {
+ /** Unix epoch milliseconds, inclusive. Selection uses WhatsApp's message time. */
+ since?: number;
+ /** Unix epoch milliseconds, exclusive. */
+ until?: number;
+ direction?: MessageDirection;
+ /** Exact stored message type (e.g. text, image or voice), no full-text search required. */
+ type?: string;
+ /** Exact WhatsApp message reference, scoped to the same session/chat. */
+ messageId?: string;
+ /** Existing ingestion-time order remains the default. Both orders are newest first. */
+ orderBy?: 'createdAt' | 'timestamp';
+}
+
+/** Match integer message seconds without underflowing a positive millisecond bound to zero. */
+export function messageTimeBoundarySeconds(milliseconds: number): number {
+ return milliseconds === 0 ? 0 : Math.max(1, Math.ceil(milliseconds / 1000));
+}
+
+export function validateMessageWindow(window: MessageWindow): void {
+ for (const key of ['since', 'until'] as const) {
+ const value = window[key];
+ if (
+ value !== undefined &&
+ (typeof value !== 'number' || !Number.isFinite(value) || value < 0 || value > Number.MAX_SAFE_INTEGER)
+ ) {
+ throw new BadRequestException(`${key} must be Unix epoch milliseconds between 0 and ${Number.MAX_SAFE_INTEGER}`);
+ }
+ }
+ if (window.since !== undefined && window.until !== undefined && window.since >= window.until) {
+ throw new BadRequestException('since must be earlier than the exclusive until boundary');
+ }
+ if (window.direction !== undefined && !Object.values(MessageDirection).includes(window.direction)) {
+ throw new BadRequestException('direction must be incoming or outgoing');
+ }
+ if (window.type !== undefined && (typeof window.type !== 'string' || !/^[a-z][a-z0-9_]{0,49}$/.test(window.type))) {
+ throw new BadRequestException('type must be a non-empty message type token');
+ }
+ if (
+ window.messageId !== undefined &&
+ (typeof window.messageId !== 'string' ||
+ !window.messageId.length ||
+ window.messageId.length > 200 ||
+ /\s/.test(window.messageId) ||
+ [...window.messageId].some(char => char.charCodeAt(0) < 32 || char.charCodeAt(0) === 127))
+ ) {
+ throw new BadRequestException('messageId must be a non-empty WhatsApp message reference up to 200 characters');
+ }
+ if (window.orderBy !== undefined && !['createdAt', 'timestamp'].includes(window.orderBy)) {
+ throw new BadRequestException('orderBy must be createdAt or timestamp');
+ }
+}
+
+export function parseMessageWindow(query: {
+ since?: string;
+ until?: string;
+ direction?: string;
+ type?: string;
+ /** Exact WhatsApp message reference, scoped to the same session/chat. */
+ messageId?: string;
+ orderBy?: string;
+}): MessageWindow {
+ const milliseconds = (key: 'since' | 'until'): number | undefined => {
+ const value = query[key];
+ if (value === undefined) return undefined;
+ // Number('') and Number('0x10') are valid JavaScript, not valid HTTP timestamps.
+ if (typeof value !== 'string' || !/^\d+(?:\.\d+)?(?:[eE][+-]?\d+)?$/.test(value))
+ throw new BadRequestException(`${key} must be Unix epoch milliseconds`);
+ return Number(value);
+ };
+ const window: MessageWindow = {
+ since: milliseconds('since'),
+ until: milliseconds('until'),
+ direction: query.direction as MessageDirection | undefined,
+ type: query.type,
+ messageId: query.messageId,
+ orderBy: query.orderBy as MessageWindow['orderBy'],
+ };
+ validateMessageWindow(window);
+ return Object.fromEntries(Object.entries(window).filter(([, value]) => value !== undefined));
+}
diff --git a/src/modules/message/message.controller.spec.ts b/src/modules/message/message.controller.spec.ts
index 3fd848b6d..6310ff589 100644
--- a/src/modules/message/message.controller.spec.ts
+++ b/src/modules/message/message.controller.spec.ts
@@ -13,6 +13,79 @@ import type { ApiKey } from '../auth/entities/api-key.entity';
import type { SendBulkMessageDto } from './dto/bulk-message.dto';
import type { Response } from 'express';
+describe('MessageController - message-time selection', () => {
+ const getMessages = jest.fn().mockResolvedValue({ messages: [], total: 0, unknownTimestampTotal: 0 });
+ const controller = new MessageController(
+ { getMessages } as unknown as MessageService,
+ {} as unknown as BulkMessageService,
+ new ChatScopeService(),
+ );
+ beforeEach(() => getMessages.mockClear());
+
+ it('passes new HTTP options into the existing bounded list', async () => {
+ await controller.getMessages(
+ 's1',
+ undefined,
+ undefined,
+ undefined,
+ undefined,
+ undefined,
+ 'false',
+ undefined,
+ '1000000.5',
+ '2000000',
+ 'incoming',
+ 'timestamp',
+ );
+ expect(getMessages).toHaveBeenCalledWith(
+ 's1',
+ expect.objectContaining({
+ since: 1000000.5,
+ until: 2000000,
+ direction: 'incoming',
+ orderBy: 'timestamp',
+ inlineMedia: false,
+ }),
+ );
+ });
+ it('rejects invalid selection before accessing storage', async () => {
+ await expect(
+ controller.getMessages(
+ 's1',
+ undefined,
+ undefined,
+ undefined,
+ undefined,
+ undefined,
+ undefined,
+ undefined,
+ '2000',
+ '1000',
+ ),
+ ).rejects.toThrow('since');
+ expect(getMessages).not.toHaveBeenCalled();
+ });
+ it('keeps chat-restricted reads fenced even for new filters', async () => {
+ await expect(
+ controller.getMessages(
+ 's1',
+ undefined,
+ undefined,
+ undefined,
+ undefined,
+ undefined,
+ undefined,
+ { allowedChats: ['[email protected]'] } as ApiKey,
+ '1000',
+ '2000',
+ 'incoming',
+ 'timestamp',
+ ),
+ ).rejects.toThrow();
+ expect(getMessages).not.toHaveBeenCalled();
+ });
+});
+
/**
* `getChatMedia` serves third-party bytes from the API origin. The route shape, the roles and the
* status codes are already held by the OpenAPI snapshot and the route-fence gates; the two headers
diff --git a/src/modules/message/message.controller.ts b/src/modules/message/message.controller.ts
index eef14f4bb..4d20bb83a 100644
--- a/src/modules/message/message.controller.ts
+++ b/src/modules/message/message.controller.ts
@@ -49,6 +49,7 @@ import {
import { ChatQuotedAllowed, ChatScoped, CurrentApiKey, RequireRole } from '../auth/decorators/auth.decorators';
import { type ApiKey, ApiKeyRole } from '../auth/entities/api-key.entity';
import { ChatScopeService } from '../auth/chat-scope.service';
+import { parseMessageWindow } from './message-window';
import {
CHANNEL_MEDIA_501,
CUSTOM_LINK_PREVIEW_501,
@@ -81,6 +82,11 @@ export class MessageController {
// Fenced on the optional ?chatId=: the guard checks it when present, and requireChat refuses a
// chat-restricted key that omits it, so such a key reads only its own chats' stored history.
+ @ApiQuery({
+ name: 'messageId',
+ required: false,
+ description: 'Exact WhatsApp message reference; remains scoped to the session and chat.',
+ })
@ChatScoped('fenced')
@Get()
@ApiOperation({ summary: 'Get message history for a session' })
@@ -119,12 +125,37 @@ export class MessageController {
description: 'Message history',
type: MessageListResponseDto,
})
+ @ApiQuery({
+ name: 'since',
+ required: false,
+ type: Number,
+ description: 'Inclusive message-time lower bound, Unix epoch milliseconds (not ingestion time).',
+ })
+ @ApiQuery({
+ name: 'until',
+ required: false,
+ type: Number,
+ description: 'Exclusive message-time upper bound, Unix epoch milliseconds.',
+ })
+ @ApiQuery({ name: 'direction', required: false, enum: ['incoming', 'outgoing'] })
+ @ApiQuery({
+ name: 'type',
+ required: false,
+ type: String,
+ description: 'Exact stored message type token, e.g. text, image, voice.',
+ })
+ @ApiQuery({
+ name: 'orderBy',
+ required: false,
+ enum: ['createdAt', 'timestamp'],
+ description:
+ 'Newest first. Default createdAt preserves existing ordering; timestamp uses message time and excludes rows without a known timestamp.',
+ })
@ApiResponse({
status: 400,
description:
- '`after` names no message in this session. The keyset comparison would otherwise return an ' +
- 'empty page, which reads exactly like the end of the history, so a walk resumed from a stale ' +
- 'or foreign cursor would stop silently instead of reporting the cursor.',
+ 'Invalid message filters or an unknown cursor. With time, direction, type or message-reference filters, ' +
+ 'the cursor must belong to the same selection.',
})
@ApiResponse({
status: 403,
@@ -141,9 +172,16 @@ export class MessageController {
@Query('after') after?: string,
@Query('inlineMedia') inlineMedia?: string,
@CurrentApiKey() apiKey?: ApiKey,
+ @Query('since') since?: string,
+ @Query('until') until?: string,
+ @Query('direction') direction?: string,
+ @Query('orderBy') orderBy?: string,
+ @Query('type') type?: string,
+ @Query('messageId') messageId?: string,
) {
this.chatScope.requireChat(apiKey, chatId);
return this.messageService.getMessages(sessionId, {
+ ...parseMessageWindow({ since, until, direction, orderBy, type, messageId }),
chatId,
from,
limit: limit ? parseInt(limit, 10) : undefined,
diff --git a/src/modules/message/message.service.ts b/src/modules/message/message.service.ts
index a03e58d99..d8d717acf 100644
--- a/src/modules/message/message.service.ts
+++ b/src/modules/message/message.service.ts
@@ -18,11 +18,12 @@ import { MessageSendService, SaveOutgoingMessageData } from './message-send.serv
// Type-only: the module binds this class to PLUGIN_MESSAGE_PORT with a `useExisting` alias, which
// TypeScript does not check, so `implements` is what keeps the two in step.
import type { PluginMessagePort } from '../../core/plugins/plugin-host-ports';
+import { MessageWindow, messageTimeBoundarySeconds, validateMessageWindow } from './message-window';
// Re-exported for existing importers (bulk send shares the rendered-template cap with the send path).
export { DEFAULT_TEMPLATE_RENDER_MAX_CHARS } from './message-send.service';
-export interface GetMessagesOptions {
+export interface GetMessagesOptions extends MessageWindow {
chatId?: string;
/** Filter by sender. A phone matches stored `@c.us`/`@s.whatsapp.net` ids AND any lid resolving to it. Group messages match on `author` (the real sender) as well as `from` (which holds the group JID). */
from?: string;
@@ -297,8 +298,10 @@ export class MessageService implements PluginMessagePort {
async getMessages(
sessionId: string,
options: GetMessagesOptions = {},
- ): Promise<{ messages: Message[]; total: number }> {
+ ): Promise<{ messages: Message[]; total: number; unknownTimestampTotal?: number }> {
const { chatId, from, after, inlineMedia } = options;
+ validateMessageWindow(options);
+ const messageTimeOrder = options.orderBy === 'timestamp';
// Sanitize pagination: a non-finite limit/offset — e.g. `?limit=abc` -> NaN —
// must never reach TypeORM's take()/skip(). Clamp to sane bounds; fall back to defaults.
const rawLimit = options.limit;
@@ -307,7 +310,7 @@ export class MessageService implements PluginMessagePort {
typeof rawLimit === 'number' && Number.isFinite(rawLimit) ? Math.min(Math.max(Math.trunc(rawLimit), 1), 100) : 50;
const offset = typeof rawOffset === 'number' && Number.isFinite(rawOffset) ? Math.max(Math.trunc(rawOffset), 0) : 0;
- const tiebreak = this.orderTiebreak;
+ const tiebreak = messageTimeOrder ? 'id' : this.orderTiebreak;
// Match across dialects: a stored chatId may be `@s.whatsapp.net` (e.g. an outbound send addressed
// by a raw engine id) while the caller filters by the neutral `@c.us` from the chat list - same
// chat, different dialect. Resolving both sides through the table keeps them equal.
@@ -318,18 +321,18 @@ export class MessageService implements PluginMessagePort {
// consideration, so it seeks (sessionId, chatId, createdAt) and sorts only that chat's rows, the
// plan ANALYZE would pick. A single candidate already uses that index in order, and PostgreSQL
// (no unary `+` on timestamps) has statistics, so both keep the plain key.
- const createdAtKey =
- tiebreak === 'rowid' && chatIds !== undefined && chatIds.length > 1 ? '+message.createdAt' : 'message.createdAt';
+ const orderColumn = messageTimeOrder ? 'timestamp' : 'createdAt';
+ const orderKey =
+ this.orderTiebreak === 'rowid' && chatIds !== undefined && chatIds.length > 1
+ ? `+message.${orderColumn}`
+ : `message.${orderColumn}`;
const query = this.messageRepository
.createQueryBuilder('message')
.where('message.sessionId = :sessionId', { sessionId })
- .orderBy(createdAtKey, 'DESC')
- // `createdAt` is not unique: SQLite stores whole seconds, Postgres NOW() is transaction-scoped
- // so a bulk write ties every row, and a history backfill stamps WhatsApp's own second-resolution
- // timestamp. Without a tiebreaker the tie group's order is whatever the plan produces, and
- // Postgres sorts it differently between two statements, so a page walk repeats some rows and
- // never returns others. See `orderTiebreak` for why the key differs by dialect.
+ .orderBy(orderKey, 'DESC')
+ // Message times and bulk ingestion times can tie. The second key keeps pages stable;
+ // legacy ingestion order retains its dialect-specific tiebreaker.
.addOrderBy(`message.${tiebreak}`, 'DESC')
.take(limit);
@@ -343,6 +346,10 @@ export class MessageService implements PluginMessagePort {
if (chatIds) {
query.andWhere('message.chatId IN (:...chatIds)', { chatIds });
}
+ if (options.direction !== undefined) {
+ query.andWhere('message.direction = :direction', { direction: options.direction });
+ }
+ if (options.type !== undefined) query.andWhere('message.type = :messageType', { messageType: options.type });
if (from) {
// Resolve the filter through the lid->phone table so a phone matches not just the stored
@@ -361,6 +368,24 @@ export class MessageService implements PluginMessagePort {
});
}
+ if (options.messageId !== undefined) {
+ query.andWhere('message.waMessageId = :messageId', { messageId: options.messageId });
+ }
+ const hasTimeSelection = messageTimeOrder || options.since !== undefined || options.until !== undefined;
+ // Unknown times cannot belong to a verified time window. Count the gap within the same
+ // non-time filters, before applying time bounds, without loading message bodies.
+ const unknownTimestampTotal = hasTimeSelection
+ ? await query.clone().andWhere('message.timestamp IS NULL').getCount()
+ : undefined;
+ if (messageTimeOrder) query.andWhere('message.timestamp IS NOT NULL');
+ // Integer seconds preserve both boundaries with ceil; fractional bigint parameters fail on PostgreSQL.
+ if (options.since !== undefined) {
+ query.andWhere('message.timestamp >= :since', { since: messageTimeBoundarySeconds(options.since) });
+ }
+ if (options.until !== undefined) {
+ query.andWhere('message.timestamp < :until', { until: messageTimeBoundarySeconds(options.until) });
+ }
+
// A budget of 0 means "never inline" and grants no single-payload allowance, which is exactly
// what an opted-out caller asks for, so the flag picks the budget rather than a second code path.
const inlineMediaBudget = inlineMedia === false ? 0 : resolveMessageListInlineMediaBudgetBytes();
@@ -369,6 +394,19 @@ export class MessageService implements PluginMessagePort {
const total = await query.clone().getCount();
if (after !== undefined) {
+ // A selection cursor must name a row inside this same selection, not merely a row from the
+ // session. Otherwise an unrelated chat/time/direction could silently truncate a valid walk.
+ if (
+ (messageTimeOrder ||
+ options.since !== undefined ||
+ options.until !== undefined ||
+ options.direction !== undefined ||
+ options.type !== undefined ||
+ options.messageId !== undefined) &&
+ !(await query.clone().andWhere('message.id = :cursorId', { cursorId: after }).getCount())
+ ) {
+ throw new BadRequestException('Unknown cursor for this message selection');
+ }
// The anchor's sort key is resolved INSIDE the statement. Carrying it in the cursor instead
// would mean round-tripping a timestamp through JSON, and neither dialect survives that:
// SQLite holds two text shapes for one instant (`datetime('now')` writes 19 chars, a stamped
@@ -376,8 +414,8 @@ export class MessageService implements PluginMessagePort {
// microseconds to a millisecond Date. Both mis-seek silently, which is the very failure this
// cursor exists to remove.
query.andWhere(
- `(message.createdAt, message.${tiebreak}) < ` +
- `(SELECT anchor."createdAt", anchor."${tiebreak}" FROM messages anchor ` +
+ `(message.${orderColumn}, message.${tiebreak}) < ` +
+ `(SELECT anchor."${orderColumn}", anchor."${tiebreak}" FROM messages anchor ` +
'WHERE anchor."id" = :after AND anchor."sessionId" = :sessionId)',
{ after, sessionId },
);
@@ -402,9 +440,13 @@ export class MessageService implements PluginMessagePort {
}
// The 1..100 clamp above bounds the ROW COUNT, not the response: each row carries its inline
- // base64 in metadata.media.data. Spent newest-first (the query orders createdAt DESC), so the
+ // base64 in metadata.media.data. Spent newest-first in the selected order, so the
// most recently viewed media still arrives inline and the rest keeps its omitted marker.
- return { messages: await this.readPageSpendingBudget(sessionId, page, inlineMediaBudget), total };
+ return {
+ messages: await this.readPageSpendingBudget(sessionId, page, inlineMediaBudget),
+ total,
+ ...(unknownTimestampTotal !== undefined ? { unknownTimestampTotal } : {}),
+ };
}
/**