Free, Open Source, Self-Hosted WhatsApp API Gateway
TypeScript apibotgatewayself-hostedwhatsappwhatsapp-apiwhatsapp-automationwhatsapp-botwhatsapp-gatewaywhatsapp-gateway-apiwhatsapp-webwhatsapp-web-api
fix(search): withhold the relevance score from a chat-scoped query
Yudhi Armyndharis830b6e6ce231
diff --git a/docs/06-api-specification.md b/docs/06-api-specification.mdindex b8373d383..4ce0f75e5 100644--- a/docs/06-api-specification.md+++ b/docs/06-api-specification.md@@ -6545,7 +6545,8 @@ Search messages across sessions (active search provider).- `provider` names which backend answered (e.g. `builtin-fts`); it is the registered `SearchProvider`id, so it changes when a plugin backend is selected via `SEARCH_PROVIDER`.- `score` is optional and provider-specific (rank ordering is stable within a provider; cross-provider- scores are not comparable).+ scores are not comparable). The built-in provider on SQLite leaves it out for a key restricted to+ selected chats, since it is computed over the whole index.**Errors:** `400` empty/whitespace `q`, a non-numeric `dateFrom`/`dateTo`/`limit`/`offset`, an `offset` above `100000`, or a malformedSQLite FTS5 query (unbalanced quote/paren, bare operator) — Postgres's `websearch_to_tsquery` isdiff --git a/docs/26-global-search.md b/docs/26-global-search.mdindex 4378419ff..d4c367219 100644--- a/docs/26-global-search.md+++ b/docs/26-global-search.md@@ -144,8 +144,9 @@ GET /api/search?q=<term>&sessionId=<id>&chatId=<id>&direction=<incoming|outgoing- **Chat scope applies the same way.** A key with a non-empty `allowedChats` searches only thosechats: the built-in provider narrows its hits before `limit`, `offset` and `total` apply, an explicit`chatId` outside the allowlist answers `403`, and a plugin provider answers `403` for such a key- (set `SEARCH_PROVIDER=builtin-fts` to allow scoped search). On SQLite a hit's relevance `score` is- computed over the whole index, so it reflects messages outside the allowed chats too.+ (set `SEARCH_PROVIDER=builtin-fts` to allow scoped search). On SQLite such a key's hits carry no+ `score`, because the value is computed over the whole index and would reflect messages outside the+ allowed chats; the hits stay in relevance order.- **Response** is a `SearchResults` object: `{ hits: SearchHit[], total, tookMs, provider }`. Each hitcarries `messageId`, `waMessageId`, `sessionId`, `chatId`, `body`, `snippet`, `timestamp`, `type`,`direction`, `from`, and optional `score`. `total` is an exact count (bounded; computed lazily onlydiff --git a/src/modules/search/providers/builtin-fts.provider.spec.ts b/src/modules/search/providers/builtin-fts.provider.spec.tsindex 51a27d80f..57372b9b2 100644--- a/src/modules/search/providers/builtin-fts.provider.spec.ts+++ b/src/modules/search/providers/builtin-fts.provider.spec.ts@@ -162,6 +162,20 @@ describe('BuiltInFtsProvider (sqlite)', () => {expect((await provider.search({ q: 'hello', chatIds: ['c2'], sessionIds: ['s1'] })).total).toBe(0);});+ // bm25 weighs a term by how many messages the whole index holds, so the rank of a hit a chat-scoped+ // caller may read still reflects chats it may not. The hits stay ordered by it; the value is withheld.+ it('returns no score to a chat-scoped caller, and still returns one to the others', async () => {+ const unscoped = await provider.search({ q: 'hello' });+ expect(unscoped.hits.map(hit => typeof hit.score)).toEqual(['number', 'number']);++ const scoped = await provider.search({ q: 'hello', chatIds: ['c2'] });+ expect(scoped.hits.map(hit => hit.chatId)).toEqual(['c2']);+ expect(scoped.hits[0].score).toBeUndefined();++ const sessions = await provider.search({ q: 'hello', sessionIds: ['s1'] });+ expect(sessions.hits.map(hit => typeof hit.score)).toEqual(['number']);+ });+it.each([0, 1])('returns no hits or count for an empty compiled chat scope at offset %s', async offset => {const result = await provider.search({ q: 'hello', chatIds: [], limit: 1, offset });expect(result.hits).toEqual([]);diff --git a/src/modules/search/providers/builtin-fts.provider.ts b/src/modules/search/providers/builtin-fts.provider.tsindex fb4cd1f97..462e46013 100644--- a/src/modules/search/providers/builtin-fts.provider.ts+++ b/src/modules/search/providers/builtin-fts.provider.ts@@ -336,7 +336,10 @@ export class BuiltInFtsProvider implements SearchProvider, OnModuleInit {const where: string[] = [`messages_fts MATCH ${ph()}`];params.push(BuiltInFtsProvider.toFts5Query(q.q));this.applyFilters(where, params, q, 'm.', ph);- const cols = `m."id", m."waMessageId" AS wa_message_id, m."sessionId" AS session_id, m."chatId" AS chat_id, m."from" AS "from", m."body", m."timestamp", m."type", m."direction", snippet(messages_fts, 0, '<mark>', '</mark>', '…', ${MAX_SNIPPET_WORDS}) AS snippet, rank AS score`;+ // bm25 weighs a term by how many messages the whole index holds, so the value of a hit a chat-scoped+ // caller may read still reflects chats it may not. It keeps ordering the hits; only the value is withheld.+ const score = q.chatIds === undefined ? 'rank' : 'NULL';+ const cols = `m."id", m."waMessageId" AS wa_message_id, m."sessionId" AS session_id, m."chatId" AS chat_id, m."from" AS "from", m."body", m."timestamp", m."type", m."direction", snippet(messages_fts, 0, '<mark>', '</mark>', '…', ${MAX_SNIPPET_WORDS}) AS snippet, ${score} AS score`;const sql = `SELECT ${cols} FROM messages_fts JOIN messages m ON m."rowid" = messages_fts."rowid" WHERE ${where.join(' AND ')} ORDER BY rank, m."timestamp" DESC, m."id" DESC LIMIT ${ph()} OFFSET ${ph()}`;params.push(limit, offset);return { sql, params };