Skip to content

Free, Open Source, Self-Hosted WhatsApp API Gateway

fix(search): withhold the relevance score from a chat-scoped query

Yudhi Armyndharis830b6e6ce231
diff --git a/docs/06-api-specification.md b/docs/06-api-specification.md
index b8373d383..4ce0f75e5 100644
--- a/docs/06-api-specification.md
+++ b/docs/06-api-specification.md
@@ -6545,7 +6545,8 @@ Search messages across sessions (active search provider).
- `provider` names which backend answered (e.g. `builtin-fts`); it is the registered `SearchProvider`
id, so it changes when a plugin backend is selected via `SEARCH_PROVIDER`.
- `score` is optional and provider-specific (rank ordering is stable within a provider; cross-provider
- scores are not comparable).
+ scores are not comparable). The built-in provider on SQLite leaves it out for a key restricted to
+ selected chats, since it is computed over the whole index.
**Errors:** `400` empty/whitespace `q`, a non-numeric `dateFrom`/`dateTo`/`limit`/`offset`, an `offset` above `100000`, or a malformed
SQLite FTS5 query (unbalanced quote/paren, bare operator) — Postgres's `websearch_to_tsquery` is
diff --git a/docs/26-global-search.md b/docs/26-global-search.md
index 4378419ff..d4c367219 100644
--- a/docs/26-global-search.md
+++ b/docs/26-global-search.md
@@ -144,8 +144,9 @@ GET /api/search?q=<term>&sessionId=<id>&chatId=<id>&direction=<incoming|outgoing
- **Chat scope applies the same way.** A key with a non-empty `allowedChats` searches only those
chats: the built-in provider narrows its hits before `limit`, `offset` and `total` apply, an explicit
`chatId` outside the allowlist answers `403`, and a plugin provider answers `403` for such a key
- (set `SEARCH_PROVIDER=builtin-fts` to allow scoped search). On SQLite a hit's relevance `score` is
- computed over the whole index, so it reflects messages outside the allowed chats too.
+ (set `SEARCH_PROVIDER=builtin-fts` to allow scoped search). On SQLite such a key's hits carry no
+ `score`, because the value is computed over the whole index and would reflect messages outside the
+ allowed chats; the hits stay in relevance order.
- **Response** is a `SearchResults` object: `{ hits: SearchHit[], total, tookMs, provider }`. Each hit
carries `messageId`, `waMessageId`, `sessionId`, `chatId`, `body`, `snippet`, `timestamp`, `type`,
`direction`, `from`, and optional `score`. `total` is an exact count (bounded; computed lazily only
diff --git a/src/modules/search/providers/builtin-fts.provider.spec.ts b/src/modules/search/providers/builtin-fts.provider.spec.ts
index 51a27d80f..57372b9b2 100644
--- a/src/modules/search/providers/builtin-fts.provider.spec.ts
+++ b/src/modules/search/providers/builtin-fts.provider.spec.ts
@@ -162,6 +162,20 @@ describe('BuiltInFtsProvider (sqlite)', () => {
expect((await provider.search({ q: 'hello', chatIds: ['c2'], sessionIds: ['s1'] })).total).toBe(0);
});
+ // bm25 weighs a term by how many messages the whole index holds, so the rank of a hit a chat-scoped
+ // caller may read still reflects chats it may not. The hits stay ordered by it; the value is withheld.
+ it('returns no score to a chat-scoped caller, and still returns one to the others', async () => {
+ const unscoped = await provider.search({ q: 'hello' });
+ expect(unscoped.hits.map(hit => typeof hit.score)).toEqual(['number', 'number']);
+
+ const scoped = await provider.search({ q: 'hello', chatIds: ['c2'] });
+ expect(scoped.hits.map(hit => hit.chatId)).toEqual(['c2']);
+ expect(scoped.hits[0].score).toBeUndefined();
+
+ const sessions = await provider.search({ q: 'hello', sessionIds: ['s1'] });
+ expect(sessions.hits.map(hit => typeof hit.score)).toEqual(['number']);
+ });
+
it.each([0, 1])('returns no hits or count for an empty compiled chat scope at offset %s', async offset => {
const result = await provider.search({ q: 'hello', chatIds: [], limit: 1, offset });
expect(result.hits).toEqual([]);
diff --git a/src/modules/search/providers/builtin-fts.provider.ts b/src/modules/search/providers/builtin-fts.provider.ts
index fb4cd1f97..462e46013 100644
--- a/src/modules/search/providers/builtin-fts.provider.ts
+++ b/src/modules/search/providers/builtin-fts.provider.ts
@@ -336,7 +336,10 @@ export class BuiltInFtsProvider implements SearchProvider, OnModuleInit {
const where: string[] = [`messages_fts MATCH ${ph()}`];
params.push(BuiltInFtsProvider.toFts5Query(q.q));
this.applyFilters(where, params, q, 'm.', ph);
- const cols = `m."id", m."waMessageId" AS wa_message_id, m."sessionId" AS session_id, m."chatId" AS chat_id, m."from" AS "from", m."body", m."timestamp", m."type", m."direction", snippet(messages_fts, 0, '<mark>', '</mark>', '…', ${MAX_SNIPPET_WORDS}) AS snippet, rank AS score`;
+ // bm25 weighs a term by how many messages the whole index holds, so the value of a hit a chat-scoped
+ // caller may read still reflects chats it may not. It keeps ordering the hits; only the value is withheld.
+ const score = q.chatIds === undefined ? 'rank' : 'NULL';
+ const cols = `m."id", m."waMessageId" AS wa_message_id, m."sessionId" AS session_id, m."chatId" AS chat_id, m."from" AS "from", m."body", m."timestamp", m."type", m."direction", snippet(messages_fts, 0, '<mark>', '</mark>', '…', ${MAX_SNIPPET_WORDS}) AS snippet, ${score} AS score`;
const sql = `SELECT ${cols} FROM messages_fts JOIN messages m ON m."rowid" = messages_fts."rowid" WHERE ${where.join(' AND ')} ORDER BY rank, m."timestamp" DESC, m."id" DESC LIMIT ${ph()} OFFSET ${ph()}`;
params.push(limit, offset);
return { sql, params };