Skip to content

Free, Open Source, Self-Hosted WhatsApp API Gateway

fix(search): bound the words of a query and run a repeated one once

Yudhi Armyndharis65f868c1a8f7
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 2311903be..89cfdbf8d 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -112,6 +112,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Keep one terminal webhook failure per receiver idempotency key.
- Reject poll votes with no matching options while preserving explicit vote clearing.
- Read the duration and waveform of an outgoing Baileys audio file in a separate process, so a crafted file can no longer abort or exhaust the gateway.
+- Bound `GET /api/search` to 1000 characters and 32 distinct words and run a repeated word once, so one request can no longer hold the event loop on SQLite for seconds.
### Documentation
@@ -146,6 +147,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- With `OPENWA_DATA_DIR` set, `backup.sh` and `restore.sh` read the default SQLite databases and `./data/...` paths in `./.env` under it; pass a path in the environment to keep it in the working directory.
- A storage import stops at `STORAGE_IMPORT_MAX_TOTAL_BYTES` (default 10 GiB); raise it before importing a larger export.
- Creating or updating an API key with more than 2000 `allowedChats` entries answers `400`; a stored longer list keeps working until it is edited.
+- `GET /api/search` answers `400` for a `q` over 1000 characters or with more than 32 distinct words; a repeated word counts once.
- With `DATABASE_SSL=true`, the PostgreSQL certificate is now verified against `DATABASE_HOST` even when it is an IP address (`pg` 8.23.1). A certificate that names only `localhost` stops matching a connection by IP: use the certificate's DNS name as the host or add the IP to its subject alternative names. `DATABASE_SSL_REJECT_UNAUTHORIZED=false` remains the escape hatch.
- The first boot builds three indexes on `messages` (`IDX_messages_session_timestamp_id`, `IDX_messages_session_chat_timestamp_id` and `IDX_messages_session_direction_timestamp_id`) not concurrently; on a large table boot takes longer, on PostgreSQL writes to that table wait until each build finishes, and the table with its indexes grows by about 240 MB per million messages.
- Baileys answers `400` for an animated sticker with more than 500 frames.
diff --git a/docs/06-api-specification.md b/docs/06-api-specification.md
index 6d8c51614..805610466 100644
--- a/docs/06-api-specification.md
+++ b/docs/06-api-specification.md
@@ -6500,18 +6500,18 @@ Search messages across sessions (active search provider).
**Query parameters**
-| Name | Type | Required | Default | Description |
-| ----------- | ------------------------------- | -------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `q` | string | **Yes** | — | Search term. Must be non-empty after trim; whitespace-only is rejected with `400`. Passed to the active provider's native full-text matcher. |
-| `sessionId` | string | No | — | Restrict to a single session id (intersected with the key's `allowedSessions` scope). |
-| `chatId` | string | No | — | Restrict to a single chat id. For a key restricted to selected chats this must name an allowed chat (`403` otherwise), and its hits are additionally narrowed to the allowlist. |
-| `direction` | enum (`incoming` \| `outgoing`) | No | — | Filter by message direction. |
-| `type` | string | No | — | Filter by stored message `type` (e.g. `text`, `image`, `video`). Compared against `messages.type`; not an enum validation, any string is accepted and unmatched values simply return no hits. |
-| `from` | string | No | — | Filter by sender. |
-| `dateFrom` | integer (epoch ms) | No | — | Inclusive lower bound on `timestamp`. A non-numeric value is rejected with `400`. |
-| `dateTo` | integer (epoch ms) | No | — | Inclusive upper bound on `timestamp`. A non-numeric value is rejected with `400`. |
-| `limit` | integer (≥ 1) | No | `50` | Max hits to return. Clamped to `SEARCH_LIMIT_MAX` (default `100`). A non-integer value is rejected with `400`. |
-| `offset` | integer (≥ 0) | No | `0` | Pagination offset, at most `100000`. A non-integer or larger value is rejected with `400`. |
+| Name | Type | Required | Default | Description |
+| ----------- | ------------------------------- | -------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `q` | string | **Yes** | — | Search term. Must be non-empty after trim; whitespace-only is rejected with `400`, and so is one longer than 1000 characters or with more than 32 distinct words (a repeated word counts once). Passed to the active provider's native full-text matcher. |
+| `sessionId` | string | No | — | Restrict to a single session id (intersected with the key's `allowedSessions` scope). |
+| `chatId` | string | No | — | Restrict to a single chat id. For a key restricted to selected chats this must name an allowed chat (`403` otherwise), and its hits are additionally narrowed to the allowlist. |
+| `direction` | enum (`incoming` \| `outgoing`) | No | — | Filter by message direction. |
+| `type` | string | No | — | Filter by stored message `type` (e.g. `text`, `image`, `video`). Compared against `messages.type`; not an enum validation, any string is accepted and unmatched values simply return no hits. |
+| `from` | string | No | — | Filter by sender. |
+| `dateFrom` | integer (epoch ms) | No | — | Inclusive lower bound on `timestamp`. A non-numeric value is rejected with `400`. |
+| `dateTo` | integer (epoch ms) | No | — | Inclusive upper bound on `timestamp`. A non-numeric value is rejected with `400`. |
+| `limit` | integer (≥ 1) | No | `50` | Max hits to return. Clamped to `SEARCH_LIMIT_MAX` (default `100`). A non-integer value is rejected with `400`. |
+| `offset` | integer (≥ 0) | No | `0` | Pagination offset, at most `100000`. A non-integer or larger value is rejected with `400`. |
**Response** `200` — `SearchResults`
diff --git a/openapi.json b/openapi.json
index fe6bc8d4f..c8ef1f718 100644
--- a/openapi.json
+++ b/openapi.json
@@ -17723,8 +17723,9 @@
"name": "q",
"required": true,
"in": "query",
- "description": "Search term (required, non-empty)",
+ "description": "Search term (required, non-empty, at most 1000 characters and 32 distinct words)",
"schema": {
+ "maxLength": 1000,
"type": "string"
}
},
@@ -17829,7 +17830,7 @@
}
},
"400": {
- "description": "Empty or whitespace-only \"q\"",
+ "description": "Empty or whitespace-only \"q\", or one over 1000 characters or 32 distinct words",
"content": {
"application/json": {
"schema": {
diff --git a/src/modules/search/dto/search-query.dto.spec.ts b/src/modules/search/dto/search-query.dto.spec.ts
index 4c9f276f1..e5b548dd4 100644
--- a/src/modules/search/dto/search-query.dto.spec.ts
+++ b/src/modules/search/dto/search-query.dto.spec.ts
@@ -2,7 +2,7 @@ import { DECORATORS } from '@nestjs/swagger';
import { validateSync } from 'class-validator';
import { plainToInstance } from 'class-transformer';
import { SearchQueryDto } from './search-query.dto';
-import { SEARCH_OFFSET_MAX } from '../search.constants';
+import { SEARCH_OFFSET_MAX, SEARCH_QUERY_MAX_LENGTH } from '../search.constants';
describe('SearchQueryDto', () => {
// Mirrors how the global ValidationPipe (transform:true + enableImplicitConversion) instantiates
@@ -36,6 +36,13 @@ describe('SearchQueryDto', () => {
expect(validateSync(fromQuery({ q: 'hello', offset: '0.5' })).some(e => e.property === 'offset')).toBe(true);
});
+ it('rejects a q longer than SEARCH_QUERY_MAX_LENGTH and accepts one at the bound', () => {
+ expect(validateSync(fromQuery({ q: 'a'.repeat(SEARCH_QUERY_MAX_LENGTH) }))).toHaveLength(0);
+ expect(validateSync(fromQuery({ q: 'a'.repeat(SEARCH_QUERY_MAX_LENGTH + 1) })).some(e => e.property === 'q')).toBe(
+ true,
+ );
+ });
+
it('rejects limit < 1 (@Min(1))', () => {
const dto = fromQuery({ q: 'hello', limit: '0' });
expect(validateSync(dto).some(e => e.property === 'limit')).toBe(true);
diff --git a/src/modules/search/dto/search-query.dto.ts b/src/modules/search/dto/search-query.dto.ts
index ffa0f04dd..0bfe02a94 100644
--- a/src/modules/search/dto/search-query.dto.ts
+++ b/src/modules/search/dto/search-query.dto.ts
@@ -1,10 +1,10 @@
import { ApiProperty, ApiPropertyOptional } from '@nestjs/swagger';
-import { IsString, IsNotEmpty, IsOptional, IsEnum, IsInt, IsNumber, Max, Min } from 'class-validator';
+import { IsString, IsNotEmpty, IsOptional, IsEnum, IsInt, IsNumber, Max, MaxLength, Min } from 'class-validator';
import { Type } from 'class-transformer';
import { MessageDirection } from '../../message/entities/message.entity';
import type { MessageType } from '../../../engine/interfaces/whatsapp-engine.interface';
import { ToStrictNumber } from '../../../common/utils/strict-boolean';
-import { SEARCH_OFFSET_MAX } from '../search.constants';
+import { SEARCH_OFFSET_MAX, SEARCH_QUERY_MAX_LENGTH, SEARCH_TERMS_MAX } from '../search.constants';
/**
* Request shape for GET /search. Numeric fields are coerced from query-string strings via
@@ -17,9 +17,13 @@ import { SEARCH_OFFSET_MAX } from '../search.constants';
* API-key allowedSessions/allowedChats (never user-supplied).
*/
export class SearchQueryDto {
- @ApiProperty({ description: 'Search term (required, non-empty)' })
+ @ApiProperty({
+ description: `Search term (required, non-empty, at most ${SEARCH_QUERY_MAX_LENGTH} characters and ${SEARCH_TERMS_MAX} distinct words)`,
+ maxLength: SEARCH_QUERY_MAX_LENGTH,
+ })
@IsString()
@IsNotEmpty()
+ @MaxLength(SEARCH_QUERY_MAX_LENGTH)
q!: string;
@ApiPropertyOptional({ description: 'Restrict to a single session' })
diff --git a/src/modules/search/providers/builtin-fts.provider.spec.ts b/src/modules/search/providers/builtin-fts.provider.spec.ts
index 1e04a9fdc..51a27d80f 100644
--- a/src/modules/search/providers/builtin-fts.provider.spec.ts
+++ b/src/modules/search/providers/builtin-fts.provider.spec.ts
@@ -187,6 +187,17 @@ describe('BuiltInFtsProvider (sqlite)', () => {
}
});
+ it('runs a repeated word once, since a repeat adds nothing under AND but costs a pass per match', async () => {
+ const query = jest.spyOn(ds, 'query');
+ const repeated = await provider.search({ q: Array.from({ length: 200 }, () => 'hello').join(' ') });
+ const single = await provider.search({ q: 'hello' });
+
+ const matched = query.mock.calls.filter(([sql]) => String(sql).includes('MATCH')).map(([, params]) => params);
+ expect((matched[0] as unknown[])[0]).toBe('"hello"');
+ expect((matched[1] as unknown[])[0]).toBe('"hello"');
+ expect(repeated.hits.map(h => h.messageId)).toEqual(single.hits.map(h => h.messageId));
+ });
+
it('keeps multi-token AND semantics after quoting', async () => {
// Both rows carry 'hello' AND 'world'... exactly one does ('goodbye world' lacks 'hello').
const res = await provider.search({ q: 'hello world' });
diff --git a/src/modules/search/providers/builtin-fts.provider.ts b/src/modules/search/providers/builtin-fts.provider.ts
index 079caf54a..fb4cd1f97 100644
--- a/src/modules/search/providers/builtin-fts.provider.ts
+++ b/src/modules/search/providers/builtin-fts.provider.ts
@@ -314,7 +314,9 @@ export class BuiltInFtsProvider implements SearchProvider, OnModuleInit {
* unchanged. Postgres already uses the tolerant websearch_to_tsquery and needs no equivalent.
*/
private static toFts5Query(raw: string): string {
- const tokens = raw.trim().split(/\s+/).filter(Boolean);
+ // A repeated token is dropped: `a a` is `a` under implicit AND, but FTS5 would pay for each copy
+ // on every matching row.
+ const tokens = [...new Set(raw.trim().split(/\s+/).filter(Boolean))];
if (tokens.length === 0) return '""';
return tokens.map(tok => `"${tok.replace(/"/g, '""')}"`).join(' ');
}
diff --git a/src/modules/search/search.constants.ts b/src/modules/search/search.constants.ts
index be8419d4a..c6439091e 100644
--- a/src/modules/search/search.constants.ts
+++ b/src/modules/search/search.constants.ts
@@ -18,3 +18,13 @@ export const SEARCH_OFFSET_MAX = 100_000;
/** Default page size when a caller omits `limit` (matches the built-in provider's historical default). */
export const SEARCH_DEFAULT_LIMIT = 50;
+
+/** Longest `q`, in characters. A generous bound: it exists so the request cannot be a megabyte of terms. */
+export const SEARCH_QUERY_MAX_LENGTH = 1000;
+
+/**
+ * Most distinct whitespace-separated words in `q`. Each one costs a pass over every message that
+ * matches, and the built-in provider runs on the event loop, so a query of hundreds of words froze every
+ * session for seconds. A word repeated is still one word: the provider collapses repeats.
+ */
+export const SEARCH_TERMS_MAX = 32;
diff --git a/src/modules/search/search.controller.ts b/src/modules/search/search.controller.ts
index 8e6256fbe..b062519b9 100644
--- a/src/modules/search/search.controller.ts
+++ b/src/modules/search/search.controller.ts
@@ -22,7 +22,10 @@ export class SearchController {
@RequireRole(ApiKeyRole.OPERATOR)
@ApiOperation({ summary: 'Search messages across sessions (active search provider)' })
@ApiResponse({ status: 200, description: 'Search results from the active provider', type: SearchResultsResponseDto })
- @ApiResponse({ status: 400, description: 'Empty or whitespace-only "q"' })
+ @ApiResponse({
+ status: 400,
+ description: 'Empty or whitespace-only "q", or one over 1000 characters or 32 distinct words',
+ })
@ApiResponse({ status: 403, description: 'Chat scope denied or chat-restricted search uses a plugin provider' })
@ApiResponse({ status: 501, description: 'No search provider configured' })
@ApiResponse({
@@ -37,7 +40,11 @@ export class SearchController {
'The active plugin search provider did not answer: its worker is not running, timed out, or reported ' +
'a failure. The built-in provider never returns this. Retryable.',
})
- @ApiQuery({ name: 'q', required: true, description: 'Search term (required, non-empty)' })
+ @ApiQuery({
+ name: 'q',
+ required: true,
+ description: 'Search term (required, non-empty, at most 1000 characters and 32 distinct words)',
+ })
@ApiQuery({ name: 'sessionId', required: false, description: 'Restrict to a single session' })
@ApiQuery({ name: 'chatId', required: false, description: 'Restrict to a single chat id' })
@ApiQuery({ name: 'direction', required: false, description: 'incoming | outgoing' })
diff --git a/src/modules/search/search.service.spec.ts b/src/modules/search/search.service.spec.ts
index 91285385d..c004d2095 100644
--- a/src/modules/search/search.service.spec.ts
+++ b/src/modules/search/search.service.spec.ts
@@ -1,7 +1,7 @@
-import { NotImplementedException } from '@nestjs/common';
+import { BadRequestException, NotImplementedException } from '@nestjs/common';
import { SearchService } from './search.service';
import { SearchProviderRegistry } from './search-provider.registry';
-import { SEARCH_DEFAULT_LIMIT, SEARCH_LIMIT_MAX, SEARCH_OFFSET_MAX } from './search.constants';
+import { SEARCH_DEFAULT_LIMIT, SEARCH_LIMIT_MAX, SEARCH_OFFSET_MAX, SEARCH_TERMS_MAX } from './search.constants';
import type { SearchProvider, SearchResults } from './search.types';
function mkProvider(id: string, search: SearchProvider['search']): SearchProvider {
@@ -51,6 +51,22 @@ describe('SearchService', () => {
expect(search).toHaveBeenCalledWith(expect.objectContaining({ chatIds: ['[email protected]', '111@lid'] }));
});
+ it('refuses a query of more than SEARCH_TERMS_MAX distinct words before any provider runs it', async () => {
+ const reg = new SearchProviderRegistry();
+ const search = jest.fn().mockResolvedValue(emptyResults);
+ reg.register(mkProvider('builtin-fts', search));
+ const svc = new SearchService(reg);
+ const words = (n: number): string => Array.from({ length: n }, (_, i) => `w${i}`).join(' ');
+
+ await expect(svc.search({ q: words(SEARCH_TERMS_MAX + 1) })).rejects.toBeInstanceOf(BadRequestException);
+ expect(search).not.toHaveBeenCalled();
+
+ await svc.search({ q: words(SEARCH_TERMS_MAX) });
+ // A word said many times is one word.
+ await svc.search({ q: Array.from({ length: 300 }, () => 'hello').join(' ') });
+ expect(search).toHaveBeenCalledTimes(2);
+ });
+
it('clamps an excessive limit to SEARCH_LIMIT_MAX before it reaches any provider', async () => {
// Without the host-side clamp a plugin provider would receive the raw caller value (9999) and
// could pressure host heap by returning an oversized page. The built-in already rejected this in
diff --git a/src/modules/search/search.service.ts b/src/modules/search/search.service.ts
index e6631bf19..a7be40bff 100644
--- a/src/modules/search/search.service.ts
+++ b/src/modules/search/search.service.ts
@@ -1,6 +1,6 @@
-import { Injectable, NotImplementedException } from '@nestjs/common';
+import { BadRequestException, Injectable, NotImplementedException } from '@nestjs/common';
import { SearchProviderRegistry } from './search-provider.registry';
-import { SEARCH_DEFAULT_LIMIT, SEARCH_LIMIT_MAX, SEARCH_OFFSET_MAX } from './search.constants';
+import { SEARCH_DEFAULT_LIMIT, SEARCH_LIMIT_MAX, SEARCH_OFFSET_MAX, SEARCH_TERMS_MAX } from './search.constants';
import type { SearchQuery, SearchResults } from './search.types';
@Injectable()
@@ -10,6 +10,11 @@ export class SearchService {
async search(query: SearchQuery, callerSessionIds?: string[], callerChatIds?: string[]): Promise<SearchResults> {
const provider = this.registry.active();
if (!provider) throw new NotImplementedException('Search is not configured (no active search provider).');
+ // Bounded here, ahead of every provider: the cost of a query grows with its words, whatever backend
+ // runs it. A repeated word is one word.
+ if (new Set(query.q.trim().split(/\s+/)).size > SEARCH_TERMS_MAX) {
+ throw new BadRequestException(`q may hold at most ${SEARCH_TERMS_MAX} distinct words.`);
+ }
// Auth scoping is authoritative — a caller cannot override it via the query. This covers both
// fences: sessions (allowedSessions) and chats (allowedChats, expanded lid-aware by the caller),
// each undefined for an unrestricted key. Bound pagination host-side so EVERY provider