Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

fix(search): resolve slug handles per OR alternative

garrytan533a86591d55
diff --git a/docs/architecture/key-files/core-search-cjk.md b/docs/architecture/key-files/core-search-cjk.md
index d60cf6cf4e..1a8d574e28 100644
--- a/docs/architecture/key-files/core-search-cjk.md
+++ b/docs/architecture/key-files/core-search-cjk.md
@@ -6,4 +6,4 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL
- `src/core/search/cjk-keyword-sql.ts` + `src/core/engine-sql/cjk-search.ts` + `src/core/search/cjk-keyword-deadline.ts` — the CJK keyword fallback shared by both engines. `buildCJKKeywordSql(query, ctx, variant)` keeps today's strict-AND SQL byte-identical by default; `match: 'any'` (the OR fallback, #6043) qualifies a chunk on any term and orders by matched-term count, then the TF score (the count enters the score as 1000 x matched outside the source factor, so a boosted partial match never outranks an all-term chunk); `candidateIds` scores only a candidate set by joining `unnest($n::bigint[])` to `content_chunks` (an `= ANY` filter was a sequential scan). `buildCJKCandidateSql` is the unordered capped LIKE scan for ids (all-term, or any-term excluding ids already taken). `searchKeywordCJK(scoped, query, ctx, engine)` runs strict AND and, with `opts.orFallback` and 2+ terms, the OR pass only when strict AND returns fewer than `limit` rows; with `opts.cjkKeyword` (hybrid only, deadline from `search.cjk_keyword_deadline_ms`, integer ms 500..30000, default 3000) it bounds the arm: Postgres runs full scoring in its own scoped read transaction within 2/3 of the deadline (counted from before the pool wait; every statement of an attempt gets `statement_timeout` = what is left of that attempt, minus a 40 ms rollback margin), and on SQLSTATE 57014 a capped retry (`CJK_CANDIDATE_CAP` = 2000; all-term ids first, partial ids only when the all-term rows fall short) in a fresh scoped transaction with the remaining third (a fresh transaction, not a savepoint, because the driver rethrows any failed statement at the end of its transaction scope); PGLite runs the capped stage whenever `pg_class.reltuples` exceeds `PGLITE_CJK_CAPPED_CHUNKS` = 50,000. `onMeta` gets `{ incomplete, reason (timeout | candidate_budget), capped, arm_ms }` (test seams: `candidateCap`, `pgliteCappedChunks`, `fullBudgetMs`, and `onStatementBudget`, which reports each Postgres statement's timeout and the arm ... (line truncated to 2000 chars)
- `src/core/search/keyword-or-rarity.ts` — rare words first in both engines' keyword OR retry. For at most `RARITY_MAX_KEYWORD_TERMS` (12) words (the title arm keeps `RARITY_MAX_TERMS` = 4: it runs one statement per word), `rareTermPagesSql` reads the pages of each word's first `RARE_TERM_CHUNK_CAP` + 1 visible chunks (brain-wide, weights only; stopwords and, on Postgres, words `commonOrTokens` in keyword-or-narrow.ts estimates past the cap are skipped); a fitting word weighs 1 / pages. `rankOrFallbackByRarity` re-runs the retry with the rarest words' OR and sorts by summed weight.
- `src/core/search/keyword-identifier-probe.ts` — #7491: `keywordArmRows` builds both engines' `searchKeyword` rows from the strict rows. With `orFallback`, offset 0 and no quote or negation, a query naming an identifier (`identifierTokens`: a hash number `#412`, or a token `identifier-query.ts` recognizes: a 6+ digit run, hex id, code, email, URL or phone) and other words runs the arm's own statement once more with the AND of the identifier tokens as the match query and the any-word query as the rank query: same scope, visibility, chunk predicate, filters and candidate cap as strict. Its rows are untagged (they vote like strict rows): after the strict rows when strict matched, ahead of the relaxed rows (`keyword_relaxed`) when it did not; one row per page, cut to the limit. A failed probe adds nothing. `holdsIdentifiers` lets the title arm's OR retry (`engine-sql/titles.ts`) lead with, untagged, the titles holding every identifier. Pinned by `test/search/identifier-probe.serial.test.ts` (+ Postgres E2E) and `test/search/keyword-identifier-probe.test.ts`.
-- `src/core/search/keyword-handles.ts` — GBRA-108: `slugHandles` finds the query's slug handles (a hyphenated slug-leaf token with a letter, bare or quoted on its own, at most `MAX_SLUG_HANDLES` = 16; a query with a negation or a longer quoted phrase resolves none, and the strict match (`slugHandles(query, true)`) resolves them only when every term is a handle or `OR`). A failed handle or identifier-probe read adds no rows and calls `SearchOpts.onKeywordIncomplete` (`keyword_candidates_incomplete`, reason `timeout` or `provider_error`). They resolve by exact slug basename (`handleMatchSql`, served by `idx_pages_slug_basename` like `whois`) as an arm of the keyword statement (`search/keyword-statement.ts`) under its filters, scope and visibility, so `--types`, granted sources, private pages and quarantine apply. `match: "keyword"` lists and counts the named pages first; in `searchKeyword` (orFallback, offset 0) `keywordArmRows` puts them ahead of every other row, untagged. A query without a handle builds the statements it built before. Pinned by `test/search/slug-handles.serial.test.ts` (+ Postgres E2E) and `test/search/keyword-handles.test.ts`.
+- `src/core/search/keyword-handles.ts` — GBRA-108: `slugHandles` finds the query's slug handles (a hyphenated slug-leaf token with a letter, bare or quoted on its own, at most `MAX_SLUG_HANDLES` = 16; each alternative of a bare `OR` is judged on its own: an alternative with a negation or a longer quoted phrase resolves none, and the strict match (`slugHandles(query, true)`) resolves an alternative's handles only when every term of it is a handle). A failed handle or identifier-probe read adds no rows and calls `SearchOpts.onKeywordIncomplete` (`keyword_candidates_incomplete`, reason `timeout` or `provider_error`). They resolve by exact slug basename (`handleMatchSql`, served by `idx_pages_slug_basename` like `whois`) as an arm of the keyword statement (`search/keyword-statement.ts`) under its filters, scope and visibility, so `--types`, granted sources, private pages and quarantine apply. `match: "keyword"` lists and counts the named pages first; in `searchKeyword` (orFallback, offset 0) `keywordArmRows` puts them ahead of every other row, untagged. A query without a handle builds the statements it built before. Pinned by `test/search/slug-handles.serial.test.ts` (+ Postgres E2E) and `test/search/keyword-handles.test.ts`.
diff --git a/src/core/search/keyword-handles.ts b/src/core/search/keyword-handles.ts
index 5c035a72bf..a9759566f2 100644
--- a/src/core/search/keyword-handles.ts
+++ b/src/core/search/keyword-handles.ts
@@ -8,13 +8,14 @@
* pages they name, and with `--types '["person"]'` it matched nothing.
*
* A handle is a hyphenated slug-leaf token with a letter in it (`dan-example`,
- * `acme-example-2`), bare or quoted on its own; each quoted handle of an OR is
- * resolved on its own. A query with a negation (`-dan-example`, `-draft`) or
- * a longer quoted phrase resolves no handle, so a handle never returns a page
- * the query excludes or a phrase it does not hold, and a strict keyword match
- * (`match: "keyword"` and its count) resolves handles only when every term is
- * one (a standalone handle or an OR list of them), so it keeps the AND of the
- * query's other words. Resolution is an exact match of the
+ * `acme-example-2`), bare or quoted on its own; each alternative of a bare OR
+ * is judged on its own (#8618: the alias rewrite `"Acme-Example" OR "acme
+ * corp"`). An alternative with a negation (`-dan-example`, `-draft`) or a
+ * longer quoted phrase resolves no handle, so a handle never returns a page
+ * its alternative excludes or a phrase it does not hold, and a strict keyword
+ * match (`match: "keyword"` and its count) resolves an alternative's handles
+ * only when every term of that alternative is one, so it keeps the AND of the
+ * alternative's other words. Resolution is an exact match of the
* slug's basename (the part after the last `/`, the whole slug for a root
* page), served by idx_pages_slug_basename like `whois`. The match is a second
* arm of the keyword statement (search/keyword-statement.ts) under the same
@@ -32,19 +33,30 @@ export const MAX_SLUG_HANDLES = 16;
const isHandle = (token: string): boolean => token.length <= 128 && HANDLE_RE.test(token) && /[a-z]/.test(token);
/**
- * The query's slug handles, lowercased, or null when it has none. With `standalone`, only when every term is a handle
- * or OR (the strict keyword match). A negation or a quoted phrase that is not a handle resolves none.
+ * The query's slug handles, lowercased, or null when it has none. Each alternative of a bare OR is judged on its
+ * own: a negation or a quoted phrase that is not a handle (an unclosed quote runs to the end, as in
+ * websearch_to_tsquery) resolves none of its alternative's handles, and with
+ * `standalone` (the strict keyword match) neither does any other word.
*/
export function slugHandles(query: string, standalone = false): string[] | null {
const handles = new Set<string>();
- for (const m of query.normalize('NFKC').matchAll(/"([^"]*)"|(\S+)/g)) {
+ let alternative: string[] = [];
+ let constrained = false;
+ const endAlternative = () => {
+ if (!constrained) for (const h of alternative) if (handles.size < MAX_SLUG_HANDLES) handles.add(h);
+ alternative = [];
+ constrained = false;
+ };
+ for (const m of query.normalize('NFKC').matchAll(/"([^"]*)(?:"|$)|([^\s"]+)/g)) {
const bare = m[2];
- if (bare?.startsWith('-')) return null;
const token = (bare === undefined ? m[1]!.trim() : bare.replace(EDGE_PUNCTUATION, '')).toLowerCase();
- if (isHandle(token)) {
- if (handles.size < MAX_SLUG_HANDLES) handles.add(token);
- } else if (bare === undefined || (standalone && token !== 'or')) return null;
+ if (bare === undefined && token === '') continue;
+ if (bare !== undefined && token === 'or') endAlternative();
+ else if (bare?.startsWith('-')) constrained = true;
+ else if (isHandle(token)) alternative.push(token);
+ else if (bare === undefined || standalone) constrained = true;
}
+ endAlternative();
return handles.size ? [...handles] : null;
}
diff --git a/test/search/keyword-handles-or-alternatives-8618.test.ts b/test/search/keyword-handles-or-alternatives-8618.test.ts
new file mode 100644
index 0000000000..64038fc0cd
--- /dev/null
+++ b/test/search/keyword-handles-or-alternatives-8618.test.ts
@@ -0,0 +1,38 @@
+/**
+ * #8618: `match: "keyword"` for a page's handle rewrites the query to every
+ * name the entity has (`"Acme-Example" OR "acme corp"`). The handle names
+ * the page through its slug only, so the rewrite must keep the slug-handle
+ * arm beside the alias phrase: the named page is listed and counted. PGLite,
+ * synthetic pages.
+ */
+import { afterAll, beforeAll, describe, expect, test } from 'bun:test';
+import { PGLiteEngine } from '../../src/core/pglite-engine.ts';
+import { callSearch, seedPage } from '../helpers/keyword-paging-fixture.ts';
+
+let engine: PGLiteEngine;
+
+beforeAll(async () => {
+ engine = new PGLiteEngine();
+ await engine.connect({});
+ await engine.initSchema();
+ await seedPage(engine, 'companies/acme-example', 'A company that builds quokka widgets.', { type: 'company', title: 'Acme-Example', frontmatter: { aliases: ['acme corp'] } });
+ await seedPage(engine, 'notes/renewal-call', 'The acme corp renewal call went well.');
+ await seedPage(engine, 'notes/unrelated', 'Nothing about that company here.');
+}, 60_000);
+
+afterAll(async () => { await engine?.disconnect(); });
+
+describe('#8618 slug handles across OR alternatives', () => {
+ test('a keyword search for a handle lists and counts the page it names beside its alias matches', async () => {
+ const { rows, retrieval } = await callSearch(engine, { query: 'Acme-Example', match: 'keyword' });
+ expect(retrieval.match_names).toEqual(['Acme-Example', 'acme corp']);
+ expect(rows.map(r => r.slug).sort()).toEqual(['companies/acme-example', 'notes/renewal-call']);
+ expect(retrieval.total).toBe(2);
+ });
+
+ test('a typed handle OR name phrase keeps the named page', async () => {
+ const { rows, retrieval } = await callSearch(engine, { query: '"acme-example" OR "renewal call"', match: 'keyword' });
+ expect(rows.map(r => r.slug).sort()).toEqual(['companies/acme-example', 'notes/renewal-call']);
+ expect(retrieval.total).toBe(2);
+ });
+});
diff --git a/test/search/keyword-handles.test.ts b/test/search/keyword-handles.test.ts
index 3c05d00011..35feb92596 100644
--- a/test/search/keyword-handles.test.ts
+++ b/test/search/keyword-handles.test.ts
@@ -33,6 +33,26 @@ describe('slugHandles', () => {
expect(slugHandles('dan-example quokka')).toEqual(['dan-example']);
});
+ test('#8618: a quoted phrase or a negation constrains only its own OR alternative', () => {
+ expect(slugHandles('"Acme-Example" OR "acme corp"', true)).toEqual(['acme-example']);
+ expect(slugHandles('"dan-example" OR "Dan Example"')).toEqual(['dan-example']);
+ expect(slugHandles('"dan-example" OR "Dan Example"', true)).toEqual(['dan-example']);
+ expect(slugHandles('dan-example OR northwind -draft')).toEqual(['dan-example']);
+ expect(slugHandles('"dan-example" "renewal due" OR "erin-example"')).toEqual(['erin-example']);
+ expect(slugHandles('dan-example quokka OR "erin-example"', true)).toEqual(['erin-example']);
+ expect(slugHandles('"acme corp" OR "Dan Example"')).toBeNull();
+ expect(slugHandles('"dan-example OR erin-example', true)).toBeNull();
+ expect(slugHandles('"dan-example OR erin-example')).toBeNull();
+ expect(slugHandles('dan-example OR "erin-example')).toEqual(['dan-example', 'erin-example']);
+ });
+
+ test('#8618: a quote mark is a word boundary and an empty phrase constrains nothing, as in websearch_to_tsquery', () => {
+ expect(slugHandles('dan-example "')).toEqual(['dan-example']);
+ expect(slugHandles('dan-example "" OR erin-example', true)).toEqual(['dan-example', 'erin-example']);
+ expect(slugHandles('acme"dan-example OR erin-example')).toBeNull();
+ expect(slugHandles('dan-example" OR erin-example')).toBeNull();
+ });
+
test('at most MAX_SLUG_HANDLES handles', () => {
const q = Array.from({ length: 40 }, (_, i) => 'p-' + i + 'x').join(' ');
expect(slugHandles(q)).toHaveLength(MAX_SLUG_HANDLES);