Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
search: question-framing words do not count toward the saved-fact threshold; every correction of a returned fact comes back
garrytan7ff7cbb5bd0e
diff --git a/docs/architecture/RETRIEVAL.md b/docs/architecture/RETRIEVAL.mdindex 8c96dd7942..9720e25e39 100644--- a/docs/architecture/RETRIEVAL.md+++ b/docs/architecture/RETRIEVAL.md@@ -345,14 +345,17 @@ Two cross-cutting seams sit around the pipeline rather than inside it:full-text language, the facts keyword index's expression, so "renewal" finds"renews" and "count" does not match inside "countersigned"; words of a CJKscript match as substrings. A fact needs three quarters of the query's words- (both of a two-word query); the most matched words come first, then the+ (both of a two-word query), not counting words that frame a question about a+ figure (count, number, total, amount); the most matched words come first, then thenewest. This is the only place `search` returns saved facts: thefacts arm, the facts block and the temporal reserve run in `query` alone.With `search.saved_facts_supersede` (default on; `false` turns it off), a newer fact with acorrection cue ("correction", "ignore the ...", "not what I posted") thatshares at least two matched words with a returned fact is returned too, evenbelow the threshold, every fact it corrects moves below the facts nothing- corrects, and those carry `corrected_by` (the correcting fact's id). An+ corrects, and those carry `corrected_by` (the correcting fact's id, the+ newest when several correct it). Every correction comes back, so an+ acknowledgement saved after a correction does not crowd it out. Anacknowledgement corrects nothing: a cue fact that adds no figure (a numberother than a date or time) to the older fact, when it restates one of itsfigures or the older fact is itself a correction. The cue is a fixed word list, sodiff --git a/docs/architecture/key-files/entity-recall.md b/docs/architecture/key-files/entity-recall.mdindex b8e42bec4e..9289086ab4 100644--- a/docs/architecture/key-files/entity-recall.md+++ b/docs/architecture/key-files/entity-recall.md@@ -24,7 +24,7 @@ card and `get_backlinks`. Guide: [entity recall](../../guides/entity-recall.md).- `src/core/verbs/entity-card-identity.ts` — the `entity` card's `aka_sources`, `identity_siblings`, `identity_excerpt` (verbatim lines of the page and its siblings from `sanitizeRemoteBody`, tiered: declarations, label lines with a name-like value, other label lines, lines with another capitalized run or code; 600 characters per page, 1,200 per card, `identity_excerpt_omitted`), `alias_guidance` (surface-aware) and the `identity_siblings_capped` notice. A segmented page is excerpted from its trusted, withdrawal-overlaid lead (`truthLead`, `EXCERPT_LEAD_BYTES` = 64 KiB, cut back to a whole line), never its body getters; the card equals the inline page's when the qualifying lines sit in that lead (`test/large-pages/segmented-read-refusals.test.ts`). `readSiblingIdentity` is its siblings-and-aliases half (one alias read for the page and its siblings; optional preloaded mention policy and pack), used alone by `entity names[]`. Tests: `test/mentions-identity-siblings.test.ts`.- `src/core/search/alias-fanout.ts` — search and query's alias fan-out: `resolveQueryEntity` (query n-grams up to 4 tokens through `resolveAliases` and linkable exact titles, longest first, capitalized or code-shaped unless a multi-word n-gram resolves to one page, case-sensitive aliases as written, a tie only when the pages are identity siblings), the names of the entity and its siblings by origin then length, at most `search.alias_fanout_max` (4) keyword queries of 5 rows (`searchAliasRequired`: the name as a required websearch phrase, ranked by the query's other terms through `SearchOpts.rankQuery` on both engines), at most 8 spliced rows with `matched_alias`, and the `fanout` meta (`alias_fanout` notice when capped). Offset 0 only. `resolving` takes the handler's `resolveQueryEntity` started before the search, so its reads overlap it. Tests: `test/search-alias-fanout.test.ts`, `test/search-other-names.test.ts`, `test/e2e/mentions-parity.test.ts`. `wholeQueryEntityNames`: when the whole query is one entity's name or alias, `match: "keyword"` lists pages matching any of its names (one websearch OR of phrases; `match_names` in the meta and the count line; rows found by another name carry `matched_alias`). `resolveQueryEntities` returns every entity a text names; its alias, linkable-title and case-sensitive alias reads run concurrently (`test/search-entity-resolve-concurrency.test.ts`).- `src/core/search/entity-facts.ts` — `entitySavedFacts`: when a search names an entity, its newest active saved facts (`factEntitySlugs`, 5) lead the model-visible saved-facts block, then up to 3 facts each of up to 2 entities those facts name (`linked_from`), before the word-overlap matches. Same active, audit-row and visibility rules as recall. Tests: `test/memory-sibling-facts.test.ts`.-- `src/core/search/saved-facts-match.ts` — `matchingSavedFacts`, the word-matched saved facts `search` and `query` report in `retrieval.saved_facts` after the named entity's facts (`ops/search.ts` `buildRetrievalResponseMeta`). Words match whole after stemming with the facts keyword index's expression (`factsFtsDocument`), CJK words as substrings; a fact needs three quarters of the query's words; most matched words first, then newest (time, then id), at most five after the #7043 collapse. One statement behind the facts-only EXISTS probe (`test/search/saved-facts-probe.test.ts`). `search.saved_facts_supersede` (default on; `false` turns it off): a newer fact with a correction cue that shares two matched words with a returned fact is returned too, even below the threshold, and the facts it corrects move last with `corrected_by`; a cue fact that adds no figure to the older fact (`figures`: numbers, dates and times left out) corrects nothing when it restates one of its figures or the older fact is itself a correction. Also owns `FACT_MATCH_STOPWORDS`. Tests: `test/search-saved-facts-match.test.ts`, `test/search-saved-facts.test.ts`.+- `src/core/search/saved-facts-match.ts` — `matchingSavedFacts`, the word-matched saved facts `search` and `query` report in `retrieval.saved_facts` after the named entity's facts (`ops/search.ts` `buildRetrievalResponseMeta`). Words match whole after stemming with the facts keyword index's expression (`factsFtsDocument`), CJK words as substrings; a fact needs three quarters of the query's words, question-framing words (`QUESTION_FRAME_WORDS`: count, number, total, amount) not counted; most matched words first, then newest (time, then id), at most five after the #7043 collapse. One statement behind the facts-only EXISTS probe (`test/search/saved-facts-probe.test.ts`). `search.saved_facts_supersede` (default on; `false` turns it off): a newer fact with a correction cue that shares two matched words with a returned fact is returned too, even below the threshold, and the facts it corrects move last with `corrected_by` (the newest correction); every correction of a returned fact comes back; a cue fact that adds no figure to the older fact (`figures`: numbers, dates and times left out) corrects nothing when it restates one of its figures or the older fact is itself a correction. Also owns `FACT_MATCH_STOPWORDS`. Tests: `test/search-saved-facts-match.test.ts`, `test/search-saved-facts.test.ts`.- `src/core/schema-migrations/v206-entity-mention-index.ts` — additive: `page_aliases.origin` (`frontmatter` default, `declared`, `subject`), `case_sensitive`, `alias_text`, unique key `(source_id, alias_norm, slug, origin)`; tables `page_mention_state`, `mention_gazetteer_entries`, `mention_index_status` (RLS on when the role bypasses RLS). `GUARDED_TABLE_COLUMNS` lists the new `page_aliases` columns.Alias readers apply precedence frontmatter > declared > subject per source, and a derived alias never answers for another live page's exact title that the caller can read (the title competitor gets the caller's `pageReadFilter`, so a private title a private-excluding caller can't see never hides a readable alias) (`readAliases` in `src/core/search/read-enrichment.ts`, shared by both engines' `resolveAliases`), so `resolveEntitySlug`, the retrieval reflex, the alias hop, intent weights and `search-diagnose` rank an exact title first. `buildGazetteer` (`src/core/by-mention.ts`) applies the same precedence, drops an alias claimed by two pages at its best origin (unless the claimants are identity siblings, which all keep it), yields to a same-source title, drops a single-word alias equal to the first word of another multi-word entity name, applies the generic-word list to every single-word name, and carries `caseTokens` for case-sensitive entries; `opts.dropped` records every dropped name with its reason.diff --git a/src/core/behavior-change-notice.ts b/src/core/behavior-change-notice.tsindex 616db2ae81..dc25f68cc1 100644--- a/src/core/behavior-change-notice.ts+++ b/src/core/behavior-change-notice.ts@@ -1321,7 +1321,7 @@ export const BEHAVIOR_CHANGES: ReadonlyArray<{ since: string; text: ChangeText;// #7679 / #7685 orphaned worktree refresh. `since` is stamped at the merge slot.{ since: '0.60.472.0', id: 'refresh_orphan_7679', text: 'A managed `gbrain sources refresh` whose process died (for example a container replaced mid-refresh) no longer wedges the checkout: the next plain `gbrain sources refresh` on the owner host finishes the orphaned refresh through the `--resume` path before its own refresh and reports it as `recovered`, and the owner completes an orphaned `syncing` refresh once every member has synced to its target. A refresh whose holder is still running, or is recorded on another host, is never taken over. `gbrain sources writer status` and `gbrain sync status` now name the holder and its `liveness` (`live`, `orphaned`, `unknown`) with a fix naming `--resume`. With `--json`, a refusal that carries a structured fix (`refresh_in_progress`, `refresh_diverged`, a git read failure) now prints `fix` as an object (argv, actor, next) instead of a string; the prose stays in `suggestion`, so read that when a script needs text. A plain refresh of a checkout whose interrupted refresh needs a reset decision now refuses `refresh_recovery_required` (it refused `refresh_in_progress`).' },// GBRA-205 saved-facts wave (harness case cr-05). `since` is stamped at the merge slot.- { since: 'TBD', id: 'saved_facts_stems', text: 'Search `saved_facts` matches a saved fact\'s words the way the facts keyword index does: whole words after stemming with the brain\'s full-text language, so a query for "seats renewal" finds a fact that says "renews at 52 seats", and "count" no longer matches inside "countersigned". A fact still needs three quarters of the query\'s words; words of a CJK script still match as substrings. Among facts matching as many words, the newest comes first, and at most five facts ride a reply. On by default (`gbrain config set search.saved_facts_supersede false` turns it off): search also returns a newer fact with a correction cue that shares two matched words with a returned fact, ahead of the fact it corrects, which is labelled `corrected_by`; an acknowledgement that adds no figure corrects nothing.' },+ { since: 'TBD', id: 'saved_facts_stems', text: 'Search `saved_facts` matches a saved fact\'s words the way the facts keyword index does: whole words after stemming with the brain\'s full-text language, so a query for "seats renewal" finds a fact that says "renews at 52 seats", and "count" no longer matches inside "countersigned". A fact still needs three quarters of the query\'s words, not counting words that frame a question about a figure (count, number, total, amount); words of a CJK script still match as substrings. Among facts matching as many words, the newest comes first, and at most five facts ride a reply. On by default (`gbrain config set search.saved_facts_supersede false` turns it off): search also returns a newer fact with a correction cue that shares two matched words with a returned fact, ahead of the fact it corrects, which is labelled `corrected_by`; every such correction comes back, not only the newest, and an acknowledgement that adds no figure corrects nothing.' },];/** The newest disclosed change's release: the notice id moves only when a release adds rows. */diff --git a/src/core/search/saved-facts-match.ts b/src/core/search/saved-facts-match.tsindex 62c27aa39e..443386a836 100644--- a/src/core/search/saved-facts-match.ts+++ b/src/core/search/saved-facts-match.ts@@ -19,6 +19,9 @@ import { SAVED_FACT_COLLAPSE_COLUMNS, type CollapseColumns } from './entity-fact/** Words too common to count toward a saved-fact match; also the coverage relevance stopwords in ops/search.ts. */export const FACT_MATCH_STOPWORDS = new Set(['the', 'and', 'for', 'who', 'what', 'when', 'where', 'which', 'with', 'from', 'that', 'this', 'are', 'was', 'were', 'our', 'your', 'their', 'now', 'current', 'currently', 'should', 'does', 'did', 'has', 'have', 'how', 'any', 'all', 'about', 'into', 'its', 'next']);+/** Words that frame a question about a figure ("seat count", "total revenue") and that the fact stating it rarely repeats; they count toward neither side of the threshold. */+const QUESTION_FRAME_WORDS = new Set(['count', 'number', 'total', 'amount']);+export interface SavedFactMatch { id: number; fact: string; entity_slug: string | null; kind: string; valid_from: string; source: string;/** #7043: every entity of the copies this row stands for, and how many rows it stands for. */entity_slugs?: string[]; collapsed_count?: number;@@ -68,7 +71,7 @@ export async function matchingSavedFacts(ctx: OperationContext, scope: { sourceI// A query naming one of two declared names also matches facts saved under the other.if (otherNames.length) queryText += ` ${otherNames.join(' ')}`;const terms = [...new Set(queryText.toLowerCase().match(/[\p{L}\p{N}][\p{L}\p{N}'-]+/gu) ?? [])]- .filter(t => t.length >= 3 && !FACT_MATCH_STOPWORDS.has(t)).slice(0, 12);+ .filter(t => t.length >= 3 && !FACT_MATCH_STOPWORDS.has(t) && !QUESTION_FRAME_WORDS.has(t)).slice(0, 12);if (terms.length === 0 || !ctx.emitResponseMeta) return [];const sources = scope.sourceIds?.length ? scope.sourceIds : [scope.sourceId ?? ctx.sourceId ?? 'default'];const [excludePrivate, supersede] = await Promise.all([resolveExcludePrivatePages(ctx.engine, ctx.remote), savedFactsSupersedeEnabled(ctx.engine)]);@@ -122,17 +125,20 @@ export async function matchingSavedFacts(ctx: OperationContext, scope: { sourceIat: new Date(at as string).getTime(), words: new Set(matched.map(Number)), need: Number(need), cue, figures: cue ? figures(fact.fact) : new Set<string>() }));const admitted = found.filter(r => r.words.size >= r.need);if (supersede) {+ const correcting = new Set<number>();for (const older of admitted) {// Posts saved in one flush share a time; the later id is the later post.// An acknowledgement is not a correction: a cue fact that adds no figure to the older fact// corrects it only when it restates no figure and the older fact is not a correction itself.const olderFigures = figures(older.fact.fact);- const correction = found.filter(n => n.cue && (n.at > older.at || (n.at === older.at && n.fact.id > older.fact.id))+ const corrections = found.filter(n => n.cue && (n.at > older.at || (n.at === older.at && n.fact.id > older.fact.id))&& [...n.words].filter(w => older.words.has(w)).length >= Math.min(2, n.need)- && !([...n.figures].every(x => olderFigures.has(x)) && (n.figures.size > 0 || older.cue))).sort((a, b) => b.at - a.at || b.fact.id - a.fact.id)[0];- if (correction) older.fact.corrected_by = correction.fact.id;+ && !([...n.figures].every(x => olderFigures.has(x)) && (n.figures.size > 0 || older.cue))).sort((a, b) => b.at - a.at || b.fact.id - a.fact.id);+ if (corrections.length) older.fact.corrected_by = corrections[0].fact.id;+ for (const c of corrections) correcting.add(c.fact.id);}- const correcting = new Set(admitted.flatMap(r => r.fact.corrected_by ? [r.fact.corrected_by] : []));+ // Every correction of a returned fact comes back, not only the newest: an acknowledgement+ // saved after the correction must not crowd out the post that made it.admitted.push(...found.filter(r => correcting.has(r.fact.id) && r.words.size < r.need));}const ordered = supersede ? [...admitted.filter(r => !r.fact.corrected_by), ...admitted.filter(r => r.fact.corrected_by)] : admitted;diff --git a/test/search-saved-facts-match.test.ts b/test/search-saved-facts-match.test.tsindex 0b91117cf8..0659fe658f 100644--- a/test/search-saved-facts-match.test.ts+++ b/test/search-saved-facts-match.test.ts@@ -66,6 +66,13 @@ beforeAll(async () => {ids.olderLease = await remember('The warehouse lease runs five years.', '2026-08-01T09:00:00Z');ids.newerLease = await remember('The warehouse lease renews in March.', '2026-08-15T09:00:00Z');ids.otherAccount = await remember('Kappa Example renews at 30 seats.', '2026-09-01T10:00:00Z');+ // A correction posted days before the question that asks for the figure, in the words a question uses.+ ids.zentra = await remember('Dana Example (dana-example, Strategic account manager) wrote in #pod-a on 2026-08-14 09:00: For your records, no action needed: the Q3 digest has Zentra wrong. Zentra Medical Supply is at 50 seats, not 55.', '2026-08-14T09:00:00Z');+ ids.zentraQuestion = await remember('Erin Example (erin-example, revenue lead) wrote in #pod-a on 2026-08-16 09:00: What is the current Zentra Medical Supply seat count? I am filling in the pod sheet.', '2026-08-16T09:00:00Z');+ // A correction and an acknowledgement of it saved in one flush: the acknowledgement is the later post.+ ids.quellin = await remember('Quellin Freight renews at 30 seats.', '2026-05-01T09:00:00Z');+ ids.quellinFix = await remember('Correction: Quellin Freight is at 24 seats, ignore the 30.', '2026-05-02T09:00:00Z');+ ids.quellinAck = await remember('Thanks for the correction, noted: Quellin Freight at 24 seats.', '2026-05-02T09:00:00Z');ids.delta = await remember('Delta Example has 50 seats, not 55.', '2026-08-01T09:00:00Z');ids.deltaAck = await remember('Correction noted: Delta Example has 50 seats, not 55.', '2026-08-01T09:05:00Z');ids.omega = await remember('Correction for the record: Omega Example stays with Carol Example.', '2026-07-01T09:00:00Z');@@ -93,7 +100,7 @@ describe('search saved facts: stem-aware whole-word matching', () => {});test('a query word matches whole words only', async () => {- const { facts } = await search('count lease');+ const { facts } = await search('counter lease');expect(facts.map(f => f.id)).not.toContain(ids.lease);});@@ -103,6 +110,15 @@ describe('search saved facts: stem-aware whole-word matching', () => {expect(order(facts, ids.newerLease)).toBeLessThan(order(facts, ids.olderLease));});+ test('a question\'s framing words ("count") do not keep out the fact that states the figure', async () => {+ for (const flag of ['false', 'true']) {+ await engine.setConfig(SAVED_FACTS_SUPERSEDE_KEY, flag);+ const { facts } = await search('Zentra seat count');+ expect(facts.map(f => f.id), flag).toEqual(expect.arrayContaining([ids.zentra, ids.zentraQuestion]));+ }+ await engine.setConfig(SAVED_FACTS_SUPERSEDE_KEY, 'false');+ });+test('a fact the remote caller may not read never comes back', async () => {for (const q of WORDINGS) expect((await search(q)).text, q).not.toContain('are 40');});@@ -162,6 +178,19 @@ describe('search.saved_facts_supersede on', () => {}});+ test('an acknowledgement saved after a correction does not crowd it out', async () => {+ await engine.setConfig(SAVED_FACTS_SUPERSEDE_KEY, 'true');+ try {+ const { facts } = await search('Quellin seats renewal');+ expect(order(facts, ids.quellinFix)).toBeGreaterThanOrEqual(0);+ expect(order(facts, ids.quellinFix)).toBeLessThan(order(facts, ids.quellin));+ expect(order(facts, ids.quellinAck)).toBeLessThan(order(facts, ids.quellin));+ expect(facts.find(f => f.id === ids.quellin)?.corrected_by).toBeDefined();+ } finally {+ await engine.setConfig(SAVED_FACTS_SUPERSEDE_KEY, 'false');+ }+ });+test('a newer fact with no correction cue relabels nothing', async () => {await engine.setConfig(SAVED_FACTS_SUPERSEDE_KEY, 'true');try {