Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

search: question-framing words do not count toward the saved-fact threshold; every correction of a returned fact comes back

garrytan7ff7cbb5bd0e
diff --git a/docs/architecture/RETRIEVAL.md b/docs/architecture/RETRIEVAL.md
index 8c96dd7942..9720e25e39 100644
--- a/docs/architecture/RETRIEVAL.md
+++ b/docs/architecture/RETRIEVAL.md
@@ -345,14 +345,17 @@ Two cross-cutting seams sit around the pipeline rather than inside it:
full-text language, the facts keyword index's expression, so "renewal" finds
"renews" and "count" does not match inside "countersigned"; words of a CJK
script match as substrings. A fact needs three quarters of the query's words
- (both of a two-word query); the most matched words come first, then the
+ (both of a two-word query), not counting words that frame a question about a
+ figure (count, number, total, amount); the most matched words come first, then the
newest. This is the only place `search` returns saved facts: the
facts arm, the facts block and the temporal reserve run in `query` alone.
With `search.saved_facts_supersede` (default on; `false` turns it off), a newer fact with a
correction cue ("correction", "ignore the ...", "not what I posted") that
shares at least two matched words with a returned fact is returned too, even
below the threshold, every fact it corrects moves below the facts nothing
- corrects, and those carry `corrected_by` (the correcting fact's id). An
+ corrects, and those carry `corrected_by` (the correcting fact's id, the
+ newest when several correct it). Every correction comes back, so an
+ acknowledgement saved after a correction does not crowd it out. An
acknowledgement corrects nothing: a cue fact that adds no figure (a number
other than a date or time) to the older fact, when it restates one of its
figures or the older fact is itself a correction. The cue is a fixed word list, so
diff --git a/docs/architecture/key-files/entity-recall.md b/docs/architecture/key-files/entity-recall.md
index b8e42bec4e..9289086ab4 100644
--- a/docs/architecture/key-files/entity-recall.md
+++ b/docs/architecture/key-files/entity-recall.md
@@ -24,7 +24,7 @@ card and `get_backlinks`. Guide: [entity recall](../../guides/entity-recall.md).
- `src/core/verbs/entity-card-identity.ts` — the `entity` card's `aka_sources`, `identity_siblings`, `identity_excerpt` (verbatim lines of the page and its siblings from `sanitizeRemoteBody`, tiered: declarations, label lines with a name-like value, other label lines, lines with another capitalized run or code; 600 characters per page, 1,200 per card, `identity_excerpt_omitted`), `alias_guidance` (surface-aware) and the `identity_siblings_capped` notice. A segmented page is excerpted from its trusted, withdrawal-overlaid lead (`truthLead`, `EXCERPT_LEAD_BYTES` = 64 KiB, cut back to a whole line), never its body getters; the card equals the inline page's when the qualifying lines sit in that lead (`test/large-pages/segmented-read-refusals.test.ts`). `readSiblingIdentity` is its siblings-and-aliases half (one alias read for the page and its siblings; optional preloaded mention policy and pack), used alone by `entity names[]`. Tests: `test/mentions-identity-siblings.test.ts`.
- `src/core/search/alias-fanout.ts` — search and query's alias fan-out: `resolveQueryEntity` (query n-grams up to 4 tokens through `resolveAliases` and linkable exact titles, longest first, capitalized or code-shaped unless a multi-word n-gram resolves to one page, case-sensitive aliases as written, a tie only when the pages are identity siblings), the names of the entity and its siblings by origin then length, at most `search.alias_fanout_max` (4) keyword queries of 5 rows (`searchAliasRequired`: the name as a required websearch phrase, ranked by the query's other terms through `SearchOpts.rankQuery` on both engines), at most 8 spliced rows with `matched_alias`, and the `fanout` meta (`alias_fanout` notice when capped). Offset 0 only. `resolving` takes the handler's `resolveQueryEntity` started before the search, so its reads overlap it. Tests: `test/search-alias-fanout.test.ts`, `test/search-other-names.test.ts`, `test/e2e/mentions-parity.test.ts`. `wholeQueryEntityNames`: when the whole query is one entity's name or alias, `match: "keyword"` lists pages matching any of its names (one websearch OR of phrases; `match_names` in the meta and the count line; rows found by another name carry `matched_alias`). `resolveQueryEntities` returns every entity a text names; its alias, linkable-title and case-sensitive alias reads run concurrently (`test/search-entity-resolve-concurrency.test.ts`).
- `src/core/search/entity-facts.ts` — `entitySavedFacts`: when a search names an entity, its newest active saved facts (`factEntitySlugs`, 5) lead the model-visible saved-facts block, then up to 3 facts each of up to 2 entities those facts name (`linked_from`), before the word-overlap matches. Same active, audit-row and visibility rules as recall. Tests: `test/memory-sibling-facts.test.ts`.
-- `src/core/search/saved-facts-match.ts` — `matchingSavedFacts`, the word-matched saved facts `search` and `query` report in `retrieval.saved_facts` after the named entity's facts (`ops/search.ts` `buildRetrievalResponseMeta`). Words match whole after stemming with the facts keyword index's expression (`factsFtsDocument`), CJK words as substrings; a fact needs three quarters of the query's words; most matched words first, then newest (time, then id), at most five after the #7043 collapse. One statement behind the facts-only EXISTS probe (`test/search/saved-facts-probe.test.ts`). `search.saved_facts_supersede` (default on; `false` turns it off): a newer fact with a correction cue that shares two matched words with a returned fact is returned too, even below the threshold, and the facts it corrects move last with `corrected_by`; a cue fact that adds no figure to the older fact (`figures`: numbers, dates and times left out) corrects nothing when it restates one of its figures or the older fact is itself a correction. Also owns `FACT_MATCH_STOPWORDS`. Tests: `test/search-saved-facts-match.test.ts`, `test/search-saved-facts.test.ts`.
+- `src/core/search/saved-facts-match.ts` — `matchingSavedFacts`, the word-matched saved facts `search` and `query` report in `retrieval.saved_facts` after the named entity's facts (`ops/search.ts` `buildRetrievalResponseMeta`). Words match whole after stemming with the facts keyword index's expression (`factsFtsDocument`), CJK words as substrings; a fact needs three quarters of the query's words, question-framing words (`QUESTION_FRAME_WORDS`: count, number, total, amount) not counted; most matched words first, then newest (time, then id), at most five after the #7043 collapse. One statement behind the facts-only EXISTS probe (`test/search/saved-facts-probe.test.ts`). `search.saved_facts_supersede` (default on; `false` turns it off): a newer fact with a correction cue that shares two matched words with a returned fact is returned too, even below the threshold, and the facts it corrects move last with `corrected_by` (the newest correction); every correction of a returned fact comes back; a cue fact that adds no figure to the older fact (`figures`: numbers, dates and times left out) corrects nothing when it restates one of its figures or the older fact is itself a correction. Also owns `FACT_MATCH_STOPWORDS`. Tests: `test/search-saved-facts-match.test.ts`, `test/search-saved-facts.test.ts`.
- `src/core/schema-migrations/v206-entity-mention-index.ts` — additive: `page_aliases.origin` (`frontmatter` default, `declared`, `subject`), `case_sensitive`, `alias_text`, unique key `(source_id, alias_norm, slug, origin)`; tables `page_mention_state`, `mention_gazetteer_entries`, `mention_index_status` (RLS on when the role bypasses RLS). `GUARDED_TABLE_COLUMNS` lists the new `page_aliases` columns.
Alias readers apply precedence frontmatter > declared > subject per source, and a derived alias never answers for another live page's exact title that the caller can read (the title competitor gets the caller's `pageReadFilter`, so a private title a private-excluding caller can't see never hides a readable alias) (`readAliases` in `src/core/search/read-enrichment.ts`, shared by both engines' `resolveAliases`), so `resolveEntitySlug`, the retrieval reflex, the alias hop, intent weights and `search-diagnose` rank an exact title first. `buildGazetteer` (`src/core/by-mention.ts`) applies the same precedence, drops an alias claimed by two pages at its best origin (unless the claimants are identity siblings, which all keep it), yields to a same-source title, drops a single-word alias equal to the first word of another multi-word entity name, applies the generic-word list to every single-word name, and carries `caseTokens` for case-sensitive entries; `opts.dropped` records every dropped name with its reason.
diff --git a/src/core/behavior-change-notice.ts b/src/core/behavior-change-notice.ts
index 616db2ae81..dc25f68cc1 100644
--- a/src/core/behavior-change-notice.ts
+++ b/src/core/behavior-change-notice.ts
@@ -1321,7 +1321,7 @@ export const BEHAVIOR_CHANGES: ReadonlyArray<{ since: string; text: ChangeText;
// #7679 / #7685 orphaned worktree refresh. `since` is stamped at the merge slot.
{ since: '0.60.472.0', id: 'refresh_orphan_7679', text: 'A managed `gbrain sources refresh` whose process died (for example a container replaced mid-refresh) no longer wedges the checkout: the next plain `gbrain sources refresh` on the owner host finishes the orphaned refresh through the `--resume` path before its own refresh and reports it as `recovered`, and the owner completes an orphaned `syncing` refresh once every member has synced to its target. A refresh whose holder is still running, or is recorded on another host, is never taken over. `gbrain sources writer status` and `gbrain sync status` now name the holder and its `liveness` (`live`, `orphaned`, `unknown`) with a fix naming `--resume`. With `--json`, a refusal that carries a structured fix (`refresh_in_progress`, `refresh_diverged`, a git read failure) now prints `fix` as an object (argv, actor, next) instead of a string; the prose stays in `suggestion`, so read that when a script needs text. A plain refresh of a checkout whose interrupted refresh needs a reset decision now refuses `refresh_recovery_required` (it refused `refresh_in_progress`).' },
// GBRA-205 saved-facts wave (harness case cr-05). `since` is stamped at the merge slot.
- { since: 'TBD', id: 'saved_facts_stems', text: 'Search `saved_facts` matches a saved fact\'s words the way the facts keyword index does: whole words after stemming with the brain\'s full-text language, so a query for "seats renewal" finds a fact that says "renews at 52 seats", and "count" no longer matches inside "countersigned". A fact still needs three quarters of the query\'s words; words of a CJK script still match as substrings. Among facts matching as many words, the newest comes first, and at most five facts ride a reply. On by default (`gbrain config set search.saved_facts_supersede false` turns it off): search also returns a newer fact with a correction cue that shares two matched words with a returned fact, ahead of the fact it corrects, which is labelled `corrected_by`; an acknowledgement that adds no figure corrects nothing.' },
+ { since: 'TBD', id: 'saved_facts_stems', text: 'Search `saved_facts` matches a saved fact\'s words the way the facts keyword index does: whole words after stemming with the brain\'s full-text language, so a query for "seats renewal" finds a fact that says "renews at 52 seats", and "count" no longer matches inside "countersigned". A fact still needs three quarters of the query\'s words, not counting words that frame a question about a figure (count, number, total, amount); words of a CJK script still match as substrings. Among facts matching as many words, the newest comes first, and at most five facts ride a reply. On by default (`gbrain config set search.saved_facts_supersede false` turns it off): search also returns a newer fact with a correction cue that shares two matched words with a returned fact, ahead of the fact it corrects, which is labelled `corrected_by`; every such correction comes back, not only the newest, and an acknowledgement that adds no figure corrects nothing.' },
];
/** The newest disclosed change's release: the notice id moves only when a release adds rows. */
diff --git a/src/core/search/saved-facts-match.ts b/src/core/search/saved-facts-match.ts
index 62c27aa39e..443386a836 100644
--- a/src/core/search/saved-facts-match.ts
+++ b/src/core/search/saved-facts-match.ts
@@ -19,6 +19,9 @@ import { SAVED_FACT_COLLAPSE_COLUMNS, type CollapseColumns } from './entity-fact
/** Words too common to count toward a saved-fact match; also the coverage relevance stopwords in ops/search.ts. */
export const FACT_MATCH_STOPWORDS = new Set(['the', 'and', 'for', 'who', 'what', 'when', 'where', 'which', 'with', 'from', 'that', 'this', 'are', 'was', 'were', 'our', 'your', 'their', 'now', 'current', 'currently', 'should', 'does', 'did', 'has', 'have', 'how', 'any', 'all', 'about', 'into', 'its', 'next']);
+/** Words that frame a question about a figure ("seat count", "total revenue") and that the fact stating it rarely repeats; they count toward neither side of the threshold. */
+const QUESTION_FRAME_WORDS = new Set(['count', 'number', 'total', 'amount']);
+
export interface SavedFactMatch { id: number; fact: string; entity_slug: string | null; kind: string; valid_from: string; source: string;
/** #7043: every entity of the copies this row stands for, and how many rows it stands for. */
entity_slugs?: string[]; collapsed_count?: number;
@@ -68,7 +71,7 @@ export async function matchingSavedFacts(ctx: OperationContext, scope: { sourceI
// A query naming one of two declared names also matches facts saved under the other.
if (otherNames.length) queryText += ` ${otherNames.join(' ')}`;
const terms = [...new Set(queryText.toLowerCase().match(/[\p{L}\p{N}][\p{L}\p{N}'-]+/gu) ?? [])]
- .filter(t => t.length >= 3 && !FACT_MATCH_STOPWORDS.has(t)).slice(0, 12);
+ .filter(t => t.length >= 3 && !FACT_MATCH_STOPWORDS.has(t) && !QUESTION_FRAME_WORDS.has(t)).slice(0, 12);
if (terms.length === 0 || !ctx.emitResponseMeta) return [];
const sources = scope.sourceIds?.length ? scope.sourceIds : [scope.sourceId ?? ctx.sourceId ?? 'default'];
const [excludePrivate, supersede] = await Promise.all([resolveExcludePrivatePages(ctx.engine, ctx.remote), savedFactsSupersedeEnabled(ctx.engine)]);
@@ -122,17 +125,20 @@ export async function matchingSavedFacts(ctx: OperationContext, scope: { sourceI
at: new Date(at as string).getTime(), words: new Set(matched.map(Number)), need: Number(need), cue, figures: cue ? figures(fact.fact) : new Set<string>() }));
const admitted = found.filter(r => r.words.size >= r.need);
if (supersede) {
+ const correcting = new Set<number>();
for (const older of admitted) {
// Posts saved in one flush share a time; the later id is the later post.
// An acknowledgement is not a correction: a cue fact that adds no figure to the older fact
// corrects it only when it restates no figure and the older fact is not a correction itself.
const olderFigures = figures(older.fact.fact);
- const correction = found.filter(n => n.cue && (n.at > older.at || (n.at === older.at && n.fact.id > older.fact.id))
+ const corrections = found.filter(n => n.cue && (n.at > older.at || (n.at === older.at && n.fact.id > older.fact.id))
&& [...n.words].filter(w => older.words.has(w)).length >= Math.min(2, n.need)
- && !([...n.figures].every(x => olderFigures.has(x)) && (n.figures.size > 0 || older.cue))).sort((a, b) => b.at - a.at || b.fact.id - a.fact.id)[0];
- if (correction) older.fact.corrected_by = correction.fact.id;
+ && !([...n.figures].every(x => olderFigures.has(x)) && (n.figures.size > 0 || older.cue))).sort((a, b) => b.at - a.at || b.fact.id - a.fact.id);
+ if (corrections.length) older.fact.corrected_by = corrections[0].fact.id;
+ for (const c of corrections) correcting.add(c.fact.id);
}
- const correcting = new Set(admitted.flatMap(r => r.fact.corrected_by ? [r.fact.corrected_by] : []));
+ // Every correction of a returned fact comes back, not only the newest: an acknowledgement
+ // saved after the correction must not crowd out the post that made it.
admitted.push(...found.filter(r => correcting.has(r.fact.id) && r.words.size < r.need));
}
const ordered = supersede ? [...admitted.filter(r => !r.fact.corrected_by), ...admitted.filter(r => r.fact.corrected_by)] : admitted;
diff --git a/test/search-saved-facts-match.test.ts b/test/search-saved-facts-match.test.ts
index 0b91117cf8..0659fe658f 100644
--- a/test/search-saved-facts-match.test.ts
+++ b/test/search-saved-facts-match.test.ts
@@ -66,6 +66,13 @@ beforeAll(async () => {
ids.olderLease = await remember('The warehouse lease runs five years.', '2026-08-01T09:00:00Z');
ids.newerLease = await remember('The warehouse lease renews in March.', '2026-08-15T09:00:00Z');
ids.otherAccount = await remember('Kappa Example renews at 30 seats.', '2026-09-01T10:00:00Z');
+ // A correction posted days before the question that asks for the figure, in the words a question uses.
+ ids.zentra = await remember('Dana Example (dana-example, Strategic account manager) wrote in #pod-a on 2026-08-14 09:00: For your records, no action needed: the Q3 digest has Zentra wrong. Zentra Medical Supply is at 50 seats, not 55.', '2026-08-14T09:00:00Z');
+ ids.zentraQuestion = await remember('Erin Example (erin-example, revenue lead) wrote in #pod-a on 2026-08-16 09:00: What is the current Zentra Medical Supply seat count? I am filling in the pod sheet.', '2026-08-16T09:00:00Z');
+ // A correction and an acknowledgement of it saved in one flush: the acknowledgement is the later post.
+ ids.quellin = await remember('Quellin Freight renews at 30 seats.', '2026-05-01T09:00:00Z');
+ ids.quellinFix = await remember('Correction: Quellin Freight is at 24 seats, ignore the 30.', '2026-05-02T09:00:00Z');
+ ids.quellinAck = await remember('Thanks for the correction, noted: Quellin Freight at 24 seats.', '2026-05-02T09:00:00Z');
ids.delta = await remember('Delta Example has 50 seats, not 55.', '2026-08-01T09:00:00Z');
ids.deltaAck = await remember('Correction noted: Delta Example has 50 seats, not 55.', '2026-08-01T09:05:00Z');
ids.omega = await remember('Correction for the record: Omega Example stays with Carol Example.', '2026-07-01T09:00:00Z');
@@ -93,7 +100,7 @@ describe('search saved facts: stem-aware whole-word matching', () => {
});
test('a query word matches whole words only', async () => {
- const { facts } = await search('count lease');
+ const { facts } = await search('counter lease');
expect(facts.map(f => f.id)).not.toContain(ids.lease);
});
@@ -103,6 +110,15 @@ describe('search saved facts: stem-aware whole-word matching', () => {
expect(order(facts, ids.newerLease)).toBeLessThan(order(facts, ids.olderLease));
});
+ test('a question\'s framing words ("count") do not keep out the fact that states the figure', async () => {
+ for (const flag of ['false', 'true']) {
+ await engine.setConfig(SAVED_FACTS_SUPERSEDE_KEY, flag);
+ const { facts } = await search('Zentra seat count');
+ expect(facts.map(f => f.id), flag).toEqual(expect.arrayContaining([ids.zentra, ids.zentraQuestion]));
+ }
+ await engine.setConfig(SAVED_FACTS_SUPERSEDE_KEY, 'false');
+ });
+
test('a fact the remote caller may not read never comes back', async () => {
for (const q of WORDINGS) expect((await search(q)).text, q).not.toContain('are 40');
});
@@ -162,6 +178,19 @@ describe('search.saved_facts_supersede on', () => {
}
});
+ test('an acknowledgement saved after a correction does not crowd it out', async () => {
+ await engine.setConfig(SAVED_FACTS_SUPERSEDE_KEY, 'true');
+ try {
+ const { facts } = await search('Quellin seats renewal');
+ expect(order(facts, ids.quellinFix)).toBeGreaterThanOrEqual(0);
+ expect(order(facts, ids.quellinFix)).toBeLessThan(order(facts, ids.quellin));
+ expect(order(facts, ids.quellinAck)).toBeLessThan(order(facts, ids.quellin));
+ expect(facts.find(f => f.id === ids.quellin)?.corrected_by).toBeDefined();
+ } finally {
+ await engine.setConfig(SAVED_FACTS_SUPERSEDE_KEY, 'false');
+ }
+ });
+
test('a newer fact with no correction cue relabels nothing', async () => {
await engine.setConfig(SAVED_FACTS_SUPERSEDE_KEY, 'true');
try {