Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
docs(key-files): saved-facts-match entry beside entity-facts
garrytan3629be11ed6c
diff --git a/docs/architecture/key-files/core-search-2.md b/docs/architecture/key-files/core-search-2.mdindex e3c1300052..4e74cd4de0 100644--- a/docs/architecture/key-files/core-search-2.md+++ b/docs/architecture/key-files/core-search-2.md@@ -17,7 +17,6 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL- `src/core/search/entity-anchor.ts` — entity-anchored retrieval: `entityAnchoredPages` (the entity page plus up to 10 pages that link to it by a ranking link type or name it as a word, newest by `COALESCE(effective_date, updated_at)`; question/synthesis/pinned-question pages excluded) and `prefixAnchoredPages`, shared by pinned-question refresh (`questions/refresh.ts` `anchoredSlugs`) and the `query`/`search` ops; `asksForCurrentState` (deterministic cue regex), `namedEntity` (exactly one `isFactEntityPage` page whose title `isTitleMentionedInQuery` matches, the title match before the read-policy filter; a title inside a longer matched title is the same mention); `applyEntityAnchoring` (anchored rows first, at most half the row count, organic rows of the same page moved up, missing pages read through `getChunkWindows` under the caller's scope, rows stamped `entity_anchored`, token budget re-applied, fail-open); `entityAnchoringEnabled` (`search.entity_anchoring`, default off). Tests: `test/search-entity-anchor.test.ts`. Gates: `docs/eval/decisions/entity-anchoring-query/`.- `src/core/search/facts-arm.ts` — the `query` facts arm behind `search.query_facts_arm` (default on since gates 1b and 2 passed; `false` turns it off). `matchQueryFacts` reads the candidate pool from `fact-relevance.ts` (active facts under recall's read policy) and matches them by query-embedding cosine (`FACT_COSINE_MIN` 0.6, the embedding `hybridSearch` already computed, via `HybridSearchOpts.onQueryEmbedding`) or by query-term share (half the terms, substring match) or a named entity (in that page's source), newest `valid_from` first, at most `MAX_FACT_ROWS` (3). `applyFactsArm` adds them as `fact_row` rows in spare capacity only: free slots under the caller's row count, and whole rows that fit in what the page rows leave of the token budget; a page row is never displaced `factRowOutput` strips page fields (slug, id, type, chunk_id) from fact rows on output, leaving `result_type: 'fact'`, `fact_id`, `follow_up` (`recall` by entity, or `grep` without one) and `page_slug` when the caller can read the entity page (`readableEntityPages`); and stamps `superseded_claim` on page rows whose typed `claim_metric` (plus `claim_period` when both carry it) a newer active fact covers. No model call. Tests: `test/search-facts-arm.test.ts`. Gates and verdict: `docs/eval/decisions/query-facts-arm/`. Temporal fact reserve: `applyTemporalFactReserve` behind `search.temporal_fact_reserve` (default off): `hasTemporalCue` (the preregistered word, phrase, ISO date and month-name regex), candidates from `collectFactCandidates` (50 cosine, 50 keyword, 20 entity facts, cosine for every candidate), scored and ordered by the shared `scoreFactCandidates` and kept by its `reserve` admission rule (fail-open on any error), packed into 15% of the token budget and 30% of the row count (1-20), rendered as fact rows whose text starts with `factDateHeader` (evidence delivery adds no second header), oldest first after the pages. Tests: `test/search-temporal-fact-reserve.test.ts`. Gates: `docs/eval/decisions/temporal-fact-reserve/`.- `src/core/search/fact-relevance.ts` — question relevance for saved facts, shared by recall's `question` (`facts/question-recall.ts`), the temporal fact reserve and (pool only) the facts arm. `collectFactCandidates`: one bounded pool under recall's read policy (source scope, active and not superseded, audit rows excluded, quarantine and rederive hiding through `projectionEligibleSql`, the trust floor, remote world-only, `privateProvenanceFilterFragment` when `excludePrivate`) plus recall's filters inside every arm before its LIMIT (`(source_id, entity_slug)` pairs, `source_session`, event-time since, escaped `ILIKE` grep); arms, run concurrently and merged in a fixed order, are keyword FTS (`factsFtsDocument`, served by `idx_facts_fts`), cosine (cast to the facts column type, filtered on model only so the HNSW index serves it, a stale hash or other dimensions yielding no similarity; `ef_search` sized to the depth and pgvector's iterative scan when `hnswIterativeScan` is set) and the named entity page's facts in its own source; depths are bound parameters; `cosineForAll` scores every candidate's cosine whichever arm found it. A missing embedding column or other dimensions skips the cosine arm; any other DB error throws. `scoreFactCandidates`: cosine + whole-word term share + 0.1 when dated, the share's denominator growing past `TERM_STUFFING_RATIO` times the question's terms for a long fact; order score, newest valid_from, id. `admitted`: the `reserve` rule (cosine >= 0.5 or share >= 0.34) or the `facts_arm` rule (cosine >= 0.6, half the terms, or the named entity). `countUncomparableFacts` counts the pool's facts the cosine arm cannot compare, per source: candidates from `idx_facts_unembedded` and `idx_facts_embedding_model` first, the read policy on those rows only. The keyword arm and the count run under `customPlan` (`plan_cache_mode = force_custom_plan` in a transaction): a cached generic plan cannot see the source arrays and nests the policy anti-joins over every match. p95 at 100k facts: Postgres 51 ms local / 65 ms remote, PGLite 152 / 139 ms. Tests: `test/fact-relevance.test.ts`, `test/recall-question.test.ts`.-- `src/core/search/saved-facts-match.ts` — `matchingSavedFacts`, the word-matched saved facts `search` and `query` report in `retrieval.saved_facts` after the named entity's facts (`ops/search.ts` `buildRetrievalResponseMeta`). Words match whole after stemming with the facts keyword index's expression (`factsFtsDocument`), CJK words as substrings; a fact needs two words of a two- or three-word query and three quarters of a longer one; most matched words first, then newest (time, then id), at most five after the #7043 collapse. One statement behind the facts-only EXISTS probe (`test/search/saved-facts-probe.test.ts`). `search.saved_facts_supersede` (default off): a newer fact with a correction cue that shares two matched words with a returned fact is returned too, and the facts it corrects move last with `corrected_by`. Also owns `FACT_MATCH_STOPWORDS`. Tests: `test/search-saved-facts-match.test.ts`, `test/search-saved-facts.test.ts`.- `src/core/search/query-arms.ts` — `applyQueryArms`: the `query` op's post-retrieval arms in order (entity anchoring, then the facts arm), each behind its own key (entity anchoring default off, the facts arm default on); a temporal-cue query with a token budget takes the temporal fact reserve instead of the facts arm when `search.temporal_fact_reserve` is on; with all off it returns the rows untouched.- `src/core/search/image-vector-presence.ts` — `brainHasImageVectors` (one `EXISTS embedding_image IS NOT NULL` probe per engine, cached 60 s, fail-open to true). `hybrid/arms.ts` uses it so an inferred image/both modality on a brain with no image embeddings stays text: no multimodal embed, keyword arm kept, no `vector_arm_failed`; the opt-in LLM modality tie-break still runs, and an image or both answer is demoted to text. An explicit `crossModal` is honored. Tests: `test/search-image-arm-text-only.test.ts`, `test/cross-modal-hybrid-integration.serial.test.ts`.diff --git a/docs/architecture/key-files/entity-recall.md b/docs/architecture/key-files/entity-recall.mdindex f963f16d65..c2f2be1a5d 100644--- a/docs/architecture/key-files/entity-recall.md+++ b/docs/architecture/key-files/entity-recall.md@@ -24,6 +24,7 @@ card and `get_backlinks`. Guide: [entity recall](../../guides/entity-recall.md).- `src/core/verbs/entity-card-identity.ts` — the `entity` card's `aka_sources`, `identity_siblings`, `identity_excerpt` (verbatim lines of the page and its siblings from `sanitizeRemoteBody`, tiered: declarations, label lines with a name-like value, other label lines, lines with another capitalized run or code; 600 characters per page, 1,200 per card, `identity_excerpt_omitted`), `alias_guidance` (surface-aware) and the `identity_siblings_capped` notice. A segmented page is excerpted from its trusted, withdrawal-overlaid lead (`truthLead`, `EXCERPT_LEAD_BYTES` = 64 KiB, cut back to a whole line), never its body getters; the card equals the inline page's when the qualifying lines sit in that lead (`test/large-pages/segmented-read-refusals.test.ts`). `readSiblingIdentity` is its siblings-and-aliases half (one alias read for the page and its siblings; optional preloaded mention policy and pack), used alone by `entity names[]`. Tests: `test/mentions-identity-siblings.test.ts`.- `src/core/search/alias-fanout.ts` — search and query's alias fan-out: `resolveQueryEntity` (query n-grams up to 4 tokens through `resolveAliases` and linkable exact titles, longest first, capitalized or code-shaped unless a multi-word n-gram resolves to one page, case-sensitive aliases as written, a tie only when the pages are identity siblings), the names of the entity and its siblings by origin then length, at most `search.alias_fanout_max` (4) keyword queries of 5 rows (`searchAliasRequired`: the name as a required websearch phrase, ranked by the query's other terms through `SearchOpts.rankQuery` on both engines), at most 8 spliced rows with `matched_alias`, and the `fanout` meta (`alias_fanout` notice when capped). Offset 0 only. `resolving` takes the handler's `resolveQueryEntity` started before the search, so its reads overlap it. Tests: `test/search-alias-fanout.test.ts`, `test/search-other-names.test.ts`, `test/e2e/mentions-parity.test.ts`. `wholeQueryEntityNames`: when the whole query is one entity's name or alias, `match: "keyword"` lists pages matching any of its names (one websearch OR of phrases; `match_names` in the meta and the count line; rows found by another name carry `matched_alias`). `resolveQueryEntities` returns every entity a text names; its alias, linkable-title and case-sensitive alias reads run concurrently (`test/search-entity-resolve-concurrency.test.ts`).- `src/core/search/entity-facts.ts` — `entitySavedFacts`: when a search names an entity, its newest active saved facts (`factEntitySlugs`, 5) lead the model-visible saved-facts block, then up to 3 facts each of up to 2 entities those facts name (`linked_from`), before the word-overlap matches. Same active, audit-row and visibility rules as recall. Tests: `test/memory-sibling-facts.test.ts`.+- `src/core/search/saved-facts-match.ts` — `matchingSavedFacts`, the word-matched saved facts `search` and `query` report in `retrieval.saved_facts` after the named entity's facts (`ops/search.ts` `buildRetrievalResponseMeta`). Words match whole after stemming with the facts keyword index's expression (`factsFtsDocument`), CJK words as substrings; a fact needs two words of a two- or three-word query and three quarters of a longer one; most matched words first, then newest (time, then id), at most five after the #7043 collapse. One statement behind the facts-only EXISTS probe (`test/search/saved-facts-probe.test.ts`). `search.saved_facts_supersede` (default off): a newer fact with a correction cue that shares two matched words with a returned fact is returned too, and the facts it corrects move last with `corrected_by`. Also owns `FACT_MATCH_STOPWORDS`. Tests: `test/search-saved-facts-match.test.ts`, `test/search-saved-facts.test.ts`.- `src/core/schema-migrations/v206-entity-mention-index.ts` — additive: `page_aliases.origin` (`frontmatter` default, `declared`, `subject`), `case_sensitive`, `alias_text`, unique key `(source_id, alias_norm, slug, origin)`; tables `page_mention_state`, `mention_gazetteer_entries`, `mention_index_status` (RLS on when the role bypasses RLS). `GUARDED_TABLE_COLUMNS` lists the new `page_aliases` columns.Alias readers apply precedence frontmatter > declared > subject per source, and a derived alias never answers for another live page's exact title that the caller can read (the title competitor gets the caller's `pageReadFilter`, so a private title a private-excluding caller can't see never hides a readable alias) (`readAliases` in `src/core/search/read-enrichment.ts`, shared by both engines' `resolveAliases`), so `resolveEntitySlug`, the retrieval reflex, the alias hop, intent weights and `search-diagnose` rank an exact title first. `buildGazetteer` (`src/core/by-mention.ts`) applies the same precedence, drops an alias claimed by two pages at its best origin (unless the claimants are identity siblings, which all keep it), yields to a same-source title, drops a single-word alias equal to the first word of another multi-word entity name, applies the generic-word list to every single-word name, and carries `caseTokens` for case-sensitive entries; `opts.dropped` records every dropped name with its reason.