Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

recall: score a long question on the terms its candidates hold, stamp facts_below_floor (GST-1304)

garrytan73c2c3cedf66
diff --git a/docs/architecture/key-files/core-search-2.md b/docs/architecture/key-files/core-search-2.md
index 4e261a2612..9e83b52faf 100644
--- a/docs/architecture/key-files/core-search-2.md
+++ b/docs/architecture/key-files/core-search-2.md
@@ -16,7 +16,7 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL
- Evidence date headers (`search/evidence-date.ts`) and the tail trim (`search/evidence-tail-trim.ts`): [Core Search (evidence blocks)](core-search-evidence.md).
- `src/core/search/entity-anchor.ts` — entity-anchored retrieval: `entityAnchoredPages` (the entity page plus up to 10 pages that link to it by a ranking link type or name it as a word, newest by `COALESCE(effective_date, updated_at)`; question/synthesis/pinned-question pages excluded) and `prefixAnchoredPages`, shared by pinned-question refresh (`questions/refresh.ts` `anchoredSlugs`) and the `query`/`search` ops; `asksForCurrentState` (deterministic cue regex), `namedEntity` (exactly one `isFactEntityPage` page whose title `isTitleMentionedInQuery` matches, the title match before the read-policy filter; a title inside a longer matched title is the same mention); `applyEntityAnchoring` (anchored rows first, at most half the row count, organic rows of the same page moved up, missing pages read through `getChunkWindows` under the caller's scope, rows stamped `entity_anchored`, token budget re-applied, fail-open); `entityAnchoringEnabled` (`search.entity_anchoring`, default off). Tests: `test/search-entity-anchor.test.ts`. Gates: `docs/eval/decisions/entity-anchoring-query/`.
- The facts arm, temporal fact reserve, facts block and fact source pages (`search/facts-arm.ts`): [Core Search (evidence blocks)](core-search-evidence.md).
-- `src/core/search/fact-relevance.ts` — question relevance for saved facts, shared by recall's `question` (`facts/question-recall.ts`), the temporal fact reserve and (pool only) the facts arm. `collectFactCandidates`: one bounded pool under recall's read policy (source scope, active and not superseded, audit rows excluded, quarantine and rederive hiding through `projectionEligibleSql`, the trust floor, remote world-only, `privateProvenanceFilterFragment` when `excludePrivate`) plus recall's filters inside every arm before its LIMIT (`(source_id, entity_slug)` pairs, `source_session`, event-time since, escaped `ILIKE` grep); arms, run concurrently and merged in a fixed order, are keyword FTS (`factsFtsDocument`, served by `idx_facts_fts`), cosine (cast to the facts column type, filtered on model only so the HNSW index serves it, a stale hash or other dimensions yielding no similarity; `ef_search` sized to the depth and pgvector's iterative scan when `hnswIterativeScan` is set) and the named entity page's facts in its own source; depths are bound parameters; `cosineForAll` scores every candidate's cosine whichever arm found it. A missing embedding column or other dimensions skips the cosine arm; any other DB error throws. `scoreFactCandidates`: cosine + whole-word term share + 0.1 when dated, the share's denominator growing past `TERM_STUFFING_RATIO` times the question's terms for a long fact; order score, newest valid_from, id. `admitted`: the `reserve` rule (cosine >= 0.5 or share >= 0.34) or the `facts_arm` rule (cosine >= 0.6, half the terms, or the named entity). `countUncomparableFacts` counts the pool's facts the cosine arm cannot compare, per source: candidates from `idx_facts_unembedded` and `idx_facts_embedding_model` first, the read policy on those rows only. The keyword arm and the count run under `customPlan` (`plan_cache_mode = force_custom_plan` in a transaction): a cached generic plan cannot see the source arrays and nests the policy anti-joins over every match. p95 at 100k facts: Postgres 51 ms local / 65 ms remote, PGLite 152 / 139 ms. Tests: `test/fact-relevance.test.ts`, `test/recall-question.test.ts`.
+- `src/core/search/fact-relevance.ts` — question relevance for saved facts, shared by recall's `question` (`facts/question-recall.ts`), the temporal fact reserve and (pool only) the facts arm. `collectFactCandidates`: one bounded pool under recall's read policy (source scope, active and not superseded, audit rows excluded, quarantine and rederive hiding through `projectionEligibleSql`, the trust floor, remote world-only, `privateProvenanceFilterFragment` when `excludePrivate`) plus recall's filters inside every arm before its LIMIT (`(source_id, entity_slug)` pairs, `source_session`, event-time since, escaped `ILIKE` grep); arms, run concurrently and merged in a fixed order, are keyword FTS (`factsFtsDocument`, served by `idx_facts_fts`), cosine (cast to the facts column type, filtered on model only so the HNSW index serves it, a stale hash or other dimensions yielding no similarity; `ef_search` sized to the depth and pgvector's iterative scan when `hnswIterativeScan` is set) and the named entity page's facts in its own source; depths are bound parameters; `cosineForAll` scores every candidate's cosine whichever arm found it; `distinctiveTerms` (recall's question, GST-1304) searches a question over `MAX_QUERY_TERMS` (12) content terms on its last 24 and returns in `terms` the 12 of those the fewest candidates hold (`heldTerms`; later word first on a tie), which `scoreFactCandidates` takes as its third argument, so a provider preamble no fact holds stops filling the share's denominator. A missing embedding column or other dimensions skips the cosine arm; any other DB error throws. `scoreFactCandidates`: cosine + whole-word term share + 0.1 when dated, the share's denominator growing past `TERM_STUFFING_RATIO` times the question's terms for a long fact; order score, newest valid_from, id. `admitted`: the `reserve` rule (cosine >= 0.5 or share >= 0.34) or the `facts_arm` rule (cosine >= 0.6, half the terms, or the named entity). `countUncomparableFacts` counts the pool's facts the cosine arm cannot compare, per source: candidates from `idx_facts_unembedded` and `idx_facts_embedding_model` first, the read policy on those rows only. The keyword arm and the count run under `customPlan` (`plan_cache_mode = force_custom_plan` in a transaction): a cached generic plan cannot see the source arrays and nests the policy anti-joins over every match. p95 at 100k facts: Postgres 51 ms local / 65 ms remote, PGLite 152 / 139 ms. Tests: `test/fact-relevance.test.ts`, `test/recall-question.test.ts`.
- `src/core/search/query-arms.ts` — `applyQueryArms`: the `query` op's post-retrieval arms in order (entity anchoring, fact source pages, then the facts arm), each behind its own key (the facts arm default on, the rest off); a query whose caller passed `token_budget` takes the facts block (`search.query_facts_block`, default on; at most `limit` rows, scored at the top page's score; #8411) or, with a temporal cue, the temporal fact reserve (`search.temporal_fact_reserve`) instead of the facts arm; with all off it returns the rows untouched.
- `src/core/search/image-vector-presence.ts` — `brainHasImageVectors` (one `EXISTS embedding_image IS NOT NULL` probe per engine, cached 60 s, fail-open to true). `hybrid/arms.ts` uses it so an inferred image/both modality on a brain with no image embeddings stays text: no multimodal embed, keyword arm kept, no `vector_arm_failed`; the opt-in LLM modality tie-break still runs, and an image or both answer is demoted to text. An explicit `crossModal` is honored. Tests: `test/search-image-arm-text-only.test.ts`, `test/cross-modal-hybrid-integration.serial.test.ts`.
diff --git a/docs/mcp/TOOL_REFERENCE.md b/docs/mcp/TOOL_REFERENCE.md
index 5795d4e53f..487e81a815 100644
--- a/docs/mcp/TOOL_REFERENCE.md
+++ b/docs/mcp/TOOL_REFERENCE.md
@@ -354,7 +354,7 @@ MEMORY VERB (v1): retrieve saved facts/snippets — the protocol read verb. Filt
| Parameter | Type | Guidance |
| --- | --- | --- |
| `entity` | string | Entity slug (canonical). Returns facts about this entity newest first, each labelled with its source_id. Across several granted sources, same-slug entities that no entity-identity group links are different entities: facts comes back empty and ambiguous_entity names each (source_id, entity_slug); pass source_id to read one. |
-| `question` | string | Ranks active facts by relevance to this question (cosine to its embedding plus whole-word term share, +0.1 for a writer-dated fact) instead of newest first. Facts below the admission rule are left out, so an empty facts[] is a valid answer. Each fact carries `relevance` (a ranking score, about 0 to 2.1, not comparable across calls); the response carries `facts_order: "relevance"` and, when ranking ran without full semantic matching, `facts_degraded`. When semantic matching ran on an embedding model whose fact admission floors were never measured, the response also carries the informational `facts_calibration: {status: "unmeasured", model_key}` (one `facts_calibration_unmeasured` notice per session explains it; `gbrain doctor --only fact_admission_floors --json` names the model): treat a short facts list as weaker evidence of "nothing matched". Applies inside `entity`, `session_id`, `since` and `grep`. Does not search pages (`query` does; both may be passed). With `include_expired`, expired and superseded facts are ranked too. Rejected with `supersessions`, and over 2,000 characters (invalid_params, with the corrected call as `fix`). One query embedding per call, shared with a `query` of the same text. |
+| `question` | string | Ranks active facts by relevance to this question (cosine to its embedding plus whole-word term share, +0.1 for a writer-dated fact) instead of newest first. Facts below the admission rule are left out, so an empty facts[] is a valid answer. Each fact carries `relevance` (a ranking score, about 0 to 2.1, not comparable across calls); the response carries `facts_order: "relevance"` and, when ranking ran without full semantic matching, `facts_degraded`. When semantic matching ran on an embedding model whose fact admission floors were never measured, the response also carries the informational `facts_calibration: {status: "unmeasured", model_key}` (one `facts_calibration_unmeasured` notice per session explains it; `gbrain doctor --only fact_admission_floors --json` names the model): treat a short facts list as weaker evidence of "nothing matched". Applies inside `entity`, `session_id`, `since` and `grep`. Does not search pages (`query` does; both may be passed). With `include_expired`, expired and superseded facts are ranked too. A question over 12 content words is searched on its last 24 and scored on the 12 of those the fewest candidates hold. `facts_below_floor: <n>` is present when candidates were found and none cleared the admission rule (absent when nothing matched). Rejected with `supersessions`, and over 2,000 characters (invalid_params reason `question_too_long`, detail `max_chars=2000 received_chars=<n>`, with the corrected call as `fix`; never cut). One query embedding per call, shared with a `query` of the same text. |
| `query` | string | MEMORY_VERBS v1: free-text retrieval over pages (hybrid search arm). Response adds results[] (slug, title, chunk, evidence, create_safety, provenance). Combinable with entity (both arms run). Degrades to keyword-only search when no embedding provider is configured (search_degraded notes it; never an error). |
| `budget_tokens` | number | MEMORY_VERBS v1: server-side token budget (char/4 estimate). Facts pack first, then results. Response adds budget_tokens, budget_used, dropped_count. |
| `budget_policy` | string (`facts_first`, `query_first`) | Optional packing order. facts_first (default) packs facts first, then page results. query_first packs the ranked page prefix before facts only with a nonblank query and positive finite budget; a positive budget below one token keeps neither arm. Without a nonblank query and an active budget, packing stays facts_first. Neither policy skips oversized items or truncates. Supplying this option adds budget_packing accounting. Keep fact-focused/entity-filtered questions on facts_first. |
diff --git a/docs/protocol/MEMORY_VERBS_v1.md b/docs/protocol/MEMORY_VERBS_v1.md
index 12a9c9ebc6..1672f34d9f 100644
--- a/docs/protocol/MEMORY_VERBS_v1.md
+++ b/docs/protocol/MEMORY_VERBS_v1.md
@@ -144,7 +144,13 @@ Retrieve saved facts and (with `query`) budget-packed page snippets.
error's `fix.mcp` is the call without it. One query
embedding per call, no model call. Without an embedding provider, with
embeddings turned off, or when the embed fails, facts rank by term share and
- `facts_degraded.reason` says why. A server without question ranking ignores
+ `facts_degraded.reason` says why. A question over 12 content words is searched
+ on its last 24 and scored on the 12 of those the fewest candidate facts hold,
+ so an instruction preamble ahead of the turn does not crowd out its words.
+ When candidates were found and none cleared the admission rule, the response
+ carries `facts_below_floor: <n>`; it is absent when nothing matched. Over
+ 2,000 characters is `invalid_params` reason `question_too_long` with detail
+ `max_chars=2000 received_chars=<n>`; the question is never cut. A server without question ranking ignores
the parameter (or rejects it under `mcp.strict_params=reject`); clients detect
support by `facts_order: "relevance"` in the response or `question` in the
live tool schema, not by `protocol_version`.
diff --git a/llms-full.txt b/llms-full.txt
index 28e9762b23..a3614749d0 100644
--- a/llms-full.txt
+++ b/llms-full.txt
@@ -8517,7 +8517,13 @@ Retrieve saved facts and (with `query`) budget-packed page snippets.
error's `fix.mcp` is the call without it. One query
embedding per call, no model call. Without an embedding provider, with
embeddings turned off, or when the embed fails, facts rank by term share and
- `facts_degraded.reason` says why. A server without question ranking ignores
+ `facts_degraded.reason` says why. A question over 12 content words is searched
+ on its last 24 and scored on the 12 of those the fewest candidate facts hold,
+ so an instruction preamble ahead of the turn does not crowd out its words.
+ When candidates were found and none cleared the admission rule, the response
+ carries `facts_below_floor: <n>`; it is absent when nothing matched. Over
+ 2,000 characters is `invalid_params` reason `question_too_long` with detail
+ `max_chars=2000 received_chars=<n>`; the question is never cut. A server without question ranking ignores
the parameter (or rejects it under `mcp.strict_params=reject`); clients detect
support by `facts_order: "relevance"` in the response or `question` in the
live tool schema, not by `protocol_version`.
diff --git a/scripts/module-size-limits.tsv b/scripts/module-size-limits.tsv
index 9b7eb67f22..c3382525d8 100644
--- a/scripts/module-size-limits.tsv
+++ b/scripts/module-size-limits.tsv
@@ -47,4 +47,4 @@ src/commands/doctor/checks/extraction-sync.ts 1500 ratchet +7 GBRA-130 A-1: link
src/core/search/evidence-delivery.ts 1759 ratchet +5 W5: EvidencePlan.rankCut resolved with the plan (logic in search/page-rank-cut.ts); +20 W5 whole_pages: wholeCost prices a block as delivery would emit it whole (needs the private Block/enrichBlock internals) and the cut is wired before allocate (cutWholePages lives in search/page-rank-cut.ts); new row: crossed the 1500 unlisted cap at the #6367 merge (master 1479, its explicit-budget cap) by the C1 date header threaded through every block path, cap included; pagePlanHits (search/page-plan-hits.ts) and the header loader (evidence-date.ts blockDateHeaders) were peeled first; peel candidate: allocateCapped; +8 at the #6396 merge: page evidence text cut through the fence-chunk trust overlay and the caller minTrust floor (re-summed: our 1504 + master +8); +119 Cat 40 next wave F2: the search.long_page_unit policy (withLongPagePolicy, selectLongPages size preflight with page/byte/deadline limits) and the long-page fetch and fallback in deliverEvidence, kept in the shared seam the plan names (gbrain-evals docs/plans/2026-10-11-cat40-next-wave, F2); +17 F2 rework to the frozen-comparison policy: search.long_page_min_chars (rendered-body characters, default 20,000) read in withLongPagePolicy, a character count in the preflight, the rendered-body confirmation in deliverEvidence, and the long-page row exemption in capDeliveredSnippets (re-summed at the 0.60.225.0 merge: master 1537 + F2 136); +39 Wave B PR 2 lane 2b: planSegmentedBlock joins the windows search/segmented-evidence.ts reads around a segmented page's hits into one run-marked document (needs the private Doc/Piece/buildDoc/locateAnchor internals), planBlock's unit cut split into blockFromDoc for both, sections bounded by their run, selectionCost counts a gap between runs, and the fetch skips the whole-body fence overlay for a segmented page; +4 segmented pages read at once before planning (segmentedReads; the read is search/segmented-evidence.ts, the statements page-body/read-range.ts); +17 GBRA-156: tail trim plan flag, DeliverOptions.query, DeliveryMeta.tail_trim, the trim call and the frozen-hit score/query library fields (logic in search/evidence-tail-trim.ts; interface fields cannot move); +7 GBRA-205: EvidencePlan.titleHeader and DeliveryMeta.title_header (interface fields cannot move) and the titles option on the header loader (logic in search/evidence-date.ts; the query budget key lives in search/evidence-packing.ts); +11 #7600: assemble_evidence quotes conversation rows and capEvidenceToBudget counts the quote fence (logic in search/transcript-quote.ts); +2 GBRA-156 /ship review: the cut-path admission and title room count the quote fence; +2 GBRA-156 /ship review pass 2: capEvidenceToBudget counts the fence only for the paths that fence (quote flag; recall counts the text as returned); +4 GST-687: page evidence and the extractive fallback quote below-owner-tier text for untrusted readers (untrusted flag; logic in search/transcript-quote.ts)
src/commands/config.ts 1524 ratchet new row: crossed the 1500 unlisted cap in wave 15 P7.4 #5461: set-time validation of think.pages_budget_chars / think.excerpt_ceiling_chars (range + parser live in core/think/pages-budget.ts), mirroring the cycle.consolidate.cluster_threshold hook; +1 wave 16 G5.1 (#6495): set-time refusal hook for search.keyword_statement_timeout_ms (check lives in commands/config/keyword-timeout-value.ts); +1 wave 18b lane 5 (#5012 item 6): search.contextual_retrieval prints the existing-pages hint after the write (text lives in commands/config/contextual-mode-hint.ts); +1 #6712 PR 2: retired/rollback sync key dispatch line (logic in core/sync-config-keys.ts handleSyncKeyConfig); +1 #6865: sync.slug_conflicts.auto_remove boolean validation at set time (refusal text in repair/slug-autoheal.ts); +1 #6948: sync.origin_twins.auto_resolve boolean validation at set time (refusal text in repair/origin-twins-heal.ts); +2 #6795: config set chat_model/expansion_model help line and the model-pin import (shadow report and file-pin unset live in commands/config/model-pin-shadow.ts); +1 #6712 PR 4b: sync.admission set-time hook (value logic in commands/config/sync-admission-value.ts); +4 #7130: set-time refusal hook for sync.max_deletes / sync.max_delete_ratio (and .source.<id>) / purge.retention_hours (check lives in core/sync-reconcile.ts deleteSafetyConfigRefusal); +1 GBRA-150: sync.metafile_pages set-time route (body in config/sync-metafile-pages-value.ts); re-summed at the v0.60.330.0 merge (1520 measured); +2 #7242 minions.terminal_hook file-plane key and its set branch (logic in minions/terminal-hook.ts); +1 GBRA-161: generations.* set-time route (refusal in commands/config/generations-values.ts); +1 GBRA-207 #7630: sync.repo_path set-time refusal hook (refusal in core/sync-config-keys.ts syncRepoPathRefusal)
src/commands/mcp-expose.ts 1519 ratchet hosted_option: Session carries the gbrain.io coaching notice on the first-receipt run and Tailscale/service friction exits (copy, refs and gating live in core/onboard/hosted-option.ts); +7 GBRA-199: --surface help lines, surfaceChosen option field and the advertise pass-through (registrationAdvertise in core/mcp-registration.ts)
-src/core/behavior-change-notice.ts 1615 ratchet the BEHAVIOR_CHANGES table grows by one row per disclosed change: +4 the #7891 provenance-drift and #7917/#7918 maintenance-lowering rows and their comments (re-summed at the 0.60.495.0 merge: 1547 + 4); +2 GBRA-205 saved_facts_stems row and its comment (re-summed at the #7901 0.60.492.0 merge: 1541 + 2); +2 GBRA-205 search.query_facts_block default-on row and its approval comment (re-summed at the #7610 0.60.490.0 merge: 1539 + 2); +2 the #7900 content checkpoint host-move row (since TBD until the merge slot), +2 the #7897 private-hold sweep row (since TBD until the merge slot), +2 the #7880 ingest_capture and company-brain private-hold row (since TBD until the merge slot), +5 the frozen-source wave comment and four rows (#7868, #7850, #7869/#7870, #7871), +2 the #7848 visibility_noncanonical hold row (since TBD until the merge slot), +2 the #7846 extract_atoms session-source row (re-summed at the 0.60.481.0 merge: master 1524 + 2), +2 the #7805 synthesize session-source and #7841 visibility_not_applied rows (re-summed at the 0.60.480.0 merge: master 1522 + 2), +2 strict door planned-state row (#7705, re-summed at the 0.60.470.0 merge: master 1514 + 2), +2 Bun floor 1.4.4 row (#7644), the GBRA-95 RLS backfill row (+2), +3 rows for #7596 pass_interrupted, #7599 read-only serve and the --access=read-only ceiling, +1 G5 lane C trust_release row, +1 #7619 cli_flag_equals, +3 GBRA-207 #7630 relative-anchor and stalled-write rows, +2 the #7632 function search_path disclosure row and +2 the #7679 orphaned worktree refresh row (re-summed at the 0.60.471.0 merge: master 1516 + 2), +2 the #7814 retired v0.13.1 grandfather row, +2 the #7618 conversion withhold/progress/push row (master 1520 + 2); the behavior-change table split (#7607) retires this row; +4 GBRA-188 #7489: the wave comment and three rows (brain-first instructions, starter surface default, per-turn rule); +1 GBRA-188 plugin_turn_hook_gbra188 row; +1 GBRA-242 provider_key_rules_7959 row (#7959), +2 the GBRA-199 registration_lists_verbs row (since TBD until the merge slot); +2 GBRA-133 #7652: the #7611 pass_abandoned reader row and the lean read-only serve socket row (re-summed at the 0.60.503.0 merge: master 1555 + 2); +1 GBRA-243: the per-turn hook disclosure row (rows are data and cannot move); +4 GBRA-156 #7598 harness wave comment and rows (remote page cap, sources_not_readable refusal, sources add next step); +4 GBRA-156 #7601 search_quotes_transcript_rows comment and row; +1 GBRA-175 #7475 autopilot_restarts_serve_7475 row (re-summed at the #8012 merge: 1566 + 1); +3 #7997/#8014/#8015 rows (redaction markers, company db_only keep, ignored supabase_only scope); +2 #7199 (re-port of #7228) classic_drift_rescreen_7199 comment and row; +2 #7931 file_drift_agrees_7931 comment and row; +1 #7950 file_drift_slug_release_7950 row; +1 #7974 file_drift_followups_7974 row; +1 #8059 quoted_fence_marker_8059 row; +2 GBRA-247 #8009 provenance_repair_held_8009 and #7982 pass_transient_streak_7982 rows; +4 GBRA-247 #7973 plain_directory_sync_7973, #7988 managed_marker_own_directory_7988, #7985 former_binding_parked_group_7985 and settle_effects_retire_7985 rows Train S merge: 1570 base + 5 (master since the #8077 base) + 13 (#8077) = 1588.; +2 GST-687 remote_quotes_unverified_text_gst687 row and its comment (re-summed at the gate-ahead on train S 0.60.534.0: 1588 + 2); +2 #8377: the quote fence follow-ups row and its comment; +2 #8384: the quote fence follow-ups 2 row and its comment; +2 #8384 wave 3: the quote fence follow-ups 3 row and its comment; +5 GBRA-218 train M rows (A1 member_read_access_gbra218 and member_access_hatch_scope_gbra218, B shared_agent_audience_gbra231, A2 folder_access_rules_gbra218, folder_access_scoped_role_gbra218), re-summed at the train M merge of master v0.60.541.0: 1596 + 5 = 1601 measured; +3 GST-359 train C rows (C1 qm_attested_actor_gst359, C2 member_facts_gst359, D qm_memory_provider_gst359), re-summed at the train C merge of master v0.60.543.0: 1601 + 3 = 1604 measured; +2 #8424: hook_launcher_8424 row and its comment (rows are data and cannot move), re-summed at the #8418 v0.60.546.0 merge: 1604 + 2 = 1606 measured; +2 #8426: mcp_lane_launcher_8426 row and its comment (rows are data and cannot move); +2 GST-927: empty_nonstring_args_gst927 row and its comment, re-summed at the gate-ahead on master v0.60.550.0 plus #8420: 1610 measured; +4 GST-1094/GST-900: the memory_recall scope-read, member-bound acting_user, recall max_chars and unknown_param _meta-only rows (since TBD until the merge slot); +1 GST-1113: the qm_scope_writes_gst1113 row (since TBD until the merge slot; rows are data and cannot move)
+src/core/behavior-change-notice.ts 1616 ratchet the BEHAVIOR_CHANGES table grows by one row per disclosed change: +4 the #7891 provenance-drift and #7917/#7918 maintenance-lowering rows and their comments (re-summed at the 0.60.495.0 merge: 1547 + 4); +2 GBRA-205 saved_facts_stems row and its comment (re-summed at the #7901 0.60.492.0 merge: 1541 + 2); +2 GBRA-205 search.query_facts_block default-on row and its approval comment (re-summed at the #7610 0.60.490.0 merge: 1539 + 2); +2 the #7900 content checkpoint host-move row (since TBD until the merge slot), +2 the #7897 private-hold sweep row (since TBD until the merge slot), +2 the #7880 ingest_capture and company-brain private-hold row (since TBD until the merge slot), +5 the frozen-source wave comment and four rows (#7868, #7850, #7869/#7870, #7871), +2 the #7848 visibility_noncanonical hold row (since TBD until the merge slot), +2 the #7846 extract_atoms session-source row (re-summed at the 0.60.481.0 merge: master 1524 + 2), +2 the #7805 synthesize session-source and #7841 visibility_not_applied rows (re-summed at the 0.60.480.0 merge: master 1522 + 2), +2 strict door planned-state row (#7705, re-summed at the 0.60.470.0 merge: master 1514 + 2), +2 Bun floor 1.4.4 row (#7644), the GBRA-95 RLS backfill row (+2), +3 rows for #7596 pass_interrupted, #7599 read-only serve and the --access=read-only ceiling, +1 G5 lane C trust_release row, +1 #7619 cli_flag_equals, +3 GBRA-207 #7630 relative-anchor and stalled-write rows, +2 the #7632 function search_path disclosure row and +2 the #7679 orphaned worktree refresh row (re-summed at the 0.60.471.0 merge: master 1516 + 2), +2 the #7814 retired v0.13.1 grandfather row, +2 the #7618 conversion withhold/progress/push row (master 1520 + 2); the behavior-change table split (#7607) retires this row; +4 GBRA-188 #7489: the wave comment and three rows (brain-first instructions, starter surface default, per-turn rule); +1 GBRA-188 plugin_turn_hook_gbra188 row; +1 GBRA-242 provider_key_rules_7959 row (#7959), +2 the GBRA-199 registration_lists_verbs row (since TBD until the merge slot); +2 GBRA-133 #7652: the #7611 pass_abandoned reader row and the lean read-only serve socket row (re-summed at the 0.60.503.0 merge: master 1555 + 2); +1 GBRA-243: the per-turn hook disclosure row (rows are data and cannot move); +4 GBRA-156 #7598 harness wave comment and rows (remote page cap, sources_not_readable refusal, sources add next step); +4 GBRA-156 #7601 search_quotes_transcript_rows comment and row; +1 GBRA-175 #7475 autopilot_restarts_serve_7475 row (re-summed at the #8012 merge: 1566 + 1); +3 #7997/#8014/#8015 rows (redaction markers, company db_only keep, ignored supabase_only scope); +2 #7199 (re-port of #7228) classic_drift_rescreen_7199 comment and row; +2 #7931 file_drift_agrees_7931 comment and row; +1 #7950 file_drift_slug_release_7950 row; +1 #7974 file_drift_followups_7974 row; +1 #8059 quoted_fence_marker_8059 row; +2 GBRA-247 #8009 provenance_repair_held_8009 and #7982 pass_transient_streak_7982 rows; +4 GBRA-247 #7973 plain_directory_sync_7973, #7988 managed_marker_own_directory_7988, #7985 former_binding_parked_group_7985 and settle_effects_retire_7985 rows Train S merge: 1570 base + 5 (master since the #8077 base) + 13 (#8077) = 1588.; +2 GST-687 remote_quotes_unverified_text_gst687 row and its comment (re-summed at the gate-ahead on train S 0.60.534.0: 1588 + 2); +2 #8377: the quote fence follow-ups row and its comment; +2 #8384: the quote fence follow-ups 2 row and its comment; +2 #8384 wave 3: the quote fence follow-ups 3 row and its comment; +5 GBRA-218 train M rows (A1 member_read_access_gbra218 and member_access_hatch_scope_gbra218, B shared_agent_audience_gbra231, A2 folder_access_rules_gbra218, folder_access_scoped_role_gbra218), re-summed at the train M merge of master v0.60.541.0: 1596 + 5 = 1601 measured; +3 GST-359 train C rows (C1 qm_attested_actor_gst359, C2 member_facts_gst359, D qm_memory_provider_gst359), re-summed at the train C merge of master v0.60.543.0: 1601 + 3 = 1604 measured; +2 #8424: hook_launcher_8424 row and its comment (rows are data and cannot move), re-summed at the #8418 v0.60.546.0 merge: 1604 + 2 = 1606 measured; +2 #8426: mcp_lane_launcher_8426 row and its comment (rows are data and cannot move); +2 GST-927: empty_nonstring_args_gst927 row and its comment, re-summed at the gate-ahead on master v0.60.550.0 plus #8420: 1610 measured; +4 GST-1094/GST-900: the memory_recall scope-read, member-bound acting_user, recall max_chars and unknown_param _meta-only rows (since TBD until the merge slot); +1 GST-1113: the qm_scope_writes_gst1113 row (since TBD until the merge slot; rows are data and cannot move); +1 GST-1304: the recall_question_terms_gst1304 row (since TBD until the merge slot; rows are data and cannot move)
diff --git a/src/core/behavior-change-notice.ts b/src/core/behavior-change-notice.ts
index 453997915f..41a904873b 100644
--- a/src/core/behavior-change-notice.ts
+++ b/src/core/behavior-change-notice.ts
@@ -1415,6 +1415,7 @@ export const BEHAVIOR_CHANGES: ReadonlyArray<{ since: string; text: ChangeText;
{ since: 'TBD', id: 'memory_recall_scope_read_gst1094', text: '`memory_recall` now takes `source_id` and `source_ids`. A call that names sources reads as their audience, never as the attested person: no own recent facts first and no fact from any member page; a page restricted to named readers stays hidden and `visibility: private` is never relaxed. A source outside the credential\'s read grant is refused (`permission_denied`) and `__all__` is refused (`invalid_params`). Without sources, `memory_recall` is unchanged. The contract is `docs/contracts/qm-memory-provider/v2.json`; `v1.json` is unchanged.' },
{ since: 'TBD', id: 'member_bound_acting_user_gst900', text: 'A credential bound to a member may now send `acting_user` with no `x-qm-actor` header: its own member id, or a canonical id linked to that member, is dropped and the call runs exactly as the credential; any other value is refused (`actor_not_attested` reason `argument_mismatch`). A credential bound to no member is still refused (`argument_without_attestation`).' },
{ since: 'TBD', id: 'qm_scope_writes_gst1113', text: '`memory_write` now takes `source_id`: a credential that lists the source in `gbrain auth rescope --attest-sources` saves the facts on that scope source\'s `memory/scope` page (`visibility: world`, attested to the person); any other source is refused (`permission_denied` reason `source_not_attested`, or `invalid_params` reason `source_is_member_source` for its own write source). Listing a source marks it a QM scope source for good: it is never federated (`sources federate` refuses, `sources restore` leaves it unfederated) and no other grant, token or registration may read it. Remote writes to `memory/scope` other than a scope `memory_write` are refused, and `memory_recall` without sources never reads a scope source.' },
+ { since: 'TBD', id: 'recall_question_terms_gst1304', text: '`recall` with a `question` over 12 content words (a provider\'s instruction preamble ahead of the turn) is now searched on its last 24 content words and scored on the 12 of those the fewest candidate facts hold, instead of on its first 12, so a keyword-only brain no longer returns no facts because the preamble filled the 12. Shorter questions rank as before. A response whose candidates all fell under the admission floor carries `facts_below_floor: <n>`, and a question over 2,000 characters is refused with reason `question_too_long` and detail `max_chars=2000 received_chars=<n>`.' },
{ since: 'TBD', id: 'recall_max_chars_harn15', text: '`recall` now takes `max_chars` (a whole number from 200 to 12000, as `memory_recall`): it keeps the whole facts, then results, whose text fits, adds no marker and returns `truncated`; without it `recall` is unchanged.' },
{ since: 'TBD', id: 'unknown_param_meta_only_harn15', text: 'A read that ignores a parameter the tool does not declare (`mcp.strict_params` `warn`, the default) no longer adds a `[gbrain notice unknown_param kind=info]` text block to the result: the warning is only in `_meta.warnings` (`{code: \'unknown_param\', param, suggestion?}`), so a harness that stores tool output in its turns never stores it. Writes still refuse unknown parameters, and `reject` mode is unchanged.' },
];
diff --git a/src/core/facts/question-recall.ts b/src/core/facts/question-recall.ts
index c7f9ed30e1..ad8579109d 100644
--- a/src/core/facts/question-recall.ts
+++ b/src/core/facts/question-recall.ts
@@ -181,6 +181,8 @@ export interface QuestionRecall {
degraded?: FactsDegraded;
/** Present when the cosine arm ran on a model with no measured admission floor. */
calibration?: FactsCalibration;
+ /** Candidates found when none cleared the admission rule (an empty answer that is not an empty match). */
+ belowFloor?: number;
/** The source holding the most uncomparable facts, for the `embed --stale --facts` fix. */
unembeddedSource?: string;
ambiguousEntity: EntityCandidate[] | null;
@@ -212,12 +214,13 @@ export async function rankFactsByQuestion(ctx: OperationContext, input: Question
const [pool, uncomparable, rule] = await Promise.all([
collectFactCandidates(ctx.engine, question, scope, embedding, {
depth: { keyword: QUESTION_POOL_DEPTH, cosine: QUESTION_POOL_DEPTH, entity: QUESTION_POOL_DEPTH }, filters, cosineForAll: true,
- hnswIterativeScan: resolveHnswIterativeScan(ctx.config),
+ hnswIterativeScan: resolveHnswIterativeScan(ctx.config), distinctiveTerms: true,
}),
embedding ? countUncomparableFacts(ctx.engine, scope, filters, embedding.length) : [],
admissionRule(ctx.engine),
]);
- const ranked = scoreFactCandidates(question, pool.candidates).filter(s => admitted(s, rule));
+ const scored = scoreFactCandidates(question, pool.candidates, pool.terms);
+ const ranked = scored.filter(s => admitted(s, rule));
const unembedded = uncomparable.reduce((n, r) => n + r.n, 0);
const why = reason ?? (embedding && !pool.column ? 'facts_embedding_column_missing' : undefined);
const degraded: FactsDegraded | undefined = why || unembedded ? { ...(why ? { reason: why } : {}), ...(unembedded ? { unembedded } : {}) } : undefined;
@@ -231,6 +234,7 @@ export async function rankFactsByQuestion(ctx: OperationContext, input: Question
rows: (await collapseFactReads(ctx.engine, ids.flatMap(id => byId.get(id) ?? []))).slice(0, input.limit),
relevance: new Map(ranked.map(s => [s.candidate.id, Math.round(s.score * 1e4) / 1e4])),
ambiguousEntity: null,
+ ...(scored.length && !ranked.length ? { belowFloor: scored.length } : {}),
...(degraded ? { degraded } : {}),
...(modelKey && FACT_ADMISSION_FLOOR_EVIDENCE[modelKey] === undefined ? { calibration: { status: 'unmeasured' as const, model_key: modelKey } } : {}),
...(uncomparable[0] ? { unembeddedSource: uncomparable[0].source_id } : {}),
diff --git a/src/core/ops/facts.ts b/src/core/ops/facts.ts
index 40a165d1f4..41ceaf7e8f 100644
--- a/src/core/ops/facts.ts
+++ b/src/core/ops/facts.ts
@@ -640,7 +640,7 @@ const recall: Operation = {
total: packedFacts.length,
facts_order: ranking ? 'relevance' : 'newest',
...(quoted.withheld ? { facts_degraded: { reason: 'saved_facts_withheld' } } : ranking?.degraded ? { facts_degraded: ranking.degraded } : {}),
- ...(ranking?.calibration ? { facts_calibration: ranking.calibration } : {}),
+ ...(ranking?.calibration ? { facts_calibration: ranking.calibration } : {}), ...(ranking?.belowFloor ? { facts_below_floor: ranking.belowFloor } : {}),
...(ambiguousEntity ? { ambiguous_entity: { candidates: ambiguousEntity, suggestion: AMBIGUOUS_ENTITY_SUGGESTION } } : {}),
...(pending_consolidation_count !== undefined ? { pending_consolidation_count } : {}),
// MEMORY_VERBS v1 envelope (G1B superset — additive on every response).
diff --git a/src/core/search/fact-relevance.ts b/src/core/search/fact-relevance.ts
index 450d2451ab..a417c38b3d 100644
--- a/src/core/search/fact-relevance.ts
+++ b/src/core/search/fact-relevance.ts
@@ -42,9 +42,15 @@ const STOPWORDS = new Set(['the', 'and', 'for', 'who', 'what', 'when', 'where',
'now', 'current', 'currently', 'should', 'does', 'did', 'has', 'have', 'how', 'any', 'all', 'about', 'into', 'its', 'next', 'use', 'uses', 'tell']);
const WORD_RE = /[\p{L}\p{N}][\p{L}\p{N}'-]+/gu;
+/** A question with more content terms than this is scored on the ones its candidates hold (`heldTerms`). */
+export const MAX_QUERY_TERMS = 12;
+/** The content terms such a question is searched on: its last ones, where a provider puts the turn after its preamble. */
+const SEARCH_TERMS = 24;
+const contentTerms = (query: string) => [...new Set(query.toLowerCase().match(WORD_RE) ?? [])].filter(t => t.length >= 3 && !STOPWORDS.has(t));
+
/** The question's content terms: distinct words of 3+ characters, stopwords dropped, at most 12. */
export function queryTerms(query: string): string[] {
- return [...new Set(query.toLowerCase().match(WORD_RE) ?? [])].filter(t => t.length >= 3 && !STOPWORDS.has(t)).slice(0, 12);
+ return contentTerms(query).slice(0, MAX_QUERY_TERMS);
}
export const DATED_BONUS = 0.1;
@@ -102,6 +108,8 @@ export interface FactPoolOpts {
cosineForAll?: boolean;
/** Set for filtered recall pools: hnsw.ef_search sized to the depth plus pgvector's iterative scan when supported. */
hnswIterativeScan?: HnswIterativeScanMode;
+ /** Search a question over MAX_QUERY_TERMS content terms on its last SEARCH_TERMS and score it on those its candidates hold (recall). */
+ distinctiveTerms?: boolean;
}
export interface FactCandidate {
@@ -118,6 +126,8 @@ export interface FactPool {
cosine: boolean;
/** Whether the facts table has an embedding column (probed only when an embedding was given). */
column: boolean;
+ /** The content terms the keyword arm searched and `scoreFactCandidates` scores with. */
+ terms: string[];
}
const COLS = 'f.id, f.fact, f.kind, f.entity_slug, f.source_id, f.source, f.valid_from, f.valid_until, f.created_at, f.claim_metric, f.claim_period';
@@ -170,13 +180,29 @@ function customPlan<T>(engine: BrainEngine, sql: string, params: unknown[]): Pro
});
}
+/**
+ * GST-1304: a question of up to MAX_QUERY_TERMS content terms is scored on them all. A longer one (a provider's
+ * instruction preamble ahead of the turn) is searched on its last SEARCH_TERMS and scored on the MAX_QUERY_TERMS of
+ * those the fewest pooled candidates hold, the later word first on a tie, so preamble words no readable fact holds stop
+ * filling the term share's denominator. The pool is already under the caller's read policy; when no candidate holds a
+ * term, the first MAX_QUERY_TERMS searched.
+ */
+function heldTerms(searched: string[], candidates: FactCandidate[]): string[] {
+ if (searched.length <= MAX_QUERY_TERMS) return searched;
+ const words = candidates.map(wordsOf);
+ const held = searched.map((t, i) => ({ i, n: words.filter(w => w.has(t)).length })).filter(r => r.n > 0)
+ .sort((a, b) => a.n - b.n || b.i - a.i).slice(0, MAX_QUERY_TERMS);
+ return held.length ? held.sort((a, b) => a.i - b.i).map(r => searched[r.i]!) : searched.slice(0, MAX_QUERY_TERMS);
+}
+
/**
* The bounded candidate pool for `question`. Throws on a database error
* (callers choose fail-open or `unavailable`).
*/
export async function collectFactCandidates(engine: BrainEngine, question: string, scope: FactPoolScope, queryEmbedding: Float32Array | null | undefined,
opts: FactPoolOpts): Promise<FactPool> {
- const terms = queryTerms(question);
+ const all = contentTerms(question);
+ const searched = opts.distinctiveTerms && all.length > MAX_QUERY_TERMS ? all.slice(-SEARCH_TERMS) : all.slice(0, MAX_QUERY_TERMS);
const { where, params } = poolPredicate(scope, opts.filters);
const column = queryEmbedding?.length ? await factsEmbeddingColumn(engine) : null;
const cosine = !!column && (column.dims <= 0 || column.dims === queryEmbedding!.length);
@@ -197,14 +223,14 @@ export async function collectFactCandidates(engine: BrainEngine, question: strin
type Row = FactCandidate & { similarity?: number | null };
const keyword = async (): Promise<Row[]> => {
- if (!terms.length) return [];
+ if (!searched.length) return [];
const p = armParams(allSim ? simParams : []);
const doc = factsFtsDocument('f');
const tsq = `NULLIF(replace(plainto_tsquery('${getFtsLanguage()}'::regconfig, $${p.length + 1})::text, ' & ', ' | '), '')::tsquery`;
return customPlan<Row>(engine,
`SELECT ${COLS}, ${allSim ? simSql : 'NULL::float8 AS similarity'} FROM facts f WHERE ${where} AND ${doc} @@ ${tsq}
ORDER BY ts_rank_cd(${doc}, ${tsq}) DESC, f.valid_from DESC, f.id DESC LIMIT $${p.length + 2}`,
- [...p, terms.join(' '), opts.depth.keyword]);
+ [...p, searched.join(' '), opts.depth.keyword]);
};
const nearest = async (): Promise<Row[]> => {
if (!vec) return [];
@@ -235,7 +261,8 @@ export async function collectFactCandidates(engine: BrainEngine, question: strin
add(byKeyword);
add(byCosine);
add(byEntity, true);
- return { candidates: [...found.values()], cosine, column: !!column };
+ const candidates = [...found.values()];
+ return { candidates, cosine, column: !!column, terms: heldTerms(searched, candidates) };
}
/**
@@ -289,8 +316,7 @@ export const isDatedFact = (f: Pick<FactCandidate, 'valid_from' | 'created_at'>)
!!f.created_at && Math.abs(new Date(f.valid_from).getTime() - new Date(f.created_at).getTime()) > DATED_MIN_MS;
/** Score every candidate (cosine + term share + DATED_BONUS when dated), ordered by score, newest valid_from, then id. */
-export function scoreFactCandidates(question: string, candidates: FactCandidate[]): ScoredFact[] {
- const terms = queryTerms(question);
+export function scoreFactCandidates(question: string, candidates: FactCandidate[], terms = queryTerms(question)): ScoredFact[] {
return candidates.map(c => {
const words = wordsOf(c);
const matched = terms.filter(t => words.has(t)).length;
diff --git a/src/core/verbs.ts b/src/core/verbs.ts
index 14ded0861a..a5dd73b672 100644
--- a/src/core/verbs.ts
+++ b/src/core/verbs.ts
@@ -604,6 +604,7 @@ export const RESPONSE_SCHEMAS: Record<VerbName, Record<string, unknown>> = {
unembedded: { type: 'integer', description: 'Facts in scope without a comparable embedding (ranked by word overlap only).' },
},
},
+ facts_below_floor: { type: 'integer', description: 'Present when question ranking found candidate facts and none cleared the admission floor: how many. Absent when nothing matched.' },
facts_calibration: {
type: 'object',
description: 'Informational, present when question ranking used semantic matching on an embedding model whose fact admission floors were never measured: a short facts list is weaker evidence of "nothing matched". Never replaces facts_degraded.',
diff --git a/test/recall-question-preamble.test.ts b/test/recall-question-preamble.test.ts
new file mode 100644
index 0000000000..42832da9c7
--- /dev/null
+++ b/test/recall-question-preamble.test.ts
@@ -0,0 +1,74 @@
+/**
+ * GST-1304 (HARN-31): with embeddings off, recall's question ranking scored facts on the question's first 12 distinct
+ * content words. A provider question that opens with a long instruction preamble filled those 12 slots with preamble
+ * words no fact holds, so no fact cleared the term-share floor and the agent got 0 facts. A question over 12 content
+ * terms now ranks on the ones the readable facts hold, rarest first; and a 0 because candidates fell under the floor is
+ * stamped (facts_below_floor), apart from a store with nothing to match.
+ *
+ * 1. Protects: a preamble-led long question finds the fact its tail asks about; a short question is unchanged; the
+ * below-floor stamp appears only when candidates were found and none was admitted.
+ * 2. Fails when: term selection goes back to the first 12 words, or the stamp disappears or fires on an empty match.
+ * 3. Existing coverage: test/fact-relevance.test.ts (scoring), test/recall-question.test.ts (short questions).
+ * 4. Real seams: PGLite, MCP dispatch as a remote caller, keyless gateway; no providers.
+ */
+import { afterAll, beforeAll, describe, expect, test } from 'bun:test';
+import type { BrainEngine } from '../src/core/engine.ts';
+import { PGLiteEngine } from '../src/core/pglite-engine.ts';
+import { dispatchToolCall } from '../src/mcp/dispatch.ts';
+import { configureGateway, resetGateway } from '../src/core/ai/gateway.ts';
+import { queryTerms } from '../src/core/search/fact-relevance.ts';
+import type { AuthInfo } from '../src/core/ops/contract.ts';
+
+const TARGET = 'The team offsite is in Lisbon; alice-example booked the venue.';
+const NOISE = 'The user prefers concise answers.';
+const PREAMBLE = 'You are a memory assistant embedded in a chat product. Read the conversation excerpt below carefully and '
+ + 'return only the saved notes that help answer the latest message. Ignore greetings, small talk, signatures and '
+ + 'formatting instructions. Never invent information, never summarise the excerpt itself, and keep answers concise for '
+ + 'the user. Prioritise durable preferences, plans, decisions, commitments and relationships. Conversation excerpt '
+ + 'follows, oldest turn first, newest turn last, separated by blank lines. ';
+const ASK = 'Latest message: where is the team offsite, and who booked the Lisbon venue?';
+
+describe('recall question with a preamble (pglite, keyless)', () => {
+ let engine: BrainEngine;
+ beforeAll(async () => {
+ configureGateway({ env: {} });
+ engine = new PGLiteEngine();
+ await engine.connect({ engine: 'pglite' } as never);
+ await engine.initSchema();
+ for (const fact of [TARGET, NOISE, 'Dark roast coffee every morning.', 'The quarterly review moved to Thursday.']) {
+ await engine.insertFact({ fact, kind: 'event', entity_slug: null, source: 'test', visibility: 'world' } as never, { source_id: 'default' });
+ }
+ }, 120_000);
+ afterAll(async () => { resetGateway(); await engine.disconnect(); });
+
+ const recall = async (question: string) => {
+ const auth = { token: 't', clientId: 'c', scopes: ['read'], sourceId: 'default' } as unknown as AuthInfo;
+ const out = await dispatchToolCall(engine, 'recall', { question, caller: 'provider' }, { remote: true, transport: 'http', auth, sourceId: 'default', takesHoldersAllowList: ['world'] });
+ let json: Record<string, any> = {};
+ try { json = JSON.parse(out.content[0]?.text ?? ''); } catch { /* not JSON */ }
+ return { isError: out.isError === true, json };
+ };
+ const facts = (json: Record<string, any>) => (json.facts ?? []).map((f: { fact: string }) => [TARGET, NOISE].find(t => f.fact.includes(t)) ?? f.fact);
+
+ test('a preamble-led question finds the fact its last message asks about', async () => {
+ const question = PREAMBLE + ASK;
+ expect(queryTerms(question).some(t => TARGET.toLowerCase().includes(t))).toBe(false);
+ const { isError, json } = await recall(question);
+ expect(isError).toBe(false);
+ expect(facts(json)[0]).toBe(TARGET);
+ expect(json.facts_below_floor).toBeUndefined();
+ });
+
+ test('the same ask without the preamble still finds it', async () => {
+ expect(facts((await recall(ASK)).json)[0]).toBe(TARGET);
+ });
+
+ test('candidates under the floor are stamped; a question nothing matches is not', async () => {
+ const under = await recall('lisbon weather forecast tomorrow morning'); // the Lisbon fact and the morning coffee fact, each 1 of 4 terms
+ expect(facts(under.json)).toEqual([]);
+ expect(under.json.facts_below_floor).toBe(2);
+ const none = await recall('kayak paddle rental');
+ expect(facts(none.json)).toEqual([]);
+ expect(none.json.facts_below_floor).toBeUndefined();
+ });
+});