Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript
51.9 KB
Raw Blame

Why the hybrid + graph stack works

Vector search alone underdelivers on real personal-knowledge queries. This doc explains why gbrain layers four strategies together and how they compound.

The four strategies in concert

  1. Vector (HNSW on pgvector) — semantic similarity. Catches "who works on retrieval quality at acme-example?" → pages mentioning "alice-example + retrieval" even when the user never typed "acme".
  2. BM25 keyword — lexical match. Catches names, exact phrases, code identifiers, anything where the user remembers the literal token. Survives the cases where vector search drifts into thematic neighbors.
  3. Reciprocal-rank fusion (RRF) — merges vector + keyword rankings without weighting one over the other globally. Each strategy gets to vote, and the vote is for a PAGE: arms return each page's best chunk, often different chunks, so fusion sums a page's votes across arms onto the page's lead chunk (largest own vote, ties to the vector arm's chunk); the page's other chunks keep their own votes so they cannot crowd other pages out of a chunk-limited result.
  4. Knowledge graph traversal — follows recorded typed edges. It can answer relationship queries whose relevant endpoints are not close in embedding space; it depends on the edges being present and correct, and does not by itself establish causality.

Why each one alone fails

Vector only. Returns chunks semantically close to the query. Misses any factual relationship not directly encoded in the embedding. "Companies in alice-example's portfolio" returns essays about portfolios, not company pages.

Keyword only (ripgrep-style). Brittle to phrasing. "Who works on retrieval?" misses pages that say "search ranking" instead of "retrieval." Garbage on synonyms, near-misses, or paraphrases.

Graph only. Excellent at "neighbors of Alice" but blind to anything not yet linked. Sparse on fresh pages until backlinks accumulate.

Hybrid (vector + keyword + RRF), no graph. Can retrieve relationships stated in text, but may miss answers that require joining several records. Typed graph traversal offers a separate recall path for those cases; this is not an exclusivity claim about GBrain or a guarantee that every graph answer is correct.

The benchmark

BrainBench (corpus + harness in the sibling gbrain-evals repo) measures retrieval P@5, R@5, MRR, nDCG@5 on a 240-page Opus-generated rich-prose corpus. (This is the retrieval-ranking benchmark; the in-repo gbrain eval brainbench suite — docs/eval/BRAINBENCH.md — gates the memory behaviors above retrieval: unprompted context push, write-back fidelity, cross-session continuity.)

The September 9, 2026 refresh (gbrain v0.48.4.0, three ingestion orders, fixed-denominator P@5) measured the 145 relationship questions two ways.

The production pipeline. One switch, search.relational_retrieval, over a shared index with everything else held fixed: mean R@5 rose from 0.6626 to 0.7241, better on 45 of 435 question runs and worse on none; investor questions moved most, attendance not at all (production relationship retrieval). On reworded one-hop questions the same switch measured 0.411 to 0.537 (development data, not a held-out set; N9).

Whole-system adapters, including an oracle upper bound:

Adapter Mean P@5 Mean R@5
graph-oracle-parse (knows the four question templates in advance, follows the fixture graph; upper bound, not a product score) 0.3421 0.9791
Reference hybrid 0.1917 0.6874
Keyword ranker 0.1710 0.6244
Vector only 0.1076 0.4069

graph-oracle-parse was called gbrain in these tables until September 28, 2026; the gbrain-evals gbrain adapter now runs the product's hybrid search with relationship retrieval. The oracle is strong on the templates it parses, but this is a comparison between whole systems. The refresh found that the older headline (P@5 49.1%, "+31.4 points over graph-disabled") was not a graph-only effect and used metric helpers since corrected; do not cite it. On the fuzzy and externally authored families the keyword and vector paths win some comparisons. None of this predicts accuracy on a different corpus.

Auto-link: why zero-LLM-call edge extraction works

Trusted local put_page runs inline link extraction on the markdown body. Remote put_page deliberately skips it at the trust boundary. Stdio serving sweeps eligible pages at startup and idle; HTTP serving does not self-sweep. For HTTP-written pages, a trusted operator can run gbrain extract links --source db (select the intended source with --source-id <id>), or a permitted client can create explicit links through add_link. Extraction recognizes:

  • Standard markdown links: [Alice Example](wiki/people/alice-example)
  • Obsidian wikilinks: [[wiki/people/alice-example|Alice Example]]
  • Typed-link blockquotes: > **Convention:** see [path](path).

The link parser uses no LLM tokens and batches SQL insertion through addLinksBatch. Graph freshness depends on the write path and when extraction runs; accepting a remote write is not evidence its links have been extracted.

Heuristic link-type inference (attended, works_at, invested_in, founded, advises) fires from surrounding sentence context — also LLM-free. Power users who want richer types add them via the typed-link blockquote convention.

Cross-encoder reranker: 60% top-1 reshuffle

The reranker is on for the balanced and tokenmax mode bundles, off for conservative. The mode-bundle default is Voyage rerank-3-lite (DEFAULT_RERANKER_MODEL, same VOYAGE_API_KEY as the embedding default); a brain with no search.reranker.model row reranks with it, and a brain with one keeps its own model. It replaced rerank-2.5 after gbrain-evals #148 (v0.10.93) measured 60% lower rerank cost at 50 candidates × 1,400 tokens with quality within noise; gbrain config set search.reranker.model voyage:rerank-2.5 switches a brain back. Without the key, search fails open in RRF order: the gateway skips the HTTP call (RerankError('no_key')), writes ONE audit row per process, prints nothing, and stamps reranker_skipped (no_key) on the search meta — gbrain search --explain shows it, gbrain search modes prints a Reranker: readiness line, and gbrain doctor's reranker_health names the fix (export VOYAGE_API_KEY=… or gbrain config set search.reranker.enabled false). Keyed installs without a Voyage key get reranking explicitly disabled at init. The retained query-cache key includes reranker_model; persisted result reuse is disabled until every response dependency can be verified. On a real-corpus benchmark across 20 queries, the cross-encoder reshuffled 60% of top-1 results after the hybrid + RRF + graph stack (historical measurement on the retired hosted reranker, not Voyage; original identifiers are preserved at Git revision 6040075c6cb95be5881cc2e1b76ef7d71f4e5d29, retained on 2026-09-23). The Voyage default's paired LongMemEval numbers live in docs/eval-bench.md.

The mechanical reason: hybrid ranking is locally optimal per strategy but globally suboptimal. A cross-encoder reranker reads the query + each candidate document jointly, with full attention. It catches the cases where the vector + keyword + graph signals all agreed on a document that's semantically related but topically wrong.

The cost: +150ms p50 latency, ~$0.025–0.05/M tokens depending on the reranker. Disabled with gbrain config set search.reranker.enabled false. For agent loops that do downstream LLM work after retrieval, the latency is invisible.

Source-aware ranking

Hybrid search applies a source-factor CASE expression at the SQL layer (lives in src/core/search/sql-ranking.ts). Curated content like originals/, concepts/, writing/ outranks bulk content like your-openclaw/chat/, daily/, media/x/. Hard-exclude prefixes (test/, attachments/, .raw/) filter at retrieval, not post-rank.

archive/ is deliberately NOT hard-excluded: it holds high-signal historical content users expect to find, so it is demoted (0.5x in DEFAULT_SOURCE_BOOSTS), not hidden. The demote is a prior applied in the outer SQL re-rank; the cross-encoder reranker (balanced/tokenmax modes) can still PROMOTE an archive page that survives the demote into the rerank candidate window — it is not an unconditional suppression. gbrain doctor's hidden_by_search_policy check reports how many chunked pages remain hidden by the surviving exclude prefixes.

The defaults fit one vault layout, so each brain can carry its own map: gbrain config set search.source_boosts "wiki/:1.3,daily/:1.0" overrides or extends the defaults per prefix, and a none entry ("none,wiki/:1.3") drops them entirely. The GBRAIN_SOURCE_BOOST env var (same format) wins over the brain config for one process. Hard exclusions are separate: the exclusion set is defaults ∪ GBRAIN_SEARCH_EXCLUDE (env, comma-separated prefixes) ∪ per-call SearchOpts.exclude_slug_prefixes. Temporal queries (detail: 'high') bypass the boost so chat pages re-surface for time-sensitive lookups.

Named-thing retrieval (per-page pool + title + alias + evidence)

A brain organized around chosen names (project codenames, place nicknames — say a project named "Helios" whose page is also known as "the Sun Room") needs more than embedding proximity. Four layers (the failure mode they close is written up in RETRIEVAL_MAXPOOL_INCIDENT.md):

  • Per-page max-pool — searchVector (both engines) collapses chunk-grain candidates to the best chunk per page (DISTINCT ON (slug)) over the full candidate set before the user LIMIT, via the shared buildBestPerPagePoolCte in sql-ranking.ts. The vector side returns N distinct pages by best chunk, not N chunks that collapse to fewer pages downstream. When one dense page's chunks fill the inner candidate pool, the engines escalate the pool in a bounded loop (×4 per step, at most 3 escalations). SQL candidate limits and offsets are independent of ef_search; supported pgvector versions use relaxed_order iterative scans with bounded visits (ef_search is still sized to the candidate pool; each pooled attempt may visit 20,000 index tuples, so a source or visibility filter covering a tenth of the brain fills its candidate window instead of stopping at the first few hundred eligible chunks). Relaxed order keeps a closer candidate the scan finds after farther ones, which strict order drops, so a selective source or visibility filter fills more of its true neighbours; output order never reaches callers because the inner ORDER BY distance and the pooled outer sort re-sort it. search.hnsw_iterative_scan (relaxed_order | strict_order | off; env GBRAIN_HNSW_ITERATIVE_SCAN wins) restores the older mode without a release; it is latched once per engine (once per process on a one-brain server), so restart serve and autopilot after changing it. A filtered short pool is not proof that the corpus is exhausted. Postgres may make one exact fallback inside the remaining eight-second arm budget; PGLite does not pretend that a JavaScript timeout can cancel its WASM work. Unresolved shortfalls appear as vector_candidates_incomplete in degraded, with scoped vector_pool_underfilled details in the public retrieval metadata. The content-freshness check runs after the HNSW candidate scan on indexed columns (the planner cannot estimate it and would drop the index); doctor vector_plan warns when the emitted statement still misses the index.
  • Title-phrase boost — when the normalized query is a contiguous token-run inside page.title (or an exact full-title match), a floor-ratio-gated, bounded multiplier fires (applyTitleBoost, search.title_boost knob). A query that is a phrase from the title can't lose to a body chunk by luck. Opt-in search.title_qualified (off by default) covers the reverse case: a person or company title (2+ content words) mentioned inside a longer query ("acme alice example") gets the same multiplier, unfloored, stamped title_mention_boost, unless an identity boost already raised the row. Evidence never reads that stamp as an exact title match.
  • Alias hop — free-text aliases: frontmatter is projected into a page_aliases table (separate from the slug_aliases wikilink redirect) and consulted at query time: a full normalized-query match injects/boosts the canonical page (applyAliasHop). The only layer that bridges true synonyms with zero surface overlap ("the Sun Room" → the Helios page). Backfill existing pages with gbrain reindex --aliases. Opt-in search.qualified_aliases (off by default) also matches a 3-6 word sub-phrase with 3+ content words inside a query of up to 14 words: when exactly one page the caller can read claims it, that page leads, stamped qualified_alias_hit; two claimants suppress the hop. The stamp keeps the row past autocut and the remote page cap but never sets evidence alias_hit, and the rerank gate does not grade it. A common three-word alias can lead a query about something else, so confirm on your own eval before turning it on: gbrain config set search.qualified_aliases true (false turns it back off).
  • Evidence contract — every result carries evidence (alias_hit | exact_title_match | high_vector_match | keyword_exact | weak_semantic) and create_safety (exists | probable | unknown). An agent deciding "is this page already here, safe to NOT write a duplicate?" keys off create_safety, not a raw blended score. high_vector_match is grounded in the result's real query↔chunk cosine (SearchResult.cosine at/above search.evidence_cosine_floor, default 0.80) — never the blended score, so a keyword+boost pile-up can't read as semantic support; keyless runs have no cosine and degrade to honest keyword-based labels. gbrain search --explain prints each result's raw cosine next to its blended score.

Extraction quarantine lane: pages carrying the unverified auto-extracted markers (frontmatter provenance: auto-extracted + status: unverified, see src/core/extraction-review.ts) rank as ordinary content — they are skipped by the compiled-truth fusion boost and by the people//companies/ namespace source-boost, and every search result from such a page carries unverified: true so agents can label the provenance. Promote or reject them via gbrain extraction-pending / gbrain extraction-review.

The search MCP/CLI op is cheap-hybrid (vector + keyword + RRF + pool + title + alias, expansion off); query is the full-control variant. Route concept / landscape / "all-of-X" questions to query — expansion recovers synonym-phrased matches search can miss, and a populated search result set is not proof of coverage (both are top-K; exhaustive enumeration belongs to list_pages). NamedThingBench (gbrain eval retrieval-quality) gates these families on every PR. Diagnose a specific miss with gbrain search diagnose "<q>" --target <slug>.

Intent-aware query rewriting

src/core/search/query-intent.ts classifies queries into entity, temporal, event, concept, or general. Each routes through different ranking knobs:

  • Entity queries ("who works at X?") apply a higher graph-traversal weight.
  • Temporal queries ("what happened last week?") bypass source-boost so chat/daily pages surface.
  • Event queries ("Acme AI Series A") engage the timeline index.
  • Concept queries ("what is the ownership economy?", "find all the companies doing offshore wind" — definitional paraphrases and landscape/quantifier phrasings with no proper noun) rank vector-lean, so keyword-decoy pages don't outrank the page that actually explains the idea. Proper nouns, quoted phrases, and sub-3-word queries never classify as concept.
  • General queries hit the standard hybrid stack.

The classifier is deterministic (no LLM call). Wrong classification degrades gracefully — the hybrid stack still works without it.

Multi-query expansion

When requested and available, src/core/search/expansion.ts asks the configured expansion model for up to two alternatives in addition to the original query. This is independent of detail; queries shorter than three words also skip the provider call. Each variant's vector list enters RRF fusion alongside the original's. Expansion can reduce recall: on LongMemEval-S (470 scored questions, k=5, the 2026-09-02 receipt in docs/eval-bench.md) plain hybrid scores 93.19% strict recall_all@5 while hybrid plus equal-weight expansion scores 54.89% (paired +3 / -183 questions). Variant lists with the original's weight can outvote it on small-k retrieval.

Budget-normalized weighted RRF, composed in src/core/search/fusion-lists.ts. Every vector list is a role-tagged arm (original | variant | clause | image) — tagged objects, never a positional convention, so a failed arm or a fell-open image branch can't mis-tag a list. The original arm always fuses at weight 1; the non-empty variant/clause arms share ONE total weight budget, search.expansion_variant_budget (weight_i = b / n_voting_arms, each row scored weight / (k + rank)), so total expansion influence is exactly b however many variants the LLM produced. null, the default in all three mode bundles, is equal-weight fusion (every list weight 1). A budget in (0, 4] is set with gbrain config set search.expansion_variant_budget <b>, per call via HybridSearchOpts.expansionVariantBudget, or pinned per eval arm with gbrain eval longmemeval --expansion-variant-budget <b> (sweep it against frozen --expansion-replay variants so cells differ only in b). Arithmetic: two variants agreeing on a distractor at rank 0 tie the original's rank-0 vote exactly at b = 1.0; equal weighting with two variants is ≈ b = 2.0; b = 0.5 subordinates them. The knob is a no-op when expansion is off and folds into the query-cache key (evb=). Measured 2026-09-06 with recorded Haiku variants replayed at every budget: the mechanism is real — strict recall_all@5 rises from 255/470 at equal weighting to 394/470 at budget 0.25 — but its pre-registered rule (≥ plain hybrid − 2 on the 430-question decision set, no type losing > 1) failed at every budget (0.25: −43; plain hybrid 439/470), so every bundle sets expansion_variant_budget: null and the knob is an operator lever. The receipts point at a trigger rather than a weight (expand only when the original query's evidence is weak) as the next mechanism to pre-register.

The mode bundles carry an expansion default, false in all three (tokenmax turned it off after the 2026-10-14 expansion-latency eval: query p50 fell from 2,022 to 227 ms with retrieval unchanged). It applies when a caller leaves expansion unset and supplies an expandFn. The query operation leaves it unset unless the caller passes expand, so query does not expand by default; expand: true / --expand forces it and expand: false / --no-expand skips it. search never expands; search.expansion overrides the mode default for query only. Keyless and image-only paths skip expansion, and expansion_applied reports actual variant use. The search-mode guide is the authoritative rule for defaults, overrides and provider costs; gbrain search modes prints the operation-level exception before the retained bundle knobs.

Putting it together

The full pipeline for a trusted local query op follows. Remote retrieval omits the optional code-graph augmentation stage; the policy-filtered typed-edge relational recall arm remains available.

intent classify (query-intent.ts — deterministic, no LLM)
       │
       ▼
expansion (query: the mode's default, off in all three; --expand forces,
           --no-expand skips; search: off; keyless and image-only paths skip it)
       │
       ▼
hybrid recall + fusion:
   ├── vector  (HNSW on chunk embeddings, per-page max-pool)
   ├── keyword (BM25 via tsvector; text indexed through gbrain_fts_input)
   ├── title-phrase arm
   ├── relational (typed-edge recall arm — relational queries only; 2-3
   │      relationship questions walk typed hop chains when
   │      search.relational_planner is on)
   ├── source-aware re-rank (CASE in SQL)
   ├── role-tagged arms; variant/clause lists weighted by search.expansion_variant_budget INSIDE the fusion (fusion-lists.ts)
   └── page-grain RRF fusion → cosine re-score → post-fusion boosts
       (backlink / salience / recency / graph signals / exact-match,
        which also fires when a title, slug or alias is named in the query;
        the metadata boosts are skipped when the vector arm was the only
        voter — search.metadata_boost_gate=lexical, metadata-boost-gate.ts)
       │
       ▼
graph augment (optional two-pass structural expansion — walkDepth > 0)
       │
       ▼
deduplication (4-layer: per-page cap, same-page Jaccard, type diversity;
   with search.diverse_pool_depth N > 0, one row per session — off by default,
   src/core/search/diverse-pool.ts)
       │
       ▼
reranker (cross-encoder — balanced/tokenmax; fail-open; reads
   reranker.top_n_in rows, at least N with search.diverse_pool_depth)
       │
       ▼
feedback (use-attributed page weights × the ordering score, bounded ±λ;
   no-op when no page was rated — src/core/search/feedback-boost.ts)
       │
       ▼
relational re-pin (relational-arm rows back above the reranked text rows, in
   fused order, ≤ search.relational_rerank_pin; only when the reranker actually
   reordered — src/core/search/relational-rerank-pin.ts)
       │
       ▼
alias hop (exact alias match injects/boosts the canonical page; opt-in
   qualified sub-phrase match leads one page; like the
   exact-lookup tier below it never re-sorts, so the reranked order holds)
       │
       ▼
exact-lookup tier (lookup-shaped queries only: slug + exact-title probes
   promote/inject the identity page at rank-1; supersession-filtered;
   fail-open — src/core/search/exact-lookup.ts)
       │
       ▼
evidence stamp → adaptive return (opt-in) → autocut (reranked modes)
       │
       ▼
limit slice → token-budget enforcement (per mode bundle)
       │
       ▼
results (+ retrieval-confidence grade in query-op meta — crag.ts)

CJK/Latin boundaries in the keyword index (#6370). Postgres' default parser reads Han, Kana and Hangul as letters, so 升级PostgreSQL17后查询Redis7 would be one lexeme and an ASCII query for Redis7 would miss it. Every indexing to_tsvector (both trigger functions, the page seal, the import seal and reindex-search-vector) and the title arm's websearch_to_tsquery read their text through gbrain_fts_input() (src/core/fts-language.ts), which inserts a space at each CJK↔ASCII-letter/digit boundary. It is a no-op on text without CJK, so non-CJK vectors are byte-identical. Pure-CJK queries still take the raw-text CJK keyword path. The fts_cjk_boundary migration rebuilds only the rows whose text holds such a boundary.

Per-arm fail-open has one deliberate exception: when BOTH lexical arms (searchKeyword + searchTitles) come back empty because of an ACCESS-class failure (isDbAccessFailure in src/core/pg-access-classify.ts — the DB itself is unreachable, not a schema gap), hybridSearch rethrows the arm error instead of returning an empty result set. A dead database must surface as the classified database_error envelope, never as a silent "no results". Schema-class arm failures (pre-migration brains) fail open.

The stage order is pinned by hybridSearch in src/core/search/hybrid.ts: dedup runs BEFORE the reranker (so the reranker sees a diverse candidate pool, capped by its own topNIn; the reranker runs only on the full hybrid path — the no-embedding and keyword-fallback paths are never reranked), the alias hop runs AFTER the reranker (so a query that is a page's declared name reliably surfaces that page regardless of how the reranker scored body chunks), and the token budget is enforced last, on the final slice.

Two cross-cutting seams sit around the pipeline rather than inside it:

  • Private-page visibility. For untrusted (remote/MCP) callers, every recall arm filters visibility: private pages via the shared predicate in src/core/search/private-visibility.ts (fail-closed default; operator opt-outs documented in docs/operations/mcp-surface-runbook.md). The same policy also authorizes contributing pages, link origins, dates and annotations before enrichment. Semantic result caching is temporarily disabled regardless of configuration; each request performs fresh retrieval. Query embeddings can still be reused within that request. Repeated searches may have higher latency and provider usage until caching can verify every response dependency.
  • CRAG-style confidence gate. src/core/search/crag.ts grades every query op result (strong/moderate/weak) from the already-stamped honesty signals — zero LLM, zero added latency — and attaches the grade to response meta. An OR-relaxed keyword top (no chunk matched every query term) grades weak (keyword_relaxed_top) unless the top five rows corroborate it: at most one query content term appears in none of them, that term is not a name, and one row holds every other matched term. Then it grades moderate (keyword_relaxed_corroborated): the question used a framing word the corpus never writes, not an attribute or entity the evidence lacks. Config-gated and default OFF: search.crag_escalation=true re-runs a weak retrieval once at a higher ceiling (expansion + relational on, autocut off, limit = the caller's explicit limit or the mode-derived default — never a hardcoded row count — floored at 50) and keeps the better-graded run; the re-run fires only when the first pass did not already run with the caller's expansion on (shouldEscalateRetrieval in crag.ts), so default-shape callers never pay a second expansion call for a near-identical candidate set. search.crag_think=true (local callers) escalates a still-weak result to think.
  • Entity-anchored retrieval (default off). With search.entity_anchoring=true, a query or search that asks for a current state ("now", "currently", "latest" and similar) and names exactly one readable entity page by title gets that page first, then the pages that link to it or name it, newest first (src/core/search/entity-anchor.ts). The function is shared with pinned-question refresh. Detection is deterministic, with no model call. Added rows are re-authorized through getChunkWindows. The row count and token budget stay what the caller asked for, and anchored rows take at most half the rows. It is skipped with offset, type, date, language or symbol filters. Verdict and gates: docs/eval/decisions/entity-anchoring-query/.
  • Facts arm in query (default on). query adds up to three active saved facts that match the question (by the query embedding it already computed, by shared terms or by a named entity) as fact rows in spare capacity only: free slots under the caller's row count, and what the page rows leave of the token budget. A page row is never displaced. It also marks a page row superseded_claim when a newer fact covers its typed claim (src/core/search/facts-arm.ts). A fact row is never page-shaped: it carries result_type: "fact", fact_id and a follow_up recall call, no slug or id, and page_slug only when the caller can read the entity page. No model call. In the B2 corrections suite with correction-dated writes it lifted forget-then-remember from 92% to 100% correct. search.query_facts_arm=false turns it off. Verdict and gates: docs/eval/decisions/query-facts-arm/.
  • Saved facts beside search and query results. Both ops return, in retrieval.saved_facts and a text block after the rows, up to five active saved facts: first those about the entity page the query names, then facts that share the query's words (matchingSavedFacts in src/core/search/saved-facts-match.ts). Words match whole after stemming with the brain's full-text language, the facts keyword index's expression, so "renewal" finds "renews" and "count" does not match inside "countersigned"; words of a CJK script match as substrings. A fact needs three quarters of the query's words (both of a two-word query, and never more than four, so a question asked in full finds the short fact that answers it), not counting words that frame a question about a figure (count, number, total, amount); the most matched words come first, then the newest. This is the only place search returns saved facts: the facts arm, the facts block and the temporal reserve run in query alone. With search.saved_facts_supersede (default on; false turns it off), a newer fact with a correction cue ("correction", "ignore the ...", "not what I posted"; a reply that says "corrected" or "correcting" reports a correction and makes none) that shares at least two matched words with a returned fact is returned too, even below the threshold, every fact it corrects moves below the facts nothing corrects, and those carry corrected_by (the correcting fact's id, the newest when several correct it). Every correction comes back, so an acknowledgement saved after a correction does not crowd it out. An acknowledgement corrects nothing: a cue fact that adds no figure (a number other than a date such as "14 Oct" or a time) to the older fact, when it restates one of its figures or the older fact is itself a correction, or when it states no figure and withdraws nothing ("thanks for the correction"). A cue fact corrects nothing saved under another entity, nothing of higher trust, and nothing whose matched names (the first word of a capitalized name, "Orbix" in "Orbix Home Goods") it shares none of. Corrections rank newest first under the five-fact cap, and a corrected fact whose correction does not fit is left out. The cue is a fixed word list, so an unmarked correction is not caught; nothing stored changes.
  • Temporal fact reserve (default off). With search.temporal_fact_reserve=true, a query with a deterministic temporal cue (when, before, after, how long, since, until, ordinal-time words, ISO dates, month names) and a token budget (token_budget, or the one evidence delivery resolves) gives facts ranked by the question (cosine + term share, +0.1 for a real date) up to 15% of the budget and 30% of the rows. They render with their date header, oldest first after the pages, in place of the facts arm. No model call. Gates: docs/eval/decisions/temporal-fact-reserve/.
  • Facts block (default on) and fact source pages (default off), GBRA-205. Every query whose caller passes token_budget gets the reserve's dated facts, up to 22% of the budget, 40 rows and the caller's limit, oldest first ahead of the pages, in place of the facts arm and the temporal reserve. The no-budget evidence plan's implicit budget does not run it, so an unbudgeted query returns at most limit rows (#8411). Block rows take the top page's score: they are placed ahead, not ranked above the pages. On the sealed held-out set it answered 183 of 200 questions against 151 without it (gbrain-evals docs/benchmarks/2026-10-26-gbrain-8k-conversion.md). search.query_facts_block=false turns it off; a config read error also leaves it off. With search.fact_source_pages=true, the conversations the best-matching facts came from join the hits after the top three rows; it stays off (inert on development data).

Use-attributed feedback

Answers from query, search, think, synthesize and recall record the pages they used (with the revision read) and the typed edges on their relational paths. A rating (rate_answer, or a think citation for the brain owner) moves each element's weight w by w + α·(r − w); the feedback stage then multiplies the ordering score by 1 + λ·2·(w − 0.5). It runs after the reranker, on the reranker's score when it reordered the list, because a boost applied before the cross-encoder would be erased; raw scores stay untouched, so autocut and evidence grading are unchanged. No retrieval path runs a query-language string: the relational arm and traverse_graph take only typed parameters into fixed SQL, and test/raw-query-routing-guard.test.ts pins that no op accepts query text as SQL or graph syntax. Guide: retrieval feedback.

Relational re-pin: edge answers bypass reranker demotion

The cross-encoder scores chunk TEXT against the query. The relational arm's rows are typed-EDGE answers — "who invested in acme-co" resolves to investor pages whose text need not mention acme-co at all — so a reranker ranks them below any page that merely contains the query's words. Measured on NamedThingBench's relational fixture (39 graph-relationship questions, the shipped balanced default, voyage:rerank-2.5, scripts/r1-namedthing-rerank-ab.ts --relational, paired per query): with the reranker on and no pin, hit@1 fell from 21/39 to 3/39 (19 paired losses) and hit@3 from 27/39 to 5/39 (22 paired losses), while the 11 non-relational core questions showed 0 losses. With the pin at its default 3 (measured with --autocut on; autocut is off by default) the same paired comparison shows 0 hit@1 and 0 hit@3 losses (21/39 and 27/39, the reranker-off numbers) and the 11 core questions unchanged, which is why the balanced reranker is on. pinRelationalRows (src/core/search/relational-rerank-pin.ts) runs immediately after the reranker and re-pins the arm's rows above the reranked text rows in their fused order, bounded by search.relational_rerank_pin (3 in every bundle; 0/off disables the pin). It is a permutation of the pool (nothing added or removed; one row per page; a relational row the reranker itself ranked higher keeps that position; ties go to the fused order), fires only when the reranker actually reordered (fail-open and reranker-off runs are untouched — the fused order already carries the arm), and is a pure no-op for non-relational queries. Pinned rows are stamped relational_pinned so autocut keeps them and leaves them out of its cliff math (text rows are cut as usual). The evidence slot runs afterwards as the page-1 guarantee for pin 0 / fail-open runs. The pin trusts the arm: a false-positive arm puts as many edge pages at the top as the pin allows, where an unpinned run would place one at limit. Turn it off per brain with gbrain config set search.relational_rerank_pin off. The knob folds into the query-cache key (rrp=).

Breadth-first graph walks

Since v0.60.294.0 every graph walk (traverse_graph, gbrain graph, the graph context think gathers) is breadth-first. Each page is visited once, at its shortest distance from the start page, and each edge comes back once, one hop past its shallower walked endpoint. The old walk followed every possible route, so a densely linked cluster multiplied the work with every hop (a 12-page clique at depth 5 took 22 s before a 20,000-path cap bounded it) and returned the same page and edge again at every depth some route reached it.

What that changes for you:

  • Faster on the graphs that were slow. On Postgres, a depth-5 walk over a 12-page clique dropped from 44–91 ms to 4–9 ms, and walks from a 400-spoke hub got 1.8x to 4x faster (scripts/bench-graph-walk.ts, medians, against v0.60.292.0).
  • Complete answers. The old cap counted routes, so a busy neighbourhood could stop the walk before it reached deeper pages. The new cap counts visited pages (20,000, shallowest first), so a walk reaches every page within its depth unless the brain itself is that large.
  • Cleaner output. No repeated pages or edges at deeper depths, so an agent's context holds each relationship once and gbrain graph prints each edge once.
  • Same safety. Soft delete, private pages and private link origins, source scope, link type and temporal policy still apply at every hop: a page the caller may not read never relays a walk (test/traverse-bfs.test.ts).

The walk is one SQL statement on both engines: a recursive CTE whose rows are whole layers (bfsWalkCtes in src/core/engine-sql/links.ts). Walks that reach most of a large graph in both directions are still dominated by turning tens of thousands of edges into results before the 5,000-row cap, about half a second on a 10,000-page, 60,000-link graph; speeding that projection up is open work.

Multi-hop relationship chains

A question that chains two or three relationships ("who founded the companies Alice invested in?") is planned by parseRelationalPlan (src/core/search/relational-plan.ts): a bounded lexer, a fixed relationship vocabulary, exactly one named entity, hops ordered from that entity outward (the main-clause verb is outermost), page types checked along the chain. Coordination, negation, time constraints, counting, quoted names and multi-entity questions are refused with a reason; single-relationship questions stay on parseRelationalQuery. With search.relational_planner on, the relational arm resolves the entity, runs runRelationalChain (src/core/search/relational-chain.ts) and emits answers first, then the intermediate and origin pages of every retained path, each row carrying a relational evidence field (role, seed, hop, path count, up to three edges). Up to search.relational_chain_slots (default 10) chain rows lead page 1. A chain that finds nothing falls back to the one-hop path; a refused question runs no relational arm. meta.relational_plan and a relational_chain notice report the outcome.

Execution is one bounded query per hop (BrainEngine.relationalChainHop, SQL in src/core/search/read-enrichment.ts): at most 50 frontier pages, 100 logical edges per page (lowest link id first) and 10 retained paths per page. Links are read by each relationship's page-type signature (stored, flipped, uncertain at half weight; canonical frontmatter, manual and attendance-section links keep stored direction), duplicate rows between a pair collapse into one logical edge, and every endpoint, origin and degree contributor passes the read policy before scoring. Path weight is the product of hub weights of the intermediate pages (hubWeight, src/core/search/hub-dampening.ts; degree saturates at 300 link rows); ties order by source and slug. The same executor serves traverse_graph hops and gbrain graph-query --hop.

Chain evidence is page-level through fusion: whichever chunk of a page survives carries the page's relational field. Only answers count toward the relational re-pin and the page-1 evidence slot; intermediate and origin pages never claim them. The planner keys fold into the query-cache key only when on (rp=1, ro=1). Guide: multi-hop relationship questions.

Link types that take no part in ranking

Every reader that turns links into a ranking signal (backlink and adjacency boosts, relational fanout and chain hops, chain hub degree, enrich order, canonical entity picks, generated-page demotion, entity anchoring) skips the link types in NON_RANKING_LINK_TYPES (src/core/link-ranking-policy.ts): today amends. Those edges stay ordinary references in graph reads and cards. Guide: amendments.

Metadata boost gate: vector-only voters keep the vector order

The post-fusion metadata boosts (backlink, salience, recency + chronicle, graph signals, alias resolution) reward well-connected pages. That is right when a lexical arm agreed the page is about the query; it is wrong on paraphrase-style concept questions where nothing but the vector arm voted — there the boosts promote hub pages (1.03–1.12x) over the gold concept page, which carries none. decideMetadataBoosts (src/core/search/metadata-boost-gate.ts) runs before runPostFusionStages and, under search.metadata_boost_gate=lexical (every bundle), skips those boosts when no strict keyword, title-phrase or relational row fused (relaxed OR-fallback rows do not count). Supersede downrank, exact-match boost, title-phrase boost, compiled-truth boost, cosine re-score, dedup, reranker and autocut are untouched either way. Receipt (Cat 13 conceptual recall, gbrain-evals): held-out nDCG@5 53.0 → 57.8 (bare vector 60.5 remains the stretch), NamedThingBench, BrainBench, the retrieval canary and the LongMemEval dev slice byte-identical. always applies the boosts whatever voted; the decision is on HybridSearchMeta.metadata_boost_gate; the knob folds into the query-cache key (mbg=). The companion search.keyword_arm_confidence_floor (src/core/search/arm-confidence.ts, off in every bundle) down-weights a weak keyword arm in the fusion; its pre-registered receipt did not move the held-out score, so it is an operator knob only.

Autocut: score-discontinuity result-sizing

Off by default in every bundle. Pre-registered rule R2 measured it on LongMemEval from the default path's captured post-rerank pool: with the reranker on, the score cliff after the top session is the normal shape on multi-part questions, and the cut removed the second gold session — strict recall_all@5 449/470 → 379/470 (−68 paired on the 430-question decision set), with no floor in the sweep {0.10 … 0.80} within two questions of "off" on either seeded half (0.80 still lost 9, all knowledge-update). Any-hit stayed ≥ 99.4% throughout: autocut keeps the best session and drops the rest, which is a token saving (mean returned window 3256 → 1633 estimated tokens at 0.35) paid for with the questions that need more than one session. gbrain config set search.autocut true re-enables it with the knobs below; a session-aware cut (never below k distinct sessions) is a filed follow-up. applyAutocut (src/core/search/autocut.ts) cuts the ranked set at the largest cross-encoder rerank-score cliff, before the limit slice, first page only. Never-empty failsafe (minKeep), no-op when fewer than 2 results carry a finite rerank score (covers the fail-open reranker path), and alias-hop exact matches are preserved through the cut. Weak-top floor: when the top rerank score is below minTopScore (default 0.35, config search.autocut_min_top), cliff trimming is skipped entirely — a low-confidence list returns the full cluster for the caller to judge instead of collapsing to one result. Knobs: per-call SearchOpts.autocut → search.autocut / search.autocut_jump / search.autocut_min_top config → mode bundle. The pre-autocut pool can be captured for offline floor replay: hybridSearch exposes an eval-only onRerankPool hook that fires with the exact pool applyAutocut is about to cut (post-rerank, post alias-hop / exact-lookup, unscored injections included), gbrain eval longmemeval --capture-pool records it per row as rerank_pool, and scripts/replay-autocut-floor.ts replays every floor — including off — from that single capture, validating byte-for-byte against the live decisions before any other cell is read.

Each stage is testable in isolation. Each stage is replaceable. The whole pipeline is < 1ms of orchestration cost; the latency budget goes to the upstream HTTP calls (embedding, rerank) and the index scans.

Evidence delivery: whole evidence after ranking

Ranking decides which pages matter; the reader still needs enough of each page to answer. return_unit (chunk, window, section, page, auto; default auto) adds a stage after ranking, cache and capture, and before output redaction and snippet capping, shared by search, query, recall and think (via think.return_unit). It groups the ranked hits by page, reads every needed neighbor chunk in ONE batched, page_id-keyed query that re-authorizes each page under the caller's current scope, sanitizes each page's complete body before slicing, locates the hit chunks in it (so chunk overlap never duplicates text), cuts the requested unit around the hits, and packs blocks by rank into the token budget with a per-page floor so lower ranked sessions keep their matching span. The strict protected-body sanitizer runs on the whole body first, so delivered evidence never contains more than get_page returns to the same caller. The default auto expands only conversation pages (by page type or chat/ / conversations/ slug) to the whole page and leaves every other hit its ranked chunk; with no conversation hit, or with chunk, the stage does not run and responses are byte-identical. The measured motivation: on 400 held-out LongMemEval questions whole sessions answered 361 against 253 for top-5 chunks, while neighbor windows reached only 285–292, so auto gives conversations exactly the page unit. auto itself answered 445 of 500 LongMemEval-S questions against 312 for chunks (development data), and the preregistered sealed check was inconclusive at a ceiling (149 against 147 of 150); see docs/evidence-delivery.md for the numbers and the token cost. A lexical excerpt selector that failed its evaluation is recorded in docs/eval/ANSWER_PACKET_RESULTS.md. Contract, algorithms and latency: docs/evidence-delivery.md.

How to verify on your own brain

# Reproduce the public LongMemEval receipt (cleaned S split, k=5) like-for-like:
# --by-type prints strict recall_all@5 (headline) and recall_any@5 (diagnostic);
# the like-for-like row pins the reranker and autocut off.
gbrain eval longmemeval ~/datasets/longmemeval/longmemeval_s_cleaned.json \
  --retrieval-only --top-k 5 --by-type --no-trajectory --mode balanced --reranker off --autocut off
# The shipped default path (what balanced/tokenmax run): --reranker on --autocut off

# Capture your own queries and replay against retrieval changes
export GBRAIN_CONTRIBUTOR_MODE=1
# ... use gbrain normally ...
gbrain eval export > before.ndjson
# ... change something ...
gbrain eval replay --against before.ndjson

# A/B retrieval strategies on a labeled fixture
gbrain eval --qrels qrels.json --config-a baseline.json --config-b balanced.json

The current measured LongMemEval result (95.53% session-level recall_all@5 on the release default path, 449/470, and 93.40% with the reranker off, 439/470; cleaned S split, 470 scored questions, k=5, measured 2026-09-06 by the in-repo harness), its per-type table, every measured ranker arm and the judged answer-accuracy row live in docs/eval-bench.md.

Methodology + metric glossary in docs/eval/SEARCH_MODE_METHODOLOGY.md.

Restricted salience

Remote reads default to the world take holder when no grant is supplied; an explicit empty grant permits no takes. Counts and average weights use only permitted active takes. Stored emotional weight contributes zero because it combines all holders. Recent-salience inclusion uses updated_at with the same recency formula as unrestricted local reads, which use their full formulas. Deleted, quarantined and archived contributors are excluded before ranking, limits and anomaly-baseline aggregation. Default remote page privacy also excludes private pages; its documented opt-outs do not widen take-holder permissions.

Chunk rebuilds after upgrading

Say to your agent: "Check whether my search index is ready, and repair code metadata without spending on embeddings." Search and query report projection_readiness for empty and nonempty results. projection_pending means visible pages lack a current revision seal and are not still served from the index they had before migration 153's rebuild; projection_status_unknown means the diagnostic could not establish readiness. Neither is a clean miss. projection_by_design means the only unsealed pages in scope are protected-OCR images (or pages of an archived source), which stay out of the index by design: nothing is pending and no repair applies. The CLI names these states; --json retains its result-array format and sends incompleteness notices to stderr. MCP carries them in _meta.retrieval.

A resident gbrain serve drains queued Markdown and code rebuilds without provider calls. Code repair can also run through the current owner: gbrain reindex-code --force --no-embed. Rebuilds preserve only exact, provenance-compatible vectors: each vector records the embedding input it was built from (content_chunks.embedding_input_hash: column, model, dimensions, wrapping tier and wrapped text), and a rebuild keeps it only when the current page would produce the same input, so an unchanged contextual page keeps its vectors and a synopsis-mode body edit nulls every synopsis-tier chunk. Vectors written before that record existed are grandfathered on pages whose mode is still NULL; contextual-mode repair and reindex stamp compatible raw inputs when moving those pages to explicit none. An explicit none page may have inherited a title-wrapped vector, so a rebuild clears unstamped vectors there, and switching from title or synopsis to none clears active text vectors immediately. Genuine old raw vectors on explicit none pages may need a one-time re-embed; it never starts automatically. Remaining NULL vectors still need an explicitly authorized gbrain embed --stale run. A text-ready index is not a promise that every page has a vector. Diagnostics do not disclose private or foreign-source pending pages and never start repair themselves.

Markdown chunk creation applies the strict protected-body sanitizer before splitting text. For remote reads, all existing chunks of every page kind are withheld until a successful rebuild records the current chunker version. Public pages require this rebuild too; trusted local chunk reads remain available. Body or chunk changes invalidate that record until the next successful rebuild. Re-importing unchanged content (gbrain sync --full, gbrain import) re-seals such a page without a page write or journal admission. While pages are withheld, remote search / query report degraded: [safe_index_pending] on empty and partial results alike (the MCP empty-result block names it) instead of a clean or complete answer, and gbrain doctor counts the withheld markdown and code pages and points at gbrain repair safe-chunks, which re-seals them on any brain, managed included. Direct page reads continue to use current source and visibility policy plus body sanitization.

Run rebuild commands from a local installation on the brain host; thin clients cannot rebuild the host's indexes. gbrain reindex --markdown --dry-run --no-embed previews the existing rebuild. gbrain reindex --markdown --no-embed rebuilds without embedding calls, replacing previous vectors; use gbrain embed --stale later to restore semantic retrieval when provider usage is authorized. The regular reindex path rebuilds and embeds. Code pages use gbrain reindex-code --force --no-embed. Existing image indexes require reimporting the source files; images whose OCR contains protected sections remain unavailable to remote chunk retrieval. Schema migrations 160 and 161 install projection statistics and the pending-projection lookup index; they do not rebuild vector indexes or call an embedding provider.

query --since includes its exact lower boundary. An exact --until timestamp is inclusive without losing fractional precision; a date-only upper bound includes the entire day, implemented as an exclusive next-midnight boundary. Legacy direct engine afterDate and beforeDate bounds remain strict unless their explicit inclusivity flags are set.

Optional code-graph expansion is omitted from remote search. The six dedicated code-inspection operations are also temporarily local-only; see the MCP surface runbook. These restrictions are separate from the chunk rebuild requirement.