Garry's Opinionated OpenClaw/Hermes Agent Brain
Why the hybrid + graph stack works
Vector search alone underdelivers on real personal-knowledge queries. This doc explains why gbrain layers four strategies together and how they compound.
The four strategies in concert
- Vector (HNSW on pgvector) — semantic similarity. Catches "who works on retrieval quality at acme-example?" → pages mentioning "alice-example + retrieval" even when the user never typed "acme".
- BM25 keyword — lexical match. Catches names, exact phrases, code identifiers, anything where the user remembers the literal token. Survives the cases where vector search drifts into thematic neighbors.
- Reciprocal-rank fusion (RRF) — merges vector + keyword rankings without weighting one over the other globally. Each strategy gets to vote, and the vote is for a PAGE: arms return each page's best chunk, often different chunks, so fusion sums a page's votes across arms onto the page's lead chunk (largest own vote, ties to the vector arm's chunk); the page's other chunks keep their own votes so they cannot crowd other pages out of a chunk-limited result.
- Knowledge graph traversal — follows recorded typed edges. It can answer relationship queries whose relevant endpoints are not close in embedding space; it depends on the edges being present and correct, and does not by itself establish causality.
Why each one alone fails
Vector only. Returns chunks semantically close to the query. Misses any factual relationship not directly encoded in the embedding. "Companies in alice-example's portfolio" returns essays about portfolios, not company pages.
Keyword only (ripgrep-style). Brittle to phrasing. "Who works on retrieval?" misses pages that say "search ranking" instead of "retrieval." Garbage on synonyms, near-misses, or paraphrases.
Graph only. Excellent at "neighbors of Alice" but blind to anything not yet linked. Sparse on fresh pages until backlinks accumulate.
Hybrid (vector + keyword + RRF), no graph. Can retrieve relationships stated in text, but may miss answers that require joining several records. Typed graph traversal offers a separate recall path for those cases; this is not an exclusivity claim about GBrain or a guarantee that every graph answer is correct.
The benchmark
BrainBench (corpus + harness in the sibling gbrain-evals repo) measures retrieval P@5, R@5, MRR, nDCG@5 on a 240-page Opus-generated rich-prose corpus. (This is the retrieval-ranking benchmark; the in-repo gbrain eval brainbench suite — docs/eval/BRAINBENCH.md — gates the memory behaviors above retrieval: unprompted context push, write-back fidelity, cross-session continuity.)
The September 9, 2026 refresh (gbrain v0.48.4.0, three ingestion orders, fixed-denominator P@5) measured the 145 relationship questions two ways.
The production pipeline. One switch, search.relational_retrieval, over a
shared index with everything else held fixed: mean R@5 rose from 0.6626 to
0.7241, better on 45 of 435 question runs and worse on none; investor
questions moved most, attendance not at all
(production relationship retrieval).
On reworded one-hop questions the same switch measured 0.411 to 0.537
(development data, not a held-out set; N9).
Whole-system adapters, including an oracle upper bound:
| Adapter | Mean P@5 | Mean R@5 |
|---|---|---|
graph-oracle-parse (knows the four question templates in advance, follows the fixture graph; upper bound, not a product score) |
0.3421 | 0.9791 |
| Reference hybrid | 0.1917 | 0.6874 |
| Keyword ranker | 0.1710 | 0.6244 |
| Vector only | 0.1076 | 0.4069 |
graph-oracle-parse was called gbrain in these tables until September 28,
2026; the gbrain-evals gbrain adapter now runs the product's hybrid search
with relationship retrieval. The oracle is strong on the templates it parses,
but this is a comparison between whole systems. The refresh found that the older headline
(P@5 49.1%, "+31.4 points over graph-disabled") was not a graph-only effect and
used metric helpers since corrected; do not cite it. On the fuzzy and externally
authored families the keyword and vector paths win some comparisons. None of
this predicts accuracy on a different corpus.
Auto-link: why zero-LLM-call edge extraction works
Trusted local put_page runs inline link extraction on the markdown body.
Remote put_page deliberately skips it at the trust boundary. Stdio serving
sweeps eligible pages at startup and idle; HTTP serving does not self-sweep.
For HTTP-written pages, a trusted operator can run
gbrain extract links --source db (select the intended source with
--source-id <id>), or a permitted client can create explicit links through
add_link. Extraction recognizes:
- Standard markdown links:
[Alice Example](wiki/people/alice-example) - Obsidian wikilinks:
[[wiki/people/alice-example|Alice Example]] - Typed-link blockquotes:
> **Convention:** see [path](path).
The link parser uses no LLM tokens and batches SQL insertion through
addLinksBatch. Graph freshness depends on the write path and when extraction
runs; accepting a remote write is not evidence its links have been extracted.
Heuristic link-type inference (attended, works_at, invested_in, founded, advises) fires from surrounding sentence context — also LLM-free. Power users who want richer types add them via the typed-link blockquote convention.
Cross-encoder reranker: 60% top-1 reshuffle
The reranker is on for the balanced and tokenmax mode bundles, off for conservative. The mode-bundle default is Voyage rerank-3-lite (DEFAULT_RERANKER_MODEL, same VOYAGE_API_KEY as the embedding default); a brain with no search.reranker.model row reranks with it, and a brain with one keeps its own model. It replaced rerank-2.5 after gbrain-evals #148 (v0.10.93) measured 60% lower rerank cost at 50 candidates × 1,400 tokens with quality within noise; gbrain config set search.reranker.model voyage:rerank-2.5 switches a brain back. Without the key, search fails open in RRF order: the gateway skips the HTTP call (RerankError('no_key')), writes ONE audit row per process, prints nothing, and stamps reranker_skipped (no_key) on the search meta — gbrain search --explain shows it, gbrain search modes prints a Reranker: readiness line, and gbrain doctor's reranker_health names the fix (export VOYAGE_API_KEY=… or gbrain config set search.reranker.enabled false). Keyed installs without a Voyage key get reranking explicitly disabled at init. The retained query-cache key includes reranker_model; persisted result reuse is disabled until every response dependency can be verified. On a real-corpus benchmark across 20 queries, the cross-encoder reshuffled 60% of top-1 results after the hybrid + RRF + graph stack (historical measurement on the retired hosted reranker, not Voyage; original identifiers are preserved at Git revision 6040075c6cb95be5881cc2e1b76ef7d71f4e5d29, retained on 2026-09-23). The Voyage default's paired LongMemEval numbers live in docs/eval-bench.md.
The mechanical reason: hybrid ranking is locally optimal per strategy but globally suboptimal. A cross-encoder reranker reads the query + each candidate document jointly, with full attention. It catches the cases where the vector + keyword + graph signals all agreed on a document that's semantically related but topically wrong.
The cost: +150ms p50 latency, ~$0.025–0.05/M tokens depending on the reranker. Disabled with gbrain config set search.reranker.enabled false. For agent loops that do downstream LLM work after retrieval, the latency is invisible.
Source-aware ranking
Hybrid search applies a source-factor CASE expression at the SQL layer (lives in src/core/search/sql-ranking.ts). Curated content like originals/, concepts/, writing/ outranks bulk content like your-openclaw/chat/, daily/, media/x/. Hard-exclude prefixes (test/, attachments/, .raw/) filter at retrieval, not post-rank.
archive/ is deliberately NOT hard-excluded: it holds high-signal historical content users expect to find, so it is demoted (0.5x in DEFAULT_SOURCE_BOOSTS), not hidden. The demote is a prior applied in the outer SQL re-rank; the cross-encoder reranker (balanced/tokenmax modes) can still PROMOTE an archive page that survives the demote into the rerank candidate window — it is not an unconditional suppression. gbrain doctor's hidden_by_search_policy check reports how many chunked pages remain hidden by the surviving exclude prefixes.
The defaults fit one vault layout, so each brain can carry its own map: gbrain config set search.source_boosts "wiki/:1.3,daily/:1.0" overrides or extends the defaults per prefix, and a none entry ("none,wiki/:1.3") drops them entirely. The GBRAIN_SOURCE_BOOST env var (same format) wins over the brain config for one process. Hard exclusions are separate: the exclusion set is defaults ∪ GBRAIN_SEARCH_EXCLUDE (env, comma-separated prefixes) ∪ per-call SearchOpts.exclude_slug_prefixes. Temporal queries (detail: 'high') bypass the boost so chat pages re-surface for time-sensitive lookups.
Named-thing retrieval (per-page pool + title + alias + evidence)
A brain organized around chosen names (project codenames, place nicknames —
say a project named "Helios" whose page is also known as "the Sun Room") needs
more than embedding proximity. Four layers (the failure mode they close is
written up in RETRIEVAL_MAXPOOL_INCIDENT.md):
- Per-page max-pool —
searchVector(both engines) collapses chunk-grain candidates to the best chunk per page (DISTINCT ON (slug)) over the full candidate set before the userLIMIT, via the sharedbuildBestPerPagePoolCteinsql-ranking.ts. The vector side returns N distinct pages by best chunk, not N chunks that collapse to fewer pages downstream. When one dense page's chunks fill the inner candidate pool, the engines escalate the pool in a bounded loop (×4 per step, at most 3 escalations). SQL candidate limits and offsets are independent ofef_search; supported pgvector versions userelaxed_orderiterative scans with bounded visits (ef_searchis still sized to the candidate pool; each pooled attempt may visit 20,000 index tuples, so a source or visibility filter covering a tenth of the brain fills its candidate window instead of stopping at the first few hundred eligible chunks). Relaxed order keeps a closer candidate the scan finds after farther ones, which strict order drops, so a selective source or visibility filter fills more of its true neighbours; output order never reaches callers because the innerORDER BY distanceand the pooled outer sort re-sort it.search.hnsw_iterative_scan(relaxed_order|strict_order|off; envGBRAIN_HNSW_ITERATIVE_SCANwins) restores the older mode without a release; it is latched once per engine (once per process on a one-brain server), so restart serve and autopilot after changing it. A filtered short pool is not proof that the corpus is exhausted. Postgres may make one exact fallback inside the remaining eight-second arm budget; PGLite does not pretend that a JavaScript timeout can cancel its WASM work. Unresolved shortfalls appear asvector_candidates_incompleteindegraded, with scopedvector_pool_underfilleddetails in the public retrieval metadata. The content-freshness check runs after the HNSW candidate scan on indexed columns (the planner cannot estimate it and would drop the index); doctorvector_planwarns when the emitted statement still misses the index. - Title-phrase boost — when the normalized query is a contiguous token-run
inside
page.title(or an exact full-title match), a floor-ratio-gated, bounded multiplier fires (applyTitleBoost,search.title_boostknob). A query that is a phrase from the title can't lose to a body chunk by luck. Opt-insearch.title_qualified(off by default) covers the reverse case: a person or company title (2+ content words) mentioned inside a longer query ("acme alice example") gets the same multiplier, unfloored, stampedtitle_mention_boost, unless an identity boost already raised the row. Evidence never reads that stamp as an exact title match. - Alias hop — free-text
aliases:frontmatter is projected into apage_aliasestable (separate from theslug_aliaseswikilink redirect) and consulted at query time: a full normalized-query match injects/boosts the canonical page (applyAliasHop). The only layer that bridges true synonyms with zero surface overlap ("the Sun Room" → the Helios page). Backfill existing pages withgbrain reindex --aliases. Opt-insearch.qualified_aliases(off by default) also matches a 3-6 word sub-phrase with 3+ content words inside a query of up to 14 words: when exactly one page the caller can read claims it, that page leads, stampedqualified_alias_hit; two claimants suppress the hop. The stamp keeps the row past autocut and the remote page cap but never sets evidencealias_hit, and the rerank gate does not grade it. A common three-word alias can lead a query about something else, so confirm on your own eval before turning it on:gbrain config set search.qualified_aliases true(falseturns it back off). - Evidence contract — every result carries
evidence(alias_hit | exact_title_match | high_vector_match | keyword_exact | weak_semantic) andcreate_safety(exists | probable | unknown). An agent deciding "is this page already here, safe to NOT write a duplicate?" keys offcreate_safety, not a raw blended score.high_vector_matchis grounded in the result's real query↔chunk cosine (SearchResult.cosineat/abovesearch.evidence_cosine_floor, default 0.80) — never the blended score, so a keyword+boost pile-up can't read as semantic support; keyless runs have no cosine and degrade to honest keyword-based labels.gbrain search --explainprints each result's raw cosine next to its blended score.
Extraction quarantine lane: pages carrying the unverified
auto-extracted markers (frontmatter provenance: auto-extracted +
status: unverified, see src/core/extraction-review.ts) rank as ordinary
content — they are skipped by the compiled-truth fusion boost and by the
people//companies/ namespace source-boost, and every search result from
such a page carries unverified: true so agents can label the provenance.
Promote or reject them via gbrain extraction-pending / gbrain extraction-review.
The search MCP/CLI op is cheap-hybrid (vector + keyword + RRF + pool +
title + alias, expansion off); query is the full-control variant. Route
concept / landscape / "all-of-X" questions to query — expansion recovers
synonym-phrased matches search can miss, and a populated search result set
is not proof of coverage (both are top-K; exhaustive enumeration belongs to
list_pages). NamedThingBench
(gbrain eval retrieval-quality) gates these families on every PR. Diagnose a
specific miss with gbrain search diagnose "<q>" --target <slug>.
Intent-aware query rewriting
src/core/search/query-intent.ts classifies queries into entity, temporal, event, concept, or general. Each routes through different ranking knobs:
- Entity queries ("who works at X?") apply a higher graph-traversal weight.
- Temporal queries ("what happened last week?") bypass source-boost so chat/daily pages surface.
- Event queries ("Acme AI Series A") engage the timeline index.
- Concept queries ("what is the ownership economy?", "find all the companies doing offshore wind" — definitional paraphrases and landscape/quantifier phrasings with no proper noun) rank vector-lean, so keyword-decoy pages don't outrank the page that actually explains the idea. Proper nouns, quoted phrases, and sub-3-word queries never classify as concept.
- General queries hit the standard hybrid stack.
The classifier is deterministic (no LLM call). Wrong classification degrades gracefully — the hybrid stack still works without it.
Multi-query expansion
When requested and available, src/core/search/expansion.ts asks the configured
expansion model for up to two alternatives in addition to the original query.
This is independent of detail; queries shorter than three words also skip
the provider call. Each variant's vector list enters RRF fusion alongside the
original's. Expansion can reduce recall: on LongMemEval-S (470 scored questions,
k=5, the 2026-09-02 receipt in docs/eval-bench.md) plain hybrid scores 93.19%
strict recall_all@5 while hybrid plus equal-weight expansion scores 54.89%
(paired +3 / -183 questions). Variant lists with the original's weight can
outvote it on small-k retrieval.
Budget-normalized weighted RRF, composed in src/core/search/fusion-lists.ts. Every vector list is a role-tagged arm (original | variant | clause | image) — tagged objects, never a positional convention, so a failed arm or a fell-open image branch can't mis-tag a list. The original arm always fuses at weight 1; the non-empty variant/clause arms share ONE total weight budget, search.expansion_variant_budget (weight_i = b / n_voting_arms, each row scored weight / (k + rank)), so total expansion influence is exactly b however many variants the LLM produced. null, the default in all three mode bundles, is equal-weight fusion (every list weight 1). A budget in (0, 4] is set with gbrain config set search.expansion_variant_budget <b>, per call via HybridSearchOpts.expansionVariantBudget, or pinned per eval arm with gbrain eval longmemeval --expansion-variant-budget <b> (sweep it against frozen --expansion-replay variants so cells differ only in b). Arithmetic: two variants agreeing on a distractor at rank 0 tie the original's rank-0 vote exactly at b = 1.0; equal weighting with two variants is ≈ b = 2.0; b = 0.5 subordinates them. The knob is a no-op when expansion is off and folds into the query-cache key (evb=). Measured 2026-09-06 with recorded Haiku variants replayed at every budget: the mechanism is real — strict recall_all@5 rises from 255/470 at equal weighting to 394/470 at budget 0.25 — but its pre-registered rule (≥ plain hybrid − 2 on the 430-question decision set, no type losing > 1) failed at every budget (0.25: −43; plain hybrid 439/470), so every bundle sets expansion_variant_budget: null and the knob is an operator lever. The receipts point at a trigger rather than a weight (expand only when the original query's evidence is weak) as the next mechanism to pre-register.
The mode bundles carry an expansion default, false in all three (tokenmax
turned it off after the 2026-10-14 expansion-latency eval: query p50 fell
from 2,022 to 227 ms with retrieval unchanged). It applies when a caller leaves
expansion unset and supplies an expandFn. The query operation leaves it
unset unless the caller passes expand, so query does not expand by default;
expand: true / --expand forces it and expand: false / --no-expand
skips it. search never expands; search.expansion overrides the mode default for query only. Keyless and image-only paths
skip expansion, and expansion_applied reports actual variant use. The
search-mode guide is the authoritative rule for
defaults, overrides and provider costs; gbrain search modes prints the
operation-level exception before the retained bundle knobs.
Putting it together
The full pipeline for a trusted local query op follows. Remote retrieval
omits the optional code-graph augmentation stage; the policy-filtered typed-edge
relational recall arm remains available.
intent classify (query-intent.ts — deterministic, no LLM)
│
▼
expansion (query: the mode's default, off in all three; --expand forces,
--no-expand skips; search: off; keyless and image-only paths skip it)
│
▼
hybrid recall + fusion:
├── vector (HNSW on chunk embeddings, per-page max-pool)
├── keyword (BM25 via tsvector; text indexed through gbrain_fts_input)
├── title-phrase arm
├── relational (typed-edge recall arm — relational queries only; 2-3
│ relationship questions walk typed hop chains when
│ search.relational_planner is on)
├── source-aware re-rank (CASE in SQL)
├── role-tagged arms; variant/clause lists weighted by search.expansion_variant_budget INSIDE the fusion (fusion-lists.ts)
└── page-grain RRF fusion → cosine re-score → post-fusion boosts
(backlink / salience / recency / graph signals / exact-match,
which also fires when a title, slug or alias is named in the query;
the metadata boosts are skipped when the vector arm was the only
voter — search.metadata_boost_gate=lexical, metadata-boost-gate.ts)
│
▼
graph augment (optional two-pass structural expansion — walkDepth > 0)
│
▼
deduplication (4-layer: per-page cap, same-page Jaccard, type diversity;
with search.diverse_pool_depth N > 0, one row per session — off by default,
src/core/search/diverse-pool.ts)
│
▼
reranker (cross-encoder — balanced/tokenmax; fail-open; reads
reranker.top_n_in rows, at least N with search.diverse_pool_depth)
│
▼
feedback (use-attributed page weights × the ordering score, bounded ±λ;
no-op when no page was rated — src/core/search/feedback-boost.ts)
│
▼
relational re-pin (relational-arm rows back above the reranked text rows, in
fused order, ≤ search.relational_rerank_pin; only when the reranker actually
reordered — src/core/search/relational-rerank-pin.ts)
│
▼
alias hop (exact alias match injects/boosts the canonical page; opt-in
qualified sub-phrase match leads one page; like the
exact-lookup tier below it never re-sorts, so the reranked order holds)
│
▼
exact-lookup tier (lookup-shaped queries only: slug + exact-title probes
promote/inject the identity page at rank-1; supersession-filtered;
fail-open — src/core/search/exact-lookup.ts)
│
▼
evidence stamp → adaptive return (opt-in) → autocut (reranked modes)
│
▼
limit slice → token-budget enforcement (per mode bundle)
│
▼
results (+ retrieval-confidence grade in query-op meta — crag.ts)
CJK/Latin boundaries in the keyword index (#6370). Postgres' default
parser reads Han, Kana and Hangul as letters, so 升级PostgreSQL17后查询Redis7
would be one lexeme and an ASCII query for Redis7 would miss it. Every
indexing to_tsvector (both trigger functions, the page seal, the import seal
and reindex-search-vector) and the title arm's websearch_to_tsquery read
their text through gbrain_fts_input() (src/core/fts-language.ts), which
inserts a space at each CJK↔ASCII-letter/digit boundary. It is a no-op on text
without CJK, so non-CJK vectors are byte-identical. Pure-CJK queries still take
the raw-text CJK keyword path. The fts_cjk_boundary migration rebuilds only
the rows whose text holds such a boundary.
Per-arm fail-open has one deliberate exception: when BOTH lexical arms
(searchKeyword + searchTitles) come back empty because of an ACCESS-class
failure (isDbAccessFailure in src/core/pg-access-classify.ts — the DB
itself is unreachable, not a schema gap), hybridSearch rethrows the arm
error instead of returning an empty result set. A dead database must surface
as the classified database_error envelope, never as a silent "no results".
Schema-class arm failures (pre-migration brains) fail open.
The stage order is pinned by hybridSearch in src/core/search/hybrid.ts:
dedup runs BEFORE the reranker (so the reranker sees a diverse candidate pool,
capped by its own topNIn; the reranker runs only on the full hybrid path — the
no-embedding and keyword-fallback paths are never reranked), the alias hop runs AFTER the reranker (so a query
that is a page's declared name reliably surfaces that page regardless of how
the reranker scored body chunks), and the token budget is enforced last, on
the final slice.
Two cross-cutting seams sit around the pipeline rather than inside it:
- Private-page visibility. For untrusted (remote/MCP) callers, every
recall arm filters
visibility: privatepages via the shared predicate insrc/core/search/private-visibility.ts(fail-closed default; operator opt-outs documented indocs/operations/mcp-surface-runbook.md). The same policy also authorizes contributing pages, link origins, dates and annotations before enrichment. Semantic result caching is temporarily disabled regardless of configuration; each request performs fresh retrieval. Query embeddings can still be reused within that request. Repeated searches may have higher latency and provider usage until caching can verify every response dependency. - CRAG-style confidence gate.
src/core/search/crag.tsgrades everyqueryop result (strong/moderate/weak) from the already-stamped honesty signals — zero LLM, zero added latency — and attaches the grade to response meta. An OR-relaxed keyword top (no chunk matched every query term) gradesweak(keyword_relaxed_top) unless the top five rows corroborate it: at most one query content term appears in none of them, that term is not a name, and one row holds every other matched term. Then it gradesmoderate(keyword_relaxed_corroborated): the question used a framing word the corpus never writes, not an attribute or entity the evidence lacks. Config-gated and default OFF:search.crag_escalation=truere-runs a weak retrieval once at a higher ceiling (expansion + relational on, autocut off, limit = the caller's explicitlimitor the mode-derived default — never a hardcoded row count — floored at 50) and keeps the better-graded run; the re-run fires only when the first pass did not already run with the caller's expansion on (shouldEscalateRetrievalincrag.ts), so default-shape callers never pay a second expansion call for a near-identical candidate set.search.crag_think=true(local callers) escalates a still-weak result tothink. - Entity-anchored retrieval (default off). With
search.entity_anchoring=true, aqueryorsearchthat asks for a current state ("now", "currently", "latest" and similar) and names exactly one readable entity page by title gets that page first, then the pages that link to it or name it, newest first (src/core/search/entity-anchor.ts). The function is shared with pinned-question refresh. Detection is deterministic, with no model call. Added rows are re-authorized throughgetChunkWindows. The row count and token budget stay what the caller asked for, and anchored rows take at most half the rows. It is skipped withoffset, type, date, language or symbol filters. Verdict and gates:docs/eval/decisions/entity-anchoring-query/. - Facts arm in
query(default on).queryadds up to three active saved facts that match the question (by the query embedding it already computed, by shared terms or by a named entity) as fact rows in spare capacity only: free slots under the caller's row count, and what the page rows leave of the token budget. A page row is never displaced. It also marks a page rowsuperseded_claimwhen a newer fact covers its typed claim (src/core/search/facts-arm.ts). A fact row is never page-shaped: it carriesresult_type: "fact",fact_idand afollow_uprecall call, noslugorid, andpage_slugonly when the caller can read the entity page. No model call. In the B2 corrections suite with correction-dated writes it lifted forget-then-remember from 92% to 100% correct.search.query_facts_arm=falseturns it off. Verdict and gates:docs/eval/decisions/query-facts-arm/. - Saved facts beside
searchandqueryresults. Both ops return, inretrieval.saved_factsand a text block after the rows, up to five active saved facts: first those about the entity page the query names, then facts that share the query's words (matchingSavedFactsinsrc/core/search/saved-facts-match.ts). Words match whole after stemming with the brain's full-text language, the facts keyword index's expression, so "renewal" finds "renews" and "count" does not match inside "countersigned"; words of a CJK script match as substrings. A fact needs three quarters of the query's words (both of a two-word query, and never more than four, so a question asked in full finds the short fact that answers it), not counting words that frame a question about a figure (count, number, total, amount); the most matched words come first, then the newest. This is the only placesearchreturns saved facts: the facts arm, the facts block and the temporal reserve run inqueryalone. Withsearch.saved_facts_supersede(default on;falseturns it off), a newer fact with a correction cue ("correction", "ignore the ...", "not what I posted"; a reply that says "corrected" or "correcting" reports a correction and makes none) that shares at least two matched words with a returned fact is returned too, even below the threshold, every fact it corrects moves below the facts nothing corrects, and those carrycorrected_by(the correcting fact's id, the newest when several correct it). Every correction comes back, so an acknowledgement saved after a correction does not crowd it out. An acknowledgement corrects nothing: a cue fact that adds no figure (a number other than a date such as "14 Oct" or a time) to the older fact, when it restates one of its figures or the older fact is itself a correction, or when it states no figure and withdraws nothing ("thanks for the correction"). A cue fact corrects nothing saved under another entity, nothing of higher trust, and nothing whose matched names (the first word of a capitalized name, "Orbix" in "Orbix Home Goods") it shares none of. Corrections rank newest first under the five-fact cap, and a corrected fact whose correction does not fit is left out. The cue is a fixed word list, so an unmarked correction is not caught; nothing stored changes. - Temporal fact reserve (default off). With
search.temporal_fact_reserve=true, aquerywith a deterministic temporal cue (when, before, after, how long, since, until, ordinal-time words, ISO dates, month names) and a token budget (token_budget, or the one evidence delivery resolves) gives facts ranked by the question (cosine + term share, +0.1 for a real date) up to 15% of the budget and 30% of the rows. They render with their date header, oldest first after the pages, in place of the facts arm. No model call. Gates:docs/eval/decisions/temporal-fact-reserve/. - Facts block (default on) and fact source pages (default off), GBRA-205.
Every
querywhose caller passestoken_budgetgets the reserve's dated facts, up to 22% of the budget, 40 rows and the caller'slimit, oldest first ahead of the pages, in place of the facts arm and the temporal reserve. The no-budget evidence plan's implicit budget does not run it, so an unbudgetedqueryreturns at mostlimitrows (#8411). Block rows take the top page's score: they are placed ahead, not ranked above the pages. On the sealed held-out set it answered 183 of 200 questions against 151 without it (gbrain-evalsdocs/benchmarks/2026-10-26-gbrain-8k-conversion.md).search.query_facts_block=falseturns it off; a config read error also leaves it off. Withsearch.fact_source_pages=true, the conversations the best-matching facts came from join the hits after the top three rows; it stays off (inert on development data).
Use-attributed feedback
Answers from query, search, think, synthesize and recall record the
pages they used (with the revision read) and the typed edges on their
relational paths. A rating (rate_answer, or a think citation for the brain
owner) moves each element's weight w by w + α·(r − w); the feedback stage
then multiplies the ordering score by 1 + λ·2·(w − 0.5). It runs after the
reranker, on the reranker's score when it reordered the list, because a boost
applied before the cross-encoder would be erased; raw scores stay untouched,
so autocut and evidence grading are unchanged. No retrieval path runs a
query-language string: the relational arm
and traverse_graph take only typed parameters into fixed SQL, and
test/raw-query-routing-guard.test.ts pins that no op accepts query text as
SQL or graph syntax. Guide: retrieval feedback.
Relational re-pin: edge answers bypass reranker demotion
The cross-encoder scores chunk TEXT against the query. The relational arm's
rows are typed-EDGE answers — "who invested in acme-co" resolves to investor
pages whose text need not mention acme-co at all — so a reranker ranks them
below any page that merely contains the query's words. Measured on
NamedThingBench's relational fixture (39 graph-relationship questions, the
shipped balanced default, voyage:rerank-2.5,
scripts/r1-namedthing-rerank-ab.ts --relational, paired per query): with
the reranker on and no pin, hit@1 fell from 21/39 to 3/39 (19 paired losses)
and hit@3 from 27/39 to 5/39 (22 paired losses), while the 11 non-relational
core questions showed 0 losses. With the pin at its default 3 (measured with
--autocut on; autocut is off by default) the same paired comparison shows
0 hit@1 and 0 hit@3 losses (21/39 and 27/39, the reranker-off numbers) and the
11 core questions unchanged, which is why the balanced reranker is on.
pinRelationalRows
(src/core/search/relational-rerank-pin.ts) runs immediately after the
reranker and re-pins the arm's rows above the reranked text rows in their fused
order, bounded by search.relational_rerank_pin (3 in every bundle; 0/off
disables the pin). It is a permutation of the pool (nothing added
or removed; one row per page; a relational row the reranker itself ranked
higher keeps that position; ties go to the fused order), fires only when the
reranker actually reordered (fail-open and reranker-off runs are untouched —
the fused order already carries the arm), and is a pure no-op for
non-relational queries. Pinned rows are stamped relational_pinned so autocut
keeps them and leaves them out of its cliff math (text rows are cut as
usual). The evidence slot runs afterwards as the page-1 guarantee for pin 0 /
fail-open runs. The pin trusts the arm: a false-positive arm puts as many edge
pages at the top as the pin allows, where an unpinned run would place one at
limit. Turn it off per brain with gbrain config set search.relational_rerank_pin off.
The knob folds into the query-cache key (rrp=).
Breadth-first graph walks
Since v0.60.294.0 every graph walk (traverse_graph, gbrain graph, the graph
context think gathers) is breadth-first. Each page is visited once, at its
shortest distance from the start page, and each edge comes back once, one hop
past its shallower walked endpoint. The old walk followed every possible route,
so a densely linked cluster multiplied the work with every hop (a 12-page
clique at depth 5 took 22 s before a 20,000-path cap bounded it) and returned
the same page and edge again at every depth some route reached it.
What that changes for you:
- Faster on the graphs that were slow. On Postgres, a depth-5 walk over a
12-page clique dropped from 44–91 ms to 4–9 ms, and walks from a 400-spoke hub
got 1.8x to 4x faster (
scripts/bench-graph-walk.ts, medians, against v0.60.292.0). - Complete answers. The old cap counted routes, so a busy neighbourhood could stop the walk before it reached deeper pages. The new cap counts visited pages (20,000, shallowest first), so a walk reaches every page within its depth unless the brain itself is that large.
- Cleaner output. No repeated pages or edges at deeper depths, so an
agent's context holds each relationship once and
gbrain graphprints each edge once. - Same safety. Soft delete, private pages and private link origins, source
scope, link type and temporal policy still apply at every hop: a page the
caller may not read never relays a walk (
test/traverse-bfs.test.ts).
The walk is one SQL statement on both engines: a recursive CTE whose rows are
whole layers (bfsWalkCtes in src/core/engine-sql/links.ts). Walks that reach
most of a large graph in both directions are still dominated by turning tens
of thousands of edges into results before the 5,000-row cap, about half a
second on a 10,000-page, 60,000-link graph; speeding that projection up is
open work.
Multi-hop relationship chains
A question that chains two or three relationships ("who founded the companies
Alice invested in?") is planned by parseRelationalPlan
(src/core/search/relational-plan.ts): a bounded lexer, a fixed relationship
vocabulary, exactly one named entity, hops ordered from that entity outward
(the main-clause verb is outermost), page types checked along the chain.
Coordination, negation, time constraints, counting, quoted names and
multi-entity questions are refused with a reason; single-relationship
questions stay on parseRelationalQuery. With search.relational_planner on,
the relational arm resolves the entity, runs runRelationalChain
(src/core/search/relational-chain.ts) and emits answers first, then the
intermediate and origin pages of every retained path, each row carrying a
relational evidence field (role, seed, hop, path count, up to three edges).
Up to search.relational_chain_slots (default 10) chain rows lead page 1.
A chain that finds nothing falls back to the one-hop path; a refused question
runs no relational arm. meta.relational_plan and a relational_chain
notice report the outcome.
Execution is one bounded query per hop (BrainEngine.relationalChainHop, SQL
in src/core/search/read-enrichment.ts): at most 50 frontier pages, 100
logical edges per page (lowest link id first) and 10 retained paths per page.
Links are read by each relationship's page-type signature (stored,
flipped, uncertain at half weight; canonical frontmatter, manual and
attendance-section links keep stored direction), duplicate rows between a
pair collapse into one logical edge, and every endpoint, origin and degree
contributor passes the read policy before scoring. Path weight is the product
of hub weights of the intermediate pages (hubWeight,
src/core/search/hub-dampening.ts; degree saturates at 300 link rows);
ties order by source and slug. The same executor serves traverse_graph
hops and gbrain graph-query --hop.
Chain evidence is page-level through fusion: whichever chunk of a page
survives carries the page's relational field. Only answers count toward the
relational re-pin and the page-1 evidence slot; intermediate and origin pages
never claim them. The planner keys fold into the query-cache key only when on
(rp=1, ro=1). Guide: multi-hop relationship questions.
Link types that take no part in ranking
Every reader that turns links into a ranking signal (backlink and adjacency
boosts, relational fanout and chain hops, chain hub degree, enrich order,
canonical entity picks, generated-page demotion, entity anchoring) skips the
link types in NON_RANKING_LINK_TYPES (src/core/link-ranking-policy.ts):
today amends. Those edges stay ordinary references in graph reads and cards.
Guide: amendments.
Metadata boost gate: vector-only voters keep the vector order
The post-fusion metadata boosts (backlink, salience, recency + chronicle,
graph signals, alias resolution) reward well-connected pages. That is right
when a lexical arm agreed the page is about the query; it is wrong on
paraphrase-style concept questions where nothing but the vector arm voted —
there the boosts promote hub pages (1.03–1.12x) over the gold concept page,
which carries none. decideMetadataBoosts
(src/core/search/metadata-boost-gate.ts) runs before runPostFusionStages
and, under search.metadata_boost_gate=lexical (every bundle), skips those
boosts when no strict keyword, title-phrase or relational row fused (relaxed
OR-fallback rows do not count). Supersede downrank, exact-match boost,
title-phrase boost, compiled-truth boost, cosine re-score, dedup, reranker
and autocut are untouched either way. Receipt (Cat 13 conceptual recall,
gbrain-evals): held-out nDCG@5 53.0 → 57.8 (bare vector 60.5 remains the
stretch), NamedThingBench, BrainBench, the retrieval canary and the
LongMemEval dev slice byte-identical. always applies the boosts whatever
voted; the decision is on HybridSearchMeta.metadata_boost_gate; the
knob folds into the query-cache key (mbg=). The companion
search.keyword_arm_confidence_floor (src/core/search/arm-confidence.ts,
off in every bundle) down-weights a weak keyword arm in the fusion; its
pre-registered receipt did not move the held-out score, so it is an
operator knob only.
Autocut: score-discontinuity result-sizing
Off by default in every bundle. Pre-registered rule R2 measured it on
LongMemEval from the default path's captured post-rerank pool: with the
reranker on, the
score cliff after the top session is the normal shape on multi-part questions,
and the cut removed the second gold session — strict recall_all@5 449/470 →
379/470 (−68 paired on the 430-question decision set), with no floor in the
sweep {0.10 … 0.80} within two questions of "off" on either seeded half
(0.80 still lost 9, all knowledge-update). Any-hit stayed ≥ 99.4% throughout:
autocut keeps the best session and drops the rest, which is a token saving
(mean returned window 3256 → 1633 estimated tokens at 0.35) paid for with the
questions that need more than one session. gbrain config set search.autocut true re-enables it with the knobs below; a session-aware cut (never below k
distinct sessions) is a filed follow-up. applyAutocut
(src/core/search/autocut.ts) cuts the ranked set at the largest
cross-encoder rerank-score cliff, before the limit slice, first page only.
Never-empty failsafe (minKeep), no-op when fewer than 2 results carry a
finite rerank score (covers the fail-open reranker path), and alias-hop exact
matches are preserved through the cut. Weak-top floor: when the top rerank
score is below minTopScore (default 0.35, config search.autocut_min_top),
cliff trimming is skipped entirely — a low-confidence list returns the full
cluster for the caller to judge instead of collapsing to one result. Knobs:
per-call SearchOpts.autocut → search.autocut / search.autocut_jump /
search.autocut_min_top config → mode bundle. The pre-autocut pool can be
captured for offline floor replay: hybridSearch exposes an eval-only
onRerankPool hook that fires with the exact pool applyAutocut is about to
cut (post-rerank, post alias-hop / exact-lookup, unscored injections included),
gbrain eval longmemeval --capture-pool records it per row as rerank_pool,
and scripts/replay-autocut-floor.ts replays every floor — including off —
from that single capture, validating byte-for-byte against the live decisions
before any other cell is read.
Each stage is testable in isolation. Each stage is replaceable. The whole pipeline is < 1ms of orchestration cost; the latency budget goes to the upstream HTTP calls (embedding, rerank) and the index scans.
Evidence delivery: whole evidence after ranking
Ranking decides which pages matter; the reader still needs enough of each page
to answer. return_unit (chunk, window, section, page, auto; default auto)
adds a stage after ranking, cache and capture, and before output
redaction and snippet capping, shared by search, query, recall and
think (via think.return_unit). It groups the ranked hits by page, reads
every needed neighbor chunk in ONE batched, page_id-keyed query that
re-authorizes each page under the caller's current scope, sanitizes each
page's complete body before slicing, locates the hit chunks in it (so chunk
overlap never duplicates text), cuts the requested unit around the hits,
and packs blocks by rank into the token budget with a per-page floor so lower
ranked sessions keep their matching span. The strict protected-body sanitizer
runs on the whole body first, so delivered evidence never contains more than
get_page returns to the same caller. The default auto expands only
conversation pages (by page type or chat/ / conversations/ slug) to the
whole page and leaves every other hit its ranked chunk; with no conversation
hit, or with chunk, the stage does not run and responses are byte-identical. The measured motivation: on 400 held-out
LongMemEval questions whole sessions answered 361 against 253 for top-5
chunks, while neighbor windows reached only 285–292, so auto gives
conversations exactly the page unit. auto itself answered 445 of 500
LongMemEval-S questions against 312 for chunks (development data), and the
preregistered sealed check was inconclusive at a ceiling (149 against 147 of
150); see docs/evidence-delivery.md for the numbers and the token cost. A lexical
excerpt selector that failed its evaluation is recorded in
docs/eval/ANSWER_PACKET_RESULTS.md. Contract, algorithms and latency:
docs/evidence-delivery.md.
How to verify on your own brain
# Reproduce the public LongMemEval receipt (cleaned S split, k=5) like-for-like:
# --by-type prints strict recall_all@5 (headline) and recall_any@5 (diagnostic);
# the like-for-like row pins the reranker and autocut off.
gbrain eval longmemeval ~/datasets/longmemeval/longmemeval_s_cleaned.json \
--retrieval-only --top-k 5 --by-type --no-trajectory --mode balanced --reranker off --autocut off
# The shipped default path (what balanced/tokenmax run): --reranker on --autocut off
# Capture your own queries and replay against retrieval changes
export GBRAIN_CONTRIBUTOR_MODE=1
# ... use gbrain normally ...
gbrain eval export > before.ndjson
# ... change something ...
gbrain eval replay --against before.ndjson
# A/B retrieval strategies on a labeled fixture
gbrain eval --qrels qrels.json --config-a baseline.json --config-b balanced.json
The current measured LongMemEval result (95.53% session-level recall_all@5 on the release default path, 449/470, and 93.40% with the reranker off, 439/470; cleaned S split, 470 scored questions, k=5, measured 2026-09-06 by the in-repo harness), its per-type table, every measured ranker arm and the judged answer-accuracy row live in docs/eval-bench.md.
Methodology + metric glossary in docs/eval/SEARCH_MODE_METHODOLOGY.md.
Restricted salience
Remote reads default to the world take holder when no grant is supplied; an
explicit empty grant permits no takes. Counts and average weights use only
permitted active takes. Stored emotional weight contributes zero because it
combines all holders. Recent-salience inclusion uses updated_at with the
same recency formula as unrestricted local reads, which use their full
formulas. Deleted, quarantined and archived contributors are excluded
before ranking, limits and anomaly-baseline aggregation. Default remote page
privacy also excludes private pages; its documented opt-outs do not widen
take-holder permissions.
Chunk rebuilds after upgrading
Say to your agent: "Check whether my search index is ready, and repair
code metadata without spending on embeddings." Search and query report
projection_readiness for empty and nonempty results. projection_pending
means visible pages lack a current revision seal and are not still served from
the index they had before migration 153's rebuild; projection_status_unknown
means the diagnostic could not establish readiness. Neither is a clean miss.
projection_by_design means the only unsealed pages in scope are
protected-OCR images (or pages of an archived source), which stay out of the
index by design: nothing is pending and no repair applies.
The CLI names these states; --json retains its result-array format and sends
incompleteness notices to stderr. MCP carries them in _meta.retrieval.
A resident gbrain serve drains queued Markdown and code rebuilds
without provider calls. Code repair can also run through the current owner:
gbrain reindex-code --force --no-embed. Rebuilds preserve only exact,
provenance-compatible vectors: each vector records the embedding input it was
built from (content_chunks.embedding_input_hash: column, model, dimensions,
wrapping tier and wrapped text), and a rebuild keeps it only when the current
page would produce the same input, so an unchanged contextual page keeps its
vectors and a synopsis-mode body edit nulls every synopsis-tier chunk. Vectors
written before that record existed are grandfathered on pages whose mode
is still NULL; contextual-mode repair and reindex stamp compatible raw inputs
when moving those pages to explicit none. An explicit none page may have
inherited a title-wrapped vector, so a rebuild clears unstamped vectors there,
and switching from title or synopsis to none clears active text vectors
immediately. Genuine old raw vectors on explicit none pages may need a
one-time re-embed; it never starts automatically. Remaining NULL vectors
still need an explicitly authorized gbrain embed --stale run. A text-ready
index is not a promise that every page has a vector. Diagnostics do not
disclose private or foreign-source pending pages and never start repair
themselves.
Markdown chunk creation applies the strict protected-body sanitizer before
splitting text. For remote reads, all existing chunks of every page kind are
withheld until a successful rebuild records the current chunker version. Public
pages require this rebuild too; trusted local chunk reads remain available. Body or
chunk changes invalidate that record until the next successful rebuild. Re-importing
unchanged content (gbrain sync --full, gbrain import) re-seals such a page
without a page write or journal admission. While pages are withheld, remote
search / query report degraded: [safe_index_pending] on empty and partial
results alike (the MCP empty-result block names it) instead of a clean or complete
answer, and gbrain doctor counts the withheld markdown and code pages and points
at gbrain repair safe-chunks, which re-seals them on any brain, managed included.
Direct page reads continue to use current source and visibility policy plus body
sanitization.
Run rebuild commands from a local installation on the brain host; thin clients
cannot rebuild the host's indexes.
gbrain reindex --markdown --dry-run --no-embed previews the existing rebuild.
gbrain reindex --markdown --no-embed rebuilds without embedding calls, replacing
previous vectors; use gbrain embed --stale later to restore semantic retrieval
when provider usage is authorized. The regular reindex path rebuilds and embeds.
Code pages use gbrain reindex-code --force --no-embed. Existing image indexes
require reimporting the source files; images whose OCR contains protected
sections remain unavailable to remote chunk retrieval. Schema migrations 160
and 161 install projection statistics and the pending-projection lookup index;
they do not rebuild vector indexes or call an embedding provider.
query --since includes its exact lower boundary. An exact --until timestamp
is inclusive without losing fractional precision; a date-only upper bound
includes the entire day, implemented as an exclusive next-midnight boundary.
Legacy direct engine afterDate and beforeDate bounds remain strict unless
their explicit inclusivity flags are set.
Optional code-graph expansion is omitted from remote search. The six dedicated code-inspection operations are also temporarily local-only; see the MCP surface runbook. These restrictions are separate from the chunk rebuild requirement.