Skip to content

Local-first 🧠 memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly 😴 dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.

TypeScript

feat: Search & Recall Quality Suite - explainable scores, trust, threshold, reinforce, eval (#93)

Sol Aitkene4df212cf243
diff --git a/CHANGELOG.md b/CHANGELOG.md
index f89f431d..488845f7 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,6 +5,51 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+## [1.5.0] - 2026-06-13
+
+### Added
+
+- **Search & Recall Quality Suite.** Seven language-agnostic enhancements to
+ the retrieval core. The five `brain_search` enhancements are each gated behind
+ their own flag, so a call that sets none of them returns byte-identical
+ results, ordering, and shape to before; the hybrid-degrade warning fires only
+ in a genuinely degraded state, and `brain_eval` is a new read-only tool.
+ - **Structured score breakdown.** `brain_search` accepts `explain: true` and
+ returns a per-result `score_breakdown` object (keyword, semantic, rrf,
+ entity, activation, coAccess, link, recency, tier, trend, sessionFocus)
+ alongside the existing human-readable `reasons[]`. The ranker now emits the
+ breakdown first-class, and the learned-weights fold reads it directly
+ instead of re-parsing reason strings.
+ - **Hybrid-degrade warning.** When a caller wants the semantic lane but it
+ cannot run (missing embeddings, unloaded vec extension, unconfigured key),
+ the result set carries one greppable `hybrid_degraded:` warning instead of
+ silently serving keyword-only. A keyword-only-by-choice query and a
+ semantic-disabled vault never carry it.
+ - **Inline trust metadata.** `brain_search` accepts `trust: true` and stamps
+ each hit with a computed-at-read-time `trust` object: `age_days` from the
+ document mtime, plus `superseded` and `conflict` derived from the
+ `superseded_by` / `contradicts` typed relations the recall path already
+ surfaces. Never stored.
+ - **Relevance threshold and rerank.** `threshold` (a `[0,1]` score floor
+ applied before the diversity rerank) drops weak hits so an irrelevant query
+ returns no match instead of noise; `rerank` re-orders the qualified set by
+ core textual relevance. Both off by default.
+ - **Self-tuning reinforce.** `reinforce: [paths]` records proven-useful
+ memories to a conflict-free one-file-per-signal ledger under
+ `Brain/search/reinforce/` and lifts ledger-strong memories by a bounded
+ boost before the top_k cut. Surfaced-only frequency is never recorded and
+ never boosts; the ledger is resettable.
+ - **Answer-containment@k benchmark metric.** The recall benchmark accepts an
+ optional per-query `answer` and reports `answerContainmentAtK` - whether the
+ answer text appears (folded substring) in the retrieved content within the
+ top k. The committed fixture gains answer-bearing queries with a CI floor;
+ datasets without `answer` keep hit@k / MRR behaviour unchanged.
+ - **`brain_eval` MCP tool.** A read-only tool that scores retrieval quality
+ over a caller-supplied dataset against the active vault, returning hit@k,
+ MRR, answer-containment@k, source-utilization@k, citation-depth, and a
+ source-warnings count a CI gate can cap. The fast path needs no embedding
+ key.
+
## [1.4.0] - 2026-06-12
### Added
@@ -5220,6 +5265,8 @@ plugin config (vault field)`, and exits with a clear
- Sandbox vault and plugin manifest fixtures for tests.
- GitHub release workflow for tag-based and manually dispatched releases.
+[1.5.0]: https://github.com/itechmeat/open-second-brain/compare/v1.4.0...v1.5.0
+[1.4.0]: https://github.com/itechmeat/open-second-brain/compare/v1.3.1...v1.4.0
[1.3.1]: https://github.com/itechmeat/open-second-brain/compare/v1.3.0...v1.3.1
[1.3.0]: https://github.com/itechmeat/open-second-brain/compare/v1.2.0...v1.3.0
[1.2.0]: https://github.com/itechmeat/open-second-brain/compare/v1.1.0...v1.2.0
diff --git a/README.md b/README.md
index bb840c13..820dc336 100644
--- a/README.md
+++ b/README.md
@@ -82,7 +82,7 @@ Each non-Hermes target writes a sidecar manifest under `<vault>/.open-second-bra
- **Memory that learns, and forgets, on its own.** A nightly `dream` pass turns repeated corrections into rules and retires the ones nothing uses any more. Deterministic by design: counters and atomic file moves, no LLM guessing inside your memory.
- **One brain, every agent.** Teach a rule in one agent and the next one already knows it - Hermes, Claude Code, Codex, Cursor, and the rest read and write the same vault.
- **You stay in control.** Pin, merge, retire, or roll back any rule from the `o2b` CLI. Every Brain mutation takes a verified snapshot first, so a bad change is one `o2b brain rollback` away.
-- **Search that explains itself.** Keyword plus an optional semantic layer over your vault, with results that show why they surfaced and what was missing - not a black box.
+- **Search that explains itself.** Keyword plus an optional semantic layer over your vault, with results that show why they surfaced and what was missing - not a black box. Opt into a structured per-result score breakdown (`explain`), inline trust metadata (age, superseded, conflict), a relevance threshold that returns nothing rather than weak noise, and reinforcement that lifts memories you have marked useful. Track retrieval quality over time with `brain_eval` and the recall benchmark (hit@k, MRR, answer-containment@k).
- **Conversations survive compaction.** When the host compresses context and rotates the session id, capture and recall stitch the segments back into one conversation - any segment id returns the whole lineage.
- **Memory that cleans itself, on your terms.** `o2b brain hygiene scan` surfaces contested facts, near-duplicate rules, stale derived pages, and never-recalled memories; `apply` executes only the findings you select, and stale pages recompile from their recorded sources with a dry-run preview.
diff --git a/docs/brainstorm/search-recall-quality/cli-output/claude.md b/docs/brainstorm/search-recall-quality/cli-output/claude.md
new file mode 100644
index 00000000..689f3020
--- /dev/null
+++ b/docs/brainstorm/search-recall-quality/cli-output/claude.md
@@ -0,0 +1,34 @@
+### Variant 1: Concern-located two-layer split
+- **Approach**: Land each unit where its concern already lives rather than introducing shared scaffolding. Read-time projections that need only data already on `SearchOutcome` (score_breakdown serialization, hybrid-degrade warning surfacing) are shaped at the `src/mcp/search-tools.ts` surface; ranking-behavior changes (threshold+rerank, reinforce re-rank) extend the pure ranker behind `ResolvedRecallConfig` kill switches; trust metadata reuses `recall-hint.ts` as a sibling read-time deriver in core; eval/benchmark units extend `benchmark.ts` and add a standalone MCP `run_eval` tool.
+- **Trade-offs**:
+ - Pro: smallest conceptual footprint; no new abstraction to learn or maintain; each commit is a tight, reviewable diff against the file that owns the concern.
+ - Pro: strongest match to "pure core stays side-effect free; surfaces opt into telemetry" β€” the reinforce ledger and eval recording naturally sit at the MCP edge.
+ - Con: score_breakdown projection at the MCP layer can only see fields `BrainSearchResult` already carries; `entityBoost`/`activationBoost`/`coAccessBoost`/`sessionFocus` are computed inside the ranker and would need to be threaded out anyway, so the "thin surface" promise partially leaks back into core.
+ - Con: read-time enrichment logic ends up split across two layers (some in core `recall-hint`-style, some in MCP), making the read-time contract harder to reason about as a whole.
+- **Complexity**: medium
+- **Risk**: low
+
+### Variant 2: Unified read-time enrichment module in core
+- **Approach**: Add one pure core module (`src/core/search/enrich.ts`) as an explicit sibling to `recall-hint.ts` that owns every read-time projection β€” `score_breakdown`, inline trust fields, and hybrid-degrade detection β€” as pure functions over a ranker output that carries all numeric components. `SearchOptions` gains a single additive `enrich`/`explain`/`threshold`/`reinforce` opt-in bag; threshold+rerank and reinforce re-rank extend the ranker/MMR path behind `ResolvedRecallConfig` switches; the MCP surface only serializes and owns the side-effecting reinforce ledger and the `run_eval` exposure; benchmark gains answer-containment@k in place.
+- **Trade-offs**:
+ - Pro: all read-time, never-stored projections live in one cohesive, unit-tested core module mirroring the proven `recall-hint` pattern, so the "computed-at-read-time, never stored" invariant is enforced in one place.
+ - Pro: ranker carries every score component once, so score_breakdown is a trivial structural projection with no leak back-and-forth; byte-identical legacy output is easy to assert because every field is gated by an explicit flag default.
+ - Pro: keeps the side-effect boundary clean β€” pure enrichment in core, ledger/eval recording at the surface.
+ - Con: requires widening the ranker's output type to surface all components, a slightly larger up-front type change touching `types.ts`.
+ - Con: one module owning three distinct projections risks becoming a mild grab-bag if not internally sectioned.
+- **Complexity**: medium
+- **Risk**: low
+
+### Variant 3: Composable gated enrichment pipeline with registry
+- **Approach**: Introduce a generalized read-time enrichment pipeline where each of the seven units registers as a gated transform stage over the ranked outcome, dispatched by a flag/kill-switch map, plus a parallel pluggable structure for eval dimensions (source-utilization, citation-depth, answer-containment) as registered metrics. The MCP surface and benchmark both consume the same registry so new signals/metrics drop in uniformly.
+- **Trade-offs**:
+ - Pro: maximally uniform and future-proof β€” an eighth signal or metric is a one-stage registration, and ordering/composition with the MMR pass is explicit and centrally controlled.
+ - Pro: forces a single declarative answer to "which flag gates which behavior," reducing the chance any unit silently mutates default output.
+ - Con: heaviest abstraction for a fixed set of seven units shipping at once; the registry indirection obscures otherwise-trivial projections and works against the repo's concrete, per-signal, one-file conventions.
+ - Con: a shared pipeline core makes the seven "atomic, independently TDD'd commits" harder to land cleanly β€” early commits must build scaffolding the later ones use, coupling the commit sequence.
+ - Con: higher risk of perturbing byte-identical legacy output, since all results now flow through a new common path even when no flag is set.
+- **Complexity**: large
+- **Risk**: medium-high
+
+### Recommended: Variant 2
+**Rationale**: It directly follows the established `recall-hint.ts` precedent the task itself names as the model pattern, keeping all read-time/never-stored projections cohesive and pure while leaving ledger writes and eval exposure at the MCP edge where side effects belong. It avoids Variant 1's score-component leak between layers and sidesteps Variant 3's registry over-engineering, which would couple the seven supposedly-atomic commits and threaten the byte-identical-when-flags-off guarantee that is a hard constraint here.
diff --git a/docs/brainstorm/search-recall-quality/cli-output/prompt.md b/docs/brainstorm/search-recall-quality/cli-output/prompt.md
new file mode 100644
index 00000000..f5fa91b8
--- /dev/null
+++ b/docs/brainstorm/search-recall-quality/cli-output/prompt.md
@@ -0,0 +1,72 @@
+You are brainstorming architectural variants for the following task. Do not write code. Do not write a final design. Only produce variants and a recommendation.
+
+# Task
+
+Ship a "Search & Recall Quality" suite for Open Second Brain: seven related, atomic enhancements to the retrieval core (all in `src/core/search/` plus the MCP search surface `src/mcp/search-tools.ts`). They share one release. Each is additive, opt-in or backward-compatible, and language-agnostic (NO hardcoded natural-language word lists in any language; structural signals, frontmatter fields, document-frequency, or LLM extraction only). The seven units:
+
+1. score_breakdown (explain): add an opt-in `explain` flag to brain_search that returns a structured per-result `score_breakdown` object (semantic, keyword, entity_boost, recency_boost, link_boost, activation, co_access, etc.) alongside the existing human-readable `reasons[]` strings. The ranker already computes every numeric component; the gap is a structured projection in the output schema.
+
+2. relevance threshold + rerank: add an opt-in `threshold` (similarity/score floor, default disabled) and `rerank` flag to brain_search so a query with no sufficiently relevant memory returns no match instead of weak noise. Must compose with the existing post-rank MMR diversity pass.
+
+3. hybrid-degrade warning: emit a warning when the caller expects hybrid (keyword+semantic) search but it silently degrades to a single lane (e.g. semantic enabled but the vec extension/embedding key is unavailable, or keyword-only fallback). Today this degradation is silent.
+
+4. inline trust metadata on hits: stamp each recall hit with computed-at-read-time trust fields (age in days, superseded status, conflict status). The signals already exist in separate passes (brain_truth conflicts, bi-temporal `supersedes` frontmatter) but are not projected onto search hits. Must be read-time/computed like the existing recall-hint, never stored.
+
+5. self-tuning reinforce: add an opt-in `reinforce=[id,...]` parameter to recall that lifts specific memories proven useful (via a small plugin-side ledger) by re-ranking them before the top_k cut. Distinct from the existing feedback loop (which folds up/down verdicts into learned per-layer weights). Surfaced-only frequency is never a positive boost.
+
+6. MCP eval suite: expose the existing recall benchmark over MCP (a `run_eval`-style tool), add source-utilization and citation-depth eval dimensions, and a `source_warnings_max` CI gate. Read-only eval report/history resources.
+
+7. answer-containment metric + reproducible corpus: extend the existing recall benchmark (`src/core/search/benchmark.ts`, currently hit@k + MRR over a committed fixture) with an answer-containment@k metric and ensure a curated reproducible corpus + CI guard.
+
+# Project context
+
+Open Second Brain β€” an agent-owned Obsidian/Markdown second-brain memory system. TypeScript on the Bun runtime; SQLite (FTS5 + sqlite-vec) search index; MCP server exposing brain tools; CLI `o2b`. Single authoring language (English). Versioned via package.json mirrored to several manifests.
+
+Recent commits (suite-style PRs are the norm):
+2e74afe feat: native Grok Build CLI integration (v1.4.0)
+0340560 feat: Continuity, Hygiene & Freshness Suite (v1.3.0)
+957a403 feat!: Stability & Trust - 1.0.0 API freeze, safeguard, staged dream
+6d09d3c feat: Link & Recall Intelligence Suite - alias resolution, communities, recall benchmark, self-tuning
+b16c37d feat: Time-Aware Recall & Activation Suite - usage-aware ranking, event-time recall, two-pass recovery
+
+Related files (already read, accurate):
+- src/core/search/ranker.ts β€” pure ranker; computes keywordScore, semanticScore, linkBoost, recencyBoost, entityBoost, activationBoost, coAccessBoost, sessionFocus; emits reasons[] strings via buildReasons().
+- src/core/search/types.ts β€” BrainSearchResult (carries keywordScore, semanticScore, linkBoost, recencyBoost), SearchOptions, SearchOutcome (results, warnings, total), ResolvedRecallConfig (per-layer kill switches: learnedWeightsEnabled, selfTuningEnabled, relationPolarityEnabled, activationEnabled...).
+- src/mcp/search-tools.ts β€” brain_search input/output JSON schema; SEARCH_OUTPUT_SCHEMA already has results[].reasons, warnings[], total, recall_hint.
+- src/core/search/recall-hint.ts β€” deriveRecallHint(): computed-at-read-time, never-stored, single English template + numbers. The model pattern for inline trust metadata.
+- src/core/search/feedback.ts β€” explicit up/down feedback folded into bounded learned per-layer weights (Brain/search/feedback/, learned-weights.json). Pure, replayable fold.
+- src/core/search/benchmark.ts β€” runRecallBenchmark(): hit@k + MRR over a dataset; consumed by CI test, `o2b brain benchmark run`, and tuneRecall objective.
+- src/core/search/tuning.ts β€” grid-evaluates recall params against the benchmark, persists winner to Brain/search/tuning.json.
+- src/core/search/search.ts β€” resolveSemanticPolicy() decides semantic on/off; runSemanticPhase() attempts the semantic lane; warnings[] already collected and surfaced.
+- src/core/search/mmr.ts β€” mmrRerank() post-rank diversity pass.
+
+Conventions:
+- Every new ranking/recall behaviour is opt-in or has an explicit kill switch in ResolvedRecallConfig, and a vault that does not opt in ranks bit-identically to before.
+- Pure core stays side-effect free; CLI/MCP surfaces opt into recording/telemetry.
+- Per-result `reasons[]` and set-level recall_hint are computed at read time, never stored.
+- One-file-per-signal under Brain/search/* for conflict-free multi-device sync (feedback pattern).
+- Strict TypeScript; no `as` cast crutches; build values with the correct type from the start.
+- Language-agnostic: no natural-language keyword lists.
+
+Constraints:
+- Do not change existing public API result shapes by default β€” new fields must be additive and gated behind the opt-in flag (explain/threshold/reinforce), preserving byte-identical legacy output when the flag is off.
+- No new external runtime dependencies unless strongly justified.
+- All seven units ship on ONE feature branch as atomic commits, ONE CHANGELOG version, implemented one-by-one via TDD.
+- Trust metadata and score_breakdown must reuse the existing computed-at-read-time pattern, not add stored columns.
+
+# Required output format
+
+Produce exactly 3 distinct architectural variants. For each variant:
+
+### Variant N: <short name>
+- **Approach**: 2-3 sentences describing the variant.
+- **Trade-offs**: bullet list of pros and cons.
+- **Complexity**: small | medium | large
+- **Risk**: low | medium | high
+
+After the three variants, add exactly one recommendation:
+
+### Recommended: Variant N
+**Rationale**: 2-3 sentences explaining why this variant over the others, considering the project context and constraints above.
+
+Output nothing outside of these sections.
diff --git a/docs/brainstorm/search-recall-quality/design.md b/docs/brainstorm/search-recall-quality/design.md
new file mode 100644
index 00000000..02efd070
--- /dev/null
+++ b/docs/brainstorm/search-recall-quality/design.md
@@ -0,0 +1,162 @@
+# Search & Recall Quality - retrieval transparency, trust, and evaluation
+
+**Status:** draft
+**Author:** claude-dev-agent (via feature-release-playbook)
+**Audience:** implementation
+
+## Problem statement
+
+Open Second Brain's retrieval core ranks well but is opaque and untunable at the
+edges: callers get human-readable `reasons[]` strings but no structured score
+components; a query with no genuinely relevant memory still returns weak noise;
+hybrid search degrades to a single lane silently; recall hits carry no validity
+signal (age, superseded, conflict) even though those signals exist in separate
+passes; there is no per-call way to reinforce memories an agent just proved
+useful; and retrieval quality is measured only by an internal hit@k/MRR test
+with no MCP-accessible eval or answer-containment metric. This suite closes those
+gaps as seven additive, opt-in, language-agnostic enhancements that ship in one
+release.
+
+## Scope
+
+Seven atomic units, all in `src/core/search/` and `src/mcp/search-tools.ts`:
+
+1. **score_breakdown (`explain`)** - structured per-result score components in
+ `brain_search` output, gated behind an `explain` flag.
+2. **relevance threshold + rerank** - opt-in score floor and second-pass rerank
+ so an irrelevant query returns no match instead of weak noise.
+3. **hybrid-degrade warning** - a warning when expected hybrid search silently
+ falls back to a single lane.
+4. **inline trust metadata** - computed-at-read-time age / superseded / conflict
+ fields stamped on each recall hit.
+5. **self-tuning reinforce** - opt-in `reinforce=[id,...]` that lifts proven
+ memories via a plugin-side ledger before the top_k cut.
+6. **answer-containment@k + reproducible corpus** - a new benchmark metric and a
+ committed curated corpus with a CI guard.
+7. **MCP eval suite** - the recall benchmark exposed over MCP with
+ source-utilization / citation-depth dimensions and a `source_warnings_max`
+ CI gate, plus read-only eval report/history resources.
+
+## Out of scope
+
+- Cross-encoder / neural reranking models (the `rerank` flag uses the existing
+ deterministic signals; a model-backed reranker is a separate future task).
+- Any stored ranking column or new SQLite migration for trust/score data - all
+ enrichment stays read-time and never-stored.
+- Owner/agent-scoped fact isolation, typed-edge relational retrieval, and the
+ indexer-durability family (separate triage clusters).
+- New natural-language word lists of any kind.
+
+## Chosen approach
+
+Adopt the consultant's **Variant 2 - unified read-time enrichment module in
+core** (see `variants.md` for the full audit trail and the three variants
+considered).
+
+A single pure module `src/core/search/enrich.ts` is added as a sibling to the
+proven `recall-hint.ts`. It owns the read-time, never-stored projections:
+`score_breakdown`, inline trust metadata, and hybrid-degrade detection - each a
+pure function over a ranker output that now carries every numeric component
+first-class. Ranking-behaviour changes (threshold filter, reinforce re-rank)
+extend the pure ranker / post-rank path behind explicit options and
+`ResolvedRecallConfig` kill switches. The MCP surface only serializes the
+enrichment and owns the two side-effecting pieces: the reinforce ledger
+(`Brain/search/reinforce/`, the conflict-free one-file-per-signal pattern) and
+the `run_eval` exposure. The benchmark gains the answer-containment metric in
+place.
+
+The byte-identical-when-flags-off guarantee holds because every new output field
+is gated behind its opt-in flag (`explain`, `threshold`, `reinforce`,
+`trust`) and the MCP serializer picks fields explicitly; a vault that opts into
+nothing produces byte-identical results and ranks bit-identically.
+
+## Design decisions
+
+- **Ranker emits a structured per-layer breakdown** (refinement over the bare
+ consultant recommendation): `BrainSearchResult` gains an always-present
+ `breakdown` object carrying `keyword`, `semantic`, `entity`, `recency`, `link`,
+ `activation`, `coAccess`, `sessionFocus`, and (when rrf) `rrf`. This removes the
+ existing string-parse crutch in `feedback.ts` (`contributionsFromResult` parses
+ `entity_match:` back out of a reason string) - it reads the structured field
+ instead. `score_breakdown` is then a pure projection, not a re-parse. The field
+ is core-internal; MCP only emits `score_breakdown` when `explain` is set.
+- **Threshold composes before MMR diversity.** The relevance floor filters
+ candidates after scoring but before the MMR rerank, so diversity is computed
+ over the already-qualified set. `threshold` defaults to disabled (0); the
+ default path is byte-identical. `rerank` triggers a deterministic second pass
+ over the top candidates using the existing signals - no new model dependency.
+- **Trust metadata is read-time, never stored**, mirroring `recall-hint.ts`. Age
+ derives from document mtime; `superseded` from the bi-temporal `superseded_by`
+ frontmatter already materialized into the links table; `conflict` from the
+ existing `computeTruthStateWithConflicts` pass. Absent signals degrade to a
+ neutral (untagged) state - no per-locale phrasing, just structural fields plus
+ numbers.
+- **Reinforce is an explicit per-call boost list, distinct from the feedback
+ loop.** The feedback loop folds up/down verdicts into bounded learned per-layer
+ weights; reinforce lifts caller-named ids (chunk/document/path) before the
+ top_k cut via a small ledger that records which ids were marked useful.
+ Surfaced-only frequency is never a positive signal (only an explicit reinforce
+ mark is). Bounded, opt-in, resettable - same discipline as `feedback.ts`.
+- **Hybrid-degrade detection is structural.** A degrade is "caller wanted both
+ lanes but only one ran" - derived from `resolveSemanticPolicy` intent vs the
+ actual `semanticAttempted` / lane-availability state already tracked in
+ `search.ts`. The warning string is a single English template plus identifiers,
+ consistent with the language-agnostic stance.
+- **Eval dimensions reuse the benchmark runner.** `source-utilization` (fraction
+ of expected sources surfaced) and `citation-depth` (rank depth of expected
+ hits) are computed from the same `runRecallBenchmark` outputs;
+ `answer-containment@k` checks whether an expected answer substring appears in
+ the retrieved content, not just whether the path matched. The MCP `run_eval`
+ tool wraps the runner read-only; the `source_warnings_max` gate is a threshold
+ the CI test asserts.
+
+## File changes
+
+New files:
+- `src/core/search/enrich.ts` - read-time enrichment: score_breakdown projection,
+ trust metadata deriver, hybrid-degrade detector. Pure, no I/O.
+- `src/core/search/reinforce.ts` - reinforce ledger (one-file-per-signal under
+ `Brain/search/reinforce/`) + the bounded re-rank applied before top_k.
+- `tests/core/search/enrich.test.ts`, `tests/core/search/reinforce.test.ts`,
+ `tests/core/search/threshold.test.ts`, `tests/core/search/answer-containment.test.ts`,
+ `tests/core/search/hybrid-degrade.test.ts`, `tests/mcp/eval-tools.test.ts` (and
+ extensions to existing ranker/search/search-tools/benchmark test files).
+- `docs/brainstorm/search-recall-quality/*` (this phase).
+
+Modified files (expected):
+- `src/core/search/ranker.ts` - emit the structured `breakdown` object.
+- `src/core/search/types.ts` - `BrainSearchResult.breakdown`; `SearchOptions`
+ (`explain`, `threshold`, `rerank`, `reinforce`, `trust`); `SearchOutcome`
+ (degrade warnings already flow through `warnings[]`).
+- `src/core/search/search.ts` - threshold filter before MMR, reinforce re-rank,
+ hybrid-degrade detection wiring, trust/score_breakdown enrichment on results.
+- `src/core/search/feedback.ts` - read the structured breakdown instead of
+ re-parsing reason strings.
+- `src/core/search/benchmark.ts` - answer-containment@k + source-utilization +
+ citation-depth metrics.
+- `src/mcp/search-tools.ts` - `explain` / `threshold` / `rerank` / `reinforce` /
+ `trust` input schema; `score_breakdown` + `trust` output schema; reinforce
+ ledger write at the surface.
+- New MCP eval tool registration (`src/mcp/eval-tools.ts` or extend
+ `search-tools.ts`) + `tools.ts` wiring + eval report/history resources.
+- `README.md`, `CHANGELOG.md`, `docs/` search-design notes, version manifests.
+- A committed benchmark corpus fixture + dataset under `tests/fixtures/` (or the
+ existing recall-benchmark fixture location) for the CI guard.
+
+## Risks and open questions
+
+- **Public shape additivity.** Adding `breakdown` to `BrainSearchResult` is a
+ core-internal change; verify no existing test asserts exact `BrainSearchResult`
+ key sets that would break. Mitigation: the field is always present in core but
+ never surfaced by MCP unless `explain` is set; assert byte-identical MCP output
+ with flags off.
+- **Reinforce id namespace.** Decide the id form callers pass (path vs
+ documentId:chunkId). Resolve in implementation against how agents already
+ reference results (`recall_feedback` uses `result_path`); prefer path for
+ consistency.
+- **answer-containment corpus.** The metric needs an `answer` field per query in
+ the dataset; keep it optional so existing datasets stay valid (containment is
+ computed only for queries that declare an answer).
+- **Threshold semantics with rrf fusion.** RRF scores are not comparable to
+ linear scores; the threshold must be applied against the final normalized
+ `score` (already clamped to [0,1]) so it is meaningful in both fusion modes.
diff --git a/docs/brainstorm/search-recall-quality/plan.md b/docs/brainstorm/search-recall-quality/plan.md
new file mode 100644
index 00000000..a97ba27f
--- /dev/null
+++ b/docs/brainstorm/search-recall-quality/plan.md
@@ -0,0 +1,95 @@
+# Search & Recall Quality - implementation plan
+
+Seven atomic units, implemented one-by-one via TDD on `feat/search-recall-quality`,
+each its own conventional commit. Format + lint must pass before every commit.
+Tasks are ordered so each builds on the prior; the byte-identical-when-flags-off
+guarantee is re-asserted by the existing suite after every commit.
+
+## Tasks
+
+### Task 1: Structured score breakdown + `explain`
+- **Files**: `src/core/search/ranker.ts`, `src/core/search/types.ts`,
+ `src/core/search/enrich.ts` (new), `src/core/search/search.ts`,
+ `src/core/search/feedback.ts`, `src/mcp/search-tools.ts`,
+ `tests/core/search/enrich.test.ts` (new), `tests/core/search/ranker*.test.ts`,
+ `tests/core/search/feedback.test.ts`, `tests/mcp/*search*`.
+- **Acceptance**: ranker emits an always-present `breakdown` object; `enrich.ts`
+ projects it into a `score_breakdown`; `brain_search` returns `score_breakdown`
+ only when `explain: true`; output is byte-identical when `explain` is absent;
+ `feedback.ts` reads the structured field instead of re-parsing reason strings;
+ all existing tests stay green.
+- **Depends on**: none (foundation).
+
+### Task 2: Hybrid-degrade warning
+- **Files**: `src/core/search/search.ts`, `src/core/search/enrich.ts`,
+ `tests/core/search/hybrid-degrade.test.ts` (new), `tests/core/search/search.test.ts`.
+- **Acceptance**: when the caller's resolved policy wants both lanes but only one
+ ran (vec/key unavailable, or keyword-only fallback), a single structural
+ warning is appended to `warnings[]`; no warning when hybrid genuinely ran or
+ when the caller explicitly asked for one lane; language-agnostic template.
+- **Depends on**: Task 1 (shares `enrich.ts`).
+
+### Task 3: Inline trust metadata on hits
+- **Files**: `src/core/search/enrich.ts`, `src/core/search/types.ts`,
+ `src/core/search/search.ts`, `src/mcp/search-tools.ts`,
+ `tests/core/search/enrich.test.ts`, `tests/core/search/search.test.ts`,
+ `tests/mcp/*search*`.
+- **Acceptance**: with `trust: true`, each hit carries computed-at-read-time
+ `age_days`, `superseded`, and `conflict` fields derived from mtime, the
+ `superseded_by` links, and `computeTruthStateWithConflicts`; never stored;
+ absent signals degrade to a neutral untagged state; off by default and
+ byte-identical when absent.
+- **Depends on**: Task 1.
+
+### Task 4: Relevance threshold + rerank
+- **Files**: `src/core/search/search.ts`, `src/core/search/types.ts`,
+ `src/core/search/mmr.ts` (compose), `src/mcp/search-tools.ts`,
+ `tests/core/search/threshold.test.ts` (new), `tests/core/search/mmr.test.ts`,
+ `tests/mcp/*search*`.
+- **Acceptance**: `threshold` (default 0/disabled) filters results whose final
+ normalized `score` is below the floor, applied before the MMR diversity pass;
+ a query with no qualifying hit returns an empty result set (no weak noise);
+ `rerank` runs a deterministic second pass over top candidates using existing
+ signals; default path byte-identical.
+- **Depends on**: Task 1.
+
+### Task 5: Self-tuning reinforce
+- **Files**: `src/core/search/reinforce.ts` (new), `src/core/search/search.ts`,
+ `src/core/search/types.ts`, `src/mcp/search-tools.ts`,
+ `tests/core/search/reinforce.test.ts` (new), `tests/mcp/*search*`.
+- **Acceptance**: `reinforce: [path,...]` lifts the named results before the
+ top_k cut via a bounded, resettable ledger under `Brain/search/reinforce/`
+ (one-file-per-signal); surfaced-only frequency never boosts; the ledger write
+ happens at the MCP surface, the re-rank is pure; off by default and
+ byte-identical when absent.
+- **Depends on**: Task 1, Task 4 (re-rank ordering interacts with threshold).
+
+### Task 6: Answer-containment@k + reproducible corpus
+- **Files**: `src/core/search/benchmark.ts`,
+ `tests/core/search/answer-containment.test.ts` (new),
+ `tests/core/search/recall-benchmark.test.ts`, benchmark corpus fixture + dataset.
+- **Acceptance**: `RecallBenchmarkQuery` gains an optional `answer` field;
+ `runRecallBenchmark` reports `answerContainmentAtK` computed only for queries
+ that declare an answer; a committed curated corpus + dataset run under the CI
+ test with a pinned floor; existing hit@k/MRR behaviour unchanged for datasets
+ without `answer`.
+- **Depends on**: none (benchmark is independent of the search-options work).
+
+### Task 7: MCP eval suite
+- **Files**: `src/mcp/eval-tools.ts` (new) or extend `src/mcp/search-tools.ts`,
+ `src/mcp/tools.ts`, `src/core/search/benchmark.ts` (source-utilization +
+ citation-depth), `tests/mcp/eval-tools.test.ts` (new),
+ `tests/core/search/benchmark.test.ts`.
+- **Acceptance**: a read-only `brain_eval` (`run_eval`-style) MCP tool runs the
+ benchmark and returns hit@k / MRR / answer-containment@k / source-utilization /
+ citation-depth; a `source_warnings_max` gate is asserted by CI; eval
+ report/history are exposed read-only; no API key required for the fast path.
+- **Depends on**: Task 6 (consumes the extended benchmark).
+
+## Cross-cutting acceptance (after every commit)
+- `bun test` green, `bun run typecheck` clean, `bun run lint` clean.
+- `bun run scripts/sync-version.ts --check` (version sync) unaffected during
+ implementation; the single version bump happens at release (Phase 9).
+- No natural-language word lists introduced anywhere.
+- `brain_search` output byte-identical to `main` for a call that sets none of the
+ new flags.
diff --git a/docs/brainstorm/search-recall-quality/variants.md b/docs/brainstorm/search-recall-quality/variants.md
new file mode 100644
index 00000000..f8c3f578
--- /dev/null
+++ b/docs/brainstorm/search-recall-quality/variants.md
@@ -0,0 +1,114 @@
+# Search & Recall Quality - brainstorm variants (audit trail)
+
+Phase 0 of the feature-release-playbook. Primary consultant: `claude -p`
+(exit 0, three parseable variants - no fallback to Codex required).
+
+Prompt: `cli-output/prompt.md`. Raw consultant output: `cli-output/claude.md`.
+
+## Variants (verbatim from the consultant)
+
+### Variant 1: Concern-located two-layer split
+- **Approach**: Land each unit where its concern already lives rather than
+ introducing shared scaffolding. Read-time projections that need only data
+ already on `SearchOutcome` (score_breakdown serialization, hybrid-degrade
+ warning surfacing) are shaped at the `src/mcp/search-tools.ts` surface;
+ ranking-behavior changes (threshold+rerank, reinforce re-rank) extend the
+ pure ranker behind `ResolvedRecallConfig` kill switches; trust metadata
+ reuses `recall-hint.ts` as a sibling read-time deriver in core; eval/benchmark
+ units extend `benchmark.ts` and add a standalone MCP `run_eval` tool.
+- **Trade-offs**:
+ - Pro: smallest conceptual footprint; no new abstraction; each commit is a
+ tight diff against the file that owns the concern.
+ - Pro: strongest match to "pure core stays side-effect free; surfaces opt into
+ telemetry".
+ - Con: score_breakdown projection at the MCP layer can only see fields
+ `BrainSearchResult` already carries; `entityBoost`/`activationBoost`/
+ `coAccessBoost`/`sessionFocus` are computed inside the ranker and would need
+ to be threaded out anyway, so the "thin surface" promise leaks back into core.
+ - Con: read-time enrichment logic ends up split across two layers, making the
+ read-time contract harder to reason about as a whole.
+- **Complexity**: medium
+- **Risk**: low
+
+### Variant 2: Unified read-time enrichment module in core
+- **Approach**: Add one pure core module (`src/core/search/enrich.ts`) as an
+ explicit sibling to `recall-hint.ts` that owns every read-time projection -
+ `score_breakdown`, inline trust fields, and hybrid-degrade detection - as pure
+ functions over a ranker output that carries all numeric components.
+ `SearchOptions` gains a single additive `enrich`/`explain`/`threshold`/
+ `reinforce` opt-in bag; threshold+rerank and reinforce re-rank extend the
+ ranker/MMR path behind `ResolvedRecallConfig` switches; the MCP surface only
+ serializes and owns the side-effecting reinforce ledger and the `run_eval`
+ exposure; benchmark gains answer-containment@k in place.
+- **Trade-offs**:
+ - Pro: all read-time, never-stored projections live in one cohesive, unit-tested
+ core module mirroring the proven `recall-hint` pattern, so the
+ "computed-at-read-time, never stored" invariant is enforced in one place.
+ - Pro: ranker carries every score component once, so score_breakdown is a
+ trivial structural projection with no leak back-and-forth; byte-identical
+ legacy output is easy to assert because every field is gated by an explicit
+ flag default.
+ - Pro: keeps the side-effect boundary clean - pure enrichment in core,
+ ledger/eval recording at the surface.
+ - Con: requires widening the ranker's output type to surface all components, a
+ slightly larger up-front type change touching `types.ts`.
+ - Con: one module owning three distinct projections risks becoming a mild
+ grab-bag if not internally sectioned.
+- **Complexity**: medium
+- **Risk**: low
+
+### Variant 3: Composable gated enrichment pipeline with registry
+- **Approach**: Introduce a generalized read-time enrichment pipeline where each
+ of the seven units registers as a gated transform stage over the ranked
+ outcome, dispatched by a flag/kill-switch map, plus a parallel pluggable
+ structure for eval dimensions as registered metrics. The MCP surface and
+ benchmark both consume the same registry so new signals/metrics drop in
+ uniformly.
+- **Trade-offs**:
+ - Pro: maximally uniform and future-proof - an eighth signal or metric is a
+ one-stage registration; ordering/composition with the MMR pass is explicit.
+ - Pro: forces a single declarative answer to "which flag gates which behavior".
+ - Con: heaviest abstraction for a fixed set of seven units; the registry
+ indirection obscures otherwise-trivial projections and works against the
+ repo's concrete, per-signal, one-file conventions.
+ - Con: a shared pipeline core makes the seven "atomic, independently TDD'd
+ commits" harder to land cleanly - early commits must build scaffolding the
+ later ones use, coupling the commit sequence.
+ - Con: higher risk of perturbing byte-identical legacy output, since all
+ results now flow through a new common path even when no flag is set.
+- **Complexity**: large
+- **Risk**: medium-high
+
+### Recommended (consultant): Variant 2
+**Rationale**: It follows the established `recall-hint.ts` precedent the task
+names as the model pattern, keeping all read-time/never-stored projections
+cohesive and pure while leaving ledger writes and eval exposure at the MCP edge.
+It avoids Variant 1's score-component leak between layers and Variant 3's
+registry over-engineering, which would couple the seven atomic commits and
+threaten the byte-identical-when-flags-off guarantee.
+
+## Orchestrator decision: adopt Variant 2, with one refinement
+
+I agree with the consultant. Variant 2 is the right fit: it reuses the proven
+read-time `recall-hint.ts` pattern, concentrates the never-stored invariant in
+one place, and keeps side effects (the reinforce ledger, eval recording) at the
+MCP edge. Variant 3's registry is over-engineering for a fixed set of seven
+units and threatens the byte-identical-off guarantee; Variant 1 leaks score
+components across layers.
+
+Refinement (project-context override the consultant lacked): rather than have
+`enrich.ts` re-parse the ranker's `reasons[]` strings to recover
+`entityBoost`/`activationBoost`/`coAccessBoost`, the ranker emits a structured
+per-layer breakdown object on `BrainSearchResult` as a first-class, always-present
+field. This:
+- removes the existing string-parse crutch in `feedback.ts`
+ (`contributionsFromResult` parses `entity_match:` back out of a reason string)
+ by letting it read the structured field instead - a net DRY win;
+- makes `score_breakdown` a pure structural projection with no re-parsing;
+- is additive to the core type and does NOT change MCP output unless the
+ `explain` flag is set (the MCP serializer picks fields explicitly), so the
+ byte-identical-off guarantee holds at the surface.
+
+The breakdown field is core-internal data; the MCP `score_breakdown` is its
+gated projection. Trust metadata and hybrid-degrade detection live in `enrich.ts`
+as pure read-time functions, exactly per Variant 2.
diff --git a/src/cli/brain/verbs/benchmark.ts b/src/cli/brain/verbs/benchmark.ts
index 5403f86a..0b959b08 100644
--- a/src/cli/brain/verbs/benchmark.ts
+++ b/src/cli/brain/verbs/benchmark.ts
@@ -70,6 +70,11 @@ export async function cmdBrainBenchmark(argv: string[]): Promise<number> {
expand: report.expand,
hit_at_k: report.hitAtK,
mrr: report.mrr,
+ answer_queries: report.answerQueries,
+ answer_containment_at_k: report.answerContainmentAtK,
+ source_utilization_at_k: report.sourceUtilizationAtK,
+ citation_depth: report.citationDepth,
+ source_warnings: report.sourceWarnings,
misses: report.perQuery.filter((q) => !q.hit).map((q) => q.id),
},
});
@@ -83,12 +88,20 @@ export async function cmdBrainBenchmark(argv: string[]): Promise<number> {
expand: report.expand,
hit_at_k: report.hitAtK,
mrr: report.mrr,
+ answer_queries: report.answerQueries,
+ answer_containment_at_k: report.answerContainmentAtK,
+ source_utilization_at_k: report.sourceUtilizationAtK,
+ citation_depth: report.citationDepth,
+ source_warnings: report.sourceWarnings,
per_query: report.perQuery,
});
} else {
ok(
`benchmark: ${report.total} queries, hit@${report.k} ${report.hitAtK.toFixed(3)}, ` +
- `MRR ${report.mrr.toFixed(3)}${report.expand ? " (expanded)" : ""}`,
+ `MRR ${report.mrr.toFixed(3)}, ` +
+ `answer@${report.k} ${report.answerContainmentAtK.toFixed(3)} (${report.answerQueries}), ` +
+ `src-util ${report.sourceUtilizationAtK.toFixed(3)}, warnings ${report.sourceWarnings}` +
+ `${report.expand ? " (expanded)" : ""}`,
);
for (const q of report.perQuery) {
ok(` ${q.hit ? "hit " : "MISS"} ${q.id} rank=${q.rank ?? "-"}`);
diff --git a/src/core/search/benchmark.ts b/src/core/search/benchmark.ts
index 45fd161f..55bc286c 100644
--- a/src/core/search/benchmark.ts
+++ b/src/core/search/benchmark.ts
@@ -33,6 +33,13 @@ export interface RecallBenchmarkQuery {
readonly expected: ReadonlyArray<string>;
/** Per-query rank depth override. */
readonly k?: number;
+ /**
+ * Optional expected answer text. When present, the query also scores
+ * answer-containment@k: whether this substring appears (folded) in the
+ * retrieved content within the top k. Absent leaves the query measuring
+ * hit@k / MRR only, so existing datasets stay valid.
+ */
+ readonly answer?: string;
}
export interface RecallBenchmarkDataset {
@@ -53,6 +60,15 @@ export interface RecallBenchmarkQueryResult {
/** 1-based rank of the first expected path, null on a miss. */
readonly rank: number | null;
readonly reciprocalRank: number;
+ /**
+ * Whether the declared answer appeared (folded) in the retrieved
+ * content within the top k. `null` when the query declares no answer.
+ */
+ readonly answerContained: boolean | null;
+ /** Expected paths that surfaced in the top k for this query. */
+ readonly expectedFound: number;
+ /** Total expected paths declared for this query. */
+ readonly expectedTotal: number;
}
export interface RecallBenchmarkReport {
@@ -63,9 +79,45 @@ export interface RecallBenchmarkReport {
readonly hitAtK: number;
/** Mean reciprocal rank over all queries (misses contribute 0). */
readonly mrr: number;
+ /**
+ * Number of queries that declared an answer (the answer-containment
+ * denominator). 0 when the dataset uses no answers.
+ */
+ readonly answerQueries: number;
+ /**
+ * Fraction of answer-bearing queries whose answer appeared in the
+ * top-k content. Vacuously 1 when no query declares an answer, so a CI
+ * floor never fails a dataset that does not use the metric.
+ */
+ readonly answerContainmentAtK: number;
+ /**
+ * Source utilization: fraction of all declared expected paths (summed
+ * across queries) that surfaced in their query's top k. Measures source
+ * coverage rather than per-query hit; equals hitAtK when every query
+ * declares exactly one expected path.
+ */
+ readonly sourceUtilizationAtK: number;
+ /**
+ * Citation depth: mean 1-based rank of the first expected hit over the
+ * queries that hit (lower is better). 0 when no query hit.
+ */
+ readonly citationDepth: number;
+ /**
+ * Source warnings: queries where NO expected path surfaced in the top
+ * k - an expected source the retrieval failed to use. The CI gate caps
+ * this with `source_warnings_max`.
+ */
+ readonly sourceWarnings: number;
readonly perQuery: ReadonlyArray<RecallBenchmarkQueryResult>;
}
+/** Folded substring containment, language-agnostic (no word lists). */
+function answerContainedIn(answer: string, contents: ReadonlyArray<string>): boolean {
+ const needle = answer.replace(/\s+/gu, " ").trim().toLowerCase();
+ if (needle.length === 0) return false;
+ return contents.some((c) => c.replace(/\s+/gu, " ").toLowerCase().includes(needle));
+}
+
/**
* Validate a parsed dataset JSON value. Throws `SearchError`
* (INVALID_INPUT) naming the first offending entry.
@@ -83,7 +135,13 @@ export function parseRecallBenchmarkDataset(raw: unknown): RecallBenchmarkDatase
if (entry === null || typeof entry !== "object" || Array.isArray(entry)) {
throw new SearchError("INVALID_INPUT", `benchmark query #${i} must be an object`);
}
- const q = entry as { id?: unknown; query?: unknown; expected?: unknown; k?: unknown };
+ const q = entry as {
+ id?: unknown;
+ query?: unknown;
+ expected?: unknown;
+ k?: unknown;
+ answer?: unknown;
+ };
if (typeof q.id !== "string" || q.id.trim().length === 0) {
throw new SearchError("INVALID_INPUT", `benchmark query #${i} needs a non-empty string id`);
}
@@ -107,11 +165,18 @@ export function parseRecallBenchmarkDataset(raw: unknown): RecallBenchmarkDatase
if (q.k !== undefined && (!Number.isInteger(q.k) || (q.k as number) < 1)) {
throw new SearchError("INVALID_INPUT", `benchmark query '${q.id}' k must be a positive int`);
}
+ if (q.answer !== undefined && (typeof q.answer !== "string" || q.answer.trim().length === 0)) {
+ throw new SearchError(
+ "INVALID_INPUT",
+ `benchmark query '${q.id}' answer must be a non-empty string when present`,
+ );
+ }
return Object.freeze({
id: q.id,
query: q.query,
expected: Object.freeze([...(q.expected as string[])]) as ReadonlyArray<string>,
...(q.k !== undefined ? { k: q.k as number } : {}),
+ ...(q.answer !== undefined ? { answer: q.answer as string } : {}),
});
});
return Object.freeze({ queries: Object.freeze(parsed) });
@@ -144,31 +209,56 @@ export async function runRecallBenchmark(
...(expand ? { expand: true } : {}),
});
const expected = new Set(q.expected);
+ const topK = outcome.results.slice(0, depth);
+ const topKPaths = new Set(topK.map((r) => r.path));
+ const expectedFound = [...expected].filter((p) => topKPaths.has(p)).length;
let rank: number | null = null;
- for (let i = 0; i < outcome.results.length && i < depth; i++) {
- if (expected.has(outcome.results[i]!.path)) {
+ for (let i = 0; i < topK.length; i++) {
+ if (expected.has(topK[i]!.path)) {
rank = i + 1;
break;
}
}
+ const answerContained =
+ q.answer === undefined
+ ? null
+ : answerContainedIn(
+ q.answer,
+ topK.map((r) => r.content),
+ );
return Object.freeze({
id: q.id,
query: q.query,
hit: rank !== null,
rank,
reciprocalRank: rank === null ? 0 : 1 / rank,
+ answerContained,
+ expectedFound,
+ expectedTotal: expected.size,
});
}),
);
const hits = perQuery.filter((r) => r.hit).length;
const mrr = perQuery.reduce((sum, r) => sum + r.reciprocalRank, 0) / perQuery.length;
+ const answerScored = perQuery.filter((r) => r.answerContained !== null);
+ const answerHits = answerScored.filter((r) => r.answerContained === true).length;
+ const expectedFoundTotal = perQuery.reduce((sum, r) => sum + r.expectedFound, 0);
+ const expectedDeclaredTotal = perQuery.reduce((sum, r) => sum + r.expectedTotal, 0);
+ const hitRanks = perQuery.filter((r) => r.rank !== null).map((r) => r.rank!);
return Object.freeze({
total: perQuery.length,
k,
expand,
hitAtK: hits / perQuery.length,
mrr,
+ answerQueries: answerScored.length,
+ answerContainmentAtK: answerScored.length === 0 ? 1 : answerHits / answerScored.length,
+ sourceUtilizationAtK:
+ expectedDeclaredTotal === 0 ? 1 : expectedFoundTotal / expectedDeclaredTotal,
+ citationDepth:
+ hitRanks.length === 0 ? 0 : hitRanks.reduce((sum, r) => sum + r, 0) / hitRanks.length,
+ sourceWarnings: perQuery.length - hits,
perQuery: Object.freeze(perQuery),
});
}
diff --git a/src/core/search/enrich.ts b/src/core/search/enrich.ts
new file mode 100644
index 00000000..bd420c1b
--- /dev/null
+++ b/src/core/search/enrich.ts
@@ -0,0 +1,142 @@
+/**
+ * Read-time enrichment for brain_search results (Search & Recall Quality
+ * Suite). The sibling of `recall-hint.ts`: every projection here is
+ * computed at read time over an already-ranked result and is NEVER
+ * stored. Pure - no I/O, no clock unless injected.
+ *
+ * Three projections live here as the suite lands:
+ * - `projectScoreBreakdown` - the structured per-layer score components
+ * surfaced under the MCP `explain` flag;
+ * - inline trust metadata (age / superseded / conflict);
+ * - hybrid-degrade detection.
+ *
+ * Language-agnostic by construction: every output is a number, a boolean,
+ * or an identifier already present in the data - no per-locale phrase
+ * table, consistent with the project's single-authoring-language stance.
+ */
+
+import type { BrainSearchResult, ScoreBreakdown, TrustMetadata } from "./types.ts";
+
+const DAY_MS = 24 * 60 * 60 * 1000;
+
+/** Typed relations that mark a hit superseded by a successor. */
+const SUPERSEDED_RELATION = "superseded_by";
+/** Typed relations that mark a hit as declaring a contradiction. */
+const CONFLICT_RELATION = "contradicts";
+
+/**
+ * Project a result's structured score breakdown. A primary ranked result
+ * carries `breakdown` verbatim; a synthetic result (link-traversal
+ * expansion, relation-polarity successor pull-in) carries none, so the
+ * breakdown is derived from the first-class lane/boost fields it does
+ * expose. The non-derivable layers are honestly zero / neutral rather
+ * than guessed - a synthetic hop result genuinely has no entity,
+ * activation, or co-access contribution.
+ */
+export function projectScoreBreakdown(result: BrainSearchResult): ScoreBreakdown {
+ if (result.breakdown !== undefined) return result.breakdown;
+ return Object.freeze({
+ keyword: result.keywordScore,
+ semantic: result.semanticScore,
+ rrf: 0,
+ entity: 0,
+ activation: 0,
+ coAccess: 0,
+ link: result.linkBoost,
+ recency: result.recencyBoost,
+ tier: 1,
+ trend: 1,
+ sessionFocus: 0,
+ });
+}
+
+export interface HybridDegradeInput {
+ /** Did the caller's resolved policy want the semantic lane at all? */
+ readonly wantSemantic: boolean;
+ /** Did the semantic lane actually run and return (vs degrade out)? */
+ readonly semanticAttempted: boolean;
+ /** Number of keyword (FTS5) candidates the query produced. */
+ readonly keywordHitCount: number;
+}
+
+/**
+ * Detect the genuine silent single-lane fallback: the caller wanted
+ * hybrid (semantic + keyword) but the semantic lane did not run, so the
+ * query was served keyword-only without the caller being told. Returns a
+ * single greppable `hybrid_degraded:` warning, or null when retrieval
+ * matched the caller's hybrid intent.
+ *
+ * Scope note: in this engine the keyword (FTS5) lane is always available,
+ * so the realistic silent degrade is the loss of the semantic lane
+ * (missing embeddings, unloaded vec extension, unconfigured key). A query
+ * with simply no keyword match is NOT flagged - that is an empty lexical
+ * result, not a configuration fallback, and flagging it would be
+ * misleading noise. The granular `runSemanticPhase` warnings still
+ * explain WHY the lane dropped; this is the one structural signal a
+ * caller can test for.
+ */
+export function detectHybridDegrade(input: HybridDegradeInput): string | null {
+ if (input.wantSemantic && !input.semanticAttempted && input.keywordHitCount > 0) {
+ return "hybrid_degraded: semantic lane unavailable, served keyword-only";
+ }
+ return null;
+}
+
+export interface DeriveTrustInput {
+ /** Document modification time in unix-ms. */
+ readonly mtimeMs: number;
+ /** Reference time in unix-ms (injected for deterministic tests). */
+ readonly nowMs: number;
+ /**
+ * Typed relations the hit's page declares (the same array surfaced on
+ * the result). `superseded_by` marks the hit superseded; `contradicts`
+ * marks it conflicted. Absent = neither.
+ */
+ readonly relations?: ReadonlyArray<{ readonly relation: string; readonly target: string }>;
+}
+
+/**
+ * Inline per-hit trust metadata (Search & Recall Quality Suite): the
+ * validity signals recall already computes, projected onto a hit so the
+ * agent weights stale or contested memories without a second audit pass.
+ *
+ * Scope: this is note-level validity, derived from the typed relation
+ * edges the search pipeline surfaces (`superseded_by`, `contradicts`) and
+ * the document mtime - NOT the entity/aspect claim-ledger truth fold,
+ * which is a different granularity (facts, not notes) and is not
+ * well-defined per recall hit. Read-time and never stored, like the
+ * recall hint. Language-agnostic: a whole-day count plus two booleans.
+ */
+export function deriveTrust(input: DeriveTrustInput): TrustMetadata {
+ const ageDays = Math.max(0, Math.floor((input.nowMs - input.mtimeMs) / DAY_MS));
+ const relations = input.relations ?? [];
+ return Object.freeze({
+ age_days: ageDays,
+ superseded: relations.some((r) => r.relation === SUPERSEDED_RELATION),
+ conflict: relations.some((r) => r.relation === CONFLICT_RELATION),
+ });
+}
+
+/**
+ * Opt-in relevance rerank (Search & Recall Quality Suite): re-order an
+ * already-ranked, threshold-qualified set by core textual relevance - the
+ * keyword + semantic lane contributions only, ignoring the recency /
+ * usage / link boosts that shaped the primary `score`. This surfaces the
+ * genuinely-most-relevant hit among qualifying results, the
+ * deeper-relevance ordering a threshold query expects. Deterministic and
+ * stable (equal relevance preserves input order); does not mutate the
+ * input array.
+ */
+export function rerankByRelevance(
+ results: ReadonlyArray<BrainSearchResult>,
+): ReadonlyArray<BrainSearchResult> {
+ return results
+ .map((r, i) => ({ r, i }))
+ .toSorted((a, b) => coreRelevance(b.r) - coreRelevance(a.r) || a.i - b.i)
+ .map((x) => x.r);
+}
+
+/** Core textual relevance: the keyword + semantic lane contributions. */
+function coreRelevance(r: BrainSearchResult): number {
+ return r.keywordScore + r.semanticScore;
+}
diff --git a/src/core/search/feedback.ts b/src/core/search/feedback.ts
index b9c24ac4..3c40615a 100644
--- a/src/core/search/feedback.ts
+++ b/src/core/search/feedback.ts
@@ -91,22 +91,17 @@ function round6(x: number): number {
/**
* Read the per-layer contributions off a search result. Keyword,
- * semantic, and recency are first-class fields; the entity layer only
- * surfaces through its `entity_match:` reason, so it is parsed back out.
+ * semantic, and recency are first-class fields; the entity layer is read
+ * from the structured `breakdown` the ranker emits. A result without a
+ * breakdown (a synthetic traversal expansion or relation-polarity
+ * pull-in) genuinely has no entity contribution, so 0 is correct - not a
+ * lossy fallback.
*/
export function contributionsFromResult(result: BrainSearchResult): LayerContributions {
- let entity = 0;
- for (const reason of result.reasons) {
- if (reason.startsWith("entity_match: ")) {
- const parsed = Number(reason.slice("entity_match: ".length));
- if (Number.isFinite(parsed)) entity = parsed;
- break;
- }
- }
return Object.freeze({
keyword: result.keywordScore,
semantic: result.semanticScore,
- entity,
+ entity: result.breakdown?.entity ?? 0,
recency: result.recencyBoost,
});
}
diff --git a/src/core/search/ranker.ts b/src/core/search/ranker.ts
index 5f981c8d..2f3a69df 100644
--- a/src/core/search/ranker.ts
+++ b/src/core/search/ranker.ts
@@ -14,7 +14,7 @@ import { weibullDecay, DEFAULT_RECENCY, type WeibullRecencyOptions } from "./rec
import { scoreSessionFocusTarget, type SearchSessionFocus } from "./session-focus.ts";
import { rrfFuse, DEFAULT_RRF_K, type FusionMode } from "./fusion.ts";
import type { KeywordHit, SemanticHit, HydratedChunk } from "./store.ts";
-import type { BrainSearchResult, WeightProfile } from "./types.ts";
+import type { BrainSearchResult, ScoreBreakdown, WeightProfile } from "./types.ts";
export interface RankerInputs {
readonly keyword: ReadonlyArray<KeywordHit>;
@@ -204,6 +204,41 @@ function buildReasons(parts: {
return Object.freeze(reasons);
}
+/**
+ * Structured sibling of {@link buildReasons} over the same per-layer
+ * values. Every component is a number: zero for an additive layer that
+ * did not fire, 1 for a neutral multiplier. No omission and no
+ * formatting, so callers (the MCP `explain` projection, `feedback.ts`)
+ * read the contributions without re-parsing the reason strings.
+ */
+function buildBreakdown(parts: {
+ keywordScore: number;
+ semanticScore: number;
+ linkBoost: number;
+ recency: number;
+ tierMul: number;
+ entityBoost?: number;
+ activationBoost?: number;
+ coAccessBoost?: number;
+ trendMul?: number;
+ sessionFocus?: number;
+ rrf?: number;
+}): ScoreBreakdown {
+ return Object.freeze({
+ keyword: parts.keywordScore,
+ semantic: parts.semanticScore,
+ rrf: parts.rrf ?? 0,
+ entity: parts.entityBoost ?? 0,
+ activation: parts.activationBoost ?? 0,
+ coAccess: parts.coAccessBoost ?? 0,
+ link: parts.linkBoost,
+ recency: parts.recency,
+ tier: parts.tierMul,
+ trend: parts.trendMul ?? 1,
+ sessionFocus: parts.sessionFocus ?? 0,
+ });
+}
+
export function rankResults(inputs: RankerInputs, opts: RankerOptions): BrainSearchResult[] {
const nowMs = opts.nowMs ?? Date.now();
const semanticEnabled = opts.semanticEnabled !== false;
@@ -406,6 +441,19 @@ export function rankResults(inputs: RankerInputs, opts: RankerOptions): BrainSea
sessionFocus,
rrf: rrfByChunk !== null ? rrf : 0,
}),
+ breakdown: buildBreakdown({
+ keywordScore: c.keywordScore,
+ semanticScore: semanticEnabled ? c.semanticScore : 0,
+ linkBoost,
+ recency,
+ tierMul,
+ entityBoost,
+ activationBoost,
+ coAccessBoost,
+ trendMul,
+ sessionFocus,
+ rrf: rrfByChunk !== null ? rrf : 0,
+ }),
}),
);
}
diff --git a/src/core/search/reinforce.ts b/src/core/search/reinforce.ts
new file mode 100644
index 00000000..32f00a78
--- /dev/null
+++ b/src/core/search/reinforce.ts
@@ -0,0 +1,176 @@
+/**
+ * Self-tuning reinforce ledger (Search & Recall Quality Suite).
+ *
+ * Reinforcement is an EXPLICIT signal: a caller (operator or downstream
+ * agent) naming a memory that proved useful. It is never inferred from a
+ * memory merely being surfaced - surfaced-only frequency carries no
+ * positive signal, by design, so popularity can never snowball.
+ *
+ * Each reinforce event is one small JSON file under
+ * `Brain/search/reinforce/` - the same conflict-free one-file-per-signal
+ * pattern `Brain/inbox/` and the feedback ledger use, so multi-device
+ * vaults never produce sync conflicts on a shared append-only file. The
+ * per-path strength is a PURE FOLD over the event set (count of distinct
+ * events naming the path, normalized and bounded), so it is replayable
+ * and resettable: deleting the derived view loses nothing.
+ *
+ * The boost is bounded ({@link REINFORCE_BOOST_CAP}) and applied before
+ * the top_k cut, so a reinforced memory is lifted into the window without
+ * ever floating an irrelevant chunk.
+ */
+
+import { mkdirSync, readdirSync, readFileSync, rmSync, writeFileSync, existsSync } from "node:fs";
+import { join } from "node:path";
+
+import { fnv1aHex } from "./feedback.ts";
+import type { BrainSearchResult } from "./types.ts";
+
+/** Maximum additive boost a fully-reinforced path receives. */
+export const REINFORCE_BOOST_CAP = 0.05;
+/** Event count at which a path reaches full reinforcement strength. */
+export const REINFORCE_SATURATION = 5;
+
+export interface ReinforceEvent {
+ /** Unix-ms timestamp the reinforcement was recorded. */
+ readonly ts: number;
+ /** Vault-relative paths reinforced in this event. */
+ readonly paths: ReadonlyArray<string>;
+}
+
+export interface RecordReinforceOptions {
+ /** Injected clock for deterministic tests. Defaults to Date.now(). */
+ readonly nowMs?: number;
+}
+
+export function reinforceDir(vault: string): string {
+ return join(vault, "Brain", "search", "reinforce");
+}
+
+/**
+ * Record one reinforce event as its own file. The filename derives from
+ * the timestamp plus a content hash, so recording the identical event
+ * (same timestamp and paths) twice is idempotent. Empty path lists are a
+ * no-op.
+ */
+export function recordReinforce(
+ vault: string,
+ paths: ReadonlyArray<string>,
+ opts: RecordReinforceOptions = {},
+): string | null {
+ const cleaned = [...new Set(paths.filter((p) => p.length > 0))].toSorted();
+ if (cleaned.length === 0) return null;
+ const dir = reinforceDir(vault);
+ mkdirSync(dir, { recursive: true });
+ const event: ReinforceEvent = Object.freeze({ ts: opts.nowMs ?? Date.now(), paths: cleaned });
+ const body = JSON.stringify(event, null, 2) + "\n";
+ const file = join(dir, `${event.ts}-${fnv1aHex(body)}.json`);
+ writeFileSync(file, body);
+ return file;
+}
+
+/** Load all reinforce events, sorted by timestamp (stable). */
+export function loadReinforceEvents(vault: string): ReinforceEvent[] {
+ let files: string[];
+ try {
+ files = readdirSync(reinforceDir(vault)).filter((f) => f.endsWith(".json"));
+ } catch {
+ return [];
+ }
+ const out: ReinforceEvent[] = [];
+ for (const f of files.toSorted()) {
+ try {
+ const parsed = JSON.parse(
+ readFileSync(join(reinforceDir(vault), f), "utf8"),
+ ) as ReinforceEvent;
+ if (
+ typeof parsed.ts === "number" &&
+ Array.isArray(parsed.paths) &&
+ parsed.paths.every((p) => typeof p === "string")
+ ) {
+ out.push(parsed);
+ }
+ } catch {
+ // One malformed file never breaks the fold.
+ }
+ }
+ out.sort((a, b) => a.ts - b.ts);
+ return out;
+}
+
+/**
+ * Pure fold: per-path strength in [0, 1], the count of distinct events
+ * naming the path normalized by {@link REINFORCE_SATURATION} and capped
+ * at 1. Order-insensitive by construction.
+ */
+export function loadReinforceStrengths(vault: string): Map<string, number> {
+ const counts = new Map<string, number>();
+ for (const event of loadReinforceEvents(vault)) {
+ for (const path of new Set(event.paths)) {
+ counts.set(path, (counts.get(path) ?? 0) + 1);
+ }
+ }
+ const strengths = new Map<string, number>();
+ for (const [path, count] of counts) {
+ strengths.set(path, Math.min(1, count / REINFORCE_SATURATION));
+ }
+ return strengths;
+}
+
+/** Delete the reinforce ledger. The fold replays as empty afterwards. */
+export function resetReinforce(vault: string): void {
+ rmSync(reinforceDir(vault), { recursive: true, force: true });
+}
+
+/**
+ * Stable fingerprint of the ledger state for the query-cache key: "off"
+ * when no ledger exists, else a hash of the sorted event filenames (the
+ * filenames already encode ts + content hash, so they change iff the
+ * ledger does).
+ */
+export function reinforceFingerprint(vault: string): string {
+ const dir = reinforceDir(vault);
+ if (!existsSync(dir)) return "off";
+ try {
+ const files = readdirSync(dir)
+ .filter((f) => f.endsWith(".json"))
+ .toSorted();
+ if (files.length === 0) return "off";
+ return fnv1aHex(files.join("|"));
+ } catch {
+ return "off";
+ }
+}
+
+function fmt(x: number): string {
+ return x.toFixed(3);
+}
+
+/**
+ * Apply the bounded reinforcement boost to a ranked pool and re-sort.
+ * A result whose path carries ledger strength gets `CAP * strength` added
+ * to its score (clamped to 1) and a `reinforce:` reason; everything else
+ * is untouched. Pure: returns a new array, never mutates the input. The
+ * tie-break matches the ranker family (score desc, keywordScore desc,
+ * chunkId asc) so the ordering stays deterministic.
+ */
+export function applyReinforceBoost(
+ results: ReadonlyArray<BrainSearchResult>,
+ strengthByPath: ReadonlyMap<string, number>,
+): BrainSearchResult[] {
+ if (strengthByPath.size === 0) return results.slice();
+ const boosted = results.map((r) => {
+ const strength = strengthByPath.get(r.path);
+ if (strength === undefined || strength <= 0) return r;
+ const boost = REINFORCE_BOOST_CAP * Math.min(1, strength);
+ return Object.freeze({
+ ...r,
+ score: Math.min(1, r.score + boost),
+ reasons: Object.freeze([...r.reasons, `reinforce: +${fmt(boost)}`]),
+ });
+ });
+ return boosted.toSorted((a, b) => {
+ if (b.score !== a.score) return b.score - a.score;
+ if (b.keywordScore !== a.keywordScore) return b.keywordScore - a.keywordScore;
+ return a.chunkId - b.chunkId;
+ });
+}
diff --git a/src/core/search/search.ts b/src/core/search/search.ts
index 7931abe4..a320976a 100644
--- a/src/core/search/search.ts
+++ b/src/core/search/search.ts
@@ -49,6 +49,8 @@ import { deriveExpansionTerms, tokenizeForExpansion, DEFAULT_EXPANSION } from ".
import { filterByProperties } from "./property-filter.ts";
import { applyRelationPolarity } from "./relation-polarity.ts";
import { rankResults } from "./ranker.ts";
+import { deriveTrust, detectHybridDegrade, rerankByRelevance } from "./enrich.ts";
+import { applyReinforceBoost, loadReinforceStrengths, reinforceFingerprint } from "./reinforce.ts";
import { readActiveSessionFocus } from "./session-focus.ts";
import { applyTemporalBridge } from "./temporal-bridge.ts";
import { resolveTimeRange } from "./time-range.ts";
@@ -216,6 +218,9 @@ export async function search(
if (tuned !== null) config = applyTunedParameters(config, tuned);
const expandActive = opts.expand ?? (tuned !== null && tuned.expansion);
const limit = Math.max(1, Math.min(100, opts.limit ?? 10));
+ if (opts.threshold !== undefined && (!Number.isFinite(opts.threshold) || opts.threshold < 0)) {
+ throw new SearchError("INVALID_INPUT", "threshold must be a finite number >= 0");
+ }
const pathPrefix = assertSafePathPrefix(opts.pathPrefix);
const policy = resolveSemanticPolicy(config, opts);
const warnings: string[] = [];
@@ -297,10 +302,15 @@ export async function search(
? activationStateFingerprint(config.vault)
: "off";
const tuneFp = tuned !== null ? JSON.stringify(tuned) : "off";
+ // The reinforce ledger changes results only when the caller opted
+ // in (Search & Recall Quality Suite); its fingerprint joins the
+ // key just for those calls so an unrelated ledger write never
+ // invalidates ordinary searches.
+ const reinfFp = opts.reinforce !== undefined ? reinforceFingerprint(config.vault) : "off";
cacheKey = buildCacheKey(
keyOpts,
basePlan.planHash,
- `${configFingerprint(config)}|lw:${lwFp}|act:${actFp}|tune:${tuneFp}`,
+ `${configFingerprint(config)}|lw:${lwFp}|act:${actFp}|tune:${tuneFp}|reinf:${reinfFp}`,
);
const hit = getCachedOutcome(store, cacheKey, generation, ttlMs, Date.now());
if (hit) return hit;
@@ -381,6 +391,20 @@ export async function search(
for (const w of semOutcome.warnings) warnings.push(w);
}
+ // Hybrid-degrade signal (Search & Recall Quality Suite): one
+ // structural warning when the caller wanted the semantic lane but it
+ // did not run, so the query was served keyword-only. The granular
+ // runSemanticPhase warnings above explain WHY; this is the single
+ // greppable flag a caller can test for.
+ {
+ const degrade = detectHybridDegrade({
+ wantSemantic: policy.wantSemantic,
+ semanticAttempted,
+ keywordHitCount: kwHits.length,
+ });
+ if (degrade !== null) warnings.push(degrade);
+ }
+
// Hydrate.
const allChunkIds = new Set<number>();
for (const h of kwHits) allChunkIds.add(h.chunkId);
@@ -626,6 +650,13 @@ export async function search(
const hasStructuredExclusions = (structured?.lex.exclude.length ?? 0) > 0;
const mmrLambda = opts.mmrLambda ?? config.recall.mmrLambda;
const mmrActive = mmrLambda < 1;
+ // Relevance floor + rerank (Search & Recall Quality Suite). The floor
+ // drops sub-threshold candidates before the diversity rerank so a
+ // query with no sufficiently relevant memory returns no match; the
+ // rerank re-orders the qualified set by core textual relevance. Both
+ // off by default keep results byte-identical.
+ const scoreFloor = opts.threshold ?? 0;
+ const rerankActive = opts.rerank === true;
const maxHops = opts.maxHops ?? config.recall.maxHops;
const traversalActive = maxHops > 0;
const baseRankLimit =
@@ -705,10 +736,22 @@ export async function search(
});
}
}
+ // Relevance floor (Search & Recall Quality Suite): drop
+ // sub-threshold candidates BEFORE diversity so the rerank works over
+ // the qualified set only. No-op when the floor is 0.
+ if (scoreFloor > 0) {
+ ranked = ranked.filter((r) => r.score >= scoreFloor);
+ }
// Diversity rerank (v0.13.0). No-op when lambda >= 1 or < 2 results.
if (mmrActive) {
ranked = mmrRerank(ranked, { lambda: mmrLambda });
}
+ // Relevance rerank (Search & Recall Quality Suite): re-order the
+ // qualified set by core textual relevance, a deeper-relevance second
+ // pass. Opt-in; replaces the diversity ordering when requested.
+ if (rerankActive) {
+ ranked = rerankByRelevance(ranked).slice();
+ }
// Optional post-rank property filter (v0.10.17). Reads each
// result's source frontmatter and drops rows whose scalars do not
// match the requested key/value pairs, then visibility scoping (v3)
@@ -746,7 +789,16 @@ export async function search(
const polarized = config.recall.relationPolarityEnabled
? applyRelationPolarityPhase(store, excluded, opts.includeSuperseded === true)
: excluded;
- const sliced = polarized.slice(0, limit);
+ // Self-tuning reinforce (Search & Recall Quality Suite): opt-in. When
+ // the caller passes a reinforce set, the persisted ledger lifts
+ // proven-useful memories by a bounded boost BEFORE the top_k cut, so
+ // a reinforced hit can enter the window. Absent leaves the pool
+ // untouched; an empty ledger is a no-op either way.
+ const reinforced =
+ opts.reinforce !== undefined
+ ? applyReinforceBoost(polarized, loadReinforceStrengths(config.vault))
+ : polarized;
+ const sliced = reinforced.slice(0, limit);
// Explainability: when learned weights affected this ranking, every
// surfaced result says so (acceptance: "search explanations show
// when learned weights affected a result").
@@ -843,11 +895,18 @@ export async function search(
}
}
+ // Inline trust metadata (Search & Recall Quality Suite): opt-in,
+ // computed at read time from the document mtime and the surfaced
+ // typed relations, never stored. Off by default keeps the result
+ // shape byte-identical.
+ const resultsOut =
+ opts.trust === true ? attachTrustMetadata(config.vault, finalResults) : finalResults;
+
return finalize(
Object.freeze({
- results: Object.freeze(finalResults),
+ results: Object.freeze(resultsOut),
warnings: Object.freeze(warnings),
- total: finalResults.length,
+ total: resultsOut.length,
...(evidencePack !== undefined ? { evidencePack } : {}),
...(secondPass !== undefined ? { secondPass } : {}),
}),
@@ -1113,6 +1172,33 @@ function applyPropertyFilter(
return filterByProperties(ranked, filters, reader);
}
+/**
+ * Stamp inline trust metadata (Search & Recall Quality Suite) onto each
+ * result: age from the document mtime, plus the superseded / conflict
+ * flags from the typed relations the result already carries. Read-time
+ * and never stored. One `statSync` per surfaced result (≀ limit); a path
+ * that cannot be stat'd is left without trust rather than reporting a
+ * bogus age.
+ */
+function attachTrustMetadata(
+ vault: string,
+ results: ReadonlyArray<BrainSearchResult>,
+): ReadonlyArray<BrainSearchResult> {
+ const nowMs = Date.now();
+ return results.map((r) => {
+ let mtimeMs: number;
+ try {
+ mtimeMs = statSync(join(vault, r.path)).mtimeMs;
+ } catch {
+ return r;
+ }
+ return Object.freeze({
+ ...r,
+ trust: deriveTrust({ mtimeMs, nowMs, ...(r.relations ? { relations: r.relations } : {}) }),
+ });
+ });
+}
+
function applyVisibilityScope(
ranked: ReadonlyArray<BrainSearchResult>,
scope: ReadonlySet<string>,
diff --git a/src/core/search/types.ts b/src/core/search/types.ts
index 35b6ca09..964fe408 100644
--- a/src/core/search/types.ts
+++ b/src/core/search/types.ts
@@ -36,6 +36,45 @@ export class SearchError extends Error {
}
}
+/**
+ * Structured per-layer score components (Search & Recall Quality Suite).
+ * The numeric sibling of `reasons[]`: where `reasons` formats only the
+ * layers that fired as strings, `breakdown` carries every component of
+ * the final score as a number, zero for a layer that did not fire and 1
+ * for a neutral multiplier. Additive layers (keyword, semantic, rrf,
+ * entity, activation, coAccess, link, recency, sessionFocus) are the raw
+ * contributions; `tier` and `trend` are the relevance-portion multipliers
+ * (1.0 = neutral). The ranker emits it for every primary result; the MCP
+ * `explain` projection and `feedback.ts` read it directly instead of
+ * re-parsing reason strings.
+ */
+export interface ScoreBreakdown {
+ readonly keyword: number;
+ readonly semantic: number;
+ readonly rrf: number;
+ readonly entity: number;
+ readonly activation: number;
+ readonly coAccess: number;
+ readonly link: number;
+ readonly recency: number;
+ readonly tier: number;
+ readonly trend: number;
+ readonly sessionFocus: number;
+}
+
+/**
+ * Inline per-hit trust metadata (Search & Recall Quality Suite). Computed
+ * at read time, never stored. `age_days` is the whole-day distance from
+ * the document mtime; `superseded` / `conflict` are derived from the
+ * typed relation edges the recall pipeline surfaces (`superseded_by` /
+ * `contradicts`). Present on a result only when the caller set `trust`.
+ */
+export interface TrustMetadata {
+ readonly age_days: number;
+ readonly superseded: boolean;
+ readonly conflict: boolean;
+}
+
export interface BrainSearchResult {
readonly documentId: number;
readonly chunkId: number;
@@ -68,6 +107,22 @@ export interface BrainSearchResult {
readonly relation: string;
readonly target: string;
}>;
+ /**
+ * Structured per-layer score components (Search & Recall Quality
+ * Suite). Always present on a primary ranked result; absent on
+ * synthetic results (link-traversal expansions, relation-polarity
+ * successor pull-ins) whose score is not a per-layer sum - the
+ * `explain` projection derives a faithful breakdown from the
+ * first-class lane/boost fields for those. Never serialized to the MCP
+ * output unless the caller sets `explain`.
+ */
+ readonly breakdown?: ScoreBreakdown;
+ /**
+ * Inline trust metadata (Search & Recall Quality Suite). Present only
+ * when the caller set `trust`; computed at read time from the document
+ * mtime and the surfaced typed relations, never stored.
+ */
+ readonly trust?: TrustMetadata;
/**
* Kind-namespaced origin label (Workspace Insight Suite, cross-vault
* search): "local", "profile/<name>", or "source/<alias>". Only set
@@ -297,6 +352,40 @@ export interface SearchOptions {
*/
readonly since?: string;
readonly until?: string;
+ /**
+ * Inline trust metadata (Search & Recall Quality Suite). When true,
+ * each surfaced result carries a computed-at-read-time `trust` object
+ * (age in days, superseded, conflict) derived from the document mtime
+ * and the surfaced typed relations. Off by default; absent leaves the
+ * result shape byte-identical.
+ */
+ readonly trust?: boolean;
+ /**
+ * Relevance floor (Search & Recall Quality Suite). When > 0, results
+ * whose final normalized score is below this value are dropped before
+ * the diversity rerank, so a query with no sufficiently relevant memory
+ * returns no match instead of weak noise. Absent / 0 disables the
+ * filter and keeps results byte-identical. Applied against the clamped
+ * [0, 1] final score, so it is meaningful in both linear and rrf
+ * fusion.
+ */
+ readonly threshold?: number;
+ /**
+ * Relevance rerank (Search & Recall Quality Suite). When true, the
+ * threshold-qualified candidates are re-ordered by core textual
+ * relevance (keyword + semantic lanes) before the final slice - a
+ * deeper-relevance second pass. Off by default; absent leaves ordering
+ * unchanged.
+ */
+ readonly rerank?: boolean;
+ /**
+ * Self-tuning reinforce (Search & Recall Quality Suite). When present
+ * (even empty), the persisted reinforce ledger lifts proven-useful
+ * memories by a bounded boost before the top_k cut. A non-empty list is
+ * also recorded to the ledger by the calling surface. Absent leaves
+ * ranking byte-identical; surfaced-only frequency never boosts.
+ */
+ readonly reinforce?: ReadonlyArray<string>;
/**
* Self-healing index policy (Workspace Insight Suite). Default true:
* a missing or schema-stale index is rebuilt once and the search
diff --git a/src/mcp/search-tools.ts b/src/mcp/search-tools.ts
index 9c4175a5..70242b73 100644
--- a/src/mcp/search-tools.ts
+++ b/src/mcp/search-tools.ts
@@ -28,6 +28,9 @@ import type { ServerContext, ToolDefinition } from "./tools.ts";
import { coerceBoolOptional, coerceStr, coerceStringOptional } from "./coerce.ts";
import { MCP_PREVIEW_BUDGET } from "./preview-budget.ts";
import { deriveRecallHint } from "../core/search/recall-hint.ts";
+import { projectScoreBreakdown } from "../core/search/enrich.ts";
+import { recordReinforce } from "../core/search/reinforce.ts";
+import { parseRecallBenchmarkDataset, runRecallBenchmark } from "../core/search/benchmark.ts";
import { emitRecallTelemetry } from "../core/brain/recall-telemetry.ts";
import { emitGateTelemetry } from "../core/brain/gate-telemetry.ts";
import { emitGatedTelemetry } from "../core/brain/continuity/emit.ts";
@@ -70,6 +73,35 @@ const SEARCH_INPUT_SCHEMA: Record<string, unknown> = {
limit: { type: "integer", minimum: 1, maximum: MCP_LIMIT_MAX },
semantic: { type: "boolean" },
keyword_only: { type: "boolean" },
+ explain: {
+ type: "boolean",
+ description:
+ "Include a structured score_breakdown (per-layer numeric components) on each result. Default false.",
+ },
+ trust: {
+ type: "boolean",
+ description:
+ "Stamp each result with inline trust metadata (age_days, superseded, conflict), computed at read time. Default false.",
+ },
+ threshold: {
+ type: "number",
+ minimum: 0,
+ maximum: 1,
+ description:
+ "Relevance floor in [0,1] on the final score; drops weaker hits so an irrelevant query returns no match. Default 0 (disabled).",
+ },
+ rerank: {
+ type: "boolean",
+ description:
+ "Re-order the threshold-qualified results by core textual relevance (keyword + semantic). Default false.",
+ },
+ reinforce: {
+ type: "array",
+ maxItems: 50,
+ items: { type: "string", minLength: 1, maxLength: 512 },
+ description:
+ "Paths proven useful: recorded to the reinforce ledger and lifted (bounded) before the top_k cut. Default absent.",
+ },
record_access: {
type: "boolean",
description:
@@ -132,6 +164,22 @@ const SEARCH_OUTPUT_SCHEMA: NonNullable<ToolDefinition["outputSchema"]> = {
endLine: { type: "integer" },
searchType: { type: "string" },
reasons: { type: "array", items: { type: "string" } },
+ score_breakdown: {
+ type: "object",
+ properties: {
+ keyword: { type: "number" },
+ semantic: { type: "number" },
+ rrf: { type: "number" },
+ entity: { type: "number" },
+ activation: { type: "number" },
+ coAccess: { type: "number" },
+ link: { type: "number" },
+ recency: { type: "number" },
+ tier: { type: "number" },
+ trend: { type: "number" },
+ sessionFocus: { type: "number" },
+ },
+ },
origin: { type: "string" },
why_retrieved: { type: "array", items: { type: "string" } },
relations: {
@@ -145,6 +193,14 @@ const SEARCH_OUTPUT_SCHEMA: NonNullable<ToolDefinition["outputSchema"]> = {
},
},
},
+ trust: {
+ type: "object",
+ properties: {
+ age_days: { type: "integer" },
+ superseded: { type: "boolean" },
+ conflict: { type: "boolean" },
+ },
+ },
},
},
},
@@ -233,6 +289,21 @@ function parseVisibilityArgument(raw: unknown): string[] | undefined {
return out;
}
+function parseReinforceArgument(raw: unknown): string[] | undefined {
+ if (raw === undefined || raw === null) return undefined;
+ if (!Array.isArray(raw)) {
+ throw new MCPError(INVALID_PARAMS, "argument 'reinforce' must be an array of strings");
+ }
+ const out: string[] = [];
+ for (const item of raw) {
+ if (typeof item !== "string") {
+ throw new MCPError(INVALID_PARAMS, "argument 'reinforce' must contain only strings");
+ }
+ if (item.length > 0) out.push(item);
+ }
+ return out;
+}
+
function truncateContent(c: string, max: number): string {
if (c.length <= max) return c;
return c.slice(0, max - 1) + "…";
@@ -287,6 +358,17 @@ async function toolBrainSearch(
const semantic = coerceBoolOptional(args, "semantic");
const keywordOnly = coerceBoolOptional(args, "keyword_only") ?? false;
+ const explain = coerceBoolOptional(args, "explain") ?? false;
+ const trust = coerceBoolOptional(args, "trust") ?? false;
+ const rerank = coerceBoolOptional(args, "rerank") ?? false;
+ let threshold: number | undefined;
+ if ("threshold" in args && args["threshold"] !== undefined && args["threshold"] !== null) {
+ const raw = args["threshold"];
+ if (typeof raw !== "number" || !Number.isFinite(raw) || raw < 0 || raw > 1) {
+ throw new MCPError(INVALID_PARAMS, "argument 'threshold' must be a number between 0 and 1");
+ }
+ threshold = raw;
+ }
const globalSearch = coerceBoolOptional(args, "global") ?? false;
const pathPrefix = coerceStringOptional(args, "path_prefix", 256);
const evidencePack = coerceBoolOptional(args, "evidence_pack") ?? false;
@@ -315,12 +397,26 @@ async function toolBrainSearch(
const focusSession = coerceStringOptional(args, "focus_session", 128);
const properties = parsePropertiesArgument(args["properties"]);
const visibility = parseVisibilityArgument(args["visibility"]);
+ const reinforce = parseReinforceArgument(args["reinforce"]);
const config = resolveSearchConfig({
vault: ctx.vault,
configPath: ctx.configPath ?? undefined,
});
+ // Self-tuning reinforce (Search & Recall Quality Suite): the ledger
+ // write is the surface's side effect, recorded BEFORE the query so the
+ // just-named paths participate in this query's bounded boost. The pure
+ // re-rank lives in core. Best-effort: a failed write never breaks the
+ // search.
+ if (reinforce !== undefined && reinforce.length > 0) {
+ try {
+ recordReinforce(ctx.vault, reinforce);
+ } catch {
+ // Ledger persistence is best-effort.
+ }
+ }
+
let outcome: SearchOutcome;
const startedAtMs = Date.now();
const searchOpts = {
@@ -336,6 +432,10 @@ async function toolBrainSearch(
...(focusSession !== undefined ? { focusSession } : {}),
...(evidencePack ? { evidencePack: true } : {}),
...(includeSuperseded ? { includeSuperseded: true } : {}),
+ ...(trust ? { trust: true } : {}),
+ ...(threshold !== undefined ? { threshold } : {}),
+ ...(rerank ? { rerank: true } : {}),
+ ...(reinforce !== undefined ? { reinforce } : {}),
...(since !== undefined ? { since } : {}),
...(until !== undefined ? { until } : {}),
// Access recording (Time-Aware Recall & Activation Suite): the MCP
@@ -418,6 +518,8 @@ async function toolBrainSearch(
endLine: r.endLine,
searchType: r.searchType,
reasons: r.reasons,
+ ...(explain ? { score_breakdown: projectScoreBreakdown(r) } : {}),
+ ...(r.trust !== undefined ? { trust: r.trust } : {}),
...(r.origin !== undefined ? { origin: r.origin } : {}),
...(outcome.evidencePack ? { why_retrieved: r.reasons } : {}),
...(r.relations && r.relations.length > 0 ? { relations: r.relations } : {}),
@@ -576,6 +678,155 @@ async function toolBrainRecallFeedback(
};
}
+const EVAL_INPUT_SCHEMA: Record<string, unknown> = {
+ type: "object",
+ properties: {
+ dataset: {
+ type: "object",
+ description:
+ "Eval dataset: { queries: [{ id, query, expected[], k?, answer? }] }. Scored against the active vault.",
+ properties: {
+ queries: {
+ type: "array",
+ minItems: 1,
+ items: {
+ type: "object",
+ required: ["id", "query", "expected"],
+ properties: {
+ id: { type: "string", minLength: 1 },
+ query: { type: "string", minLength: 1 },
+ expected: { type: "array", minItems: 1, items: { type: "string", minLength: 1 } },
+ k: { type: "integer", minimum: 1, maximum: MCP_LIMIT_MAX },
+ answer: { type: "string", minLength: 1 },
+ },
+ },
+ },
+ },
+ required: ["queries"],
+ },
+ k: { type: "integer", minimum: 1, maximum: MCP_LIMIT_MAX },
+ expand: { type: "boolean" },
+ },
+ required: ["dataset"],
+ additionalProperties: false,
+};
+
+const EVAL_OUTPUT_SCHEMA: NonNullable<ToolDefinition["outputSchema"]> = {
+ type: "object",
+ required: [
+ "total",
+ "k",
+ "hit_at_k",
+ "mrr",
+ "answer_queries",
+ "answer_containment_at_k",
+ "source_utilization_at_k",
+ "citation_depth",
+ "source_warnings",
+ ],
+ properties: {
+ total: { type: "integer" },
+ k: { type: "integer" },
+ expand: { type: "boolean" },
+ hit_at_k: { type: "number" },
+ mrr: { type: "number" },
+ answer_queries: { type: "integer" },
+ answer_containment_at_k: { type: "number" },
+ source_utilization_at_k: { type: "number" },
+ citation_depth: { type: "number" },
+ source_warnings: { type: "integer" },
+ per_query: {
+ type: "array",
+ items: {
+ type: "object",
+ properties: {
+ id: { type: "string" },
+ hit: { type: "boolean" },
+ rank: { type: "integer" },
+ answer_contained: { type: "boolean" },
+ },
+ },
+ },
+ },
+};
+
+const EVAL_TIMEOUT_MS = 60_000;
+
+/**
+ * `brain_eval` (Search & Recall Quality Suite): run the recall benchmark
+ * over a caller-supplied dataset against the active vault and return the
+ * quality metrics - hit@k, MRR, answer-containment@k, source-utilization,
+ * citation-depth, and the source-warnings count a CI gate can cap.
+ * Read-only; the fast path needs no embedding key.
+ */
+async function toolBrainEval(
+ ctx: ServerContext,
+ args: Record<string, unknown>,
+): Promise<Record<string, unknown>> {
+ let dataset;
+ try {
+ dataset = parseRecallBenchmarkDataset(args["dataset"]);
+ } catch (e) {
+ if (e instanceof SearchError) throw searchErrorToMcp(e);
+ throw new MCPError(INVALID_PARAMS, e instanceof Error ? e.message : String(e));
+ }
+ // Bound per-query rank depth at the untrusted MCP boundary. The library
+ // accepts any positive `k`, but an over-MCP caller must not bypass the
+ // top-level `k <= MCP_LIMIT_MAX` guard with a deep per-query override and
+ // trigger expensive searches.
+ for (const q of dataset.queries) {
+ if (q.k !== undefined && q.k > MCP_LIMIT_MAX) {
+ throw new MCPError(INVALID_PARAMS, `query '${q.id}' k must not exceed ${MCP_LIMIT_MAX}`);
+ }
+ }
+ let k: number | undefined;
+ if ("k" in args && args["k"] !== undefined && args["k"] !== null) {
+ const raw = args["k"];
+ if (typeof raw !== "number" || !Number.isInteger(raw) || raw < 1 || raw > MCP_LIMIT_MAX) {
+ throw new MCPError(
+ INVALID_PARAMS,
+ `argument 'k' must be an integer between 1 and ${MCP_LIMIT_MAX}`,
+ );
+ }
+ k = raw;
+ }
+ const expand = coerceBoolOptional(args, "expand") ?? false;
+ const config = resolveSearchConfig({
+ vault: ctx.vault,
+ configPath: ctx.configPath ?? undefined,
+ });
+ let report;
+ try {
+ report = await withTimeout(
+ runRecallBenchmark(config, dataset, { ...(k !== undefined ? { k } : {}), expand }),
+ EVAL_TIMEOUT_MS,
+ searchTimeoutError,
+ );
+ } catch (e) {
+ if (e instanceof SearchError) throw searchErrorToMcp(e);
+ if (e instanceof MCPError) throw e;
+ throw new MCPError(INTERNAL_ERROR, e instanceof Error ? e.message : String(e));
+ }
+ return {
+ total: report.total,
+ k: report.k,
+ expand: report.expand,
+ hit_at_k: report.hitAtK,
+ mrr: report.mrr,
+ answer_queries: report.answerQueries,
+ answer_containment_at_k: report.answerContainmentAtK,
+ source_utilization_at_k: report.sourceUtilizationAtK,
+ citation_depth: report.citationDepth,
+ source_warnings: report.sourceWarnings,
+ per_query: report.perQuery.map((q) => ({
+ id: q.id,
+ hit: q.hit,
+ ...(q.rank !== null ? { rank: q.rank } : {}),
+ ...(q.answerContained !== null ? { answer_contained: q.answerContained } : {}),
+ })),
+ };
+}
+
export const SEARCH_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([
{
name: "brain_recall_feedback",
@@ -602,6 +853,15 @@ export const SEARCH_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([
previewBudget: MCP_PREVIEW_BUDGET,
handler: toolBrainSearch,
},
+ {
+ name: "brain_eval",
+ description:
+ "Score retrieval quality over a dataset against the active vault: hit@k, MRR, answer-containment@k, source-utilization, citation-depth, source warnings. Read-only.",
+ inputSchema: EVAL_INPUT_SCHEMA,
+ outputSchema: EVAL_OUTPUT_SCHEMA,
+ previewBudget: MCP_PREVIEW_BUDGET,
+ handler: toolBrainEval,
+ },
]);
/**
diff --git a/tests/core/search/enrich.test.ts b/tests/core/search/enrich.test.ts
new file mode 100644
index 00000000..53de2f37
--- /dev/null
+++ b/tests/core/search/enrich.test.ts
@@ -0,0 +1,182 @@
+/**
+ * Read-time enrichment projections (Search & Recall Quality Suite).
+ * Pure functions over a ranked result - never stored, computed at read
+ * time like recall-hint. This file covers the score_breakdown projection;
+ * trust-metadata and hybrid-degrade cases live alongside as the suite lands.
+ */
+
+import { describe, test, expect } from "bun:test";
+
+import {
+ deriveTrust,
+ detectHybridDegrade,
+ projectScoreBreakdown,
+ rerankByRelevance,
+} from "../../../src/core/search/enrich.ts";
+import type { BrainSearchResult, ScoreBreakdown } from "../../../src/core/search/types.ts";
+
+function result(over: Partial<BrainSearchResult>): BrainSearchResult {
+ return Object.freeze({
+ documentId: 1,
+ chunkId: 1,
+ path: "doc.md",
+ title: "Doc",
+ content: "body",
+ startLine: 1,
+ endLine: 1,
+ score: 0.5,
+ keywordScore: 0.4,
+ semanticScore: 0.2,
+ linkBoost: 0.01,
+ recencyBoost: 0.03,
+ searchType: "keyword",
+ reasons: Object.freeze([]),
+ ...over,
+ }) as BrainSearchResult;
+}
+
+test("projects the ranker breakdown verbatim when present", () => {
+ const breakdown: ScoreBreakdown = Object.freeze({
+ keyword: 0.4,
+ semantic: 0.2,
+ rrf: 0,
+ entity: 0.02,
+ activation: 0,
+ coAccess: 0,
+ link: 0.01,
+ recency: 0.03,
+ tier: 1.2,
+ trend: 1,
+ sessionFocus: 0,
+ });
+ const out = projectScoreBreakdown(result({ breakdown }));
+ expect(out).toEqual(breakdown);
+});
+
+test("derives a faithful breakdown from first-class fields when breakdown is absent", () => {
+ // Synthetic results (traversal expansions, successor pull-ins) carry no
+ // breakdown; the projection falls back to the lane/boost fields they DO
+ // expose, with non-derivable layers honestly zero.
+ const out = projectScoreBreakdown(
+ result({ keywordScore: 0.7, semanticScore: 0.1, linkBoost: 0.02, recencyBoost: 0.05 }),
+ );
+ expect(out.keyword).toBe(0.7);
+ expect(out.semantic).toBe(0.1);
+ expect(out.link).toBe(0.02);
+ expect(out.recency).toBe(0.05);
+ expect(out.entity).toBe(0);
+ expect(out.activation).toBe(0);
+ expect(out.coAccess).toBe(0);
+ expect(out.rrf).toBe(0);
+ expect(out.tier).toBe(1);
+ expect(out.trend).toBe(1);
+ expect(out.sessionFocus).toBe(0);
+});
+
+test("projection is frozen", () => {
+ expect(Object.isFrozen(projectScoreBreakdown(result({})))).toBe(true);
+});
+
+describe("detectHybridDegrade", () => {
+ test("flags a keyword-only degrade when semantic was wanted but did not run", () => {
+ const w = detectHybridDegrade({
+ wantSemantic: true,
+ semanticAttempted: false,
+ keywordHitCount: 5,
+ });
+ expect(w).not.toBeNull();
+ expect(w!.startsWith("hybrid_degraded:")).toBe(true);
+ });
+
+ test("no warning when both lanes ran (genuine hybrid)", () => {
+ expect(
+ detectHybridDegrade({ wantSemantic: true, semanticAttempted: true, keywordHitCount: 5 }),
+ ).toBeNull();
+ });
+
+ test("no warning when the caller did not want semantic (keyword-only by choice)", () => {
+ expect(
+ detectHybridDegrade({ wantSemantic: false, semanticAttempted: false, keywordHitCount: 5 }),
+ ).toBeNull();
+ });
+
+ test("no warning when there are no keyword candidates to serve", () => {
+ // Nothing was served, so there is no degraded result set to flag - that
+ // is an empty query, not a silent single-lane fallback.
+ expect(
+ detectHybridDegrade({ wantSemantic: true, semanticAttempted: false, keywordHitCount: 0 }),
+ ).toBeNull();
+ });
+});
+
+describe("deriveTrust", () => {
+ const DAY_MS = 24 * 60 * 60 * 1000;
+ const NOW = 1_750_000_000_000;
+
+ test("age_days is the whole-day distance from mtime, floored at zero", () => {
+ const t = deriveTrust({ mtimeMs: NOW - 10 * DAY_MS, nowMs: NOW });
+ expect(t.age_days).toBe(10);
+ // A future mtime never reports a negative age.
+ expect(deriveTrust({ mtimeMs: NOW + 5 * DAY_MS, nowMs: NOW }).age_days).toBe(0);
+ });
+
+ test("superseded and conflict default false with no relations", () => {
+ const t = deriveTrust({ mtimeMs: NOW, nowMs: NOW });
+ expect(t.superseded).toBe(false);
+ expect(t.conflict).toBe(false);
+ });
+
+ test("superseded_by relation marks the hit superseded", () => {
+ const t = deriveTrust({
+ mtimeMs: NOW,
+ nowMs: NOW,
+ relations: [{ relation: "superseded_by", target: "newer.md" }],
+ });
+ expect(t.superseded).toBe(true);
+ expect(t.conflict).toBe(false);
+ });
+
+ test("contradicts relation marks the hit conflicted", () => {
+ const t = deriveTrust({
+ mtimeMs: NOW,
+ nowMs: NOW,
+ relations: [{ relation: "contradicts", target: "other.md" }],
+ });
+ expect(t.conflict).toBe(true);
+ expect(t.superseded).toBe(false);
+ });
+
+ test("unrelated relation types leave both flags false", () => {
+ const t = deriveTrust({
+ mtimeMs: NOW,
+ nowMs: NOW,
+ relations: [{ relation: "related", target: "x.md" }],
+ });
+ expect(t.superseded).toBe(false);
+ expect(t.conflict).toBe(false);
+ });
+
+ test("result is frozen", () => {
+ expect(Object.isFrozen(deriveTrust({ mtimeMs: NOW, nowMs: NOW }))).toBe(true);
+ });
+});
+
+describe("rerankByRelevance", () => {
+ test("orders by core lexical+semantic relevance, ignoring boosts", () => {
+ // `a` wins on final score thanks to a recency/usage boost, but `b` is
+ // the more textually relevant hit; a relevance rerank surfaces `b`.
+ const a = result({ path: "a.md", score: 0.9, keywordScore: 0.2, semanticScore: 0.1 });
+ const b = result({ path: "b.md", score: 0.6, keywordScore: 0.7, semanticScore: 0.2 });
+ const out = rerankByRelevance([a, b]);
+ expect(out.map((r) => r.path)).toEqual(["b.md", "a.md"]);
+ });
+
+ test("stable for equal relevance and does not mutate the input", () => {
+ const a = result({ path: "a.md", keywordScore: 0.5, semanticScore: 0.1 });
+ const b = result({ path: "b.md", keywordScore: 0.5, semanticScore: 0.1 });
+ const input = [a, b];
+ const out = rerankByRelevance(input);
+ expect(out.map((r) => r.path)).toEqual(["a.md", "b.md"]);
+ expect(input.map((r) => r.path)).toEqual(["a.md", "b.md"]);
+ });
+});
diff --git a/tests/core/search/feedback.test.ts b/tests/core/search/feedback.test.ts
index b143fb0a..0f00f01d 100644
--- a/tests/core/search/feedback.test.ts
+++ b/tests/core/search/feedback.test.ts
@@ -149,6 +149,19 @@ describe("learned-weight fold", () => {
recencyBoost: 0.05,
searchType: "hybrid",
reasons: ["fts5_bm25: 0.600", "semantic_cos: 0.200", "entity_match: 0.020"],
+ breakdown: {
+ keyword: 0.6,
+ semantic: 0.2,
+ rrf: 0,
+ entity: 0.02,
+ activation: 0,
+ coAccess: 0,
+ link: 0,
+ recency: 0.05,
+ tier: 1,
+ trend: 1,
+ sessionFocus: 0,
+ },
} as unknown as BrainSearchResult;
const c = contributionsFromResult(result);
expect(c.keyword).toBeCloseTo(0.6, 5);
@@ -156,6 +169,30 @@ describe("learned-weight fold", () => {
expect(c.entity).toBeCloseTo(0.02, 5);
expect(c.recency).toBeCloseTo(0.05, 5);
});
+
+ test("contributionsFromResult reports zero entity when no breakdown is present", () => {
+ // A synthetic result (e.g. a traversal expansion) carries no breakdown;
+ // the entity layer is honestly zero, not a parse artifact.
+ const result = {
+ documentId: 1,
+ chunkId: 11,
+ path: "y.md",
+ title: null,
+ content: "",
+ startLine: 1,
+ endLine: 1,
+ score: 0.4,
+ keywordScore: 0.4,
+ semanticScore: 0,
+ linkBoost: 0,
+ recencyBoost: 0,
+ searchType: "keyword",
+ reasons: ["fts5_bm25: 0.400"],
+ } as unknown as BrainSearchResult;
+ const c = contributionsFromResult(result);
+ expect(c.entity).toBe(0);
+ expect(c.keyword).toBeCloseTo(0.4, 5);
+ });
});
describe("derived weights file", () => {
diff --git a/tests/core/search/hybrid-degrade.test.ts b/tests/core/search/hybrid-degrade.test.ts
new file mode 100644
index 00000000..ea3e8fa4
--- /dev/null
+++ b/tests/core/search/hybrid-degrade.test.ts
@@ -0,0 +1,61 @@
+/**
+ * Hybrid-degrade warning (Search & Recall Quality Suite): when a caller
+ * wants the semantic lane but it cannot run (here: no embeddings indexed),
+ * search() serves keyword-only and appends ONE greppable
+ * `hybrid_degraded:` warning. A keyword-only-by-choice query and a
+ * genuine hybrid query never carry it.
+ */
+
+import { test, expect, beforeEach, afterEach } from "bun:test";
+
+import { indexVault } from "../../../src/core/search/indexer.ts";
+import { search } from "../../../src/core/search/search.ts";
+import { createTempVault, makeConfig, writeMd } from "../../helpers/search-fixtures.ts";
+
+let vault: string;
+let dbPath: string;
+let cleanup: () => void;
+
+beforeEach(() => {
+ const v = createTempVault("hybrid-degrade");
+ vault = v.vault;
+ dbPath = v.dbPath;
+ cleanup = v.cleanup;
+ writeMd(vault, "Notes/foo.md", "# Foo\n\nThe quick brown fox jumps over the lazy dog.");
+});
+
+afterEach(() => {
+ cleanup();
+});
+
+function semanticEnabledConfig() {
+ // Semantic is enabled in config, but the index below is built WITHOUT
+ // embeddings, so the semantic lane degrades out at query time.
+ return makeConfig({
+ vault,
+ dbPath,
+ semantic: { enabled: true, provider: "local", model: "local", dimension: 8 },
+ });
+}
+
+test("semantic wanted but unavailable -> keyword-only with a hybrid_degraded warning", async () => {
+ const cfg = semanticEnabledConfig();
+ await indexVault(cfg, {}); // no embeddings
+ const out = await search(cfg, { query: "fox", limit: 5, semantic: true });
+ expect(out.results.length).toBeGreaterThan(0);
+ expect(out.warnings.some((w) => w.startsWith("hybrid_degraded:"))).toBe(true);
+});
+
+test("keyword-only by choice carries no hybrid_degraded warning", async () => {
+ const cfg = semanticEnabledConfig();
+ await indexVault(cfg, {});
+ const out = await search(cfg, { query: "fox", limit: 5, keywordOnly: true });
+ expect(out.warnings.some((w) => w.startsWith("hybrid_degraded:"))).toBe(false);
+});
+
+test("a vault with semantic disabled in config carries no hybrid_degraded warning", async () => {
+ const cfg = makeConfig({ vault, dbPath }); // semantic disabled (default)
+ await indexVault(cfg, {});
+ const out = await search(cfg, { query: "fox", limit: 5 });
+ expect(out.warnings.some((w) => w.startsWith("hybrid_degraded:"))).toBe(false);
+});
diff --git a/tests/core/search/ranker-breakdown.test.ts b/tests/core/search/ranker-breakdown.test.ts
new file mode 100644
index 00000000..fcebef2f
--- /dev/null
+++ b/tests/core/search/ranker-breakdown.test.ts
@@ -0,0 +1,99 @@
+/**
+ * Structured score breakdown: every ranked result carries a `breakdown`
+ * object with the per-layer numeric components the final score is built
+ * from. Pure exposure of values `rankResults` already computes - the
+ * structured sibling of the human-readable `reasons` strings, and the
+ * source `feedback.ts` and the MCP `explain` projection read.
+ */
+
+import { test, expect } from "bun:test";
+
+import { rankResults } from "../../../src/core/search/ranker.ts";
+import type { KeywordHit, SemanticHit, HydratedChunk } from "../../../src/core/search/store.ts";
+
+function hyd(
+ chunkId: number,
+ docId: number,
+ mtime: number,
+ path = `doc${docId}.md`,
+): HydratedChunk {
+ return Object.freeze({
+ chunkId,
+ documentId: docId,
+ path,
+ title: `Doc ${docId}`,
+ content: `chunk ${chunkId}`,
+ startLine: 1,
+ endLine: 1,
+ mtime,
+ });
+}
+
+const NOW = 1_750_000_000_000;
+const OLD = NOW / 1000 - 365 * 24 * 3600; // 0 recency
+
+test("keyword-only hit breakdown mirrors the first-class lane fields", () => {
+ const ranked = rankResults(
+ {
+ keyword: [{ chunkId: 1, documentId: 10, bm25: -5 }] as KeywordHit[],
+ semantic: [] as SemanticHit[],
+ hydrated: new Map([[1, hyd(1, 10, OLD)]]),
+ inboundLinkSources: new Map(),
+ tagsByDoc: new Map(),
+ },
+ { keywordWeight: 0.6, semanticWeight: 0.4, limit: 10, nowMs: NOW },
+ );
+ const r = ranked.find((x) => x.chunkId === 1)!;
+ expect(r.breakdown).toBeDefined();
+ expect(typeof r.breakdown).toBe("object");
+ // The normalized lane components equal the first-class fields the
+ // result already exposes - no divergent computation.
+ expect(r.breakdown!.keyword).toBe(r.keywordScore);
+ expect(r.breakdown!.semantic).toBe(r.semanticScore);
+ expect(r.breakdown!.link).toBe(r.linkBoost);
+ expect(r.breakdown!.recency).toBe(r.recencyBoost);
+ // Layers that did not fire are exactly zero (not omitted, unlike reasons).
+ expect(r.breakdown!.entity).toBe(0);
+ expect(r.breakdown!.activation).toBe(0);
+ expect(r.breakdown!.coAccess).toBe(0);
+ expect(r.breakdown!.rrf).toBe(0);
+ // Neutral multipliers report as 1.
+ expect(r.breakdown!.tier).toBe(1);
+ expect(r.breakdown!.trend).toBe(1);
+});
+
+test("entity boost surfaces as a structured number, not only a reason string", () => {
+ const ranked = rankResults(
+ {
+ keyword: [{ chunkId: 1, documentId: 10, bm25: -5 }] as KeywordHit[],
+ semantic: [] as SemanticHit[],
+ hydrated: new Map([[1, hyd(1, 10, OLD)]]),
+ inboundLinkSources: new Map(),
+ tagsByDoc: new Map(),
+ entityMatchByChunk: new Map([[1, 1]]),
+ },
+ { keywordWeight: 0.6, semanticWeight: 0.4, limit: 10, nowMs: NOW },
+ );
+ const r = ranked.find((x) => x.chunkId === 1)!;
+ expect(r.breakdown!.entity).toBeGreaterThan(0);
+ // And it equals the value the reason string reports.
+ const reason = r.reasons.find((x) => x.startsWith("entity_match: "))!;
+ expect(r.breakdown!.entity).toBeCloseTo(Number(reason.slice("entity_match: ".length)), 3);
+});
+
+test("breakdown is frozen", () => {
+ const ranked = rankResults(
+ {
+ keyword: [{ chunkId: 1, documentId: 10, bm25: -5 }] as KeywordHit[],
+ semantic: [] as SemanticHit[],
+ hydrated: new Map([[1, hyd(1, 10, OLD)]]),
+ inboundLinkSources: new Map(),
+ tagsByDoc: new Map(),
+ },
+ { keywordWeight: 0.6, semanticWeight: 0.4, limit: 10, nowMs: NOW },
+ );
+ // Assert presence before freeze: Object.isFrozen(undefined) is true, so a
+ // missing breakdown would otherwise pass this test silently.
+ expect(ranked[0]!.breakdown).toBeDefined();
+ expect(Object.isFrozen(ranked[0]!.breakdown)).toBe(true);
+});
diff --git a/tests/core/search/recall-benchmark.test.ts b/tests/core/search/recall-benchmark.test.ts
index 647b0602..4a4c8b49 100644
--- a/tests/core/search/recall-benchmark.test.ts
+++ b/tests/core/search/recall-benchmark.test.ts
@@ -32,6 +32,11 @@ const FIXTURE = join(import.meta.dir, "..", "..", "fixtures", "recall-benchmark"
*/
const MIN_HIT_AT_5 = 0.9;
const MIN_MRR = 0.85;
+// Answer-containment floor over the answer-bearing fixture queries. The
+// three answers are verbatim substrings of their source notes, so a
+// correct retrieval contains them; the pin sits one failing-direction
+// margin below the measured 1.0.
+const MIN_ANSWER_CONTAINMENT_AT_5 = 0.99;
let vault: string;
let config: ResolvedSearchConfig;
@@ -80,6 +85,11 @@ describe("parseRecallBenchmarkDataset", () => {
expect(() =>
parseRecallBenchmarkDataset({ queries: [{ id: "x", query: "q", expected: ["a.md"], k: 0 }] }),
).toThrow(/k must be/);
+ expect(() =>
+ parseRecallBenchmarkDataset({
+ queries: [{ id: "x", query: "q", expected: ["a.md"], answer: " " }],
+ }),
+ ).toThrow(/answer must be/);
});
});
@@ -91,6 +101,47 @@ describe("runRecallBenchmark", () => {
expect(report.mrr).toBeGreaterThanOrEqual(MIN_MRR);
});
+ test("answer-containment@k scores the answer-bearing queries and holds the pinned floor", async () => {
+ const report = await runRecallBenchmark(config, loadDataset(), { k: 5 });
+ expect(report.answerQueries).toBe(3);
+ expect(report.answerContainmentAtK).toBeGreaterThanOrEqual(MIN_ANSWER_CONTAINMENT_AT_5);
+ // Queries without an answer report null containment, not a false miss.
+ const noAnswer = report.perQuery.find((q) => q.id === "style")!;
+ expect(noAnswer.answerContained).toBeNull();
+ });
+
+ test("an answer absent from the retrieved content scores a containment miss", async () => {
+ const report = await runRecallBenchmark(
+ config,
+ parseRecallBenchmarkDataset({
+ queries: [
+ {
+ id: "miss",
+ query: "canary rollout",
+ expected: ["deploy-canary.md"],
+ answer: "this phrase appears in no note",
+ },
+ ],
+ }),
+ { k: 5 },
+ );
+ expect(report.answerQueries).toBe(1);
+ expect(report.answerContainmentAtK).toBe(0);
+ expect(report.perQuery[0]!.answerContained).toBe(false);
+ });
+
+ test("answer-containment is vacuously 1 for a dataset with no answers", async () => {
+ const report = await runRecallBenchmark(
+ config,
+ parseRecallBenchmarkDataset({
+ queries: [{ id: "plain", query: "canary rollout", expected: ["deploy-canary.md"] }],
+ }),
+ { k: 5 },
+ );
+ expect(report.answerQueries).toBe(0);
+ expect(report.answerContainmentAtK).toBe(1);
+ });
+
test("the benchmark is deterministic across runs", async () => {
const first = await runRecallBenchmark(config, loadDataset(), { k: 5 });
const second = await runRecallBenchmark(config, loadDataset(), { k: 5 });
diff --git a/tests/core/search/reinforce.test.ts b/tests/core/search/reinforce.test.ts
new file mode 100644
index 00000000..2a61db90
--- /dev/null
+++ b/tests/core/search/reinforce.test.ts
@@ -0,0 +1,129 @@
+/**
+ * Self-tuning reinforce ledger (Search & Recall Quality Suite): an
+ * explicit, opt-in signal that a memory proved useful. Events land one
+ * file per signal under Brain/search/reinforce/; the fold yields a
+ * bounded per-path strength that lifts named memories before the top_k
+ * cut. Surfaced-only frequency is never recorded, so it never boosts.
+ */
+
+import { test, expect, beforeEach, afterEach, describe } from "bun:test";
+import { existsSync } from "node:fs";
+
+import {
+ REINFORCE_BOOST_CAP,
+ applyReinforceBoost,
+ loadReinforceStrengths,
+ recordReinforce,
+ reinforceDir,
+ resetReinforce,
+} from "../../../src/core/search/reinforce.ts";
+import { createTempVault, makeConfig, writeMd } from "../../helpers/search-fixtures.ts";
+import { indexVault } from "../../../src/core/search/indexer.ts";
+import { search } from "../../../src/core/search/search.ts";
+import type { BrainSearchResult } from "../../../src/core/search/types.ts";
+
+let vault: string;
+let cleanup: () => void;
+
+beforeEach(() => {
+ ({ vault, cleanup } = createTempVault("reinforce"));
+});
+afterEach(() => cleanup());
+
+function result(path: string, score: number): BrainSearchResult {
+ return Object.freeze({
+ documentId: 1,
+ chunkId: 1,
+ path,
+ title: path,
+ content: "body",
+ startLine: 1,
+ endLine: 1,
+ score,
+ keywordScore: score,
+ semanticScore: 0,
+ linkBoost: 0,
+ recencyBoost: 0,
+ searchType: "keyword",
+ reasons: Object.freeze([]),
+ }) as BrainSearchResult;
+}
+
+describe("ledger", () => {
+ test("recording grows a path's strength; reset clears the derived state", () => {
+ expect(loadReinforceStrengths(vault).size).toBe(0);
+ recordReinforce(vault, ["a.md"], { nowMs: 1 });
+ recordReinforce(vault, ["a.md", "b.md"], { nowMs: 2 });
+ const strengths = loadReinforceStrengths(vault);
+ expect(strengths.get("a.md")).toBeGreaterThan(strengths.get("b.md")!);
+ expect(existsSync(reinforceDir(vault))).toBe(true);
+ resetReinforce(vault);
+ expect(loadReinforceStrengths(vault).size).toBe(0);
+ });
+
+ test("strength is bounded to [0,1] regardless of count", () => {
+ for (let i = 0; i < 50; i++) recordReinforce(vault, ["hot.md"], { nowMs: i + 1 });
+ expect(loadReinforceStrengths(vault).get("hot.md")).toBeLessThanOrEqual(1);
+ });
+
+ test("recording the identical event twice is idempotent", () => {
+ recordReinforce(vault, ["a.md"], { nowMs: 5 });
+ recordReinforce(vault, ["a.md"], { nowMs: 5 });
+ // One distinct event file -> the fold counts it once.
+ const a1 = loadReinforceStrengths(vault).get("a.md");
+ recordReinforce(vault, ["a.md"], { nowMs: 6 });
+ expect(loadReinforceStrengths(vault).get("a.md")).toBeGreaterThan(a1!);
+ });
+});
+
+describe("applyReinforceBoost", () => {
+ test("lifts a reinforced path above an unreinforced higher-scored one", () => {
+ const results = [result("top.md", 0.6), result("reinforced.md", 0.58)];
+ const boosted = applyReinforceBoost(results, new Map([["reinforced.md", 1]]));
+ expect(boosted[0]!.path).toBe("reinforced.md");
+ expect(
+ boosted
+ .find((r) => r.path === "reinforced.md")!
+ .reasons.some((x) => x.startsWith("reinforce:")),
+ ).toBe(true);
+ });
+
+ test("boost is bounded by the cap and clamped to 1", () => {
+ const boosted = applyReinforceBoost([result("x.md", 0.99)], new Map([["x.md", 1]]));
+ expect(boosted[0]!.score).toBeLessThanOrEqual(1);
+ expect(boosted[0]!.score - 0.99).toBeLessThanOrEqual(REINFORCE_BOOST_CAP + 1e-9);
+ });
+
+ test("no reinforced paths -> input order and scores unchanged", () => {
+ const results = [result("a.md", 0.6), result("b.md", 0.4)];
+ const out = applyReinforceBoost(results, new Map());
+ expect(out.map((r) => r.path)).toEqual(["a.md", "b.md"]);
+ expect(out.map((r) => r.score)).toEqual([0.6, 0.4]);
+ });
+});
+
+describe("search integration", () => {
+ test("absent reinforce leaves the ranking byte-identical even with a populated ledger", async () => {
+ writeMd(vault, "a.md", "# A\n\nfox fox fox quick brown fox");
+ writeMd(vault, "b.md", "# B\n\na single fox mention among cats");
+ const cfg = makeConfig({ vault, dbPath: `${vault}/.idx.sqlite` });
+ await indexVault(cfg);
+ recordReinforce(vault, ["b.md"], { nowMs: 1 });
+ const base = await search(cfg, { query: "fox", limit: 10 });
+ // No reinforce key -> the ledger must not touch the result.
+ for (const r of base.results)
+ expect(r.reasons.some((x) => x.startsWith("reinforce:"))).toBe(false);
+ });
+
+ test("opt-in reinforce lifts a ledger-strong path with a reinforce reason", async () => {
+ writeMd(vault, "a.md", "# A\n\nfox fox fox quick brown fox");
+ writeMd(vault, "b.md", "# B\n\na single fox mention among cats");
+ const cfg = makeConfig({ vault, dbPath: `${vault}/.idx.sqlite` });
+ await indexVault(cfg);
+ for (let i = 0; i < 5; i++) recordReinforce(vault, ["b.md"], { nowMs: i + 1 });
+ const out = await search(cfg, { query: "fox", limit: 10, reinforce: [] });
+ const b = out.results.find((r) => r.path === "b.md");
+ expect(b).toBeDefined();
+ expect(b!.reasons.some((x) => x.startsWith("reinforce:"))).toBe(true);
+ });
+});
diff --git a/tests/core/search/threshold.test.ts b/tests/core/search/threshold.test.ts
new file mode 100644
index 00000000..801bc957
--- /dev/null
+++ b/tests/core/search/threshold.test.ts
@@ -0,0 +1,75 @@
+/**
+ * Relevance threshold + rerank (Search & Recall Quality Suite): an
+ * opt-in score floor drops weak hits (returning no match when nothing is
+ * relevant enough), and an opt-in rerank re-orders the qualified set by
+ * core textual relevance. Both off by default keep results byte-identical.
+ */
+
+import { test, expect, beforeEach, afterEach } from "bun:test";
+
+import { indexVault } from "../../../src/core/search/indexer.ts";
+import { search } from "../../../src/core/search/search.ts";
+import { SearchError } from "../../../src/core/search/types.ts";
+import { createTempVault, makeConfig, writeMd } from "../../helpers/search-fixtures.ts";
+
+let vault: string;
+let dbPath: string;
+let cleanup: () => void;
+
+beforeEach(() => {
+ ({ vault, dbPath, cleanup } = createTempVault("threshold"));
+ writeMd(vault, "strong.md", "# Strong\n\nfox fox fox fox the quick brown fox jumps.");
+ writeMd(vault, "weak.md", "# Weak\n\nA note mostly about cats, with one fox mention.");
+ writeMd(vault, "none.md", "# None\n\nTurtles and cats only.");
+});
+afterEach(() => cleanup());
+
+test("threshold absent keeps results byte-identical", async () => {
+ const cfg = makeConfig({ vault, dbPath });
+ await indexVault(cfg);
+ const base = await search(cfg, { query: "fox", limit: 10 });
+ const withZero = await search(cfg, { query: "fox", limit: 10, threshold: 0 });
+ expect(withZero.results.map((r) => r.path)).toEqual(base.results.map((r) => r.path));
+});
+
+test("a floor between the top and bottom scores drops the weak hits", async () => {
+ const cfg = makeConfig({ vault, dbPath });
+ await indexVault(cfg);
+ const base = await search(cfg, { query: "fox", limit: 10 });
+ expect(base.results.length).toBeGreaterThanOrEqual(2);
+ const scores = base.results.map((r) => r.score);
+ const top = scores[0]!;
+ const bottom = scores[scores.length - 1]!;
+ expect(top).toBeGreaterThan(bottom);
+ const floor = (top + bottom) / 2;
+ const filtered = await search(cfg, { query: "fox", limit: 10, threshold: floor });
+ expect(filtered.results.length).toBeLessThan(base.results.length);
+ expect(filtered.results.length).toBeGreaterThan(0);
+ for (const r of filtered.results) expect(r.score).toBeGreaterThanOrEqual(floor);
+});
+
+test("a floor above every score returns no match", async () => {
+ const cfg = makeConfig({ vault, dbPath });
+ await indexVault(cfg);
+ const out = await search(cfg, { query: "fox", limit: 10, threshold: 1.0001 });
+ expect(out.results).toEqual([]);
+ expect(out.total).toBe(0);
+});
+
+test("a negative threshold is rejected", async () => {
+ const cfg = makeConfig({ vault, dbPath });
+ await indexVault(cfg);
+ await expect(search(cfg, { query: "fox", limit: 10, threshold: -1 })).rejects.toBeInstanceOf(
+ SearchError,
+ );
+});
+
+test("rerank does not change the returned membership, only ordering", async () => {
+ const cfg = makeConfig({ vault, dbPath });
+ await indexVault(cfg);
+ const base = await search(cfg, { query: "fox", limit: 10 });
+ const reranked = await search(cfg, { query: "fox", limit: 10, rerank: true });
+ expect(reranked.results.map((r) => r.path).toSorted()).toEqual(
+ base.results.map((r) => r.path).toSorted(),
+ );
+});
diff --git a/tests/core/search/trust-metadata.test.ts b/tests/core/search/trust-metadata.test.ts
new file mode 100644
index 00000000..ac86343e
--- /dev/null
+++ b/tests/core/search/trust-metadata.test.ts
@@ -0,0 +1,77 @@
+/**
+ * Inline trust metadata (Search & Recall Quality Suite): with `trust`
+ * set, each hit carries a computed-at-read-time `trust` object - age in
+ * days plus the superseded / conflict flags read from the surfaced typed
+ * relations. Off by default keeps the result shape byte-identical.
+ */
+
+import { test, expect, beforeEach, afterEach } from "bun:test";
+
+import { indexVault } from "../../../src/core/search/indexer.ts";
+import { search } from "../../../src/core/search/search.ts";
+import { createTempVault, makeConfig, writeMd } from "../../helpers/search-fixtures.ts";
+
+let vault: string;
+let dbPath: string;
+let cleanup: () => void;
+
+beforeEach(() => {
+ ({ vault, dbPath, cleanup } = createTempVault("trust-metadata"));
+});
+afterEach(() => cleanup());
+
+const OLD_PAGE = [
+ "---",
+ "title: Old deploy decision",
+ 'superseded_by: "[[new-deploy]]"',
+ "---",
+ "# Old deploy decision",
+ "",
+ "We deploy with the blue-green rollout strategy on fridays.",
+].join("\n");
+
+const NEW_PAGE = [
+ "---",
+ "title: New deploy decision",
+ "---",
+ "# New deploy decision",
+ "",
+ "Deployment notes for the canary path.",
+].join("\n");
+
+test("no trust field by default", async () => {
+ writeMd(vault, "plain.md", "# Plain\n\nThe quick brown fox jumps over the lazy dog.");
+ const cfg = makeConfig({ vault, dbPath });
+ await indexVault(cfg);
+ const out = await search(cfg, { query: "fox", limit: 5 });
+ expect(out.results.length).toBeGreaterThan(0);
+ for (const r of out.results) expect(r.trust).toBeUndefined();
+});
+
+test("trust stamps age and a neutral status on a plain recent hit", async () => {
+ writeMd(vault, "plain.md", "# Plain\n\nThe quick brown fox jumps over the lazy dog.");
+ const cfg = makeConfig({ vault, dbPath });
+ await indexVault(cfg);
+ const out = await search(cfg, { query: "fox", limit: 5, trust: true });
+ const hit = out.results.find((r) => r.path === "plain.md")!;
+ expect(hit.trust).toBeDefined();
+ expect(hit.trust!.age_days).toBeGreaterThanOrEqual(0);
+ expect(hit.trust!.superseded).toBe(false);
+ expect(hit.trust!.conflict).toBe(false);
+});
+
+test("trust marks a superseded predecessor", async () => {
+ writeMd(vault, "old.md", OLD_PAGE);
+ writeMd(vault, "new-deploy.md", NEW_PAGE);
+ const cfg = makeConfig({ vault, dbPath });
+ await indexVault(cfg);
+ const out = await search(cfg, {
+ query: "blue-green rollout strategy",
+ limit: 5,
+ trust: true,
+ includeSuperseded: true,
+ });
+ const old = out.results.find((r) => r.path === "old.md")!;
+ expect(old.trust).toBeDefined();
+ expect(old.trust!.superseded).toBe(true);
+});
diff --git a/tests/fixtures/recall-benchmark/dataset.json b/tests/fixtures/recall-benchmark/dataset.json
index 87e4fe77..e5bb9f8a 100644
--- a/tests/fixtures/recall-benchmark/dataset.json
+++ b/tests/fixtures/recall-benchmark/dataset.json
@@ -1,14 +1,29 @@
{
"queries": [
- { "id": "canary", "query": "canary rollout", "expected": ["deploy-canary.md"] },
+ {
+ "id": "canary",
+ "query": "canary rollout",
+ "expected": ["deploy-canary.md"],
+ "answer": "expand the rollout gradually"
+ },
{
"id": "bluegreen",
"query": "blue green deployment swap",
"expected": ["deploy-bluegreen.md"]
},
- { "id": "oauth", "query": "oauth token refresh", "expected": ["auth-oauth.md"] },
+ {
+ "id": "oauth",
+ "query": "oauth token refresh",
+ "expected": ["auth-oauth.md"],
+ "answer": "exchanges the refresh token"
+ },
{ "id": "passwords", "query": "password hashing salts", "expected": ["auth-passwords.md"] },
- { "id": "backups", "query": "nightly backup strategy", "expected": ["db-backups.md"] },
+ {
+ "id": "backups",
+ "query": "nightly backup strategy",
+ "expected": ["db-backups.md"],
+ "answer": "rotates the last fourteen backup archives offsite"
+ },
{ "id": "indexing", "query": "slow queries missing index", "expected": ["db-indexing.md"] },
{ "id": "incident", "query": "outage root cause timeout", "expected": ["incident-2026-05.md"] },
{ "id": "alias-hop", "query": "PA importer progress", "expected": ["project-alpha.md"] },
diff --git a/tests/mcp/brain-search-explain.test.ts b/tests/mcp/brain-search-explain.test.ts
new file mode 100644
index 00000000..4a02e287
--- /dev/null
+++ b/tests/mcp/brain-search-explain.test.ts
@@ -0,0 +1,105 @@
+/**
+ * `brain_search` exposes a structured `score_breakdown` per result only
+ * when the request sets `explain: true`. With the flag off the output is
+ * byte-identical to the legacy shape (no score_breakdown key). Exercises
+ * the tool through the same handler path the MCP server uses.
+ */
+
+import { afterEach, beforeEach, describe, expect, test } from "bun:test";
+import { mkdtempSync, rmSync, mkdirSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { SEARCH_TOOLS } from "../../src/mcp/search-tools.ts";
+import { indexVault, resolveSearchConfig } from "../../src/core/search/index.ts";
+import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
+
+let vault: string;
+let configHome: string;
+let ctx: { vault: string; configPath: string };
+
+const BREAKDOWN_KEYS = [
+ "keyword",
+ "semantic",
+ "rrf",
+ "entity",
+ "activation",
+ "coAccess",
+ "link",
+ "recency",
+ "tier",
+ "trend",
+ "sessionFocus",
+];
+
+beforeEach(async () => {
+ vault = mkdtempSync(join(tmpdir(), "o2b-explain-vault-"));
+ configHome = mkdtempSync(join(tmpdir(), "o2b-explain-cfg-"));
+ const configPath = join(configHome, "config.yaml");
+ atomicWriteFileSync(configPath, `vault: ${vault}\n`);
+ mkdirSync(join(vault, "notes"), { recursive: true });
+ writeFileSync(
+ join(vault, "notes", "alpha.md"),
+ "---\ntitle: Alpha\n---\n\nThe quick brown fox jumps over the lazy dog.\n",
+ );
+ ctx = { vault, configPath };
+ const config = resolveSearchConfig({ vault, configPath });
+ await indexVault(config, {});
+});
+
+afterEach(() => {
+ rmSync(vault, { recursive: true, force: true });
+ rmSync(configHome, { recursive: true, force: true });
+});
+
+describe("brain_search explain", () => {
+ const tool = () => SEARCH_TOOLS.find((t) => t.name === "brain_search")!;
+
+ test("no score_breakdown key when explain is absent", async () => {
+ // eslint-disable-next-line @typescript-eslint/no-explicit-any
+ const out = (await tool().handler(ctx as any, { query: "quick fox" })) as {
+ results: Array<Record<string, unknown>>;
+ total: number;
+ };
+ expect(out.total).toBeGreaterThan(0);
+ for (const r of out.results) {
+ expect("score_breakdown" in r).toBe(false);
+ }
+ });
+
+ test("structured score_breakdown on every result when explain is true", async () => {
+ // eslint-disable-next-line @typescript-eslint/no-explicit-any
+ const out = (await tool().handler(ctx as any, { query: "quick fox", explain: true })) as {
+ results: Array<{ score_breakdown?: Record<string, number> }>;
+ total: number;
+ };
+ expect(out.total).toBeGreaterThan(0);
+ for (const r of out.results) {
+ expect(r.score_breakdown).toBeDefined();
+ for (const key of BREAKDOWN_KEYS) {
+ expect(typeof r.score_breakdown![key]).toBe("number");
+ }
+ // keyword-only scratch vault: the bm25 lane carries the weight.
+ expect(r.score_breakdown!.keyword).toBeGreaterThan(0);
+ }
+ });
+
+ test("no trust key when trust is absent; structured trust when trust is true", async () => {
+ // eslint-disable-next-line @typescript-eslint/no-explicit-any
+ const off = (await tool().handler(ctx as any, { query: "quick fox" })) as {
+ results: Array<Record<string, unknown>>;
+ };
+ for (const r of off.results) expect("trust" in r).toBe(false);
+
+ // eslint-disable-next-line @typescript-eslint/no-explicit-any
+ const on = (await tool().handler(ctx as any, { query: "quick fox", trust: true })) as {
+ results: Array<{ trust?: { age_days: number; superseded: boolean; conflict: boolean } }>;
+ };
+ for (const r of on.results) {
+ expect(r.trust).toBeDefined();
+ expect(typeof r.trust!.age_days).toBe("number");
+ expect(typeof r.trust!.superseded).toBe("boolean");
+ expect(typeof r.trust!.conflict).toBe("boolean");
+ }
+ });
+});
diff --git a/tests/mcp/eval-tool.test.ts b/tests/mcp/eval-tool.test.ts
new file mode 100644
index 00000000..f47374db
--- /dev/null
+++ b/tests/mcp/eval-tool.test.ts
@@ -0,0 +1,97 @@
+/**
+ * `brain_eval` runs the recall benchmark over a caller-supplied dataset
+ * against the active vault and returns the quality metrics (hit@k, MRR,
+ * answer-containment@k, source-utilization, citation-depth, source
+ * warnings). Exercised through the same handler path the MCP server uses.
+ */
+
+import { afterEach, beforeEach, describe, expect, test } from "bun:test";
+import { mkdtempSync, rmSync, mkdirSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { SEARCH_TOOLS } from "../../src/mcp/search-tools.ts";
+import { indexVault, resolveSearchConfig } from "../../src/core/search/index.ts";
+import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
+
+let vault: string;
+let configHome: string;
+let ctx: { vault: string; configPath: string };
+
+beforeEach(async () => {
+ vault = mkdtempSync(join(tmpdir(), "o2b-eval-vault-"));
+ configHome = mkdtempSync(join(tmpdir(), "o2b-eval-cfg-"));
+ const configPath = join(configHome, "config.yaml");
+ atomicWriteFileSync(configPath, `vault: ${vault}\n`);
+ mkdirSync(join(vault, "notes"), { recursive: true });
+ writeFileSync(
+ join(vault, "notes", "canary.md"),
+ "---\ntitle: Canary\n---\n\n# Canary rollout\n\nShip to one instance, then expand the rollout gradually.\n",
+ );
+ ctx = { vault, configPath };
+ const config = resolveSearchConfig({ vault, configPath });
+ await indexVault(config, {});
+});
+
+afterEach(() => {
+ rmSync(vault, { recursive: true, force: true });
+ rmSync(configHome, { recursive: true, force: true });
+});
+
+describe("brain_eval", () => {
+ const tool = () => SEARCH_TOOLS.find((t) => t.name === "brain_eval")!;
+
+ test("is registered and read-only", () => {
+ expect(tool()).toBeDefined();
+ });
+
+ test("returns the full metric set for a matching dataset", async () => {
+ const dataset = {
+ queries: [
+ {
+ id: "canary",
+ query: "canary rollout",
+ expected: ["notes/canary.md"],
+ answer: "expand the rollout gradually",
+ },
+ ],
+ };
+ // eslint-disable-next-line @typescript-eslint/no-explicit-any
+ const out = (await tool().handler(ctx as any, { dataset, k: 5 })) as Record<string, number>;
+ expect(out.total).toBe(1);
+ expect(out.hit_at_k).toBe(1);
+ expect(out.mrr).toBeGreaterThan(0);
+ expect(out.answer_queries).toBe(1);
+ expect(out.answer_containment_at_k).toBe(1);
+ expect(out.source_utilization_at_k).toBe(1);
+ expect(out.citation_depth).toBeGreaterThanOrEqual(1);
+ expect(out.source_warnings).toBe(0);
+ });
+
+ test("counts a source warning when an expected source is not retrieved", async () => {
+ const dataset = {
+ queries: [{ id: "missing", query: "canary rollout", expected: ["notes/absent.md"] }],
+ };
+ // eslint-disable-next-line @typescript-eslint/no-explicit-any
+ const out = (await tool().handler(ctx as any, { dataset })) as Record<string, number>;
+ expect(out.source_warnings).toBe(1);
+ expect(out.hit_at_k).toBe(0);
+ });
+
+ test("rejects a malformed dataset", async () => {
+ await expect(
+ // eslint-disable-next-line @typescript-eslint/no-explicit-any
+ tool().handler(ctx as any, { dataset: { queries: [] } }),
+ ).rejects.toBeDefined();
+ });
+
+ test("rejects a per-query k above the MCP cap", async () => {
+ const dataset = {
+ queries: [{ id: "deep", query: "canary", expected: ["notes/canary.md"], k: 1000 }],
+ };
+ await expect(
+ // eslint-disable-next-line @typescript-eslint/no-explicit-any
+ tool().handler(ctx as any, { dataset }),
+ ).rejects.toBeDefined();
+ });
+});
diff --git a/tests/mcp/mcp.test.ts b/tests/mcp/mcp.test.ts
index df3460dd..a586e897 100644
--- a/tests/mcp/mcp.test.ts
+++ b/tests/mcp/mcp.test.ts
@@ -236,6 +236,7 @@ describe("tool listing", () => {
"brain_recall_feedback",
"brain_recall_gate",
"brain_search",
+ "brain_eval",
// Schema admin + watchdog recovery probes.
"schema_apply_mutations",
"schema_inspect",
@@ -524,7 +525,8 @@ describe("stdio loop", () => {
// (Write-Time Integrity & Governance Suite) = 65.
// + brain_hygiene / brain_anticipatory_context
// (continuity-hygiene-freshness suite, v1.3.0) = 71.
- expect(list.result.tools.length).toBe(71);
+ // + brain_eval (Search & Recall Quality Suite) = 72.
+ expect(list.result.tools.length).toBe(72);
});
test("returns parse error for invalid JSON", async () => {
diff --git a/tests/mcp/removed-tools.test.ts b/tests/mcp/removed-tools.test.ts
index b7f6fc34..f4c8fe59 100644
--- a/tests/mcp/removed-tools.test.ts
+++ b/tests/mcp/removed-tools.test.ts
@@ -138,5 +138,5 @@ test("the shadow surface is gone: no hidden tools, removed names unlisted", asyn
method: "tools/list",
params: {},
})) as { result: { tools: Array<{ name: string }> } };
- expect(list.result.tools.length).toBe(71);
+ expect(list.result.tools.length).toBe(72);
});