Skip to content

Local-first 🧠 memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly 😴 dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.

TypeScript

feat: recall precision, coverage, and provenance hardening (v1.18.0) (#110)

Sol Aitken33b4fba5d35f
diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json
index 8012afeb..ca2ad323 100644
--- a/.claude-plugin/plugin.json
+++ b/.claude-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "1.17.0",
+ "version": "1.18.0",
"description": "Plugin-first second brain package for AI agents and humans.",
"author": {
"name": "Open Second Brain contributors"
diff --git a/.codex-plugin/plugin.json b/.codex-plugin/plugin.json
index c10236b6..8d7a06ef 100644
--- a/.codex-plugin/plugin.json
+++ b/.codex-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "1.17.0",
+ "version": "1.18.0",
"description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.",
"skills": "./skills",
"hooks": "./hooks/hooks.json",
diff --git a/CHANGELOG.md b/CHANGELOG.md
index ecec3d50..b54a2391 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,6 +5,86 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+## [1.18.0] - 2026-06-24
+
+Recall precision, coverage, and provenance hardening. Six related
+improvements make recall more accurate, more complete, and more auditable:
+an end-to-end provenance reader, exact line-span citations, progressive
+disclosure for all recall, normalized-confidence cross-vault early
+termination, coverage-driven targeted self-correction, and a configurable
+chunk floor. Every new decision is deterministic and LLM-free, and each new
+behaviour is off by default or otherwise byte-identical until opted in.
+
+### Added
+
+- **Configurable chunk minimum size (`t_122b2cbc`).** The markdown chunker's
+ packing floor (`minTokens`) is now operator-tunable per vault via
+ `search_chunk_min_size` / `OPEN_SECOND_BRAIN_SEARCH_CHUNK_MIN_SIZE`,
+ threaded through `chunkMarkdown` and the indexer alongside the existing
+ `search_chunk_size` and `search_chunk_overlap` knobs. Default `100` is
+ unchanged, so vaults that do not set it hash identical chunks across
+ Syncthing peers.
+
+- **Event-to-context provenance reader (`t_8da11868`).** A new read-only
+ correlation-join surface attaches a logged event's context traces via its
+ correlation IDs, so an operator can answer "why did the agent do this"
+ from one surface: the `o2b brain event-trace` CLI verb and the
+ `brain_event_trace` MCP tool. Read-only; joins existing records and writes
+ nothing.
+
+- **Read-time line-span citations (`t_0b83c97b`).** A shared `LinePointer`
+ type and `line-numbering.ts` helpers resolve recalled content to exact
+ `path:Lstart-Lend` spans at read time (no Markdown mutation), and evidence
+ records and session-recall hits now carry those line anchors for sharper
+ citation-depth and answer-containment signals.
+
+- **Progressive disclosure for all recall (`t_468190f5`).** The compact-card
+ to expand to raw-transcript disclosure already used by session-recall is
+ generalized to the main vault search via `disclosure: "cards"`, so callers
+ can pay layer-1 token cost up front and drill deeper only when needed. The
+ default `full` path stays byte-identical.
+
+- **Normalized-confidence cross-vault chain-stop (`t_23c1b929`).** Opt-in via
+ `search_chain_stop_enabled` / `OPEN_SECOND_BRAIN_SEARCH_CHAIN_STOP`:
+ `searchAcrossVaults` stops querying further origins once a completed
+ origin's top NORMALIZED `[0,1]` result score reaches
+ `search_chain_stop_score` (default `0.8`), recording the skipped origins on
+ the new `SearchOutcome.chainStop`. The gate reads the normalized result
+ score, never the raw lane score, so a tiny-corpus origin with a high raw
+ score never short-circuits. Off by default and byte-identical when no
+ origin is skipped.
+
+- **Targeted retry on partial coverage (`t_8eb5ca32`).** In evidence-pack
+ mode, when the first pass returns candidates but their IDF-weighted
+ coverage of the query is below `COMPLETENESS_COMPLETE_THRESHOLD` with at
+ least one RARE significant term still uncovered, `search()` issues
+ exactly ONE follow-up FTS pass built from those uncovered rare terms
+ (the specifically-missing high-signal facts), merges the recovered
+ candidates into the pool, and lets them flow through the normal ranking
+ and a regenerated evidence pack. The new pure helper
+ `planTargetedRetry(coverage)` (in `coverage.ts`) is the deterministic
+ trigger: it fires only on a below-threshold coverage WITH uncovered rare
+ terms (a rare gate keeps it off when only corpus-common terms are
+ missing) and returns exactly those terms to re-query. This retry is
+ mutually exclusive with the existing zero-candidate broadened OR retry
+ (one needs an empty pool, the other a non-empty one), so at most one
+ retry fires per query — the same single-retry discipline. The
+ regenerated pack still abstains on any term left uncovered after the
+ retry, so the conservative-on-final-miss posture is unchanged.
+
+- **`SearchOutcome.secondPass.kind` (`t_8eb5ca32`).** The two-pass marker
+ now carries `kind: "broadened" | "targeted"` to distinguish the two
+ triggers, plus `targetedTerms` (the uncovered rare terms re-queried) for
+ the targeted variant. Recovered results carry a
+ `second_pass: targeted retry on uncovered rare terms` reason; first-pass
+ hits they merge with do not.
+
+### Fixed
+
+- **MCP search output contracts.** Aligned the MCP search tool output with
+ the new line-span and disclosure result shapes so the typed-relation and
+ evidence fields surface consistently across the CLI and MCP surfaces.
+
## [1.17.0] - 2026-06-21
CodeGraph link-graph depth and MCP exposure. A set of strictly additive
@@ -5869,6 +5949,7 @@ plugin config (vault field)`, and exits with a clear
- Sandbox vault and plugin manifest fixtures for tests.
- GitHub release workflow for tag-based and manually dispatched releases.
+[1.18.0]: https://github.com/itechmeat/open-second-brain/compare/v1.17.0...v1.18.0
[1.17.0]: https://github.com/itechmeat/open-second-brain/compare/v1.16.0...v1.17.0
[1.16.0]: https://github.com/itechmeat/open-second-brain/compare/v1.15.0...v1.16.0
[1.15.0]: https://github.com/itechmeat/open-second-brain/compare/v1.14.0...v1.15.0
diff --git a/README.md b/README.md
index 3eb2494b..675403f6 100644
--- a/README.md
+++ b/README.md
@@ -98,6 +98,7 @@ Each non-Hermes target writes a sidecar manifest under `<vault>/.open-second-bra
- **Operational readability for code partners and large vaults.** Open Second Brain now exposes a read-only CodeGraph report (`o2b partner codegraph report`, `brain_codegraph_report`) that resolves the in-scope code project, reports the codegraph index state with node and edge counts, and structurally parses Cargo.toml for Rust workspace members. The report is honest about missing CLIs, missing indexes, and non-Rust projects. For large vaults, community materialization can run in fixed-size batches (`o2b brain clusters run --batch-size N`, `brain_clusters` `batch_size`) with per-batch success or isolated failure reporting, while the default run stays byte-identical.
- **Consistent feedback categorization.** A vault-local `feedback.default_scope` in `Brain/_brain.yaml` gives agent-recorded signals a consistent category (for example `coding`) when no explicit scope is provided, with the same precedence across the inbox signal, its shared-namespace mirror, and any force-confirmed preference. The effective scope is computed once at the signal write boundary and byte-identical output is preserved when the setting is absent or no explicit scope is given.
- **Path-safe vault writes.** The write-session commit chokepoint now re-resolves every target through `ensureInsideVault` before any directory creation, read, or write, catching symlinked ancestors that point outside the vault root. The backstop fails closed: a target resolving outside the configured vault is rejected and nothing is written. All other caller-derived vault paths already funnel through guarded constructors; regression tests pin both invariants.
+- **Recall that pays for depth only on demand.** General vault search now offers the same progressive 3-layer disclosure session-recall already had. `o2b search "<q>" --disclosure cards` (and the `brain_search` `disclosure: "cards"` field) returns compact layer-1 cards - path, title, score, reasons, a bounded snippet, and a `path:Lstart-Lend` pointer - instead of full content per hit, so recall stays token-cheap. Drill a hit with `o2b search expand --chunk <id>` (or `brain_search_expand`) to get layer 2 (the fuller note) and layer 3 (the raw chunk transcript, paginated by cursor). It reuses the existing index read - no new index, no model - and the default `full` mode is byte-identical to before.
That is the day-to-day picture. The full capability surface, every CLI verb, and the mental model live in the [documentation](#documentation) below.
diff --git a/docs/brainstorm/recall-precision-provenance/cli-output/claude.md b/docs/brainstorm/recall-precision-provenance/cli-output/claude.md
new file mode 100644
index 00000000..bb802b99
--- /dev/null
+++ b/docs/brainstorm/recall-precision-provenance/cli-output/claude.md
@@ -0,0 +1,26 @@
+## Variant 1
+Approach: Drive the six cards one at a time as additive, independently-switched modules, ordering only the two pairs that share a file so the second builds on a seam the first already laid: land D2's read-time `Lstart-Lend` pointer primitive before D3's disclosure-depth layering (both touch citation/evidence rendering), and land D4's chain-stop before D5's coverage-retry (both touch `search.ts`/cross-vault). D1 (read-only correlation-join reader) and D7 (expose `minTokens`) are orthogonal and can land in any slot. Each card integrates at a single guarded call site whose default-off branch is bit-identical to today.
+Trade-offs:
+- Minimal blast radius and easy per-card review/revert; matches the "one PR, many cards, each behind a switch" cadence the git log already follows.
+- Ordering is a discipline, not a guarantee — a mis-sequenced D5-before-D4 would force a rebase, and the shared-file pairs still serialize (no parallel authoring of D2/D3 or D4/D5).
+Complexity: medium
+Risk: low
+
+## Variant 2
+Approach: Spend the first commit defining the shared seams before any card logic: a `PointerSpec` citation type consumed by both D2 and D5, a single ordered recall-decision policy hook in `search.ts`/`cross-vault.ts` into which D4 (chain-stop) and D5 (coverage follow-up) register as distinct deterministic policies, and a disclosure-layer contract that D3 implements over the existing store read. Each card then plugs into its seam as an isolated, separately-flagged implementation.
+Trade-offs:
+- Lowest late-stage merge risk: D4/D5 stop colliding because they share an explicit registration point rather than both editing the same hot path, and D2/D3 share one citation type.
+- The seam commit is speculative design ahead of two unwritten cards; if D5's targeted-requery shape diverges from the guessed policy interface, the abstraction churns — and an over-built hook risks accidentally altering the off-path, threatening the bit-identical guarantee.
+Complexity: large
+Risk: medium
+
+## Variant 3
+Approach: Refactor `search.ts`, `cross-vault.ts`, and `evidence-pack.ts` into one unified recall pipeline that natively expresses layered disclosure (D3), normalized-confidence chain-stop (D4), coverage-driven follow-up (D5), and line-anchored pointers (D2/D5), with per-vault config knobs (D7) and the provenance reader (D1) reading the pipeline's emitted traces.
+Trade-offs:
+- Most cohesive end state — one place to reason about precision, coverage, and disclosure together, with no duplicated guard logic across cards.
+- Largest blast radius on the most-tested hot paths; breaks the one-at-a-time shared-branch discipline (every card now depends on the refactor landing first), and a single pipeline makes "default-off ⇒ ranking bit-identical" far harder to prove per-card, directly stressing the core convention.
+Complexity: large
+Risk: high
+
+## Recommended: Variant 1
+Variant 1 is the only option that natively honors o2b's load-bearing conventions. Each card stays a self-contained module behind its own explicit switch whose off-branch is provably bit-identical — the precondition the repo demands for any ranking/cost change (D4, D5) and the easiest property to verify when integration is one guarded call site rather than a shared pipeline (Variant 3) or a pre-committed policy interface (Variant 2). It preserves the demonstrated release pattern (themed multi-card suite, driven sequentially, landing as one PR) and keeps the kernel LLM-free: D4's stop and D5's trigger remain deterministic guards at their own sites. Non-conflict on the shared branch is achieved by the cheapest sufficient mechanism — ordering the only two real collision pairs (D2→D3 on citation rendering; D4→D5 on `search.ts`) so the later card extends an already-merged seam, while D1 and D7 stay fully orthogonal. D2 stays read-time-only (pure pointer slicing, deterministic across Syncthing peers), D3 reuses the existing store read instead of a new index, D5 reuses `coverage.uncoveredRareTerms` + existing expansion capped at one pass, and D7 narrows to exposing `minTokens` without perturbing chunk hashes. Telemetry on every card routes through `emitGatedTelemetry` and stays fail-open. Borrow exactly one idea from Variant 2 — a shared `Lstart-Lend` pointer type used by both D2 and D5 — but introduce it inside the D2 card rather than as a speculative up-front seam, so no abstraction is committed ahead of the card that proves its shape.
diff --git a/docs/brainstorm/recall-precision-provenance/cli-output/prompt.md b/docs/brainstorm/recall-precision-provenance/cli-output/prompt.md
new file mode 100644
index 00000000..644953a4
--- /dev/null
+++ b/docs/brainstorm/recall-precision-provenance/cli-output/prompt.md
@@ -0,0 +1,101 @@
+You are a senior backend architect reviewing a feature scope for an existing, mature TypeScript codebase.
+Produce an architecture review. NO CODE. Follow the output format EXACTLY.
+
+# Project: Open Second Brain (o2b)
+
+- Runtime: Bun (also runs on Node). Language: TypeScript. Pure filesystem + SQLite, no daemon.
+- Domain: an Obsidian-vault-native memory layer for AI agents. Plain Markdown on disk under `Brain/`. Deterministic by design: counters, atomic file moves, hash-stable chunks. The kernel never calls an LLM for its own logic; agents own generation, the vault owns the durable provenanced record.
+- Core principle: "you own plain Markdown." Storage stays verbatim; presentation is computed at read time. Syncthing replication across peers, so any persisted value must be hash-stable across machines.
+- Conventions: SOLID/KISS/DRY, no misleading fallbacks, no hardcoding, English-only strings (logic must be language-agnostic, no per-language word lists). All new behavior ships behind an explicit switch whose default keeps ranking/behavior bit-identical when the switch is off. Continuity records are append-only JSONL; existing JSONL is never migrated; additive optional fields do NOT bump schema version. Telemetry is fail-open (must never block the primary op) and routed through `emitGatedTelemetry(gate, build)` so the build thunk is never invoked when the gate is off.
+
+# Release scope: "Recall precision, coverage & provenance hardening"
+
+Six concrete-leaf cards ship together on branch `feat/recall-precision-provenance`. They are the primary user-facing function of a second brain: accurate, complete, auditable recall.
+
+## D1 — Context traces attached to logged events (provenance join)
+Upstream (signetai v0.144.0) attaches context traces directly to logged events so an operator can answer "why did the agent do this" from one surface.
+Status in o2b: present_weaker. o2b ALREADY emits both Brain log events (`appendLogEvent()` → `Brain/log/<date>.md` + JSONL sidecar) AND rich continuity records (`recall_telemetry`, `context_receipt`, `gate_telemetry`, `generation_report`, `session_turn`, `session_summary_node`), but they are queried by SEPARATE readers. The correlation-ID scaffolding (`sourceRefs`, `session_id`, `turn_id`, `handoffKind/handoffRef`, `timestamp`, `agent`) already exists and is the natural join key.
+NOT FOUND: a single reader that, given a log event via its correlation IDs, resolves and displays the attached context trace (context_receipt / recall_telemetry / generation_report). o2b has no web dashboard, so the o2b form is a CLI/MCP reader. Reader infrastructure exists: `brain_recall_telemetry`, `brain_context_receipts`, `brain_generation_reports` (MCP) + `o2b brain recall-telemetry|context-receipts|generation-reports` (CLI). Continuity read-model (`src/core/brain/continuity/read-model.ts`) already normalizes records and lifts `session_id`/`turn_id`/handoff fields to first-class join fields.
+
+## D2 — Read-time virtual line numbering for precise line-span citations
+Upstream (mempalace) ships two pure functions: `render_with_line_numbers(text, start_line=1)` and `extract_line_range(text, line_start, line_end)` that apply `[N]` line markers at READ time only. A pointer `path:Lstart-Lend` resolves by slicing the range; bytes on disk are untouched.
+Status in o2b: not_in_project_useful. o2b cites at path level + a char-offset snippet (`snippet()` in `src/core/brain/session-recall.ts:510`; `EvidencePack` in `src/core/search/evidence-pack.ts:44`). NO line-anchored citation exists. Line-range pointers sharpen the recall benchmark's citation-depth/answer-containment signals and point a reader to exact lines of a long note.
+Pure read-time functions, no storage migration. Implement as helpers next to the search/snippet path; plumb a `Lstart-Lend` pointer form into evidence-pack/citation rendering.
+
+## D3 — Generalize progressive search→expand→transcript disclosure to ALL recall
+Upstream (memsearch) describes 3-layer recall (search → expand → transcript): compact cards (layer 1), expand a hit to the fuller note (layer 2), drill to raw transcript/source (layer 3). Token-cheap by default; pay for depth on demand.
+Status in o2b: present_weaker. o2b ALREADY has this pattern but ONLY for session recall: `searchSessionRecall` returns snippet hits, `expandSessionRecall` walks hit → immediate sources → paginated raw turns (`src/core/brain/session-recall.ts`). The GENERAL vault search (`src/core/search/search.ts`) returns a FLAT ranked list with full content per result and per-layer score breakdowns (`src/core/search/types.ts`) — no compact-card-then-expand depth contract. NOTE: `expand` flag in search/types.ts is query-lane expansion, NOT result-depth disclosure (different meaning). Generalizing search→expand→transcript to the main recall surface serves context-budget goals (context-presets, pre-compress). The session-recall expand machinery is the proven blueprint to lift up a layer. The expand step must reuse the existing store read, not a new index.
+
+## D4 — Normalized-confidence chain-stop for early termination of multi-scope recall
+Upstream (mem9) gates early termination of multi-scope recall on a NORMALIZED-confidence threshold (`MNEMO_CHAIN_RECALL_STOP_SCORE`, default 0.8): stop querying later scopes once one reaches the threshold. Crucially uses normalized confidence, not raw score (a high raw score on a tiny corpus must not trigger the stop).
+Status in o2b: present_weaker. o2b's cross-vault recall (`src/core/search/cross-vault.ts`) runs the SAME query across every configured vault and merges — no early termination, so N vaults always pay N searches even when the first confidently answers. o2b already has confidence, query planning (`query-plan.ts`), query caching (`query-cache.ts`), and a retrieve-or-not surfacing-gate (`surfacing-gate.ts` — a DIFFERENT decision: WHETHER to retrieve, keep distinct). NOT FOUND: nothing decides WHEN to stop chaining scopes. Threshold MUST gate on normalized confidence, not raw score. Deterministic, no LLM. BrainSearchResult already carries a final normalized [0,1] score (relevance threshold drops results below it).
+
+## D5 — Coverage-driven targeted self-correcting recall on partial misses
+Upstream (argus) extends self-correcting recall to fire on PARTIAL evidence (low coverage), not only zero-result, and to issue TARGETED follow-up queries built from specifically-missing facts rather than generic broadening.
+Status in o2b: present_weaker. o2b computes IDF-weighted coverage and uncovered rare terms (`src/core/search/coverage.ts` `CoverageReport.uncoveredRareTerms`, `COMPLETENESS_COMPLETE_THRESHOLD`), AND has a self-correcting two-pass recall — but that retry triggers ONLY on a ZERO-candidate first pass in evidence-pack mode and broadens generically with a single OR pass (`src/core/search/types.ts:424` `secondPass`, `twoPassEnabled`; `search.ts`). So when the first pass returns results that nonetheless leave rare terms uncovered (partial miss), o2b does NOT re-query. Wiring coverage's `uncoveredRareTerms` into a targeted follow-up pass (queries derived from the uncovered terms via existing expansion `query-expansion.ts`/`synonyms.ts`), gated on the existing completeness threshold, closes the loop. o2b already abstains rather than guesses. Deterministic trigger (coverage below threshold), no LLM for the trigger. Cap the loop (one targeted follow-up pass) to bound cost, matching the existing single-retry discipline.
+
+## D7 — Make the markdown chunker's token budgets configurable per vault (NOT hardcoded)
+Upstream (hindsight v0.8.3) exposes chunker granularity as operator-tunable config.
+**IMPORTANT CORRECTION TO THE CARD'S PREMISE (verified in source):** The card claims "the sole caller `indexInto` passes nothing, so chunk size is pinned to hardcoded constants." This is PARTIALLY STALE. Verified facts in the live tree:
+- `chunkMarkdown(text, filenameBase, opts)` at `src/core/search/chunker.ts:449` accepts optional `ChunkOptions { maxTokens?, minTokens?, overlapTokens? }`.
+- Hardcoded defaults at `chunker.ts:11-13`: `DEFAULT_MAX_TOKENS=800`, `DEFAULT_MIN_TOKENS=100`, `DEFAULT_OVERLAP_TOKENS=100`.
+- The sole caller `indexInto` at `src/core/search/indexer.ts:263-265` ALREADY passes `{ maxTokens: config.chunkSize, overlapTokens: config.chunkOverlap }`.
+- `resolveSearchConfig` (`src/core/search/index.ts`) ALREADY resolves `chunkSize` (default 800) and `chunkOverlap` (default 100) from config key `search_chunk_size` / `search_chunk_overlap` AND env `OPEN_SECOND_BRAIN_SEARCH_CHUNK_SIZE` / `OPEN_SECOND_BRAIN_SEARCH_CHUNK_OVERLAP`, with validation (`chunkOverlap < chunkSize`).
+So `maxTokens` and `overlapTokens` ARE already operator-tunable via config. What is NOT yet configurable: `minTokens` (the chunk-packing floor, hardcoded DEFAULT_MIN_TOKENS=100, has no config/env surface and is not threaded into the indexer call). The chunker also carries a determinism contract ("hashes the same chunks on every Syncthing peer") — any per-vault config must stay stable/shared across peers or it churns chunk hashes. Consider whether this card collapses to "expose `minTokens`" only, or whether it should be reframed.
+
+# Recent git log (most recent first)
+```
+da2e3cc feat: memory subsystem alignment - honest pinned budgets, atomic batch writes, on_memory_write host bridge (v1.16.0) (#107)
+4db7862 fix(hermes): pass --repo so bridge skill discovery resolves repoRoot (#103) (#106)
+0a4b6da feat: calendar obligations, agenda synthesis, OKF portability, Obsidian Bases and steelman synthesis (v1.15.0) (#105)
+f8b4abf feat(brain): add feedback default scope and vault write containment (#104)
+20ea7ef feat: per-handoff LLM generation tracing and prompt-prefix stability metric (#102)
+9c1d48f feat: CodeGraph and MCP operational readability (v1.12.0) (#101)
+c2c3ff4 feat: Session Knowledge Synthesis Suite - structured session summaries, idea-lineage, episodic note history (v1.11.0) (#100)
+56dd3dd fix(hermes): bridge EOF - byte streams, stderr drain, retry loop (#92)
+35b824e feat: Recall & Working-Memory Quality Suite - selectable profiles, usage decay, co-occurrence, file-context (v1.10.0) (#99)
+929d54c feat: Brain Portability & Interop Suite - bank export/import, page contract, brain_create_note, in-process SDK (v1.9.0) (#98)
+8b679fe feat: Knowledge Provenance Suite - ingest, research, NER, derived facts, owner-scope, standing-query (v1.7.0) (#96)
+6e59a42 feat: Vault Integrity & Trust Suite - untrusted-source containment, NFC identity, watch-sync, O(1) graph, agent-scope (v1.6.0) (#95)
+e4df212 feat: Search & Recall Quality Suite - explainable scores, trust, threshold, reinforce, eval (#93)
+```
+Pattern: each release is a themed multi-card suite landing as ONE PR. Recall/search quality is a recurring theme. o2b already has a rich search-quality substrate (coverage, IDF, evidence-pack, two-pass, cross-vault, query-plan/cache, surfacing-gate, confidence, reinforcement).
+
+# Constraints to honor
+- Default-off / bit-identical-when-off switches for any new ranking or cost behavior (D4, D5 especially).
+- Telemetry fail-open via emitGatedTelemetry; never block the primary op.
+- Read-time-only for D2: never mutate stored Markdown; pointers must not invalidate on re-mine (deterministic numbering).
+- D3 expand must reuse existing store reads, not a new index.
+- D4 must gate on NORMALIZED confidence; stay distinct from the surfacing-gate (whether-to-retrieve) and from query-cache (which is per-request, not cross-scope chain order).
+- D5 must cap at one targeted follow-up pass; deterministic trigger; reuse existing coverage + expansion machinery.
+- D7: the per-vault determinism contract across Syncthing peers must not break; chunk hashes must stay stable.
+- Cards are driven ONE AT A TIME on the shared branch, so designs must not overlap/conflict (e.g. D4 and D5 both touch recall search.ts; D3 and D2 both touch result/citation rendering).
+
+# Your output — EXACTLY this structure, nothing else
+
+## Variant 1
+Approach: <2-3 sentences>
+Trade-offs:
+- <bullet>
+- <bullet>
+Complexity: small|medium|large
+Risk: low|medium|high
+
+## Variant 2
+Approach: <2-3 sentences>
+Trade-offs:
+- <bullet>
+- <bullet>
+Complexity: small|medium|large
+Risk: low|medium|high
+
+## Variant 3
+Approach: <2-3 sentences>
+Trade-offs:
+- <bullet>
+- <bullet>
+Complexity: small|medium|large
+Risk: low|medium|high
+
+## Recommended: Variant N
+<rationale: which variant best fits o2b's "plain Markdown you own, deterministic, default-off, fail-open, no-LLM-kernel, one-PR-suite" conventions, and how it keeps the 6 cards non-conflicting on the shared branch.>
diff --git a/docs/brainstorm/recall-precision-provenance/design.md b/docs/brainstorm/recall-precision-provenance/design.md
new file mode 100644
index 00000000..16a515bf
--- /dev/null
+++ b/docs/brainstorm/recall-precision-provenance/design.md
@@ -0,0 +1,166 @@
+# Design — Recall precision, coverage & provenance hardening
+
+Branch: `feat/recall-precision-provenance`
+Slug: `recall-precision-provenance`
+Cards: D1 `t_8da11868`, D2 `t_0b83c97b`, D3 `t_468190f5`, D4 `t_23c1b929`, D5 `t_8eb5ca32`, D7 `t_122b2cbc`
+
+## Problem
+
+Open Second Brain's primary user-facing function is recall that is accurate, complete, and
+auditable. Today the recall and observability substrate is rich (coverage engine, IDF-weighted
+evidence packs, cross-vault search, query planning/caching, a retrieve-or-not surfacing gate,
+session-recall expand, continuity records, generation tracing) but six concrete gaps leave recall
+less precise, less complete, and less auditable than it can be:
+
+1. **Provenance is split across surfaces.** Log events and context traces both exist but are read
+ by separate readers; an operator cannot answer "why did the agent do this" from one surface.
+2. **Citations are path + char-offset only.** A recalled note points at "somewhere in this file",
+ not at an exact line span, blunting citation-depth/answer-containment signals.
+3. **General recall is flat.** The session-recall surface already does compact-card → expand →
+ raw-transcript disclosure, but the main vault search returns full content per hit, paying full
+ token cost on every result regardless of whether the agent drills deeper.
+4. **Multi-scope recall never short-circuits.** Cross-vault search always pays N searches even
+ when the first vault already answers confidently.
+5. **Self-correction only fires on zero candidates.** A first pass that returns results but leaves
+ rare query terms uncovered (a partial miss) does not re-query for those terms.
+6. **The chunker's min-token floor is hardcoded.** (max-tokens and overlap-tokens are already
+ operator-tunable; only the packing floor is not.)
+
+## Scope
+
+All six cards ship in this release, driven one at a time on the shared branch:
+
+- **D7** `t_122b2cbc` — expose the markdown chunker's `minTokens` floor as per-vault config.
+- **D1** `t_8da11868` — a read-only correlation-join reader that attaches context traces to a
+ logged event via its correlation IDs.
+- **D2** `t_0b83c97b` — read-time virtual line numbering (`Lstart-Lend` pointers), introducing a
+ shared `LinePointer` type reused by D5.
+- **D3** `t_468190f5` — generalize progressive search→expand→transcript disclosure to all recall.
+- **D4** `t_23c1b929` — normalized-confidence chain-stop for cross-vault early termination.
+- **D5** `t_8eb5ca32` — coverage-driven targeted self-correcting recall on partial misses.
+
+## Out of scope
+
+- A web dashboard (o2b has none; provenance ships as CLI/MCP readers).
+- Server-managed shared "Spaces", per-agent tokens, hosted control-planes (mem9 product direction
+ o2b deliberately avoids — only the chain-stop transfers).
+- An LLM inside any kernel logic. Every new decision (chain-stop threshold, coverage trigger,
+ pointer resolution) is deterministic.
+- HyDE-style generative query rewriting. D5 derives targeted follow-up queries from uncovered rare
+ terms via the existing deterministic expansion machinery, not a model call.
+- Migrating existing JSONL or stored Markdown. D2 is read-time-only; D1 reads existing records.
+
+## Chosen approach
+
+**Variant 1 — additive, independently-switched modules, ordered by file-collision.**
+
+Each card lands as a self-contained module behind its own explicit switch whose off-branch is
+provably bit-identical to today — the precondition o2b demands for any ranking or cost change (D4,
+D5) and the easiest property to verify when integration is one guarded call site rather than a
+shared pipeline. The cards are driven one at a time on the shared branch in an order that resolves
+the only two real collision pairs:
+
+- **D2 → D3** (both touch citation/evidence rendering): D2 introduces the `LinePointer` primitive
+ and citation rendering seam; D3 extends result rendering with disclosure tiers over it.
+- **D2 → D5** (shared pointer type): D2 introduces `LinePointer`; D5 reuses it to cite newly-found
+ evidence in the targeted follow-up pass.
+- **D4 → D5** (both touch `search.ts`/`cross-vault.ts`): D4 adds the chain-stop guard at its own
+ call site; D5 adds the coverage-retry guard at a distinct site, building on the seam D4 left.
+
+D1 (read-only correlation-join reader) and D7 (config-only `minTokens` exposure) are fully
+orthogonal and slot in at the front of the order. Telemetry on every card routes through
+`emitGatedTelemetry` and stays fail-open.
+
+**Drive order:** D7 → D1 → D2 → D3 → D4 → D5.
+
+## Design decisions
+
+### D7 — expose `minTokens` (card premise partially corrected)
+The card claims the sole caller passes nothing so chunk size is pinned to hardcoded constants.
+Verified in the live tree this is **partially stale**: `indexer.ts:263-265` already passes
+`{ maxTokens: config.chunkSize, overlapTokens: config.chunkOverlap }`, and `resolveSearchConfig`
+already resolves both from `search_chunk_size`/`search_chunk_overlap` config + env with validation.
+What is **not** yet configurable is `minTokens` (the chunk-packing floor, `DEFAULT_MIN_TOKENS`),
+which has no config/env surface and is not threaded into the indexer call. D7 narrows to exposing
+`minTokens` symmetrically. The default stays equal to the existing constant so chunk hashes are
+unchanged when no config is set (honoring the Syncthing-peer determinism contract).
+
+### D2 — read-time-only `Lstart-Lend`, no storage mutation
+Two pure functions (`renderWithLineNumbers`, `extractLineRange`) plus a `LinePointer` type and
+parse/format helpers for `path:Lstart-Lend`. Numbering is deterministic (1-based from file start),
+computed at read time only; bytes on disk are never mutated, so existing pointers never invalidate
+and idempotent re-mining never shifts numbering. D2 owns the `LinePointer` type so D5 can import it
+rather than re-defining it.
+
+Line-span provenance for an evidence record comes from `BrainSearchResult.startLine`/`endLine`
+(types.ts:84-85), which are persisted in the `chunks` table (`start_line`/`end_line`,
+schema.ts:42-43). `EvidenceRecord` (evidence-pack.ts:10-20) does NOT currently carry them —
+`buildEvidencePack` drops them when projecting `BrainSearchResult` → `EvidenceRecord`. D2 threads
+`lineStart`/`lineEnd` into `EvidenceRecord` (additive, defaulting to the source result's values) so a
+`LinePointer` can be rendered per record without a chunk-table lookup.
+
+### D3 — disclosure tiers over the existing store read
+The session-recall expand machinery (`searchSessionRecall` / `expandSessionRecall`) is the proven
+blueprint. D3 adds a compact-card result form and an `expandHit` step to the general search that
+reuses the existing store read for layer 2 (fuller note) and layer 3 (raw chunk), **not** a new
+index. Default behavior stays full-content (bit-identical); `disclosure: 'cards'` opts into the
+tiered contract. The `expand` flag already in `search/types.ts` is query-lane expansion and stays
+distinct.
+
+### D4 — normalized-confidence chain-stop, distinct from the surfacing gate and cache
+After each origin's cross-vault search completes, if the top result's **normalized** [0,1] score is
+>= the configured threshold (default 0.8), remaining origins are skipped and the outcome records
+which were skipped. The stop gates on normalized confidence, never raw score (a high raw score on a
+tiny corpus must not trigger it). It stays distinct from the retrieve-or-not `surfacing-gate`
+(whether to retrieve at all) and from `query-cache` (per-request, not cross-scope chain order).
+Default-off (`chainStopEnabled`) → bit-identical to today.
+
+### D5 — partial-coverage targeted follow-up, capped at one pass
+After the first evidence-pack pass, if `uncoveredRareTerms` is non-empty and results exist (partial
+miss, coverage below `COMPLETENESS_COMPLETE_THRESHOLD`), run exactly **one** targeted follow-up
+pass: derive queries from the uncovered rare terms via existing `query-expansion.ts`/`synonyms.ts`,
+re-query, merge. This is distinct from the existing zero-candidate broadened-OR `secondPass`.
+Deterministic trigger, no LLM, capped at one pass (matching the existing single-retry discipline).
+Default-off (`coverageRetryEnabled`) → bit-identical. Reuses D2's `LinePointer` to cite the
+newly-found evidence.
+
+### D1 — read-only correlation join, fail-open
+A join function resolves, for a given log event's correlation IDs (`session_id`, `turn_id`,
+`handoffRef`, `timestamp`, `agent`, `sourceRefs`), the attached `context_receipt` /
+`recall_telemetry` / `generation_report` via the existing continuity read-model
+(`read-model.ts`, which already lifts session/handoff fields to first-class join keys). Surfaced as
+`brain_context_trace` (MCP) + `o2b brain context-trace` (CLI). Missing records return empty, never
+throw (fail-open).
+
+## File changes (summary; per-card detail in plan.md)
+
+- D7: `src/core/search/index.ts`, `src/core/search/types.ts`, `src/core/search/indexer.ts`; test
+ `tests/core/search/config.test.ts`.
+- D2: `src/core/search/line-citation.ts` (new), `src/core/search/evidence-pack.ts`,
+ `src/core/brain/session-recall.ts`; test `tests/core/search/line-citation.test.ts` (new).
+- D3: `src/core/search/types.ts`, `src/core/search/search.ts`, `src/cli/search.ts`,
+ `src/mcp/brain/recall-tools.ts`; test `tests/core/search/disclosure.test.ts` (new).
+- D4: `src/core/search/cross-vault.ts`, `src/core/search/index.ts`, `src/core/search/types.ts`;
+ test `tests/core/search/cross-vault.test.ts`.
+- D5: `src/core/search/search.ts`, `src/core/search/types.ts`; test
+ `tests/core/search/coverage-retry.test.ts` (new).
+- D1: `src/core/brain/provenance/context-trace.ts` (new), `src/core/brain/log.ts`,
+ `src/mcp/brain/recall-tools.ts`, `src/cli/brain.ts`; test
+ `tests/core/brain/context-trace.test.ts` (new).
+
+## Risks
+
+- **Shared-branch ordering discipline.** D2-before-D3/D5 and D4-before-D5 are enforced by drive
+ order, not the compiler. Mitigation: the plan records `Depends on` per card; a worker starting a
+ card before its dependency lands must `git pull`/rebase first.
+- **Default-off bit-identity for D4/D5.** The hot recall path is the most-tested in the repo.
+ Mitigation: each card's acceptance test asserts the off-branch is bit-identical to current
+ output (golden/snapshot), not just "runs without error".
+- **D7 determinism contract.** A per-vault `minTokens` that differs across Syncthing peers churns
+ chunk hashes. Mitigation: default == existing constant; document that per-vault deviation must be
+ set identically on all peers (mirrors the existing chunkSize/overlap guidance).
+- **D1 join correctness.** Correlation IDs span several fields; a too-loose join could attach the
+ wrong trace. Mitigation: join on the read-model's lifted first-class fields with explicit
+ match semantics; tests cover the no-match (empty) and partial-match cases.
+- **D5 follow-up cost.** An unbounded retry loop would defeat the purpose. Mitigation: hard cap at
+ one targeted follow-up pass, deterministic trigger from the existing completeness threshold.
diff --git a/docs/brainstorm/recall-precision-provenance/plan.md b/docs/brainstorm/recall-precision-provenance/plan.md
new file mode 100644
index 00000000..d70eea1d
--- /dev/null
+++ b/docs/brainstorm/recall-precision-provenance/plan.md
@@ -0,0 +1,208 @@
+# Implementation plan — Recall precision, coverage & provenance hardening
+
+Branch: `feat/recall-precision-provenance`. Cards are driven ONE AT A TIME on this shared branch.
+Each worker MUST `git pull` / build on the commits previously-driven in-scope cards already landed,
+and must not duplicate or conflict with sibling tasks. Follow TDD: write the failing test first.
+
+Combined design: `docs/brainstorm/recall-precision-provenance/design.md`.
+Drive order: **D7 → D1 → D2 → D3 → D4 → D5**.
+
+---
+
+## D7 — t_122b2cbc — Expose chunker `minTokens` as per-vault config
+
+**Note:** `maxTokens` and `overlapTokens` are ALREADY operator-tunable
+(`indexer.ts:263-265` passes `config.chunkSize`/`config.chunkOverlap`; `resolveSearchConfig` reads
+`search_chunk_size`/`search_chunk_overlap` config + env with validation). D7 narrows to exposing
+the **only** missing knob: the chunk-packing floor `minTokens` (`DEFAULT_MIN_TOKENS`). The default
+MUST equal the existing constant so chunk hashes are unchanged when no config is set (Syncthing-peer
+determinism contract).
+
+### Files
+- `src/core/search/index.ts` — add `minTokens` to `DEFAULTS` (== existing `DEFAULT_MIN_TOKENS`),
+ resolve it from `search_chunk_min_tokens` config + `OPEN_SECOND_BRAIN_SEARCH_CHUNK_MIN_TOKENS` env
+ via `parseInteger`, with validation (`0 <= minTokens < chunkSize`), add to `ResolvedSearchConfig`
+ return shape.
+- `src/core/search/types.ts` — add `readonly chunkMinTokens: number;` to `ResolvedSearchConfig`.
+- `src/core/search/indexer.ts:263-265` — thread `minTokens: config.chunkMinTokens` into the
+ existing `chunkMarkdown(...)` call (alongside the already-present `maxTokens`/`overlapTokens`).
+- `tests/core/search/config.test.ts` — new/extended cases.
+
+### Acceptance (passing test)
+A test in `tests/core/search/config.test.ts` that:
+1. Resolves config with `search_chunk_min_tokens` set and asserts `chunkMinTokens` carries through;
+2. Asserts the DEFAULT (no config) yields a value equal to `chunker.DEFAULT_MIN_TOKENS` (import the
+ constant) so chunk hashes are byte-identical to pre-change — the bit-identity guard;
+3. Asserts validation rejects `minTokens >= chunkSize`.
+
+### Depends on
+None. Orthogonal. (Driven first.)
+
+---
+
+## D1 — t_8da11868 — Context traces attached to logged events (provenance join)
+
+A read-only correlation-join reader. Given a log event's correlation IDs, resolve the attached
+`context_receipt` / `recall_telemetry` / `generation_report` via the existing continuity
+read-model (`read-model.ts`, which already lifts `session_id`/`turn_id`/handoff fields to
+first-class join keys). Missing records return empty, never throw (fail-open). Surfaced as
+`brain_context_trace` (MCP) + `o2b brain context-trace` (CLI).
+
+### Files
+- `src/core/brain/provenance/context-trace.ts` (new) — pure join function
+ `resolveContextTrace(vault, correlation)` returning a frozen shape `{ logEvent, receipts,
+ recallTelemetry, generationReports }`. Join on the read-model's lifted first-class fields
+ (`session_id`, `turn_id`, `handoffRef`, `timestamp`, `agent`, `sourceRefs`). No new writes.
+- `src/core/brain/log.ts` — add a small read helper to load a single log event row by its JSONL
+ sidecar id/timestamp if a join-by-id is needed (read-only; do not touch `appendLogEvent`).
+- `src/mcp/brain/recall-tools.ts` — add `brain_context_trace` op (input: correlation fields or a
+ log event id; output: the joined trace). Fail-open: missing → empty arrays.
+- `src/cli/brain.ts` — add `context-trace` subcommand.
+- `tests/core/brain/context-trace.test.ts` (new).
+
+### Acceptance (passing test)
+A test in `tests/core/brain/context-trace.test.ts` over a fixture vault with a log event and
+matching + non-matching continuity records that asserts:
+1. Given a log event's `session_id`+`turn_id`, the reader returns exactly the matching
+ `context_receipt` / `recall_telemetry` / `generation_report`;
+2. A log event with NO matching records returns empty arrays (not an error) — the fail-open guard;
+3. A partial-match (only `recall_telemetry` present) returns that one and empty for the rest.
+
+### Depends on
+None. Orthogonal.
+
+---
+
+## D2 — t_0b83c97b — Read-time virtual line numbering (`Lstart-Lend` pointers)
+
+Pure read-time functions + a `LinePointer` type (introduced here, reused by D5). Deterministic
+1-based numbering from file start; bytes on disk never mutated.
+
+### Files
+- `src/core/search/line-citation.ts` (new) — `LinePointer { path, lineStart, lineEnd }`, pure
+ `renderWithLineNumbers(text, startLine?)`, `extractLineRange(text, lineStart, lineEnd)`,
+ `parseLinePointer(str)` / `formatLinePointer(ptr)` for `path:Lstart-Lend` form.
+- `src/core/search/evidence-pack.ts` — add `lineStart`/`lineEnd` to `EvidenceRecord` (additive;
+ `buildEvidencePack` carries them over from `BrainSearchResult.startLine`/`endLine`, which it
+ already receives). Render a `LinePointer` for each `EvidenceRecord` when line-citation is
+ requested. NOTE: `EvidenceRecord` does not currently carry line spans — the source values live on
+ `BrainSearchResult` (types.ts:84-85) and the `chunks` table (schema.ts:42-43); threading them in
+ here is the additive change D2 makes.
+- `src/core/brain/session-recall.ts` — plumb the `Lstart-Lend` form into the snippet/citation path
+ (`snippet()` at ~510) as an alternative to char-offset.
+- `tests/core/search/line-citation.test.ts` (new).
+
+### Acceptance (passing test)
+A test in `tests/core/search/line-citation.test.ts` that:
+1. `renderWithLineNumbers` applies `[N]` markers 1-based and is a pure function (same input → same
+ output, never mutates the input string);
+2. `extractLineRange(text, 55, 72)` returns exactly lines 55-72 inclusive;
+3. `parseLinePointer("note.md:L55-L72")` round-trips through `formatLinePointer`;
+4. Re-running render on an unchanged file yields identical numbering (idempotent — the
+ "pointers never invalidate" invariant).
+
+### Depends on
+None (introduces the `LinePointer` type D5 will import).
+
+---
+
+## D3 — t_468190f5 — Generalize progressive disclosure to all recall
+
+Compact-card → expand → raw-transcript disclosure for the general search, mirroring
+`searchSessionRecall`/`expandSessionRecall`. Reuse the existing store read for layer 2/3, NOT a new
+index. Default behavior stays full-content (bit-identical); `disclosure: 'cards'` opts in. The
+existing query-lane `expand` flag stays distinct.
+
+### Files
+- `src/core/search/types.ts` — add a `disclosure` option (`'full' | 'cards'`, default `'full'`) and
+ a compact `SearchCard` result shape (path, title, score, reasons, snippet, pointer) for layer 1.
+- `src/core/search/search.ts` — when `disclosure === 'cards'`, return compact cards (layer 1); add
+ an `expandHit(vault, hitId)` returning layer 2 (fuller note) and a layer-3 raw chunk read via the
+ existing store, reusing `expandSessionRecall`'s paginated-cursor pattern.
+- `src/cli/search.ts` — add `--disclosure cards` flag + a `search expand` subcommand.
+- `src/mcp/brain/recall-tools.ts` — add `disclosure` field to `brain_search` + a
+ `brain_search_expand` op.
+- `tests/core/search/disclosure.test.ts` (new).
+
+### Acceptance (passing test)
+A test in `tests/core/search/disclosure.test.ts` that:
+1. `disclosure: 'cards'` returns compact cards (no full content) for layer 1;
+2. `expandHit` on a card returns the fuller note (layer 2) and the raw chunk (layer 3);
+3. Default (no `disclosure`) is bit-identical to pre-change output — the bit-identity guard;
+4. The layer-2/3 expand reuses the existing store read (assert no new index file is created).
+
+### Depends on
+D2 (renders `LinePointer` citations in the compact cards / expanded note).
+
+---
+
+## D4 — t_23c1b929 — Normalized-confidence chain-stop for cross-vault recall
+
+After each origin completes, if the top result's NORMALIZED [0,1] score >= threshold (default 0.8),
+skip remaining origins and record which were skipped. Gates on normalized confidence, never raw
+score. Distinct from the retrieve-or-not `surfacing-gate` and from `query-cache`. Default-off
+(`chainStopEnabled`) → bit-identical.
+
+### Files
+- `src/core/search/types.ts` — add `chainStopEnabled: boolean` and `chainStopScore: number` to
+ `ResolvedRecallConfig`; add a `chainStop?` field to the cross-vault outcome recording
+ `{ triggered: true, stoppedAfter: <label>, skipped: [<labels>] }`.
+- `src/core/search/index.ts` — resolve `chain_stop_enabled` (config + env, default false) and
+ `chain_stop_score` (default 0.8) with `[0,1]` range validation.
+- `src/core/search/cross-vault.ts` — in the per-origin loop, after pushing results, if
+ `chainStopEnabled && merged.length && topNormalizedScore >= chainStopScore`, break and record
+ `skipped` = remaining origin labels. Top score is the normalized result `score` (already [0,1]).
+- `tests/core/search/cross-vault.test.ts` — new cases.
+
+### Acceptance (passing test)
+A test in `tests/core/search/cross-vault.test.ts` with a multi-origin fixture that:
+1. With `chainStopEnabled` on and the first origin's top normalized score >= 0.8, later origins are
+ NOT searched and `chainStop.skipped` lists them;
+2. A high RAW score on a tiny-corpus origin that yields a low NORMALIZED score does NOT trigger the
+ stop (the normalized-not-raw guard);
+3. `chainStopEnabled` off (default) runs all origins, bit-identical to today, and `chainStop` is
+ absent — the bit-identity guard.
+
+### Depends on
+None for logic; drives before D5 so D5 builds on a settled `search.ts`/`cross-vault.ts` seam.
+
+---
+
+## D5 — t_8eb5ca32 — Coverage-driven targeted self-correcting recall on partial misses
+
+After the first evidence-pack pass, if `uncoveredRareTerms` is non-empty AND results exist (partial
+miss, coverage below `COMPLETENESS_COMPLETE_THRESHOLD`), run exactly ONE targeted follow-up pass:
+derive queries from the uncovered rare terms via existing `query-expansion.ts`/`synonyms.ts`,
+re-query, merge. Distinct from the existing zero-candidate broadened-OR `secondPass`. Deterministic
+trigger, no LLM, capped at one pass. Default-off (`coverageRetryEnabled`) → bit-identical. Reuses
+D2's `LinePointer` to cite newly-found evidence.
+
+### Files
+- `src/core/search/types.ts` — add `coverageRetryEnabled: boolean` to `ResolvedRecallConfig`; add a
+ `coverageRetry?` field to `SearchOutcome` recording `{ triggered: true, uncoveredTerms:
+ [...], added: <n> }`.
+- `src/core/search/index.ts` — resolve `coverage_retry_enabled` (config + env, default false).
+- `src/core/search/search.ts` — after the existing zero-candidate `secondPass` block, add a distinct
+ branch: when `evidencePack && coverageRetryEnabled && results.length > 0 &&
+ evidencePack.uncoveredRareTerms?.length` , build ONE targeted query from the uncovered rare terms
+ via `deriveExpansionTerms`/`tokenizeForExpansion`, re-query (`runFtsQueryDetailed`), merge any new
+ hits, update the evidence pack, record `coverageRetry`. Hard cap: one pass. NOTE:
+ `deriveExpansionTerms` lives in **`synonyms.ts`** (not `query-expansion.ts`); `tokenizeForExpansion`
+ is also in `synonyms.ts`. `query-expansion.ts` imports it for internal use only (no re-export), so
+ import both helpers directly from `synonyms.ts`.
+- `tests/core/search/coverage-retry.test.ts` (new).
+
+### Acceptance (passing test)
+A test in `tests/core/search/coverage-retry.test.ts` over a fixture vault where the first pass
+returns results but leaves a rare term uncovered that:
+1. With `coverageRetryEnabled` on, the targeted follow-up re-queries for the uncovered rare term and
+ merges newly-found evidence, recording `coverageRetry.triggered` with the uncovered terms;
+2. The retry runs exactly ONCE (a second uncovered term after the retry does not trigger a third
+ pass) — the cap guard;
+3. `coverageRetryEnabled` off (default) is bit-identical to today and `coverageRetry` is absent —
+ the bit-identity guard;
+4. A ZERO-candidate first pass still uses the existing `secondPass` (not this path) — the
+ distinctness guard.
+
+### Depends on
+D2 (`LinePointer` import for citing new evidence) and D4 (settled `search.ts` seam to extend).
diff --git a/docs/brainstorm/recall-precision-provenance/variants.md b/docs/brainstorm/recall-precision-provenance/variants.md
new file mode 100644
index 00000000..428a6de2
--- /dev/null
+++ b/docs/brainstorm/recall-precision-provenance/variants.md
@@ -0,0 +1,85 @@
+# Architecture variants — Recall precision, coverage & provenance hardening
+
+Branch: `feat/recall-precision-provenance`. Consultant output is preserved verbatim in
+`cli-output/claude.md`; this file restates the three variants verbatim and records the final
+decision and rationale.
+
+## Variant 1
+Approach: Drive the six cards one at a time as additive, independently-switched modules, ordering
+only the two pairs that share a file so the second builds on a seam the first already laid: land
+D2's read-time `Lstart-Lend` pointer primitive before D3's disclosure-depth layering (both touch
+citation/evidence rendering), and land D4's chain-stop before D5's coverage-retry (both touch
+`search.ts`/cross-vault). D1 (read-only correlation-join reader) and D7 (expose `minTokens`) are
+orthogonal and can land in any slot. Each card integrates at a single guarded call site whose
+default-off branch is bit-identical to today.
+Trade-offs:
+- Minimal blast radius and easy per-card review/revert; matches the "one PR, many cards, each
+ behind a switch" cadence the git log already follows.
+- Ordering is a discipline, not a guarantee — a mis-sequenced D5-before-D4 would force a rebase,
+ and the shared-file pairs still serialize (no parallel authoring of D2/D3 or D4/D5).
+Complexity: medium
+Risk: low
+
+## Variant 2
+Approach: Spend the first commit defining the shared seams before any card logic: a `PointerSpec`
+citation type consumed by both D2 and D5, a single ordered recall-decision policy hook in
+`search.ts`/`cross-vault.ts` into which D4 (chain-stop) and D5 (coverage follow-up) register as
+distinct deterministic policies, and a disclosure-layer contract that D3 implements over the
+existing store read. Each card then plugs into its seam as an isolated, separately-flagged
+implementation.
+Trade-offs:
+- Lowest late-stage merge risk: D4/D5 stop colliding because they share an explicit registration
+ point rather than both editing the same hot path, and D2/D3 share one citation type.
+- The seam commit is speculative design ahead of two unwritten cards; if D5's targeted-requery
+ shape diverges from the guessed policy interface, the abstraction churns — and an over-built hook
+ risks accidentally altering the off-path, threatening the bit-identical guarantee.
+Complexity: large
+Risk: medium
+
+## Variant 3
+Approach: Refactor `search.ts`, `cross-vault.ts`, and `evidence-pack.ts` into one unified recall
+pipeline that natively expresses layered disclosure (D3), normalized-confidence chain-stop (D4),
+coverage-driven follow-up (D5), and line-anchored pointers (D2/D5), with per-vault config knobs
+(D7) and the provenance reader (D1) reading the pipeline's emitted traces.
+Trade-offs:
+- Most cohesive end state — one place to reason about precision, coverage, and disclosure together,
+ with no duplicated guard logic across cards.
+- Largest blast radius on the most-tested hot paths; breaks the one-at-a-time shared-branch
+ discipline (every card now depends on the refactor landing first), and a single pipeline makes
+ "default-off ⇒ ranking bit-identical" far harder to prove per-card, directly stressing the core
+ convention.
+Complexity: large
+Risk: high
+
+## Recommended: Variant 1
+Variant 1 is the only option that natively honors o2b's load-bearing conventions. Each card stays a
+self-contained module behind its own explicit switch whose off-branch is provably bit-identical —
+the precondition the repo demands for any ranking/cost change (D4, D5) and the easiest property to
+verify when integration is one guarded call site rather than a shared pipeline (Variant 3) or a
+pre-committed policy interface (Variant 2). It preserves the demonstrated release pattern (themed
+multi-card suite, driven sequentially, landing as one PR) and keeps the kernel LLM-free: D4's stop
+and D5's trigger remain deterministic guards at their own sites. Non-conflict on the shared branch
+is achieved by the cheapest sufficient mechanism — ordering the only two real collision pairs
+(D2→D3 on citation rendering; D4→D5 on `search.ts`) so the later card extends an already-merged
+seam, while D1 and D7 stay fully orthogonal. D2 stays read-time-only (pure pointer slicing,
+deterministic across Syncthing peers), D3 reuses the existing store read instead of a new index, D5
+reuses `coverage.uncoveredRareTerms` + existing expansion capped at one pass, and D7 narrows to
+exposing `minTokens` without perturbing chunk hashes. Telemetry on every card routes through
+`emitGatedTelemetry` and stays fail-open. Borrow exactly one idea from Variant 2 — a shared
+`Lstart-Lend` pointer type used by both D2 and D5 — but introduce it inside the D2 card rather than
+as a speculative up-front seam, so no abstraction is committed ahead of the card that proves its
+shape.
+
+## Final decision: Variant 1 (accepted, with the Variant-2 pointer-type borrow)
+
+Agreed with the consultant's recommendation. Variant 1 is the only variant that natively honors
+o2b's load-bearing conventions — default-off/bit-identical switches, fail-open telemetry, an
+LLM-free kernel, and the demonstrated one-PR-suite release cadence — while keeping the six cards
+non-conflicting on the shared branch via the cheapest sufficient mechanism (ordering the two
+collision pairs). Variant 3's unified-pipeline refactor would break the one-at-a-time discipline
+and make the per-card bit-identity guarantee far harder to prove on the most-tested hot paths.
+Variant 2's speculative seam commit risks churning if D5's requery shape diverges and threatens the
+bit-identical guarantee; the single good idea (a shared pointer type) is borrowed but introduced
+inside the D2 card, not pre-committed.
+
+Drive order: **D7 → D1 → D2 → D3 → D4 → D5**.
diff --git a/docs/cli-reference.md b/docs/cli-reference.md
index 060dd1b1..b9ad118a 100644
--- a/docs/cli-reference.md
+++ b/docs/cli-reference.md
@@ -370,6 +370,8 @@ Recall and ranking quality (v0.20.0), each tunable and bounded:
| `search_synonym_max_terms` | `OPEN_SECOND_BRAIN_SEARCH_SYNONYM_MAX_TERMS` | `3` | Cap on expansion terms OR'd onto the query |
| `search_cache_enabled` | `OPEN_SECOND_BRAIN_SEARCH_CACHE_ENABLED` | `false` | Opt-in persistent query cache, gated by corpus generation |
| `search_cache_ttl_seconds` | `OPEN_SECOND_BRAIN_SEARCH_CACHE_TTL` | `300` | Cache row time-to-live in seconds |
+| `search_chain_stop_enabled`| `OPEN_SECOND_BRAIN_SEARCH_CHAIN_STOP` | `false` | Opt-in cross-vault early termination once an origin answers confidently |
+| `search_chain_stop_score` | `OPEN_SECOND_BRAIN_SEARCH_CHAIN_STOP_SCORE` | `0.8` | Normalized `[0,1]` top-score threshold that triggers the chain-stop |
`brain_context_pack` also accepts `max_chars_per_memory` and
`max_total_chars` (code-point caps). Pass `--lanes` to keep the legacy flat
diff --git a/docs/observability.md b/docs/observability.md
index e3e14858..b7b5f5bd 100644
--- a/docs/observability.md
+++ b/docs/observability.md
@@ -89,6 +89,7 @@ No-consumer regression tests cover each gated surface, so a future call site can
- `brain_recall_telemetry` (MCP) and `o2b brain recall-telemetry list|summary|gate-list|gate-summary` aggregate recall and gate records.
- `brain_context_receipts` and `o2b brain context-receipts` inspect served-context receipts.
+- `brain_event_trace` (MCP) and `o2b brain event-trace` join the Brain log to the continuity store: given a logged event (by day, `--at` stamp, kind, or `--session-id`), they resolve the continuity records attached to it via the stable correlation ids above (`session_id`, `turn_id`, and artifact `sourceRefs`), so an operator can answer "which context was supplied to this action?" from one surface. Each attached record reports `joinedBy` (`session` / `turn` / `artifact`); open the full record through the per-kind readers (`context-receipts show`, `recall-telemetry`, `generation-reports show`). Read-only; `private` records are dropped unless explicitly kept.
- `brain_generation_reports` (MCP, actions `record`/`list`/`summary`) and `o2b brain generation-reports record|list|summary|show` post and read inbound LLM generation traces. `summary` rolls up call counts, the always-present local token estimate, agent-reported usage where present, and a per-path linkage map so a memory path resolves back to the generation reports that produced or consumed it.
- `o2b brain continuity export --format atof|atif` renders the store as standard trajectory formats (ATOF JSONL event stream; ATIF v1.7 one document per session) for replay and eval tooling. Read-only; `private` records never reach an export file. Mapping decisions: `docs/brainstorm/memory-observability-suite/atof-atif-mapping.md`.
- `o2b brain bench memory` measures recall quality over disposable fixture vaults and reports quality, latency, and context cost as separate families (`o2b.bench.v1` report schema).
diff --git a/openclaw.plugin.json b/openclaw.plugin.json
index 7a5984f3..4c0d3097 100644
--- a/openclaw.plugin.json
+++ b/openclaw.plugin.json
@@ -2,7 +2,7 @@
"id": "open-second-brain",
"name": "Open Second Brain",
"description": "Second brain for AI agents using Obsidian-compatible Markdown vaults.",
- "version": "1.17.0",
+ "version": "1.18.0",
"activation": { "onStartup": true },
"skills": ["./skills"],
"contracts": {
diff --git a/package.json b/package.json
index d63a1b85..d1c12836 100644
--- a/package.json
+++ b/package.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "1.17.0",
+ "version": "1.18.0",
"private": false,
"description": "Second brain for AI agents using Obsidian-compatible Markdown vaults. Works with Hermes, Claude Code, Codex, and OpenClaw.",
"keywords": [
diff --git a/plugin.yaml b/plugin.yaml
index c420e6a4..9940b580 100644
--- a/plugin.yaml
+++ b/plugin.yaml
@@ -1,5 +1,5 @@
name: open-second-brain
-version: "1.17.0"
+version: "1.18.0"
description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."
author: "Open Second Brain contributors"
memory_provider: true
diff --git a/plugins/codex/.codex-plugin/plugin.json b/plugins/codex/.codex-plugin/plugin.json
index cd64d223..c302cf33 100644
--- a/plugins/codex/.codex-plugin/plugin.json
+++ b/plugins/codex/.codex-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "1.17.0",
+ "version": "1.18.0",
"description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.",
"author": {
"name": "Open Second Brain contributors",
diff --git a/plugins/hermes/plugin.yaml b/plugins/hermes/plugin.yaml
index c420e6a4..9940b580 100644
--- a/plugins/hermes/plugin.yaml
+++ b/plugins/hermes/plugin.yaml
@@ -1,5 +1,5 @@
name: open-second-brain
-version: "1.17.0"
+version: "1.18.0"
description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."
author: "Open Second Brain contributors"
memory_provider: true
diff --git a/pyproject.toml b/pyproject.toml
index 37431d8d..7ac5c877 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -10,7 +10,7 @@ build-backend = "setuptools.build_meta"
# CLI entry points (those moved to `package.json` `bin`).
[project]
name = "open-second-brain"
-version = "1.17.0"
+version = "1.18.0"
description = "Hermes Python shim for Open Second Brain. Most of the project (CLI, MCP server, OpenClaw plugin) is TypeScript on Bun; see package.json."
readme = "README.md"
requires-python = ">=3.11"
diff --git a/src/cli/brain.ts b/src/cli/brain.ts
index 54c6b786..59ffb396 100644
--- a/src/cli/brain.ts
+++ b/src/cli/brain.ts
@@ -98,6 +98,7 @@ import {
cmdBrainTokenFootprint,
cmdBrainContextPack,
cmdBrainContextReceipts,
+ cmdBrainEventTrace,
cmdBrainContextPresets,
cmdBrainPreCompactExtract,
cmdBrainRecallTelemetry,
@@ -323,6 +324,8 @@ export async function handleBrainSubcommand(argv: ReadonlyArray<string>): Promis
return await cmdBrainContextPack(rest);
case "context-receipts":
return await cmdBrainContextReceipts(rest);
+ case "event-trace":
+ return await cmdBrainEventTrace(rest);
case "context-presets":
return await cmdBrainContextPresets(rest);
case "pre-compact-extract":
diff --git a/src/cli/brain/help-text.ts b/src/cli/brain/help-text.ts
index 69a2a68a..2e83df3a 100644
--- a/src/cli/brain/help-text.ts
+++ b/src/cli/brain/help-text.ts
@@ -81,6 +81,7 @@ Brain verbs (observing memory):
token-footprint Report per-category vault token size with a warn threshold
context-pack Return a tier-then-recency vault slice under a token budget
context-receipts List/show prompt context receipt records
+ event-trace Show context traces attached to logged events by correlation id
context-presets Show/suggest/diff read-only context budget presets
pre-compact-extract Extract typed continuity records from bounded text
recall-telemetry List/summarize opt-in recall telemetry records
@@ -563,6 +564,14 @@ export const VERB_HELP: Record<string, string> = {
"usage: o2b brain context-receipts list [--trigger context_pack|pre_compress] [--host <name>] [--session-id <id>] [--limit <n>] [--vault <path>] [--json]\n" +
" o2b brain context-receipts show <receipt-id> [--vault <path>] [--json]\n" +
"Read prompt context receipt continuity records emitted by opt-in callers.\n",
+ "event-trace":
+ "usage: o2b brain event-trace [--date <YYYY-MM-DD>] [--at <HH:MM:SS>] [--kind <event-kind>] [--session-id <id>] [--limit <n>] [--keep-private] [--vault <path>] [--json]\n" +
+ "Join logged Brain events to the continuity records (recall telemetry, context\n" +
+ "receipts, generation reports, …) attached to them by shared correlation ids\n" +
+ "(session_id, turn_id, artifact refs). Defaults to today's log. Use the per-kind\n" +
+ "readers (context-receipts show, recall-telemetry, generation-reports show) to\n" +
+ "open a listed record in full. Read-only; private records are dropped unless\n" +
+ "--keep-private is set.\n",
"context-presets":
"usage: o2b brain context-presets show [preset-id] [--json]\n" +
" o2b brain context-presets suggest [--model <name>] [--context-window <tokens>] [--json]\n" +
diff --git a/src/cli/brain/verbs/event-trace.ts b/src/cli/brain/verbs/event-trace.ts
new file mode 100644
index 00000000..87d60621
--- /dev/null
+++ b/src/cli/brain/verbs/event-trace.ts
@@ -0,0 +1,108 @@
+import {
+ resolveLogEventTraces,
+ type AttachedTrace,
+ type LogEventTrace,
+} from "../../../core/brain/event-trace.ts";
+import { BRAIN_LOG_EVENT_KIND_SET, type BrainLogEventKind } from "../../../core/brain/types.ts";
+import { brainVerbContext, parse, usageError } from "../helpers.ts";
+
+/**
+ * `o2b brain event-trace` — given one or more logged Brain events,
+ * resolve and display the continuity records (recall telemetry, context
+ * receipts, generation reports, …) attached to each via shared
+ * correlation ids. The integrated "why did the agent do this?" reader:
+ * one surface joining the event log to the served-context trace.
+ */
+export async function cmdBrainEventTrace(argv: string[]): Promise<number> {
+ const { flags } = parse(argv, {
+ vault: { type: "string" },
+ json: { type: "boolean" },
+ date: { type: "string" },
+ at: { type: "string" },
+ kind: { type: "string" },
+ "session-id": { type: "string" },
+ limit: { type: "string" },
+ "keep-private": { type: "boolean" },
+ });
+
+ const kindRaw = trimOrUndefined(flags["kind"]);
+ if (kindRaw !== undefined && !BRAIN_LOG_EVENT_KIND_SET.has(kindRaw)) {
+ // Bad flag value is a usage error (exit 2, plain stderr), not a runtime
+ // failure (exit 1). Same for --limit and the resolver's selector checks.
+ return usageError(`brain event-trace: unknown event kind '${kindRaw}'`);
+ }
+ const limitRaw = trimOrUndefined(flags["limit"]);
+ if (limitRaw !== undefined && (!/^[0-9]+$/.test(limitRaw) || Number.parseInt(limitRaw, 10) < 1)) {
+ return usageError(`brain event-trace: --limit must be a positive integer`);
+ }
+ const limit = limitRaw !== undefined ? Number.parseInt(limitRaw, 10) : undefined;
+ const vault = brainVerbContext(flags).vault;
+
+ let results: ReadonlyArray<LogEventTrace>;
+ try {
+ results = resolveLogEventTraces(vault, {
+ ...(trimOrUndefined(flags["date"]) !== undefined
+ ? { date: trimOrUndefined(flags["date"])! }
+ : {}),
+ ...(trimOrUndefined(flags["at"]) !== undefined ? { at: trimOrUndefined(flags["at"])! } : {}),
+ ...(kindRaw !== undefined ? { kind: kindRaw as BrainLogEventKind } : {}),
+ ...(trimOrUndefined(flags["session-id"]) !== undefined
+ ? { sessionId: trimOrUndefined(flags["session-id"])! }
+ : {}),
+ ...(limit !== undefined ? { limit } : {}),
+ ...(flags["keep-private"] === true ? { keepPrivate: true } : {}),
+ });
+ } catch (err) {
+ // The resolver validates selectors (--date, --at, --kind) before any IO
+ // and a missing log day reads as empty rather than throwing, so a throw
+ // here is a selector usage error: exit 2, not the runtime exit-1 path.
+ return usageError(`brain event-trace: ${(err as Error).message}`);
+ }
+
+ if (flags["json"]) {
+ process.stdout.write(
+ JSON.stringify({ total: results.length, events: results }, null, 2) + "\n",
+ );
+ return 0;
+ }
+
+ const totalTraces = results.reduce((sum, r) => sum + r.traceCount, 0);
+ process.stdout.write(
+ `${results.length} logged event(s), ${totalTraces} attached context trace(s):\n`,
+ );
+ for (const result of results) {
+ const e = result.event;
+ const stamp = e.timestamp.slice(11, 19);
+ const tags = [
+ e.sessionId ? `session=${e.sessionId}` : null,
+ e.turnId ? `turn=${e.turnId}` : null,
+ e.agent ? `agent=${e.agent}` : null,
+ ].filter(Boolean);
+ process.stdout.write(
+ `\n${stamp} ${e.eventType}${tags.length > 0 ? " " + tags.join(" ") : ""}\n`,
+ );
+ if (result.traceCount === 0) {
+ process.stdout.write(" (no attached context trace)\n");
+ continue;
+ }
+ for (const trace of result.traces) {
+ process.stdout.write(` ${formatTrace(trace)}\n`);
+ }
+ }
+ return 0;
+}
+
+function formatTrace(trace: AttachedTrace): string {
+ const flags: string[] = [`via ${trace.joinedBy.join("+")}`];
+ if (trace.handoffRef) flags.push(`handoff=${trace.handoffKind ?? "?"}:${trace.handoffRef}`);
+ if (trace.sourceCount > 0) flags.push(`sources=${trace.sourceCount}`);
+ if (trace.private) flags.push("private");
+ if (trace.redacted) flags.push("redacted");
+ return `${trace.createdAt} ${trace.kind} ${trace.id} [${flags.join(", ")}]`;
+}
+
+function trimOrUndefined(value: string | boolean | string[] | undefined): string | undefined {
+ if (typeof value !== "string") return undefined;
+ const trimmed = value.trim();
+ return trimmed.length > 0 ? trimmed : undefined;
+}
diff --git a/src/cli/brain/verbs/index.ts b/src/cli/brain/verbs/index.ts
index e4bbb8fb..89ce2920 100644
--- a/src/cli/brain/verbs/index.ts
+++ b/src/cli/brain/verbs/index.ts
@@ -82,6 +82,7 @@ export { cmdBrainPageDedup } from "./page-dedup.ts";
export { cmdBrainTokenFootprint } from "./token-footprint.ts";
export { cmdBrainContextPack } from "./context-pack.ts";
export { cmdBrainContextReceipts } from "./context-receipts.ts";
+export { cmdBrainEventTrace } from "./event-trace.ts";
export { cmdBrainContextPresets } from "./context-presets.ts";
export { cmdBrainPreCompactExtract } from "./pre-compact-extract.ts";
export { cmdBrainRecallTelemetry } from "./recall-telemetry.ts";
diff --git a/src/cli/search.ts b/src/cli/search.ts
index 9df3e10f..01e7f23c 100644
--- a/src/cli/search.ts
+++ b/src/cli/search.ts
@@ -21,6 +21,7 @@ import {
} from "../core/brain/safeguard.ts";
import {
captureRecallFeedback,
+ expandHit,
indexCheck,
indexStatus,
indexVault,
@@ -65,6 +66,7 @@ import { CronTemplateError, renderCronTemplate } from "./search-cron-template.ts
const KNOWN_VERBS = new Set([
"query",
+ "expand",
"index",
"reindex",
"status",
@@ -90,6 +92,8 @@ export async function handleSearchSubcommand(argv: ReadonlyArray<string>): Promi
switch (verb) {
case "query":
return await cmdSearchQuery(rest);
+ case "expand":
+ return await cmdSearchExpand(rest);
case "index":
return await cmdSearchIndex(rest);
case "reindex":
@@ -435,6 +439,7 @@ async function cmdSearchQuery(argv: ReadonlyArray<string>): Promise<number> {
visibility: { type: "string-array" },
"query-doc": { type: "string" },
expand: { type: "boolean" },
+ disclosure: { type: "string" },
profile: { type: "string" },
"evidence-pack": { type: "boolean" },
"include-superseded": { type: "boolean" },
@@ -468,6 +473,10 @@ async function cmdSearchQuery(argv: ReadonlyArray<string>): Promise<number> {
if (!Number.isInteger(limitNum) || limitNum < 1 || limitNum > 100) {
throw new CliError("--limit must be an integer in 1..100");
}
+ const disclosureRaw = typeof flags["disclosure"] === "string" ? flags["disclosure"] : undefined;
+ if (disclosureRaw !== undefined && disclosureRaw !== "full" && disclosureRaw !== "cards") {
+ throw new CliError("--disclosure must be 'full' or 'cards'");
+ }
const cfg = resolveConfig(flags);
@@ -499,6 +508,7 @@ async function cmdSearchQuery(argv: ReadonlyArray<string>): Promise<number> {
...(visibility !== undefined && visibility.length > 0 ? { visibility } : {}),
...(structuredQuery !== undefined ? { structuredQuery } : {}),
...(flags["expand"] === true ? { expand: true } : {}),
+ ...(disclosureRaw === "cards" ? { disclosure: "cards" as const } : {}),
...(typeof flags["profile"] === "string" ? { profile: flags["profile"] as string } : {}),
...(flags["evidence-pack"] === true ? { evidencePack: true } : {}),
...(flags["include-superseded"] === true ? { includeSuperseded: true } : {}),
@@ -584,6 +594,21 @@ function jsonForOutcome(o: SearchOutcome): unknown {
})),
warnings: o.warnings,
total: o.total,
+ ...(o.cards
+ ? {
+ cards: o.cards.map((c) => ({
+ path: c.path,
+ title: c.title,
+ score: c.score,
+ snippet: c.snippet,
+ pointer: c.pointer,
+ reasons: c.reasons,
+ document_id: c.documentId,
+ chunk_id: c.chunkId,
+ ...(c.origin !== undefined ? { origin: c.origin } : {}),
+ })),
+ }
+ : {}),
...(o.evidencePack ? { evidence_pack: jsonForEvidencePack(o.evidencePack) } : {}),
};
}
@@ -598,6 +623,7 @@ function jsonForEvidencePack(pack: NonNullable<SearchOutcome["evidencePack"]>):
path: record.path,
document_id: record.documentId,
chunk_id: record.chunkId,
+ line_pointer: record.linePointer,
matched_terms: record.matchedTerms,
missing_terms: record.missingTerms,
support_coverage: record.supportCoverage,
@@ -639,6 +665,22 @@ function jsonForEvidencePack(pack: NonNullable<SearchOutcome["evidencePack"]>):
}
function renderOutcomeHuman(o: SearchOutcome, verbose: boolean): string {
+ // Progressive disclosure layer 1: render compact cards. The agent
+ // drills a hit with `o2b search expand --chunk <chunk_id>`.
+ if (o.cards !== undefined) {
+ const lines: string[] = [];
+ if (o.cards.length === 0) lines.push("(no results)");
+ o.cards.forEach((c, i) => {
+ const originSuffix = c.origin !== undefined ? ` • ${c.origin}` : "";
+ lines.push(`[${i + 1}] ${c.pointer} • ${c.score.toFixed(2)}${originSuffix}`);
+ lines.push(` ${c.snippet}`);
+ lines.push(` expand: o2b search expand --chunk ${c.chunkId}`);
+ if (verbose && c.reasons.length > 0) lines.push(` why: ${c.reasons.join(", ")}`);
+ lines.push("");
+ });
+ for (const w of o.warnings) lines.push(`warning: ${w}`);
+ return lines.join("\n") + (lines.length > 0 ? "" : "\n");
+ }
const lines: string[] = [];
if (o.results.length === 0) {
lines.push("(no results)");
@@ -668,6 +710,94 @@ function renderOutcomeHuman(o: SearchOutcome, verbose: boolean): string {
return lines.join("\n") + (lines.length > 0 ? "" : "\n");
}
+// ─── expand ───────────────────────────────────────────────────────────────────
+
+/**
+ * Progressive disclosure layers 2 + 3: drill a layer-1 card (from
+ * `o2b search --disclosure cards`) into the fuller note and the paginated
+ * raw chunk transcript. Read-only; never rebuilds the index.
+ */
+async function cmdSearchExpand(argv: ReadonlyArray<string>): Promise<number> {
+ const { flags } = parseFlags(argv, {
+ vault: { type: "string" },
+ config: { type: "string" },
+ db: { type: "string" },
+ chunk: { type: "string" },
+ "raw-limit": { type: "string" },
+ cursor: { type: "string" },
+ json: { type: "boolean" },
+ });
+ const chunkId = Number(flags["chunk"]);
+ if (!Number.isInteger(chunkId) || chunkId < 1) {
+ throw new CliError("--chunk must be a positive integer chunk id (from a card)");
+ }
+ let rawLimit: number | undefined;
+ if (typeof flags["raw-limit"] === "string") {
+ rawLimit = Number(flags["raw-limit"]);
+ if (!Number.isInteger(rawLimit) || rawLimit < 1) {
+ throw new CliError("--raw-limit must be a positive integer");
+ }
+ }
+ const cfg = resolveConfig(flags);
+ let result;
+ try {
+ result = await expandHit(cfg, {
+ chunkId,
+ ...(rawLimit !== undefined ? { rawLimit } : {}),
+ ...(typeof flags["cursor"] === "string" ? { cursor: flags["cursor"] as string } : {}),
+ });
+ } catch (e) {
+ if (e instanceof SearchError) {
+ process.stderr.write(`error: ${e.message}\n`);
+ return 2;
+ }
+ throw e;
+ }
+ if (flags["json"]) {
+ process.stdout.write(
+ JSON.stringify({
+ chunk_id: result.chunkId,
+ note: {
+ document_id: result.note.documentId,
+ path: result.note.path,
+ title: result.note.title,
+ line_start: result.note.lineStart,
+ line_end: result.note.lineEnd,
+ pointer: result.note.pointer,
+ content: result.note.content,
+ },
+ raw_content: result.raw_content.map((c) => ({
+ chunk_id: c.chunkId,
+ chunk_index: c.chunkIndex,
+ start_line: c.startLine,
+ end_line: c.endLine,
+ pointer: c.pointer,
+ content: c.content,
+ })),
+ next_cursor: result.next_cursor,
+ }) + "\n",
+ );
+ return 0;
+ }
+ const lines: string[] = [];
+ lines.push(`note: ${result.note.pointer}`);
+ if (result.note.title !== null) lines.push(`title: ${result.note.title}`);
+ lines.push("");
+ lines.push(result.note.content);
+ lines.push("");
+ lines.push(`── raw chunks (${result.raw_content.length}) ──`);
+ for (const c of result.raw_content) {
+ lines.push(`[${c.pointer}]`);
+ lines.push(c.content);
+ lines.push("");
+ }
+ if (result.next_cursor !== null) {
+ lines.push(`more: o2b search expand --chunk ${result.chunkId} --cursor ${result.next_cursor}`);
+ }
+ process.stdout.write(lines.join("\n") + "\n");
+ return 0;
+}
+
// ─── index ────────────────────────────────────────────────────────────────────
async function cmdSearchIndex(argv: ReadonlyArray<string>): Promise<number> {
diff --git a/src/core/brain/event-trace.ts b/src/core/brain/event-trace.ts
new file mode 100644
index 00000000..3072eea2
--- /dev/null
+++ b/src/core/brain/event-trace.ts
@@ -0,0 +1,300 @@
+/**
+ * Event → context-trace join reader.
+ *
+ * The Brain log (`Brain/log/<date>.md` + JSONL sidecar) records WHAT
+ * happened; the continuity store (`Brain/log/continuity/<month>.jsonl`)
+ * records WHICH CONTEXT was supplied to those operations. The two
+ * surfaces are written independently and, until now, were read
+ * independently — an operator asking "why did the agent do this?" had
+ * to query the log, copy a correlation id, and then query each
+ * continuity reader by hand.
+ *
+ * This module is the single reader that joins them: given a selector
+ * for one (or several) logged events, it resolves the continuity
+ * records attached to each via the stable correlation ids the
+ * observability contract already guarantees (`docs/observability.md`
+ * §Correlation IDs):
+ *
+ * - `session_id` — the cross-surface session join. Log events written
+ * by the write-session chokepoint and session-lifecycle capture
+ * carry `session_id` in their body; recall_telemetry /
+ * context_receipt / session_turn / generation_report payloads carry
+ * the same id (lifted to `sessionId` by the read-model).
+ * - `turn_id` — the finer within-session join.
+ * - artifact refs — wikilinks / vault paths in the event body (e.g.
+ * `apply-evidence`'s `artifact`, write-session's `target`) matched
+ * against a record's `sourceRefs[].id` / `.path` (or the record id
+ * itself when the body links a continuity id directly).
+ *
+ * Read-only and fail-soft by construction: it composes `readLogDay`
+ * (tolerant shard merge) and `loadNormalizedContinuityRecords` (schema
+ * dispatch + masking policy), so it inherits their guarantees — malformed
+ * rows are skipped, unknown kinds stay readable, and `private` records
+ * are dropped unless explicitly kept. It never writes to either surface.
+ */
+
+import { readLogDay } from "./log-jsonl.ts";
+import type { BrainLogEntry, BrainLogEntryPayload } from "./log.ts";
+import { loadNormalizedContinuityRecords } from "./continuity/read-model.ts";
+import type { NormalizedContinuityRecord } from "./continuity/read-model.ts";
+import { validateIsoDate } from "./paths.ts";
+import { BRAIN_LOG_EVENT_KIND_SET, type BrainLogEventKind } from "./types.ts";
+
+/** Why a continuity record was attached to a given log event. */
+export type TraceJoinReason = "session" | "turn" | "artifact";
+
+/** Body keys whose values are treated as artifact references. */
+const ARTIFACT_BODY_KEYS = Object.freeze([
+ "artifact",
+ "target",
+ "preference",
+ "signal",
+ "source",
+ "note_ref",
+ "ref",
+]);
+
+const WIKILINK_RE = /\[\[([^\]|#]+)(?:[#|][^\]]*)?\]\]/g;
+
+export interface EventTraceSelector {
+ /** Log day to read (`YYYY-MM-DD`, UTC). Defaults to today when omitted. */
+ readonly date?: string;
+ /** Pin a single event by its `HH:MM:SS` UTC stamp. */
+ readonly at?: string;
+ /** Restrict to one event kind. */
+ readonly kind?: BrainLogEventKind;
+ /** Restrict to events bound to this session id. */
+ readonly sessionId?: string;
+ /** Cap the number of events returned (after filtering). */
+ readonly limit?: number;
+ /** Keep continuity records flagged `private` (default: drop them). */
+ readonly keepPrivate?: boolean;
+}
+
+/** One continuity record attached to a log event, with the join provenance. */
+export interface AttachedTrace {
+ readonly id: string;
+ readonly kind: string;
+ readonly createdAt: string;
+ /** Distinct join reasons that matched, in fixed order session→turn→artifact. */
+ readonly joinedBy: ReadonlyArray<TraceJoinReason>;
+ readonly sessionId?: string;
+ readonly turnId?: string;
+ readonly handoffKind?: string;
+ readonly handoffRef?: string;
+ readonly sourceCount: number;
+ readonly private: boolean;
+ readonly redacted: boolean;
+}
+
+/** Correlation ids lifted out of a log event body. */
+export interface EventCorrelation {
+ readonly sessionId?: string;
+ readonly turnId?: string;
+ /** Distinct artifact identifiers (wikilink targets + path/id-shaped values). */
+ readonly artifacts: ReadonlyArray<string>;
+}
+
+export interface LogEventTrace {
+ readonly event: {
+ readonly timestamp: string;
+ readonly eventType: string;
+ readonly agent?: string;
+ readonly sessionId?: string;
+ readonly turnId?: string;
+ readonly artifacts: ReadonlyArray<string>;
+ readonly body: BrainLogEntryPayload;
+ };
+ readonly traces: ReadonlyArray<AttachedTrace>;
+ readonly traceCount: number;
+}
+
+/**
+ * Extract the join keys from a single log entry body. Pure and
+ * defensive: it never throws on an odd payload shape.
+ */
+export function extractEventCorrelation(event: BrainLogEntry): EventCorrelation {
+ const body = event.body;
+ const sessionId = readBodyString(body["session_id"]);
+ const turnId = readBodyString(body["turn_id"]);
+ const artifacts = new Set<string>();
+ for (const key of ARTIFACT_BODY_KEYS) {
+ collectArtifacts(body[key], artifacts);
+ }
+ return {
+ ...(sessionId !== undefined ? { sessionId } : {}),
+ ...(turnId !== undefined ? { turnId } : {}),
+ artifacts: Object.freeze([...artifacts]),
+ };
+}
+
+/**
+ * Compute the continuity records attached to one event, given the
+ * already-normalized store. Returned records are sorted by
+ * (createdAt, id) — the same stable order the store guarantees.
+ */
+export function attachTracesToEvent(
+ records: ReadonlyArray<NormalizedContinuityRecord>,
+ correlation: EventCorrelation,
+): ReadonlyArray<AttachedTrace> {
+ const artifactSet = new Set(correlation.artifacts);
+ const attached: AttachedTrace[] = [];
+ for (const record of records) {
+ const reasons = joinReasons(record, correlation, artifactSet);
+ if (reasons.length === 0) continue;
+ attached.push({
+ id: record.id,
+ kind: record.kind,
+ createdAt: record.createdAt,
+ joinedBy: reasons,
+ ...(record.sessionId !== undefined ? { sessionId: record.sessionId } : {}),
+ ...(record.turnId !== undefined ? { turnId: record.turnId } : {}),
+ ...(record.handoffKind !== undefined ? { handoffKind: record.handoffKind } : {}),
+ ...(record.handoffRef !== undefined ? { handoffRef: record.handoffRef } : {}),
+ sourceCount: record.sourceRefs.length,
+ private: record.private,
+ redacted: record.redacted,
+ });
+ }
+ attached.sort((a, b) => a.createdAt.localeCompare(b.createdAt) || a.id.localeCompare(b.id));
+ return Object.freeze(attached);
+}
+
+/**
+ * Resolve logged events for `selector.date` and attach the continuity
+ * trace to each. Events are returned in log order (timestamp, then
+ * shard/line) after filtering; an event with no attached records is
+ * still returned with an empty `traces` list so the operator sees that
+ * the join produced nothing rather than that the event is missing.
+ */
+export function resolveLogEventTraces(
+ vault: string,
+ selector: EventTraceSelector = {},
+): ReadonlyArray<LogEventTrace> {
+ const date = selector.date !== undefined ? validateIsoDate(selector.date) : todayUtc();
+ if (selector.at !== undefined && !/^\d{2}:\d{2}:\d{2}$/.test(selector.at)) {
+ throw new Error(`event-trace: --at must be HH:MM:SS (UTC); got ${JSON.stringify(selector.at)}`);
+ }
+ if (selector.kind !== undefined && !BRAIN_LOG_EVENT_KIND_SET.has(selector.kind)) {
+ throw new Error(`event-trace: unknown event kind '${selector.kind}'`);
+ }
+
+ const { entries } = readLogDay(vault, date);
+ const records = loadNormalizedContinuityRecords(
+ vault,
+ selector.keepPrivate === true ? { keepPrivate: true } : {},
+ );
+
+ const wantStamp = selector.at !== undefined ? `${date}T${selector.at}Z` : undefined;
+ // Cap is evaluated BEFORE the push, so `limit: 0` yields an empty list and a
+ // limit of N never collects N+1 events (the cap is the count returned, not
+ // the count plus one).
+ const cap = selector.limit !== undefined ? Math.max(0, Math.floor(selector.limit)) : undefined;
+ const results: LogEventTrace[] = [];
+ for (const entry of entries) {
+ if (selector.kind !== undefined && entry.eventType !== selector.kind) continue;
+ if (wantStamp !== undefined && entry.timestamp !== wantStamp) continue;
+ const correlation = extractEventCorrelation(entry);
+ if (selector.sessionId !== undefined && correlation.sessionId !== selector.sessionId) continue;
+ if (cap !== undefined && results.length >= cap) break;
+ const traces = attachTracesToEvent(records, correlation);
+ results.push({
+ event: {
+ timestamp: entry.timestamp,
+ eventType: entry.eventType,
+ ...(entry.agent !== undefined ? { agent: entry.agent } : {}),
+ ...(correlation.sessionId !== undefined ? { sessionId: correlation.sessionId } : {}),
+ ...(correlation.turnId !== undefined ? { turnId: correlation.turnId } : {}),
+ artifacts: correlation.artifacts,
+ body: entry.body,
+ },
+ traces,
+ traceCount: traces.length,
+ });
+ }
+ return Object.freeze(results);
+}
+
+// ----- internals ------------------------------------------------------------
+
+function joinReasons(
+ record: NormalizedContinuityRecord,
+ correlation: EventCorrelation,
+ artifactSet: ReadonlySet<string>,
+): ReadonlyArray<TraceJoinReason> {
+ const reasons: TraceJoinReason[] = [];
+ if (correlation.sessionId !== undefined && record.sessionId === correlation.sessionId) {
+ reasons.push("session");
+ }
+ if (correlation.turnId !== undefined && record.turnId === correlation.turnId) {
+ reasons.push("turn");
+ }
+ if (artifactSet.size > 0 && recordMatchesArtifact(record, artifactSet)) {
+ reasons.push("artifact");
+ }
+ return reasons;
+}
+
+function recordMatchesArtifact(
+ record: NormalizedContinuityRecord,
+ artifactSet: ReadonlySet<string>,
+): boolean {
+ if (artifactSet.has(record.id)) return true;
+ for (const ref of record.sourceRefs) {
+ const id = ref["id"];
+ if (typeof id === "string" && artifactSet.has(id)) return true;
+ const path = ref["path"];
+ if (typeof path === "string" && artifactSet.has(path)) return true;
+ }
+ return false;
+}
+
+function collectArtifacts(value: unknown, out: Set<string>): void {
+ if (typeof value === "string") {
+ addArtifactToken(value, out);
+ } else if (Array.isArray(value)) {
+ for (const item of value) {
+ if (typeof item === "string") addArtifactToken(item, out);
+ }
+ }
+}
+
+function addArtifactToken(raw: string, out: Set<string>): void {
+ const value = raw.trim();
+ if (value.length === 0) return;
+ let sawWikilink = false;
+ for (const match of value.matchAll(WIKILINK_RE)) {
+ const inner = match[1]?.trim();
+ if (inner) {
+ out.add(inner);
+ // A wikilink target may itself be a vault path (`Brain/foo.md`);
+ // its basename without extension is the artifact id most readers use.
+ const base = basenameNoExt(inner);
+ if (base !== inner) out.add(base);
+ sawWikilink = true;
+ }
+ }
+ if (sawWikilink) return;
+ // Bare path or id-shaped value (no wikilink syntax). Record both the
+ // raw value and its basename so a `target: Brain/x.md` body joins a
+ // sourceRef carrying either form.
+ out.add(value);
+ const base = basenameNoExt(value);
+ if (base !== value) out.add(base);
+}
+
+function basenameNoExt(value: string): string {
+ const slash = value.lastIndexOf("/");
+ const tail = slash >= 0 ? value.slice(slash + 1) : value;
+ return tail.endsWith(".md") ? tail.slice(0, -3) : tail;
+}
+
+function readBodyString(value: unknown): string | undefined {
+ if (typeof value !== "string") return undefined;
+ const trimmed = value.trim();
+ return trimmed.length > 0 ? trimmed : undefined;
+}
+
+function todayUtc(): string {
+ return new Date().toISOString().slice(0, 10);
+}
diff --git a/src/core/brain/session-recall.ts b/src/core/brain/session-recall.ts
index 49fa5e64..5f77296c 100644
--- a/src/core/brain/session-recall.ts
+++ b/src/core/brain/session-recall.ts
@@ -1,5 +1,6 @@
import { createHash } from "node:crypto";
+import { charSpanToLineSpan } from "../search/line-numbering.ts";
import { appendContinuityRecord, listContinuityRecords } from "./continuity/store.ts";
import type { ContinuityRecord, ContinuitySourceRef } from "./continuity/types.ts";
import { readLineageLedger } from "./lineage/ledger.ts";
@@ -36,6 +37,14 @@ export interface SessionRecallHit {
readonly kind: "session_turn" | "session_summary_node";
readonly score: number;
readonly snippet: string;
+ /**
+ * 1-based line span of the matched text within the record, computed at
+ * read time from the snippet match's char offset. Line-anchors the
+ * char-offset snippet so a hit can be cited to exact lines, not just the
+ * record as a whole; the stored text is never mutated.
+ */
+ readonly line_start: number;
+ readonly line_end: number;
readonly turn_id?: string;
readonly role?: string;
readonly depth?: number;
@@ -401,6 +410,24 @@ function lineageScope(
return scope;
}
+/**
+ * Case-insensitive search that returns an offset into the ORIGINAL `text`,
+ * so `snippet()` and `charSpanToLineSpan()` (both of which slice the
+ * original text) stay aligned. `text.toLowerCase()` is not always
+ * length-preserving (e.g. U+0130 lowercases to two code units), which would
+ * otherwise shift the offset taken from the lowercased copy. The fast path
+ * keeps normal text O(n) and bit-identical; the scan runs only when
+ * lowercasing changed the length.
+ */
+function caseInsensitiveIndex(text: string, needleLower: string): number {
+ const lower = text.toLowerCase();
+ if (lower.length === text.length) return lower.indexOf(needleLower);
+ for (let i = 0; i < text.length; i++) {
+ if (text.slice(i).toLowerCase().startsWith(needleLower)) return i;
+ }
+ return -1;
+}
+
function hitFor(
record: ContinuityRecord,
needle: string,
@@ -408,14 +435,17 @@ function hitFor(
): SessionRecallHit | null {
const text = recordText(record);
const haystack = text.toLowerCase();
- const index = haystack.indexOf(needle);
+ const index = caseInsensitiveIndex(text, needle);
if (index < 0) return null;
const score = (record.kind === "session_turn" ? 2 : 1) + occurrenceCount(haystack, needle);
+ const span = charSpanToLineSpan(text, index, needle.length);
const base = {
id: record.id,
kind: record.kind as "session_turn" | "session_summary_node",
score,
snippet: snippet(text, index, snippetChars),
+ line_start: span.lineStart,
+ line_end: span.lineEnd,
};
if (record.kind === "session_turn") {
return Object.freeze({
diff --git a/src/core/search/coverage.ts b/src/core/search/coverage.ts
index 8102fa7b..b09350a0 100644
--- a/src/core/search/coverage.ts
+++ b/src/core/search/coverage.ts
@@ -132,6 +132,35 @@ export function buildCompletenessReport(coverage: CoverageReport): CompletenessR
});
}
+/**
+ * Targeted self-correcting retry plan (t_8eb5ca32): the deterministic
+ * decision that connects coverage to a follow-up retrieval. A retry
+ * fires only when the IDF-weighted coverage is below the completeness
+ * threshold AND at least one RARE significant term is still uncovered;
+ * the follow-up is then aimed at exactly those uncovered rare terms —
+ * the specifically-missing high-signal facts — never a generic
+ * broadening of the whole query. The rare gate keeps the retry off when
+ * only corpus-common terms are missing (low IDF, low value) and bounds
+ * how often it can fire. `terms` is empty iff `fire` is false.
+ *
+ * Pure and deterministic: a verdict over an already-built report, no
+ * I/O. The caller decides how to turn the terms into a query (FTS OR,
+ * expansion, …) and how to cap the number of passes.
+ */
+export interface TargetedRetryPlan {
+ readonly fire: boolean;
+ readonly terms: ReadonlyArray<string>;
+}
+
+export function planTargetedRetry(coverage: CoverageReport): TargetedRetryPlan {
+ const belowThreshold = coverage.idfWeightedCoverage < COMPLETENESS_COMPLETE_THRESHOLD;
+ const fire = belowThreshold && coverage.uncoveredRareTerms.length > 0;
+ return Object.freeze({
+ fire,
+ terms: fire ? coverage.uncoveredRareTerms : Object.freeze([] as string[]),
+ });
+}
+
export function buildCoverageReport(inputs: CoverageInputs): CoverageReport {
const terms: TermCoverage[] = inputs.significantTerms.map((term) => {
const df = inputs.dfByTerm.get(term) ?? 0;
diff --git a/src/core/search/cross-vault.ts b/src/core/search/cross-vault.ts
index 77ef6ec8..05801ad1 100644
--- a/src/core/search/cross-vault.ts
+++ b/src/core/search/cross-vault.ts
@@ -84,9 +84,20 @@ export async function searchAcrossVaults(
}
}
+ // Normalized-confidence chain-stop policy is an active-origin decision
+ // (t_23c1b929): the active vault's resolved config governs whether and
+ // when the union short-circuits. External origins resolve fresh and their
+ // own knob never gates the union. Resolved once, before the loop.
+ const activeRecall = (
+ activeConfig ?? resolveSearchConfig({ vault: resolve(activeVault), configPath })
+ ).recall;
+
+ let chainStop: SearchOutcome["chainStop"];
+
// Origins run sequentially: each opens its own SQLite store, and a
// handful of local index reads gains nothing from interleaving.
- for (const origin of origins) {
+ for (let i = 0; i < origins.length; i++) {
+ const origin = origins[i]!;
const isActive = origin.kind === "active";
try {
const base =
@@ -110,6 +121,27 @@ export async function searchAcrossVaults(
merged.push(...outcome.results.map((result) => labelled(result, origin.label)));
warnings.push(...outcome.warnings.map((warning) => `[${origin.label}] ${warning}`));
total += outcome.total;
+ // Chain-stop: if this origin answered confidently (its top NORMALIZED
+ // [0,1] result score reached the threshold) and origins remain, skip
+ // them. `results` is ranked score-desc, so `[0]` is the top score; it
+ // is the normalized score, never the raw lane score, so a tiny-corpus
+ // origin with a high raw score does not short-circuit. Only recorded
+ // when origins were actually skipped, keeping the single-origin and
+ // never-triggered paths byte-identical.
+ const remaining = origins.slice(i + 1);
+ if (
+ activeRecall.chainStopEnabled &&
+ remaining.length > 0 &&
+ outcome.results.length > 0 &&
+ outcome.results[0]!.score >= activeRecall.chainStopScore
+ ) {
+ chainStop = Object.freeze({
+ triggered: true as const,
+ stoppedAfter: origin.label,
+ skipped: Object.freeze(remaining.map((o) => o.label)),
+ });
+ break;
+ }
} catch (exc) {
warnings.push(`[${origin.label}] ${(exc as Error).message ?? String(exc)}`);
}
@@ -122,5 +154,6 @@ export async function searchAcrossVaults(
// Sum of per-origin totals - informational, mirrors single-vault
// semantics where `total` can exceed the capped `results` length.
total,
+ ...(chainStop ? { chainStop } : {}),
});
}
diff --git a/src/core/search/evidence-pack.ts b/src/core/search/evidence-pack.ts
index 53e48e04..7b4a2e15 100644
--- a/src/core/search/evidence-pack.ts
+++ b/src/core/search/evidence-pack.ts
@@ -5,12 +5,21 @@ import {
termIncludedIn,
} from "./coverage.ts";
import type { CompletenessReport, CoverageReport } from "./coverage.ts";
+import { formatLinePointer } from "./line-numbering.ts";
import type { BrainSearchResult } from "./types.ts";
export interface EvidenceRecord {
readonly path: string;
readonly documentId: number;
readonly chunkId: number;
+ /**
+ * Read-time line-anchored citation for this chunk, formatted
+ * `path:Lstart-Lend` (or `path:Lstart` for a single line) from the chunk's
+ * 1-based `startLine`/`endLine`. Resolves by opening the file and slicing
+ * the range with {@link extractLineRange}; the stored bytes are never
+ * mutated, so the pointer stays valid across idempotent re-mining.
+ */
+ readonly linePointer: string;
readonly matchedTerms: ReadonlyArray<string>;
readonly missingTerms: ReadonlyArray<string>;
readonly supportCoverage: number;
@@ -184,6 +193,7 @@ export function buildEvidencePack(
path: result.path,
documentId: result.documentId,
chunkId: result.chunkId,
+ linePointer: formatLinePointer(result.path, result.startLine, result.endLine),
matchedTerms: Object.freeze(matched),
missingTerms: Object.freeze(missing),
supportCoverage: supportCoverage(matched, significant),
diff --git a/src/core/search/index.ts b/src/core/search/index.ts
index 2fca57d0..240da29a 100644
--- a/src/core/search/index.ts
+++ b/src/core/search/index.ts
@@ -33,12 +33,18 @@ type SearchConfigOverrides = Partial<Omit<ResolvedSearchConfig, "ignoreRules" |
export type {
BrainSearchResult,
+ DisclosureMode,
+ ExpandedNote,
+ ExpandedRawChunk,
+ ExpandHitInput,
+ ExpandHitResult,
IndexCheckReport,
IndexStats,
IndexStatusSnapshot,
ResolvedEmbeddingConfig,
ResolvedRecallConfig,
ResolvedSearchConfig,
+ SearchCard,
SearchErrorCode,
SearchOptions,
SearchOutcome,
@@ -79,7 +85,7 @@ export {
type IndexVaultOptions,
type IndexProgressEvent,
} from "./indexer.ts";
-export { search } from "./search.ts";
+export { search, expandHit } from "./search.ts";
export {
captureRecallFeedback,
computeLearnedWeights,
@@ -97,6 +103,7 @@ export {
const DEFAULTS = {
chunkSize: 800,
chunkOverlap: 100,
+ chunkMinSize: 100,
keywordWeight: 0.6,
semanticWeight: 0.4,
provider: "openai-compat" as const,
@@ -117,6 +124,7 @@ const DEFAULTS = {
rrfK: DEFAULT_RRF_K,
shutdownGraceSeconds: 5,
resumeReindex: false,
+ chainStopScore: 0.8,
};
function parseFusionMode(raw: string | null): "linear" | "rrf" {
@@ -203,6 +211,13 @@ function validateResolvedConfig(config: ResolvedSearchConfig): void {
`search_chunk_overlap must be smaller than search_chunk_size, got ${config.chunkOverlap} >= ${config.chunkSize}`,
);
}
+ validateIntegerRange(config.chunkMinSize, "search_chunk_min_size", { min: 1 });
+ if (config.chunkMinSize > config.chunkSize) {
+ throw new SearchError(
+ "INVALID_INPUT",
+ `search_chunk_min_size must not exceed search_chunk_size, got ${config.chunkMinSize} > ${config.chunkSize}`,
+ );
+ }
validateWeight(config.keywordWeight, "search_keyword_weight");
validateWeight(config.semanticWeight, "search_semantic_weight");
if (config.keywordWeight + config.semanticWeight > 1.0 + 1e-9) {
@@ -285,6 +300,12 @@ export function resolveSearchConfig(opts: {
"search_chunk_overlap",
{ min: 0 },
);
+ const chunkMinSize = parseInteger(
+ envOrConfig(env, config, "OPEN_SECOND_BRAIN_SEARCH_CHUNK_MIN_SIZE", "search_chunk_min_size"),
+ DEFAULTS.chunkMinSize,
+ "search_chunk_min_size",
+ { min: 1 },
+ );
const keywordWeight = parseFloat01(
envOrConfig(env, config, "OPEN_SECOND_BRAIN_SEARCH_KW_WEIGHT", "search_keyword_weight"),
DEFAULTS.keywordWeight,
@@ -510,6 +531,21 @@ export function resolveSearchConfig(opts: {
false,
"search_self_tuning_enabled",
);
+ const chainStopEnabled = parseBool(
+ envOrConfig(env, config, "OPEN_SECOND_BRAIN_SEARCH_CHAIN_STOP", "search_chain_stop_enabled"),
+ false,
+ "search_chain_stop_enabled",
+ );
+ const chainStopScore = parseFloat01(
+ envOrConfig(
+ env,
+ config,
+ "OPEN_SECOND_BRAIN_SEARCH_CHAIN_STOP_SCORE",
+ "search_chain_stop_score",
+ ),
+ DEFAULTS.chainStopScore,
+ "search_chain_stop_score",
+ );
const shutdownGraceSeconds = parseInteger(
envOrConfig(
env,
@@ -545,6 +581,8 @@ export function resolveSearchConfig(opts: {
twoPassEnabled,
poolMultiplier,
selfTuningEnabled,
+ chainStopEnabled,
+ chainStopScore,
});
const base: ResolvedSearchConfig = Object.freeze({
@@ -553,6 +591,7 @@ export function resolveSearchConfig(opts: {
ignoreRules,
chunkSize,
chunkOverlap,
+ chunkMinSize,
keywordWeight,
semanticWeight,
fusionMode,
diff --git a/src/core/search/indexer.ts b/src/core/search/indexer.ts
index 32662852..6ccbf51e 100644
--- a/src/core/search/indexer.ts
+++ b/src/core/search/indexer.ts
@@ -289,6 +289,7 @@ async function indexInto(
const filenameBase = basename(file.relPath, ".md");
const chunkResult = chunkMarkdown(content, filenameBase, {
maxTokens: config.chunkSize,
+ minTokens: config.chunkMinSize,
overlapTokens: config.chunkOverlap,
});
for (const w of chunkResult.warnings) {
@@ -697,6 +698,7 @@ function reindexStagingSignature(config: ResolvedSearchConfig, embeddings: boole
schema: LATEST_SCHEMA_VERSION,
chunkSize: config.chunkSize,
chunkOverlap: config.chunkOverlap,
+ chunkMinSize: config.chunkMinSize,
embedding,
});
}
diff --git a/src/core/search/line-numbering.ts b/src/core/search/line-numbering.ts
new file mode 100644
index 00000000..e5a834c2
--- /dev/null
+++ b/src/core/search/line-numbering.ts
@@ -0,0 +1,114 @@
+/**
+ * Read-time virtual line numbering — pure, I/O-free helpers for precise
+ * line-span citations.
+ *
+ * The stored Markdown is never mutated. Line markers and line slices are
+ * computed on read, so a pointer like `path:L55-L72` stays valid across
+ * idempotent re-mining: the bytes on disk do not shift, so the numbering a
+ * pointer was minted against does not shift either.
+ *
+ * `renderWithLineNumbers` / `extractLineRange` mirror the two pure functions
+ * the source design ships; `formatLinePointer` / `parseLinePointer` define the
+ * `path:Lstart-Lend` pointer grammar; `charSpanToLineSpan` bridges the existing
+ * char-offset snippet world to a line span.
+ *
+ * All helpers are total functions: out-of-range or inverted inputs clamp or
+ * return empty rather than throwing — they sit on a read path.
+ */
+
+/** A parsed `path:Lstart-Lend` (or single-line `path:Lstart`) pointer. */
+export interface LinePointer {
+ readonly path: string;
+ readonly lineStart: number;
+ readonly lineEnd: number;
+}
+
+const POINTER_RE = /^(.+):L(\d+)(?:-L(\d+))?$/;
+
+function clampStart(line: number): number {
+ return Math.max(1, Math.floor(line));
+}
+
+/**
+ * Prefix each line of `text` with a `[N] ` marker, counting from `startLine`
+ * (1-based, clamped to >= 1). Blank lines are preserved and still numbered.
+ * Empty input has no line 1, so it renders to the empty string.
+ */
+export function renderWithLineNumbers(text: string, startLine = 1): string {
+ if (text === "") return "";
+ const base = clampStart(startLine);
+ return text
+ .split("\n")
+ .map((line, index) => `[${base + index}] ${line}`)
+ .join("\n");
+}
+
+/**
+ * Return the verbatim bytes of `text` for the inclusive 1-based line range
+ * `[lineStart, lineEnd]`, with no markers. The range is clamped into
+ * `[1, lineCount]`; an inverted range, a start past the last line, or empty
+ * input yields the empty string. This is the resolution half of a line pointer:
+ * open the file, then slice.
+ */
+export function extractLineRange(text: string, lineStart: number, lineEnd: number): string {
+ if (text === "") return "";
+ const lines = text.split("\n");
+ const start = clampStart(lineStart);
+ const end = Math.min(lines.length, Math.floor(lineEnd));
+ if (start > end || start > lines.length) return "";
+ return lines.slice(start - 1, end).join("\n");
+}
+
+/**
+ * Format a `path:Lstart-Lend` pointer, collapsing to `path:Lstart` for a single
+ * line. The range is normalized: `start` clamps to >= 1 and `end` to >= start.
+ */
+export function formatLinePointer(path: string, lineStart: number, lineEnd: number): string {
+ const start = clampStart(lineStart);
+ const end = Math.max(start, Math.floor(lineEnd));
+ return start === end ? `${path}:L${start}` : `${path}:L${start}-L${end}`;
+}
+
+/**
+ * Parse a `path:Lstart-Lend` (or `path:Lstart`) pointer. Returns `null` for any
+ * malformed input — including a sub-1 start or an end below the start. The path
+ * capture is greedy, so colons that belong to the path are kept.
+ */
+export function parseLinePointer(pointer: string): LinePointer | null {
+ const match = POINTER_RE.exec(pointer);
+ if (match === null) return null;
+ const path = match[1]!;
+ const lineStart = Number(match[2]);
+ const lineEnd = match[3] !== undefined ? Number(match[3]) : lineStart;
+ if (path === "" || lineStart < 1 || lineEnd < lineStart) return null;
+ return { path, lineStart, lineEnd };
+}
+
+function countNewlines(text: string): number {
+ let count = 0;
+ for (let index = text.indexOf("\n"); index >= 0; index = text.indexOf("\n", index + 1)) {
+ count += 1;
+ }
+ return count;
+}
+
+/**
+ * Map a char-offset match `[index, index + length)` to the 1-based line span it
+ * touches. Indices are clamped into `[0, text.length]`. Bridges the existing
+ * char-offset snippet path to a line-anchored citation.
+ */
+export function charSpanToLineSpan(
+ text: string,
+ index: number,
+ length = 0,
+): { readonly lineStart: number; readonly lineEnd: number } {
+ const start = Math.max(0, Math.min(Math.floor(index), text.length));
+ const end = Math.max(start, Math.min(Math.floor(index) + Math.max(0, length), text.length));
+ // The line of the span's last *included* character: a half-open range that
+ // ends exactly on a newline must not count the line that newline opens.
+ const lastChar = end > start ? end - 1 : start;
+ return {
+ lineStart: 1 + countNewlines(text.slice(0, start)),
+ lineEnd: 1 + countNewlines(text.slice(0, lastChar)),
+ };
+}
diff --git a/src/core/search/query-cache.ts b/src/core/search/query-cache.ts
index 8a775d2b..a3d9d06c 100644
--- a/src/core/search/query-cache.ts
+++ b/src/core/search/query-cache.ts
@@ -86,6 +86,10 @@ export function buildCacheKey(
properties: canonicalProperties(opts.properties),
visibility: opts.visibility ? [...opts.visibility].toSorted() : null,
agentScope: opts.agentScope ?? null,
+ // Disclosure depth (D3) partitions the cache: a `cards` outcome and a
+ // `full` outcome must not collide. Folded in only for `cards`, so the
+ // default `full` key (and every pre-D3 cached row) stays byte-identical.
+ disclosure: opts.disclosure === "cards" ? "cards" : undefined,
structuredQuery: canonicalStructuredQuery(opts),
sessionFocus: canonicalSessionFocus(opts),
plan: planHash,
diff --git a/src/core/search/search.ts b/src/core/search/search.ts
index fd46a1ed..6bcb1205 100644
--- a/src/core/search/search.ts
+++ b/src/core/search/search.ts
@@ -35,7 +35,14 @@ import {
} from "./activation/store.ts";
import { extractEntities } from "./entities.ts";
import { expandQueryEntities } from "./entity-alias.ts";
-import { buildCoverageReport, significantTerms, termIncludedIn } from "./coverage.ts";
+import {
+ buildCoverageReport,
+ COMPLETENESS_COMPLETE_THRESHOLD,
+ planTargetedRetry,
+ significantTerms,
+ termIncludedIn,
+ type CoverageReport,
+} from "./coverage.ts";
import {
buildEvidencePack,
downrankTerminalEvidenceResults,
@@ -58,10 +65,14 @@ import { resolveTimeRange } from "./time-range.ts";
import { eventTimeInRange, parseValidityWindow, type ValidityWindow } from "./validity.ts";
import { expandByTraversal, type TraversalOptions } from "./traversal.ts";
import { Store } from "./store.ts";
+import { formatLinePointer } from "./line-numbering.ts";
import { SearchError } from "./types.ts";
import type {
BrainSearchResult,
+ ExpandHitInput,
+ ExpandHitResult,
ResolvedSearchConfig,
+ SearchCard,
SearchOptions,
SearchOutcome,
} from "./types.ts";
@@ -418,6 +429,24 @@ export async function search(
for (const h of semHits) allChunkIds.add(h.chunkId);
let idsList = Array.from(allChunkIds);
+ // Validity-window resolver (hoisted): used both by the time-range
+ // filter below and by the targeted-retry coverage gate, which must
+ // judge coverage over the IN-RANGE pool (see below). One cached
+ // frontmatter read per candidate path.
+ const validityWindowCache = new Map<string, ValidityWindow | null>();
+ const validityWindowFor = (path: string): ValidityWindow | null => {
+ if (validityWindowCache.has(path)) return validityWindowCache.get(path) ?? null;
+ let window: ValidityWindow | null = null;
+ try {
+ const [meta] = parseFrontmatter(join(config.vault, path));
+ window = parseValidityWindow(meta as Record<string, unknown>);
+ } catch {
+ window = null;
+ }
+ validityWindowCache.set(path, window);
+ return window;
+ };
+
// Self-correcting two-pass recall (t_ef92dfdc): a zero-candidate
// first pass in evidence-pack mode means the implicit-AND keyword
// match was too strict - the classic abstention dead end. Instead
@@ -426,6 +455,10 @@ export async function search(
// as alternatives, then let the merged pool flow through the normal
// ranking, filters, and a recomputed evidence pack.
let secondPass: SearchOutcome["secondPass"];
+ // Chunk ids the targeted retry below recovered, so only those
+ // results (not the first-pass hits they merge with) get the
+ // second-pass attribution reason.
+ const targetedChunkIds = new Set<number>();
if (idsList.length === 0 && opts.evidencePack === true && config.recall.twoPassEnabled) {
const terms = significantTerms(query);
if (terms.length >= 2) {
@@ -441,6 +474,7 @@ export async function search(
idsList = Array.from(allChunkIds);
secondPass = Object.freeze({
triggered: true,
+ kind: "broadened",
reason: "zero-candidate first pass; broadened OR retry",
added: kwHits.length,
});
@@ -448,6 +482,67 @@ export async function search(
}
}
+ // Coverage-driven targeted follow-up (t_8eb5ca32): the first pass
+ // DID return candidates, but their IDF-weighted coverage of the
+ // query is below the completeness threshold with rare query terms
+ // still uncovered - a PARTIAL miss, distinct from the zero-candidate
+ // dead end above. Issue exactly ONE targeted retry built from the
+ // specifically-uncovered rare terms (not a generic broadening of the
+ // whole query), merge the recovered candidates into the pool, and
+ // let them flow through the normal ranking and a recomputed evidence
+ // pack. Mutually exclusive with the broadened retry above (that needs
+ // an empty pool, this a non-empty one), so at most one retry fires -
+ // the same single-retry discipline. The trigger is deterministic and
+ // LLM-free; the recomputed pack still abstains on any term left
+ // uncovered after the retry.
+ if (
+ secondPass === undefined &&
+ idsList.length > 0 &&
+ opts.evidencePack === true &&
+ config.recall.twoPassEnabled
+ ) {
+ // Judge coverage over the pool that will actually survive ranking:
+ // for a time-scoped query, exclude out-of-range candidates first, so
+ // a rare term is not marked "covered" only because an out-of-range
+ // chunk matched it (which would wrongly suppress the retry while the
+ // final in-range result set still misses that term).
+ let coverageIds = idsList;
+ if (timeRange !== null) {
+ const hydratedForCoverage = store.hydrateChunks(idsList);
+ coverageIds = idsList.filter((chunkId) => {
+ const chunk = hydratedForCoverage.get(chunkId);
+ if (chunk === undefined) return false;
+ return eventTimeInRange(validityWindowFor(chunk.path), chunk.mtime, timeRange);
+ });
+ }
+ const poolCoverage = coverageOverChunks(store, query, coverageIds);
+ const plan = planTargetedRetry(poolCoverage);
+ if (plan.fire) {
+ const targeted = runFtsQueryDetailed(store, plan.terms[0]!, {
+ expandedTerms: plan.terms.slice(1),
+ limit: Math.max(limit * 5, 50),
+ pathPrefix: pathPrefix ?? null,
+ });
+ for (const w of targeted.warnings) warnings.push(w);
+ const newHits = targeted.hits.filter((h) => !allChunkIds.has(h.chunkId));
+ if (newHits.length > 0) {
+ kwHits = kwHits.concat(newHits);
+ for (const h of newHits) {
+ allChunkIds.add(h.chunkId);
+ targetedChunkIds.add(h.chunkId);
+ }
+ idsList = Array.from(allChunkIds);
+ secondPass = Object.freeze({
+ triggered: true,
+ kind: "targeted",
+ reason: `partial coverage ${poolCoverage.idfWeightedCoverage.toFixed(2)} < ${COMPLETENESS_COMPLETE_THRESHOLD}; targeted retry on uncovered rare terms: ${plan.terms.join(", ")}`,
+ added: newHits.length,
+ targetedTerms: plan.terms,
+ });
+ }
+ }
+ }
+
if (idsList.length === 0) {
const evidencePack =
opts.evidencePack === true
@@ -471,22 +566,9 @@ export async function search(
// Event-time discipline (t_b7191486): a document declaring
// `valid_from` / `valid_until` is tested by validity-window
// OVERLAP - storage mtime is the fallback, never the authority,
- // when explicit event time exists. One cached frontmatter read per
- // candidate path; an unparseable declared value warns once and
- // falls back to mtime.
- const validityWindowCache = new Map<string, ValidityWindow | null>();
- const validityWindowFor = (path: string): ValidityWindow | null => {
- if (validityWindowCache.has(path)) return validityWindowCache.get(path) ?? null;
- let window: ValidityWindow | null = null;
- try {
- const [meta] = parseFrontmatter(join(config.vault, path));
- window = parseValidityWindow(meta as Record<string, unknown>);
- } catch {
- window = null;
- }
- validityWindowCache.set(path, window);
- return window;
- };
+ // when explicit event time exists. The `validityWindowFor` resolver
+ // is hoisted above (shared with the targeted-retry coverage gate); an
+ // unparseable declared value warns once and falls back to mtime.
if (timeRange !== null) {
const warnedInvalid = new Set<string>();
const windowFor = validityWindowFor;
@@ -852,18 +934,24 @@ export async function search(
: r,
)
: withStructuredReasons;
- // Two-pass attribution (t_ef92dfdc): every surfaced result of a
- // broadened retry says so - the operator can tell recovered
- // evidence from a first-pass hit.
+ // Two-pass attribution (t_ef92dfdc; targeted retry t_8eb5ca32): a
+ // surfaced result of a retry says so - the operator can tell
+ // recovered evidence from a first-pass hit. The broadened retry
+ // replaced the whole pool, so every result is recovered; the
+ // targeted retry only ADDED candidates, so only those (tracked in
+ // `targetedChunkIds`) carry the reason - the first-pass hits they
+ // merge with do not.
const withSecondPassReasons =
- secondPass !== undefined
- ? withCanonicalReasons.map((r) =>
- Object.freeze({
- ...r,
- reasons: Object.freeze([...r.reasons, "second_pass: or-broadened retry"]),
- }),
- )
- : withCanonicalReasons;
+ secondPass === undefined
+ ? withCanonicalReasons
+ : withCanonicalReasons.map((r) => {
+ if (secondPass.kind === "targeted" && !targetedChunkIds.has(r.chunkId)) return r;
+ const reason =
+ secondPass.kind === "targeted"
+ ? "second_pass: targeted retry on uncovered rare terms"
+ : "second_pass: or-broadened retry";
+ return Object.freeze({ ...r, reasons: Object.freeze([...r.reasons, reason]) });
+ });
// Terminal-state downrank (recall-trust-suite) is now structural and
// language-agnostic: a result is terminal when its frontmatter
// `status:` field declares a terminal value (controlled vocabulary),
@@ -917,6 +1005,26 @@ export async function search(
const resultsOut =
opts.trust === true ? attachTrustMetadata(config.vault, finalResults) : finalResults;
+ // Progressive disclosure (D3): layer 1. When the caller opts into
+ // `cards`, project the SAME ranked rows into token-cheap cards and
+ // return them on `cards` with an empty `results`. The ranking,
+ // filtering, and evidence pack are computed identically to full mode -
+ // only the surfaced depth differs - so the contract stays
+ // deterministic and the default `full` path is byte-identical.
+ if ((opts.disclosure ?? "full") === "cards") {
+ const cards = resultsOut.map(toSearchCard);
+ return finalize(
+ Object.freeze({
+ results: Object.freeze([] as ReadonlyArray<BrainSearchResult>),
+ cards: Object.freeze(cards),
+ warnings: Object.freeze(warnings),
+ total: cards.length,
+ ...(evidencePack !== undefined ? { evidencePack } : {}),
+ ...(secondPass !== undefined ? { secondPass } : {}),
+ }),
+ );
+ }
+
return finalize(
Object.freeze({
results: Object.freeze(resultsOut),
@@ -931,6 +1039,102 @@ export async function search(
}
}
+/** Max chars of a layer-1 card snippet — enough to judge a hit, cheap to carry. */
+const CARD_SNIPPET_CHARS = 240;
+/** Default layer-3 raw-chunk page size for `expandHit`. */
+const DEFAULT_EXPAND_RAW_LIMIT = 10;
+
+/**
+ * Project a ranked result into a layer-1 card (progressive disclosure):
+ * identity + score + the same `reasons`, a whitespace-collapsed snippet
+ * capped at {@link CARD_SNIPPET_CHARS}, and a `path:Lstart-Lend` pointer
+ * (D2 grammar) over the chunk's stored line span. No full content.
+ */
+function toSearchCard(result: BrainSearchResult): SearchCard {
+ return Object.freeze({
+ chunkId: result.chunkId,
+ documentId: result.documentId,
+ path: result.path,
+ title: result.title,
+ score: result.score,
+ reasons: result.reasons,
+ snippet: cardSnippet(result.content),
+ pointer: formatLinePointer(result.path, result.startLine, result.endLine),
+ ...(result.origin !== undefined ? { origin: result.origin } : {}),
+ });
+}
+
+function cardSnippet(content: string): string {
+ const collapsed = content.replace(/\s+/g, " ").trim();
+ return collapsed.length <= CARD_SNIPPET_CHARS
+ ? collapsed
+ : `${collapsed.slice(0, CARD_SNIPPET_CHARS)}...`;
+}
+
+/**
+ * Progressive disclosure (D3): layers 2 and 3 of a hit the agent already
+ * holds as a layer-1 card. Given the card's `chunkId`, reconstruct the
+ * fuller note (layer 2) from the document's stored chunks and return a
+ * paginated slice of those raw chunks (layer 3), mirroring
+ * `expandSessionRecall`'s cursor contract.
+ *
+ * Read-only by construction: it opens the index in read mode and never
+ * self-heals — a card can only exist because a prior search built the
+ * index, and a rebuild would WRITE it. The layer-2/3 data is pure store
+ * reads (`hydrateChunks` + `getChunksByDocument`), never a new index.
+ */
+export async function expandHit(
+ config: ResolvedSearchConfig,
+ input: ExpandHitInput,
+): Promise<ExpandHitResult> {
+ if (!Number.isInteger(input.chunkId) || input.chunkId < 1) {
+ throw new SearchError("INVALID_INPUT", "chunkId must be a positive integer");
+ }
+ const store = await Store.open(config, { mode: "read" });
+ try {
+ const hit = store.hydrateChunks([input.chunkId]).get(input.chunkId);
+ if (hit === undefined) {
+ throw new SearchError("INVALID_INPUT", `chunk not found: ${input.chunkId}`);
+ }
+ // Document chunks in `chunkIndex` order: the fuller note (layer 2) is
+ // their concatenation; the raw transcript (layer 3) is the same rows.
+ const chunks = store.getChunksByDocument(hit.documentId);
+ const lineStart = chunks.length > 0 ? chunks[0]!.startLine : hit.startLine;
+ const lineEnd = chunks.length > 0 ? chunks[chunks.length - 1]!.endLine : hit.endLine;
+ const note = Object.freeze({
+ documentId: hit.documentId,
+ path: hit.path,
+ title: hit.title,
+ lineStart,
+ lineEnd,
+ pointer: formatLinePointer(hit.path, lineStart, lineEnd),
+ content: chunks.map((c) => c.content).join("\n"),
+ });
+
+ const offset = Math.max(0, Number.parseInt(input.cursor ?? "0", 10) || 0);
+ const rawLimit = Math.max(1, input.rawLimit ?? DEFAULT_EXPAND_RAW_LIMIT);
+ const page = chunks.slice(offset, offset + rawLimit).map((c) =>
+ Object.freeze({
+ chunkId: c.id,
+ chunkIndex: c.chunkIndex,
+ startLine: c.startLine,
+ endLine: c.endLine,
+ pointer: formatLinePointer(hit.path, c.startLine, c.endLine),
+ content: c.content,
+ }),
+ );
+ const nextOffset = offset + rawLimit;
+ return Object.freeze({
+ chunkId: input.chunkId,
+ note,
+ raw_content: Object.freeze(page),
+ next_cursor: nextOffset < chunks.length ? String(nextOffset) : null,
+ });
+ } finally {
+ await store.close();
+ }
+}
+
/**
* Walk outbound links from the ranked hits and merge in related
* documents. Fetches the outbound adjacency level-by-level (each
@@ -1069,6 +1273,40 @@ const UNION_RECORDS_TOTAL = 8;
* fetch up to {@link UNION_RECORDS_PER_TERM} records that DO cover it
* (evidence can span records the primary ranking never surfaced).
*/
+/**
+ * IDF-weighted coverage of the query over a candidate POOL (the partial
+ * self-correcting retry trigger, t_8eb5ca32). Mirrors the result-set
+ * coverage in {@link buildEvidenceVerification} but scores the
+ * pre-ranking candidate chunks: a term is covered when any candidate's
+ * path/title/content contains it. Corpus document frequencies and the
+ * document count come from the store, exactly as the result-set pass
+ * does, so the two reports share one definition of "covered" and one
+ * IDF scale.
+ */
+function coverageOverChunks(
+ store: Store,
+ query: string,
+ chunkIds: ReadonlyArray<number>,
+): CoverageReport {
+ const terms = significantTerms(query);
+ const dfByTerm = store.documentFrequencies(terms);
+ const documentCount = store.counts().documents;
+ const hydrated = store.hydrateChunks(chunkIds);
+ const covered = new Set<string>();
+ for (const h of hydrated.values()) {
+ const haystack = `${h.path}\n${h.title ?? ""}\n${h.content}`;
+ for (const t of terms) {
+ if (!covered.has(t) && termIncludedIn(haystack, t)) covered.add(t);
+ }
+ }
+ return buildCoverageReport({
+ significantTerms: terms,
+ coveredTerms: covered,
+ documentCount,
+ dfByTerm,
+ });
+}
+
function buildEvidenceVerification(
store: Store,
query: string,
diff --git a/src/core/search/types.ts b/src/core/search/types.ts
index 880c51c8..cd07f20c 100644
--- a/src/core/search/types.ts
+++ b/src/core/search/types.ts
@@ -288,6 +288,86 @@ export interface IndexCheckReport {
readonly recommendations: ReadonlyArray<string>;
}
+/**
+ * Result-depth disclosure mode (progressive 3-layer recall). `full`
+ * (default) is the historical flat search: every hit carries its full
+ * chunk content. `cards` returns compact layer-1 {@link SearchCard}s
+ * instead — path/title/score/reasons/snippet/pointer, no full content —
+ * so recall stays token-cheap and the agent pays for depth only by
+ * calling `expandHit` (layer 2 fuller note, layer 3 raw transcript).
+ *
+ * This is NOT the query-lane `expand` flag on {@link SearchOptions}:
+ * that broadens the candidate query, this shapes how much of each
+ * surfaced result is disclosed.
+ */
+export type DisclosureMode = "full" | "cards";
+
+/**
+ * Layer-1 compact card (progressive disclosure). The token-cheap
+ * projection of a ranked {@link BrainSearchResult}: identity, score,
+ * the same explainable `reasons`, a bounded `snippet`, and a
+ * `path:Lstart-Lend` line pointer (D2 grammar) — but never the full
+ * chunk content. Drill to layer 2/3 via `expandHit({ chunkId })`.
+ */
+export interface SearchCard {
+ readonly chunkId: number;
+ readonly documentId: number;
+ readonly path: string;
+ readonly title: string | null;
+ readonly score: number;
+ readonly reasons: ReadonlyArray<string>;
+ readonly snippet: string;
+ /** `path:Lstart-Lend` (or single-line `path:Lstart`) line pointer. */
+ readonly pointer: string;
+ /** Cross-vault origin label, mirrored from the source result when set. */
+ readonly origin?: string;
+}
+
+export interface ExpandHitInput {
+ readonly chunkId: number;
+ /** Raw-chunk page size for layer 3 (default 10). */
+ readonly rawLimit?: number;
+ /** Opaque pagination cursor returned as `next_cursor` by a prior call. */
+ readonly cursor?: string;
+}
+
+/**
+ * Layer-2 fuller note: the hit's whole document, reconstructed from the
+ * store's chunk rows (no new index, no disk read), with the line span it
+ * occupies and a `path:Lstart-Lend` pointer.
+ */
+export interface ExpandedNote {
+ readonly documentId: number;
+ readonly path: string;
+ readonly title: string | null;
+ readonly lineStart: number;
+ readonly lineEnd: number;
+ readonly pointer: string;
+ readonly content: string;
+}
+
+/** Layer-3 raw chunk (the indexed transcript), one page entry. */
+export interface ExpandedRawChunk {
+ readonly chunkId: number;
+ readonly chunkIndex: number;
+ readonly startLine: number;
+ readonly endLine: number;
+ readonly pointer: string;
+ readonly content: string;
+}
+
+/**
+ * `expandHit` result, mirroring `expandSessionRecall`: the fuller note
+ * (layer 2) and a paginated slice of the document's raw chunks (layer 3),
+ * with a `next_cursor` that is null once the transcript is exhausted.
+ */
+export interface ExpandHitResult {
+ readonly chunkId: number;
+ readonly note: ExpandedNote;
+ readonly raw_content: ReadonlyArray<ExpandedRawChunk>;
+ readonly next_cursor: string | null;
+}
+
export interface SearchOptions {
readonly query: string;
readonly limit?: number;
@@ -339,6 +419,14 @@ export interface SearchOptions {
* retrieval. Never silently active.
*/
readonly expand?: boolean;
+ /**
+ * Result-depth disclosure (progressive 3-layer recall). `full`
+ * (default) keeps the flat full-content result shape byte-identical;
+ * `cards` returns compact {@link SearchCard}s on `SearchOutcome.cards`
+ * and an empty `results`, so the caller pays for depth only by calling
+ * `expandHit`. Distinct from the query-lane `expand` flag above.
+ */
+ readonly disclosure?: DisclosureMode;
/**
* Optional named recall profile (Recall & Working-Memory Quality Suite,
* t_98c39dd6): `fast | balanced | thorough` expand to a fixed knob tuple
@@ -433,17 +521,49 @@ export interface SearchOutcome {
readonly results: ReadonlyArray<BrainSearchResult>;
readonly warnings: ReadonlyArray<string>;
readonly total: number;
+ /**
+ * Layer-1 compact cards (progressive disclosure). Present only when the
+ * caller set `disclosure: "cards"`; in that mode `results` is empty and
+ * `total` counts the cards. Absent on the default `full` path, keeping
+ * the legacy outcome shape byte-identical.
+ */
+ readonly cards?: ReadonlyArray<SearchCard>;
readonly evidencePack?: EvidencePack;
/**
* Self-correcting two-pass recall (Time-Aware Recall & Activation
- * Suite, t_ef92dfdc). Present only when a zero-candidate first pass
- * in evidence-pack mode triggered the single broadened OR retry.
+ * Suite, t_ef92dfdc; coverage-driven targeted retry, t_8eb5ca32).
+ * Present only when a single follow-up retry fired in evidence-pack
+ * mode. `kind` distinguishes the two triggers, which are mutually
+ * exclusive (at most one retry per query):
+ * - `"broadened"`: a ZERO-candidate first pass ran one broadened
+ * OR retry over all significant terms.
+ * - `"targeted"`: a non-empty first pass left rare query terms
+ * uncovered (partial coverage below the completeness threshold)
+ * and ran one retry aimed at exactly those uncovered rare terms,
+ * listed in `targetedTerms`.
*/
readonly secondPass?: {
readonly triggered: true;
+ readonly kind: "broadened" | "targeted";
readonly reason: string;
- /** Candidate hits the broadened pass contributed. */
+ /** Candidate hits the retry pass contributed to the pool. */
readonly added: number;
+ /** The uncovered rare terms re-queried by a `"targeted"` retry. */
+ readonly targetedTerms?: ReadonlyArray<string>;
+ };
+ /**
+ * Normalized-confidence chain-stop (t_23c1b929). Present only when
+ * `searchAcrossVaults` ran with `chainStopEnabled` and a completed origin's
+ * top normalized score reached the threshold, so the remaining origins were
+ * skipped. `stoppedAfter` is the origin label that cleared the threshold;
+ * `skipped` lists the origin labels that were not searched. Absent whenever
+ * the chain-stop is off or never triggered, keeping the outcome shape
+ * byte-identical to single-vault search.
+ */
+ readonly chainStop?: {
+ readonly triggered: true;
+ readonly stoppedAfter: string;
+ readonly skipped: ReadonlyArray<string>;
};
}
@@ -542,10 +662,14 @@ export interface ResolvedRecallConfig {
*/
readonly activationEnabled: boolean;
/**
- * Self-correcting two-pass recall (t_ef92dfdc). On by default: a
- * zero-candidate first pass in evidence-pack mode runs exactly one
- * broadened OR retry instead of dead-ending in an abstention.
- * Plain (non-evidence-pack) searches never broaden either way.
+ * Self-correcting two-pass recall (t_ef92dfdc; coverage-driven
+ * targeted retry, t_8eb5ca32). On by default and the kill switch for
+ * BOTH retry triggers in evidence-pack mode: a zero-candidate first
+ * pass runs one broadened OR retry, and a non-empty first pass whose
+ * IDF-weighted coverage falls below the completeness threshold with
+ * rare terms still uncovered runs one targeted retry aimed at those
+ * uncovered rare terms. At most one retry fires per query. Plain
+ * (non-evidence-pack) searches never retry either way.
*/
readonly twoPassEnabled: boolean;
/**
@@ -563,6 +687,18 @@ export interface ResolvedRecallConfig {
* silently.
*/
readonly selfTuningEnabled: boolean;
+ /**
+ * Normalized-confidence chain-stop for cross-vault recall (t_23c1b929).
+ * Off by default: when true, `searchAcrossVaults` stops querying further
+ * origins as soon as a completed origin's top NORMALIZED [0,1] result
+ * score reaches `chainStopScore`, recording the skipped origins. Gates on
+ * the normalized result score, never the raw lane score, so a tiny-corpus
+ * origin with a high raw score but a low normalized score never
+ * short-circuits. Default-off keeps cross-vault results bit-identical.
+ */
+ readonly chainStopEnabled: boolean;
+ /** Normalized-score threshold in [0, 1] that triggers the chain-stop. */
+ readonly chainStopScore: number;
}
export interface ResolvedSearchConfig {
@@ -580,6 +716,15 @@ export interface ResolvedSearchConfig {
readonly ignoreRules: ReadonlyArray<VaultIgnoreRule>;
readonly chunkSize: number;
readonly chunkOverlap: number;
+ /**
+ * Chunk floor (min tokens) — the heading-boundary flush threshold in
+ * the markdown chunker (`minTokens`). Resolved from
+ * `search_chunk_min_size` / `OPEN_SECOND_BRAIN_SEARCH_CHUNK_MIN_SIZE`.
+ * Default 100 (the chunker's `DEFAULT_MIN_TOKENS`); kept stable so
+ * vaults that don't set it hash identical chunks across Syncthing
+ * peers. Must be ≥ 1 and ≤ `chunkSize`.
+ */
+ readonly chunkMinSize: number;
readonly keywordWeight: number;
readonly semanticWeight: number;
/**
diff --git a/src/mcp/brain/pack-tools.ts b/src/mcp/brain/pack-tools.ts
index d3f819c3..2a3a1a41 100644
--- a/src/mcp/brain/pack-tools.ts
+++ b/src/mcp/brain/pack-tools.ts
@@ -33,6 +33,8 @@ import {
type ContextPresetCurrentConfig,
} from "../../core/brain/context-presets.ts";
import { extractPreCompactRecords } from "../../core/brain/pre-compact-extract.ts";
+import { resolveLogEventTraces } from "../../core/brain/event-trace.ts";
+import { BRAIN_LOG_EVENT_KIND_SET, type BrainLogEventKind } from "../../core/brain/types.ts";
import { INVALID_PARAMS, MCPError } from "../protocol.ts";
import type { ServerContext, ToolDefinition } from "../tools.ts";
import { MCP_PREVIEW_BUDGET } from "../preview-budget.ts";
@@ -197,6 +199,44 @@ async function toolBrainContextReceipts(
throw new MCPError(INVALID_PARAMS, "brain_context_receipts: operation must be list or show");
}
+// ----- brain_event_trace ---------------------------------------------------
+
+async function toolBrainEventTrace(
+ ctx: ServerContext,
+ args: Record<string, unknown>,
+): Promise<Record<string, unknown>> {
+ const kind = optionalStringArg("brain_event_trace", args, "kind");
+ if (kind !== undefined && !BRAIN_LOG_EVENT_KIND_SET.has(kind)) {
+ throw new MCPError(INVALID_PARAMS, `brain_event_trace: unknown event kind '${kind}'`);
+ }
+ const date = optionalStringArg("brain_event_trace", args, "date");
+ const at = optionalStringArg("brain_event_trace", args, "at");
+ const sessionId = optionalStringArg("brain_event_trace", args, "session_id");
+ const limit = coercePositiveInteger("brain_event_trace", "limit", args["limit"]);
+ const keepPrivate = coerceBool(args, "keep_private");
+
+ let events;
+ try {
+ events = resolveLogEventTraces(ctx.vault, {
+ ...(date !== undefined ? { date } : {}),
+ ...(at !== undefined ? { at } : {}),
+ ...(kind !== undefined ? { kind: kind as BrainLogEventKind } : {}),
+ ...(sessionId !== undefined ? { sessionId } : {}),
+ ...(limit !== undefined ? { limit } : {}),
+ ...(keepPrivate === true ? { keepPrivate: true } : {}),
+ });
+ } catch (err) {
+ throw new MCPError(INVALID_PARAMS, `brain_event_trace: ${(err as Error).message}`);
+ }
+
+ return {
+ vault_path: ctx.vault,
+ total: events.length,
+ trace_total: events.reduce((sum, e) => sum + e.traceCount, 0),
+ events,
+ };
+}
+
function receiptOptionsFromArgs(
tool: string,
args: Record<string, unknown>,
@@ -529,6 +569,46 @@ export const PACK_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([
},
handler: toolBrainContextReceipts,
},
+ {
+ name: "brain_event_trace",
+ description:
+ "Join logged Brain events to the continuity records (recall telemetry, context receipts, generation reports) attached to them by shared correlation ids (session_id, turn_id, artifact refs) — the integrated 'why did the agent do this?' reader. Defaults to today's log. Read-only.",
+ inputSchema: {
+ type: "object",
+ properties: {
+ date: {
+ type: "string",
+ pattern: "^\\d{4}-\\d{2}-\\d{2}$",
+ description: "Log day to read (YYYY-MM-DD, UTC). Defaults to today.",
+ },
+ at: {
+ type: "string",
+ pattern: "^\\d{2}:\\d{2}:\\d{2}$",
+ description: "Pin a single event by its HH:MM:SS UTC stamp.",
+ },
+ kind: {
+ type: "string",
+ description:
+ "Optional filter by Brain log event kind (e.g. write-session, apply-evidence).",
+ },
+ session_id: {
+ type: "string",
+ description: "Restrict to events whose body carries this session id.",
+ },
+ limit: {
+ type: "integer",
+ minimum: 1,
+ description: "Optional maximum number of events to return.",
+ },
+ keep_private: {
+ type: "boolean",
+ description: "Keep continuity records flagged private (default: drop them).",
+ },
+ },
+ additionalProperties: false,
+ },
+ handler: toolBrainEventTrace,
+ },
{
name: "brain_context_presets",
description:
diff --git a/src/mcp/registry-guard.ts b/src/mcp/registry-guard.ts
index 60766513..2f55d091 100644
--- a/src/mcp/registry-guard.ts
+++ b/src/mcp/registry-guard.ts
@@ -117,6 +117,7 @@ export const PREVIEW_BUDGET_EXEMPT: Readonly<Record<string, string>> = Object.fr
brain_mcp_landscape: "fixed-shape landscape summary",
brain_session_describe: "single-session metadata only",
brain_context_receipts: "bounded receipt list per session",
+ brain_event_trace: "bounded event→trace join summary; full records via per-kind readers",
brain_context_presets: "small preset list",
brain_pre_compact_extract: "bounded extract; host injects it whole by design",
brain_skill_proposals: "bounded proposal list",
diff --git a/src/mcp/search-tools.ts b/src/mcp/search-tools.ts
index d6705ea3..598d8a22 100644
--- a/src/mcp/search-tools.ts
+++ b/src/mcp/search-tools.ts
@@ -13,6 +13,7 @@
import {
captureRecallFeedback,
evaluateSurfacingGate,
+ expandHit,
indexStatus,
resolveSearchConfig,
search,
@@ -75,6 +76,12 @@ const SEARCH_INPUT_SCHEMA: Record<string, unknown> = {
limit: { type: "integer", minimum: 1, maximum: MCP_LIMIT_MAX },
semantic: { type: "boolean" },
keyword_only: { type: "boolean" },
+ disclosure: {
+ type: "string",
+ enum: ["full", "cards"],
+ description:
+ "Result depth: 'full' (default) returns full chunk content; 'cards' returns token-cheap layer-1 cards — drill a hit with brain_search_expand.",
+ },
profile: {
type: "string",
enum: [...RECALL_PROFILE_NAMES],
@@ -170,7 +177,10 @@ const SEARCH_OUTPUT_SCHEMA: NonNullable<ToolDefinition["outputSchema"]> = {
],
properties: {
path: { type: "string" },
- title: { type: "string" },
+ // Titles are nullable for markdown files without frontmatter/title.
+ // The lightweight contract validator has no union types, so leave
+ // this field unconstrained while keeping it required in the shape.
+ title: {},
content: { type: "string" },
score: { type: "number" },
startLine: { type: "integer" },
@@ -217,6 +227,34 @@ const SEARCH_OUTPUT_SCHEMA: NonNullable<ToolDefinition["outputSchema"]> = {
},
},
},
+ cards: {
+ type: "array",
+ items: {
+ type: "object",
+ required: [
+ "path",
+ "title",
+ "score",
+ "snippet",
+ "pointer",
+ "reasons",
+ "document_id",
+ "chunk_id",
+ ],
+ properties: {
+ path: { type: "string" },
+ // Nullable, same as full search result titles above.
+ title: {},
+ score: { type: "number" },
+ snippet: { type: "string" },
+ pointer: { type: "string" },
+ reasons: { type: "array", items: { type: "string" } },
+ document_id: { type: "integer" },
+ chunk_id: { type: "integer" },
+ origin: { type: "string" },
+ },
+ },
+ },
warnings: { type: "array", items: { type: "string" } },
total: { type: "integer" },
recall_hint: { type: "string" },
@@ -371,6 +409,10 @@ async function toolBrainSearch(
const semantic = coerceBoolOptional(args, "semantic");
const keywordOnly = coerceBoolOptional(args, "keyword_only") ?? false;
+ const disclosure = coerceStringOptional(args, "disclosure", 16);
+ if (disclosure !== undefined && disclosure !== "full" && disclosure !== "cards") {
+ throw new MCPError(INVALID_PARAMS, "argument 'disclosure' must be 'full' or 'cards'");
+ }
const explain = coerceBoolOptional(args, "explain") ?? false;
const trust = coerceBoolOptional(args, "trust") ?? false;
const rerank = coerceBoolOptional(args, "rerank") ?? false;
@@ -441,6 +483,7 @@ async function toolBrainSearch(
keywordOnly,
pathPrefix,
...(profile !== undefined ? { profile } : {}),
+ ...(disclosure === "cards" ? { disclosure: "cards" as const } : {}),
...(properties !== undefined ? { properties } : {}),
...(visibility !== undefined ? { visibility } : {}),
...(agentScope !== undefined ? { agentScope } : {}),
@@ -499,16 +542,20 @@ async function toolBrainSearch(
}
const recallHint = deriveRecallHint(outcome.results, outcome.total);
+ // Under disclosure:'cards' the surfaced rows live on `cards`, not
+ // `results`; both shapes carry documentId/chunkId/path/score, so the
+ // telemetry count/status/top-artifacts stay honest either way.
+ const surfaced = outcome.cards ?? outcome.results;
const telemetryRecord = emitGatedTelemetry(telemetry || undefined, () =>
emitRecallTelemetry(ctx.vault, {
host: telemetryHost,
...(telemetrySessionId !== undefined ? { sessionId: telemetrySessionId } : {}),
...(telemetryTurnId !== undefined ? { turnId: telemetryTurnId } : {}),
mode: "search",
- status: outcome.results.length > 0 ? "ok" : "empty",
+ status: surfaced.length > 0 ? "ok" : "empty",
durationMs: Date.now() - startedAtMs,
- resultCount: outcome.results.length,
- topArtifacts: outcome.results.slice(0, 10).map((result) => ({
+ resultCount: surfaced.length,
+ topArtifacts: surfaced.slice(0, 10).map((result) => ({
id: `${result.documentId}:${result.chunkId}`,
path: result.path,
score: result.score,
@@ -541,6 +588,21 @@ async function toolBrainSearch(
...(outcome.evidencePack ? { why_retrieved: r.reasons } : {}),
...(r.relations && r.relations.length > 0 ? { relations: r.relations } : {}),
})),
+ ...(outcome.cards
+ ? {
+ cards: outcome.cards.map((c) => ({
+ path: c.path,
+ title: c.title,
+ score: c.score,
+ snippet: c.snippet,
+ pointer: c.pointer,
+ reasons: c.reasons,
+ document_id: c.documentId,
+ chunk_id: c.chunkId,
+ ...(c.origin !== undefined ? { origin: c.origin } : {}),
+ })),
+ }
+ : {}),
warnings: outcome.warnings,
total: outcome.total,
...(outcome.evidencePack ? { evidence_pack: mcpEvidencePack(outcome.evidencePack) } : {}),
@@ -570,6 +632,7 @@ function mcpEvidencePack(
path: record.path,
document_id: record.documentId,
chunk_id: record.chunkId,
+ line_pointer: record.linePointer,
matched_terms: record.matchedTerms,
missing_terms: record.missingTerms,
support_coverage: record.supportCoverage,
@@ -939,6 +1002,132 @@ function coerceIntInRange(
return raw;
}
+const SEARCH_EXPAND_INPUT_SCHEMA: Record<string, unknown> = {
+ type: "object",
+ properties: {
+ chunk_id: {
+ type: "integer",
+ minimum: 1,
+ description: "The chunk_id of a layer-1 card (from brain_search disclosure:'cards').",
+ },
+ raw_limit: {
+ type: "integer",
+ minimum: 1,
+ maximum: MCP_LIMIT_MAX,
+ description: "Layer-3 raw-chunk page size (default 10).",
+ },
+ cursor: {
+ type: "string",
+ maxLength: 32,
+ description: "Pagination cursor returned as next_cursor by a prior expand call.",
+ },
+ },
+ required: ["chunk_id"],
+ additionalProperties: false,
+};
+
+const SEARCH_EXPAND_OUTPUT_SCHEMA: NonNullable<ToolDefinition["outputSchema"]> = {
+ type: "object",
+ required: ["chunk_id", "note", "raw_content", "next_cursor"],
+ properties: {
+ chunk_id: { type: "integer" },
+ note: {
+ type: "object",
+ required: ["document_id", "path", "title", "line_start", "line_end", "pointer", "content"],
+ properties: {
+ document_id: { type: "integer" },
+ path: { type: "string" },
+ // Nullable for title-less notes; the local schema validator does not
+ // support union types, so keep the field present but unconstrained.
+ title: {},
+ line_start: { type: "integer" },
+ line_end: { type: "integer" },
+ pointer: { type: "string" },
+ content: { type: "string" },
+ },
+ },
+ raw_content: {
+ type: "array",
+ items: {
+ type: "object",
+ required: ["chunk_id", "chunk_index", "start_line", "end_line", "pointer", "content"],
+ properties: {
+ chunk_id: { type: "integer" },
+ chunk_index: { type: "integer" },
+ start_line: { type: "integer" },
+ end_line: { type: "integer" },
+ pointer: { type: "string" },
+ content: { type: "string" },
+ },
+ },
+ },
+ // String cursor or null when the raw transcript is exhausted.
+ next_cursor: {},
+ },
+};
+
+/**
+ * `brain_search_expand` (progressive disclosure layers 2 + 3): drill a
+ * layer-1 card into the fuller note and the paginated raw chunk
+ * transcript. Read-only; reuses the existing store read, never rebuilds
+ * the index.
+ */
+async function toolBrainSearchExpand(
+ ctx: ServerContext,
+ args: Record<string, unknown>,
+): Promise<Record<string, unknown>> {
+ const rawChunk = args["chunk_id"];
+ if (typeof rawChunk !== "number" || !Number.isInteger(rawChunk) || rawChunk < 1) {
+ throw new MCPError(INVALID_PARAMS, "argument 'chunk_id' must be a positive integer");
+ }
+ const rawLimit = coerceIntInRange(args, "raw_limit", 1, MCP_LIMIT_MAX);
+ const cursor = coerceStringOptional(args, "cursor", 32);
+ const config = resolveSearchConfig({
+ vault: ctx.vault,
+ configPath: ctx.configPath ?? undefined,
+ });
+ let result;
+ try {
+ result = await withTimeout(
+ expandHit(config, {
+ chunkId: rawChunk,
+ ...(rawLimit !== undefined ? { rawLimit } : {}),
+ ...(cursor !== undefined ? { cursor } : {}),
+ }),
+ SEARCH_TIMEOUT_MS,
+ searchTimeoutError,
+ );
+ } catch (e) {
+ if (e instanceof SearchError) throw searchErrorToMcp(e);
+ if (e instanceof MCPError) throw e;
+ throw new MCPError(INTERNAL_ERROR, e instanceof Error ? e.message : String(e));
+ }
+ return {
+ chunk_id: result.chunkId,
+ note: {
+ document_id: result.note.documentId,
+ path: result.note.path,
+ title: result.note.title,
+ line_start: result.note.lineStart,
+ line_end: result.note.lineEnd,
+ pointer: result.note.pointer,
+ // Drill-down tool: return the full note (layer 2) and raw chunks
+ // (layer 3), not a snippet. The preview budget caps the envelope
+ // and hands back an artifact_id when the payload is large.
+ content: result.note.content,
+ },
+ raw_content: result.raw_content.map((c) => ({
+ chunk_id: c.chunkId,
+ chunk_index: c.chunkIndex,
+ start_line: c.startLine,
+ end_line: c.endLine,
+ pointer: c.pointer,
+ content: c.content,
+ })),
+ next_cursor: result.next_cursor,
+ };
+}
+
export const SEARCH_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([
{
name: "brain_recall_feedback",
@@ -965,6 +1154,15 @@ export const SEARCH_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([
previewBudget: MCP_PREVIEW_BUDGET,
handler: toolBrainSearch,
},
+ {
+ name: "brain_search_expand",
+ description:
+ "Progressive disclosure layers 2 + 3: drill a brain_search card (by chunk_id) into the fuller note and the paginated raw chunk transcript. Read-only; reuses the existing index.",
+ inputSchema: SEARCH_EXPAND_INPUT_SCHEMA,
+ outputSchema: SEARCH_EXPAND_OUTPUT_SCHEMA,
+ previewBudget: MCP_PREVIEW_BUDGET,
+ handler: toolBrainSearchExpand,
+ },
{
name: "brain_eval",
description:
diff --git a/tests/cli/brain-event-trace.test.ts b/tests/cli/brain-event-trace.test.ts
new file mode 100644
index 00000000..eb40b8ed
--- /dev/null
+++ b/tests/cli/brain-event-trace.test.ts
@@ -0,0 +1,47 @@
+/**
+ * `o2b brain event-trace` CLI surface: usage errors (bad --kind / --limit,
+ * malformed selectors) exit with code 2 and a plain stderr message, distinct
+ * from the exit-1 runtime-failure path. A clean run over an empty log exits 0.
+ */
+
+import { afterEach, beforeEach, expect, test } from "bun:test";
+import { mkdirSync, mkdtempSync, rmSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { runCli } from "../helpers/run-cli.ts";
+
+let tmp: string;
+let vault: string;
+
+beforeEach(() => {
+ tmp = mkdtempSync(join(tmpdir(), "o2b-cli-event-trace-"));
+ vault = join(tmp, "vault");
+ mkdirSync(join(vault, "Brain", "log"), { recursive: true });
+});
+
+afterEach(() => {
+ rmSync(tmp, { recursive: true, force: true });
+});
+
+test("an unknown --kind is a usage error (exit 2, plain stderr)", async () => {
+ const r = await runCli(["brain", "event-trace", "--vault", vault, "--kind", "not-a-kind"]);
+ expect(r.returncode).toBe(2);
+ expect(r.stderr).toContain("unknown event kind");
+});
+
+test("a non-positive --limit is a usage error (exit 2)", async () => {
+ const r = await runCli(["brain", "event-trace", "--vault", vault, "--limit", "0"]);
+ expect(r.returncode).toBe(2);
+ expect(r.stderr).toContain("--limit");
+});
+
+test("a malformed --at selector is a usage error (exit 2)", async () => {
+ const r = await runCli(["brain", "event-trace", "--vault", vault, "--at", "10am"]);
+ expect(r.returncode).toBe(2);
+});
+
+test("a clean run over an empty log exits 0", async () => {
+ const r = await runCli(["brain", "event-trace", "--vault", vault, "--json"]);
+ expect(r.returncode).toBe(0);
+});
diff --git a/tests/core/brain/event-trace.test.ts b/tests/core/brain/event-trace.test.ts
new file mode 100644
index 00000000..b6c29999
--- /dev/null
+++ b/tests/core/brain/event-trace.test.ts
@@ -0,0 +1,213 @@
+import { afterEach, beforeEach, describe, expect, test } from "bun:test";
+import { mkdirSync, mkdtempSync, rmSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { appendLogEvent } from "../../../src/core/brain/log.ts";
+import { appendContinuityRecord } from "../../../src/core/brain/continuity/store.ts";
+import {
+ attachTracesToEvent,
+ extractEventCorrelation,
+ resolveLogEventTraces,
+} from "../../../src/core/brain/event-trace.ts";
+import { BRAIN_LOG_EVENT_KIND } from "../../../src/core/brain/types.ts";
+
+let vault: string;
+
+beforeEach(() => {
+ vault = mkdtempSync(join(tmpdir(), "o2b-event-trace-"));
+ mkdirSync(join(vault, "Brain", "log"), { recursive: true });
+});
+
+afterEach(() => {
+ rmSync(vault, { recursive: true, force: true });
+});
+
+const DATE = "2026-06-15";
+
+function logSessionEvent(session: string, at: string): void {
+ appendLogEvent(vault, {
+ timestamp: `${DATE}T${at}Z`,
+ eventType: BRAIN_LOG_EVENT_KIND.writeSession,
+ agent: "tester",
+ body: { session_id: session, kind: "note", status: "committed", target: "Brain/x.md" },
+ });
+}
+
+function recallTelemetry(session: string, at: string): string {
+ return appendContinuityRecord(vault, {
+ kind: "recall_telemetry",
+ createdAt: `${DATE}T${at}Z`,
+ sourceRefs: [],
+ payload: { session_id: session, mode: "context_pack", status: "ok" },
+ }).id;
+}
+
+describe("extractEventCorrelation", () => {
+ test("lifts session_id, turn_id, and artifact wikilinks/paths", () => {
+ const corr = extractEventCorrelation({
+ timestamp: `${DATE}T10:00:00Z`,
+ eventType: BRAIN_LOG_EVENT_KIND.applyEvidence,
+ body: {
+ session_id: "sess-1",
+ turn_id: "turn-9",
+ preference: "[[pref-keep-short]]",
+ artifact: "Brain/notes/foo.md",
+ },
+ });
+ expect(corr.sessionId).toBe("sess-1");
+ expect(corr.turnId).toBe("turn-9");
+ // wikilink target + bare path + their basenames are all join candidates
+ expect(corr.artifacts).toContain("pref-keep-short");
+ expect(corr.artifacts).toContain("Brain/notes/foo.md");
+ expect(corr.artifacts).toContain("foo");
+ });
+
+ test("returns no artifacts and no ids for a bare note event", () => {
+ const corr = extractEventCorrelation({
+ timestamp: `${DATE}T10:00:00Z`,
+ eventType: BRAIN_LOG_EVENT_KIND.note,
+ body: { text: "shipped v1", agent: "tester" },
+ });
+ expect(corr.sessionId).toBeUndefined();
+ expect(corr.artifacts).toEqual([]);
+ });
+});
+
+describe("attachTracesToEvent", () => {
+ test("joins by session and reports the reason", () => {
+ const records = [
+ {
+ schema: "o2b.continuity.v1",
+ legacy: false,
+ id: "ctn_a",
+ kind: "recall_telemetry",
+ createdAt: `${DATE}T10:00:00Z`,
+ sourceRefs: [],
+ payload: {},
+ private: false,
+ redacted: false,
+ sessionId: "sess-1",
+ },
+ {
+ schema: "o2b.continuity.v1",
+ legacy: false,
+ id: "ctn_b",
+ kind: "recall_telemetry",
+ createdAt: `${DATE}T10:01:00Z`,
+ sourceRefs: [],
+ payload: {},
+ private: false,
+ redacted: false,
+ sessionId: "other",
+ },
+ ] as const;
+ const attached = attachTracesToEvent(records, { sessionId: "sess-1", artifacts: [] });
+ expect(attached.map((t) => t.id)).toEqual(["ctn_a"]);
+ expect(attached[0]!.joinedBy).toEqual(["session"]);
+ });
+
+ test("joins by artifact via sourceRefs id and path", () => {
+ const records = [
+ {
+ schema: "o2b.continuity.v1",
+ legacy: false,
+ id: "ctn_c",
+ kind: "context_receipt",
+ createdAt: `${DATE}T10:00:00Z`,
+ sourceRefs: [{ id: "pref-keep-short", path: "Brain/preferences/pref-keep-short.md" }],
+ payload: {},
+ private: false,
+ redacted: false,
+ },
+ ] as const;
+ const attached = attachTracesToEvent(records, { artifacts: ["pref-keep-short"] });
+ expect(attached).toHaveLength(1);
+ expect(attached[0]!.joinedBy).toEqual(["artifact"]);
+ });
+});
+
+describe("resolveLogEventTraces", () => {
+ test("attaches session-correlated continuity records to a logged event", () => {
+ logSessionEvent("sess-1", "10:00:00");
+ const want = recallTelemetry("sess-1", "10:00:05");
+ recallTelemetry("sess-other", "10:00:06");
+
+ const result = resolveLogEventTraces(vault, { date: DATE });
+ expect(result).toHaveLength(1);
+ expect(result[0]!.event.sessionId).toBe("sess-1");
+ expect(result[0]!.traceCount).toBe(1);
+ expect(result[0]!.traces[0]!.id).toBe(want);
+ expect(result[0]!.traces[0]!.joinedBy).toEqual(["session"]);
+ });
+
+ test("limit caps the number of events; limit 0 yields an empty list", () => {
+ logSessionEvent("sess-1", "10:00:00");
+ logSessionEvent("sess-2", "11:00:00");
+ logSessionEvent("sess-3", "12:00:00");
+
+ // The cap is the count returned, evaluated before each push: limit 0 is
+ // an empty list (not one event), limit 2 returns exactly two.
+ expect(resolveLogEventTraces(vault, { date: DATE, limit: 0 })).toEqual([]);
+ expect(resolveLogEventTraces(vault, { date: DATE, limit: 2 })).toHaveLength(2);
+ expect(resolveLogEventTraces(vault, { date: DATE })).toHaveLength(3);
+ });
+
+ test("--at pins a single event by HH:MM:SS", () => {
+ logSessionEvent("sess-1", "10:00:00");
+ logSessionEvent("sess-2", "11:00:00");
+ recallTelemetry("sess-2", "11:00:01");
+
+ const result = resolveLogEventTraces(vault, { date: DATE, at: "11:00:00" });
+ expect(result).toHaveLength(1);
+ expect(result[0]!.event.sessionId).toBe("sess-2");
+ expect(result[0]!.traceCount).toBe(1);
+ });
+
+ test("--session-id filters events; --kind filters by event kind", () => {
+ logSessionEvent("sess-1", "10:00:00");
+ logSessionEvent("sess-2", "11:00:00");
+
+ const bySession = resolveLogEventTraces(vault, { date: DATE, sessionId: "sess-2" });
+ expect(bySession.map((r) => r.event.sessionId)).toEqual(["sess-2"]);
+
+ const byKind = resolveLogEventTraces(vault, {
+ date: DATE,
+ kind: BRAIN_LOG_EVENT_KIND.note,
+ });
+ expect(byKind).toEqual([]);
+ });
+
+ test("events with no attached context still surface with an empty trace list", () => {
+ appendLogEvent(vault, {
+ timestamp: `${DATE}T10:00:00Z`,
+ eventType: BRAIN_LOG_EVENT_KIND.note,
+ body: { text: "no correlation here", agent: "tester" },
+ });
+ const result = resolveLogEventTraces(vault, { date: DATE });
+ expect(result).toHaveLength(1);
+ expect(result[0]!.traceCount).toBe(0);
+ expect(result[0]!.traces).toEqual([]);
+ });
+
+ test("private continuity records are dropped unless keepPrivate is set", () => {
+ logSessionEvent("sess-1", "10:00:00");
+ appendContinuityRecord(vault, {
+ kind: "recall_telemetry",
+ createdAt: `${DATE}T10:00:05Z`,
+ sourceRefs: [],
+ payload: { session_id: "sess-1", note: "<private>secret</private>" },
+ });
+
+ const dropped = resolveLogEventTraces(vault, { date: DATE });
+ expect(dropped[0]!.traceCount).toBe(0);
+
+ const kept = resolveLogEventTraces(vault, { date: DATE, keepPrivate: true });
+ expect(kept[0]!.traceCount).toBe(1);
+ expect(kept[0]!.traces[0]!.private).toBe(true);
+ });
+
+ test("rejects a malformed --at", () => {
+ expect(() => resolveLogEventTraces(vault, { date: DATE, at: "10am" })).toThrow(/HH:MM:SS/);
+ });
+});
diff --git a/tests/core/brain/session-recall.test.ts b/tests/core/brain/session-recall.test.ts
index 52fb34df..73335dee 100644
--- a/tests/core/brain/session-recall.test.ts
+++ b/tests/core/brain/session-recall.test.ts
@@ -94,4 +94,46 @@ describe("session recall DAG", () => {
expect(expanded.raw_content.map((item) => item.turn_id)).toEqual(["t1", "t2"]);
expect(expanded.next_cursor).toBe("2");
});
+
+ test("annotates each hit with the 1-based line span of the snippet match", () => {
+ importSessionRecall(vault, {
+ sessionId: "session-b",
+ turns: [turn("t1", "user", "First line.\nSecond line mentions receipts.")],
+ summaryGroupSize: 2,
+ createdAt: "2026-05-20T17:00:00.000Z",
+ });
+
+ const search = searchSessionRecall(vault, {
+ query: "receipts",
+ sessionId: "session-b",
+ limit: 4,
+ });
+ const hit = search.hits.find((candidate) => candidate.kind === "session_turn");
+ expect(hit?.line_start).toBe(2);
+ expect(hit?.line_end).toBe(2);
+ });
+
+ test("line span stays correct when an earlier line lowercases to more code units", () => {
+ // U+0130 (Turkish dotted capital I) lowercases to two code units, so a
+ // match offset taken from the lowercased copy would be shifted ahead of
+ // its true position in the original text. Nine of them on line 1 shift a
+ // line-2 match onto line 3 under the bug; the offset must index the
+ // original text so the line span stays line 2.
+ importSessionRecall(vault, {
+ sessionId: "session-locale",
+ turns: [turn("t1", "user", `${"İ".repeat(9)}\nreceipts\nfinal line three`)],
+ summaryGroupSize: 2,
+ createdAt: "2026-05-20T18:00:00.000Z",
+ });
+
+ const search = searchSessionRecall(vault, {
+ query: "receipts",
+ sessionId: "session-locale",
+ limit: 4,
+ });
+ const hit = search.hits.find((candidate) => candidate.kind === "session_turn");
+ expect(hit).toBeDefined();
+ expect(hit?.line_start).toBe(2);
+ expect(hit?.line_end).toBe(2);
+ });
});
diff --git a/tests/core/search/config.test.ts b/tests/core/search/config.test.ts
index e84039e1..89cc5a40 100644
--- a/tests/core/search/config.test.ts
+++ b/tests/core/search/config.test.ts
@@ -14,6 +14,7 @@ const ENV_KEYS = [
"OPEN_SECOND_BRAIN_SEARCH_SEMANTIC",
"OPEN_SECOND_BRAIN_SEARCH_CHUNK_SIZE",
"OPEN_SECOND_BRAIN_SEARCH_CHUNK_OVERLAP",
+ "OPEN_SECOND_BRAIN_SEARCH_CHUNK_MIN_SIZE",
"OPEN_SECOND_BRAIN_SEARCH_KW_WEIGHT",
"OPEN_SECOND_BRAIN_SEARCH_SEM_WEIGHT",
"OPEN_SECOND_BRAIN_SEARCH_IGNORE",
@@ -35,6 +36,8 @@ const ENV_KEYS = [
"OPEN_SECOND_BRAIN_SEARCH_CACHE_TTL",
"OPEN_SECOND_BRAIN_SEARCH_SHUTDOWN_GRACE",
"OPEN_SECOND_BRAIN_SEARCH_RESUME_REINDEX",
+ "OPEN_SECOND_BRAIN_SEARCH_CHAIN_STOP",
+ "OPEN_SECOND_BRAIN_SEARCH_CHAIN_STOP_SCORE",
];
beforeEach(() => {
@@ -62,6 +65,7 @@ test("defaults are returned when config and env are empty", () => {
expect(cfg.dbPath).toBe(join(tmp, ".open-second-brain", "brain.sqlite"));
expect(cfg.chunkSize).toBe(800);
expect(cfg.chunkOverlap).toBe(100);
+ expect(cfg.chunkMinSize).toBe(100);
expect(cfg.keywordWeight).toBe(0.6);
expect(cfg.semanticWeight).toBe(0.4);
expect(cfg.semantic.enabled).toBe(false);
@@ -124,6 +128,23 @@ test("query cache defaults off with a 300s TTL and is toggleable", () => {
expect(on.cacheTtlSeconds).toBe(60);
});
+test("cross-vault chain-stop defaults off at a 0.8 threshold and is toggleable", () => {
+ writeFileSync(configPath, `vault: "${tmp}"\n`);
+ const def = resolveSearchConfig({ vault: tmp, configPath }).recall;
+ expect(def.chainStopEnabled).toBe(false);
+ expect(def.chainStopScore).toBe(0.8);
+ process.env["OPEN_SECOND_BRAIN_SEARCH_CHAIN_STOP"] = "true";
+ process.env["OPEN_SECOND_BRAIN_SEARCH_CHAIN_STOP_SCORE"] = "0.9";
+ const on = resolveSearchConfig({ vault: tmp, configPath }).recall;
+ expect(on.chainStopEnabled).toBe(true);
+ expect(on.chainStopScore).toBe(0.9);
+});
+
+test("chain-stop score outside [0,1] is rejected", () => {
+ writeFileSync(configPath, `vault: "${tmp}"\nsearch_chain_stop_score: "1.5"\n`);
+ expect(() => resolveSearchConfig({ vault: tmp, configPath })).toThrow(/search_chain_stop_score/);
+});
+
test("non-positive recency scale is rejected", () => {
writeFileSync(configPath, `vault: "${tmp}"\nsearch_recency_scale: "0"\n`);
expect(() => resolveSearchConfig({ vault: tmp, configPath })).toThrow(/search_recency_scale/);
@@ -265,6 +286,25 @@ test("chunk overlap must be non-negative and smaller than chunk size", () => {
expect(() => resolveSearchConfig({ vault: tmp, configPath })).toThrow(/smaller than/);
});
+test("chunk minimum size resolves from config and env overrides config", () => {
+ writeFileSync(configPath, `vault: "${tmp}"\nsearch_chunk_min_size: "50"\n`);
+ expect(resolveSearchConfig({ vault: tmp, configPath }).chunkMinSize).toBe(50);
+
+ process.env["OPEN_SECOND_BRAIN_SEARCH_CHUNK_MIN_SIZE"] = "75";
+ expect(resolveSearchConfig({ vault: tmp, configPath }).chunkMinSize).toBe(75);
+});
+
+test("chunk minimum size must be a positive integer not exceeding chunk size", () => {
+ writeFileSync(configPath, `vault: "${tmp}"\nsearch_chunk_min_size: "0"\n`);
+ expect(() => resolveSearchConfig({ vault: tmp, configPath })).toThrow(/search_chunk_min_size/);
+
+ writeFileSync(
+ configPath,
+ `vault: "${tmp}"\nsearch_chunk_size: "300"\nsearch_chunk_min_size: "400"\n`,
+ );
+ expect(() => resolveSearchConfig({ vault: tmp, configPath })).toThrow(/exceed/);
+});
+
test("overrides are validated after merging", () => {
writeFileSync(configPath, `vault: "${tmp}"\n`);
expect(() =>
diff --git a/tests/core/search/coverage-targeted-recall.test.ts b/tests/core/search/coverage-targeted-recall.test.ts
new file mode 100644
index 00000000..4b16ae01
--- /dev/null
+++ b/tests/core/search/coverage-targeted-recall.test.ts
@@ -0,0 +1,181 @@
+/**
+ * Coverage-driven targeted self-correcting recall on partial misses
+ * (t_8eb5ca32). Extends the zero-candidate broadened retry: when a
+ * non-empty first pass leaves rare query terms uncovered (IDF-weighted
+ * coverage below the completeness threshold), search issues ONE targeted
+ * follow-up built from exactly those uncovered rare terms, merges the
+ * recovered candidates, and regenerates the evidence pack. The trigger
+ * is deterministic and LLM-free; the pack still abstains on any term
+ * left uncovered after the retry.
+ */
+
+import { test, expect, beforeEach, afterEach, describe } from "bun:test";
+
+import { buildCoverageReport, planTargetedRetry } from "../../../src/core/search/coverage.ts";
+import { indexVault } from "../../../src/core/search/indexer.ts";
+import { search } from "../../../src/core/search/search.ts";
+import { createTempVault, makeConfig, writeMd } from "../../helpers/search-fixtures.ts";
+import type { StructuredRecallQueryDocument } from "../../../src/core/search/structured-query.ts";
+
+describe("planTargetedRetry (pure decision)", () => {
+ const report = (
+ terms: ReadonlyArray<string>,
+ covered: ReadonlyArray<string>,
+ documentCount: number,
+ df: ReadonlyArray<[string, number]>,
+ ) =>
+ buildCoverageReport({
+ significantTerms: terms,
+ coveredTerms: new Set(covered),
+ documentCount,
+ dfByTerm: new Map(df),
+ });
+
+ test("does not fire when coverage is complete", () => {
+ const cov = report(["alpha", "beta"], ["alpha", "beta"], 10, [
+ ["alpha", 3],
+ ["beta", 3],
+ ]);
+ const plan = planTargetedRetry(cov);
+ expect(plan.fire).toBe(false);
+ expect(plan.terms).toEqual([]);
+ });
+
+ test("fires on a partial miss, aimed at the uncovered rare term", () => {
+ const cov = report(["warehouse", "rotation", "chandelier"], ["warehouse", "rotation"], 6, [
+ ["warehouse", 4],
+ ["rotation", 4],
+ ["chandelier", 1],
+ ]);
+ expect(cov.idfWeightedCoverage).toBeLessThan(0.8);
+ const plan = planTargetedRetry(cov);
+ expect(plan.fire).toBe(true);
+ expect(plan.terms).toEqual(["chandelier"]);
+ });
+
+ test("the rare gate: a below-threshold miss on only common terms does not fire", () => {
+ const cov = report(["alpha", "common"], ["alpha"], 10, [
+ ["alpha", 1],
+ ["common", 9],
+ ]);
+ expect(cov.idfWeightedCoverage).toBeLessThan(0.8);
+ expect(cov.uncoveredRareTerms).toEqual([]);
+ expect(planTargetedRetry(cov).fire).toBe(false);
+ });
+
+ test("zero coverage aims at every rare term", () => {
+ const cov = report(["fennel", "warehouse"], [], 4, [
+ ["fennel", 1],
+ ["warehouse", 1],
+ ]);
+ expect(cov.idfWeightedCoverage).toBe(0);
+ expect(planTargetedRetry(cov).terms).toEqual(["fennel", "warehouse"]);
+ });
+});
+
+describe("targeted recall on a partial miss (integration)", () => {
+ let vault: string;
+ let dbPath: string;
+ let cleanup: () => void;
+
+ beforeEach(() => {
+ ({ vault, dbPath, cleanup } = createTempVault("targeted-recall"));
+ });
+ afterEach(() => cleanup());
+
+ // The first-pass keyword lane is restricted to the common terms via a
+ // supplied structured query (lex lane omits the rare term), so the
+ // pool is NON-EMPTY yet misses the rare term `chandelier` - the
+ // partial miss the targeted retry exists for. `chandelier` is still a
+ // significant term of the original query, so coverage flags it.
+ const structured: StructuredRecallQueryDocument = Object.freeze({
+ intent: null,
+ lex: Object.freeze({
+ include: Object.freeze(["warehouse", "rotation"]) as ReadonlyArray<string>,
+ exclude: Object.freeze([]) as ReadonlyArray<string>,
+ }),
+ vec: Object.freeze(["warehouse rotation"]) as ReadonlyArray<string>,
+ hyde: Object.freeze([]) as ReadonlyArray<string>,
+ });
+ const QUERY = "warehouse rotation chandelier";
+
+ async function seed(): Promise<void> {
+ // Four docs make warehouse/rotation common (low IDF); chandelier is
+ // unique (high IDF), present in a doc the common-term lane never
+ // surfaces.
+ writeMd(vault, "Brain/notes/wh1.md", "# WH1\n\nWarehouse rotation schedule, bay seven.\n");
+ writeMd(vault, "Brain/notes/wh2.md", "# WH2\n\nWarehouse rotation plan for spring.\n");
+ writeMd(vault, "Brain/notes/wh3.md", "# WH3\n\nWarehouse rotation audit notes.\n");
+ writeMd(vault, "Brain/notes/wh4.md", "# WH4\n\nWarehouse rotation duties roster.\n");
+ writeMd(
+ vault,
+ "Brain/notes/chandelier.md",
+ "# Chandelier\n\nChandelier handoff log, east hall.\n",
+ );
+ await indexVault(makeConfig({ vault, dbPath, maxHops: 0, mmrLambda: 1 }));
+ }
+
+ test("a partial first pass re-queries the uncovered rare term and recovers it", async () => {
+ await seed();
+ const config = makeConfig({ vault, dbPath, maxHops: 0, mmrLambda: 1 });
+ const outcome = await search(config, {
+ query: QUERY,
+ evidencePack: true,
+ structuredQuery: structured,
+ });
+
+ expect(outcome.secondPass?.triggered).toBe(true);
+ expect(outcome.secondPass?.kind).toBe("targeted");
+ expect(outcome.secondPass?.targetedTerms).toContain("chandelier");
+ expect(outcome.secondPass?.added).toBeGreaterThan(0);
+
+ const paths = outcome.results.map((r) => r.path);
+ expect(paths).toContain("Brain/notes/chandelier.md");
+ // The first-pass common-term docs are still present.
+ expect(paths).toContain("Brain/notes/wh1.md");
+
+ // The recovered record carries the targeted attribution; a first-pass
+ // hit does not.
+ const recovered = outcome.results.find((r) => r.path === "Brain/notes/chandelier.md");
+ expect(recovered?.reasons.some((x) => x.startsWith("second_pass: targeted"))).toBe(true);
+ const firstPass = outcome.results.find((r) => r.path === "Brain/notes/wh1.md");
+ expect(firstPass?.reasons.some((x) => x.startsWith("second_pass:"))).toBe(false);
+
+ // The regenerated pack no longer abstains on the now-covered term.
+ expect(outcome.evidencePack?.uncoveredRareTerms ?? []).not.toContain("chandelier");
+ });
+
+ test("the kill switch disables the targeted retry", async () => {
+ await seed();
+ const off = makeConfig({ vault, dbPath, maxHops: 0, mmrLambda: 1, twoPassEnabled: false });
+ const outcome = await search(off, {
+ query: QUERY,
+ evidencePack: true,
+ structuredQuery: structured,
+ });
+ expect(outcome.secondPass).toBeUndefined();
+ expect(outcome.results.map((r) => r.path)).not.toContain("Brain/notes/chandelier.md");
+ // Conservative on a final miss: the pack still abstains on the rare term.
+ expect(outcome.evidencePack?.uncoveredRareTerms ?? []).toContain("chandelier");
+ });
+
+ test("a plain (non-evidence-pack) search never retries", async () => {
+ await seed();
+ const config = makeConfig({ vault, dbPath, maxHops: 0, mmrLambda: 1 });
+ const outcome = await search(config, { query: QUERY, structuredQuery: structured });
+ expect(outcome.secondPass).toBeUndefined();
+ });
+
+ test("a fully-covered first pass never triggers the retry", async () => {
+ await seed();
+ const config = makeConfig({ vault, dbPath, maxHops: 0, mmrLambda: 1 });
+ // No uncovered rare term: the query is exactly the common-term lane.
+ const outcome = await search(config, {
+ query: "warehouse rotation",
+ evidencePack: true,
+ structuredQuery: structured,
+ });
+ expect(outcome.secondPass).toBeUndefined();
+ expect(outcome.evidencePack?.abstention ?? null).toBeNull();
+ });
+});
diff --git a/tests/core/search/cross-vault.test.ts b/tests/core/search/cross-vault.test.ts
index 84ef56ef..3dc049d3 100644
--- a/tests/core/search/cross-vault.test.ts
+++ b/tests/core/search/cross-vault.test.ts
@@ -93,3 +93,47 @@ test("single-origin union (no profiles, no sources) matches plain search shape",
expect(outcome.results.length).toBeGreaterThan(0);
expect(outcome.results.every((r) => r.origin === "local")).toBe(true);
});
+
+// D4 t_23c1b929 - normalized-confidence chain-stop for cross-vault early termination.
+function withChainStop(score: number): void {
+ // Re-resolve every origin from config.yaml, so the knob reaches the
+ // active origin the cross-vault loop gates on.
+ writeFileSync(
+ configPath,
+ `vault: "${active}"\nsearch_chain_stop_enabled: true\nsearch_chain_stop_score: ${score}\n`,
+ );
+}
+
+test("chain-stop on: a confident active origin skips the remaining origins", async () => {
+ addRecallSource(configPath, active, "team", external);
+ // Threshold 0: any non-empty origin clears it, so the active origin
+ // alone answers and the external origin is never searched.
+ withChainStop(0);
+ const outcome = await searchAcrossVaults(configPath, active, { query: "griffin", limit: 10 });
+ expect(outcome.results.every((r) => r.origin === "local")).toBe(true);
+ expect(outcome.chainStop).toBeDefined();
+ expect(outcome.chainStop?.triggered).toBe(true);
+ expect(outcome.chainStop?.stoppedAfter).toBe("local");
+ expect(outcome.chainStop?.skipped).toEqual(["source/team"]);
+});
+
+test("chain-stop gates on the normalized score, never raw: a sub-threshold top does not stop", async () => {
+ addRecallSource(configPath, active, "team", external);
+ // The active origin's top NORMALIZED score for this fixture is ~0.65,
+ // well under 0.9, so the gate must not fire and every origin is searched.
+ // The raw FTS/BM25 lane score is far higher than 0.9, so this also proves
+ // the gate reads the normalized result score, not the raw lane score.
+ withChainStop(0.9);
+ const outcome = await searchAcrossVaults(configPath, active, { query: "griffin", limit: 10 });
+ const labels = new Set(outcome.results.map((r) => r.origin));
+ expect(labels).toEqual(new Set(["local", "source/team"]));
+ expect(outcome.chainStop).toBeUndefined();
+});
+
+test("chain-stop off (default) runs every origin bit-identically and records no chainStop", async () => {
+ addRecallSource(configPath, active, "team", external);
+ const outcome = await searchAcrossVaults(configPath, active, { query: "griffin", limit: 10 });
+ const labels = new Set(outcome.results.map((r) => r.origin));
+ expect(labels).toEqual(new Set(["local", "source/team"]));
+ expect(outcome.chainStop).toBeUndefined();
+});
diff --git a/tests/core/search/disclosure.test.ts b/tests/core/search/disclosure.test.ts
new file mode 100644
index 00000000..5e47650e
--- /dev/null
+++ b/tests/core/search/disclosure.test.ts
@@ -0,0 +1,179 @@
+/**
+ * Progressive 3-layer disclosure for the general vault search
+ * (search → expand → transcript), the contract session-recall already
+ * ships lifted up to the main recall surface.
+ *
+ * Layer 1 — `search(..., { disclosure: "cards" })` returns compact cards
+ * (path/title/score/reasons/snippet/pointer), never full content.
+ * Layer 2 — `expandHit` returns the fuller note (the document's chunks
+ * reconstructed from the store).
+ * Layer 3 — `expandHit` returns the raw chunk transcript, paginated by a
+ * cursor, exactly like `expandSessionRecall`.
+ *
+ * The default (`disclosure` absent) path stays byte-identical to the
+ * pre-change full-content search, and expand reuses the existing store
+ * read — it never rebuilds or writes the index.
+ */
+
+import { afterEach, beforeEach, expect, test } from "bun:test";
+import { statSync } from "node:fs";
+
+import { expandHit, indexVault, search, SearchError } from "../../../src/core/search/index.ts";
+import { makeConfig, createTempVault, writeMd } from "../../helpers/search-fixtures.ts";
+
+let tmp: ReturnType<typeof createTempVault>;
+
+beforeEach(async () => {
+ tmp = createTempVault("disclosure");
+ // A note long enough to split into several chunks (> the 800-token
+ // budget), so layer-3 raw pagination has more than one page to walk.
+ const body = Array.from(
+ { length: 260 },
+ (_, i) => `Paragraph ${i} discusses the migration plan and rollback strategy in detail.`,
+ ).join("\n\n");
+ writeMd(
+ tmp.vault,
+ "notes/migration.md",
+ `---\ntitle: Migration Plan\n---\n\n# Migration\n\n${body}\n`,
+ );
+ writeMd(
+ tmp.vault,
+ "notes/rollback.md",
+ `---\ntitle: Rollback\n---\n\n# Rollback\n\nThe rollback strategy reverts the migration in one step.\n`,
+ );
+ const config = makeConfig({ vault: tmp.vault, dbPath: tmp.dbPath, maxHops: 0 });
+ await indexVault(config, {});
+});
+
+afterEach(() => {
+ tmp.cleanup();
+});
+
+function config() {
+ return makeConfig({ vault: tmp.vault, dbPath: tmp.dbPath, maxHops: 0 });
+}
+
+test("layer 1: disclosure 'cards' returns compact cards, never full content", async () => {
+ const out = await search(config(), {
+ query: "migration rollback strategy",
+ limit: 10,
+ disclosure: "cards",
+ });
+
+ expect(out.cards).toBeDefined();
+ expect(out.cards!.length).toBeGreaterThan(0);
+ // The cards lane is the payload; the heavy `results` array is empty.
+ expect(out.results.length).toBe(0);
+ expect(out.total).toBe(out.cards!.length);
+
+ for (const card of out.cards!) {
+ expect(typeof card.chunkId).toBe("number");
+ expect(typeof card.documentId).toBe("number");
+ expect(typeof card.path).toBe("string");
+ expect(Array.isArray(card.reasons)).toBe(true);
+ expect(typeof card.snippet).toBe("string");
+ // The pointer is a D2 `path:Lstart-Lend` (or `path:Lstart`) form.
+ expect(card.pointer).toMatch(/:L\d+(-L\d+)?$/);
+ // A card carries a snippet, not the chunk's full content field.
+ expect("content" in card).toBe(false);
+ }
+
+ // The big note's card snippet is strictly shorter than its full chunk
+ // content — token economy is the whole point of the cards lane.
+ const full = await search(config(), { query: "migration rollback strategy", limit: 10 });
+ const bigResult = full.results.find((r) => r.path === "notes/migration.md");
+ const bigCard = out.cards!.find((c) => c.path === "notes/migration.md");
+ expect(bigResult).toBeDefined();
+ expect(bigCard).toBeDefined();
+ expect(bigCard!.snippet.length).toBeLessThan(bigResult!.content.length);
+});
+
+test("layer 2 + 3: expandHit returns the fuller note and the paginated raw transcript", async () => {
+ const out = await search(config(), {
+ query: "migration rollback strategy",
+ limit: 10,
+ disclosure: "cards",
+ });
+ const card = out.cards!.find((c) => c.path === "notes/migration.md");
+ expect(card).toBeDefined();
+
+ // First raw page: one chunk only, so the multi-chunk note still has more.
+ const first = await expandHit(config(), { chunkId: card!.chunkId, rawLimit: 1 });
+
+ // Layer 2 — the fuller note.
+ expect(first.note.path).toBe("notes/migration.md");
+ expect(first.note.title).toBe("Migration Plan");
+ expect(first.note.content.length).toBeGreaterThan(card!.snippet.length);
+ expect(first.note.content).toContain("migration");
+ expect(first.note.pointer).toMatch(/:L\d+(-L\d+)?$/);
+
+ // Layer 3 — the raw chunk transcript, paginated.
+ expect(first.raw_content.length).toBe(1);
+ expect(typeof first.raw_content[0]!.content).toBe("string");
+ expect(first.raw_content[0]!.pointer).toMatch(/:L\d+(-L\d+)?$/);
+ expect(first.next_cursor).not.toBeNull();
+
+ // Walking the cursor yields the next raw chunk.
+ const second = await expandHit(config(), {
+ chunkId: card!.chunkId,
+ rawLimit: 1,
+ cursor: first.next_cursor!,
+ });
+ expect(second.raw_content.length).toBe(1);
+ expect(second.raw_content[0]!.chunkId).not.toBe(first.raw_content[0]!.chunkId);
+});
+
+test("expandHit on an unknown chunk fails with INVALID_INPUT", async () => {
+ await expect(expandHit(config(), { chunkId: 999_999 })).rejects.toThrow(SearchError);
+});
+
+test("bit-identity: default (no disclosure) output is unchanged full-content search", async () => {
+ const implicitFull = await search(config(), { query: "migration rollback strategy", limit: 10 });
+ const explicitFull = await search(config(), {
+ query: "migration rollback strategy",
+ limit: 10,
+ disclosure: "full",
+ });
+
+ // No cards lane on the default path, and the result rows carry full content.
+ expect("cards" in implicitFull).toBe(false);
+ expect(implicitFull.results.length).toBeGreaterThan(0);
+ expect(implicitFull.results[0]!.content.length).toBeGreaterThan(0);
+
+ // Implicit-default and explicit-'full' are the same search. The score's
+ // recency layer folds in Date.now(), which drifts by microseconds
+ // between two live calls, so compare the stable identity/content/order
+ // projection rather than the raw floats.
+ const stable = (o: typeof implicitFull) =>
+ o.results.map((r) => ({
+ path: r.path,
+ chunkId: r.chunkId,
+ documentId: r.documentId,
+ title: r.title,
+ content: r.content,
+ startLine: r.startLine,
+ endLine: r.endLine,
+ searchType: r.searchType,
+ }));
+ expect(stable(explicitFull)).toEqual(stable(implicitFull));
+ expect("cards" in explicitFull).toBe(false);
+});
+
+test("expand reuses the existing store read — no index rebuild, no new index file", async () => {
+ const out = await search(config(), {
+ query: "migration rollback strategy",
+ limit: 10,
+ disclosure: "cards",
+ });
+ const card = out.cards![0]!;
+
+ // The search above built the index; capture its identity, then expand.
+ const before = statSync(tmp.dbPath);
+ const result = await expandHit(config(), { chunkId: card.chunkId });
+ const after = statSync(tmp.dbPath);
+
+ // A rebuild swaps the index file in place (new mtime); a pure read does not.
+ expect(after.mtimeMs).toBe(before.mtimeMs);
+ expect(after.size).toBe(before.size);
+ expect(result.note.documentId).toBe(card.documentId);
+});
diff --git a/tests/core/search/evidence-pack.test.ts b/tests/core/search/evidence-pack.test.ts
index f9d20658..10e3e5f5 100644
--- a/tests/core/search/evidence-pack.test.ts
+++ b/tests/core/search/evidence-pack.test.ts
@@ -69,3 +69,20 @@ test("buildEvidencePack reports terminal downrank only when applied", () => {
expect(pack.records[0]?.droppedCandidateReasons).toContain("terminal_state_downranked");
});
+
+test("buildEvidencePack carries a single-line linePointer for a one-line chunk", () => {
+ const pack = buildEvidencePack("alpha", [result("active.md", "alpha beta")]);
+
+ expect(pack.records[0]?.linePointer).toBe("active.md:L1");
+});
+
+test("buildEvidencePack carries a line-range linePointer from the chunk span", () => {
+ const multiLine: BrainSearchResult = Object.freeze({
+ ...result("notes.md", "alpha"),
+ startLine: 5,
+ endLine: 12,
+ });
+ const pack = buildEvidencePack("alpha", [multiLine]);
+
+ expect(pack.records[0]?.linePointer).toBe("notes.md:L5-L12");
+});
diff --git a/tests/core/search/line-numbering.test.ts b/tests/core/search/line-numbering.test.ts
new file mode 100644
index 00000000..8aab4e34
--- /dev/null
+++ b/tests/core/search/line-numbering.test.ts
@@ -0,0 +1,134 @@
+import { expect, test } from "bun:test";
+
+import {
+ charSpanToLineSpan,
+ extractLineRange,
+ formatLinePointer,
+ parseLinePointer,
+ renderWithLineNumbers,
+} from "../../../src/core/search/line-numbering.ts";
+
+test("renderWithLineNumbers prefixes [N] from line 1 by default", () => {
+ expect(renderWithLineNumbers("a\nb\nc")).toBe("[1] a\n[2] b\n[3] c");
+});
+
+test("renderWithLineNumbers honours a custom start line", () => {
+ expect(renderWithLineNumbers("a\nb", 10)).toBe("[10] a\n[11] b");
+});
+
+test("renderWithLineNumbers preserves blank lines", () => {
+ expect(renderWithLineNumbers("a\n\nc")).toBe("[1] a\n[2] \n[3] c");
+});
+
+test("renderWithLineNumbers clamps a sub-1 start line to 1", () => {
+ expect(renderWithLineNumbers("a", 0)).toBe("[1] a");
+ expect(renderWithLineNumbers("a", -5)).toBe("[1] a");
+});
+
+test("renderWithLineNumbers treats empty text as zero lines", () => {
+ expect(renderWithLineNumbers("")).toBe("");
+});
+
+test("extractLineRange slices an inclusive 1-based range", () => {
+ expect(extractLineRange("a\nb\nc\nd", 2, 3)).toBe("b\nc");
+});
+
+test("extractLineRange returns a single line when start equals end", () => {
+ expect(extractLineRange("a\nb\nc", 2, 2)).toBe("b");
+});
+
+test("extractLineRange clamps an over-long end to the last line", () => {
+ expect(extractLineRange("a\nb", 1, 99)).toBe("a\nb");
+});
+
+test("extractLineRange clamps a sub-1 start to line 1", () => {
+ expect(extractLineRange("a\nb", 0, 1)).toBe("a");
+});
+
+test("extractLineRange returns empty for an inverted range", () => {
+ expect(extractLineRange("a\nb\nc", 3, 1)).toBe("");
+});
+
+test("extractLineRange returns empty when the start is past the last line", () => {
+ expect(extractLineRange("a\nb", 5, 9)).toBe("");
+});
+
+test("extractLineRange returns empty for empty text", () => {
+ expect(extractLineRange("", 1, 1)).toBe("");
+});
+
+test("extractLineRange yields verbatim bytes with no markers (idempotent)", () => {
+ const text = "alpha\nbeta\ngamma\ndelta";
+ const once = extractLineRange(text, 2, 3);
+ expect(once).toBe("beta\ngamma");
+ // Re-extracting the full slice from itself is a no-op on the bytes.
+ expect(extractLineRange(once, 1, 2)).toBe(once);
+});
+
+test("formatLinePointer renders a range pointer", () => {
+ expect(formatLinePointer("Brain/foo.md", 5, 12)).toBe("Brain/foo.md:L5-L12");
+});
+
+test("formatLinePointer collapses a single line", () => {
+ expect(formatLinePointer("a.md", 7, 7)).toBe("a.md:L7");
+});
+
+test("formatLinePointer normalizes an inverted or sub-1 range", () => {
+ expect(formatLinePointer("a.md", 9, 3)).toBe("a.md:L9");
+ expect(formatLinePointer("a.md", 0, 2)).toBe("a.md:L1-L2");
+});
+
+test("parseLinePointer parses a range pointer", () => {
+ expect(parseLinePointer("Brain/foo.md:L5-L12")).toEqual({
+ path: "Brain/foo.md",
+ lineStart: 5,
+ lineEnd: 12,
+ });
+});
+
+test("parseLinePointer parses a single-line pointer", () => {
+ expect(parseLinePointer("a.md:L7")).toEqual({ path: "a.md", lineStart: 7, lineEnd: 7 });
+});
+
+test("parseLinePointer keeps colons that belong to the path", () => {
+ expect(parseLinePointer("a:b:L1-L2")).toEqual({ path: "a:b", lineStart: 1, lineEnd: 2 });
+});
+
+test("parseLinePointer round-trips formatLinePointer", () => {
+ for (const [path, s, e] of [
+ ["Brain/foo.md", 5, 12],
+ ["a.md", 7, 7],
+ ] as const) {
+ const pointer = formatLinePointer(path, s, e);
+ expect(parseLinePointer(pointer)).toEqual({ path, lineStart: s, lineEnd: e });
+ }
+});
+
+test("parseLinePointer rejects malformed input", () => {
+ expect(parseLinePointer("nope")).toBeNull();
+ expect(parseLinePointer("")).toBeNull();
+ expect(parseLinePointer("a.md:L0")).toBeNull();
+ expect(parseLinePointer("a.md:L3-L1")).toBeNull();
+ expect(parseLinePointer(":L1")).toBeNull();
+});
+
+test("charSpanToLineSpan maps an in-line match to its line", () => {
+ const text = "a\nbb\nccc";
+ expect(charSpanToLineSpan(text, text.indexOf("bb"), 2)).toEqual({ lineStart: 2, lineEnd: 2 });
+});
+
+test("charSpanToLineSpan spans multiple lines", () => {
+ const text = "a\nbb\nccc";
+ expect(charSpanToLineSpan(text, 0, 5)).toEqual({ lineStart: 1, lineEnd: 2 });
+});
+
+test("charSpanToLineSpan defaults length to a zero-width point", () => {
+ const text = "a\nbb\nccc";
+ expect(charSpanToLineSpan(text, text.indexOf("ccc"))).toEqual({ lineStart: 3, lineEnd: 3 });
+});
+
+test("charSpanToLineSpan clamps out-of-range indices", () => {
+ const text = "a\nbb\nccc";
+ expect(charSpanToLineSpan(text, -10, 0)).toEqual({ lineStart: 1, lineEnd: 1 });
+ expect(charSpanToLineSpan(text, 999, 5)).toEqual({ lineStart: 3, lineEnd: 3 });
+});
diff --git a/tests/core/search/store.test.ts b/tests/core/search/store.test.ts
index 0565dbb4..2697f317 100644
--- a/tests/core/search/store.test.ts
+++ b/tests/core/search/store.test.ts
@@ -33,6 +33,7 @@ function makeConfig(overrides?: Partial<ResolvedSearchConfig>): ResolvedSearchCo
ignoreRules: Object.freeze([{ raw: ".git", kind: "name" as const }]),
chunkSize: 800,
chunkOverlap: 100,
+ chunkMinSize: 100,
keywordWeight: 0.6,
semanticWeight: 0.4,
fusionMode: "linear" as const,
@@ -57,6 +58,8 @@ function makeConfig(overrides?: Partial<ResolvedSearchConfig>): ResolvedSearchCo
twoPassEnabled: true,
poolMultiplier: 3,
selfTuningEnabled: false,
+ chainStopEnabled: false,
+ chainStopScore: 0.8,
}),
shutdownGraceMs: 5_000,
resumeReindex: false,
diff --git a/tests/core/search/store.vec.test.ts b/tests/core/search/store.vec.test.ts
index 79dfbeea..489ff638 100644
--- a/tests/core/search/store.vec.test.ts
+++ b/tests/core/search/store.vec.test.ts
@@ -36,6 +36,7 @@ function semanticConfig(
ignoreRules: Object.freeze([{ raw: ".git", kind: "name" as const }]),
chunkSize: 800,
chunkOverlap: 100,
+ chunkMinSize: 100,
keywordWeight: 0.6,
semanticWeight: 0.4,
fusionMode: "linear" as const,
@@ -60,6 +61,8 @@ function semanticConfig(
twoPassEnabled: true,
poolMultiplier: 3,
selfTuningEnabled: false,
+ chainStopEnabled: false,
+ chainStopScore: 0.8,
}),
shutdownGraceMs: 5_000,
resumeReindex: false,
diff --git a/tests/helpers/search-fixtures.ts b/tests/helpers/search-fixtures.ts
index b4adad9c..e6691fc5 100644
--- a/tests/helpers/search-fixtures.ts
+++ b/tests/helpers/search-fixtures.ts
@@ -102,6 +102,7 @@ export function makeConfig(opts: {
ignoreRules,
chunkSize: 800,
chunkOverlap: 100,
+ chunkMinSize: 100,
keywordWeight: 0.6,
semanticWeight: 0.4,
fusionMode: opts.fusionMode ?? "linear",
@@ -126,6 +127,8 @@ export function makeConfig(opts: {
twoPassEnabled: opts.twoPassEnabled ?? true,
poolMultiplier: 3,
selfTuningEnabled: opts.selfTuningEnabled ?? false,
+ chainStopEnabled: false,
+ chainStopScore: 0.8,
}),
shutdownGraceMs: 5_000,
resumeReindex: false,
diff --git a/tests/mcp/brain-tools-parity.test.ts b/tests/mcp/brain-tools-parity.test.ts
index 771a1525..6833d117 100644
--- a/tests/mcp/brain-tools-parity.test.ts
+++ b/tests/mcp/brain-tools-parity.test.ts
@@ -20,7 +20,9 @@
* with a cadence-driven next-due date); the memory-subsystem-alignment
* release added `brain_memory_bridge` (the Hermes on_memory_write host
* bridge — persists native built-in-memory writes as durable
- * host_memory_write continuity records).
+ * host_memory_write continuity records); the dashboard-context-trace
+ * release added `brain_event_trace` (joins logged Brain events to the
+ * continuity records attached to them by shared correlation ids).
*/
import { describe, expect, test } from "bun:test";
@@ -52,6 +54,7 @@ const FROZEN_BRAIN_TOOL_NAMES = [
"brain_doctor",
"brain_dream",
"brain_entity",
+ "brain_event_trace",
"brain_feedback",
"brain_foresight",
"brain_generation_reports",
diff --git a/tests/mcp/event-trace-tool.test.ts b/tests/mcp/event-trace-tool.test.ts
new file mode 100644
index 00000000..33374213
--- /dev/null
+++ b/tests/mcp/event-trace-tool.test.ts
@@ -0,0 +1,124 @@
+import { afterEach, beforeEach, describe, expect, test } from "bun:test";
+import { mkdirSync, mkdtempSync, rmSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { appendLogEvent } from "../../src/core/brain/log.ts";
+import { appendContinuityRecord } from "../../src/core/brain/continuity/store.ts";
+import { BRAIN_LOG_EVENT_KIND } from "../../src/core/brain/types.ts";
+import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
+import { JSONRPC_VERSION, MCPServer, PROTOCOL_VERSION } from "../../src/mcp/index.ts";
+import { buildToolTable } from "../../src/mcp/tools.ts";
+
+let tmp: string;
+let vault: string;
+let configHome: string;
+let configPath: string;
+const savedEnv: Record<string, string | undefined> = {};
+
+beforeEach(() => {
+ tmp = mkdtempSync(join(tmpdir(), "o2b-mcp-event-trace-"));
+ vault = join(tmp, "vault");
+ mkdirSync(join(vault, "Brain", "log"), { recursive: true });
+ configHome = mkdtempSync(join(tmpdir(), "o2b-mcp-event-trace-cfg-"));
+ configPath = join(configHome, "config.yaml");
+ for (const key of [
+ "VAULT_AGENT_NAME",
+ "VAULT_TIMEZONE",
+ "VAULT_DIR",
+ "OPEN_SECOND_BRAIN_CONFIG",
+ ]) {
+ savedEnv[key] = process.env[key];
+ delete process.env[key];
+ }
+ process.env["OPEN_SECOND_BRAIN_CONFIG"] = configPath;
+ atomicWriteFileSync(configPath, `vault: ${vault}\nagent_name: claude\n`);
+});
+
+afterEach(() => {
+ rmSync(tmp, { recursive: true, force: true });
+ rmSync(configHome, { recursive: true, force: true });
+ for (const [key, value] of Object.entries(savedEnv)) {
+ if (value === undefined) delete process.env[key];
+ else process.env[key] = value;
+ }
+});
+
+async function initialize(server: MCPServer): Promise<void> {
+ await server.handleRequest({
+ jsonrpc: JSONRPC_VERSION,
+ id: 1,
+ method: "initialize",
+ params: {
+ protocolVersion: PROTOCOL_VERSION,
+ capabilities: {},
+ clientInfo: { name: "event-trace-test", version: "0" },
+ },
+ });
+ await server.handleRequest({ jsonrpc: JSONRPC_VERSION, method: "notifications/initialized" });
+}
+
+async function callEventTrace(
+ server: MCPServer,
+ args: Record<string, unknown>,
+): Promise<Record<string, unknown>> {
+ const response = (await server.handleRequest({
+ jsonrpc: JSONRPC_VERSION,
+ id: 9,
+ method: "tools/call",
+ params: { name: "brain_event_trace", arguments: args },
+ })) as { result: { content: ReadonlyArray<{ type: string; text: string }> } };
+ return JSON.parse(response.result.content[0]!.text);
+}
+
+describe("brain_event_trace tool registration", () => {
+ test("registered in the full tool table", () => {
+ const tools = buildToolTable("full");
+ expect(tools.find((tool) => tool.name === "brain_event_trace")).toBeDefined();
+ });
+
+ test("not registered in the writer-only tool table", () => {
+ const tools = buildToolTable("writer");
+ expect(tools.find((tool) => tool.name === "brain_event_trace")).toBeUndefined();
+ });
+});
+
+describe("brain_event_trace tool", () => {
+ test("joins continuity records to a logged event by session id", async () => {
+ appendLogEvent(vault, {
+ timestamp: "2026-06-15T10:00:00Z",
+ eventType: BRAIN_LOG_EVENT_KIND.writeSession,
+ agent: "tester",
+ body: { session_id: "sess-1", kind: "note", status: "committed" },
+ });
+ const want = appendContinuityRecord(vault, {
+ kind: "recall_telemetry",
+ createdAt: "2026-06-15T10:00:05Z",
+ sourceRefs: [],
+ payload: { session_id: "sess-1", mode: "context_pack", status: "ok" },
+ }).id;
+
+ const server = new MCPServer({ vault, configPath });
+ await initialize(server);
+
+ const result = await callEventTrace(server, { date: "2026-06-15" });
+ expect(result["total"]).toBe(1);
+ expect(result["trace_total"]).toBe(1);
+ const events = result["events"] as Array<Record<string, unknown>>;
+ const traces = events[0]!["traces"] as Array<Record<string, unknown>>;
+ expect(traces[0]!["id"]).toBe(want);
+ expect(traces[0]!["joinedBy"]).toEqual(["session"]);
+ });
+
+ test("rejects an unknown event kind", async () => {
+ const server = new MCPServer({ vault, configPath });
+ await initialize(server);
+ const response = (await server.handleRequest({
+ jsonrpc: JSONRPC_VERSION,
+ id: 9,
+ method: "tools/call",
+ params: { name: "brain_event_trace", arguments: { kind: "not-a-kind" } },
+ })) as { error?: { code: number } };
+ expect(response.error?.code).toBe(-32602);
+ });
+});
diff --git a/tests/mcp/mcp.test.ts b/tests/mcp/mcp.test.ts
index 0372667b..b429cda2 100644
--- a/tests/mcp/mcp.test.ts
+++ b/tests/mcp/mcp.test.ts
@@ -218,6 +218,7 @@ describe("tool listing", () => {
"brain_pre_compress_pack",
// Context continuity and receipts (added in v0.29.0).
"brain_context_receipts",
+ "brain_event_trace",
"brain_recall_telemetry",
"brain_context_presets",
"brain_pre_compact_extract",
@@ -234,10 +235,12 @@ describe("tool listing", () => {
"brain_brief",
"brain_analytics",
"brain_anticipatory_context",
- // Search (added in v0.10.0; recall gate added in v0.27.0).
+ // Search (added in v0.10.0; recall gate added in v0.27.0;
+ // brain_search_expand added with progressive disclosure).
"brain_recall_feedback",
"brain_recall_gate",
"brain_search",
+ "brain_search_expand",
"brain_eval",
"brain_file_context",
// Session Knowledge Synthesis Suite (v1.11.0).
@@ -559,7 +562,9 @@ describe("stdio loop", () => {
// + brain_obligation / brain_agenda (Calendar integration) = 85.
// + brain_memory_bridge (Hermes on_memory_write host bridge,
// memory-subsystem-alignment) = 86.
- expect(list.result.tools.length).toBe(86);
+ // + brain_event_trace (dashboard-context-trace: event→trace join) = 87.
+ // + brain_search_expand (progressive disclosure: search→expand→transcript) = 88.
+ expect(list.result.tools.length).toBe(88);
});
test("returns parse error for invalid JSON", async () => {
diff --git a/tests/mcp/removed-tools.test.ts b/tests/mcp/removed-tools.test.ts
index dcbba23b..a4767b8d 100644
--- a/tests/mcp/removed-tools.test.ts
+++ b/tests/mcp/removed-tools.test.ts
@@ -146,5 +146,7 @@ test("the shadow surface is gone: no hidden tools, removed names unlisted", asyn
// + brain_generation_reports (Hindsight brain-loop ops) = 83.
// + brain_obligation, brain_agenda (Calendar integration) = 85.
// + brain_memory_bridge (Hermes on_memory_write host bridge) = 86.
- expect(list.result.tools.length).toBe(86);
+ // + brain_event_trace (dashboard-context-trace: event→trace join) = 87.
+ // + brain_search_expand (progressive disclosure: search→expand→transcript) = 88.
+ expect(list.result.tools.length).toBe(88);
});
diff --git a/tests/mcp/search-relations.test.ts b/tests/mcp/search-relations.test.ts
index f396bb4f..bbbb028c 100644
--- a/tests/mcp/search-relations.test.ts
+++ b/tests/mcp/search-relations.test.ts
@@ -12,11 +12,39 @@ import { join } from "node:path";
import { SEARCH_TOOLS } from "../../src/mcp/search-tools.ts";
import { indexVault, resolveSearchConfig } from "../../src/core/search/index.ts";
import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
+import { JSONRPC_VERSION, MCPServer, PROTOCOL_VERSION } from "../../src/mcp/index.ts";
let vault: string;
let configHome: string;
let ctx: { vault: string; configPath: string };
+async function initialize(server: MCPServer): Promise<void> {
+ await server.handleRequest({
+ jsonrpc: JSONRPC_VERSION,
+ id: 1,
+ method: "initialize",
+ params: {
+ protocolVersion: PROTOCOL_VERSION,
+ capabilities: {},
+ clientInfo: { name: "search-relations-test", version: "0" },
+ },
+ });
+ await server.handleRequest({ jsonrpc: JSONRPC_VERSION, method: "notifications/initialized" });
+}
+
+async function callTool(
+ server: MCPServer,
+ name: string,
+ args: Record<string, unknown>,
+): Promise<Record<string, any>> {
+ return (await server.handleRequest({
+ jsonrpc: JSONRPC_VERSION,
+ id: 2,
+ method: "tools/call",
+ params: { name, arguments: args },
+ })) as Record<string, any>;
+}
+
beforeEach(async () => {
vault = mkdtempSync(join(tmpdir(), "o2b-rel-vault-"));
configHome = mkdtempSync(join(tmpdir(), "o2b-rel-cfg-"));
@@ -76,4 +104,29 @@ describe("brain_search typed relations", () => {
expect(hit).toBeDefined();
expect(hit!.relations).toBeUndefined();
});
+
+ test("cards and expand output contract accepts exhausted cursor", async () => {
+ writeFileSync(join(vault, "notes", "untitled.md"), "Unique exhausted cursor search term.\n");
+ const config = resolveSearchConfig({ vault, configPath: ctx.configPath });
+ await indexVault(config, {});
+
+ const server = new MCPServer({ vault, configPath: ctx.configPath });
+ await initialize(server);
+ const searchResponse = await callTool(server, "brain_search", {
+ query: "exhausted cursor",
+ disclosure: "cards",
+ limit: 1,
+ });
+ expect(searchResponse.result?.isError).toBe(false);
+ const card = searchResponse.result.structuredContent.cards[0];
+ expect(card.title).toBe("untitled");
+
+ const expandResponse = await callTool(server, "brain_search_expand", {
+ chunk_id: card.chunk_id,
+ raw_limit: 50,
+ });
+ expect(expandResponse.result?.isError).toBe(false);
+ expect(expandResponse.result.structuredContent.note.title).toBe("untitled");
+ expect(expandResponse.result.structuredContent.next_cursor).toBeNull();
+ });
});