Skip to content

Local-first 🧠 memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly 😴 dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.

TypeScript

feat(search): ship recall control and trust surfaces (#54)

Sergey Eroshenkov794ee450e5f2
diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json
index a5c9971c..aa4871ec 100644
--- a/.claude-plugin/plugin.json
+++ b/.claude-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "0.26.0",
+ "version": "0.27.0",
"description": "Plugin-first second brain package for AI agents and humans.",
"author": {
"name": "Open Second Brain contributors"
diff --git a/.codex-plugin/plugin.json b/.codex-plugin/plugin.json
index a1344476..14c3dc65 100644
--- a/.codex-plugin/plugin.json
+++ b/.codex-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "0.26.0",
+ "version": "0.27.0",
"description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.",
"skills": "./skills",
"hooks": "./hooks/hooks.json",
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 7478cc1c..385e24a1 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,6 +5,46 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+## [0.27.0] - 2026-05-31
+
+Recall control and trust surfaces. Search can now expose the evidence behind a
+retrieval decision, agents get explicit gates before automatic recall, and
+context packs can separate directives from constraints and softer context.
+
+### Added
+
+- FTS5 safety hardening for malformed/operator-only lexical queries and external-content
+ drift. Search now repairs a desynchronised FTS table, retries safely, and returns warnings
+ instead of letting recall fail opaquely.
+- Structured recall query documents for CLI and MCP search. `--query-doc` and
+ `query_document` accept line-oriented `intent:`, `lex:`, `vec:`, and `hyde:` lanes,
+ including quoted lexical phrases and `-excluded` terms.
+- Session-scoped search focus through `o2b search focus set|status|clear` and explicit
+ MCP `focus_query` / `focus_path_prefix` inputs. Focus nudges ranking toward the active
+ task window without hiding unfocused matches.
+- MCP `brain_recall_gate`, a read-only classifier that tells an agent whether an
+ automatic recall attempt should run and why it was allowed or skipped.
+- Polarity-aware context lanes for `o2b brain context-pack --lanes` and
+ `brain_context_pack` with `lanes: true`. The legacy flat `items` list remains present
+ while `directives`, `constraints`, and `consider` lanes give host agents safer prompts.
+- Verified evidence packs for search via `--evidence-pack` and MCP `evidence_pack`.
+ Evidence packs report significant, matched, and missing terms; support coverage;
+ abstention text for unsupported terms; per-record `why_retrieved`; and terminal-state
+ downrank reasons for retired/superseded support.
+
+### Changed
+
+- Evidence-pack mode is opt-in and part of the search cache key, so legacy search output
+ and cache rows remain stable unless callers request the diagnostic payload.
+- `brain_search` and CLI JSON search results include per-result `why_retrieved` when an
+ evidence pack is requested.
+- MCP full-server tool count is now 58 after adding `brain_recall_gate`.
+
+### Notes
+
+- `bun run lint` remains warning-only on the existing repository baseline.
+- Files touched by this release were formatted with targeted `oxfmt --write` runs.
+
## [0.26.0] - 2026-05-30
CJK search, schema administration, real-time lifecycle capture, and safe
@@ -2227,7 +2267,7 @@ with `source_type: inline`, the source-file wikilink in `source`,
and a `dedup_hash`over the normalised payload. After capture
the source line is annotated`@osb✓ [[sig-...]]`(inline form)
or the info-string flips to`osb-checked`with a`<!-- @osb✓
- [[sig-...]] -->`comment line (block form), making re-runs
+ [[sig-...]] -->`comment line (block form), making re-runs
idempotent. Default ignore set covers`Brain/`, `.git`,
`node_modules`, `.obsidian`, `.trash`, `.stversions`,
`.open-second-brain`; additional excludes via `--exclude`,
@@ -3573,6 +3613,7 @@ plugin config (vault field)`, and exits with a clear
- Sandbox vault and plugin manifest fixtures for tests.
- GitHub release workflow for tag-based and manually dispatched releases.
+[0.27.0]: https://github.com/itechmeat/open-second-brain/compare/v0.26.0...v0.27.0
[0.26.0]: https://github.com/itechmeat/open-second-brain/compare/v0.25.0...v0.26.0
[0.25.0]: https://github.com/itechmeat/open-second-brain/compare/v0.24.0...v0.25.0
[0.24.0]: https://github.com/itechmeat/open-second-brain/compare/v0.23.0...v0.24.0
diff --git a/README.md b/README.md
index 13c89fc8..abcf1b2a 100644
--- a/README.md
+++ b/README.md
@@ -99,22 +99,22 @@ Three repeat signals on the same topic graduate to a confirmed rule. Evidence sh
The capabilities you actually feel day to day:
-| # | Feature | What it means for you |
-| --- | ------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| 1 | Your memory, in your Obsidian vault | Every rule the agent learns about you is a Markdown file you can open, read, and edit. Wikilinks, backlinks, and the graph view in Obsidian all work — no separate UI to learn. |
-| 2 | `dream` - the agent learns what you actually want | The nightly `dream` pass turns repeat corrections into rules: three signals on the same topic graduate to a confirmed preference. Its confidence rises every time the agent applies the rule and drops when it slips. No LLM inside the algorithm - counters and atomic file moves, nothing else. |
-| 3 | One brain, every agent | Hermes Agent, Claude Code, Codex, Cursor, Aider, opencode, kiro, Copilot CLI, Gemini CLI, Pi - they all read the same memory. Teach one of them a rule and the next one already knows; `brain_agent_query` and `brain_agent_diff` let you inspect which agent contributed what. |
-| 4 | Teach a rule in one line | Drop `@osb feedback negative topic=... principle="..."` into any note. Next time `o2b brain scan-inline` runs (or you call it manually) the marker becomes a real taste signal in `Brain/inbox/` and the note gets a `@osb✓` checkmark so re-runs skip it. |
-| 5 | Look back at how your AI grew | The time axis surfaces how the agent's view of you evolved. `o2b brain evolution` walks a single preference's life from first signal to confirmed to retired; `o2b brain daily` / `weekly` / `monthly` summarise what changed; `o2b brain intent-review` previews whether signal clusters are ready before the next `dream`; `o2b brain retention` recommends keep/improve/park/prune actions without deleting anything. |
-| 6 | Browse and govern your memory schema | `o2b brain explorer` opens a force-directed HTML graph of every rule, every link, every retirement. Preference files can declare typed relationships such as `depends_on:` and `refines:`, plus `memory_layer` / `memory_branch` labels and optional `schema_type:` taxonomy metadata; `o2b brain schema` stats, lint, graph, explain, orphans, and apply audit and mutate those tokens through locked, audited writes. Double-click a node — Obsidian opens the underlying note. |
-| 7 | Undo the agent if it gets it wrong | Every Brain mutation lands a verified snapshot before touching anything. One command (`o2b brain rollback <id>`) restores yesterday's state, and drift detection refuses to clobber unintended local edits. |
-| 8 | Pin, merge, retire by hand | When a rule is wrong, you fix it. `o2b brain pin` keeps a good one safe from auto-retire, `merge` folds duplicates, `reject` puts a bad rule in the bin with a reason. The CLI is the operator's seat. |
-| 9 | Hybrid search that explains itself | `o2b search "<query>" --property type=decision --property status=open` — SQLite + FTS5 and an optional semantic layer, with CJK-aware token expansion for unspaced Chinese/Japanese/Korean text. Recall is sharpened by MMR diversity, link-graph traversal, entity boosting, and header-anchored chunking; every result carries a `why_retrieved` breakdown of the layers that ranked it. |
-| 10 | Pay Memory — audit every paid action | When the agent spends money (Solana-Foundation `pay`, third-party APIs) every receipt lands in `Brain/payments/<date>/`. Spending policy, approval gate, daily Telegram report. The agent never holds wallet keys. |
+| # | Feature | What it means for you |
+| --- | ------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| 1 | Your memory, in your Obsidian vault | Every rule the agent learns about you is a Markdown file you can open, read, and edit. Wikilinks, backlinks, and the graph view in Obsidian all work — no separate UI to learn. |
+| 2 | `dream` - the agent learns what you actually want | The nightly `dream` pass turns repeat corrections into rules: three signals on the same topic graduate to a confirmed preference. Its confidence rises every time the agent applies the rule and drops when it slips. No LLM inside the algorithm - counters and atomic file moves, nothing else. |
+| 3 | One brain, every agent | Hermes Agent, Claude Code, Codex, Cursor, Aider, opencode, kiro, Copilot CLI, Gemini CLI, Pi - they all read the same memory. Teach one of them a rule and the next one already knows; `brain_agent_query` and `brain_agent_diff` let you inspect which agent contributed what. |
+| 4 | Teach a rule in one line | Drop `@osb feedback negative topic=... principle="..."` into any note. Next time `o2b brain scan-inline` runs (or you call it manually) the marker becomes a real taste signal in `Brain/inbox/` and the note gets a `@osb✓` checkmark so re-runs skip it. |
+| 5 | Look back at how your AI grew | The time axis surfaces how the agent's view of you evolved. `o2b brain evolution` walks a single preference's life from first signal to confirmed to retired; `o2b brain daily` / `weekly` / `monthly` summarise what changed; `o2b brain intent-review` previews whether signal clusters are ready before the next `dream`; `o2b brain retention` recommends keep/improve/park/prune actions without deleting anything. |
+| 6 | Browse and govern your memory schema | `o2b brain explorer` opens a force-directed HTML graph of every rule, every link, every retirement. Preference files can declare typed relationships such as `depends_on:` and `refines:`, plus `memory_layer` / `memory_branch` labels and optional `schema_type:` taxonomy metadata; `o2b brain schema` stats, lint, graph, explain, orphans, and apply audit and mutate those tokens through locked, audited writes. Double-click a node — Obsidian opens the underlying note. |
+| 7 | Undo the agent if it gets it wrong | Every Brain mutation lands a verified snapshot before touching anything. One command (`o2b brain rollback <id>`) restores yesterday's state, and drift detection refuses to clobber unintended local edits. |
+| 8 | Pin, merge, retire by hand | When a rule is wrong, you fix it. `o2b brain pin` keeps a good one safe from auto-retire, `merge` folds duplicates, `reject` puts a bad rule in the bin with a reason. The CLI is the operator's seat. |
+| 9 | Hybrid search that explains itself | `o2b search "<query>" --property type=decision --property status=open --evidence-pack` — SQLite + FTS5 and an optional semantic layer, with CJK-aware token expansion for unspaced Chinese/Japanese/Korean text. Recall is sharpened by MMR diversity, link-graph traversal, entity boosting, structured query lanes, session focus, and header-anchored chunking; opt-in evidence packs show matched/missing terms, abstention text, terminal-state downranks, and `why_retrieved` reasons. |
+| 10 | Pay Memory — audit every paid action | When the agent spends money (Solana-Foundation `pay`, third-party APIs) every receipt lands in `Brain/payments/<date>/`. Spending policy, approval gate, daily Telegram report. The agent never holds wallet keys. |
These are the headline capabilities. The full surface also includes: importing Claude Code memory directories, daily logging-discipline cron with a complexity-to-thinking warning, cross-project pointers for shared vaults, transient pinned session context through `brain_pinned_context` / `Brain/pinned.md`, source-agent query and comparison (`brain_agent_query`, `brain_agent_diff`, `o2b brain agent-query`, `o2b brain agent-diff`), the codegraph partner skill, vault hygiene lints, per-MOC coverage audit, concept synthesis, an operator dashboard, configurable Markdown-link presentation output, MCP output-contract checks for core agent-facing tools, private-region stripping with `<private>...</private>`, the v0.12.0 Brain Integrity Suite (content-hash drift detection, durable workrun checkpoints, destructive-from-confirmed retire gate, and the `brain_review_candidates` MCP tool that previews what the next dream pass would do without writing anything), the v0.13.0 Hybrid Search and Recall Quality suite (explainable recall, MMR diversity, link-graph traversal, entity-boosted retrieval, header-anchored chunking), the v0.14.0 Semantic Brain Health and Self-Maintenance suite (cross-preference contradiction detection, concept-gap and stale-claim surfacing, per-preference edit-history, a clean/watch/investigate reconciliation verdict via `brain_health`, and dependency-ordered `doctor --remediate`), the v0.17.0 Brain Lifecycle Review Suite (`intent-review`, `retention`, `monthly`, lifecycle schemas, and discipline complexity ratio), and the v0.18.0 MCP context economy (a per-tool preview budget that returns a bounded inline preview and parks the full payload in a vault-local artifact the agent fetches on demand with `brain_artifact_get`, plus a `recall_hint` one-liner on search results), and the v0.19.0 typed graph semantics (typed frontmatter relationships - `related` / `extends` / `contradicts` / `superseded_by` - that surface inline on search results, a `visibility:` scoping axis for `brain_search`, and a `brain_mcp_landscape` view of the Model Context Protocol servers configured across the vault), and the v0.20.0 Recall and Ranking Quality suite (a configurable Weibull recency-decay curve, structural query-intent classification that re-weights ranking by query shape, language-agnostic synonym expansion via local co-occurrence, per-memory and total recall character budgets for `brain_context_pack`, a persistent query cache that self-invalidates on a corpus-generation change, and a `brain_pre_compress_pack` tool that returns a budgeted top-preferences addendum for a host runtime to inject before context compression), and the v0.21.0 Brain Lifecycle Suite (a per-preference mutation audit trail keyed by preference id - create / promote / update / retire / merge with revision + content-hash before/after, via `o2b brain audit` and `brain_audit`; a multi-phase `dream` pipeline that names its close / reconcile / synthesize / heal / log seams and reports a structured per-phase summary without rewriting the proven internals; deterministic reconcile-phase domain classification that buckets contradictions into claims / entity / decisions / source-freshness, auto-resolves only the decisive source-freshness case, and surfaces the rest as open questions instead of forcing a merge; a budgeted `o2b brain morning-brief` / `brain_morning_brief` session-start summary of top preferences, recent open questions, and recent notes; language-agnostic ISO-8601 temporal extraction that fills preference validity windows from signal text; and opt-in heal-phase vault enrichment that links exact title mentions across user pages), and the v0.22.0 Vault portability + session economy suite (a deterministic lossless session codec - `expand(compress(x)) === x` for all input, opt-in on the signal store and exposed as `o2b brain codec`; an `o2b brain sources` / `brain_sources` read-only dashboard of signals by agent and source type; vault-map `{{role}}` tokens resolved via an optional `Brain/_vault-map.yaml` and wired into scan-inline read paths and graph import; named multi-vault profiles with `o2b vault profile` + a `brain_switch_vault` tool, activated by a pointer in config; and `o2b brain graph-export` / `graph-import` for moving a vault's knowledge graph between layouts with skip/overwrite/merge conflict modes). Browse [`docs/cli-reference.md`](docs/cli-reference.md) for every verb and [`docs/how-it-works.md`](docs/how-it-works.md) for the mental model.
-The v0.26.0 surface adds audited schema mutation/admin tools, real-time lifecycle capture hooks for prompt markers and `brain_feedback` tool calls, CJK-aware FTS recall, and `brain_watchdog` / `o2b brain watchdog` recovery probes.
+The v0.26.0 surface adds audited schema mutation/admin tools, real-time lifecycle capture hooks for prompt markers and `brain_feedback` tool calls, CJK-aware FTS recall, and `brain_watchdog` / `o2b brain watchdog` recovery probes. The v0.27.0 surface adds recall-control and trust tools: structured search query documents, session focus, MCP `brain_recall_gate`, polarity-aware context lanes, and opt-in evidence packs for CLI/MCP search.
## CLI
@@ -137,7 +137,8 @@ o2b brain semantics-backfill --json # Dry-run typed preference-edge repair
o2b vault profile switch work # Activate a named multi-vault profile
o2b brain agent-query --agent claude --json # Source-agent provenance query
o2b brain agent-diff --mode diff --agent claude --agent codex
-o2b search "<query>" # FTS5 over the vault
+o2b search "<query>" --evidence-pack # FTS5/hybrid search with matched/missing evidence
+o2b search focus set --query "current task" --path Notes/Project
o2b help --json # Machine-readable command/flag manifest
o2b completions --shell zsh # Completion script for bash/zsh/fish/elvish/nushell/powershell
o2b mcp --probe --json --disable-tool second_brain_query # Capability-aware MCP probe
diff --git a/docs/brainstorm/recall-control-trust-suite/cli-output/claude.md b/docs/brainstorm/recall-control-trust-suite/cli-output/claude.md
new file mode 100644
index 00000000..b582a7f8
--- /dev/null
+++ b/docs/brainstorm/recall-control-trust-suite/cli-output/claude.md
@@ -0,0 +1,39 @@
+### Variant 1: Unified recall pipeline orchestrator
+
+- **Approach**: Introduce an explicit, ordered `RecallPipeline` in the search core that threads a single mutable `RecallContext` (raw input, parsed query document, gate decision, heal status, per-lane candidates, verification verdicts, polarity lanes) through typed stages: parse -> surfacing-gate -> FTS-self-heal -> lane-route -> fuse -> multi-record-verify -> polarity-lane. Each of the six tasks becomes one stage with a declared input/output contract, and `search.ts` is refactored to drive the pipeline instead of calling keyword/semantic/rank/MMR directly.
+- **Trade-offs**:
+ - Pro: one coherent mental model; every feature has an obvious home and ordering is explicit and testable end-to-end.
+ - Pro: `why_retrieved` provenance flows naturally as accumulated context across stages.
+ - Con: forces a large refactor of the central, backward-compatible `search.ts`, putting the plain-string path at risk.
+ - Con: couples six otherwise-independent features into one abstraction - a bug or churn in the pipeline contract touches all of them.
+ - Con: over-engineered for features like session-focus and the retrieval gate that don't naturally belong on the same path.
+- **Complexity**: large
+- **Risk**: high
+
+### Variant 2: Surgical pure helpers around existing search
+
+- **Approach**: Keep `search.ts`, `ranker.ts`, and `fts.ts` structurally intact and add each task as a focused, deterministic helper module (query-document parser, FTS-heal wrapper, surfacing gate, session-focus store, polarity classifier, multi-record verifier), each with its own focused test suite. Wire each helper in at the minimal existing call site - the parser in front of `brain_search`/`o2b search` argument handling, the heal wrapper inside `keywordTopK`, the polarity/verify layers at result-assembly - gated behind new explicit options so the plain-string path is untouched by default.
+- **Trade-offs**:
+ - Pro: directly honors the conventions (pure helpers, focused tests) and the backward-compat / bounded-self-heal constraints.
+ - Pro: features ship and are reviewable independently; one feature's risk is contained to its call site.
+ - Pro: lowest blast radius on the hot read path.
+ - Con: no single place describes the overall recall flow; cross-feature ordering (gate before heal before fuse before lanes) lives implicitly in call-site sequencing.
+ - Con: `why_retrieved` provenance must be plumbed through several seams rather than accumulated in one context object.
+- **Complexity**: medium
+- **Risk**: low
+
+### Variant 3: Two-tier request / trust split
+
+- **Approach**: Organize the six tasks into two cohesive subsystems sharing one extended `SearchOptions`/`SearchOutcome` contract: a **request tier** (structured query document, retrieval gate, session focus - "what to ask and whether to ask") and a **trust tier** (FTS self-heal, multi-record verification, polarity lanes - "make results reliable and well-presented"). The request tier produces a normalized intent/lane spec consumed by the existing search, and the trust tier wraps and post-processes its output; `search.ts` gains two thin seams rather than a full rewrite.
+- **Trade-offs**:
+ - Pro: meaningful conceptual grouping that aids review and documentation without a monolithic pipeline.
+ - Pro: contained changes to `search.ts` (two seams) while still giving provenance a shared contract to ride on.
+ - Pro: request-tier features can short-circuit (gate skips, focus-only) before touching retrieval.
+ - Con: the two-tier boundary is somewhat artificial - session-focus spans both asking and ranking, blurring the split.
+ - Con: more upfront contract design than Variant 2 for marginal organizational gain on an already-large PR.
+- **Complexity**: medium
+- **Risk**: medium
+
+### Recommended: Variant 2
+
+**Rationale**: The project conventions explicitly call for pure parser/ranker helpers with focused tests, backward-compatible read paths gated behind explicit options, and bounded self-healing - all of which Variant 2 satisfies directly while keeping the hot `search.ts` path stable across a six-feature PR. The unified pipeline (Variant 1) buys coherence at the cost of a high-risk rewrite of the backward-compatible search core, and the two-tier split (Variant 3) imposes a contract boundary that session-focus and the gate don't cleanly respect; surgical helpers let each feature ship, test, and fail in isolation while provenance is plumbed through the existing `reasons[]` seam.
diff --git a/docs/brainstorm/recall-control-trust-suite/cli-output/codex.md b/docs/brainstorm/recall-control-trust-suite/cli-output/codex.md
new file mode 100644
index 00000000..96be748d
--- /dev/null
+++ b/docs/brainstorm/recall-control-trust-suite/cli-output/codex.md
@@ -0,0 +1,11 @@
+# Codex fallback attempt
+
+Codex fallback was attempted after the first Claude invocation produced no output.
+
+Result: no parseable variants. The CLI failed before model output due authentication errors:
+
+```text
+ERROR: Your access token could not be refreshed because your refresh token was already used. Please log out and sign in again.
+```
+
+The successful consultant output for this Phase 0 is recorded in `claude.md`.
diff --git a/docs/brainstorm/recall-control-trust-suite/cli-output/prompt.md b/docs/brainstorm/recall-control-trust-suite/cli-output/prompt.md
new file mode 100644
index 00000000..14dde0cb
--- /dev/null
+++ b/docs/brainstorm/recall-control-trust-suite/cli-output/prompt.md
@@ -0,0 +1,263 @@
+You are brainstorming architectural variants for the following task. Do not write code. Do not write a final design. Only produce variants and a recommendation.
+
+# Task
+
+Multi-task PR scope selected by the operator: Recall Control & Trust Suite.
+
+## Task t_b25d9891 - [upstream:flowstate-qmd] Structured recall query documents with lex, vector, HyDE, and intent lanes
+
+**Source**: https://github.com/amanning3390/flowstate-qmd
+**Upstream files studied**: `docs/SYNTAX.md`, `README.md`, `src/mcp/server.ts`, `src/store.ts`, `src/embedded-skills.ts`
+
+### What
+
+FlowState-QMD lets capable agents submit a structured recall request instead of one ambiguous string. A query document can include an optional `intent:` line plus typed search lanes such as `lex:`, `vec:`, and `hyde:`. Lexical lanes support exact phrases and negation; semantic lanes are validated separately; the server routes each lane to the right backend, fuses the ranked lists, and can expose explanation traces. The transferable feature is an agent-authored retrieval grammar, not a new ranking trick by itself.
+
+### Why useful for OSB
+
+OSB already has hybrid search, query intent classification, synonyms, MMR, link traversal, entity boost, recency, filters, and `why_retrieved`. Those are mostly internal ranking controls around a user query. Agents often know more structure than a single string can carry: an exact filename or phrase that must match, a semantic paraphrase for fuzzy recall, a hypothetical answer shape, and an intent that disambiguates overloaded terms. A small query-document grammar would let agents ask OSB for evidence in those lanes explicitly while keeping results auditable.
+
+### Current OSB status
+
+- **Verdict**: present_weaker
+- **Local evidence**: OSB search can classify intent and combine multiple recall signals, but I did not find an agent-facing structured query document format that accepts typed lexical, semantic, hypothetical-answer, and intent lanes as one request.
+- **Related existing work, not a duplicate**: existing query expansion rewrites the query inside OSB; property filters narrow result sets; `why_retrieved` explains selected evidence. This task gives callers a precise, inspectable request grammar and records which lane contributed each result.
+
+### Proposal
+
+Add an OSB structured recall query format for CLI/MCP. Start with a conservative parser for `intent:`, `lex:`, `vec:`, and `hyde:` lines, validate syntax per lane, route lexical requests to FTS and semantic requests to the configured embedding provider, fuse results through the existing ranker, and include lane-level contribution details in `why_retrieved`.
+
+### Acceptance criteria
+
+- CLI and MCP accept a structured recall document as input without breaking the existing plain-string search path.
+- Lexical lanes support quoted phrases and safe negation with clear validation errors for malformed syntax.
+- Semantic and HyDE lanes degrade gracefully when semantic search is disabled or embeddings are stale.
+- Result explanations show the source lane, backend, rank contribution, score contribution, and final selected snippet.
+- Tests cover mixed lex/vector requests, intent-only disambiguation, malformed lane syntax, semantic-disabled fallback, and duplicate result fusion.
+
+## Task t_38ec86bd - [upstream:Sibyl-Memory] Search-path FTS5 self-healing and natural-query hardening
+
+**Source**: https://github.com/Sibyl-Labs/Sibyl-Memory
+**Upstream files studied**: `sibyl-memory-client/src/sibyl_memory_client/client.py`, `sibyl-memory-client/CHANGELOG.md`
+
+### What
+
+Sibyl-Memory hardened its FTS5 query path in two useful ways. First, it classifies FTS5 failures so malformed user queries, missing schema, and backend corruption produce different outcomes. Second, when an external-content FTS5 index is corrupted or desynchronized, search attempts a bounded `rebuild` from the intact base table and retries once; if the index is still broken, the failure is contained instead of crashing the caller. It also drops standalone uppercase FTS5 operator tokens such as `AND`, `OR`, `NOT`, and `NEAR` from natural-language queries so users do not accidentally require those words literally.
+
+### Why useful for OSB
+
+OSB search is a core Second Brain surface. It already has CJK-aware FTS, query expansion, semantic ranking, MMR, link traversal, entity boost, recency, query cache, `why_retrieved`, and watchdog repair recommendations. The remaining reliability gap is query-path containment: a corrupted FTS shadow table or an awkward natural query should not make the agent lose memory access or silently miss obvious results. The system should repair what is rebuildable, fail loudly for true query errors, and preserve recall for common natural-language connector words.
+
+### Current OSB status
+
+- **Verdict**: present_weaker
+- **Local evidence**: `src/core/search/fts.ts` safely phrase-quotes tokens and handles CJK tokenization, while `src/core/search/store.ts` probes FTS5 availability and `brain_watchdog` can recommend search-index repair. CodeGraph did not show a search-path FTS rebuild/retry helper, and current token handling keeps operator words as literal required tokens.
+- **Related existing work, not a duplicate**: Watchdog recovery is an operator/probe surface. This task is about bounded self-healing inside the read path and improved natural-query tokenization.
+
+### Proposal
+
+Add an internal FTS safety layer around OSB keyword retrieval that classifies FTS failures, rebuilds rebuildable FTS indexes once, retries the query, and preserves clear error semantics. Extend `buildFtsMatch` tokenization to drop standalone FTS operator tokens when doing so does not empty the query.
+
+### Acceptance criteria
+
+- A corrupted rebuildable FTS table is rebuilt once and the original search retries successfully.
+- If rebuild fails, search returns a controlled `SearchError` or warning shape rather than an uncaught SQLite exception.
+- Programming/binding errors remain visible and are not converted to empty results.
+- Natural-language queries containing uppercase `AND`, `OR`, `NOT`, or `NEAR` do not require those words literally when other meaningful terms exist.
+- Queries made only of those words still search for the literal words.
+- CLI and MCP search outputs include enough warning metadata for operators to know a self-heal happened.
+
+## Task t_0e2f3a60 - [upstream:ClawMem] Adaptive retrieval gate for automatic context surfacing
+
+**Source**: https://github.com/yoloshii/ClawMem
+**Upstream files studied**: `README.md`, `src/retrieval-gate.ts`, `src/hooks.ts`, `src/hooks/context-surfacing.ts`
+
+### What
+
+ClawMem does not run expensive or noisy memory retrieval for every possible prompt. Its context-surfacing hook skips slash commands, very short prompts, greetings, confirmations, shell-command-shaped turns, and repeated heartbeat/dedupe prompts. It also filters obviously useless retrieved bodies before injection and records empty-turn usage rows when needed so later recall attribution stays aligned with transcript turns. The transferable feature is a prompt-aware gate around automatic retrieval, separate from the search engine itself.
+
+### Why useful for OSB
+
+OSB has strong explicit search and context-pack tools, and it is increasingly used through hooks and multiple agent runtimes. As automatic context surfacing grows, quality depends on knowing when _not_ to retrieve: pings, status checks, trivial acknowledgements, command-only turns, or noisy tool chatter should not burn token budget, create misleading usage telemetry, or reinforce irrelevant memories. This is a read-path complement to capture-boundary filtering.
+
+### Current OSB status
+
+- **Verdict**: not_in_osb_useful
+- **Local evidence**: Existing OSB work covers context-pack size limits, query intent, search ranking, source-aware session/message capture boundaries, and token-footprint diagnostics. I did not find a dedicated automatic-surfacing gate that classifies a prompt before retrieval, suppresses empty/noisy injections, and preserves telemetry alignment for skipped turns.
+- **Related existing work, not a duplicate**: source-aware capture boundaries decide which runtime sessions/messages become Brain evidence. This task decides whether a particular live prompt deserves Brain recall/injection at all.
+
+### Proposal
+
+Add a retrieval gate for OSB host hooks and automatic context-pack injection. The gate should be deterministic, configurable, cheap, and observable: it should skip obvious non-memory prompts, avoid raw prompt persistence before meaningful retrieval, and report why surfacing was skipped in debug mode.
+
+### Acceptance criteria
+
+- Automatic context surfacing skips configured noise prompt classes without invoking semantic/vector retrieval.
+- Skipped turns have explicit reasons in diagnostics and do not pollute learned recall signals.
+- Transcript-visible skipped turns preserve attribution alignment for later feedback loops.
+- The gate is configurable per runtime/profile and fail-open for unknown prompt shapes.
+- Tests cover greetings, slash commands, shell commands, duplicate prompts, real memory questions, and post-retrieval noise filtering.
+
+## Task t_ff693b7f - [upstream:ClawMem] Session-scoped focus topic for retrieval steering
+
+**Source**: https://github.com/yoloshii/ClawMem
+**Upstream files studied**: `README.md`, `src/session-focus.ts`, `src/hooks/context-surfacing.ts`
+
+### What
+
+ClawMem supports a per-session focus topic. A command writes a small session-specific focus file, and the context-surfacing path uses that topic to steer query expansion, reranking, snippet extraction, chunk selection, and final score adjustment. Matching results get a boost, non-matching results are gently demoted, and the signal is intentionally session-scoped: it does not become permanent memory and does not modify lifecycle metadata.
+
+### Why useful for OSB
+
+OSB has core memory pinning for transient session context, strong search ranking, context packs, and source-agent filters. What is still missing is a light way to tell recall: for this current session, bias searches toward this working topic without teaching the Brain a permanent preference. This is useful during multi-step research, release work, comparative upstream analysis, or debugging sessions where the operator says something like same process for ClawMem and expects follow-up turns to inherit the active topic.
+
+### Current OSB status
+
+- **Verdict**: present_weaker
+- **Local evidence**: transient pinning shipped, and OSB search already has recency, intent, MMR, entity boost, link traversal, and query expansion. Pinning preserves explicit facts, but it does not appear to provide a per-session retrieval-bias topic that influences search/context assembly while staying outside durable Brain state.
+- **Related existing work, not a duplicate**: Pinned memory keeps specific facts visible. This task adds a scoped ranking signal for which Brain evidence should be favored during the current session.
+
+### Proposal
+
+Add a session focus surface for OSB that can be set, cleared, inspected, and passed through search/context pack APIs. The focus should be explicit, temporary, and visible in `why_retrieved` so the operator can tell when it influenced recall.
+
+### Acceptance criteria
+
+- A focus topic can be set for one session without affecting other sessions.
+- Search/context-pack results show measurable but bounded ranking changes for focus-matching evidence.
+- `why_retrieved` reports the focus contribution.
+- Clearing or expiring the focus returns ranking behavior to baseline.
+- Focus state never becomes a confirmed preference or durable Brain fact unless explicitly promoted by the operator.
+
+## Task t_71942f88 - [upstream:plur] Polarity-aware Brain context lanes with consider pool
+
+**Source**: https://github.com/plur-ai/plur
+**Upstream files studied**: `README.md`, `packages/core/src/inject.ts`, `packages/core/src/polarity.ts`, `packages/core/src/schemas/engram.ts`, `packages/claw/src/assembler.ts`, `packages/mcp/src/tools.ts`
+
+### What
+
+PLUR classifies memories into practical injection lanes instead of dumping every recalled item into one undifferentiated block. Its injection pipeline detects `do not` / `never` / `avoid` style rules as constraints, keeps normal rules as directives, and puts lower-confidence or activation-spread matches into an `also consider` pool. It then formats the lanes with progressive disclosure: high-priority directives get fuller detail, constraints stay separate, and speculative context is only included when budget remains.
+
+### Why useful for OSB
+
+OSB already has context packs, recall ranking, token budgets, visibility filters, prompt-injection guardrails, and `why_retrieved`. The missing product affordance is semantic separation of the recalled Brain context before it reaches an agent. A hard prohibition like `never run destructive git commands`, a positive convention like `use Bun for this project`, and a weakly related memory should not arrive in the same paragraph with the same implied authority. Separating them would make OSB context easier for agents to follow and easier for operators to audit.
+
+### Current OSB status
+
+- **Verdict**: present_weaker
+- **Local evidence**: Existing tasks cover prompt-injection protection, MCP output boundaries, preference quality gates, score reasons, and context budgets. I did not find a Brain injection format that explicitly routes recalled items into directive, constraint, and low-confidence consideration lanes with different authority levels.
+- **Related existing work, not a duplicate**: preference quality gates improve what gets stored. This task changes how stored context is assembled and presented at retrieval/injection time.
+
+### Proposal
+
+Add polarity-aware context lanes to OSB recall/context-pack output. The feature should classify Brain items by operational authority, keep negative constraints visually and structurally separate from positive directives, and reserve a small optional pool for nearby but lower-confidence memories.
+
+### Acceptance criteria
+
+- Recalled Brain context can be returned as separate directive, constraint, and consider lanes.
+- Negative/prohibition preferences are classified deterministically and can be overridden in frontmatter.
+- `why_retrieved` and source provenance are preserved for every lane.
+- The consider pool is budget-capped and never crowds out higher-authority context.
+- Tests cover polarity classification, manual override, lane ordering, JSON output, and prompt rendering.
+- Existing plain recall output remains backwards compatible unless lane output is requested.
+
+## Task t_85581d59 - [upstream:Sibyl-Memory] Verified multi-record retrieval for evidence-spanning questions
+
+**Source**: https://github.com/Sibyl-Labs/Sibyl-Memory
+**Upstream files studied**: `sibyl-memory-client/src/sibyl_memory_client/multi_record.py`, `sibyl-memory-client/CHANGELOG.md`
+
+### What
+
+Sibyl-Memory added a `multi_record_search` helper for questions whose answer spans multiple records. Instead of requiring one record to contain every query term, it performs per-token recall, unions candidates, tracks which terms each record supports, then verifies the candidate set. The verification gates abstain when a significant term has no corpus support, require rare/selective term coverage, drop purely preparatory records for terminal-state queries, and rank by IDF-weighted coverage. The key insight is that recall-only multipass search can regress by pulling distractors; the verify gates are what make multi-record retrieval trustworthy.
+
+### Why useful for OSB
+
+OSB's search is already strong at ranking individual chunks and expanding recall through semantic, link, entity, recency, and graph layers. A Second Brain often needs answers that span several artifacts: a preference plus its apply-evidence log, a retired rule plus the superseding rule, a project note plus recent session facts, or a payment receipt plus its generated asset. A verified multi-record retrieval mode would return an evidence pack instead of a flat list, making agent answers more grounded and less likely to overfit the top single chunk.
+
+### Current OSB status
+
+- **Verdict**: not_in_osb_useful
+- **Local evidence**: OSB has hybrid search, `why_retrieved`, link traversal, `brain_context_pack`, `brain_synthesise`, graph export, backlinks, and timeline/evolution tools. It does not appear to have a dedicated retrieve-then-verify evidence-pack mode that checks query-term support across multiple records and abstains on unsupported terms.
+- **Related existing work, not a duplicate**: `brain_synthesise` and context packs gather bounded context; this task adds a query-time verification layer for evidence-spanning questions.
+
+### Proposal
+
+Add an optional OSB search mode that returns verified evidence packs for multi-record questions. The mode should gather candidates across search layers, compute support coverage for significant query terms, apply conservative verification gates, and return a structured pack with support/abstention reasons.
+
+### Acceptance criteria
+
+- Multi-record search can return several complementary Brain artifacts when no single chunk contains the full query vocabulary.
+- Unsupported significant terms produce an explicit abstention or missing-support warning.
+- Rare/selective term coverage is required so broad neighboring clusters do not pollute evidence packs.
+- Terminal-state queries down-rank or drop draft/planning-only records unless they also contain terminal evidence.
+- The output is available from CLI and MCP in JSON with matched/missing term diagnostics.
+- Tests cover evidence-spanning preference/log queries, retired/superseded chains, no-support abstention, distractor rejection, and CJK or non-English token handling.
+
+# Project context
+
+Project: Open Second Brain, TypeScript on Bun. Obsidian-compatible Markdown vault memory layer with CLI, MCP server, and runtime adapters.
+
+Recent commits:
+
+- 40d4e2b (v0.26.0) feat: cjk schema lifecycle recovery - CJK search, schema admin, lifecycle hooks, watchdog (#53)
+- f62918c (v0.25.0) feat: runtime schema packs foundation - schema vocabulary, artifact taxonomy, schema inspection (#52)
+- 14d1ee1 (v0.24.0) feat: brain model semantics foundation - typed preference relations, memory labels, dry-run backfill (#51)
+- 3a5d5c3 (v0.23.0) feat: agent capability CLI integration - runtime MCP capabilities, inherited JSON, completions (#50)
+- a085bfa (v0.22.0) feat: vault portability + session economy - codec, sources dashboard, vault-map tokens, multi-vault profiles, graph export/import (#49)
+- 73e4a28 (v0.21.0) feat: brain lifecycle suite - mutation audit, multi-phase dream, reconcile domains, morning brief, temporal extraction, heal enrichment (#48)
+- bd49cd5 (v0.20.0) feat: recall and ranking quality - Weibull recency, query intent, synonym expansion, recall budgets, query cache, pre-compress pack (#47)
+- cbbe18f (v0.19.0) feat: typed graph semantics - typed relations, visibility scoping, MCP landscape (#46)
+- 5fa7eb0 (v0.18.0) feat: MCP context economy - preview budget, artifact fetch, recall hint (#45)
+- 2147640 (v0.13.0) feat: Hybrid Search and Recall Quality: explainable recall, MMR, link traversal, entity boost, header anchoring (#37)
+
+Related files:
+
+- `src/core/search/fts.ts`: builds safe FTS5 MATCH expressions and runs keyword retrieval.
+- `src/core/search/store.ts`: central SQLite boundary; `keywordTopK`, `semanticTopK`, hydration, typed relation reads.
+- `src/core/search/search.ts`: orchestrates keyword, semantic, query plan, cache, rank, traversal, MMR, filters, visibility, relations, warnings.
+- `src/core/search/ranker.ts`: pure scoring and `reasons` layer assembly.
+- `src/core/search/query-plan.ts`: pure structural query intent and weight profile.
+- `src/core/search/types.ts`: `SearchOptions`, `SearchOutcome`, `BrainSearchResult`, warnings and error types.
+- `src/core/brain/context-pack.ts`: bounded context pack assembly from Brain preferences/retired pages.
+- `src/cli/search.ts`: `o2b search`, JSON and human rendering, property/visibility flags.
+- `src/mcp/search-tools.ts`: `brain_search` schema, argument parsing, error mapping, MCP output shape.
+- Existing tests: `tests/core/search/fts.test.ts`, `tests/core/search/search.test.ts`, `tests/core/search/ranker-reasons.test.ts`, `tests/mcp/search.test.ts`, `tests/mcp/brain-search-reasons.test.ts`.
+
+Conventions:
+
+- Core modules stay deterministic and do not depend on runtime adapters.
+- CLI wrappers parse flags, resolve config, shape exit codes, and render output; core owns behavior.
+- MCP tools validate inputs, map typed errors, and expose structured content plus compact text.
+- Search JSON exposes structured `reasons[]`; human output shows them only with `--verbose`.
+- Read paths should preserve backward compatibility unless a new explicit flag or option requests richer output.
+- Search index is rebuildable under `.open-second-brain/`; `Brain/active.md` is derived.
+- Public docs use the full project name, not OSB abbreviation.
+- Package scripts: `bun run typecheck`, `bun run lint`, `bun run fmt`, `bun run fmt:check`, `bun run test`, `bun run validate`, `bun run sync-version:check`.
+
+Constraints:
+
+- Do not add a vector database or new external retrieval dependency in this PR.
+- Keep plain string `o2b search` and `brain_search` backward-compatible.
+- Prefer pure parser/ranker helpers with focused tests.
+- Keep new automatic-surfacing behavior opt-in/configurable; do not change explicit `brain_search` semantics silently.
+- Keep self-healing bounded: at most one rebuild/retry, with a visible warning.
+- No new public AI-authorship markers.
+
+# Required output format
+
+Produce exactly 3 distinct architectural variants. For each variant:
+
+### Variant N: <short name>
+
+- **Approach**: 2-3 sentences describing the variant.
+- **Trade-offs**: bullet list of pros and cons.
+- **Complexity**: small | medium | large
+- **Risk**: low | medium | high
+
+After the three variants, add exactly one recommendation:
+
+### Recommended: Variant N
+
+**Rationale**: 2-3 sentences explaining why this variant over the others, considering the project context and constraints above.
+
+Output nothing outside of these sections.
diff --git a/docs/brainstorm/recall-control-trust-suite/design.md b/docs/brainstorm/recall-control-trust-suite/design.md
new file mode 100644
index 00000000..f94f9c79
--- /dev/null
+++ b/docs/brainstorm/recall-control-trust-suite/design.md
@@ -0,0 +1,94 @@
+# Recall Control & Trust Suite - precise, auditable recall without a search rewrite
+
+**Status:** draft
+**Author:** GitHub Copilot (via feature-release-playbook)
+**Audience:** implementation
+
+## Problem statement
+
+Open Second Brain already has strong hybrid search, structural query intent, CJK-aware FTS, query expansion, MMR, link traversal, entity boost, recency, context packs, and explainable `reasons[]`. The remaining gap is caller control and trust: agents cannot submit a structured recall request, the FTS read path cannot self-heal a rebuildable desync, automatic context surfacing has no cheap skip gate, session-local focus cannot steer ranking, context packs do not separate constraints from directives, and evidence-spanning questions still return a flat result list.
+
+This suite improves recall precision and auditability while preserving the existing search architecture. The default plain-string path must remain backward-compatible; richer behavior is opt-in through explicit options, CLI flags, or MCP arguments.
+
+## Scope
+
+- Add a deterministic structured recall query parser for `intent:`, `lex:`, `vec:`, and `hyde:` lanes.
+- Add bounded FTS5 read-path hardening: natural-language operator-token cleanup, failure classification, one rebuild/retry for rebuildable FTS desync, and visible warnings.
+- Add a pure retrieval surfacing gate for automatic context injection callers.
+- Add session-scoped focus state and ranking contribution plumbing without teaching the Brain a permanent preference.
+- Add polarity-aware context lanes (`directives`, `constraints`, `consider`) for context-pack style output.
+- Add optional verified multi-record evidence packs with matched/missing term diagnostics and abstention reasons.
+- Expose the new surfaces through focused CLI/MCP options and JSON fields while preserving current outputs by default.
+- Add focused unit and integration tests for each atomic task.
+
+## Out of scope
+
+- Adding a vector database backend or changing the storage model.
+- Replacing the existing `search.ts` orchestration with a new pipeline abstraction.
+- Changing default result ranking for callers that do not opt into the new features.
+- Learning retrieval weights from user feedback.
+- LLM-based answer generation or LLM-based verification.
+- Persisting session focus as a confirmed preference or durable Brain fact.
+- Blocking explicit `brain_search` when the surfacing gate would skip automatic context.
+
+## Chosen approach
+
+Use surgical pure helpers around the existing search architecture. Each selected task gets a small, deterministic module and focused tests, then minimal wiring at the current owning boundary:
+
+- parser before CLI/MCP search dispatch;
+- FTS safety at the keyword retrieval boundary;
+- gate before automatic context-surfacing callers;
+- focus as an optional ranking input and reason contribution;
+- polarity classification during context-pack assembly;
+- evidence-pack verification as an optional post-search mode.
+
+The implementation should prefer explicit data contracts over a new framework. The core stays deterministic and runtime-adapter independent; CLI and MCP remain thin parse/render layers.
+
+## Design decisions
+
+- Keep plain query behavior byte-compatible where possible; new behavior requires `--query-doc`, `--evidence-pack`, `--session-focus`, `--lanes`, or MCP equivalents.
+- Represent structured query documents as parsed lane specs, not as a new query language for storage. `lex` routes to existing FTS, `vec` and `hyde` route through existing semantic policy and degrade to warnings when semantic search cannot run.
+- Model FTS self-heal as a typed keyword retrieval outcome so warnings can flow through `SearchOutcome.warnings` without hiding real programmer errors.
+- Keep retrieval gate pure and caller-owned. It classifies prompt text and returns `{ shouldSurface, reason }`; hook/adapter code decides whether to record a skipped-turn diagnostic.
+- Store session focus in a small runtime file keyed by session id, with explicit set/status/clear operations and expiry. Search accepts focus as an option rather than reading global state implicitly.
+- Add focus and lane contributions to the existing `reasons[]`/`why_retrieved` concept instead of inventing a second explanation channel.
+- Build polarity lanes from existing context-pack candidates first; add deterministic classifier and optional frontmatter override, but keep the old flat context-pack output unless lane output is requested.
+- Implement evidence packs as a separate optional result mode. It may internally run multiple searches, but it returns structured coverage diagnostics and abstains when significant terms have no support.
+
+## File changes
+
+Expected new files:
+
+- `src/core/search/structured-query.ts`
+- `src/core/search/fts-safety.ts`
+- `src/core/search/surfacing-gate.ts`
+- `src/core/search/session-focus.ts`
+- `src/core/brain/context-lanes.ts`
+- `src/core/search/evidence-pack.ts`
+- focused test files under `tests/core/search/` and `tests/core/brain/`
+- MCP/CLI integration tests under `tests/mcp/` and `tests/cli/`
+
+Expected modified files:
+
+- `src/core/search/fts.ts`
+- `src/core/search/store.ts`
+- `src/core/search/search.ts`
+- `src/core/search/ranker.ts`
+- `src/core/search/types.ts`
+- `src/core/search/index.ts`
+- `src/core/brain/context-pack.ts`
+- `src/cli/search.ts`
+- `src/cli/brain.ts` or related brain subcommand wiring for focus/context lanes
+- `src/cli/command-manifest.ts`
+- `src/mcp/search-tools.ts`
+- `src/mcp/brain-tools.ts` if context-pack/focus MCP surfaces live there
+- `README.md`, `docs/cli-reference.md`, `docs/mcp.md`, `CHANGELOG.md`
+
+## Risks and open questions
+
+- FTS rebuild tests must corrupt/desync the FTS table without creating nondeterministic SQLite failures.
+- Structured `hyde` lanes may need to be treated as semantic text only; no LLM hypothetical-answer generation belongs in this PR.
+- Session focus storage must avoid cross-session bleed and must not require a daemon.
+- Evidence-pack verification can become expensive if implemented naively; cap candidate counts and keep it opt-in.
+- The context-lane output shape must be explicit enough for MCP clients but should not bloat default tool output.
+- The final implementation may trim one lower-priority task if Phase 2 proves the selected six-task scope risks a platform rewrite; any trim must be recorded in this design and plan before code completion.
diff --git a/docs/brainstorm/recall-control-trust-suite/plan.md b/docs/brainstorm/recall-control-trust-suite/plan.md
new file mode 100644
index 00000000..34b726f4
--- /dev/null
+++ b/docs/brainstorm/recall-control-trust-suite/plan.md
@@ -0,0 +1,52 @@
+# Recall Control & Trust Suite - implementation plan
+
+## Tasks
+
+### Task 1: FTS safety foundation
+
+- **Board task**: `t_38ec86bd`
+- **Files**: `src/core/search/fts.ts`, `src/core/search/fts-safety.ts`, `src/core/search/store.ts`, `src/core/search/search.ts`, `src/core/search/types.ts`, `tests/core/search/fts.test.ts`, `tests/core/search/search.test.ts`
+- **Acceptance**: Failing tests first show operator tokens in mixed natural queries currently over-constrain search and rebuildable FTS desync currently escapes as an uncontained SQLite error. Passing implementation drops standalone operator tokens only when meaningful tokens remain, retries one rebuildable FTS failure once, preserves programming errors, and emits a search warning when self-heal occurs.
+- **Depends on**: none
+
+### Task 2: Structured recall query documents
+
+- **Board task**: `t_b25d9891`
+- **Files**: `src/core/search/structured-query.ts`, `src/core/search/search.ts`, `src/core/search/types.ts`, `src/cli/search.ts`, `src/mcp/search-tools.ts`, `tests/core/search/structured-query.test.ts`, `tests/core/search/search.test.ts`, `tests/cli/search.test.ts`, `tests/mcp/search.test.ts`
+- **Acceptance**: Failing tests first show CLI/MCP reject or treat query documents as plain strings. Passing implementation parses `intent:`, `lex:`, `vec:`, and `hyde:` lanes behind explicit CLI/MCP input, validates malformed lane syntax, preserves plain string behavior, degrades semantic lanes with warnings when semantic search is unavailable, and includes lane contribution reasons.
+- **Depends on**: Task 1
+
+### Task 3: Session focus steering
+
+- **Board task**: `t_ff693b7f`
+- **Files**: `src/core/search/session-focus.ts`, `src/core/search/ranker.ts`, `src/core/search/search.ts`, `src/core/search/types.ts`, `src/cli/search.ts`, `src/cli/brain.ts`, `src/mcp/search-tools.ts`, `tests/core/search/session-focus.test.ts`, `tests/core/search/ranker-reasons.test.ts`, `tests/cli/search.test.ts`, `tests/mcp/search.test.ts`
+- **Acceptance**: Failing tests first show focus cannot be set or influence ranking. Passing implementation supports set/status/clear or equivalent explicit focus input, applies a bounded boost/demotion only for the selected session/query, reports focus contribution in reasons, and returns to baseline after clear/expiry.
+- **Depends on**: Task 2
+
+### Task 4: Retrieval surfacing gate
+
+- **Board task**: `t_0e2f3a60`
+- **Files**: `src/core/search/surfacing-gate.ts`, relevant hook/context-surfacing caller if present, `src/mcp/search-tools.ts` or hook diagnostics surface, `tests/core/search/surfacing-gate.test.ts`, integration test for the chosen caller
+- **Acceptance**: Failing tests first show automatic surfacing callers do not distinguish greetings, slash commands, shell-only prompts, duplicates, and real memory questions. Passing implementation exposes a pure fail-open gate with explicit skip reasons, avoids invoking retrieval for skipped automatic surfacing, and leaves explicit `brain_search` unchanged.
+- **Depends on**: Task 2
+
+### Task 5: Polarity-aware context lanes
+
+- **Board task**: `t_71942f88`
+- **Files**: `src/core/brain/context-lanes.ts`, `src/core/brain/context-pack.ts`, `src/mcp/brain-tools.ts`, CLI context-pack wiring, `tests/core/brain/context-lanes.test.ts`, context-pack integration tests
+- **Acceptance**: Failing tests first show context packs are flat and cannot separate constraints from directives. Passing implementation classifies deterministic negative/prohibition rules, honors a manual override, returns `directives`, `constraints`, and budget-capped `consider` lanes when requested, preserves source IDs/provenance, and keeps legacy flat output unless lanes are requested.
+- **Depends on**: Task 3
+
+### Task 6: Verified multi-record evidence packs
+
+- **Board task**: `t_85581d59`
+- **Files**: `src/core/search/evidence-pack.ts`, `src/core/search/search.ts`, `src/core/search/types.ts`, `src/cli/search.ts`, `src/mcp/search-tools.ts`, `tests/core/search/evidence-pack.test.ts`, `tests/core/search/search.test.ts`, `tests/cli/search.test.ts`, `tests/mcp/search.test.ts`
+- **Acceptance**: Failing tests first show no opt-in mode can return complementary evidence across multiple records with missing-term diagnostics. Passing implementation returns evidence packs with matched/missing significant terms, support coverage, dropped-candidate reasons, explicit abstention for unsupported significant terms, terminal-state down-ranking, and existing `why_retrieved` detail in JSON from CLI and MCP.
+- **Depends on**: Task 2
+
+### Task 7: Documentation, changelog, and version release prep
+
+- **Board tasks**: all selected tasks
+- **Files**: `README.md`, `docs/cli-reference.md`, `docs/mcp.md`, `CHANGELOG.md`, `package.json`, generated version sync targets
+- **Acceptance**: User-facing docs describe new explicit flags/options and backward compatibility. `CHANGELOG.md` has one new version entry for the PR. Version sync check passes after the release version bump phase required by the operator.
+- **Depends on**: Tasks 1-6
diff --git a/docs/brainstorm/recall-control-trust-suite/variants.md b/docs/brainstorm/recall-control-trust-suite/variants.md
new file mode 100644
index 00000000..073616f9
--- /dev/null
+++ b/docs/brainstorm/recall-control-trust-suite/variants.md
@@ -0,0 +1,56 @@
+# Recall Control & Trust Suite - brainstorm variants
+
+## Source tasks
+
+- `t_b25d9891` P4 - Structured recall query documents with lex, vector, HyDE, and intent lanes.
+- `t_38ec86bd` P3 - Search-path FTS5 self-healing and natural-query hardening.
+- `t_0e2f3a60` P3 - Adaptive retrieval gate for automatic context surfacing.
+- `t_ff693b7f` P2 - Session-scoped focus topic for retrieval steering.
+- `t_71942f88` P2 - Polarity-aware Brain context lanes with consider pool.
+- `t_85581d59` P2 - Verified multi-record retrieval for evidence-spanning questions.
+
+## Consultant output
+
+### Variant 1: Unified recall pipeline orchestrator
+
+- **Approach**: Introduce an explicit, ordered `RecallPipeline` in the search core that threads a single mutable `RecallContext` (raw input, parsed query document, gate decision, heal status, per-lane candidates, verification verdicts, polarity lanes) through typed stages: parse -> surfacing-gate -> FTS-self-heal -> lane-route -> fuse -> multi-record-verify -> polarity-lane. Each of the six tasks becomes one stage with a declared input/output contract, and `search.ts` is refactored to drive the pipeline instead of calling keyword/semantic/rank/MMR directly.
+- **Trade-offs**:
+ - Pro: one coherent mental model; every feature has an obvious home and ordering is explicit and testable end-to-end.
+ - Pro: `why_retrieved` provenance flows naturally as accumulated context across stages.
+ - Con: forces a large refactor of the central, backward-compatible `search.ts`, putting the plain-string path at risk.
+ - Con: couples six otherwise-independent features into one abstraction - a bug or churn in the pipeline contract touches all of them.
+ - Con: over-engineered for features like session-focus and the retrieval gate that don't naturally belong on the same path.
+- **Complexity**: large
+- **Risk**: high
+
+### Variant 2: Surgical pure helpers around existing search
+
+- **Approach**: Keep `search.ts`, `ranker.ts`, and `fts.ts` structurally intact and add each task as a focused, deterministic helper module (query-document parser, FTS-heal wrapper, surfacing gate, session-focus store, polarity classifier, multi-record verifier), each with its own focused test suite. Wire each helper in at the minimal existing call site - the parser in front of `brain_search`/`o2b search` argument handling, the heal wrapper inside `keywordTopK`, the polarity/verify layers at result-assembly - gated behind new explicit options so the plain-string path is untouched by default.
+- **Trade-offs**:
+ - Pro: directly honors the conventions (pure helpers, focused tests) and the backward-compat / bounded-self-heal constraints.
+ - Pro: features ship and are reviewable independently; one feature's risk is contained to its call site.
+ - Pro: lowest blast radius on the hot read path.
+ - Con: no single place describes the overall recall flow; cross-feature ordering (gate before heal before fuse before lanes) lives implicitly in call-site sequencing.
+ - Con: `why_retrieved` provenance must be plumbed through several seams rather than accumulated in one context object.
+- **Complexity**: medium
+- **Risk**: low
+
+### Variant 3: Two-tier request / trust split
+
+- **Approach**: Organize the six tasks into two cohesive subsystems sharing one extended `SearchOptions`/`SearchOutcome` contract: a **request tier** (structured query document, retrieval gate, session focus - "what to ask and whether to ask") and a **trust tier** (FTS self-heal, multi-record verification, polarity lanes - "make results reliable and well-presented"). The request tier produces a normalized intent/lane spec consumed by the existing search, and the trust tier wraps and post-processes its output; `search.ts` gains two thin seams rather than a full rewrite.
+- **Trade-offs**:
+ - Pro: meaningful conceptual grouping that aids review and documentation without a monolithic pipeline.
+ - Pro: contained changes to `search.ts` (two seams) while still giving provenance a shared contract to ride on.
+ - Pro: request-tier features can short-circuit (gate skips, focus-only) before touching retrieval.
+ - Con: the two-tier boundary is somewhat artificial - session-focus spans both asking and ranking, blurring the split.
+ - Con: more upfront contract design than Variant 2 for marginal organizational gain on an already-large PR.
+- **Complexity**: medium
+- **Risk**: medium
+
+## Orchestrator decision
+
+Chosen approach: Variant 2, surgical pure helpers around existing search.
+
+I agree with the consultant recommendation. Open Second Brain already has a layered search core: `fts.ts` owns FTS expressions, `query-plan.ts` owns pure query analysis, `ranker.ts` owns scoring reasons, `search.ts` orchestrates retrieval, and CLI/MCP modules only parse and render. The selected tasks can be implemented as small, explicit helper surfaces without replacing that architecture or changing the default plain-string search behavior.
+
+The main override to the consultant wording is scope discipline: each helper must expose only the minimum contract needed by the selected task, and richer outputs must stay behind explicit CLI/MCP flags or typed options. The default `o2b search "query"` and `brain_search` behavior remains backward-compatible unless callers opt into structured documents, evidence packs, focus, lanes, or surfacing diagnostics.
diff --git a/docs/cli-reference.md b/docs/cli-reference.md
index a2b119bd..b1608e8c 100644
--- a/docs/cli-reference.md
+++ b/docs/cli-reference.md
@@ -87,7 +87,7 @@ o2b brain summary Operator dashboard - trust verdict (clean | watch
o2b brain page-dedup Page-level duplicate detector (by content hash + frontmatter similarity)
o2b brain lint Self-healing structural drift fixer; --consolidate folds multi-source duplicates
o2b brain token-footprint Token-budget monitor across instruction files and active.md
-o2b brain context-pack Bounded-token vault slice for priming an agent's context window (--max-tokens N)
+o2b brain context-pack Bounded-token vault slice for priming an agent's context window (--max-tokens N, --lanes for directives/constraints/consider)
o2b brain synthesise Concept-scoped JSON envelope: target node + linkers + optional unlinked mentions
o2b brain moc-audit Per-MOC coverage audit: classify cluster members into well-covered / fragile / candidate-missing
o2b brain unlinked Raw-text mentions outside `[[...]]` (Unicode-aware boundaries)
@@ -138,13 +138,30 @@ Full Pay Memory walkthrough: [`pay-memory.md`](pay-memory.md).
o2b search "<query>" Hybrid full-text + semantic search across the vault
--property type=decision --property status=open
filters on frontmatter scalars (post-FTS phase)
+ --query-doc '<lanes>' separates intent/lex/vec/hyde recall lanes
+ --evidence-pack adds matched/missing term diagnostics and abstention text
--verbose adds per-result why_retrieved reasons
--json for structured output (includes reasons[])
CJK text is expanded for FTS recall without polluting returned content
+o2b search focus set Persist a 120-minute ranking focus (--query Q and/or --path P; --ttl-minutes N)
+o2b search focus status Show the active focus; --json emits { active, focus }
+o2b search focus clear Clear the persisted focus file next to the search index
o2b search reindex Rebuild the SQLite + FTS5 index from scratch
(required after upgrading to v0.13.0 recall schema or v0.26.0 CJK FTS content)
```
+Structured recall query documents are line-oriented. `intent:` accepts
+`neutral`, `exact`, `entity`, or `broad`; `lex:` accepts bare or quoted terms
+and `-excluded` tokens; `vec:` and `hyde:` provide semantic text lanes when the
+semantic layer is configured. Example:
+
+```text
+intent: entity
+lex: "project alpha" -archived
+vec: active implementation context
+hyde: a note that explains the current project alpha decision
+```
+
The fused ranking is sharpened by a recall-quality suite (v0.13.0), each
layer config-tunable and bounded:
@@ -169,7 +186,9 @@ Recall and ranking quality (v0.20.0), each tunable and bounded:
| `search_cache_ttl_seconds` | `OPEN_SECOND_BRAIN_SEARCH_CACHE_TTL` | `300` | Cache row time-to-live in seconds |
`brain_context_pack` also accepts `max_chars_per_memory` and
-`max_total_chars` (code-point caps), and the read-only
+`max_total_chars` (code-point caps). Pass `--lanes` to keep the legacy flat
+items while also returning `directives`, `constraints`, and `consider` lanes
+derived from polarity cues and page tier. The read-only
`brain_pre_compress_pack` MCP tool returns a budgeted
top-preferences-plus-`active.md` addendum for a host runtime to inject
before a context-compression event.
diff --git a/docs/mcp.md b/docs/mcp.md
index df62572a..aaa5a23b 100644
--- a/docs/mcp.md
+++ b/docs/mcp.md
@@ -18,7 +18,7 @@ in Open Second Brain depends on the MCP server being running.
## Tool Highlights
-The full server currently advertises 57 tools. The table below highlights the
+The full server currently advertises 58 tools. The table below highlights the
operator-facing core, schema, agent-source, health, recovery, and Pay Memory
tools; the full surface also includes Brain writer, review, query, temporal,
link-graph, and search tools. In Claude Code, that full schema can push MCP definitions beyond
@@ -38,6 +38,9 @@ capability flags for a narrower per-process full server.
| `brain_agent_diff` | Read-only comparison between source agents using browse/search/diff/map modes over the same provenance foundation. | — |
| `brain_audit` | Read-only per-preference mutation trail (create / promote / update / retire / merge) with agent, reason, revision + content-hash before/after. | `pref_id` |
| `brain_morning_brief` | Read-only session-start summary: top confirmed preferences, recent reconcile open questions, recent notes; character-budgeted. | — |
+| `brain_search` | Read-only vault search with optional structured query lanes, explicit focus hints, and evidence-pack diagnostics. | `query` |
+| `brain_recall_gate` | Read-only classifier for whether an automatic recall attempt should run; returns `retrieve` plus a stable reason. | `prompt` |
+| `brain_context_pack` | Budgeted context slice; pass `lanes: true` to return directives, constraints, and consider lanes alongside legacy flat items. | `max_tokens` |
| `brain_sources` | Read-only dashboard of signals grouped by (agent, source_type) with active/processed and distinct-topic counts. | — |
| `get_active_schema_pack` | Return the active runtime schema pack resolved from `Brain/_brain.yaml`. | — |
| `list_schema_packs` | List schema packs available to the vault/runtime. | — |
@@ -65,6 +68,12 @@ capability flags for a narrower per-process full server.
(`signal`, `preference`, `log`), and `limit` (1-500, default 50).
`brain_agent_diff` accepts the same filters plus `mode` (`browse`, `search`,
`diff`, `map`). Omitting `agents` means all known source agents.
+`brain_search` accepts `query_document` with line-oriented `intent:`, `lex:`,
+`vec:`, and `hyde:` lanes; `focus_query` / `focus_path_prefix` to steer a
+single call; and `evidence_pack: true` to return significant/matched/missing
+terms, abstention text, terminal-state downrank reasons, and per-result
+`why_retrieved`. `brain_recall_gate` accepts optional `previous_prompt` and
+`explicit`; `explicit: true` always returns `retrieve: true`.
`payment_memory_init` accepts `agent` (string) and `overwrite` (boolean — to
refresh the policy template). `payment_receipt_append` accepts the same
@@ -303,7 +312,7 @@ server to your Codex MCP config the same way as Hermes.
The plugin's `.mcp.json` ships **two** MCP-server entries:
-- `open-second-brain` - the full surface (57 tools, including `brain_health`, `brain_mcp_landscape`, `brain_agent_query`, `brain_agent_diff`, `brain_pinned_context`, `brain_pre_compress_pack`, `brain_audit`, `brain_morning_brief`, `brain_sources`, and `brain_switch_vault`); subject to Claude Code's `MCPSearch` tool-search deferral when MCP definitions push the system prompt past 10% of the context window.
+- `open-second-brain` - the full surface (58 tools, including `brain_health`, `brain_mcp_landscape`, `brain_agent_query`, `brain_agent_diff`, `brain_recall_gate`, `brain_pinned_context`, `brain_pre_compress_pack`, `brain_audit`, `brain_morning_brief`, `brain_sources`, and `brain_switch_vault`); subject to Claude Code's `MCPSearch` tool-search deferral when MCP definitions push the system prompt past 10% of the context window.
- `open-second-brain-writer` - a minimal always-loaded surface of five tools: `brain_feedback`, `brain_apply_evidence`, `brain_note`, `brain_pinned_context` (writers) and `brain_context` (read-only pull-bootstrap of `Brain/active.md` plus pinned context, v0.16.0). The agent records taste signals, evidence events, milestone notes, and current-task pinned facts - and fetches the active rule digest at session start in runtimes without a SessionStart hook - without a ToolSearch round-trip on every session boot.
Both servers reuse the same backing CLI (`o2b mcp --scope writer` vs the default `--scope full`). Handlers are byte-identical; the writer-mode instructions text explicitly tells the agent to prefer the writer copy over any duplicate the full server still exposes (both call the same code path).
diff --git a/openclaw.plugin.json b/openclaw.plugin.json
index 9fc566eb..2078577e 100644
--- a/openclaw.plugin.json
+++ b/openclaw.plugin.json
@@ -2,7 +2,7 @@
"id": "open-second-brain",
"name": "Open Second Brain",
"description": "Second brain for AI agents using Obsidian-compatible Markdown vaults.",
- "version": "0.26.0",
+ "version": "0.27.0",
"activation": { "onStartup": true },
"skills": ["./skills"],
"contracts": {
diff --git a/package.json b/package.json
index 621cf5e5..9531c7b0 100644
--- a/package.json
+++ b/package.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "0.26.0",
+ "version": "0.27.0",
"private": false,
"description": "Second brain for AI agents using Obsidian-compatible Markdown vaults. Works with Hermes, Claude Code, Codex, and OpenClaw.",
"keywords": [
diff --git a/plugin.yaml b/plugin.yaml
index 11b7e293..2dded011 100644
--- a/plugin.yaml
+++ b/plugin.yaml
@@ -1,5 +1,5 @@
name: open-second-brain
-version: "0.26.0"
+version: "0.27.0"
description: "Hermes plugin entrypoint for OpenSecondBrain vault health checks and the optional MCP tool server."
author: "OpenSecondBrain contributors"
kind: standalone
diff --git a/plugins/codex/.codex-plugin/plugin.json b/plugins/codex/.codex-plugin/plugin.json
index f53c82ff..7d7997d2 100644
--- a/plugins/codex/.codex-plugin/plugin.json
+++ b/plugins/codex/.codex-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "0.26.0",
+ "version": "0.27.0",
"description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.",
"author": {
"name": "Open Second Brain contributors",
diff --git a/plugins/hermes/plugin.yaml b/plugins/hermes/plugin.yaml
index 4427db36..3ef47b54 100644
--- a/plugins/hermes/plugin.yaml
+++ b/plugins/hermes/plugin.yaml
@@ -1,5 +1,5 @@
name: open-second-brain
-version: "0.26.0"
+version: "0.27.0"
description: "Hermes adapter for OpenSecondBrain vault health checks and optional MCP tool server."
author: "OpenSecondBrain contributors"
kind: standalone
diff --git a/pyproject.toml b/pyproject.toml
index 13fb1424..93a89b40 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -10,7 +10,7 @@ build-backend = "setuptools.build_meta"
# CLI entry points (those moved to `package.json` `bin`).
[project]
name = "open-second-brain"
-version = "0.26.0"
+version = "0.27.0"
description = "Hermes Python shim for Open Second Brain. Most of the project (CLI, MCP server, OpenClaw plugin) is TypeScript on Bun; see package.json."
readme = "README.md"
requires-python = ">=3.11"
diff --git a/src/cli/brain/help-text.ts b/src/cli/brain/help-text.ts
index e7e6f1b8..b6d6ec1c 100644
--- a/src/cli/brain/help-text.ts
+++ b/src/cli/brain/help-text.ts
@@ -326,12 +326,13 @@ export const VERB_HELP: Record<string, string> = {
"threshold (default 200000; override via --warn-threshold or the\n" +
"BRAIN_TOKEN_WARN_THRESHOLD env var).\n",
"context-pack":
- "usage: o2b brain context-pack --max-tokens <n> [--query <q>] [--vault <path>] [--json]\n" +
+ "usage: o2b brain context-pack --max-tokens <n> [--query <q>] [--lanes] [--vault <path>] [--json]\n" +
"Return the highest-tier, most recent vault slice that fits under\n" +
"<n> tokens. Items ordered core → supporting → peripheral, then\n" +
"newest first. Stops adding pages when the next page would exceed\n" +
"the budget. --query <q> filters by NFKC+casefold substring match\n" +
- "on topic + principle.\n",
+ "on topic + principle. --lanes also returns directives, constraints,\n" +
+ "and consider lanes alongside the legacy flat items list.\n",
lint:
"usage: o2b brain lint --consolidate [--apply] [--yes] [--vault <path>] [--json]\n" +
"Self-healing structural lint. Dry-run by default; --apply writes\n" +
diff --git a/src/cli/brain/verbs/context-pack.ts b/src/cli/brain/verbs/context-pack.ts
index 9bba34d8..82ea52b1 100644
--- a/src/cli/brain/verbs/context-pack.ts
+++ b/src/cli/brain/verbs/context-pack.ts
@@ -16,6 +16,7 @@ export async function cmdBrainContextPack(argv: string[]): Promise<number> {
json: { type: "boolean" },
"max-tokens": { type: "string" },
query: { type: "string" },
+ lanes: { type: "boolean" },
});
const config = defaultConfigPath();
const vault = resolveBrainVault(flags["vault"] as string | undefined, config);
@@ -34,6 +35,7 @@ export async function cmdBrainContextPack(argv: string[]): Promise<number> {
const report = packContext(vault, {
maxTokens,
...(flags["query"] ? { query: flags["query"] as string } : {}),
+ ...(flags["lanes"] === true ? { includeLanes: true } : {}),
});
if (flags["json"]) {
@@ -47,6 +49,7 @@ export async function cmdBrainContextPack(argv: string[]): Promise<number> {
tokens: i.tokens,
})),
skipped: report.skipped,
+ ...(report.lanes ? { lanes: report.lanes } : {}),
});
return 0;
}
diff --git a/src/cli/search.ts b/src/cli/search.ts
index d4279a13..e5b4a5c2 100644
--- a/src/cli/search.ts
+++ b/src/cli/search.ts
@@ -22,6 +22,12 @@ import {
resolveSearchConfig,
search,
SearchError,
+ clearSessionFocus,
+ normalizeSessionFocus,
+ parseStructuredRecallQueryDocument,
+ readSessionFocus,
+ structuredRecallQueryText,
+ writeSessionFocus,
} from "../core/search/index.ts";
import type {
IndexCheckReport,
@@ -29,12 +35,13 @@ import type {
IndexStats,
IndexStatusSnapshot,
ResolvedSearchConfig,
+ SearchSessionFocus,
SearchOutcome,
} from "../core/search/index.ts";
import { CliError, parseFlags } from "./argparse.ts";
import { CronTemplateError, renderCronTemplate } from "./search-cron-template.ts";
-const KNOWN_VERBS = new Set(["query", "index", "reindex", "status", "check"]);
+const KNOWN_VERBS = new Set(["query", "index", "reindex", "status", "check", "focus"]);
export async function handleSearchSubcommand(argv: ReadonlyArray<string>): Promise<number> {
// First positional is verb iff it matches a known verb. Otherwise the
@@ -58,6 +65,8 @@ export async function handleSearchSubcommand(argv: ReadonlyArray<string>): Promi
return await cmdSearchStatus(rest);
case "check":
return await cmdSearchCheck(rest);
+ case "focus":
+ return await cmdSearchFocus(rest);
default:
process.stderr.write(`error: unknown search verb: ${verb}\n`);
return 2;
@@ -104,6 +113,70 @@ function resolveConfig(
return resolveSearchConfig({ vault, configPath, overrides });
}
+// ─── focus ───────────────────────────────────────────────────────────────────
+
+async function cmdSearchFocus(argv: ReadonlyArray<string>): Promise<number> {
+ const action = argv[0];
+ if (!action || !["set", "status", "clear"].includes(action)) {
+ throw new CliError("usage: o2b search focus <set|status|clear> [--query Q] [--path P]");
+ }
+ const { flags } = parseFlags(argv.slice(1), {
+ vault: { type: "string" },
+ config: { type: "string" },
+ db: { type: "string" },
+ query: { type: "string" },
+ path: { type: "string" },
+ "ttl-minutes": { type: "string", default: "120" },
+ json: { type: "boolean" },
+ });
+ const cfg = resolveConfig(flags);
+
+ if (action === "set") {
+ const ttlMinutes = Number(flags["ttl-minutes"] ?? "120");
+ const focus = normalizeSessionFocus(
+ {
+ query: typeof flags["query"] === "string" ? (flags["query"] as string) : null,
+ pathPrefix: typeof flags["path"] === "string" ? (flags["path"] as string) : null,
+ ttlMinutes,
+ },
+ Date.now(),
+ );
+ writeSessionFocus(cfg, focus);
+ writeFocusResponse(focus, flags["json"] === true);
+ return 0;
+ }
+
+ if (action === "clear") {
+ clearSessionFocus(cfg);
+ writeFocusResponse(null, flags["json"] === true);
+ return 0;
+ }
+
+ writeFocusResponse(readSessionFocus(cfg), flags["json"] === true);
+ return 0;
+}
+
+function focusJson(focus: SearchSessionFocus | null): Record<string, unknown> {
+ return { active: focus !== null, focus };
+}
+
+function writeFocusResponse(focus: SearchSessionFocus | null, json: boolean): void {
+ if (json) {
+ process.stdout.write(JSON.stringify(focusJson(focus)) + "\n");
+ return;
+ }
+ if (!focus) {
+ process.stdout.write("search focus: inactive\n");
+ return;
+ }
+ const parts = [
+ focus.query !== null ? `query=${JSON.stringify(focus.query)}` : null,
+ focus.pathPrefix !== null ? `path=${JSON.stringify(focus.pathPrefix)}` : null,
+ focus.expiresAt !== null ? `expires_at=${new Date(focus.expiresAt).toISOString()}` : null,
+ ].filter((part): part is string => part !== null);
+ process.stdout.write(`search focus: active ${parts.join(" ")}\n`);
+}
+
// ─── query ────────────────────────────────────────────────────────────────────
async function cmdSearchQuery(argv: ReadonlyArray<string>): Promise<number> {
@@ -120,17 +193,30 @@ async function cmdSearchQuery(argv: ReadonlyArray<string>): Promise<number> {
"auto-refresh": { type: "boolean" },
property: { type: "string-array" },
visibility: { type: "string-array" },
+ "query-doc": { type: "string" },
+ "evidence-pack": { type: "boolean" },
json: { type: "boolean" },
verbose: { type: "boolean" },
});
- if (positional.length === 0) {
+ const rawQueryDocument =
+ typeof flags["query-doc"] === "string" ? (flags["query-doc"] as string) : undefined;
+ const structuredQuery =
+ rawQueryDocument !== undefined
+ ? parseStructuredRecallQueryDocument(rawQueryDocument)
+ : undefined;
+
+ if (positional.length === 0 && structuredQuery === undefined) {
throw new CliError("query string is required");
}
if (flags["semantic"] === true && flags["keyword-only"] === true) {
throw new CliError("--semantic and --keyword-only are mutually exclusive");
}
- const query = positional.join(" ");
+ const query =
+ positional.length > 0 ? positional.join(" ") : structuredRecallQueryText(structuredQuery!);
+ if (query.trim().length === 0) {
+ throw new CliError("query string is required when --query-doc has no searchable lanes");
+ }
const limitNum = Number(flags["limit"] ?? "10");
if (!Number.isInteger(limitNum) || limitNum < 1 || limitNum > 100) {
throw new CliError("--limit must be an integer in 1..100");
@@ -164,6 +250,8 @@ async function cmdSearchQuery(argv: ReadonlyArray<string>): Promise<number> {
pathPrefix: typeof flags["path"] === "string" ? (flags["path"] as string) : undefined,
...(properties !== undefined ? { properties } : {}),
...(visibility !== undefined && visibility.length > 0 ? { visibility } : {}),
+ ...(structuredQuery !== undefined ? { structuredQuery } : {}),
+ ...(flags["evidence-pack"] === true ? { evidencePack: true } : {}),
});
if (flags["json"]) {
@@ -219,12 +307,36 @@ function jsonForOutcome(o: SearchOutcome): unknown {
end_line: r.endLine,
search_type: r.searchType,
reasons: r.reasons,
+ ...(o.evidencePack ? { why_retrieved: r.reasons } : {}),
document_id: r.documentId,
chunk_id: r.chunkId,
...(r.relations && r.relations.length > 0 ? { relations: r.relations } : {}),
})),
warnings: o.warnings,
total: o.total,
+ ...(o.evidencePack ? { evidence_pack: jsonForEvidencePack(o.evidencePack) } : {}),
+ };
+}
+
+function jsonForEvidencePack(pack: NonNullable<SearchOutcome["evidencePack"]>): unknown {
+ return {
+ significant_terms: pack.significantTerms,
+ matched_terms: pack.matchedTerms,
+ missing_terms: pack.missingTerms,
+ support_coverage: pack.supportCoverage,
+ records: pack.records.map((record) => ({
+ path: record.path,
+ document_id: record.documentId,
+ chunk_id: record.chunkId,
+ matched_terms: record.matchedTerms,
+ missing_terms: record.missingTerms,
+ support_coverage: record.supportCoverage,
+ terminal_state: record.terminalState,
+ why_retrieved: record.whyRetrieved,
+ dropped_candidate_reasons: record.droppedCandidateReasons,
+ })),
+ dropped_candidates: pack.droppedCandidates,
+ abstention: pack.abstention,
};
}
diff --git a/src/core/brain/context-lanes.ts b/src/core/brain/context-lanes.ts
new file mode 100644
index 00000000..4ada194e
--- /dev/null
+++ b/src/core/brain/context-lanes.ts
@@ -0,0 +1,78 @@
+import type { PageTier } from "./page-meta/tier.ts";
+
+export type ContextLaneName = "directives" | "constraints" | "consider";
+
+export interface ContextLaneSource {
+ readonly id: string;
+ readonly path: string;
+ readonly tier: PageTier;
+ readonly tokens: number;
+ readonly body: string;
+ readonly trimmed: boolean;
+ readonly sourceId: string;
+ readonly sourcePath: string;
+}
+
+export interface ContextLanesReport {
+ readonly directives: ReadonlyArray<ContextLaneSource>;
+ readonly constraints: ReadonlyArray<ContextLaneSource>;
+ readonly consider: ReadonlyArray<ContextLaneSource>;
+}
+
+export interface ContextLaneInput {
+ readonly id: string;
+ readonly path: string;
+ readonly tier: PageTier;
+ readonly tokens: number;
+ readonly body: string;
+ readonly trimmed: boolean;
+ readonly principle: string;
+ readonly manualLane?: ContextLaneName | null;
+}
+
+const CONSTRAINT_RE =
+ /\b(?:avoid|do not|don't|must not|never|no longer|forbid|forbidden)\b|\b(?:не делай|никогда|запрещено|избегай)\b/iu;
+
+export function normalizeContextLane(raw: unknown): ContextLaneName | null {
+ if (typeof raw !== "string") return null;
+ const value = raw.trim().toLowerCase();
+ if (value === "directives" || value === "constraints" || value === "consider") return value;
+ return null;
+}
+
+export function classifyContextLane(input: ContextLaneInput): ContextLaneName {
+ if (input.manualLane) return input.manualLane;
+ const text = `${input.principle}\n${input.body}`;
+ if (CONSTRAINT_RE.test(text)) return "constraints";
+ if (input.tier === "peripheral") return "consider";
+ return "directives";
+}
+
+export function buildContextLanes(inputs: ReadonlyArray<ContextLaneInput>): ContextLanesReport {
+ const directives: ContextLaneSource[] = [];
+ const constraints: ContextLaneSource[] = [];
+ const consider: ContextLaneSource[] = [];
+
+ for (const input of inputs) {
+ const source = Object.freeze({
+ id: input.id,
+ path: input.path,
+ tier: input.tier,
+ tokens: input.tokens,
+ body: input.body,
+ trimmed: input.trimmed,
+ sourceId: input.id,
+ sourcePath: input.path,
+ });
+ const lane = classifyContextLane(input);
+ if (lane === "constraints") constraints.push(source);
+ else if (lane === "consider") consider.push(source);
+ else directives.push(source);
+ }
+
+ return Object.freeze({
+ directives: Object.freeze(directives),
+ constraints: Object.freeze(constraints),
+ consider: Object.freeze(consider),
+ });
+}
diff --git a/src/core/brain/context-pack.ts b/src/core/brain/context-pack.ts
index b8a6e571..d6312dbc 100644
--- a/src/core/brain/context-pack.ts
+++ b/src/core/brain/context-pack.ts
@@ -22,6 +22,12 @@ import { PAGE_TIER, readTier, type PageTier } from "./page-meta/tier.ts";
import { estimateTokens } from "./text/tokenizer.ts";
import { normalizeForDedup } from "./text/normalize.ts";
import { applyCharBudget } from "./recall-budget.ts";
+import {
+ buildContextLanes,
+ normalizeContextLane,
+ type ContextLaneName,
+ type ContextLanesReport,
+} from "./context-lanes.ts";
const TIER_ORDER: ReadonlyArray<PageTier> = [
PAGE_TIER.core,
@@ -35,6 +41,8 @@ export interface ContextPackItem {
readonly tier: PageTier;
readonly tokens: number;
readonly body: string;
+ readonly principle: string;
+ readonly contextLane: ContextLaneName | null;
/** True when `body` was truncated by `maxCharsPerMemory` (v0.20.0). */
readonly trimmed: boolean;
}
@@ -50,6 +58,7 @@ export interface ContextPackReport {
readonly tokensUsed: number;
readonly items: ReadonlyArray<ContextPackItem>;
readonly skipped: ReadonlyArray<ContextPackSkipped>;
+ readonly lanes?: ContextLanesReport;
}
export interface ContextPackOptions {
@@ -69,6 +78,8 @@ export interface ContextPackOptions {
* `over-char-budget` skip reason. <= 0 / undefined disables.
*/
readonly maxTotalChars?: number;
+ /** Opt-in polarity-aware lanes. Omitted preserves the legacy flat output shape. */
+ readonly includeLanes?: boolean;
}
interface Candidate {
@@ -78,10 +89,32 @@ interface Candidate {
readonly createdAtMs: number;
readonly topic: string;
readonly principle: string;
+ readonly contextLane: ContextLaneName | null;
readonly body: string;
readonly tokens: number;
}
+function withOptionalLanes(
+ opts: ContextPackOptions,
+ items: ReadonlyArray<ContextPackItem>,
+): { readonly lanes?: ContextLanesReport } {
+ if (opts.includeLanes !== true) return {};
+ return {
+ lanes: buildContextLanes(
+ items.map((item) => ({
+ id: item.id,
+ path: item.path,
+ tier: item.tier,
+ tokens: item.tokens,
+ body: item.body,
+ trimmed: item.trimmed,
+ principle: item.principle,
+ manualLane: item.contextLane,
+ })),
+ ),
+ };
+}
+
function collectCandidates(vault: string): Candidate[] {
const dirs = brainDirs(vault);
const out: Candidate[] = [];
@@ -111,6 +144,7 @@ function collectCandidates(vault: string): Candidate[] {
const createdAtMs = created ? Date.parse(created) : fallbackMtimeMs;
const topic = typeof meta["topic"] === "string" ? meta["topic"] : "";
const principle = typeof meta["principle"] === "string" ? meta["principle"] : "";
+ const contextLane = normalizeContextLane(meta["context_lane"]);
// Token budget is computed against the body the pack actually
// emits, not the full file - frontmatter tokens are never
// returned to the caller, so charging them would under-fill
@@ -122,6 +156,7 @@ function collectCandidates(vault: string): Candidate[] {
createdAtMs: Number.isFinite(createdAtMs) ? createdAtMs : 0,
topic,
principle,
+ contextLane,
body,
tokens: estimateTokens(body),
});
@@ -137,6 +172,7 @@ export function packContext(vault: string, opts: ContextPackOptions): ContextPac
tokensUsed: 0,
items: Object.freeze([]),
skipped: Object.freeze([]),
+ ...withOptionalLanes(opts, []),
});
}
const query = opts.query ? normalizeForDedup(opts.query) : null;
@@ -181,6 +217,8 @@ export function packContext(vault: string, opts: ContextPackOptions): ContextPac
tier: c.tier,
tokens,
body,
+ principle: c.principle,
+ contextLane: c.contextLane,
trimmed,
});
used += tokens;
@@ -202,7 +240,11 @@ export function packContext(vault: string, opts: ContextPackOptions): ContextPac
for (const i of keptItems) recomputed += i.tokens;
for (const d of items) {
if (droppedSet.has(d)) {
- skipped.push({ id: d.id, tokens: d.tokens, reason: "over-char-budget" });
+ skipped.push({
+ id: d.id,
+ tokens: d.tokens,
+ reason: "over-char-budget",
+ });
}
}
return Object.freeze({
@@ -210,6 +252,7 @@ export function packContext(vault: string, opts: ContextPackOptions): ContextPac
tokensUsed: recomputed,
items: Object.freeze(keptItems),
skipped: Object.freeze(skipped),
+ ...withOptionalLanes(opts, keptItems),
});
}
}
@@ -219,5 +262,6 @@ export function packContext(vault: string, opts: ContextPackOptions): ContextPac
tokensUsed: used,
items: Object.freeze(items),
skipped: Object.freeze(skipped),
+ ...withOptionalLanes(opts, items),
});
}
diff --git a/src/core/search/evidence-pack.ts b/src/core/search/evidence-pack.ts
new file mode 100644
index 00000000..577e6265
--- /dev/null
+++ b/src/core/search/evidence-pack.ts
@@ -0,0 +1,134 @@
+import type { BrainSearchResult } from "./types.ts";
+
+export interface EvidenceRecord {
+ readonly path: string;
+ readonly documentId: number;
+ readonly chunkId: number;
+ readonly matchedTerms: ReadonlyArray<string>;
+ readonly missingTerms: ReadonlyArray<string>;
+ readonly supportCoverage: number;
+ readonly terminalState: boolean;
+ readonly whyRetrieved: ReadonlyArray<string>;
+ readonly droppedCandidateReasons: ReadonlyArray<string>;
+}
+
+export interface EvidencePack {
+ readonly significantTerms: ReadonlyArray<string>;
+ readonly matchedTerms: ReadonlyArray<string>;
+ readonly missingTerms: ReadonlyArray<string>;
+ readonly supportCoverage: number;
+ readonly records: ReadonlyArray<EvidenceRecord>;
+ readonly droppedCandidates: ReadonlyArray<{
+ readonly path: string;
+ readonly reason: string;
+ }>;
+ readonly abstention: string | null;
+}
+
+const STOPWORDS = new Set([
+ "about",
+ "and",
+ "are",
+ "for",
+ "from",
+ "how",
+ "the",
+ "this",
+ "that",
+ "what",
+ "when",
+ "where",
+ "with",
+]);
+
+const TERMINAL_STATE_RE =
+ /\b(?:archived|closed|deprecated|done|resolved|retired|superseded|terminal)\b/iu;
+
+function significantTerms(query: string): string[] {
+ const terms = new Set<string>();
+ for (const token of query.toLocaleLowerCase().split(/[^\p{L}\p{N}_-]+/u)) {
+ if (token.length >= 3 && !STOPWORDS.has(token)) terms.add(token);
+ }
+ return [...terms];
+}
+
+function includesTerm(result: BrainSearchResult, term: string): boolean {
+ const haystack = `${result.path}\n${result.title ?? ""}\n${result.content}`.toLocaleLowerCase();
+ return haystack.includes(term);
+}
+
+function supportCoverage(
+ matched: ReadonlyArray<string>,
+ significant: ReadonlyArray<string>,
+): number {
+ if (significant.length === 0) return 1;
+ return matched.length / significant.length;
+}
+
+export function evidenceTerminalState(result: BrainSearchResult): boolean {
+ return TERMINAL_STATE_RE.test(`${result.path}\n${result.title ?? ""}\n${result.content}`);
+}
+
+function withTerminalReason(result: BrainSearchResult): BrainSearchResult {
+ if (!evidenceTerminalState(result)) return result;
+ if (result.reasons.some((reason) => reason.startsWith("evidence_terminal_downrank:"))) {
+ return result;
+ }
+ return Object.freeze({
+ ...result,
+ reasons: Object.freeze([...result.reasons, "evidence_terminal_downrank: true"]),
+ });
+}
+
+export function downrankTerminalEvidenceResults(
+ results: ReadonlyArray<BrainSearchResult>,
+): ReadonlyArray<BrainSearchResult> {
+ return results.map(withTerminalReason).toSorted((left, right) => {
+ const leftTerminal = evidenceTerminalState(left);
+ const rightTerminal = evidenceTerminalState(right);
+ if (leftTerminal !== rightTerminal) return leftTerminal ? 1 : -1;
+ if (right.score !== left.score) return right.score - left.score;
+ return left.chunkId - right.chunkId;
+ });
+}
+
+export function buildEvidencePack(
+ query: string,
+ results: ReadonlyArray<BrainSearchResult>,
+): EvidencePack {
+ const significant = Object.freeze(significantTerms(query));
+ const matchedSet = new Set<string>();
+ const records = results.map((result) => {
+ const matched = significant.filter((term) => includesTerm(result, term));
+ for (const term of matched) matchedSet.add(term);
+ const missing = significant.filter((term) => !matched.includes(term));
+ const terminalState = evidenceTerminalState(result);
+ const terminalDownranked = result.reasons.some((reason) =>
+ reason.startsWith("evidence_terminal_downrank:"),
+ );
+ return Object.freeze({
+ path: result.path,
+ documentId: result.documentId,
+ chunkId: result.chunkId,
+ matchedTerms: Object.freeze(matched),
+ missingTerms: Object.freeze(missing),
+ supportCoverage: supportCoverage(matched, significant),
+ terminalState,
+ whyRetrieved: Object.freeze([...result.reasons]),
+ droppedCandidateReasons: Object.freeze(
+ terminalDownranked ? ["terminal_state_downranked"] : [],
+ ),
+ });
+ });
+ const matched = significant.filter((term) => matchedSet.has(term));
+ const missing = significant.filter((term) => !matchedSet.has(term));
+ return Object.freeze({
+ significantTerms: significant,
+ matchedTerms: Object.freeze(matched),
+ missingTerms: Object.freeze(missing),
+ supportCoverage: supportCoverage(matched, significant),
+ records: Object.freeze(records),
+ droppedCandidates: Object.freeze([]),
+ abstention: missing.length > 0 ? `Unsupported significant terms: ${missing.join(", ")}` : null,
+ });
+}
diff --git a/src/core/search/fts-safety.ts b/src/core/search/fts-safety.ts
new file mode 100644
index 00000000..83a6a551
--- /dev/null
+++ b/src/core/search/fts-safety.ts
@@ -0,0 +1,58 @@
+import type { KeywordHit, Store } from "./store.ts";
+import { SearchError } from "./types.ts";
+
+export interface SafeKeywordOutcome {
+ readonly hits: KeywordHit[];
+ readonly warnings: ReadonlyArray<string>;
+}
+
+function errorMessage(e: unknown): string {
+ return e instanceof Error ? e.message : String(e);
+}
+
+function isRebuildableFtsError(e: unknown): boolean {
+ const msg = errorMessage(e);
+ return /SQLITE_CORRUPT|database disk image is malformed|malformed|corrupt/i.test(msg);
+}
+
+function ftsEmptyWhileChunksExist(store: Store): boolean {
+ const counts = store.ftsIntegrityCounts();
+ return counts.chunks > 0 && counts.ftsRows === 0;
+}
+
+function rebuildWarning(cause: string): string {
+ return `rebuilt FTS index after ${cause}; retried keyword search`;
+}
+
+export function keywordTopKWithFtsSafety(
+ store: Store,
+ fts5Query: string,
+ opts: { readonly limit: number; readonly pathPrefix?: string | null },
+): SafeKeywordOutcome {
+ try {
+ const hits = store.keywordTopK(fts5Query, opts);
+ if (hits.length > 0 || !ftsEmptyWhileChunksExist(store)) {
+ return Object.freeze({ hits, warnings: Object.freeze([]) });
+ }
+
+ store.rebuildFtsIndexWithWriterLock();
+ return Object.freeze({
+ hits: store.keywordTopK(fts5Query, opts),
+ warnings: Object.freeze([rebuildWarning("empty chunk_fts with indexed chunks")]),
+ });
+ } catch (e) {
+ if (!isRebuildableFtsError(e)) throw e;
+ try {
+ store.rebuildFtsIndexWithWriterLock();
+ return Object.freeze({
+ hits: store.keywordTopK(fts5Query, opts),
+ warnings: Object.freeze([rebuildWarning("rebuildable FTS error")]),
+ });
+ } catch (retryError) {
+ throw new SearchError(
+ "INDEX_UNREADABLE",
+ `keyword FTS rebuild/retry failed: ${errorMessage(retryError)}`,
+ );
+ }
+ }
+}
diff --git a/src/core/search/fts.ts b/src/core/search/fts.ts
index f2461bf8..1f9959ad 100644
--- a/src/core/search/fts.ts
+++ b/src/core/search/fts.ts
@@ -2,26 +2,35 @@
* FTS5 query construction and keyword retrieval.
*
* The user-typed query is treated as a bag of phrase tokens joined by
- * implicit AND. Each token is wrapped in double quotes so FTS5's
- * operator words (`AND`, `OR`, `NOT`, `NEAR`) and metacharacters
- * (`*`, `(`, `)`, `:`, `^`) lose their special meaning. Internal `"`
- * is escaped as `""` per the FTS5 grammar.
+ * implicit AND. Natural-language connector words that are also uppercase
+ * FTS5 operators (`AND`, `OR`, `NOT`, `NEAR`) are dropped when other
+ * meaningful tokens remain. Each remaining token is wrapped in double
+ * quotes so FTS5 metacharacters (`*`, `(`, `)`, `:`, `^`) lose their
+ * special meaning. Internal `"` is escaped as `""` per the FTS5 grammar.
*
* Anchored in docs/plans/2026-05-16-brain-search-design.md §7.
*/
import type { KeywordHit, Store } from "./store.ts";
import { containsCjk, tokenizeCjkSearchText } from "./cjk-tokenizer.ts";
+import { keywordTopKWithFtsSafety, type SafeKeywordOutcome } from "./fts-safety.ts";
+
+const FTS5_OPERATOR_TOKENS = new Set(["AND", "OR", "NOT", "NEAR"]);
function quoteToken(t: string): string {
return `"${t.replace(/"/g, '""')}"`;
}
+function dropStandaloneOperators(tokens: ReadonlyArray<string>): ReadonlyArray<string> {
+ const withoutOperators = tokens.filter((t) => !FTS5_OPERATOR_TOKENS.has(t));
+ return withoutOperators.length > 0 ? withoutOperators : tokens;
+}
+
export function buildFtsMatch(rawQuery: string): string {
const tokens = containsCjk(rawQuery) ? tokenizeCjkSearchText(rawQuery) : rawQuery.split(/\s+/);
const cleaned = tokens.map((t) => t.trim()).filter((t) => t.length > 0);
if (cleaned.length === 0) return "";
- return cleaned.map(quoteToken).join(" ");
+ return dropStandaloneOperators(cleaned).map(quoteToken).join(" ");
}
/**
@@ -56,11 +65,19 @@ export interface RunFtsOptions {
readonly expandedTerms?: ReadonlyArray<string>;
}
-export function runFtsQuery(store: Store, rawQuery: string, opts: RunFtsOptions): KeywordHit[] {
+export function runFtsQueryDetailed(
+ store: Store,
+ rawQuery: string,
+ opts: RunFtsOptions,
+): SafeKeywordOutcome {
const match =
opts.expandedTerms && opts.expandedTerms.length > 0
? buildExpandedFtsMatch(rawQuery, opts.expandedTerms)
: buildFtsMatch(rawQuery);
- if (match === "") return [];
- return store.keywordTopK(match, opts);
+ if (match === "") return Object.freeze({ hits: [], warnings: Object.freeze([]) });
+ return keywordTopKWithFtsSafety(store, match, opts);
+}
+
+export function runFtsQuery(store: Store, rawQuery: string, opts: RunFtsOptions): KeywordHit[] {
+ return runFtsQueryDetailed(store, rawQuery, opts).hits;
}
diff --git a/src/core/search/index.ts b/src/core/search/index.ts
index e22814fe..0a311a63 100644
--- a/src/core/search/index.ts
+++ b/src/core/search/index.ts
@@ -39,6 +39,21 @@ export type {
VaultIgnoreRule,
} from "./types.ts";
export { SearchError, SEARCH_ERROR_CODES } from "./types.ts";
+export {
+ parseStructuredRecallQueryDocument,
+ structuredRecallQueryText,
+ type StructuredRecallQueryDocument,
+} from "./structured-query.ts";
+export {
+ clearSessionFocus,
+ normalizeSessionFocus,
+ readSessionFocus,
+ sessionFocusIsActive,
+ writeSessionFocus,
+ type SearchSessionFocus,
+} from "./session-focus.ts";
+export { evaluateSurfacingGate, type SurfacingGateDecision } from "./surfacing-gate.ts";
+export { buildEvidencePack, type EvidencePack } from "./evidence-pack.ts";
export { resolveIndexPath } from "./paths.ts";
export {
@@ -145,11 +160,19 @@ function validateResolvedConfig(config: ResolvedSearchConfig): void {
);
}
if (config.semantic.dimension !== null) {
- validateIntegerRange(config.semantic.dimension, "embedding_dimension", { min: 1 });
+ validateIntegerRange(config.semantic.dimension, "embedding_dimension", {
+ min: 1,
+ });
}
- validateIntegerRange(config.semantic.timeoutMs, "embedding_timeout_ms", { min: 1 });
- validateIntegerRange(config.semantic.concurrency, "embedding_concurrency", { min: 1 });
- validateIntegerRange(config.semantic.batchSize, "embedding_batch_size", { min: 1 });
+ validateIntegerRange(config.semantic.timeoutMs, "embedding_timeout_ms", {
+ min: 1,
+ });
+ validateIntegerRange(config.semantic.concurrency, "embedding_concurrency", {
+ min: 1,
+ });
+ validateIntegerRange(config.semantic.batchSize, "embedding_batch_size", {
+ min: 1,
+ });
}
function parseProvider(raw: string | null): ResolvedEmbeddingConfig["provider"] {
diff --git a/src/core/search/query-cache.ts b/src/core/search/query-cache.ts
index 91ead65f..7b7b7899 100644
--- a/src/core/search/query-cache.ts
+++ b/src/core/search/query-cache.ts
@@ -38,6 +38,29 @@ function canonicalProperties(
.toSorted((a, b) => (a[0] < b[0] ? -1 : a[0] > b[0] ? 1 : 0));
}
+function canonicalStructuredQuery(opts: SearchOptions): unknown {
+ const structured = opts.structuredQuery;
+ if (!structured) return null;
+ return {
+ intent: structured.intent,
+ lexInclude: [...structured.lex.include],
+ lexExclude: [...structured.lex.exclude],
+ vec: [...structured.vec],
+ hyde: [...structured.hyde],
+ };
+}
+
+function canonicalSessionFocus(opts: SearchOptions): unknown {
+ const focus = opts.sessionFocus;
+ if (focus === undefined) return "persisted";
+ if (focus === null) return null;
+ return {
+ query: focus.query,
+ pathPrefix: focus.pathPrefix,
+ expiresAt: focus.expiresAt,
+ };
+}
+
/**
* Build a stable cache key from the result-affecting request. Every
* option that can change the result set is folded in, in a canonical
@@ -59,8 +82,11 @@ export function buildCacheKey(
semanticWeight: opts.semanticWeight ?? null,
mmrLambda: opts.mmrLambda ?? null,
maxHops: opts.maxHops ?? null,
+ evidencePack: opts.evidencePack === true,
properties: canonicalProperties(opts.properties),
visibility: opts.visibility ? [...opts.visibility].toSorted() : null,
+ structuredQuery: canonicalStructuredQuery(opts),
+ sessionFocus: canonicalSessionFocus(opts),
plan: planHash,
cfg: configFingerprint,
});
diff --git a/src/core/search/query-plan.ts b/src/core/search/query-plan.ts
index 14a0c3dc..e35c523f 100644
--- a/src/core/search/query-plan.ts
+++ b/src/core/search/query-plan.ts
@@ -35,11 +35,26 @@ export const NEUTRAL_PROFILE: WeightProfile = Object.freeze({
const PROFILES: Record<QueryIntent, WeightProfile> = Object.freeze({
neutral: NEUTRAL_PROFILE,
// Literal lookup: trust the keyword/FTS layer, discount fuzzy semantic.
- exact: Object.freeze({ keywordMul: 1.3, semanticMul: 0.7, entityMul: 1, recencyMul: 1 }),
+ exact: Object.freeze({
+ keywordMul: 1.3,
+ semanticMul: 0.7,
+ entityMul: 1,
+ recencyMul: 1,
+ }),
// Proper-noun lookup: amplify the entity layer, nudge keyword.
- entity: Object.freeze({ keywordMul: 1.15, semanticMul: 0.9, entityMul: 1.4, recencyMul: 1 }),
+ entity: Object.freeze({
+ keywordMul: 1.15,
+ semanticMul: 0.9,
+ entityMul: 1.4,
+ recencyMul: 1,
+ }),
// Open-ended exploration: lean on semantic similarity and recency.
- broad: Object.freeze({ keywordMul: 0.9, semanticMul: 1.2, entityMul: 1, recencyMul: 1.1 }),
+ broad: Object.freeze({
+ keywordMul: 0.9,
+ semanticMul: 1.2,
+ entityMul: 1,
+ recencyMul: 1.1,
+ }),
});
const QUOTED_PHRASE_RE = /"[^"\n]{2,}"/u;
@@ -98,9 +113,10 @@ function classify(query: string, normalized: string): QueryIntent {
export function buildQueryPlan(
query: string,
expandedTerms: ReadonlyArray<string> = [],
+ intentOverride?: QueryIntent | null,
): QueryPlan {
const normalized = normalize(query);
- const intent = classify(query, normalized);
+ const intent = intentOverride ?? classify(query, normalized);
const terms = Object.freeze([...expandedTerms]);
const planHash = fnv1a(`${normalized}|${intent}|${terms.join(",")}`);
return Object.freeze({
diff --git a/src/core/search/ranker.ts b/src/core/search/ranker.ts
index 0dc0a713..b1238013 100644
--- a/src/core/search/ranker.ts
+++ b/src/core/search/ranker.ts
@@ -11,6 +11,7 @@
import { PAGE_TIER_DEFAULT, tierWeight, type PageTier } from "../brain/page-meta/tier.ts";
import { weibullDecay, DEFAULT_RECENCY, type WeibullRecencyOptions } from "./recency.ts";
+import { scoreSessionFocusTarget, type SearchSessionFocus } from "./session-focus.ts";
import type { KeywordHit, SemanticHit, HydratedChunk } from "./store.ts";
import type { BrainSearchResult, WeightProfile } from "./types.ts";
@@ -58,6 +59,7 @@ export interface RankerOptions {
* weight, so ranking is bit-identical to pre-intent behaviour.
*/
readonly weightProfile?: WeightProfile;
+ readonly sessionFocus?: SearchSessionFocus | null;
}
const DAY_MS = 24 * 60 * 60 * 1000;
@@ -128,6 +130,7 @@ function buildReasons(parts: {
recency: number;
tierMul: number;
entityBoost?: number;
+ sessionFocus?: number;
}): ReadonlyArray<string> {
const reasons: string[] = [];
if (parts.keywordScore > 0) reasons.push(`fts5_bm25: ${fmt(parts.keywordScore)}`);
@@ -138,6 +141,9 @@ function buildReasons(parts: {
if (parts.linkBoost > 0) reasons.push(`link_boost: ${fmt(parts.linkBoost)}`);
if (parts.recency > 0) reasons.push(`recency: ${fmt(parts.recency)}`);
if (parts.tierMul !== 1) reasons.push(`tier: ${fmt(parts.tierMul)}`);
+ if (parts.sessionFocus && parts.sessionFocus !== 0) {
+ reasons.push(`session_focus: ${parts.sessionFocus >= 0 ? "+" : ""}${fmt(parts.sessionFocus)}`);
+ }
return Object.freeze(reasons);
}
@@ -255,7 +261,8 @@ export function rankResults(inputs: RankerInputs, opts: RankerOptions): BrainSea
// relevant set - never enough to float an irrelevant chunk.
const entityMatches = inputs.entityMatchByChunk?.get(c.chunkId) ?? 0;
const entityBoost = Math.min(0.04, entityMatches * 0.02 * entMul);
- const score = clamp01(weighted * tierMul + linkBoost + recency + entityBoost);
+ const sessionFocus = scoreSessionFocusTarget(hyd, opts.sessionFocus, nowMs);
+ const score = clamp01(weighted * tierMul + linkBoost + recency + entityBoost + sessionFocus);
ranked.push(
Object.freeze({
@@ -279,6 +286,7 @@ export function rankResults(inputs: RankerInputs, opts: RankerOptions): BrainSea
recency,
tierMul,
entityBoost,
+ sessionFocus,
}),
}),
);
diff --git a/src/core/search/search.ts b/src/core/search/search.ts
index 8385b6d3..00d9c632 100644
--- a/src/core/search/search.ts
+++ b/src/core/search/search.ts
@@ -15,13 +15,15 @@ import { parseFrontmatter } from "../vault.ts";
import { isVisible, normalizeVisibilityScope, pageVisibility } from "../graph/visibility.ts";
import { makeProvider } from "./embeddings/provider.ts";
import { extractEntities } from "./entities.ts";
-import { runFtsQuery } from "./fts.ts";
+import { buildEvidencePack, downrankTerminalEvidenceResults } from "./evidence-pack.ts";
+import { runFtsQueryDetailed } from "./fts.ts";
import { mmrRerank } from "./mmr.ts";
import { buildQueryPlan } from "./query-plan.ts";
import { buildCacheKey, getCachedOutcome, putCachedOutcome } from "./query-cache.ts";
import { deriveExpansionTerms, tokenizeForExpansion, DEFAULT_EXPANSION } from "./synonyms.ts";
import { filterByProperties } from "./property-filter.ts";
import { rankResults } from "./ranker.ts";
+import { readSessionFocus } from "./session-focus.ts";
import { expandByTraversal, type TraversalOptions } from "./traversal.ts";
import { Store } from "./store.ts";
import { SearchError } from "./types.ts";
@@ -31,6 +33,7 @@ import type {
SearchOptions,
SearchOutcome,
} from "./types.ts";
+import type { StructuredRecallQueryDocument } from "./structured-query.ts";
interface SemanticPolicy {
/** caller asked for semantic on or off (true), or accepted the default (false). */
@@ -82,6 +85,62 @@ function assertSafePathPrefix(prefix: string | undefined): string | undefined {
return prefix;
}
+function structuredKeywordQuery(
+ query: string,
+ structured: StructuredRecallQueryDocument | undefined,
+): string {
+ if (!structured || structured.lex.include.length === 0) return query;
+ return structured.lex.include.join(" ");
+}
+
+function structuredSemanticQuery(
+ structured: StructuredRecallQueryDocument | undefined,
+): string | null {
+ if (!structured) return null;
+ const text = [...structured.vec, ...structured.hyde].join("\n\n").trim();
+ return text.length > 0 ? text : null;
+}
+
+function includesFolded(haystack: string, needle: string): boolean {
+ return haystack.toLocaleLowerCase().includes(needle.toLocaleLowerCase());
+}
+
+function applyStructuredExclusions(
+ results: ReadonlyArray<BrainSearchResult>,
+ structured: StructuredRecallQueryDocument | undefined,
+): ReadonlyArray<BrainSearchResult> {
+ if (!structured || structured.lex.exclude.length === 0) return results;
+ return results.filter((result) => {
+ const haystack = `${result.path}\n${result.title ?? ""}\n${result.content}`;
+ return !structured.lex.exclude.some((term) => includesFolded(haystack, term));
+ });
+}
+
+function addStructuredReasons(
+ results: ReadonlyArray<BrainSearchResult>,
+ structured: StructuredRecallQueryDocument | undefined,
+): ReadonlyArray<BrainSearchResult> {
+ if (!structured) return results;
+ return results.map((result) => {
+ const additions: string[] = [];
+ if (structured.lex.include.length > 0 && result.keywordScore > 0) {
+ additions.push(`lane:lex/fts5 ${result.keywordScore.toFixed(3)}`);
+ }
+ if (structured.vec.length > 0 && result.semanticScore > 0) {
+ additions.push(`lane:vec/semantic ${result.semanticScore.toFixed(3)}`);
+ }
+ if (structured.hyde.length > 0 && result.semanticScore > 0) {
+ additions.push(`lane:hyde/semantic ${result.semanticScore.toFixed(3)}`);
+ }
+ if (structured.intent !== null) additions.push(`intent:${structured.intent}`);
+ if (additions.length === 0) return result;
+ return Object.freeze({
+ ...result,
+ reasons: Object.freeze([...result.reasons, ...additions]),
+ });
+ });
+}
+
export async function search(
config: ResolvedSearchConfig,
opts: SearchOptions,
@@ -94,11 +153,16 @@ export async function search(
const pathPrefix = assertSafePathPrefix(opts.pathPrefix);
const policy = resolveSemanticPolicy(config, opts);
const warnings: string[] = [];
+ const structured = opts.structuredQuery;
+ const sessionFocus =
+ opts.sessionFocus === undefined ? readSessionFocus(config, Date.now()) : opts.sessionFocus;
+ const keywordQuery = structuredKeywordQuery(query, structured);
+ const semanticLaneQuery = structuredSemanticQuery(structured);
// Query plan (v0.20.0): one structural pass yields the intent weight
// profile and the cache key. Pure; no I/O. Expanded terms (if any) are
// folded in once they have been derived from the store below.
- const basePlan = buildQueryPlan(query);
+ const basePlan = buildQueryPlan(keywordQuery, [], structured?.intent);
const store = await Store.open(config, { mode: "read" });
try {
@@ -121,7 +185,13 @@ export async function search(
// semantic decision) so equivalent calls share a cache entry.
try {
generation = store.corpusGeneration();
- const keyOpts = { ...opts, limit, semantic: policy.wantSemantic, keywordOnly: false };
+ const keyOpts = {
+ ...opts,
+ limit,
+ semantic: policy.wantSemantic,
+ keywordOnly: false,
+ sessionFocus,
+ };
cacheKey = buildCacheKey(keyOpts, basePlan.planHash, configFingerprint(config));
const hit = getCachedOutcome(store, cacheKey, generation, ttlMs, Date.now());
if (hit) return hit;
@@ -142,10 +212,12 @@ export async function search(
};
// Keyword candidates.
- let kwHits = runFtsQuery(store, query, {
+ let kwOutcome = runFtsQueryDetailed(store, keywordQuery, {
limit: limit * 3,
pathPrefix,
});
+ let kwHits = kwOutcome.hits;
+ for (const w of kwOutcome.warnings) warnings.push(w);
// Synonym / query expansion (v0.20.0): opt-in and never for an
// exact-intent (quoted/wildcard) query. Derive related terms from
@@ -156,16 +228,20 @@ export async function search(
if (config.recall.synonymEnabled && basePlan.intent !== "exact" && kwHits.length > 0) {
const topIds = kwHits.slice(0, 10).map((h) => h.chunkId);
const ctx = store.hydrateChunks(topIds);
- const texts = topIds
- .map((id) => ctx.get(id)?.content ?? "")
- .filter((t) => t.length > 0);
+ const texts = topIds.map((id) => ctx.get(id)?.content ?? "").filter((t) => t.length > 0);
const expandedTerms = deriveExpansionTerms(tokenizeForExpansion(query), texts, {
...DEFAULT_EXPANSION,
maxTerms: config.recall.synonymMaxTerms,
});
if (expandedTerms.length > 0) {
- plan = buildQueryPlan(query, expandedTerms);
- kwHits = runFtsQuery(store, query, { limit: limit * 3, pathPrefix, expandedTerms });
+ plan = buildQueryPlan(keywordQuery, expandedTerms, structured?.intent);
+ kwOutcome = runFtsQueryDetailed(store, keywordQuery, {
+ limit: limit * 3,
+ pathPrefix,
+ expandedTerms,
+ });
+ kwHits = kwOutcome.hits;
+ for (const w of kwOutcome.warnings) warnings.push(w);
}
}
const weightProfile = config.recall.intentEnabled ? plan.weightProfile : undefined;
@@ -173,8 +249,11 @@ export async function search(
// Semantic candidates (may be skipped).
let semHits: ReturnType<Store["semanticTopK"]> = [];
let semanticAttempted = false;
+ if (semanticLaneQuery !== null && !policy.wantSemantic) {
+ warnings.push("semantic structured lanes skipped: semantic search is disabled");
+ }
if (policy.wantSemantic) {
- const semOutcome = await runSemanticPhase(store, config, query, {
+ const semOutcome = await runSemanticPhase(store, config, semanticLaneQuery ?? query, {
limit: Math.max(limit * 5, 50),
pathPrefix,
explicit: policy.explicit,
@@ -190,11 +269,13 @@ export async function search(
for (const h of semHits) allChunkIds.add(h.chunkId);
const idsList = Array.from(allChunkIds);
if (idsList.length === 0) {
+ const evidencePack = opts.evidencePack === true ? buildEvidencePack(query, []) : undefined;
return finalize(
Object.freeze({
results: Object.freeze([] as ReadonlyArray<BrainSearchResult>),
warnings: Object.freeze(warnings),
total: 0,
+ ...(evidencePack !== undefined ? { evidencePack } : {}),
}),
);
}
@@ -232,11 +313,13 @@ export async function search(
// narrow pool lets a high-parent expansion crowd a genuine but
// lower-ranked hit out of the final window). When both are disabled
// the pool collapses back to the historical rankLimit.
+ const hasStructuredExclusions = (structured?.lex.exclude.length ?? 0) > 0;
const mmrLambda = opts.mmrLambda ?? config.recall.mmrLambda;
const mmrActive = mmrLambda < 1;
const maxHops = opts.maxHops ?? config.recall.maxHops;
const traversalActive = maxHops > 0;
- const baseRankLimit = hasFrontmatterFilter ? Math.max(limit * 5, 50) : limit;
+ const baseRankLimit =
+ hasFrontmatterFilter || hasStructuredExclusions ? Math.max(limit * 5, 50) : limit;
const rankLimit =
mmrActive || traversalActive ? Math.max(baseRankLimit, limit * 3, 30) : baseRankLimit;
@@ -246,7 +329,11 @@ export async function search(
// caller can tell "the pool ran out" from "the cap truncated more").
const assemble = (
rankCap: number,
- ): { preVisibility: number; visible: ReadonlyArray<BrainSearchResult>; capHit: boolean } => {
+ ): {
+ preVisibility: number;
+ visible: ReadonlyArray<BrainSearchResult>;
+ capHit: boolean;
+ } => {
let ranked = rankResults(
{
keyword: kwHits,
@@ -267,6 +354,7 @@ export async function search(
amplitude: config.recall.recencyAmplitude,
},
...(weightProfile !== undefined ? { weightProfile } : {}),
+ ...(sessionFocus !== undefined ? { sessionFocus } : {}),
},
);
const capHit = ranked.length >= rankCap;
@@ -314,7 +402,7 @@ export async function search(
const wideCap = Math.max(limit * 5, 50, limit * 3, 30);
if (wideCap > rankLimit) assembled = assemble(wideCap);
}
- const filtered = assembled.visible.slice(0, limit);
+ const filtered = applyStructuredExclusions(assembled.visible, structured).slice(0, limit);
// Typed graph semantics (v3): surface the typed relations each
// result page declares in its frontmatter. Computed here from the
@@ -324,12 +412,20 @@ export async function search(
const rels = relByDoc.get(r.documentId);
return rels && rels.length > 0 ? { ...r, relations: Object.freeze(rels) } : r;
});
+ const withStructuredReasons = addStructuredReasons(withRelations, structured);
+ const finalResults =
+ opts.evidencePack === true
+ ? downrankTerminalEvidenceResults(withStructuredReasons)
+ : withStructuredReasons;
+ const evidencePack =
+ opts.evidencePack === true ? buildEvidencePack(query, finalResults) : undefined;
return finalize(
Object.freeze({
- results: Object.freeze(withRelations),
+ results: Object.freeze(finalResults),
warnings: Object.freeze(warnings),
- total: withRelations.length,
+ total: finalResults.length,
+ ...(evidencePack !== undefined ? { evidencePack } : {}),
}),
);
} finally {
diff --git a/src/core/search/session-focus.ts b/src/core/search/session-focus.ts
new file mode 100644
index 00000000..019dfb40
--- /dev/null
+++ b/src/core/search/session-focus.ts
@@ -0,0 +1,146 @@
+import { existsSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs";
+import { dirname, join } from "node:path";
+
+import { resolveIndexPath } from "./paths.ts";
+import { SearchError, type ResolvedSearchConfig } from "./types.ts";
+
+export interface SearchSessionFocus {
+ readonly query: string | null;
+ readonly pathPrefix: string | null;
+ readonly expiresAt: number | null;
+}
+
+export interface SessionFocusInput {
+ readonly query?: string | null;
+ readonly pathPrefix?: string | null;
+ readonly ttlMinutes?: number | null;
+}
+
+export interface SessionFocusTarget {
+ readonly path: string;
+ readonly title: string | null;
+ readonly content: string;
+}
+
+const DEFAULT_TTL_MINUTES = 120;
+const MAX_TTL_MINUTES = 24 * 60;
+const FOCUS_FILE = "search-focus.json";
+
+function normalizePathPrefix(pathPrefix: string | null | undefined): string | null {
+ if (pathPrefix === undefined || pathPrefix === null) return null;
+ const normalized = pathPrefix.trim().replaceAll("\\", "/").replace(/^\.\//u, "");
+ if (normalized.length === 0) return null;
+ if (normalized.startsWith("/") || normalized.split("/").includes("..")) {
+ throw new SearchError("INVALID_INPUT", "session focus path prefix must stay inside the vault");
+ }
+ return normalized;
+}
+
+function normalizeQuery(query: string | null | undefined): string | null {
+ if (query === undefined || query === null) return null;
+ const normalized = query.trim().replace(/\s+/gu, " ");
+ return normalized.length > 0 ? normalized : null;
+}
+
+function normalizeTtlMinutes(raw: number | null | undefined): number {
+ const ttl = raw ?? DEFAULT_TTL_MINUTES;
+ if (!Number.isFinite(ttl) || !Number.isInteger(ttl) || ttl < 1 || ttl > MAX_TTL_MINUTES) {
+ throw new SearchError(
+ "INVALID_INPUT",
+ `session focus ttl must be an integer in 1..${MAX_TTL_MINUTES}`,
+ );
+ }
+ return ttl;
+}
+
+export function normalizeSessionFocus(
+ input: SessionFocusInput,
+ nowMs = Date.now(),
+): SearchSessionFocus {
+ const query = normalizeQuery(input.query);
+ const pathPrefix = normalizePathPrefix(input.pathPrefix);
+ if (query === null && pathPrefix === null) {
+ throw new SearchError("INVALID_INPUT", "session focus requires a query or path prefix");
+ }
+ const ttlMinutes = normalizeTtlMinutes(input.ttlMinutes);
+ return Object.freeze({
+ query,
+ pathPrefix,
+ expiresAt: nowMs + ttlMinutes * 60 * 1000,
+ });
+}
+
+export function sessionFocusIsActive(
+ focus: SearchSessionFocus | null | undefined,
+ nowMs = Date.now(),
+): boolean {
+ if (!focus) return false;
+ return focus.expiresAt === null || focus.expiresAt > nowMs;
+}
+
+export function sessionFocusPath(config: ResolvedSearchConfig): string {
+ return join(dirname(resolveIndexPath(config.vault, config.dbPath)), FOCUS_FILE);
+}
+
+export function readSessionFocus(
+ config: ResolvedSearchConfig,
+ nowMs = Date.now(),
+): SearchSessionFocus | null {
+ const path = sessionFocusPath(config);
+ if (!existsSync(path)) return null;
+ try {
+ const parsed = JSON.parse(readFileSync(path, "utf8")) as SearchSessionFocus;
+ const query = normalizeQuery(typeof parsed.query === "string" ? parsed.query : null);
+ const pathPrefix = normalizePathPrefix(
+ typeof parsed.pathPrefix === "string" ? parsed.pathPrefix : null,
+ );
+ const expiresAt = typeof parsed.expiresAt === "number" ? parsed.expiresAt : null;
+ if (expiresAt === null || (query === null && pathPrefix === null)) return null;
+ const focus = Object.freeze({ query, pathPrefix, expiresAt });
+ return sessionFocusIsActive(focus, nowMs) ? focus : null;
+ } catch {
+ return null;
+ }
+}
+
+export function writeSessionFocus(config: ResolvedSearchConfig, focus: SearchSessionFocus): void {
+ const path = sessionFocusPath(config);
+ mkdirSync(dirname(path), { recursive: true });
+ writeFileSync(path, JSON.stringify(focus, null, 2) + "\n");
+}
+
+export function clearSessionFocus(config: ResolvedSearchConfig): void {
+ const path = sessionFocusPath(config);
+ if (existsSync(path)) rmSync(path, { force: true });
+}
+
+function focusTokens(query: string): string[] {
+ const tokens = new Set<string>();
+ for (const token of query.toLocaleLowerCase().split(/[^\p{L}\p{N}_-]+/u)) {
+ if (token.length >= 2) tokens.add(token);
+ }
+ return [...tokens];
+}
+
+function clampContribution(value: number): number {
+ if (value > 0.08) return 0.08;
+ if (value < -0.03) return -0.03;
+ return value;
+}
+
+export function scoreSessionFocusTarget(
+ target: SessionFocusTarget,
+ focus: SearchSessionFocus | null | undefined,
+ nowMs = Date.now(),
+): number {
+ if (!sessionFocusIsActive(focus, nowMs)) return 0;
+ let score = 0;
+ if (focus!.pathPrefix !== null)
+ score += target.path.startsWith(focus!.pathPrefix) ? 0.06 : -0.015;
+ if (focus!.query !== null) {
+ const haystack = `${target.path}\n${target.title ?? ""}\n${target.content}`.toLocaleLowerCase();
+ const matches = focusTokens(focus!.query).filter((token) => haystack.includes(token)).length;
+ score += matches > 0 ? Math.min(0.04, matches * 0.02) : -0.01;
+ }
+ return clampContribution(score);
+}
diff --git a/src/core/search/store.ts b/src/core/search/store.ts
index e8660389..53610e23 100644
--- a/src/core/search/store.ts
+++ b/src/core/search/store.ts
@@ -189,6 +189,24 @@ function nowIso(): string {
return new Date().toISOString();
}
+function acquireWriterLockSync(path: string): () => void {
+ const maxAttempts = 10;
+ const sleepMs = 50;
+ let lastError: unknown;
+ for (let attempt = 0; attempt < maxAttempts; attempt++) {
+ try {
+ return lockfile.lockSync(path, { stale: 60_000, realpath: false });
+ } catch (err) {
+ const code = (err as NodeJS.ErrnoException).code;
+ if (code !== "ELOCKED") throw err;
+ lastError = err;
+ if (attempt < maxAttempts - 1) Bun.sleepSync(sleepMs);
+ }
+ }
+ const msg = lastError instanceof Error ? lastError.message : String(lastError);
+ throw new SearchError("INDEX_LOCKED", `another writer holds the search index lock: ${msg}`);
+}
+
function vecToBuffer(values: ReadonlyArray<number> | Float32Array): Buffer {
const arr = values instanceof Float32Array ? values : Float32Array.from(values);
return Buffer.from(arr.buffer, arr.byteOffset, arr.byteLength);
@@ -878,6 +896,31 @@ export class Store {
// ── search ─────────────────────────────────────────────────────────────────
+ ftsIntegrityCounts(): { readonly chunks: number; readonly ftsRows: number } {
+ const chunks =
+ this.db.query<{ c: number }, []>("SELECT count(*) AS c FROM chunks").get()?.c ?? 0;
+ const ftsRows =
+ this.db.query<{ c: number }, []>("SELECT count(*) AS c FROM chunk_fts_docsize").get()?.c ?? 0;
+ return Object.freeze({ chunks, ftsRows });
+ }
+
+ rebuildFtsIndex(): void {
+ this.db.run("INSERT INTO chunk_fts(chunk_fts) VALUES('rebuild')");
+ }
+
+ rebuildFtsIndexWithWriterLock(): void {
+ if (this.release) {
+ this.rebuildFtsIndex();
+ return;
+ }
+ const release = acquireWriterLockSync(this.config.dbPath);
+ try {
+ this.rebuildFtsIndex();
+ } finally {
+ release();
+ }
+ }
+
/**
* Top-K BM25 keyword hits. `fts5Query` is an already-escaped FTS5
* MATCH expression; building it is fts.ts's job.
diff --git a/src/core/search/structured-query.ts b/src/core/search/structured-query.ts
new file mode 100644
index 00000000..d3042823
--- /dev/null
+++ b/src/core/search/structured-query.ts
@@ -0,0 +1,147 @@
+import { SearchError, type QueryIntent } from "./types.ts";
+
+export interface StructuredLexLane {
+ readonly include: ReadonlyArray<string>;
+ readonly exclude: ReadonlyArray<string>;
+}
+
+export interface StructuredRecallQueryDocument {
+ readonly intent: QueryIntent | null;
+ readonly lex: StructuredLexLane;
+ readonly vec: ReadonlyArray<string>;
+ readonly hyde: ReadonlyArray<string>;
+}
+
+const ALLOWED_INTENTS = new Set<QueryIntent>(["neutral", "exact", "entity", "broad"]);
+const ALLOWED_LANES = new Set(["intent", "lex", "vec", "hyde"]);
+
+function invalid(lineNumber: number, message: string): SearchError {
+ return new SearchError("INVALID_INPUT", `structured query line ${lineNumber}: ${message}`);
+}
+
+function readQuoted(
+ value: string,
+ lineNumber: number,
+ start: number,
+): { text: string; next: number } {
+ let charIndex = start + 1;
+ let text = "";
+ while (charIndex < value.length) {
+ const char = value[charIndex]!;
+ if (char === '"') return { text, next: charIndex + 1 };
+ text += char;
+ charIndex++;
+ }
+ throw invalid(lineNumber, "unterminated quoted lex token");
+}
+
+function readBare(value: string, start: number): { text: string; next: number } {
+ let charIndex = start;
+ while (charIndex < value.length && !/\s/u.test(value[charIndex]!)) charIndex++;
+ return { text: value.slice(start, charIndex), next: charIndex };
+}
+
+function parseLexLane(value: string, lineNumber: number): StructuredLexLane {
+ const include: string[] = [];
+ const exclude: string[] = [];
+ let charIndex = 0;
+
+ while (charIndex < value.length) {
+ while (charIndex < value.length && /\s/u.test(value[charIndex]!)) charIndex++;
+ if (charIndex >= value.length) break;
+
+ let negated = false;
+ if (value[charIndex] === "-") {
+ negated = true;
+ charIndex++;
+ if (charIndex >= value.length || /\s/u.test(value[charIndex]!)) {
+ throw invalid(lineNumber, "negation must be followed by a lex token");
+ }
+ }
+
+ const token =
+ value[charIndex] === '"'
+ ? readQuoted(value, lineNumber, charIndex)
+ : readBare(value, charIndex);
+ const text = token.text.trim();
+ if (text.length === 0) throw invalid(lineNumber, "empty lex token");
+ if (negated) exclude.push(text);
+ else include.push(text);
+ charIndex = token.next;
+ }
+
+ if (include.length === 0 && exclude.length === 0) {
+ throw invalid(lineNumber, "lex lane must not be empty");
+ }
+ return Object.freeze({
+ include: Object.freeze(include),
+ exclude: Object.freeze(exclude),
+ });
+}
+
+export function parseStructuredRecallQueryDocument(
+ document: string,
+): StructuredRecallQueryDocument {
+ let intent: QueryIntent | null = null;
+ const lexInclude: string[] = [];
+ const lexExclude: string[] = [];
+ const vec: string[] = [];
+ const hyde: string[] = [];
+
+ const lines = document.split(/\r?\n/u);
+ for (let lineIndex = 0; lineIndex < lines.length; lineIndex++) {
+ const lineNumber = lineIndex + 1;
+ const line = lines[lineIndex]!.trim();
+ if (line.length === 0 || line.startsWith("#")) continue;
+
+ const separator = line.indexOf(":");
+ if (separator <= 0) throw invalid(lineNumber, "expected '<lane>: <value>'");
+ const lane = line.slice(0, separator).trim();
+ const value = line.slice(separator + 1).trim();
+ if (!ALLOWED_LANES.has(lane)) throw invalid(lineNumber, `unknown lane '${lane}'`);
+ if (value.length === 0) throw invalid(lineNumber, `${lane} lane must not be empty`);
+
+ if (lane === "intent") {
+ if (intent !== null) throw invalid(lineNumber, "intent lane must not be repeated");
+ if (!ALLOWED_INTENTS.has(value as QueryIntent)) {
+ throw invalid(lineNumber, `intent must be neutral, exact, entity, or broad`);
+ }
+ intent = value as QueryIntent;
+ } else if (lane === "lex") {
+ const parsed = parseLexLane(value, lineNumber);
+ lexInclude.push(...parsed.include);
+ lexExclude.push(...parsed.exclude);
+ } else if (lane === "vec") {
+ vec.push(value);
+ } else {
+ hyde.push(value);
+ }
+ }
+
+ if (
+ intent === null &&
+ lexInclude.length === 0 &&
+ lexExclude.length === 0 &&
+ vec.length === 0 &&
+ hyde.length === 0
+ ) {
+ throw new SearchError(
+ "INVALID_INPUT",
+ "structured query document must contain at least one lane",
+ );
+ }
+
+ return Object.freeze({
+ intent,
+ lex: Object.freeze({
+ include: Object.freeze(lexInclude),
+ exclude: Object.freeze(lexExclude),
+ }),
+ vec: Object.freeze(vec),
+ hyde: Object.freeze(hyde),
+ });
+}
+
+export function structuredRecallQueryText(document: StructuredRecallQueryDocument): string {
+ return [...document.lex.include, ...document.vec, ...document.hyde].join(" ").trim();
+}
diff --git a/src/core/search/surfacing-gate.ts b/src/core/search/surfacing-gate.ts
new file mode 100644
index 00000000..74c16996
--- /dev/null
+++ b/src/core/search/surfacing-gate.ts
@@ -0,0 +1,109 @@
+export type SurfacingGateReason =
+ | "explicit"
+ | "memory_question"
+ | "uncertain_question"
+ | "duplicate"
+ | "empty"
+ | "greeting"
+ | "slash_command"
+ | "shell_command"
+ | "no_recall_intent";
+
+export interface SurfacingGateInput {
+ readonly prompt: string;
+ readonly previousPrompt?: string | null;
+ readonly explicit?: boolean;
+}
+
+export interface SurfacingGateDecision {
+ readonly retrieve: boolean;
+ readonly reason: SurfacingGateReason;
+}
+
+const GREETINGS = new Set([
+ "hello",
+ "hi",
+ "hey",
+ "привет",
+ "здравствуй",
+ "здравствуйте",
+ "добрый день",
+]);
+
+const SHELL_COMMANDS = new Set([
+ "awk",
+ "bun",
+ "cat",
+ "cd",
+ "chmod",
+ "cp",
+ "curl",
+ "find",
+ "git",
+ "grep",
+ "ls",
+ "mkdir",
+ "mv",
+ "npm",
+ "pnpm",
+ "python",
+ "rg",
+ "rm",
+ "sed",
+ "touch",
+ "yarn",
+]);
+
+const MEMORY_PATTERNS = [
+ /\b(remember|recall|memory|context|notes?|decid(?:e|ed)|decision|search|find)\b/iu,
+ /\b(what did we|where did we|when did we|have we|did we)\b/iu,
+ /\b(помнишь|вспомни|найди|контекст|решили|обсуждали|заметки)\b/iu,
+];
+
+function normalizePrompt(prompt: string): string {
+ return prompt.trim().replace(/\s+/gu, " ").toLocaleLowerCase();
+}
+
+function isGreeting(normalized: string): boolean {
+ const stripped = normalized.replace(/[!.?]+$/u, "");
+ return GREETINGS.has(stripped);
+}
+
+function isSlashCommand(normalized: string): boolean {
+ return normalized.startsWith("/") && !normalized.includes(" ");
+}
+
+function isShellOnlyPrompt(normalized: string): boolean {
+ if (normalized.includes("?") || normalized.includes("\n")) return false;
+ const firstToken = normalized.replace(/^\$\s*/u, "").split(/\s+/u)[0] ?? "";
+ return SHELL_COMMANDS.has(firstToken);
+}
+
+function hasMemoryIntent(normalized: string): boolean {
+ return MEMORY_PATTERNS.some((pattern) => pattern.test(normalized));
+}
+
+export function evaluateSurfacingGate(input: SurfacingGateInput): SurfacingGateDecision {
+ if (input.explicit === true) return Object.freeze({ retrieve: true, reason: "explicit" });
+
+ const normalized = normalizePrompt(input.prompt);
+ if (normalized.length === 0) return Object.freeze({ retrieve: false, reason: "empty" });
+
+ const previous = input.previousPrompt ? normalizePrompt(input.previousPrompt) : null;
+ if (previous !== null && previous === normalized) {
+ return Object.freeze({ retrieve: false, reason: "duplicate" });
+ }
+ if (isGreeting(normalized)) return Object.freeze({ retrieve: false, reason: "greeting" });
+ if (isSlashCommand(normalized))
+ return Object.freeze({ retrieve: false, reason: "slash_command" });
+ if (isShellOnlyPrompt(normalized)) {
+ return Object.freeze({ retrieve: false, reason: "shell_command" });
+ }
+ if (hasMemoryIntent(normalized)) {
+ return Object.freeze({ retrieve: true, reason: "memory_question" });
+ }
+ if (normalized.endsWith("?") && normalized.length >= 60) {
+ return Object.freeze({ retrieve: true, reason: "uncertain_question" });
+ }
+ return Object.freeze({ retrieve: false, reason: "no_recall_intent" });
+}
diff --git a/src/core/search/types.ts b/src/core/search/types.ts
index 211367d4..bdb0b422 100644
--- a/src/core/search/types.ts
+++ b/src/core/search/types.ts
@@ -5,6 +5,9 @@
*/
import type { VaultIgnoreRule } from "../vault-scope/defaults.ts";
+import type { EvidencePack } from "./evidence-pack.ts";
+import type { SearchSessionFocus } from "./session-focus.ts";
+import type { StructuredRecallQueryDocument } from "./structured-query.ts";
export type { VaultIgnoreRule };
@@ -60,7 +63,10 @@ export interface BrainSearchResult {
* Computed at query time from the links table, never stored on the
* result row. Absent when the page declares no typed relations.
*/
- readonly relations?: ReadonlyArray<{ readonly relation: string; readonly target: string }>;
+ readonly relations?: ReadonlyArray<{
+ readonly relation: string;
+ readonly target: string;
+ }>;
}
/**
@@ -105,7 +111,10 @@ export interface IndexStats {
readonly chunksTotal: number;
readonly embeddingsComputed: number;
readonly embeddingsRetries: number;
- readonly errors: ReadonlyArray<{ readonly path: string; readonly message: string }>;
+ readonly errors: ReadonlyArray<{
+ readonly path: string;
+ readonly message: string;
+ }>;
readonly durationMs: number;
}
@@ -189,12 +198,19 @@ export interface SearchOptions {
* untagged pages only). See src/core/graph/visibility.ts.
*/
readonly visibility?: ReadonlyArray<string>;
+ /** Optional parsed structured recall query document. Plain-string search ignores this. */
+ readonly structuredQuery?: StructuredRecallQueryDocument;
+ /** Optional per-query or persisted session focus steering. Undefined means load persisted focus. */
+ readonly sessionFocus?: SearchSessionFocus | null;
+ /** Opt-in verified evidence pack diagnostics. Omitted preserves the legacy search outcome shape. */
+ readonly evidencePack?: boolean;
}
export interface SearchOutcome {
readonly results: ReadonlyArray<BrainSearchResult>;
readonly warnings: ReadonlyArray<string>;
readonly total: number;
+ readonly evidencePack?: EvidencePack;
}
export interface ResolvedEmbeddingConfig {
diff --git a/src/mcp/brain-tools.ts b/src/mcp/brain-tools.ts
index 61846022..9b9e39d1 100644
--- a/src/mcp/brain-tools.ts
+++ b/src/mcp/brain-tools.ts
@@ -889,7 +889,9 @@ async function toolBrainAudit(
// through the shared wikilink normaliser first (handles `[[id]]`,
// `[[id|Alias]]`, and `Brain/.../id.md` forms), then strip the
// pref-/ret- prefix so every reference resolves to one trail.
- const slug = normaliseWikilinkTarget(raw).replace(/^(?:pref-|ret-)/, "").trim();
+ const slug = normaliseWikilinkTarget(raw)
+ .replace(/^(?:pref-|ret-)/, "")
+ .trim();
if (slug.length === 0) {
throw new Error(`brain_audit: empty preference slug after normalising '${raw}'`);
}
@@ -909,7 +911,11 @@ async function toolBrainMorningBrief(
): Promise<Record<string, unknown>> {
const topK = optionalPositiveInt(args, "top_k", "brain_morning_brief") ?? 10;
const lookbackDays = optionalPositiveInt(args, "lookback_days", "brain_morning_brief") ?? 7;
- const maxCharsPerMemory = optionalPositiveInt(args, "max_chars_per_memory", "brain_morning_brief");
+ const maxCharsPerMemory = optionalPositiveInt(
+ args,
+ "max_chars_per_memory",
+ "brain_morning_brief",
+ );
const maxTotalChars = optionalPositiveInt(args, "max_total_chars", "brain_morning_brief");
const brief = buildMorningBrief(ctx.vault, {
now: new Date(),
@@ -1694,11 +1700,13 @@ async function toolBrainContextPack(
throw new MCPError(INVALID_PARAMS, "brain_context_pack: max_tokens must be a positive integer");
}
const query = typeof args["query"] === "string" ? (args["query"] as string) : undefined;
+ const includeLanes = coerceBool(args, "lanes");
const maxCharsPerMemory = optionalPositiveInt(args, "max_chars_per_memory", "brain_context_pack");
const maxTotalChars = optionalPositiveInt(args, "max_total_chars", "brain_context_pack");
const report = packContext(ctx.vault, {
maxTokens,
...(query ? { query } : {}),
+ ...(includeLanes ? { includeLanes: true } : {}),
...(maxCharsPerMemory !== undefined ? { maxCharsPerMemory } : {}),
...(maxTotalChars !== undefined ? { maxTotalChars } : {}),
});
@@ -1719,6 +1727,7 @@ async function toolBrainContextPack(
tokens: s.tokens,
reason: s.reason,
})),
+ ...(report.lanes ? { lanes: report.lanes } : {}),
};
}
@@ -1754,7 +1763,11 @@ async function toolBrainPreCompressPack(
args: Record<string, unknown>,
): Promise<Record<string, unknown>> {
const topK = optionalPositiveInt(args, "top_k", "brain_pre_compress_pack") ?? 10;
- const maxCharsPerMemory = optionalPositiveInt(args, "max_chars_per_memory", "brain_pre_compress_pack");
+ const maxCharsPerMemory = optionalPositiveInt(
+ args,
+ "max_chars_per_memory",
+ "brain_pre_compress_pack",
+ );
const maxTotalChars = optionalPositiveInt(args, "max_total_chars", "brain_pre_compress_pack");
const pack = buildPreCompressPack(ctx.vault, {
topK,
@@ -1766,7 +1779,11 @@ async function toolBrainPreCompressPack(
text: pack.text,
active_head_included: pack.activeHeadIncluded,
total_chars: pack.totalChars,
- items: pack.items.map((i) => ({ id: i.id, principle: i.principle, trimmed: i.trimmed })),
+ items: pack.items.map((i) => ({
+ id: i.id,
+ principle: i.principle,
+ trimmed: i.trimmed,
+ })),
};
}
@@ -2301,7 +2318,11 @@ export const BRAIN_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([
name: "brain_sources",
description:
"Read-only dashboard of the brain's signals grouped by (agent, source_type) with active/processed and distinct-topic counts plus totals.",
- inputSchema: { type: "object", properties: {}, additionalProperties: false },
+ inputSchema: {
+ type: "object",
+ properties: {},
+ additionalProperties: false,
+ },
handler: toolBrainSources,
},
{
@@ -2347,6 +2368,11 @@ export const BRAIN_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([
description:
"Optional second ceiling (code points) on the cumulative size of the returned slice. Lowest-priority overflow is dropped with an `over-char-budget` skip reason.",
},
+ lanes: {
+ type: "boolean",
+ description:
+ "When true, also return polarity-aware directives, constraints, and consider lanes. Legacy flat `items` remains present.",
+ },
},
required: ["max_tokens"],
additionalProperties: false,
@@ -2365,17 +2391,20 @@ export const BRAIN_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([
top_k: {
type: "integer",
minimum: 1,
- description: "Maximum number of preferences to include, highest-confidence first (default 10).",
+ description:
+ "Maximum number of preferences to include, highest-confidence first (default 10).",
},
max_chars_per_memory: {
type: "integer",
minimum: 1,
- description: "Optional per-entry character cap (code points); trimmed entries carry `trimmed: true`.",
+ description:
+ "Optional per-entry character cap (code points); trimmed entries carry `trimmed: true`.",
},
max_total_chars: {
type: "integer",
minimum: 1,
- description: "Optional total character cap (code points) across the addendum; lowest-priority overflow is dropped.",
+ description:
+ "Optional total character cap (code points) across the addendum; lowest-priority overflow is dropped.",
},
},
additionalProperties: false,
diff --git a/src/mcp/search-tools.ts b/src/mcp/search-tools.ts
index b2ea7977..bcf1e1a4 100644
--- a/src/mcp/search-tools.ts
+++ b/src/mcp/search-tools.ts
@@ -10,7 +10,14 @@
* Anchored in docs/plans/2026-05-16-brain-search-design.md §9.
*/
-import { indexStatus, resolveSearchConfig, search, SearchError } from "../core/search/index.ts";
+import {
+ evaluateSurfacingGate,
+ indexStatus,
+ resolveSearchConfig,
+ search,
+ SearchError,
+} from "../core/search/index.ts";
+import { normalizeSessionFocus, parseStructuredRecallQueryDocument } from "../core/search/index.ts";
import type { BrainSearchResult, SearchOutcome } from "../core/search/index.ts";
import { withTimeout } from "../core/search/with-timeout.ts";
import { INTERNAL_ERROR, INVALID_PARAMS, MCPError } from "./protocol.ts";
@@ -27,6 +34,10 @@ const SEARCH_INPUT_SCHEMA: Record<string, unknown> = {
type: "object",
properties: {
query: { type: "string", minLength: 1, maxLength: 2000 },
+ query_document: { type: "string", minLength: 1, maxLength: 4000 },
+ focus_query: { type: "string", minLength: 1, maxLength: 1000 },
+ focus_path_prefix: { type: "string", minLength: 1, maxLength: 256 },
+ evidence_pack: { type: "boolean" },
limit: { type: "integer", minimum: 1, maximum: MCP_LIMIT_MAX },
semantic: { type: "boolean" },
keyword_only: { type: "boolean" },
@@ -78,6 +89,7 @@ const SEARCH_OUTPUT_SCHEMA: NonNullable<ToolDefinition["outputSchema"]> = {
endLine: { type: "integer" },
searchType: { type: "string" },
reasons: { type: "array", items: { type: "string" } },
+ why_retrieved: { type: "array", items: { type: "string" } },
relations: {
type: "array",
items: {
@@ -95,6 +107,27 @@ const SEARCH_OUTPUT_SCHEMA: NonNullable<ToolDefinition["outputSchema"]> = {
warnings: { type: "array", items: { type: "string" } },
total: { type: "integer" },
recall_hint: { type: "string" },
+ evidence_pack: { type: "object" },
+ },
+};
+
+const RECALL_GATE_INPUT_SCHEMA: Record<string, unknown> = {
+ type: "object",
+ properties: {
+ prompt: { type: "string", minLength: 1, maxLength: 4000 },
+ previous_prompt: { type: "string", maxLength: 4000 },
+ explicit: { type: "boolean" },
+ },
+ required: ["prompt"],
+ additionalProperties: false,
+};
+
+const RECALL_GATE_OUTPUT_SCHEMA: NonNullable<ToolDefinition["outputSchema"]> = {
+ type: "object",
+ required: ["retrieve", "reason"],
+ properties: {
+ retrieve: { type: "boolean" },
+ reason: { type: "string" },
},
};
@@ -208,6 +241,21 @@ async function toolBrainSearch(
const semantic = coerceBoolOptional(args, "semantic");
const keywordOnly = coerceBoolOptional(args, "keyword_only") ?? false;
const pathPrefix = coerceStringOptional(args, "path_prefix", 256);
+ const evidencePack = coerceBoolOptional(args, "evidence_pack") ?? false;
+ const rawQueryDocument = coerceStringOptional(args, "query_document", 4000);
+ const structuredQuery =
+ rawQueryDocument !== undefined
+ ? parseStructuredRecallQueryDocument(rawQueryDocument)
+ : undefined;
+ const focusQuery = coerceStringOptional(args, "focus_query", 1000);
+ const focusPathPrefix = coerceStringOptional(args, "focus_path_prefix", 256);
+ const sessionFocus =
+ focusQuery !== undefined || focusPathPrefix !== undefined
+ ? normalizeSessionFocus({
+ query: focusQuery ?? null,
+ pathPrefix: focusPathPrefix ?? null,
+ })
+ : undefined;
const properties = parsePropertiesArgument(args["properties"]);
const visibility = parseVisibilityArgument(args["visibility"]);
@@ -227,6 +275,9 @@ async function toolBrainSearch(
pathPrefix,
...(properties !== undefined ? { properties } : {}),
...(visibility !== undefined ? { visibility } : {}),
+ ...(structuredQuery !== undefined ? { structuredQuery } : {}),
+ ...(sessionFocus !== undefined ? { sessionFocus } : {}),
+ ...(evidencePack ? { evidencePack: true } : {}),
}),
SEARCH_TIMEOUT_MS,
searchTimeoutError,
@@ -248,15 +299,71 @@ async function toolBrainSearch(
endLine: r.endLine,
searchType: r.searchType,
reasons: r.reasons,
+ ...(outcome.evidencePack ? { why_retrieved: r.reasons } : {}),
...(r.relations && r.relations.length > 0 ? { relations: r.relations } : {}),
})),
warnings: outcome.warnings,
total: outcome.total,
+ ...(outcome.evidencePack ? { evidence_pack: mcpEvidencePack(outcome.evidencePack) } : {}),
...(recallHint !== null ? { recall_hint: recallHint } : {}),
};
}
+function mcpEvidencePack(
+ pack: NonNullable<SearchOutcome["evidencePack"]>,
+): Record<string, unknown> {
+ return {
+ significant_terms: pack.significantTerms,
+ matched_terms: pack.matchedTerms,
+ missing_terms: pack.missingTerms,
+ support_coverage: pack.supportCoverage,
+ records: pack.records.map((record) => ({
+ path: record.path,
+ document_id: record.documentId,
+ chunk_id: record.chunkId,
+ matched_terms: record.matchedTerms,
+ missing_terms: record.missingTerms,
+ support_coverage: record.supportCoverage,
+ terminal_state: record.terminalState,
+ why_retrieved: record.whyRetrieved,
+ dropped_candidate_reasons: record.droppedCandidateReasons,
+ })),
+ dropped_candidates: pack.droppedCandidates,
+ abstention: pack.abstention,
+ };
+}
+
+async function toolBrainRecallGate(
+ _ctx: ServerContext,
+ args: Record<string, unknown>,
+): Promise<Record<string, unknown>> {
+ const prompt = args["prompt"];
+ if (typeof prompt !== "string" || prompt.trim() === "") {
+ throw new MCPError(INVALID_PARAMS, "missing required argument: prompt");
+ }
+ if (prompt.length > 4000) {
+ throw new MCPError(INVALID_PARAMS, "argument 'prompt' exceeds 4000 characters");
+ }
+ const previousPrompt = coerceStringOptional(args, "previous_prompt", 4000);
+ const explicit = coerceBoolOptional(args, "explicit") ?? false;
+ return {
+ ...evaluateSurfacingGate({
+ prompt,
+ previousPrompt: previousPrompt ?? null,
+ explicit,
+ }),
+ };
+}
+
export const SEARCH_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([
+ {
+ name: "brain_recall_gate",
+ description:
+ "Classify whether an automatic recall/surfacing attempt should run. Diagnostics only; does not search.",
+ inputSchema: RECALL_GATE_INPUT_SCHEMA,
+ outputSchema: RECALL_GATE_OUTPUT_SCHEMA,
+ handler: toolBrainRecallGate,
+ },
{
name: "brain_search",
description:
diff --git a/tests/cli/brain-context-pack.test.ts b/tests/cli/brain-context-pack.test.ts
new file mode 100644
index 00000000..50707b93
--- /dev/null
+++ b/tests/cli/brain-context-pack.test.ts
@@ -0,0 +1,41 @@
+import { afterEach, beforeEach, expect, test } from "bun:test";
+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { runCli } from "../helpers/run-cli.ts";
+
+let vault: string;
+
+beforeEach(() => {
+ vault = mkdtempSync(join(tmpdir(), "o2b-context-pack-cli-"));
+ mkdirSync(join(vault, "Brain", "preferences"), { recursive: true });
+});
+
+afterEach(() => {
+ rmSync(vault, { recursive: true, force: true });
+});
+
+function writePref(slug: string, principle: string, extra = ""): void {
+ writeFileSync(
+ join(vault, "Brain", "preferences", `pref-${slug}.md`),
+ ["---", `id: pref-${slug}`, "topic: t", `principle: ${principle}`, extra, "---", ""].join("\n"),
+ );
+}
+
+test("brain context-pack --lanes --json returns polarity lanes", async () => {
+ writePref("directive", "Prefer concise answers", "tier: core");
+ writePref("constraint", "Never expose tokens", "tier: core");
+
+ const out = await runCli(
+ ["brain", "context-pack", "--vault", vault, "--max-tokens", "10000", "--lanes", "--json"],
+ {},
+ );
+
+ expect(out.returncode).toBe(0);
+ const json = JSON.parse(out.stdout);
+ expect(json.lanes.directives.map((item: { id: string }) => item.id)).toContain("pref-directive");
+ expect(json.lanes.constraints.map((item: { id: string }) => item.id)).toContain(
+ "pref-constraint",
+ );
+});
diff --git a/tests/cli/search.test.ts b/tests/cli/search.test.ts
index fdeae552..9fd465c5 100644
--- a/tests/cli/search.test.ts
+++ b/tests/cli/search.test.ts
@@ -56,7 +56,9 @@ test("search status without an index reports 'not initialised' and exits 0", asy
test("search status --json after an index returns documents count", async () => {
writeVaultFile("a.md", "# A\n\nbody");
writeVaultFile("b.md", "# B\n\nbody");
- await runCli(["search", "index"], { env: { OPEN_SECOND_BRAIN_CONFIG: config } });
+ await runCli(["search", "index"], {
+ env: { OPEN_SECOND_BRAIN_CONFIG: config },
+ });
const out = await runCli(["search", "status", "--json"], {
env: { OPEN_SECOND_BRAIN_CONFIG: config },
});
@@ -69,7 +71,9 @@ test("search status --json after an index returns documents count", async () =>
test("search query returns a human-readable hit for indexed content", async () => {
writeVaultFile("notes/foo.md", "# Foo\n\nthe quick brown fox jumps over the lazy dog");
- await runCli(["search", "index"], { env: { OPEN_SECOND_BRAIN_CONFIG: config } });
+ await runCli(["search", "index"], {
+ env: { OPEN_SECOND_BRAIN_CONFIG: config },
+ });
const out = await runCli(["search", "fox"], {
env: { OPEN_SECOND_BRAIN_CONFIG: config },
});
@@ -79,7 +83,9 @@ test("search query returns a human-readable hit for indexed content", async () =
test("search query --json returns structured results", async () => {
writeVaultFile("notes/foo.md", "# Foo\n\nfox content");
- await runCli(["search", "index"], { env: { OPEN_SECOND_BRAIN_CONFIG: config } });
+ await runCli(["search", "index"], {
+ env: { OPEN_SECOND_BRAIN_CONFIG: config },
+ });
const out = await runCli(["search", "fox", "--json", "--limit", "5"], {
env: { OPEN_SECOND_BRAIN_CONFIG: config },
});
@@ -90,6 +96,77 @@ test("search query --json returns structured results", async () => {
expect(obj.results[0].path).toBe("notes/foo.md");
});
+test("search query --query-doc accepts structured recall documents", async () => {
+ writeVaultFile("notes/final.md", "# Final\n\nrelease notes mention recall diagnostics.");
+ writeVaultFile("notes/draft.md", "# Draft\n\ndraft release notes mention recall diagnostics.");
+ await runCli(["search", "index"], {
+ env: { OPEN_SECOND_BRAIN_CONFIG: config },
+ });
+
+ const out = await runCli(
+ ["search", "--query-doc", 'lex: "release notes" -draft', "--json", "--limit", "10"],
+ { env: { OPEN_SECOND_BRAIN_CONFIG: config } },
+ );
+
+ expect(out.returncode).toBe(0);
+ const obj = JSON.parse(out.stdout);
+ expect(obj.results.map((r: { path: string }) => r.path)).toContain("notes/final.md");
+ expect(obj.results.map((r: { path: string }) => r.path)).not.toContain("notes/draft.md");
+ expect(obj.results[0].reasons.some((reason: string) => reason.includes("lane:lex/fts5"))).toBe(
+ true,
+ );
+});
+
+test("search query --evidence-pack returns missing terms and why_retrieved", async () => {
+ writeVaultFile("notes/foo.md", "# Foo\n\nalpha beta current support.");
+ await runCli(["search", "index"], {
+ env: { OPEN_SECOND_BRAIN_CONFIG: config },
+ });
+
+ const out = await runCli(
+ ["search", "alpha gamma", "--query-doc", "lex: alpha", "--json", "--evidence-pack"],
+ {
+ env: { OPEN_SECOND_BRAIN_CONFIG: config },
+ },
+ );
+
+ expect(out.returncode).toBe(0);
+ const obj = JSON.parse(out.stdout);
+ expect(obj.evidence_pack.missing_terms).toContain("gamma");
+ expect(Array.isArray(obj.results[0].why_retrieved)).toBe(true);
+});
+
+test("search focus set/status/clear steers only the focused query window", async () => {
+ writeVaultFile("archive/other.md", "# Other\n\nshared recall topic.");
+ writeVaultFile("sessions/focus.md", "# Focus\n\nshared recall topic.");
+ await runCli(["search", "index"], {
+ env: { OPEN_SECOND_BRAIN_CONFIG: config },
+ });
+
+ const set = await runCli(
+ ["search", "focus", "set", "--path", "sessions/", "--ttl-minutes", "60", "--json"],
+ { env: { OPEN_SECOND_BRAIN_CONFIG: config } },
+ );
+ expect(set.returncode).toBe(0);
+ expect(JSON.parse(set.stdout).active).toBe(true);
+
+ const focused = await runCli(["search", "shared", "--json", "--limit", "2"], {
+ env: { OPEN_SECOND_BRAIN_CONFIG: config },
+ });
+ expect(focused.returncode).toBe(0);
+ const focusedJson = JSON.parse(focused.stdout);
+ expect(focusedJson.results[0].path).toBe("sessions/focus.md");
+ expect(
+ focusedJson.results[0].reasons.some((reason: string) => reason.startsWith("session_focus:")),
+ ).toBe(true);
+
+ const clear = await runCli(["search", "focus", "clear", "--json"], {
+ env: { OPEN_SECOND_BRAIN_CONFIG: config },
+ });
+ expect(clear.returncode).toBe(0);
+ expect(JSON.parse(clear.stdout).active).toBe(false);
+});
+
test("search query on missing index fails with exit 1", async () => {
const out = await runCli(["search", "nothing-here"], {
env: { OPEN_SECOND_BRAIN_CONFIG: config },
@@ -111,7 +188,9 @@ test("search check reports vault_readable and sqlite_ok on a fresh vault", async
test("search reindex rebuilds the index atomically", async () => {
writeVaultFile("a.md", "# A");
- await runCli(["search", "index"], { env: { OPEN_SECOND_BRAIN_CONFIG: config } });
+ await runCli(["search", "index"], {
+ env: { OPEN_SECOND_BRAIN_CONFIG: config },
+ });
writeVaultFile("b.md", "# B");
const out = await runCli(["search", "reindex"], {
env: { OPEN_SECOND_BRAIN_CONFIG: config },
@@ -144,7 +223,9 @@ test("invalid numeric search flags exit with code 2 before touching the index",
test("path-prefix escaping returns exit 2 with INVALID_INPUT", async () => {
writeVaultFile("a.md", "# A");
- await runCli(["search", "index"], { env: { OPEN_SECOND_BRAIN_CONFIG: config } });
+ await runCli(["search", "index"], {
+ env: { OPEN_SECOND_BRAIN_CONFIG: config },
+ });
const out = await runCli(["search", "A", "--path", "../etc/"], {
env: { OPEN_SECOND_BRAIN_CONFIG: config },
});
@@ -154,7 +235,9 @@ test("path-prefix escaping returns exit 2 with INVALID_INPUT", async () => {
test("the default verb is `query` when first positional is unknown", async () => {
writeVaultFile("a.md", "# A\n\nalpha word");
- await runCli(["search", "index"], { env: { OPEN_SECOND_BRAIN_CONFIG: config } });
+ await runCli(["search", "index"], {
+ env: { OPEN_SECOND_BRAIN_CONFIG: config },
+ });
// No explicit verb, just a query token:
const out = await runCli(["search", "alpha"], {
env: { OPEN_SECOND_BRAIN_CONFIG: config },
diff --git a/tests/core/brain/context-pack.test.ts b/tests/core/brain/context-pack.test.ts
index 9119d7b1..b1812a25 100644
--- a/tests/core/brain/context-pack.test.ts
+++ b/tests/core/brain/context-pack.test.ts
@@ -3,6 +3,7 @@ import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
+import { normalizeContextLane } from "../../../src/core/brain/context-lanes.ts";
import { packContext } from "../../../src/core/brain/context-pack.ts";
let vault: string;
@@ -18,7 +19,13 @@ afterEach(() => {
function writePref(
slug: string,
- fields: { topic: string; principle: string; tier?: string; created_at?: string },
+ fields: {
+ topic: string;
+ principle: string;
+ tier?: string;
+ created_at?: string;
+ context_lane?: string;
+ },
) {
const lines = [
"---",
@@ -28,11 +35,16 @@ function writePref(
];
if (fields.tier) lines.push(`tier: ${fields.tier}`);
if (fields.created_at) lines.push(`created_at: ${fields.created_at}`);
+ if (fields.context_lane) lines.push(`context_lane: ${fields.context_lane}`);
lines.push("---", "");
writeFileSync(join(vault, "Brain", "preferences", `pref-${slug}.md`), lines.join("\n"));
}
describe("packContext", () => {
+ test("normalizeContextLane handles uppercase lane names deterministically", () => {
+ expect(normalizeContextLane("DIRECTIVES")).toBe("directives");
+ });
+
test("empty vault returns empty pack", () => {
const r = packContext(vault, { maxTokens: 1000 });
expect(r.tokensUsed).toBe(0);
@@ -51,7 +63,10 @@ describe("packContext", () => {
expect(full.items[0]!.trimmed).toBe(false);
expect([...full.items[0]!.body].length).toBeGreaterThanOrEqual(500);
- const capped = packContext(vault, { maxTokens: 100_000, maxCharsPerMemory: 100 });
+ const capped = packContext(vault, {
+ maxTokens: 100_000,
+ maxCharsPerMemory: 100,
+ });
expect([...capped.items[0]!.body].length).toBe(100);
expect(capped.items[0]!.trimmed).toBe(true);
});
@@ -60,22 +75,40 @@ describe("packContext", () => {
const writeBody = (slug: string, tier: string, body: string) =>
writeFileSync(
join(vault, "Brain", "preferences", `pref-${slug}.md`),
- ["---", `id: pref-${slug}`, "topic: t", "principle: p", `tier: ${tier}`, "---", "", body].join(
- "\n",
- ),
+ [
+ "---",
+ `id: pref-${slug}`,
+ "topic: t",
+ "principle: p",
+ `tier: ${tier}`,
+ "---",
+ "",
+ body,
+ ].join("\n"),
);
// Core (highest priority) emitted first, then peripheral.
writeBody("a", "core", "a".repeat(60));
writeBody("z", "peripheral", "z".repeat(60));
- const capped = packContext(vault, { maxTokens: 100_000, maxTotalChars: 60 });
+ const capped = packContext(vault, {
+ maxTokens: 100_000,
+ maxTotalChars: 60,
+ });
expect(capped.items.map((i) => i.id)).toEqual(["pref-a"]);
expect(capped.skipped.find((s) => s.id === "pref-z")?.reason).toBe("over-char-budget");
});
test("orders core → supporting → peripheral", () => {
- writePref("p", { topic: "x", principle: "peripheral one", tier: "peripheral" });
- writePref("s", { topic: "x", principle: "supporting one", tier: "supporting" });
+ writePref("p", {
+ topic: "x",
+ principle: "peripheral one",
+ tier: "peripheral",
+ });
+ writePref("s", {
+ topic: "x",
+ principle: "supporting one",
+ tier: "supporting",
+ });
writePref("c", { topic: "x", principle: "core one", tier: "core" });
const r = packContext(vault, { maxTokens: 10_000 });
const ids = r.items.map((i) => i.id);
@@ -154,4 +187,51 @@ describe("packContext", () => {
expect(packContext(vault, { maxTokens: 0 }).items.length).toBe(0);
expect(packContext(vault, { maxTokens: -5 }).items.length).toBe(0);
});
+
+ test("includeLanes classifies directives, constraints, consider, and manual overrides", () => {
+ writePref("directive", {
+ topic: "style",
+ principle: "Prefer short direct answers",
+ tier: "core",
+ });
+ writePref("constraint", {
+ topic: "security",
+ principle: "Never expose secret tokens",
+ tier: "core",
+ });
+ writePref("consider", {
+ topic: "taste",
+ principle: "Maybe use a quiet tone",
+ tier: "peripheral",
+ });
+ writePref("manual", {
+ topic: "ops",
+ principle: "Use release notes",
+ tier: "supporting",
+ context_lane: "constraints",
+ });
+
+ const lanes = packContext(vault, {
+ maxTokens: 10_000,
+ includeLanes: true,
+ }).lanes!;
+
+ expect(lanes.directives.map((item) => item.id)).toContain("pref-directive");
+ expect(lanes.constraints.map((item) => item.id)).toContain("pref-manual");
+ expect(lanes.constraints.map((item) => item.id)).toContain("pref-constraint");
+ expect(lanes.consider.map((item) => item.id)).toContain("pref-consider");
+ const manual = lanes.constraints.find((item) => item.id === "pref-manual")!;
+ expect(manual.sourceId).toBe("pref-manual");
+ expect(manual.sourcePath.endsWith("pref-manual.md")).toBe(true);
+ });
+
+ test("legacy context pack output omits lanes unless requested", () => {
+ writePref("directive", {
+ topic: "style",
+ principle: "Prefer short direct answers",
+ tier: "core",
+ });
+
+ expect(packContext(vault, { maxTokens: 10_000 }).lanes).toBeUndefined();
+ });
});
diff --git a/tests/core/search/evidence-pack.test.ts b/tests/core/search/evidence-pack.test.ts
new file mode 100644
index 00000000..8296a761
--- /dev/null
+++ b/tests/core/search/evidence-pack.test.ts
@@ -0,0 +1,55 @@
+import { expect, test } from "bun:test";
+
+import { buildEvidencePack } from "../../../src/core/search/evidence-pack.ts";
+import type { BrainSearchResult } from "../../../src/core/search/types.ts";
+
+function result(
+ path: string,
+ content: string,
+ reasons: string[] = ["fts5_bm25: 1.000"],
+): BrainSearchResult {
+ return Object.freeze({
+ documentId: path === "active.md" ? 1 : 2,
+ chunkId: path === "active.md" ? 1 : 2,
+ path,
+ title: path,
+ content,
+ startLine: 1,
+ endLine: 1,
+ score: 1,
+ keywordScore: 1,
+ semanticScore: 0,
+ linkBoost: 0,
+ recencyBoost: 0,
+ searchType: "keyword",
+ reasons: Object.freeze(reasons),
+ });
+}
+
+test("buildEvidencePack reports matched and missing significant terms", () => {
+ const pack = buildEvidencePack("alpha gamma", [result("active.md", "alpha beta")]);
+
+ expect(pack.significantTerms).toEqual(["alpha", "gamma"]);
+ expect(pack.matchedTerms).toEqual(["alpha"]);
+ expect(pack.missingTerms).toEqual(["gamma"]);
+ expect(pack.abstention).toContain("gamma");
+ expect(pack.records[0]?.whyRetrieved).toEqual(["fts5_bm25: 1.000"]);
+});
+
+test("buildEvidencePack marks terminal-state records", () => {
+ const pack = buildEvidencePack("alpha", [result("done.md", "alpha superseded by another note")]);
+
+ expect(pack.records[0]?.terminalState).toBe(true);
+ expect(pack.records[0]?.droppedCandidateReasons).toEqual([]);
+});
+
+test("buildEvidencePack reports terminal downrank only when applied", () => {
+ const pack = buildEvidencePack("alpha", [
+ result("done.md", "alpha superseded by another note", [
+ "fts5_bm25: 1.000",
+ "evidence_terminal_downrank: true",
+ ]),
+ ]);
+
+ expect(pack.records[0]?.droppedCandidateReasons).toContain("terminal_state_downranked");
+});
diff --git a/tests/core/search/fts.test.ts b/tests/core/search/fts.test.ts
index fe4f477c..090190aa 100644
--- a/tests/core/search/fts.test.ts
+++ b/tests/core/search/fts.test.ts
@@ -1,4 +1,5 @@
import { test, expect, beforeEach, afterEach } from "bun:test";
+import { Database } from "bun:sqlite";
import { buildFtsMatch, runFtsQuery } from "../../../src/core/search/fts.ts";
import { Store } from "../../../src/core/search/store.ts";
@@ -20,7 +21,10 @@ afterEach(() => {
});
async function fixture() {
- const store = await Store.open(makeConfig({ vault, dbPath }), { mode: "write", loadVec: false });
+ const store = await Store.open(makeConfig({ vault, dbPath }), {
+ mode: "write",
+ loadVec: false,
+ });
const d1 = store.upsertDocument({
path: "Notes/alpha.md",
title: "Alpha",
@@ -66,6 +70,15 @@ async function fixture() {
return { store, d1, d2 };
}
+function clearFtsRows(): void {
+ const db = new Database(dbPath);
+ try {
+ db.run("DELETE FROM chunk_fts");
+ } finally {
+ db.close();
+ }
+}
+
test("buildFtsMatch quotes tokens for safety", () => {
expect(buildFtsMatch("hello")).toBe('"hello"');
expect(buildFtsMatch("two words")).toBe('"two" "words"');
@@ -82,7 +95,11 @@ test("buildFtsMatch ignores empty input", () => {
test("buildFtsMatch defangs FTS5 operator words and metacharacters", () => {
// Without quoting, "AND" would be a boolean operator and "*" a prefix glob.
- expect(buildFtsMatch("AND foo*")).toBe('"AND" "foo*"');
+ // With other meaningful tokens present, standalone uppercase operators are
+ // dropped so natural-language connector words do not over-constrain recall.
+ expect(buildFtsMatch("AND foo*")).toBe('"foo*"');
+ expect(buildFtsMatch("fox AND OR")).toBe('"fox"');
+ expect(buildFtsMatch("AND OR")).toBe('"AND" "OR"');
});
test("runFtsQuery returns BM25-ordered hits across documents", async () => {
@@ -124,9 +141,25 @@ test("runFtsQuery survives FTS5-operator-like queries safely", async () => {
// must not throw and must not return spurious matches.
expect(() => runFtsQuery(store, "AND", { limit: 5 })).not.toThrow();
expect(runFtsQuery(store, "AND", { limit: 5 })).toEqual([]);
- // A query mixing operator-like tokens with a real token returns matches
- // when all tokens are present in the document.
- const hits = runFtsQuery(store, "fox", { limit: 5 });
+ // A query mixing operator-like tokens with a real token drops the operator
+ // token and still returns natural-language matches.
+ const hits = runFtsQuery(store, "fox AND", { limit: 5 });
expect(hits.length).toBeGreaterThan(0);
await store.close();
});
+
+test("runFtsQuery rebuilds an empty desynced FTS table once", async () => {
+ const { store, d2 } = await fixture();
+ await store.close();
+ clearFtsRows();
+
+ const readStore = await Store.open(makeConfig({ vault, dbPath }), {
+ mode: "read",
+ loadVec: false,
+ });
+ const hits = runFtsQuery(readStore, "fox", { limit: 10 });
+
+ expect(hits.length).toBeGreaterThanOrEqual(2);
+ expect(hits[0]?.documentId).toBe(d2);
+ await readStore.close();
+});
diff --git a/tests/core/search/ranker-reasons.test.ts b/tests/core/search/ranker-reasons.test.ts
index 07cfc39b..2d654949 100644
--- a/tests/core/search/ranker-reasons.test.ts
+++ b/tests/core/search/ranker-reasons.test.ts
@@ -9,11 +9,16 @@ import { test, expect } from "bun:test";
import { rankResults } from "../../../src/core/search/ranker.ts";
import type { KeywordHit, SemanticHit, HydratedChunk } from "../../../src/core/search/store.ts";
-function hyd(chunkId: number, docId: number, mtime: number): HydratedChunk {
+function hyd(
+ chunkId: number,
+ docId: number,
+ mtime: number,
+ path = `doc${docId}.md`,
+): HydratedChunk {
return Object.freeze({
chunkId,
documentId: docId,
- path: `doc${docId}.md`,
+ path,
title: `Doc ${docId}`,
content: `chunk ${chunkId}`,
startLine: 1,
@@ -128,3 +133,35 @@ test("reasons array is frozen", () => {
);
expect(Object.isFrozen(ranked[0]!.reasons)).toBe(true);
});
+
+test("session focus can re-rank a focused path and explains the contribution", () => {
+ const ranked = rankResults(
+ {
+ keyword: [
+ { chunkId: 1, documentId: 10, bm25: -5 },
+ { chunkId: 2, documentId: 11, bm25: -5 },
+ ] as KeywordHit[],
+ semantic: [] as SemanticHit[],
+ hydrated: new Map([
+ [1, hyd(1, 10, OLD, "archive/other.md")],
+ [2, hyd(2, 11, OLD, "sessions/focus.md")],
+ ]),
+ inboundLinkSources: new Map(),
+ tagsByDoc: new Map(),
+ },
+ {
+ keywordWeight: 0.6,
+ semanticWeight: 0.4,
+ limit: 10,
+ nowMs: NOW,
+ sessionFocus: {
+ query: null,
+ pathPrefix: "sessions/",
+ expiresAt: NOW + 60_000,
+ },
+ },
+ );
+
+ expect(ranked[0]?.path).toBe("sessions/focus.md");
+ expect(ranked[0]?.reasons.some((reason) => reason.startsWith("session_focus:"))).toBe(true);
+});
diff --git a/tests/core/search/search.test.ts b/tests/core/search/search.test.ts
index 647fc4b2..0602ee5e 100644
--- a/tests/core/search/search.test.ts
+++ b/tests/core/search/search.test.ts
@@ -3,6 +3,7 @@ import { Database } from "bun:sqlite";
import { indexVault } from "../../../src/core/search/indexer.ts";
import { search } from "../../../src/core/search/search.ts";
+import { parseStructuredRecallQueryDocument } from "../../../src/core/search/structured-query.ts";
import { SearchError } from "../../../src/core/search/types.ts";
import { createTempVault, makeConfig, writeMd } from "../../helpers/search-fixtures.ts";
import { startFakeHttp, type FakeHttp } from "../../helpers/fake-http.ts";
@@ -143,6 +144,96 @@ test("search returns keyword-only results when semantic is disabled", async () =
expect(out.warnings).toEqual([]);
});
+test("search warns when keyword retrieval rebuilds a desynced FTS table", async () => {
+ const cfg = await seedKeyword();
+ await indexVault(cfg);
+ const db = new Database(dbPath);
+ try {
+ db.run("DELETE FROM chunk_fts");
+ } finally {
+ db.close();
+ }
+
+ const out = await search(cfg, { query: "fox", limit: 5 });
+
+ expect(out.results.map((r) => r.path)).toContain("Notes/foo.md");
+ expect(out.warnings.some((w) => w.includes("rebuilt FTS"))).toBe(true);
+});
+
+test("search accepts structured lexical lanes with safe exclusions", async () => {
+ writeMd(vault, "Notes/final.md", "# Final\n\nrelease notes mention recall diagnostics.");
+ writeMd(vault, "Notes/draft.md", "# Draft\n\ndraft release notes mention recall diagnostics.");
+ const cfg = makeConfig({ vault, dbPath });
+ await indexVault(cfg);
+ const structuredQuery = parseStructuredRecallQueryDocument('lex: "release notes" -draft');
+
+ const out = await search(cfg, {
+ query: "release notes",
+ structuredQuery,
+ limit: 10,
+ });
+
+ expect(out.results.map((r) => r.path)).toContain("Notes/final.md");
+ expect(out.results.map((r) => r.path)).not.toContain("Notes/draft.md");
+ expect(out.results[0]?.reasons.some((reason) => reason.includes("lane:lex/fts5"))).toBe(true);
+});
+
+test("search backfills structured lexical exclusions before applying limit", async () => {
+ writeMd(vault, "Notes/draft-a.md", "# Draft A\n\nalpha alpha alpha draft.");
+ writeMd(vault, "Notes/draft-b.md", "# Draft B\n\nalpha alpha draft.");
+ writeMd(vault, "Notes/final-a.md", "# Final A\n\nalpha current.");
+ writeMd(vault, "Notes/final-b.md", "# Final B\n\nalpha stable.");
+ const cfg = makeConfig({ vault, dbPath, mmrLambda: 1, maxHops: 0 });
+ await indexVault(cfg);
+ const structuredQuery = parseStructuredRecallQueryDocument("lex: alpha -draft");
+
+ const out = await search(cfg, {
+ query: "alpha",
+ structuredQuery,
+ limit: 2,
+ mmrLambda: 1,
+ maxHops: 0,
+ });
+
+ expect(out.results.map((r) => r.path).toSorted()).toEqual([
+ "Notes/final-a.md",
+ "Notes/final-b.md",
+ ]);
+});
+
+test("search degrades semantic structured lanes when semantic search is disabled", async () => {
+ const cfg = await seedKeyword();
+ await indexVault(cfg);
+ const structuredQuery = parseStructuredRecallQueryDocument("vec: fuzzy fox memory");
+
+ const out = await search(cfg, { query: "fox", structuredQuery, limit: 5 });
+
+ expect(out.results.map((r) => r.path)).toContain("Notes/foo.md");
+ expect(
+ out.warnings.some((warning) => warning.includes("semantic structured lanes skipped")),
+ ).toBe(true);
+});
+
+test("search can return an evidence pack and downrank terminal-state support", async () => {
+ writeMd(vault, "Notes/terminal.md", "# Terminal\n\nalpha beta superseded by active note.");
+ writeMd(vault, "Notes/active.md", "# Active\n\nalpha beta current support.");
+ const cfg = makeConfig({ vault, dbPath });
+ await indexVault(cfg);
+
+ const structuredQuery = parseStructuredRecallQueryDocument('lex: "alpha beta"');
+ const out = await search(cfg, {
+ query: "alpha beta gamma",
+ structuredQuery,
+ limit: 5,
+ evidencePack: true,
+ });
+
+ expect(out.results[0]?.path).toBe("Notes/active.md");
+ expect(out.evidencePack?.missingTerms).toContain("gamma");
+ expect(
+ out.evidencePack?.records.find((record) => record.path === "Notes/terminal.md")?.terminalState,
+ ).toBe(true);
+});
test("search returns empty (no error) when no results match", async () => {
const cfg = await seedKeyword();
await indexVault(cfg);
diff --git a/tests/core/search/session-focus.test.ts b/tests/core/search/session-focus.test.ts
new file mode 100644
index 00000000..51e700e2
--- /dev/null
+++ b/tests/core/search/session-focus.test.ts
@@ -0,0 +1,49 @@
+import { expect, test } from "bun:test";
+import { mkdirSync, writeFileSync } from "node:fs";
+import { dirname } from "node:path";
+
+import {
+ normalizeSessionFocus,
+ readSessionFocus,
+ sessionFocusPath,
+ sessionFocusIsActive,
+} from "../../../src/core/search/session-focus.ts";
+import { createTempVault, makeConfig } from "../../helpers/search-fixtures.ts";
+
+const NOW = 1_750_000_000_000;
+
+test("normalizeSessionFocus trims query and path with bounded ttl", () => {
+ const focus = normalizeSessionFocus(
+ {
+ query: " release decision ",
+ pathPrefix: " Sessions/May/ ",
+ ttlMinutes: 30,
+ },
+ NOW,
+ );
+
+ expect(focus.query).toBe("release decision");
+ expect(focus.pathPrefix).toBe("Sessions/May/");
+ expect(focus.expiresAt).toBe(NOW + 30 * 60 * 1000);
+ expect(sessionFocusIsActive(focus, NOW)).toBe(true);
+});
+
+test("sessionFocusIsActive rejects expired focus", () => {
+ const focus = normalizeSessionFocus({ query: "alpha", ttlMinutes: 1 }, NOW);
+
+ expect(sessionFocusIsActive(focus, NOW + 61_000)).toBe(false);
+});
+
+test("readSessionFocus treats malformed persisted focus as missing", () => {
+ const temp = createTempVault("session-focus");
+ try {
+ const config = makeConfig({ vault: temp.vault, dbPath: temp.dbPath });
+ const path = sessionFocusPath(config);
+ mkdirSync(dirname(path), { recursive: true });
+ writeFileSync(path, JSON.stringify({ query: " alpha ", expiresAt: "forever" }));
+
+ expect(readSessionFocus(config, NOW)).toBeNull();
+ } finally {
+ temp.cleanup();
+ }
+});
diff --git a/tests/core/search/structured-query.test.ts b/tests/core/search/structured-query.test.ts
new file mode 100644
index 00000000..8a82d6bd
--- /dev/null
+++ b/tests/core/search/structured-query.test.ts
@@ -0,0 +1,61 @@
+import { expect, test } from "bun:test";
+
+import { parseStructuredRecallQueryDocument } from "../../../src/core/search/structured-query.ts";
+import { SearchError } from "../../../src/core/search/types.ts";
+
+test("parseStructuredRecallQueryDocument parses intent and typed lanes", () => {
+ const parsed = parseStructuredRecallQueryDocument(`
+intent: exact
+lex: "release notes" -draft
+vec: semantic memory recall
+hyde: the accepted release decision mentions recall diagnostics
+`);
+
+ expect(parsed.intent).toBe("exact");
+ expect(parsed.lex.include).toEqual(["release notes"]);
+ expect(parsed.lex.exclude).toEqual(["draft"]);
+ expect(parsed.vec).toEqual(["semantic memory recall"]);
+ expect(parsed.hyde).toEqual(["the accepted release decision mentions recall diagnostics"]);
+});
+
+test("parseStructuredRecallQueryDocument allows repeated lanes", () => {
+ const parsed = parseStructuredRecallQueryDocument(`
+lex: alpha
+lex: beta
+vec: gamma
+vec: delta
+`);
+
+ expect(parsed.lex.include).toEqual(["alpha", "beta"]);
+ expect(parsed.vec).toEqual(["gamma", "delta"]);
+});
+
+test("parseStructuredRecallQueryDocument rejects repeated intent lanes", () => {
+ expect(() =>
+ parseStructuredRecallQueryDocument(`
+intent: neutral
+intent: broad
+`),
+ ).toThrow(SearchError);
+});
+
+test("parseStructuredRecallQueryDocument rejects malformed lane syntax", () => {
+ let err: SearchError | null = null;
+ try {
+ parseStructuredRecallQueryDocument("lex without colon");
+ } catch (e) {
+ err = e as SearchError;
+ }
+
+ expect(err).toBeInstanceOf(SearchError);
+ expect(err?.code).toBe("INVALID_INPUT");
+ expect(err?.message).toContain("line 1");
+});
+
+test("parseStructuredRecallQueryDocument rejects unknown lanes", () => {
+ expect(() => parseStructuredRecallQueryDocument("sql: drop table chunks")).toThrow(SearchError);
+});
+
+test("parseStructuredRecallQueryDocument rejects unterminated quotes", () => {
+ expect(() => parseStructuredRecallQueryDocument('lex: "unfinished')).toThrow(SearchError);
+});
diff --git a/tests/core/search/surfacing-gate.test.ts b/tests/core/search/surfacing-gate.test.ts
new file mode 100644
index 00000000..444cee36
--- /dev/null
+++ b/tests/core/search/surfacing-gate.test.ts
@@ -0,0 +1,39 @@
+import { expect, test } from "bun:test";
+
+import { evaluateSurfacingGate } from "../../../src/core/search/surfacing-gate.ts";
+
+test("evaluateSurfacingGate skips greetings, slash commands, shell-only prompts, and duplicates", () => {
+ expect(evaluateSurfacingGate({ prompt: "hello" }).reason).toBe("greeting");
+ expect(evaluateSurfacingGate({ prompt: "/help" }).reason).toBe("slash_command");
+ expect(evaluateSurfacingGate({ prompt: "git status" }).reason).toBe("shell_command");
+ expect(
+ evaluateSurfacingGate({
+ prompt: "find recall notes",
+ previousPrompt: "Find recall notes",
+ }).reason,
+ ).toBe("duplicate");
+});
+
+test("evaluateSurfacingGate retrieves real memory questions and explicit requests", () => {
+ expect(
+ evaluateSurfacingGate({
+ prompt: "what did we decide about recall diagnostics?",
+ }),
+ ).toEqual({
+ retrieve: true,
+ reason: "memory_question",
+ });
+ expect(evaluateSurfacingGate({ prompt: "hello", explicit: true })).toEqual({
+ retrieve: true,
+ reason: "explicit",
+ });
+});
+
+test("evaluateSurfacingGate fails open for uncertain long questions", () => {
+ expect(
+ evaluateSurfacingGate({
+ prompt:
+ "Can you compare the options we discussed and tell me which implementation path is less risky?",
+ }),
+ ).toEqual({ retrieve: true, reason: "uncertain_question" });
+});
diff --git a/tests/mcp/context-pack-tool.test.ts b/tests/mcp/context-pack-tool.test.ts
index b7c95338..8563ea54 100644
--- a/tests/mcp/context-pack-tool.test.ts
+++ b/tests/mcp/context-pack-tool.test.ts
@@ -104,6 +104,27 @@ describe("brain_context_pack tool — round trip", () => {
expect(items[1]!.id).toBe("pref-supp");
});
+ test("returns polarity lanes when requested", async () => {
+ writeFileSync(
+ join(vault, "Brain", "preferences", "pref-directive.md"),
+ "---\nid: pref-directive\ntopic: t\nprinciple: Prefer concise answers\ntier: core\n---\n",
+ );
+ writeFileSync(
+ join(vault, "Brain", "preferences", "pref-constraint.md"),
+ "---\nid: pref-constraint\ntopic: t\nprinciple: Never expose tokens\ntier: core\n---\n",
+ );
+ const server = new MCPServer({ vault, configPath });
+ await initialize(server);
+ const out = await callPack(server, { max_tokens: 10_000, lanes: true });
+ const lanes = out["lanes"] as {
+ directives: Array<{ id: string }>;
+ constraints: Array<{ id: string }>;
+ };
+
+ expect(lanes.directives.map((item) => item.id)).toContain("pref-directive");
+ expect(lanes.constraints.map((item) => item.id)).toContain("pref-constraint");
+ });
+
test("rejects non-positive max_tokens via INVALID_PARAMS", async () => {
const server = new MCPServer({ vault, configPath });
await initialize(server);
diff --git a/tests/mcp/mcp.test.ts b/tests/mcp/mcp.test.ts
index 4ffe02dd..374d72b8 100644
--- a/tests/mcp/mcp.test.ts
+++ b/tests/mcp/mcp.test.ts
@@ -229,7 +229,8 @@ describe("tool listing", () => {
"payment_request_approval",
"payment_request_status",
"payment_request_consume",
- // Search (added in v0.10.0).
+ // Search (added in v0.10.0; recall gate added in v0.27.0).
+ "brain_recall_gate",
"brain_search",
// Schema admin + watchdog recovery probes.
"get_active_schema_pack",
@@ -480,7 +481,8 @@ describe("stdio loop", () => {
// + brain_sources + brain_switch_vault (v0.22.0) = 46.
// + second_brain_capabilities (v0.23.0) = 47.
// + 9 schema admin tools + brain_watchdog = 57.
- expect(list.result.tools.length).toBe(57);
+ // + brain_recall_gate (v0.27.0) = 58.
+ expect(list.result.tools.length).toBe(58);
});
test("returns parse error for invalid JSON", async () => {
diff --git a/tests/mcp/search.test.ts b/tests/mcp/search.test.ts
index 4e12556b..75d15414 100644
--- a/tests/mcp/search.test.ts
+++ b/tests/mcp/search.test.ts
@@ -130,6 +130,92 @@ test("brain_search happy path returns paths and respects 600-char content cap",
expect((results[0] as Record<string, unknown>)["keywordScore"]).toBeUndefined();
});
+test("brain_search accepts structured query_document", async () => {
+ writeMd("notes/final.md", "# Final\n\nrelease notes mention recall diagnostics.");
+ writeMd("notes/draft.md", "# Draft\n\ndraft release notes mention recall diagnostics.");
+ const cfg = resolveSearchConfig({ vault, configPath });
+ await indexVault(cfg);
+
+ const server = makeServer();
+ await initialize(server);
+ const resp = await call(server, "brain_search", {
+ query: "release notes",
+ query_document: 'lex: "release notes" -draft',
+ limit: 10,
+ });
+ const body = extractToolResult(resp);
+ const results = body["results"] as Array<{ path: string; reasons: string[] }>;
+
+ expect(results.map((result) => result.path)).toContain("notes/final.md");
+ expect(results.map((result) => result.path)).not.toContain("notes/draft.md");
+ expect(results[0]?.reasons.some((reason) => reason.includes("lane:lex/fts5"))).toBe(true);
+});
+
+test("brain_search accepts explicit session focus input", async () => {
+ writeMd("archive/other.md", "# Other\n\nshared recall topic.");
+ writeMd("sessions/focus.md", "# Focus\n\nshared recall topic.");
+ const cfg = resolveSearchConfig({ vault, configPath });
+ await indexVault(cfg);
+
+ const server = makeServer();
+ await initialize(server);
+ const resp = await call(server, "brain_search", {
+ query: "shared",
+ focus_path_prefix: "sessions/",
+ limit: 2,
+ });
+ const body = extractToolResult(resp);
+ const results = body["results"] as Array<{ path: string; reasons: string[] }>;
+
+ expect(results[0]?.path).toBe("sessions/focus.md");
+ expect(results[0]?.reasons.some((reason) => reason.startsWith("session_focus:"))).toBe(true);
+});
+
+test("brain_recall_gate reports skip reasons without affecting explicit brain_search", async () => {
+ writeMd("notes/shell.md", "# Shell\n\ngit status troubleshooting note.");
+ const cfg = resolveSearchConfig({ vault, configPath });
+ await indexVault(cfg);
+
+ const server = makeServer();
+ await initialize(server);
+ const gate = await call(server, "brain_recall_gate", {
+ prompt: "git status",
+ });
+ expect(extractToolResult(gate)).toEqual({
+ retrieve: false,
+ reason: "shell_command",
+ });
+
+ const explicit = await call(server, "brain_search", {
+ query: "git status",
+ limit: 1,
+ });
+ const body = extractToolResult(explicit);
+ const results = body["results"] as Array<{ path: string }>;
+ expect(results[0]?.path).toBe("notes/shell.md");
+});
+
+test("brain_search evidence_pack returns missing terms and why_retrieved", async () => {
+ writeMd("notes/foo.md", "# Foo\n\nalpha beta current support.");
+ const cfg = resolveSearchConfig({ vault, configPath });
+ await indexVault(cfg);
+
+ const server = makeServer();
+ await initialize(server);
+ const resp = await call(server, "brain_search", {
+ query: "alpha gamma",
+ query_document: "lex: alpha",
+ evidence_pack: true,
+ limit: 5,
+ });
+ const body = extractToolResult(resp);
+ const evidencePack = body["evidence_pack"] as { missing_terms: string[] };
+ const results = body["results"] as Array<{ why_retrieved: string[] }>;
+
+ expect(evidencePack.missing_terms).toContain("gamma");
+ expect(Array.isArray(results[0]?.why_retrieved)).toBe(true);
+});
+
test("brain_search rejects missing query with INVALID_PARAMS", async () => {
const server = makeServer();
await initialize(server);
@@ -145,7 +231,10 @@ test("brain_search rejects path_prefix that escapes vault", async () => {
const server = makeServer();
await initialize(server);
- const resp = (await call(server, "brain_search", { query: "A", path_prefix: "../etc/" })) as any;
+ const resp = (await call(server, "brain_search", {
+ query: "A",
+ path_prefix: "../etc/",
+ })) as any;
expect(resp?.error?.code).toBe(-32602);
expect(resp.error.message).toContain("path_prefix");
});
@@ -165,7 +254,10 @@ test("brain_search rejects limit > 50", async () => {
const server = makeServer();
await initialize(server);
- const resp = (await call(server, "brain_search", { query: "A", limit: 100 })) as any;
+ const resp = (await call(server, "brain_search", {
+ query: "A",
+ limit: 100,
+ })) as any;
expect(resp?.error?.code).toBe(-32602);
});