Local-first 🧠memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly 😴 dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.
TypeScript
feat(brain): retrieval precision and quality loop (v1.22.0) (#118)
Sol Aitkena98bed1dab01
diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.jsonindex 567c145e..e9c7b95a 100644--- a/.claude-plugin/plugin.json+++ b/.claude-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.21.0",+ "version": "1.22.0","description": "Plugin-first second brain package for AI agents and humans.","author": {"name": "Open Second Brain contributors"diff --git a/.codex-plugin/plugin.json b/.codex-plugin/plugin.jsonindex f2783853..504d17ef 100644--- a/.codex-plugin/plugin.json+++ b/.codex-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.21.0",+ "version": "1.22.0","description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.","skills": "./skills","hooks": "./hooks/hooks.json",diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 71f494bd..23a7423c 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -5,6 +5,60 @@ All notable changes to this project will be documented in this file.The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).+## [1.22.0] - 2026-07-05++A retrieval-precision and quality-loop pass. Four new surfaces close the+loop between what recall surfaces, how honestly it reports its own+grounding, and how observed failures turn into a prioritized backlog.+Each new field is optional and absent by default, so every call site+whose caller does not opt in produces byte-identical output, and the+kernel still calls no LLM.++### Added++- **Epistemic provenance markers on packed and recalled items.** Every+ item emitted by `brain_context_pack`, `brain_context_receipts`, and+ recall results now carries a structural `epistemic` status+ (`observed` | `derived` | `hypothesis` | `plan` | `unknown`) plus+ `evidence_refs`, derived from existing graph metadata with no manual+ tagging and no natural-language word lists. A consuming model can now+ tell a source-backed fact from a conjecture or a plan. The status is+ language-agnostic by construction; fields are absent when the status+ is `unknown`, so the off-path is byte-identical to today.+- **Recall adequacy verdict with explicit low-adequacy actions.** A thin+ verdict layer classifies each recall attempt as `sufficient`, `weak`,+ or `insufficient` and emits a recommended action (`proceed`,+ `re_recall`, `abstain`) with an optional `escalate` flag, reusing the+ existing gate-telemetry relevance scores plus the new epistemic mix.+ The verdict appears in `brain_recall_gate` output and the+ context-receipts summary when the caller passes recall scores;+ otherwise the field is absent and output is unchanged. Thresholds are+ configurable (`recall_adequacy_sufficient`, `recall_adequacy_weak`,+ `recall_adequacy_min_results`).+- **Cross-query demand log surfacing recurring poorly-answered queries.**+ A rolling, size-capped demand log under `Brain/log/query-demand.jsonl`+ records each recall (normalized terms plus timestamp, result count,+ and the existing coverage score) and an aggregation reader ranks+ recurring queries the vault answers poorly, ranked by frequency+ times (1 minus IDF-weighted coverage). Surfaced read-only as the+ `brain_knowledge_gaps` MCP tool and `o2b brain knowledge-gaps` CLI.+ The log is written only when recall gate telemetry is opt-in (default+ off) and queries are normalized before append; the reader is always+ available and never writes.+- **Unified outcome-tagged lessons loop.** A `Brain/lessons.md` digest+ folds positive knowledge (preferences and their applied evidence) and+ negative knowledge (`violated` / `outdated` rows and `Brain/dead-ends/`)+ into one outcome-tagged corpus scored by a signed, recency-decayed+ weight (shared 30-day half-life from working-memory continuity). Each+ lesson lands in a tier: `preferred` (net-positive and corroborated by+ at least N distinct results), `tentative` (net-positive but under the+ corroboration floor), `contested` (both positive and negative recent+ evidence, recency-wins), or `avoid` (dead-ends and net-negative). The+ digest is regenerated at the tail of every `dream` pass alongside+ `active.md`, exposed as the `osb://lessons` MCP resource, and injected+ on SessionStart and PostCompact. Tunable via `lessons.limit`,+ `lessons.half_life_days`, and `lessons.corroboration_min`.+## [1.21.0] - 2026-07-04A safety-hardening pass around the surfaces that write to or trust the@@ -6118,6 +6172,7 @@ plugin config (vault field)`, and exits with a clear- Sandbox vault and plugin manifest fixtures for tests.- GitHub release workflow for tag-based and manually dispatched releases.+[1.22.0]: https://github.com/itechmeat/open-second-brain/compare/v1.21.0...v1.22.0[1.21.0]: https://github.com/itechmeat/open-second-brain/compare/v1.20.0...v1.21.0[1.20.0]: https://github.com/itechmeat/open-second-brain/compare/v1.19.1...v1.20.0[1.19.1]: https://github.com/itechmeat/open-second-brain/compare/v1.19.0...v1.19.1diff --git a/README.md b/README.mdindex 4dd89985..662d3074 100644--- a/README.md+++ b/README.md@@ -8,7 +8,7 @@ Open Second Brain plugs into [Hermes Agent](https://github.com/NousResearch/herm## What is new-Open Second Brain 1.21.0 is a safety-hardening pass around the surfaces that write to or trust the vault and the codegraph. The secret redactor now enforces a fail-closed 1 MiB scan window instead of silently truncating oversized input (always on), with opt-in infrastructure-topology detectors that extend coverage beyond secrets when enabled. Before the partner surface mutates the codegraph, a read-only health check refuses the write when the index is stale or inconsistent rather than importing into a graph it cannot trust (always on). A report-only hardcoded-path check surfaces path leakage in shipped source without failing on pre-existing hits (always on), and the recall-telemetry cost meter now separates memory write volume from read volume (always on). Two quiet loss and collision gaps close as well: punctuation-only note titles get a deterministic `unnamed-<hash>` slug instead of collapsing into one shared note (always on), and session provenance is recorded as a stable logical key so a vault moved between machines keeps its lookups (always on). The opt-in pieces (the extra infra-topology detectors) are off by default and byte-identical when the new path is unused, and the kernel still calls no LLM.+Open Second Brain 1.22.0 closes the retrieval quality loop on four surfaces. Packed and recalled items now carry a structural epistemic status (observed, derived, hypothesis, plan, or unknown) plus evidence refs, derived from existing graph metadata so a consuming model can tell a source-backed fact from a conjecture (always on, fields absent when unknown). A thin recall adequacy verdict classifies each recall as sufficient, weak, or insufficient with a recommended action (proceed, re-recall, abstain) and an optional escalate flag, surfaced in recall gate output and context receipts when the caller passes scores (thresholds configurable). A persisted cross-query demand log records each recall and an aggregation reader ranks recurring queries the vault answers poorly, exposed as the brain_knowledge_gaps tool and o2b brain knowledge-gaps (the log is written only when recall gate telemetry is opt-in, queries normalized before append). A unified lessons digest folds positive knowledge and dead-ends into one signed, recency-scored, corroboration-tiered corpus under Brain/lessons.md, regenerated every dream pass and loaded on session start. Every new field is optional and absent by default, so callers that do not opt in see byte-identical output, and the kernel still calls no LLM.## Whydiff --git a/docs/brainstorm/retrieval-precision-quality-loop/cli-output/claude.md b/docs/brainstorm/retrieval-precision-quality-loop/cli-output/claude.mdnew file mode 100644index 00000000..938ede38--- /dev/null+++ b/docs/brainstorm/retrieval-precision-quality-loop/cli-output/claude.md@@ -0,0 +1,32 @@+## Variant 1+Approach: Introduce one shared per-item annotation envelope early on the branch — a `RetrievalAnnotations` extension threaded through `context-pack.ts`/`context-receipts.ts` and the reader tail — and have every card attach its field to that single struct (A: rerank score, C: epistemic status + `evidence_refs`, D: adequacy verdict, E: satisfaction signal). Cards land in sequence, each extending the same envelope rather than opening a new output seam. B folds its outcome tag onto the same annotation vocabulary.+Trade-offs:+- Cleanest composition: C→D and A→enrich share one growth point, so the C/D output-shape collision and the A/rerank-tail collision collapse into "extend the struct."+- Broad blast radius on the shared type; a defect in the envelope touches all five read paths at once.+- Per-card bit-identity is harder to verify: each card mutates a struct other cards also read, so the off-branch proof is no longer a single guarded call site but "this field defaults absent and nothing consumes it."+- Front-loads an abstraction before the second consumer exists, risking speculative shape.+Complexity: large+Risk: medium++## Variant 2+Approach: Keep each card a self-contained stage with its own module, its own default-off config key, and one guarded call site, minimizing shared types. Resolve the real collisions purely by ordering so the later card extends an already-merged seam: A → C → D → E → B. A appends after the existing heuristic reranks in the reader tail; C adds the epistemic field to context items; D extends C's now-merged item shape with the verdict and reads gate-telemetry/coverage; E instruments the recall path D already touched; B lands last as the capstone assembly over a fully-merged branch.+Trade-offs:+- Maximal per-card bit-identity: every off-branch is a single guarded call site whose unchanged output is trivially verifiable.+- Smallest blast radius per card; a regression is isolated to one stage.+- Directly mirrors invariant #6 and the demonstrated one-PR-many-cards cadence — no up-front shared abstraction to design or defend.+- Mild duplication: C's status and D's verdict are emitted as adjacent-but-separate fields; B and E each own their persistence (`Brain/lessons/*` vs `Brain/demand-log/*`) with no shared writer.+Complexity: medium+Risk: low++## Variant 3+Approach: Land the two persistence substrates and a shared recall-event record first — B's outcome-tagged lessons corpus and E's demand log both emitted as continuity/`gate_telemetry`-style records — then rebuild the read-side cards (A, C, D) as consumers of that common event stream. D and E share one recall-event emission; B and E share the record substrate; the dream pass folds everything.+Trade-offs:+- Resolves the B/E persistence collision and the D/E recall-instrumentation collision via one shared emission point.+- Inverts the natural read→write flow, forcing the persistence/event layer to be designed before its consumers exist.+- Centralized emission on the hot path stresses the fail-open telemetry contract and Syncthing determinism, and makes the off-branch bit-identity proof depend on the whole bus being inert, not a local call site.+- Largest blast radius and the weakest fit with driving cards one at a time — the first card carries most of the risk for cards that land later.+Complexity: large+Risk: high++## Recommended: Variant 2+Variant 2 is the only option whose off-branch state is provable the way invariant #2 asks for it — a single guarded call site per card, each off-branch byte-identical to today — because it refuses a shared mutable struct or a shared emission bus that would make bit-identity a whole-subsystem property. The ordering A → C → D → E → B is exactly invariant #6 in practice: A is pure search-side with no vault writes and lowest blast radius, so it seeds the cross-encoder-provider pattern first (mirroring `embeddings/registry.ts`) without touching any other card; C then establishes the epistemic field on context items so D extends an already-merged shape rather than colliding on `context-pack.ts`/`context-receipts.ts`; D reuses C's status plus the existing `gate-telemetry`/`coverage` scores as a thin verdict+action layer (honoring its "do not build a separate attention organ" guardrail); E appends its demand log at the recall seam D has already opened, reusing `coverage.ts`'s IDF score as the satisfaction axis; and B lands last as the heaviest assembly, unifying `dead-ends.ts`/`preference.ts` under the `usage-signal.ts` decay scorer and the dream pass over a fully-merged branch. This maximizes reuse of existing scoring/telemetry/coverage primitives, keeps every decision in-kernel deterministic (the cross-encoder stays an opt-in external call, graceful no-op when unset), and matches the repository's proven multi-card-suite cadence. Variant 1's shared envelope and Variant 3's event bus both trade that per-card verifiability and small blast radius for a composition elegance the suite does not need — and both raise, rather than lower, the risk to the load-bearing bit-identity and fail-open invariants.diff --git a/docs/brainstorm/retrieval-precision-quality-loop/cli-output/prompt.md b/docs/brainstorm/retrieval-precision-quality-loop/cli-output/prompt.mdnew file mode 100644index 00000000..e25a14aa--- /dev/null+++ b/docs/brainstorm/retrieval-precision-quality-loop/cli-output/prompt.md@@ -0,0 +1,222 @@+# Architecture consultation - retrieval-precision-quality-loop++You are a senior software architect advising on the design of a new feature suite for an+existing TypeScript project. Produce **exactly 3 distinct architectural variants** and then+**exactly one recommendation**. Variants and recommendation only - no code, nothing outside+those sections.++## Output format (follow EXACTLY)++```+## Variant 1+Approach: <2-3 sentences>+Trade-offs:+- <bullet>+Complexity: small | medium | large+Risk: low | medium | high++## Variant 2+Approach: <2-3 sentences>+Trade-offs:+- <bullet>+Complexity: small | medium | large+Risk: low | medium | high++## Variant 3+Approach: <2-3 sentences>+Trade-offs:+- <bullet>+Complexity: small | medium | large+Risk: low | medium | high++## Recommended: Variant N+<rationale - why this variant over the others, in the context of the project conventions below>+```++Do not include code, pseudo-code, or file listings. No preamble, no closing remarks.++## Project++- **Name**: Open Second Brain (package `open-second-brain`, v1.21.0).+- **Language/runtime**: TypeScript (strict), runs on Bun. Obsidian-native second-brain memory+ layer for AI agents. Plain Markdown vault under `Brain/`. Ships a deterministic CLI (`o2b`),+ an MCP server, and native plugins (Hermes/Claude Code/Codex/OpenClaw/opencode/Grok).+- **Invariants you MUST honor** (load-bearing conventions; violating any breaks the release):+ 1. **The kernel calls no LLM.** Every decision in search/recall/ranking/lessons logic is+ deterministic. An optional cross-encoder endpoint is an external service call, not an+ in-kernel model - it must be opt-in and a graceful no-op when unconfigured.+ 2. **Default-off switches are bit-identical to today.** Any ranking/score/cost change must+ produce byte-identical results when the new path is off. The easiest property to verify is+ a single guarded call site whose off-branch is unchanged.+ 3. **Telemetry is fail-open** via `emitGatedTelemetry` - never throws into the hot path.+ 4. **Language-agnostic.** No hardcoded natural-language word lists (stopwords, keywords,+ negations) in any language. Use structural signals, frontmatter fields, corpus document+ frequency / IDF, or explicit agent/LLM extraction.+ 5. **Syncthing-peer determinism.** Anything written to the vault is deterministic across+ machines (stable sort keys, stable slugs, no wall-clock-only tiebreaks).+ 6. **One release = one PR = many cards, driven ONE AT A TIME on a shared branch.** Each card+ builds on commits the previous cards already landed. Non-conflict is achieved by ordering+ the real collision pairs so the later card extends an already-merged seam.+ 7. **Markdown-first persistence.** New knowledge (lessons, demand logs) is plain `.md` /+ JSON sidecar under `Brain/` so FTS indexes it and the operator can grep/edit it.+ 8. No TypeScript cast crutches (`as unknown as T`). Build correct shapes with conditional+ spreads / narrowing. English-only strings in code; abstract multi-language.++## Release scope: retrieval-precision-quality-loop++Closing the full recall/search quality loop: rank (cross-encoder) -> verify (epistemic+provenance) -> observe (adequacy verdict + cross-query demand log) -> learn (outcome-tagged+lessons loop). Five cards ship together as one PR on branch+`feat/retrieval-precision-quality-loop`, driven one at a time:++### Card A (t_110867f5, p4) - Optional pluggable cross-encoder rerank stage for the search reader+OSB has hybrid search (rank-fusion of semantic + keyword) and several *heuristic* rerank stages+(relevance rerank, entity-match boost, MMR diversity, usage/activation decay) but no learned+cross-encoder re-scoring pass. Add an optional, opt-in cross-encoder rerank stage+(OpenAI-compatible / pluggable endpoint) that re-scores the top-K fused candidates as a final+reader step. Mirror the existing embedding-provider config pattern (key + endpoint, graceful+no-op when unset) so the stage is zero-cost when disabled. Default behavior unchanged when no+rerank endpoint is configured.++### Card B (t_62363378, p4) - Outcome-tagged, recency-scored lessons loop unifying dead-ends and preferences with corroboration tiering+A closed-loop outcome-driven feedback system over one artifact class (saved results/Q&A): an+agent stamps each saved result with an outcome (`useful | dead_end | corrected`), and a+deterministic aggregation pass folds those outcomes into a single `LESSONS` file using a SIGNED,+recency-decayed score (configurable half-life). A node is promoted to PREFERRED only once+corroborated by >=N distinct results, otherwise TENTATIVE, with mixed-signal nodes rendered as+CONTESTED (recency-wins). The lessons file is auto-loaded at session start and auto-refreshed.+**Assembly/integration gap, not from-scratch**: OSB already owns negative knowledge+(`src/core/brain/dead-ends.ts`, bounded active set N=100), positive knowledge (preferences with+confidence band + `retired` state via the dream pass), AND a signed recency-decay scorer+(`src/core/brain/continuity/usage-signal.ts` `decayWeight()`/`rankByUsageDecay()`, exponential+half-life default 30 days). The delta is (1) collapsing the two registries into one+outcome-tagged corpus, (2) reusing usage-signal-style scoring as the promotion signal with an+explicit corroboration threshold, (3) a single auto-loaded/auto-refreshed lessons digest. Watch+for overlap with the existing dream pass (repeat signals -> rules, retire stale) - the+improvement should extend that machinery (signed + corroboration-gated tiers) rather than+introduce a parallel loop. Auto-load maps onto OSB's existing `active.md`/SessionStart-hook+surface and the Hermes cron (dream runs on cron), not git hooks.++### Card C (t_e163feb2, p3) - Epistemic provenance markers on recalled/packed items+Attach an epistemic status (`Observed | Derived | Hypothesis | Plan | Unknown`) + `evidence_refs`+to each item emitted by `brain_context_pack` (and recall results). Today `brain_truth` /+`brain_derive_fact` exist but "derived" means preference keys (`normalizeDerivedKeys`), not an+epistemic status on context items. `evidence_ref` = 0 hits, `epistemic` = 0 hits in src.+`brain_context_pack` / `brain_context_receipts` emit structured context without a per-item+epistemic status. Derive the status from existing graph metadata (source-backed vs inferred vs+preference/hypothesis tiers) - no manual tagging. Lets downstream reasoning (and the+recall-adequacy gate, Card D) distinguish grounded facts from conjecture, cutting confident+fabrication. Pairs with Card D.++### Card D (t_b8f66fec, p3) - Recall adequacy verdict with explicit low-adequacy actions (abstain / re-recall / escalate)+On top of the existing gate telemetry (`brain_recall_gate` + `src/core/brain/gate-telemetry.ts`+emit gate/relevance scores; `brain_recall_telemetry` / `brain_recall_feedback` exist), classify+each recall result (`sufficient | weak | insufficient`) and wire explicit outcomes:+- `sufficient` -> proceed+- `weak` -> automatic re-recall via an alternate strategy (broaden scope / different path)+ before answering+- `insufficient` -> **abstain** (return an explicit "insufficient grounding" signal rather than a+ low-confidence answer) and/or **escalate** (flag for review / surface to operator)+Expose the verdict in `brain_recall_gate` output and `brain_context_receipts` so callers can+branch on it. **Scope guardrail**: reuse existing telemetry scores; thresholds configurable. Do+NOT build a separate "attention organ" - this is a thin verdict + action layer over what already+exists. Complements Card C (that feeds this gate).++### Card E (t_97091fff, p3) - Cross-query demand log surfacing queries that are asked often but recall poorly+Log every recall query against the vault together with a deterministic satisfaction signal+(result count + the IDF-weighted coverage score already computed per query in+`src/core/search/coverage.ts`), then aggregate over time to surface queries that recur+frequently but consistently return weak/empty results. This is a cross-query *demand* signal -+"what do operators keep asking that the Brain can't answer" - distinct from OSB's existing+structural gaps (dangling wikilink targets, `deep-synthesis.ts:407` iterates `report.gaps`) and+from its in-the-moment per-query coverage check (`coverage.ts:141` `planTargetedRetry`, scoped to+the current query, not persisted/aggregated). **Keep it LLM-free and deterministic**: append each+recall to a rolling log under `Brain/` (query text or normalized terms + timestamp + result count++ coverage score), then provide an aggregation (a `knowledge_gaps`-style read tool / CLI) that+buckets by normalized query and ranks by frequency x low-satisfaction. **Watch privacy/size**:+queries may contain sensitive text - normalize/redact and cap/rotate the log. Reuse the existing+coverage score as the "answered poorly" axis rather than inventing a new metric.++## Key existing files / surfaces (read-only references)++- `src/core/search/fusion.ts` - rank-fusion (`linear` weighted-sum + `rrf` reciprocal-rank),+ min-max-normalised fused relevance on [0,1] composing with downstream boosts.+- `src/core/search/ranker.ts` - primary scorer (BM25 + cosine + intent multipliers + boosts).+- `src/core/search/enrich.ts` - heuristic reranks: `rerankByRelevance` (core textual relevance =+ keyword + semantic lane contributions, ignoring recency/usage/link boosts); `deriveTrust`+ (age_days, superseded, conflict).+- `src/core/search/coverage.ts` - single source of truth for query-term verification:+ `significantTerms` (no stopword list, language-agnostic), `idfForTerm` (smoothed IDF:+ `ln(1 + N/(1+df))`), `isRareTerm`, `buildCoverageReport`, `planTargetedRetry` (deterministic+ follow-up over uncovered RARE terms, pure verdict over an already-built report).+- `src/core/search/embeddings/` - provider pattern to mirror for the cross-encoder:+ - `registry.ts`: named provider profiles persisted to `Brain/search/embedding-providers.json`+ (CLI-added, fail-soft load), stores only env-key NAMES never secret values, safe to sync.+ - `openai-compat.ts`: OpenAI-compatible `/v1/embeddings` provider (base_url + model ++ env-resolved api key, fail-closed validation when semantic enabled, HTTP client with caching).+ - `provider.ts` / `registry.ts`: `expandRegisteredProvider` resolves a registered name to+ `openai-compat` config at config-resolution time, after built-ins.+- `src/core/search/profiles.ts` - search profiles ("thorough" etc.); cross-encoder most valuable+ for the thorough profile where recall quality matters most.+- `src/core/brain/dead-ends.ts` - negative-knowledge class: one `.md` note under+ `Brain/dead-ends/` (approach + why-failed + context), bounded active set N=100, overflow+ archives oldest to `Brain/dead-ends/archive/`. `sanitiseTextField`, `writeFrontmatterAtomic`.+- `src/core/brain/preference.ts` + `src/core/brain/apply-evidence.ts` - positive knowledge:+ signals -> rules via the dream pass, confidence/confidence_value band, `last_evidence_at`,+ `retired` state (`Brain/retired/`).+- `src/core/brain/continuity/usage-signal.ts` - SIGNED recency-decay scorer: `decayWeight()`+ (exponential half-life, `DEFAULT_HALF_LIFE_DAYS = 30`, configurable), frequency gain,+ `rankByUsageDecay()`. Currently over continuity records ranked by recall telemetry, NOT over+ an outcome-tagged corpus.+- `src/core/brain/context-pack.ts` - `packContext()` emits `ContextPackItem[]` with `tier`+ (core/supporting/peripheral), bounded-token vault slice. Each item extends+ `ContextTransformAnnotations`.+- `src/core/brain/context-receipts.ts` - `emitContextReceipt` / `listContextReceipts` /+ `summarizeContextReceipt` over continuity records.+- `src/core/brain/gate-telemetry.ts` - `emitGateTelemetry` (records every automatic-recall gate+ decision as a `gate_telemetry` continuity record; never stores raw prompt, only SHA-256 prefix+ + length; default OFF behind `recall_gate_telemetry` config key).+- `src/core/brain/deep-synthesis.ts` - `report.gaps` iterates dangling-wikilink targets+ (structural gaps only); corroboration concept appears only narratively at line 171.+- `src/core/brain/active.ts` - `Brain/active.md` auto-loaded at session start via SessionStart+ hook; auto-generated by `dream`. (Reusable surface for a lessons digest auto-load.)+- `src/core/brain/dream.ts` / `dream-refresh.ts` / `dream-stage.ts` - the deterministic dream+ pass (repeat signals -> rules, retire stale) that runs on the Hermes cron.++## Git log (last 20, most recent first) - shows the one-PR-themed-suite cadence++```+4281605 feat(brain): integrity & safety hardening suite (1.21.0) (#115)+313d061 feat: configurable skills_dir + trigger-keyword auto-attach scoring (#114)+a3ea315 fix: v1.19.1 - cross-vault cards, event-trace exit codes, registry-guard hygiene (#113)+bb5f320 feat(brain): session-boundary capture durability and post-compaction pinned-anchor survival audit (v1.19.0) (#112)+c5e30b8 fix: cross-vault chain-stop reads the max normalized score (v1.18.1) (#111)+33b4fba feat: recall precision, coverage, and provenance hardening (v1.18.0) (#110)+254b580 feat: codegraph link-graph depth and MCP exposure (v1.17.0) (#108)+da2e3cc feat: memory subsystem alignment - honest pinned budgets, atomic batch writes, on_memory_write host bridge (v1.16.0) (#107)+4db7862 fix(hermes): pass --repo so bridge skill discovery resolves repoRoot (#103) (#106)+0a4b6da feat: calendar obligations, agenda synthesis, OKF portability, Obsidian Bases and steelman synthesis (v1.15.0) (#105)+f8b4abf feat(brain): add feedback default scope and vault write containment (#104)+20ea7ef feat: per-handoff LLM generation tracing and prompt-prefix stability metric (#102)+9c1d48f feat: CodeGraph and MCP operational readability (v1.12.0) (#101)+c2c3ff4 feat: Session Knowledge Synthesis Suite - structured session summaries, idea-lineage, episodic note history (v1.11.0) (#100)+56dd3dd fix(hermes): bridge EOF - byte streams, stderr drain, retry loop (#92)+35b824e feat: Recall & Working-Memory Quality Suite - selectable profiles, usage decay, co-occurrence, file-context (v1.10.0) (#99)+929d54c feat: Brain Portability & Interop Suite - bank export/import, page contract, brain_create_note, in-process SDK (v1.9.0) (#98)+7cdbfc0 feat: Indexer Durability & Resilience Suite - cooperative abort, graceful watch shutdown, resumable reindex (v1.8.0) (#97)+8b679fe feat: Knowledge Provenance Suite - ingest, research, NER, derived facts, owner-scope, standing-query (v1.7.0) (#96)+6e59a42 feat: Vault Integrity & Trust Suite - untrusted-source containment, NFC identity, watch-sync, O(1) graph, agent-scope (v1.6.0) (#95)+```++## What to design++An architecture that lands all five cards as one themed suite on the shared branch, each behind+its own explicit default-off switch whose off-branch is bit-identical to today, driven one at a+time in a non-conflicting order, honoring every invariant above. Identify the real cross-card+collision points (e.g. Card C and Card D both touch `context-pack.ts`/`context-receipts.ts`+output shape; Card A and the existing heuristic reranks both touch the reader tail;+Cards B/E both add new persistence under `Brain/`; Card D re-uses Card C's status and+`gate-telemetry`/`coverage` scores) and say how each variant resolves them. Weigh: blast radius,+per-card bit-identity verifiability, LLM-free-kernel preservation, reuse of existing+scoring/telemetry/coverage primitives, and fit with the demonstrated multi-card-suite release+cadence.diff --git a/docs/brainstorm/retrieval-precision-quality-loop/design.md b/docs/brainstorm/retrieval-precision-quality-loop/design.mdnew file mode 100644index 00000000..52cd27d7--- /dev/null+++ b/docs/brainstorm/retrieval-precision-quality-loop/design.md@@ -0,0 +1,198 @@+# Design - retrieval-precision-quality-loop++Branch: `feat/retrieval-precision-quality-loop`+Slug: `retrieval-precision-quality-loop`+Cards: A `t_110867f5`, B `t_62363378`, C `t_e163feb2`, D `t_b8f66fec`, E `t_97091fff`++## Problem++Open Second Brain's recall/search pipeline is rich (hybrid rank-fusion, heuristic reranks,+IDF-weighted coverage, gate telemetry, continuity records, dream-promoted preferences,+dead-ends) but the full quality loop is not yet closed. Five concrete gaps leave recall less+precise, less honest about its own grounding, and less able to turn observed failures into+improvements:++1. **No learned re-scoring pass.** Heuristic reranks (relevance, entity boost, MMR, usage+ decay) order the top-K, but no cross-encoder jointly re-scores query/document pairs, so+ precision-at-top stays bounded by what lexical + cosine heuristics can express.+2. **Context items have no epistemic status.** `brain_context_pack` / `brain_context_receipts`+ emit structured context but the consuming model cannot tell a source-backed fact from a+ conjecture or a plan, so it reasons over all of it with the same confidence.+3. **No adequacy verdict that drives behavior.** Gate telemetry emits scores, but there is no+ classification that says "this recall is too weak to answer from" and then abstains or+ re-recalls; the system hands top-K to the model regardless of fitness.+4. **No cross-query demand signal.** Per-query coverage and structural dangling-link gaps+ exist, but nothing persists or aggregates which queries recur yet consistently answer+ poorly, so the operator never sees the prioritized backlog of "what to write next".+5. **Lessons are split across two registries.** Negative knowledge (dead-ends) and positive+ knowledge (dream-promoted preferences) are separate registries with a coarse confidence+ band and no signed recency-decayed scoring or corroboration gate over a single outcome+ corpus, so a fresh dead-end does not outweigh a stale "useful", and a one-off outcome can+ become a rule.++## Scope++All five cards ship in this release, driven one at a time on the shared branch in blast-radius+order so each later card extends an already-merged seam rather than colliding:++- **A** `t_110867f5` - optional, opt-in cross-encoder rerank stage (OpenAI-compatible / pluggable+ endpoint) re-scoring the top-K fused candidates as a final reader step.+- **C** `t_e163feb2` - epistemic provenance markers (`Observed | Derived | Hypothesis | Plan |+ Unknown` + `evidence_refs`) on each item emitted by `brain_context_pack` and recall results.+- **D** `t_b8f66fec` - recall adequacy verdict (`sufficient | weak | insufficient`) with explicit+ low-adequacy actions (proceed / re-recall / abstain / escalate), a thin layer over existing+ telemetry + coverage scores plus C's epistemic mix.+- **E** `t_97091fff` - persisted cross-query demand log + aggregation reader surfacing queries+ asked often but answered poorly, reusing the existing coverage score as the satisfaction axis.+- **B** `t_62363378` - outcome-tagged, recency-scored lessons loop unifying dead-ends and+ preferences with corroboration tiering, reusing the `usage-signal.ts` decay scorer and the+ dream pass.++**Drive order:** A → C → D → E → B.++## Out of scope++- A learned model living inside the kernel. The cross-encoder is an opt-in external service+ call, graceful no-op when unconfigured; every other decision (adequacy verdict, corroboration+ tier, demand aggregation, epistemic derivation) stays deterministic and LLM-free.+- Any change to default ranking/score/output when a feature is off. Each card is a single+ guarded call site whose off-branch is byte-identical to today.+- A web dashboard. Every new surface ships as CLI / MCP reader or as plain Markdown under+ `Brain/`.+- Manual epistemic tagging. C derives status from existing graph metadata only.+- A separate "attention organ" for adequacy. D is a thin verdict + action layer over what+ already exists, per its own scope guardrail.+- Migrating or rewriting the existing dead-ends or preference registries. B collapses them into+ one outcome-tagged view and extends the dream pass; it does not delete or rewrite the source+ notes.+- HyDE-style generative query rewriting or learned query expansion. D's re-recall uses the+ existing deterministic expansion / alternate-strategy machinery.++## Chosen approach++**Variant 2 - each card a self-contained stage behind its own default-off switch, ordered so the+later card extends an already-merged seam (A → C → D → E → B).**++Every card lands as its own module with its own config key and exactly one guarded call site+whose off-branch is provably bit-identical to today. This is the only variant whose off-branch+state is verifiable the way the project demands (a single guarded call site per card, not a+shared mutable struct or a shared emission bus that turns bit-identity into a whole-subsystem+property). Cross-card collisions are resolved by the cheapest sufficient mechanism - ordering -+rather than by a speculative up-front abstraction:++- **A → heuristic reranks** (both touch the reader tail): A appends after the existing heuristic+ reranks as an isolated final stage; it does not alter them.+- **C → D** (both touch `context-pack.ts` / `context-receipts.ts` output shape): C establishes+ the epistemic field on context items; D extends C's now-merged item shape with the verdict+ instead of colliding on the same files.+- **D → E** (both touch the recall path): D opens the recall seam with its verdict + action+ layer; E appends its demand-log instrumentation at that seam, reusing the coverage score D+ already surfaces.+- **B ↔ E** (both add persistence under `Brain/`): independent directories+ (`Brain/lessons/*` vs `Brain/recall/demand-log*`), no shared writer; B lands last over a+ fully-merged branch so a later shift to the recall/telemetry surfaces never forces a rebase.++Telemetry on every card routes through `emitGatedTelemetry` and stays fail-open. The kernel+calls no LLM.++## Design decisions++### A - cross-encoder rerank (search-side, no vault writes)++Mirror the existing embedding-provider pattern (`src/core/search/embeddings/registry.ts` ++`openai-compat.ts`): a `rerank/` module with a named-provider registry persisted to+`Brain/search/rerank-providers.json` (env-key NAMES only, fail-soft load) and an+OpenAI-compatible rerank provider (base_url + model + env-resolved api key, fail-closed+validation when enabled-but-unconfigured). The stage re-scores the top-K fused candidates as a+final reader step appended after the heuristic reranks. Config surface (`search_rerank_*`+keys: enabled, model, base_url, env_key, top_k, min_score) resolves through+`resolveSearchConfig` (config + env, with validation), and is most valuable on the "thorough"+search profile. When disabled (the default), the stage returns its input unchanged - bit+identical ordering, zero HTTP cost. On endpoint error it degrades gracefully to the+heuristic-ordered input and logs via `emitGatedTelemetry` (fail-open), never throwing into the+hot path.++### C - epistemic provenance (read-time derivation, optional fields)++A pure `epistemic-status.ts` under `src/core/brain/provenance/` derives an `EpistemicStatus`+(`observed | derived | hypothesis | plan | unknown`) plus `evidence_refs` from existing graph+metadata: a source-backed note/wikilink -> `observed` (carry the source as evidence_ref); an+inferred/derived fact -> `derived` (link the basis); a preference/hypothesis tier ->+`hypothesis`; an intended/not-yet-true item -> `plan`; an acknowledged gap -> `unknown`. No+manual tagging and no natural-language word lists (language-agnostic by construction). The+fields are added to `ContextPackItem` and recall results as **optional** absent-by-default+fields, so the off-branch (feature flag off) produces byte-identical pack/receipt output. D+reads this status.++### D - adequacy verdict (thin layer over existing scores)++A pure `adequacy.ts` under `src/core/brain/recall/` classifies each recall into+`sufficient | weak | insufficient` and emits a recommended action+(`proceed | re_recall | abstain | escalate`), reusing the existing `gate-telemetry` relevance+scores, the `coverage.ts` IDF-weighted coverage, and C's epistemic mix (a recall that is mostly+`hypothesis`/`unknown` scores lower fitness than one grounded in `observed` facts). Thresholds+are configurable. The verdict + action are exposed in `brain_recall_gate` output and the+`context-receipts` summary so callers can branch on them. `weak` triggers an alternate-strategy+re-recall (broaden scope / different path) via existing recall machinery; `insufficient`+returns an explicit "insufficient grounding" abstain signal and/or an escalate flag instead of+a low-confidence answer. This honors the card's guardrail: reuse existing telemetry, do NOT+build a separate attention organ.++### E - cross-query demand log (LLM-free, privacy-capped persistence)++A `demand-log.ts` under `src/core/brain/recall/` appends each recall (normalized terms or a+redacted query form + timestamp + result count + the coverage score already computed per query+in `coverage.ts`) to a rolling, size-capped log under `Brain/recall/demand-log.jsonl`.+Privacy/size discipline: queries may contain sensitive text, so the query is normalized/redacted+before append and the log is capped/rotated. An aggregation reader (a `knowledge_gaps`-style+read tool / CLI) buckets by normalized query and ranks by frequency x low-satisfaction,+reusing the existing coverage score as the "answered poorly" axis rather than inventing a new+metric. Appends at the recall seam D has already opened. Deterministic across Syncthing peers.++### B - outcome-tagged lessons loop (assembly over existing primitives)++A `lessons/` module under `src/core/brain/` unifies `dead-ends.ts` (negative outcomes) and+`preference.ts`/`apply-evidence.ts` (positive outcomes) into one outcome-tagged corpus+(`useful | dead_end | corrected`), reusing `usage-signal.ts` `decayWeight()` (configurable+half-life, signed) as the promotion signal with an explicit corroboration threshold: a node is+`PREFERRED` only once corroborated by >=N distinct results, otherwise `TENTATIVE`, with+mixed-signal nodes rendered `CONTESTED` (recency-wins). A deterministic aggregation pass+folds the corpus into a single `Brain/lessons/LESSONS.md` digest that is auto-loaded at session+start (extending the existing `active.ts` / SessionStart-hook surface) and auto-refreshed by+the dream cron (extending `dream-refresh.ts`). This extends the dream pass with signed ++corroboration-gated tiers rather than introducing a parallel loop. It lands last over a+fully-merged branch because it is the heaviest assembly and depends on the recall/telemetry+surfaces not shifting underneath it.++## File changes (per card, indicative)++- **A**: new `src/core/search/rerank/{cross-encoder,registry,provider}.ts`; config keys via+ `resolveSearchConfig`; one guarded call site in the reader tail; MCP/CLI exposure for provider+ registration (mirror embedding registry).+- **C**: new `src/core/brain/provenance/epistemic-status.ts`; optional `epistemic_status` ++ `evidence_refs` fields on `ContextPackItem` and recall result type; derivation wired into+ `context-pack.ts` / recall rendering behind a feature flag.+- **D**: new `src/core/brain/recall/adequacy.ts`; verdict + action added to `brain_recall_gate`+ output and `context-receipts` summary; re-recall trigger and abstain/escalate paths behind a+ feature flag.+- **E**: new `src/core/brain/recall/demand-log.ts` (append + rotate/cap + redact) and an+ aggregation reader (CLI/MCP); append wired at the recall seam behind a feature flag.+- **B**: new `src/core/brain/lessons/{outcome-corpus,score,digest}.ts`; `active.ts` /+ `dream-refresh.ts` extension for auto-load + auto-refresh; corroboration-tiered dream pass+ extension behind a feature flag.++## Risks++- **Per-card bit-identity under shared output shapes (C/D).** Mitigated by landing C before D so+ D extends a merged shape, and by making every new field optional/absent-by-default so the+ off-branch is byte-identical. Acceptance tests assert off-branch output equality.+- **Cross-encoder latency/availability leaking into the hot path (A).** Mitigated by default-off,+ a strict timeout, and fail-open degradation to heuristic-ordered input on endpoint error; zero+ HTTP cost when unconfigured.+- **Demand-log privacy growth (E).** Mitigated by query normalization/redaction before append+ and size cap/rotation; the log is operator-inspectable Markdown/JSONL so it can be audited.+- **Lessons loop double-counting the dream pass (B).** Mitigated by extending the dream pass+ (signed + corroboration-gated) rather than running a parallel aggregation; the corroboration+ threshold (>=N distinct results) prevents one-off outcomes from promoting.+- **Syncthing-peer determinism (B, E).** Mitigated by stable sort keys, stable slugs, and+ deterministic aggregation (no wall-clock-only tiebreaks) in every vault write.diff --git a/docs/brainstorm/retrieval-precision-quality-loop/plan.md b/docs/brainstorm/retrieval-precision-quality-loop/plan.mdnew file mode 100644index 00000000..00fe893e--- /dev/null+++ b/docs/brainstorm/retrieval-precision-quality-loop/plan.md@@ -0,0 +1,190 @@+# Plan - retrieval-precision-quality-loop++Branch: `feat/retrieval-precision-quality-loop`+Slug: `retrieval-precision-quality-loop`+Combined design: `docs/brainstorm/retrieval-precision-quality-loop/design.md`+Drive order: **A → C → D → E → B** (blast-radius order; each later card extends an already-merged+seam). Cards are driven ONE AT A TIME on the shared branch. Each worker builds on the commits the+previously-driven in-scope cards already landed and must not duplicate or conflict with sibling+tasks.++Global conventions for every card:++- TDD: write the failing test first, then implement until it passes. Run `bun run validate`+ (typecheck + lint + test) before claiming the card done.+- Default-off switch per card (one config key), off-branch byte-identical to today.+- Telemetry via `emitGatedTelemetry`, fail-open.+- Kernel calls no LLM. The cross-encoder (A) is an opt-in external service call.+- No TypeScript cast crutches; build correct shapes with conditional spreads / narrowing.+- English-only strings in code; logic is language-agnostic (no hardcoded NL word lists).++In-scope cards shipping together in this one release:++| id | card | one-line title |+|---|---|---|+| `t_110867f5` | A | Optional pluggable cross-encoder rerank stage for the search reader |+| `t_e163feb2` | C | Epistemic provenance markers on recalled/packed items |+| `t_b8f66fec` | D | Recall adequacy verdict with explicit low-adequacy actions |+| `t_97091fff` | E | Cross-query demand log surfacing queries asked often but recall poorly |+| `t_62363378` | B | Outcome-tagged, recency-scored lessons loop unifying dead-ends and preferences with corroboration tiering |++---++## A (t_110867f5) - Optional pluggable cross-encoder rerank stage++### Files+- `src/core/search/rerank/cross-encoder.ts` - the rerank provider (OpenAI-compatible endpoint:+ base_url + model + env-resolved api key, re-scores top-K query/doc pairs).+- `src/core/search/rerank/registry.ts` - named-provider registry persisted to+ `Brain/search/rerank-providers.json` (env-key NAMES only, fail-soft load), mirroring+ `embeddings/registry.ts`.+- `src/core/search/rerank/provider.ts` - provider interface + built-in/default resolution.+- `src/core/search/rerank/index.ts` - `applyCrossEncoderRerank(results, query, config)` final+ reader step; returns input unchanged when disabled or on endpoint error.+- config keys threaded through `resolveSearchConfig`: `search_rerank_enabled` (default false),+ `search_rerank_model`, `search_rerank_base_url`, `search_rerank_env_key`, `search_rerank_top_k`,+ `search_rerank_min_score`, with env parity + validation.+- one guarded call site in the reader tail (after `enrich.ts` heuristic reranks).+- MCP/CLI exposure for provider registration (mirror the embedding registry CLI verbs).+- tests under the module's sibling `.test.ts` (follow repo convention).++### Acceptance (a passing test)+- Disabled (default): the reader tail output is byte-identical to the pre-feature baseline for+ the same query/results (snapshot/diff equality asserted in a test).+- Enabled + endpoint unconfigured: validation fails closed with a clear error (mirrors+ `openai-compat.ts` when semantic is enabled without base_url/model).+- Enabled + endpoint returns scores: top-K are re-ordered by the cross-encoder scores; a doc the+ heuristic ranker placed 3rd but the cross-encoder scored highest lands 1st.+- Enabled + endpoint errors: stage degrades to the heuristic-ordered input (no throw into the+ hot path) and emits one fail-open telemetry record.++### Depends on+- None. Lands first (pure search-side, no vault writes, lowest blast radius, seeds the+ provider-registry pattern). Must not touch `context-pack.ts`, `context-receipts.ts`, or any+ brain/ persistence.++---++## C (t_e163feb2) - Epistemic provenance markers++### Files+- `src/core/brain/provenance/epistemic-status.ts` - pure derivation:+ `deriveEpistemicStatus(meta)` -> `EpistemicStatus`+ (`observed | derived | hypothesis | plan | unknown`) + `evidence_refs`, from existing graph+ metadata (source-backed vs inferred vs preference/hypothesis vs intended vs gap). Language+ agnostic, no manual tagging.+- optional fields `epistemic_status` and `evidence_refs` added to `ContextPackItem` and the+ recall result type (absent by default).+- derivation wired into `context-pack.ts` and recall result rendering behind a feature flag.+- `src/mcp/brain/*` (or the pack/recall surface) so the field surfaces in+ `brain_context_pack` / `brain_context_receipts` output when enabled.++### Acceptance (a passing test)+- Flag off: `brain_context_pack` and recall output is byte-identical to the pre-feature baseline+ (the new fields are absent, not present-with-empty).+- Flag on: a source-backed note item is tagged `observed` with the source wikilink as+ `evidence_refs`; an inferred/derived-fact item is `derived` with its basis linked; a+ preference-tier item is `hypothesis`; an intended/not-yet-true item is `plan`; an+ acknowledged gap is `unknown`.+- Derivation is deterministic across two runs on the same input (Syncthing-peer safe).++### Depends on+- A landed (so the branch is stable at the reader tail). Establishes the item-shape seam D+ extends. Must not collide with `context-receipts.ts` summary shape beyond adding optional+ fields.++---++## D (t_b8f66fec) - Recall adequacy verdict with explicit low-adequacy actions++### Files+- `src/core/brain/recall/adequacy.ts` - pure verdict:+ `classifyAdequacy(gateScores, coverageReport, epistemicMix, thresholds)` ->+ `{ verdict: sufficient|weak|insufficient, action: proceed|re_recall|abstain|escalate }`.+ Reuses `gate-telemetry` relevance scores, `coverage.ts` IDF-weighted coverage, and C's+ epistemic mix; thresholds configurable.+- verdict + action exposed in `brain_recall_gate` output and `context-receipts` summary.+- `weak` action: trigger an alternate-strategy re-recall (broaden scope / different path) via+ existing recall machinery, behind the feature flag.+- `insufficient` action: abstain signal ("insufficient grounding") and/or escalate flag, behind+ the feature flag.++### Acceptance (a passing test)+- Flag off: `brain_recall_gate` output and `context-receipts` summary are byte-identical to the+ pre-feature baseline.+- Flag on, high-coverage + mostly-`observed` recall: verdict `sufficient`, action `proceed`.+- Flag on, partial coverage / mixed `hypothesis`-heavy recall: verdict `weak`, action `re_recall`+ (one alternate-strategy re-recall fires, deterministic).+- Flag on, near-zero coverage / mostly `unknown`: verdict `insufficient`, action `abstain`+ (explicit "insufficient grounding" signal returned, not a low-confidence answer) and/or+ `escalate`.+- Verdict is deterministic over the same scores.++### Depends on+- C landed (D extends C's merged item shape with the verdict; D reads C's epistemic mix). Reuses+ existing `gate-telemetry.ts` and `coverage.ts` scores - must NOT introduce a new score or a+ separate attention organ (card guardrail).++---++## E (t_97091fff) - Cross-query demand log surfacing weak-recall queries++### Files+- `src/core/brain/recall/demand-log.ts` - append + rotate/cap + redact: each recall appends+ (normalized terms or redacted query form + timestamp + result count + the coverage score+ already computed per query in `coverage.ts`) to a rolling, size-capped+ `Brain/recall/demand-log.jsonl`. Query normalized/redacted before append; log capped/rotated.+- aggregation reader (a `knowledge_gaps`-style read tool / CLI) bucketing by normalized query+ and ranking by frequency x low-satisfaction, reusing the existing coverage score as the+ "answered poorly" axis.+- append wired at the recall seam D opened, behind a feature flag.++### Acceptance (a passing test)+- Flag off: no file is written under `Brain/`, recall path byte-identical to baseline.+- Flag on: three recalls of the same normalized query that each return low coverage produce one+ bucket ranked by frequency x low-satisfaction; a different query asked once with good coverage+ ranks far lower.+- Privacy: the on-disk log contains the normalized/redacted form, never the raw sensitive query+ (asserted by inspecting the written file in the test).+- Size cap: after N appends the oldest entries are rotated out; the file never grows unbounded.+- Aggregation is deterministic across two runs on the same log (Syncthing-peer safe).++### Depends on+- D landed (E appends at the recall seam D opened and reuses the coverage score D surfaces).+ Independent persistence directory `Brain/recall/` - must not share a writer with B's+ `Brain/lessons/`.++---++## B (t_62363378) - Outcome-tagged, recency-scored lessons loop with corroboration tiering++### Files+- `src/core/brain/lessons/outcome-corpus.ts` - unify `dead-ends.ts` (negative) and+ `preference.ts`/`apply-evidence.ts` (positive) into one outcome-tagged corpus+ (`useful | dead_end | corrected`).+- `src/core/brain/lessons/score.ts` - signed recency-decayed score reusing+ `usage-signal.ts` `decayWeight()` (configurable half-life) as the promotion signal.+- `src/core/brain/lessons/digest.ts` - corroboration-gated aggregation: a node is `PREFERRED`+ only once corroborated by >=N distinct results (configurable N), else `TENTATIVE`;+ mixed-signal nodes render `CONTESTED` (recency-wins). Folds the corpus into a single+ `Brain/lessons/LESSONS.md`.+- `active.ts` / `dream-refresh.ts` extension for auto-load (SessionStart) + auto-refresh (dream+ cron) of the digest.+- dream-pass extension (`dream.ts`/`dream-stage.ts`) adding signed + corroboration-gated tiers,+ behind a feature flag.++### Acceptance (a passing test)+- Flag off: dream pass and `active.md` byte-identical to baseline; no `Brain/lessons/` written.+- Flag on: a node corroborated by <N distinct results is `TENTATIVE`; once >=N distinct results+ corroborate it, it promotes to `PREFERRED`; a fresh `dead_end` outcome outweighs a stale+ `useful` on the same node (recency + sign), rendering it `CONTESTED` when mixed.+- A one-off outcome does NOT promote to `PREFERRED` (corroboration threshold prevents it).+- The `LESSONS.md` digest is deterministic across two runs on the same corpus (stable sort keys,+ stable slugs; Syncthing-peer safe).+- Auto-load: the digest surface is reachable via the existing `active.ts`/SessionStart path; no+ git-hook dependency.++### Depends on+- A, C, D, E landed (B is the capstone assembly over a fully-merged branch; it must not force a+ rebase of the recall/telemetry surfaces). Extends the dream pass rather than running a parallel+ aggregation; does not rewrite or delete the source dead-ends/preference notes.diff --git a/docs/brainstorm/retrieval-precision-quality-loop/variants.md b/docs/brainstorm/retrieval-precision-quality-loop/variants.mdnew file mode 100644index 00000000..c1474bab--- /dev/null+++ b/docs/brainstorm/retrieval-precision-quality-loop/variants.md@@ -0,0 +1,108 @@+# Architecture variants - retrieval-precision-quality-loop++Branch: `feat/retrieval-precision-quality-loop`. Consultant output is preserved verbatim in+`cli-output/claude.md`; this file restates the three variants verbatim and records the final+decision and rationale.++## Variant 1+Approach: Introduce one shared per-item annotation envelope early on the branch - a+`RetrievalAnnotations` extension threaded through `context-pack.ts`/`context-receipts.ts` and+the reader tail - and have every card attach its field to that single struct (A: rerank score,+C: epistemic status + `evidence_refs`, D: adequacy verdict, E: satisfaction signal). Cards land+in sequence, each extending the same envelope rather than opening a new output seam. B folds its+outcome tag onto the same annotation vocabulary.+Trade-offs:+- Cleanest composition: C->D and A->enrich share one growth point, so the C/D output-shape+ collision and the A/rerank-tail collision collapse into "extend the struct."+- Broad blast radius on the shared type; a defect in the envelope touches all five read paths+ at once.+- Per-card bit-identity is harder to verify: each card mutates a struct other cards also read,+ so the off-branch proof is no longer a single guarded call site but "this field defaults+ absent and nothing consumes it."+- Front-loads an abstraction before the second consumer exists, risking speculative shape.+Complexity: large+Risk: medium++## Variant 2+Approach: Keep each card a self-contained stage with its own module, its own default-off config+key, and one guarded call site, minimizing shared types. Resolve the real collisions purely by+ordering so the later card extends an already-merged seam: A -> C -> D -> E -> B. A appends+after the existing heuristic reranks in the reader tail; C adds the epistemic field to context+items; D extends C's now-merged item shape with the verdict and reads gate-telemetry/coverage; E+instruments the recall path D already touched; B lands last as the capstone assembly over a+fully-merged branch.+Trade-offs:+- Maximal per-card bit-identity: every off-branch is a single guarded call site whose unchanged+ output is trivially verifiable.+- Smallest blast radius per card; a regression is isolated to one stage.+- Directly mirrors invariant #6 and the demonstrated one-PR-many-cards cadence - no up-front+ shared abstraction to design or defend.+- Mild duplication: C's status and D's verdict are emitted as adjacent-but-separate fields; B+ and E each own their persistence (`Brain/lessons/*` vs `Brain/demand-log/*`) with no shared+ writer.+Complexity: medium+Risk: low++## Variant 3+Approach: Land the two persistence substrates and a shared recall-event record first - B's+outcome-tagged lessons corpus and E's demand log both emitted as continuity/`gate_telemetry`-+style records - then rebuild the read-side cards (A, C, D) as consumers of that common event+stream. D and E share one recall-event emission; B and E share the record substrate; the dream+pass folds everything.+Trade-offs:+- Resolves the B/E persistence collision and the D/E recall-instrumentation collision via one+ shared emission point.+- Inverts the natural read->write flow, forcing the persistence/event layer to be designed+ before its consumers exist.+- Centralized emission on the hot path stresses the fail-open telemetry contract and Syncthing+ determinism, and makes the off-branch bit-identity proof depend on the whole bus being inert,+ not a local call site.+- Largest blast radius and the weakest fit with driving cards one at a time - the first card+ carries most of the risk for cards that land later.+Complexity: large+Risk: high++## Recommended: Variant 2+Variant 2 is the only option whose off-branch state is provable the way invariant #2 asks for+it - a single guarded call site per card, each off-branch byte-identical to today - because it+refuses a shared mutable struct or a shared emission bus that would make bit-identity a+whole-subsystem property. The ordering A -> C -> D -> E -> B is exactly invariant #6 in+practice: A is pure search-side with no vault writes and lowest blast radius, so it seeds the+cross-encoder-provider pattern first (mirroring `embeddings/registry.ts`) without touching any+other card; C then establishes the epistemic field on context items so D extends an+already-merged shape rather than colliding on `context-pack.ts`/`context-receipts.ts`; D reuses+C's status plus the existing `gate-telemetry`/`coverage` scores as a thin verdict+action layer+(honoring its "do not build a separate attention organ" guardrail); E appends its demand log at+the recall seam D has already opened, reusing `coverage.ts`'s IDF score as the satisfaction+axis; and B lands last as the heaviest assembly, unifying `dead-ends.ts`/`preference.ts` under+the `usage-signal.ts` decay scorer and the dream pass over a fully-merged branch. This maximizes+reuse of existing scoring/telemetry/coverage primitives, keeps every decision in-kernel+deterministic (the cross-encoder stays an opt-in external call, graceful no-op when unset), and+matches the repository's proven multi-card-suite cadence. Variant 1's shared envelope and+Variant 3's event bus both trade that per-card verifiability and small blast radius for a+composition elegance the suite does not need - and both raise, rather than lower, the risk to+the load-bearing bit-identity and fail-open invariants.++## Final decision: Variant 2 (accepted)++Agreed with the consultant's recommendation. Variant 2 is the only variant whose off-branch+state is provable per-card as a single guarded call site - the precondition the repo demands for+any ranking/score/output change (invariant #2) and the easiest property to verify. It preserves+the load-bearing conventions: an LLM-free kernel (the cross-encoder is an opt-in external call,+graceful no-op when unconfigured), fail-open telemetry, language-agnostic logic, Syncthing-peer+determinism, and the demonstrated one-PR-themed-suite release cadence. The drive order+A -> C -> D -> E -> B resolves every real cross-card collision by the cheapest sufficient+mechanism (ordering) rather than a speculative up-front abstraction: A seeds the provider-registry+pattern at the reader tail with no vault writes; C establishes the item-shape seam so D extends+a merged shape rather than colliding on `context-pack.ts`/`context-receipts.ts`; D reuses+existing gate-telemetry + coverage scores as a thin verdict+action layer (honoring the card's+"no separate attention organ" guardrail); E appends at the recall seam D opened, reusing the+coverage score as the satisfaction axis; B lands last as the capstone assembly over a+fully-merged branch, extending the dream pass rather than running a parallel loop.++Variant 1's shared annotation envelope turns bit-identity into a whole-subsystem property and+front-loads an abstraction before its consumers exist. Variant 3's shared emission bus+inverts the read->write flow and stresses the fail-open + Syncthing-determinism invariants on+the hot path. Neither is warranted for this suite.++Drive order: **A -> C -> D -> E -> B**.diff --git a/docs/how-it-works.md b/docs/how-it-works.mdindex d6d5c227..8661c415 100644--- a/docs/how-it-works.md+++ b/docs/how-it-works.md@@ -426,6 +426,21 @@ flowchart LRThe writer is idempotent — if the rendered body matches the fileon disk, no I/O happens. The same window / limit drive a mirrored`Most-applied (Nd)` section in `brain_digest` output.+- **`Brain/lessons.md`** is a second derived digest that folds the+ positive-knowledge (preferences + their `apply-evidence applied`+ rows) and negative-knowledge (`violated` / `outdated` rows and+ `Brain/dead-ends/`) into ONE outcome-tagged corpus scored by a+ **signed, recency-decayed** weight (shared 30-day half-life from+ working-memory continuity; tunable via `lessons.half_life_days`).+ Each lesson lands in a tier: **preferred** (net-positive and+ corroborated by ≥N distinct results — `lessons.corroboration_min`,+ default 2), **tentative** (net-positive but under the corroboration+ floor), **contested** (both positive and negative recent evidence —+ the shown stance follows the sign of the decayed score, so recency+ wins), or **avoid** (dead-ends and rules whose recent evidence is+ net-negative). Rendered top-N (`lessons.limit`, default 20), ranked+ by decayed salience. Regenerated at the tail of every `dream` pass+ alongside `active.md`, with the same idempotent write.- **SessionStart hook** (`startup | resume | clear | compact`) injectsthe body as `additionalContext` so the agent sees current rules atthe start of every session and again after `/compact` - the@@ -434,7 +449,9 @@ flowchart LRbudgeted (`active.inject_budget_chars`, default 8,000 chars):sections drop deterministically (recently retired first, thenquarantine, then most-applied) and a one-line notice points the- agent at `brain_context` for the full set. Fails closed - any error+ agent at `brain_context` for the full set. When `Brain/lessons.md`+ exists, its (separately budgeted) body is appended so the unified+ lessons corpus loads on the same surface. Fails closed - any errorpath exits 0 with no output so the runtime proceeds unaffected.- **MCP Resources** expose the same content for hosts that preferpull access (`osb://preferences/active` and friends in the tablediff --git a/docs/mcp.md b/docs/mcp.mdindex 1eaca976..16c4b8a3 100644--- a/docs/mcp.md+++ b/docs/mcp.md@@ -46,10 +46,11 @@ flags for a narrower per-process full server.| `brain_analytics` | Read-only Brain analytics for any lens: `view: timeline \| attention_flows \| belief_evolution \| concept_synthesis`. | `view` || `brain_search` | Read-only vault search with optional structured query lanes, explicit focus hints, time ranges, evidence-pack diagnostics, and a selectable recall `profile` (`fast \| balanced \| thorough`). | `query` || `brain_recall_feedback` | Record explicit up/down recall feedback for one search result; feeds the deterministic learned-weight fold. | `query`, `result_path`, `verdict` |-| `brain_recall_gate` | Read-only classifier for whether an automatic recall attempt should run; returns `retrieve` plus a stable reason. | `prompt` |-| `brain_context_pack` | Budgeted context slice; pass `lanes: true` to return directives, constraints, and consider lanes. Filtered items include `safety.reasons`. | `max_tokens` |+| `brain_recall_gate` | Read-only classifier for whether an automatic recall attempt should run; returns `retrieve` plus a stable reason. When the caller passes `scores`, also attaches an adequacy verdict (`sufficient` \| `weak` \| `insufficient`), a recommended action (`proceed` \| `re_recall` \| `abstain`), and an optional `escalate` flag over the gate-telemetry relevance scores plus the epistemic mix; thresholds via `recall_adequacy_sufficient` / `recall_adequacy_weak` / `recall_adequacy_min_results`. | `prompt` |+| `brain_context_pack` | Budgeted context slice; pass `lanes: true` to return directives, constraints, and consider lanes. Filtered items include `safety.reasons`. Each item carries a structural `epistemic` status (`observed` \| `derived` \| `hypothesis` \| `plan` \| `unknown`) plus `evidence_refs` derived from existing graph metadata; fields are absent when the status is `unknown`. | `max_tokens` || `brain_context_receipts` | List or show opt-in prompt context receipt continuity records with budgets, hashes, source refs, safety/redaction metadata, and item IDs. | `operation` || `brain_recall_telemetry` | List or summarise opt-in recall telemetry records for search, context-pack, and pre-compress calls. | `operation` |+| `brain_knowledge_gaps` | Aggregate the persisted cross-query demand log into recurring queries the vault answers poorly, ranked by frequency × (1 − IDF-weighted coverage). Read-only; the log is written only by opt-in recall telemetry. | — || `brain_generation_reports` | Inbound, opt-in LLM generation tracing: `record` posts a generation's usage for a handoff (gated, default off; stores prompt hash + token counts only); `list`/`summary` read records and join them to memory paths. Kernel never calls an LLM. | `action` || `brain_obligation` | Recurring obligations under `Brain/obligations/` with a deterministic cadence-driven next-due date: `add`, `done` (advances next_due by one cadence interval), `list` (optionally overdue-only), `show`, `remove`. Cadences: daily/weekly/biweekly/monthly/quarterly/yearly/every-<N>-days. | `operation` || `brain_agenda` | Stateless agenda synthesis over caller-provided calendar events (the host fetches them; the Brain never calls a calendar API): overlap conflicts, free focus blocks (optionally clipped to a workday window), and events organised outside the operator's own email domain(s). No vault writes. | `events` |@@ -148,6 +149,12 @@ come back from `resources/list`:generated digest of confirmed + quarantined preferences plus thelast three retired entries. Auto-regenerated on first read if thefile does not exist yet.+- `osb://lessons` — body of `Brain/lessons.md`, the auto-generated,+ signed and recency-scored lessons corpus that unifies preferences and+ dead-ends into corroboration-tiered lessons (`preferred` / `tentative`+ / `contested` / `avoid`). Auto-regenerated on first read if the file+ does not exist yet. The SessionStart / PostCompact hook injects it+ alongside `active.md`.- `osb://digest/latest` — same body as `brain_brief` `view="digest"`in its default (24h) Markdown window.- `osb://status` — Brain operational snapshot: counts (inbox /diff --git a/hooks/active-inject.ts b/hooks/active-inject.tsindex fc40283a..96fa363c 100644--- a/hooks/active-inject.ts+++ b/hooks/active-inject.ts@@ -31,7 +31,7 @@ import { existsSync, readFileSync } from "node:fs";import { resolveVault } from "../src/core/config.ts";import { parseFrontmatterText } from "../src/core/vault.ts";-import { brainActivePath } from "../src/core/brain/paths.ts";+import { brainActivePath, brainLessonsPath } from "../src/core/brain/paths.ts";import { budgetActiveBody } from "../src/core/brain/active-budget.ts";import { INJECT_BUDGET_CHARS_DEFAULT, loadBrainConfig } from "../src/core/brain/policy.ts";import { healCliSymlinks } from "../src/cli/install-cli.ts";@@ -121,15 +121,45 @@ async function main(): Promise<void> {// intentional fallback - a corrupted _brain.yaml is doctor's job}+ // Auto-load the lessons digest alongside active.md so the agent gets+ // the unified, signed, recency-scored corpus (preferences + dead-ends)+ // on the same SessionStart surface. Fail-soft and budgeted separately:+ // a missing / unreadable / oversized lessons file must never disturb+ // the active-preferences injection above.+ const lessonsBody = readLessonsBody(brainLessonsPath(vault), budget);++ const additionalContext =+ lessonsBody === null+ ? budgetActiveBody(trimmed, budget)+ : `${budgetActiveBody(trimmed, budget)}\n\n${lessonsBody}`;+const out = {hookSpecificOutput: {hookEventName,- additionalContext: budgetActiveBody(trimmed, budget),+ additionalContext,},};process.stdout.write(JSON.stringify(out) + "\n");}+/**+ * Read and budget the `Brain/lessons.md` body for injection. Returns+ * `null` on any failure mode (missing file, unreadable, empty body) so+ * the caller falls back to injecting active.md alone.+ */+function readLessonsBody(lessonsPath: string, budget: number): string | null {+ if (!existsSync(lessonsPath)) return null;+ try {+ const raw = readFileSync(lessonsPath, "utf8");+ const [, body] = parseFrontmatterText(raw);+ const trimmed = body.trim();+ if (trimmed.length === 0) return null;+ return budgetActiveBody(trimmed, budget);+ } catch {+ return null;+ }+}+main().catch(() => {// Never crash the runtime; the session start should proceed// regardless of any hook misbehaviour.diff --git a/openclaw.plugin.json b/openclaw.plugin.jsonindex 67d20cbe..eb80dddc 100644--- a/openclaw.plugin.json+++ b/openclaw.plugin.json@@ -2,7 +2,7 @@"id": "open-second-brain","name": "Open Second Brain","description": "Second brain for AI agents using Obsidian-compatible Markdown vaults.",- "version": "1.21.0",+ "version": "1.22.0","activation": { "onStartup": true },"skills": ["./skills"],"contracts": {diff --git a/package.json b/package.jsonindex 92c8623a..a7f13e59 100644--- a/package.json+++ b/package.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.21.0",+ "version": "1.22.0","private": false,"description": "Second brain for AI agents using Obsidian-compatible Markdown vaults. Works with Hermes, Claude Code, Codex, and OpenClaw.","keywords": [diff --git a/plugin.yaml b/plugin.yamlindex 821e81fb..1f5c1458 100644--- a/plugin.yaml+++ b/plugin.yaml@@ -1,5 +1,5 @@name: open-second-brain-version: "1.21.0"+version: "1.22.0"description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."author: "Open Second Brain contributors"memory_provider: truediff --git a/plugins/codex/.codex-plugin/plugin.json b/plugins/codex/.codex-plugin/plugin.jsonindex 75b37097..1237650e 100644--- a/plugins/codex/.codex-plugin/plugin.json+++ b/plugins/codex/.codex-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.21.0",+ "version": "1.22.0","description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.","author": {"name": "Open Second Brain contributors",diff --git a/plugins/hermes/plugin.yaml b/plugins/hermes/plugin.yamlindex 821e81fb..1f5c1458 100644--- a/plugins/hermes/plugin.yaml+++ b/plugins/hermes/plugin.yaml@@ -1,5 +1,5 @@name: open-second-brain-version: "1.21.0"+version: "1.22.0"description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."author: "Open Second Brain contributors"memory_provider: truediff --git a/pyproject.toml b/pyproject.tomlindex d193117f..2c5a61ab 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -10,7 +10,7 @@ build-backend = "setuptools.build_meta"# CLI entry points (those moved to `package.json` `bin`).[project]name = "open-second-brain"-version = "1.21.0"+version = "1.22.0"description = "Hermes Python shim for Open Second Brain. Most of the project (CLI, MCP server, OpenClaw plugin) is TypeScript on Bun; see package.json."readme = "README.md"requires-python = ">=3.11"diff --git a/src/cli/brain.ts b/src/cli/brain.tsindex 77f6f62d..d1b3a93c 100644--- a/src/cli/brain.ts+++ b/src/cli/brain.ts@@ -103,6 +103,7 @@ import {cmdBrainPreCompactExtract,cmdBrainPostCompactAudit,cmdBrainRecallTelemetry,+ cmdBrainKnowledgeGaps,cmdBrainGenerationReports,cmdBrainSkillProposals,cmdBrainProceduralMemory,@@ -335,6 +336,8 @@ export async function handleBrainSubcommand(argv: ReadonlyArray<string>): Promisreturn await cmdBrainPostCompactAudit(rest);case "recall-telemetry":return await cmdBrainRecallTelemetry(rest);+ case "knowledge-gaps":+ return await cmdBrainKnowledgeGaps(rest);case "generation-reports":return await cmdBrainGenerationReports(rest);case "skill-proposals":diff --git a/src/cli/brain/help-text.ts b/src/cli/brain/help-text.tsindex d38c09d6..25c90a7d 100644--- a/src/cli/brain/help-text.ts+++ b/src/cli/brain/help-text.ts@@ -86,6 +86,7 @@ Brain verbs (observing memory):pre-compact-extract Extract typed continuity records from bounded textpost-compact-audit Audit pinned-anchor survival after a compaction; re-assert drifted anchorsrecall-telemetry List/summarize opt-in recall telemetry records+ knowledge-gaps Rank recurring queries the vault answers poorly (unmet demand)generation-reports Record/list/summarize opt-in LLM generation tracesskill-proposals Learn/list/review deterministic skill proposalsprocedural-memory Reconcile/list procedural memory index and usage@@ -589,6 +590,9 @@ export const VERB_HELP: Record<string, string> = {" o2b brain recall-telemetry summary [same filters] [--vault <path>] [--json]\n" +" o2b brain recall-telemetry cost [--since <iso>] [--until <iso>] [--write-cost <n>] [--read-cost <n>] [--write-heavy-ratio <n>] [--vault <path>] [--json]\n" +"Read opt-in recall telemetry continuity records and aggregate coverage gaps. cost folds write volume (feedback/apply-evidence/note/host writes) against reads into a write-vs-read ratio, a write-heavy flag, and a rough weighted cost signal per period.\n",+ "knowledge-gaps":+ "usage: o2b brain knowledge-gaps [--min-occurrences <n>] [--max-satisfaction <0..1>] [--since <iso>] [--until <iso>] [--limit <n>] [--vault <path>] [--json]\n" ++ "Aggregate the persisted cross-query demand log (Brain/log/query-demand.jsonl) into recurring queries the vault answers poorly. Buckets by normalized query terms and ranks by frequency x (1 - satisfaction), where satisfaction is the reused IDF-weighted coverage (or the non-empty-result fraction when coverage was not recorded). Turns repeated recall failures into a prioritized backlog of what to write next. Read-only; the log is written only by opt-in recall telemetry.\n","generation-reports":"usage: o2b brain generation-reports record <write_session|context_pack|dream_stage> --ref <id> --agent <name> --prompt <text> [--enable] [--provider <p>] [--model <m>] [--finish-reason <r>] [--latency-ms <n>] [--input-tokens <n>] [--output-tokens <n>] [--cached-tokens <n>] [--total-tokens <n>] [--scope <s>] [--source <id[=path]>...] [--created-at <iso>] [--vault <path>] [--json]\n" +" o2b brain generation-reports list [--handoff <kind>] [--agent <name>] [--since <iso>] [--until <iso>] [--limit <n>] [--vault <path>] [--json]\n" +diff --git a/src/cli/brain/verbs/context-pack.ts b/src/cli/brain/verbs/context-pack.tsindex 27c62326..628b483b 100644--- a/src/cli/brain/verbs/context-pack.ts+++ b/src/cli/brain/verbs/context-pack.ts@@ -88,6 +88,8 @@ export async function cmdBrainContextPack(argv: string[]): Promise<number> {path: i.path,tier: i.tier,tokens: i.tokens,+ epistemic: i.epistemic,+ ...(i.evidenceRefs.length > 0 ? { evidence_refs: i.evidenceRefs } : {}),...(i.originalRank !== undefined ? { original_rank: i.originalRank } : {}),...(i.stableRank !== undefined ? { stable_rank: i.stableRank } : {}),...(i.dedupedFrom !== undefined ? { deduped_from: i.dedupedFrom } : {}),@@ -106,7 +108,7 @@ export async function cmdBrainContextPack(argv: string[]): Promise<number> {process.stdout.write(`pages skipped: ${report.skipped.length}\n\n`);if (report.receiptId) process.stdout.write(`receipt: ${report.receiptId}\n`);for (const i of report.items) {- process.stdout.write(`[${i.tier}] ${i.id} (${i.tokens} tokens)\n`);+ process.stdout.write(`[${i.tier}] ${i.epistemic} ${i.id} (${i.tokens} tokens)\n`);}return 0;}diff --git a/src/cli/brain/verbs/context-receipts.ts b/src/cli/brain/verbs/context-receipts.tsindex fe3cb7b6..f1f670db 100644--- a/src/cli/brain/verbs/context-receipts.ts+++ b/src/cli/brain/verbs/context-receipts.ts@@ -82,6 +82,10 @@ function showReceipt(argv: string[]): number {if (summary.turn_id) process.stdout.write(`turn_id: ${summary.turn_id}\n`);process.stdout.write(`item_count: ${summary.item_count ?? "?"}\n`);process.stdout.write(`source_count: ${summary.source_count}\n`);+ if (summary.adequacy) {+ const { level, action, escalate } = summary.adequacy;+ process.stdout.write(`adequacy: ${level} -> ${action}${escalate ? " (escalate)" : ""}\n`);+ }return 0;}diff --git a/src/cli/brain/verbs/index.ts b/src/cli/brain/verbs/index.tsindex 1bc20fd4..fb90913b 100644--- a/src/cli/brain/verbs/index.ts+++ b/src/cli/brain/verbs/index.ts@@ -87,6 +87,7 @@ export { cmdBrainContextPresets } from "./context-presets.ts";export { cmdBrainPreCompactExtract } from "./pre-compact-extract.ts";export { cmdBrainPostCompactAudit } from "./post-compact-audit.ts";export { cmdBrainRecallTelemetry } from "./recall-telemetry.ts";+export { cmdBrainKnowledgeGaps } from "./knowledge-gaps.ts";export { cmdBrainGenerationReports } from "./generation-reports.ts";export { cmdBrainSkillProposals } from "./skill-proposals.ts";export { cmdBrainProceduralMemory } from "./procedural-memory.ts";diff --git a/src/cli/brain/verbs/knowledge-gaps.ts b/src/cli/brain/verbs/knowledge-gaps.tsnew file mode 100644index 00000000..dedcb821--- /dev/null+++ b/src/cli/brain/verbs/knowledge-gaps.ts@@ -0,0 +1,86 @@+import {+ aggregateQueryDemand,+ serializeQueryDemandReport,+} from "../../../core/brain/query-demand.ts";+import { CliError, brainVerbContext, parse } from "../helpers.ts";++/**+ * `o2b brain knowledge-gaps` — surface recurring queries the vault+ * answers poorly from the persisted cross-query demand log+ * (t_97091fff). Read-only aggregation; no LLM.+ */+export async function cmdBrainKnowledgeGaps(argv: string[]): Promise<number> {+ const { flags } = parse(argv, {+ vault: { type: "string" },+ json: { type: "boolean" },+ since: { type: "string" },+ until: { type: "string" },+ "min-occurrences": { type: "string" },+ "max-satisfaction": { type: "string" },+ limit: { type: "string" },+ });+ const vault = brainVerbContext(flags).vault;+ const since = trimOrUndefined(flags["since"]);+ const until = trimOrUndefined(flags["until"]);+ const minOccurrences = parsePositiveInteger(+ trimOrUndefined(flags["min-occurrences"]),+ "--min-occurrences",+ );+ const limit = parsePositiveInteger(trimOrUndefined(flags["limit"]), "--limit");+ const maxSatisfaction = parseUnitInterval(+ trimOrUndefined(flags["max-satisfaction"]),+ "--max-satisfaction",+ );++ const report = aggregateQueryDemand(vault, {+ ...(since !== undefined ? { since } : {}),+ ...(until !== undefined ? { until } : {}),+ ...(minOccurrences !== undefined ? { minOccurrences } : {}),+ ...(maxSatisfaction !== undefined ? { maxSatisfaction } : {}),+ ...(limit !== undefined ? { limit } : {}),+ });++ if (flags["json"]) {+ process.stdout.write(JSON.stringify(serializeQueryDemandReport(report), null, 2) + "\n");+ return 0;+ }++ process.stdout.write(+ `${report.totalRecords} demand record(s), ${report.distinctQueries} distinct quer(y/ies), ` ++ `${report.gaps.length} gap(s):\n`,+ );+ for (const gap of report.gaps) {+ const coverage = gap.meanCoverage === null ? "n/a" : gap.meanCoverage.toFixed(2);+ process.stdout.write(+ ` [${gap.demandScore.toFixed(2)}] ${gap.verdict} x${gap.occurrences} ` ++ `coverage=${coverage} empty=${gap.emptyCount}/${gap.occurrences} ` ++ `${gap.terms.join(" ")}\n`,+ );+ }+ return 0;+}++function trimOrUndefined(value: string | boolean | string[] | undefined): string | undefined {+ if (typeof value !== "string") return undefined;+ const trimmed = value.trim();+ return trimmed.length > 0 ? trimmed : undefined;+}++function parsePositiveInteger(value: string | undefined, flag: string): number | undefined {+ if (value === undefined) return undefined;+ if (!/^[0-9]+$/.test(value)) {+ throw new CliError(`brain knowledge-gaps: ${flag} must be a positive integer`);+ }+ const parsed = Number.parseInt(value, 10);+ if (parsed < 1) throw new CliError(`brain knowledge-gaps: ${flag} must be a positive integer`);+ return parsed;+}++function parseUnitInterval(value: string | undefined, flag: string): number | undefined {+ if (value === undefined) return undefined;+ const parsed = Number(value);+ if (!Number.isFinite(parsed) || parsed < 0 || parsed > 1) {+ throw new CliError(`brain knowledge-gaps: ${flag} must be a number in [0, 1]`);+ }+ return parsed;+}diff --git a/src/core/brain/context-pack.ts b/src/core/brain/context-pack.tsindex 66738ed5..e1e03c88 100644--- a/src/core/brain/context-pack.ts+++ b/src/core/brain/context-pack.ts@@ -26,10 +26,16 @@ import {} from "./safety/context-guard.ts";import { brainDirs } from "./paths.ts";import { PAGE_TIER, readTier, type PageTier } from "./page-meta/tier.ts";+import {+ deriveEpistemicStatus,+ EPISTEMIC_STATUS,+ type EpistemicStatus,+} from "./provenance/epistemic.ts";import { estimateTokens } from "./text/tokenizer.ts";import { normalizeForDedup } from "./text/normalize.ts";import { applyCharBudget, type CharBudgetDegradationMode } from "./recall-budget.ts";import { emitContextReceipt, type ContextReceiptOptions } from "./context-receipts.ts";+import type { RecallAdequacyVerdict } from "./recall-adequacy.ts";import { emitGatedTelemetry } from "./continuity/emit.ts";import {canonicalSegment,@@ -73,6 +79,15 @@ export interface ContextPackItem extends ContextTransformAnnotations {readonly contextLane: ContextLaneName | null;/** True when `body` was truncated by `maxCharsPerMemory` (v0.20.0). */readonly trimmed: boolean;+ /**+ * Epistemic grounding of this item (ACM), derived from the page's+ * provenance / status / lifecycle metadata: `observed | derived |+ * hypothesis | plan | unknown`. Lets downstream reasoning distinguish a+ * grounded fact from a conjecture.+ */+ readonly epistemic: EpistemicStatus;+ /** `evidenced_by` wikilinks grounding this item; empty when it cites none. */+ readonly evidenceRefs: ReadonlyArray<string>;/** Present when the surfaced body was filtered or explicitly trusted. */readonly safety?: ContextSafetyReport;}@@ -120,6 +135,13 @@ export interface ContextPackOptions {readonly includeLanes?: boolean;/** Opt-in audit receipt for the final emitted context. */readonly receipt?: ContextReceiptOptions;+ /**+ * Recall adequacy verdict (t_b8f66fec) for the recall that produced+ * this pack. When present alongside `receipt`, it is persisted in the+ * receipt so the audit trail records the sufficient/weak/insufficient+ * verdict and the action the caller was told to take.+ */+ readonly recallAdequacy?: RecallAdequacyVerdict;/** Opt-in telemetry for recall coverage and gap diagnostics. */readonly telemetry?: RecallTelemetryOptions;/**@@ -153,6 +175,8 @@ interface Candidate {readonly contextLane: ContextLaneName | null;readonly body: string;readonly tokens: number;+ readonly epistemic: EpistemicStatus;+ readonly evidenceRefs: ReadonlyArray<string>;readonly safety?: ContextSafetyReport;}@@ -207,6 +231,7 @@ function collectCandidates(vault: string, delimitUntrusted: boolean): Candidate[const topic = typeof meta["topic"] === "string" ? meta["topic"] : "";const principle = typeof meta["principle"] === "string" ? meta["principle"] : "";const contextLane = normalizeContextLane(meta["context_lane"]);+ const epistemic = deriveEpistemicStatus(meta);const guarded = guardBrainContextSnippet(body, {source: { id, path: full, metadata: meta },...(meta["context_safety"] === "trusted-instruction"@@ -231,6 +256,8 @@ function collectCandidates(vault: string, delimitUntrusted: boolean): Candidate[contextLane,body: guarded.safeText,tokens: estimateTokens(guarded.safeText),+ epistemic: epistemic.status,+ evidenceRefs: epistemic.evidenceRefs,...(contextSafetyReport(guarded) ? { safety: contextSafetyReport(guarded) } : {}),});}@@ -323,6 +350,8 @@ export function packContext(vault: string, opts: ContextPackOptions): ContextPacprinciple: c.principle,contextLane: c.contextLane,trimmed,+ epistemic: c.epistemic,+ evidenceRefs: c.evidenceRefs,...(c.safety ? { safety: c.safety } : {}),});used += tokens;@@ -354,6 +383,10 @@ export function packContext(vault: string, opts: ContextPackOptions): ContextPacprinciple: "Declarative attention flow output",contextLane: "directives",trimmed: false,+ // Synthesized from the vault's attention-flow graph, not a+ // source-backed page, so it is `derived` with no evidence refs.+ epistemic: EPISTEMIC_STATUS.derived,+ evidenceRefs: [],...(safetyReport ? { safety: safetyReport } : {}),});used += tokens;@@ -445,9 +478,12 @@ function finalizeContextPackReport(tier: item.tier,trimmed: item.trimmed,safetyFiltered: item.safety?.filtered,+ epistemic: item.epistemic,+ evidenceRefs: item.evidenceRefs,})),finalText: report.items.map((item) => item.body).join("\n\n"),budget: contextPackBudgetMetadata(opts, report),+ ...(opts.recallAdequacy ? { adequacy: opts.recallAdequacy } : {}),extra: {skipped_count: report.skipped.length,lanes: opts.includeLanes === true,diff --git a/src/core/brain/context-receipts.ts b/src/core/brain/context-receipts.tsindex 8a79cbc0..8c103bf8 100644--- a/src/core/brain/context-receipts.ts+++ b/src/core/brain/context-receipts.ts@@ -2,6 +2,7 @@ import { createHash } from "node:crypto";import { appendContinuityRecord, listContinuityRecords } from "./continuity/store.ts";import type { ContinuityRecord } from "./continuity/types.ts";+import type { RecallAdequacyVerdict } from "./recall-adequacy.ts";export type ContextReceiptTrigger = "context_pack" | "pre_compress";@@ -21,6 +22,10 @@ export interface ContextReceiptItemInput {readonly tier?: string;readonly trimmed?: boolean;readonly safetyFiltered?: boolean;+ /** Epistemic grounding of the item (ACM): observed | derived | ... */+ readonly epistemic?: string;+ /** `evidenced_by` wikilinks grounding the item. */+ readonly evidenceRefs?: ReadonlyArray<string>;}export interface EmitContextReceiptInput {@@ -29,6 +34,26 @@ export interface EmitContextReceiptInput {readonly finalText: string;readonly budget?: Readonly<Record<string, unknown>>;readonly extra?: Readonly<Record<string, unknown>>;+ /**+ * Recall adequacy verdict (t_b8f66fec) for the recall that produced+ * this context. When present it is persisted so the audit trail records+ * whether the surfaced grounding was judged sufficient / weak /+ * insufficient and what action the caller was told to take.+ */+ readonly adequacy?: RecallAdequacyVerdict;+}++/** Flatten a verdict into the snake_cased receipt payload shape. */+export function serializeAdequacy(verdict: RecallAdequacyVerdict): Record<string, unknown> {+ return {+ level: verdict.level,+ action: verdict.action,+ escalate: verdict.escalate,+ result_count: verdict.resultCount,+ top_score: verdict.topScore,+ mean_score: verdict.meanScore,+ reason: verdict.reason,+ };}export interface ContextReceiptFilter {@@ -49,6 +74,12 @@ export interface ContextReceiptSummary {readonly source_count: number;readonly private: boolean;readonly redacted: boolean;+ /** Recall adequacy verdict summary, when the receipt recorded one. */+ readonly adequacy?: {+ readonly level: string;+ readonly action: string;+ readonly escalate: boolean;+ };}export function emitContextReceipt(@@ -64,6 +95,10 @@ export function emitContextReceipt(...(item.tier ? { tier: item.tier } : {}),...(item.trimmed !== undefined ? { trimmed: item.trimmed } : {}),...(item.safetyFiltered !== undefined ? { safety_filtered: item.safetyFiltered } : {}),+ ...(item.epistemic !== undefined ? { epistemic: item.epistemic } : {}),+ ...(item.evidenceRefs && item.evidenceRefs.length > 0+ ? { evidence_refs: [...item.evidenceRefs] }+ : {}),...(item.text ? { text_hash: sha256(item.text) } : {}),}));const payload: Record<string, unknown> = {@@ -76,6 +111,7 @@ export function emitContextReceipt(final_text_chars: [...input.finalText].length,items: itemPayloads,...(input.budget ? { budget: input.budget } : {}),+ ...(input.adequacy ? { adequacy: serializeAdequacy(input.adequacy) } : {}),};mergeReceiptExtra(payload, input.extra);return appendContinuityRecord(vault, {@@ -136,9 +172,21 @@ export function summarizeContextReceipt(record: ContinuityRecord): ContextReceipsource_count: record.sourceRefs.length,private: record.private,redacted: record.redacted,+ ...adequacySummary(payload["adequacy"]),};}+function adequacySummary(+ raw: unknown,+): { adequacy: { level: string; action: string; escalate: boolean } } | Record<string, never> {+ if (typeof raw !== "object" || raw === null) return {};+ const record = raw as Record<string, unknown>;+ const level = record["level"];+ const action = record["action"];+ if (typeof level !== "string" || typeof action !== "string") return {};+ return { adequacy: { level, action, escalate: record["escalate"] === true } };+}+export function isContextReceiptTrigger(value: unknown): value is ContextReceiptTrigger {return value === "context_pack" || value === "pre_compress";}diff --git a/src/core/brain/dream.ts b/src/core/brain/dream.tsindex 479284e8..32392155 100644--- a/src/core/brain/dream.ts+++ b/src/core/brain/dream.ts@@ -39,6 +39,7 @@ import { basename, join } from "node:path";import { parseFrontmatter } from "../vault.ts";import { regenerateActiveQuiet } from "./active.ts";+import { regenerateLessonsQuiet } from "./lessons.ts";import { openWorkrun, WORKRUN_PHASE, type WorkrunHandle } from "./dream-workrun.ts";import { DREAM_PHASE, type DreamPhase, type DreamPhaseSummary } from "./dream-phases.ts";import { extractTemporalConstraints } from "./temporal-extract.ts";@@ -345,7 +346,10 @@ export function dream(vault: string, opts: DreamOptions = {}): DreamRunSummary {scan.corrupted.length > 0;if (!changed) {- if (!dryRun) regenerateActiveQuiet(vault, { now });+ if (!dryRun) {+ regenerateActiveQuiet(vault, { now });+ regenerateLessonsQuiet(vault, { now });+ }return Object.freeze({run_id: runId,changed: false,@@ -772,6 +776,7 @@ export function dream(vault: string, opts: DreamOptions = {}): DreamRunSummary {process.stderr.write(`warning: prune snapshots failed: ${(err as Error).message}\n`);}regenerateActiveQuiet(vault, { now });+ regenerateLessonsQuiet(vault, { now });}// v0.12.0 Brain Integrity Suite: finalise the durable workrundiff --git a/src/core/brain/index.ts b/src/core/brain/index.tsindex 15adb286..dabf244f 100644--- a/src/core/brain/index.ts+++ b/src/core/brain/index.ts@@ -73,6 +73,23 @@ export {export { regenerateActive, regenerateActiveQuiet } from "./active.ts";export type { RegenerateActiveOptions, RegenerateActiveResult } from "./active.ts";+// ----- Lessons digest (signed, recency-scored, corroboration-tiered) --------+export {+ computeLessons,+ regenerateLessons,+ regenerateLessonsQuiet,+ LESSON_TIER,+ LESSON_STANCE,+} from "./lessons.ts";+export type {+ LessonEntry,+ LessonTier,+ LessonStance,+ ComputeLessonsOptions,+ RegenerateLessonsOptions,+ RegenerateLessonsResult,+} from "./lessons.ts";+// ----- Backlink index -------------------------------------------------------export { buildBacklinkIndex, backlinkCount } from "./backlinks.ts";export type { BacklinkIndex, BacklinkRef, BacklinkSourceKind } from "./backlinks.ts";diff --git a/src/core/brain/lessons.ts b/src/core/brain/lessons.tsnew file mode 100644index 00000000..7866949b--- /dev/null+++ b/src/core/brain/lessons.ts@@ -0,0 +1,555 @@+/**+ * `Brain/lessons.md` — the unified, signed, recency-scored lessons+ * digest (Outcome-tagged lessons loop, t_62363378).+ *+ * OSB already owns the two halves of a lessons corpus as separate+ * registries:+ *+ * - NEGATIVE knowledge — `Brain/dead-ends/` ("avoid X"), and the+ * `violated` / `outdated` apply-evidence rows against a preference;+ * - POSITIVE knowledge — `Brain/preferences/` confirmed rules and the+ * `applied` apply-evidence rows that corroborate them.+ *+ * This module folds both classes into ONE outcome-tagged corpus scored+ * by a SIGNED, recency-decayed weight so that a fresh dead-end+ * outweighs a stale "useful", and promotes a lesson to a corroborated+ * tier only once ≥N distinct results back it — otherwise it stays+ * tentative, and a rule with both positive and negative recent+ * evidence renders as CONTESTED (the displayed stance follows the sign+ * of the decayed score, so recency wins).+ *+ * It is NOT a parallel loop: the corpus is derived from the existing+ * dead-end notes and apply-evidence log, the decay curve is the shared+ * `decayWeight` from continuity's usage-signal, and the digest is+ * regenerated at the tail of every `dream` pass exactly like+ * `Brain/active.md`. The SessionStart / PostCompact hook injects it+ * alongside `active.md`, so the agent gets a single auto-loaded lessons+ * surface without querying dead-ends and preferences independently.+ *+ * Pure derivation: no LLM, no network, no clock-dependent content+ * beyond the `generated_at` stamp. Identical inputs produce identical+ * bodies; the write is idempotent and atomic (mirrors `active.ts`).+ */++import { existsSync, readdirSync } from "node:fs";+import { join } from "node:path";++import { atomicWriteFileSync } from "../fs-atomic.ts";+import { parseFrontmatter } from "../vault.ts";+import { decayWeight } from "./continuity/usage-signal.ts";+import { listDeadEnds, type DeadEndEntry } from "./dead-ends.ts";+import { listLogDates, readLogDay } from "./log-jsonl.ts";+import { brainDirs, brainLessonsPath } from "./paths.ts";+import { parsePreference } from "./preference.ts";+import {+ LESSONS_CORROBORATION_MIN_DEFAULT,+ LESSONS_HALF_LIFE_DAYS_DEFAULT,+ LESSONS_LIMIT_DEFAULT,+ loadBrainConfig,+} from "./policy.ts";+import { isoSecond } from "./time.ts";+import { normaliseWikilinkTarget } from "./wikilink.ts";+import {+ BRAIN_APPLY_RESULT,+ BRAIN_LOG_EVENT_KIND,+ BRAIN_PREFERENCE_STATUS,+ type BrainPreference,+} from "./types.ts";++const FRONTMATTER_KIND = "brain-lessons";++/**+ * A net-positive lesson below the corroboration floor is `tentative`;+ * once ≥N distinct results back it, `preferred`. A rule carrying BOTH+ * positive and negative recent evidence is `contested` (recency-wins:+ * the displayed stance follows the sign of the decayed score). A+ * purely-negative node (a dead-end, or a rule only ever violated) is+ * `avoid`.+ */+export const LESSON_TIER = {+ preferred: "preferred",+ tentative: "tentative",+ contested: "contested",+ avoid: "avoid",+} as const;+export type LessonTier = (typeof LESSON_TIER)[keyof typeof LESSON_TIER];++/** The stance a `contested` lesson currently leans toward. */+export const LESSON_STANCE = {+ positive: "positive",+ negative: "negative",+} as const;+export type LessonStance = (typeof LESSON_STANCE)[keyof typeof LESSON_STANCE];++export interface LessonEntry {+ /** `pref-<slug>` or `de-<date>-<slug>`. */+ readonly id: string;+ readonly kind: "preference" | "dead-end";+ /** Preference principle or dead-end approach. */+ readonly title: string;+ readonly scope: string | null;+ readonly tier: LessonTier;+ /** Positive decayed mass minus negative decayed mass, rounded to 4dp. */+ readonly signedScore: number;+ readonly positiveMass: number;+ readonly negativeMass: number;+ /** Distinct positive-outcome artifacts corroborating the lesson. */+ readonly corroboration: number;+ /** For `contested` lessons, the recency-decided leaning; else null. */+ readonly stance: LessonStance | null;+ /** ISO timestamp of the most recent contributing outcome, or null. */+ readonly lastOutcomeAt: string | null;+}++export interface ComputeLessonsOptions {+ /** Scoring anchor. Defaults to `new Date()`. */+ readonly now?: Date;+ /** Exponential half-life of the recency decay, in days. Default 30. */+ readonly halfLifeDays?: number;+ /** Distinct-result count required for the `preferred` tier. Default 2. */+ readonly corroborationMin?: number;+ /** Max lessons returned. Default 20. */+ readonly limit?: number;+}++/** Mutable per-node accumulator during the scoring pass. */+interface Accumulator {+ positiveMass: number;+ negativeMass: number;+ readonly positiveArtifacts: Set<string>;+ lastOutcomeMs: number;+}++const EPSILON = 1e-9;++/**+ * Single-event recency weight from the shared continuity decay curve.+ * Modelled as a record with zero access count and no last-access, so+ * `decayWeight` reduces to the pure `0.5^(age/halfLife)` exponential+ * (floored at its `minWeight`) — the same half-life OSB uses for+ * working-memory continuity.+ */+function eventWeight(eventMs: number, nowMs: number, halfLifeDays: number): number {+ return decayWeight({ createdAtMs: eventMs, accessCount: 0, lastAccessAtMs: null }, nowMs, {+ halfLifeDays,+ });+}++/**+ * Fold the preference corpus and the dead-end corpus into one ranked,+ * signed, recency-scored, corroboration-tiered lesson list.+ *+ * Pure read. The caller passes the candidate preferences (already+ * filtered to `confirmed | quarantine`) and the active dead-ends;+ * apply-evidence outcomes are harvested from `Brain/log/`.+ */+export function computeLessons(+ vault: string,+ preferences: ReadonlyArray<BrainPreference>,+ deadEnds: ReadonlyArray<DeadEndEntry>,+ opts: ComputeLessonsOptions = {},+): ReadonlyArray<LessonEntry> {+ const now = opts.now ?? new Date();+ const nowMs = now.getTime();+ const halfLifeDays = opts.halfLifeDays ?? LESSONS_HALF_LIFE_DAYS_DEFAULT;+ const corroborationMin = opts.corroborationMin ?? LESSONS_CORROBORATION_MIN_DEFAULT;+ const limit = opts.limit ?? LESSONS_LIMIT_DEFAULT;++ // Index the candidate preferences by their normalised id so an+ // apply-evidence row spelled `[[pref-x|principle]]` resolves to one+ // node. Retired rules are excluded upstream and never scored here.+ const prefByKey = new Map<string, BrainPreference>();+ for (const p of preferences) {+ if (+ p.status !== BRAIN_PREFERENCE_STATUS.confirmed &&+ p.status !== BRAIN_PREFERENCE_STATUS.quarantine+ ) {+ continue;+ }+ const key = normaliseWikilinkTarget(p.id);+ if (key !== null) prefByKey.set(key, p);+ }++ const accByKey = new Map<string, Accumulator>();+ const accFor = (key: string): Accumulator => {+ let acc = accByKey.get(key);+ if (acc === undefined) {+ acc = { positiveMass: 0, negativeMass: 0, positiveArtifacts: new Set(), lastOutcomeMs: 0 };+ accByKey.set(key, acc);+ }+ return acc;+ };++ // One pass over every log day, bucketing signed outcomes by pref.+ if (prefByKey.size > 0) scoreApplyEvidence(vault, prefByKey, accFor, nowMs, halfLifeDays);++ const entries: LessonEntry[] = [];++ // Preference-derived lessons.+ for (const [key, pref] of prefByKey) {+ const acc = accByKey.get(key);+ if (acc === undefined) continue; // no outcome evidence → not a lesson yet+ entries.push(+ buildEntry({+ id: pref.id,+ kind: "preference",+ title: pref.principle,+ scope: pref.scope ?? null,+ acc,+ corroborationMin,+ }),+ );+ }++ // Dead-end lessons: one negative outcome stamped at creation. They+ // are inherently `avoid` — a recorded failed approach.+ for (const de of deadEnds) {+ const createdMs = Date.parse(de.created_at);+ if (Number.isNaN(createdMs)) continue;+ const weight = eventWeight(createdMs, nowMs, halfLifeDays);+ const acc: Accumulator = {+ positiveMass: 0,+ negativeMass: weight,+ positiveArtifacts: new Set(),+ lastOutcomeMs: createdMs,+ };+ entries.push(+ buildEntry({+ id: de.id,+ kind: "dead-end",+ title: de.approach,+ scope: null,+ acc,+ corroborationMin,+ }),+ );+ }++ // Rank by salience — the dominant decayed mass — so the strongest+ // signal (positive, negative, or contested) surfaces first. Ties+ // break to the most recent outcome, then to a stable id order.+ entries.sort((a, b) => {+ const sa = Math.max(a.positiveMass, a.negativeMass);+ const sb = Math.max(b.positiveMass, b.negativeMass);+ if (sb !== sa) return sb - sa;+ const ta = a.lastOutcomeAt ? Date.parse(a.lastOutcomeAt) : 0;+ const tb = b.lastOutcomeAt ? Date.parse(b.lastOutcomeAt) : 0;+ if (tb !== ta) return tb - ta;+ return a.id.localeCompare(b.id);+ });++ return Object.freeze(entries.slice(0, limit).map((e) => Object.freeze(e)));+}++/** Walk the log once, accumulating signed apply-evidence per pref. */+function scoreApplyEvidence(+ vault: string,+ prefByKey: ReadonlyMap<string, BrainPreference>,+ accFor: (key: string) => Accumulator,+ nowMs: number,+ halfLifeDays: number,+): void {+ const dirs = brainDirs(vault);+ if (!existsSync(dirs.log)) return;++ for (const datePrefix of listLogDates(vault)) {+ let parsed;+ try {+ parsed = readLogDay(vault, datePrefix);+ } catch (err) {+ process.stderr.write(+ `warning: lessons: failed to read ${datePrefix}: ${(err as Error).message}\n`,+ );+ continue;+ }+ for (const e of parsed.entries) {+ if (e.eventType !== BRAIN_LOG_EVENT_KIND.applyEvidence) continue;+ const prefField = e.body["preference"];+ if (typeof prefField !== "string") continue;+ const key = normaliseWikilinkTarget(prefField);+ if (key === null) continue;+ const pref = prefByKey.get(key);+ if (pref === undefined) continue;+ // Never harvest rows that predate the pref's existence — that would+ // be a different rule under a reused slug history.+ if (e.timestamp < pref.created_at) continue;+ const eventMs = Date.parse(e.timestamp);+ if (Number.isNaN(eventMs)) continue;+ const result = e.body["result"];+ if (typeof result !== "string") continue;++ const weight = eventWeight(eventMs, nowMs, halfLifeDays);+ const acc = accFor(key);+ acc.lastOutcomeMs = Math.max(acc.lastOutcomeMs, eventMs);+ if (result === BRAIN_APPLY_RESULT.applied) {+ acc.positiveMass += weight;+ const artifact = e.body["artifact"];+ if (typeof artifact === "string" && artifact.trim().length > 0) {+ acc.positiveArtifacts.add(normaliseWikilinkTarget(artifact) ?? artifact.trim());+ }+ } else if (result === BRAIN_APPLY_RESULT.violated || result === BRAIN_APPLY_RESULT.outdated) {+ acc.negativeMass += weight;+ }+ }+ }+}++interface BuildEntryInput {+ readonly id: string;+ readonly kind: "preference" | "dead-end";+ readonly title: string;+ readonly scope: string | null;+ readonly acc: Accumulator;+ readonly corroborationMin: number;+}++function buildEntry(input: BuildEntryInput): LessonEntry {+ const { acc } = input;+ const signed = acc.positiveMass - acc.negativeMass;+ const corroboration = acc.positiveArtifacts.size;++ let tier: LessonTier;+ let stance: LessonStance | null = null;+ if (acc.positiveMass > EPSILON && acc.negativeMass > EPSILON) {+ tier = LESSON_TIER.contested;+ // Recency-wins: the decayed sum already leans toward whichever+ // outcome class is fresher / stronger. Treat a dead-even score as+ // negative — an unresolved conflict is not an endorsement.+ stance = signed > EPSILON ? LESSON_STANCE.positive : LESSON_STANCE.negative;+ } else if (signed > EPSILON) {+ tier = corroboration >= input.corroborationMin ? LESSON_TIER.preferred : LESSON_TIER.tentative;+ } else {+ tier = LESSON_TIER.avoid;+ }++ return {+ id: input.id,+ kind: input.kind,+ title: input.title,+ scope: input.scope,+ tier,+ signedScore: round4(signed),+ positiveMass: round4(acc.positiveMass),+ negativeMass: round4(acc.negativeMass),+ corroboration,+ stance,+ lastOutcomeAt: acc.lastOutcomeMs > 0 ? new Date(acc.lastOutcomeMs).toISOString() : null,+ };+}++function round4(n: number): number {+ return Math.round(n * 10000) / 10000;+}++// ----- Regeneration (mirrors active.ts) ------------------------------------++export interface RegenerateLessonsOptions {+ /** Wall clock for `generated_at`. Defaults to `new Date()`. */+ readonly now?: Date;+}++export interface RegenerateLessonsResult {+ readonly path: string;+ readonly changed: boolean;+ readonly counts: {+ readonly preferred: number;+ readonly tentative: number;+ readonly contested: number;+ readonly avoid: number;+ readonly total: number;+ };+}++/**+ * Regenerate `<vault>/Brain/lessons.md`. Idempotent: the rendered body+ * is compared byte-for-byte against the existing file before writing so+ * a no-op `dream` rerun leaves the file (and its mtime) untouched.+ *+ * Parser failures on individual preference / dead-end files are+ * swallowed and the offending file is omitted, exactly like+ * `regenerateActive` — a single corrupted frontmatter must not blank+ * the whole lessons view. `brain_doctor` is the surface for corruption.+ */+export function regenerateLessons(+ vault: string,+ opts: RegenerateLessonsOptions = {},+): RegenerateLessonsResult {+ const now = opts.now ?? new Date();+ const path = brainLessonsPath(vault);++ const preferences = readActivePreferences(vault);+ const { entries: deadEnds } = listDeadEnds(vault);++ // Tunables from `_brain.yaml:lessons.*`. A malformed config never+ // blocks the digest — fall back to the defaults (doctor's job).+ let halfLifeDays = LESSONS_HALF_LIFE_DAYS_DEFAULT;+ let corroborationMin = LESSONS_CORROBORATION_MIN_DEFAULT;+ let limit = LESSONS_LIMIT_DEFAULT;+ try {+ const cfg = loadBrainConfig(vault);+ if (cfg.lessons) {+ halfLifeDays = cfg.lessons.half_life_days ?? halfLifeDays;+ corroborationMin = cfg.lessons.corroboration_min ?? corroborationMin;+ limit = cfg.lessons.limit ?? limit;+ }+ } catch {+ // intentional fallback — config error is doctor's job to surface+ }++ const lessons = computeLessons(vault, preferences, deadEnds, {+ now,+ halfLifeDays,+ corroborationMin,+ limit,+ });++ const body = renderBody(lessons, { corroborationMin, halfLifeDays });+ const existingBody = readExistingBody(path);+ const changed = existingBody === null || existingBody !== body.trim();+ if (changed) {+ atomicWriteFileSync(path, renderDocument(body, isoSecond(now)));+ }++ const counts = {+ preferred: lessons.filter((l) => l.tier === LESSON_TIER.preferred).length,+ tentative: lessons.filter((l) => l.tier === LESSON_TIER.tentative).length,+ contested: lessons.filter((l) => l.tier === LESSON_TIER.contested).length,+ avoid: lessons.filter((l) => l.tier === LESSON_TIER.avoid).length,+ total: lessons.length,+ };+ return { path, changed, counts };+}++/**+ * Fire-and-warn wrapper around {@link regenerateLessons}, matching+ * `regenerateActiveQuiet`. The lessons digest is a derived view: a+ * failed write (disk full, permissions) must not mask the success of+ * the `dream` pass that triggered it. The next pass retries.+ */+export function regenerateLessonsQuiet(vault: string, opts: RegenerateLessonsOptions = {}): void {+ try {+ regenerateLessons(vault, opts);+ } catch (err) {+ process.stderr.write(`warning: regenerate lessons.md failed: ${(err as Error).message}\n`);+ }+}++// ----- Scan helpers --------------------------------------------------------++function readActivePreferences(vault: string): BrainPreference[] {+ const dirs = brainDirs(vault);+ if (!existsSync(dirs.preferences)) return [];+ const out: BrainPreference[] = [];+ for (const name of readdirSync(dirs.preferences)) {+ if (!name.endsWith(".md")) continue;+ try {+ out.push(parsePreference(join(dirs.preferences, name)));+ } catch {+ // Corrupted / status-folder-mismatched — omit; doctor flags it.+ }+ }+ return out;+}++// ----- Render helpers ------------------------------------------------------++interface RenderMeta {+ readonly corroborationMin: number;+ readonly halfLifeDays: number;+}++function renderBody(lessons: ReadonlyArray<LessonEntry>, meta: RenderMeta): string {+ const out: string[] = [];+ out.push("# Lessons");+ out.push("");+ out.push("Auto-generated by `dream`. Do not edit — changes will be overwritten.");+ out.push("");+ out.push(+ `Signed, recency-decayed outcome corpus (half-life ${meta.halfLifeDays}d). ` ++ `A rule reaches **preferred** once ≥${meta.corroborationMin} distinct results corroborate it.`,+ );+ out.push("");++ if (lessons.length === 0) {+ out.push("_No lessons yet — record apply-evidence outcomes and dead-ends to seed the corpus._");+ while (out.length > 0 && out[out.length - 1] === "") out.pop();+ return out.join("\n") + "\n";+ }++ const sections: ReadonlyArray<{ tier: LessonTier; heading: string; blurb: string }> = [+ {+ tier: LESSON_TIER.preferred,+ heading: "Preferred",+ blurb: "Net-positive and corroborated by enough distinct results.",+ },+ {+ tier: LESSON_TIER.tentative,+ heading: "Tentative",+ blurb: "Net-positive but not yet corroborated by enough distinct results.",+ },+ {+ tier: LESSON_TIER.contested,+ heading: "Contested",+ blurb: "Mixed recent evidence — the stance shown is the one recency currently favours.",+ },+ {+ tier: LESSON_TIER.avoid,+ heading: "Avoid",+ blurb: "Dead-ends and rules whose recent evidence is net-negative.",+ },+ ];++ for (const section of sections) {+ const group = lessons.filter((l) => l.tier === section.tier);+ if (group.length === 0) continue;+ out.push(`## ${section.heading} (${group.length})`);+ out.push("");+ out.push(`_${section.blurb}_`);+ out.push("");+ for (const l of group) out.push(renderLine(l));+ out.push("");+ }++ while (out.length > 0 && out[out.length - 1] === "") out.pop();+ return out.join("\n") + "\n";+}++function renderLine(l: LessonEntry): string {+ const tags: string[] = [];+ if (l.scope) tags.push(`scope: ${l.scope}`);+ tags.push(`score: ${formatSigned(l.signedScore)}`);+ if (l.tier === LESSON_TIER.preferred || l.tier === LESSON_TIER.tentative) {+ tags.push(`corroboration: ${l.corroboration}`);+ }+ if (l.tier === LESSON_TIER.contested && l.stance) {+ tags.push(`leaning: ${l.stance}`);+ }+ return `- \`${l.id}\` (${tags.join(", ")}) — ${l.title}`;+}++function formatSigned(n: number): string {+ const fixed = n.toFixed(2);+ return n > 0 ? `+${fixed}` : fixed;+}++function renderDocument(body: string, generatedAt: string): string {+ return [+ "---",+ `kind: ${FRONTMATTER_KIND}`,+ `generated_at: ${generatedAt}`,+ "---",+ "",+ body.trimEnd(),+ "",+ ].join("\n");+}++function readExistingBody(path: string): string | null {+ if (!existsSync(path)) return null;+ try {+ const [, body] = parseFrontmatter(path);+ return body;+ } catch {+ return null;+ }+}diff --git a/src/core/brain/paths.ts b/src/core/brain/paths.tsindex d7633c26..ee51ac4f 100644--- a/src/core/brain/paths.ts+++ b/src/core/brain/paths.ts@@ -72,6 +72,7 @@ export const BRAIN_ARTIFACTS_REL = posix.join(BRAIN_ROOT_REL, ".artifacts");export const BRAIN_CONFIG_FILE = "_brain.yaml";export const BRAIN_MANUAL_FILE = "_BRAIN.md";export const BRAIN_ACTIVE_FILE = "active.md";+export const BRAIN_LESSONS_FILE = "lessons.md";export const BRAIN_PINNED_FILE = "pinned.md";export const BRAIN_INDEX_FILE = "_INDEX.md";@@ -143,6 +144,17 @@ export function brainActivePath(vault: string): string {return ensureInsideVault(join(brainDirs(vault).brain, BRAIN_ACTIVE_FILE), vault);}+/**+ * Path of the auto-generated lessons digest (`Brain/lessons.md`) written+ * by `dream`: the unified, signed, recency-scored corpus over+ * preferences and dead-ends. Read by the `SessionStart` / `PostCompact`+ * hook alongside `active.md` and exposed as the MCP resource+ * `osb://lessons`.+ */+export function brainLessonsPath(vault: string): string {+ return ensureInsideVault(join(brainDirs(vault).brain, BRAIN_LESSONS_FILE), vault);+}+/** Path of the transient current-task scratchpad read by `brain_context`. */export function brainPinnedPath(vault: string): string {return ensureInsideVault(join(brainDirs(vault).brain, BRAIN_PINNED_FILE), vault);@@ -274,6 +286,16 @@ export function proceduralRecurrencePath(vault: string): string {return ensureInsideVault(join(vault, BRAIN_LOG_REL, "recurrence-support.jsonl"), vault);}+/**+ * Cross-query demand ledger path: `Brain/log/query-demand.jsonl`. A+ * rolling, byte-budget-capped append-only log of normalized recall+ * queries with their result count and IDF-weighted coverage, aggregated+ * to surface recurring queries the vault answers poorly (unmet demand).+ */+export function queryDemandLogPath(vault: string): string {+ return ensureInsideVault(join(vault, BRAIN_LOG_REL, "query-demand.jsonl"), vault);+}+/** Log file for the given UTC date: `Brain/log/<YYYY-MM-DD>.md`. */export function logPath(vault: string, date: string): string {const d = validateIsoDate(date);diff --git a/src/core/brain/policy.ts b/src/core/brain/policy.tsindex 670a88fc..ceea5337 100644--- a/src/core/brain/policy.ts+++ b/src/core/brain/policy.ts@@ -30,6 +30,7 @@ import type {BrainGuardrailConfig,BrainHealthConfig,BrainHygieneConfig,+ BrainLessonsConfig,BrainRecallConfig,BrainLinkGraphConfig,BrainNotesConfig,@@ -87,6 +88,25 @@ export const INJECT_BUDGET_CHARS_MAX = 200_000;/** Default top-N limit when `_brain.yaml` lacks `active.most_applied.limit`. */export const MOST_APPLIED_LIMIT_DEFAULT = 10;+/**+ * Bounds and defaults for the `lessons:` block (t_62363378) that tunes+ * the signed, recency-scored lessons digest (`Brain/lessons.md`).+ * Out-of-range values are hard errors — an operator-tunable knob must+ * never silently clamp. The half-life default matches the working-memory+ * continuity decay (30 days); the corroboration floor (2 distinct+ * results) keeps a one-off application from promoting straight to+ * `preferred`.+ */+export const LESSONS_HALF_LIFE_DAYS_DEFAULT = 30;+export const LESSONS_HALF_LIFE_DAYS_MIN = 1;+export const LESSONS_HALF_LIFE_DAYS_MAX = 365;+export const LESSONS_CORROBORATION_MIN_DEFAULT = 2;+export const LESSONS_CORROBORATION_MIN_MIN = 1;+export const LESSONS_CORROBORATION_MIN_MAX = 100;+export const LESSONS_LIMIT_DEFAULT = 20;+export const LESSONS_LIMIT_MIN = 1;+export const LESSONS_LIMIT_MAX = 200;+/*** Hard upper bound on `guardrails.instruction_file_max_lines`. Any* value above this is a misconfiguration - vault-root instruction@@ -913,6 +933,56 @@ export function validateBrainConfigDetailed(};}+ // Optional `lessons:` block (t_62363378). Tunes the signed,+ // recency-scored lessons digest. Hard-error on out-of-range values —+ // operator-tunable knobs must never silently fall back to defaults.+ let lessons: BrainLessonsConfig | undefined;+ if ("lessons" in obj) {+ const rawLessons = obj["lessons"];+ if (typeof rawLessons !== "object" || rawLessons === null || Array.isArray(rawLessons)) {+ throw new BrainConfigError(+ `block must be a map of keys; got ${describe(rawLessons)}`,+ "lessons",+ source,+ );+ }+ const lessonsMap = rawLessons as Record<string, unknown>;+ const halfLife = readBoundedInt(+ lessonsMap,+ "half_life_days",+ LESSONS_HALF_LIFE_DAYS_MIN,+ LESSONS_HALF_LIFE_DAYS_MAX,+ source,+ );+ const corroborationMin = readBoundedInt(+ lessonsMap,+ "corroboration_min",+ LESSONS_CORROBORATION_MIN_MIN,+ LESSONS_CORROBORATION_MIN_MAX,+ source,+ );+ const lessonsLimit = readBoundedInt(+ lessonsMap,+ "limit",+ LESSONS_LIMIT_MIN,+ LESSONS_LIMIT_MAX,+ source,+ );+ for (const k of Object.keys(lessonsMap)) {+ if (k !== "half_life_days" && k !== "corroboration_min" && k !== "limit") {+ warnings.push({+ path: source ?? "<config>",+ message: `lessons.${k}: unknown field ignored (forward-compat)`,+ });+ }+ }+ lessons = {+ ...(halfLife !== undefined ? { half_life_days: halfLife } : {}),+ ...(corroborationMin !== undefined ? { corroboration_min: corroborationMin } : {}),+ ...(lessonsLimit !== undefined ? { limit: lessonsLimit } : {}),+ };+ }+// Optional `discipline_report` section. On any type mismatch, emit a// warning and drop the section (return undefined) rather than throwing —// the rest of the CLI surface must keep working.@@ -1563,6 +1633,7 @@ export function validateBrainConfigDetailed("snapshots","vault","active",+ "lessons","discipline_report","guardrails","link_graph",@@ -1757,6 +1828,7 @@ export function validateBrainConfigDetailed(},...(vault !== undefined ? { vault } : {}),...(active !== undefined ? { active } : {}),+ ...(lessons !== undefined ? { lessons } : {}),...(disciplineReport !== undefined ? { discipline_report: disciplineReport } : {}),...(guardrails !== undefined ? { guardrails } : {}),...(linkGraph !== undefined ? { link_graph: linkGraph } : {}),@@ -1827,6 +1899,32 @@ function requireUnitInterval(field: string, value: unknown, source: string | nul}}+/**+ * Read an optional integer field bounded to `[min, max]` from a config+ * sub-map. Returns `undefined` when the key is absent; throws a+ * {@link BrainConfigError} (never clamps) when present but out of range+ * or non-integer. Shared by the `lessons:` block loader.+ */+function readBoundedInt(+ map: Record<string, unknown>,+ key: string,+ min: number,+ max: number,+ source: string | null,+ field: string = `lessons.${key}`,+): number | undefined {+ if (!(key in map)) return undefined;+ const raw = map[key];+ if (typeof raw !== "number" || !Number.isInteger(raw) || raw < min || raw > max) {+ throw new BrainConfigError(+ `must be an integer between ${min} and ${max}; got ${describe(raw)}`,+ field,+ source,+ );+ }+ return raw;+}+function describe(value: unknown): string {if (value === null) return "null";if (value === undefined) return "undefined";diff --git a/src/core/brain/provenance/epistemic.ts b/src/core/brain/provenance/epistemic.tsnew file mode 100644index 00000000..655fca7c--- /dev/null+++ b/src/core/brain/provenance/epistemic.ts@@ -0,0 +1,120 @@+/**+ * Epistemic provenance markers (ACM - Artifact-Constrained Metacognition).+ *+ * Every item placed into a model's context should say how grounded it is, so+ * the consuming model can tell a source-backed fact from a conjecture and stop+ * treating both as equally true. This module owns the fixed status vocabulary+ * and the deterministic derivation of a status from a page's existing+ * frontmatter - no manual tagging required.+ *+ * The five statuses, most-grounded first:+ * - `observed` - an operator-stated, evidence-backed fact.+ * - `derived` - inferred/deduced from cited premises (a derived fact).+ * - `hypothesis` - a conjecture on trial (unconfirmed / quarantined).+ * - `plan` - intended, not-yet-true (explicit tag only).+ * - `unknown` - an acknowledged gap or contested/disputed page.+ *+ * Each marker carries `evidenceRefs`: the page's `evidenced_by` wikilinks+ * (origin signals for a stated rule, premise facts for a derived one), so the+ * consumer can follow the grounding.+ *+ * Derivation reuses the existing graph metadata (provenance trust level,+ * preference status, lifecycle) rather than a new hand-maintained field. A+ * page MAY still override the result with an explicit `epistemic:` frontmatter+ * value - the only path that can yield `plan`, which no automatic signal maps+ * to. This mirrors provenance.ts: the primitive owns the vocabulary and the+ * mapping, never a specific record's frontmatter schema.+ *+ * Language-agnostic: the status is a fixed structural token set, never derived+ * from natural-language vocabulary.+ */++import { asProvenanceLevel } from "./provenance.ts";+import { readLifecycle, PAGE_LIFECYCLE } from "../page-meta/lifecycle.ts";++/** Epistemic grounding of a context item, most-grounded first. */+export const EPISTEMIC_STATUS = Object.freeze({+ observed: "observed",+ derived: "derived",+ hypothesis: "hypothesis",+ plan: "plan",+ unknown: "unknown",+} as const);++export type EpistemicStatus = (typeof EPISTEMIC_STATUS)[keyof typeof EPISTEMIC_STATUS];++const ALL: ReadonlySet<string> = new Set(Object.values(EPISTEMIC_STATUS));++/**+ * Narrow an arbitrary value to an {@link EpistemicStatus}, or null when it is+ * not one. Case- and whitespace-insensitive, so an explicit `epistemic:`+ * frontmatter value round-trips without an `as` cast.+ */+export function asEpistemicStatus(value: unknown): EpistemicStatus | null {+ if (typeof value !== "string") return null;+ const token = value.trim().toLowerCase();+ return ALL.has(token) ? (token as EpistemicStatus) : null;+}++/** An epistemic marker: the status plus the evidence links that ground it. */+export interface EpistemicMarker {+ readonly status: EpistemicStatus;+ /** `evidenced_by` wikilinks - origin signals or premise facts. */+ readonly evidenceRefs: readonly string[];+}++/**+ * Read `evidenced_by` into a clean string list. Accepts an array (the normal+ * shape) or a lone string, drops blanks, and trims incidental whitespace so a+ * marker's refs are stable regardless of how the frontmatter serialized them.+ *+ * Reads the derived `_evidenced_by` key or its `normalizeDerivedKeys`-renamed+ * `evidenced_by` form, so it works on both raw and normalized frontmatter -+ * the same dual-read the `_lifecycle`/`lifecycle` readers use.+ */+export function readEvidenceRefs(meta: Readonly<Record<string, unknown>>): string[] {+ const raw = meta["_evidenced_by"] ?? meta["evidenced_by"];+ const values = Array.isArray(raw) ? raw : raw === undefined || raw === null ? [] : [raw];+ const out: string[] = [];+ for (const v of values) {+ if (typeof v !== "string") continue;+ const trimmed = v.trim();+ if (trimmed) out.push(trimmed);+ }+ return out;+}++/**+ * Derive an {@link EpistemicMarker} from a page's frontmatter.+ *+ * Precedence:+ * 1. an explicit, valid `epistemic:` value always wins (the only `plan` path);+ * 2. a `disputed` lifecycle is `unknown` (its truth is contested);+ * 3. a `deduced`/`inferred` provenance level is `derived`;+ * 4. otherwise (a `stated`/absent level) an `unconfirmed`/`quarantine`+ * preference status is `hypothesis`, and anything else is `observed`.+ *+ * The evidence refs are the page's `evidenced_by` links regardless of status.+ */+export function deriveEpistemicStatus(meta: Readonly<Record<string, unknown>>): EpistemicMarker {+ const evidenceRefs = readEvidenceRefs(meta);++ const explicit = asEpistemicStatus(meta["epistemic"]);+ if (explicit !== null) return { status: explicit, evidenceRefs };++ if (readLifecycle(meta) === PAGE_LIFECYCLE.disputed) {+ return { status: EPISTEMIC_STATUS.unknown, evidenceRefs };+ }++ const level = asProvenanceLevel(meta["provenance"]) ?? "stated";+ if (level === "deduced" || level === "inferred") {+ return { status: EPISTEMIC_STATUS.derived, evidenceRefs };+ }++ const statusRaw = meta["_status"] ?? meta["status"];+ const status = typeof statusRaw === "string" ? statusRaw.trim() : "";+ if (status === "unconfirmed" || status === "quarantine") {+ return { status: EPISTEMIC_STATUS.hypothesis, evidenceRefs };+ }+ return { status: EPISTEMIC_STATUS.observed, evidenceRefs };+}diff --git a/src/core/brain/query-demand.ts b/src/core/brain/query-demand.tsnew file mode 100644index 00000000..2234afd7--- /dev/null+++ b/src/core/brain/query-demand.ts@@ -0,0 +1,462 @@+/**+ * Cross-query demand log (t_97091fff).+ *+ * A persisted, deterministic record of every recall query against the+ * vault, together with two satisfaction signals already computed+ * elsewhere: the result count and the IDF-weighted coverage score from+ * {@link ../search/coverage.ts}. Aggregated over time, the log surfaces+ * the *demand* gap — queries operators keep asking that the vault+ * answers poorly (weak or empty) — as a prioritized backlog of what to+ * write into `Brain/` next.+ *+ * This is distinct from the existing structural gaps (dangling wikilink+ * targets in deep-synthesis) and from the in-the-moment per-query+ * coverage guard (coverage.ts / evidence-pack.ts): those look at one+ * query or one link at a time and are never persisted. Here the signal+ * is cross-query and durable.+ *+ * No LLM, no ranking model — pure counting and the reused coverage+ * score. Privacy: only the normalized significant terms are stored+ * (never the raw query), each run through the secret redactor, so a+ * query carrying a token or path cannot leak into the log.+ *+ * Storage: `Brain/log/query-demand.jsonl`, one JSON record per line,+ * rolling and byte-budget-capped (see {@link DEMAND_LOG_MAX_BYTES}).+ */++import {+ appendFileSync,+ existsSync,+ mkdirSync,+ readFileSync,+ statSync,+ writeFileSync,+} from "node:fs";+import { dirname } from "node:path";++import { ensureInsideVault } from "../path-safety.ts";+import { redactRawOutput } from "../redactor.ts";+import { queryDemandLogPath } from "./paths.ts";+import { acquireLockSync } from "./sync-lockfile.ts";+import {+ COMPLETENESS_COMPLETE_THRESHOLD,+ COMPLETENESS_PARTIAL_THRESHOLD,+ significantTerms,+ type CompletenessVerdict,+} from "../search/coverage.ts";++/** Longest single normalized term kept; longer tokens are dropped as a+ * privacy guard (a 40+ char alnum run is far likelier a secret/id than a+ * word). */+export const DEMAND_TERM_MAX_LEN = 40;++/** Most terms retained per record — bounds one pathological query's line+ * size while keeping the bucket key stable for realistic queries. */+export const DEMAND_MAX_TERMS_PER_RECORD = 24;++/** Compaction trigger: once the log file exceeds this many bytes, the+ * oldest lines are dropped so the tail fits in {@link DEMAND_LOG_COMPACT_BYTES}.+ * Bounds unbounded growth without reading the whole file on every append+ * (the common path only stats the file). */+export const DEMAND_LOG_MAX_BYTES = 1_000_000;++/** Target size after compaction — keep the newest lines whose combined+ * size is at most this. Kept below {@link DEMAND_LOG_MAX_BYTES} so+ * compaction is amortized, not run on every subsequent append. */+export const DEMAND_LOG_COMPACT_BYTES = 750_000;++/** A query must recur at least this many times to count as demand. */+export const DEMAND_DEFAULT_MIN_OCCURRENCES = 2;++/**+ * One persisted recall observation. `coverage` is absent when the caller+ * did not compute the evidence-pack coverage (the common non-verified+ * search path); the aggregator falls back to the result-presence signal+ * for those buckets.+ */+export interface QueryDemandRecord {+ readonly ts: string;+ readonly terms: ReadonlyArray<string>;+ readonly results: number;+ readonly coverage?: number;+}++export interface RecordQueryDemandInput {+ /** Raw query text — normalized/redacted to terms before it is stored. */+ readonly query?: string;+ /** Pre-normalized terms; used verbatim (after redaction) if given. */+ readonly terms?: ReadonlyArray<string>;+ readonly resultCount: number;+ /** IDF-weighted coverage (0..1) from coverage.ts, when available. */+ readonly coverage?: number | null;+ /** ISO timestamp; defaults to now. */+ readonly at?: string;+}++export interface QueryDemandFilter {+ readonly since?: string;+ readonly until?: string;+}++export interface AggregateQueryDemandOptions extends QueryDemandFilter {+ /** Minimum occurrences to surface a bucket (default 2). */+ readonly minOccurrences?: number;+ /**+ * Surface only buckets whose mean satisfaction is at or below this+ * (default {@link COMPLETENESS_COMPLETE_THRESHOLD}) — i.e. queries not+ * already answered well. Ranking by demand pushes the worst to the top.+ */+ readonly maxSatisfaction?: number;+ /** Cap on returned gaps. */+ readonly limit?: number;+}++export interface QueryDemandGap {+ /** Sorted normalized terms — the bucket key, split for display. */+ readonly terms: ReadonlyArray<string>;+ readonly occurrences: number;+ /** Mean IDF-weighted coverage over records that carried one, else null. */+ readonly meanCoverage: number | null;+ readonly meanResultCount: number;+ /** Records with zero results. */+ readonly emptyCount: number;+ readonly emptyRate: number;+ /**+ * Satisfaction in [0,1]: the coverage mean when any record carried+ * coverage, otherwise the fraction of runs that returned any result.+ */+ readonly satisfaction: number;+ /** Whether satisfaction came from coverage (true) or the result-count+ * fallback (false). */+ readonly coverageKnown: boolean;+ /** occurrences × (1 − satisfaction): recurring + unmet ranks highest. */+ readonly demandScore: number;+ /** Completeness verdict over the satisfaction score. */+ readonly verdict: CompletenessVerdict;+ readonly firstSeen: string;+ readonly lastSeen: string;+}++export interface QueryDemandReport {+ readonly totalRecords: number;+ readonly distinctQueries: number;+ readonly gaps: ReadonlyArray<QueryDemandGap>;+}++/** Length at/above which a mixed letter+digit token is treated as a+ * high-entropy secret/id (API key, UUID, hash) rather than a query word+ * and dropped. Real words this long are essentially never both alpha and+ * numeric. */+export const DEMAND_SECRET_TOKEN_MIN_LEN = 20;++/**+ * Normalize a raw query into the terms actually stored: the significant+ * terms (length ≥ 3, deduped) sorted for a stable bucket key, with the+ * privacy filters below applied. Capped at+ * {@link DEMAND_MAX_TERMS_PER_RECORD}.+ */+export function normalizeQueryTerms(query: string): string[] {+ return sanitizeTerms(significantTerms(query));+}++function sanitizeTerms(terms: ReadonlyArray<string>): string[] {+ const out = new Set<string>();+ for (const raw of terms) {+ const term = raw.toLocaleLowerCase().trim();+ if (term.length < 3 || term.length > DEMAND_TERM_MAX_LEN) continue;+ if (isSecretShapedTerm(term)) continue;+ // Belt-and-suspenders: a term the key/value redactor still rewrites+ // is secret-shaped — never a legitimate query word, so drop it rather+ // than store a `***REDACTED***` bucket key.+ if (redactRawOutput(term) !== term) continue;+ out.add(term);+ }+ return [...out].toSorted().slice(0, DEMAND_MAX_TERMS_PER_RECORD);+}++/**+ * A bare token is secret-shaped when it is long AND mixes letters with+ * digits — the signature of an API key, UUID, or hash. The redactor's+ * passes are context-based (`key: value`, bearer headers, env), so a+ * single high-entropy word carries no shape they can match; this catches+ * it before it becomes a persisted bucket key. Kept deliberately narrow+ * (length gate + both classes) so ordinary words and short alnum terms+ * (`oauth2`, `sha256`, `utf8`) are never dropped.+ */+function isSecretShapedTerm(term: string): boolean {+ if (term.length < DEMAND_SECRET_TOKEN_MIN_LEN) return false;+ return /[a-z]/.test(term) && /[0-9]/.test(term);+}++/**+ * Append one recall observation to the demand log. Terms are derived+ * from `terms` (redacted) or `query` (normalized+redacted); a query with+ * no significant terms is not recorded (returns null). Best-effort by+ * contract — callers gate this behind opt-in telemetry and must not let+ * a log write fail the search.+ */+export function recordQueryDemand(+ vault: string,+ input: RecordQueryDemandInput,+): QueryDemandRecord | null {+ const terms =+ input.terms !== undefined ? sanitizeTerms(input.terms) : normalizeQueryTerms(input.query ?? "");+ if (terms.length === 0) return null;+ const record: QueryDemandRecord = {+ ts: normalizeDemandTimestamp(input.at),+ terms,+ results: Number.isFinite(input.resultCount) ? Math.max(0, Math.floor(input.resultCount)) : 0,+ ...(typeof input.coverage === "number" && Number.isFinite(input.coverage)+ ? { coverage: clamp01(input.coverage) }+ : {}),+ };+ const path = queryDemandLogPath(vault);+ mkdirSync(ensureInsideVault(dirname(path), vault), { recursive: true });+ const handle = acquireLockSync(path);+ try {+ appendFileSync(path, `${JSON.stringify(record)}\n`, { encoding: "utf8" });+ compactIfNeeded(path);+ } finally {+ handle.release();+ }+ return record;+}++/** Read all demand records, filtered by [since, until] and sorted by ts. */+export function readQueryDemand(+ vault: string,+ filter: QueryDemandFilter = {},+): ReadonlyArray<QueryDemandRecord> {+ const path = queryDemandLogPath(vault);+ if (!existsSync(path)) return Object.freeze([]);+ const out: QueryDemandRecord[] = [];+ for (const line of readFileSync(path, "utf8").split("\n")) {+ if (!line.trim()) continue;+ const record = coerceRecord(line);+ if (record === null) continue;+ if (filter.since !== undefined && record.ts < filter.since) continue;+ if (filter.until !== undefined && record.ts > filter.until) continue;+ out.push(record);+ }+ out.sort((a, b) => (a.ts < b.ts ? -1 : a.ts > b.ts ? 1 : 0));+ return Object.freeze(out);+}++/**+ * Aggregate the demand log into ranked knowledge gaps: recurring queries+ * the vault answers poorly. Buckets by the sorted-terms key, scores each+ * by occurrences × (1 − satisfaction), and returns the worst first.+ */+export function aggregateQueryDemand(+ vault: string,+ options: AggregateQueryDemandOptions = {},+): QueryDemandReport {+ const records = readQueryDemand(vault, {+ ...(options.since !== undefined ? { since: options.since } : {}),+ ...(options.until !== undefined ? { until: options.until } : {}),+ });+ const minOccurrences = Math.max(+ 1,+ Math.floor(options.minOccurrences ?? DEMAND_DEFAULT_MIN_OCCURRENCES),+ );+ const maxSatisfaction = clamp01(options.maxSatisfaction ?? COMPLETENESS_COMPLETE_THRESHOLD);++ interface Bucket {+ terms: ReadonlyArray<string>;+ occurrences: number;+ resultSum: number;+ emptyCount: number;+ coverageSum: number;+ coverageSamples: number;+ firstSeen: string;+ lastSeen: string;+ }+ const buckets = new Map<string, Bucket>();+ for (const record of records) {+ const key = record.terms.join(" ");+ let bucket = buckets.get(key);+ if (!bucket) {+ bucket = {+ terms: record.terms,+ occurrences: 0,+ resultSum: 0,+ emptyCount: 0,+ coverageSum: 0,+ coverageSamples: 0,+ firstSeen: record.ts,+ lastSeen: record.ts,+ };+ buckets.set(key, bucket);+ }+ bucket.occurrences += 1;+ bucket.resultSum += record.results;+ if (record.results === 0) bucket.emptyCount += 1;+ if (typeof record.coverage === "number") {+ bucket.coverageSum += record.coverage;+ bucket.coverageSamples += 1;+ }+ if (record.ts < bucket.firstSeen) bucket.firstSeen = record.ts;+ if (record.ts > bucket.lastSeen) bucket.lastSeen = record.ts;+ }++ const gaps: QueryDemandGap[] = [];+ for (const bucket of buckets.values()) {+ if (bucket.occurrences < minOccurrences) continue;+ const coverageKnown = bucket.coverageSamples > 0;+ const meanCoverage = coverageKnown ? bucket.coverageSum / bucket.coverageSamples : null;+ // Prefer the reused coverage score; only when no record in this+ // bucket carried one do we fall back to "did it return anything".+ const satisfaction = coverageKnown+ ? meanCoverage!+ : (bucket.occurrences - bucket.emptyCount) / bucket.occurrences;+ if (satisfaction > maxSatisfaction) continue;+ gaps.push(+ Object.freeze({+ terms: bucket.terms,+ occurrences: bucket.occurrences,+ meanCoverage: meanCoverage === null ? null : round4(meanCoverage),+ meanResultCount: round4(bucket.resultSum / bucket.occurrences),+ emptyCount: bucket.emptyCount,+ emptyRate: round4(bucket.emptyCount / bucket.occurrences),+ satisfaction: round4(satisfaction),+ coverageKnown,+ demandScore: round4(bucket.occurrences * (1 - satisfaction)),+ verdict: verdictFor(satisfaction),+ firstSeen: bucket.firstSeen,+ lastSeen: bucket.lastSeen,+ }),+ );+ }++ gaps.sort(+ (a, b) =>+ b.demandScore - a.demandScore ||+ b.occurrences - a.occurrences ||+ a.terms.join(" ").localeCompare(b.terms.join(" ")),+ );+ const limited =+ options.limit !== undefined ? gaps.slice(0, Math.max(0, Math.floor(options.limit))) : gaps;++ return Object.freeze({+ totalRecords: records.length,+ distinctQueries: buckets.size,+ gaps: Object.freeze(limited),+ });+}++/**+ * Snake_case JSON projection of a report for the MCP tool and CLI+ * `--json` surfaces, matching the sibling recall-telemetry convention.+ * Kept here so both surfaces serialize identically.+ */+export function serializeQueryDemandReport(report: QueryDemandReport): Record<string, unknown> {+ return {+ total_records: report.totalRecords,+ distinct_queries: report.distinctQueries,+ gaps: report.gaps.map((gap) => ({+ terms: gap.terms,+ occurrences: gap.occurrences,+ mean_coverage: gap.meanCoverage,+ mean_result_count: gap.meanResultCount,+ empty_count: gap.emptyCount,+ empty_rate: gap.emptyRate,+ satisfaction: gap.satisfaction,+ coverage_known: gap.coverageKnown,+ demand_score: gap.demandScore,+ verdict: gap.verdict,+ first_seen: gap.firstSeen,+ last_seen: gap.lastSeen,+ })),+ };+}++function verdictFor(satisfaction: number): CompletenessVerdict {+ if (satisfaction >= COMPLETENESS_COMPLETE_THRESHOLD) return "complete";+ if (satisfaction >= COMPLETENESS_PARTIAL_THRESHOLD) return "partial";+ return "sparse";+}++function coerceRecord(line: string): QueryDemandRecord | null {+ let parsed: unknown;+ try {+ parsed = JSON.parse(line);+ } catch {+ return null;+ }+ if (parsed === null || typeof parsed !== "object") return null;+ const obj = parsed as Record<string, unknown>;+ const ts = obj["ts"];+ const terms = obj["terms"];+ const results = obj["results"];+ if (typeof ts !== "string") return null;+ if (!Array.isArray(terms) || !terms.every((t) => typeof t === "string")) return null;+ if (typeof results !== "number" || !Number.isFinite(results)) return null;+ const coverage = obj["coverage"];+ return {+ ts,+ terms: Object.freeze(terms as string[]),+ results: Math.max(0, Math.floor(results)),+ ...(typeof coverage === "number" && Number.isFinite(coverage)+ ? { coverage: clamp01(coverage) }+ : {}),+ };+}++/**+ * Drop the oldest lines once the file grows past+ * {@link DEMAND_LOG_MAX_BYTES}, keeping the newest lines that fit in+ * {@link DEMAND_LOG_COMPACT_BYTES}. Called under the append lock. The+ * common path only stats the file — no read — so per-append cost stays+ * O(1) until the rare compaction.+ */+function compactIfNeeded(path: string): void {+ let size: number;+ try {+ size = statSync(path).size;+ } catch {+ return;+ }+ if (size <= DEMAND_LOG_MAX_BYTES) return;+ const lines = readFileSync(path, "utf8")+ .split("\n")+ .filter((line) => line.trim().length > 0);+ const kept: string[] = [];+ let bytes = 0;+ for (let i = lines.length - 1; i >= 0; i--) {+ const line = lines[i]!;+ const lineBytes = Buffer.byteLength(line, "utf8") + 1;+ if (bytes + lineBytes > DEMAND_LOG_COMPACT_BYTES && kept.length > 0) break;+ kept.push(line);+ bytes += lineBytes;+ }+ kept.reverse();+ writeFileSync(path, kept.length > 0 ? `${kept.join("\n")}\n` : "", { encoding: "utf8" });+}++function clamp01(value: number): number {+ if (value < 0) return 0;+ if (value > 1) return 1;+ return value;+}++/**+ * Normalize a caller-supplied (or absent) timestamp into a canonical+ * millisecond-precision ISO string. `readQueryDemand` and+ * `aggregateQueryDemand` compare `ts` as raw strings (sort + range+ * filter), which is only stable when every record uses the same precision+ * — `2026-07-01T00:00:00.000Z` must sort after `2026-07-01T00:00:00Z`,+ * but lexically `.` < `Z` flips them. Parsing-and-reformatting through+ * `Date` collapses the difference and also rejects non-ISO input by+ * falling back to now, so a malformed `at` can't silently misorder the+ * log. Non-finite `Date` (invalid input) resolves to the current time.+ */+function normalizeDemandTimestamp(at: string | undefined): string {+ if (at === undefined) return new Date().toISOString();+ const parsed = new Date(at);+ const ms = parsed.getTime();+ return Number.isFinite(ms) ? parsed.toISOString() : new Date().toISOString();+}++function round4(value: number): number {+ return Math.round(value * 10_000) / 10_000;+}diff --git a/src/core/brain/recall-adequacy.ts b/src/core/brain/recall-adequacy.tsnew file mode 100644index 00000000..db8d2d5c--- /dev/null+++ b/src/core/brain/recall-adequacy.ts@@ -0,0 +1,169 @@+/**+ * Recall adequacy verdict (retrieval-precision-quality-loop, t_b8f66fec).+ *+ * A thin verdict + action layer over the relevance scores the recall+ * stack already produces (search / recall-telemetry `top_artifacts`+ * scores). It does NOT search or re-rank; given the top-k scores of a+ * recall attempt it classifies grounding fitness and names the explicit+ * low-adequacy action so callers can branch instead of always feeding+ * top-k to the LLM:+ *+ * sufficient -> proceed (grounding is strong enough to answer)+ * weak -> re_recall (broaden scope / try an alternate path first)+ * insufficient -> abstain (return an explicit 'insufficient grounding'+ * signal, and escalate for review)+ *+ * Language-agnostic and deterministic: it reads only numeric scores, so+ * it behaves identically for any prompt language. Complements the+ * epistemic-provenance card (which feeds grounded scores into this gate).+ */++export const RECALL_ADEQUACY_LEVELS = ["sufficient", "weak", "insufficient"] as const;+export type RecallAdequacyLevel = (typeof RECALL_ADEQUACY_LEVELS)[number];++export const RECALL_ADEQUACY_ACTIONS = ["proceed", "re_recall", "abstain"] as const;+export type RecallAdequacyAction = (typeof RECALL_ADEQUACY_ACTIONS)[number];++export interface RecallAdequacyThresholds {+ /** Top score at/above which recall is sufficient. */+ readonly sufficient: number;+ /** Top score at/above which recall is at least weak (below => insufficient). */+ readonly weak: number;+ /** Minimum usable-result count below which recall cannot be sufficient. */+ readonly minResults: number;+}++export const DEFAULT_RECALL_ADEQUACY_THRESHOLDS: RecallAdequacyThresholds = Object.freeze({+ sufficient: 0.6,+ weak: 0.3,+ minResults: 1,+});++export interface RecallAdequacyVerdict {+ readonly level: RecallAdequacyLevel;+ readonly action: RecallAdequacyAction;+ /** True when the result should be flagged for review / surfaced to an operator. */+ readonly escalate: boolean;+ readonly resultCount: number;+ /** Highest usable score, or 0 when there are no usable results. */+ readonly topScore: number;+ /** Mean of usable scores, or 0 when there are no usable results. */+ readonly meanScore: number;+ readonly reason: string;+}++const ACTION_BY_LEVEL: Readonly<Record<RecallAdequacyLevel, RecallAdequacyAction>> = Object.freeze({+ sufficient: "proceed",+ weak: "re_recall",+ insufficient: "abstain",+});++function resolveThresholds(+ overrides?: Partial<RecallAdequacyThresholds>,+): RecallAdequacyThresholds {+ const merged = { ...DEFAULT_RECALL_ADEQUACY_THRESHOLDS, ...overrides };+ const { sufficient, weak, minResults } = merged;+ for (const [name, value] of [+ ["sufficient", sufficient],+ ["weak", weak],+ ] as const) {+ if (!Number.isFinite(value) || value < 0 || value > 1) {+ throw new Error(`recall adequacy: ${name} threshold must be in [0,1]; got ${value}`);+ }+ }+ if (weak > sufficient) {+ throw new Error(+ `recall adequacy: weak threshold (${weak}) must not exceed sufficient threshold (${sufficient})`,+ );+ }+ if (!Number.isInteger(minResults) || minResults < 1) {+ throw new Error(`recall adequacy: minResults must be a positive integer; got ${minResults}`);+ }+ return Object.freeze({ sufficient, weak, minResults });+}++/** Round to 4 decimals so verdict payloads stay stable and compact. */+function round4(value: number): number {+ return Math.round(value * 1e4) / 1e4;+}++/**+ * Classify recall fitness from the relevance scores of a recall attempt+ * and name the explicit action. Non-finite scores are dropped; negative+ * scores clamp to 0 (search scores are normalized to [0,1]).+ */+export function assessRecallAdequacy(+ scores: ReadonlyArray<number>,+ overrides?: Partial<RecallAdequacyThresholds>,+): RecallAdequacyVerdict {+ const thresholds = resolveThresholds(overrides);+ const usable = scores.filter((s) => Number.isFinite(s)).map((s) => Math.max(0, s));+ const resultCount = usable.length;++ if (resultCount === 0) {+ return finalize("insufficient", {+ resultCount: 0,+ topScore: 0,+ meanScore: 0,+ reason: "no recall results — insufficient grounding",+ });+ }++ const topScore = Math.max(...usable);+ const meanScore = usable.reduce((a, b) => a + b, 0) / resultCount;++ if (topScore >= thresholds.sufficient) {+ if (resultCount >= thresholds.minResults) {+ return finalize("sufficient", {+ resultCount,+ topScore,+ meanScore,+ reason: `top score ${round4(topScore)} >= sufficient ${thresholds.sufficient} across ${resultCount} result(s) — sufficient grounding`,+ });+ }+ // Strong single hit but too few corroborating results: re-recall to+ // broaden before answering rather than proceed on a lone signal.+ return finalize("weak", {+ resultCount,+ topScore,+ meanScore,+ reason: `strong top score ${round4(topScore)} but only ${resultCount} result(s) < min_results ${thresholds.minResults} — re-recall to broaden`,+ });+ }++ if (topScore >= thresholds.weak) {+ return finalize("weak", {+ resultCount,+ topScore,+ meanScore,+ reason: `top score ${round4(topScore)} in [${thresholds.weak}, ${thresholds.sufficient}) — weak grounding, re-recall via alternate strategy`,+ });+ }++ return finalize("insufficient", {+ resultCount,+ topScore,+ meanScore,+ reason: `top score ${round4(topScore)} < weak ${thresholds.weak} — insufficient grounding, abstain`,+ });+}++function finalize(+ level: RecallAdequacyLevel,+ parts: {+ readonly resultCount: number;+ readonly topScore: number;+ readonly meanScore: number;+ readonly reason: string;+ },+): RecallAdequacyVerdict {+ return Object.freeze({+ level,+ action: ACTION_BY_LEVEL[level],+ escalate: level === "insufficient",+ resultCount: parts.resultCount,+ topScore: round4(parts.topScore),+ meanScore: round4(parts.meanScore),+ reason: parts.reason,+ });+}diff --git a/src/core/brain/types.ts b/src/core/brain/types.tsindex 7bc4e2f8..a2f60f2f 100644--- a/src/core/brain/types.ts+++ b/src/core/brain/types.ts@@ -905,6 +905,21 @@ export interface BrainActiveConfig {readonly inject_budget_chars?: number;}+/**+ * Container for the `lessons:` block of `_brain.yaml` (t_62363378).+ * Tunes the signed, recency-scored lessons digest (`Brain/lessons.md`).+ * Every field is optional; absence falls back to the `LESSONS_*`+ * defaults in policy.ts.+ */+export interface BrainLessonsConfig {+ /** Exponential half-life of the recency decay, in days. */+ readonly half_life_days?: number;+ /** Distinct-result count required to promote a lesson to `preferred`. */+ readonly corroboration_min?: number;+ /** Max lessons rendered into the digest. */+ readonly limit?: number;+}+/*** Optional configuration for the daily discipline report (§D of the* agent-discipline-tail design). Absent on vaults that have not opted@@ -1059,6 +1074,13 @@ export interface BrainConfig {* mirrored `most_applied` block in `brain_digest`.*/readonly active?: BrainActiveConfig;+ /**+ * Optional `lessons:` block (t_62363378). Tunes the signed,+ * recency-scored lessons digest (`Brain/lessons.md`): decay+ * half-life, corroboration threshold, and rendered limit. Absent:+ * callers fall back to the `LESSONS_*` defaults in policy.ts.+ */+ readonly lessons?: BrainLessonsConfig;/** Optional daily discipline-report configuration (§D). Absent when not configured. */readonly discipline_report?: DisciplineReportConfig;/**diff --git a/src/core/config.ts b/src/core/config.tsindex a4043337..a7dfd11d 100644--- a/src/core/config.ts+++ b/src/core/config.ts@@ -411,6 +411,68 @@ export function resolveRecallGateTelemetry(configPath?: string): boolean {return raw === "true" || raw === "1";}+/**+ * Recall adequacy thresholds (retrieval-precision-quality-loop,+ * t_b8f66fec). Configurable floors that drive the sufficient / weak /+ * insufficient verdict over existing recall relevance scores. Defaults+ * mirror DEFAULT_RECALL_ADEQUACY_THRESHOLDS (0.6 / 0.3 / 1). Invalid+ * values fail fast rather than silently reverting to a default.+ */+export function resolveRecallAdequacyThresholds(configPath?: string): {+ sufficient: number;+ weak: number;+ minResults: number;+} {+ const data = discoverConfig(configPath).data;+ const sufficient = resolveAdequacyFloor(+ "recall_adequacy_sufficient",+ process.env["OPEN_SECOND_BRAIN_RECALL_ADEQUACY_SUFFICIENT"],+ data["recall_adequacy_sufficient"],+ 0.6,+ );+ const weak = resolveAdequacyFloor(+ "recall_adequacy_weak",+ process.env["OPEN_SECOND_BRAIN_RECALL_ADEQUACY_WEAK"],+ data["recall_adequacy_weak"],+ 0.3,+ );+ if (weak > sufficient) {+ throw new Error(+ `recall_adequacy_weak (${weak}) must not exceed recall_adequacy_sufficient (${sufficient})`,+ );+ }+ const minResults = resolveAdequacyMinResults(+ process.env["OPEN_SECOND_BRAIN_RECALL_ADEQUACY_MIN_RESULTS"],+ data["recall_adequacy_min_results"],+ );+ return { sufficient, weak, minResults };+}++function resolveAdequacyFloor(+ key: string,+ env: string | undefined,+ fileValue: string | undefined,+ fallback: number,+): number {+ const raw = (env?.trim() || fileValue?.trim()) ?? "";+ if (raw === "") return fallback;+ const value = Number(raw);+ if (!Number.isFinite(value) || value < 0 || value > 1) {+ throw new Error(`${key} must be a number in [0,1]; got '${raw}'`);+ }+ return value;+}++function resolveAdequacyMinResults(env: string | undefined, fileValue: string | undefined): number {+ const raw = (env?.trim() || fileValue?.trim()) ?? "";+ if (raw === "") return 1;+ const value = Number(raw);+ if (!Number.isInteger(value) || value < 1) {+ throw new Error(`recall_adequacy_min_results must be a positive integer; got '${raw}'`);+ }+ return value;+}+/*** Generation-report tracing gate (Hindsight brain-loop ops, t_281c3edc).* Default OFF: the inbound `generation_report` continuity path staysdiff --git a/src/mcp/brain/pack-tools.ts b/src/mcp/brain/pack-tools.tsindex afaacfaf..8cc2ab45 100644--- a/src/mcp/brain/pack-tools.ts+++ b/src/mcp/brain/pack-tools.ts@@ -8,10 +8,14 @@* BRAIN_TOOLS surface.*/-import { resolveSearchFocusContextPack } from "../../core/config.ts";+import {+ resolveRecallAdequacyThresholds,+ resolveSearchFocusContextPack,+} from "../../core/config.ts";import { resolveSearchConfig } from "../../core/search/index.ts";import { readActiveSessionFocus } from "../../core/search/session-focus.ts";import { packContext } from "../../core/brain/context-pack.ts";+import { assessRecallAdequacy } from "../../core/brain/recall-adequacy.ts";import {readAnticipatoryContext,refreshAnticipatoryCache,@@ -74,6 +78,17 @@ async function toolBrainContextPack(const maxTotalChars = optionalPositiveInt(args, "max_total_chars", "brain_context_pack");const receipt = receiptOptionsFromArgs("brain_context_pack", args, "context_pack", "mcp");const telemetry = telemetryOptionsFromArgs("brain_context_pack", args, "mcp");+ // Adequacy verdict (t_b8f66fec): when the caller passes the recall+ // scores that produced this material, classify grounding fitness and+ // name the action; also persist it in the receipt for the audit trail.+ const recallScores = parseRecallScores(args["recall_scores"]);+ const adequacy =+ recallScores !== undefined+ ? assessRecallAdequacy(+ recallScores,+ resolveRecallAdequacyThresholds(ctx.configPath ?? undefined),+ )+ : undefined;// Focus wiring (Agent Surface Suite, t_5b478e47): gated on the// search_focus_context_pack config key (default off) so the default// pack stays byte-identical. Fail-soft - a broken search config@@ -100,6 +115,7 @@ async function toolBrainContextPack(...(query ? { query } : {}),...(includeLanes ? { includeLanes: true } : {}),...(receipt !== undefined ? { receipt } : {}),+ ...(adequacy !== undefined ? { recallAdequacy: adequacy } : {}),...(cacheStable || dedupRepeated? {transforms: {@@ -127,6 +143,8 @@ async function toolBrainContextPack(tokens: i.tokens,body: i.body,trimmed: i.trimmed,+ epistemic: i.epistemic,+ ...(i.evidenceRefs.length > 0 ? { evidence_refs: i.evidenceRefs } : {}),...(i.originalRank !== undefined ? { original_rank: i.originalRank } : {}),...(i.stableRank !== undefined ? { stable_rank: i.stableRank } : {}),...(i.dedupedFrom !== undefined ? { deduped_from: i.dedupedFrom } : {}),@@ -141,9 +159,52 @@ async function toolBrainContextPack(...(report.receiptId ? { receipt_id: report.receiptId } : {}),...(report.telemetryId ? { telemetry_id: report.telemetryId } : {}),...(report.lanes ? { lanes: report.lanes } : {}),+ ...(adequacy !== undefined+ ? {+ adequacy: {+ level: adequacy.level,+ action: adequacy.action,+ escalate: adequacy.escalate,+ result_count: adequacy.resultCount,+ top_score: adequacy.topScore,+ mean_score: adequacy.meanScore,+ reason: adequacy.reason,+ },+ }+ : {}),};}+/**+ * Parse the optional `recall_scores` argument. Returns `undefined` when+ * absent (verdict skipped) and throws INVALID_PARAMS on a malformed+ * shape. An empty array is a valid "no results" signal.+ */+function parseRecallScores(raw: unknown): ReadonlyArray<number> | undefined {+ if (raw === undefined || raw === null) return undefined;+ if (!Array.isArray(raw)) {+ throw new MCPError(+ INVALID_PARAMS,+ "brain_context_pack: recall_scores must be an array of numbers",+ );+ }+ if (raw.length > 200) {+ throw new MCPError(+ INVALID_PARAMS,+ "brain_context_pack: recall_scores must not exceed 200 items",+ );+ }+ for (const item of raw) {+ if (typeof item !== "number") {+ throw new MCPError(+ INVALID_PARAMS,+ "brain_context_pack: recall_scores must contain only numbers",+ );+ }+ }+ return raw as ReadonlyArray<number>;+}+// ----- brain_context_receipts ---------------------------------------------async function toolBrainContextReceipts(@@ -514,6 +575,13 @@ export const PACK_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([type: "string",description: "Optional host/runtime name for emitted receipts; defaults to `mcp`.",},+ recall_scores: {+ type: "array",+ maxItems: 200,+ items: { type: "number" },+ description:+ "Optional relevance scores of the recall behind this material. When given, the response adds an adequacy verdict (level + action) and persists it in the receipt.",+ },telemetry: {type: "boolean",description:diff --git a/src/mcp/brain/recall-tools.ts b/src/mcp/brain/recall-tools.tsindex 5424ed3c..6808ff1b 100644--- a/src/mcp/brain/recall-tools.ts+++ b/src/mcp/brain/recall-tools.ts@@ -26,6 +26,7 @@ import {expandSessionRecall,searchSessionRecall,} from "../../core/brain/session-recall.ts";+import { aggregateQueryDemand, serializeQueryDemandReport } from "../../core/brain/query-demand.ts";import { isoSecond } from "../../core/brain/time.ts";import { INVALID_PARAMS, MCPError } from "../protocol.ts";import type { ServerContext, ToolDefinition } from "../tools.ts";@@ -272,6 +273,39 @@ function coerceRecallTelemetryStatus(raw: unknown): RecallTelemetryStatus | undereturn trimmed;}+// ----- brain_knowledge_gaps (t_97091fff) -----------------------------------++async function toolBrainKnowledgeGaps(+ ctx: ServerContext,+ args: Record<string, unknown>,+): Promise<Record<string, unknown>> {+ const since = optionalStringArg("brain_knowledge_gaps", args, "since");+ const until = optionalStringArg("brain_knowledge_gaps", args, "until");+ const minOccurrences = coercePositiveInteger(+ "brain_knowledge_gaps",+ "min_occurrences",+ args["min_occurrences"],+ );+ const limit = coercePositiveInteger("brain_knowledge_gaps", "limit", args["limit"]);+ const maxSatisfaction = coerceUnitInterval("max_satisfaction", args["max_satisfaction"]);+ const report = aggregateQueryDemand(ctx.vault, {+ ...(since !== undefined ? { since } : {}),+ ...(until !== undefined ? { until } : {}),+ ...(minOccurrences !== undefined ? { minOccurrences } : {}),+ ...(maxSatisfaction !== undefined ? { maxSatisfaction } : {}),+ ...(limit !== undefined ? { limit } : {}),+ });+ return { vault_path: ctx.vault, ...serializeQueryDemandReport(report) };+}++function coerceUnitInterval(name: string, raw: unknown): number | undefined {+ if (raw === undefined || raw === null) return undefined;+ if (typeof raw !== "number" || !Number.isFinite(raw) || raw < 0 || raw > 1) {+ throw new MCPError(INVALID_PARAMS, `brain_knowledge_gaps: ${name} must be a number in [0, 1]`);+ }+ return raw;+}+// ----- brain_context_presets ----------------------------------------------async function toolBrainSessionGrep(@@ -433,6 +467,34 @@ export const RECALL_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([},handler: toolBrainRecallTelemetry,},+ {+ name: "brain_knowledge_gaps",+ previewBudget: MCP_PREVIEW_BUDGET,+ description:+ "Cross-query demand gaps: aggregate the persisted recall demand log into recurring queries the vault answers poorly, ranked by frequency × (1 − IDF-weighted coverage). Read-only; log written only by opt-in recall telemetry.",+ inputSchema: {+ type: "object",+ properties: {+ min_occurrences: {+ type: "integer",+ minimum: 1,+ description: "Minimum times a query must recur to surface (default 2).",+ },+ max_satisfaction: {+ type: "number",+ minimum: 0,+ maximum: 1,+ description:+ "Surface only queries at/below this mean satisfaction (default 0.8) — i.e. not already answered well.",+ },+ since: { type: "string", description: "Optional inclusive lower timestamp bound." },+ until: { type: "string", description: "Optional inclusive upper timestamp bound." },+ limit: { type: "integer", minimum: 1, description: "Optional maximum gaps to return." },+ },+ additionalProperties: false,+ },+ handler: toolBrainKnowledgeGaps,+ },{name: "brain_session_grep",previewBudget: MCP_PREVIEW_BUDGET,diff --git a/src/mcp/resources.ts b/src/mcp/resources.tsindex ae92d6b4..abdee416 100644--- a/src/mcp/resources.ts+++ b/src/mcp/resources.ts@@ -37,12 +37,14 @@ import { existsSync, readFileSync } from "node:fs";import { listLogMarkdownFiles } from "../core/brain/log-jsonl.ts";import { regenerateActive } from "../core/brain/active.ts";+import { regenerateLessons } from "../core/brain/lessons.ts";import { buildBacklinkIndex } from "../core/brain/backlinks.ts";import type { BacklinkRef } from "../core/brain/backlinks.ts";import { renderDigest } from "../core/brain/digest.ts";import { computeBrainStatus, type BrainStatusSnapshot } from "../core/brain/status.ts";import {brainActivePath,+ brainLessonsPath,preferencePath,retiredPath,validateIsoDate,@@ -87,6 +89,13 @@ const CONCRETE: ReadonlyArray<ResourceDescriptor> = ["Auto-generated digest of confirmed and quarantined preferences plus the three most recently retired entries. Regenerated at the tail of every dream run.",mimeType: MIME_MARKDOWN,},+ {+ uri: "osb://lessons",+ name: "Brain lessons digest",+ description:+ "Auto-generated, signed and recency-scored lessons corpus unifying preferences and dead-ends into corroboration-tiered lessons (preferred / tentative / contested / avoid). Regenerated at the tail of every dream run.",+ mimeType: MIME_MARKDOWN,+ },{uri: "osb://digest/latest",name: "Latest Brain digest",@@ -145,6 +154,8 @@ export function readResource(ctx: ResourceContext, uri: string): ResourceContentswitch (parsed.kind) {case "active":return readActive(ctx, uri);+ case "lessons":+ return readLessons(ctx, uri);case "digestLatest":return readDigestLatest(ctx, uri);case "status":@@ -164,6 +175,7 @@ export function readResource(ctx: ResourceContext, uri: string): ResourceContenttype Parsed =| { readonly kind: "active" }+ | { readonly kind: "lessons" }| { readonly kind: "digestLatest" }| { readonly kind: "status" }| { readonly kind: "preference"; readonly id: string }@@ -183,6 +195,9 @@ function parseUri(uri: string): Parsed {if (head === "preferences" && parts[1] === "active" && parts.length === 2) {return { kind: "active" };}+ if (head === "lessons" && parts.length === 1) {+ return { kind: "lessons" };+ }if (head === "digest" && parts[1] === "latest" && parts.length === 2) {return { kind: "digestLatest" };}@@ -227,6 +242,24 @@ function readActive(ctx: ResourceContext, uri: string): ResourceContent {return readMarkdown(uri, path);}+function readLessons(ctx: ResourceContext, uri: string): ResourceContent {+ const path = brainLessonsPath(ctx.vault);+ // Same on-demand-on-first-miss policy as readActive: a fresh vault+ // that has never been dreamed has no lessons.md yet — generate it+ // once so the resource is always meaningful, but never on every read.+ if (!existsSync(path)) {+ try {+ regenerateLessons(ctx.vault);+ } catch (err) {+ throw new MCPError(+ INTERNAL_ERROR,+ `lessons.md not present and on-demand regeneration failed: ${(err as Error).message}`,+ );+ }+ }+ return readMarkdown(uri, path);+}+function readDigestLatest(ctx: ResourceContext, uri: string): ResourceContent {const rendered = renderDigest(ctx.vault, { format: "markdown" });return {diff --git a/src/mcp/search-tools.ts b/src/mcp/search-tools.tsindex 598d8a22..48588e27 100644--- a/src/mcp/search-tools.ts+++ b/src/mcp/search-tools.ts@@ -25,7 +25,12 @@ import { searchAcrossVaults } from "../core/search/cross-vault.ts";import { RECALL_PROFILE_NAMES } from "../core/search/profiles.ts";import { fileContextRecall } from "../core/brain/file-recall.ts";import { withTimeout } from "../core/search/with-timeout.ts";-import { defaultConfigPath, resolveRecallGateTelemetry } from "../core/config.ts";+import {+ defaultConfigPath,+ resolveRecallAdequacyThresholds,+ resolveRecallGateTelemetry,+} from "../core/config.ts";+import { assessRecallAdequacy } from "../core/brain/recall-adequacy.ts";import { INTERNAL_ERROR, INVALID_PARAMS, MCPError } from "./protocol.ts";import type { ServerContext, ToolDefinition } from "./tools.ts";import { coerceBoolOptional, coerceStr, coerceStringOptional } from "./coerce.ts";@@ -37,6 +42,7 @@ import { parseRecallBenchmarkDataset, runRecallBenchmark } from "../core/search/import { emitRecallTelemetry } from "../core/brain/recall-telemetry.ts";import { emitGateTelemetry } from "../core/brain/gate-telemetry.ts";import { emitGatedTelemetry } from "../core/brain/continuity/emit.ts";+import { recordQueryDemand } from "../core/brain/query-demand.ts";const MCP_LIMIT_MAX = 50;const MCP_CONTENT_MAX = 600;@@ -271,6 +277,13 @@ const RECALL_GATE_INPUT_SCHEMA: Record<string, unknown> = {explicit: { type: "boolean" },telemetry_host: { type: "string", maxLength: 200 },session_id: { type: "string", maxLength: 512 },+ scores: {+ type: "array",+ maxItems: 200,+ items: { type: "number" },+ description:+ "Optional top-k recall relevance scores. When given, the gate adds an adequacy verdict: sufficient/proceed, weak/re_recall, or insufficient/abstain.",+ },},required: ["prompt"],additionalProperties: false,@@ -282,6 +295,27 @@ const RECALL_GATE_OUTPUT_SCHEMA: NonNullable<ToolDefinition["outputSchema"]> = {properties: {retrieve: { type: "boolean" },reason: { type: "string" },+ adequacy: {+ type: "object",+ required: [+ "level",+ "action",+ "escalate",+ "result_count",+ "top_score",+ "mean_score",+ "reason",+ ],+ properties: {+ level: { type: "string", enum: ["sufficient", "weak", "insufficient"] },+ action: { type: "string", enum: ["proceed", "re_recall", "abstain"] },+ escalate: { type: "boolean" },+ result_count: { type: "integer" },+ top_score: { type: "number" },+ mean_score: { type: "number" },+ reason: { type: "string" },+ },+ },},};@@ -572,6 +606,18 @@ async function toolBrainSearch(},}),);+ // Cross-query demand log (t_97091fff): persist the normalized query+ // terms, result count, and (when the evidence pack computed it) the+ // IDF-weighted coverage so recurring poorly-answered queries can be+ // surfaced as unmet-demand knowledge gaps. Gated behind the same+ // telemetry opt-in and fail-open — a log write never breaks search.+ emitGatedTelemetry(telemetry || undefined, () =>+ recordQueryDemand(ctx.vault, {+ query,+ resultCount: surfaced.length,+ coverage: outcome.evidencePack?.idfWeightedCoverage ?? null,+ }),+ );return {results: outcome.results.map((r: BrainSearchResult) => ({path: r.path,@@ -706,7 +752,47 @@ async function toolBrainRecallGate(...(sessionId !== undefined ? { sessionId } : {}),});});- return { ...decision };+ // Adequacy verdict (t_b8f66fec): thin verdict + action layer over the+ // relevance scores of a recall attempt. Only computed when the caller+ // passes `scores`, keeping the pure structural-gate contract otherwise.+ const scores = parseRecallScores(args["scores"]);+ if (scores === undefined) return { ...decision };+ const thresholds = resolveRecallAdequacyThresholds(ctx.configPath ?? undefined);+ const verdict = assessRecallAdequacy(scores, thresholds);+ return {+ ...decision,+ adequacy: {+ level: verdict.level,+ action: verdict.action,+ escalate: verdict.escalate,+ result_count: verdict.resultCount,+ top_score: verdict.topScore,+ mean_score: verdict.meanScore,+ reason: verdict.reason,+ },+ };+}++/**+ * Parse the optional `scores` argument for the recall gate. Returns+ * `undefined` when absent (verdict skipped) and throws INVALID_PARAMS on+ * a malformed shape so callers get a clear error rather than a silently+ * dropped verdict. An empty array is a valid "no results" signal.+ */+function parseRecallScores(raw: unknown): ReadonlyArray<number> | undefined {+ if (raw === undefined || raw === null) return undefined;+ if (!Array.isArray(raw)) {+ throw new MCPError(INVALID_PARAMS, "argument 'scores' must be an array of numbers");+ }+ if (raw.length > 200) {+ throw new MCPError(INVALID_PARAMS, "argument 'scores' must not exceed 200 items");+ }+ for (const item of raw) {+ if (typeof item !== "number") {+ throw new MCPError(INVALID_PARAMS, "argument 'scores' must contain only numbers");+ }+ }+ return raw as ReadonlyArray<number>;}const RECALL_FEEDBACK_INPUT_SCHEMA: Record<string, unknown> = {@@ -1140,7 +1226,7 @@ export const SEARCH_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([{name: "brain_recall_gate",description:- "Classify whether an automatic recall/surfacing attempt should run. Diagnostics only; does not search.",+ "Classify whether an automatic recall/surfacing attempt should run. Diagnostics only; does not search. Pass `scores` (a recall attempt's top-k relevance scores) to also get an adequacy verdict — sufficient (proceed) / weak (re_recall) / insufficient (abstain + escalate).",inputSchema: RECALL_GATE_INPUT_SCHEMA,outputSchema: RECALL_GATE_OUTPUT_SCHEMA,handler: toolBrainRecallGate,diff --git a/tests/cli/brain-knowledge-gaps.test.ts b/tests/cli/brain-knowledge-gaps.test.tsnew file mode 100644index 00000000..f038c498--- /dev/null+++ b/tests/cli/brain-knowledge-gaps.test.ts@@ -0,0 +1,81 @@+import { afterEach, beforeEach, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { recordQueryDemand } from "../../src/core/brain/query-demand.ts";+import { runCli } from "../helpers/run-cli.ts";++let vault: string;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-knowledge-gaps-cli-"));+ mkdirSync(join(vault, "Brain"), { recursive: true });+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+});++test("brain knowledge-gaps ranks recurring poorly-answered queries as JSON", async () => {+ for (let i = 0; i < 3; i++) {+ recordQueryDemand(vault, {+ query: "incident response runbook",+ resultCount: 0,+ coverage: 0.0,+ at: `2026-07-0${i + 1}T00:00:00.000Z`,+ });+ }+ recordQueryDemand(vault, {+ query: "deployment pipeline overview",+ resultCount: 4,+ coverage: 0.9,+ at: "2026-07-01T02:00:00.000Z",+ });+ recordQueryDemand(vault, {+ query: "deployment pipeline overview",+ resultCount: 4,+ coverage: 0.9,+ at: "2026-07-02T02:00:00.000Z",+ });++ const result = await runCli(["brain", "knowledge-gaps", "--vault", vault, "--json"]);+ expect(result.returncode).toBe(0);+ const report = JSON.parse(result.stdout);+ expect(report.total_records).toBe(5);+ expect(report.gaps).toHaveLength(1);+ expect(report.gaps[0].terms).toEqual(["incident", "response", "runbook"]);+ expect(report.gaps[0].occurrences).toBe(3);+ expect(report.gaps[0].empty_count).toBe(3);+});++test("brain knowledge-gaps renders a human summary", async () => {+ recordQueryDemand(vault, {+ query: "cron secret rotation",+ resultCount: 0,+ at: "2026-07-01T00:00:00Z",+ });+ recordQueryDemand(vault, {+ query: "cron secret rotation",+ resultCount: 0,+ at: "2026-07-02T00:00:00Z",+ });++ const result = await runCli(["brain", "knowledge-gaps", "--vault", vault]);+ expect(result.returncode).toBe(0);+ expect(result.stdout).toContain("gap(s)");+ expect(result.stdout).toContain("cron rotation secret");+});++test("brain knowledge-gaps rejects a bad --max-satisfaction", async () => {+ const result = await runCli([+ "brain",+ "knowledge-gaps",+ "--vault",+ vault,+ "--max-satisfaction",+ "5",+ ]);+ expect(result.returncode).not.toBe(0);+ expect(result.stderr).toContain("--max-satisfaction");+});diff --git a/tests/core/brain/context-pack.test.ts b/tests/core/brain/context-pack.test.tsindex 47cb5c8e..ffeec845 100644--- a/tests/core/brain/context-pack.test.ts+++ b/tests/core/brain/context-pack.test.ts@@ -370,3 +370,40 @@ describe("packContext - untrusted_source_delimiting flag (Unit 1)", () => {expect(body).toContain(HOSTILE); // content preserved, not blanked});});++describe("packContext - epistemic provenance markers (ACM)", () => {+ function writeRaw(slug: string, frontLines: string[], body = "x") {+ writeFileSync(+ join(vault, "Brain", "preferences", `pref-${slug}.md`),+ ["---", `id: pref-${slug}`, "topic: t", "principle: p", ...frontLines, "---", "", body].join(+ "\n",+ ),+ );+ }++ test("a plain preference defaults to observed with no evidence refs", () => {+ writePref("plain", { topic: "t", principle: "p", tier: "core" });+ const item = packContext(vault, { maxTokens: 10_000 }).items[0]!;+ expect(item.epistemic).toBe("observed");+ expect(item.evidenceRefs).toEqual([]);+ });++ test("a confirmed stated rule is observed and surfaces its evidence refs", () => {+ // Real preferences store derived fields `_`-prefixed with inline arrays.+ writeRaw("obs", ["provenance: stated", "_status: confirmed", '_evidenced_by: ["[[sig-1]]"]']);+ const item = packContext(vault, { maxTokens: 10_000 }).items[0]!;+ expect(item.epistemic).toBe("observed");+ expect(item.evidenceRefs).toEqual(["[[sig-1]]"]);+ });++ test("a deduced/inferred fact is derived; unconfirmed is hypothesis; disputed is unknown", () => {+ writeRaw("der", ["provenance: deduced", '_evidenced_by: ["[[pref-obs]]"]']);+ writeRaw("hyp", ["_status: unconfirmed"]);+ writeRaw("unk", ["_lifecycle: disputed", "_status: confirmed"]);+ const byId = new Map(packContext(vault, { maxTokens: 10_000 }).items.map((i) => [i.id, i]));+ expect(byId.get("pref-der")!.epistemic).toBe("derived");+ expect(byId.get("pref-der")!.evidenceRefs).toEqual(["[[pref-obs]]"]);+ expect(byId.get("pref-hyp")!.epistemic).toBe("hypothesis");+ expect(byId.get("pref-unk")!.epistemic).toBe("unknown");+ });+});diff --git a/tests/core/brain/lessons.test.ts b/tests/core/brain/lessons.test.tsnew file mode 100644index 00000000..6f8ad3a3--- /dev/null+++ b/tests/core/brain/lessons.test.ts@@ -0,0 +1,284 @@+/**+ * Tests for `src/core/brain/lessons.ts` (t_62363378).+ *+ * Drives the unified, signed, recency-scored lessons corpus:+ * - positive (apply-evidence `applied`) vs negative (`violated` /+ * `outdated`, plus dead-end notes) outcomes folded into one score;+ * - a signed, exponentially-decayed weight so a fresh dead-end+ * outweighs a stale "useful";+ * - a ≥N-distinct-results corroboration gate promoting `tentative`+ * to `preferred`, with mixed-signal nodes rendered `contested`+ * (recency-wins) and purely-negative nodes `avoid`.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdtempSync, mkdirSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import {+ computeLessons,+ regenerateLessons,+ LESSON_TIER,+ LESSON_STANCE,+} from "../../../src/core/brain/lessons.ts";+import { appendLogEvent } from "../../../src/core/brain/log.ts";+import { recordDeadEnd } from "../../../src/core/brain/dead-ends.ts";+import type { BrainPreference } from "../../../src/core/brain/types.ts";+import {+ BRAIN_APPLY_RESULT,+ BRAIN_LOG_EVENT_KIND,+ BRAIN_PREFERENCE_STATUS,+} from "../../../src/core/brain/types.ts";+import type { DeadEndEntry } from "../../../src/core/brain/dead-ends.ts";++const NOW = new Date("2026-05-20T00:00:00Z");++function buildPref(id: string, overrides: Partial<BrainPreference> = {}): BrainPreference {+ const base: BrainPreference = {+ kind: "brain-preference",+ id,+ created_at: "2026-01-01T00:00:00Z",+ confirmed_at: null,+ unconfirmed_until: "2026-01-08T00:00:00Z",+ tags: [],+ topic: id.replace(/^pref-/, ""),+ status: BRAIN_PREFERENCE_STATUS.confirmed,+ principle: `principle for ${id}`,+ evidenced_by: [],+ applied_count: 0,+ violated_count: 0,+ last_evidence_at: null,+ confidence: "medium",+ confidence_value: 0.5,+ pinned: false,+ };+ return { ...base, ...overrides };+}++function seed(+ vault: string,+ timestamp: string,+ prefWikilink: string,+ result: string,+ artifact = "[[src/foo.ts]]",+): void {+ appendLogEvent(vault, {+ timestamp,+ eventType: BRAIN_LOG_EVENT_KIND.applyEvidence,+ body: { preference: prefWikilink, artifact, agent: "tester", result },+ });+}++/** A lightweight dead-end entry (only the fields computeLessons reads). */+function deadEnd(id: string, approach: string, created_at: string): DeadEndEntry {+ return { id, path: `${id}.md`, approach, reason: "why", context: null, agent: "t", created_at };+}++describe("computeLessons", () => {+ let vault: string;+ beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "osb-lessons-"));+ mkdirSync(join(vault, "Brain"), { recursive: true });+ });+ afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+ });++ test("empty corpus yields no lessons", () => {+ expect(computeLessons(vault, [], [], { now: NOW })).toEqual([]);+ });++ test("a preference with no outcome evidence is not yet a lesson", () => {+ const result = computeLessons(vault, [buildPref("pref-a")], [], { now: NOW });+ expect(result).toEqual([]);+ });++ test("two distinct applied artifacts promote to preferred", () => {+ seed(vault, "2026-05-18T00:00:00Z", "[[pref-a]]", BRAIN_APPLY_RESULT.applied, "[[a.ts]]");+ seed(vault, "2026-05-19T00:00:00Z", "[[pref-a]]", BRAIN_APPLY_RESULT.applied, "[[b.ts]]");+ const [lesson] = computeLessons(vault, [buildPref("pref-a")], [], { now: NOW });+ expect(lesson!.tier).toBe(LESSON_TIER.preferred);+ expect(lesson!.corroboration).toBe(2);+ expect(lesson!.signedScore).toBeGreaterThan(0);+ });++ test("a single applied artifact stays tentative (below corroboration floor)", () => {+ seed(vault, "2026-05-18T00:00:00Z", "[[pref-a]]", BRAIN_APPLY_RESULT.applied, "[[a.ts]]");+ seed(vault, "2026-05-19T00:00:00Z", "[[pref-a]]", BRAIN_APPLY_RESULT.applied, "[[a.ts]]");+ const [lesson] = computeLessons(vault, [buildPref("pref-a")], [], { now: NOW });+ expect(lesson!.tier).toBe(LESSON_TIER.tentative);+ expect(lesson!.corroboration).toBe(1); // same artifact twice = one distinct result+ });++ test("corroborationMin option gates promotion", () => {+ seed(vault, "2026-05-18T00:00:00Z", "[[pref-a]]", BRAIN_APPLY_RESULT.applied, "[[a.ts]]");+ seed(vault, "2026-05-19T00:00:00Z", "[[pref-a]]", BRAIN_APPLY_RESULT.applied, "[[b.ts]]");+ const [strict] = computeLessons(vault, [buildPref("pref-a")], [], {+ now: NOW,+ corroborationMin: 3,+ });+ expect(strict!.tier).toBe(LESSON_TIER.tentative);+ });++ test("mixed applied + violated evidence renders contested", () => {+ seed(vault, "2026-05-18T00:00:00Z", "[[pref-a]]", BRAIN_APPLY_RESULT.applied, "[[a.ts]]");+ seed(vault, "2026-05-18T00:00:00Z", "[[pref-a]]", BRAIN_APPLY_RESULT.violated, "[[b.ts]]");+ const [lesson] = computeLessons(vault, [buildPref("pref-a")], [], { now: NOW });+ expect(lesson!.tier).toBe(LESSON_TIER.contested);+ expect(lesson!.stance).not.toBeNull();+ });++ test("contested stance follows recency: a fresh violation outweighs a stale applied", () => {+ // Stale positive (~4 months old), fresh negative (2 days old).+ seed(vault, "2026-01-20T00:00:00Z", "[[pref-a]]", BRAIN_APPLY_RESULT.applied, "[[a.ts]]");+ seed(vault, "2026-05-18T00:00:00Z", "[[pref-a]]", BRAIN_APPLY_RESULT.violated, "[[b.ts]]");+ const [lesson] = computeLessons(vault, [buildPref("pref-a")], [], { now: NOW });+ expect(lesson!.tier).toBe(LESSON_TIER.contested);+ expect(lesson!.stance).toBe(LESSON_STANCE.negative);+ expect(lesson!.signedScore).toBeLessThan(0);+ });++ test("a rule only ever violated is avoid", () => {+ seed(vault, "2026-05-18T00:00:00Z", "[[pref-a]]", BRAIN_APPLY_RESULT.violated, "[[a.ts]]");+ const [lesson] = computeLessons(vault, [buildPref("pref-a")], [], { now: NOW });+ expect(lesson!.tier).toBe(LESSON_TIER.avoid);+ expect(lesson!.signedScore).toBeLessThan(0);+ });++ test("dead-ends are avoid lessons with a negative score", () => {+ const de = deadEnd("de-2026-05-18-x", "tried the flaky retry loop", "2026-05-18T00:00:00Z");+ const [lesson] = computeLessons(vault, [], [de], { now: NOW });+ expect(lesson!.kind).toBe("dead-end");+ expect(lesson!.tier).toBe(LESSON_TIER.avoid);+ expect(lesson!.negativeMass).toBeGreaterThan(0);+ expect(lesson!.positiveMass).toBe(0);+ expect(lesson!.title).toBe("tried the flaky retry loop");+ });++ test("recency: a fresh dead-end outscores a stale preference in salience ranking", () => {+ // Stale applied preference (~4 months old) vs a 1-day-old dead-end.+ seed(vault, "2026-01-20T00:00:00Z", "[[pref-old]]", BRAIN_APPLY_RESULT.applied, "[[a.ts]]");+ const de = deadEnd("de-2026-05-19-x", "fresh failed approach", "2026-05-19T00:00:00Z");+ const result = computeLessons(vault, [buildPref("pref-old")], [de], { now: NOW });+ // The fresh dead-end has larger decayed mass, so it ranks first.+ expect(result[0]!.id).toBe("de-2026-05-19-x");+ });++ test("outdated counts as a negative outcome", () => {+ seed(vault, "2026-05-18T00:00:00Z", "[[pref-a]]", BRAIN_APPLY_RESULT.outdated, "[[a.ts]]");+ const [lesson] = computeLessons(vault, [buildPref("pref-a")], [], { now: NOW });+ expect(lesson!.tier).toBe(LESSON_TIER.avoid);+ });++ test("evidence predating the preference's created_at is ignored", () => {+ const pref = buildPref("pref-a", { created_at: "2026-05-01T00:00:00Z" });+ seed(vault, "2026-04-01T00:00:00Z", "[[pref-a]]", BRAIN_APPLY_RESULT.applied, "[[a.ts]]");+ const result = computeLessons(vault, [pref], [], { now: NOW });+ expect(result).toEqual([]);+ });++ test("retired-status preferences are excluded from scoring", () => {+ seed(vault, "2026-05-18T00:00:00Z", "[[pref-a]]", BRAIN_APPLY_RESULT.applied, "[[a.ts]]");+ const pref = buildPref("pref-a", { status: "retired" as BrainPreference["status"] });+ expect(computeLessons(vault, [pref], [], { now: NOW })).toEqual([]);+ });++ test("wikilink aliases collapse to the bare id", () => {+ seed(+ vault,+ "2026-05-18T00:00:00Z",+ "[[pref-a|principle text]]",+ BRAIN_APPLY_RESULT.applied,+ "[[a.ts]]",+ );+ seed(+ vault,+ "2026-05-19T00:00:00Z",+ "[[pref-a|principle text]]",+ BRAIN_APPLY_RESULT.applied,+ "[[b.ts]]",+ );+ const [lesson] = computeLessons(vault, [buildPref("pref-a")], [], { now: NOW });+ expect(lesson!.tier).toBe(LESSON_TIER.preferred);+ });++ test("halfLifeDays option sharpens the decay", () => {+ // 30 days old applied event. A short half-life shrinks its weight.+ seed(vault, "2026-04-20T00:00:00Z", "[[pref-a]]", BRAIN_APPLY_RESULT.applied, "[[a.ts]]");+ const [gentle] = computeLessons(vault, [buildPref("pref-a")], [], {+ now: NOW,+ halfLifeDays: 60,+ });+ const [sharp] = computeLessons(vault, [buildPref("pref-a")], [], { now: NOW, halfLifeDays: 5 });+ expect(gentle!.signedScore).toBeGreaterThan(sharp!.signedScore);+ });++ test("limit truncates the ranked output", () => {+ const des = [+ deadEnd("de-1", "a", "2026-05-18T00:00:00Z"),+ deadEnd("de-2", "b", "2026-05-17T00:00:00Z"),+ deadEnd("de-3", "c", "2026-05-16T00:00:00Z"),+ ];+ const result = computeLessons(vault, [], des, { now: NOW, limit: 2 });+ expect(result.length).toBe(2);+ });++ test("output is deterministic and frozen", () => {+ seed(vault, "2026-05-18T00:00:00Z", "[[pref-a]]", BRAIN_APPLY_RESULT.applied, "[[a.ts]]");+ const first = computeLessons(vault, [buildPref("pref-a")], [], { now: NOW });+ const second = computeLessons(vault, [buildPref("pref-a")], [], { now: NOW });+ expect(first).toEqual(second);+ expect(Object.isFrozen(first)).toBe(true);+ expect(Object.isFrozen(first[0])).toBe(true);+ });+});++describe("regenerateLessons", () => {+ let vault: string;+ beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "osb-lessons-regen-"));+ mkdirSync(join(vault, "Brain"), { recursive: true });+ });+ afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+ });++ test("writes a lessons.md digest and reports counts", () => {+ recordDeadEnd(vault, {+ approach: "polling loop that never converges",+ reason: "burned the token budget",+ agent: "tester",+ now: new Date("2026-05-18T00:00:00Z"),+ });+ const result = regenerateLessons(vault, { now: NOW });+ expect(result.changed).toBe(true);+ expect(result.counts.avoid).toBe(1);+ expect(result.counts.total).toBe(1);++ const body = require("node:fs").readFileSync(result.path, "utf8") as string;+ expect(body).toContain("kind: brain-lessons");+ expect(body).toContain("## Avoid (1)");+ expect(body).toContain("polling loop that never converges");+ });++ test("idempotent: a second regeneration with the same clock does not rewrite", () => {+ recordDeadEnd(vault, {+ approach: "x",+ reason: "y",+ agent: "tester",+ now: new Date("2026-05-18T00:00:00Z"),+ });+ const first = regenerateLessons(vault, { now: NOW });+ expect(first.changed).toBe(true);+ const second = regenerateLessons(vault, { now: NOW });+ expect(second.changed).toBe(false);+ });++ test("empty corpus renders a placeholder body", () => {+ const result = regenerateLessons(vault, { now: NOW });+ expect(result.counts.total).toBe(0);+ const body = require("node:fs").readFileSync(result.path, "utf8") as string;+ expect(body).toContain("No lessons yet");+ });+});diff --git a/tests/core/brain/policy-lessons.test.ts b/tests/core/brain/policy-lessons.test.tsnew file mode 100644index 00000000..7e60d2d6--- /dev/null+++ b/tests/core/brain/policy-lessons.test.ts@@ -0,0 +1,77 @@+/**+ * `lessons:` config block (t_62363378) — tunables for the signed,+ * recency-scored lessons digest (`Brain/lessons.md`).+ */++import { describe, expect, test } from "bun:test";+import { parseBrainYaml } from "../../../src/core/brain/yaml-parse.ts";+import { BrainConfigError, validateBrainConfigDetailed } from "../../../src/core/brain/policy.ts";++function validate(yaml: string) {+ return validateBrainConfigDetailed(parseBrainYaml(yaml), "<test>");+}++const HEAD = `schema_version: 1\n`;++describe("lessons config block", () => {+ test("absent block → cfg.lessons is undefined; consumers use defaults", () => {+ const { config } = validate(HEAD);+ expect(config.lessons).toBeUndefined();+ });++ test("present values load into BrainLessonsConfig", () => {+ const { config } = validate(+ HEAD + `lessons:\n half_life_days: 14\n corroboration_min: 3\n limit: 50\n`,+ );+ expect(config.lessons?.half_life_days).toBe(14);+ expect(config.lessons?.corroboration_min).toBe(3);+ expect(config.lessons?.limit).toBe(50);+ });++ test("partial block keeps other fields undefined (defaults apply downstream)", () => {+ const { config } = validate(HEAD + `lessons:\n half_life_days: 45\n`);+ expect(config.lessons?.half_life_days).toBe(45);+ expect(config.lessons?.corroboration_min).toBeUndefined();+ expect(config.lessons?.limit).toBeUndefined();+ });++ test("half_life_days: 0 rejected", () => {+ expect(() => validate(HEAD + `lessons:\n half_life_days: 0\n`)).toThrow(BrainConfigError);+ });++ test("half_life_days: 366 rejected", () => {+ expect(() => validate(HEAD + `lessons:\n half_life_days: 366\n`)).toThrow(BrainConfigError);+ });++ test("corroboration_min: 0 rejected", () => {+ expect(() => validate(HEAD + `lessons:\n corroboration_min: 0\n`)).toThrow(BrainConfigError);+ });++ test("limit: 201 rejected", () => {+ expect(() => validate(HEAD + `lessons:\n limit: 201\n`)).toThrow(BrainConfigError);+ });++ test("non-integer half_life_days rejected", () => {+ expect(() => validate(HEAD + `lessons:\n half_life_days: 30.5\n`)).toThrow(BrainConfigError);+ });++ test("string value rejected", () => {+ expect(() => validate(HEAD + `lessons:\n half_life_days: "thirty"\n`)).toThrow(+ BrainConfigError,+ );+ });++ test("unknown sub-key warns, doesn't throw", () => {+ const { warnings } = validate(HEAD + `lessons:\n half_life_days: 30\n future_thing: 1\n`);+ expect(warnings.find((w) => w.message.includes("future_thing"))).toBeDefined();+ });++ test("non-object lessons value is a hard error", () => {+ expect(() => validate(HEAD + `lessons: not-a-map\n`)).toThrow(BrainConfigError);+ });++ test("empty lessons block is permitted", () => {+ const { config } = validate(HEAD + `lessons:\n __reserved: 1\n`);+ expect(config.lessons).toEqual({});+ });+});diff --git a/tests/core/brain/provenance/epistemic.test.ts b/tests/core/brain/provenance/epistemic.test.tsnew file mode 100644index 00000000..4d1a639f--- /dev/null+++ b/tests/core/brain/provenance/epistemic.test.ts@@ -0,0 +1,98 @@+import { describe, expect, test } from "bun:test";++import {+ asEpistemicStatus,+ deriveEpistemicStatus,+ EPISTEMIC_STATUS,+ readEvidenceRefs,+} from "../../../../src/core/brain/provenance/epistemic.ts";++describe("asEpistemicStatus", () => {+ test("narrows a valid token case/space-insensitively", () => {+ expect(asEpistemicStatus("observed")).toBe("observed");+ expect(asEpistemicStatus(" Derived ")).toBe("derived");+ expect(asEpistemicStatus("PLAN")).toBe("plan");+ });++ test("rejects unknown strings and non-strings", () => {+ expect(asEpistemicStatus("guess")).toBeNull();+ expect(asEpistemicStatus(42)).toBeNull();+ expect(asEpistemicStatus(undefined)).toBeNull();+ });+});++describe("readEvidenceRefs", () => {+ test("reads an array, dropping blanks and trimming", () => {+ expect(readEvidenceRefs({ evidenced_by: ["[[sig-a]]", " [[sig-b]] ", "", 7] })).toEqual([+ "[[sig-a]]",+ "[[sig-b]]",+ ]);+ });++ test("accepts a lone string and tolerates absence", () => {+ expect(readEvidenceRefs({ evidenced_by: "[[pref-x]]" })).toEqual(["[[pref-x]]"]);+ expect(readEvidenceRefs({})).toEqual([]);+ expect(readEvidenceRefs({ evidenced_by: null })).toEqual([]);+ });+});++describe("deriveEpistemicStatus", () => {+ test("stated + confirmed is observed and carries evidence refs", () => {+ const m = deriveEpistemicStatus({+ provenance: "stated",+ status: "confirmed",+ evidenced_by: ["[[sig-1]]"],+ });+ expect(m.status).toBe(EPISTEMIC_STATUS.observed);+ expect(m.evidenceRefs).toEqual(["[[sig-1]]"]);+ });++ test("absent provenance defaults to stated -> observed", () => {+ expect(deriveEpistemicStatus({ status: "confirmed" }).status).toBe(EPISTEMIC_STATUS.observed);+ // a page with no status at all is still an authored (observed) statement+ expect(deriveEpistemicStatus({}).status).toBe(EPISTEMIC_STATUS.observed);+ });++ test("deduced / inferred provenance is derived", () => {+ expect(deriveEpistemicStatus({ provenance: "deduced", evidenced_by: ["[[pref-a]]"] })).toEqual({+ status: EPISTEMIC_STATUS.derived,+ evidenceRefs: ["[[pref-a]]"],+ });+ expect(deriveEpistemicStatus({ provenance: "inferred" }).status).toBe(EPISTEMIC_STATUS.derived);+ });++ test("unconfirmed / quarantine stated preference is hypothesis", () => {+ expect(deriveEpistemicStatus({ status: "unconfirmed" }).status).toBe(+ EPISTEMIC_STATUS.hypothesis,+ );+ expect(deriveEpistemicStatus({ status: "quarantine" }).status).toBe(+ EPISTEMIC_STATUS.hypothesis,+ );+ });++ test("a disputed lifecycle is unknown regardless of provenance", () => {+ expect(+ deriveEpistemicStatus({ _lifecycle: "disputed", provenance: "stated", status: "confirmed" })+ .status,+ ).toBe(EPISTEMIC_STATUS.unknown);+ expect(deriveEpistemicStatus({ lifecycle: "disputed" }).status).toBe(EPISTEMIC_STATUS.unknown);+ });++ test("an explicit epistemic value overrides derivation - the only plan path", () => {+ const m = deriveEpistemicStatus({+ epistemic: "plan",+ provenance: "stated",+ status: "confirmed",+ evidenced_by: ["[[sig-2]]"],+ });+ expect(m.status).toBe(EPISTEMIC_STATUS.plan);+ // evidence refs still surface under an override+ expect(m.evidenceRefs).toEqual(["[[sig-2]]"]);+ });++ test("an invalid explicit value falls through to derivation", () => {+ expect(deriveEpistemicStatus({ epistemic: "bogus", status: "unconfirmed" }).status).toBe(+ EPISTEMIC_STATUS.hypothesis,+ );+ });+});diff --git a/tests/core/brain/query-demand.test.ts b/tests/core/brain/query-demand.test.tsnew file mode 100644index 00000000..0c373e23--- /dev/null+++ b/tests/core/brain/query-demand.test.ts@@ -0,0 +1,227 @@+import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { existsSync, mkdtempSync, readFileSync, rmSync, statSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { queryDemandLogPath } from "../../../src/core/brain/paths.ts";+import {+ DEMAND_LOG_MAX_BYTES,+ aggregateQueryDemand,+ normalizeQueryTerms,+ readQueryDemand,+ recordQueryDemand,+} from "../../../src/core/brain/query-demand.ts";++let vault: string;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-query-demand-"));+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+});++describe("normalizeQueryTerms", () => {+ test("keeps significant terms, sorts, dedupes, drops short tokens", () => {+ expect(normalizeQueryTerms("How does the AUTH flow work flow")).toEqual([+ "auth",+ "does",+ "flow",+ "how",+ "the",+ "work",+ ]);+ });++ test("drops secret-shaped tokens the redactor rewrites", () => {+ const terms = normalizeQueryTerms("api key sk-abcdef0123456789abcdef0123456789 rotation");+ expect(terms).toContain("api");+ expect(terms).toContain("key");+ expect(terms).toContain("rotation");+ expect(terms.some((t) => t.includes("sk-abcdef"))).toBe(false);+ expect(terms.some((t) => t.includes("REDACTED"))).toBe(false);+ });++ test("empty when no significant terms", () => {+ expect(normalizeQueryTerms("a it")).toEqual([]);+ });+});++describe("recordQueryDemand", () => {+ test("appends a normalized record and never stores the raw query", () => {+ const record = recordQueryDemand(vault, {+ query: "deployment rollback procedure",+ resultCount: 0,+ coverage: 0.1,+ at: "2026-07-01T10:00:00.000Z",+ });+ expect(record).not.toBeNull();+ expect(record!.terms).toEqual(["deployment", "procedure", "rollback"]);+ expect(record!.results).toBe(0);+ expect(record!.coverage).toBe(0.1);++ const raw = readFileSync(queryDemandLogPath(vault), "utf8");+ expect(raw).not.toContain("deployment rollback procedure");+ expect(raw.trim().split("\n")).toHaveLength(1);+ });++ test("omits coverage when not provided", () => {+ const record = recordQueryDemand(vault, { query: "orphaned notes cleanup", resultCount: 3 });+ expect(record!.coverage).toBeUndefined();+ });++ test("does not record a query with no significant terms", () => {+ const record = recordQueryDemand(vault, { query: "a of to", resultCount: 5 });+ expect(record).toBeNull();+ expect(existsSync(queryDemandLogPath(vault))).toBe(false);+ });++ test("clamps out-of-range coverage and negative result counts", () => {+ const r = recordQueryDemand(vault, {+ terms: ["alpha"],+ resultCount: -4,+ coverage: 1.7,+ at: "2026-07-01T10:00:00.000Z",+ });+ expect(r!.results).toBe(0);+ expect(r!.coverage).toBe(1);+ });+});++describe("readQueryDemand", () => {+ test("filters by since/until and sorts by timestamp", () => {+ recordQueryDemand(vault, { terms: ["aaa"], resultCount: 1, at: "2026-07-03T00:00:00.000Z" });+ recordQueryDemand(vault, { terms: ["bbb"], resultCount: 1, at: "2026-07-01T00:00:00.000Z" });+ recordQueryDemand(vault, { terms: ["ccc"], resultCount: 1, at: "2026-07-02T00:00:00.000Z" });++ const all = readQueryDemand(vault);+ expect(all.map((r) => r.ts)).toEqual([+ "2026-07-01T00:00:00.000Z",+ "2026-07-02T00:00:00.000Z",+ "2026-07-03T00:00:00.000Z",+ ]);++ const windowed = readQueryDemand(vault, {+ since: "2026-07-02T00:00:00.000Z",+ until: "2026-07-02T23:59:59.000Z",+ });+ expect(windowed.map((r) => r.terms[0])).toEqual(["ccc"]);+ });++ test("empty when the log does not exist", () => {+ expect(readQueryDemand(vault)).toEqual([]);+ });+});++describe("aggregateQueryDemand", () => {+ test("buckets recurring poorly-answered queries and ranks by demand", () => {+ // "vault backups" asked 3x, always weak coverage -> high demand.+ for (let i = 0; i < 3; i++) {+ recordQueryDemand(vault, {+ query: "vault backups schedule",+ resultCount: 0,+ coverage: 0.05,+ at: `2026-07-0${i + 1}T00:00:00.000Z`,+ });+ }+ // "release process" asked 2x, well covered -> excluded by default.+ for (let i = 0; i < 2; i++) {+ recordQueryDemand(vault, {+ query: "release process steps",+ resultCount: 5,+ coverage: 0.95,+ at: `2026-07-0${i + 1}T01:00:00.000Z`,+ });+ }+ // "token budget" asked once -> below minOccurrences.+ recordQueryDemand(vault, {+ query: "token budget tuning",+ resultCount: 1,+ coverage: 0.2,+ at: "2026-07-01T02:00:00.000Z",+ });++ const report = aggregateQueryDemand(vault);+ expect(report.totalRecords).toBe(6);+ expect(report.distinctQueries).toBe(3);+ expect(report.gaps).toHaveLength(1);++ const gap = report.gaps[0]!;+ expect(gap.terms).toEqual(["backups", "schedule", "vault"]);+ expect(gap.occurrences).toBe(3);+ expect(gap.meanCoverage).toBeCloseTo(0.05, 5);+ expect(gap.emptyCount).toBe(3);+ expect(gap.emptyRate).toBe(1);+ expect(gap.coverageKnown).toBe(true);+ expect(gap.verdict).toBe("sparse");+ expect(gap.demandScore).toBeCloseTo(3 * (1 - 0.05), 4);+ expect(gap.firstSeen).toBe("2026-07-01T00:00:00.000Z");+ expect(gap.lastSeen).toBe("2026-07-03T00:00:00.000Z");+ });++ test("falls back to result-presence when no coverage was recorded", () => {+ recordQueryDemand(vault, { query: "cron secrets", resultCount: 0, at: "2026-07-01T00:00:00Z" });+ recordQueryDemand(vault, { query: "cron secrets", resultCount: 0, at: "2026-07-02T00:00:00Z" });++ const report = aggregateQueryDemand(vault);+ expect(report.gaps).toHaveLength(1);+ const gap = report.gaps[0]!;+ expect(gap.coverageKnown).toBe(false);+ expect(gap.meanCoverage).toBeNull();+ expect(gap.satisfaction).toBe(0);+ expect(gap.demandScore).toBe(2);+ });++ test("honors minOccurrences, maxSatisfaction, and limit", () => {+ for (let i = 0; i < 4; i++) {+ recordQueryDemand(vault, {+ query: "graph traversal depth",+ resultCount: 2,+ coverage: 0.5,+ at: `2026-07-0${i + 1}T00:00:00.000Z`,+ });+ }+ // Excluded when maxSatisfaction is stricter than 0.5.+ expect(aggregateQueryDemand(vault, { maxSatisfaction: 0.4 }).gaps).toHaveLength(0);+ // Included at the default 0.8 ceiling.+ expect(aggregateQueryDemand(vault, { maxSatisfaction: 0.8 }).gaps).toHaveLength(1);+ // Excluded when minOccurrences exceeds the count.+ expect(aggregateQueryDemand(vault, { minOccurrences: 5 }).gaps).toHaveLength(0);++ for (let i = 0; i < 3; i++) {+ recordQueryDemand(vault, {+ query: "another weak query",+ resultCount: 0,+ coverage: 0.0,+ at: `2026-07-0${i + 1}T03:00:00.000Z`,+ });+ }+ const limited = aggregateQueryDemand(vault, { limit: 1 });+ expect(limited.gaps).toHaveLength(1);+ // The zero-coverage query outranks the 0.5-coverage one.+ expect(limited.gaps[0]!.terms).toEqual(["another", "query", "weak"]);+ });+});++describe("rolling cap", () => {+ test("compacts the log once it exceeds the byte budget", () => {+ const path = queryDemandLogPath(vault);+ // Each record ~ small; write enough to blow past the 1MB cap.+ const terms = ["compaction", "stress", "budget"];+ for (let i = 0; i < 20_000; i++) {+ recordQueryDemand(vault, {+ terms,+ resultCount: 0,+ coverage: 0.1,+ at: "2026-07-01T00:00:00.000Z",+ });+ }+ const size = statSync(path).size;+ expect(size).toBeLessThanOrEqual(DEMAND_LOG_MAX_BYTES);+ // Compaction keeps recent lines readable and parseable.+ const records = readQueryDemand(vault);+ expect(records.length).toBeGreaterThan(0);+ expect(records.every((r) => r.terms.join(" ") === "budget compaction stress")).toBe(true);+ });+});diff --git a/tests/core/brain/recall-adequacy.test.ts b/tests/core/brain/recall-adequacy.test.tsnew file mode 100644index 00000000..1e25a263--- /dev/null+++ b/tests/core/brain/recall-adequacy.test.ts@@ -0,0 +1,84 @@+import { expect, test } from "bun:test";++import {+ assessRecallAdequacy,+ DEFAULT_RECALL_ADEQUACY_THRESHOLDS,+} from "../../../src/core/brain/recall-adequacy.ts";++test("a strong top hit is sufficient and proceeds", () => {+ const verdict = assessRecallAdequacy([0.82, 0.5, 0.31]);+ expect(verdict.level).toBe("sufficient");+ expect(verdict.action).toBe("proceed");+ expect(verdict.escalate).toBe(false);+ expect(verdict.topScore).toBeCloseTo(0.82);+ expect(verdict.resultCount).toBe(3);+});++test("a middling top hit is weak and triggers re-recall", () => {+ const verdict = assessRecallAdequacy([0.42, 0.3]);+ expect(verdict.level).toBe("weak");+ expect(verdict.action).toBe("re_recall");+ expect(verdict.escalate).toBe(false);+});++test("a poor top hit is insufficient, abstains, and escalates", () => {+ const verdict = assessRecallAdequacy([0.12, 0.08]);+ expect(verdict.level).toBe("insufficient");+ expect(verdict.action).toBe("abstain");+ expect(verdict.escalate).toBe(true);+});++test("no results is insufficient with zero scores and escalates", () => {+ const verdict = assessRecallAdequacy([]);+ expect(verdict.level).toBe("insufficient");+ expect(verdict.action).toBe("abstain");+ expect(verdict.escalate).toBe(true);+ expect(verdict.topScore).toBe(0);+ expect(verdict.meanScore).toBe(0);+ expect(verdict.resultCount).toBe(0);+});++test("min_results downgrades a strong-but-lonely hit to weak/re-recall", () => {+ const verdict = assessRecallAdequacy([0.9], { minResults: 2 });+ expect(verdict.level).toBe("weak");+ expect(verdict.action).toBe("re_recall");+ expect(verdict.escalate).toBe(false);+});++test("custom thresholds move the boundaries", () => {+ // Raise the sufficient floor so 0.7 is no longer sufficient.+ const verdict = assessRecallAdequacy([0.7, 0.4], { sufficient: 0.8, weak: 0.5 });+ expect(verdict.level).toBe("weak");+});++test("non-finite scores are ignored; the rest still classify", () => {+ const verdict = assessRecallAdequacy([Number.NaN, Infinity, 0.75]);+ expect(verdict.resultCount).toBe(1);+ expect(verdict.level).toBe("sufficient");+ expect(verdict.topScore).toBeCloseTo(0.75);+});++test("negative scores clamp to zero", () => {+ const verdict = assessRecallAdequacy([-0.3, 0.05]);+ expect(verdict.topScore).toBeCloseTo(0.05);+ expect(verdict.level).toBe("insufficient");+});++test("defaults are the documented 0.6 / 0.3 / 1", () => {+ expect(DEFAULT_RECALL_ADEQUACY_THRESHOLDS).toEqual({+ sufficient: 0.6,+ weak: 0.3,+ minResults: 1,+ });+});++test("invalid thresholds throw", () => {+ expect(() => assessRecallAdequacy([0.5], { sufficient: 0.2, weak: 0.5 })).toThrow();+ expect(() => assessRecallAdequacy([0.5], { sufficient: 1.5 })).toThrow();+ expect(() => assessRecallAdequacy([0.5], { minResults: 0 })).toThrow();+});++test("verdict carries a human-readable reason", () => {+ expect(assessRecallAdequacy([0.9]).reason).toContain("sufficient");+ expect(assessRecallAdequacy([]).reason.length).toBeGreaterThan(0);+});diff --git a/tests/core/config.test.ts b/tests/core/config.test.tsindex 71216c01..b1749336 100644--- a/tests/core/config.test.ts+++ b/tests/core/config.test.ts@@ -10,6 +10,7 @@ import {redactMapping,resolveAgentName,resolveLinkOutputFormat,+ resolveRecallAdequacyThresholds,resolveSkillsAttachTriggers,resolveSkillsDir,resolveTimezone,@@ -32,6 +33,9 @@ beforeEach(() => {"OBSIDIAN_LINK_FORMAT","OPEN_SECOND_BRAIN_SKILLS_DIR","OPEN_SECOND_BRAIN_SKILLS_ATTACH_TRIGGERS",+ "OPEN_SECOND_BRAIN_RECALL_ADEQUACY_SUFFICIENT",+ "OPEN_SECOND_BRAIN_RECALL_ADEQUACY_WEAK",+ "OPEN_SECOND_BRAIN_RECALL_ADEQUACY_MIN_RESULTS",]) {savedEnv[k] = process.env[k];delete process.env[k];@@ -357,3 +361,53 @@ describe("resolveSkillsAttachTriggers", () => {expect(resolveSkillsAttachTriggers(cfg)).toBe(true);});});++describe("resolveRecallAdequacyThresholds", () => {+ test("defaults to 0.6 / 0.3 / 1 when unset", () => {+ const cfg = join(tmp, "config.yaml");+ writeFileSync(cfg, "");+ expect(resolveRecallAdequacyThresholds(cfg)).toEqual({+ sufficient: 0.6,+ weak: 0.3,+ minResults: 1,+ });+ });++ test("reads configured floors and min_results", () => {+ const cfg = join(tmp, "config.yaml");+ writeFileSync(+ cfg,+ 'recall_adequacy_sufficient: "0.75"\nrecall_adequacy_weak: "0.4"\nrecall_adequacy_min_results: "2"\n',+ );+ expect(resolveRecallAdequacyThresholds(cfg)).toEqual({+ sufficient: 0.75,+ weak: 0.4,+ minResults: 2,+ });+ });++ test("env overrides the config file", () => {+ const cfg = join(tmp, "config.yaml");+ writeFileSync(cfg, 'recall_adequacy_sufficient: "0.6"\n');+ process.env["OPEN_SECOND_BRAIN_RECALL_ADEQUACY_SUFFICIENT"] = "0.9";+ expect(resolveRecallAdequacyThresholds(cfg).sufficient).toBe(0.9);+ });++ test("rejects out-of-range floors", () => {+ const cfg = join(tmp, "config.yaml");+ writeFileSync(cfg, 'recall_adequacy_sufficient: "1.5"\n');+ expect(() => resolveRecallAdequacyThresholds(cfg)).toThrow();+ });++ test("rejects weak above sufficient", () => {+ const cfg = join(tmp, "config.yaml");+ writeFileSync(cfg, 'recall_adequacy_sufficient: "0.4"\nrecall_adequacy_weak: "0.6"\n');+ expect(() => resolveRecallAdequacyThresholds(cfg)).toThrow();+ });++ test("rejects a non-positive min_results", () => {+ const cfg = join(tmp, "config.yaml");+ writeFileSync(cfg, 'recall_adequacy_min_results: "0"\n');+ expect(() => resolveRecallAdequacyThresholds(cfg)).toThrow();+ });+});diff --git a/tests/hooks/active-inject.test.ts b/tests/hooks/active-inject.test.tsindex 5537217f..5f84a2f0 100644--- a/tests/hooks/active-inject.test.ts+++ b/tests/hooks/active-inject.test.ts@@ -60,6 +60,10 @@ function writeActive(body: string): void {writeFileSync(join(vault, "Brain", "active.md"), body, "utf8");}+function writeLessons(body: string): void {+ writeFileSync(join(vault, "Brain", "lessons.md"), body, "utf8");+}+describe("active-inject hook", () => {test("injects active.md content as SessionStart additionalContext", async () => {writeActive(@@ -206,6 +210,36 @@ describe("active-inject hook", () => {expect(injected).toContain("brain_context");});+ test("appends lessons.md body alongside active.md when present", async () => {+ writeActive(+ "---\nkind: brain-active\ngenerated_at: 2026-05-15T10:00:00Z\n---\n\n# Active Brain Preferences\n\n## Confirmed (1)\n\n- `pref-foo` — Rule body\n",+ );+ writeLessons(+ "---\nkind: brain-lessons\ngenerated_at: 2026-05-15T10:00:00Z\n---\n\n# Lessons\n\n## Avoid (1)\n\n- `de-2026-05-18-x` (score: -0.95) — the flaky retry loop\n",+ );++ const r = await runHook({ hook_event_name: "SessionStart" }, { VAULT_DIR: vault });+ expect(r.exit).toBe(0);+ const out = JSON.parse(r.stdout);+ const injected: string = out.hookSpecificOutput.additionalContext;+ expect(injected).toContain("# Active Brain Preferences");+ expect(injected).toContain("pref-foo");+ expect(injected).toContain("# Lessons");+ expect(injected).toContain("the flaky retry loop");+ });++ test("injects active.md alone when lessons.md is absent", async () => {+ writeActive(+ "---\nkind: brain-active\ngenerated_at: 2026-05-15T10:00:00Z\n---\n\n# Active Brain Preferences\n\n## Confirmed (1)\n\n- `pref-foo` — Rule body\n",+ );+ const r = await runHook({ hook_event_name: "SessionStart" }, { VAULT_DIR: vault });+ expect(r.exit).toBe(0);+ const out = JSON.parse(r.stdout);+ const injected: string = out.hookSpecificOutput.additionalContext;+ expect(injected).toContain("pref-foo");+ expect(injected).not.toContain("# Lessons");+ });+test("stays silent when active.md is empty whitespace only", async () => {writeActive(" \n \n");const r = await runHook({ hook_event_name: "SessionStart" }, { VAULT_DIR: vault });diff --git a/tests/mcp/brain-tools-parity.test.ts b/tests/mcp/brain-tools-parity.test.tsindex 6833d117..2294da0b 100644--- a/tests/mcp/brain-tools-parity.test.ts+++ b/tests/mcp/brain-tools-parity.test.ts@@ -66,6 +66,7 @@ const FROZEN_BRAIN_TOOL_NAMES = ["brain_intake_entities","brain_intent_review","brain_intention",+ "brain_knowledge_gaps","brain_labels","brain_maintenance","brain_mcp_landscape",diff --git a/tests/mcp/context-pack-tool.test.ts b/tests/mcp/context-pack-tool.test.tsindex 7c8a4e5b..2514e2b5 100644--- a/tests/mcp/context-pack-tool.test.ts+++ b/tests/mcp/context-pack-tool.test.ts@@ -100,9 +100,24 @@ describe("brain_context_pack tool — round trip", () => {const out = await callPack(server, { max_tokens: 10_000 });expect(out["max_tokens"]).toBe(10000);expect(typeof out["tokens_used"]).toBe("number");- const items = out["items"] as Array<{ id: string; tier: string }>;+ const items = out["items"] as Array<{ id: string; tier: string; epistemic: string }>;expect(items[0]!.id).toBe("pref-core");expect(items[1]!.id).toBe("pref-supp");+ // Every emitted item carries an epistemic provenance marker (ACM).+ expect(items[0]!.epistemic).toBe("observed");+ });++ test("emits derived epistemic status and evidence_refs from provenance metadata", async () => {+ writeFileSync(+ join(vault, "Brain", "preferences", "pref-derived.md"),+ '---\nid: pref-derived\ntopic: x\nprinciple: derived principle\ntier: core\nprovenance: inferred\n_evidenced_by: ["[[pref-a]]"]\n---\n',+ );+ const server = new MCPServer({ vault, configPath });+ await initialize(server);+ const out = await callPack(server, { max_tokens: 10_000 });+ const items = out["items"] as Array<{ epistemic: string; evidence_refs?: string[] }>;+ expect(items[0]!.epistemic).toBe("derived");+ expect(items[0]!.evidence_refs).toEqual(["[[pref-a]]"]);});test("returns polarity lanes when requested", async () => {diff --git a/tests/mcp/knowledge-gaps-tool.test.ts b/tests/mcp/knowledge-gaps-tool.test.tsnew file mode 100644index 00000000..6473128c--- /dev/null+++ b/tests/mcp/knowledge-gaps-tool.test.ts@@ -0,0 +1,115 @@+import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { recordQueryDemand } from "../../src/core/brain/query-demand.ts";+import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";+import { JSONRPC_VERSION, MCPServer, PROTOCOL_VERSION } from "../../src/mcp/index.ts";+import { buildToolTable } from "../../src/mcp/tools.ts";++let tmp: string;+let vault: string;+let configHome: string;+let configPath: string;++beforeEach(() => {+ tmp = mkdtempSync(join(tmpdir(), "o2b-mcp-knowledge-gaps-"));+ vault = join(tmp, "vault");+ mkdirSync(join(vault, "Brain"), { recursive: true });+ configHome = mkdtempSync(join(tmpdir(), "o2b-mcp-knowledge-gaps-cfg-"));+ configPath = join(configHome, "config.yaml");+ atomicWriteFileSync(configPath, `vault: ${vault}\nagent_name: claude\n`);+});++afterEach(() => {+ rmSync(tmp, { recursive: true, force: true });+ rmSync(configHome, { recursive: true, force: true });+});++async function initialize(server: MCPServer): Promise<void> {+ await server.handleRequest({+ jsonrpc: JSONRPC_VERSION,+ id: 1,+ method: "initialize",+ params: {+ protocolVersion: PROTOCOL_VERSION,+ capabilities: {},+ clientInfo: { name: "knowledge-gaps-test", version: "0" },+ },+ });+ await server.handleRequest({ jsonrpc: JSONRPC_VERSION, method: "notifications/initialized" });+}++async function callTool(+ server: MCPServer,+ name: string,+ args: Record<string, unknown>,+): Promise<Record<string, unknown>> {+ const response = (await server.handleRequest({+ jsonrpc: JSONRPC_VERSION,+ id: 9,+ method: "tools/call",+ params: { name, arguments: args },+ })) as+ | { result: { content: ReadonlyArray<{ type: string; text: string }> } }+ | { error: { message: string } };+ if ("error" in response) throw new Error(response.error.message);+ return JSON.parse(response.result.content[0]!.text);+}++describe("brain_knowledge_gaps tool registration", () => {+ test("registered in the full tool table only", () => {+ expect(+ buildToolTable("full").find((tool) => tool.name === "brain_knowledge_gaps"),+ ).toBeDefined();+ expect(+ buildToolTable("writer").find((tool) => tool.name === "brain_knowledge_gaps"),+ ).toBeUndefined();+ });+});++describe("brain_knowledge_gaps tool", () => {+ test("ranks recurring poorly-answered queries and honors filters", async () => {+ for (let i = 0; i < 3; i++) {+ recordQueryDemand(vault, {+ query: "vault backup schedule",+ resultCount: 0,+ coverage: 0.05,+ at: `2026-07-0${i + 1}T00:00:00.000Z`,+ });+ }+ recordQueryDemand(vault, {+ query: "release checklist steps",+ resultCount: 5,+ coverage: 0.95,+ at: "2026-07-01T01:00:00.000Z",+ });+ recordQueryDemand(vault, {+ query: "release checklist steps",+ resultCount: 5,+ coverage: 0.95,+ at: "2026-07-02T01:00:00.000Z",+ });++ const server = new MCPServer({ vault, configPath });+ await initialize(server);++ const report = await callTool(server, "brain_knowledge_gaps", {});+ expect(report["vault_path"]).toBe(vault);+ expect(report["total_records"]).toBe(5);+ const gaps = report["gaps"] as ReadonlyArray<Record<string, unknown>>;+ expect(gaps).toHaveLength(1);+ expect(gaps[0]!["terms"]).toEqual(["backup", "schedule", "vault"]);+ expect(gaps[0]!["occurrences"]).toBe(3);+ expect(gaps[0]!["verdict"]).toBe("sparse");+ });++ test("rejects an out-of-range max_satisfaction", async () => {+ const server = new MCPServer({ vault, configPath });+ await initialize(server);+ await expect(callTool(server, "brain_knowledge_gaps", { max_satisfaction: 2 })).rejects.toThrow(+ /max_satisfaction/,+ );+ });+});diff --git a/tests/mcp/mcp.test.ts b/tests/mcp/mcp.test.tsindex cf4c66fe..21cc0780 100644--- a/tests/mcp/mcp.test.ts+++ b/tests/mcp/mcp.test.ts@@ -221,6 +221,7 @@ describe("tool listing", () => {"brain_context_receipts","brain_event_trace","brain_recall_telemetry",+ "brain_knowledge_gaps","brain_context_presets","brain_pre_compact_extract","brain_session_grep",@@ -565,7 +566,8 @@ describe("stdio loop", () => {// memory-subsystem-alignment) = 86.// + brain_event_trace (dashboard-context-trace: event→trace join) = 87.// + brain_search_expand (progressive disclosure: search→expand→transcript) = 88.- expect(list.result.tools.length).toBe(88);+ // + brain_knowledge_gaps (cross-query demand log, t_97091fff) = 89.+ expect(list.result.tools.length).toBe(89);});test("returns parse error for invalid JSON", async () => {diff --git a/tests/mcp/recall-adequacy-tool.test.ts b/tests/mcp/recall-adequacy-tool.test.tsnew file mode 100644index 00000000..2b477d84--- /dev/null+++ b/tests/mcp/recall-adequacy-tool.test.ts@@ -0,0 +1,124 @@+/**+ * Recall adequacy verdict wiring (retrieval-precision-quality-loop,+ * t_b8f66fec): `brain_recall_gate` returns a verdict when `scores` is+ * supplied, `brain_context_pack` persists it in the receipt, and+ * `brain_context_receipts` surfaces it.+ */++import { afterEach, beforeEach, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { buildToolTable, findTool, type ServerContext } from "../../src/mcp/tools.ts";++let tmp: string;+let vault: string;+let configPath: string;++beforeEach(() => {+ tmp = mkdtempSync(join(tmpdir(), "o2b-adequacy-mcp-"));+ vault = join(tmp, "vault");+ mkdirSync(join(vault, "Brain"), { recursive: true });+ configPath = join(tmp, "config.yaml");+ writeFileSync(configPath, `vault: "${vault}"\n`);+});++afterEach(() => {+ rmSync(tmp, { recursive: true, force: true });+});++function ctx(): ServerContext {+ return { vault, configPath, repoRoot: null };+}++function tool(name: string) {+ return findTool(buildToolTable("full"), name);+}++test("gate omits adequacy when no scores are supplied", async () => {+ const out = (await tool("brain_recall_gate").handler(ctx(), {+ prompt: "what did we decide?",+ })) as Record<string, unknown>;+ expect(out["retrieve"]).toBe(true);+ expect(out["adequacy"]).toBeUndefined();+});++test("gate returns a sufficient/proceed verdict for strong scores", async () => {+ const out = (await tool("brain_recall_gate").handler(ctx(), {+ prompt: "what did we decide?",+ scores: [0.83, 0.4],+ })) as { adequacy: Record<string, unknown> };+ expect(out.adequacy["level"]).toBe("sufficient");+ expect(out.adequacy["action"]).toBe("proceed");+ expect(out.adequacy["escalate"]).toBe(false);+ expect(out.adequacy["result_count"]).toBe(2);+});++test("gate returns weak/re_recall and insufficient/abstain per scores", async () => {+ const weak = (await tool("brain_recall_gate").handler(ctx(), {+ prompt: "x",+ scores: [0.4],+ })) as { adequacy: Record<string, unknown> };+ expect(weak.adequacy["level"]).toBe("weak");+ expect(weak.adequacy["action"]).toBe("re_recall");++ const insufficient = (await tool("brain_recall_gate").handler(ctx(), {+ prompt: "x",+ scores: [],+ })) as { adequacy: Record<string, unknown> };+ expect(insufficient.adequacy["level"]).toBe("insufficient");+ expect(insufficient.adequacy["action"]).toBe("abstain");+ expect(insufficient.adequacy["escalate"]).toBe(true);+});++test("gate honours configurable thresholds", async () => {+ writeFileSync(configPath, `vault: "${vault}"\nrecall_adequacy_sufficient: "0.9"\n`);+ const out = (await tool("brain_recall_gate").handler(ctx(), {+ prompt: "x",+ scores: [0.7],+ })) as { adequacy: Record<string, unknown> };+ // 0.7 would be sufficient at the default 0.6 floor, but not at 0.9.+ expect(out.adequacy["level"]).toBe("weak");+});++test("gate rejects a malformed scores argument", async () => {+ await expect(+ tool("brain_recall_gate").handler(ctx(), { prompt: "x", scores: ["nope"] }),+ ).rejects.toThrow();+});++test("context_pack persists the verdict into the receipt and returns it", async () => {+ const packed = (await tool("brain_context_pack").handler(ctx(), {+ max_tokens: 1000,+ receipt: true,+ recall_scores: [0.2, 0.1],+ })) as { adequacy?: Record<string, unknown>; receipt_id?: string };+ expect(packed.adequacy).toBeDefined();+ expect(packed.adequacy!["level"]).toBe("insufficient");+ expect(packed.receipt_id).toBeDefined();++ const summaries = (await tool("brain_context_receipts").handler(ctx(), {+ operation: "list",+ })) as { receipts: Array<{ adequacy?: Record<string, unknown> }> };+ expect(summaries.receipts.length).toBeGreaterThan(0);+ const withVerdict = summaries.receipts.find((r) => r.adequacy);+ expect(withVerdict?.adequacy?.["level"]).toBe("insufficient");+ expect(withVerdict?.adequacy?.["action"]).toBe("abstain");++ const shown = (await tool("brain_context_receipts").handler(ctx(), {+ operation: "show",+ id: packed.receipt_id,+ })) as { payload: Record<string, unknown> };+ const adequacy = shown.payload["adequacy"] as Record<string, unknown>;+ expect(adequacy["level"]).toBe("insufficient");+ expect(adequacy["top_score"]).toBeCloseTo(0.2);+});++test("context_pack omits adequacy when recall_scores is absent", async () => {+ const packed = (await tool("brain_context_pack").handler(ctx(), {+ max_tokens: 1000,+ receipt: true,+ })) as { adequacy?: unknown };+ expect(packed.adequacy).toBeUndefined();+});diff --git a/tests/mcp/removed-tools.test.ts b/tests/mcp/removed-tools.test.tsindex a4767b8d..914ed406 100644--- a/tests/mcp/removed-tools.test.ts+++ b/tests/mcp/removed-tools.test.ts@@ -148,5 +148,6 @@ test("the shadow surface is gone: no hidden tools, removed names unlisted", asyn// + brain_memory_bridge (Hermes on_memory_write host bridge) = 86.// + brain_event_trace (dashboard-context-trace: event→trace join) = 87.// + brain_search_expand (progressive disclosure: search→expand→transcript) = 88.- expect(list.result.tools.length).toBe(88);+ // + brain_knowledge_gaps (cross-query demand log, t_97091fff) = 89.+ expect(list.result.tools.length).toBe(89);});diff --git a/tests/mcp/resources.test.ts b/tests/mcp/resources.test.tsindex a938eadd..b53aa0d6 100644--- a/tests/mcp/resources.test.ts+++ b/tests/mcp/resources.test.ts@@ -111,7 +111,12 @@ describe("MCP resources — capabilities and discovery", () => {await initialize(server);const r = await listResources(server);const uris = r.result.resources.map((x: any) => x.uri).toSorted();- expect(uris).toEqual(["osb://digest/latest", "osb://preferences/active", "osb://status"]);+ expect(uris).toEqual([+ "osb://digest/latest",+ "osb://lessons",+ "osb://preferences/active",+ "osb://status",+ ]);for (const desc of r.result.resources) {expect(desc.mimeType).toBe("text/markdown");expect(typeof desc.name).toBe("string");@@ -187,6 +192,19 @@ describe("MCP resources — read osb://preferences/active", () => {});});+describe("MCP resources — read osb://lessons", () => {+ test("regenerates lessons.md on demand when missing and returns its body", async () => {+ const server = makeServer();+ await initialize(server);+ const r = await read(server, "osb://lessons");+ expect(r.result.contents).toHaveLength(1);+ const c = r.result.contents[0];+ expect(c.uri).toBe("osb://lessons");+ expect(c.mimeType).toBe("text/markdown");+ expect(c.text).toContain("# Lessons");+ });+});+describe("MCP resources — read osb://digest/latest", () => {test("returns rendered digest markdown (empty window is non-empty markdown)", async () => {const server = makeServer();diff --git a/tests/mcp/search.test.ts b/tests/mcp/search.test.tsindex b89de650..5f2d532d 100644--- a/tests/mcp/search.test.ts+++ b/tests/mcp/search.test.ts@@ -18,6 +18,7 @@ import { resolveSearchConfig } from "../../src/core/search/index.ts";import { LATEST_SCHEMA_VERSION } from "../../src/core/search/schema.ts";import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";import { listRecallTelemetry } from "../../src/core/brain/recall-telemetry.ts";+import { readQueryDemand } from "../../src/core/brain/query-demand.ts";let tmp: string;let vault: string;@@ -153,6 +154,24 @@ test("brain_search can opt in to recall telemetry", async () => {});expect(records).toHaveLength(1);expect(records[0]!.payload).toMatchObject({ status: "ok", result_count: 1 });++ // The same opt-in also persists a cross-query demand record with the+ // normalized query terms and result count (t_97091fff).+ const demand = readQueryDemand(vault);+ expect(demand).toHaveLength(1);+ expect(demand[0]!.terms).toEqual(["telemetry"]);+ expect(demand[0]!.results).toBe(1);+});++test("brain_search does not persist demand records without the telemetry opt-in", async () => {+ writeMd("notes/quiet.md", "# Quiet\n\nsilent recall leaves no demand trace.");+ const cfg = resolveSearchConfig({ vault, configPath });+ await indexVault(cfg);++ const server = makeServer();+ await initialize(server);+ await call(server, "brain_search", { query: "silent", limit: 5 });+ expect(readQueryDemand(vault)).toHaveLength(0);});test("brain_search accepts structured query_document", async () => {