Skip to content

Local-first 🧠 memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly 😴 dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.

TypeScript

feat: nothing writes silently, nothing degrades silently (v1.52.0) (#178)

Sol Aitken323e2f097173
diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json
index 719a9b8b..3f99c56d 100644
--- a/.claude-plugin/plugin.json
+++ b/.claude-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "1.51.0",
+ "version": "1.52.0",
"description": "Plugin-first second brain package for AI agents and humans.",
"author": {
"name": "Open Second Brain contributors"
diff --git a/.codex-plugin/plugin.json b/.codex-plugin/plugin.json
index 8832d750..507c1751 100644
--- a/.codex-plugin/plugin.json
+++ b/.codex-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "1.51.0",
+ "version": "1.52.0",
"description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.",
"skills": "./skills",
"hooks": "./hooks/hooks.json",
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 78710a73..3fad92f4 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,6 +5,30 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+## [1.52.0] - 2026-08-23
+
+Nine tracker cards read against the live source before anything was designed, and the reading did most of the work. One card described a feature that already shipped end to end - closed, not built. One asked for privacy by riding a predicate that is inert in every default install - refused as designed, redesigned down to the truth it can tell today. Four more had premises the source contradicted: the "localized" stamp would have touched sixty-eight writers, the counter had no chokepoint to count at, the resume half already existed on the ingest lane, and the raise-on-attach the card wanted was already there behind a documented default. What survived is one rule applied eight ways: nothing writes silently, nothing degrades silently.
+
+The units share their machinery on purpose. One pure reconciliation vocabulary - attempted, found, and missing with every missing key named, no bare-count constructor - is spoken by the import read-back census, the multi-artifact write accounting, and the embedder record audit, and is pinned by the same verdict-vocabulary census that guards the project's other closed vocabularies. Two facts the wave needed - which process a write came through, and which transport carried a request - are derived by the server at the entry point, never accepted from a caller: twenty-five tool schemas already take a caller-supplied `agent` string verbatim, and this wave declined to add one more unverifiable claim beside them.
+
+### Added
+
+- **`o2b search check` measures the pending-vector count instead of inferring it.** The old recommendation branch fired on a provider-reachable proxy with no term for existing vectors, so a fully embedded, healthy vault was told to "compute the first vectors" - a false statement on the happy path, and one no test pinned. A COUNT(*) anti-join through the read-only seam replaces it: a fully embedded vault gets no recommendation, a partial one gets the count and the dry-run backfill verb that prices it, and an absent or unreadable index reports unrecorded - never zero, because an unmeasured quantity is not a measured nothing.
+- **An embedder record audit, and a restamp for the drift that costs nothing.** Nothing compared the recorded index dimension against the width of the vectors actually stored; a meta row claiming one dimension over data holding another was invisible. The audit reports that disagreement as its own third state - the record contradicts the data - distinct from drifted and from unrecorded. `o2b search restamp` repairs the one drift that needs no provider, a stale `vec_version` token: dry-run by default, `--apply` writes one cell, and a real model or dimension disagreement refuses by name because that repair costs money and belongs to the apply-gated backfill.
+- **Note updates refuse to destroy what they could not read.** An existing-but-unreadable note - permissions, a directory standing where a file should be, a mid-sync fault - parsed as an empty one, so an update destroyed frontmatter and body while reporting `updated: true`, and an append destroyed the prior body the same way. The read now raises `target_unreadable` (and a scanner-lossy frontmatter raises `target_frontmatter_lossy`) before anything is projected, and a blank body over a non-empty note refuses unless the caller passes the new `allow_empty` flag on `brain_update_note` or `brain_write_batch`. Genuinely new empty notes are untouched; the guard's docblock records the three placements it deliberately refused.
+- **Every structured Brain record carries a server-derived `origin_channel`.** `mcp-tool`, `cli`, or `import`, resolved once at the process entry point and threaded as an option - never a tool or CLI argument, and a census test proves no schema offers one. Log events, signals, continuity records, and caller-named notes are stamped; existing records are not rewritten, because backfilled provenance is the opposite of provenance; and the write-site census asserts the measured remainder that carries no stamp rather than describing it.
+- **Brain status derives its maintenance debt instead of counting it.** `log_events_since_dream` is computed from the log shards the status reader already opens, and is named for what it measures - caller-named note writes emit no log event, and the docblock says so instead of borrowing a bigger name. An unreadable shard turns the figure into a named lower bound, a dry-run dream moves nothing (pinned by a test), and the always-loaded `brain_context` carries only a newest-shard overdue flag so the bootstrap path stays cheap.
+- **The three note-producing envelope lanes hand the model its wikilink vocabulary.** Rollup, diarization, and design-note envelopes now carry a bounded `link_candidates` manifest - built from the basename walker with no embedding, filtered through the ownership rule so a withheld note is indistinguishable from an absent one, and carrying its own total and truncation flag so a partial list cannot read as complete. The envelope spine is byte-identical; the three lanes that produce no wikilinked note carry nothing new.
+- **Multi-artifact envelope writes account for themselves, durably.** The one lane that commits several artifacts from one validated payload reports attempted, found, and missing in the shared vocabulary with the first real error - never a partial success - and a mid-batch failure now also leaves a dead letter under the vault's state directory, content-addressed so a retry rewrites one file instead of accumulating stamps. A caller that drops the response is no longer the only record that a partial write happened.
+- **Imports read back what they claim.** Both import lanes re-read the vault after committing and report claimed, found, and missing with the keys named - a count alone is the misleading no-op this wave exists to remove. The sessions lane also gains resume on the ingest lane's existing checkpoint substrate - extended, not introduced - so an interrupted transcript import continues at the turn boundary instead of re-reading everything, a resumed run re-collects what its recall analysis needs so resumption never loses data, and a malformed checkpoint discards itself by name instead of silently resetting to zero.
+- **A visibility surface census, and the honesty it enforces.** `visibility: private` frontmatter is a caller-supplied view filter, not a privacy boundary - any caller may lift it, one read lane honors it, and the index stores tagged content regardless. A standing architecture census now enumerates every note-returning surface as covered or excluded-with-reason, and `o2b search check` states the boundary's absence on vaults that use the field, with the surface counts derived from the census itself rather than hand-written. The census corrected the wave's own reconnaissance on two surfaces, once in each direction.
+
+### Notes
+
+- The wikilink-stub card is closed as already shipped: `scaffold-stub` landed earlier end to end - core, tool, verb, repair-lane integration, tests - and the one unbuilt clause, auto-materialize on capture, is a silent write the shipped design declined in writing. The per-page privacy card stays open with its redesign verdict: real enforcement must reach the walker, the search pipeline, and the indexer, and belongs to a wave of its own; the census above is the coverage map that wave starts from.
+- Also refused, each with its reason recorded in the design documents: a stored writes counter (no transaction, no chokepoint, and a counter would be a new state surface where a derivation cannot desync), a caller-supplied channel field (an unverifiable stamp is a lie with a schema), backfilled provenance, a flipped embedding-ABI default (the rebuild-loop argument stands), and any re-embed verification outside the explicit apply gate.
+- An independent reviewer was given the finished branch with no knowledge of how it was built and returned two blocking findings and eleven non-blocking ones, all applied: a resumed import's recall analysis silently dropped every turn before the checkpoint boundary while reporting success; a frontmatter-only update deleted a line the scanner could not express while reporting success; a malformed checkpoint silently reset to zero; the visibility census could not see a tool registered under a constant name; the origin-channel vocabulary was invisible to the vocabulary census that should have pinned it; dead letters accumulated one file per retry against their own docblock; an unreadable log shard under-counted the maintenance debt with no signal; the dream pass built link-candidate manifests it then discarded; and a bad ingest plan id disabled resume through a bare catch. The other blocking finding was the version bump this entry is part of.
+
## [1.51.0] - 2026-08-22
Eight tracker cards picked by priority, and reconnaissance against the real source before any design - a pass that corrected three of the eight rather than following them. The salience gate the first card asked for had nothing to stand in front of: the dream pass owns no per-item model lane, only a count-triggered rollup ladder, so the gate governs what enters that fold instead. The architecture diagram the second card wanted could not draw dependencies, because the scanner's own design doc excludes import analysis and its module facts carry no edges - so the diagram draws what the scan truthfully knows, containment, and says so. And the codegraph enrichment the third card described would have required inventing a partner query surface against a constant whose docstring calls its four tokens the entire surface, absolute - so what ships is the honest half, a verdict region that states what the partner index holds without pretending to read symbols it cannot see.
@@ -7325,6 +7349,7 @@ plugin config (vault field)`, and exits with a clear
- Sandbox vault and plugin manifest fixtures for tests.
- GitHub release workflow for tag-based and manually dispatched releases.
+[1.52.0]: https://github.com/itechmeat/open-second-brain/compare/v1.51.0...v1.52.0
[1.51.0]: https://github.com/itechmeat/open-second-brain/compare/v1.50.3...v1.51.0
[1.50.3]: https://github.com/itechmeat/open-second-brain/compare/v1.50.2...v1.50.3
[1.50.2]: https://github.com/itechmeat/open-second-brain/compare/v1.50.1...v1.50.2
diff --git a/README.md b/README.md
index d4f3b4a7..0829e067 100644
--- a/README.md
+++ b/README.md
@@ -8,13 +8,13 @@ Open Second Brain plugs into [Hermes Agent](https://github.com/NousResearch/herm
## What is new
-Open Second Brain 1.51.0 is about which memories earn their keep and how generated artifacts explain what they rest on. Eight tracker cards picked by priority, read against the real source before anything was designed - a pass that corrected three of them, because the lane one wanted to gate did not exist, the dependency diagram another wanted cannot be drawn from a scanner that deliberately collects no edges, and the partner enrichment a third described would have crossed a boundary the code declares absolute. A ninth card was refused outright rather than shipped as a shell: a local transformer embedding provider stands against a written no-ML-runtime invariant, and a registered provider that cannot embed is a stub, not a seam.
+Open Second Brain 1.52.0 is the honesty wave: nothing writes silently, nothing degrades silently. Nine tracker cards were read against the live source before anything was designed, and the reading did most of the work - one card described a feature that had already shipped end to end, one asked for privacy by riding a predicate that is inert in every default install, and four more had premises the source contradicted. What survived is one rule applied eight ways.
-What shipped instead: a deterministic salience gate decides which facts enter the dream pass's expensive fold, from three signals the system already computes, and names every exclusion; a batch verb mines memory signals from imported sessions through a validated model lane that can only reach the speculative inbox, never a confirmed preference; a note delete can follow structured provenance - and only structured provenance - under one snapshot and an exact count guard; expiration can finally be set at creation and changed afterwards from every writing surface, through one normalizer; mature vault pages become staged Agent Skill drafts that materialize outside the vault only on explicit accept; the architect overview states its codegraph verdict, draws a containment diagram it can actually prove, and lists the repository's mined decision candidates; and a one-shot design-note verb grounds a topic in tension, decision, and truth records and demands exactly one recommended alternative back.
+A note update now refuses to destroy what it could not read - an unreadable note raises `target_unreadable` instead of parsing as empty, and a blank body over a non-empty note needs an explicit `allow_empty`. `o2b search check` measures instead of inferring: a real pending-vector count replaces a recommendation that told fully embedded vaults to compute their first vectors, an embedder record audit reports when the recorded dimension contradicts the stored data, and `o2b search restamp` repairs the one drift that needs no provider. Every structured Brain record carries a server-derived `origin_channel` no caller can supply; Brain status derives `log_events_since_dream` from the shards it already reads and names an unreadable shard a lower bound; imports read back what they claim and report every missing key by name, and the sessions lane resumes an interrupted transcript at the turn boundary; the three note-producing envelope lanes hand the generating agent a bounded, ownership-filtered wikilink candidate manifest; a failed multi-artifact write leaves a durable dead letter; and a standing census documents - truthfully - that `visibility: private` is a view filter, not a privacy boundary, with `o2b search check` saying so on vaults that use the field.
-Under all of it sits one new piece of shared structure: the three divergent needs-llm-step envelope shapes the project had accumulated now share a named type, and the payload rules the deliberately shallow shape language cannot express - cardinality, caps, confidence floors - live in one fail-closed semantic-check registry. An independent reviewer given the finished branch returned one blocking finding and eleven non-blocking ones, all applied; the [CHANGELOG](CHANGELOG.md) names each.
+The units speak one vocabulary on purpose: attempted, found, and missing with every missing key named, one pure module consumed by the import census, the write accounting, and the embedder audit, pinned by the same census that guards the project's other closed vocabularies. An independent reviewer given the finished branch returned two blocking findings and eleven non-blocking ones, all applied; the [CHANGELOG](CHANGELOG.md) names each.
-Between this wave and the last feature wave, 1.49.0 through 1.50.3 made a label into a real boundary, taught the installer to see what the machine already has, and fixed a Hermes bridge that discarded the one line of stderr naming why it could not start - the [CHANGELOG](CHANGELOG.md) has the detail.
+Previous release, 1.51.0, was about which memories earn their keep: a deterministic salience gate in front of the dream pass's expensive fold, session mining that can only reach the speculative inbox, a provenance-following note delete under one snapshot and an exact count guard, expiration settable and changeable through one normalizer, vault pages staged as Agent Skill drafts that leave the vault only on explicit accept, an architect overview that names its grounding, and the shared needs-llm-step envelope type under all of it. Before it, 1.49.0 through 1.50.3 made a label into a real boundary, taught the installer to see what the machine already has, and fixed a Hermes bridge that discarded the one line of stderr naming why it could not start - the [CHANGELOG](CHANGELOG.md) has the detail.
Previous release, 1.48.0, was about work you can watch, bound and stop: the long passes emit progress and observe interrupts where an interrupt can physically be observed, a successful install states what it measured rather than what it hoped, the project scanner stopped entering what the repository ignores (24148 files visited became 2729), and five independent reviewers returned forty-four reproduced findings against the finished branch, all resolved by measurement rather than argument.
diff --git a/docs/architecture.md b/docs/architecture.md
index 1730293e..734d57bd 100644
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -198,7 +198,7 @@ Recommended behavior:
What "the vault" contains is enumerated rather than assumed.
`src/core/state/surfaces.ts` declares every durable location this build
-keeps INSIDE a vault — 40 of them — each with the layer that can move it
+keeps INSIDE a vault — 42 of them — each with the layer that can move it
and what its loss costs, split into `derived` (rebuildable, so deleting it
costs time) and `vault-content` (the memory itself, permanent without a
backup). `o2b state status` measures that catalogue against one vault and
@@ -383,7 +383,7 @@ is the point rather than its size. Ninety-nine classified tools carry 226 call
recipes - one per mode, view or operation, because one executed view is not an
executed classification. Every recipe runs TWICE against the same two-owner
fixture, once with the gate closed and once with it open, and the classification
-decides what the open run must show: the 31 recipes classified as reaching
+decides what the open run must show: the 32 recipes classified as reaching
owner-taggable content must surface the marker with the gate off, which is what
stops the closed run from passing vacuously. The other 194 carry a claim of
unreachability, and that claim is executed directly - the marker must be absent
diff --git a/docs/brainstorm/nothing-writes-silently/cli-output/claude.md b/docs/brainstorm/nothing-writes-silently/cli-output/claude.md
new file mode 100644
index 00000000..271368a4
--- /dev/null
+++ b/docs/brainstorm/nothing-writes-silently/cli-output/claude.md
@@ -0,0 +1,39 @@
+### Variant 1: Eight local truths
+
+- **Approach**: Each unit lands with its own local mechanism and no new shared abstractions: A's `countChunksWithoutEmbeddings` and G's record-vs-data audit are the only coupled pair (sequenced A→G on the shared `buildRecommendations`/`IndexCheckReport` surface). Unit H takes form B (enumeration census + doctor/search-check finding stating the boundary does not exist). Unit E's dead-letter knob resolves to in-response accounting only, with the durable dead-letter named and deferred in the design doc; F's read-back census and E's write accounting each define their own small report shape.
+- **Trade-offs**:
+ - Pro: smallest diff and review burden for an already-large 8-unit single PR; each unit is independently revertable inside the squash.
+ - Pro: no new STATE_SURFACES row, no indexRevision move — zero migration cost for existing vaults.
+ - Con: three near-identical "claimed/found/missing" shapes (E accounting, F census, G audit third-state) grow independently and will drift in vocabulary — the verdict-vocabulary census will eventually have to reconcile them anyway.
+ - Con: a failed lane write that exists only in a response the caller drops is itself a silent loss — in-response-only accounting undercuts the wave's own theme.
+ - Con: H form B plus deferred enforcement means the privacy gap survives the wave with only a warning attached.
+- **Complexity**: small
+- **Risk**: low
+
+### Variant 2: Shared reconciliation vocabulary
+
+- **Approach**: Introduce one small pure module — a reconciliation report type (`attempted/found/missing`, missing keys always named) — consumed by F's post-import read-back census, E's multi-artifact write accounting, and G's "record contradicts data" third state, so all three surfaces speak one vocabulary pinned by the existing verdict-vocabulary census. H takes form B, but its note-returning-surface enumeration is built as an extension of the same architecture-census machinery C uses for the write-site boundary assertion, so C and H share one census substrate. E's dead-letter knob resolves to a durable dead-letter under `.open-second-brain/` with its STATE_SURFACES row (a dropped response must not be the only record of a failed write). Sequencing: census/vocabulary substrate first, then A→G sequentially, then B/C/D/E/F against the settled substrate.
+- **Trade-offs**:
+ - Pro: one report shape means the wave's honesty guarantees are uniform and census-enforceable rather than three ad-hoc dialects.
+ - Pro: durable dead-letter is the only resolution of the knob consistent with "nothing writes silently"; the STATE_SURFACES convention already prices it.
+ - Pro: H form B ships something true and small now while its census artifact is exactly the coverage map form A needs later — no throwaway work.
+ - Con: the shared module is a new coordination point; B/E/F/G all block on its shape settling before parallel work can start (mitigates the known parallel-agent pin-collision problem, but costs wall-clock).
+ - Con: still defers real `visibility: private` enforcement; the doctor finding documents a hole rather than closing it.
+- **Complexity**: medium
+- **Risk**: medium
+
+### Variant 3: Boundary-enforcing wave
+
+- **Approach**: Unit H takes form A: `isRemotelyReadable` in graph/visibility.ts, transport-keyed (stdio vs HTTP) trusted-local bypass, wired at listVaultPages, pool-filters, and the indexer — and the entry-point derivation that C needs for `origin_channel` is generalized into one server-derived context object (channel + transport) threaded from the MCP server / CLI dispatcher, serving both C's stamp and H's bypass so no caller string ever carries privilege. Durable dead-letter for E rides the same STATE_SURFACES addition wave. Sequencing is strict: A→G on `buildRecommendations`, then H's indexer change last since it moves indexRevision and touches the same index surfaces A and G just instrumented.
+- **Trade-offs**:
+ - Pro: actually closes the private-content-in-chunks hole this wave's theme points at; one shared entry-point context is the architecturally clean home for both server-derived facts.
+ - Pro: no second wave needed for H; the census lands as enforcement coverage proof, not an apology.
+ - Con: indexRevision move forces a full reindex in the same release that ships new index-health recommendations (A, G) — every user's first post-upgrade `o2b search check` fires on a freshly invalidated index, muddying the very signals A and G introduce.
+ - Con: largest blast radius in a single PR/CHANGELOG version; the indexer chokepoint touches the query hot path the constraints say A must stay off.
+ - Con: three-chokepoint enforcement plus a covered/excluded census across every note-returning surface (brain_file_context, context_pack, query, backlinks, knowledge lane…) is a wave-sized unit by itself; bundled with seven others it dominates review and delays everything.
+- **Complexity**: large
+- **Risk**: high
+
+### Recommended: Variant 2
+
+**Rationale**: The wave's theme is honesty about writes and degradation, and Variant 2 is the only one that applies that theme to its own machinery: a single census-pinned reconciliation vocabulary keeps E, F, and G from drifting into three dialects of "missing", and a durable dead-letter is the only dead-letter answer where a failed write survives a dropped response. Variant 3's form-A enforcement is the right destination but its indexRevision move poisons A's and G's brand-new health signals inside the same mandatory single PR, while Variant 1 saves little over Variant 2 and leaves debts (report-shape drift, response-only failure records) that contradict the release's stated promise; H form B ships true statements now and produces exactly the coverage map a later form-A wave needs.
diff --git a/docs/brainstorm/nothing-writes-silently/cli-output/prompt.md b/docs/brainstorm/nothing-writes-silently/cli-output/prompt.md
new file mode 100644
index 00000000..9bb0baf1
--- /dev/null
+++ b/docs/brainstorm/nothing-writes-silently/cli-output/prompt.md
@@ -0,0 +1,77 @@
+You are brainstorming architectural variants for the following release wave. Do not write code. Do not write a final design. Only produce variants and a recommendation.
+
+# Task
+
+One release wave for Open Second Brain bundling eight units under the theme "nothing writes silently, nothing degrades silently". Every unit's scope below has already been corrected by a reconnaissance pass against the live source at main (v1.51.0); treat these corrected scopes as ground truth, not the original tracker cards.
+
+## Unit A (t_39b9b633, do first): measured pending-vector count in `o2b search check`
+The current recommendation branch in `buildRecommendations` (src/core/search/indexer.ts:1600-1647) fires on a provider-reachable + vec-loaded proxy with no term for existing vectors, so a fully-embedded healthy vault is told to "compute the first vectors" - a live false statement. Replace the proxy with a measured count: a new `countChunksWithoutEmbeddings(db)` (SELECT COUNT(*) anti-join beside the existing findChunksWithoutEmbeddings at store/chunks.ts:214) read through the existing `withReadonlyIndex` seam (store/embedding-abi.ts:65). Absent/unreadable index surfaces as "unrecorded", never zero. Fully-embedded vault gets no recommendation; pending > 0 names the count and points at the shipped `o2b search vector-backfill` dry-run verb for pricing. No auto-spend, off the query hot path.
+
+## Unit B (t_589f04fb): blank-overwrite refusal + unreadable-read raise in the note update path
+Two layers at one seam. (1) `readExistingNote` (write-batch.ts:519) currently discards `parseFrontmatter`'s unreadable-file notice, so an existing-but-unreadable note is treated as empty and update/append destroys frontmatter and body while reporting success - raise a typed `WriteBatchError("target_unreadable")` instead. (2) A pure shared guard `refuseBlankOverwrite({existingBody, nextBody, allowEmpty})` wired at `projectUpdateNote` only (covers brain_update_note and brain_write_batch's update_note op - the only two callers). New `allow_empty` boolean on those two schemas for deliberate clears. Creates of genuinely-new empty notes untouched. Guard placement in fs-atomic/host-memory-write/inline is refused (wrong layers; documented in the guard docblock). The ~40 other parseFrontmatter->rewrite sites are measured and deferred, named in the design doc.
+
+## Unit C (t_34f4e3ab): server-derived origin_channel stamp on structured Brain records
+Channel derived once per process at the entry point (MCP server / CLI dispatcher / import verbs), threaded as an option, NEVER a tool or CLI argument (25 MCP schemas already take `agent` verbatim - unverifiable; a caller-named channel would be a lie with a schema). Population bounded to four record families that already carry structured metadata: log events (appendLogEvent), signals (writeSignal - as a sibling of the existing source_type, with a documented mapping), continuity records (appendContinuityRecord), caller-named notes (the one resolveNoteTarget envelope). Backfill of existing records refused. The uncovered remainder (68 direct-fs census sites, fully-derived writers) asserted as a measured boundary in tests/core/architecture/write-site-census.test.ts.
+
+## Unit D (t_6285e06f): derived maintenance-debt block on brain status
+No stored counter (no transaction, no chokepoint; a counter would need a STATE_SURFACES row and 68-site instrumentation - refused). Derive from existing JSONL log shards after `last_dream_at` in `computeBrainStatus` (which already imports the shard readers). Field named for what it measures: `log_events_since_dream` - caller-named note writes emit no log event and the docblock says so (that sentence is the honesty deliverable). Wet-only clear is free (dry-run dream emits no dream log event - pin with a test). Ride on `second_brain_status` + `osb://status`; `brain_context` gets at most a cheap bounded overdue flag from the newest shard. `open_conflicts` descoped (needs a semantic-health run); `pending_triggers` only if the trigger store answers with a cheap count.
+
+## Unit E (t_5b27eb0f): link candidates on note-producing envelopes + write accounting
+No change to LlmStepFields/NEEDS_LLM_STEP_KEYS (the spine docblock forbids it; only 3 of 6 consumers produce wikilinked notes). `link_candidates` added to the three note-producing consumer types (rollup, diarization, design-note) the same way tier/produces already ride RollupEnvelope. Manifest built zero-embed from listVaultBasenames / note-title-resolver (fail-closed, lists candidates rather than guessing). Not smuggled into schema_hints (that channel is constraints, not data). Write accounting: lanes committing multiple artifacts from one envelope report attempted/written/failed + first real error, never bare success. Open design knob: durable dead-letter under .open-second-brain/ (needs a STATE_SURFACES row) vs in-response accounting only.
+
+## Unit F (t_2e58d10f): import read-back census + sessions-import resume
+Adapters (zep/letta/mnemosyne) stay dropped. Deliverable 1: post-import read-back census - after an import completes, re-read what landed and report claimed/found/missing with missing keys NAMED (a bare count is the misleading no-op this unit removes); shared helper over existing keys (dedup hashes for sessions, content-manifest hashes for ingest). Deliverable 2: extend the existing plan-scoped ingest checkpoint substrate (ingest/checkpoint.ts - atomic, vault-identity-asserted, schema-version-refusing) to the sessions import lane, which today has dedup-only re-do-and-discard; checkpoint keyed on (file, turn boundary). CHANGELOG must say resume exists for ingest and is being extended, not introduced.
+
+## Unit G (t_808c91b0, first cut line): embedder-identity record-vs-data audit + free restamp
+The attach-time raise mostly exists (resolveEmbeddingAbi throws under `fail`; `warn` default is deliberate - rebuild-loop argument at types.ts:2061 stands, do NOT flip it). Genuinely new: nothing compares the recorded index_state dimension against the ACTUAL stored embeddings width - a pure SQL audit (DISTINCT dimension vs meta vs vec0 declared width), reported as a third state "record contradicts data", landing in IndexCheckReport beside unit A (same buildRecommendations surface - sequence after A, do not parallelize). Plus a restamp verb for vec_version-only drift (a pure ABI token; no re-embed, no provider call), dry-run by default, CLI-only (operator repair; EMBEDDING_ABI_FIX_COMMAND is already a CLI string). Any re-embed verification is deferred or sits behind the planVectorBackfill apply/cost gate with three distinct outcomes (verified match / verified mismatch / cannot verify).
+
+## Unit H (t_92c26f69): per-page visibility:private - form to be decided
+The card is unbuildable as written (visibility: frontmatter dimension already exists in graph/visibility.ts as caller-liftable view scoping - any caller passes visibility:["private"] and reads the page; gatedOwnerScopeView is inert unless owner_scope_delivery:fail, gates only Brain artifacts, fails open on unresolvable refs). Real enforcement map: applyVisibilityScope has exactly one call site (search pool-filters); brain_file_context, context_pack, query, backlinks, unlinked_mentions, audit, sources, and the whole knowledge lane are unfiltered; the index itself stores private page content in chunks; listVaultPages has no frontmatter predicate. Two honest forms:
+(A) deny-by-default: one predicate isRemotelyReadable in graph/visibility.ts honoring a reserved `private` token independent of owner-scope config, trusted-local bypass keyed on transport (stdio vs HTTP), never on a caller string; wired at three chokepoints (listVaultPages predicate, pool-filters, the indexer so private content never enters chunks); a new architecture census test enumerating every note-returning surface as covered/excluded-with-reason; the "no closed enum" docblock amended in writing. Corpus/indexRevision moves - named cost.
+(B) honesty-only: ship the enumeration census + a doctor/search-check finding stating "visibility: private is a caller-supplied view filter, not a boundary; N surfaces do not honor it". Small, true, deferrable to (A) later.
+The wave doc must pick A or B. Refuse the card's own design (riding gatedOwnerScopeView) unconditionally.
+
+Also part of the wave (not a design question): tracker card t_5080cc53 (wikilink stub materialization) is closed as already-shipped (scaffold-stub landed end to end; the one unbuilt clause - auto-materialize on capture - is a silent write the shipped design refused in writing).
+
+# Project context
+
+Open Second Brain - TypeScript, Bun runtime. CLI (`o2b`) + MCP server over a file-based Obsidian-compatible vault with a SQLite (sqlite-vec) search index.
+Recent commits:
+6d3e8b23 feat: what earns its keep (v1.51.0) (#177)
+86544b9b fix(hermes): the bridge child inherits a PATH that can find Bun (v1.50.3) (#176)
+ba1a9678 fix(hermes): preserve context-pack recall (v1.50.2) (#172)
+4ba335d4 ci: adopt Bun 1.4.0 - pin the toolchain and rebuild the bundle it gates (#175)
+1e8792cb fix: a provider the host cannot read is not an unconfigured one (v1.50.1) (#171)
+
+Related files: src/core/search/{indexer,vector-backfill}.ts, src/core/search/store/{chunks,embedding-abi,vectors}.ts, src/core/brain/{write-batch,log,signal,status,llm-step}.ts, src/core/brain/ingest/checkpoint.ts, src/core/brain/sessions/import.ts, src/core/graph/visibility.ts, src/core/write-binding/index.ts, tests/core/architecture/write-site-census.test.ts.
+
+Conventions:
+- Fail loud: refusals name their reason; partial writes report how far they got; no silent fallbacks, no stubs.
+- OSB never calls an LLM; model work goes through needs-llm-step envelopes; the calling agent owns generation.
+- Anything contacting an embedding provider sits behind an explicit apply gate, dry-run by default.
+- Architecture censuses are standing tests (write-site, state-surface, verdict-vocabulary); new durable state needs a STATE_SURFACES row; MCP tools need previewBudget + closed schemas + parity pins; CLI verbs need dispatch + help text + command-manifest rows.
+- English-only strings; no natural-language hardcoding for other languages.
+
+Constraints:
+- No new caller-supplied field may carry provenance or privilege (server-derived only).
+- Derived quantities are named for what they actually measure.
+- Units A and G share buildRecommendations - they must land sequentially.
+- One PR, one CHANGELOG version for the whole wave.
+- No new external dependencies. Do not change existing public API semantics except where a unit explicitly says so.
+
+# Required output format
+
+Produce exactly 3 distinct architectural variants. For each variant:
+
+### Variant N: <short name>
+- **Approach**: 2-3 sentences describing the variant (how the eight units compose: shared mechanisms, unit H form, dead-letter knob, sequencing).
+- **Trade-offs**: bullet list of pros and cons.
+- **Complexity**: small | medium | large
+- **Risk**: low | medium | high
+
+After the three variants, add exactly one recommendation:
+
+### Recommended: Variant N
+**Rationale**: 2-3 sentences explaining why this variant over the others, considering the project context and constraints above.
+
+Output nothing outside of these sections.
diff --git a/docs/brainstorm/nothing-writes-silently/design.md b/docs/brainstorm/nothing-writes-silently/design.md
new file mode 100644
index 00000000..dca00591
--- /dev/null
+++ b/docs/brainstorm/nothing-writes-silently/design.md
@@ -0,0 +1,63 @@
+# Nothing writes silently - integrity and degradation honesty wave
+
+**Status:** accepted
+**Author:** claude-dev-agent (via feature-release-playbook)
+**Audience:** implementation
+
+## Problem statement
+
+Eight tracker cards converge on one defect class: Open Second Brain can lose or degrade data without saying so. A note update can blank a page it failed to read; a fully-embedded vault is told to "compute the first vectors"; an interrupted sessions import re-does everything and reports nothing; a failed envelope write can exist only in a response the caller dropped; a `visibility: private` flag reads as privacy and is not. Reconnaissance against main (v1.51.0) corrected or refuted every card's premise before this design; the corrected scopes in `docs/brainstorm/nothing-writes-silently/cli-output/prompt.md` and the recon report are ground truth, not the cards.
+
+## Scope
+
+Eight units, one PR, one CHANGELOG version (chosen approach: consultant Variant 2, "shared reconciliation vocabulary"):
+
+- **Unit A** (`t_39b9b633`): measured pending-vector count replacing the false proxy branch in `buildRecommendations`; `countChunksWithoutEmbeddings(db)` COUNT(*) anti-join read through `withReadonlyIndex`; absent index surfaces as unrecorded, never zero; a fully-embedded vault gets no recommendation.
+- **Unit B** (`t_589f04fb`): `readExistingNote` raises typed `WriteBatchError("target_unreadable")` instead of treating an unreadable note as empty; pure `refuseBlankOverwrite` guard at the single `projectUpdateNote` seam; `allow_empty` boolean on `brain_update_note` and `brain_write_batch`'s `update_note` op.
+- **Unit C** (`t_34f4e3ab`): server-derived `origin_channel` (resolved once per process at the entry point, threaded as an option, never an argument) stamped on four record families: log events, signals (sibling of `source_type`), continuity records, caller-named notes. Uncovered remainder asserted in the write-site census.
+- **Unit D** (`t_6285e06f`): derived `maintenance_debt` block in `computeBrainStatus` from existing JSONL log shards after `last_dream_at`; field named `log_events_since_dream` for what it measures; rides `second_brain_status` + `osb://status`; `brain_context` gets only a cheap newest-shard overdue flag.
+- **Unit E** (`t_5b27eb0f`): `link_candidates` on the three note-producing envelope consumer types (rollup, diarization, design-note) built zero-embed from the basename walker / note-title resolver; write accounting in the shared reconciliation vocabulary; durable dead-letter under `<vault>/.open-second-brain/` with its STATE_SURFACES row.
+- **Unit F** (`t_2e58d10f`): post-import read-back census (claimed/found/missing with missing keys named) over existing dedup/content hashes, in the shared reconciliation vocabulary; sessions-import resume extending the existing `ingest/checkpoint.ts` substrate.
+- **Unit G** (`t_808c91b0`): record-vs-data embedder audit (recorded `index_state` dimension vs actual stored width vs vec0 declared width) as a third state "record contradicts data" in `IndexCheckReport`; `vec_version`-only restamp CLI verb, dry-run by default, no provider contact.
+- **Unit H** (`t_92c26f69`, form B): a standing architecture census enumerating every note-returning surface as visibility-covered or excluded-with-reason, plus a `search check` / doctor finding stating that `visibility: private` is a caller-supplied view filter, not a boundary, naming the uncovered surface count.
+- **Shared substrate** (new, first): one pure reconciliation report module (`attempted/found/missing`-shaped, missing keys always named) consumed by units E, F, and G's third state, pinned by the verdict-vocabulary census.
+
+## Out of scope
+
+- `t_5080cc53` (wikilink stub materialization): REFUSED - already shipped end to end as `scaffold-stub` (core + `brain_scaffold_stub` + CLI verb + repair-lane + tests). The one unbuilt clause (auto-materialize on capture) is a silent write the shipped design declined in writing (`repair-lane.ts:110-115`).
+- Import adapters (zep/letta/mnemosyne) - external formats with no fixtures.
+- A stored writes counter (no transaction, no chokepoint; refused - derive only).
+- Any change to `LlmStepFields` / `NEEDS_LLM_STEP_KEYS` (the spine docblock forbids it; only 3 of 6 consumers produce wikilinked notes).
+- Flipping `integrity.embedding_abi` default to `fail` (the rebuild-loop argument at `types.ts:2061` stands).
+- Re-embed verification for model/dimension drift (deferred; if ever shipped it sits behind the `planVectorBackfill` apply/cost gate).
+- Unit H form A (deny-by-default enforcement at walker/search/indexer chokepoints): deferred to a dedicated wave. The indexRevision move would invalidate every index in the same release that ships new index-health signals, and the three-chokepoint change is wave-sized by itself. The Unit H census is exactly the coverage map form A needs.
+- The ~40 other `parseFrontmatter` -> rewrite sites (labels, tombstone, backfills, ...): measured and deferred; unit B fixes the write-batch seam only.
+- Backfilling `origin_channel` onto existing records: rewriting history to add provenance is the opposite of provenance.
+
+## Chosen approach
+
+Consultant Variant 2 with two orchestrator amendments:
+
+1. **Unit H card disposition**: form B ships the truth, not the card's ask. The tracker card is not closed as done at release; it receives the redesign verdict (form A spec, census as coverage map) and stays parked for a dedicated enforcement wave - mirroring the t_916f9953 refusal handling.
+2. **Dead-letter boundary**: the durable dead-letter covers only lanes that commit multiple artifacts from one validated envelope payload (rollup fold, extract-signals batch); single-artifact lanes keep in-response accounting (their one failure IS the response). This bounds the STATE_SURFACES row to where a dropped response could actually hide a partial write.
+
+## Design decisions
+
+- **Server-derived beats caller-asserted, uniformly.** No new caller-supplied field may carry provenance or privilege (25 MCP schemas already take `agent` verbatim - the census recount this wave found 25, up from the documented 22). Unit C derives the channel at the process entry point; unit H's future bypass would key on transport, never a string. Cited by both units.
+- **Derived quantities are named for what they measure.** `log_events_since_dream`, not `writes_since_dream` - caller-named note writes emit no log event, and the docblock says so. A false label on a true number is the dishonesty this wave removes.
+- **One reconciliation vocabulary, not three dialects.** E's accounting, F's census, and G's third state share one pure report shape, pinned by the verdict-vocabulary census, so "missing" cannot drift into three spellings.
+- **Units A and G land sequentially** - both edit `buildRecommendations` / `IndexCheckReport`.
+- **Unit B refuses the wrong layers explicitly**: no guard in `fs-atomic` (writes bytes, not notes; 212 call sites), `host-memory-write` (append-only, already rejects empty), or `inline*` (parser / line-preserving insert). The guard docblock records why.
+- **Wet-only debt clear is free**: dry-run dream emits no `dream` log event, so `last_dream_at` cannot move on a preview - pinned by a test rather than implemented as a mechanism.
+
+## File changes
+
+New: `src/core/reconciliation-report.ts` (core root - its consumers span the search layer (G) and the brain layer (E, F)), `src/core/brain/notes/blank-overwrite-guard.ts`, `src/core/search/store/` count query beside `findChunksWithoutEmbeddings`, sessions checkpoint extension, dead-letter store module, restamp CLI verb file, `tests/core/architecture/visibility-surface-census.test.ts`.
+Modified: `write-batch.ts`, `log.ts`, `signal.ts`, `continuity/store.ts`, `notes/create-note.ts` (envelope option only), `status.ts`, `indexer.ts` (`buildRecommendations`, `IndexCheckReport`), `serialize.ts`, `search/verbs/check.ts` output, `rollup-ladder.ts` / `diarization.ts` / `design-note.ts` (consumer envelope types), `sessions/import.ts`, `mcp` schemas for `allow_empty`, `command-manifest.ts` + help text (restamp verb), `state/surfaces.ts` (+1 or +2 rows), write-site census (boundary assertion), verdict-vocabulary census (reconciliation vocabulary), docs.
+
+## Risks and open questions
+
+- The stale-recommendation pin tests that assert today's (wrong) "compute the first vectors" behavior will move - that movement is the fix, not a regression; expect and rename them.
+- Signals gain a frontmatter key (`origin_channel`) - fixtures, `schema_inspect` vocabulary, and the frontmatter-key allowlist all move together or the wave fails its own parity gates.
+- The sessions checkpoint may fold under the existing ingest-checkpoints STATE_SURFACES row or need its own - verify the existing row's key-shape coverage rather than assuming (recon flagged this explicitly).
+- Unit H's census must enumerate the knowledge lane honestly; a census that quietly omits a surface reproduces the defect it documents.
diff --git a/docs/brainstorm/nothing-writes-silently/plan.md b/docs/brainstorm/nothing-writes-silently/plan.md
new file mode 100644
index 00000000..66f7cca3
--- /dev/null
+++ b/docs/brainstorm/nothing-writes-silently/plan.md
@@ -0,0 +1,56 @@
+# Nothing writes silently - implementation plan
+
+Ten tasks. T1 is the substrate; T2->T3 is the one strict sequence (shared `buildRecommendations` surface); T4, T5, T6, T9 are independent; T7 and T8 depend on T1. TDD per task: failing test first, then implementation, then refactor green.
+
+## Tasks
+
+### Task 1: Shared reconciliation report module
+- **Files**: new `src/core/reconciliation-report.ts`; `tests/core/reconciliation-report.test.ts`; `tests/core/architecture/verdict-vocabulary-census.test.ts` (vocabulary pin).
+- **Shape**: pure module, no I/O. A report is `{ attempted, found, missing }` where `missing` always names its keys (a bare count constructor does not exist); a builder that refuses `attempted < found` and NaN inputs by name. Closed outcome vocabulary shared by consumers (e.g. `complete | partial | contradicted`).
+- **Acceptance**: unit tests incl. refusal cases; vocabulary census pins the outcome tokens.
+- **Depends on**: none. Blocks T7, T8, and T3's third-state naming.
+
+### Task 2: Unit A - measured pending-vector count (`t_39b9b633`)
+- **Files**: `src/core/search/store/chunks.ts` (new `countChunksWithoutEmbeddings`), `src/core/search/indexer.ts` (`IndexCheckReport` fields + `buildRecommendations` branch), `src/core/search/serialize.ts`, `src/cli/search/verbs/check.ts` output; tests: store unit test, check-report tests, moved recommendation pins.
+- **Acceptance**: fully-embedded vault emits NO reindex recommendation (the old always-fires pin moves and is renamed as the fix); pending > 0 names the count and points at `o2b search vector-backfill`; absent/unreadable index reports unrecorded, never zero. No call from the query hot path.
+- **Depends on**: none. Blocks T3.
+
+### Task 3: Unit G - record-vs-data audit + restamp verb (`t_808c91b0`)
+- **Files**: `src/core/search/store/` audit query (DISTINCT stored dimension vs `index_state` vs vec0 declared width), `indexer.ts` (third state in `IndexCheckReport`, after T2's shape settles), new `src/cli/search/verbs/` restamp verb (vec_version-only, dry-run default, no provider contact), `command-manifest.ts`, search help text; tests: audit unit tests, verb tests, check-output pins.
+- **Acceptance**: a meta row claiming dimension N over stored width M reports "record contradicts data" (T1 vocabulary), distinct from drifted and unrecorded; restamp dry-run prints the would-be change and writes nothing; `--apply` restamps `vec_version` only and refuses when model/dimension differ (that repair is deferred by design). CLI-only - MCP tool count stays 113.
+- **Depends on**: T2 (same `buildRecommendations` surface), T1 (vocabulary).
+
+### Task 4: Unit B - unreadable-read raise + blank-overwrite guard (`t_589f04fb`)
+- **Files**: `src/core/brain/write-batch.ts` (`readExistingNote` -> `parseFrontmatterWithNotices`, new codes `target_unreadable`, `blank_overwrite_refused`), new `src/core/brain/notes/blank-overwrite-guard.ts`, `src/mcp/brain/notes-tools.ts` + `src/mcp/brain/write-batch-tools.ts` (`allow_empty`); tests: write-batch unit tests (unreadable note -> typed refusal, no write; blank body over non-empty -> refusal naming the note; `allow_empty: true` clears; new empty note untouched), MCP schema/parity pins.
+- **Acceptance**: an existing-but-unreadable note can no longer be blanked by update or append; refusals name their reason; the guard docblock records the refused placements (fs-atomic, host-memory-write, inline).
+- **Depends on**: none.
+
+### Task 5: Unit C - server-derived origin_channel (`t_34f4e3ab`)
+- **Files**: new resolver module (entry-point-set process constant, `mcp-tool | cli | import` + explicit unset refusal), threading through `appendLogEvent` (`log.ts`), `writeSignal` (`signal.ts`, sibling of `source_type` + mapping table in docblock), `appendContinuityRecord` (`continuity/store.ts`), the `resolveNoteTarget` envelope (`notes/create-note.ts`); `tests/core/architecture/write-site-census.test.ts` boundary assertion; signal fixtures, frontmatter-key allowlist, `schema_inspect` vocabulary.
+- **Acceptance**: all four families stamp the channel with zero new schema fields (grep proves no tool/CLI argument named channel); the census asserts the measured uncovered remainder; existing records are not rewritten.
+- **Depends on**: none.
+
+### Task 6: Unit D - derived maintenance-debt block (`t_6285e06f`)
+- **Files**: `src/core/brain/status.ts` (`maintenance_debt` block: `log_events_since_dream` + `pending_triggers` only if answerable by a cheap count), status serialization surfaces (`second_brain_status`, `osb://status`), bounded newest-shard overdue flag on `brain_context`; tests: status tests, the dry-run-dream-moves-nothing pin, `brain_context` payload test.
+- **Acceptance**: field names match what is measured; the docblock states caller-named note writes are not counted and why; a `--dry-run` dream does not move the ledger; `brain_context` cost stays bounded (newest shard only).
+- **Depends on**: none.
+
+### Task 7: Unit E - link candidates + write accounting + dead letter (`t_5b27eb0f`)
+- **Files**: `rollup-ladder.ts`, `diarization.ts`, `design-note.ts` (consumer-type `link_candidates` via the zero-embed basename walker / note-title resolver; NOT on `LlmStepFields`, NOT in `schema_hints`), write accounting in T1 vocabulary for multi-artifact commit lanes, new durable dead-letter module under `<vault>/.open-second-brain/` + `state/surfaces.ts` row + `state-surface-census`; tests: envelope shape tests per consumer, accounting tests (all-writes-failed reports attempted/written/failed + first real error, never success), dead-letter round-trip.
+- **Acceptance**: `NEEDS_LLM_STEP_KEYS` byte-identical (pin); three note-producing envelopes carry candidates; non-note lanes untouched; a multi-artifact lane interrupted mid-commit leaves a dead-letter record naming the unwritten items.
+- **Depends on**: T1.
+
+### Task 8: Unit F - import read-back census + sessions resume (`t_2e58d10f`)
+- **Files**: shared read-back helper over existing keys (sessions dedup hashes, ingest content-manifest hashes) reporting in T1 vocabulary; `sessions/import.ts` checkpoint extension reusing `ingest/checkpoint.ts` substrate (keyed on file + turn boundary); STATE_SURFACES: verify whether the existing ingest-checkpoints row covers the new key shape, else add a row; tests: census tests (missing keys named), interrupted-import resume test (resumes at turn boundary, does not re-hash completed turns), checkpoint schema-version refusal.
+- **Acceptance**: an import that lost items reports claimed/found/missing with keys named; an interrupted sessions import resumes instead of re-doing; CHANGELOG wording says ingest resume is extended, not introduced.
+- **Depends on**: T1.
+
+### Task 9: Unit H form B - visibility surface census + honesty finding (`t_92c26f69`)
+- **Files**: new `tests/core/architecture/visibility-surface-census.test.ts` (every note-returning surface: covered by `applyVisibilityScope` or excluded-with-written-reason - search lane, file_context, context_pack, query, backlinks, unlinked_mentions, audit, sources, knowledge lane, resources); `search check` / doctor finding module stating the filter is caller-supplied view scoping, not a boundary, with the measured uncovered count.
+- **Acceptance**: census enumerates and fails on an unclassified new surface; the finding appears in check output with the count derived from the census's own numbers (no hand-written N); no enforcement change, no owner-scope path touched, no indexRevision move.
+- **Depends on**: none (lands after T2/T3 settle check output, coordinate strings).
+
+### Task 10: Docs, version, release prep
+- **Files**: `CHANGELOG.md` (new version heading + link ref; states the Unit A fix as a fixed false statement, F's resume as an extension, H as honesty-not-enforcement, the two refusals), `README.md` "What is new", `docs/architecture.md` / `docs/mcp.md` counts if moved, `package.json` bump + `bun run scripts/sync-version.ts`.
+- **Acceptance**: `sync-version.ts --check` green; docs count tests green; full suite, typecheck, lint green on clean HOME.
+- **Depends on**: all.
diff --git a/docs/brainstorm/nothing-writes-silently/variants.md b/docs/brainstorm/nothing-writes-silently/variants.md
new file mode 100644
index 00000000..f36c7651
--- /dev/null
+++ b/docs/brainstorm/nothing-writes-silently/variants.md
@@ -0,0 +1,24 @@
+# Nothing writes silently - variants audit trail
+
+Consultant: Claude Code (`claude -p`), prompt at `cli-output/prompt.md`, raw output at `cli-output/claude.md`. Three variants returned; the consultant recommended Variant 2.
+
+## Variant 1: Eight local truths (consultant)
+
+Each unit lands its own local mechanism, no shared abstractions; unit H form B; dead-letter resolves to in-response accounting only. Complexity small, risk low. Consultant's own cons: three near-identical claimed/found/missing shapes drift into dialects; a failed lane write that exists only in a dropped response is itself a silent loss - undercutting the wave's theme.
+
+## Variant 2: Shared reconciliation vocabulary (consultant, recommended)
+
+One pure reconciliation report module consumed by E's write accounting, F's read-back census, and G's third state, pinned by the verdict-vocabulary census; H form B with its enumeration built on the same census machinery C's boundary assertion uses; durable dead-letter with its STATE_SURFACES row; sequencing substrate-first, then A->G sequential, the rest parallel. Complexity medium, risk medium. Cons: the shared module is a coordination point that costs wall-clock before parallel work starts; H form B defers real enforcement.
+
+## Variant 3: Boundary-enforcing wave (consultant)
+
+H takes form A (isRemotelyReadable at walker/search/indexer chokepoints, transport-keyed bypass), generalized entry-point context serving both C's stamp and H's bypass; strict sequencing with the indexer change last. Complexity large, risk high. Cons the consultant named: the indexRevision move forces a full reindex in the same release that ships the new index-health signals, muddying them; the three-chokepoint change plus full surface census is wave-sized by itself and dominates review.
+
+## Decision
+
+**Variant 2 accepted**, with two orchestrator amendments:
+
+1. **Unit H card disposition.** Form B ships the truth (census + finding), not the card's ask (enforcement). The tracker card therefore does not close as done at release: it gets the redesign verdict recorded (form A spec; the census as its coverage map) and stays parked for a dedicated enforcement wave. Closing it on a doctor warning would be the misleading no-op this wave exists to remove.
+2. **Dead-letter boundary.** Durable dead-letter only for lanes committing multiple artifacts from one validated payload (rollup fold, extract-signals batch); single-artifact lanes keep in-response accounting, because their single failure IS the response. This bounds the new STATE_SURFACES row to where a dropped response could actually hide a partial write, keeping Variant 2's honesty without Variant 3's blast radius.
+
+Rationale for agreeing with the consultant: the wave's theme applied to its own machinery - one census-pinned vocabulary instead of three drifting dialects, and a durable record for exactly the failures a response cannot be trusted to carry. Variant 3's destination (real per-page enforcement) is right but belongs to its own wave; its cost lands on the same index surfaces units A and G are instrumenting, and the census Variant 2 ships is the coverage map that future wave starts from.
diff --git a/docs/cli-reference.md b/docs/cli-reference.md
index 78322242..f37e9fb9 100644
--- a/docs/cli-reference.md
+++ b/docs/cli-reference.md
@@ -1421,6 +1421,34 @@ o2b search check Pre-flight diagnostics: vault, index directory, SQ
not-configured, reachable, unreachable, timed-out, skipped. The boolean
had to answer four questions with two values, so a provider that refused
and one that never answered were the same `false`.
+
+ Two censuses ride the report in EVERY state, because a check that could
+ not measure something must not answer with silence. `pending_vectors` is
+ the measured count of chunks with no vector - `{verdict: measured,
+ pending, chunks}`, or `{verdict: unrecorded, reason}` when the index is
+ absent or will not open, which is never reported as a count of zero. It
+ is what the reindex recommendation now gates on: a fully embedded vault
+ is told nothing, an index with no vectors at all is told to compute its
+ first ones, and a partially embedded index is pointed at
+ `o2b search vector-backfill`, whose dry run prices the work.
+ `embedder_record` is the record-vs-data audit: the dimension
+ `index_state` claims against the widths the `embeddings` rows carry and
+ the width `chunk_vec` declares. Its outcome is `complete` or
+ `contradicted` - a record the data itself disproves, which is a
+ different finding from ABI drift (the record disagrees with this build)
+ and from an unrecorded token (no claim was ever made).
+o2b search restamp Record this build's sqlite-vec version as the one the stored vectors are
+ accepted under - the repair for a drift confined to
+ embedding_vec_version, which is an ABI marker rather than a property of
+ any vector (two peers on different sqlite-vec builds each read the
+ other's index as drifted). Dry-run by DEFAULT: it prints the change it
+ would make and writes nothing. --apply writes that one index_state cell
+ and nothing else. It REFUSES by name when the recorded model or
+ dimension disagrees with this build - those describe the vectors
+ themselves, and repairing them needs re-embedding under a verified
+ identity, which is deferred by design; the refusal names
+ `o2b search reindex --embeddings` instead. No path contacts a provider.
+ --json emits dry_run, field, recorded, runtime, changed, applied
o2b search provider add NAME Register an OpenAI-compatible embedding endpoint (since v0.36.0)
--base-url U --model M --env-key K (K is the env var NAME holding the key);
persisted to Brain/search/embedding-providers.json, resolved after built-ins
diff --git a/docs/mcp.md b/docs/mcp.md
index 96fc2de3..28dec739 100644
--- a/docs/mcp.md
+++ b/docs/mcp.md
@@ -1232,3 +1232,26 @@ log line is machine-composed rather than authored.
outputs. Zero and two-plus recommendations are both refused and the
refusal states the count found; a note for the same topic on the same day
is refused rather than overwriting the first.
+- Since v1.52.0 the three needs-llm-step lanes whose artifact is a wikilinked note -
+ `brain_diarize`, `brain_design_note`, and the dream pass's rollup rungs -
+ additively carry `link_candidates` on their envelope: the note basenames a
+ `[[wikilink]]` in the answer can actually resolve to, with the `total` the
+ vault holds and a `truncated` flag when the bound could not carry them
+ all. It is built from the basename walker alone - no embedding, no vector
+ store, no index - and a truncated manifest says so in its own schema hint
+ rather than reading as the whole vault. The envelope spine is unchanged
+ and the three lanes that produce no wikilinked note (`brain_extract_signals`,
+ `brain_skill_proposals` page drafts, `brain_write_session`) carry nothing new.
+- Since v1.52.0 `brain_extract_signals` is the one lane that commits SEVERAL artifacts from
+ one validated payload - a file per mined item - so its result additively
+ carries `reconciliation` (`attempted` / `found` / `missing`, the missing
+ keys named, in the wave's shared vocabulary; a deduped or gate-rejected
+ item is a named refusal, not an attempted write). A commit that fails part
+ way still refuses with the same accounting AND the first real error, never
+ a partial success, and now also leaves a durable record of the partial
+ write under `<vault>/.open-second-brain/dead-letters/` - the new
+ `write_dead_letters` state surface - so a caller that drops the response is
+ not the only record that it happened. Nothing in this tool removes a dead
+ letter; `o2b state status` names the location and the records are listable
+ from core. Single-artifact lanes are deliberately excluded: their one
+ failure IS the response. No new tool - the surface stays at 113.
diff --git a/openclaw.plugin.json b/openclaw.plugin.json
index f2703c72..d8c72027 100644
--- a/openclaw.plugin.json
+++ b/openclaw.plugin.json
@@ -2,7 +2,7 @@
"id": "open-second-brain",
"name": "Open Second Brain",
"description": "Second brain for AI agents using Obsidian-compatible Markdown vaults.",
- "version": "1.51.0",
+ "version": "1.52.0",
"activation": { "onStartup": true },
"skills": ["./skills"],
"contracts": {
diff --git a/package.json b/package.json
index dcff22d7..9fb65f73 100644
--- a/package.json
+++ b/package.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "1.51.0",
+ "version": "1.52.0",
"private": false,
"description": "Second brain for AI agents using Obsidian-compatible Markdown vaults. Works with Hermes, Claude Code, Codex, and OpenClaw.",
"keywords": [
diff --git a/plugin.yaml b/plugin.yaml
index 2566e86f..ddc17f8a 100644
--- a/plugin.yaml
+++ b/plugin.yaml
@@ -1,5 +1,5 @@
name: open-second-brain
-version: "1.51.0"
+version: "1.52.0"
description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."
author: "Open Second Brain contributors"
memory_provider: true
diff --git a/plugins/codex/.codex-plugin/plugin.json b/plugins/codex/.codex-plugin/plugin.json
index a44a7ad9..eee4a201 100644
--- a/plugins/codex/.codex-plugin/plugin.json
+++ b/plugins/codex/.codex-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "1.51.0",
+ "version": "1.52.0",
"description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.",
"author": {
"name": "Open Second Brain contributors",
diff --git a/plugins/hermes/plugin.yaml b/plugins/hermes/plugin.yaml
index 2566e86f..ddc17f8a 100644
--- a/plugins/hermes/plugin.yaml
+++ b/plugins/hermes/plugin.yaml
@@ -1,5 +1,5 @@
name: open-second-brain
-version: "1.51.0"
+version: "1.52.0"
description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."
author: "Open Second Brain contributors"
memory_provider: true
diff --git a/pyproject.toml b/pyproject.toml
index 185cd1f3..e5a2e2c1 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -10,7 +10,7 @@ build-backend = "setuptools.build_meta"
# CLI entry points (those moved to `package.json` `bin`).
[project]
name = "open-second-brain"
-version = "1.51.0"
+version = "1.52.0"
description = "Hermes Python shim for Open Second Brain. Most of the project (CLI, MCP server, OpenClaw plugin) is TypeScript on Bun; see package.json."
readme = "README.md"
requires-python = ">=3.11"
diff --git a/src/cli/brain.ts b/src/cli/brain.ts
index 4d0f052e..11623a62 100644
--- a/src/cli/brain.ts
+++ b/src/cli/brain.ts
@@ -156,6 +156,42 @@ import {
cmdBrainWeekly,
} from "./brain/verbs/index.ts";
+/**
+ * Brain verbs whose whole purpose is replaying records authored
+ * somewhere else, spelled as the verb path the caller types. The CLI
+ * entry point reads this to claim the `import` origin channel for the
+ * process (`src/core/origin-channel.ts`) instead of the `cli` every
+ * other verb gets.
+ *
+ * It lives here, beside the `switch` that dispatches these verbs, so a
+ * renamed verb moves both in one edit; a copy of the list in `main.ts`
+ * would be a copy that goes stale the first time one is renamed.
+ *
+ * Membership is "replays records this operator did not author now",
+ * which is why `scan-inline` is absent: it captures `@osb` markers the
+ * operator wrote in their own vault during this session, and that is a
+ * `cli` capture wearing an unusual input format.
+ */
+export const BRAIN_IMPORT_VERB_PATHS: ReadonlyArray<string> = Object.freeze([
+ "bank-import",
+ "git ingest",
+ "graph-import",
+ "import-claude-memory",
+ "import-session",
+]);
+
+/**
+ * Whether `o2b brain <argv…>` names one of {@link BRAIN_IMPORT_VERB_PATHS}.
+ * Matches on the first two words so a nested verb (`git ingest`) is
+ * distinguishable from its read-only siblings (`git status`, `git find`).
+ */
+export function isBrainImportVerb(argv: ReadonlyArray<string>): boolean {
+ if (argv.length === 0) return false;
+ const one = argv[0]!;
+ const two = argv.length > 1 ? `${one} ${argv[1]!}` : one;
+ return BRAIN_IMPORT_VERB_PATHS.includes(one) || BRAIN_IMPORT_VERB_PATHS.includes(two);
+}
+
export async function handleBrainSubcommand(argv: ReadonlyArray<string>): Promise<number> {
if (argv.length === 0 || argv[0] === "-h" || argv[0] === "--help") {
process.stdout.write(BRAIN_HELP);
diff --git a/src/cli/brain/verbs/batch-plan.ts b/src/cli/brain/verbs/batch-plan.ts
index 41995e76..b844b7f8 100644
--- a/src/cli/brain/verbs/batch-plan.ts
+++ b/src/cli/brain/verbs/batch-plan.ts
@@ -11,6 +11,7 @@
*/
import { planBatches } from "../../../core/brain/ingest/batch-plan.ts";
+import { serializeImportCensus } from "../../../core/brain/import-census.ts";
import { reconcilePlan } from "../../../core/brain/ingest/reconcile.ts";
import { formatIgnoreWarning } from "../../../core/fs/git-discovery.ts";
import { serializeBatchPlan } from "../../../mcp/brain/ingest-tools.ts";
@@ -80,6 +81,7 @@ export async function cmdBrainBatchPlan(argv: string[]): Promise<number> {
ingested: [...report.ingested],
missing: [...report.missing],
complete: report.complete,
+ census: serializeImportCensus(report.census),
},
}
: {}),
@@ -117,6 +119,16 @@ export async function cmdBrainBatchPlan(argv: string[]): Promise<number> {
info(` reconcile: ${report.missing.length} dispatched source(s) never ingested:`);
for (const p of report.missing) info(` - ${p}`);
}
+ // The read-back half: of the sources the checkpoint CLAIMS, which the
+ // content manifest can still confirm. Named, never a bare count.
+ const census = report.census;
+ if (census.attempted > 0) {
+ ok(
+ ` census: ${census.found}/${census.attempted} ingested source(s) read back ` +
+ `(${census.outcome})`,
+ );
+ for (const p of census.missing) info(` unconfirmed by the manifest: ${p}`);
+ }
}
return 0;
} catch (err) {
diff --git a/src/cli/brain/verbs/design-note.ts b/src/cli/brain/verbs/design-note.ts
index 0c31dfe2..3d4046ab 100644
--- a/src/cli/brain/verbs/design-note.ts
+++ b/src/cli/brain/verbs/design-note.ts
@@ -20,6 +20,7 @@ import {
DesignNoteError,
planDesignNote,
} from "../../../core/brain/design-note.ts";
+import { gatedOwnerScopeView } from "../../../core/brain/owner-scope-view.ts";
import { ResponseCheckError } from "../../../core/brain/response-checks.ts";
import { ResponseShapeError } from "../../../core/brain/response-shape.ts";
import {
@@ -65,7 +66,10 @@ export async function cmdBrainDesignNote(argv: string[]): Promise<number> {
: null;
if (rawPayload === null) {
- const report = planDesignNote(vault, topic, { now: new Date() });
+ const report = planDesignNote(vault, topic, {
+ now: new Date(),
+ ownerScope: gatedOwnerScopeView(vault, resolveBrainAgent(flags, config)).scope,
+ });
const g = report.grounding;
if (asJson) {
okJson({
diff --git a/src/cli/brain/verbs/diarize.ts b/src/cli/brain/verbs/diarize.ts
index e0b2149d..07ccc674 100644
--- a/src/cli/brain/verbs/diarize.ts
+++ b/src/cli/brain/verbs/diarize.ts
@@ -7,6 +7,7 @@
*/
import { diarize, DiarizationError } from "../../../core/brain/diarization.ts";
+import { gatedOwnerScopeView } from "../../../core/brain/owner-scope-view.ts";
import {
brainVerbContext,
fail,
@@ -14,6 +15,7 @@ import {
okJson,
normalizeFlagString,
parse,
+ resolveBrainAgent,
usageError,
} from "../helpers.ts";
@@ -27,13 +29,18 @@ export async function cmdBrainDiarize(argv: string[]): Promise<number> {
if (!query || query.trim() === "") {
return usageError("usage: o2b brain diarize <entity> [--category C] [--vault <path>] [--json]");
}
- const { vault } = brainVerbContext(flags);
+ const { config, vault } = brainVerbContext(flags);
const category = normalizeFlagString(flags["category"]);
try {
const report = diarize(
vault,
{ query, ...(category !== null ? { category } : {}) },
- { now: new Date() },
+ // Same gated view the MCP tool uses: the candidate list names Brain
+ // artifact ids, and this verb reads the same tree.
+ {
+ now: new Date(),
+ ownerScope: gatedOwnerScopeView(vault, resolveBrainAgent(flags, config)).scope,
+ },
);
if (flags["json"] === true) {
okJson({
diff --git a/src/cli/brain/verbs/import-session.ts b/src/cli/brain/verbs/import-session.ts
index c423109e..e4a2e282 100644
--- a/src/cli/brain/verbs/import-session.ts
+++ b/src/cli/brain/verbs/import-session.ts
@@ -37,7 +37,12 @@ import {
type SessionDiscovery,
type SessionImportRecord,
} from "../../../core/brain/sessions/discover.ts";
+import {
+ SESSION_RESUME_DISCARD,
+ type SessionResumeDiscard,
+} from "../../../core/brain/sessions/checkpoint.ts";
import { SessionImportError, type SessionAdapterId } from "../../../core/brain/sessions/types.ts";
+import { serializeImportCensus, type ImportCensus } from "../../../core/brain/import-census.ts";
import {
createSafeguard,
OPERATION,
@@ -455,6 +460,21 @@ function emitCoverage(
for (const line of discovery.unreadable) info(` unreadable: ${line}`);
}
+/**
+ * What each discard reason means, in one clause. Keyed by the vocabulary so
+ * a member added without a sentence fails to compile - the alternative was
+ * one sentence about the session log that was already wrong for a reason
+ * that is about the checkpoint's own bytes.
+ */
+const RESUME_DISCARD_EXPLANATION: Readonly<Record<SessionResumeDiscard, string>> = Object.freeze({
+ [SESSION_RESUME_DISCARD.headChanged]:
+ "the session log is not the one the boundary was taken against",
+ [SESSION_RESUME_DISCARD.fileShrank]:
+ "the session log has shrunk since the boundary was taken, so the boundary is past its end",
+ [SESSION_RESUME_DISCARD.payloadInvalid]:
+ "the checkpoint on disk carries a turn boundary this build could not have written",
+});
+
/** The per-file import report, shared by the path form and the sweep. */
function emitImportReport(
result: {
@@ -485,6 +505,9 @@ function emitImportReport(
filtered_turns: f.filtered_turns,
recall_turns_imported: f.recall_turns_imported,
recall_summary_nodes: f.recall_summary_nodes,
+ turns_resumed: f.turns_resumed,
+ resume_discarded: f.resume_discarded,
+ census: serializeImportCensus(f.census),
errors: f.errors,
})),
warnings: result.warnings,
@@ -507,6 +530,16 @@ function emitImportReport(
ok(` signals_deduped: ${f.signals_deduped}`);
ok(` tool_replays: ${f.tool_replays}`);
ok(` filtered_turns: ${f.filtered_turns}`);
+ // Printed only when a checkpoint actually changed what the run did, so a
+ // first import's output is unchanged.
+ if (f.turns_resumed > 0) ok(` turns_resumed: ${f.turns_resumed}`);
+ if (f.resume_discarded !== null) {
+ info(
+ ` resume checkpoint discarded (${f.resume_discarded}): ${RESUME_DISCARD_EXPLANATION[f.resume_discarded]}, ` +
+ "so the file was re-imported from the start.",
+ );
+ }
+ emitCensus(f.census);
if (opts.recall) {
ok(` recall_turns_imported: ${f.recall_turns_imported}`);
ok(` recall_summary_nodes: ${f.recall_summary_nodes}`);
@@ -518,6 +551,18 @@ function emitImportReport(
for (const f of failures) process.stderr.write(`error: ${f.path}: ${f.message}\n`);
}
+/**
+ * The post-import read-back census, printed only when it has something to
+ * say: a run that wrote nothing has nothing to reconcile, and a clean census
+ * is one line rather than a list. A shortfall NAMES its hashes - the count
+ * alone is what this line exists not to be.
+ */
+function emitCensus(census: ImportCensus): void {
+ if (census.attempted === 0) return;
+ ok(` census: ${census.found}/${census.attempted} written signals read back (${census.outcome})`);
+ for (const hash of census.missing) info(` not found on disk: ${hash}`);
+}
+
/** Map a session-import throw onto this verb's exit codes. */
function reportImportFailure(exc: unknown): number {
if (exc instanceof SessionImportError) {
diff --git a/src/cli/command-manifest.ts b/src/cli/command-manifest.ts
index 99cf604c..2aef3c04 100644
--- a/src/cli/command-manifest.ts
+++ b/src/cli/command-manifest.ts
@@ -675,6 +675,19 @@ export const CLI_COMMAND_MANIFEST: CliRootManifest = Object.freeze({
command("rerank-provider", "Manage reranker provider profiles"),
command("rerank-fit", "Diagnose whether the configured reranker fits this vault's queries"),
command("plan", "Preview the should-read shortlist for a query without reading the notes"),
+ // The whole schema, for the reason the neighbours give: this
+ // family has no per-verb `--help`, so an unmodelled flag is
+ // unreachable from help and from completions alike.
+ command(
+ "restamp",
+ "Record this build's sqlite-vec version as the one the stored vectors are accepted under (dry-run unless --apply; refuses a model or dimension disagreement)",
+ [
+ flag("vault", "string"),
+ flag("config", "string"),
+ flag("db", "string"),
+ flag("apply", "boolean"),
+ ],
+ ),
// The whole `parseFlags` schema, for the reason the two builders
// above give: there is no per-verb `--help` on this family, so a
// flag the manifest does not model is a flag neither help nor
diff --git a/src/cli/main.ts b/src/cli/main.ts
index f2c36bf1..d7d7a28d 100644
--- a/src/cli/main.ts
+++ b/src/cli/main.ts
@@ -31,7 +31,8 @@ import { listVaultPages, writeFrontmatter } from "../core/vault.ts";
import { CliError, parseFlags } from "./argparse.ts";
import { installStdoutEpipeGuard, isEpipeError } from "./stdout-guard.ts";
import { handleAiderSubcommand } from "./aider.ts";
-import { handleBrainSubcommand } from "./brain.ts";
+import { handleBrainSubcommand, isBrainImportVerb } from "./brain.ts";
+import { ORIGIN_CHANNEL, setOriginChannel, type OriginChannel } from "../core/origin-channel.ts";
import { handleDisciplineSubcommand } from "./discipline.ts";
import { handlePartnerSubcommand } from "./partner.ts";
import { handleSearchSubcommand } from "./search.ts";
@@ -962,6 +963,28 @@ function cmdCompletions(argv: ReadonlyArray<string>): number {
return 0;
}
+/** The command that hands this process over to the MCP transport. */
+const MCP_COMMAND = "mcp";
+
+/**
+ * Which origin channel this invocation is, derived from the command line
+ * and from nothing else (Unit C, `src/core/origin-channel.ts`).
+ *
+ * Deriving it HERE, in the dispatcher, is the point: there is no flag
+ * that names the channel and no way to reach {@link setOriginChannel}
+ * from a tool argument, so a record's channel is a fact about how the
+ * process was started rather than a claim anyone made.
+ *
+ * `o2b tool-call` stays `cli` deliberately. It invokes an MCP tool
+ * HANDLER, but the channel names the transport that carried the request,
+ * and that transport was a shell.
+ */
+function originChannelForArgv(command: string, rest: ReadonlyArray<string>): OriginChannel {
+ if (command === MCP_COMMAND) return ORIGIN_CHANNEL.mcpTool;
+ if (command === "brain" && isBrainImportVerb(rest)) return ORIGIN_CHANNEL.import;
+ return ORIGIN_CHANNEL.cli;
+}
+
export async function main(argv: ReadonlyArray<string>): Promise<number> {
if (argv.length === 0) {
// No command named: the terse banner, not the complete index. The
@@ -977,6 +1000,10 @@ export async function main(argv: ReadonlyArray<string>): Promise<number> {
const command = argv[0]!;
const rest = argv.slice(1);
+ // Claim the origin channel for this process before anything can write.
+ // The two help arms above return without writing, so they need none.
+ setOriginChannel(originChannelForArgv(command, rest));
+
// Per-command --help support: print the dedicated help line plus generic.
// The `brain` subcommand has its own dispatcher with per-verb help, so we
// skip the generic shortcut and hand control over directly.
diff --git a/src/cli/search.ts b/src/cli/search.ts
index cf1eb70d..9b7e0d5a 100644
--- a/src/cli/search.ts
+++ b/src/cli/search.ts
@@ -24,6 +24,7 @@ import { cmdSearchPlan } from "./search/verbs/plan.ts";
import { cmdSearchProvider, cmdSearchRerankProvider } from "./search/verbs/provider-registry.ts";
import { cmdSearchQuery } from "./search/verbs/query.ts";
import { cmdSearchRerankFit } from "./search/verbs/rerank-fit.ts";
+import { cmdSearchRestamp } from "./search/verbs/restamp.ts";
import { cmdSearchStatus } from "./search/verbs/status.ts";
import { cmdSearchVectorBackfill } from "./search/verbs/vector-backfill.ts";
import { cmdSearchWatch } from "./search/verbs/watch.ts";
@@ -42,6 +43,7 @@ const KNOWN_VERBS = new Set([
"rerank-provider",
"rerank-fit",
"plan",
+ "restamp",
"watch",
"vector-backfill",
"event-anchor-backfill",
@@ -87,6 +89,8 @@ export async function handleSearchSubcommand(argv: ReadonlyArray<string>): Promi
return await cmdSearchRerankFit(rest);
case "plan":
return await cmdSearchPlan(rest);
+ case "restamp":
+ return await cmdSearchRestamp(rest);
case "vector-backfill":
return await cmdSearchVectorBackfill(rest);
case "event-anchor-backfill":
diff --git a/src/cli/search/verbs/check.ts b/src/cli/search/verbs/check.ts
index b29c12dd..b825b001 100644
--- a/src/cli/search/verbs/check.ts
+++ b/src/cli/search/verbs/check.ts
@@ -75,8 +75,20 @@ import {
} from "../../../core/brain/diagnostics.ts";
import { nextCommandField } from "../../../core/brain/next-step.ts";
import { formatStampMismatch } from "../../../core/integrity/stamp.ts";
-import { indexCheck, serializeStampMismatches } from "../../../core/search/index.ts";
-import type { IndexCheckReport, ResolvedSearchConfig } from "../../../core/search/index.ts";
+import {
+ indexCheck,
+ serializeEmbedderRecordCensus,
+ serializePendingVectorCensus,
+ serializeStampMismatches,
+ serializeVisibilityHonestyFinding,
+} from "../../../core/search/index.ts";
+import type {
+ EmbedderRecordCensus,
+ IndexCheckReport,
+ PendingVectorCensus,
+ ResolvedSearchConfig,
+ VisibilityHonestyFinding,
+} from "../../../core/search/index.ts";
import { PROVIDER_PROBE } from "../../../core/search/provider-probe.ts";
import { acquireWriterLock } from "../../../core/search/store.ts";
import { runIntegrityCheck } from "../../../core/search/store/lifecycle.ts";
@@ -390,6 +402,20 @@ function jsonForCheck(r: IndexCheckReport): Record<string, unknown> {
// the failure mode worth having.
provider_probe: r.providerProbe,
provider_reason: r.providerReason,
+ // Emitted in every state, unlike the drift block below: an index
+ // whose pending-vector count could not be taken is the one state
+ // this key exists to report, and an absent key would be read as a
+ // count of zero (nothing-writes-silently, unit A).
+ pending_vectors: serializePendingVectorCensus(r.pendingVectors),
+ // Same terms again: the audit says what it compared, or that it
+ // compared nothing (nothing-writes-silently, unit G).
+ embedder_record: serializeEmbedderRecordCensus(r.embedderRecord),
+ // Present only when the vault/index carries at least one
+ // visibility:-tagged page, so a vault that never uses the field
+ // stays byte-identical (nothing-writes-silently, unit H, form B).
+ ...(r.visibilityHonesty !== undefined
+ ? { visibility_honesty: serializeVisibilityHonestyFinding(r.visibilityHonesty) }
+ : {}),
// Emitted only on drift, so a matching store's JSON is byte-identical
// to the pre-gate output (context-integrity-gates, Unit E).
...(r.embeddingAbi.length > 0
@@ -425,6 +451,44 @@ function jsonForIntegrity(r: IntegrityReport): Record<string, unknown> {
};
}
+/** The record-vs-data audit as one operator-facing value. */
+function describeEmbedderRecord(census: EmbedderRecordCensus): string {
+ if (census.verdict === "unrecorded") return `${census.verdict} (${census.reason})`;
+ const stored = census.storedDimensions.join(", ");
+ return (
+ `${census.outcome} (records ${census.recordedDimension}; ` +
+ `stored ${stored === "" ? "none" : stored}; ` +
+ `chunk_vec declares ${census.vecDeclaredWidth ?? "nothing"})`
+ );
+}
+
+/** The pending-vector census as one operator-facing value. */
+function describePendingVectors(census: PendingVectorCensus): string {
+ if (census.verdict === "unrecorded") return `${census.verdict} (${census.reason})`;
+ return `${census.pending} of ${census.chunks} chunk(s) have no vector`;
+}
+
+/**
+ * The visibility honesty finding as one operator-facing line
+ * (nothing-writes-silently, unit H, form B). Both counts are the
+ * registry's own numbers, read at render time - never hand-written here.
+ *
+ * The line names the ENUMERATED population rather than implying it is
+ * the product's whole surface area. It is not: the MCP half is swept
+ * mechanically and its rule has stated blind spots, and the CLI half is
+ * hand-enumerated one row per MCP mirror. A reader who took the
+ * denominator for "every note-returning surface there is" would size the
+ * enforcement work against a number that is a floor.
+ */
+function describeVisibilityHonesty(finding: VisibilityHonestyFinding): string {
+ return (
+ "visibility: frontmatter is a caller-supplied view filter, not a privacy boundary; " +
+ `${finding.excludedSurfaceCount} of the ${finding.totalSurfaceCount} note-returning ` +
+ "surfaces the visibility census enumerates do not honor it, and the census does not " +
+ "claim to enumerate them all (see tests/core/architecture/visibility-surface-census.test.ts)"
+ );
+}
+
function renderCheckHuman(r: IndexCheckReport): string {
const lines: string[] = [];
lines.push(`vault_readable: ${ok(r.vaultReadable)}`);
@@ -443,6 +507,16 @@ function renderCheckHuman(r: IndexCheckReport): string {
// about what could not be checked must not say.
lines.push(`provider_probe: ${r.providerProbe}`);
if (r.providerReason) lines.push(`provider_reason: ${r.providerReason}`);
+ // Always emitted, on the same terms and for the same reason: a count
+ // nobody could take is reported as unrecorded, never as zero.
+ lines.push(`pending_vectors: ${describePendingVectors(r.pendingVectors)}`);
+ lines.push(`embedder_record: ${describeEmbedderRecord(r.embedderRecord)}`);
+ // Present only when the vault/index carries at least one
+ // visibility:-tagged page - a vault that never uses the field gets no
+ // line at all, never a zero-count one.
+ if (r.visibilityHonesty !== undefined) {
+ lines.push(`visibility_honesty: ${describeVisibilityHonesty(r.visibilityHonesty)}`);
+ }
for (const w of r.warnings) lines.push(`warning: ${w}`);
for (const f of r.fatal) lines.push(`fatal: ${f}`);
if (r.recommendations.length > 0) {
diff --git a/src/cli/search/verbs/restamp.ts b/src/cli/search/verbs/restamp.ts
new file mode 100644
index 00000000..82818e07
--- /dev/null
+++ b/src/cli/search/verbs/restamp.ts
@@ -0,0 +1,200 @@
+/**
+ * `o2b search restamp` — record this build's sqlite-vec version as the
+ * one the stored vectors are accepted under (nothing-writes-silently,
+ * unit G).
+ *
+ * `embedding_vec_version` is an ABI marker, not a property of any
+ * vector: it says which sqlite-vec build wrote the storage, and two
+ * peers on different builds each read the other's index as drifted.
+ * That is precisely why `integrity.embedding_abi` defaults to `warn`
+ * rather than refusing - a hard gate would loop a synced vault through
+ * rebuilds - and until this verb existed the only way to clear the
+ * warning was `o2b search reindex --embeddings`, which pays provider
+ * prices to re-derive vectors that were never the problem.
+ *
+ * What this verb claims, exactly: that the operator accepts the stored
+ * vectors under the extension this build loads. It does not verify
+ * them. Nothing can - a vector's bytes do not carry the version that
+ * wrote them - and a verb that implied otherwise would be the kind of
+ * unearned reassurance this wave removes.
+ *
+ * Three properties are deliberate:
+ *
+ * - Dry run is the DEFAULT, as on every other repair verb in this
+ * family. `--apply` is the only path that writes a cell.
+ * - It refuses BY NAME when the recorded model or dimension disagrees
+ * with this build. Those two describe the vectors themselves;
+ * writing a new token over them would record an identity nobody
+ * checked. Repairing that needs re-embedding under a verified
+ * identity, which is deferred by design - the recon that scoped
+ * this wave put it behind the same explicit cost gate
+ * `planVectorBackfill` uses, and it is not built.
+ * - It contacts no provider on any path. The only outside thing it
+ * touches is the sqlite-vec extension, loaded into an in-memory
+ * database to ask its version.
+ *
+ * The write does NOT go through `Store.open(…, { mode: "write" })`.
+ * That path runs `ensureEmbeddingModel`, which CLEARS every stored
+ * vector when the model or dimension changed - exactly the destructive
+ * repair this verb refuses to perform. It takes the writer lock and
+ * writes the one cell instead, the same way `search check --integrity`
+ * records its verdict.
+ */
+
+import { Database } from "bun:sqlite";
+
+import { compareStamps, formatStampMismatch } from "../../../core/integrity/stamp.ts";
+import {
+ acquireWriterLock,
+ contradictedAbiFields,
+ EMBEDDING_ABI_FIX_COMMAND,
+ EMBEDDING_VEC_VERSION_STATE_KEY,
+ peekEmbeddingAbiSync,
+ runtimeEmbeddingAbi,
+} from "../../../core/search/store.ts";
+import { describeUnreadableIndex, setState } from "../../../core/search/store/state.ts";
+import { loadVecExtension } from "../../../core/search/store/vectors.ts";
+import { SearchError } from "../../../core/search/types.ts";
+import { info, ok } from "../../output.ts";
+import { flagBoolean, parseFlags, resolveConfig, VAULT_FLAGS } from "../helpers.ts";
+
+/** What one restamp run concluded, before it is rendered. */
+interface RestampResult {
+ /** The token the index records, or null when it never recorded one. */
+ readonly recorded: string | null;
+ /** The version this build's extension reports. */
+ readonly runtime: string;
+ /** Whether the two differ at all. */
+ readonly changed: boolean;
+ /** Whether this run wrote the cell. False on every dry run. */
+ readonly applied: boolean;
+}
+
+/**
+ * The sqlite-vec version this build reports, from an in-memory database
+ * so the real index is not opened to answer a question about the
+ * extension.
+ */
+function runtimeVecVersion(): string {
+ const probe = new Database(":memory:");
+ try {
+ const version = loadVecExtension(probe);
+ if (version === null) {
+ throw new SearchError(
+ "VEC_EXTENSION_UNAVAILABLE",
+ "sqlite-vec did not load in this build, so there is no version to record. " +
+ "Run `o2b search check` for the extension's own diagnosis",
+ );
+ }
+ return version;
+ } finally {
+ probe.close();
+ }
+}
+
+/**
+ * Refuse, naming every field, when the record disagrees with this build
+ * about something a restamp cannot fix. Only fields the index actually
+ * RECORDED count - an unrecorded token is a store written before that
+ * token existed, which `contradictedAbiFields` already declines to
+ * treat as wrong.
+ */
+function refuseUnrepairableDrift(
+ recorded: Record<string, string | null>,
+ runtime: Record<string, string | null>,
+): void {
+ const blocking = contradictedAbiFields(compareStamps(recorded, runtime)).filter(
+ (m) => m.field !== EMBEDDING_VEC_VERSION_STATE_KEY,
+ );
+ if (blocking.length === 0) return;
+ throw new SearchError(
+ "EMBEDDING_DIMENSION_MISMATCH",
+ `restamp refuses: ${blocking.map(formatStampMismatch).join("; ")}. Those fields describe ` +
+ `the stored vectors, so recording a new token over them would claim an identity nothing ` +
+ `verified. The repair is re-embedding under a verified identity, which is deferred by ` +
+ `design and not built; today's honest fix is a rebuild. Run: ${EMBEDDING_ABI_FIX_COMMAND}`,
+ );
+}
+
+/** Write the one cell, under the writer lock, and nothing else. */
+async function applyRestamp(dbPath: string, version: string): Promise<void> {
+ const release = await acquireWriterLock(dbPath);
+ try {
+ const db = new Database(dbPath);
+ try {
+ setState(db, EMBEDDING_VEC_VERSION_STATE_KEY, version);
+ } finally {
+ db.close();
+ }
+ } finally {
+ await release();
+ }
+}
+
+function jsonForResult(r: RestampResult): Record<string, unknown> {
+ return {
+ dry_run: !r.applied,
+ field: EMBEDDING_VEC_VERSION_STATE_KEY,
+ recorded: r.recorded,
+ runtime: r.runtime,
+ changed: r.changed,
+ applied: r.applied,
+ };
+}
+
+function renderHuman(r: RestampResult): void {
+ if (!r.changed) {
+ ok(`restamp: ${EMBEDDING_VEC_VERSION_STATE_KEY} already records ${r.runtime}; nothing to do`);
+ return;
+ }
+ const change = `${EMBEDDING_VEC_VERSION_STATE_KEY} ${r.recorded ?? "unrecorded"} -> ${r.runtime}`;
+ if (r.applied) {
+ ok(`restamp: ${change} recorded`);
+ return;
+ }
+ ok(`restamp dry-run: ${change}`);
+ info(" nothing was written; pass --apply to record it");
+}
+
+export async function cmdSearchRestamp(argv: ReadonlyArray<string>): Promise<number> {
+ const { flags } = parseFlags(argv, {
+ ...VAULT_FLAGS,
+ apply: { type: "boolean" },
+ json: { type: "boolean" },
+ });
+ const cfg = resolveConfig(flags);
+ if (!cfg.semantic.enabled) {
+ throw new SearchError(
+ "EMBEDDING_DISABLED",
+ "semantic search is disabled in this configuration, so there is no embedder identity to restamp",
+ );
+ }
+
+ const peek = peekEmbeddingAbiSync(cfg.dbPath);
+ if (peek.kind !== "read") {
+ // An index that is not there and one that will not open are
+ // different faults and are named as the different faults they are.
+ throw new SearchError(
+ peek.kind === "absent" ? "INDEX_MISSING" : "INDEX_UNREADABLE",
+ peek.kind === "absent"
+ ? `no search index at ${cfg.dbPath}; there is nothing to restamp`
+ : `the search index at ${describeUnreadableIndex(cfg.dbPath, peek.detail)}`,
+ );
+ }
+
+ const runtime = runtimeVecVersion();
+ refuseUnrepairableDrift(peek.value, runtimeEmbeddingAbi(cfg, runtime));
+
+ const recorded = peek.value[EMBEDDING_VEC_VERSION_STATE_KEY] ?? null;
+ const changed = recorded !== runtime;
+ const apply = flagBoolean(flags, "apply");
+ if (changed && apply) await applyRestamp(cfg.dbPath, runtime);
+
+ const result: RestampResult = { recorded, runtime, changed, applied: changed && apply };
+ if (flagBoolean(flags, "json")) {
+ process.stdout.write(JSON.stringify(jsonForResult(result)) + "\n");
+ } else {
+ renderHuman(result);
+ }
+ return 0;
+}
diff --git a/src/core/brain/checkpoint-store.ts b/src/core/brain/checkpoint-store.ts
new file mode 100644
index 00000000..b9d616f2
--- /dev/null
+++ b/src/core/brain/checkpoint-store.ts
@@ -0,0 +1,151 @@
+/**
+ * The checkpoint substrate two resume lanes share.
+ *
+ * `ingest/checkpoint.ts` shipped first and owned all of this privately: the
+ * opt-out env var, the id validation, the location under the machine-artifact
+ * directory, the read that REFUSES an unknown `schema_version` instead of
+ * silently resetting, and the write that takes the sync lock, asserts vault
+ * identity and lands atomically. The sessions lane now needs the same
+ * discipline for a different payload - a turn boundary rather than a set of
+ * completed paths - and the honest way to give it that is to extract what the
+ * two share rather than copy a second, drifting version of it.
+ *
+ * What is shared is the MECHANISM; what is not is the record. Each lane keeps
+ * its own schema, its own serializer, and its own no-op rule, because those
+ * are the parts that differ: the ingest checkpoint no-ops when the completed
+ * SET is unchanged, and the sessions checkpoint when the serialized bytes are.
+ *
+ * Why an unknown `schema_version` is a hard error in both lanes: a resume
+ * point that cannot be read is not "no resume point". Treating it as one
+ * would present completed work as pending, which for the sessions lane means
+ * re-hashing every turn a previous run already finished - the exact cost this
+ * substrate exists to avoid.
+ *
+ * Language-agnostic: keys are hex ids and paths; no natural-language content
+ * is inspected.
+ */
+
+import { existsSync, readFileSync, rmSync } from "node:fs";
+import { join } from "node:path";
+
+import { atomicWriteFileSync } from "../fs-atomic.ts";
+import { DERIVED_STORE_DIR } from "./path-constants.ts";
+import { acquireLockSyncWithRetry, type LockHandle } from "./sync-lockfile.ts";
+import { assertVaultIdentityForWrite } from "./vault-identity.ts";
+
+/**
+ * Env var that, when truthy, disables checkpoint reads and writes in EVERY
+ * lane. One spelling on purpose: an operator who has turned resumability off
+ * for a deterministic run means it for the whole process, and a second var
+ * would be a second thing to remember and a second thing to forget.
+ */
+export const NO_CHECKPOINT_ENV = "OSB_INGEST_NO_CHECKPOINT";
+
+/** Shape of a checkpoint id: a lowercase hex digest prefix. */
+const CHECKPOINT_ID_RE = /^[0-9a-f]{6,64}$/;
+
+/**
+ * Whether checkpointing is active. Off only when {@link NO_CHECKPOINT_ENV}
+ * holds a truthy value; the empty string, `0`, and `false` are all treated as
+ * unset so an accidentally-exported empty var does not silently disable
+ * resumability.
+ */
+export function checkpointingEnabled(): boolean {
+ const raw = process.env[NO_CHECKPOINT_ENV];
+ if (raw === undefined) return true;
+ const v = raw.trim().toLowerCase();
+ return v === "" || v === "0" || v === "false";
+}
+
+/**
+ * Refuse an id that is not a hex digest, naming the lane. The id becomes a
+ * filename, so a caller-supplied `../../etc/passwd` has to fail here rather
+ * than reach `join`.
+ */
+export function assertCheckpointId(label: string, id: string): void {
+ if (!CHECKPOINT_ID_RE.test(id)) {
+ throw new Error(`invalid ${label} id (expected lowercase hex): ${JSON.stringify(id)}`);
+ }
+}
+
+/** Absolute path of one checkpoint file inside a lane's own directory. */
+export function checkpointFilePath(vault: string, dir: string, label: string, id: string): string {
+ assertCheckpointId(label, id);
+ return join(vault, DERIVED_STORE_DIR, dir, `${id}.json`);
+}
+
+/**
+ * Read one checkpoint file as a plain object. Returns `null` when the file
+ * does not exist. Throws - naming `label` and the path - when the bytes are
+ * not JSON, are not an object, or carry a `schema_version` this build does
+ * not understand. Never a silent reset.
+ */
+export function readCheckpointObject(
+ path: string,
+ label: string,
+ schemaVersion: number,
+): Record<string, unknown> | null {
+ if (!existsSync(path)) return null;
+ let parsed: unknown;
+ try {
+ parsed = JSON.parse(readFileSync(path, "utf8"));
+ } catch (e) {
+ throw new Error(`${label} is corrupted JSON: ${path}`, { cause: e });
+ }
+ if (parsed === null || typeof parsed !== "object" || Array.isArray(parsed)) {
+ throw new Error(`${label} is not an object: ${path}`);
+ }
+ const obj = parsed as Record<string, unknown>;
+ if (obj["schema_version"] !== schemaVersion) {
+ throw new Error(
+ `${label} schema_version ${String(obj["schema_version"])} not supported (expected ${schemaVersion}): ${path}`,
+ );
+ }
+ return obj;
+}
+
+/**
+ * Write one checkpoint file behind the vault-identity guard, atomically, and
+ * only when the bytes differ from what is already there. Returns `true` when
+ * a write happened, `false` on the byte-identical no-op - which is what keeps
+ * a re-recorded boundary from touching the file's mtime.
+ */
+export function writeCheckpointObject(vault: string, path: string, body: string): boolean {
+ // Vault-identity write guard (context-integrity-gates, Unit J).
+ assertVaultIdentityForWrite(vault);
+ if (existsSync(path) && readFileSync(path, "utf8") === body) return false;
+ atomicWriteFileSync(path, body);
+ return true;
+}
+
+/**
+ * Remove one checkpoint file. Returns `true` when a file was removed, `false`
+ * when there was none. Absence is the terminal state for a resume point: a
+ * lane with no checkpoint starts from the beginning, which is the same thing
+ * a checkpoint rewritten to empty would have to mean.
+ */
+export function removeCheckpointFile(vault: string, path: string): boolean {
+ // Vault-identity write guard (context-integrity-gates, Unit J).
+ assertVaultIdentityForWrite(vault);
+ if (!existsSync(path)) return false;
+ rmSync(path);
+ return true;
+}
+
+/**
+ * Run `fn` with the sync lock held on a checkpoint file.
+ *
+ * The read, the merge and the write of a checkpoint are ONE critical section.
+ * A plan's items - and a machine's session logs - are dispatched to parallel
+ * workers, so two of them finishing at once would otherwise each write back
+ * the state they read, and the later write would erase the earlier progress:
+ * done work silently turned back into pending work on the next resume.
+ */
+export function withCheckpointLock<T>(path: string, fn: () => T): T {
+ const handle: LockHandle = acquireLockSyncWithRetry(path);
+ try {
+ return fn();
+ } finally {
+ handle.release();
+ }
+}
diff --git a/src/core/brain/continuity/store.ts b/src/core/brain/continuity/store.ts
index 99043a95..09cccf58 100644
--- a/src/core/brain/continuity/store.ts
+++ b/src/core/brain/continuity/store.ts
@@ -2,6 +2,7 @@ import { createHash } from "node:crypto";
import { existsSync, mkdirSync, readFileSync, readdirSync, statSync, writeFileSync } from "node:fs";
import { join } from "node:path";
+import { originChannelStamp } from "../../origin-channel.ts";
import { BRAIN_LOG_REL, ensureInsideVault } from "../paths.ts";
import { acquireLockSync } from "../sync-lockfile.ts";
import { assertVaultIdentityForWrite } from "../vault-identity.ts";
@@ -266,12 +267,18 @@ function buildRecord(
assertCanonicalCreatedAt(input.createdAt);
const payloadResult = safeContinuityPayload(input.payload ?? {});
const sourceRefs = Object.freeze([...(input.sourceRefs ?? [])]);
- // `schema` stays OUT of recordId(): identical records must keep
- // identical dedup ids across the version-stamp transition.
+ // `schema` and `originChannel` both stay OUT of recordId(): identical
+ // records must keep identical dedup ids across the version-stamp
+ // transition, and across the channel that happened to append them.
const id = recordId(input.kind, input.createdAt, sourceRefs, payloadResult.payload);
return Object.freeze({
schema: CONTINUITY_SCHEMA_VERSION,
id,
+ // Server-derived (Unit C), read off the module rather than the input:
+ // AppendContinuityRecordInput has no channel field and must not grow
+ // one. Distinct from the payload's `channel` key, which recall
+ // telemetry uses for the transport a recall was DELIVERED over.
+ originChannel: originChannelStamp(),
kind: input.kind,
createdAt: input.createdAt,
sourceRefs,
diff --git a/src/core/brain/continuity/types.ts b/src/core/brain/continuity/types.ts
index fa72a3af..d3178572 100644
--- a/src/core/brain/continuity/types.ts
+++ b/src/core/brain/continuity/types.ts
@@ -1,3 +1,5 @@
+import type { OriginChannelStamp } from "../../origin-channel.ts";
+
/**
* Contract-wide continuity schema version (Memory Observability Suite,
* t_26040ee8), stamped on every new record at `buildRecord()`.
@@ -89,6 +91,20 @@ export interface ContinuityRecord {
*/
readonly schema?: string;
readonly id: string;
+ /**
+ * Server-derived channel of the process that appended this record
+ * (Unit C of the nothing-writes-silently wave): `mcp-tool`, `cli`,
+ * `import`, or the explicit `unset` when no entry point claimed the
+ * process. Never caller-supplied — {@link AppendContinuityRecordInput}
+ * has no field for it and `src/core/origin-channel.ts` says why.
+ *
+ * `undefined` on records written before the stamp existed, which are
+ * never rewritten to add one. Deliberately EXCLUDED from `recordId()`,
+ * for the same reason `schema` is: adding provenance must not
+ * re-identify a record, or the same append from two channels would
+ * stop deduping against itself.
+ */
+ readonly originChannel?: OriginChannelStamp;
readonly kind: ContinuityRecordKind;
readonly createdAt: string;
readonly sourceRefs: ReadonlyArray<ContinuitySourceRef>;
diff --git a/src/core/brain/dead-letter.ts b/src/core/brain/dead-letter.ts
new file mode 100644
index 00000000..370e2bb0
--- /dev/null
+++ b/src/core/brain/dead-letter.ts
@@ -0,0 +1,344 @@
+/**
+ * Durable dead letters for multi-artifact write lanes
+ * (nothing-writes-silently, unit E).
+ *
+ * One lane in this project commits SEVERAL artifacts from one validated
+ * envelope payload: `commitExtractedSignals` writes a file per mined
+ * signal. A failure on the fourth leaves three on disk, and until now the
+ * only record of that state was the thrown error - which lives exactly as
+ * long as the response that carried it. A caller that dropped the
+ * response left a vault holding three signals nobody knows about and a
+ * payload nobody knows was half-applied.
+ *
+ * This module is the record that outlives the response. It is deliberately
+ * NOT for single-artifact lanes (the design's dead-letter boundary): a
+ * design note or a research report either lands or does not, and its one
+ * failure IS the response. Widening this to every write would make a
+ * durable file out of every transient error and bury the case that
+ * matters.
+ *
+ * ## Shape and vocabulary
+ *
+ * A record answers "what did this lane try, what landed, and what did
+ * not" in the wave's SHARED reconciliation vocabulary - `attempted`,
+ * `found`, `missing` with the missing keys NAMED (see
+ * `src/core/reconciliation-report.ts`). No dialect: the same three words
+ * the import read-back census and the embedder audit use. Beside them it
+ * carries the lane, the envelope identity the payload answered, and the
+ * first real error, so an operator reading the file knows which payload
+ * to re-run and why the run stopped.
+ *
+ * ## Where it lives, and why nothing here deletes it
+ *
+ * Under `<vault>/.open-second-brain/dead-letters/`, the machine-artifact
+ * root the ingest checkpoint and the content manifest already use, with
+ * the same atomic-write and vault-identity discipline. It is a declared
+ * `STATE_SURFACES` row, so `o2b state status` names the location and a
+ * state migration moves it with everything else.
+ *
+ * RETENTION: nothing in this project removes a dead letter. The whole
+ * point of the file is to be the only surviving evidence of a partial
+ * write, and a tool that swept it on a schedule - or cleared it on the
+ * next successful run - would delete exactly the evidence somebody had
+ * not read yet. Removing one is an operator action on a named file, taken
+ * once the payload has been reconciled. The records are small and are
+ * written only when a multi-artifact commit fails part way, so there is
+ * no growth to manage; `listDeadLetters` is the read API that says what
+ * is there, and no MCP tool is added for it.
+ *
+ * ## Reading refuses rather than skips
+ *
+ * A corrupt file or an unknown `schema_version` is a hard error, on the
+ * single read and on the listing alike. A skip would turn "the record of
+ * a partial write is unreadable" into "there was no partial write", which
+ * is the failure this module exists to prevent, one level up.
+ */
+
+import { existsSync, readFileSync, readdirSync } from "node:fs";
+import { join } from "node:path";
+
+import { atomicWriteFileSync } from "../fs-atomic.ts";
+import { canonicalJson, sha256Hex } from "../integrity/digest.ts";
+import {
+ deriveReconciliationOutcome,
+ isReconciliationOutcome,
+ type ReconciliationOutcome,
+ type ReconciliationReport,
+} from "../reconciliation-report.ts";
+import { DERIVED_STORE_DIR } from "./path-constants.ts";
+import { isoSecond } from "./time.ts";
+import { assertVaultIdentityForWrite } from "./vault-identity.ts";
+
+/** Only schema version currently understood. Unknown versions are refused. */
+export const DEAD_LETTER_SCHEMA_VERSION = 1 as const;
+
+/** Subdirectory under the machine-artifact root holding one JSON per record. */
+const DEAD_LETTER_DIR = "dead-letters";
+
+/** Hex characters of the content digest that makes an id unique. */
+const ID_DIGEST_CHARS = 12;
+
+/** The `<lane>-<digest>` id shape, as the reader validates it. */
+const ID_RE = new RegExp(`^[a-z][a-z0-9-]*-[0-9a-f]{${ID_DIGEST_CHARS}}$`);
+
+/**
+ * Lanes that commit several artifacts from one validated payload, and so
+ * may record a dead letter. A union rather than an open string: a new
+ * lane joining this list is a decision somebody types, and the compiler
+ * asks for it at every call site.
+ */
+export type DeadLetterLane = "extract-signals";
+
+/** Which payload, from which lane, the unwritten items belonged to. */
+export interface DeadLetterEnvelopeIdentity {
+ readonly lane: DeadLetterLane;
+ /** The needs-llm-step step name the payload answered. */
+ readonly step: string;
+ /** The lane's own identity for this payload - a session id, a plan id. */
+ readonly reference: string;
+ /** Vault-relative destination the artifacts were bound for. */
+ readonly target: string;
+}
+
+/** One persisted dead letter. */
+export interface DeadLetterRecord {
+ readonly schema_version: typeof DEAD_LETTER_SCHEMA_VERSION;
+ readonly id: string;
+ readonly lane: DeadLetterLane;
+ readonly step: string;
+ readonly reference: string;
+ readonly target: string;
+ /** Artifacts the commit set out to write. */
+ readonly attempted: number;
+ /** Artifacts whose bytes reached disk. */
+ readonly found: number;
+ /** Keys naming every artifact that did NOT reach disk. */
+ readonly missing: ReadonlyArray<string>;
+ readonly outcome: ReconciliationOutcome;
+ /** Message of the first real error, verbatim. */
+ readonly first_error: string;
+ readonly recorded_at: string;
+}
+
+export interface RecordDeadLetterInput {
+ readonly envelope: DeadLetterEnvelopeIdentity;
+ readonly report: ReconciliationReport;
+ /** The error that stopped the commit; anything throwable. */
+ readonly firstError: unknown;
+ readonly now: Date;
+}
+
+/** What {@link tryRecordDeadLetter} managed to do, either way. */
+export type DeadLetterOutcome =
+ | { readonly recorded: true; readonly id: string; readonly path: string }
+ | { readonly recorded: false; readonly reason: string };
+
+/** A dead letter could not be written, or one on disk could not be read. */
+export class DeadLetterError extends Error {
+ constructor(message: string, options?: ErrorOptions) {
+ super(message, options);
+ this.name = "DeadLetterError";
+ }
+}
+
+/** Directory holding this vault's dead letters. */
+export function deadLetterDir(vault: string): string {
+ return join(vault, DERIVED_STORE_DIR, DEAD_LETTER_DIR);
+}
+
+/** Absolute path of one record. */
+export function deadLetterPath(vault: string, id: string): string {
+ if (!ID_RE.test(id)) {
+ throw new DeadLetterError(`invalid dead-letter id: ${JSON.stringify(id)}`);
+ }
+ return join(deadLetterDir(vault), `${id}.json`);
+}
+
+/**
+ * Write one dead letter and return it.
+ *
+ * The id is derived from the record's own content and NOTHING else - no
+ * clock - so re-recording an identical failure rewrites one file rather
+ * than accumulating one per retry, while a different failure - different
+ * items missing, different error - gets its own. The retry rewrites
+ * `recorded_at`, so the one file names the most recent attempt; a
+ * timestamp in the id instead would have made a deterministically-failing
+ * payload retried twenty times into twenty byte-identical files in a
+ * directory nothing prunes. No lock: each record is a whole file written
+ * atomically under a content-derived name, so there is no read-modify-
+ * write for two writers to race on.
+ */
+export function recordDeadLetter(vault: string, input: RecordDeadLetterInput): DeadLetterRecord {
+ const { envelope, report, firstError, now } = input;
+ requireContent("step", envelope.step);
+ requireContent("reference", envelope.reference);
+ requireContent("target", envelope.target);
+ if (report.missing.length === 0) {
+ throw new DeadLetterError(
+ "a dead letter must name at least one unwritten item; this report names none",
+ );
+ }
+ const message = firstError instanceof Error ? firstError.message : String(firstError);
+ requireContent("first_error", message);
+
+ // Vault-identity write guard (context-integrity-gates, Unit J).
+ assertVaultIdentityForWrite(vault);
+
+ const body = {
+ schema_version: DEAD_LETTER_SCHEMA_VERSION,
+ lane: envelope.lane,
+ step: envelope.step.trim(),
+ reference: envelope.reference.trim(),
+ target: envelope.target.trim(),
+ attempted: report.attempted,
+ found: report.found,
+ missing: Object.freeze([...report.missing]),
+ outcome: deriveReconciliationOutcome(report),
+ first_error: message,
+ };
+ const id = `${envelope.lane}-${sha256Hex(canonicalJson(body)).slice(0, ID_DIGEST_CHARS)}`;
+ const record: DeadLetterRecord = Object.freeze({
+ ...body,
+ id,
+ recorded_at: isoSecond(now),
+ });
+ atomicWriteFileSync(deadLetterPath(vault, id), `${JSON.stringify(serialize(record), null, 2)}\n`);
+ return record;
+}
+
+/**
+ * {@link recordDeadLetter} for a caller that is already inside a catch
+ * block. It NEVER throws: the error the caller is holding is the one that
+ * has to reach the operator, and a recorder that raised over it would
+ * replace a partial-write report with a filesystem complaint. The failure
+ * is not swallowed either - the reason comes back on the outcome, for the
+ * caller to name in its own refusal.
+ */
+export function tryRecordDeadLetter(
+ vault: string,
+ input: RecordDeadLetterInput,
+): DeadLetterOutcome {
+ try {
+ const record = recordDeadLetter(vault, input);
+ return Object.freeze({
+ recorded: true as const,
+ id: record.id,
+ path: deadLetterPath(vault, record.id),
+ });
+ } catch (err) {
+ return Object.freeze({
+ recorded: false as const,
+ reason: err instanceof Error ? err.message : String(err),
+ });
+ }
+}
+
+/** Read one record, or null when nothing was ever written under that id. */
+export function readDeadLetter(vault: string, id: string): DeadLetterRecord | null {
+ const path = deadLetterPath(vault, id);
+ if (!existsSync(path)) return null;
+ return parseRecord(path);
+}
+
+/**
+ * Every record this vault holds, newest last. Ordered by `recorded_at`
+ * off the parsed records rather than by filename: the id is a content
+ * digest and carries no clock, so lexical order over the directory would
+ * be arbitrary. The id breaks a tie, so the order is total. An absent
+ * directory is an empty list - nothing has ever failed part way - while
+ * an unreadable record throws, because a listing that skipped it would
+ * under-report the very thing it is for.
+ */
+export function listDeadLetters(vault: string): ReadonlyArray<DeadLetterRecord> {
+ const dir = deadLetterDir(vault);
+ if (!existsSync(dir)) return Object.freeze([]);
+ const records = readdirSync(dir)
+ .filter((name) => name.endsWith(".json"))
+ .map((name) => parseRecord(join(dir, name)));
+ return Object.freeze(
+ records.toSorted(
+ (a, b) => a.recorded_at.localeCompare(b.recorded_at) || a.id.localeCompare(b.id),
+ ),
+ );
+}
+
+function parseRecord(path: string): DeadLetterRecord {
+ let parsed: unknown;
+ try {
+ parsed = JSON.parse(readFileSync(path, "utf8"));
+ } catch (err) {
+ throw new DeadLetterError(`dead letter is corrupt JSON: ${path}`, { cause: err });
+ }
+ if (parsed === null || typeof parsed !== "object" || Array.isArray(parsed)) {
+ throw new DeadLetterError(`dead letter is not an object: ${path}`);
+ }
+ const obj = parsed as Record<string, unknown>;
+ if (obj["schema_version"] !== DEAD_LETTER_SCHEMA_VERSION) {
+ throw new DeadLetterError(
+ `dead letter schema_version ${String(obj["schema_version"])} not supported ` +
+ `(expected ${DEAD_LETTER_SCHEMA_VERSION}): ${path}`,
+ );
+ }
+ const missing = obj["missing"];
+ if (!Array.isArray(missing) || missing.some((key) => typeof key !== "string")) {
+ throw new DeadLetterError(`dead letter has no readable missing key list: ${path}`);
+ }
+ const outcome = obj["outcome"];
+ if (!isReconciliationOutcome(outcome)) {
+ throw new DeadLetterError(`dead letter carries an unknown outcome: ${path}`);
+ }
+ return Object.freeze({
+ schema_version: DEAD_LETTER_SCHEMA_VERSION,
+ id: requireString(obj, "id", path),
+ lane: requireString(obj, "lane", path) as DeadLetterLane,
+ step: requireString(obj, "step", path),
+ reference: requireString(obj, "reference", path),
+ target: requireString(obj, "target", path),
+ attempted: requireNumber(obj, "attempted", path),
+ found: requireNumber(obj, "found", path),
+ missing: Object.freeze([...(missing as ReadonlyArray<string>)]),
+ outcome,
+ first_error: requireString(obj, "first_error", path),
+ recorded_at: requireString(obj, "recorded_at", path),
+ });
+}
+
+/** Serialized field order, fixed so a re-record is byte-identical. */
+function serialize(record: DeadLetterRecord): Record<string, unknown> {
+ return {
+ schema_version: record.schema_version,
+ id: record.id,
+ lane: record.lane,
+ step: record.step,
+ reference: record.reference,
+ target: record.target,
+ attempted: record.attempted,
+ found: record.found,
+ missing: [...record.missing],
+ outcome: record.outcome,
+ first_error: record.first_error,
+ recorded_at: record.recorded_at,
+ };
+}
+
+function requireContent(field: string, value: string): void {
+ if (value.trim().length === 0) {
+ throw new DeadLetterError(`dead letter: ${field} must be a non-empty string`);
+ }
+}
+
+function requireString(obj: Record<string, unknown>, field: string, path: string): string {
+ const value = obj[field];
+ if (typeof value !== "string" || value.trim().length === 0) {
+ throw new DeadLetterError(`dead letter has no readable ${field}: ${path}`);
+ }
+ return value;
+}
+
+function requireNumber(obj: Record<string, unknown>, field: string, path: string): number {
+ const value = obj[field];
+ if (typeof value !== "number" || !Number.isInteger(value) || value < 0) {
+ throw new DeadLetterError(`dead letter has no readable ${field}: ${path}`);
+ }
+ return value;
+}
diff --git a/src/core/brain/design-note.ts b/src/core/brain/design-note.ts
index b592e778..a2a4b265 100644
--- a/src/core/brain/design-note.ts
+++ b/src/core/brain/design-note.ts
@@ -48,6 +48,12 @@ import {
listDecisions,
} from "./decisions/record.ts";
import { buildNeedsLlmStep, type NeedsLlmStep } from "./llm-step.ts";
+import {
+ buildLinkCandidateManifest,
+ linkCandidateSchemaHint,
+ type LinkCandidateManifest,
+} from "./notes/link-candidates.ts";
+import { ownerScopeView } from "./owner-scope-view.ts";
import { decisionsDir } from "./paths.ts";
import {
assertResponseCheck,
@@ -169,6 +175,17 @@ export interface DesignNoteGrounding {
readonly emptyStores: ReadonlyArray<DesignNoteStore>;
}
+/**
+ * The one deferred generation step: the shared spine plus the wikilink
+ * targets the note may cite. A design note grounded in named records is
+ * the case the manifest exists for - the grounding below tells the agent
+ * WHAT the vault already argued, and this tells it which of those records
+ * a link can actually reach.
+ */
+export interface DesignNoteLlmStep extends NeedsLlmStep {
+ readonly link_candidates: LinkCandidateManifest;
+}
+
export interface DesignNoteReport {
readonly topic: string;
readonly slug: string;
@@ -176,11 +193,19 @@ export interface DesignNoteReport {
readonly grounding: DesignNoteGrounding;
/** Vault-relative path the committed note will occupy. */
readonly targetPath: string;
- readonly llmStep: NeedsLlmStep;
+ readonly llmStep: DesignNoteLlmStep;
}
export interface PlanDesignNoteOptions {
readonly now: Date;
+ /**
+ * Enforced owner scope for this caller, or null when none applies. It
+ * filters the envelope's link candidates - see
+ * {@link DesignNoteLlmStep} and `notes/link-candidates.ts` for why a
+ * candidate list is a disclosure. Required, so a surface cannot forget
+ * it and leak in silence.
+ */
+ readonly ownerScope: string | null;
}
export interface CommitDesignNoteOptions {
@@ -353,6 +378,10 @@ export function planDesignNote(
const grounding = designNoteGrounding(vault, trimmed);
const slug = slugify(trimmed);
const targetPath = posix.join(DESIGN_NOTE_DIR_REL, noteBasename(slug, opts.now));
+ const linkCandidates = buildLinkCandidateManifest(vault, {
+ query: trimmed,
+ visible: ownerScopeView(vault, opts.ownerScope).visible,
+ });
return Object.freeze({
topic: trimmed,
slug,
@@ -372,8 +401,10 @@ export function planDesignNote(
'payload: { "title", "summary"?, "alternatives": [ { "name", "approach", "tradeoffs", "recommended" } ] }',
"alternatives: at least two, each field non-empty",
"recommended: boolean; exactly one alternative may set it true",
+ linkCandidateSchemaHint(linkCandidates),
],
target_path: targetPath,
+ link_candidates: linkCandidates,
}),
});
}
diff --git a/src/core/brain/destructive-sites.ts b/src/core/brain/destructive-sites.ts
index 1e140efb..0ff23127 100644
--- a/src/core/brain/destructive-sites.ts
+++ b/src/core/brain/destructive-sites.ts
@@ -218,13 +218,13 @@ export const DESTRUCTIVE_SITES: Readonly<Record<string, DestructiveSiteDeclarati
"clears one aspect's exact-state file. Absence IS the cleared state, so the " +
"removal is the state transition rather than a loss of one.",
},
- "src/core/brain/ingest/checkpoint.ts": {
+ "src/core/brain/checkpoint-store.ts": {
calls: ["rmSync"],
recovery: UNARCHIVED_BRAIN,
reason:
- "clears one ingest plan's checkpoint once the plan reached its terminal state. " +
- "The checkpoint exists to let an interrupted plan resume; after the plan is " +
- "done there is nothing it could resume into.",
+ "clears one checkpoint - an ingest plan's or a session import's - once that run " +
+ "reached its terminal state. A checkpoint exists to let an interrupted run resume; " +
+ "after the run is done there is nothing it could resume into.",
},
"src/core/brain/skill-accept-journal.ts": {
calls: ["rmSync"],
diff --git a/src/core/brain/diarization.ts b/src/core/brain/diarization.ts
index a1876b85..1360834e 100644
--- a/src/core/brain/diarization.ts
+++ b/src/core/brain/diarization.ts
@@ -28,6 +28,12 @@ import { getEntity } from "./entities/registry.ts";
import type { EntityRef } from "./entities/types.ts";
import { getIngestedSource, listIngestedSources } from "./ingest/sources-registry.ts";
import { buildNeedsLlmStep, type NeedsLlmStep } from "./llm-step.ts";
+import {
+ buildLinkCandidateManifest,
+ linkCandidateSchemaHint,
+ type LinkCandidateManifest,
+} from "./notes/link-candidates.ts";
+import { ownerScopeView } from "./owner-scope-view.ts";
/** Vault-relative directory a fleshed profile note lands in. */
export const PROFILE_DIR_REL = "Brain/profiles";
@@ -75,11 +81,14 @@ export interface DiarizationGapLine {
}
/**
- * The single deferred generation step: the shared envelope spine, exactly.
- * It stays plain data - diarization is read-only and opens no durable
- * session - so it carries the spine and nothing beyond it.
+ * The single deferred generation step: the shared envelope spine plus the
+ * one field a profile needs and the spine will not carry - the wikilink
+ * targets the prose may cite. It stays plain data otherwise: diarization
+ * is read-only and opens no durable session.
*/
-export type DiarizationLlmStep = NeedsLlmStep;
+export interface DiarizationLlmStep extends NeedsLlmStep {
+ readonly link_candidates: LinkCandidateManifest;
+}
export interface DiarizationReport {
readonly entityId: string;
@@ -97,6 +106,16 @@ export interface DiarizationReport {
export interface DiarizationOptions {
readonly now: Date;
+ /**
+ * Enforced owner scope for this caller, or null when none applies. It
+ * filters the envelope's link candidates: the basenames of the
+ * preference, retired, inbox and entity trees are exactly the ids the
+ * ownership rule governs, so an unscoped manifest would offer another
+ * owner's private artifact as a citation target. Required rather than
+ * optional - a surface that forgot it would leak silently, which is
+ * the one failure mode this option exists for.
+ */
+ readonly ownerScope: string | null;
}
function toPosixRel(vault: string, abs: string): string {
@@ -227,6 +246,13 @@ export function diarize(
lines,
documentSet,
});
+ // The subject and its aliases are the query: a profile cites the notes
+ // that are about its subject, and those are the ones an over-bound
+ // vault has to keep when the manifest truncates.
+ const linkCandidates = buildLinkCandidateManifest(vault, {
+ query: [entity.name, ...entity.aliases].join(" "),
+ visible: ownerScopeView(vault, opts.ownerScope).visible,
+ });
const llmStep: DiarizationLlmStep = buildNeedsLlmStep({
step: PROFILE_PROSE_STEP,
prompt:
@@ -236,8 +262,10 @@ export function diarize(
schema_hints: [
"frontmatter: preserve the skeleton block verbatim",
"body: replace only the prose marker; keep the structured sections intact",
+ linkCandidateSchemaHint(linkCandidates),
],
target_path: targetPath,
+ link_candidates: linkCandidates,
});
return Object.freeze({
diff --git a/src/core/brain/dream.ts b/src/core/brain/dream.ts
index ddfe2735..6e1feb46 100644
--- a/src/core/brain/dream.ts
+++ b/src/core/brain/dream.ts
@@ -75,6 +75,8 @@ import {
type ProgressCounter,
} from "./progress.ts";
import { regenerateLessonsQuiet } from "./lessons.ts";
+import { buildLinkCandidateManifest, type LinkCandidateManifest } from "./notes/link-candidates.ts";
+import { gatedOwnerScopeView } from "./owner-scope-view.ts";
import { brainDirsForWrite, dreamWorkrunPath } from "./paths.ts";
import { loadBrainConfig } from "./policy.ts";
import { buildReconcileOutcomes } from "./reconcile-outcomes.ts";
@@ -271,13 +273,21 @@ function dreamRun(
// which is why an un-opted-in vault pays no I/O here.
const salienceGate = applySalienceGate({ vault, preferences: scan.preferences, cfg, now });
+ // The wikilink targets a fired rung's envelope offers, built at most
+ // ONCE per dream run and only if a rung fires. The manifest is a whole
+ // -vault walk (a frontmatter read per note under owner-scope delivery),
+ // the ladder is planned before the `changed` gate, and the runId
+ // collision path below re-plans - so an eager build paid that walk on
+ // every dry run that fires nothing, and twice on a collision.
+ const linkCandidates = memoizedLinkCandidates(vault, opts.agentName);
+
// Count-triggered fact rollup ladder (S3): pure counters over the
// ADMITTED fact artifacts (preferences) against the persisted per-tier
// baselines. Computed before the `changed` gate so a run whose ONLY
// effect is a rollup still counts as changed; below threshold it fires
// nothing and the ledger is never written, so the run stays
// byte-identical.
- let rollupPlan = buildRollupPlan(vault, cfg, salienceGate.admitted, runId);
+ let rollupPlan = buildRollupPlan(vault, cfg, salienceGate.admitted, runId, linkCandidates);
if (!hasStateChange(plan, refresh, scan.corrupted.length, rollupPlan)) {
if (!dryRun) {
@@ -338,7 +348,7 @@ function dreamRun(
// the pre-collision runId; if the ladder corrected it, rebuild the
// plan so every target_path embeds the final run_id.
if (rollupPlan.fired && runId !== baseRunId) {
- rollupPlan = buildRollupPlan(vault, cfg, salienceGate.admitted, runId);
+ rollupPlan = buildRollupPlan(vault, cfg, salienceGate.admitted, runId, linkCandidates);
}
opts.safeguard?.checkpoint();
const handle = openWorkrun(vault, runId);
@@ -528,15 +538,39 @@ function buildRollupPlan(
cfg: BrainConfig,
factCount: number,
runId: string,
+ linkCandidates: () => LinkCandidateManifest,
): RollupLadderPlan {
return planRollupLadder({
factCount,
ledger: readRollupLedger(vault),
thresholds: resolveRollupThresholds(cfg),
runId,
+ linkCandidates,
});
}
+/**
+ * The link-candidate manifest for this run, walked at most once and only
+ * if something asks for it. Both callers of {@link buildRollupPlan} share
+ * one of these, so a runId collision re-plans off the manifest already
+ * built rather than walking the vault a second time.
+ *
+ * No query: the ladder is count-only by design and never learns which
+ * items it is folding, so ranking the candidates would be inventing a
+ * relevance nothing measured. The manifest says `alphabetical` and names
+ * the total, which is the honest form of that.
+ */
+function memoizedLinkCandidates(
+ vault: string,
+ agentName: string | undefined,
+): () => LinkCandidateManifest {
+ let manifest: LinkCandidateManifest | null = null;
+ return () =>
+ (manifest ??= buildLinkCandidateManifest(vault, {
+ visible: gatedOwnerScopeView(vault, agentName).visible,
+ }));
+}
+
function formatRunId(d: Date): string {
// <reason>-YYYY-MM-DD-HHMMSS. The prefix is the snapshot-reason
// constant rather than a literal, so the archive's filename and the
diff --git a/src/core/brain/extract-signals.ts b/src/core/brain/extract-signals.ts
index e4023ed7..cef7b188 100644
--- a/src/core/brain/extract-signals.ts
+++ b/src/core/brain/extract-signals.ts
@@ -38,7 +38,9 @@
import { posix } from "node:path";
+import { buildReconciliationReport, type ReconciliationReport } from "../reconciliation-report.ts";
import { buildCaptureBoundary, type SessionCaptureDecision } from "./capture-boundary.ts";
+import { tryRecordDeadLetter, type DeadLetterLane, type DeadLetterOutcome } from "./dead-letter.ts";
import { buildDedupIndex, computeDedupHash, type DedupIndexEntry } from "./dedup-hash.ts";
import { classifyDurability, resolveDurabilityDenylist } from "./gates/durability.ts";
import { buildNeedsLlmStep, type NeedsLlmStep } from "./llm-step.ts";
@@ -68,6 +70,13 @@ export const AUTO_EXTRACT_TARGET_DIR_REL = "Brain/inbox";
/** The needs-llm-step step name for the deferred signal mining. */
export const EXTRACT_SIGNALS_STEP = "extract-signals";
+/**
+ * This lane's dead-letter name. It is the ONE multi-artifact commit lane
+ * in the tree - one file per mined item from one validated payload - and
+ * so the only one that can leave a partial write behind.
+ */
+const EXTRACT_SIGNALS_LANE: DeadLetterLane = "extract-signals";
+
/**
* Most signals one session may yield. A transcript that genuinely states
* more than this many durable preferences is a transcript worth reading by
@@ -108,6 +117,16 @@ export class ExtractSignalsError extends Error {
* ON it, so an operator can find them, and re-running the same payload is
* safe because the dedup index recognises every one of them.
*
+ * Since nothing-writes-silently it carries two further things. The first
+ * is {@link reconciliation}: the same `attempted` / `found` / `missing`
+ * report shape the import read-back census and the embedder audit use, so
+ * this lane's accounting is not a fourth dialect of the same three
+ * numbers. The second is {@link deadLetter}: this error's own accounting
+ * lives exactly as long as the response carrying it, and a caller that
+ * drops the response must not be the only record that a partial write
+ * happened - so the accounting is ALSO on disk, and this field says
+ * where, or why it could not be.
+ *
* Not an {@link ExtractSignalsError}: that class is the caller's fault
* and the surfaces report it as such. A filesystem that refused a write
* is not.
@@ -119,29 +138,50 @@ export class ExtractSignalsWriteError extends Error {
readonly written: ReadonlyArray<ExtractedSignalWrite>;
/** Payload items this call never reached, the failing one included. */
readonly remaining: number;
+ /** Attempted / found / missing for this commit, missing keys named. */
+ readonly reconciliation: ReconciliationReport;
+ /** The error that stopped the commit, kept as thrown. */
+ readonly firstError: unknown;
+ /** Where the durable record of this partial write landed, or why not. */
+ readonly deadLetter: DeadLetterOutcome;
- constructor(
- topic: string,
- written: ReadonlyArray<ExtractedSignalWrite>,
- remaining: number,
- cause: unknown,
- ) {
+ constructor(params: {
+ readonly topic: string;
+ readonly written: ReadonlyArray<ExtractedSignalWrite>;
+ readonly remaining: number;
+ readonly reconciliation: ReconciliationReport;
+ readonly cause: unknown;
+ readonly deadLetter: DeadLetterOutcome;
+ }) {
+ const { topic, written, remaining, reconciliation, cause, deadLetter } = params;
super(
`failed to write extracted signal ${JSON.stringify(topic)}: ` +
`${cause instanceof Error ? cause.message : String(cause)} - ` +
+ `attempted ${reconciliation.attempted}, written ${reconciliation.found}, ` +
+ `failed ${reconciliation.missing.length} (${reconciliation.missing.join(", ")}); ` +
`${written.length} signal(s) are already on disk ` +
`(${written.length === 0 ? "none" : written.map((w) => w.id).join(", ")}) and ` +
- `${remaining} payload item(s) were not reached; fix the cause and re-run the ` +
- "same payload, which dedups what is already there",
+ `${remaining} payload item(s) were not reached; ${describeDeadLetter(deadLetter)}; ` +
+ "fix the cause and re-run the same payload, which dedups what is already there",
{ cause },
);
this.name = "ExtractSignalsWriteError";
this.topic = topic;
this.written = Object.freeze([...written]);
this.remaining = remaining;
+ this.reconciliation = reconciliation;
+ this.firstError = cause;
+ this.deadLetter = deadLetter;
}
}
+/** The dead-letter half of the message above, either way it went. */
+function describeDeadLetter(outcome: DeadLetterOutcome): string {
+ return outcome.recorded
+ ? `a dead letter recording this is at ${outcome.path} (${outcome.id})`
+ : `NO dead letter could be recorded for it (${outcome.reason})`;
+}
+
/** One user turn offered to the caller as mining material. */
export interface MinedTurn {
readonly turnId: string;
@@ -198,6 +238,19 @@ export interface CommitExtractedSignalsResult {
/** Items the durability gate refused; each is named in {@link rejected}. */
readonly durabilityRejected: number;
readonly rejected: ReadonlyArray<ExtractedSignalRejection>;
+ /**
+ * Write accounting in the wave's shared vocabulary: `attempted` is the
+ * items that reached the write call, `found` the ones whose bytes
+ * landed, `missing` the keys of the rest. On this path `missing` is
+ * always empty - a failure throws {@link ExtractSignalsWriteError},
+ * which carries the same shape with the gap named.
+ *
+ * A deduped or durability-rejected item is NOT attempted: both are
+ * deliberate refusals this result already names in {@link deduped} and
+ * {@link rejected}, and counting a refusal the lane made as a write it
+ * lost would misreport the one number an operator reads first.
+ */
+ readonly reconciliation: ReconciliationReport;
}
export interface PlanExtractSignalsOptions {
@@ -454,7 +507,36 @@ export function commitExtractedSignals(
targetDir !== undefined ? { targetDir } : {},
);
} catch (err) {
- throw new ExtractSignalsWriteError(item.topic, written, items.length - index, err);
+ // Everything from the failing item onward is unwritten. Some of the
+ // items this call never reached might have deduped away had it got
+ // to them - unknowable from here, and "not written" is true of all
+ // of them either way, so they are NAMED rather than guessed at.
+ const remaining = items.slice(index);
+ const reconciliation = buildReconciliationReport({
+ attempted: written.length + remaining.length,
+ found: written.length,
+ missing: remaining.map((unwritten, offset) =>
+ unwrittenKey(index + offset, unwritten.topic),
+ ),
+ });
+ throw new ExtractSignalsWriteError({
+ topic: item.topic,
+ written,
+ remaining: remaining.length,
+ reconciliation,
+ cause: err,
+ deadLetter: tryRecordDeadLetter(vault, {
+ envelope: {
+ lane: EXTRACT_SIGNALS_LANE,
+ step: EXTRACT_SIGNALS_STEP,
+ reference: trimmed,
+ target: AUTO_EXTRACT_TARGET_DIR_REL,
+ },
+ report: reconciliation,
+ firstError: err,
+ now: opts.now,
+ }),
+ });
}
dedup.set(hash, { id: res.id, path: res.path });
written.push(Object.freeze({ id: res.id, path: res.path, topic: item.topic }));
@@ -467,5 +549,20 @@ export function commitExtractedSignals(
deduped,
durabilityRejected: rejected.length,
rejected: Object.freeze(rejected),
+ reconciliation: buildReconciliationReport({
+ attempted: written.length,
+ found: written.length,
+ missing: [],
+ }),
});
}
+
+/**
+ * The key a missing item is NAMED by: its payload index and its topic.
+ * The index carries the uniqueness (two items may share a topic) and the
+ * topic carries the meaning, which is what an operator matches against
+ * the payload they are about to re-run.
+ */
+function unwrittenKey(index: number, topic: string): string {
+ return `items[${index}]:${topic}`;
+}
diff --git a/src/core/brain/import-census.ts b/src/core/brain/import-census.ts
new file mode 100644
index 00000000..b22d4244
--- /dev/null
+++ b/src/core/brain/import-census.ts
@@ -0,0 +1,143 @@
+/**
+ * Post-import read-back census (nothing-writes-silently, Unit F).
+ *
+ * Every import surface in this build reported what it BELIEVED it had done:
+ * `signals_created: 40`, `ingested: [...]`. Nothing ever went back to the
+ * vault afterwards and asked whether those forty items are there. When a
+ * write is lost - a crash between the write and the fsync, a sync conflict
+ * that resolved the other way, a directory an operator cleaned - the counter
+ * still says forty, and the loss is invisible until somebody misses the note.
+ *
+ * This module is that second look. It re-reads the vault through the keys the
+ * import ALREADY uses, and reports `attempted / found / missing` in the wave's
+ * shared reconciliation vocabulary ({@link ../reconciliation-report.ts}) -
+ * which is what makes `missing` a list of NAMED keys rather than a number.
+ * A count on its own is the misleading no-op this unit removes: "40 attempted,
+ * 38 found" reads as accounted for while nobody can say which two vanished.
+ *
+ * ## No new index, no new hash
+ *
+ * Both lanes read back over the key the lane already keys on:
+ *
+ * - sessions: the signal dedup hash ({@link ./dedup-hash.ts}), read back
+ * through the same `buildDedupIndex` walk of `Brain/inbox/` and
+ * `Brain/inbox/processed/` that the import consults before every write.
+ * A hash the index cannot find is a signal that is not on disk.
+ * - ingest: the content-hash manifest ({@link ./ingest/content-manifest.ts}),
+ * read back through the same `classifyPaths`. A claimed path counts as
+ * found only when the manifest recorded it AND the bytes on disk still
+ * hash to what was recorded - anything else is a claim nobody can
+ * reproduce.
+ *
+ * ## What the sessions census does and does not cover
+ *
+ * It covers the signals the import wrote through its own `writeSignal` call,
+ * which always land in `Brain/inbox/`. It does NOT cover extracted facts:
+ * those route through `routeExtractedFacts`, whose write target moves to
+ * `Brain/pending/` when the write-approval gate is on, and a read-back over
+ * the inbox alone would report every staged fact as lost. Naming the scope is
+ * the point - a census that quietly covered two thirds of a run would be the
+ * same lie in a smaller font.
+ *
+ * Read-only and idempotent: nothing here writes a byte.
+ */
+
+import { canonicalNotePath } from "../path-safety.ts";
+import { buildDedupIndex } from "./dedup-hash.ts";
+import { classifyPaths, readManifest } from "./ingest/content-manifest.ts";
+import {
+ buildReconciliationReport,
+ deriveReconciliationOutcome,
+ type ReconciliationOutcome,
+} from "../reconciliation-report.ts";
+
+/**
+ * One import's read-back result, flattened for the result payloads it rides
+ * on. `missing` names its keys; there is no constructor here that takes a
+ * bare count instead.
+ */
+export interface ImportCensus {
+ /** Distinct keys the import claims to have written. */
+ readonly attempted: number;
+ /** Claimed keys a fresh read of the vault can still find. */
+ readonly found: number;
+ /** Every claimed key the read-back could NOT find, named and sorted. */
+ readonly missing: ReadonlyArray<string>;
+ /** `complete` when nothing is missing, `partial` otherwise. */
+ readonly outcome: ReconciliationOutcome;
+}
+
+/**
+ * Build a census from a claim list and a presence test. `claimed` is deduped
+ * first: a key claimed twice is one claim, and counting it twice would inflate
+ * `attempted` into a number no read-back could ever satisfy.
+ */
+export function buildReadBackCensus(
+ claimed: ReadonlyArray<string>,
+ present: (key: string) => boolean,
+): ImportCensus {
+ const keys = [...new Set(claimed)];
+ const missing: string[] = [];
+ let found = 0;
+ for (const key of keys) {
+ if (present(key)) found++;
+ else missing.push(key);
+ }
+ // Sorted so the finding reads the same whatever order the run happened to
+ // claim its keys in - a report an operator diffs across runs.
+ missing.sort();
+ const report = buildReconciliationReport({ attempted: keys.length, found, missing });
+ return Object.freeze({
+ attempted: report.attempted,
+ found: report.found,
+ missing: report.missing,
+ outcome: deriveReconciliationOutcome(report),
+ });
+}
+
+/**
+ * Sessions lane: read back the dedup hashes of the signals an import claims to
+ * have written. The index is built FRESH from disk here - the map the import
+ * mutated as it ran is its own belief, and asking a belief to confirm itself
+ * measures nothing.
+ */
+export function censusSessionSignals(
+ vault: string,
+ claimedHashes: ReadonlyArray<string>,
+): ImportCensus {
+ if (claimedHashes.length === 0) return buildReadBackCensus([], () => true);
+ const index = buildDedupIndex(vault);
+ return buildReadBackCensus(claimedHashes, (hash) => index.has(hash));
+}
+
+/**
+ * Ingest lane: read back the content-manifest entries of the sources an
+ * ingest claims to have completed. Found means `unchanged` - recorded, still
+ * on disk, and still hashing to the recorded digest. A path that is `new`
+ * (never recorded), `modified` (recorded hash no longer matches) or `missing`
+ * (gone from disk) is a claim the manifest cannot confirm.
+ */
+export function censusIngestedPaths(
+ vault: string,
+ claimedPaths: ReadonlyArray<string>,
+): ImportCensus {
+ if (claimedPaths.length === 0) return buildReadBackCensus([], () => true);
+ // Canonical on both sides: `classifyPaths` reports canonical keys, so a
+ // claim spelled any other way would never match its own classification.
+ const keys = [...new Set(claimedPaths.map((p) => canonicalNotePath(p)))];
+ const confirmed = new Set(classifyPaths(vault, keys, readManifest(vault)).unchanged);
+ return buildReadBackCensus(keys, (path) => confirmed.has(path));
+}
+
+/**
+ * The one wire shape both lanes emit, so a CLI reading and an MCP reading of
+ * the same census cannot drift into two spellings of the same finding.
+ */
+export function serializeImportCensus(census: ImportCensus): Record<string, unknown> {
+ return {
+ attempted: census.attempted,
+ found: census.found,
+ missing: [...census.missing],
+ outcome: census.outcome,
+ };
+}
diff --git a/src/core/brain/ingest/checkpoint.ts b/src/core/brain/ingest/checkpoint.ts
index 335c1ccf..48501142 100644
--- a/src/core/brain/ingest/checkpoint.ts
+++ b/src/core/brain/ingest/checkpoint.ts
@@ -19,31 +19,46 @@
* read inert - the deterministic-test escape hatch mirroring upstream graphify's
* `GRAPHIFY_NO_INCREMENTAL_CACHE`.
*
+ * The mechanism below the record - the id validation, the location, the
+ * schema-version refusal, the lock / identity / atomic write - lives in
+ * {@link ../checkpoint-store.ts}, shared with the session-import lane. What
+ * stays here is this lane's record: a plan id, a source dir, and a set of
+ * completed paths that no-ops when the SET is unchanged.
+ *
* Language-agnostic: keys are canonical vault-relative paths and content hashes;
* no natural-language content is inspected.
*/
import { createHash } from "node:crypto";
-import { existsSync, readFileSync, rmSync } from "node:fs";
-import { join } from "node:path";
-import { atomicWriteFileSync } from "../../fs-atomic.ts";
import { canonicalNotePath } from "../../path-safety.ts";
-import { acquireLockSyncWithRetry } from "../sync-lockfile.ts";
+import {
+ checkpointFilePath,
+ checkpointingEnabled,
+ NO_CHECKPOINT_ENV,
+ readCheckpointObject,
+ removeCheckpointFile,
+ withCheckpointLock,
+ writeCheckpointObject,
+} from "../checkpoint-store.ts";
import { isoSecond } from "../time.ts";
-import { assertVaultIdentityForWrite } from "../vault-identity.ts";
/** Only schema version currently understood. Unknown versions are refused. */
const SCHEMA_VERSION = 1 as const;
-/** Vault-relative directory holding OSB machine artifacts (not curated memory). */
-const MACHINE_ARTIFACT_DIR = ".open-second-brain";
-
/** Subdirectory holding one JSON checkpoint per batch plan. */
const CHECKPOINT_DIR = "ingest-checkpoints";
-/** Env var that, when truthy, disables checkpoint reads and writes. */
-export const NO_CHECKPOINT_ENV = "OSB_INGEST_NO_CHECKPOINT";
+/**
+ * How a refusal names this lane's id and its file. Exported because the
+ * boundaries that accept a caller-supplied plan id validate it against the
+ * shared store's grammar and must name the same thing this lane does.
+ */
+export const PLAN_ID_LABEL = "plan";
+const ID_LABEL = PLAN_ID_LABEL;
+const FILE_LABEL = "ingest checkpoint";
+
+export { checkpointingEnabled, NO_CHECKPOINT_ENV };
/** The persisted per-plan checkpoint. */
export interface IngestCheckpoint {
@@ -57,18 +72,6 @@ export interface IngestCheckpoint {
readonly updated_at: string;
}
-/**
- * Whether checkpointing is active. Off only when {@link NO_CHECKPOINT_ENV} holds
- * a truthy value; the empty string, `0`, and `false` are all treated as unset so
- * an accidentally-exported empty var does not silently disable resumability.
- */
-export function checkpointingEnabled(): boolean {
- const raw = process.env[NO_CHECKPOINT_ENV];
- if (raw === undefined) return true;
- const v = raw.trim().toLowerCase();
- return v === "" || v === "0" || v === "false";
-}
-
/**
* Deterministic plan id: a short SHA-256 hex over the canonical source dir and
* the sorted full discovered path set. Keying on the FULL set (not the remaining
@@ -87,16 +90,9 @@ export function computePlanId(sourceDir: string, discoveredPaths: readonly strin
return hash.digest("hex").slice(0, 16);
}
-function assertPlanId(planId: string): void {
- if (!/^[0-9a-f]{6,64}$/.test(planId)) {
- throw new Error(`invalid plan id (expected lowercase hex): ${JSON.stringify(planId)}`);
- }
-}
-
/** Absolute path of one plan's checkpoint file. */
export function checkpointPath(vault: string, planId: string): string {
- assertPlanId(planId);
- return join(vault, MACHINE_ARTIFACT_DIR, CHECKPOINT_DIR, `${planId}.json`);
+ return checkpointFilePath(vault, CHECKPOINT_DIR, ID_LABEL, planId);
}
function serialize(cp: IngestCheckpoint): string {
@@ -123,22 +119,8 @@ function serialize(cp: IngestCheckpoint): string {
export function readCheckpoint(vault: string, planId: string): IngestCheckpoint | null {
if (!checkpointingEnabled()) return null;
const path = checkpointPath(vault, planId);
- if (!existsSync(path)) return null;
- let parsed: unknown;
- try {
- parsed = JSON.parse(readFileSync(path, "utf8"));
- } catch (e) {
- throw new Error(`ingest checkpoint is corrupted JSON: ${path}`, { cause: e });
- }
- if (parsed === null || typeof parsed !== "object" || Array.isArray(parsed)) {
- throw new Error(`ingest checkpoint is not an object: ${path}`);
- }
- const obj = parsed as Record<string, unknown>;
- if (obj["schema_version"] !== SCHEMA_VERSION) {
- throw new Error(
- `ingest checkpoint schema_version ${String(obj["schema_version"])} not supported (expected ${SCHEMA_VERSION}): ${path}`,
- );
- }
+ const obj = readCheckpointObject(path, FILE_LABEL, SCHEMA_VERSION);
+ if (obj === null) return null;
const rawCompleted = obj["completed"];
const completed = Array.isArray(rawCompleted)
? rawCompleted
@@ -182,11 +164,8 @@ export function recordCompleted(
now: Date,
): boolean {
if (!checkpointingEnabled()) return false;
- // Vault-identity write guard (context-integrity-gates, Unit J).
- assertVaultIdentityForWrite(vault);
const path = checkpointPath(vault, planId);
- const handle = acquireLockSyncWithRetry(path);
- try {
+ return withCheckpointLock(path, () => {
const prev = readCheckpoint(vault, planId);
const merged = new Set<string>(prev?.completed ?? []);
for (const p of paths) merged.add(canonicalNotePath(p));
@@ -199,11 +178,8 @@ export function recordCompleted(
completed,
updated_at: isoSecond(now),
};
- atomicWriteFileSync(path, serialize(next));
- return true;
- } finally {
- handle.release();
- }
+ return writeCheckpointObject(vault, path, serialize(next));
+ });
}
/**
@@ -212,10 +188,5 @@ export function recordCompleted(
* existed.
*/
export function clearCheckpoint(vault: string, planId: string): boolean {
- // Vault-identity write guard (context-integrity-gates, Unit J).
- assertVaultIdentityForWrite(vault);
- const path = checkpointPath(vault, planId);
- if (!existsSync(path)) return false;
- rmSync(path);
- return true;
+ return removeCheckpointFile(vault, checkpointPath(vault, planId));
}
diff --git a/src/core/brain/ingest/ingest.ts b/src/core/brain/ingest/ingest.ts
index bba412d7..60a41d2f 100644
--- a/src/core/brain/ingest/ingest.ts
+++ b/src/core/brain/ingest/ingest.ts
@@ -24,6 +24,7 @@ import { dirname, join, relative } from "node:path";
import type { FrontmatterMap } from "../../types.ts";
import { canonicalNotePath } from "../../path-safety.ts";
+import { assertCheckpointId } from "../checkpoint-store.ts";
import {
formatFrontmatter,
parseFrontmatter,
@@ -41,7 +42,7 @@ import {
sourceIdentityHash,
type Provenance,
} from "../provenance/provenance.ts";
-import { recordCompleted } from "./checkpoint.ts";
+import { PLAN_ID_LABEL, recordCompleted } from "./checkpoint.ts";
import { updateManifest } from "./content-manifest.ts";
import { preExtractCodeStructure, type PreExtractResult } from "./pre-extract.ts";
@@ -65,6 +66,13 @@ export interface IngestSourceOptions {
* source records the source into that plan's resume checkpoint
* (union-as-you-go). The content manifest stays the authoritative final
* state; the checkpoint only tracks plan progress. Absent → no checkpoint.
+ *
+ * Held to the checkpoint store's id grammar (lowercase hex - the id becomes
+ * a filename) and REFUSED here, before any write, when it is not. The
+ * checkpoint write below is best-effort, and a caller-supplied id that can
+ * never work is not the transient fault that tolerance is for: swallowed,
+ * it left every source ingested with no plan progress recorded at all and
+ * `--reconcile` reporting the whole batch as never ingested.
*/
readonly planId?: string;
/**
@@ -115,6 +123,12 @@ export function ingestSource(
): IngestSourceResult {
// Vault-identity write guard (context-integrity-gates, Unit J).
assertVaultIdentityForWrite(vault);
+ // Before anything is written: an id the checkpoint store cannot use is a
+ // resume that will never engage, and the caller has to hear it now rather
+ // than discover it from a reconciliation report.
+ if (opts.planId !== undefined && opts.planId.length > 0) {
+ assertCheckpointId(PLAN_ID_LABEL, opts.planId);
+ }
// The SAME normaliser the trust classifier uses. This used to be a bare
// `canonicalNotePath`, which left a caller-supplied `[[Articles/x.md]]`
// wrapped: the summary page then wrote `[[[[Articles/x.md]]]]` and keyed on
@@ -200,7 +214,11 @@ export function ingestSource(
try {
recordCompleted(vault, opts.planId, dirname(canonicalSource), [canonicalSource], opts.now);
} catch {
- // Checkpointing is a resumability optimization, not correctness.
+ // Checkpointing is a resumability optimization, not correctness -
+ // and what is tolerated here is now only the TRANSIENT half: a lock
+ // this run could not take, a write that failed. The permanent,
+ // caller-caused half (an id the store cannot use) was refused at the
+ // top of this function, before the ingest wrote anything.
}
}
}
diff --git a/src/core/brain/ingest/reconcile.ts b/src/core/brain/ingest/reconcile.ts
index c9af26a3..f70e69e8 100644
--- a/src/core/brain/ingest/reconcile.ts
+++ b/src/core/brain/ingest/reconcile.ts
@@ -19,8 +19,18 @@
* `unchanged` (already ingested in a prior run of the same plan). Reconciling
* that union against the checkpoint means a plan re-run after a partial ingest
* still names exactly the sources that never landed.
+ *
+ * Read-back census (nothing-writes-silently, Unit F). The gap above is
+ * computed from what the CHECKPOINT says - a record of the run's own claims.
+ * A claim is not evidence: a source recorded as completed whose bytes are no
+ * longer on disk, or whose content hash no longer matches what the manifest
+ * recorded, is a completed ingest nobody can reproduce. The census re-reads
+ * the manifest for exactly the sources this report calls `ingested` and names
+ * the ones that do not read back. Still read-only, still idempotent - it
+ * hashes files it is already reporting on and writes nothing.
*/
+import { censusIngestedPaths, type ImportCensus } from "../import-census.ts";
import type { BatchPlan } from "./batch-plan.ts";
import { readCheckpoint } from "./checkpoint.ts";
@@ -36,6 +46,13 @@ export interface ReconcileReport {
readonly missing: readonly string[];
/** True when the gap is empty (every dispatched source ingested). */
readonly complete: boolean;
+ /**
+ * Read-back census over {@link ReconcileReport.ingested}: of the sources
+ * this report calls ingested, how many the content manifest can still
+ * confirm, and the named paths it cannot. `complete` above answers "did
+ * everything dispatched get claimed"; this answers "is every claim true".
+ */
+ readonly census: ImportCensus;
}
/**
@@ -75,5 +92,6 @@ export function reconcilePlan(vault: string, plan: BatchPlan): ReconcileReport {
ingested,
missing,
complete: missing.length === 0,
+ census: censusIngestedPaths(vault, ingested),
};
}
diff --git a/src/core/brain/log.ts b/src/core/brain/log.ts
index 4586821e..c807a80d 100644
--- a/src/core/brain/log.ts
+++ b/src/core/brain/log.ts
@@ -23,6 +23,11 @@
* log is append-only. Atomicity is achieved by reading the
* existing contents, appending the new block, and writing the
* result back through `fs-atomic` in one shot.
+ *
+ * Every appended event carries an `origin_channel` bullet beside the
+ * `agent` one: the server-derived channel of the process that wrote it
+ * (`src/core/origin-channel.ts`). Events written before that shipped
+ * carry no such bullet and are never rewritten to add one.
*/
import { existsSync, mkdirSync, readFileSync } from "node:fs";
@@ -38,6 +43,7 @@ import {
validateIsoDate,
} from "./paths.ts";
import { isValidDeviceId, resolveDeviceId } from "../config.ts";
+import { ORIGIN_CHANNEL_FIELD, originChannelStamp } from "../origin-channel.ts";
import { BRAIN_LOG_EVENT_KIND, BRAIN_LOG_EVENT_KIND_SET, type BrainLogEventKind } from "./types.ts";
// ----- Public types ---------------------------------------------------------
@@ -311,11 +317,24 @@ export function appendLogEvent(
const jsonlPath = logShardJsonlPath(vault, ts.date, deviceId);
const logDir = brainDirsForWrite(vault).log;
const topLevelAgent = (event as { agent?: unknown }).agent;
- const eventBody =
+ const withAgent =
typeof topLevelAgent === "string" && typeof event.body["agent"] !== "string"
- ? Object.freeze({ ...event.body, agent: topLevelAgent })
- : event.body;
- const diskEvent: BrainLogEntry = eventBody === event.body ? event : { ...event, body: eventBody };
+ ? { ...event.body, agent: topLevelAgent }
+ : { ...event.body };
+ // The server-derived origin channel (Unit C), folded into the body the
+ // same way `agent` is so both surfaces this appender writes carry it
+ // and `parseLogDay` reads it back with no parser change.
+ //
+ // Assigned LAST and unconditionally: `agent` yields to a body key
+ // because the caller may legitimately name a different agent for the
+ // event than for the call, but the channel is derived from the process
+ // and a body that spells it is a caller naming its own provenance -
+ // the one thing this field exists not to be.
+ const eventBody: BrainLogEntryPayload = Object.freeze({
+ ...withAgent,
+ [ORIGIN_CHANNEL_FIELD]: originChannelStamp(),
+ });
+ const diskEvent: BrainLogEntry = { ...event, body: eventBody };
// §23 (v0.10.8): each event lands in both `<date>.jsonl` (machine
// surface, primary for `readLogDay`) and `<date>.md` (human-facing
diff --git a/src/core/brain/notes/blank-overwrite-guard.ts b/src/core/brain/notes/blank-overwrite-guard.ts
new file mode 100644
index 00000000..eed1a6f8
--- /dev/null
+++ b/src/core/brain/notes/blank-overwrite-guard.ts
@@ -0,0 +1,88 @@
+/**
+ * The blank-overwrite predicate (nothing-writes-silently, unit B).
+ *
+ * A note update carries a replacement body. When that body is empty, the
+ * write is indistinguishable at the byte level from "delete everything
+ * this page says" - and until this guard, that is exactly what happened,
+ * reported as `updated: true`. An agent that composed an empty string
+ * (a truncated generation, a variable that never got assigned, a
+ * template that rendered to nothing) emptied the page and was told the
+ * write succeeded.
+ *
+ * The rule is one line of logic and the whole of this module: replacing
+ * a body that carries text with one that does not is refused unless the
+ * caller SAYS it means to clear the page. Deleting a page's contents is
+ * a legitimate thing to want; doing it by accident and being told it
+ * worked is not.
+ *
+ * ## What counts as blank
+ *
+ * `trim().length === 0`. Whitespace-only is blank, because
+ * `parseFrontmatter` trims the body it reads
+ * (`vault.ts` - `text.slice(match[0].length).trim()`), so a page whose
+ * body is three spaces reads back as an empty body on the next pass:
+ * the bytes differ, the note does not. The same predicate already
+ * governs `projectAppendNote`'s empty-content refusal, so the two note
+ * writers agree on what "nothing" is rather than each deciding.
+ *
+ * A character that is not whitespace is content, however invisible -
+ * U+200B ZERO WIDTH SPACE survives `trim` and therefore survives this
+ * guard. The predicate deliberately does not invent a wider notion of
+ * emptiness than the parser has: a guard stricter than the format it
+ * protects would refuse writes the format would have kept.
+ *
+ * An existing body that is ALREADY blank has nothing to lose, so
+ * blank-over-blank is not a refusal. A brand-new note is not an update
+ * and never reaches here: `createNote` resolves an absent body to `""`
+ * and writes a genuinely empty page, which is a page nobody had.
+ *
+ * ## Where this is deliberately NOT wired
+ *
+ * One seam: `projectUpdateNote` in `write-batch.ts`. That single call
+ * covers both callers - `brain_update_note` (a one-operation batch) and
+ * `brain_write_batch`'s `update_note` op. There is no third. The three
+ * other write surfaces a reader might expect to find it on were
+ * examined and refused, so nobody re-opens the question:
+ *
+ * - `core/fs-atomic.ts` writes BYTES, not notes. It is the substrate
+ * under JSON state, manifests, JSONL shards, sqlite sidecars and
+ * install adapters across most of the tree; it has no concept of a
+ * body, and a "the new content is shorter" heuristic there would
+ * refuse legitimate writes of every other file format.
+ * - `core/brain/host-memory-write.ts` is append-only into a continuity
+ * record and already refuses empty content with a typed
+ * `empty_content` error. It replaces nothing, so there is nothing to
+ * blank.
+ * - `core/brain/inline.ts` is a parser and writes nothing at all;
+ * `inline-rewrite.ts` is the writer beside it and is line-preserving
+ * and insert-only, returning early when the text is unchanged. It
+ * cannot empty a body either.
+ */
+
+/** Inputs to {@link refuseBlankOverwrite}. */
+export interface BlankOverwriteInput {
+ /** The body currently on disk, as the frontmatter parser returned it. */
+ readonly existingBody: string;
+ /** The body the caller wants written in its place. */
+ readonly nextBody: string;
+ /** The caller explicitly asked to clear the page. Defaults to false. */
+ readonly allowEmpty?: boolean;
+}
+
+/** Whether `body` says nothing the format would preserve. */
+function isBlank(body: string): boolean {
+ return body.trim().length === 0;
+}
+
+/**
+ * Whether this update must be refused: a blank replacement over a body
+ * that carries text, without an explicit `allowEmpty`.
+ *
+ * Pure - it reads no file and reaches nothing outside its argument, so
+ * the caller decides what to do with the answer and owns the error type
+ * its own surface speaks.
+ */
+export function refuseBlankOverwrite(input: BlankOverwriteInput): boolean {
+ if (input.allowEmpty === true) return false;
+ return isBlank(input.nextBody) && !isBlank(input.existingBody);
+}
diff --git a/src/core/brain/notes/create-note.ts b/src/core/brain/notes/create-note.ts
index deea4350..6fc56357 100644
--- a/src/core/brain/notes/create-note.ts
+++ b/src/core/brain/notes/create-note.ts
@@ -48,6 +48,11 @@ import { dirname, join, posix, sep, win32 } from "node:path";
import type { FrontmatterMap } from "../../types.ts";
import { ensureInsideVault } from "../../path-safety.ts";
+import {
+ ORIGIN_CHANNEL_FIELD,
+ originChannelStamp,
+ type OriginChannelStamp,
+} from "../../origin-channel.ts";
import { isFileAlreadyExists } from "../../fs-atomic.ts";
import { formatFrontmatter, writeFrontmatterAtomic } from "../../vault.ts";
import {
@@ -173,6 +178,17 @@ export interface ResolvedNoteTarget {
readonly relPath: string;
/** Absolute filesystem path, guaranteed inside the vault. */
readonly abs: string;
+ /**
+ * Server-derived channel of the process resolving this target (Unit C).
+ * Resolved HERE, in the one envelope all four caller-named note tools
+ * share, so no arm can reach a write without having been handed the
+ * channel, and no arm can be given a different one.
+ *
+ * It is a resolution output, not an input: nothing on
+ * {@link CreateNoteInput} names it, and `src/core/origin-channel.ts`
+ * says why a caller-nameable channel would be worthless.
+ */
+ readonly originChannel: OriginChannelStamp;
}
/**
@@ -371,7 +387,7 @@ export function resolveNoteTarget(vault: string, path: string): ResolvedNoteTarg
} catch (err) {
throw new CreateNoteError("outside_vault", err instanceof Error ? err.message : String(err));
}
- return { relPath, abs };
+ return { relPath, abs, originChannel: originChannelStamp() };
}
/**
@@ -460,13 +476,33 @@ export function createNote(vault: string, input: CreateNoteInput): CreateNoteRes
// backs `brain_create_note`, `brain_append_note`, and
// `brain_update_note` - the headline note writers.
assertVaultIdentityForWrite(vault);
- const { relPath, abs } = resolveNoteTarget(vault, input.path);
- const frontmatter = input.frontmatter ?? {};
+ const { relPath, abs, originChannel } = resolveNoteTarget(vault, input.path);
+ const callerFrontmatter = input.frontmatter ?? {};
const body = resolveBody(input);
// Validation judges the INPUT, so it runs before the target is
// consulted: a caller must not learn that its document is invalid
// only on the runs where the path happened to be free.
- if (input.strict === true) assertValidDocument(vault, frontmatter, body);
+ //
+ // It also runs before the origin-channel stamp is merged, and that
+ // ordering is load-bearing rather than incidental: the validator's
+ // `frontmatter-missing` violation asks whether the CALLER wrote a
+ // frontmatter block, and a server-derived key merged first would
+ // satisfy it on every document, silently retiring a refusal that
+ // exists.
+ if (input.strict === true) assertValidDocument(vault, callerFrontmatter, body);
+
+ // The server-derived stamp (Unit C) goes on LAST, so a caller that
+ // supplied a key of the same name loses to the process that actually
+ // wrote the note.
+ //
+ // Stamped on CREATION only. `update_note` and `append_note` rewrite a
+ // note this call did not author, and re-stamping there would overwrite
+ // the creating channel with the mutating one - the same objection that
+ // refuses backfilling the field onto existing records.
+ const frontmatter: FrontmatterMap = {
+ ...callerFrontmatter,
+ [ORIGIN_CHANNEL_FIELD]: originChannel,
+ };
const skipOccupied = input.ifExists === "skip";
// Return before mkdirSync so a no-op leaves no parent directories
diff --git a/src/core/brain/notes/link-candidates.ts b/src/core/brain/notes/link-candidates.ts
new file mode 100644
index 00000000..297ca6f3
--- /dev/null
+++ b/src/core/brain/notes/link-candidates.ts
@@ -0,0 +1,227 @@
+/**
+ * The link-candidate manifest carried by note-producing envelopes
+ * (nothing-writes-silently, unit E).
+ *
+ * Three of the six needs-llm-step lanes ask the calling agent for a note
+ * that CITES the vault - the rollup ladder ("cite the items you fold
+ * in"), the diarization profile, the design note. All three hand over a
+ * prompt and no way to know which wikilink targets exist, so every link
+ * the agent writes is a guess, and a guess that misses lands a dangling
+ * link in a note nobody will re-read.
+ *
+ * This module is the answer they share: a bounded list of the basenames a
+ * `[[wikilink]]` can actually resolve to, built from
+ * {@link listVaultNotePaths} - the cheap readdir-only walker with
+ * Obsidian's own resolution semantics. NOTHING here opens the search
+ * index or contacts an embedding provider: which notes exist is an
+ * existence question, and existence is answerable from directory entries
+ * alone. `tests/core/brain/notes/link-candidates.test.ts` walks this
+ * module's whole import closure to keep that true. (The visibility
+ * predicate a lane hands in does read page frontmatter - see below - but
+ * it is the LANE's dependency, which is the point of asking for it.)
+ *
+ * ## The manifest is not the field it carries
+ *
+ * A bare array of names would be the misleading shape this wave removes:
+ * a forty-name list from a five-hundred-note vault reads exactly like a
+ * complete one. So the manifest reports the TOTAL it measured beside the
+ * subset it can carry, says whether it truncated, and names how the
+ * subset was chosen. {@link linkCandidateSchemaHint} turns those numbers
+ * into the one constraint line the envelope's `schema_hints` carries -
+ * the DATA stays in `link_candidates`, because `schema_hints` is the
+ * constraint channel and smuggling a payload into it makes both
+ * unreadable.
+ *
+ * ## A candidate list is a disclosure, so it is filtered
+ *
+ * The basenames of `Brain/preferences/`, `Brain/retired/`, the inbox and
+ * the entity registry are exactly the ids the ownership rule governs, so
+ * an unfiltered manifest hands one agent the id of another agent's
+ * private memory - the leak `a-label-is-not-a-boundary` closed on
+ * fourteen report surfaces. This module therefore ASKS rather than
+ * deciding: {@link LinkCandidateOptions.visible} is required and has no
+ * default, and each lane binds it to the vault's one ownership rule
+ * through `ownerScopeView`. Keeping the rule out of here is deliberate -
+ * that adapter reaches the search layer, and a manifest that imported it
+ * would no longer be the leaf the zero-embed proof rests on.
+ *
+ * A withheld basename is dropped BEFORE `total` is counted, following the
+ * IDENTICAL TO ABSENT convention every owner-filtered surface obeys: a
+ * total that included hidden pages would tell the caller how many
+ * memories it may not see, which is the existence leak by arithmetic.
+ *
+ * ## Why the ranking is token overlap and not similarity
+ *
+ * The vault's own deterministic matcher (`tokenise`/`jaccard`, the same
+ * pair `findSimilarDecisions` ranks with) is reused rather than
+ * re-invented, so a lane that ranks candidates reaches them the same way
+ * every other surface reaches related records - and no model is consulted
+ * to decide relevance. The one adjustment is that basenames are SLUGS:
+ * the shared tokeniser deliberately keeps hyphens (Brain principles are
+ * prose, and `fail-closed` is one word there), which would make
+ * `ada-lovelace` share no token with the query `Ada Lovelace`. Both sides
+ * are split on the slug separators here, and only here.
+ */
+
+import { pathStem } from "../../fs-utils.ts";
+import { jaccard, tokenise } from "../similarity.ts";
+import { listVaultNotePaths } from "../../vault.ts";
+
+/**
+ * How many candidates an envelope may carry.
+ *
+ * Bounded by the MCP tool-result preview budget rather than by taste:
+ * `brain_diarize` and its siblings return their envelope through a
+ * 2000-character budget, above which the whole result is parked in the
+ * artifact store and the caller gets a preview instead. Forty basenames
+ * is a kilobyte at most, so adding candidates never pushes a lane's own
+ * answer out of the response - and a vault larger than that is reported
+ * as truncated rather than trimmed in silence.
+ */
+export const LINK_CANDIDATE_LIMIT = 40;
+
+/** How the carried subset was chosen. */
+export type LinkCandidateSelection =
+ /** The vault fits inside the bound; the list is every name there is. */
+ | "all"
+ /** Over the bound with a query: the subset is the highest token overlap. */
+ | "relevance"
+ /** Over the bound with no query: the subset is the first N by name. */
+ | "alphabetical";
+
+/** Wikilink targets an envelope offers the calling agent, and their bound. */
+export interface LinkCandidateManifest {
+ /** Note basenames a `[[wikilink]]` resolves to, in selection order. */
+ readonly candidates: ReadonlyArray<string>;
+ /**
+ * DISTINCT visible basenames the walk found, before the bound - not a
+ * note count: two notes in different folders share one basename, and
+ * one a caller may not see is not in it at all.
+ */
+ readonly total: number;
+ /** True when {@link candidates} is a subset of what {@link total} counts. */
+ readonly truncated: boolean;
+ readonly selection: LinkCandidateSelection;
+}
+
+export interface LinkCandidateOptions {
+ /**
+ * May the caller see the note at this VAULT-RELATIVE PATH? Required,
+ * with no default: a manifest built without asking would offer another
+ * owner's private artifact as a citation target. Lanes bind it to
+ * `ownerScopeView(vault, scope).visible`, which is a no-op comparison
+ * on every vault that has not enabled owner-scope delivery.
+ *
+ * The path, not the basename: the ownership rule reads a page's own
+ * frontmatter, and a bare basename is only resolvable for the handful
+ * of `Brain/` trees whose files are id-addressable. An owner-tagged
+ * note under `notes/` would pass a basename check that found no
+ * artifact of that id and fail open, which is the leak by a longer
+ * route.
+ */
+ readonly visible: (relPath: string) => boolean;
+ /**
+ * What the note is about. Used ONLY to rank an over-bound vault; it
+ * never filters, so a query that matches nothing still yields the
+ * bound's worth of candidates.
+ */
+ readonly query?: string;
+ /** Override for {@link LINK_CANDIDATE_LIMIT}; must be a positive integer. */
+ readonly limit?: number;
+}
+
+/** A candidate manifest was asked for with a bound that cannot hold one. */
+export class LinkCandidateError extends Error {
+ constructor(message: string) {
+ super(message);
+ this.name = "LinkCandidateError";
+ }
+}
+
+/**
+ * Build one manifest for `vault`. Read-only and deterministic: the same
+ * vault and the same query give the same list, in the same order.
+ */
+export function buildLinkCandidateManifest(
+ vault: string,
+ opts: LinkCandidateOptions,
+): LinkCandidateManifest {
+ const limit = opts.limit ?? LINK_CANDIDATE_LIMIT;
+ if (!Number.isInteger(limit) || limit <= 0) {
+ throw new LinkCandidateError(
+ `link candidates: limit must be a positive integer, got ${String(limit)}`,
+ );
+ }
+
+ // Basename, from the path the visibility question was asked about. Two
+ // notes in different folders can share one - and a wikilink cannot tell
+ // them apart either, so the manifest offers the name once.
+ const all = [
+ ...new Set(
+ listVaultNotePaths(vault)
+ .filter((rel) => opts.visible(rel))
+ .map(pathStem),
+ ),
+ ].toSorted((a, b) => a.localeCompare(b));
+ const total = all.length;
+ if (total <= limit) return freezeManifest(all, total, false, "all");
+
+ const query = opts.query?.trim() ?? "";
+ if (query.length === 0) return freezeManifest(all.slice(0, limit), total, true, "alphabetical");
+
+ const queryTokens = slugTokens(query);
+ const ranked = all
+ .map((name) => ({ name, score: jaccard(queryTokens, slugTokens(name)) }))
+ // Name is the tie-break, so two candidates with the same score never
+ // depend on the order the directory walk happened to return them in.
+ .toSorted((a, b) => b.score - a.score || a.name.localeCompare(b.name))
+ .slice(0, limit)
+ .map((entry) => entry.name);
+ return freezeManifest(ranked, total, true, "relevance");
+}
+
+/**
+ * The one `schema_hints` line a lane adds for its manifest: a CONSTRAINT
+ * on the artifact, stating what the list in `link_candidates` is and -
+ * when it is a subset - that a target outside it may still exist. A hint
+ * claiming the list is exhaustive over a truncated manifest would be the
+ * confident-sounding falsehood this wave exists to remove.
+ */
+export function linkCandidateSchemaHint(manifest: LinkCandidateManifest): string {
+ if (manifest.total === 0) {
+ return "wikilinks: no note in this vault is available to cite, so link_candidates is empty and any wikilink you write will dangle";
+ }
+ if (manifest.truncated) {
+ return (
+ `wikilinks: link_candidates carries ${manifest.candidates.length} of the ${manifest.total} ` +
+ "note names this vault offers; a target it does not name may still exist, so cite one only " +
+ "when the note itself is in your grounding"
+ );
+ }
+ return (
+ `wikilinks: link_candidates carries all ${manifest.total} note names this vault offers; ` +
+ "a name it does not carry is not one you can cite and the link will dangle"
+ );
+}
+
+/**
+ * Tokens of a slug. Hyphens and underscores are separators here even
+ * though the shared tokeniser keeps them - see the module docblock.
+ */
+function slugTokens(text: string): ReadonlySet<string> {
+ return tokenise(text.replace(/[-_]+/g, " "));
+}
+
+function freezeManifest(
+ candidates: ReadonlyArray<string>,
+ total: number,
+ truncated: boolean,
+ selection: LinkCandidateSelection,
+): LinkCandidateManifest {
+ return Object.freeze({
+ candidates: Object.freeze([...candidates]),
+ total,
+ truncated,
+ selection,
+ });
+}
diff --git a/src/core/brain/rollup-ladder.ts b/src/core/brain/rollup-ladder.ts
index c285e6c0..4d4ee77a 100644
--- a/src/core/brain/rollup-ladder.ts
+++ b/src/core/brain/rollup-ladder.ts
@@ -23,6 +23,7 @@ import { existsSync, readFileSync } from "node:fs";
import { atomicWriteFileSync } from "../fs-atomic.ts";
import { buildNeedsLlmStep, type NeedsLlmStep } from "./llm-step.ts";
+import { linkCandidateSchemaHint, type LinkCandidateManifest } from "./notes/link-candidates.ts";
import type { BrainConfig } from "./types.ts";
import { rollupLedgerPath } from "./paths.ts";
import { assertVaultIdentityForWrite } from "./vault-identity.ts";
@@ -63,12 +64,17 @@ export interface RollupLedger {
/**
* The needs-llm-step envelope emitted for one fired rung: the shared
- * envelope spine plus the two fields only a ladder rung has - which rung
- * fired and which tier its summary note becomes.
+ * envelope spine plus the fields only a ladder rung has - which rung
+ * fired, which tier its summary note becomes, and the wikilink targets
+ * the fold may cite. The manifest rides HERE rather than on the spine
+ * for the reason `llm-step.ts` gives: three of the six lanes that speak
+ * the grammar produce a wikilinked note, and a field half the consumers
+ * want is not spine.
*/
export interface RollupEnvelope extends NeedsLlmStep {
readonly tier: string;
readonly produces: string;
+ readonly link_candidates: LinkCandidateManifest;
}
/** One fired rung: the counter reset plus its emitted envelope. */
@@ -99,6 +105,24 @@ export interface RollupLadderInput {
readonly thresholds: RollupThresholds;
/** Run id, for a stable, unique rollup target path. */
readonly runId: string;
+ /**
+ * Wikilink targets the emitted envelopes offer the calling agent.
+ *
+ * A SUPPLIER, not a value, and called at most once - only when a rung
+ * actually fires. The manifest is a whole-vault directory walk, and
+ * under owner-scope delivery a frontmatter read per note; only a fired
+ * rung's envelope carries one, and most passes fire nothing (every dry
+ * run among them). Building it up front paid that walk to throw the
+ * result away.
+ *
+ * The caller still owns the walk - this function does no I/O of its
+ * own and plans identically whatever the supplier returns, so the
+ * ladder stays replayable. Required rather than optional for the same
+ * reason a fallback is refused everywhere else here: a lane that
+ * forgot it would emit an envelope claiming the vault has nothing to
+ * cite.
+ */
+ readonly linkCandidates: () => LinkCandidateManifest;
}
/** Resolve the rollup thresholds from config, else the named defaults. */
@@ -140,12 +164,18 @@ interface Rung {
}
/**
- * Plan the rollup ladder. Pure: no I/O, deterministic in its inputs. The
- * rungs are processed base-to-top so a fact rollup fired this pass counts
- * toward the identity rung in the same pass.
+ * Plan the rollup ladder. Deterministic in its inputs and doing no I/O of
+ * its own; the one thing it may reach for is the link-candidate manifest,
+ * and only for a rung that fires (see {@link RollupLadderInput.linkCandidates}).
+ * The rungs are processed base-to-top so a fact rollup fired this pass
+ * counts toward the identity rung in the same pass.
*/
export function planRollupLadder(input: RollupLadderInput): RollupLadderPlan {
- const { factCount, ledger, thresholds, runId } = input;
+ const { factCount, ledger, thresholds, runId, linkCandidates } = input;
+ // At most one call, however many rungs fire: two envelopes in one pass
+ // offer the same targets, and the walk behind them is the expensive part.
+ let manifest: LinkCandidateManifest | null = null;
+ const resolveCandidates = (): LinkCandidateManifest => (manifest ??= linkCandidates());
const baselines: Record<string, number> = { ...ledger?.baselines };
const produced: Record<string, number> = { ...ledger?.produced };
@@ -174,7 +204,7 @@ export function planRollupLadder(input: RollupLadderInput): RollupLadderPlan {
fromCount: baseline,
toCount: source,
newSinceLast,
- envelope: buildEnvelope(rung, newSinceLast, runId),
+ envelope: buildEnvelope(rung, newSinceLast, runId, resolveCandidates()),
}),
);
}
@@ -190,7 +220,12 @@ export function planRollupLadder(input: RollupLadderInput): RollupLadderPlan {
});
}
-function buildEnvelope(rung: Rung, newSinceLast: number, runId: string): RollupEnvelope {
+function buildEnvelope(
+ rung: Rung,
+ newSinceLast: number,
+ runId: string,
+ linkCandidates: LinkCandidateManifest,
+): RollupEnvelope {
const targetPath = `Brain/rollups/rollup-${rung.produces}-${runId}.md`;
return buildNeedsLlmStep({
step: `rollup:${rung.tier}`,
@@ -202,7 +237,9 @@ function buildEnvelope(rung: Rung, newSinceLast: number, runId: string): RollupE
schema_hints: [
"frontmatter: required YAML block with at least a `kind` key",
`tier: ${rung.produces} (the rollup's tier weight)`,
+ linkCandidateSchemaHint(linkCandidates),
],
target_path: targetPath,
+ link_candidates: linkCandidates,
});
}
diff --git a/src/core/brain/sessions/checkpoint.ts b/src/core/brain/sessions/checkpoint.ts
new file mode 100644
index 00000000..d1189155
--- /dev/null
+++ b/src/core/brain/sessions/checkpoint.ts
@@ -0,0 +1,362 @@
+/**
+ * Per-file session-import checkpoint (nothing-writes-silently, Unit F).
+ *
+ * Resume is NOT introduced here. `ingest/checkpoint.ts` has shipped
+ * plan-scoped resume for the source-ingest lane since the Ingestion & Import
+ * Robustness suite; this module extends that substrate
+ * ({@link ../checkpoint-store.ts}) to the sessions lane, which had none.
+ *
+ * What the sessions lane had instead was DEDUP: `importSession` rebuilt a
+ * dedup index from the inbox at the start of every run, so a second run over
+ * the same transcript found every hash already present and discarded its
+ * writes. That is re-do-and-discard, not resume - an interrupted 50k-turn
+ * import re-read and re-hashed all fifty thousand turns to conclude it had
+ * nothing to write. This checkpoint records the turn boundary instead, so the
+ * next run skips straight past the turns a previous one finished.
+ *
+ * ## The key: (file identity, turn boundary)
+ *
+ * The turn boundary is a count of turns the adapter has already yielded and
+ * the import has already processed. The file identity is what makes that
+ * count meaningful, and it is two cheap facts rather than a content hash: the
+ * SHA-256 of the transcript's first line, and its size in bytes at the moment
+ * the boundary was taken. Hashing the whole file would defeat the purpose -
+ * re-reading every byte is the cost being avoided - and would invalidate the
+ * checkpoint on every append to a log that is append-only by construction.
+ *
+ * Stated precisely, because the guarantee is narrower than "the file is
+ * unchanged": a transcript whose first line differs, or which has SHRUNK
+ * since the boundary, is a different log and is refused. A log rewritten in
+ * place to the same-or-greater length with the same first line would not be
+ * caught. Every session adapter this build ships reads an append-only JSONL
+ * whose first line carries the session's own identifiers, so that case is not
+ * one the format produces; the limit is written down rather than implied.
+ *
+ * ## Scoped to the import's question, not just the file
+ *
+ * The id folds the import's filters in - role and text filters, `since`,
+ * event-time preservation, recall, the ingest scope label, the agent. Two
+ * runs asking different questions of one transcript reach different turn sets
+ * and must never resume off each other's boundary; giving them different ids
+ * is what makes that structural rather than remembered.
+ *
+ * Language-agnostic: the id is a hex digest over paths and option values; no
+ * natural-language content is inspected.
+ */
+
+import { createHash } from "node:crypto";
+
+import {
+ checkpointFilePath,
+ checkpointingEnabled,
+ readCheckpointObject,
+ removeCheckpointFile,
+ withCheckpointLock,
+ writeCheckpointObject,
+} from "../checkpoint-store.ts";
+import { isoSecond } from "../time.ts";
+
+/** Only schema version currently understood. Unknown versions are refused. */
+const SCHEMA_VERSION = 1 as const;
+
+/** Subdirectory holding one JSON checkpoint per scoped session import. */
+const CHECKPOINT_DIR = "session-import-checkpoints";
+
+/** How a refusal names this lane's id and its file. */
+const ID_LABEL = "session import";
+const FILE_LABEL = "session import checkpoint";
+
+/**
+ * Turns between boundary writes.
+ *
+ * A boundary per turn would take the sync lock and land an atomic write fifty
+ * thousand times on a large transcript, which costs more than the extraction
+ * it protects. A boundary every 250 turns bounds an interruption's lost work
+ * to at most 250 turns while keeping the whole run to a couple of hundred
+ * checkpoint writes.
+ */
+export const SESSION_CHECKPOINT_TURN_INTERVAL = 250;
+
+/** The persisted per-import checkpoint. */
+export interface SessionImportCheckpoint {
+ readonly schema_version: typeof SCHEMA_VERSION;
+ /** Stable id derived from the session file and the import's scope. */
+ readonly import_id: string;
+ /**
+ * The session log this boundary belongs to, as this machine addressed it.
+ * Machine-local by design and for the same reason the session import ledger
+ * is: a transcript lives outside the vault, under a per-runtime root that
+ * differs on every device, so there is no portable spelling of it.
+ */
+ readonly session_file: string;
+ /** SHA-256 of the transcript's first line when the boundary was taken. */
+ readonly head_hash: string;
+ /** Size of the transcript in bytes when the boundary was taken. */
+ readonly bytes: number;
+ /** Turns the import had fully processed at the boundary. */
+ readonly turns_completed: number;
+ readonly updated_at: string;
+}
+
+/** The subset of import options that changes which turns a run processes. */
+export interface SessionImportScope {
+ readonly agent: string;
+ readonly format?: string;
+ readonly since?: Date;
+ readonly filterRoles?: ReadonlyArray<string>;
+ readonly filterTextIncludes?: string;
+ readonly preserveEventTime?: boolean;
+ readonly recall?: boolean;
+ readonly recallSessionId?: string;
+ readonly recallSummaryGroupSize?: number;
+ readonly ingestScope?: string;
+ readonly rawCodec?: boolean;
+}
+
+/** Why a checkpoint that existed was refused instead of resumed from. */
+export const SESSION_RESUME_DISCARD = Object.freeze({
+ /** The transcript's first line is not the one the boundary was taken against. */
+ headChanged: "head_changed",
+ /** The transcript is smaller than it was, so the boundary is past its end. */
+ fileShrank: "file_shrank",
+ /**
+ * The file parsed, and its payload is not one this writer could have
+ * written - a `turns_completed` that is not a whole number, or a negative
+ * one. Coercing it to zero would present finished turns as pending and be
+ * indistinguishable from "there was no checkpoint".
+ */
+ payloadInvalid: "payload_invalid",
+} as const);
+
+export type SessionResumeDiscard =
+ (typeof SESSION_RESUME_DISCARD)[keyof typeof SESSION_RESUME_DISCARD];
+
+/** What a run measured about the transcript it is about to read. */
+export interface SessionFileIdentity {
+ readonly headHash: string;
+ readonly bytes: number;
+}
+
+/** The resume decision: where to start, and why a checkpoint was rejected. */
+export interface SessionResumeDecision {
+ /** Turns to skip. Zero when there is no usable checkpoint. */
+ readonly turns: number;
+ /** Set only when a checkpoint existed and was refused. */
+ readonly discarded: SessionResumeDiscard | null;
+}
+
+/** SHA-256 hex of a transcript's first line - the cheap half of file identity. */
+export function computeSessionHeadHash(firstLine: string): string {
+ return createHash("sha256").update(firstLine).digest("hex");
+}
+
+/**
+ * Deterministic import id: a short SHA-256 hex over the session file's path
+ * and every option that changes which turns the run processes. Options are
+ * folded in a fixed order with NUL separators so no two distinct scopes can
+ * collide by concatenation.
+ */
+export function computeSessionImportId(sessionFile: string, scope: SessionImportScope): string {
+ const parts: string[] = [
+ sessionFile,
+ scope.agent,
+ scope.format ?? "",
+ scope.since ? String(scope.since.getTime()) : "",
+ [...(scope.filterRoles ?? [])].toSorted().join(","),
+ scope.filterTextIncludes ?? "",
+ scope.preserveEventTime === true ? "1" : "",
+ scope.recall === true ? "1" : "",
+ scope.recallSessionId ?? "",
+ scope.recallSummaryGroupSize !== undefined ? String(scope.recallSummaryGroupSize) : "",
+ scope.ingestScope ?? "",
+ scope.rawCodec === true ? "1" : "",
+ ];
+ return createHash("sha256").update(parts.join("\u0000")).digest("hex").slice(0, 16);
+}
+
+/** Absolute path of one scoped import's checkpoint file. */
+export function sessionCheckpointPath(vault: string, importId: string): string {
+ return checkpointFilePath(vault, CHECKPOINT_DIR, ID_LABEL, importId);
+}
+
+function serialize(cp: SessionImportCheckpoint): string {
+ return (
+ JSON.stringify(
+ {
+ schema_version: cp.schema_version,
+ import_id: cp.import_id,
+ session_file: cp.session_file,
+ head_hash: cp.head_hash,
+ bytes: cp.bytes,
+ turns_completed: cp.turns_completed,
+ updated_at: cp.updated_at,
+ },
+ null,
+ 2,
+ ) + "\n"
+ );
+}
+
+function requireNonNegativeInteger(field: string, value: number): void {
+ if (!Number.isInteger(value) || value < 0) {
+ throw new Error(
+ `session import checkpoint: ${field} must be a non-negative integer, got ${value}`,
+ );
+ }
+}
+
+function requireHex64(field: string, value: string): void {
+ if (!/^[0-9a-f]{64}$/.test(value)) {
+ throw new Error(
+ `session import checkpoint: ${field} must be a 64-character lowercase hex digest`,
+ );
+ }
+}
+
+/**
+ * A checkpoint file parsed, and carried a payload the writer could not have
+ * produced. Its own class so {@link resolveSessionResume} can turn exactly
+ * this condition into a named discard and let every other read failure -
+ * corrupt bytes, an unknown `schema_version` - keep propagating.
+ */
+export class SessionCheckpointPayloadError extends Error {
+ constructor(message: string) {
+ super(message);
+ this.name = "SessionCheckpointPayloadError";
+ }
+}
+
+/**
+ * Read one scoped import's checkpoint. A missing file (or checkpointing
+ * disabled) returns `null`. Corrupt bytes and an unknown `schema_version` are
+ * hard errors - never a silent reset, which would present finished turns as
+ * pending and re-hash every one of them.
+ *
+ * The PAYLOAD is held to the same rule as the envelope, and to exactly the
+ * rule {@link recordSessionProgress} enforces on the way in: a
+ * `turns_completed` or `bytes` that is not a non-negative integer raises
+ * {@link SessionCheckpointPayloadError} rather than being coerced to zero.
+ * The coercion was the silent reset by another route - zero turns completed
+ * reads back as "there was no checkpoint" - and a negative count, which no
+ * writer here can produce, was honoured verbatim.
+ */
+export function readSessionCheckpoint(
+ vault: string,
+ importId: string,
+): SessionImportCheckpoint | null {
+ if (!checkpointingEnabled()) return null;
+ const path = sessionCheckpointPath(vault, importId);
+ const obj = readCheckpointObject(path, FILE_LABEL, SCHEMA_VERSION);
+ if (obj === null) return null;
+ return Object.freeze({
+ schema_version: SCHEMA_VERSION,
+ import_id: importId,
+ session_file: typeof obj["session_file"] === "string" ? obj["session_file"] : "",
+ head_hash: typeof obj["head_hash"] === "string" ? obj["head_hash"] : "",
+ bytes: requireStoredCount(path, "bytes", obj["bytes"]),
+ turns_completed: requireStoredCount(path, "turns_completed", obj["turns_completed"]),
+ updated_at: typeof obj["updated_at"] === "string" ? obj["updated_at"] : "",
+ });
+}
+
+function requireStoredCount(path: string, field: string, value: unknown): number {
+ if (!Number.isInteger(value) || (value as number) < 0) {
+ throw new SessionCheckpointPayloadError(
+ `${FILE_LABEL}: ${field} must be a non-negative integer, got ${JSON.stringify(value)}: ${path}`,
+ );
+ }
+ return value as number;
+}
+
+/** What {@link recordSessionProgress} is told about the boundary just reached. */
+export interface SessionProgressInput {
+ readonly sessionFile: string;
+ readonly headHash: string;
+ readonly bytes: number;
+ readonly turnsCompleted: number;
+}
+
+/**
+ * Record the turn boundary a run has reached, atomically. Returns `true` when
+ * the checkpoint was written, `false` on a no-op - checkpointing disabled, or
+ * the serialized bytes unchanged.
+ *
+ * The read and the write are ONE critical section under the sync lock, for
+ * the reason the shared substrate states: two runs over one transcript that
+ * both wrote back the state they read would leave the lower boundary, silently
+ * turning finished turns back into pending work.
+ */
+export function recordSessionProgress(
+ vault: string,
+ importId: string,
+ input: SessionProgressInput,
+ now: Date,
+): boolean {
+ requireNonNegativeInteger("turns_completed", input.turnsCompleted);
+ requireNonNegativeInteger("bytes", input.bytes);
+ requireHex64("head_hash", input.headHash);
+ if (!checkpointingEnabled()) return false;
+ const path = sessionCheckpointPath(vault, importId);
+ return withCheckpointLock(path, () =>
+ writeCheckpointObject(
+ vault,
+ path,
+ serialize({
+ schema_version: SCHEMA_VERSION,
+ import_id: importId,
+ session_file: input.sessionFile,
+ head_hash: input.headHash,
+ bytes: input.bytes,
+ turns_completed: input.turnsCompleted,
+ updated_at: isoSecond(now),
+ }),
+ ),
+ );
+}
+
+/**
+ * Decide where a run should start. Returns `turns: 0` with `discarded: null`
+ * when there is no checkpoint at all, and `turns: 0` with a named reason when
+ * one exists but was taken against a transcript this is no longer. The caller
+ * surfaces that reason: a boundary quietly ignored is a full re-import the
+ * operator was never told about.
+ */
+export function resolveSessionResume(
+ vault: string,
+ importId: string,
+ live: SessionFileIdentity,
+): SessionResumeDecision {
+ let cp: SessionImportCheckpoint | null;
+ try {
+ cp = readSessionCheckpoint(vault, importId);
+ } catch (err) {
+ // A payload the writer could not have produced is a boundary this run
+ // cannot use - reported as a discard, the same as a boundary taken
+ // against another transcript, so the re-import is the operator's to see.
+ // Every other read failure (corrupt bytes, an unknown schema_version)
+ // still propagates: those say the substrate is broken, not that one
+ // boundary is unusable.
+ if (err instanceof SessionCheckpointPayloadError) {
+ return { turns: 0, discarded: SESSION_RESUME_DISCARD.payloadInvalid };
+ }
+ throw err;
+ }
+ if (cp === null) return { turns: 0, discarded: null };
+ if (cp.head_hash !== live.headHash) {
+ return { turns: 0, discarded: SESSION_RESUME_DISCARD.headChanged };
+ }
+ if (live.bytes < cp.bytes) {
+ return { turns: 0, discarded: SESSION_RESUME_DISCARD.fileShrank };
+ }
+ return { turns: cp.turns_completed, discarded: null };
+}
+
+/**
+ * Drop one scoped import's checkpoint once the run has drained it - the same
+ * settle the ingest lane performs on a fully drained plan, where the content
+ * manifest (here, the dedup index) is the authoritative final state from then
+ * on. Returns `true` when a file was removed.
+ */
+export function clearSessionCheckpoint(vault: string, importId: string): boolean {
+ return removeCheckpointFile(vault, sessionCheckpointPath(vault, importId));
+}
diff --git a/src/core/brain/sessions/import.ts b/src/core/brain/sessions/import.ts
index 4390cf56..d127579d 100644
--- a/src/core/brain/sessions/import.ts
+++ b/src/core/brain/sessions/import.ts
@@ -23,6 +23,32 @@
* Idempotency: dedup index is built once at the start of each
* `importSession` run by reading the inbox and processed dirs. A
* second run on the same file finds every hash already present.
+ *
+ * Resume (nothing-writes-silently, Unit F). Dedup alone is
+ * re-do-and-discard: it makes a second run harmless, not cheap, so an
+ * interrupted 50k-turn import re-read and re-hashed every turn to
+ * conclude it had nothing to write. An applied run now records the turn
+ * boundary it has reached into {@link ./checkpoint.ts} - the sessions
+ * lane's extension of the ingest checkpoint substrate, not a second one
+ * of its own - and the next run over the same file with the same filters
+ * skips straight past those turns. What that saves is the extraction
+ * work: the marker scan, the dedup hashing, the fact pass and the writes.
+ * The lines themselves are still read, because a JSONL transcript has no
+ * index to seek into.
+ *
+ * Recall is the one thing the boundary does not vouch for, and the resumed
+ * run re-collects it. `--recall` accumulates turns in memory and commits
+ * them to the recall DAG AFTER the loop, so a run that stopped at the
+ * boundary committed none of them; skipping those turns on the resumed run
+ * would leave the head of the transcript permanently missing from the DAG
+ * while both runs reported success. Re-collection costs nothing extra (the
+ * lines are read regardless) and is idempotent, because the recall importer
+ * keys every turn on a content dedupe key.
+ *
+ * Census (same unit). After the run, the dedup hashes of the signals it
+ * claims to have written are read back from disk, and the result carries
+ * `claimed / found / missing` with the missing hashes NAMED. See
+ * {@link ../import-census.ts} for what the census does and does not cover.
*/
import { existsSync, statSync } from "node:fs";
@@ -30,13 +56,24 @@ import { basename, resolve } from "node:path";
import { delegatedAgentName } from "../../agent-identity.ts";
import { readSkillOfferId, SKILL_OFFER_ID_KEY } from "../../surface/skill-offer.ts";
+import { checkpointingEnabled } from "../checkpoint-store.ts";
import { buildDedupIndex, computeDedupHash, type DedupIndexEntry } from "../dedup-hash.ts";
+import { censusSessionSignals, type ImportCensus } from "../import-census.ts";
import { appendContinuityRecord } from "../continuity/store.ts";
import { discoverMarkersDetailed, isFeedbackMarker } from "../inline.ts";
import { writeSignal } from "../signal.ts";
import { importSessionRecall } from "../session-recall.ts";
import { isoDate, isoSecond } from "../time.ts";
import { BRAIN_SIGNAL_SOURCE_TYPE } from "../types.ts";
+import {
+ clearSessionCheckpoint,
+ computeSessionHeadHash,
+ computeSessionImportId,
+ recordSessionProgress,
+ resolveSessionResume,
+ SESSION_CHECKPOINT_TURN_INTERVAL,
+ type SessionResumeDiscard,
+} from "./checkpoint.ts";
import { readFirstLine } from "./read-lines.ts";
import { detectAdapter, getAdapter } from "./registry.ts";
import { sessionFilesUnder } from "./session-files.ts";
@@ -190,6 +227,25 @@ export interface ImportSessionResult {
readonly facts_deduped: number;
readonly recall_turns_imported: number;
readonly recall_summary_nodes: number;
+ /**
+ * Turns a checkpoint from an earlier, interrupted run let this one skip.
+ * Always 0 on a first run, a dry run, and a run with checkpointing opted
+ * out. `turns_scanned` still counts every turn the adapter yielded, so the
+ * pair says how much of the file was re-read and how little was re-worked.
+ */
+ readonly turns_resumed: number;
+ /**
+ * Why a checkpoint that existed was NOT resumed from, or `null` when none
+ * was refused. A boundary silently ignored is a full re-import nobody was
+ * told about, which is the class of quiet this unit removes.
+ */
+ readonly resume_discarded: SessionResumeDiscard | null;
+ /**
+ * Read-back census of the signals this run claims to have written:
+ * `attempted / found / missing`, with the missing dedup hashes named. A
+ * dry run claims nothing and censuses zero.
+ */
+ readonly census: ImportCensus;
readonly errors: ReadonlyArray<{ path: string; message: string }>;
}
@@ -220,13 +276,20 @@ export function sessionRefIdentity(absPath: string, recallSessionId?: string): s
*
* Autodetect needs exactly one line, and `readFirstLine` stops at the first
* newline: the file used to be read whole here and then read a second time by
- * `adapter.iterate`. An explicit `--format` reads nothing at all.
+ * `adapter.iterate`. An explicit `--format` reads nothing at all - hence the
+ * lazy `readHead`, shared with the resume checkpoint's file-identity probe so
+ * the head line is read at most once per import and not at all when neither
+ * caller needs it.
*/
-async function chooseAdapter(path: string, format?: SessionAdapterId): Promise<SessionAdapter> {
+async function chooseAdapter(
+ path: string,
+ readHead: () => Promise<string>,
+ format?: SessionAdapterId,
+): Promise<SessionAdapter> {
if (format !== undefined) {
return getAdapter(format);
}
- const first = await readFirstLine(path);
+ const first = await readHead();
const a = detectAdapter(first);
if (!a) {
throw new SessionImportError(
@@ -245,7 +308,9 @@ export async function importSession(
if (!existsSync(path)) {
throw new SessionImportError("IO", `session file does not exist: ${path}`);
}
- const adapter = await chooseAdapter(path, opts.format);
+ let headLine: string | null = null;
+ const readHead = async (): Promise<string> => (headLine ??= await readFirstLine(path));
+ const adapter = await chooseAdapter(path, readHead, opts.format);
// Reuse the caller-supplied index when present (directory walk lifts
// the build out of the per-file loop). Otherwise build our own.
const dedup = opts.dedupIndex ?? buildDedupIndex(vault);
@@ -278,6 +343,11 @@ export async function importSession(
* one write and one dedup.
*/
const withheldHashes = new Set<string>();
+ /**
+ * Dedup hashes of the signals this run actually wrote - the claim the
+ * post-import census reads back against the vault.
+ */
+ const createdHashes: string[] = [];
const recallTurns: SessionTurn[] = [];
const filterRoles =
opts.filterRoles && opts.filterRoles.length > 0 ? new Set(opts.filterRoles) : null;
@@ -355,6 +425,7 @@ export async function importSession(
...(opts.rawCodec === true ? { rawCodec: true } : {}),
});
dedup.set(input.dedupHash, { id: res.id, path: res.path });
+ createdHashes.push(input.dedupHash);
signalsCreated++;
} catch (err) {
errors.push({
@@ -390,33 +461,105 @@ export async function importSession(
facts_deduped: 0,
recall_turns_imported: 0,
recall_summary_nodes: 0,
+ turns_resumed: 0,
+ resume_discarded: null,
+ census: censusSessionSignals(vault, []),
errors: Object.freeze([]),
});
if (boundaryDecision === "ignore") return emptyResult();
- for await (const turn of adapter.iterate(path)) {
- turnsScanned++;
+ // Resume (Unit F). Only an APPLIED capture run checkpoints: a dry run
+ // writes nothing to resume, and a stateless/ignored file is scanned
+ // read-only. The id folds the import's filters in, so a differently
+ // filtered run never inherits this one's boundary.
+ const checkpointActive = opts.dryRun !== true && mayWrite && checkpointingEnabled();
+ const importId = computeSessionImportId(absPath, opts);
+ let resumeFrom = 0;
+ let resumeDiscarded: SessionResumeDiscard | null = null;
+ let turnsResumed = 0;
+ let fileIdentity = { headHash: "", bytes: 0 };
+ if (checkpointActive) {
+ fileIdentity = {
+ headHash: computeSessionHeadHash(await readHead()),
+ bytes: statSync(absPath).size,
+ };
+ const decision = resolveSessionResume(vault, importId, fileIdentity);
+ resumeFrom = decision.turns;
+ resumeDiscarded = decision.discarded;
+ }
+
+ /**
+ * Whether a turn belongs to this run's question, and which counter says
+ * so when it does not. One function because the resumed prefix has to ask
+ * the same question the main body does - a filter applied on one path and
+ * not the other would put a turn in the recall DAG that the run's own
+ * filters exclude.
+ */
+ const admitTurn = (turn: SessionTurn): "admit" | "before_since" | "filtered" | "suppressed" => {
if (sinceMs !== undefined) {
const t = Date.parse(turn.timestamp);
- if (Number.isFinite(t) && t < sinceMs) continue;
- }
- if (filterRoles !== null && !filterRoles.has(turn.role)) {
- filteredTurns++;
- continue;
+ if (Number.isFinite(t) && t < sinceMs) return "before_since";
}
+ if (filterRoles !== null && !filterRoles.has(turn.role)) return "filtered";
if (filterNeedle !== null) {
const haystack = turn.text?.toLowerCase() ?? "";
- if (!haystack.includes(filterNeedle)) {
- filteredTurns++;
- continue;
- }
+ if (!haystack.includes(filterNeedle)) return "filtered";
}
// Message-level boundary: suppressed text never reaches marker or
// fact extraction (and is not stored for recall).
- if (turn.text && boundary.suppressMessage(turn.text)) {
+ if (turn.text && boundary.suppressMessage(turn.text)) return "suppressed";
+ return "admit";
+ };
+
+ for await (const turn of adapter.iterate(path)) {
+ turnsScanned++;
+ // Past the boundary an earlier run recorded: the extraction, the
+ // hashing and the writes for these turns already happened. The lines
+ // are still read because a JSONL transcript cannot be seeked into.
+ if (turnsScanned <= resumeFrom) {
+ turnsResumed++;
+ // Recall is the one thing the boundary does NOT vouch for. It is
+ // accumulated in memory and committed after this loop, so the run
+ // that stopped at the boundary committed none of it - skipping these
+ // turns would leave the head of the transcript permanently absent
+ // from the recall DAG while the run reported success. Re-collecting
+ // them is cheap (the lines are read either way) and idempotent (the
+ // recall importer keys each turn on a content dedupe key).
+ if (opts.recall === true && mayWrite && admitTurn(turn) === "admit") recallTurns.push(turn);
+ continue;
+ }
+ // Everything before THIS turn is done, so the boundary is recorded
+ // here rather than at the foot of the loop - the body has several
+ // early `continue`s, and a turn skipped by a filter needs no re-work
+ // either.
+ const completedTurns = turnsScanned - 1;
+ if (
+ checkpointActive &&
+ completedTurns > resumeFrom &&
+ completedTurns % SESSION_CHECKPOINT_TURN_INTERVAL === 0
+ ) {
+ recordSessionProgress(
+ vault,
+ importId,
+ {
+ sessionFile: absPath,
+ headHash: fileIdentity.headHash,
+ bytes: fileIdentity.bytes,
+ turnsCompleted: completedTurns,
+ },
+ now,
+ );
+ }
+ const admission = admitTurn(turn);
+ if (admission === "filtered") {
+ filteredTurns++;
+ continue;
+ }
+ if (admission === "suppressed") {
suppressedTurns++;
continue;
}
+ if (admission === "before_since") continue;
if (opts.recall === true && mayWrite) recallTurns.push(turn);
// Facts from USER turns only (the HANDOFF carve-out's conservative
@@ -569,6 +712,11 @@ export async function importSession(
});
}
+ // The run drained the file, so its resume point has nothing left to say -
+ // the same settle the ingest lane performs on a fully drained plan, after
+ // which the dedup index is the authoritative final state.
+ if (checkpointActive) clearSessionCheckpoint(vault, importId);
+
return Object.freeze({
file: absPath,
format: adapter.id,
@@ -588,6 +736,11 @@ export async function importSession(
filtered_turns: filteredTurns,
recall_turns_imported: recallTurnsImported,
recall_summary_nodes: recallSummaryNodes,
+ turns_resumed: turnsResumed,
+ resume_discarded: resumeDiscarded,
+ // Read back AFTER every write this run makes, including the recall
+ // import above: the point is to ask the disk, not the run's own belief.
+ census: censusSessionSignals(vault, createdHashes),
errors: Object.freeze(errors),
});
}
diff --git a/src/core/brain/signal.ts b/src/core/brain/signal.ts
index a73f0946..6b97b809 100644
--- a/src/core/brain/signal.ts
+++ b/src/core/brain/signal.ts
@@ -35,6 +35,12 @@ import {
rememberKey,
} from "./idempotency-ledger.ts";
import { sanitisePrinciple } from "./text/sanitize-principle.ts";
+import {
+ ORIGIN_CHANNEL_FIELD,
+ isOriginChannelStamp,
+ originChannelStamp,
+ type OriginChannelStamp,
+} from "../origin-channel.ts";
import { EXPIRATION_DATE_FIELD, normalizeExpirationDate } from "./expiration.ts";
import { writeFrontmatterAtomic, parseFrontmatter } from "../vault.ts";
import { compress, expand, CODEC_VERSION } from "./portability/codec.ts";
@@ -63,6 +69,28 @@ function renderSourceTypes(): string {
return BRAIN_SIGNAL_SOURCE_TYPES.map((v) => `'${v}'`).join(", ");
}
+/**
+ * Read the `origin_channel` frontmatter key back, refusing a value this
+ * build cannot interpret. `undefined` means the key is absent, which is
+ * what every signal written before Unit C carries.
+ */
+function readOriginChannel(meta: FrontmatterMap, path: string): OriginChannelStamp | undefined {
+ const value = meta[ORIGIN_CHANNEL_FIELD];
+ if (value === undefined) return undefined;
+ if (typeof value !== "string") {
+ throw new Error(`signal field '${ORIGIN_CHANNEL_FIELD}' must be a string (${path})`);
+ }
+ const trimmed = value.trim();
+ if (trimmed === "") return undefined;
+ if (!isOriginChannelStamp(trimmed)) {
+ throw new Error(
+ `signal field '${ORIGIN_CHANNEL_FIELD}' is not a channel this build reads: ` +
+ `${JSON.stringify(trimmed)} (${path})`,
+ );
+ }
+ return trimmed;
+}
+
/** Filename prefix without the trailing dash, e.g. `sig-2026-05-14`. */
function signalPrefix(date: string): string {
return `sig-${validateIsoDate(date)}`;
@@ -241,6 +269,39 @@ const REQUIRED_INPUT_FIELDS: ReadonlyArray<keyof WriteSignalInput> = [
* section is rendered with an `_(not provided)_` placeholder so every
* signal has the same heading structure — consistent with the Pay
* Memory receipt style.
+ *
+ * ## `origin_channel` is a SIBLING of `source_type`, not a rename
+ *
+ * Every signal written from here carries `origin_channel`: the
+ * server-derived channel of the process that wrote it
+ * ({@link originChannelStamp}). It sits beside `source_type` and neither
+ * field may be folded into the other, because they are different axes
+ * that only look alike:
+ *
+ * | `source_type` | what it says | axis |
+ * |----------------|-----------------------------------------------|------------|
+ * | `live` | written by live `brain_feedback` | origin |
+ * | `inline` | captured from an `@osb` marker in a vault file| origin |
+ * | `session` | replayed from a session JSONL | origin |
+ * | `extracted` | mined by a regex over structure | extraction |
+ * | `auto_extract` | mined by a model reading prose | extraction |
+ *
+ * Three of the five name where the content came from and two name how it
+ * was derived, so `source_type` is a MIXED axis and cannot answer "which
+ * transport wrote this". `origin_channel` answers only that:
+ *
+ * | `origin_channel` | who wrote the record |
+ * |------------------|-----------------------------------------------|
+ * | `mcp-tool` | an agent called a tool over the MCP transport |
+ * | `cli` | an operator or script ran `o2b` |
+ * | `import` | a bulk replay of records authored elsewhere |
+ *
+ * Neither determines the other. A `live` signal is `mcp-tool` when a
+ * tool call wrote it and `cli` when `o2b brain feedback` did, and
+ * `extracted` says nothing about the transport at all. Renaming
+ * `source_type` to the channel spelling was considered and refused: it
+ * would silently reinterpret every signal already on disk and break the
+ * fixtures and read-side parsers that carry its meaning.
*/
export function writeSignal(
vault: string,
@@ -401,6 +462,14 @@ function renderSignalDocument(
) {
metadata["source_type"] = sanitised.source_type;
}
+ // Server-derived origin channel (Unit C), the sibling of `source_type`
+ // the table on `writeSignal` maps. Emitted ALWAYS, including the
+ // `unset` literal: unlike `source_type`, absence here already means
+ // something else - a signal written before the stamp existed - and
+ // overloading it with "no entry point claimed the process" would make
+ // the two indistinguishable. It is read off the module, never off the
+ // input: no caller may name its own channel.
+ metadata[ORIGIN_CHANNEL_FIELD] = originChannelStamp();
if (sanitised.schema_type?.trim()) {
metadata["schema_type"] = validateSchemaToken(sanitised.schema_type, "schema_type");
}
@@ -453,6 +522,11 @@ function renderSignalDocument(
* that make two signals "the same content". Timestamps, the calendar
* date, the allocated slug, and portability markers are intentionally
* excluded so a retry with a later wall-clock still dedupes.
+ *
+ * `origin_channel` is excluded for the stronger version of that reason:
+ * it is provenance, not content, and including it would make the same
+ * signal retried from a different channel collide as an
+ * {@link IdempotencyPayloadMismatchError} instead of deduping.
*/
function signalPayloadFields(input: WriteSignalInput): Record<string, unknown> {
return {
@@ -613,6 +687,15 @@ export function parseSignal(path: string, options: ParseSignalOptions = {}): Bra
// (a-label-is-not-a-boundary, U5). Absent on every non-mirrored signal,
// which is every signal a single-vault install writes.
const origin_vault = readOptionalTrimmedString(meta, "origin_vault", path);
+ // Read back rather than left write-only. `origin_vault` shipped with no
+ // reader at all and had to be given one a release later; a provenance
+ // field nothing can query is a field nothing can act on. Absent on every
+ // signal written before Unit C, and those are never rewritten - the
+ // no-backfill refusal in `src/core/origin-channel.ts`. A value outside
+ // the vocabulary is refused rather than coerced, for the same reason
+ // `source_type` refuses one: a channel this build cannot read is not a
+ // channel it may silently rename.
+ const origin_channel = readOriginChannel(meta, path);
const result: BrainSignal = {
kind: "brain-signal",
@@ -631,6 +714,7 @@ export function parseSignal(path: string, options: ParseSignalOptions = {}): Bra
...(dedup_hash !== undefined ? { dedup_hash } : {}),
...(session_ref !== undefined ? { session_ref } : {}),
...(origin_vault !== undefined ? { origin_vault } : {}),
+ ...(origin_channel !== undefined ? { origin_channel } : {}),
...readBiTemporal(meta, path),
...(readOptionalTrimmedString(meta, "expiration_date", path) !== undefined
? { expiration_date: readOptionalTrimmedString(meta, "expiration_date", path) }
diff --git a/src/core/brain/status.ts b/src/core/brain/status.ts
index 31ca84d2..aba2ca15 100644
--- a/src/core/brain/status.ts
+++ b/src/core/brain/status.ts
@@ -2,18 +2,61 @@
* Brain operational status snapshot.
*
* One function: {@link computeBrainStatus} walks `Brain/` and returns
- * counts, last-activity timestamps, and a single sanity flag. Pure
- * read — nothing is mutated, nothing is parsed deeply (we cap I/O at
- * `readdirSync` for counts and one `parseLogDay` per log file for
- * timestamps).
+ * counts, last-activity timestamps, a `maintenance_debt` block, and a
+ * single sanity flag. Pure read — nothing is mutated, nothing is parsed
+ * deeply (we cap I/O at `readdirSync` for counts and one `parseLogDay`
+ * per log file for timestamps and for the debt count).
*
* Used by:
*
* - the MCP `second_brain_status` tool (extended `brain` field)
* - the MCP resource `osb://status` (markdown render)
+ * - the MCP `brain_context` tool, which reads only the cheap, bounded
+ * `computeMaintenanceOverdueFlag` below — never this whole function
+ * — because it is the always-loaded reader (see that function's
+ * docblock for the cost bound).
*
* Callers that want the timestamps but not the counts can read either
* field independently — the shape is shallow.
+ *
+ * ## `maintenance_debt` (nothing-writes-silently, Unit D)
+ *
+ * DERIVED, not counted: there is no write-time counter and no
+ * STATE_SURFACES row backing this block. It is recomputed on every
+ * call from the same `Brain/log/` JSONL shards `scanLogTimestamps`
+ * already reads to find `last_dream_at`, so it cannot desync from the
+ * log and carries no write-path coupling.
+ *
+ * The field is named `log_events_since_dream`, not `writes_since_dream`,
+ * because that is what it measures and nothing more. `write-batch.ts`
+ * — the executor behind `brain_create_note`, `brain_update_note`,
+ * `brain_append_note`, and the note ops of `brain_write_batch` — never
+ * calls `appendLogEvent`; a caller-named note write leaves no log event
+ * to count. Calling this field "writes since dream" would be a false
+ * label on a true number, which is exactly the dishonesty this wave
+ * removes. It counts Brain LOG events (dream/feedback/apply-evidence/
+ * retire/promote/… — the full {@link BRAIN_LOG_EVENT_KIND} vocabulary),
+ * a real and useful signal of unreviewed lifecycle activity, just not
+ * the one its old provisional name would have implied.
+ *
+ * Two fields the card asked for are deliberately absent, not forgotten:
+ *
+ * - `open_conflicts` needs a semantic-health run
+ * (`health/reconcile.ts`) to produce a `SemanticHealthReport`; that
+ * report is not already computed on this path and running it here
+ * would make every `second_brain_status` call pay for a health
+ * pass. Measured too expensive for a status snapshot; callers that
+ * want it run `brain_health` / the reconcile report directly.
+ * - `pending_triggers` was investigated against
+ * `src/core/brain/triggers/store.ts`. Its only readers,
+ * `readTriggers`/`listTriggers`, fully parse frontmatter AND every
+ * body section of EVERY `.md` file under `Brain/triggers/` before a
+ * status filter is even applied — there is no status-only or
+ * count-only accessor — and terminal triggers (acted / dismissed /
+ * expired / suppressed) are never pruned from that directory, so
+ * the read cost grows with the vault's entire trigger history, not
+ * with the currently-open count. Not the cheap count this block
+ * requires; measured too expensive, not forgotten.
*/
import { existsSync, readdirSync } from "node:fs";
@@ -38,12 +81,45 @@ export interface BrainStatusCounts {
readonly snapshots: number;
}
+/**
+ * Whether `log_events_since_dream` is counted against a real dream
+ * boundary or reports the whole log because no boundary exists yet.
+ * `never_dreamed` is a distinct state from "zero events" — see the
+ * module docblock.
+ */
+export const MAINTENANCE_DEBT_STATUS = Object.freeze({
+ neverDreamed: "never_dreamed",
+ counted: "counted",
+ /**
+ * At least one log shard in the counted window could not be read, or
+ * held lines the reader could not parse, so the figure is a LOWER
+ * BOUND. Distinct from `counted` because the number an operator acts
+ * on is the one place a partial walk must not pass for a total.
+ */
+ undercounted: "undercounted",
+} as const);
+
+export type MaintenanceDebtStatus =
+ (typeof MAINTENANCE_DEBT_STATUS)[keyof typeof MAINTENANCE_DEBT_STATUS];
+
+/** See the "`maintenance_debt`" section of this module's docblock. */
+export interface MaintenanceDebt {
+ readonly status: MaintenanceDebtStatus;
+ readonly log_events_since_dream: number;
+}
+
+const NEVER_DREAMED_DEBT: MaintenanceDebt = Object.freeze({
+ status: MAINTENANCE_DEBT_STATUS.neverDreamed,
+ log_events_since_dream: 0,
+});
+
export interface BrainStatusSnapshot {
/** Whether `<vault>/Brain/` exists at all. */
readonly present: boolean;
readonly counts: BrainStatusCounts;
readonly last_dream_at: string | null;
readonly last_apply_evidence_at: string | null;
+ readonly maintenance_debt: MaintenanceDebt;
readonly sanity: {
/**
* Number of signals in `inbox/` whose `created_at` predates
@@ -81,6 +157,7 @@ export function computeBrainStatus(
},
last_dream_at: null,
last_apply_evidence_at: null,
+ maintenance_debt: NEVER_DREAMED_DEBT,
sanity: { signals_awaiting_dream: 0 },
});
}
@@ -88,16 +165,65 @@ export function computeBrainStatus(
const counts = countArtifacts(vault);
const { lastDreamAt, lastApplyEvidenceAt } = scanLogTimestamps(vault);
const signalsAwaitingDream = countSignalsAwaitingDream(vault, opts.now ?? new Date());
+ const maintenanceDebt = computeMaintenanceDebt(vault, lastDreamAt);
return Object.freeze({
present: true,
counts,
last_dream_at: lastDreamAt,
last_apply_evidence_at: lastApplyEvidenceAt,
+ maintenance_debt: maintenanceDebt,
sanity: Object.freeze({ signals_awaiting_dream: signalsAwaitingDream }),
});
}
+/**
+ * Cheap, bounded proxy for `maintenance_debt` fit for the always-loaded
+ * `brain_context` path.
+ *
+ * Cost bound: at most one {@link listLogDates} call (a `readdirSync` of
+ * `Brain/log/` plus filename parsing — no file content is read) and, if
+ * any date exists, one {@link readLogDay} call for the SINGLE newest
+ * date. That is O(1) in the number of log days regardless of how much
+ * history the vault holds — `computeMaintenanceDebt` below, in
+ * contrast, walks every day at or after `last_dream_at` (all of them,
+ * for a never-dreamed vault) and MUST NOT run on this path.
+ *
+ * Returns `null` when there is no log history yet, when the newest day's
+ * shard(s) exist but could not be read, and when the newest day holds no
+ * readable event at all — all three are "unknown", never a false
+ * `false` and never a `true` invented out of an empty or unparsable
+ * file. Returns `true` when the newest log day has at least one event
+ * and none of them is a `dream` event, `false` when it has a `dream`
+ * event. A `dream` event is decisive even beside a warning: it was read,
+ * and nothing unread can unsay it. This is a same-day signal only: a vault that
+ * dreamed yesterday and logged something unrelated today still reads
+ * `true`. Callers that need the precise count read
+ * `maintenance_debt.log_events_since_dream` from `computeBrainStatus`
+ * instead (`second_brain_status` / `osb://status`).
+ */
+export function computeMaintenanceOverdueFlag(vault: string): boolean | null {
+ const dates = listLogDates(vault);
+ const newest = dates.at(-1);
+ if (newest === undefined) return null;
+ let day;
+ try {
+ day = readLogDay(vault, newest);
+ } catch {
+ // Same tolerance as `scanLogTimestamps`: an unreadable day is
+ // reported as unknown, not silently folded into a boolean answer.
+ return null;
+ }
+ if (day.entries.some((e) => e.eventType === BRAIN_LOG_EVENT_KIND.dream)) return false;
+ // No dream event among the events that were read. That is only an
+ // answer when the read was whole and had something in it: an empty day
+ // and a day whose bytes did not parse are both "nothing to judge", and
+ // `true` there would report overdue maintenance out of a file nobody
+ // could read.
+ if (day.entries.length === 0 || day.warnings.length > 0) return null;
+ return true;
+}
+
// ----- Implementation ------------------------------------------------------
function countArtifacts(vault: string): BrainStatusCounts {
@@ -208,6 +334,73 @@ function scanLogTimestamps(vault: string): {
return { lastDreamAt, lastApplyEvidenceAt };
}
+/**
+ * Derive `maintenance_debt` from the JSONL log shards, AFTER
+ * `last_dream_at` has already been resolved by {@link scanLogTimestamps}.
+ * No stored counter, no STATE_SURFACES row — see the module docblock.
+ *
+ * `lastDreamAt === null` (never dreamed) counts every event ever
+ * logged, because there is no dream boundary to count "since". A real
+ * `last_dream_at` counts only events with a strictly later timestamp —
+ * and a shard in that window that could not be read, or a line in it
+ * that could not be parsed, makes the answer `undercounted`: the number
+ * is then a floor, and the sibling flag's discipline (never a confident
+ * answer over an incomplete read) applies to the precise figure too.
+ *
+ * the dream event itself is the boundary, not a countable event — and
+ * walks days newest-first, stopping as soon as a day predates the
+ * dream's own day, so the scan is bounded to "since the last dream",
+ * not the whole log history, on every vault that HAS dreamed.
+ */
+function computeMaintenanceDebt(vault: string, lastDreamAt: string | null): MaintenanceDebt {
+ const dirs = brainDirs(vault);
+ if (!existsSync(dirs.log)) return NEVER_DREAMED_DEBT;
+
+ const cutoffMs = lastDreamAt !== null ? Date.parse(lastDreamAt) : null;
+ const cutoffDate = lastDreamAt !== null ? lastDreamAt.slice(0, 10) : null;
+ const days = listLogDates(vault).toReversed(); // newest day first
+
+ let count = 0;
+ let complete = true;
+ for (const date of days) {
+ // Every earlier day is entirely pre-dream once we're past the
+ // dream's own day — no need to open its shards at all.
+ if (cutoffDate !== null && date < cutoffDate) break;
+ let entries;
+ try {
+ const day = readLogDay(vault, date);
+ entries = day.entries;
+ // A shard that would not open, or a line that would not parse:
+ // either way events this window contains are absent from `count`,
+ // and the figure below is a floor rather than the count it looks
+ // like. Recorded, not aborted — the events that WERE read are
+ // still the operator's best available number.
+ if (day.warnings.length > 0) complete = false;
+ } catch {
+ // Same tolerance as scanLogTimestamps: skip the day rather than
+ // abort the snapshot — but never silently, see above.
+ complete = false;
+ continue;
+ }
+ for (const e of entries) {
+ if (cutoffMs !== null) {
+ const ts = Date.parse(e.timestamp);
+ if (!Number.isFinite(ts) || ts <= cutoffMs) continue;
+ }
+ count++;
+ }
+ }
+
+ return Object.freeze({
+ status: !complete
+ ? MAINTENANCE_DEBT_STATUS.undercounted
+ : lastDreamAt === null
+ ? MAINTENANCE_DEBT_STATUS.neverDreamed
+ : MAINTENANCE_DEBT_STATUS.counted,
+ log_events_since_dream: count,
+ });
+}
+
function countSignalsAwaitingDream(vault: string, now: Date): number {
const dirs = brainDirs(vault);
if (!existsSync(dirs.inbox)) return 0;
diff --git a/src/core/brain/types.ts b/src/core/brain/types.ts
index 453adca2..db94fbbe 100644
--- a/src/core/brain/types.ts
+++ b/src/core/brain/types.ts
@@ -11,13 +11,17 @@
* sections 5 (file formats) and 10 (configuration).
*/
-// The one import this module carries, and it is TYPE-ONLY: `GateMode`
-// is owned by the integrity seam that acts on it, so the `_brain.yaml`
-// shape cannot drift from the vocabulary the gate actually understands.
-// `verbatimModuleSyntax` erases this entirely, and `stamp.ts` imports
-// nothing itself, so the "plain-data leaf with no runtime dependency"
-// property this module relies on is unchanged.
+// The only imports this module carries, and both are TYPE-ONLY, for the
+// same reason: each vocabulary is owned by the seam that acts on it, so
+// the shapes declared here cannot drift from the values that seam
+// understands. `GateMode` belongs to the integrity stamp; the origin
+// channel belongs to the resolver the entry points claim it through.
+// `verbatimModuleSyntax` erases both entirely, and neither `stamp.ts`
+// nor `origin-channel.ts` imports anything itself, so the "plain-data
+// leaf with no runtime dependency" property this module relies on is
+// unchanged.
import type { GateMode } from "../integrity/stamp.ts";
+import type { OriginChannelStamp } from "../origin-channel.ts";
// ----- Status & retire-reason enums -----------------------------------------
//
@@ -776,6 +780,16 @@ export interface BrainSignal {
* mirrored record from a local one.
*/
readonly origin_vault?: string;
+ /**
+ * Server-derived channel of the process that wrote this signal (Unit C
+ * of the nothing-writes-silently wave): `mcp-tool`, `cli`, `import`, or
+ * the explicit `unset` when no entry point claimed the process. Never
+ * caller-supplied — see `src/core/origin-channel.ts`. A SIBLING of
+ * {@link BrainSignal.source_type}, not a spelling of it; the mapping
+ * table lives on `writeSignal`. Absent on every signal written before
+ * the stamp shipped, and those are never rewritten to add one.
+ */
+ readonly origin_channel?: OriginChannelStamp;
/** Bi-temporal event-time start (additive optional, v0.10.18). */
readonly valid_from?: string;
/** Bi-temporal event-time end (additive optional, v0.10.18). */
diff --git a/src/core/brain/write-batch.ts b/src/core/brain/write-batch.ts
index 7227f118..9acfb893 100644
--- a/src/core/brain/write-batch.ts
+++ b/src/core/brain/write-batch.ts
@@ -32,7 +32,9 @@ import { dirname } from "node:path";
import type { FrontmatterMap } from "../types.ts";
import { atomicWriteFileSync } from "../fs-atomic.ts";
import { CreateNoteError, createNote, resolveNoteTarget } from "./notes/create-note.ts";
-import { formatFrontmatter, parseFrontmatter } from "../vault.ts";
+import { refuseBlankOverwrite } from "./notes/blank-overwrite-guard.ts";
+import { formatFrontmatter, parseFrontmatterWithNotices } from "../vault.ts";
+import { DEGRADATION_CODE } from "../integrity/degradation.ts";
import {
appendApplyEvidence,
type AppendApplyEvidenceInput,
@@ -76,6 +78,12 @@ export interface UpdateNoteOperation {
readonly path: string;
readonly frontmatter?: FrontmatterMap;
readonly body?: string;
+ /**
+ * Permit `body` to be blank, clearing the note's contents. Default
+ * false: see {@link refuseBlankOverwrite}, which exists because an
+ * accidentally-empty body is byte-identical to a deliberate clear.
+ */
+ readonly allowEmpty?: boolean;
}
/** Append `content` to the body of an EXISTING note. */
@@ -121,6 +129,28 @@ export type WriteBatchErrorCode =
| "outside_vault"
| "exists"
| "target_missing"
+ // nothing-writes-silently, unit B. The target exists and the process
+ // could not read it (a permission bit, a directory in its place, a
+ // transient I/O fault, a file mid-sync). Until this code existed the
+ // parser resolved that to `[{}, ""]` and the projection believed it:
+ // an update then wrote the caller's body over an empty frontmatter
+ // map and reported `updated: true`, and an append wrote just the
+ // appended text over a body it never saw. Read failure is now a
+ // refusal that names the path and the reason.
+ | "target_unreadable"
+ // nothing-writes-silently, unit B. The target was read and the
+ // frontmatter scanner reported a line it could not express as a
+ // key/value pair or a list item, so that line is absent from the parsed
+ // map. Both update and append re-serialise that map over the file, so
+ // proceeding would delete the line and answer `updated: true` - the
+ // same loss `target_unreadable` refuses, on the half of the read the
+ // parser DID manage. The operator's way through is to fix the
+ // frontmatter the scanner names.
+ | "target_frontmatter_lossy"
+ // nothing-writes-silently, unit B. The update would replace a body
+ // that carries text with a blank one. See
+ // {@link refuseBlankOverwrite}; `allowEmpty` is the way through.
+ | "blank_overwrite_refused"
| "duplicate_target"
| "too_many_operations"
| "preference_not_found"
@@ -352,6 +382,25 @@ function projectUpdateNote(
}
const target = reserveNoteTarget(vault, op.path, index, noteTargets);
const state = readExistingNote(target.abs, target.relPath, index);
+ // The one seam the blank-overwrite guard is wired at - it covers both
+ // callers, `brain_update_note` and `brain_write_batch`'s update op,
+ // because both project through here.
+ if (
+ op.body !== undefined &&
+ refuseBlankOverwrite({
+ existingBody: state.body,
+ nextBody: op.body,
+ allowEmpty: op.allowEmpty === true,
+ })
+ ) {
+ throw new WriteBatchError(
+ "blank_overwrite_refused",
+ index,
+ `operation ${index}: refusing to replace the body of ${target.relPath} with an empty ` +
+ "one; pass allow_empty to clear the note deliberately",
+ { path: target.relPath },
+ );
+ }
const frontmatter =
op.frontmatter !== undefined ? { ...state.frontmatter, ...op.frontmatter } : state.frontmatter;
const body = op.body !== undefined ? op.body : state.body;
@@ -512,9 +561,31 @@ interface ExistingNote {
readonly body: string;
}
+/** Attribution for the frontmatter notices this reader inspects. */
+const READ_EXISTING_NOTE_SITE = "brain.write-batch.read-existing-note";
+
/**
- * Read and parse an existing note, or throw a typed `target_missing`
- * error. update and append only touch notes that already exist.
+ * Read and parse an existing note, or throw a typed error naming which
+ * of the two ways it was unavailable. update and append only touch
+ * notes that already exist.
+ *
+ * An UNREADABLE file raises rather than resolving to an empty note. The
+ * two-tuple `parseFrontmatter` reports a read failure as `[{}, ""]` -
+ * correct for the fail-soft walkers it was written for, and fatal here,
+ * because this result is the base of a read-modify-write: an update
+ * would write the caller's body under an empty frontmatter map and an
+ * append would write its text over a body nobody read, both reporting
+ * success. The projection runs before any commit, so raising leaves the
+ * file byte-identical.
+ *
+ * A DROPPED frontmatter line raises for the same reason. The parse
+ * succeeded, and the scanner reported by name a line it could not
+ * express; the callers below re-serialise the parsed map with
+ * `formatFrontmatter`, so that line would be deleted from disk by a
+ * write the caller never asked to touch it - a frontmatter-only update
+ * that answers `updated: true` while removing a field it never mentioned.
+ * Both notices come off the same parse, and both mean the same thing:
+ * part of the content this write would replace is unknown to it.
*/
function readExistingNote(abs: string, relPath: string, index: number): ExistingNote {
if (!existsSync(abs)) {
@@ -525,6 +596,29 @@ function readExistingNote(abs: string, relPath: string, index: number): Existing
{ path: relPath },
);
}
- const [frontmatter, body] = parseFrontmatter(abs);
+ const [frontmatter, body, notices] = parseFrontmatterWithNotices(abs, {
+ site: READ_EXISTING_NOTE_SITE,
+ });
+ const unreadable = notices.find((n) => n.code === DEGRADATION_CODE.frontmatterUnreadable);
+ if (unreadable !== undefined) {
+ throw new WriteBatchError(
+ "target_unreadable",
+ index,
+ `operation ${index}: note ${relPath} exists but could not be read ` +
+ `(${unreadable.detail}), so the content this write would replace is unknown`,
+ { path: relPath, reason: unreadable.detail },
+ );
+ }
+ const dropped = notices.filter((n) => n.code === DEGRADATION_CODE.frontmatterLineDropped);
+ if (dropped.length > 0) {
+ const reason = dropped.map((n) => n.detail).join("; ");
+ throw new WriteBatchError(
+ "target_frontmatter_lossy",
+ index,
+ `operation ${index}: note ${relPath} has frontmatter this build cannot round-trip ` +
+ `(${reason}), and rewriting it would delete those lines`,
+ { path: relPath, reason },
+ );
+ }
return { frontmatter: { ...frontmatter }, body };
}
diff --git a/src/core/brain/write-session/validate.ts b/src/core/brain/write-session/validate.ts
index fac58b69..c341cd7d 100644
--- a/src/core/brain/write-session/validate.ts
+++ b/src/core/brain/write-session/validate.ts
@@ -18,6 +18,7 @@ import { createHash } from "node:crypto";
import { existsSync, readFileSync, statSync } from "node:fs";
import { join, normalize, sep } from "node:path";
+import { ORIGIN_CHANNEL_FIELD } from "../../origin-channel.ts";
import { parseFrontmatterText } from "../../vault.ts";
import { isKnownSchemaToken, type BrainSchemaVocabulary } from "../schema-vocab.ts";
import type { ExistingTargetInfo, WriteSessionError } from "./types.ts";
@@ -95,6 +96,14 @@ export interface ValidateArtifactOptions {
readonly vocabulary?: BrainSchemaVocabulary;
}
+/**
+ * Frontmatter keys the AUTHOR of a document put there — every key except
+ * the server-derived stamps no caller supplies and no caller can remove.
+ */
+function authoredFrontmatterKeyCount(meta: Readonly<Record<string, unknown>>): number {
+ return Object.keys(meta).filter((key) => key !== ORIGIN_CHANNEL_FIELD).length;
+}
+
/**
* Validate a submitted artifact body. Order matters: cheap structural
* checks first so the error list reads top-down like a fix list.
@@ -125,7 +134,12 @@ export function validateArtifact(
errors.push(err("frontmatter-malformed", "frontmatter", (exc as Error).message));
return Object.freeze(errors);
}
- if (Object.keys(meta).length === 0) {
+ // "No keys" means no AUTHORED keys. The origin channel is stamped by
+ // `createNote` on every note it writes (Unit C), so counting it here
+ // would make this violation unreachable from the surface that produces
+ // most of the documents it judges - a check that cannot fire is worse
+ // than no check, because the receipt still says the lint ran.
+ if (authoredFrontmatterKeyCount(meta) === 0) {
errors.push(err("frontmatter-missing", "frontmatter", "frontmatter block has no keys"));
return Object.freeze(errors);
}
diff --git a/src/core/fs-utils.ts b/src/core/fs-utils.ts
index c18767a8..fa4ff5ae 100644
--- a/src/core/fs-utils.ts
+++ b/src/core/fs-utils.ts
@@ -69,3 +69,16 @@ export function stem(filename: string): string {
const dot = filename.lastIndexOf(".");
return dot > 0 ? filename.slice(0, dot) : filename;
}
+
+/**
+ * The {@link stem} of a POSIX vault-relative path's LAST segment.
+ * `pathStem("notes/a/b.md")` -> `"b"`.
+ *
+ * Shared rather than spelled twice: `stem` alone applied to a path keeps
+ * the folder segments (`"notes/a/b"`), which is not a wikilink target and
+ * is not a basename either - it is a value that looks close enough to
+ * both to pass a review and fail a lookup.
+ */
+export function pathStem(relPath: string): string {
+ return stem(relPath.slice(relPath.lastIndexOf("/") + 1));
+}
diff --git a/src/core/origin-channel.ts b/src/core/origin-channel.ts
new file mode 100644
index 00000000..930aaa73
--- /dev/null
+++ b/src/core/origin-channel.ts
@@ -0,0 +1,207 @@
+/**
+ * The origin channel — which kind of process put a record in the vault.
+ *
+ * ## Server-derived, and that is the whole design
+ *
+ * The channel is claimed ONCE per process by the entry point that knows
+ * what the process is: the MCP server transport, the CLI dispatcher, and
+ * the import verbs' dispatch. It is never a tool argument and never a
+ * CLI flag, and `tests/core/architecture/origin-channel-census.test.ts`
+ * asserts that rather than trusting this paragraph.
+ *
+ * The reason is measured, not stylistic. Agent identity in this codebase
+ * resolves to a caller-supplied `agent` string that 25 MCP tool schemas
+ * accept verbatim (the write binding's docblock documents the recount and
+ * the command to redo it). A channel a caller could name would be the
+ * same unverifiable claim wearing a schema — a record saying `cli` would
+ * mean "someone typed cli", which is not provenance. So the value comes
+ * from the process, or it comes back as {@link ORIGIN_CHANNEL_UNSET}.
+ *
+ * ## Why an `unset` literal rather than a guessed default
+ *
+ * {@link resolveOriginChannel} FAILS LOUD when nothing claimed the
+ * process: reading a channel that no entry point set is a programming
+ * error and the refusal names itself and the setter. The record writers
+ * do not use that read. They use {@link originChannelStamp}, which
+ * absorbs the unclaimed state into the explicit `unset` literal and
+ * stamps THAT.
+ *
+ * This mirrors a decision already made one layer down, deliberately.
+ * `appendLogEvent` absorbs a `resolveDeviceId` failure into the empty
+ * shard id (`AppendLogEventOptions.deviceId`) because appending is the
+ * always-on write path behind every hook, and failing it turns one bad
+ * file mode into a dead session. The same argument holds here for all
+ * four stamped families, and the absorbed value obeys the same rule the
+ * device-id fallback does: it is a DOCUMENTED value with its own
+ * meaning, not a neutral-looking invention. `unset` serializes as
+ * `unset`, is outside {@link ORIGIN_CHANNELS}, and reads on disk as
+ * exactly what it is — no entry point claimed this process. A guessed
+ * `cli` would have been the lie this unit exists to remove.
+ *
+ * ## No backfill
+ *
+ * Records written before this shipped carry no channel and are NOT
+ * rewritten to add one. Rewriting history to add provenance is the
+ * opposite of provenance: the value stamped on a backfill pass would
+ * describe the backfill process, not the process that wrote the record.
+ * Absence of the key means "written before the stamp existed", and every
+ * reader treats it that way.
+ *
+ * ## Three adjacent axes this is NOT
+ *
+ * - `ProvenanceLevel` (`brain/provenance/provenance.ts`) is a TRUST
+ * band — `stated` > `deduced` > `inferred` — about how a claim was
+ * arrived at. A channel is not a trust band: an `mcp-tool` record can
+ * be `inferred` and a `cli` record can be `stated`.
+ * - `classifySourceOrigin` (`brain/intake/source-trust.ts`) is an
+ * INBOUND-SOURCE axis about the file an intake read. A channel says
+ * nothing about the source and everything about the writer.
+ * - `RECALL_CHANNEL` (`brain/recall-telemetry.ts`) is the nearest one
+ * and the one most worth being explicit about, because its members
+ * (`mcp` / `cli` / `hook`) nearly rhyme with these. It measures the
+ * transport a recall was DELIVERED over — the read direction — and it
+ * is caller-supplied by design, being a property of a delivery the
+ * caller performed. This is the WRITE direction and is derived rather
+ * than supplied. Two records can disagree on them honestly: a hook
+ * delivers a recall (`recall_channel: hook`) inside a process that
+ * writes its telemetry over `origin_channel: cli`. Folding either
+ * into the other would make one of those two facts unrecordable.
+ *
+ * None may be folded into this one, and none may absorb it.
+ */
+
+/**
+ * The closed vocabulary. Three members because three process shapes can
+ * put a record in the vault, and each one is a different answer to "who
+ * would I ask about this record":
+ *
+ * - `mcp-tool` — an agent called a tool over the MCP transport.
+ * - `cli` — an operator (or a script) ran `o2b`.
+ * - `import` — a bulk replay of records authored somewhere else.
+ *
+ * Spellings are on-disk values and are load-bearing; the hyphen in
+ * `mcp-tool` matches the `apply-evidence` shape the log-event vocabulary
+ * already uses.
+ */
+export const ORIGIN_CHANNEL = Object.freeze({
+ mcpTool: "mcp-tool",
+ cli: "cli",
+ import: "import",
+} as const);
+
+export type OriginChannel = (typeof ORIGIN_CHANNEL)[keyof typeof ORIGIN_CHANNEL];
+
+/**
+ * The membership list, exported so a refusal can NAME the vocabulary
+ * instead of restating it — the same rule `BRAIN_SIGNAL_SOURCE_TYPES`
+ * follows, and for the same reason: a hand-written list in a message
+ * goes stale on the first new member and teaches the wrong contract.
+ */
+export const ORIGIN_CHANNELS: ReadonlyArray<OriginChannel> = Object.freeze(
+ Object.values(ORIGIN_CHANNEL),
+);
+
+/**
+ * What a record carries when no entry point claimed the process.
+ * Deliberately OUTSIDE {@link ORIGIN_CHANNELS} so a reader can tell it
+ * apart from every real answer without knowing this module.
+ */
+export const ORIGIN_CHANNEL_UNSET = "unset";
+
+/** What a stamped record actually holds: a channel, or the unset literal. */
+export type OriginChannelStamp = OriginChannel | typeof ORIGIN_CHANNEL_UNSET;
+
+/**
+ * The on-disk key. One constant because three of the four stamped
+ * families spell it in a text surface — a log bullet, a signal
+ * frontmatter line, a note frontmatter line — and a key that drifts
+ * between them is a key no reader can query on. The continuity record is
+ * the exception and uses `originChannel`, because its record shape is
+ * camelCase JSON throughout (`createdAt`, `sourceRefs`) and a lone
+ * snake_case field there would be the drift, not the consistency.
+ */
+export const ORIGIN_CHANNEL_FIELD = "origin_channel";
+
+export function isOriginChannel(value: unknown): value is OriginChannel {
+ return typeof value === "string" && (ORIGIN_CHANNELS as ReadonlyArray<string>).includes(value);
+}
+
+export function isOriginChannelStamp(value: unknown): value is OriginChannelStamp {
+ return value === ORIGIN_CHANNEL_UNSET || isOriginChannel(value);
+}
+
+/** Raised by {@link resolveOriginChannel} when nothing claimed the process. */
+export class OriginChannelUnsetError extends Error {
+ constructor() {
+ super(
+ "resolveOriginChannel: no entry point claimed an origin channel for this process; " +
+ "the MCP transport, the CLI dispatcher and the import verbs each call " +
+ `setOriginChannel(...) with one of ${ORIGIN_CHANNELS.join(", ")}. Record writers ` +
+ `must call originChannelStamp() instead, which stamps '${ORIGIN_CHANNEL_UNSET}'.`,
+ );
+ this.name = "OriginChannelUnsetError";
+ }
+}
+
+/** Process-scoped, module-private. `null` until an entry point claims it. */
+let claimed: OriginChannel | null = null;
+
+/**
+ * Claim the channel for this process. Called by the entry point, before
+ * any dispatch.
+ *
+ * Re-claiming is allowed and the LAST claim wins. This is not laxity: a
+ * single OS process re-enters the entry point routinely — the CLI test
+ * harness calls `main()` many times, and an embedded host can start the
+ * MCP transport inside a process that already ran a command. Refusing
+ * the second claim would fail those runs over a value that is correct;
+ * refusing to let a MID-FLIGHT caller set it is what actually matters,
+ * and that is enforced by there being no argument and no flag that
+ * reaches this function, which the origin-channel census asserts.
+ */
+export function setOriginChannel(channel: OriginChannel): void {
+ if (!isOriginChannel(channel)) {
+ throw new Error(
+ `setOriginChannel: ${JSON.stringify(channel)} is not an origin channel; ` +
+ `expected one of ${ORIGIN_CHANNELS.join(", ")}`,
+ );
+ }
+ claimed = channel;
+}
+
+/**
+ * Return the process to unclaimed. The counterpart of
+ * {@link setOriginChannel} for a harness that hands the process back
+ * between runs; production entry points claim and never release.
+ */
+export function clearOriginChannel(): void {
+ claimed = null;
+}
+
+/**
+ * The strict read: the channel this process claimed, or a refusal.
+ *
+ * Use it where a wrong answer is worse than no answer. Record writers
+ * want {@link originChannelStamp} instead — see the module docblock for
+ * why the always-on write path absorbs this condition rather than
+ * propagating it.
+ */
+export function resolveOriginChannel(): OriginChannel {
+ if (claimed === null) throw new OriginChannelUnsetError();
+ return claimed;
+}
+
+/**
+ * The record-writer read: the claimed channel, or the explicit
+ * {@link ORIGIN_CHANNEL_UNSET} literal. Never throws, never guesses.
+ */
+export function originChannelStamp(): OriginChannelStamp {
+ try {
+ return resolveOriginChannel();
+ } catch {
+ // The documented absorb. See the module docblock: a write path that
+ // fails over its own provenance stamp costs the record it was
+ // stamping, and `unset` loses nothing that was ever known.
+ return ORIGIN_CHANNEL_UNSET;
+ }
+}
diff --git a/src/core/reconciliation-report.ts b/src/core/reconciliation-report.ts
new file mode 100644
index 00000000..30e2f5fc
--- /dev/null
+++ b/src/core/reconciliation-report.ts
@@ -0,0 +1,206 @@
+/**
+ * The wave's shared reconciliation vocabulary (nothing-writes-silently).
+ *
+ * Three unrelated lanes each need to answer the same question - "of what
+ * we attempted, what did we actually get, and what fell out?" - without
+ * inventing three spellings of it: the import read-back census (what a
+ * sessions import claimed to write against what a post-import scan can
+ * still find), the multi-artifact envelope write accounting (rollup /
+ * diarization / design-note lanes that commit several artifacts from one
+ * validated payload), and the embedder record-vs-data audit's third state
+ * (what a recorded `index_state` row claims against what the vector store
+ * actually holds). This module is that one shape, pure and dependency-free
+ * so all three can import it without pulling in any lane's I/O.
+ *
+ * The rule the whole module exists to enforce: `missing` NAMES its keys.
+ * There is no constructor here that accepts a bare count in place of the
+ * array - `attempted: 40, found: 38` with no further detail is the
+ * misleading no-op this wave removes, because it reads as accounted for
+ * when two items simply vanished without anyone able to say which. Every
+ * numeric input this module DOES accept (`attempted`, `found`) is
+ * validated for exactly that reason: an unvalidated count is how a NaN or
+ * a negative slips through the accounting and reports a lie as a number.
+ */
+
+/** One reconciliation report: what was attempted, found, and left out. */
+export interface ReconciliationReport {
+ /** Total items the operation attempted to write, read, or verify. */
+ readonly attempted: number;
+ /** Items confirmed present, written, or matching. */
+ readonly found: number;
+ /** Keys naming every item accounted for by neither of the above. */
+ readonly missing: ReadonlyArray<string>;
+}
+
+/** Input to {@link buildReconciliationReport}, before validation. */
+export interface ReconciliationReportInput {
+ readonly attempted: number;
+ readonly found: number;
+ readonly missing: ReadonlyArray<string>;
+}
+
+/** Machine-readable reason {@link buildReconciliationReport} refused. */
+export type ReconciliationReportErrorCode =
+ | "invalid_count"
+ | "found_exceeds_attempted"
+ | "count_mismatch"
+ | "duplicate_missing_key";
+
+/**
+ * A reconciliation report could not be built from the given numbers. Every
+ * refusal names a `code` a caller can branch on and `details` a caller can
+ * log without re-deriving them from the message string.
+ */
+export class ReconciliationReportError extends Error {
+ readonly code: ReconciliationReportErrorCode;
+ readonly details: Readonly<Record<string, unknown>>;
+
+ constructor(
+ code: ReconciliationReportErrorCode,
+ message: string,
+ details: Readonly<Record<string, unknown>> = {},
+ ) {
+ super(message);
+ this.name = "ReconciliationReportError";
+ this.code = code;
+ this.details = details;
+ }
+}
+
+/** Refuses anything that is not a non-negative integer, naming the field. */
+function requireCount(field: "attempted" | "found", value: number): void {
+ if (!Number.isInteger(value) || value < 0) {
+ throw new ReconciliationReportError(
+ "invalid_count",
+ `reconciliation report: ${field} must be a non-negative integer, got ${value}`,
+ { field, value },
+ );
+ }
+}
+
+/**
+ * Validate and freeze one reconciliation report. Refuses, by name:
+ *
+ * - `attempted` or `found` that is NaN, negative, or non-integer
+ * (`invalid_count`).
+ * - `found` greater than `attempted` (`found_exceeds_attempted`) - a
+ * nonsensical claim independent of `missing`, checked before the
+ * accounting below so it is never masked by a `count_mismatch` instead.
+ * - `attempted !== found + missing.length` (`count_mismatch`) - the named
+ * keys do not account for the gap between attempted and found.
+ * - a key repeated inside `missing` (`duplicate_missing_key`) - a
+ * duplicate cannot both name a distinct absence and keep the count
+ * honest.
+ *
+ * `missing` is copied before freezing so the returned report cannot change
+ * under a reference the caller kept.
+ */
+export function buildReconciliationReport(input: ReconciliationReportInput): ReconciliationReport {
+ const { attempted, found, missing } = input;
+ requireCount("attempted", attempted);
+ requireCount("found", found);
+
+ if (found > attempted) {
+ throw new ReconciliationReportError(
+ "found_exceeds_attempted",
+ `reconciliation report: found (${found}) exceeds attempted (${attempted})`,
+ { attempted, found },
+ );
+ }
+
+ const seen = new Set<string>();
+ for (const key of missing) {
+ if (seen.has(key)) {
+ throw new ReconciliationReportError(
+ "duplicate_missing_key",
+ `reconciliation report: missing names "${key}" more than once`,
+ { key, missing },
+ );
+ }
+ seen.add(key);
+ }
+
+ if (attempted !== found + missing.length) {
+ throw new ReconciliationReportError(
+ "count_mismatch",
+ `reconciliation report: attempted (${attempted}) does not equal found (${found}) + missing.length (${missing.length})`,
+ { attempted, found, missingCount: missing.length },
+ );
+ }
+
+ return Object.freeze({ attempted, found, missing: Object.freeze([...missing]) });
+}
+
+/**
+ * Closed outcome vocabulary derived from a reconciliation report.
+ *
+ * - `complete`: everything attempted was found; `missing` is empty.
+ * - `partial`: `missing` names at least one key and no recorded claim
+ * contradicts the report's own numbers.
+ * - `contradicted`: a recorded claim - a value some OTHER record asserts,
+ * such as an `index_state` row's declared dimension - disagrees with
+ * what this report itself measured. This is the embedder audit's third
+ * state: not "some items are missing" but "the record about them is
+ * itself wrong", which is worse than a plain shortfall because it is
+ * evidence of nothing rather than evidence of a gap.
+ */
+export const RECONCILIATION_OUTCOME = Object.freeze({
+ complete: "complete",
+ partial: "partial",
+ contradicted: "contradicted",
+});
+export type ReconciliationOutcome =
+ (typeof RECONCILIATION_OUTCOME)[keyof typeof RECONCILIATION_OUTCOME];
+
+/**
+ * The membership list, exported so a refusal or a schema can name the
+ * vocabulary instead of restating it by hand.
+ */
+export const RECONCILIATION_OUTCOMES: ReadonlyArray<ReconciliationOutcome> = Object.freeze(
+ Object.values(RECONCILIATION_OUTCOME),
+);
+
+/** Type-guard for the outcome union - used by writers and parsers alike. */
+export function isReconciliationOutcome(value: unknown): value is ReconciliationOutcome {
+ return (
+ typeof value === "string" && (RECONCILIATION_OUTCOMES as ReadonlyArray<string>).includes(value)
+ );
+}
+
+/**
+ * What some other record claims about the same numbers a report measured
+ * - e.g. an `index_state` row's declared `attempted`/`found` before this
+ * report re-measured them from the vector store directly. Either field
+ * may be omitted when that half of the claim was never recorded; an
+ * omitted field cannot contradict anything.
+ */
+export interface ReconciliationRecordedClaim {
+ readonly attempted?: number;
+ readonly found?: number;
+}
+
+/**
+ * Derive the closed outcome for a report, deterministically. `missing`
+ * alone decides `complete` versus `partial`; a `recordedClaim` that
+ * disagrees with the report's own `attempted` or `found` always wins as
+ * `contradicted`, even over an otherwise-clean report, because a record
+ * asserting numbers the data itself disproves is not "complete with a
+ * stale label" - it is the third state this vocabulary exists to name.
+ */
+export function deriveReconciliationOutcome(
+ report: ReconciliationReport,
+ recordedClaim?: ReconciliationRecordedClaim,
+): ReconciliationOutcome {
+ if (recordedClaim !== undefined) {
+ const { attempted: claimedAttempted, found: claimedFound } = recordedClaim;
+ if (claimedAttempted !== undefined && claimedAttempted !== report.attempted) {
+ return RECONCILIATION_OUTCOME.contradicted;
+ }
+ if (claimedFound !== undefined && claimedFound !== report.found) {
+ return RECONCILIATION_OUTCOME.contradicted;
+ }
+ }
+ return report.missing.length === 0
+ ? RECONCILIATION_OUTCOME.complete
+ : RECONCILIATION_OUTCOME.partial;
+}
diff --git a/src/core/search/index.ts b/src/core/search/index.ts
index 9aa7fb61..1449b67a 100644
--- a/src/core/search/index.ts
+++ b/src/core/search/index.ts
@@ -49,8 +49,10 @@ export type {
ExpandHitInput,
ExpandHitResult,
IndexCheckReport,
+ EmbedderRecordCensus,
IndexStats,
IndexStatusSnapshot,
+ PendingVectorCensus,
ResolvedEmbeddingConfig,
ResolvedRecallConfig,
ResolvedRerankConfig,
@@ -63,6 +65,7 @@ export type {
StructuredRecallQueryDocument,
VaultPathRule,
VaultScopeRules,
+ VisibilityHonestyFinding,
} from "./types.ts";
export {
chunkWindowDiagnosticCode,
@@ -86,10 +89,24 @@ export { evaluateSurfacingGate, type SurfacingGateDecision } from "./surfacing-g
export { buildEvidencePack, serializeEvidencePack } from "./evidence-pack.ts";
export {
serializeChunkWindowCensus,
+ serializeEmbedderRecordCensus,
+ serializePendingVectorCensus,
serializeSearchCard,
serializeIndexStatus,
serializeStampMismatches,
+ serializeVisibilityHonestyFinding,
} from "./serialize.ts";
+export {
+ callableVisibilitySurfaces,
+ excludedCallableVisibilitySurfaces,
+ excludedVisibilitySurfaces,
+ VISIBILITY_SURFACE_CATEGORY,
+ VISIBILITY_SURFACE_KIND,
+ VISIBILITY_SURFACE_REGISTRY,
+ type VisibilitySurfaceCategory,
+ type VisibilitySurfaceEntry,
+ type VisibilitySurfaceKind,
+} from "./visibility-surface-registry.ts";
export {
loadProviderRegistry,
addProviderProfile,
diff --git a/src/core/search/indexer.ts b/src/core/search/indexer.ts
index 79c9f0e3..09b3ee4a 100644
--- a/src/core/search/indexer.ts
+++ b/src/core/search/indexer.ts
@@ -77,8 +77,13 @@ import { pathCovers } from "../vault-scope/defaults.ts";
import {
acquireWriterLock,
- EMBEDDING_ABI_FIX_COMMAND,
+ describeUnreadableIndex,
+ embeddingAbiFixCommand,
+ formatEmbedderRecordContradiction,
formatEmbeddingAbiDrift,
+ peekPendingVectorsSync,
+ peekVisibilityTagPresence,
+ readEmbedderRecordCensusSync,
readEmbeddingAbiSync,
runtimeEmbeddingAbi,
Store,
@@ -90,13 +95,20 @@ import {
import { LATEST_SCHEMA_VERSION } from "./schema.ts";
import { chunkWindowDiagnosticCode, SearchError } from "./types.ts";
import { walkVault } from "./walker.ts";
+import {
+ callableVisibilitySurfaces,
+ excludedCallableVisibilitySurfaces,
+} from "./visibility-surface-registry.ts";
import type { ChunkInput, LinkInput } from "./store.ts";
import type {
ChunkWindowCensus,
+ EmbedderRecordCensus,
IndexCheckReport,
IndexStats,
IndexStatusSnapshot,
+ PendingVectorCensus,
ResolvedSearchConfig,
+ VisibilityHonestyFinding,
} from "./types.ts";
// ─────────────────────────────────────────────────────────────────────────────
@@ -1561,6 +1573,26 @@ export async function indexCheck(
: compareStamps(recordedAbi, runtimeEmbeddingAbi(config, vecVersion));
if (embeddingAbi.length > 0) warnings.push(formatEmbeddingAbiDrift(embeddingAbi));
+ // The measured pending-vector count (nothing-writes-silently, unit A),
+ // read through the same explicit peek the ABI tokens use and for the
+ // same reason. Two `COUNT(*)`s on a read-only handle; the report is a
+ // diagnostic an operator ran, never the query path.
+ const pendingVectors = readPendingVectorCensus(config.dbPath);
+
+ // The record-vs-data embedder audit (nothing-writes-silently, unit G).
+ // Ungated by the semantic switch on purpose: it compares the index's
+ // record against the index's own data, so what this build is
+ // configured to do next cannot change the answer.
+ const embedderRecord = readEmbedderRecordCensusSync(config.dbPath);
+ const contradiction = formatEmbedderRecordContradiction(embedderRecord);
+ if (contradiction !== null) warnings.push(contradiction);
+
+ // The visibility honesty finding (nothing-writes-silently, unit H,
+ // form B). Ungated by the semantic switch, like embedderRecord beside
+ // it: this reads chunk content the indexer already wrote, regardless
+ // of whether embeddings are configured at all.
+ const visibilityHonesty = readVisibilityHonestyFinding(config.dbPath);
+
// §E.2 — Actionable hints derived from the check state.
// Rules match the design doc table; agents and operators read the
// list to know what command to run next without learning the
@@ -1571,6 +1603,8 @@ export async function indexCheck(
vecExtension,
providerProbe,
embeddingAbi,
+ pendingVectors,
+ embedderRecord,
});
return Object.freeze({
@@ -1579,6 +1613,11 @@ export async function indexCheck(
sqliteOk,
fts5Ok,
embeddingAbi,
+ pendingVectors,
+ embedderRecord,
+ // Absent, not null: a vault that has never used the field gets no
+ // key at all, so its JSON stays byte-identical to before this unit.
+ ...(visibilityHonesty === null ? {} : { visibilityHonesty }),
vecExtension,
embeddingKeyResolved,
providerProbe,
@@ -1589,24 +1628,97 @@ export async function indexCheck(
});
}
+/**
+ * The pending-vector census for one index path, with the peek's three
+ * outcomes folded into the report's two. Both non-`read` outcomes are
+ * UNRECORDED - they differ in their reason, not in what they prove -
+ * and neither is a count of zero.
+ */
+function readPendingVectorCensus(dbPath: string): PendingVectorCensus {
+ const peek = peekPendingVectorsSync(dbPath);
+ if (peek.kind === "read") {
+ return Object.freeze({
+ verdict: "measured" as const,
+ pending: peek.value.pending,
+ chunks: peek.value.chunks,
+ });
+ }
+ return Object.freeze({
+ verdict: "unrecorded" as const,
+ reason:
+ peek.kind === "absent"
+ ? `no search index at ${dbPath}`
+ : describeUnreadableIndex(dbPath, peek.detail),
+ });
+}
+
+/**
+ * The visibility honesty finding for an index path, or `null` when there
+ * is nothing to be honest ABOUT: the index does not exist, would not
+ * open, or - the common case - has never carried a `visibility:`-tagged
+ * page. Both counts are read off the registry's own exported list at
+ * call time, never hand-written, so the finding cannot drift from the
+ * census that backs it (nothing-writes-silently, unit H, form B).
+ *
+ * CALLABLE rows only. The registry also carries one `index_store` row -
+ * the fact that `chunks` stores a private page's text whatever a
+ * read-side filter hides - and counting a storage fact as a surface an
+ * operator could call would inflate a number reported to operators.
+ */
+function readVisibilityHonestyFinding(dbPath: string): VisibilityHonestyFinding | null {
+ const peek = peekVisibilityTagPresence(dbPath);
+ if (peek.kind !== "read" || !peek.value) return null;
+ return Object.freeze({
+ excludedSurfaceCount: excludedCallableVisibilitySurfaces().length,
+ totalSurfaceCount: callableVisibilitySurfaces().length,
+ });
+}
+
interface BuildRecommendationsInput {
readonly config: ResolvedSearchConfig;
readonly embeddingKeyResolved: boolean;
readonly vecExtension: "loaded" | "unavailable" | "not-attempted";
readonly providerProbe: ProviderProbeState;
readonly embeddingAbi: ReadonlyArray<StampMismatch>;
+ readonly pendingVectors: PendingVectorCensus;
+ readonly embedderRecord: EmbedderRecordCensus;
}
+/**
+ * The first-vectors recipe, unchanged from the release that wrote it -
+ * what changed is WHEN it fires. It is now reached only by an index
+ * that holds chunks and no vectors for them, which is the state the
+ * sentence has always described.
+ */
+const FIRST_VECTORS_RECOMMENDATION =
+ "Run `o2b search reindex --embeddings` to compute the first vectors, then optionally " +
+ "`o2b search reindex --cron-template` for periodic refresh.";
+
+/**
+ * The verb that prices a partial backfill. Named here rather than
+ * priced here: `planVectorBackfill`'s dry run already counts the tokens
+ * and applies the model's rate, and a second estimator on this surface
+ * could quote a different number for the same work.
+ */
+const VECTOR_BACKFILL_COMMAND = "o2b search vector-backfill";
+
function buildRecommendations(input: BuildRecommendationsInput): string[] {
const recs: string[] = [];
+ // A record the data itself disproves comes first: every other hint
+ // below reasons from what the index claims about itself, and this is
+ // the one finding that says those claims cannot be trusted. Same
+ // sentence as the warning - one finding, one wording.
+ const contradiction = formatEmbedderRecordContradiction(input.embedderRecord);
+ if (contradiction !== null) recs.push(contradiction);
+
// Embedding-ABI drift is the one condition here whose remediation is a
// single command, so it is named verbatim and copy-pasteable.
if (input.embeddingAbi.length > 0) {
recs.push(
`Stored vectors were written under a different embedding ABI ` +
`(${input.embeddingAbi.map(formatStampMismatch).join("; ")}). Run: ` +
- `${EMBEDDING_ABI_FIX_COMMAND}`,
+ `${embeddingAbiFixCommand(input.embeddingAbi)}`,
);
}
@@ -1631,16 +1743,36 @@ function buildRecommendations(input: BuildRecommendationsInput): string[] {
}
}
- // "Everything wired, no embeddings yet" → suggest the first
- // reindex plus the optional cron template. The probe reports
- // `reachable` only after both key and vec are present, so it is the
- // tightest proxy for "ready to compute but never did" - and a probe
- // that was skipped or timed out proves no such thing, which is why the
- // comparison is against that one state rather than "not a failure".
+ // Everything wired: what to do next is decided by the MEASURED
+ // pending-vector count (nothing-writes-silently, unit A).
+ //
+ // The gate on a reachable provider and a loaded extension is the same
+ // one this branch has always carried - a probe that was skipped or
+ // timed out proves nothing, and advising a command that cannot
+ // succeed is worse than saying nothing. What it no longer does is
+ // stand IN for the count: those two facts say the machine is ready to
+ // embed, never that anything is waiting to be embedded, and a fully
+ // embedded vault was told to compute its first vectors for as long as
+ // they were the whole test. A vault with every chunk vectorised now
+ // gets no recommendation at all.
if (input.providerProbe === PROVIDER_PROBE.reachable && input.vecExtension === "loaded") {
- recs.push(
- "Run `o2b search reindex --embeddings` to compute the first vectors, then optionally `o2b search reindex --cron-template` for periodic refresh.",
- );
+ const census = input.pendingVectors;
+ if (census.verdict === "unrecorded") {
+ recs.push(
+ `The pending-vector count is unrecorded, not zero (${census.reason}). ` +
+ FIRST_VECTORS_RECOMMENDATION,
+ );
+ } else if (census.pending === census.chunks) {
+ // Every chunk there is - including none at all, on an index that
+ // holds no chunks yet - is waiting for its first vector.
+ recs.push(FIRST_VECTORS_RECOMMENDATION);
+ } else if (census.pending > 0) {
+ recs.push(
+ `${census.pending} of ${census.chunks} indexed chunk(s) have no vector. ` +
+ `Run \`${VECTOR_BACKFILL_COMMAND}\` to see what embedding them would cost - it is a ` +
+ `dry run unless \`--apply\` is passed - and \`${VECTOR_BACKFILL_COMMAND} --apply\` to compute them.`,
+ );
+ }
}
return recs;
diff --git a/src/core/search/serialize.ts b/src/core/search/serialize.ts
index 42d00b25..6b9c15d0 100644
--- a/src/core/search/serialize.ts
+++ b/src/core/search/serialize.ts
@@ -5,7 +5,14 @@
*/
import type { StampMismatch } from "../integrity/stamp.ts";
-import type { ChunkWindowCensus, IndexStatusSnapshot, SearchCard } from "./types.ts";
+import type {
+ ChunkWindowCensus,
+ EmbedderRecordCensus,
+ IndexStatusSnapshot,
+ PendingVectorCensus,
+ SearchCard,
+ VisibilityHonestyFinding,
+} from "./types.ts";
/**
* Wire shape for a stamp comparison. `recorded` is what the index
@@ -43,6 +50,69 @@ export function serializeSearchCard(c: SearchCard): Record<string, unknown> {
};
}
+/**
+ * Wire shape of the pending-vector census (nothing-writes-silently,
+ * unit A). Emitted in EVERY state, which is what separates it from the
+ * drift fields on the same report: those say nothing when there is
+ * nothing to say, while a census whose count could not be taken has
+ * something to say precisely then. The unrecorded arm carries no
+ * `pending` key at all, so a consumer cannot read a default zero out of
+ * a state that measured nothing.
+ */
+export function serializePendingVectorCensus(census: PendingVectorCensus): Record<string, unknown> {
+ if (census.verdict === "unrecorded") {
+ return { verdict: census.verdict, reason: census.reason };
+ }
+ return { verdict: census.verdict, pending: census.pending, chunks: census.chunks };
+}
+
+/**
+ * Wire shape of the record-vs-data embedder audit
+ * (nothing-writes-silently, unit G). Emitted in every state on the same
+ * terms as the pending-vector census beside it, and the `unrecorded`
+ * arm carries no numbers at all - there is no dimension to report when
+ * nothing was compared.
+ *
+ * `reconciliation` keeps the shared vocabulary's own field names, so a
+ * consumer reading this audit and one reading the import census read
+ * the same three words for the same three quantities.
+ */
+export function serializeEmbedderRecordCensus(
+ census: EmbedderRecordCensus,
+): Record<string, unknown> {
+ if (census.verdict === "unrecorded") {
+ return { verdict: census.verdict, reason: census.reason };
+ }
+ return {
+ verdict: census.verdict,
+ outcome: census.outcome,
+ recorded_dimension: census.recordedDimension,
+ stored_dimensions: census.storedDimensions,
+ vec_declared_width: census.vecDeclaredWidth,
+ reconciliation: {
+ attempted: census.reconciliation.attempted,
+ found: census.reconciliation.found,
+ missing: census.reconciliation.missing,
+ },
+ };
+}
+
+/**
+ * Wire shape of the visibility honesty finding (nothing-writes-silently,
+ * unit H, form B). Present only when the caller already established
+ * there is something to report - `search check` omits the key entirely
+ * rather than serializing a zero-count finding, the same convention
+ * `embeddingAbi` uses for a matching store.
+ */
+export function serializeVisibilityHonestyFinding(
+ finding: VisibilityHonestyFinding,
+): Record<string, unknown> {
+ return {
+ excluded_surface_count: finding.excludedSurfaceCount,
+ total_surface_count: finding.totalSurfaceCount,
+ };
+}
+
/**
* Wire shape of an oversize-chunk census verdict, shared by the index
* run's report and the status snapshot so the two cannot describe one
diff --git a/src/core/search/store.ts b/src/core/search/store.ts
index bbd3b11e..890eed08 100644
--- a/src/core/search/store.ts
+++ b/src/core/search/store.ts
@@ -55,6 +55,7 @@ import * as vectors from "./store/vectors.ts";
// ─────────────────────────────────────────────────────────────────────────────
export {
+ describeUnreadableIndex,
EMBEDDING_DIMENSION_STATE_KEY,
EMBEDDING_MODEL_STATE_KEY,
EMBEDDING_PREFIX_PASSAGE_STATE_KEY,
@@ -71,7 +72,10 @@ export type { IndexPeek } from "./store/state.ts";
export {
contradictedAbiFields,
EMBEDDING_ABI_FIX_COMMAND,
+ EMBEDDING_VEC_VERSION_FIX_COMMAND,
+ embeddingAbiFixCommand,
formatEmbeddingAbiDrift,
+ peekEmbeddingAbiSync,
readEmbeddingAbiSync,
runtimeEmbeddingAbi,
} from "./store/embedding-abi.ts";
@@ -82,6 +86,16 @@ export {
WRITER_LOCK_STALE_MS,
} from "./store/writer-lock.ts";
+export { peekPendingVectorsSync } from "./store/counts.ts";
+
+export { peekVisibilityTagPresence } from "./store/visibility-tag.ts";
+
+export {
+ auditEmbedderRecord,
+ formatEmbedderRecordContradiction,
+ readEmbedderRecordCensusSync,
+} from "./store/embedder-audit.ts";
+
export { normalizeAlias } from "./store/aliases.ts";
export type { DocumentInput, DocumentSummary } from "./store/documents.ts";
@@ -90,7 +104,7 @@ export type { ChunkInput, ChunkRow, HydratedChunk } from "./store/chunks.ts";
export type { KeywordHit } from "./store/keyword.ts";
export type { DanglingLinkTarget, LinkInput, LinkResolutionCounts } from "./store/links.ts";
export type { EmbeddingPrefixPair, ModelChangeOutcome, SemanticHit } from "./store/vectors.ts";
-export type { StoreCounts } from "./store/counts.ts";
+export type { PendingVectorTally, StoreCounts } from "./store/counts.ts";
export interface StoreOpenOptions {
/** "read" never locks; "write" acquires an exclusive proper-lockfile. */
diff --git a/src/core/search/store/chunks.ts b/src/core/search/store/chunks.ts
index 6f558e42..f583e882 100644
--- a/src/core/search/store/chunks.ts
+++ b/src/core/search/store/chunks.ts
@@ -224,6 +224,29 @@ export function findChunksWithoutEmbeddings(
return rows.map((r) => ({ chunkId: r.id, content: r.content }));
}
+/**
+ * How MANY chunks have no row in `embeddings`, without materialising a
+ * single one of their bodies.
+ *
+ * The same anti-join {@link findChunksWithoutEmbeddings} walks, as a
+ * `COUNT(*)`. The row-returning form is the indexer's work queue and
+ * loads every pending chunk's full `content` into JS memory to hand it
+ * to a provider; a diagnostic that only wants the number must not pay
+ * that, which is the whole reason this second query exists rather than
+ * a `.length` on the first.
+ */
+export function countChunksWithoutEmbeddings(db: Database): number {
+ return (
+ db
+ .query<{ n: number }, []>(
+ "SELECT COUNT(*) AS n FROM chunks c " +
+ "LEFT JOIN embeddings e ON e.chunk_id = c.id " +
+ "WHERE e.chunk_id IS NULL",
+ )
+ .get()?.n ?? 0
+ );
+}
+
export function hydrateChunks(
db: Database,
chunkIds: ReadonlyArray<number>,
diff --git a/src/core/search/store/counts.ts b/src/core/search/store/counts.ts
index e7d17a4d..136af012 100644
--- a/src/core/search/store/counts.ts
+++ b/src/core/search/store/counts.ts
@@ -7,8 +7,9 @@
import { Database } from "bun:sqlite";
import { SearchError } from "../search-error.ts";
-import { countChunks } from "./chunks.ts";
+import { countChunks, countChunksWithoutEmbeddings } from "./chunks.ts";
import { countDocuments } from "./documents.ts";
+import { peekReadonlyIndex, type IndexPeek } from "./state.ts";
import { countEmbeddings, staleEmbeddings } from "./vectors.ts";
import {
charLengthOverTokenBudget,
@@ -38,6 +39,37 @@ export function counts(
});
}
+/** Chunks that carry no vector, against the chunks there are. */
+export interface PendingVectorTally {
+ /** Chunks with no row in `embeddings`. */
+ readonly pending: number;
+ /** Chunks in the index at all, so the pending count has a denominator. */
+ readonly chunks: number;
+}
+
+/** Both counts from one connection, so they describe one instant. */
+export function pendingVectorTally(db: Database): PendingVectorTally {
+ return Object.freeze({
+ pending: countChunksWithoutEmbeddings(db),
+ chunks: countChunks(db),
+ });
+}
+
+/**
+ * {@link pendingVectorTally} taken WITHOUT opening a `Store`, for
+ * `indexCheck` - which deliberately probes an in-memory database and
+ * never touches the real index, so anything it reports about stored
+ * state has to be read explicitly.
+ *
+ * The peek keeps its three outcomes rather than collapsing to a number:
+ * "no index here" and "an index that will not open" are not a pending
+ * count of zero, and zero is exactly what a caller would read as a
+ * fully-embedded vault.
+ */
+export function peekPendingVectorsSync(dbPath: string): IndexPeek<PendingVectorTally> {
+ return peekReadonlyIndex(dbPath, (_read, db) => pendingVectorTally(db));
+}
+
/** Split of the chunk table by what the census can decide about it. */
export interface ChunkWindowTally {
/** Chunks whose LOW estimate already exceeds the window. */
diff --git a/src/core/search/store/embedder-audit.ts b/src/core/search/store/embedder-audit.ts
new file mode 100644
index 00000000..42db0a42
--- /dev/null
+++ b/src/core/search/store/embedder-audit.ts
@@ -0,0 +1,199 @@
+/**
+ * What the index RECORDS about its embedder against what it actually
+ * HOLDS (nothing-writes-silently, unit G).
+ *
+ * Two comparisons already exist and neither answers this question.
+ * `embedding-abi.ts` compares the recorded tokens against the tokens
+ * THIS BUILD would produce; `staleEmbeddings` compares the stored rows
+ * against the CONFIGURED model. Both take the record's word for what
+ * the index contains. So an `index_state` row claiming dimension 8 over
+ * 4-wide stored vectors was invisible to every surface: the record is
+ * not drifted from anything, it is simply wrong, and a wrong record is
+ * evidence of nothing rather than evidence of a gap.
+ *
+ * The audit is pure SQL and reads three facts:
+ *
+ * - the dimension `index_state` records;
+ * - the DISTINCT widths the `embeddings` rows carry;
+ * - the width the `chunk_vec` virtual table declares, read out of
+ * `sqlite_master` rather than by querying the table, so no
+ * sqlite-vec extension has to load for the audit to run.
+ *
+ * It contacts no provider, costs nothing, and its verdict rides the
+ * wave's shared reconciliation vocabulary so "the record contradicts the
+ * data" is one named outcome rather than a sentence each surface spells
+ * for itself.
+ *
+ * Scope, stated so a reader does not go looking for the other half: this
+ * compares DIMENSIONS. `embeddings.model` versus the recorded model is a
+ * separate comparison the wave did not take on - a row written by
+ * another model is what `staleEmbeddings` already counts, against the
+ * configured model rather than the recorded one - and the restamp verb
+ * refuses a model disagreement rather than repairing it.
+ */
+
+import { Database } from "bun:sqlite";
+
+import {
+ buildReconciliationReport,
+ deriveReconciliationOutcome,
+ RECONCILIATION_OUTCOME,
+} from "../../reconciliation-report.ts";
+import type { EmbedderRecordCensus } from "../types.ts";
+import { EMBEDDING_ABI_FIX_COMMAND } from "./embedding-abi.ts";
+import {
+ describeUnreadableIndex,
+ EMBEDDING_DIMENSION_STATE_KEY,
+ peekReadonlyIndex,
+} from "./state.ts";
+
+/**
+ * `CREATE VIRTUAL TABLE chunk_vec USING vec0(embedding float[N])` - the
+ * declaration `ensureVecTable` writes, read back for its N. A table
+ * declared in a shape this pattern does not match contributes no
+ * observation: an unreadable declaration cannot contradict anything,
+ * the same rule an unrecorded ABI token already follows.
+ */
+const VEC0_DECLARED_WIDTH_RE = /float\s*\[\s*(\d+)\s*\]/i;
+
+/** The distinct widths the `embeddings` rows carry, ascending. */
+function storedDimensions(db: Database): ReadonlyArray<number> {
+ return db
+ .query<{ dimension: number }, []>(
+ "SELECT DISTINCT dimension FROM embeddings ORDER BY dimension ASC",
+ )
+ .all()
+ .map((r) => r.dimension);
+}
+
+/** The width `chunk_vec` declares, or null when there is no such table. */
+function vecDeclaredWidth(db: Database): number | null {
+ const row = db
+ .query<{ sql: string | null }, []>(
+ "SELECT sql FROM sqlite_master WHERE type = 'table' AND name = 'chunk_vec'",
+ )
+ .get();
+ if (row?.sql == null) return null;
+ const match = VEC0_DECLARED_WIDTH_RE.exec(row.sql);
+ return match === null ? null : Number(match[1]);
+}
+
+/** One thing the index holds, named as it will appear in `missing`. */
+interface DimensionObservation {
+ readonly key: string;
+ readonly dimension: number;
+}
+
+/**
+ * Audit one open index. `read` is the `index_state` accessor and `db`
+ * the same connection, so both halves of the comparison come from one
+ * instant.
+ *
+ * The UNRECORDED arm covers the three states that leave nothing to
+ * compare - no recorded dimension, a recorded dimension that is not a
+ * number, and an index holding neither vectors nor a vec table. None of
+ * them is a contradiction and none is a clean audit; reporting either
+ * would be a claim this query cannot back.
+ */
+export function auditEmbedderRecord(
+ read: (key: string) => string | null,
+ db: Database,
+): EmbedderRecordCensus {
+ const raw = read(EMBEDDING_DIMENSION_STATE_KEY);
+ if (raw === null) {
+ return Object.freeze({
+ verdict: "unrecorded" as const,
+ reason: `index_state records no ${EMBEDDING_DIMENSION_STATE_KEY}, so there is no claim to check`,
+ });
+ }
+ const recordedDimension = Number(raw);
+ if (!Number.isInteger(recordedDimension) || recordedDimension <= 0) {
+ return Object.freeze({
+ verdict: "unrecorded" as const,
+ reason: `index_state records ${EMBEDDING_DIMENSION_STATE_KEY} as "${raw}", which is not a dimension`,
+ });
+ }
+
+ const stored = storedDimensions(db);
+ const declared = vecDeclaredWidth(db);
+ const observations: ReadonlyArray<DimensionObservation> = [
+ ...stored.map((dimension) => ({ key: `embeddings.dimension=${dimension}`, dimension })),
+ ...(declared === null
+ ? []
+ : [{ key: `chunk_vec.embedding float[${declared}]`, dimension: declared }]),
+ ];
+ if (observations.length === 0) {
+ return Object.freeze({
+ verdict: "unrecorded" as const,
+ reason: "the index holds no stored vectors and no vector table to compare the record against",
+ });
+ }
+
+ const missing = observations.filter((o) => o.dimension !== recordedDimension).map((o) => o.key);
+ const reconciliation = buildReconciliationReport({
+ attempted: observations.length,
+ found: observations.length - missing.length,
+ missing,
+ });
+ // What the record ASSERTS about these same observations: that every
+ // one of them was written at the dimension it names. Handing that
+ // claim to the shared derivation is what makes a disagreement come
+ // back as `contradicted` - the record is wrong - rather than as a
+ // plain shortfall in what was found.
+ const outcome = deriveReconciliationOutcome(reconciliation, {
+ attempted: observations.length,
+ found: observations.length,
+ });
+
+ return Object.freeze({
+ verdict: "audited" as const,
+ recordedDimension,
+ storedDimensions: Object.freeze([...stored]),
+ vecDeclaredWidth: declared,
+ reconciliation,
+ outcome,
+ });
+}
+
+/**
+ * {@link auditEmbedderRecord} for an index path, without opening a
+ * `Store` - the same seam `readEmbeddingAbiSync` uses, and for the same
+ * reason: `indexCheck` probes an in-memory database and never touches
+ * the real index otherwise.
+ *
+ * An absent or unreadable index folds into the UNRECORDED arm with the
+ * reason the open gave. It is not a clean audit: nothing was compared.
+ */
+export function readEmbedderRecordCensusSync(dbPath: string): EmbedderRecordCensus {
+ const peek = peekReadonlyIndex(dbPath, auditEmbedderRecord);
+ if (peek.kind === "read") return peek.value;
+ return Object.freeze({
+ verdict: "unrecorded" as const,
+ reason:
+ peek.kind === "absent"
+ ? `no search index at ${dbPath}`
+ : describeUnreadableIndex(dbPath, peek.detail),
+ });
+}
+
+/**
+ * One operator-facing line for a record the data contradicts, or `null`
+ * when the audit found no contradiction to describe. Shared by the
+ * `search check` warning and its recommendation so the two cannot word
+ * one finding differently or name different repairs.
+ *
+ * The repair is the full rebuild, not the restamp verb: a recorded
+ * dimension the stored vectors disprove cannot be corrected by writing
+ * a different number over it - the vectors would still be whatever
+ * width they are, and the record would still be a claim nobody checked.
+ */
+export function formatEmbedderRecordContradiction(census: EmbedderRecordCensus): string | null {
+ if (census.verdict !== "audited") return null;
+ if (census.outcome !== RECONCILIATION_OUTCOME.contradicted) return null;
+ return (
+ `the index records embedding dimension ${census.recordedDimension}, and the vectors it ` +
+ `holds contradict that record (${census.reconciliation.missing.join("; ")}); the ` +
+ `recorded identity is evidence of nothing about the stored vectors. Run: ` +
+ `${EMBEDDING_ABI_FIX_COMMAND}`
+ );
+}
diff --git a/src/core/search/store/embedding-abi.ts b/src/core/search/store/embedding-abi.ts
index 27106b02..2d1e5939 100644
--- a/src/core/search/store/embedding-abi.ts
+++ b/src/core/search/store/embedding-abi.ts
@@ -15,7 +15,8 @@ import {
EMBEDDING_DIMENSION_STATE_KEY,
EMBEDDING_MODEL_STATE_KEY,
EMBEDDING_VEC_VERSION_STATE_KEY,
- withReadonlyIndex,
+ peekReadonlyIndex,
+ type IndexPeek,
} from "./state.ts";
/**
@@ -25,6 +26,32 @@ import {
*/
export const EMBEDDING_ABI_FIX_COMMAND = "o2b search reindex --embeddings";
+/**
+ * The remediation for a drift confined to the sqlite-vec version
+ * (nothing-writes-silently, unit G).
+ *
+ * That token is an ABI marker, not a property of any vector: two peers
+ * on different sqlite-vec builds each read the other's store as
+ * drifted, which is why the gate defaults to `warn`. Sending them to
+ * the full reindex charges provider prices to correct a version string
+ * no stored vector depends on, so this drift - and only this one -
+ * names the restamp verb instead.
+ */
+export const EMBEDDING_VEC_VERSION_FIX_COMMAND = "o2b search restamp";
+
+/**
+ * Which remediation a given drift earns. The full rebuild, unless every
+ * drifted field is the sqlite-vec version, in which case restamping is
+ * the whole repair. Shared by the refusal, the warning and the
+ * recommendation so no surface can name a different command for the
+ * same drift.
+ */
+export function embeddingAbiFixCommand(mismatches: ReadonlyArray<StampMismatch>): string {
+ const vecVersionOnly =
+ mismatches.length > 0 && mismatches.every((m) => m.field === EMBEDDING_VEC_VERSION_STATE_KEY);
+ return vecVersionOnly ? EMBEDDING_VEC_VERSION_FIX_COMMAND : EMBEDDING_ABI_FIX_COMMAND;
+}
+
/** The ABI tokens an index recorded, read through any state accessor. */
export function recordedEmbeddingAbi(state: (key: string) => string | null): StampTokens {
return {
@@ -63,7 +90,17 @@ export function runtimeEmbeddingAbi(
* matches.
*/
export function readEmbeddingAbiSync(dbPath: string): StampTokens | null {
- return withReadonlyIndex(dbPath, (read) => recordedEmbeddingAbi(read));
+ const peek = peekEmbeddingAbiSync(dbPath);
+ return peek.kind === "read" ? peek.value : null;
+}
+
+/**
+ * {@link readEmbeddingAbiSync} with the three outcomes kept apart, for
+ * a caller that must tell an index it could not open from one that is
+ * not there - a repair verb refusing by name is exactly that caller.
+ */
+export function peekEmbeddingAbiSync(dbPath: string): IndexPeek<StampTokens> {
+ return peekReadonlyIndex(dbPath, (read) => recordedEmbeddingAbi(read));
}
/**
@@ -94,6 +131,6 @@ export function formatEmbeddingAbiDrift(mismatches: ReadonlyArray<StampMismatch>
return (
`embedding ABI drift between the stored index and this build ` +
`(${mismatches.map(formatStampMismatch).join("; ")}); stored vectors are not ` +
- `comparable to queries embedded now. Run: ${EMBEDDING_ABI_FIX_COMMAND}`
+ `comparable to queries embedded now. Run: ${embeddingAbiFixCommand(mismatches)}`
);
}
diff --git a/src/core/search/store/state.ts b/src/core/search/store/state.ts
index 76eff4ea..54161f12 100644
--- a/src/core/search/store/state.ts
+++ b/src/core/search/store/state.ts
@@ -147,6 +147,20 @@ export type IndexPeek<T> =
| { readonly kind: "absent" }
| { readonly kind: "unreadable"; readonly detail: string };
+/**
+ * How an `unreadable` peek reads in a report.
+ *
+ * "Could not be read", not "did not open": the arm covers a database
+ * that opened and then threw on the read as well as one that never
+ * opened - an older schema with no `embeddings` table is the common
+ * case - and blaming the open misattributes the fault to the file
+ * instead of the query. Shared so the peeks that fold this arm into an
+ * `unrecorded` verdict all word it the same way.
+ */
+export function describeUnreadableIndex(dbPath: string, detail: string): string {
+ return `${dbPath} could not be read: ${detail}`;
+}
+
/**
* Run `read` against a read-only handle on the index, naming which of
* the three outcomes occurred. Shared by the synchronous peeks so the
diff --git a/src/core/search/store/visibility-tag.ts b/src/core/search/store/visibility-tag.ts
new file mode 100644
index 00000000..fc5c2b35
--- /dev/null
+++ b/src/core/search/store/visibility-tag.ts
@@ -0,0 +1,78 @@
+/**
+ * Whether the index holds ANY evidence that a page declares
+ * `visibility:` frontmatter, read without a vault walk
+ * (nothing-writes-silently, unit H).
+ *
+ * The chunker (`chunker.ts`'s `packBlocks`) emits a page's frontmatter
+ * block as its own chunk, verbatim, ahead of every other chunk the page
+ * produces - so `chunks.content` at `chunk_index = 0` IS the page's raw
+ * YAML text when the page has frontmatter at all, and the FIRST real
+ * content chunk (never frontmatter) when it does not. That is a
+ * structural fact about how the index is built, not a coincidence this
+ * module is trusting: it is exercised as `INTRUDER_CHUNK_INDEX_ZERO` in
+ * `tests/core/search/visibility-tag-presence.test.ts`.
+ *
+ * That makes "does any page use `visibility:`" answerable from the
+ * ALREADY-BUILT index - no filesystem walk, no per-page frontmatter
+ * parse, and no new table. It is imprecise in the direction that costs
+ * nothing: a
+ * frontmatter VALUE that happens to contain the literal substring
+ * `visibility:` (a title field quoting the word, say) reads as a tagged
+ * vault when none exists. `search check`'s honesty finding this feeds
+ * is worded to survive that - it names a caller-supplied filter, not an
+ * exact count of tagged pages - and the alternative, parsing every
+ * frontmatter block back out of `chunks` to confirm the key rather than
+ * the substring, re-derives the vault-walk cost this module exists to
+ * avoid.
+ *
+ * ## What it costs, stated rather than implied
+ *
+ * It is a SCAN of `chunks`, not an index lookup. No index serves it and
+ * none could: the only index on the table is `idx_chunks_document`
+ * (`schema.ts`), and a leading-wildcard `LIKE` is unindexable in any
+ * case, so `chunk_index = ?` is a filter applied per row rather than a
+ * seek. `EXISTS` stops at the first match, which makes a vault that DOES
+ * use the field cheap and the common case - a vault that never has -
+ * the worst one: every chunk body is read to conclude "no".
+ *
+ * What makes that affordable is WHERE it runs, not how small it is: once
+ * per `indexCheck`, an operator-invoked diagnostic, and nowhere on a
+ * query path. A caller wanting this answer per search would need a
+ * recorded flag rather than this scan.
+ */
+
+import { Database } from "bun:sqlite";
+
+import { peekReadonlyIndex, type IndexPeek } from "./state.ts";
+
+/** The `chunk_index` a page's frontmatter block lands at, when it has one. */
+const FRONTMATTER_CHUNK_INDEX = 0;
+
+/** Substring a frontmatter chunk carries when the page declares the key. */
+const VISIBILITY_KEY_PATTERN = "%visibility:%";
+
+/**
+ * True when at least one document's `chunk_index = 0` row contains the
+ * literal substring `visibility:`. One `LIKE` scan of `chunks`, stopping
+ * at the first hit - see the module docblock for what that costs and why
+ * this is not the indexed lookup its shape suggests.
+ */
+export function anyVisibilityTagPresent(db: Database): boolean {
+ const row = db
+ .query<{ present: number }, [string, number]>(
+ "SELECT EXISTS(SELECT 1 FROM chunks WHERE content LIKE ?1 AND chunk_index = ?2) AS present",
+ )
+ .get(VISIBILITY_KEY_PATTERN, FRONTMATTER_CHUNK_INDEX);
+ return row?.present === 1;
+}
+
+/**
+ * {@link anyVisibilityTagPresent} for an index path, without opening a
+ * `Store` - the same seam {@link peekPendingVectorsSync} and
+ * {@link readEmbedderRecordCensusSync} use, and for the same reason:
+ * `indexCheck` probes an in-memory database and never touches the real
+ * index otherwise.
+ */
+export function peekVisibilityTagPresence(dbPath: string): IndexPeek<boolean> {
+ return peekReadonlyIndex(dbPath, (_read, db) => anyVisibilityTagPresent(db));
+}
diff --git a/src/core/search/types.ts b/src/core/search/types.ts
index e6ec9eca..abfb6fc9 100644
--- a/src/core/search/types.ts
+++ b/src/core/search/types.ts
@@ -15,6 +15,7 @@
import type { DegradationNotice } from "../integrity/degradation.ts";
import type { StampMismatch } from "../integrity/stamp.ts";
+import type { ReconciliationOutcome, ReconciliationReport } from "../reconciliation-report.ts";
import type { VaultPathRule, VaultScopeRules } from "../vault-scope/defaults.ts";
import type { DegreePredicate } from "./property-filter.ts";
import type { TemporalIntent } from "./temporal-intent.ts";
@@ -404,6 +405,102 @@ export interface IndexStatusSnapshot {
readonly warnings: ReadonlyArray<string>;
}
+/**
+ * What `search check` could establish about chunks that carry no vector
+ * (nothing-writes-silently, unit A).
+ *
+ * Two states, and the second is the point. The recommendation this
+ * census feeds used to be derived from a proxy - a reachable provider
+ * plus a loaded extension - which has no term for whether any vector
+ * exists, so a fully embedded vault was told to compute its first ones.
+ * A measured count replaces the proxy, and a count that could not be
+ * taken is reported as UNRECORDED rather than as zero: an absent or
+ * unreadable index proves nothing about how many chunks are waiting,
+ * and zero is precisely the answer that reads as "nothing is waiting".
+ *
+ * Emitted in every state, unlike the drift fields beside it. A report
+ * about what could not be checked must not answer with silence.
+ */
+export type PendingVectorCensus =
+ | {
+ readonly verdict: "measured";
+ /** Chunks with no row in `embeddings`. */
+ readonly pending: number;
+ /** Chunks the index holds at all. */
+ readonly chunks: number;
+ }
+ | {
+ /** No count was taken; the index was absent or would not open. */
+ readonly verdict: "unrecorded";
+ /** Which of those it was, in the words the open used. */
+ readonly reason: string;
+ };
+
+/**
+ * What the index's own record of its embedder says, measured against
+ * the vectors it holds (nothing-writes-silently, unit G).
+ *
+ * The `audited` arm's `outcome` is the wave's shared reconciliation
+ * vocabulary, and `contradicted` is the state this census exists for: a
+ * recorded dimension the stored data itself disproves. That is a
+ * different finding from the two that already had names - `embeddingAbi`
+ * drift, where the record disagrees with THIS BUILD, and an unrecorded
+ * token, where the index never made the claim at all - and folding it
+ * into either would report a wrong record as a stale one.
+ *
+ * `unrecorded` is every state with nothing to compare: no index, an
+ * index that would not open, no recorded dimension, or no stored vector
+ * to check it against. None of them is a clean audit.
+ */
+export type EmbedderRecordCensus =
+ | {
+ readonly verdict: "audited";
+ /** The dimension `index_state` claims. */
+ readonly recordedDimension: number;
+ /** Distinct widths the `embeddings` rows carry, ascending. */
+ readonly storedDimensions: ReadonlyArray<number>;
+ /** The width `chunk_vec` declares, or null when there is no such table. */
+ readonly vecDeclaredWidth: number | null;
+ /** Observations attempted, matched, and - by name - not matched. */
+ readonly reconciliation: ReconciliationReport;
+ /** `complete` or `contradicted`; see the note above. */
+ readonly outcome: ReconciliationOutcome;
+ }
+ | {
+ /** Nothing was compared, and why. */
+ readonly verdict: "unrecorded";
+ readonly reason: string;
+ };
+
+/**
+ * The visibility honesty finding (nothing-writes-silently, unit H, form
+ * B): `visibility:` frontmatter is a caller-supplied view filter, not a
+ * privacy boundary, and `excludedSurfaceCount` of `totalSurfaceCount`
+ * ENUMERATED note-returning surfaces do not honour it. Both counts are
+ * read off `visibility-surface-registry.ts`'s own exported list - never
+ * hand-written - so the number this finding names cannot drift from the
+ * census that backs it.
+ *
+ * The denominator is the census's population, not the product's whole
+ * surface area, and the surface that renders it says so: the MCP half is
+ * swept mechanically with stated blind spots and the CLI half is
+ * hand-enumerated. It also counts CALLABLE rows only - the registry's one
+ * `index_store` row is a structural fact about the index's storage, not
+ * something a caller invokes.
+ *
+ * Present on {@link IndexCheckReport} only when the vault/index carries
+ * at least one page tagged with the field; a vault that has never used
+ * `visibility:` has nothing this finding would be honest ABOUT, and
+ * warning it anyway would train operators to ignore the line on every
+ * vault that never triggers it.
+ */
+export interface VisibilityHonestyFinding {
+ /** Enumerated callable surfaces that never consult `visibility:`. */
+ readonly excludedSurfaceCount: number;
+ /** Every callable surface the registry classifies, covered or excluded. */
+ readonly totalSurfaceCount: number;
+}
+
export interface IndexCheckReport {
readonly vaultReadable: boolean;
readonly indexDirWritable: boolean;
@@ -436,6 +533,26 @@ export interface IndexCheckReport {
* field was stamped, which is reported and never treated as wrong.
*/
readonly embeddingAbi: ReadonlyArray<StampMismatch>;
+ /**
+ * Chunks awaiting a vector, measured rather than inferred - or the
+ * statement that no count could be taken. See
+ * {@link PendingVectorCensus}; it is the term the reindex
+ * recommendation below now gates on.
+ */
+ readonly pendingVectors: PendingVectorCensus;
+ /**
+ * The index's recorded embedder identity measured against the vectors
+ * it holds. See {@link EmbedderRecordCensus}; its `contradicted`
+ * outcome is a statement neither {@link embeddingAbi} nor
+ * {@link pendingVectors} can make.
+ */
+ readonly embedderRecord: EmbedderRecordCensus;
+ /**
+ * The visibility honesty finding, present only when the vault/index
+ * carries at least one `visibility:`-tagged page. See
+ * {@link VisibilityHonestyFinding}.
+ */
+ readonly visibilityHonesty?: VisibilityHonestyFinding;
readonly warnings: ReadonlyArray<string>;
readonly fatal: ReadonlyArray<string>;
/**
diff --git a/src/core/search/visibility-surface-registry.ts b/src/core/search/visibility-surface-registry.ts
new file mode 100644
index 00000000..814527df
--- /dev/null
+++ b/src/core/search/visibility-surface-registry.ts
@@ -0,0 +1,717 @@
+/**
+ * Every surface that can hand a vault note's path, title, or body back to
+ * a caller, classified as visibility-COVERED (routes through
+ * `applyVisibilityScope`) or EXCLUDED with a written reason
+ * (nothing-writes-silently, unit H, form B).
+ *
+ * `graph/visibility.ts`'s `visibility:` frontmatter field is real and
+ * wired into exactly one place: `pipeline/pool-filters.ts:130`, inside
+ * `search()`. It is caller-liftable scoping, not a boundary - any caller
+ * may pass `visibility: ["private"]` and read a tagged page - but even
+ * that weaker guarantee only holds for the ONE lane that calls `search()`.
+ * Everywhere else, a page's tag is decorative. This module is the
+ * enumeration that makes the gap a measured fact instead of a claim:
+ * {@link tests/core/architecture/visibility-surface-census.test.ts} pins
+ * that a mechanical sweep of `src/mcp/` and `src/cli/` finds nothing this
+ * list does not already carry, and `search check` derives its honesty
+ * finding's count from {@link excludedCallableVisibilitySurfaces} rather
+ * than a hand-written number.
+ *
+ * The list is the census's POPULATION, not a claim to be every surface
+ * there is: the MCP half is swept mechanically with the blind spots that
+ * test's docblock states, and the CLI half is hand-enumerated one row per
+ * MCP mirror. The `search check` line that reports it says so.
+ *
+ * This module makes NO enforcement change. It does not touch
+ * `graph/visibility.ts`, the indexer, `listVaultPages`, or the
+ * owner-scope path - see the census test's docblock for the coverage map
+ * a future enforcement wave (Unit H form A, parked) would need.
+ */
+
+/** Whether a surface's data path passes through `applyVisibilityScope`. */
+export const VISIBILITY_SURFACE_CATEGORY = Object.freeze({
+ /** Routes through `applyVisibilityScope`, directly or via `search()`. */
+ covered: "covered",
+ /** Can return note path/title/body without ever consulting `visibility:`. */
+ excluded: "excluded",
+} as const);
+
+export type VisibilitySurfaceCategory =
+ (typeof VISIBILITY_SURFACE_CATEGORY)[keyof typeof VISIBILITY_SURFACE_CATEGORY];
+
+/** Which interface a surface is reached through. */
+export const VISIBILITY_SURFACE_KIND = Object.freeze({
+ mcpTool: "mcp_tool",
+ cliVerb: "cli_verb",
+ mcpResource: "mcp_resource",
+ /** A structural fact about the index's own storage, not a callable surface. */
+ indexStore: "index_store",
+} as const);
+
+export type VisibilitySurfaceKind =
+ (typeof VISIBILITY_SURFACE_KIND)[keyof typeof VISIBILITY_SURFACE_KIND];
+
+export interface VisibilitySurfaceEntry {
+ /** The tool name, `<group> <verb>` CLI path, resource URI, or fixed label. */
+ readonly surface: string;
+ readonly kind: VisibilitySurfaceKind;
+ readonly category: VisibilitySurfaceCategory;
+ /** What the category alone does not say - the source-verified argument. */
+ readonly reason: string;
+}
+
+const K = VISIBILITY_SURFACE_KIND;
+const C = VISIBILITY_SURFACE_CATEGORY;
+
+/**
+ * The population, hand-verified against source (main clone at
+ * `feat/nothing-writes-silently`, HEAD `31561384` and after). The census
+ * test's mechanical sweep is the check that this list has not gone
+ * stale; the reasons below are what that sweep cannot itself produce.
+ */
+export const VISIBILITY_SURFACE_REGISTRY: ReadonlyArray<VisibilitySurfaceEntry> = Object.freeze([
+ // --- Covered: reaches search()'s pool-filters pipeline -------------------
+ {
+ surface: "brain_search",
+ kind: K.mcpTool,
+ category: C.covered,
+ reason:
+ "calls core/search/search.ts's search() directly, which runs assembleRankedResults -> " +
+ "applyPoolFilters -> applyVisibilityScope (pool-filters.ts:130) on every result before " +
+ "returning it, unconditionally - the empty-scope-hides-tagged-pages rule applies even when " +
+ "the caller passes no visibility argument at all.",
+ },
+ {
+ surface: "brain_file_context",
+ kind: K.mcpTool,
+ category: C.covered,
+ reason:
+ "src/mcp/search-tools.ts's toolBrainFileContext calls fileContextRecall " +
+ "(core/brain/file-recall.ts), which calls search() with the derived query and no visibility " +
+ "override, so it inherits the same pool-filters gate as brain_search. It is covered " +
+ "transitively, not by an independent check of its own.",
+ },
+ {
+ surface: "search query",
+ kind: K.cliVerb,
+ category: C.covered,
+ reason:
+ "src/cli/search/verbs/query.ts calls search() with the CLI's --visibility flag threaded " +
+ "through, the same function and the same pool-filters gate brain_search uses.",
+ },
+
+ // --- Excluded: MCP tools ---------------------------------------------------
+ {
+ surface: "brain_search_expand",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "expandHit() (core/search/cards.ts) hydrates a chunk_id directly off the store and checks " +
+ "only isPathOwnerVisible, and only when the caller supplies agentScope - it never calls " +
+ "applyVisibilityScope or isVisible. A caller holding any valid chunk_id for a " +
+ "visibility-tagged page - not necessarily one it received from a filtered brain_search call " +
+ "- can read the full note through this tool.",
+ },
+ {
+ surface: "second_brain_query",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "toolQuery (src/mcp/tools.ts) lists pages with listVaultPages(), which accepts only " +
+ "skipDirs/skipFiles, and applies its own owner-scope check over the parsed frontmatter - " +
+ "visibility: is never read on this path.",
+ },
+ {
+ surface: "brain_context_pack",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "packContext() (core/brain/context-pack.ts) draws candidates only from Brain/preferences " +
+ "and Brain/retired via collectPreferencePages, filtered by isTombstoned and ownerScope - " +
+ "never by visibility. A Brain preference page tagged visibility: private is returned " +
+ "verbatim, path and body both.",
+ },
+ {
+ surface: "brain_query",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "queryByPreference / queryByTopic (core/brain/query.ts) return Brain preference, retired " +
+ "and log records - id, principle, evidence trail - straight off disk with no visibility " +
+ "check anywhere in the read path.",
+ },
+ {
+ surface: "brain_backlinks",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "buildBacklinkIndex (core/brain/backlinks.ts) walks the vault with readdirSync + " +
+ "parseFrontmatter directly, gated only by ownerScopeView; visibility: is not among the " +
+ "fields it consults.",
+ },
+ {
+ surface: "brain_unlinked_mentions",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "findUnlinkedMentions (core/brain/link-graph/unlinked-mentions.ts) walks the vault the same " +
+ "way backlinks.ts does - readdirSync + parseFrontmatter, ownerScopeView only - and surfaces " +
+ "the mentioning page's path regardless of any visibility tag on it.",
+ },
+ {
+ surface: "brain_sources",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "aggregateSources (core/brain/portability/sources.ts) reads Brain's own signal registry " +
+ "under readdirSync, with no visibility check - the field is not part of a signal's schema " +
+ "and the reader never looks at a general vault page.",
+ },
+ {
+ surface: "brain_audit",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "readPrefAudit (core/brain/pref-audit.ts) returns preference-mutation ledger records - " +
+ "timestamp, pref_id, op, agent, revision hashes - never a note body or title, and no " +
+ "visibility check runs over them. Included for completeness though its disclosure is " +
+ "narrower than the other entries here: no page content crosses this surface at all.",
+ },
+ {
+ surface: "brain_bridges",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "discoverBridges (core/brain/link-graph/bridge-discovery.ts) proposes links between " +
+ "embedding-near notes read via listVaultPages and a raw Store handle, and list reads " +
+ "Brain/proposals/bridges.md back - source/target note paths in both cases, with no " +
+ "visibility check on either side of a proposed pair.",
+ },
+ {
+ surface: "brain_clusters",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "detectCommunities / materializeClusterNotes (core/brain/link-graph/communities.ts) read " +
+ "the link graph off a raw Store handle and member titles via parseFrontmatter, then " +
+ "materialize cluster notes naming every member page - no visibility check anywhere in the " +
+ "pass.",
+ },
+ {
+ surface: "brain_moc_audit",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "auditMoc (core/brain/link-graph/moc-audit.ts) classifies a hub's outbound links through " +
+ "buildBacklinkIndex, the same ownerScopeView-only, visibility-blind path brain_backlinks " +
+ "uses.",
+ },
+ {
+ surface: "brain_deep_synthesis",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "deepSynthesis (core/brain/deep-synthesis.ts) is mixed rather than uniformly unfiltered: " +
+ "its matched-notes component calls search() and inherits pool-filters, but its " +
+ "knowledge-gap component (dangling wikilinks) walks the vault directly via walkVault " +
+ "(core/search/walker.ts), which applies no visibility check. The dossier as a whole cannot " +
+ "be called covered because one of its two note-reading paths is not.",
+ },
+ {
+ surface: "brain_idea_discovery",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "discoverIdeas / ideaCandidates (core/brain/idea-discovery.ts) walk the vault via " +
+ "readdirSync + parseFrontmatter to find orphan research notes, with no visibility check on " +
+ "any candidate page.",
+ },
+ {
+ surface: "brain_dead_ends",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "listDeadEnds / recordDeadEnd (core/brain/dead-ends.ts) read and write Brain/dead-ends/ " +
+ "notes directly via parseFrontmatterText, with no visibility check - though these are " +
+ "Brain-authored records rather than general vault notes.",
+ },
+ {
+ surface: "brain_claims",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "toolBrainClaims's own docblock (knowledge-tools.ts) states a claim row carries the " +
+ "artifact's id, vault-relative path, topic and full principle text; every row-returning " +
+ "operation is filtered by gatedOwnerScopeView (agent-scope), never by visibility.",
+ },
+ {
+ surface: "brain_truth",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "the entity claim ledger (core/brain/truth/*) returns claim values, provenance wikilinks " +
+ "and cross-agent collision records keyed by entity, with no visibility check anywhere in " +
+ "ingest, slots, conflicts, aggregate or collisions.",
+ },
+ {
+ surface: "brain_labels",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "show reads a caller-named page with parseFrontmatter and returns the caller's own path " +
+ "plus its label set - no title or body crosses this surface, and no visibility check runs " +
+ "over the read either way.",
+ },
+ {
+ surface: "brain_tiers",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "check returns store.listTierDrift() unscoped across the whole index - every drifted " +
+ "document's path, with no visibility check - and restore additionally reads the named " +
+ "page's full frontmatter and body via parseFrontmatter.",
+ },
+ {
+ surface: "brain_maintenance",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "the bridges and clusters lanes of this maintenance runner call discoverBridges / " +
+ "detectCommunities directly in their own `run` steps, the same unfiltered vec-index and " +
+ "link-graph reads brain_bridges/brain_clusters make - the response surfaces only per-lane " +
+ "counts, but the reads themselves are unfiltered.",
+ },
+ {
+ surface: "brain_secrets",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "swept into population because admin-tools.ts (its home file) also imports parseFrontmatter " +
+ "and Store for brain_labels/brain_tiers/brain_maintenance beside it - the handler itself " +
+ "returns only secret names and run results, never a note path, title, or body. Listed for " +
+ "completeness of the file-level population rule rather than as a real disclosure.",
+ },
+ {
+ surface: "brain_artifact_get",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "a generic preview-artifact cache read (src/mcp/tools.ts's toolArtifactGet): it returns " +
+ "whatever string another tool call stored under artifact_id when its own response was " +
+ "preview-truncated, with no check of its own. When the truncated response came from a " +
+ "note-returning excluded tool, the cached payload is that same unfiltered content.",
+ },
+ {
+ surface: "brain_diarize",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "diarize() (core/brain/diarization.ts) returns document_set - the entity's assembled " +
+ "document set from the registry and sources - with no visibility check on any member path.",
+ },
+ {
+ surface: "brain_context",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "toolBrainContext (context-tools.ts) returns Brain/active.md's rendered content via " +
+ "parseFrontmatter - the standing-rules digest, a Brain-authored artifact - with no " +
+ "visibility check; it is the always-loaded writer tool, not a general note reader.",
+ },
+ {
+ surface: "brain_agent_query",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "queryAgentSources (core/brain/agent-source/query.ts) returns, by the handler's own " +
+ "comment, 'contribution rows - ids, topics and the record text' from the provenance fold, " +
+ "gated by an explicit agent_scope argument only - never by visibility.",
+ },
+ {
+ surface: "brain_agent_diff",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "diffAgentSources returns the same contribution rows brain_agent_query does, gated by the " +
+ "GATED server identity (owner_scope_delivery, off by default) rather than an argument - " +
+ "visibility is not part of either gate.",
+ },
+ {
+ surface: "brain_anticipatory_context",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "reads the anticipatory-context cache (core/brain/anticipatory-cache.ts), which mirrors " +
+ "brain_context_pack's own candidate items - no visibility check runs over the cached read.",
+ },
+ {
+ surface: "brain_context_receipts",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "show returns a stored receipt's full payload verbatim - the same content a prior " +
+ "brain_context_pack call recorded - with no visibility re-check at read time; only a " +
+ "receipt-level private/redacted flag applies, which is a different axis.",
+ },
+ {
+ surface: "brain_event_trace",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "resolveLogEventTraces (core/brain/event-trace.ts) returns Brain log event bodies, which " +
+ "can name a note path per event; gated by a keep_private argument over the event's own " +
+ "flag, not by the named artifact's visibility.",
+ },
+ {
+ surface: "brain_foresight",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "assembles recurring routines, open commitments and open questions from Brain's own " +
+ "recurrence/obligation/question stores - Brain-authored records, not general vault notes - " +
+ "with no visibility check.",
+ },
+ {
+ surface: "brain_pinned_context",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "reads/writes Brain/pinned.md, a single Brain-authored scratchpad file, with no visibility " +
+ "check - the field would be meaningless on a file with exactly one owner-less instance.",
+ },
+ {
+ surface: "brain_write_session",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "dispatchSubmit / listWriteSessions / readWriteSession (core/brain/write-session/*) read " +
+ "and mutate staged write-session records - Brain's own queue, not general vault notes - " +
+ "with no visibility check.",
+ },
+ {
+ surface: "brain_pre_compress_pack",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "buildPreCompressPack (core/brain/pre-compress-pack.ts) returns id/principle items from " +
+ "the same Brain/preferences pool brain_context_pack draws from, gated by agentScope only.",
+ },
+ {
+ surface: "brain_pre_compact_extract",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "extractPreCompactRecords (core/brain/pre-compact-extract.ts) extracts records from " +
+ "caller-SUPPLIED conversation text, not from vault notes - included for completeness of " +
+ "the file-level sweep rather than as a real vault-content disclosure.",
+ },
+ {
+ surface: "brain_recall_gate",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "a diagnostic classifier over caller-supplied scores/match_quality - it runs no search and " +
+ "returns no note content; included for completeness of the file-level sweep only.",
+ },
+ {
+ surface: "brain_recall_feedback",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "records a relevance verdict for a prior recall result and returns a confirmation, not note " +
+ "content; included for completeness of the file-level sweep only.",
+ },
+ {
+ surface: "brain_eval",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "scores retrieval quality (hit@k, MRR, …) over an operator-supplied dataset and returns " +
+ "metrics, not note bodies; included for completeness of the file-level sweep only.",
+ },
+ {
+ surface: "brain_codegraph_report",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "reports codegraph index state and counts - no vault note crosses this surface at all; " +
+ "included for completeness of the file-level sweep only.",
+ },
+ {
+ surface: "brain_context_presets",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "diagnoses/suggests context-pack preset configuration and returns config diffs, not note " +
+ "content; included for completeness of the file-level sweep only.",
+ },
+ {
+ surface: "second_brain_capabilities",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "returns the process's own capability report (tool counts, withheld-tool reasons) from " +
+ "ctx.capabilityReport - it never touches the vault, let alone a note path, title or body; " +
+ "included for completeness of the file-level sweep only. Registered as " +
+ "`name: CAPABILITY_DIAGNOSTIC_TOOL` rather than a literal, which is exactly why the sweep " +
+ "resolves constants: this row was missing while the census passed.",
+ },
+ {
+ surface: "second_brain_status",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "reports install/config/vault status blocks - no note path, title, or body crosses this " +
+ "surface; included for completeness of the file-level sweep only (tools.ts imports " +
+ "listVaultPages for second_brain_query, defined in the same file).",
+ },
+ {
+ surface: "vault_health",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "runs vault/config/plugin-manifest health checks and state-surface inventory - no note " +
+ "content crosses this surface; included for completeness of the file-level sweep only.",
+ },
+
+ // --- Excluded: CLI verbs, one per MCP tool above that has a CLI mirror ----
+ {
+ surface: "search expand",
+ kind: K.cliVerb,
+ category: C.excluded,
+ reason:
+ "src/cli/search/verbs/expand.ts calls expandHit() directly - the same chunk_id hydration " +
+ "brain_search_expand uses, with the same absence of a visibility check.",
+ },
+ {
+ surface: "brain backlinks",
+ kind: K.cliVerb,
+ category: C.excluded,
+ reason: "src/cli/brain/verbs/backlinks.ts calls buildBacklinkIndex, same as brain_backlinks.",
+ },
+ {
+ surface: "brain bridges",
+ kind: K.cliVerb,
+ category: C.excluded,
+ reason:
+ "src/cli/brain/verbs/bridges.ts drives the same bridge-discovery.ts core as brain_bridges.",
+ },
+ {
+ surface: "brain clusters",
+ kind: K.cliVerb,
+ category: C.excluded,
+ reason:
+ "src/cli/brain/verbs/clusters.ts drives the same communities.ts core as brain_clusters.",
+ },
+ {
+ surface: "brain moc-audit",
+ kind: K.cliVerb,
+ category: C.excluded,
+ reason:
+ "src/cli/brain/verbs/moc-audit.ts calls auditMoc, the same buildBacklinkIndex-backed, " +
+ "visibility-blind path brain_moc_audit uses.",
+ },
+ {
+ surface: "brain deep-synthesis",
+ kind: K.cliVerb,
+ category: C.excluded,
+ reason:
+ "src/cli/brain/verbs/deep-synthesis.ts drives the same mixed deepSynthesis() core as " +
+ "brain_deep_synthesis.",
+ },
+ {
+ surface: "brain ideas",
+ kind: K.cliVerb,
+ category: C.excluded,
+ reason:
+ "src/cli/brain/verbs/ideas.ts drives the same idea-discovery.ts core as brain_idea_discovery.",
+ },
+ {
+ surface: "brain dead-end",
+ kind: K.cliVerb,
+ category: C.excluded,
+ reason: "src/cli/brain/verbs/dead-end.ts drives the same dead-ends.ts core as brain_dead_ends.",
+ },
+ {
+ surface: "brain file-context",
+ kind: K.cliVerb,
+ category: C.excluded,
+ reason:
+ "src/cli/brain/verbs/file-context.ts calls fileContextRecall(), the same wrapper " +
+ "brain_file_context uses - listed here as an entry of record even though the underlying " +
+ "path is visibility-covered, so a reader of this registry does not have to cross-reference " +
+ "the MCP list to learn the CLI verb's answer.",
+ },
+ {
+ surface: "brain context-pack",
+ kind: K.cliVerb,
+ category: C.excluded,
+ reason: "src/cli/brain/verbs/context-pack.ts calls packContext, same as brain_context_pack.",
+ },
+ {
+ surface: "brain query",
+ kind: K.cliVerb,
+ category: C.excluded,
+ reason:
+ "src/cli/brain/verbs/query.ts calls queryByPreference / queryByTopic, the same " +
+ "visibility-blind Brain-record path brain_query uses.",
+ },
+ {
+ surface: "brain sources",
+ kind: K.cliVerb,
+ category: C.excluded,
+ reason:
+ "src/cli/brain/verbs/sources.ts calls aggregateSources, the same Brain signal-registry " +
+ "read brain_sources uses, with no visibility check either.",
+ },
+ {
+ surface: "brain audit",
+ kind: K.cliVerb,
+ category: C.excluded,
+ reason:
+ "src/cli/brain/verbs/audit.ts calls readPrefAudit, the same preference-mutation ledger " +
+ "read brain_audit uses; no note body or title crosses either surface.",
+ },
+ {
+ surface: "brain claims",
+ kind: K.cliVerb,
+ category: C.excluded,
+ reason:
+ "src/cli/brain/verbs/claims.ts drives the same claim-graph core (path, topic, principle " +
+ "text per row) that brain_claims's own docblock describes as agent-scope-gated only.",
+ },
+ {
+ surface: "brain truth",
+ kind: K.cliVerb,
+ category: C.excluded,
+ reason: "src/cli/brain/verbs/truth.ts drives the same entity claim ledger as brain_truth.",
+ },
+ {
+ surface: "brain label",
+ kind: K.cliVerb,
+ category: C.excluded,
+ reason:
+ "src/cli/brain/verbs/label.ts drives the same labels.ts core as brain_labels; neither " +
+ "surface returns a note title or body, only the caller's own path and a label set.",
+ },
+ {
+ surface: "brain tiers",
+ kind: K.cliVerb,
+ category: C.excluded,
+ reason:
+ "src/cli/brain/verbs/tiers.ts drives the same Store.listTierDrift() path as brain_tiers.",
+ },
+ {
+ surface: "brain secret",
+ kind: K.cliVerb,
+ category: C.excluded,
+ reason: "src/cli/brain/verbs/secret.ts mirrors brain_secrets; no note content crosses either.",
+ },
+ {
+ surface: "brain maintenance",
+ kind: K.cliVerb,
+ category: C.excluded,
+ reason:
+ "src/cli/brain/verbs/maintenance.ts mirrors brain_maintenance; no note content crosses either.",
+ },
+
+ // --- Excluded: MCP resources -----------------------------------------------
+ {
+ surface: "osb://preferences/active",
+ kind: K.mcpResource,
+ category: C.excluded,
+ reason:
+ "one of resources.ts's three whole-vault readers, deliberately unfiltered by its own " +
+ "docblock (Brain/active.md is shared by construction) - and visibility: is not part of " +
+ "that decision either way; the field is never consulted.",
+ },
+ {
+ surface: "osb://lessons",
+ kind: K.mcpResource,
+ category: C.excluded,
+ reason:
+ "same whole-vault-reader class as osb://preferences/active; visibility: is never consulted.",
+ },
+ {
+ surface: "osb://digest/latest",
+ kind: K.mcpResource,
+ category: C.excluded,
+ reason: "renderDigest() output, same unfiltered whole-vault-reader class; no visibility check.",
+ },
+ {
+ surface: "osb://status",
+ kind: K.mcpResource,
+ category: C.excluded,
+ reason:
+ "computeBrainStatus() output, same unfiltered whole-vault-reader class; no visibility check.",
+ },
+ {
+ surface: "osb://preference/{id}",
+ kind: K.mcpResource,
+ category: C.excluded,
+ reason:
+ "gated by gatedOwnerScopeView (agent-scope), per resources.ts's own docblock on why the " +
+ "four templated readers exist - a visibility-tagged preference page is returned in full, " +
+ "frontmatter and body, once the owner-scope check passes.",
+ },
+ {
+ surface: "osb://topic/{slug}",
+ kind: K.mcpResource,
+ category: C.excluded,
+ reason:
+ "same gatedOwnerScopeView-only path as osb://preference/{id}; visibility: is never read.",
+ },
+ {
+ surface: "osb://log/{date}",
+ kind: K.mcpResource,
+ category: C.excluded,
+ reason:
+ "withVisibleLogEvents filters rendered log sections by gatedOwnerScopeView over the " +
+ "artifacts each event names; no visibility check on those artifacts.",
+ },
+ {
+ surface: "osb://backlinks/{id}",
+ kind: K.mcpResource,
+ category: C.excluded,
+ reason:
+ "calls buildBacklinkIndex directly - resources.ts's own comment calls it 'the last " +
+ "unscoped buildBacklinkIndex call in the product' - gated only by gatedOwnerScopeView.",
+ },
+
+ // --- Excluded: the index's own storage --------------------------------------
+ {
+ surface: "search index chunks/chunk_fts tables",
+ kind: K.indexStore,
+ category: C.excluded,
+ reason:
+ "chunker.ts's packBlocks emits a page's frontmatter block as its own chunk verbatim " +
+ "(chunk_index 0), and the body as the chunks after it - indexer.ts applies no visibility " +
+ "predicate anywhere in that pass, so a private page's full text, frontmatter included, is " +
+ "stored in chunks and mirrored into chunk_fts regardless of any read-side filter. A copy of " +
+ "the index carries the content a read-side filter only ever hides at query time.",
+ },
+]);
+
+/** {@link VISIBILITY_SURFACE_REGISTRY}, restricted to the excluded rows. */
+export function excludedVisibilitySurfaces(): ReadonlyArray<VisibilitySurfaceEntry> {
+ return VISIBILITY_SURFACE_REGISTRY.filter((entry) => entry.category === C.excluded);
+}
+
+/**
+ * The rows that name something a caller can INVOKE - every kind but
+ * `index_store`, whose one row is a structural fact about the index's own
+ * storage and says so in its own vocabulary comment.
+ *
+ * The distinction exists because a count is reported to operators. "N of
+ * M note-returning surfaces" has to be M surfaces somebody could call;
+ * folding the storage fact into that denominator quietly inflates it by
+ * one, which is the kind of arithmetic this wave is about.
+ */
+export function callableVisibilitySurfaces(): ReadonlyArray<VisibilitySurfaceEntry> {
+ return VISIBILITY_SURFACE_REGISTRY.filter((entry) => entry.kind !== K.indexStore);
+}
+
+/** {@link callableVisibilitySurfaces}, restricted to the excluded rows. */
+export function excludedCallableVisibilitySurfaces(): ReadonlyArray<VisibilitySurfaceEntry> {
+ return callableVisibilitySurfaces().filter((entry) => entry.category === C.excluded);
+}
diff --git a/src/core/state/surfaces.ts b/src/core/state/surfaces.ts
index 1756b515..c7f4a149 100644
--- a/src/core/state/surfaces.ts
+++ b/src/core/state/surfaces.ts
@@ -105,6 +105,8 @@ export const STATE_SURFACE_ID = Object.freeze({
secretCustody: "secret_custody",
ingestContentManifest: "ingest_content_manifest",
ingestCheckpoints: "ingest_checkpoints",
+ sessionImportCheckpoints: "session_import_checkpoints",
+ writeDeadLetters: "write_dead_letters",
sessionImportLedger: "session_import_ledger",
installManifest: "install_manifest",
protectManifest: "protect_manifest",
@@ -255,6 +257,8 @@ const WRITER_LOCK_SUFFIX = ".lock";
const SECRETS_DIR = "secrets";
const INGEST_MANIFEST_FILE = "ingest-manifest.json";
const INGEST_CHECKPOINT_DIR = "ingest-checkpoints";
+const SESSION_CHECKPOINT_DIR = "session-import-checkpoints";
+const DEAD_LETTER_DIR = "dead-letters";
const SESSION_LEDGER_FILE = "session-import-ledger.json";
const INSTALL_MANIFEST_FILE = "install.lock.json";
const PROTECT_MANIFEST_FILE = "protect.lock.json";
@@ -417,7 +421,41 @@ export const STATE_SURFACES: ReadonlyArray<StateSurface> = Object.freeze([
reason:
"One resume point per ingest plan, so an interrupted batch continues rather than restarts. " +
"`OSB_INGEST_NO_CHECKPOINT` suppresses writing them; it does not move them.",
- sources: ["src/core/brain/ingest/checkpoint.ts"],
+ sources: ["src/core/brain/ingest/checkpoint.ts", "src/core/brain/checkpoint-store.ts"],
+ },
+ {
+ id: STATE_SURFACE_ID.sessionImportCheckpoints,
+ label: "session import checkpoints",
+ tier: STATE_TIER.derived,
+ derive: derivedStore(SESSION_CHECKPOINT_DIR),
+ override_env: null,
+ override_config_key: null,
+ carries_memory: false,
+ reason:
+ "One resume point per scoped session import, keyed on the transcript's identity and the " +
+ "turn boundary a run reached, so an interrupted import continues instead of re-reading and " +
+ "re-hashing every turn it already finished. Separate from the ingest checkpoints beside it " +
+ "because the key is a turn count in one file, not a set of completed paths in a plan. " +
+ "Deleting it costs re-work on the next interrupted run, never correctness: the dedup index " +
+ "still suppresses the repeated writes. `OSB_INGEST_NO_CHECKPOINT` suppresses writing them; " +
+ "it does not move them.",
+ sources: ["src/core/brain/sessions/checkpoint.ts", "src/core/brain/checkpoint-store.ts"],
+ },
+ {
+ id: STATE_SURFACE_ID.writeDeadLetters,
+ label: "multi-artifact write dead letters",
+ tier: STATE_TIER.vaultContent,
+ derive: derivedStore(DEAD_LETTER_DIR),
+ override_env: null,
+ override_config_key: null,
+ carries_memory: true,
+ reason:
+ "One record per multi-artifact commit that failed part way, naming the items that never " +
+ "reached disk, the payload they belonged to, and the first error. Nothing rebuilds it and " +
+ "nothing in this tool removes it: it is the only evidence that a partial write happened " +
+ "once the response carrying the error is gone, so a sweep would delete exactly what an " +
+ "operator has not read yet. Deleting one is a deliberate act after the payload is re-run.",
+ sources: ["src/core/brain/dead-letter.ts"],
},
{
id: STATE_SURFACE_ID.sessionImportLedger,
diff --git a/src/core/vault.ts b/src/core/vault.ts
index c970b576..9287d418 100644
--- a/src/core/vault.ts
+++ b/src/core/vault.ts
@@ -57,7 +57,7 @@ import {
atomicWriteFileSync,
isFileAlreadyExists,
} from "./fs-atomic.ts";
-import { stem } from "./fs-utils.ts";
+import { pathStem, stem } from "./fs-utils.ts";
import {
DEGRADATION_CODE,
type DegradationNotice,
@@ -621,24 +621,47 @@ export function listVaultBasenames(
vaultDir: string,
opts: ListVaultPagesOptions = {},
): Set<string> {
+ return new Set(
+ listVaultNotePaths(vaultDir, {
+ ...opts,
+ site: opts.site ?? LIST_VAULT_BASENAMES_SITE,
+ }).map(pathStem),
+ );
+}
+
+/**
+ * The same walk as {@link listVaultBasenames}, reported as vault-relative
+ * POSIX paths instead of basenames — no frontmatter parsing here either.
+ *
+ * Two callers need the two shapes for one reason: a basename is what a
+ * wikilink resolves against, and a PATH is what the ownership and
+ * visibility rules take. A caller that must ask "may this caller see it"
+ * before publishing the basename needs both, and deriving the path back
+ * from a basename is exactly the guess this project refuses.
+ */
+export function listVaultNotePaths(
+ vaultDir: string,
+ opts: ListVaultPagesOptions = {},
+): Array<string> {
const skipDirs = new Set(opts.skipDirs ?? DEFAULT_SKIP_DIRS);
const skipFiles = new Set((opts.skipFiles ?? DEFAULT_SKIP_FILES).map((f) => f.toLowerCase()));
- const out = new Set<string>();
+ const out: string[] = [];
walkBasenames(vaultDir, vaultDir, skipDirs, skipFiles, out, {
sink: opts.notices,
- site: opts.site ?? LIST_VAULT_BASENAMES_SITE,
+ site: opts.site ?? LIST_VAULT_NOTE_PATHS_SITE,
});
return out;
}
const LIST_VAULT_BASENAMES_SITE = "vault.listVaultBasenames";
+const LIST_VAULT_NOTE_PATHS_SITE = "vault.listVaultNotePaths";
function walkBasenames(
root: string,
dir: string,
skipDirs: Set<string>,
skipFiles: Set<string>,
- out: Set<string>,
+ out: string[],
notices: WalkNoticeSink,
): void {
let entries;
@@ -672,7 +695,7 @@ function walkBasenames(
const rel = relative(root, full);
const parts = rel.split(/[\\/]/);
if (parts.some((p) => skipDirs.has(p))) continue;
- out.add(stem(entry.name));
+ out.push(parts.join("/"));
}
}
diff --git a/src/core/write-binding/index.ts b/src/core/write-binding/index.ts
index fafc02d7..2543c87d 100644
--- a/src/core/write-binding/index.ts
+++ b/src/core/write-binding/index.ts
@@ -10,11 +10,14 @@
*
* There is no credential in this system to key a fence to. Agent
* identity resolves as `VAULT_AGENT_NAME`, else the config `agent_name`,
- * else the literal `agent`, and twenty-two MCP tool schemas across
- * fifteen modules additionally accept a caller-supplied `agent` string
- * that overrides it, taken verbatim after a placeholder check. (Count it
- * with `grep -rn '^\s*agent:\s*{' src/mcp/`; it has only ever grown, and
- * understating it understates this argument.) A caller who can call the tool can
+ * else the literal `agent`, and twenty-five MCP tool schemas
+ * additionally accept a caller-supplied `agent` string that overrides
+ * it, taken verbatim after a placeholder check. (That count is pinned as
+ * an equality by `tests/core/architecture/origin-channel-census.test.ts`,
+ * which measures it with `grep -rn '^\s*agent:\s*{' src/mcp/` - it used
+ * to be a number kept here by hand, and this docblock still said
+ * twenty-two after it had reached twenty-five. Understating it
+ * understates this argument.) A caller who can call the tool can
* name itself anything, so a fence keyed to that name is bypassed by
* passing a different string. The binding therefore reads NO identity at
* all: its whole authority is `<vault>/Brain/_brain.yaml`, which the
diff --git a/src/mcp/brain/context-tools.ts b/src/mcp/brain/context-tools.ts
index 26e70db3..44a34880 100644
--- a/src/mcp/brain/context-tools.ts
+++ b/src/mcp/brain/context-tools.ts
@@ -23,6 +23,7 @@ import {
type RegenerateActiveResult,
} from "../../core/brain/active.ts";
import { resolveOwnerScopeDelivery } from "../../core/brain/preferences-collect.ts";
+import { computeMaintenanceOverdueFlag } from "../../core/brain/status.ts";
import { parseFrontmatter } from "../../core/vault.ts";
import { readVaultInstructionFile } from "../../core/brain/vault-instruction-file.ts";
import { normalizeAgentArgument } from "../../core/agent-identity.ts";
@@ -323,6 +324,11 @@ async function toolBrainContext(ctx: ServerContext): Promise<Record<string, unkn
const dirs = brainDirs(ctx.vault);
const activePath = brainActivePath(ctx.vault);
const pinned = readPinnedContext(ctx.vault);
+ // Bounded to the single newest `Brain/log/` day - see
+ // `computeMaintenanceOverdueFlag`'s docblock for the cost bound. This
+ // is the always-loaded path, so the full `maintenance_debt` derivation
+ // on `computeBrainStatus` never runs here.
+ const maintenanceOverdue = computeMaintenanceOverdueFlag(ctx.vault);
// The operator's standing rules, resolved before anything else this
// surface assembles - INCLUDING the no-Brain return below. `brain_context`
@@ -347,6 +353,7 @@ async function toolBrainContext(ctx: ServerContext): Promise<Record<string, unkn
counts: EMPTY_CONTEXT_COUNTS,
generated_at: null,
pinned: serializePinnedContext(ctx, pinned),
+ maintenance_overdue: maintenanceOverdue,
...standingRulesField(ctx, standing),
};
}
@@ -435,6 +442,7 @@ async function toolBrainContext(ctx: ServerContext): Promise<Record<string, unkn
counts,
generated_at: generatedAt,
pinned: serializePinnedContext(ctx, pinned),
+ maintenance_overdue: maintenanceOverdue,
...(error ? { error } : {}),
...standingRulesField(ctx, standing),
...(vaultInstruction
@@ -534,7 +542,16 @@ const PINNED_CONTEXT_OUTPUT_SCHEMA: NonNullable<ToolDefinition["outputSchema"]>
const BRAIN_CONTEXT_OUTPUT_SCHEMA: NonNullable<ToolDefinition["outputSchema"]> = {
type: "object",
- required: ["vault_path", "present", "active_path", "content", "counts", "generated_at", "pinned"],
+ required: [
+ "vault_path",
+ "present",
+ "active_path",
+ "content",
+ "counts",
+ "generated_at",
+ "pinned",
+ "maintenance_overdue",
+ ],
properties: {
vault_path: VAULT_PATH_OUTPUT_SCHEMA,
present: { type: "boolean" },
@@ -553,6 +570,14 @@ const BRAIN_CONTEXT_OUTPUT_SCHEMA: NonNullable<ToolDefinition["outputSchema"]> =
},
generated_at: {},
pinned: PINNED_CONTEXT_OUTPUT_SCHEMA,
+ // `boolean | null` (empty schema, matching `generated_at` above): a
+ // cheap, bounded proxy over the single newest `Brain/log/` day only -
+ // `null` when there is no log history yet or the newest day could
+ // not be read. See `computeMaintenanceOverdueFlag`'s docblock
+ // (`src/core/brain/status.ts`) for exactly what it measures and its
+ // cost bound; it is NOT the precise `maintenance_debt` figure that
+ // `second_brain_status` / `osb://status` report.
+ maintenance_overdue: {},
// Declared but deliberately NOT required: a vault whose operator has
// written no standing rules omits the key entirely, exactly as the
// vault-instruction field does, so hosts that strip unknown fields
diff --git a/src/mcp/brain/design-note-tools.ts b/src/mcp/brain/design-note-tools.ts
index ff64de40..8aa10698 100644
--- a/src/mcp/brain/design-note-tools.ts
+++ b/src/mcp/brain/design-note-tools.ts
@@ -11,6 +11,7 @@
*/
import { commitDesignNote, DesignNoteError, planDesignNote } from "../../core/brain/design-note.ts";
+import { gatedOwnerScopeView } from "../../core/brain/owner-scope-view.ts";
import { ResponseCheckError } from "../../core/brain/response-checks.ts";
import { ResponseShapeError } from "../../core/brain/response-shape.ts";
import { resolveAgentName } from "../../core/config.ts";
@@ -37,7 +38,10 @@ async function toolBrainDesignNote(
return wrapToolErrors(TOOL, NAMED_REFUSALS, async () => {
if (args["note"] === undefined) {
- const report = planDesignNote(ctx.vault, topic, { now: new Date() });
+ const report = planDesignNote(ctx.vault, topic, {
+ now: new Date(),
+ ownerScope: gatedOwnerScopeView(ctx.vault, ctx.agentName).scope,
+ });
const g = report.grounding;
return {
phase: "plan",
diff --git a/src/mcp/brain/ingest-tools.ts b/src/mcp/brain/ingest-tools.ts
index bbff20fb..365fa71b 100644
--- a/src/mcp/brain/ingest-tools.ts
+++ b/src/mcp/brain/ingest-tools.ts
@@ -9,8 +9,10 @@
* Idempotent on the source path.
*/
+import { serializeImportCensus } from "../../core/brain/import-census.ts";
import { planBatches, type BatchPlan } from "../../core/brain/ingest/batch-plan.ts";
-import { clearCheckpoint } from "../../core/brain/ingest/checkpoint.ts";
+import { clearCheckpoint, PLAN_ID_LABEL } from "../../core/brain/ingest/checkpoint.ts";
+import { assertCheckpointId } from "../../core/brain/checkpoint-store.ts";
import { ingestSource } from "../../core/brain/ingest/ingest.ts";
import { reconcilePlan } from "../../core/brain/ingest/reconcile.ts";
import { IntakeValidationError } from "../../core/brain/intake/extract-intake.ts";
@@ -31,6 +33,7 @@ import {
coerceStrList,
} from "../coerce.ts";
import { MCP_PREVIEW_BUDGET } from "../preview-budget.ts";
+import { INVALID_PARAMS, MCPError } from "../protocol.ts";
import type { ServerContext, ToolDefinition } from "../tool-contract.ts";
import { parseExtractionIntakeArgs } from "./intake-args.ts";
import { enforceCountGuard, readCountGuardArgs, wrapToolErrors } from "./shared.ts";
@@ -51,6 +54,19 @@ async function toolBrainIngestSource(
const sourcePath = coerceStr(args, "source_path", true)!;
const summary = coerceStr(args, "summary", true)!;
const planId = coerceStr(args, "plan_id", false) ?? undefined;
+ // `plan_id` is caller-supplied and becomes a checkpoint filename, so the
+ // store's grammar is enforced HERE, where the caller can be told it is
+ // their argument that is wrong. Left to the ingest it would surface as an
+ // INTERNAL_ERROR, and before that it was swallowed entirely: the ingest
+ // succeeded, resume never engaged, and `--reconcile` reported the whole
+ // batch as never ingested.
+ if (planId !== undefined) {
+ try {
+ assertCheckpointId(PLAN_ID_LABEL, planId);
+ } catch (err) {
+ throw new MCPError(INVALID_PARAMS, `${TOOL}: ${(err as Error).message}`);
+ }
+ }
const preExtract = coerceBoolOptional(args, "pre_extract") ?? false;
// This tool names its source as `source_path` and hands it to the pipeline
// itself, so `source` is not part of its contract. Declaring that stops the
@@ -292,6 +308,7 @@ async function toolBrainIngestBatchPlan(
ingested: [...report.ingested],
missing: [...report.missing],
complete: report.complete,
+ census: serializeImportCensus(report.census),
},
};
}
diff --git a/src/mcp/brain/knowledge-tools.ts b/src/mcp/brain/knowledge-tools.ts
index fcc0b1de..701075ca 100644
--- a/src/mcp/brain/knowledge-tools.ts
+++ b/src/mcp/brain/knowledge-tools.ts
@@ -440,7 +440,11 @@ async function toolBrainDiarize(
const report = diarize(
ctx.vault,
{ query, ...(category ? { category } : {}) },
- { now: new Date() },
+ // The envelope's candidate list names Brain artifact ids, so it is
+ // filtered through the same gated view every other report surface
+ // here uses - an unscoped list would publish another owner's
+ // preference ids as citation targets.
+ { now: new Date(), ownerScope: gatedOwnerScopeView(ctx.vault, ctx.agentName).scope },
);
return {
entity_id: report.entityId,
diff --git a/src/mcp/brain/notes-tools.ts b/src/mcp/brain/notes-tools.ts
index 4e5eccf1..56a5483e 100644
--- a/src/mcp/brain/notes-tools.ts
+++ b/src/mcp/brain/notes-tools.ts
@@ -90,11 +90,19 @@ export function advisoryFields(code: string): Readonly<Record<string, unknown>>
/**
* Surface codes that are NOT the caller's fault. A malformed
* `Brain/_brain.yaml` refuses every write on this surface no matter what
- * arguments arrive, so reporting it as INVALID_PARAMS would send the
- * agent looking at its own request forever. It is reported as an
- * INTERNAL_ERROR that is nonetheless typed and carries its exit.
+ * arguments arrive, and a target the process cannot read refuses this
+ * one no matter how the request is rephrased - the path is right and the
+ * host is what is broken. Reporting either as INVALID_PARAMS would send
+ * the agent looking at its own request forever. Both are reported as an
+ * INTERNAL_ERROR that is nonetheless typed and carries its details.
+ * Frontmatter the scanner cannot round-trip joins them for the same
+ * reason: the note on disk is what needs fixing, not the request.
*/
-const OPERATOR_FAULT_CODES: ReadonlySet<string> = new Set(["config_invalid"]);
+const OPERATOR_FAULT_CODES: ReadonlySet<string> = new Set([
+ "config_invalid",
+ "target_unreadable",
+ "target_frontmatter_lossy",
+]);
/** JSON-RPC code for a refusal, split on whose fault the refusal is. */
function rpcCodeFor(surfaceCode: string): number {
@@ -277,6 +285,29 @@ async function toolBrainCreateNote(
}
}
+/**
+ * The `content` an update carries, distinguishing ABSENT from EMPTY.
+ *
+ * {@link coerceStr} collapses `""` and a whitespace-only string to
+ * `null`, which every other surface reads as "the caller did not send
+ * this argument". On an update that reading is wrong in a way that loses
+ * the caller's intent: `{path, frontmatter, content: ""}` asked for the
+ * body to be cleared and was answered with `updated: true` over a body
+ * left exactly as it was, and `{path, content: ""}` was refused as if no
+ * argument had arrived at all. Both are silent about what happened to
+ * the request. Reading presence directly lets the empty body reach the
+ * kernel, where it is either refused by name or - with `allow_empty` -
+ * honoured.
+ */
+function updateBodyArgument(args: Record<string, unknown>): string | undefined {
+ const raw = args["content"];
+ if (raw === undefined || raw === null) return undefined;
+ if (typeof raw !== "string") {
+ throw new MCPError(INVALID_PARAMS, "brain_update_note: argument 'content' must be a string");
+ }
+ return raw;
+}
+
/**
* Update an existing note: merge frontmatter keys and/or replace the
* body. A single-operation batch over kernel 2 so a mid-write failure
@@ -289,8 +320,9 @@ async function toolBrainUpdateNote(
): Promise<Record<string, unknown>> {
const path = coerceStr(args, "path", true)!;
const frontmatter = parseFrontmatterArg(args["frontmatter"], "brain_update_note");
- const content = coerceStr(args, "content", false);
- if (frontmatter === undefined && content === null) {
+ const content = updateBodyArgument(args);
+ const allowEmpty = coerceBoolOptional(args, "allow_empty");
+ if (frontmatter === undefined && content === undefined) {
throw new MCPError(
INVALID_PARAMS,
"brain_update_note: provide 'frontmatter', 'content', or both",
@@ -300,7 +332,8 @@ async function toolBrainUpdateNote(
kind: "update_note",
path,
...(frontmatter !== undefined ? { frontmatter } : {}),
- ...(content !== null ? { body: content } : {}),
+ ...(content !== undefined ? { body: content } : {}),
+ ...(allowEmpty !== undefined ? { allowEmpty } : {}),
};
const result = runSingleWrite(ctx, op, "brain_update_note");
// The flag comes off the kernel result rather than being restated here:
@@ -460,6 +493,11 @@ export const NOTES_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([
type: "string",
description: "Replacement Markdown body. Omit to keep the existing body.",
},
+ allow_empty: {
+ type: "boolean",
+ description:
+ "Allow empty or whitespace-only content to clear the note. Default false: a blank body over a note that has text is refused as an accidental clear.",
+ },
},
required: ["path"],
additionalProperties: false,
diff --git a/src/mcp/brain/write-batch-tools.ts b/src/mcp/brain/write-batch-tools.ts
index bcddadb0..798e49b6 100644
--- a/src/mcp/brain/write-batch-tools.ts
+++ b/src/mcp/brain/write-batch-tools.ts
@@ -56,6 +56,15 @@ function optionalStr(raw: Record<string, unknown>, key: string): string | undefi
return value;
}
+function optionalBool(raw: Record<string, unknown>, key: string): boolean | undefined {
+ const value = raw[key];
+ if (value === undefined || value === null) return undefined;
+ if (typeof value !== "boolean") {
+ throw new MCPError(INVALID_PARAMS, `brain_write_batch: '${key}' must be a boolean`);
+ }
+ return value;
+}
+
/**
* Map one untrusted request operation object onto a typed
* {@link WriteOperation}. `resolveAgent` supplies the caller identity for
@@ -86,6 +95,7 @@ function mapOperation(
case "update_note": {
const frontmatter = parseFrontmatterArg(obj["frontmatter"], "brain_write_batch");
const content = optionalStr(obj, "content");
+ const allowEmpty = optionalBool(obj, "allow_empty");
if (frontmatter === undefined && content === undefined) {
throw new MCPError(
INVALID_PARAMS,
@@ -97,6 +107,7 @@ function mapOperation(
path: requireStr(obj, "path", "update_note"),
...(frontmatter !== undefined ? { frontmatter } : {}),
...(content !== undefined ? { body: content } : {}),
+ ...(allowEmpty !== undefined ? { allowEmpty } : {}),
};
}
case "append_note":
@@ -221,6 +232,11 @@ export const WRITE_BATCH_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([
description:
"Note body: create_note/update_note replacement, or append_note appended text.",
},
+ allow_empty: {
+ type: "boolean",
+ description:
+ "update_note only: allow empty or whitespace-only content to clear the note. Default false; a blank body over a note that has text is refused.",
+ },
pref_id: { type: "string", description: "Preference id for apply_evidence." },
artifact: { type: "string", description: "Artifact wikilink for apply_evidence." },
result: {
diff --git a/src/mcp/http.ts b/src/mcp/http.ts
index e2566755..c23b15a5 100644
--- a/src/mcp/http.ts
+++ b/src/mcp/http.ts
@@ -28,6 +28,7 @@ import { MCPServer, type MCPServerOptions, type MCPServerRuntimeOptions } from "
import { errorResponse, type JsonRpcResponse } from "./server.ts";
import { INTERNAL_ERROR, INVALID_REQUEST, PARSE_ERROR } from "./protocol.ts";
import { DRAIN_STATE, RequestDrain, resolveDrainDeadlineMs, type DrainOutcome } from "./drain.ts";
+import { ORIGIN_CHANNEL, setOriginChannel } from "../core/origin-channel.ts";
export interface ServeHttpOptions {
readonly host?: string;
@@ -62,6 +63,9 @@ export async function startHttp(
opts: ServeHttpOptions = {},
runtimeOpts: MCPServerRuntimeOptions = {},
): Promise<HttpServerHandle> {
+ // See the same claim in `serveStdio`: the transport is what knows this
+ // process serves MCP tool calls (Unit C).
+ setOriginChannel(ORIGIN_CHANNEL.mcpTool);
const apiKey = opts.apiKey ?? null;
const host = opts.host ?? "127.0.0.1";
// Safe by default: on the loopback default a bearer is optional (the
diff --git a/src/mcp/resources.ts b/src/mcp/resources.ts
index 36448a83..942208bc 100644
--- a/src/mcp/resources.ts
+++ b/src/mcp/resources.ts
@@ -65,7 +65,11 @@ import { regenerateLessons } from "../core/brain/lessons.ts";
import { buildBacklinkIndex } from "../core/brain/backlinks.ts";
import type { BacklinkRef } from "../core/brain/backlinks.ts";
import { renderDigest } from "../core/brain/digest.ts";
-import { computeBrainStatus, type BrainStatusSnapshot } from "../core/brain/status.ts";
+import {
+ computeBrainStatus,
+ MAINTENANCE_DEBT_STATUS,
+ type BrainStatusSnapshot,
+} from "../core/brain/status.ts";
import {
brainActivePath,
brainLessonsPath,
@@ -584,6 +588,24 @@ function renderStatusMarkdown(s: BrainStatusSnapshot): string {
lines.push(`- last dream: ${s.last_dream_at ?? "_(never)_"}`);
lines.push(`- last apply-evidence: ${s.last_apply_evidence_at ?? "_(never)_"}`);
lines.push("");
+ lines.push("## Maintenance debt");
+ lines.push("");
+ const debtCount = s.maintenance_debt.log_events_since_dream;
+ const debtNoun = debtCount === 1 ? "log event" : "log events";
+ if (s.maintenance_debt.status === MAINTENANCE_DEBT_STATUS.neverDreamed) {
+ lines.push(`- never dreamed — ${debtCount} ${debtNoun} recorded since inception`);
+ } else if (s.maintenance_debt.status === MAINTENANCE_DEBT_STATUS.undercounted) {
+ // The figure is a floor: at least one shard in the window would not
+ // read or would not parse. Printing it as a count would be the
+ // partial walk presented as a total.
+ lines.push(
+ `- at least ${debtCount} ${debtNoun} since the last dream — a log shard could not be ` +
+ "read in full, so this is a lower bound",
+ );
+ } else {
+ lines.push(`- ${debtCount} ${debtNoun} since the last dream`);
+ }
+ lines.push("");
if (s.sanity.signals_awaiting_dream > 0) {
lines.push("## Sanity");
lines.push("");
diff --git a/src/mcp/stdio.ts b/src/mcp/stdio.ts
index a8c681e2..16314cc2 100644
--- a/src/mcp/stdio.ts
+++ b/src/mcp/stdio.ts
@@ -22,6 +22,7 @@ import { MCPServer, type MCPServerOptions, type MCPServerRuntimeOptions } from "
import { errorResponse, type JsonRpcResponse } from "./server.ts";
import { INVALID_REQUEST, PARSE_ERROR, type JsonRpcNotification } from "./protocol.ts";
import { RequestDrain, resolveDrainDeadlineMs, type DrainOutcome } from "./drain.ts";
+import { ORIGIN_CHANNEL, setOriginChannel } from "../core/origin-channel.ts";
/**
* Any frame this transport writes: a response to a request, or a
@@ -66,6 +67,14 @@ export async function serveStdio(
ioOpts: ServeStdioOptions = {},
runtimeOpts: MCPServerRuntimeOptions = {},
): Promise<number> {
+ // This process serves MCP tool calls, so it claims the `mcp-tool`
+ // origin channel for every record written under it (Unit C). Claimed at
+ // the TRANSPORT and not in the `MCPServer` constructor: `o2b tool-call`
+ // and the MCP probe build a server to invoke a handler from a shell, and
+ // stamping those `mcp-tool` would name a transport that carried nothing.
+ // Redundant when `o2b mcp` started us - it claimed the same value - and
+ // load-bearing when a host embeds this transport directly.
+ setOriginChannel(ORIGIN_CHANNEL.mcpTool);
const stdin = ioOpts.stdin ?? process.stdin;
const stdout = ioOpts.stdout ?? process.stdout;
// This transport owns a duplex stream, so it CAN write a frame nobody
diff --git a/tests/cli/brain-query-as-of.test.ts b/tests/cli/brain-query-as-of.test.ts
index 45a8cdae..1c738d0c 100644
--- a/tests/cli/brain-query-as-of.test.ts
+++ b/tests/cli/brain-query-as-of.test.ts
@@ -36,7 +36,18 @@ const AFTER_EXPIRY = "2026-08-01T00:00:00Z";
const LAPSED_PRINCIPLE = "Principle for lapsed";
const EVERGREEN_PRINCIPLE = "Principle for evergreen";
-/** Exact pre-change stdout for `--topic deploy --json`, no new flags. */
+/**
+ * Exact stdout for `--topic deploy --json`, no new flags.
+ *
+ * `origin_channel` is the server-derived channel of the process that
+ * WROTE the signal (Unit C of the nothing-writes-silently wave), read
+ * back by `parseSignal` and surfaced here because this payload is the
+ * parsed signal. `unset` is the explicit literal for "no entry point
+ * claimed this process": the fixture writes its signals through
+ * `writeSignal` directly rather than through `o2b`, and the suite
+ * preload unclaims the channel before every test so this value cannot
+ * depend on which test file ran first.
+ */
const PINNED_JSON_STDOUT = `{
"signals": [
{
@@ -51,7 +62,8 @@ const PINNED_JSON_STDOUT = `{
"topic": "deploy",
"signal": "positive",
"agent": "tester",
- "principle": "Principle for evergreen"
+ "principle": "Principle for evergreen",
+ "origin_channel": "unset"
}
],
"preference": null,
diff --git a/tests/cli/search-check-integrity.test.ts b/tests/cli/search-check-integrity.test.ts
index 2a0b546b..61bdcd39 100644
--- a/tests/cli/search-check-integrity.test.ts
+++ b/tests/cli/search-check-integrity.test.ts
@@ -334,6 +334,15 @@ test("without --integrity the verb spends nothing and emits the bytes it always
// code now depends on the difference.
"provider_probe",
"provider_reason",
+ // The pending-vector census (nothing-writes-silently, unit A) sits
+ // in a slot of its own rather than replacing anything, and it is
+ // present in every state - including the one where no count could
+ // be taken, which is the state it exists to report.
+ "pending_vectors",
+ // The record-vs-data embedder audit (unit G) rides beside it on the
+ // same terms: present in every state, including the one where
+ // nothing could be compared.
+ "embedder_record",
"warnings",
"fatal",
"recommendations",
diff --git a/tests/cli/search-check-visibility-honesty.test.ts b/tests/cli/search-check-visibility-honesty.test.ts
new file mode 100644
index 00000000..a3050581
--- /dev/null
+++ b/tests/cli/search-check-visibility-honesty.test.ts
@@ -0,0 +1,90 @@
+/**
+ * `o2b search check`'s visibility honesty finding
+ * (nothing-writes-silently, unit H, form B).
+ *
+ * The defect: `visibility: private` frontmatter reads as a privacy
+ * boundary and is a caller-supplied view filter honoured by exactly one
+ * lane (`search()`'s pool-filters pipeline). This finding names the gap,
+ * with a count derived from `visibility-surface-registry.ts`'s own
+ * exported list - never a hand-written number - and only when the
+ * vault/index actually carries at least one tagged page.
+ *
+ * The claims pinned here:
+ *
+ * 1. A vault that never uses `visibility:` renders neither the JSON key
+ * nor the human line - both shapes stay silent on it.
+ * 2. A vault with a tagged page renders `visibility_honesty` in JSON
+ * with `excluded_surface_count` / `total_surface_count`, and the
+ * human line under the same label, beside `pending_vectors:` and
+ * `embedder_record:`.
+ * 3. The finding never changes the exit code - it is an honesty
+ * statement, not a fault.
+ */
+
+import { expect, test } from "bun:test";
+
+import { runCli } from "../helpers/run-cli.ts";
+import { createTempVault, makeConfig, writeMd } from "../helpers/search-fixtures.ts";
+import { indexVault } from "../../src/core/search/indexer.ts";
+import {
+ callableVisibilitySurfaces,
+ excludedCallableVisibilitySurfaces,
+} from "../../src/core/search/visibility-surface-registry.ts";
+
+test("a vault that never uses visibility: renders neither the JSON key nor the human line", async () => {
+ const { vault, dbPath, cleanup } = createTempVault("check-visibility-honesty-absent");
+ try {
+ writeMd(vault, "a.md", "# A\n\nAn ordinary note.");
+ await indexVault(makeConfig({ vault, dbPath }));
+
+ const json = await runCli(["search", "check", "--json", "--vault", vault, "--db", dbPath]);
+ expect(json.returncode).toBe(0);
+ const parsed = JSON.parse(json.stdout) as Record<string, unknown>;
+ expect("visibility_honesty" in parsed).toBe(false);
+
+ const human = await runCli(["search", "check", "--vault", vault, "--db", dbPath]);
+ expect(human.stdout).not.toContain("visibility_honesty");
+ } finally {
+ cleanup();
+ }
+});
+
+test("a vault with one visibility-tagged page renders the finding in both shapes", async () => {
+ const { vault, dbPath, cleanup } = createTempVault("check-visibility-honesty-present");
+ try {
+ writeMd(vault, "a.md", "---\nvisibility: private\n---\n\n# A\n\nA private page.");
+ await indexVault(makeConfig({ vault, dbPath }));
+
+ const json = await runCli(["search", "check", "--json", "--vault", vault, "--db", dbPath]);
+ expect(json.returncode).toBe(0);
+ const parsed = JSON.parse(json.stdout) as Record<string, unknown>;
+ const finding = parsed["visibility_honesty"] as Record<string, unknown>;
+ expect(finding).toBeDefined();
+ expect(finding["excluded_surface_count"]).toBe(excludedCallableVisibilitySurfaces().length);
+ expect(finding["total_surface_count"]).toBe(callableVisibilitySurfaces().length);
+
+ const human = await runCli(["search", "check", "--vault", vault, "--db", dbPath]);
+ expect(human.returncode).toBe(0);
+ expect(human.stdout).toContain("visibility_honesty:");
+ expect(human.stdout).toContain("caller-supplied view filter, not a privacy boundary");
+ expect(human.stdout).toContain(`${excludedCallableVisibilitySurfaces().length} of the `);
+ // The line names its own scope: the census's population, not the
+ // product's whole surface area.
+ expect(human.stdout).toContain("the visibility census enumerates");
+ } finally {
+ cleanup();
+ }
+});
+
+test("the finding never moves the exit code", async () => {
+ const { vault, dbPath, cleanup } = createTempVault("check-visibility-honesty-exit");
+ try {
+ writeMd(vault, "a.md", "---\nvisibility: private\n---\n\n# A\n\nA private page.");
+ await indexVault(makeConfig({ vault, dbPath }));
+
+ const out = await runCli(["search", "check", "--vault", vault, "--db", dbPath]);
+ expect(out.returncode).toBe(0);
+ } finally {
+ cleanup();
+ }
+});
diff --git a/tests/cli/search-query-flag-manifest.test.ts b/tests/cli/search-query-flag-manifest.test.ts
index 858558a9..885c7bf9 100644
--- a/tests/cli/search-query-flag-manifest.test.ts
+++ b/tests/cli/search-query-flag-manifest.test.ts
@@ -76,6 +76,16 @@ const COVERED: ReadonlyArray<CoveredVerb> = Object.freeze([
minFlags: 5,
witness: ["no-probe", "boolean"],
},
+ {
+ // Added with the verb itself (nothing-writes-silently, unit G): a
+ // repair verb whose `--apply` gate is not modelled advertises a
+ // dry run with no way to reach the write it exists for.
+ verb: "restamp",
+ source: "restamp.ts",
+ marker: "export async function cmdSearchRestamp",
+ minFlags: 4,
+ witness: ["apply", "boolean"],
+ },
{
// Added with `--progress` (nothing-runs-unwatched, U1): this verb
// grew the whole progress spine in core - an options field, a
diff --git a/tests/cli/search-restamp.test.ts b/tests/cli/search-restamp.test.ts
new file mode 100644
index 00000000..f4619864
--- /dev/null
+++ b/tests/cli/search-restamp.test.ts
@@ -0,0 +1,218 @@
+/**
+ * `o2b search restamp` (nothing-writes-silently, unit G).
+ *
+ * The sqlite-vec version an index recorded is a pure ABI token: it
+ * describes the extension build the vectors were written under, not the
+ * vectors themselves. Two peers on different sqlite-vec builds each see
+ * the other's store as drifted, which is exactly why the ABI gate
+ * defaults to `warn` rather than refusing - and until now the only way
+ * to clear that warning was to re-embed the whole vault at provider
+ * prices for a token no vector depends on.
+ *
+ * The claims pinned here:
+ *
+ * 1. Dry run is the DEFAULT: it prints the change it would make and
+ * leaves `index_state` byte-identical.
+ * 2. `--apply` restamps the vec_version cell and nothing else.
+ * 3. A second run has nothing to do and says so, rather than
+ * re-writing a cell that already holds the value.
+ * 4. A recorded model or dimension that disagrees with this build is
+ * REFUSED by name, with the deferral stated: that repair needs
+ * re-embedding, which this verb will not do behind an operator's
+ * back.
+ * 5. An absent index is refused, naming the path.
+ * 6. No path through the verb contacts an embedding provider.
+ */
+
+import { afterEach, beforeEach, expect, test } from "bun:test";
+import { Database } from "bun:sqlite";
+import { join } from "node:path";
+
+import { indexVault } from "../../src/core/search/indexer.ts";
+import {
+ EMBEDDING_DIMENSION_STATE_KEY,
+ EMBEDDING_MODEL_STATE_KEY,
+ EMBEDDING_VEC_VERSION_STATE_KEY,
+} from "../../src/core/search/store.ts";
+import { getState, setState } from "../../src/core/search/store/state.ts";
+import { createTempVault, makeConfig, writeMd } from "../helpers/search-fixtures.ts";
+import { startFakeHttp, type FakeHttp } from "../helpers/fake-http.ts";
+import { runCli } from "../helpers/run-cli.ts";
+import { sqliteVecLoadable } from "../helpers/sqlite-vec.ts";
+
+const VEC_LOADABLE = sqliteVecLoadable();
+
+/** A version string no sqlite-vec build reports, so the drift is real. */
+const STALE_VEC_VERSION = "v0.0.0-from-another-machine";
+
+let vault: string;
+let dbPath: string;
+let configPath: string;
+let cleanup: () => void;
+let server: FakeHttp;
+
+beforeEach(async () => {
+ const v = createTempVault("search-restamp");
+ vault = v.vault;
+ dbPath = v.dbPath;
+ cleanup = v.cleanup;
+ server = await startFakeHttp();
+ configPath = join(vault, "cli-config.yaml");
+});
+
+afterEach(async () => {
+ cleanup();
+ await server.close();
+});
+
+function semanticConfig() {
+ return makeConfig({
+ vault,
+ dbPath,
+ semantic: {
+ enabled: true,
+ provider: "openai-compat",
+ baseUrl: server.url,
+ model: "fake-model",
+ apiKey: "test-key",
+ dimension: 4,
+ timeoutMs: 5_000,
+ },
+ });
+}
+
+async function writeConfig(): Promise<void> {
+ await Bun.write(
+ configPath,
+ [
+ `vault: "${vault}"`,
+ "search_semantic_enabled: true",
+ "embedding_provider: openai-compat",
+ `embedding_base_url: "${server.url}"`,
+ "embedding_model: fake-model",
+ "embedding_api_key: test-key",
+ "embedding_dimension: 4",
+ "",
+ ].join("\n"),
+ );
+}
+
+function readState(key: string): string | null {
+ const db = new Database(dbPath, { readonly: true });
+ try {
+ return getState(db, key);
+ } finally {
+ db.close();
+ }
+}
+
+function writeStateDirectly(key: string, value: string): void {
+ const db = new Database(dbPath);
+ try {
+ setState(db, key, value);
+ } finally {
+ db.close();
+ }
+}
+
+async function indexWithVectors(): Promise<void> {
+ writeMd(vault, "a.md", "# A\n\nA note with vectors.");
+ await indexVault(semanticConfig(), { embeddings: true });
+ await writeConfig();
+}
+
+function restamp(...extra: ReadonlyArray<string>) {
+ return runCli(
+ ["search", "restamp", "--vault", vault, "--db", dbPath, "--config", configPath, ...extra],
+ { env: { OPEN_SECOND_BRAIN_CONFIG: configPath } },
+ );
+}
+
+test.skipIf(!VEC_LOADABLE)("the dry run prints the change and writes nothing", async () => {
+ await indexWithVectors();
+ const current = readState(EMBEDDING_VEC_VERSION_STATE_KEY);
+ expect(typeof current).toBe("string");
+ writeStateDirectly(EMBEDDING_VEC_VERSION_STATE_KEY, STALE_VEC_VERSION);
+
+ const callsBefore = server.callCount();
+ const run = await restamp();
+ expect(run.returncode).toBe(0);
+ expect(run.stdout).toContain("dry-run");
+ expect(run.stdout).toContain(STALE_VEC_VERSION);
+ expect(run.stdout).toContain(current as string);
+ expect(run.stdout).toContain("--apply");
+ // The cell is exactly as the dry run found it.
+ expect(readState(EMBEDDING_VEC_VERSION_STATE_KEY)).toBe(STALE_VEC_VERSION);
+ // Nothing here asks a provider anything: the only calls on the record
+ // are the ones the fixture's own indexing run made.
+ expect(server.callCount()).toBe(callsBefore);
+});
+
+test.skipIf(!VEC_LOADABLE)("--apply restamps the vec_version cell and nothing else", async () => {
+ await indexWithVectors();
+ const current = readState(EMBEDDING_VEC_VERSION_STATE_KEY) as string;
+ const model = readState(EMBEDDING_MODEL_STATE_KEY);
+ const dimension = readState(EMBEDDING_DIMENSION_STATE_KEY);
+ writeStateDirectly(EMBEDDING_VEC_VERSION_STATE_KEY, STALE_VEC_VERSION);
+
+ const callsBefore = server.callCount();
+ const run = await restamp("--apply", "--json");
+ expect(run.returncode).toBe(0);
+ const payload = JSON.parse(run.stdout) as Record<string, unknown>;
+ expect(payload["applied"]).toBe(true);
+ expect(payload["field"]).toBe(EMBEDDING_VEC_VERSION_STATE_KEY);
+ expect(payload["recorded"]).toBe(STALE_VEC_VERSION);
+ expect(payload["runtime"]).toBe(current);
+
+ expect(readState(EMBEDDING_VEC_VERSION_STATE_KEY)).toBe(current);
+ expect(readState(EMBEDDING_MODEL_STATE_KEY)).toBe(model);
+ expect(readState(EMBEDDING_DIMENSION_STATE_KEY)).toBe(dimension);
+ expect(server.callCount()).toBe(callsBefore);
+});
+
+test.skipIf(!VEC_LOADABLE)("a store already at this version has nothing to do", async () => {
+ await indexWithVectors();
+
+ const run = await restamp("--json");
+ expect(run.returncode).toBe(0);
+ const payload = JSON.parse(run.stdout) as Record<string, unknown>;
+ expect(payload["changed"]).toBe(false);
+ expect(payload["applied"]).toBe(false);
+});
+
+test.skipIf(!VEC_LOADABLE)(
+ "a recorded dimension that disagrees is refused by name, with the deferral stated",
+ async () => {
+ await indexWithVectors();
+ writeStateDirectly(EMBEDDING_VEC_VERSION_STATE_KEY, STALE_VEC_VERSION);
+ writeStateDirectly(EMBEDDING_DIMENSION_STATE_KEY, "8");
+
+ const callsBefore = server.callCount();
+ const run = await restamp("--apply");
+ expect(run.returncode).not.toBe(0);
+ expect(run.stderr).toContain(EMBEDDING_DIMENSION_STATE_KEY);
+ expect(run.stderr).toContain("deferred");
+ // The refusal wrote nothing: the stale token is still there.
+ expect(readState(EMBEDDING_VEC_VERSION_STATE_KEY)).toBe(STALE_VEC_VERSION);
+ expect(server.callCount()).toBe(callsBefore);
+ },
+);
+
+test.skipIf(!VEC_LOADABLE)("a recorded model that disagrees is refused by name too", async () => {
+ await indexWithVectors();
+ writeStateDirectly(EMBEDDING_VEC_VERSION_STATE_KEY, STALE_VEC_VERSION);
+ writeStateDirectly(EMBEDDING_MODEL_STATE_KEY, "some-other-model");
+
+ const run = await restamp("--apply");
+ expect(run.returncode).not.toBe(0);
+ expect(run.stderr).toContain(EMBEDDING_MODEL_STATE_KEY);
+ expect(readState(EMBEDDING_VEC_VERSION_STATE_KEY)).toBe(STALE_VEC_VERSION);
+});
+
+test("an absent index is refused, naming the path", async () => {
+ await writeConfig();
+ const run = await restamp();
+ expect(run.returncode).not.toBe(0);
+ expect(run.stderr).toContain(dbPath);
+ expect(server.callCount()).toBe(0);
+});
diff --git a/tests/core/architecture/origin-channel-census.test.ts b/tests/core/architecture/origin-channel-census.test.ts
new file mode 100644
index 00000000..c69ce0cb
--- /dev/null
+++ b/tests/core/architecture/origin-channel-census.test.ts
@@ -0,0 +1,210 @@
+/**
+ * Unit C — the origin channel is server-derived, and that is measured.
+ *
+ * The cardinal rule of this unit is a negative, so it is asserted as one.
+ * Agent identity resolves to a caller-supplied `agent` string on 25 MCP
+ * tool schemas, taken verbatim after a placeholder check; the write
+ * binding's own docblock says so and tells you to recount it. A channel
+ * that a caller could name would be exactly that again - an unverifiable
+ * claim with a schema in front of it. So no schema property, no CLI flag,
+ * and no writer input field may spell the channel, and this census is
+ * what proves it rather than a sentence in a docblock.
+ *
+ * Claims pinned here:
+ * 1. No MCP tool schema declares a property spelling the channel, in
+ * any of its spellings, anywhere under `src/mcp/` - except the one
+ * recall-telemetry read filter excused by name below.
+ * 2. No CLI flag or command-manifest entry names it, with the same one
+ * exception on the CLI side of that filter.
+ * 3. The caller-facing input types of the four stamped writers declare
+ * no channel field, so TypeScript cannot be talked past either.
+ * 4. The `agent` recount that motivates the rule is still what this
+ * wave measured - an equality, so this fails when the unverifiable
+ * surface grows AND when it shrinks. Growth is the event worth
+ * learning about, and a floor could not report it.
+ */
+
+import { describe, expect, test } from "bun:test";
+import { readdirSync, readFileSync } from "node:fs";
+import { join, relative } from "node:path";
+
+import { ORIGIN_CHANNELS } from "../../../src/core/origin-channel.ts";
+
+const REPO_ROOT = join(import.meta.dir, "..", "..", "..");
+
+interface SourceFile {
+ readonly path: string;
+ readonly text: string;
+}
+
+function readTree(rel: string): SourceFile[] {
+ const files: SourceFile[] = [];
+ const walk = (dir: string): void => {
+ for (const entry of readdirSync(dir, { withFileTypes: true })) {
+ const abs = join(dir, entry.name);
+ if (entry.isDirectory()) walk(abs);
+ else if (entry.name.endsWith(".ts")) {
+ files.push({
+ path: relative(REPO_ROOT, abs).split("\\").join("/"),
+ text: readFileSync(abs, "utf8"),
+ });
+ }
+ }
+ };
+ walk(join(REPO_ROOT, rel));
+ return files.toSorted((a, b) => (a.path < b.path ? -1 : a.path > b.path ? 1 : 0));
+}
+
+/**
+ * Every spelling the channel could arrive under. `channel` alone is in
+ * the set because the field does not have to carry the full name to be
+ * the same lie - a tool taking `channel: "cli"` is the refused design
+ * wearing a shorter word.
+ */
+const CHANNEL_SPELLINGS: ReadonlyArray<string> = Object.freeze([
+ "origin_channel",
+ "originChannel",
+ "origin-channel",
+ "channel",
+]);
+
+/**
+ * The one caller-supplied `channel` in the tree, excused by name.
+ *
+ * `RECALL_CHANNEL` (`brain/recall-telemetry.ts`) is a DIFFERENT axis that
+ * shipped first: the transport a recall was DELIVERED over, whose members
+ * (`mcp` / `cli` / `hook`) rhyme with the origin channel's closely enough
+ * that a bare `channel` grep cannot tell them apart. Both surfaces below
+ * take it as a read-side FILTER over telemetry the caller itself
+ * recorded, which is a query parameter and not a provenance claim.
+ *
+ * Excused rather than dropped from {@link CHANNEL_SPELLINGS}: the bare
+ * word is exactly how the refused design would arrive - a tool taking
+ * `channel: "cli"` is the same lie in fewer letters - so the grep keeps
+ * it and this list keeps the two apart by name. A THIRD `channel`
+ * property fails here and has to argue its way in.
+ */
+const RECALL_CHANNEL_FILTER_SITES: ReadonlyArray<string> = Object.freeze([
+ "src/cli/brain/verbs/recall-telemetry.ts: channel x1",
+ "src/mcp/brain/recall-tools.ts: channel x1",
+]);
+
+/**
+ * A JSON-schema property declaration, in the shape the write binding's
+ * own `agent` recount uses (`grep -rn '^\s*agent:\s*{' src/mcp/`). The
+ * schemas in this tree declare one property per line with the key at the
+ * start of it, so the anchor is what distinguishes a DECLARATION from a
+ * mention in prose or a local variable.
+ */
+function propertyDeclarationRe(key: string): RegExp {
+ return new RegExp(`^\\s*(?:"${key}"|'${key}'|${key})\\s*:\\s*\\{`, "gm");
+}
+
+/**
+ * A `parseFlags` spec entry or a command-manifest `flag(...)` entry. Both
+ * shapes, because the CLI declares a flag in two places and a rule that
+ * reads one of them is a rule about that one: `parseFlags` takes an
+ * object whose keys are the flag names (quoted or bare), and the manifest
+ * takes `flag("name", …)`.
+ */
+function flagDeclarationRe(key: string): RegExp {
+ return new RegExp(
+ `^\\s*(?:"${key}"|'${key}'|${key})\\s*:\\s*\\{|flag\\(\\s*["']${key}["']`,
+ "gm",
+ );
+}
+
+function hits(files: ReadonlyArray<SourceFile>, re: (key: string) => RegExp): string[] {
+ const found: string[] = [];
+ for (const file of files) {
+ for (const spelling of CHANNEL_SPELLINGS) {
+ const matches = file.text.match(re(spelling));
+ if (matches !== null) found.push(`${file.path}: ${spelling} x${matches.length}`);
+ }
+ }
+ return found.toSorted();
+}
+
+const MCP_FILES = readTree("src/mcp");
+const CLI_FILES = readTree("src/cli");
+
+describe("no caller can name the origin channel", () => {
+ test("no MCP tool schema declares a channel property", () => {
+ // Named, not counted: a failure has to say which schema reopened it.
+ expect(hits(MCP_FILES, propertyDeclarationRe)).toEqual(
+ RECALL_CHANNEL_FILTER_SITES.filter((site) => site.startsWith("src/mcp/")),
+ );
+ });
+
+ test("no CLI flag or command-manifest entry names it", () => {
+ expect(hits(CLI_FILES, flagDeclarationRe)).toEqual(
+ RECALL_CHANNEL_FILTER_SITES.filter((site) => site.startsWith("src/cli/")),
+ );
+ });
+
+ test("the vocabulary values are not reachable as caller input either", () => {
+ // The complement of the two greps above: a schema could take the
+ // channel under some other key and still be the refused design, so
+ // the enum members themselves must not appear in a schema `enum`.
+ const enumHits: string[] = [];
+ for (const file of MCP_FILES) {
+ for (const channel of ORIGIN_CHANNELS) {
+ const re = new RegExp(`enum\\s*:\\s*\\[[^\\]]*["']${channel}["']`, "g");
+ if (re.test(file.text)) enumHits.push(`${file.path}: ${channel}`);
+ }
+ }
+ expect(enumHits.toSorted()).toEqual([]);
+ });
+
+ test("the four stamped writers declare no caller-facing channel field", () => {
+ // The type-level half. Each of these is the one input interface a
+ // caller fills in for that family; a field here would let an
+ // in-process caller assert what the process alone may derive.
+ const WRITER_INPUT_TYPES: ReadonlyArray<readonly [string, string]> = Object.freeze([
+ ["src/core/brain/log.ts", "AppendLogEventOptions"],
+ ["src/core/brain/signal.ts", "WriteSignalInput"],
+ ["src/core/brain/continuity/types.ts", "AppendContinuityRecordInput"],
+ ["src/core/brain/notes/create-note.ts", "CreateNoteInput"],
+ ]);
+ const offenders: string[] = [];
+ for (const [path, typeName] of WRITER_INPUT_TYPES) {
+ const text = readFileSync(join(REPO_ROOT, path), "utf8");
+ const start = text.indexOf(`interface ${typeName} {`);
+ expect(start).toBeGreaterThanOrEqual(0);
+ const end = text.indexOf("\n}", start);
+ const block = text.slice(start, end);
+ for (const spelling of CHANNEL_SPELLINGS) {
+ if (new RegExp(`readonly\\s+${spelling}\\??\\s*:`).test(block)) {
+ offenders.push(`${path}: ${typeName}.${spelling}`);
+ }
+ }
+ }
+ expect(offenders.toSorted()).toEqual([]);
+ });
+});
+
+describe("the unverifiable surface this rule exists for", () => {
+ /**
+ * The caller-supplied `agent` property count, MEASURED and pinned as an
+ * equality - the convention the write-site census states for its own
+ * row counts: a moved number is a finding to name in the commit that
+ * moves it, not a re-measurement chore.
+ *
+ * It was a floor, and a floor could not fail on the event both its
+ * comments said it watched for. `>= 25` passes at 35, so ten new
+ * schemas taking `agent` verbatim would have widened the unverifiable
+ * surface this whole rule is argued from and reported nothing. An
+ * equality fails in both directions; when it does, recount with the
+ * regex below, update this constant, and say in the commit which
+ * schemas moved it.
+ */
+ const CALLER_SUPPLIED_AGENT_PROPERTIES = 25;
+
+ test("is exactly what the rule was argued from, in both directions", () => {
+ let count = 0;
+ for (const file of MCP_FILES) {
+ count += (file.text.match(/^\s*agent:\s*\{/gm) ?? []).length;
+ }
+ expect(count).toBe(CALLER_SUPPLIED_AGENT_PROPERTIES);
+ });
+});
diff --git a/tests/core/architecture/state-surface-census.test.ts b/tests/core/architecture/state-surface-census.test.ts
index 0a78a780..2fea0ce1 100644
--- a/tests/core/architecture/state-surface-census.test.ts
+++ b/tests/core/architecture/state-surface-census.test.ts
@@ -77,10 +77,10 @@ const STORE_DIR_IDENTIFIER = "DERIVED_STORE_DIR";
* glob that stopped matching would sweep an empty set clean and every
* assertion below would pass over nothing.
*/
-const SWEPT_POPULATION_SIZE = 20;
+const SWEPT_POPULATION_SIZE = 21;
/** Declared surfaces today. Pinned for the same reason. */
-const DECLARED_SURFACE_COUNT = 40;
+const DECLARED_SURFACE_COUNT = 42;
/** An exclusion reason has to be an argument, not a label. */
const MIN_REASON_LENGTH = 80;
diff --git a/tests/core/architecture/verdict-vocabulary-census.test.ts b/tests/core/architecture/verdict-vocabulary-census.test.ts
index f57325b6..d8a6b95d 100644
--- a/tests/core/architecture/verdict-vocabulary-census.test.ts
+++ b/tests/core/architecture/verdict-vocabulary-census.test.ts
@@ -391,6 +391,16 @@ import {
STATE_TIER,
STATE_TIERS,
} from "../../../src/core/state/surfaces.ts";
+import {
+ isReconciliationOutcome,
+ RECONCILIATION_OUTCOME,
+ RECONCILIATION_OUTCOMES,
+} from "../../../src/core/reconciliation-report.ts";
+import {
+ isOriginChannel,
+ ORIGIN_CHANNEL,
+ ORIGIN_CHANNELS,
+} from "../../../src/core/origin-channel.ts";
import { lexCode } from "../../helpers/source-lexer.ts";
interface VocabularyUnderCensus {
@@ -1202,6 +1212,32 @@ const CENSUS: ReadonlyArray<VocabularyUnderCensus> = Object.freeze([
members: EXPORT_FORMATS,
guard: isExportFormat,
},
+ {
+ // nothing-writes-silently, shared substrate. The wave's one
+ // reconciliation report shape (attempted/found/missing) is consumed
+ // by three unrelated lanes - import read-back, envelope write
+ // accounting, embedder record-vs-data audit - and `contradicted` is
+ // the third state none of the pair-of-booleans vocabularies it
+ // replaces could name: a recorded claim that disagrees with what the
+ // report itself measured, not merely a shortfall against it.
+ name: "RECONCILIATION_OUTCOME",
+ values: RECONCILIATION_OUTCOME,
+ members: RECONCILIATION_OUTCOMES,
+ guard: isReconciliationOutcome,
+ },
+ {
+ // nothing-writes-silently, unit C. The wave's newest on-disk
+ // vocabulary, and the one that nearly escaped: the trio was complete
+ // from the start, but the object shipped as a bare `as const` rather
+ // than frozen, so the scan below - which enumerates frozen bindings -
+ // could not see it. Its members are load-bearing on-disk strings, and
+ // a rename that missed the guard would have failed the census for
+ // every other vocabulary in the tree and passed for this one.
+ name: "ORIGIN_CHANNEL",
+ values: ORIGIN_CHANNEL,
+ members: ORIGIN_CHANNELS,
+ guard: isOriginChannel,
+ },
]);
// ---------------------------------------------------------------------------
@@ -1466,7 +1502,7 @@ const SCANNED = scanVocabularies(SOURCE_TREE);
* How many four-piece vocabularies `src/` currently holds. Measured, and
* kept as an equality rather than a floor - see the population test.
*/
-const VOCABULARY_POPULATION = 71;
+const VOCABULARY_POPULATION = 73;
const REGISTERED = new Map(CENSUS.map((entry) => [entry.name, entry] as const));
describe("verdict vocabulary census", () => {
diff --git a/tests/core/architecture/visibility-surface-census.test.ts b/tests/core/architecture/visibility-surface-census.test.ts
new file mode 100644
index 00000000..6aeb900f
--- /dev/null
+++ b/tests/core/architecture/visibility-surface-census.test.ts
@@ -0,0 +1,642 @@
+/**
+ * Unit H (form B) — every surface that can hand a vault note's path,
+ * title, or body back to a caller, measured rather than described
+ * (nothing-writes-silently).
+ *
+ * `graph/visibility.ts`'s `visibility:` frontmatter is caller-liftable
+ * scoping, not a boundary, and it is wired into exactly ONE place:
+ * `search()`'s pool-filters pipeline. This file is the coverage map that
+ * fact needs, in the shape `write-site-census.test.ts` already
+ * established for a different gap: a closed category vocabulary, a
+ * per-entry written reason, a population pin, and a mechanism that fails
+ * on an unclassified newcomer rather than describing the population in
+ * prose.
+ *
+ * The claims pinned here:
+ *
+ * 1. Every MCP tool file that imports a known note-content-returning
+ * primitive is swept into a population; every REAL registered tool
+ * name (from `buildToolTable("full")`) that a swept file defines
+ * has an entry in `VISIBILITY_SURFACE_REGISTRY`, in both
+ * directions - a newly-swept tool with no entry fails, and a
+ * registry entry for a tool no swept file defines fails too.
+ * 2. Every MCP resource `listResources()` / `listResourceTemplates()`
+ * advertises has a registry entry, exactly.
+ * 3. Every CLI-verb registry entry names a `<group> <verb>` pair that
+ * actually exists in `command-manifest.ts`.
+ * 4. Every registry entry carries exactly one of the two closed
+ * categories and a reason of meaningful length that is not a
+ * placeholder.
+ * 5. The population is pinned as an equality, the way
+ * `DIRECT_WRITE_ROWS` pins write-site-census's count - a moved
+ * number is a finding to name, not a re-measurement chore.
+ * 6. The census can fail: a synthetic file importing a known
+ * note-content primitive and defining a tool name in the (also
+ * synthetic) real-tool set is reported unclassified.
+ * 7. `search()` is the only lane wired to `applyVisibilityScope` -
+ * pinned by grepping `src/core/search/` for the call site.
+ *
+ * ## The mechanical anchor, and what it cannot catch
+ *
+ * MCP population: a file under `src/mcp/` is a candidate the moment it
+ * imports one of {@link NOTE_CONTENT_PRODUCERS} - an identifier taken
+ * from a module suffix already verified, by reading the source, to hand
+ * back a page's path, title, or body. That is exactly write-site-census's
+ * `VAULT_VOCABULARY_IMPORT_RE` shape, aimed at reads instead of writes.
+ * Within a candidate FILE, every tool-registration `name:` is extracted -
+ * both the string literals and the `name: SOME_CONSTANT` form, resolved
+ * through the constants the file declares and the ones it imports - and
+ * kept only when the resulting name ALSO appears in
+ * `buildToolTable("full")`'s real, live tool list, which turns a
+ * merely-plausible match (a `name:` field on some unrelated object) into a
+ * real, currently-registered tool with no hand-picked per-tool filter to
+ * go stale.
+ *
+ * Constant resolution is not decoration. Roughly a dozen MCP tool files
+ * register their tools through constants, and `src/mcp/tools.ts` - in
+ * population, and swept since this census shipped - registers
+ * `second_brain_capabilities` as `name: CAPABILITY_DIAGNOSTIC_TOOL`. A
+ * literals-only sweep reported a population of 42 that was 43 and left
+ * that tool with no registry row, which is the failure mode the design's
+ * risk section names: a census that quietly omits a surface reproduces
+ * the defect it documents.
+ *
+ * The population unit is the FILE, not the individual tool: a file that
+ * imports one primitive for one of its tools sweeps in every OTHER real
+ * tool name that file also defines (over-inclusion is the safe direction,
+ * matching write-site-census's own stated principle), so several entries
+ * in the registry below carry a reason of "swept in for file-level
+ * completeness; this handler returns no note content" rather than a real
+ * disclosure - stated in each such entry rather than left to be guessed.
+ *
+ * What this cannot see, stated rather than implied: a tool that reads
+ * note content through a NEW primitive not yet in the vocabulary (the
+ * same blind spot write-site-census states for a new write shape); a
+ * tool name built at runtime rather than written down - a template
+ * literal, a concatenation, a value read from a table - since constant
+ * resolution reads `const NAME = "literal"` declarations only, and a
+ * constant re-exported through a third module (the resolver follows one
+ * hop, from the registering file to the file it imported the name from);
+ * a tool that reaches a listed primitive only through a transitive import
+ * two modules away (the population rule reads the MCP file's own
+ * imports, not its whole call graph - deep-synthesis.ts is in
+ * population via `deepSynthesis`, not because knowledge-tools.ts itself
+ * imports `search`); and `src/openclaw/` (the OpenClaw native plugin
+ * adapter), which the charter scoped this sweep away from even though
+ * `src/openclaw/index.ts` hosts its own `listVaultPages`-backed page
+ * search with no visibility check - named here, not swept, and worth a
+ * follow-up census of its own.
+ *
+ * CLI population is NOT mechanically discovered the way the MCP one is:
+ * it is hand-enumerated, one row per MCP tool above that has a CLI
+ * mirror, and each row is only CROSS-CHECKED for existence against
+ * `command-manifest.ts` via `nestedCommand`. A wholly new CLI verb with
+ * no MCP counterpart would not be caught by this file at all - a real
+ * gap, named so a reader does not assume otherwise.
+ */
+
+import { describe, expect, test } from "bun:test";
+import { readdirSync, readFileSync } from "node:fs";
+import { join, relative } from "node:path";
+
+import { lexSource, type LexedSource } from "../../helpers/source-lexer.ts";
+import { buildToolTable } from "../../../src/mcp/tools.ts";
+import { listResources, listResourceTemplates } from "../../../src/mcp/resources.ts";
+import { nestedCommand } from "../../../src/cli/command-manifest.ts";
+import {
+ VISIBILITY_SURFACE_CATEGORY,
+ VISIBILITY_SURFACE_KIND,
+ VISIBILITY_SURFACE_REGISTRY,
+ type VisibilitySurfaceEntry,
+} from "../../../src/core/search/visibility-surface-registry.ts";
+
+const REPO_ROOT = join(import.meta.dir, "..", "..", "..");
+const MCP_ROOT = join(REPO_ROOT, "src", "mcp");
+const SRC_ROOT = join(REPO_ROOT, "src");
+
+// ─────────────────────────────────────────────────────────────────────────────
+// Tree reading, shared with the intruder fixtures below
+// ─────────────────────────────────────────────────────────────────────────────
+
+interface CensusFile {
+ readonly path: string;
+ readonly text: string;
+}
+
+function readTree(root: string): CensusFile[] {
+ const files: CensusFile[] = [];
+ const walk = (dir: string): void => {
+ for (const entry of readdirSync(dir, { withFileTypes: true })) {
+ const abs = join(dir, entry.name);
+ if (entry.isDirectory()) walk(abs);
+ else if (entry.name.endsWith(".ts")) {
+ files.push({
+ path: relative(REPO_ROOT, abs).split("\\").join("/"),
+ text: readFileSync(abs, "utf8"),
+ });
+ }
+ }
+ };
+ walk(root);
+ files.sort((a, b) => (a.path < b.path ? -1 : a.path > b.path ? 1 : 0));
+ return files;
+}
+
+const LEXED = new Map<string, { readonly text: string; readonly views: LexedSource }>();
+
+function lexedViews(file: CensusFile): LexedSource {
+ const cached = LEXED.get(file.path);
+ if (cached !== undefined && cached.text === file.text) return cached.views;
+ const views = lexSource(file.text);
+ LEXED.set(file.path, { text: file.text, views });
+ return views;
+}
+
+// ─────────────────────────────────────────────────────────────────────────────
+// MCP tool population
+// ─────────────────────────────────────────────────────────────────────────────
+
+interface ProducerRule {
+ /** A substring the import specifier must contain. */
+ readonly specifierIncludes: string;
+ /** Any one of these named imports, present alongside that specifier, qualifies. */
+ readonly identifiers: ReadonlyArray<string>;
+}
+
+/**
+ * Known note-content-returning primitives, keyed by the module each is
+ * imported from. Every entry is source-verified: the reason field on the
+ * matching {@link VISIBILITY_SURFACE_REGISTRY} row cites exactly how.
+ */
+const NOTE_CONTENT_PRODUCERS: ReadonlyArray<ProducerRule> = Object.freeze([
+ {
+ specifierIncludes: "/vault.ts",
+ identifiers: [
+ "parseFrontmatter",
+ "parseFrontmatterWithNotices",
+ "parseFrontmatterText",
+ "parseFrontmatterTextWithNotices",
+ "listVaultPages",
+ ],
+ },
+ { specifierIncludes: "/search/search.ts", identifiers: ["search"] },
+ { specifierIncludes: "/search/index.ts", identifiers: ["search", "expandHit"] },
+ { specifierIncludes: "/search/cards.ts", identifiers: ["expandHit"] },
+ { specifierIncludes: "/search/store.ts", identifiers: ["Store"] },
+ { specifierIncludes: "/brain/file-recall.ts", identifiers: ["fileContextRecall"] },
+ { specifierIncludes: "/brain/context-pack.ts", identifiers: ["packContext"] },
+ { specifierIncludes: "/brain/backlinks.ts", identifiers: ["buildBacklinkIndex"] },
+ {
+ specifierIncludes: "/brain/link-graph/unlinked-mentions.ts",
+ identifiers: ["findUnlinkedMentions"],
+ },
+ { specifierIncludes: "/brain/portability/sources.ts", identifiers: ["aggregateSources"] },
+ { specifierIncludes: "/brain/query.ts", identifiers: ["queryByPreference", "queryByTopic"] },
+ { specifierIncludes: "/brain/pref-audit.ts", identifiers: ["readPrefAudit"] },
+ {
+ specifierIncludes: "/brain/link-graph/bridge-discovery.ts",
+ identifiers: [
+ "discoverBridges",
+ "writeBridgeProposals",
+ "readDismissedBridges",
+ "dismissBridge",
+ "acceptBridge",
+ ],
+ },
+ {
+ specifierIncludes: "/brain/link-graph/communities.ts",
+ identifiers: ["detectCommunities", "materializeClusterNotes"],
+ },
+ {
+ specifierIncludes: "/brain/deep-synthesis.ts",
+ identifiers: ["deepSynthesis", "synthesisCandidates", "synthesisFindingsJson"],
+ },
+ {
+ specifierIncludes: "/brain/idea-discovery.ts",
+ identifiers: ["discoverIdeas", "ideaCandidates"],
+ },
+ { specifierIncludes: "/brain/link-graph/moc-audit.ts", identifiers: ["auditMoc"] },
+ { specifierIncludes: "/brain/dead-ends.ts", identifiers: ["listDeadEnds", "recordDeadEnd"] },
+ {
+ specifierIncludes: "/brain/claim-graph.ts",
+ identifiers: [
+ "currentTruth",
+ "rebuildClaimGraph",
+ "whatReplaced",
+ "whatContests",
+ "truthAt",
+ "allClaims",
+ ],
+ },
+ {
+ specifierIncludes: "/brain/truth/",
+ identifiers: ["computeTruthStateWithConflicts", "aggregateQuantities", "detectAgentCollisions"],
+ },
+]);
+
+/**
+ * Every `import { … } from "…"` statement's named-import list and
+ * specifier, read off {@link LexedSource.withoutComments} - the import
+ * specifier and the imported NAMES are both strings/identifiers a
+ * structural scan must read as written, not the `code` view that blanks
+ * literal contents (the same split write-site-census's own import
+ * clause uses, and for the same reason).
+ */
+const IMPORT_RE = /import\s*(?:type\s*)?\{([^}]*)\}\s*from\s*"([^"]+)"/g;
+
+/** Whether `text` imports any {@link NOTE_CONTENT_PRODUCERS} primitive. */
+function importsNoteContentPrimitive(text: string): boolean {
+ for (const match of text.matchAll(IMPORT_RE)) {
+ const names = new Set(
+ match[1]!.split(",").map((n) =>
+ n
+ .trim()
+ .split(/\s+as\s+/)[0]!
+ .trim(),
+ ),
+ );
+ const specifier = match[2]!;
+ for (const rule of NOTE_CONTENT_PRODUCERS) {
+ if (!specifier.includes(rule.specifierIncludes)) continue;
+ if (rule.identifiers.some((id) => names.has(id))) return true;
+ }
+ }
+ return false;
+}
+
+/** Every `name: "…"` tool-registration string literal a file's text contains. */
+const TOOL_NAME_LITERAL_RE = /\bname:\s*"([a-z][a-z0-9_]*)"/g;
+
+/**
+ * Every `name: SOME_CONSTANT` registration. A tool name does not have to
+ * be written at the registration site: `src/mcp/tools.ts` registers
+ * `second_brain_capabilities` as `name: CAPABILITY_DIAGNOSTIC_TOOL`, and
+ * a dozen other MCP files register theirs through constants too. Reading
+ * literals only left those invisible INSIDE an already-swept file, which
+ * is the one blind spot a reader of this census would not expect.
+ */
+const TOOL_NAME_CONST_RE = /\bname:\s*([A-Z][A-Z0-9_]*)\b/g;
+
+/** A `const NAME = "value"` declaration, exported or not. */
+const STRING_CONST_RE =
+ /\b(?:export\s+)?const\s+([A-Z][A-Z0-9_]*)\s*(?::[^=;]+)?=\s*"([a-z][a-z0-9_]*)"/g;
+
+/** Named imports and the specifier they came from, for constant resolution. */
+const NAMED_IMPORT_RE = /import\s*(?:type\s*)?\{([^}]*)\}\s*from\s*"([^"]+)"/g;
+
+function toolNameLiteralsIn(text: string): string[] {
+ return [...text.matchAll(TOOL_NAME_LITERAL_RE)].map((m) => m[1]!);
+}
+
+/** `const NAME = "value"` pairs a file declares, by constant name. */
+function stringConstantsIn(text: string): Map<string, string> {
+ const found = new Map<string, string>();
+ for (const match of text.matchAll(STRING_CONST_RE)) found.set(match[1]!, match[2]!);
+ return found;
+}
+
+/**
+ * Resolve a relative import specifier against the importing file's path,
+ * to the same repo-relative spelling {@link readTree} produces. Only
+ * `./` and `../` specifiers resolve; a bare package specifier has no
+ * file in the tree and returns `null`.
+ */
+function resolveSpecifier(fromPath: string, specifier: string): string | null {
+ if (!specifier.startsWith(".")) return null;
+ const segments = fromPath.split("/").slice(0, -1);
+ for (const part of specifier.split("/")) {
+ if (part === "." || part === "") continue;
+ if (part === "..") segments.pop();
+ else segments.push(part);
+ }
+ return segments.join("/");
+}
+
+/** Which file each named import in `file` was imported from. */
+function importSources(file: CensusFile, text: string): Map<string, string> {
+ const sources = new Map<string, string>();
+ for (const match of text.matchAll(NAMED_IMPORT_RE)) {
+ const resolved = resolveSpecifier(file.path, match[2]!);
+ if (resolved === null) continue;
+ for (const raw of match[1]!.split(",")) {
+ const local = raw
+ .trim()
+ .split(/\s+as\s+/)[0]!
+ .trim();
+ if (local.length > 0) sources.set(local, resolved);
+ }
+ }
+ return sources;
+}
+
+/**
+ * The MCP tool population: real, currently-registered tool names defined
+ * in a file that imports a note-content primitive - whether the name is
+ * written at the registration site or reached through a constant the
+ * file declares or imports. `realToolNames` is passed in rather than read
+ * globally so the fixtures below can exercise the same function with a
+ * synthetic registry.
+ *
+ * A constant resolves through the file's own declarations first, then
+ * through the file it was imported from. It resolves to NOTHING when
+ * neither answers - an unresolved identifier contributes no name rather
+ * than a guessed one - and the real-tool filter is the backstop either
+ * way.
+ */
+function discoverMcpToolPopulation(
+ files: ReadonlyArray<CensusFile>,
+ realToolNames: ReadonlySet<string>,
+): ReadonlySet<string> {
+ const constantsByFile = new Map<string, Map<string, string>>();
+ for (const file of files) {
+ constantsByFile.set(file.path, stringConstantsIn(lexedViews(file).withoutComments));
+ }
+ const found = new Set<string>();
+ for (const file of files) {
+ const text = lexedViews(file).withoutComments;
+ if (!importsNoteContentPrimitive(text)) continue;
+ for (const literal of toolNameLiteralsIn(text)) {
+ if (realToolNames.has(literal)) found.add(literal);
+ }
+ const own = constantsByFile.get(file.path)!;
+ const imported = importSources(file, text);
+ for (const match of text.matchAll(TOOL_NAME_CONST_RE)) {
+ const identifier = match[1]!;
+ const source = imported.get(identifier);
+ const value =
+ own.get(identifier) ??
+ (source === undefined ? undefined : constantsByFile.get(source)?.get(identifier));
+ if (value !== undefined && realToolNames.has(value)) found.add(value);
+ }
+ }
+ return found;
+}
+
+const MCP_SOURCE_TREE = readTree(MCP_ROOT);
+const REAL_TOOL_NAMES: ReadonlySet<string> = new Set(buildToolTable("full").map((t) => t.name));
+const MCP_TOOL_POPULATION = discoverMcpToolPopulation(MCP_SOURCE_TREE, REAL_TOOL_NAMES);
+
+const REGISTRY_BY_KIND = (
+ kind: (typeof VISIBILITY_SURFACE_KIND)[keyof typeof VISIBILITY_SURFACE_KIND],
+) => VISIBILITY_SURFACE_REGISTRY.filter((e) => e.kind === kind);
+
+// ─────────────────────────────────────────────────────────────────────────────
+// Reason quality, matching write-site-census's bar
+// ─────────────────────────────────────────────────────────────────────────────
+
+const MIN_REASON_LENGTH = 80;
+const LAZY_REASON_RE = /\bTODO\b|\bfor now\b|\blater\b/i;
+
+function reasonProblems(entries: ReadonlyArray<VisibilitySurfaceEntry>): {
+ thin: string[];
+ lazy: string[];
+} {
+ const thin: string[] = [];
+ const lazy: string[] = [];
+ for (const e of entries) {
+ if (e.reason.trim().length < MIN_REASON_LENGTH) thin.push(`${e.surface} (${e.reason.length})`);
+ if (LAZY_REASON_RE.test(e.reason)) lazy.push(e.surface);
+ }
+ return { thin, lazy };
+}
+
+// ─────────────────────────────────────────────────────────────────────────────
+// Population pins - equalities, the way write-site-census pins its own counts
+// ─────────────────────────────────────────────────────────────────────────────
+
+/** Measured: MCP tools whose file imports a note-content primitive. */
+const MCP_TOOL_POPULATION_SIZE = 43;
+/** Measured: MCP resources + templates, all excluded. */
+const MCP_RESOURCE_POPULATION_SIZE = 8;
+/** Measured: hand-enumerated CLI verb mirrors. */
+const CLI_VERB_POPULATION_SIZE = 20;
+
+describe("visibility surface census", () => {
+ test("every category is one of the two closed values", () => {
+ const bad = VISIBILITY_SURFACE_REGISTRY.filter(
+ (e) =>
+ e.category !== VISIBILITY_SURFACE_CATEGORY.covered &&
+ e.category !== VISIBILITY_SURFACE_CATEGORY.excluded,
+ );
+ expect(bad).toEqual([]);
+ });
+
+ test("every entry carries a reason of meaningful length, not a placeholder", () => {
+ const { thin, lazy } = reasonProblems(VISIBILITY_SURFACE_REGISTRY);
+ expect(thin.toSorted().join("\n")).toBe("");
+ expect(lazy.toSorted().join("\n")).toBe("");
+ });
+
+ test("no surface name is registered twice under the same kind", () => {
+ const seen = new Map<string, number>();
+ for (const e of VISIBILITY_SURFACE_REGISTRY) {
+ const key = `${e.kind}:${e.surface}`;
+ seen.set(key, (seen.get(key) ?? 0) + 1);
+ }
+ const dupes = [...seen.entries()].filter(([, n]) => n > 1).map(([k]) => k);
+ expect(dupes).toEqual([]);
+ });
+
+ describe("MCP tools", () => {
+ test("every swept tool name has a registry entry, and every registry entry names a swept tool", () => {
+ const registered = new Set(
+ REGISTRY_BY_KIND(VISIBILITY_SURFACE_KIND.mcpTool).map((e) => e.surface),
+ );
+ const unclassified = [...MCP_TOOL_POPULATION].filter((name) => !registered.has(name));
+ const stale = [...registered].filter((name) => !MCP_TOOL_POPULATION.has(name));
+ expect(unclassified.toSorted()).toEqual([]);
+ expect(stale.toSorted()).toEqual([]);
+ });
+
+ test("the swept population is measured, as an equality", () => {
+ // A moved number is a finding to name in the report, not a
+ // re-measurement chore - see write-site-census's own convention.
+ expect(MCP_TOOL_POPULATION.size).toBe(MCP_TOOL_POPULATION_SIZE);
+ });
+
+ test("brain_search and brain_file_context are the only two covered MCP tools", () => {
+ const covered = REGISTRY_BY_KIND(VISIBILITY_SURFACE_KIND.mcpTool)
+ .filter((e) => e.category === VISIBILITY_SURFACE_CATEGORY.covered)
+ .map((e) => e.surface)
+ .toSorted();
+ expect(covered).toEqual(["brain_file_context", "brain_search"]);
+ });
+ });
+
+ describe("MCP resources", () => {
+ test("every advertised resource and template has a registry entry, exactly", () => {
+ const advertised = new Set<string>([
+ ...listResources().map((r) => r.uri),
+ ...listResourceTemplates().map((t) => t.uriTemplate),
+ ]);
+ const registered = new Set(
+ REGISTRY_BY_KIND(VISIBILITY_SURFACE_KIND.mcpResource).map((e) => e.surface),
+ );
+ expect([...advertised].toSorted()).toEqual([...registered].toSorted());
+ expect(advertised.size).toBe(MCP_RESOURCE_POPULATION_SIZE);
+ });
+
+ test("no MCP resource is covered - resources.ts never calls applyVisibilityScope", () => {
+ const anyCovered = REGISTRY_BY_KIND(VISIBILITY_SURFACE_KIND.mcpResource).some(
+ (e) => e.category === VISIBILITY_SURFACE_CATEGORY.covered,
+ );
+ expect(anyCovered).toBe(false);
+ });
+ });
+
+ describe("CLI verbs", () => {
+ test("every registered CLI-verb entry names a <group> <verb> pair that exists", () => {
+ const missing: string[] = [];
+ for (const entry of REGISTRY_BY_KIND(VISIBILITY_SURFACE_KIND.cliVerb)) {
+ const [group, verb] = entry.surface.split(" ");
+ if (group === undefined || verb === undefined || nestedCommand(group, verb) === undefined) {
+ missing.push(entry.surface);
+ }
+ }
+ expect(missing).toEqual([]);
+ });
+
+ test("the hand-enumerated CLI population is measured, as an equality", () => {
+ expect(REGISTRY_BY_KIND(VISIBILITY_SURFACE_KIND.cliVerb).length).toBe(
+ CLI_VERB_POPULATION_SIZE,
+ );
+ });
+
+ test("no CLI verb is covered except the two search lanes", () => {
+ const covered = REGISTRY_BY_KIND(VISIBILITY_SURFACE_KIND.cliVerb)
+ .filter((e) => e.category === VISIBILITY_SURFACE_CATEGORY.covered)
+ .map((e) => e.surface)
+ .toSorted();
+ expect(covered).toEqual(["search query"]);
+ });
+ });
+
+ test("the index store's own chunk tables carry exactly one entry", () => {
+ const rows = REGISTRY_BY_KIND(VISIBILITY_SURFACE_KIND.indexStore);
+ expect(rows.length).toBe(1);
+ expect(rows[0]!.category).toBe(VISIBILITY_SURFACE_CATEGORY.excluded);
+ });
+
+ test("applyVisibilityScope has exactly one call site in src/core/search/", () => {
+ // The fact the whole registry rests on: search()'s pool-filters
+ // pipeline is the ONE place the field is consulted. A second call
+ // site would mean a covered lane this file does not know about.
+ const hits: string[] = [];
+ for (const file of readTree(SRC_ROOT)) {
+ if (!file.path.startsWith("src/core/search/")) continue;
+ if (/\bapplyVisibilityScope\s*\(/.test(lexedViews(file).code)) hits.push(file.path);
+ }
+ // The declaration site itself (`export function applyVisibilityScope`)
+ // is a definition, not a call, and the lexer's `code` view still
+ // matches its parameter list open-paren - so the DEFINING file is
+ // expected to appear once alongside its one real caller.
+ expect(hits.toSorted()).toEqual([
+ "src/core/search/pipeline/pool-filters.ts",
+ "src/core/search/result-filters.ts",
+ ]);
+ });
+});
+
+describe("the census can fail", () => {
+ test("a synthetic MCP tool file with a known primitive and a real tool name is reported unclassified", () => {
+ const syntheticRealNames = new Set(["brain_synthetic_intruder"]);
+ const intruder: CensusFile = {
+ path: "src/mcp/brain/synthetic-tools.ts",
+ text:
+ 'import { parseFrontmatter } from "../../core/vault.ts";\n' +
+ "export const SYNTHETIC_TOOLS = [\n" +
+ ' { name: "brain_synthetic_intruder", description: "x", handler: () => parseFrontmatter },\n' +
+ "];\n",
+ };
+ const population = discoverMcpToolPopulation([intruder], syntheticRealNames);
+ expect([...population]).toEqual(["brain_synthetic_intruder"]);
+ // And the registry, as it stands, does not carry this synthetic name -
+ // exactly the state the real test above fails the build on.
+ const registered = new Set(
+ REGISTRY_BY_KIND(VISIBILITY_SURFACE_KIND.mcpTool).map((e) => e.surface),
+ );
+ expect(registered.has("brain_synthetic_intruder")).toBe(false);
+ });
+
+ test("a file with no note-content import is not swept in, even with a matching tool-name literal", () => {
+ const syntheticRealNames = new Set(["brain_synthetic_bystander"]);
+ const bystander: CensusFile = {
+ path: "src/mcp/brain/synthetic-bystander.ts",
+ text: 'export const X = [{ name: "brain_synthetic_bystander", description: "x" }];\n',
+ };
+ const population = discoverMcpToolPopulation([bystander], syntheticRealNames);
+ expect([...population]).toEqual([]);
+ });
+
+ test("a tool-name literal that is not a REAL registered tool is not swept in (noise filter)", () => {
+ const noisy: CensusFile = {
+ path: "src/mcp/brain/synthetic-noise.ts",
+ text:
+ 'import { listVaultPages } from "../../core/vault.ts";\n' +
+ 'const unrelatedObject = { name: "not_a_real_tool" };\n',
+ };
+ const population = discoverMcpToolPopulation([noisy], new Set(["brain_search"]));
+ expect([...population]).toEqual([]);
+ });
+
+ test("a tool registered under a constant declared in the same file is swept in", () => {
+ const named: CensusFile = {
+ path: "src/mcp/brain/synthetic-const-local.ts",
+ text:
+ 'import { listVaultPages } from "../../core/vault.ts";\n' +
+ 'const MY_TOOL = "brain_synthetic_local_const";\n' +
+ "export const T = [{ name: MY_TOOL, handler: listVaultPages }];\n",
+ };
+ const population = discoverMcpToolPopulation([named], new Set(["brain_synthetic_local_const"]));
+ expect([...population]).toEqual(["brain_synthetic_local_const"]);
+ });
+
+ test("a tool registered under a constant IMPORTED from another swept file is swept in", () => {
+ // The real shape this blind spot had: `src/mcp/tools.ts` registers
+ // `second_brain_capabilities` as `name: CAPABILITY_DIAGNOSTIC_TOOL`,
+ // declared in `src/mcp/capabilities.ts`.
+ const declaring: CensusFile = {
+ path: "src/mcp/synthetic-names.ts",
+ text: 'export const REMOTE_TOOL = "brain_synthetic_remote_const";\n',
+ };
+ const registering: CensusFile = {
+ path: "src/mcp/synthetic-registry.ts",
+ text:
+ 'import { listVaultPages } from "../core/vault.ts";\n' +
+ 'import { REMOTE_TOOL } from "./synthetic-names.ts";\n' +
+ "export const T = [{ name: REMOTE_TOOL, handler: listVaultPages }];\n",
+ };
+ const population = discoverMcpToolPopulation(
+ [declaring, registering],
+ new Set(["brain_synthetic_remote_const"]),
+ );
+ expect([...population]).toEqual(["brain_synthetic_remote_const"]);
+ });
+
+ test("a constant the file never imports and never declares resolves to nothing", () => {
+ // Over-reach is the failure the resolution has to avoid: an unresolved
+ // identifier contributes no name rather than a guessed one.
+ const dangling: CensusFile = {
+ path: "src/mcp/synthetic-dangling.ts",
+ text:
+ 'import { listVaultPages } from "../core/vault.ts";\n' +
+ "export const T = [{ name: UNRESOLVED_TOOL, handler: listVaultPages }];\n",
+ };
+ const population = discoverMcpToolPopulation([dangling], new Set(["brain_search"]));
+ expect([...population]).toEqual([]);
+ });
+
+ test("a renamed import binding is still detected (the same shape write-site-census pins)", () => {
+ const renamed: CensusFile = {
+ path: "src/mcp/brain/synthetic-renamed.ts",
+ text:
+ 'import { parseFrontmatter as pf } from "../../core/vault.ts";\n' +
+ 'export const T = [{ name: "brain_synthetic_renamed", handler: pf }];\n',
+ };
+ // The vocabulary check reads the IMPORTED name (before `as`), not the
+ // local alias `pf` the file goes on to use - so a rename cannot hide
+ // the import the way it could not hide a write-site-census call.
+ const population = discoverMcpToolPopulation([renamed], new Set(["brain_synthetic_renamed"]));
+ expect([...population]).toEqual(["brain_synthetic_renamed"]);
+ });
+});
diff --git a/tests/core/architecture/write-site-census.test.ts b/tests/core/architecture/write-site-census.test.ts
index ea9c593f..843e1a66 100644
--- a/tests/core/architecture/write-site-census.test.ts
+++ b/tests/core/architecture/write-site-census.test.ts
@@ -597,14 +597,15 @@ const DIRECT_WRITE_EXCLUSIONS: Readonly<Record<string, WriteExclusion>> = Object
"clears the git ingest `state.json` on a rescan that found no head, so the next " +
"run starts as an initial walk rather than inheriting a stale marker.",
},
- "src/core/brain/ingest/checkpoint.ts": {
+ "src/core/brain/checkpoint-store.ts": {
categories: [C.retentionDelete],
calls: ["rmSync"],
reason:
- "clears one ingest plan's checkpoint behind `assertVaultIdentityForWrite`, reporting false " +
- "rather than throwing when there was nothing to clear. Absence is the terminal state for a " +
- "resume point: a plan with no checkpoint starts from the beginning, which is the same thing " +
- "a checkpoint rewritten to empty would have to mean.",
+ "clears one checkpoint - an ingest plan's or a session import's - behind " +
+ "`assertVaultIdentityForWrite`, reporting false rather than throwing when there was nothing " +
+ "to clear. Absence is the terminal state for a resume point: a lane with no checkpoint " +
+ "starts from the beginning, which is the same thing a checkpoint rewritten to empty would " +
+ "have to mean.",
},
"src/core/brain/ingest/sources-registry.ts": {
categories: [C.retentionDelete],
@@ -1099,8 +1100,87 @@ const DIRECT_WRITE_ROWS = 68;
* 98 -> 99: `src/core/brain/design-note.ts` commits its note through
* `writeFrontmatterAtomic`, exclusively, so it needs no exclusion either -
* it arrives in the class the record wants modules to arrive in.
+ *
+ * 99 -> 100: `src/core/brain/dead-letter.ts` writes one record per failed
+ * multi-artifact commit through `atomicWriteFileSync`, and removes
+ * nothing - deleting the only evidence of a partial write is exactly what
+ * that module declines to do - so it too arrives with no exclusion owed.
+ */
+const SHARED_HELPER_ROWS = 100;
+
+// ----- Origin-channel coverage boundary (Unit C) ----------------------------
+
+/**
+ * The origin-channel stamp reaches four writers, and this census is where
+ * the OTHER ones are counted rather than described.
+ *
+ * Unit C derives a channel (`mcp-tool` / `cli` / `import`) at the process
+ * entry point and stamps it on four record families. The population is
+ * bounded on purpose - a stamp is only worth having on records that
+ * already carry structured metadata - so the honest statement of what
+ * ships is "four writers carry it and N measured write sites do not",
+ * and N has to come from the same walk that produces every other number
+ * in this file. A prose claim of uniformity would be the exact
+ * dishonesty the wave removes.
+ *
+ * Against the write-binding taxonomy (`src/core/write-binding/index.ts`)
+ * the four are one CALLER-NAMED envelope (the note target resolver), one
+ * SLUG-DERIVED writer (signals), and two FULLY-DERIVED writers (the log
+ * pair and the continuity ledger). The rest of the fully-derived class -
+ * dream workruns, decision receipts, capture decisions, every telemetry
+ * ledger - is inside the uncovered counts below and stays there.
+ */
+const ORIGIN_CHANNEL_STAMPED: ReadonlyArray<string> = Object.freeze([
+ "src/core/brain/continuity/store.ts",
+ "src/core/brain/log.ts",
+ "src/core/brain/notes/create-note.ts",
+ "src/core/brain/signal.ts",
+]);
+
+/** Import specifier of the resolver, in either quote style. */
+const ORIGIN_CHANNEL_IMPORT_RE = /from\s*["'][^"']*\/origin-channel\.ts["']/;
+
+/**
+ * The record-writer read. Keyed on the READ and not on the import,
+ * because the entry points that CLAIM the channel import the same module
+ * and write no record - counting them would inflate the covered class
+ * with modules that stamp nothing.
+ */
+const ORIGIN_CHANNEL_READ_RE = /\boriginChannelStamp\s*\(/;
+
+function stampsOriginChannel(file: CensusFile): boolean {
+ const views = lexedViews(file);
+ return (
+ ORIGIN_CHANNEL_IMPORT_RE.test(views.withoutComments) && ORIGIN_CHANNEL_READ_RE.test(views.code)
+ );
+}
+
+const STAMPED_PATHS: ReadonlySet<string> = new Set(
+ SOURCE_TREE.filter(stampsOriginChannel).map((file) => file.path),
+);
+
+/**
+ * Direct-`fs` write sites the stamp does not reach - MEASURED off the
+ * sweep, not derived from {@link DIRECT_WRITE_ROWS}.
+ *
+ * It used to be written as `DIRECT_WRITE_ROWS - 1` and asserted against
+ * `DIRECT_ROWS.length - stampedDirect.length`, which is the same
+ * arithmetic on both sides of an `expect`: the assertion restated pins
+ * the file had already made and could not fail on its own. The number
+ * below is what the sweep actually reports, so the assertion measures
+ * the swept tree; the one excused site that IS stamped is the continuity
+ * ledger, which appends its own record shape and carries the channel on
+ * the record rather than in frontmatter.
*/
-const SHARED_HELPER_ROWS = 99;
+const UNSTAMPED_DIRECT_ROWS = 67;
+
+/**
+ * Shared-helper write sites the stamp does not reach, measured the same
+ * way. The three stamped writers that route through the shared helpers -
+ * the log pair through `atomicWriteFileSync`, signals and notes through
+ * `writeFrontmatterAtomic` - are the ones missing from this count.
+ */
+const UNSTAMPED_SHARED_ROWS = 97;
describe("in-vault write-site census", () => {
test("every direct-fs write site carries a written exclusion", () => {
@@ -1165,6 +1245,70 @@ describe("in-vault write-site census", () => {
});
});
+describe("origin-channel coverage boundary", () => {
+ test("exactly the four declared writers stamp the channel", () => {
+ // Named, not counted, in both directions: a new stamper and a writer
+ // that stopped stamping are different findings and read differently.
+ expect([...STAMPED_PATHS].toSorted()).toEqual([...ORIGIN_CHANNEL_STAMPED]);
+ });
+
+ test("the direct-fs exclusion record is uncovered but for the continuity ledger", () => {
+ const stampedDirect = DIRECT_ROWS.filter((row) => STAMPED_PATHS.has(row.path)).map(
+ (row) => row.path,
+ );
+ expect(stampedDirect.toSorted()).toEqual(["src/core/brain/continuity/store.ts"]);
+ // The measured remainder, counted off the sweep rather than
+ // subtracted from a pin. This is the honest sizing of the unit: the
+ // stamp lands on four writers, and this many swept direct-`fs`
+ // sites - append-only ledgers, lifecycle moves, retention deletes,
+ // machine artifacts - put bytes in the vault without one.
+ const unstampedDirect = DIRECT_ROWS.filter((row) => !STAMPED_PATHS.has(row.path));
+ expect(unstampedDirect.length).toBe(UNSTAMPED_DIRECT_ROWS);
+ });
+
+ test("the shared-helper class is uncovered but for the log, signal and note writers", () => {
+ const sharedRows = ROWS.filter((row) => row.sharedCalls > 0);
+ const stampedShared = sharedRows
+ .filter((row) => STAMPED_PATHS.has(row.path))
+ .map((row) => row.path);
+ expect(stampedShared.toSorted()).toEqual([
+ "src/core/brain/log.ts",
+ "src/core/brain/notes/create-note.ts",
+ "src/core/brain/signal.ts",
+ ]);
+ const unstampedShared = sharedRows.filter((row) => !STAMPED_PATHS.has(row.path));
+ expect(unstampedShared.length).toBe(UNSTAMPED_SHARED_ROWS);
+ });
+
+ test("a new stamper anywhere in the tree is reported, not absorbed", () => {
+ // The detector's own failure mode: a module that reads the stamp and
+ // is not in the declared four has widened the covered class, and the
+ // boundary numbers above are stale the moment it lands.
+ const intruder: CensusFile = {
+ path: "src/core/brain/synthetic-stamper.ts",
+ text:
+ 'import { originChannelStamp } from "../origin-channel.ts";\n' +
+ "export const channel = originChannelStamp();\n",
+ };
+ expect(stampsOriginChannel(intruder)).toBe(true);
+ expect(STAMPED_PATHS.has(intruder.path)).toBe(false);
+ });
+
+ test("claiming the channel is not stamping it", () => {
+ // The entry points import the same module to CLAIM the channel and
+ // write no record. Keying the detector on the import alone would put
+ // them in the covered class and overstate the coverage by three.
+ expect(
+ stampsOriginChannel({
+ path: "src/cli/synthetic-entry-point.ts",
+ text:
+ 'import { ORIGIN_CHANNEL, setOriginChannel } from "../core/origin-channel.ts";\n' +
+ "setOriginChannel(ORIGIN_CHANNEL.cli);\n",
+ }),
+ ).toBe(false);
+ });
+});
+
describe("the census can fail", () => {
/** Run the real census over the real tree plus one synthetic module. */
function unlistedWith(intruder: CensusFile): string[] {
diff --git a/tests/core/brain.log.test.ts b/tests/core/brain.log.test.ts
index 95bcf097..7ccd303b 100644
--- a/tests/core/brain.log.test.ts
+++ b/tests/core/brain.log.test.ts
@@ -247,10 +247,21 @@ describe("appendLogEvent — JSONL sidecar (§23, v0.10.8)", () => {
const lines = jsonl.trim().split("\n");
expect(lines).toHaveLength(1);
const parsed = JSON.parse(lines[0]!);
+ // `origin_channel` is the server-derived channel of the writing
+ // process (Unit C of the nothing-writes-silently wave), folded into
+ // the body the same way `agent` is so the markdown block and this row
+ // stay one-to-one. `unset` because a test harness is not one of the
+ // entry points that claims a channel - the explicit literal for that,
+ // never a guessed `cli`.
expect(parsed).toEqual({
ts: "2026-05-19T10:00:00Z",
kind: "feedback",
- payload: { signal: "[[sig-x]]", topic: "x", sign: "positive" },
+ payload: {
+ signal: "[[sig-x]]",
+ topic: "x",
+ sign: "positive",
+ origin_channel: "unset",
+ },
});
});
diff --git a/tests/core/brain/continuity/record-envelope.test.ts b/tests/core/brain/continuity/record-envelope.test.ts
index 23d548e9..411332be 100644
--- a/tests/core/brain/continuity/record-envelope.test.ts
+++ b/tests/core/brain/continuity/record-envelope.test.ts
@@ -10,11 +10,19 @@
*
* Two properties carry it:
*
- * - the envelope is EIGHT fields in one order. `JSON.stringify` emits
+ * - the envelope is NINE fields in one order. `JSON.stringify` emits
* insertion order, so a field added anywhere - including at the end -
* changes the bytes on disk and fails here. That is the point: a new
* envelope field is a schema change, and this test is where it has to
- * be argued for rather than slipped in.
+ * be argued for rather than slipped in. It was eight, and the ninth
+ * was argued: `originChannel` is the server-derived origin channel
+ * (Unit C of the nothing-writes-silently wave), stamped beside `id`
+ * because it is part of the record's header rather than its content.
+ * It reads `unset` in this fixture because a test harness is not one
+ * of the entry points that claims a channel, and `unset` is the
+ * explicit literal for that rather than a guessed `cli`. Per the
+ * evolution rule on `CONTINUITY_SCHEMA_VERSION` an additive optional
+ * field does not bump the version, so `o2b.continuity.v1` stands.
* - the dedup id is sha-256 over `kind`, `createdAt`, `sourceRefs` and
* `payload`, and NOTHING else. `docs/observability.md` documents that
* formula so a reader can reproduce an id; the second assertion below
@@ -68,7 +76,7 @@ const INPUT: AppendContinuityRecordInput = Object.freeze({
*/
const GOLDEN_LINE =
'{"schema":"o2b.continuity.v1","id":"ctn_20260614093000_dfa03f037ab006d8",' +
- '"kind":"recall_telemetry","createdAt":"2026-06-14T09:30:00Z",' +
+ '"originChannel":"unset","kind":"recall_telemetry","createdAt":"2026-06-14T09:30:00Z",' +
'"sourceRefs":[{"id":"note-a","path":"Brain/notes/a.md"}],' +
'"payload":{"status":"ok","session_id":"sess-1"},"private":false,"redacted":false}';
@@ -79,6 +87,7 @@ const GOLDEN_RECORD = JSON.parse(GOLDEN_LINE) as ContinuityRecord;
const GOLDEN_KEYS: ReadonlyArray<string> = Object.freeze([
"schema",
"id",
+ "originChannel",
"kind",
"createdAt",
"sourceRefs",
@@ -108,7 +117,10 @@ describe("continuity record envelope", () => {
// + payload, id-stamped with the digits of createdAt. Recomputed from
// the documented inputs alone - if anything else enters the hash, an
// operator reproducing an id from the documentation gets a miss and
- // this assertion is what tells us before they do.
+ // this assertion is what tells us before they do. `schema` and
+ // `originChannel` are both outside it, so neither the version stamp
+ // nor the writing channel can re-identify an otherwise identical
+ // record.
const digest = createHash("sha256")
.update(
JSON.stringify({
diff --git a/tests/core/brain/dead-letter.test.ts b/tests/core/brain/dead-letter.test.ts
new file mode 100644
index 00000000..b10f66bf
--- /dev/null
+++ b/tests/core/brain/dead-letter.test.ts
@@ -0,0 +1,269 @@
+/**
+ * The durable multi-artifact write dead letter (nothing-writes-silently,
+ * unit E).
+ *
+ * Claims pinned here:
+ *
+ * 1. A record round-trips: what `recordDeadLetter` wrote is what
+ * `readDeadLetter` and `listDeadLetters` return, in the shared
+ * reconciliation vocabulary (`attempted` / `found` / `missing`, the
+ * missing keys NAMED).
+ * 2. It lands under `<vault>/.open-second-brain/`, the machine-artifact
+ * root the ingest checkpoint and content manifest already use, at a
+ * path the module resolves and the state catalogue can name.
+ * 3. Two records of the same failure collapse onto one file, and two
+ * different failures do not.
+ * 4. An unknown `schema_version` and a corrupt file are HARD errors on
+ * read - never a silent skip, which would turn "the record of a
+ * partial write is unreadable" into "there was no partial write".
+ * 5. A report with nothing missing is refused: a dead letter that names
+ * no unwritten item is not a record of anything.
+ * 6. `tryRecordDeadLetter` never throws - it reports why it could not
+ * write - because it runs inside a catch block whose original error
+ * must survive.
+ */
+
+import { describe, expect, test } from "bun:test";
+import { existsSync, mkdtempSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { basename, dirname, join } from "node:path";
+
+import {
+ DEAD_LETTER_SCHEMA_VERSION,
+ DeadLetterError,
+ deadLetterDir,
+ deadLetterPath,
+ listDeadLetters,
+ readDeadLetter,
+ recordDeadLetter,
+ tryRecordDeadLetter,
+ type DeadLetterEnvelopeIdentity,
+} from "../../../src/core/brain/dead-letter.ts";
+import { DERIVED_STORE_DIR } from "../../../src/core/brain/path-constants.ts";
+import { buildReconciliationReport } from "../../../src/core/reconciliation-report.ts";
+
+const NOW = new Date("2026-08-23T10:15:00Z");
+
+const ENVELOPE: DeadLetterEnvelopeIdentity = Object.freeze({
+ lane: "extract-signals",
+ step: "extract-signals",
+ reference: "sess-2026-08-23",
+ target: "Brain/inbox",
+});
+
+function vault(): string {
+ return mkdtempSync(join(tmpdir(), "o2b-dead-letter-"));
+}
+
+function report(missing: ReadonlyArray<string>, found = 1) {
+ return buildReconciliationReport({ attempted: found + missing.length, found, missing });
+}
+
+describe("a dead letter round-trips", () => {
+ test("it names the unwritten items, the lane, the envelope and the first error", () => {
+ const v = vault();
+ const record = recordDeadLetter(v, {
+ envelope: ENVELOPE,
+ report: report(["items[1]:module-names", "items[2]:no-abbrev"]),
+ firstError: new Error("EACCES: permission denied"),
+ now: NOW,
+ });
+
+ expect(record.schema_version).toBe(DEAD_LETTER_SCHEMA_VERSION);
+ expect(record.lane).toBe("extract-signals");
+ expect(record.step).toBe("extract-signals");
+ expect(record.reference).toBe("sess-2026-08-23");
+ expect(record.target).toBe("Brain/inbox");
+ expect(record.attempted).toBe(3);
+ expect(record.found).toBe(1);
+ expect(record.missing).toEqual(["items[1]:module-names", "items[2]:no-abbrev"]);
+ expect(record.outcome).toBe("partial");
+ expect(record.first_error).toContain("EACCES");
+ expect(record.recorded_at).toBe("2026-08-23T10:15:00Z");
+
+ const back = readDeadLetter(v, record.id);
+ expect(back).toEqual(record);
+ expect(listDeadLetters(v)).toEqual([record]);
+ });
+
+ test("it lands in the machine-artifact root, at the path the module resolves", () => {
+ const v = vault();
+ const record = recordDeadLetter(v, {
+ envelope: ENVELOPE,
+ report: report(["items[0]:only"]),
+ firstError: "disk full",
+ now: NOW,
+ });
+ const path = deadLetterPath(v, record.id);
+ expect(existsSync(path)).toBe(true);
+ expect(dirname(path)).toBe(deadLetterDir(v));
+ expect(deadLetterDir(v).startsWith(join(v, DERIVED_STORE_DIR))).toBe(true);
+ expect(basename(path)).toBe(`${record.id}.json`);
+ });
+
+ test("the same failure collapses onto one file; a different one does not", () => {
+ const v = vault();
+ const input = {
+ envelope: ENVELOPE,
+ report: report(["items[1]:module-names"]),
+ firstError: "disk full",
+ now: NOW,
+ };
+ const first = recordDeadLetter(v, input);
+ const again = recordDeadLetter(v, input);
+ expect(again.id).toBe(first.id);
+ expect(listDeadLetters(v)).toHaveLength(1);
+
+ const other = recordDeadLetter(v, {
+ ...input,
+ report: report(["items[2]:something-else"]),
+ });
+ expect(other.id).not.toBe(first.id);
+ expect(listDeadLetters(v)).toHaveLength(2);
+ });
+
+ test("a retry minutes later still collapses onto the same file", () => {
+ // The id is derived from the record's CONTENT and nothing else. When
+ // the clock was part of it, a deterministically-failing payload
+ // retried twenty times left twenty byte-identical files in a
+ // directory nothing prunes - and an operator could not tell twenty
+ // partial writes from one repeated one.
+ const v = vault();
+ const input = {
+ envelope: ENVELOPE,
+ report: report(["items[1]:module-names"]),
+ firstError: "disk full",
+ now: NOW,
+ };
+ const first = recordDeadLetter(v, input);
+ const later = recordDeadLetter(v, { ...input, now: new Date("2026-08-23T10:26:00Z") });
+ expect(later.id).toBe(first.id);
+ expect(listDeadLetters(v)).toHaveLength(1);
+ // The one file names the most recent attempt.
+ expect(listDeadLetters(v)[0]!.recorded_at).toBe("2026-08-23T10:26:00Z");
+ });
+
+ test("the listing is chronological across lanes, not lexical by id", () => {
+ const v = vault();
+ const older = recordDeadLetter(v, {
+ envelope: ENVELOPE,
+ report: report(["items[0]:zulu"]),
+ firstError: "disk full",
+ now: NOW,
+ });
+ const newer = recordDeadLetter(v, {
+ envelope: ENVELOPE,
+ report: report(["items[0]:alpha"]),
+ firstError: "disk full",
+ now: new Date("2026-08-23T11:00:00Z"),
+ });
+ expect(listDeadLetters(v).map((r) => r.id)).toEqual([older.id, newer.id]);
+ });
+
+ test("an empty vault lists nothing rather than failing", () => {
+ expect(listDeadLetters(vault())).toEqual([]);
+ });
+});
+
+describe("an unreadable record is an error, never a skip", () => {
+ test("an unknown schema_version is refused by name", () => {
+ const v = vault();
+ const record = recordDeadLetter(v, {
+ envelope: ENVELOPE,
+ report: report(["items[1]:x"]),
+ firstError: "boom",
+ now: NOW,
+ });
+ const path = deadLetterPath(v, record.id);
+ writeFileSync(path, JSON.stringify({ ...record, schema_version: 99 }));
+ expect(() => readDeadLetter(v, record.id)).toThrow(DeadLetterError);
+ expect(() => readDeadLetter(v, record.id)).toThrow(/schema_version/);
+ expect(() => listDeadLetters(v)).toThrow(/schema_version/);
+ });
+
+ test("a corrupt file is refused by name", () => {
+ const v = vault();
+ const record = recordDeadLetter(v, {
+ envelope: ENVELOPE,
+ report: report(["items[1]:x"]),
+ firstError: "boom",
+ now: NOW,
+ });
+ writeFileSync(deadLetterPath(v, record.id), "{ not json");
+ expect(() => readDeadLetter(v, record.id)).toThrow(/corrupt/);
+ });
+
+ test("a record missing a field the vocabulary needs is refused by name", () => {
+ const v = vault();
+ const record = recordDeadLetter(v, {
+ envelope: ENVELOPE,
+ report: report(["items[1]:x"]),
+ firstError: "boom",
+ now: NOW,
+ });
+ const { missing: _dropped, ...withoutMissing } = record;
+ writeFileSync(deadLetterPath(v, record.id), JSON.stringify(withoutMissing));
+ expect(() => readDeadLetter(v, record.id)).toThrow(/missing/);
+ });
+
+ test("a record that was never written reads as null", () => {
+ expect(readDeadLetter(vault(), "extract-signals-0011aabb2233")).toBeNull();
+ });
+});
+
+describe("what a dead letter refuses to be", () => {
+ test("a report with nothing missing is not a dead letter", () => {
+ expect(() =>
+ recordDeadLetter(vault(), {
+ envelope: ENVELOPE,
+ report: buildReconciliationReport({ attempted: 2, found: 2, missing: [] }),
+ firstError: "boom",
+ now: NOW,
+ }),
+ ).toThrow(DeadLetterError);
+ });
+
+ test("an envelope identity with a blank field is refused by name", () => {
+ expect(() =>
+ recordDeadLetter(vault(), {
+ envelope: { ...ENVELOPE, reference: " " },
+ report: report(["items[1]:x"]),
+ firstError: "boom",
+ now: NOW,
+ }),
+ ).toThrow(/reference/);
+ });
+});
+
+describe("the recorder used inside a catch block cannot throw", () => {
+ test("a write it cannot perform is reported, not raised", () => {
+ // A vault path that is a FILE, not a directory: the write has nowhere
+ // to land, and the caller is mid-catch holding the real error.
+ const parent = mkdtempSync(join(tmpdir(), "o2b-dead-letter-"));
+ const notAVault = join(parent, "file-not-a-dir");
+ writeFileSync(notAVault, "");
+ const outcome = tryRecordDeadLetter(notAVault, {
+ envelope: ENVELOPE,
+ report: report(["items[1]:x"]),
+ firstError: "boom",
+ now: NOW,
+ });
+ expect(outcome.recorded).toBe(false);
+ if (!outcome.recorded) expect(outcome.reason.length).toBeGreaterThan(0);
+ });
+
+ test("a write it can perform reports where the record landed", () => {
+ const v = vault();
+ const outcome = tryRecordDeadLetter(v, {
+ envelope: ENVELOPE,
+ report: report(["items[1]:x"]),
+ firstError: "boom",
+ now: NOW,
+ });
+ expect(outcome.recorded).toBe(true);
+ if (outcome.recorded) {
+ expect(existsSync(outcome.path)).toBe(true);
+ expect(readDeadLetter(v, outcome.id)).not.toBeNull();
+ }
+ });
+});
diff --git a/tests/core/brain/design-note.test.ts b/tests/core/brain/design-note.test.ts
index d958bd2c..e3ab8679 100644
--- a/tests/core/brain/design-note.test.ts
+++ b/tests/core/brain/design-note.test.ts
@@ -88,7 +88,7 @@ test("an empty vault grounds the note in a NAMED emptiness, not a failure", () =
[DESIGN_NOTE_STORE.decisions, DESIGN_NOTE_STORE.tensions, DESIGN_NOTE_STORE.truth].toSorted(),
);
// The report is still produced, with its envelope.
- const report = planDesignNote(vault, TOPIC, { now: NOW });
+ const report = planDesignNote(vault, TOPIC, { now: NOW, ownerScope: null });
expect(report.llmStep.status).toBe(NEEDS_LLM_STEP);
expect(report.grounding.emptyStores.length).toBe(3);
});
@@ -143,7 +143,7 @@ test("a populated store that matches nothing grounds nothing but is not reported
});
test("the report carries exactly one envelope, naming the note it will become", () => {
- const report = planDesignNote(vault, TOPIC, { now: NOW });
+ const report = planDesignNote(vault, TOPIC, { now: NOW, ownerScope: null });
expect(report.llmStep.step).toBe("design-note");
expect(report.llmStep.target_path).toBe(report.targetPath);
expect(report.targetPath).toBe("Brain/decisions/design-2026-08-22-vector-index-storage.md");
diff --git a/tests/core/brain/diarization.test.ts b/tests/core/brain/diarization.test.ts
index 3911b4b1..9c519767 100644
--- a/tests/core/brain/diarization.test.ts
+++ b/tests/core/brain/diarization.test.ts
@@ -88,7 +88,7 @@ afterEach(() => {
});
test("stated claims corroborated by evidence carry frequency, recency, and identity", () => {
- const report = diarize(vault, { query: "Ada Lovelace" }, { now: NOW });
+ const report = diarize(vault, { query: "Ada Lovelace" }, { now: NOW, ownerScope: null });
expect(report.entityId).toContain("ent-person");
expect(report.entityName).toBe("Ada Lovelace");
@@ -105,7 +105,7 @@ test("stated claims corroborated by evidence carry frequency, recency, and ident
});
test("stated but unevidenced claims report a zero-frequency gap", () => {
- const report = diarize(vault, { query: "Grace Hopper" }, { now: NOW });
+ const report = diarize(vault, { query: "Grace Hopper" }, { now: NOW, ownerScope: null });
const unevidenced = report.statedVsEvidenced.filter((l) => l.kind === "stated_unevidenced");
expect(unevidenced.length).toBeGreaterThanOrEqual(1);
expect(unevidenced[0]!.evidenceFrequency).toBe(0);
@@ -113,7 +113,7 @@ test("stated but unevidenced claims report a zero-frequency gap", () => {
});
test("evidenced but unstated subjects surface the source pages as the gap", () => {
- const report = diarize(vault, { query: "Alan Turing" }, { now: NOW });
+ const report = diarize(vault, { query: "Alan Turing" }, { now: NOW, ownerScope: null });
const unstated = report.statedVsEvidenced.filter((l) => l.kind === "evidenced_unstated");
expect(unstated.length).toBeGreaterThanOrEqual(1);
expect(unstated[0]!.evidence.kind).toBe("source_page");
@@ -121,7 +121,7 @@ test("evidenced but unstated subjects surface the source pages as the gap", () =
});
test("the report emits a profile skeleton and exactly one needs-llm-step envelope", () => {
- const report = diarize(vault, { query: "Ada Lovelace" }, { now: NOW });
+ const report = diarize(vault, { query: "Ada Lovelace" }, { now: NOW, ownerScope: null });
expect(report.skeleton).toContain("kind: brain-profile");
expect(report.skeleton).toContain("## Stated vs evidenced");
// The prose is deferred, never generated inline.
@@ -133,12 +133,14 @@ test("the report emits a profile skeleton and exactly one needs-llm-step envelop
});
test("the document set includes the entity plus every corroborating source", () => {
- const report = diarize(vault, { query: "Ada Lovelace" }, { now: NOW });
+ const report = diarize(vault, { query: "Ada Lovelace" }, { now: NOW, ownerScope: null });
const kinds = report.documentSet.map((d) => d.kind);
expect(kinds).toContain("entity");
expect(report.documentSet.filter((d) => d.kind === "source_page").length).toBe(2);
});
test("an unknown entity is a typed error", () => {
- expect(() => diarize(vault, { query: "Nobody At All" }, { now: NOW })).toThrow(DiarizationError);
+ expect(() => diarize(vault, { query: "Nobody At All" }, { now: NOW, ownerScope: null })).toThrow(
+ DiarizationError,
+ );
});
diff --git a/tests/core/brain/dream-topic-folding.test.ts b/tests/core/brain/dream-topic-folding.test.ts
index 56861799..32e153a0 100644
--- a/tests/core/brain/dream-topic-folding.test.ts
+++ b/tests/core/brain/dream-topic-folding.test.ts
@@ -356,6 +356,15 @@ describe("a corpus with no folding variants", () => {
// `readdirSync`, which yields entries in whatever order the filesystem
// stores them, and a golden over a larger corpus would be a flake, not a
// proof.
+ //
+ // They moved once since, and not by the fold: every log event now
+ // carries an `origin_channel` bullet, the server-derived channel of the
+ // writing process (Unit C of the nothing-writes-silently wave). It
+ // reads `unset` because a test harness is not one of the entry points
+ // that claims a channel. What these goldens still prove is unchanged —
+ // the fold authors no byte of a variant-free report — because the new
+ // bullet is on every event of both goldens and of every other log in
+ // the tree.
test("promotion writes a byte-identical dream report", () => {
const vault = newVault("golden-promote");
@@ -404,6 +413,7 @@ tags: [brain, brain/log]
- run_id: dream-2026-06-05-120000
- reason: dream
- size_bytes: <masked>
+- origin_channel: unset
## 12:00:00Z — dream
- run_id: dream-2026-06-05-120000
@@ -411,6 +421,7 @@ tags: [brain, brain/log]
- [[pref-deploy-cadence|Ship the fold with the rule it folds.]]
- moved_to_processed:
- sig-2026-06-01-only
+- origin_channel: unset
`;
const GOLDEN_REBUT = `---
@@ -425,6 +436,7 @@ tags: [brain, brain/log]
- run_id: dream-2026-06-05-120000
- reason: dream
- size_bytes: <masked>
+- origin_channel: unset
## 12:00:00Z — dream
- run_id: dream-2026-06-05-120000
@@ -434,4 +446,5 @@ tags: [brain, brain/log]
- [[ret-deploy-cadence|Rule of record for deploy-cadence.]] (rebutted)
- moved_to_processed:
- sig-2026-06-01-only
+- origin_channel: unset
`;
diff --git a/tests/core/brain/extract-signals.test.ts b/tests/core/brain/extract-signals.test.ts
index 1339eb4c..c8cbdc62 100644
--- a/tests/core/brain/extract-signals.test.ts
+++ b/tests/core/brain/extract-signals.test.ts
@@ -32,10 +32,12 @@ import {
AUTO_EXTRACT_CONFIDENCE_FLOOR,
AUTO_EXTRACT_PER_SESSION_CAP,
commitExtractedSignals,
+ EXTRACT_SIGNALS_STEP,
ExtractSignalsError,
ExtractSignalsWriteError,
planExtractSignals,
} from "../../../src/core/brain/extract-signals.ts";
+import { listDeadLetters } from "../../../src/core/brain/dead-letter.ts";
import { NEEDS_LLM_STEP } from "../../../src/core/brain/llm-step.ts";
import { ResponseCheckError } from "../../../src/core/brain/response-checks.ts";
import { ResponseShapeError } from "../../../src/core/brain/response-shape.ts";
@@ -299,3 +301,98 @@ test("the new source_type round-trips and an unknown member is still refused", (
expect(() => parseSignal(path)).toThrow(/auto_extract/);
expect(() => parseSignal(path)).toThrow(/sideways/);
});
+
+// ----- Write accounting and the durable dead letter (unit E) ---------------
+//
+// 12. Every commit reports its write accounting in the wave's shared
+// reconciliation vocabulary - attempted / found / missing, the missing
+// keys NAMED - on the success path and on the refusal alike.
+// 13. A commit that lands nothing does not read as a success: the accounting
+// says found 0, names every unwritten item, and carries the first real
+// error.
+// 14. A partial write leaves a DURABLE dead letter naming the unwritten
+// items, so a caller that drops the response is not the only record of
+// it - and the error says where that record landed.
+
+test("a completed commit reports its accounting in the shared vocabulary", () => {
+ const res = commitExtractedSignals(
+ vault,
+ SESSION,
+ { items: [item(), item({ topic: "no-abbrev", principle: "Never abbreviate a module name." })] },
+ { agent: "tester", now: NOW },
+ );
+ expect(res.reconciliation).toEqual({ attempted: 2, found: 2, missing: [] });
+ expect(listDeadLetters(vault)).toEqual([]);
+});
+
+test("a deduped or gate-rejected item is not counted as an attempted write", () => {
+ // Both are deliberate skips the result already names; folding them into
+ // `missing` would report a refusal the lane made as a loss it suffered.
+ const res = commitExtractedSignals(
+ vault,
+ SESSION,
+ { items: [item(), item({ topic: "noise", principle: "temporary scratch note" })] },
+ { agent: "tester", now: NOW, durabilityDenylist: [/scratch/] },
+ );
+ expect(res.reconciliation).toEqual({ attempted: 1, found: 1, missing: [] });
+ expect(res.durabilityRejected).toBe(1);
+});
+
+test("a mid-loop write failure reports attempted, written and missing plus the first error", () => {
+ const items = [item(), item({ topic: "module:names", principle: "Never abbreviate a module." })];
+ let caught: unknown;
+ try {
+ commitExtractedSignals(vault, SESSION, { items }, { agent: "tester", now: NOW });
+ } catch (err) {
+ caught = err;
+ }
+ const err = caught as ExtractSignalsWriteError;
+ expect(err).toBeInstanceOf(ExtractSignalsWriteError);
+ expect(err.reconciliation.attempted).toBe(2);
+ expect(err.reconciliation.found).toBe(1);
+ expect(err.reconciliation.missing).toEqual(["items[1]:module:names"]);
+ expect(err.firstError).toBeDefined();
+});
+
+test("a commit that lands nothing never reads as a success", () => {
+ // The FIRST item fails, so nothing at all reaches disk.
+ const items = [item({ topic: "module:names" }), item({ topic: "no-abbrev" })];
+ let caught: unknown;
+ try {
+ commitExtractedSignals(vault, SESSION, { items }, { agent: "tester", now: NOW });
+ } catch (err) {
+ caught = err;
+ }
+ const err = caught as ExtractSignalsWriteError;
+ expect(err).toBeInstanceOf(ExtractSignalsWriteError);
+ expect(err.reconciliation.found).toBe(0);
+ expect(err.reconciliation.attempted).toBe(2);
+ expect(err.reconciliation.missing).toEqual(["items[0]:module:names", "items[1]:no-abbrev"]);
+ expect(inboxFiles()).toEqual([]);
+});
+
+test("a partial write leaves a durable dead letter naming the unwritten items", () => {
+ const items = [item(), item({ topic: "module:names", principle: "Never abbreviate a module." })];
+ let caught: unknown;
+ try {
+ commitExtractedSignals(vault, SESSION, { items }, { agent: "tester", now: NOW });
+ } catch (err) {
+ caught = err;
+ }
+ const err = caught as ExtractSignalsWriteError;
+ const letters = listDeadLetters(vault);
+ expect(letters).toHaveLength(1);
+ const letter = letters[0]!;
+ expect(letter.lane).toBe("extract-signals");
+ expect(letter.step).toBe(EXTRACT_SIGNALS_STEP);
+ expect(letter.reference).toBe(SESSION);
+ expect(letter.attempted).toBe(2);
+ expect(letter.found).toBe(1);
+ expect(letter.missing).toEqual(["items[1]:module:names"]);
+ expect(letter.outcome).toBe("partial");
+ expect(letter.first_error.length).toBeGreaterThan(0);
+ // The response says where the durable record is, so a caller that keeps
+ // the response can go straight to it.
+ expect(err.deadLetter.recorded).toBe(true);
+ if (err.deadLetter.recorded) expect(err.message).toContain(err.deadLetter.id);
+});
diff --git a/tests/core/brain/import-census.test.ts b/tests/core/brain/import-census.test.ts
new file mode 100644
index 00000000..72fbe971
--- /dev/null
+++ b/tests/core/brain/import-census.test.ts
@@ -0,0 +1,238 @@
+/**
+ * Post-import read-back census (nothing-writes-silently, Unit F).
+ *
+ * The claims this suite pins:
+ *
+ * 1. A census NAMES the keys it could not read back. `attempted: 2,
+ * found: 1` with nothing else is the misleading no-op this unit
+ * removes, so the shared reconciliation vocabulary is the only
+ * constructor and a bare count cannot be built.
+ * 2. A clean import reports `complete`; one whose items no longer read
+ * back reports `partial` with the losses named.
+ * 3. The sessions lane reads back over the dedup hashes the import
+ * already keys its signals on - no second index, no second hash.
+ * 4. The ingest lane reads back over the content manifest, and a claimed
+ * path whose recorded hash no longer matches the bytes on disk is not
+ * counted as found. A recorded claim nobody can reproduce is exactly
+ * the gap the census exists to name.
+ * 5. A key claimed twice is one claim: the census cannot inflate
+ * `attempted` by repeating itself.
+ * 6. The serialized payload both lanes emit is one shape, so a CLI and an
+ * MCP reading of the same census cannot drift.
+ */
+
+import { afterEach, beforeEach, describe, expect, test } from "bun:test";
+import { mkdirSync, mkdtempSync, readdirSync, rmSync, unlinkSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { DEFAULT_BRAIN_CONFIG_YAML } from "../../../src/core/brain/config-template.ts";
+import { atomicWriteFileSync } from "../../../src/core/fs-atomic.ts";
+import { brainDirs } from "../../../src/core/brain/paths.ts";
+import { computeDedupHash } from "../../../src/core/brain/dedup-hash.ts";
+import { updateManifest } from "../../../src/core/brain/ingest/content-manifest.ts";
+import { importSession } from "../../../src/core/brain/sessions/import.ts";
+import {
+ buildReadBackCensus,
+ censusIngestedPaths,
+ censusSessionSignals,
+ serializeImportCensus,
+} from "../../../src/core/brain/import-census.ts";
+import { RECONCILIATION_OUTCOME } from "../../../src/core/reconciliation-report.ts";
+
+let vault: string;
+
+const NOW = new Date("2026-08-15T10:00:00Z");
+
+beforeEach(() => {
+ vault = mkdtempSync(join(tmpdir(), "o2b-import-census-"));
+ const dirs = brainDirs(vault);
+ for (const d of [dirs.brain, dirs.inbox, dirs.processed, dirs.preferences, dirs.log]) {
+ mkdirSync(d, { recursive: true });
+ }
+ atomicWriteFileSync(join(dirs.brain, "config.yaml"), DEFAULT_BRAIN_CONFIG_YAML);
+});
+
+afterEach(() => {
+ rmSync(vault, { recursive: true, force: true });
+});
+
+/** One Claude-format transcript carrying the given turn texts. */
+function writeSessionFile(name: string, texts: readonly string[]): string {
+ const path = join(vault, name);
+ const lines = texts.map((text, index) =>
+ JSON.stringify({
+ type: "user",
+ parentUuid: null,
+ entrypoint: "cli",
+ uuid: `u${index}`,
+ timestamp: "2026-08-15T09:00:00.000Z",
+ sessionId: "s1",
+ message: { role: "user", content: [{ type: "text", text }] },
+ }),
+ );
+ writeFileSync(path, lines.join("\n") + "\n", "utf8");
+ return path;
+}
+
+function marker(topic: string, principle: string): string {
+ return `@osb feedback positive topic=${topic} principle="${principle}"`;
+}
+
+function hashOf(topic: string, principle: string): string {
+ return computeDedupHash({ topic, signal: "positive", principle });
+}
+
+function inboxFiles(): string[] {
+ return readdirSync(brainDirs(vault).inbox).filter((n) => n.endsWith(".md"));
+}
+
+function writeVaultFile(rel: string, content: string): void {
+ const abs = join(vault, rel);
+ mkdirSync(join(abs, ".."), { recursive: true });
+ writeFileSync(abs, content, "utf8");
+}
+
+describe("buildReadBackCensus", () => {
+ test("names every key the read-back could not find and calls the run partial", () => {
+ const present = new Set(["a", "c"]);
+ const census = buildReadBackCensus(["a", "b", "c", "d"], (key) => present.has(key));
+ expect(census.attempted).toBe(4);
+ expect(census.found).toBe(2);
+ expect([...census.missing]).toEqual(["b", "d"]);
+ expect(census.outcome).toBe(RECONCILIATION_OUTCOME.partial);
+ });
+
+ test("a clean read-back is complete with an empty missing list", () => {
+ const census = buildReadBackCensus(["a", "b"], () => true);
+ expect(census.attempted).toBe(2);
+ expect(census.found).toBe(2);
+ expect([...census.missing]).toEqual([]);
+ expect(census.outcome).toBe(RECONCILIATION_OUTCOME.complete);
+ });
+
+ test("a key claimed twice is one claim, so attempted cannot be inflated", () => {
+ const census = buildReadBackCensus(["a", "a", "b"], (key) => key === "a");
+ expect(census.attempted).toBe(2);
+ expect(census.found).toBe(1);
+ expect([...census.missing]).toEqual(["b"]);
+ });
+
+ test("claiming nothing is complete, not an error", () => {
+ const census = buildReadBackCensus([], () => false);
+ expect(census.attempted).toBe(0);
+ expect(census.outcome).toBe(RECONCILIATION_OUTCOME.complete);
+ });
+
+ test("the serialized payload carries the named keys, never a bare count", () => {
+ const wire = serializeImportCensus(buildReadBackCensus(["a", "b"], (k) => k === "a"));
+ expect(wire).toEqual({
+ attempted: 2,
+ found: 1,
+ missing: ["b"],
+ outcome: RECONCILIATION_OUTCOME.partial,
+ });
+ });
+});
+
+describe("censusSessionSignals - the sessions lane reads back its dedup hashes", () => {
+ test("every signal the import wrote reads back, and the run is complete", async () => {
+ const path = writeSessionFile("s.jsonl", [
+ marker("census-one", "Keep the first rule"),
+ marker("census-two", "Keep the second rule"),
+ ]);
+ const result = await importSession(vault, path, { agent: "@t", now: NOW });
+ expect(result.signals_created).toBe(2);
+
+ const claimed = [
+ hashOf("census-one", "Keep the first rule"),
+ hashOf("census-two", "Keep the second rule"),
+ ];
+ const census = censusSessionSignals(vault, claimed);
+ expect(census.attempted).toBe(2);
+ expect(census.found).toBe(2);
+ expect(census.outcome).toBe(RECONCILIATION_OUTCOME.complete);
+ });
+
+ test("a signal removed after the write is named by its hash, not merely counted", async () => {
+ const path = writeSessionFile("s.jsonl", [
+ marker("census-one", "Keep the first rule"),
+ marker("census-two", "Keep the second rule"),
+ ]);
+ await importSession(vault, path, { agent: "@t", now: NOW });
+
+ // The synthetic loss: one signal disappears between the write and the
+ // read-back, which is precisely the case a bare counter reports as fine.
+ const victim = inboxFiles().find((n) => n.includes("census-two"));
+ expect(victim).toBeDefined();
+ unlinkSync(join(brainDirs(vault).inbox, victim!));
+
+ const lost = hashOf("census-two", "Keep the second rule");
+ const census = censusSessionSignals(vault, [hashOf("census-one", "Keep the first rule"), lost]);
+ expect(census.attempted).toBe(2);
+ expect(census.found).toBe(1);
+ expect([...census.missing]).toEqual([lost]);
+ expect(census.outcome).toBe(RECONCILIATION_OUTCOME.partial);
+ });
+
+ test("the import result carries the census of what it claims to have written", async () => {
+ const path = writeSessionFile("s.jsonl", [marker("census-rides", "Ride the result")]);
+ const result = await importSession(vault, path, { agent: "@t", now: NOW });
+ expect(result.census.attempted).toBe(result.signals_created);
+ expect(result.census.found).toBe(1);
+ expect([...result.census.missing]).toEqual([]);
+ expect(result.census.outcome).toBe(RECONCILIATION_OUTCOME.complete);
+ });
+
+ test("a dry run claims nothing, so its census is complete over zero", async () => {
+ const path = writeSessionFile("s.jsonl", [marker("census-dry", "Rehearse only")]);
+ const result = await importSession(vault, path, { agent: "@t", dryRun: true, now: NOW });
+ expect(result.signals_created).toBe(0);
+ expect(result.census.attempted).toBe(0);
+ expect(result.census.outcome).toBe(RECONCILIATION_OUTCOME.complete);
+ });
+});
+
+describe("censusIngestedPaths - the ingest lane reads back its content hashes", () => {
+ test("paths the manifest still confirms are found", () => {
+ writeVaultFile("Docs/a.md", "alpha\n");
+ writeVaultFile("Docs/b.md", "beta\n");
+ updateManifest(vault, ["Docs/a.md", "Docs/b.md"]);
+
+ const census = censusIngestedPaths(vault, ["Docs/a.md", "Docs/b.md"]);
+ expect(census.attempted).toBe(2);
+ expect(census.found).toBe(2);
+ expect(census.outcome).toBe(RECONCILIATION_OUTCOME.complete);
+ });
+
+ test("a claimed path deleted after the write is named", () => {
+ writeVaultFile("Docs/a.md", "alpha\n");
+ writeVaultFile("Docs/b.md", "beta\n");
+ updateManifest(vault, ["Docs/a.md", "Docs/b.md"]);
+ unlinkSync(join(vault, "Docs/b.md"));
+
+ const census = censusIngestedPaths(vault, ["Docs/a.md", "Docs/b.md"]);
+ expect(census.found).toBe(1);
+ expect([...census.missing]).toEqual(["Docs/b.md"]);
+ expect(census.outcome).toBe(RECONCILIATION_OUTCOME.partial);
+ });
+
+ test("a claimed path the manifest never recorded is missing, not silently fine", () => {
+ writeVaultFile("Docs/a.md", "alpha\n");
+ writeVaultFile("Docs/b.md", "beta\n");
+ updateManifest(vault, ["Docs/a.md"]);
+
+ const census = censusIngestedPaths(vault, ["Docs/a.md", "Docs/b.md"]);
+ expect([...census.missing]).toEqual(["Docs/b.md"]);
+ });
+
+ test("a claimed path whose bytes no longer match the recorded hash is missing", () => {
+ writeVaultFile("Docs/a.md", "alpha\n");
+ updateManifest(vault, ["Docs/a.md"]);
+ writeVaultFile("Docs/a.md", "alpha rewritten\n");
+
+ const census = censusIngestedPaths(vault, ["Docs/a.md"]);
+ expect(census.found).toBe(0);
+ expect([...census.missing]).toEqual(["Docs/a.md"]);
+ });
+});
diff --git a/tests/core/brain/ingest/ingest.test.ts b/tests/core/brain/ingest/ingest.test.ts
index 74209c32..ea7abd51 100644
--- a/tests/core/brain/ingest/ingest.test.ts
+++ b/tests/core/brain/ingest/ingest.test.ts
@@ -7,7 +7,7 @@
*/
import { afterEach, beforeEach, describe, expect, test } from "bun:test";
-import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
+import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
import { join } from "node:path";
import { tmpdir } from "node:os";
@@ -135,6 +135,19 @@ describe("ingestSource", () => {
ingestSource(vault, INPUT, { agent: "claude", now: NOW });
expect(readCheckpoint(vault, planId)).toBeNull();
});
+
+ test("a plan id the checkpoint store cannot use is refused before anything is written", () => {
+ // A plan id becomes a checkpoint filename, so the store refuses
+ // anything but lowercase hex. Swallowed - as it was - the ingest
+ // succeeded, no checkpoint was EVER written for that plan, and
+ // `--reconcile` then reported every source as never ingested.
+ seedSourceFile();
+ expect(() =>
+ ingestSource(vault, INPUT, { agent: "claude", now: NOW, planId: "docs-migration" }),
+ ).toThrow(/plan id/);
+ // Refused at the boundary: the summary page was not written either.
+ expect(existsSync(join(vault, "Brain", "sources"))).toBe(false);
+ });
});
describe("ingestSource pre-extract pass (P4, t_ef786747)", () => {
diff --git a/tests/core/brain/ingest/reconcile.test.ts b/tests/core/brain/ingest/reconcile.test.ts
index 6af9b725..0744a7f3 100644
--- a/tests/core/brain/ingest/reconcile.test.ts
+++ b/tests/core/brain/ingest/reconcile.test.ts
@@ -26,6 +26,8 @@ let configHome: string;
const NOW = new Date("2026-06-13T12:00:00Z");
const CAPS = { maxBatchBytes: 100_000, maxBatchFiles: 100 } as const;
+/** A plan id the checkpoint store accepts, for fixtures that need any id. */
+const SEED_PLAN_ID = "5eed01";
beforeEach(() => {
vault = mkdtempSync(join(tmpdir(), "o2b-reconcile-vault-"));
@@ -110,8 +112,12 @@ describe("reconcilePlan", () => {
write("Docs/b.md");
// Ingest both so the content manifest records them; a re-plan then
// classifies every source `unchanged` (manifest skips), not new work.
- ingest("Docs/a.md", "seed-plan");
- ingest("Docs/b.md", "seed-plan");
+ // A valid (hex) plan id that is not the plan under test: the checkpoint
+ // it writes is irrelevant here, and the assertions below are about the
+ // manifest path. The old fixture passed "seed-plan", which the
+ // checkpoint store rejects - the argument did nothing at all.
+ ingest("Docs/a.md", SEED_PLAN_ID);
+ ingest("Docs/b.md", SEED_PLAN_ID);
const plan = planBatches(vault, "Docs", CAPS);
expect(plan.batches).toEqual([]); // nothing new to dispatch
@@ -154,3 +160,51 @@ describe("reconcilePlan", () => {
expect(readFileSync(cpPath, "utf8")).toBe(before);
});
});
+
+/**
+ * The gap report above answers "what did the checkpoint never record". The
+ * census answers the question one step further out: of the sources the
+ * checkpoint DOES claim, which can still be read back from the content
+ * manifest? A claim nobody can reproduce is not a completed ingest, and
+ * before this it was reported as one.
+ */
+describe("reconcilePlan - the post-import read-back census", () => {
+ test("a plan whose ingested sources all read back is complete", () => {
+ write("Docs/a.md");
+ write("Docs/b.md");
+ const plan = planBatches(vault, "Docs", CAPS);
+ ingest("Docs/a.md", plan.planId);
+ ingest("Docs/b.md", plan.planId);
+
+ const report = reconcilePlan(vault, plan);
+ expect(report.census.attempted).toBe(2);
+ expect(report.census.found).toBe(2);
+ expect([...report.census.missing]).toEqual([]);
+ expect(report.census.outcome).toBe("complete");
+ });
+
+ test("a source the checkpoint claims but the manifest cannot confirm is named", () => {
+ write("Docs/a.md");
+ write("Docs/b.md");
+ const plan = planBatches(vault, "Docs", CAPS);
+ ingest("Docs/a.md", plan.planId);
+ ingest("Docs/b.md", plan.planId);
+ // The write landed in the checkpoint and then the source vanished: the
+ // gap report still calls the plan complete, the census does not.
+ rmSync(join(vault, "Docs/b.md"));
+
+ const report = reconcilePlan(vault, plan);
+ expect(report.complete).toBe(true);
+ expect(report.census.found).toBe(1);
+ expect([...report.census.missing]).toEqual(["Docs/b.md"]);
+ expect(report.census.outcome).toBe("partial");
+ });
+
+ test("a plan that ingested nothing censuses zero rather than claiming success", () => {
+ write("Docs/a.md");
+ const plan = planBatches(vault, "Docs", CAPS);
+ const report = reconcilePlan(vault, plan);
+ expect(report.census.attempted).toBe(0);
+ expect(report.census.outcome).toBe("complete");
+ });
+});
diff --git a/tests/core/brain/link-candidate-envelopes.test.ts b/tests/core/brain/link-candidate-envelopes.test.ts
new file mode 100644
index 00000000..1d4d27b6
--- /dev/null
+++ b/tests/core/brain/link-candidate-envelopes.test.ts
@@ -0,0 +1,144 @@
+/**
+ * Link candidates on the three note-producing envelopes
+ * (nothing-writes-silently, unit E).
+ *
+ * Claims pinned here:
+ *
+ * 1. The SPINE does not move. `LlmStepFields` and `NEEDS_LLM_STEP_KEYS`
+ * carry exactly what they carried; `link_candidates` rides on the
+ * consumer's own type, exactly as the rollup's `tier`/`produces`
+ * already do, and lands AFTER the spine's own keys so no existing
+ * serialized position changes.
+ * 2. All three note-producing lanes - rollup ladder, diarization,
+ * design note - carry a manifest, and each carries the one schema
+ * hint that says what the manifest is.
+ * 3. The three lanes that produce no wikilinked note - skill page
+ * drafts, extract-signals, the durable write session - carry NO
+ * manifest. A field three of six consumers want is not spine, and
+ * this is the assertion that keeps it off the other three.
+ * 4. The rollup ladder stays PURE: its manifest arrives on the input
+ * rather than being walked from a vault inside the planner.
+ */
+
+import { beforeEach, describe, expect, test } from "bun:test";
+import { mkdirSync, mkdtempSync, readFileSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { dirname, join } from "node:path";
+
+import { diarize } from "../../../src/core/brain/diarization.ts";
+import { planDesignNote } from "../../../src/core/brain/design-note.ts";
+import { upsertEntity } from "../../../src/core/brain/entities/registry.ts";
+import { NEEDS_LLM_STEP_KEYS } from "../../../src/core/brain/llm-step.ts";
+import {
+ buildLinkCandidateManifest,
+ type LinkCandidateManifest,
+} from "../../../src/core/brain/notes/link-candidates.ts";
+import { planRollupLadder } from "../../../src/core/brain/rollup-ladder.ts";
+
+const NOW = new Date("2026-08-23T10:00:00Z");
+const RUN_ID = "dream-2026-08-23-100000";
+const REPO_ROOT = join(import.meta.dir, "..", "..", "..");
+
+/** The three envelope lanes whose artifact carries no wikilink. */
+const NON_NOTE_LANES: ReadonlyArray<string> = Object.freeze([
+ "src/core/brain/skill-page-drafts.ts",
+ "src/core/brain/extract-signals.ts",
+ "src/core/brain/write-session/types.ts",
+]);
+
+let vault: string;
+
+function writeNote(rel: string, body: string): void {
+ const abs = join(vault, rel);
+ mkdirSync(dirname(abs), { recursive: true });
+ writeFileSync(abs, body);
+}
+
+beforeEach(() => {
+ vault = mkdtempSync(join(tmpdir(), "o2b-link-envelopes-"));
+ mkdirSync(join(vault, "Brain"), { recursive: true });
+ writeNote("notes/analytical-engine.md", "# Analytical engine\n");
+ writeNote("notes/vector-index-storage.md", "# Vector index storage\n");
+});
+
+describe("the spine does not move", () => {
+ test("NEEDS_LLM_STEP_KEYS is byte-identical", () => {
+ expect([...NEEDS_LLM_STEP_KEYS]).toEqual([
+ "status",
+ "step",
+ "prompt",
+ "schema_hints",
+ "target_path",
+ ]);
+ });
+});
+
+describe("the three note-producing lanes carry a manifest", () => {
+ test("the rollup envelope carries the manifest it was handed, last in key order", () => {
+ const linkCandidates = buildLinkCandidateManifest(vault, { visible: () => true });
+ const plan = planRollupLadder({
+ factCount: 5,
+ ledger: null,
+ thresholds: { fact: 5, identity: 2 },
+ runId: RUN_ID,
+ linkCandidates: () => linkCandidates,
+ });
+ const envelope = plan.entries[0]!.envelope;
+ expect(envelope.link_candidates).toEqual(linkCandidates);
+ expect(Object.keys(envelope)).toEqual([
+ "status",
+ "step",
+ "tier",
+ "produces",
+ "prompt",
+ "schema_hints",
+ "target_path",
+ "link_candidates",
+ ]);
+ expect(envelope.schema_hints.some((hint) => hint.includes("link_candidates"))).toBe(true);
+ });
+
+ test("the diarization envelope ranks candidates on the subject it profiles", () => {
+ upsertEntity(vault, {
+ category: "person",
+ name: "Ada Lovelace",
+ agent: "test",
+ now: NOW,
+ body: "Ada Lovelace worked on the analytical engine.",
+ });
+ const report = diarize(vault, { query: "Ada Lovelace" }, { now: NOW, ownerScope: null });
+ const manifest: LinkCandidateManifest = report.llmStep.link_candidates;
+ expect(manifest.candidates).toContain("analytical-engine");
+ expect(manifest.total).toBeGreaterThan(0);
+ expect(report.llmStep.schema_hints.some((hint) => hint.includes("link_candidates"))).toBe(true);
+ // The spine's own keys keep their positions; the manifest is appended.
+ expect(Object.keys(report.llmStep).at(-1)).toBe("link_candidates");
+ });
+
+ test("the design-note envelope carries a manifest ranked on the topic", () => {
+ const report = planDesignNote(vault, "vector index storage", { now: NOW, ownerScope: null });
+ expect(report.llmStep.link_candidates.candidates).toContain("vector-index-storage");
+ expect(report.llmStep.schema_hints.some((hint) => hint.includes("link_candidates"))).toBe(true);
+ expect(Object.keys(report.llmStep).at(-1)).toBe("link_candidates");
+ });
+});
+
+describe("the lanes that produce no wikilinked note carry none", () => {
+ /**
+ * Read off the source rather than driven, because each of these three
+ * needs a whole world to build one envelope - imported session turns,
+ * observed-use verdicts, an open durable session - and the claim is
+ * about the FIELD, which the module either names or does not. A lane
+ * that started carrying a manifest would have to name it here.
+ */
+ test("skill page drafts, extract-signals and the write session never name the field", () => {
+ const carriers = NON_NOTE_LANES.filter((rel) =>
+ readFileSync(join(REPO_ROOT, rel), "utf8").includes("link_candidates"),
+ );
+ expect(carriers.join("\n")).toBe("");
+ // The read has to be reaching real envelope lanes, or it proves nothing.
+ for (const rel of NON_NOTE_LANES) {
+ expect(readFileSync(join(REPO_ROOT, rel), "utf8")).toMatch(/LlmStepFields|NeedsLlmStep/);
+ }
+ });
+});
diff --git a/tests/core/brain/llm-step.test.ts b/tests/core/brain/llm-step.test.ts
index a9d8bd46..3d5ab0e0 100644
--- a/tests/core/brain/llm-step.test.ts
+++ b/tests/core/brain/llm-step.test.ts
@@ -41,6 +41,7 @@ import {
type LlmStepFields,
type NeedsLlmStep,
} from "../../../src/core/brain/llm-step.ts";
+import type { LinkCandidateManifest } from "../../../src/core/brain/notes/link-candidates.ts";
import { planRollupLadder } from "../../../src/core/brain/rollup-ladder.ts";
import type { WriteSessionEnvelope } from "../../../src/core/brain/write-session/types.ts";
@@ -48,17 +49,38 @@ const RUN_ID = "dream-2026-07-19-100000";
const NOW = new Date("2026-07-19T10:00:00Z");
/**
- * The rollup envelope exactly as it serialized before the spine existed,
- * captured from the ladder on the parent commit. Byte-for-byte, key order
- * included - this envelope crosses the CLI and MCP boundaries as JSON.
+ * The manifest a fired rung is handed. A value, because the planner is
+ * pure - see `RollupLadderInput.linkCandidates`.
*/
-const ROLLUP_ENVELOPE_JSON_BEFORE_SPINE =
+const LINK_CANDIDATES: LinkCandidateManifest = Object.freeze({
+ candidates: Object.freeze(["analytical-engine"]),
+ total: 1,
+ truncated: false,
+ selection: "all",
+});
+
+/**
+ * The rollup envelope, byte-for-byte and in key order - it crosses the
+ * CLI and MCP boundaries as JSON.
+ *
+ * The spine's own keys are exactly where they were when this pin was
+ * first captured (before the spine existed). What moved in
+ * nothing-writes-silently is that the lane's own `link_candidates` is
+ * appended AFTER `target_path`, with the one constraint line naming it
+ * added to `schema_hints`: a consumer field, in the consumer's position,
+ * leaving every existing position where it was.
+ */
+const ROLLUP_ENVELOPE_JSON =
'{"status":"needs-llm-step","step":"rollup:fact","tier":"fact","produces":"rollup",' +
'"prompt":"Consolidate the 5 new fact items since the last rollup into one rollup-tier ' +
'summary note. Cite the items you fold in; submit the full note.",' +
'"schema_hints":["frontmatter: required YAML block with at least a `kind` key",' +
- '"tier: rollup (the rollup\'s tier weight)"],' +
- '"target_path":"Brain/rollups/rollup-rollup-dream-2026-07-19-100000.md"}';
+ '"tier: rollup (the rollup\'s tier weight)",' +
+ '"wikilinks: link_candidates carries all 1 note names this vault offers; a name it does ' +
+ 'not carry is not one you can cite and the link will dangle"],' +
+ '"target_path":"Brain/rollups/rollup-rollup-dream-2026-07-19-100000.md",' +
+ '"link_candidates":{"candidates":["analytical-engine"],"total":1,"truncated":false,' +
+ '"selection":"all"}}';
/** Compile-time pin: anything passed here carries the whole spine. */
function acceptsSpine(_envelope: NeedsLlmStep): void {}
@@ -71,6 +93,7 @@ function fired() {
ledger: null,
thresholds: { fact: 5, identity: 2 },
runId: RUN_ID,
+ linkCandidates: () => LINK_CANDIDATES,
});
return plan.entries[0]!.envelope;
}
@@ -173,7 +196,7 @@ test("the rollup envelope is the spine, and serializes as it did before adoption
const envelope = fired();
acceptsSpine(envelope);
acceptsSpineFields(envelope);
- expect(JSON.stringify(envelope)).toBe(ROLLUP_ENVELOPE_JSON_BEFORE_SPINE);
+ expect(JSON.stringify(envelope)).toBe(ROLLUP_ENVELOPE_JSON);
for (const key of NEEDS_LLM_STEP_KEYS) expect(Object.hasOwn(envelope, key)).toBe(true);
});
@@ -215,15 +238,19 @@ afterEach(() => {
rmSync(vault, { recursive: true, force: true });
});
-test("the diarization step is the spine, with its key order unchanged", () => {
- const report = diarize(vault, { query: "Ada Lovelace" }, { now: NOW });
+test("the diarization step is the spine, with the spine's key order unchanged", () => {
+ const report = diarize(vault, { query: "Ada Lovelace" }, { now: NOW, ownerScope: null });
acceptsSpine(report.llmStep);
+ // The lane's own `link_candidates` is appended after the spine, in the
+ // consumer position the rollup's `tier`/`produces` already occupy: the
+ // spine's five keys are still the first five, in order.
expect(Object.keys(report.llmStep)).toEqual([
"status",
"step",
"prompt",
"schema_hints",
"target_path",
+ "link_candidates",
]);
expect(report.llmStep.status).toBe(NEEDS_LLM_STEP);
});
diff --git a/tests/core/brain/notes/blank-overwrite-guard.test.ts b/tests/core/brain/notes/blank-overwrite-guard.test.ts
new file mode 100644
index 00000000..3ac5a989
--- /dev/null
+++ b/tests/core/brain/notes/blank-overwrite-guard.test.ts
@@ -0,0 +1,60 @@
+/**
+ * The pure blank-overwrite predicate (nothing-writes-silently, unit B).
+ *
+ * Claims pinned here:
+ *
+ * 1. A blank replacement over a body that carries text is REFUSED.
+ * 2. `allowEmpty` is the one way through, so a deliberate clear stays
+ * possible and is never confused with an accidental one.
+ * 3. "Blank" is `trim().length === 0`, so a whitespace-only body is
+ * blank - the parser trims, so those bytes are read back as an
+ * empty body and the note is emptied either way.
+ * 4. A body that is ALREADY blank has nothing to lose, so replacing it
+ * with another blank body is not a refusal.
+ * 5. The predicate is pure: same inputs, same answer, no I/O.
+ */
+
+import { describe, expect, test } from "bun:test";
+
+import { refuseBlankOverwrite } from "../../../../src/core/brain/notes/blank-overwrite-guard.ts";
+
+describe("refuseBlankOverwrite", () => {
+ test("an empty replacement over a body with text is refused", () => {
+ expect(refuseBlankOverwrite({ existingBody: "real content", nextBody: "" })).toBe(true);
+ });
+
+ test("a replacement that carries text is never refused", () => {
+ expect(refuseBlankOverwrite({ existingBody: "real content", nextBody: "new" })).toBe(false);
+ });
+
+ test("allowEmpty lets a deliberate clear through", () => {
+ expect(
+ refuseBlankOverwrite({ existingBody: "real content", nextBody: "", allowEmpty: true }),
+ ).toBe(false);
+ });
+
+ test("whitespace-only counts as blank on the replacement side", () => {
+ // The last entry is U+00A0 NO-BREAK SPACE, which `trim` also removes.
+ for (const nextBody of [" ", "\n", "\t\n \n", " "]) {
+ expect(refuseBlankOverwrite({ existingBody: "real content", nextBody })).toBe(true);
+ }
+ });
+
+ test("a character that is not whitespace is content, however invisible", () => {
+ // U+200B ZERO WIDTH SPACE is not ECMAScript whitespace, so `trim`
+ // keeps it and the note keeps a body. The predicate does not invent
+ // a wider notion of emptiness than the parser has.
+ expect(refuseBlankOverwrite({ existingBody: "real content", nextBody: "​" })).toBe(false);
+ });
+
+ test("an already-blank body has nothing to lose", () => {
+ expect(refuseBlankOverwrite({ existingBody: "", nextBody: "" })).toBe(false);
+ expect(refuseBlankOverwrite({ existingBody: " \n", nextBody: "" })).toBe(false);
+ });
+
+ test("the predicate is pure - repeated calls agree", () => {
+ const input = { existingBody: "keep me", nextBody: " " } as const;
+ expect(refuseBlankOverwrite(input)).toBe(true);
+ expect(refuseBlankOverwrite(input)).toBe(true);
+ });
+});
diff --git a/tests/core/brain/notes/create-note-modes.test.ts b/tests/core/brain/notes/create-note-modes.test.ts
index 75e01fd5..5ce373b8 100644
--- a/tests/core/brain/notes/create-note-modes.test.ts
+++ b/tests/core/brain/notes/create-note-modes.test.ts
@@ -68,26 +68,41 @@ const BASELINE_INPUT = Object.freeze({
});
/**
- * The exact bytes {@link BASELINE_INPUT} produced with none of the three
- * authoring options set, captured from the primitive as it stood before
- * they existed. Pinned as a literal rather than recomputed from the
- * implementation, because an expectation derived from the implementation
- * cannot notice the implementation moving: an unconditional spread adding
- * one frontmatter key, or a reordering of the block, changes this string
- * and nothing else in this file.
+ * The exact bytes {@link BASELINE_INPUT} produces with none of the three
+ * authoring options set. Pinned as a literal rather than recomputed from
+ * the implementation, because an expectation derived from the
+ * implementation cannot notice the implementation moving: an
+ * unconditional spread adding one frontmatter key, or a reordering of the
+ * block, changes this string and nothing else in this file.
+ *
+ * It caught exactly that, once, on purpose. The trailing
+ * `origin_channel` line is the server-derived origin channel (Unit C of
+ * the nothing-writes-silently wave): `createNote` now stamps the channel
+ * of the process that wrote the note, merged LAST so a caller cannot
+ * name it. It reads `unset` here because a test harness is not one of
+ * the entry points that claims a channel, and `unset` is the explicit
+ * literal for that rather than a guess. The three authoring options are
+ * still byte-neutral when absent, which is what these tests measure; the
+ * stamp is unconditional by design and is part of the golden now.
*/
const GOLDEN_BASELINE_NOTE =
- "---\ntitle: Golden\ntype: note\ntags: [a, b]\n---\n\nBody line one.\n\nBody line two.\n";
+ "---\ntitle: Golden\ntype: note\ntags: [a, b]\norigin_channel: unset\n---\n\n" +
+ "Body line one.\n\nBody line two.\n";
-/** Frontmatter key order of the pre-feature note, top to bottom. */
+/** Frontmatter key order of the note, top to bottom, stamp last. */
const GOLDEN_BASELINE_FRONTMATTER_KEYS: ReadonlyArray<string> = Object.freeze([
"title",
"type",
"tags",
+ "origin_channel",
]);
-/** The pre-feature bytes for a note with no frontmatter map at all. */
-const GOLDEN_BARE_NOTE = "---\n---\n\njust a body\n";
+/**
+ * The bytes for a note with no frontmatter MAP at all. The block is no
+ * longer empty: the origin-channel stamp is the one key every created
+ * note carries, whatever the caller supplied.
+ */
+const GOLDEN_BARE_NOTE = "---\norigin_channel: unset\n---\n\njust a body\n";
/** Result field order the MCP surface and the SDK both read. */
const GOLDEN_RESULT_KEYS: ReadonlyArray<string> = Object.freeze(["path", "outcome", "created"]);
@@ -170,7 +185,9 @@ describe("createNote - the new options are byte-identical when absent", () => {
});
test("a document the strict validator would reject is still written when strict is unset", () => {
- // No frontmatter at all: `validateArtifact` reports frontmatter-missing.
+ // No CALLER frontmatter at all: `validateArtifact` reports
+ // frontmatter-missing, which it still can because the origin-channel
+ // stamp is merged after validation, not before it.
const res = createNote(vault, { path: "Notes/Bare.md", content: "just a body" });
expect(res.created).toBe(true);
// The whole file, not a substring: an empty frontmatter block is part
diff --git a/tests/core/brain/notes/link-candidates.test.ts b/tests/core/brain/notes/link-candidates.test.ts
new file mode 100644
index 00000000..fb5dc39d
--- /dev/null
+++ b/tests/core/brain/notes/link-candidates.test.ts
@@ -0,0 +1,207 @@
+/**
+ * The zero-embed link-candidate manifest (nothing-writes-silently, unit E).
+ *
+ * Claims pinned here:
+ *
+ * 1. The manifest is built from the vault's own basenames and nothing
+ * else: no embedding provider, no vector store, no search index is
+ * reachable from this module by any import path.
+ * 2. A vault inside the bound reports every basename it holds, says the
+ * list is whole, and names how it was chosen.
+ * 3. A vault over the bound NAMES the total it could not carry rather
+ * than handing back a short list that reads as the whole vault.
+ * 4. A query ranks the kept subset by the vault's own deterministic
+ * token overlap; without one the subset is alphabetical, and both
+ * selections are named on the manifest.
+ * 5. A limit that is not a positive integer is refused by name.
+ * 5a. A basename the caller may not see is DROPPED AND UNCOUNTED - the
+ * IDENTICAL TO ABSENT convention: a total that included hidden pages
+ * would tell one agent how many of another's memories exist.
+ * 6. The schema hint states the bound honestly in all three cases -
+ * whole vault, truncated subset, and a vault with no notes at all.
+ */
+
+import { describe, expect, test } from "bun:test";
+import { mkdirSync, mkdtempSync, readFileSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { dirname, join, relative, resolve } from "node:path";
+
+import {
+ buildLinkCandidateManifest,
+ LINK_CANDIDATE_LIMIT,
+ LinkCandidateError,
+ linkCandidateSchemaHint,
+} from "../../../../src/core/brain/notes/link-candidates.ts";
+
+const REPO_ROOT = join(import.meta.dir, "..", "..", "..", "..");
+const MODULE_REL = "src/core/brain/notes/link-candidates.ts";
+
+/** Roots no zero-embed module may reach, by any chain of imports. */
+const FORBIDDEN_ROOTS: ReadonlyArray<string> = Object.freeze([
+ "src/core/search/embeddings",
+ "src/core/search/store",
+ "src/core/search/indexer.ts",
+]);
+
+/** The unfiltered caller - what `ownerScopeView(vault, null)` hands back. */
+const ALL_VISIBLE = Object.freeze({ visible: () => true });
+
+function vaultWith(names: ReadonlyArray<string>): string {
+ const vault = mkdtempSync(join(tmpdir(), "o2b-link-candidates-"));
+ mkdirSync(join(vault, "Brain"), { recursive: true });
+ for (const name of names) {
+ const abs = join(vault, `${name}.md`);
+ mkdirSync(dirname(abs), { recursive: true });
+ writeFileSync(abs, `# ${name}\n`);
+ }
+ return vault;
+}
+
+/** Every `src/` module reachable from `entry`, entry included. */
+function importClosure(entry: string): ReadonlyArray<string> {
+ const seen = new Set<string>();
+ const queue = [entry];
+ while (queue.length > 0) {
+ const rel = queue.pop()!;
+ if (seen.has(rel)) continue;
+ seen.add(rel);
+ const text = readFileSync(join(REPO_ROOT, rel), "utf8");
+ for (const match of text.matchAll(/from\s+"(\.[^"]+)"/g)) {
+ const target = resolve(dirname(join(REPO_ROOT, rel)), match[1]!);
+ queue.push(relative(REPO_ROOT, target).split("\\").join("/"));
+ }
+ }
+ return [...seen].toSorted();
+}
+
+describe("the manifest is zero-embed by construction", () => {
+ test("no provider, vector store or index module is reachable from it", () => {
+ const closure = importClosure(MODULE_REL);
+ const reached = closure.filter((path) =>
+ FORBIDDEN_ROOTS.some((root) => path === root || path.startsWith(`${root}/`)),
+ );
+ expect(reached.join("\n")).toBe("");
+ // …and the walk has to have found something, or it proves nothing.
+ expect(closure.length).toBeGreaterThan(1);
+ });
+});
+
+describe("a vault inside the bound", () => {
+ test("reports every basename, whole and named as whole", () => {
+ const vault = vaultWith(["beta", "alpha", "notes/gamma"]);
+ const manifest = buildLinkCandidateManifest(vault, ALL_VISIBLE);
+ expect(manifest.candidates).toEqual(["alpha", "beta", "gamma"]);
+ expect(manifest.total).toBe(3);
+ expect(manifest.truncated).toBe(false);
+ expect(manifest.selection).toBe("all");
+ });
+
+ test("is frozen, so an envelope cannot be edited after it is built", () => {
+ const manifest = buildLinkCandidateManifest(vaultWith(["alpha"]), ALL_VISIBLE);
+ expect(Object.isFrozen(manifest)).toBe(true);
+ expect(Object.isFrozen(manifest.candidates)).toBe(true);
+ });
+});
+
+describe("a vault over the bound", () => {
+ const names = Array.from({ length: 12 }, (_, i) => `note-${String(i).padStart(2, "0")}`);
+
+ test("names the total it could not carry rather than reading as the whole vault", () => {
+ const manifest = buildLinkCandidateManifest(vaultWith(names), { ...ALL_VISIBLE, limit: 4 });
+ expect(manifest.candidates).toHaveLength(4);
+ expect(manifest.total).toBe(12);
+ expect(manifest.truncated).toBe(true);
+ expect(manifest.selection).toBe("alphabetical");
+ expect(manifest.candidates).toEqual(["note-00", "note-01", "note-02", "note-03"]);
+ });
+
+ test("a query ranks the kept subset by the vault's own token overlap", () => {
+ const vault = vaultWith([...names, "ada-lovelace", "charles-babbage"]);
+ const manifest = buildLinkCandidateManifest(vault, {
+ ...ALL_VISIBLE,
+ limit: 2,
+ query: "Ada Lovelace",
+ });
+ expect(manifest.selection).toBe("relevance");
+ expect(manifest.candidates[0]).toBe("ada-lovelace");
+ expect(manifest.total).toBe(14);
+ expect(manifest.truncated).toBe(true);
+ });
+
+ test("ranking is deterministic: the same vault and query give the same list", () => {
+ const vault = vaultWith([...names, "ada-lovelace"]);
+ const first = buildLinkCandidateManifest(vault, { ...ALL_VISIBLE, limit: 3, query: "ada" });
+ const second = buildLinkCandidateManifest(vault, { ...ALL_VISIBLE, limit: 3, query: "ada" });
+ expect(first.candidates).toEqual(second.candidates);
+ });
+});
+
+describe("a candidate the caller may not see", () => {
+ test("is dropped, and does not survive in the total either", () => {
+ const vault = vaultWith(["pref-mine", "pref-theirs", "shared"]);
+ const manifest = buildLinkCandidateManifest(vault, {
+ visible: (rel) => rel !== "pref-theirs.md",
+ });
+ expect(manifest.candidates).toEqual(["pref-mine", "shared"]);
+ expect(manifest.total).toBe(2);
+ expect(manifest.truncated).toBe(false);
+ });
+
+ test("cannot reappear through the ranking path either", () => {
+ const names = Array.from({ length: 9 }, (_, i) => `note-${i}`);
+ const manifest = buildLinkCandidateManifest(vaultWith([...names, "pref-theirs"]), {
+ visible: (rel) => rel !== "pref-theirs.md",
+ query: "pref theirs",
+ limit: 3,
+ });
+ expect(manifest.candidates).not.toContain("pref-theirs");
+ expect(manifest.total).toBe(9);
+ });
+});
+
+describe("the bound itself", () => {
+ test("the default is the named constant, not a literal at the call site", () => {
+ expect(LINK_CANDIDATE_LIMIT).toBeGreaterThan(0);
+ expect(Number.isInteger(LINK_CANDIDATE_LIMIT)).toBe(true);
+ });
+
+ test("a limit that is not a positive integer is refused by name", () => {
+ const vault = vaultWith(["alpha"]);
+ expect(() => buildLinkCandidateManifest(vault, { ...ALL_VISIBLE, limit: 0 })).toThrow(
+ LinkCandidateError,
+ );
+ expect(() => buildLinkCandidateManifest(vault, { ...ALL_VISIBLE, limit: 0 })).toThrow(/limit/);
+ expect(() => buildLinkCandidateManifest(vault, { ...ALL_VISIBLE, limit: -3 })).toThrow(/limit/);
+ expect(() => buildLinkCandidateManifest(vault, { ...ALL_VISIBLE, limit: 2.5 })).toThrow(
+ /limit/,
+ );
+ expect(() => buildLinkCandidateManifest(vault, { ...ALL_VISIBLE, limit: Number.NaN })).toThrow(
+ /limit/,
+ );
+ });
+});
+
+describe("the schema hint says what the list is", () => {
+ test("a whole-vault manifest says the list is the whole vault", () => {
+ const hint = linkCandidateSchemaHint(
+ buildLinkCandidateManifest(vaultWith(["alpha", "beta"]), ALL_VISIBLE),
+ );
+ expect(hint).toContain("link_candidates");
+ expect(hint).toContain("2");
+ expect(hint).not.toContain("of the");
+ });
+
+ test("a truncated manifest says how much it left out", () => {
+ const names = Array.from({ length: 9 }, (_, i) => `note-${i}`);
+ const hint = linkCandidateSchemaHint(
+ buildLinkCandidateManifest(vaultWith(names), { ...ALL_VISIBLE, limit: 3 }),
+ );
+ expect(hint).toContain("3");
+ expect(hint).toContain("9");
+ });
+
+ test("an empty vault says so rather than claiming a complete list of nothing", () => {
+ const hint = linkCandidateSchemaHint(buildLinkCandidateManifest(vaultWith([]), ALL_VISIBLE));
+ expect(hint).toMatch(/no note in this vault is available to cite/);
+ });
+});
diff --git a/tests/core/brain/origin-channel-stamp.test.ts b/tests/core/brain/origin-channel-stamp.test.ts
new file mode 100644
index 00000000..8dfaa796
--- /dev/null
+++ b/tests/core/brain/origin-channel-stamp.test.ts
@@ -0,0 +1,225 @@
+/**
+ * Unit C — the four record families that carry the origin channel.
+ *
+ * Claims pinned here:
+ * 1. Log events carry `origin_channel` beside `agent` in BOTH surfaces
+ * the appender writes — the markdown block and the JSONL sidecar —
+ * and `parseLogDay` reads it back with no parser change.
+ * 2. A caller-shaped log body cannot name the channel: a body key
+ * spelling `origin_channel` is overwritten by the server value.
+ * 3. Signals carry `origin_channel` as a SIBLING of `source_type`; the
+ * two vocabularies are not folded and `source_type` keeps its
+ * historical emit rule (absent means `live`).
+ * 4. `parseSignal` reads `origin_channel` back, so the field is not
+ * write-only.
+ * 5. Continuity records carry `originChannel` and the dedup id does NOT
+ * move: adding provenance must not re-identify existing records.
+ * 6. The caller-named note envelope resolves the channel, and
+ * `createNote` stamps it into the written frontmatter LAST, so a
+ * caller-supplied `origin_channel` key cannot win.
+ * 7. Every family stamps the explicit `unset` literal when no entry
+ * point claimed the process — never a guessed channel.
+ */
+
+import { afterEach, beforeEach, describe, expect, test } from "bun:test";
+import { mkdirSync, mkdtempSync, readFileSync, rmSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import {
+ ORIGIN_CHANNEL,
+ ORIGIN_CHANNEL_FIELD,
+ ORIGIN_CHANNEL_UNSET,
+ clearOriginChannel,
+ setOriginChannel,
+} from "../../../src/core/origin-channel.ts";
+import { appendLogEvent, parseLogDay } from "../../../src/core/brain/log.ts";
+import { parseSignal, writeSignal, type WriteSignalInput } from "../../../src/core/brain/signal.ts";
+import {
+ appendContinuityRecord,
+ buildContinuityRecord,
+} from "../../../src/core/brain/continuity/store.ts";
+import { createNote, resolveNoteTarget } from "../../../src/core/brain/notes/create-note.ts";
+
+let vault: string;
+
+beforeEach(() => {
+ vault = mkdtempSync(join(tmpdir(), "o2b-origin-channel-"));
+ mkdirSync(join(vault, "Brain", "inbox"), { recursive: true });
+ clearOriginChannel();
+});
+afterEach(() => {
+ clearOriginChannel();
+ rmSync(vault, { recursive: true, force: true });
+});
+
+function baseSignal(slug: string, overrides: Partial<WriteSignalInput> = {}): WriteSignalInput {
+ return {
+ topic: "deploy",
+ signal: "positive",
+ agent: "tester",
+ principle: `Principle for ${slug}`,
+ created_at: "2026-05-01T00:00:00Z",
+ date: "2026-05-01",
+ slug,
+ ...overrides,
+ };
+}
+
+describe("family 1 — log events", () => {
+ test("stamp the claimed channel into the markdown block and the JSONL row", () => {
+ setOriginChannel(ORIGIN_CHANNEL.cli);
+ const { logPath } = appendLogEvent(vault, {
+ timestamp: "2026-05-01T10:00:00Z",
+ eventType: "note",
+ agent: "tester",
+ body: { text: "a milestone" },
+ });
+ const markdown = readFileSync(logPath, "utf8");
+ expect(markdown).toContain(`- ${ORIGIN_CHANNEL_FIELD}: cli`);
+ const jsonl = readFileSync(logPath.replace(/\.md$/, ".jsonl"), "utf8").trim();
+ expect(JSON.parse(jsonl).payload[ORIGIN_CHANNEL_FIELD]).toBe("cli");
+ });
+
+ test("round-trip through parseLogDay with no parser change", () => {
+ setOriginChannel(ORIGIN_CHANNEL.mcpTool);
+ appendLogEvent(vault, {
+ timestamp: "2026-05-01T10:00:00Z",
+ eventType: "note",
+ body: { text: "a milestone" },
+ });
+ const { entries } = parseLogDay(vault, "2026-05-01");
+ expect(entries[0]?.body[ORIGIN_CHANNEL_FIELD]).toBe("mcp-tool");
+ });
+
+ test("a caller-shaped body cannot name the channel", () => {
+ setOriginChannel(ORIGIN_CHANNEL.cli);
+ const { logPath } = appendLogEvent(vault, {
+ timestamp: "2026-05-01T10:00:00Z",
+ eventType: "note",
+ body: { text: "a milestone", [ORIGIN_CHANNEL_FIELD]: "mcp-tool" },
+ });
+ const markdown = readFileSync(logPath, "utf8");
+ expect(markdown).toContain(`- ${ORIGIN_CHANNEL_FIELD}: cli`);
+ expect(markdown).not.toContain(`- ${ORIGIN_CHANNEL_FIELD}: mcp-tool`);
+ });
+
+ test("stamp `unset` when no entry point claimed the process", () => {
+ const { logPath } = appendLogEvent(vault, {
+ timestamp: "2026-05-01T10:00:00Z",
+ eventType: "note",
+ body: { text: "a milestone" },
+ });
+ expect(readFileSync(logPath, "utf8")).toContain(
+ `- ${ORIGIN_CHANNEL_FIELD}: ${ORIGIN_CHANNEL_UNSET}`,
+ );
+ });
+});
+
+describe("family 2 — signals", () => {
+ test("stamp the channel as a sibling of source_type, not a replacement", () => {
+ setOriginChannel(ORIGIN_CHANNEL.import);
+ const res = writeSignal(vault, baseSignal("sibling", { source_type: "session" }));
+ const text = readFileSync(res.path, "utf8");
+ expect(text).toContain(`${ORIGIN_CHANNEL_FIELD}: import`);
+ expect(text).toContain("source_type: session");
+ });
+
+ test("leave the historical source_type emit rule alone", () => {
+ setOriginChannel(ORIGIN_CHANNEL.cli);
+ const res = writeSignal(vault, baseSignal("live-default"));
+ const text = readFileSync(res.path, "utf8");
+ // `live` is still the implicit default and is still not emitted.
+ expect(text).not.toContain("source_type:");
+ expect(text).toContain(`${ORIGIN_CHANNEL_FIELD}: cli`);
+ });
+
+ test("parseSignal reads the channel back, so the field is not write-only", () => {
+ setOriginChannel(ORIGIN_CHANNEL.mcpTool);
+ const res = writeSignal(vault, baseSignal("readback"));
+ expect(parseSignal(res.path).origin_channel).toBe("mcp-tool");
+ });
+
+ test("stamp `unset` when no entry point claimed the process", () => {
+ const res = writeSignal(vault, baseSignal("unclaimed"));
+ expect(readFileSync(res.path, "utf8")).toContain(
+ `${ORIGIN_CHANNEL_FIELD}: ${ORIGIN_CHANNEL_UNSET}`,
+ );
+ });
+});
+
+describe("family 3 — continuity records", () => {
+ test("carry the claimed channel", () => {
+ setOriginChannel(ORIGIN_CHANNEL.mcpTool);
+ const record = appendContinuityRecord(vault, {
+ kind: "context_receipt",
+ createdAt: "2026-05-01T10:00:00Z",
+ payload: { note: "checkpoint" },
+ });
+ expect(record.originChannel).toBe("mcp-tool");
+ const shard = readFileSync(
+ join(vault, "Brain", "log", "continuity", "2026-05.jsonl"),
+ "utf8",
+ ).trim();
+ expect(JSON.parse(shard).originChannel).toBe("mcp-tool");
+ });
+
+ test("keep the dedup id stable across channels — provenance must not re-identify", () => {
+ setOriginChannel(ORIGIN_CHANNEL.cli);
+ const viaCli = buildContinuityRecord({
+ kind: "context_receipt",
+ createdAt: "2026-05-01T10:00:00Z",
+ payload: { note: "checkpoint" },
+ });
+ setOriginChannel(ORIGIN_CHANNEL.mcpTool);
+ const viaMcp = buildContinuityRecord({
+ kind: "context_receipt",
+ createdAt: "2026-05-01T10:00:00Z",
+ payload: { note: "checkpoint" },
+ });
+ expect(viaMcp.id).toBe(viaCli.id);
+ expect(viaMcp.originChannel).not.toBe(viaCli.originChannel);
+ });
+
+ test("stamp `unset` when no entry point claimed the process", () => {
+ const record = buildContinuityRecord({
+ kind: "context_receipt",
+ createdAt: "2026-05-01T10:00:00Z",
+ });
+ expect(record.originChannel).toBe(ORIGIN_CHANNEL_UNSET);
+ });
+});
+
+describe("family 4 — caller-named notes", () => {
+ test("the shared envelope resolves the channel for all four caller-named tools", () => {
+ setOriginChannel(ORIGIN_CHANNEL.cli);
+ expect(resolveNoteTarget(vault, "Notes/Target.md").originChannel).toBe("cli");
+ });
+
+ test("createNote stamps it into the written frontmatter", () => {
+ setOriginChannel(ORIGIN_CHANNEL.cli);
+ createNote(vault, { path: "Notes/Stamped.md", frontmatter: { title: "T" }, content: "body" });
+ const text = readFileSync(join(vault, "Notes", "Stamped.md"), "utf8");
+ expect(text).toContain(`${ORIGIN_CHANNEL_FIELD}: cli`);
+ expect(text).toContain("title: T");
+ });
+
+ test("a caller-supplied origin_channel key loses to the server value", () => {
+ setOriginChannel(ORIGIN_CHANNEL.cli);
+ createNote(vault, {
+ path: "Notes/Liar.md",
+ frontmatter: { [ORIGIN_CHANNEL_FIELD]: "mcp-tool" },
+ content: "body",
+ });
+ const text = readFileSync(join(vault, "Notes", "Liar.md"), "utf8");
+ expect(text).toContain(`${ORIGIN_CHANNEL_FIELD}: cli`);
+ expect(text).not.toContain(`${ORIGIN_CHANNEL_FIELD}: mcp-tool`);
+ });
+
+ test("stamp `unset` when no entry point claimed the process", () => {
+ createNote(vault, { path: "Notes/Unclaimed.md", content: "body" });
+ expect(readFileSync(join(vault, "Notes", "Unclaimed.md"), "utf8")).toContain(
+ `${ORIGIN_CHANNEL_FIELD}: ${ORIGIN_CHANNEL_UNSET}`,
+ );
+ });
+});
diff --git a/tests/core/brain/rollup-ladder.test.ts b/tests/core/brain/rollup-ladder.test.ts
index d2c8f2c8..b3aeb4ee 100644
--- a/tests/core/brain/rollup-ladder.test.ts
+++ b/tests/core/brain/rollup-ladder.test.ts
@@ -13,17 +13,31 @@ import {
ROLLUP_TIER,
type RollupThresholds,
} from "../../../src/core/brain/rollup-ladder.ts";
+import type { LinkCandidateManifest } from "../../../src/core/brain/notes/link-candidates.ts";
import { DEFAULT_BRAIN_CONFIG } from "../../../src/core/brain/policy.ts";
const THRESHOLDS: RollupThresholds = { fact: 5, identity: 2 };
const RUN_ID = "dream-2026-07-19-100000";
+/**
+ * The wikilink targets a fired rung offers, as a VALUE. The planner is
+ * pure and takes the manifest rather than walking a vault for one, which
+ * is exactly what lets this suite stay a pure-function suite.
+ */
+const LINK_CANDIDATES: LinkCandidateManifest = Object.freeze({
+ candidates: Object.freeze(["analytical-engine", "vector-index-storage"]),
+ total: 2,
+ truncated: false,
+ selection: "all",
+});
+
test("below threshold nothing fires and no counters move", () => {
const plan = planRollupLadder({
factCount: 4,
ledger: null,
thresholds: THRESHOLDS,
runId: RUN_ID,
+ linkCandidates: () => LINK_CANDIDATES,
});
expect(plan.fired).toBe(false);
expect(plan.entries).toHaveLength(0);
@@ -36,6 +50,7 @@ test("reaching the fact threshold fires one rollup and resets the counter", () =
ledger: null,
thresholds: THRESHOLDS,
runId: RUN_ID,
+ linkCandidates: () => LINK_CANDIDATES,
});
expect(plan.fired).toBe(true);
expect(plan.entries).toHaveLength(1);
@@ -58,12 +73,14 @@ test("a fired plan is idempotent: replaying it moves no counter", () => {
ledger: null,
thresholds: THRESHOLDS,
runId: RUN_ID,
+ linkCandidates: () => LINK_CANDIDATES,
});
const second = planRollupLadder({
factCount: 5,
ledger: first.ledger,
thresholds: THRESHOLDS,
runId: RUN_ID,
+ linkCandidates: () => LINK_CANDIDATES,
});
expect(second.fired).toBe(false);
expect(second.entries).toHaveLength(0);
@@ -77,6 +94,7 @@ test("the ladder composes: enough fact rollups cascade into an identity rollup",
ledger: null,
thresholds: { fact: 5, identity: 1 },
runId: RUN_ID,
+ linkCandidates: () => LINK_CANDIDATES,
});
expect(plan.entries).toHaveLength(2);
expect(plan.entries.map((e) => e.tier)).toEqual([ROLLUP_TIER.fact, ROLLUP_TIER.rollup]);
@@ -90,6 +108,7 @@ test("new facts beyond the last rollup re-arm the fact rung", () => {
ledger: null,
thresholds: THRESHOLDS,
runId: RUN_ID,
+ linkCandidates: () => LINK_CANDIDATES,
});
// Five more facts since the reset (10 total) crosses the threshold again.
const second = planRollupLadder({
@@ -97,6 +116,7 @@ test("new facts beyond the last rollup re-arm the fact rung", () => {
ledger: first.ledger,
thresholds: THRESHOLDS,
runId: RUN_ID,
+ linkCandidates: () => LINK_CANDIDATES,
});
expect(second.fired).toBe(true);
expect(second.entries[0]!.fromCount).toBe(5);
@@ -115,3 +135,41 @@ test("thresholds resolve from config, falling back to the named constants", () =
expect(overridden.fact).toBe(3);
expect(overridden.identity).toBe(DEFAULT_ROLLUP_IDENTITY_THRESHOLD);
});
+
+test("the link-candidate manifest is not built when no rung fires", () => {
+ // The manifest is a whole-vault walk - and, under owner-scope delivery, a
+ // frontmatter read per note. Only a FIRED rung carries one, so a run that
+ // fires nothing (the common case, and every dry run) must not pay for it.
+ let built = 0;
+ const plan = planRollupLadder({
+ factCount: 4,
+ ledger: null,
+ thresholds: THRESHOLDS,
+ runId: RUN_ID,
+ linkCandidates: () => {
+ built += 1;
+ return LINK_CANDIDATES;
+ },
+ });
+ expect(plan.fired).toBe(false);
+ expect(built).toBe(0);
+});
+
+test("two rungs firing in one pass build the manifest once", () => {
+ let built = 0;
+ const plan = planRollupLadder({
+ factCount: 5,
+ ledger: null,
+ thresholds: { fact: 5, identity: 1 },
+ runId: RUN_ID,
+ linkCandidates: () => {
+ built += 1;
+ return LINK_CANDIDATES;
+ },
+ });
+ expect(plan.entries.length).toBeGreaterThan(1);
+ expect(built).toBe(1);
+ for (const entry of plan.entries) {
+ expect(entry.envelope.link_candidates).toEqual(LINK_CANDIDATES);
+ }
+});
diff --git a/tests/core/brain/sessions/import-resume.test.ts b/tests/core/brain/sessions/import-resume.test.ts
new file mode 100644
index 00000000..a0aba78e
--- /dev/null
+++ b/tests/core/brain/sessions/import-resume.test.ts
@@ -0,0 +1,464 @@
+/**
+ * Session-import resume on the ingest checkpoint substrate
+ * (nothing-writes-silently, Unit F).
+ *
+ * Resume is not new in this codebase: `ingest/checkpoint.ts` has shipped
+ * plan-scoped, atomic, vault-identity-asserted resume for the source-ingest
+ * lane. What was missing is the sessions lane, whose only idempotency was a
+ * dedup index rebuilt from scratch on every run - re-do-and-discard, so an
+ * interrupted 50k-turn import re-read and re-hashed everything. This suite
+ * pins the extension:
+ *
+ * 1. A boundary written by the checkpoint writer reads back with the turn
+ * count, the session file it belongs to, and the file identity it was
+ * taken against.
+ * 2. An unknown `schema_version` and corrupt bytes are refused BY NAME -
+ * never a silent reset, which would masquerade completed turns as
+ * pending and re-hash them all.
+ * 3. `OSB_INGEST_NO_CHECKPOINT` makes record and read inert, the same env
+ * var and the same semantics as the ingest lane; two spellings of the
+ * opt-out would be two things to remember.
+ * 4. The checkpoint id is scoped to the import's filters, so a run that
+ * asks a different question of the same file never resumes off another
+ * run's boundary.
+ * 5. An interrupted import resumes AT the boundary: the turns before it
+ * are not re-extracted, proved by deleting a signal one of those turns
+ * wrote and showing the resumed run does not write it again.
+ * 6. A completed applied import clears its checkpoint, exactly as the
+ * ingest lane drops a drained plan's.
+ * 7. A dry run records nothing, and a checkpoint taken against a session
+ * file that no longer matches is discarded and SAID so rather than
+ * silently resuming at a turn boundary that has moved.
+ * 8. A checkpoint whose payload is not what the writer would ever have
+ * written - a non-integer or negative `turns_completed` - is discarded
+ * by name too, rather than coerced to zero and reported as "there was
+ * no checkpoint".
+ * 9. A resumed run with `recall` re-collects the turns before the boundary.
+ * Recall is accumulated in memory and committed AFTER the loop, so an
+ * interrupted run wrote none of it; skipping those turns would leave the
+ * head of the transcript permanently absent from the recall DAG while
+ * the run reported success.
+ */
+
+import { afterEach, beforeEach, describe, expect, test } from "bun:test";
+import {
+ existsSync,
+ mkdirSync,
+ mkdtempSync,
+ readdirSync,
+ readFileSync,
+ rmSync,
+ statSync,
+ unlinkSync,
+ writeFileSync,
+} from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { DEFAULT_BRAIN_CONFIG_YAML } from "../../../../src/core/brain/config-template.ts";
+import { atomicWriteFileSync } from "../../../../src/core/fs-atomic.ts";
+import { brainDirs } from "../../../../src/core/brain/paths.ts";
+import { importSession } from "../../../../src/core/brain/sessions/import.ts";
+import {
+ clearSessionCheckpoint,
+ computeSessionHeadHash,
+ computeSessionImportId,
+ readSessionCheckpoint,
+ recordSessionProgress,
+ resolveSessionResume,
+ SESSION_CHECKPOINT_TURN_INTERVAL,
+ SESSION_RESUME_DISCARD,
+ sessionCheckpointPath,
+} from "../../../../src/core/brain/sessions/checkpoint.ts";
+import { listSessionRawTurns } from "../../../../src/core/brain/session-recall.ts";
+import { RECONCILIATION_OUTCOME } from "../../../../src/core/reconciliation-report.ts";
+
+let vault: string;
+
+const NOW = new Date("2026-08-15T10:00:00Z");
+const AGENT = "@t";
+/** Explicit recall session id, so the DAG can be read back by name. */
+const RECALL_SESSION = "resume-recall-session";
+/** Enough turns to cross the boundary interval more than once. */
+const TURNS = SESSION_CHECKPOINT_TURN_INTERVAL * 2 + 100;
+const EARLY_TURN = Math.floor(SESSION_CHECKPOINT_TURN_INTERVAL / 2);
+const LATE_TURN = SESSION_CHECKPOINT_TURN_INTERVAL + 50;
+
+beforeEach(() => {
+ vault = mkdtempSync(join(tmpdir(), "o2b-session-resume-"));
+ const dirs = brainDirs(vault);
+ for (const d of [dirs.brain, dirs.inbox, dirs.processed, dirs.preferences, dirs.log]) {
+ mkdirSync(d, { recursive: true });
+ }
+ atomicWriteFileSync(join(dirs.brain, "config.yaml"), DEFAULT_BRAIN_CONFIG_YAML);
+});
+
+afterEach(() => {
+ rmSync(vault, { recursive: true, force: true });
+ delete process.env["OSB_INGEST_NO_CHECKPOINT"];
+});
+
+function marker(topic: string, principle: string): string {
+ return `@osb feedback positive topic=${topic} principle="${principle}"`;
+}
+
+/**
+ * A Claude-format transcript of `count` ASSISTANT turns. Assistant on
+ * purpose: fact extraction runs over user turns only, so the only writes
+ * this fixture can produce are the two markers it plants, which is what
+ * makes the resume assertions below unambiguous.
+ */
+function writeTranscript(
+ name: string,
+ count: number,
+ markers: ReadonlyMap<number, string>,
+): string {
+ const path = join(vault, name);
+ const lines: string[] = [];
+ for (let turn = 1; turn <= count; turn++) {
+ lines.push(
+ JSON.stringify({
+ type: "assistant",
+ parentUuid: null,
+ entrypoint: "cli",
+ uuid: `u${turn}`,
+ timestamp: "2026-08-15T09:00:00.000Z",
+ sessionId: "s1",
+ message: {
+ role: "assistant",
+ content: [{ type: "text", text: markers.get(turn) ?? `turn ${turn}` }],
+ },
+ }),
+ );
+ }
+ writeFileSync(path, lines.join("\n") + "\n", "utf8");
+ return path;
+}
+
+function standardTranscript(name = "session.jsonl"): string {
+ return writeTranscript(
+ name,
+ TURNS,
+ new Map([
+ [EARLY_TURN, marker("resume-early", "The early rule")],
+ [LATE_TURN, marker("resume-late", "The late rule")],
+ ]),
+ );
+}
+
+function inboxFiles(): string[] {
+ return readdirSync(brainDirs(vault).inbox).filter((n) => n.endsWith(".md"));
+}
+
+function headHashOf(path: string): string {
+ return computeSessionHeadHash(readFileSync(path, "utf8").split("\n")[0] ?? "");
+}
+
+function identityOf(path: string): { headHash: string; bytes: number } {
+ return { headHash: headHashOf(path), bytes: statSync(path).size };
+}
+
+/** Put arbitrary bytes where a checkpoint lives, bypassing the writer's checks. */
+function writeCheckpointPayload(importId: string, payload: Record<string, unknown>): void {
+ atomicWriteFileSync(
+ sessionCheckpointPath(vault, importId),
+ `${JSON.stringify({ schema_version: 1, import_id: importId, updated_at: NOW.toISOString(), ...payload }, null, 2)}\n`,
+ );
+}
+
+/** The checkpoint an interrupted run would have left at `turnsCompleted`. */
+function seedCheckpoint(path: string, importId: string, turnsCompleted: number): boolean {
+ return recordSessionProgress(
+ vault,
+ importId,
+ {
+ sessionFile: path,
+ headHash: headHashOf(path),
+ bytes: statSync(path).size,
+ turnsCompleted,
+ },
+ NOW,
+ );
+}
+
+describe("the session-import checkpoint substrate", () => {
+ test("a recorded boundary reads back with its turn count and file identity", () => {
+ const path = standardTranscript();
+ const importId = computeSessionImportId(path, { agent: AGENT });
+ expect(seedCheckpoint(path, importId, SESSION_CHECKPOINT_TURN_INTERVAL)).toBe(true);
+
+ const cp = readSessionCheckpoint(vault, importId);
+ expect(cp?.turns_completed).toBe(SESSION_CHECKPOINT_TURN_INTERVAL);
+ expect(cp?.session_file).toBe(path);
+ expect(cp?.head_hash).toBe(headHashOf(path));
+ expect(cp?.bytes).toBe(statSync(path).size);
+ });
+
+ test("re-recording the same boundary is a no-op rather than a rewrite", () => {
+ const path = standardTranscript();
+ const importId = computeSessionImportId(path, { agent: AGENT });
+ expect(seedCheckpoint(path, importId, SESSION_CHECKPOINT_TURN_INTERVAL)).toBe(true);
+ expect(seedCheckpoint(path, importId, SESSION_CHECKPOINT_TURN_INTERVAL)).toBe(false);
+ });
+
+ test("an unknown schema_version is refused by name, never silently reset", () => {
+ const path = standardTranscript();
+ const importId = computeSessionImportId(path, { agent: AGENT });
+ seedCheckpoint(path, importId, SESSION_CHECKPOINT_TURN_INTERVAL);
+ const file = sessionCheckpointPath(vault, importId);
+ writeFileSync(file, JSON.stringify({ schema_version: 99, turns_completed: 1 }), "utf8");
+ expect(() => readSessionCheckpoint(vault, importId)).toThrow(/schema_version/);
+ });
+
+ test("corrupt bytes are refused by name", () => {
+ const path = standardTranscript();
+ const importId = computeSessionImportId(path, { agent: AGENT });
+ seedCheckpoint(path, importId, SESSION_CHECKPOINT_TURN_INTERVAL);
+ writeFileSync(sessionCheckpointPath(vault, importId), "{ not json", "utf8");
+ expect(() => readSessionCheckpoint(vault, importId)).toThrow(/corrupt/i);
+ });
+
+ test("a negative or fractional turn count is refused rather than recorded", () => {
+ const path = standardTranscript();
+ const importId = computeSessionImportId(path, { agent: AGENT });
+ expect(() => seedCheckpoint(path, importId, -1)).toThrow(/turns_completed/);
+ expect(() => seedCheckpoint(path, importId, 1.5)).toThrow(/turns_completed/);
+ });
+
+ test("OSB_INGEST_NO_CHECKPOINT makes record and read inert, as it does for ingest", () => {
+ const path = standardTranscript();
+ const importId = computeSessionImportId(path, { agent: AGENT });
+ process.env["OSB_INGEST_NO_CHECKPOINT"] = "1";
+ expect(seedCheckpoint(path, importId, SESSION_CHECKPOINT_TURN_INTERVAL)).toBe(false);
+ expect(existsSync(sessionCheckpointPath(vault, importId))).toBe(false);
+ expect(readSessionCheckpoint(vault, importId)).toBeNull();
+ });
+
+ test("clearing reports whether there was anything to clear", () => {
+ const path = standardTranscript();
+ const importId = computeSessionImportId(path, { agent: AGENT });
+ seedCheckpoint(path, importId, SESSION_CHECKPOINT_TURN_INTERVAL);
+ expect(clearSessionCheckpoint(vault, importId)).toBe(true);
+ expect(clearSessionCheckpoint(vault, importId)).toBe(false);
+ });
+
+ test("the id is scoped to the import's question, not just the file", () => {
+ const path = standardTranscript();
+ const base = computeSessionImportId(path, { agent: AGENT });
+ expect(base).toMatch(/^[0-9a-f]{16}$/);
+ expect(computeSessionImportId(path, { agent: AGENT, filterRoles: ["user"] })).not.toBe(base);
+ expect(computeSessionImportId(path, { agent: AGENT, filterTextIncludes: "x" })).not.toBe(base);
+ expect(
+ computeSessionImportId(path, { agent: AGENT, since: new Date("2026-01-01T00:00:00Z") }),
+ ).not.toBe(base);
+ expect(computeSessionImportId(path, { agent: AGENT, recall: true })).not.toBe(base);
+ // Same question, same id - otherwise no run could ever resume.
+ expect(computeSessionImportId(path, { agent: AGENT })).toBe(base);
+ });
+});
+
+describe("an interrupted session import resumes at the turn boundary", () => {
+ test("turns before the boundary are not re-extracted", async () => {
+ const path = standardTranscript();
+ const opts = { agent: AGENT, now: NOW };
+ const first = await importSession(vault, path, opts);
+ expect(first.signals_created).toBe(2);
+
+ // The early marker's signal is removed, so a resumed run that re-read
+ // those turns would write it again - and the dedup index could not stop
+ // it, because the hash is no longer on disk. Its continued absence is
+ // the proof that the completed turns were never re-hashed.
+ const early = inboxFiles().find((n) => n.includes("resume-early"));
+ expect(early).toBeDefined();
+ unlinkSync(join(brainDirs(vault).inbox, early!));
+
+ const importId = computeSessionImportId(path, opts);
+ seedCheckpoint(path, importId, SESSION_CHECKPOINT_TURN_INTERVAL);
+
+ const resumed = await importSession(vault, path, opts);
+ expect(resumed.turns_resumed).toBe(SESSION_CHECKPOINT_TURN_INTERVAL);
+ expect(resumed.resume_discarded).toBeNull();
+ expect(resumed.signals_created).toBe(0);
+ // The late marker sits past the boundary, so it IS re-seen - and deduped
+ // against the signal the first run wrote.
+ expect(resumed.signals_deduped).toBe(1);
+ expect(inboxFiles().some((n) => n.includes("resume-early"))).toBe(false);
+ });
+
+ test("turns_scanned still counts every turn the adapter yielded", async () => {
+ const path = standardTranscript();
+ const opts = { agent: AGENT, now: NOW };
+ const importId = computeSessionImportId(path, opts);
+ seedCheckpoint(path, importId, SESSION_CHECKPOINT_TURN_INTERVAL);
+
+ const resumed = await importSession(vault, path, opts);
+ expect(resumed.turns_scanned).toBe(TURNS);
+ expect(resumed.turns_resumed).toBe(SESSION_CHECKPOINT_TURN_INTERVAL);
+ expect(resumed.census.outcome).toBe(RECONCILIATION_OUTCOME.complete);
+ });
+
+ test("a completed applied import clears its checkpoint", async () => {
+ const path = standardTranscript();
+ const opts = { agent: AGENT, now: NOW };
+ const importId = computeSessionImportId(path, opts);
+ await importSession(vault, path, opts);
+ expect(existsSync(sessionCheckpointPath(vault, importId))).toBe(false);
+ expect(readSessionCheckpoint(vault, importId)).toBeNull();
+ });
+
+ test("a dry run records no checkpoint at all", async () => {
+ const path = standardTranscript();
+ const opts = { agent: AGENT, now: NOW, dryRun: true };
+ const importId = computeSessionImportId(path, opts);
+ const result = await importSession(vault, path, opts);
+ expect(result.turns_resumed).toBe(0);
+ expect(existsSync(sessionCheckpointPath(vault, importId))).toBe(false);
+ });
+
+ test("a checkpoint taken against a different file is discarded and said so", async () => {
+ const path = standardTranscript();
+ const opts = { agent: AGENT, now: NOW };
+ const importId = computeSessionImportId(path, opts);
+ recordSessionProgress(
+ vault,
+ importId,
+ {
+ sessionFile: path,
+ // The head line of some OTHER transcript: the log this boundary was
+ // taken against is not the log on disk now.
+ headHash: computeSessionHeadHash("a completely different first line"),
+ bytes: statSync(path).size,
+ turnsCompleted: SESSION_CHECKPOINT_TURN_INTERVAL,
+ },
+ NOW,
+ );
+
+ const result = await importSession(vault, path, opts);
+ expect(result.turns_resumed).toBe(0);
+ expect(result.resume_discarded).toBe("head_changed");
+ // Nothing was skipped, so both markers are seen and written.
+ expect(result.signals_created).toBe(2);
+ });
+
+ test("a session file that shrank since the boundary is not resumed from", async () => {
+ const path = standardTranscript();
+ const opts = { agent: AGENT, now: NOW };
+ const importId = computeSessionImportId(path, opts);
+ recordSessionProgress(
+ vault,
+ importId,
+ {
+ sessionFile: path,
+ headHash: headHashOf(path),
+ bytes: statSync(path).size + 1_000,
+ turnsCompleted: SESSION_CHECKPOINT_TURN_INTERVAL,
+ },
+ NOW,
+ );
+
+ const result = await importSession(vault, path, opts);
+ expect(result.turns_resumed).toBe(0);
+ expect(result.resume_discarded).toBe("file_shrank");
+ });
+
+ test("a checkpoint payload the writer would never have written is discarded by name", async () => {
+ const path = standardTranscript();
+ const opts = { agent: AGENT, now: NOW };
+ const importId = computeSessionImportId(path, opts);
+ // A partially-flushed or externally-touched file: the count is a string,
+ // which no writer here can produce. Coerced to 0 it is indistinguishable
+ // from "there was no checkpoint" - the module's own stated anti-goal.
+ writeCheckpointPayload(importId, {
+ session_file: path,
+ head_hash: headHashOf(path),
+ bytes: statSync(path).size,
+ turns_completed: "250",
+ });
+
+ expect(resolveSessionResume(vault, importId, identityOf(path))).toEqual({
+ turns: 0,
+ discarded: SESSION_RESUME_DISCARD.payloadInvalid,
+ });
+ const result = await importSession(vault, path, opts);
+ expect(result.turns_resumed).toBe(0);
+ expect(result.resume_discarded).toBe(SESSION_RESUME_DISCARD.payloadInvalid);
+ // Nothing was skipped, so the whole transcript is imported.
+ expect(result.signals_created).toBe(2);
+ });
+
+ test("a negative turn boundary is refused rather than honoured verbatim", () => {
+ const path = standardTranscript();
+ const importId = computeSessionImportId(path, { agent: AGENT });
+ writeCheckpointPayload(importId, {
+ session_file: path,
+ head_hash: headHashOf(path),
+ bytes: statSync(path).size,
+ turns_completed: -5,
+ });
+ expect(resolveSessionResume(vault, importId, identityOf(path))).toEqual({
+ turns: 0,
+ discarded: SESSION_RESUME_DISCARD.payloadInvalid,
+ });
+ });
+
+ test("with checkpointing opted out the import neither resumes nor records", async () => {
+ const path = standardTranscript();
+ const opts = { agent: AGENT, now: NOW };
+ const importId = computeSessionImportId(path, opts);
+ seedCheckpoint(path, importId, SESSION_CHECKPOINT_TURN_INTERVAL);
+ process.env["OSB_INGEST_NO_CHECKPOINT"] = "1";
+
+ const result = await importSession(vault, path, opts);
+ expect(result.turns_resumed).toBe(0);
+ expect(result.resume_discarded).toBeNull();
+ expect(result.signals_created).toBe(2);
+ // The opt-out suppresses writes; it must not delete a checkpoint either.
+ expect(existsSync(sessionCheckpointPath(vault, importId))).toBe(true);
+ });
+});
+
+describe("a resumed import with recall keeps the recall DAG whole", () => {
+ test("the turns before the boundary still reach the recall DAG", async () => {
+ const path = standardTranscript();
+ const opts = { agent: AGENT, now: NOW, recall: true, recallSessionId: RECALL_SESSION };
+ const importId = computeSessionImportId(path, opts);
+ seedCheckpoint(path, importId, SESSION_CHECKPOINT_TURN_INTERVAL);
+
+ const resumed = await importSession(vault, path, opts);
+ expect(resumed.turns_resumed).toBe(SESSION_CHECKPOINT_TURN_INTERVAL);
+ expect(resumed.resume_discarded).toBeNull();
+ // `recallTurns` is an in-memory accumulator committed after the loop, so
+ // the interrupted run wrote NONE of it: the resumed run owes the whole
+ // transcript, not the tail past the boundary.
+ expect(resumed.recall_turns_imported).toBe(TURNS);
+
+ const stored = listSessionRawTurns(vault, RECALL_SESSION);
+ expect(stored.length).toBe(TURNS);
+ expect(stored.some((t) => t.text === "turn 1")).toBe(true);
+ expect(stored.some((t) => t.text === `turn ${TURNS}`)).toBe(true);
+ });
+
+ test("a re-collected turn still passes the run's own filters", async () => {
+ const path = standardTranscript();
+ // The fixture is assistant turns only, so a user-role filter admits none
+ // of them. Re-collection must apply that filter, not smuggle the head of
+ // the transcript in behind it.
+ const opts = {
+ agent: AGENT,
+ now: NOW,
+ recall: true,
+ recallSessionId: RECALL_SESSION,
+ filterRoles: ["user"] as const,
+ };
+ const importId = computeSessionImportId(path, opts);
+ seedCheckpoint(path, importId, SESSION_CHECKPOINT_TURN_INTERVAL);
+
+ const resumed = await importSession(vault, path, opts);
+ expect(resumed.turns_resumed).toBe(SESSION_CHECKPOINT_TURN_INTERVAL);
+ expect(resumed.recall_turns_imported).toBe(0);
+ expect(listSessionRawTurns(vault, RECALL_SESSION)).toEqual([]);
+ // The filtered count still describes only the turns this run processed,
+ // so a resumed run's counters keep meaning what they meant before.
+ expect(resumed.filtered_turns).toBe(TURNS - SESSION_CHECKPOINT_TURN_INTERVAL);
+ });
+});
diff --git a/tests/core/brain/snapshot-gate-coverage.test.ts b/tests/core/brain/snapshot-gate-coverage.test.ts
index 58959c46..b7a43d96 100644
--- a/tests/core/brain/snapshot-gate-coverage.test.ts
+++ b/tests/core/brain/snapshot-gate-coverage.test.ts
@@ -223,10 +223,18 @@ describe("the dream pass runs behind the gate, not beside it", () => {
// the same exclusion applied, and this branch reproduces it - so it
// is still a number from a tree without the units it guards, not a
// number this branch invented about itself.
+ //
+ // Re-measured a second time, in nothing-writes-silently, and for a
+ // byte the pass DOES author: every log event and every signal now
+ // carries an `origin_channel` stamp, the server-derived channel of
+ // the writing process (Unit C). The digest below is the tree with
+ // that unit and WITHOUT the snapshot gate this file guards, taken
+ // the same way as the two before it, so what it still proves is the
+ // same claim: the gate authors none of these bytes.
seedSignal("tidy");
dream(vault, { now: NOW, agentName: "tester" });
expect(dreamOutputDigest(join(vault, "Brain"))).toBe(
- "1944a3340e6aee531205e0e3042ca957b23cb67a9254d10b649d49a51e504446",
+ "345b742a592bbfd51305e16fc9e63031f88ba54371ba619e01eecff129f4103a",
);
});
diff --git a/tests/core/brain/status-maintenance-debt.test.ts b/tests/core/brain/status-maintenance-debt.test.ts
new file mode 100644
index 00000000..f2387e17
--- /dev/null
+++ b/tests/core/brain/status-maintenance-debt.test.ts
@@ -0,0 +1,235 @@
+/**
+ * `computeBrainStatus`'s derived `maintenance_debt` block
+ * (nothing-writes-silently, Unit D / t_6285e06f).
+ *
+ * Claims pinned here:
+ *
+ * 1. A vault that has never dreamed reports `status: "never_dreamed"`
+ * with `log_events_since_dream` equal to every log event ever
+ * recorded — never `0`, which would read as "no debt" rather than
+ * "no dream has ever run".
+ * 2. Once a `dream` event exists, `status` becomes `"counted"` and
+ * `log_events_since_dream` counts only events strictly AFTER the
+ * dream's own timestamp — the dream event itself is the boundary,
+ * not a countable event.
+ * 3. Events recorded before `last_dream_at` are excluded even when
+ * they share the same log day as the dream event.
+ * 4. A `--dry-run` dream emits no `dream` log event, so it moves
+ * neither `last_dream_at` nor `maintenance_debt` — the wet/dry
+ * split already in `dream.ts` is the whole clearing mechanism;
+ * there is no separate counter to reset.
+ * 5. A vault with no `Brain/` layer at all reports the same honest
+ * `never_dreamed` / zero shape as an empty-but-present Brain, not
+ * an error.
+ * 6. A log shard the process cannot read makes the figure a LOWER
+ * BOUND, and it says so (`status: "undercounted"`) rather than
+ * presenting a partial walk as a count.
+ * 7. `computeMaintenanceOverdueFlag` answers `null` for a newest day
+ * that holds no events at all - the same "unknown" it already gives
+ * an unreadable day, not `true`, which would report overdue
+ * maintenance from an empty file.
+ */
+
+import { afterEach, beforeEach, describe, expect, test } from "bun:test";
+import { mkdtempSync, rmSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import {
+ computeBrainStatus,
+ computeMaintenanceOverdueFlag,
+ MAINTENANCE_DEBT_STATUS,
+} from "../../../src/core/brain/status.ts";
+import { brainDirs } from "../../../src/core/brain/paths.ts";
+import { bootstrapBrain } from "../../../src/core/brain/init.ts";
+import { dream } from "../../../src/core/brain/dream.ts";
+import { appendLogEvent } from "../../../src/core/brain/log.ts";
+import { writeSignal } from "../../../src/core/brain/signal.ts";
+import { atomicWriteFileSync } from "../../../src/core/fs-atomic.ts";
+import { BRAIN_LOG_EVENT_KIND } from "../../../src/core/brain/types.ts";
+
+let vault: string;
+let configHome: string;
+let configPath: string;
+
+beforeEach(() => {
+ vault = mkdtempSync(join(tmpdir(), "o2b-status-maint-debt-"));
+ configHome = mkdtempSync(join(tmpdir(), "o2b-status-maint-debt-cfg-"));
+ configPath = join(configHome, "config.yaml");
+ atomicWriteFileSync(configPath, `vault: ${vault}\n`);
+ bootstrapBrain(vault, { configPath });
+});
+
+afterEach(() => {
+ rmSync(vault, { recursive: true, force: true });
+ rmSync(configHome, { recursive: true, force: true });
+});
+
+function logEvent(timestamp: string): void {
+ appendLogEvent(vault, {
+ timestamp,
+ eventType: BRAIN_LOG_EVENT_KIND.note,
+ agent: "tester",
+ body: { text: "note" },
+ });
+}
+
+/** Three same-sign signals: enough for `dream` to plan a real change. */
+function seedPromotion(topic = "maint-debt-topic"): void {
+ for (const [i, date] of ["2026-05-20", "2026-05-21", "2026-05-22"].entries()) {
+ writeSignal(vault, {
+ topic,
+ signal: "positive",
+ agent: "claude",
+ principle: `Prefer the ${topic} approach`,
+ created_at: `${date}T10:00:00Z`,
+ date,
+ slug: `${topic}-${i}`,
+ scope: "writing",
+ });
+ }
+}
+
+describe("computeBrainStatus — maintenance_debt (never dreamed)", () => {
+ test("reports never_dreamed with the true event count, not zero-debt", () => {
+ logEvent("2026-06-01T10:00:00Z");
+ logEvent("2026-06-01T11:00:00Z");
+ logEvent("2026-06-02T09:00:00Z");
+
+ const snapshot = computeBrainStatus(vault);
+ expect(snapshot.last_dream_at).toBeNull();
+ expect(snapshot.maintenance_debt.status).toBe(MAINTENANCE_DEBT_STATUS.neverDreamed);
+ expect(snapshot.maintenance_debt.log_events_since_dream).toBe(3);
+ });
+
+ test("an empty, never-dreamed Brain reports zero — not an error, not a lie", () => {
+ const snapshot = computeBrainStatus(vault);
+ expect(snapshot.maintenance_debt.status).toBe(MAINTENANCE_DEBT_STATUS.neverDreamed);
+ expect(snapshot.maintenance_debt.log_events_since_dream).toBe(0);
+ });
+});
+
+describe("computeBrainStatus — maintenance_debt (dreamed)", () => {
+ test("counts only events strictly after the dream boundary", () => {
+ logEvent("2026-06-01T10:00:00Z"); // before dream — excluded
+ appendLogEvent(vault, {
+ timestamp: "2026-06-01T12:00:00Z",
+ eventType: BRAIN_LOG_EVENT_KIND.dream,
+ agent: "tester",
+ body: {},
+ });
+ logEvent("2026-06-01T12:00:00Z"); // same instant as the dream — NOT after it, excluded
+ logEvent("2026-06-01T13:00:00Z"); // after dream, same day — included
+ logEvent("2026-06-02T09:00:00Z"); // after dream, later day — included
+
+ const snapshot = computeBrainStatus(vault);
+ expect(snapshot.last_dream_at).toBe("2026-06-01T12:00:00Z");
+ expect(snapshot.maintenance_debt.status).toBe(MAINTENANCE_DEBT_STATUS.counted);
+ expect(snapshot.maintenance_debt.log_events_since_dream).toBe(2);
+ });
+
+ test("zero events since the last dream reports counted, not never_dreamed", () => {
+ appendLogEvent(vault, {
+ timestamp: "2026-06-01T12:00:00Z",
+ eventType: BRAIN_LOG_EVENT_KIND.dream,
+ agent: "tester",
+ body: {},
+ });
+
+ const snapshot = computeBrainStatus(vault);
+ expect(snapshot.maintenance_debt.status).toBe(MAINTENANCE_DEBT_STATUS.counted);
+ expect(snapshot.maintenance_debt.log_events_since_dream).toBe(0);
+ });
+});
+
+describe("computeBrainStatus — maintenance_debt (Brain absent)", () => {
+ test("reports the same honest never_dreamed shape as an empty Brain", () => {
+ const bareVault = mkdtempSync(join(tmpdir(), "o2b-status-maint-debt-bare-"));
+ try {
+ const snapshot = computeBrainStatus(bareVault);
+ expect(snapshot.present).toBe(false);
+ expect(snapshot.maintenance_debt.status).toBe(MAINTENANCE_DEBT_STATUS.neverDreamed);
+ expect(snapshot.maintenance_debt.log_events_since_dream).toBe(0);
+ } finally {
+ rmSync(bareVault, { recursive: true, force: true });
+ }
+ });
+});
+
+describe("a log day the reader cannot fully parse is a lower bound, and says so", () => {
+ /** A shard whose lines are not JSON: the reader reports them, and counts none. */
+ function writeUnparsableShard(date: string): void {
+ atomicWriteFileSync(
+ join(brainDirs(vault).log, `${date}.broken.jsonl`),
+ "not json at all\nnor this\n",
+ );
+ }
+
+ test("maintenance_debt reports undercounted rather than presenting a partial walk as a count", () => {
+ logEvent("2026-06-01T10:00:00Z");
+ writeUnparsableShard("2026-06-01");
+
+ const snapshot = computeBrainStatus(vault);
+ expect(snapshot.maintenance_debt.status).toBe(MAINTENANCE_DEBT_STATUS.undercounted);
+ // The events it COULD read are still reported - the number is a floor,
+ // not a refusal to answer.
+ expect(snapshot.maintenance_debt.log_events_since_dream).toBe(1);
+ });
+
+ test("a fully readable log still reports counted", () => {
+ appendLogEvent(vault, {
+ timestamp: "2026-06-01T12:00:00Z",
+ eventType: BRAIN_LOG_EVENT_KIND.dream,
+ agent: "tester",
+ body: {},
+ });
+ logEvent("2026-06-01T13:00:00Z");
+ expect(computeBrainStatus(vault).maintenance_debt.status).toBe(MAINTENANCE_DEBT_STATUS.counted);
+ });
+
+ test("the overdue flag answers unknown for a newest day that holds no readable event", () => {
+ writeUnparsableShard("2026-06-03");
+ // `true` here would report overdue maintenance out of a file nobody
+ // could read; `false` would claim a dream that may be in the bytes
+ // that did not parse.
+ expect(computeMaintenanceOverdueFlag(vault)).toBeNull();
+ });
+
+ test("the overdue flag still answers when the newest day is readable", () => {
+ logEvent("2026-06-04T09:00:00Z");
+ expect(computeMaintenanceOverdueFlag(vault)).toBe(true);
+ appendLogEvent(vault, {
+ timestamp: "2026-06-04T10:00:00Z",
+ eventType: BRAIN_LOG_EVENT_KIND.dream,
+ agent: "tester",
+ body: {},
+ });
+ expect(computeMaintenanceOverdueFlag(vault)).toBe(false);
+ });
+});
+
+describe("computeBrainStatus — dry-run dream moves nothing in the ledger", () => {
+ test("a --dry-run dream leaves last_dream_at and maintenance_debt untouched", () => {
+ seedPromotion();
+ logEvent("2026-05-19T09:00:00Z");
+ const before = computeBrainStatus(vault);
+ expect(before.last_dream_at).toBeNull();
+
+ dream(vault, { now: new Date("2026-05-23T12:00:00Z"), dryRun: true });
+
+ const after = computeBrainStatus(vault);
+ expect(after.last_dream_at).toBeNull();
+ expect(after.maintenance_debt).toEqual(before.maintenance_debt);
+ });
+
+ test("the same seeded state DOES move the ledger on a wet run", () => {
+ seedPromotion();
+ logEvent("2026-05-19T09:00:00Z");
+
+ dream(vault, { now: new Date("2026-05-23T12:00:00Z") });
+
+ const after = computeBrainStatus(vault);
+ expect(after.last_dream_at).not.toBeNull();
+ expect(after.maintenance_debt.status).toBe(MAINTENANCE_DEBT_STATUS.counted);
+ });
+});
diff --git a/tests/core/brain/sync-lockfile.test.ts b/tests/core/brain/sync-lockfile.test.ts
index d3f4989e..3513894d 100644
--- a/tests/core/brain/sync-lockfile.test.ts
+++ b/tests/core/brain/sync-lockfile.test.ts
@@ -189,17 +189,19 @@ describe("every module that takes a lock is under a scanned root", () => {
/**
* Declared lock sites, each with the vault-relative root
* {@link lockScanRoots} reaches it through. Verified against the path
- * builders: every entry resolves under `Brain/` except the two ingest
- * artefacts, which live in the derived-store directory.
+ * builders: every entry resolves under `Brain/` except the three that
+ * live in the derived-store directory - the shared checkpoint substrate
+ * (both the ingest and the session-import lanes lock through it), the
+ * ingest content manifest, and the session import ledger.
*/
const DECLARED: ReadonlyMap<string, string> = new Map([
["core/brain/architect/generate.ts", "Brain"],
+ ["core/brain/checkpoint-store.ts", ".open-second-brain"],
["core/brain/continuity/store.ts", "Brain"],
["core/brain/diagnostics.ts", "Brain"],
["core/brain/git/store.ts", "Brain"],
["core/brain/health/remediation.ts", "Brain"],
["core/brain/idempotency-ledger.ts", "Brain"],
- ["core/brain/ingest/checkpoint.ts", ".open-second-brain"],
["core/brain/ingest/content-manifest.ts", ".open-second-brain"],
["core/brain/lineage/ledger.ts", "Brain"],
["core/brain/preference-txn.ts", "Brain"],
diff --git a/tests/core/brain/write-batch.test.ts b/tests/core/brain/write-batch.test.ts
index 67d33992..803b0664 100644
--- a/tests/core/brain/write-batch.test.ts
+++ b/tests/core/brain/write-batch.test.ts
@@ -8,6 +8,21 @@
* and no disk write happens. Note operations reuse the exact
* create-note safety envelope (path traversal, Brain machinery root,
* vault-scope exclusions) and atomic-write semantics.
+ *
+ * The nothing-writes-silently wave (unit B) adds two refusals to that
+ * vocabulary, pinned below:
+ *
+ * 1. `target_unreadable` - a note that exists and cannot be read is no
+ * longer projected as an empty note, for update and for append. The
+ * refusal names the path and the reason, and the file is untouched.
+ * 1b. `target_frontmatter_lossy` - the same refusal for the other thing
+ * the parser can report: a frontmatter line it could not express and
+ * dropped. The read-modify-write re-serialises the parsed map, so
+ * writing would delete that line and report success.
+ * 2. `blank_overwrite_refused` - an update may not replace a body that
+ * carries text with a blank one unless `allowEmpty` says so.
+ * Whitespace-only is blank; a create of a genuinely new empty note
+ * is not an update and is unaffected.
*/
import { afterEach, beforeEach, describe, expect, test } from "bun:test";
@@ -18,6 +33,7 @@ import {
mkdtempSync,
readFileSync,
rmSync,
+ statSync,
writeFileSync,
} from "node:fs";
import { tmpdir } from "node:os";
@@ -210,6 +226,165 @@ describe("applyWriteBatch note operations", () => {
}
});
+ test("update_note refuses a blank body over a note that has one", () => {
+ const abs = seedNote("Notes/Doc.md", "worth keeping", "title: Doc");
+ const before = readFileSync(abs, "utf8");
+ try {
+ applyWriteBatch(vault, [{ kind: "update_note", path: "Notes/Doc.md", body: "" }]);
+ throw new Error("expected throw");
+ } catch (err) {
+ expect(err).toBeInstanceOf(WriteBatchError);
+ expect((err as WriteBatchError).code).toBe("blank_overwrite_refused");
+ expect((err as WriteBatchError).index).toBe(0);
+ // The refusal names the note, not just the operation index.
+ expect((err as WriteBatchError).message).toContain("Notes/Doc.md");
+ expect((err as WriteBatchError).details).toMatchObject({ path: "Notes/Doc.md" });
+ }
+ expect(readFileSync(abs, "utf8")).toBe(before);
+ });
+
+ test("a whitespace-only body is a blank one: the parser would trim it away", () => {
+ const abs = seedNote("Notes/Doc.md", "worth keeping", "title: Doc");
+ const before = readFileSync(abs, "utf8");
+ expect(() =>
+ applyWriteBatch(vault, [{ kind: "update_note", path: "Notes/Doc.md", body: " \n\t\n" }]),
+ ).toThrow(WriteBatchError);
+ expect(readFileSync(abs, "utf8")).toBe(before);
+ });
+
+ test("allowEmpty clears the body deliberately, keeping the frontmatter", () => {
+ const abs = seedNote("Notes/Doc.md", "worth keeping", "title: Doc");
+ applyWriteBatch(vault, [
+ { kind: "update_note", path: "Notes/Doc.md", body: "", allowEmpty: true },
+ ]);
+ const md = readFileSync(abs, "utf8");
+ expect(md).toContain("title: Doc");
+ expect(md).not.toContain("worth keeping");
+ });
+
+ test("a frontmatter-only update never trips the blank guard", () => {
+ seedNote("Notes/Doc.md", "worth keeping", "title: Doc");
+ applyWriteBatch(vault, [
+ { kind: "update_note", path: "Notes/Doc.md", frontmatter: { status: "final" } },
+ ]);
+ const md = readFileSync(join(vault, "Notes/Doc.md"), "utf8");
+ expect(md).toContain("worth keeping");
+ expect(md).toContain("status: final");
+ });
+
+ test("creating a genuinely new empty note is untouched by the guard", () => {
+ const res = applyWriteBatch(vault, [
+ { kind: "create_note", path: "Notes/Empty.md", frontmatter: { title: "Empty" } },
+ ]);
+ expect(res.applied).toBe(1);
+ const md = readFileSync(join(vault, "Notes/Empty.md"), "utf8");
+ expect(md).toContain("title: Empty");
+ });
+
+ test("an update on a note that cannot be read is refused, and nothing is written", () => {
+ // A directory standing where a note should be: `existsSync` says
+ // yes and `readFileSync` raises EISDIR - an existing target the
+ // process cannot read, without depending on permission bits.
+ const abs = join(vault, "Notes/Doc.md");
+ mkdirSync(abs, { recursive: true });
+ try {
+ applyWriteBatch(vault, [{ kind: "update_note", path: "Notes/Doc.md", body: "replacement" }]);
+ throw new Error("expected throw");
+ } catch (err) {
+ expect(err).toBeInstanceOf(WriteBatchError);
+ expect((err as WriteBatchError).code).toBe("target_unreadable");
+ expect((err as WriteBatchError).message).toContain("Notes/Doc.md");
+ // The reason travels with the refusal; a bare "could not read"
+ // would leave the operator guessing between a permission bit and
+ // a missing mount.
+ expect(String((err as WriteBatchError).details["reason"])).toContain("EISDIR");
+ }
+ expect(statSync(abs).isDirectory()).toBe(true);
+ });
+
+ test("an append onto a note that cannot be read is refused, and nothing is written", () => {
+ const abs = join(vault, "Notes/Doc.md");
+ mkdirSync(abs, { recursive: true });
+ try {
+ applyWriteBatch(vault, [{ kind: "append_note", path: "Notes/Doc.md", content: "more" }]);
+ throw new Error("expected throw");
+ } catch (err) {
+ expect(err).toBeInstanceOf(WriteBatchError);
+ expect((err as WriteBatchError).code).toBe("target_unreadable");
+ }
+ expect(statSync(abs).isDirectory()).toBe(true);
+ });
+
+ test("an update on a note whose frontmatter the scanner cannot express is refused", () => {
+ // The line scanner has no branch for a continuation line, so it drops
+ // it and says so. A read-modify-write re-serialises the PARSED map, so
+ // proceeding would delete that line from disk and answer updated: true.
+ const abs = seedNote("Notes/Odd.md", "body", "title: Odd\n continued indent line");
+ const before = readFileSync(abs, "utf8");
+ try {
+ applyWriteBatch(vault, [
+ { kind: "update_note", path: "Notes/Odd.md", frontmatter: { reviewed: "2026-08-23" } },
+ ]);
+ throw new Error("expected throw");
+ } catch (err) {
+ expect(err).toBeInstanceOf(WriteBatchError);
+ expect((err as WriteBatchError).code).toBe("target_frontmatter_lossy");
+ expect((err as WriteBatchError).message).toContain("Notes/Odd.md");
+ expect(String((err as WriteBatchError).details["reason"])).toContain("continued indent line");
+ }
+ expect(readFileSync(abs, "utf8")).toBe(before);
+ });
+
+ test("an append onto such a note is refused too - it rewrites the same frontmatter", () => {
+ const abs = seedNote("Notes/Odd.md", "body", "title: Odd\n continued indent line");
+ const before = readFileSync(abs, "utf8");
+ expect(() =>
+ applyWriteBatch(vault, [{ kind: "append_note", path: "Notes/Odd.md", content: "more" }]),
+ ).toThrow(WriteBatchError);
+ expect(readFileSync(abs, "utf8")).toBe(before);
+ });
+
+ test("frontmatter the scanner does express is updated as before", () => {
+ // The refusal is scoped to a line the parser reported dropping: a
+ // comment and a block list are both consumed, not dropped.
+ const abs = seedNote("Notes/Fine.md", "body", "# a comment\ntags:\n - one\ntitle: Fine");
+ applyWriteBatch(vault, [
+ { kind: "update_note", path: "Notes/Fine.md", frontmatter: { status: "final" } },
+ ]);
+ const md = readFileSync(abs, "utf8");
+ expect(md).toContain("status: final");
+ expect(md).toContain("title: Fine");
+ });
+
+ test("an unreadable-by-permission note is refused rather than blanked", () => {
+ // Running as root bypasses the permission bits this case needs.
+ if (typeof process.getuid === "function" && process.getuid() === 0) return;
+ const abs = seedNote("Notes/Secret.md", "the body that must survive", "title: Secret");
+ const before = readFileSync(abs, "utf8");
+ chmodSync(abs, 0o000);
+ try {
+ expect(() =>
+ applyWriteBatch(vault, [{ kind: "update_note", path: "Notes/Secret.md", body: "new" }]),
+ ).toThrow(WriteBatchError);
+ } finally {
+ chmodSync(abs, 0o600);
+ }
+ expect(readFileSync(abs, "utf8")).toBe(before);
+ });
+
+ test("an unreadable target at op 1 aborts the batch before op 0 lands", () => {
+ const kept = seedNote("Notes/A.md", "unchanged", "title: A");
+ const before = readFileSync(kept, "utf8");
+ mkdirSync(join(vault, "Notes/B.md"), { recursive: true });
+ expect(() =>
+ applyWriteBatch(vault, [
+ { kind: "update_note", path: "Notes/A.md", body: "would change" },
+ { kind: "update_note", path: "Notes/B.md", body: "y" },
+ ]),
+ ).toThrow(WriteBatchError);
+ expect(readFileSync(kept, "utf8")).toBe(before);
+ });
+
test("a mid-write failure leaves the target byte-identical", () => {
// Running as root bypasses filesystem permission bits, so the
// read-only-directory injection cannot force a write failure there.
diff --git a/tests/core/origin-channel.test.ts b/tests/core/origin-channel.test.ts
new file mode 100644
index 00000000..a810f688
--- /dev/null
+++ b/tests/core/origin-channel.test.ts
@@ -0,0 +1,103 @@
+/**
+ * Unit C — the server-derived origin channel, at its resolver.
+ *
+ * Claims pinned here:
+ * 1. The vocabulary is closed at three members and spelled exactly
+ * `mcp-tool` / `cli` / `import`; `ORIGIN_CHANNELS` is that list.
+ * 2. A value outside the vocabulary is refused at the setter, by name,
+ * so a bad channel can never be claimed for a process.
+ * 3. Reading the channel before any entry point claimed it FAILS LOUD:
+ * `resolveOriginChannel` throws and the message names the function
+ * and the entry points that are supposed to have called the setter.
+ * 4. The record-writer read never throws and never guesses: it returns
+ * the explicit `unset` literal, which serializes as `unset`.
+ * 5. `unset` is not a member of the channel vocabulary — a reader can
+ * tell "no entry point claimed this process" from every real answer.
+ * 6. A claim is readable by both accessors; `clearOriginChannel` puts
+ * the process back to unclaimed (entry-point re-entry / test
+ * isolation).
+ * 7. Re-claiming is allowed and last-claim-wins: one process can re-enter
+ * the entry point (the CLI test harness calls `main()` many times).
+ */
+
+import { afterEach, describe, expect, test } from "bun:test";
+
+import {
+ ORIGIN_CHANNEL,
+ ORIGIN_CHANNEL_UNSET,
+ ORIGIN_CHANNELS,
+ OriginChannelUnsetError,
+ clearOriginChannel,
+ isOriginChannel,
+ originChannelStamp,
+ resolveOriginChannel,
+ setOriginChannel,
+} from "../../src/core/origin-channel.ts";
+
+afterEach(() => {
+ clearOriginChannel();
+});
+
+describe("the origin-channel vocabulary", () => {
+ test("is closed at three members with exact spellings", () => {
+ expect([...ORIGIN_CHANNELS]).toEqual(["mcp-tool", "cli", "import"]);
+ expect(ORIGIN_CHANNEL.mcpTool).toBe("mcp-tool");
+ expect(ORIGIN_CHANNEL.cli).toBe("cli");
+ expect(ORIGIN_CHANNEL.import).toBe("import");
+ });
+
+ test("`unset` is outside it, so absence of a claim is distinguishable", () => {
+ expect(ORIGIN_CHANNEL_UNSET).toBe("unset");
+ expect(isOriginChannel(ORIGIN_CHANNEL_UNSET)).toBe(false);
+ expect(ORIGIN_CHANNELS).not.toContain(ORIGIN_CHANNEL_UNSET as never);
+ });
+
+ test("refuses a value outside the vocabulary at the setter, naming it", () => {
+ expect(() => setOriginChannel("http" as never)).toThrow(/"http"/);
+ expect(() => setOriginChannel("http" as never)).toThrow(/mcp-tool, cli, import/);
+ // The refusal left the process unclaimed rather than half-claimed.
+ expect(originChannelStamp()).toBe(ORIGIN_CHANNEL_UNSET);
+ });
+});
+
+describe("reading before any entry point claimed the process", () => {
+ test("the strict read fails loud and names itself", () => {
+ expect(() => resolveOriginChannel()).toThrow(OriginChannelUnsetError);
+ expect(() => resolveOriginChannel()).toThrow(/resolveOriginChannel/);
+ });
+
+ test("the strict refusal names the entry points that owe the claim", () => {
+ let message = "";
+ try {
+ resolveOriginChannel();
+ } catch (err) {
+ message = (err as Error).message;
+ }
+ expect(message).toContain("setOriginChannel");
+ });
+
+ test("the record-writer read returns the explicit unset literal instead", () => {
+ expect(originChannelStamp()).toBe(ORIGIN_CHANNEL_UNSET);
+ });
+});
+
+describe("a claimed process", () => {
+ test("reports the claim through both accessors", () => {
+ setOriginChannel(ORIGIN_CHANNEL.cli);
+ expect(resolveOriginChannel()).toBe(ORIGIN_CHANNEL.cli);
+ expect(originChannelStamp()).toBe(ORIGIN_CHANNEL.cli);
+ });
+
+ test("last claim wins, so a re-entered entry point is not a fault", () => {
+ setOriginChannel(ORIGIN_CHANNEL.cli);
+ setOriginChannel(ORIGIN_CHANNEL.import);
+ expect(resolveOriginChannel()).toBe(ORIGIN_CHANNEL.import);
+ });
+
+ test("clearing puts it back to unclaimed", () => {
+ setOriginChannel(ORIGIN_CHANNEL.mcpTool);
+ clearOriginChannel();
+ expect(originChannelStamp()).toBe(ORIGIN_CHANNEL_UNSET);
+ expect(() => resolveOriginChannel()).toThrow(OriginChannelUnsetError);
+ });
+});
diff --git a/tests/core/reconciliation-report.test.ts b/tests/core/reconciliation-report.test.ts
new file mode 100644
index 00000000..cf65f88f
--- /dev/null
+++ b/tests/core/reconciliation-report.test.ts
@@ -0,0 +1,264 @@
+/**
+ * The shared reconciliation vocabulary (nothing-writes-silently, shared
+ * substrate). Pins:
+ *
+ * 1. `buildReconciliationReport` accepts a valid `{ attempted, found,
+ * missing }` triple and returns a report exposing exactly those values.
+ * 2. The returned report and its `missing` array are both frozen, and the
+ * array is a defensive copy - mutating the caller's input array after
+ * the call does not change the stored report.
+ * 3. `deriveReconciliationOutcome` reads `complete` when nothing is
+ * missing.
+ * 4. It reads `partial` when `missing` names at least one key and no
+ * recorded claim contradicts the report.
+ * 5. It reads `contradicted` when a recorded claim's `attempted` disagrees
+ * with the report's own `attempted`.
+ * 6. It reads `contradicted` when a recorded claim's `found` disagrees
+ * with the report's own `found` - even when `missing` is empty, because
+ * a claim contradicting a clean report is still a contradiction.
+ * 7. It is deterministic: the same report (and the same recorded claim)
+ * derives the same outcome on repeated calls.
+ * 8. Refusal: a non-integer `attempted` (NaN and a fractional value) is
+ * named `invalid_count` and names the offending field.
+ * 9. Refusal: a non-integer `found` (NaN and a fractional value) is named
+ * `invalid_count` and names the offending field.
+ * 10. Refusal: a negative `attempted` or `found` is named `invalid_count`.
+ * 11. Refusal: `found` greater than `attempted` is named
+ * `found_exceeds_attempted`, independent of `missing`.
+ * 12. Refusal: `attempted !== found + missing.length` is named
+ * `count_mismatch` - the missing keys do not account for the gap.
+ * 13. Refusal: a duplicate key inside `missing` is named
+ * `duplicate_missing_key` and the error names the offending key.
+ * 14. There is no constructor that accepts a bare missing COUNT in place
+ * of the named array - `ReconciliationReportInput["missing"]` is a
+ * `ReadonlyArray<string>`, never a `number` (a compile-time claim
+ * checked here by a runtime probe: a report built from N distinct
+ * generated keys carries exactly those N keys back, never a count).
+ * 15. `RECONCILIATION_OUTCOME` is frozen, `RECONCILIATION_OUTCOMES` lists
+ * every one of its values with no duplicates, and
+ * `isReconciliationOutcome` accepts every declared value and rejects
+ * non-member strings, non-strings, and `undefined`.
+ */
+
+import { describe, expect, test } from "bun:test";
+
+import {
+ buildReconciliationReport,
+ deriveReconciliationOutcome,
+ isReconciliationOutcome,
+ RECONCILIATION_OUTCOME,
+ RECONCILIATION_OUTCOMES,
+ ReconciliationReportError,
+ type ReconciliationReport,
+} from "../../src/core/reconciliation-report.ts";
+
+describe("buildReconciliationReport", () => {
+ test("returns a report exposing the validated triple", () => {
+ const report = buildReconciliationReport({
+ attempted: 5,
+ found: 3,
+ missing: ["a", "b"],
+ });
+ expect(report.attempted).toBe(5);
+ expect(report.found).toBe(3);
+ expect(report.missing).toEqual(["a", "b"]);
+ });
+
+ test("the report and its missing array are frozen", () => {
+ const report = buildReconciliationReport({ attempted: 1, found: 0, missing: ["only"] });
+ expect(Object.isFrozen(report)).toBe(true);
+ expect(Object.isFrozen(report.missing)).toBe(true);
+ });
+
+ test("missing is copied defensively - mutating the input after the call is inert", () => {
+ const input = ["a", "b"];
+ const report = buildReconciliationReport({ attempted: 3, found: 1, missing: input });
+ input.push("c");
+ input[0] = "mutated";
+ expect(report.missing).toEqual(["a", "b"]);
+ });
+
+ test("carries exactly the named keys back, never a count in their place", () => {
+ const keys = ["alpha", "beta", "gamma"];
+ const report = buildReconciliationReport({ attempted: 3, found: 0, missing: keys });
+ expect(report.missing).toEqual(keys);
+ expect(typeof report.missing).not.toBe("number");
+ });
+
+ describe("refusals", () => {
+ test("NaN attempted is invalid_count naming the field", () => {
+ expect(() =>
+ buildReconciliationReport({ attempted: Number.NaN, found: 0, missing: [] }),
+ ).toThrow(ReconciliationReportError);
+ try {
+ buildReconciliationReport({ attempted: Number.NaN, found: 0, missing: [] });
+ throw new Error("expected throw");
+ } catch (err) {
+ expect(err).toBeInstanceOf(ReconciliationReportError);
+ expect((err as ReconciliationReportError).code).toBe("invalid_count");
+ expect((err as ReconciliationReportError).details["field"]).toBe("attempted");
+ }
+ });
+
+ test("NaN found is invalid_count naming the field", () => {
+ try {
+ buildReconciliationReport({ attempted: 1, found: Number.NaN, missing: [] });
+ throw new Error("expected throw");
+ } catch (err) {
+ expect(err).toBeInstanceOf(ReconciliationReportError);
+ expect((err as ReconciliationReportError).code).toBe("invalid_count");
+ expect((err as ReconciliationReportError).details["field"]).toBe("found");
+ }
+ });
+
+ test("a fractional attempted is invalid_count", () => {
+ try {
+ buildReconciliationReport({ attempted: 2.5, found: 0, missing: [] });
+ throw new Error("expected throw");
+ } catch (err) {
+ expect(err).toBeInstanceOf(ReconciliationReportError);
+ expect((err as ReconciliationReportError).code).toBe("invalid_count");
+ }
+ });
+
+ test("a fractional found is invalid_count", () => {
+ try {
+ buildReconciliationReport({ attempted: 2, found: 1.5, missing: [] });
+ throw new Error("expected throw");
+ } catch (err) {
+ expect(err).toBeInstanceOf(ReconciliationReportError);
+ expect((err as ReconciliationReportError).code).toBe("invalid_count");
+ }
+ });
+
+ test("a negative attempted is invalid_count", () => {
+ try {
+ buildReconciliationReport({ attempted: -1, found: 0, missing: [] });
+ throw new Error("expected throw");
+ } catch (err) {
+ expect(err).toBeInstanceOf(ReconciliationReportError);
+ expect((err as ReconciliationReportError).code).toBe("invalid_count");
+ }
+ });
+
+ test("a negative found is invalid_count", () => {
+ try {
+ buildReconciliationReport({ attempted: 1, found: -1, missing: [] });
+ throw new Error("expected throw");
+ } catch (err) {
+ expect(err).toBeInstanceOf(ReconciliationReportError);
+ expect((err as ReconciliationReportError).code).toBe("invalid_count");
+ }
+ });
+
+ test("found greater than attempted is found_exceeds_attempted", () => {
+ try {
+ buildReconciliationReport({ attempted: 2, found: 5, missing: [] });
+ throw new Error("expected throw");
+ } catch (err) {
+ expect(err).toBeInstanceOf(ReconciliationReportError);
+ expect((err as ReconciliationReportError).code).toBe("found_exceeds_attempted");
+ }
+ });
+
+ test("found exceeding attempted is refused even when missing would balance it", () => {
+ // found=5 attempted=2 missing=[] never reaches the accounting check;
+ // the nonsensical count is refused first, by its own name.
+ try {
+ buildReconciliationReport({ attempted: 2, found: 5, missing: [] });
+ throw new Error("expected throw");
+ } catch (err) {
+ expect((err as ReconciliationReportError).code).toBe("found_exceeds_attempted");
+ }
+ });
+
+ test("attempted not accounted for by found + missing.length is count_mismatch", () => {
+ try {
+ buildReconciliationReport({ attempted: 5, found: 2, missing: ["a"] }); // 2+1 != 5
+ throw new Error("expected throw");
+ } catch (err) {
+ expect(err).toBeInstanceOf(ReconciliationReportError);
+ expect((err as ReconciliationReportError).code).toBe("count_mismatch");
+ }
+ });
+
+ test("a duplicate missing key is duplicate_missing_key, naming the key", () => {
+ try {
+ buildReconciliationReport({ attempted: 4, found: 2, missing: ["dup", "dup"] });
+ throw new Error("expected throw");
+ } catch (err) {
+ expect(err).toBeInstanceOf(ReconciliationReportError);
+ expect((err as ReconciliationReportError).code).toBe("duplicate_missing_key");
+ expect((err as ReconciliationReportError).details["key"]).toBe("dup");
+ }
+ });
+ });
+});
+
+describe("RECONCILIATION_OUTCOME vocabulary", () => {
+ test("the values object is frozen", () => {
+ expect(Object.isFrozen(RECONCILIATION_OUTCOME)).toBe(true);
+ });
+
+ test("the membership list is frozen and has no duplicates", () => {
+ expect(Object.isFrozen(RECONCILIATION_OUTCOMES)).toBe(true);
+ expect(new Set(RECONCILIATION_OUTCOMES).size).toBe(RECONCILIATION_OUTCOMES.length);
+ });
+
+ test("the guard accepts every declared value", () => {
+ for (const value of Object.values(RECONCILIATION_OUTCOME)) {
+ expect(isReconciliationOutcome(value)).toBe(true);
+ }
+ });
+
+ test("the guard rejects non-members", () => {
+ expect(isReconciliationOutcome("unknown-outcome")).toBe(false);
+ expect(isReconciliationOutcome("")).toBe(false);
+ expect(isReconciliationOutcome(undefined)).toBe(false);
+ expect(isReconciliationOutcome(null)).toBe(false);
+ expect(isReconciliationOutcome(42)).toBe(false);
+ });
+});
+
+describe("deriveReconciliationOutcome", () => {
+ function report(
+ overrides: Partial<{ attempted: number; found: number; missing: string[] }> = {},
+ ) {
+ const attempted = overrides.attempted ?? 3;
+ const missing = overrides.missing ?? [];
+ const found = overrides.found ?? attempted - missing.length;
+ return buildReconciliationReport({ attempted, found, missing });
+ }
+
+ test("complete when nothing is missing", () => {
+ const r = report({ attempted: 3, missing: [] });
+ expect(deriveReconciliationOutcome(r)).toBe(RECONCILIATION_OUTCOME.complete);
+ });
+
+ test("partial when missing names at least one key and no claim contradicts it", () => {
+ const r = report({ attempted: 3, missing: ["x"] });
+ expect(deriveReconciliationOutcome(r)).toBe(RECONCILIATION_OUTCOME.partial);
+ });
+
+ test("contradicted when a recorded claim's attempted disagrees", () => {
+ const r = report({ attempted: 3, missing: [] });
+ expect(deriveReconciliationOutcome(r, { attempted: 4 })).toBe(
+ RECONCILIATION_OUTCOME.contradicted,
+ );
+ });
+
+ test("contradicted when a recorded claim's found disagrees, even on a clean report", () => {
+ const r: ReconciliationReport = report({ attempted: 3, missing: [] });
+ expect(r.missing.length).toBe(0);
+ expect(deriveReconciliationOutcome(r, { found: 0 })).toBe(RECONCILIATION_OUTCOME.contradicted);
+ });
+
+ test("is deterministic across repeated calls", () => {
+ const r = report({ attempted: 5, missing: ["a", "b"] });
+ const claim = { attempted: 5 };
+ const first = deriveReconciliationOutcome(r, claim);
+ const second = deriveReconciliationOutcome(r, claim);
+ expect(first).toBe(second);
+ expect(first).toBe(RECONCILIATION_OUTCOME.partial);
+ });
+});
diff --git a/tests/core/search/embedder-record-audit.test.ts b/tests/core/search/embedder-record-audit.test.ts
new file mode 100644
index 00000000..36ecc607
--- /dev/null
+++ b/tests/core/search/embedder-record-audit.test.ts
@@ -0,0 +1,189 @@
+/**
+ * The record-vs-data embedder audit (nothing-writes-silently, unit G).
+ *
+ * `index_state` records the dimension an index was written under, and
+ * until now nothing compared that record against the width the index
+ * ACTUALLY stores. A meta row claiming 8 over 4-wide vectors was
+ * invisible: the ABI stamp compares the record against this build, and
+ * `staleEmbeddings` compares the rows against the CONFIGURED model -
+ * neither compares the record against the data it describes.
+ *
+ * The claims pinned here:
+ *
+ * 1. A store whose record matches its vectors audits `complete`, with
+ * the stored width and the vec0 declared width both accounted for.
+ * 2. A record contradicting the stored width audits `contradicted` in
+ * the wave's shared reconciliation vocabulary, and `missing` NAMES
+ * the observation that disagrees rather than reporting a count.
+ * 3. That state is distinct from ABI drift: a tampered record produces
+ * both, and they say different things - one that the record
+ * disagrees with this build, one that it disagrees with the data.
+ * 4. An index that records no dimension, or holds no vectors to
+ * compare, is UNRECORDED - the same distinction
+ * `contradictedAbiFields` already draws for a null recorded token.
+ * 5. An absent index is unrecorded and names the path.
+ * 6. The contradiction reaches `search check` as a warning and a
+ * recommendation naming the rebuild, and the JSON carries the
+ * audit in every state.
+ */
+
+import { afterEach, beforeEach, expect, test } from "bun:test";
+import { mkdirSync, writeFileSync } from "node:fs";
+import { dirname } from "node:path";
+
+import { RECONCILIATION_OUTCOME } from "../../../src/core/reconciliation-report.ts";
+import { indexCheck, indexVault } from "../../../src/core/search/indexer.ts";
+import {
+ EMBEDDING_ABI_FIX_COMMAND,
+ EMBEDDING_DIMENSION_STATE_KEY,
+ Store,
+} from "../../../src/core/search/store.ts";
+import { readEmbedderRecordCensusSync } from "../../../src/core/search/store/embedder-audit.ts";
+import type { ResolvedEmbeddingConfig } from "../../../src/core/search/types.ts";
+import { startFakeHttp, type FakeHttp } from "../../helpers/fake-http.ts";
+import { runCli } from "../../helpers/run-cli.ts";
+import { createTempVault, makeConfig, writeMd } from "../../helpers/search-fixtures.ts";
+import { sqliteVecLoadable } from "../../helpers/sqlite-vec.ts";
+
+let vault: string;
+let dbPath: string;
+let cleanup: () => void;
+let server: FakeHttp;
+
+beforeEach(async () => {
+ const v = createTempVault("embedder-record");
+ vault = v.vault;
+ dbPath = v.dbPath;
+ cleanup = v.cleanup;
+ server = await startFakeHttp();
+});
+
+afterEach(async () => {
+ cleanup();
+ await server.close();
+});
+
+function cfg(semantic: Partial<ResolvedEmbeddingConfig> = {}) {
+ return makeConfig({
+ vault,
+ dbPath,
+ semantic: {
+ enabled: true,
+ provider: "openai-compat",
+ baseUrl: server.url,
+ model: "fake-model",
+ apiKey: "test-key",
+ dimension: 4,
+ timeoutMs: 5_000,
+ concurrency: 2,
+ batchSize: 8,
+ costGateUsd: 0,
+ maxRetries: 3,
+ ...semantic,
+ },
+ });
+}
+
+/** Mutate recorded state behind the stamp's back, as another build would. */
+async function tamper(edit: (store: Store) => void): Promise<void> {
+ const store = await Store.open(cfg(), { mode: "write" });
+ edit(store);
+ await store.close();
+}
+
+async function indexWithVectors(): Promise<void> {
+ writeMd(vault, "a.md", "# A\n\nA note with vectors.");
+ await indexVault(cfg(), { embeddings: true });
+}
+
+test("a record that matches the vectors it describes audits complete", async () => {
+ if (!sqliteVecLoadable()) return;
+ await indexWithVectors();
+
+ const census = readEmbedderRecordCensusSync(dbPath);
+ expect(census.verdict).toBe("audited");
+ if (census.verdict !== "audited") return;
+ expect(census.recordedDimension).toBe(4);
+ expect(census.storedDimensions).toEqual([4]);
+ expect(census.vecDeclaredWidth).toBe(4);
+ expect(census.outcome).toBe(RECONCILIATION_OUTCOME.complete);
+ // Both observations - the stored rows and the vec0 declaration - were
+ // compared, and the reconciliation accounts for each one.
+ expect(census.reconciliation.attempted).toBe(2);
+ expect(census.reconciliation.found).toBe(2);
+ expect(census.reconciliation.missing).toEqual([]);
+});
+
+test("a record contradicting the stored width names the observation that disagrees", async () => {
+ if (!sqliteVecLoadable()) return;
+ await indexWithVectors();
+ await tamper((s) => s.setState(EMBEDDING_DIMENSION_STATE_KEY, "8"));
+
+ const census = readEmbedderRecordCensusSync(dbPath);
+ expect(census.verdict).toBe("audited");
+ if (census.verdict !== "audited") return;
+ expect(census.recordedDimension).toBe(8);
+ expect(census.outcome).toBe(RECONCILIATION_OUTCOME.contradicted);
+ // A count would say two observations went unaccounted for. The keys
+ // say WHICH, which is the whole point of the shared vocabulary.
+ expect(census.reconciliation.missing).toContain("embeddings.dimension=4");
+ expect(census.reconciliation.found).toBe(0);
+});
+
+test("the contradiction is a different statement from ABI drift", async () => {
+ if (!sqliteVecLoadable()) return;
+ await indexWithVectors();
+ await tamper((s) => s.setState(EMBEDDING_DIMENSION_STATE_KEY, "8"));
+
+ const report = await indexCheck(cfg());
+ // The record disagrees with this build...
+ expect(report.embeddingAbi.map((m) => m.field)).toContain(EMBEDDING_DIMENSION_STATE_KEY);
+ // ...and, separately, with the data it claims to describe.
+ expect(report.embedderRecord.verdict).toBe("audited");
+ if (report.embedderRecord.verdict !== "audited") return;
+ expect(report.embedderRecord.outcome).toBe(RECONCILIATION_OUTCOME.contradicted);
+ expect(report.warnings.some((w) => w.includes("contradict that record"))).toBe(true);
+ expect(
+ report.recommendations.some(
+ (r) => r.includes("contradict that record") && r.includes(EMBEDDING_ABI_FIX_COMMAND),
+ ),
+ ).toBe(true);
+});
+
+test("an index with nothing stored to compare is unrecorded, never a clean audit", async () => {
+ writeMd(vault, "a.md", "# A\n\nA note with no vectors.");
+ await indexVault(makeConfig({ vault, dbPath }));
+
+ const census = readEmbedderRecordCensusSync(dbPath);
+ expect(census.verdict).toBe("unrecorded");
+ if (census.verdict !== "unrecorded") return;
+ expect(census.reason.length).toBeGreaterThan(0);
+});
+
+test("an absent index is unrecorded and names the path", () => {
+ const census = readEmbedderRecordCensusSync(dbPath);
+ expect(census.verdict).toBe("unrecorded");
+ if (census.verdict !== "unrecorded") return;
+ expect(census.reason).toContain(dbPath);
+});
+
+test("a file that is not an index is unrecorded, not audited", () => {
+ mkdirSync(dirname(dbPath), { recursive: true });
+ writeFileSync(dbPath, "not a database");
+ expect(readEmbedderRecordCensusSync(dbPath).verdict).toBe("unrecorded");
+});
+
+test("`o2b search check --json` carries the audit in every state", async () => {
+ writeMd(vault, "a.md", "# A\n\nA note.");
+ await indexVault(makeConfig({ vault, dbPath }));
+
+ const out = await runCli(["search", "check", "--json", "--vault", vault, "--db", dbPath]);
+ expect(out.returncode).toBe(0);
+ const parsed = JSON.parse(out.stdout) as Record<string, unknown>;
+ const audit = parsed["embedder_record"] as Record<string, unknown>;
+ expect(audit["verdict"]).toBe("unrecorded");
+ expect(typeof audit["reason"]).toBe("string");
+
+ const human = await runCli(["search", "check", "--vault", vault, "--db", dbPath]);
+ expect(human.stdout).toContain("embedder_record:");
+});
diff --git a/tests/core/search/pending-vectors.test.ts b/tests/core/search/pending-vectors.test.ts
new file mode 100644
index 00000000..f784c433
--- /dev/null
+++ b/tests/core/search/pending-vectors.test.ts
@@ -0,0 +1,188 @@
+/**
+ * The MEASURED pending-vector count on `search check`
+ * (nothing-writes-silently, unit A).
+ *
+ * The claims pinned here:
+ *
+ * 1. `countChunksWithoutEmbeddings` answers the same anti-join
+ * `findChunksWithoutEmbeddings` walks, as a count, without
+ * materialising a single chunk body.
+ * 2. An absent index reports the census as UNRECORDED and names the
+ * path - never as a pending count of zero, which would read as a
+ * fully-embedded vault.
+ * 3. A file at the index path that is not a readable index reports
+ * unrecorded too, with the reason the open gave.
+ * 4. A fully-embedded vault emits NO reindex recommendation. This is
+ * the fix: the branch it replaces had no term for whether vectors
+ * already existed, so a healthy vault was told to "compute the
+ * first vectors".
+ * 5. An index whose chunks carry no vector at all still gets that
+ * first-vectors recommendation.
+ * 6. A partially embedded index names the measured count and points
+ * at `o2b search vector-backfill`, whose dry run is the surface
+ * that prices the work. The report itself does no cost arithmetic.
+ * 7. The unrecorded census recommends building the index and says the
+ * count is unrecorded rather than zero.
+ * 8. Both `o2b search check` shapes render the field in every state.
+ */
+
+import { afterEach, beforeEach, expect, test } from "bun:test";
+import { Database } from "bun:sqlite";
+import { mkdirSync, writeFileSync } from "node:fs";
+import { dirname } from "node:path";
+
+import { indexCheck, indexVault } from "../../../src/core/search/indexer.ts";
+import {
+ countChunksWithoutEmbeddings,
+ findChunksWithoutEmbeddings,
+} from "../../../src/core/search/store/chunks.ts";
+import { peekPendingVectorsSync } from "../../../src/core/search/store/counts.ts";
+import type { ResolvedEmbeddingConfig } from "../../../src/core/search/types.ts";
+import { startFakeHttp, type FakeHttp } from "../../helpers/fake-http.ts";
+import { runCli } from "../../helpers/run-cli.ts";
+import { createTempVault, makeConfig, writeMd } from "../../helpers/search-fixtures.ts";
+import { sqliteVecLoadable } from "../../helpers/sqlite-vec.ts";
+
+let vault: string;
+let dbPath: string;
+let cleanup: () => void;
+let server: FakeHttp;
+
+beforeEach(async () => {
+ const v = createTempVault("pending-vectors");
+ vault = v.vault;
+ dbPath = v.dbPath;
+ cleanup = v.cleanup;
+ server = await startFakeHttp();
+});
+
+afterEach(async () => {
+ cleanup();
+ await server.close();
+});
+
+function cfg(semantic: Partial<ResolvedEmbeddingConfig> = {}) {
+ return makeConfig({
+ vault,
+ dbPath,
+ semantic: {
+ enabled: true,
+ provider: "openai-compat",
+ baseUrl: server.url,
+ model: "fake-model",
+ apiKey: "test-key",
+ dimension: 4,
+ timeoutMs: 5_000,
+ concurrency: 2,
+ batchSize: 8,
+ costGateUsd: 0,
+ maxRetries: 3,
+ ...semantic,
+ },
+ });
+}
+
+test("the count agrees with the row walk it replaces", async () => {
+ writeMd(vault, "a.md", "# A\n\nFirst note.");
+ writeMd(vault, "b.md", "# B\n\nSecond note.");
+ await indexVault(makeConfig({ vault, dbPath }));
+
+ const db = new Database(dbPath, { readonly: true });
+ try {
+ const rows = findChunksWithoutEmbeddings(db);
+ expect(rows.length).toBeGreaterThan(0);
+ expect(countChunksWithoutEmbeddings(db)).toBe(rows.length);
+ } finally {
+ db.close();
+ }
+});
+
+test("an absent index is unrecorded and names the path, never a pending count of zero", () => {
+ const peek = peekPendingVectorsSync(dbPath);
+ expect(peek.kind).toBe("absent");
+});
+
+test("a file that is not a readable index is unrecorded with the reason the open gave", () => {
+ mkdirSync(dirname(dbPath), { recursive: true });
+ writeFileSync(dbPath, "not a database");
+ const peek = peekPendingVectorsSync(dbPath);
+ expect(peek.kind).toBe("unreadable");
+});
+
+test("a fully embedded vault emits no reindex recommendation", async () => {
+ if (!sqliteVecLoadable()) return;
+ writeMd(vault, "a.md", "# A\n\nA note with vectors.");
+ await indexVault(cfg(), { embeddings: true });
+
+ const report = await indexCheck(cfg());
+ expect(report.pendingVectors.verdict).toBe("measured");
+ if (report.pendingVectors.verdict === "measured") {
+ expect(report.pendingVectors.pending).toBe(0);
+ expect(report.pendingVectors.chunks).toBeGreaterThan(0);
+ }
+ expect(report.recommendations).toEqual([]);
+});
+
+test("an index whose chunks carry no vector at all is told to compute the first vectors", async () => {
+ if (!sqliteVecLoadable()) return;
+ writeMd(vault, "a.md", "# A\n\nA note with no vectors.");
+ await indexVault(cfg());
+
+ const report = await indexCheck(cfg());
+ expect(report.pendingVectors.verdict).toBe("measured");
+ if (report.pendingVectors.verdict === "measured") {
+ expect(report.pendingVectors.pending).toBe(report.pendingVectors.chunks);
+ }
+ expect(report.recommendations.some((r) => r.includes("compute the first vectors"))).toBe(true);
+});
+
+test("a partially embedded index names the count and the verb that prices the work", async () => {
+ if (!sqliteVecLoadable()) return;
+ writeMd(vault, "a.md", "# A\n\nA note with vectors.");
+ await indexVault(cfg(), { embeddings: true });
+ writeMd(vault, "b.md", "# B\n\nA note indexed without them.");
+ await indexVault(cfg());
+
+ const report = await indexCheck(cfg());
+ expect(report.pendingVectors.verdict).toBe("measured");
+ if (report.pendingVectors.verdict !== "measured") return;
+ const { pending, chunks } = report.pendingVectors;
+ expect(pending).toBeGreaterThan(0);
+ expect(pending).toBeLessThan(chunks);
+
+ const line = report.recommendations.find((r) => r.includes("o2b search vector-backfill"));
+ expect(line).toBeDefined();
+ expect(line!).toContain(String(pending));
+ expect(line!).toContain(String(chunks));
+ // The report prices nothing itself: the dry run is where a dollar
+ // figure comes from, and a second estimator here could disagree with it.
+ expect(line!).not.toContain("$");
+ expect(report.recommendations.some((r) => r.includes("compute the first vectors"))).toBe(false);
+});
+
+test("an index that could not be read recommends building it and says so", async () => {
+ if (!sqliteVecLoadable()) return;
+ writeMd(vault, "a.md", "# A\n\nA note.");
+
+ const report = await indexCheck(cfg());
+ expect(report.pendingVectors.verdict).toBe("unrecorded");
+ if (report.pendingVectors.verdict === "unrecorded") {
+ expect(report.pendingVectors.reason.length).toBeGreaterThan(0);
+ }
+ expect(report.recommendations.some((r) => r.includes("unrecorded"))).toBe(true);
+});
+
+test("both `o2b search check` shapes render the census in every state", async () => {
+ writeMd(vault, "a.md", "# A\n\nA note.");
+ await indexVault(makeConfig({ vault, dbPath }));
+
+ const json = await runCli(["search", "check", "--json", "--vault", vault, "--db", dbPath]);
+ expect(json.returncode).toBe(0);
+ const parsed = JSON.parse(json.stdout) as Record<string, unknown>;
+ const census = parsed["pending_vectors"] as Record<string, unknown>;
+ expect(census["verdict"]).toBe("measured");
+ expect(census["pending"]).toBe(census["chunks"]);
+
+ const human = await runCli(["search", "check", "--vault", vault, "--db", dbPath]);
+ expect(human.stdout).toContain("pending_vectors:");
+});
diff --git a/tests/core/search/visibility-honesty-finding.test.ts b/tests/core/search/visibility-honesty-finding.test.ts
new file mode 100644
index 00000000..01122834
--- /dev/null
+++ b/tests/core/search/visibility-honesty-finding.test.ts
@@ -0,0 +1,93 @@
+/**
+ * The visibility honesty finding on `search check`
+ * (nothing-writes-silently, unit H, form B).
+ *
+ * The claims pinned here:
+ *
+ * 1. A vault with no `visibility:` frontmatter anywhere gets no
+ * finding at all - the field is absent, not a zero count.
+ * 2. A vault with at least one `visibility:`-tagged page gets the
+ * finding, and its counts are exactly
+ * `excludedCallableVisibilitySurfaces().length` /
+ * `callableVisibilitySurfaces().length` - never a hand-written
+ * number that could drift from the registry backing it.
+ * 3. An index that does not exist yet gets no finding (nothing was
+ * measured, so nothing is claimed).
+ * 4. The denominator counts CALLABLE surfaces only: the registry's
+ * `index_store` row is a structural fact about the index's own
+ * storage, and counting it as a surface an operator could call
+ * inflates a number reported to operators by one.
+ */
+
+import { afterEach, beforeEach, expect, test } from "bun:test";
+
+import { indexCheck, indexVault } from "../../../src/core/search/indexer.ts";
+import {
+ callableVisibilitySurfaces,
+ excludedCallableVisibilitySurfaces,
+ excludedVisibilitySurfaces,
+ VISIBILITY_SURFACE_KIND,
+ VISIBILITY_SURFACE_REGISTRY,
+} from "../../../src/core/search/visibility-surface-registry.ts";
+import { createTempVault, makeConfig, writeMd } from "../../helpers/search-fixtures.ts";
+
+let vault: string;
+let dbPath: string;
+let cleanup: () => void;
+
+beforeEach(() => {
+ const v = createTempVault("visibility-honesty");
+ vault = v.vault;
+ dbPath = v.dbPath;
+ cleanup = v.cleanup;
+});
+
+afterEach(() => {
+ cleanup();
+});
+
+test("a vault that never uses visibility: gets no finding at all", async () => {
+ writeMd(vault, "a.md", "# A\n\nAn ordinary note.");
+ await indexVault(makeConfig({ vault, dbPath }));
+
+ const report = await indexCheck(makeConfig({ vault, dbPath }));
+ expect(report.visibilityHonesty).toBeUndefined();
+});
+
+test("a vault with one visibility-tagged page gets the finding, counted from the registry", async () => {
+ writeMd(vault, "a.md", "---\nvisibility: private\n---\n\n# A\n\nA private page.");
+ await indexVault(makeConfig({ vault, dbPath }));
+
+ const report = await indexCheck(makeConfig({ vault, dbPath }));
+ expect(report.visibilityHonesty).toBeDefined();
+ expect(report.visibilityHonesty?.excludedSurfaceCount).toBe(
+ excludedCallableVisibilitySurfaces().length,
+ );
+ expect(report.visibilityHonesty?.totalSurfaceCount).toBe(callableVisibilitySurfaces().length);
+ // Not a trivial zero-of-zero: the registry actually carries excluded rows.
+ expect(report.visibilityHonesty!.excludedSurfaceCount).toBeGreaterThan(0);
+});
+
+test("the denominator leaves out the row the registry itself calls not a surface", async () => {
+ writeMd(vault, "a.md", "---\nvisibility: private\n---\n\n# A\n\nA private page.");
+ await indexVault(makeConfig({ vault, dbPath }));
+
+ const indexStoreRows = VISIBILITY_SURFACE_REGISTRY.filter(
+ (row) => row.kind === VISIBILITY_SURFACE_KIND.indexStore,
+ );
+ expect(indexStoreRows).toHaveLength(1);
+
+ const report = await indexCheck(makeConfig({ vault, dbPath }));
+ expect(report.visibilityHonesty?.totalSurfaceCount).toBe(
+ VISIBILITY_SURFACE_REGISTRY.length - indexStoreRows.length,
+ );
+ // ...and it is an excluded row, so it would have moved both counts.
+ expect(report.visibilityHonesty?.excludedSurfaceCount).toBe(
+ excludedVisibilitySurfaces().length - indexStoreRows.length,
+ );
+});
+
+test("an index that does not exist yet gets no finding", async () => {
+ const report = await indexCheck(makeConfig({ vault, dbPath }));
+ expect(report.visibilityHonesty).toBeUndefined();
+});
diff --git a/tests/core/search/visibility-tag-presence.test.ts b/tests/core/search/visibility-tag-presence.test.ts
new file mode 100644
index 00000000..26452b6a
--- /dev/null
+++ b/tests/core/search/visibility-tag-presence.test.ts
@@ -0,0 +1,108 @@
+/**
+ * The visibility-tag presence peek `search check`'s honesty finding
+ * gates on (nothing-writes-silently, unit H).
+ *
+ * The claims pinned here:
+ *
+ * 1. A vault with no `visibility:` frontmatter anywhere reports absent,
+ * read straight off the built index - no page is opened again.
+ * 2. A vault with one `visibility:`-tagged page reports present.
+ * 3. Merely mentioning the WORD "visibility" (no colon, no key shape)
+ * does not flip the peek - the pattern requires the key phrase.
+ * 4. The heuristic is charged honestly for what it cannot tell apart:
+ * a page with no frontmatter block puts its own first content
+ * chunk at `chunk_index = 0`, so a body line that happens to spell
+ * the exact phrase `visibility:` reads as tagged. This is the
+ * documented false-positive shape, not a defect to close.
+ * 5. An absent index reports `absent`, never `false` folded into a
+ * read - the two are different claims and stay distinguishable.
+ * 6. A file at the index path that will not open reports `unreadable`
+ * with the open's own reason.
+ */
+
+import { afterEach, beforeEach, expect, test } from "bun:test";
+import { mkdirSync, writeFileSync } from "node:fs";
+import { dirname } from "node:path";
+
+import { indexVault } from "../../../src/core/search/indexer.ts";
+import { peekVisibilityTagPresence } from "../../../src/core/search/store/visibility-tag.ts";
+import { createTempVault, makeConfig, writeMd } from "../../helpers/search-fixtures.ts";
+
+let vault: string;
+let dbPath: string;
+let cleanup: () => void;
+
+beforeEach(() => {
+ const v = createTempVault("visibility-tag-presence");
+ vault = v.vault;
+ dbPath = v.dbPath;
+ cleanup = v.cleanup;
+});
+
+afterEach(() => {
+ cleanup();
+});
+
+test("a vault with no visibility frontmatter anywhere reads absent", async () => {
+ writeMd(vault, "a.md", "# A\n\nFirst note, no frontmatter at all.");
+ writeMd(vault, "b.md", "---\ntitle: B\n---\n\n# B\n\nTagged with an unrelated key.");
+ await indexVault(makeConfig({ vault, dbPath }));
+
+ const peek = peekVisibilityTagPresence(dbPath);
+ expect(peek.kind).toBe("read");
+ if (peek.kind === "read") expect(peek.value).toBe(false);
+});
+
+test("a vault with one visibility-tagged page reads present", async () => {
+ writeMd(vault, "a.md", "# A\n\nFirst note, no frontmatter at all.");
+ writeMd(vault, "b.md", "---\nvisibility: private\n---\n\n# B\n\nA private page.");
+ await indexVault(makeConfig({ vault, dbPath }));
+
+ const peek = peekVisibilityTagPresence(dbPath);
+ expect(peek.kind).toBe("read");
+ if (peek.kind === "read") expect(peek.value).toBe(true);
+});
+
+test("the word 'visibility' in body prose, with no 'visibility:' key text, does not flip the peek", async () => {
+ writeMd(
+ vault,
+ "a.md",
+ "# A\n\nThis paragraph discusses page visibility and how visible a UI element is to a user.",
+ );
+ await indexVault(makeConfig({ vault, dbPath }));
+
+ const peek = peekVisibilityTagPresence(dbPath);
+ expect(peek.kind).toBe("read");
+ if (peek.kind === "read") expect(peek.value).toBe(false);
+});
+
+test("a 'visibility:' phrase in body prose (no frontmatter block) still flips the peek, by design", async () => {
+ // The documented imprecision is not scoped to frontmatter VALUES: the
+ // heuristic reads chunk_index 0 wholesale, and a page with no
+ // frontmatter block puts its own first content chunk there. A body
+ // line spelling the exact key phrase is charged as a false positive
+ // for the same reason a quoted title would be - pinned here as the
+ // shape of the tradeoff, not as an oversight to close.
+ writeMd(
+ vault,
+ "a.md",
+ "# A\n\nThis paragraph discusses visibility: how visible a UI element is to a user.",
+ );
+ await indexVault(makeConfig({ vault, dbPath }));
+
+ const peek = peekVisibilityTagPresence(dbPath);
+ expect(peek.kind).toBe("read");
+ if (peek.kind === "read") expect(peek.value).toBe(true);
+});
+
+test("an absent index reads absent, never a false folded into a read", () => {
+ const peek = peekVisibilityTagPresence(dbPath);
+ expect(peek.kind).toBe("absent");
+});
+
+test("a file that is not a readable index reads unreadable with the open's reason", () => {
+ mkdirSync(dirname(dbPath), { recursive: true });
+ writeFileSync(dbPath, "not a database");
+ const peek = peekVisibilityTagPresence(dbPath);
+ expect(peek.kind).toBe("unreadable");
+});
diff --git a/tests/core/state/surfaces.test.ts b/tests/core/state/surfaces.test.ts
index d7b3a60c..97d0b042 100644
--- a/tests/core/state/surfaces.test.ts
+++ b/tests/core/state/surfaces.test.ts
@@ -50,7 +50,9 @@ import {
captureDecisionLogPath,
captureWatermarkPath,
} from "../../../src/core/brain/paths.ts";
+import { deadLetterDir } from "../../../src/core/brain/dead-letter.ts";
import { checkpointPath } from "../../../src/core/brain/ingest/checkpoint.ts";
+import { sessionCheckpointPath } from "../../../src/core/brain/sessions/checkpoint.ts";
import { manifestPath as ingestManifestPath } from "../../../src/core/brain/ingest/content-manifest.ts";
import { sessionLedgerPath } from "../../../src/core/brain/sessions/discover.ts";
import { secretsDir } from "../../../src/core/brain/secrets/store.ts";
@@ -232,6 +234,8 @@ const RESOLVER_BINDINGS: ReadonlyArray<readonly [StateSurfaceId, (vault: string)
["secret_custody", (v) => secretsDir(v)],
["ingest_content_manifest", (v) => ingestManifestPath(v)],
["ingest_checkpoints", (v) => dirname(checkpointPath(v, "0f1e2d3c"))],
+ ["session_import_checkpoints", (v) => dirname(sessionCheckpointPath(v, "0f1e2d3c"))],
+ ["write_dead_letters", (v) => deadLetterDir(v)],
["session_import_ledger", (v) => sessionLedgerPath(v)],
["install_manifest", (v) => installManifestPath(v)],
["hook_audit", (v) => hookAuditDir(v)],
diff --git a/tests/core/write-binding.test.ts b/tests/core/write-binding.test.ts
index 52376702..55493535 100644
--- a/tests/core/write-binding.test.ts
+++ b/tests/core/write-binding.test.ts
@@ -20,9 +20,9 @@
* 3. THE AUTHORITY IS THE CONFIG FILE, NOT THE CALLER'S CLAIM. This
* system has no credential: agent identity is an environment
* variable, else a config key, else the literal `agent`, and
- * twenty-two MCP tool schemas across fifteen modules accept a
- * caller-supplied `agent` string that
- * overrides it verbatim. A fence keyed to that would be bypassed by
+ * twenty-five MCP tool schemas accept a caller-supplied `agent`
+ * string that overrides it verbatim (the count is pinned as an
+ * equality by `tests/core/architecture/origin-channel-census.test.ts`). A fence keyed to that would be bypassed by
* passing a different string, so the binding never reads it - and
* that is asserted here by passing a different one and by moving
* the environment variable underneath the same call.
@@ -75,12 +75,24 @@ const PREFIX = "Projects";
* only show the two agree; a change to the shared write envelope every
* caller-named write funnels through moves BOTH arms identically and the
* comparison stays green. Only a fixed pre-feature reference can see it.
+ *
+ * It saw one, on purpose. `origin_channel` is the server-derived channel
+ * of the process that CREATED the note (Unit C of the
+ * nothing-writes-silently wave), stamped by `createNote` and by the batch
+ * `create_note` arm. `unset` is the explicit literal for "no entry point
+ * claimed this process" - a test harness is not one - and the suite
+ * preload unclaims before every test so it cannot depend on file order.
+ *
+ * Its POSITION in the `outside` note is the other half of the record: the
+ * stamp sits where creation put it, ahead of the `tag` a later
+ * `update_note` merged in. Update and append do not re-stamp, so this
+ * golden also pins that a mutation cannot overwrite the creating channel.
*/
const GOLDEN_ABSENT_BINDING_BYTES: Readonly<Record<string, string>> = Object.freeze({
/** `createNote`, then the batch `update_note` and `append_note` arms. */
- outside: "---\ntitle: T\ntag: x\n---\n\nbody\n\nmore\n",
+ outside: "---\ntitle: T\norigin_channel: unset\ntag: x\n---\n\nbody\n\nmore\n",
/** The batch `create_note` arm. */
- inside: "---\n---\n\ninside\n",
+ inside: "---\norigin_channel: unset\n---\n\ninside\n",
});
const AGENT_ENV = "VAULT_AGENT_NAME";
diff --git a/tests/mcp/agent-scope-matrix.test.ts b/tests/mcp/agent-scope-matrix.test.ts
index 3182bb89..2b19921d 100644
--- a/tests/mcp/agent-scope-matrix.test.ts
+++ b/tests/mcp/agent-scope-matrix.test.ts
@@ -326,7 +326,7 @@ const REASONS_REACHING_OWNER_CONTENT: ReadonlySet<string> = new Set([REASON.owne
*/
const PROBE_ENTRY_COUNT = 99;
const PROBE_RECIPE_COUNT = 226;
-const PROBE_TWO_SIDED_COUNT = 31;
+const PROBE_TWO_SIDED_COUNT = 32;
/**
* `brain_trigger` read arguments, with the queue populated first.
@@ -604,7 +604,11 @@ const NON_CONTENT: ReadonlyArray<ProbeEntry> = [
{
name: "brain_design_note",
calls: [
- { args: { topic: "probe topic" }, reason: REASON.ownerlessLane },
+ // The grounding stores carry no `owner:`, but the plan envelope's
+ // `link_candidates` names Brain artifact ids - so this recipe DOES
+ // reach owner-taggable content and is filtered, rather than being
+ // unable to name it.
+ { args: { topic: "probe topic" }, reason: REASON.ownerFiltered },
{
args: {
topic: "probe topic",
diff --git a/tests/mcp/brain-context.test.ts b/tests/mcp/brain-context.test.ts
index 7e825757..f57cb11e 100644
--- a/tests/mcp/brain-context.test.ts
+++ b/tests/mcp/brain-context.test.ts
@@ -21,6 +21,8 @@ import { bootstrapBrain } from "../../src/core/brain/init.ts";
import { brainActivePath } from "../../src/core/brain/paths.ts";
import { writePreference } from "../../src/core/brain/preference.ts";
import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
+import { appendLogEvent } from "../../src/core/brain/log.ts";
+import { BRAIN_LOG_EVENT_KIND } from "../../src/core/brain/types.ts";
let tmp: string;
let vault: string;
@@ -195,3 +197,66 @@ describe("brain_context tool — Brain present", () => {
expect(second["generated_at"]).toBe(first["generated_at"]);
});
});
+
+/**
+ * `maintenance_overdue` (nothing-writes-silently, Unit D / t_6285e06f):
+ * a cheap proxy bounded to the single newest `Brain/log/` day, never a
+ * full history scan, since `brain_context` is always-loaded.
+ */
+describe("brain_context tool — maintenance_overdue (bounded, newest shard only)", () => {
+ beforeEach(() => {
+ bootstrapBrain(vault, { configPath });
+ });
+
+ test("no log history yet -> null (nothing to be overdue about)", async () => {
+ const server = new MCPServer({ vault, configPath });
+ await initialize(server);
+ const out = await callContext(server);
+ expect(out["maintenance_overdue"]).toBeNull();
+ });
+
+ test("newest day holds events but no dream -> true", async () => {
+ appendLogEvent(vault, {
+ timestamp: "2026-06-01T10:00:00Z",
+ eventType: BRAIN_LOG_EVENT_KIND.note,
+ agent: "tester",
+ body: { text: "note" },
+ });
+ const server = new MCPServer({ vault, configPath });
+ await initialize(server);
+ const out = await callContext(server);
+ expect(out["maintenance_overdue"]).toBe(true);
+ });
+
+ test("newest day holds a dream event -> false", async () => {
+ appendLogEvent(vault, {
+ timestamp: "2026-06-01T10:00:00Z",
+ eventType: BRAIN_LOG_EVENT_KIND.dream,
+ agent: "tester",
+ body: {},
+ });
+ const server = new MCPServer({ vault, configPath });
+ await initialize(server);
+ const out = await callContext(server);
+ expect(out["maintenance_overdue"]).toBe(false);
+ });
+
+ test("an older day's dream does not clear a newer day with no dream (bounded to newest only)", async () => {
+ appendLogEvent(vault, {
+ timestamp: "2026-06-01T10:00:00Z",
+ eventType: BRAIN_LOG_EVENT_KIND.dream,
+ agent: "tester",
+ body: {},
+ });
+ appendLogEvent(vault, {
+ timestamp: "2026-06-02T10:00:00Z",
+ eventType: BRAIN_LOG_EVENT_KIND.note,
+ agent: "tester",
+ body: { text: "note" },
+ });
+ const server = new MCPServer({ vault, configPath });
+ await initialize(server);
+ const out = await callContext(server);
+ expect(out["maintenance_overdue"]).toBe(true);
+ });
+});
diff --git a/tests/mcp/brain-update-append-note.test.ts b/tests/mcp/brain-update-append-note.test.ts
index 19b77447..f3df75f6 100644
--- a/tests/mcp/brain-update-append-note.test.ts
+++ b/tests/mcp/brain-update-append-note.test.ts
@@ -4,6 +4,14 @@
* atomic write-batch core (kernel 2). They reuse the create-note safety
* envelope and refuse a missing target with a typed error mapped to
* INVALID_PARAMS.
+ *
+ * The nothing-writes-silently wave (unit B) adds the `allow_empty`
+ * argument to `brain_update_note` and three refusals pinned here: a
+ * blank body over a note that has one (INVALID_PARAMS,
+ * `blank_overwrite_refused`, naming the note), an existing note the host
+ * cannot read (INTERNAL_ERROR, `target_unreadable` - the caller's
+ * request is right and the host is not), and an explicit empty `content`
+ * read as a clear request rather than as an absent argument.
*/
import { afterEach, beforeEach, describe, expect, test } from "bun:test";
@@ -96,6 +104,82 @@ describe("brain_update_note", () => {
expect(err.code).toBe(INVALID_PARAMS);
});
+ test("declares the optional allow_empty boolean with a description", () => {
+ const schema = updateTool.inputSchema as {
+ properties: Record<string, { type?: string; description?: string }>;
+ required: ReadonlyArray<string>;
+ };
+ expect(schema.properties["allow_empty"]?.type).toBe("boolean");
+ expect((schema.properties["allow_empty"]?.description ?? "").length).toBeGreaterThan(0);
+ expect(schema.required).not.toContain("allow_empty");
+ });
+
+ test("a blank body over a note that has one is refused, naming the note", async () => {
+ seedNote("Notes/Doc.md", "worth keeping", "title: Doc");
+ const err = await rejectedMcpError(
+ updateTool.handler(ctx, { path: "Notes/Doc.md", content: " " }),
+ );
+ expect(err.code).toBe(INVALID_PARAMS);
+ expect(err.data).toMatchObject({ code: "blank_overwrite_refused", path: "Notes/Doc.md" });
+ expect(err.message).toContain("Notes/Doc.md");
+ expect(readFileSync(join(vault, "Notes/Doc.md"), "utf8")).toContain("worth keeping");
+ });
+
+ test("allow_empty clears the body deliberately", async () => {
+ seedNote("Notes/Doc.md", "worth keeping", "title: Doc");
+ const res = await updateTool.handler(ctx, {
+ path: "Notes/Doc.md",
+ content: "",
+ allow_empty: true,
+ });
+ expect(res).toMatchObject({ updated: true, path: "Notes/Doc.md" });
+ const md = readFileSync(join(vault, "Notes/Doc.md"), "utf8");
+ expect(md).toContain("title: Doc");
+ expect(md).not.toContain("worth keeping");
+ });
+
+ test("an explicit empty content beside frontmatter is a clear request, not an absent one", async () => {
+ // It used to collapse to "argument not sent": the frontmatter merge
+ // landed, the body the caller asked to clear stayed, and the receipt
+ // said updated: true.
+ seedNote("Notes/Doc.md", "worth keeping", "title: Doc");
+ const err = await rejectedMcpError(
+ updateTool.handler(ctx, {
+ path: "Notes/Doc.md",
+ frontmatter: { status: "final" },
+ content: "",
+ }),
+ );
+ expect(err.data).toMatchObject({ code: "blank_overwrite_refused", path: "Notes/Doc.md" });
+ const md = readFileSync(join(vault, "Notes/Doc.md"), "utf8");
+ expect(md).toContain("worth keeping");
+ expect(md).not.toContain("status: final");
+ });
+
+ test("allow_empty clears the body while merging frontmatter", async () => {
+ seedNote("Notes/Doc.md", "worth keeping", "title: Doc");
+ await updateTool.handler(ctx, {
+ path: "Notes/Doc.md",
+ frontmatter: { status: "final" },
+ content: "",
+ allow_empty: true,
+ });
+ const md = readFileSync(join(vault, "Notes/Doc.md"), "utf8");
+ expect(md).toContain("status: final");
+ expect(md).not.toContain("worth keeping");
+ });
+
+ test("an existing note the host cannot read is refused, not blanked", async () => {
+ // A directory where the note should be: it exists, and reading it
+ // raises EISDIR regardless of the running user.
+ mkdirSync(join(vault, "Notes/Doc.md"), { recursive: true });
+ const err = await rejectedMcpError(
+ updateTool.handler(ctx, { path: "Notes/Doc.md", content: "replacement" }),
+ );
+ expect(err.code).toBe(INTERNAL_ERROR);
+ expect(err.data).toMatchObject({ code: "target_unreadable", path: "Notes/Doc.md" });
+ });
+
test("path traversal is refused with INVALID_PARAMS", async () => {
const err = await rejectedMcpError(
updateTool.handler(ctx, { path: "../escape.md", content: "x" }),
@@ -131,6 +215,20 @@ describe("brain_append_note", () => {
expect(err.data).toMatchObject({ code: "target_missing", index: 0, path: "Notes/Ghost.md" });
});
+ test("an existing note the host cannot read is refused, not overwritten", async () => {
+ mkdirSync(join(vault, "Notes/Doc.md"), { recursive: true });
+ const err = await rejectedMcpError(
+ appendTool.handler(ctx, { path: "Notes/Doc.md", content: "more" }),
+ );
+ expect(err.code).toBe(INTERNAL_ERROR);
+ expect(err.data).toMatchObject({ code: "target_unreadable", path: "Notes/Doc.md" });
+ });
+
+ test("declares no allow_empty: an append never replaces a body", () => {
+ const schema = appendTool.inputSchema as { properties: Record<string, unknown> };
+ expect("allow_empty" in schema.properties).toBe(false);
+ });
+
test("refuses to author into the Brain machinery root", async () => {
const err = await rejectedMcpError(
appendTool.handler(ctx, { path: "Brain/x.md", content: "y" }),
diff --git a/tests/mcp/brain-write-batch.test.ts b/tests/mcp/brain-write-batch.test.ts
index ec19e3bc..5fd8c73d 100644
--- a/tests/mcp/brain-write-batch.test.ts
+++ b/tests/mcp/brain-write-batch.test.ts
@@ -7,6 +7,12 @@
* happens. Single-operation batches produce results equal to the
* dedicated brain_create_note / brain_update_note / brain_append_note
* tools.
+ *
+ * The nothing-writes-silently wave (unit B) adds `allow_empty` to the
+ * `update_note` op object and pins the two refusals the batch inherits
+ * from the shared projection: a blank body over a note that has one, and
+ * a target the host cannot read - each aborting the whole batch before
+ * any operation commits.
*/
import { afterEach, beforeEach, describe, expect, test } from "bun:test";
@@ -178,6 +184,63 @@ describe("brain_write_batch", () => {
expect(schema.properties.operations.maxItems).toBe(MAX_BATCH_OPERATIONS);
});
+ test("the operation object declares allow_empty with a description", () => {
+ const schema = tool.inputSchema as {
+ properties: {
+ operations: {
+ items: { properties: Record<string, { type?: string; description?: string }> };
+ };
+ };
+ };
+ const allowEmpty = schema.properties.operations.items.properties["allow_empty"];
+ expect(allowEmpty?.type).toBe("boolean");
+ expect((allowEmpty?.description ?? "").length).toBeGreaterThan(0);
+ });
+
+ test("an update_note op refuses to blank a note that has a body", async () => {
+ seedNote("Notes/Doc.md", "worth keeping", "title: Doc");
+ let thrown: unknown;
+ try {
+ await runBatch([{ op: "update_note", path: "Notes/Doc.md", content: "" }]);
+ } catch (err) {
+ thrown = err;
+ }
+ expect(thrown).toBeInstanceOf(MCPError);
+ expect((thrown as MCPError).data).toMatchObject({
+ code: "blank_overwrite_refused",
+ index: 0,
+ path: "Notes/Doc.md",
+ });
+ expect(readFileSync(join(vault, "Notes/Doc.md"), "utf8")).toContain("worth keeping");
+ });
+
+ test("allow_empty on the op clears the body deliberately", async () => {
+ seedNote("Notes/Doc.md", "worth keeping", "title: Doc");
+ const res = await runBatch([
+ { op: "update_note", path: "Notes/Doc.md", content: "", allow_empty: true },
+ ]);
+ expect(res.applied).toBe(1);
+ const md = readFileSync(join(vault, "Notes/Doc.md"), "utf8");
+ expect(md).toContain("title: Doc");
+ expect(md).not.toContain("worth keeping");
+ });
+
+ test("an unreadable note aborts the batch before any op lands", async () => {
+ mkdirSync(join(vault, "Notes/Unreadable.md"), { recursive: true });
+ let thrown: unknown;
+ try {
+ await runBatch([
+ { op: "create_note", path: "Notes/First.md", content: "one" },
+ { op: "update_note", path: "Notes/Unreadable.md", content: "two" },
+ ]);
+ } catch (err) {
+ thrown = err;
+ }
+ expect(thrown).toBeInstanceOf(MCPError);
+ expect((thrown as MCPError).data).toMatchObject({ code: "target_unreadable", index: 1 });
+ expect(existsSync(join(vault, "Notes/First.md"))).toBe(false);
+ });
+
test("single-op create parity with brain_create_note", async () => {
const createTool = NOTES_TOOLS.find((t) => t.name === "brain_create_note")!;
const direct = await createTool.handler(ctx, { path: "Notes/Direct.md", content: "d" });
diff --git a/tests/mcp/context-pack-evidence-tool.test.ts b/tests/mcp/context-pack-evidence-tool.test.ts
index 38a15294..98d25623 100644
--- a/tests/mcp/context-pack-evidence-tool.test.ts
+++ b/tests/mcp/context-pack-evidence-tool.test.ts
@@ -47,10 +47,18 @@ const GOLDEN_CLAIMLESS_PAYLOAD_KEYS: ReadonlyArray<string> = Object.freeze([
* hash over a wall-clock timestamp - so they are interpolated from the
* record under test and everything else, envelope key order included, is
* pinned as a literal.
+ *
+ * `originChannel` is the server-derived channel of the process that
+ * appended the record (Unit C of the nothing-writes-silently wave). It is
+ * `unset` - the explicit literal for "no entry point claimed this
+ * process" - because this suite drives the tool handler directly rather
+ * than through the MCP transport, and the suite preload unclaims the
+ * channel before every test so the value cannot depend on file order.
*/
function goldenOutcomeLine(id: string, createdAt: string): string {
return (
- `{"schema":"o2b.continuity.v1","id":"${id}","kind":"context_pack_outcome",` +
+ `{"schema":"o2b.continuity.v1","id":"${id}","originChannel":"unset",` +
+ `"kind":"context_pack_outcome",` +
`"createdAt":"${createdAt}","sourceRefs":[],"payload":${GOLDEN_CLAIMLESS_PAYLOAD},` +
`"private":false,"redacted":false}`
);
diff --git a/tests/mcp/ingest-tool.test.ts b/tests/mcp/ingest-tool.test.ts
index 586fc9cc..d1ff0eca 100644
--- a/tests/mcp/ingest-tool.test.ts
+++ b/tests/mcp/ingest-tool.test.ts
@@ -5,7 +5,7 @@
*/
import { afterEach, beforeEach, describe, expect, test } from "bun:test";
-import { mkdtempSync, readFileSync, readdirSync, rmSync } from "node:fs";
+import { existsSync, mkdtempSync, readFileSync, readdirSync, rmSync } from "node:fs";
import { join } from "node:path";
import { tmpdir } from "node:os";
@@ -15,7 +15,7 @@ import { bootstrapBrain } from "../../src/core/brain/init.ts";
import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
import { listEntities } from "../../src/core/brain/entities/registry.ts";
import { INGEST_TOOLS } from "../../src/mcp/brain/ingest-tools.ts";
-import { MCPError } from "../../src/mcp/protocol.ts";
+import { INVALID_PARAMS, MCPError } from "../../src/mcp/protocol.ts";
import type { ServerContext } from "../../src/mcp/tool-contract.ts";
let vault: string;
@@ -85,6 +85,29 @@ describe("brain_ingest_source", () => {
expect(listEntities(vault)).toHaveLength(0);
});
+ test("a plan_id outside the checkpoint grammar is the caller's fault, and named", async () => {
+ seed("Articles/eth.md");
+ // Swallowed, this took the whole batch: every call succeeded, no
+ // checkpoint was ever written, and reconciliation reported all of it as
+ // never ingested. INVALID_PARAMS, not INTERNAL_ERROR - the argument is
+ // what is wrong and rephrasing it is the fix.
+ let thrown: unknown;
+ try {
+ await handler(ctx, {
+ source_path: "Articles/eth.md",
+ summary: "x",
+ entities: [{ category: "concept", name: "A" }],
+ plan_id: "docs-migration",
+ });
+ } catch (err) {
+ thrown = err;
+ }
+ expect(thrown).toBeInstanceOf(MCPError);
+ expect((thrown as MCPError).code).toBe(INVALID_PARAMS);
+ expect((thrown as MCPError).message).toContain("plan id");
+ expect(existsSync(join(vault, "Brain", "sources"))).toBe(false);
+ });
+
test("missing required source_path is rejected", async () => {
await expect(
handler(ctx, { summary: "x", entities: [{ category: "concept", name: "A" }] }),
diff --git a/tests/mcp/mcp.test.ts b/tests/mcp/mcp.test.ts
index 9f1b01d6..0777b87d 100644
--- a/tests/mcp/mcp.test.ts
+++ b/tests/mcp/mcp.test.ts
@@ -15,6 +15,9 @@ import {
import { createPluginRepo, createSandboxVault } from "../helpers/fixtures.ts";
import { REDACTION_PLACEHOLDER } from "../../src/core/redactor.ts";
import { PARTNER_CODEGRAPH_DISABLED_ENV } from "../../src/core/config.ts";
+import { bootstrapBrain } from "../../src/core/brain/init.ts";
+import { appendLogEvent } from "../../src/core/brain/log.ts";
+import { BRAIN_LOG_EVENT_KIND } from "../../src/core/brain/types.ts";
let tmp: string;
const savedEnv: Record<string, string | undefined> = {};
@@ -423,6 +426,41 @@ describe("tool calls", () => {
expect(s.brain.counts.preferences).toBe(0);
expect(s.brain.last_dream_at).toBeNull();
expect(s.brain.sanity.signals_awaiting_dream).toBe(0);
+ // Brain absent -> the same honest never_dreamed/zero shape as an
+ // empty-but-present Brain, not an error (nothing-writes-silently, D).
+ expect(s.brain.maintenance_debt).toEqual({
+ status: "never_dreamed",
+ log_events_since_dream: 0,
+ });
+ });
+
+ test("second_brain_status `brain.maintenance_debt` derives from the log shards", async () => {
+ const vault = join(tmp, "vault");
+ mkdirSync(vault);
+ const config = join(tmp, "config.yaml");
+ writeFileSync(config, `vault_path: ${vault}\n`);
+ bootstrapBrain(vault, { configPath: config });
+ appendLogEvent(vault, {
+ timestamp: "2026-05-10T09:00:00Z",
+ eventType: BRAIN_LOG_EVENT_KIND.note,
+ agent: "tester",
+ body: { text: "note" },
+ });
+ appendLogEvent(vault, {
+ timestamp: "2026-05-11T09:00:00Z",
+ eventType: BRAIN_LOG_EVENT_KIND.note,
+ agent: "tester",
+ body: { text: "note" },
+ });
+
+ const server = new MCPServer({ vault, configPath: config });
+ await initialize(server);
+ const r = (await callTool(server, "second_brain_status"))! as any;
+ const s = r.result.structuredContent;
+ expect(s.brain.maintenance_debt).toEqual({
+ status: "never_dreamed",
+ log_events_since_dream: 2,
+ });
});
test("second_brain_status includes a `vault` block (v0.10.9)", async () => {
diff --git a/tests/mcp/resources.test.ts b/tests/mcp/resources.test.ts
index b53aa0d6..3bb6a59e 100644
--- a/tests/mcp/resources.test.ts
+++ b/tests/mcp/resources.test.ts
@@ -22,6 +22,8 @@ import { writePreference } from "../../src/core/brain/preference.ts";
import { writeSignal } from "../../src/core/brain/signal.ts";
import { regenerateActive } from "../../src/core/brain/active.ts";
import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
+import { appendLogEvent } from "../../src/core/brain/log.ts";
+import { BRAIN_LOG_EVENT_KIND } from "../../src/core/brain/types.ts";
let tmp: string;
let vault: string;
@@ -381,6 +383,46 @@ describe("MCP resources — read osb://status", () => {
expect(text).toContain("confirmed: 1");
expect(text).toContain("## Activity");
});
+
+ test("reports never_dreamed maintenance debt when no dream has run", async () => {
+ appendLogEvent(vault, {
+ timestamp: "2026-05-10T09:00:00Z",
+ eventType: BRAIN_LOG_EVENT_KIND.note,
+ agent: "tester",
+ body: { text: "note" },
+ });
+
+ const server = makeServer();
+ await initialize(server);
+ const r = await read(server, "osb://status");
+ const text = r.result.contents[0].text as string;
+ expect(text).toContain("## Maintenance debt");
+ expect(text).toContain("never dreamed");
+ expect(text).toContain("1 log event");
+ });
+
+ test("reports the counted debt since the last dream", async () => {
+ appendLogEvent(vault, {
+ timestamp: "2026-05-10T09:00:00Z",
+ eventType: BRAIN_LOG_EVENT_KIND.dream,
+ agent: "tester",
+ body: {},
+ });
+ appendLogEvent(vault, {
+ timestamp: "2026-05-11T09:00:00Z",
+ eventType: BRAIN_LOG_EVENT_KIND.note,
+ agent: "tester",
+ body: { text: "note" },
+ });
+
+ const server = makeServer();
+ await initialize(server);
+ const r = await read(server, "osb://status");
+ const text = r.result.contents[0].text as string;
+ expect(text).toContain("## Maintenance debt");
+ expect(text).toContain("1 log event");
+ expect(text).not.toContain("never dreamed");
+ });
});
describe("MCP resources — read osb://backlinks/{id}", () => {
diff --git a/tests/setup.ts b/tests/setup.ts
index e1b045e0..6345245e 100644
--- a/tests/setup.ts
+++ b/tests/setup.ts
@@ -16,10 +16,31 @@
* Tests that need their own vault still set `OPEN_SECOND_BRAIN_CONFIG` /
* `XDG_CONFIG_HOME` themselves; this only provides a safe default.
*/
+import { beforeEach } from "bun:test";
import { mkdtempSync, mkdirSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
+import { clearOriginChannel } from "../src/core/origin-channel.ts";
+
+// The origin channel (`src/core/origin-channel.ts`) is process-scoped by
+// design: an entry point claims it once and every record written under
+// that process carries it. Bun runs many test files in ONE process, so a
+// file that invokes `main()` leaves `cli` claimed for every file that
+// runs after it - and a byte-identity golden over a log line, a signal or
+// a continuity record would then pass or fail on file ORDER.
+//
+// Unclaiming before each test makes the default deterministic (`unset`,
+// the explicit literal for "no entry point claimed this process") without
+// pinning a real channel that would hide the difference. A test that
+// wants a channel claims it, exactly as the entry point does. Registered
+// here rather than in each suite for the same reason `O2B_DEVICE_ID` is
+// pinned above: a hermeticity rule every file needs is not a rule every
+// file should have to remember.
+beforeEach(() => {
+ clearOriginChannel();
+});
+
// Per-device log sharding (Memory Integrity Suite): pin the device id
// to the empty string so the suite writes the legacy un-sharded log
// pair deterministically. Shard-specific tests clear / override