Local-first 🧠 memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly 😴 dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.
TypeScript
feat: provenance at the boundary (v1.43.0) (#154)
Sol Aitken0ae4b097bd3a
diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.jsonindex 06d94157..8e5a958c 100644--- a/.claude-plugin/plugin.json+++ b/.claude-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.42.0",+ "version": "1.43.0","description": "Plugin-first second brain package for AI agents and humans.","author": {"name": "Open Second Brain contributors"diff --git a/.codex-plugin/plugin.json b/.codex-plugin/plugin.jsonindex 4309f8a1..eb407794 100644--- a/.codex-plugin/plugin.json+++ b/.codex-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.42.0",+ "version": "1.43.0","description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.","skills": "./skills","hooks": "./hooks/hooks.json",diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 67fe4eb8..42a02c8b 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -5,6 +5,51 @@ All notable changes to this project will be documented in this file.The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).+## [1.43.0] - 2026-08-02++Ten units on one seam: the boundary where content, authority and claims enter the vault. Open Second Brain could say a great deal about what it holds and almost nothing about where any of it came from, who was entitled to put it there, or what backs a claim made about it. An entity extracted from a scraped page became a first-class Brain entity on the same terms as one the operator typed. A write named any path it liked. A note was created with whatever frontmatter the caller assembled. A record was ranked by when its file was touched rather than by what it is about. An agent posted its own success and nothing recomputed it.++A reconnaissance pass over the code preceded the design and changed the work substantially: load-bearing premises were falsified in nine of the ten tasks. Two described a capability that cannot be built honestly on this architecture and were scoped down rather than faked. An independent verifier is not definable here - agent identity is an environment variable, else a config key, else the literal string `agent`, with any caller-supplied string accepted verbatim, and the server is a child process of the agent it labels, so two records asserting two names chosen by one process are not two actors. A schema pack fetched by URL is not implementable either - a pack is a block inside one config file, the listing returns a single hardcoded entry, and there is no registry to install into and no signature verification anywhere. Three tasks described gaps that are already closed while the open gap sat one layer away unmentioned: ranking already used a content-derived event time, skill invocation was already observed, and a document validator already existed.++Five reviewers were then given the finished branch and the claims made for it, with no knowledge of how the work was done, one of them told to treat every test as guilty until proven innocent. They found thirty defects, nine introduced by this wave. Every one is fixed or named below. Three were a mechanism nothing could make fire, which is the shape this repository reverted a unit for one release ago, and one of those three was removed rather than repaired.++### Added++- Untrusted intake lands in a quarantine lane instead of the registry. Trust is originated where content arrives and derived structurally from the source identity's shape, with no host list, scheme list or word list in any language. The entity status filter, previously re-implemented at every caller and absent at eight of them, is one shared predicate a census keeps singular, and a status nothing can leave would be a dead end so release from quarantine is the named exit.+- A write boundary the operator declares in the vault config. A caller-named write outside the declared path prefixes is refused through a registered code. Its authority is the config file, never the caller: identity here is self-asserted and additionally overridable by an argument on twenty-two tool schemas, so a fence keyed to it is bypassed by passing a different string. This is a write boundary over caller-named paths and not a security boundary, and nothing describes it as one.+- Note creation gains an idempotent skip whose result says which happened, pre-write validation over the existing artifact validator, and a template mode with a closed two-construct grammar - typed variables, and presence-driven sections with list iteration - parsed by recursive descent with no dependency.+- A body-derived date anchor, materialised at index time with the rung that won recorded beside it as a registered token, so a back-dated note ranks by what it is about. `o2b search event-anchor-backfill` resolves anchors for documents indexed before this release; an index run counts what is still pending and names that command as its exit.+- `embedding_batch_tokens` closes an embedding batch on accumulated tokens as well as on item count, on both batching providers.+- One capability-tier resolver shared by the five sites that computed those facts separately, behind registered codes, with a contract predicate replacing a provider-name string comparison duplicated at five sites. `o2b search vector-backfill` runs the vector phase alone.+- `schema_apply_mutations` and `o2b brain schema apply` gain a dry run returning the pack that would result and its diff, with the same validator rejections and no write.+- A second ledger record carrying the kernel's own on-disk evidence against the acting agent's claim, joined by the existing sample id. It carries no verifier identity and no surface calls it independently witnessed, because it cannot be.+- A skill offer now has a content-addressed identity an invocation can be joined back to, a discriminating-term ranking floor derived from corpus document frequency, and retained provenance where a skill is shadowed by a same-named one in a later root.++### Fixed++- `o2b brain schema apply --dry-run` parsed the flag and wrote the vault schema anyway. It previews now.+- An untrusted source that guessed an entity name captured it permanently: every later trusted write landed inside the quarantined record, returned a not-created result, and read back as absent. Trusted and untrusted writes are separate lanes that never resolve across.+- The trust classifier had the wrong polarity in three ways. A bare host classified as trusted, an intake naming no source classified as trusted, and a vault filename containing a colon parsed as an external URI and quarantined everything it touched. It trusts only what is shaped like a location inside this vault, and an intake naming no source is refused before any write.+- A symlink under a declared write prefix widened the binding to the whole vault while the response named a path the bytes were not at. A backslash was a separator to the prefix matcher and an ordinary character to the filesystem, so a binding admitted a write that landed at the vault root - and the same mismatch had been hiding the Brain machinery root from its own guard with no config present at all. Both are pre-existing and both are closed.+- The anchor ladder had promoted two frontmatter keys into event time, and event time feeds the hard `since`/`until` filter rather than only the ranking boost, so a note carrying an old `created_at` and edited today disappeared from a query for recent work. Those keys stay recorded with their provenance; they are no longer consulted by the query side.+- A frontmatter key containing a newline emitted extra lines, and because an update merges last-wins a caller could overwrite a key it was not asked to touch. Keys are refused at the emitter rather than escaped: the reader has no quoted-key form to escape into, so a quoted key would be lost on the next parse, trading an injection for a silent loss.+- An explicit semantic request degraded to lexical results with a success exit on one rung of the capability ladder. A token budget above the 32-bit boundary truncated to one item per request, and the sibling count cap had the same defect; both are refused rather than quietly reinterpreted.+- Quarantined records were readable through page walkers, which no entity-reader rule can see because entity records are ordinary vault pages.+- A capability probe made an outbound request with no credential whose only possible outcome was an authentication error, a working local provider was reported as having an unresolved key and advised to set one it does not use, and a fourth branch of the deferred-reason function was unreachable because its only call site passed a constant.++### Changed++- The declared continuity scope was built and removed before release rather than shipped. Nothing could supply one, and its single vocabulary member would have made a record less visible, since a declared scope is dropped by every reader that does not name it. The record envelope is unchanged, dedup id included, and is now pinned as a known answer.+- Three censuses were added and two of them were the disease they were written to cure. The write-site census read only the first `node:fs` import statement per file and was already missing a live vault append. The entity-read census missed the registry API whose explicit status branch bypasses the shared predicate, had the quarantine value absent from its own vocabulary, and its second rule reached nothing at all. Every rule now has to reach modules no other rule reaches, and that is asserted rather than described.+- Four tests claimed byte-identity without comparing bytes, one by sorting the keys whose order was the property. Each compares a golden now and each was shown failing against a real mutation of the code it pins.++### Notes++- Three modules carried literal NUL bytes as key separators, which makes `grep` classify the file as binary and print nothing for it. They were invisible to every grep-based sweep, including this release's own reconnaissance, which is why it undercounted the continuity readers as five of twenty-one when the measured figure is four of twenty-four. All three now use the two-character escape.+- Found and deliberately not closed: roughly forty test sites report a silent pass when the vector extension is absent, so a host without it loses the coverage while the suite reads green - two files were converted and the shared helper is a follow-up. Twenty continuity readers bypass the masking read model, nineteen of them predating this release and the twentieth added by it. Page walkers outside the link-graph tree are not asserted by the new rule. The concurrency cap has the same 32-bit truncation as the batch caps, left alone because there the clamp is protective and a maximum is a policy decision. And `brain_token_impact` still has no actor wiring - the asymmetry one unit set out to close did not exist in the direction stated.+- Two review findings were measured false and no fix was written for them.+- The vault schema goes to version 11. The migration is additive and runs in place; it does not reindex, which is what the new backfill verb and the index run's changed exit exist to say out loud.+## [1.42.0] - 2026-07-27A structural wave with no new capability: the six modules that cost the most to change, plus the two import cycles in the graph, plus the silent fallbacks the split uncovered on the way. The targets were chosen by measurement rather than by impression - a maintainability index, a cognitive-complexity count and a fan-in count per file - and the six worst were a config validator with a single twelve-hundred-line function, the SQLite store as one class holding fourteen tables, the recall pipeline as one function running twelve stages, every search subcommand in one file, the vault doctor as twenty-six unrelated diagnostics in one body, and the dream pass with its orchestration fused into the mechanics it orchestrates.@@ -6800,6 +6845,7 @@ plugin config (vault field)`, and exits with a clear- Sandbox vault and plugin manifest fixtures for tests.- GitHub release workflow for tag-based and manually dispatched releases.+[1.43.0]: https://github.com/itechmeat/open-second-brain/compare/v1.42.0...v1.43.0[1.42.0]: https://github.com/itechmeat/open-second-brain/compare/v1.41.0...v1.42.0[1.41.0]: https://github.com/itechmeat/open-second-brain/compare/v1.40.0...v1.41.0[1.40.0]: https://github.com/itechmeat/open-second-brain/compare/v1.39.0...v1.40.0diff --git a/README.md b/README.mdindex 7f1ea368..8dba38e8 100644--- a/README.md+++ b/README.md@@ -8,7 +8,11 @@ Open Second Brain plugs into [Hermes Agent](https://github.com/NousResearch/herm## What is new-Open Second Brain 1.42.0 is a structural wave with no new capability, and one behavioural correction worth knowing about: a `_brain.yaml` that is present but malformed no longer reads as one that is absent, so a typo in your config stops being served back as the documented defaults while every surface reports healthy, and a diagnostic that could not read a directory now says so instead of rendering as a clean bill of health. The rest is the six modules that cost the most to change, chosen by measurement rather than impression - a config validator with a single twelve-hundred-line function, the search store as one class over fourteen tables, the recall pipeline as one function running twelve stages, every `o2b search` verb in one file, the vault doctor as twenty-six unrelated checks in one body, and the dream pass with its orchestration fused into the mechanics it orchestrates. Both import cycles in the graph are gone, and a ratchet now fails the build on any new one - after review proved that ratchet could pass on an empty set, which is fixed and demonstrated. Behaviour is byte-identical everywhere it is claimed to be, and where that guarantee is load-bearing it was measured rather than asserted: the validator against its predecessor over 3135 generated configs, the command line over 92 invocations with stdout, stderr and exit code diffed byte for byte, and the store's schema statements and the pipeline's cache-key composition diffed line for line, since either changing would strand data already on disk without failing a test. The [CHANGELOG](CHANGELOG.md) has the detail.+Open Second Brain 1.43.0 is about provenance at the boundary: what enters your vault, under whose authority, validated against what, and backed by what proof. Entities an agent extracts from an untrusted source - a scraped page, a fetched article - now land in a quarantine lane instead of becoming first-class Brain entities, and trust is derived from the shape of the source identity rather than from any word list. You can declare in `_brain.yaml` which path prefixes a caller-named write may touch, and a write outside them is refused with the command that resolves it; its authority is the config file, not the caller, because a caller can name itself anything. Note creation gains an idempotent skip whose result tells you which happened, validation before the write lands, and a template mode with a deliberately small grammar. A back-dated note - an imported log, a meeting record - now ranks by the date its body declares rather than by when the file was touched, with the source of that date recorded beside it. Schema mutations can be previewed before they touch the vault, which matters more than it sounds: `--dry-run` was previously parsed and ignored. And an outcome an agent posts about its own work now carries the kernel's own on-disk evidence beside the claim, with a mismatch recorded rather than resolved.++Two things this release deliberately did not build are worth knowing about, because their absence is a decision rather than an omission. There is no independent verifier, because agent identity here is an unauthenticated string and two records asserting two names chosen by one process are not two actors. And there is no pack fetched by URL, because a schema pack has no portable representation and no registry to install into. Five independent reviewers were given the finished branch and found thirty defects, nine of them introduced by this work and three of them mechanisms nothing could make fire; one of those three was removed rather than repaired. The [CHANGELOG](CHANGELOG.md) has the detail, including what was found and deliberately left open.++Previous release, 1.42.0, was a structural wave with no new capability: the six modules that cost the most to change, both import cycles removed, and nine places where absence and inability-to-examine had been collapsed into one answer. Details live in the [CHANGELOG](CHANGELOG.md).Previous release, 1.41.0, was a signals-that-survive wave: eight units that carry a signal the system already computes the last few metres, to the decision it should have changed. Ingest now honours the repository's own `.gitignore`, nested ignore files, `.git/info/exclude` and submodule boundaries, through the same discovery module the hygiene scan uses - so a repo path stops pulling in the build artifacts it already declares as noise. A recall graded weak no longer evaporates when the call returns: repeated weak recall on the same question becomes a tracked, self-closing gap task. A query that names a period is ranked for that period instead of against it, detected from ISO and `since:`/`until:` tokens only so it behaves the same in every language. A scope-less capture says which routing signal it was missing and which scopes the vault actually has. Model-authored payloads are checked against a declared shape before they become knowledge, unconditionally - a validator that can be switched off is one the write path cannot rely on. Plain text can now seed a fact or a skill, not only a taste signal. And resumed work is recognised by a declared identity rather than by a fifteen-minute clock, so a branch or worktree switch stops breaking the chain. A ninth unit was built, reviewed, and reverted rather than shipped, because proving it inert was easier than pretending it was not - the [CHANGELOG](CHANGELOG.md) says why.diff --git a/docs/architecture.md b/docs/architecture.mdindex 66019ee6..e4daba85 100644--- a/docs/architecture.md+++ b/docs/architecture.md@@ -136,7 +136,14 @@ with backup/sync. It describes:- optional `integrity:` gates (`owner_scope_delivery`, `embedding_abi`,`pack_validity_seconds`) that decide where a mismatch is reported andwhere it is refused;-- `vault.ignore_paths` (exclusion policy for every vault walker).+- `vault.ignore_paths` (exclusion policy for every vault walker);+- optional `write_binding.path_prefixes` (where a CALLER-NAMED write may+ land: `brain_create_note`, `brain_update_note`, `brain_append_note`,+ `brain_write_batch`). This is a write boundary over caller-named paths,+ not a security boundary and not per-agent — its authority is this file,+ which no tool call can rewrite, and it reads no caller identity. Writes+ whose destination is derived rather than named are outside it; the+ boundary is recorded in `tests/core/architecture/write-site-census.test.ts`.It must not contain secrets.diff --git a/docs/brainstorm/provenance-at-the-boundary/cli-output/claude.md b/docs/brainstorm/provenance-at-the-boundary/cli-output/claude.mdnew file mode 100644index 00000000..51787a20--- /dev/null+++ b/docs/brainstorm/provenance-at-the-boundary/cli-output/claude.md@@ -0,0 +1,43 @@+### Variant 1: Shared vocabulary, enforced by conformance tests++- **Approach**: Ship no new shared runtime module. Instead, freeze a written contract up front — envelope field names (origin, trust grade, anchor source, scope), a registered code namespace for every refusal and every "capability absent" answer, and a canonical sha256 digest helper — and let each of the ten units implement against it in its own files. The only shared artifact landed before the parallel work is a conformance test suite that every unit's write path must pass.+- **Trade-offs**:+ - Pro: near-zero merge contention; ten atomic commits touch ten disjoint file sets, which is exactly what parallel implementation by separate agents wants.+ - Pro: additive by construction — nothing central changes, so "byte-identical when the flag is absent" is trivially true for all ten units.+ - Pro: reuses the rails that already exist (the advisory diagnostic rail, `SearchError` codes, `UNTRUSTED_SOURCE_TAG`, the preference lifecycle statuses) rather than superseding them.+ - Con: a credential write fence enforced by convention is not a fence — nothing structurally prevents a write path (existing or future) from skipping the check, which defeats the point of t_a3c4b13b.+ - Con: provenance drifts. The quarantine lane, the content-date anchor, the pack installer and the skill sha256 collapse will each grow their own near-identical origin descriptor, and the wave ends up with four provenance shapes instead of one.+ - Con: conformance tests only catch drift where someone thought to write the test; absence-vs-inability distinctions are easy to collapse silently in a unit nobody tested for it.+- **Complexity**: small+- **Risk**: medium++### Variant 2: Single write-admission pipeline++- **Approach**: Factor by control flow. Introduce one ordered admission chain — authority check, shape/schema validation, provenance stamp and trust routing, capability resolution, then land — and funnel every write-capable surface through it: create/append/update note, write-batch, intake extraction, ingest, schema mutation, pack apply. Each of the ten units becomes a stage implementation or a stage consumer rather than an independent code path.+- **Trade-offs**:+ - Pro: the write-prefix fence becomes genuinely unskippable, and every future write surface inherits it for free — the strongest possible answer to "authority to write".+ - Pro: one place defines what a refusal looks like, so the refusal contract, its registered codes and its tests exist exactly once.+ - Pro: quarantine routing (t_444349f2), `--strict` validation and `if_exists: skip` (t_c0fce0b9), and pack preview-before-apply (t_50033859) become three configurations of the same chain rather than three hand-rolled gates.+ - Con: blast radius across roughly every write surface in a ~900-module, ~8100-test codebase, with high odds of incidental behaviour change where the "absent flag is a no-op" guarantee is hardest to prove.+ - Con: it inverts the parallelism constraint — ten agents all editing one chain is contention concentrated precisely where the wave cannot afford it.+ - Con: it does not actually unify the wave. Embedding batch packing, capability tiers with deferred backfill, the content-date anchor, the verification ledger and skill receipts are not writes through this chain, so half the units sit outside the substrate and the "one coherent substrate" claim is only half true.+- **Complexity**: large+- **Risk**: high++### Variant 3: Two substrates plus a capability rail++- **Approach**: Accept that the six cross-cutting concerns cluster into two, not one, and factor accordingly. Substrate A, the intake boundary: a single provenance descriptor (origin, trust grade, digest) plus an admission decision that composes authority, validation and routing-to-quarantine, serving the quarantine lane, the write-prefix fence, template/strict/skip creation and HTTP pack install, and supplying the anchor-provenance field the content-date unit needs and the positive scope value the shared-observation unit needs. Substrate B, the attestation ledger: one witnessed-record shape (witness identity, witness kind, evidence digest, pass/fail) keyed by the existing sample id, serving the split verification ledger and the skill efficacy receipt chain. Alongside them, extend the existing registered-code rail with an explicit capability state — configured, absent, deferred — so embedding batch budgeting, tiered degradation with NULL-vector backfill, and the pack fetcher's network capability all answer "not configured" the same way, distinct from "nothing found".+- **Trade-offs**:+ - Pro: each shared piece is small enough for one agent to land as a seed commit before the eight dependent units fan out, satisfying the "decided up front, not discovered at merge" constraint without a central chokepoint.+ - Pro: the clustering is real — provenance, authority and validation genuinely meet at intake; claim-versus-proof genuinely meets at the ledger; capability-absence is a rail concern in both and in neither substrate. Nothing is forced together that does not share semantics.+ - Pro: gives enforcement teeth where it matters (one admission decision at intake, one witness-record identity) while leaving mechanically unrelated units — embedding token budgeting, the content-date column — as local additive changes that merely consume the vocabulary.+ - Pro: the capability rail is where the absence-versus-inability convention is naturally honoured once, instead of three times.+ - Con: three shared pieces must land before most of the wave, serialising the first stretch of an otherwise parallel effort.+ - Con: both substrates want a content digest and an origin notion; without deliberately sharing one digest primitive between them, the wave grows exactly the duplication it was meant to avoid.+ - Con: the boundary between the new capability state and the existing `SearchError` / advisory codes needs an explicit ruling, or two vocabularies will describe the same unconfigured provider.+- **Complexity**: medium+- **Risk**: medium++### Recommended: Variant 3++**Rationale**: The six concerns do not collapse into one substrate — intake admission and outcome attestation share almost no semantics, and Variant 2 pays a large-blast-radius, high-contention price to pretend otherwise while still leaving half the units outside the chain. Variant 1 is cheap and parallel-friendly but enforces the one concern that must be structural, authority to write, by convention alone, and will leave four divergent provenance shapes behind. Variant 3 buys real enforcement where it is load-bearing, keeps the seed commits small enough to land ahead of the fan-out as the parallel-agent constraint requires, and respects the per-task carve-outs by leaving the purely mechanical units local rather than dragging them into a substrate they do not need.diff --git a/docs/brainstorm/provenance-at-the-boundary/cli-output/prompt.md b/docs/brainstorm/provenance-at-the-boundary/cli-output/prompt.mdnew file mode 100644index 00000000..af827ba8--- /dev/null+++ b/docs/brainstorm/provenance-at-the-boundary/cli-output/prompt.md@@ -0,0 +1,302 @@+You are brainstorming architectural variants for the following task. Do not write code. Do not write a final design. Only produce variants and a recommendation.++# Task++Ship one release wave for Open Second Brain covering the ten kanban tasks below. They were selected together because they all sit on the same seam: the boundary where content, authority and claims enter the vault. The architectural question you are brainstorming is NOT how to implement each task separately - it is how the ten units should be factored so the wave has one coherent substrate instead of ten independent bolt-ons.++Specifically, decide how these cross-cutting concerns should be shared or kept separate across the ten units:++- provenance of a record (where it came from, how trusted that origin is)+- authority to write (which identity may write where, and what a refusal looks like)+- validation before a write lands (schema, shape, existence)+- what a record says about its own scope and its own time anchor+- what happens when a capability the write path wants is not configured+- what independent proof, if any, backs an outcome claim+++## t_444349f2 (board priority 4) - [upstream:gbrain] Quarantine/review lane for auto-extracted entities carrying untrusted provenance++**Source**: https://github.com/garrytan/gbrain/pull/3458+**Repo**: garrytan/gbrain (18439★)+**PR**: #3458 feat(extract): quarantine lane for auto-extracted entities from untrusted input (#160) (2026-07-28)++## What+Route entities that an agent auto-extracts from untrusted content (ingested web/chat, scraped pages) into a quarantine/review lane instead of promoting them straight into the knowledge registry, so untrusted provenance cannot contaminate trusted entities. This is a deterministic provenance-routing policy on the intake primitive, not an extraction model — the calling agent still extracts; OSB decides where the typed record lands.++## Why useful for OSB+OSB already has both halves of the machinery but not the join: a `quarantine` status exists for preferences and an `UNTRUSTED_SOURCE_TAG` fence exists for note content, yet intake entities are written straight into the entity registry with no review lane. Gating untrusted-provenance intake entities into a quarantine lane before promotion closes a real contamination gap and fits OSB's deterministic, agent-owns-the-model contract — extracted entities from a scraped web page should not become first-class Brain entities until a human/agent promotes them.++## Status in OSB+- **Verdict**: not_in_osb_useful+- **Codegraph hints**: Quarantine status exists for PREFERENCES only — `src/core/brain/preference.ts:353-356` (BRAIN_PREFERENCE_STATUS.quarantine), surfaced in `src/mcp/brain/feedback-tools.ts:518` and digest at `src/mcp/resources.ts:89,527`. Untrusted-content fencing exists — `src/core/brain/untrusted-source.ts` (UNTRUSTED_SOURCE_TAG, fenceUntrustedContent, neutralizeUntrustedText), consumed by `src/core/brain/navmap.ts:16`. No quarantine/review lane for extracted ENTITIES — entities enter the registry directly via `src/core/brain/intake/extract-intake.ts` → `intakeExtraction` (called from `src/mcp/brain/ner-tools.ts:17`); not found.++## Notes+Adopt as a deterministic provenance policy on `intakeExtraction`, not an extraction model: reuse the existing `UNTRUSTED_SOURCE_TAG` signal to flag intake records, and add a `quarantine`-style entity status mirroring the existing preference lifecycle (unconfirmed → confirmed/retired). Conceptually adjacent to the review-queue idea already observed on `t_041c571f` (llm-wiki-compiler quarantines low-confidence/provenance-violating generated pages); this task is the entity-intake-specific instance of it.+++## t_a3c4b13b (board priority 4) - [upstream:gbrain] Per-credential enforced write-prefix fence (bound-slug-prefixes) refusing cross-prefix writes at the server++**Source**: https://github.com/garrytan/gbrain/releases/tag/v0.42.72.0+**Repo**: garrytan/gbrain (18439★)+**Released**: v0.42.72.0 (2026-08-01T23:06:18Z)++## What+Bind a credential/client to a set of folder-prefix write boundaries so the server REFUSES any write op that names a page outside those prefixes — a per-credential, server-enforced, write-only fence. Ops that write by something other than a page slug (entity/fact extraction, ontology propose, source add/remove, ingest, put-page auto-fact-extraction) are refused outright to bound credentials rather than left unfenced. Bindings are rescopable in place with existing tokens picking up the change on the next request (no secret rotation).++## Why useful for OSB+Lets multiple agents/people share one source while each is confined to its own write territory, preventing accidental or cross-agent overwrites without splitting the vault or rotating credentials. Complements OSB's existing read/ingest scope filters by adding a hard, credential-level write authorization boundary that OSB currently lacks.++## Status in OSB+- **Verdict**: not_in_osb_useful+- **Codegraph hints**: OSB has scope-based read/ingest isolation only, not a per-credential write-prefix fence. Owner-scope read/ingest filtering: src/mcp/brain/query-tools.ts:182, src/mcp/brain/context-tools.ts:360, src/mcp/brain/ingest-tools.ts:129, src/mcp/tools.ts:236, src/mcp/server.ts:123, src/cli/search/verbs/query.ts:129, src/core/search/types.ts:329. Agent-ownership recall narrowing: src/core/graph/agent-scope.ts:2-43, src/core/scope-key.ts:25,100, src/core/search/pipeline/pool-filters.ts:130, src/core/search/result-filters.ts:270,307. No symbol found for credential-bound write-prefix enforcement or server-side refusal of writes outside a bound prefix.++## Notes+Distinct mechanism, not a weaker version of existing OSB isolation. OSB's models are request/scope-axis filters (owner, agent-scope) applied on READ and INGEST; gbrain's is a WRITE-only boundary keyed to the credential and enforced by server refusal at every page-naming op. Different axis (folder prefix vs owner/agent identity) and different enforcement point (write-refusal vs read/ingest narrowing), so this is a capability gap rather than an improvement to an existing feature. Per upstream, this is a write boundary, not a privacy boundary — reads stay source-granular; scope any OSB implementation the same way to avoid implying read isolation.+++## t_c0fce0b9 (board priority 3) - [upstream:iwe] Template-mode note creation with typed variables, pre-write schema validation, and idempotent if_exists:skip++**Source**: https://github.com/iwe-org/iwe/releases/tag/iwe-v0.17.0+**Repo**: iwe-org/iwe (1,086★)+**Released**: iwe-v0.17.0 (2026-07-29T01:26:08Z)++## What+Extend Open Second Brain's note-write primitive with a second creation mode alongside today's verbatim CONTENT path: a TEMPLATE mode that names a stored template and fills it with variables — `--var NAME=VALUE` (verbatim string), `--vars-yaml`/`--vars-json` for a whole typed variable set so templates can branch on booleans and loop over lists — plus `--set FIELD=VALUE` frontmatter fields written above the rendered body. Add a `--strict` flag that validates the resulting document against the Brain schema before writing, and an `if_exists: "skip"` option that makes a repeated create a safe no-op (existence checked against the file on disk). Mirror the same shape into the `brain_create_note` MCP tool so an agent writes exactly the document it intended.++## Why useful for OSB+Agents and users get repeatable, schema-correct note creation: structured note types (decisions, sources, MOCs) render from one canonical template with typed inputs instead of hand-assembled markdown, `--strict` catches malformed frontmatter before it lands in the vault, and `if_exists: "skip"` lets automations and cron flows call create idempotently without pre-checking or catching the loud "exists" error.++## Status in OSB+- **Verdict**: not_in_osb_useful+- **Codegraph hints**:+ - CONTENT-only, no template/variables: `CreateNoteInput` has only `path`/`frontmatter`/`content` in `src/core/brain/notes/create-note.ts`.+ - No idempotent skip — loud refusal on clobber: `CreateNoteErrorCode` includes `"exists"`, writer called with `overwrite: false` + `existsErrorKind: "note"` at `src/core/brain/notes/create-note.ts:25,134-135`.+ - No note-body template engine: existing "templates" are config/export/cron only — `src/core/brain/config-template.ts`, `src/core/brain/templates.ts`, `renderCronTemplate` in `src/cli/search/verbs/indexing.ts`.+ - No `--strict` pre-write schema gate: schema vocabulary exists via `cmdBrainSchema` (`src/cli/brain.ts:283-284`, `src/cli/command-manifest.ts:185-189`) but is not wired into `createNote`.+ - Building blocks present: `writeFrontmatterAtomic` and frontmatter split/parse in `src/core/vault.ts` parallel iwe's `split_raw_frontmatter`/`prepend_frontmatter` additions.++## Notes+Scope naturally into three layers that can ship independently: (1) `if_exists: "skip"` idempotent create — smallest, but note it deliberately relaxes the current "refuse loudly, never a silent skip" contract, so it must be opt-in and leave the default refusing; (2) `--strict` pre-write schema validation reusing the existing Brain schema vocabulary; (3) the template engine with typed `--vars-yaml`/`--vars-json` variables, the largest piece. All three should land in both the CLI and the `brain_create_note` MCP tool to keep parity.+++## t_ac1c4176 (board priority 3) - [upstream:gbrain] Content-derived date anchor for temporal ranking when a note's mtime/created_at is not content-meaningful++**Source**: https://github.com/garrytan/gbrain/pull/2341+**Repo**: garrytan/gbrain (18439★)+**PR**: #2341 feat(extract): --infer-dates anchors timeline from a page's content date when its body has none (merged-date: unknown — not fetchable, non-interactive session)++## What+Adds an ingest-time concept of a *content date*: when a note carries no explicit date, the extractor infers one from the note's own content — a frontmatter `created`/`published`/`date` field or a date token in the body — and uses it as the temporal anchor for sorting/ranking instead of the crawl/mtime timestamp. In Open Second Brain terms, this is a new content-meaningful date column promoted from frontmatter/body, feeding temporal recall.++## Why useful for OSB+Open Second Brain notes are routinely back-dated — imported logs, meeting notes, historical references — yet carry today's `mtime`/`created_at`, and mtime drifts across synced machines, so a note written today about a 6-month-old event ranks as "recent." A content-derived anchor makes temporal recall correct for exactly these notes and lets the existing query-side `temporal-intent` (`since:`/`until:`/named periods) rank against when the content is *about*, not when the file was touched.++## Status in OSB+- **Verdict**: not_in_osb_useful+- **Codegraph hints**: `src/core/search/schema.ts:102` (`mtime INTEGER`), `schema.ts:104` (`created_at TEXT`) — both filesystem/crawl timestamps, no content-date column; `src/core/search/indexer.ts:295` (`file.stat.mtimeMs` fastpath, `created_at` set at index time, no body/frontmatter date extraction); `src/core/search/chunker.ts` `extractFrontmatter` and `src/core/vault.ts` `parseFrontmatterTextWithNotices` (frontmatter parsed and indexed as a block but date-ish fields not promoted); `src/core/search/temporal-intent.ts` (query-side `since:`/`until:`/named-period detection ranks off `mtime`/`created_at` only). No "content date" / "body date" / "infer date" concept found in `src/core` — confirmed absent.++## Notes+- Query and ingest are separable: the query side (`temporal-intent.ts`) already exists; the gap is a stable ingest-side anchor for it to rank against. A new nullable `content_date` column (fall back to `mtime` when unset) keeps existing ranking behavior intact for notes with no inferrable date.+- Date extraction must stay language-agnostic per repo convention (`pref-language-agnostic-search`): prefer explicit frontmatter fields (`created`/`published`/`date`) and structural ISO-like tokens over natural-language date phrases; avoid hardcoded month-name word lists across locales.+- Edge cases: multiple candidate dates in one body (need a precedence rule — frontmatter over body, earliest/latest), ambiguous formats (DD/MM vs MM/DD), and future dates. Consider recording provenance (which source the anchor came from) so mis-anchoring is auditable.+++## t_39ec3fef (board priority 3) - [upstream:gbrain] Token-budget embedding batch packing to respect provider per-request input-token caps++**Source**: https://github.com/garrytan/gbrain/pull/3651+**Repo**: garrytan/gbrain (18,439★)+**PR**: #3651 declare Gemini embedding batch-token budget (2026-07-31)++## What+Add a per-request token budget to embedding batching so batches are packed up to an accumulated-token limit rather than a fixed chunk count. The batcher would sum each text's token estimate and close a batch once the next chunk would exceed the configured provider input-token cap, falling back to the count cap for cheap endpoints that only limit by chunk count.++## Why useful for OSB+OSB's embedding batches are count-only (`batchSize` 32), so a worst-case batch of 32 × ~800-token chunks is ~25,600 input tokens in one request — enough to blow a token-capped provider's per-request limit. A token budget lets OSB target Gemini and similar token-capped embedding endpoints without 429/413 errors while keeping dense batches for count-limited providers.++## Status in OSB+- **Verdict**: not_in_osb_useful+- **Codegraph hints**:+ - Count-only batching: `src/core/search/embeddings/openai-compat.ts:269-271` (`chunkArray(texts, this.config.batchSize)`), sent per-request at `openai-compat.ts:286,340` (`embedBatchWithRetry`).+ - Default `batchSize: 32` at `src/core/search/index.ts:148`; validated as `embedding_batch_size` (min 1) at `src/core/search/index.ts:322`. No `embedding_batch_tokens`-style budget exists.+ - Chunks already token-sized: `src/core/search/chunker.ts` (`maxTokens` default 800), so ~800 tokens/chunk × 32 = worst-case ~25,600 input tokens/request.+ - Token-cap 429/413 responses hit hard-quota handling `QUOTA_ERROR_TOKENS` at `src/core/search/embeddings/openai-compat.ts:79-97`, not transient size handling.+ - No Gemini-specific provider — Gemini embeddings route through the generic `openai-compat` path.++## Notes+- Reuse the chunker's existing token-estimation logic (`src/core/search/chunker.ts`) so batch-token accounting matches how chunks were sized; avoid a divergent tokenizer.+- Budget should be additive to the count cap (whichever fills first closes the batch), preserving current dense batching for count-limited endpoints.+- Consider distinguishing token-cap size errors (413 / token-limit 429) from true quota exhaustion in `QUOTA_ERROR_TOKENS` handling so an oversized batch can be split and retried rather than treated as a hard quota stop.+- Interacts with `embedding_concurrency` semaphore slots (one batch per slot) — token budgeting changes batch count but not the concurrency model.+++## t_76b89833 (board priority 2) - [upstream:EverOS] Optional embedding/rerank with capability tiers and deferred vector backfill at ingest++**Source**: https://github.com/EverMind-AI/EverOS/releases/tag/v1.2.1+**Repo**: EverMind-AI/EverOS (6,886★)+**Released**: v1.2.1 (2026-07-29T17:21:27Z)++## What+Make embedding and rerank soft runtime dependencies so Open Second Brain boots with only an LLM configured and degrades into capability tiers: LLM-only → keyword search + writes + cascade sync; +embedding → vector/hybrid search + reflection + skill extraction; +rerank → agentic search. Rows ingested without an embedding provider are stored with NULL vectors and filled in retroactively by a dedicated backfill CLI (vectors → clusters → skills), instead of aborting startup or failing the write.++## Why useful for OSB+Today ingestion fails loud when no embedding provider is wired (`NullProvider.embed()` throws), so an operator with only an LLM cannot write to the vault at all. Tiered degradation plus deferred backfill would let agents keep capturing knowledge while embeddings are unavailable or unconfigured, then enrich those rows once a provider appears — removing embedding as a hard boot/ingest gate.++## Status in OSB+- **Verdict**: not_in_osb_useful+- **Codegraph hints**: `src/core/search/embeddings/null-provider.ts` (`embed()` throws `SearchError("EMBEDDING_DISABLED")` — fails loud, cannot persist NULL-vector rows at ingest); `src/core/search/index.ts:331-343` (`embedding_provider` = `openai-compat | zeroentropy | local | disabled`, `disabled` → NullProvider, no tier model); `src/core/search/indexer.ts:175` and `indexer.ts:594` (binary semantic on/off + offline lexical fallback at SEARCH time, not a tiered capability model nor a write-now-embed-later ingest path); `src/cli/search.ts` / `src/cli/search/verbs/indexing.ts` (`o2b search reindex --embeddings` re-embeds already-vectored chunks on model change, not deferred NULL-vector backfill); `src/cli/brain.ts:295-298` (`brain semantics-backfill` / `brain authored-at-backfill` backfill metadata fields, not deferred embedding vectors). No tiered capability model or NULL-vector-await-backfill concept found.++## Notes+- Adapting to OSB implies: a tier resolver keyed on which providers are configured; a schema allowing NULL vectors (parallel to EverOS's LanceDB schema v2); a NULL-vector variant of NullProvider (or a distinct "deferred" mode) that defers rather than throws; a new `cascade backfill` CLI with vectors → clusters → skills phases; and an "unbackfilled rows" startup banner.+- The existing content-hash manifest in the reindex path can likely be reused to drive the vector-backfill phase.+- Consider mapping EverOS's `ProviderNotConfiguredError → HTTP 422 CAPABILITY_UNAVAILABLE` onto OSB's `SearchError` codes for read-time capability gating.+++## t_77efc212 (board priority 2) - [upstream:hindsight] Named shared observation scope for explicitly sharing extracted observations across contexts++**Source**: https://github.com/vectorize-io/hindsight/releases/tag/v0.8.3+**Repo**: vectorize-io/hindsight (15638★)+**Released**: v0.8.3 (2026-06-18T09:36:01Z)++## What+Upstream adds a `shared` keyword to the `observation_scopes` taxonomy used during consolidation, so an extracted observation can be explicitly marked as shareable across contexts/sessions rather than being confined to private/local-only scoping. For OSB this maps onto the continuity-record envelope: today records carry `private`/`redacted` flags, but there is no positive, named scope that says "this observation is intended to be shared/promoted beyond its origin context."++## Why useful for OSB+A first-class `shared` scope gives extraction and read-side gating a clear, intentional signal for cross-context promotion instead of inferring it from the absence of `private`. That sharpens recall (consumers can opt into shared observations deliberately) and makes the privacy model symmetric — explicit private vs. explicit shared rather than private vs. default.++## Status in OSB+- **Verdict**: unverified+- **Codegraph hints**: unverified — codegraph MCP unavailable in this run; textual search found docs/observability.md:32 (continuity envelope fields `private`/`redacted`), docs/observability.md:77 (read-side consumers drop `private` by default, keep only on explicit request), and docs/architecture.md:208 (an always-load scope naming decision), but no explicit shared observation scope model was found.++## Notes+The current OSB model appears to express scope negatively (drop `private` unless requested) rather than via an explicit shared/promotable scope. Before implementing, confirm against the actual scope enum / continuity-record schema whether a `shared` value already exists under a different name (e.g. an "always-load" scope per architecture.md:208) — if so this may be present_parity rather than a net-new addition.+++## t_50033859 (board priority 1) - [upstream:plur] Add HTTP pack install/preview for Brain schema packs (fetch a pack from a URL)++**Source**: https://github.com/plur-ai/plur/releases/tag/v0.16.0+**Repo**: plur-ai/plur (154★)+**Released**: v0.16.0 (2026-07-28T19:57:14Z)++## What+Upstream `plur packs install <url>` / `plur packs preview <url>` (#746) fetch a pack over HTTP: preview inspects the remote pack before applying, install pulls it down and registers it. This turns packs from a purely local artifact into a shareable, distributable unit addressable by URL.++## Why useful for OSB+OSB already has a pack concept — Brain **schema** packs (token/type/alias/prefix/link-type bundles) surfaced via `schema_inspect(view: packs | active_pack)`. Today those packs are local/schema-driven only: authored or shipped in-repo, with no way to pull a community-authored or team-shared schema pack by URL. A `preview <url>` + `install <url>` path would let the community distribute domain schema packs (e.g. a research-vault taxonomy, a legal-notes ontology) the same way plur distributes memory packs — a natural fit for OSB's "bring your own vault shape" story. Preview-before-apply also matches OSB's cautious, user-owns-the-data posture: inspect exactly what tokens/types a remote pack would mutate before it touches the schema.++## Status in OSB+- **Verdict**: not_in_osb_useful+- **Codegraph hints**: Pack tooling exists but is local-only — no HTTP fetch-and-install. `src/mcp/brain/pack-tools.ts` (Brain schema pack tools, local/schema-driven); pack views exposed via `schema_inspect(view: packs)` / `schema_inspect(view: active_pack)`. Grep for a URL/HTTP pack-install path returned no hits — the only network-adjacent code found is `src/core/install/opencode-plugin-asset.ts` (in-repo plugin packaging) and `src/core/search/embeddings/local-provider.ts` (local embeddings), neither of which fetches-and-installs a pack. The "install/preview from URL" capability is absent.++## Notes+- Scope this to OSB's schema packs, not a general memory backend — the two projects' "pack" refer to different artifacts.+- Ship `preview` before `install`, and require it: show the exact schema mutations (new/renamed tokens, types, aliases, link-types) a remote pack would apply, so nothing mutates the vault schema sight-unseen.+- Local-first guardrails: pin/verify the fetched pack (checksum or signature), sandbox it to schema-only changes (no arbitrary code execution on install), and keep it fully opt-in. A remote URL is an external-input trust boundary — treat a fetched pack as untrusted until previewed (ties into the quarantine-lane idea on t_444349f2).+++## t_b654e25d (board priority 2) - [upstream:contextlattice] Split verification ledger — independent VERIFIER-witnessed result distinct from the acting agent's outcome claim, with matched-control causal gain++**Source**: https://github.com/sheawinkler/contextlattice/releases/tag/v3.20.0+**Repo**: sheawinkler/contextlattice (61★)+**Released**: v3.20.0 (2026-07-18T07:02:33Z)++## What+Add a two-record verification ledger where the acting agent posts an outcome CLAIM and a *separate* verifier posts an independently-witnessed RESULT (own verifier-id, verifier-kind, evidence-digest sha256, verification-passed bool), joined to the claim by exact project/agent/session/sample identity. Causal-gain claims require a leakage-free matched control (identical task/assignment/model/runner/harness/reconstruction/exact model-visible token count/tokenizer); without a control only observed verified yield is reported. Persist fsync-backed and restart-safe; 503 on persistence failure rather than acknowledging a memory-only claim.++## Why useful for OSB+OSB's outcome loop is single-actor: the same agent that did the work posts the OUTCOME ROW and the calibration record by the same sample id (deriveOutcome, recordTokenImpactOutcome). Nothing witnesses that claim independently, so a self-serving or buggy agent can inflate its own token-savings/utility with no adversarial check. A distinct verifier record + matched-control gating would give OSB trustworthy, non-self-reported utility accounting.++## Status in OSB+- **Verdict**: not_in_osb_useful+- **Codegraph hints**: single-actor outcome path exists — src/core/brain/context-pack-outcome.ts:322 (deriveOutcome -> TokenImpactOutcome), src/core/brain/token-impact.ts:218 (recordTokenImpactOutcome, both keyed by the acting agent's sample). Trust primitives that could seed a verifier but do NOT record an independently-witnessed pass: src/core/brain/trust/compute-verification-delta.ts:58 (computeVerificationDelta, dream-phase claims-vs-disk, no separate verifier actor), src/core/search/evidence-verification.ts:67 (buildEvidenceVerification, retrieval IDF/recall coverage), src/core/brain/trust/self-approval-guardrail.ts:41 (will/preference promote|quarantine, not utility). No verifier-id / evidence-digest / verification-passed record and no matched-control causal-gain semantics found.++## Notes+Related but distinct: src/core/brain/reconcile-outcomes.ts is dream-phase contradiction reconcile, not verification. Scope the new record to sit alongside the existing token-impact ledger, keyed by the same sample id.+++## t_ccb05134 (board priority 2) - [upstream:contextlattice] Skill efficacy receipt chain — outcome-verified lifecycle, discriminating-term ranking floor, SHA-256 identity collapse++**Source**: https://github.com/sheawinkler/ContextLattice/releases/tag/v4.0.6+**Repo**: sheawinkler/contextlattice (61★)+**Released**: v4.0.6 (2026-07-31T08:48:50Z)++## What+Add an efficacy receipt chain to Open Second Brain's skills catalog: skills advance through four lifecycle stages (searched → selected → invoked → independently-verified-outcome), where search/attachment alone earns no efficacy credit and only an independently-verified outcome closes the chain. Harden ranking with a deterministic discriminating-term coverage floor so generic query words can no longer dominate skill selection, and collapse byte-identical SKILL.md files to a single SHA-256 identity while retaining every source path as provenance. An efficacy-review pass emits only bounded candidates (inactive-retain, note, revision, retirement, abstention) and never edits, activates, or executes a skill.++## Why useful for OSB+Open Second Brain today scores skills deterministically per turn but keeps no record of whether an attached skill was actually selected, invoked, or helped — so brain_skill_proposals can only reason about relevance, not proven usefulness. Outcome-verified receipts let proposals rank and retire skills on evidence, the discriminating-term floor stops broad boilerplate queries from surfacing weak matches, and SHA-256 collapse deduplicates identical skills across roots without losing provenance. The bounded, read-only review pass fits OSB's existing observe-don't-mutate posture for the skill surface.++## Status in OSB+- **Verdict**: not_in_osb_useful+- **Codegraph hints**: No efficacy/receipt/lifecycle tracking exists — grep for skill usage/outcome/efficacy/receipt/verified-outcome/invoked-skill/skill-rank/discriminating found only unrelated "discriminator" union fields (write-batch, decisions, session-lifecycle). Current skill surface: discovery `src/core/surface/skills.ts` (`discoverSkills`/`readSkillFile`/`skillRoots`); per-turn relevance `src/core/surface/skill-attach.ts` (`buildSkillAttachment`); MCP tools `src/mcp/skill-tools.ts` (`list_skills`/`get_skill`/`skills_attach`/`brain_skill_proposals`) + `src/mcp/registry-guard.ts:131,136`; config `src/core/config.ts:600-612` (`resolveSkillsDir`) and default-OFF gates `src/core/config.ts:638-652` (`resolveSkillAutoAttach`/`resolveSkillsAttachTriggers`). Ranking is deterministic relevance only, with no discriminating-term floor and no SHA-256 identity/provenance collapse.++## Notes+Scope aligns with OSB conventions: the review pass stays read-only (proposals only, never mutating/activating skills), matching the default-OFF auto-attach gates. Per OSB's language-agnostic-search preference, the discriminating-term coverage floor must derive discrimination from corpus document-frequency/IDF over skill text, not any hardcoded natural-language stop-word list.++# Project context++Open Second Brain - TypeScript on the Bun runtime; a CLI (`o2b`) plus an MCP server over an Obsidian-compatible Markdown vault. Around 900 modules under `src/`, ~8100 tests under `tests/`.++Recent commits on main:+7e6a5672 refactor: module boundaries and the fallbacks behind them (v1.42.0) (#153)+5ac866eb feat: signals that survive (v1.41.0) (#152)+0963ef0a feat: no dead ends - every diagnosis names its exit (v1.40.0) (#151)+f91a698b feat: context integrity gates (v1.39.0) (#150)+c31a2574 feat: semantic-health baseline watermark (v1.38.0) (#148)+b0c37977 feat: retrieval quality and context delivery (v1.37.0) (#146)+842d690f feat: knowledge intake and consolidation (v1.36.0) (#145)+95dc8577 feat: trusted recall and memory write surface (v1.35.0) (#144)+426d06f8 fix(vault): parse block-style YAML lists in frontmatter (not just inline arrays) (#142)+4b8100ca feat: source pipeline integrity and operator tooling (v1.34.0) (#143)+77513f2b feat: belief lifecycle and decision memory (v1.33.0) (#141)+61e93d24 fix(config): derive vault store reference from a keyed installation secret (#140)+9a649dd6 feat: memory write-path integrity and store safety wave (v1.32.0) (#139)+f2a037eb feat: today operator surface - dashboard, open loops, marker write-back (v1.31.0) (#138)+13bde6c3 refactor: remove all import cycles, decompose search.ts (v1.30.1) (#137)+fd5661f9 feat: governance visibility - vitals scorecard + batch-inflation lint (v1.30.0) (#136)+a99b0e71 feat(brain): add o2b brain vitals scorecard + batch-concept-inflation lint (#135)+70fb36e1 feat: operability, safety & first-run experience (v1.29.0) (#134)+ac26a675 feat: retrieval & ranking quality (v1.28.0) (#133)+5cd52e70 fix(hermes): resolve o2b when memory provider PATH is tiny (v1.27.1) (#131)++Related files (verified anchors are inside each task body above):+- src/core/brain/intake/extract-intake.ts, src/core/brain/untrusted-source.ts, src/core/brain/preference.ts+- src/mcp/server.ts, src/mcp/tools.ts, src/mcp/brain/*.ts, src/core/scope-key.ts, src/core/graph/agent-scope.ts+- src/core/brain/notes/create-note.ts, src/core/vault.ts+- src/core/search/schema.ts, src/core/search/indexer.ts, src/core/search/chunker.ts, src/core/search/temporal-intent.ts+- src/core/search/embeddings/{openai-compat,null-provider,local-provider,contract,signature}.ts+- src/core/brain/continuity/read-model.ts, src/core/brain/continuity/emit.ts+- src/mcp/brain/pack-tools.ts, src/core/brain/portability/{bundle,okf}.ts+- src/core/brain/context-pack-outcome.ts, src/core/brain/token-impact.ts, src/core/brain/trust/*.ts+- src/core/surface/skills.ts, src/core/surface/skill-attach.ts, src/mcp/skill-tools.ts++Conventions:+- The kernel calls no model. Every decision in the deterministic path is a rule or a score table, never an LLM call.+- No natural-language word lists, in any language, anywhere in gating, classification, ranking or extraction. Structural signals, explicit frontmatter fields, or corpus document-frequency instead.+- A new flag or configuration key must be byte-identical in behaviour when absent.+- Findings and diagnoses resolve a REGISTERED diagnostic code through a single advisory rail; never a sentence assembled at the call site.+- Absence and inability-to-examine are different answers and must never be collapsed into one.+- No silent fallback, no stub, no placeholder. If something failed, the failure is named at the surface the caller can see.+- Tests never touch the real vault or the network; they build temp vaults.++Constraints:+- Do not change existing public CLI or MCP surfaces in a breaking way; additive only.+- No new runtime dependency without a strong argument; an embedding transformer or a network fetch is exactly the kind of dependency that needs one.+- The ten units land as separate atomic commits on one branch, implemented largely in parallel by separate agents, so the substrate they share must be decided up front rather than discovered during merge.+- Some units are explicitly scoped down by their own task body (for example the verification ledger's matched-control causal-gain semantics). Respect those carve-outs.++# Required output format++Produce exactly 3 distinct architectural variants. For each variant:++### Variant N: <short name>+- **Approach**: 2-3 sentences describing the variant.+- **Trade-offs**: bullet list of pros and cons.+- **Complexity**: small | medium | large+- **Risk**: low | medium | high++After the three variants, add exactly one recommendation:++### Recommended: Variant N+**Rationale**: 2-3 sentences explaining why this variant over the others, considering the project context and constraints above.++Output nothing outside of these sections.diff --git a/docs/brainstorm/provenance-at-the-boundary/design.md b/docs/brainstorm/provenance-at-the-boundary/design.mdnew file mode 100644index 00000000..ab770ffd--- /dev/null+++ b/docs/brainstorm/provenance-at-the-boundary/design.md@@ -0,0 +1,342 @@+# Provenance at the boundary - what enters the vault, under what authority, backed by what proof++**Status:** draft+**Author:** claude-dev-agent (via feature-release-playbook)+**Audience:** implementation++## Problem statement++Ten kanban tasks were selected together because they sit on one seam: the+boundary where content, authority and claims enter the vault. Open Second Brain+can say a great deal about what it holds and almost nothing about where any of+it came from, who was entitled to put it there, or what backs a claim made about+it. An entity extracted from a scraped page becomes a first-class Brain entity+on the same terms as one the operator typed. A write names any path it likes. A+note is created with whatever frontmatter the caller assembled. A record is+ranked by when its file was touched rather than by what it is about. An agent+posts its own success and nothing recomputes it.++A reconnaissance pass over the actual code preceded this document, and it+changed the work substantially. Load-bearing premises were falsified in nine of+the ten tasks. Two tasks describe a capability that cannot be built honestly on+this architecture at all. Three describe gaps that are already closed, while the+gap that is genuinely open sits one layer away and goes unmentioned. What+follows is scoped to what is true.++## Scope++Ten units, one branch, one release.++| Unit | Task | Ships |+| --- | --- | --- |+| A | t_444349f2 | A trust level originated at intake, a `quarantine` entity status that read paths honour, and a census that keeps it non-vacuous |+| B | t_a3c4b13b | A config-declared path-prefix write binding over caller-named write ops, plus a write-site census recording every path it cannot cover |+| C | t_c0fce0b9 | `if_exists: "skip"` with a discriminated result, `strict` pre-write validation over the existing validator, and template-mode creation with a closed two-construct grammar |+| D | t_ac1c4176 | A body-derived date anchor materialised at index time, schema v11, with provenance recorded for the anchor it chose |+| E | t_39ec3fef | A per-request token budget alongside the count cap, on both batching providers |+| F | t_76b89833 | One named capability-tier resolver shared by the four sites that compute those facts separately, a registered code in place of call-site prose, a capability predicate in place of a stringly-typed sentinel, and a vector-only backfill verb |+| G | t_77efc212 | A caller-declared scope on the continuity envelope, honoured by the read model, plus a census of which readers honour it |+| H | t_50033859 | Dry-run preview for schema mutations, so nothing mutates the vault schema sight-unseen |+| I | t_b654e25d | A second ledger record carrying the kernel's own on-disk evidence against the acting agent's claim |+| J | t_ccb05134 | An identified skill offer that an invocation can be joined back to, a discriminating-term ranking floor, and retained provenance where a skill is currently shadowed silently |++## Out of scope++Stated here rather than discovered at review, with the reason for each.++- **An independent verifier (part of t_b654e25d, and stage four of t_ccb05134).**+ Agent identity in this system is a string from an environment variable, else a+ config key, else the literal `agent`; fifteen modules under `src/mcp/brain/`+ declare twenty-two tool schemas that additionally accept a caller-supplied+ `agent` argument overriding it, accepted verbatim. There+ is no token, key, signature or channel binding anywhere, and the MCP server is+ a child process of the agent it labels. Two records asserting two names,+ chosen by one process from one config in one session, are not two actors.+ Shipping that under the word "independent" would be a false claim in a ledger+ whose entire purpose is to be trustworthy.+- **`fsync`-backed ledger persistence and refusal on persistence failure (part of+ t_b654e25d).** Continuity appends are deliberately unfsynced and the emit path+ is deliberately fail-open, on a documented invariant that telemetry must never+ fail the primary operation. Inverting that is a regression wearing a feature's+ clothes.+- **Fetching and installing a schema pack from a URL (part of t_50033859).** A+ schema pack is not an artifact. It is the `schema:` block inside the vault's+ `_brain.yaml`, and the pack listing returns one hardcoded entry. There is no+ registry, no packs directory, no second pack, no name-to-pack resolution, and+ no signature verification anywhere in the repository. Fetching a thing that has+ no portable representation is not implementable; inventing the distribution+ format is a different task. The preview half, which the task itself says must+ ship first and be required, is what ships.+- **Extending the write binding to derived-destination writes (part of+ t_a3c4b13b).** Most write operations do not name a page - daily logs, dream+ runs, continuity records and all telemetry derive their destination. A prefix+ fence can only refuse those wholesale, which is not a fence but an off switch.+ The census records the boundary instead of pretending it is not there.+- **Auditing the twenty continuity readers that bypass the read model (part of+ t_77efc212).** Nineteen of them predate this wave. The twentieth is unit I's+ own `context-pack-evidence.ts`, which calls `listContinuityRecords` on the+ store directly and is enumerated as such in the census+ (`tests/core/brain/continuity/reader-census.test.ts`). So this wave does not+ only make a pre-existing gap visible through the census in unit G - it adds+ one reader to that gap and defers the audit, because closing it is a+ judgement per reader.+- **Splitting an oversized embedding batch and retrying it (part of t_39ec3fef).**+ The error category is a closed five-member union with wait-or-fail semantics,+ consumed on a second surface that renders it to the operator. A sixth,+ orthogonal disposition is a two-surface change. The token budget prevents the+ oversized request instead, which is the actual fix.+- **A schema change for vectors, a deferred provider class, and a NULL-vector+ concept (part of t_76b89833).** Vectors live in a virtual table that cannot+ hold a null vector, and it never needs to: a chunk without a vector is a chunk+ row with no embeddings row, which is already a first-class, already-queried,+ already-resumable state.+- **A new command-line verb for note creation (part of t_c0fce0b9).** The task+ asks for command-line and tool parity. No command-line verb reaches note+ creation today; inventing one is wider than the task.++## Chosen approach++The consultant produced three variants and recommended the third; the audit+trail with the full text and the orchestrator's two amendments is in+[`variants.md`](variants.md). The wave is factored into two substrates and one+rail, because the six cross-cutting concerns cluster into two and not one.++**Substrate one - the intake boundary.** Provenance, authority and validation+genuinely meet where content arrives. One trust level originated where content+enters (unit A), one path-prefix authority decision for writes that name their+destination (unit B), one validation gate before a note lands (unit C). Unit D's+anchor provenance and unit G's declared scope consume the same vocabulary+without being dragged into the same code path.++**Substrate two - the attestation ledger.** Claim and proof meet at the outcome+record. One witnessed-record shape keyed by the sample id that already joins two+records today, carrying evidence the acting agent cannot author (unit I), and one+receipt chain that can actually advance because the offer it starts from now has+an identity (unit J).++**The capability rail.** Whether a capability is configured is a different+question from whether a call failed, and today four sites answer the first+question separately and one of them answers it in prose assembled at the call+site. Unit F makes that one resolver behind one registered code, and units E and+H consume the same distinction.++Both substrates need a content digest. Per the first amendment, that digest is+not left to whichever unit needs it first: the existing manifest and stamp+helpers are the source, and no unit introduces a second hashing path.++## Design decisions++Each decision below is a place where the code contradicted the task, or where+two honest implementations existed and one had to be chosen.++**A1. The trust signal is originated, not reused.** The task says to reuse+`UNTRUSTED_SOURCE_TAG`. That constant is a prompt-delimiter tag name used to+wrap text on its way to a model; the module states it never rewrites a note, and+its four consumers are all model-payload assembly running nowhere near intake.+There is no untrusted signal at intake to join to. So intake originates one. The+source ingest path currently hardcodes a single trust level for every source,+which makes a scraped URL and a local file indistinguishable; that is itself the+defect, and it is fixed here rather than worked around.++**A2. The dangling read is closed by giving it a writer.** The retrieval gate+reads an `untrusted_source` frontmatter key. Nothing in the repository writes it;+the only other occurrences are two tests that hand-construct it. It is a gate+branch reachable only from a field no writer produces. The intake path now writes+it when provenance is untrusted. The task's instinct was correct and its stated+mechanism was not.++**A3. A quarantine that nothing filters on is a no-op, so the filter is+centralised.** Entities already carry a status. The cost is not the enum: the+`active` filter is re-implemented per caller across sixteen read sites, eight of+which apply no status filter at all and would treat a new value as visible. The+filter becomes one predicate those sites share, and a census fails the build if a+new entity read path bypasses it. This project reverted a unit in the previous+release for being unable to fire; the same failure mode is designed out here+rather than hoped against.++**B1. The binding's authority is the config file, never the caller's claim.**+There are no credentials. The nearest identity is self-asserted and additionally+overridable by a caller-supplied argument on twenty-two tool schemas, so a fence keyed+to it is bypassed by passing a different string. The binding is declared in the+vault config, which the operator controls and which no MCP call can rewrite. It+is described as a write boundary over caller-named paths, never as a security+boundary and never as per-credential.++**B2. The census is part of the deliverable, not a follow-up.** Sixty-two sites+write inside the vault through `node:fs` directly, bypassing every shared+helper - including one that writes a complete vault note with hand-rolled+frontmatter, and one in a command-line verb. That figure is the census's, not+this document's: `DIRECT_WRITE_EXCLUSIONS` in+`tests/core/architecture/write-site-census.test.ts` IS the count, and its own+docblock keeps the number out of prose precisely so a stated figure cannot drift+away from the list. Shipping a fence over a surface with that many known holes+and describing it as enforcement would be theatre. The census+enumerates every in-vault write site and requires each to either pass through a+shared writer or carry a written exclusion, in the same shape as the vault-guard+and terminal-state censuses this repository already runs. Two writers whose+divergence is a plain duplication defect are routed through the shared helper in+this unit; the rest become an asserted, readable list.++**C1. A skip must be distinguishable from a create.** The result type currently+declares creation as a literal, so a skip cannot be expressed at all. Returning+the same shape for both would be precisely the silent fallback this wave forbids.+The result carries a discriminated outcome, so a caller that skipped can never+read as a caller that created. Skip is decided on the existing exclusive-link+refusal, not by relaxing the overwrite flag, and returns before parent+directories are created so a no-op leaves nothing behind.++**C2. Validation is wiring, not machinery.** A document validator that takes an+artifact and returns coded violations already exists and is already bound to the+schema vocabulary. The sibling path validator is not reusable - its policy is the+exact complement of the note resolver's - and only the artifact half is used.++**C3. The template grammar is closed at two constructs.** Typed variable+substitution with canonical rendering, plus presence-driven sections and list+iteration. No expression language, no general-purpose mini-language. Unknown+placeholders are preserved intact, because the existing renderer documents that+rule deliberately so a typo surfaces in the output rather than vanishing.++**D1. The gap is the body, not the anchor.** Ranking already uses a+content-derived event time from frontmatter. The query-side module the task+names as the existing half in fact ranks nothing - it is pure query-text parsing.+What is missing is a date derived from the note's body, and materialisation at+index time so a candidate's body is not re-read on every query.++**D2. The existing extractor is reused, minus its clock.** A language-agnostic+body-text date extractor already exists, with negative tests in three+non-English scripts asserting that word forms are not recognised. No second+extractor is written. Its duration branch anchors to the current time and must+not be reached at ingest, because a stored value derived from a clock ages+silently while both index fastpaths, which gate on content identity, correctly+decline to recompute it.++**D3. Slash-formatted dates are not recognised, and future dates are kept.**+Resolving day-month against month-day requires a locale, and a locale is a+natural-language signal. The existing query-side parser refuses an entire class+rather than encoding such an assumption, and this follows it. A future date is a+real declaration and is stored; dropping it would be inference dressed as+validation.++**D4. A derived point becomes a day-width window.** Three time filters take an+interval rather than a point, and the task does not mention them. A bare date+snaps to day start and day end, matching what the existing validity parser+already does, so the anchor composes with those filters instead of being wired+only into the ranking boost.++**E1. The budget uses the estimator already resident in the embeddings layer.**+The task says to reuse the chunker's estimator to avoid a divergent tokenizer.+That estimator is not exported, and the premise is wrong in a more interesting+way: a second estimator already serves this layer and governs the cost gate. Two+notions of a token already exist and they differ by roughly a quarter on prose.+Using the layer's own means the cost gate and the batch budget describe the same+texts identically. The budget accounts for the instruction prefix the provider+prepends before batching.++**E2. Both batching providers get the budget.** A second provider carries the+identical count-only shape and would silently keep it.++**F1. The tier facts exist; the resolver does not.** Four sites compute whether+semantic search is available and why, separately. One of them assembles+operator-facing sentences at the call site, contrary to the rail this repository+adopted two releases ago. They share one resolver behind one registered code.++**F2. A capability predicate replaces a stringly-typed sentinel.** Four sites+test a provider by comparing its name to the literal `null`. Any new provider+name bypasses all four silently. The contract answers the question instead.++**G1. The scope is declared by the caller, because the alternative is a second+inference.** The premise that this system has explicit private and implicit+default is false: private is derived from a marker found in the record's own+content. A content marker for shared would be a second inference, which is the+thing the task objects to. An optional declared field is added, absent behaving+byte-identically to today.++**G2. The bypass becomes an asserted fact.** Four of twenty-four continuity+readers go through the filtering read model; the other twenty read the store+directly and already bypass the private drop, while the documentation states the+policy as universal. A census records which readers honour the read model, and+the documentation is corrected. Auditing the twenty is out of scope and said so.+The counts here are the measured ones; the reconnaissance pass reported five of+twenty-one because one direct reader carries a literal NUL byte and was+invisible to its grep.++**H1. Preview is the deliverable.** The mutation application function is already+pure and already runs full validation, so a dry run returning the resulting pack+and its diff touches no disk and reuses everything.++**I1. The kernel's own evidence, not an independent verifier.** The second+record carries the acting agent's claim, the digest the kernel wrote at pack time+and reads back off disk, and the match verdict. Read back rather than recomputed:+re-deriving it would mean hashing text the receipt does not retain, which is the+second hashing path the wave's first amendment forbids. The agent still cannot+author it, and the record says only what is on disk. There is no+verifier-identity field, and no surface describes the record as independently+witnessed.++**I2. The sibling asymmetry is fixed while the file is open.** One outcome record+carries an actor field and its sibling carries none.++**J1. The chain starts at the offer, because that is the missing link.** Skill+invocation is already observed and already aggregated; the task's claim that it+is not is false. What is missing is that an invocation record carries no+reference to the offer that produced it, so the two cannot be joined. The offer+gains an identity.++**J2. No third inverse-document-frequency implementation.** Two exist - one over+the skill descriptor corpus, one over the vault corpus with an explicit+no-stopword-list design. The floor combines the shape of the second with the+corpus of the first.++**J3. Provenance is retained where a skill is currently shadowed.** Duplicate+skills do not reach the ranker; they collapse on a name key first. The real+defect is the other half of that collapse - the shadowed path is discarded+silently and the entry has nowhere to record it. That is what is fixed, rather+than building a deduplicator for a case that does not occur.++## File changes++Expected, per unit. Implementers own their unit's files and report rather than+edit another unit's.++- **A** - `src/core/brain/intake/extract-intake.ts`, `src/core/brain/ingest/ingest.ts`, `src/core/brain/entities/{types,registry}.ts`, `src/mcp/brain/ner-tools.ts`, `src/core/brain/trust/retrieval-gate.ts`, a shared entity-status predicate module, and a new census test.+- **B** - a new write-binding module under `src/core/`, `src/core/vault.ts` and `src/core/brain/write-batch.ts` at the enforcement points, `src/core/brain/handoff.ts`, `src/cli/brain/verbs/links.ts`, config policy for the binding, and a new write-site census test.+- **C** - `src/core/brain/notes/create-note.ts`, a new template module, `src/mcp/brain/notes-tools.ts`.+- **D** - `src/core/search/{schema,indexer}.ts`, `src/core/search/store/documents.ts`, `src/core/search/pipeline/{candidate-signals,event-time}.ts`, reusing `src/core/brain/temporal-extract.ts` unchanged where possible.+- **E** - `src/core/search/embeddings/{openai-compat,zeroentropy}.ts`, `src/core/search/{index,types}.ts`.+- **F** - a new tier-resolver module under `src/core/search/`, `src/core/search/{semantic-phase,indexer}.ts`, `src/core/doctor-readiness.ts`, `src/core/search/embeddings/contract.ts`, `src/core/brain/diagnostics.ts`, a new backfill verb and its planner.+- **G** - `src/core/brain/continuity/{types,store,read-model}.ts`, `docs/observability.md`, a new census test.+- **H** - `src/core/brain/schema-mutate.ts`, `src/mcp/schema-tools.ts`.+- **I** - `src/core/brain/{context-pack-outcome,token-impact}.ts`, `src/core/brain/continuity/types.ts`, `src/mcp/brain/recall-tools.ts`.+- **J** - `src/core/surface/{skills,skill-attach}.ts`, `src/core/brain/skill-usage.ts`, `src/mcp/skill-tools.ts`, `src/mcp/registry-guard.ts`.++Shared, owned by the orchestrator: `CHANGELOG.md`, `README.md`, `package.json`+and the mirrored manifests.++## Risks and open questions++- **The entity read paths are the largest single risk in unit A.** Sixteen sites,+ eight with no status filter. If centralisation turns out to change what an+ existing surface returns, that is a behaviour change and must be reported, not+ absorbed.+- **Unit B's census may find that a direct writer is unreachable from any+ agent-triggered path.** The reconnaissance did not establish that distinction+ and it changes how much of the surface the fence must cover. The census records+ what it finds; it does not assume.+- **Unit D bumps the schema version, and a bump triggers NO reindex.**+ `applyMigrations` (`src/core/search/schema.ts`) runs the pending migrations in+ place on an existing index and raises only when the version on disk is NEWER+ than the binary supports; nothing anywhere re-indexes on a bump. So on an+ already-populated index the migration alone does not populate the body-derived+ date anchor for documents it leaves untouched: the schema v11 column exists and+ stays empty for them until those documents are re-indexed. The existing+ migration test asserts the version literal and needs a sibling. A migration+ that strands data on disk fails no test, so the migration is checked against a+ rewound index rather than only a fresh one.+- **Unit C's template grammar is the one place a mini-language could grow.** The+ two constructs are a hard boundary; anything needing a third is out of scope+ for this release.+- **Unit J depends on the offer and the invocation being observable in the same+ vault.** Invocation is recorded only when a session log is imported through an+ adapter, so the join is retrospective rather than live. That is a property of+ the existing mechanism, not a defect introduced here, and it is stated in the+ release notes rather than implied away.diff --git a/docs/brainstorm/provenance-at-the-boundary/plan.md b/docs/brainstorm/provenance-at-the-boundary/plan.mdnew file mode 100644index 00000000..230620f6--- /dev/null+++ b/docs/brainstorm/provenance-at-the-boundary/plan.md@@ -0,0 +1,76 @@+# Provenance at the boundary - implementation plan++Ten atomic units on `feat/provenance-at-the-boundary`, one commit each,+implemented largely in parallel. Every unit is test-first: the failing test runs+and fails for the expected reason before the implementation exists.++Standing rules for every unit, from the wave contract: no fallback that reads as+success, no stub, no hardcoded natural-language word list in any language,+absence and inability-to-examine stay different answers, a new flag or config key+is byte-identical when absent and a test proves it, diagnoses resolve a+registered code through the advisory rail, tests build temp vaults and make no+network calls, and no existing test may be edited to make a change pass.++## Tasks++### Task A: Trust at intake, and a quarantine that read paths honour+- **Files**: `src/core/brain/intake/extract-intake.ts`, `src/core/brain/ingest/ingest.ts`, `src/core/brain/entities/types.ts`, `src/core/brain/entities/registry.ts`, `src/mcp/brain/ner-tools.ts`, `src/core/brain/trust/retrieval-gate.ts`, a new shared entity-status predicate, `tests/core/brain/entities/*`, a new census test+- **Acceptance**: an entity extracted under untrusted provenance lands with `quarantine` status and is absent from every entity read surface; the same extraction under trusted provenance is byte-identical to today; the `untrusted_source` frontmatter key the retrieval gate reads now has a writer, proved by a test that drives the gate from a real intake rather than a hand-built fixture; the census fails when a new entity read path bypasses the shared status predicate, demonstrated by a test that adds one+- **Depends on**: none++### Task B: A config-declared write binding, and a census of what it cannot cover+- **Files**: a new write-binding module under `src/core/`, enforcement in `src/core/vault.ts` and `src/core/brain/write-batch.ts`, `src/core/brain/handoff.ts`, `src/cli/brain/verbs/links.ts`, the config policy block declaring the binding, a new write-site census test+- **Acceptance**: with no binding declared, every write path is byte-identical to today; with a binding declared, a caller-named write outside it is refused with a registered code naming the binding, and a write inside it succeeds; a caller-supplied agent argument cannot widen the binding, proved by a test that passes a different agent string and still gets refused; the census enumerates every in-vault write site and fails when a new direct-`fs` writer appears without a written exclusion+- **Depends on**: none++### Task C: Idempotent skip, pre-write validation, template-mode creation+- **Files**: `src/core/brain/notes/create-note.ts`, a new template module under `src/core/brain/notes/`, `src/mcp/brain/notes-tools.ts`, `tests/core/brain/notes/*`, `tests/mcp/brain-create-note.test.ts` additions+- **Acceptance**: default creation still refuses to clobber, with the existing tests untouched and green; `if_exists: "skip"` on an existing note returns a result a caller can tell apart from a create and leaves no parent directory behind; `strict` rejects a document the existing artifact validator rejects, reporting its coded violations, and is byte-identical when absent; template mode renders typed variables, a presence-driven section and a list iteration, and leaves an unknown placeholder intact; the frozen tool-name parity test is verified rather than assumed unaffected+- **Depends on**: none++### Task D: A body-derived date anchor, materialised at index time+- **Files**: `src/core/search/schema.ts`, `src/core/search/indexer.ts`, `src/core/search/store/documents.ts`, `src/core/search/pipeline/candidate-signals.ts`, `src/core/search/pipeline/event-time.ts`, a new schema-migration test, `tests/core/search/*`+- **Acceptance**: a note whose body carries an ISO date and whose frontmatter carries none ranks by that date under a temporal query; a note with neither is byte-identical to today; the chosen anchor's source is recorded as a registered token; a v10 index rewound and re-migrated preserves existing rows with the new column null; body text in three non-English scripts yields no anchor; a slash-formatted date yields no anchor; a future date is stored rather than dropped; the clock-relative branch of the reused extractor is unreachable from the ingest path, proved by a test that would catch a value drifting between two runs over unchanged content+- **Depends on**: none++### Task E: A per-request token budget for embedding batches+- **Files**: `src/core/search/embeddings/openai-compat.ts`, `src/core/search/embeddings/zeroentropy.ts`, `src/core/search/index.ts`, `src/core/search/types.ts`, `tests/core/search/*`+- **Acceptance**: with the budget unset, batching is byte-identical to today; with it set, a batch closes on the accumulated token estimate before the count cap when the estimate fills first, and on the count cap otherwise; the estimate includes the instruction prefix the provider prepends; both batching providers honour it; the config key is rejected at zero the way its sibling is, both as a string and as a programmatic override; the stale "three attempts" docblock and the super-batch comment are corrected+- **Depends on**: none++### Task F: One capability-tier resolver, one registered code, one predicate+- **Files**: a new tier-resolver module under `src/core/search/`, `src/core/search/semantic-phase.ts`, `src/core/search/indexer.ts`, `src/core/doctor-readiness.ts`, `src/core/search/embeddings/contract.ts`, `src/core/brain/diagnostics.ts`, a new backfill planner under `src/core/` and its verb under `src/cli/`, `tests/core/search/*`, `tests/cli/*`+- **Acceptance**: the four sites that computed tier facts separately return the same tier for the same configuration, proved by a test that drives all four; the deferred-reason prose is replaced by a registered code and the advisory-rail ratchet passes; a provider whose name is neither of the two current literals is classified by the contract predicate rather than by name, proved by a test provider; the backfill verb reports pending vector work in dry-run by default, applies only under the explicit flag, emits both report shapes, and records a registered log event whose append failure surfaces rather than being swallowed+- **Depends on**: none++### Task G: A declared continuity scope, and a census of who honours it+- **Files**: `src/core/brain/continuity/types.ts`, `src/core/brain/continuity/store.ts`, `src/core/brain/continuity/read-model.ts`, `docs/observability.md`, a new census test, `tests/core/brain/continuity/*`+- **Acceptance**: a record appended without a scope is byte-identical to today, field for field; a record appended with the shared scope is retained by a read-model filter that requests it and excluded by one that does not; the census names every continuity reader and asserts which go through the read model and which read the store directly, failing when a new reader joins either set without being listed; the documentation sentence stating the masking policy as universal is corrected to match+- **Depends on**: none++### Task H: Preview before a schema mutation lands+- **Files**: `src/core/brain/schema-mutate.ts`, `src/mcp/schema-tools.ts`, `tests/core/brain/*`, `tests/mcp/*`+- **Acceptance**: a preview returns the pack that would result and the difference from the current one, and the config file's modification time and bytes are unchanged afterwards; a preview of a mutation the validator rejects returns the same coded rejection an apply would, without writing; an apply with no preview behaves exactly as today+- **Depends on**: none++### Task I: The kernel's own evidence recorded beside the agent's claim+- **Files**: `src/core/brain/context-pack-outcome.ts`, `src/core/brain/token-impact.ts`, `src/core/brain/continuity/types.ts`, `src/mcp/brain/recall-tools.ts`, `tests/core/brain/*`+- **Acceptance**: posting an outcome also records a second entry, joined by the existing sample id, carrying the digest the kernel wrote at pack time and reads back off disk, and a match verdict; a claim contradicted by that evidence records a mismatch rather than being rejected or silently corrected; the record carries no verifier-identity field and no surface calls it independently witnessed; the outcome record now carries the actor field its sibling already had; the emit path stays fail-open and unfsynced+- **Depends on**: none++### Task J: An identified skill offer, a ranking floor, retained provenance+- **Files**: `src/core/surface/skills.ts`, `src/core/surface/skill-attach.ts`, `src/core/brain/skill-usage.ts`, `src/mcp/skill-tools.ts`, `src/mcp/registry-guard.ts`, `tests/core/surface/*`, `tests/mcp/*`+- **Acceptance**: an offer carries an identity that a subsequent invocation record can be joined to, proved end to end over an imported session fixture; the discriminating-term floor drops a skill matched only on terms common across the descriptor corpus and keeps one matched on a rare term, with no word list anywhere in the derivation; a skill shadowed by a same-named skill in a later root records the shadowed path instead of discarding it; any new tool surface declares a preview budget or is listed in the exemption map, so the registry guard passes+- **Depends on**: none++## Sequencing++All ten are independent by file ownership and can run concurrently. Two shared+concerns are decided here rather than at merge, per the design document's+amendments: the content digest comes from the existing manifest and stamp+helpers and no unit introduces a second hashing path, and the capability state+from task F answers what the operator configured while the existing search error+codes answer what a call attempted and could not do, with nothing reporting both.++Commits land in the orchestrator's hands, one per unit, after that unit's+formatter, linter, typecheck and tests are green.diff --git a/docs/brainstorm/provenance-at-the-boundary/variants.md b/docs/brainstorm/provenance-at-the-boundary/variants.mdnew file mode 100644index 00000000..22f24c37--- /dev/null+++ b/docs/brainstorm/provenance-at-the-boundary/variants.md@@ -0,0 +1,84 @@+# Provenance at the boundary - brainstorm audit trail++**Consultant:** Claude Code (`claude -p`), primary. Exit code 0, three parseable+variants plus a recommendation, so the Codex fallback was not run.++**Prompt:** [`cli-output/prompt.md`](cli-output/prompt.md). The task put to the+consultant was not any single kanban item but the wave-level architectural+question: how ten units that all sit on the vault's write boundary should be+factored so they share one substrate instead of ten bolt-ons.++## Variants, verbatim++### Variant 1: Shared vocabulary, enforced by conformance tests++- **Approach**: Ship no new shared runtime module. Instead, freeze a written contract up front — envelope field names (origin, trust grade, anchor source, scope), a registered code namespace for every refusal and every "capability absent" answer, and a canonical sha256 digest helper — and let each of the ten units implement against it in its own files. The only shared artifact landed before the parallel work is a conformance test suite that every unit's write path must pass.+- **Trade-offs**:+ - Pro: near-zero merge contention; ten atomic commits touch ten disjoint file sets, which is exactly what parallel implementation by separate agents wants.+ - Pro: additive by construction — nothing central changes, so "byte-identical when the flag is absent" is trivially true for all ten units.+ - Pro: reuses the rails that already exist (the advisory diagnostic rail, `SearchError` codes, `UNTRUSTED_SOURCE_TAG`, the preference lifecycle statuses) rather than superseding them.+ - Con: a credential write fence enforced by convention is not a fence — nothing structurally prevents a write path (existing or future) from skipping the check, which defeats the point of t_a3c4b13b.+ - Con: provenance drifts. The quarantine lane, the content-date anchor, the pack installer and the skill sha256 collapse will each grow their own near-identical origin descriptor, and the wave ends up with four provenance shapes instead of one.+ - Con: conformance tests only catch drift where someone thought to write the test; absence-vs-inability distinctions are easy to collapse silently in a unit nobody tested for it.+- **Complexity**: small+- **Risk**: medium++### Variant 2: Single write-admission pipeline++- **Approach**: Factor by control flow. Introduce one ordered admission chain — authority check, shape/schema validation, provenance stamp and trust routing, capability resolution, then land — and funnel every write-capable surface through it: create/append/update note, write-batch, intake extraction, ingest, schema mutation, pack apply. Each of the ten units becomes a stage implementation or a stage consumer rather than an independent code path.+- **Trade-offs**:+ - Pro: the write-prefix fence becomes genuinely unskippable, and every future write surface inherits it for free — the strongest possible answer to "authority to write".+ - Pro: one place defines what a refusal looks like, so the refusal contract, its registered codes and its tests exist exactly once.+ - Pro: quarantine routing (t_444349f2), `--strict` validation and `if_exists: skip` (t_c0fce0b9), and pack preview-before-apply (t_50033859) become three configurations of the same chain rather than three hand-rolled gates.+ - Con: blast radius across roughly every write surface in a ~900-module, ~8100-test codebase, with high odds of incidental behaviour change where the "absent flag is a no-op" guarantee is hardest to prove.+ - Con: it inverts the parallelism constraint — ten agents all editing one chain is contention concentrated precisely where the wave cannot afford it.+ - Con: it does not actually unify the wave. Embedding batch packing, capability tiers with deferred backfill, the content-date anchor, the verification ledger and skill receipts are not writes through this chain, so half the units sit outside the substrate and the "one coherent substrate" claim is only half true.+- **Complexity**: large+- **Risk**: high++### Variant 3: Two substrates plus a capability rail++- **Approach**: Accept that the six cross-cutting concerns cluster into two, not one, and factor accordingly. Substrate A, the intake boundary: a single provenance descriptor (origin, trust grade, digest) plus an admission decision that composes authority, validation and routing-to-quarantine, serving the quarantine lane, the write-prefix fence, template/strict/skip creation and HTTP pack install, and supplying the anchor-provenance field the content-date unit needs and the positive scope value the shared-observation unit needs. Substrate B, the attestation ledger: one witnessed-record shape (witness identity, witness kind, evidence digest, pass/fail) keyed by the existing sample id, serving the split verification ledger and the skill efficacy receipt chain. Alongside them, extend the existing registered-code rail with an explicit capability state — configured, absent, deferred — so embedding batch budgeting, tiered degradation with NULL-vector backfill, and the pack fetcher's network capability all answer "not configured" the same way, distinct from "nothing found".+- **Trade-offs**:+ - Pro: each shared piece is small enough for one agent to land as a seed commit before the eight dependent units fan out, satisfying the "decided up front, not discovered at merge" constraint without a central chokepoint.+ - Pro: the clustering is real — provenance, authority and validation genuinely meet at intake; claim-versus-proof genuinely meets at the ledger; capability-absence is a rail concern in both and in neither substrate. Nothing is forced together that does not share semantics.+ - Pro: gives enforcement teeth where it matters (one admission decision at intake, one witness-record identity) while leaving mechanically unrelated units — embedding token budgeting, the content-date column — as local additive changes that merely consume the vocabulary.+ - Pro: the capability rail is where the absence-versus-inability convention is naturally honoured once, instead of three times.+ - Con: three shared pieces must land before most of the wave, serialising the first stretch of an otherwise parallel effort.+ - Con: both substrates want a content digest and an origin notion; without deliberately sharing one digest primitive between them, the wave grows exactly the duplication it was meant to avoid.+ - Con: the boundary between the new capability state and the existing `SearchError` / advisory codes needs an explicit ruling, or two vocabularies will describe the same unconfigured provider.+- **Complexity**: medium+- **Risk**: medium++### Recommended: Variant 3++**Rationale**: The six concerns do not collapse into one substrate — intake admission and outcome attestation share almost no semantics, and Variant 2 pays a large-blast-radius, high-contention price to pretend otherwise while still leaving half the units outside the chain. Variant 1 is cheap and parallel-friendly but enforces the one concern that must be structural, authority to write, by convention alone, and will leave four divergent provenance shapes behind. Variant 3 buys real enforcement where it is load-bearing, keeps the seed commits small enough to land ahead of the fan-out as the parallel-agent constraint requires, and respects the per-task carve-outs by leaving the purely mechanical units local rather than dragging them into a substrate they do not need.++## Orchestrator decision++Variant 3 is adopted, in agreement with the consultant, with two amendments+recorded here because they close the two cons it names against itself.++**Amendment 1 - one digest primitive, named before the fan-out.** The+consultant's own second con is that both substrates want a content digest and+an origin notion, and that without deliberately sharing one they reproduce the+duplication the wave exists to avoid. So the digest helper is not left to+whichever unit needs it first: it is part of the seed commit, sourced from+whatever `src/core/integrity/` already provides, and both substrates consume+it. No unit may introduce a second hashing path.++**Amendment 2 - the capability rail must rule on its boundary with the existing+error vocabularies before any unit uses it.** The third con is that a new+capability state and the existing `SearchError` codes will otherwise both+describe an unconfigured provider. The ruling is made in the design document+rather than deferred: the capability state answers *what the operator has+configured*, and `SearchError` answers *what a call attempted and could not+do*. A provider that is absent by configuration is the former; a provider that+is present and refused the request is the latter. Nothing may report both.++The two rejected variants were rejected for the reasons the consultant gives.+Variant 1 is worth one further note: this repository has already shipped a+release whose central finding was that a ratchet enforcing a convention passed+vacuously over an empty set. A write fence enforced by conformance test rather+than by structure is the same class of artefact, and this project has direct+evidence of how that ends.diff --git a/docs/cli-reference.md b/docs/cli-reference.mdindex f38f80bf..c1032c3c 100644--- a/docs/cli-reference.md+++ b/docs/cli-reference.md@@ -58,6 +58,8 @@ o2b brain morning-brief Read-only session-start summary (since v0.21.0): to2b brain codec Deterministic lossless session codec (since v0.22.0): --compress | --expand over stdin or --in <file>; structured content preserved byte-for-byteo2b brain sources Read-only dashboard of signals by (agent, source_type) (since v0.22.0): active/processed + distinct-topic counts; --jsono2b brain schema Runtime schema report/admin (since v0.26.0): report|stats|lint|graph|explain|orphans|apply|sync; mutation writes are locked and audited+o2b brain schema apply --mutation '<json>' (repeatable) applies audited, locked mutations to Brain/_brain.yaml; --actor NAME --reason TEXT annotate the audit record+ --dry-run previews instead (since v1.43.0): it returns the pack that would result plus its leaf-level diff and writes nothing at all - no config write, no audit record, no lock file. Before v1.43.0 `apply` parsed the flag and mutated anyway. The preview runs the same pack validator the apply runs, but NOT the two checks that exist only because the apply writes (the vault-identity write guard, and the atomic writer's re-parse of the rendered YAML), so a batch that renders to unparseable YAML previews clean and fails on applyo2b brain watchdog Probe Brain config/dirs/search index and plan safe recovery (since v0.26.0): --remediate [--dry-run], --restore <run_id> [--force-restore], --jsono2b brain graph-export Serialise the vault knowledge graph (pages, wikilinks, typed relations) to a stable graph.json (since v0.22.0): stdout or --out <file>o2b brain graph-import Reconstruct vault page stubs from a graph.json (since v0.22.0): --mode skip|overwrite|merge; vault-guarded writes@@ -479,6 +481,42 @@ with no `Brain/` layer is now reported as a named `brain-root-absent`warning instead of `clean`, which means `o2b brain doctor --strict`exits 2 on an un-initialized vault where it previously exited 0.+### Write binding (since v1.43.0)++An optional `write_binding:` block in `Brain/_brain.yaml` declares where a+CALLER-NAMED write may land:++```yaml+write_binding:+ path_prefixes:+ - Projects+ - Journal/Weekly+```++It covers the four tools that take a vault-relative path from the caller -+`brain_create_note`, `brain_update_note`, `brain_append_note` and+`brain_write_batch` - because all four resolve that path through one shared+envelope. Slug-derived writes (a signal, a preference, a dead end) and fully+derived ones (daily logs, dream runs, continuity records, every telemetry+surface) compute their own destination and are out of scope: a prefix rule+could only refuse those wholesale, which is an off switch rather than a+boundary. `tests/core/architecture/write-site-census.test.ts` is the standing+record of every in-vault write site the binding does not cover.++The check reads no identity at all. Agent names here are self-asserted and+additionally overridable per call, so a fence keyed to one is bypassed by+passing a different string; the binding's whole authority is `_brain.yaml`,+which the operator controls and which no MCP call can rewrite. It is a write+boundary over caller-named paths, never a security boundary and never+per-credential.++Both the name and the realpath must be admitted, so a symlinked directory+under a declared prefix cannot widen it. A refusal names the path the caller+gave, where that name actually resolves when the two differ (or `a location+outside the vault`), the prefixes in force, and the registered exit for+`write-binding-refused`. An absent block, or a block without `path_prefixes`,+is inert: every write path behaves byte-identically to before the key existed.+## Stability and trust (since v1.0.0)```text@@ -509,7 +547,7 @@ Single exclusion policy for every vault walker.```texto2b vault status Walks the vault under the active policy; reports include / exclude counts and which rules fired-o2b vault inspect <relpath> Point-check one vault-relative path; reports matched rule, source, and whether the path exists on disk+o2b vault inspect <relpath> Point-check one vault-relative path; reports matched rule, source, whether the path exists on disk, and the write-binding verdict (none declared | admits | refuses)o2b vault profile <sub> Manage named multi-vault profiles (since v0.22.0): list | create <name> <vault> | switch <name>; pointer-based activation in profiles.jsono2b vault map [show] Print the resolved vault-map role tokens -> folders (since v0.22.0), merging an optional Brain/_vault-map.yaml over defaults; read-only```@@ -562,6 +600,17 @@ o2b search reindex Rebuild the SQLite + FTS5 index from scratch--force-cost bypasses the embedding cost gate for this run (since v0.36.0)o2b search index Incrementally update the index; --embeddings computes vectors--force-cost bypasses the embedding cost gate (since v0.36.0)+o2b search vector-backfill Run the vector phase ALONE for indexed chunks that have no vector -+ no vault walk, no re-chunking, no frontmatter pass (since v1.43.0).+ Dry-run by DEFAULT: it counts the pending chunks, reports the+ configured semantic capability tier, and contacts no provider.+ --apply is the only path that reaches a provider or writes a vector+ --force-cost bypasses the embedding cost gate for that run+ --json emits dry_run, capability_tier, capability_code, chunks_total,+ pending, embedded, retries, plus estimated_cost_usd only when the+ model's price is known - a missing price is an absent key, never 0+ Idempotent; an --apply run that wrote vectors appends one+ vector-backfill Brain log evento2b search status Index status; since v0.36.0 also reports the active embeddingsignature (<provider>:<model>:<dimension>) and a refresh-cost estimateo2b search provider add NAME Register an OpenAI-compatible embedding endpoint (since v0.36.0)@@ -582,7 +631,16 @@ built-in `openai-compat`, the offline `local` feature-hashing embedder(no cloud, no key, no model download; `embedding_dimension` default 256),`disabled`, or any name registered via `o2b search provider add`.`embedding_cost_gate_usd` (default 0 = off) refuses an embedding run whose-estimated spend exceeds it unless `--force-cost`. `search_fusion_mode`+estimated spend exceeds it unless `--force-cost`.+`embedding_batch_tokens` (`OPEN_SECOND_BRAIN_EMBEDDING_BATCH_TOKENS`, since+v1.43.0) adds a per-request token budget beside `embedding_batch_size`: a+batch closes on whichever cap fills first, so a run of long chunks cannot+assemble a request past a provider's per-request token ceiling. The estimate+is the same one the cost gate uses, applied to the text as it will be sent+(instruction prefix included). With the key unset the field is absent and+batching is byte-identical to the fixed `embedding_batch_size` stride; a+single text whose own estimate exceeds the budget is sent alone rather than+dropped or split. `search_fusion_mode`(default `linear`) may be set to `rrf` to fuse the keyword and semanticlanes by reciprocal rank (`search_rrf_k`, default 60); `linear` keepsranking bit-identical.diff --git a/docs/how-it-works.md b/docs/how-it-works.mdindex 7d062151..ab9b68ee 100644--- a/docs/how-it-works.md+++ b/docs/how-it-works.md@@ -906,9 +906,12 @@ ids. Gated telemetry surfaces (context-pack receipts and telemetry,pre-compress, search, the recall gate) route through one lazy emitkernel: with the gate off the payload thunk never runs, and a brokencontinuity store can never fail the operation it was observing. A-read-model normalizes stamped and legacy records identically and-drops `private` records by default, so every read-side consumer-agrees on masking.+read-model normalizes stamped and legacy records identically, drops+`private` records by default, and honours a caller-declared record+scope - for the four consumers that read through it. Most readers+call the store directly and see raw records; the split is asserted in+`tests/core/brain/continuity/reader-census.test.ts` and detailed in+`docs/observability.md`.Two consumers ship with the suite. `o2b brain continuity export`renders the store as standard trajectory formats - ATOF JSONL eventsdiff --git a/docs/mcp.md b/docs/mcp.mdindex 09b9e852..6604aacb 100644--- a/docs/mcp.md+++ b/docs/mcp.md@@ -53,7 +53,7 @@ flags for a narrower per-process full server.| `brain_route_metrics` | List or summarise opt-in route-level MCP tool latency (`mcp_route_latency` records); `summary` rolls each tool up into count, error count, and min/avg/max + p50/p95/p99 latency, slowest-first. Emitted only when `mcp_route_metrics_enabled` is on; payload-safe (tool, scope, status, duration, arg key names). Read-only. | `operation` || `brain_retrieval_plan` | Shadow-only retrieval advisor for one `question`: composes query intent/weights, the summary-surface route, the context-pack density allocation, the token-impact ledger, and observed route p95 latency into a strategy, token-budget allocation, graph-expansion advice, reliability, and a marginal-value stop. Optional `token_budget`. Read-only; exposes no mutating parameters and changes no ranking. | `question` || `brain_token_impact` | Durable value-of-memory ledger: `record` posts a context pack's tokenizer-exact prompt-token delta (`baseline` − `packed`, `method` exact/fallback) plus an optional modeled inference-avoidance estimate; `outcome` posts first-pass/repair/retry to calibrate the model; `summary` keeps EXACT prompt-token savings strictly separate from the MODELED (outcome-calibrated) figure; `list` reads raw samples. Writes gated on `token_impact_ledger_enabled` (default off); payload-safe (counts + opaque pack id only). Reads ignore the gate. | `operation` |-| `brain_context_pack_outcome`| Agent-operable outcome loop over the context-pack quality ledger: `post` records one compact outcome row for a carried context-pack quality-sample id — first-pass/repair/retry counters plus three STRICTLY SEPARATE token signals (`exact_prompt_token_savings`, `modeled_inference_avoidance`, `observed_provider_tokens`) — and composes the token-impact ledger by posting a matching first-pass/repair/retry calibration outcome; `list`/`summary` read the rows keeping the signals separate. Writes gated on `context_pack_outcome_enabled` (default off); payload-safe (counters + opaque sample id only), a field the caller omits is never invented. Reads ignore the gate. | `operation` |+| `brain_context_pack_outcome`| Agent-operable outcome loop over the context-pack quality ledger: `post` records one compact outcome row for a carried context-pack quality-sample id — first-pass/repair/retry counters plus three STRICTLY SEPARATE token signals (`exact_prompt_token_savings`, `modeled_inference_avoidance`, `observed_provider_tokens`) — and composes the token-impact ledger by posting a matching first-pass/repair/retry calibration outcome; `list`/`summary` read the rows keeping the signals separate. Writes gated on `context_pack_outcome_enabled` (default off); payload-safe (counters + opaque sample id only), a field the caller omits is never invented. An optional `agent_id` names the ACTING agent and is recorded on every row the post lands; it is a tool ARGUMENT rather than the server's own config identity, so a config the server cannot read can never fail a telemetry post, and omitting it records no actor rather than a guessed one. Reads ignore the gate. | `operation` || `brain_knowledge_gaps` | Aggregate the persisted cross-query demand log into recurring queries the vault answers poorly, ranked by frequency × (1 − IDF-weighted coverage). Read-only; the log is written only by opt-in recall telemetry. | — || `brain_generation_reports` | Inbound, opt-in LLM generation tracing: `record` posts a generation's usage for a handoff (gated, default off; stores prompt hash + token counts only); `list`/`summary` read records and join them to memory paths. Kernel never calls an LLM. | `action` || `brain_obligation` | Recurring obligations under `Brain/obligations/` with a deterministic cadence-driven next-due date: `add`, `done` (advances next_due by one cadence interval), `list` (optionally overdue-only), `show`, `remove`. Cadences: daily/weekly/biweekly/monthly/quarterly/yearly/every-<N>-days. | `operation` |@@ -66,13 +66,13 @@ flags for a narrower per-process full server.| `brain_session_describe` | Describe raw-turn counts and summary depths for one imported session recall DAG. | `session_id` || `brain_session_expand` | Expand a raw or summary session recall node to immediate sources and paginated raw turn content. | `id` || `brain_sources` | Read-only dashboard of signals grouped by (agent, source_type) with active/processed and distinct-topic counts. | — |-| `brain_create_note` | Write an actual vault note file (path + frontmatter + content) atomically inside the vault. Distinct from `brain_note` (log append); refuses traversal, the Brain root, excluded paths, and clobbering. | `path` |+| `brain_create_note` | Write an actual vault note file (path + frontmatter + content) atomically inside the vault. Distinct from `brain_note` (log append); refuses traversal, the Brain root, excluded paths, any destination outside the declared write binding, and — by default — clobbering. Opt-in: `if_exists: "skip"` returns `outcome: "skipped"` instead of creating, `strict` validates the document before writing and reports coded violations, and `template` + `template_variables` render the body through a closed two-construct grammar (`{{name}}` substitution; `{{#name}}…{{/name}}` presence sections and list iteration with `{{.}}`). Unknown placeholders are left intact. | `path` || `brain_file_context` | Given a file path, surface prior vault work that mentions it (decisions, bug notes, refactor history) by querying the index with path-derived terms. Size gate skips trivial files. Read-only; no LLM. | `file_path` || `brain_session_summary` | Session-scoped structured digest over request/decisions/learnings/next_steps: `write` stores agent-extracted categories, `get` returns a session's latest digest, `list` returns all. Append-only, deduped; an all-empty digest is rejected. | `operation` || `brain_idea_lineage` | Read-only provenance tracer: reconstruct how a derived artifact was reached as an observation -> synthesis -> conclusion graph. A `ctn_` id walks the sourceRefs graph; a `pref-`/`ret-` id adapts belief-evolution. Cycle-guarded, depth-bounded; unknown id errors. | `id` || `brain_note_history` | Decompose a note's git history into recallable episodic phases split on a deterministic commit-time gap (default 72h, language-agnostic). Each phase carries subjects/dates/authors. Missing repo → `available: false`; no commits → zero phases. Read-only. | `path` || `schema_inspect` | Read-only schema inspection for any view: `view: graph \| lint \| stats \| orphans \| explain_type \| active_pack \| packs`. | `view` (`token` for `explain_type`) |-| `schema_apply_mutations` | Apply audited, locked schema mutations to `Brain/_brain.yaml`. | `mutations` |+| `schema_apply_mutations` | Apply audited, locked schema mutations to `Brain/_brain.yaml`. `dry_run: true` previews instead: the pack that would result plus its leaf-level `diff`, no config write, no audit record. The preview runs the same pack validator the apply runs, so a batch that validator rejects raises identically in both. It does NOT cover the two checks that exist only because the apply writes: the vault-identity write guard, and the atomic writer's re-parse of the rendered YAML. A batch that renders to unparseable YAML therefore previews clean and fails on apply. | `mutations` (`dry_run` to preview) || `brain_watchdog` | Probe Brain config, required dirs, and search-index health; optionally apply safe directory remediation. | — || `brain_switch_vault` | Activate a named vault profile; the change takes effect on the next server launch. | `name` |@@ -302,6 +302,56 @@ scorer and returns a char-budgeted block of top matches; it returnsconfig key is `"true"`, so default per-turn injection is unchanged. Thenative Hermes provider calls it from `prefetch()` fail-soft.+Three properties of that surface come from the offer chain (v1.43.0):++- **`list_skills` reports what a skill shadows.** Each entry carries a+ `shadowed` array of the same-named skill directories it overrode, in+ discovery order. Previously the losing path was discarded, so an+ operator reading a surprising skill body had nothing telling them a+ second copy existed. An empty array means discovery found none.+- **`skills_attach` returns an `offer_id`.** It is a content-addressed+ digest of the offer itself - the normalised turn text plus the offered+ `(name, path)` pairs in rank order - so anyone holding the offer can+ recompute it and no offer store has to exist. It is `null` when nothing+ was offered; an offer that was never made has no identity. The rendered+ block cites the id too, so an agent that reads only the block can quote+ it back.+- **`get_skill` accepts that `offer_id`.** The runtime's session log then+ records which offer the fetch came from, and `o2b brain import-session`+ stamps it onto the `skill_invoked` continuity record, where+ `joinSkillInvocationsToOffers` joins the invocation back to its offer.+ A malformed id is refused rather than dropped AT `get_skill`, which can+ answer its caller: `isSkillOfferId` rejects it with `INVALID_PARAMS`. The+ import boundary cannot refuse anyone - the session log is historical and+ its author is gone - so `readSkillOfferId` reports a non-conforming value+ the same as absence, and a host-native skill call carrying a junk id is+ stamped with no offer and reads as unattributed. The two surfaces answer+ the same malformed value differently, on purpose, and only one of them+ can tell anybody. A call citing no offer is stamped exactly as before and+ reads as unattributed, never as belonging+ to a nearby offer. The id is the acting agent's claim about where an+ invocation came from: checkable (well formed, and recomputable from the+ offer), not authenticated - this system has no credentials to+ authenticate it with. The join is also retrospective, because an+ invocation is only observed when a session log is imported.++The join surfaces as ONE count under two spellings, which is worth stating+because they look like two figures: `brain_skill_proposals` with+`operation: "usage"` returns `offerAttributedCount` per skill, and the same+number prints as `from_offer=` on `o2b brain skill-proposals usage`. It+counts the invocations of that skill which cited an offer, out of+`invocationCount` total. The remainder are not unattributed by error - most+runtimes' skill calls follow no offer at all.++`skills_attach` additionally applies a discriminating-term floor: a+candidate whose entire match rests on terms more than half the descriptor+corpus carries is dropped rather than offered, because such a term is+evidence for no skill in particular. The only input is corpus document+frequency, so the rule holds in any language and there is no word list+anywhere in its derivation. A single-skill corpus carries no+discrimination at all, so the floor abstains there rather than dropping+the only candidate.+Two optional config keys (each with a matching environment variable)tune the surface; both are off/unset by default, so behaviour isunchanged unless an operator opts in:@@ -672,3 +722,38 @@ Both servers reuse the same backing CLI (`o2b mcp --scope writer` vs the defaultthese warnings fails the plan - the repository is an input, not operatorintent - while a malformed operator `exclude` pattern still fails the call,naming the pattern and the reason it would not compile.+- Since v1.43.0 the four caller-named write tools -- `brain_create_note`,+ `brain_update_note`, `brain_append_note`, and the note operations of+ `brain_write_batch` -- name the write binding among their refusals. All+ four share one envelope, so all four raise it; the tool descriptions+ previously enumerated a refusal list it was missing from, which a caller+ reads as complete. The binding is declared in+ `write_binding.path_prefixes` in `Brain/_brain.yaml`; absent block, no+ binding, and every write path is byte-identical to before the key+ existed.+- Since v1.43.0 a refused write reports BOTH spellings of its reason, so an+ agent can act on the one it received. `data.code` is the surface code+ (`write_binding`, `invalid_path`, `excluded`, `exists`, `config_invalid`,+ ...): snake_case, the same enumeration every other refusal on the write+ surface uses, and what a client branches on. `data.diagnostic_code` is the+ advisory registry spelling of the same state (`write_binding` ->+ `write-binding-refused`, `config_invalid` -> `config-invalid`), and+ `data.next_command` is the command that registry code resolves to (`o2b+ vault inspect <relpath>` and `o2b brain doctor` respectively). A surface+ code with no registered advisory code carries neither key rather than a+ null.+- Since v1.43.0 a vault whose `Brain/_brain.yaml` exists and does not+ validate refuses caller-named writes with `data.code: "config_invalid"`+ naming the config by its VAULT-RELATIVE path (`Brain/_brain.yaml`) and+ carrying `next_command`, instead of a bare fault carrying an absolute+ host filesystem path. It stays a JSON-RPC INTERNAL_ERROR rather than+ INVALID_PARAMS on purpose: no argument the caller could send would+ succeed, and the operator holds the fix.+- Since v1.43.0 a `frontmatter` key must be a key this format can read+ back -- a letter or underscore followed by letters, digits, `_` or `-`.+ The emitter escaped values but wrote keys raw, so a key containing a+ newline emitted extra frontmatter lines; combined with `brain_update_note`+ merging caller keys over existing ones and a last-wins reader, that let a+ caller overwrite a key it never named. `template_variables` keys are+ unaffected: they are not frontmatter keys and never become a line of a+ file.diff --git a/docs/observability.md b/docs/observability.mdindex 998a7eda..9a27b5be 100644--- a/docs/observability.md+++ b/docs/observability.md@@ -29,7 +29,7 @@ The `prompt_prefix` metric surface measures STRUCTURAL prompt-prefix stability (## Continuity record kinds-Every continuity record shares one envelope: `schema`, `id`, `kind`, `createdAt`, `sourceRefs`, `payload`, `private`, `redacted`. The table marks how each kind is gated - this is the always-on vs opt-in matrix, verified against the call sites named in the right column.+Every continuity record shares one envelope, and exactly these eight fields in this order: `schema`, `id`, `kind`, `createdAt`, `sourceRefs`, `payload`, `private`, `redacted`. The table marks how each kind is gated - this is the always-on vs opt-in matrix, verified against the call sites named in the right column.| Kind | Gate | Emitted by ||---|---|---|@@ -40,7 +40,8 @@ Every continuity record shares one envelope: `schema`, `id`, `kind`, `createdAt`| `mcp_route_latency` | config key `mcp_route_metrics_enabled` (default off) | the MCP server's tool-call seam (`MCPServer.invokeToolHandler` in `src/mcp/server.ts`) - one record per `tools/call` and per CLI-bridge `callTool`; payload carries the tool name, scope, status, duration, and argument KEY NAMES only, never argument values || `token_impact` | config key `token_impact_ledger_enabled` (default off) | `brain_token_impact` action `record` (`emitTokenImpact` in `src/core/brain/token-impact.ts`) - one sample per context pack; payload carries the tokenizer-exact prompt-token counts (`baseline_tokens`, `packed_tokens`, signed `delta_tokens`), the `method` label (`exact`/`fallback`), an optional opaque `pack_id`, and the optional modeled inference-avoidance estimate - never raw prompts or recalled text || `token_impact_outcome` | config key `token_impact_ledger_enabled` (default off) | `brain_token_impact` action `outcome` (`recordTokenImpactOutcome`) - one first-pass/repair/retry post used only to calibrate the modeled ledger |-| `context_pack_outcome` | config key `context_pack_outcome_enabled` (default off) | `brain_context_pack_outcome` action `post` (`emitContextPackOutcome` in `src/core/brain/context-pack-outcome.ts`) - one compact per-sample outcome row for a carried context-pack quality-sample id: first-pass/repair/retry counters plus three STRICTLY SEPARATE token signals (`exact_prompt_token_savings`, `modeled_inference_avoidance`, `observed_provider_tokens`), never merged; also composes a matching `token_impact_outcome` calibration post; payload carries counters + an opaque `sample_id` only, never raw prompts or recalled text, and a caller-omitted field is not invented |+| `context_pack_outcome` | config key `context_pack_outcome_enabled` (default off) | `brain_context_pack_outcome` action `post` (`postContextPackOutcome` in `src/core/brain/context-pack-outcome.ts`) - one compact per-sample outcome row for a carried context-pack quality-sample id: first-pass/repair/retry counters plus three STRICTLY SEPARATE token signals (`exact_prompt_token_savings`, `modeled_inference_avoidance`, `observed_provider_tokens`), never merged; payload carries counters + an opaque `sample_id` only, never raw prompts or recalled text, and a caller-omitted field is not invented. One `post` writes THREE records, all joined on the same `sample_id`: this row, a `token_impact_outcome` calibration post, and a `context_pack_evidence` row (both listed separately below and in this table). The thin wrapper `emitContextPackOutcome` returns only this row; the two composed records are written either way |+| `context_pack_evidence` | config key `context_pack_outcome_enabled` (default off - the same gate as the outcome row it accompanies) | composed by every `brain_context_pack_outcome` action `post` (`recordContextPackEvidence` in `src/core/brain/context-pack-evidence.ts`) - one row per outcome row, joined on the same `sample_id`, carrying the evidence the kernel reads back off disk (the `context_receipt` that id names: `final_text_hash`, `item_count`, `final_text_chars`) and a closed `verdict` over the acting agent's optional claim: `match`, `mismatch`, `unclaimed` (evidence obtained, nothing asserted), or `unresolved`. A `mismatch` lists the drifted fields and is RECORDED, never resolved - the outcome row is neither rejected nor silently corrected by it. An `unresolved` verdict names why through a separate token, `sample_absent` (the log was read and holds no such receipt) or `evidence_unreadable` (the log could not be read at all), so absence is never reported as inability. Payload carries those tokens + the opaque `sample_id` only, never raw prompts or recalled text. NOT an independent attestation: the agent id on the row is self-asserted and this same process wrote the receipt it now reads. The `post` response echoes the payload under `evidence`, so a contradicted claim is visible to the poster and not only to a later reader || `ingest_dedup` | always-on, fail-open; written ONLY by a run that actually dropped something | the three ingest runs that drop an item on an exact sha-256 `dedup_hash` collision: `captureSessionLifecycleEvent`, `scanInline`, `importSession` (`emitIngestDedupReport` in `src/core/brain/dedup-telemetry.ts`) - one record per run, payload carries the `surface`, the run's `exact_deduped` total, and a `by_source` map keyed by the session id or vault-relative note path being re-ingested. A run that deduped nothing writes NO record, so a real zero stays distinguishable from an absent measurement. Every count is an EXACT-hash drop; the semantic detectors nominate merge candidates and never drop, so no semantic figure appears here || `session_turn`, `session_summary_node` | always-on within its operation | session-recall import (`src/core/brain/session-recall.ts`) - the write IS the operation || `pre_compact_extract` | always-on within its operation | explicit pre-compact extraction (`src/core/brain/pre-compact-extract.ts`) |@@ -80,7 +81,8 @@ Every continuity payload passes `safeContinuityPayload()` (`src/core/brain/conti- `gate_telemetry` never stores the raw prompt by construction - only a SHA-256 prefix (`prompt_hash`) and the length (`prompt_chars`). The test suite asserts this against the persisted files, not just returned values.- `generation_report` follows the same rule: the handoff prompt is hashed and counted but never persisted - only `prompt_hash` (full SHA-256 hex) and `prompt_chars` land on disk, plus token counts. The kernel never calls an LLM; the report is the agent's INBOUND record after it performed a generation for a brain handoff, so the local token estimate (`local_estimate.input_tokens`) is always present while the agent-reported `usage` block is present only when supplied - absent is reported as absent, never fabricated. A grep-guarded regression test pins that no `fetch`/provider HTTP call exists under `src/core` for this feature, and a persisted-file assertion confirms no raw prompt survives.- `mcp_route_latency` records the route (tool name), MCP scope, status, duration, and the sorted set of argument KEY NAMES only - the tool's own JSON-Schema property names, never operator-supplied argument values - so no prompt, note body, or preference id can land on disk. A persisted-file test asserts key names survive while values never do.-- Read-side consumers (trajectory export, bench) get the masking policy from the read-model: records flagged `private` are dropped by default and kept only on explicit request; masked text is never un-masked.+- The masking policy lives in the read-model (`src/core/brain/continuity/read-model.ts`) and governs only the readers that go through it: records flagged `private` are dropped by default and kept only on explicit request; masked text is never un-masked. That is FOUR modules - the `o2b brain continuity` verbs (trajectory export and listing), the bench harness, `brain_event_trace`, and skill-usage aggregation. Twenty other modules call `listContinuityRecords` on the store directly and see raw records: no `private` drop and no schema-version dispatch. The policy is read-model-wide, NOT store-wide. Both lists are asserted in `tests/core/brain/continuity/reader-census.test.ts`, which fails when a module joins or leaves either set; auditing the direct readers is tracked separately.+- `private` is the ONLY visibility signal a record carries, and it is inferred from a `<private>` region in the payload rather than declared. A caller-declared `scope` field was built for this envelope and removed before release: nothing in the codebase could supply one - no tool argument, no command-line flag, no config key and no library entry point - and its single vocabulary member would have narrowed a record's visibility rather than widened it, since a declared scope is dropped by every reader that does not name it. Adding a declaration to the envelope is a schema change; `tests/core/brain/continuity/record-envelope.test.ts` pins the eight fields and their order so the next attempt has to be argued for.## Fail-open rulesdiff --git a/openclaw.plugin.json b/openclaw.plugin.jsonindex 3cf19f0b..e5ab4458 100644--- a/openclaw.plugin.json+++ b/openclaw.plugin.json@@ -2,7 +2,7 @@"id": "open-second-brain","name": "Open Second Brain","description": "Second brain for AI agents using Obsidian-compatible Markdown vaults.",- "version": "1.42.0",+ "version": "1.43.0","activation": { "onStartup": true },"skills": ["./skills"],"contracts": {diff --git a/package.json b/package.jsonindex f05c2b11..a737f7ab 100644--- a/package.json+++ b/package.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.42.0",+ "version": "1.43.0","private": false,"description": "Second brain for AI agents using Obsidian-compatible Markdown vaults. Works with Hermes, Claude Code, Codex, and OpenClaw.","keywords": [diff --git a/plugin.yaml b/plugin.yamlindex 22b2df8c..4dc40c09 100644--- a/plugin.yaml+++ b/plugin.yaml@@ -1,5 +1,5 @@name: open-second-brain-version: "1.42.0"+version: "1.43.0"description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."author: "Open Second Brain contributors"memory_provider: truediff --git a/plugins/codex/.codex-plugin/plugin.json b/plugins/codex/.codex-plugin/plugin.jsonindex 70fa7a53..f4bd1109 100644--- a/plugins/codex/.codex-plugin/plugin.json+++ b/plugins/codex/.codex-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.42.0",+ "version": "1.43.0","description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.","author": {"name": "Open Second Brain contributors",diff --git a/plugins/hermes/plugin.yaml b/plugins/hermes/plugin.yamlindex 22b2df8c..4dc40c09 100644--- a/plugins/hermes/plugin.yaml+++ b/plugins/hermes/plugin.yaml@@ -1,5 +1,5 @@name: open-second-brain-version: "1.42.0"+version: "1.43.0"description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."author: "Open Second Brain contributors"memory_provider: truediff --git a/pyproject.toml b/pyproject.tomlindex a7a15380..35ca90a1 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -10,7 +10,7 @@ build-backend = "setuptools.build_meta"# CLI entry points (those moved to `package.json` `bin`).[project]name = "open-second-brain"-version = "1.42.0"+version = "1.43.0"description = "Hermes Python shim for Open Second Brain. Most of the project (CLI, MCP server, OpenClaw plugin) is TypeScript on Bun; see package.json."readme = "README.md"requires-python = ">=3.11"diff --git a/skills/schema-author/SKILL.md b/skills/schema-author/SKILL.mdindex 3173700f..05665171 100644--- a/skills/schema-author/SKILL.md+++ b/skills/schema-author/SKILL.md@@ -12,8 +12,9 @@ Use this skill when a task changes the active Brain schema pack in `Brain/_brain1. Inspect the active schema pack with `schema_inspect` (`view="active_pack"`) or `o2b brain schema --json`.2. Check current usage and findings with `schema_inspect` views `stats`, `lint`, and `orphans`.3. Explain candidate tokens with `schema_inspect` (`view="explain_type"`) before renaming or deleting them.-4. Apply schema changes only through `schema_apply_mutations` or `o2b brain schema apply --mutation ...` so writes are locked, atomic, and audited.-5. Re-run lint/stats after mutation and report changed tokens plus any remaining findings.+4. Preview the batch first with `schema_apply_mutations` (`dry_run=true`) or `o2b brain schema apply --dry-run --mutation ...`: it returns the pack that would result and its diff and writes nothing at all - no config write, no audit record, no lock file. It runs the same pack validator the apply runs, so a batch that validator rejects raises identically in both. It does not run the two checks that exist only because the apply writes: the vault-identity write guard, and the atomic writer's re-parse of the rendered YAML. A batch that renders to unparseable YAML previews clean and fails on apply, so treat a clean preview as "the mutations are valid", not as "the apply will succeed".+5. Apply schema changes only through `schema_apply_mutations` or `o2b brain schema apply --mutation ...` so writes are locked, atomic, and audited.+6. Re-run lint/stats after mutation and report changed tokens plus any remaining findings.## Mutation Notesdiff --git a/src/cli/brain/verbs/entity.ts b/src/cli/brain/verbs/entity.tsindex 183a2d40..64898d58 100644--- a/src/cli/brain/verbs/entity.ts+++ b/src/cli/brain/verbs/entity.ts@@ -12,7 +12,11 @@ import {relateEntities,upsertEntity,} from "../../../core/brain/entities/registry.ts";-import { BRAIN_ENTITY_STATUS, type BrainEntity } from "../../../core/brain/entities/types.ts";+import {+ BRAIN_ENTITY_STATUS_VALUES,+ isBrainEntityStatus,+ type BrainEntity,+} from "../../../core/brain/entities/types.ts";import { pruneEntityLabels } from "../../../core/brain/entities/label-hygiene.ts";import {brainVerbContext,@@ -147,12 +151,11 @@ function entityList(argv: string[]): number {const vault = brainVerbContext(flags).vault;const category = normalizeFlagString(flags["category"]);const statusRaw = normalizeFlagString(flags["status"]);- if (- statusRaw !== null &&- statusRaw !== BRAIN_ENTITY_STATUS.active &&- statusRaw !== BRAIN_ENTITY_STATUS.archived- ) {- return fail("--status must be 'active' or 'archived'");+ // Validated against the vocabulary itself rather than a hand-written pair,+ // so a new status is listable the day it exists. `--status quarantine` is+ // how an operator sees what an untrusted intake put aside.+ if (statusRaw !== null && !isBrainEntityStatus(statusRaw)) {+ return fail(`--status must be one of: ${BRAIN_ENTITY_STATUS_VALUES.join(", ")}`);}try {const entities = listEntities(vault, {diff --git a/src/cli/brain/verbs/links.ts b/src/cli/brain/verbs/links.tsindex 0d452420..e67dd3da 100644--- a/src/cli/brain/verbs/links.ts+++ b/src/cli/brain/verbs/links.ts@@ -4,9 +4,10 @@* path format. Dry-run by default; `--write` applies.*/-import { readFileSync, writeFileSync } from "node:fs";+import { readFileSync } from "node:fs";import { join } from "node:path";+import { atomicWriteFileSync } from "../../../core/fs-atomic.ts";import { defaultConfigPath, resolveWikiLinkFormat } from "../../../core/config.ts";import {isWikiLinkFormat,@@ -81,7 +82,14 @@ export async function cmdBrainLinks(argv: string[]): Promise<number> {});totalChanged += result.changed;if (write && result.changed > 0) {- writeFileSync(join(vault, file.relPath), result.content);+ // Through the shared atomic writer, not a raw `writeFileSync`.+ // This verb rewrites notes the operator is also editing in+ // Obsidian, in a loop over the whole Brain tree: a plain write+ // truncates first, so an interrupted run leaves the note it was+ // in the middle of empty or half-rewritten, with the original+ // gone. Temp file plus `rename(2)` makes each note either the+ // old bytes or the new ones, never neither.+ atomicWriteFileSync(join(vault, file.relPath), result.content);}}diff --git a/src/cli/brain/verbs/schema.ts b/src/cli/brain/verbs/schema.tsindex e038d15c..38781181 100644--- a/src/cli/brain/verbs/schema.ts+++ b/src/cli/brain/verbs/schema.ts@@ -10,6 +10,7 @@ import {parseSchemaMutationPayloads,reviewSchemaOrphans,} from "../../../core/brain/schema-admin.ts";+import { previewSchemaMutations } from "../../../core/brain/schema-mutate.ts";import {buildSchemaReport,type BrainSchemaReport,@@ -89,14 +90,21 @@ export async function cmdBrainSchema(argv: string[]): Promise<number> {case "apply": {const mutationPayloads = (flags["mutation"] as string[] | undefined) ?? [];if (mutationPayloads.length === 0) throw new Error("schema apply requires --mutation JSON");- const result = await applySchemaAdminMutations(- vault,- parseSchemaMutationPayloads(mutationPayloads),- {- actor: flags["actor"] as string,- reason: flags["reason"] as string | undefined,- },- );+ const mutations = parseSchemaMutationPayloads(mutationPayloads);+ // `--dry-run` is parsed for every schema subcommand. Until the+ // preview existed, `apply` accepted the flag and wrote anyway - an+ // operator asking to look first got a mutated vault schema.+ if (flags["dry-run"]) {+ return writeResult(+ previewSchemaMutations(vault, mutations),+ Boolean(flags["json"]),+ renderGenericText,+ );+ }+ const result = await applySchemaAdminMutations(vault, mutations, {+ actor: flags["actor"] as string,+ reason: flags["reason"] as string | undefined,+ });return writeResult(result, Boolean(flags["json"]), renderGenericText);}case "sync": {diff --git a/src/cli/brain/verbs/skill-proposals.ts b/src/cli/brain/verbs/skill-proposals.tsindex 290fc7ad..0e369f41 100644--- a/src/cli/brain/verbs/skill-proposals.ts+++ b/src/cli/brain/verbs/skill-proposals.ts@@ -86,7 +86,9 @@ function usage(argv: string[]): number {process.stdout.write(`${rows.length} skill(s) with recorded invocations:\n`);for (const row of rows) {- process.stdout.write(` ${row.skill} invocations=${row.invocationCount}\n`);+ process.stdout.write(+ ` ${row.skill} invocations=${row.invocationCount} from_offer=${row.offerAttributedCount}\n`,+ );}return 0;}diff --git a/src/cli/command-manifest.ts b/src/cli/command-manifest.tsindex 9c3aedf8..b50f385d 100644--- a/src/cli/command-manifest.ts+++ b/src/cli/command-manifest.ts@@ -444,6 +444,14 @@ export const CLI_COMMAND_MANIFEST: CliRootManifest = Object.freeze({command("rerank-provider", "Manage reranker provider profiles"),command("rerank-fit", "Diagnose whether the configured reranker fits this vault's queries"),command("plan", "Preview the should-read shortlist for a query without reading the notes"),+ command(+ "vector-backfill",+ "Compute vectors for indexed chunks that have none (dry-run unless --apply)",+ ),+ command(+ "event-anchor-backfill",+ "Resolve event anchors for indexed documents that have none (dry-run unless --apply)",+ ),],),command(diff --git a/src/cli/search.ts b/src/cli/search.tsindex 653dcec6..cf1eb70d 100644--- a/src/cli/search.ts+++ b/src/cli/search.ts@@ -17,6 +17,7 @@ import { CliError } from "./argparse.ts";import { cmdSearchCheck } from "./search/verbs/check.ts";import { cmdSearchExpand } from "./search/verbs/expand.ts";import { cmdSearchFocus } from "./search/verbs/focus.ts";+import { cmdSearchEventAnchorBackfill } from "./search/verbs/event-anchor-backfill.ts";import { cmdSearchIndex, cmdSearchReindex } from "./search/verbs/indexing.ts";import { cmdSearchFeedback, cmdSearchWeights } from "./search/verbs/learned-weights.ts";import { cmdSearchPlan } from "./search/verbs/plan.ts";@@ -24,6 +25,7 @@ import { cmdSearchProvider, cmdSearchRerankProvider } from "./search/verbs/proviimport { cmdSearchQuery } from "./search/verbs/query.ts";import { cmdSearchRerankFit } from "./search/verbs/rerank-fit.ts";import { cmdSearchStatus } from "./search/verbs/status.ts";+import { cmdSearchVectorBackfill } from "./search/verbs/vector-backfill.ts";import { cmdSearchWatch } from "./search/verbs/watch.ts";const KNOWN_VERBS = new Set([@@ -41,6 +43,8 @@ const KNOWN_VERBS = new Set(["rerank-fit","plan","watch",+ "vector-backfill",+ "event-anchor-backfill",]);export async function handleSearchSubcommand(argv: ReadonlyArray<string>): Promise<number> {@@ -83,6 +87,10 @@ export async function handleSearchSubcommand(argv: ReadonlyArray<string>): Promireturn await cmdSearchRerankFit(rest);case "plan":return await cmdSearchPlan(rest);+ case "vector-backfill":+ return await cmdSearchVectorBackfill(rest);+ case "event-anchor-backfill":+ return await cmdSearchEventAnchorBackfill(rest);default:process.stderr.write(`error: unknown search verb: ${verb}\n`);return 2;diff --git a/src/cli/search/verbs/event-anchor-backfill.ts b/src/cli/search/verbs/event-anchor-backfill.tsnew file mode 100644index 00000000..0a01d7e4--- /dev/null+++ b/src/cli/search/verbs/event-anchor-backfill.ts@@ -0,0 +1,132 @@+/**+ * `o2b search event-anchor-backfill` — resolve the event anchors an+ * in-place schema migration could not (provenance at the boundary).+ *+ * A schema bump migrates the index in place and reindexes nothing, and+ * the indexer resolves an anchor only below its two content-identity+ * fastpaths. A document indexed by the previous binary therefore never+ * gets one: its content does not change, so it is never re-read. This+ * verb reads exactly those documents and nothing else - no vault walk,+ * no re-chunking, no embedding provider.+ *+ * Dry-run is the DEFAULT; `--apply` is the only path that writes. The+ * shape follows `search/verbs/vector-backfill.ts` line for line,+ * including the log append whose failure is reported on stderr rather+ * than swallowed.+ */++import { appendLogEvent } from "../../../core/brain/log.ts";+import { nextCommandField } from "../../../core/brain/next-step.ts";+import { isoSecond } from "../../../core/brain/time.ts";+import { BRAIN_LOG_EVENT_KIND } from "../../../core/brain/types.ts";+import { resolveAgentName } from "../../../core/config.ts";+import {+ planEventAnchorBackfill,+ type EventAnchorBackfillResult,+} from "../../../core/search/event-anchor-backfill.ts";+import { emitNextStep } from "../../advisory-rail.ts";+import { info, ok } from "../../output.ts";+import {+ flagBoolean,+ parseFlags,+ resolveConfig,+ resolveConfigPath,+ searchAdvisoryStream,+ VAULT_FLAGS,+} from "../helpers.ts";++/**+ * The state this verb leaves an operator in when work remains: indexed+ * documents whose event anchor has never been resolved. Its registered+ * exit is this same verb under `--apply`, which is the point of naming it.+ */+const EVENT_ANCHORS_PENDING = "event-anchors-pending";++/** JSON payload shape. Snake case, matching every other search verb. */+function jsonForResult(result: EventAnchorBackfillResult): Record<string, unknown> {+ return {+ dry_run: !result.applied,+ documents_total: result.documentsTotal,+ pending: result.pending,+ examined: result.examined,+ anchored: result.anchored,+ // Absent rather than empty when every file was readable, so a clean+ // run emits the payload it would have emitted without this field.+ ...(result.unreadable.length > 0+ ? { unreadable: result.unreadable.map((u) => ({ path: u.path, message: u.message })) }+ : {}),+ };+}++function renderHuman(result: EventAnchorBackfillResult): void {+ if (result.applied) {+ ok(+ `event-anchor-backfill: examined ${result.examined} of ${result.pending} pending document(s), ` ++ `${result.anchored} of which declare an event time`,+ );+ } else {+ ok(+ `event-anchor-backfill dry-run: ${result.pending} of ${result.documentsTotal} document(s) ` ++ `have no event anchor resolved`,+ );+ }+ // A file that could not be read is NAMED. It keeps no verdict and+ // stays pending, so it must not vanish into the counts above.+ if (result.unreadable.length > 0) {+ info(` ${result.unreadable.length} indexed document(s) could not be read:`);+ for (const u of result.unreadable) info(` - ${u.path}: ${u.message}`);+ }+}++export async function cmdSearchEventAnchorBackfill(argv: ReadonlyArray<string>): Promise<number> {+ const { flags } = parseFlags(argv, {+ ...VAULT_FLAGS,+ apply: { type: "boolean" },+ json: { type: "boolean" },+ });+ const cfg = resolveConfig(flags);+ const apply = flagBoolean(flags, "apply");+ const jsonRequested = flagBoolean(flags, "json");++ const result = await planEventAnchorBackfill(cfg, { apply });++ if (apply && result.examined > 0) {+ try {+ appendLogEvent(cfg.vault, {+ timestamp: isoSecond(new Date()),+ eventType: BRAIN_LOG_EVENT_KIND.eventAnchorBackfill,+ body: {+ agent: resolveAgentName(resolveConfigPath(flags)),+ examined: String(result.examined),+ anchored: String(result.anchored),+ pending: String(result.pending),+ documents_total: String(result.documentsTotal),+ },+ });+ } catch (err) {+ process.stderr.write(+ `warning: append event-anchor-backfill log failed: ${(err as Error).message}\n`,+ );+ }+ }++ // Work remains exactly when the run did not consume it - which+ // includes every document whose file could not be read, since those+ // were deliberately left unexamined rather than recorded as dateless.+ const pendingAfter = result.pending - result.examined;++ if (jsonRequested) {+ process.stdout.write(+ JSON.stringify({+ ...jsonForResult(result),+ ...(pendingAfter > 0 ? nextCommandField(EVENT_ANCHORS_PENDING) : {}),+ }) + "\n",+ );+ } else {+ renderHuman(result);+ }+ if (pendingAfter > 0) {+ emitNextStep(EVENT_ANCHORS_PENDING, searchAdvisoryStream(argv, jsonRequested));+ }+ return 0;+}diff --git a/src/cli/search/verbs/indexing.ts b/src/cli/search/verbs/indexing.tsindex 1055d740..ce8c88c4 100644--- a/src/cli/search/verbs/indexing.ts+++ b/src/cli/search/verbs/indexing.ts@@ -34,6 +34,19 @@ const SAFEGUARD_OPERATION = "reindex";/** Terminal state both builders reach when the index covers every document. */const INDEX_BUILT = "search-index-built";+/**+ * The state a run leaves when the index holds documents whose EVENT+ * ANCHOR has never been resolved (provenance at the boundary).+ *+ * It outranks `INDEX_BUILT` as this run's exit, and deliberately: a+ * schema bump migrates in place and reindexes nothing, so every+ * pre-anchor document takes the unchanged fastpath and the run truly did+ * cover the vault - but a hard `since` / `until` filter is now judged on+ * a column those rows do not carry, and naming `o2b search query` next+ * would send the operator at exactly the query that answers wrongly.+ */+const EVENT_ANCHORS_PENDING = "event-anchors-pending";+/** Default flush cadence for the generated `reindex --cron-template`. */const DEFAULT_CRON_INTERVAL = "30m";@@ -78,6 +91,7 @@ function reportIndexRun(jsonRequested: boolean,): void {const complete = indexIsComplete(stats);+ const exitCode = stats.eventAnchorsPending > 0 ? EVENT_ANCHORS_PENDING : INDEX_BUILT;if (jsonRequested) {// no-dead-ends, phase 3: the rail suppresses its line on this stream// and returns the resolved step precisely so the payload can carry@@ -85,14 +99,14 @@ function reportIndexRun(process.stdout.write(JSON.stringify({...(jsonForStats(stats, cfg) as Record<string, unknown>),- ...(complete ? nextCommandField(INDEX_BUILT) : {}),+ ...(complete ? nextCommandField(exitCode) : {}),}) + "\n",);} else {process.stdout.write(renderStatsHuman(stats, cfg));}if (complete) {- emitNextStep(INDEX_BUILT, searchAdvisoryStream(argv, jsonRequested));+ emitNextStep(exitCode, searchAdvisoryStream(argv, jsonRequested));}}@@ -174,6 +188,11 @@ function jsonForStats(stats: IndexStats, cfg: ResolvedSearchConfig): unknown {chunks_total: stats.chunksTotal,embeddings_computed: stats.embeddingsComputed,embeddings_retries: stats.embeddingsRetries,+ // Conditional: a vault with nothing pending emits exactly the+ // payload it emitted before this field existed.+ ...(stats.eventAnchorsPending > 0+ ? { event_anchors_pending: stats.eventAnchorsPending }+ : {}),},errors: stats.errors.map((e) => ({ path: e.path, message: e.message })),// Unit F. Conditional so a vault with no malformed frontmatter emits@@ -206,6 +225,13 @@ function renderStatsHuman(stats: IndexStats, cfg: ResolvedSearchConfig): string` embeddings: ${stats.embeddingsComputed} computed (${stats.embeddingsRetries} retries)`,);}+ // Above the errors block: these documents indexed fine, under a+ // binary that had no event anchor to compute for them.+ if (stats.eventAnchorsPending > 0) {+ lines.push(+ ` event anchors: ${stats.eventAnchorsPending} document(s) indexed before anchors existed`,+ );+ }if (stats.errors.length > 0) {lines.push(` errors:`);for (const e of stats.errors) lines.push(` - ${e.path}: ${e.message}`);diff --git a/src/cli/search/verbs/vector-backfill.ts b/src/cli/search/verbs/vector-backfill.tsnew file mode 100644index 00000000..e82aa904--- /dev/null+++ b/src/cli/search/verbs/vector-backfill.ts@@ -0,0 +1,143 @@+/**+ * `o2b search vector-backfill` — run the vector phase on its own+ * (provenance-at-the-boundary, unit F).+ *+ * Indexing does not compute vectors unless it is asked to: no automatic+ * `indexVault` call site requests embeddings, and only the explicit+ * `--embeddings` flag on `search index` / `search reindex` reaches a+ * provider. A vault therefore accumulates chunks with no `embeddings`+ * row perfectly normally, and until now the only way to fill them was to+ * re-index the whole vault. This verb fills them and nothing else.+ *+ * Dry-run is the DEFAULT; `--apply` is the only path that contacts a+ * provider or writes a vector. The shape follows+ * `brain/verbs/authored-at-backfill.ts` line for line, including the log+ * append whose failure is reported on stderr rather than swallowed.+ */++import { appendLogEvent } from "../../../core/brain/log.ts";+import { nextCommandField } from "../../../core/brain/next-step.ts";+import { isoSecond } from "../../../core/brain/time.ts";+import { BRAIN_LOG_EVENT_KIND } from "../../../core/brain/types.ts";+import { resolveAgentName } from "../../../core/config.ts";+import {+ semanticCapabilityIsBlocked,+ semanticCapabilityLabel,+ SEMANTIC_VECTOR_CODE,+} from "../../../core/search/capability-tier.ts";+import {+ planVectorBackfill,+ type VectorBackfillResult,+} from "../../../core/search/vector-backfill.ts";+import { emitNextStep } from "../../advisory-rail.ts";+import { info, ok } from "../../output.ts";+import {+ flagBoolean,+ parseFlags,+ resolveConfig,+ resolveConfigPath,+ searchAdvisoryStream,+ VAULT_FLAGS,+} from "../helpers.ts";++/**+ * The state this verb leaves an operator in when work remains: an index+ * holding chunks with no vector. Its registered exit is this same verb+ * under `--apply`, which is the whole point of naming it.+ */+const VECTORS_PENDING = SEMANTIC_VECTOR_CODE.pending;++/** JSON payload shape. Snake case, matching every other search verb. */+function jsonForResult(result: VectorBackfillResult): Record<string, unknown> {+ return {+ dry_run: !result.applied,+ capability_tier: result.capability.tier,+ capability_code: result.capability.code,+ chunks_total: result.chunksTotal,+ pending: result.pending,+ embedded: result.embedded,+ retries: result.retries,+ // Absent rather than zero when the model carries no known price: a+ // missing price is not a free run.+ ...(result.costKnown ? { estimated_cost_usd: result.estimatedCostUsd } : {}),+ };+}++async function renderHuman(result: VectorBackfillResult): Promise<void> {+ if (result.applied) {+ ok(`vector-backfill: wrote ${result.embedded} of ${result.pending} pending vector(s)`);+ } else {+ ok(+ `vector-backfill dry-run: ${result.pending} of ${result.chunksTotal} chunk(s) have no vector`,+ );+ }+ if (result.costKnown && result.pending > 0) {+ info(` estimated cost: $${result.estimatedCostUsd.toFixed(4)}`);+ }+ if (result.retries > 0) info(` provider retries: ${result.retries}`);+ // What the operator CONFIGURED, resolved from the registry - never a+ // sentence built here.+ if (semanticCapabilityIsBlocked(result.capability)) {+ info(` semantic capability: ${await semanticCapabilityLabel(result.capability.code)}`);+ }+}++export async function cmdSearchVectorBackfill(argv: ReadonlyArray<string>): Promise<number> {+ const { flags } = parseFlags(argv, {+ ...VAULT_FLAGS,+ apply: { type: "boolean" },+ "force-cost": { type: "boolean" },+ json: { type: "boolean" },+ });+ const cfg = resolveConfig(flags);+ const apply = flagBoolean(flags, "apply");+ const jsonRequested = flagBoolean(flags, "json");++ const result = await planVectorBackfill(cfg, {+ apply,+ forceCost: flagBoolean(flags, "force-cost"),+ });++ if (apply && result.embedded > 0) {+ try {+ appendLogEvent(cfg.vault, {+ timestamp: isoSecond(new Date()),+ eventType: BRAIN_LOG_EVENT_KIND.vectorBackfill,+ body: {+ agent: resolveAgentName(resolveConfigPath(flags)),+ embedded: String(result.embedded),+ pending: String(result.pending),+ chunks_total: String(result.chunksTotal),+ },+ });+ } catch (err) {+ process.stderr.write(+ `warning: append vector-backfill log failed: ${(err as Error).message}\n`,+ );+ }+ }++ // Work remains exactly when the run did not consume it.+ const pendingAfter = result.pending - result.embedded;+ // WHICH exit, though, depends on what is in the operator's way. Naming+ // `--apply` while the credential is missing would advise a command that+ // cannot succeed, so a blocked tier names the tier's own exit instead.+ const exitCode = semanticCapabilityIsBlocked(result.capability)+ ? result.capability.code+ : VECTORS_PENDING;++ if (jsonRequested) {+ process.stdout.write(+ JSON.stringify({+ ...jsonForResult(result),+ ...(pendingAfter > 0 ? nextCommandField(exitCode) : {}),+ }) + "\n",+ );+ } else {+ await renderHuman(result);+ }+ if (pendingAfter > 0) {+ emitNextStep(exitCode, searchAdvisoryStream(argv, jsonRequested));+ }+ return 0;+}diff --git a/src/cli/vault/verbs/inspect.ts b/src/cli/vault/verbs/inspect.tsindex c27c18af..92e29444 100644--- a/src/cli/vault/verbs/inspect.ts+++ b/src/cli/vault/verbs/inspect.ts@@ -2,10 +2,26 @@* `o2b vault inspect <relpath>` — point-check one vault-relative path.** Anchored in `docs/plans/2026-05-19-vault-scope-design.md` §7.2.+ *+ * Reports TWO independent verdicts about the same path, because they+ * answer different questions and an operator who conflates them will+ * look in the wrong file:+ *+ * - VAULT SCOPE — is this path part of the vault at all? Governs every+ * walker (indexer, scanners) and is declared in `vault.ignore_paths`.+ * - WRITE BINDING — may a CALLER-NAMED write author here? Governs+ * `brain_create_note` and the update / append / batch arms, and is+ * declared in `write_binding.path_prefixes`+ * (provenance-at-the-boundary, unit B).+ *+ * This verb is the registered exit for `write-binding-refused`, so the+ * binding half is not decoration: it is the surface that answers the+ * question the refusal raises.*/import { defaultConfigPath } from "../../../core/config.ts";import { inspectPath, resolveVaultScope } from "../../../core/vault-scope/index.ts";+import { resolveWriteBinding, writeBindingAdmits } from "../../../core/write-binding/index.ts";import { CliError, parseFlags } from "../../argparse.ts";import { resolveBrainVault } from "../../brain/helpers.ts";import { fail, info, writeJson } from "../../output.ts";@@ -29,13 +45,20 @@ export async function cmdVaultInspect(argv: ReadonlyArray<string>): Promise<numbthrow exc;}let result: ReturnType<typeof inspectPath>;+ let binding: ReturnType<typeof resolveWriteBinding>;try {const scope = resolveVaultScope(vault);result = inspectPath(relpath, scope, vault);+ binding = resolveWriteBinding(vault);} catch (exc) {process.stderr.write(`error: ${(exc as Error).message ?? exc}\n`);return 2;}+ // Three states, not two. "No binding is declared" is a different+ // answer from "a binding is declared and admits this path", and+ // collapsing them would tell an operator their boundary is working+ // when they never declared one.+ const bindingAdmits = binding === null ? null : writeBindingAdmits(binding, result.relPath);// Per design §7.2: the rule decision is meaningful even for paths// that do not exist on disk (operator may be checking the policy// before authoring the file), but the operator should know@@ -49,17 +72,40 @@ export async function cmdVaultInspect(argv: ReadonlyArray<string>): Promise<numbmatched_rule: result.rule ? { raw: result.rule.raw, kind: result.rule.kind } : null,matched_at: result.matchedAt,source: result.source,+ write_binding:+ binding === null+ ? { declared: false }+ : {+ declared: true,+ admits: bindingAdmits,+ path_prefixes: binding.pathPrefixes,+ },});return 0;}info(`relpath: ${result.relPath}`);if (!result.excluded) {info(`status: included${notFoundSuffix}`);+ reportWriteBinding(binding, bindingAdmits);return 0;}info(`status: excluded${notFoundSuffix}`);if (result.rule) info(`matched rule: ${result.rule.raw} (${result.rule.kind})`);if (result.matchedAt) info(`matched at: ${result.matchedAt}`);info(`source: ${result.source}`);+ reportWriteBinding(binding, bindingAdmits);return 0;}++/** Human-stream rendering of the three write-binding states. */+function reportWriteBinding(+ binding: ReturnType<typeof resolveWriteBinding>,+ admits: boolean | null,+): void {+ if (binding === null) {+ info("write binding: none declared");+ return;+ }+ info(`write binding: ${admits === true ? "admits" : "refuses"} caller-named writes`);+ info(` prefixes: ${binding.pathPrefixes.join(", ")}`);+}diff --git a/src/core/brain/atomic-facts.ts b/src/core/brain/atomic-facts.tsindex d3a5939c..3615496e 100644--- a/src/core/brain/atomic-facts.ts+++ b/src/core/brain/atomic-facts.ts@@ -11,6 +11,7 @@*/import { entityMatchForms, normalizeEntityName } from "./entities/canonical.ts";+import { ENTITY_STATUS_SCOPE, entityStatusInScope } from "./entities/status-scope.ts";/** Cap on one assertion's text length. */export const MAX_ASSERTION_CHARS = 300;@@ -125,7 +126,7 @@ function anchorEntities(text: string, entities: ReadonlyArray<AtomicEntityLike>)const haystack = normalizeEntityName(text);const ids: string[] = [];for (const entity of entities) {- if (entity.status !== "active") continue;+ if (!entityStatusInScope(entity.status, ENTITY_STATUS_SCOPE.canonical)) continue;const forms = entityMatchForms([entity.name, ...entity.aliases]).filter((f) => f.length >= MIN_ANCHOR_FORM_LENGTH,);diff --git a/src/core/brain/config-template.ts b/src/core/brain/config-template.tsindex 7d70032e..95dd6e13 100644--- a/src/core/brain/config-template.ts+++ b/src/core/brain/config-template.ts@@ -444,6 +444,26 @@ export const BRAIN_CONFIG_TEMPLATE: ReadonlyArray<BrainTemplateBlock> = Object.fdef("daily_window_offset_hours", T.daily_window_offset_hours),],},+ {+ key: "write_binding",+ doc: [+ "Where a CALLER-NAMED write may land: brain_create_note,",+ "brain_update_note, brain_append_note and brain_write_batch, which",+ "take a vault-relative path from the caller. Unset by default,",+ "which leaves those surfaces unbound exactly as before.",+ "",+ "This is a write boundary over caller-named paths. It is NOT a",+ "security boundary and not per-agent: its authority is this file,",+ "which no tool call can rewrite, and it reads no caller identity.",+ "Writes whose destination is derived rather than named - daily",+ "logs, dream runs, continuity records, telemetry - are outside it.",+ "",+ "The values below are an illustration, not defaults. An empty list",+ "is refused: remove the block to run unbound.",+ ],+ emit: "commented-example",+ keys: [example("path_prefixes", Object.freeze(["Projects", "Journal/Weekly"]), [])],+ },]) as ReadonlyArray<BrainTemplateBlock>;/**diff --git a/src/core/brain/context-pack-evidence.ts b/src/core/brain/context-pack-evidence.tsnew file mode 100644index 00000000..88def8cb--- /dev/null+++ b/src/core/brain/context-pack-evidence.ts@@ -0,0 +1,423 @@+/**+ * Kernel-recomputed evidence beside the acting agent's claim+ * (provenance-at-the-boundary, unit I).+ *+ * The outcome ledger ({@link ./context-pack-outcome.ts}) is entirely+ * self-reported: the agent posts a sample id and its own counters, and+ * nothing on the read side can tell a row describing a real context pack+ * from a row describing one that never existed. This module is the other+ * half of that row - a SECOND continuity record, joined by the same+ * sample id, carrying evidence the kernel derives itself and the verdict+ * of comparing the agent's claim against it.+ *+ * ## What this is, stated exactly+ *+ * The evidence is the `context_receipt` the sample id names, read back+ * from the durable continuity log at post time. Its `final_text_hash` is+ * a SHA-256 the kernel computed over the assembled pack text when the+ * pack was built ({@link emitContextReceipt}); `item_count` and+ * `final_text_chars` are the measurements taken alongside it. The+ * posting call supplies a sample id and nothing else that enters those+ * values, so a claim can be checked against a record the claiming call+ * did not write.+ *+ * ## What this is NOT+ *+ * It is NOT an independent attestation, and no field, verdict or+ * docstring here says otherwise. Agent identity in this system is a+ * string from an environment variable, else a config key, else a+ * literal, with no token, key, signature or channel binding anywhere,+ * and this process wrote the receipt it now reads. A `verifier_id` would+ * be a second name chosen by the same process from the same config in+ * the same session, which is not a second actor. The honest claim is+ * narrow and is the whole claim: THESE ARE THE BYTES ON DISK, and the+ * agent's assertion either agrees with them or does not.+ *+ * ## Disagreement is recorded, never resolved+ *+ * A contradicted claim produces {@link CONTEXT_PACK_EVIDENCE_VERDICT}+ * `mismatch` with the drifted fields listed. The claim is not rejected+ * (the outcome row still lands, unaltered) and it is not silently+ * corrected to match the disk. A ledger that hides disagreement is worse+ * than no ledger.+ *+ * ## Absence is not inability+ *+ * A sample id with no receipt behind it is an ordinary, expected state -+ * the id may be a telemetry id or an opaque request hash. A continuity+ * log that could not be READ is a different answer, and the two carry+ * different {@link CONTEXT_PACK_EVIDENCE_UNRESOLVED} tokens so a reader+ * can never mistake one for the other.+ *+ * ## No second hashing path+ *+ * Per this wave's architecture amendment, the digest is the one the+ * receipt already carries. Nothing here hashes anything; the comparison+ * runs on `integrity/stamp.ts`'s token vocabulary, which is the shared+ * mechanism for "recorded at write time vs. observed now".+ *+ * Gated + fail-open, like every other writer on this ledger: with the+ * gate off no payload is built and no write happens, and a throwing+ * write never fails the operation being measured. Appends are the plain+ * unfsynced continuity appends every telemetry writer uses.+ */++import { compareStamps, type StampMismatch, type StampTokens } from "../integrity/stamp.ts";+import { emitGatedTelemetry } from "./continuity/emit.ts";+import { appendContinuityRecord, listContinuityRecords } from "./continuity/store.ts";+import {+ CONTINUITY_AGENT_ID_KEY,+ CONTINUITY_SESSION_ID_KEY,+ type ContinuityPayload,+ type ContinuityRecord,+} from "./continuity/types.ts";+import { getContextReceipt } from "./context-receipts.ts";++/**+ * The evidence fields, in `integrity/stamp.ts`'s caller-owned naming.+ * Named once here and reused on both sides of every comparison so the+ * claim and the disk reading can never be spelled differently.+ */+export const CONTEXT_PACK_EVIDENCE_FIELD = Object.freeze({+ /** SHA-256 the kernel computed over the assembled pack text. */+ finalTextHash: "final_text_hash",+ /** How many artifacts the pack injected. */+ itemCount: "item_count",+ /** Codepoint length of the assembled pack text. */+ finalTextChars: "final_text_chars",+} as const);++/**+ * What the kernel read the evidence OFF. A registered token rather than+ * free text, so a reader can enumerate the surfaces that can back a+ * sample instead of parsing a sentence.+ */+export const CONTEXT_PACK_EVIDENCE_BASIS = Object.freeze({+ contextReceipt: "context_receipt",+} as const);++export type ContextPackEvidenceBasis =+ (typeof CONTEXT_PACK_EVIDENCE_BASIS)[keyof typeof CONTEXT_PACK_EVIDENCE_BASIS];++/**+ * Why no evidence could be obtained. The two members are deliberately+ * distinct: a sample with nothing behind it is a normal state, an+ * unreadable log is a fault, and collapsing them would report a broken+ * ledger as an empty one.+ */+export const CONTEXT_PACK_EVIDENCE_UNRESOLVED = Object.freeze({+ /** The log was read and holds no receipt with that id. */+ sampleAbsent: "sample_absent",+ /** The log could not be read at all, so nothing is known either way. */+ evidenceUnreadable: "evidence_unreadable",+} as const);++export type ContextPackEvidenceUnresolved =+ (typeof CONTEXT_PACK_EVIDENCE_UNRESOLVED)[keyof typeof CONTEXT_PACK_EVIDENCE_UNRESOLVED];++/** The four states a claim/evidence comparison can be in. Closed. */+export const CONTEXT_PACK_EVIDENCE_VERDICT = Object.freeze({+ /** Every field the agent asserted agrees with the disk. */+ match: "match",+ /** At least one asserted field disagrees. Recorded, never resolved. */+ mismatch: "mismatch",+ /** Evidence was obtained; the agent asserted nothing to compare. */+ unclaimed: "unclaimed",+ /** No evidence was obtained; see the unresolved reason token. */+ unresolved: "unresolved",+} as const);++export type ContextPackEvidenceVerdictToken =+ (typeof CONTEXT_PACK_EVIDENCE_VERDICT)[keyof typeof CONTEXT_PACK_EVIDENCE_VERDICT];++/**+ * What the acting agent asserts about the sample it is reporting on.+ * A closed set of fields - a caller cannot introduce a token name, so+ * the comparison can only ever run over evidence the kernel also has.+ * Every field is optional: an agent asserts what it knows and omits the+ * rest rather than inventing a value to fill the shape.+ */+export interface ContextPackEvidenceClaim {+ readonly finalTextHash?: string;+ readonly itemCount?: number;+ readonly finalTextChars?: number;+}++/** Evidence the kernel obtained for a sample, or the reason it did not. */+export type ContextPackEvidence =+ | {+ readonly resolved: true;+ readonly basis: ContextPackEvidenceBasis;+ readonly tokens: StampTokens;+ }+ | {+ readonly resolved: false;+ readonly reason: ContextPackEvidenceUnresolved;+ };++/** A verdict and the evidence behind it. Mismatches are empty unless `mismatch`. */+export interface ContextPackEvidenceJudgement {+ readonly verdict: ContextPackEvidenceVerdictToken;+ readonly mismatches: ReadonlyArray<StampMismatch>;+}++export interface ContextPackEvidenceInput {+ readonly createdAt?: string;+ readonly host?: string;+ readonly sessionId?: string;+ readonly turnId?: string;+ /**+ * The ACTING agent - the one posting the outcome. Self-asserted, like+ * every identity in this system; recorded as the claimant, never as a+ * verifier.+ */+ readonly agentId?: string;+ /** The sample id joining this record to its outcome row. */+ readonly sampleId: string;+ /** What the acting agent asserts about the sample, if anything. */+ readonly claim?: ContextPackEvidenceClaim;+}++export interface ContextPackEvidenceFilter {+ readonly host?: string;+ readonly sampleId?: string;+ readonly verdict?: ContextPackEvidenceVerdictToken;+ readonly since?: string;+ readonly until?: string;+ readonly limit?: number;+}++/**+ * Read the evidence for `sampleId` off disk.+ *+ * Total by construction: a store read that throws returns the+ * `evidenceUnreadable` token rather than propagating, because this runs+ * inside a telemetry emit that must not fail the operation it measures -+ * and the inability is NAMED in the return value, so no caller can+ * mistake it for "no evidence exists".+ */+export function resolveContextPackEvidence(vault: string, sampleId: string): ContextPackEvidence {+ let receipt: ContinuityRecord | null;+ try {+ receipt = getContextReceipt(vault, sampleId);+ } catch {+ return Object.freeze({+ resolved: false as const,+ reason: CONTEXT_PACK_EVIDENCE_UNRESOLVED.evidenceUnreadable,+ });+ }+ if (receipt === null) {+ return Object.freeze({+ resolved: false as const,+ reason: CONTEXT_PACK_EVIDENCE_UNRESOLVED.sampleAbsent,+ });+ }+ return Object.freeze({+ resolved: true as const,+ basis: CONTEXT_PACK_EVIDENCE_BASIS.contextReceipt,+ tokens: receiptEvidenceTokens(receipt.payload),+ });+}++/**+ * Compare the agent's claim against the evidence.+ *+ * Only the fields the claim ACTUALLY asserts are compared. Asking about+ * a field nobody claimed would report the agent's silence as a+ * disagreement, which is the opposite of what a stamp comparison means:+ * an unasserted field is unrecorded, and two unrecorded sides are not a+ * finding (`integrity/stamp.ts`).+ */+export function judgeContextPackEvidence(+ evidence: ContextPackEvidence,+ claim?: ContextPackEvidenceClaim,+): ContextPackEvidenceJudgement {+ return judgeClaimTokens(evidence, claimTokens(claim));+}++/**+ * The judgement over an already-canonicalised claim. Split out so the+ * writer below can put the same token map on the record AND judge it+ * without canonicalising twice - two canonicalisations of one claim is+ * exactly how a payload and its verdict drift apart.+ */+function judgeClaimTokens(+ evidence: ContextPackEvidence,+ claimed: StampTokens,+): ContextPackEvidenceJudgement {+ if (!evidence.resolved) return frozenJudgement(CONTEXT_PACK_EVIDENCE_VERDICT.unresolved, []);+ const fields = Object.keys(claimed);+ if (fields.length === 0) return frozenJudgement(CONTEXT_PACK_EVIDENCE_VERDICT.unclaimed, []);+ const observed: Record<string, string | null> = {};+ for (const field of fields) observed[field] = evidence.tokens[field] ?? null;+ const mismatches = compareStamps(claimed, Object.freeze(observed));+ return frozenJudgement(+ mismatches.length === 0+ ? CONTEXT_PACK_EVIDENCE_VERDICT.match+ : CONTEXT_PACK_EVIDENCE_VERDICT.mismatch,+ mismatches,+ );+}++/**+ * Append one `context_pack_evidence` record, gated and fail-open.+ *+ * `gate` doubles as the opt-in switch: with `false | null | undefined`+ * no payload is built and no write happens. A throwing build - a blank+ * sample id, a non-finite claimed count - is swallowed and reported as+ * `null`, so the evidence half can never fail the outcome half.+ */+export function recordContextPackEvidence<G>(+ vault: string,+ input: ContextPackEvidenceInput,+ gate: G | false | null | undefined,+): ContinuityRecord | null {+ return emitGatedTelemetry(gate, () => {+ const sampleId = requireSampleId(input.sampleId);+ const evidence = resolveContextPackEvidence(vault, sampleId);+ const claimed = claimTokens(input.claim);+ const judgement = judgeClaimTokens(evidence, claimed);+ const payload: Record<string, unknown> = {+ ...(input.host !== undefined ? { host: input.host } : {}),+ ...(input.sessionId !== undefined ? { [CONTINUITY_SESSION_ID_KEY]: input.sessionId } : {}),+ ...(input.turnId !== undefined ? { turn_id: input.turnId } : {}),+ ...(input.agentId !== undefined ? { [CONTINUITY_AGENT_ID_KEY]: input.agentId } : {}),+ sample_id: sampleId,+ verdict: judgement.verdict,+ // Omit-don't-invent on both sides: a field the receipt never+ // recorded, and a field the agent never asserted, are absent+ // rather than rendered as a null that reads like a measurement.+ ...(evidence.resolved+ ? { evidence_basis: evidence.basis, kernel_evidence: definedTokens(evidence.tokens) }+ : { unresolved_reason: evidence.reason }),+ ...(Object.keys(claimed).length > 0 ? { agent_claim: claimed } : {}),+ ...(judgement.mismatches.length > 0 ? { mismatches: judgement.mismatches } : {}),+ };+ return appendContinuityRecord(vault, {+ kind: "context_pack_evidence",+ createdAt: input.createdAt ?? new Date().toISOString(),+ sourceRefs: [],+ payload,+ });+ });+}++/** List `context_pack_evidence` rows newest-first, after applying filters. */+export function listContextPackEvidence(+ vault: string,+ filter: ContextPackEvidenceFilter = {},+): ReadonlyArray<ContinuityRecord> {+ let records = listContinuityRecords(vault, {+ kind: "context_pack_evidence",+ ...(filter.since !== undefined ? { since: filter.since } : {}),+ ...(filter.until !== undefined ? { until: filter.until } : {}),+ }).filter((record) => matchesFilter(record, filter));+ records = records.toReversed();+ if (filter.limit !== undefined) records = records.slice(0, Math.max(0, Math.floor(filter.limit)));+ return Object.freeze(records);+}++/**+ * The evidence a `context_receipt` payload carries, in stamp-token form.+ * A field the receipt did not record reads `null` - unrecorded, per+ * `integrity/stamp.ts`, and distinct from a recorded empty string.+ */+function receiptEvidenceTokens(payload: ContinuityPayload): StampTokens {+ return Object.freeze({+ [CONTEXT_PACK_EVIDENCE_FIELD.finalTextHash]: stringToken(+ payload[CONTEXT_PACK_EVIDENCE_FIELD.finalTextHash],+ ),+ [CONTEXT_PACK_EVIDENCE_FIELD.itemCount]: countToken(+ payload[CONTEXT_PACK_EVIDENCE_FIELD.itemCount],+ ),+ [CONTEXT_PACK_EVIDENCE_FIELD.finalTextChars]: countToken(+ payload[CONTEXT_PACK_EVIDENCE_FIELD.finalTextChars],+ ),+ });+}++/**+ * The agent's claim in the same token form, through the same+ * canonicalisation - so a claimed count and a recorded count can never+ * disagree because one side stringified differently from the other.+ * Unasserted fields are absent, not null: the claim asserts what it+ * asserts and nothing more.+ */+function claimTokens(claim: ContextPackEvidenceClaim | undefined): StampTokens {+ if (claim === undefined) return Object.freeze({});+ const tokens: Record<string, string> = {};+ if (claim.finalTextHash !== undefined) {+ tokens[CONTEXT_PACK_EVIDENCE_FIELD.finalTextHash] = requireClaimString(+ CONTEXT_PACK_EVIDENCE_FIELD.finalTextHash,+ claim.finalTextHash,+ );+ }+ if (claim.itemCount !== undefined) {+ tokens[CONTEXT_PACK_EVIDENCE_FIELD.itemCount] = requireClaimCount(+ CONTEXT_PACK_EVIDENCE_FIELD.itemCount,+ claim.itemCount,+ );+ }+ if (claim.finalTextChars !== undefined) {+ tokens[CONTEXT_PACK_EVIDENCE_FIELD.finalTextChars] = requireClaimCount(+ CONTEXT_PACK_EVIDENCE_FIELD.finalTextChars,+ claim.finalTextChars,+ );+ }+ return Object.freeze(tokens);+}++/** Drop the unrecorded fields so the payload states only what disk holds. */+function definedTokens(tokens: StampTokens): Readonly<Record<string, string>> {+ const out: Record<string, string> = {};+ for (const [field, value] of Object.entries(tokens)) {+ if (value !== null) out[field] = value;+ }+ return Object.freeze(out);+}++function stringToken(value: unknown): string | null {+ return typeof value === "string" ? value : null;+}++/** A recorded count becomes its canonical decimal string; anything else is unrecorded. */+function countToken(value: unknown): string | null {+ return typeof value === "number" && Number.isFinite(value) ? String(value) : null;+}++function requireClaimString(field: string, value: string): string {+ if (typeof value !== "string" || value.trim().length === 0) {+ throw new TypeError(`context pack evidence: claimed ${field} must be a non-empty string`);+ }+ return value.trim();+}++function requireClaimCount(field: string, value: number): string {+ const token = countToken(value);+ if (token === null || value < 0) {+ throw new TypeError(`context pack evidence: claimed ${field} must be a finite number >= 0`);+ }+ return token;+}++function requireSampleId(raw: unknown): string {+ if (typeof raw !== "string" || raw.trim().length === 0) {+ throw new TypeError("context pack evidence: sampleId must be a non-empty string");+ }+ return raw.trim();+}++function frozenJudgement(+ verdict: ContextPackEvidenceVerdictToken,+ mismatches: ReadonlyArray<StampMismatch>,+): ContextPackEvidenceJudgement {+ return Object.freeze({ verdict, mismatches: Object.freeze([...mismatches]) });+}++function matchesFilter(record: ContinuityRecord, filter: ContextPackEvidenceFilter): boolean {+ const payload = record.payload;+ if (filter.host !== undefined && payload["host"] !== filter.host) return false;+ if (filter.sampleId !== undefined && payload["sample_id"] !== filter.sampleId) return false;+ if (filter.verdict !== undefined && payload["verdict"] !== filter.verdict) return false;+ return true;+}diff --git a/src/core/brain/context-pack-outcome.ts b/src/core/brain/context-pack-outcome.tsindex f1cb6910..b61304bf 100644--- a/src/core/brain/context-pack-outcome.ts+++ b/src/core/brain/context-pack-outcome.ts@@ -34,14 +34,25 @@* {@link recordTokenImpactOutcome}), correlated by the same sample id, so the* modeled figure there is calibrated by measured recall outcomes.*+ * Composes the evidence half (provenance-at-the-boundary, unit I): every+ * outcome row also posts a `context_pack_evidence` record on the same sample+ * id, carrying the evidence the kernel read back off disk and the verdict of+ * the acting agent's claim against it (via {@link recordContextPackEvidence}).+ * The outcome row itself is never altered by that verdict - a contradicted+ * claim is recorded as a mismatch, not rejected and not silently corrected.+ ** Gated + fail-open: emits route through {@link emitGatedTelemetry}, so with* the gate off no payload is built and no write happens, and a throwing write* never fails the operation being measured.*/+import {+ recordContextPackEvidence,+ type ContextPackEvidenceClaim,+} from "./context-pack-evidence.ts";import { emitGatedTelemetry } from "./continuity/emit.ts";import { appendContinuityRecord, listContinuityRecords } from "./continuity/store.ts";-import type { ContinuityRecord } from "./continuity/types.ts";+import { CONTINUITY_AGENT_ID_KEY, type ContinuityRecord } from "./continuity/types.ts";import { recordTokenImpactOutcome, type TokenImpactOutcome } from "./token-impact.ts";export interface ContextPackOutcomeInput {@@ -55,6 +66,22 @@ export interface ContextPackOutcomeInput {* a raw prompt or recalled text.*/readonly sampleId: string;+ /**+ * The ACTING agent posting this outcome, clip-protected beside+ * session_id (t_5be0654d) and spread onto all three rows this post+ * lands. Self-asserted, like every identity in this system, and never+ * presented as a verifier.+ *+ * Supplied by the CALLER, not resolved from config here. The sibling+ * `token_impact*` payloads have declared the same field since+ * t_5be0654d and no production caller has ever set it, so "the siblings+ * already carry it" was never true of anything on disk; the surface that+ * makes it true is `brain_context_pack_outcome`'s `agent_id` argument,+ * which reaches this row, the token-impact calibration row and the+ * evidence row from one place. Omitted records no actor - a guessed+ * identity is worse than a missing one.+ */+ readonly agentId?: string;/** Whether the packed context led to a first-pass success (no repair/retry). */readonly firstPassSuccess: boolean;/** Whether the agent had to repair the first completion. */@@ -69,6 +96,13 @@ export interface ContextPackOutcomeInput {readonly modeledInferenceAvoidance?: number;/** OBSERVED provider-reported token usage for this inference. */readonly observedProviderTokens?: number;+ /**+ * What the acting agent asserts about the sample it is reporting on,+ * checked against the evidence the kernel reads off disk. Omitted when+ * the agent asserts nothing - the evidence record is still written, and+ * says so.+ */+ readonly evidenceClaim?: ContextPackEvidenceClaim;}export interface ContextPackOutcomeFilter {@@ -102,21 +136,36 @@ export interface ContextPackOutcomeSummary {readonly follow_up: { readonly samples: number; readonly tokens: number };}+/** The records one `post` lands, so a caller need not read them back. */+export interface ContextPackOutcomePost {+ /** The `context_pack_outcome` row. */+ readonly outcome: ContinuityRecord;+ /**+ * The `context_pack_evidence` row joined to it on the sample id.+ * `null` only when that half fail-opened - the evidence writer is+ * gated and fail-open in its own right, so it can never take the+ * outcome row down with it.+ */+ readonly evidence: ContinuityRecord | null;+}+/**- * Emit one `context_pack_outcome` row, gated and fail-open, AND post a- * matching first-pass/repair/retry calibration record to the token-impact- * ledger (C3). `gate` doubles as the opt-in switch: with- * `false | null | undefined` no payload is built and no write happens- * (returns `null`). A throwing build - including a blank sample id - is- * swallowed and reported as `null` so the loop can never fail the operation- * it measures. Returns the `context_pack_outcome` record (the calibration- * record is a durable side effect, listable via `listTokenImpactOutcomes`).+ * Post one outcome: the `context_pack_outcome` row, the matching+ * first-pass/repair/retry calibration record on the token-impact ledger+ * (C3), and the `context_pack_evidence` record carrying the kernel's+ * reading of the sample against the agent's claim (unit I). All three are+ * joined by the one sample id.+ *+ * `gate` doubles as the opt-in switch: with `false | null | undefined` no+ * payload is built and no write happens (returns `null`). A throwing build -+ * including a blank sample id - is swallowed and reported as `null` so the+ * loop can never fail the operation it measures.*/-export function emitContextPackOutcome<G>(+export function postContextPackOutcome<G>(vault: string,input: ContextPackOutcomeInput,gate: G | false | null | undefined,-): ContinuityRecord | null {+): ContextPackOutcomePost | null {return emitGatedTelemetry(gate, (openGate) => {const sampleId = requireSampleId(input.sampleId);if (typeof input.firstPassSuccess !== "boolean") {@@ -127,6 +176,7 @@ export function emitContextPackOutcome<G>(...(input.host !== undefined ? { host: input.host } : {}),...(input.sessionId !== undefined ? { session_id: input.sessionId } : {}),...(input.turnId !== undefined ? { turn_id: input.turnId } : {}),+ ...(input.agentId !== undefined ? { [CONTINUITY_AGENT_ID_KEY]: input.agentId } : {}),sample_id: sampleId,first_pass_success: input.firstPassSuccess,// Omit-don't-invent: every optional counter is written only when supplied.@@ -178,6 +228,7 @@ export function emitContextPackOutcome<G>(createdAt,...(input.host !== undefined ? { host: input.host } : {}),...(input.sessionId !== undefined ? { sessionId: input.sessionId } : {}),+ ...(input.agentId !== undefined ? { agentId: input.agentId } : {}),packId: sampleId,outcome: deriveOutcome(input),...(input.observedProviderTokens !== undefined@@ -186,10 +237,42 @@ export function emitContextPackOutcome<G>(},openGate,);- return record;+ // Compose the evidence half: what the kernel reads off disk for this+ // sample, and how the agent's claim compares. Also gated + fail-open,+ // so it never throws through this thunk either - the outcome row above+ // is already durable and stays that way regardless of the verdict.+ const evidence = recordContextPackEvidence(+ vault,+ {+ createdAt,+ ...(input.host !== undefined ? { host: input.host } : {}),+ ...(input.sessionId !== undefined ? { sessionId: input.sessionId } : {}),+ ...(input.turnId !== undefined ? { turnId: input.turnId } : {}),+ ...(input.agentId !== undefined ? { agentId: input.agentId } : {}),+ sampleId,+ ...(input.evidenceClaim !== undefined ? { claim: input.evidenceClaim } : {}),+ },+ openGate,+ );+ return Object.freeze({ outcome: record, evidence });});}+/**+ * Emit one `context_pack_outcome` row and its two joined side effects,+ * returning the outcome record alone. The thin form of+ * {@link postContextPackOutcome} for callers that only need the row; the+ * calibration and evidence records are durable side effects, listable via+ * `listTokenImpactOutcomes` and `listContextPackEvidence`.+ */+export function emitContextPackOutcome<G>(+ vault: string,+ input: ContextPackOutcomeInput,+ gate: G | false | null | undefined,+): ContinuityRecord | null {+ return postContextPackOutcome(vault, input, gate)?.outcome ?? null;+}+/*** Extract the recall/context-pack quality-sample id from a pack report to* carry through the session. The context-receipt id ({@linkdiff --git a/src/core/brain/continuity/read-model.ts b/src/core/brain/continuity/read-model.tsindex ec2687cf..b66c8d05 100644--- a/src/core/brain/continuity/read-model.ts+++ b/src/core/brain/continuity/read-model.ts@@ -1,10 +1,9 @@/*** Continuity read-model (Memory Observability Suite kernel).*- * The single normalization layer between the raw JSONL continuity- * store and every read-side consumer (ATOF/ATIF export, bench- * harness). It absorbs three concerns exactly once so consumers- * cannot disagree on them:+ * The normalization layer between the raw JSONL continuity store and the+ * read-side consumers that go through it. It absorbs three concerns+ * exactly once so those consumers cannot disagree on them:** - schema-version dispatch: records written before the stamp* existed carry no `schema` field and read as v1 (`legacy: true`);@@ -14,6 +13,13 @@* - fail-soft reads: malformed rows normalize to null and unknown* kinds stay readable (the evolution rule is additive).*+ * It is NOT the only door onto the store, and the docblock used to imply+ * it was. Four modules read through here; twenty call+ * `listContinuityRecords` directly and see raw records with none of the+ * above applied. `tests/core/brain/continuity/reader-census.test.ts`+ * holds both lists and fails when either changes; auditing the direct+ * readers is a separate task, by design.+ ** Read-only by construction - this module never writes to the store.*/diff --git a/src/core/brain/continuity/types.ts b/src/core/brain/continuity/types.tsindex 314a9cec..9a6beb55 100644--- a/src/core/brain/continuity/types.ts+++ b/src/core/brain/continuity/types.ts@@ -43,6 +43,7 @@ export type ContinuityRecordKind =| "token_impact"| "token_impact_outcome"| "context_pack_outcome"+ | "context_pack_evidence"| "host_memory_write"| "recall_observed_use"| "skill_invoked"diff --git a/src/core/brain/dedup-telemetry.ts b/src/core/brain/dedup-telemetry.tsindex 6950b50c..9feec4b5 100644--- a/src/core/brain/dedup-telemetry.ts+++ b/src/core/brain/dedup-telemetry.ts@@ -250,7 +250,7 @@ export function summarizeIngestDedup(for (const [ref, count] of readSourceMap(payload["by_source"])) {// Axis label folded into the key so the same ref on two surfaces// never merges into one misleading rollup.- const key = `${surface} ${ref}`;+ const key = `${surface}\0${ref}`;const previous = bySource.get(key)?.rollup;bySource.set(key, {rollup: Object.freeze({diff --git a/src/core/brain/diagnostics.ts b/src/core/brain/diagnostics.tsindex 1a0c20e1..1fea48a2 100644--- a/src/core/brain/diagnostics.ts+++ b/src/core/brain/diagnostics.ts@@ -284,6 +284,77 @@ export const DIAGNOSTIC_SIGNALS: ReadonlyMap<string, DiagnosticSignal> = new MapnextCommand: "o2b search index",autoRepairable: false,},+ // --- Semantic capability tiers (provenance-at-the-boundary, F1) ---+ // WHAT THE OPERATOR CONFIGURED, resolved by+ // `core/search/capability-tier.ts` and reported by the four+ // surfaces that used to each derive it themselves - one of them by+ // assembling these very sentences beside the call. The `SearchError`+ // codes answer the different question of what a call attempted and+ // could not do; nothing reports both for one condition.+ //+ // The exit for the two blocked tiers is the pre-flight diagnostic+ // rather than an edit command: the remediation is a config key, and+ // no verb in this tool writes one. `search check` is what names the+ // key, the provider and the env var for the operator to set.+ {+ code: "semantic-capability-disabled",+ // Two configurations reach this rung - `search_semantic_enabled=false`+ // and `embedding_provider=disabled` - so the sentence names the state+ // rather than one of its causes. It is a thrown error message on the+ // explicit arm now, where naming the wrong key would send an operator+ // to edit a setting that is already correct. `o2b search check` reports+ // which of the two it is.+ issueClass: "semantic search not configured (disabled by config or provider)",+ nextCommand: "o2b search check",+ autoRepairable: false,+ },+ {+ code: "semantic-capability-key-missing",+ issueClass: "embedding_api_key not configured",+ nextCommand: "o2b search check",+ autoRepairable: false,+ },+ {+ // Not a fault - the tier an operator wants to be at. It is+ // registered because the ladder must be total: a surface that+ // resolves a label per tier cannot have one rung with no entry,+ // and the honest exit from "configured" is to populate vectors.+ code: "semantic-capability-configured",+ issueClass: "semantic search configured",+ nextCommand: "o2b search vector-backfill",+ autoRepairable: false,+ },+ // --- Vector-population states (provenance-at-the-boundary, F) ---+ // Index facts, deliberately named apart from the capability tiers:+ // a chunk with no vector is a `chunks` row with no `embeddings` row,+ // which is a first-class, already-resumable state and not a+ // degraded one.+ {+ code: "semantic-vectors-deferred",+ issueClass: "embeddings not requested this run",+ nextCommand: "o2b search vector-backfill --apply",+ autoRepairable: false,+ },+ {+ code: "semantic-vectors-pending",+ issueClass: "indexed chunks with no vector",+ nextCommand: "o2b search vector-backfill --apply",+ autoRepairable: false,+ },+ // --- Event-anchor population (provenance-at-the-boundary) ---+ // The same class of index fact as the vector states, for the same+ // reason: a schema bump migrates in place and reindexes nothing,+ // and the indexer resolves an anchor only for content that+ // changed, so a document carried over from a pre-anchor binary+ // never gets one. It is a distinct code because the remedy is a+ // distinct verb, and because "no anchor resolved" must never be+ // reported as the note declaring no date.+ {+ code: "event-anchors-pending",+ issueClass: "indexed documents with no event anchor resolved",+ nextCommand: "o2b search event-anchor-backfill --apply",+ autoRepairable: false,+ },{code: "git-history-absent",issueClass: "no ingested git history",@@ -296,6 +367,18 @@ export const DIAGNOSTIC_SIGNALS: ReadonlyMap<string, DiagnosticSignal> = new MapnextCommand: "o2b brain bridges discover",autoRepairable: false,},+ {+ // provenance-at-the-boundary, unit B. A caller-named write+ // landed outside the `write_binding.path_prefixes` the operator+ // declared. The exit is the per-path policy inspector, which+ // reports the binding verdict for one path beside the+ // vault-scope verdict, so the operator sees WHY the write was+ // refused and what the declaration currently admits.+ code: "write-binding-refused",+ issueClass: "caller-named write outside the declared write binding",+ nextCommand: "o2b vault inspect <relpath>",+ autoRepairable: false,+ },{code: "cli-config-absent",issueClass: "no machine configuration file",diff --git a/src/core/brain/entities/index-builder.ts b/src/core/brain/entities/index-builder.tsindex c6f90645..171b2dca 100644Binary files a/src/core/brain/entities/index-builder.ts and b/src/core/brain/entities/index-builder.ts differdiff --git a/src/core/brain/entities/page-scope.ts b/src/core/brain/entities/page-scope.tsnew file mode 100644index 00000000..55327d1e--- /dev/null+++ b/src/core/brain/entities/page-scope.ts@@ -0,0 +1,45 @@+/**+ * The status scope applied to a vault PAGE rather than to a parsed record.+ *+ * `Brain/entities/**` are ordinary Markdown pages, so every reader that+ * walks the vault - not only the ones that go through the registry - sees+ * them. A quarantined entity page carries a title, a body and wikilinks+ * that were authored by whatever untrusted source the intake quarantined,+ * and a link-graph walker that reads pages by path never touches+ * `buildEntityIndex`, so `status-scope.ts` alone cannot reach it.+ *+ * This is that reach: the same {@link entityStatusInScope} decision, taken+ * over the frontmatter `listVaultPages` already parsed, so a walker pays no+ * extra read.+ *+ * A page that does not declare the entity kind is not an entity record and+ * is admitted unchanged - this predicate narrows nothing outside+ * `Brain/entities/`. A page that DOES declare the kind but carries a status+ * outside the vocabulary is admitted by no scope, exactly as+ * {@link entityStatusInScope} answers for a value that never parsed: an+ * unrecognised status is not a reason to show the record.+ */++import {+ ENTITY_STATUS_SCOPE,+ entityStatusInScope,+ type EntityStatusScope,+} from "./status-scope.ts";+import { BRAIN_ENTITY_KIND } from "./types.ts";++export { ENTITY_STATUS_SCOPE };++/**+ * May a read at `scope` see the page whose frontmatter is `metadata`?+ *+ * Pages that are not entity records answer `true`. See the module docblock+ * for why an unparseable status on an entity page answers `false`.+ */+export function vaultPageInStatusScope(+ metadata: Readonly<Record<string, unknown>>,+ scope: EntityStatusScope,+): boolean {+ if (metadata["kind"] !== BRAIN_ENTITY_KIND) return true;+ const status = metadata["status"];+ return typeof status === "string" && entityStatusInScope(status, scope);+}diff --git a/src/core/brain/entities/registry.ts b/src/core/brain/entities/registry.tsindex bbeedcd3..e9260ed0 100644--- a/src/core/brain/entities/registry.ts+++ b/src/core/brain/entities/registry.ts@@ -7,6 +7,12 @@* doctor lints catch the ones that arrive by hand-editing or sync.* All operations are deterministic - the caller injects the clock.*+ * Writes run in one of two LANES ({@link resolveWriteTarget}): the canonical+ * one, and the quarantine holding untrusted-provenance records. A name can be+ * claimed once in each, which is not a duplicate - only one of the two is a+ * canonical entity, and the quarantined claim is a record OF a claim rather+ * than a record the vault stands behind.+ ** Files stay plain Obsidian Markdown. Every rewrite preserves unknown* frontmatter keys the operator may have added by hand.*/@@ -29,6 +35,12 @@ import {} from "./canonical.ts";import { resolveEntityLabelDenylist } from "./label-hygiene.ts";import { buildEntityIndex, parseEntityFile, type EntityIndex } from "./index-builder.ts";+import { ENTITY_STATUS_SCOPE, entityStatusInScope } from "./status-scope.ts";+import {+ INTAKE_TRUST,+ UNTRUSTED_SOURCE_FRONTMATTER_KEY,+ untrustedSourceFrontmatter,+} from "../trust/untrusted-provenance.ts";import {BRAIN_ENTITY_ID_PREFIX,BRAIN_ENTITY_KIND,@@ -50,8 +62,30 @@ export interface UpsertEntityInput {readonly confidence?: string;/** Markdown body (current structured state). Replaces on update. */readonly body?: string;+ /**+ * Markdown body used only if this upsert CREATES the record; an update+ * keeps the body that is there. The intake path stamps a source citation+ * this way: whether the write creates or updates is decided here, in the+ * lane it resolves in, and a caller that answered that question for itself+ * with a canonical read would get it wrong for the quarantine lane - and+ * would silently drop the citation of the source that introduced the+ * record. {@link body} still wins when both are given.+ */+ readonly bodyOnCreate?: string;/** Config path for denylist resolution; env still wins when set. */readonly configPath?: string;+ /**+ * This write is being made under untrusted provenance (see+ * `intake/source-trust.ts`), which selects the LANE it resolves and writes+ * in - see {@link resolveWriteTarget}. A record it introduces lands+ * `quarantine` and carries the untrusted-source marker the retrieval gate+ * reads, so it is absent from every ordinary read until an operator+ * releases it with {@link archiveEntity}'s `restore`.+ *+ * Absent → the canonical lane, byte-identical to an upsert that never knew+ * about trust.+ */+ readonly untrustedOrigin?: boolean;}export interface UpsertEntityResult {@@ -69,11 +103,27 @@ export interface RelateEntitiesInput {readonly relation: string;readonly to: EntityRef;readonly now: Date;+ /**+ * This edge is being written under untrusted provenance, which selects the+ * lane both endpoints resolve in ({@link resolveWriteTarget}) - an intake+ * that just quarantined two entities links THOSE, and never the operator's+ * records of the same names. Absent → the canonical lane, byte-identical+ * to a relate that never knew about trust.+ */+ readonly untrustedOrigin?: boolean;}export interface ArchiveEntityOptions {readonly now: Date;- /** When true, return an archived entity to active lookup. */+ /**+ * Return a record that is outside the canonical scope - archived OR+ * quarantined - to active lookup. This is the named exit from quarantine:+ * without it a record that entered under untrusted provenance would be+ * invisible with no way back, and a status nothing can leave is a dead end+ * rather than a lane. A release that would collide with a canonical record+ * already holding the same name is refused with both ids - see the check in+ * {@link archiveEntity}.+ */readonly restore?: boolean;}@@ -89,7 +139,11 @@ export class EntityAmbiguityError extends Error {}}-/** Resolve a ref against ACTIVE entities: canonical name first, then alias. */+/**+ * Resolve a ref at the CANONICAL status scope: canonical name first, then+ * alias. The index's lookup maps are built at the same scope, so the two+ * branches below agree by construction rather than by coincidence.+ */function resolveActive(index: EntityIndex, ref: EntityRef): BrainEntity | null {const query = normalizeEntityName(ref.query);if (!query) return null;@@ -102,7 +156,7 @@ function resolveActive(index: EntityIndex, ref: EntityRef): BrainEntity | null {}const matches = index.entities.filter((e) =>- e.status === BRAIN_ENTITY_STATUS.active &&+ entityStatusInScope(e.status, ENTITY_STATUS_SCOPE.canonical) &&(normalizeEntityName(e.name) === query ||e.aliases.some((a) => normalizeEntityName(a) === query)),);@@ -115,16 +169,90 @@ function resolveActive(index: EntityIndex, ref: EntityRef): BrainEntity | null {return matches[0] ?? null;}+/**+ * Resolve a ref inside the QUARANTINE lane: the records that hold a name+ * without being entitled to it, which no read scope admits. Kept separate+ * from {@link resolveActive} rather than chained after it, because the two+ * answer for different authorities and a write belongs to exactly one.+ */+function resolveQuarantined(index: EntityIndex, ref: EntityRef): BrainEntity | null {+ const query = normalizeEntityName(ref.query);+ if (!query) return null;+ const category = ref.category !== undefined ? validateEntityCategory(ref.category) : undefined;+ const matches = index.entities.filter(+ (e) =>+ e.status === BRAIN_ENTITY_STATUS.quarantine &&+ (category === undefined || e.category === category) &&+ (normalizeEntityName(e.name) === query ||+ e.aliases.some((a) => normalizeEntityName(a) === query)),+ );+ if (matches.length > 1) {+ throw new EntityAmbiguityError(+ ref.query,+ matches.map((m) => m.id),+ );+ }+ return matches[0] ?? null;+}++/**+ * Resolve the record a WRITE lands on, in the lane the write belongs to.+ *+ * This is the seam where an untrusted source could otherwise take a name for+ * good. A single "who holds this name" resolution that answered canonical+ * first and quarantined second looked like the careful choice - it kept a+ * re-ingest from forking a second page - but it answered the same for a+ * write the vault had vouched for. So the first scraped page to mention+ * `Acme Corp` created the quarantined record, and every later TRUSTED write+ * of that name landed INSIDE it: the update arm copies the target's status,+ * so the record stayed quarantined, kept the untrusted-source marker in its+ * carried-forward frontmatter, and stayed invisible to `getEntity`, to the+ * default listing, to alias resolution and to the retrieval gate - reported+ * to the writer as `created: false`, which reads as "already there".+ *+ * The lanes are separate identity spaces instead:+ *+ * - a TRUSTED write resolves canonical records only. A quarantined+ * namesake does not hold the name against it: the write creates its own+ * record, `created: true`, readable, unmarked. It is also not refused,+ * which matters - refusing would let any hostile page reserve a name the+ * operator then could not use.+ * - an UNTRUSTED write resolves quarantined records only. Re-ingesting the+ * same untrusted source still updates its own record rather than forking+ * one, but an untrusted mention of a name the operator already holds can+ * no longer reach that record - not its aliases, not its `source_agent`,+ * not its body, not its edges. Nothing it carries can arrive unmarked.+ *+ * Neither direction can therefore capture the other, and the two spaces are+ * joined only where a human is present: {@link archiveEntity}'s `restore`,+ * which refuses a release that would collide.+ */+function resolveWriteTarget(+ index: EntityIndex,+ ref: EntityRef,+ untrustedOrigin: boolean,+): BrainEntity | null {+ return untrustedOrigin ? resolveQuarantined(index, ref) : resolveActive(index, ref);+}+export function getEntity(vault: string, ref: EntityRef): BrainEntity | null {return resolveActive(buildEntityIndex(vault), ref);}+/**+ * List entities. Without a `status`, the listing is the READABLE scope: the+ * statuses a surface may show, which is `active` and `archived` exactly as+ * before, and never `quarantine`. A `status` is an explicit ask and is+ * honoured verbatim - that is how an operator sees what is quarantined.+ */export function listEntities(vault: string, opts: ListEntitiesOptions = {}): BrainEntity[] {const category = opts.category !== undefined ? validateEntityCategory(opts.category) : undefined;return buildEntityIndex(vault).entities.filter((e) =>(category === undefined || e.category === category) &&- (opts.status === undefined || e.status === opts.status),+ (opts.status === undefined+ ? entityStatusInScope(e.status, ENTITY_STATUS_SCOPE.readable)+ : e.status === opts.status),);}@@ -233,13 +361,11 @@ export function upsertEntity(vault: string, input: UpsertEntityInput): UpsertEntconst index = buildEntityIndex(vault);const stamp = isoSecond(input.now);- // Resolve to an existing canonical entity: name key first, alias second.+ // Resolve inside this write's own lane (see resolveWriteTarget): name key+ // first, alias second, and never across the trusted/untrusted boundary.+ const untrusted = input.untrustedOrigin === true;const key = entityIdentityKey(category, name);- let target = index.byKey.get(key) ?? null;- if (target === null) {- const viaAlias = index.byAlias.get(normalizeEntityName(name));- if (viaAlias && viaAlias.category === category) target = viaAlias;- }+ const target = resolveWriteTarget(index, { category, query: name }, untrusted);if (target === null) {// The name may be held by an archived entity - refuse with the remedy@@ -289,7 +415,17 @@ export function upsertEntity(vault: string, input: UpsertEntityInput): UpsertEntupdated_at: stamp,tags: ["brain", "brain/entity"],};- writeEntityFile(target.path, fields, meta, input.body ?? existingBody, { overwrite: true });+ // The target's status is carried forward, which is now safe by+ // construction: the lane guarantees a trusted write never sees a+ // quarantined target and an untrusted one never sees a canonical target.+ // The marker is re-asserted on the untrusted lane rather than trusted to+ // the frontmatter that was there, so status and marker cannot drift apart+ // through a hand edit or a partial sync.+ const extras: FrontmatterMap = {+ ...meta,+ ...(untrusted ? untrustedSourceFrontmatter(INTAKE_TRUST.untrusted) : {}),+ };+ writeEntityFile(target.path, fields, extras, input.body ?? existingBody, { overwrite: true });const entity = parseEntityFile(target.path);if (entity === null) throw new Error(`entity file unreadable after write: ${target.path}`);return { entity, created: false };@@ -297,25 +433,50 @@ export function upsertEntity(vault: string, input: UpsertEntityInput): UpsertEntconst id = allocateEntityId(index, category, name);const path = entityPath(vault, category, id);+ // Untrusted provenance decides two things about the new record: the status+ // it lands at, and the marker the retrieval gate reads. Both come from the+ // one lane flag, so they cannot disagree.const fields: FrontmatterMap = {kind: BRAIN_ENTITY_KIND,entity_id: id,category,name,...(aliases.length > 0 ? { aliases } : {}),- status: BRAIN_ENTITY_STATUS.active,+ status: untrusted ? BRAIN_ENTITY_STATUS.quarantine : BRAIN_ENTITY_STATUS.active,source_agent: input.agent,...(input.confidence !== undefined ? { confidence: input.confidence } : {}),created_at: stamp,updated_at: stamp,tags: ["brain", "brain/entity"],};- writeEntityFile(path, fields, {}, input.body ?? `# ${name}`, { overwrite: false });+ const marker = untrustedSourceFrontmatter(+ untrusted ? INTAKE_TRUST.untrusted : INTAKE_TRUST.trusted,+ );+ writeEntityFile(path, fields, marker, input.body ?? input.bodyOnCreate ?? `# ${name}`, {+ overwrite: false,+ });const entity = parseEntityFile(path);if (entity === null) throw new Error(`entity file unreadable after write: ${path}`);return { entity, created: true };}+/**+ * Why an endpoint did not resolve, and what to do about it. A canonical+ * relate that misses a name a quarantined record holds is not a missing+ * entity - it is a record the caller is not entitled to reach yet, and the+ * message names the release that changes that rather than leaving a dead end.+ */+function missingEndpoint(index: EntityIndex, ref: EntityRef, untrustedOrigin: boolean): string {+ const base = `entity not found: ${JSON.stringify(ref.query)}`;+ if (untrustedOrigin) return base;+ const quarantined = resolveQuarantined(index, ref);+ if (quarantined === null) return base;+ return (+ `${base} - ${quarantined.id} holds that name in quarantine, where an untrusted source ` ++ "introduced it. Release it (entity archive --restore) to relate it."+ );+}+export function relateEntities(vault: string, input: RelateEntitiesInput): BrainEntity {// Vault-identity write guard (context-integrity-gates, Unit J).assertVaultIdentityForWrite(vault);@@ -326,10 +487,16 @@ export function relateEntities(vault: string, input: RelateEntitiesInput): Brain);}const index = buildEntityIndex(vault);- const from = resolveActive(index, input.from);- if (from === null) throw new Error(`entity not found: ${JSON.stringify(input.from)}`);- const to = resolveActive(index, input.to);- if (to === null) throw new Error(`entity not found: ${JSON.stringify(input.to)}`);+ // Both endpoints resolve in the edge's own lane: an intake that just+ // quarantined two entities must be able to link them, and the edge stays+ // inside the quarantine because both endpoints are unreadable - while an+ // untrusted edge can never attach itself to the operator's record of the+ // same name.+ const untrusted = input.untrustedOrigin === true;+ const from = resolveWriteTarget(index, input.from, untrusted);+ if (from === null) throw new Error(missingEndpoint(index, input.from, untrusted));+ const to = resolveWriteTarget(index, input.to, untrusted);+ if (to === null) throw new Error(missingEndpoint(index, input.to, untrusted));if (from.id === to.id) throw new Error("an entity cannot relate to itself");const [meta, body] = parseFrontmatter(from.path);@@ -372,10 +539,13 @@ export function archiveEntity(const index = buildEntityIndex(vault);let target: BrainEntity | null;if (opts.restore) {+ // Restore resolves what canonical lookup cannot see - archived AND+ // quarantined - because both are records that hold a name without being+ // reachable through it.const query = normalizeEntityName(ref.query);const matches = index.entities.filter((e) =>- e.status === BRAIN_ENTITY_STATUS.archived &&+ !entityStatusInScope(e.status, ENTITY_STATUS_SCOPE.canonical) &&(ref.category === undefined || e.category === validateEntityCategory(ref.category)) &&(normalizeEntityName(e.name) === query ||e.aliases.some((a) => normalizeEntityName(a) === query)),@@ -398,6 +568,29 @@ export function archiveEntity(const nextStatus = opts.restore ? BRAIN_ENTITY_STATUS.active : BRAIN_ENTITY_STATUS.archived;if (target.status === nextStatus) return target;+ if (opts.restore) {+ // Release is where the two write lanes meet again, and a trusted record+ // may have taken this name while the quarantined one was out of scope+ // (that is exactly what a trusted write does with a quarantined+ // namesake). Promoting into that collision would put two active records+ // on one identity key - a duplicate the index reports and no read can+ // resolve deterministically. Which of the two survives is a judgement+ // about the material, so it goes back to the operator with both ids.+ const key = entityIdentityKey(target.category, target.name);+ const holder = index.entities.find(+ (e) =>+ e.id !== target.id &&+ entityStatusInScope(e.status, ENTITY_STATUS_SCOPE.canonical) &&+ entityIdentityKey(e.category, e.name) === key,+ );+ if (holder) {+ throw new Error(+ `cannot restore ${target.id}: ${holder.id} already holds '${target.name}' ` ++ `(${target.category}). Archive or rename ${holder.id} first, or merge the two.`,+ );+ }+ }+const [meta, body] = parseFrontmatter(target.path);const stamp = isoSecond(opts.now);const fields: FrontmatterMap = {@@ -416,7 +609,12 @@ export function archiveEntity(};// `archived_at` must disappear on restore: writeEntityFile only emits// the keys present in `fields`, and extras never override own fields.- writeEntityFile(target.path, fields, meta, body, { overwrite: true });+ // The untrusted-source marker must disappear too, and it is an EXTRA, so+ // it is dropped explicitly: a released record the retrieval gate still+ // excluded would be a restore that reads as success without being one.+ const extras: FrontmatterMap = { ...meta };+ if (opts.restore) delete extras[UNTRUSTED_SOURCE_FRONTMATTER_KEY];+ writeEntityFile(target.path, fields, extras, body, { overwrite: true });const entity = parseEntityFile(target.path);if (entity === null) throw new Error(`entity file unreadable after write: ${target.path}`);return entity;diff --git a/src/core/brain/entities/semantic-dedup.ts b/src/core/brain/entities/semantic-dedup.tsindex 1a342b99..e2b01bf4 100644--- a/src/core/brain/entities/semantic-dedup.ts+++ b/src/core/brain/entities/semantic-dedup.ts@@ -27,7 +27,9 @@ import type { EmbeddingProvider } from "../../search/embeddings/contract.ts";import { jaccard, tokenise } from "../similarity.ts";import { buildEntityIndex } from "./index-builder.ts";import { normalizeEntityName } from "./canonical.ts";-import { BRAIN_ENTITY_STATUS, type BrainEntity } from "./types.ts";+import { ENTITY_STATUS_SCOPE, entityStatusInScope } from "./status-scope.ts";+import { providerProducesVectors } from "../../search/embeddings/contract.ts";+import type { BrainEntity } from "./types.ts";/*** Cosine threshold for the embedding layer. Default deliberately high so@@ -168,8 +170,8 @@ export function entityEvolutionChain(entity: BrainEntity): ReadonlyArray<string>/** Active entities, id-sorted, capped — the comparison universe. */function activeEntities(vault: string): BrainEntity[] {- const entities = buildEntityIndex(vault).entities.filter(- (e) => e.status === BRAIN_ENTITY_STATUS.active,+ const entities = buildEntityIndex(vault).entities.filter((e) =>+ entityStatusInScope(e.status, ENTITY_STATUS_SCOPE.canonical),);return entities.slice(0, ENTITY_DEDUP_CANDIDATE_CAP);}@@ -316,7 +318,7 @@ export async function detectEntityAliasCandidates(),});- if (provider === null || provider.name === "null") return lexical();+ if (provider === null || !providerProducesVectors(provider)) return lexical();const entities = activeEntities(vault);const pairs = comparablePairs(entities);diff --git a/src/core/brain/entities/status-scope.ts b/src/core/brain/entities/status-scope.tsnew file mode 100644index 00000000..45b1cf8e--- /dev/null+++ b/src/core/brain/entities/status-scope.ts@@ -0,0 +1,78 @@+/**+ * The one place that decides which entity statuses a read may see.+ *+ * Before this module the `active` filter was re-implemented at every caller+ * and eight callers applied none at all, so adding a third status value would+ * have read as "not archived, therefore visible" at almost every read path in+ * the tree - a quarantine nothing filters on, which is a no-op wearing a+ * feature's name. The filter is one predicate now, and+ * `tests/core/brain/entities/entity-read-census.test.ts` fails the build when+ * a new entity read path bypasses it.+ *+ * Two scopes, because read paths ask two different questions:+ *+ * - `canonical` - may this record HOLD the identity a lookup resolves to?+ * Only `active` does. An archived name is retired; a quarantined name was+ * never vouched for. This is the scope the identity index and every+ * name/alias resolution use.+ * - `readable` - may a surface show this record, or reason over it at all?+ * `active` and `archived` may; `quarantine` may not. This is the scope an+ * unfiltered listing, a guard, or an anchoring pass uses.+ *+ * `quarantine` is admitted by NEITHER. That is the entire content of the+ * value: a record that entered under untrusted provenance stays out of every+ * ordinary read until an operator releases it (`entity archive --restore`).+ * Reaching it at all takes an explicit status ask.+ *+ * Structural only. The decision reads a controlled-vocabulary token and never+ * note prose, so there is no natural-language word list here or downstream.+ */++import { BRAIN_ENTITY_STATUS, type BrainEntityStatus } from "./types.ts";++/** The question a read path is asking. See the module docblock. */+export const ENTITY_STATUS_SCOPE = {+ canonical: "canonical",+ readable: "readable",+} as const;++export type EntityStatusScope = (typeof ENTITY_STATUS_SCOPE)[keyof typeof ENTITY_STATUS_SCOPE];++/**+ * Which scopes admit each status. Keyed by {@link BrainEntityStatus}, so a+ * new member of the vocabulary cannot be added in `types.ts` without failing+ * the type check here until someone decides what may see it - the decision is+ * never made by omission.+ */+const SCOPES_ADMITTING: Readonly<Record<BrainEntityStatus, ReadonlyArray<EntityStatusScope>>> =+ Object.freeze({+ [BRAIN_ENTITY_STATUS.active]: Object.freeze([+ ENTITY_STATUS_SCOPE.canonical,+ ENTITY_STATUS_SCOPE.readable,+ ]),+ [BRAIN_ENTITY_STATUS.archived]: Object.freeze([ENTITY_STATUS_SCOPE.readable]),+ [BRAIN_ENTITY_STATUS.quarantine]: Object.freeze([]),+ });++/**+ * Lookup keyed by the raw string. The entity-shaped slices the guards+ * consume (`atomic-facts`, `truth/contamination`, `truth/merge-guard`) carry+ * `status: string` rather than the narrowed type, so the predicate has to+ * answer for a value that never parsed - and a `Map` answers it without an+ * `as` cast that would let an unparsed value pretend to be vocabulary.+ */+const SCOPES_BY_STATUS: ReadonlyMap<string, ReadonlyArray<EntityStatusScope>> = new Map(+ Object.entries(SCOPES_ADMITTING),+);++/**+ * May a read at `scope` see a record carrying `status`?+ *+ * A status outside the vocabulary is admitted by no scope. That is the only+ * answer that cannot leak an unrecognised record into a surface, and it is+ * different from "the record is absent": the record exists, the caller was+ * simply not entitled to it at this scope.+ */+export function entityStatusInScope(status: string, scope: EntityStatusScope): boolean {+ return SCOPES_BY_STATUS.get(status)?.includes(scope) ?? false;+}diff --git a/src/core/brain/entities/types.ts b/src/core/brain/entities/types.tsindex 36016493..2ae59592 100644--- a/src/core/brain/entities/types.ts+++ b/src/core/brain/entities/types.ts@@ -10,17 +10,33 @@import type { RelationEdge } from "../../graph/frontmatter-relations.ts";+/**+ * Lifecycle status of a stored entity.+ *+ * `quarantine` is not a third kind of retirement: it records that the entity+ * ENTERED the vault under untrusted provenance and has never been vouched+ * for. Which reads may see each value is not decided here - it is decided+ * once, in `status-scope.ts`, so a new value cannot be added without every+ * read path getting an answer.+ */export const BRAIN_ENTITY_STATUS = {active: "active",archived: "archived",+ quarantine: "quarantine",} as const;export type BrainEntityStatus = (typeof BRAIN_ENTITY_STATUS)[keyof typeof BRAIN_ENTITY_STATUS];-const STATUS_VALUES: ReadonlyArray<string> = Object.values(BRAIN_ENTITY_STATUS);+/** Every status, in declaration order. The one list a surface may render. */+export const BRAIN_ENTITY_STATUS_VALUES: ReadonlyArray<BrainEntityStatus> = Object.freeze(+ Object.values(BRAIN_ENTITY_STATUS),+);export function isBrainEntityStatus(value: unknown): value is BrainEntityStatus {- return typeof value === "string" && STATUS_VALUES.includes(value);+ return (+ typeof value === "string" &&+ BRAIN_ENTITY_STATUS_VALUES.some((status): boolean => status === value)+ );}/** Frontmatter `kind:` marker of an entity file. */diff --git a/src/core/brain/fact-extract.ts b/src/core/brain/fact-extract.tsindex 6f7f8114..bfb47c58 100644--- a/src/core/brain/fact-extract.ts+++ b/src/core/brain/fact-extract.ts@@ -41,7 +41,7 @@ import {sanitizeEntityLabel,validateEntityLabel,} from "./entities/canonical.ts";-import { BRAIN_ENTITY_STATUS } from "./entities/types.ts";+import { ENTITY_STATUS_SCOPE, entityStatusInScope } from "./entities/status-scope.ts";export type FactFamily = "url" | "email" | "quantity";@@ -257,7 +257,7 @@ function buildAnchorables(): AnchorableEntity[] {const out: AnchorableEntity[] = [];for (const entity of index.entities) {- if (entity.status !== BRAIN_ENTITY_STATUS.active) continue;+ if (!entityStatusInScope(entity.status, ENTITY_STATUS_SCOPE.canonical)) continue;const nameVerdict = validateEntityLabel(sanitizeEntityLabel(entity.name));if (!nameVerdict.valid) {try {diff --git a/src/core/brain/handoff.ts b/src/core/brain/handoff.tsindex 5655aaca..d33e2f57 100644--- a/src/core/brain/handoff.ts+++ b/src/core/brain/handoff.ts@@ -9,10 +9,11 @@* the source session file remains the lossless record.*/-import { mkdirSync, writeFileSync } from "node:fs";import { join } from "node:path";-import { computeSourceStamp, formatSourceStampFrontmatter } from "./freshness.ts";+import type { FrontmatterMap } from "../types.ts";+import { formatFrontmatter, writeFrontmatterAtomic } from "../vault.ts";+import { computeSourceStamp } from "./freshness.ts";import { isoDate, isoSecond } from "./time.ts";import { resolveSessionScope } from "./session-scope.ts";import { assertVaultIdentityForWrite } from "./vault-identity.ts";@@ -143,26 +144,32 @@ export function writeHandoffNote(vault: string, input: WriteHandoffNoteInput): Hconst now = input.now ?? new Date();const scope = resolveSessionScope(input.sessionId);const dir = join(vault, "Brain", "handoffs");- mkdirSync(dir, { recursive: true });const path = input.targetPath ?? join(dir, `${isoDate(now)}-${scope}.md`);const body = buildHandoffNote(input.turns, input);- // JSON.stringify-quote the caller-supplied scalars: YAML-significant- // characters or newlines in a session id / agent name must not be- // able to break or inject frontmatter fields.- const content = [- "---",- `session_id: ${JSON.stringify(input.sessionId)}`,- `scope: ${scope}`,- `agent: ${JSON.stringify(input.agent)}`,- `created_at: ${isoSecond(now)}`,- `turns: ${input.turns.length}`,+ // The frontmatter is a MAP handed to the shared emitter, not a string+ // assembled here. The hand-rolled version quoted the caller-supplied+ // scalars with `JSON.stringify` to stop a YAML-significant character+ // or a newline in a session id breaking the block; `formatFrontmatter`+ // already owns that rule, and owns it more completely - it quotes on+ // the same structural test the PARSER uses and escapes the backslash+ // and control characters `JSON.stringify` alone left to chance. One+ // emitter, so a note this module writes and a note the vault reader+ // parses cannot disagree about quoting.+ const metadata: FrontmatterMap = {+ session_id: input.sessionId,+ scope,+ agent: input.agent,+ created_at: isoSecond(now),+ turns: input.turns.length,...(input.sourcePaths !== undefined && input.sourcePaths.length > 0- ? [formatSourceStampFrontmatter(computeSourceStamp(vault, input.sourcePaths))]- : []),- "---",- "",- body,- ].join("\n");- writeFileSync(path, content);- return Object.freeze({ path, scope, content });+ ? computeSourceStamp(vault, input.sourcePaths)+ : {}),+ };+ // `overwrite` because both callers re-derive in place: the SessionEnd+ // hook rewrites the day's note and the targeted-recompile executor+ // refreshes an existing one by path. The atomic writer creates the+ // parent directory, so the note is never observed half-written and no+ // `mkdirSync` is needed ahead of it.+ writeFrontmatterAtomic(path, metadata, body, { overwrite: true });+ return Object.freeze({ path, scope, content: formatFrontmatter(metadata, body) });}diff --git a/src/core/brain/hygiene/detectors/dedup.ts b/src/core/brain/hygiene/detectors/dedup.tsindex 45c22679..07813e10 100644--- a/src/core/brain/hygiene/detectors/dedup.ts+++ b/src/core/brain/hygiene/detectors/dedup.ts@@ -21,6 +21,7 @@ import { join } from "node:path";import { resolveSearchConfig } from "../../../search/index.ts";import { makeProvider } from "../../../search/embeddings/provider.ts";+import { providerProducesVectors } from "../../../search/embeddings/contract.ts";import type { EmbeddingProvider } from "../../../search/embeddings/contract.ts";import { findMergeCandidates } from "../../merge-candidates.ts";import { brainDirs } from "../../paths.ts";@@ -103,7 +104,7 @@ function cosine(a: ReadonlyArray<number>, b: ReadonlyArray<number>): number {function resolveConfiguredProvider(vault: string): EmbeddingProvider | null {try {const provider = makeProvider(resolveSearchConfig({ vault }).semantic);- return provider.name === "null" ? null : provider;+ return providerProducesVectors(provider) ? provider : null;} catch {return null;}@@ -130,7 +131,10 @@ export async function detectSemanticDedup(opts.lexicalThreshold !== undefined ? { lexicalThreshold: opts.lexicalThreshold } : {},),});- if (provider === null || provider.name === "null") return lexical();+ // An INJECTED provider reaches here without passing through+ // `resolveConfiguredProvider`, so the contract predicate is asked again+ // rather than assumed (provenance-at-the-boundary, F2).+ if (provider === null || !providerProducesVectors(provider)) return lexical();const candidates = listPreferenceCandidates(vault);if (candidates.length < 2) return lexical();diff --git a/src/core/brain/ingest/ingest.ts b/src/core/brain/ingest/ingest.tsindex ec2cbffd..bc0cb7a8 100644--- a/src/core/brain/ingest/ingest.ts+++ b/src/core/brain/ingest/ingest.ts@@ -34,6 +34,8 @@ import { isoSecond } from "../time.ts";import { sourcePagePath } from "../paths.ts";import { assertVaultIdentityForWrite } from "../vault-identity.ts";import { intakeExtraction, type ExtractionIntake } from "../intake/extract-intake.ts";+import { classifySourceTrust } from "../intake/source-trust.ts";+import { untrustedSourceFrontmatter } from "../trust/untrusted-provenance.ts";import {renderProvenanceSection,sourceIdentityHash,@@ -117,11 +119,17 @@ export function ingestSource(const preExtract = opts.preExtract === true ? runPreExtract(vault, canonicalSource) : undefined;const sourceLink = `[[${canonicalSource}]]`;const provenance: Provenance = { level: "stated", sources: [sourceLink], premises: [] };+ // Derived per source, not stamped uniformly: until this unit every source+ // carried the same provenance regardless of where it came from, which made+ // a scraped URL and a file in the operator's own vault indistinguishable at+ // the moment they entered the brain.+ const trust = classifySourceTrust(vault, canonicalSource);const intake = intakeExtraction(vault, input.extraction, {agent: opts.agent,now: opts.now,provenance,+ trust,});const connections = intake.entitiesUpdated;const allEntities = [...intake.entitiesCreated, ...intake.entitiesUpdated];@@ -143,6 +151,11 @@ export function ingestSource(source_path: canonicalSource,source_hash: sourceHash,provenance: provenance.level,+ // Marks the summary page itself when the material behind it came from+ // outside the vault, so the retrieval trust gate excludes it with a+ // reason rather than ranking it beside the operator's own notes. Trusted+ // sources add nothing, keeping their page byte-identical to before.+ ...untrustedSourceFrontmatter(trust),created_at: createdAt,updated_at: stamp,tags: ["brain", "brain/source"],diff --git a/src/core/brain/ingest/pre-extract.ts b/src/core/brain/ingest/pre-extract.tsindex c503e8b0..f935fb92 100644--- a/src/core/brain/ingest/pre-extract.ts+++ b/src/core/brain/ingest/pre-extract.ts@@ -234,11 +234,11 @@ function extensionOf(path: string): string {}function entityKey(e: CodeEntitySeed): string {- return `${e.kind} ${e.name}`;+ return `${e.kind}\0${e.name}`;}function edgeKey(e: CodeEdgeSeed): string {- return `${e.kind} ${e.from} ${e.to}`;+ return `${e.kind}\0${e.from}\0${e.to}`;}/** Dedupe by a stable key and sort by that key, so output is deterministic. */diff --git a/src/core/brain/intake/extract-intake.ts b/src/core/brain/intake/extract-intake.tsindex 8e0368b8..715845fe 100644--- a/src/core/brain/intake/extract-intake.ts+++ b/src/core/brain/intake/extract-intake.ts@@ -20,12 +20,23 @@* authoritative cross-source citation list, so an entity page cites the* source that first introduced it without being clobbered on every later* mention.+ *+ * Trust: the caller may declare where the material came from+ * ({@link IntakeOptions.trust}); when it does not, the trust is DERIVED from+ * the sources the provenance cites, through the same structural classifier+ * both callers use (see {@link resolveIntakeTrust} for why the undeclared+ * case cannot simply read as trusted). An untrusted intake writes in the+ * registry's quarantine lane: the entities it introduces land quarantined and+ * marked, and they are records of what an untrusted source claimed rather+ * than edits to what the operator already holds.*/import { isKnownRelation, normalizeRelation } from "../../graph/relation-vocab.ts";import { validateEntityCategory, normalizeEntityName } from "../entities/canonical.ts";-import { getEntity, relateEntities, upsertEntity } from "../entities/registry.ts";+import { relateEntities, upsertEntity } from "../entities/registry.ts";import { renderProvenanceSection, type Provenance } from "../provenance/provenance.ts";+import { INTAKE_TRUST, type IntakeTrust } from "../trust/untrusted-provenance.ts";+import { classifySourceTrust } from "./source-trust.ts";/** One entity the agent extracted from a source. */export interface IntakeEntity {@@ -58,6 +69,13 @@ export interface IntakeOptions {readonly now: Date;/** When set, its Sources section is stamped into newly created entity bodies. */readonly provenance?: Provenance;+ /**+ * Whether the material this extraction came from carries the vault's own+ * authority. Both in-repo callers declare it, deriving it from the source+ * identity through `classifySourceTrust`. Absent → derived here from the+ * cited provenance; see {@link resolveIntakeTrust}.+ */+ readonly trust?: IntakeTrust;}export interface IntakeResult {@@ -118,6 +136,35 @@ function validateIntake(intake: ExtractionIntake): void {}}+/**+ * The trust this intake commits under.+ *+ * A declared trust wins: the caller classified the identity it actually read+ * from, which is more than this primitive can see. When nothing is declared,+ * the CITED sources decide, through the one structural classifier - reading+ * an undeclared trust as trusted would have made "the caller forgot" and "the+ * operator wrote it" the same answer, and a hostile page is exactly the input+ * that arrives with a source cited and no classification done. Several+ * sources resolve conservatively: an extraction drawn from a mixed batch+ * cannot be split back apart afterwards, so one source outside the vault+ * makes the whole intake untrusted.+ *+ * An intake that cites NO source and declares no trust is the one case this+ * function cannot decide, and it keeps the pre-change verdict so a caller+ * that never knew about trust behaves identically. That is deliberately not+ * where the question is answered: nothing here can ask the caller what it+ * read, so the MCP boundary - which can - refuses an intake that names no+ * source at all rather than routing an unanswerable one down either lane.+ */+function resolveIntakeTrust(vault: string, opts: IntakeOptions): IntakeTrust {+ if (opts.trust !== undefined) return opts.trust;+ const sources = opts.provenance?.sources ?? [];+ if (sources.length === 0) return INTAKE_TRUST.trusted;+ return sources.every((source) => classifySourceTrust(vault, source) === INTAKE_TRUST.trusted)+ ? INTAKE_TRUST.trusted+ : INTAKE_TRUST.untrusted;+}+/*** Intake a typed extraction into the entity registry. Validates the full* payload first (throwing {@link IntakeValidationError} with no write on@@ -131,19 +178,20 @@ export function intakeExtraction(validateIntake(intake);const provenanceSection = opts.provenance ? renderProvenanceSection(opts.provenance) : "";+ const untrustedOrigin = resolveIntakeTrust(vault, opts) === INTAKE_TRUST.untrusted;const entitiesCreated: string[] = [];const entitiesUpdated: string[] = [];for (const entity of intake.entities) {const category = validateEntityCategory(entity.category);- const existing = getEntity(vault, { category, query: entity.name });// Stamp provenance into the body only when creating the page, so a later- // mention of the same entity does not overwrite its first citation.- const body =- existing === null && provenanceSection- ? `# ${entity.name.trim()}\n\n${provenanceSection}`- : undefined;+ // mention of the same entity does not overwrite its first citation. The+ // registry decides whether this is a creation, in the lane it writes in;+ // asking here would answer for the canonical lane only.+ const bodyOnCreate = provenanceSection+ ? `# ${entity.name.trim()}\n\n${provenanceSection}`+ : undefined;const res = upsertEntity(vault, {category,name: entity.name,@@ -151,7 +199,8 @@ export function intakeExtraction(agent: opts.agent,now: opts.now,...(entity.confidence !== undefined ? { confidence: entity.confidence } : {}),- ...(body !== undefined ? { body } : {}),+ ...(bodyOnCreate !== undefined ? { bodyOnCreate } : {}),+ ...(untrustedOrigin ? { untrustedOrigin } : {}),});if (res.created) entitiesCreated.push(res.entity.id);else entitiesUpdated.push(res.entity.id);@@ -167,6 +216,10 @@ export function intakeExtraction(relation: rel.relation,to: { query: rel.to, ...(rel.toCategory !== undefined ? { category: rel.toCategory } : {}) },now: opts.now,+ // Same lane as the entities this intake just wrote, so an untrusted+ // extraction links its own quarantined records instead of reaching the+ // operator's records that happen to carry those names.+ ...(untrustedOrigin ? { untrustedOrigin } : {}),});relationsApplied += 1;}diff --git a/src/core/brain/intake/source-trust.ts b/src/core/brain/intake/source-trust.tsnew file mode 100644index 00000000..2cb663a1--- /dev/null+++ b/src/core/brain/intake/source-trust.ts@@ -0,0 +1,155 @@+/**+ * Where a source came from, decided once for both intake callers.+ *+ * The source-ingest pipeline used to stamp one hardcoded provenance level on+ * every source it saw, which made a scraped URL and a file in the operator's+ * own vault indistinguishable at the moment they entered the brain. There was+ * no untrusted signal at intake to join to, so this module originates one.+ *+ * The decision is STRUCTURAL - it reads the shape of the source identity and+ * nothing else. No word list, no host list, no scheme list: a URI scheme is+ * recognised by its grammar (RFC 3986 §3.1, via {@link hasUriScheme}) and+ * containment is decided by the same vault-boundary check every writer in+ * this repository already funnels through.+ *+ * POLARITY. The first version of this module asked "is this identity provably+ * outside the vault?" and trusted everything else, which handed trust to every+ * identity it simply could not parse: `evil.com/article` and+ * `www.example.com/x` are not absolute, carry no scheme and do not climb out+ * of the vault, so they entered as if the operator had written them, and an+ * agent reading a hostile page could be told by that page to drop the scheme.+ * The question asked here is the other one: is this identity SHAPED like a+ * location inside this vault? Only a yes is trusted. A shape that establishes+ * nothing is not a shape that establishes authority.+ *+ * WHAT THE SHAPE CANNOT SETTLE. `evil.com` and `readme.md` are the same+ * structure - dot-separated labels, no separator - because a hostname and a+ * vault-relative filename are drawn from the same character set. Nothing short+ * of asking the filesystem tells them apart, and asking it would make a note+ * the operator has not written yet untrusted, which confuses absence with+ * authority. So a single-segment identity is resolved by the one structural+ * signal available: it is trusted when it carries this vault's note+ * extension, and untrusted otherwise. `evil.com` therefore lands untrusted,+ * `readme.md` trusted, and the residual ambiguity (`.md` is also a country+ * TLD) costs nothing an attacker did not already have - an identity is a+ * claim, and an attacker willing to write `evil.md` was equally free to write+ * `primer.md`. The shape gate stops the identities that could not be ours; it+ * was never able to stop a lie about one that could.+ *+ * A COLON IS NOT A SCHEME. {@link hasUriScheme} is shared with the markdown+ * link reader, where its input is a link target and a leading `word:` can only+ * be a scheme. A source identity is a FILENAME, a different domain: this+ * project's own vault names notes `Meeting: Q3 planning.md`. Reused unguarded+ * the predicate read that colon as a scheme and quarantined the operator's own+ * note - and quarantine is one-way, so the entities of that intake left every+ * ordinary read with nothing but a success response to show for it. The fix is+ * the reuse, not the predicate: it is applied here only where a scheme can+ * legally appear (before the first separator) and only to a segment that could+ * be a URI at all (a URI admits no whitespace, RFC 3986 §2).+ */++import { isAbsolute, join } from "node:path";++import { canonicalNotePath, ensureInsideVault, hasUriScheme } from "../../path-safety.ts";+import { INTAKE_TRUST, type IntakeTrust } from "../trust/untrusted-provenance.ts";++/** `[[Articles/x.md]]` - the wikilink form the NER tool's `source` arrives in. */+const WIKILINK_WRAPPER_RE = /^\[\[(.*)\]\]$/s;++/** The separator {@link canonicalNotePath} normalises every identity to. */+const PATH_SEPARATOR = "/";++/** The no-op path segment, dropped before the shape is read. */+const SAME_DIR_SEGMENT = ".";++/** The climbing segment, left in place for the vault-boundary check to judge. */+const PARENT_DIR_SEGMENT = "..";++/**+ * The extension a note in this vault carries. Used for ONE decision - telling+ * a single-segment filename apart from a bare hostname - because that is the+ * only place where the path shape alone cannot decide.+ */+const VAULT_NOTE_EXTENSION = ".md";++/** Any whitespace character; a URI contains none (RFC 3986 §2). */+const WHITESPACE_RE = /\s/;++/**+ * Could this segment be a URI prefix? Whitespace rules it out before the+ * scheme grammar is consulted, so a filename that merely contains a colon+ * (`Meeting: Q3 planning.md`) is never mistaken for `mailto:`.+ */+function bearsUriScheme(segment: string): boolean {+ return !WHITESPACE_RE.test(segment) && hasUriScheme(segment);+}++/**+ * Is this leading segment shaped like an authority rather than a directory?+ * A dot in the FIRST segment is the residue of a scheme-less address+ * (`evil.com/article`); vault directories are named, not dotted. `..` is+ * excluded because it is a path operator, and the vault-boundary check below+ * is the one entitled to answer for it.+ */+function isAuthorityShaped(segment: string): boolean {+ return segment !== PARENT_DIR_SEGMENT && segment.includes(".");+}++/** Does this segment name a note file in this vault's own extension? */+function hasVaultNoteExtension(segment: string): boolean {+ return segment.toLowerCase().endsWith(VAULT_NOTE_EXTENSION);+}++/**+ * Classify a source identity as trusted or untrusted.+ *+ * Trusted means exactly one thing: the identity is shaped like a location+ * inside this vault, which is the only authority this system actually has.+ * Every other shape - an address with a URI scheme, a protocol-relative or+ * absolute path, a leading segment shaped like a host, a bare name that is+ * not a note, a path that climbs out of the vault, or no identity at all - is+ * untrusted.+ *+ * Note what is NOT tested: whether the file is there. An absent file and an+ * unreadable one are different answers to a different question, and neither+ * changes who was entitled to name the location. A source the operator has+ * not written yet is still inside their own namespace.+ *+ * An EMPTY identity is untrusted here rather than an error, because this+ * function answers about an identity it was given. "The caller named no+ * source at all" is a different failure with a different remedy - the caller+ * can be asked - and it belongs to the boundary that can still ask, which is+ * why `brain_intake_entities` refuses it instead of routing it here.+ */+export function classifySourceTrust(vault: string, sourcePath: string): IntakeTrust {+ const inner = sourcePath.trim().replace(WIKILINK_WRAPPER_RE, "$1").trim();+ if (inner.length === 0) return INTAKE_TRUST.untrusted;++ const canonical = canonicalNotePath(inner);+ if (isAbsolute(canonical)) return INTAKE_TRUST.untrusted;++ const segments = canonical.split(PATH_SEPARATOR).filter((s) => s !== SAME_DIR_SEGMENT);+ // An empty segment is a doubled separator: the `//host` authority marker,+ // the remains of `scheme://host`, or a trailing slash naming a directory.+ // None of the three is a note identity this vault owns.+ if (segments.some((segment) => segment.length === 0)) return INTAKE_TRUST.untrusted;++ // A scheme precedes the path, so only the segments before the last one can+ // carry it. The final segment is a basename and its colons are its own.+ if (segments.slice(0, -1).some(bearsUriScheme)) return INTAKE_TRUST.untrusted;++ const head = segments[0] ?? "";+ const establishedShape =+ segments.length > 1 ? !isAuthorityShaped(head) : hasVaultNoteExtension(head);+ if (!establishedShape) return INTAKE_TRUST.untrusted;++ try {+ ensureInsideVault(join(vault, canonical), vault);+ } catch {+ // The identity climbs out of the vault (or escapes it through a+ // symlink). It names a location this vault does not own, which is the+ // same verdict as naming one on another host.+ return INTAKE_TRUST.untrusted;+ }+ return INTAKE_TRUST.trusted;+}diff --git a/src/core/brain/link-graph/bridge-discovery.ts b/src/core/brain/link-graph/bridge-discovery.tsindex 506cc0cd..57caaa3e 100644--- a/src/core/brain/link-graph/bridge-discovery.ts+++ b/src/core/brain/link-graph/bridge-discovery.ts@@ -34,6 +34,7 @@ import {parseFrontmatter,writeFrontmatterAtomic,} from "../../vault.ts";+import { ENTITY_STATUS_SCOPE, vaultPageInStatusScope } from "../entities/page-scope.ts";import { resolveNotePath } from "../note-path.ts";import type { SchemaPack } from "../schema-pack.ts";import { assertVaultIdentityForWrite } from "../vault-identity.ts";@@ -339,10 +340,17 @@ export function acceptBridge(return Object.freeze({ changed: true, related: Object.freeze(next) });}-/** Number of vault pages sharing one basename (without `.md`). */+/**+ * Number of vault pages sharing one basename (without `.md`).+ *+ * The walk has no skip list, so it reaches `Brain/entities/` like any other+ * subtree. A quarantined entity page must not make an operator's link+ * target read as ambiguous, so it is not counted.+ */function countBasenameMatches(vault: string, base: string): number {let count = 0;for (const page of listVaultPages(vault)) {+ if (!vaultPageInStatusScope(page.metadata, ENTITY_STATUS_SCOPE.readable)) continue;if (basename(page.path, ".md") === base) count++;}return count;diff --git a/src/core/brain/link-graph/graph-holdout.ts b/src/core/brain/link-graph/graph-holdout.tsindex f108da06..2ac74f67 100644--- a/src/core/brain/link-graph/graph-holdout.ts+++ b/src/core/brain/link-graph/graph-holdout.ts@@ -21,6 +21,7 @@ import { join } from "node:path";import { ensureInsideVault } from "../../path-safety.ts";import { EXCLUDED_DIRS, extractWikilinks, listVaultPages, parseFrontmatter } from "../../vault.ts";+import { ENTITY_STATUS_SCOPE, vaultPageInStatusScope } from "../entities/page-scope.ts";import { canonicalCoOccurrenceKey } from "./co-occurrence.ts";/** Upper bound on the evidence hydrated from a target note. */@@ -66,10 +67,18 @@ export interface HoldoutGateResult {readonly resolutions: readonly HoldoutResolution[];}-/** Canonical-key adjacency over the vault's resolved wikilink structure. */+/**+ * Canonical-key adjacency over the vault's resolved wikilink structure.+ *+ * The walk covers `Brain/`, so entity pages are in it. A quarantined+ * record's links were authored by the untrusted source that put it aside,+ * and admitting them would let that source shape the graph this lane+ * measures recall over.+ */function buildKeyAdjacency(vault: string): Map<string, Set<string>> {const adjacency = new Map<string, Set<string>>();for (const page of listVaultPages(vault, { skipDirs: [...EXCLUDED_DIRS] })) {+ if (!vaultPageInStatusScope(page.metadata, ENTITY_STATUS_SCOPE.readable)) continue;const sourceKey = canonicalCoOccurrenceKey(page.path);if (sourceKey === null) continue;let body = "";diff --git a/src/core/brain/link-graph/repair-lane.ts b/src/core/brain/link-graph/repair-lane.tsindex d8287d7d..24108439 100644--- a/src/core/brain/link-graph/repair-lane.ts+++ b/src/core/brain/link-graph/repair-lane.ts@@ -34,6 +34,7 @@ import {writeFrontmatterAtomic,} from "../../vault.ts";import { listContinuityRecords } from "../continuity/store.ts";+import { ENTITY_STATUS_SCOPE, vaultPageInStatusScope } from "../entities/page-scope.ts";import { canonicalCoOccurrenceKey, computeCoOccurrenceSuggestions } from "./co-occurrence.ts";import { assertVaultIdentityForWrite } from "../vault-identity.ts";@@ -344,9 +345,18 @@ interface CollectedPage {readonly linkedKeys: ReadonlySet<string>;}+/**+ * Every page the repair lane may reason over.+ *+ * The walk covers `Brain/`, so entity pages are in it. A quarantined+ * record's title, body and links come from an untrusted source; proposing+ * repairs that cite them would put that source's text in front of the+ * operator as a suggestion.+ */function loadPages(vault: string): CollectedPage[] {const out: CollectedPage[] = [];for (const page of listVaultPages(vault, { skipDirs: [...EXCLUDED_DIRS] })) {+ if (!vaultPageInStatusScope(page.metadata, ENTITY_STATUS_SCOPE.readable)) continue;const rel = canonicalNotePath(relative(vault, page.path));const key = canonicalCoOccurrenceKey(rel);if (key === null) continue;diff --git a/src/core/brain/notes/create-note.ts b/src/core/brain/notes/create-note.tsindex fcb151e4..95d89e02 100644--- a/src/core/brain/notes/create-note.ts+++ b/src/core/brain/notes/create-note.ts@@ -1,69 +1,284 @@/**- * createNote primitive (Brain Portability & Interop suite, Unit D).+ * createNote primitive (Brain Portability & Interop suite, Unit D;+ * authoring modes from provenance-at-the-boundary, Unit C).** The single write primitive behind the `brain_create_note` MCP tool and* the SDK `createNote` method. It writes one Markdown note atomically,* funnelled through `ensureInsideVault`, and refuses - with a typed* {@link CreateNoteError}, never a silent skip - any path that is not a- * `.md` file, traverses outside the vault, lands in the Brain machinery- * root, is excluded by the vault scope, or would clobber an existing- * file. Refusing loudly is deliberate: a connected agent must learn its- * write was rejected rather than believe a note exists when it does not.+ * `.md` file, traverses outside the vault, spells its separators in a+ * way this host would read differently, lands in the Brain machinery+ * root, is excluded by the vault scope, falls outside the operator's+ * declared write binding, or would clobber an existing file. A vault+ * whose own `Brain/_brain.yaml` does not load is refused the same way+ * rather than raising an untyped fault. Refusing loudly is deliberate: a+ * connected agent must learn its write was rejected rather than believe+ * a note exists when it does not.+ *+ * The path this primitive REPORTS is the path the bytes are at. That is+ * a property of the envelope, not a coincidence: the reported string is+ * the one `inspectPath` normalised and the one every guard judged, and+ * the two shapes that used to break it - a caller separator rewritten on+ * the way out, and a symlinked directory redirecting the write past the+ * binding - are refused rather than reported.+ *+ * Three opt-in authoring modes sit on top of that envelope, each inert+ * when its field is absent:+ *+ * - `ifExists: "skip"` turns the existing-target refusal into a+ * DISCRIMINATED no-op. The result says `outcome: "skipped"`, so a+ * caller that skipped can never read as a caller that created -+ * returning the same shape for both would be precisely the silent+ * fallback this project spent two releases removing. The default+ * stays `"refuse"`.+ * - `strict` runs the shared write-session artifact validator over the+ * document that would be written and reports its coded violations.+ * It is wiring, not new machinery: the validator and its schema+ * vocabulary binding already exist.+ * - `template` + `templateVariables` author the body through the+ * closed two-construct grammar in `./note-template.ts`.+ *+ * The batch surface (`../write-batch.ts`) deliberately exposes NONE of+ * these. A batch is all-or-nothing, and a per-operation skip would make+ * one `applied` count mean two different things in one result list.*/-import { mkdirSync } from "node:fs";-import { dirname, join, posix } from "node:path";+import { existsSync, mkdirSync } from "node:fs";+import { dirname, join, posix, sep, win32 } from "node:path";import type { FrontmatterMap } from "../../types.ts";import { ensureInsideVault } from "../../path-safety.ts";-import { writeFrontmatterAtomic } from "../../vault.ts";+import { formatFrontmatter, writeFrontmatterAtomic } from "../../vault.ts";import { inspectPath, resolveVaultScope } from "../../vault-scope/index.ts";-import { BRAIN_ROOT_REL } from "../paths.ts";+import { BRAIN_CONFIG_FILE, BRAIN_ROOT_REL } from "../paths.ts";+import { requireNextStep } from "../next-step.ts";+import { BrainConfigError } from "../policy.ts";+import { loadSchemaPack } from "../schema-pack.ts";import { assertVaultIdentityForWrite } from "../vault-identity.ts";+import { checkWriteBinding } from "../../write-binding/index.ts";+import { validateArtifact } from "../write-session/validate.ts";+import type { WriteSessionError } from "../write-session/types.ts";+import {+ NoteTemplateError,+ renderNoteTemplate,+ type NoteTemplateVariables,+} from "./note-template.ts";/** Machine-readable reason a {@link createNote} call was refused. */-export type CreateNoteErrorCode = "invalid_path" | "excluded" | "exists" | "outside_vault";+export type CreateNoteErrorCode =+ | "invalid_path"+ | "excluded"+ | "exists"+ | "outside_vault"+ | "invalid_document"+ | "invalid_template"+ // provenance-at-the-boundary, unit B. The operator's declared+ // boundary over caller-named destinations, kept distinct from+ // `excluded` on purpose: vault scope says a path is not part of the+ // vault at all, the binding says it is a real vault path that this+ // write surface may not author into.+ | "write_binding"+ // The vault's own `Brain/_brain.yaml` exists and does not validate, so+ // neither the vault scope nor the write binding can be determined.+ // Distinct from every other code here because it is NOT about the path+ // the caller named: no argument the caller could send would succeed,+ // and the operator - not the caller - holds the fix.+ | "config_invalid";export class CreateNoteError extends Error {readonly code: CreateNoteErrorCode;- constructor(code: CreateNoteErrorCode, message: string) {+ /**+ * Coded validator violations. Populated only for `invalid_document`,+ * where the caller needs the fix list rather than one sentence; empty+ * for every other code.+ */+ readonly violations: ReadonlyArray<WriteSessionError>;+ constructor(+ code: CreateNoteErrorCode,+ message: string,+ violations: ReadonlyArray<WriteSessionError> = [],+ ) {super(message);this.name = "CreateNoteError";this.code = code;+ this.violations = violations;}}+/** What to do when the target path is already occupied. */+export const CREATE_NOTE_IF_EXISTS = Object.freeze(["refuse", "skip"] as const);++/** Policy for an occupied target; `refuse` is the default and the pre-existing behaviour. */+export type CreateNoteIfExists = (typeof CREATE_NOTE_IF_EXISTS)[number];+export interface CreateNoteInput {/** Vault-relative target path; must end in `.md`. */readonly path: string;/** Optional frontmatter map written above the body. */readonly frontmatter?: FrontmatterMap;- /** Optional Markdown body. */+ /** Optional Markdown body. Mutually exclusive with `template`. */readonly content?: string;+ /** Occupied-target policy; absent means `refuse`. */+ readonly ifExists?: CreateNoteIfExists;+ /** Run the shared artifact validator before writing. Absent means no validation. */+ readonly strict?: boolean;+ /** Body template rendered through the closed two-construct grammar. */+ readonly template?: string;+ /** Variables the template may reference; requires `template`. */+ readonly templateVariables?: NoteTemplateVariables;}-export interface CreateNoteResult {+/** What a {@link createNote} call actually did. */+export type CreateNoteOutcome = "created" | "skipped";++/** A note written by this call. */+export interface CreatedNoteResult {/** Vault-relative POSIX path of the created note. */readonly path: string;+ readonly outcome: "created";readonly created: true;}+/** An occupied target left exactly as it was, under `ifExists: "skip"`. */+export interface SkippedNoteResult {+ /** Vault-relative POSIX path of the note that was left alone. */+ readonly path: string;+ readonly outcome: "skipped";+ readonly created: false;+}++/**+ * Discriminated on `outcome`. `created` is the boolean the MCP surface+ * has returned since the tool shipped and is kept in lockstep with it;+ * `outcome` is what new callers should branch on, because it stays+ * readable if a third disposition is ever added.+ */+export type CreateNoteResult = CreatedNoteResult | SkippedNoteResult;+/** A note path resolved through the shared write safety envelope. */export interface ResolvedNoteTarget {- /** Normalised vault-relative path (native separators). */+ /**+ * Normalised vault-relative path, POSIX. `inspectPath` joins its+ * segments with `/` on every platform, and {@link resolveNoteTarget}+ * refuses any caller path this host would read differently, so this is+ * the path the surfaces report AND the path the bytes are at — the two+ * cannot diverge.+ */readonly relPath: string;/** Absolute filesystem path, guaranteed inside the vault. */readonly abs: string;}+/**+ * The separator that is structure on Windows and an ordinary filename+ * character everywhere else. Named once because the reason it matters is+ * that this host's answer differs from the other host's.+ */+const WINDOWS_SEP = win32.sep;++/**+ * Registry code for a vault whose Brain config does not load, resolved+ * at module scope so registry drift fails at import rather than inside+ * the refusal it was meant to explain (same rule as the write binding's+ * own exit).+ */+export const NOTE_CONFIG_INVALID_CODE = "config-invalid";+const CONFIG_INVALID_EXIT = requireNextStep(NOTE_CONFIG_INVALID_CODE).nextCommand;++/** Vault-relative spelling of the config file every refusal here names. */+const BRAIN_CONFIG_REL = posix.join(BRAIN_ROOT_REL, BRAIN_CONFIG_FILE);++/**+ * Refuse a caller-named path whose separators this host does not read+ * the way the caller wrote them.+ *+ * THE DECISION, and why it is a refusal rather than a normalisation.+ * On POSIX a backslash is an ordinary filename character, and three+ * layers of this envelope disagreed about that: the write-binding+ * matcher split on it (so a binding on `Projects` admitted+ * `Projects\evil.md`), the result renderer rewrote it to `/` (so the+ * tool reported `Projects/evil.md`), and `inspectPath`, the `..`+ * traversal guard, the `Brain/` machinery-root check and every+ * POSIX-written vault-scope ignore rule treated it as literal (so the+ * bytes landed in ONE file at the vault ROOT, named `Projects\evil.md`).+ *+ * Picking a reading is the tempting fix and the wrong one. Normalising+ * `\` to `/` silently reinterprets the caller's string into a different+ * path than the one it sent, and treating it as literal leaves a+ * boundary that a caller can talk past. This repository forbids the+ * quiet reinterpretation, so the path is refused and the caller is told+ * which character made it ambiguous. A caller that meant a subdirectory+ * re-sends it with `/`; a caller that genuinely meant a backslash in a+ * filename has asked for something no surface here can report+ * faithfully.+ *+ * On Windows this never fires: `\` IS the separator there, and+ * `inspectPath` converts it to POSIX before anything downstream sees it.+ */+function assertHostPathSeparators(path: string): void {+ if (WINDOWS_SEP === sep) return;+ if (!path.includes(WINDOWS_SEP)) return;+ throw new CreateNoteError(+ "invalid_path",+ `note path must use '${posix.sep}' to separate segments; on this host ` ++ `'${WINDOWS_SEP}' is an ordinary filename character, so the path would not ` ++ `mean what it looks like: ${path}`,+ );+}++/**+ * Translate a Brain config failure into this surface's typed refusal.+ *+ * The raw {@link BrainConfigError} names the config by its ABSOLUTE host+ * path, which is the one thing no other refusal on this surface leaks+ * and which a connected agent cannot act on. The re-raised error keeps+ * the validator's own diagnosis, renames the source to its vault-+ * relative spelling, and appends the registered exit, so a caller that+ * hits it learns what to run instead of receiving a bare fault.+ */+function configFault(err: BrainConfigError): CreateNoteError {+ const source = err.source;+ const prefix = source === null ? null : `${source}: `;+ const detail =+ prefix !== null && err.message.startsWith(prefix)+ ? err.message.slice(prefix.length)+ : err.message;+ return new CreateNoteError(+ "config_invalid",+ `${BRAIN_CONFIG_REL}: ${detail}; ${CONFIG_INVALID_EXIT}`,+ );+}++/**+ * Run `read`, translating a Brain config failure into {@link configFault}.+ * Shared by the two config-backed checks in the envelope — the vault+ * scope and the write binding — so a malformed config reads the same on+ * whichever of them reaches it first.+ */+function withConfigFault<T>(read: () => T): T {+ try {+ return read();+ } catch (err) {+ if (err instanceof BrainConfigError) throw configFault(err);+ throw err;+ }+}+/*** Resolve and safety-check a vault-relative note path - the exact* envelope enforced by {@link createNote}: `.md` suffix, vault-relative- * (no absolute path), no `..` traversal, not the Brain machinery root,- * not a vault-scope-excluded location, and inside the vault. Every- * refusal is a typed {@link CreateNoteError}. Extracted so the atomic- * write-batch core (kernel 2) reuses the same envelope for update and- * append operations rather than re-deriving it.+ * (no absolute path), separators this host reads the way the caller+ * wrote them, no `..` traversal, not the Brain machinery root, not a+ * vault-scope-excluded location, inside the operator's declared write+ * binding, and inside the vault. Every refusal is a typed+ * {@link CreateNoteError} - including the one case that is not about the+ * path at all, a `Brain/_brain.yaml` that does not load. Extracted so the+ * atomic write-batch core (kernel 2) reuses the same envelope for update+ * and append operations rather than re-deriving it.+ *+ * Order is load-bearing. The separator check runs FIRST, before anything+ * normalises the string, because every guard after it reads segments and+ * they must all be reading the same segments the caller meant.*/export function resolveNoteTarget(vault: string, path: string): ResolvedNoteTarget {if (!path.toLowerCase().endsWith(".md")) {@@ -71,14 +286,17 @@ export function resolveNoteTarget(vault: string, path: string): ResolvedNoteTarg}// The tool addresses notes by a vault-relative path; an absolute path is// ambiguous (which root?) and is refused rather than silently re-rooted.- if (path.startsWith("/") || path.startsWith("\\")) {+ if (path.startsWith(posix.sep) || path.startsWith(WINDOWS_SEP)) {throw new CreateNoteError("invalid_path", `note path must be vault-relative: ${path}`);}+ // Before any normalisation reads the string: refuse a separator this+ // host would not read the way the caller wrote it. See the function.+ assertHostPathSeparators(path);// inspectPath normalises the relative path and throws on `..` traversal;// an absolute path also has no place here. Translate both into a typed// CreateNoteError so callers get one error surface.- const scope = resolveVaultScope(vault);+ const scope = withConfigFault(() => resolveVaultScope(vault));let inspected;try {inspected = inspectPath(path, scope, vault);@@ -93,7 +311,7 @@ export function resolveNoteTarget(vault: string, path: string): ResolvedNoteTarg// The Brain machinery root is owned by the brain's own writers; a// free-form note tool must never author into it (default vault-scope// rules ignore Brain/.snapshots only, not the whole Brain root).- const firstSegment = relPath.split("/")[0];+ const firstSegment = relPath.split(posix.sep)[0];if (firstSegment === BRAIN_ROOT_REL) {throw new CreateNoteError("excluded",@@ -108,6 +326,18 @@ export function resolveNoteTarget(vault: string, path: string): ResolvedNoteTarg);}+ // The operator's declared write binding (provenance-at-the-boundary,+ // unit B). This function is the ONE envelope every caller-named write+ // shares - `createNote` and the batch core's create / update / append+ // arms all resolve their target here - so the boundary is checked+ // exactly once, before any directory is created or byte written. The+ // refusal is composed in the binding module, never here; absent+ // declaration returns null and this whole branch is inert.+ const refusal = withConfigFault(() => checkWriteBinding(vault, relPath));+ if (refusal !== null) {+ throw new CreateNoteError("write_binding", refusal.message);+ }+let abs: string;try {abs = ensureInsideVault(join(vault, relPath), vault);@@ -118,8 +348,85 @@ export function resolveNoteTarget(vault: string, path: string): ResolvedNoteTarg}/**- * Create one Markdown note in the vault. Returns the created note's- * vault-relative path; throws {@link CreateNoteError} on any refusal.+ * The one no-op result. Built in a single place because it is reached+ * from two arms - the pre-write existence check and the exclusive-link+ * refusal that closes the race behind it - and the two must not be able+ * to drift into reporting a skip differently.+ *+ * `relPath` is reported verbatim. It used to be re-split on `[\\/]` and+ * re-joined with `/`, which was the only place the reported path could+ * differ from the path that holds the bytes: on POSIX that rewrote a+ * file literally named `Projects\evil.md` at the vault root into the+ * report `Projects/evil.md`. Such a path is now refused at the envelope,+ * and {@link ResolvedNoteTarget.relPath} is already POSIX, so there is+ * nothing left to convert.+ */+function skippedResult(relPath: string): SkippedNoteResult {+ return { path: relPath, outcome: "skipped", created: false };+}++/**+ * Resolve the body to write. Exactly one of `content` and `template` may+ * be supplied; providing both, or variables with no template, is a+ * caller fault named rather than resolved by letting one win.+ */+function resolveBody(input: CreateNoteInput): string {+ if (input.template === undefined) {+ if (input.templateVariables !== undefined) {+ throw new CreateNoteError(+ "invalid_template",+ "templateVariables requires 'template'; supply a template or drop the variables",+ );+ }+ return input.content ?? "";+ }+ if (input.content !== undefined) {+ throw new CreateNoteError(+ "invalid_template",+ "provide either 'content' or 'template', not both",+ );+ }+ try {+ return renderNoteTemplate(input.template, input.templateVariables ?? {});+ } catch (err) {+ if (err instanceof NoteTemplateError) {+ throw new CreateNoteError("invalid_template", `${err.code}: ${err.message}`);+ }+ throw err;+ }+}++/**+ * Run the shared write-session artifact validator over the document that+ * would be written. The schema type is whatever the frontmatter itself+ * declares, checked against the vault's `page_types` vocabulary - the+ * same binding `validateArtifactForSession` uses, so `strict` here and a+ * write session there judge a document identically.+ *+ * Note this is the CLI's `--strict` in name only: there it escalates+ * warnings into an exit code, here it is a pre-write gate. The two never+ * meet, because no command-line verb reaches this primitive.+ */+function assertValidDocument(vault: string, frontmatter: FrontmatterMap, body: string): void {+ const declaredType = frontmatter["type"];+ const schemaType = typeof declaredType === "string" ? declaredType : null;+ const violations = validateArtifact(formatFrontmatter(frontmatter, body), {+ schemaType,+ vocabulary: loadSchemaPack(vault).vocabulary,+ });+ if (violations.length > 0) {+ throw new CreateNoteError(+ "invalid_document",+ `note failed strict validation: ${violations.map((v) => v.code).join(", ")}`,+ violations,+ );+ }+}++/**+ * Create one Markdown note in the vault. Returns a discriminated+ * {@link CreateNoteResult}; throws {@link CreateNoteError} on any+ * refusal.*/export function createNote(vault: string, input: CreateNoteInput): CreateNoteResult {// Vault-identity write guard (context-integrity-gates, Unit J). This@@ -127,10 +434,24 @@ export function createNote(vault: string, input: CreateNoteInput): CreateNoteRes// `brain_update_note` - the headline note writers.assertVaultIdentityForWrite(vault);const { relPath, abs } = resolveNoteTarget(vault, input.path);+ const frontmatter = input.frontmatter ?? {};+ const body = resolveBody(input);+ // Validation judges the INPUT, so it runs before the target is+ // consulted: a caller must not learn that its document is invalid+ // only on the runs where the path happened to be free.+ if (input.strict === true) assertValidDocument(vault, frontmatter, body);++ const skipOccupied = input.ifExists === "skip";+ // Return before mkdirSync so a no-op leaves no parent directories+ // behind. The authoritative decision is still the exclusive-link+ // refusal below, which closes the residual TOCTOU race race-free.+ if (skipOccupied && existsSync(abs)) {+ return skippedResult(relPath);+ }mkdirSync(dirname(abs), { recursive: true });try {- writeFrontmatterAtomic(abs, input.frontmatter ?? {}, input.content ?? "", {+ writeFrontmatterAtomic(abs, frontmatter, body, {overwrite: false,existsErrorKind: "note",vaultForRelativePath: vault,@@ -140,10 +461,13 @@ export function createNote(vault: string, input: CreateNoteInput): CreateNoteResconst exists =code === "EEXIST" || (err instanceof Error && /already exists/.test(err.message));if (exists) {+ if (skipOccupied) {+ return skippedResult(relPath);+ }throw new CreateNoteError("exists", `note already exists: ${relPath}`);}throw err;}- return { path: relPath.split(/[\\/]/).join(posix.sep), created: true };+ return { path: relPath, outcome: "created", created: true };}diff --git a/src/core/brain/notes/note-template.ts b/src/core/brain/notes/note-template.tsnew file mode 100644index 00000000..493c1ef2--- /dev/null+++ b/src/core/brain/notes/note-template.ts@@ -0,0 +1,299 @@+/**+ * Note-body templates: a CLOSED two-construct grammar+ * (provenance-at-the-boundary, Unit C).+ *+ * `createNote` can author a body from a caller-supplied template plus a+ * typed variable map instead of a literal string. The grammar has+ * exactly two constructs and is not going to grow a third:+ *+ * 1. Typed variable substitution. `{{name}}` renders the variable's+ * canonical text: a string as-is, a number through `String`, a+ * boolean as `true`/`false`, a list joined by {@link LIST_JOIN}.+ * 2. Presence-driven sections and list iteration.+ * `{{#name}}…{{/name}}` renders its body when `name` is present and+ * non-empty, and once per entry when `name` is a list, where+ * `{{.}}` is the current entry. Sections nest.+ *+ * There is no expression language, no comparison, no filter, no+ * partial, no inversion and no dotted path. Anything needing a third+ * construct is out of scope by design: a general template engine is not+ * something this kernel wants to own, and every one of them starts as+ * two constructs.+ *+ * Two failure modes, kept deliberately distinct because collapsing them+ * is exactly the silent-fallback defect this project removes:+ *+ * - An UNKNOWN name is left byte-intact. `{{ typo }}` renders as+ * `{{ typo }}`, and an unknown section keeps its whole span, body+ * included. This is the rule `renderTemplate` in `../templates.ts`+ * already documents for managed templates - a typo must surface in+ * the output rather than vanish into an empty string.+ * - A MALFORMED template - a section opened and never closed - is+ * refused with a typed {@link NoteTemplateError}, because there is+ * no rendering of it to return. "Not a construct" and "a construct+ * that does not parse" are different answers and get different+ * treatment.+ *+ * Pure and clock-free: the same template and variables always render to+ * the same bytes.+ */++import type { FrontmatterValue } from "../../types.ts";++/**+ * Variables a template may reference. Deliberately the vault's own+ * frontmatter value domain, so the MCP surface coerces template+ * variables through the same parser it already uses for frontmatter+ * instead of growing a second value model.+ */+export type NoteTemplateVariables = Readonly<Record<string, FrontmatterValue>>;++/** Machine-readable reason a template could not be rendered. */+export type NoteTemplateErrorCode = "unbalanced_section" | "section_too_deep" | "invalid_variable";++export class NoteTemplateError extends Error {+ readonly code: NoteTemplateErrorCode;+ constructor(code: NoteTemplateErrorCode, message: string) {+ super(message);+ this.name = "NoteTemplateError";+ this.code = code;+ }+}++/** Construct delimiters. */+const OPEN = "{{";+const CLOSE = "}}";++/** Sigils that turn a delimited name into a section open or close. */+const SECTION_OPEN_SIGIL = "#";+const SECTION_CLOSE_SIGIL = "/";++/** The current entry inside a list iteration. */+const ITEM_NAME = ".";++/**+ * Legal variable and section names. Anything else between the+ * delimiters is not a construct at all and survives as literal text,+ * which is what keeps `{{a.b}}`, `{{a | upper}}` and `{{^a}}` from+ * being a path lookup, a filter and an inverted section.+ */+const NAME_PATTERN = /^[A-Za-z0-9_][A-Za-z0-9_-]*$/;++/** Canonical separator when a list renders as one value. */+const LIST_JOIN = ", ";++/**+ * Deepest section nesting accepted. Templates arrive from an MCP+ * caller, and the parser is recursive; a bound keeps a pathological+ * template a typed refusal instead of a stack overflow.+ */+const MAX_SECTION_DEPTH = 16;++type TemplateNode =+ | { readonly kind: "text"; readonly text: string }+ | { readonly kind: "variable"; readonly name: string; readonly raw: string }+ | {+ readonly kind: "section";+ readonly name: string;+ readonly raw: string;+ readonly body: ReadonlyArray<TemplateNode>;+ };++interface ParsedSpan {+ readonly nodes: ReadonlyArray<TemplateNode>;+ /** Index just past the matching close tag, or the end of the source. */+ readonly next: number;+ /** True when a matching close tag was consumed. */+ readonly closed: boolean;+}++function isName(candidate: string): boolean {+ return NAME_PATTERN.test(candidate);+}++/** A list value, narrowed so the scalar branch narrows too. */+function isList(value: FrontmatterValue): value is ReadonlyArray<string> {+ return Array.isArray(value);+}++/**+ * Own-property read. `variables` may be a prototype-free object built by+ * the MCP argument parser, so `in` and direct indexing are both wrong+ * here for different reasons.+ */+function readVariable(+ variables: NoteTemplateVariables,+ name: string,+): FrontmatterValue | undefined {+ return Object.prototype.hasOwnProperty.call(variables, name) ? variables[name] : undefined;+}++/**+ * Canonical text for one variable. Rendering is total for every value+ * in the domain except a non-finite number, which is a caller fault and+ * is named rather than rendered as `NaN`.+ */+function canonical(value: FrontmatterValue, name: string): string {+ if (isList(value)) return value.join(LIST_JOIN);+ if (typeof value === "string") return value;+ if (typeof value === "boolean") return value ? "true" : "false";+ if (!Number.isFinite(value)) {+ throw new NoteTemplateError(+ "invalid_variable",+ `template variable '${name}' is not a finite number`,+ );+ }+ return String(value);+}++/**+ * Presence, NOT JavaScript truthiness. A string is present when it is+ * non-empty and a boolean when it is `true`; a number is ALWAYS present,+ * because `0` is a value the caller supplied and reading it as absence+ * is the classic falsy-zero defect. Lists are handled by the iteration+ * branch before this is reached.+ */+function isPresent(value: string | number | boolean): boolean {+ if (typeof value === "string") return value !== "";+ if (typeof value === "boolean") return value;+ return true;+}++/**+ * Parse `source` from `start` until either the close tag for+ * `sectionName` or the end of input. `sectionName` is null at the top+ * level, where a close tag has nothing to match and is therefore not a+ * construct.+ */+function parseSpan(+ source: string,+ start: number,+ sectionName: string | null,+ depth: number,+): ParsedSpan {+ const nodes: TemplateNode[] = [];+ let cursor = start;+ let literalStart = start;+ const pushText = (end: number): void => {+ if (end > literalStart) nodes.push({ kind: "text", text: source.slice(literalStart, end) });+ };++ while (cursor < source.length) {+ const open = source.indexOf(OPEN, cursor);+ if (open < 0) break;+ const close = source.indexOf(CLOSE, open + OPEN.length);+ // An unterminated `{{` is not a construct; the remainder is text.+ if (close < 0) break;+ const after = close + CLOSE.length;+ const inner = source.slice(open + OPEN.length, close).trim();++ if (inner.startsWith(SECTION_CLOSE_SIGIL)) {+ const name = inner.slice(SECTION_CLOSE_SIGIL.length).trim();+ if (isName(name) && name === sectionName) {+ pushText(open);+ return { nodes, next: after, closed: true };+ }+ // A close tag that matches no open section is an unknown token,+ // not a broken construct: leave it intact so it surfaces.+ cursor = after;+ continue;+ }++ if (inner.startsWith(SECTION_OPEN_SIGIL)) {+ const name = inner.slice(SECTION_OPEN_SIGIL.length).trim();+ if (!isName(name)) {+ cursor = after;+ continue;+ }+ if (depth >= MAX_SECTION_DEPTH) {+ throw new NoteTemplateError(+ "section_too_deep",+ `template nests sections deeper than ${MAX_SECTION_DEPTH}`,+ );+ }+ const body = parseSpan(source, after, name, depth + 1);+ if (!body.closed) {+ throw new NoteTemplateError(+ "unbalanced_section",+ `template opens section '${name}' and never closes it`,+ );+ }+ pushText(open);+ nodes.push({+ kind: "section",+ name,+ raw: source.slice(open, body.next),+ body: body.nodes,+ });+ cursor = body.next;+ literalStart = cursor;+ continue;+ }++ if (inner === ITEM_NAME || isName(inner)) {+ pushText(open);+ nodes.push({ kind: "variable", name: inner, raw: source.slice(open, after) });+ cursor = after;+ literalStart = cursor;+ continue;+ }++ // Not one of the two constructs: the delimiters stay literal text.+ cursor = after;+ }++ pushText(source.length);+ return { nodes, next: source.length, closed: false };+}++/**+ * Render a parsed span. `item` is the current list entry inside an+ * iteration and `undefined` everywhere else, which is what makes+ * `{{.}}` outside an iteration an unknown placeholder rather than an+ * empty one.+ */+function renderNodes(+ nodes: ReadonlyArray<TemplateNode>,+ variables: NoteTemplateVariables,+ item: FrontmatterValue | undefined,+): string {+ let out = "";+ for (const node of nodes) {+ if (node.kind === "text") {+ out += node.text;+ continue;+ }+ if (node.kind === "variable") {+ const value = node.name === ITEM_NAME ? item : readVariable(variables, node.name);+ out += value === undefined ? node.raw : canonical(value, node.name);+ continue;+ }+ const value = readVariable(variables, node.name);+ if (value === undefined) {+ // An undeclared section name is an unknown placeholder, not an+ // empty one: the whole construct survives so the typo surfaces.+ out += node.raw;+ continue;+ }+ if (isList(value)) {+ for (const entry of value) out += renderNodes(node.body, variables, entry);+ continue;+ }+ if (isPresent(value)) out += renderNodes(node.body, variables, item);+ }+ return out;+}++/**+ * Render `template` against `variables`. Throws+ * {@link NoteTemplateError} only for a template that cannot be parsed+ * or a variable outside the value domain; an unknown name is never an+ * error, it is preserved.+ */+export function renderNoteTemplate(+ template: string,+ variables: NoteTemplateVariables = {},+): string {+ const parsed = parseSpan(template, 0, null, 0);+ return renderNodes(parsed.nodes, variables, undefined);+}diff --git a/src/core/brain/policy/blocks/write-binding.ts b/src/core/brain/policy/blocks/write-binding.tsnew file mode 100644index 00000000..09f75f1f--- /dev/null+++ b/src/core/brain/policy/blocks/write-binding.ts@@ -0,0 +1,113 @@+/**+ * The `write_binding:` block — where a CALLER-NAMED write may land.+ *+ * One reason to change: what an operator may declare as a write+ * boundary, and which shapes of declaration are refused outright.+ *+ * Shape problems are hard errors rather than dropped entries, for the+ * same reason `vault.ignore_paths` treats them that way: an entry that+ * silently disabled itself would widen a boundary the operator believes+ * is in force, which is worse than declaring no boundary at all.+ *+ * The block is OPTIONAL and its absence is the historical behaviour. So+ * is a block present without `path_prefixes` — the binding is made of+ * that key, and a block that omits it has declared nothing, exactly as+ * `vault:` without `ignore_paths` falls back to the defaults.+ *+ * An EMPTY list is the one shape refused rather than honoured. A prefix+ * list that admits no path is not a boundary, it is an off switch on the+ * caller-named write surface; the way to run unbound is to remove the+ * block, and conflating the two would leave an operator who mistyped a+ * list unable to tell a disabled surface from a bounded one.+ */++import type { BrainWriteBindingConfig } from "../../types.ts";+import { BrainConfigError } from "../errors.ts";+import { assertNoYamlUnsafeChars, describe, requireArrayField } from "../field-checks.ts";+import { openBlock, warnUnknownKeys, type BlockParseContext } from "../key-index.ts";+import { normaliseWriteBindingPrefix } from "../../../write-binding/prefix.ts";++const BLOCK = "write_binding";+const PREFIXES_KEY = "path_prefixes";+const PREFIXES_FIELD = `${BLOCK}.${PREFIXES_KEY}`;++/**+ * Shape:+ * write_binding:+ * path_prefixes:+ * - Projects+ * - Journal/Weekly+ */+export function parseWriteBindingBlock(+ ctx: BlockParseContext,+): BrainWriteBindingConfig | undefined {+ const rawMap = openBlock(ctx, BLOCK);+ if (rawMap === undefined) return undefined;++ let block: BrainWriteBindingConfig | undefined;+ if (PREFIXES_KEY in rawMap) {+ const list = requireArrayField(+ rawMap[PREFIXES_KEY],+ PREFIXES_FIELD,+ ctx.source,+ `must be a list of vault-relative paths; got ${describe(rawMap[PREFIXES_KEY])}`,+ );+ if (list.length === 0) {+ throw new BrainConfigError(+ "must declare at least one prefix; a list that admits no path is an off switch " ++ `on caller-named writes rather than a boundary. Remove the '${BLOCK}' block to ` ++ "run unbound.",+ PREFIXES_FIELD,+ ctx.source,+ );+ }+ const validated = list.map((entry, i) =>+ cleanPrefix(entry, `${PREFIXES_FIELD}[${i}]`, ctx.source),+ );+ block = { path_prefixes: Object.freeze(validated) };+ }+ warnUnknownKeys(ctx, rawMap, [PREFIXES_KEY], BLOCK);+ return block;+}++/**+ * One declared prefix, validated and normalised to the vault-relative+ * POSIX form the matcher compares against. Every refusal names the+ * indexed field, so the operator sees which entry is wrong rather than+ * that some entry is.+ */+function cleanPrefix(entry: unknown, field: string, source: string | null): string {+ if (typeof entry !== "string") {+ throw new BrainConfigError(`must be a string; got ${describe(entry)}`, field, source);+ }+ const trimmed = entry.trim();+ if (trimmed.length === 0) {+ throw new BrainConfigError("must be a non-empty string", field, source);+ }+ assertNoYamlUnsafeChars(trimmed, field, source, "use a simple one-line path");+ if (trimmed.startsWith("/") || trimmed.startsWith("\\")) {+ throw new BrainConfigError(+ "must be a vault-relative path without a leading slash",+ field,+ source,+ );+ }+ if (trimmed.split(/[\\/]/).some((segment) => segment === "..")) {+ throw new BrainConfigError(+ "must not contain '..' segments; a prefix that can traverse out of the vault bounds " ++ "nothing",+ field,+ source,+ );+ }+ const normalised = normaliseWriteBindingPrefix(trimmed);+ if (normalised.length === 0) {+ throw new BrainConfigError(+ "normalises to the empty string, which would admit the whole vault; name a real " ++ "folder or vault-relative path",+ field,+ source,+ );+ }+ return normalised;+}diff --git a/src/core/brain/policy/validate.ts b/src/core/brain/policy/validate.tsindex 52dd8738..d91a33da 100644--- a/src/core/brain/policy/validate.ts+++ b/src/core/brain/policy/validate.ts@@ -34,6 +34,7 @@ import { parseTemporalBlock } from "./blocks/temporal.ts";import { parseHealthBlock } from "./blocks/health.ts";import { parseIntegrityBlock } from "./blocks/integrity.ts";import { parseNotesBlock } from "./blocks/notes.ts";+import { parseWriteBindingBlock } from "./blocks/write-binding.ts";import { parseSessionsBlock } from "./blocks/sessions.ts";import { parseSchemaBlock } from "./blocks/schema.ts";import {@@ -108,6 +109,7 @@ export function validateBrainConfigDetailed(const health = parseHealthBlock(ctx);const integrity = parseIntegrityBlock(ctx);const notes = parseNotesBlock(ctx);+ const writeBinding = parseWriteBindingBlock(ctx);const sessions = parseSessionsBlock(ctx);const schema = parseSchemaBlock(ctx);const hygiene = parseHygieneBlock(ctx);@@ -133,6 +135,7 @@ export function validateBrainConfigDetailed(...(linkGraph !== undefined ? { link_graph: linkGraph } : {}),...(temporal !== undefined ? { temporal } : {}),...(notes !== undefined ? { notes } : {}),+ ...(writeBinding !== undefined ? { write_binding: writeBinding } : {}),...(sessions !== undefined ? { sessions } : {}),...(health !== undefined ? { health } : {}),...(integrity !== undefined ? { integrity } : {}),diff --git a/src/core/brain/schema-mutate.ts b/src/core/brain/schema-mutate.tsindex ff2d4815..f68ca19d 100644--- a/src/core/brain/schema-mutate.ts+++ b/src/core/brain/schema-mutate.ts@@ -7,6 +7,7 @@ import { withFileLock } from "../reliability/lock.ts";import { brainConfigPath, brainDirsForWrite } from "./paths.ts";import { assertVaultIdentityForWrite } from "./vault-identity.ts";import {+ loadSchemaPack,parseSchemaPack,renderSchemaBlock,replaceSchemaBlock,@@ -109,6 +110,51 @@ export interface ApplySchemaMutationsResult {readonly pack: SchemaPack;}+/**+ * One leaf-level difference between two schema packs.+ *+ * `before === null` means the leaf would appear, `after === null` means it+ * would disappear, and two non-null values mean a scalar leaf's value would+ * change. A `path` is the dotted address of the leaf inside the pack+ * (`declarations.preference_types`, `prefixes.pref`, `attributes.paper.doi`);+ * list-valued paths report one entry per member, so a rename shows as a+ * removal and an addition under the same path rather than as a mutation of+ * an anonymous list.+ */+export interface SchemaPackDiffEntry {+ readonly path: string;+ readonly before: string | null;+ readonly after: string | null;+}++/**+ * The result of {@link previewSchemaMutations}. Deliberately NOT the shape of+ * {@link ApplySchemaMutationsResult}: a preview has no `audit_path` because it+ * wrote no audit record, and it reports `would_apply` rather than `applied`+ * because nothing was applied. A caller can therefore never mistake a dry run+ * for a real mutation.+ */+export interface PreviewSchemaMutationsResult {+ readonly dry_run: true;+ /** How many mutations were submitted; see `diff` for what would change. */+ readonly would_apply: number;+ readonly pack: SchemaPack;+ readonly diff: ReadonlyArray<SchemaPackDiffEntry>;+}++/**+ * Separates a leaf's path from its value when keying a list MEMBER, so two+ * members of the same list do not collide in the flat map the diff walks. A+ * NUL byte cannot occur in a schema token, an endpoint pair, or a rendered+ * single-line description, so the key is unambiguous.+ */+const LEAF_KEY_SEPARATOR = "\u0000";++interface PackLeaf {+ readonly path: string;+ readonly value: string;+}+interface MutableSchemaPack {declarations: Record<SchemaVocabularyCategory, string[]>;aliases: Record<string, string[]>;@@ -177,6 +223,136 @@ export function applyMutationsToPack(return next;}+/**+ * Build the pack {@link applySchemaMutations} would persist for this batch,+ * WITHOUT touching disk, together with its difference from the pack currently+ * on disk.+ *+ * The preview runs the same {@link applyMutationsToPack} the write path runs,+ * so a batch the validator rejects raises here exactly what it would raise+ * there, and a batch it accepts yields exactly the pack that would land. A+ * preview that were more permissive than the apply it previews would be worse+ * than no preview at all.+ *+ * Two deliberate asymmetries with the write path, both because nothing is+ * written: no file lock is taken (a lock is a file, and a preview creates no+ * files), and the vault-identity WRITE guard is not run - its job is to refuse+ * a write to a store this process did not open, and running it here would pin+ * this process's identity as a side effect of a read.+ */+export function previewSchemaMutations(+ vault: string,+ mutations: ReadonlyArray<SchemaMutation>,+): PreviewSchemaMutationsResult {+ const current = loadSchemaPack(vault);+ const next = applyMutationsToPack(current, mutations);+ return {+ dry_run: true,+ would_apply: mutations.length,+ pack: next,+ diff: diffSchemaPacks(current, next),+ };+}++/**+ * Leaf-level difference between two schema packs, ordered by path so the same+ * two packs always produce the same list.+ *+ * `vocabulary` is excluded: it is the closure of `declarations` over the+ * built-in vocabulary, so every difference in it is already reported under+ * `declarations` and listing both would double-count one change.+ */+export function diffSchemaPacks(+ before: SchemaPack,+ after: SchemaPack,+): ReadonlyArray<SchemaPackDiffEntry> {+ const left = flattenPack(before);+ const right = flattenPack(after);+ const entries: SchemaPackDiffEntry[] = [];+ for (const [key, leaf] of left) {+ const next = right.get(key);+ if (next === undefined) {+ entries.push({ path: leaf.path, before: leaf.value, after: null });+ } else if (next.value !== leaf.value) {+ entries.push({ path: leaf.path, before: leaf.value, after: next.value });+ }+ }+ for (const [key, leaf] of right) {+ if (!left.has(key)) entries.push({ path: leaf.path, before: null, after: leaf.value });+ }+ // Ordered by path, then by the value each entry is about - so a rename+ // under one path reports the departing member before the arriving one.+ return entries.toSorted(+ (a, b) => a.path.localeCompare(b.path) || diffEntryValue(a).localeCompare(diffEntryValue(b)),+ );+}++/** The value a diff entry is about: what left, or - for an addition - what arrived. */+function diffEntryValue(entry: SchemaPackDiffEntry): string {+ return entry.before ?? entry.after ?? "";+}++/** Flatten every authored leaf of a pack to `key -> {path, value}`. */+function flattenPack(pack: SchemaPack): Map<string, PackLeaf> {+ const leaves = new Map<string, PackLeaf>();+ for (const category of SCHEMA_VOCAB_CATEGORIES) {+ addMembers(leaves, `declarations.${category}`, pack.declarations[category] ?? []);+ }+ addRecordMembers(leaves, "aliases", pack.aliases);+ addScalars(leaves, "prefixes", pack.prefixes);+ addMembers(leaves, "link_types", pack.link_types);+ addMembers(leaves, "extractable", pack.extractable);+ addScalars(leaves, "expert_routing", pack.expert_routing);+ addRecordMembers(leaves, "labels", pack.labels);+ addRecordMembers(leaves, "link_constraints", pack.link_constraints);+ addNestedScalars(leaves, "attributes", pack.attributes);+ addNestedScalars(leaves, "frontmatter_tiers", pack.frontmatter_tiers);+ return leaves;+}++/**+ * List members: the value is part of the key, so membership changes read as+ * an addition or a removal and never as an in-place edit of a list slot.+ */+function addMembers(+ leaves: Map<string, PackLeaf>,+ path: string,+ values: ReadonlyArray<string>,+): void {+ for (const value of values) {+ leaves.set(`${path}${LEAF_KEY_SEPARATOR}${value}`, { path, value });+ }+}++/** Scalar map: the key is the leaf path, so a changed value reads as an edit. */+function addScalars(+ leaves: Map<string, PackLeaf>,+ prefix: string,+ record: Readonly<Record<string, string>>,+): void {+ for (const [key, value] of Object.entries(record)) {+ const path = `${prefix}.${key}`;+ leaves.set(path, { path, value });+ }+}++function addRecordMembers(+ leaves: Map<string, PackLeaf>,+ prefix: string,+ record: Readonly<Record<string, ReadonlyArray<string>>>,+): void {+ for (const [key, values] of Object.entries(record))+ addMembers(leaves, `${prefix}.${key}`, values);+}++function addNestedScalars(+ leaves: Map<string, PackLeaf>,+ prefix: string,+ record: Readonly<Record<string, Readonly<Record<string, string>>>>,+): void {+ for (const [key, inner] of Object.entries(record)) addScalars(leaves, `${prefix}.${key}`, inner);+}+function applyOne(pack: MutableSchemaPack, mutation: SchemaMutation): void {switch (mutation.op) {case "add_type": {diff --git a/src/core/brain/sessions/import.ts b/src/core/brain/sessions/import.tsindex 1ceb8f7c..fdc1602b 100644--- a/src/core/brain/sessions/import.ts+++ b/src/core/brain/sessions/import.ts@@ -28,6 +28,7 @@import { existsSync, lstatSync, readdirSync, readFileSync, statSync } from "node:fs";import { basename, join, resolve } from "node:path";+import { readSkillOfferId, SKILL_OFFER_ID_KEY } from "../../surface/skill-offer.ts";import { buildDedupIndex, computeDedupHash, type DedupIndexEntry } from "../dedup-hash.ts";import { appendContinuityRecord } from "../continuity/store.ts";import { discoverMarkersDetailed, isFeedbackMarker } from "../inline.ts";@@ -420,6 +421,12 @@ export async function importSession(// double-counts). Skipped on a dry run.const invokedSkill = resolveInvokedSkill(call);if (invokedSkill !== null && opts.dryRun !== true) {+ // Offer provenance (t_ccb05134): when the call cites the offer that+ // produced it, the id rides onto the record so the invocation can be+ // joined back to that offer. A call citing none is stamped exactly as+ // before - the key is omitted, never defaulted, so "no offer was+ // cited" can never be read as "offer unknown-but-present".+ const citedOffer = readSkillOfferId(call.input);try {appendContinuityRecord(vault, {kind: "skill_invoked",@@ -430,6 +437,7 @@ export async function importSession(agent: agentLabelForTurn(turn, adapter.id, opts.agent),tool: call.name,runtime: adapter.id,+ ...(citedOffer !== null ? { [SKILL_OFFER_ID_KEY]: citedOffer } : {}),},});skillInvocations++;diff --git a/src/core/brain/skill-usage.ts b/src/core/brain/skill-usage.tsindex 5aa8f6c4..316259a8 100644--- a/src/core/brain/skill-usage.ts+++ b/src/core/brain/skill-usage.ts@@ -1,5 +1,5 @@/**- * Per-skill invocation telemetry (t_56a12bde).+ * Per-skill invocation telemetry (t_56a12bde) and the offer join (t_ccb05134).** How often each installed skill is actually invoked by an agent runtime is a* usage signal distinct from proposing, ranking, or verifying skills. Each@@ -9,54 +9,117 @@* mutated counter - mirroring the working-memory usage-signal shape, so the* dream / synthesis pass can rank, retain, or retire skills on real evidence.* Deterministic; no LLM.+ *+ * An invocation that cites the offer it came from carries that offer's id+ * (see `../surface/skill-offer.ts`), which closes the link the efficacy chain+ * was missing: {@link joinSkillInvocationsToOffers} groups invocations under+ * the offer that produced them. An invocation citing no offer is reported+ * separately and is never attributed to one by proximity or by guesswork -+ * "we do not know which offer this came from" is a different answer from+ * "this came from that offer", and the two are not collapsed.*/import { loadNormalizedContinuityRecords } from "./continuity/read-model.ts";import { decayWeight } from "./continuity/usage-signal.ts";+import { readSkillOfferId } from "../surface/skill-offer.ts";export interface SkillUsage {readonly skill: string;/** How many times the skill was invoked across all captured runtimes. */readonly invocationCount: number;+ /**+ * How many of those invocations cited the offer they came from. The+ * remainder are not unattributed-by-error: most runtimes' skill calls are+ * not preceded by an offer at all.+ */+ readonly offerAttributedCount: number;/** Most recent invocation time, epoch ms, or null when never dated. */readonly lastInvokedAtMs: number | null;/** Recency/frequency decay weight in (0, 1], for ranking. */readonly weight: number;}+/** One `skill_invoked` record, normalised to the fields the join needs. */+export interface SkillInvocation {+ /** Content-addressed continuity record id. */+ readonly recordId: string;+ readonly skill: string;+ /** Offer this invocation cited, or null when it cited none. */+ readonly offerId: string | null;+ readonly invokedAtMs: number | null;+}++export interface SkillOfferJoin {+ readonly offerId: string;+ /** Invocations citing this offer, in record order. */+ readonly invocations: ReadonlyArray<SkillInvocation>;+}++export interface SkillOfferJoinReport {+ /** Offers with at least one invocation, by offer id ascending. */+ readonly offers: ReadonlyArray<SkillOfferJoin>;+ /** Invocations citing no offer. Never folded into an offer above. */+ readonly unattributed: ReadonlyArray<SkillInvocation>;+}+interface MutableUsage {count: number;+ attributed: number;firstAtMs: number | null;lastAtMs: number | null;}/**- * Derive per-skill invocation counts from `skill_invoked` continuity records.- * Ranked by count descending, then skill name ascending for stable output.+ * Every `skill_invoked` record in the vault, deduplicated and normalised.+ *+ * Continuity logs are append-only with no write-time dedup, so a re-imported+ * session re-appends byte-identical records. Their content-addressed id+ * (kind + createdAt + sourceRefs + payload) is stable, so keying on DISTINCT+ * ids makes every derivation below idempotent across re-imports. A record+ * carrying no usable skill name is not an invocation and is skipped.*/-export function deriveSkillUsage(vault: string, opts: { nowMs?: number } = {}): SkillUsage[] {+function readSkillInvocations(vault: string): SkillInvocation[] {const records = loadNormalizedContinuityRecords(vault, { kind: "skill_invoked" });- const bySkill = new Map<string, MutableUsage>();- // Continuity logs are append-only with no write-time dedup, so a re-imported- // session re-appends byte-identical records. Their content-addressed id- // (kind + createdAt + sourceRefs + payload) is stable, so counting DISTINCT- // ids makes the derivation idempotent across re-imports.const seen = new Set<string>();-+ const out: SkillInvocation[] = [];for (const record of records) {if (seen.has(record.id)) continue;seen.add(record.id);const skill = typeof record.payload["skill"] === "string" ? record.payload["skill"] : null;if (skill === null || skill.length === 0) continue;const atMs = Date.parse(record.createdAt);- const dated = Number.isFinite(atMs) ? atMs : null;- const entry = bySkill.get(skill) ?? { count: 0, firstAtMs: null, lastAtMs: null };+ out.push({+ recordId: record.id,+ skill,+ offerId: readSkillOfferId(record.payload),+ invokedAtMs: Number.isFinite(atMs) ? atMs : null,+ });+ }+ return out;+}++/**+ * Derive per-skill invocation counts from `skill_invoked` continuity records.+ * Ranked by count descending, then skill name ascending for stable output.+ */+export function deriveSkillUsage(vault: string, opts: { nowMs?: number } = {}): SkillUsage[] {+ const bySkill = new Map<string, MutableUsage>();++ for (const invocation of readSkillInvocations(vault)) {+ const entry = bySkill.get(invocation.skill) ?? {+ count: 0,+ attributed: 0,+ firstAtMs: null,+ lastAtMs: null,+ };entry.count += 1;+ if (invocation.offerId !== null) entry.attributed += 1;+ const dated = invocation.invokedAtMs;if (dated !== null) {entry.firstAtMs = entry.firstAtMs === null ? dated : Math.min(entry.firstAtMs, dated);entry.lastAtMs = entry.lastAtMs === null ? dated : Math.max(entry.lastAtMs, dated);}- bySkill.set(skill, entry);+ bySkill.set(invocation.skill, entry);}const nowMs = opts.nowMs ?? Date.now();@@ -75,6 +138,7 @@ export function deriveSkillUsage(vault: string, opts: { nowMs?: number } = {}):usage.push({skill,invocationCount: entry.count,+ offerAttributedCount: entry.attributed,lastInvokedAtMs: entry.lastAtMs,weight,});@@ -83,3 +147,39 @@ export function deriveSkillUsage(vault: string, opts: { nowMs?: number } = {}):usage.sort((a, b) => b.invocationCount - a.invocationCount || a.skill.localeCompare(b.skill));return usage;}++/**+ * Join invocations back to the offers that produced them.+ *+ * The join key is the offer id an invocation cited - the acting agent's+ * claim about where the invocation came from, which is checkable (a+ * malformed id was never stamped, and a real offer id can be recomputed from+ * the offer) but is not an authenticated one; nothing in this system has+ * credentials to authenticate it with, and no surface describes it as more+ * than it is.+ */+export function joinSkillInvocationsToOffers(vault: string): SkillOfferJoinReport {+ const byOffer = new Map<string, SkillInvocation[]>();+ const unattributed: SkillInvocation[] = [];++ for (const invocation of readSkillInvocations(vault)) {+ if (invocation.offerId === null) {+ unattributed.push(invocation);+ continue;+ }+ const group = byOffer.get(invocation.offerId);+ if (group === undefined) byOffer.set(invocation.offerId, [invocation]);+ else group.push(invocation);+ }++ const offers = [...byOffer.entries()]+ .map(([offerId, invocations]) =>+ Object.freeze({ offerId, invocations: Object.freeze(invocations) }),+ )+ .toSorted((a, b) => (a.offerId < b.offerId ? -1 : a.offerId > b.offerId ? 1 : 0));++ return Object.freeze({+ offers: Object.freeze(offers),+ unattributed: Object.freeze(unattributed),+ });+}diff --git a/src/core/brain/trust/retrieval-gate.ts b/src/core/brain/trust/retrieval-gate.tsindex 96134738..db1bd277 100644--- a/src/core/brain/trust/retrieval-gate.ts+++ b/src/core/brain/trust/retrieval-gate.ts@@ -12,8 +12,9 @@* - self-approval guardrail state: a preference whose persisted status* is `quarantine` ({@link BRAIN_PREFERENCE_STATUS.quarantine}, written* by the dream self-approval guardrail),- * - untrusted-source provenance: a page flagged with the untrusted-source- * module's own tag ({@link UNTRUSTED_SOURCE_TAG}),+ * - untrusted-source provenance: a page carrying the untrusted-source+ * marker ({@link hasUntrustedSourceMarker}), which the intake boundary+ * stamps when content enters the vault under untrusted provenance,* - entity contamination: a page carrying the contamination module's* marker ({@link ENTITY_CONTAMINATION_FRONTMATTER_KEY}).*@@ -30,7 +31,7 @@ import {} from "../../search/rank-adjust.ts";import type { BrainSearchResult } from "../../search/types.ts";import { BRAIN_PREFERENCE_STATUS } from "../types.ts";-import { UNTRUSTED_SOURCE_TAG } from "../untrusted-source.ts";+import { hasUntrustedSourceMarker } from "./untrusted-provenance.ts";import { hasEntityContaminationMarker } from "../truth/contamination.ts";/** Namespace name kernel 1 uses when attributing the gate's exclusions. */@@ -64,10 +65,6 @@ function statusScalar(meta: Readonly<Record<string, unknown>>): string | null {return typeof raw === "string" ? raw.trim().toLowerCase() : null;}-function truthy(value: unknown): boolean {- return value === true || value === "true";-}-/*** Classify a candidate's frontmatter as quarantined or clean. Pure and* O(1): three controlled-vocabulary key reads. Reasons are returned in a@@ -83,7 +80,7 @@ export function classifyRetrievalTrust(if (statusScalar(meta) === BRAIN_PREFERENCE_STATUS.quarantine) {reasons.push(RETRIEVAL_TRUST_EXCLUSION_REASON.selfApprovalQuarantine);}- if (truthy(meta[UNTRUSTED_SOURCE_TAG])) {+ if (hasUntrustedSourceMarker(meta)) {reasons.push(RETRIEVAL_TRUST_EXCLUSION_REASON.untrustedSourceProvenance);}if (reasons.length === 0) return CLEAN;diff --git a/src/core/brain/trust/untrusted-provenance.ts b/src/core/brain/trust/untrusted-provenance.tsnew file mode 100644index 00000000..75e60e87--- /dev/null+++ b/src/core/brain/trust/untrusted-provenance.ts@@ -0,0 +1,70 @@+/**+ * The untrusted-provenance marker: one spelling, one reader, one writer.+ *+ * The retrieval trust gate has always branched on an `untrusted_source`+ * frontmatter key, and until this unit nothing in the repository wrote it -+ * the only other occurrences were two tests hand-constructing the field. A+ * gate branch reachable only from a key no writer produces is not a gate.+ * This module is the seam that closes it: the intake boundary stamps the+ * marker through {@link untrustedSourceFrontmatter}, and the gate reads it+ * through {@link hasUntrustedSourceMarker}, so the two halves cannot drift+ * apart over the spelling of a token.+ *+ * A deliberate leaf: it imports the delimiter module for the token name and+ * nothing else, so both the trust layer (which reads) and the entity+ * registry (which writes) can depend on it without an initialisation-order+ * loop.+ */++import type { FrontmatterMap } from "../../types.ts";+import { UNTRUSTED_SOURCE_TAG } from "../untrusted-source.ts";++/**+ * Whether content entering the vault carries the vault's own authority.+ *+ * Structural, and orthogonal to `ProvenanceLevel`: that vocabulary bands how+ * a conclusion was DERIVED (stated / deduced / inferred), which says nothing+ * about where the material came from. A scraped page states its claims just+ * as firmly as the operator states theirs; what differs is who is entitled+ * to have put them in the vault.+ */+export const INTAKE_TRUST = {+ /** Named a location inside the operator's own vault. */+ trusted: "trusted",+ /** Named an address outside it, or no location this vault owns. */+ untrusted: "untrusted",+} as const;++export type IntakeTrust = (typeof INTAKE_TRUST)[keyof typeof INTAKE_TRUST];++/**+ * The frontmatter key marking a page whose content entered under untrusted+ * provenance.+ *+ * Deliberately the same token as the delimiter tag: a page carrying the+ * marker and a span wrapped in `<untrusted_source>` are the same claim about+ * the same material, made at write time and at model-payload time, and the+ * gate already reads this name. Bound to the delimiter constant rather than+ * respelled so the two can never diverge.+ */+export const UNTRUSTED_SOURCE_FRONTMATTER_KEY = UNTRUSTED_SOURCE_TAG;++/**+ * Does this frontmatter declare untrusted provenance? Tolerant of the+ * boolean the writer emits and of the string form a hand edit or a YAML+ * round-trip can leave behind.+ */+export function hasUntrustedSourceMarker(meta: Readonly<Record<string, unknown>>): boolean {+ const value = meta[UNTRUSTED_SOURCE_FRONTMATTER_KEY];+ return value === true || value === "true";+}++/**+ * The frontmatter fragment a writer merges in for the given trust. Trusted+ * content adds NOTHING - an absent marker is the untouched, pre-existing+ * shape, so a trusted write stays byte-identical to one that never knew+ * about trust at all.+ */+export function untrustedSourceFrontmatter(trust: IntakeTrust): FrontmatterMap {+ return trust === INTAKE_TRUST.untrusted ? { [UNTRUSTED_SOURCE_FRONTMATTER_KEY]: true } : {};+}diff --git a/src/core/brain/truth/contamination.ts b/src/core/brain/truth/contamination.tsindex 7057781a..41f5c4e1 100644--- a/src/core/brain/truth/contamination.ts+++ b/src/core/brain/truth/contamination.ts@@ -9,6 +9,7 @@*/import { normalizeEntityName } from "../entities/canonical.ts";+import { ENTITY_STATUS_SCOPE, entityStatusInScope } from "../entities/status-scope.ts";/*** Frontmatter marker key stamped on a page whose synthesized conclusion@@ -73,7 +74,7 @@ export function checkEntityContamination(input: ContaminationInput): Contaminaticonst violations: ContaminationViolation[] = [];for (const entity of input.entities) {- if (entity.status !== "active") continue;+ if (!entityStatusInScope(entity.status, ENTITY_STATUS_SCOPE.canonical)) continue;if (!mentions(conclusion, entity)) continue;const cited = sources.some((source) => mentions(source, entity));if (!cited) violations.push(Object.freeze({ entityId: entity.id, name: entity.name }));diff --git a/src/core/brain/truth/merge-guard.ts b/src/core/brain/truth/merge-guard.tsindex d3cbdbf8..6adef6c0 100644--- a/src/core/brain/truth/merge-guard.ts+++ b/src/core/brain/truth/merge-guard.ts@@ -10,6 +10,7 @@*/import { normalizeEntityName } from "../entities/canonical.ts";+import { ENTITY_STATUS_SCOPE, entityStatusInScope } from "../entities/status-scope.ts";/** Categories guarded by default: people and organisations. */export const GUARDED_ENTITY_CATEGORIES: ReadonlyArray<string> = Object.freeze([@@ -61,7 +62,7 @@ function guardedAnchors(const haystack = normalizeEntityName(text);const ids: string[] = [];for (const entity of entities) {- if (entity.status !== "active") continue;+ if (!entityStatusInScope(entity.status, ENTITY_STATUS_SCOPE.canonical)) continue;if (!categories.has(entity.category.toLowerCase())) continue;const forms = [entity.name, ...entity.aliases].map((f) => normalizeEntityName(f));if (forms.some((f) => f.length >= 3 && haystack.includes(f))) ids.push(entity.id);diff --git a/src/core/brain/types.ts b/src/core/brain/types.tsindex 485840ee..3342a768 100644--- a/src/core/brain/types.ts+++ b/src/core/brain/types.ts@@ -424,6 +424,27 @@ export const BRAIN_LOG_EVENT_KIND = {* double-counts a fix.*/doctorRepair: "doctor-repair",+ /**+ * `vector-backfill` (provenance-at-the-boundary, unit F) - the operator+ * ran `o2b search vector-backfill --apply` and the vector phase wrote+ * embeddings for indexed chunks that had none. Payload carries the+ * number `embedded`, the number `pending` when the run started, the+ * `chunks_total` in the index, and the `agent`. A dry run and an apply+ * that finds nothing pending emit nothing, so the merged timeline+ * records only runs that actually populated vectors.+ */+ vectorBackfill: "vector-backfill",+ /**+ * `event-anchor-backfill` (provenance-at-the-boundary) - the operator+ * ran `o2b search event-anchor-backfill --apply` and event anchors were+ * resolved for indexed documents that a pre-anchor binary had left with+ * none. Payload carries the number `examined`, how many of those turned+ * out to be `anchored`, the number `pending` when the run started, the+ * `documents_total` in the index, and the `agent`. A dry run and an+ * apply that finds nothing pending emit nothing, so the merged timeline+ * records only runs that actually examined documents.+ */+ eventAnchorBackfill: "event-anchor-backfill",} as const;export type BrainLogEventKind = (typeof BRAIN_LOG_EVENT_KIND)[keyof typeof BRAIN_LOG_EVENT_KIND];@@ -1449,6 +1470,13 @@ export interface BrainConfig {* to these paths - the type is `read_paths` for a reason.*/readonly notes?: BrainNotesConfig;+ /**+ * Optional `write_binding:` block (provenance-at-the-boundary, unit+ * B). Bounds where a CALLER-NAMED write may land. Absent: no binding+ * is in force and every write path is byte-identical to a tree that+ * has never heard of the key.+ */+ readonly write_binding?: BrainWriteBindingConfig;/*** Optional `sessions:` block (Memory Integrity Suite). Capture* boundaries for session/message ingestion. Absent: every session@@ -1543,6 +1571,32 @@ export interface ResolvedBrainNotesConfig {readonly read_paths: ReadonlyArray<string>;}+/**+ * Optional `write_binding:` block (provenance-at-the-boundary, unit B).+ * The vault-relative prefixes a CALLER-NAMED write may land under.+ *+ * The authority is this file, which the operator controls and which no+ * MCP call can rewrite - never the caller's claimed identity. This+ * system has no credential: agent identity is an environment variable,+ * else a config key, else the literal `agent`, and ten tool families+ * accept a caller-supplied `agent` string that overrides it verbatim, so+ * a fence keyed to it would be bypassed by passing a different string.+ *+ * Absent block, or a block that omits `path_prefixes`, means no binding+ * is in force and every write path behaves exactly as it did before the+ * key existed. An empty list is rejected at load: a list that admits no+ * path is an off switch, and the way to turn the binding off is to+ * remove the block.+ */+export interface BrainWriteBindingConfig {+ /**+ * Vault-relative POSIX prefixes, normalised (no leading `./`, no+ * trailing `/`). A target is admitted when it equals one of these or+ * lies under it segment-wise.+ */+ readonly path_prefixes?: ReadonlyArray<string>;+}+/*** Optional `sessions:` block (Memory Integrity Suite). Capture* boundaries for runtime session ingestion: ignored sessions producediff --git a/src/core/brain/write-batch.ts b/src/core/brain/write-batch.tsindex 99418414..7227f118 100644--- a/src/core/brain/write-batch.ts+++ b/src/core/brain/write-batch.ts@@ -123,7 +123,31 @@ export type WriteBatchErrorCode =| "target_missing"| "duplicate_target"| "too_many_operations"- | "preference_not_found";+ | "preference_not_found"+ // provenance-at-the-boundary, unit B. Propagated from the shared+ // create-note envelope by `envelopeError`, which preserves the+ // envelope's code rather than flattening every refusal into+ // `invalid_operation`. The check runs in the projection phase, so an+ // operation outside the operator's declared binding aborts the whole+ // batch before any commit - including the operations that WERE+ // admitted.+ | "write_binding"+ // provenance-at-the-boundary, unit C. Declared because+ // `envelopeError` preserves whatever code the shared create-note+ // envelope raises, so the two unions have to stay in step. Neither is+ // reachable from a batch today: `create_note` operations expose+ // neither the `strict` validator nor template-mode bodies, and they+ // are not going to. A batch is all-or-nothing, and per-operation+ // authoring modes - above all `if_exists: "skip"` - would make one+ // `applied` count mean two different things in one result list. The+ // batch keeps refusing an occupied target outright.+ | "invalid_document"+ | "invalid_template"+ // The vault's `Brain/_brain.yaml` exists and does not validate, so+ // neither the vault scope nor the write binding can be determined.+ // Propagated from the same envelope: no operation in the batch can be+ // projected, and the operator - not the caller - holds the fix.+ | "config_invalid";/*** All-or-nothing failure for {@link applyWriteBatch}. Thrown during thediff --git a/src/core/doctor-readiness.ts b/src/core/doctor-readiness.tsindex b97c93b7..7fc89f3d 100644--- a/src/core/doctor-readiness.ts+++ b/src/core/doctor-readiness.ts@@ -23,6 +23,13 @@*/import { discoverConfig } from "./config.ts";+import {+ resolveSemanticCapability,+ SEMANTIC_CAPABILITY_CODE,+ semanticCapabilityIsBlocked,+ semanticCapabilityLabel,+} from "./search/capability-tier.ts";+import { providerProducesVectors } from "./search/embeddings/contract.ts";import { makeProvider } from "./search/embeddings/provider.ts";import { resolveSearchConfig } from "./search/index.ts";import { buildPayload } from "./install/payload.ts";@@ -174,18 +181,47 @@ export async function probeLlmKey(opts: ReadinessOptions): Promise<ReadinessVerd* dimension. The provider's `ping()` is the authoritative source of the* dimension (a cloud provider learns it from the first response), so the* probe pings under the per-check timeout rather than trusting config.+ *+ * A configuration the operator has not finished is `skipped`, not+ * `failed`, and WHICH configuration it is comes from the shared+ * capability tier (provenance-at-the-boundary, F1). Two consequences,+ * both deliberate: the sentinel provider is now recognised by the+ * contract predicate rather than by the literal name it happens to+ * carry, and a key-less remote provider is refused HERE instead of+ * making a credential-free outbound request whose only possible outcome+ * is an auth error. `probeLlmKey` above is the probe that reports the+ * missing key as a failure, so nothing is lost by skipping here.*/export async function probeEmbeddingProvider(opts: ReadinessOptions): Promise<ReadinessVerdict> {const configPath = resolveConfigPath(opts);+ let semantic;+ try {+ semantic = resolveSearchConfig({ vault: opts.vault, configPath }).semantic;+ } catch (err) {+ return { status: "fail", detail: `config failed to resolve: ${(err as Error).message}` };+ }+ // The tier is read BEFORE the provider is constructed: a provider whose+ // credential is missing refuses at construction, and reporting that as+ // "failed to load" described the wrong thing entirely.+ const capability = resolveSemanticCapability(semantic);+ if (semanticCapabilityIsBlocked(capability)) {+ return { status: "skipped", detail: await semanticCapabilityLabel(capability.code) };+ }let provider;try {- const { semantic } = resolveSearchConfig({ vault: opts.vault, configPath });provider = makeProvider(semantic);} catch (err) {return { status: "fail", detail: `provider failed to load: ${(err as Error).message}` };}- if (provider.name === "null") {- return { status: "skipped", detail: "semantic search disabled; no embedding provider" };+ // A provider that embeds nothing IS the `disabled` capability, whatever+ // the rest of the configuration says, so it names that code rather than+ // the tier's - the two agree for every provider in the tree and this+ // keeps them agreeing for one that is added later.+ if (!providerProducesVectors(provider)) {+ return {+ status: "skipped",+ detail: await semanticCapabilityLabel(SEMANTIC_CAPABILITY_CODE.disabled),+ };}const pong = await provider.ping();if (!pong.ok) {diff --git a/src/core/path-safety.ts b/src/core/path-safety.tsindex 70026c4a..6f2737a0 100644--- a/src/core/path-safety.ts+++ b/src/core/path-safety.ts@@ -137,6 +137,28 @@ export function canonicalNotePath(p: string): string {return toPosix(p).normalize("NFC");}+/**+ * A URI scheme per RFC 3986 §3.1: an ASCII letter followed by letters,+ * digits, `+`, `-` and `.`, terminated by a colon. Structural - never a+ * list of known scheme names, which could not be complete.+ */+const URI_SCHEME_RE = /^[a-z][a-z0-9+.-]*:/i;++/**+ * Does `target` name a location somewhere other than inside this vault -+ * either by carrying a URI scheme (`https:`, `mailto:`, `obsidian:`) or by+ * being protocol-relative (`//host/x`)?+ *+ * This is the boundary between "a path in the vault" and "an address+ * elsewhere", and both readings of that boundary want the same answer: a+ * markdown link target with a scheme is an external link rather than a note+ * reference, and a source identity with one entered the vault from outside+ * it. One predicate, so the two cannot drift apart.+ */+export function hasUriScheme(target: string): boolean {+ return URI_SCHEME_RE.test(target) || target.startsWith("//");+}+/*** Vault-relative path with forward slashes.*diff --git a/src/core/search/capability-tier.ts b/src/core/search/capability-tier.tsnew file mode 100644index 00000000..490d75f5--- /dev/null+++ b/src/core/search/capability-tier.ts@@ -0,0 +1,155 @@+/**+ * Semantic capability tier — the one resolver for "what did the operator+ * configure" (provenance-at-the-boundary, F1).+ *+ * Four surfaces used to derive this from `ResolvedEmbeddingConfig`+ * separately: the semantic candidate lane (`semantic-phase.ts`), the+ * index run's deferred-backend explanation (`indexer.ts`), the readiness+ * probe (`../doctor-readiness.ts`), and `indexCheck` (`indexer.ts`). Four+ * copies of one ladder drift, and one of them had drifted into building+ * operator-facing sentences at the call site. They share this module now.+ *+ * ## The boundary this module sits on+ *+ * A capability tier answers WHAT THE OPERATOR CONFIGURED. The+ * {@link SearchError} codes answer WHAT A CALL ATTEMPTED AND COULD NOT+ * DO. They are not two spellings of one fact and nothing reports both for+ * one condition: a surface that refused before attempting anything+ * reports the tier, and a surface that attempted and failed throws the+ * typed error. `EMBEDDING_KEY_MISSING` therefore still belongs to the+ * explicit `--semantic` request that was refused; the implicit+ * degradation beside it reports the tier.+ *+ * Runtime facts are deliberately NOT tiers. Whether sqlite-vec loaded and+ * whether the index holds any vectors are properties of this machine and+ * this index, not of the configuration, and each already has its own+ * report (`vecLoaded()`, the store counts, `VEC_EXTENSION_UNAVAILABLE`).+ * Folding them in here would make the tier mean two things at once.+ *+ * ## Why the label lookup is async+ *+ * Every operator-facing sentence resolves a REGISTERED code through+ * `core/brain/diagnostics.ts` (the v1.40.0 advisory rail). A STATIC+ * import of that registry from `core/search/` closes a module cycle -+ * `brain/next-step.ts` -> `brain/diagnostics.ts` -> `brain/doctor.ts` ->+ * `brain/doctor/entity-checks.ts` -> `brain/entities/semantic-dedup.ts`+ * -> `search/embeddings/configured-provider.ts` -> `search/index.ts` ->+ * back into this layer - which the import-cycle ratchet fails, and+ * rightly: a cycle makes module-scope initialisation order undefined. A+ * deferred `await import` is evaluated after both modules have+ * initialised and is the sanctioned cure this tree already uses, so the+ * label lookup is async and every caller of it is already async.+ */++import type { ResolvedEmbeddingConfig } from "./types.ts";++/**+ * The configured-capability ladder. Ordered from least to most capable;+ * a configuration sits at exactly one rung.+ */+export const SEMANTIC_CAPABILITY_TIER = Object.freeze({+ /** Semantic search is switched off, or the provider is the sentinel. */+ disabled: "disabled",+ /** Switched on with a provider that needs a credential, and none resolves. */+ credentialMissing: "credential-missing",+ /** Everything the configuration must supply is supplied. */+ configured: "configured",+} as const);++export type SemanticCapabilityTier =+ (typeof SEMANTIC_CAPABILITY_TIER)[keyof typeof SEMANTIC_CAPABILITY_TIER];++/**+ * Registered diagnostic code per tier. Every one of these has an entry in+ * `DIAGNOSTIC_SIGNALS`; the codes are written as literals on both sides+ * (the same convention the CLI's `search-index-built` follows) so this+ * module stays a leaf, and a test asserts each one resolves.+ */+export const SEMANTIC_CAPABILITY_CODE = Object.freeze({+ disabled: "semantic-capability-disabled",+ credentialMissing: "semantic-capability-key-missing",+ configured: "semantic-capability-configured",+} as const);++/**+ * Registered codes for the two VECTOR-population states, which are index+ * facts rather than capability tiers and so are named apart from them:+ * `deferred` is a run that never asked for embeddings, `pending` is an+ * index holding chunks that have no vector.+ */+export const SEMANTIC_VECTOR_CODE = Object.freeze({+ deferred: "semantic-vectors-deferred",+ pending: "semantic-vectors-pending",+} as const);++/**+ * Providers that cannot embed without a credential. `local` embeds+ * offline and `disabled` embeds not at all, so neither appears.+ */+const PROVIDERS_REQUIRING_CREDENTIAL: ReadonlySet<string> = new Set([+ "openai-compat",+ "zeroentropy",+]);++/** The configured-capability verdict: one tier and its registered code. */+export interface SemanticCapability {+ readonly tier: SemanticCapabilityTier;+ /** The registered diagnostic code for {@link tier}. Always registered. */+ readonly code: string;+}++const CAPABILITY_BY_TIER: Readonly<Record<SemanticCapabilityTier, SemanticCapability>> =+ Object.freeze({+ [SEMANTIC_CAPABILITY_TIER.disabled]: Object.freeze({+ tier: SEMANTIC_CAPABILITY_TIER.disabled,+ code: SEMANTIC_CAPABILITY_CODE.disabled,+ }),+ [SEMANTIC_CAPABILITY_TIER.credentialMissing]: Object.freeze({+ tier: SEMANTIC_CAPABILITY_TIER.credentialMissing,+ code: SEMANTIC_CAPABILITY_CODE.credentialMissing,+ }),+ [SEMANTIC_CAPABILITY_TIER.configured]: Object.freeze({+ tier: SEMANTIC_CAPABILITY_TIER.configured,+ code: SEMANTIC_CAPABILITY_CODE.configured,+ }),+ });++/** True when at least one API key resolved for this configuration. */+function credentialResolves(semantic: ResolvedEmbeddingConfig): boolean {+ if ((semantic.apiKeys?.length ?? 0) > 0) return true;+ return typeof semantic.apiKey === "string" && semantic.apiKey.length > 0;+}++/**+ * Classify what the operator configured. Pure, synchronous, and derived+ * from the resolved config alone - never from `process.env`, never from+ * the index, never from a live provider.+ */+export function resolveSemanticCapability(semantic: ResolvedEmbeddingConfig): SemanticCapability {+ if (!semantic.enabled || semantic.provider === "disabled") {+ return CAPABILITY_BY_TIER[SEMANTIC_CAPABILITY_TIER.disabled];+ }+ if (PROVIDERS_REQUIRING_CREDENTIAL.has(semantic.provider) && !credentialResolves(semantic)) {+ return CAPABILITY_BY_TIER[SEMANTIC_CAPABILITY_TIER.credentialMissing];+ }+ return CAPABILITY_BY_TIER[SEMANTIC_CAPABILITY_TIER.configured];+}++/** True when the configuration cannot reach a provider as it stands. */+export function semanticCapabilityIsBlocked(capability: SemanticCapability): boolean {+ return capability.tier !== SEMANTIC_CAPABILITY_TIER.configured;+}++/**+ * The operator-facing label for a registered capability or vector code,+ * read from the diagnostics registry.+ *+ * Throws {@link UnregisteredNextStepError} for a code the registry does+ * not hold. That is deliberate: this function's whole job is to keep the+ * sentence out of the call site, and returning the code itself - or an+ * invented sentence - would put it back while reading as success.+ */+export async function semanticCapabilityLabel(code: string): Promise<string> {+ const { requireNextStep } = await import("../brain/next-step.ts");+ return requireNextStep(code).issueClass;+}diff --git a/src/core/search/coverage.ts b/src/core/search/coverage.tsindex ceb3b3f1..72800ddd 100644--- a/src/core/search/coverage.ts+++ b/src/core/search/coverage.ts@@ -55,12 +55,25 @@ export function idfForTerm(df: number, documentCount: number): number {}/**- * A term is rare (high-signal) when it appears in at most- * `RARE_TERM_CORPUS_SHARE` of the corpus, with a floor of one document- * so tiny corpora still classify their unique terms as rare.+ * A term is rare (high-signal) when it appears in at most `corpusShare` of+ * the corpus, with a floor of one document so tiny corpora still classify+ * their unique terms as rare.+ *+ * `corpusShare` defaults to {@link RARE_TERM_CORPUS_SHARE}, the vault-corpus+ * value this function was written for. It is a parameter because the same+ * classification - corpus document frequency against a share of the corpus,+ * with no vocabulary list anywhere in it - is what separates a+ * discriminating term from a corpus-common one in the far smaller skill+ * descriptor corpus (t_ccb05134). The share differs by corpus; the shape+ * must not, or the two would drift into disagreeing about what "common"+ * means.*/-export function isRareTerm(df: number, documentCount: number): boolean {- return df <= Math.max(1, Math.floor(RARE_TERM_CORPUS_SHARE * documentCount));+export function isRareTerm(+ df: number,+ documentCount: number,+ corpusShare: number = RARE_TERM_CORPUS_SHARE,+): boolean {+ return df <= Math.max(1, Math.floor(corpusShare * documentCount));}export interface TermCoverage {diff --git a/src/core/search/embeddings/configured-provider.ts b/src/core/search/embeddings/configured-provider.tsindex 57414613..b47ad534 100644--- a/src/core/search/embeddings/configured-provider.ts+++ b/src/core/search/embeddings/configured-provider.ts@@ -11,13 +11,19 @@import { discoverConfig } from "../../config.ts";import { resolveSearchConfig } from "../index.ts";import { makeProvider } from "./provider.ts";+import { providerProducesVectors } from "./contract.ts";import type { EmbeddingProvider } from "./contract.ts";/*** Resolve the vault's configured embedding provider. Returns `null` when- * semantic search is disabled (the null provider) or when resolution+ * semantic search is disabled (the sentinel provider) or when resolution* throws — callers then fall back to a deterministic path rather than* failing. Mirrors the inline guard in the hygiene dedup detector.+ *+ * The sentinel is recognised by the contract predicate, not by its name+ * (provenance-at-the-boundary, F2): a provider registered under any other+ * name used to slip past this guard and be handed to a caller that then+ * called `embed()` on it.*/export function resolveConfiguredEmbeddingProvider(vault: string,@@ -26,7 +32,7 @@ export function resolveConfiguredEmbeddingProvider(try {const configPath = opts.configPath ?? discoverConfig().path;const provider = makeProvider(resolveSearchConfig({ vault, configPath }).semantic);- return provider.name === "null" ? null : provider;+ return providerProducesVectors(provider) ? provider : null;} catch {return null;}diff --git a/src/core/search/embeddings/contract.ts b/src/core/search/embeddings/contract.tsindex af1d8668..08f9fd05 100644--- a/src/core/search/embeddings/contract.ts+++ b/src/core/search/embeddings/contract.ts@@ -28,4 +28,32 @@ export interface EmbeddingProvider {* (NullProvider, MockEmbeddingProvider) leave this undefined.*/consumeRetryCount?(): number;+ /**+ * Whether this provider produces vectors at all+ * (provenance-at-the-boundary, F2). Read through+ * {@link providerProducesVectors}, never directly.+ *+ * Four call sites used to answer this by comparing `name` to the+ * literal `"null"`, so any provider registered under a different name+ * bypassed all four silently. The question belongs to the contract, so+ * the contract answers it.+ *+ * Optional and additive, in the same shape as `consumeRetryCount`+ * above. An absent declaration means "produces vectors", which is the+ * truthful default for every implementation that is a real embedder -+ * it is not a fallback covering a failure, because there is no failure+ * here to cover: a provider that CANNOT embed is a configuration+ * sentinel, and a sentinel is written knowing it is one.+ */+ readonly producesVectors?: boolean;+}++/**+ * True when `provider` embeds text. The single predicate every site that+ * needs to tell a real provider from the configuration sentinel calls -+ * `resolveConfiguredEmbeddingProvider`, the hygiene dedup detector, and+ * the readiness probe.+ */+export function providerProducesVectors(provider: EmbeddingProvider): boolean {+ return provider.producesVectors ?? true;}diff --git a/src/core/search/embeddings/http-util.ts b/src/core/search/embeddings/http-util.tsindex d74d5365..d99d93c1 100644--- a/src/core/search/embeddings/http-util.ts+++ b/src/core/search/embeddings/http-util.ts@@ -6,7 +6,9 @@* unit-normalisation semantics.*/+import { SearchError } from "../types.ts";import { assertValidVector } from "../vector-guard.ts";+import { estimateTokens } from "./signature.ts";/** Statuses that warrant a transient retry with backoff. */export const RETRYABLE_STATUSES: ReadonlySet<number> = new Set([429, 500, 502, 503, 504]);@@ -113,10 +115,93 @@ export function unitNormaliseInPlace(v: number[]): number[] {return v;}-/** Split an array into fixed-size chunks (step >= 1). */+/**+ * The two batching caps, named for the operator-facing config key each one+ * carries, so a refusal says which key to change.+ */+const CAP_FIELD = Object.freeze({+ /** `embedding_batch_size`: how many items one request may carry. */+ size: "batch size",+ /** `embedding_batch_tokens`: how many estimated tokens one request may carry. */+ tokens: "token budget",+});++/**+ * Take a batching cap EXACTLY as the caller gave it, or refuse it.+ *+ * `Math.max(1, cap | 0)` used to stand at both call sites. `| 0` truncates+ * to a signed 32-bit int, so a cap above 2^31-1 wrapped NEGATIVE and the+ * clamp then lifted it to 1: the largest batch an operator can ask for+ * silently became the smallest one possible - one HTTP request per item,+ * at full `embedding_concurrency`. Nothing is truncated now, and nothing is+ * clamped: a cap this packer cannot step by is named rather than quietly+ * reinterpreted into a different request plan than the one configured.+ *+ * Both keys are already refused below 1, at a fraction, and at a+ * non-finite value by config validation (`search/index.ts`), so a resolved+ * config never reaches this guard; it holds the line for programmatic+ * callers that construct a `ResolvedEmbeddingConfig` directly.+ */+function requireBatchCap(cap: number, field: string): number {+ if (!Number.isInteger(cap) || cap < 1) {+ throw new SearchError("INVALID_INPUT", `${field} must be an integer >= 1, got ${cap}`);+ }+ return cap;+}++/** Split an array into fixed-size chunks. Refuses a step below 1 or fractional. */export function chunkArray<T>(arr: ReadonlyArray<T>, size: number): T[][] {- const step = Math.max(1, size | 0);+ const step = requireBatchCap(size, CAP_FIELD.size);const out: T[][] = [];for (let i = 0; i < arr.length; i += step) out.push(arr.slice(i, i + step) as T[]);return out;}++/**+ * Split an array into batches that close on whichever of two caps fills+ * first: the item count (`size`, i.e. `embedding_batch_size`) or the+ * accumulated token estimate (`tokenBudget`, i.e. `embedding_batch_tokens`).+ *+ * `tokenBudget` of `undefined` - the state of an unset config key - delegates+ * straight to {@link chunkArray}, so batching is byte-identical to the fixed+ * stride that preceded the budget.+ *+ * The estimate is {@link estimateTokens}, the estimator already resident in+ * this layer and already governing the indexer's cost gate, so the gate and+ * the batch budget describe the same texts identically. It deliberately+ * differs from the chunker's word-count estimator: that one sizes chunks of+ * a document, this one sizes a request. `textOf` must return the text+ * exactly as it will be sent, instruction prefix included, because the+ * provider prepends the prefix before batching and the provider's token+ * ceiling counts it.+ *+ * An item whose own estimate already exceeds the budget cannot be made to fit+ * by closing the batch around it, so it travels alone. Splitting the text+ * itself is the chunker's job, not the request packer's; dropping it would be+ * silent data loss.+ */+export function chunkArrayByTokenBudget<T>(+ arr: ReadonlyArray<T>,+ size: number,+ tokenBudget: number | undefined,+ textOf: (item: T) => string,+): T[][] {+ if (tokenBudget === undefined) return chunkArray(arr, size);+ const step = requireBatchCap(size, CAP_FIELD.size);+ const budget = requireBatchCap(tokenBudget, CAP_FIELD.tokens);+ const out: T[][] = [];+ let current: T[] = [];+ let currentTokens = 0;+ for (const item of arr) {+ const tokens = estimateTokens([textOf(item)]);+ if (current.length > 0 && (current.length >= step || currentTokens + tokens > budget)) {+ out.push(current);+ current = [];+ currentTokens = 0;+ }+ current.push(item);+ currentTokens += tokens;+ }+ if (current.length > 0) out.push(current);+ return out;+}diff --git a/src/core/search/embeddings/null-provider.ts b/src/core/search/embeddings/null-provider.tsindex 389778f2..5470534c 100644--- a/src/core/search/embeddings/null-provider.ts+++ b/src/core/search/embeddings/null-provider.ts@@ -14,6 +14,13 @@ export class NullProvider implements EmbeddingProvider {readonly name = "null";readonly model = "";readonly dimension = null;+ /**+ * The one implementation in the tree that embeds nothing, and the+ * reason the contract carries this flag at all+ * (provenance-at-the-boundary, F2). Callers ask+ * `providerProducesVectors(provider)`, never `provider.name === "null"`.+ */+ readonly producesVectors = false;embed(_texts: ReadonlyArray<string>): Promise<number[][]> {return Promise.reject(diff --git a/src/core/search/embeddings/openai-compat.ts b/src/core/search/embeddings/openai-compat.tsindex c4bc2a07..896419c4 100644--- a/src/core/search/embeddings/openai-compat.ts+++ b/src/core/search/embeddings/openai-compat.ts@@ -5,11 +5,13 @@** Network rules:* - One concurrent batch per semaphore slot (`embedding_concurrency`).- * - Each batch contains up to `embedding_batch_size` texts.+ * - Each batch contains up to `embedding_batch_size` texts and, when+ * `embedding_batch_tokens` is set, at most that many estimated tokens+ * of prefixed text; the batch closes on whichever cap fills first.* - Per-request timeout: `embedding_timeout_ms`.* - Retry on `429`, `5xx`, and network/timeout errors with exponential- * backoff `1s/2s` (+ ±25% jitter), three attempts total. Other 4xx- * fail fast.+ * backoff `1s/2s` (+ ±25% jitter), for `embedding_max_retries` attempts+ * in total (default 6). Other 4xx fail fast.* - Vectors are unit-normalised so cosine similarity equals* `1 - L2² / 2` (see ranker).*@@ -26,7 +28,7 @@ import {RATE_LIMIT_STATUS,RETRYABLE_STATUSES,Semaphore,- chunkArray,+ chunkArrayByTokenBudget,jittered,parseRetryAfterMs,sleep,@@ -266,9 +268,13 @@ export class OpenAICompatProvider implements EmbeddingProvider {if (texts.length === 0) return [];const prefix = this.prefixFor(kind);const prepared = prefix === "" ? texts : texts.map((t) => prefix + t);- const batches = chunkArray(+ // Batch over the PREFIXED text: the prefix is part of every request the+ // provider will count, so the token budget must see it too.+ const batches = chunkArrayByTokenBudget(prepared.map((t, i) => ({ text: t, originalIndex: i })),this.config.batchSize,+ this.config.batchTokens,+ (b) => b.text,);const sem = new Semaphore(this.config.concurrency);const out: number[][] = new Array(texts.length);diff --git a/src/core/search/embeddings/zeroentropy.ts b/src/core/search/embeddings/zeroentropy.tsindex 4baa2f35..8747610c 100644--- a/src/core/search/embeddings/zeroentropy.ts+++ b/src/core/search/embeddings/zeroentropy.ts@@ -11,9 +11,10 @@** `results` preserves input order (there is no per-item index field), so* vectors map positionally. Network discipline mirrors the OpenAI-compat- * provider: bounded concurrency, per-request timeout, retry on 429/5xx and- * network/timeout with jittered backoff, vectors unit-normalised so cosine- * equals 1 - L2²/2.+ * provider: bounded concurrency, per-request timeout, batches closing on+ * whichever of `embedding_batch_size` / `embedding_batch_tokens` fills+ * first, retry on 429/5xx and network/timeout with jittered backoff,+ * vectors unit-normalised so cosine equals 1 - L2²/2.** The interface is symmetric (one `embed`), so `input_type` is fixed to* "document" - consistent with how OSB treats query and passage embeddings@@ -26,7 +27,7 @@ import type { EmbeddingProvider } from "./contract.ts";import {RETRYABLE_STATUSES,Semaphore,- chunkArray,+ chunkArrayByTokenBudget,jittered,sleep,unitNormaliseInPlace,@@ -97,9 +98,11 @@ export class ZeroEntropyProvider implements EmbeddingProvider {async embed(texts: ReadonlyArray<string>): Promise<number[][]> {if (texts.length === 0) return [];- const batches = chunkArray(+ const batches = chunkArrayByTokenBudget(texts.map((t, i) => ({ text: t, originalIndex: i })),this.config.batchSize,+ this.config.batchTokens,+ (b) => b.text,);const sem = new Semaphore(this.config.concurrency);const out: number[][] = new Array(texts.length);diff --git a/src/core/search/entity-alias.ts b/src/core/search/entity-alias.tsindex 719a6bb4..17b8acf2 100644--- a/src/core/search/entity-alias.ts+++ b/src/core/search/entity-alias.ts@@ -12,7 +12,7 @@import { buildEntityIndex } from "../brain/entities/index-builder.ts";import { normalizeEntityName } from "../brain/entities/canonical.ts";-import { BRAIN_ENTITY_STATUS } from "../brain/entities/types.ts";+import { ENTITY_STATUS_SCOPE, entityStatusInScope } from "../brain/entities/status-scope.ts";export interface QueryEntityExpansion {/** Original query entities plus every form added by the registry. */@@ -54,7 +54,7 @@ export function expandQueryEntities(const sourceIds: string[] = [];for (const entity of index.entities) {- if (entity.status !== BRAIN_ENTITY_STATUS.active) continue;+ if (!entityStatusInScope(entity.status, ENTITY_STATUS_SCOPE.canonical)) continue;const forms = [entity.name, ...entity.aliases].map((f) => normalizeEntityName(f));if (!forms.some((f) => querySet.has(f))) continue;let contributed = false;diff --git a/src/core/search/event-anchor-backfill.ts b/src/core/search/event-anchor-backfill.tsnew file mode 100644index 00000000..18d37fa6--- /dev/null+++ b/src/core/search/event-anchor-backfill.ts@@ -0,0 +1,127 @@+/**+ * Event-anchor backfill (provenance at the boundary, t_ac1c4176).+ *+ * The v11 migration adds the anchor columns to an existing index IN+ * PLACE. A schema bump reindexes nothing - `applyMigrations` alters the+ * table and only a version newer than this binary refuses - and the+ * indexer resolves an anchor only below its two content-identity+ * fastpaths. Both of those are right for content: unchanged content can+ * only ever resolve to the same anchor, so declining to recompute cannot+ * stale it. Neither is right across a BINARY upgrade: a document indexed+ * by the previous binary has no anchor, its content never changes, and+ * so no run ever computes one. The feature is inert on every vault that+ * already existed, and the column now gates a hard `since` / `until`+ * filter.+ *+ * This module is the operator-facing way to close that gap without+ * re-walking, re-chunking or re-embedding the vault. It reads each+ * unexamined document's file, resolves the anchor with the SAME function+ * the indexer uses, and writes the four anchor columns on that row.+ * Nothing else on the row moves - not the stat fingerprint, not the+ * chunks, not the links - so a backfill cannot disturb the fastpaths.+ *+ * Contract, taken verbatim from `brain/authored-at-backfill.ts` and+ * `search/vector-backfill.ts`:+ * - Dry-run by DEFAULT. {@link planEventAnchorBackfill} without `apply`+ * opens the index read-only, counts, and writes nothing at all.+ * - `apply` is the only mutating path.+ * - Idempotent: an examined row is never re-examined, so a re-run over+ * a backfilled index finds nothing pending and does nothing.+ *+ * A file that cannot be read is NAMED and left pending. The run does not+ * get to record "this note declares no date" for a note it never opened;+ * that is the exact conflation the examination marker exists to prevent.+ */++import { readFileSync } from "node:fs";+import { join } from "node:path";++import { parseFrontmatterText } from "../vault.ts";+import { resolveEventAnchor } from "./event-anchor.ts";+import { Store } from "./store.ts";+import type { ResolvedSearchConfig } from "./types.ts";++export interface EventAnchorBackfillOptions {+ /** When true, resolve and store the missing anchors. Default false. */+ readonly apply?: boolean;+}++/** One indexed document whose file the run could not read. */+export interface UnreadableDocument {+ /** Vault-relative POSIX path, as the index holds it. */+ readonly path: string;+ readonly message: string;+}++export interface EventAnchorBackfillResult {+ /** Whether the run mutated the index (`false` for a dry run). */+ readonly applied: boolean;+ /** Documents in the index. */+ readonly documentsTotal: number;+ /** Documents no anchor-aware binary had examined when the run started. */+ readonly pending: number;+ /** Documents this run read and resolved an answer for (0 on a dry run). */+ readonly examined: number;+ /**+ * Of those, the ones that turned out to DECLARE a date. The remainder+ * declare none - which is now a recorded verdict rather than an+ * unexamined row.+ */+ readonly anchored: number;+ /** Indexed documents whose file could not be read; each stays pending. */+ readonly unreadable: ReadonlyArray<UnreadableDocument>;+}++/**+ * Count the documents no anchor-aware binary has examined and, when+ * `apply` is set, examine them.+ *+ * Throws `SearchError("INDEX_MISSING")` from {@link Store.open} when+ * there is no index to read: an absent index is a different answer from+ * an index with nothing pending, and collapsing the two would report+ * "nothing to do" for a vault that has never been indexed.+ */+export async function planEventAnchorBackfill(+ config: ResolvedSearchConfig,+ opts: EventAnchorBackfillOptions = {},+): Promise<EventAnchorBackfillResult> {+ const apply = opts.apply === true;+ const store = await Store.open(config, { mode: apply ? "write" : "read" });+ try {+ const pending = store.unexaminedEventAnchorPaths();+ const documentsTotal = store.counts().documents;+ const unreadable: UnreadableDocument[] = [];+ let examined = 0;+ let anchored = 0;++ if (apply) {+ for (const path of pending) {+ let content: string;+ try {+ content = readFileSync(join(config.vault, path), "utf8");+ } catch (err) {+ unreadable.push(+ Object.freeze({ path, message: err instanceof Error ? err.message : String(err) }),+ );+ continue;+ }+ const [frontmatter, body] = parseFrontmatterText(content);+ const anchor = resolveEventAnchor(frontmatter as Record<string, unknown>, body);+ store.setEventAnchor(path, anchor);+ examined++;+ if (anchor !== null) anchored++;+ }+ }++ return Object.freeze({+ applied: apply,+ documentsTotal,+ pending: pending.length,+ examined,+ anchored,+ unreadable: Object.freeze(unreadable),+ });+ } finally {+ await store.close();+ }+}diff --git a/src/core/search/event-anchor.ts b/src/core/search/event-anchor.tsnew file mode 100644index 00000000..1fcab90a--- /dev/null+++ b/src/core/search/event-anchor.ts@@ -0,0 +1,232 @@+/**+ * The event anchor: ONE event-time interval per document, resolved from+ * what the document itself says, and materialised onto the `documents`+ * row at index time (provenance at the boundary, t_ac1c4176).+ *+ * ## Why this exists+ *+ * Ranking already judged a note by a content-derived event time, but only+ * when the note declared `valid_from` / `valid_until` in frontmatter. A+ * note whose only statement of when it is ABOUT sits in its BODY was+ * ranked by when its file was last written. This module closes that gap,+ * and the index stores the result so a candidate's body is not re-scanned+ * on every query.+ *+ * ## The ladder+ *+ * First rung that DECLARES anything wins, and the winning rung is+ * recorded as a registered {@link EventAnchorSource} token - never as+ * free text, so a consumer can branch on provenance:+ *+ * 1. `valid_from` / `valid_until` frontmatter (the existing bi-temporal+ * validity window);+ * 2. `created_at` frontmatter;+ * 3. `date` frontmatter;+ * 4. an ISO token in the note body.+ *+ * A rung that is DECLARED but cannot be read stops the ladder and yields+ * no anchor. Absence and inability-to-examine are different answers: a+ * malformed `created_at` is not an absent one, and quietly substituting a+ * lower rung's answer for it would report a date the document never+ * stated.+ *+ * ## Recorded provenance is not the same as event time+ *+ * All four rungs are RECORDED - that is what "provenance at the+ * boundary" means, and it is why the rung is stored as a registered+ * token instead of being thrown away. Which of them the query side is+ * allowed to JUDGE a document by is a separate, narrower question, and+ * {@link anchorIsEventTime} answers it: `created_at` and `date` are+ * stamped by every Brain writer, describe when a file was authored, and+ * were never event time before this release. The query-side event-time+ * resolver feeds a filter that DROPS candidates, so promoting them into+ * it would silently change what a time-ranged query means - "what did I+ * touch since January" would start answering about authorship. The body+ * rung is the one this release adds, and it is the one that judges.+ *+ * ## Two deliberate refusals and one deliberate acceptance+ *+ * - **Slash-formatted dates are not recognised.** Resolving `04/03/2026`+ * as day-month or month-day requires a locale, and a locale is a+ * natural-language signal. The extractor below recognises formal ISO+ * tokens only, in every script alike.+ * - **A clock-relative derivation is not stored.** The shared extractor+ * anchors a bare ISO-8601 duration to `now`. A value derived from a+ * clock ages silently, while both index fastpaths gate on content+ * identity and so correctly decline to recompute it. Rather than+ * assuming which branch is clock-free, {@link resolveEventAnchor} runs+ * the extractor under two distant probe clocks and refuses any result+ * that differs between them - a structural proof that survives future+ * changes to the extractor.+ * - **A future date is kept.** A note that says an embargo lifts in 2099+ * has declared an event time. Dropping it would be inference dressed as+ * validation.+ *+ * ## Day-width windows+ *+ * A bare date is a DAY, not an instant, so it snaps to day start and day+ * end - the same snapping `parseValidityPoint` already applies to a bare+ * frontmatter date. That makes the anchor an interval, which is what the+ * hard `since`/`until` filter, the targeted-retry coverage gate and the+ * temporal bridge all consume, so the anchor composes with all of them+ * instead of being wired only into the ranking boost.+ *+ * Pure and deterministic: same frontmatter and body, same anchor, on any+ * machine at any time.+ */++import { extractTemporalConstraints } from "../brain/temporal-extract.ts";+import { parseValidityWindow } from "./validity.ts";++/**+ * Registered vocabulary for the ladder rung an anchor came from. Stored+ * verbatim in `documents.event_anchor_source`.+ */+export const EVENT_ANCHOR_SOURCE = {+ /** The note's declared `valid_from` / `valid_until` frontmatter window. */+ validity: "validity",+ /** The note's `created_at` frontmatter field. */+ createdAt: "created_at",+ /** The note's `date` frontmatter field (the brain-log convention). */+ date: "date",+ /** An ISO token found in the note body. */+ body: "body",+} as const;++export type EventAnchorSource = (typeof EVENT_ANCHOR_SOURCE)[keyof typeof EVENT_ANCHOR_SOURCE];++const EVENT_ANCHOR_SOURCE_VALUES: ReadonlyArray<string> = Object.freeze(+ Object.values(EVENT_ANCHOR_SOURCE),+);++export function isEventAnchorSource(value: unknown): value is EventAnchorSource {+ return typeof value === "string" && EVENT_ANCHOR_SOURCE_VALUES.includes(value);+}++/**+ * The rungs that are EVENT TIME, and may therefore decide whether a+ * candidate survives the hard `since` / `until` filter. See the+ * "recorded provenance is not the same as event time" section above for+ * why the two frontmatter point fields are not among them.+ */+const EVENT_TIME_ANCHOR_SOURCES: ReadonlyArray<EventAnchorSource> = Object.freeze([+ EVENT_ANCHOR_SOURCE.validity,+ EVENT_ANCHOR_SOURCE.body,+]);++/** True when `anchor` came from a rung the query side judges a document by. */+export function anchorIsEventTime(anchor: EventAnchor): boolean {+ return EVENT_TIME_ANCHOR_SOURCES.includes(anchor.source);+}++/**+ * A document's resolved event interval. At least one side is non-null;+ * an open side means the document declared no bound there.+ *+ * Milliseconds, not seconds: a day-end bound is `dayStart + 86_400_000 -+ * 1`, and truncating that to whole seconds would move the boundary by+ * almost a second on every interval comparison. `mtime` and `authored_at`+ * are unix SECONDS on the same table, so the unit is spelled into these+ * field and column names rather than left to be inferred.+ */+export interface EventAnchor {+ readonly startMs: number | null;+ readonly endMs: number | null;+ readonly source: EventAnchorSource;+}++/** Frontmatter point fields, in ladder order below the validity window. */+const POINT_FIELDS: ReadonlyArray<readonly [string, EventAnchorSource]> = Object.freeze([+ ["created_at", EVENT_ANCHOR_SOURCE.createdAt],+ ["date", EVENT_ANCHOR_SOURCE.date],+]);++/**+ * Two distant, fixed instants used to prove a body derivation is+ * independent of the clock. Any extractor branch that consults `now`+ * produces different output under these two, and is refused.+ */+const CLOCK_PROBE_EARLY = new Date("2001-01-01T00:00:00Z");+const CLOCK_PROBE_LATE = new Date("2031-01-01T00:00:00Z");++/** Leading `YYYY-MM-DD` of an ISO instant. */+const ISO_DATE_PREFIX_RE = /^(\d{4}-\d{2}-\d{2})/;++/**+ * Resolve a document's event anchor, or null when it declares nothing+ * readable. `body` is the note text with frontmatter already stripped, so+ * a frontmatter value can never be mistaken for a body statement.+ */+export function resolveEventAnchor(+ frontmatter: Record<string, unknown>,+ body: string,+): EventAnchor | null {+ const declared = parseValidityWindow(frontmatter);+ if (declared !== null) {+ if (declared.invalid) return null;+ return freezeAnchor(declared.validFromMs, declared.validUntilMs, EVENT_ANCHOR_SOURCE.validity);+ }++ for (const [field, source] of POINT_FIELDS) {+ const raw = frontmatter[field];+ if (raw === undefined || raw === null) continue;+ return windowAnchor(raw, raw, source);+ }++ const constraints = clockIndependentConstraints(body);+ if (constraints === null) return null;+ const from = isoDatePart(constraints.from);+ const until = isoDatePart(constraints.until);+ if (from === null || until === null) return null;+ return windowAnchor(from, until, EVENT_ANCHOR_SOURCE.body);+}++/**+ * Build an anchor by handing the two bounds to the SAME parser the+ * frontmatter validity window uses, so bare dates snap to day start /+ * day end exactly as a declared `valid_from` / `valid_until` pair would.+ * A bound that cannot be parsed yields no anchor.+ */+function windowAnchor(+ from: unknown,+ until: unknown,+ source: EventAnchorSource,+): EventAnchor | null {+ const window = parseValidityWindow({ valid_from: from, valid_until: until });+ if (window === null || window.invalid) return null;+ return freezeAnchor(window.validFromMs, window.validUntilMs, source);+}++function freezeAnchor(+ startMs: number | null,+ endMs: number | null,+ source: EventAnchorSource,+): EventAnchor | null {+ if (startMs === null && endMs === null) return null;+ return Object.freeze({ startMs, endMs, source });+}++/** The two bounds a body statement resolves to, before day snapping. */+interface BodyConstraints {+ readonly from: string;+ /** The lone-date case declares one day, so the end bound is that day. */+ readonly until: string;+}++/**+ * Run the shared extractor under two distant clocks and return its result+ * only when the two agree. A disagreement means the derivation consulted+ * `now`, which must never reach stored data.+ */+function clockIndependentConstraints(body: string): BodyConstraints | null {+ const early = extractTemporalConstraints(body, { now: CLOCK_PROBE_EARLY });+ const late = extractTemporalConstraints(body, { now: CLOCK_PROBE_LATE });+ if (early.valid_from !== late.valid_from || early.valid_until !== late.valid_until) return null;+ if (early.valid_from === undefined) return null;+ return { from: early.valid_from, until: early.valid_until ?? early.valid_from };+}++function isoDatePart(iso: string): string | null {+ const m = ISO_DATE_PREFIX_RE.exec(iso);+ return m === null ? null : m[1]!;+}diff --git a/src/core/search/index.ts b/src/core/search/index.tsindex 959b7fb0..039e8bda 100644--- a/src/core/search/index.ts+++ b/src/core/search/index.ts@@ -270,6 +270,14 @@ function validateIntegerRange(n: number, fieldName: string, range?: IntegerRangeif (!Number.isFinite(n)) {throw new SearchError("INVALID_INPUT", `${fieldName} must be a finite number, got '${n}'`);}+ // Integrality, for the same reason: a string-sourced value has already+ // been through `parseInteger`, which refuses a fraction, so accepting one+ // from a programmatic override would leave two entry paths disagreeing+ // about what the field means - and hand a downstream consumer (the batch+ // packer, the chunker's stride) a value it cannot step by.+ if (!Number.isInteger(n)) {+ throw new SearchError("INVALID_INPUT", `${fieldName} must be an integer, got ${n}`);+ }if (range?.min !== undefined && n < range.min) {throw new SearchError("INVALID_INPUT", `${fieldName} must be >= ${range.min}, got ${n}`);}@@ -322,6 +330,15 @@ function validateResolvedConfig(config: ResolvedSearchConfig): void {validateIntegerRange(config.semantic.batchSize, "embedding_batch_size", {min: 1,});+ // Optional: absent means no token budget. Present-but-zero is a+ // misconfiguration (a batch that can hold nothing), rejected the same way+ // `embedding_batch_size` is - including when supplied as a programmatic+ // override that never passed through `parseInteger`.+ if (config.semantic.batchTokens !== undefined) {+ validateIntegerRange(config.semantic.batchTokens, "embedding_batch_tokens", {+ min: 1,+ });+ }validateIntegerRange(config.semantic.maxRetries, "embedding_max_retries", {min: 1,});@@ -489,6 +506,20 @@ export function resolveSearchConfig(opts: {"embedding_batch_size",{ min: 1 },);+ // Per-request token budget. No DEFAULTS entry, because there is no+ // provider-independent ceiling to default to and an absent key must stay+ // absent: the providers read `undefined` as "count cap only", which is+ // byte-identical to the pre-feature fixed stride.+ const batchTokensRaw = envOrConfig(+ env,+ config,+ "OPEN_SECOND_BRAIN_EMBEDDING_BATCH_TOKENS",+ "embedding_batch_tokens",+ );+ const batchTokens =+ batchTokensRaw === null+ ? null+ : parseInteger(batchTokensRaw, 0, "embedding_batch_tokens", { min: 1 });const maxRetries = parseInteger(envOrConfig(env, config, "OPEN_SECOND_BRAIN_EMBEDDING_MAX_RETRIES", "embedding_max_retries"),DEFAULTS.maxRetries,@@ -527,6 +558,7 @@ export function resolveSearchConfig(opts: {timeoutMs,concurrency,batchSize,+ ...(batchTokens === null ? {} : { batchTokens }),maxRetries,costGateUsd,queryPrefix,diff --git a/src/core/search/indexer.ts b/src/core/search/indexer.tsindex 7db42d69..cafd7483 100644--- a/src/core/search/indexer.ts+++ b/src/core/search/indexer.ts@@ -26,6 +26,14 @@ import {} from "node:fs";import { basename, dirname } from "node:path";+import {+ resolveSemanticCapability,+ SEMANTIC_CAPABILITY_CODE,+ SEMANTIC_CAPABILITY_TIER,+ semanticCapabilityIsBlocked,+ semanticCapabilityLabel,+ SEMANTIC_VECTOR_CODE,+} from "./capability-tier.ts";import { chunkMarkdown } from "./chunker.ts";import { expandTextForCjkFts } from "./cjk-tokenizer.ts";import { makeProvider } from "./embeddings/provider.ts";@@ -37,6 +45,7 @@ import {pricePerMillionTokens,LOCAL_EMBEDDING_MODEL,} from "./embeddings/signature.ts";+import { resolveEventAnchor } from "./event-anchor.ts";import { extractLinks } from "./links.ts";import { extractFrontmatterRelations } from "../graph/frontmatter-relations.ts";import { loadSchemaPack, type SchemaPack } from "../brain/schema-pack.ts";@@ -119,6 +128,7 @@ interface MutableStats {relationViolations: IndexStats["relationViolations"];tierDrift: IndexStats["tierDrift"];aliasResolved: number;+ eventAnchorsPending: number;backend: IndexStats["backend"];deferredReason: IndexStats["deferredReason"];}@@ -137,6 +147,9 @@ function newStats(): MutableStats {relationViolations: [],tierDrift: [],aliasResolved: 0,+ // Counted at the end of the run, once every changed document has+ // been upserted: what is left is what this run did not reach.+ eventAnchorsPending: 0,// Resolved lazily at the end of the run, after content detection.// Defaults to the deterministic offline backend.backend: "offline",@@ -158,6 +171,7 @@ function freezeStats(s: MutableStats, durationMs: number): IndexStats {relationViolations: Object.freeze([...s.relationViolations]),tierDrift: Object.freeze([...s.tierDrift]),aliasResolved: s.aliasResolved,+ eventAnchorsPending: s.eventAnchorsPending,backend: s.backend,deferredReason: s.deferredReason,durationMs,@@ -166,21 +180,28 @@ function freezeStats(s: MutableStats, durationMs: number): IndexStats {/*** Explain why the semantic backend was not engaged, inspecting only the- * already-resolved config (never `process.env`). Used for an offline run- * so operators see whether the cause is an unset option, disabled- * semantic search, or a missing credential.+ * already-resolved config (never `process.env`).+ *+ * The sentence is the diagnostics registry's, resolved from a REGISTERED+ * code through the advisory rail (provenance-at-the-boundary, F1). It+ * used to be assembled here from four hand-written branches, which is the+ * one thing the v1.40.0 rail exists to stop.+ *+ * Two of those branches are now the shared capability tier. The third -+ * a fully configured vault that simply was not asked for embeddings this+ * run - is an INDEX state rather than a capability fault and carries its+ * own code, whose registered exit is the vector backfill verb. The+ * fourth was unreachable: the only call site passed a constant `false`+ * for `embeddingsRequested`, because the caller reaches this function+ * exclusively on the else-arm of `if (opts?.embeddings)`. The parameter+ * and the branch are gone rather than kept as a shape that could never+ * be observed.*/-function offlineDeferredReason(config: ResolvedSearchConfig, embeddingsRequested: boolean): string {- if (!config.semantic.enabled) {- return "semantic search disabled (search_semantic_enabled=false); ran offline lexical backend";- }- if (config.semantic.provider !== "local" && !config.semantic.apiKey) {- return "embedding_api_key not configured; semantic backend deferred, ran offline lexical backend";- }- if (!embeddingsRequested) {- return "embeddings not requested this run; ran offline lexical backend";- }- return "semantic backend not engaged; ran offline lexical backend";+async function offlineDeferredReason(config: ResolvedSearchConfig): Promise<string> {+ const capability = resolveSemanticCapability(config.semantic);+ return semanticCapabilityLabel(+ semanticCapabilityIsBlocked(capability) ? capability.code : SEMANTIC_VECTOR_CODE.deferred,+ );}function sha256(text: string): string {@@ -339,7 +360,7 @@ async function indexInto(// failing, so a note keeps indexing while a field silently// vanishes. Collect what it dropped; the run reports it and// nothing about the indexing changes.- const [frontmatter, , fmNotices] = parseFrontmatterTextWithNotices(content, {+ const [frontmatter, body, fmNotices] = parseFrontmatterTextWithNotices(content, {site: INDEX_FRONTMATTER_SITE,path: file.relPath,});@@ -352,6 +373,14 @@ async function indexInto(size: file.stat.size,pageType: pageTypeFromFrontmatter(frontmatter),authoredAt: authoredAtFromFrontmatter(frontmatter),+ // The event anchor is materialised HERE, on the same pass that+ // already holds the parsed frontmatter and the body, so the+ // query side never re-scans a candidate's body. It is a pure+ // function of this file's content, which is exactly what makes+ // it safe to store behind the two content-identity fastpaths+ // above: unchanged content can only ever resolve to the same+ // anchor, so declining to recompute it cannot stale it.+ eventAnchor: resolveEventAnchor(frontmatter, body),});// Framework-kind files feed the tier-guard post-pass: keep the// parsed frontmatter of this run's changed docs that declare a@@ -507,21 +536,28 @@ async function indexInto(// semantic backend is genuinely active. Otherwise the run is offline// and we record why the semantic backend was deferred.if (opts?.embeddings) {- await populateEmbeddings(- store,- config,- stats,- opts?.forceCost === true,- opts?.safeguard,- opts?.signal,- );+ await runEmbeddingPhase(store, config, stats, {+ forceCost: opts?.forceCost === true,+ ...(opts?.safeguard !== undefined ? { safeguard: opts.safeguard } : {}),+ ...(opts?.signal !== undefined ? { signal: opts.signal } : {}),+ });stats.backend = "semantic";stats.deferredReason = null;} else {stats.backend = "offline";- stats.deferredReason = offlineDeferredReason(config, false);+ stats.deferredReason = await offlineDeferredReason(config);}+ // Documents this run left with no event anchor RESOLVED - not+ // documents that declare no date. A schema bump migrates in place+ // and reindexes nothing, and both fastpaths above correctly decline+ // to recompute an anchor for content that did not change, so a+ // document carried over from a pre-anchor binary is never examined+ // by any number of index runs. Reported here because this run is the+ // surface an upgrading operator reaches first, and because a hard+ // `since` / `until` filter now depends on the answer.+ stats.eventAnchorsPending = store.countUnexaminedEventAnchors();+const now = new Date().toISOString();store.setState(LAST_INDEXED_AT_STATE_KEY, now);// The SAME instant on a forced run: their equality is the@@ -581,14 +617,46 @@ function activeEmbeddingSignature(return embeddingSignature({ provider, model, dimension });}-async function populateEmbeddings(+/**+ * The two counters the vector phase writes back. Declared as its own+ * shape so the phase can serve the index run's `MutableStats` and the+ * standalone vector backfill without either knowing about the other;+ * `MutableStats` satisfies it structurally.+ */+export interface EmbeddingPhaseTally {+ embeddingsComputed: number;+ embeddingsRetries: number;+}++export interface EmbeddingPhaseOptions {+ /** Bypass the configured spend ceiling for this run. */+ readonly forceCost?: boolean;+ readonly safeguard?: import("../brain/safeguard.ts").Safeguard;+ readonly signal?: AbortSignal;+}++/**+ * Compute and store a vector for every indexed chunk that has none.+ *+ * Exported (provenance-at-the-boundary, F) because it is the whole of+ * `o2b search vector-backfill --apply`: the population step is already+ * resumable by construction - the anti-join finds exactly the chunks+ * still missing a row and `vecUpsert` commits per chunk - so the backfill+ * verb is this phase run on its own, not a second implementation of it.+ *+ * Every refusal here is a SearchError, because reaching this function is+ * an attempt: the capability tier is what a caller reports when it+ * declines to attempt.+ */+export async function runEmbeddingPhase(store: Store,config: ResolvedSearchConfig,- stats: MutableStats,- forceCost: boolean,- safeguard?: import("../brain/safeguard.ts").Safeguard,- signal?: AbortSignal,+ stats: EmbeddingPhaseTally,+ opts: EmbeddingPhaseOptions = {},): Promise<void> {+ const forceCost = opts.forceCost === true;+ const safeguard = opts.safeguard;+ const signal = opts.signal;if (!config.semantic.enabled) {throw new SearchError("EMBEDDING_DISABLED",@@ -633,10 +701,14 @@ async function populateEmbeddings();}const batchSize = Math.max(1, config.semantic.batchSize);- // Hand the provider a super-batch sized to fully saturate its- // internal `embedding_concurrency` semaphore. Without this multiplier- // the indexer's outer loop would serialise provider.embed() calls and- // the configured concurrency would never kick in.+ // Hand the provider a super-batch large enough to keep its internal+ // `embedding_concurrency` semaphore busy. Without this multiplier the+ // indexer's outer loop would serialise provider.embed() calls and the+ // configured concurrency would never kick in. It is an upper bound, not+ // an exact fill: when `embedding_batch_tokens` is set the provider closes+ // a batch on the token estimate as well as on the count, so a super-batch+ // may split into more - and therefore smaller - requests than+ // `embedding_concurrency`.const superBatch = batchSize * Math.max(1, config.semantic.concurrency);for (let i = 0; i < pending.length; i += superBatch) {@@ -870,12 +942,14 @@ export async function indexStatus(config: ResolvedSearchConfig): Promise<IndexStif (config.semantic.enabled && !store.vecLoaded()) {warnings.push("sqlite-vec unavailable; semantic search disabled this session");}+ // A FIFTH site that derived the credential fact for itself, in a+ // fourth spelling, and which the unit-F brief did not name. Routed+ // through the same resolver while the file is open: two of these+ // copies disagreed about whether the local provider needs a key.if (- config.semantic.enabled &&- config.semantic.provider !== "local" &&- !config.semantic.apiKey+ resolveSemanticCapability(config.semantic).tier === SEMANTIC_CAPABILITY_TIER.credentialMissing) {- warnings.push("embedding_api_key not configured; semantic search disabled");+ warnings.push(await semanticCapabilityLabel(SEMANTIC_CAPABILITY_CODE.credentialMissing));}// Instruction-prefix drift (memory-write-path-integrity B2). Stored@@ -974,6 +1048,10 @@ function isDirectoryWritable(dir: string): boolean {export async function indexCheck(config: ResolvedSearchConfig): Promise<IndexCheckReport> {const warnings: string[] = [];const fatal: string[] = [];+ // One resolver for what the operator configured, shared with the index+ // run, the semantic lane and the readiness probe+ // (provenance-at-the-boundary, F1).+ const capability = resolveSemanticCapability(config.semantic);let vaultReadable = false;try {@@ -1009,7 +1087,10 @@ export async function indexCheck(config: ResolvedSearchConfig): Promise<IndexCheconst msg = e instanceof Error ? e.message : String(e);fatal.push(`FTS5 not available: ${msg}`);}- if (config.semantic.enabled) {+ // Probed whenever semantic search is switched on at all - including+ // while the credential is still missing, because the extension is a+ // machine fact the operator can fix independently of the key.+ if (capability.tier !== SEMANTIC_CAPABILITY_TIER.disabled) {try {// eslint-disable-next-line @typescript-eslint/no-require-importsconst vec = require("sqlite-vec") as { getLoadablePath(): string };@@ -1027,14 +1108,18 @@ export async function indexCheck(config: ResolvedSearchConfig): Promise<IndexChefatal.push(`bun:sqlite open failed: ${e instanceof Error ? e.message : String(e)}`);}- const embeddingKeyResolved = !!(config.semantic.enabled && config.semantic.apiKey);- if (config.semantic.enabled && !config.semantic.apiKey) {- warnings.push("embedding_api_key not configured");+ // "The credential side of the configuration is satisfied" - which for+ // the offline local provider means it needed none. This used to read+ // `enabled && apiKey`, which reported a fully working local install as+ // key-less and then recommended setting a key it does not use.+ const embeddingKeyResolved = !semanticCapabilityIsBlocked(capability);+ if (capability.tier === SEMANTIC_CAPABILITY_TIER.credentialMissing) {+ warnings.push(await semanticCapabilityLabel(capability.code));}let providerReachable: boolean | null = null;let providerReason: string | null = null;- if (config.semantic.enabled && embeddingKeyResolved) {+ if (embeddingKeyResolved) {try {const provider = makeProvider(config.semantic);const probe = await withTimeout(provider.ping(), 5_000);diff --git a/src/core/search/links.ts b/src/core/search/links.tsindex 40e5b4fb..49cce7c3 100644--- a/src/core/search/links.ts+++ b/src/core/search/links.ts@@ -21,6 +21,7 @@* not cross-resolved, mirroring the existing per-chunk extraction unit.*/+import { hasUriScheme } from "../path-safety.ts";import { WIKILINK_ALIAS_RE } from "../brain/wikilink.ts";export type LinkType = "wikilink" | "markdown_link" | "tag";@@ -59,9 +60,8 @@ function stripCode(text: string): string {return out;}-function isUrl(target: string): boolean {- return /^[a-z][a-z0-9+.-]*:/i.test(target) || target.startsWith("//");-}+/** An external address rather than a note path - see {@link hasUriScheme}. */+const isUrl = hasUriScheme;function isMailto(target: string): boolean {return target.toLowerCase().startsWith("mailto:");diff --git a/src/core/search/pipeline/candidate-signals.ts b/src/core/search/pipeline/candidate-signals.tsindex 97c6bb48..bf6dec6b 100644--- a/src/core/search/pipeline/candidate-signals.ts+++ b/src/core/search/pipeline/candidate-signals.ts@@ -258,6 +258,12 @@ function collectReuseRates(input: CandidateSignalsInput): ReadonlyMap<number, nu* judges a note by when it is about rather than by when it was last* written. Candidates with no declaration are omitted and the ranker falls* back to their mtime.+ *+ * "Declared" here means whatever the shared event-time resolver counts as+ * a declaration: validity frontmatter first, then the event anchor the+ * indexer materialised from the note's `created_at` / `date` frontmatter+ * or its body (t_ac1c4176). This function does not resolve any of that+ * itself - one definition, one resolver, every time filter agreeing.*/function collectDeclaredEventTimes(input: CandidateSignalsInput,diff --git a/src/core/search/pipeline/event-time.ts b/src/core/search/pipeline/event-time.tsindex e4b45bec..667830ba 100644--- a/src/core/search/pipeline/event-time.ts+++ b/src/core/search/pipeline/event-time.ts@@ -7,8 +7,25 @@* A document declaring `valid_from` / `valid_until` is judged by* validity-window OVERLAP; storage mtime is the fallback, never the* authority, when explicit event time exists.+ *+ * Provenance at the boundary (t_ac1c4176) widened what "declared" means:+ * a note may state when it is about in its BODY, and the indexer+ * materialises that as the document's EVENT ANCHOR (see+ * `event-anchor.ts`). Frontmatter validity still wins - it is read live+ * here and is not re-implemented - and the stored anchor is consulted+ * only for a page that declares no validity window at all. A page that+ * declares nothing anywhere still falls back to mtime, byte-identically+ * to before the anchor existed.+ *+ * Only the rungs `anchorIsEventTime` admits are consulted. The anchor+ * also records a note's `created_at` / `date` frontmatter, which every+ * Brain writer stamps and which describes when the file was authored;+ * judging a document by it here would remove it from a hard `since` /+ * `until` filter it was in before this release, changing what a+ * time-ranged query means. That rung stays recorded and unconsulted.*/+import { anchorIsEventTime, type EventAnchor, type EventAnchorSource } from "../event-anchor.ts";import { readCachedFrontmatter, type FrontmatterCache } from "../result-filters.ts";import { eventTimeInRange, parseValidityWindow, type ValidityWindow } from "../validity.ts";import type { HydratedChunk, KeywordHit, SemanticHit } from "../store.ts";@@ -22,31 +39,85 @@ export interface EventTimeResolver {* declares none: validity start, else validity end.*/readonly declaredEventTimeMs: (path: string) => number | null;+ /**+ * The anchor rung that supplied this path's window, or null when the+ * page's own frontmatter supplied it and nothing was inferred. A+ * caller that REMOVES a candidate uses this to say so.+ */+ readonly inferredEventTimeSourceFor: (path: string) => EventAnchorSource | null;}+/** The materialised event anchor of a path, or null when it has none. */+export type EventAnchorLookup = (path: string) => EventAnchor | null;++/** One path's resolved event-time window and where it came from. */+interface ResolvedEventTime {+ readonly window: ValidityWindow | null;+ readonly anchorSource: EventAnchorSource | null;+}++/** The answer for a page that states nothing this layer can use. */+const NO_EVENT_TIME: ResolvedEventTime = Object.freeze({ window: null, anchorSource: null });+export function createEventTimeResolver(vault: string,frontmatterCache: FrontmatterCache,+ eventAnchorFor: EventAnchorLookup,): EventTimeResolver {- const windowCache = new Map<string, ValidityWindow | null>();- const validityWindowFor = (path: string): ValidityWindow | null => {- if (windowCache.has(path)) return windowCache.get(path) ?? null;- let window: ValidityWindow | null = null;+ const cache = new Map<string, ResolvedEventTime>();+ const resolve = (path: string): ResolvedEventTime => {+ const cached = cache.get(path);+ if (cached !== undefined) return cached;+ let declared: ValidityWindow | null = null;try {const meta = readCachedFrontmatter(frontmatterCache, vault, path);- window = parseValidityWindow(meta as Record<string, unknown>);+ declared = parseValidityWindow(meta as Record<string, unknown>);} catch {- window = null;+ declared = null;}- windowCache.set(path, window);- return window;+ // No validity fields at all is an ABSENCE, so the note's other+ // statements about its own event time get their turn. The anchor+ // lookup sits OUTSIDE the catch above on purpose: that catch answers+ // "the page could not be examined" with a fallback to mtime, and an+ // index row this binary cannot interpret is a different failure that+ // must reach the caller. Answering it from here would report "no+ // declared event time" for a document whose declaration was simply+ // unreadable - and, because the result is memoised, would do so for+ // every candidate in the call.+ const resolved =+ declared !== null+ ? { window: declared, anchorSource: null }+ : anchorEventTime(eventAnchorFor(path));+ cache.set(path, resolved);+ return resolved;};+ const validityWindowFor = (path: string): ValidityWindow | null => resolve(path).window;const declaredEventTimeMs = (path: string): number | null => {const window = validityWindowFor(path);if (window === null || window.invalid) return null;return window.validFromMs ?? window.validUntilMs;};- return { validityWindowFor, declaredEventTimeMs };+ const inferredEventTimeSourceFor = (path: string): EventAnchorSource | null =>+ resolve(path).anchorSource;+ return { validityWindowFor, declaredEventTimeMs, inferredEventTimeSourceFor };+}++/**+ * Present the materialised anchor in the shape every downstream time+ * filter already consumes, for the rungs that ARE event time. `invalid`+ * is always false: an anchor that could not be resolved was never+ * stored, so a stored one is by construction readable.+ */+function anchorEventTime(anchor: EventAnchor | null): ResolvedEventTime {+ if (anchor === null || !anchorIsEventTime(anchor)) return NO_EVENT_TIME;+ return Object.freeze({+ window: Object.freeze({+ validFromMs: anchor.startMs,+ validUntilMs: anchor.endMs,+ invalid: false,+ }),+ anchorSource: anchor.source,+ });}export interface RangeFilterInput {@@ -55,6 +126,8 @@ export interface RangeFilterInput {readonly semanticHits: SemanticHit[];readonly timeRange: ResolvedTimeRange;readonly validityWindowFor: (path: string) => ValidityWindow | null;+ /** See {@link EventTimeResolver.inferredEventTimeSourceFor}. */+ readonly inferredEventTimeSourceFor: (path: string) => EventAnchorSource | null;}export interface RangeFilterOutcome {@@ -63,6 +136,15 @@ export interface RangeFilterOutcome {readonly warnings: string[];}+/**+ * The one place the anchor-drop line is written. Names the path and the+ * registered rung token, so the reader can tell an inferred body date+ * from a declared validity window without reading this file.+ */+function anchorDropWarning(path: string, source: EventAnchorSource): string {+ return `event-anchor: ${path} is outside the requested range by its ${source}-declared event time, not its mtime`;+}+/*** Time-aware recall (recall-trust-suite): drop out-of-range candidates* BEFORE ranking so every later phase (traversal seeds, MMR, relation@@ -70,9 +152,10 @@ export interface RangeFilterOutcome {* warns once per path and falls back to mtime.*/export function filterCandidatesInRange(input: RangeFilterInput): RangeFilterOutcome {- const { hydrated, timeRange, validityWindowFor } = input;+ const { hydrated, timeRange, validityWindowFor, inferredEventTimeSourceFor } = input;const warnings: string[] = [];const warnedInvalid = new Set<string>();+ const warnedDropped = new Set<string>();const inRange = (chunkId: number): boolean => {const h = hydrated.get(chunkId);if (h === undefined) return false;@@ -81,7 +164,19 @@ export function filterCandidatesInRange(input: RangeFilterInput): RangeFilterOutwarnedInvalid.add(h.path);warnings.push(`validity: unparseable valid_from/valid_until in ${h.path}; using mtime`);}- return eventTimeInRange(window, h.mtime, timeRange);+ const kept = eventTimeInRange(window, h.mtime, timeRange);+ // A candidate removed on the strength of a date this tool INFERRED+ // rather than one the page declared in frontmatter is the one case+ // where the filter's answer differs from what the operator wrote+ // down. It is never silent: the path and the rung are named.+ if (!kept && !warnedDropped.has(h.path)) {+ const source = inferredEventTimeSourceFor(h.path);+ if (source !== null) {+ warnedDropped.add(h.path);+ warnings.push(anchorDropWarning(h.path, source));+ }+ }+ return kept;};return {keywordHits: input.keywordHits.filter((h) => inRange(h.chunkId)),diff --git a/src/core/search/query-expansion.ts b/src/core/search/query-expansion.tsindex 0a3563db..f3b593ab 100644--- a/src/core/search/query-expansion.ts+++ b/src/core/search/query-expansion.ts@@ -24,6 +24,7 @@*/import { listEntities } from "../brain/entities/registry.ts";+import { ENTITY_STATUS_SCOPE, entityStatusInScope } from "../brain/entities/status-scope.ts";import { tokenizeForExpansion } from "./synonyms.ts";import type { StructuredRecallQueryDocument } from "./types.ts";@@ -98,13 +99,22 @@ export function expandQuery(* Registry entities whose name (or alias) shares a token with the* query, sorted by name for determinism. Fail-soft: a vault without a* registry simply matches nothing.+ *+ * Anchoring is a CANONICAL-scope read: only a record that may hold an+ * identity may lend its name to a query. The scope comes from the shared+ * predicate rather than from an explicit `{ status }` ask, because the+ * explicit ask is the registry's deliberate bypass of that predicate -+ * it exists so an operator can look at what is quarantined, and a+ * machine read path taking it would re-implement the filter it is meant+ * to defer to.*/function matchEntities(vault: string, queryTokens: ReadonlyArray<string>): string[] {if (queryTokens.length === 0) return [];const wanted = new Set(queryTokens);let names: string[];try {- names = listEntities(vault, { status: "active" })+ names = listEntities(vault)+ .filter((entity) => entityStatusInScope(entity.status, ENTITY_STATUS_SCOPE.canonical)).filter((entity) =>[entity.name, ...entity.aliases].some((label) =>tokenizeForExpansion(label).some((token) => wanted.has(token)),diff --git a/src/core/search/schema.ts b/src/core/search/schema.tsindex 8c1c0928..7a999e3f 100644--- a/src/core/search/schema.ts+++ b/src/core/search/schema.ts@@ -16,7 +16,7 @@ import { SearchError } from "./types.ts";* this raises `SCHEMA_MISMATCH` on open — the operator must reindex* with a newer binary.*/-export const LATEST_SCHEMA_VERSION = 10;+export const LATEST_SCHEMA_VERSION = 11;/*** The wikilink-resolution basename of a stored document path: the final@@ -490,6 +490,56 @@ export const MIGRATIONS: ReadonlyArray<Migration> = Object.freeze([}},},+ {+ // v11 (provenance at the boundary, t_ac1c4176) - the materialised+ // EVENT ANCHOR: the interval a note is ABOUT, resolved from its own+ // frontmatter or body by `event-anchor.ts`, plus the registered token+ // naming which rung of that ladder produced it. Materialised here so+ // the query side never re-scans a candidate's body.+ //+ // Naming: `event_anchor_*`, deliberately NOT `created_at`-adjacent.+ // `documents.created_at` on this same table is the row-INSERT+ // timestamp and has nothing to do with the `created_at` FRONTMATTER+ // key that is one rung of the anchor ladder; a column named after+ // either would be read as the other. The `_ms` suffix is likewise+ // load-bearing: `mtime` and `authored_at` beside it are unix SECONDS,+ // and a day-end bound truncated to whole seconds would move every+ // interval comparison by almost a second.+ //+ // Additive and reindex-safe: existing rows default to NULL on all+ // three until a reindex repopulates them, so a vault whose notes+ // declare no dates ranks byte-identically.+ //+ // `event_anchor_examined` is the fourth column and the one that+ // makes the other three honest. This migration runs IN PLACE - a+ // schema bump reindexes nothing, and the indexer resolves an anchor+ // only below its two content-identity fastpaths, so a document+ // carried over from v10 never has one computed: its content does not+ // change, so it is never re-read. Without this marker that row's+ // three NULLs are indistinguishable from a note that was examined+ // and declares no date, and the column now gates a hard `since` /+ // `until` filter. `0` means "no anchor-aware binary has ever looked+ // at this document", which is what every pre-existing row is and+ // what `o2b search event-anchor-backfill` finds.+ version: 11,+ up(db) {+ const docCols = db.query<{ name: string }, []>("PRAGMA table_info(documents)").all();+ if (!docCols.some((c) => c.name === "event_anchor_start_ms")) {+ db.exec("ALTER TABLE documents ADD COLUMN event_anchor_start_ms INTEGER");+ }+ if (!docCols.some((c) => c.name === "event_anchor_end_ms")) {+ db.exec("ALTER TABLE documents ADD COLUMN event_anchor_end_ms INTEGER");+ }+ if (!docCols.some((c) => c.name === "event_anchor_source")) {+ db.exec("ALTER TABLE documents ADD COLUMN event_anchor_source TEXT");+ }+ if (!docCols.some((c) => c.name === "event_anchor_examined")) {+ db.exec(+ "ALTER TABLE documents ADD COLUMN event_anchor_examined INTEGER NOT NULL DEFAULT 0",+ );+ }+ },+ },]);/** Read the current schema version from `index_state`. Returns 0 if not yet recorded. */diff --git a/src/core/search/search.ts b/src/core/search/search.tsindex 8327decb..2e0dcf66 100644--- a/src/core/search/search.ts+++ b/src/core/search/search.ts@@ -162,7 +162,9 @@ export async function search(// One event-time resolver for the whole call: the targeted-retry// coverage gate, the hard time filter, the temporal bridge and the// ranker's declared-event-time map all judge a page the same way.- const eventTime = createEventTimeResolver(effectiveConfig.vault, frontmatterCache);+ const eventTime = createEventTimeResolver(effectiveConfig.vault, frontmatterCache, (path) =>+ store.eventAnchorForPath(path),+ );// Second-pass recall (t_ef92dfdc, t_8eb5ca32): evidence-pack mode only,// at most one retry, merged into the pool before ranking.@@ -197,6 +199,7 @@ export async function search(semanticHits,timeRange,validityWindowFor: eventTime.validityWindowFor,+ inferredEventTimeSourceFor: eventTime.inferredEventTimeSourceFor,});keywordHits = inRange.keywordHits;semanticHits = inRange.semanticHits;diff --git a/src/core/search/semantic-phase.ts b/src/core/search/semantic-phase.tsindex 89d89ae8..509fec83 100644--- a/src/core/search/semantic-phase.ts+++ b/src/core/search/semantic-phase.ts@@ -5,11 +5,18 @@* to keyword-only with warnings when the lane cannot run).*/+import {+ resolveSemanticCapability,+ SEMANTIC_CAPABILITY_TIER,+ semanticCapabilityIsBlocked,+ semanticCapabilityLabel,+} from "./capability-tier.ts";import { classifyEmbeddingError } from "./embeddings/openai-compat.ts";import { makeProvider } from "./embeddings/provider.ts";import { Store } from "./store.ts";import { EMBEDDING_QUOTA_MESSAGE, SearchError } from "./types.ts";-import type { ResolvedSearchConfig, SearchOptions } from "./types.ts";+import type { SemanticCapability, SemanticCapabilityTier } from "./capability-tier.ts";+import type { ResolvedSearchConfig, SearchErrorCode, SearchOptions } from "./types.ts";export interface SemanticPolicy {/** caller asked for semantic on or off (true), or accepted the default (false). */@@ -44,6 +51,41 @@ export function semanticPoolSize(limit: number): number {return Math.max(limit * POOL_OVERFETCH, POOL_FLOOR);}+/** The rungs of the capability ladder that BLOCK; `configured` is not one. */+type BlockedCapabilityTier = Exclude<+ SemanticCapabilityTier,+ typeof SEMANTIC_CAPABILITY_TIER.configured+>;++/**+ * The typed failure an EXPLICIT semantic request reports per blocked rung.+ *+ * One entry per rung, so adding a rung to the ladder is a type error here+ * rather than a request that silently degrades. `credential-missing` keeps+ * `EMBEDDING_KEY_MISSING`, the code that arm has always raised; `disabled`+ * raises `EMBEDDING_DISABLED`, the code the null provider already raises+ * for the same configuration when the indexer meets it, so one condition+ * has one code across the tree.+ */+const BLOCKED_TIER_ERROR_CODE: Readonly<Record<BlockedCapabilityTier, SearchErrorCode>> =+ Object.freeze({+ [SEMANTIC_CAPABILITY_TIER.disabled]: "EMBEDDING_DISABLED",+ [SEMANTIC_CAPABILITY_TIER.credentialMissing]: "EMBEDDING_KEY_MISSING",+ });++/**+ * {@link semanticCapabilityIsBlocked} as a TYPE predicate. The shared+ * resolver exports it as a plain boolean and lives in a module this lane+ * does not own, so the narrowing lives here - which lets the table above be+ * total over exactly the blocked rungs, with no cast and no unreachable+ * default arm.+ */+function isBlockedCapability(+ capability: SemanticCapability,+): capability is SemanticCapability & { readonly tier: BlockedCapabilityTier } {+ return semanticCapabilityIsBlocked(capability);+}+interface SemanticPhaseOutcome {readonly attempted: boolean;readonly hits: ReturnType<Store["semanticTopK"]>;@@ -74,18 +116,26 @@ export async function runSemanticPhase(warnings.push("sqlite-vec unavailable, semantic disabled this session");return { attempted: false, hits: [], warnings };}- if (!config.semantic.enabled) {- // Defensive: should be handled at policy layer, but in case caller- // forced wantSemantic without enabling, treat as implicit warning.- warnings.push("semantic not enabled in config; using keyword-only");- return { attempted: false, hits: [], warnings };- }- // The offline local provider needs no key; every remote provider does.- if (config.semantic.provider !== "local" && !config.semantic.apiKey) {- if (opts.explicit) {- throw new SearchError("EMBEDDING_KEY_MISSING", "embedding key not configured");- }- warnings.push("embedding key not configured; semantic disabled");+ // What the OPERATOR CONFIGURED, from the one shared resolver+ // (provenance-at-the-boundary, F1). The two runtime guards above stay+ // where they are: an empty vector table and a missing extension are+ // facts about this index and this machine, not about the configuration.+ //+ // The explicit arm is the one that ATTEMPTED something and could not,+ // so it keeps reporting the typed `SearchError`; the implicit arm+ // refused before attempting and reports the capability. Neither+ // condition is reported both ways.+ //+ // EVERY blocked rung throws on the explicit arm, not just the missing+ // credential: a caller who asked for semantic recall in so many words and+ // was served lexical results with exit 0 was told the search succeeded.+ // Both arms name the condition with the SAME registry sentence, so the+ // warning and the error message cannot drift apart.+ const capability = resolveSemanticCapability(config.semantic);+ if (isBlockedCapability(capability)) {+ const label = await semanticCapabilityLabel(capability.code);+ if (opts.explicit) throw new SearchError(BLOCKED_TIER_ERROR_CODE[capability.tier], label);+ warnings.push(label);return { attempted: false, hits: [], warnings };}diff --git a/src/core/search/store.ts b/src/core/search/store.tsindex bdecbe0f..34ea3036 100644--- a/src/core/search/store.ts+++ b/src/core/search/store.ts@@ -31,6 +31,7 @@ import {import { SearchError } from "./types.ts";import type { ResolvedSearchConfig } from "./types.ts";+import type { EventAnchor } from "./event-anchor.ts";import { readSchemaVersion } from "./schema.ts";import * as aliases from "./store/aliases.ts";@@ -84,6 +85,7 @@ export {export { normalizeAlias } from "./store/aliases.ts";export type { DocumentInput, DocumentSummary } from "./store/documents.ts";+export type { EventAnchor } from "./event-anchor.ts";export type { ChunkInput, ChunkRow, HydratedChunk } from "./store/chunks.ts";export type { KeywordHit } from "./store/keyword.ts";export type { LinkInput, LinkResolutionCounts } from "./store/links.ts";@@ -307,6 +309,38 @@ export class Store {return documents.getDocumentIdByPath(this.db, path);}+ /**+ * The materialised event anchor of one path (v11), or null when the+ * document is absent or declared no readable date. The query-side+ * event-time resolver consults this instead of re-scanning the note's+ * body on every query.+ */+ eventAnchorForPath(path: string): EventAnchor | null {+ return documents.eventAnchorForPath(this.db, path);+ }++ /**+ * Vault-relative paths of documents no anchor-aware binary has ever+ * examined (v11) - the rows a pre-anchor index carried across the+ * migration, which are NOT the same as documents that declare no date.+ */+ unexaminedEventAnchorPaths(): string[] {+ return documents.unexaminedEventAnchorPaths(this.db);+ }++ /** How many documents no anchor-aware binary has ever examined (v11). */+ countUnexaminedEventAnchors(): number {+ return documents.countUnexaminedEventAnchors(this.db);+ }++ /**+ * Record one already-indexed document's event anchor and mark it+ * examined, touching nothing else on the row. The backfill's only write.+ */+ setEventAnchor(path: string, anchor: EventAnchor | null): void {+ documents.setEventAnchor(this.db, path, anchor);+ }+upsertDocument(doc: documents.DocumentInput): number {return documents.upsertDocument(this.db, doc);}diff --git a/src/core/search/store/documents.ts b/src/core/search/store/documents.tsindex b6bd962c..a946b3ad 100644--- a/src/core/search/store/documents.ts+++ b/src/core/search/store/documents.ts@@ -6,11 +6,21 @@import { Database } from "bun:sqlite";+import { isEventAnchorSource, type EventAnchor } from "../event-anchor.ts";import { documentBasename } from "../schema.ts";import { SearchError } from "../types.ts";import { nowIso } from "./sql.ts";import { purgeVecRowsForDocument } from "./vectors.ts";+/**+ * `documents.event_anchor_examined`: whether an anchor-aware binary has+ * ever resolved this document's event anchor. SQLite has no boolean, so+ * the two values are named once here rather than written as bare 0/1 at+ * each of the four sites that read or write them.+ */+const EXAMINED_YES = 1;+const EXAMINED_NO = 0;+export interface DocumentInput {readonly path: string; // vault-relative POSIXreadonly title: string | null;@@ -30,6 +40,24 @@ export interface DocumentInput {* turn instant, which then ranks byte-identically to pre-feature.*/readonly authoredAt?: number | null;+ /**+ * The interval this note is ABOUT, resolved from its own frontmatter or+ * body by `resolveEventAnchor` (provenance at the boundary, v11), with+ * the registered token naming which rung produced it.+ *+ * The three states are distinct and all three are recorded:+ *+ * - an `EventAnchor` - the caller resolved one;+ * - `null` - the caller resolved the anchor and the note declares no+ * readable date, so it keeps ranking on storage mtime exactly as+ * before the anchor existed;+ * - ABSENT - the caller did not resolve an anchor at all. The row is+ * then marked unexamined and `o2b search event-anchor-backfill`+ * finds it. Omitting the field is not the same statement as+ * passing `null`, and the column that separates them exists+ * because a hard time filter now depends on the difference.+ */+ readonly eventAnchor?: EventAnchor | null;}export interface DocumentSummary {@@ -86,13 +114,19 @@ export function upsertDocument(db: Database, doc: DocumentInput): number {number,string | null,number | null,+ number | null,+ number | null,+ string | null,+ number,string,string,string,]>(- "INSERT INTO documents(path, basename, title, content_hash, mtime, size, page_type, authored_at, created_at, updated_at, indexed_at) " +- "VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) " ++ "INSERT INTO documents(path, basename, title, content_hash, mtime, size, page_type, authored_at, " ++ " event_anchor_start_ms, event_anchor_end_ms, event_anchor_source, event_anchor_examined, " ++ " created_at, updated_at, indexed_at) " ++ "VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) " +"ON CONFLICT(path) DO UPDATE SET " +" basename = excluded.basename, " +" title = excluded.title, " +@@ -101,6 +135,10 @@ export function upsertDocument(db: Database, doc: DocumentInput): number {" size = excluded.size, " +" page_type = excluded.page_type, " +" authored_at = excluded.authored_at, " ++ " event_anchor_start_ms = excluded.event_anchor_start_ms, " ++ " event_anchor_end_ms = excluded.event_anchor_end_ms, " ++ " event_anchor_source = excluded.event_anchor_source, " ++ " event_anchor_examined = excluded.event_anchor_examined, " +" updated_at = excluded.updated_at, " +" indexed_at = excluded.indexed_at " +"RETURNING id",@@ -114,6 +152,12 @@ export function upsertDocument(db: Database, doc: DocumentInput): number {doc.size,doc.pageType ?? null,doc.authoredAt ?? null,+ doc.eventAnchor?.startMs ?? null,+ doc.eventAnchor?.endMs ?? null,+ doc.eventAnchor?.source ?? null,+ // Examined iff the caller stated an answer. `null` IS an answer+ // ("this note declares nothing"); leaving the field out is not.+ doc.eventAnchor === undefined ? EXAMINED_NO : EXAMINED_YES,now,now,now,@@ -124,6 +168,112 @@ export function upsertDocument(db: Database, doc: DocumentInput): number {return row.id;}+/**+ * The materialised event anchor of one path, or null when the document is+ * absent from the index or declared no readable date (v11).+ *+ * A row carrying an anchor whose source token is not in the registered+ * vocabulary, or whose bounds are both NULL beside a non-NULL source, is+ * an index this binary cannot interpret - it raises rather than returning+ * null, because "no anchor" and "an anchor I cannot read" are different+ * answers and the caller must not receive the first when the truth is the+ * second.+ *+ * A row that was never EXAMINED also returns null, and deliberately does+ * not raise: it holds exactly the answer the previous binary held, so+ * the query degrades to storage mtime precisely as it did before this+ * feature, rather than failing every query on an index that predates it.+ * That state is not left silent either - it is counted by+ * {@link countUnexaminedEventAnchors}, reported by every index run, and+ * closed by `o2b search event-anchor-backfill --apply`.+ */+export function eventAnchorForPath(db: Database, path: string): EventAnchor | null {+ const row = db+ .query<+ {+ event_anchor_start_ms: number | null;+ event_anchor_end_ms: number | null;+ event_anchor_source: string | null;+ },+ [string]+ >(+ "SELECT event_anchor_start_ms, event_anchor_end_ms, event_anchor_source " ++ "FROM documents WHERE path = ?",+ )+ .get(path);+ if (!row || row.event_anchor_source === null) return null;+ if (!isEventAnchorSource(row.event_anchor_source)) {+ throw new SearchError(+ "INDEX_UNREADABLE",+ `documents.event_anchor_source for '${path}' is not a registered anchor source: '${row.event_anchor_source}'`,+ );+ }+ if (row.event_anchor_start_ms === null && row.event_anchor_end_ms === null) {+ throw new SearchError(+ "INDEX_UNREADABLE",+ `documents row for '${path}' names an anchor source with no bounds on either side`,+ );+ }+ return Object.freeze({+ startMs: row.event_anchor_start_ms,+ endMs: row.event_anchor_end_ms,+ source: row.event_anchor_source,+ });+}++/**+ * Vault-relative paths of the documents no anchor-aware binary has ever+ * examined (v11), in stable path order.+ *+ * These are the rows a v10 index carried across the migration: the+ * columns exist, the fastpaths correctly decline to recompute anything+ * for content that did not change, and so nothing ever populates them.+ * They are NOT documents that declare no date - that is the state this+ * list exists to be distinguishable from.+ */+export function unexaminedEventAnchorPaths(db: Database): string[] {+ return db+ .query<{ path: string }, [number]>(+ "SELECT path FROM documents WHERE event_anchor_examined = ? ORDER BY path",+ )+ .all(EXAMINED_NO)+ .map((r) => r.path);+}++/** How many documents no anchor-aware binary has ever examined (v11). */+export function countUnexaminedEventAnchors(db: Database): number {+ return (+ db+ .query<{ c: number }, [number]>(+ "SELECT count(*) AS c FROM documents WHERE event_anchor_examined = ?",+ )+ .get(EXAMINED_NO)?.c ?? 0+ );+}++/**+ * Record the event anchor of an already-indexed document and mark it+ * examined, touching nothing else on the row - no chunks, no links, no+ * stat fingerprint. This is the backfill's only write.+ *+ * `null` stores the verdict "this note declares no readable date", which+ * is a result and not an absence of one.+ */+export function setEventAnchor(db: Database, path: string, anchor: EventAnchor | null): void {+ db.run(+ "UPDATE documents SET event_anchor_start_ms = ?, event_anchor_end_ms = ?, " ++ "event_anchor_source = ?, event_anchor_examined = ?, updated_at = ? WHERE path = ?",+ [+ anchor?.startMs ?? null,+ anchor?.endMs ?? null,+ anchor?.source ?? null,+ EXAMINED_YES,+ nowIso(),+ path,+ ],+ );+}+/*** Delete a document and everything that hangs off it. The vec rows* are removed first because the FK cascade does not reach thediff --git a/src/core/search/types.ts b/src/core/search/types.tsindex f6f03b6a..0486bcfc 100644--- a/src/core/search/types.ts+++ b/src/core/search/types.ts@@ -146,6 +146,19 @@ export interface IndexStats {* unconditionally) for a full sweep.*/readonly frontmatterNotices: ReadonlyArray<DegradationNotice>;+ /**+ * Indexed documents whose EVENT ANCHOR has never been resolved by an+ * anchor-aware binary (v11) - the rows a pre-anchor index carried+ * across the in-place schema migration.+ *+ * Not the same as "documents that declare no date": that verdict is+ * recorded, this is its absence. It cannot fall to zero on its own,+ * because both content-identity fastpaths correctly decline to+ * recompute an anchor for content that did not change and those+ * documents' content never changes. `o2b search event-anchor-backfill+ * --apply` is what closes it.+ */+ readonly eventAnchorsPending: number;/*** Typed edges blocked by the schema pack's `link_constraints` during* this run's materialization post-pass@@ -728,6 +741,20 @@ export interface ResolvedEmbeddingConfig {readonly timeoutMs: number;readonly concurrency: number;readonly batchSize: number;+ /**+ * Per-request token budget for one embedding batch+ * (provenance-at-the-boundary E1). When set, a batch closes on whichever+ * cap fills first - {@link batchSize} items or this many estimated tokens+ * - so a run of long chunks cannot assemble a request that exceeds the+ * provider's per-request token ceiling. The estimate is `estimateTokens`+ * from `embeddings/signature.ts`, the same estimator the indexer's cost+ * gate uses, applied to the text as it will be sent (instruction prefix+ * included). Optional: when the `embedding_batch_tokens` key is absent the+ * field is absent too and batching is byte-identical to the fixed+ * `batchSize` stride. A single text whose own estimate exceeds the budget+ * is sent alone rather than dropped or split.+ */+ readonly batchTokens?: number;/*** Per-batch transient-retry budget (attempts, not extra retries) for* 429 / 5xx / network errors. Default 6, raised from the former hardcodeddiff --git a/src/core/search/vector-backfill.ts b/src/core/search/vector-backfill.tsnew file mode 100644index 00000000..c250d5d8--- /dev/null+++ b/src/core/search/vector-backfill.ts@@ -0,0 +1,124 @@+/**+ * Vector-only backfill (provenance-at-the-boundary, unit F).+ *+ * A chunk without a vector is NOT a broken row. Vectors live in a `vec0`+ * virtual table that cannot hold a null vector, so the absence of a+ * vector is simply a `chunks` row with no `embeddings` row - a+ * first-class state the store already queries (`findChunksWithoutEmbeddings`)+ * and that the population phase already resumes from, because the+ * anti-join finds exactly what is still missing and `vecUpsert` commits+ * per chunk. Nothing here introduces a schema change, a deferred provider+ * class, or a null-vector concept; the state already existed and had no+ * operator-facing verb.+ *+ * That verb is what this module plans. It is the vector phase run ALONE:+ * no vault walk, no re-chunking, no frontmatter pass. An operator who+ * indexed offline (the default - no automatic index call requests+ * embeddings) and later configures a provider closes the gap with it+ * instead of re-indexing the whole vault.+ *+ * Contract, taken verbatim from `brain/authored-at-backfill.ts`:+ * - Dry-run by DEFAULT. {@link planVectorBackfill} without `apply`+ * opens the index read-only, counts, and writes nothing at all.+ * - `apply` is the only mutating path, and it is the only path that+ * contacts a provider.+ * - Idempotent: a re-run over a fully vectorised index finds nothing+ * pending and does nothing.+ *+ * ## Which question each refusal answers+ *+ * The dry run does not attempt anything, so it REPORTS the capability+ * tier - what the operator configured - and still counts the pending+ * work, because how much work is waiting is true regardless of whether+ * the credential is in place yet. The apply path does attempt, so its+ * refusals are the typed `SearchError`s the embedding phase already+ * raises. One condition is never reported both ways.+ */++import { resolveSemanticCapability, type SemanticCapability } from "./capability-tier.ts";+import { estimateCostUsd, estimateTokens, pricePerMillionTokens } from "./embeddings/signature.ts";+import { runEmbeddingPhase } from "./indexer.ts";+import { Store } from "./store.ts";+import type { ResolvedSearchConfig } from "./types.ts";++export interface VectorBackfillOptions {+ /** When true, compute and store the missing vectors. Default false. */+ readonly apply?: boolean;+ /** Bypass the configured spend ceiling for this run. */+ readonly forceCost?: boolean;+ readonly safeguard?: import("../brain/safeguard.ts").Safeguard;+ readonly signal?: AbortSignal;+}++export interface VectorBackfillResult {+ /** Whether the run mutated the index (`false` for a dry run). */+ readonly applied: boolean;+ /** What the operator configured, from the shared tier resolver. */+ readonly capability: SemanticCapability;+ /** Chunks in the index. */+ readonly chunksTotal: number;+ /** Chunks that had no vector when the run started. */+ readonly pending: number;+ /** Vectors written by this run (0 on a dry run). */+ readonly embedded: number;+ /** Provider retries this run consumed (0 on a dry run). */+ readonly retries: number;+ /**+ * Estimated spend for the pending set, from the estimator that already+ * governs the indexer's cost gate. `0` when the model carries no known+ * price - which is an absent price, not a free run, and the human+ * report says so rather than printing `$0.0000`.+ */+ readonly estimatedCostUsd: number;+ /** True when {@link estimatedCostUsd} could be derived at all. */+ readonly costKnown: boolean;+}++/**+ * Count the vectorless chunks and, when `apply` is set, populate them.+ *+ * Throws `SearchError("INDEX_MISSING")` when there is no index to read:+ * an absent index is a different answer from an index with no pending+ * work, and collapsing the two would report "nothing to do" for a vault+ * that has never been indexed.+ */+export async function planVectorBackfill(+ config: ResolvedSearchConfig,+ opts: VectorBackfillOptions = {},+): Promise<VectorBackfillResult> {+ const apply = opts.apply === true;+ const capability = resolveSemanticCapability(config.semantic);+ const store = await Store.open(config, { mode: apply ? "write" : "read" });+ try {+ const pendingChunks = store.findChunksWithoutEmbeddings();+ const chunksTotal = store.counts().chunks;+ const model = config.semantic.model;+ const tokens = estimateTokens(pendingChunks.map((c) => c.content));+ const estimatedCostUsd = estimateCostUsd(tokens, model);+ const tally = { embeddingsComputed: 0, embeddingsRetries: 0 };++ if (apply && pendingChunks.length > 0) {+ await runEmbeddingPhase(store, config, tally, {+ forceCost: opts.forceCost === true,+ ...(opts.safeguard !== undefined ? { safeguard: opts.safeguard } : {}),+ ...(opts.signal !== undefined ? { signal: opts.signal } : {}),+ });+ }++ return Object.freeze({+ applied: apply,+ capability,+ chunksTotal,+ pending: pendingChunks.length,+ embedded: tally.embeddingsComputed,+ retries: tally.embeddingsRetries,+ estimatedCostUsd,+ // Whether the MODEL has a price, not whether this run costs+ // anything: an empty pending set costs nothing and that is not the+ // same statement as "the price of this model is unknown".+ costKnown: pricePerMillionTokens(model) > 0,+ });+ } finally {+ await store.close();+ }+}diff --git a/src/core/surface/lexical-score.ts b/src/core/surface/lexical-score.tsindex c6c7ab70..4b3ca45f 100644--- a/src/core/surface/lexical-score.ts+++ b/src/core/surface/lexical-score.ts@@ -12,9 +12,23 @@import type { SurfaceDescriptor } from "./descriptor.ts";+/**+ * One query token a descriptor matched, with the document frequency of that+ * token over the SCORED CORPUS (not the vault). Exposed because the score+ * alone cannot say whether a match rests on a term that separates this+ * descriptor from the others or on one every descriptor carries.+ */+export interface MatchedTerm {+ readonly term: string;+ /** Descriptors in the scored corpus containing the term. Always >= 1. */+ readonly df: number;+}+export interface ScoredDescriptor {readonly descriptor: SurfaceDescriptor;readonly score: number;+ /** Query tokens this descriptor matched, in query order. Never empty. */+ readonly matched: ReadonlyArray<MatchedTerm>;}export interface LexicalScoreOptions {@@ -124,17 +138,21 @@ export function scoreDescriptors(const scored: ScoredDescriptor[] = [];for (const doc of docs) {let score = 0;+ const matched: MatchedTerm[] = [];for (const token of queryTokens) {const tf = doc.tf.get(token);if (tf === undefined) continue;const docFreq = df.get(token) ?? 0;+ matched.push({ term: token, df: docFreq });// BM25+ style idf floor: keeps a term present in every document// from zeroing out, which matters in tiny descriptor corpora.const idf = Math.log(1 + (n - docFreq + 0.5) / (docFreq + 0.5));const norm = tf + opts.k1 * (1 - opts.b + opts.b * (doc.length / avgLength));score += idf * ((tf * (opts.k1 + 1)) / norm);}- if (score > 0) scored.push({ descriptor: doc.descriptor, score });+ if (score > 0) {+ scored.push({ descriptor: doc.descriptor, score, matched: Object.freeze(matched) });+ }}scored.sort(diff --git a/src/core/surface/skill-attach.ts b/src/core/surface/skill-attach.tsindex 5abc6611..29ba43ad 100644--- a/src/core/surface/skill-attach.ts+++ b/src/core/surface/skill-attach.ts@@ -7,10 +7,23 @@* loop; the same query and skill set always produce the same block.* The char budget drops whole trailing entries - a half-rendered* skill line would read as a different skill.+ *+ * Two properties were added in t_ccb05134:+ *+ * - A discriminating-term floor. BM25 already down-weights a term the+ * whole corpus carries, but down-weighted is not dropped: a candidate+ * whose ENTIRE match is such terms still outscores nothing and still+ * gets offered. It should not be. See+ * {@link DISCRIMINATING_TERM_CORPUS_SHARE}.+ * - An offer identity. The block cites the id of the offer it renders, so+ * an agent invoking one of these skills can quote it back and the+ * invocation becomes joinable to the offer. See `./skill-offer.ts`.*/+import { isRareTerm } from "../search/coverage.ts";import { skillDescriptors } from "./descriptor.ts";-import { scoreDescriptors } from "./lexical-score.ts";+import { scoreDescriptors, type ScoredDescriptor } from "./lexical-score.ts";+import { computeSkillOfferId, SKILL_OFFER_ID_KEY, SKILL_OFFER_ID_LENGTH } from "./skill-offer.ts";import type { SkillEntry } from "./skills.ts";export interface SkillAttachItem {@@ -24,6 +37,12 @@ export interface SkillAttachment {/** Markdown block ("## Relevant skills" + bullets), "" when empty. */readonly block: string;readonly items: ReadonlyArray<SkillAttachItem>;+ /**+ * Content-addressed id of this offer, or null when nothing was offered.+ * Null is not a failure: an offer that was never made has no identity,+ * and inventing one would put an unbacked id into the join.+ */+ readonly offerId: string | null;}export interface BuildSkillAttachmentOptions {@@ -45,23 +64,65 @@ export interface BuildSkillAttachmentOptions {const DEFAULT_MAX_SKILLS = 3;const DEFAULT_MAX_CHARS = 1200;const BLOCK_HEADER = "## Relevant skills";+const OFFER_LINE_PREFIX = "Offer ";+const OFFER_LINE_SUFFIX = ` - cite it as ${SKILL_OFFER_ID_KEY} when calling get_skill.`;+/**+ * Chars the offer line costs: a fixed-width id between two literals, plus+ * the blank line separating it from the bullets. Fixed width is why the+ * budget can reserve it before the id exists, without a second pass.+ */+const OFFER_LINE_BUDGET =+ OFFER_LINE_PREFIX.length + SKILL_OFFER_ID_LENGTH + OFFER_LINE_SUFFIX.length + 2;++/**+ * Share of the descriptor corpus a query term may appear in and still+ * discriminate between its members.+ *+ * A term more than half the descriptors carry cannot separate them: nearly+ * every candidate matches it, so it is evidence for none of them in+ * particular. A candidate whose entire match rests on such terms has not+ * been shown to be relevant to this turn and is dropped rather than offered.+ *+ * The only input is corpus document frequency, so the rule holds in any+ * language and there is no vocabulary list anywhere in its derivation - the+ * standing project position, and the reason the vault-side coverage engine+ * this shares {@link isRareTerm} with carries no stopword list either.+ *+ * The one-document floor inside `isRareTerm` matters here: a single-skill+ * corpus has df = the whole corpus for every term, so the floor makes the+ * rule abstain rather than drop the only candidate on no evidence.+ */+export const DISCRIMINATING_TERM_CORPUS_SHARE = 0.5;++/**+ * Whether at least one term this candidate matched separates it from the+ * rest of the corpus.+ */+function hasDiscriminatingMatch(scored: ScoredDescriptor, corpusSize: number): boolean {+ return scored.matched.some((m) => isRareTerm(m.df, corpusSize, DISCRIMINATING_TERM_CORPUS_SHARE));+}function renderLine(item: SkillAttachItem): string {return `- ${item.name} - ${item.description} (load with get_skill: ${item.name})`;}+function emptyAttachment(): SkillAttachment {+ return Object.freeze({ block: "", items: Object.freeze([]), offerId: null });+}+export function buildSkillAttachment(opts: BuildSkillAttachmentOptions): SkillAttachment {const maxSkills = opts.maxSkills ?? DEFAULT_MAX_SKILLS;const maxChars = opts.maxChars ?? DEFAULT_MAX_CHARS;const byName = new Map(opts.skills.map((s) => [s.name, s]));- const ranked = scoreDescriptors(- opts.query,- skillDescriptors(opts.skills, opts.includeTriggers),- ).slice(0, maxSkills);+ const descriptors = skillDescriptors(opts.skills, opts.includeTriggers);+ const ranked = scoreDescriptors(opts.query, descriptors)+ // The floor filters, it never reorders: ranking stays the scorer's.+ .filter((scored) => hasDiscriminatingMatch(scored, descriptors.length))+ .slice(0, maxSkills);const items: SkillAttachItem[] = [];const lines: string[] = [];- let used = BLOCK_HEADER.length + 2; // header + blank line+ let used = BLOCK_HEADER.length + 2 + OFFER_LINE_BUDGET; // header + blank line + offer linefor (const { descriptor, score } of ranked) {const skill = byName.get(descriptor.name);if (skill === undefined) continue;@@ -78,9 +139,14 @@ export function buildSkillAttachment(opts: BuildSkillAttachmentOptions): SkillAtlines.push(line);}- if (items.length === 0) return Object.freeze({ block: "", items: Object.freeze([]) });+ if (items.length === 0) return emptyAttachment();+ const offerId = computeSkillOfferId(+ opts.query,+ items.map((item) => ({ name: item.name, path: item.path })),+ );return Object.freeze({- block: `${BLOCK_HEADER}\n\n${lines.join("\n")}`,+ block: `${BLOCK_HEADER}\n\n${lines.join("\n")}\n\n${OFFER_LINE_PREFIX}${offerId}${OFFER_LINE_SUFFIX}`,items: Object.freeze(items),+ offerId,});}diff --git a/src/core/surface/skill-offer.ts b/src/core/surface/skill-offer.tsnew file mode 100644index 00000000..76a5c085--- /dev/null+++ b/src/core/surface/skill-offer.ts@@ -0,0 +1,84 @@+/**+ * Skill offer identity (t_ccb05134).+ *+ * Skill invocation was already observed - the session importer writes a+ * `skill_invoked` continuity record for every real skill fetch. What was+ * missing is the other end of the arrow: an invocation carried nothing+ * naming the offer that produced it, so the two could never be joined and+ * the efficacy chain could not advance past its first link.+ *+ * An offer is the set of skills `skills_attach` put in front of an agent for+ * one turn. Its identity is content-addressed over the offer itself - the+ * normalised turn text plus the offered (name, path) pairs in rank order -+ * through the repository's one canonical-JSON digest helper. Two+ * consequences follow, and both are the point:+ *+ * - The id is reproducible. Anyone holding the offer can recompute it, so+ * no offer store has to exist for the join to be verifiable.+ * - The id is specific. A different turn, a different offered set, or a+ * different rank order is a different offer, so an id cannot be replayed+ * across turns to claim credit for an offer that was never made.+ *+ * The id an invocation carries is the AGENT'S CLAIM about where the+ * invocation came from - there are no credentials in this system and none+ * are implied here. What the id buys is a claim that is checkable (it must+ * be well formed, and it must match an offer that can be recomputed),+ * rather than no claim at all.+ */++import { computePayloadHash } from "../brain/idempotency-ledger.ts";++/** Payload / tool-argument key carrying an offer id. */+export const SKILL_OFFER_ID_KEY = "offer_id";++/**+ * Hex characters kept from the digest. Long enough that an accidental+ * collision across a vault's offers is not a practical concern, short+ * enough to sit in a rendered attach block an agent has to quote back.+ */+export const SKILL_OFFER_ID_LENGTH = 16;++const OFFER_ID_PATTERN = new RegExp(`^[0-9a-f]{${SKILL_OFFER_ID_LENGTH}}$`);++/** The part of an offered skill that identifies it: which skill, from where. */+export interface SkillOfferedItem {+ readonly name: string;+ readonly path: string;+}++/**+ * Content-addressed id for one offer. The query is trimmed and its internal+ * whitespace collapsed so a turn reformatted by a host runtime still yields+ * the same offer; everything else is taken verbatim, in the order offered.+ */+export function computeSkillOfferId(query: string, items: ReadonlyArray<SkillOfferedItem>): string {+ const normalisedQuery = query.trim().replace(/\s+/gu, " ");+ return computePayloadHash({+ query: normalisedQuery,+ // An array preserves rank order under the canonical encoding, which a+ // keyed object would not.+ offered: items.map((item) => [item.name, item.path]),+ }).slice(0, SKILL_OFFER_ID_LENGTH);+}++/** Whether a value is a well-formed offer id. */+export function isSkillOfferId(value: unknown): value is string {+ return typeof value === "string" && OFFER_ID_PATTERN.test(value);+}++/**+ * Lift the cited offer id out of a tool call's input, or null when the call+ * cited none. A value that is present but not a well-formed offer id is not+ * an offer either: it names nothing that could have been offered, so it is+ * reported the same as absence rather than stamped onto a record where it+ * would read as provenance. Surfaces that can refuse the caller outright+ * (the `get_skill` tool) use {@link isSkillOfferId} and say so instead; this+ * reader exists for a historical session log, which cannot be refused.+ */+export function readSkillOfferId(+ input: Readonly<Record<string, unknown>> | undefined | null,+): string | null {+ if (input === undefined || input === null) return null;+ const value = input[SKILL_OFFER_ID_KEY];+ return isSkillOfferId(value) ? value : null;+}diff --git a/src/core/surface/skills.ts b/src/core/surface/skills.tsindex f7671b95..3d79da34 100644--- a/src/core/surface/skills.ts+++ b/src/core/surface/skills.ts@@ -42,6 +42,25 @@ export interface SkillEntry {readonly path: string;/** Absolute path to the SKILL.md file. */readonly skillFile: string;+ /**+ * Absolute paths of same-named skill directories this entry overrode, in+ * discovery order (earliest first). Empty means discovery checked and+ * found nothing shadowed; `undefined` means the entry did not come from+ * {@link discoverSkills} and therefore carries no shadow information at+ * all. Those are different answers and are not collapsed.+ */+ readonly shadowed?: ReadonlyArray<string>;+}++/**+ * A {@link SkillEntry} that came out of {@link discoverSkills}, where the+ * shadow list is always present. Narrowing it here means a caller reading+ * discovery output never needs a fallback for a field discovery always sets -+ * and a fallback there would be exactly the "absent read as empty" collapse+ * the optional field exists to prevent.+ */+export interface DiscoveredSkill extends SkillEntry {+ readonly shadowed: ReadonlyArray<string>;}export interface SkillRootsOptions {@@ -96,7 +115,7 @@ function flattenTriggers(raw: FrontmatterValue): string {return "";}-function readSkillEntry(root: string, dir: string): SkillEntry | null {+function readSkillEntry(root: string, dir: string): DiscoveredSkill | null {const path = join(root, dir);const skillFile = join(path, SKILL_FILE_NAME);if (!existsSync(skillFile)) return null;@@ -111,6 +130,7 @@ function readSkillEntry(root: string, dir: string): SkillEntry | null {triggers,path,skillFile,+ shadowed: Object.freeze([] as string[]),});}@@ -128,26 +148,48 @@ function firstBodyLine(body: string): string {* Discover skills across roots. A later root overrides an earlier one* on name collision (vault-local skills shadow shipped ones). Output* is sorted by name. Fail-soft: unreadable roots/directories skip.+ *+ * The override RETAINS the loser: the winning entry lists every path it+ * shadowed in `shadowed`, so an operator reading a surprising skill body can+ * see that a second copy exists and where it lives. Dropping that path was+ * the whole of the defect - the collapse itself is correct and deliberate.+ *+ * Directory entries are sorted within each root so a collision between two+ * same-named skills in ONE root resolves the same way on every filesystem;+ * `readdirSync` order is not specified, and an unordered winner would make+ * both the surviving entry and the recorded shadow list arbitrary.*/-export function discoverSkills(roots: ReadonlyArray<string>): SkillEntry[] {- const byName = new Map<string, SkillEntry>();+export function discoverSkills(roots: ReadonlyArray<string>): DiscoveredSkill[] {+ const byName = new Map<string, DiscoveredSkill>();for (const root of roots) {let dirs: string[];try {- dirs = readdirSync(root).filter((d) => {- try {- return statSync(join(root, d)).isDirectory();- } catch {- return false;- }- });+ dirs = readdirSync(root)+ .filter((d) => {+ try {+ return statSync(join(root, d)).isDirectory();+ } catch {+ return false;+ }+ })+ .toSorted((a, b) => (a < b ? -1 : a > b ? 1 : 0));} catch {continue;}for (const dir of dirs) {try {const entry = readSkillEntry(root, dir);- if (entry !== null) byName.set(entry.name, entry);+ if (entry === null) continue;+ const shadowedEntry = byName.get(entry.name);+ byName.set(+ entry.name,+ shadowedEntry === undefined+ ? entry+ : Object.freeze({+ ...entry,+ shadowed: Object.freeze([...shadowedEntry.shadowed, shadowedEntry.path]),+ }),+ );} catch {// One malformed skill never hides the rest.}diff --git a/src/core/vault.ts b/src/core/vault.tsindex 697c2d65..5254aee6 100644--- a/src/core/vault.ts+++ b/src/core/vault.ts@@ -61,7 +61,19 @@ import {import type { FrontmatterMap, FrontmatterValue, VaultPage } from "./types.ts";const FRONTMATTER_RE = /^---\s*\n([\s\S]*?)\n---\s*\n?/;-const KEY_VALUE_RE = /^([a-zA-Z_][a-zA-Z0-9_-]*)\s*:\s*(.*?)\s*$/;+/**+ * The key grammar, written once and used by BOTH directions.+ *+ * The scanner has only ever recognised this shape, so it is also the+ * only shape the emitter may write: a key outside it is a key the+ * emitter would put on disk and the parser would then drop or, worse,+ * read as something else. Deriving `KEY_VALUE_RE` from the same source+ * is what makes "round-trips" a fact rather than a claim two regexes+ * happen to agree on.+ */+const FRONTMATTER_KEY_PATTERN = "[a-zA-Z_][a-zA-Z0-9_-]*";+const FRONTMATTER_KEY_RE = new RegExp(`^${FRONTMATTER_KEY_PATTERN}$`);+const KEY_VALUE_RE = new RegExp(`^(${FRONTMATTER_KEY_PATTERN})\\s*:\\s*(.*?)\\s*$`);// Block-sequence dash item: `- item` or a bare `-` (empty item). The// leading whitespace is required so `-foo` (a dash-prefixed string, not a// list item) is NOT matched. Captures the item text after the whitespace.@@ -314,14 +326,61 @@ function clipNoticeLine(line: string): string {: line.slice(0, NOTICE_LINE_MAX) + NOTICE_LINE_ELLIPSIS;}+/**+ * Whether `key` is a frontmatter key this format can round-trip: one the+ * emitter may write and the scanner will read back as the same key.+ *+ * Exported so a boundary that wants to reject a bad key as the CALLER's+ * fault, naming the argument, can ask the same question the emitter asks+ * rather than restating the grammar.+ */+export function isFrontmatterKey(key: string): boolean {+ return FRONTMATTER_KEY_RE.test(key);+}++/** A frontmatter key this emitter will not write. See {@link formatFrontmatter}. */+export class FrontmatterKeyError extends Error {+ /** The offending key, verbatim, so a caller can point at which one. */+ readonly key: string;+ constructor(key: string) {+ super(+ `frontmatter key ${JSON.stringify(key)} is not a key this format can round-trip; ` ++ "keys are a letter or underscore followed by letters, digits, '_' or '-'",+ );+ this.name = "FrontmatterKeyError";+ this.key = key;+ }+}+/*** Render a Markdown file with YAML-like frontmatter to a string. Pure* function — exposed so callers can decide *how* to persist (atomic,* exclusive, plain) without duplicating the YAML formatter.+ *+ * ## Keys are checked, not escaped+ *+ * The value side was already sound: {@link formatYamlScalar} quotes and+ * escapes any scalar that would break the scanner, so no VALUE has ever+ * been able to add a line to the block. Keys were emitted raw, which+ * made the same guarantee false in the other direction — a key+ * containing a newline emitted extra frontmatter lines, and because the+ * scanner is last-wins and `update_note` merges `{...existing,+ * ...caller}`, a caller could overwrite a key it was never asked to+ * touch. That reach was bounded by the `Brain/` machinery-root refusal,+ * so it was latent rather than live; it is fixed here, at the emitter,+ * so every writer in the tree inherits the guarantee rather than each+ * entry point having to remember it.+ *+ * Escaping is not the fix, because the reader has no quoted-key form to+ * escape INTO: a quoted key would fail `KEY_VALUE_RE` and be dropped on+ * the next parse, which trades an injection for a silent loss. So a key+ * outside the grammar is a typed {@link FrontmatterKeyError}, raised+ * before a single byte is rendered.*/export function formatFrontmatter(metadata: FrontmatterMap, body: string): string {const lines: string[] = ["---"];for (const [key, value] of Object.entries(metadata)) {+ if (!isFrontmatterKey(key)) throw new FrontmatterKeyError(key);lines.push(`${key}: ${formatYamlValue(value)}`);}lines.push("---");diff --git a/src/core/write-binding/index.ts b/src/core/write-binding/index.tsnew file mode 100644index 00000000..fafc02d7--- /dev/null+++ b/src/core/write-binding/index.ts@@ -0,0 +1,311 @@+/**+ * The write binding — a config-declared boundary over CALLER-NAMED+ * write destinations (provenance-at-the-boundary, unit B).+ *+ * ## What this is, said precisely+ *+ * It is a WRITE BOUNDARY over the paths a caller names. It is NOT a+ * security boundary and it is NOT per-credential, and neither wording+ * belongs on any surface that reports it.+ *+ * There is no credential in this system to key a fence to. Agent+ * identity resolves as `VAULT_AGENT_NAME`, else the config `agent_name`,+ * else the literal `agent`, and twenty-two MCP tool schemas across+ * fifteen modules additionally accept a caller-supplied `agent` string+ * that overrides it, taken verbatim after a placeholder check. (Count it+ * with `grep -rn '^\s*agent:\s*{' src/mcp/`; it has only ever grown, and+ * understating it understates this argument.) A caller who can call the tool can+ * name itself anything, so a fence keyed to that name is bypassed by+ * passing a different string. The binding therefore reads NO identity at+ * all: its whole authority is `<vault>/Brain/_brain.yaml`, which the+ * operator controls and which no MCP call can rewrite.+ *+ * ## What it covers, and what it cannot+ *+ * Write destinations in this codebase split three ways, and only the+ * first admits a prefix boundary:+ *+ * - CALLER-NAMED — `brain_create_note`, `brain_update_note`,+ * `brain_append_note` and `brain_write_batch` take a vault-relative+ * path from the caller. All four resolve it through one shared+ * envelope (`resolveNoteTarget`), which is where this check runs.+ * - SLUG-DERIVED — a signal, a preference, a dead end: the caller+ * supplies a topic and the writer derives the filename.+ * - FULLY DERIVED — daily logs, dream runs, continuity records and all+ * telemetry compute their own destination.+ *+ * A prefix boundary over the last two could only refuse them wholesale,+ * which is an off switch rather than a boundary, so they are out of+ * scope and said so. `tests/core/architecture/write-site-census.test.ts`+ * is the standing record of every in-vault write site this does not+ * cover, so the gap is an asserted fact rather than a silent one.+ *+ * ## Absent is inert+ *+ * No block, or a block without `path_prefixes`, means no binding: every+ * write path behaves exactly as it did before the key existed, and+ * `tests/core/write-binding.test.ts` proves it by running all three+ * caller-named arms against a vault with no config and a vault whose+ * config omits the block and comparing the bytes.+ *+ * Inert means inert on the FAILURE paths too. A vault whose+ * `Brain/_brain.yaml` exists but does not validate raises here, and that+ * is not a new failure mode the binding introduced: the same shared+ * envelope already resolves the vault scope through the same loader one+ * step earlier, so a malformed config has always stopped every+ * caller-named write. What the envelope now guarantees is that the stop+ * is a TYPED refusal naming a vault-relative source and a registered+ * exit, on the binding arm and the scope arm alike.+ *+ * ## What the binding compares+ *+ * The REALPATH, not the name. A lexical comparison is not a boundary: one+ * symlinked directory under a declared prefix+ * (`Projects/link -> <vault>/Journal`) would turn `path_prefixes:+ * [Projects]` into "may write anywhere in the vault", and the success+ * payload would name a path that does not hold the bytes. Every check+ * therefore resolves the destination through the filesystem and requires+ * BOTH the name the caller gave and the place it resolves to to lie+ * under a declared prefix.+ */++import { realpathSync, statSync } from "node:fs";+import { basename, dirname, join, relative, resolve, sep } from "node:path";++import { brainConfigPath } from "../brain/paths.ts";+import { requireNextStep } from "../brain/next-step.ts";+import { loadBrainConfig } from "../brain/policy.ts";+import { toPosix } from "../path-safety.ts";+import { normaliseWriteBindingPrefix, writeBindingPrefixCovers } from "./prefix.ts";++export { normaliseWriteBindingPrefix, writeBindingPrefixCovers };++/**+ * Registry code the refusal resolves its forward pointer through. The+ * refusal NAMES no command of its own — the registry does, so the exit+ * is the same structural CLI string on every surface that reports it.+ */+export const WRITE_BINDING_REFUSED_CODE = "write-binding-refused";++/**+ * Resolved at module scope on purpose. A refusal whose value is telling+ * the operator where to look is meaningless without its exit, so an+ * unregistered code is registry drift and fails at import rather than+ * inside the very refusal it was meant to explain (see `next-step.ts`).+ */+const REFUSED_EXIT = requireNextStep(WRITE_BINDING_REFUSED_CODE).nextCommand;++/** The declared binding in force for one vault. Never empty when present. */+export interface WriteBinding {+ /**+ * Vault-relative POSIX prefixes, already normalised by the block+ * parser. A target is admitted when it equals one of these or lies+ * under it segment-wise.+ */+ readonly pathPrefixes: ReadonlyArray<string>;+}++/** A caller-named target the declared binding does not admit. */+export interface WriteBindingRefusal {+ /** Always {@link WRITE_BINDING_REFUSED_CODE}; carried so a surface can resolve it. */+ readonly code: string;+ /** The vault-relative target that was refused, as the caller named it. */+ readonly relPath: string;+ /**+ * Where that name actually resolves on disk, vault-relative — the path+ * that would have held the bytes. Equal to {@link relPath} in the+ * ordinary case; different when a symlinked directory component+ * redirects the write, which is the case a lexical check missed. Null+ * when the destination resolves outside the vault entirely, which is a+ * different answer from "somewhere else inside it".+ */+ readonly resolvedRelPath: string | null;+ /** The prefixes in force, so the operator sees what WOULD be admitted. */+ readonly pathPrefixes: ReadonlyArray<string>;+ /** The registered exit for {@link code}. */+ readonly nextCommand: string;+ /**+ * The refusal, composed HERE and only here. Call sites raise it in+ * whatever error type their surface already speaks; none of them+ * assembles a sentence, so the wording cannot drift between the four+ * tools that share the envelope.+ */+ readonly message: string;+}++/** Separator between prefixes when the refusal lists them. */+const PREFIX_LIST_SEPARATOR = ", ";++/** How the refusal names a destination that leaves the vault entirely. */+const OUTSIDE_VAULT_LABEL = "a location outside the vault";++/**+ * One resolved binding plus the identity of the config bytes it was read+ * from, so the answer can be reused for as long as those bytes are the+ * ones on disk.+ */+interface CachedBinding {+ readonly vault: string;+ readonly identity: string;+ readonly binding: WriteBinding | null;+}++/**+ * Single-slot memo over {@link resolveWriteBinding}.+ *+ * A write batch resolves its target through the shared envelope once per+ * operation, so without this the whole `_brain.yaml` was read, parsed and+ * revalidated N times for one atomic batch against one unchanging file.+ * One slot is the right size: the loop that pays the cost hits the same+ * vault every iteration.+ *+ * The key is the config file's own identity, never a clock — see+ * {@link brainConfigIdentity}. A file that changed by any of inode, size+ * or mtime misses, so a cached answer can never outlive the declaration+ * it came from.+ */+let cachedBinding: CachedBinding | null = null;++/**+ * Identity of the config bytes at `path`, or `null` when the file is+ * ABSENT. Absence and inability-to-examine are different answers: an+ * ENOENT is a vault that predates `_brain.yaml`, while a permission+ * error is a declaration that may exist and cannot be read, and+ * answering "no binding" for the latter would silently widen the+ * boundary. Only ENOENT becomes `null`; every other errno is raised.+ */+function brainConfigIdentity(path: string): string | null {+ try {+ const st = statSync(path, { bigint: true });+ return `${st.ino}:${st.size}:${st.mtimeNs}`;+ } catch (err) {+ if ((err as NodeJS.ErrnoException)?.code === "ENOENT") return null;+ throw err;+ }+}++/**+ * The binding declared for `vault`, or `null` when none is.+ *+ * `null` is the absent result rather than an empty-prefix record because+ * the two are different answers: an empty record would have to be read+ * as "admits nothing", and the block parser refuses that declaration+ * precisely so it can never arrive here.+ *+ * A vault with no `_brain.yaml` is absent, not broken — existing vaults+ * predate the file. A config that is present but does not load raises,+ * matching `loadBrainConfig`: the operator's settings exist and are not+ * the ones in force, and answering "no binding" would silently widen the+ * boundary they declared. Callers on a caller-named write path translate+ * that raise into their own typed refusal rather than letting it surface+ * as an untyped fault.+ */+export function resolveWriteBinding(vault: string): WriteBinding | null {+ const configPath = brainConfigPath(vault);+ const identity = brainConfigIdentity(configPath);+ if (identity === null) return null;+ const hit = cachedBinding;+ if (hit !== null && hit.vault === vault && hit.identity === identity) return hit.binding;+ const declared = loadBrainConfig(vault).write_binding?.path_prefixes;+ const binding =+ declared === undefined || declared.length === 0+ ? null+ : Object.freeze({ pathPrefixes: Object.freeze([...declared]) });+ cachedBinding = Object.freeze({ vault, identity, binding });+ return binding;+}++/** Whether `binding` admits the vault-relative `relPath`. */+export function writeBindingAdmits(binding: WriteBinding, relPath: string): boolean {+ return binding.pathPrefixes.some((prefix) => writeBindingPrefixCovers(prefix, relPath));+}++/**+ * The realpath of `target`, resolving every symlink in the components+ * that exist and re-appending the components that do not yet.+ *+ * A write target is normally the LAST component and does not exist yet,+ * so `realpathSync` on it alone would only ever raise; what matters is+ * the directory chain above it, which does exist and is where a symlink+ * redirects the write. ENOENT walks one level up; any other errno is+ * raised, because a directory we cannot examine is not a directory we+ * know to be safe.+ */+function realpathOfDeepestExisting(target: string): string {+ try {+ return realpathSync(target);+ } catch (err) {+ if ((err as NodeJS.ErrnoException)?.code !== "ENOENT") throw err;+ const parent = dirname(target);+ if (parent === target) return target;+ return join(realpathOfDeepestExisting(parent), basename(target));+ }+}++/**+ * Where `relPath` actually lands, as a vault-relative POSIX path, or+ * `null` when it lands outside the vault altogether.+ */+function resolveRealVaultRelative(vault: string, relPath: string): string | null {+ const realVault = realpathOfDeepestExisting(resolve(vault));+ const realTarget = realpathOfDeepestExisting(resolve(join(vault, relPath)));+ const rel = relative(realVault, realTarget);+ // `relative` emits the platform separator, so the "climbs out of the+ // root" test uses it too. An empty result means the target IS the+ // vault root, which is not a place a note can be written either.+ if (rel === "" || rel === ".." || rel.startsWith(".." + sep)) return null;+ return toPosix(rel);+}++/**+ * The refusal for a caller-named write to `relPath`, or `null` when the+ * write is admitted — including when no binding is declared at all.+ *+ * Both the name and the destination must be admitted. The name alone is+ * not enough (a symlinked component redirects the bytes), and the+ * destination alone is not enough either (a caller must not be able to+ * name a path outside the binding and have it accepted because it+ * happens to resolve inside one).+ *+ * Takes no identity, by construction. See the module docblock for why a+ * fence keyed to the caller's claimed name would fence nothing.+ */+export function checkWriteBinding(vault: string, relPath: string): WriteBindingRefusal | null {+ const binding = resolveWriteBinding(vault);+ if (binding === null) return null;+ if (!writeBindingAdmits(binding, relPath)) return writeBindingRefusal(binding, relPath, relPath);+ const resolved = resolveRealVaultRelative(vault, relPath);+ if (resolved !== null && writeBindingAdmits(binding, resolved)) return null;+ return writeBindingRefusal(binding, relPath, resolved);+}++/**+ * Build the refusal record for a target the binding does not admit.+ *+ * `resolvedRelPath` is where the bytes would have gone; the message+ * names it beside the caller's own spelling whenever the two differ, so+ * an operator reading the refusal sees the redirection rather than+ * having to discover it.+ */+export function writeBindingRefusal(+ binding: WriteBinding,+ relPath: string,+ resolvedRelPath: string | null,+): WriteBindingRefusal {+ const normalised = normaliseWriteBindingPrefix(relPath);+ const prefixes = binding.pathPrefixes.join(PREFIX_LIST_SEPARATOR);+ const redirect =+ resolvedRelPath === normalised+ ? ""+ : ` (resolves to ${resolvedRelPath ?? OUTSIDE_VAULT_LABEL})`;+ return Object.freeze({+ code: WRITE_BINDING_REFUSED_CODE,+ relPath: normalised,+ resolvedRelPath,+ pathPrefixes: binding.pathPrefixes,+ nextCommand: REFUSED_EXIT,+ message:+ `${normalised}${redirect} is outside the write binding declared in Brain/_brain.yaml ` ++ `(write_binding.path_prefixes: ${prefixes}); ${REFUSED_EXIT}`,+ });+}diff --git a/src/core/write-binding/prefix.ts b/src/core/write-binding/prefix.tsnew file mode 100644index 00000000..5c39a47a--- /dev/null+++ b/src/core/write-binding/prefix.ts@@ -0,0 +1,63 @@+/**+ * The prefix grammar of the write binding, and nothing else.+ *+ * This module IMPORTS NOTHING, deliberately. It is the cycle-safe home+ * shared by the two layers that must agree on what a declared prefix+ * means — the `_brain.yaml` block parser+ * (`brain/policy/blocks/write-binding.ts`) and the runtime matcher+ * (`./index.ts`) — in exactly the shape `vault-scope/defaults.ts`+ * already uses for the ignore-rule grammar.+ *+ * The split is load-bearing rather than tidy: the matcher reads the+ * configuration through `brain/policy.ts`, which reaches the block+ * parser, so a parser that imported the matcher would close a cycle and+ * make the initialisation order of both undefined.+ */++/**+ * The ONE separator this grammar knows. Declarations are POSIX, and so+ * is every path the matcher is handed: `inspectPath` returns its+ * segments joined with `/` on both platforms (on Windows it converts the+ * native separator first, on POSIX there is nothing to convert).+ *+ * A backslash is DELIBERATELY not a separator here. On POSIX it is an+ * ordinary filename character, so splitting on it made the matcher read+ * structure into a name: a binding on `Projects` admitted the literal+ * one-segment filename `Projects\evil.md`, which lands at the vault+ * ROOT. The caller-named write envelope+ * (`brain/notes/create-note.ts`, `assertHostPathSeparators`) refuses a+ * backslash-bearing path outright rather than reinterpreting it, so the+ * matcher never has to guess which reading the caller meant.+ */+const POSIX_SEP = "/";++/**+ * Canonical form of one declared prefix: no `./` segments, no repeated+ * or trailing slashes.+ *+ * Returns the empty string for a declaration that carries no segments at+ * all (`.`, `/`, `///`). That is not a usable prefix — it would admit+ * the whole vault — and the block parser refuses it rather than letting+ * a typo silently widen the boundary.+ */+export function normaliseWriteBindingPrefix(raw: string): string {+ const segments: string[] = [];+ for (const segment of raw.split(POSIX_SEP)) {+ if (segment === "" || segment === ".") continue;+ segments.push(segment);+ }+ return segments.join(POSIX_SEP);+}++/**+ * Whether `relPath` lies at or under `prefix`, SEGMENT-WISE.+ *+ * Segment-wise is the whole point. A character-prefix test would let a+ * binding on `Projects` admit `ProjectsArchive/`, so the declaration+ * would silently cover a folder the operator never named.+ */+export function writeBindingPrefixCovers(prefix: string, relPath: string): boolean {+ const target = normaliseWriteBindingPrefix(relPath);+ if (target === prefix) return true;+ return target.startsWith(prefix + POSIX_SEP);+}diff --git a/src/mcp/brain/entity-tools.ts b/src/mcp/brain/entity-tools.tsindex bad6b73f..3a0e267f 100644--- a/src/mcp/brain/entity-tools.ts+++ b/src/mcp/brain/entity-tools.ts@@ -12,6 +12,10 @@ import {listEntities,} from "../../core/brain/entities/registry.ts";import { validateEntityCategory } from "../../core/brain/entities/canonical.ts";+import {+ BRAIN_ENTITY_STATUS_VALUES,+ isBrainEntityStatus,+} from "../../core/brain/entities/types.ts";import { switchProfile, listProfiles } from "../../core/brain/portability/profiles.ts";import { defaultConfigPath } from "../../core/config.ts";import { INTERNAL_ERROR, INVALID_PARAMS, MCPError } from "../protocol.ts";@@ -59,8 +63,13 @@ async function toolBrainEntity(}}const status = coerceStr(args, "status", false);- if (status !== null && status !== "active" && status !== "archived") {- throw new MCPError(INVALID_PARAMS, "brain_entity: status must be 'active' or 'archived'");+ // Validated against the vocabulary itself, so the enum below and this+ // check cannot drift and a new status is reachable the day it exists.+ if (status !== null && !isBrainEntityStatus(status)) {+ throw new MCPError(+ INVALID_PARAMS,+ `brain_entity: status must be one of: ${BRAIN_ENTITY_STATUS_VALUES.join(", ")}`,+ );}const query = view === "get" ? coerceStr(args, "query", true)! : null;@@ -137,8 +146,10 @@ export const ENTITY_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([},status: {type: "string",- enum: ["active", "archived"],- description: "Optional status filter for view: list.",+ enum: [...BRAIN_ENTITY_STATUS_VALUES],+ description:+ "Optional status filter for view: list. Omits quarantined entities " ++ "(they entered under untrusted provenance) unless you pass `quarantine`.",},},required: ["view"],diff --git a/src/mcp/brain/ner-tools.ts b/src/mcp/brain/ner-tools.tsindex 23de61a0..51677ca6 100644--- a/src/mcp/brain/ner-tools.ts+++ b/src/mcp/brain/ner-tools.ts@@ -15,7 +15,9 @@*/import { intakeExtraction, IntakeValidationError } from "../../core/brain/intake/extract-intake.ts";+import { classifySourceTrust } from "../../core/brain/intake/source-trust.ts";import { resolveAgentName } from "../../core/config.ts";+import { INVALID_PARAMS, MCPError } from "../protocol.ts";import type { ServerContext, ToolDefinition } from "../tool-contract.ts";import { parseExtractionIntakeArgs } from "./intake-args.ts";import { wrapToolErrors } from "./shared.ts";@@ -31,6 +33,30 @@ async function toolBrainIntakeEntities(parsed.agent && parsed.agent.trim().length > 0? parsed.agent: resolveAgentName(ctx.configPath ?? undefined);+ // Trust is derived from the source identity the caller named, through the+ // same structural classifier the ingest pipeline uses.+ //+ // A call that names NO source is a third answer, and collapsing it into+ // either lane is a defect in one direction or the other. Trusting it makes+ // the omission itself the way in: an agent reading a hostile page, told by+ // that page to leave the source out, lands its entities active and+ // unmarked. Quarantining it punishes the caller for a question nobody+ // asked, and quarantine is one-way - the records leave every ordinary read+ // and only an explicit release brings them back, while the response says+ // the write succeeded. This is the surface that can still ask, so it asks:+ // an intake whose provenance cannot be established is refused here, with+ // the exit named, before anything is written.+ const source = parsed.provenance?.sources[0];+ if (source === undefined) {+ throw new MCPError(+ INVALID_PARAMS,+ `${TOOL}: 'source' is required - name the note this extraction came from ` ++ "(a vault wikilink, e.g. `[[Articles/primer.md]]`) or the address it was read from. " ++ "Entities are committed under the provenance of their source, so an unnamed source " ++ "has no provenance to commit under.",+ );+ }+ const trust = classifySourceTrust(ctx.vault, source);// A malformed extraction is a client-resolvable input problem, not a// server fault - surface it as INVALID_PARAMS, never a fabricated result.return wrapToolErrors(TOOL, [IntakeValidationError], async () => {@@ -38,11 +64,16 @@ async function toolBrainIntakeEntities(agent,now: new Date(),...(parsed.provenance !== undefined ? { provenance: parsed.provenance } : {}),+ ...(trust !== undefined ? { trust } : {}),});return {entities_created: [...result.entitiesCreated],entities_updated: [...result.entitiesUpdated],relations_applied: result.relationsApplied,+ // The lane the entities landed in. An untrusted intake quarantines what+ // it introduces, so a caller told only which ids it created would be+ // told nothing about whether it can read them back.+ trust,};});}@@ -51,7 +82,7 @@ export const NER_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([{name: TOOL,description:- "Intake entities the agent extracted from note text into the entity registry (OSB runs no model). Supply `entities` (category, name, optional aliases/confidence), optional typed `relations` (from, relation, to), and an optional `source` wikilink cited on new pages. Validated and idempotent.",+ "Intake entities the agent extracted from note text into the entity registry (OSB runs no model). Supply `entities` (category, name, optional aliases), the `source` they came from, and optional typed `relations`. Entities from a source outside the vault are quarantined; the response says so.",inputSchema: {type: "object",properties: {@@ -101,14 +132,15 @@ export const NER_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([},source: {type: "string",- description: "Optional source wikilink cited in the Sources section of new entity pages.",+ description:+ "Where this extraction came from: a vault wikilink (`[[Articles/x.md]]`) or the address read. Cited on new entity pages; decides their provenance.",},agent: {type: "string",description: "Optional agent identity override; defaults to the server-resolved name.",},},- required: ["entities"],+ required: ["entities", "source"],additionalProperties: false,},handler: toolBrainIntakeEntities,diff --git a/src/mcp/brain/notes-tools.ts b/src/mcp/brain/notes-tools.tsindex 861fc49c..715552ed 100644--- a/src/mcp/brain/notes-tools.ts+++ b/src/mcp/brain/notes-tools.ts@@ -5,40 +5,152 @@* daily log), this tool writes an actual vault note file - path,* frontmatter, and body - through the shared `createNote` primitive.* The primitive enforces the vault-scope, path-traversal, Brain-root,- * and no-clobber guards; this handler only coerces arguments and maps a- * typed `CreateNoteError` to a client-side INVALID_PARAMS.+ * write-binding, and no-clobber guards; this handler only coerces+ * arguments and maps the typed `CreateNoteError` onto an MCPError.+ *+ * That mapping splits on WHOSE fault the refusal is, not on which error+ * class arrived: a caller-input fault is an INVALID_PARAMS the caller can+ * act on, while a vault whose own config does not load is an+ * INTERNAL_ERROR, because no argument would have succeeded. Both carry+ * the typed `code`, and both carry the advisory registry spelling of that+ * code plus its next command when one is registered - see+ * {@link advisoryFields}.*/import type { FrontmatterMap, FrontmatterValue } from "../../core/types.ts";-import { createNote, CreateNoteError } from "../../core/brain/notes/create-note.ts";+import {+ createNote,+ CreateNoteError,+ CREATE_NOTE_IF_EXISTS,+ NOTE_CONFIG_INVALID_CODE,+ type CreateNoteIfExists,+} from "../../core/brain/notes/create-note.ts";import {applyWriteBatch,WriteBatchError,type WriteOperation,} from "../../core/brain/write-batch.ts";+import { nextCommandField } from "../../core/brain/next-step.ts";+import { WRITE_BINDING_REFUSED_CODE } from "../../core/write-binding/index.ts";+import { isFrontmatterKey } from "../../core/vault.ts";import { INTERNAL_ERROR, INVALID_PARAMS, MCPError } from "../protocol.ts";import type { ServerContext, ToolDefinition } from "../tool-contract.ts";-import { coerceStr } from "../coerce.ts";+import { coerceBoolOptional, coerceStr, coerceStringOptional } from "../coerce.ts";/**- * Narrow an untrusted `frontmatter` argument to a {@link FrontmatterMap}.- * Accepts a plain object whose values are strings, numbers, booleans, or- * string arrays (the frontmatter value domain); rejects anything else- * with INVALID_PARAMS rather than silently dropping it. `tool` names the- * calling surface so the rejection message points at the right tool.+ * The two vocabularies a refused write speaks, and the bridge between+ * them.+ *+ * A refusal has always carried TWO names for one concept: the surface+ * code in `data.code` (`write_binding`), which is snake_case because it+ * sits in the same enumeration as `invalid_path` and `excluded` and is+ * what a client branches on; and the advisory registry code+ * (`write-binding-refused`), which is kebab-case because that is the+ * registry's spelling and is what resolves the next command. Renaming+ * either would break the convention of the other, so instead the+ * relationship is made explicit: every refusal that HAS a registered+ * code reports it as `data.diagnostic_code` beside `data.code`, together+ * with the `next_command` it resolves to. An agent that received a code+ * can now find it, and `docs/mcp.md` documents both spellings.+ *+ * A surface code absent from this map has no registered advisory code -+ * stated by leaving it out, never by inventing one.*/-export function parseFrontmatterArg(value: unknown, tool: string): FrontmatterMap | undefined {+const DIAGNOSTIC_CODE_FOR_SURFACE_CODE: Readonly<Record<string, string>> = Object.freeze({+ write_binding: WRITE_BINDING_REFUSED_CODE,+ config_invalid: NOTE_CONFIG_INVALID_CODE,+});++/** Key under which a refusal reports the registry spelling of its code. */+const DIAGNOSTIC_CODE_KEY = "diagnostic_code";++/**+ * The advisory fields a surface `code` contributes to an MCP error's+ * `data`, or nothing at all when the code has no registered exit.+ */+export function advisoryFields(code: string): Readonly<Record<string, unknown>> {+ const diagnosticCode = DIAGNOSTIC_CODE_FOR_SURFACE_CODE[code];+ if (diagnosticCode === undefined) return {};+ return Object.freeze({+ [DIAGNOSTIC_CODE_KEY]: diagnosticCode,+ ...nextCommandField(diagnosticCode),+ });+}++/**+ * Surface codes that are NOT the caller's fault. A malformed+ * `Brain/_brain.yaml` refuses every write on this surface no matter what+ * arguments arrive, so reporting it as INVALID_PARAMS would send the+ * agent looking at its own request forever. It is reported as an+ * INTERNAL_ERROR that is nonetheless typed and carries its exit.+ */+const OPERATOR_FAULT_CODES: ReadonlySet<string> = new Set(["config_invalid"]);++/** JSON-RPC code for a refusal, split on whose fault the refusal is. */+function rpcCodeFor(surfaceCode: string): number {+ return OPERATOR_FAULT_CODES.has(surfaceCode) ? INTERNAL_ERROR : INVALID_PARAMS;+}++/**+ * Template variable names are NOT frontmatter keys, and holding them to+ * the frontmatter key grammar would narrow a surface this change has no+ * business narrowing: a variable name never becomes a line of a file,+ * the template parser has its own name grammar, and an unmatched+ * placeholder is deliberately left intact so a typo surfaces in the+ * rendered note. Passing this keeps that arm byte-identical - only the+ * shared VALUE domain was ever meant to be shared.+ */+const ANY_TEMPLATE_VARIABLE_NAME = (): boolean => true;++/**+ * Longest accepted body template. A template is a note skeleton, not a+ * payload; the rendered note is bounded separately by the artifact cap+ * that `strict` enforces.+ */+const TEMPLATE_MAX_LEN = 32_768;++/**+ * Narrow an untrusted argument to a {@link FrontmatterMap}. Accepts a+ * plain object whose values are strings, numbers, booleans, or string+ * arrays (the frontmatter value domain); rejects anything else with+ * INVALID_PARAMS rather than silently dropping it. `tool` names the+ * calling surface and `field` the argument, so the rejection points at+ * the exact thing the caller got wrong - template variables share this+ * value domain deliberately rather than growing a second one.+ */+export function parseFrontmatterArg(+ value: unknown,+ tool: string,+ field = "frontmatter",+ isKeyLegal: (key: string) => boolean = isFrontmatterKey,+): FrontmatterMap | undefined {if (value === undefined || value === null) return undefined;if (typeof value !== "object" || Array.isArray(value)) {- throw new MCPError(INVALID_PARAMS, `${tool}: frontmatter must be an object`);+ throw new MCPError(INVALID_PARAMS, `${tool}: ${field} must be an object`);}// Prototype-free target + explicit rejection of prototype-mutating keys:// `frontmatter` is untrusted, and a `__proto__`/`constructor`/`prototype`// key with an array value would otherwise pollute the object prototype.+ //+ // The prototype triad is not the whole check. A key is also a line of+ // the file the emitter writes, so it is held to the grammar the+ // frontmatter scanner can read back - the same rule+ // `formatFrontmatter` enforces, applied here so the caller learns+ // WHICH argument was wrong instead of receiving the emitter's fault.+ // `FrontmatterKeyError` rejects the prototype triad too (none matches+ // the key grammar); the triad stays named because prototype pollution+ // and an unreadable key are different reasons to say no.const out: FrontmatterMap = Object.create(null) as FrontmatterMap;for (const [key, raw] of Object.entries(value)) {if (key === "__proto__" || key === "constructor" || key === "prototype") {- throw new MCPError(INVALID_PARAMS, `${tool}: invalid frontmatter key "${key}"`);+ throw new MCPError(INVALID_PARAMS, `${tool}: invalid ${field} key "${key}"`);+ }+ if (!isKeyLegal(key)) {+ throw new MCPError(+ INVALID_PARAMS,+ `${tool}: ${field} key ${JSON.stringify(key)} is not a key this format can ` ++ "round-trip; use a letter or underscore followed by letters, digits, '_' or '-'",+ );}let coerced: FrontmatterValue;if (typeof raw === "string" || typeof raw === "number" || typeof raw === "boolean") {@@ -48,7 +160,7 @@ export function parseFrontmatterArg(value: unknown, tool: string): FrontmatterMa} else {throw new MCPError(INVALID_PARAMS,- `${tool}: frontmatter.${key} must be a string, number, boolean, or string array`,+ `${tool}: ${field}.${key} must be a string, number, boolean, or string array`,);}out[key] = coerced;@@ -57,24 +169,48 @@ export function parseFrontmatterArg(value: unknown, tool: string): FrontmatterMa}/**- * Map a core {@link WriteBatchError} onto a structured INVALID_PARAMS so- * the agent gets a machine-readable rejection (`code`, offending `index`)- * instead of opaque prose. `tool` prefixes the message. Any other error is- * a genuine I/O fault; wrap it in an INTERNAL_ERROR MCPError (mirroring the- * fallback in {@link toolBrainCreateNote}) so every write surface returns a- * consistent structured MCPError rather than an opaque throw.+ * Map a core {@link WriteBatchError} onto a structured MCPError so the+ * agent gets a machine-readable rejection (`code`, offending `index`,+ * and the advisory code / next command when the state has one) instead+ * of opaque prose. `tool` prefixes the message. The JSON-RPC code splits+ * exactly as it does for a single note write - see {@link rpcCodeFor} -+ * so a batch and a one-shot report the same state the same way. Any+ * other error is a genuine I/O fault; wrap it in an INTERNAL_ERROR+ * MCPError (mirroring the fallback in {@link toolBrainCreateNote}) so+ * every write surface returns a consistent structured MCPError rather+ * than an opaque throw.*/export function writeBatchErrorToMcp(err: unknown, tool: string): MCPError {if (err instanceof WriteBatchError) {- return new MCPError(INVALID_PARAMS, `${tool}: ${err.message}`, {+ return new MCPError(rpcCodeFor(err.code), `${tool}: ${err.message}`, {code: err.code,index: err.index,+ ...advisoryFields(err.code),...err.details,});}return new MCPError(INTERNAL_ERROR, err instanceof Error ? err.message : String(err));}+/**+ * Read the occupied-target policy. An unrecognised value is refused+ * rather than falling back to the default: a caller that asked for a+ * disposition this tool does not have must not be told its write+ * succeeded under a different one.+ */+function coerceIfExists(args: Record<string, unknown>): CreateNoteIfExists | undefined {+ const raw = coerceStringOptional(args, "if_exists", 16);+ if (raw === undefined) return undefined;+ const match = CREATE_NOTE_IF_EXISTS.find((policy) => policy === raw);+ if (match === undefined) {+ throw new MCPError(+ INVALID_PARAMS,+ `brain_create_note: if_exists must be one of ${CREATE_NOTE_IF_EXISTS.join(", ")}`,+ );+ }+ return match;+}+async function toolBrainCreateNote(ctx: ServerContext,args: Record<string, unknown>,@@ -82,20 +218,45 @@ async function toolBrainCreateNote(const path = coerceStr(args, "path", true)!;const content = coerceStr(args, "content", false);const frontmatter = parseFrontmatterArg(args["frontmatter"], "brain_create_note");+ const ifExists = coerceIfExists(args);+ const strict = coerceBoolOptional(args, "strict");+ const template = coerceStringOptional(args, "template", TEMPLATE_MAX_LEN);+ const templateVariables = parseFrontmatterArg(+ args["template_variables"],+ "brain_create_note",+ "template_variables",+ ANY_TEMPLATE_VARIABLE_NAME,+ );try {const res = createNote(ctx.vault, {path,...(frontmatter !== undefined ? { frontmatter } : {}),...(content !== null && content !== undefined ? { content } : {}),+ ...(ifExists !== undefined ? { ifExists } : {}),+ ...(strict !== undefined ? { strict } : {}),+ ...(template !== undefined ? { template } : {}),+ ...(templateVariables !== undefined ? { templateVariables } : {}),});- return { created: res.created, path: res.path };+ // `outcome` is the discriminant; `created` is the boolean this tool+ // has always returned and stays in lockstep with it, so a skip can+ // never be read as a create by either field.+ return { created: res.created, outcome: res.outcome, path: res.path };} catch (err) {- // Every CreateNoteError is a client-input fault (bad path, excluded- // location, or an existing target); report it as INVALID_PARAMS with- // the typed message. Anything else is a genuine I/O fault.+ // Almost every CreateNoteError is a client-input fault (bad path,+ // excluded location, an existing target, an invalid document, a+ // malformed template, a destination outside the declared binding);+ // report it with the typed code, its registry spelling and exit, and+ // the validator's fix list when there is one. `config_invalid` is+ // the exception the split in `rpcCodeFor` exists for: no argument+ // the caller could send would succeed. Anything else is a genuine+ // I/O fault.if (err instanceof CreateNoteError) {- throw new MCPError(INVALID_PARAMS, `brain_create_note: ${err.message}`);+ throw new MCPError(rpcCodeFor(err.code), `brain_create_note: ${err.message}`, {+ code: err.code,+ ...advisoryFields(err.code),+ ...(err.violations.length > 0 ? { violations: err.violations } : {}),+ });}throw new MCPError(INTERNAL_ERROR, err instanceof Error ? err.message : String(err));}@@ -168,7 +329,7 @@ export const NOTES_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([{name: "brain_create_note",description:- "Create an actual vault note file (path + frontmatter + content), written atomically inside the vault. Distinct from brain_note, which only appends a log line. Refuses path traversal, the Brain machinery root, vault-scope-excluded paths, and overwriting an existing note.",+ "Create an actual vault note file (path + frontmatter + content), written atomically inside the vault. Distinct from brain_note, which only appends a log line. Refuses path traversal, the Brain root, vault-scope-excluded paths, writes outside the declared write binding, and by default clobbering.",inputSchema: {type: "object",properties: {@@ -184,7 +345,30 @@ export const NOTES_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([},content: {type: "string",- description: "Optional Markdown body written below the frontmatter.",+ description:+ "Optional Markdown body written below the frontmatter. Mutually exclusive with template.",+ },+ if_exists: {+ type: "string",+ enum: [...CREATE_NOTE_IF_EXISTS],+ description:+ "Occupied-target policy. Default refuse. skip leaves the note untouched and returns outcome=skipped, never created.",+ },+ strict: {+ type: "boolean",+ description:+ "Validate the document before writing (frontmatter present, size, control chars, declared type in page_types) and refuse with coded violations.",+ },+ template: {+ type: "string",+ description:+ "Body template instead of content. Two constructs: {{name}} substitution, and {{#name}}..{{/name}} presence sections / list iteration where {{.}} is the item.",+ },+ template_variables: {+ type: "object",+ description:+ "Values the template references; same domain as frontmatter. Unknown placeholders are left intact so a typo surfaces.",+ additionalProperties: { type: ["string", "number", "boolean", "array"] },},},required: ["path"],@@ -195,7 +379,7 @@ export const NOTES_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([{name: "brain_update_note",description:- "Update an existing vault note: merge frontmatter keys and/or replace the body, written atomically. A missing target is refused. Reuses the create-note safety envelope: path traversal, the Brain machinery root, and vault-scope-excluded paths are refused.",+ "Update an existing vault note: merge frontmatter keys and/or replace the body, written atomically. A missing target is refused. Reuses the create-note safety envelope: path traversal, the Brain root, vault-scope-excluded paths, and writes outside the declared write binding are refused.",inputSchema: {type: "object",properties: {@@ -222,7 +406,7 @@ export const NOTES_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([{name: "brain_append_note",description:- "Append Markdown text to the body of an existing vault note, written atomically. A missing target is refused. Reuses the create-note safety envelope: path traversal, the Brain machinery root, and vault-scope-excluded paths are refused.",+ "Append Markdown text to the body of an existing vault note, written atomically. A missing target is refused. Reuses the create-note safety envelope: path traversal, the Brain root, vault-scope-excluded paths, and writes outside the declared write binding are refused.",inputSchema: {type: "object",properties: {diff --git a/src/mcp/brain/procedure-tools.ts b/src/mcp/brain/procedure-tools.tsindex b077795e..670617d9 100644--- a/src/mcp/brain/procedure-tools.ts+++ b/src/mcp/brain/procedure-tools.ts@@ -95,6 +95,10 @@ async function toolBrainSkillProposals(// Per-skill invocation telemetry (t_56a12bde): deterministic counts// derived from skill_invoked continuity records, distinct from proposing// or ranking. Real usage evidence for the dream/synthesis pass.+ // Each row also carries `offerAttributedCount` (t_ccb05134) - how many of+ // those invocations cited the offer they came from. The command-line+ // surface prints the same figure as `from_offer=`. The remainder are not+ // unattributed-by-error: most runtimes' skill calls follow no offer.const usage = deriveSkillUsage(ctx.vault);return { total: usage.length, usage };}@@ -264,7 +268,7 @@ export const PROCEDURE_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([{name: "brain_skill_proposals",description:- "Learn/list/review deterministic skill proposals from continuity records (learn, list, accept, reject), resolve abandoned accept sequences (recover), read per-skill invocation counts (usage), and resolve a proposal's self-reported support against the recorded procedural outcome ledger (evidence).",+ "Deterministic skill proposals from continuity records: learn, list, accept, reject; resolve abandoned accept sequences (recover); read per-skill invocation and offer-attribution counts (usage); resolve a proposal's self-reported support against the recorded procedural outcome ledger (evidence).",inputSchema: {type: "object",properties: {diff --git a/src/mcp/brain/recall-tools.ts b/src/mcp/brain/recall-tools.tsindex 76eaf072..6e20e8b9 100644--- a/src/mcp/brain/recall-tools.ts+++ b/src/mcp/brain/recall-tools.ts@@ -42,11 +42,15 @@ import {type TokenImpactOutcome,} from "../../core/brain/token-impact.ts";import {- emitContextPackOutcome,listContextPackOutcomes,+ postContextPackOutcome,summarizeContextPackOutcomes,type ContextPackOutcomeFilter,} from "../../core/brain/context-pack-outcome.ts";+import {+ CONTEXT_PACK_EVIDENCE_FIELD,+ type ContextPackEvidenceClaim,+} from "../../core/brain/context-pack-evidence.ts";import {resolveContextPackOutcomeEnabled,resolveTokenImpactLedgerEnabled,@@ -599,11 +603,14 @@ function coerceNonNegativeTokenValue(/*** Agent-operable context-pack outcome loop. `post` writes one compact* outcome row (gated on `context_pack_outcome_enabled`, payload-safe:- * counters + an opaque sample id only) AND composes the C3 ledger by posting- * a matching first-pass/repair/retry calibration record. `list`/`summary`- * read the rows regardless of the gate so historical aggregates stay- * inspectable. The three token signals (exact/modeled/observed) stay strictly- * separate; a field the caller omits is never invented.+ * counters + an opaque sample id only), composes the C3 ledger by posting+ * a matching first-pass/repair/retry calibration record, and composes the+ * evidence half by posting the kernel's own reading of the sample beside+ * the agent's claim. All three rows join on the one sample id and carry+ * the same acting agent. `list`/`summary` read the rows regardless of the+ * gate so historical aggregates stay inspectable. The three token signals+ * (exact/modeled/observed) stay strictly separate; a field the caller omits+ * is never invented.*/async function toolBrainContextPackOutcome(ctx: ServerContext,@@ -646,8 +653,9 @@ async function toolBrainContextPackOutcome("observed_provider_tokens",args["observed_provider_tokens"],);+ const evidenceClaim = contextPackEvidenceClaim(args["evidence_claim"]);const enabled = resolveContextPackOutcomeEnabled(ctx.configPath ?? undefined);- const record = emitContextPackOutcome(+ const post = postContextPackOutcome(ctx.vault,{sampleId,@@ -660,19 +668,25 @@ async function toolBrainContextPackOutcome(...(exact !== undefined ? { exactPromptTokenSavings: exact } : {}),...(modeled !== undefined ? { modeledInferenceAvoidance: modeled } : {}),...(observed !== undefined ? { observedProviderTokens: observed } : {}),+ ...(evidenceClaim !== undefined ? { evidenceClaim } : {}),},enabled || undefined,);- if (record === null) {+ if (post === null) {return { vault_path: ctx.vault, recorded: false, enabled };}return {vault_path: ctx.vault,recorded: true,enabled,- id: record.id,+ id: post.outcome.id,sample_id: sampleId,first_pass_success: firstPassSuccess,+ // The kernel's reading of this sample, surfaced so a contradicted+ // claim is visible to the poster and not only to a later reader.+ // Absent when the evidence half fail-opened - reporting a verdict+ // that was never written would be the fallback this project bans.+ ...(post.evidence !== null ? { evidence: post.evidence.payload } : {}),};}@@ -690,18 +704,75 @@ async function toolBrainContextPackOutcome();}+/**+ * The correlation keys a `post` records, read from the caller's arguments.+ *+ * `agent_id` is the ACTING agent, and it is an ARGUMENT rather than+ * `ServerContext.agentName` deliberately. That getter resolves the plugin+ * config on every access and RAISES on a config that is present but+ * unreadable - correctly, since the alternative is writing under a guessed+ * identity - so consulting it here would let one broken line in an+ * unrelated file turn a working telemetry post into a tool error. The+ * identity a client asserts is exactly as self-asserted as the one the+ * config holds and costs nothing when the config is broken, so it is the+ * one this row carries. Omitted stays omitted: no identity is invented.+ */function contextPackOutcomeCorrelation(args: Record<string, unknown>): {host?: string;sessionId?: string;turnId?: string;+ agentId?: string;} {const host = optionalStringArg("brain_context_pack_outcome", args, "host");const sessionId = optionalStringArg("brain_context_pack_outcome", args, "session_id");const turnId = optionalStringArg("brain_context_pack_outcome", args, "turn_id");+ const agentId = optionalStringArg("brain_context_pack_outcome", args, "agent_id");return {...(host !== undefined ? { host } : {}),...(sessionId !== undefined ? { sessionId } : {}),...(turnId !== undefined ? { turnId } : {}),+ ...(agentId !== undefined ? { agentId } : {}),+ };+}++/**+ * The acting agent's assertion about the sample, coerced at the tool+ * boundary. A malformed member is REFUSED here rather than dropped: a+ * claim silently discarded would land an evidence record reading+ * `unclaimed`, which asserts the agent said nothing when in fact it said+ * something unusable.+ */+function contextPackEvidenceClaim(raw: unknown): ContextPackEvidenceClaim | undefined {+ if (raw === undefined || raw === null) return undefined;+ if (typeof raw !== "object" || Array.isArray(raw)) {+ throw new MCPError(+ INVALID_PARAMS,+ "brain_context_pack_outcome: evidence_claim must be an object",+ );+ }+ const claim = raw as Record<string, unknown>;+ const finalTextHash = optionalStringArg(+ "brain_context_pack_outcome",+ claim,+ CONTEXT_PACK_EVIDENCE_FIELD.finalTextHash,+ );+ const itemCount = coerceNonNegativeInteger(+ "brain_context_pack_outcome",+ CONTEXT_PACK_EVIDENCE_FIELD.itemCount,+ claim[CONTEXT_PACK_EVIDENCE_FIELD.itemCount],+ );+ const finalTextChars = coerceNonNegativeInteger(+ "brain_context_pack_outcome",+ CONTEXT_PACK_EVIDENCE_FIELD.finalTextChars,+ claim[CONTEXT_PACK_EVIDENCE_FIELD.finalTextChars],+ );+ if (finalTextHash === undefined && itemCount === undefined && finalTextChars === undefined) {+ return undefined;+ }+ return {+ ...(finalTextHash !== undefined ? { finalTextHash } : {}),+ ...(itemCount !== undefined ? { itemCount } : {}),+ ...(finalTextChars !== undefined ? { finalTextChars } : {}),};}@@ -1121,7 +1192,7 @@ export const RECALL_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([name: "brain_context_pack_outcome",previewBudget: MCP_PREVIEW_BUDGET,description:- "Agent-operable context-pack outcome loop. `post` records a compact outcome row for a carried sample id — first-pass/repair/retry counters plus three SEPARATE token signals (exact, modeled, observed) — and calibrates the token-impact ledger. `list`/`summary` read rows. Gated, payload-safe.",+ "Context-pack outcome loop. `post` records an outcome row for a carried sample id — first-pass/repair/retry counters plus three SEPARATE token signals (exact, modeled, observed) — calibrates the token-impact ledger, and records the kernel's on-disk evidence. `list`/`summary` read rows. Gated.",inputSchema: {type: "object",properties: {@@ -1171,9 +1242,43 @@ export const RECALL_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([description:"post (optional): OBSERVED provider-reported token usage. Kept separate from the exact and modeled signals; also calibrates the token-impact ledger.",},+ evidence_claim: {+ type: "object",+ // Every client pays for this string on every request (the+ // registry guard caps it at 160 chars), so the reasoning lives+ // on `contextPackEvidenceClaim` below and the wire text states+ // only what a caller must know to use the field correctly: the+ // FOUR verdicts it can produce, and that a malformed member is+ // refused rather than downgraded - the earlier "Never rejects"+ // described the disagreement path and read as covering both.+ description:+ "post (optional): what you assert about this sample; kernel reads its receipt off disk, records match|mismatch|unclaimed|unresolved. Malformed = INVALID_PARAMS.",+ properties: {+ final_text_hash: {+ type: "string",+ description: "Claimed SHA-256 of the assembled pack text.",+ },+ item_count: {+ type: "integer",+ minimum: 0,+ description: "Claimed number of artifacts the pack injected.",+ },+ final_text_chars: {+ type: "integer",+ minimum: 0,+ description: "Claimed codepoint length of the assembled pack text.",+ },+ },+ additionalProperties: false,+ },host: { type: "string", description: "Optional host/runtime label; also a filter." },session_id: { type: "string", description: "Optional session id recorded on the row." },turn_id: { type: "string", description: "Optional turn id recorded on the row." },+ agent_id: {+ type: "string",+ description:+ "post (optional): the ACTING agent, recorded on all three rows this post lands. Self-asserted, never a verifier; omitted records no actor rather than a guess.",+ },since: { type: "string", description: "Optional inclusive lower timestamp bound." },until: { type: "string", description: "Optional inclusive upper timestamp bound." },limit: {diff --git a/src/mcp/brain/write-batch-tools.ts b/src/mcp/brain/write-batch-tools.tsindex 97fd2805..fc2c4450 100644--- a/src/mcp/brain/write-batch-tools.ts+++ b/src/mcp/brain/write-batch-tools.ts@@ -184,7 +184,7 @@ export const WRITE_BATCH_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([{name: "brain_write_batch",description:- "Apply an ordered batch of Brain write operations atomically. Ops: create_note, update_note, append_note, apply_evidence, append_log_line. Validated and projected in memory first; the first invalid op aborts with a typed error naming its index and nothing is written.",+ "Apply an ordered batch of Brain write operations atomically. Ops: create_note, update_note, append_note, apply_evidence, append_log_line. Validated first; the first invalid op aborts with a typed error naming its index and nothing is written. Note ops honour the declared write binding.",inputSchema: {type: "object",properties: {diff --git a/src/mcp/registry-guard.ts b/src/mcp/registry-guard.tsindex 649415f7..f283cc22 100644--- a/src/mcp/registry-guard.ts+++ b/src/mcp/registry-guard.ts@@ -84,7 +84,8 @@ export const PREVIEW_BUDGET_EXEMPT: Readonly<Record<string, string>> = Object.frbrain_apply_evidence: "write; returns a small fixed-shape ack",brain_note: "write; returns a small fixed-shape ack",brain_observed_use: "write; returns a small fixed-shape ack (records/aggregates count)",- brain_create_note: "write; returns the created note path and a created flag",+ brain_create_note:+ "write; returns the created note path, a created flag, and the outcome discriminant",brain_update_note: "write; returns the updated note path and an updated flag",brain_append_note: "write; returns the appended note path and an appended flag",brain_write_batch: "write; returns a bounded per-operation result list and an applied count",@@ -92,7 +93,6 @@ export const PREVIEW_BUDGET_EXEMPT: Readonly<Record<string, string>> = Object.frbrain_recall_feedback: "write; returns one event receipt plus bounded weights",brain_switch_vault: "write; returns a small profile ack",brain_write_session: "lifecycle ops return one fixed-shape envelope; prompts are kernel-bounded",- schema_apply_mutations: "write; returns a bounded mutation receipt",brain_intake_entities: "write; returns created/updated id lists and a relation count",brain_ingest_source: "write; returns the summary path plus bounded id lists",brain_distill_source: "write; returns the distillation path plus a claim count and source hash",@@ -101,6 +101,12 @@ export const PREVIEW_BUDGET_EXEMPT: Readonly<Record<string, string>> = Object.frbrain_memory_bridge:"write; returns a small fixed-shape receipt (recorded flag, kind, count, ids)",+ // Bounded by the operator-curated schema block, not by a receipt. Under+ // `dry_run: true` this tool is not a write at all and returns no receipt:+ // it returns the entire resulting pack plus a leaf-level diff.+ schema_apply_mutations:+ "returns the whole resulting schema pack - plus a leaf diff under dry_run, which writes nothing; bounded by the size of the schema block in Brain/_brain.yaml, which the operator curates",+// Bounded-by-construction reads.second_brain_capabilities: "fixed-size capability report",second_brain_status:diff --git a/src/mcp/schema-tools.ts b/src/mcp/schema-tools.tsindex d42f7a18..4bb9619a 100644--- a/src/mcp/schema-tools.ts+++ b/src/mcp/schema-tools.ts@@ -9,10 +9,10 @@ import {listSchemaPacks,reviewSchemaOrphans,} from "../core/brain/schema-admin.ts";-import type { SchemaMutation } from "../core/brain/schema-mutate.ts";+import { previewSchemaMutations, type SchemaMutation } from "../core/brain/schema-mutate.ts";import { resolveSearchConfig } from "../core/search/index.ts";import { INVALID_PARAMS, MCPError } from "./protocol.ts";-import { coerceStr } from "./coerce.ts";+import { coerceBool, coerceStr } from "./coerce.ts";import { MCP_PREVIEW_BUDGET } from "./preview-budget.ts";import type { ServerContext, ToolDefinition } from "./tool-contract.ts";@@ -77,7 +77,7 @@ export const SCHEMA_TOOLS: ReadonlyArray<ToolDefinition> = [{name: "schema_apply_mutations",description:- "Apply an atomic batch of schema mutations to Brain/_brain.yaml and write an audit record.",+ "Apply an atomic batch of schema mutations to Brain/_brain.yaml and write an audit record. With dry_run, returns the pack that would result and its diff instead, writing nothing.",inputSchema: {type: "object",properties: {@@ -94,6 +94,11 @@ export const SCHEMA_TOOLS: ReadonlyArray<ToolDefinition> = [type: "string",description: "Optional audit reason.",},+ dry_run: {+ type: "boolean",+ description:+ "Preview WITHOUT writing: the pack that would result plus its diff, same validator rejections, no config write, no audit record. Absent by default.",+ },},required: ["mutations"],additionalProperties: false,@@ -102,13 +107,18 @@ export const SCHEMA_TOOLS: ReadonlyArray<ToolDefinition> = [let mutations: SchemaMutation[];let actor: string;let reason: string | undefined;+ let dryRun: boolean;try {mutations = coerceSchemaMutations(args["mutations"]);actor = coerceStr(args, "actor", false, "mcp")!;reason = coerceStr(args, "reason", false) ?? undefined;+ dryRun = coerceBool(args, "dry_run");} catch (err) {throw new MCPError(INVALID_PARAMS, (err as Error).message);}+ // The preview shape carries no `audit_path` and no `applied` count, so+ // a dry run can never be read back as a mutation that landed.+ if (dryRun) return previewSchemaMutations(ctx.vault, mutations);return await applySchemaAdminMutations(ctx.vault, mutations, {actor,reason,diff --git a/src/mcp/skill-tools.ts b/src/mcp/skill-tools.tsindex 8e812113..25f1ffd5 100644--- a/src/mcp/skill-tools.ts+++ b/src/mcp/skill-tools.ts@@ -16,6 +16,7 @@ import {resolveSkillsAttachTriggers,} from "../core/config.ts";import { buildSkillAttachment } from "../core/surface/skill-attach.ts";+import { isSkillOfferId, SKILL_OFFER_ID_KEY } from "../core/surface/skill-offer.ts";import { discoverSkills, readSkillFile, skillRoots, SkillError } from "../core/surface/skills.ts";import { coerceInt, coerceStr } from "./coerce.ts";import { MCP_PREVIEW_BUDGET } from "./preview-budget.ts";@@ -35,13 +36,36 @@ function toolListSkills(ctx: ServerContext): Record<string, unknown> {name: s.name,description: s.description,path: s.path,+ // Same-named skills in an earlier root: the entry that won, plus where+ // the copies it hid live. Empty means discovery found none.+ shadowed: s.shadowed,})),};}+/**+ * The offer this fetch cites, or undefined when it cites none.+ *+ * A present-but-malformed value is refused rather than dropped: the caller is+ * here and can be told, and silently continuing would put a skill fetch into+ * the runtime's session log looking exactly like one that cited no offer.+ */+function readCitedOffer(args: Record<string, unknown>): string | undefined {+ const raw = args[SKILL_OFFER_ID_KEY];+ if (raw === undefined || raw === null) return undefined;+ if (!isSkillOfferId(raw)) {+ throw new MCPError(+ INVALID_PARAMS,+ `${SKILL_OFFER_ID_KEY} must be an offer id returned by skills_attach`,+ );+ }+ return raw;+}+function toolGetSkill(ctx: ServerContext, args: Record<string, unknown>): Record<string, unknown> {const name = coerceStr(args, "name", true)!;const filePath = coerceStr(args, "file_path", false) ?? undefined;+ const citedOffer = readCitedOffer(args);const skills = discoverSkills(rootsFor(ctx));const skill = skills.find((s) => s.name === name);if (skill === undefined) {@@ -60,6 +84,7 @@ function toolGetSkill(ctx: ServerContext, args: Record<string, unknown>): Recorddescription: skill.description,path: skill.path,...(filePath !== undefined ? { file_path: filePath } : {}),+ ...(citedOffer !== undefined ? { [SKILL_OFFER_ID_KEY]: citedOffer } : {}),content,};}@@ -71,7 +96,7 @@ function toolSkillsAttach(const query = coerceStr(args, "query", true)!;const maxSkills = coerceInt(args, "max_skills", 3, 1, 10);if (!resolveSkillAutoAttach(ctx.configPath ?? undefined)) {- return { enabled: false, block: "", skills: [] };+ return { enabled: false, block: "", skills: [], [SKILL_OFFER_ID_KEY]: null };}const includeTriggers = resolveSkillsAttachTriggers(ctx.configPath ?? undefined);const attachment = buildSkillAttachment({@@ -83,6 +108,9 @@ function toolSkillsAttach(return {enabled: true,block: attachment.block,+ // Identity of this offer, for the agent to cite back on get_skill so the+ // invocation can be joined to it. Null when nothing was offered.+ [SKILL_OFFER_ID_KEY]: attachment.offerId,skills: attachment.items.map((item) => ({name: item.name,description: item.description,@@ -96,7 +124,7 @@ export const SKILL_TOOLS: ReadonlyArray<ToolDefinition> = [{name: "list_skills",description:- "List agent skills shipped with Open Second Brain (and vault-local Brain/skills/) with one-line descriptions. Read-only.",+ "List agent skills shipped with Open Second Brain (and vault-local Brain/skills/) with one-line descriptions. Each entry carries a shadowed array naming the same-named skill directories it overrode, empty when discovery found none. Read-only.",inputSchema: {type: "object",properties: {},@@ -108,7 +136,7 @@ export const SKILL_TOOLS: ReadonlyArray<ToolDefinition> = [{name: "get_skill",description:- "Fetch a skill's SKILL.md content by name; optional file_path reads an auxiliary file inside the same skill directory. Read-only.",+ "Fetch a skill's SKILL.md content by name; optional file_path reads an auxiliary file inside the same skill directory. An offer_id from skills_attach is echoed back on the result; a malformed one is refused, never dropped. Read-only.",inputSchema: {type: "object",properties: {@@ -120,6 +148,11 @@ export const SKILL_TOOLS: ReadonlyArray<ToolDefinition> = [type: "string",description: "Optional relative path to an auxiliary file inside the skill directory.",},+ [SKILL_OFFER_ID_KEY]: {+ type: "string",+ description:+ "Optional offer_id from skills_attach, recording which offer this fetch came from.",+ },},required: ["name"],additionalProperties: false,@@ -129,7 +162,7 @@ export const SKILL_TOOLS: ReadonlyArray<ToolDefinition> = [{name: "skills_attach",description:- "Score available skills against the current turn text and return a char-budgeted block of relevant skill summaries. Returns enabled:false unless skill_auto_attach is configured.",+ "Score skills against the current turn text; returns a char-budgeted block of relevant summaries and the offer_id to cite on get_skill (null when none was offered). A candidate matching only terms most of the corpus carries is dropped, not offered. enabled:false unless skill_auto_attach is set.",inputSchema: {type: "object",properties: {diff --git a/src/openclaw/index.ts b/src/openclaw/index.tsindex 12846138..04fc561c 100644--- a/src/openclaw/index.ts+++ b/src/openclaw/index.ts@@ -16,6 +16,7 @@ import { probeVaultDirectory } from "../core/vault-presence.ts";import { doctor } from "../core/doctor.ts";import { buildReminder } from "../core/identity-reminder.ts";import { listVaultPages } from "../core/vault.ts";+import { ENTITY_STATUS_SCOPE, vaultPageInStatusScope } from "../core/brain/entities/page-scope.ts";import { deriveRuntimeAgentName, normalizeAgentArgument } from "../core/agent-identity.ts";import { vaultRelative as vaultRelativePath } from "../core/path-safety.ts";@@ -117,7 +118,14 @@ export default definePluginEntry({const limit = typeof params["limit"] === "number" ? (params["limit"] as number) : 50;if (limit < 1 || limit > 500) throw new Error("argument 'limit' must be between 1 and 500");- const pages = listVaultPages(vault);+ // Entity records under quarantine entered the vault under untrusted+ // provenance, so their title is attacker-chosen. This listing filters+ // by title, which is exactly the surface that would hand one back, and+ // it is a page walker rather than an entity reader so the entity read+ // census does not cover it. Non-entity pages are unaffected.+ const pages = listVaultPages(vault).filter((p) =>+ vaultPageInStatusScope(p.metadata, ENTITY_STATUS_SCOPE.readable),+ );const needle = pattern ? pattern.toLowerCase() : null;const matched = (needle === null ? pages : pages.filter((p) => p.title.toLowerCase().includes(needle))diff --git a/tests/cli/brain-schema-cli.test.ts b/tests/cli/brain-schema-cli.test.tsindex 9bb38c56..1499ca57 100644--- a/tests/cli/brain-schema-cli.test.ts+++ b/tests/cli/brain-schema-cli.test.ts@@ -1,5 +1,13 @@import { afterEach, beforeEach, describe, expect, test } from "bun:test";-import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";+import {+ mkdirSync,+ mkdtempSync,+ readFileSync,+ rmSync,+ statSync,+ utimesSync,+ writeFileSync,+} from "node:fs";import { tmpdir } from "node:os";import { join } from "node:path";@@ -173,6 +181,48 @@ describe("o2b brain schema", () => {expect(JSON.parse(report.stdout).vocabulary.preference_types).toContain("strategy");});+ test("apply --dry-run previews the resulting pack and writes nothing", async () => {+ const mutation = JSON.stringify({+ op: "add_type",+ category: "preference_types",+ token: "strategy",+ });+ const brainConfig = join(vault, "Brain", "_brain.yaml");+ // Stamped far in the past so any write during the preview moves the+ // modification time to now rather than staying within timestamp noise.+ const pastSeconds = Date.UTC(2020, 0, 1) / 1000;+ utimesSync(brainConfig, pastSeconds, pastSeconds);+ // Base64 so the later comparison is over raw bytes, not decoded text.+ const bytesBefore = readFileSync(brainConfig).toString("base64");+ const mtimeBefore = statSync(brainConfig).mtimeMs;++ const preview = await runCli(+ ["brain", "schema", "apply", "--mutation", mutation, "--dry-run", "--json"],+ { env: env() },+ );++ expect(preview.returncode).toBe(0);+ const payload = JSON.parse(preview.stdout) as {+ dry_run: boolean;+ would_apply: number;+ pack: { declarations: { preference_types: string[] } };+ diff: Array<{ path: string; before: string | null; after: string | null }>;+ audit_path?: string;+ };+ expect(payload.dry_run).toBe(true);+ expect(payload.would_apply).toBe(1);+ expect(payload.pack.declarations.preference_types).toContain("strategy");+ expect(payload.diff).toEqual([+ { path: "declarations.preference_types", before: null, after: "strategy" },+ ]);+ expect(payload.audit_path).toBeUndefined();++ expect(readFileSync(brainConfig).toString("base64")).toBe(bytesBefore);+ expect(statSync(brainConfig).mtimeMs).toBe(mtimeBefore);+ const report = await runCli(["brain", "schema", "--json"], { env: env() });+ expect(JSON.parse(report.stdout).vocabulary.preference_types).not.toContain("strategy");+ });+test("rejects unknown schema subcommands", async () => {const result = await runCli(["brain", "schema", "reprot", "--json"], {env: env(),diff --git a/tests/cli/search-event-anchor-backfill.test.ts b/tests/cli/search-event-anchor-backfill.test.tsnew file mode 100644index 00000000..87f65156--- /dev/null+++ b/tests/cli/search-event-anchor-backfill.test.ts@@ -0,0 +1,199 @@+/**+ * `o2b search event-anchor-backfill` (provenance at the boundary).+ *+ * The event-anchor phase run on its own. A schema bump migrates the+ * index in place and reindexes nothing, and the indexer resolves an+ * anchor only below its two content-identity fastpaths, so a document+ * carried over from a pre-anchor binary never gets one - and the column+ * now gates a hard `since` / `until` filter. This verb is the+ * operator-facing way to close that, and the index run is the surface+ * that tells an operator it is open.+ *+ * It follows the `vector-backfill` contract exactly: a `plan*` function+ * in core does the finding, dry-run is the DEFAULT, `--apply` is the+ * only mutating path, both report shapes exist, and the registered log+ * event's append failure surfaces on stderr instead of being swallowed.+ *+ * Nothing here reaches the network or an embedding provider: the anchor+ * is a pure function of a note's own frontmatter and body.+ */++import { afterEach, beforeEach, expect, test } from "bun:test";+import { Database } from "bun:sqlite";+import { chmodSync, mkdirSync, readFileSync, readdirSync, utimesSync } from "node:fs";+import { join } from "node:path";++import { bootstrapBrain } from "../../src/core/brain/init.ts";+import { DIAGNOSTIC_SIGNALS } from "../../src/core/brain/diagnostics.ts";+import { indexVault } from "../../src/core/search/indexer.ts";+import { createTempVault, makeConfig, writeMd } from "../helpers/search-fixtures.ts";+import { runCli } from "../helpers/run-cli.ts";++/** Root ignores the directory mode bits the unwritable-log test drives. */+const RUNNING_AS_ROOT = process.getuid?.() === 0;++/** The registered exit for an index holding unexamined documents. */+const PENDING_CODE = "event-anchors-pending";++/** The columns v11 adds; dropping them puts an index back at v10. */+const ANCHOR_COLUMNS = [+ "event_anchor_start_ms",+ "event_anchor_end_ms",+ "event_anchor_source",+ "event_anchor_examined",+];++let vault: string;+let dbPath: string;+let cleanup: () => void;+let configPath: string;++beforeEach(() => {+ const v = createTempVault("cli-event-anchor-backfill");+ vault = v.vault;+ dbPath = v.dbPath;+ cleanup = v.cleanup;+ configPath = join(vault, "cli-config.yaml");+});++afterEach(() => {+ cleanup();+});++/**+ * Build a real index, then put it back in the state the previous binary+ * left it and let the next run migrate it forward - the operator's+ * upgrade, performed for real rather than mocked.+ */+async function indexThenDowngrade(): Promise<void> {+ writeMd(vault, "notes/dated.md", "# Ledger\n\nThe reconciliation entry 2024-03-01 closed it.\n");+ writeMd(vault, "notes/plain.md", "# Ledger\n\nThe reconciliation entry ref-xy-1234 closed it.\n");+ const pinned = new Date("2026-07-01T00:00:00Z");+ for (const rel of ["notes/dated.md", "notes/plain.md"]) {+ utimesSync(join(vault, rel), pinned, pinned);+ }+ await Bun.write(configPath, [`vault: "${vault}"`, ""].join("\n"));+ const config = makeConfig({ vault, dbPath });+ await indexVault(config, { embeddings: false });+ const db = new Database(dbPath);+ for (const col of ANCHOR_COLUMNS) db.run(`ALTER TABLE documents DROP COLUMN ${col}`);+ db.run("UPDATE index_state SET value = '10' WHERE key = 'schema_version'");+ db.close();+ // The write open migrates in place; every file takes the unchanged+ // fastpath, so no anchor is computed for any of them.+ await indexVault(config, { embeddings: false });+}++function cli(...args: string[]): Promise<Awaited<ReturnType<typeof runCli>>> {+ return runCli(["search", ...args, "--vault", vault, "--db", dbPath, "--config", configPath], {+ env: { OPEN_SECOND_BRAIN_CONFIG: configPath },+ });+}++test("the pending state resolves a registered exit rather than a sentence", () => {+ const signal = DIAGNOSTIC_SIGNALS.get(PENDING_CODE);+ expect(signal).toBeDefined();+ expect(signal!.nextCommand).toBe("o2b search event-anchor-backfill --apply");+});++test("the dry run reports the gap on both streams and writes nothing", async () => {+ await indexThenDowngrade();++ const json = await cli("event-anchor-backfill", "--json");+ expect(json.returncode).toBe(0);+ const payload = JSON.parse(json.stdout) as Record<string, unknown>;+ expect(payload["dry_run"]).toBe(true);+ expect(payload["pending"]).toBe(2);+ expect(payload["examined"]).toBe(0);+ expect(payload["documents_total"]).toBe(2);+ // The registered exit travels on the machine stream as a field.+ expect(payload["next_command"]).toBe("o2b search event-anchor-backfill --apply");++ const human = await cli("event-anchor-backfill");+ expect(human.returncode).toBe(0);+ expect(human.stdout).toContain("dry-run");+ expect(human.stdout).toContain("next: o2b search event-anchor-backfill --apply");++ // Still pending: a dry run is a pure read.+ const again = await cli("event-anchor-backfill", "--json");+ expect(JSON.parse(again.stdout)["pending"]).toBe(2);+});++test("an index run over an upgraded vault names the backfill, not the query verb", async () => {+ await indexThenDowngrade();++ const human = await cli("index");+ expect(human.returncode).toBe(0);+ // The run genuinely covered every document - and would have claimed+ // the index was ready to query, sending the operator at the very+ // query the missing anchors answer wrongly.+ expect(human.stdout).toContain("2 document(s) indexed before anchors existed");+ expect(human.stdout).toContain("next: o2b search event-anchor-backfill --apply");+ expect(human.stdout).not.toContain("next: o2b search query");++ const json = await cli("index", "--json");+ const payload = JSON.parse(json.stdout) as Record<string, unknown>;+ expect((payload["stats"] as Record<string, unknown>)["event_anchors_pending"]).toBe(2);+ expect(payload["next_command"]).toBe("o2b search event-anchor-backfill --apply");+});++test("a vault with nothing pending is byte-identical to before the field existed", async () => {+ writeMd(vault, "notes/plain.md", "# Ledger\n\nThe reconciliation entry ref-xy-1234 closed it.\n");+ await Bun.write(configPath, [`vault: "${vault}"`, ""].join("\n"));++ const json = await cli("index", "--json");+ const stats = JSON.parse(json.stdout)["stats"] as Record<string, unknown>;+ expect(stats).not.toHaveProperty("event_anchors_pending");+ expect(JSON.parse(json.stdout)["next_command"]).toBe("o2b search query <text>");++ const human = await cli("index");+ expect(human.stdout).not.toContain("event anchors:");+});++test("--apply examines the pending documents and records a registered log event", async () => {+ await indexThenDowngrade();+ bootstrapBrain(vault, { configPath });++ const applied = await cli("event-anchor-backfill", "--apply", "--json");+ expect(applied.returncode).toBe(0);+ const payload = JSON.parse(applied.stdout) as Record<string, unknown>;+ expect(payload["dry_run"]).toBe(false);+ expect(payload["examined"]).toBe(2);+ // Only one of the two notes declares a date; the other is now KNOWN+ // to declare none rather than merely never looked at.+ expect(payload["anchored"]).toBe(1);+ // Nothing left to do, so no exit is named.+ expect(payload).not.toHaveProperty("next_command");++ const logDir = join(vault, "Brain", "log");+ const logged = readdirSync(logDir)+ .map((f) => readFileSync(join(logDir, f), "utf8"))+ .join("\n");+ expect(logged).toContain("event-anchor-backfill");++ // And the index run that follows is quiet again.+ const after = await cli("index");+ expect(after.stdout).not.toContain("event anchors:");+ expect(after.stdout).toContain("next: o2b search query");+});++test.skipIf(RUNNING_AS_ROOT)(+ "an unwritable Brain log surfaces on stderr rather than being swallowed",+ async () => {+ await indexThenDowngrade();+ bootstrapBrain(vault, { configPath });+ const logDir = join(vault, "Brain", "log");+ mkdirSync(logDir, { recursive: true });+ chmodSync(logDir, 0o500);+ try {+ const applied = await cli("event-anchor-backfill", "--apply");+ // The anchors were still written - telemetry never fails the+ // primary operation - but the operator is told the record did not+ // land, rather than the failure being swallowed.+ expect(applied.returncode).toBe(0);+ expect(applied.stderr).toContain("append event-anchor-backfill log failed");+ } finally {+ chmodSync(logDir, 0o700);+ }+ },+);diff --git a/tests/cli/search-vector-backfill.test.ts b/tests/cli/search-vector-backfill.test.tsnew file mode 100644index 00000000..b9181d8a--- /dev/null+++ b/tests/cli/search-vector-backfill.test.ts@@ -0,0 +1,338 @@+/**+ * `o2b search vector-backfill` (provenance-at-the-boundary, unit F).+ *+ * The vector phase run on its own: an index whose chunks have no+ * `embeddings` row is a first-class, already-resumable state, and this+ * verb is the operator-facing way to close it without re-walking the+ * vault. It follows the `authored-at-backfill` contract exactly - a+ * `plan*` function in core does the finding, dry-run is the DEFAULT,+ * `--apply` is the only mutating path, both report shapes exist, and the+ * registered log event's append failure surfaces on stderr instead of+ * being swallowed.+ *+ * The only embedding endpoint used is the loopback fake the search+ * fixtures ship; nothing here reaches the network.+ */++import { afterEach, beforeEach, expect, test } from "bun:test";+import { chmodSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { bootstrapBrain } from "../../src/core/brain/init.ts";+import { indexVault } from "../../src/core/search/indexer.ts";+import { planVectorBackfill } from "../../src/core/search/vector-backfill.ts";+import { SEMANTIC_CAPABILITY_TIER } from "../../src/core/search/capability-tier.ts";+import { createTempVault, makeConfig, writeMd } from "../helpers/search-fixtures.ts";+import { startFakeHttp, type FakeHttp } from "../helpers/fake-http.ts";+import { sqliteVecLoadable } from "../helpers/sqlite-vec.ts";+import { runCli } from "../helpers/run-cli.ts";++/**+ * Whether `sqlite-vec` loaded in THIS process, resolved once at module+ * load. Every test below needs a vector index, so the answer decides+ * whether this file has any coverage at all - and it is therefore+ * reported, never swallowed: the tests are declared with `skipIf` so a+ * missing extension shows up as SKIPPED in the runner rather than as a+ * body that returned before its first assertion and read as a pass.+ */+const VEC_LOADABLE = sqliteVecLoadable();++/**+ * Set only on a platform that genuinely cannot load `sqlite-vec`. Absent+ * - the default, and what CI runs - an unloadable extension is a FAULT,+ * not a platform fact, and the loadability guard below turns the whole+ * file red instead of letting its coverage evaporate into skips.+ */+const ALLOW_MISSING_VEC_ENV = "O2B_ALLOW_MISSING_SQLITE_VEC";++/** Whether this environment has declared the extension optional. */+const VEC_DECLARED_OPTIONAL = process.env[ALLOW_MISSING_VEC_ENV] !== undefined;++/** Root ignores the directory mode bits the unwritable-log test drives. */+const RUNNING_AS_ROOT = process.getuid?.() === 0;++let vault: string;+let dbPath: string;+let cleanup: () => void;+let server: FakeHttp;++beforeEach(async () => {+ const v = createTempVault("vector-backfill");+ vault = v.vault;+ dbPath = v.dbPath;+ cleanup = v.cleanup;+ server = await startFakeHttp();+});++afterEach(async () => {+ await server.close();+ cleanup();+});++/** A config whose semantic tier is `configured` against the fake server. */+function semanticConfig(): ReturnType<typeof makeConfig> {+ return makeConfig({+ vault,+ dbPath,+ semantic: {+ enabled: true,+ provider: "openai-compat",+ baseUrl: server.url,+ model: "fake-model",+ apiKey: "test-key",+ dimension: 4,+ timeoutMs: 5_000,+ },+ });+}++/** Index the vault WITHOUT embeddings, leaving every chunk vectorless. */+async function indexWithoutVectors(): Promise<void> {+ writeMd(vault, "a.md", "# A\n\nFirst note about something.");+ writeMd(vault, "b.md", "# B\n\nSecond note discussing other things.");+ await indexVault(semanticConfig());+}++test.skipIf(VEC_DECLARED_OPTIONAL)(+ "the vector extension every test in this file depends on is loadable",+ () => {+ expect(VEC_LOADABLE).toBe(true);+ },+);++test.skipIf(!VEC_LOADABLE)(+ "the planner reports pending vectors and writes nothing by default",+ async () => {+ await indexWithoutVectors();++ const plan = await planVectorBackfill(semanticConfig());++ expect(plan.applied).toBe(false);+ expect(plan.capability.tier).toBe(SEMANTIC_CAPABILITY_TIER.configured);+ expect(plan.chunksTotal).toBeGreaterThan(0);+ expect(plan.pending).toBe(plan.chunksTotal);+ expect(plan.embedded).toBe(0);++ // A second dry run sees exactly the same work: nothing was consumed.+ const again = await planVectorBackfill(semanticConfig());+ expect(again.pending).toBe(plan.pending);+ },+);++test.skipIf(!VEC_LOADABLE)(+ "--apply is the only path that computes vectors, and it is resumable",+ async () => {+ await indexWithoutVectors();++ const applied = await planVectorBackfill(semanticConfig(), { apply: true });+ expect(applied.applied).toBe(true);+ expect(applied.embedded).toBe(applied.pending);+ expect(applied.embedded).toBeGreaterThan(0);++ // Idempotent: a re-run finds nothing left to do.+ const rerun = await planVectorBackfill(semanticConfig(), { apply: true });+ expect(rerun.pending).toBe(0);+ expect(rerun.embedded).toBe(0);+ },+);++test.skipIf(!VEC_LOADABLE)(+ "a blocked capability tier is reported by the dry run without attempting anything",+ async () => {+ await indexWithoutVectors();++ // Same index, semantic switched off: the plan still counts the work and+ // names the tier rather than pretending there is none.+ const plan = await planVectorBackfill(makeConfig({ vault, dbPath }));+ expect(plan.capability.tier).toBe(SEMANTIC_CAPABILITY_TIER.disabled);+ expect(plan.pending).toBeGreaterThan(0);+ expect(plan.embedded).toBe(0);+ },+);++test.skipIf(!VEC_LOADABLE)(+ "--apply on a blocked tier refuses with the typed attempt error",+ async () => {+ await indexWithoutVectors();++ let code: string | null = null;+ try {+ await planVectorBackfill(makeConfig({ vault, dbPath }), { apply: true });+ } catch (err) {+ code = (err as { code?: string }).code ?? null;+ }+ expect(code).toBe("EMBEDDING_DISABLED");+ },+);++test.skipIf(!VEC_LOADABLE)("the CLI dry-runs by default and emits both report shapes", async () => {+ await indexWithoutVectors();+ const config = join(vault, "cli-config.yaml");+ await Bun.write(+ config,+ [+ `vault: "${vault}"`,+ "search_semantic_enabled: true",+ "embedding_provider: openai-compat",+ `embedding_base_url: "${server.url}"`,+ "embedding_model: fake-model",+ "embedding_api_key: test-key",+ "embedding_dimension: 4",+ "",+ ].join("\n"),+ );++ const json = await runCli(+ ["search", "vector-backfill", "--vault", vault, "--db", dbPath, "--config", config, "--json"],+ { env: { OPEN_SECOND_BRAIN_CONFIG: config } },+ );+ expect(json.returncode).toBe(0);+ const payload = JSON.parse(json.stdout) as Record<string, unknown>;+ expect(payload["dry_run"]).toBe(true);+ expect(payload["embedded"]).toBe(0);+ expect(payload["pending"]).toBeGreaterThan(0);+ // The registered exit travels on the machine stream as a field.+ expect(payload["next_command"]).toBe("o2b search vector-backfill --apply");++ const human = await runCli(+ ["search", "vector-backfill", "--vault", vault, "--db", dbPath, "--config", config],+ { env: { OPEN_SECOND_BRAIN_CONFIG: config } },+ );+ expect(human.returncode).toBe(0);+ expect(human.stdout).toContain("dry-run");+ expect(human.stdout).toContain("next: o2b search vector-backfill --apply");+});++test.skipIf(!VEC_LOADABLE)(+ "a blocked tier names the tier's exit, never an --apply that cannot succeed",+ async () => {+ await indexWithoutVectors();+ const config = join(vault, "cli-config.yaml");+ // Semantic on, no key: work is pending but `--apply` would refuse.+ await Bun.write(+ config,+ [+ `vault: "${vault}"`,+ "search_semantic_enabled: true",+ "embedding_provider: openai-compat",+ "embedding_model: fake-model",+ "",+ ].join("\n"),+ );++ const json = await runCli(+ ["search", "vector-backfill", "--vault", vault, "--db", dbPath, "--config", config, "--json"],+ { env: { OPEN_SECOND_BRAIN_CONFIG: config } },+ );+ expect(json.returncode).toBe(0);+ const payload = JSON.parse(json.stdout) as Record<string, unknown>;+ expect(payload["capability_tier"]).toBe(SEMANTIC_CAPABILITY_TIER.credentialMissing);+ expect(payload["pending"]).toBeGreaterThan(0);+ expect(payload["next_command"]).toBe("o2b search check");+ },+);++test.skipIf(!VEC_LOADABLE)(+ "the CLI --apply records a registered log event and surfaces an append failure",+ async () => {+ await indexWithoutVectors();+ const config = join(vault, "cli-config.yaml");+ await Bun.write(+ config,+ [+ `vault: "${vault}"`,+ "search_semantic_enabled: true",+ "embedding_provider: openai-compat",+ `embedding_base_url: "${server.url}"`,+ "embedding_model: fake-model",+ "embedding_api_key: test-key",+ "embedding_dimension: 4",+ "",+ ].join("\n"),+ );+ bootstrapBrain(vault, { configPath: config });++ const applied = await runCli(+ [+ "search",+ "vector-backfill",+ "--vault",+ vault,+ "--db",+ dbPath,+ "--config",+ config,+ "--apply",+ "--json",+ ],+ { env: { OPEN_SECOND_BRAIN_CONFIG: config } },+ );+ expect(applied.returncode).toBe(0);+ expect(JSON.parse(applied.stdout)["embedded"]).toBeGreaterThan(0);++ const logDir = join(vault, "Brain", "log");+ const logged = readdirSync(logDir)+ .map((f) => readFileSync(join(logDir, f), "utf8"))+ .join("\n");+ expect(logged).toContain("vector-backfill");+ },+);++test.skipIf(!VEC_LOADABLE || RUNNING_AS_ROOT)(+ "an unwritable Brain log surfaces on stderr rather than being swallowed",+ async () => {+ await indexWithoutVectors();+ const config = join(vault, "cli-config.yaml");+ await Bun.write(+ config,+ [+ `vault: "${vault}"`,+ "search_semantic_enabled: true",+ "embedding_provider: openai-compat",+ `embedding_base_url: "${server.url}"`,+ "embedding_model: fake-model",+ "embedding_api_key: test-key",+ "embedding_dimension: 4",+ "",+ ].join("\n"),+ );+ bootstrapBrain(vault, { configPath: config });+ const logDir = join(vault, "Brain", "log");+ mkdirSync(logDir, { recursive: true });+ chmodSync(logDir, 0o500);+ try {+ const applied = await runCli(+ [+ "search",+ "vector-backfill",+ "--vault",+ vault,+ "--db",+ dbPath,+ "--config",+ config,+ "--apply",+ ],+ { env: { OPEN_SECOND_BRAIN_CONFIG: config } },+ );+ // The vectors were still written - telemetry never fails the primary+ // operation - but the operator is told the record did not land.+ expect(applied.returncode).toBe(0);+ expect(applied.stderr).toContain("vector-backfill log");+ } finally {+ chmodSync(logDir, 0o700);+ }+ },+);++test("a temp directory is never the real vault", () => {+ // Guard on the fixtures themselves: every path this file writes to is+ // created under the OS temp root by `createTempVault`.+ const probe = mkdtempSync(join(tmpdir(), "o2b-vector-backfill-probe-"));+ try {+ expect(probe.startsWith(tmpdir())).toBe(true);+ } finally {+ rmSync(probe, { recursive: true, force: true });+ }+});diff --git a/tests/core/architecture/write-site-census.test.ts b/tests/core/architecture/write-site-census.test.tsnew file mode 100644index 00000000..d93de2da--- /dev/null+++ b/tests/core/architecture/write-site-census.test.ts@@ -0,0 +1,932 @@+/**+ * Unit B — every in-vault write site, measured rather than described.+ *+ * The write binding this release ships covers ONE class of destination:+ * the vault-relative path a caller names. That is a real boundary and it+ * is also a small fraction of what puts bytes into a vault. Shipping it+ * while describing the surface as "enforced" would be theatre: dozens of+ * modules reach `node:fs` directly and pass no shared writer at all —+ * including, before this unit, one that wrote a complete vault note with+ * hand-rolled frontmatter and one inside a CLI verb.+ *+ * How many there are is not written down in this docblock, because a+ * number in prose is the thing that goes stale. {@link+ * DIRECT_WRITE_EXCLUSIONS} below IS the count, and it is exact by+ * construction: it walks the tree, finds every direct-`fs` write into+ * the vault, and requires each to carry a written exclusion naming the+ * category it falls in and the reason it does not route through a shared+ * writer. A new direct writer fails here and is named in the failure.+ *+ * ## The population, defined structurally+ *+ * A module is IN POPULATION when it can address the vault: it lives+ * under one of {@link VAULT_WRITE_ROOTS}, or it is one of+ * {@link VAULT_WRITER_FILES}, or it imports the Brain vault-path+ * vocabulary (`paths.ts`) from anywhere in the tree. The last clause is+ * what stops a new in-vault writer escaping by living somewhere new.+ * Modules that write only outside the vault — the install adapters, the+ * machine-config exporter, the benchmark harness — are out by the rule+ * rather than by twenty hand-written "not a vault" entries.+ *+ * ## What counts as a WRITE+ *+ * A CONTENT write: a call that puts file bytes on disk, or removes,+ * moves, or re-permissions a file. `mkdirSync` is deliberately NOT in+ * the set. A directory holds no content, every atomic writer creates its+ * own parent, and counting it would make this census a list of every+ * module that writes anything — which is what the vault-guard census+ * (`tests/core/brain/vault-guard-census.test.ts`) already is. The two+ * censuses answer different questions: that one asks whether a+ * write-capable MODULE asserts vault identity, this one asks whether a+ * write SITE goes through a shared writer.+ *+ * Only identifiers actually bound from `node:fs` or `node:fs/promises`+ * count, so a local helper that happens to be called `writeFileSync` is+ * not miscounted, and a module that reaches `fs` through the shared+ * writers shows up in the shared class instead. "Bound" covers every+ * form the language offers - several statements in one file, a renamed+ * binding, a namespace import, either quote style - because the detector+ * previously read only a file's FIRST `node:fs` statement and the live+ * vault append in `src/core/brain/git/store.ts` sat on the second one.+ * Runtime writers that need no import (`Bun.write`) are matched on the+ * receiver instead.+ *+ * What it still does not see, stated rather than implied: a write issued+ * through a `node:fs/promises` FileHandle method, or through a binding+ * re-exported by an intermediate module. Both would be new shapes in+ * this tree, and the shape table in "the census can fail" is where a new+ * one gets added the day it appears.+ *+ * ## Why the exclusions are categorised+ *+ * Fifty free-text reasons are a wall nobody reads. Every entry carries a+ * category from a closed vocabulary, so the shape of the gap is legible+ * at a glance — how much of it is append-only ledgers, how much is+ * lifecycle moves that author no content, how much is retention pruning+ * — and the free text says only what the category cannot.+ */++import { describe, expect, test } from "bun:test";+import { readdirSync, readFileSync } from "node:fs";+import { join, relative } from "node:path";++const REPO_ROOT = join(import.meta.dir, "..", "..", "..");+const SRC_ROOT = join(REPO_ROOT, "src");++/** Trees whose modules address the vault by construction. */+const VAULT_WRITE_ROOTS: ReadonlyArray<string> = Object.freeze([+ "src/core/brain/",+ "src/core/search/",+ "src/cli/brain/",+ "src/mcp/",+]);++/** Vault writers that live outside those trees, named individually. */+const VAULT_WRITER_FILES: ReadonlyArray<string> = Object.freeze([+ "src/core/vault.ts",+ "src/core/fs-atomic.ts",+]);++/** The vault path vocabulary. Importing it is addressing the vault. */+const VAULT_PATHS_IMPORT_RE = /^[ \t]*import\b[^;]*?\bfrom\s*"\.[^"]*\bpaths\.ts"/m;++/**+ * Calls that put file bytes on disk, or remove, move, or re-permission a+ * file. See the docblock for why `mkdirSync` is not among them.+ */+const CONTENT_WRITE_CALLS: ReadonlyArray<string> = Object.freeze([+ // `node:fs`+ "writeFileSync",+ "appendFileSync",+ "writeSync",+ "renameSync",+ "rmSync",+ "unlinkSync",+ "rmdirSync",+ "cpSync",+ "copyFileSync",+ "createWriteStream",+ "truncateSync",+ "ftruncateSync",+ "symlinkSync",+ "linkSync",+ "utimesSync",+ "chmodSync",+ // `node:fs/promises` twins of the same operations. Nothing in `src/`+ // imports them today; they are here so the FIRST one is reported rather+ // than admitted. The import gate below is what keeps a local helper+ // named `rm` or `link` from being miscounted as one of these.+ "writeFile",+ "appendFile",+ "rename",+ "rm",+ "unlink",+ "rmdir",+ "cp",+ "copyFile",+ "truncate",+ "symlink",+ "link",+ "utimes",+ "chmod",+]);++const CONTENT_WRITE_SET: ReadonlySet<string> = new Set(CONTENT_WRITE_CALLS);++/**+ * Writers reachable with no import at all, so no import gate can see them.+ * Named in full (`Bun.write`) because the receiver is what identifies them.+ */+const GLOBAL_WRITE_CALLS: ReadonlyArray<string> = Object.freeze(["Bun.write"]);++/** The shared writers a site may route through instead. */+const SHARED_WRITE_CALLS: ReadonlyArray<string> = Object.freeze([+ "atomicWriteFileSync",+ "atomicWriteText",+ "atomicCreateFileSyncExclusive",+ "writeFrontmatterAtomic",+ "writeFrontmatter",+]);++/** A dotted call name as a pattern that tolerates spacing around the dot. */+function callPattern(name: string): string {+ return name.split(".").join(String.raw`\s*\.\s*`);+}++/** Longest name first, so a prefix cannot shadow the longer name after it. */+function alternation(names: ReadonlyArray<string>): string {+ return [...names]+ .toSorted((a, b) => b.length - a.length)+ .map(callPattern)+ .join("|");+}++function callRe(names: ReadonlyArray<string>): RegExp {+ return new RegExp(String.raw`\b(${alternation(names)})\s*\(`, "g");+}++/** Member calls on a namespace import, e.g. `fs.writeFileSync(...)`. */+function namespaceCallRe(namespace: string): RegExp {+ return new RegExp(String.raw`\b${namespace}\s*\.\s*(${CONTENT_WRITE_ALTERNATION})\s*\(`, "g");+}++/** The matched call text, with the spacing the source happened to use removed. */+function normalizeCallName(matched: string): string {+ return matched.replace(/\s+/g, "");+}++const CONTENT_WRITE_ALTERNATION = alternation(CONTENT_WRITE_CALLS);+const GLOBAL_WRITE_RE = callRe(GLOBAL_WRITE_CALLS);+const SHARED_WRITE_RE = callRe(SHARED_WRITE_CALLS);++/**+ * Every `node:fs` / `node:fs/promises` import statement, in both binding+ * forms and both quote styles.+ *+ * GLOBAL on purpose. It used to be non-global and read with `.exec()`, so+ * only a module's FIRST `node:fs` statement counted - and the live vault+ * append in `src/core/brain/git/store.ts`, whose `appendFileSync` arrives+ * on the second statement, was invisible to a census that reported a clean+ * sweep. `import type { … }` does not match, because a type cannot be+ * called.+ */+const FS_IMPORT_RE =+ /import\s*(?:\*\s*as\s+([A-Za-z_$][\w$]*)|\{([^}]*)\})\s*from\s*["']node:fs(?:\/promises)?["']/g;++/**+ * Why a direct-`fs` write site does not go through a shared writer.+ * Closed on purpose: a new shape of gap should have to be named, not+ * absorbed into free text.+ */+const WRITE_CATEGORY = Object.freeze({+ /** This module IS a shared writer; the raw call is its implementation. */+ sharedWriterItself: "shared-writer-itself",+ /** Append to a line-oriented ledger. A rewrite would not be an append. */+ appendOnlyLedger: "append-only-ledger",+ /** Move an existing artifact between lifecycle locations; authors no content. */+ lifecycleMove: "lifecycle-move",+ /** Delete an artifact under a retention, reset, or cleanup policy. */+ retentionDelete: "retention-delete",+ /** Machine-readable artifact (JSON, JSONL, sqlite, archive) - never a note. */+ machineArtifact: "machine-artifact",+ /** A lock, marker, or other concurrency primitive. */+ lockPrimitive: "lock-primitive",+ /** Bulk copy of a whole subtree, in or out. */+ archiveTransfer: "archive-transfer",+ /** Permission or metadata change on an existing file; writes no bytes. */+ metadataOnly: "metadata-only",+} as const);++type WriteCategory = (typeof WRITE_CATEGORY)[keyof typeof WRITE_CATEGORY];++interface WriteExclusion {+ /** Every category this file's direct calls fall in. Never empty. */+ readonly categories: ReadonlyArray<WriteCategory>;+ /** The `node:fs` calls this file makes directly, sorted. */+ readonly calls: ReadonlyArray<string>;+ /** What the categories alone do not say. Never empty. */+ readonly reason: string;+}++const C = WRITE_CATEGORY;++/**+ * Every in-vault write site that reaches `node:fs` directly, with the+ * argument for why it is not behind a shared writer.+ *+ * This is the record the write binding cannot cover. Two entries were+ * removed by this unit rather than written: `core/brain/handoff.ts` now+ * emits its frontmatter through `writeFrontmatterAtomic` instead of+ * hand-rolling the block, and `cli/brain/verbs/links.ts` rewrites notes+ * through `atomicWriteFileSync` instead of truncating them in place.+ */+const DIRECT_WRITE_EXCLUSIONS: Readonly<Record<string, WriteExclusion>> = Object.freeze({+ // --- The shared writers themselves ------------------------------------+ "src/core/fs-atomic.ts": {+ categories: [C.sharedWriterItself],+ calls: ["linkSync", "renameSync", "unlinkSync", "writeSync"],+ reason:+ "this module IS the atomic-write pipeline every other site routes through; " ++ "the raw calls are its implementation of temp-file plus link(2)/rename(2).",+ },+ "src/core/vault.ts": {+ categories: [C.sharedWriterItself],+ calls: ["writeFileSync"],+ reason:+ "`writeFrontmatter`, the deliberately non-atomic sibling of " ++ "`writeFrontmatterAtomic`, used where a torn file on crash is acceptable " ++ "because the target is regenerated on the next run.",+ },++ // --- Append-only ledgers ----------------------------------------------+ "src/core/brain/capture/telegram-capture.ts": {+ categories: [C.appendOnlyLedger],+ calls: ["appendFileSync"],+ reason: "one JSON line per capture-routing decision, appended to the decision log.",+ },+ "src/core/brain/continuity/store.ts": {+ categories: [C.appendOnlyLedger],+ calls: ["writeFileSync"],+ reason:+ "`writeFileSync(..., { flag: 'a' })` under the shard lock - an O_APPEND write, " ++ "not a rewrite. Continuity appends are deliberately unfsynced and fail-open so " ++ "telemetry can never fail the operation it observes.",+ },+ "src/core/brain/decisions/receipts.ts": {+ categories: [C.appendOnlyLedger],+ calls: ["appendFileSync"],+ reason: "one receipt JSON line per decision, appended to a per-config shard.",+ },+ "src/core/brain/diagnostics.ts": {+ categories: [C.appendOnlyLedger],+ calls: ["appendFileSync"],+ reason:+ "the WAL-gap fixer appends the missing terminal phase line to a dream workrun " ++ "log. Additive by contract: the forensic content before the gap is preserved.",+ },+ "src/core/brain/dream-workrun.ts": {+ categories: [C.appendOnlyLedger],+ calls: ["appendFileSync"],+ reason:+ "the write-ahead phase log itself. Rewriting it would destroy the very record " ++ "that lets an interrupted run be detected.",+ },+ "src/core/brain/git/store.ts": {+ categories: [C.appendOnlyLedger],+ calls: ["appendFileSync"],+ reason:+ "one JSON line per ingested commit or tag appended to the per-repo " ++ "`commits.jsonl`, behind `assertVaultIdentityForWrite` and after the dedup pass " ++ "that drops shas already on disk. The `state.json` watermark beside it already " ++ "goes through `atomicWriteFileSync`, because a watermark is a rewrite and a " ++ "record is an append. This entry exists because the detector was fixed: the " ++ "import arrives on the module's SECOND `node:fs` statement, and a non-global " ++ "regex read only the first.",+ },+ "src/core/brain/health/edit-history.ts": {+ categories: [C.appendOnlyLedger],+ calls: ["appendFileSync"],+ reason: "deduplicated edit-history lines appended to the per-page history log.",+ },+ "src/core/brain/idempotency-ledger.ts": {+ categories: [C.appendOnlyLedger],+ calls: ["writeFileSync"],+ reason:+ "`writeFileSync(..., { flag: 'a' })` inside the shard lock: one record line per " ++ "remembered key, appended.",+ },+ "src/core/brain/lineage/ledger.ts": {+ categories: [C.appendOnlyLedger, C.lockPrimitive],+ calls: ["appendFileSync", "unlinkSync"],+ reason:+ "appends one record line; the COMPACTION arm beside it already uses " ++ "`atomicWriteFileSync`, because rewriting is a different operation from " ++ "appending. The `unlinkSync` breaks a lock proven stale by its own mtime, with " ++ "exactly one retry.",+ },+ "src/core/brain/maintenance/journal.ts": {+ categories: [C.appendOnlyLedger],+ calls: ["appendFileSync", "renameSync", "writeFileSync"],+ reason:+ "O_APPEND per line so gate-refusal writers - which run BEFORE the lease is " ++ "held - interleave instead of racing a read-modify-rewrite. The tmp-plus-rename " ++ "sweep is the one safe rewrite point and runs only while the lease is held.",+ },+ "src/core/brain/metrics.ts": {+ categories: [C.appendOnlyLedger],+ calls: ["appendFileSync"],+ reason: "one JSON line per surface metrics record.",+ },+ "src/core/brain/pref-audit.ts": {+ categories: [C.appendOnlyLedger],+ calls: ["appendFileSync"],+ reason: "one rendered line per preference mutation, appended to that preference's audit.",+ },+ "src/core/brain/query-demand.ts": {+ categories: [C.appendOnlyLedger],+ calls: ["appendFileSync"],+ reason: "one JSON line per query-demand record, appended under the log lock.",+ },+ "src/core/brain/recurrence.ts": {+ categories: [C.appendOnlyLedger],+ calls: ["appendFileSync"],+ reason: "one JSON line per recurrence event.",+ },++ // --- Lifecycle moves: an existing artifact changes location ------------+ "src/core/brain/capture/capture-note.ts": {+ categories: [C.lifecycleMove],+ calls: ["renameSync", "rmSync"],+ reason:+ "archives a staged capture into `processed/`. The `rmSync` is the source half of " ++ "the cross-device copy-then-remove fallback taken when `rename(2)` cannot span " ++ "filesystems.",+ },+ "src/core/brain/dead-ends.ts": {+ categories: [C.lifecycleMove],+ calls: ["renameSync"],+ reason: "moves dead-end notes past the active cap into `archive/`; the bytes are unchanged.",+ },+ "src/core/brain/dream-apply.ts": {+ categories: [C.lifecycleMove],+ calls: ["renameSync"],+ reason: "moves every consumed signal out of `inbox/` into its processed location.",+ },+ "src/core/brain/dream-stage.ts": {+ categories: [C.lifecycleMove, C.retentionDelete],+ calls: ["renameSync", "rmSync"],+ reason:+ "renames a whole staged bundle DIRECTORY into `applied/` - a directory move has " ++ "no file-writer form - and discards a bundle the operator dropped.",+ },+ "src/core/brain/intentions.ts": {+ categories: [C.lifecycleMove],+ calls: ["renameSync", "rmSync"],+ reason:+ "archives an active intention into `history/` under a collision-suffixed name, " ++ "with the same cross-device copy-then-remove fallback.",+ },+ "src/core/brain/obligations.ts": {+ categories: [C.lifecycleMove],+ calls: ["renameSync", "rmSync"],+ reason:+ "archives a closed obligation; the fallback arm writes the copy through " ++ "`atomicWriteFileSync` and removes the source.",+ },+ "src/core/brain/pending.ts": {+ categories: [C.lifecycleMove],+ calls: ["unlinkSync"],+ reason:+ "removes the source only AFTER `atomicCreateFileSyncExclusive` landed the " ++ "destination, so the exclusive create is the real gate and the unlink cannot " ++ "lose the record.",+ },+ "src/core/brain/preference.ts": {+ categories: [C.lifecycleMove],+ calls: ["unlinkSync"],+ reason:+ "removes the source only after `writeFrontmatterAtomic` landed the retired copy " ++ "AND its presence on disk was re-confirmed - the retire is a move whose write " ++ "half already goes through the shared writer.",+ },+ "src/core/brain/recompile.ts": {+ categories: [C.lifecycleMove],+ calls: ["renameSync"],+ reason: "moves an orphaned generated page into the cleanup directory under a free name.",+ },+ "src/core/brain/signal-retire.ts": {+ categories: [C.lifecycleMove],+ calls: ["unlinkSync"],+ reason:+ "the signal twin of the preference retire: shared-writer write first, existence " ++ "re-confirmed, then the source unlinked.",+ },+ "src/core/search/store/lifecycle.ts": {+ categories: [C.lifecycleMove],+ calls: ["renameSync"],+ reason:+ "restores the `.bak` sqlite index over a missing database after a crashed " ++ "reindex, re-checked under the lock. A database file has no text-writer form.",+ },++ // --- Retention, reset and cleanup deletes ------------------------------+ "src/core/brain/entities/label-hygiene.ts": {+ categories: [C.retentionDelete],+ calls: ["rmSync"],+ reason:+ "prunes entity records inside `withDestructiveSnapshot`, which is the safety " ++ "mechanism here; the surviving records' rewrites go through " ++ "`writeFrontmatterAtomic` in the same block.",+ },+ "src/core/brain/exact-state.ts": {+ categories: [C.retentionDelete],+ calls: ["rmSync"],+ reason: "clears one aspect's exact-state file; the absence IS the cleared state.",+ },+ "src/core/brain/gaps/gap-loop.ts": {+ categories: [C.retentionDelete],+ calls: ["rmSync"],+ reason: "prunes closed gap tasks past their retention window.",+ },+ "src/core/brain/git/ingest.ts": {+ categories: [C.retentionDelete],+ calls: ["rmSync"],+ reason:+ "clears the git ingest `state.json` on a rescan that found no head, so the next " ++ "run starts as an initial walk rather than inheriting a stale marker.",+ },+ "src/core/brain/ingest/checkpoint.ts": {+ categories: [C.retentionDelete],+ calls: ["rmSync"],+ reason: "clears one ingest plan's checkpoint; absence is the terminal state.",+ },+ "src/core/brain/ingest/sources-registry.ts": {+ categories: [C.retentionDelete],+ calls: ["rmSync"],+ reason:+ "deletes a source page after confirming from its own frontmatter that it is one; " ++ "a concurrent delete is reported, not swallowed.",+ },+ "src/core/brain/lineage/identity.ts": {+ categories: [C.retentionDelete],+ calls: ["unlinkSync"],+ reason:+ "clears a worktree identity marker, and prunes the oldest markers past the cap. " ++ "A marker's whole content is its existence.",+ },+ "src/core/brain/link-graph/communities.ts": {+ categories: [C.retentionDelete],+ calls: ["rmSync"],+ reason:+ "removes generated cluster notes the current run no longer expects, and only " ++ "those whose own frontmatter declares the generated kind.",+ },+ "src/core/brain/skill-accept-journal.ts": {+ categories: [C.retentionDelete],+ calls: ["rmSync"],+ reason:+ "removes the accept-journal marker once the accept committed; the path is put " ++ "through `ensureInsideVault` first because the unreadable arm cannot trust the " ++ "slug in the file.",+ },+ "src/core/brain/source-cleanup.ts": {+ categories: [C.retentionDelete],+ calls: ["rmSync"],+ reason: "deletes pages belonging to a removed source, each re-checked by `ensureInsideVault`.",+ },+ "src/core/brain/watchdog.ts": {+ categories: [C.retentionDelete],+ calls: ["rmSync"],+ reason:+ "under `--remediate --apply` only: removes a non-directory occupying a path the " ++ "tree requires to be a directory, immediately before creating it.",+ },+ "src/core/brain/write-session/store.ts": {+ categories: [C.retentionDelete],+ calls: ["rmSync"],+ reason: "deletes write-session files, individually and by sweep. Idempotent by `force`.",+ },+ "src/core/search/tuning-store.ts": {+ categories: [C.retentionDelete],+ calls: ["rmSync"],+ reason: "deletes the persisted tuning state; absence is the reset state.",+ },+ "src/mcp/artifact-store.ts": {+ categories: [C.retentionDelete],+ calls: ["rmSync"],+ reason:+ "prunes preview-artifact directories past their TTL. Directory removal has no " ++ "shared-writer form.",+ },++ // --- Machine-readable artifacts: JSON, sqlite, archives ----------------+ "src/cli/brain/verbs/continuity.ts": {+ categories: [C.machineArtifact],+ calls: ["writeFileSync"],+ reason:+ "`o2b brain continuity export` writes JSONL/JSON export files into an " ++ "operator-named output directory, not vault notes.",+ },+ "src/cli/main.ts": {+ categories: [C.machineArtifact],+ calls: ["writeFileSync"],+ reason:+ "`o2b config export --output` writes a redacted machine-config snapshot to an " ++ "operator-named path. In population only because the module imports the vault " ++ "path vocabulary elsewhere; this write never targets the vault.",+ },+ "src/core/brain/link-graph/bridge-discovery.ts": {+ categories: [C.machineArtifact],+ calls: ["writeFileSync"],+ reason: "rewrites the dismissed-bridge-pairs JSON set in full; there is no note involved.",+ },+ "src/core/brain/link-graph/co-occurrence.ts": {+ categories: [C.machineArtifact],+ calls: ["writeFileSync"],+ reason: "regenerates the co-occurrence JSON artifact in full from the current graph.",+ },+ "src/core/brain/portability/pointer.ts": {+ categories: [C.machineArtifact],+ calls: ["rmSync"],+ reason:+ "the project-side pointer NAMING a vault lives outside that vault " ++ "(`assertNotInsideVault`); removing it is not an in-vault write at all.",+ },+ "src/core/brain/secrets/crypto.ts": {+ categories: [C.machineArtifact],+ calls: ["writeSync"],+ reason:+ "writes key bytes into a descriptor opened `wx` at mode 0600. The exclusivity " ++ "and the mode are the point, and no shared writer takes a mode.",+ },+ "src/core/brain/secrets/store.ts": {+ categories: [C.machineArtifact],+ calls: ["renameSync", "writeFileSync"],+ reason:+ "hand-rolled tmp-plus-rename because it must write at mode 0600, which " ++ "`atomicWriteFileSync` cannot express. Routing it through the shared writer " ++ "would leave the secrets file world-readable for the length of the swap.",+ },+ "src/core/brain/truth/store.ts": {+ categories: [C.appendOnlyLedger, C.machineArtifact, C.retentionDelete],+ calls: ["appendFileSync", "rmSync", "writeFileSync"],+ reason:+ "appends one claim event per line, rewrites the DERIVED state JSON in full, and " ++ "compacts shards by rewriting or removing them.",+ },+ "src/core/search/activation/store.ts": {+ categories: [C.machineArtifact, C.retentionDelete],+ calls: ["rmSync", "writeFileSync"],+ reason:+ "one content-addressed JSON file per access event, the derived activation state " ++ "beside it, and a retention sweep over the events.",+ },+ "src/core/search/embeddings/registry.ts": {+ categories: [C.machineArtifact],+ calls: ["writeFileSync"],+ reason: "rewrites the embedding-provider registry JSON in full, name-sorted for determinism.",+ },+ "src/core/search/feedback.ts": {+ categories: [C.machineArtifact, C.retentionDelete],+ calls: ["rmSync", "writeFileSync"],+ reason:+ "one content-addressed JSON file per recall-feedback event, the derived learned " ++ "weights beside it, and the reset that removes them.",+ },+ "src/core/search/indexer.ts": {+ categories: [C.machineArtifact, C.lifecycleMove],+ calls: ["renameSync", "unlinkSync"],+ reason:+ "swaps a freshly built sqlite index into place under the index lock, keeping the " ++ "previous file as `.bak`. A database is not text and has no shared-writer form.",+ },+ "src/core/search/link-ratchet.ts": {+ categories: [C.archiveTransfer],+ calls: ["cpSync", "rmSync"],+ reason:+ "copies the subject vault into a `mkdtemp` workdir to measure it, then removes " ++ "the workdir. Nothing is written into the vault at all.",+ },+ "src/core/search/reinforce.ts": {+ categories: [C.machineArtifact, C.retentionDelete],+ calls: ["rmSync", "writeFileSync"],+ reason:+ "one content-addressed JSON file per reinforcement event, plus the sweep that " ++ "clears the directory.",+ },+ "src/core/search/rerank/registry.ts": {+ categories: [C.machineArtifact],+ calls: ["writeFileSync"],+ reason: "rewrites the rerank-provider registry JSON in full, name-sorted for determinism.",+ },+ "src/core/search/session-focus.ts": {+ categories: [C.machineArtifact, C.retentionDelete],+ calls: ["rmSync", "writeFileSync"],+ reason: "the per-session focus JSON and the call that clears it.",+ },+ "src/core/search/tuning.ts": {+ categories: [C.machineArtifact],+ calls: ["writeFileSync"],+ reason: "persists the tuning result JSON, keyed by a hash of the dataset it came from.",+ },++ // --- Bulk subtree transfer ---------------------------------------------+ "src/core/brain/init.ts": {+ categories: [C.archiveTransfer],+ calls: ["cpSync"],+ reason:+ "one recursive copy per bootstrap subdirectory into a destination the pre-check " ++ "already proved empty. This is also the path that CREATES the tree, so it can " ++ "not be gated on anything the tree is about to contain.",+ },+ "src/core/brain/snapshot.ts": {+ categories: [C.archiveTransfer, C.retentionDelete],+ calls: ["cpSync", "rmSync", "writeFileSync"],+ reason:+ "writes the compressed archive BYTES into `.snapshots/`, prunes archives past " ++ "the retention count, and restores by recursive copy. A torn archive fails on " ++ "restore exactly as any interrupted snapshot does; no Markdown parser reads it.",+ },++ // --- Concurrency primitives --------------------------------------------+ "src/core/brain/sync-lockfile.ts": {+ categories: [C.lockPrimitive],+ calls: ["unlinkSync", "writeSync"],+ reason:+ "the lock itself. Its semantics come from the exclusive create; the body it " ++ "writes is a diagnostic pid/timestamp stamp whose failure is deliberately " ++ "non-fatal, and the unlink is the release.",+ },++ // --- Metadata only -----------------------------------------------------+ "src/core/brain/health/remediation.ts": {+ categories: [C.metadataOnly],+ calls: ["chmodSync"],+ reason:+ "narrows non-owner permission bits on an existing file or directory. No bytes " ++ "are written, so there is nothing for a writer to make atomic.",+ },++ // --- Mixed: an accept protocol that moves, appends and rolls back ------+ "src/core/brain/skill-proposals.ts": {+ categories: [C.appendOnlyLedger, C.lifecycleMove, C.retentionDelete],+ calls: ["appendFileSync", "rmSync", "unlinkSync"],+ reason:+ "appends one JSON line per verifier rejection; unlinks the pending proposal only " ++ "after the accepted copy landed through `writeFrontmatterAtomic`; and removes " ++ "the accepted copy and its procedure when a journalled accept is rolled back.",+ },+});++interface CensusFile {+ readonly path: string;+ readonly text: string;+}++interface CensusRow {+ readonly path: string;+ /** Direct `node:fs` content-write calls this file makes, sorted, deduped. */+ readonly directCalls: ReadonlyArray<string>;+ /** How many shared-writer calls it makes. */+ readonly sharedCalls: number;+}++/** Every `.ts` file under `src/`, as path + text. */+function readSourceTree(): CensusFile[] {+ const files: CensusFile[] = [];+ const walk = (dir: string): void => {+ for (const entry of readdirSync(dir, { withFileTypes: true })) {+ const abs = join(dir, entry.name);+ if (entry.isDirectory()) walk(abs);+ else if (entry.name.endsWith(".ts")) {+ files.push({+ path: relative(REPO_ROOT, abs).split("\\").join("/"),+ text: readFileSync(abs, "utf8"),+ });+ }+ }+ };+ walk(SRC_ROOT);+ files.sort((a, b) => (a.path < b.path ? -1 : a.path > b.path ? 1 : 0));+ return files;+}++/** Whether this module can address the vault - see the docblock. */+function addressesVault(file: CensusFile): boolean {+ if (VAULT_WRITE_ROOTS.some((root) => file.path.startsWith(root))) return true;+ if (VAULT_WRITER_FILES.includes(file.path)) return true;+ return VAULT_PATHS_IMPORT_RE.test(file.text);+}++interface FsImports {+ /**+ * Local binding -> the `node:fs` name it was imported under. The two+ * differ under `import { writeFileSync as put }`, and matching on the+ * LOCAL name is what makes the renamed form countable at all.+ */+ readonly bindings: ReadonlyMap<string, string>;+ /** Local names of namespace imports, e.g. `fs` in `import * as fs`. */+ readonly namespaces: ReadonlySet<string>;+}++/** What this module actually imported from `node:fs` / `node:fs/promises`. */+function fsImports(text: string): FsImports {+ const bindings = new Map<string, string>();+ const namespaces = new Set<string>();+ for (const match of text.matchAll(FS_IMPORT_RE)) {+ const namespace = match[1];+ if (namespace !== undefined) {+ namespaces.add(namespace);+ continue;+ }+ for (const raw of match[2]!.split(",")) {+ const [imported, local] = raw+ .trim()+ .split(/\s+as\s+/)+ .map((part) => part.trim());+ if (imported === undefined || imported.length === 0) continue;+ bindings.set(local !== undefined && local.length > 0 ? local : imported, imported);+ }+ }+ return { bindings, namespaces };+}++/** The `node:fs` write calls this module makes directly, by their fs name. */+function directWriteCalls(text: string, imported: FsImports): Set<string> {+ const direct = new Set<string>();+ const locals = [...imported.bindings.keys()];+ if (locals.length > 0) {+ for (const match of text.matchAll(callRe(locals))) {+ const name = imported.bindings.get(normalizeCallName(match[1]!));+ if (name !== undefined && CONTENT_WRITE_SET.has(name)) direct.add(name);+ }+ }+ for (const namespace of imported.namespaces) {+ for (const match of text.matchAll(namespaceCallRe(namespace))) {+ direct.add(normalizeCallName(match[1]!));+ }+ }+ for (const match of text.matchAll(GLOBAL_WRITE_RE)) {+ direct.add(normalizeCallName(match[1]!));+ }+ return direct;+}++/** Classify one file into its direct and shared write sites. */+function classify(file: CensusFile): CensusRow | null {+ if (!addressesVault(file)) return null;+ const direct = directWriteCalls(file.text, fsImports(file.text));+ const shared = [...file.text.matchAll(SHARED_WRITE_RE)].length;+ if (direct.size === 0 && shared === 0) return null;+ return { path: file.path, directCalls: [...direct].toSorted(), sharedCalls: shared };+}++function census(files: ReadonlyArray<CensusFile>): CensusRow[] {+ const rows: CensusRow[] = [];+ for (const file of files) {+ const row = classify(file);+ if (row !== null) rows.push(row);+ }+ return rows;+}++const SOURCE_TREE = readSourceTree();+const ROWS = census(SOURCE_TREE);+const DIRECT_ROWS = ROWS.filter((row) => row.directCalls.length > 0);++describe("in-vault write-site census", () => {+ test("every direct-fs write site carries a written exclusion", () => {+ const unlisted = DIRECT_ROWS.filter((row) => !(row.path in DIRECT_WRITE_EXCLUSIONS)).map(+ (row) => `${row.path} [${row.directCalls.join(",")}]`,+ );+ // Named, not counted: the failure has to say which file.+ expect(unlisted).toEqual([]);+ });++ test("no exclusion outlives the site it excuses", () => {+ const present = new Set(DIRECT_ROWS.map((row) => row.path));+ const stale = Object.keys(DIRECT_WRITE_EXCLUSIONS).filter((path) => !present.has(path));+ expect(stale).toEqual([]);+ });++ test("each exclusion still names exactly the calls its site makes", () => {+ // A new KIND of direct write inside an already-excused file is a new+ // decision, not one the existing argument already covered.+ const drifted: string[] = [];+ for (const row of DIRECT_ROWS) {+ const entry = DIRECT_WRITE_EXCLUSIONS[row.path];+ if (entry === undefined) continue;+ if (entry.calls.join(",") !== row.directCalls.join(",")) {+ drifted.push(+ `${row.path}: declared [${entry.calls.join(",")}] found [${row.directCalls.join(",")}]`,+ );+ }+ }+ expect(drifted).toEqual([]);+ });++ test("every exclusion carries a non-empty reason", () => {+ for (const [path, entry] of Object.entries(DIRECT_WRITE_EXCLUSIONS)) {+ expect(`${path}: ${entry.reason.trim().length > 0} ${entry.categories.length > 0}`).toBe(+ `${path}: true true`,+ );+ }+ });++ test("the two writers this unit routed through a shared helper are gone from the record", () => {+ // Both wrote vault content by hand: one a complete note with+ // hand-rolled frontmatter, one a truncating rewrite of notes the+ // operator edits concurrently. Their absence here is the assertion.+ const direct = new Set(DIRECT_ROWS.map((row) => row.path));+ expect(direct.has("src/core/brain/handoff.ts")).toBe(false);+ expect(direct.has("src/cli/brain/verbs/links.ts")).toBe(false);+ });+});++describe("the census can fail", () => {+ /** Run the real census over the real tree plus one synthetic module. */+ function unlistedWith(intruder: CensusFile): string[] {+ return census([...SOURCE_TREE, intruder])+ .filter((row) => row.directCalls.length > 0)+ .filter((row) => !(row.path in DIRECT_WRITE_EXCLUSIONS))+ .map((row) => row.path);+ }++ /**+ * Every source shape a direct write can arrive in. Each must be reported+ * on its own: the detector used to see only the first `node:fs` import+ * statement of a file, in the one binding form and the one quote style,+ * so the shapes below were unreachable by construction and the suite+ * still passed. A shape that stops being detected shows up here as a+ * named failure rather than as a silently smaller census.+ */+ const INTRUDER_SHAPES: ReadonlyArray<readonly [string, string]> = Object.freeze([+ [+ "a plain named import",+ 'import { writeFileSync } from "node:fs";\nwriteFileSync("x", "y");\n',+ ],+ [+ "a second `node:fs` import statement",+ 'import { readFileSync } from "node:fs";\nimport { appendFileSync } from "node:fs";\nappendFileSync("x", "y");\n',+ ],+ ["a renamed binding", 'import { writeFileSync as put } from "node:fs";\nput("x", "y");\n'],+ ["a namespace import", 'import * as fs from "node:fs";\nfs.writeFileSync("x", "y");\n'],+ [+ "the promise API",+ 'import { writeFile } from "node:fs/promises";\nawait writeFile("x", "y");\n',+ ],+ [+ "a single-quoted specifier",+ "import { writeFileSync } from 'node:fs';\nwriteFileSync('x', 'y');\n",+ ],+ ["a runtime built-in needing no import", 'await Bun.write("x", "y");\n'],+ ]);++ for (const [shape, source] of INTRUDER_SHAPES) {+ test(`a new direct writer using ${shape} is reported unlisted`, () => {+ const path = "src/core/brain/synthetic-intruder.ts";+ expect(unlistedWith({ path, text: source })).toEqual([path]);+ });+ }++ test("a type-only fs import is not a write site", () => {+ // The complement of the import gate: naming a type cannot write bytes,+ // and counting it would put every module that annotates a `Dirent` in+ // the record.+ expect(+ classify({+ path: "src/core/brain/synthetic-types-only.ts",+ text: 'import type { WriteFileOptions } from "node:fs";\nexport type T = WriteFileOptions;\n',+ }),+ ).toBeNull();+ });++ test("the live second-statement import stays visible", () => {+ // The concrete defect this detector fix closes, pinned by name: the+ // vault append in the git record store arrives through the module's+ // SECOND `node:fs` statement. A regression to a single-statement read+ // makes this row vanish and the sweep read as clean.+ const row = DIRECT_ROWS.find((candidate) => candidate.path === "src/core/brain/git/store.ts");+ expect(row?.directCalls).toEqual(["appendFileSync"]);+ });++ test("a writer outside every vault root is out of population, not silently listed", () => {+ // The complement of the rule above: over-reach would drag the+ // install adapters in and make the record meaningless.+ const outsider: CensusFile = {+ path: "src/core/install/adapters/synthetic.ts",+ text: 'import { writeFileSync } from "node:fs";\nwriteFileSync("x", "y");\n',+ };+ expect(classify(outsider)).toBeNull();+ });++ test("the detectors still match the shapes they measure", () => {+ // A regex that stopped matching would report a clean sweep over an+ // empty set. Pin the measurement, not only its verdict.+ // Set just under the measurement, not an order of magnitude under it:+ // a floor of 40 against 63 would let a third of the tree stop being+ // seen and still report a clean sweep.+ expect(DIRECT_ROWS.length).toBeGreaterThan(58);+ expect(ROWS.filter((row) => row.sharedCalls > 0).length).toBeGreaterThan(88);+ });+});diff --git a/tests/core/brain.types.test.ts b/tests/core/brain.types.test.tsindex 3b4b496d..1335edfc 100644--- a/tests/core/brain.types.test.ts+++ b/tests/core/brain.types.test.ts@@ -120,6 +120,10 @@ describe("BRAIN_* const enums", () => {"source-citation",// Source pipeline integrity suite (t_bd6cc4cb) guarded doctor repair"doctor-repair",+ // provenance-at-the-boundary unit F (t_76b89833) vector-only backfill+ "vector-backfill",+ // provenance-at-the-boundary (t_ac1c4176) event-anchor-only backfill+ "event-anchor-backfill",]);const actual = new Set<string>(Object.values(BRAIN_LOG_EVENT_KIND));expect(actual).toEqual(expected);diff --git a/tests/core/brain/context-pack-evidence.test.ts b/tests/core/brain/context-pack-evidence.test.tsnew file mode 100644index 00000000..3363b318--- /dev/null+++ b/tests/core/brain/context-pack-evidence.test.ts@@ -0,0 +1,300 @@+import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { emitContextReceipt } from "../../../src/core/brain/context-receipts.ts";+import {+ CONTEXT_PACK_EVIDENCE_BASIS,+ CONTEXT_PACK_EVIDENCE_FIELD,+ CONTEXT_PACK_EVIDENCE_UNRESOLVED,+ CONTEXT_PACK_EVIDENCE_VERDICT,+ judgeContextPackEvidence,+ listContextPackEvidence,+ resolveContextPackEvidence,+} from "../../../src/core/brain/context-pack-evidence.ts";+import {+ emitContextPackOutcome,+ listContextPackOutcomes,+ postContextPackOutcome,+} from "../../../src/core/brain/context-pack-outcome.ts";+import { listTokenImpactOutcomes } from "../../../src/core/brain/token-impact.ts";+import { CONTINUITY_AGENT_ID_KEY } from "../../../src/core/brain/continuity/types.ts";++let tmp: string;+let vault: string;++/** Pack text whose digest the receipt records; the tests never re-hash it. */+const PACKED_TEXT = "preference one\npreference two\n";++beforeEach(() => {+ tmp = mkdtempSync(join(tmpdir(), "o2b-ctx-evidence-"));+ vault = join(tmp, "vault");+ mkdirSync(join(vault, "Brain"), { recursive: true });+});++afterEach(() => {+ rmSync(tmp, { recursive: true, force: true });+});++/** Write one real `context_receipt` and hand back its durable id + payload. */+function seedReceipt(): { sampleId: string; finalTextHash: string; itemCount: number } {+ const record = emitContextReceipt(vault, {+ options: { host: "test", trigger: "context_pack", createdAt: "2026-07-01T00:00:00.000Z" },+ items: [+ { id: "pref-a", path: "Brain/preferences/pref-a.md", text: "preference one" },+ { id: "pref-b", path: "Brain/preferences/pref-b.md", text: "preference two" },+ ],+ finalText: PACKED_TEXT,+ });+ return {+ sampleId: record.id,+ finalTextHash: record.payload["final_text_hash"] as string,+ itemCount: record.payload["item_count"] as number,+ };+}++/** Make the continuity log unreadable: the directory is replaced by a file. */+function breakContinuityLog(): void {+ const dir = join(vault, "Brain", "log");+ mkdirSync(dir, { recursive: true });+ writeFileSync(join(dir, "continuity"), "not a directory\n", "utf8");+}++describe("resolveContextPackEvidence reads the sample's receipt back from disk", () => {+ test("a sample backed by a receipt resolves to the digest the kernel recorded", () => {+ const seeded = seedReceipt();+ const evidence = resolveContextPackEvidence(vault, seeded.sampleId);+ expect(evidence.resolved).toBe(true);+ if (!evidence.resolved) return;+ expect(evidence.basis).toBe(CONTEXT_PACK_EVIDENCE_BASIS.contextReceipt);+ expect(evidence.tokens[CONTEXT_PACK_EVIDENCE_FIELD.finalTextHash]).toBe(seeded.finalTextHash);+ expect(evidence.tokens[CONTEXT_PACK_EVIDENCE_FIELD.itemCount]).toBe(String(seeded.itemCount));+ expect(evidence.tokens[CONTEXT_PACK_EVIDENCE_FIELD.finalTextChars]).toBe(+ String([...PACKED_TEXT].length),+ );+ });++ test("absence and inability-to-read are DIFFERENT answers", () => {+ const absent = resolveContextPackEvidence(vault, "ctn_nothing_here");+ expect(absent.resolved).toBe(false);+ if (absent.resolved) return;+ expect(absent.reason).toBe(CONTEXT_PACK_EVIDENCE_UNRESOLVED.sampleAbsent);++ breakContinuityLog();+ const unreadable = resolveContextPackEvidence(vault, "ctn_nothing_here");+ expect(unreadable.resolved).toBe(false);+ if (unreadable.resolved) return;+ expect(unreadable.reason).toBe(CONTEXT_PACK_EVIDENCE_UNRESOLVED.evidenceUnreadable);+ expect(unreadable.reason).not.toBe(absent.reason);+ });+});++describe("judgeContextPackEvidence compares only what the agent asserted", () => {+ test("an agreeing claim is a match", () => {+ const seeded = seedReceipt();+ const verdict = judgeContextPackEvidence(resolveContextPackEvidence(vault, seeded.sampleId), {+ finalTextHash: seeded.finalTextHash,+ itemCount: seeded.itemCount,+ });+ expect(verdict.verdict).toBe(CONTEXT_PACK_EVIDENCE_VERDICT.match);+ expect(verdict.mismatches).toHaveLength(0);+ });++ test("a contradicted claim is a mismatch naming the field, not a rejection", () => {+ const seeded = seedReceipt();+ const verdict = judgeContextPackEvidence(resolveContextPackEvidence(vault, seeded.sampleId), {+ itemCount: 99,+ });+ expect(verdict.verdict).toBe(CONTEXT_PACK_EVIDENCE_VERDICT.mismatch);+ expect(verdict.mismatches).toHaveLength(1);+ expect(verdict.mismatches[0]).toEqual({+ field: CONTEXT_PACK_EVIDENCE_FIELD.itemCount,+ expected: "99",+ actual: String(seeded.itemCount),+ });+ });++ test("no claim is `unclaimed`, never a silent match", () => {+ const seeded = seedReceipt();+ const evidence = resolveContextPackEvidence(vault, seeded.sampleId);+ expect(judgeContextPackEvidence(evidence).verdict).toBe(+ CONTEXT_PACK_EVIDENCE_VERDICT.unclaimed,+ );+ expect(judgeContextPackEvidence(evidence, {}).verdict).toBe(+ CONTEXT_PACK_EVIDENCE_VERDICT.unclaimed,+ );+ });++ test("evidence the kernel could not obtain is `unresolved`, not a match", () => {+ const verdict = judgeContextPackEvidence(resolveContextPackEvidence(vault, "ctn_absent"), {+ itemCount: 1,+ });+ expect(verdict.verdict).toBe(CONTEXT_PACK_EVIDENCE_VERDICT.unresolved);+ expect(verdict.mismatches).toHaveLength(0);+ });+});++describe("posting an outcome also records the kernel-evidence entry", () => {+ test("the second record joins the outcome on the SAME sample id", () => {+ const seeded = seedReceipt();+ const post = postContextPackOutcome(+ vault,+ { sampleId: seeded.sampleId, firstPassSuccess: true },+ true,+ );+ expect(post).not.toBeNull();+ expect(post!.evidence).not.toBeNull();+ expect(post!.evidence!.kind).toBe("context_pack_evidence");+ // one sample id joins all three ledger rows+ expect(post!.outcome.payload["sample_id"]).toBe(seeded.sampleId);+ expect(post!.evidence!.payload["sample_id"]).toBe(seeded.sampleId);+ expect(listTokenImpactOutcomes(vault)[0]!.payload["pack_id"]).toBe(seeded.sampleId);+ });++ test("the evidence record carries the digest the kernel read from disk", () => {+ const seeded = seedReceipt();+ const post = postContextPackOutcome(+ vault,+ { sampleId: seeded.sampleId, firstPassSuccess: true },+ true,+ );+ const payload = post!.evidence!.payload;+ expect(payload["evidence_basis"]).toBe(CONTEXT_PACK_EVIDENCE_BASIS.contextReceipt);+ expect(payload["kernel_evidence"]).toMatchObject({+ [CONTEXT_PACK_EVIDENCE_FIELD.finalTextHash]: seeded.finalTextHash,+ [CONTEXT_PACK_EVIDENCE_FIELD.itemCount]: String(seeded.itemCount),+ });+ expect(payload["verdict"]).toBe(CONTEXT_PACK_EVIDENCE_VERDICT.unclaimed);+ });++ test("a claim CONTRADICTED by the recomputation is recorded, not rejected or corrected", () => {+ const seeded = seedReceipt();+ const post = postContextPackOutcome(+ vault,+ {+ sampleId: seeded.sampleId,+ firstPassSuccess: true,+ evidenceClaim: { finalTextHash: "0".repeat(64), itemCount: 99 },+ },+ true,+ );+ // the outcome row still landed, unchanged+ expect(listContextPackOutcomes(vault)).toHaveLength(1);+ expect(post!.outcome.payload["first_pass_success"]).toBe(true);+ // and the disagreement is on the record, in full+ const payload = post!.evidence!.payload;+ expect(payload["verdict"]).toBe(CONTEXT_PACK_EVIDENCE_VERDICT.mismatch);+ expect(payload["agent_claim"]).toMatchObject({+ [CONTEXT_PACK_EVIDENCE_FIELD.finalTextHash]: "0".repeat(64),+ [CONTEXT_PACK_EVIDENCE_FIELD.itemCount]: "99",+ });+ expect(payload["kernel_evidence"]).toMatchObject({+ [CONTEXT_PACK_EVIDENCE_FIELD.finalTextHash]: seeded.finalTextHash,+ });+ const mismatches = payload["mismatches"] as ReadonlyArray<Record<string, unknown>>;+ expect(mismatches.map((entry) => entry["field"]).toSorted()).toEqual([+ CONTEXT_PACK_EVIDENCE_FIELD.finalTextHash,+ CONTEXT_PACK_EVIDENCE_FIELD.itemCount,+ ]);+ });++ test("a sample with no receipt on disk records the absence, and the outcome still lands", () => {+ const post = postContextPackOutcome(+ vault,+ { sampleId: "opaque_request_hash", firstPassSuccess: false, repairRequired: true },+ true,+ );+ expect(post!.outcome.payload["sample_id"]).toBe("opaque_request_hash");+ const payload = post!.evidence!.payload;+ expect(payload["verdict"]).toBe(CONTEXT_PACK_EVIDENCE_VERDICT.unresolved);+ expect(payload["unresolved_reason"]).toBe(CONTEXT_PACK_EVIDENCE_UNRESOLVED.sampleAbsent);+ expect("kernel_evidence" in payload).toBe(false);+ });+});++describe("the evidence record names a claimant, never a verifier", () => {+ test("no verifier-identity field, and no wording asserting a second actor", () => {+ const seeded = seedReceipt();+ const post = postContextPackOutcome(+ vault,+ {+ sampleId: seeded.sampleId,+ firstPassSuccess: true,+ agentId: "claude-dev-agent",+ evidenceClaim: { itemCount: seeded.itemCount },+ },+ true,+ );+ const payload = post!.evidence!.payload;+ for (const key of ["verifier_id", "verifier", "witness", "witnessed_by", "attested_by"]) {+ expect(key in payload).toBe(false);+ }+ // the only actor named is the ACTING agent, and it is labelled as such+ expect(payload[CONTINUITY_AGENT_ID_KEY]).toBe("claude-dev-agent");+ const serialized = JSON.stringify(payload).toLowerCase();+ expect(serialized).not.toContain("independent");+ expect(serialized).not.toContain("witness");+ });+});++describe("the ledger stays gated, fail-open and byte-identical when unused", () => {+ test("gate off writes neither an outcome nor an evidence record", () => {+ const seeded = seedReceipt();+ expect(+ postContextPackOutcome(vault, { sampleId: seeded.sampleId, firstPassSuccess: true }, false),+ ).toBeNull();+ expect(listContextPackEvidence(vault)).toHaveLength(0);+ });++ test("a blank sample id fails open and writes no evidence record", () => {+ expect(emitContextPackOutcome(vault, { sampleId: " ", firstPassSuccess: true }, true)).toBe(+ null,+ );+ expect(listContextPackEvidence(vault)).toHaveLength(0);+ });++ test("an outcome posted without an actor omits agent_id rather than inventing one", () => {+ const post = postContextPackOutcome(vault, { sampleId: "s1", firstPassSuccess: true }, true);+ expect(CONTINUITY_AGENT_ID_KEY in post!.outcome.payload).toBe(false);+ expect(CONTINUITY_AGENT_ID_KEY in post!.evidence!.payload).toBe(false);+ });++ test("emitContextPackOutcome keeps returning the outcome record alone", () => {+ const record = emitContextPackOutcome(vault, { sampleId: "s1", firstPassSuccess: true }, true);+ expect(record!.kind).toBe("context_pack_outcome");+ });+});++describe("the outcome row carries the actor field its sibling already had (I2)", () => {+ test("agent_id lands on the outcome row AND on the calibration sibling", () => {+ postContextPackOutcome(+ vault,+ { sampleId: "s1", firstPassSuccess: true, agentId: "claude-dev-agent" },+ true,+ );+ expect(listContextPackOutcomes(vault)[0]!.payload[CONTINUITY_AGENT_ID_KEY]).toBe(+ "claude-dev-agent",+ );+ expect(listTokenImpactOutcomes(vault)[0]!.payload[CONTINUITY_AGENT_ID_KEY]).toBe(+ "claude-dev-agent",+ );+ });+});++describe("listContextPackEvidence", () => {+ test("returns evidence rows newest-first and filters by sample id", () => {+ postContextPackOutcome(+ vault,+ { createdAt: "2026-07-01T00:00:00.000Z", sampleId: "a", firstPassSuccess: true },+ true,+ );+ postContextPackOutcome(+ vault,+ { createdAt: "2026-07-02T00:00:00.000Z", sampleId: "b", firstPassSuccess: true },+ true,+ );+ expect(listContextPackEvidence(vault).map((r) => r.payload["sample_id"])).toEqual(["b", "a"]);+ expect(listContextPackEvidence(vault, { sampleId: "a" })).toHaveLength(1);+ });+});diff --git a/tests/core/brain/continuity/reader-census.test.ts b/tests/core/brain/continuity/reader-census.test.tsnew file mode 100644index 00000000..13a09b26--- /dev/null+++ b/tests/core/brain/continuity/reader-census.test.ts@@ -0,0 +1,264 @@+/**+ * Continuity reader census (provenance-at-the-boundary, unit G,+ * t_77efc212).+ *+ * `docs/observability.md` described the read-model's masking policy as if+ * it governed the read side: `private` records dropped by default, kept+ * only on explicit request. The read-model does exactly that - and almost+ * nothing goes through it. Four modules call+ * `loadNormalizedContinuityRecords`; twenty call `listContinuityRecords`+ * straight off the store, where no `private` drop, no schema-version+ * dispatch, and now no declared-scope filter applies. A policy sentence+ * cannot notice that; a census can.+ *+ * So the split is asserted rather than described. Each side is an explicit+ * inventory: a module that starts reading continuity records - through+ * either door - fails this test until it is listed, and a listed module+ * that stops reading fails it too. The direct readers are RECORDED, not+ * fixed: auditing them is one judgement per reader and is out of scope for+ * this unit by design (see the design document's out-of-scope list).+ *+ * Non-vacuity is proved in the test itself rather than argued: the+ * classifier is run over an injected module that reads through each door+ * and is required to name it. A census that stopped matching would report+ * a clean split over an empty set, which is the failure mode this repo+ * has already shipped once.+ *+ * The classifier reads FILE BYTES, not `grep` output, and that is+ * load-bearing: `src/core/brain/dedup-telemetry.ts` contains a literal NUL+ * in a template literal (a key separator), which makes `grep` treat the+ * whole file as binary and print nothing for it. A grep-built census would+ * silently miss a real reader, so that file's presence is asserted below+ * in its own right.+ */++import { describe, expect, test } from "bun:test";+import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join, relative, resolve } from "node:path";++const REPO_ROOT = resolve(import.meta.dir, "..", "..", "..", "..");+const SRC_ROOT = resolve(REPO_ROOT, "src");++/** Store entry points that return raw records with no policy applied. */+const STORE_READ_RE = /\b(?:listContinuityRecords|paginateContinuityRecords)\s*\(/;+/** Read-model entry points, where masking, schema and scope are applied. */+const READ_MODEL_READ_RE = /\b(?:loadNormalizedContinuityRecords|normalizeContinuityRecord)\s*\(/;++/**+ * The two modules that IMPLEMENT the layer rather than consume it. They+ * match the patterns by construction - the store defines its own reads,+ * and the read-model is the thing that wraps them - so counting them as+ * readers would mean the census could never reach zero unlisted. Their+ * existence is asserted below, so this exclusion cannot outlive them.+ */+const CONTINUITY_LAYER: ReadonlyArray<string> = Object.freeze([+ "src/core/brain/continuity/read-model.ts",+ "src/core/brain/continuity/store.ts",+]);++/**+ * Readers that go through the read-model, and therefore honour the+ * `private` drop, the schema-version dispatch, and the declared-scope+ * filter added by this unit.+ */+const READ_MODEL_READERS: Readonly<Record<string, string>> = Object.freeze({+ "src/cli/brain/verbs/continuity.ts":+ "`o2b brain continuity export|list` - ATOF/ATIF trajectory export and the " ++ "usage-decay-ranked listing.",+ "src/core/bench/phases.ts": "memory bench harness; replays `session_turn` records per question.",+ "src/core/brain/event-trace.ts":+ "joins Brain-log events to their continuity records; passes `keepPrivate` " ++ "through from the caller rather than deciding it.",+ "src/core/brain/skill-usage.ts": "aggregates `skill_invoked` records into per-skill usage.",+});++/**+ * Readers that call the store directly. Every one of them sees `private`+ * records, legacy unstamped records, and scoped records regardless of what+ * any filter asked for. Recorded here as an asserted fact; changing them is+ * a per-reader judgement and a separate task.+ */+const DIRECT_STORE_READERS: Readonly<Record<string, string>> = Object.freeze({+ "src/core/brain/context-pack-evidence.ts":+ "reads `context_pack_evidence` for the kernel-recomputed evidence ledger.",+ "src/core/brain/context-pack-outcome.ts": "reads `context_pack_outcome` for the outcome ledger.",+ "src/core/brain/context-receipts.ts": "reads `context_receipt` for list, summary and show.",+ "src/core/brain/dedup-telemetry.ts":+ "reads `ingest_dedup` rollups. Carries a literal NUL byte, so text tools " ++ "that classify by content type do not see it at all.",+ "src/core/brain/gate-telemetry.ts": "reads `gate_telemetry` for recall-gate summaries.",+ "src/core/brain/generation-reports.ts": "reads `generation_report` for list, summary and show.",+ "src/core/brain/idea-lineage.ts":+ "reads the WHOLE store to build id / turn-id / session-turn indexes.",+ "src/core/brain/link-graph/repair-lane.ts":+ "reads the WHOLE store to harvest `sourceRefs` paths for link repair.",+ "src/core/brain/mcp-route-metrics.ts": "reads `mcp_route_latency` for per-route rollups.",+ "src/core/brain/memory-cost-meter.ts": "reads `host_memory_write` for the write-vs-read ratio.",+ "src/core/brain/observed-use.ts": "reads `recall_observed_use` for observed-use aggregation.",+ "src/core/brain/post-compact-audit.ts":+ "reads `post_compact_audit` for the append-time dedupe-key lookup.",+ "src/core/brain/pre-compact-extract.ts":+ "reads `pre_compact_extract` for the append-time dedupe-key lookup.",+ "src/core/brain/recall-telemetry.ts": "reads `recall_telemetry` for list and summary.",+ "src/core/brain/recent-turns.ts": "reads `recent_turn` for the sequence counter and the window.",+ "src/core/brain/session-recall.ts":+ "reads the WHOLE store for `session_turn` / `session_summary_node`, plus a " ++ "dedupe-key lookup.",+ "src/core/brain/session-summary.ts": "reads `session_summary_digest` for listing and dedupe.",+ "src/core/brain/skill-proposals.ts":+ "reads the WHOLE store forward from a watermark to derive skill proposals.",+ "src/core/brain/temporal/foresight.ts":+ "reads `pre_compact_extract` since a horizon for foresight items.",+ "src/core/brain/token-impact.ts": "reads `token_impact` and `token_impact_outcome` samples.",+});++/**+ * Floors under the MEASUREMENT, not a second copy of the inventories+ * above - those are the exact assertion. These catch the other failure:+ * a classifier that stopped matching would report a clean, empty split+ * and both `toEqual`s would fail on emptiness rather than on a floor, so+ * these say how far the counts may fall before the split stops describing+ * anything.+ *+ * Four modules read through the read-model and twenty read the store+ * directly. The read-model floor is AT today's count because losing one+ * of four is a real regression; the direct floor sits one under, because+ * that set is the one this wave is actively adding to and an exact pin+ * would fail on another unit's honest new reader rather than on a lost+ * measurement.+ */+const MIN_READ_MODEL_READERS = 4;+const MIN_DIRECT_STORE_READERS = 19;++/** A file the classifier must see, and that `grep` alone does not. */++interface CensusRow {+ readonly path: string;+ readonly viaReadModel: boolean;+ readonly viaStore: boolean;+}++interface SourceFile {+ readonly path: string;+ readonly text: string;+}++function sourceFiles(): ReadonlyArray<SourceFile> {+ const files = [...new Bun.Glob("**/*.ts").scanSync({ cwd: SRC_ROOT, absolute: true })].toSorted();+ return files.map((file) => ({+ path: relative(REPO_ROOT, file).split("\\").join("/"),+ text: readFileSync(file, "utf8"),+ }));+}++/**+ * Classify every module that reads the continuity log. `injected` lets a+ * test add a module that does not exist on disk, which is how the+ * non-vacuity assertion below shows the census can actually fail.+ */+function census(injected: ReadonlyArray<SourceFile> = []): ReadonlyArray<CensusRow> {+ const layer = new Set(CONTINUITY_LAYER);+ const rows: CensusRow[] = [];+ for (const file of [...sourceFiles(), ...injected]) {+ if (layer.has(file.path)) continue;+ const viaReadModel = READ_MODEL_READ_RE.test(file.text);+ const viaStore = STORE_READ_RE.test(file.text);+ if (!viaReadModel && !viaStore) continue;+ rows.push({ path: file.path, viaReadModel, viaStore });+ }+ return rows.toSorted((left, right) => left.path.localeCompare(right.path));+}++const pathsVia = (rows: ReadonlyArray<CensusRow>, door: keyof CensusRow): ReadonlyArray<string> =>+ rows.filter((row) => row[door] === true).map((row) => row.path);++const unlisted = (+ rows: ReadonlyArray<CensusRow>,+ door: keyof CensusRow,+ listed: Readonly<Record<string, string>>,+): ReadonlyArray<string> => pathsVia(rows, door).filter((path) => !(path in listed));++describe("continuity reader census", () => {+ test("every read-model reader is listed, and every listed one still reads", () => {+ expect(pathsVia(census(), "viaReadModel")).toEqual(Object.keys(READ_MODEL_READERS).toSorted());+ });++ test("every direct store reader is listed, and every listed one still reads", () => {+ expect(pathsVia(census(), "viaStore")).toEqual(Object.keys(DIRECT_STORE_READERS).toSorted());+ });++ test("the read-model is the minority door, which is what the docs must say", () => {+ const rows = census();+ // The claim `docs/observability.md` now makes, measured rather than+ // asserted: most readers bypass the policy layer entirely.+ expect(pathsVia(rows, "viaStore").length).toBeGreaterThan(+ pathsVia(rows, "viaReadModel").length,+ );+ expect(pathsVia(rows, "viaReadModel").length).toBeGreaterThanOrEqual(MIN_READ_MODEL_READERS);+ expect(pathsVia(rows, "viaStore").length).toBeGreaterThanOrEqual(MIN_DIRECT_STORE_READERS);+ });++ test("the census reads bytes, so a NUL-carrying reader is not invisible to it", () => {+ // `grep` classifies a file holding a NUL byte as binary and prints+ // nothing for it, so a census built on text tooling reports a clean+ // result over a reader it never looked at. Three modules under `src/`+ // carried literal NUL bytes as key separators until this release+ // replaced them with the two-character escape - which is exactly why+ // this is proved against an injected module rather than against+ // whichever file happens to carry one today. The hazard belongs to the+ // tooling, not to those files.+ const nul = String.fromCharCode(0);+ const rogue: SourceFile = {+ path: "src/core/brain/rogue-nul-reader.ts",+ text: `const key = kind + "${nul}" + ref;\nlistContinuityRecords(vault, {});`,+ };+ expect(rogue.text).toContain(nul);+ expect(pathsVia(census([rogue]), "viaStore")).toContain(rogue.path);++ // The half above proves the classifier survives a NUL. This half proves+ // the enumeration does: the census reads every candidate with the same+ // `readFileSync(file, "utf8")` call used below, and a NUL-carrying file+ // read that way keeps its bytes - which is the exact point where a+ // grep-based sweep would have dropped the file instead.+ const dir = mkdtempSync(join(tmpdir(), "osb-nul-"));+ try {+ const probe = join(dir, "nul-reader.ts");+ writeFileSync(probe, rogue.text);+ expect(readFileSync(probe, "utf8")).toContain(nul);+ } finally {+ rmSync(dir, { recursive: true, force: true });+ }+ });++ test("no exclusion outlives the module it excuses", () => {+ const onDisk = new Set(sourceFiles().map((file) => file.path));+ for (const path of CONTINUITY_LAYER) expect(onDisk.has(path)).toBe(true);+ });++ test("the census is demonstrably able to fail", () => {+ // A new reader through either door is named, not absorbed. Injected+ // rather than written to disk so the proof costs nothing and cannot+ // be left behind.+ const rogues: ReadonlyArray<SourceFile> = [+ { path: "src/core/brain/rogue-store-reader.ts", text: "listContinuityRecords(vault, {});" },+ {+ path: "src/core/brain/rogue-read-model-reader.ts",+ text: "loadNormalizedContinuityRecords(vault, {});",+ },+ ];+ const rogueRows = census(rogues);++ expect(unlisted(rogueRows, "viaStore", DIRECT_STORE_READERS)).toEqual([+ "src/core/brain/rogue-store-reader.ts",+ ]);+ expect(unlisted(rogueRows, "viaReadModel", READ_MODEL_READERS)).toEqual([+ "src/core/brain/rogue-read-model-reader.ts",+ ]);++ // ...and the real tree has none, so the assertions above are not+ // passing because the comparison is loose.+ expect(unlisted(census(), "viaStore", DIRECT_STORE_READERS)).toEqual([]);+ expect(unlisted(census(), "viaReadModel", READ_MODEL_READERS)).toEqual([]);+ });+});diff --git a/tests/core/brain/continuity/record-envelope.test.ts b/tests/core/brain/continuity/record-envelope.test.tsnew file mode 100644index 00000000..23d548e9--- /dev/null+++ b/tests/core/brain/continuity/record-envelope.test.ts@@ -0,0 +1,141 @@+/**+ * The continuity record envelope, pinned as a known answer.+ *+ * This file is what survives the caller-declared `scope` field that the+ * provenance-at-the-boundary wave built and then removed: the golden+ * fixture below was captured from the store BEFORE that field existed and+ * is unchanged by its removal, which is the exact property the removal had+ * to preserve. Nothing here mentions a scope, because nothing in the store+ * has one any more - what is pinned is the envelope every writer produces.+ *+ * Two properties carry it:+ *+ * - the envelope is EIGHT fields in one order. `JSON.stringify` emits+ * insertion order, so a field added anywhere - including at the end -+ * changes the bytes on disk and fails here. That is the point: a new+ * envelope field is a schema change, and this test is where it has to+ * be argued for rather than slipped in.+ * - the dedup id is sha-256 over `kind`, `createdAt`, `sourceRefs` and+ * `payload`, and NOTHING else. `docs/observability.md` documents that+ * formula so a reader can reproduce an id; the second assertion below+ * recomputes it from the documented inputs rather than from the+ * implementation, so a hidden input silently joining the hash fails+ * here instead of stranding everyone who reproduced it from the docs.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { createHash } from "node:crypto";+import { mkdtempSync, readFileSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import {+ appendContinuityRecord,+ buildContinuityRecord,+ continuityLogPath,+} from "../../../../src/core/brain/continuity/store.ts";+import { loadNormalizedContinuityRecords } from "../../../../src/core/brain/continuity/read-model.ts";+import type {+ AppendContinuityRecordInput,+ ContinuityRecord,+} from "../../../../src/core/brain/continuity/types.ts";++let vault: string;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-record-envelope-"));+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+});++/** Month shard every fixture below lands in. */+const MONTH = "2026-06";++/** One fixed append input, so every assertion below describes the same write. */+const INPUT: AppendContinuityRecordInput = Object.freeze({+ kind: "recall_telemetry",+ createdAt: "2026-06-14T09:30:00Z",+ sourceRefs: Object.freeze([Object.freeze({ id: "note-a", path: "Brain/notes/a.md" })]),+ payload: Object.freeze({ status: "ok", session_id: "sess-1" }),+});++/**+ * The exact JSONL line {@link INPUT} produces. Pinned as a literal rather+ * than recomputed, because a test that derives its expectation from the+ * implementation cannot notice the implementation moving.+ */+const GOLDEN_LINE =+ '{"schema":"o2b.continuity.v1","id":"ctn_20260614093000_dfa03f037ab006d8",' ++ '"kind":"recall_telemetry","createdAt":"2026-06-14T09:30:00Z",' ++ '"sourceRefs":[{"id":"note-a","path":"Brain/notes/a.md"}],' ++ '"payload":{"status":"ok","session_id":"sess-1"},"private":false,"redacted":false}';++/** The same golden fixture, parsed, for the field-by-field comparison. */+const GOLDEN_RECORD = JSON.parse(GOLDEN_LINE) as ContinuityRecord;++/** Field order of the envelope, in `JSON.stringify` order. */+const GOLDEN_KEYS: ReadonlyArray<string> = Object.freeze([+ "schema",+ "id",+ "kind",+ "createdAt",+ "sourceRefs",+ "payload",+ "private",+ "redacted",+]);++/** Read the month shard back as raw lines, so disk is checked, not memory. */+function shardLines(month: string = MONTH): ReadonlyArray<string> {+ return readFileSync(continuityLogPath(vault, month), "utf8")+ .split("\n")+ .filter((line) => line.trim().length > 0);+}++describe("continuity record envelope", () => {+ test("an append produces the golden envelope, in memory and byte for byte on disk", () => {+ const record = appendContinuityRecord(vault, INPUT);++ expect({ ...record }).toEqual({ ...GOLDEN_RECORD });+ expect(Object.keys(record)).toEqual([...GOLDEN_KEYS]);+ expect(shardLines()).toEqual([GOLDEN_LINE]);+ });++ test("the dedup id is reproducible from the formula the docs publish", () => {+ // `docs/observability.md`: sha-256 over kind + createdAt + sourceRefs+ // + payload, id-stamped with the digits of createdAt. Recomputed from+ // the documented inputs alone - if anything else enters the hash, an+ // operator reproducing an id from the documentation gets a miss and+ // this assertion is what tells us before they do.+ const digest = createHash("sha256")+ .update(+ JSON.stringify({+ kind: INPUT.kind,+ createdAt: INPUT.createdAt,+ sourceRefs: INPUT.sourceRefs,+ payload: INPUT.payload,+ }),+ "utf8",+ )+ .digest("hex")+ .slice(0, 16);++ expect(buildContinuityRecord(INPUT).id).toBe(`ctn_20260614093000_${digest}`);+ });++ test("the dry-run builder agrees with the append path", () => {+ // `buildContinuityRecord` exists so a caller can predict the write.+ // A field threaded into one path and not the other would break that.+ expect({ ...buildContinuityRecord(INPUT) }).toEqual({ ...GOLDEN_RECORD });+ });++ test("the read-model surfaces the envelope with no extra declaration on it", () => {+ appendContinuityRecord(vault, INPUT);+ const [normalized] = loadNormalizedContinuityRecords(vault);+ expect(normalized).toBeDefined();+ expect(normalized!.id).toBe(GOLDEN_RECORD.id);+ expect("scope" in normalized!).toBe(false);+ });+});diff --git a/tests/core/brain/entities/entity-read-census.test.ts b/tests/core/brain/entities/entity-read-census.test.tsnew file mode 100644index 00000000..dbcec223--- /dev/null+++ b/tests/core/brain/entities/entity-read-census.test.ts@@ -0,0 +1,502 @@+/**+ * Entity read-path census (provenance-at-the-boundary, unit A).+ *+ * A quarantine that nothing filters on is a no-op. This repository reverted+ * a unit in the previous release for shipping a layer that could never fire,+ * and the same failure mode is available here for free: the `active` filter+ * used to be re-implemented at every caller, eight callers applied none at+ * all, and a third status value would have read as "not archived, therefore+ * visible" at every one of them. Centralising the filter is what makes+ * `quarantine` mean something; this census is what keeps it centralised.+ *+ * Four ways a module can reach an entity record, so four rules over one+ * exclusion inventory. Each rule reaches modules no other rule reaches -+ * asserted below, because a rule whose members are all already caught by+ * another rule is a decision layer that never decides, and deleting it+ * would change no outcome.+ *+ * 1. It builds the index itself (`buildEntityIndex` / `parseEntityFile`)+ * and reads the records out.+ * 2. It decides on the status of records handed to it - visible either as+ * a call to the shared predicate, or as a comparison between a+ * `status`-named expression and the entity status vocabulary.+ * 3. It asks the registry's list API (`listEntities`). That API has a+ * deliberate bypass: an explicit `{ status }` is honoured verbatim and+ * skips the shared predicate entirely, which is how an operator sees+ * what is quarantined - and is also exactly how a machine read path+ * would re-implement the filter by accident. `query-expansion.ts` did+ * precisely that (`{ status: "active" }`) and matched no rule at all+ * before this one existed.+ * 4. It walks vault PAGES inside the link-graph tree. `Brain/entities/**`+ * are ordinary Markdown pages, so a walker that never touches the+ * registry still reads a quarantined record's title, body and links.+ * A walker is scoped either by the shared page predicate or by+ * excluding the Brain root from every one of its walks.+ *+ * ## What this census does NOT claim+ *+ * Stated rather than implied, because a bound that lives only in prose is+ * the same defect as a rule that cannot fire:+ *+ * - Rule 4 covers the link-graph tree, not every `listVaultPages` caller+ * in the repository. The trees outside it (the freshness scan, the+ * schema report, the doctor probes, the CLI page listing, the OpenCLAW+ * adapter) walk pages too and are NOT asserted here. `quarantine` is+ * absent from every entity read surface this census names; it is not+ * yet asserted absent from every page-listing surface in the tree.+ * - Rule 2's literal branch only fires in a module that already names the+ * entity vocabulary, because `active` and `quarantine` are ALSO+ * preference statuses and an ungated literal sweeps in four modules+ * that have nothing to do with entities. The gate errs toward+ * over-inclusion - `merge.ts` matches rule 2 on a preference status+ * because it also imports from `entities/` - since a module named twice+ * costs a line and a filter unseen costs the release.+ *+ * The census is proved capable of failing, below, over fixture modules that+ * bypass the predicate under each rule - the same way the import-cycle+ * ratchet proves its own search is not vacuous.+ */++import { describe, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, readdirSync, readFileSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { dirname, join, relative } from "node:path";++const REPO_ROOT = join(import.meta.dir, "..", "..", "..", "..");+const SRC_ROOT = join(REPO_ROOT, "src");++/** Rule 1: the module builds the entity index and reads records out of it. */+const INDEX_READ_RE = /\b(?:buildEntityIndex|parseEntityFile)\s*\(/;++/**+ * Rule 2, literal branch: a `status`-named expression compared against the+ * entity status vocabulary. `quarantine` is IN the vocabulary here - it was+ * absent when this rule shipped, so the one value the release introduced was+ * the one value the rule could not see.+ */+const STATUS_LITERAL_DECISION_RE = new RegExp(+ [+ String.raw`\b\w*[Ss]tatus\w*\s*(?:!==|===)\s*(?:"(?:active|archived|quarantine)"|BRAIN_ENTITY_STATUS\s*\.\s*\w+)`,+ String.raw`BRAIN_ENTITY_STATUS\s*\.\s*\w+\s*(?:!==|===)`,+ ].join("|"),+);++/**+ * The gate on that branch: the module is about ENTITIES rather than about+ * one of the other vocabularies that spell the same words. Preference+ * status, profile names and vault-origin kinds all use `active`, and+ * preference status also uses `quarantine`.+ */+const ENTITY_VOCABULARY_RE =+ /\bBRAIN_ENTITY_STATUS\b|\bBrainEntityStatus\b|\bENTITY_STATUS_SCOPE\b|from\s*"[^"]*\bentities\/[^"]*\.ts"/;++/** Rule 2, predicate branch: deciding on status THROUGH the shared predicate. */+const SHARED_PREDICATE_RE = /\bentityStatusInScope\s*\(/;++/** Rule 3: the registry's list API, whose explicit `{ status }` ask bypasses the scope. */+const REGISTRY_LIST_RE = /(?<!function\s)\blistEntities\s*\(/;++/** Rule 4's population: the link-graph tree, whose walkers reach `Brain/entities/`. */+const LINK_GRAPH_ROOT = "src/core/brain/link-graph/";++/** Rule 4: a walk over vault pages. The declaration itself is not a walk. */+const PAGE_WALK_RE = /(?<!function\s)\blistVaultPages\s*\(/g;++/**+ * A page walk that excludes the Brain root cannot reach an entity page at+ * all, so it is scoped by construction rather than by a predicate. Counted+ * rather than merely detected: a module with two walks, only one of which+ * excludes the Brain root, is not scoped.+ */+const BRAIN_EXCLUDING_WALK_RE =+ /(?<!function\s)\blistVaultPages\s*\([^;]*?skipDirs[^;]*?(?:BRAIN_ROOT_REL|brainDir)/g;++/** A module is scoped when it names the shared predicate or its scopes. */+const SCOPED_RE = /entityStatusInScope|ENTITY_STATUS_SCOPE|vaultPageInStatusScope/;++/**+ * Every entity read path that deliberately does not go through the shared+ * predicate, with the reason. A module may only be here because scoping it+ * would be wrong, never because scoping it was inconvenient.+ */+const UNSCOPED_WITH_REASON: Readonly<Record<string, string>> = Object.freeze({+ "src/core/brain/doctor/entity-checks.ts":+ "registry-integrity surface. It reports duplicate identities and broken " ++ "relation edges over what is STORED; scoping it would stop the doctor " ++ "reporting on exactly the records most likely to be malformed - the ones " ++ "a read path is already hiding.",+ "src/core/brain/entities/label-hygiene.ts":+ "runs the label validator over every stored node (its own docblock says " ++ "'any status'), and the prune pass must see every survivor or it would " ++ "strand inbound edges pointing at a removed node.",+ "src/core/brain/deep-synthesis.ts":+ "hands the whole index to `checkEntityContamination`, which applies the " ++ "shared predicate at its own boundary. A second filter here would be a " ++ "duplicate of that decision, not an independent one.",+ "src/core/brain/merge.ts":+ "hands the whole index to `guardEntityMerge`, which applies the shared " ++ "predicate at its own boundary. It is also in the census under rule 2, on " ++ "a PREFERENCE status comparison - see the docblock on why that branch errs " ++ "toward over-inclusion.",+ "src/cli/brain/verbs/facts.ts":+ "hands the whole index to `decomposeAtomicFacts`, which applies the " ++ "shared predicate at its own boundary.",+ "src/cli/brain/verbs/entity.ts":+ "`brain entity list --status quarantine` IS the named exit from quarantine. " ++ "The explicit ask is the operator asking to see what a read scope hides, it " ++ "is validated against the vocabulary itself before it reaches the registry, " ++ "and scoping it would leave a status nothing can leave - a dead end rather " ++ "than a lane. Without an explicit `--status` this verb takes the registry's " ++ "predicate-applying branch like every other caller.",+ "src/mcp/brain/entity-tools.ts":+ "the MCP twin of that verb: `brain_entity` with an explicit `status` is the " ++ "agent-facing form of the same operator ask, validated against the same " ++ "vocabulary. Its `get` view resolves through `getEntity`, which applies the " ++ "canonical scope inside the registry.",+});++interface CensusRow {+ readonly path: string;+ /** Rule 1. */+ readonly buildsIndex: boolean;+ /** Rule 2. */+ readonly decidesOnStatus: boolean;+ /** Rule 3. */+ readonly asksRegistry: boolean;+ /** Rule 4. */+ readonly walksPages: boolean;+ readonly scoped: boolean;+}++function tsFiles(root: string): string[] {+ const out: string[] = [];+ const walk = (dir: string): void => {+ for (const entry of readdirSync(dir, { withFileTypes: true }).toSorted((a, b) =>+ a.name.localeCompare(b.name),+ )) {+ const abs = join(dir, entry.name);+ if (entry.isDirectory()) walk(abs);+ else if (entry.name.endsWith(".ts")) out.push(abs);+ }+ };+ walk(root);+ return out;+}++function matchCount(text: string, re: RegExp): number {+ return [...text.matchAll(re)].length;+}++/** Classify a set of modules. Parameterised on the file list so the+ * non-vacuity tests below can run the identical classifier over fixtures. */+function censusOver(files: ReadonlyArray<string>, root: string): CensusRow[] {+ const rows: CensusRow[] = [];+ for (const file of files) {+ const text = readFileSync(file, "utf8");+ const path = relative(root, file).split("\\").join("/");+ const walks = matchCount(text, PAGE_WALK_RE);+ const walksPages = path.startsWith(LINK_GRAPH_ROOT) && walks > 0;+ const buildsIndex = INDEX_READ_RE.test(text);+ const decidesOnStatus =+ SHARED_PREDICATE_RE.test(text) ||+ (ENTITY_VOCABULARY_RE.test(text) && STATUS_LITERAL_DECISION_RE.test(text));+ const asksRegistry = REGISTRY_LIST_RE.test(text);+ if (!buildsIndex && !decidesOnStatus && !asksRegistry && !walksPages) continue;+ // A walk that never reaches `Brain/` cannot see an entity page, which is+ // a stronger guarantee than filtering one out.+ const brainRootExcluded = walks > 0 && matchCount(text, BRAIN_EXCLUDING_WALK_RE) === walks;+ rows.push({+ path,+ buildsIndex,+ decidesOnStatus,+ asksRegistry,+ walksPages,+ scoped: SCOPED_RE.test(text) || (walksPages && brainRootExcluded),+ });+ }+ return rows;+}++/**+ * The census over the real tree, computed once. Every assertion below reads+ * the SAME rows, and the whole-tree walk is not repeated per test.+ */+let realCensus: CensusRow[] | null = null;+function census(): CensusRow[] {+ realCensus ??= censusOver(tsFiles(SRC_ROOT), REPO_ROOT);+ return realCensus;+}++/** The rules, by the row field each sets. Used by the no-dead-rule assertions. */+const RULES = Object.freeze({+ buildsIndex: "buildsIndex",+ decidesOnStatus: "decidesOnStatus",+ asksRegistry: "asksRegistry",+ walksPages: "walksPages",+} as const);++type RuleField = (typeof RULES)[keyof typeof RULES];++const RULE_FIELDS: ReadonlyArray<RuleField> = Object.freeze(Object.values(RULES));++/** Rows this rule reaches that no other rule reaches. */+function exclusiveTo(rows: ReadonlyArray<CensusRow>, rule: RuleField): string[] {+ return rows+ .filter((row) => row[rule] && RULE_FIELDS.every((other) => other === rule || !row[other]))+ .map((row) => row.path);+}++describe("entity read-path census", () => {+ test("every entity read path is scoped by the shared predicate or excluded with a reason", () => {+ const unscoped = census()+ .filter((row) => !row.scoped)+ .map((row) => row.path);+ // Named, not counted: the failure message is the file list.+ expect(unscoped.toSorted()).toEqual(Object.keys(UNSCOPED_WITH_REASON).toSorted());+ });++ test("no exclusion outlives the module it excuses", () => {+ const seen = new Set(census().map((row) => row.path));+ for (const excluded of Object.keys(UNSCOPED_WITH_REASON)) {+ expect(seen.has(excluded)).toBe(true);+ }+ });++ test("the search ran over the real tree, not an empty one", () => {+ const rows = census();+ // Floors just under the measurement, so "everything is scoped" cannot be+ // reported over a file list that was never built, and so a real loss of+ // input cannot pass underneath. Today: 22 rows, 10 building the index,+ // 15 scoped. The previous floors (5 index readers against a measured 10)+ // would have let half the population vanish and still read as healthy.+ expect(rows.length).toBeGreaterThan(20);+ expect(rows.filter((row) => row.buildsIndex).length).toBeGreaterThan(9);+ expect(rows.filter((row) => row.scoped).length).toBeGreaterThan(14);+ });++ test("no rule is dead: each reaches modules no other rule reaches", () => {+ // The assertion the previous shape of this file got wrong. It proved+ // rule 2 "alive" by naming a module rule 1 already caught, so deleting+ // rule 2 outright would have changed no outcome. What makes a rule load+ // bearing is the modules ONLY it sees.+ const rows = census();+ for (const rule of RULE_FIELDS) {+ expect(`${rule}: ${exclusiveTo(rows, rule).length > 0}`).toBe(`${rule}: true`);+ }+ });++ test("the modules each rule alone reaches are the ones the rule was written for", () => {+ const rows = census();+ // Rule 3 exists because the registry's explicit-status branch bypasses+ // the predicate, and these two surfaces are its only legitimate users.+ expect(exclusiveTo(rows, RULES.asksRegistry).toSorted()).toEqual([+ "src/cli/brain/verbs/entity.ts",+ "src/mcp/brain/entity-tools.ts",+ ]);+ // Rule 4 exists because entity pages are ordinary vault pages. None of+ // these four touches the registry, so rule 4 is the ONLY rule that sees+ // them - which is exactly why the leak they carried was invisible.+ expect(exclusiveTo(rows, RULES.walksPages).toSorted()).toEqual([+ "src/core/brain/link-graph/bridge-discovery.ts",+ "src/core/brain/link-graph/co-occurrence.ts",+ "src/core/brain/link-graph/graph-holdout.ts",+ "src/core/brain/link-graph/repair-lane.ts",+ ]);+ // Rule 2 exists for modules handed entity-shaped records by a caller;+ // they never build an index and never call the registry.+ expect(exclusiveTo(rows, RULES.decidesOnStatus)).toContain(+ "src/core/brain/truth/contamination.ts",+ );+ });++ test("the registry is in the census under three rules, so its own read API is watched", () => {+ // The registry owns the decision: it builds the index, it names the+ // status vocabulary, and it defines the list API rule 3 watches. If the+ // census could not see the module that OWNS the decision, it is+ // measuring the wrong tree and every verdict above is meaningless.+ const registry = census().find((row) => row.path === "src/core/brain/entities/registry.ts");+ expect(registry).toBeDefined();+ expect(registry!.buildsIndex).toBe(true);+ expect(registry!.decidesOnStatus).toBe(true);+ expect(registry!.scoped).toBe(true);+ });++ test("the link-graph walkers that reach Brain/entities are all scoped", () => {+ // The concrete leak this unit closed, pinned by name: three walkers read+ // page titles, bodies and links straight out of `Brain/entities/` with no+ // status filter at all, and the census could not see them because they+ // never touch `buildEntityIndex`.+ const rows = census().filter((row) => row.walksPages);+ expect(rows.map((row) => row.path).toSorted()).toEqual([+ "src/core/brain/link-graph/bridge-discovery.ts",+ "src/core/brain/link-graph/co-occurrence.ts",+ "src/core/brain/link-graph/graph-holdout.ts",+ "src/core/brain/link-graph/repair-lane.ts",+ ]);+ expect(rows.filter((row) => !row.scoped).map((row) => row.path)).toEqual([]);+ });+});++describe("the census can fail", () => {+ /** Write fixture modules and run the real classifier over them. */+ function classifyFixtures(fixtures: Readonly<Record<string, string>>): CensusRow[] {+ const dir = mkdtempSync(join(tmpdir(), "o2b-entity-census-"));+ try {+ const files = Object.entries(fixtures).map(([name, source]) => {+ const path = join(dir, name);+ mkdirSync(dirname(path), { recursive: true });+ writeFileSync(path, source, "utf8");+ return path;+ });+ return censusOver(files, dir);+ } finally {+ rmSync(dir, { recursive: true, force: true });+ }+ }++ const BYPASSING_INDEX_READER = [+ 'import { buildEntityIndex } from "./index-builder.ts";',+ "export function surfaceEntities(vault: string) {",+ " return buildEntityIndex(vault).entities;",+ "}",+ "",+ ].join("\n");++ const BYPASSING_STATUS_DECIDER = [+ 'import { normalizeEntityName } from "./entities/canonical.ts";',+ "export function anchor(entities: ReadonlyArray<{ status: string; name: string }>) {",+ ' return entities.filter((e) => e.status !== "quarantine").map((e) => normalizeEntityName(e.name));',+ "}",+ "",+ ].join("\n");++ const BYPASSING_REGISTRY_ASK = [+ 'import { listEntities } from "./entities/registry.ts";',+ "export function anchors(vault: string) {",+ ' return listEntities(vault, { status: "active" }).map((e) => e.name);',+ "}",+ "",+ ].join("\n");++ const LINK_GRAPH_WALKER = `${LINK_GRAPH_ROOT}rogue-walker.ts`;+ const BYPASSING_PAGE_WALKER = [+ 'import { EXCLUDED_DIRS, listVaultPages } from "../../../vault.ts";',+ "export function titles(vault: string) {",+ " return listVaultPages(vault, { skipDirs: [...EXCLUDED_DIRS] }).map((p) => p.title);",+ "}",+ "",+ ].join("\n");++ const SCOPED_READER = [+ 'import { buildEntityIndex } from "./index-builder.ts";',+ 'import { ENTITY_STATUS_SCOPE, entityStatusInScope } from "./status-scope.ts";',+ "export function surfaceEntities(vault: string) {",+ " return buildEntityIndex(vault).entities.filter((e) =>",+ " entityStatusInScope(e.status, ENTITY_STATUS_SCOPE.readable),",+ " );",+ "}",+ "",+ ].join("\n");++ test("a new read path that builds the index without the predicate is caught", () => {+ const rows = classifyFixtures({ "rogue-index-reader.ts": BYPASSING_INDEX_READER });+ expect(rows).toHaveLength(1);+ expect(rows[0]!.buildsIndex).toBe(true);+ expect(rows[0]!.scoped).toBe(false);+ });++ test("a new read path that re-implements the status filter is caught", () => {+ const rows = classifyFixtures({ "rogue-status-decider.ts": BYPASSING_STATUS_DECIDER });+ expect(rows).toHaveLength(1);+ expect(rows[0]!.decidesOnStatus).toBe(true);+ expect(rows[0]!.scoped).toBe(false);+ });++ test("a status filter naming `quarantine` is caught - the value this release added", () => {+ // The literal was absent from the vocabulary when the rule shipped, so+ // the one status the release introduced was invisible to the rule meant+ // to police it. The fixture above compares against `quarantine` alone.+ expect(STATUS_LITERAL_DECISION_RE.test(BYPASSING_STATUS_DECIDER)).toBe(true);+ const withoutQuarantine = new RegExp(+ String.raw`\b\w*[Ss]tatus\w*\s*(?:!==|===)\s*"(?:active|archived)"`,+ );+ expect(withoutQuarantine.test(BYPASSING_STATUS_DECIDER)).toBe(false);+ });++ test("a new read path that takes the registry's explicit-status bypass is caught", () => {+ const rows = classifyFixtures({ "rogue-registry-ask.ts": BYPASSING_REGISTRY_ASK });+ expect(rows).toHaveLength(1);+ expect(rows[0]!.asksRegistry).toBe(true);+ expect(rows[0]!.buildsIndex).toBe(false);+ expect(rows[0]!.scoped).toBe(false);+ });++ test("a new link-graph walker that reaches Brain/entities unfiltered is caught", () => {+ const rows = classifyFixtures({ [LINK_GRAPH_WALKER]: BYPASSING_PAGE_WALKER });+ expect(rows).toHaveLength(1);+ expect(rows[0]!.walksPages).toBe(true);+ expect(rows[0]!.scoped).toBe(false);+ });++ test("a link-graph walker that excludes the Brain root is scoped by construction", () => {+ const rows = classifyFixtures({+ [LINK_GRAPH_WALKER]: BYPASSING_PAGE_WALKER.replace(+ "[...EXCLUDED_DIRS]",+ "[...EXCLUDED_DIRS, BRAIN_ROOT_REL]",+ ),+ });+ expect(rows).toHaveLength(1);+ expect(rows[0]!.walksPages).toBe(true);+ expect(rows[0]!.scoped).toBe(true);+ });++ test("a page walker outside the link-graph tree is out of rule 4's population", () => {+ // The bound stated in the docblock, asserted rather than described: the+ // census does not claim to cover every `listVaultPages` caller.+ const rows = classifyFixtures({ "elsewhere/rogue-walker.ts": BYPASSING_PAGE_WALKER });+ expect(rows).toEqual([]);+ });++ test("a read path that uses the predicate is not flagged", () => {+ const rows = classifyFixtures({ "scoped-reader.ts": SCOPED_READER });+ expect(rows).toHaveLength(1);+ expect(rows[0]!.scoped).toBe(true);+ });++ test("a module that touches no entity record is not in the census at all", () => {+ const rows = classifyFixtures({+ "unrelated.ts": 'export const NAME = "active";\nexport const K = 1;\n',+ });+ expect(rows).toEqual([]);+ });++ test("a preference status comparison alone is not an entity read", () => {+ // Both vocabularies spell `active` and `quarantine`. Without the entity+ // gate, rule 2 sweeps in the digest, the epistemic ladder and the MCP+ // resource index, none of which touch an entity record.+ const rows = classifyFixtures({+ "preference-surface.ts": [+ 'import { BRAIN_PREFERENCE_STATUS } from "./preference.ts";',+ "export function quarantined(prefs: ReadonlyArray<{ status: string }>) {",+ ' return prefs.filter((p) => p.status === "quarantine").length;',+ "}",+ "",+ ].join("\n"),+ });+ expect(rows).toEqual([]);+ });++ test("the assertion the census makes fails when a bypassing path is present", () => {+ // The verdict itself, not just the classification: an unscoped module+ // that is not in the exclusion inventory makes the top-level assertion+ // fail. Without this, a classifier that worked over a list nobody+ // asserted against would still look healthy.+ const rows = classifyFixtures({+ "rogue-index-reader.ts": BYPASSING_INDEX_READER,+ "scoped-reader.ts": SCOPED_READER,+ });+ const unscoped = rows.filter((row) => !row.scoped).map((row) => row.path);+ expect(unscoped).toEqual(["rogue-index-reader.ts"]);+ expect(() => expect(unscoped.toSorted()).toEqual([])).toThrow();+ });+});diff --git a/tests/core/brain/entities/page-scope.test.ts b/tests/core/brain/entities/page-scope.test.tsnew file mode 100644index 00000000..33297128--- /dev/null+++ b/tests/core/brain/entities/page-scope.test.ts@@ -0,0 +1,157 @@+/**+ * The status scope applied to vault PAGES (provenance-at-the-boundary).+ *+ * `Brain/entities/**` are ordinary Markdown pages, so the registry's status+ * scope stops at the registry: a link-graph walker reads the same files by+ * path and never calls `buildEntityIndex`. Three walkers did exactly that,+ * and a quarantined record's title, body and links reached the repair+ * proposals, the holdout adjacency and the bridge ambiguity count.+ *+ * Two levels of proof here: the predicate's own answers, and one walker+ * driven end to end. The end-to-end pair runs the SAME vault twice - once+ * with the record active, once quarantined - so a test that stopped+ * producing candidates for an unrelated reason fails the active half+ * instead of passing the quarantined half vacuously.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { bootstrapBrain } from "../../../../src/core/brain/init.ts";+import {+ ENTITY_STATUS_SCOPE,+ vaultPageInStatusScope,+} from "../../../../src/core/brain/entities/page-scope.ts";+import { BRAIN_ENTITY_KIND } from "../../../../src/core/brain/entities/types.ts";+import {+ IDENTITY_STRENGTH,+ collectRepairCandidates,+} from "../../../../src/core/brain/link-graph/repair-lane.ts";++/** A title long enough to clear the repair lane's explicit-reference floor. */+const ENTITY_TITLE = "Shadowbroker Ledger";++describe("vaultPageInStatusScope", () => {+ test("a page that is not an entity record is admitted unchanged", () => {+ expect(+ vaultPageInStatusScope({ kind: "brain-note", title: "x" }, ENTITY_STATUS_SCOPE.readable),+ ).toBe(true);+ expect(vaultPageInStatusScope({}, ENTITY_STATUS_SCOPE.canonical)).toBe(true);+ });++ test("an active entity page is admitted at both scopes", () => {+ const meta = { kind: BRAIN_ENTITY_KIND, status: "active" };+ expect(vaultPageInStatusScope(meta, ENTITY_STATUS_SCOPE.readable)).toBe(true);+ expect(vaultPageInStatusScope(meta, ENTITY_STATUS_SCOPE.canonical)).toBe(true);+ });++ test("an archived entity page is readable but cannot hold an identity", () => {+ const meta = { kind: BRAIN_ENTITY_KIND, status: "archived" };+ expect(vaultPageInStatusScope(meta, ENTITY_STATUS_SCOPE.readable)).toBe(true);+ expect(vaultPageInStatusScope(meta, ENTITY_STATUS_SCOPE.canonical)).toBe(false);+ });++ test("a quarantined entity page is admitted by no scope", () => {+ const meta = { kind: BRAIN_ENTITY_KIND, status: "quarantine" };+ expect(vaultPageInStatusScope(meta, ENTITY_STATUS_SCOPE.readable)).toBe(false);+ expect(vaultPageInStatusScope(meta, ENTITY_STATUS_SCOPE.canonical)).toBe(false);+ });++ test("an entity page whose status never parsed is admitted by no scope", () => {+ // Absence of a recognised status is not a reason to show the record, and+ // it is not the same answer as "this is not an entity page" - which is+ // why the kind is checked first and separately.+ for (const status of [undefined, "", "released", 3]) {+ expect(+ vaultPageInStatusScope(+ { kind: BRAIN_ENTITY_KIND, ...(status !== undefined ? { status } : {}) },+ ENTITY_STATUS_SCOPE.readable,+ ),+ ).toBe(false);+ }+ });+});++describe("a quarantined entity page stays out of the link graph", () => {+ let vault: string;++ beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-page-scope-"));+ bootstrapBrain(vault);+ mkdirSync(join(vault, "Notes"), { recursive: true });+ });++ afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+ });++ /**+ * An entity page carrying an attacker-chosen `title`. The registry+ * preserves frontmatter keys it does not own, and `listVaultPages` titles+ * a page from that key - so the title an untrusted intake wrote is the+ * title the link graph sees.+ */+ function writeEntityPage(status: string): void {+ const dir = join(vault, "Brain", "entities", "concept");+ mkdirSync(dir, { recursive: true });+ writeFileSync(+ join(dir, "ent-concept-shadowbroker-ledger.md"),+ [+ "---",+ `kind: ${BRAIN_ENTITY_KIND}`,+ "entity_id: ent-concept-shadowbroker-ledger",+ "category: concept",+ `name: ${ENTITY_TITLE}`,+ `title: ${ENTITY_TITLE}`,+ `status: ${status}`,+ "created_at: 2026-01-01T00:00:00Z",+ "updated_at: 2026-01-01T00:00:00Z",+ "---",+ "",+ "Body authored by whatever the intake quarantined.",+ "",+ ].join("\n"),+ "utf8",+ );+ }++ function writeNamingNote(): void {+ writeFileSync(+ join(vault, "Notes", "alpha.md"),+ [+ "---",+ "kind: brain-note",+ "title: Alpha",+ "---",+ "",+ `This note discusses the ${ENTITY_TITLE} at length.`,+ "",+ ].join("\n"),+ "utf8",+ );+ }++ function candidatesNamingTheEntity(): number {+ return collectRepairCandidates(vault).filter(+ (candidate) =>+ candidate.strength === IDENTITY_STRENGTH.explicitReference &&+ candidate.target.includes("shadowbroker-ledger"),+ ).length;+ }++ test("an ACTIVE entity page does reach the repair lane", () => {+ // The half that proves the assertion below is not vacuous: the same+ // vault, the same walker, the same note naming the same title.+ writeEntityPage("active");+ writeNamingNote();+ expect(candidatesNamingTheEntity()).toBeGreaterThan(0);+ });++ test("a QUARANTINED entity page reaches nothing", () => {+ writeEntityPage("quarantine");+ writeNamingNote();+ expect(candidatesNamingTheEntity()).toBe(0);+ });+});diff --git a/tests/core/brain/entities/status-scope.test.ts b/tests/core/brain/entities/status-scope.test.tsnew file mode 100644index 00000000..562b0443--- /dev/null+++ b/tests/core/brain/entities/status-scope.test.ts@@ -0,0 +1,169 @@+/**+ * The shared entity status scope (provenance-at-the-boundary, unit A).+ *+ * `quarantine` is only worth having if the read paths honour it. Before this+ * unit the `active` filter was re-implemented at every caller and eight+ * callers applied none at all, so a third status value would have read as+ * "not archived, therefore visible" almost everywhere. One predicate now+ * answers the question and this file pins what it answers.+ *+ * The two scopes are deliberately different questions:+ * - `canonical` - may this record HOLD the identity a lookup resolves to,+ * - `readable` - may a surface show or reason over this record at all.+ * `quarantine` is admitted by neither; `archived` by the second only.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdtempSync, readFileSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { bootstrapBrain } from "../../../../src/core/brain/init.ts";+import { atomicWriteFileSync } from "../../../../src/core/fs-atomic.ts";+import {+ archiveEntity,+ getEntity,+ listEntities,+ upsertEntity,+} from "../../../../src/core/brain/entities/registry.ts";+import {+ BRAIN_ENTITY_STATUS,+ isBrainEntityStatus,+} from "../../../../src/core/brain/entities/types.ts";+import {+ ENTITY_STATUS_SCOPE,+ entityStatusInScope,+} from "../../../../src/core/brain/entities/status-scope.ts";++let vault: string;+let configHome: string;++const NOW = new Date("2026-06-13T12:00:00Z");++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-scope-vault-"));+ configHome = mkdtempSync(join(tmpdir(), "o2b-scope-cfg-"));+ const configPath = join(configHome, "config.yaml");+ atomicWriteFileSync(configPath, `vault: ${vault}\n`);+ bootstrapBrain(vault, { configPath });+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+ rmSync(configHome, { recursive: true, force: true });+});++describe("entityStatusInScope", () => {+ test("quarantine is admitted by no scope - that is what the value means", () => {+ for (const scope of Object.values(ENTITY_STATUS_SCOPE)) {+ expect(entityStatusInScope(BRAIN_ENTITY_STATUS.quarantine, scope)).toBe(false);+ }+ });++ test("active holds the identity and is readable; archived is readable only", () => {+ expect(entityStatusInScope(BRAIN_ENTITY_STATUS.active, ENTITY_STATUS_SCOPE.canonical)).toBe(+ true,+ );+ expect(entityStatusInScope(BRAIN_ENTITY_STATUS.active, ENTITY_STATUS_SCOPE.readable)).toBe(+ true,+ );+ expect(entityStatusInScope(BRAIN_ENTITY_STATUS.archived, ENTITY_STATUS_SCOPE.canonical)).toBe(+ false,+ );+ expect(entityStatusInScope(BRAIN_ENTITY_STATUS.archived, ENTITY_STATUS_SCOPE.readable)).toBe(+ true,+ );+ });++ test("a status outside the vocabulary is admitted by no scope", () => {+ // The entity-like slices the guards consume carry `status: string`, so+ // the predicate has to answer for a value that never parsed. Refusing+ // is the only answer that cannot leak an unknown record into a surface.+ expect(isBrainEntityStatus("smuggled")).toBe(false);+ for (const scope of Object.values(ENTITY_STATUS_SCOPE)) {+ expect(entityStatusInScope("smuggled", scope)).toBe(false);+ }+ });++ test("every status in the vocabulary has a decision, none by omission", () => {+ // The scope table is keyed by BrainEntityStatus, so a new value cannot+ // be added to the vocabulary without a decision here. Measured rather+ // than trusted to the type checker alone.+ const decided = Object.values(BRAIN_ENTITY_STATUS).filter((status) =>+ Object.values(ENTITY_STATUS_SCOPE).some((scope) => entityStatusInScope(status, scope)),+ );+ expect(decided.toSorted()).toEqual([BRAIN_ENTITY_STATUS.active, BRAIN_ENTITY_STATUS.archived]);+ });+});++describe("registry reads honour the scope", () => {+ function quarantined(name: string): void {+ upsertEntity(vault, {+ category: "concept",+ name,+ agent: "intake",+ now: NOW,+ untrustedOrigin: true,+ });+ }++ test("a quarantined entity is absent from an unfiltered list and from lookup", () => {+ upsertEntity(vault, { category: "concept", name: "Restaking", agent: "op", now: NOW });+ quarantined("Scraped Claim");++ expect(listEntities(vault, { category: "concept" }).map((e) => e.name)).toEqual(["Restaking"]);+ expect(getEntity(vault, { category: "concept", query: "Scraped Claim" })).toBeNull();+ expect(getEntity(vault, { query: "Scraped Claim" })).toBeNull();+ });++ test("an archived entity stays in an unfiltered list, exactly as before", () => {+ upsertEntity(vault, { category: "concept", name: "Restaking", agent: "op", now: NOW });+ archiveEntity(vault, { category: "concept", query: "Restaking" }, { now: NOW });+ const listed = listEntities(vault, { category: "concept" });+ expect(listed).toHaveLength(1);+ expect(listed[0]!.status).toBe(BRAIN_ENTITY_STATUS.archived);+ });++ test("an explicit status ask still reaches quarantined records", () => {+ quarantined("Scraped Claim");+ const listed = listEntities(vault, { status: BRAIN_ENTITY_STATUS.quarantine });+ expect(listed.map((e) => e.name)).toEqual(["Scraped Claim"]);+ });++ test("a second untrusted intake updates the quarantined record, it does not fork one", () => {+ quarantined("Scraped Claim");+ quarantined("Scraped Claim");+ const all = listEntities(vault, { status: BRAIN_ENTITY_STATUS.quarantine });+ expect(all).toHaveLength(1);+ expect(all[0]!.id).toBe("ent-concept-scraped-claim");+ });++ test("a trusted mention does not promote a quarantined record on its own", () => {+ quarantined("Scraped Claim");+ upsertEntity(vault, { category: "concept", name: "Scraped Claim", agent: "op", now: NOW });+ const quarantinedRecords = listEntities(vault, { status: BRAIN_ENTITY_STATUS.quarantine });+ expect(quarantinedRecords).toHaveLength(1);+ // ...and the trusted write is not swallowed by it either. Until v1.43.0+ // this asserted zero readable records, which is the defect rather than+ // the guarantee: an untrusted source that guessed a name captured it, and+ // every later trusted write landed inside the quarantined record and read+ // back as absent. A trusted write now creates its own canonical record.+ const readable = listEntities(vault, { category: "concept" });+ expect(readable).toHaveLength(1);+ expect(readable[0]!.status).toBe(BRAIN_ENTITY_STATUS.active);+ });++ test("restore is the named exit from quarantine, not a dead end", () => {+ quarantined("Scraped Claim");+ const released = archiveEntity(+ vault,+ { category: "concept", query: "Scraped Claim" },+ { now: NOW, restore: true },+ );+ expect(released.status).toBe(BRAIN_ENTITY_STATUS.active);+ expect(getEntity(vault, { category: "concept", query: "Scraped Claim" })).not.toBeNull();+ // The release also clears the untrusted-source marker: the record is+ // now held on the operator's authority, not the source's.+ expect(readFileSync(released.path, "utf8")).not.toContain("untrusted_source");+ });+});diff --git a/tests/core/brain/entities/write-lane.test.ts b/tests/core/brain/entities/write-lane.test.tsnew file mode 100644index 00000000..f2140723--- /dev/null+++ b/tests/core/brain/entities/write-lane.test.ts@@ -0,0 +1,216 @@+/**+ * The two write lanes of the entity registry (provenance-at-the-boundary).+ *+ * `quarantine` was introduced as a lane for records an untrusted source+ * introduced, but the write seam resolved a quarantined holder for EVERY+ * upsert. That gave an untrusted source permanent capture of a name: the+ * first scraped page that mentioned `Acme Corp` created the quarantined+ * record, and every later TRUSTED write of that name landed inside it -+ * `created: false`, status copied from the quarantined target, the+ * `untrusted_source` marker carried forward as an extra, and the result+ * invisible to `getEntity`, to the default listing, to alias resolution and+ * to the retrieval gate. The writer was told the write succeeded.+ *+ * What is pinned here is the lane rule that replaced it: a write resolves+ * only inside its OWN identity space. A trusted write never lands on a+ * quarantined record (so an untrusted source cannot capture a name), and an+ * untrusted write never lands on a canonical one (so untrusted material+ * cannot ride into a trusted record's aliases, agent stamp or edges). The+ * two spaces are joined again only by the operator's explicit release, which+ * is why the collision that release can now produce is refused with its+ * remedy rather than silently forked into two active records.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdtempSync, readFileSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { bootstrapBrain } from "../../../../src/core/brain/init.ts";+import { atomicWriteFileSync } from "../../../../src/core/fs-atomic.ts";+import {+ archiveEntity,+ getEntity,+ listEntities,+ relateEntities,+ upsertEntity,+} from "../../../../src/core/brain/entities/registry.ts";+import { BRAIN_ENTITY_STATUS } from "../../../../src/core/brain/entities/types.ts";+import { UNTRUSTED_SOURCE_FRONTMATTER_KEY } from "../../../../src/core/brain/trust/untrusted-provenance.ts";++let vault: string;+let configHome: string;++const NOW = new Date("2026-06-13T12:00:00Z");+const CATEGORY = "org";+const NAME = "Acme Corp";++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-lane-vault-"));+ configHome = mkdtempSync(join(tmpdir(), "o2b-lane-cfg-"));+ const configPath = join(configHome, "config.yaml");+ atomicWriteFileSync(configPath, `vault: ${vault}\n`);+ bootstrapBrain(vault, { configPath });+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+ rmSync(configHome, { recursive: true, force: true });+});++function untrustedUpsert(name: string, aliases?: readonly string[]): string {+ return upsertEntity(vault, {+ category: CATEGORY,+ name,+ ...(aliases !== undefined ? { aliases } : {}),+ agent: "scraper",+ now: NOW,+ untrustedOrigin: true,+ }).entity.id;+}++function trustedUpsert(name: string, aliases?: readonly string[]) {+ return upsertEntity(vault, {+ category: CATEGORY,+ name,+ ...(aliases !== undefined ? { aliases } : {}),+ agent: "operator",+ now: NOW,+ });+}++describe("a trusted write does not disappear into an untrusted-seeded record", () => {+ test("the trusted write creates a readable record of its own", () => {+ untrustedUpsert(NAME);+ const res = trustedUpsert(NAME);++ expect(res.created).toBe(true);+ expect(res.entity.status).toBe(BRAIN_ENTITY_STATUS.active);+ const found = getEntity(vault, { category: CATEGORY, query: NAME });+ expect(found).not.toBeNull();+ expect(found!.id).toBe(res.entity.id);+ expect(listEntities(vault, { category: CATEGORY })).toHaveLength(1);+ });++ test("the trusted record carries no untrusted-source marker", () => {+ untrustedUpsert(NAME);+ const res = trustedUpsert(NAME);+ expect(readFileSync(res.entity.path, "utf8")).not.toContain(UNTRUSTED_SOURCE_FRONTMATTER_KEY);+ });++ test("the quarantined record is neither promoted nor rewritten by it", () => {+ const quarantinedId = untrustedUpsert(NAME);+ const before = readFileSync(+ listEntities(vault, { status: BRAIN_ENTITY_STATUS.quarantine })[0]!.path,+ "utf8",+ );+ trustedUpsert(NAME);++ const quarantined = listEntities(vault, { status: BRAIN_ENTITY_STATUS.quarantine });+ expect(quarantined).toHaveLength(1);+ expect(quarantined[0]!.id).toBe(quarantinedId);+ expect(quarantined[0]!.source_agent).toBe("scraper");+ expect(readFileSync(quarantined[0]!.path, "utf8")).toBe(before);+ });+});++describe("an untrusted write does not launder itself into a trusted record", () => {+ test("it lands in the quarantine lane even when a trusted namesake exists", () => {+ const trusted = trustedUpsert(NAME);+ untrustedUpsert(NAME, ["Acme Holdings"]);++ const found = getEntity(vault, { category: CATEGORY, query: NAME })!;+ expect(found.id).toBe(trusted.entity.id);+ expect(found.status).toBe(BRAIN_ENTITY_STATUS.active);+ expect(found.aliases).toHaveLength(0);+ expect(found.source_agent).toBe("operator");+ expect(readFileSync(found.path, "utf8")).not.toContain(UNTRUSTED_SOURCE_FRONTMATTER_KEY);+ expect(listEntities(vault, { status: BRAIN_ENTITY_STATUS.quarantine })).toHaveLength(1);+ });++ test("an alias claimed only by an untrusted record never resolves a trusted lookup", () => {+ untrustedUpsert(NAME, ["Acme Holdings"]);+ expect(getEntity(vault, { category: CATEGORY, query: "Acme Holdings" })).toBeNull();+ });++ test("re-ingesting the same untrusted source updates its record rather than forking one", () => {+ const first = untrustedUpsert(NAME);+ const second = untrustedUpsert(NAME);+ expect(second).toBe(first);+ expect(listEntities(vault, { status: BRAIN_ENTITY_STATUS.quarantine })).toHaveLength(1);+ });++ test("a quarantined record keeps its marker across an untrusted update", () => {+ untrustedUpsert(NAME);+ untrustedUpsert(NAME);+ const path = listEntities(vault, { status: BRAIN_ENTITY_STATUS.quarantine })[0]!.path;+ expect(readFileSync(path, "utf8")).toContain(UNTRUSTED_SOURCE_FRONTMATTER_KEY);+ });+});++describe("edges stay inside the lane that wrote them", () => {+ test("an untrusted intake can link the records it quarantined", () => {+ untrustedUpsert("Acme Corp");+ untrustedUpsert("Acme Labs");+ const from = relateEntities(vault, {+ from: { category: CATEGORY, query: "Acme Corp" },+ relation: "related",+ to: { category: CATEGORY, query: "Acme Labs" },+ now: NOW,+ untrustedOrigin: true,+ });+ expect(from.status).toBe(BRAIN_ENTITY_STATUS.quarantine);+ expect(from.relations.some((r) => r.relation === "related")).toBe(true);+ });++ test("an untrusted edge cannot attach itself to a trusted record", () => {+ const trusted = trustedUpsert("Acme Corp");+ untrustedUpsert("Acme Corp");+ untrustedUpsert("Acme Labs");+ relateEntities(vault, {+ from: { category: CATEGORY, query: "Acme Corp" },+ relation: "related",+ to: { category: CATEGORY, query: "Acme Labs" },+ now: NOW,+ untrustedOrigin: true,+ });+ const stillClean = getEntity(vault, { category: CATEGORY, query: "Acme Corp" })!;+ expect(stillClean.id).toBe(trusted.entity.id);+ expect(stillClean.relations).toHaveLength(0);+ });++ test("a trusted relate that finds only a quarantined holder names the exit", () => {+ trustedUpsert("Acme Labs");+ untrustedUpsert("Acme Corp");+ expect(() =>+ relateEntities(vault, {+ from: { category: CATEGORY, query: "Acme Corp" },+ relation: "related",+ to: { category: CATEGORY, query: "Acme Labs" },+ now: NOW,+ }),+ ).toThrow(/quarantine/);+ });+});++describe("release is refused when it would collide, not silently duplicated", () => {+ test("restoring a quarantined record whose name a trusted record holds throws with the ids", () => {+ untrustedUpsert(NAME);+ const trusted = trustedUpsert(NAME);+ expect(() =>+ archiveEntity(vault, { category: CATEGORY, query: NAME }, { now: NOW, restore: true }),+ ).toThrow(new RegExp(trusted.entity.id));+ expect(listEntities(vault, { status: BRAIN_ENTITY_STATUS.quarantine })).toHaveLength(1);+ });++ test("restore still releases a quarantined record when no trusted record holds the name", () => {+ untrustedUpsert(NAME);+ const released = archiveEntity(+ vault,+ { category: CATEGORY, query: NAME },+ { now: NOW, restore: true },+ );+ expect(released.status).toBe(BRAIN_ENTITY_STATUS.active);+ expect(getEntity(vault, { category: CATEGORY, query: NAME })).not.toBeNull();+ });+});diff --git a/tests/core/brain/handoff.test.ts b/tests/core/brain/handoff.test.tsindex 98e633f0..313ca098 100644--- a/tests/core/brain/handoff.test.ts+++ b/tests/core/brain/handoff.test.ts@@ -4,6 +4,7 @@ import { tmpdir } from "node:os";import { join } from "node:path";import { buildHandoffNote, writeHandoffNote } from "../../../src/core/brain/handoff.ts";+import { parseFrontmatterTextWithNotices } from "../../../src/core/vault.ts";import type { SessionTurn } from "../../../src/core/brain/sessions/types.ts";const NOW = new Date("2026-06-03T12:00:00Z");@@ -113,8 +114,29 @@ test("writeHandoffNote lands Brain/handoffs/<date>-<scope>.md with frontmatter",expect(result.path.endsWith(join("Brain", "handoffs", "2026-06-03-sess-42.md"))).toBe(true);expect(existsSync(result.path)).toBe(true);const content = readFileSync(result.path, "utf8");- // Caller-supplied scalars are JSON-quoted against YAML injection.- expect(content).toContain('session_id: "Sess 42"');- expect(content).toContain('agent: "test-agent"');+ // The property is that a caller-supplied scalar survives the round trip+ // without becoming structure - not that it is always quoted. This writer+ // used to hand-roll its frontmatter and quote unconditionally; it now+ // shares the vault's emitter, which quotes on the same structural test+ // the parser unquotes on, so a value needing no quoting is written plain+ // and reads back byte-identical. The injection case below is the property+ // the old assertions were reaching for.+ expect(content).toContain("session_id: Sess 42");+ expect(content).toContain("agent: test-agent");+ const [meta] = parseFrontmatterTextWithNotices(content);+ expect(meta["session_id"]).toBe("Sess 42");+ expect(meta["agent"]).toBe("test-agent");expect(content).toContain("## Request");});++test("a caller-supplied scalar cannot become a frontmatter key", () => {+ const result = writeHandoffNote(vault, {+ turns: TURNS,+ sessionId: "s42",+ agent: "x\nowner: attacker",+ now: NOW,+ });+ const [meta] = parseFrontmatterTextWithNotices(readFileSync(result.path, "utf8"));+ expect(meta["owner"]).toBeUndefined();+ expect(meta["agent"]).toBe("x\nowner: attacker");+});diff --git a/tests/core/brain/intake/intake-trust-polarity.test.ts b/tests/core/brain/intake/intake-trust-polarity.test.tsnew file mode 100644index 00000000..353d55bb--- /dev/null+++ b/tests/core/brain/intake/intake-trust-polarity.test.ts@@ -0,0 +1,176 @@+/**+ * The polarity of the intake trust decision (provenance-at-the-boundary).+ *+ * `classifySourceTrust` originally answered "is this identity provably+ * OUTSIDE the vault?" and called everything else trusted, so a source+ * identity that established nothing at all - a bare host such as+ * `evil.com/article`, an address with the scheme dropped - entered as if the+ * operator had written it. The polarity here is the other way round: an+ * identity is trusted only when its SHAPE is that of a location inside this+ * vault, and every identity the shape cannot establish is untrusted.+ *+ * The same seam had the mirrored defect. `hasUriScheme` is written for+ * markdown link targets, where a colon can only be a scheme; a filename is a+ * different domain, and this project's own vault names notes like+ * `Meeting: Q3 planning.md`. Reused unguarded it read that colon as a scheme+ * and quarantined a note the operator wrote - damage that is one-way, since+ * the record then leaves every ordinary read and only an explicit release+ * brings it back. So the two failures a source identity can have are kept+ * apart: an identity that names something external, and an identity that+ * cannot be established at all. The first is a lane; the second is a+ * question for whoever named it.+ *+ * The MCP-boundary cases live here rather than under `tests/mcp/` because+ * they are the same decision as the classifier cases below it - the tool is+ * the surface where "no source was named at all" can still be handed back to+ * the caller with an exit, and testing it apart from the classifier would+ * split one rule across two files.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdtempSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { bootstrapBrain } from "../../../../src/core/brain/init.ts";+import { atomicWriteFileSync } from "../../../../src/core/fs-atomic.ts";+import { getEntity, listEntities } from "../../../../src/core/brain/entities/registry.ts";+import { BRAIN_ENTITY_STATUS } from "../../../../src/core/brain/entities/types.ts";+import { intakeExtraction } from "../../../../src/core/brain/intake/extract-intake.ts";+import { classifySourceTrust } from "../../../../src/core/brain/intake/source-trust.ts";+import { INTAKE_TRUST } from "../../../../src/core/brain/trust/untrusted-provenance.ts";+import { NER_TOOLS } from "../../../../src/mcp/brain/ner-tools.ts";+import { MCPError } from "../../../../src/mcp/protocol.ts";+import type { ServerContext } from "../../../../src/mcp/tool-contract.ts";++let vault: string;+let configHome: string;+let ctx: ServerContext;++const NOW = new Date("2026-06-13T12:00:00Z");+const EXTRACTION = { entities: [{ category: "concept", name: "Restaking" }] };+/** A note name this project's own vault conventions produce. */+const COLON_NOTE = "Meeting: Q3 planning.md";++const handler = NER_TOOLS[0]!.handler;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-polarity-vault-"));+ configHome = mkdtempSync(join(tmpdir(), "o2b-polarity-cfg-"));+ const configPath = join(configHome, "config.yaml");+ atomicWriteFileSync(configPath, `vault: ${vault}\nagent_name: claude\n`);+ bootstrapBrain(vault, { configPath });+ ctx = { vault, configPath, repoRoot: null };+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+ rmSync(configHome, { recursive: true, force: true });+});++describe("classifySourceTrust - an identity that establishes nothing is not trusted", () => {+ test("a bare host, with or without a path, is untrusted", () => {+ for (const source of [+ "evil.com/article",+ "www.example.com/x",+ "example.com",+ "example.co.uk/posts/1",+ "[[evil.com/article]]",+ ]) {+ expect(classifySourceTrust(vault, source)).toBe(INTAKE_TRUST.untrusted);+ }+ });++ test("a scheme in front of a note-shaped path does not buy trust", () => {+ for (const source of ["file:/etc/passwd.md", "https://example.com/a.md", "//evil.com/x.md"]) {+ expect(classifySourceTrust(vault, source)).toBe(INTAKE_TRUST.untrusted);+ }+ });+});++describe("classifySourceTrust - a colon in a filename is not a scheme", () => {+ test("a vault note whose name carries a colon stays trusted", () => {+ for (const source of [+ COLON_NOTE,+ `[[${COLON_NOTE}]]`,+ "Meetings/Meeting: Q3 planning.md",+ "Meetings/Q3: planning/notes.md",+ ]) {+ expect(classifySourceTrust(vault, source)).toBe(INTAKE_TRUST.trusted);+ }+ });++ test("the ordinary vault-relative identities are unchanged", () => {+ for (const source of [+ "Articles/primer.md",+ "notes/deep/nested.md",+ "Code/widget.ts",+ "readme.md",+ ]) {+ expect(classifySourceTrust(vault, source)).toBe(INTAKE_TRUST.trusted);+ }+ });+});++describe("intakeExtraction derives trust from the source it was given", () => {+ test("a cited source outside the vault quarantines even when no trust is declared", () => {+ intakeExtraction(vault, EXTRACTION, {+ agent: "claude",+ now: NOW,+ provenance: { level: "stated", sources: ["[[evil.com/article]]"], premises: [] },+ });+ expect(getEntity(vault, { category: "concept", query: "Restaking" })).toBeNull();+ expect(listEntities(vault, { status: BRAIN_ENTITY_STATUS.quarantine })).toHaveLength(1);+ });++ test("one untrusted source among several makes the whole intake untrusted", () => {+ intakeExtraction(vault, EXTRACTION, {+ agent: "claude",+ now: NOW,+ provenance: {+ level: "stated",+ sources: ["[[Articles/primer.md]]", "[[https://example.com/a]]"],+ premises: [],+ },+ });+ expect(getEntity(vault, { category: "concept", query: "Restaking" })).toBeNull();+ });++ test("a declared trust still wins over the derivation", () => {+ intakeExtraction(vault, EXTRACTION, {+ agent: "claude",+ now: NOW,+ provenance: { level: "stated", sources: ["[[evil.com/article]]"], premises: [] },+ trust: INTAKE_TRUST.trusted,+ });+ expect(getEntity(vault, { category: "concept", query: "Restaking" })).not.toBeNull();+ });+});++describe("brain_intake_entities - the boundary that can still ask", () => {+ test("naming no source at all is refused with an exit, and writes nothing", async () => {+ await expect(+ handler(ctx, { entities: [{ category: "concept", name: "Layer 2s" }] }),+ ).rejects.toThrow(MCPError);+ expect(listEntities(vault, { status: BRAIN_ENTITY_STATUS.quarantine })).toHaveLength(0);+ expect(listEntities(vault)).toHaveLength(0);+ });++ test("a source with the scheme dropped lands in quarantine, not in the registry", async () => {+ await handler(ctx, {+ entities: [{ category: "concept", name: "Layer 2s" }],+ source: "evil.com/article",+ });+ expect(getEntity(vault, { category: "concept", query: "Layer 2s" })).toBeNull();+ expect(listEntities(vault, { status: BRAIN_ENTITY_STATUS.quarantine })).toHaveLength(1);+ });++ test("a vault note whose name carries a colon is not quarantined", async () => {+ await handler(ctx, {+ entities: [{ category: "concept", name: "Layer 2s" }],+ source: `[[${COLON_NOTE}]]`,+ });+ expect(getEntity(vault, { category: "concept", query: "Layer 2s" })).not.toBeNull();+ expect(listEntities(vault, { status: BRAIN_ENTITY_STATUS.quarantine })).toHaveLength(0);+ });+});diff --git a/tests/core/brain/intake/intake-trust.test.ts b/tests/core/brain/intake/intake-trust.test.tsnew file mode 100644index 00000000..be337f17--- /dev/null+++ b/tests/core/brain/intake/intake-trust.test.ts@@ -0,0 +1,269 @@+/**+ * Trust originated at the intake boundary (provenance-at-the-boundary, unit A).+ *+ * Before this unit nothing at intake knew where content came from. The source+ * pipeline stamped one provenance level on every source, so a scraped URL and+ * a file in the operator's own vault were indistinguishable, and the+ * `untrusted_source` frontmatter key that the retrieval trust gate reads had+ * no writer anywhere in the repository - a gate branch reachable only from a+ * field nothing produced.+ *+ * Two things are pinned here. First, that the trusted path did not move: the+ * entity page and the summary page a trusted intake writes are asserted+ * against bytes captured from the code as it stood BEFORE the change, so+ * "byte-identical" is a measurement and not a claim. Second, that the+ * untrusted path now lands the record quarantined, marks it with the key the+ * gate reads, and that driving the real gate over the real file it wrote+ * returns the untrusted-source verdict.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdtempSync, readFileSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { bootstrapBrain } from "../../../../src/core/brain/init.ts";+import { atomicWriteFileSync } from "../../../../src/core/fs-atomic.ts";+import { parseFrontmatter } from "../../../../src/core/vault.ts";+import { getEntity, listEntities } from "../../../../src/core/brain/entities/registry.ts";+import { BRAIN_ENTITY_STATUS } from "../../../../src/core/brain/entities/types.ts";+import { intakeExtraction } from "../../../../src/core/brain/intake/extract-intake.ts";+import { classifySourceTrust } from "../../../../src/core/brain/intake/source-trust.ts";+import { ingestSource } from "../../../../src/core/brain/ingest/ingest.ts";+import {+ classifyRetrievalTrust,+ RETRIEVAL_TRUST_EXCLUSION_REASON,+} from "../../../../src/core/brain/trust/retrieval-gate.ts";+import {+ INTAKE_TRUST,+ UNTRUSTED_SOURCE_FRONTMATTER_KEY,+} from "../../../../src/core/brain/trust/untrusted-provenance.ts";++let vault: string;+let configHome: string;++const NOW = new Date("2026-06-13T12:00:00Z");++const EXTRACTION = { entities: [{ category: "concept", name: "Restaking" }] };+const TRUSTED_SOURCE = "Articles/primer.md";+const UNTRUSTED_SOURCE = "https://example.com/a";++/**+ * Bytes the pre-change code wrote for a trusted intake, captured from the+ * tree before this unit existed. Pinned literally so a regression in the+ * trusted path fails here rather than being absorbed by a helper that+ * changed alongside it.+ */+const GOLDEN_TRUSTED_ENTITY_PAGE =+ "---\n" ++ "kind: brain-entity\n" ++ "entity_id: ent-concept-restaking\n" ++ "category: concept\n" ++ "name: Restaking\n" ++ "status: active\n" ++ "source_agent: ingest-agent\n" ++ 'created_at: "2026-06-13T12:00:00Z"\n' ++ 'updated_at: "2026-06-13T12:00:00Z"\n' ++ "tags: [brain, brain/entity]\n" ++ "---\n" ++ "\n" ++ "# Restaking\n" ++ "\n" ++ "## Sources\n" ++ "\n" ++ "- [[Articles/primer.md]]\n";++const GOLDEN_TRUSTED_SUMMARY_PAGE =+ "---\n" ++ "kind: brain-source\n" ++ "source_path: Articles/primer.md\n" ++ "source_hash: 51b0f811072b41378ce56e723de332a315cd3c2841915c355606b294f1304019\n" ++ "provenance: stated\n" ++ 'created_at: "2026-06-13T12:00:00Z"\n' ++ 'updated_at: "2026-06-13T12:00:00Z"\n' ++ "tags: [brain, brain/source]\n" ++ "---\n" ++ "\n" ++ "An overview.\n" ++ "\n" ++ "## Sources\n" ++ "\n" ++ "- [[Articles/primer.md]]\n" ++ "\n" ++ "## Entities\n" ++ "\n" ++ "- [[ent-concept-restaking]]\n";++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-trust-vault-"));+ configHome = mkdtempSync(join(tmpdir(), "o2b-trust-cfg-"));+ const configPath = join(configHome, "config.yaml");+ atomicWriteFileSync(configPath, `vault: ${vault}\n`);+ bootstrapBrain(vault, { configPath });+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+ rmSync(configHome, { recursive: true, force: true });+});++describe("classifySourceTrust", () => {+ test("a vault-relative source identity is trusted", () => {+ expect(classifySourceTrust(vault, TRUSTED_SOURCE)).toBe(INTAKE_TRUST.trusted);+ expect(classifySourceTrust(vault, "notes/deep/nested.md")).toBe(INTAKE_TRUST.trusted);+ expect(classifySourceTrust(vault, "[[Articles/primer.md]]")).toBe(INTAKE_TRUST.trusted);+ });++ test("a source identity carrying a URI scheme is untrusted", () => {+ for (const source of [+ "https://example.com/a",+ "HTTP://Example.com/a",+ "ftp://host/x",+ "//example.com/a",+ "[[https://example.com/a]]",+ ]) {+ expect(classifySourceTrust(vault, source)).toBe(INTAKE_TRUST.untrusted);+ }+ });++ test("a source identity that leaves the vault is untrusted", () => {+ expect(classifySourceTrust(vault, "../outside/x.md")).toBe(INTAKE_TRUST.untrusted);+ expect(classifySourceTrust(vault, "/etc/passwd")).toBe(INTAKE_TRUST.untrusted);+ });++ test("an empty source identity is untrusted - absence is not authority", () => {+ expect(classifySourceTrust(vault, "")).toBe(INTAKE_TRUST.untrusted);+ expect(classifySourceTrust(vault, " ")).toBe(INTAKE_TRUST.untrusted);+ });+});++describe("intakeExtraction - the trusted path did not move", () => {+ test("an intake with no trust declared writes the pre-change bytes", () => {+ intakeExtraction(vault, EXTRACTION, {+ agent: "ingest-agent",+ now: NOW,+ provenance: { level: "stated", sources: [`[[${TRUSTED_SOURCE}]]`], premises: [] },+ });+ const entity = getEntity(vault, { category: "concept", query: "Restaking" })!;+ expect(readFileSync(entity.path, "utf8")).toBe(GOLDEN_TRUSTED_ENTITY_PAGE);+ });++ test("an intake declared trusted writes the same bytes as one declaring nothing", () => {+ intakeExtraction(vault, EXTRACTION, {+ agent: "ingest-agent",+ now: NOW,+ provenance: { level: "stated", sources: [`[[${TRUSTED_SOURCE}]]`], premises: [] },+ trust: INTAKE_TRUST.trusted,+ });+ const entity = getEntity(vault, { category: "concept", query: "Restaking" })!;+ expect(readFileSync(entity.path, "utf8")).toBe(GOLDEN_TRUSTED_ENTITY_PAGE);+ });+});++describe("intakeExtraction - the untrusted path", () => {+ function untrustedIntake(): void {+ intakeExtraction(vault, EXTRACTION, {+ agent: "ingest-agent",+ now: NOW,+ provenance: { level: "stated", sources: [`[[${UNTRUSTED_SOURCE}]]`], premises: [] },+ trust: INTAKE_TRUST.untrusted,+ });+ }++ test("the entity lands quarantined and reaches no ordinary read surface", () => {+ untrustedIntake();+ expect(getEntity(vault, { category: "concept", query: "Restaking" })).toBeNull();+ expect(listEntities(vault, { category: "concept" })).toHaveLength(0);+ const quarantined = listEntities(vault, { status: BRAIN_ENTITY_STATUS.quarantine });+ expect(quarantined).toHaveLength(1);+ expect(quarantined[0]!.name).toBe("Restaking");+ });++ test("the entity page carries the untrusted_source key the gate reads", () => {+ untrustedIntake();+ const [meta] = parseFrontmatter(+ listEntities(vault, { status: BRAIN_ENTITY_STATUS.quarantine })[0]!.path,+ );+ expect(meta[UNTRUSTED_SOURCE_FRONTMATTER_KEY]).toBeDefined();+ });++ test("the retrieval gate excludes the page this intake actually wrote", () => {+ untrustedIntake();+ // Driven from the real file the intake produced, not a hand-built+ // fixture: this is the join that did not exist before, because the key+ // the gate reads had no writer.+ const path = listEntities(vault, { status: BRAIN_ENTITY_STATUS.quarantine })[0]!.path;+ const [meta] = parseFrontmatter(path);+ const verdict = classifyRetrievalTrust(meta);+ expect(verdict.quarantined).toBe(true);+ expect(verdict.reasons).toContain(RETRIEVAL_TRUST_EXCLUSION_REASON.untrustedSourceProvenance);+ });++ test("a trusted intake leaves the gate's verdict clean", () => {+ intakeExtraction(vault, EXTRACTION, { agent: "ingest-agent", now: NOW });+ const entity = getEntity(vault, { category: "concept", query: "Restaking" })!;+ const [meta] = parseFrontmatter(entity.path);+ expect(classifyRetrievalTrust(meta).quarantined).toBe(false);+ });++ test("an entity that already exists is not downgraded by a later untrusted mention", () => {+ intakeExtraction(vault, EXTRACTION, { agent: "operator", now: NOW });+ untrustedIntake();+ const entity = getEntity(vault, { category: "concept", query: "Restaking" });+ expect(entity).not.toBeNull();+ expect(entity!.status).toBe(BRAIN_ENTITY_STATUS.active);+ const [meta] = parseFrontmatter(entity!.path);+ expect(meta[UNTRUSTED_SOURCE_FRONTMATTER_KEY]).toBeUndefined();+ });+});++describe("ingestSource derives trust from the source identity", () => {+ const INPUT = { summary: "An overview.", extraction: EXTRACTION };++ test("a vault-relative source writes the pre-change summary bytes", () => {+ const res = ingestSource(+ vault,+ { ...INPUT, sourcePath: TRUSTED_SOURCE },+ { agent: "claude", now: NOW },+ );+ expect(readFileSync(join(vault, res.summaryPath), "utf8")).toBe(GOLDEN_TRUSTED_SUMMARY_PAGE);+ expect(listEntities(vault, { category: "concept" })).toHaveLength(1);+ });++ test("a URL source quarantines its entities and marks its summary page", () => {+ const res = ingestSource(+ vault,+ { ...INPUT, sourcePath: UNTRUSTED_SOURCE },+ { agent: "claude", now: NOW },+ );+ expect(listEntities(vault, { category: "concept" })).toHaveLength(0);+ expect(listEntities(vault, { status: BRAIN_ENTITY_STATUS.quarantine })).toHaveLength(1);++ const [meta] = parseFrontmatter(join(vault, res.summaryPath));+ expect(meta[UNTRUSTED_SOURCE_FRONTMATTER_KEY]).toBeDefined();+ expect(classifyRetrievalTrust(meta).reasons).toContain(+ RETRIEVAL_TRUST_EXCLUSION_REASON.untrustedSourceProvenance,+ );+ });++ test("a scraped URL and a local file are no longer indistinguishable", () => {+ const local = ingestSource(+ vault,+ { ...INPUT, sourcePath: TRUSTED_SOURCE },+ { agent: "claude", now: NOW },+ );+ const scraped = ingestSource(+ vault,+ {+ ...INPUT,+ sourcePath: UNTRUSTED_SOURCE,+ extraction: { entities: [{ category: "concept", name: "Scraped" }] },+ },+ { agent: "claude", now: NOW },+ );+ const [localMeta] = parseFrontmatter(join(vault, local.summaryPath));+ const [scrapedMeta] = parseFrontmatter(join(vault, scraped.summaryPath));+ expect(localMeta[UNTRUSTED_SOURCE_FRONTMATTER_KEY]).toBeUndefined();+ expect(scrapedMeta[UNTRUSTED_SOURCE_FRONTMATTER_KEY]).toBeDefined();+ });+});diff --git a/tests/core/brain/notes/create-note-modes.test.ts b/tests/core/brain/notes/create-note-modes.test.tsnew file mode 100644index 00000000..75e01fd5--- /dev/null+++ b/tests/core/brain/notes/create-note-modes.test.ts@@ -0,0 +1,415 @@+/**+ * createNote authoring modes (provenance-at-the-boundary, Unit C):+ * `ifExists: "skip"`, `strict` pre-write validation, and template-mode+ * bodies.+ *+ * Three properties are load-bearing and each has its own test here:+ *+ * - With none of the three options set, the primitive is+ * byte-identical to the release before them. The sharp form of that+ * is a document the strict validator would reject (no frontmatter at+ * all) still being written when `strict` is unset.+ * - A skip is DISCRIMINATED from a create. Returning the same shape+ * for both would be the silent fallback this wave forbids, so the+ * result carries an `outcome` a caller cannot misread, and a skipped+ * call leaves the vault tree byte-for-byte unchanged.+ * - The write-batch surface keeps refusing an existing target. Batches+ * are all-or-nothing; a per-operation skip would make "applied"+ * mean two different things in one result list.+ *+ * The pre-existing refusal behaviour is pinned by `create-note.test.ts`,+ * which stays untouched and green.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import {+ existsSync,+ mkdirSync,+ mkdtempSync,+ readdirSync,+ readFileSync,+ rmSync,+ statSync,+ writeFileSync,+} from "node:fs";+import { tmpdir } from "node:os";+import { join, relative } from "node:path";++import {+ createNote,+ CreateNoteError,+ type CreateNoteResult,+} from "../../../../src/core/brain/notes/create-note.ts";+import { applyWriteBatch, WriteBatchError } from "../../../../src/core/brain/write-batch.ts";++/**+ * One raw C0 control character, built rather than typed so it is+ * visible in source and survives a formatter. `validateArtifact`+ * rejects any C0 byte except tab, newline and carriage return.+ */+const CONTROL_CHAR = String.fromCharCode(7);++/** The YAML fence `formatFrontmatter` opens and closes the block with. */+const FRONTMATTER_FENCE = "---";+/** What separates a frontmatter key from its value on one line. */+const FRONTMATTER_KEY_SEPARATOR = ":";++/**+ * One fixed create input, reused so the with-options and without-options+ * cases differ in exactly one thing: whether an option was supplied. It+ * carries three frontmatter keys in a deliberate, non-alphabetical order,+ * because a reordering is one of the two ways the bytes can move without+ * any assertion on `created`/`outcome` noticing.+ */+const BASELINE_INPUT = Object.freeze({+ path: "Notes/Golden.md",+ frontmatter: Object.freeze({ title: "Golden", type: "note", tags: Object.freeze(["a", "b"]) }),+ content: "Body line one.\n\nBody line two.",+});++/**+ * The exact bytes {@link BASELINE_INPUT} produced with none of the three+ * authoring options set, captured from the primitive as it stood before+ * they existed. Pinned as a literal rather than recomputed from the+ * implementation, because an expectation derived from the implementation+ * cannot notice the implementation moving: an unconditional spread adding+ * one frontmatter key, or a reordering of the block, changes this string+ * and nothing else in this file.+ */+const GOLDEN_BASELINE_NOTE =+ "---\ntitle: Golden\ntype: note\ntags: [a, b]\n---\n\nBody line one.\n\nBody line two.\n";++/** Frontmatter key order of the pre-feature note, top to bottom. */+const GOLDEN_BASELINE_FRONTMATTER_KEYS: ReadonlyArray<string> = Object.freeze([+ "title",+ "type",+ "tags",+]);++/** The pre-feature bytes for a note with no frontmatter map at all. */+const GOLDEN_BARE_NOTE = "---\n---\n\njust a body\n";++/** Result field order the MCP surface and the SDK both read. */+const GOLDEN_RESULT_KEYS: ReadonlyArray<string> = Object.freeze(["path", "outcome", "created"]);++let vault: string;+beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-create-note-modes-"));+});+afterEach(() => rmSync(vault, { recursive: true, force: true }));++/** Every path under `root`, with file bytes, so "changed nothing" is checkable. */+function snapshotTree(root: string): ReadonlyArray<string> {+ const out: string[] = [];+ const walk = (dir: string): void => {+ for (const entry of readdirSync(dir).toSorted()) {+ const abs = join(dir, entry);+ const rel = relative(root, abs);+ if (statSync(abs).isDirectory()) {+ out.push(`dir ${rel}`);+ walk(abs);+ } else {+ out.push(`file ${rel} ${readFileSync(abs, "utf8")}`);+ }+ }+ };+ walk(root);+ return out;+}++/**+ * Frontmatter keys of a written note, in the order they appear on disk.+ * Order is half of what byte-identity means here: the same set of keys+ * emitted in a different sequence is a different file, and a test that+ * compared sets would not see it. A note with no frontmatter block is a+ * caller error rather than an empty list - absence and "could not parse"+ * must not collapse into one answer.+ */+function frontmatterKeys(markdown: string): ReadonlyArray<string> {+ const lines = markdown.split("\n");+ const end = lines[0] === FRONTMATTER_FENCE ? lines.indexOf(FRONTMATTER_FENCE, 1) : -1;+ if (end < 0) {+ throw new Error(`note carries no frontmatter block: ${JSON.stringify(markdown)}`);+ }+ return lines.slice(1, end).map((line) => {+ const colon = line.indexOf(FRONTMATTER_KEY_SEPARATOR);+ if (colon < 0) throw new Error(`frontmatter line carries no key: ${JSON.stringify(line)}`);+ return line.slice(0, colon);+ });+}++/** Read one written note back as raw text, so disk is checked, not memory. */+function writtenNote(path: string): string {+ return readFileSync(join(vault, path), "utf8");+}++describe("createNote - the new options are byte-identical when absent", () => {+ test("with none of the options set the written bytes are the pre-feature golden", () => {+ const res = createNote(vault, { ...BASELINE_INPUT });+ expect(res.outcome).toBe("created");+ // Byte for byte on disk...+ expect(writtenNote(BASELINE_INPUT.path)).toBe(GOLDEN_BASELINE_NOTE);+ // ...frontmatter key order included.+ expect(frontmatterKeys(writtenNote(BASELINE_INPUT.path))).toEqual([+ ...GOLDEN_BASELINE_FRONTMATTER_KEYS,+ ]);+ });++ test("options spread in as undefined write the same bytes as options omitted", () => {+ // That is the shape every conditional spread in this codebase+ // produces when its condition is false, so an option threaded as an+ // explicit `undefined` must not become a different document.+ createNote(vault, {+ ...BASELINE_INPUT,+ ifExists: undefined,+ strict: undefined,+ template: undefined,+ templateVariables: undefined,+ });+ expect(writtenNote(BASELINE_INPUT.path)).toBe(GOLDEN_BASELINE_NOTE);+ });++ test("a document the strict validator would reject is still written when strict is unset", () => {+ // No frontmatter at all: `validateArtifact` reports frontmatter-missing.+ const res = createNote(vault, { path: "Notes/Bare.md", content: "just a body" });+ expect(res.created).toBe(true);+ // The whole file, not a substring: an empty frontmatter block is part+ // of the pre-feature shape and a body-only file is not the same note.+ expect(writtenNote("Notes/Bare.md")).toBe(GOLDEN_BARE_NOTE);+ });++ test("with ifExists unset an existing target is still refused with the exists code", () => {+ createNote(vault, { path: "Notes/Once.md", content: "first" });+ try {+ createNote(vault, { path: "Notes/Once.md", content: "second" });+ throw new Error("expected throw");+ } catch (err) {+ expect(err).toBeInstanceOf(CreateNoteError);+ expect((err as CreateNoteError).code).toBe("exists");+ }+ expect(readFileSync(join(vault, "Notes/Once.md"), "utf8")).toContain("first");+ });++ test("a create reports the created outcome alongside the frozen created flag", () => {+ const res = createNote(vault, { path: "Notes/New.md", content: "x" });+ expect(res.outcome).toBe("created");+ expect(res.created).toBe(true);+ // The result envelope is the one addition this unit makes, and it is+ // pinned in full so a fourth field cannot arrive unannounced.+ expect(Object.keys(res)).toEqual([...GOLDEN_RESULT_KEYS]);+ });+});++describe('createNote - ifExists: "skip"', () => {+ test("an existing target returns a skipped outcome a caller cannot read as a create", () => {+ createNote(vault, { path: "Notes/Dup.md", content: "original" });+ const res: CreateNoteResult = createNote(vault, {+ path: "Notes/Dup.md",+ content: "replacement",+ ifExists: "skip",+ });+ expect(res.outcome).toBe("skipped");+ expect(res.created).toBe(false);+ expect(res.path).toBe("Notes/Dup.md");+ expect(readFileSync(join(vault, "Notes/Dup.md"), "utf8")).toContain("original");+ });++ test("a skip leaves the vault tree byte-for-byte unchanged, directories included", () => {+ createNote(vault, { path: "A/B/C/Deep.md", content: "original" });+ const before = snapshotTree(vault);+ const res = createNote(vault, {+ path: "A/B/C/Deep.md",+ content: "replacement",+ ifExists: "skip",+ });+ expect(res.outcome).toBe("skipped");+ expect(snapshotTree(vault)).toEqual([...before]);+ });++ test("a free target under skip is created normally", () => {+ const res = createNote(vault, { path: "Notes/Fresh.md", content: "x", ifExists: "skip" });+ expect(res.outcome).toBe("created");+ expect(res.created).toBe(true);+ expect(existsSync(join(vault, "Notes/Fresh.md"))).toBe(true);+ });++ test("an explicit refuse behaves exactly like the default", () => {+ createNote(vault, { path: "Notes/R.md", content: "first" });+ expect(() =>+ createNote(vault, { path: "Notes/R.md", content: "second", ifExists: "refuse" }),+ ).toThrow(CreateNoteError);+ });++ test("skip does not relax any other refusal - traversal is still typed", () => {+ expect(() => createNote(vault, { path: "../escape.md", ifExists: "skip" })).toThrow(+ CreateNoteError,+ );+ });+});++describe("createNote - strict pre-write validation", () => {+ test("a valid document passes and is written", () => {+ const res = createNote(vault, {+ path: "Notes/Good.md",+ frontmatter: { title: "Good", type: "note" },+ content: "Body.",+ strict: true,+ });+ expect(res.outcome).toBe("created");+ });++ test("a document with no frontmatter is refused with the validator's coded violation", () => {+ try {+ createNote(vault, { path: "Notes/Bad.md", content: "body only", strict: true });+ throw new Error("expected throw");+ } catch (err) {+ expect(err).toBeInstanceOf(CreateNoteError);+ const e = err as CreateNoteError;+ expect(e.code).toBe("invalid_document");+ expect(e.violations.map((v) => v.code)).toContain("frontmatter-missing");+ }+ expect(existsSync(join(vault, "Notes/Bad.md"))).toBe(false);+ });++ test("a body carrying raw control characters is refused", () => {+ try {+ createNote(vault, {+ path: "Notes/Ctrl.md",+ frontmatter: { title: "C" },+ content: `a${CONTROL_CHAR}b`,+ strict: true,+ });+ throw new Error("expected throw");+ } catch (err) {+ expect((err as CreateNoteError).violations.map((v) => v.code)).toContain(+ "artifact-control-chars",+ );+ }+ expect(existsSync(join(vault, "Notes/Ctrl.md"))).toBe(false);+ });++ test("a declared type outside the vault's page_types vocabulary is refused", () => {+ try {+ createNote(vault, {+ path: "Notes/Typed.md",+ frontmatter: { title: "T", type: "not-a-declared-page-type" },+ content: "body",+ strict: true,+ });+ throw new Error("expected throw");+ } catch (err) {+ expect((err as CreateNoteError).violations.map((v) => v.code)).toContain(+ "schema-type-unknown",+ );+ }+ });++ test("strict validates the rendered template body, not the raw template", () => {+ // The template text itself is clean; only the RENDERED body carries+ // the control character, so a strict refusal proves the validator+ // ran after rendering rather than before it.+ try {+ createNote(vault, {+ path: "Notes/Tpl.md",+ template: "start {{body}} end",+ templateVariables: { body: CONTROL_CHAR },+ strict: true,+ frontmatter: { title: "T" },+ });+ throw new Error("expected throw");+ } catch (err) {+ expect((err as CreateNoteError).violations.map((v) => v.code)).toContain(+ "artifact-control-chars",+ );+ }+ expect(existsSync(join(vault, "Notes/Tpl.md"))).toBe(false);+ });+});++describe("createNote - template mode", () => {+ test("renders typed variables, a presence section and a list iteration into the body", () => {+ const res = createNote(vault, {+ path: "Notes/Rendered.md",+ frontmatter: { title: "R" },+ template: "# {{title}}\n{{#draft}}(draft){{/draft}}\n{{#tags}}- {{.}}\n{{/tags}}{{ typo }}",+ templateVariables: { title: "R", draft: true, tags: ["a", "b"] },+ });+ expect(res.outcome).toBe("created");+ const md = readFileSync(join(vault, "Notes/Rendered.md"), "utf8");+ expect(md).toContain("# R");+ expect(md).toContain("(draft)");+ expect(md).toContain("- a\n- b\n");+ // The unknown placeholder survives so the typo surfaces in the note.+ expect(md).toContain("{{ typo }}");+ });++ test("a template with no variables renders its literal text", () => {+ createNote(vault, { path: "Notes/Lit.md", template: "plain body" });+ expect(readFileSync(join(vault, "Notes/Lit.md"), "utf8")).toContain("plain body");+ });++ test("content and template together are refused rather than one silently winning", () => {+ try {+ createNote(vault, { path: "Notes/Both.md", content: "a", template: "b" });+ throw new Error("expected throw");+ } catch (err) {+ expect(err).toBeInstanceOf(CreateNoteError);+ expect((err as CreateNoteError).code).toBe("invalid_template");+ }+ expect(existsSync(join(vault, "Notes/Both.md"))).toBe(false);+ });++ test("template variables without a template are refused", () => {+ expect(() =>+ createNote(vault, { path: "Notes/Orphan.md", templateVariables: { a: "b" } }),+ ).toThrow(CreateNoteError);+ expect(existsSync(join(vault, "Notes/Orphan.md"))).toBe(false);+ });++ test("a malformed template is refused and writes nothing", () => {+ try {+ createNote(vault, { path: "Notes/Broken.md", template: "{{#a}}x", templateVariables: {} });+ throw new Error("expected throw");+ } catch (err) {+ expect((err as CreateNoteError).code).toBe("invalid_template");+ }+ expect(existsSync(join(vault, "Notes/Broken.md"))).toBe(false);+ });+});++describe("write_batch keeps refusing an existing target", () => {+ test("a create_note operation on an occupied path aborts the whole batch", () => {+ mkdirSync(join(vault, "Notes"), { recursive: true });+ writeFileSync(join(vault, "Notes/Taken.md"), "original", "utf8");+ try {+ applyWriteBatch(vault, [+ { kind: "create_note", path: "Notes/Fresh.md", content: "x" },+ { kind: "create_note", path: "Notes/Taken.md", content: "y" },+ ]);+ throw new Error("expected throw");+ } catch (err) {+ expect(err).toBeInstanceOf(WriteBatchError);+ expect((err as WriteBatchError).code).toBe("exists");+ }+ // All-or-nothing: the first, valid operation did not land either.+ expect(readFileSync(join(vault, "Notes/Taken.md"), "utf8")).toBe("original");+ expect(existsSync(join(vault, "Notes/Fresh.md"))).toBe(false);+ });++ test("the batch create_note operation exposes no skip policy to opt into", () => {+ const withSkip = {+ kind: "create_note",+ path: "Notes/Taken.md",+ content: "y",+ if_exists: "skip",+ };+ mkdirSync(join(vault, "Notes"), { recursive: true });+ writeFileSync(join(vault, "Notes/Taken.md"), "original", "utf8");+ expect(() =>+ applyWriteBatch(vault, [+ withSkip as unknown as Parameters<typeof applyWriteBatch>[1][number],+ ]),+ ).toThrow(WriteBatchError);+ expect(readFileSync(join(vault, "Notes/Taken.md"), "utf8")).toBe("original");+ });+});diff --git a/tests/core/brain/notes/create-note.test.ts b/tests/core/brain/notes/create-note.test.tsindex 0f98a07a..dcc17a1c 100644--- a/tests/core/brain/notes/create-note.test.ts+++ b/tests/core/brain/notes/create-note.test.ts@@ -9,7 +9,15 @@*/import { afterEach, beforeEach, describe, expect, test } from "bun:test";-import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync, mkdirSync } from "node:fs";+import {+ existsSync,+ mkdtempSync,+ readFileSync,+ readdirSync,+ rmSync,+ writeFileSync,+ mkdirSync,+} from "node:fs";import { tmpdir } from "node:os";import { join } from "node:path";@@ -98,3 +106,35 @@ describe("createNote", () => {}});});++describe("a caller-named path is read with this host's separator only", () => {+ // Defect 2 of the v1.43.0 security review, reproduced at the envelope.+ // On POSIX a backslash is an ordinary filename character, and three+ // layers disagreed about it: the write-binding matcher split on it, the+ // result renderer rewrote it to `/`, and the Brain-root and vault-scope+ // checks treated it as literal. The envelope refuses the path rather+ // than picking one reading.+ for (const [label, path] of [+ ["a backslash inside a segment", "Projects\\evil.md"],+ ["a backslash traversal", "Projects\\..\\..\\..\\etc\\x.md"],+ ["a backslash-hidden Brain root", "Brain\\x.md"],+ ] as const) {+ test(`${label} is refused, not reinterpreted`, () => {+ let caught: CreateNoteError | null = null;+ try {+ createNote(vault, { path, content: "x" });+ } catch (err) {+ caught = err as CreateNoteError;+ }+ expect(caught).toBeInstanceOf(CreateNoteError);+ expect(caught!.code).toBe("invalid_path");+ expect(readdirSync(vault)).toEqual([]);+ });+ }++ test("a POSIX path with the same segments is still created", () => {+ const res = createNote(vault, { path: "Projects/evil.md", content: "x" });+ expect(res.path).toBe("Projects/evil.md");+ expect(existsSync(join(vault, "Projects", "evil.md"))).toBe(true);+ });+});diff --git a/tests/core/brain/notes/note-template.test.ts b/tests/core/brain/notes/note-template.test.tsnew file mode 100644index 00000000..a0f5cad3--- /dev/null+++ b/tests/core/brain/notes/note-template.test.ts@@ -0,0 +1,152 @@+/**+ * Note-body template grammar (provenance-at-the-boundary, Unit C).+ *+ * The grammar is CLOSED at two constructs - typed variable substitution+ * and presence-driven sections / list iteration. These tests pin both+ * halves of that boundary: what the two constructs do, and that nothing+ * else is a construct. An unknown placeholder survives byte-intact (the+ * rule `renderTemplate` in `src/core/brain/templates.ts` already+ * documents, so a typo surfaces instead of vanishing); a malformed+ * section is a typed refusal, because it cannot be rendered at all.+ */++import { describe, expect, test } from "bun:test";++import {+ NoteTemplateError,+ renderNoteTemplate,+} from "../../../../src/core/brain/notes/note-template.ts";++describe("renderNoteTemplate - construct 1: typed variable substitution", () => {+ test("renders a string variable as-is", () => {+ expect(renderNoteTemplate("# {{title}}", { title: "Release notes" })).toBe("# Release notes");+ });++ test("renders a number canonically, including zero", () => {+ expect(renderNoteTemplate("{{count}} open", { count: 0 })).toBe("0 open");+ expect(renderNoteTemplate("{{count}} open", { count: 12 })).toBe("12 open");+ });++ test("renders a boolean as true/false", () => {+ expect(renderNoteTemplate("draft={{draft}}", { draft: true })).toBe("draft=true");+ expect(renderNoteTemplate("draft={{draft}}", { draft: false })).toBe("draft=false");+ });++ test("renders a list with the canonical separator", () => {+ expect(renderNoteTemplate("tags: {{tags}}", { tags: ["a", "b", "c"] })).toBe("tags: a, b, c");+ });++ test("tolerates whitespace inside the delimiters", () => {+ expect(renderNoteTemplate("{{ title }}", { title: "T" })).toBe("T");+ });++ test("substitutes literally - a value containing $& is not a backreference", () => {+ expect(renderNoteTemplate("{{name}}", { name: "pay-$& and $1" })).toBe("pay-$& and $1");+ });++ test("replaces every occurrence of the same variable", () => {+ expect(renderNoteTemplate("{{a}}/{{a}}", { a: "x" })).toBe("x/x");+ });+});++describe("renderNoteTemplate - construct 2: presence sections and list iteration", () => {+ test("renders a section when its variable is present and non-empty", () => {+ expect(renderNoteTemplate("{{#draft}}WIP{{/draft}}", { draft: true })).toBe("WIP");+ expect(renderNoteTemplate("{{#note}}[{{note}}]{{/note}}", { note: "hi" })).toBe("[hi]");+ });++ test("omits a section whose variable is present but empty", () => {+ expect(renderNoteTemplate("a{{#draft}}WIP{{/draft}}b", { draft: false })).toBe("ab");+ expect(renderNoteTemplate("a{{#note}}x{{/note}}b", { note: "" })).toBe("ab");+ expect(renderNoteTemplate("a{{#tags}}x{{/tags}}b", { tags: [] })).toBe("ab");+ });++ test("presence is not JavaScript truthiness - a zero renders its section", () => {+ expect(renderNoteTemplate("{{#count}}n={{count}}{{/count}}", { count: 0 })).toBe("n=0");+ });++ test("iterates a list section once per item, binding the item to {{.}}", () => {+ expect(renderNoteTemplate("{{#tags}}- {{.}}\n{{/tags}}", { tags: ["a", "b"] })).toBe(+ "- a\n- b\n",+ );+ });++ test("an outer variable stays readable inside an iteration", () => {+ expect(+ renderNoteTemplate("{{#tags}}{{prefix}}{{.}} {{/tags}}", { prefix: "#", tags: ["x", "y"] }),+ ).toBe("#x #y ");+ });++ test("sections nest", () => {+ expect(+ renderNoteTemplate("{{#on}}{{#tags}}<{{.}}>{{/tags}}{{/on}}", { on: true, tags: ["a", "b"] }),+ ).toBe("<a><b>");+ });++ test("{{.}} outside an iteration is an unknown placeholder, left intact", () => {+ expect(renderNoteTemplate("x {{.}} y", {})).toBe("x {{.}} y");+ });+});++describe("renderNoteTemplate - unknown placeholders stay intact", () => {+ test("an unknown variable keeps its exact source text, spacing included", () => {+ expect(renderNoteTemplate("a {{ typo }} b", { title: "T" })).toBe("a {{ typo }} b");+ });++ test("an unknown section keeps the whole construct, body included", () => {+ expect(renderNoteTemplate("a{{#typo}}body{{/typo}}b", {})).toBe("a{{#typo}}body{{/typo}}b");+ });+});++describe("renderNoteTemplate - the grammar is closed", () => {+ test("there is no path/dotted lookup", () => {+ expect(renderNoteTemplate("{{a.b}}", { a: "x" })).toBe("{{a.b}}");+ });++ test("there is no expression language", () => {+ expect(renderNoteTemplate("{{a == b}}", { a: "x", b: "x" })).toBe("{{a == b}}");+ expect(renderNoteTemplate("{{a | upper}}", { a: "x" })).toBe("{{a | upper}}");+ });++ test("there is no inverted section", () => {+ expect(renderNoteTemplate("{{^a}}none{{/a}}", {})).toBe("{{^a}}none{{/a}}");+ });++ test("an unterminated delimiter is literal text", () => {+ expect(renderNoteTemplate("{{ oops", { oops: "x" })).toBe("{{ oops");+ });+});++describe("renderNoteTemplate - a malformed template is refused, not guessed", () => {+ test("an unclosed section throws a typed unbalanced_section", () => {+ try {+ renderNoteTemplate("{{#a}}body", { a: true });+ throw new Error("expected throw");+ } catch (err) {+ expect(err).toBeInstanceOf(NoteTemplateError);+ expect((err as NoteTemplateError).code).toBe("unbalanced_section");+ }+ });++ test("a close with no open is not a construct, so it stays intact", () => {+ expect(renderNoteTemplate("body{{/a}}", { a: true })).toBe("body{{/a}}");+ });++ test("a mismatched close leaves the section open, which throws", () => {+ expect(() => renderNoteTemplate("{{#a}}x{{/b}}", { a: true, b: true })).toThrow(+ NoteTemplateError,+ );+ });++ test("nesting deeper than the cap throws a typed section_too_deep", () => {+ let template = "x";+ for (let i = 0; i < 40; i++) template = `{{#s${i}}}${template}{{/s${i}}}`;+ try {+ renderNoteTemplate(template, {});+ throw new Error("expected throw");+ } catch (err) {+ expect(err).toBeInstanceOf(NoteTemplateError);+ expect((err as NoteTemplateError).code).toBe("section_too_deep");+ }+ });+});diff --git a/tests/core/brain/schema-mutate.test.ts b/tests/core/brain/schema-mutate.test.tsindex 4f41e252..e81b3736 100644--- a/tests/core/brain/schema-mutate.test.ts+++ b/tests/core/brain/schema-mutate.test.ts@@ -1,16 +1,57 @@import { afterEach, beforeEach, describe, expect, test } from "bun:test";-import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";+import {+ existsSync,+ mkdirSync,+ mkdtempSync,+ readFileSync,+ rmSync,+ statSync,+ utimesSync,+ writeFileSync,+} from "node:fs";import { tmpdir } from "node:os";import { join } from "node:path";import {applySchemaMutations,+ previewSchemaMutations,type SchemaMutation,} from "../../../src/core/brain/schema-mutate.ts";import { loadSchemaPack } from "../../../src/core/brain/schema-pack.ts";let vault: string;+/**+ * Epoch seconds stamped onto `_brain.yaml` before a preview, far enough in+ * the past that ANY write during the preview would move the modification+ * time to now. Without this the assertion would be satisfied by a coarse+ * filesystem timestamp rather than by the preview keeping its hands off.+ */+const PAST_MTIME_SECONDS = Date.UTC(2020, 0, 1) / 1000;++function configPathFor(root: string): string {+ return join(root, "Brain", "_brain.yaml");+}++interface ConfigSnapshot {+ /** Base64 of the raw file, so the comparison is over bytes, not text. */+ readonly bytes: string;+ readonly mtimeMs: number;+}++/** Freeze `_brain.yaml`'s observable state so a later write is detectable. */+function snapshotConfig(root: string): ConfigSnapshot {+ const path = configPathFor(root);+ utimesSync(path, PAST_MTIME_SECONDS, PAST_MTIME_SECONDS);+ return { bytes: readFileSync(path).toString("base64"), mtimeMs: statSync(path).mtimeMs };+}++function expectConfigUntouched(root: string, snapshot: ConfigSnapshot): void {+ const path = configPathFor(root);+ expect(readFileSync(path).toString("base64")).toBe(snapshot.bytes);+ expect(statSync(path).mtimeMs).toBe(snapshot.mtimeMs);+}+beforeEach(() => {vault = mkdtempSync(join(tmpdir(), "o2b-schema-mutate-"));mkdirSync(join(vault, "Brain"), { recursive: true });@@ -137,3 +178,140 @@ describe("applySchemaMutations", () => {expect(audit).not.toContain("secret-value");});});++describe("previewSchemaMutations", () => {+ test("returns the pack that would result and the diff from the current one", () => {+ const snapshot = snapshotConfig(vault);++ const preview = previewSchemaMutations(vault, [+ { op: "add_type", category: "preference_types", token: "decision" },+ { op: "add_alias", token: "decision", alias: "choice" },+ { op: "set_expert_routing", token: "decision", expert: "schema-author" },+ ]);++ expect(preview.dry_run).toBe(true);+ expect(preview.would_apply).toBe(3);+ expect(preview.pack.declarations.preference_types).toEqual(["research", "decision"]);+ expect(preview.diff).toEqual([+ { path: "aliases.decision", before: null, after: "choice" },+ { path: "declarations.preference_types", before: null, after: "decision" },+ { path: "expert_routing.decision", before: null, after: "schema-author" },+ ]);++ expectConfigUntouched(vault, snapshot);+ expect(loadSchemaPack(vault).declarations.preference_types).toEqual(["research"]);+ });++ test("reports a scalar leaf whose value changed as a before/after pair", async () => {+ await applySchemaMutations(+ vault,+ [{ op: "set_expert_routing", token: "research", expert: "first-expert" }],+ { actor: "tester" },+ );+ const snapshot = snapshotConfig(vault);++ const preview = previewSchemaMutations(vault, [+ { op: "set_expert_routing", token: "research", expert: "second-expert" },+ ]);++ expect(preview.diff).toEqual([+ { path: "expert_routing.research", before: "first-expert", after: "second-expert" },+ ]);+ expectConfigUntouched(vault, snapshot);+ });++ test("reports a rename as one removed and one added declaration member", () => {+ const preview = previewSchemaMutations(vault, [+ {+ op: "update_type",+ category: "preference_types",+ token: "research",+ new_token: "research-note",+ },+ ]);++ expect(preview.diff).toEqual([+ { path: "declarations.preference_types", before: "research", after: null },+ { path: "declarations.preference_types", before: null, after: "research-note" },+ ]);+ });++ test("returns an empty diff for a batch that would change nothing", () => {+ const preview = previewSchemaMutations(vault, [+ { op: "add_type", category: "preference_types", token: "research" },+ ]);++ expect(preview.would_apply).toBe(1);+ expect(preview.diff).toEqual([]);+ });++ test("previews against a vault with no _brain.yaml without creating one", () => {+ const configPath = configPathFor(vault);+ rmSync(configPath);++ const preview = previewSchemaMutations(vault, [+ { op: "add_type", category: "preference_types", token: "decision" },+ ]);++ expect(preview.diff).toEqual([+ { path: "declarations.preference_types", before: null, after: "decision" },+ ]);+ // Absent must stay absent: a preview creates nothing, not even the file+ // the apply path would materialise from its default.+ expect(existsSync(configPath)).toBe(false);+ });++ const REJECTED_BATCHES: ReadonlyArray<{+ readonly name: string;+ readonly mutations: ReadonlyArray<SchemaMutation>;+ readonly message: string;+ }> = [+ {+ name: "a malformed token",+ mutations: [{ op: "add_type", category: "preference_types", token: "123bad" }],+ message:+ "schema.preference_types: must start with a letter and contain only letters, numbers, underscores, or hyphens",+ },+ {+ name: "a prefix pointing at an undeclared token",+ mutations: [{ op: "add_prefix", prefix: "pref", token: "undeclared" }],+ message: "schema.prefixes.pref: token is not declared",+ },+ {+ name: "a rename of a token that is not declared",+ mutations: [+ {+ op: "update_type",+ category: "preference_types",+ token: "missing",+ new_token: "renamed",+ },+ ],+ message: "schema.preference_types: missing is not declared",+ },+ ];++ for (const rejected of REJECTED_BATCHES) {+ test(`preview of ${rejected.name} raises exactly what an apply raises, without writing`, async () => {+ const snapshot = snapshotConfig(vault);++ let previewMessage = "";+ try {+ previewSchemaMutations(vault, rejected.mutations);+ } catch (err) {+ previewMessage = (err as Error).message;+ }+ expectConfigUntouched(vault, snapshot);++ let applyMessage = "";+ try {+ await applySchemaMutations(vault, rejected.mutations, { actor: "tester" });+ } catch (err) {+ applyMessage = (err as Error).message;+ }++ expect(previewMessage).toBe(rejected.message);+ expect(applyMessage).toBe(previewMessage);+ });+ }+});diff --git a/tests/core/brain/skill-offer-join.test.ts b/tests/core/brain/skill-offer-join.test.tsnew file mode 100644index 00000000..9cb3552d--- /dev/null+++ b/tests/core/brain/skill-offer-join.test.ts@@ -0,0 +1,192 @@+/**+ * Offer-to-invocation join, end to end (Unit J, t_ccb05134, decision J1).+ *+ * The chain under test runs all the way through: `skills_attach` mints an+ * offer id over what it offered, an agent quotes that id back on the+ * `get_skill` call, the session log records the call, the session importer+ * stamps the cited offer onto the `skill_invoked` continuity record, and the+ * read side joins the invocation back to the offer it came from.+ *+ * Nothing here is asserted in the abstract: the offer id the join reports is+ * recomputed from the attachment, and the invocation record is the one the+ * real importer wrote from a real session fixture.+ */++import { afterEach, beforeEach, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { DEFAULT_BRAIN_CONFIG_YAML } from "../../../src/core/brain/config-template.ts";+import { loadNormalizedContinuityRecords } from "../../../src/core/brain/continuity/read-model.ts";+import { brainDirs } from "../../../src/core/brain/paths.ts";+import { importSession } from "../../../src/core/brain/sessions/import.ts";+import {+ deriveSkillUsage,+ joinSkillInvocationsToOffers,+} from "../../../src/core/brain/skill-usage.ts";+import { atomicWriteFileSync } from "../../../src/core/fs-atomic.ts";+import { buildSkillAttachment } from "../../../src/core/surface/skill-attach.ts";+import { SKILL_OFFER_ID_KEY } from "../../../src/core/surface/skill-offer.ts";+import type { SkillEntry } from "../../../src/core/surface/skills.ts";++let vault: string;++function entry(name: string, description: string): SkillEntry {+ return Object.freeze({+ name,+ description,+ triggers: "",+ path: `/skills/${name}`,+ skillFile: `/skills/${name}/SKILL.md`,+ shadowed: Object.freeze([]),+ });+}++const SKILLS = [+ entry("release", "Cut a release: changelog, version bump, tag."),+ entry("triage", "Sort incoming issues by severity."),+ entry("schema-author", "Author Brain schema vocabulary packs."),+];++const TURN = "cut a release with a changelog and a version bump";++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "osb-skill-offer-join-"));+ const dirs = brainDirs(vault);+ for (const d of [+ dirs.brain,+ dirs.inbox,+ dirs.processed,+ dirs.preferences,+ dirs.retired,+ dirs.log,+ dirs.snapshots,+ ]) {+ mkdirSync(d, { recursive: true });+ }+ atomicWriteFileSync(join(dirs.brain, "_brain.yaml"), DEFAULT_BRAIN_CONFIG_YAML);+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+});++/** A Claude session log whose get_skill calls carry the offers they cite. */+function writeSessionFixture(+ path: string,+ calls: ReadonlyArray<{ id: string; skill: string; offerId?: string }>,+): void {+ const lines = [+ `{"parentUuid":null,"sessionId":"offer-join","entrypoint":"sdk-cli","type":"user","message":{"role":"user","content":${JSON.stringify(TURN)}},"uuid":"u1","timestamp":"2026-06-01T10:00:00.000Z"}`,+ ];+ calls.forEach((call, i) => {+ const input: Record<string, unknown> = { name: call.skill };+ if (call.offerId !== undefined) input[SKILL_OFFER_ID_KEY] = call.offerId;+ lines.push(+ JSON.stringify({+ parentUuid: "u1",+ sessionId: "offer-join",+ type: "assistant",+ message: {+ role: "assistant",+ content: [{ type: "tool_use", id: call.id, name: "get_skill", input }],+ },+ uuid: `a${i + 1}`,+ timestamp: `2026-06-01T10:00:0${i + 1}.000Z`,+ }),+ );+ });+ lines.push("");+ writeFileSync(path, lines.join("\n"));+}++test("an invocation citing an offer joins back to it across a real import", async () => {+ const attachment = buildSkillAttachment({ query: TURN, skills: SKILLS });+ expect(attachment.items.map((i) => i.name)).toContain("release");+ const offerId = attachment.offerId!;+ expect(offerId).not.toBeNull();++ const fixture = join(vault, "session.jsonl");+ writeSessionFixture(fixture, [{ id: "c1", skill: "release", offerId }]);+ const res = await importSession(vault, fixture, { agent: "test" });+ expect(res.skill_invocations).toBe(1);++ // The importer put the cited offer on the record, not next to it.+ const records = loadNormalizedContinuityRecords(vault, { kind: "skill_invoked" });+ expect(records).toHaveLength(1);+ expect(records[0]!.payload[SKILL_OFFER_ID_KEY]).toBe(offerId);++ const report = joinSkillInvocationsToOffers(vault);+ expect(report.offers).toHaveLength(1);+ expect(report.offers[0]!.offerId).toBe(offerId);+ expect(report.offers[0]!.invocations.map((i) => i.skill)).toEqual(["release"]);+ expect(report.unattributed).toHaveLength(0);+});++test("an invocation citing no offer is reported unattributed, never guessed into one", async () => {+ const offerId = buildSkillAttachment({ query: TURN, skills: SKILLS }).offerId!;+ const fixture = join(vault, "session.jsonl");+ writeSessionFixture(fixture, [+ { id: "c1", skill: "release", offerId },+ { id: "c2", skill: "triage" },+ ]);+ await importSession(vault, fixture, { agent: "test" });++ const report = joinSkillInvocationsToOffers(vault);+ expect(report.offers.map((o) => o.offerId)).toEqual([offerId]);+ expect(report.offers[0]!.invocations.map((i) => i.skill)).toEqual(["release"]);+ expect(report.unattributed.map((i) => i.skill)).toEqual(["triage"]);+ expect(report.unattributed[0]!.offerId).toBeNull();+});++test("two invocations of the same offer group under one offer id", async () => {+ const offerId = buildSkillAttachment({ query: TURN, skills: SKILLS }).offerId!;+ const fixture = join(vault, "session.jsonl");+ writeSessionFixture(fixture, [+ { id: "c1", skill: "release", offerId },+ { id: "c2", skill: "triage", offerId },+ ]);+ await importSession(vault, fixture, { agent: "test" });++ const report = joinSkillInvocationsToOffers(vault);+ expect(report.offers).toHaveLength(1);+ expect(report.offers[0]!.invocations.map((i) => i.skill).toSorted()).toEqual([+ "release",+ "triage",+ ]);+});++test("a malformed offer id on the call is not an offer, and is not stamped", async () => {+ const fixture = join(vault, "session.jsonl");+ writeSessionFixture(fixture, [{ id: "c1", skill: "release", offerId: "not-an-offer-id" }]);+ await importSession(vault, fixture, { agent: "test" });++ const records = loadNormalizedContinuityRecords(vault, { kind: "skill_invoked" });+ expect(records).toHaveLength(1);+ expect(records[0]!.payload[SKILL_OFFER_ID_KEY]).toBeUndefined();+ expect(joinSkillInvocationsToOffers(vault).unattributed).toHaveLength(1);+});++test("per-skill usage reports how much of it came from a recorded offer", async () => {+ const offerId = buildSkillAttachment({ query: TURN, skills: SKILLS }).offerId!;+ const fixture = join(vault, "session.jsonl");+ writeSessionFixture(fixture, [+ { id: "c1", skill: "release", offerId },+ { id: "c2", skill: "release" },+ { id: "c3", skill: "triage" },+ ]);+ await importSession(vault, fixture, { agent: "test" });++ const usage = deriveSkillUsage(vault, { nowMs: Date.parse("2026-06-02T00:00:00Z") });+ const release = usage.find((u) => u.skill === "release")!;+ expect(release.invocationCount).toBe(2);+ expect(release.offerAttributedCount).toBe(1);+ const triage = usage.find((u) => u.skill === "triage")!;+ expect(triage.invocationCount).toBe(1);+ expect(triage.offerAttributedCount).toBe(0);+});++test("a vault with no invocations joins to an empty report", () => {+ expect(joinSkillInvocationsToOffers(vault)).toEqual({ offers: [], unattributed: [] });+});diff --git a/tests/core/search/capability-tier.test.ts b/tests/core/search/capability-tier.test.tsnew file mode 100644index 00000000..5c69be3c--- /dev/null+++ b/tests/core/search/capability-tier.test.ts@@ -0,0 +1,402 @@+/**+ * The semantic capability tier is resolved ONCE and agreed on by every+ * surface that reports it (provenance-at-the-boundary, unit F).+ *+ * Four sites used to compute "is semantic search configured, and if not+ * why" from the resolved config separately, in four spellings, one of+ * them assembling operator-facing sentences at the call site. This file+ * drives all four against one config matrix and requires them to say the+ * same thing:+ *+ * 1. `runSemanticPhase` (`search/semantic-phase.ts`)+ * 2. `indexVault().deferredReason` (`search/indexer.ts`)+ * 3. `probeEmbeddingProvider` (`doctor-readiness.ts`)+ * 4. `indexCheck()` (`search/indexer.ts`)+ *+ * The second half covers the capability PREDICATE that replaced the+ * `provider.name === "null"` sentinel: a provider whose name is neither+ * of the literals in the tree must be classified by what it declares it+ * can do, never by what it is called.+ *+ * No test here reaches the network. The one configuration that needs an+ * embedding endpoint points at the loopback fake server the search+ * fixtures already ship, and `probeEmbeddingProvider` is deliberately+ * driven only on configurations it cannot make an outbound call for.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import {+ resolveSemanticCapability,+ SEMANTIC_CAPABILITY_CODE,+ SEMANTIC_CAPABILITY_TIER,+ semanticCapabilityIsBlocked,+ semanticCapabilityLabel,+ SEMANTIC_VECTOR_CODE,+} from "../../../src/core/search/capability-tier.ts";+import {+ providerProducesVectors,+ type EmbeddingProvider,+} from "../../../src/core/search/embeddings/contract.ts";+import { NullProvider } from "../../../src/core/search/embeddings/null-provider.ts";+import { indexCheck, indexVault } from "../../../src/core/search/indexer.ts";+import { runSemanticPhase } from "../../../src/core/search/semantic-phase.ts";+import { Store } from "../../../src/core/search/store.ts";+import type { ResolvedEmbeddingConfig } from "../../../src/core/search/types.ts";+import { probeEmbeddingProvider } from "../../../src/core/doctor-readiness.ts";+import { resolveNextStep } from "../../../src/core/brain/next-step.ts";+import { detectSemanticDedup } from "../../../src/core/brain/hygiene/detectors/dedup.ts";+import { createTempVault, makeConfig, writeMd } from "../../helpers/search-fixtures.ts";+import { startFakeHttp, type FakeHttp } from "../../helpers/fake-http.ts";+import { sqliteVecLoadable } from "../../helpers/sqlite-vec.ts";++/** An endpoint nothing listens on; used where no call may be made. */+const REFUSED_ENDPOINT = "http://127.0.0.1:9";++/**+ * Whether `sqlite-vec` loaded in THIS process, resolved once at module+ * load. Only the `runSemanticPhase` test needs it - that arm requires a+ * store that really holds vectors - and it is declared with `skipIf` so a+ * missing extension is REPORTED as a skip rather than returning before+ * its first assertion and reading as a pass.+ */+const VEC_LOADABLE = sqliteVecLoadable();++/**+ * Set only on a platform that genuinely cannot load `sqlite-vec`. Absent+ * - the default, and what CI runs - an unloadable extension is a FAULT,+ * not a platform fact, and the loadability guard below turns this file+ * red instead of letting the fourth site's coverage disappear silently.+ */+const ALLOW_MISSING_VEC_ENV = "O2B_ALLOW_MISSING_SQLITE_VEC";++/** Whether this environment has declared the extension optional. */+const VEC_DECLARED_OPTIONAL = process.env[ALLOW_MISSING_VEC_ENV] !== undefined;++/** How many of the five variants are blocked, and so measurable below. */+const BLOCKED_VARIANT_COUNT = 3;++let vault: string;+let dbPath: string;+let cleanup: () => void;++beforeEach(() => {+ const v = createTempVault("capability-tier");+ vault = v.vault;+ dbPath = v.dbPath;+ cleanup = v.cleanup;+});++afterEach(() => {+ cleanup();+});++interface Variant {+ readonly label: string;+ readonly semantic: Partial<ResolvedEmbeddingConfig>;+ readonly tier: string;+}++/** The configurations the tier ladder must distinguish. */+function semanticVariants(baseUrl: string): ReadonlyArray<Variant> {+ return [+ { label: "disabled", semantic: { enabled: false }, tier: SEMANTIC_CAPABILITY_TIER.disabled },+ {+ label: "provider disabled while enabled",+ semantic: { enabled: true, provider: "disabled" },+ tier: SEMANTIC_CAPABILITY_TIER.disabled,+ },+ {+ label: "remote provider, no key",+ semantic: { enabled: true, provider: "openai-compat", apiKey: null },+ tier: SEMANTIC_CAPABILITY_TIER.credentialMissing,+ },+ {+ label: "local provider needs no key",+ semantic: { enabled: true, provider: "local", apiKey: null, dimension: 4 },+ tier: SEMANTIC_CAPABILITY_TIER.configured,+ },+ {+ label: "remote provider with a key",+ semantic: {+ enabled: true,+ provider: "openai-compat",+ baseUrl,+ model: "fake-model",+ apiKey: "test-key",+ dimension: 4,+ },+ tier: SEMANTIC_CAPABILITY_TIER.configured,+ },+ ];+}++describe("the tier resolver", () => {+ test("classifies each configuration into exactly one tier", () => {+ for (const variant of semanticVariants(REFUSED_ENDPOINT)) {+ const cfg = makeConfig({ vault, dbPath, semantic: variant.semantic });+ expect(`${variant.label}: ${resolveSemanticCapability(cfg.semantic).tier}`).toBe(+ `${variant.label}: ${variant.tier}`,+ );+ }+ });++ test("every code the resolver can produce is registered on the advisory rail", async () => {+ const codes = [+ ...Object.values(SEMANTIC_CAPABILITY_CODE),+ ...Object.values(SEMANTIC_VECTOR_CODE),+ ];+ expect(codes.length).toBeGreaterThan(4);+ for (const code of codes) {+ expect(`${code} registered: ${resolveNextStep(code) !== null}`).toBe(+ `${code} registered: true`,+ );+ // The label a surface renders is the registry's, never a sentence+ // assembled beside the call.+ expect(await semanticCapabilityLabel(code)).toBe(resolveNextStep(code)!.issueClass);+ }+ });++ test("only the configured tier is unblocked", () => {+ for (const variant of semanticVariants(REFUSED_ENDPOINT)) {+ const cfg = makeConfig({ vault, dbPath, semantic: variant.semantic });+ const capability = resolveSemanticCapability(cfg.semantic);+ expect(`${variant.label}: ${semanticCapabilityIsBlocked(capability)}`).toBe(+ `${variant.label}: ${variant.tier !== SEMANTIC_CAPABILITY_TIER.configured}`,+ );+ }+ });+});++describe("the four sites agree with the resolver", () => {+ test("indexVault's deferred reason is the registry label for the resolved tier", async () => {+ for (const variant of semanticVariants(REFUSED_ENDPOINT)) {+ const v = createTempVault("capability-defer");+ try {+ writeMd(v.vault, "a.md", "# A\n\nFirst note about something.");+ const cfg = makeConfig({ vault: v.vault, dbPath: v.dbPath, semantic: variant.semantic });+ const stats = await indexVault(cfg);+ const capability = resolveSemanticCapability(cfg.semantic);+ // Configured-but-not-asked-for is its own registered state, not a+ // capability fault.+ const expected = semanticCapabilityIsBlocked(capability)+ ? await semanticCapabilityLabel(capability.code)+ : await semanticCapabilityLabel(SEMANTIC_VECTOR_CODE.deferred);+ expect(`${variant.label}: ${stats.deferredReason}`).toBe(`${variant.label}: ${expected}`);+ } finally {+ v.cleanup();+ }+ }+ });++ test("indexCheck reports the key as resolved exactly on the configured tier", async () => {+ for (const variant of semanticVariants(REFUSED_ENDPOINT)) {+ // A remote provider with a key would ping; the refused base URL+ // keeps that on loopback and off the network.+ const cfg = makeConfig({+ vault,+ dbPath,+ semantic: { ...variant.semantic, baseUrl: REFUSED_ENDPOINT },+ });+ const report = await indexCheck(cfg);+ expect(`${variant.label}: ${report.embeddingKeyResolved}`).toBe(+ `${variant.label}: ${variant.tier === SEMANTIC_CAPABILITY_TIER.configured}`,+ );+ }+ });++ test("probeEmbeddingProvider skips exactly the blocked tiers", async () => {+ const tmp = mkdtempSync(join(tmpdir(), "o2b-capability-probe-"));+ const configPath = join(tmp, "config.yaml");+ const envKeys = [+ "OPEN_SECOND_BRAIN_SEARCH_SEMANTIC",+ "OPEN_SECOND_BRAIN_EMBEDDING_PROVIDER",+ "OPEN_SECOND_BRAIN_EMBEDDING_BASE_URL",+ "OPEN_SECOND_BRAIN_EMBEDDING_MODEL",+ "OPEN_SECOND_BRAIN_EMBEDDING_KEY",+ "OPEN_SECOND_BRAIN_EMBEDDING_DIM",+ ];+ const saved: Record<string, string | undefined> = {};+ for (const k of envKeys) {+ saved[k] = process.env[k];+ delete process.env[k];+ }+ try {+ // Only configurations that cannot make an outbound call: disabled+ // (no provider at all), key-missing (which must be refused BEFORE+ // any ping), and the offline local provider.+ const cases = [+ { body: "", tier: SEMANTIC_CAPABILITY_TIER.disabled },+ {+ body: "search_semantic_enabled: true\nembedding_provider: openai-compat\n",+ tier: SEMANTIC_CAPABILITY_TIER.credentialMissing,+ },+ {+ body: "search_semantic_enabled: true\nembedding_provider: local\n",+ tier: SEMANTIC_CAPABILITY_TIER.configured,+ },+ ];+ for (const c of cases) {+ writeFileSync(configPath, `vault: "${tmp}"\n${c.body}`);+ const verdict = await probeEmbeddingProvider({ vault: tmp, config: configPath });+ const blocked = c.tier !== SEMANTIC_CAPABILITY_TIER.configured;+ expect(`${c.tier}: ${verdict.status === "skipped"}`).toBe(`${c.tier}: ${blocked}`);+ }+ } finally {+ for (const k of envKeys) {+ if (saved[k] === undefined) delete process.env[k];+ else process.env[k] = saved[k];+ }+ rmSync(tmp, { recursive: true, force: true });+ }+ });++ test.skipIf(VEC_DECLARED_OPTIONAL)(+ "the vector extension the runSemanticPhase arm depends on is loadable",+ () => {+ expect(VEC_LOADABLE).toBe(true);+ },+ );++ test.skipIf(!VEC_LOADABLE)(+ "runSemanticPhase degrades with the registry label for the resolved tier",+ async () => {+ const server: FakeHttp = await startFakeHttp();+ try {+ writeMd(vault, "a.md", "# A\n\nFirst note about something.");+ const indexed = makeConfig({+ vault,+ dbPath,+ semantic: {+ enabled: true,+ provider: "openai-compat",+ baseUrl: server.url,+ model: "fake-model",+ apiKey: "test-key",+ dimension: 4,+ timeoutMs: 5_000,+ },+ });+ // A store that HAS vectors and HAS the extension, so the runtime+ // guards pass and what is measured is the config-derived arm.+ await indexVault(indexed, { embeddings: true });+ const store = await Store.open(indexed, { mode: "read" });+ try {+ expect(store.counts().embeddings).toBeGreaterThan(0);+ let measured = 0;+ for (const variant of semanticVariants(server.url)) {+ const cfg = makeConfig({ vault, dbPath, semantic: variant.semantic });+ const capability = resolveSemanticCapability(cfg.semantic);+ if (!semanticCapabilityIsBlocked(capability)) continue;+ measured++;+ const outcome = await runSemanticPhase(store, cfg, "something", {+ limit: 5,+ pathPrefix: undefined,+ explicit: false,+ });+ expect(`${variant.label}: ${outcome.attempted}`).toBe(`${variant.label}: false`);+ expect(`${variant.label}: ${outcome.warnings.join("|")}`).toBe(+ `${variant.label}: ${await semanticCapabilityLabel(capability.code)}`,+ );+ }+ // A loop that measured nothing would report agreement over an+ // empty set.+ expect(measured).toBe(BLOCKED_VARIANT_COUNT);+ } finally {+ await store.close();+ }+ } finally {+ await server.close();+ }+ },+ );+});++/**+ * A provider whose name is neither `null` nor any name this tree ships.+ * It DECLARES that it produces no vectors while still returning some, so+ * a site that honours the declaration falls back and a site that still+ * compares names does not - which is exactly the defect the predicate+ * removes.+ */+class SentinelNamedProvider implements EmbeddingProvider {+ readonly name = "capability-test-sentinel";+ readonly model = "capability-test-sentinel-model";+ readonly dimension = 3;+ readonly producesVectors = false;+ calls = 0;++ embed(texts: ReadonlyArray<string>): Promise<number[][]> {+ this.calls++;+ return Promise.resolve(texts.map(() => [1, 0, 0]));+ }++ async ping(): Promise<{ ok: true; dimension: number }> {+ return { ok: true, dimension: 3 };+ }+}++/** A real provider that declares nothing: the default must be "it embeds". */+class UndeclaredProvider implements EmbeddingProvider {+ readonly name = "capability-test-working";+ readonly model = "capability-test";+ readonly dimension = 3;++ embed(texts: ReadonlyArray<string>): Promise<number[][]> {+ return Promise.resolve(texts.map(() => [1, 0, 0]));+ }++ async ping(): Promise<{ ok: true; dimension: number }> {+ return { ok: true, dimension: 3 };+ }+}++describe("the capability predicate replaces the name sentinel", () => {+ test("classification follows the declaration, not the name", () => {+ expect(providerProducesVectors(new SentinelNamedProvider())).toBe(false);+ expect(providerProducesVectors(new UndeclaredProvider())).toBe(true);+ expect(providerProducesVectors(new NullProvider())).toBe(false);+ });++ test("the semantic dedup detector falls back on a differently-named sentinel", async () => {+ const prefVault = mkdtempSync(join(tmpdir(), "o2b-capability-dedup-"));+ try {+ const dir = join(prefVault, "Brain", "preferences");+ mkdirSync(dir, { recursive: true });+ const writePref = (slug: string, topic: string, principle: string): void => {+ writeFileSync(+ join(dir, `pref-${slug}.md`),+ [+ "---",+ "kind: brain-preference",+ `id: pref-${slug}`,+ "tags: [brain, brain/preference]",+ `topic: ${topic}`,+ "_status: confirmed",+ `principle: ${principle}`,+ "created_at: 2026-01-01T00:00:00Z",+ "unconfirmed_until: 2026-01-15T00:00:00Z",+ "---",+ "",+ ].join("\n"),+ "utf8",+ );+ };+ writePref("a", "writing-style", "Never use exclamation marks in docs");+ writePref("b", "doc-tone", "Do not use exclamation marks in documentation");++ const provider = new SentinelNamedProvider();+ const result = await detectSemanticDedup(prefVault, { provider });+ // Under the old `provider.name === "null"` sentinel this provider+ // passed the guard, embedded both principles, and the report read+ // `embedding`.+ expect(result.method).toBe("lexical");+ expect(provider.calls).toBe(0);+ } finally {+ rmSync(prefVault, { recursive: true, force: true });+ }+ });+});diff --git a/tests/core/search/embeddings.batch-tokens.test.ts b/tests/core/search/embeddings.batch-tokens.test.tsnew file mode 100644index 00000000..ac68d51e--- /dev/null+++ b/tests/core/search/embeddings.batch-tokens.test.ts@@ -0,0 +1,370 @@+/**+ * Per-request token budget for embedding batches (provenance-at-the-boundary,+ * unit E / t_39ec3fef).+ *+ * A batch closes on whichever fills first: the count cap+ * (`embedding_batch_size`) or the accumulated token estimate+ * (`embedding_batch_tokens`). With the budget absent, batching is+ * byte-identical to the fixed-stride behaviour that preceded it.+ *+ * The estimate is `estimateTokens` from `embeddings/signature.ts` - the+ * estimator already resident in this layer and already governing the cost+ * gate - so the gate and the budget describe the same texts identically.+ */++import { test, expect, beforeEach, afterEach } from "bun:test";+import { mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import {+ chunkArray,+ chunkArrayByTokenBudget,+} from "../../../src/core/search/embeddings/http-util.ts";+import { estimateTokens } from "../../../src/core/search/embeddings/signature.ts";+import { OpenAICompatProvider } from "../../../src/core/search/embeddings/openai-compat.ts";+import { ZeroEntropyProvider } from "../../../src/core/search/embeddings/zeroentropy.ts";+import { resolveSearchConfig } from "../../../src/core/search/index.ts";+import type { ResolvedEmbeddingConfig } from "../../../src/core/search/types.ts";+import { startFakeHttp, type FakeHttp } from "../../helpers/fake-http.ts";++// ── fixtures ─────────────────────────────────────────────────────────────────++/** 40 chars => ceil(40/4) = 10 estimated tokens. */+const TEXT_10_TOKENS = "a".repeat(40);+/** 36 chars => 9 tokens bare; 45 chars => 12 tokens once a 9-char prefix lands. */+const TEXT_9_TOKENS = "b".repeat(36);+/** 100 chars => 25 estimated tokens - larger than any budget used below. */+const TEXT_25_TOKENS = "c".repeat(100);++let server: FakeHttp;++function openAiCfg(overrides: Partial<ResolvedEmbeddingConfig> = {}): ResolvedEmbeddingConfig {+ return Object.freeze({+ enabled: true,+ provider: "openai-compat",+ baseUrl: server.url,+ model: "fake-model",+ apiKey: "test-key",+ dimension: 4,+ timeoutMs: 5_000,+ // Serial batches keep the captured request order deterministic.+ concurrency: 1,+ batchSize: 32,+ costGateUsd: 0,+ maxRetries: 3,+ ...overrides,+ });+}++function zeroEntropyCfg(overrides: Partial<ResolvedEmbeddingConfig> = {}): ResolvedEmbeddingConfig {+ return Object.freeze({+ enabled: true,+ provider: "zeroentropy",+ baseUrl: server.url,+ model: "zembed-1",+ apiKey: "ze-test-key",+ dimension: 4,+ timeoutMs: 5_000,+ concurrency: 1,+ batchSize: 32,+ costGateUsd: 0,+ maxRetries: 3,+ ...overrides,+ });+}++/** Records each request's `input` array and answers with OpenAI-shaped vectors. */+function captureOpenAi(sent: string[][]) {+ return (req: { body: unknown }) => {+ const body = req.body as { input: string[]; model: string };+ sent.push([...body.input]);+ const data = body.input.map((t, i) => ({+ object: "embedding",+ embedding: [t.length, i, 1, 1],+ index: i,+ }));+ return { status: 200, body: { data, model: body.model } };+ };+}++/** Records each request's `input` array and answers with ZeroEntropy-shaped vectors. */+function captureZeroEntropy(sent: string[][]) {+ return (req: { method: string; path: string; body: unknown }) => {+ if (!req.path.endsWith("/models/embed") || req.method !== "POST") {+ return { status: 404, body: { error: "not_found" } };+ }+ const body = req.body as { input: string[] };+ sent.push([...body.input]);+ const results = body.input.map((text, index) => ({+ embedding: [text.length, index, 1, 1],+ }));+ return { status: 200, body: { results } };+ };+}++beforeEach(async () => {+ server = await startFakeHttp();+});++afterEach(async () => {+ await server.close();+});++// ── the packer itself ────────────────────────────────────────────────────────++test("chunkArrayByTokenBudget with no budget is identical to the fixed stride", () => {+ const texts = [TEXT_10_TOKENS, TEXT_9_TOKENS, TEXT_25_TOKENS, "d", ""];+ for (const size of [1, 2, 3, 32]) {+ expect(chunkArrayByTokenBudget(texts, size, undefined, (t) => t)).toEqual(+ chunkArray(texts, size),+ );+ }+});++test("a batch closes on the token budget before the count cap when the estimate fills first", () => {+ const texts = [TEXT_10_TOKENS, TEXT_10_TOKENS, TEXT_10_TOKENS, TEXT_10_TOKENS];+ const batches = chunkArrayByTokenBudget(texts, 32, 25, (t) => t);+ expect(batches.map((b) => b.length)).toEqual([2, 2]);+ for (const b of batches) expect(estimateTokens(b)).toBeLessThanOrEqual(25);+});++test("a batch closes on the count cap when the cap fills first", () => {+ const texts = [TEXT_10_TOKENS, TEXT_10_TOKENS, TEXT_10_TOKENS, TEXT_10_TOKENS, TEXT_10_TOKENS];+ const batches = chunkArrayByTokenBudget(texts, 2, 1000, (t) => t);+ expect(batches.map((b) => b.length)).toEqual([2, 2, 1]);+});++test("an item larger than the whole budget travels alone rather than being dropped", () => {+ const texts = [TEXT_10_TOKENS, TEXT_25_TOKENS, TEXT_10_TOKENS];+ const batches = chunkArrayByTokenBudget(texts, 32, 12, (t) => t);+ expect(batches).toEqual([[TEXT_10_TOKENS], [TEXT_25_TOKENS], [TEXT_10_TOKENS]]);+ expect(batches.flat().length).toBe(texts.length);+});++test("every input appears exactly once, in order, whatever the budget", () => {+ const texts = Array.from({ length: 17 }, (_, i) => "x".repeat(i * 7));+ for (const budget of [1, 5, 13, 40, 10_000]) {+ expect(chunkArrayByTokenBudget(texts, 4, budget, (t) => t).flat()).toEqual(texts);+ }+});++// ── openai-compat provider ───────────────────────────────────────────────────++test("openai-compat with no token budget batches exactly as the count cap alone dictates", async () => {+ const sent: string[][] = [];+ server.setHandler(captureOpenAi(sent));+ const texts = [TEXT_10_TOKENS, TEXT_10_TOKENS, TEXT_10_TOKENS, TEXT_10_TOKENS];+ await new OpenAICompatProvider(openAiCfg()).embed(texts);+ expect(sent).toEqual([texts]);+ expect(server.callCount()).toBe(1);+});++test("openai-compat closes a batch on the token budget", async () => {+ const sent: string[][] = [];+ server.setHandler(captureOpenAi(sent));+ const texts = [TEXT_10_TOKENS, TEXT_10_TOKENS, TEXT_10_TOKENS, TEXT_10_TOKENS];+ await new OpenAICompatProvider(openAiCfg({ batchTokens: 25 })).embed(texts);+ expect(sent).toEqual([+ [TEXT_10_TOKENS, TEXT_10_TOKENS],+ [TEXT_10_TOKENS, TEXT_10_TOKENS],+ ]);+});++test("the instruction prefix counts against the token budget", async () => {+ const texts = [TEXT_9_TOKENS, TEXT_9_TOKENS, TEXT_9_TOKENS, TEXT_9_TOKENS];+ const budget = 38;++ // Bare: 4 x 9 = 36 tokens, inside the budget -> one request.+ const bare: string[][] = [];+ server.setHandler(captureOpenAi(bare));+ await new OpenAICompatProvider(openAiCfg({ batchTokens: budget })).embed(texts, "passage");+ expect(bare.map((b) => b.length)).toEqual([4]);++ // With a 9-char prefix each text estimates at 12 tokens, so the fourth+ // no longer fits and the batch closes at three.+ const prefixed: string[][] = [];+ server.setHandler(captureOpenAi(prefixed));+ await new OpenAICompatProvider(+ openAiCfg({ batchTokens: budget, passagePrefix: "passage: " }),+ ).embed(texts, "passage");+ expect(prefixed.map((b) => b.length)).toEqual([3, 1]);+ for (const b of prefixed) {+ for (const t of b) expect(t.startsWith("passage: ")).toBe(true);+ expect(estimateTokens(b)).toBeLessThanOrEqual(budget);+ }+});++test("openai-compat still returns every vector in input order under a budget", async () => {+ server.setHandler(captureOpenAi([]));+ const texts = [TEXT_10_TOKENS, TEXT_25_TOKENS, TEXT_9_TOKENS];+ const out = await new OpenAICompatProvider(openAiCfg({ batchTokens: 12 })).embed(texts);+ expect(out.length).toBe(3);+ for (const v of out) expect(v.length).toBe(4);+});++// ── zeroentropy provider ─────────────────────────────────────────────────────++test("zeroentropy with no token budget batches exactly as the count cap alone dictates", async () => {+ const sent: string[][] = [];+ server.setHandler(captureZeroEntropy(sent));+ const texts = [TEXT_10_TOKENS, TEXT_10_TOKENS, TEXT_10_TOKENS, TEXT_10_TOKENS];+ await new ZeroEntropyProvider(zeroEntropyCfg()).embed(texts);+ expect(sent).toEqual([texts]);+});++test("zeroentropy closes a batch on the token budget", async () => {+ const sent: string[][] = [];+ server.setHandler(captureZeroEntropy(sent));+ const texts = [TEXT_10_TOKENS, TEXT_10_TOKENS, TEXT_10_TOKENS, TEXT_10_TOKENS];+ const out = await new ZeroEntropyProvider(zeroEntropyCfg({ batchTokens: 25 })).embed(texts);+ expect(sent).toEqual([+ [TEXT_10_TOKENS, TEXT_10_TOKENS],+ [TEXT_10_TOKENS, TEXT_10_TOKENS],+ ]);+ expect(out.length).toBe(4);+});++// ── config resolution ────────────────────────────────────────────────────────++const CONFIG_ENV_KEYS = [+ "OPEN_SECOND_BRAIN_EMBEDDING_BATCH",+ "OPEN_SECOND_BRAIN_EMBEDDING_BATCH_TOKENS",+] as const;++let vault: string;+let configPath: string;+let savedEnv: Record<string, string | undefined>;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "osb-batch-tokens-"));+ configPath = join(vault, "_brain.yaml");+ savedEnv = {};+ for (const k of CONFIG_ENV_KEYS) {+ savedEnv[k] = process.env[k];+ delete process.env[k];+ }+});++afterEach(() => {+ for (const k of CONFIG_ENV_KEYS) {+ if (savedEnv[k] === undefined) delete process.env[k];+ else process.env[k] = savedEnv[k];+ }+ rmSync(vault, { recursive: true, force: true });+});++test("with embedding_batch_tokens absent the resolved embedding config is byte-identical", () => {+ writeFileSync(configPath, `vault: "${vault}"\n`);+ const semantic = resolveSearchConfig({ vault, configPath }).semantic;+ // Absence is absence: the key is not present at all, so JSON round-trips+ // and key-order-sensitive consumers see exactly the pre-feature object.+ expect("batchTokens" in semantic).toBe(false);+ expect(semantic.batchTokens).toBeUndefined();+ expect(Object.keys(semantic)).toEqual([+ "enabled",+ "provider",+ "baseUrl",+ "model",+ "apiKey",+ "apiKeys",+ "dimension",+ "timeoutMs",+ "concurrency",+ "batchSize",+ "maxRetries",+ "costGateUsd",+ "queryPrefix",+ "passagePrefix",+ ]);+ expect(JSON.stringify(semantic)).not.toContain("batchTokens");+});++test("embedding_batch_tokens resolves from config and env overrides config", () => {+ writeFileSync(configPath, `vault: "${vault}"\nembedding_batch_tokens: "5000"\n`);+ expect(resolveSearchConfig({ vault, configPath }).semantic.batchTokens).toBe(5000);++ process.env["OPEN_SECOND_BRAIN_EMBEDDING_BATCH_TOKENS"] = "7000";+ expect(resolveSearchConfig({ vault, configPath }).semantic.batchTokens).toBe(7000);+});++test("a very large embedding_batch_tokens resolves to the number the operator wrote", () => {+ writeFileSync(configPath, `vault: "${vault}"\nembedding_batch_tokens: "${OVER_INT32_CAP}"\n`);+ expect(resolveSearchConfig({ vault, configPath }).semantic.batchTokens).toBe(OVER_INT32_CAP);+});++test("a fractional integer field is refused at config load, whatever its source", () => {+ // The string path already refuses it; a programmatic override must too,+ // or the packer downstream is handed a cap it cannot step by.+ writeFileSync(configPath, `vault: "${vault}"\nembedding_batch_tokens: "2.5"\n`);+ expect(() => resolveSearchConfig({ vault, configPath })).toThrow(/embedding_batch_tokens/);+ writeFileSync(configPath, `vault: "${vault}"\n`);+ expect(() =>+ resolveSearchConfig({ vault, configPath, overrides: { semantic: { batchTokens: 2.5 } } }),+ ).toThrow(/embedding_batch_tokens/);+ expect(() =>+ resolveSearchConfig({ vault, configPath, overrides: { semantic: { batchSize: 2.5 } } }),+ ).toThrow(/embedding_batch_size/);+});++test("embedding_batch_tokens is rejected at zero as a string, like embedding_batch_size", () => {+ for (const key of ["embedding_batch_size", "embedding_batch_tokens"]) {+ writeFileSync(configPath, `vault: "${vault}"\n${key}: "0"\n`);+ expect(() => resolveSearchConfig({ vault, configPath })).toThrow(key);+ }+});++// ── caps beyond the 32-bit boundary ──────────────────────────────────────────++/**+ * Above 2^31-1. `Math.max(1, n | 0)` used to truncate a cap this size into a+ * NEGATIVE int32 and then lift it to 1, so the largest budget an operator can+ * ask for became the smallest one possible: a batch that closes after every+ * item, i.e. one HTTP request per chunk at full concurrency.+ */+const OVER_INT32_CAP = 3_000_000_000;++test("a token budget above the 32-bit boundary is honoured, not truncated", () => {+ const texts = [TEXT_10_TOKENS, TEXT_10_TOKENS, TEXT_10_TOKENS, TEXT_10_TOKENS];+ expect(chunkArrayByTokenBudget(texts, 32, OVER_INT32_CAP, (t) => t)).toEqual([texts]);+});++test("a count cap above the 32-bit boundary is honoured, not truncated", () => {+ const texts = [TEXT_10_TOKENS, TEXT_9_TOKENS, TEXT_25_TOKENS];+ expect(chunkArray(texts, OVER_INT32_CAP)).toEqual([texts]);+ expect(chunkArrayByTokenBudget(texts, OVER_INT32_CAP, 10_000, (t) => t)).toEqual([texts]);+});++test("openai-compat sends one request for a budget no batch can exhaust", async () => {+ const sent: string[][] = [];+ server.setHandler(captureOpenAi(sent));+ const texts = [TEXT_10_TOKENS, TEXT_10_TOKENS, TEXT_10_TOKENS, TEXT_10_TOKENS];+ await new OpenAICompatProvider(openAiCfg({ batchTokens: OVER_INT32_CAP })).embed(texts);+ expect(sent).toEqual([texts]);+ expect(server.callCount()).toBe(1);+});++test("a cap that cannot be honoured is refused, never quietly reinterpreted", () => {+ const texts = [TEXT_10_TOKENS, TEXT_10_TOKENS];+ for (const bad of [0, -1, 2.5, Number.NaN, Number.POSITIVE_INFINITY]) {+ expect(() => chunkArrayByTokenBudget(texts, 32, bad, (t) => t)).toThrow(/token budget/);+ expect(() => chunkArray(texts, bad)).toThrow(/batch size/);+ }+});++test("embedding_batch_tokens is rejected at zero as a programmatic override", () => {+ writeFileSync(configPath, `vault: "${vault}"\n`);+ expect(() =>+ resolveSearchConfig({ vault, configPath, overrides: { semantic: { batchTokens: 0 } } }),+ ).toThrow(/embedding_batch_tokens/);+ expect(() =>+ resolveSearchConfig({ vault, configPath, overrides: { semantic: { batchTokens: -1 } } }),+ ).toThrow(/embedding_batch_tokens/);+ expect(() =>+ resolveSearchConfig({+ vault,+ configPath,+ overrides: { semantic: { batchTokens: Number.NaN } },+ }),+ ).toThrow(/embedding_batch_tokens/);+});diff --git a/tests/core/search/event-anchor-backfill.test.ts b/tests/core/search/event-anchor-backfill.test.tsnew file mode 100644index 00000000..eae8bbcb--- /dev/null+++ b/tests/core/search/event-anchor-backfill.test.ts@@ -0,0 +1,250 @@+/**+ * The event-anchor backfill (provenance at the boundary, t_ac1c4176).+ *+ * The v11 migration adds the anchor columns to an EXISTING index in+ * place - nothing reindexes - and the indexer populates them only on the+ * changed-document path, below both content-identity fastpaths. A+ * document indexed by the previous binary therefore has no anchor, its+ * content never changes, and so no anchor is ever computed for it: the+ * feature is inert on every vault that already existed, and its+ * inertness reads exactly like a note that declares no date.+ *+ * These tests pin the two halves of the cure:+ *+ * 1. The index distinguishes "examined, declares nothing" from "never+ * examined by an anchor-aware binary". Without that column the two+ * are one NULL and no surface can tell an operator which they have.+ * 2. `planEventAnchorBackfill` closes the gap under the same contract+ * `authored-at-backfill` and `vector-backfill` follow: a `plan*`+ * function in core does the finding, dry-run is the DEFAULT,+ * `apply` is the only mutating path, and a file it could not read+ * is NAMED rather than counted as done.+ *+ * The upgrade is simulated the only faithful way: a real index is built,+ * its anchor columns are dropped, its schema version is rewound to v10,+ * and the next `Store.open` runs the real v11 migration over it.+ */++import { test, expect, beforeEach, afterEach } from "bun:test";+import { Database } from "bun:sqlite";+import { rmSync, utimesSync } from "node:fs";+import { join } from "node:path";++import { planEventAnchorBackfill } from "../../../src/core/search/event-anchor-backfill.ts";+import { EVENT_ANCHOR_SOURCE, type EventAnchor } from "../../../src/core/search/event-anchor.ts";+import { indexVault } from "../../../src/core/search/indexer.ts";+import { search } from "../../../src/core/search/search.ts";+import { Store } from "../../../src/core/search/store.ts";+import { createTempVault, makeConfig, writeMd } from "../../helpers/search-fixtures.ts";++const DAY_MS = 24 * 60 * 60 * 1000;+/** Budget for a test that builds an index; matches the other index suites. */+const INDEX_TEST_TIMEOUT_MS = 30_000;+/** Every fixture is written inside the queried window ... */+const MTIME_IN_WINDOW = new Date("2026-07-01T00:00:00Z");+/** ... and the dated one DECLARES a day two years outside it. */+const DECLARED = "2024-03-01";++/** The columns v11 adds; dropping them rewinds a real index to v10. */+const ANCHOR_COLUMNS = [+ "event_anchor_start_ms",+ "event_anchor_end_ms",+ "event_anchor_source",+ "event_anchor_examined",+];++let vault: string;+let dbPath: string;+let cleanup: () => void;++beforeEach(() => {+ const v = createTempVault("event-anchor-backfill");+ vault = v.vault;+ dbPath = v.dbPath;+ cleanup = v.cleanup;+});++afterEach(() => {+ cleanup();+});++/** Two notes: one declaring a body date, one declaring nothing at all. */+function seedVault(): void {+ writeMd(vault, "notes/dated.md", `# Ledger\n\nThe reconciliation entry ${DECLARED} closed it.\n`);+ writeMd(vault, "notes/plain.md", "# Ledger\n\nThe reconciliation entry ref-xy-1234 closed it.\n");+ for (const rel of ["notes/dated.md", "notes/plain.md"]) {+ utimesSync(join(vault, rel), MTIME_IN_WINDOW, MTIME_IN_WINDOW);+ }+}++/**+ * Put a built index back in the state the PREVIOUS binary left it: the+ * anchor columns and the examination marker gone, the recorded schema+ * version back at v10.+ *+ * Then do what an operator upgrading the binary does - run the indexer,+ * whose write open applies the real v11 migration in place. Every file+ * matches its stored mtime and size, so every one of them takes the+ * fastpath and no anchor is computed for any of them: the exact state+ * this backfill exists for. The run's stats are returned because what it+ * reports about that state is itself under test.+ */+async function simulateBinaryUpgrade(+ config: ReturnType<typeof makeConfig>,+): Promise<Awaited<ReturnType<typeof indexVault>>> {+ const db = new Database(dbPath);+ for (const col of ANCHOR_COLUMNS) db.run(`ALTER TABLE documents DROP COLUMN ${col}`);+ db.run("UPDATE index_state SET value = '10' WHERE key = 'schema_version'");+ db.close();+ return await indexVault(config, { embeddings: false });+}++async function anchors(+ config: ReturnType<typeof makeConfig>,+ paths: ReadonlyArray<string>,+): Promise<Array<EventAnchor | null>> {+ const store = await Store.open(config, { mode: "read" });+ try {+ return paths.map((p) => store.eventAnchorForPath(p));+ } finally {+ await store.close();+ }+}++const PATHS = ["notes/dated.md", "notes/plain.md"] as const;++const DATED_ANCHOR: EventAnchor = Object.freeze({+ startMs: Date.parse(`${DECLARED}T00:00:00Z`),+ endMs: Date.parse(`${DECLARED}T00:00:00Z`) + DAY_MS - 1,+ source: EVENT_ANCHOR_SOURCE.body,+});++test(+ "an upgraded index reports its documents as never examined, and the dry run writes nothing",+ async () => {+ seedVault();+ const config = makeConfig({ vault, dbPath });+ await indexVault(config, { embeddings: false });+ expect(await anchors(config, PATHS)).toEqual([DATED_ANCHOR, null]);++ const upgrade = await simulateBinaryUpgrade(config);+ expect(upgrade.unchanged).toBe(2);++ // Both rows are now indistinguishable from "declares nothing" on+ // their columns alone - which is precisely why the examination+ // marker exists to tell them apart.+ expect(await anchors(config, PATHS)).toEqual([null, null]);++ const dry = await planEventAnchorBackfill(config);+ expect(dry.applied).toBe(false);+ expect(dry.documentsTotal).toBe(2);+ expect(dry.pending).toBe(2);+ expect(dry.examined).toBe(0);+ expect(dry.anchored).toBe(0);+ expect(dry.unreadable).toEqual([]);+ // A dry run is a pure read: the index is untouched.+ expect(await anchors(config, PATHS)).toEqual([null, null]);+ expect((await planEventAnchorBackfill(config)).pending).toBe(2);+ },+ INDEX_TEST_TIMEOUT_MS,+);++test(+ "apply materialises exactly what a full reindex would have, and is idempotent",+ async () => {+ seedVault();+ const config = makeConfig({ vault, dbPath });+ await indexVault(config, { embeddings: false });+ const fresh = await anchors(config, PATHS);+ await simulateBinaryUpgrade(config);++ const applied = await planEventAnchorBackfill(config, { apply: true });+ expect(applied.applied).toBe(true);+ expect(applied.pending).toBe(2);+ expect(applied.examined).toBe(2);+ // Only one of the two notes declares anything, and the other is now+ // KNOWN to declare nothing rather than merely unexamined.+ expect(applied.anchored).toBe(1);+ expect(applied.unreadable).toEqual([]);+ expect(await anchors(config, PATHS)).toEqual(fresh);++ // A second run finds nothing pending and does nothing: an examined+ // row that declares nothing is not a row waiting to be examined.+ const again = await planEventAnchorBackfill(config, { apply: true });+ expect(again.pending).toBe(0);+ expect(again.examined).toBe(0);+ expect(await anchors(config, PATHS)).toEqual(fresh);+ },+ INDEX_TEST_TIMEOUT_MS,+);++test(+ "a file the backfill cannot read is named and stays pending, never counted as examined",+ async () => {+ seedVault();+ const config = makeConfig({ vault, dbPath });+ await indexVault(config, { embeddings: false });+ await simulateBinaryUpgrade(config);+ rmSync(join(vault, "notes/dated.md"));++ const applied = await planEventAnchorBackfill(config, { apply: true });+ expect(applied.unreadable.map((u) => u.path)).toEqual(["notes/dated.md"]);+ expect(applied.examined).toBe(1);+ expect(applied.anchored).toBe(0);+ // Still pending, because it was never examined - the run does not+ // get to record a verdict on a file it could not open.+ expect((await planEventAnchorBackfill(config)).pending).toBe(1);+ },+ INDEX_TEST_TIMEOUT_MS,+);++test(+ "the hard since/until filter judges a backfilled note by what it declares",+ async () => {+ seedVault();+ const config = makeConfig({ vault, dbPath });+ await indexVault(config, { embeddings: false });+ await simulateBinaryUpgrade(config);++ // Inert: the note declares 2024 in its body, but the upgraded index+ // has no anchor for it, so it is judged by its 2026 mtime and stays.+ const before = await search(config, {+ query: "reconciliation entry",+ since: "2026-01-01",+ until: "2026-12-31",+ limit: 5,+ });+ expect(before.results.map((r) => r.path)).toContain("notes/dated.md");++ await planEventAnchorBackfill(config, { apply: true });++ const after = await search(config, {+ query: "reconciliation entry",+ since: "2026-01-01",+ until: "2026-12-31",+ limit: 5,+ });+ expect(after.results.map((r) => r.path)).not.toContain("notes/dated.md");+ expect(after.results.map((r) => r.path)).toContain("notes/plain.md");+ },+ INDEX_TEST_TIMEOUT_MS,+);++test(+ "an index run over an upgraded vault reports the pending anchors it did not compute",+ async () => {+ seedVault();+ const config = makeConfig({ vault, dbPath });+ await indexVault(config, { embeddings: false });++ // Every file hits the mtime/size fastpath, so the upgrade run+ // changes nothing - which is exactly the silence this field breaks.+ const stats = await simulateBinaryUpgrade(config);+ expect(stats.unchanged).toBe(2);+ expect(stats.eventAnchorsPending).toBe(2);++ await planEventAnchorBackfill(config, { apply: true });+ expect((await indexVault(config, { embeddings: false })).eventAnchorsPending).toBe(0);+ },+ INDEX_TEST_TIMEOUT_MS,+);diff --git a/tests/core/search/event-anchor-hard-filter.test.ts b/tests/core/search/event-anchor-hard-filter.test.tsnew file mode 100644index 00000000..93b73117--- /dev/null+++ b/tests/core/search/event-anchor-hard-filter.test.ts@@ -0,0 +1,221 @@+/**+ * The event anchor where it REMOVES results: the hard `since` / `until`+ * filter (provenance at the boundary, t_ac1c4176).+ *+ * `event-anchor-index.test.ts` pins both of its fixtures to one storage+ * mtime so the anchor is the only difference, and then asserts ranking+ * ORDER. That leaves the case where a result DISAPPEARS untested - a note+ * whose declared date and whose file mtime fall on opposite sides of the+ * queried window - which is the only case in which the anchor can turn a+ * hit into a non-hit. Every test here is that case, and each asserts+ * presence or absence explicitly.+ *+ * Two properties are pinned:+ *+ * 1. An anchor row this binary cannot interpret raises. The store+ * raises `INDEX_UNREADABLE` for exactly that row, and the resolver+ * must let it through: "declared no event time" and "I could not+ * read what was declared" are different answers, and the second+ * silently answered as the first would demote a document to mtime+ * for every candidate in the call, through a memoised cache.+ * 2. `created_at` / `date` frontmatter is NOT event time in this+ * release. Those keys are stamped by every Brain writer, they were+ * never event time before it, and promoting them into a filter that+ * DROPS candidates would silently change what a time-ranged query+ * means.+ */++import { test, expect, beforeEach, afterEach } from "bun:test";+import { Database } from "bun:sqlite";+import { utimesSync } from "node:fs";+import { join } from "node:path";++import { indexVault } from "../../../src/core/search/indexer.ts";+import { createEventTimeResolver } from "../../../src/core/search/pipeline/event-time.ts";+import { search } from "../../../src/core/search/search.ts";+import { SearchError } from "../../../src/core/search/types.ts";+import { createTempVault, makeConfig, writeMd } from "../../helpers/search-fixtures.ts";++/** Budget for a test that builds an index; matches the other index suites. */+const INDEX_TEST_TIMEOUT_MS = 30_000;+/** Storage mtime every fixture below is pinned to: inside the query window. */+const MTIME_IN_WINDOW = new Date("2026-07-01T00:00:00Z");+/** The date every fixture DECLARES: two years outside that window. */+const DECLARED_OUT_OF_WINDOW = "2024-03-01";++let vault: string;+let dbPath: string;+let cleanup: () => void;++beforeEach(() => {+ const v = createTempVault("event-anchor-filter");+ vault = v.vault;+ dbPath = v.dbPath;+ cleanup = v.cleanup;+});++afterEach(() => {+ cleanup();+});++function pinMtime(relPath: string): void {+ utimesSync(join(vault, relPath), MTIME_IN_WINDOW, MTIME_IN_WINDOW);+}++/** A note whose only date sits in the named frontmatter field. */+function frontmatterNote(field: string): string {+ return [+ "---",+ `${field}: ${DECLARED_OUT_OF_WINDOW}`,+ "---",+ "",+ "# Ledger",+ "",+ "The reconciliation entry closed the batch.",+ "",+ ].join("\n");+}++/** A note whose only date sits in its body. */+function bodyNote(): string {+ return [+ "# Ledger",+ "",+ `The reconciliation entry ${DECLARED_OUT_OF_WINDOW} closed the batch.`,+ "",+ ].join("\n");+}++test("an anchor row this binary cannot interpret reaches the caller instead of reading as 'no declared event time'", () => {+ writeMd(vault, "notes/a.md", "# A\n\nNo validity frontmatter anywhere.\n");+ const unreadable = new SearchError(+ "INDEX_UNREADABLE",+ "documents.event_anchor_source for 'notes/a.md' is not a registered anchor source: 'wat'",+ );+ const resolver = createEventTimeResolver(vault, new Map(), () => {+ throw unreadable;+ });++ expect(() => resolver.validityWindowFor("notes/a.md")).toThrow(unreadable);+ // And again: a first call must not have memoised a null window, which+ // would answer every later candidate in the same call with "declared+ // nothing" on the strength of a read that failed.+ expect(() => resolver.validityWindowFor("notes/a.md")).toThrow(unreadable);+ expect(() => resolver.declaredEventTimeMs("notes/a.md")).toThrow(unreadable);+});++test(+ "a corrupt anchor token fails the whole query instead of quietly demoting the note to mtime",+ async () => {+ writeMd(vault, "notes/body.md", bodyNote());+ pinMtime("notes/body.md");+ const config = makeConfig({ vault, dbPath });+ await indexVault(config, { embeddings: false });++ // A token no registered vocabulary contains: an index written by a+ // binary this one cannot interpret.+ const db = new Database(dbPath);+ db.run("UPDATE documents SET event_anchor_source = 'wat' WHERE path = 'notes/body.md'");+ db.close();++ await expect(+ search(config, {+ query: "reconciliation entry",+ since: "2026-01-01",+ until: "2026-12-31",+ limit: 5,+ }),+ ).rejects.toThrow(SearchError);+ },+ INDEX_TEST_TIMEOUT_MS,+);++test(+ "a note whose created_at disagrees with its mtime stays in a since/until filter",+ async () => {+ writeMd(vault, "notes/created-at.md", frontmatterNote("created_at"));+ writeMd(vault, "notes/date.md", frontmatterNote("date"));+ pinMtime("notes/created-at.md");+ pinMtime("notes/date.md");+ const config = makeConfig({ vault, dbPath });+ await indexVault(config, { embeddings: false });++ const outcome = await search(config, {+ query: "reconciliation entry",+ since: "2026-01-01",+ until: "2026-12-31",+ limit: 5,+ });++ // Both notes were WRITTEN inside the window and neither declares a+ // validity window, so "what did I touch since January" must still+ // answer with both. Before this release neither key was event time;+ // this release does not make them event time either.+ const paths = outcome.results.map((r) => r.path);+ expect(paths).toContain("notes/created-at.md");+ expect(paths).toContain("notes/date.md");+ },+ INDEX_TEST_TIMEOUT_MS,+);++test(+ "a body-declared date does remove the note from the filter, and says so",+ async () => {+ writeMd(vault, "notes/body.md", bodyNote());+ pinMtime("notes/body.md");+ const config = makeConfig({ vault, dbPath });+ await indexVault(config, { embeddings: false });++ const outcome = await search(config, {+ query: "reconciliation entry",+ since: "2026-01-01",+ until: "2026-12-31",+ limit: 5,+ });++ // The body rung IS event time - that is the whole scope of this+ // release - so the note is judged by the date it states rather than+ // by when its file was last written, and it drops out.+ expect(outcome.results.map((r) => r.path)).not.toContain("notes/body.md");+ // Dropping a result is not something to do quietly: the candidate+ // was removed by an INFERRED date, and the caller is told which path+ // and which rung inferred it.+ expect(outcome.warnings.some((w) => w.includes("notes/body.md") && w.includes("body"))).toBe(+ true,+ );+ },+ INDEX_TEST_TIMEOUT_MS,+);++test(+ "a declared validity window still removes the note, exactly as it did before the anchor existed",+ async () => {+ writeMd(+ vault,+ "notes/validity.md",+ [+ "---",+ `valid_from: ${DECLARED_OUT_OF_WINDOW}`,+ `valid_until: ${DECLARED_OUT_OF_WINDOW}`,+ "---",+ "",+ "# Ledger",+ "",+ "The reconciliation entry closed the batch.",+ "",+ ].join("\n"),+ );+ pinMtime("notes/validity.md");+ const config = makeConfig({ vault, dbPath });+ await indexVault(config, { embeddings: false });++ const outcome = await search(config, {+ query: "reconciliation entry",+ since: "2026-01-01",+ until: "2026-12-31",+ limit: 5,+ });+ expect(outcome.results.map((r) => r.path)).not.toContain("notes/validity.md");+ },+ INDEX_TEST_TIMEOUT_MS,+);diff --git a/tests/core/search/event-anchor-index.test.ts b/tests/core/search/event-anchor-index.test.tsnew file mode 100644index 00000000..a3dfe513--- /dev/null+++ b/tests/core/search/event-anchor-index.test.ts@@ -0,0 +1,206 @@+/**+ * The event anchor end to end (provenance at the boundary, t_ac1c4176):+ * materialised at index time onto `documents`, then consumed by the+ * query-side event-time resolver so a note is ranked by when it is ABOUT+ * rather than by when its file was last written.+ *+ * The point of materialising it is that a candidate's body is not+ * re-scanned on every query; these tests therefore assert the stored row+ * as well as the ranking it produces.+ *+ * Each test drives a real index build, so they carry the same explicit+ * 30s budget the other index-driving suites use rather than the 5s+ * default, and each shares one build across as many assertions as it can.+ */++import { test, expect, beforeEach, afterEach } from "bun:test";+import { utimesSync } from "node:fs";+import { join } from "node:path";++import { EVENT_ANCHOR_SOURCE, type EventAnchor } from "../../../src/core/search/event-anchor.ts";+import { indexVault } from "../../../src/core/search/indexer.ts";+import { TEMPORAL_INTENT_BOOST_CAP } from "../../../src/core/search/ranker.ts";+import { search } from "../../../src/core/search/search.ts";+import { Store } from "../../../src/core/search/store.ts";+import { createTempVault, makeConfig, writeMd } from "../../helpers/search-fixtures.ts";++const DAY_MS = 24 * 60 * 60 * 1000;+/** Budget for a test that builds an index; matches the other index suites. */+const INDEX_TEST_TIMEOUT_MS = 30_000;+/** Two fixtures sharing this storage mtime leave the anchor as the only difference. */+const PINNED_MTIME = new Date("2026-07-01T00:00:00Z");++let vault: string;+let dbPath: string;+let cleanup: () => void;++beforeEach(() => {+ const v = createTempVault("event-anchor");+ vault = v.vault;+ dbPath = v.dbPath;+ cleanup = v.cleanup;+});++afterEach(() => {+ cleanup();+});++/**+ * Two notes built from this helper are identical in token count and in+ * query-term frequency, so their BM25 scores tie exactly; `token` is the+ * only difference and only some of its forms are ISO dates.+ */+function noteMd(token: string): string {+ return [+ "# Ledger reconciliation",+ "",+ `The reconciliation entry ${token} closed the batch.`,+ "",+ ].join("\n");+}++function pinMtime(relPath: string): void {+ utimesSync(join(vault, relPath), PINNED_MTIME, PINNED_MTIME);+}++function dayWindow(iso: string): { startMs: number; endMs: number } {+ const startMs = Date.parse(`${iso}T00:00:00Z`);+ return { startMs, endMs: startMs + DAY_MS - 1 };+}++async function anchors(+ config: ReturnType<typeof makeConfig>,+ paths: ReadonlyArray<string>,+): Promise<Array<EventAnchor | null>> {+ const store = await Store.open(config, { mode: "read" });+ try {+ return paths.map((p) => store.eventAnchorForPath(p));+ } finally {+ store.close();+ }+}++test(+ "each ladder rung is materialised onto the document with its registered source token",+ async () => {+ writeMd(vault, "notes/dated.md", noteMd("2026-03-04"));+ writeMd(vault, "notes/plain.md", noteMd("ref-xy-1234"));+ writeMd(vault, "notes/embargo.md", noteMd("2099-12-31"));+ writeMd(vault, "notes/duration.md", noteMd("P1Y"));+ writeMd(+ vault,+ "notes/fm.md",+ ["---", "created_at: 2025-11-20", "---", "", noteMd("2026-03-04")].join("\n"),+ );+ const config = makeConfig({ vault, dbPath });+ await indexVault(config, { embeddings: false });++ const paths = [+ "notes/dated.md",+ "notes/plain.md",+ "notes/embargo.md",+ "notes/duration.md",+ "notes/fm.md",+ ];+ const first = await anchors(config, paths);++ expect(first).toEqual([+ // A body ISO date becomes a day-width window sourced to the body.+ { ...dayWindow("2026-03-04"), source: EVENT_ANCHOR_SOURCE.body },+ // A note that declares nothing anywhere stores nothing.+ null,+ // A future date is STORED, not dropped: it is a real declaration.+ { ...dayWindow("2099-12-31"), source: EVENT_ANCHOR_SOURCE.body },+ // A clock-relative derivation is never materialised.+ null,+ // Frontmatter outranks the body, and says so in its own token.+ {+ startMs: Date.parse("2025-11-20T00:00:00Z"),+ endMs: Date.parse("2025-11-20T00:00:00Z") + DAY_MS - 1,+ source: EVENT_ANCHOR_SOURCE.createdAt,+ },+ ]);++ // Re-index over UNCHANGED content. Both fastpaths gate on content+ // identity and correctly decline to recompute, so a clock-derived+ // value stored on the first run would still be sitting here on the+ // second, silently ageing. Nothing may drift between the two runs.+ await indexVault(config, { embeddings: false, force: true });+ expect(await anchors(config, paths)).toEqual(first);+ },+ INDEX_TEST_TIMEOUT_MS,+);++test(+ "a body ISO date ranks the note into a query-declared window over an equally relevant dateless peer",+ async () => {+ writeMd(vault, "notes/dated.md", noteMd("2026-03-04"));+ writeMd(vault, "notes/plain.md", noteMd("ref-xy-1234"));+ pinMtime("notes/dated.md");+ pinMtime("notes/plain.md");+ const config = makeConfig({ vault, dbPath });+ await indexVault(config, { embeddings: false });++ const outcome = await search(config, {+ query: "since:2026-03-01 until:2026-03-31 reconciliation entry",+ limit: 5,+ });+ const dated = outcome.results.find((r) => r.path === "notes/dated.md");+ const plain = outcome.results.find((r) => r.path === "notes/plain.md");+ expect(dated).toBeDefined();+ expect(plain).toBeDefined();+ // The anchor puts the dated note inside the declared window; the+ // dateless one falls back to its storage mtime, months outside it.+ expect(dated!.breakdown!["temporal"]).toBeCloseTo(TEMPORAL_INTENT_BOOST_CAP, 6);+ expect(plain!.breakdown!["temporal"]).toBeLessThan(dated!.breakdown!["temporal"]!);+ expect(outcome.results[0]!.path).toBe("notes/dated.md");+ },+ INDEX_TEST_TIMEOUT_MS,+);++test(+ "two dateless notes keep the mtime ordering the temporal layer had before the anchor existed",+ async () => {+ writeMd(vault, "notes/old.md", noteMd("ref-xy-1234"));+ writeMd(vault, "notes/new.md", noteMd("ref-xy-5678"));+ const older = new Date("2026-01-05T00:00:00Z");+ utimesSync(join(vault, "notes/old.md"), older, older);+ pinMtime("notes/new.md");+ const config = makeConfig({ vault, dbPath });+ await indexVault(config, { embeddings: false });++ expect(await anchors(config, ["notes/old.md", "notes/new.md"])).toEqual([null, null]);++ const outcome = await search(config, {+ query: "since:2026-06-15 until:2026-07-15 reconciliation entry",+ limit: 5,+ });+ // Neither note declares anything, so both are judged on mtime exactly+ // as they were before this feature; the one inside the window leads.+ expect(outcome.results[0]!.path).toBe("notes/new.md");+ },+ INDEX_TEST_TIMEOUT_MS,+);++test(+ "the anchor also composes with the hard since/until filter, not only the ranking boost",+ async () => {+ writeMd(vault, "notes/dated.md", noteMd("2026-03-04"));+ writeMd(vault, "notes/plain.md", noteMd("ref-xy-1234"));+ pinMtime("notes/dated.md");+ pinMtime("notes/plain.md");+ const config = makeConfig({ vault, dbPath });+ await indexVault(config, { embeddings: false });++ const outcome = await search(config, {+ query: "reconciliation entry",+ since: "2026-03-01",+ until: "2026-03-31",+ limit: 5,+ });+ const paths = outcome.results.map((r) => r.path);+ expect(paths).toContain("notes/dated.md");+ expect(paths).not.toContain("notes/plain.md");+ },+ INDEX_TEST_TIMEOUT_MS,+);diff --git a/tests/core/search/event-anchor.test.ts b/tests/core/search/event-anchor.test.tsnew file mode 100644index 00000000..e530209f--- /dev/null+++ b/tests/core/search/event-anchor.test.ts@@ -0,0 +1,226 @@+/**+ * Body-derived event anchor (provenance at the boundary, t_ac1c4176).+ *+ * The anchor ladder resolves ONE event interval per document from its+ * frontmatter, falling back to an ISO token in its body, and records+ * which rung won as a REGISTERED token. These tests pin the ladder, the+ * two forms the design deliberately refuses (slash dates, clock-relative+ * derivations), and the one it deliberately keeps (a future date).+ */++import { test, expect, describe } from "bun:test";++import { extractTemporalConstraints } from "../../../src/core/brain/temporal-extract.ts";+import {+ EVENT_ANCHOR_SOURCE,+ isEventAnchorSource,+ resolveEventAnchor,+} from "../../../src/core/search/event-anchor.ts";++const DAY_MS = 24 * 60 * 60 * 1000;++function dayStart(iso: string): number {+ return Date.parse(`${iso}T00:00:00Z`);+}+function dayEnd(iso: string): number {+ return dayStart(iso) + DAY_MS - 1;+}++describe("no anchor at all", () => {+ test("a document with neither frontmatter nor a body ISO token has no anchor", () => {+ expect(resolveEventAnchor({}, "Plain prose with no temporal token whatsoever.")).toBeNull();+ });++ test("an empty body and empty frontmatter has no anchor", () => {+ expect(resolveEventAnchor({}, "")).toBeNull();+ });+});++describe("the body rung", () => {+ test("a lone ISO date in the body becomes a day-width window sourced to the body", () => {+ const anchor = resolveEventAnchor({}, "The migration landed on 2026-03-04 after review.");+ expect(anchor).not.toBeNull();+ expect(anchor!.source).toBe(EVENT_ANCHOR_SOURCE.body);+ expect(anchor!.startMs).toBe(dayStart("2026-03-04"));+ expect(anchor!.endMs).toBe(dayEnd("2026-03-04"));+ });++ test("an ISO interval in the body spans day start to day end", () => {+ const anchor = resolveEventAnchor({}, "Window 2026-03-01/2026-03-05 covers the rollout.");+ expect(anchor).not.toBeNull();+ expect(anchor!.source).toBe(EVENT_ANCHOR_SOURCE.body);+ expect(anchor!.startMs).toBe(dayStart("2026-03-01"));+ expect(anchor!.endMs).toBe(dayEnd("2026-03-05"));+ });++ test("a duration anchored to a co-occurring ISO date is content-derived and is kept", () => {+ const anchor = resolveEventAnchor({}, "Valid from 2026-06-01 for P1Y.");+ expect(anchor).not.toBeNull();+ expect(anchor!.startMs).toBe(dayStart("2026-06-01"));+ expect(anchor!.endMs).toBe(dayEnd("2027-06-01"));+ });++ test("an impossible ISO-shaped date yields no anchor", () => {+ expect(resolveEventAnchor({}, "Ticket 2026-02-31 is not a day.")).toBeNull();+ });+});++describe("forms the design refuses", () => {+ test("a slash-formatted date yields no anchor, because resolving it needs a locale", () => {+ expect(resolveEventAnchor({}, "Signed 04/03/2026 in the meeting.")).toBeNull();+ expect(resolveEventAnchor({}, "Signed 2026/03/04 in the meeting.")).toBeNull();+ });++ test("body text in three non-English scripts yields no anchor", () => {+ expect(+ resolveEventAnchor(+ {},+ "Встреча состоялась четвёртого марта две тысячи двадцать шестого года.",+ ),+ ).toBeNull();+ expect(+ resolveEventAnchor({}, "La reunión fue el cuatro de marzo de dos mil veintiséis."),+ ).toBeNull();+ expect(resolveEventAnchor({}, "会议于二零二六年三月四日举行。")).toBeNull();+ });++ test("a clock-relative body derivation yields no anchor, because a stored value derived from a clock drifts", () => {+ const body = "This authorisation runs for P1Y from the moment it was written.";+ // The shared extractor DOES drift for this body: two runs over+ // UNCHANGED content under two clocks disagree.+ const early = extractTemporalConstraints(body, { now: new Date("2001-01-01T00:00:00Z") });+ const late = extractTemporalConstraints(body, { now: new Date("2031-01-01T00:00:00Z") });+ expect(early).not.toEqual(late);+ // So the anchor refuses to materialise it. Both index fastpaths gate+ // on content identity and would correctly decline to recompute a+ // stored value, so a drifting one would age silently forever.+ expect(resolveEventAnchor({}, body)).toBeNull();+ });++ test("the resolved anchor is a pure function of content across repeated runs", () => {+ const frontmatter = { created_at: "2026-03-04" };+ const body = "Renewal cadence P2Y is reviewed alongside 2026-03-04.";+ const first = resolveEventAnchor(frontmatter, body);+ const second = resolveEventAnchor(frontmatter, body);+ expect(second).toEqual(first);+ });+});++describe("a future date is stored, not dropped", () => {+ test("a body date beyond any plausible clock still produces an anchor", () => {+ const anchor = resolveEventAnchor({}, "The embargo lifts on 2099-12-31.");+ expect(anchor).not.toBeNull();+ expect(anchor!.startMs).toBe(dayStart("2099-12-31"));+ expect(anchor!.endMs).toBe(dayEnd("2099-12-31"));+ });++ test("a future frontmatter validity window still produces an anchor", () => {+ const anchor = resolveEventAnchor({ valid_from: "2099-01-01" }, "no body token");+ expect(anchor).not.toBeNull();+ expect(anchor!.startMs).toBe(dayStart("2099-01-01"));+ });+});++describe("the ladder, top rung first", () => {+ const body = "Body token 2020-01-01 sits at the bottom of the ladder.";++ test("valid_from outranks every lower rung and is sourced to the validity window", () => {+ const anchor = resolveEventAnchor(+ {+ valid_from: "2026-05-01",+ valid_until: "2026-05-31",+ created_at: "2024-01-01",+ date: "2023-01-01",+ },+ body,+ );+ expect(anchor).toEqual({+ startMs: dayStart("2026-05-01"),+ endMs: dayEnd("2026-05-31"),+ source: EVENT_ANCHOR_SOURCE.validity,+ });+ });++ test("created_at wins when no validity window is declared", () => {+ const anchor = resolveEventAnchor({ created_at: "2024-02-03", date: "2023-01-01" }, body);+ expect(anchor).toEqual({+ startMs: dayStart("2024-02-03"),+ endMs: dayEnd("2024-02-03"),+ source: EVENT_ANCHOR_SOURCE.createdAt,+ });+ });++ test("date wins when neither validity nor created_at is declared", () => {+ const anchor = resolveEventAnchor({ date: "2023-07-09" }, body);+ expect(anchor).toEqual({+ startMs: dayStart("2023-07-09"),+ endMs: dayEnd("2023-07-09"),+ source: EVENT_ANCHOR_SOURCE.date,+ });+ });++ test("the body rung is reached only when every frontmatter rung is absent", () => {+ const anchor = resolveEventAnchor({}, body);+ expect(anchor).toEqual({+ startMs: dayStart("2020-01-01"),+ endMs: dayEnd("2020-01-01"),+ source: EVENT_ANCHOR_SOURCE.body,+ });+ });++ test("a frontmatter datetime keeps its instant rather than snapping to the day", () => {+ const anchor = resolveEventAnchor({ created_at: "2024-02-03T11:22:33Z" }, body);+ expect(anchor!.startMs).toBe(Date.parse("2024-02-03T11:22:33Z"));+ expect(anchor!.endMs).toBe(Date.parse("2024-02-03T11:22:33Z"));+ });++ test("only valid_until declared still anchors on the validity rung", () => {+ const anchor = resolveEventAnchor({ valid_until: "2026-05-31" }, body);+ expect(anchor).toEqual({+ startMs: null,+ endMs: dayEnd("2026-05-31"),+ source: EVENT_ANCHOR_SOURCE.validity,+ });+ });+});++describe("a declared-but-unreadable rung stops the ladder", () => {+ test("an unparseable valid_from does not fall through to a lower rung", () => {+ expect(+ resolveEventAnchor(+ { valid_from: "sometime last spring", created_at: "2024-01-01" },+ "2020-01-01",+ ),+ ).toBeNull();+ });++ test("an unparseable created_at does not fall through to date or the body", () => {+ expect(+ resolveEventAnchor({ created_at: "not a date", date: "2023-01-01" }, "2020-01-01"),+ ).toBeNull();+ });++ test("an empty created_at is a declaration that cannot be read, not an absence", () => {+ expect(resolveEventAnchor({ created_at: "" }, "2020-01-01")).toBeNull();+ });+});++describe("the source vocabulary is registered", () => {+ test("every ladder rung reports a token the registry recognises", () => {+ const cases = [+ resolveEventAnchor({ valid_from: "2026-05-01" }, ""),+ resolveEventAnchor({ created_at: "2026-05-01" }, ""),+ resolveEventAnchor({ date: "2026-05-01" }, ""),+ resolveEventAnchor({}, "2026-05-01"),+ ];+ expect(cases.every((a) => a !== null)).toBe(true);+ for (const a of cases) expect(isEventAnchorSource(a!.source)).toBe(true);+ expect(new Set(cases.map((a) => a!.source)).size).toBe(4);+ });++ test("the predicate rejects free text", () => {+ expect(isEventAnchorSource("frontmatter")).toBe(false);+ expect(isEventAnchorSource("")).toBe(false);+ expect(isEventAnchorSource(undefined)).toBe(false);+ });+});diff --git a/tests/core/search/schema-migration-v10.test.ts b/tests/core/search/schema-migration-v10.test.tsindex efbf9cd5..2ca3bd8b 100644--- a/tests/core/search/schema-migration-v10.test.ts+++ b/tests/core/search/schema-migration-v10.test.ts@@ -37,8 +37,8 @@ function hasColumn(db: Database, table: string, name: string): boolean {.some((c) => c.name === name);}-test("latest schema version is 10", () => {- expect(LATEST_SCHEMA_VERSION).toBe(10);+test("latest schema version is 11", () => {+ expect(LATEST_SCHEMA_VERSION).toBe(11);});test("fresh migration reaches latest with the authored_at column", () => {diff --git a/tests/core/search/schema-migration-v11.test.ts b/tests/core/search/schema-migration-v11.test.tsnew file mode 100644index 00000000..98f698fa--- /dev/null+++ b/tests/core/search/schema-migration-v11.test.ts@@ -0,0 +1,122 @@+/**+ * Schema migration v10 -> v11: the materialised event anchor+ * (`documents.event_anchor_start_ms` / `event_anchor_end_ms` /+ * `event_anchor_source`, provenance at the boundary, t_ac1c4176).+ *+ * Additive and reindex-safe: existing rows keep every column they had and+ * default to NULL on the three new ones until a reindex repopulates them,+ * so a vault whose notes declare no dates ranks byte-identically.+ *+ * The rewind case is the one that matters. A migration that strands data+ * on disk passes every fresh-database test there is, so v11 is checked+ * against an index rewound to v10 WITH A ROW IN IT, not only against an+ * empty new one.+ */++import { test, expect, beforeEach, afterEach } from "bun:test";+import { Database } from "bun:sqlite";+import { mkdtempSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import {+ applyMigrations,+ LATEST_SCHEMA_VERSION,+ readSchemaVersion,+} from "../../../src/core/search/schema.ts";++const ANCHOR_COLUMNS = ["event_anchor_start_ms", "event_anchor_end_ms", "event_anchor_source"];++let tmp: string;+let dbPath: string;++beforeEach(() => {+ tmp = mkdtempSync(join(tmpdir(), "osb-mig-v11-"));+ dbPath = join(tmp, "test.sqlite");+});+afterEach(() => {+ rmSync(tmp, { recursive: true, force: true });+});++function hasColumn(db: Database, table: string, name: string): boolean {+ return db+ .query<{ name: string }, []>(`PRAGMA table_info(${table})`)+ .all()+ .some((c) => c.name === name);+}++test("latest schema version is 11", () => {+ expect(LATEST_SCHEMA_VERSION).toBe(11);+});++test("fresh migration reaches latest with all three event-anchor columns", () => {+ const db = new Database(dbPath);+ expect(applyMigrations(db)).toBe(LATEST_SCHEMA_VERSION);+ for (const col of ANCHOR_COLUMNS) expect(hasColumn(db, "documents", col)).toBe(true);+ db.close();+});++test("a v10 index rewound and re-migrated preserves existing rows with the new columns null", () => {+ const db = new Database(dbPath);+ applyMigrations(db); // -> latest+ db.run(+ "INSERT INTO documents(id, path, basename, title, content_hash, mtime, size, page_type, authored_at, created_at, updated_at, indexed_at) " ++ "VALUES (1, 'notes/alpha.md', 'alpha', 'A', 'h', 42, 7, 'note', 99, 't1', 't2', 't3')",+ );+ // Simulate an older v10 index: drop the three columns and rewind.+ for (const col of ANCHOR_COLUMNS) db.run(`ALTER TABLE documents DROP COLUMN ${col}`);+ db.run("UPDATE index_state SET value = '10' WHERE key = 'schema_version'");+ expect(readSchemaVersion(db)).toBe(10);++ expect(applyMigrations(db)).toBe(LATEST_SCHEMA_VERSION);+ for (const col of ANCHOR_COLUMNS) expect(hasColumn(db, "documents", col)).toBe(true);++ const row = db+ .query<+ {+ id: number;+ path: string;+ basename: string;+ title: string;+ content_hash: string;+ mtime: number;+ size: number;+ page_type: string | null;+ authored_at: number | null;+ event_anchor_start_ms: number | null;+ event_anchor_end_ms: number | null;+ event_anchor_source: string | null;+ },+ []+ >(+ "SELECT id, path, basename, title, content_hash, mtime, size, page_type, authored_at, " ++ "event_anchor_start_ms, event_anchor_end_ms, event_anchor_source FROM documents WHERE id = 1",+ )+ .get();+ expect(row).toEqual({+ id: 1,+ path: "notes/alpha.md",+ basename: "alpha",+ title: "A",+ content_hash: "h",+ mtime: 42,+ size: 7,+ page_type: "note",+ authored_at: 99,+ event_anchor_start_ms: null,+ event_anchor_end_ms: null,+ event_anchor_source: null,+ });+ db.close();+});++test("re-running the migration over an already-migrated index is a no-op", () => {+ const db = new Database(dbPath);+ applyMigrations(db);+ db.run("UPDATE index_state SET value = '10' WHERE key = 'schema_version'");+ // The PRAGMA guard makes the ALTERs idempotent, so a rewind with the+ // columns still present must not raise "duplicate column name".+ expect(() => applyMigrations(db)).not.toThrow();+ expect(readSchemaVersion(db)).toBe(LATEST_SCHEMA_VERSION);+ db.close();+});diff --git a/tests/core/search/semantic-explicit-blocked.test.ts b/tests/core/search/semantic-explicit-blocked.test.tsnew file mode 100644index 00000000..60d924c8--- /dev/null+++ b/tests/core/search/semantic-explicit-blocked.test.ts@@ -0,0 +1,129 @@+/**+ * An EXPLICIT semantic request that the CONFIGURATION cannot serve fails+ * typed; it is never answered with keyword-only results and exit 0.+ *+ * `semantic-phase.ts` says so in prose - "the explicit arm is the one that+ * ATTEMPTED something and could not, so it keeps reporting the typed+ * SearchError" - but the tier ladder it now resolves through had only the+ * `credential-missing` rung wired to a throw, so an explicit `--semantic`+ * against `embedding_provider: disabled` silently degraded. This file+ * covers BOTH values of `explicit` against BOTH blocked rungs, so neither+ * cell can regress unobserved again.+ */++import { test, expect } from "bun:test";++import {+ resolveSemanticCapability,+ semanticCapabilityLabel,+ SEMANTIC_CAPABILITY_TIER,+} from "../../../src/core/search/capability-tier.ts";+import { runSemanticPhase } from "../../../src/core/search/semantic-phase.ts";+import { SearchError } from "../../../src/core/search/types.ts";+import type { ResolvedEmbeddingConfig } from "../../../src/core/search/types.ts";+import type { Store } from "../../../src/core/search/store.ts";+import { makeConfig } from "../../helpers/search-fixtures.ts";++/**+ * A store that HAS vectors and HAS the extension, so the two runtime+ * guards ahead of the capability arm pass and what a case measures is the+ * configuration-derived arm alone.+ */+function readyStore(): Store {+ return {+ counts: () => ({ documents: 1, chunks: 1, embeddings: 1, staleEmbeddings: 0 }),+ vecLoaded: () => true,+ semanticTopK: () => [],+ } as unknown as Store;+}++/** Every rung of the ladder that BLOCKS, with the configuration that reaches it. */+const BLOCKED_CASES = [+ {+ label: "provider disabled while semantic is switched on",+ semantic: { enabled: true, provider: "disabled" } satisfies Partial<ResolvedEmbeddingConfig>,+ tier: SEMANTIC_CAPABILITY_TIER.disabled,+ code: "EMBEDDING_DISABLED",+ },+ {+ label: "semantic switched off outright",+ semantic: {+ enabled: false,+ provider: "openai-compat",+ } satisfies Partial<ResolvedEmbeddingConfig>,+ tier: SEMANTIC_CAPABILITY_TIER.disabled,+ code: "EMBEDDING_DISABLED",+ },+ {+ label: "remote provider with no credential",+ semantic: {+ enabled: true,+ provider: "openai-compat",+ apiKey: undefined,+ apiKeys: [],+ } satisfies Partial<ResolvedEmbeddingConfig>,+ tier: SEMANTIC_CAPABILITY_TIER.credentialMissing,+ code: "EMBEDDING_KEY_MISSING",+ },+] as const;++function configFor(semantic: Partial<ResolvedEmbeddingConfig>) {+ return makeConfig({+ vault: "/tmp/does-not-matter",+ dbPath: "/tmp/does-not-matter/db.sqlite",+ semantic,+ });+}++/** The registry sentence for each case, resolved once and in case order. */+function registryLabels(): Promise<string[]> {+ return Promise.all(+ BLOCKED_CASES.map((c) =>+ semanticCapabilityLabel(resolveSemanticCapability(configFor(c.semantic).semantic).code),+ ),+ );+}++test("each blocked configuration sits on the rung the case names", () => {+ const observed = BLOCKED_CASES.map(+ (c) => `${c.label}: ${resolveSemanticCapability(configFor(c.semantic).semantic).tier}`,+ );+ expect(observed).toEqual(BLOCKED_CASES.map((c) => `${c.label}: ${c.tier}`));+});++test("an explicit semantic request throws the typed error for every blocked rung", async () => {+ const labels = await registryLabels();+ const observed = await Promise.all(+ BLOCKED_CASES.map(async (c) => {+ try {+ await runSemanticPhase(readyStore(), configFor(c.semantic), "q", {+ limit: 10,+ pathPrefix: undefined,+ explicit: true,+ });+ return `${c.label}: returned keyword-only results`;+ } catch (e) {+ const err = e as SearchError;+ return `${c.label}: ${err.code} / ${err.message}`;+ }+ }),+ );+ // The sentence is the registry's, not one assembled at the throw site, so+ // the explicit and implicit arms name the same condition identically.+ expect(observed).toEqual(BLOCKED_CASES.map((c, i) => `${c.label}: ${c.code} / ${labels[i]}`));+});++test("an implicit request degrades with the registry label for every blocked rung", async () => {+ const labels = await registryLabels();+ const observed = await Promise.all(+ BLOCKED_CASES.map(async (c) => {+ const outcome = await runSemanticPhase(readyStore(), configFor(c.semantic), "q", {+ limit: 10,+ pathPrefix: undefined,+ explicit: false,+ });+ return `${c.label}: ${outcome.attempted} / ${outcome.warnings.join("|")}`;+ }),+ );+ expect(observed).toEqual(BLOCKED_CASES.map((c, i) => `${c.label}: false / ${labels[i]}`));+});diff --git a/tests/core/search/store-aliases.test.ts b/tests/core/search/store-aliases.test.tsindex b9205889..a903de41 100644--- a/tests/core/search/store-aliases.test.ts+++ b/tests/core/search/store-aliases.test.ts@@ -53,8 +53,8 @@ function targetDocId(store: Store, sourceId: number): number | null {return row?.target_document_id ?? null;}-test("schema version is 10", () => {- expect(LATEST_SCHEMA_VERSION).toBe(10);+test("schema version is 11", () => {+ expect(LATEST_SCHEMA_VERSION).toBe(11);});test("normalizeAlias lower-cases, trims, and NFC-normalises", () => {diff --git a/tests/core/surface/skill-discriminating-floor.test.ts b/tests/core/surface/skill-discriminating-floor.test.tsnew file mode 100644index 00000000..79c1bbd9--- /dev/null+++ b/tests/core/surface/skill-discriminating-floor.test.ts@@ -0,0 +1,112 @@+/**+ * Discriminating-term floor for skill attach (Unit J, t_ccb05134).+ *+ * A term carried by most of the descriptor corpus separates nothing: every+ * candidate matches it, so it is evidence for none of them. The floor drops+ * a candidate whose entire match rests on such terms. Discrimination is read+ * from corpus document frequency alone - there is no vocabulary list in the+ * derivation, which is why the same corpus shape behaves identically in a+ * language the repository has never seen.+ */++import { test, expect } from "bun:test";++import { buildSkillAttachment } from "../../../src/core/surface/skill-attach.ts";+import type { SkillEntry } from "../../../src/core/surface/skills.ts";++function entry(name: string, description: string): SkillEntry {+ return Object.freeze({+ name,+ description,+ triggers: "",+ path: `/skills/${name}`,+ skillFile: `/skills/${name}/SKILL.md`,+ shadowed: Object.freeze([]),+ });+}++/**+ * Five descriptors sharing one term. "workflow" has df 5 of 5; "quantise"+ * has df 1 of 5. Nothing distinguishes them but the counts.+ */+const CORPUS = [+ entry("alpha", "workflow helper for alpha things"),+ entry("beta", "workflow helper for beta things"),+ entry("gamma", "workflow helper for gamma things"),+ entry("delta", "workflow helper for delta things"),+ entry("epsilon", "workflow quantise tuning helper"),+];++test("a skill matched only on a corpus-common term is dropped", () => {+ const attachment = buildSkillAttachment({+ query: "workflow quantise",+ skills: CORPUS,+ maxSkills: 10,+ });+ expect(attachment.items.map((i) => i.name)).toEqual(["epsilon"]);+});++test("a query of nothing but corpus-common terms offers nothing", () => {+ const attachment = buildSkillAttachment({+ query: "workflow helper",+ skills: CORPUS,+ maxSkills: 10,+ });+ expect(attachment.items).toHaveLength(0);+ expect(attachment.block).toBe("");+ expect(attachment.offerId).toBeNull();+});++test("the floor is document frequency, not a word list: same shape, other language", () => {+ const cyrillic = [+ entry("альфа", "документ для альфа задач"),+ entry("бета", "документ для бета задач"),+ entry("гамма", "документ для гамма задач"),+ entry("дельта", "документ для дельта задач"),+ entry("эпсилон", "документ квантование настройка"),+ ];+ const attachment = buildSkillAttachment({+ query: "документ квантование",+ skills: cyrillic,+ maxSkills: 10,+ });+ expect(attachment.items.map((i) => i.name)).toEqual(["эпсилон"]);+});++test("a term in exactly half the corpus still discriminates", () => {+ const corpus = [+ entry("alpha", "shared marker alpha"),+ entry("beta", "shared marker beta"),+ entry("gamma", "gamma only"),+ entry("delta", "delta only"),+ ];+ const attachment = buildSkillAttachment({+ query: "shared",+ skills: corpus,+ maxSkills: 10,+ });+ expect(attachment.items.map((i) => i.name)).toEqual(["alpha", "beta"]);+});++test("a one-skill corpus carries no discrimination, so the floor abstains", () => {+ // With a single descriptor every term has df = 1 = the whole corpus. A+ // floor that fired here would drop the only answer on no evidence.+ const attachment = buildSkillAttachment({+ query: "configure embedding search",+ skills: [entry("embeddings-setup", "Configure embedding providers for semantic search.")],+ });+ expect(attachment.items.map((i) => i.name)).toEqual(["embeddings-setup"]);+});++test("the floor filters, it never reorders: survivors keep their ranking", () => {+ const attachment = buildSkillAttachment({+ query: "workflow quantise tuning",+ skills: [...CORPUS, entry("zeta", "workflow tuning helper")],+ maxSkills: 10,+ });+ // Both survivors match a discriminating term; the one matching two of+ // them still ranks first, and the four common-only descriptors are gone.+ expect(attachment.items.map((i) => i.name)).toEqual(["epsilon", "zeta"]);+ const scores = attachment.items.map((i) => i.score);+ expect(scores.toSorted((a, b) => b - a)).toEqual(scores);+});diff --git a/tests/core/surface/skill-offer.test.ts b/tests/core/surface/skill-offer.test.tsnew file mode 100644index 00000000..e4f76b77--- /dev/null+++ b/tests/core/surface/skill-offer.test.ts@@ -0,0 +1,155 @@+/**+ * Skill offer identity (Unit J, t_ccb05134).+ *+ * An offer is the set of skills `skills_attach` put in front of an agent+ * for one turn. Until it has an identity, a later invocation cannot be+ * joined back to it. The id is content-addressed over the offer itself,+ * so it is reproducible from the offer and needs no offer store.+ */++import { test, expect } from "bun:test";++import { buildSkillAttachment } from "../../../src/core/surface/skill-attach.ts";+import {+ computeSkillOfferId,+ isSkillOfferId,+ readSkillOfferId,+ SKILL_OFFER_ID_KEY,+ SKILL_OFFER_ID_LENGTH,+} from "../../../src/core/surface/skill-offer.ts";+import type { SkillEntry } from "../../../src/core/surface/skills.ts";++const OFFERED = [+ { name: "embeddings-setup", path: "/skills/embeddings-setup" },+ { name: "brain-memory", path: "/skills/brain-memory" },+];++function entry(name: string, description: string): SkillEntry {+ return Object.freeze({+ name,+ description,+ triggers: "",+ path: `/skills/${name}`,+ skillFile: `/skills/${name}/SKILL.md`,+ shadowed: Object.freeze([]),+ });+}++test("the same query and offered set mint the same offer id", () => {+ expect(computeSkillOfferId("set up embeddings", OFFERED)).toBe(+ computeSkillOfferId("set up embeddings", OFFERED),+ );+});++test("an offer id is lowercase hex of the declared length", () => {+ const id = computeSkillOfferId("set up embeddings", OFFERED);+ expect(id).toHaveLength(SKILL_OFFER_ID_LENGTH);+ expect(/^[0-9a-f]+$/.test(id)).toBe(true);+ expect(isSkillOfferId(id)).toBe(true);+});++test("a different offered set mints a different offer id", () => {+ const other = [OFFERED[0]!];+ expect(computeSkillOfferId("set up embeddings", other)).not.toBe(+ computeSkillOfferId("set up embeddings", OFFERED),+ );+});++test("rank order is part of the offer, so reordering mints a different id", () => {+ const reversed = OFFERED.toReversed();+ expect(computeSkillOfferId("q", reversed)).not.toBe(computeSkillOfferId("q", OFFERED));+});++test("a different query mints a different offer id", () => {+ expect(computeSkillOfferId("something else", OFFERED)).not.toBe(+ computeSkillOfferId("set up embeddings", OFFERED),+ );+});++test("query whitespace is normalised so trivial reformatting keeps the id", () => {+ expect(computeSkillOfferId(" set up embeddings\n", OFFERED)).toBe(+ computeSkillOfferId("set up embeddings", OFFERED),+ );+});++test("the same skill name at a different path is a different offer", () => {+ const moved = [{ name: OFFERED[0]!.name, path: "/elsewhere/embeddings-setup" }, OFFERED[1]!];+ expect(computeSkillOfferId("q", moved)).not.toBe(computeSkillOfferId("q", OFFERED));+});++test("isSkillOfferId rejects anything that is not a well-formed id", () => {+ expect(isSkillOfferId(undefined)).toBe(false);+ expect(isSkillOfferId("")).toBe(false);+ expect(isSkillOfferId("not-hex-at-all!!")).toBe(false);+ expect(isSkillOfferId("ABCDEF0123456789")).toBe(false);+ expect(isSkillOfferId("0123456789abcde")).toBe(false);+ expect(isSkillOfferId(1234567890123456)).toBe(false);+});++test("readSkillOfferId lifts a well-formed id from a tool input", () => {+ const id = computeSkillOfferId("set up embeddings", OFFERED);+ expect(readSkillOfferId({ name: "embeddings-setup", [SKILL_OFFER_ID_KEY]: id })).toBe(id);+});++test("readSkillOfferId reports absence and malformation alike as no cited offer", () => {+ expect(readSkillOfferId(undefined)).toBeNull();+ expect(readSkillOfferId({})).toBeNull();+ expect(readSkillOfferId({ [SKILL_OFFER_ID_KEY]: " " })).toBeNull();+ expect(readSkillOfferId({ [SKILL_OFFER_ID_KEY]: "nope" })).toBeNull();+ expect(readSkillOfferId({ [SKILL_OFFER_ID_KEY]: 42 })).toBeNull();+});++const SKILLS = [+ entry("brain-memory", "Record taste signals and apply-evidence events into the Brain."),+ entry("embeddings-setup", "Configure embedding providers for semantic search."),+ entry("schema-author", "Author Brain schema vocabulary packs."),+];++test("an attachment carries the offer id of exactly what it offered", () => {+ const attachment = buildSkillAttachment({+ query: "configure embedding providers for semantic search",+ skills: SKILLS,+ });+ expect(attachment.items.length).toBeGreaterThan(0);+ expect(attachment.offerId).toBe(+ computeSkillOfferId(+ "configure embedding providers for semantic search",+ attachment.items.map((i) => ({ name: i.name, path: i.path })),+ ),+ );+});++test("the rendered block cites the offer id so an agent can quote it back", () => {+ const attachment = buildSkillAttachment({+ query: "configure embedding providers for semantic search",+ skills: SKILLS,+ });+ expect(attachment.block).toContain(attachment.offerId!);+ expect(attachment.block).toContain(SKILL_OFFER_ID_KEY);+});++test("an empty attachment has no offer id, because nothing was offered", () => {+ const attachment = buildSkillAttachment({ query: "qqq zzz xxyyzz", skills: SKILLS });+ expect(attachment.items).toHaveLength(0);+ expect(attachment.offerId).toBeNull();+ expect(attachment.block).toBe("");+});++test("the offer id covers only the skills that survived the char budget", () => {+ const wide = buildSkillAttachment({+ query: "brain schema memory embedding search signals",+ skills: SKILLS,+ });+ const narrow = buildSkillAttachment({+ query: "brain schema memory embedding search signals",+ skills: SKILLS,+ maxSkills: 1,+ });+ expect(narrow.items).toHaveLength(1);+ expect(narrow.offerId).not.toBe(wide.offerId);+ expect(narrow.offerId).toBe(+ computeSkillOfferId("brain schema memory embedding search signals", [+ { name: narrow.items[0]!.name, path: narrow.items[0]!.path },+ ]),+ );+});diff --git a/tests/core/surface/skill-shadow.test.ts b/tests/core/surface/skill-shadow.test.tsnew file mode 100644index 00000000..b8dd6fc4--- /dev/null+++ b/tests/core/surface/skill-shadow.test.ts@@ -0,0 +1,87 @@+/**+ * Shadowed-skill provenance (Unit J, t_ccb05134, decision J3).+ *+ * Same-named skills already collapse on a name key in `discoverSkills`, so+ * duplicates never reach the ranker. The defect is the other half of that+ * collapse: the losing skill's path was dropped on the floor, so an operator+ * reading a surprising skill body had nothing telling them a second copy+ * exists. The winner now carries the paths it shadows.+ */++import { test, expect, beforeEach, afterEach } from "bun:test";+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { discoverSkills } from "../../../src/core/surface/skills.ts";++let tmp: string;++beforeEach(() => {+ tmp = mkdtempSync(join(tmpdir(), "osb-skill-shadow-"));+});++afterEach(() => {+ rmSync(tmp, { recursive: true, force: true });+});++function writeSkill(root: string, dir: string, frontmatter: string): string {+ const skillDir = join(root, dir);+ mkdirSync(skillDir, { recursive: true });+ writeFileSync(join(skillDir, "SKILL.md"), `---\n${frontmatter}\n---\n\nBody.`);+ return skillDir;+}++test("the winning skill records the path it shadowed instead of discarding it", () => {+ const repo = join(tmp, "repo-skills");+ const vault = join(tmp, "vault-skills");+ const shadowedDir = writeSkill(repo, "shared", "name: shared\ndescription: repo copy");+ const winnerDir = writeSkill(vault, "shared", "name: shared\ndescription: vault copy");++ const skills = discoverSkills([repo, vault]);+ expect(skills).toHaveLength(1);+ expect(skills[0]!.path).toBe(winnerDir);+ expect(skills[0]!.description).toBe("vault copy");+ expect(skills[0]!.shadowed).toEqual([shadowedDir]);+});++test("every shadowed path is kept, in discovery order", () => {+ const first = join(tmp, "a-skills");+ const second = join(tmp, "b-skills");+ const third = join(tmp, "c-skills");+ const firstDir = writeSkill(first, "shared", "name: shared\ndescription: first");+ const secondDir = writeSkill(second, "shared", "name: shared\ndescription: second");+ const thirdDir = writeSkill(third, "shared", "name: shared\ndescription: third");++ const skills = discoverSkills([first, second, third]);+ expect(skills[0]!.path).toBe(thirdDir);+ expect(skills[0]!.shadowed).toEqual([firstDir, secondDir]);+});++test("an unshadowed skill reports an empty list, not an absent one", () => {+ // Empty means "checked, nothing shadowed". Absent would mean the entry+ // never went through discovery, which is a different answer.+ const root = join(tmp, "skills");+ writeSkill(root, "solo", "name: solo\ndescription: only copy");+ expect(discoverSkills([root])[0]!.shadowed).toEqual([]);+});++test("shadowing keys on the skill name, not the directory name", () => {+ const first = join(tmp, "a-skills");+ const second = join(tmp, "b-skills");+ writeSkill(first, "same-dir", "name: alpha\ndescription: a");+ writeSkill(second, "same-dir", "name: beta\ndescription: b");+ const skills = discoverSkills([first, second]);+ expect(skills.map((s) => s.name)).toEqual(["alpha", "beta"]);+ for (const skill of skills) expect(skill.shadowed).toEqual([]);+});++test("a same-named skill inside one root shadows within that root too", () => {+ const root = join(tmp, "skills");+ const dupA = writeSkill(root, "aaa", "name: shared\ndescription: from aaa");+ const dupB = writeSkill(root, "bbb", "name: shared\ndescription: from bbb");+ const skills = discoverSkills([root]);+ expect(skills).toHaveLength(1);+ expect(skills[0]!.path).toBe(dupB);+ expect(skills[0]!.shadowed).toEqual([dupA]);+});diff --git a/tests/core/vault.test.ts b/tests/core/vault.test.tsindex 3bc9d929..83d3f551 100644--- a/tests/core/vault.test.ts+++ b/tests/core/vault.test.ts@@ -1,11 +1,21 @@import { afterEach, beforeEach, describe, expect, test } from "bun:test";-import { chmodSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";+import {+ chmodSync,+ existsSync,+ mkdirSync,+ mkdtempSync,+ readFileSync,+ rmSync,+ writeFileSync,+} from "node:fs";import { tmpdir } from "node:os";import { join } from "node:path";import { DEGRADATION_CODE, type DegradationNotice } from "../../src/core/integrity/degradation.ts";import {+ FrontmatterKeyError,extractWikilinks,+ formatFrontmatter,listVaultPages,parseFrontmatter,parseFrontmatterText,@@ -437,3 +447,35 @@ describe("listVaultPages", () => {}});});++describe("formatFrontmatter refuses a key the parser could not read back", () => {+ // The emitter escaped VALUES but emitted KEYS raw, so a key carrying a+ // newline emitted extra frontmatter lines. Because `update_note` merges+ // `{...existing, ...caller}` and the scanner is last-wins, that let a+ // caller override a frontmatter key it was not asked to touch. The+ // guarantee belongs at the emitter so every writer inherits it.+ for (const [label, key] of [+ ["a newline", "title\nowner"],+ ["a carriage return", "title\rowner"],+ ["a colon", "ti:tle"],+ ["a leading digit", "1title"],+ ["a space", "my title"],+ ["an empty key", ""],+ ] as const) {+ test(`${label} is refused`, () => {+ expect(() => formatFrontmatter({ [key]: "v" }, "body")).toThrow(FrontmatterKeyError);+ });+ }++ test("a conforming key still round-trips through the parser", () => {+ const text = formatFrontmatter({ title: "T", owner_name: "a-b", x2: "v" }, "body");+ const [meta] = parseFrontmatterText(text);+ expect(meta).toEqual({ title: "T", owner_name: "a-b", x2: "v" });+ });++ test("the refusal fires before any byte reaches disk", () => {+ const path = join(tmp, "note.md");+ expect(() => writeFrontmatter(path, { "a\nb: c": "v" }, "body")).toThrow(FrontmatterKeyError);+ expect(existsSync(path)).toBe(false);+ });+});diff --git a/tests/core/write-binding.test.ts b/tests/core/write-binding.test.tsnew file mode 100644index 00000000..52376702--- /dev/null+++ b/tests/core/write-binding.test.ts@@ -0,0 +1,475 @@+/**+ * The config-declared write binding over caller-named write paths+ * (provenance-at-the-boundary, unit B).+ *+ * Three properties are under test, and the third is the one that makes+ * the other two mean anything:+ *+ * 1. ABSENT IS INERT. No `write_binding:` block - and a block that+ * declares no `path_prefixes` - leaves every write path exactly as+ * it was. Proved by running the three caller-named writes against a+ * vault with no config at all and against a vault whose config+ * omits the block, and comparing both against the PRE-FEATURE bytes+ * in {@link GOLDEN_ABSENT_BINDING_BYTES}. The two runs are also+ * compared to each other, but that comparison is the weaker of the+ * two: both are the same inert path, so a change to the shared write+ * envelope shows up in both and only the golden catches it.+ * 2. DECLARED REFUSES OUTSIDE, ADMITS INSIDE, on all four caller-named+ * surfaces (`createNote`, and the update / append / batch arms that+ * share its envelope).+ * 3. THE AUTHORITY IS THE CONFIG FILE, NOT THE CALLER'S CLAIM. This+ * system has no credential: agent identity is an environment+ * variable, else a config key, else the literal `agent`, and+ * twenty-two MCP tool schemas across fifteen modules accept a+ * caller-supplied `agent` string that+ * overrides it verbatim. A fence keyed to that would be bypassed by+ * passing a different string, so the binding never reads it - and+ * that is asserted here by passing a different one and by moving+ * the environment variable underneath the same call.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import {+ mkdirSync,+ mkdtempSync,+ readFileSync,+ readdirSync,+ rmSync,+ symlinkSync,+ writeFileSync,+} from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import {+ WRITE_BINDING_REFUSED_CODE,+ checkWriteBinding,+ normaliseWriteBindingPrefix,+ resolveWriteBinding,+ writeBindingAdmits,+ writeBindingPrefixCovers,+} from "../../src/core/write-binding/index.ts";+import { resolveNextStep } from "../../src/core/brain/next-step.ts";+import { CreateNoteError, createNote } from "../../src/core/brain/notes/create-note.ts";+import { WriteBatchError, applyWriteBatch } from "../../src/core/brain/write-batch.ts";+import { BrainConfigError } from "../../src/core/brain/policy.ts";+import { validateBrainConfig } from "../../src/core/brain/policy.ts";+import { parseBrainYaml } from "../../src/core/brain/yaml-parse.ts";+import { runCli } from "../helpers/run-cli.ts";++/** A caller-named note path that no declared prefix in these tests admits. */+const OUTSIDE_PATH = "Elsewhere/Note.md";+/** A caller-named note path under the prefix the tests declare. */+const INSIDE_PATH = "Projects/Note.md";+/** The single prefix the declared-binding fixtures use. */+const PREFIX = "Projects";++/**+ * The exact bytes the three caller-named write arms left behind BEFORE+ * this unit existed, captured by running the same sequence against the+ * store as it stood at HEAD~ of the binding.+ *+ * Pinned as literals on purpose. Comparing one absent-binding run against+ * another absent-binding run - which is what "no config file" versus "a+ * config without the block" are, both of them the same inert path - can+ * only show the two agree; a change to the shared write envelope every+ * caller-named write funnels through moves BOTH arms identically and the+ * comparison stays green. Only a fixed pre-feature reference can see it.+ */+const GOLDEN_ABSENT_BINDING_BYTES: Readonly<Record<string, string>> = Object.freeze({+ /** `createNote`, then the batch `update_note` and `append_note` arms. */+ outside: "---\ntitle: T\ntag: x\n---\n\nbody\n\nmore\n",+ /** The batch `create_note` arm. */+ inside: "---\n---\n\ninside\n",+});++const AGENT_ENV = "VAULT_AGENT_NAME";++let vault: string;++function writeBrainConfig(root: string, yaml: string): void {+ mkdirSync(join(root, "Brain"), { recursive: true });+ writeFileSync(join(root, "Brain", "_brain.yaml"), yaml, "utf8");+}++/** `_brain.yaml` declaring exactly one write-binding prefix. */+function declaredBindingYaml(...prefixes: ReadonlyArray<string>): string {+ return `schema_version: 1\nwrite_binding:\n path_prefixes:\n${prefixes+ .map((p) => ` - ${p}\n`)+ .join("")}`;+}++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-write-binding-"));+});+afterEach(() => rmSync(vault, { recursive: true, force: true }));++describe("the binding is absent unless the operator declares it", () => {+ test("a vault with no Brain config has no binding", () => {+ expect(resolveWriteBinding(vault)).toBeNull();+ expect(checkWriteBinding(vault, OUTSIDE_PATH)).toBeNull();+ });++ test("a config without the block has no binding", () => {+ writeBrainConfig(vault, "schema_version: 1\n");+ expect(resolveWriteBinding(vault)).toBeNull();+ expect(checkWriteBinding(vault, OUTSIDE_PATH)).toBeNull();+ });++ test("a block that declares no path_prefixes has no binding", () => {+ // Present-but-empty BLOCK is not a declaration of anything: the key+ // the binding is made of is absent, exactly as `vault:` treats a+ // missing `ignore_paths`.+ writeBrainConfig(vault, "schema_version: 1\nwrite_binding:\n other_key: 1\n");+ expect(resolveWriteBinding(vault)).toBeNull();+ expect(checkWriteBinding(vault, OUTSIDE_PATH)).toBeNull();+ });++ test("an empty prefix list is refused at load rather than silently admitting nothing", () => {+ // A list that admits no path is an off switch wearing a binding's+ // clothes; the operator turns the binding off by removing the block.+ expect(() =>+ validateBrainConfig(+ parseBrainYaml("schema_version: 1\nwrite_binding:\n path_prefixes: []\n"),+ "<test>",+ ),+ ).toThrow(BrainConfigError);+ });+});++describe("absent binding is byte-identical on every caller-named write path", () => {+ /** Run the three caller-named write arms and return what landed. */+ function runAllWritePaths(root: string): Record<string, string> {+ createNote(root, { path: OUTSIDE_PATH, frontmatter: { title: "T" }, content: "body" });+ applyWriteBatch(root, [+ { kind: "update_note", path: OUTSIDE_PATH, frontmatter: { tag: "x" } },+ { kind: "create_note", path: INSIDE_PATH, content: "inside" },+ ]);+ applyWriteBatch(root, [{ kind: "append_note", path: OUTSIDE_PATH, content: "more" }]);+ return {+ outside: readFileSync(join(root, OUTSIDE_PATH), "utf8"),+ inside: readFileSync(join(root, INSIDE_PATH), "utf8"),+ };+ }++ test("a vault with no config at all writes the pre-feature bytes", () => {+ expect(runAllWritePaths(vault)).toEqual({ ...GOLDEN_ABSENT_BINDING_BYTES });+ });++ test("a config without the block writes the same pre-feature bytes", () => {+ const withConfig = mkdtempSync(join(tmpdir(), "o2b-write-binding-cfg-"));+ try {+ writeBrainConfig(withConfig, "schema_version: 1\n");+ const written = runAllWritePaths(withConfig);+ expect(written).toEqual({ ...GOLDEN_ABSENT_BINDING_BYTES });+ // And the two inert arms still agree with each other, which is the+ // property the golden alone does not state.+ expect(written).toEqual(runAllWritePaths(vault));+ } finally {+ rmSync(withConfig, { recursive: true, force: true });+ }+ });+});++describe("a declared binding is rooted and segment-wise", () => {+ test("the prefix admits itself and its descendants", () => {+ const binding = { pathPrefixes: [PREFIX] };+ expect(writeBindingAdmits(binding, "Projects")).toBe(true);+ expect(writeBindingAdmits(binding, "Projects/Note.md")).toBe(true);+ expect(writeBindingAdmits(binding, "Projects/deep/nested/Note.md")).toBe(true);+ });++ test("a longer sibling name is not a descendant", () => {+ // Character-prefix matching would admit `ProjectsArchive/`; segment+ // matching is what makes the declaration mean the folder.+ expect(writeBindingAdmits({ pathPrefixes: [PREFIX] }, "ProjectsArchive/Note.md")).toBe(false);+ });++ test("a path under no declared prefix is refused", () => {+ expect(writeBindingAdmits({ pathPrefixes: [PREFIX] }, OUTSIDE_PATH)).toBe(false);+ });++ test("any one of several prefixes admits", () => {+ const binding = { pathPrefixes: ["Projects", "Journal/Weekly"] };+ expect(writeBindingAdmits(binding, "Journal/Weekly/W1.md")).toBe(true);+ expect(writeBindingAdmits(binding, "Journal/Daily/D1.md")).toBe(false);+ });+});++describe("a declared binding refuses caller-named writes outside it", () => {+ beforeEach(() => writeBrainConfig(vault, declaredBindingYaml(PREFIX)));++ test("createNote refuses outside and admits inside", () => {+ expect(() => createNote(vault, { path: OUTSIDE_PATH, content: "x" })).toThrow(CreateNoteError);+ const res = createNote(vault, { path: INSIDE_PATH, content: "x" });+ expect(res.created).toBe(true);+ });++ test("the refusal carries the registered code and names the declared prefixes", () => {+ let caught: CreateNoteError | null = null;+ try {+ createNote(vault, { path: OUTSIDE_PATH, content: "x" });+ } catch (err) {+ caught = err as CreateNoteError;+ }+ expect(caught).not.toBeNull();+ expect(caught!.code).toBe("write_binding");+ expect(caught!.message).toContain(PREFIX);+ expect(caught!.message).toContain(OUTSIDE_PATH);+ });++ test("the refusal names the exit its registered code resolves to", () => {+ const step = resolveNextStep(WRITE_BINDING_REFUSED_CODE);+ expect(step).not.toBeNull();+ const refusal = checkWriteBinding(vault, OUTSIDE_PATH);+ expect(refusal).not.toBeNull();+ expect(refusal!.code).toBe(WRITE_BINDING_REFUSED_CODE);+ expect(refusal!.nextCommand).toBe(step!.nextCommand);+ expect(refusal!.message).toContain(step!.nextCommand);+ });++ test("the batch update and append arms are refused before any write", () => {+ createNote(vault, { path: INSIDE_PATH, content: "x" });+ // Seed a target outside the binding by hand, so the refusal cannot be+ // mistaken for a missing-target error.+ mkdirSync(join(vault, "Elsewhere"), { recursive: true });+ writeFileSync(join(vault, OUTSIDE_PATH), "---\n---\n\nseed\n", "utf8");++ for (const op of [+ { kind: "update_note", path: OUTSIDE_PATH, body: "replaced" },+ { kind: "append_note", path: OUTSIDE_PATH, content: "appended" },+ { kind: "create_note", path: "Elsewhere/Fresh.md", content: "x" },+ ] as const) {+ let caught: WriteBatchError | null = null;+ try {+ applyWriteBatch(vault, [op]);+ } catch (err) {+ caught = err as WriteBatchError;+ }+ expect(caught).not.toBeNull();+ expect(caught!.code).toBe("write_binding");+ }+ // Nothing landed: the seeded file is untouched and no new file exists.+ expect(readFileSync(join(vault, OUTSIDE_PATH), "utf8")).toBe("---\n---\n\nseed\n");+ });++ test("a batch aborts entirely when one operation lands outside the binding", () => {+ expect(() =>+ applyWriteBatch(vault, [+ { kind: "create_note", path: INSIDE_PATH, content: "inside" },+ { kind: "create_note", path: OUTSIDE_PATH, content: "outside" },+ ]),+ ).toThrow(WriteBatchError);+ // Validate-all-then-commit: the admitted operation must not have run.+ expect(() => readFileSync(join(vault, INSIDE_PATH), "utf8")).toThrow();+ });+});++describe("the caller's claimed identity cannot widen the binding", () => {+ beforeEach(() => writeBrainConfig(vault, declaredBindingYaml(PREFIX)));++ test("a caller-supplied agent string on the same batch does not widen it", () => {+ // `brain_write_batch` accepts a caller-supplied `agent` on its+ // evidence and log-line operations, and it is taken verbatim. The+ // binding never reads it, so an operation outside the binding is+ // refused with the same code whichever name rides along.+ const codes = ["agent", "someone-else", ""].map((agent) => {+ try {+ applyWriteBatch(vault, [+ { kind: "append_log_line", input: { text: "note", agent } },+ { kind: "create_note", path: OUTSIDE_PATH, content: "x" },+ ]);+ return "not-refused";+ } catch (err) {+ return (err as WriteBatchError).code;+ }+ });+ expect(codes).toEqual(["write_binding", "write_binding", "write_binding"]);+ });++ test("moving the identity environment variable does not widen it", () => {+ const before = process.env[AGENT_ENV];+ try {+ for (const name of ["agent", "operator", "root"]) {+ process.env[AGENT_ENV] = name;+ const refusal = checkWriteBinding(vault, OUTSIDE_PATH);+ expect(refusal?.code).toBe(WRITE_BINDING_REFUSED_CODE);+ }+ } finally {+ if (before === undefined) delete process.env[AGENT_ENV];+ else process.env[AGENT_ENV] = before;+ }+ });+});++describe("the registered exit answers the question the refusal raises", () => {+ // The rail forbids inventing a command, and a command that does not+ // report the binding would be an invented one in all but name. So the+ // exit this code resolves to is driven here, end to end, against a+ // path the binding refuses.+ test("`o2b vault inspect` reports the binding verdict and the declared prefixes", async () => {+ writeBrainConfig(vault, declaredBindingYaml(PREFIX));+ const configPath = join(vault, "cli-config.yaml");+ const run = await runCli(["vault", "inspect", OUTSIDE_PATH, "--vault", vault, "--json"], {+ env: { OPEN_SECOND_BRAIN_CONFIG: configPath },+ });+ expect(run.returncode).toBe(0);+ const payload = JSON.parse(run.stdout) as {+ write_binding: { declared: boolean; admits: boolean; path_prefixes: string[] };+ };+ expect(payload.write_binding).toEqual({+ declared: true,+ admits: false,+ path_prefixes: [PREFIX],+ });+ });++ test("an undeclared binding reads as undeclared, not as admitted", () => {+ // The third state. Reporting `admits: true` for a vault that never+ // declared a binding would tell an operator a boundary is holding+ // when none exists.+ expect(resolveWriteBinding(vault)).toBeNull();+ });+});++describe("prefix declarations are validated at load", () => {+ for (const [label, raw] of [+ ["a leading slash", "/Projects"],+ ["a traversal segment", "Projects/../etc"],+ ["an empty entry", '""'],+ ] as const) {+ test(`${label} is refused`, () => {+ expect(() =>+ validateBrainConfig(+ parseBrainYaml(`schema_version: 1\nwrite_binding:\n path_prefixes:\n - ${raw}\n`),+ "<test>",+ ),+ ).toThrow(BrainConfigError);+ });+ }++ test("a declared prefix normalises to its POSIX vault-relative form", () => {+ writeBrainConfig(vault, declaredBindingYaml("./Projects/"));+ expect(resolveWriteBinding(vault)?.pathPrefixes).toEqual(["Projects"]);+ });+});++describe("the binding is checked against where the bytes land, not the name given", () => {+ // Defect 1 of the v1.43.0 security review. `checkWriteBinding` used to+ // compare the LEXICAL relative path, so one symlink under a declared+ // prefix turned the binding into "may write anywhere in the vault" and+ // the success payload named a path that did not hold the bytes.+ const LINK_PATH = "Projects/link/escaped.md";++ /** `<vault>/Projects/link` -> `<vault>/Journal`, both inside the vault. */+ function linkProjectsToJournal(root: string): void {+ mkdirSync(join(root, "Projects"), { recursive: true });+ mkdirSync(join(root, "Journal"), { recursive: true });+ symlinkSync(join(root, "Journal"), join(root, "Projects", "link"));+ }++ test("a symlinked directory under a declared prefix does not widen it", () => {+ writeBrainConfig(vault, declaredBindingYaml(PREFIX));+ linkProjectsToJournal(vault);+ let caught: CreateNoteError | null = null;+ try {+ createNote(vault, { path: LINK_PATH, content: "x" });+ } catch (err) {+ caught = err as CreateNoteError;+ }+ expect(caught?.code).toBe("write_binding");+ expect(readdirSync(join(vault, "Journal"))).toEqual([]);+ });++ test("the refusal names the destination the bytes would have reached", () => {+ writeBrainConfig(vault, declaredBindingYaml(PREFIX));+ linkProjectsToJournal(vault);+ const refusal = checkWriteBinding(vault, LINK_PATH);+ expect(refusal).not.toBeNull();+ expect(refusal!.resolvedRelPath).toBe("Journal/escaped.md");+ expect(refusal!.message).toContain("Journal/escaped.md");+ expect(refusal!.message).toContain(LINK_PATH);+ });++ test("a symlink that stays inside the declared prefix is still admitted", () => {+ writeBrainConfig(vault, declaredBindingYaml(PREFIX));+ mkdirSync(join(vault, "Projects", "real"), { recursive: true });+ symlinkSync(join(vault, "Projects", "real"), join(vault, "Projects", "alias"));+ expect(checkWriteBinding(vault, "Projects/alias/Note.md")).toBeNull();+ });++ test("with no binding declared the same symlink write is untouched", () => {+ // Absent stays byte-identical: the realpath comparison is reached+ // only once a binding exists, so an unbound vault behaves exactly as+ // it did before the key existed.+ linkProjectsToJournal(vault);+ const res = createNote(vault, { path: LINK_PATH, content: "x" });+ expect(res.path).toBe(LINK_PATH);+ expect(readdirSync(join(vault, "Journal"))).toEqual(["escaped.md"]);+ });+});++describe("a backslash is a filename character on POSIX, not a separator", () => {+ // Defect 2 of the v1.43.0 security review. The matcher split on+ // `[\\/]`, so a binding on `Projects` admitted `Projects\evil.md` -+ // which on Linux is one filename that lands at the VAULT ROOT.+ test("the matcher does not read a backslash as a segment boundary", () => {+ expect(writeBindingPrefixCovers(PREFIX, "Projects\\evil.md")).toBe(false);+ expect(writeBindingPrefixCovers(PREFIX, "Projects/evil.md")).toBe(true);+ });++ test("a declared prefix keeps its POSIX segments", () => {+ expect(normaliseWriteBindingPrefix("./Projects//Sub/")).toBe("Projects/Sub");+ });+});++describe("a config that does not load is a typed refusal, not a server fault", () => {+ // Defect 3 of the v1.43.0 security review. A `_brain.yaml` that fails+ // validation made every caller-named write throw a raw BrainConfigError+ // carrying an ABSOLUTE host path, with no registered next step.+ const MALFORMED = "schema_version: 99\n";++ test("createNote refuses with a typed code, a vault-relative source, and a registered exit", () => {+ writeBrainConfig(vault, MALFORMED);+ let caught: CreateNoteError | null = null;+ try {+ createNote(vault, { path: INSIDE_PATH, content: "x" });+ } catch (err) {+ caught = err as CreateNoteError;+ }+ expect(caught).toBeInstanceOf(CreateNoteError);+ expect(caught!.code).toBe("config_invalid");+ expect(caught!.message).toContain("Brain/_brain.yaml");+ expect(caught!.message).not.toContain(vault);+ const step = resolveNextStep("config-invalid");+ expect(step).not.toBeNull();+ expect(caught!.message).toContain(step!.nextCommand);+ });++ test("the batch arms carry the same typed code", () => {+ writeBrainConfig(vault, MALFORMED);+ let caught: WriteBatchError | null = null;+ try {+ applyWriteBatch(vault, [{ kind: "create_note", path: INSIDE_PATH, content: "x" }]);+ } catch (err) {+ caught = err as WriteBatchError;+ }+ expect(caught).toBeInstanceOf(WriteBatchError);+ expect(caught!.code).toBe("config_invalid");+ expect(caught!.message).not.toContain(vault);+ });+});++describe("a re-declared binding takes effect on the next call", () => {+ test("rewriting the prefix list changes what is admitted", () => {+ // Guards the per-vault memo that keeps a batch from re-reading and+ // re-validating the whole config once per operation: the cached+ // answer must never outlive the file it was read from.+ writeBrainConfig(vault, declaredBindingYaml(PREFIX));+ expect(checkWriteBinding(vault, INSIDE_PATH)).toBeNull();+ expect(checkWriteBinding(vault, OUTSIDE_PATH)).not.toBeNull();+ writeBrainConfig(vault, declaredBindingYaml("Elsewhere"));+ expect(checkWriteBinding(vault, OUTSIDE_PATH)).toBeNull();+ expect(checkWriteBinding(vault, INSIDE_PATH)).not.toBeNull();+ });+});diff --git a/tests/mcp/brain-context-pack-outcome-agent.test.ts b/tests/mcp/brain-context-pack-outcome-agent.test.tsnew file mode 100644index 00000000..55e799c5--- /dev/null+++ b/tests/mcp/brain-context-pack-outcome-agent.test.ts@@ -0,0 +1,196 @@+/**+ * The acting agent on a `brain_context_pack_outcome` post.+ *+ * `context-pack-outcome.ts` spreads an `agentId` onto three rows - the+ * outcome row, the `token_impact_outcome` calibration row and the+ * `context_pack_evidence` row, which calls it "the ACTING agent recorded+ * as the claimant". Until this file existed the only caller,+ * `toolBrainContextPackOutcome`, never supplied one and the tool schema+ * had no property for it, so the claimant key could not be written by any+ * shipped path. The exercise here runs the REAL surface - a JSON-RPC+ * `tools/call` against a live `MCPServer` - rather than the core function,+ * because a test that calls the core function directly is exactly what let+ * that gap survive.+ *+ * Three properties:+ *+ * - supplied, the identity reaches all three rows. One argument, three+ * joined records, so the outcome row and its two side records can+ * never disagree about who posted.+ * - omitted, the post is byte-identical to one made before the property+ * existed. Nothing is invented and nothing is guessed.+ * - a config that cannot be READ does not fail the post. The identity is+ * an argument, never `ServerContext.agentName`, which resolves the+ * config on access and RAISES on one it cannot read. Reading it here+ * would turn a working telemetry post into a failure over a broken+ * line in an unrelated file; the third test forbids that by standing a+ * directory where the config belongs.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { resolveAgentName } from "../../src/core/config.ts";+import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";+import { listContextPackEvidence } from "../../src/core/brain/context-pack-evidence.ts";+import { emitContextReceipt } from "../../src/core/brain/context-receipts.ts";+import { listTokenImpactOutcomes } from "../../src/core/brain/token-impact.ts";+import { CONTINUITY_AGENT_ID_KEY } from "../../src/core/brain/continuity/types.ts";+import { JSONRPC_VERSION, MCPServer, PROTOCOL_VERSION } from "../../src/mcp/index.ts";++/** The identity a client asserts. Self-asserted, like every identity here. */+const ACTING_AGENT = "claude-dev-agent";++let tmp: string;+let vault: string;+let configHome: string;+let configPath: string;+const savedEnv: Record<string, string | undefined> = {};++const ENV_KEYS: ReadonlyArray<string> = Object.freeze([+ "VAULT_AGENT_NAME",+ "VAULT_TIMEZONE",+ "VAULT_DIR",+ "OPEN_SECOND_BRAIN_CONFIG",+ "OPEN_SECOND_BRAIN_CONTEXT_PACK_OUTCOME_ENABLED",+ "OPEN_SECOND_BRAIN_TOKEN_IMPACT_LEDGER_ENABLED",+]);++function writeConfig(): void {+ atomicWriteFileSync(configPath, `vault: ${vault}\ncontext_pack_outcome_enabled: "true"\n`);+}++function seedReceipt(): string {+ return emitContextReceipt(vault, {+ options: { host: "test", trigger: "context_pack", createdAt: "2026-07-01T00:00:00.000Z" },+ items: [{ id: "pref-a", path: "Brain/preferences/pref-a.md", text: "preference one" }],+ finalText: "preference one\n",+ }).id;+}++beforeEach(() => {+ tmp = mkdtempSync(join(tmpdir(), "o2b-ctx-outcome-agent-"));+ vault = join(tmp, "vault");+ mkdirSync(join(vault, "Brain"), { recursive: true });+ configHome = mkdtempSync(join(tmpdir(), "o2b-ctx-outcome-agent-cfg-"));+ configPath = join(configHome, "config.yaml");+ for (const key of ENV_KEYS) {+ savedEnv[key] = process.env[key];+ delete process.env[key];+ }+ process.env["OPEN_SECOND_BRAIN_CONFIG"] = configPath;+});++afterEach(() => {+ rmSync(tmp, { recursive: true, force: true });+ rmSync(configHome, { recursive: true, force: true });+ for (const [key, value] of Object.entries(savedEnv)) {+ if (value === undefined) delete process.env[key];+ else process.env[key] = value;+ }+});++async function startServer(): Promise<MCPServer> {+ const server = new MCPServer({ vault, configPath });+ await server.handleRequest({+ jsonrpc: JSONRPC_VERSION,+ id: 1,+ method: "initialize",+ params: {+ protocolVersion: PROTOCOL_VERSION,+ capabilities: {},+ clientInfo: { name: "ctx-outcome-agent-test", version: "0" },+ },+ });+ return server;+}++async function callTool(+ server: MCPServer,+ args: Record<string, unknown>,+): Promise<Record<string, unknown>> {+ const response = (await server.handleRequest({+ jsonrpc: JSONRPC_VERSION,+ id: 9,+ method: "tools/call",+ params: { name: "brain_context_pack_outcome", arguments: args },+ })) as { result: { content: ReadonlyArray<{ type: string; text: string }> } };+ return JSON.parse(response.result.content[0]!.text);+}++describe("brain_context_pack_outcome acting agent", () => {+ test("an agent_id supplied at the tool boundary reaches all three joined rows", async () => {+ writeConfig();+ const sampleId = seedReceipt();+ const server = await startServer();++ const out = await callTool(server, {+ operation: "post",+ sample_id: sampleId,+ first_pass_success: true,+ agent_id: ACTING_AGENT,+ });+ expect(out["recorded"]).toBe(true);++ // 1. the outcome row itself+ const list = await callTool(server, { operation: "list" });+ const rows = list["records"] as ReadonlyArray<{ payload: Record<string, unknown> }>;+ expect(rows).toHaveLength(1);+ expect(rows[0]!.payload[CONTINUITY_AGENT_ID_KEY]).toBe(ACTING_AGENT);++ // 2. the evidence row - where the identity is the CLAIMANT+ const evidence = listContextPackEvidence(vault);+ expect(evidence).toHaveLength(1);+ expect(evidence[0]!.payload[CONTINUITY_AGENT_ID_KEY]).toBe(ACTING_AGENT);++ // 3. the token-impact calibration row composed off the same post+ const calibration = listTokenImpactOutcomes(vault);+ expect(calibration).toHaveLength(1);+ expect(calibration[0]!.payload[CONTINUITY_AGENT_ID_KEY]).toBe(ACTING_AGENT);+ });++ test("a post with no agent_id is byte-identical to one made before the property existed", async () => {+ writeConfig();+ const sampleId = seedReceipt();+ const server = await startServer();++ await callTool(server, { operation: "post", sample_id: sampleId, first_pass_success: true });++ const list = await callTool(server, { operation: "list" });+ const rows = list["records"] as ReadonlyArray<{ payload: Record<string, unknown> }>;+ expect(Object.keys(rows[0]!.payload).toSorted()).toEqual(["first_pass_success", "sample_id"]);+ // Nothing is guessed on the joined rows either - no server identity+ // leaks in through the back door when the client asserted none.+ expect(CONTINUITY_AGENT_ID_KEY in listContextPackEvidence(vault)[0]!.payload).toBe(false);+ expect(CONTINUITY_AGENT_ID_KEY in listTokenImpactOutcomes(vault)[0]!.payload).toBe(false);+ });++ test("a config that cannot be read does not fail the post", async () => {+ // A directory where the config file belongs: present, and unreadable+ // as a config. The gate comes from the environment, which short-+ // circuits ahead of the config read, so the ONLY thing that could+ // touch that file on this path is an identity lookup.+ mkdirSync(configPath, { recursive: true });+ process.env["OPEN_SECOND_BRAIN_CONTEXT_PACK_OUTCOME_ENABLED"] = "true";++ // Non-vacuity: the config really is unreadable, and the server-side+ // identity really would raise on it. Without this the test would pass+ // just as well against a config that was merely absent.+ expect(() => resolveAgentName(configPath)).toThrow();++ const server = await startServer();+ const out = await callTool(server, {+ operation: "post",+ sample_id: "opaque_hash",+ first_pass_success: true,+ agent_id: ACTING_AGENT,+ });++ expect(out["recorded"]).toBe(true);+ const list = await callTool(server, { operation: "list" });+ const rows = list["records"] as ReadonlyArray<{ payload: Record<string, unknown> }>;+ expect(rows[0]!.payload[CONTINUITY_AGENT_ID_KEY]).toBe(ACTING_AGENT);+ });+});diff --git a/tests/mcp/brain-create-note.test.ts b/tests/mcp/brain-create-note.test.tsindex a7203bf9..e8141b43 100644--- a/tests/mcp/brain-create-note.test.ts+++ b/tests/mcp/brain-create-note.test.ts@@ -14,6 +14,9 @@ import { tmpdir } from "node:os";import { bootstrapBrain } from "../../src/core/brain/init.ts";import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";import { NOTES_TOOLS } from "../../src/mcp/brain/notes-tools.ts";+import { WRITE_BATCH_TOOLS } from "../../src/mcp/brain/write-batch-tools.ts";+import { resolveNextStep } from "../../src/core/brain/next-step.ts";+import { WRITE_BINDING_REFUSED_CODE } from "../../src/core/write-binding/index.ts";import { MCPError } from "../../src/mcp/protocol.ts";import type { ServerContext } from "../../src/mcp/tool-contract.ts";@@ -90,3 +93,143 @@ describe("brain_create_note", () => {expect(readFileSync(join(vault, "Notes/Once.md"), "utf8")).toContain("first");});});++/**+ * Authoring modes added by provenance-at-the-boundary, Unit C:+ * `if_exists`, `strict`, and template-mode bodies. The pre-existing+ * cases above stay untouched and green - a refusal is still the+ * default, and the three new arguments are inert when absent.+ */+describe("brain_create_note - authoring modes", () => {+ test("advertises the new arguments without changing the required set", () => {+ const props = tool.inputSchema.properties as Record<string, { type?: unknown }>;+ expect(Object.keys(props)).toEqual(+ expect.arrayContaining(["if_exists", "strict", "template", "template_variables"]),+ );+ expect(tool.inputSchema.required).toEqual(["path"]);+ expect(tool.inputSchema.additionalProperties).toBe(false);+ });++ test("with the new arguments absent, a note with no frontmatter is still written", async () => {+ const res = await handler(ctx, { path: "Notes/Absent.md", content: "body only" });+ expect(res).toMatchObject({ created: true, outcome: "created", path: "Notes/Absent.md" });+ });++ test('if_exists "skip" returns a skipped outcome a caller cannot read as a create', async () => {+ await handler(ctx, { path: "Notes/Idem.md", content: "first" });+ const res = await handler(ctx, {+ path: "Notes/Idem.md",+ content: "second",+ if_exists: "skip",+ });+ expect(res).toMatchObject({ created: false, outcome: "skipped", path: "Notes/Idem.md" });+ expect(readFileSync(join(vault, "Notes/Idem.md"), "utf8")).toContain("first");+ });++ test("an unknown if_exists value is rejected rather than defaulted", async () => {+ await expect(handler(ctx, { path: "Notes/Bad.md", if_exists: "overwrite" })).rejects.toThrow(+ MCPError,+ );+ expect(existsSync(join(vault, "Notes/Bad.md"))).toBe(false);+ });++ test("strict reports the validator's coded violations and writes nothing", async () => {+ try {+ await handler(ctx, { path: "Notes/Strict.md", content: "body only", strict: true });+ throw new Error("expected throw");+ } catch (err) {+ expect(err).toBeInstanceOf(MCPError);+ const data = (err as MCPError).data as { violations?: ReadonlyArray<{ code: string }> };+ expect(data.violations?.map((v) => v.code)).toContain("frontmatter-missing");+ }+ expect(existsSync(join(vault, "Notes/Strict.md"))).toBe(false);+ });++ test("template mode renders variables, a section, an iteration, and keeps a typo intact", async () => {+ const res = await handler(ctx, {+ path: "Notes/Tpl.md",+ frontmatter: { title: "T" },+ template: "# {{title}}\n{{#draft}}(draft){{/draft}}\n{{#tags}}- {{.}}\n{{/tags}}{{ typo }}",+ template_variables: { title: "T", draft: true, tags: ["a", "b"] },+ });+ expect(res).toMatchObject({ created: true, outcome: "created" });+ const md = readFileSync(join(vault, "Notes/Tpl.md"), "utf8");+ expect(md).toContain("# T");+ expect(md).toContain("(draft)");+ expect(md).toContain("- a\n- b\n");+ expect(md).toContain("{{ typo }}");+ });++ test("a malformed template is an INVALID_PARAMS refusal, not a half-written note", async () => {+ await expect(handler(ctx, { path: "Notes/Broken.md", template: "{{#a}}x" })).rejects.toThrow(+ MCPError,+ );+ expect(existsSync(join(vault, "Notes/Broken.md"))).toBe(false);+ });++ test("template_variables values are held to the frontmatter value domain", async () => {+ await expect(+ handler(ctx, { path: "Notes/BadVars.md", template: "{{a}}", template_variables: { a: {} } }),+ ).rejects.toThrow(MCPError);+ expect(existsSync(join(vault, "Notes/BadVars.md"))).toBe(false);+ });+});++describe("the write surface names the write binding it enforces", () => {+ // Every tool that traverses the shared create-note envelope spells out+ // its refusals; a caller reads that list as complete, so the binding+ // has to appear in it.+ const BINDING_TERM = "write binding";+ for (const t of [...NOTES_TOOLS, ...WRITE_BATCH_TOOLS]) {+ test(`${t.name} lists the binding among its refusals`, () => {+ expect(t.description.toLowerCase()).toContain(BINDING_TERM);+ });+ }+});++describe("a refused write can be traced back to a registered code", () => {+ const MALFORMED = "schema_version: 99\n";++ test("a binding refusal carries the advisory code and its exit", async () => {+ atomicWriteFileSync(+ join(vault, "Brain", "_brain.yaml"),+ "schema_version: 1\nwrite_binding:\n path_prefixes:\n - Projects\n",+ );+ let caught: MCPError | null = null;+ try {+ await handler(ctx, { path: "Elsewhere/N.md", content: "x" });+ } catch (err) {+ caught = err as MCPError;+ }+ expect(caught).toBeInstanceOf(MCPError);+ const data = caught!.data as Record<string, unknown>;+ expect(data["code"]).toBe("write_binding");+ expect(data["diagnostic_code"]).toBe(WRITE_BINDING_REFUSED_CODE);+ expect(data["next_command"]).toBe(resolveNextStep(WRITE_BINDING_REFUSED_CODE)!.nextCommand);+ });++ test("a malformed config is a typed refusal with a vault-relative source", async () => {+ atomicWriteFileSync(join(vault, "Brain", "_brain.yaml"), MALFORMED);+ let caught: MCPError | null = null;+ try {+ await handler(ctx, { path: "Notes/N.md", content: "x" });+ } catch (err) {+ caught = err as MCPError;+ }+ expect(caught).toBeInstanceOf(MCPError);+ expect(caught!.message).toContain("Brain/_brain.yaml");+ expect(caught!.message).not.toContain(vault);+ const data = caught!.data as Record<string, unknown>;+ expect(data["code"]).toBe("config_invalid");+ expect(data["next_command"]).toBe(resolveNextStep("config-invalid")!.nextCommand);+ });+});++describe("a frontmatter key must be a key the parser can read back", () => {+ test("a newline-bearing key is refused before anything is written", async () => {+ await expect(+ handler(ctx, { path: "Notes/Inject.md", frontmatter: { "a\nowner: mallory": "v" } }),+ ).rejects.toThrow(MCPError);+ expect(existsSync(join(vault, "Notes/Inject.md"))).toBe(false);+ });+});diff --git a/tests/mcp/context-pack-evidence-tool.test.ts b/tests/mcp/context-pack-evidence-tool.test.tsnew file mode 100644index 00000000..38a15294--- /dev/null+++ b/tests/mcp/context-pack-evidence-tool.test.ts@@ -0,0 +1,260 @@+import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, readFileSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";+import { emitContextReceipt } from "../../src/core/brain/context-receipts.ts";+import {+ CONTEXT_PACK_EVIDENCE_VERDICT,+ listContextPackEvidence,+} from "../../src/core/brain/context-pack-evidence.ts";+import { continuityLogPath } from "../../src/core/brain/continuity/store.ts";+import { JSONRPC_VERSION, MCPServer, PROTOCOL_VERSION } from "../../src/mcp/index.ts";++let tmp: string;+let vault: string;+let configHome: string;+let configPath: string;+const savedEnv: Record<string, string | undefined> = {};++const PACKED_TEXT = "preference one\npreference two\n";++/** The opaque sample id the claimless byte-identity fixture posts on. */+const CLAIMLESS_SAMPLE_ID = "opaque_hash";++/** Length of the `YYYY-MM` month key a continuity shard is named for. */+const MONTH_KEY_LENGTH = 7;++/**+ * The serialized `payload` of a claimless `post`, exactly as it stood+ * before `evidence_claim` existed. `JSON.stringify` emits insertion+ * order, so this literal pins the key ORDER as well as the values -+ * sorting the keys before comparing, which is what this test used to do,+ * destroys precisely the property "byte-identical" names.+ */+const GOLDEN_CLAIMLESS_PAYLOAD = '{"sample_id":"opaque_hash","first_pass_success":true}';++/** Payload key order of the pre-claim outcome row, in emission order. */+const GOLDEN_CLAIMLESS_PAYLOAD_KEYS: ReadonlyArray<string> = Object.freeze([+ "sample_id",+ "first_pass_success",+]);++/**+ * The whole JSONL record line a claimless `post` writes. `id` and+ * `createdAt` are the only fields a run cannot fix - the id is a content+ * hash over a wall-clock timestamp - so they are interpolated from the+ * record under test and everything else, envelope key order included, is+ * pinned as a literal.+ */+function goldenOutcomeLine(id: string, createdAt: string): string {+ return (+ `{"schema":"o2b.continuity.v1","id":"${id}","kind":"context_pack_outcome",` ++ `"createdAt":"${createdAt}","sourceRefs":[],"payload":${GOLDEN_CLAIMLESS_PAYLOAD},` ++ `"private":false,"redacted":false}`+ );+}++/** Raw lines of the month shard a record with this timestamp lands in. */+function shardLines(createdAt: string): ReadonlyArray<string> {+ return readFileSync(continuityLogPath(vault, createdAt.slice(0, MONTH_KEY_LENGTH)), "utf8")+ .split("\n")+ .filter((line) => line.trim().length > 0);+}++function writeConfig(enabled: boolean): void {+ atomicWriteFileSync(+ configPath,+ `vault: ${vault}\nagent_name: claude\ncontext_pack_outcome_enabled: "${enabled}"\n`,+ );+}++function seedReceipt(): { sampleId: string; finalTextHash: string; itemCount: number } {+ const record = emitContextReceipt(vault, {+ options: { host: "test", trigger: "context_pack", createdAt: "2026-07-01T00:00:00.000Z" },+ items: [+ { id: "pref-a", path: "Brain/preferences/pref-a.md", text: "preference one" },+ { id: "pref-b", path: "Brain/preferences/pref-b.md", text: "preference two" },+ ],+ finalText: PACKED_TEXT,+ });+ return {+ sampleId: record.id,+ finalTextHash: record.payload["final_text_hash"] as string,+ itemCount: record.payload["item_count"] as number,+ };+}++beforeEach(() => {+ tmp = mkdtempSync(join(tmpdir(), "o2b-ctx-evidence-tool-"));+ vault = join(tmp, "vault");+ mkdirSync(join(vault, "Brain"), { recursive: true });+ configHome = mkdtempSync(join(tmpdir(), "o2b-ctx-evidence-tool-cfg-"));+ configPath = join(configHome, "config.yaml");+ for (const key of [+ "VAULT_AGENT_NAME",+ "VAULT_TIMEZONE",+ "VAULT_DIR",+ "OPEN_SECOND_BRAIN_CONFIG",+ "OPEN_SECOND_BRAIN_CONTEXT_PACK_OUTCOME_ENABLED",+ ]) {+ savedEnv[key] = process.env[key];+ delete process.env[key];+ }+ process.env["OPEN_SECOND_BRAIN_CONFIG"] = configPath;+});++afterEach(() => {+ rmSync(tmp, { recursive: true, force: true });+ rmSync(configHome, { recursive: true, force: true });+ for (const [key, value] of Object.entries(savedEnv)) {+ if (value === undefined) delete process.env[key];+ else process.env[key] = value;+ }+});++async function initialize(server: MCPServer): Promise<void> {+ await server.handleRequest({+ jsonrpc: JSONRPC_VERSION,+ id: 1,+ method: "initialize",+ params: {+ protocolVersion: PROTOCOL_VERSION,+ capabilities: {},+ clientInfo: { name: "ctx-evidence-test", version: "0" },+ },+ });+}++async function callTool(+ server: MCPServer,+ name: string,+ args: Record<string, unknown>,+): Promise<Record<string, unknown>> {+ const response = (await server.handleRequest({+ jsonrpc: JSONRPC_VERSION,+ id: 9,+ method: "tools/call",+ params: { name, arguments: args },+ })) as { result: { content: ReadonlyArray<{ type: string; text: string }> } };+ return JSON.parse(response.result.content[0]!.text);+}++describe("brain_context_pack_outcome evidence half", () => {+ test("post reports the kernel's verdict for a sample with no claim", async () => {+ writeConfig(true);+ const seeded = seedReceipt();+ const server = new MCPServer({ vault, configPath });+ await initialize(server);+ const out = await callTool(server, "brain_context_pack_outcome", {+ operation: "post",+ sample_id: seeded.sampleId,+ first_pass_success: true,+ });+ expect(out["recorded"]).toBe(true);+ const evidence = out["evidence"] as Record<string, unknown>;+ expect(evidence["verdict"]).toBe(CONTEXT_PACK_EVIDENCE_VERDICT.unclaimed);+ expect(listContextPackEvidence(vault)).toHaveLength(1);+ });++ test("an agreeing claim reports match; a contradicted one reports mismatch and still records the outcome", async () => {+ writeConfig(true);+ const seeded = seedReceipt();+ const server = new MCPServer({ vault, configPath });+ await initialize(server);++ const agreeing = await callTool(server, "brain_context_pack_outcome", {+ operation: "post",+ sample_id: seeded.sampleId,+ first_pass_success: true,+ evidence_claim: { item_count: seeded.itemCount, final_text_hash: seeded.finalTextHash },+ });+ expect((agreeing["evidence"] as Record<string, unknown>)["verdict"]).toBe(+ CONTEXT_PACK_EVIDENCE_VERDICT.match,+ );++ const contradicted = await callTool(server, "brain_context_pack_outcome", {+ operation: "post",+ sample_id: seeded.sampleId,+ first_pass_success: true,+ evidence_claim: { item_count: 99 },+ });+ expect(contradicted["recorded"]).toBe(true);+ const evidence = contradicted["evidence"] as Record<string, unknown>;+ expect(evidence["verdict"]).toBe(CONTEXT_PACK_EVIDENCE_VERDICT.mismatch);+ expect(evidence["mismatches"]).toEqual([+ { field: "item_count", expected: "99", actual: String(seeded.itemCount) },+ ]);+ // the contradicted post is a RECORD, not a refusal: the outcome landed+ const list = await callTool(server, "brain_context_pack_outcome", { operation: "list" });+ expect(list["total"]).toBe(2);+ });++ test("a malformed claim is refused at the boundary rather than written as a guess", async () => {+ writeConfig(true);+ const server = new MCPServer({ vault, configPath });+ await initialize(server);+ const response = (await server.handleRequest({+ jsonrpc: JSONRPC_VERSION,+ id: 4,+ method: "tools/call",+ params: {+ name: "brain_context_pack_outcome",+ arguments: {+ operation: "post",+ sample_id: "s1",+ first_pass_success: true,+ evidence_claim: { item_count: -3 },+ },+ },+ })) as { error?: { message: string } };+ expect(response.error?.message).toContain("item_count");+ });++ test("gate off writes no evidence record and reports none", async () => {+ writeConfig(false);+ const seeded = seedReceipt();+ const server = new MCPServer({ vault, configPath });+ await initialize(server);+ const out = await callTool(server, "brain_context_pack_outcome", {+ operation: "post",+ sample_id: seeded.sampleId,+ first_pass_success: true,+ evidence_claim: { item_count: 99 },+ });+ expect(out["recorded"]).toBe(false);+ expect("evidence" in out).toBe(false);+ expect(listContextPackEvidence(vault)).toHaveLength(0);+ });++ test("a post with no evidence_claim is byte-identical to one made before the field existed", async () => {+ writeConfig(true);+ const server = new MCPServer({ vault, configPath });+ await initialize(server);+ const out = await callTool(server, "brain_context_pack_outcome", {+ operation: "post",+ sample_id: CLAIMLESS_SAMPLE_ID,+ first_pass_success: true,+ });+ const list = await callTool(server, "brain_context_pack_outcome", { operation: "list" });+ const rows = list["records"] as ReadonlyArray<{ id: string; createdAt: string }>;+ const row = rows[0]!;+ expect(row.id).toBe(out["id"] as string);++ // The record as it actually sits on disk. Key order has to be read+ // off the shard rather than off the MCP envelope, which re-serializes+ // with sorted keys - and sorting before comparing, which this test+ // used to do, destroys exactly the property "byte-identical" names.+ const line = shardLines(row.createdAt).find((entry) => entry.includes(`"id":"${row.id}"`));+ expect(line).toBe(goldenOutcomeLine(row.id, row.createdAt));+ const written = JSON.parse(line!) as { payload: Record<string, unknown> };+ expect(Object.keys(written.payload)).toEqual([...GOLDEN_CLAIMLESS_PAYLOAD_KEYS]);+ expect(JSON.stringify(written.payload)).toBe(GOLDEN_CLAIMLESS_PAYLOAD);++ // the evidence row records that the sample has nothing behind it+ const evidence = out["evidence"] as Record<string, unknown>;+ expect(evidence["verdict"]).toBe(CONTEXT_PACK_EVIDENCE_VERDICT.unresolved);+ expect(evidence["unresolved_reason"]).toBe("sample_absent");+ });+});diff --git a/tests/mcp/ner-tool.test.ts b/tests/mcp/ner-tool.test.tsindex 6bd06fed..accc5cea 100644--- a/tests/mcp/ner-tool.test.ts+++ b/tests/mcp/ner-tool.test.ts@@ -43,6 +43,7 @@ const handler = NER_TOOLS[0]!.handler;describe("brain_intake_entities", () => {test("intakes agent-supplied entities into the registry", async () => {const res = await handler(ctx, {+ source: "[[Notes/scaling.md]]",entities: [{ category: "concept", name: "Layer 2s" },{ category: "people", name: "Vitalik", aliases: ["V."] },@@ -52,6 +53,7 @@ describe("brain_intake_entities", () => {entities_created: [expect.any(String), expect.any(String)],entities_updated: [],relations_applied: 0,+ trust: "trusted",});expect(listEntities(vault)).toHaveLength(2);expect(getEntity(vault, { category: "concept", query: "Layer 2s" })?.name).toBe("Layer 2s");@@ -59,6 +61,7 @@ describe("brain_intake_entities", () => {test("applies typed relations between extracted entities", async () => {const res = await handler(ctx, {+ source: "[[Notes/restaking.md]]",entities: [{ category: "concept", name: "Restaking" },{ category: "concept", name: "Validators" },diff --git a/tests/mcp/schema-tools.test.ts b/tests/mcp/schema-tools.test.tsindex 5d94c971..86d8b807 100644--- a/tests/mcp/schema-tools.test.ts+++ b/tests/mcp/schema-tools.test.ts@@ -1,5 +1,5 @@import { afterEach, beforeEach, describe, expect, test } from "bun:test";-import { mkdtempSync, rmSync } from "node:fs";+import { mkdtempSync, readFileSync, rmSync, statSync, utimesSync } from "node:fs";import { tmpdir } from "node:os";import { join } from "node:path";@@ -13,6 +13,14 @@ let vault: string;let configPath: string;const savedEnv: Record<string, string | undefined> = {};+/**+ * Epoch seconds stamped onto `Brain/_brain.yaml` before a dry run, far+ * enough in the past that any write during the preview would move the+ * modification time to now rather than leaving it within a coarse+ * filesystem timestamp of where it started.+ */+const PAST_MTIME_SECONDS = Date.UTC(2020, 0, 1) / 1000;+beforeEach(() => {tmp = mkdtempSync(join(tmpdir(), "o2b-mcp-schema-"));vault = join(tmp, "vault");@@ -126,6 +134,93 @@ describe("schema MCP tools", () => {expect(nodes.map((node) => node.id)).toContain("link:decision");});+ test("dry_run previews the resulting pack and its diff without touching _brain.yaml", async () => {+ const server = makeServer();+ await initialize(server);+ const brainConfig = join(vault, "Brain", "_brain.yaml");+ utimesSync(brainConfig, PAST_MTIME_SECONDS, PAST_MTIME_SECONDS);+ // Base64 so the later comparison is over raw bytes, not decoded text.+ const bytesBefore = readFileSync(brainConfig).toString("base64");+ const mtimeBefore = statSync(brainConfig).mtimeMs;++ const preview = await call(server, "schema_apply_mutations", {+ mutations: [{ op: "add_type", category: "preference_types", token: "decision" }],+ dry_run: true,+ });++ const structured = (preview as any).result.structuredContent;+ expect(structured.dry_run).toBe(true);+ expect(structured.would_apply).toBe(1);+ expect(structured.pack.declarations.preference_types).toContain("decision");+ expect(structured.diff).toEqual([+ { path: "declarations.preference_types", before: null, after: "decision" },+ ]);+ // A preview must never look like an apply: no audit record was written,+ // so there is no audit path to report.+ expect(structured.audit_path).toBeUndefined();++ expect(readFileSync(brainConfig).toString("base64")).toBe(bytesBefore);+ expect(statSync(brainConfig).mtimeMs).toBe(mtimeBefore);+ const pack = await call(server, "schema_inspect", { view: "active_pack" });+ expect(+ (pack as any).result.structuredContent.pack.declarations.preference_types ?? [],+ ).not.toContain("decision");+ });++ test("an apply with no dry_run argument behaves exactly as today", async () => {+ const server = makeServer();+ await initialize(server);++ const applied = await call(server, "schema_apply_mutations", {+ mutations: [{ op: "add_type", category: "preference_types", token: "decision" }],+ });++ const structured = (applied as any).result.structuredContent;+ expect(structured.applied).toBe(1);+ expect(typeof structured.audit_path).toBe("string");+ expect(structured.pack.declarations.preference_types).toContain("decision");+ // The dry-run key exists only on a preview; an apply carries neither it+ // nor the diff, exactly as before dry_run was added.+ expect("dry_run" in structured).toBe(false);+ expect("diff" in structured).toBe(false);++ const pack = await call(server, "schema_inspect", { view: "active_pack" });+ expect((pack as any).result.structuredContent.pack.declarations.preference_types).toContain(+ "decision",+ );+ });++ test("a rejected preview returns the same error envelope the apply returns", async () => {+ const server = makeServer();+ await initialize(server);+ const rejected = [{ op: "add_prefix", prefix: "pref", token: "undeclared" }];++ const preview = await call(server, "schema_apply_mutations", {+ mutations: rejected,+ dry_run: true,+ });+ const apply = await call(server, "schema_apply_mutations", { mutations: rejected });++ expect((preview as any).result.isError).toBe(true);+ expect((preview as any).result.content[0].text).toBe(+ "schema.prefixes.pref: token is not declared",+ );+ expect((preview as any).result).toEqual((apply as any).result);+ });++ test("a non-boolean dry_run is an invalid param", async () => {+ const server = makeServer();+ await initialize(server);++ const response = await call(server, "schema_apply_mutations", {+ mutations: [{ op: "add_type", category: "preference_types", token: "decision" }],+ dry_run: "yes",+ });++ expect((response as any).error.code).toBe(INVALID_PARAMS);+ expect((response as any).error.message).toContain("dry_run");+ });+test("schema apply reports coercion failures as invalid params", async () => {const server = makeServer();await initialize(server);diff --git a/tests/mcp/skill-offer-tools.test.ts b/tests/mcp/skill-offer-tools.test.tsnew file mode 100644index 00000000..4063b498--- /dev/null+++ b/tests/mcp/skill-offer-tools.test.ts@@ -0,0 +1,146 @@+/**+ * Skill tool surface for the offer chain (Unit J, t_ccb05134).+ *+ * `skills_attach` hands back the id of the offer it just made, `get_skill`+ * accepts that id so the runtime's session log records which offer the+ * invocation came from, and `list_skills` surfaces the paths a skill+ * shadows. No new tool is added, so the registry guard's preview-budget+ * contract has to stay green - asserted here rather than assumed.+ */++import { test, expect, beforeEach, afterEach } from "bun:test";+import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { auditPreviewBudgets } from "../../src/mcp/registry-guard.ts";+import { SKILL_TOOLS } from "../../src/mcp/skill-tools.ts";+import { buildToolTable } from "../../src/mcp/tools.ts";+import { isSkillOfferId, SKILL_OFFER_ID_KEY } from "../../src/core/surface/skill-offer.ts";+import type { ServerContext } from "../../src/mcp/tool-contract.ts";++let tmp: string;+let vault: string;+let configPath: string;+let ctx: ServerContext;++function writeSkill(root: string, dir: string, name: string, description: string): string {+ const skillDir = join(root, dir);+ mkdirSync(skillDir, { recursive: true });+ writeFileSync(+ join(skillDir, "SKILL.md"),+ `---\nname: ${name}\ndescription: ${description}\n---\n\n# ${name}\n\nBody.`,+ );+ return skillDir;+}++beforeEach(() => {+ tmp = mkdtempSync(join(tmpdir(), "osb-skill-offer-tools-"));+ vault = join(tmp, "vault");+ mkdirSync(vault, { recursive: true });+ configPath = join(tmp, "config.yaml");+ writeFileSync(configPath, `vault: "${vault}"\nskill_auto_attach: "true"\n`);+ writeSkill(join(tmp, "skills"), "embeddings-setup", "embeddings-setup", "Configure embeddings.");+ ctx = { vault, configPath, repoRoot: tmp };+});++afterEach(() => {+ rmSync(tmp, { recursive: true, force: true });+});++function tool(name: string) {+ return SKILL_TOOLS.find((t) => t.name === name)!;+}++test("skills_attach returns the id of the offer it just made", async () => {+ const result = (await tool("skills_attach").handler(ctx, { query: "configure embeddings" })) as {+ enabled: boolean;+ offer_id: string | null;+ block: string;+ };+ expect(result.enabled).toBe(true);+ expect(isSkillOfferId(result.offer_id)).toBe(true);+ expect(result.block).toContain(result.offer_id!);+});++test("a turn that is offered nothing gets no offer id", async () => {+ const result = (await tool("skills_attach").handler(ctx, { query: "qqq zzz" })) as {+ enabled: boolean;+ offer_id: string | null;+ };+ expect(result.enabled).toBe(true);+ expect(result.offer_id).toBeNull();+});++test("the disabled surface reports no offer, exactly as it reports no block", async () => {+ writeFileSync(configPath, `vault: "${vault}"\n`);+ const result = (await tool("skills_attach").handler(ctx, { query: "configure embeddings" })) as {+ enabled: boolean;+ block: string;+ offer_id: string | null;+ skills: unknown[];+ };+ expect(result.enabled).toBe(false);+ expect(result.block).toBe("");+ expect(result.skills).toHaveLength(0);+ expect(result.offer_id).toBeNull();+});++test("get_skill echoes the offer the caller cited so the session log carries it", async () => {+ const offered = (await tool("skills_attach").handler(ctx, { query: "configure embeddings" })) as {+ offer_id: string;+ };+ const result = (await tool("get_skill").handler(ctx, {+ name: "embeddings-setup",+ [SKILL_OFFER_ID_KEY]: offered.offer_id,+ })) as Record<string, unknown>;+ expect(result[SKILL_OFFER_ID_KEY]).toBe(offered.offer_id);+});++test("get_skill without an offer id omits the key rather than inventing one", async () => {+ const result = (await tool("get_skill").handler(ctx, {+ name: "embeddings-setup",+ })) as Record<string, unknown>;+ expect(SKILL_OFFER_ID_KEY in result).toBe(false);+});++test("get_skill refuses a malformed offer id instead of recording a fiction", () => {+ expect(() =>+ tool("get_skill").handler(ctx, { name: "embeddings-setup", [SKILL_OFFER_ID_KEY]: "nope" }),+ ).toThrow("offer_id");+});++test("get_skill declares offer_id in its input schema", () => {+ const schema = tool("get_skill").inputSchema as {+ properties: Record<string, { type: string; description: string }>;+ };+ expect(schema.properties[SKILL_OFFER_ID_KEY]!.type).toBe("string");+ expect(schema.properties[SKILL_OFFER_ID_KEY]!.description.length).toBeLessThanOrEqual(160);+});++test("list_skills surfaces the paths a skill shadows", async () => {+ const vaultSkills = join(vault, "Brain", "skills");+ const shadowed = join(tmp, "skills", "embeddings-setup");+ writeSkill(vaultSkills, "embeddings-setup", "embeddings-setup", "Operator override.");++ const result = (await tool("list_skills").handler(ctx, {})) as {+ skills: Array<{ name: string; description: string; shadowed: string[] }>;+ };+ expect(result.skills).toHaveLength(1);+ expect(result.skills[0]!.description).toBe("Operator override.");+ expect(result.skills[0]!.shadowed).toEqual([shadowed]);+});++test("an unshadowed skill reports an empty shadowed list", async () => {+ const result = (await tool("list_skills").handler(ctx, {})) as {+ skills: Array<{ shadowed: string[] }>;+ };+ expect(result.skills[0]!.shadowed).toEqual([]);+});++test("the preview-budget contract still holds over the whole tool table", () => {+ const audit = auditPreviewBudgets(buildToolTable("full"));+ expect(audit.unbudgetedAndUnexempted).toEqual([]);+ expect(audit.exemptButBudgeted).toEqual([]);+ expect(audit.exemptButUnknown).toEqual([]);+});