Local-first 🧠memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly 😴 dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.
TypeScript
feat: write-time honesty and egress consent (v1.92.0) (#273)
Serg Eroshe5572997d74ea
diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.jsonindex 3a6138e5..d8666348 100644--- a/.claude-plugin/plugin.json+++ b/.claude-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.91.0",+ "version": "1.92.0","description": "Plugin-first second brain package for AI agents and humans.","author": {"name": "Open Second Brain contributors"diff --git a/.codex-plugin/plugin.json b/.codex-plugin/plugin.jsonindex 15f36409..71bb044d 100644--- a/.codex-plugin/plugin.json+++ b/.codex-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.91.0",+ "version": "1.92.0","description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.","author": {"name": "Open Second Brain contributors",diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 231fe972..e446837e 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -5,6 +5,26 @@ All notable changes to this project will be documented in this file.The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).+## [1.92.0] - 2026-10-22++Open Second Brain 1.92.0 makes the write path and the egress path keep promises the rest of the system already made. Informal privacy markers in frontmatter - `access: private`, `confidentiality: secret`, `sensitivity: confidential` - now resolve to the reserved remote-deny token at every remote read root, and a preview-first verb stamps the formal token onto the pages that only ever carried the informal spelling. Chunks the declared embedding window cannot hold are held out of embedding under a persistent record instead of being truncated and indexed as something shorter, with the hold reported and released when its premise moves. People resolve through derived name forms - first initial plus surname, word order, and operator-supplied nicknames - while declared spellings keep precedence and ambiguity is reported rather than guessed. A fifth page-lint detector asks a count or an amount for its date, prompt-driven writes carry a server-stamped content class that shared exports honor by default, and a provider decline is a persisted decision honored before any network call. Review of the release closed four defects, among them an OKF bundle that left out its own reserved root files.++### Fixed++- **An OKF bundle carries its reserved root files, and an index that agrees with the manifest.** `o2b brain okf-export` composes its bundle from the pages the content-class split kept, and `index.md` and `log.md` were not reaching the written directory, so a bundle arrived without the two files the rest of its layout assumes. Both now ride every bundle. The index is re-rendered from the filtered manifest by the same renderer the builder uses rather than copied from the source vault, so a page the export excluded is named neither in `okf.json` nor in `index.md` - an index listing pages the shipped manifest does not carry would make the bundle disagree with itself - and `log.md` stays whole, because its entries are day-keyed events rather than page content, while still passing the same scan as every page file.+- **The relational arm reads the indexed visibility column at remote reach before it names a candidate.** A remote answer from the relational arm joined its candidates through the page index and resolved them against the vault, leaving a window in which a page whose remote reservation had just been lifted could still be named from rows the previous index pass had written. The arm now consults the indexed visibility column at remote reach, so the answer holds at the joined view rather than at one of its two sources, and a page readable at local reach is unaffected.+- **The undated-quantity exclusion scan is order-independent.** The scan now settles its exclusions over the page as a whole before it reports, so the findings depend on what the page contains rather than on the order its regions were visited, and the same bytes yield the same findings on every run.+- **A research report page now carries the prompt-derived class stamp.** The write path applies it at creation beside the rest of the report's frontmatter, and the byte-identity fixture is updated to the stamped shape as the only byte change. A report page therefore reads as prompt-derived to the shared exports, and the report lane's output is subject to the same default exclusion as a session import or an MCP note write.++### Added++- **Informal privacy markers resolve to the reserved remote-deny token at every remote read root, and a preview-first verb writes the reservation the pages never carried.** A page that says `access: private`, `confidentiality: secret` or `sensitivity: confidential` was already withheld from a remote caller, but every consumer that reads `visibility:` directly - a derived page stamped through the strictest-visibility fold, an export filtering on the token, an operator auditing reservations - could not see the claim. The three markers now resolve to the reserved token at every remote read root: search results, vault page lists and the indexed snippet column answer the same deny the formal key would. `o2b brain privacy-marker-restamp` makes the claim formal - dry run by default, `--apply` to write - unioning the token into `visibility:` additively and idempotently, re-reading each candidate immediately before its write so an edit that landed in between is honored, refusing by name (in the verb's `skipped` list) a page whose frontmatter cannot be rewritten losslessly instead of quietly absorbing it into a lower updated count, never removing a token and never re-embedding, with the write guard asserted at the applying entry point only. A doctor finding counts the marker pages that are still unstamped, and `update_note` refuses changes to the three marker keys exactly as it refuses changes to the reserved `visibility` key, so the informal spelling cannot be removed from a page whose reservation depends on it.+- **A chunk that cannot fit the embedding model's declared input window is held out of embedding instead of silently truncated, under a record that says why.** The serving stack used to cut such a chunk to the window, so the index held an embedding of text the page does not contain, with nothing on any surface saying so. A chunk provably over the declared window is now skipped and recorded under a persistent hold keyed by the model, the window and the chunk signature, so the same chunk under the same configuration stays held without re-deciding, and the record is released automatically when the model, the window or the signature changes - a moved premise reconsiders, a static one stays quiet. A held chunk is excluded from the pending-vectors census, so no backfill is recommended for text this configuration cannot embed, and the estimator's undecided band is reported rather than resolved to one side. `o2b search status --json` carries a bounded `held_chunks` report - position-ordered, capped at 20 with the whole named by `total`, present only while the active configuration holds something - and a doctor check counts the holds, so the state is visible without reading the index.+- **Person entities resolve through derived name forms, with a declared spelling always outranking a derived one and ambiguity reported instead of guessed.** Two derivation families join the resolver: forms computed from a declared name - a first-initial-plus-surname spelling, and the word-order variants of a multi-part name - and operator-supplied nicknames read from `Brain/entities/person-nicknames.json`; nothing ships a built-in word list, so a nickname exists only because the operator wrote it. The forms answer in query expansion and in registry lookups, and ranking is explicit: a match on a declared name or alias outranks a match that only a derived form reaches, so a derived form can widen recall but never displace the spelling the vault declares. Ambiguity is reported rather than resolved - a derived form that matches more than one person refuses and names the competing ids - write lanes stay exact-only, and a doctor warning names a derived variant that collides with another person's declared name.+- **A fifth write-time page-lint detector warns when a count or an amount is written with no date it could come from.** The recognized shapes are structural, never natural language: a currency amount by symbol or ISO code, a percentage, a number with a unit from the declared set, and a bare thousands-separated count. A page that supplies a date anywhere the quantity could take one stays quiet - a `valid_from` boundary in frontmatter, any ISO date in the body, or a `[Source: <name>, YYYY-MM-DD]` citation marker - and the finding carries the matched lexeme and its offset, never the authored line, so a receipt cannot repeat prose the caller did not ask it to carry. Each finding names `o2b brain claims` as the next command, where the quantity can be placed in time.+- **Prompt-driven writes now carry a server-stamped `content_class: prompt-derived`, and every shared export leaves those pages out by default.** The stamp lands at creation on the deterministic prompt-driven paths - a note write that arrives over MCP, the session import, and the in-process report writers - and it is derived by the writer rather than accepted from the caller: a caller-supplied value is dropped, an operator-authored CLI write stays unmarked, and no path removes the stamp afterwards. The shared exports - bank, graph, OKF and knowledge packs - exclude class-marked pages by default and name every omission with the page's path and class, so a smaller export is never a silent one; the bank, graph and OKF verbs and the transcript export (`o2b brain export --format transcripts-jsonl`) take `--include-prompt-derived`, which carries the pages back in and says so on the run, while the knowledge pack always blocks class-marked pages and names them in its `blocked` list. Pages written before this release carry no mark and export exactly as they did; that era boundary is documented in `docs/cli-reference.md` and `SECURITY.md` rather than migrated, because a stamp applied by a later pass would describe the pass and not the writer.+- **A provider decline is a persisted, fail-closed record honored before any network call.** `search_semantic_enabled=false` and `embedding_provider=disabled` used to reach the same capability verdict as a machine that had never configured semantic search at all, so the two situations were indistinguishable to every surface that asked. A decline written in the configuration is now read as a record of its own, distinct from never-configured, and it is fail-closed: an unreadable or unparseable record is a decline, not consent. The decline is honored at provider construction, before a request is built, and the refusal names the keys that record it and the keys that would re-enable the provider. `o2b doctor --readiness` gains a consent probe that reports the decline as a decision rather than a fault, and lists the provider environment variables it found by NAME only, never by value. Beside the record, a new egress consent-stamp primitive ships with a presence-based kill switch, `OPEN_SECOND_BRAIN_EGRESS_CONSENT_KILL`; it declares no destination of its own, adds no egress registry row, and leaves every network path in this release unchanged.+## [1.91.0] - 2026-10-21Open Second Brain 1.91.0 turns one-off promises into first-class memory. A promise made in prose becomes a pledge page under `Brain/pledges/` - title, promise text, due date, the source page it was read from, and evidence quotes the kernel verified against that source at write time - with direct CRUD and a validate-everything-first extraction intake over MCP, a pledges section on the today dashboard, and two delivery surfaces for the moment of need: an opt-in context-pack rollup that groups what a query's entities already kept, and an opt-in counterparty brief that hands the agent at most three shareable facts about the person a prompt names, once per session. Both delivery surfaces are off by default and byte-identical when off, and nothing in the release calls a model.@@ -8703,6 +8723,7 @@ Hermes / Claude Code / Codex / OpenClaw configurations do not change.- Sandbox vault and plugin manifest fixtures for tests.- GitHub release workflow for tag-based and manually dispatched releases.+[1.92.0]: https://github.com/itechmeat/open-second-brain/compare/v1.91.0...v1.92.0[1.91.0]: https://github.com/itechmeat/open-second-brain/compare/v1.90.0...v1.91.0[1.90.0]: https://github.com/itechmeat/open-second-brain/compare/v1.89.0...v1.90.0[1.89.0]: https://github.com/itechmeat/open-second-brain/compare/v1.88.0...v1.89.0diff --git a/README.md b/README.mdindex 477710e6..18ad7d87 100644--- a/README.md+++ b/README.md@@ -123,7 +123,7 @@ The full router with readiness criteria is [`install.md`](install.md); native Wi## What is new-Open Second Brain 1.91.0 turns one-off promises into first-class memory. A promise made in prose becomes a pledge page under `Brain/pledges/` - title, promise text, due date, the source page it was read from, and evidence quotes the kernel verified against that source when the page was written, refusing by name an empty evidence list, a source page the vault does not hold, and any quote the cited source does not contain. `brain_pledge` carries direct CRUD over the family and `brain_intake_pledges` is the extraction intake, which validates every item before writing any page, anchors counterparty names against canonical entities instead of trusting the caller, and reports a repeat of the same payload as a no-op. The today dashboard gains a pledges section beside obligations and open loops, and two opt-in delivery surfaces serve the moment of need: an `entity_facts` rollup on the context pack that groups what a query's entities already kept, and a counterparty brief on UserPromptSubmit that hands the agent at most three shareable facts about the person a prompt names, once per session, gated on the operator-declared trigger vocabulary. Both delivery surfaces are off by default and byte-identical when off, and the kernel never calls a model. Every release is described in the [CHANGELOG](CHANGELOG.md).+Open Second Brain 1.92.0 makes the write path and the egress path keep the promises the rest of the system already made. Informal privacy markers in frontmatter now resolve to the reserved remote-deny token at every remote read root, and `o2b brain privacy-marker-restamp` previews and then stamps the formal reservation onto pages that only carried the informal spelling. A chunk over the embedding model's declared input window is held out of embedding under a record naming the model, the window and the signature, reported on `search status` and released when its premise moves, instead of being silently truncated. Person entities resolve through initials, word order and operator-supplied nicknames, with declared spellings always outranking derived ones and ambiguity reported rather than guessed. A fifth page-lint detector asks a count or an amount for its date, prompt-driven writes carry a server-stamped content class that shared exports leave out by default, and a provider decline is honored before any network call. Every release is described in the [CHANGELOG](CHANGELOG.md).## Documentationdiff --git a/SECURITY.md b/SECURITY.mdindex d7becdec..55acd505 100644--- a/SECURITY.md+++ b/SECURITY.md@@ -28,3 +28,24 @@ network calls only to the embedding, rerank or research endpoints the operatorconfigures. Reports about how those boundaries hold are in scope. Issues inthird-party agent hosts (Claude Code, Codex, Hermes, OpenClaw, opencode) belongwith their maintainers.++## What shared exports carry++Every export that writes a file an operator names runs the shared redactor+over its bytes before they land. Since v1.92.0 the shared exports also apply+a content-class policy: pages whose `content_class` frontmatter carries the+server-stamped value `prompt-derived` - material an agent produced from a+prompt - are left out of the export by default, and each omission is named+on stderr. The per-verb `--include-prompt-derived` flag carries those pages+back in and says so on the same run. The knowledge pack blocks such pages in+its selection step and names them in its `blocked` list.++Pages written before this shipped carry no `content_class` key and are never+rewritten to add one; absence of the mark is the documented pre-stamp era,+and such a page exports exactly as it did before. A hand-edited class value+this build does not recognise reads as unmarked rather than hiding the page.++The full decision record for every export path, scanned and unscanned, is+`src/core/egress/registry.ts`, and the operator-facing summaries are the+"content class" and "network egress" sections of+[`docs/cli-reference.md`](docs/cli-reference.md).diff --git a/docs/brainstorm/write-honesty-egress-consent/cli-output/codex.md b/docs/brainstorm/write-honesty-egress-consent/cli-output/codex.mdnew file mode 100644index 00000000..7f729f92--- /dev/null+++ b/docs/brainstorm/write-honesty-egress-consent/cli-output/codex.md@@ -0,0 +1,34 @@+### Variant 1: Feature lanes with queued integration+- **Approach**: Assign one lane to each feature, combining lanes when only four or five coders are available. Each lane implements its feature against existing seams, with explicit ownership of shared files such as the indexer and doctor registries. Owners apply other lanes’ prepared changes to those files through a serialized integration queue. Integrate each feature’s census and registry updates with its implementation, then run the combined architecture checks after every merge.+- **Trade-offs**:+ - Pro: Closely follows the verified task slices and keeps feature acceptance accountable to one coder.+ - Pro: Independent leaf modules and detectors can progress immediately.+ - Con: Shared-file owners become bottlenecks, especially around indexer changes and doctor registration.+ - Con: Independently developed policy assumptions can surface late during integration.+- **Complexity**: medium+- **Risk**: high++### Variant 2: Contract-first dependency waves+- **Approach**: Begin with a short substrate wave that establishes leaf-module contracts, shared-file ownership, and the census registration sequence without building a general policy framework. Then use six lanes: integration/substrate, F1+F5, F2, F3, F4, and F6; with four coders, combine integration with F1+F5 and combine F3 with F4. Feature lanes own disjoint implementation files, while the integration owner applies their prepared changes to shared files in dependency order: predicates and resolvers, enforcement and persistence, then reporting surfaces. Register doctor codes with their required census entries, egress changes with their registry entries, and durable surfaces only when their actual storage decisions are settled. Finish with combined acceptance checks for read filtering, held-item lifecycle, egress exclusions, and provider construction before network access.+- **Trade-offs**:+ - Pro: Makes shared-file coordination explicit while preserving parallel implementation.+ - Pro: Groups privacy and content-class work for consistent policy handling while keeping their metadata and semantics distinct.+ - Pro: Sequences census updates alongside real implementation rather than reserving speculative rows or counts.+ - Con: Requires an initial coordination pass and disciplined handoffs to the integration owner.+ - Con: Incorrect early contracts can require changes across several lanes.+- **Complexity**: medium+- **Risk**: low++### Variant 3: Enforcement spine first, advisory features second+- **Approach**: Complete F6 first, then implement F1 and F5 as the next gated stage, establishing provider consent, remote-read protection, and content-class egress exclusion before broadening the wave. During those stages, other coders prepare the isolated F2 hold-decision logic, F3 person-variant module, and F4 detector against agreed interfaces. After the enforcement stage passes its acceptance checks and censuses, integrate F2, F3, and F4 in parallel under shared-file ownership. Each stage carries its own doctor, state-surface, and registry changes, with final combined verification across all six features.+- **Trade-offs**:+ - Pro: Concentrates early review on the features governing disclosure and outbound calls.+ - Pro: Smaller integration stages make regressions easier to localize.+ - Con: Imposes ordering between features that are not all technical dependencies.+ - Con: Four to six coders have less useful parallel work while waiting for enforcement integration.+ - Con: Later indexer and reporting changes still require checking earlier guarantees.+- **Complexity**: large+- **Risk**: medium++### Recommended: Variant 2+**Rationale**: It gives four to six coders useful parallel work with disjoint file ownership and an explicit route for shared-file edits. Coordinating doctor codes, state surfaces, and the egress registry through one integration lane reduces census conflicts while preserving fail-closed checks. The substrate stays limited to concrete contracts and leaf primitives, avoiding premature storage choices for the consent stamp. Compared with feature-first integration or a largely sequential enforcement stage, it offers the strongest balance of throughput and integration control.diff --git a/docs/brainstorm/write-honesty-egress-consent/cli-output/prompt.md b/docs/brainstorm/write-honesty-egress-consent/cli-output/prompt.mdnew file mode 100644index 00000000..5f96ad73--- /dev/null+++ b/docs/brainstorm/write-honesty-egress-consent/cli-output/prompt.md@@ -0,0 +1,34 @@+You are brainstorming architectural variants for a 6-feature release wave. Do not write code. Do not write a final design. Only produce variants and a recommendation.++# Task++Open Second Brain (local-first personal knowledge vault, TypeScript/Bun, MCP server + CLI; kernel in src/core, MCP tools in src/mcp, CLI verbs in src/cli, architecture censuses in tests/core/architecture). One release ships six validator-confirmed features on the spine "write-time honesty and egress consent":++1. F1 privacy-markers (t_e095afac): informal frontmatter privacy markers (access: private, confidentiality: secret) currently get zero protection - pageVisibility (src/core/graph/visibility.ts:44-48) reads only meta["visibility"]. Plan: marker predicate, remote-deny enforcement at the three read roots (result-filters.ts isPathReadableAtReach/applyReachFilter, vault.ts listVaultPages, indexer visibility stamp), preview-first tighten-only restamp verb, doctor finding. Tighten-only: only ADD the reserved token.+2. F2 chunk skip-and-hold (t_f029b413): chunks over the embedding model input window are silently truncated by the serving stack (indexer.ts:1184 embed call; presets carry windows). Plan: hold decision in runEmbeddingPhase (no embed, no vecUpsert), persistent hold record excluded from the pending anti-join (chunks.ts PENDING_ANTI_JOIN), bounded held-items report in IndexStats/status wire, doctor check, census warning update, release-on-config-change.+3. F3 person-name variants (t_46296794): entity lookup is exact normalized equality only (entity-alias.ts:58-59; registry.ts resolveActive). Plan: pure person-variants leaf module (initials, reversed order, operator-extensible nickname data - no hardcoded English word lists), fold into query-side expansion with declared-vs-derived tracking, exact-declared-first ranking tier (declared keeps existing boost, derived gets strictly smaller capped lane), registry lookup decision (extend resolveActive with ambiguity-throws or ranking-only), doctor entity-alias-candidate on collisions.+4. F4 undated-quantity lint (t_fb442a24): page-lint.ts has four detectors; add a fifth warning detector flagging counts/amounts written without any date (currency, percent, number+unit, thousands-separated) with escape hatches (valid_from, ISO date in body, [Source: ..., YYYY-MM-DD] marker); register in diagnostics for next_command; receipt lint key stays advisory and absent-when-clean.+5. F5 content-class egress (t_5f545a9b): exports/bundles apply credential-shape redaction only; no content-class policy. Plan: new server-derived creation-time frontmatter key (e.g. content_class, reserved value prompt-derived, spelling outside origin-channel census CHANNEL_SPELLINGS) stamped at deterministic prompt-derived write paths (session import, MCP note writes, research/dream pipelines that bypass createNote), excluded by default at the egress guard seam with per-verb operator override, named omissions, no-backfill era boundary, knowledge-pack block reason, census extension.+6. F6 provider-decline consent (t_fa467133): explicit provider decline (search_semantic_enabled=false / embedding_provider=disabled) and never-configured collapse into the same disabled tier; nothing persists an explicit decline decision and no doctor check names provider env vars. Plan: provider-consent resolver distinguishing unset vs declined (fail-closed on unreadable config), doctor consent probe naming env vars NAME-ONLY, honor decline at provider construction (single makeProvider decision point) before any network call, consent-stamp primitive with env kill switch (no destination declared yet), census pin bumps if durable files appear.++# Project context++Conventions: version bump + CHANGELOG inside the PR (not this phase); censuses fail closed (a new doctor code must join doctor-exits.ts + applier-capability.ts; new durable files join state surfaces with pinned counts; new registry rows need >=80-char reasons; import-cycles ratchet - shared modules must be leaves); TDD red-first; tests run with `env HOME=$(mktemp -d) bun test <files>`; Bun 1.4.0 pin; Windows-safe tests (no chmod-deny, normalize paths); no secret-shaped identifiers before quoted values; English, no exclamation marks; ox formatter at ./node_modules/.bin/oxfmt.+Verified premise reports for all six features exist with per-feature task slices (3-6 tasks each, ~30 tasks total, 45-65 files).++# Required output format++Produce exactly 3 distinct architectural variants for STRUCTURING this wave (lane decomposition, dependency ordering, shared-substrate strategy, census/registry sequencing, integration risks). For each variant:++### Variant N: <short name>+- **Approach**: 3-5 sentences describing the variant.+- **Trade-offs**: bullet list of pros and cons.+- **Complexity**: small | medium | large+- **Risk**: low | medium | high++After the three variants, add exactly one recommendation:++### Recommended: Variant N+**Rationale**: 3-4 sentences explaining why this variant over the others, considering parallel implementation by 4-6 coders on disjoint file sets, the census coupling (doctor codes, state surfaces, egress registry), and integration risk.++Output nothing outside of these sections.diff --git a/docs/brainstorm/write-honesty-egress-consent/design.md b/docs/brainstorm/write-honesty-egress-consent/design.mdnew file mode 100644index 00000000..7cf50f17--- /dev/null+++ b/docs/brainstorm/write-honesty-egress-consent/design.md@@ -0,0 +1,192 @@+# Write-time honesty and egress consent - six-feature release wave++**Status:** draft+**Author:** osb-feature-release run osb-feature-release-1792650839871 (via feature-release-playbook)+**Audience:** implementation lanes++## Problem statement++Four classes of dishonesty survive in the current tree: informal privacy frontmatter+markers (`access: private`, `confidentiality: secret`, `sensitivity: confidential`) resolve to default visibility+because the visibility layer reads only `meta["visibility"]` (src/core/graph/visibility.ts:44-48);+embedding chunks larger than the model input window are silently truncated by the+serving stack while the full text stays searchable (src/core/search/indexer.ts:1184);+person lookups match only exact normalized name-or-alias equality+(src/core/search/entity-alias.ts:58-59); and counts or amounts written without a+date become claims that cannot be placed in time (page-lint.ts runs four detectors,+none checks date presence). On the egress side, shared exports apply credential-shape+redaction only (src/core/egress/guard.ts:141-144) with no content-class policy, and an+explicit provider decline collapses into the same disabled tier as never-configured,+so no surface honors a written operator decision (src/core/doctor-readiness.ts:294-299).++All six premises were re-verified against the live worktree on 2026-10-22 (premise+reports in the wave scratch; anchors below are from that pass). Every premise is+confirmed; two framing corrections were folded in: an undated claim answers as true at+every instant (isValidAt treats absent valid_from as unbounded), and provider consent+today never asks - it is implicit configuration, so the feature is a persisted decline+record plus name-only doctor disclosure, not a per-surface refusal fix.++## Scope++- F1 privacy markers (kanban t_e095afac): detect informal privacy frontmatter+ markers; remote-deny enforcement at the three read roots plus the indexed+ visibility column; preview-first tighten-only restamp verb; doctor finding.+- F2 chunk skip-and-hold (t_f029b413): over-window chunks are held, not embedded;+ persistent hold record excluded from the pending anti-join; bounded held-items+ report on the status wire; doctor check; census warning update; holds release on+ window/model/chunk-size change.+- F3 person-name variants (t_46296794): pure person-variant derivation (initials,+ word order, operator-extensible nickname data); query-side expansion with+ declared-vs-derived tracking; exact-declared-first ranking; registry lookup+ resolves derived variants with ambiguity-throws; doctor collision finding.+- F4 undated-quantity lint (t_fb442a24): fifth page-lint detector flagging+ counts/amounts written without any date, with structural escape hatches;+ diagnostics registration for next_command; advisory receipt stays+ absent-when-clean.+- F5 content-class egress (t_5f545a9b): server-derived creation-time+ `content_class` frontmatter key with reserved value `prompt-derived`, stamped at+ deterministic prompt-derived write paths, excluded by default from shared export+ verbs at the guard seam with per-verb operator override and named omissions;+ no-backfill era boundary; census extension.+- F6 provider-decline consent (t_fa467133): persisted decline resolution distinct+ from unset (fail-closed on unreadable config); honor the decline at provider+ construction before any network call; doctor consent probe naming provider env+ vars by name only; egress consent-stamp primitive with an environment kill+ switch (no destination declared in this release).++## Out of scope++- Backfilling `content_class` onto pre-existing pages (pre-stamp era is documented,+ never migrated).+- Any natural-language date recognizer for F4 (structural ISO shapes only, per+ src/core/search/temporal-intents invariant).+- Hardcoded English nickname word lists for F3 (operator-extensible data only;+ structural rules only).+- Widening the reserved visibility token set beyond `private` (visibility.ts:80-92+ docblock refuses a wider policy); informal markers enforce remote-deny only, not+ caller-scope restriction.+- Declaring a new egress destination for the consent stamp (primitive and kill+ switch only; the absorbed relay sibling stays deferred).+- Any new runtime dependency.++## Chosen approach++Six implementation lanes on disjoint file sets, one feature per lane, each lane a+red-first TDD sequence of 4-6 tasks committing per task. The lane split follows the+module ownership the premise reports verified: visibility roots (F1), the embedding+phase (F2), entity resolution (F3), page lint (F4), the egress seam (F5), provider+construction and doctor readiness (F6). Two shared files get single primary owners+with append-with-poll for the second writer: `src/core/brain/diagnostics.ts`+(primary F4, F1 appends its doctor signal in its final task after polling for F4's+commit) and `docs/cli-reference.md` (primary F5, F1 and F2 append their CLI doc+lines in their final tasks). `src/core/search/indexer.ts` is owned outright by F2;+F6's decline is resolved by `resolveProviderConsent` and handed to+`makeProvider(config.semantic, consent)`: the indexer's embed phase and its+embedding probe call the resolver directly (a coordinated edit with F2) and the+probe reports a decline as skipped. Integration is one integrator pass: census recounts, full local gates, the+docs and version bump land in the osb-pr-prepare sub-playbook.++## Design decisions++- F1 enforces remote-deny only, at the read roots (`isPathReadableAtReach`,+ `applyReachFilter`, `listVaultPages`) plus the indexed visibility stamp - not+ inside `pageVisibility` - so informal markers never become caller-scope-liftable+ tokens. Derived-page stamping is not untouched in the shipped tree: the derived+ drafters (tensions, pledges, the dream lane) union an informal marker's+ reservation into the `strictestVisibility` fold, and the counterparty brief's+ entity and preference gates read the same union, so a marker page's+ reservation reaches the pages derived from it. The restamp+ is additive-only: it unions the reserved token into `visibility:`, never removes+ keys or tokens, follows the dry-run-by-default repair-verb family contract, and+ rewrites through `writeFrontmatterAtomic` behind `assertVaultIdentityForWrite`.+- F2 holds decide on the conservative side of the token estimator: only+ provably-over-window chunks are held; the undecided band stays reported, never+ silently resolved. Held chunks leave the pending anti-join (a persistent hold+ marker keyed by model/window/signature), so vector-backfill neither re-embeds nor+ recommends forever, and a config change releases holds back into pending. The+ batch 1:1 invariant survives because holding filters before batching.+- F3 variant derivation is a leaf module (import-cycles ratchet) emitting frozen,+ bit-stable forms for person-category entities only, with a minimum-form-length+ floor and no variants for unspaced scripts. Ranking keeps the never-retrieval+ invariant: declared-form matches keep the existing boost lane, derived variants+ get a strictly smaller capped lane, both omitted when zero. Registry lookup+ consults derived variants only after an exact name/alias miss and throws+ `EntityAmbiguityError` on multi-match; write-target resolution never follows+ quarantined namesakes' variants.+- F4 reuses the existing detector skeleton (code constant, `finding()` factory,+ warning severity, 25-finding cap, absent-key-when-clean) and the structural date+ recognizers (`DATE_ONLY_RE`, `ISO_INSTANT_RE`) as escape hatches. Trigger+ classes are high-precision only: currency, percent, number-with-unit,+ thousands-separated counts; versions, list markers, table cells and code spans+ are excluded. Findings quote the matched lexeme and offset, never the line.+- F5 stamps `content_class` server-side at creation on the write paths that are+ definitionally prompt-derived (session import, MCP note writes, in-process+ generation pipelines that bypass createNote), reserves the key against caller+ updates beside `visibility`/`origin_channel`, and enforces the default exclusion+ once at the shared guard so every `shared_redactor` site inherits it. Spelling+ avoids every origin-channel census spelling. Override is per-verb, names every+ omission in output, and the transcript export refuses-or-names rather than+ silently emptying.+- F6 resolves decline through one resolver consumed by provider construction;+ an unreadable or unparseable consent record resolves to declined (fail-closed).+ The doctor probe extends the readiness probe registry and prints env var NAMES+ only - tests assert absence of values. The consent-stamp primitive ships with an+ environment kill switch that no env value can re-enable, and declares no+ destination.++## File changes++Lane ownership (new files marked +):++- F1: `src/core/graph/privacy-markers.ts` (+), `src/core/graph/visibility.ts`,+ `src/core/search/result-filters.ts`, `src/core/vault.ts`,+ `src/core/search/indexer.ts` (visibility stamp only, coordinated with F2),+ `src/core/brain/privacy-marker-restamp.ts` (+), `src/core/brain/doctor/privacy-marker-check.ts` (+),+ `src/core/brain/doctor.ts`, `src/core/brain/applier-capability.ts`,+ `src/core/brain/doctor-exits.ts`, `tests/core/graph/`, `tests/core/search/`,+ `tests/core/brain/`.+- F2 (owns `src/core/search/indexer.ts`): `src/core/search/indexer.ts`,+ `src/core/search/schema.ts` (migration), `src/core/search/store/chunks.ts`,+ `src/core/search/store/counts.ts`, `src/core/search/embedding-spend.ts`,+ `src/core/search/serialize.ts`, `src/core/search/types.ts`,+ `src/core/brain/doctor/` (held-sources check), `tests/core/search/`.+- F3: `src/core/brain/entities/person-variants.ts` (+),+ `src/core/search/entity-alias.ts`, `src/core/search/pipeline/candidate-signals.ts`,+ `src/core/search/ranker.ts`, `src/core/brain/entities/registry.ts`,+ `src/core/brain/doctor/entity-checks.ts`, `tests/core/search/`, `tests/core/brain/`.+- F4: `src/core/brain/undated-quantity.ts` (+), `src/core/brain/page-lint.ts`,+ `src/core/brain/diagnostics.ts` (primary owner), `src/cli/brain/help-text.ts`,+ `tests/core/brain/`, `tests/mcp/`.+- F5: `src/core/brain/provenance/content-class.ts` (+),+ `src/core/egress/guard.ts`, `src/core/egress/registry.ts`,+ `src/core/brain/write-batch.ts`, `src/core/brain/notes/create-note.ts`,+ `src/core/brain/sessions/import.ts`, `src/core/brain/research/research.ts`,+ `src/core/brain/portability/knowledge-pack.ts`, the CLI export verbs,+ `tests/core/architecture/egress-census.test.ts` (extension),+ `docs/cli-reference.md` (primary owner), `SECURITY.md`.+- F6: `src/core/search/provider-consent.ts` (+),+ `src/core/egress/consent-stamp.ts` (+), `src/core/search/embeddings/provider.ts`,+ `src/core/search/semantic-phase.ts`, `src/core/brain/config-template.ts`,+ `src/core/doctor-readiness.ts`, `src/core/state/surfaces.ts` (only if a durable+ file appears), `tests/core/search/`, `tests/core/`.++## Risks and open questions++- Census coupling is the integration risk: a lane that ships detection without its+ census/registry rows does not build green (doctor-exit census, applier-capability,+ egress census population, state-surface counts, visibility-surface pins). Every+ lane brief carries its census checklist; the integrator recounts.+- F1 behavior change on import-heavy vaults: pages using `access:` for non-privacy+ purposes vanish from remote readers after upgrade. The marker set and value+ matching are pinned by tests; the doctor finding is the operator's discovery path.+- F2 recall drops for held sources (semantic lane skips them, FTS still matches);+ the held list must stay visible on status and doctor or one silence replaces+ another.+- F3 false-positive person matches (initials, shared nicknames): mitigated by+ exact-declared-first ordering, ambiguity-throws-over-guessing, person-folder+ scoping, and the doctor collision finding.+- F5 transcript export has a contract to export session material; the class+ default must refuse-or-name there, never quietly empty.+- The consent decline record must not become a third parallel "off" spelling beside+ the capability tier; the resolver extends the tier ladder rather than duplicating+ it (import-cycle ratchet limits the home).diff --git a/docs/brainstorm/write-honesty-egress-consent/plan.md b/docs/brainstorm/write-honesty-egress-consent/plan.mdnew file mode 100644index 00000000..7c7b0240--- /dev/null+++ b/docs/brainstorm/write-honesty-egress-consent/plan.md@@ -0,0 +1,85 @@+# Write-time honesty and egress consent - implementation plan++Lane structure: six feature lanes (A-F) on disjoint file sets plus one integrator.+Chosen variant: contract-first dependency waves (consultant Variant 2, adapted):+each lane lands its leaf/substrate primitive first, census and registry rows land+with the implementation that needs them, shared files have one primary owner and+append-with-poll for the second writer. TDD is red-first per task: the failing+test, then the fix, then green. Commit per task with `git commit --only <paths>`.+Format only your own files with ./node_modules/.bin/oxfmt; run only your own tests+with `env HOME=$(mktemp -d) bun test <files>`.++Shared-file ownership:+- `src/core/search/indexer.ts`: Lane B owns outright. Lane A coordinates the+ visibility-stamp edit with B before touching it (single coordinated edit, A+ first, B rebases its in-progress work). Lane F never edits it.+- `src/core/brain/diagnostics.ts`: Lane D owns (task D3). Lane A appends its+ doctor signal row in A4 AFTER polling `git log -- src/core/brain/diagnostics.ts`+ for D3's commit (30 s interval, up to 12 min).+- `docs/cli-reference.md`: Lane E owns (E5). Lanes A and B append their CLI doc+ lines in their final tasks after polling for E5's commit.+- Census tests: each lane edits only the census test its feature moves+ (A: visibility-surface if pins move; B: chunk-window + pending-vectors;+ D: none expected; E: egress-census extension; F: state-surface if a durable+ file appears).++## Tasks++### Lane A - F1 privacy markers (t_e095afac)++- **A1: informal privacy marker predicate.** Files: src/core/graph/privacy-markers.ts (new), tests/core/graph/privacy-markers.test.ts. Acceptance: `access: private`, `confidentiality: secret`, `sensitivity: confidential` (string and array forms, NFC/case variants via normToken) resolve to remote-deny; only-formal or keyless pages unchanged; no value read as a caller-scope token.+- **A2: enforce remote-deny at the three read roots and the indexed column.** Files: src/core/search/result-filters.ts (isPathReadableAtReach, applyReachFilter), src/core/vault.ts (listVaultPages), src/core/search/indexer.ts (visibility stamp, coordinated with Lane B), tests/core/search/visibility-filter.test.ts, tests/core/graph/remote-readability.test.ts. Acceptance: a marker-only page is absent from remote listVaultPages and remote search and its indexed snippets stay reserved after rewrite; local reach byte-identical; marker + `visibility: team` still remote-denied.+- **A3: preview-first tighten-only restamp verb.** Files: src/core/brain/privacy-marker-restamp.ts (new), src/cli/brain/verbs/ (new verb registration), tests/core/brain/privacy-marker-restamp.test.ts. Acceptance: default run writes nothing and returns candidates with the would-be visibility line; apply unions the reserved token via writeFrontmatterAtomic behind assertVaultIdentityForWrite; re-run no-op; never weakens existing visibility.+- **A4: doctor finding + registrations.** Files: src/core/brain/doctor/privacy-marker-check.ts (new), src/core/brain/doctor.ts (registry), src/core/brain/applier-capability.ts, src/core/brain/doctor-exits.ts, and ONE appended row in src/core/brain/diagnostics.ts (poll for D3). Tests: doctor-exit census, applier-capability, check unit test. Acceptance: N marker pages yield one warning with count N and paths; zero-marker vaults silent; code resolves through exit tables.+- **A5: census + docs tail.** Files: tests/core/architecture/visibility-surface-census.test.ts (only if pins move), docs/cli-reference.md (poll for E5). Acceptance: census green with named recounts; restamp verb documented.++### Lane B - F2 chunk skip-and-hold (t_f029b413; owns indexer.ts)++- **B1: hold decision in the embed phase.** Files: src/core/search/indexer.ts (runEmbeddingPhase), tests/core/search/ (embed phase). Acceptance: with a declared window and one over-window chunk, the provider receives only fitting texts, vectors stay 1:1 with what was sent, held tally with reason in stats, held content never reaches vecUpsert.+- **B2: persistent hold record + pending exclusion + release-on-change.** Files: src/core/search/schema.ts (migration + test), src/core/search/store/chunks.ts (PENDING_ANTI_JOIN), src/core/search/embedding-spend.ts. Acceptance: held chunk leaves countChunksWithoutEmbeddings (no perpetual vector-backfill recommendation); model/window/signature change releases holds; unchanged config keeps them held without spend.+- **B3: bounded held-items report on the status wire.** Files: src/core/search/indexer.ts (IndexStats), src/core/search/serialize.ts, tests/core/search/chunk-window-census.test.ts (wire discipline). Acceptance: status carries a capped held list (path, chunk index, estimate, window, reason) with an N-more disclosure, presence/absence byte discipline preserved.+- **B4: doctor check for held sources.** Files: new check under src/core/brain/doctor/ (embeddings-health-check.ts pattern), src/cli/brain/verbs/doctor.ts if wiring needed, tests. Acceptance: one aggregated warning per affected source document with a registered next-step command opening the held list.+- **B5: census verdict/warning update + docs tail.** Files: src/core/search/indexer.ts (formatChunkWindowMeasured), src/core/search/types.ts (new code only if warranted), tests/core/search/chunk-window-census.test.ts, tests/core/search/pending-vectors.test.ts, docs/cli-reference.md (poll for E5). Acceptance: warning names held-list and split-or-exclude levers; both verdicts resolve to registered commands; estimator one-resolver invariant intact.++### Lane C - F3 person-name variants (t_46296794)++- **C1: person-variants leaf module.** Files: src/core/brain/entities/person-variants.ts (new, imports nothing), tests. Acceptance: deterministic bit-stable forms (initials, word order) for person-category names with a minimum-form-length floor; unspaced-script names emit no word-order variants; nickname data is operator-extensible input, never a hardcoded English list.+- **C2: query-side expansion with declared-vs-derived tracking.** Files: src/core/search/entity-alias.ts, tests/core/search/entity-alias.test.ts. Acceptance: query "Bob" resolves against a Robert-only registry; derived forms tracked separately from declared; no-person vaults return the identity expansion byte-identically.+- **C3: exact-declared-first ranking tier.** Files: src/core/search/pipeline/candidate-signals.ts, src/core/search/ranker.ts, tests. Acceptance: declared-form match ranks above derived-variant match for otherwise-equal candidates; derived lane strictly smaller and capped; signals omitted when zero; never-retrieval invariant preserved.+- **C4: registry lookup derived lane.** Files: src/core/brain/entities/registry.ts, tests. Acceptance: getEntity resolves a unique derived match after exact miss; multi-match throws EntityAmbiguityError naming both ids; write-target resolution never follows a quarantined namesake's variant.+- **C5: doctor collision finding.** Files: src/core/brain/doctor/entity-checks.ts, tests/core/brain/doctor.entities.test.ts. Acceptance: two persons sharing a derived variant produce exactly one entity-alias-candidate warning naming both ids.++### Lane D - F4 undated-quantity lint (t_fb442a24; owns diagnostics.ts, help-text.ts)++- **D1: recognizer module.** Files: src/core/brain/undated-quantity.ts (new), tests. Acceptance: currency, percent, number-with-unit, thousands-separated triggers fire; ISO dates, version tokens, list markers, table cells, code spans do not; returns lexeme + offset.+- **D2: fifth detector in lintOnePage.** Files: src/core/brain/page-lint.ts, tests/core/brain/page-lint.test.ts. Acceptance: warning findings with the new code, path "body"; escape hatches (valid_from, ISO date in body, [Source: ..., YYYY-MM-DD] marker) produce no finding; clean page keeps the absent lint key; cap/rank discipline intact.+- **D3: diagnostics registration.** Files: src/core/brain/diagnostics.ts, tests. Acceptance: nextCommandField(code) resolves; findings carry next_command. (Lane A polls this commit before appending its row.)+- **D4: receipt-level integration.** Files: tests/mcp/brain-create-note.test.ts + update/append/batch twins. Acceptance: undated numeric write succeeds AND carries lint.findings with the new code; dated write keeps the byte-identical clean receipt.+- **D5: help text.** Files: src/cli/brain/help-text.ts, tests. Acceptance: claims help names the new finding class; no stale line references.++### Lane E - F5 content-class egress (t_5f545a9b; owns egress seam, cli-reference.md)++- **E1: content-class stamp primitive.** Files: src/core/brain/provenance/content-class.ts (new), src/core/brain/write-batch.ts (reserved update key), src/core/brain/notes/create-note.ts (creation merge), tests. Acceptance: unknown values refuse; stamp server-side at creation; callers cannot forge or strip the key on update; spelling avoids origin-channel census spellings.+- **E2: stamp the deterministic prompt-derived write paths.** Files: src/core/brain/sessions/import.ts, the create-note envelope for mcp-tool claims, src/core/brain/research/research.ts and the in-process generation writers, tests. Acceptance: session import and research runs carry the class; operator-authored CLI writes do not; the four pinned writer input interfaces unchanged (origin-channel census green).+- **E3: guard class policy + per-verb override.** Files: src/core/egress/guard.ts, src/core/egress/registry.ts (reasons), the export verbs (bank-export, graph-export, okf-export, export, continuity, explorer, knowledge-pack), tests. Acceptance: default export omits class-marked items and NAMES each omission; override includes them with a notice; transcript export refuses-or-names, never silently empties; egress-census guard-call rules stay green.+- **E4: knowledge-pack block reason + census extension.** Files: src/core/brain/portability/knowledge-pack.ts, tests/core/architecture/egress-census.test.ts (or content-class-census.test.ts new). Acceptance: prompt-derived is a named block reason; census pins in both directions that every sharedRedactor site applies the class filter.+- **E5: operator disclosure + no-backfill invariant.** Files: docs/cli-reference.md, SECURITY.md, tests. Acceptance: default-excluded class and override documented where the census demands; a pre-stamp vault fixture exports unchanged (era boundary asserted).++### Lane F - F6 provider-decline consent (t_fa467133)++- **F1t: consent resolver.** Files: src/core/search/provider-consent.ts (new), src/core/brain/config-template.ts (key declarations), tests. Acceptance: unset and explicit decline resolve to distinct states; unreadable/unparseable config resolves to declined (fail-closed); resolver extends the capability-tier ladder rather than paralleling it; import-cycles green.+- **F2t: honor the decline before any network call.** Files: src/core/search/embeddings/provider.ts (makeProvider typed declined error naming the keys), src/core/search/semantic-phase.ts refusal path, src/core/search/indexer.ts (the embed phase and the embedding probe resolve the consent and hand it to makeProvider - a coordinated edit with F2, shipped as such), tests. Acceptance: declined vault refuses at provider construction with a fetch counter asserting no network call; unset-config vaults behave byte-identically to today.+- **F3t: doctor consent probe.** Files: src/core/doctor-readiness.ts (READINESS_PROBE + DEFAULT_PROBES, split the skipped-decline arm), tests. Acceptance: declined vault reports a decline sentence naming search_semantic_enabled=false / embedding_provider=disabled and lists present provider env vars BY NAME ONLY (tests assert absence of values); unset keeps the existing sentence.+- **F4t: consent-stamp primitive with kill switch.** Files: src/core/egress/consent-stamp.ts (new, no destination declared), tests. Acceptance: kill switch set refuses every stamp regardless of any other env value; refusal discloses what would leave the machine; leaf-level redaction composes via redactForEgress.+- **F5t: durable-surface registration if needed.** Files: src/core/state/surfaces.ts, tests/core/architecture/state-surface-census.test.ts (only if a durable file appears). Acceptance: new row attributed with sources, counts bumped with a written reason, naive cross-check green.++### Integrator (single pass after all lanes)++- Census recount across all moved pins; full local gates in CI order (CI-pinned Bun); cross-lane contract verification (declined error surfaces by name through the indexer's consent-resolved provider construction); remaining docs.++## Dependency notes++- A4 depends on D3 (diagnostics.ts append-with-poll). A5/B5 depend on E5 (cli-reference.md append-with-poll).+- A2's indexer stamp edit is coordinated with Lane B before B1 starts editing the indexer; A performs its indexer edit FIRST, B proceeds after.+- C2, C3, C4, C5 depend on C1. E3, E4 depend on E1. F2t, F3t, F4t depend on F1t.+- No lane waits on another lane's feature completion otherwise; all six start in parallel.diff --git a/docs/brainstorm/write-honesty-egress-consent/variants.md b/docs/brainstorm/write-honesty-egress-consent/variants.mdnew file mode 100644index 00000000..ec6cf33f--- /dev/null+++ b/docs/brainstorm/write-honesty-egress-consent/variants.md@@ -0,0 +1,72 @@+# Variants considered - write-honesty-egress-consent++Consultant: codex gpt-6.1-sol (high reasoning), prompted with the six verified+feature briefs and the repo conventions; output preserved verbatim in+`cli-output/codex.md`, prompt in `cli-output/prompt.md`.++## Variant 1: Feature lanes with queued integration (consultant)++- Approach: one lane per feature, shared files owned by single lanes, other+ lanes' changes applied through a serialized integration queue; census and+ registry updates integrated with each feature; combined architecture checks+ after every merge.+- Trade-offs: accountable per-feature ownership and immediate progress on leaf+ modules; shared-file owners become bottlenecks and policy assumptions can+ surface late.+- Complexity: medium. Risk: high.++## Variant 2: Contract-first dependency waves (consultant)++- Approach: a short substrate wave establishing leaf-module contracts,+ shared-file ownership and the census registration sequence, then six lanes+ (integration/substrate, F1+F5, F2, F3, F4, F6); the integration owner applies+ prepared changes to shared files in dependency order (predicates and+ resolvers, enforcement and persistence, reporting surfaces); census rows land+ with the implementation that needs them; combined acceptance checks at the+ end.+- Trade-offs: explicit shared-file coordination with parallel implementation;+ privacy and content-class grouped for consistent policy handling; census+ updates sequenced with real implementation; needs an initial coordination+ pass and disciplined handoffs; wrong early contracts ripple across lanes.+- Complexity: medium. Risk: low.++## Variant 3: Enforcement spine first, advisory features second (consultant)++- Approach: F6 first, then F1 and F5 as the gated enforcement stage while other+ coders prepare F2/F3/F4 against agreed interfaces; advisory features+ integrate after enforcement passes its checks; per-stage doctor,+ state-surface and registry changes; final combined verification.+- Trade-offs: early review concentrates on disclosure and outbound calls and+ regressions localize easily; imposes ordering between features with no+ technical dependency and leaves coders idle.+- Complexity: large. Risk: medium.++### Recommended: Variant 2 (consultant)++Rationale (consultant): useful parallel work with disjoint file ownership and an+explicit route for shared-file edits; one integration lane coordinating doctor+codes, state surfaces and the egress registry reduces census conflicts while+preserving fail-closed checks; the substrate stays limited to concrete contracts+and leaf primitives.++## Orchestrator decision++Variant 2 adopted, with two adaptations that keep its risk profile while+removing its serialization cost:++1. The substrate "wave" is folded into the lanes themselves: every lane lands+ its leaf/contract primitive as its first task (A1, B1, C1, D1, E1, F1t are+ all pure leaf modules or resolvers), so no separate coordination pass delays+ the start. The cross-lane contracts that would have been "established" up+ front are pinned in plan.md instead: file ownership per lane, the two+ append-with-poll shared files (diagnostics.ts, docs/cli-reference.md), the+ indexer.ts single-owner rule, and the provider-construction decline contract+ that keeps Lane F out of indexer.ts entirely.+2. The consultant's F1+F5 pairing is split back into two lanes (A and E): their+ file sets are disjoint (graph/visibility roots vs egress seam), and the+ policy consistency the consultant wanted is already guaranteed because both+ lanes enforce at their own single boundary (remote-deny at the read roots,+ content class at the shared guard) with their own census rows.++Lane graph and acceptance criteria per task: plan.md. Six lanes start in+parallel; one integrator runs the combined gates.diff --git a/docs/cli-reference.md b/docs/cli-reference.mdindex 193b27f6..b6146d69 100644--- a/docs/cli-reference.md+++ b/docs/cli-reference.md@@ -2457,6 +2457,13 @@ model. Theoffline `local` embedder has no input window, so the key is refused with`INVALID_INPUT` when `embedding_provider` is `local`.+`o2b search status --json` carries `held_chunks` - `{ total, items }`, each+item `{ path, chunk_index, estimated_tokens, window_tokens, reason }`, the+list position-ordered and capped at 20 with `total` naming the whole - only+while the index holds chunks under the ACTIVE embedding configuration; it is+absent when nothing is held, and records whose (model, window, signature)+key a config change released are not reported.+`embedding_extra_body` (`OPEN_SECOND_BRAIN_EMBEDDING_EXTRA_BODY`, sincev1.73.0) sends operator-declared fields with every embedding request to anOpenAI-compatible endpoint, for a serving stack that needs a field the@@ -2817,6 +2824,17 @@ o2b search check reports how many indexed documents rethe field, and how many documents hold no frontmatter the index measured```+```text+o2b brain privacy-marker-restamp [--apply] [--json] - stamp the reserved visibility token onto pages that declare their audience only through an informal privacy marker (access / confidentiality / sensitivity); dry-run default, additive, idempotent, never removes a token, never re-embeds+```++With `--apply`, a candidate whose frontmatter cannot be rewritten losslessly+is REFUSED, not stamped: a page whose frontmatter carries a line the+scanner drops, a comment line, an indented nested mapping, or a file that+cannot be re-read is reported in the verb's `skipped` list (JSON) and on+stderr with the reason, and its bytes stay untouched - `updated N of M`+never silently absorbs a refused page.+A vault page's `visibility:` frontmatter carries opaque, language-neutraltokens, and ONE of them is reserved: a page declaring it is not readableat `remote` reach. Reach is a transport fact rather than a caller@@ -2867,6 +2885,55 @@ and unmeasured for a document the index holds no frontmatter chunk for.`o2b search check` reports the unmeasured population rather than countingit as declaring nothing.+### The content class (since v1.92.0)++```text+o2b brain bank-export --out <file> [--include-prompt-derived]+o2b brain graph-export --out <file> [--include-prompt-derived]+o2b brain okf-export --out <dir> [--include-prompt-derived]+o2b brain export --format transcripts-jsonl --transcripts <file|dir>+ --out <file> [--include-prompt-derived]+```++Pages carry a `content_class:` frontmatter key that the server writes, the+same way the channel a record arrived on is written by the process that+received it and never named by the caller. The vocabulary is closed and+holds one member today: `prompt-derived`, stamped at creation on the write+paths that are definitionally material an agent produced from a prompt -+note writes that arrive over the MCP transport, the session import, and the+in-process report writers. A caller that sends the key has it dropped; an+operator-authored CLI write is never stamped.++Every shared export leaves class-marked pages out by default. The three+page-export verbs above split them out of the payload BEFORE it is+composed, so a marked page is absent from the export rather than redacted+inside it, and each omission is named on stderr with the page's path and+its class, so a smaller export is never a silent one. Passing+`--include-prompt-derived` carries the pages back in; the run says so on+stderr, because an operator who opted in deserves to see the opt-in+acknowledged where the copy is made. `o2b brain export` is the transcript+export, not a page export: it never drops vault pages, and its+`--include-prompt-derived` governs TRANSCRIPT CONVERSATIONS - a record+that carries the class is omitted and named by its `session_id`, the same+way the page verbs name paths (and only the `transcripts-jsonl` format+reads the flag; the other formats ignore it). The knowledge pack goes one+step earlier: a marked page is blocked by the selection step beside the+`visibility` and `owner` blocks and named in the pack's `blocked` list,+since a pack is a curated subset by design and offers no override.+The transcript format has a contract to export session material, so it+never applies the default quietly: a record that carries the class is+omitted and named, and a corpus the policy would empty is a refusal, never+an empty file.++Pages written before this shipped carry no `content_class` key, and+nothing rewrites them to add one - a stamp applied by a later pass would+describe the pass, not the writer. Absence of the mark is the documented+pre-stamp era: such a page is unmarked and keeps exporting exactly as it+did before this release. The vocabulary is closed on the write side and+forgiving on the read side: a hand-edited value this build does not know+reads as unmarked, because inventing an exclusion for it would let one+typo silently hide pages from every shared export.+## What leaves the machine unscanned (network egress)Every export that writes a FILE - `brain export`, `brain explorer --export`,diff --git a/openclaw.plugin.json b/openclaw.plugin.jsonindex 174216e2..3c654445 100644--- a/openclaw.plugin.json+++ b/openclaw.plugin.json@@ -2,7 +2,7 @@"id": "open-second-brain","name": "Open Second Brain","description": "Second brain for AI agents using Obsidian-compatible Markdown vaults.",- "version": "1.91.0",+ "version": "1.92.0","activation": { "onStartup": true },"skills": ["./skills"],"contracts": {diff --git a/openclaw/index.js b/openclaw/index.jsindex 179fdb5e..20a0cdbb 100644--- a/openclaw/index.js+++ b/openclaw/index.js@@ -4752,28 +4752,28 @@ var EGRESS_SITES = Object.freeze({verb: "o2b brain bank-export",module: "src/cli/brain/verbs/bank-export.ts",redaction: R.sharedRedactor,- reason: "the widest export in the tree: preferences, the page graph, page contracts and " + "the sources dashboard composed into one file. Redaction runs over the bundle " + "TREE and the JSON is serialised afterwards, so a note whose text happens to " + "contain a credential assignment cannot mangle the document's own quoting."+ reason: "the widest export in the tree: preferences, the page graph, page contracts and " + "the sources dashboard composed into one file. Redaction runs over the bundle " + "TREE and the JSON is serialised afterwards, so a note whose text happens to " + "contain a credential assignment cannot mangle the document's own quoting. A page " + "whose content_class carries the reserved prompt-derived value is left out of both " + "page sections - graph node and contract - and the omission is named on stderr; " + "--include-prompt-derived carries it back in. The sources dashboard is an aggregate " + "over signal records, carries no page text, and is not class-filtered."},"brain-graph-export": {id: "brain-graph-export",verb: "o2b brain graph-export",module: "src/cli/brain/verbs/graph-export.ts",redaction: R.sharedRedactor,- reason: "carries no page body, but does carry every page title, path and typed-relation " + "target - a credential pasted into a title left through here verbatim."+ reason: "carries no page body, but does carry every page title, path and typed-relation " + "target - a credential pasted into a title left through here verbatim. A page whose " + "content_class carries the reserved prompt-derived value has no node in the graph - " + "title, path and relations go with it - and the omission is named on stderr; " + "--include-prompt-derived carries it back in. Link and relation targets inside a " + "kept node are that page's own declared citations and are not rewritten."},"brain-okf-export": {id: "brain-okf-export",verb: "o2b brain okf-export",module: "src/cli/brain/verbs/okf-export.ts",redaction: R.sharedRedactor,- reason: "the only export that carries page bodies VERBATIM, which is its interchange " + "contract and also why it is the widest leak by volume. The manifest is redacted " + "as a tree and `okf.json` re-serialised from it; the markdown files are redacted " + "as text. A bundle whose secrets were removed is no longer a lossless round-trip " + "of the vault, and the verb says so rather than implying otherwise. What it does NOT " + "do is filter by label: a page carrying `private: true` in its frontmatter exports in " + "full, because the `<private>` region marker is this product's only content-derived " + "privacy primitive and these composers are content composers, not visibility filters."+ reason: "the only export that carries page bodies VERBATIM, which is its interchange " + "contract and also why it is the widest leak by volume. The manifest is redacted " + "as a tree and `okf.json` re-serialised from it; the markdown files are redacted " + "as text. A bundle whose secrets were removed is no longer a lossless round-trip " + "of the vault, and the verb says so rather than implying otherwise. What it does NOT " + "do is filter by the informal privacy labels: a page carrying `private: true` in its " + "frontmatter exports in full, because the `<private>` region marker is this product's " + "only content-derived privacy primitive and these composers are content composers, " + "not visibility filters. What it DOES filter, since the content class shipped, is the " + "server-derived class itself: a page stamped content_class: prompt-derived is absent " + "from the manifest and no file is written for it, every omission is named on stderr, " + "and --include-prompt-derived carries the pages back in."},"brain-export": {id: "brain-export",verb: "o2b brain export",module: "src/cli/brain/verbs/export.ts",redaction: R.sharedRedactor,- reason: "preference principles are free text an agent wrote, and the llms-txt form is " + "meant to be pasted into a foreign prompt - the destination least likely to be " + "read before it is shared. One entry covers all three of the verb's formats " + "because the census keys on the MODULE and the destination is declared once, in " + "the verb: the JSON and transcript forms are redacted as TREES and serialised " + "afterwards, llms-txt as text. What proves each branch scans is a per-handler " + "check in the census, not the guard-call count - counting guards against " + "DESTINATIONS is an inequality, and this module declares one destination against " + "three guards, so a fourth handler that scanned nothing would satisfy it. The " + "transcript form is the widest of the three by far - whole recorded " + "conversations with whichever runtime wrote them, which is where a key pasted " + "into a prompt actually lives. Its records are guarded ONE AT A TIME, so a " + "single oversized turn cannot put a machine's whole corpus past the scan window, " + "and a refusal names the conversation rather than the run - by its basename, " + "except when the basename is itself the secret-shaped identifier, where it is " + "named by runtime and start instant instead. The transcript branch is also the " + "one caller that passes `foreignIdentifiers`, because its `session_id` and " + "`turn_id` were named by the harness that wrote the transcript rather than by " + "this vault, so the guard's vendor-prefix-only rule for identifiers - an " + "argument about ids this build constructs - does not cover them."+ reason: "preference principles are free text an agent wrote, and the llms-txt form is " + "meant to be pasted into a foreign prompt - the destination least likely to be " + "read before it is shared. One entry covers all three of the verb's formats " + "because the census keys on the MODULE and the destination is declared once, in " + "the verb: the JSON and transcript forms are redacted as TREES and serialised " + "afterwards, llms-txt as text. What proves each branch scans is a per-handler " + "check in the census, not the guard-call count - counting guards against " + "DESTINATIONS is an inequality, and this module declares one destination against " + "three guards, so a fourth handler that scanned nothing would satisfy it. The " + "transcript form is the widest of the three by far - whole recorded " + "conversations with whichever runtime wrote them, which is where a key pasted " + "into a prompt actually lives. Its records are guarded ONE AT A TIME, so a " + "single oversized turn cannot put a machine's whole corpus past the scan window, " + "and a refusal names the conversation rather than the run - by its basename, " + "except when the basename is itself the secret-shaped identifier, where it is " + "named by runtime and start instant instead. The transcript branch is also the " + "one caller that passes `foreignIdentifiers`, because its `session_id` and " + "`turn_id` were named by the harness that wrote the transcript rather than by " + "this vault, so the guard's vendor-prefix-only rule for identifiers - an " + "argument about ids this build constructs - does not cover them. The formats " + "carry no page-classed material - the two preference forms serialise preference " + "rows, which never take the content-class stamp, and the transcript corpus is " + "runtime session logs, not vault pages - so the default content-class exclusion " + "has nothing to remove here; the transcript branch still does not apply it " + "SILENTLY: a record that carries the reserved class is omitted and named, and a " + "corpus the policy would empty is a refusal, never an empty file."},"config-export": {id: "config-export",@@ -4787,21 +4787,21 @@ var EGRESS_SITES = Object.freeze({verb: "o2b brain continuity export",module: "src/cli/brain/verbs/continuity.ts",redaction: R.sharedRedactor,- reason: "was declared as covered by its read model, which was wrong in both directions. " + "The upstream call redacts at WRITE time with the redactor's DEFAULT options, so " + "`redactTokens` and `redactUrlCredentials` were both off and a vendor key, a bare " + "high-entropy token and a `user:pass@host` URL all left through here verbatim - " + "the two flags the export boundary exists to turn on. Write-time coverage also " + "says nothing about a record already on disk or appended by another writer, since " + "the read model never re-scans. The verb now scans what it READ, which answers " + "both, and gains the truncation refusal the other five already had."+ reason: "was declared as covered by its read model, which was wrong in both directions. " + "The upstream call redacts at WRITE time with the redactor's DEFAULT options, so " + "`redactTokens` and `redactUrlCredentials` were both off and a vendor key, a bare " + "high-entropy token and a `user:pass@host` URL all left through here verbatim - " + "the two flags the export boundary exists to turn on. Write-time coverage also " + "says nothing about a record already on disk or appended by another writer, since " + "the read model never re-scans. The verb now scans what it READ, which answers " + "both, and gains the truncation refusal the other five already had. Continuity " + "records are structured events, not pages, and never take the content-class " + "stamp, so the default class exclusion removes nothing here."},"brain-explorer-export": {id: "brain-explorer-export",verb: "o2b brain explorer --export",module: "src/cli/brain/verbs/explorer.ts",redaction: R.sharedRedactor,- reason: "a self-contained HTML file with the whole rule graph embedded as JSON: every " + "preference and retired principle verbatim, plus each one's topic, scope and " + "provenance counts. It is the export most likely to be handed to a person rather " + "than a program - it opens in a browser - and it was the one export that never " + "scanned anything. Redaction runs over the graph TREE before the template " + "substitution, so a principle whose text contains a quote cannot disturb the " + "document. The label rule holds here too: a preference is not withheld for carrying " + "a `private` tag, because the `<private>` region marker is the only content-derived " + "privacy primitive in this product."+ reason: "a self-contained HTML file with the whole rule graph embedded as JSON: every " + "preference and retired principle verbatim, plus each one's topic, scope and " + "provenance counts. It is the export most likely to be handed to a person rather " + "than a program - it opens in a browser - and it was the one export that never " + "scanned anything. Redaction runs over the graph TREE before the template " + "substitution, so a principle whose text contains a quote cannot disturb the " + "document. The label rule holds here too: a preference is not withheld for carrying " + "a `private` tag, because the `<private>` region marker is the only content-derived " + "privacy primitive in this product. The content-class default removes nothing here " + "either: preferences and retired principles are operator-curated rows and never " + "take the prompt-derived stamp, which is written by note creation, the session " + "import and the report writers only."},"brain-knowledge-pack-export": {id: "brain-knowledge-pack-export",verb: "o2b brain knowledge-pack export",module: "src/cli/brain/verbs/knowledge-pack.ts",redaction: R.sharedRedactor,- reason: "a selected subset - preference principles and page bodies verbatim - built to be " + "handed to someone else, so it is the one export that also filters by LABEL before " + "it scans: a page declaring `visibility:`, an entry carrying `owner:`, and an " + "unreviewed `OKF Review/` candidate are blocked and named, never written. What is " + "carried is redacted as a tree (OKF manifest, preference rows) and as text (pages) " + "BEFORE the pack is sealed, so the sha256 table hashes the bytes that left and a " + "recipient's integrity check verifies the redacted copy, not the vault. Preference " + "rows leave without their evidence links and rendered body."+ reason: "a selected subset - preference principles and page bodies verbatim - built to be " + "handed to someone else, so it is the one export that also filters by LABEL before " + "it scans: a page declaring `visibility:`, an entry carrying `owner:`, an " + "unreviewed `OKF Review/` candidate, and - since the content class shipped - a page " + "stamped content_class: prompt-derived are blocked and named, never written. What is " + "carried is redacted as a tree (OKF manifest, preference rows) and as text (pages) " + "BEFORE the pack is sealed, so the sha256 table hashes the bytes that left and a " + "recipient's integrity check verifies the redacted copy, not the vault. Preference " + "rows leave without their evidence links and rendered body."},"brain-secret-bundle-export": {id: "brain-secret-bundle-export",@@ -5826,6 +5826,46 @@ function isRemotelyReadable(pageTags, reach) {return !pageTags.includes(REMOTE_DENY_VISIBILITY_TOKEN);}+// src/core/graph/privacy-markers.ts+var PRIVACY_MARKER_KEYS = Object.freeze([+ "access",+ "confidentiality",+ "sensitivity"+]);+var PRIVACY_DENY_MARKER_VALUES = Object.freeze([+ "private",+ "secret",+ "confidential"+]);+function declaredValues(raw) {+ if (typeof raw === "string")+ return [raw];+ if (!Array.isArray(raw))+ return [];+ return raw.filter((v) => typeof v === "string");+}+function hasPrivacyDenyMarker(meta) {+ for (const key of PRIVACY_MARKER_KEYS) {+ for (const value of declaredValues(meta[key])) {+ if (PRIVACY_DENY_MARKER_VALUES.includes(normToken(value)))+ return true;+ }+ }+ return false;+}+function markerDenyTokens(meta) {+ return hasPrivacyDenyMarker(meta) ? Object.freeze([REMOTE_DENY_VISIBILITY_TOKEN]) : Object.freeze([]);+}+function remoteDenyVisibilityTokens(meta) {+ const declared = pageVisibility(meta);+ const out = [...declared];+ for (const token of markerDenyTokens(meta)) {+ if (!out.includes(token))+ out.push(token);+ }+ return Object.freeze(out);+}+// src/core/vault.tsinit_fs_utils();var FRONTMATTER_RE = /^---[^\S\n]*\n([\s\S]*?)\n---\s*\n?/;@@ -5962,7 +6002,7 @@ function listVaultPages(vaultDir, opts) {sink: opts.notices,site: opts.site ?? LIST_VAULT_PAGES_SITE});- const pages = walked.filter((w) => isRemotelyReadable(w.unreadable ? UNMEASURABLE_PAGE_VISIBILITY : pageVisibility(w.page.metadata), opts.reach)).map((w) => w.page);+ const pages = walked.filter((w) => isRemotelyReadable(w.unreadable ? UNMEASURABLE_PAGE_VISIBILITY : remoteDenyVisibilityTokens(w.page.metadata), opts.reach)).map((w) => w.page);pages.sort((a, b) => COLLATOR.compare(a.title.toLowerCase(), b.title.toLowerCase()));return pages;}diff --git a/package.json b/package.jsonindex 0ff988b6..98a1798e 100644--- a/package.json+++ b/package.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.91.0",+ "version": "1.92.0","private": false,"description": "Second brain for AI agents using Obsidian-compatible Markdown vaults. Works with Hermes, Claude Code, Codex, and OpenClaw.","keywords": [diff --git a/plugin.yaml b/plugin.yamlindex 984b05db..99779597 100644--- a/plugin.yaml+++ b/plugin.yaml@@ -1,5 +1,5 @@name: open-second-brain-version: "1.91.0"+version: "1.92.0"description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."author: "Open Second Brain contributors"memory_provider: truediff --git a/plugins/codex/.codex-plugin/plugin.json b/plugins/codex/.codex-plugin/plugin.jsonindex 26f3353d..61dd8e99 100644--- a/plugins/codex/.codex-plugin/plugin.json+++ b/plugins/codex/.codex-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.91.0",+ "version": "1.92.0","description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.","author": {"name": "Open Second Brain contributors",diff --git a/plugins/codex/README.md b/plugins/codex/README.mdindex 4a84c4e1..825f2f82 100644--- a/plugins/codex/README.md+++ b/plugins/codex/README.md@@ -123,7 +123,7 @@ The full router with readiness criteria is [`install.md`](https://github.com/ite## What is new-Open Second Brain 1.91.0 turns one-off promises into first-class memory. A promise made in prose becomes a pledge page under `Brain/pledges/` - title, promise text, due date, the source page it was read from, and evidence quotes the kernel verified against that source when the page was written, refusing by name an empty evidence list, a source page the vault does not hold, and any quote the cited source does not contain. `brain_pledge` carries direct CRUD over the family and `brain_intake_pledges` is the extraction intake, which validates every item before writing any page, anchors counterparty names against canonical entities instead of trusting the caller, and reports a repeat of the same payload as a no-op. The today dashboard gains a pledges section beside obligations and open loops, and two opt-in delivery surfaces serve the moment of need: an `entity_facts` rollup on the context pack that groups what a query's entities already kept, and a counterparty brief on UserPromptSubmit that hands the agent at most three shareable facts about the person a prompt names, once per session, gated on the operator-declared trigger vocabulary. Both delivery surfaces are off by default and byte-identical when off, and the kernel never calls a model. Every release is described in the [CHANGELOG](https://github.com/itechmeat/open-second-brain/blob/main/CHANGELOG.md).+Open Second Brain 1.92.0 makes the write path and the egress path keep the promises the rest of the system already made. Informal privacy markers in frontmatter now resolve to the reserved remote-deny token at every remote read root, and `o2b brain privacy-marker-restamp` previews and then stamps the formal reservation onto pages that only carried the informal spelling. A chunk over the embedding model's declared input window is held out of embedding under a record naming the model, the window and the signature, reported on `search status` and released when its premise moves, instead of being silently truncated. Person entities resolve through initials, word order and operator-supplied nicknames, with declared spellings always outranking derived ones and ambiguity reported rather than guessed. A fifth page-lint detector asks a count or an amount for its date, prompt-driven writes carry a server-stamped content class that shared exports leave out by default, and a provider decline is honored before any network call. Every release is described in the [CHANGELOG](https://github.com/itechmeat/open-second-brain/blob/main/CHANGELOG.md).## Documentationdiff --git a/plugins/codex/SECURITY.md b/plugins/codex/SECURITY.mdindex 8a1bfa3a..c771744c 100644--- a/plugins/codex/SECURITY.md+++ b/plugins/codex/SECURITY.md@@ -28,3 +28,24 @@ network calls only to the embedding, rerank or research endpoints the operatorconfigures. Reports about how those boundaries hold are in scope. Issues inthird-party agent hosts (Claude Code, Codex, Hermes, OpenClaw, opencode) belongwith their maintainers.++## What shared exports carry++Every export that writes a file an operator names runs the shared redactor+over its bytes before they land. Since v1.92.0 the shared exports also apply+a content-class policy: pages whose `content_class` frontmatter carries the+server-stamped value `prompt-derived` - material an agent produced from a+prompt - are left out of the export by default, and each omission is named+on stderr. The per-verb `--include-prompt-derived` flag carries those pages+back in and says so on the same run. The knowledge pack blocks such pages in+its selection step and names them in its `blocked` list.++Pages written before this shipped carry no `content_class` key and are never+rewritten to add one; absence of the mark is the documented pre-stamp era,+and such a page exports exactly as it did before. A hand-edited class value+this build does not recognise reads as unmarked rather than hiding the page.++The full decision record for every export path, scanned and unscanned, is+`src/core/egress/registry.ts`, and the operator-facing summaries are the+"content class" and "network egress" sections of+[`docs/cli-reference.md`](https://github.com/itechmeat/open-second-brain/blob/main/docs/cli-reference.md).diff --git a/plugins/hermes/plugin.yaml b/plugins/hermes/plugin.yamlindex 984b05db..99779597 100644--- a/plugins/hermes/plugin.yaml+++ b/plugins/hermes/plugin.yaml@@ -1,5 +1,5 @@name: open-second-brain-version: "1.91.0"+version: "1.92.0"description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."author: "Open Second Brain contributors"memory_provider: truediff --git a/pyproject.toml b/pyproject.tomlindex 7df9d112..a48e2ae3 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -12,7 +12,7 @@ build-backend = "setuptools.build_meta"# `hermes_agent.memory_providers` group, not a CLI script.[project]name = "open-second-brain"-version = "1.91.0"+version = "1.92.0"description = "Hermes Python shim for Open Second Brain. Most of the project (CLI, MCP server, OpenClaw plugin) is TypeScript on Bun; see package.json."readme = "README.md"requires-python = ">=3.11"diff --git a/src/cli/brain.ts b/src/cli/brain.tsindex b36a4c63..9a85defb 100644--- a/src/cli/brain.ts+++ b/src/cli/brain.ts@@ -78,6 +78,7 @@ import {cmdBrainBacklinks,cmdBrainSemanticsBackfill,cmdBrainAuthoredAtBackfill,+ cmdBrainPrivacyMarkerRestamp,cmdBrainMcpLandscape,cmdBrainMerge,cmdBrainExplorer,@@ -367,6 +368,8 @@ export async function handleBrainSubcommand(argv: ReadonlyArray<string>): Promisreturn await cmdBrainSemanticsBackfill(rest);case "authored-at-backfill":return await cmdBrainAuthoredAtBackfill(rest);+ case "privacy-marker-restamp":+ return await cmdBrainPrivacyMarkerRestamp(rest);case "mcp-landscape":return await cmdBrainMcpLandscape(rest);case "scan-inline":diff --git a/src/cli/brain/help-text.ts b/src/cli/brain/help-text.tsindex a3d18920..1f7c0f76 100644--- a/src/cli/brain/help-text.ts+++ b/src/cli/brain/help-text.ts@@ -124,6 +124,7 @@ Brain verbs (observing memory):backlinks List inbound references to a Brain artifact idsemantics-backfill Preview deterministic typed preference-edge backfill proposalsauthored-at-backfill Backfill authored_at on session signals (dry-run default; --apply to write)+ privacy-marker-restamp Formalize informal privacy markers (dry-run default; --apply to write)mcp-landscape List MCP servers configured across the vault (packages, env names)scan-inline Capture @osb markers from folders listed under notes.read_paths in _brain.yamlscan-citations Promote inline [Source: <name>, YYYY-MM-DD] markers to dated provenance events@@ -346,6 +347,16 @@ export const VERB_HELP: Record<string, string> = {"excludes, minus code fences, and minus Brain/log, which is append-only. The\n" +"receipt names how stale the search index now is and the command that fixes it.\n" +"Distinct from `o2b brain lifecycle`, which annotates a memory and moves no bytes.\n",+ "privacy-marker-restamp":+ "usage: o2b brain privacy-marker-restamp [--apply] [--vault <path>] [--json]\n" ++ "Make an informal privacy marker formal. A page that declares its audience only\n" ++ "through an informal marker (access / confidentiality / sensitivity) is a\n" ++ "candidate; dry-run is the default and writes nothing. --apply unions the\n" ++ "reserved visibility token (`private`) into each candidate's `visibility:`.\n" ++ "A page whose frontmatter cannot be rewritten losslessly is refused by name\n" ++ "(reported in `skipped` with the reason) and left untouched. Additive and\n" ++ "idempotent: no key is removed, no declared token is dropped, and a re-run\n" ++ "writes nothing; it never removes a token.\n","scaffold-stub":"usage: o2b brain scaffold-stub list [--limit <n>] [--vault <path>] [--json]\n" +" or: o2b brain scaffold-stub write <target> [--path <rel>] [--source <rel>]...\n" +@@ -361,7 +372,11 @@ export const VERB_HELP: Record<string, string> = {"valid_from / valid_until relations. Default is current truth; --at <ISO|YYYY-MM-DD>\n" +"gives truth at that instant; --history lists every claim (tombstoned included);\n" +"--replaced <id> follows the supersede chain to the live tip; --contests <id>\n" +- "lists contesting claims; --rebuild rebuilds and persists Brain/claim-graph.json.\n",+ "lists contesting claims; --rebuild rebuilds and persists Brain/claim-graph.json.\n" ++ "The write-time lint reports a count or amount written with no date as the\n" ++ "advisory `undated-quantity` finding; it is silenced by valid_from in\n" ++ "frontmatter, any ISO date in the body, or a [Source: <name>, YYYY-MM-DD]\n" ++ "marker, and --at / --history are how such a claim is placed in time.\n",decision:"usage: o2b brain decision <record|outcome|rate|show|list|compare|similar|history|recall|open|list_open|show_open|resolve|discard> [...] [--vault <path>] [--json]\n" +"Decision-record note family under Brain/decisions/. record --title <t> --chosen <c>\n" +diff --git a/src/cli/brain/verbs/bank-export.ts b/src/cli/brain/verbs/bank-export.tsindex 2eb4c5a6..0f9366db 100644--- a/src/cli/brain/verbs/bank-export.ts+++ b/src/cli/brain/verbs/bank-export.ts@@ -1,31 +1,64 @@import { atomicWriteFileSync } from "../../../core/fs-atomic.ts";import { exportBankBundle } from "../../../core/brain/portability/bundle.ts";+import { isPromptDerivedPage } from "../../../core/brain/provenance/content-class.ts";import {EGRESS_OUTCOME,+ EGRESS_CONTENT_CLASS_INCLUDE_NOTICE,EGRESS_REDACTION_NOTICE,+ contentClassOmissionNotice,+ egressContentClassSplit,redactForEgress,} from "../../../core/egress/guard.ts";import { brainVerbContext, fail, parse } from "../helpers.ts";/**- * `o2b brain bank-export [--out <file>]` - serialise a whole-vault bank- * bundle (preferences + page graph + page contracts + sources dashboard)- * to a schema-versioned JSON. Prints to stdout, or writes to- * `--out <file>`. Read-only on the vault.+ * `o2b brain bank-export [--out <file>] [--include-prompt-derived]` -+ * serialise a whole-vault bank bundle (preferences + page graph + page+ * contracts + sources dashboard) to a schema-versioned JSON. Prints to+ * stdout, or writes to `--out <file>`. Read-only on the vault.** Every byte goes through the shared egress guard first (registry entry* `brain-bank-export`). The guard runs over the bundle TREE and the JSON* is serialised afterwards, so redaction can never disturb the* document's own quoting.+ *+ * A page whose `content_class` frontmatter carries the reserved+ * `prompt-derived` value is left out of BOTH page sections - its graph+ * node and its page contract - and every omission is named once on+ * stderr. `--include-prompt-derived` is the operator override; it+ * carries the pages back in and says so. The sources dashboard is an+ * aggregate over signal records, not pages, and carries no page text, so+ * the class policy does not reach it.*/export async function cmdBrainBankExport(argv: string[]): Promise<number> {- const { flags } = parse(argv, { vault: { type: "string" }, out: { type: "string" } });+ const { flags } = parse(argv, {+ vault: { type: "string" },+ out: { type: "string" },+ "include-prompt-derived": { type: "boolean" },+ });const { vault } = brainVerbContext(flags);+ const classPolicy = { contentClass: { include: flags["include-prompt-derived"] === true } };let json: string;let redacted: boolean;+ let omitted: ReadonlyArray<string> = [];try {- const verdict = redactForEgress("brain-bank-export", exportBankBundle(vault));+ const bundle = exportBankBundle(vault);+ const nodeSplit = egressContentClassSplit(bundle.graph.nodes, classPolicy, {+ name: (node) => node.path,+ isPromptDerived: (node) => isPromptDerivedPage(vault, node.path),+ });+ const pageSplit = egressContentClassSplit(bundle.pages, classPolicy, {+ name: (page) => page.path,+ isPromptDerived: (page) => isPromptDerivedPage(vault, page.path),+ });+ // One page can appear in both sections; an omission is named once.+ omitted = [...new Set([...nodeSplit.omitted, ...pageSplit.omitted])];+ const verdict = redactForEgress("brain-bank-export", {+ ...bundle,+ graph: { ...bundle.graph, nodes: nodeSplit.kept },+ pages: pageSplit.kept,+ });if (verdict.outcome !== EGRESS_OUTCOME.released) return fail(verdict.detail);json = JSON.stringify(verdict.payload, null, 2) + "\n";redacted = verdict.redacted;@@ -43,6 +76,11 @@ export async function cmdBrainBankExport(argv: string[]): Promise<number> {} catch (exc) {return fail(`bank-export failed to write output: ${(exc as Error).message ?? exc}`);}+ if (flags["include-prompt-derived"] === true) {+ process.stderr.write(EGRESS_CONTENT_CLASS_INCLUDE_NOTICE);+ } else if (omitted.length > 0) {+ process.stderr.write(contentClassOmissionNotice(omitted));+ }if (redacted) process.stderr.write(EGRESS_REDACTION_NOTICE);return 0;}diff --git a/src/cli/brain/verbs/export.ts b/src/cli/brain/verbs/export.tsindex 9cc1e6cd..79605bdf 100644--- a/src/cli/brain/verbs/export.ts+++ b/src/cli/brain/verbs/export.ts@@ -19,10 +19,13 @@ import {} from "../../../core/brain/export-transcripts.ts";import {EGRESS_OUTCOME,+ EGRESS_CONTENT_CLASS_INCLUDE_NOTICE,EGRESS_REDACTION_NOTICE,+ contentClassOmissionNotice,redactForEgress,type EgressRefused,} from "../../../core/egress/guard.ts";+import { treeCarriesPromptDerived } from "../../../core/brain/provenance/content-class.ts";import {brainVerbContext,fail,@@ -197,6 +200,7 @@ async function exportTranscripts(flags: BrainVerbFlags): Promise<FormatOutcome>const window = windowOf(flags);if (typeof window === "string") return { kind: "usage", detail: window };+ const includePromptDerived = flags["include-prompt-derived"] === true;const options: TranscriptExportOptions = {source,...window,@@ -209,9 +213,13 @@ async function exportTranscripts(flags: BrainVerbFlags): Promise<FormatOutcome>const handle = openSync(spool, "wx");let outcome: SpoolOutcome;try {- outcome = await spoolTranscriptCorpus(streamTranscriptConversations(options), (line) => {- writeSync(handle, line);- });+ outcome = await spoolTranscriptCorpus(+ streamTranscriptConversations(options),+ (line) => {+ writeSync(handle, line);+ },+ { includePromptDerived },+ );} finally {closeSync(handle);}@@ -221,6 +229,12 @@ async function exportTranscripts(flags: BrainVerbFlags): Promise<FormatOutcome>unlinkSync(spool);return { kind: "refused", detail: outcome.detail };}+ if (outcome.promptDerivedOmitted.length > 0) {+ process.stderr.write(contentClassOmissionNotice(outcome.promptDerivedOmitted));+ }+ if (includePromptDerived && outcome.written > 0) {+ process.stderr.write(EGRESS_CONTENT_CLASS_INCLUDE_NOTICE);+ }if (outcome.written === 0) reportEmptyCorpus(outcome.summary);return { kind: "spooled", spool, redacted: outcome.redacted };}@@ -232,9 +246,29 @@ export type SpoolOutcome =readonly written: number;readonly redacted: boolean;readonly summary: TranscriptExportSummary;+ /**+ * The conversations the default content-class policy left out,+ * named for the stderr disclosure. Empty under the override and+ * when nothing carried the class.+ */+ readonly promptDerivedOmitted: ReadonlyArray<string>;}| { readonly kind: "refused"; readonly detail: string };+/** Options the spooling loop takes from the verb's flags. */+export interface SpoolTranscriptOptions {+ /**+ * The per-verb `--include-prompt-derived` override. Default (false):+ * a conversation whose record carries the reserved `prompt-derived`+ * content class anywhere in its tree is left out of the corpus and+ * named - the transcript format has a contract to export session+ * material, so it never applies the class default SILENTLY: each+ * omission is disclosed, and a corpus the policy would empty entirely+ * is a refusal, never an empty file.+ */+ readonly includePromptDerived?: boolean;+}+/*** Guard each conversation and hand the released one straight to `write`.*@@ -261,15 +295,42 @@ export type SpoolOutcome =export async function spoolTranscriptCorpus(records: AsyncGenerator<TranscriptConversation, TranscriptExportSummary>,write: (line: string) => void,+ opts: SpoolTranscriptOptions = {},): Promise<SpoolOutcome> {let written = 0;let redacted = false;+ const promptDerivedOmitted: string[] = [];for (;;) {// The corpus is read one conversation at a time, and the summary rides// out as the generator's return value - `for await` would discard it.// oxlint-disable-next-line no-await-in-loopconst step = await records.next();- if (step.done === true) return { kind: "written", written, redacted, summary: step.value };+ if (step.done === true) {+ // A corpus the class policy would empty entirely is a REFUSAL, not+ // an empty file: the transcript format exists to export session+ // material, and an empty export under exit 0 reads as a machine+ // that recorded nothing. The refusal names what was held back.+ if (written === 0 && promptDerivedOmitted.length > 0) {+ return {+ kind: "refused",+ detail:+ `every conversation in the corpus carries content_class: prompt-derived, and the ` ++ `transcript export leaves such records out by default (${promptDerivedOmitted.length} ` ++ "conversation(s): " ++ `${promptDerivedOmitted.join(", ")}). Re-run with --include-prompt-derived to export them.`,+ };+ }+ return { kind: "written", written, redacted, summary: step.value, promptDerivedOmitted };+ }+ // The content-class tripwire (F5): a record carrying the reserved+ // class anywhere in its tree is left out and named under the default+ // policy. No record carries one today; this is the seam that keeps+ // the day one does from emptying the corpus quietly.+ if (opts.includePromptDerived !== true && treeCarriesPromptDerived(step.value)) {+ promptDerivedOmitted.push(step.value.session_id);+ // oxlint-disable-next-line no-await-in-loop+ continue;+ }// `foreignIdentifiers`: this record's `session_id` and `turn_id` were// named by whichever harness wrote the transcript, so the guard's// default "an id is a long mixed run by construction" narrowing - an@@ -327,6 +388,7 @@ export async function cmdBrainExport(argv: string[]): Promise<number> {until: { type: "string" },out: { type: "string" },force: { type: "boolean" },+ "include-prompt-derived": { type: "boolean" },});const format = flags["format"];if (!isExportFormat(format)) {diff --git a/src/cli/brain/verbs/graph-export.ts b/src/cli/brain/verbs/graph-export.tsindex e7296554..9d709c89 100644--- a/src/cli/brain/verbs/graph-export.ts+++ b/src/cli/brain/verbs/graph-export.ts@@ -1,29 +1,55 @@import { atomicWriteFileSync } from "../../../core/fs-atomic.ts";import { exportVaultGraph } from "../../../core/brain/portability/graph.ts";+import { isPromptDerivedPage } from "../../../core/brain/provenance/content-class.ts";import {EGRESS_OUTCOME,+ EGRESS_CONTENT_CLASS_INCLUDE_NOTICE,EGRESS_REDACTION_NOTICE,+ contentClassOmissionNotice,+ egressContentClassSplit,redactForEgress,} from "../../../core/egress/guard.ts";import { brainVerbContext, fail, parse } from "../helpers.ts";/**- * `o2b brain graph-export [--out <file>]` - serialise the user's vault- * pages (links + typed relations) to a stable graph.json. Prints to- * stdout, or writes to `--out <file>`. Read-only on the vault.+ * `o2b brain graph-export [--out <file>] [--include-prompt-derived]` -+ * serialise the user's vault pages (links + typed relations) to a stable+ * graph.json. Prints to stdout, or writes to `--out <file>`. Read-only on+ * the vault.** The graph carries no page body, but it does carry every title, path* and relation target, so it goes through the shared egress guard* (registry entry `brain-graph-export`) like every other export.+ *+ * A page whose `content_class` frontmatter carries the reserved+ * `prompt-derived` value is left out of the graph: its node - title,+ * path, links and typed relations - is absent from the JSON, and the+ * omission is named on stderr. `--include-prompt-derived` is the operator+ * override; it carries the nodes back in and says so. Links and relation+ * targets INSIDE a kept node are that page's own content and are not+ * rewritten: the graph serialises what the kept pages declare, it does+ * not editorialise their citations.*/export async function cmdBrainGraphExport(argv: string[]): Promise<number> {- const { flags } = parse(argv, { vault: { type: "string" }, out: { type: "string" } });+ const { flags } = parse(argv, {+ vault: { type: "string" },+ out: { type: "string" },+ "include-prompt-derived": { type: "boolean" },+ });const { vault } = brainVerbContext(flags);+ const classPolicy = { contentClass: { include: flags["include-prompt-derived"] === true } };let json: string;let redacted: boolean;+ let omitted: ReadonlyArray<string> = [];try {- const verdict = redactForEgress("brain-graph-export", exportVaultGraph(vault));+ const graph = exportVaultGraph(vault);+ const split = egressContentClassSplit(graph.nodes, classPolicy, {+ name: (node) => node.path,+ isPromptDerived: (node) => isPromptDerivedPage(vault, node.path),+ });+ omitted = split.omitted;+ const verdict = redactForEgress("brain-graph-export", { ...graph, nodes: split.kept });if (verdict.outcome !== EGRESS_OUTCOME.released) return fail(verdict.detail);json = JSON.stringify(verdict.payload, null, 2) + "\n";redacted = verdict.redacted;@@ -41,6 +67,11 @@ export async function cmdBrainGraphExport(argv: string[]): Promise<number> {} catch (exc) {return fail(`graph-export failed to write output: ${(exc as Error).message ?? exc}`);}+ if (flags["include-prompt-derived"] === true) {+ process.stderr.write(EGRESS_CONTENT_CLASS_INCLUDE_NOTICE);+ } else if (omitted.length > 0) {+ process.stderr.write(contentClassOmissionNotice(omitted));+ }if (redacted) process.stderr.write(EGRESS_REDACTION_NOTICE);return 0;}diff --git a/src/cli/brain/verbs/index.ts b/src/cli/brain/verbs/index.tsindex 5df60b9a..720d7c7a 100644--- a/src/cli/brain/verbs/index.ts+++ b/src/cli/brain/verbs/index.ts@@ -61,6 +61,7 @@ export { cmdBrainBankImport } from "./bank-import.ts";export { cmdBrainBacklinks } from "./backlinks.ts";export { cmdBrainSemanticsBackfill } from "./semantics-backfill.ts";export { cmdBrainAuthoredAtBackfill } from "./authored-at-backfill.ts";+export { cmdBrainPrivacyMarkerRestamp } from "./privacy-marker-restamp.ts";export { cmdBrainMcpLandscape } from "./mcp-landscape.ts";export { cmdBrainMerge } from "./merge.ts";export { cmdBrainExplorer } from "./explorer.ts";diff --git a/src/cli/brain/verbs/okf-export.ts b/src/cli/brain/verbs/okf-export.tsindex 80488727..b4ddb54a 100644--- a/src/cli/brain/verbs/okf-export.ts+++ b/src/cli/brain/verbs/okf-export.ts@@ -1,22 +1,30 @@import {- buildOkfBundle,+ buildOkfBundleFromPages,+ collectOkfPages,+ OKF_INDEX_FILENAME,OKF_MANIFEST_FILENAME,+ renderIndex,renderOkfManifest,writeOkfBundle,type OkfBundle,type OkfBundleFile,} from "../../../core/brain/portability/okf.ts";+import { readContentClass } from "../../../core/brain/provenance/content-class.ts";import {EGRESS_OUTCOME,+ EGRESS_CONTENT_CLASS_INCLUDE_NOTICE,EGRESS_REDACTION_NOTICE,+ contentClassOmissionNotice,+ egressContentClassSplit,redactForEgress,} from "../../../core/egress/guard.ts";import { brainVerbContext, fail, parse } from "../helpers.ts";/**- * `o2b brain okf-export --out <dir> [--force]` — write a portable Open- * Knowledge Format bundle (concepts / queries / references + date-grouped- * `log.md` + `okf.json` manifest) to a directory. Read-only on the vault.+ * `o2b brain okf-export --out <dir> [--force] [--include-prompt-derived]`+ * — write a portable Open Knowledge Format bundle (concepts / queries /+ * references + date-grouped `log.md` + `okf.json` manifest) to a+ * directory. Read-only on the vault.** This is the only export that carries page bodies verbatim, so it is the* widest leak by volume and the reason the shared egress guard exists@@ -26,6 +34,25 @@ import { brainVerbContext, fail, parse } from "../helpers.ts";* over serialised JSON can consume a closing quote; the markdown files* are scanned as text.*+ * Pages whose `content_class` frontmatter carries the reserved+ * `prompt-derived` value are left out of the bundle BEFORE it is composed+ * - they are absent from the manifest and no file is written for them -+ * and every omission is named on stderr. `--include-prompt-derived` is+ * the operator override; it carries the pages back in and says so. The+ * bundle is composed FROM the kept pages rather than by re-walking the+ * vault, so the walk that fed the split is the walk the bundle is built+ * from: a page appearing or changing between two walks could otherwise+ * re-admit what the policy just left out.+ *+ * The reserved bundle-root files ride every bundle: `index.md` is+ * re-rendered from the REDACTED manifest by the same renderer the+ * builder uses - the pre-redaction index named the excluded pages, and+ * an index that lists pages or values the shipped `okf.json` does not+ * carry would make the bundle disagree with itself - and `log.md` is+ * kept whole, because its entries are day-keyed events (a timestamp, a+ * kind, body bullets), not page content; it still rides the same+ * redaction scan as the page files.+ ** A redacted bundle is no longer a lossless round-trip of the vault. The* verb says so on stderr rather than leaving the interchange contract to* imply otherwise.@@ -35,18 +62,39 @@ export async function cmdBrainOkfExport(argv: string[]): Promise<number> {vault: { type: "string" },out: { type: "string" },force: { type: "boolean" },+ "include-prompt-derived": { type: "boolean" },});const { vault } = brainVerbContext(flags);const out = flags["out"] as string | undefined;if (out === undefined) {return fail("--out <dir> is required for okf-export");}+ const classPolicy = { contentClass: { include: flags["include-prompt-derived"] === true } };try {- const built = buildOkfBundle(vault);+ // The split runs over the pages the walk collected, and the bundle+ // below is composed FROM the kept pages: a marked page is absent+ // from the manifest and its bundle file is never composed - not+ // redacted after the fact. Composing from `split.kept` (the+ // `buildOkfBundleFromPages` entry point) is also the honest choice+ // about the walk: the alternative re-walk in `buildOkfBundle` could+ // disagree with the very split that decided what may leave.+ const split = egressContentClassSplit(collectOkfPages(vault), classPolicy, {+ name: (page) => page.path,+ isPromptDerived: (page) => readContentClass(page.frontmatter) !== null,+ });++ const built = buildOkfBundleFromPages(vault, split.kept);const verdict = redactForEgress("brain-okf-export", {manifest: built.manifest,- files: built.files.filter((file) => file.path !== OKF_MANIFEST_FILENAME),+ // The two reserved files rendered FROM the manifest are re-rendered+ // after redaction below, so only the per-page files and the log are+ // scanned as text here. The log stays whole - its entries name days+ // and events, not page bodies - but it is scanned like any other+ // markdown the bundle carries.+ files: built.files.filter(+ (file) => file.path !== OKF_MANIFEST_FILENAME && file.path !== OKF_INDEX_FILENAME,+ ),});if (verdict.outcome !== EGRESS_OUTCOME.released) return fail(verdict.detail);@@ -55,6 +103,10 @@ export async function cmdBrainOkfExport(argv: string[]): Promise<number> {path: OKF_MANIFEST_FILENAME,contents: renderOkfManifest(verdict.payload.manifest),},+ // Same renderer the builder uses, redacted manifest in: the index+ // can never list a page the manifest left out (the pre-redaction+ // index did) nor name a value the guard replaced.+ { path: OKF_INDEX_FILENAME, contents: renderIndex(verdict.payload.manifest) },...verdict.payload.files,];files.sort((a, b) => (a.path < b.path ? -1 : a.path > b.path ? 1 : 0));@@ -65,6 +117,11 @@ export async function cmdBrainOkfExport(argv: string[]): Promise<number> {`wrote OKF bundle to ${out} (${bundle.manifest.pages.length} page(s), ` +`${bundle.manifest.log_days} log day(s))\n`,);+ if (flags["include-prompt-derived"] === true) {+ process.stderr.write(EGRESS_CONTENT_CLASS_INCLUDE_NOTICE);+ } else if (split.omitted.length > 0) {+ process.stderr.write(contentClassOmissionNotice(split.omitted));+ }if (verdict.redacted) process.stderr.write(EGRESS_REDACTION_NOTICE);} catch (exc) {return fail(`okf-export failed: ${(exc as Error).message ?? exc}`);diff --git a/src/cli/brain/verbs/privacy-marker-restamp.ts b/src/cli/brain/verbs/privacy-marker-restamp.tsnew file mode 100644index 00000000..e92ecdea--- /dev/null+++ b/src/cli/brain/verbs/privacy-marker-restamp.ts@@ -0,0 +1,57 @@+import { planPrivacyMarkerRestamp } from "../../../core/brain/privacy-marker-restamp.ts";+import { brainVerbContext, info, ok, okJson, parse } from "../helpers.ts";++/**+ * `o2b brain privacy-marker-restamp` (write-honesty, F1).+ *+ * Make an informal privacy marker (`access: private`, `confidentiality:+ * secret`, `sensitivity: confidential`) formal by unioning the reserved+ * token into `visibility:`. Dry-run by DEFAULT (report only); `--apply`+ * writes. Additive only, idempotent, and never re-embeds - it edits+ * frontmatter and nothing else.+ */+export async function cmdBrainPrivacyMarkerRestamp(argv: string[]): Promise<number> {+ const { flags } = parse(argv, {+ vault: { type: "string" },+ apply: { type: "boolean" },+ json: { type: "boolean" },+ });+ const { vault } = brainVerbContext(flags);+ const apply = flags["apply"] === true;++ const result = planPrivacyMarkerRestamp(vault, { apply });++ if (flags["json"]) {+ okJson({+ dry_run: !apply,+ scanned: result.scanned,+ updated: result.updated,+ // The refusals travel with the result: a candidate the apply pass+ // refused (a frontmatter the writer cannot round-trip, a page that+ // could not be re-read) is reported by name with its reason, never+ // folded into a lower `updated` count.+ skipped: result.skipped.map((s) => ({ path: s.path, reason: s.reason })),+ candidates: result.candidates.map((c) => ({+ path: c.path,+ visibility: [...c.visibility],+ visibility_line: c.visibilityLine,+ })),+ });+ return 0;+ }++ if (apply) {+ ok(+ `privacy-marker-restamp: updated ${result.updated} of ${result.candidates.length} candidate(s)`,+ );+ for (const s of result.skipped) process.stderr.write(` refused ${s.path}: ${s.reason}\n`);+ } else {+ ok(+ `privacy-marker-restamp dry-run: ${result.candidates.length} candidate(s) ` ++ `(scanned ${result.scanned})`,+ );+ info("re-run with --apply to union the reserved token into visibility:");+ }+ for (const c of result.candidates) info(` ${c.path}: ${c.visibilityLine}`);+ return 0;+}diff --git a/src/cli/command-manifest.ts b/src/cli/command-manifest.tsindex 625cb67d..52aad2b4 100644--- a/src/cli/command-manifest.ts+++ b/src/cli/command-manifest.ts@@ -714,6 +714,11 @@ export const CLI_COMMAND_MANIFEST: CliRootManifest = Object.freeze({"authored-at-backfill","Backfill authored_at on session-imported signals (dry-run by default)",),+ command(+ "privacy-marker-restamp",+ "Restamp informal privacy markers as visibility: private (dry-run by default)",+ [flag("vault", "string"), flag("apply", "boolean")],+ ),command("distill","Condense a source into agent-supplied atomic claims with block provenance and checked quotes",diff --git a/src/core/bench/embedding-cache.ts b/src/core/bench/embedding-cache.tsindex 9c9c0eb3..c5322da9 100644--- a/src/core/bench/embedding-cache.ts+++ b/src/core/bench/embedding-cache.ts@@ -178,7 +178,10 @@ function writeVectorCache(runsDir: string, entries: Record<string, ReadonlyArray/** The stored-model spelling, matching the indexer's embedding phase. */function storedEmbeddingModel(config: ResolvedSearchConfig): string | null {- return config.semantic.model ?? makeProvider(config.semantic).model;+ return (+ config.semantic.model ??+ makeProvider({ ...config.semantic, configPath: config.configPath }).model+ );}/** Same digest formula the indexer records alongside every vector. */@@ -211,7 +214,7 @@ export async function embedBenchVaultIndex(opts: {"bench embedding: no embedding model is stored on the index; the capability path needs an indexed model to key vectors",);}- const provider = makeProvider(config.semantic);+ const provider = makeProvider({ ...config.semantic, configPath: config.configPath });const signature = benchEmbeddingSignature(config.semantic.provider,model,diff --git a/src/core/brain/applier-capability.ts b/src/core/brain/applier-capability.tsindex b5734519..e97aa5f3 100644--- a/src/core/brain/applier-capability.ts+++ b/src/core/brain/applier-capability.ts@@ -11,6 +11,13 @@* MECHANICAL (naming the fixer that performs the repair) or REFUSED* (naming the reason it is not attempted).*+ * One surface is a verb rather than one of those four: the preview-first+ * privacy-marker restamp is run by an operator against a named vault, so+ * its MECHANICAL entry names that module's planner instead of an applier+ * that routes a Finding. The verdict answers the same question - a+ * deterministic, idempotent repair exists - and the separate exits table+ * still says why the doctor names no single command for it.+ ** A refusal is a deliverable here. Every entry below that declines a* repair states why in a non-empty reason, because a finding whose exit* is "nothing happens" is exactly the dead end this release removes.@@ -84,6 +91,10 @@ export const REPAIR_SURFACE = Object.freeze({healthRemediation: "health/remediation",hygieneApply: "hygiene/apply",lintConsolidate: "lint-consolidate",+ // A repair pipeline that is a verb rather than one of the three+ // appliers: the preview-first restamp rewrites frontmatter and is+ // invoked by an operator, not routed from a Finding.+ privacyMarkerRestamp: "privacy-marker-restamp",detectorOnly: "detector-only",} as const);@@ -260,6 +271,14 @@ const ENTRIES: ReadonlyArray<RepairCapability> = [fixer: "lint-consolidate.applyDemotion",},+ // --- privacy-marker-restamp: the operator-run additive repair ------------+ {+ code: "privacy-marker-unstamped",+ surface: REPAIR_SURFACE.privacyMarkerRestamp,+ verdict: REPAIR_VERDICT.mechanical,+ fixer: "privacy-marker-restamp.planPrivacyMarkerRestamp",+ },+// --- detector-only: reported, deliberately never repaired ---------------{code: "stale-lock",@@ -296,6 +315,16 @@ const ENTRIES: ReadonlyArray<RepairCapability> = ["the ratchet measurement yields scalar counts over the search index and no " +"per-link rows, so there is no individual finding for an applier to act on",},+ {+ code: "chunk-holds-active",+ surface: REPAIR_SURFACE.detectorOnly,+ verdict: REPAIR_VERDICT.refused,+ reason:+ "a hold is the embedding configuration declining a chunk it provably cannot " ++ "take, not damage to repair; the levers - narrower chunking or leaving the " ++ "page unembedded - are per-document editorial decisions, and the index-side " ++ "release is automatic the moment the configuration changes",+ },];/**diff --git a/src/core/brain/belief-semantic.ts b/src/core/brain/belief-semantic.tsindex 0c21706f..d8defb9c 100644--- a/src/core/brain/belief-semantic.ts+++ b/src/core/brain/belief-semantic.ts@@ -268,7 +268,16 @@ export async function loadBeliefSemanticRelevance(const readable = isRemotelyReadable(indexed.get(path) ?? [], reach);vectorsByPath.set(path, readable ? store.storedEmbeddingsForDocument(doc.id) : []);}- const provider = deps.provider ?? makeProvider(config.semantic);+ // The consent travels with the configuration that would spend it:+ // the provider is built under the decline record of the SAME file+ // (or env, when there is none) this semantic configuration was+ // resolved from.+ const provider =+ deps.provider ??+ makeProvider({+ ...config.semantic,+ configPath: config.configPath,+ });const model = config.semantic.model ?? provider.model;// No belief row under the model anywhere means none in reach has one:// the pack's refusal is certain, so it is raised before the paid embed.diff --git a/src/core/brain/config-template.ts b/src/core/brain/config-template.tsindex 4bcbfb0f..94f226bd 100644--- a/src/core/brain/config-template.ts+++ b/src/core/brain/config-template.ts@@ -748,6 +748,37 @@ export const BRAIN_CONFIG_TEMPLATE_OMISSIONS: ReadonlyArray<BrainTemplateOmissio"Templating an example would invite copying folder names that do not exist; " +"the block and its contract example are documented in docs/cli-reference.md.",},+ {+ // The provider-consent record (write-honesty-egress-consent, Lane F) is+ // a MACHINE-configuration surface, and this is where that surface is+ // declared rather than in the template. The record has two spellings,+ // `search_semantic_enabled=false` and `embedding_provider=disabled`,+ // and both live in the operator's `config.yaml`; the resolver that+ // reads them is `src/core/search/provider-consent.ts`, which answers+ // UNSET apart from DECLINED and fails closed on a record it cannot+ // read.+ //+ // It is NOT a vault-configuration key, and the reason is the layer,+ // not a preference: this file travels between machines through+ // Syncthing, so a key here would carry one machine's provider+ // decision to every other one - and a key that a vault declares and+ // the resolver never reads is a knob that does nothing, which is the+ // failure the omission list exists to prevent. The vault tier of the+ // same idea is the `decision_model` block above, which can only+ // NARROW; the provider decision is the operator's, and it stays on the+ // machine that made it.+ key: "provider_consent",+ reason:+ "The provider-consent record is a machine-configuration key, not a " ++ "vault-configuration one: an explicit decline is written as " ++ "`search_semantic_enabled=false` or `embedding_provider=disabled` in the " ++ "operator's `config.yaml`. `_brain.yaml` travels between machines through " ++ "Syncthing, so templating the record here would carry one machine's " ++ "provider decision to every other one - and a vault key the resolver " ++ "never reads would advertise a knob that does nothing. The record is " ++ "resolved by `src/core/search/provider-consent.ts` and disclosed by the " ++ "readiness probe that names the provider environment variables.",+ },]) as ReadonlyArray<BrainTemplateOmission>;export interface RenderBrainConfigTemplateOptions {diff --git a/src/core/brain/counterparty-brief.ts b/src/core/brain/counterparty-brief.tsindex 7db43a89..390abf27 100644--- a/src/core/brain/counterparty-brief.ts+++ b/src/core/brain/counterparty-brief.ts@@ -76,11 +76,8 @@ import {type PledgeListItem,} from "./pledges.ts";import { neutralizeSingleLine, untrustedFramingHeader } from "./untrusted-source.ts";-import {- REMOTE_DENY_VISIBILITY_TOKEN,- pageVisibility,- strictestVisibilityOf,-} from "../graph/visibility.ts";+import { REMOTE_DENY_VISIBILITY_TOKEN, strictestVisibilityOf } from "../graph/visibility.ts";+import { remoteDenyVisibilityTokens } from "../graph/privacy-markers.ts";import {DEGRADATION_CODE,emitDegradationNotice,@@ -324,7 +321,12 @@ function entityCarriesReservedToken(entity: BrainEntity): boolean {}const [meta, , notices] = parseFrontmatterTextWithNotices(text);if (notices.length > 0) return true;- return pageVisibility(meta).includes(REMOTE_DENY_VISIBILITY_TOKEN);+ // The full union (declared tokens plus any informal marker's+ // reservation), so a page stating `access: private` under the informal+ // key is reserved exactly as a formally declared one would be - the+ // marker is the same claim in an older spelling, and the brief leaves+ // the machine either way.+ return remoteDenyVisibilityTokens(meta).includes(REMOTE_DENY_VISIBILITY_TOKEN);}/**@@ -437,7 +439,12 @@ function pledgeItem(// be shareable, so the item is withheld - the same discipline the// cited-source gate applies.if (pledgeNotices.length > 0) return null;- const tagLists: ReadonlyArray<string>[] = [[...pageVisibility(meta)]];+ // Each tag list is the page's declared tokens plus any informal marker's+ // reservation (`remoteDenyVisibilityTokens`), so a page saying+ // `access: private` under the informal key folds exactly as a formally+ // reserved one would - the brief leaves the machine, and the marker is+ // the same claim in an older spelling.+ const tagLists: ReadonlyArray<string>[] = [[...remoteDenyVisibilityTokens(meta)]];let sourceRel = pledgeRel(pledge.slug);if (pledge.source.length > 0) {const source = pledgeSourceWithinVault(pledge.source);@@ -451,7 +458,7 @@ function pledgeItem(if (!isRegularFile(sourceAbs)) return null;const [sourceMeta, , sourceNotices] = parseFrontmatterWithNotices(sourceAbs);if (sourceNotices.length > 0) return null;- tagLists.push([...pageVisibility(sourceMeta)]);+ tagLists.push([...remoteDenyVisibilityTokens(sourceMeta)]);sourceRel = source;}const folded = strictestVisibilityOf(tagLists);@@ -538,7 +545,10 @@ function preferenceItems(}const [meta, body, pageNotices] = parseFrontmatterTextWithNotices(text);if (pageNotices.length > 0) continue;- if (pageVisibility(meta).includes(REMOTE_DENY_VISIBILITY_TOKEN)) continue;+ // The full union, so a preference page reserving itself under an+ // informal key (`confidentiality: secret`) is withheld exactly as a+ // formally declared one would be.+ if (remoteDenyVisibilityTokens(meta).includes(REMOTE_DENY_VISIBILITY_TOKEN)) continue;const lifecycle = readLifecycleState(meta);if (lifecycle.tombstoned || lifecycle.supersededBy !== null) continue;if (anchorEntityForms(body, forms, anchorOpts).length === 0) continue;diff --git a/src/core/brain/diagnostics.ts b/src/core/brain/diagnostics.tsindex 7270c5fc..a36c7715 100644--- a/src/core/brain/diagnostics.ts+++ b/src/core/brain/diagnostics.ts@@ -272,6 +272,23 @@ export const DIAGNOSTIC_SIGNALS: ReadonlyMap<string, DiagnosticSignal> = new MapnextCommand: "o2b brain stale",autoRepairable: false,},+ {+ // write-honesty, F4. A count or amount authored with no structural+ // date is a claim that answers as true at every instant, because an+ // absent `valid_from` is unbounded. The exit is the claim-graph+ // query: `--at` and `--history` are what place a claim in time, and+ // the write-time finding names the lexeme the reader should check.+ //+ // Spelled as a literal for the same reason `vault-frozen` above is:+ // importing the code from `page-lint.ts` would close an import cycle+ // (page-lint -> next-step -> diagnostics -> page-lint). The producer+ // and this entry are kept in step by the producer's own test, which+ // resolves THIS code through `nextCommandField`.+ code: "undated-quantity",+ issueClass: "count or amount written with no date",+ nextCommand: "o2b brain claims",+ autoRepairable: false,+ },{// Issue #149. The detector has existed since the hygiene lints// shipped; what it never had was an exit. Recording real use isdiff --git a/src/core/brain/doctor-exits.ts b/src/core/brain/doctor-exits.tsindex 67fc3055..dfd4598c 100644--- a/src/core/brain/doctor-exits.ts+++ b/src/core/brain/doctor-exits.ts@@ -74,6 +74,15 @@ const EXCLUSIONS: ReadonlyArray<DoctorExitExclusion> = ["creating that entity or correcting the edge, and which was intended is only in the " +"author's head - the two produce different graphs",},+ {+ code: "chunk-holds-active",+ reason:+ "the hold is a consequence of the embedding configuration, not a defect in the page or " ++ "the index: the configured model provably cannot take the chunk inside its declared " ++ "input window, and the two ways to change that - chunking the source more finely, or " ++ "leaving the page unembedded - are per-page editorial decisions, which the finding " ++ "names together so the operator can make them",+ },{code: "concept-gap",reason:@@ -135,6 +144,19 @@ const EXCLUSIONS: ReadonlyArray<DoctorExitExclusion> = ["mapping is already printed in the finding. Rewriting someone else's note is an edit to " +"their content, not a repair of a record this store owns",},+ {+ // The informal privacy marker (write-honesty, F1). The marker is a+ // claim about audience, and the vocabulary is the author's: `access:`+ // can mean a narrower group than one token states. The restamp is+ // preview-first for exactly that reason.+ code: "privacy-marker-unstamped",+ reason:+ "the informal marker is a claim about audience written in the author's own vocabulary, " ++ "which can mean a narrower group than any single token states, and the additive restamp " ++ "is preview-first by contract: it writes only after an operator has read each would-be " ++ "visibility line, so no one command resolves the class without deciding for pages nobody " ++ "reviewed",+ },{code: "stale-lock",reason:@@ -195,6 +217,14 @@ const EXCLUSIONS: ReadonlyArray<DoctorExitExclusion> = ["to match is not recoverable from a pattern that does not parse, so the repair is an edit " +"by whoever wrote it",},+ {+ code: "person-nicknames-unreadable",+ reason:+ "the operator's person nickname map exists but cannot be used - it does not parse or " ++ "violates its shape. The repair is an edit to that file by whoever wrote it: the map it " ++ "was meant to hold is not derivable from the error, and minting nickname data would put " ++ "words in the operator's mouth about who knows whom",+ },{code: "iso-invalid",reason:diff --git a/src/core/brain/doctor.ts b/src/core/brain/doctor.tsindex 4b28ce16..4f39ef9f 100644--- a/src/core/brain/doctor.ts+++ b/src/core/brain/doctor.ts@@ -51,6 +51,7 @@ import { toPosix } from "../path-safety.ts";import { activeBudgetPressureCheck } from "./doctor/active-budget-check.ts";import type { DoctorCheck, DoctorCheckContext, DoctorFindings } from "./doctor/check.ts";import { inboxArchivableCheck } from "./doctor/inbox-archive-check.ts";+import { privacyMarkerCheck } from "./doctor/privacy-marker-check.ts";import { capturePatternCheck, configCheck, vaultIgnoreCheck } from "./doctor/config-checks.ts";import { entityRegistryCheck } from "./doctor/entity-checks.ts";import { brokenBacklinkCheck } from "./doctor/link-checks.ts";@@ -80,6 +81,7 @@ import {} from "./doctor/records.ts";import { embeddingSunsetCheck } from "./doctor/embedding-sunset-check.ts";import { embeddingsHealthCheck } from "./doctor/embeddings-health-check.ts";+import { chunkHoldsCheck } from "./doctor/chunk-holds-check.ts";import { rerankHealthCheck } from "./doctor/rerank-health-check.ts";import { payloadRegistryCheck } from "./doctor/payload-checks.ts";import { recallChannelCoverageCheck } from "./doctor/recall-channel-coverage.ts";@@ -212,6 +214,7 @@ const DOCTOR_CHECKS: ReadonlyArray<DoctorCheck> = Object.freeze([frozenVaultCheck,secretsSyncExposureCheck,inboxArchivableCheck,+ privacyMarkerCheck,logChainCheck,symlinkEscapeCheck,frontmatterUncertaintyProbe,@@ -224,6 +227,7 @@ const DOCTOR_CHECKS: ReadonlyArray<DoctorCheck> = Object.freeze([receiptLedgerCheck,embeddingSunsetCheck,embeddingsHealthCheck,+ chunkHoldsCheck,rerankHealthCheck,payloadRegistryCheck,permissionsDocumentCheck,diff --git a/src/core/brain/doctor/chunk-holds-check.ts b/src/core/brain/doctor/chunk-holds-check.tsnew file mode 100644index 00000000..ba04d028--- /dev/null+++ b/src/core/brain/doctor/chunk-holds-check.ts@@ -0,0 +1,129 @@+/**+ * Source documents whose chunks the active embedding configuration holds+ * back (chunk skip-and-hold, F2, task B4).+ *+ * A held chunk is deliberately invisible to every SURFACE THAT COUNTS+ * WORK: it leaves the pending-vector census, the refresh estimate and+ * the backfill, because the index does not owe it a vector. That is+ * right for spend and wrong for awareness - nothing told an operator+ * which source documents carry chunks the configured model provably+ * cannot take. This check is that telling, from the doctor pass an+ * operator already runs.+ *+ * ## One warning per affected document+ *+ * The chunk is where the hold lands but the DOCUMENT is where the repair+ * lands: the levers - narrower chunking, or leaving the page unembedded -+ * are decisions about a page, not about its twenty-third chunk. So the+ * finding aggregates per document, naming the count, the largest single+ * estimate and the window the proof ran against.+ *+ * ## What is silence here, and where the same state speaks instead+ *+ * - nothing held under the ACTIVE key - the honest absence; rows whose+ * key a config change released are decisions this configuration does+ * not make;+ * - a configuration that can hold nothing (no model, no declared+ * window, a local provider) - there is no proof this model could+ * have run;+ * - an index that could not be read at all - the embeddings-health+ * check beside this one already reports that file as census-+ * unrecorded or unmeasured, and a second uncertain stream for the+ * same read failure is what its docblock refuses.+ *+ * A page the caller may not read is not named and does not move a count+ * (`ctx.readable`, the contract every other counting check follows).+ *+ * The check is strictly read-only: one synchronous peek at the index+ * file, no provider contact.+ */++import { resolveSearchConfig } from "../../search/index.ts";+import { chunkHoldExclusion } from "../../search/embedding-spend.ts";+import {+ heldChunkDocCensus,+ type ChunkHoldExclusion,+ type HeldChunkDocRow,+} from "../../search/store/chunks.ts";+import { withReadonlyIndex } from "../../search/store/state.ts";+import type { DoctorIssue } from "../types.ts";+import type { DoctorCheck, DoctorCheckContext, DoctorFindings } from "./check.ts";++/** Indexed documents carry chunks held back from embedding. */+export const CHUNK_HOLDS_ACTIVE_CODE = "chunk-holds-active";++export type { ChunkHoldExclusion, HeldChunkDocRow };++/**+ * The seam the check reads through: the documents holding chunks under+ * `hold`, or `null` when the index could not be read. A test injects any+ * snapshot without building an index.+ */+export type ChunkHoldsProbe = (+ dbPath: string,+ hold: ChunkHoldExclusion,+) => HeldChunkDocRow[] | null;++/** The default probe: one synchronous read-only peek at the index. */+function indexProbe(dbPath: string, hold: ChunkHoldExclusion): HeldChunkDocRow[] | null {+ return withReadonlyIndex(dbPath, (_read, db) => heldChunkDocCensus(db, hold));+}++function holdMessage(row: HeldChunkDocRow, hold: ChunkHoldExclusion): string {+ return (+ `${row.heldChunks} chunk(s) in this document are held back from embedding as provably ` ++ `over the declared input window of ${hold.model}: the largest single-chunk estimate is ` ++ `${row.maxEstimatedTokens} tokens against a ${hold.windowTokens}-token window. The page ` ++ "still searches lexically and is never sent to the embedding provider; narrowing the " ++ "chunking (search_chunk_size) or leaving the page unembedded are the ways to change the " ++ "outcome, both per-document decisions. `o2b search status` lists the held chunks."+ );+}++/**+ * Build the check against `probe`, following `makeEmbeddingsHealthCheck`:+ * the index facts are data the check reads rather than logic it owns.+ */+export function makeChunkHoldsCheck(probe: ChunkHoldsProbe = indexProbe): DoctorCheck {+ return {+ failSoft: true,+ run(ctx: DoctorCheckContext, out: DoctorFindings): void {+ let search;+ try {+ search = resolveSearchConfig({ vault: ctx.vault, configPath: ctx.configPath });+ } catch {+ // The embeddings-health check resolves the same configuration+ // from the same context and reports its failure there, by name.+ // Naming the one condition twice would be two findings for it.+ return;+ }+ if (!search.semantic.enabled || search.semantic.provider === "disabled") return;++ // The index the pass was pointed at wins over the one the search+ // configuration names, so the probe reads the file the rest of the+ // pass speaks about.+ const dbPath = ctx.dbPath ?? search.dbPath;+ // Null when no chunk can be proved over anything - no model or no+ // declared window - and a configuration that could not hold is not+ // a configuration whose holds are missing.+ const hold = chunkHoldExclusion(search);+ if (hold === null) return;++ const docs = probe(dbPath, hold);+ if (docs === null) return;+ const readable =+ ctx.readable === undefined ? docs : docs.filter((doc) => ctx.readable!(doc.path));+ for (const doc of readable) {+ out.issues.push({+ severity: "warning",+ code: CHUNK_HOLDS_ACTIVE_CODE,+ path: doc.path,+ message: holdMessage(doc, hold),+ } satisfies DoctorIssue);+ }+ },+ };+}++/** The registered instance, reading the live index. */+export const chunkHoldsCheck: DoctorCheck = makeChunkHoldsCheck();diff --git a/src/core/brain/doctor/entity-checks.ts b/src/core/brain/doctor/entity-checks.tsindex b1d3f981..62742f2f 100644--- a/src/core/brain/doctor/entity-checks.ts+++ b/src/core/brain/doctor/entity-checks.ts@@ -22,7 +22,18 @@ import {ENTITY_CONFLICT_KIND,type EntityIndex,} from "../entities/index-builder.ts";+import {+ PersonNicknameConfigError,+ resolvePersonNicknameMap,+} from "../entities/person-nicknames.ts";+import {+ derivePersonNameVariants,+ EMPTY_PERSON_NICKNAME_MAP,+ isPersonCategory,+ type PersonNicknameMap,+} from "../entities/person-variants.ts";import { findMalformedEntityLabels } from "../entities/label-hygiene.ts";+import { ENTITY_STATUS_SCOPE, entityStatusInScope } from "../entities/status-scope.ts";import type { BrainEntity, EntityConflict } from "../entities/types.ts";import type { DoctorCheck } from "./check.ts";@@ -49,21 +60,110 @@ function claimedForms(index: EntityIndex, conflict: EntityConflict): string[] {);}+/** Doctor code of an unusable operator nickname map. */+export const PERSON_NICKNAMES_UNREADABLE_CODE = "person-nicknames-unreadable";+/** Doctor code of a proposal-only alias-merge candidate. */export const ENTITY_ALIAS_CANDIDATE_CODE = "entity-alias-candidate";+/**+ * The one alias-merge sentence both candidate kinds are written with.+ * Kept in a single home so the doctor's findings and the decision-model+ * annotator's message-matching can never drift apart.+ */+function aliasMergeCandidateMessage(+ nameA: string,+ idA: string,+ nameB: string,+ idB: string,+ method: string,+ similarity: number | string,+): string {+ return (+ `possible alias-merge: '${nameA}' (${idA}) ~ '${nameB}' (${idB}) ` ++ `[${method} ${similarity}]. Review and add an alias to merge — never auto-merged.`+ );+}+/*** The doctor message of one alias-merge candidate. Shared with the* decision-model annotator (`entities/alias-verdicts.ts`), which matches* the doctor's issues back to their candidates by this exact text.*/export function entityAliasCandidateMessage(c: EntityAliasCandidate): string {- return (- `possible alias-merge: '${c.name_a}' (${c.a}) ~ '${c.name_b}' (${c.b}) ` +- `[${c.method} ${c.similarity}]. Review and add an alias to merge — never auto-merged.`+ return aliasMergeCandidateMessage(c.name_a, c.a, c.name_b, c.b, c.method, c.similarity);+}++/**+ * The doctor message of one DERIVED-VARIANT collision+ * (write-honesty-egress-consent, F3/C5): the same alias-merge shape,+ * with a `person-variant` method label. The similarity is 1 because the+ * collision is exact - the derived form equals the other record's+ * declared form byte-for-byte after normalization. The method label+ * differs from every lexical/embedding candidate's, so the annotator's+ * by-message match simply never pairs this finding with one.+ */+export function personVariantCollisionMessage(a: BrainEntity, b: BrainEntity): string {+ const [first, second] = a.id.localeCompare(b.id) <= 0 ? [a, b] : [b, a];+ return aliasMergeCandidateMessage(+ first.name,+ first.id,+ second.name,+ second.id,+ "person-variant",+ 1,);}+/**+ * Every pair of canonical PERSON records where a derived variant of one+ * equals a DECLARED form (name or alias) of the other+ * (write-honesty-egress-consent, F3/C5). Lookups stay exact-declared-+ * first, so nothing mis-resolves today - but the registry holds two+ * records answering to one form, and that is the same latent alias-merge+ * the candidates above nominate, so it is reported through the same+ * warning. Exactly one finding per unordered pair, however many variants+ * collide and whichever side derives.+ *+ * A form shared only as a DERIVED form on both sides is not a collision:+ * neither record declared it, so the collision scan has nothing to offer+ * the merge that the ambiguity error at lookup time does not already+ * say better.+ */+function derivedVariantCollisionMessages(+ index: EntityIndex,+ nicknames: PersonNicknameMap,+): string[] {+ const persons = index.entities.filter(+ (e) =>+ entityStatusInScope(e.status, ENTITY_STATUS_SCOPE.canonical) && isPersonCategory(e.category),+ );+ const declared = new Map<string, ReadonlySet<string>>(+ persons.map((p) => [p.id, new Set([p.name, ...p.aliases].map((f) => normalizeEntityName(f)))]),+ );+ const seenPairs = new Set<string>();+ const messages: string[] = [];+ for (const deriver of persons) {+ const variants = new Set<string>();+ for (const form of [deriver.name, ...deriver.aliases]) {+ for (const v of derivePersonNameVariants(normalizeEntityName(form), nicknames)) {+ variants.add(v);+ }+ }+ if (variants.size === 0) continue;+ for (const claimant of persons) {+ if (claimant.id === deriver.id) continue;+ const pairKey = [deriver.id, claimant.id].toSorted().join("\u0000");+ if (seenPairs.has(pairKey)) continue;+ const claimantForms = declared.get(claimant.id)!;+ if (![...variants].some((v) => claimantForms.has(v))) continue;+ seenPairs.add(pairKey);+ messages.push(personVariantCollisionMessage(deriver, claimant));+ }+ }+ return messages;+}+/** `ent-a (path-a), ent-b (path-b)` - the records, not just their files. */function describeClaimants(index: EntityIndex, conflict: EntityConflict): string {return conflictClaimants(index, conflict)@@ -150,6 +250,44 @@ export const entityRegistryCheck: DoctorCheck = {});}+ // write-honesty-egress-consent (F3/C5): a derived person variant of+ // one person that equals another person's declared form. Same+ // warning as the candidates below - it nominates the same remedy -+ // but deterministic and always on, like the lints above. Unlike the+ // conflict findings above, which deliberately read what is STORED+ // (scoping them would stop the doctor reporting on exactly the+ // records a read path hides), this scan and the malformed-label scan+ // resolve through the shared canonical predicate: a quarantined+ // person's variants nominate no merges.+ // An unusable nickname file degrades this scan to no derived+ // collisions rather than discarding the check - the read lanes keep+ // their documented degradations too - but the doctor is the+ // surfacing point: a present-yet-unusable operator file is one+ // warning, naming the path, with the error's own short+ // (position-only) detail.+ let nicknames = EMPTY_PERSON_NICKNAME_MAP;+ try {+ nicknames = resolvePersonNicknameMap(vault);+ } catch (err) {+ if (!(err instanceof PersonNicknameConfigError)) throw err;+ nicknames = EMPTY_PERSON_NICKNAME_MAP;+ issues.push({+ severity: "warning",+ code: PERSON_NICKNAMES_UNREADABLE_CODE,+ path: err.path,+ message:+ `the person nickname map at ${err.path} is unusable and nickname-derived ` ++ `lookups run without it: ${err.detail}`,+ });+ }+ for (const message of derivedVariantCollisionMessages(index, nicknames)) {+ issues.push({+ severity: "warning",+ code: ENTITY_ALIAS_CANDIDATE_CODE,+ message,+ });+ }+// semantic-retrieval-precision (t_47fd9523): opt-in, proposal-only// alias-merge candidates. Off by default → this block is a no-op and// the report is byte-identical to the baseline. The doctor isdiff --git a/src/core/brain/doctor/privacy-marker-check.ts b/src/core/brain/doctor/privacy-marker-check.tsnew file mode 100644index 00000000..82371446--- /dev/null+++ b/src/core/brain/doctor/privacy-marker-check.ts@@ -0,0 +1,59 @@+/**+ * Informal privacy markers that are not formally stamped (write-honesty, F1).+ *+ * The read roots withhold a page saying `access: private`,+ * `confidentiality: secret` or `sensitivity: confidential` from a remote+ * caller (see `graph/privacy-markers.ts`), but the reservation lives in an+ * informal key. Nothing that reads `visibility:` - derived-page stamping,+ * an export filtering on the token, an operator auditing reservations -+ * can see it, and a page whose author believed it declared its audience+ * has a claim that only two of this tree's readers honour.+ *+ * This check is the discovery path for that gap: ONE warning carrying the+ * count and the paths, silent when there are none. The population is the+ * restamp's own candidate list ({@link planPrivacyMarkerRestamp}), so the+ * finding, its count, and the preview an operator then reads cannot drift+ * apart. A marker page already carrying `visibility: private` is not in+ * that population and does not appear here.+ *+ * A page the caller may not read is not named and does not move the count+ * (`ctx.readable`, the contract every other counting check follows): a+ * remote caller must not learn how many reserved pages exist.+ */++import { planPrivacyMarkerRestamp } from "../privacy-marker-restamp.ts";+import type { DoctorIssue } from "../types.ts";+import type { DoctorCheck, DoctorCheckContext, DoctorFindings } from "./check.ts";++/** Marker pages whose `visibility:` does not carry the reserved token. */+export const PRIVACY_MARKER_UNSTAMPED_CODE = "privacy-marker-unstamped";++/** How many paths one finding names before it counts the rest. */+const NAMED_PATH_LIMIT = 5;++export const privacyMarkerCheck: DoctorCheck = {+ failSoft: true,+ run(ctx: DoctorCheckContext, out: DoctorFindings): void {+ const candidates = planPrivacyMarkerRestamp(ctx.vault).candidates;+ const readable =+ ctx.readable === undefined+ ? candidates+ : candidates.filter((candidate) => ctx.readable!(candidate.path));+ if (readable.length === 0) return;++ const named = readable.slice(0, NAMED_PATH_LIMIT).map((candidate) => candidate.path);+ const unreadableRemainder = readable.length - named.length;+ const remainder = unreadableRemainder > 0 ? ` and ${unreadableRemainder} more` : "";+ out.issues.push({+ severity: "warning",+ code: PRIVACY_MARKER_UNSTAMPED_CODE,+ path: ctx.vault,+ message:+ `${readable.length} page(s) reserve themselves with an informal privacy marker ` ++ "(access / confidentiality / sensitivity) rather than a visibility: declaration: " ++ `${named.join(", ")}${remainder}. Remote reads are already denied, but tools that ` ++ "read visibility: cannot see the reservation. The preview-first restamp reports each " ++ "would-be visibility line and writes only on request, additively",+ } satisfies DoctorIssue);+ },+};diff --git a/src/core/brain/dream-scan.ts b/src/core/brain/dream-scan.tsindex 0e878028..f43d1b68 100644--- a/src/core/brain/dream-scan.ts+++ b/src/core/brain/dream-scan.ts@@ -13,7 +13,7 @@import { existsSync, readdirSync } from "node:fs";import { join } from "node:path";-import { pageVisibility } from "../graph/visibility.ts";+import { remoteDenyVisibilityTokens } from "../graph/privacy-markers.ts";import type { FrontmatterMap } from "../types.ts";import { parseFrontmatter } from "../vault.ts";import type {@@ -105,11 +105,20 @@ function markdownFilesIn(dir: string): string[] {}/**- * A record's `visibility:` tokens as a spreadable slice: `{}` for default- * visibility, so a record from an untagged page keeps its exact shape.+ * A record's `visibility:` tokens plus any informal privacy marker's+ * reservation (`remoteDenyVisibilityTokens`), as a spreadable slice: `{}`+ * for default visibility, so a record from an untagged page keeps its+ * exact shape.+ *+ * The scan is the one place the pass holds a page's raw frontmatter, so+ * the marker union rides in here: the planner modules are pure folds over+ * the records' token lists (`strictestVisibilityOf` in the topic plan and+ * the cross-topic promotion), and a page saying `access: private` under+ * the informal key must reserve every page drafted from it exactly as a+ * formal `visibility: private` does.*/function visibilitySlice(meta: FrontmatterMap): { visibility?: ReadonlyArray<string> } {- const tokens = pageVisibility(meta);+ const tokens = remoteDenyVisibilityTokens(meta);return tokens.length > 0 ? { visibility: Object.freeze(tokens) } : {};}diff --git a/src/core/brain/entities/person-nicknames.ts b/src/core/brain/entities/person-nicknames.tsnew file mode 100644index 00000000..6b393c49--- /dev/null+++ b/src/core/brain/entities/person-nicknames.ts@@ -0,0 +1,149 @@+/**+ * Reader for the operator-supplied person nickname map+ * (write-honesty-egress-consent, F3).+ *+ * `Brain/entities/person-nicknames.json` - a JSON object keyed by+ * nickname, whose values are the full person names the nickname can+ * stand for. Same `Brain/entities/` neighbourhood as the derived+ * lexicon artifact, but this file is OPERATOR data, never+ * machine-derived, so there is no schema-version/freshness envelope -+ * the whole document is the map.+ *+ * This module was moved here verbatim from+ * `src/core/search/entity-alias.ts`, which had taken it in for query+ * expansion (C2). The registry's derived lookup lane (C4) and the+ * doctor's collision scan (C5) need the same reader, and the brain+ * layer must not import from `src/core/search/` - so the reader lives+ * on the brain side, beside the derivation kernel it feeds, and the+ * search module re-exports it to keep its public surface identical.+ *+ * A MISSING file is not an error - it means the vault declares no+ * nickname data - but a present file that fails to parse or violates+ * the shape is an operator-data fault, and it surfaces by name; every+ * consumer decides its own posture toward it (search expansion degrades+ * to the identity expansion, read resolution degrades to exact-only,+ * the doctor's collision scan degrades to no derived collisions).+ */++import { readFileSync } from "node:fs";+import { join } from "node:path";++import { normalizeEntityName } from "./canonical.ts";+import { EMPTY_PERSON_NICKNAME_MAP, type PersonNicknameMap } from "./person-variants.ts";++/**+ * Vault-relative home of the operator-supplied nickname map.+ */+export const PERSON_NICKNAMES_ARTIFACT_REL = join("Brain", "entities", "person-nicknames.json");++/**+ * Raised when `person-nicknames.json` exists but cannot be used. A+ * MISSING file is not an error - it means the vault declares no+ * nickname data - but a present file that fails to parse or violates+ * the shape is an operator-data fault, and it surfaces by name; every+ * consumer decides its own posture toward it (search expansion degrades+ * to the identity expansion, read resolution degrades to exact-only).+ */+export class PersonNicknameConfigError extends Error {+ readonly path: string;+ readonly detail: string;+ constructor(path: string, detail: string) {+ super(`unreadable person nickname config at ${path}: ${detail}`);+ this.name = "PersonNicknameConfigError";+ this.path = path;+ this.detail = detail;+ }+}++/** True for the ENOENT family: the artifact is simply not there. */+function isMissingFileError(err: unknown): boolean {+ return typeof err === "object" && err !== null && (err as { code?: string }).code === "ENOENT";+}++/**+ * A position-only note from a JSON parser error, or a fixed sentence when+ * the runtime exposes no position.+ *+ * The parser's own message can quote the bytes it choked on, and this+ * detail reaches doctor output and logs, so the raw message never+ * travels: only a line or offset number - which says nothing about the+ * content - or the fixed sentence ever leaves here.+ */+function jsonPositionNote(err: unknown): string {+ const raw = err instanceof Error ? err.message : String(err);+ const line = /line (\d+)/i.exec(raw)?.[1];+ if (line !== undefined) return `parse failed at line ${line}`;+ const offset = /position (\d+)/i.exec(raw)?.[1];+ if (offset !== undefined) return `parse failed at offset ${offset}`;+ return "the parser reported a syntax error";+}++/**+ * Fold one parsed JSON document into the normalized nickname map the+ * derivation kernel consumes: keys and values are run through+ * `normalizeEntityName` (the comparison is post-normalization, exactly+ * like the label denylist), empty normalized forms are dropped, and+ * value lists are deduplicated. Malformed shapes surface through+ * {@link PersonNicknameConfigError}.+ */+function parsePersonNicknameDocument(text: string, path: string): PersonNicknameMap {+ let parsed: unknown;+ try {+ parsed = JSON.parse(text);+ } catch (err) {+ throw new PersonNicknameConfigError(path, `not parseable JSON (${jsonPositionNote(err)})`);+ }+ if (typeof parsed !== "object" || parsed === null || Array.isArray(parsed)) {+ throw new PersonNicknameConfigError(path, "expected a JSON object mapping nickname to names");+ }+ const out = new Map<string, string[]>();+ let entryIndex = 0;+ for (const [rawKey, rawValue] of Object.entries(parsed as Record<string, unknown>)) {+ entryIndex += 1;+ const key = normalizeEntityName(rawKey);+ if (!key) continue;+ // The detail names the entry's POSITION, never the key's text: a key+ // is a person's nickname, and this detail reaches doctor output and+ // logs - the same channel the parse-error path keeps position-only.+ const shapeDetail = `value of entry #${entryIndex} must be an array of names`;+ if (!Array.isArray(rawValue)) {+ throw new PersonNicknameConfigError(path, shapeDetail);+ }+ const values: string[] = [];+ for (const entry of rawValue) {+ if (typeof entry !== "string") {+ throw new PersonNicknameConfigError(path, shapeDetail);+ }+ const normalized = normalizeEntityName(entry);+ if (normalized && !values.includes(normalized)) values.push(normalized);+ }+ if (values.length === 0) continue;+ // Two raw keys that normalize alike ("Bob" and "bob ") are one+ // nickname: the value lists MERGE, because a silent replace would+ // make the expansion depend on the JSON key order.+ const prior = out.get(key) ?? [];+ out.set(key, [...prior, ...values.filter((v) => !prior.includes(v))]);+ }+ return Object.freeze(+ new Map([...out].map(([key, values]) => [key, Object.freeze(values)] as const)),+ );+}++/**+ * Resolve the operator nickname map for `vault`. A missing file yields+ * the empty map (no nickname data declared); a present but unreadable+ * or malformed one throws {@link PersonNicknameConfigError}, so a+ * caller that must not guess can tell "none declared" from "declared+ * but unusable".+ */+export function resolvePersonNicknameMap(vault: string): PersonNicknameMap {+ const path = join(vault, PERSON_NICKNAMES_ARTIFACT_REL);+ let text: string;+ try {+ text = readFileSync(path, "utf8");+ } catch (err) {+ if (isMissingFileError(err)) return EMPTY_PERSON_NICKNAME_MAP;+ throw new PersonNicknameConfigError(path, `not readable (${String(err)})`);+ }+ return parsePersonNicknameDocument(text, path);+}diff --git a/src/core/brain/entities/person-variants.ts b/src/core/brain/entities/person-variants.tsnew file mode 100644index 00000000..1e32232f--- /dev/null+++ b/src/core/brain/entities/person-variants.ts@@ -0,0 +1,149 @@+/**+ * Person-name variant derivation (write-honesty-egress-consent, F3).+ *+ * Pure kernel over an ALREADY-NORMALIZED person name - the same+ * normalization shape as `normalizeEntityName` in `canonical.ts`: NFC,+ * lowercase, collapsed whitespace. This module is a LEAF of the import+ * graph and imports nothing, so every consumer (query expansion, the+ * registry's derived lookup lane, the doctor's collision scan) derives+ * from the one implementation rather than from three copies of the+ * rules. The price of the leaf position is stated where it bites: the+ * floor constant and the no-space script set below MIRROR their+ * canonical.ts siblings (`MIN_ANCHOR_FORM_LENGTH`, `NO_SPACE_SCRIPT_RE`)+ * instead of importing them, and the mirror is pinned by tests on both+ * sides.+ *+ * Three structural rules, deliberately all conservative:+ *+ * - FIRST-INITIAL-PLUS-SURNAME: `ada lovelace` also answers to+ * `a lovelace`. The initial convention is a spaced-script+ * convention, so this form is emitted only for a multi-token name+ * that carries no unspaced-script character.+ * - REVERSED WORD ORDER: `ada lovelace` also answers to+ * `lovelace ada`, the "Last, First" spelling read back as words.+ * Like the initial form it is emitted only for a multi-token name+ * with no unspaced-script character - in Han, Hiragana, Katakana,+ * Thai, Lao, Khmer or Myanmar text a name's tokens are not word+ * separable, and an unspaced-script (CJK) name emits NO word-order+ * variants at all.+ * - NICKNAMES: never derived from any shipped vocabulary. There is no+ * hardcoded English word list here or downstream - the only+ * nickname source is the operator-supplied map passed as an+ * argument (a vault config map), exactly like the label denylist's+ * operator-data-only rule. A name derives a nickname when the map+ * lists the name's FULL normalized form among the values the+ * nickname stands for.+ *+ * Every emitted form meets a minimum-form-length floor (the same+ * discipline as the anchoring kernels' floor), the output order is+ * fixed (initial, reversed, then nicknames sorted), and the returned+ * array is frozen: same input, same bytes, on every peer. Derivation is+ * a re-ranking and resolution aid, never a retrieval mechanism and+ * never part of identity - `entityIdentityKey` is untouched by every+ * form this module emits.+ */++/**+ * Minimum normalized form length that a derived person variant must+ * reach to be emitted. Mirrors `MIN_ANCHOR_FORM_LENGTH` in+ * `canonical.ts` (3): a derived form is an emitted needle in its own+ * right, and an unfloored one would anchor standalone fragments no full+ * form ever matched. This module imports nothing, so the value is+ * restated here and pinned by tests rather than shared by import.+ */+export const MIN_PERSON_VARIANT_LENGTH = 3;++/**+ * The registry categories that hold PERSON entities - the person+ * members of `ENTITY_TYPE_FOLDERS` in `canonical.ts`. Mirrored here for+ * the same leaf reason as the floor above; every consumer gates+ * person-variant derivation on this predicate, so orgs, companies and+ * teams never derive variants.+ */+export const PERSON_ENTITY_CATEGORIES: ReadonlyArray<string> = Object.freeze(["people", "person"]);++const PERSON_CATEGORY_SET: ReadonlySet<string> = new Set(PERSON_ENTITY_CATEGORIES);++/** True when `category` is a person-category registry folder. */+export function isPersonCategory(category: string): boolean {+ return PERSON_CATEGORY_SET.has(category);+}++/**+ * Operator-supplied nickname data: keyed by a normalized NICKNAME,+ * whose values are the normalized FULL person names the nickname can+ * stand for. The only vocabulary source for nickname expansion; an+ * empty or absent map means no nickname variants exist.+ */+export type PersonNicknameMap = ReadonlyMap<string, ReadonlyArray<string>>;++/** The shared empty map, so callers without nickname data allocate once. */+export const EMPTY_PERSON_NICKNAME_MAP: PersonNicknameMap = Object.freeze(new Map());++/**+ * The scripts written WITHOUT word spaces - Han, Hiragana, Katakana,+ * Thai, Lao, Khmer, Myanmar. Mirrors `NO_SPACE_SCRIPT_RE` in+ * `canonical.ts` (which this leaf cannot import): Korean is+ * deliberately absent, because Hangul writes spaces between words.+ * A name containing any character from these scripts emits no+ * structural (initial or word-order) variants.+ */+const NO_SPACE_SCRIPT_RE =+ /[\p{Script=Han}\p{Script=Hiragana}\p{Script=Katakana}\p{Script=Thai}\p{Script=Lao}\p{Script=Khmer}\p{Script=Myanmar}]/u;++/**+ * Derive the person-name variants of the ALREADY-NORMALIZED name+ * `name`: first-initial-plus-surname, reversed word order, and the+ * operator-supplied nicknames the name answers to - in that fixed+ * order, nicknames sorted. Deterministic and frozen: the same input+ * yields byte-identical output on every call and every peer.+ *+ * A name is split on single spaces (normalization collapsed any run).+ * Structural variants require at least two tokens AND no+ * unspaced-script character anywhere in the name; a variant equal to+ * the input name is never emitted, and every emitted form is at least+ * {@link MIN_PERSON_VARIANT_LENGTH} characters. Nickname expansion+ * consults ONLY `nicknames` - an absent or empty map yields no+ * nickname variants, and no hardcoded list exists anywhere in this+ * module. Pure: no I/O, no state.+ */+export function derivePersonNameVariants(+ name: string,+ nicknames: PersonNicknameMap = EMPTY_PERSON_NICKNAME_MAP,+): ReadonlyArray<string> {+ const out: string[] = [];+ const seen = new Set<string>();+ const consider = (form: string): void => {+ if (form.length < MIN_PERSON_VARIANT_LENGTH) return;+ if (form === name) return;+ if (seen.has(form)) return;+ seen.add(form);+ out.push(form);+ };++ const tokens = name.split(" ").filter((token) => token !== "");+ if (tokens.length > 0 && !NO_SPACE_SCRIPT_RE.test(name)) {+ if (tokens.length >= 2) {+ // First-initial-plus-surname: the initial of the first token,+ // then the remaining tokens verbatim ("ada mary lovelace" ->+ // "a mary lovelace").+ consider(`${tokens[0]!.slice(0, 1)} ${tokens.slice(1).join(" ")}`);+ // Reversed word order: "Last, First" read back as words.+ consider(tokens.slice().reverse().join(" "));+ }+ }++ if (nicknames.size > 0) {+ const nickHits: string[] = [];+ for (const [nick, fullNames] of nicknames) {+ if (!fullNames.includes(name)) continue;+ if (nick.length < MIN_PERSON_VARIANT_LENGTH || nick === name || seen.has(nick)) continue;+ nickHits.push(nick);+ }+ // Sorted so the output is independent of the operator map's+ // insertion order.+ for (const nick of nickHits.toSorted()) consider(nick);+ }++ return Object.freeze(out);+}diff --git a/src/core/brain/entities/registry.ts b/src/core/brain/entities/registry.tsindex 79969698..af927e86 100644--- a/src/core/brain/entities/registry.ts+++ b/src/core/brain/entities/registry.ts@@ -40,6 +40,13 @@ import {} from "./canonical.ts";import { resolveEntityLexicon } from "./lexicon.ts";import { resolveEntityLabelDenylist } from "./label-hygiene.ts";+import { resolvePersonNicknameMap } from "./person-nicknames.ts";+import {+ derivePersonNameVariants,+ EMPTY_PERSON_NICKNAME_MAP,+ isPersonCategory,+ type PersonNicknameMap,+} from "./person-variants.ts";import {buildEntityIndex,conflictClaimants,@@ -206,9 +213,65 @@ export class EntityAmbiguityError extends Error {}/**- * Resolve a ref at the CANONICAL status scope: canonical name first, then- * alias. The index's lookup maps are built at the same scope, so the two- * branches below agree by construction rather than by coincidence.+ * Every derived person form `entity` answers to: the variants the+ * derivation kernel emits for each of its declared forms (name and+ * aliases), as a set. Empty by construction for non-person categories,+ * because the kernel emits nothing for them.+ */+function derivedPersonForms(+ entity: BrainEntity,+ nicknames: PersonNicknameMap,+): ReadonlySet<string> {+ const forms = new Set<string>();+ for (const declared of [entity.name, ...entity.aliases]) {+ for (const variant of derivePersonNameVariants(normalizeEntityName(declared), nicknames)) {+ forms.add(variant);+ }+ }+ return forms;+}++/**+ * The DERIVED lane of canonical read resolution: the person variants+ * (write-honesty-egress-consent, F3) - first-initial-plus-surname,+ * reversed word order, operator nicknames - consulted only after the+ * exact name/alias lanes above have missed. A unique derived match+ * resolves; more than one is the same ambiguity the exact lane reports,+ * naming both ids. Like the exact lane it resolves canonical-scope+ * records only, so a quarantined person's variants resolve nothing.+ */+function resolveDerived(+ index: EntityIndex,+ ref: EntityRef,+ nicknames: PersonNicknameMap,+): BrainEntity | null {+ const query = normalizeEntityName(ref.query);+ if (!query) return null;+ const category = ref.category !== undefined ? validateEntityCategory(ref.category) : undefined;+ const matches = index.entities.filter(+ (e) =>+ entityStatusInScope(e.status, ENTITY_STATUS_SCOPE.canonical) &&+ isPersonCategory(e.category) &&+ (category === undefined || e.category === category) &&+ derivedPersonForms(e, nicknames).has(query),+ );+ if (matches.length > 1) throw new EntityAmbiguityError(ref.query, matches);+ return matches[0] ?? null;+}++/**+ * Resolve a ref at the CANONICAL status scope for reading: canonical+ * name first, then alias - and only after BOTH miss, the derived+ * person-variant lane ({@link resolveDerived}). The index's lookup maps+ * are built at the same canonical scope, so the exact branches agree by+ * construction rather than by coincidence.+ *+ * The WRITE path deliberately does not come through here-then-derived+ * as one step: {@link resolveWriteTarget} calls {@link resolveActive}+ * alone, so a write resolves on declared forms only. A record is+ * updated or claimed by the names it chose, never by a form derivation+ * invented for it - and a quarantined namesake's variants (which the+ * canonical scope never holds anyway) stay unreachable in both lanes.*/function resolveActive(index: EntityIndex, ref: EntityRef): BrainEntity | null {const query = normalizeEntityName(ref.query);@@ -291,8 +354,24 @@ function resolveWriteTarget(return untrustedOrigin ? resolveQuarantined(index, ref) : resolveActive(index, ref);}+/**+ * Read resolution: the exact lanes, then - only on a full miss - the+ * derived person-variant lane. An unusable nickname file is an+ * operator-data fault, not a read outage, so read resolution degrades+ * to exact-only (the posture `PersonNicknameConfigError`'s docblock+ * pins for every consumer).+ */export function getEntity(vault: string, ref: EntityRef): BrainEntity | null {- return resolveActive(buildEntityIndex(vault), ref);+ const index = buildEntityIndex(vault);+ const exact = resolveActive(index, ref);+ if (exact !== null) return exact;+ let nicknames = EMPTY_PERSON_NICKNAME_MAP;+ try {+ nicknames = resolvePersonNicknameMap(vault);+ } catch {+ return null;+ }+ return resolveDerived(index, ref, nicknames);}/**diff --git a/src/core/brain/hygiene/detectors/dedup.ts b/src/core/brain/hygiene/detectors/dedup.tsindex 8b58ff35..ccd3e886 100644--- a/src/core/brain/hygiene/detectors/dedup.ts+++ b/src/core/brain/hygiene/detectors/dedup.ts@@ -112,7 +112,12 @@ function cosine(a: ReadonlyArray<number>, b: ReadonlyArray<number>): number {function resolveConfiguredProvider(vault: string): EmbeddingProvider | null {try {- const provider = makeProvider(resolveSearchConfig({ vault }).semantic);+ const config = resolveSearchConfig({ vault });+ // The consent travels with the configuration that would spend it:+ // this semantic configuration was resolved env-only (no config path),+ // so the consent reads the same source instead of whatever file the+ // process happens to default to.+ const provider = makeProvider({ ...config.semantic, configPath: config.configPath });return providerProducesVectors(provider) ? provider : null;} catch {return null;diff --git a/src/core/brain/hygiene/detectors/document-dedup.ts b/src/core/brain/hygiene/detectors/document-dedup.tsindex 90a92ec4..6ee90d24 100644--- a/src/core/brain/hygiene/detectors/document-dedup.ts+++ b/src/core/brain/hygiene/detectors/document-dedup.ts@@ -481,7 +481,12 @@ export function detectDocumentDedup(function resolveConfiguredProvider(vault: string): EmbeddingProvider | null {try {- const provider = makeProvider(resolveSearchConfig({ vault }).semantic);+ const config = resolveSearchConfig({ vault });+ // The consent travels with the configuration that would spend it:+ // this semantic configuration was resolved env-only (no config path),+ // so the consent reads the same source instead of whatever file the+ // process happens to default to.+ const provider = makeProvider({ ...config.semantic, configPath: config.configPath });return providerProducesVectors(provider) ? provider : null;} catch {return null;diff --git a/src/core/brain/notes/create-note.ts b/src/core/brain/notes/create-note.tsindex b85ebfb4..27eba2a2 100644--- a/src/core/brain/notes/create-note.ts+++ b/src/core/brain/notes/create-note.ts@@ -57,6 +57,11 @@ import {originChannelStamp,type OriginChannelStamp,} from "../../origin-channel.ts";+import {+ CONTENT_CLASS_FIELD,+ contentClassCreationStamp,+ withoutCallerContentClass,+} from "../provenance/content-class.ts";import { isFileAlreadyExists } from "../../fs-atomic.ts";import { formatFrontmatter, writeFrontmatterAtomic } from "../../vault.ts";import {@@ -771,17 +776,24 @@ export function createNote(vault: string, input: CreateNoteInput): CreateNoteRes// exists.if (input.strict === true) assertValidDocument(vault, callerFrontmatter, body);- // The server-derived stamp (Unit C) goes on LAST, so a caller that- // supplied a key of the same name loses to the process that actually- // wrote the note.+ // The server-derived stamps (Unit C, and the content class beside it)+ // go on LAST, so a caller that supplied a key of either name loses to+ // the process that actually wrote the note. A caller-named+ // `content_class` is not merely outranked but dropped: on an unmarked+ // write the honest byte is NO key, and letting the caller's through+ // would let an mcp-transport caller forge an exclusion that ripples+ // all the way to the export boundary.//// Stamped on CREATION only. `update_note` and `append_note` rewrite a// note this call did not author, and re-stamping there would overwrite// the creating channel with the mutating one - the same objection that// refuses backfilling the field onto existing records.+ const callerFrontmatterWithoutClass = withoutCallerContentClass(callerFrontmatter);+ const contentClass = contentClassCreationStamp(originChannel);const frontmatter: FrontmatterMap = {- ...callerFrontmatter,+ ...callerFrontmatterWithoutClass,[ORIGIN_CHANNEL_FIELD]: originChannel,+ ...(contentClass !== null ? { [CONTENT_CLASS_FIELD]: contentClass } : {}),};const skipOccupied = input.ifExists === "skip";diff --git a/src/core/brain/page-lint.ts b/src/core/brain/page-lint.tsindex dd585641..734886b1 100644--- a/src/core/brain/page-lint.ts+++ b/src/core/brain/page-lint.ts@@ -24,6 +24,12 @@* composite scope bucket - through the shared `tokenise`/`jaccard`* primitives (t_d30c0548). A bounded, deterministic candidate set per* write; the finding rides the receipt and never gates the write.+ * - {@link UNDATED_QUANTITY_CODE}, the authored body scanned for a count+ * or amount with no structural date on the page (t_fb442a24). The+ * trigger shapes live in `undated-quantity.ts`; this module owns the+ * escape hatches - `valid_from` in frontmatter, any ISO date in the+ * body, or a `[Source: <name>, YYYY-MM-DD]` marker - and the finding+ * quotes the lexeme and its offset, never the line. Advisory only.** `demote-stale-stable` is deliberately EXCLUDED, not overlooked: its* trigger is the page's creation age against the staleness cap, so it can@@ -106,8 +112,11 @@ import {type NearDuplicatePoolEntry,type ReadableRef,} from "./near-duplicate.ts";+import { VALID_FROM_KEY } from "./lifecycle/temporal-replace.ts";import { tokenise } from "./similarity.ts";+import { parseCitations } from "./temporal/citations.ts";import type { DoctorSeverity } from "./types.ts";+import { containsIsoDate, findUndatedQuantities } from "./undated-quantity.ts";import { WIKILINK_TARGET_RE, isBrainArtifactId, normaliseWikilinkTarget } from "./wikilink.ts";import { ARTIFACT_MAX_BYTES, validateArtifact } from "./write-session/validate.ts";@@ -140,6 +149,14 @@ export const NEAR_DUPLICATE_JACCARD = NEAR_DUPLICATE_THRESHOLDS.writeHint;/** Code the near-duplicate detector reports under. */export const NEAR_DUPLICATE_CODE = "near-duplicate";+/**+ * Code the undated-quantity detector reports under (write-honesty, F4).+ * A count or amount with no structural date on the page answers as true at+ * every instant, because an absent `valid_from` is unbounded. The code is+ * registered in `diagnostics.ts` so the finding carries a next command.+ */+export const UNDATED_QUANTITY_CODE = "undated-quantity";+/*** Most sibling pages one directory contributes as near-duplicate* candidates: the newest by mtime. Every sibling is stat'ed (cheap), but@@ -555,6 +572,48 @@ function hasReadableArtifact(vault: string, ctx: LintContext, target: string): breturn false;}+/**+ * Whether the page supplies a structural date that places its quantities+ * in time. Three hatches, all structural: a `valid_from` boundary in+ * frontmatter, any ISO date or instant in the body, or a+ * `[Source: <name>, YYYY-MM-DD]` marker. The citation marker always+ * carries its own ISO date, so the date hatch already covers a well-formed+ * one; it is checked explicitly through the canonical parser so the+ * contract reads as three hatches rather than a coincidence.+ */+function hasDatedQuantityContext(meta: FrontmatterMap, body: string): boolean {+ const declared = meta[VALID_FROM_KEY];+ // The boundary hatch is structural like the other two: the value must+ // NAME a date. Any non-empty string ("TBD", "unknown", "soon") places+ // nothing in time, and accepting it let a placeholder quiet the lint.+ if (typeof declared === "string" && containsIsoDate(declared)) return true;+ if (containsIsoDate(body)) return true;+ return parseCitations(body).markers.length > 0;+}++/**+ * The undated-quantity findings for one written page: one warning per+ * lexeme the body states with no date on the page. The lexeme and its+ * offset are the whole evidence - the authored line is not quoted, so a+ * receipt cannot carry prose the caller did not ask it to repeat.+ */+function undatedQuantityFindings(+ page: string,+ meta: FrontmatterMap,+ body: string,+): PageLintFinding[] {+ if (hasDatedQuantityContext(meta, body)) return [];+ return findUndatedQuantities(body).map(({ lexeme, offset }) =>+ finding(+ "warning",+ UNDATED_QUANTITY_CODE,+ page,+ "body",+ `quantity ${JSON.stringify(lexeme)} at offset ${offset} carries no date`,+ ),+ );+}+function lintOnePage(vault: string,ctx: LintContext,@@ -609,6 +668,7 @@ function lintOnePage();}}+ out.push(...undatedQuantityFindings(page, meta, body));out.push(...timeStageSync(ROUTE_STAGE.nearDuplicateLookup, () =>nearDuplicateFindings(ctx, page, meta, body),diff --git a/src/core/brain/pledge-intake.ts b/src/core/brain/pledge-intake.tsindex 4a76602f..8538d95e 100644--- a/src/core/brain/pledge-intake.ts+++ b/src/core/brain/pledge-intake.ts@@ -72,7 +72,8 @@ import {import { validateIsoDate } from "./paths.ts";import { acquireLockSyncWithRetry } from "./sync-lockfile.ts";import { assertVaultIdentityForWrite } from "./vault-identity.ts";-import { pageVisibility, strictestVisibilityOf } from "../graph/visibility.ts";+import { strictestVisibilityOf } from "../graph/visibility.ts";+import { remoteDenyVisibilityTokens } from "../graph/privacy-markers.ts";import { PROMISED_TO_RELATION } from "../graph/relation-vocab.ts";import { normalizeForQuoteComparison, quoteHaystack, spanOccursIn } from "./distill/quote-spans.ts";@@ -290,7 +291,11 @@ export function intakePledges(// visibility the created pages inherit. The write pass hands both to// `createPledgePage`, so a full batch never re-reads the source.const [sourceMeta] = parseFrontmatterText(sourceText);- const visibility = strictestVisibilityOf([pageVisibility(sourceMeta)]);+ // The full union, so a source reserving itself under an informal key+ // (`confidentiality: secret`) reserves the derived pledge pages too -+ // their evidence fields quote the source, so the quotes inherit the+ // reservation.+ const visibility = strictestVisibilityOf([remoteDenyVisibilityTokens(sourceMeta)]);const prepared: PreparedIntakeItem[] = [];const seenSlugs = new Set<string>();diff --git a/src/core/brain/pledges.ts b/src/core/brain/pledges.tsindex 8e013370..384c834b 100644--- a/src/core/brain/pledges.ts+++ b/src/core/brain/pledges.ts@@ -52,7 +52,8 @@ import {validateIsoDate,BRAIN_PLEDGES_REL,} from "./paths.ts";-import { pageVisibility, strictestVisibilityOf } from "../graph/visibility.ts";+import { strictestVisibilityOf } from "../graph/visibility.ts";+import { remoteDenyVisibilityTokens } from "../graph/privacy-markers.ts";import { PROMISED_TO_RELATION } from "../graph/relation-vocab.ts";import { assertVaultIdentityForWrite } from "./vault-identity.ts";import { acquireLockSyncWithRetry } from "./sync-lockfile.ts";@@ -474,7 +475,10 @@ export function addPledge(vault: string, input: AddPledgeInput): PledgePage {}const sourceText = readFileSync(sourceAbs, "utf8");const [sourceMeta] = parseFrontmatterText(sourceText);- const visibility = strictestVisibilityOf([pageVisibility(sourceMeta)]);+ // The full union, so a source reserving itself under an informal key+ // (`access: private`) reserves the derived pledge page too - the page's+ // evidence quotes the source, so the quote inherits the reservation.+ const visibility = strictestVisibilityOf([remoteDenyVisibilityTokens(sourceMeta)]);return createPledgePage(vault, {title,due,diff --git a/src/core/brain/portability/knowledge-pack.ts b/src/core/brain/portability/knowledge-pack.tsindex 15c56c52..5ee4f796 100644--- a/src/core/brain/portability/knowledge-pack.ts+++ b/src/core/brain/portability/knowledge-pack.ts@@ -46,6 +46,7 @@ import { atomicWriteFileSync } from "../../fs-atomic.ts";import { EXCLUDED_DIRS, parseFrontmatterText } from "../../vault.ts";import type { FrontmatterMap } from "../../types.ts";import { pageVisibility } from "../../graph/visibility.ts";+import { readContentClass } from "../provenance/content-class.ts";import {PRIVATE_REGION_PLACEHOLDER,REDACTION_PLACEHOLDER,@@ -207,7 +208,17 @@ export type KnowledgePackBlockReason =/** The entry carries an `owner:` claim - owner-private knowledge. */| "owner"/** The page is a staged import candidate nobody has reviewed. */- | "unreviewed";+ | "unreviewed"+ /**+ * The page carries the reserved `content_class: prompt-derived` stamp -+ * material an agent produced from a prompt, which the shared exports+ * leave out by default (the guard seam in `src/core/egress/guard.ts`).+ * A pack is handed to someone else, so it joins the block list beside+ * `visibility` and `owner` rather than composing the page and redacting+ * it after the fact; each block names the reason, so the omission is+ * never a silently smaller pack.+ */+ | "prompt-derived";export interface KnowledgePackBlocked {readonly kind: KnowledgePackEntryKind;@@ -351,6 +362,13 @@ function pageBlockReason(page: OkfPage): KnowledgePackBlockReason | null {if (pageVisibility(page.frontmatter).length > 0) return "visibility";const owner = page.frontmatter["owner"];if (typeof owner === "string" && owner.trim() !== "") return "owner";+ // The reserved class joins the operator-set scopes: a page stamped+ // `prompt-derived` is default-excluded from a shared export, and the+ // exclusion is named. First match wins, so an operator's own scoping+ // keeps its reason; a value this build does not know reads as unmarked+ // (the read side of `readContentClass` never throws), so a hand-edited+ // spelling cannot silently hide pages from every pack.+ if (readContentClass(page.frontmatter) !== null) return "prompt-derived";if (page.frontmatter["okf_review"] !== undefined) return "unreviewed";if (page.path === OKF_REVIEW_REL || page.path.startsWith(`${OKF_REVIEW_REL}/`)) {return "unreviewed";diff --git a/src/core/brain/portability/okf.ts b/src/core/brain/portability/okf.tsindex 5b3f482b..6d81e930 100644--- a/src/core/brain/portability/okf.ts+++ b/src/core/brain/portability/okf.ts@@ -104,6 +104,15 @@ const OKF_GENERATED_BY = `${OKF_PRODUCER}/${OPEN_SECOND_BRAIN_VERSION}`;*/export const OKF_MANIFEST_FILENAME = "okf.json";+/**+ * Bundle-relative filename of the human-readable index. Named for the+ * same reason as {@link OKF_MANIFEST_FILENAME}: the export verb+ * re-renders this one file from the REDACTED manifest after the guard+ * runs, and a reserved name spelled as a literal in a second module is+ * a name that can drift away from the writer that reserves it.+ */+export const OKF_INDEX_FILENAME = "index.md";+/** Vault-relative root the default (untrusted) import stages pages under. */export const OKF_REVIEW_REL = "OKF Review";@@ -532,7 +541,14 @@ function buildLog(vault: string): { contents: string; days: number } {return { contents: sections.join("\n\n"), days: bodiesByDate.size };}-function renderIndex(manifest: OkfManifest): string {+/**+ * Render the bundle-root `index.md` from a manifest. Exported because+ * the export verb re-renders the index from the REDACTED manifest after+ * the guard runs - an index naming a value the shipped `okf.json` no+ * longer carries (a redacted foreign type, an omitted page) would make+ * the bundle disagree with itself in the file a human reads first.+ */+export function renderIndex(manifest: OkfManifest): string {const lines: string[] = ["---",// OKF v0.2's version declaration. Per the spec it is a MAY in the@@ -766,6 +782,25 @@ export function buildOkfSubsetBundle(vault: string, pages: ReadonlyArray<OkfPagereturn assembleOkfBundle(vault, pages, null);}+/**+ * Build a full OKF bundle write-plan - change log included - over+ * caller-collected pages, composing from the array as given instead of+ * walking the vault again. {@link buildOkfSubsetBundle} is the existing+ * page-input entry point, but a subset bundle carries no `log.md` by+ * contract, so a caller that has ALREADY walked the vault (the shared+ * export verbs, which split the collected pages through the egress+ * content-class policy before composition) had no way to keep the+ * change log without a second walk. Composing from the caller's page+ * array closes that double-walk window: between the caller's walk and a+ * re-walk here, a page can change or appear, and the bundle would mix+ * the two reads - or, with the class policy applied to the first walk+ * only, re-admit a page the policy had just left out. The pages are+ * written as given; only the log is read from the vault here.+ */+export function buildOkfBundleFromPages(vault: string, pages: ReadonlyArray<OkfPage>): OkfBundle {+ return assembleOkfBundle(vault, pages, buildLog(vault));+}+function assembleOkfBundle(vault: string,pages: ReadonlyArray<OkfPage>,@@ -801,7 +836,7 @@ function assembleOkfBundle(const files: OkfBundleFile[] = [{ path: OKF_MANIFEST_FILENAME, contents: renderOkfManifest(manifest) },- { path: "index.md", contents: renderIndex(manifest) },+ { path: OKF_INDEX_FILENAME, contents: renderIndex(manifest) },];if (log !== null) files.push({ path: "log.md", contents: log.contents });for (const { page, sources } of derived) {@@ -947,7 +982,7 @@ const SPEC_DIR_CLASS: Readonly<Record<string, OkfPageClass>> = Object.freeze(* page's full bundle-relative path, lowercased - never the basename,* which would skip real pages at any depth.*/-const SPEC_RESERVED_PATHS: ReadonlySet<string> = new Set(["index.md", "log.md"]);+const SPEC_RESERVED_PATHS: ReadonlySet<string> = new Set([OKF_INDEX_FILENAME, "log.md"]);/*** The producer label a manifest-less spec bundle reports: there is nodiff --git a/src/core/brain/privacy-marker-restamp.ts b/src/core/brain/privacy-marker-restamp.tsnew file mode 100644index 00000000..671e5aa8--- /dev/null+++ b/src/core/brain/privacy-marker-restamp.ts@@ -0,0 +1,320 @@+/**+ * Preview-first restamp of informal privacy markers (write-honesty, F1).+ *+ * The read roots deny a remote read for a page saying `access: private`,+ * `confidentiality: secret` or `sensitivity: confidential` (see+ * `graph/privacy-markers.ts`), so those pages are already withheld from a+ * remote caller. What they do NOT carry is the formal `visibility:`+ * declaration, and every consumer that reads that key directly - derived+ * pages stamped through `strictestVisibility`, an export filtering on the+ * token, an operator auditing reservations - cannot see the claim.+ *+ * This verb makes the claim formal, and it does so the way the repair-verb+ * family already does (`authored-at-backfill.ts` is the template):+ *+ * - DRY RUN BY DEFAULT. `planPrivacyMarkerRestamp` never writes; the+ * caller opts into mutation with `{ apply: true }`.+ * - ADDITIVE ONLY. The reserved token is unioned into `visibility:`.+ * No key is removed, no declared token is dropped, and a page already+ * carrying the reserved token is not a candidate at all.+ * - WRITTEN FROM A RE-READ. The apply loop re-parses each candidate+ * immediately before its write and recomputes the token list from+ * that parse, so an edit that landed between the scan and the write+ * is honoured, and a page whose marker vanished is skipped (it stays+ * in the reported candidate list, which is what the scan found).+ * The re-parse is the notice-carrying one: a page whose frontmatter+ * carries a line the scanner drops would lose that line in a+ * rewrite, so the apply pass refuses it by name (reported in+ * `skipped`, with the reason) and leaves its bytes untouched. The+ * raw block is checked too, for the losses the scanner does not+ * report: a `#` comment line and an indented nested mapping are+ * skipped or flattened without a notice, and a rewrite would lose+ * them all the same.+ * - IDEMPOTENT. A re-run over a restamped vault reports no candidates+ * and writes nothing, byte for byte.+ * - WRITE-GUARDED AT THE ENTRY POINT, ONLY WHEN THE CALL WILL WRITE.+ * A preview is the operator's way to find out what is there, so it+ * runs against a root the guard would refuse; an apply asserts+ * `assertVaultIdentityForWrite` before touching the first file.+ * - NO RE-EMBEDDING. Only frontmatter is edited; the body and the search+ * index are untouched, and the index picks the change up on its next+ * normal run.+ *+ * The scan is the vault walk every root shares (`listVaultPages`), so a+ * page the walk skips - `.git`, `.obsidian`, a generated `index.md` - is+ * not restamped here either. Paths in the result are vault-relative with+ * `/` separators, so a preview reads the same on every platform.+ */++import { readFileSync } from "node:fs";+import { join } from "node:path";++import {+ FRONTMATTER_RE,+ listVaultPages,+ parseFrontmatterWithNotices,+ writeFrontmatterAtomic,+} from "../vault.ts";+import { DEGRADATION_CODE } from "../integrity/degradation.ts";+import { vaultRelative } from "../path-safety.ts";+import { MAINTENANCE_LANE_REACH } from "../graph/transport-reach.ts";+import { REMOTE_DENY_VISIBILITY_TOKEN, normToken } from "../graph/visibility.ts";+import { hasPrivacyDenyMarker } from "../graph/privacy-markers.ts";+import type { FrontmatterMap } from "../types.ts";+import { assertVaultIdentityForWrite } from "./vault-identity.ts";++/** U+FEFF, which may precede a page's opening fence (see vault.ts). */+const BYTE_ORDER_MARK = 0xfeff;++/** One page whose informal marker is not yet a formal reservation. */+export interface PrivacyMarkerRestampCandidate {+ /** Vault-relative path with `/` separators. */+ readonly path: string;+ /** The `visibility:` tokens the page would carry after the restamp. */+ readonly visibility: ReadonlyArray<string>;+ /** The frontmatter line those tokens render to, for a preview reader. */+ readonly visibilityLine: string;+}++export interface PrivacyMarkerRestampResult {+ /** Whether the run mutated files (`false` for a dry run). */+ readonly applied: boolean;+ /** Pages carrying an informal deny marker (candidates and stamped ones). */+ readonly scanned: number;+ /** Marker pages whose `visibility:` lacks the reserved token. */+ readonly candidates: ReadonlyArray<PrivacyMarkerRestampCandidate>;+ /** Files actually rewritten (0 on a dry run). */+ readonly updated: number;+ /**+ * Candidates the apply pass REFUSED to rewrite, by name, with the+ * reason. A page whose frontmatter parse reports dropped lines stays+ * untouched: the line-based scanner cannot round-trip those lines, so+ * a rewrite would silently drop them and break the no-key-removed+ * promise. Empty on a dry run; the candidate list still shows what the+ * scan found.+ */+ readonly skipped: ReadonlyArray<PrivacyMarkerRestampSkip>;+}++/** One page the apply pass refused, and why. */+export interface PrivacyMarkerRestampSkip {+ /** Vault-relative path with `/` separators. */+ readonly path: string;+ /** Why the page was not rewritten. */+ readonly reason: string;+}++export interface PrivacyMarkerRestampOptions {+ /**+ * When true, rewrite each candidate's frontmatter with the reserved+ * token unioned into `visibility:`. Default false - a pure preview.+ */+ readonly apply?: boolean;+}++/**+ * The page's declared `visibility:` tokens as WRITTEN: a string, or an+ * array, trimmed and with empty entries dropped. Unlike `pageVisibility`+ * this does not lower-case anything, because the restamp must not rewrite+ * a declared token's spelling on its way to adding one.+ */+function declaredTokens(meta: FrontmatterMap): string[] {+ const raw: unknown = meta["visibility"];+ const values: ReadonlyArray<unknown> = Array.isArray(raw)+ ? raw+ : raw === undefined || raw === null+ ? []+ : [raw];+ const out: string[] = [];+ for (const value of values) {+ if (typeof value !== "string") continue;+ const trimmed = value.trim();+ if (trimmed.length > 0) out.push(trimmed);+ }+ return out;+}++/** True when `tokens` already declares the reserved token, however spelled. */+function declaresReserved(tokens: ReadonlyArray<string>): boolean {+ return tokens.some((token) => normToken(token) === REMOTE_DENY_VISIBILITY_TOKEN);+}++/**+ * A frontmatter-safe rendering of ONE visibility token, matching the+ * writer's scalar rule closely enough for a preview: a plain identifier+ * renders bare, anything else is JSON-quoted.+ */+function renderToken(token: string): string {+ return /^[A-Za-z0-9_][\w.:/@+-]*$/.test(token) ? token : JSON.stringify(token);+}++/** The would-be frontmatter line for a token list. */+function renderVisibilityLine(tokens: ReadonlyArray<string>): string {+ return `visibility: [${tokens.map(renderToken).join(", ")}]`;+}++/** The absolute path of a vault-relative, `/`-separated candidate path. */+function absolutePath(vault: string, relPath: string): string {+ return join(vault, ...relPath.split("/"));+}++/**+ * The first line of the raw frontmatter block that a rewrite through the+ * writer cannot round-trip even though the scanner raised NO notice - so+ * the refuse-on-notice guard cannot see it. The scanner skips a `#`+ * comment line silently (the format documents comments as content it+ * discards) and trims each line before matching key/value, so an indented+ * nested mapping is silently flattened into top-level keys. Either way a+ * rewrite re-serialises the parsed map and the original bytes are gone:+ * a comment is deleted, a nested map is flattened. Spelled from the raw+ * text, not from the parsed map, because the loss is in the bytes the+ * parse throws away.+ */+function unroundtrippableFrontmatterLine(raw: string): string | null {+ const text = raw.charCodeAt(0) === BYTE_ORDER_MARK ? raw.slice(1) : raw;+ const match = FRONTMATTER_RE.exec(text);+ if (match === null) return null;+ const lines = match[1]!.split("\n");+ for (let i = 0; i < lines.length; i++) {+ const line = lines[i]!;+ const trimmed = line.trim();+ if (trimmed === "") continue;+ if (trimmed.startsWith("#")) return `frontmatter line ${i + 1} is a comment`;+ if (/^\s/.test(line)) return `frontmatter line ${i + 1} is indented`;+ }+ return null;+}++/**+ * Scan the vault for informal privacy markers that are not formally+ * reserved. When `apply` is set, union the reserved token into each+ * candidate's `visibility:` through {@link writeFrontmatterAtomic};+ * otherwise report the candidates and write nothing.+ */+export function planPrivacyMarkerRestamp(+ vault: string,+ opts: PrivacyMarkerRestampOptions = {},+): PrivacyMarkerRestampResult {+ const apply = opts.apply === true;+ // The write guard is a WRITE guard: a preview writes nothing, so it is+ // not gated. Asserted before the walk so a refusal cannot land after+ // the first byte.+ if (apply) assertVaultIdentityForWrite(vault);++ const candidates: PrivacyMarkerRestampCandidate[] = [];+ let scanned = 0;++ for (const page of listVaultPages(vault, { reach: MAINTENANCE_LANE_REACH })) {+ if (!hasPrivacyDenyMarker(page.metadata)) continue;+ scanned++;+ const declared = declaredTokens(page.metadata);+ // Already formal: the marker adds nothing, so the page is not a+ // candidate and the run stays a no-op on it.+ if (declaresReserved(declared)) continue;+ const visibility = Object.freeze([...declared, REMOTE_DENY_VISIBILITY_TOKEN]);+ candidates.push(+ Object.freeze({+ path: vaultRelative(page.path, vault),+ visibility,+ visibilityLine: renderVisibilityLine(visibility),+ }),+ );+ }+ candidates.sort((a, b) => (a.path < b.path ? -1 : a.path > b.path ? 1 : 0));++ let updated = 0;+ const skipped: PrivacyMarkerRestampSkip[] = [];+ if (apply) {+ for (const candidate of candidates) {+ const absolute = absolutePath(vault, candidate.path);+ // Write-time recompute: the candidate list describes the vault as+ // the scan saw it, but the page is re-parsed here, immediately+ // before its write, and an edit that landed in between (operator,+ // sync) is the state this write is responsible for. The tokens are+ // therefore recomputed from the just-parsed frontmatter - the+ // declared tokens as they stand now, plus the reserved token while+ // the marker is still present - never copied off the scanned+ // candidate, which would clobber a token the edit declared. A page+ // whose marker vanished is no longer a reservation this run owes:+ // it is skipped, while staying in the reported candidate list,+ // which is what the scan found. The already-reserved case skips+ // for the same reason the scan never nominated such a page.+ //+ // The re-parse is the NOTICE-CARRYING one, and a notice skips the+ // page by name: the scanner is line-based, so a line it drops (a+ // folded block scalar, a stray dash item) cannot round-trip through+ // the writer - rewriting would silently lose it and break the+ // no-key-removed promise above. The page keeps its informal marker+ // (the read roots still deny it) and stays in the candidate list;+ // the refusal is reported, never silent.+ const [meta, body, parseNotices] = parseFrontmatterWithNotices(absolute);+ if (parseNotices.length > 0) {+ // The reason names the failure the page actually has: a notice+ // whose code is `frontmatter-unreadable` is a READ failure (the+ // file was deleted or moved between the scan and the write), not+ // a YAML problem, and saying "dropped line" would send the+ // operator looking at frontmatter the page does not have.+ const first = parseNotices[0]!;+ skipped.push(+ Object.freeze({+ path: candidate.path,+ reason:+ first.code === DEGRADATION_CODE.frontmatterUnreadable+ ? `page could not be re-read before the write (${first.detail})`+ : parseNotices.length === 1+ ? `frontmatter parse dropped 1 line; rewriting would lose it (${first.code})`+ : `frontmatter parse dropped ${parseNotices.length} lines; rewriting would lose them (${first.code})`,+ }),+ );+ continue;+ }+ // The notice guard above covers the lines the scanner REPORTS. A+ // comment line and an indented nested mapping are lost silently -+ // the scanner skips or flattens them without a notice - so the raw+ // block is checked for them here, before any write. The re-read is+ // guarded on its own: a page that vanished between the parse above+ // and this read is refused as unreadable rather than crashing the+ // apply pass.+ let raw: string;+ try {+ raw = readFileSync(absolute, "utf8");+ } catch (err) {+ const code = (err as NodeJS.ErrnoException).code ?? String(err);+ skipped.push(+ Object.freeze({+ path: candidate.path,+ reason: `page could not be re-read before the write (${code})`,+ }),+ );+ continue;+ }+ const loss = unroundtrippableFrontmatterLine(raw);+ if (loss !== null) {+ skipped.push(+ Object.freeze({+ path: candidate.path,+ reason: `${loss}; the writer cannot round-trip it, so rewriting would lose it`,+ }),+ );+ continue;+ }+ if (!hasPrivacyDenyMarker(meta)) continue;+ const declared = declaredTokens(meta);+ if (declaresReserved(declared)) continue;+ const next: FrontmatterMap = {+ ...meta,+ visibility: Object.freeze([...declared, REMOTE_DENY_VISIBILITY_TOKEN]),+ };+ writeFrontmatterAtomic(absolute, next, body, { overwrite: true });+ updated++;+ }+ }++ return Object.freeze({+ applied: apply,+ scanned,+ candidates: Object.freeze(candidates),+ updated,+ skipped: Object.freeze(skipped),+ });+}diff --git a/src/core/brain/provenance/content-class.ts b/src/core/brain/provenance/content-class.tsnew file mode 100644index 00000000..6cd7bc89--- /dev/null+++ b/src/core/brain/provenance/content-class.ts@@ -0,0 +1,282 @@+/**+ * The content class - what kind of process produced a page's text.+ *+ * ## One member, because there is exactly one decision+ *+ * `content_class` is the sibling of `origin_channel`+ * (`src/core/origin-channel.ts`) on a different axis: the channel says+ * which process put a record in the vault, the class says what kind of+ * material the record carries. The vocabulary is closed and holds ONE+ * member, `prompt-derived` - text an agent produced from a prompt. One+ * member is enough because the stamp exists for exactly one decision:+ * shared exports leave such pages out by default (the guard seam in+ * `src/core/egress/guard.ts`), and a per-verb operator override names+ * them back in. A second member would need a second decision, and no+ * second decision is implemented; the vocabulary grows the day one is.+ *+ * ## Server-derived, like the channel it sits beside+ *+ * The class is stamped by the writer that knows the material is+ * prompt-derived, never named by the caller that sends it. Creation+ * stamps it when the write arrives over the MCP transport (an agent+ * dictated the page), and the deterministic generation pipelines - the+ * session import and the in-process report writers - stamp their own+ * output. Operator-authored CLI writes stay unmarked. A caller-supplied+ * key is dropped at the merge: a class a caller could name would be the+ * same unverifiable claim a caller-named channel would be, and the+ * census that pins the channel (`origin-channel-census.test.ts`) is why+ * this module's spellings avoid that field's every spelling.+ *+ * ## No backfill, and absence is a meaning+ *+ * Pages written before this shipped carry no key and are NOT rewritten+ * to add one, for the same reason the channel is not backfilled: a stamp+ * applied by a later pass describes the pass, not the writer. Absence+ * reads as unmarked, and the era boundary is documented rather than+ * migrated - a pre-stamp page keeps exporting exactly as it did.+ *+ * ## Two signal writers carry no stamp, on purpose+ *+ * The agent-authored signal writers behind `session-checkpoint.ts` and+ * `session-lifecycle.ts` stamp nothing. That is a NAMED SCOPING+ * DECISION, not an omission: their pages are written under `Brain/`+ * (the inbox and its lanes), and every page-carrying export walk in+ * this system excludes Brain-internal pages at the source - the OKF+ * walk and the knowledge pack compose from `collectOkfPages` (ordinary+ * pages plus the `Brain/sources/` and `Brain/reports/` lanes only), the+ * graph export skips the `Brain/` root, and the bank bundle's walk+ * skips it at the top level. A page those writers produce cannot reach+ * a shared export, so there is no egress decision for the stamp to+ * carry; the writers are left untouched rather than stamped for a leak+ * that has no path. The day an export walk starts carrying Brain-+ * internal pages, that walk is the change this module must hear about.+ *+ * ## A closed vocabulary meeting a hand editor+ *+ * On disk the key is frontmatter an operator can edit. The narrowing+ * guard refuses unknown values where a stamp is RESOLVED (a write path+ * cannot be talked into inventing a class), while the read side treats+ * any value other than the reserved member as unmarked rather than+ * throwing: a page marked by hand with a spelling this build does not+ * know is not evidence of anything, and inventing an exclusion for it+ * would let one typo silently hide pages from every shared export.+ * Hand-set scoping already has a key - `visibility` - and this axis is+ * not a second one.+ */++import { readFileSync } from "node:fs";+import { join } from "node:path";++import { atomicWriteFileSync } from "../../fs-atomic.ts";+import type { FrontmatterMap } from "../../types.ts";+import { ORIGIN_CHANNEL, type OriginChannelStamp } from "../../origin-channel.ts";+// The block-boundary regex the shared parser itself closes at, and the+// byte-order mark its text entry strips before matching: the stamp's+// insertion point and the parser's read boundary are the same decision,+// so they share the constant rather than two spellings of it.+import { FRONTMATTER_RE, parseFrontmatterText } from "../../vault.ts";++/** U+FEFF, which may precede a page's opening fence (see vault.ts). */+const BYTE_ORDER_MARK = 0xfeff;++/**+ * The closed vocabulary. `prompt-derived` names material an agent+ * produced from a prompt: session imports, MCP note writes, and the+ * in-process generation pipelines. The value is an on-disk spelling and+ * is load-bearing; it deliberately shares no substring with the origin+ * channel's field or values, so the two axes cannot be folded by a grep.+ */+export const CONTENT_CLASS = Object.freeze({+ promptDerived: "prompt-derived",+} as const);++export type ContentClass = (typeof CONTENT_CLASS)[keyof typeof CONTENT_CLASS];++/**+ * The membership list, exported so a refusal can NAME the vocabulary+ * instead of restating it - the same rule `ORIGIN_CHANNELS` follows.+ */+export const CONTENT_CLASSES: ReadonlyArray<ContentClass> = Object.freeze(+ Object.values(CONTENT_CLASS),+);++/**+ * The on-disk key. One constant because the stamp, the reserved-update+ * guard and the egress filter all spell it; a key that drifted between+ * them would be a key no reader could query on.+ */+export const CONTENT_CLASS_FIELD = "content_class";++/** Takes `unknown`: a value can arrive from frontmatter or a caller. */+export function isContentClass(value: unknown): value is ContentClass {+ return typeof value === "string" && (CONTENT_CLASSES as ReadonlyArray<string>).includes(value);+}++/** Raised by {@link resolveContentClass} for a value outside the vocabulary. */+export class ContentClassValueError extends Error {+ constructor(value: unknown) {+ super(+ `resolveContentClass: ${JSON.stringify(value)} is not a content class; ` ++ `the vocabulary is ${CONTENT_CLASSES.join(", ")}. A class is stamped by the ` ++ "writer that knows the material, never named by a caller.",+ );+ this.name = "ContentClassValueError";+ }+}++/**+ * The strict read: the class, or a named refusal. Write paths resolving+ * a class they are about to stamp go through here, so an internal+ * mistake surfaces by name instead of inventing an on-disk value.+ */+export function resolveContentClass(value: unknown): ContentClass {+ if (!isContentClass(value)) throw new ContentClassValueError(value);+ return value;+}++/**+ * The read side: the class a page carries, or `null` when it is+ * unmarked. Absence is the pre-stamp era and reads as unmarked; a value+ * outside the vocabulary reads as unmarked too (see the module+ * docblock). Never throws - this reads a disk an operator can edit.+ */+export function readContentClass(meta: FrontmatterMap): ContentClass | null {+ const value = meta[CONTENT_CLASS_FIELD];+ return isContentClass(value) ? value : null;+}++/**+ * The creation stamp for a write whose server-derived channel is known:+ * `prompt-derived` for material that arrived over the MCP transport,+ * `null` - stay unmarked, write no key - for everything else.+ *+ * The channel is the ONLY input. A CLI write and an unclaimed process+ * are the same answer, because the question is about the material, and+ * the channel is the one fact the process cannot be lied to about.+ */+export function contentClassCreationStamp(channel: OriginChannelStamp): ContentClass | null {+ return channel === ORIGIN_CHANNEL.mcpTool ? CONTENT_CLASS.promptDerived : null;+}++/**+ * Drop a caller-named class from frontmatter about to be written. Used+ * at the creation merge so a caller cannot forge the key on the way in;+ * the stamp (when there is one) is merged after it and wins regardless,+ * and on an unmarked write the key is simply absent - the honest byte.+ */+export function withoutCallerContentClass(frontmatter: FrontmatterMap): FrontmatterMap {+ if (frontmatter[CONTENT_CLASS_FIELD] === undefined) return frontmatter;+ const rest = { ...frontmatter };+ delete rest[CONTENT_CLASS_FIELD];+ return rest;+}++/**+ * Stamp `content_class: prompt-derived` onto a page an in-process+ * pipeline has just written.+ *+ * This exists for the generation writers whose bytes are composed by+ * their own writer function (the signal writer, for one, renders its+ * frontmatter internally and exposes no extension point), so the stamp+ * cannot ride the frontmatter map the way it does in `createNote`. It is+ * still a CREATION-time stamp: the caller invokes it on the same run+ * that wrote the page, the line is inserted as text so every other byte+ * of the just-written document survives untouched, the write is atomic,+ * and a page already carrying the class is a no-op. A file without a+ * frontmatter block is a named refusal rather than a silent miss - the+ * one state where the stamp could be quietly lost is the one state this+ * refuses to paper over.+ *+ * The insertion point is the block boundary the SHARED PARSER closes at+ * ({@link FRONTMATTER_RE}), never a raw fence search: a close fence that+ * carries trailing whitespace - or ends the file without a newline -+ * is a close the parser accepts and a `"\n---\n"` search misses, and a+ * search that misses runs on to the next bare `---` line in the page,+ * planting the stamp outside the parsed block where the page reads+ * unmarked. Where the stamp lands is a question with one right answer,+ * and it is whatever `parseFrontmatterText` would answer.+ */+export function stampPromptDerivedLine(abs: string): void {+ const raw = readFileSync(abs, "utf8");+ // The same byte-order-mark strip the parser applies, spliced back in+ // around the write so a BOM'd page keeps its first byte.+ const stripped = raw.charCodeAt(0) === BYTE_ORDER_MARK ? raw.slice(1) : raw;+ const prefix = raw.length - stripped.length;+ const match = FRONTMATTER_RE.exec(stripped);+ if (match === null) {+ throw new Error(`stampPromptDerivedLine: ${abs} has no frontmatter block to stamp`);+ }+ // The idempotency guard reads the RAW key, not `readContentClass`: the+ // read side maps an out-of-vocabulary value to unmarked (that is its+ // documented posture toward a hand edit or a newer build's spelling),+ // but the stamp must not read that as "no key" - planting a second+ // `content_class:` line gives the frontmatter a duplicate key, which+ // strict YAML consumers reject. A page carrying the reserved member is+ // the byte-level no-op; a page carrying anything ELSE under the key is+ // a state this stamp refuses to overwrite, named, and leaves untouched.+ const existing = parseFrontmatterText(stripped)[0][CONTENT_CLASS_FIELD];+ if (existing === CONTENT_CLASS.promptDerived) return;+ if (existing !== undefined) {+ throw new Error(+ `stampPromptDerivedLine: ${abs} already carries ${CONTENT_CLASS_FIELD}: ` ++ `${JSON.stringify(existing)}; the stamp writes only an unmarked page`,+ );+ }+ // The block match ends with the close fence (`\n---` plus any+ // trailing whitespace), and the close is the only `\n---` inside the+ // match - the lazy body stops at the first one. The line goes right+ // after the newline that precedes that fence: the last line INSIDE+ // the block, wherever the fence's own line ends.+ const closeStart = match[0].lastIndexOf("\n---");+ if (closeStart < 0) {+ throw new Error(`stampPromptDerivedLine: ${abs} has a frontmatter block with no close fence`);+ }+ const at = prefix + closeStart + 1;+ const line = `${CONTENT_CLASS_FIELD}: ${CONTENT_CLASS.promptDerived}\n`;+ atomicWriteFileSync(abs, `${raw.slice(0, at)}${line}${raw.slice(at)}`);+}++/**+ * Whether the page at `vault`-relative `relPath` carries the reserved+ * class. The predicate the export verbs hand to the guard's+ * `egressContentClassSplit`: one page, one frontmatter read. A page that+ * cannot be read (deleted between collection and the split, or never a+ * page at all) reads as unmarked - the split omits what it can PROVE is+ * marked, and everything else stays on the export path it was on.+ */+export function isPromptDerivedPage(vault: string, relPath: string): boolean {+ let text: string;+ try {+ text = readFileSync(join(vault, relPath), "utf8");+ } catch {+ return false;+ }+ return readContentClass(parseFrontmatterText(text)[0]) === CONTENT_CLASS.promptDerived;+}++/** How deep {@link treeCarriesPromptDerived} descends before it stops. */+const TREE_SEARCH_DEPTH = 8;++/**+ * Whether a JSON-shaped tree carries the reserved class anywhere - the+ * tripwire the transcript export walks each streamed record through.+ * Transcript records have no frontmatter and carry no class today, and+ * the day a producer adds one this is what keeps the corpus from+ * silently shipping (or silently emptying): the record is omitted and+ * named, or the export refuses. Depth-bounded so a pathological tree+ * cannot turn the check into a walk of unbounded cost; a class marker+ * deeper than that is not a record shape, it is data.+ */+export function treeCarriesPromptDerived(value: unknown, depth = 0): boolean {+ if (depth > TREE_SEARCH_DEPTH) return false;+ if (Array.isArray(value))+ return value.some((entry) => treeCarriesPromptDerived(entry, depth + 1));+ if (value !== null && typeof value === "object") {+ const record = value as Record<string, unknown>;+ // The key names the field; a bare string elsewhere in the tree is+ // content, not a claim about itself.+ if (readContentClass(record as FrontmatterMap) !== null) return true;+ return Object.values(record).some((entry) => treeCarriesPromptDerived(entry, depth + 1));+ }+ return false;+}diff --git a/src/core/brain/research/research.ts b/src/core/brain/research/research.tsindex e8c19abe..56dff439 100644--- a/src/core/brain/research/research.ts+++ b/src/core/brain/research/research.ts@@ -33,6 +33,7 @@ import {import { assertVaultIdentityForWrite } from "../vault-identity.ts";import { isSecretReferenceValue, resolveNamedSecret } from "../../secret-resolver.ts";import { renderProvenanceSection, type Provenance } from "../provenance/provenance.ts";+import { CONTENT_CLASS, CONTENT_CLASS_FIELD } from "../provenance/content-class.ts";import {CAPTURE_SCOPE,captureScopesFrontmatter,@@ -261,6 +262,12 @@ export function writeResearchReport(// Written only when some source is not a local file, so a report built// from vault files alone stays byte-identical to before....captureScopesFrontmatter(captureScopes),+ // Server-derived content class (F5): a research report is prompt-+ // derived material by definition - the pipeline synthesised it from+ // provider runs - so the writer that knows the material stamps it+ // here, at creation. The shared exports leave such pages out by+ // default and name the omission.+ [CONTENT_CLASS_FIELD]: CONTENT_CLASS.promptDerived,created_at: stamp,updated_at: stamp,tags: ["brain", "brain/report"],diff --git a/src/core/brain/sessions/import.ts b/src/core/brain/sessions/import.tsindex e48c0352..cb6349fa 100644--- a/src/core/brain/sessions/import.ts+++ b/src/core/brain/sessions/import.ts@@ -68,6 +68,7 @@ import { censusSessionSignals, type ImportCensus } from "../import-census.ts";import { appendContinuityRecord } from "../continuity/store.ts";import { discoverMarkersDetailed, isFeedbackMarker } from "../inline.ts";import { writeSignal } from "../signal.ts";+import { stampPromptDerivedLine } from "../provenance/content-class.ts";import { importSessionRecall } from "../session-recall.ts";import { isoDate, isoSecond } from "../time.ts";import { BRAIN_SIGNAL_SOURCE_TYPE } from "../types.ts";@@ -484,6 +485,30 @@ export async function importSession(: {}),...(opts.rawCodec === true ? { rawCodec: true } : {}),});+ // A session-replayed signal is prompt-derived material by+ // definition - it is a turn of an agent conversation - so the+ // content-class stamp rides the same run that wrote it (F5). The+ // signal writer composes its own frontmatter and exposes no+ // extension point, so the stamp lands as a line insert on the+ // bytes this call just wrote; a deduped write (empty path) has+ // nothing new to stamp. The stamp failing never loses the signal:+ // the error surfaces by name in this import's errors, separate+ // from the writeSignal arm so neither stage can be mistaken for+ // the other.+ if (res.path !== "") {+ try {+ stampPromptDerivedLine(res.path);+ } catch (err) {+ // The row names `res.path` - the signal page the failure is+ // about - never `absPath`, which is the transcript this import+ // only read. An operator following the row must land on the+ // page that went unstamped.+ errors.push({+ path: res.path,+ message: `content-class stamp failed: ${(err as Error).message ?? String(err)}`,+ });+ }+ }dedup.set(input.dedupHash, { id: res.id, path: res.path });createdHashes.push(input.dedupHash);signalsCreated++;diff --git a/src/core/brain/tensions.ts b/src/core/brain/tensions.tsindex 888a47e5..914628cf 100644--- a/src/core/brain/tensions.ts+++ b/src/core/brain/tensions.ts@@ -34,6 +34,7 @@ import { resolveAgentName } from "../config.ts";import { atomicWriteFileSync } from "../fs-atomic.ts";import { sanitiseTextField } from "../redactor.ts";import { pageVisibility, strictestVisibility } from "../graph/visibility.ts";+import { remoteDenyVisibilityTokens } from "../graph/privacy-markers.ts";import type { FrontmatterMap } from "../types.ts";import { parseFrontmatter } from "../vault.ts";import {@@ -607,8 +608,13 @@ export function detectTensionsInVault(typeof rawSubject === "string" && rawSubject.trim() !== "" ? rawSubject.trim() : undefined;notes.push({ id, ...(subject !== undefined ? { subject } : {}), text: body });// Two notes may share a frontmatter id; the id then reads under the- // stricter of their rules.- visibilityById.set(id, strictestVisibility(visibilityById.get(id) ?? [], pageVisibility(meta)));+ // stricter of their rules. The fold is the full union, so a note+ // reserving itself under an informal key (`access: private`) reserves+ // every tension page drafted from it, not just a formally declared one.+ visibilityById.set(+ id,+ strictestVisibility(visibilityById.get(id) ?? [], [...remoteDenyVisibilityTokens(meta)]),+ );}const result = detectTensions(vault, notes, {diff --git a/src/core/brain/undated-quantity.ts b/src/core/brain/undated-quantity.tsnew file mode 100644index 00000000..ea3fe899--- /dev/null+++ b/src/core/brain/undated-quantity.ts@@ -0,0 +1,397 @@+/**+ * Undated-quantity recognizer (write-honesty, F4, t_fb442a24).+ *+ * A count or amount written with no date becomes a claim that answers as+ * true at every instant: `isValidAt` treats an absent `valid_from` as+ * unbounded, so `5,000 users` written last year reads identically today.+ * This module owns the RECOGNITION half only - the write-time lint+ * (`page-lint.ts`) decides the escape hatches and emits the finding, and+ * the claims surface is where a reader places the claim in time.+ *+ * ## Structural shapes only+ *+ * The vocabulary is structural, matching the invariant+ * `search/temporal-intent.ts` states for the read side: no natural-language+ * date or unit phrase is recognised. The trigger classes are the four+ * high-precision shapes a writer actually types -+ *+ * - a currency amount, by symbol (`$4.2M`) or ISO code (`10 USD`);+ * - a percentage (`35%`);+ * - a number with a unit from the declared structural set (`359 ms`,+ * `5,000 users`, `512 MB`);+ * - a bare thousands-separated count (`12,400`).+ *+ * Everything else is left alone. A number inside an ISO date or instant, a+ * version string (`v1.73.0`), an ordered-list marker, a table row, an+ * inline code span, a fenced block and the frontmatter block are masked+ * before the scan, because a number there is not a claim about the vault.+ * Precision is deliberate: a missed quantity costs one advisory, a+ * hallucinated one teaches the operator to ignore the list.+ *+ * Pure and deterministic: the same body yields the same lexemes and+ * offsets. An offset is a UTF-16 code-unit index into the caller's own+ * string.+ */++/** One quantity the authored body states without a date. */+export interface UndatedQuantity {+ /** The matched text exactly as written. */+ readonly lexeme: string;+ /** UTF-16 code-unit offset of the first character of {@link lexeme}. */+ readonly offset: number;+}++/**+ * Unit words and symbols that make a preceding number a quantity. A closed+ * structural set, not a language model: an unknown word after a number is+ * not a unit. Longest-first matching is built from this list, so `seconds`+ * is never read as `s` followed by prose.+ */+const QUANTITY_UNITS: ReadonlyArray<string> = Object.freeze([+ // Duration.+ "ms",+ "s",+ "sec",+ "secs",+ "second",+ "seconds",+ "min",+ "mins",+ "minute",+ "minutes",+ "h",+ "hr",+ "hrs",+ "hour",+ "hours",+ "d",+ "day",+ "days",+ "wk",+ "wks",+ "week",+ "weeks",+ "mo",+ "mos",+ "month",+ "months",+ "yr",+ "yrs",+ "year",+ "years",+ // Data size.+ "b",+ "byte",+ "bytes",+ "kb",+ "mb",+ "gb",+ "tb",+ "kib",+ "mib",+ "gib",+ "tib",+ "kb/s",+ "mb/s",+ "gb/s",+ // Counts and throughput.+ "user",+ "users",+ "item",+ "items",+ "row",+ "rows",+ "page",+ "pages",+ "file",+ "files",+ "request",+ "requests",+ "req",+ "reqs",+ "rps",+ "qps",+ "op",+ "ops",+ "call",+ "calls",+ "token",+ "tokens",+ "char",+ "chars",+ "character",+ "characters",+ "word",+ "words",+ "line",+ "lines",+ "node",+ "nodes",+ "edge",+ "edges",+ "point",+ "points",+ "record",+ "records",+ "entry",+ "entries",+ "result",+ "results",+ "hit",+ "hits",+ "download",+ "downloads",+ "install",+ "installs",+ "visitor",+ "visitors",+ "view",+ "views",+ "session",+ "sessions",+ "signal",+ "signals",+ "note",+ "notes",+ "preference",+ "preferences",+ // Ratio.+ "%",+]);++/** ISO currency codes admitted after an amount (`10 USD`). */+const CURRENCY_CODES: ReadonlyArray<string> = Object.freeze([+ "USD",+ "EUR",+ "GBP",+ "JPY",+ "CHF",+ "CAD",+ "AUD",+ "CNY",+ "INR",+ "SEK",+ "NOK",+ "PLN",+]);++/** Regex-escape one literal fragment used to build a pattern. */+function escapePattern(text: string): string {+ return text.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");+}++/** Longest-first alternation, so a suffix unit cannot shadow a longer one. */+function alternation(values: ReadonlyArray<string>): string {+ return [...values]+ .toSorted((a, b) => b.length - a.length || a.localeCompare(b))+ .map(escapePattern)+ .join("|");+}++/**+ * ISO calendar-date run in prose. The digit and hyphen boundaries are the+ * same guard `search/temporal-intent.ts` uses: a date-shaped run inside a+ * longer digit run is not a date, while a date surrounded by any script+ * with no whitespace still is one. The set of accepted date VALUES is the+ * lifecycle module's (`lifecycle/temporal-replace.ts`); this module only+ * needs the shared shape for masking and for the escape hatch.+ */+const ISO_DATE_RUN = /(?<![0-9-])\d{4}-\d{2}-\d{2}(?![0-9-])/u;++/** ISO date or instant run, with an optional time and zone. */+const ISO_INSTANT_RUN =+ /(?<![0-9-])\d{4}-\d{2}-\d{2}[Tt]\d{2}:\d{2}(?::\d{2}(?:\.\d+)?)?(?:[Zz]|[+-]\d{2}:?\d{2})?(?![0-9-])/u;++/**+ * A version-like dotted run: `v1.73.0`, `1.73.0`, `192.168.1.1`. Two or+ * more dots is the structural difference from a decimal (`3.14159`), so an+ * IP address and a semantic version are both excluded without a table.+ */+const VERSION_RUN = /(?<![0-9A-Za-z_])v?\d+(?:\.\d+){2,}(?![0-9A-Za-z_.])/iu;++/** Currency amount led by a symbol, with an optional magnitude suffix. */+const CURRENCY_SYMBOL_RUN =+ /(?<![0-9A-Za-z])[$€£¥]\s?\d[\d,]*(?:\.\d+)?(?:\s?[KMBT](?![A-Za-z]))?/iu;++/** Currency amount followed by an ISO code. */+const CURRENCY_CODE_RUN = new RegExp(+ `(?<![0-9A-Za-z_])[0-9][0-9,]*(?:\\.\\d+)?\\s?(?:${alternation(CURRENCY_CODES)})(?![A-Za-z])`,+ "iu",+);++/** Percentage. */+const PERCENT_RUN = /(?<![0-9A-Za-z_.])\d[\d,]*(?:\.\d+)?\s?%/u;++/** Number followed by a unit from {@link QUANTITY_UNITS}. */+const NUMBER_UNIT_RUN = new RegExp(+ // The year guard: a bare four-digit 19xx/20xx number is a YEAR far+ // more often than a count ("In 2024 users doubled"), and the lint is+ // precision-first. A thousands separator (`2,024 users`) or a longer+ // digit run is a number, not a year shape, and still fires.+ `(?<![0-9A-Za-z_.])(?!(?:19|20)\\d{2}\\b)\\d[\\d,]*(?:\\.\\d+)?\\s?(?:${alternation(QUANTITY_UNITS)})(?![0-9A-Za-z_])`,+ "iu",+);++/**+ * Bare thousands-separated count, optionally with a decimal part. The+ * trailing lookahead rejects a following DIGIT (the run must end) and a+ * comma that ANOTHER DIGIT follows (still inside the count), but not a+ * prose comma - "the counter reads 12,400, then stops" states a count,+ * and rejecting every comma made the common shape invisible.+ */+const THOUSANDS_RUN = /(?<![0-9A-Za-z_,])\d{1,3}(?:,\d{3})+(?:\.\d+)?(?![0-9]|,\d)/u;++/** Every trigger, in the order an equal-span tie prefers. */+const TRIGGER_PATTERNS: ReadonlyArray<RegExp> = Object.freeze([+ CURRENCY_SYMBOL_RUN,+ CURRENCY_CODE_RUN,+ PERCENT_RUN,+ NUMBER_UNIT_RUN,+ THOUSANDS_RUN,+]);++/** True when `text` carries an ISO date or instant anywhere in it. */+export function containsIsoDate(text: string): boolean {+ return ISO_DATE_RUN.test(text) || ISO_INSTANT_RUN.test(text);+}++interface Range {+ readonly start: number;+ readonly end: number;+}++/** A global copy of `pattern`, keeping every flag it already carries. */+function globalCopy(pattern: RegExp): RegExp {+ return new RegExp(+ pattern.source,+ pattern.flags.includes("g") ? pattern.flags : `${pattern.flags}g`,+ );+}++/** Append a match's span when the pattern finds one. */+function maskMatches(text: string, pattern: RegExp, into: Range[]): void {+ for (const match of text.matchAll(globalCopy(pattern))) {+ const start = match.index!;+ into.push({ start, end: start + match[0].length });+ }+}++/** True when the line carries an unescaped pipe, the table-cell boundary. */+function hasTablePipe(line: string): boolean {+ return /(?<!\\)\|/u.test(line);+}++/**+ * The masked spans of a document: frontmatter, fenced blocks, table rows,+ * ordered-list markers, inline code spans, ISO dates and instants, and+ * version-like tokens. Walked line by line because every one of those+ * exclusions except the date and version shapes is a LINE property.+ */+function exclusionRanges(text: string): Range[] {+ const ranges: Range[] = [];++ // Frontmatter is a leading fence closed by another fence line. An+ // unclosed opening fence is prose to the vault parser, so it is not+ // masked here either.+ let scanFrom = 0;+ const open = /^---[ \t]*\r?\n/u.exec(text);+ if (open !== null) {+ const rest = text.slice(open[0].length);+ const close = /^---[ \t]*\r?$/mu.exec(rest);+ if (close !== null) {+ scanFrom = open[0].length + close.index + close[0].length;+ ranges.push({ start: 0, end: scanFrom });+ }+ }++ let inFence = false;+ let fenceMarker = "";+ let lineStart = scanFrom;+ for (;;) {+ const newline = text.indexOf("\n", lineStart);+ const lineEnd = newline === -1 ? text.length : newline;+ const line = text.slice(lineStart, lineEnd).replace(/\r$/u, "");+ const fence = /^[ \t]{0,3}(`{3,}|~{3,})/u.exec(line);++ if (inFence) {+ ranges.push({ start: lineStart, end: lineEnd });+ if (+ fence !== null &&+ fence[1]!.charCodeAt(0) === fenceMarker.charCodeAt(0) &&+ fence[1]!.length >= fenceMarker.length+ ) {+ inFence = false;+ }+ } else if (fence !== null) {+ inFence = true;+ fenceMarker = fence[1]!;+ ranges.push({ start: lineStart, end: lineEnd });+ } else {+ // A table row is excluded whole: every cell on it is tabular data,+ // and a quantity in one belongs to the table, not to the prose.+ if (hasTablePipe(line)) ranges.push({ start: lineStart, end: lineEnd });++ // An ordered-list marker is excluded by itself; the item's content+ // after it is still authored prose and still scanned.+ const marker = /^[ \t]{0,3}\d{1,9}[.)](?=[ \t])/u.exec(line);+ if (marker !== null) {+ ranges.push({ start: lineStart, end: lineStart + marker[0].length });+ }++ // Inline code spans, backtick runs on one line. The span is excluded+ // including its delimiters, which carry no quantity of their own.+ for (const span of line.matchAll(/(`+)([^`]*?)\1/gu)) {+ const start = lineStart + span.index!;+ ranges.push({ start, end: start + span[0].length });+ }+ }++ if (newline === -1) break;+ lineStart = newline + 1;+ }++ maskMatches(text, ISO_DATE_RUN, ranges);+ maskMatches(text, ISO_INSTANT_RUN, ranges);+ maskMatches(text, VERSION_RUN, ranges);+ return ranges;+}++/** True when `[start, end)` overlaps any excluded span. */+function isExcluded(ranges: ReadonlyArray<Range>, start: number, end: number): boolean {+ for (const range of ranges) {+ if (start < range.end && range.start < end) return true;+ }+ return false;+}++/**+ * Every undated quantity in `body`, in ascending offset order.+ *+ * Overlapping trigger classes are resolved to the single longest lexeme at+ * that offset (`5,000 users` is one quantity, not a count plus a unit), so+ * a caller never reports the same words twice. The scan is pure: the input+ * string is never modified and repeated calls return equal results.+ */+export function findUndatedQuantities(body: string): UndatedQuantity[] {+ const excluded = exclusionRanges(body);+ const candidates: Range[] = [];+ for (const pattern of TRIGGER_PATTERNS) {+ maskMatches(body, pattern, candidates);+ }+ candidates.sort((a, b) => a.start - b.start || b.end - a.end);++ const found: UndatedQuantity[] = [];+ let consumedTo = 0;+ for (const candidate of candidates) {+ if (candidate.start < consumedTo) continue;+ if (isExcluded(excluded, candidate.start, candidate.end)) continue;+ found.push(+ Object.freeze({+ lexeme: body.slice(candidate.start, candidate.end),+ offset: candidate.start,+ }),+ );+ consumedTo = candidate.end;+ }+ return found;+}diff --git a/src/core/brain/write-batch.ts b/src/core/brain/write-batch.tsindex 37021b23..3bbf8d5a 100644--- a/src/core/brain/write-batch.ts+++ b/src/core/brain/write-batch.ts@@ -46,6 +46,7 @@ import {type NoteWriteOp,} from "./notes/write-record.ts";import { formatFrontmatter, parseFrontmatterWithNotices } from "../vault.ts";+import { PRIVACY_MARKER_KEYS } from "../graph/privacy-markers.ts";import { DEGRADATION_CODE } from "../integrity/degradation.ts";import { writePathAdvisoryField, type WritePathAdvisoryField } from "./write-path-advisory.ts";import {@@ -56,6 +57,7 @@ import {} from "./apply-evidence.ts";import { appendBrainNote, type AppendBrainNoteInput } from "./note.ts";import { ORIGIN_CHANNEL_FIELD } from "../origin-channel.ts";+import { CONTENT_CLASS_FIELD } from "./provenance/content-class.ts";import { preferencePath, validateSlug } from "./paths.ts";import { refuseDocumentDeny, type WriteSubject } from "./write-disposition.ts";import { REVIEW_LANE } from "./write-gate.ts";@@ -758,17 +760,28 @@ function projectCreateNote(}/**- * Frontmatter keys a caller-named UPDATE may not set. Both are read as- * boundary state, not note content: `visibility` carries the page's- * read-scope tokens (the reserved `private` among them), and- * `origin_channel` is the server-derived stamp creation writes once and- * mutation never rewrites. Creation needs no guard - the server stamp is- * merged last there and wins - but the update merge inverts the order,- * so the guard lives at the merge.+ * Frontmatter keys a caller-named UPDATE may not set. The first three are+ * read as boundary state, not note content: `visibility` carries the+ * page's read-scope tokens (the reserved `private` among them), and+ * `origin_channel` and `content_class` are the server-derived stamps+ * creation writes once and mutation never rewrites. The marker keys+ * (`access`, `confidentiality`, `sensitivity`) are boundary state for the+ * same reason, one layer over: the read roots resolve a deny value under+ * any of them to the same reserved remote-deny token (`graph/privacy-+ * markers.ts`), so a merged `access: ""` would clear that reservation+ * without the caller ever naming the boundary - and a merged deny value+ * would forge one. The refusal posture is the visibility one: edit the+ * note directly. Creation needs no refusal - the server stamps are merged+ * last there and a caller-named class is dropped outright, and a marker a+ * caller authors at creation is the caller stating the claim about their+ * own new note - but the update merge inverts the order, so the guard+ * lives at the merge.*/const RESERVED_UPDATE_FRONTMATTER_KEYS: ReadonlySet<string> = new Set(["visibility",ORIGIN_CHANNEL_FIELD,+ CONTENT_CLASS_FIELD,+ ...PRIVACY_MARKER_KEYS,]);/** The first reserved key the caller's map names, or `null`. */diff --git a/src/core/doctor-readiness.ts b/src/core/doctor-readiness.tsindex 58bc9d43..37ec51ab 100644--- a/src/core/doctor-readiness.ts+++ b/src/core/doctor-readiness.ts@@ -94,6 +94,7 @@ import {import { providerProducesVectors } from "./search/embeddings/contract.ts";import { makeProvider } from "./search/embeddings/provider.ts";import { resolveSearchConfig } from "./search/index.ts";+import { providerConsentIsDeclined, resolveProviderConsent } from "./search/provider-consent.ts";import {COMMAND_PROBE_VERDICT,CommandProbeError,@@ -125,6 +126,7 @@ export const DEFAULT_READINESS_TIMEOUT_MS = 5_000;export const READINESS_PROBE = {llmKey: "llm_key",embeddingProvider: "embedding_provider",+ providerConsent: "provider_consent",runtimeAdapterWiring: "runtime_adapter_wiring",installedRuntimes: "installed_runtimes",registeredCommands: "registered_commands",@@ -138,6 +140,59 @@ export const READINESS_PROBE = {*/const PROVIDERS_REQUIRING_API_KEY: ReadonlySet<string> = new Set(["openai-compat", "zeroentropy"]);+/**+ * The environment-variable PREFIXES that name an embedding-provider+ * setting.+ *+ * A prefix rather than a hand-written list, because a list would be a+ * second copy of the resolver's spelling table and would drift: a new+ * `OPEN_SECOND_BRAIN_EMBEDDING_*` key added to the resolver would not+ * appear here, and the probe would then under-report what is present in+ * the environment - which is the one thing this probe exists to say.+ */+const PROVIDER_ENV_PREFIXES: ReadonlyArray<string> = Object.freeze([+ "OPEN_SECOND_BRAIN_EMBEDDING_",+]);++/**+ * The provider environment variables that do NOT carry the prefix above.+ * The switch is one of them: it is the environment twin of the decline+ * record, and an operator who set it in the environment rather than in+ * the config file is exactly the case this probe has to reach.+ */+const PROVIDER_ENV_NAMES: ReadonlyArray<string> = Object.freeze([+ "OPEN_SECOND_BRAIN_SEARCH_SEMANTIC",+]);++/**+ * The NAMES of the provider environment variables present in `env`, in+ * alphabetical order.+ *+ * NAMES ONLY, never values: a readiness detail travels into logs, into a+ * JSON payload an operator pastes into a bug report and into the+ * registry's own output, and a value that reached any of those would be+ * a credential published by a diagnostic. The count-and-name form is the+ * same one `probeLlmKey` already uses for a resolved key.+ *+ * An empty value is absent, which is this repository's rule for+ * environment values everywhere else: an operator who exported+ * `OPEN_SECOND_BRAIN_EMBEDDING_KEY=` set nothing.+ */+function presentProviderEnvNames(+ env: Readonly<Record<string, string | undefined>>,+): ReadonlyArray<string> {+ const names: string[] = [];+ for (const name of Object.keys(env).toSorted()) {+ const value = env[name];+ if (value === undefined || value === "") continue;+ const isProviderEnv =+ PROVIDER_ENV_NAMES.includes(name) ||+ PROVIDER_ENV_PREFIXES.some((prefix) => name.startsWith(prefix));+ if (isProviderEnv) names.push(name);+ }+ return Object.freeze(names);+}+// ----- Errors ---------------------------------------------------------------/** Raised when a probe exceeds its per-check timeout budget. */@@ -292,6 +347,25 @@ export async function probeLlmKey(opts: ReadinessOptions): Promise<ReadinessVerdconst configPath = resolveConfigPath(opts);const { semantic } = resolveSearchConfig({ vault: opts.vault, configPath });if (!semantic.enabled || semantic.provider === "disabled") {+ // The arm is SPLIT, because the two situations are not the same+ // fact. A configuration the operator declined is a decision, and it+ // is disclosed by the provider-consent probe beside this one - which+ // names the keys that would re-enable the provider and the provider+ // environment variables that are present. This probe answers the+ // narrower question it was asked (is a model credential required?),+ // and the answer is the same either way, so it says so without+ // repeating the disclosure.+ const consent = resolveProviderConsent({+ configPath,+ env: opts.env ?? process.env,+ semantic,+ });+ if (providerConsentIsDeclined(consent)) {+ return {+ status: READINESS_STATUS.skipped,+ detail: "provider consent declined; no model API key is required",+ };+ }return {status: READINESS_STATUS.skipped,detail: "semantic search disabled; no model API key required",@@ -318,6 +392,46 @@ export async function probeLlmKey(opts: ReadinessOptions): Promise<ReadinessVerd};}+/**+ * The provider consent: did the operator write down a decision about the+ * embedding provider, and is it a decline?+ *+ * The probe answers the question the capability tier cannot: the tier+ * says what the configuration can do, and a configuration that says+ * `search_semantic_enabled: false` looks exactly like a configuration+ * that never mentioned semantic search at all. An operator who declined+ * and an operator who never configured anything are two different+ * situations with two different repairs, and this probe is where the+ * first one is named.+ *+ * `skipped`, not `fail`: a decline is a decision, and a decision is not+ * a fault. The detail names the two spellings a decline is written in,+ * so the operator can see which one is in force, and lists the provider+ * environment variables that are PRESENT - by NAME. The names are the+ * point: an operator who set `OPEN_SECOND_BRAIN_EMBEDDING_KEY` in their+ * environment and then wrote a decline in the config file needs to know+ * that the key is still there, and a readiness detail is the wrong+ * place to publish a credential.+ */+export async function probeProviderConsent(opts: ReadinessOptions): Promise<ReadinessVerdict> {+ const configPath = resolveConfigPath(opts);+ const env = opts.env ?? process.env;+ const consent = resolveProviderConsent({ configPath, env });+ if (!providerConsentIsDeclined(consent)) {+ return {+ status: READINESS_STATUS.skipped,+ detail: consent.detail,+ };+ }+ const present = presentProviderEnvNames(env);+ return {+ status: READINESS_STATUS.skipped,+ detail:+ `${consent.detail} Present provider environment variables: ` ++ `${present.length > 0 ? present.join(", ") : "none"}.`,+ };+}+/*** The embedding provider loads and reports a model and a positive* dimension. The provider's `ping()` is the authoritative source of the@@ -345,6 +459,21 @@ export async function probeEmbeddingProvider(opts: ReadinessOptions): Promise<Redetail: `config failed to resolve: ${(err as Error).message}`,};}+ // The consent is read BEFORE the provider is constructed: an operator+ // who deliberately declined is reported as skipped with the decline+ // sentence (the consent probe's wording is the precedent) rather than+ // reporting a failure that contradicts their choice.+ const consent = resolveProviderConsent({+ configPath,+ env: opts.env ?? process.env,+ semantic,+ });+ if (providerConsentIsDeclined(consent)) {+ return {+ status: READINESS_STATUS.skipped,+ detail: consent.detail,+ };+ }// The tier is read BEFORE the provider is constructed: a provider whose// credential is missing refuses at construction, and reporting that as// "failed to load" described the wrong thing entirely.@@ -357,7 +486,7 @@ export async function probeEmbeddingProvider(opts: ReadinessOptions): Promise<Re}let provider;try {- provider = makeProvider(semantic);+ provider = makeProvider(semantic, consent);} catch (err) {return {status: READINESS_STATUS.fail,@@ -1200,6 +1329,7 @@ export interface NamedProbe {export const DEFAULT_PROBES: ReadonlyArray<NamedProbe> = [{ name: READINESS_PROBE.llmKey, fn: probeLlmKey },{ name: READINESS_PROBE.embeddingProvider, fn: probeEmbeddingProvider },+ { name: READINESS_PROBE.providerConsent, fn: probeProviderConsent },{ name: READINESS_PROBE.runtimeAdapterWiring, fn: probeRuntimeAdapterWiring },{ name: READINESS_PROBE.installedRuntimes, fn: probeInstalledRuntimes },{ name: READINESS_PROBE.registeredCommands, fn: probeRegisteredCommands },diff --git a/src/core/egress/consent-stamp.ts b/src/core/egress/consent-stamp.tsnew file mode 100644index 00000000..87c0c9a2--- /dev/null+++ b/src/core/egress/consent-stamp.ts@@ -0,0 +1,241 @@+/**+ * The egress consent stamp: a record of WHAT WOULD LEAVE the machine,+ * made before anything leaves.+ *+ * ## What a stamp is+ *+ * A stamp answers one question - "if this were to go, what would go?" -+ * and it answers it about the payload as it would ACTUALLY be sent: the+ * payload has been through the shared egress guard, so the stamp carries+ * the redacted payload rather than the caller's intent. A stamp that+ * carried the intent would be a record of a wish, and a wish is not what+ * crosses a boundary.+ *+ * ## The two refusals+ *+ * 1. THE KILL SWITCH. When the operator has set the kill-switch+ * variable, every stamp is refused - whatever any other environment+ * value says, and whatever the grant says. The switch is a+ * PRESENCE switch: a non-empty value of any kind keeps the refusal+ * in force, because the point of a kill switch is that no+ * environment value can talk it out of it. The next variable in the+ * environment does not get to say otherwise.+ * 2. NO CONSENT. A stamp is a consent record, so a process that has+ * not been granted egress consent gets no stamp. The default is a+ * refusal, not a stamp: a consent feature whose default is "yes" is+ * not a consent feature.+ *+ * The order matters and is not a detail: the kill switch is read FIRST,+ * so a machine that has both the grant and the kill switch refuses, and+ * the test matrix in `tests/core/egress-consent-stamp.test.ts` drives+ * every granting value against the kill switch to keep it that way.+ *+ * ## What a refusal discloses+ *+ * The payload class. "Egress refused" tells an operator nothing they can+ * act on; "vault page bodies were withheld" names the thing that was+ * withheld, which is the thing they can go and look at. The class is+ * supplied by the caller, because only the caller knows what it is+ * offering to the boundary, and a refusal that named a wrong class would+ * be worse than one that named none.+ *+ * ## No destination is declared here+ *+ * This release declares no egress destination, so this module declares+ * no row in the egress registry and the site is an INPUT: the caller+ * names the destination it is stamping for, and the registry row for+ * that destination lands with the destination itself. The primitive is+ * the consent half of that seam; the other half is a place to send+ * bytes, and there is not one yet.+ */++import { parseBool } from "../validate.ts";+import { EGRESS_OUTCOME, redactForEgress, type EgressPolicy } from "./guard.ts";+import type { EgressSiteId } from "./registry.ts";++/**+ * How a stamp came back. Not the guard's outcome vocabulary and not the+ * trust rail's: this one answers "may this leave", which is a different+ * question from "did the scan clear it" and from "did a person consent+ * to a guarded write".+ */+export const CONSENT_STAMP_OUTCOME = Object.freeze({+ /** The stamp is issued: consent is recorded and the payload is cleared. */+ stamped: "stamped",+ /** No stamp: nothing may leave. */+ withheld: "withheld",+} as const);++export type ConsentStampOutcome =+ (typeof CONSENT_STAMP_OUTCOME)[keyof typeof CONSENT_STAMP_OUTCOME];++/**+ * The environment variable that GRANTS egress consent. Its granting+ * values are the repository's truthy spellings, so an operator who has+ * written `OPEN_SECOND_BRAIN_EGRESS_CONSENT=true` in their shell profile+ * means what they mean everywhere else in this tree.+ */+export const CONSENT_STAMP_ENV = "OPEN_SECOND_BRAIN_EGRESS_CONSENT";++/**+ * The environment variable that REFUSES every stamp. Presence is what+ * counts: a non-empty value of any kind - including one that reads as+ * "off" to a boolean parser - keeps the refusal in force, because no+ * environment value may re-enable egress once the operator has killed+ * it.+ */+export const CONSENT_STAMP_KILL_ENV = "OPEN_SECOND_BRAIN_EGRESS_CONSENT_KILL";++/** The values of {@link CONSENT_STAMP_ENV} that grant consent. */+export const CONSENT_STAMP_GRANT_VALUES: ReadonlyArray<string> = Object.freeze(["1", "true"]);++/** A stamp that was issued. */+export interface ConsentStampStamped<T> {+ readonly outcome: typeof CONSENT_STAMP_OUTCOME.stamped;+ /** What would leave, named for the operator. */+ readonly payloadClass: string;+ /** The redacted payload: what would ACTUALLY leave. */+ readonly payload: T;+ /** True when the guard changed the payload, so the copy differs. */+ readonly redacted: boolean;+ /** The operator-facing sentence. Never empty. */+ readonly detail: string;+}++/** A stamp that was refused. */+export interface ConsentStampWithheld {+ readonly outcome: typeof CONSENT_STAMP_OUTCOME.withheld;+ /** What would have left, named for the operator. */+ readonly payloadClass: string;+ /** The operator-facing refusal. Never empty; it discloses the class. */+ readonly detail: string;+ /** Why the stamp was refused. Never empty; it discloses the class. */+ readonly reason: string;+ /**+ * The tree locations the guard reported as secret-shaped identifiers,+ * empty unless the guard was the one that refused.+ */+ readonly secretIdentifiers: ReadonlyArray<string>;+}++/** Issued or refused. A union, so a refusal cannot be read as a stamp. */+export type ConsentStampVerdict<T> = ConsentStampStamped<T> | ConsentStampWithheld;++/** What the caller offers to the boundary. */+export interface ConsentStampRequest<T> {+ /**+ * The destination the stamp is for. This release declares no egress+ * destination, so the primitive has none of its own: the caller names+ * the site it is stamping for.+ */+ readonly site: EgressSiteId;+ /**+ * WHAT WOULD LEAVE, in the caller's words. Named in every refusal,+ * because a refusal that does not say what was withheld is a sentence+ * the operator cannot act on.+ */+ readonly payloadClass: string;+ /** The payload offered to the boundary. */+ readonly payload: T;+ /** The environment the consent is read from. */+ readonly env?: NodeJS.ProcessEnv;+ /** The guard's per-site policy, when the call site has one. */+ readonly policy?: EgressPolicy;+}++/**+ * Whether the kill switch is set. A presence check over a non-empty+ * value, RAW: an empty value is the absence of a value, which is this+ * repository's rule for environment values everywhere else, and the+ * whitespace a value carries is part of the value the operator set -+ * the switch is never disarmed by trimming it.+ */+function killSwitchIsSet(env: NodeJS.ProcessEnv): boolean {+ const value = env[CONSENT_STAMP_KILL_ENV];+ return value !== undefined && value !== "";+}++/**+ * Whether the operator granted egress consent. A value that cannot be+ * parsed is NOT a grant: the default is the refusal, so an unreadable+ * grant reads as no grant rather than as a yes.+ */+function consentIsGranted(env: NodeJS.ProcessEnv): boolean {+ const raw = env[CONSENT_STAMP_ENV];+ if (raw === undefined || raw === "") return false;+ try {+ return parseBool(raw, false, CONSENT_STAMP_ENV);+ } catch {+ return false;+ }+}++/** A refusal, with the class named in both sentences. */+function withheld(+ payloadClass: string,+ cause: string,+ secretIdentifiers: ReadonlyArray<string> = [],+): ConsentStampWithheld {+ const reason = `${payloadClass} were withheld: ${cause}`;+ return Object.freeze({+ outcome: CONSENT_STAMP_OUTCOME.withheld,+ payloadClass,+ detail: `egress consent withheld for ${payloadClass}: ${cause}.`,+ reason,+ secretIdentifiers: Object.freeze([...secretIdentifiers]),+ });+}++/**+ * Stamp a payload for egress, or refuse it.+ *+ * Pure with respect to the filesystem: it reads no file and writes none.+ * The only inputs are the payload, the class the caller names for it, the+ * site it is destined for and the environment the consent is read from.+ */+export function stampEgressConsent<T>(request: ConsentStampRequest<T>): ConsentStampVerdict<T> {+ const env = request.env ?? process.env;+ const payloadClass = request.payloadClass;++ // The kill switch, FIRST. Whatever the grant says, and whatever the+ // payload would have been.+ if (killSwitchIsSet(env)) {+ return withheld(+ payloadClass,+ `the egress consent kill switch (${CONSENT_STAMP_KILL_ENV}) is set, and no environment value re-enables it`,+ );+ }++ // The grant. Without one there is no consent to record. The refusal+ // separates an ABSENT grant from an UNRECOGNIZED one: telling an+ // operator who set the variable to set it would have them re-run the+ // instruction that already failed, so a value that is not a granting+ // value is named as the problem, with the granting spellings beside it.+ if (!consentIsGranted(env)) {+ const rawGrant = env[CONSENT_STAMP_ENV];+ const cause =+ rawGrant === undefined || rawGrant === ""+ ? `no egress consent is recorded; set ${CONSENT_STAMP_ENV} to grant it`+ : `${CONSENT_STAMP_ENV} is set to an unrecognized value; ` ++ `only ${CONSENT_STAMP_GRANT_VALUES.map((v) => JSON.stringify(v)).join(" or ")} grants consent`;+ return withheld(payloadClass, cause);+ }++ // The redaction. A stamp carries what would ACTUALLY leave, so the+ // shared guard runs over the payload before the stamp is issued - and+ // a guard refusal withholds the stamp rather than issuing one that+ // would have to be revoked.+ const scanned = redactForEgress(request.site, request.payload, request.policy ?? {});+ if (scanned.outcome !== EGRESS_OUTCOME.released) {+ return withheld(payloadClass, scanned.detail, scanned.secretIdentifiers);+ }+ return Object.freeze({+ outcome: CONSENT_STAMP_OUTCOME.stamped,+ payloadClass,+ payload: scanned.payload,+ redacted: scanned.redacted,+ detail:+ `egress consent stamped for ${payloadClass}: the payload is cleared to leave` ++ `${scanned.redacted ? " with secret-shaped content redacted" : ""}.`,+ });+}diff --git a/src/core/egress/guard.ts b/src/core/egress/guard.tsindex 37deb6ba..e57cbc66 100644--- a/src/core/egress/guard.ts+++ b/src/core/egress/guard.ts@@ -155,6 +155,95 @@ const SCAN_WINDOW_MIB = MAX_REDACTOR_INPUT / (1024 * 1024);export const EGRESS_REDACTION_NOTICE ="note: secret-shaped content was redacted; this export no longer matches the vault byte for byte\n";+/**+ * Emitted on stderr by a verb run with the content-class override, so an+ * operator who passes `--include-prompt-derived` is told, on the run,+ * that the copy carries material the default leaves out. Like the+ * redaction notice it goes to stderr, so stdout stays parseable.+ */+export const EGRESS_CONTENT_CLASS_INCLUDE_NOTICE =+ "note: prompt-derived content is INCLUDED by override; this export carries material the default leaves out\n";++/**+ * The sentence naming every item the default content-class policy left+ * out of an export, in the shape the knowledge pack's `blocked` list+ * established: each omission named, with the reason beside it. An+ * omission that is not named is a silently smaller export - the exact+ * dishonesty the class policy exists to remove - so every verb that+ * omits writes this line to stderr.+ */+export function contentClassOmissionNotice(omitted: ReadonlyArray<string>): string {+ const list = omitted.map((name) => `${name} (content_class: prompt-derived)`).join(", ");+ return (+ `note: ${omitted.length} item(s) left out as prompt-derived: ${list}; ` ++ "pass --include-prompt-derived to export them\n"+ );+}++/**+ * The operator-overridable half of the egress policy: what happens to+ * pages whose `content_class` frontmatter carries the reserved+ * `prompt-derived` value.+ *+ * The DEFAULT is exclusion, and it is not represented as a field here -+ * an absent `contentClass` means the default applies, the same way an+ * absent `foreignIdentifiers` means the default identifier rule does.+ * Setting `include` is the per-verb operator override: the flag is+ * spelled `--include-prompt-derived` on every export verb that offers+ * it, and a run that sets it emits+ * {@link EGRESS_CONTENT_CLASS_INCLUDE_NOTICE} on stderr.+ */+export interface EgressContentClassPolicy {+ /** Carry class-marked items; the default leaves them out and names them. */+ readonly include?: boolean;+}++/**+ * What the default content-class policy did to one collection of items a+ * payload was composed from. `kept` is what the payload should carry;+ * `omitted` holds each excluded item's operator-facing name, ready for+ * {@link contentClassOmissionNotice}. Under the override the split is the+ * identity and `omitted` is empty - an override run has nothing to name.+ */+export interface EgressContentClassSplit<T> {+ readonly kept: ReadonlyArray<T>;+ readonly omitted: ReadonlyArray<string>;+}++/**+ * Apply the content-class policy to the items a shared export's payload+ * is composed from. This is the exclusion seam: every export that+ * carries pages splits its page collection through here BEFORE the+ * payload is composed and redacted, so a class-marked page is not in the+ * bytes the guard scans rather than redacted inside them. The census+ * (`egress-census.test.ts`) pins the page-carrying sites to this call by+ * name, in both directions.+ *+ * The predicate comes from the caller because the guard is shape-blind+ * by design - it cannot know whether a payload's items are pages, nodes+ * or contracts, and pretending otherwise would put a vault walk inside+ * the one module that stays pure of the filesystem.+ */+export function egressContentClassSplit<T>(+ items: ReadonlyArray<T>,+ policy: EgressPolicy,+ item: {+ /** The operator-facing name an omission is disclosed under. */+ readonly name: (entry: T) => string;+ /** Whether this item carries the reserved class. */+ readonly isPromptDerived: (entry: T) => boolean;+ },+): EgressContentClassSplit<T> {+ if (policy.contentClass?.include === true) return { kept: [...items], omitted: [] };+ const kept: T[] = [];+ const omitted: string[] = [];+ for (const entry of items) {+ if (item.isPromptDerived(entry)) omitted.push(item.name(entry));+ else kept.push(entry);+ }+ return { kept, omitted };+}+/*** What a call site may vary about the one policy above.*@@ -191,6 +280,17 @@ export interface EgressPolicy {* option stays absent, which is the named decision, not an omission.*/readonly resolvedLiterals?: ReadonlyArray<string>;+ /**+ * What happens to pages carrying the reserved `prompt-derived` content+ * class. Absent, the default applies: such pages are left out of the+ * payload a shared export composes - split out at+ * {@link egressContentClassSplit} BEFORE composition, so they are+ * absent rather than redacted - and every omission is named on stderr.+ * `include: true` is the per-verb `--include-prompt-derived` override+ * and turns the split into the identity, with+ * {@link EGRESS_CONTENT_CLASS_INCLUDE_NOTICE} on stderr.+ */+ readonly contentClass?: EgressContentClassPolicy;}/**diff --git a/src/core/egress/registry.ts b/src/core/egress/registry.tsindex a971317f..fa8a2ff6 100644--- a/src/core/egress/registry.ts+++ b/src/core/egress/registry.ts@@ -115,7 +115,11 @@ export const EGRESS_SITES = Object.freeze({"the widest export in the tree: preferences, the page graph, page contracts and " +"the sources dashboard composed into one file. Redaction runs over the bundle " +"TREE and the JSON is serialised afterwards, so a note whose text happens to " +- "contain a credential assignment cannot mangle the document's own quoting.",+ "contain a credential assignment cannot mangle the document's own quoting. A page " ++ "whose content_class carries the reserved prompt-derived value is left out of both " ++ "page sections - graph node and contract - and the omission is named on stderr; " ++ "--include-prompt-derived carries it back in. The sources dashboard is an aggregate " ++ "over signal records, carries no page text, and is not class-filtered.",},"brain-graph-export": {id: "brain-graph-export",@@ -124,7 +128,11 @@ export const EGRESS_SITES = Object.freeze({redaction: R.sharedRedactor,reason:"carries no page body, but does carry every page title, path and typed-relation " +- "target - a credential pasted into a title left through here verbatim.",+ "target - a credential pasted into a title left through here verbatim. A page whose " ++ "content_class carries the reserved prompt-derived value has no node in the graph - " ++ "title, path and relations go with it - and the omission is named on stderr; " ++ "--include-prompt-derived carries it back in. Link and relation targets inside a " ++ "kept node are that page's own declared citations and are not rewritten.",},"brain-okf-export": {id: "brain-okf-export",@@ -137,9 +145,13 @@ export const EGRESS_SITES = Object.freeze({"as a tree and `okf.json` re-serialised from it; the markdown files are redacted " +"as text. A bundle whose secrets were removed is no longer a lossless round-trip " +"of the vault, and the verb says so rather than implying otherwise. What it does NOT " +- "do is filter by label: a page carrying `private: true` in its frontmatter exports in " +- "full, because the `<private>` region marker is this product's only content-derived " +- "privacy primitive and these composers are content composers, not visibility filters.",+ "do is filter by the informal privacy labels: a page carrying `private: true` in its " ++ "frontmatter exports in full, because the `<private>` region marker is this product's " ++ "only content-derived privacy primitive and these composers are content composers, " ++ "not visibility filters. What it DOES filter, since the content class shipped, is the " ++ "server-derived class itself: a page stamped content_class: prompt-derived is absent " ++ "from the manifest and no file is written for it, every omission is named on stderr, " ++ "and --include-prompt-derived carries the pages back in.",},"brain-export": {id: "brain-export",@@ -166,7 +178,13 @@ export const EGRESS_SITES = Object.freeze({"one caller that passes `foreignIdentifiers`, because its `session_id` and " +"`turn_id` were named by the harness that wrote the transcript rather than by " +"this vault, so the guard's vendor-prefix-only rule for identifiers - an " +- "argument about ids this build constructs - does not cover them.",+ "argument about ids this build constructs - does not cover them. The formats " ++ "carry no page-classed material - the two preference forms serialise preference " ++ "rows, which never take the content-class stamp, and the transcript corpus is " ++ "runtime session logs, not vault pages - so the default content-class exclusion " ++ "has nothing to remove here; the transcript branch still does not apply it " ++ "SILENTLY: a record that carries the reserved class is omitted and named, and a " ++ "corpus the policy would empty is a refusal, never an empty file.",},"config-export": {id: "config-export",@@ -192,7 +210,9 @@ export const EGRESS_SITES = Object.freeze({"the two flags the export boundary exists to turn on. Write-time coverage also " +"says nothing about a record already on disk or appended by another writer, since " +"the read model never re-scans. The verb now scans what it READ, which answers " +- "both, and gains the truncation refusal the other five already had.",+ "both, and gains the truncation refusal the other five already had. Continuity " ++ "records are structured events, not pages, and never take the content-class " ++ "stamp, so the default class exclusion removes nothing here.",},"brain-explorer-export": {id: "brain-explorer-export",@@ -208,7 +228,10 @@ export const EGRESS_SITES = Object.freeze({"substitution, so a principle whose text contains a quote cannot disturb the " +"document. The label rule holds here too: a preference is not withheld for carrying " +"a `private` tag, because the `<private>` region marker is the only content-derived " +- "privacy primitive in this product.",+ "privacy primitive in this product. The content-class default removes nothing here " ++ "either: preferences and retired principles are operator-curated rows and never " ++ "take the prompt-derived stamp, which is written by note creation, the session " ++ "import and the report writers only.",},"brain-knowledge-pack-export": {id: "brain-knowledge-pack-export",@@ -218,8 +241,9 @@ export const EGRESS_SITES = Object.freeze({reason:"a selected subset - preference principles and page bodies verbatim - built to be " +"handed to someone else, so it is the one export that also filters by LABEL before " +- "it scans: a page declaring `visibility:`, an entry carrying `owner:`, and an " +- "unreviewed `OKF Review/` candidate are blocked and named, never written. What is " ++ "it scans: a page declaring `visibility:`, an entry carrying `owner:`, an " ++ "unreviewed `OKF Review/` candidate, and - since the content class shipped - a page " ++ "stamped content_class: prompt-derived are blocked and named, never written. What is " +"carried is redacted as a tree (OKF manifest, preference rows) and as text (pages) " +"BEFORE the pack is sealed, so the sha256 table hashes the bytes that left and a " +"recipient's integrity check verifies the redacted copy, not the vault. Preference " +diff --git a/src/core/graph/privacy-markers.ts b/src/core/graph/privacy-markers.tsnew file mode 100644index 00000000..7596e5c0--- /dev/null+++ b/src/core/graph/privacy-markers.ts@@ -0,0 +1,118 @@+/**+ * Informal privacy markers (write-honesty, F1).+ *+ * `graph/visibility.ts` reserves ONE token - `private` - and reads it from+ * the `visibility:` key only. Vaults written before that convention say+ * `access: private`, `confidentiality: secret` or+ * `sensitivity: confidential` instead, and those pages resolve to default+ * visibility: the reserved-token rule never sees the claim the author+ * wrote. This module is that missing reading, and nothing else.+ *+ * What it answers, in three functions:+ *+ * - {@link hasPrivacyDenyMarker} - does the frontmatter carry a value+ * under one of the three marker keys that says "not for everyone";+ * - {@link markerDenyTokens} - the reserved token to UNION into the+ * page's declared visibility, or an empty list when no marker denies;+ * - {@link remoteDenyVisibilityTokens} - the declaration and that union+ * folded into one token list, for the read roots and the indexed+ * column.+ *+ * ## Scope of the rule, stated rather than implied+ *+ * A marker reserves a page against REMOTE reads only. It is not a+ * caller-scope token: it never enters {@link pageVisibility} and no scope+ * request can name it, so it can neither widen nor narrow what a caller+ * reaches at local reach. Folding it into `pageVisibility` would have made+ * `visibility:` and an informal key one vocabulary with two spellings;+ * keeping it here leaves each statement saying exactly one thing.+ *+ * The values are token-matched through {@link normToken} - the same NFC ++ * trim + lower-case normalisation the visibility layer uses - rather than+ * through a second normaliser. A value that spells something else (a+ * phrase, a negation, a Markdown sentence) is not a deny marker: this+ * module recognises the three token spellings the design pins and does not+ * guess at prose.+ */++import type { FrontmatterMap } from "../types.ts";+import { REMOTE_DENY_VISIBILITY_TOKEN, normToken, pageVisibility } from "./visibility.ts";++/**+ * The informal frontmatter keys a pre-`visibility:` vault used to state a+ * privacy claim under. Closed: a key outside this list is not a marker,+ * whatever value it carries.+ */+export const PRIVACY_MARKER_KEYS: ReadonlyArray<string> = Object.freeze([+ "access",+ "confidentiality",+ "sensitivity",+]);++/**+ * The marker VALUES that deny a remote read, already in the normal form+ * {@link normToken} produces. Compared by exact normalised equality: a+ * value that merely contains one of these words is a different claim.+ */+export const PRIVACY_DENY_MARKER_VALUES: ReadonlyArray<string> = Object.freeze([+ "private",+ "secret",+ "confidential",+]);++/** Marker values declared as a string or an array of strings. */+function declaredValues(raw: unknown): string[] {+ if (typeof raw === "string") return [raw];+ if (!Array.isArray(raw)) return [];+ return raw.filter((v): v is string => typeof v === "string");+}++/**+ * Does this frontmatter carry a privacy-denying informal marker value?+ *+ * True when any of {@link PRIVACY_MARKER_KEYS} declares a string (or an+ * array holding a string) whose {@link normToken} form is one of+ * {@link PRIVACY_DENY_MARKER_VALUES}. Absent keys, non-string scalars,+ * empty values and every other token answer false.+ */+export function hasPrivacyDenyMarker(meta: FrontmatterMap): boolean {+ for (const key of PRIVACY_MARKER_KEYS) {+ for (const value of declaredValues(meta[key])) {+ if (PRIVACY_DENY_MARKER_VALUES.includes(normToken(value))) return true;+ }+ }+ return false;+}++/**+ * The token to union into the page's declared visibility when an informal+ * marker denies a remote read, else an empty list.+ *+ * Exactly the reserved token: the marker says the same thing the formal+ * reservation says, so it resolves to the same identifier rather than+ * minting a second one.+ */+export function markerDenyTokens(meta: FrontmatterMap): ReadonlyArray<string> {+ return hasPrivacyDenyMarker(meta)+ ? Object.freeze([REMOTE_DENY_VISIBILITY_TOKEN])+ : Object.freeze([]);+}++/**+ * The page's declared visibility tokens plus any informal marker's+ * reservation, deduplicated in declaration order.+ *+ * This is what a read root hands to {@link isRemotelyReadable} and what+ * the indexer stamps into the measured column. The declared tokens are+ * kept verbatim (already normalised by {@link pageVisibility}); the+ * reserved token is appended only when a marker denies and it is not+ * already declared.+ */+export function remoteDenyVisibilityTokens(meta: FrontmatterMap): ReadonlyArray<string> {+ const declared = pageVisibility(meta);+ const out = [...declared];+ for (const token of markerDenyTokens(meta)) {+ if (!out.includes(token)) out.push(token);+ }+ return Object.freeze(out);+}diff --git a/src/core/search/embedding-spend.ts b/src/core/search/embedding-spend.tsindex b7a4a4b1..0a049320 100644--- a/src/core/search/embedding-spend.ts+++ b/src/core/search/embedding-spend.ts@@ -20,9 +20,16 @@ import {resolveEmbeddingPrice,type PriceQuote,} from "./embeddings/pricing.ts";-import { estimateCostUsd, estimateTokens, LOCAL_EMBEDDING_MODEL } from "./embeddings/signature.ts";+import { effectiveInputWindowTokens, passagePrefixSentByProvider } from "./embeddings/presets.ts";+import {+ embeddingSignature,+ estimateCostUsd,+ estimateTokens,+ LOCAL_EMBEDDING_MODEL,+} from "./embeddings/signature.ts";+import { sha256Hex } from "../integrity/digest.ts";import type { Store } from "./store.ts";-import type { PendingVectorScope } from "./store/chunks.ts";+import type { ChunkHoldExclusion, PendingVectorScope } from "./store/chunks.ts";import type { ResolvedSearchConfig } from "./types.ts";/** Why a gate refused. The value leaves TypeScript in previews and JSON. */@@ -138,17 +145,71 @@ export interface EmbeddingSpendPlanOptions {readonly forced?: boolean;}+/**+ * The pending-census exclusion key for this config's chunk holds+ * (chunk skip-and-hold, F2): the (model, window, signature) triple under+ * which the embedding phase records a hold, and under which the pending+ * anti-join hides held chunks from every spend surface.+ *+ * Null when no chunk can be proved over anything - no model, or no+ * declared window - and a null key excludes nothing: an old hold row+ * whose key no longer matches the config is inert, which is exactly the+ * release path. The phase persists under the same rendering so the writer+ * and the readers cannot drift on what "the same hold" means.+ */+export function chunkHoldExclusion(config: ResolvedSearchConfig): ChunkHoldExclusion | null {+ const model = activeEmbeddingModel(config);+ if (model === null) return null;+ const windowTokens = effectiveInputWindowTokens(config.semantic, model);+ if (windowTokens === null) return null;+ return Object.freeze({+ model,+ windowTokens,+ embeddingSignature: holdKeySignature(config, model),+ });+}++/**+ * The hold table's signature: the embedding identity PLUS the passage+ * prefix the backend would actually send. The hold's proof is measured+ * over `prefix + content` against the window, so the key the proof is+ * recorded under has to answer for the prefix too - a shortened or+ * removed prefix changes what fits, and a key blind to it would keep+ * excluding a chunk that now fits, with nothing to release it. The prefix+ * rides as a LEADING field, so the trailing provider:model:dimension+ * triple every signature reader parses from the end stays intact; an+ * empty sent prefix (every backend but `openai-compat`, or the prefix+ * disabled) contributes no field and renders the identity alone.+ */+function holdKeySignature(config: ResolvedSearchConfig, model: string): string {+ const identity = embeddingSignature({+ provider: config.semantic.provider,+ model,+ dimension: config.semantic.dimension,+ });+ const sent = passagePrefixSentByProvider(config.semantic.provider, config.semantic.passagePrefix);+ if (sent === "") return identity;+ const digest = sha256Hex(sent);+ return `pp-${digest.slice(0, 16)}:${identity}`;+}+/*** Plan the spend of embedding every pending chunk in scope. Reads only:* the phase in `indexer.ts` remains the only spender and throws on a- * blocked verdict.+ * blocked verdict. Chunks held under the config's own key are not+ * pending: they leave the census before they can be priced.*/export function planEmbeddingSpend(store: Store,config: ResolvedSearchConfig,opts: EmbeddingSpendPlanOptions = {},): EmbeddingSpendPlan {- return priceSpendPlan(store.findChunksWithoutEmbeddings(opts.scope), config, opts.forced);+ const hold = chunkHoldExclusion(config);+ return priceSpendPlan(+ store.findChunksWithoutEmbeddings(opts.scope, hold ?? undefined),+ config,+ opts.forced,+ );}/**diff --git a/src/core/search/embeddings/configured-provider.ts b/src/core/search/embeddings/configured-provider.tsindex b47ad534..cbc112f3 100644--- a/src/core/search/embeddings/configured-provider.ts+++ b/src/core/search/embeddings/configured-provider.ts@@ -31,7 +31,12 @@ export function resolveConfiguredEmbeddingProvider(): EmbeddingProvider | null {try {const configPath = opts.configPath ?? discoverConfig().path;- const provider = makeProvider(resolveSearchConfig({ vault, configPath }).semantic);+ const config = resolveSearchConfig({ vault, configPath });+ // The consent is read from the file the semantic configuration was+ // resolved FROM, not from whatever file the process happens to+ // default to: a decline is a decision about a configuration, and a+ // caller that named another file did not have this one in force.+ const provider = makeProvider({ ...config.semantic, configPath: config.configPath });return providerProducesVectors(provider) ? provider : null;} catch {return null;diff --git a/src/core/search/embeddings/provider.ts b/src/core/search/embeddings/provider.tsindex c3360367..e96569d0 100644--- a/src/core/search/embeddings/provider.ts+++ b/src/core/search/embeddings/provider.ts@@ -4,13 +4,59 @@* store.vecUpsert().** Anchored in docs/plans/2026-05-16-brain-search-design.md §11.+ *+ * ## The construction point is where a decline is honoured+ *+ * Every embedding provider in this tree is built here, so this is the one+ * place a recorded decline can be refused BEFORE a provider exists - before+ * an endpoint is named, before a credential is read and before a byte is+ * sent. A refusal further down (at the first `embed()`) would already have+ * built the provider, which is the difference between "nothing was+ * contacted" and "something was contacted and then refused".+ *+ * The code is `EMBEDDING_DISABLED`, the same code the null provider+ * raises for the same configuration, so one condition has one code across+ * the tree; what the message adds is the DECLINE - the keys that would+ * re-enable the provider - which the null provider's message cannot know+ * and would otherwise contradict (it tells an operator who deliberately+ * wrote `embedding_provider: disabled` to go and set a provider).+ *+ * The consent is resolved by `../provider-consent.ts` and may be handed+ * in already resolved, so a caller that has read the record (the semantic+ * phase does) does not read it twice. A caller that hands in nothing gets+ * the record read from the configuration this process is pointed at.*/import { SearchError } from "../types.ts";import type { ResolvedEmbeddingConfig } from "../types.ts";import type { EmbeddingProvider } from "./contract.ts";+import {+ providerConsentIsDeclined,+ resolveProviderConsent,+ type ProviderConsent,+} from "../provider-consent.ts";-export function makeProvider(config: ResolvedEmbeddingConfig): EmbeddingProvider {+export function makeProvider(+ config: ResolvedEmbeddingConfig & { readonly configPath?: string | null },+ /**+ * The provider consent, already resolved by the caller. Absent means+ * "resolve it here", which reads the record from the configuration this+ * process is pointed at.+ */+ consent?: ProviderConsent,+): EmbeddingProvider {+ // The refusal comes FIRST, before the lazy provider imports below: a+ // declined vault must not even load the module that would have talked+ // to an endpoint.+ const resolved =+ consent ??+ resolveProviderConsent({+ ...(config.configPath !== undefined ? { configPath: config.configPath } : {}),+ semantic: config,+ });+ if (providerConsentIsDeclined(resolved)) {+ throw new SearchError("EMBEDDING_DISABLED", resolved.detail);+ }// Lazy imports to keep the module graph small for users who never// enable semantic search.if (config.provider === "disabled" || !config.enabled) {diff --git a/src/core/search/entity-alias.ts b/src/core/search/entity-alias.tsindex 17b8acf2..83c7d58a 100644--- a/src/core/search/entity-alias.ts+++ b/src/core/search/entity-alias.ts@@ -8,17 +8,54 @@* unreadable one) leaves the query entity set untouched, so vaults* without `Brain/entities/` rank bit-identically to pre-registry* behaviour.+ *+ * Person-category entities additionally derive PERSON-NAME VARIANTS+ * (write-honesty-egress-consent, F3) - first-initial-plus-surname,+ * reversed word order, and operator-supplied nicknames - which join the+ * participation test and the expanded set beside the declared forms.+ * Derived forms are tracked in their own `derived` array so attribution+ * can tell a declared-form match from a derived-variant match; the+ * derivation kernel is the leaf module `person-variants.ts`, and the+ * nickname data is operator-supplied only (`person-nicknames.json` in+ * the vault, resolved by `resolvePersonNicknameMap` in+ * `person-nicknames.ts` - never a shipped word list).*/import { buildEntityIndex } from "../brain/entities/index-builder.ts";import { normalizeEntityName } from "../brain/entities/canonical.ts";+import {+ PERSON_NICKNAMES_ARTIFACT_REL,+ PersonNicknameConfigError,+ resolvePersonNicknameMap,+} from "../brain/entities/person-nicknames.ts";+import {+ derivePersonNameVariants,+ EMPTY_PERSON_NICKNAME_MAP,+ isPersonCategory,+ type PersonNicknameMap,+} from "../brain/entities/person-variants.ts";import { ENTITY_STATUS_SCOPE, entityStatusInScope } from "../brain/entities/status-scope.ts";+// The nickname reader lives on the brain side (`person-nicknames.ts`),+// beside the kernel it feeds: the registry's derived lookup lane and the+// doctor's collision scan consume it from there, and the brain must not+// import from `src/core/search/`. The re-export keeps this module's+// public surface byte-identical to C2.+export { PERSON_NICKNAMES_ARTIFACT_REL, PersonNicknameConfigError, resolvePersonNicknameMap };+export interface QueryEntityExpansion {/** Original query entities plus every form added by the registry. */readonly expanded: ReadonlyArray<string>;/** Only the forms the registry added (for canonical-hop attribution). */readonly added: ReadonlyArray<string>;+ /**+ * Only the forms PERSON-VARIANT DERIVATION added, beside the declared+ * {@link added} - so attribution can tell a declared-form match from+ * a derived-variant match. Empty unless a person-category entity+ * participated through a derived form the query did not already+ * carry.+ */+ readonly derived: ReadonlyArray<string>;/** Ids of the canonical entities whose forms were added, sorted. */readonly sourceIds: ReadonlyArray<string>;}@@ -26,8 +63,10 @@ export interface QueryEntityExpansion {/*** Expand query entities with canonical names and aliases from the* registry. A registry entity participates when ANY of its forms- * (name or alias) appears among the query entities; all its other- * forms are then added.+ * (name or alias) appears among the query entities - or, for a+ * person-category entity, when any DERIVED person variant of its forms+ * does - and the entity's other forms (declared and derived) are then+ * added.*/export function expandQueryEntities(vault: string,@@ -36,6 +75,7 @@ export function expandQueryEntities(const identity: QueryEntityExpansion = Object.freeze({expanded: queryEntities,added: Object.freeze([] as string[]),+ derived: Object.freeze([] as string[]),sourceIds: Object.freeze([] as string[]),});if (queryEntities.length === 0) return identity;@@ -48,15 +88,40 @@ export function expandQueryEntities(}if (index.entities.length === 0) return identity;+ let nicknames = EMPTY_PERSON_NICKNAME_MAP;+ try {+ nicknames = resolvePersonNicknameMap(vault);+ } catch (err) {+ // Fail-soft for the operator data alone: an unusable nickname file+ // drops the NICKNAME lane, not the expansion the registry provides+ // without it - the declared name/alias forms and the structural+ // variants keep working, exactly as they did before operator+ // nickname data existed. Anything that is not a config fault is not+ // a degradation, it is a bug, and it surfaces by name.+ if (!(err instanceof PersonNicknameConfigError)) throw err;+ }+const querySet = new Set(queryEntities.map((e) => normalizeEntityName(e)));const added: string[] = [];const addedSet = new Set<string>();+ const derived: string[] = [];+ const derivedSet = new Set<string>();const sourceIds: string[] = [];for (const entity of index.entities) {if (!entityStatusInScope(entity.status, ENTITY_STATUS_SCOPE.canonical)) continue;const forms = [entity.name, ...entity.aliases].map((f) => normalizeEntityName(f));- if (!forms.some((f) => querySet.has(f))) continue;+ // Person-category entities derive variants per declared form; the+ // variants join the PARTICIPATION test (a query naming a derived+ // form picks the entity up) and, below, the expanded set.+ const derivedByForm = isPersonCategory(entity.category)+ ? forms.map((f) => derivePersonNameVariants(f, nicknames))+ : null;+ const participates =+ forms.some((f) => querySet.has(f)) ||+ (derivedByForm !== null &&+ derivedByForm.some((variants) => variants.some((v) => querySet.has(v))));+ if (!participates) continue;let contributed = false;for (const form of forms) {if (querySet.has(form) || addedSet.has(form) || form.length < 2) continue;@@ -64,13 +129,34 @@ export function expandQueryEntities(added.push(form);contributed = true;}+ if (derivedByForm !== null) {+ for (const variants of derivedByForm) {+ for (const variant of variants) {+ // A derived form another entity (or this one) already DECLARED+ // is tracked as declared: declared attribution wins. The guard+ // sees declarations collected SO FAR; a declaration collected+ // LATER in the walk is claimed by the post-loop filter below.+ if (querySet.has(variant) || addedSet.has(variant) || derivedSet.has(variant)) continue;+ derivedSet.add(variant);+ derived.push(variant);+ contributed = true;+ }+ }+ }if (contributed) sourceIds.push(entity.id);}- if (added.length === 0) return identity;+ // Declared attribution wins in BOTH visit orders: a form some entity+ // declared is subtracted from the derived lane even when the derivation+ // reached it first, so the two lanes stay disjoint and a form is never+ // counted - or boosted - twice.+ const derivedOnly = derived.filter((v) => !addedSet.has(v));++ if (added.length === 0 && derivedOnly.length === 0) return identity;return Object.freeze({- expanded: Object.freeze([...queryEntities, ...added]),+ expanded: Object.freeze([...queryEntities, ...added, ...derivedOnly]),added: Object.freeze(added),+ derived: Object.freeze(derivedOnly),sourceIds: Object.freeze(sourceIds.toSorted()),});}diff --git a/src/core/search/indexer.ts b/src/core/search/indexer.tsindex f0075e0c..a9b9da4e 100644--- a/src/core/search/indexer.ts+++ b/src/core/search/indexer.ts@@ -35,6 +35,7 @@ import { effectiveInputWindowTokens, passagePrefixSentByProvider } from "./embedimport {activeEmbeddingModel,activeSpendQuote,+ chunkHoldExclusion,COST_GATE_KEY,EMBEDDING_GATE_REASON,type EmbeddingGateReason,@@ -48,10 +49,13 @@ import {import { embeddingPriceRecommendations } from "./price-recommendations.ts";import type { EmbeddingPriceSource } from "./embeddings/pricing.ts";import { makeProvider } from "./embeddings/provider.ts";+import { resolveProviderConsent } from "./provider-consent.ts";import {embeddingSignature,isStaleSignature,MODEL_NATIVE_DIMENSION,+ textExtent,+ tokenEstimateFloor,type EmbeddingIdentity,} from "./embeddings/signature.ts";import { LOCAL_DEFAULT_DIMENSION } from "./embeddings/local-provider.ts";@@ -109,7 +113,7 @@ import {LINK_RESOLUTION_PENDING_STATE_KEY,} from "./store.ts";import { LATEST_SCHEMA_VERSION, documentAttributesFromFrontmatter } from "./schema.ts";-import { chunkWindowDiagnosticCode, SearchError } from "./types.ts";+import { chunkWindowDiagnosticCode, CHUNK_HOLD_REASON, SearchError } from "./types.ts";import { walkVault } from "./walker.ts";import {callableVisibilitySurfaces,@@ -120,15 +124,20 @@ import type { PendingVectorScope } from "./store/chunks.ts";import type {ChunkWindowCensus,EmbedderRecordCensus,+ HeldChunkRecord,+ HeldChunkReport,+ HeldChunkReportItem,IndexCheckReport,IndexStats,IndexStatusSnapshot,PendingVectorCensus,+ ResolvedEmbeddingConfig,ResolvedSearchConfig,VisibilityHonestyFinding,} from "./types.ts";import type { ValidityWindow } from "./validity.ts";-import { REMOTE_DENY_VISIBILITY_TOKEN, pageVisibility } from "../graph/visibility.ts";+import { REMOTE_DENY_VISIBILITY_TOKEN } from "../graph/visibility.ts";+import { remoteDenyVisibilityTokens } from "../graph/privacy-markers.ts";import { closeDatabase } from "../sqlite-close.ts";import { renameWithRetry, unlinkWithRetry } from "../fs-atomic.ts";@@ -185,6 +194,9 @@ interface MutableStats {embeddingsComputed: number;embeddingsReused: number;embeddingsRetries: number;+ /** Held-back-over-window chunks (F2); frozen out of the stats when zero. */+ embeddingsHeld: number;+ heldChunks: HeldChunkRecord[];errors: Array<{ readonly path: string; readonly message: string }>;frontmatterNotices: DegradationNotice[];relationViolations: IndexStats["relationViolations"];@@ -215,6 +227,10 @@ function newStats(): MutableStats {embeddingsComputed: 0,embeddingsReused: 0,embeddingsRetries: 0,+ // Filled by the embedding phase only when it holds a chunk back;+ // zero means the frozen stats carry neither hold field at all.+ embeddingsHeld: 0,+ heldChunks: [],errors: [],frontmatterNotices: [],relationViolations: [],@@ -257,6 +273,11 @@ function freezeStats(s: MutableStats, durationMs: number): IndexStats {embeddingsComputed: s.embeddingsComputed,embeddingsReused: s.embeddingsReused,embeddingsRetries: s.embeddingsRetries,+ // Absent together, never one without the other: a run that held+ // nothing emits the statistics it emitted before the hold existed.+ ...(s.embeddingsHeld === 0+ ? {}+ : { embeddingsHeld: s.embeddingsHeld, heldChunks: Object.freeze([...s.heldChunks]) }),errors: Object.freeze([...s.errors]),frontmatterNotices: Object.freeze([...s.frontmatterNotices]),relationViolations: Object.freeze([...s.relationViolations]),@@ -600,7 +621,15 @@ async function indexIntoRun(// defect: `seen.add` runs BEFORE the content is read, so the// deletion sweep would not purge a skipped page's stale rows and// the page would keep whatever chunks it already had.- visibility: pageVisibility(frontmatter),+ //+ // The measurement is the UNION of the declared `visibility:`+ // tokens and any informal privacy marker's reservation+ // (`graph/privacy-markers.ts`), because the column is what a read+ // root consults for bytes already captured: stamping the+ // declaration alone would let a page reserved by+ // `access: private` serve its snippet remotely after the file+ // changed.+ visibility: remoteDenyVisibilityTokens(frontmatter),});// Framework-kind files feed the tier-guard post-pass: keep the// parsed frontmatter of this run's changed docs that declare a@@ -982,15 +1011,17 @@ async function formatChunkWindowMeasured(): Promise<string> {const { requireNextStep } = await import("../brain/next-step.ts");const step = requireNextStep(chunkWindowDiagnosticCode(census));- const remedy =- `whatever does not fit is truncated by the provider, silently. ` +- `Lower ${CHUNK_SIZE_CONFIG_KEY}, then run: ${step.nextCommand}`;+ const levers =+ `Two levers change the outcome: split the source by lowering ` ++ `${CHUNK_SIZE_CONFIG_KEY}, or exclude the page from embedding. Then run: ${step.nextCommand}`;if (census.verdict === "estimate-undecided") {return (`${census.chunksUndecided} of ${census.chunksMeasured} indexed chunk(s) cannot be ` +`placed inside the ${census.windowTokens}-token input window declared for ` +`${census.model}: the census estimates tokens from character counts, which does not ` +- `bound a token window for non-Latin scripts, so this is not a pass - ${remedy}`+ `bound a token window for non-Latin scripts, so this is not a pass. Undecided chunks ` ++ `are still sent, and whatever does not fit is truncated by the provider, silently - ` ++ `${levers}`);}const undecided =@@ -999,8 +1030,10 @@ async function formatChunkWindowMeasured(: ` A further ${census.chunksUndecided} chunk(s) the estimate cannot place either way.`;return (`${census.chunksOverWindow} of ${census.chunksMeasured} indexed chunk(s) estimate above ` +- `the ${census.windowTokens}-token input window declared for ${census.model}; ` +- `${remedy}${undecided}`+ `the ${census.windowTokens}-token input window declared for ${census.model}; chunks ` ++ `proved over the window are held back from embedding rather than sent and truncated, ` ++ `and \`o2b search status\` names the held ones in its held-chunks report. ${levers}` ++ `${undecided}`);}@@ -1034,6 +1067,15 @@ const INDEX_STAGE = Object.freeze({export interface EmbeddingPhaseTally {embeddingsComputed: number;embeddingsRetries: number;+ /**+ * Chunks this phase held back as provably over the model's declared+ * input window (chunk skip-and-hold, F2), and one record per chunk.+ * Absent until the phase holds something: a tally a hold-free run+ * owns never carries either, which is what lets the stats freeze drop+ * both together.+ */+ embeddingsHeld?: number;+ heldChunks?: HeldChunkRecord[];/*** The cost gate's own verdict for this phase's census, set just before* the first provider call (t_9d155d0e). `forced` is true only when a@@ -1079,9 +1121,66 @@ export interface EmbeddingPhaseOptions {readonly progress?: import("../brain/progress.ts").ProgressCounter;}+/** One entry of the embedding phase's pending census. */+type PendingCensusEntry = EmbeddingSpendPlan["pending"][number];++/**+ * Split a pending census into what the embedding phase may send and what+ * it must HOLD back (chunk skip-and-hold, F2).+ *+ * The hold acts on proof only. A chunk is held when the estimator's FLOOR+ * - code points over four, charged through the prefix the configured+ * backend actually prepends - already exceeds the model's effective+ * declared window: then the real request is over it whatever tokenizer the+ * provider runs. The undecided band between the floor and the ceiling+ * stays embedded exactly as before this feature, because acting there+ * would hold work on a guess, and a model with no declared window (or the+ * local embedder, which has no window by construction) proves nothing at+ * all - for them nothing is held and every chunk stays pending.+ *+ * The predicate is the same measurement the oversize-chunk census makes+ * its `over-window` arm with, so the phase and the report can never+ * disagree about which chunks are provably over.+ */+function partitionHeldChunks(+ pending: ReadonlyArray<PendingCensusEntry>,+ semantic: ResolvedEmbeddingConfig,+ model: string | null,+): { readonly embeddable: Array<PendingCensusEntry>; readonly held: HeldChunkRecord[] } {+ const windowTokens = effectiveInputWindowTokens(semantic, model);+ if (windowTokens === null) return { embeddable: [...pending], held: [] };+ // Charge the text the provider will actually receive: only+ // `openai-compat` prepends the configured passage prefix, and a+ // prefix nobody sends cannot push a chunk over a window.+ const prefix = passagePrefixSentByProvider(semantic.provider, semantic.passagePrefix);+ const embeddable: Array<PendingCensusEntry> = [];+ const held: HeldChunkRecord[] = [];+ for (const chunk of pending) {+ const estimatedTokens = tokenEstimateFloor(textExtent(prefix + chunk.content));+ if (estimatedTokens > windowTokens) {+ held.push(+ Object.freeze({+ chunkId: chunk.chunkId,+ estimatedTokens,+ windowTokens,+ reason: CHUNK_HOLD_REASON.overDeclaredWindow,+ }),+ );+ } else {+ embeddable.push(chunk);+ }+ }+ return { embeddable, held };+}+/*** Compute and store a vector for every indexed chunk that has none.*+ * Chunks whose token estimate PROVABLY exceeds the model's effective+ * declared input window are held back instead (see+ * {@link partitionHeldChunks}): they are never sent, never stored and+ * never priced, and the tally records each one.+ ** Exported (provenance-at-the-boundary, F) because it is the whole of* `o2b search vector-backfill --apply`: the population step is already* resumable by construction - the anti-join finds exactly the chunks@@ -1124,16 +1223,54 @@ export async function runEmbeddingPhase(// A handed-in plan keeps its census but is priced and gated again here,// unforced: the verdict this phase acts on is always its own.- const plan =+ const initialPlan =opts.plan !== undefined? priceSpendPlan(opts.plan.pending, config): planEmbeddingSpend(store, config, opts.scope ? { scope: opts.scope } : {});- const pending = plan.pending;+ const pending = initialPlan.pending;if (pending.length === 0) return;- const provider = makeProvider(config.semantic);+ const consent = resolveProviderConsent({+ configPath: config.configPath ?? null,+ env: process.env,+ semantic: config.semantic,+ });+ const provider = makeProvider(config.semantic, consent);const model = config.semantic.model ?? provider.model;+ // Skip-and-hold (F2): chunks provably over the declared window leave+ // the census BEFORE it is priced, batched or gated, so held work can+ // neither trip the cost gate nor appear on the receipt. Filtering ahead+ // of the batch loop is also what keeps the 1:1 invariant below intact:+ // the provider is only ever asked about chunks it will answer for.+ const { embeddable, held } = partitionHeldChunks(pending, config.semantic, model);+ if (held.length > 0) {+ stats.embeddingsHeld = (stats.embeddingsHeld ?? 0) + held.length;+ (stats.heldChunks ??= []).push(...held);+ // The record is what makes the hold outlive the run: the pending+ // anti-join excludes these chunks under this key, so the status+ // estimate, the preview and the next census stop pricing work this+ // config proved it will not spend. A hold the config-side resolver+ // cannot key is not persisted - there is no stable identity to+ // release it under.+ const exclusion = chunkHoldExclusion(config);+ if (exclusion !== null) {+ store.recordChunkHolds(+ held.map((h) => ({+ chunkId: h.chunkId,+ estimatedTokens: h.estimatedTokens,+ windowTokens: h.windowTokens,+ })),+ { model: exclusion.model, embeddingSignature: exclusion.embeddingSignature },+ );+ }+ }+ // A run that holds everything pending spends nothing: no provider+ // request, no gate verdict, and no receipt on a census it will not+ // embed.+ if (embeddable.length === 0) return;+ const plan = held.length > 0 ? priceSpendPlan(embeddable, config) : initialPlan;+// Cost gate: the shared spend plan estimates the whole pending set up// front and the run is refused when it exceeds the configured ceiling,// unless forced.@@ -1150,7 +1287,7 @@ export async function runEmbeddingPhase(}throw new SearchError("EMBEDDING_COST_GATE",- `estimated embedding cost ${formatEstimatedUsd(plan.estimatedUsd)} for ${pending.length} chunk(s) ` ++ `estimated embedding cost ${formatEstimatedUsd(plan.estimatedUsd)} for ${plan.pending.length} chunk(s) ` +`exceeds ${COST_GATE_KEY} ${formatEstimatedUsd(gateUsd)}. ` +`Re-run with ${FORCE_COST_FLAG} to proceed or raise the gate.`,);@@ -1173,13 +1310,13 @@ export async function runEmbeddingPhase(// `embedding_concurrency`.const superBatch = batchSize * Math.max(1, config.semantic.concurrency);- progress?.start(INDEX_STAGE.embed, pending.length);- for (let i = 0; i < pending.length; i += superBatch) {+ progress?.start(INDEX_STAGE.embed, embeddable.length);+ for (let i = 0; i < embeddable.length; i += superBatch) {// Cooperative deadline: embedding batches are the other long// phase of an index run - abort between batches, never mid-batch.safeguard?.checkpoint();throwIfAborted(signal, "index");- const batch = pending.slice(i, i + superBatch);+ const batch = embeddable.slice(i, i + superBatch);progress?.advance(INDEX_STAGE.embed, batch.length);const texts = batch.map((p) => p.content);const vectors = await provider.embed(texts, "passage");@@ -1643,6 +1780,14 @@ function isEnoent(e: unknown): boolean {// indexStatus// ─────────────────────────────────────────────────────────────────────────────+/**+ * How many held chunks the status snapshot names. A vault can hold+ * thousands of chunks after a model swap, and the status JSON is read by+ * humans and by agents on a token budget; the report's `total` carries+ * the whole so the cap is a disclosure, not a concealment.+ */+export const HELD_CHUNK_REPORT_CAP = 20;+export async function indexStatus(config: ResolvedSearchConfig): Promise<IndexStatusSnapshot> {let store: Store;try {@@ -1764,6 +1909,32 @@ export async function indexStatus(config: ResolvedSearchConfig): Promise<IndexStwarnings.push(await formatChunkWindowMeasured(chunkWindow));}+ // What the index holds back right now (chunk skip-and-hold, F2), read+ // from the durable records under the ACTIVE configuration's key. A+ // row whose key a config change released is a decision the next run+ // will not honour, so it is not a hold this status would report, and+ // a configuration that can hold nothing (no model, no declared+ // window) reads nothing. Absent unless something matches the key -+ // the same byte discipline as the census field beside it.+ let heldChunks: HeldChunkReport | undefined;+ const holdKey = config.semantic.enabled ? chunkHoldExclusion(config) : null;+ if (holdKey !== null) {+ const heldCensus = store.heldChunkCensus(holdKey, HELD_CHUNK_REPORT_CAP);+ if (heldCensus.total > 0) {+ const items: HeldChunkReportItem[] = heldCensus.rows.map((row) =>+ Object.freeze({+ path: row.path,+ chunkIndex: row.chunkIndex,+ estimatedTokens: row.estimatedTokens,+ windowTokens: row.windowTokens,+ // The only decision the hold table records.+ reason: CHUNK_HOLD_REASON.overDeclaredWindow,+ }),+ );+ heldChunks = Object.freeze({ total: heldCensus.total, items: Object.freeze(items) });+ }+ }+// Best-effort spend estimate to bring stale/missing embeddings current,// from the shared spend plan. Chunk content is scanned only when the// price is a positive known rate: a known-free model costs 0. An@@ -1777,7 +1948,10 @@ export async function indexStatus(config: ResolvedSearchConfig): Promise<IndexStrefreshPriceSource = quote.source;if (quote.usdPerMtok !== null && quote.usdPerMtok > 0) {estimatedRefreshCostUsd = planEmbeddingSpend(store, config).estimatedUsd;- } else if (quote.usdPerMtok === null && store.countChunksWithoutEmbeddings() > 0) {+ } else if (+ quote.usdPerMtok === null &&+ store.countChunksWithoutEmbeddings(undefined, chunkHoldExclusion(config) ?? undefined) > 0+ ) {estimatedRefreshCostUsd = null;} else {estimatedRefreshCostUsd = 0;@@ -1806,6 +1980,7 @@ export async function indexStatus(config: ResolvedSearchConfig): Promise<IndexSt// Absent, not null, whenever the check could run - so a status// over a curated model serializes the bytes it always has....(chunkWindowUndeclared === null ? {} : { chunkWindowUndeclared }),+ ...(heldChunks === undefined ? {} : { heldChunks }),warnings: Object.freeze(warnings),});} finally {@@ -2123,11 +2298,16 @@ export async function indexCheck(// A probe that did not complete goes to `warnings`, because it// proved nothing, and the CLI gives it an exit code of its own so// it is not read as either verdict.+ const consent = resolveProviderConsent({+ configPath: config.configPath ?? null,+ env: process.env,+ semantic: config.semantic,+ });const probe = !embeddingKeyResolved? PROVIDER_PROBE_NOT_CONFIGURED: opts?.probeProvider === false? PROVIDER_PROBE_SKIPPED- : await probeProvider(() => makeProvider(config.semantic));+ : await probeProvider(() => makeProvider(config.semantic, consent));const providerProbe = probe.state;const providerReason = probe.reason;if (providerProbe === PROVIDER_PROBE.unreachable) {diff --git a/src/core/search/pipeline/assemble.ts b/src/core/search/pipeline/assemble.tsindex bc51176a..15da0317 100644--- a/src/core/search/pipeline/assemble.ts+++ b/src/core/search/pipeline/assemble.ts@@ -540,6 +540,9 @@ function rankCandidates(input: AssemblyInput, rankCap: number): BrainSearchResul...(signals.entityMatchByChunk !== undefined? { entityMatchByChunk: signals.entityMatchByChunk }: {}),+ ...(signals.derivedEntityMatchByChunk !== undefined+ ? { derivedEntityMatchByChunk: signals.derivedEntityMatchByChunk }+ : {}),...(signals.activationByChunk !== undefined? { activationByChunk: signals.activationByChunk }: {}),diff --git a/src/core/search/pipeline/candidate-signals.ts b/src/core/search/pipeline/candidate-signals.tsindex a6f9d8d3..665cad5a 100644--- a/src/core/search/pipeline/candidate-signals.ts+++ b/src/core/search/pipeline/candidate-signals.ts@@ -46,6 +46,13 @@ export interface CandidateSignals {readonly canonicalMatchByChunk: ReadonlyMap<number, number> | undefined;/** Canonical entity ids the expansion hopped through. */readonly canonicalSourceIds: ReadonlyArray<string>;+ /**+ * Chunks matching a DERIVED person-variant form (F3), fed to the+ * ranker's strictly-smaller derived lane. Undefined when the+ * expansion derived nothing, so the layer stays unwired and every+ * score and breakdown shape is byte-identical.+ */+ readonly derivedEntityMatchByChunk: ReadonlyMap<number, number> | undefined;readonly activationByChunk: ReadonlyMap<number, number> | undefined;readonly coAccessByChunk: ReadonlyMap<number, ReadonlyMap<number, number>> | undefined;readonly trendByDoc: ReadonlyMap<number, string> | undefined;@@ -82,9 +89,19 @@ export function collectCandidateSignals(input: CandidateSignalsInput): Candidate// bit-identical to pre-registry behaviour.const queryEntities = extractEntities(input.query);const entityExpansion = expandQueryEntities(config.vault, queryEntities);+ // The DECLARED lane counts the query's own entities plus the forms+ // the registry declared (name and aliases). A person-variant query+ // expansion also emits DERIVED forms (initials, word order,+ // nicknames); those ride their own strictly-smaller lane below, so a+ // declared-form match always outranks a derived-variant match. When+ // nothing was derived this is exactly the pre-variant `expanded` set,+ // byte for byte.+ const declaredForms: ReadonlyArray<string> = [...queryEntities, ...entityExpansion.added];const entityMatchByChunk =- entityExpansion.expanded.length > 0- ? store.chunkEntityMatches(ids, entityExpansion.expanded)+ declaredForms.length > 0 ? store.chunkEntityMatches(ids, declaredForms) : undefined;+ const derivedEntityMatchByChunk =+ entityExpansion.derived.length > 0+ ? store.chunkEntityMatches(ids, entityExpansion.derived): undefined;const canonicalMatchByChunk =entityExpansion.added.length > 0@@ -112,6 +129,7 @@ export function collectCandidateSignals(input: CandidateSignalsInput): CandidateentityMatchByChunk,canonicalMatchByChunk,canonicalSourceIds: entityExpansion.sourceIds,+ derivedEntityMatchByChunk,activationByChunk: activation.activationByChunk,coAccessByChunk: activation.coAccessByChunk,trendByDoc: collectFreshnessTrend(input),diff --git a/src/core/search/pipeline/relational-arm.ts b/src/core/search/pipeline/relational-arm.tsindex d67383ae..070476b7 100644--- a/src/core/search/pipeline/relational-arm.ts+++ b/src/core/search/pipeline/relational-arm.ts@@ -239,9 +239,11 @@ function isCallerReadable(reach: TransportReach,agentScope: string | null,cache: FrontmatterCache,+ store: Pick<Store, "indexedVisibilityByPaths">,): boolean {+ const indexedTags = store.indexedVisibilityByPaths([path]).get(path);return (- isPathReadableAtReach(vault, path, reach, cache) &&+ isPathReadableAtReach(vault, path, reach, cache, indexedTags) &&(agentScope === null || isPathOwnerVisible(vault, path, agentScope, cache)));}@@ -377,11 +379,18 @@ export function runRelationalArm(// documents or not. The enforcement verdict below stays the FULL// classification: the exclusion (enforce) or the kept// contribution (shadow) does not depend on what was recordable.- const recordable = isCallerReadable(vault, rep.path, reach, agentScope, frontmatterCache)+ const recordable = isCallerReadable(+ vault,+ rep.path,+ reach,+ agentScope,+ frontmatterCache,+ store,+ )? [...eligibility.node,...(sourcePath === null ||- isCallerReadable(vault, sourcePath, reach, agentScope, frontmatterCache)+ isCallerReadable(vault, sourcePath, reach, agentScope, frontmatterCache, store)? eligibility.source: []),]@@ -402,7 +411,16 @@ export function runRelationalArm(if (trustGateMode === SEARCH_TRUST_GATE_MODE.enforce) continue;}}- const gated = gatePath(vault, node.path, titles, reach, agentScope, frontmatterCache, nowMs);+ const gated = gatePath(+ vault,+ node.path,+ titles,+ reach,+ agentScope,+ frontmatterCache,+ nowMs,+ store,+ );outcome.rankedChunkIds.push(rep.chunkId);outcome.reachByChunk.set(rep.chunkId, {via: node.viaLinkTypes,@@ -593,6 +611,7 @@ function gatePath(agentScope: string | null,cache: FrontmatterCache,nowMs: number,+ store: Store,): {readonly path: ReadonlyArray<RelationalPathStep>;readonly supersededBy?: string;@@ -602,7 +621,7 @@ function gatePath(for (const step of steps) {const meta = titles.get(step.documentId);const unreadable =- meta === undefined || !isCallerReadable(vault, meta.path, reach, agentScope, cache);+ meta === undefined || !isCallerReadable(vault, meta.path, reach, agentScope, cache, store);if (unreadable) continue;gated.push(step);if (supersededBy === undefined) {diff --git a/src/core/search/provider-consent.ts b/src/core/search/provider-consent.tsnew file mode 100644index 00000000..6fd21380--- /dev/null+++ b/src/core/search/provider-consent.ts@@ -0,0 +1,304 @@+/**+ * The provider-consent resolver: what the operator's configuration says+ * about the embedding provider, and whether they DECLINED it+ * (write-honesty-egress-consent, Lane F).+ *+ * ## The defect this module exists for+ *+ * An operator who wrote `search_semantic_enabled: false` and an operator+ * who never configured semantic search at all reached the SAME capability+ * tier, so every surface reported the same sentence for two opposite+ * situations: one of them a decision, the other an absence. The+ * distinction is not cosmetic - "the provider is off" and "the operator+ * said no" have different repairs, and a surface that cannot tell them+ * apart cannot tell an operator which one they are looking at.+ *+ * ## What this module is NOT+ *+ * It is not a third "off" spelling beside the capability tier. The tier+ * (`./capability-tier.ts`) still answers what the configuration can do,+ * and this resolver CARRIES that tier rather than computing a second one:+ * {@link ProviderConsent.capability} is the shared resolver's answer, and+ * the consent adds exactly one fact beside it - whether a decline was+ * written down. A configuration sits on one rung of the shared ladder and+ * the consent is an annotation of that rung, never a rival ladder.+ *+ * The import-cycle ratchet is what limits the home: this module sits+ * BESIDE the capability tier rather than above it, and it reads the+ * operator's machine configuration directly (`../config.ts`) rather than+ * reaching for the vault configuration, whose loader sits above the search+ * layer and would close a loop. The cost of that choice is stated rather+ * than hidden: the resolver answers for the MACHINE's configuration, which+ * is where the two decline spellings live.+ *+ * ## The fail-closed arm+ *+ * A consent record that cannot be read, and a consent value that cannot+ * be parsed, both resolve to DECLINED. Reading either as "unset" would+ * turn a broken record back into a silent default - the same defect the+ * configuration reader itself refuses elsewhere ("an unreadable config is+ * not consent to print host paths", `src/mcp/vault-path-field.ts`). The+ * default `search_semantic_enabled` is `false`, so "unset" and "off" are+ * the same configuration; a record that cannot be read must not be allowed+ * to look like a record that says nothing, because only one of those two+ * is a decision the operator made.+ *+ * ## The two witnesses of a decline+ *+ * 1. `search_semantic_enabled=false` - the switch, in the config file+ * or in `OPEN_SECOND_BRAIN_SEARCH_SEMANTIC`.+ * 2. `embedding_provider=disabled` - the provider name, in the config+ * file or in `OPEN_SECOND_BRAIN_EMBEDDING_PROVIDER`.+ *+ * Both are READINGS OF THE RECORD, and nothing else is. The resolved+ * configuration is deliberately NOT a third witness even though it names+ * the sentinel provider whenever the record does: a resolved+ * `provider: "disabled"` is the record seen through the resolver, and+ * treating it as an independent witness would let a configuration that+ * was assembled rather than read - a fixture, an override - claim a+ * decision the operator never wrote down. The record is the only place a+ * decision lives.+ *+ * ## Why there is no membership list and no guard+ *+ * The verdict-vocabulary census enrols a vocabulary by its four pieces -+ * a frozen object, a derived union, a membership list and a guard that+ * takes `unknown`. The states here never cross a string boundary: they+ * are compared in-process and never persisted, never put on a wire and+ * never read back off disk, so a guard that policed a round trip through+ * a string would have nothing to police. The two pieces that earn their+ * place - the frozen object and the derived union - are what ship.+ */++import { discoverConfig } from "../config.ts";+import { envOrConfig, parseBool } from "../validate.ts";+import { resolveSemanticCapability, type SemanticCapability } from "./capability-tier.ts";+import type { ResolvedEmbeddingConfig } from "./types.ts";++/**+ * The two consent states. Not a third member for "unknown": an unreadable+ * record resolves to {@link PROVIDER_CONSENT_STATE.declined}, because a+ * record that cannot be read is not a record of consent.+ */+export const PROVIDER_CONSENT_STATE = Object.freeze({+ /** The operator wrote no decision about the provider. */+ unset: "unset",+ /** The operator wrote a decline, or the record could not be read. */+ declined: "declined",+} as const);++export type ProviderConsentState =+ (typeof PROVIDER_CONSENT_STATE)[keyof typeof PROVIDER_CONSENT_STATE];++/**+ * The machine-configuration keys that record a decision about the+ * embedding provider. These are the two spellings the decline has always+ * had; this module names them rather than inventing a third.+ */+export const PROVIDER_CONSENT_KEY = Object.freeze({+ /** The switch that turns the semantic lane on. */+ semantic: "search_semantic_enabled",+ /** The provider name; the sentinel value is the decline. */+ provider: "embedding_provider",+} as const);++/**+ * The environment variables that mirror {@link PROVIDER_CONSENT_KEY}. The+ * environment wins over the config file, which is the rule the search+ * configuration resolver already applies to every key it reads.+ */+export const PROVIDER_CONSENT_ENV = Object.freeze({+ semantic: "OPEN_SECOND_BRAIN_SEARCH_SEMANTIC",+ provider: "OPEN_SECOND_BRAIN_EMBEDDING_PROVIDER",+} as const);++/**+ * The provider name that records a decline. The resolver's default+ * provider is `openai-compat`, so this value is never reached by+ * accident.+ */+export const PROVIDER_CONSENT_DECLINED_PROVIDER = "disabled";++/**+ * The keys that would re-enable a declined provider. Named in every+ * refusal, because a refusal that does not say how to undo it is a+ * sentence the operator has to look up the spelling of.+ */+export const PROVIDER_CONSENT_REENABLE_KEYS: ReadonlyArray<string> = Object.freeze([+ PROVIDER_CONSENT_KEY.semantic,+ PROVIDER_CONSENT_KEY.provider,+]);++/** What the resolver is asked to resolve. */+export interface ProviderConsentRequest {+ /**+ * The configuration file the record is read from. `undefined` reads the+ * process default path; `null` reads no file at all, which is what an+ * environment-only resolution means.+ */+ readonly configPath?: string | null;+ /** The environment, for the environment-over-config rule. */+ readonly env?: NodeJS.ProcessEnv;+ /**+ * The resolved semantic configuration. Optional: the consent state does+ * not depend on it, but the capability tier does, and a caller that+ * holds one should not have to resolve the tier twice.+ */+ readonly semantic?: ResolvedEmbeddingConfig;+}++/** What the resolver concluded. */+export interface ProviderConsent {+ /** Whether a decline was recorded. */+ readonly state: ProviderConsentState;+ /**+ * The shared capability tier, when the caller supplied a resolved+ * configuration. `null` when it did not - never a second ladder.+ */+ readonly capability: SemanticCapability | null;+ /**+ * The keys that record the decline, in the order they were found. Empty+ * when the state is unset, and non-empty for every declined state whose+ * record could be read - an unreadable record names no key, because it+ * named none.+ */+ readonly declineKeys: ReadonlyArray<string>;+ /** The keys that would re-enable a declined provider. */+ readonly reenableKeys: ReadonlyArray<string>;+ /** The operator-facing sentence. Never empty. */+ readonly detail: string;+}++/**+ * The sentence a refusal carries. Named in the message as a KEY, so an+ * operator reading it can search the configuration file for the spelling+ * rather than for a phrase.+ */+function reenableSentence(): string {+ return (+ `re-enable it with ${PROVIDER_CONSENT_KEY.semantic}=true ` ++ `or ${PROVIDER_CONSENT_KEY.provider}=<provider>`+ );+}++/**+ * How a decline is WRITTEN. Both spellings are named, not only the one+ * this record used: an operator who wrote one of them and is looking at+ * a refusal that names the other needs to see that both spell the same+ * decision.+ */+function declineSpellingsSentence(): string {+ return (+ `a decline is written as ${PROVIDER_CONSENT_KEY.semantic}=false ` ++ `or ${PROVIDER_CONSENT_KEY.provider}=${PROVIDER_CONSENT_DECLINED_PROVIDER}`+ );+}++/** A declined consent, with the reason it was read that way. */+function declined(+ capability: SemanticCapability | null,+ declineKeys: ReadonlyArray<string>,+ reason: string,+): ProviderConsent {+ const recorded =+ declineKeys.length > 0 ? `; this one is recorded by ${declineKeys.join(" and ")}` : "";+ return Object.freeze({+ state: PROVIDER_CONSENT_STATE.declined,+ capability,+ declineKeys: Object.freeze([...declineKeys]),+ reenableKeys: PROVIDER_CONSENT_REENABLE_KEYS,+ detail:+ `provider consent declined: no embedding provider will be contacted. ` ++ `${reason}${declineSpellingsSentence()}${recorded}. ` ++ `${reenableSentence()}.`,+ });+}++/**+ * Resolve what the operator's configuration says about the embedding+ * provider.+ *+ * Pure with respect to the filesystem in one direction only: it READS the+ * configuration file it is given and never writes one. A record that+ * cannot be read resolves to declined rather than to unset, and a value+ * that cannot be parsed resolves to declined for the same reason.+ */+export function resolveProviderConsent(request: ProviderConsentRequest): ProviderConsent {+ const capability =+ request.semantic === undefined ? null : resolveSemanticCapability(request.semantic);+ const env = request.env ?? process.env;++ // The record. An unreadable one is a decline, never an absence: the+ // operator's settings exist and are not the ones in force, which is+ // exactly what the configuration reader refuses to paper over.+ let config: Readonly<Record<string, string>>;+ try {+ config = request.configPath === null ? {} : discoverConfig(request.configPath).data;+ } catch {+ return declined(+ capability,+ [],+ "The provider-consent record could not be read, and an unreadable record is not consent. ",+ );+ }++ const declineKeys: string[] = [];++ // The switch. An unparseable value is a decline too: the operator wrote+ // something they meant to be a decision, and reading it as "unset"+ // would silently drop it.+ const semanticRaw = envOrConfig(+ env,+ config,+ PROVIDER_CONSENT_ENV.semantic,+ PROVIDER_CONSENT_KEY.semantic,+ );+ if (semanticRaw !== null) {+ let enabled: boolean;+ try {+ enabled = parseBool(semanticRaw, false, PROVIDER_CONSENT_KEY.semantic);+ } catch {+ return declined(+ capability,+ [PROVIDER_CONSENT_KEY.semantic],+ `The provider-consent record could not be parsed, and an unparseable record is not consent. `,+ );+ }+ if (!enabled) declineKeys.push(PROVIDER_CONSENT_KEY.semantic);+ }++ // The provider name. The sentinel value is the decline; every other+ // name is a provider the operator chose.+ const providerRaw = envOrConfig(+ env,+ config,+ PROVIDER_CONSENT_ENV.provider,+ PROVIDER_CONSENT_KEY.provider,+ );+ if (+ providerRaw !== null &&+ providerRaw.trim().toLowerCase() === PROVIDER_CONSENT_DECLINED_PROVIDER+ ) {+ declineKeys.push(PROVIDER_CONSENT_KEY.provider);+ }++ if (declineKeys.length > 0) {+ return declined(capability, declineKeys, "");+ }++ return Object.freeze({+ state: PROVIDER_CONSENT_STATE.unset,+ capability,+ declineKeys: Object.freeze([]),+ reenableKeys: PROVIDER_CONSENT_REENABLE_KEYS,+ detail: "no provider-consent decision is recorded: the embedding provider was never declined.",+ });+}++/**+ * Whether the consent is a decline. A predicate rather than a comparison+ * at the call site, so the state is read the same way everywhere.+ */+export function providerConsentIsDeclined(consent: ProviderConsent): boolean {+ return consent.state === PROVIDER_CONSENT_STATE.declined;+}diff --git a/src/core/search/ranker.ts b/src/core/search/ranker.tsindex c8fdbd04..602c73e7 100644--- a/src/core/search/ranker.ts+++ b/src/core/search/ranker.ts@@ -55,6 +55,20 @@ export interface RankerInputs {* populated by a reindex.*/readonly entityMatchByChunk?: ReadonlyMap<number, number>;+ /**+ * Optional per-chunk count of DERIVED person-variant forms the chunk+ * also carries (write-honesty-egress-consent, F3): forms the query+ * expansion DERIVED (first-initial-plus-surname, reversed word order,+ * operator nicknames) rather than forms the registry declares. Kept+ * in a lane of its own so a declared-form match always outranks a+ * derived-variant match between otherwise-equal candidates: per-match+ * 0.01, capped at 0.02 - each strictly below the declared lane above.+ * Missing entries (and the absent map) leave the layer unwired: no+ * reason entry, no breakdown key, byte-identical scores. Never a+ * retrieval mechanism - the candidate set arrives fixed and this lane+ * only reorders it.+ */+ readonly derivedEntityMatchByChunk?: ReadonlyMap<number, number>;/*** Optional per-chunk effective activation in [0, 1] (Time-Aware* Recall & Activation Suite): access-reinforced strength already@@ -135,6 +149,13 @@ const LINK_ONLY_BOOST_CAP = 0.03;const TAG_ONLY_BOOST_CAP = 0.02;/** Query entities the chunk also carries. */export const ENTITY_BOOST_CAP = 0.04;+/**+ * Derived person-variant matches (write-honesty-egress-consent, F3):+ * half the declared per-match rate and half the declared cap, so a+ * declared-form match always outranks a derived-variant match between+ * otherwise-equal candidates.+ */+export const DERIVED_ENTITY_BOOST_CAP = 0.02;/** Access-reinforced, type-decayed activation. */export const ACTIVATION_BOOST_CAP = 0.04;/** Habitual co-retrieval companions inside the candidate pool. */@@ -339,6 +360,7 @@ function buildReasons(parts: {recency: number;tierMul: number;entityBoost?: number;+ derivedEntityBoost?: number;activationBoost?: number;coAccessBoost?: number;reuseBoost?: number;@@ -358,6 +380,11 @@ function buildReasons(parts: {if (parts.entityBoost && parts.entityBoost > 0) {reasons.push(`entity_match: ${fmt(parts.entityBoost)}`);}+ // Derived person-variant lane (F3), named apart from the declared+ // entity_match so attribution can tell the two tiers apart.+ if (parts.derivedEntityBoost !== undefined && parts.derivedEntityBoost > 0) {+ reasons.push(`entity_variant: ${fmt(parts.derivedEntityBoost)}`);+ }if (parts.activationBoost && parts.activationBoost > 0) {reasons.push(`activation: ${fmt(parts.activationBoost)}`);}@@ -413,6 +440,7 @@ function buildBreakdown(parts: {recency: number;tierMul: number;entityBoost?: number;+ derivedEntityBoost?: number;activationBoost?: number;coAccessBoost?: number;reuseBoost?: number;@@ -428,6 +456,10 @@ function buildBreakdown(parts: {semantic: parts.semanticScore,rrf: parts.rrf ?? 0,entity: parts.entityBoost ?? 0,+ // Derived person-variant lane (F3): present only when the chunk+ // actually matched a derived form - absent, not zero, otherwise -+ // so a result without derived matches keeps the pre-feature shape.+ ...(parts.derivedEntityBoost !== undefined ? { entityDerived: parts.derivedEntityBoost } : {}),activation: parts.activationBoost ?? 0,coAccess: parts.coAccessBoost ?? 0,reuse: parts.reuseBoost ?? 0,@@ -653,6 +685,15 @@ export function rankResults(inputs: RankerInputs, opts: RankerOptions): BrainSea// relevant set - never enough to float an irrelevant chunk.const entityMatches = inputs.entityMatchByChunk?.get(c.chunkId) ?? 0;const entityBoost = Math.min(ENTITY_BOOST_CAP, entityMatches * 0.02 * entMul);+ // Derived person-variant lane (F3): strictly below the declared+ // lane above, capped, and reported by ABSENCE when it does not+ // fire (undefined, not zero) so a result without derived matches+ // keeps its pre-feature breakdown shape.+ const derivedEntityMatches = inputs.derivedEntityMatchByChunk?.get(c.chunkId) ?? 0;+ const derivedEntityBoost =+ derivedEntityMatches > 0+ ? Math.min(DERIVED_ENTITY_BOOST_CAP, derivedEntityMatches * 0.01 * entMul)+ : undefined;// Activation boost (Time-Aware Recall & Activation Suite): the// effective (type-decayed) activation scales into a capped 0.04// contribution - a re-ranker for habitually-recalled memories,@@ -712,6 +753,10 @@ export function rankResults(inputs: RankerInputs, opts: RankerOptions): BrainSeaconst linkBoostVoted = zeroWithoutVote("link", linkBoost);const recencyVoted = zeroWithoutVote("recency", recency);const entityBoostVoted = zeroWithoutVote("entity", entityBoost);+ const derivedEntityBoostVoted =+ derivedEntityBoost === undefined+ ? undefined+ : zeroWithoutVote("entityDerived", derivedEntityBoost);const activationBoostVoted = zeroWithoutVote("activation", activationBoost);const coAccessBoostVoted = zeroWithoutVote("coAccess", coAccessBoost);const reuseBoostVoted = zeroWithoutVote("reuse", reuseBoost);@@ -731,6 +776,7 @@ export function rankResults(inputs: RankerInputs, opts: RankerOptions): BrainSealinkBoostVoted +recencyVoted +entityBoostVoted ++ (derivedEntityBoostVoted ?? 0) +activationBoostVoted +coAccessBoostVoted +reuseBoostVoted +@@ -777,6 +823,7 @@ export function rankResults(inputs: RankerInputs, opts: RankerOptions): BrainSearecency: recencyVoted,tierMul,entityBoost: entityBoostVoted,+ derivedEntityBoost: derivedEntityBoostVoted,activationBoost: activationBoostVoted,coAccessBoost: coAccessBoostVoted,...(trend !== undefined ? { trend, trendMul } : {}),@@ -798,6 +845,7 @@ export function rankResults(inputs: RankerInputs, opts: RankerOptions): BrainSearecency: recencyVoted,tierMul,entityBoost: entityBoostVoted,+ derivedEntityBoost: derivedEntityBoostVoted,activationBoost: activationBoostVoted,coAccessBoost: coAccessBoostVoted,trendMul,@@ -859,6 +907,25 @@ export function rankResults(inputs: RankerInputs, opts: RankerOptions): BrainSeaconst aRelationalOnly = relationalOnlyIds.has(a.chunkId);const bRelationalOnly = relationalOnlyIds.has(b.chunkId);if (aRelationalOnly !== bRelationalOnly) return aRelationalOnly ? 1 : -1;+ // Structural declared-before-derived entity precedence: candidates+ // are partitioned into TIERS - declared, then neutral, then+ // derived-only - and the tiers compare before scores, so a declared+ // match always outranks a derived-only one regardless of entityMul.+ // A tier, not a pairwise rule: declared-beats-derived-only answered+ // pairwise while neutral candidates fell through to the score, which+ // is not a strict weak ordering (declared A < derived B, derived B <+ // neutral C, neutral C < declared A) and made the sort output depend+ // on the input order.+ const aDeclared = (inputs.entityMatchByChunk?.get(a.chunkId) ?? 0) > 0;+ const bDeclared = (inputs.entityMatchByChunk?.get(b.chunkId) ?? 0) > 0;+ const aDerivedOnly =+ aDeclared === false && (inputs.derivedEntityMatchByChunk?.get(a.chunkId) ?? 0) > 0;+ const bDerivedOnly =+ bDeclared === false && (inputs.derivedEntityMatchByChunk?.get(b.chunkId) ?? 0) > 0;+ const tierOf = (declared: boolean, derivedOnly: boolean): number =>+ declared ? 0 : derivedOnly ? 2 : 1;+ const tierDelta = tierOf(aDeclared, aDerivedOnly) - tierOf(bDeclared, bDerivedOnly);+ if (tierDelta !== 0) return tierDelta;if (b.score !== a.score) return b.score - a.score;if (temporalIntent !== null) {const aTemporal = a.breakdown?.temporal ?? 0;diff --git a/src/core/search/result-filters.ts b/src/core/search/result-filters.tsindex d1a3db51..c7015bfa 100644--- a/src/core/search/result-filters.ts+++ b/src/core/search/result-filters.ts@@ -19,6 +19,7 @@ import {isVisible,pageVisibility,} from "../graph/visibility.ts";+import { remoteDenyVisibilityTokens } from "../graph/privacy-markers.ts";import { TRANSPORT_REACH, type TransportReach } from "../graph/transport-reach.ts";import { isOwnerVisible, pageOwner } from "../graph/agent-scope.ts";import { scopeAxisReachable, scopeFromFrontmatter, type CompositeScope } from "../scope-key.ts";@@ -360,6 +361,14 @@ export function applyVisibilityScope(* pass this check - the file said "public" - and a remote search served* the private body. Omitted by surfaces that serve the file itself.*+ * The file's verdict comes from {@link remoteDenyVisibilityTokens}, not+ * from `pageVisibility` alone: a page that states its reservation with an+ * informal marker (`access: private`, `confidentiality: secret`,+ * `sensitivity: confidential`) is remote-denied here exactly as one that+ * spells `visibility: private`. The marker adds no caller-scope token -+ * {@link applyVisibilityScope} still reads `pageVisibility` - so the+ * caller-liftable vocabulary is unchanged.+ ** A path that resolves OUTSIDE the vault answers `false` at every reach,* before any read: it names no page anyone could be shown, and some* callers hand this function a path the caller of a tool chose, which@@ -390,7 +399,7 @@ export function isPathReadableAtReach(return false;}const entry = readCachedFrontmatterEntry(frontmatterCache, vault, path);- const tags = entry.unreadable ? UNMEASURABLE_VISIBILITY : pageVisibility(entry.meta);+ const tags = entry.unreadable ? UNMEASURABLE_VISIBILITY : remoteDenyVisibilityTokens(entry.meta);return isRemotelyReadable(tags, reach) && isRemotelyReadable(indexedTags, reach);}diff --git a/src/core/search/schema.ts b/src/core/search/schema.tsindex 2f25941a..01acedee 100644--- a/src/core/search/schema.ts+++ b/src/core/search/schema.ts@@ -10,7 +10,7 @@import { Database } from "bun:sqlite";import { SearchError } from "./types.ts";-import { pageVisibility } from "../graph/visibility.ts";+import { remoteDenyVisibilityTokens } from "../graph/privacy-markers.ts";import { parseFrontmatterText } from "../vault.ts";import { closeDatabase } from "../sqlite-close.ts";@@ -19,7 +19,7 @@ import { closeDatabase } from "../sqlite-close.ts";* this raises `SCHEMA_MISMATCH` on open — the operator must reindex* with a newer binary.*/-export const LATEST_SCHEMA_VERSION = 14;+export const LATEST_SCHEMA_VERSION = 15;/*** The one command that rebuilds an index this binary cannot read. Every@@ -712,6 +712,40 @@ export const MIGRATIONS: ReadonlyArray<Migration> = Object.freeze([backfillDocumentAttributes(db);},},+ {+ // v15 (chunk skip-and-hold, t_f029b413) - the persistent hold+ // record: one row per (chunk, hold-decision key) for chunks the+ // embedding phase PROVED over a model's declared input window.+ //+ // The key is the whole design. `PENDING_ANTI_JOIN` excludes a chunk+ // only while a row matches the CURRENT (model, window, signature)+ // triple, so a config change releases holds back into pending with+ // no migration and no delete pass, while an unchanged config keeps+ // them held without a second provider contact. The FK cascade is+ // the garbage collector: re-indexing a document replaces its chunk+ // rows, and a hold against a chunk that no longer exists dies with+ // it rather than keying a ghost.+ //+ // No backfill, deliberately: a hold is a decision about a CONFIG -+ // which window which model declares - and a migration has no config+ // to consult. An index predating this version keeps its oversize+ // chunks pending until the next embedding pass decides, and records,+ // for them.+ version: 15,+ up(db) {+ db.exec(`+ CREATE TABLE IF NOT EXISTS ${CHUNK_HOLDS_TABLE} (+ chunk_id INTEGER NOT NULL REFERENCES chunks(id) ON DELETE CASCADE,+ model TEXT NOT NULL,+ window_tokens INTEGER NOT NULL,+ embedding_signature TEXT NOT NULL,+ estimated_tokens INTEGER NOT NULL,+ held_at TEXT NOT NULL,+ PRIMARY KEY (chunk_id, model, window_tokens, embedding_signature)+ ) WITHOUT ROWID;+ `);+ },+ },]);/**@@ -754,6 +788,15 @@ export const CHUNK_KIND_SECTION = "section";/** The `document_attributes` table (v14), named once for the same reason. */export const DOCUMENT_ATTRIBUTES_TABLE = "document_attributes";+/**+ * The `chunk_embedding_holds` table (v15): the persistent record of the+ * embedding phase's skip-and-hold decisions, keyed by the chunk and the+ * (model, window, signature) triple that made the proof. Named once: the+ * migration, the store's anti-join exclusion and the hold report all+ * spell it from here.+ */+export const CHUNK_HOLDS_TABLE = "chunk_embedding_holds";+/*** The frontmatter keys the attributes extractor never copies: each keeps a* dedicated column or table as the single source of truth (`type` ->@@ -919,6 +962,13 @@ export function encodeDocumentVisibility(tokens: ReadonlyArray<string>): string* Fill {@link DOCUMENT_VISIBILITY_COLUMN} for every row that has no value* yet, from the frontmatter chunk the index already holds.*+ * The tokens come from {@link remoteDenyVisibilityTokens} - the declared+ * `visibility:` plus any informal privacy marker's reservation - because+ * that is exactly what the indexer stamps on a live write. Backfilling+ * from the declaration alone would silently UN-reserve every marker page+ * this migration touches: the row would answer differently after a+ * migration than the same file answers after a re-index.+ ** A document with no chunk zero is left NULL - the index holds nothing to* measure, and saying "no tokens" about it would be a claim nobody made.*/@@ -937,7 +987,7 @@ function backfillDocumentVisibility(db: Database): void {for (const row of rows) {if (row.content === null) continue;const [meta] = parseFrontmatterText(row.content);- update.run(encodeDocumentVisibility(pageVisibility(meta)), row.id);+ update.run(encodeDocumentVisibility(remoteDenyVisibilityTokens(meta)), row.id);}}diff --git a/src/core/search/search-result.ts b/src/core/search/search-result.tsindex 00c355bb..6f6375f1 100644--- a/src/core/search/search-result.ts+++ b/src/core/search/search-result.ts@@ -27,6 +27,15 @@ export interface ScoreBreakdown {readonly semantic: number;readonly rrf: number;readonly entity: number;+ /**+ * Derived person-variant entity boost (write-honesty-egress-consent,+ * F3): the strictly smaller capped lane under `entity`, fed by forms+ * the query expansion DERIVED (initials, word order, operator+ * nicknames) rather than declared registry forms. Absent - not zero -+ * when the chunk matched no derived form, so a result without derived+ * matches keeps the pre-feature breakdown shape byte for byte.+ */+ readonly entityDerived?: number;readonly activation: number;readonly coAccess: number;/** Observed-reuse boost (t_65588d8b); 0 when no verdicts apply. */diff --git a/src/core/search/semantic-phase.ts b/src/core/search/semantic-phase.tsindex f0f7d11d..ab2ae5bc 100644--- a/src/core/search/semantic-phase.ts+++ b/src/core/search/semantic-phase.ts@@ -10,6 +10,7 @@ import {isBlockedCapability,resolveSemanticCapability,semanticCapabilityLabel,+ SEMANTIC_CAPABILITY_TIER,} from "./capability-tier.ts";import { classifyEmbeddingError } from "./embeddings/openai-compat.ts";import { makeProvider } from "./embeddings/provider.ts";@@ -19,6 +20,7 @@ import {queryEmbedEmptyFitMessage,queryEmbedRefusalMessage,} from "./embeddings/query-embed.ts";+import { providerConsentIsDeclined, resolveProviderConsent } from "./provider-consent.ts";import { RETRIEVAL_DEGRADATION, noteDegradation } from "./retrieval-trail.ts";import { Store } from "./store.ts";import { EMBEDDING_QUOTA_MESSAGE, SearchError } from "./types.ts";@@ -143,6 +145,31 @@ export async function runSemanticPhase(const warnings: string[] = [];const degraded: RetrievalDegradationSink = [];+ // The operator's WRITTEN decision comes before every other arm, and+ // before the capability ladder below. A decline is a more specific fact+ // than a tier: the tier says what the configuration can do, and it+ // cannot say that the operator said no. Reporting the tier for a+ // declined vault would answer "not configured" for a configuration that+ // was configured on purpose - and the sentence it would use is the one+ // that tells an operator to set a provider they deliberately declined.+ //+ // The record is read from the SAME configuration the search+ // configuration was resolved from (`config.configPath`), so the two can+ // never disagree about which file is in force.+ const consent = resolveProviderConsent({+ configPath: config.configPath ?? null,+ env: process.env,+ semantic: config.semantic,+ });+ if (providerConsentIsDeclined(consent)) {+ if (opts.explicit) throw new SearchError("EMBEDDING_DISABLED", consent.detail);+ warnings.push(consent.detail);+ noteDegradation(degraded, RETRIEVAL_DEGRADATION.semanticCapabilityBlocked, {+ tier: SEMANTIC_CAPABILITY_TIER.disabled,+ });+ return { attempted: false, hits: [], warnings, degraded };+ }+const counts = store.counts();if (counts.embeddings === 0) {warnings.push("no compatible embeddings; run: o2b search index --embeddings");@@ -176,7 +203,7 @@ export async function runSemanticPhase(// was served lexical results with exit 0 was told the search succeeded.// Both arms name the condition with the SAME registry sentence, so the// warning and the error message cannot drift apart.- const capability = resolveSemanticCapability(config.semantic);+ const capability = consent.capability ?? resolveSemanticCapability(config.semantic);if (isBlockedCapability(capability)) {const label = await semanticCapabilityLabel(capability.code);if (opts.explicit) throw new SearchError(BLOCKED_TIER_ERROR_CODE[capability.tier], label);@@ -231,7 +258,7 @@ export async function runSemanticPhase(const timing = opts.timing;if (timing !== undefined) timing.preCallStartMs = timing.clock();try {- const provider = makeProvider(config.semantic);+ const provider = makeProvider(config.semantic, consent);const vectors = await provider.embed([prepared.text],"query",diff --git a/src/core/search/serialize.ts b/src/core/search/serialize.tsindex b93b6267..f2c807f9 100644--- a/src/core/search/serialize.ts+++ b/src/core/search/serialize.ts@@ -194,6 +194,23 @@ export function serializeIndexStatus(s: IndexStatusSnapshot): Record<string, unk...(s.chunkWindowUndeclared === undefined? {}: { chunk_window_undeclared: serializeChunkWindowCensus(s.chunkWindowUndeclared) }),+ // The same terms as the census field above: present only when the+ // index actually holds chunks under the active key, so a status with+ // nothing held serializes the bytes it emitted before this existed.+ ...(s.heldChunks === undefined+ ? {}+ : {+ held_chunks: {+ total: s.heldChunks.total,+ items: s.heldChunks.items.map((item) => ({+ path: item.path,+ chunk_index: item.chunkIndex,+ estimated_tokens: item.estimatedTokens,+ window_tokens: item.windowTokens,+ reason: item.reason,+ })),+ },+ }),warnings: s.warnings,};}diff --git a/src/core/search/store.ts b/src/core/search/store.tsindex afdb5c00..decbd7cc 100644--- a/src/core/search/store.ts+++ b/src/core/search/store.ts@@ -533,17 +533,43 @@ export class Store {/*** Chunks that have no row in `embeddings`. Used by the indexer to* populate vectors after a fresh index or after the model-change drop,- * optionally scoped to path prefixes.+ * optionally scoped to path prefixes and optionally excluding chunks+ * held under `hold`'s key (chunk skip-and-hold, F2).*/findChunksWithoutEmbeddings(scope?: chunks.PendingVectorScope,+ hold?: chunks.ChunkHoldExclusion,): Array<{ chunkId: number; content: string }> {- return chunks.findChunksWithoutEmbeddings(this.db, scope);+ return chunks.findChunksWithoutEmbeddings(this.db, scope, hold);}/** How many chunks the scoped census above would return, without their bodies. */- countChunksWithoutEmbeddings(scope?: chunks.PendingVectorScope): number {- return chunks.countChunksWithoutEmbeddings(this.db, scope);+ countChunksWithoutEmbeddings(+ scope?: chunks.PendingVectorScope,+ hold?: chunks.ChunkHoldExclusion,+ ): number {+ return chunks.countChunksWithoutEmbeddings(this.db, scope, hold);+ }++ /** Persist embedding-phase hold decisions under one (model, signature) stamp. */+ recordChunkHolds(+ entries: ReadonlyArray<chunks.ChunkHoldEntry>,+ stamp: chunks.ChunkHoldStamp,+ ): void {+ chunks.recordChunkHolds(this.db, entries, stamp);+ }++ /** The hold rows recorded for one chunk, whatever key made them. */+ chunkHoldRowsFor(chunkId: number): chunks.ChunkHoldRow[] {+ return chunks.chunkHoldRowsFor(this.db, chunkId);+ }++ /** Held chunks under one key: capped position-ordered rows plus the total. */+ heldChunkCensus(+ hold: chunks.ChunkHoldExclusion,+ limit: number,+ ): { total: number; rows: chunks.HeldChunkCensusRow[] } {+ return chunks.heldChunkCensus(this.db, hold, limit);}/** How many documents sit under one path prefix of a pending-vector scope. */diff --git a/src/core/search/store/chunks.ts b/src/core/search/store/chunks.tsindex 3019122a..8b798d87 100644--- a/src/core/search/store/chunks.ts+++ b/src/core/search/store/chunks.ts@@ -8,6 +8,7 @@import { Database } from "bun:sqlite";import { SearchError } from "../types.ts";+import { CHUNK_HOLDS_TABLE } from "../schema.ts";import { pathPrefixRange } from "./path-prefix.ts";import { nowIso, sqlPlaceholders } from "./sql.ts";import {@@ -362,6 +363,40 @@ export interface PendingVectorScope {const PENDING_ANTI_JOIN ="FROM chunks c LEFT JOIN embeddings e ON e.chunk_id = c.id WHERE e.chunk_id IS NULL";+/**+ * The identity a persistent chunk hold is matched under (chunk+ * skip-and-hold, F2): the (model, window, signature) triple whose proof+ * put the chunk in `chunk_embedding_holds`. The pending anti-join+ * excludes a held chunk ONLY while this triple still describes the+ * active configuration, so a window or model change releases the hold+ * back into pending without any delete pass.+ */+export interface ChunkHoldExclusion {+ readonly model: string;+ readonly windowTokens: number;+ readonly embeddingSignature: string;+}++/** One recorded hold, as read back for reporting. */+export interface ChunkHoldRow {+ readonly chunkId: number;+ readonly model: string;+ readonly windowTokens: number;+ readonly embeddingSignature: string;+ readonly estimatedTokens: number;+ readonly heldAt: string;+}++/**+ * The anti-join's exclusion of holds made under `hold`: a chunk with a+ * row matching the whole key is owed no vector by this configuration.+ * Bound after the scope's bindings, matching clause order.+ */+const HOLD_NOT_EXISTS =+ ` AND NOT EXISTS (SELECT 1 FROM ${CHUNK_HOLDS_TABLE} h ` ++ "WHERE h.chunk_id = c.id AND h.model = ? AND h.window_tokens = ? " ++ "AND h.embedding_signature = ?)";+/*** The anti-join's FROM/WHERE text and bindings for a scope. A scoped* census joins `documents` and matches each prefix with the sargable@@ -370,12 +405,18 @@ const PENDING_ANTI_JOIN =* the match. A prefix whose last code point cannot increment has a null* bound and contributes its lower-bound-only clause.*/-function pendingCensusClause(scope: PendingVectorScope | undefined): {- sql: string;- bindings: string[];-} {+function pendingCensusClause(+ scope: PendingVectorScope | undefined,+ hold: ChunkHoldExclusion | undefined,+): { sql: string; bindings: (string | number)[] } {const prefixes = scope?.pathPrefixes ?? [];- if (prefixes.length === 0) return { sql: PENDING_ANTI_JOIN, bindings: [] };+ const holdBindings: (string | number)[] = hold+ ? [hold.model, hold.windowTokens, hold.embeddingSignature]+ : [];+ const holdClause = hold ? HOLD_NOT_EXISTS : "";+ if (prefixes.length === 0) {+ return { sql: PENDING_ANTI_JOIN + holdClause, bindings: holdBindings };+ }const ranges = prefixes.map((prefix) => pathPrefixRange(prefix));const matches = ranges.map((range) => (range.hi !== null ? "(d.path >= ? AND d.path < ?)" : "(d.path >= ?)"))@@ -384,23 +425,28 @@ function pendingCensusClause(scope: PendingVectorScope | undefined): {sql:"FROM chunks c JOIN documents d ON d.id = c.document_id " +"LEFT JOIN embeddings e ON e.chunk_id = c.id " +- `WHERE e.chunk_id IS NULL AND (${matches})`,- bindings: ranges.flatMap((range) => (range.hi !== null ? [range.lo, range.hi] : [range.lo])),+ `WHERE e.chunk_id IS NULL AND (${matches})${holdClause}`,+ bindings: [+ ...ranges.flatMap((range) => (range.hi !== null ? [range.lo, range.hi] : [range.lo])),+ ...holdBindings,+ ],};}/*** Chunks that have no row in `embeddings`. Used by the indexer to* populate vectors after a fresh index or after the model-change drop,- * optionally scoped to path prefixes.+ * optionally scoped to path prefixes and optionally excluding the chunks+ * held back under `hold`'s key (chunk skip-and-hold, F2).*/export function findChunksWithoutEmbeddings(db: Database,scope?: PendingVectorScope,+ hold?: ChunkHoldExclusion,): Array<{ chunkId: number; content: string }> {- const { sql, bindings } = pendingCensusClause(scope);+ const { sql, bindings } = pendingCensusClause(scope, hold);const rows = db- .query<{ id: number; content: string }, string[]>(+ .query<{ id: number; content: string }, (string | number)[]>(`SELECT c.id AS id, c.content AS content ${sql} ORDER BY c.id`,).all(...bindings);@@ -412,15 +458,195 @@ export function findChunksWithoutEmbeddings(* single one of their bodies.** The same anti-join {@link findChunksWithoutEmbeddings} walks, under the- * same scope, as a `COUNT(*)`. The row-returning form is the indexer's- * work queue and loads every pending chunk's full `content` into JS- * memory to hand it to a provider; a diagnostic that only wants the- * number must not pay that, which is the whole reason this second query- * exists rather than a `.length` on the first.+ * same scope and hold exclusion, as a `COUNT(*)`. The row-returning form+ * is the indexer's work queue and loads every pending chunk's full+ * `content` into JS memory to hand it to a provider; a diagnostic that+ * only wants the number must not pay that, which is the whole reason this+ * second query exists rather than a `.length` on the first.+ */+export function countChunksWithoutEmbeddings(+ db: Database,+ scope?: PendingVectorScope,+ hold?: ChunkHoldExclusion,+): number {+ const { sql, bindings } = pendingCensusClause(scope, hold);+ return (+ db.query<{ n: number }, (string | number)[]>(`SELECT COUNT(*) AS n ${sql}`).get(...bindings)+ ?.n ?? 0+ );+}++/** One hold decision to record: the chunk and the proof's numbers. */+export interface ChunkHoldEntry {+ readonly chunkId: number;+ readonly estimatedTokens: number;+ readonly windowTokens: number;+}++/** The (model, signature) pair every hold of one phase run shares. */+export interface ChunkHoldStamp {+ readonly model: string;+ readonly embeddingSignature: string;+}++/**+ * Record the embedding phase's hold decisions (chunk skip-and-hold, F2).+ *+ * `INSERT OR REPLACE` on the (chunk, key) primary key: re-deciding the+ * same hold refreshes its estimate and timestamp rather than stacking a+ * second row, which is what keeps a hand-in-plan re-run of the phase+ * free of duplicate state.*/-export function countChunksWithoutEmbeddings(db: Database, scope?: PendingVectorScope): number {- const { sql, bindings } = pendingCensusClause(scope);- return db.query<{ n: number }, string[]>(`SELECT COUNT(*) AS n ${sql}`).get(...bindings)?.n ?? 0;+export function recordChunkHolds(+ db: Database,+ entries: ReadonlyArray<ChunkHoldEntry>,+ stamp: ChunkHoldStamp,+): void {+ if (entries.length === 0) return;+ const heldAt = nowIso();+ const insert = db.query<never, [number, string, number, string, number, string]>(+ `INSERT OR REPLACE INTO ${CHUNK_HOLDS_TABLE}` ++ "(chunk_id, model, window_tokens, embedding_signature, estimated_tokens, held_at) " ++ "VALUES (?1, ?2, ?3, ?4, ?5, ?6)",+ );+ // One transaction, the way the sibling bulk writers commit: after a+ // model or window change a large held set is hundreds of rows, and one+ // autocommit (WAL fsync) per row prices the run in syscalls while a+ // failure halfway leaves a partial record beside an in-memory tally+ // that already counts every hold.+ db.transaction(() => {+ for (const entry of entries) {+ insert.run(+ entry.chunkId,+ stamp.model,+ entry.windowTokens,+ stamp.embeddingSignature,+ entry.estimatedTokens,+ heldAt,+ );+ }+ })();+}++/** The hold rows recorded for one chunk, whatever key made them. */+export function chunkHoldRowsFor(db: Database, chunkId: number): ChunkHoldRow[] {+ return chunkHoldRowsForMany(db, [chunkId]);+}++/** Hold rows for many chunks in one read, keyed by chunk id. */+export function chunkHoldRowsForMany(+ db: Database,+ chunkIds: ReadonlyArray<number>,+): ChunkHoldRow[] {+ if (chunkIds.length === 0) return [];+ const placeholders = sqlPlaceholders(chunkIds);+ return db+ .query<+ {+ chunk_id: number;+ model: string;+ window_tokens: number;+ embedding_signature: string;+ estimated_tokens: number;+ held_at: string;+ },+ number[]+ >(+ `SELECT chunk_id, model, window_tokens, embedding_signature, estimated_tokens, held_at ` ++ `FROM ${CHUNK_HOLDS_TABLE} WHERE chunk_id IN (${placeholders}) ORDER BY chunk_id, model`,+ )+ .all(...chunkIds)+ .map((r) => ({+ chunkId: r.chunk_id,+ model: r.model,+ windowTokens: r.window_tokens,+ embeddingSignature: r.embedding_signature,+ estimatedTokens: r.estimated_tokens,+ heldAt: r.held_at,+ }));+}++/** One held chunk with the position that names it outside the index. */+export interface HeldChunkCensusRow {+ readonly path: string;+ readonly chunkIndex: number;+ readonly estimatedTokens: number;+ readonly windowTokens: number;+}++/**+ * The held chunks recorded under one key, with the UNCAPPED total. The+ * rows come back position-ordered and at most `limit` of them, so the+ * caller caps without loading a vault's whole hold list.+ */+export function heldChunkCensus(+ db: Database,+ hold: ChunkHoldExclusion,+ limit: number,+): { total: number; rows: HeldChunkCensusRow[] } {+ // A hold row whose chunk has a vector is not a hold any more (window+ // up, chunk embedded, window back): the report answers to the same fact+ // the pending anti-join answers to, so the operator is never warned+ // about work that is already done.+ const total =+ db+ .query<{ n: number }, (string | number)[]>(+ `SELECT COUNT(*) AS n FROM ${CHUNK_HOLDS_TABLE} h ` ++ "WHERE h.model = ? AND h.window_tokens = ? AND h.embedding_signature = ? " ++ "AND NOT EXISTS (SELECT 1 FROM embeddings e WHERE e.chunk_id = h.chunk_id)",+ )+ .get(hold.model, hold.windowTokens, hold.embeddingSignature)?.n ?? 0;+ if (total === 0) return { total: 0, rows: [] };+ const rows = db+ .query<+ { path: string; chunk_index: number; estimated_tokens: number; window_tokens: number },+ (string | number)[]+ >(+ "SELECT d.path AS path, c.chunk_index AS chunk_index, " ++ "h.estimated_tokens AS estimated_tokens, h.window_tokens AS window_tokens " ++ `FROM ${CHUNK_HOLDS_TABLE} h ` ++ "JOIN chunks c ON c.id = h.chunk_id " ++ "JOIN documents d ON d.id = c.document_id " ++ "WHERE h.model = ? AND h.window_tokens = ? AND h.embedding_signature = ? " ++ "AND NOT EXISTS (SELECT 1 FROM embeddings e WHERE e.chunk_id = h.chunk_id) " ++ "ORDER BY d.path, c.chunk_index LIMIT ?",+ )+ .all(hold.model, hold.windowTokens, hold.embeddingSignature, limit)+ .map((r) => ({+ path: r.path,+ chunkIndex: r.chunk_index,+ estimatedTokens: r.estimated_tokens,+ windowTokens: r.window_tokens,+ }));+ return { total, rows };+}++/** Held chunks aggregated to the source document that carries them. */+export interface HeldChunkDocRow {+ readonly path: string;+ readonly heldChunks: number;+ readonly maxEstimatedTokens: number;+}++/**+ * Per-document counts of the holds recorded under one key, document-+ * ordered. The doctor's held-sources check speaks per document, and the+ * aggregation belongs to the SQL rather than to a caller that would+ * have to load every held row to do it.+ */+export function heldChunkDocCensus(db: Database, hold: ChunkHoldExclusion): HeldChunkDocRow[] {+ return db+ .query<{ path: string; held: number; max_est: number }, (string | number)[]>(+ "SELECT d.path AS path, COUNT(*) AS held, MAX(h.estimated_tokens) AS max_est " ++ `FROM ${CHUNK_HOLDS_TABLE} h ` ++ "JOIN chunks c ON c.id = h.chunk_id " ++ "JOIN documents d ON d.id = c.document_id " ++ "WHERE h.model = ? AND h.window_tokens = ? AND h.embedding_signature = ? " ++ "AND NOT EXISTS (SELECT 1 FROM embeddings e WHERE e.chunk_id = h.chunk_id) " ++ "GROUP BY d.path ORDER BY d.path",+ )+ .all(hold.model, hold.windowTokens, hold.embeddingSignature)+ .map((r) => ({ path: r.path, heldChunks: r.held, maxEstimatedTokens: r.max_est }));}export function hydrateChunks(diff --git a/src/core/search/types.ts b/src/core/search/types.tsindex cf280fbd..4bdf0087 100644--- a/src/core/search/types.ts+++ b/src/core/search/types.ts@@ -247,6 +247,58 @@ export function chunkWindowDiagnosticCode(census: ChunkWindowCensus): string {return CHUNK_WINDOW_CODE[census.verdict];}+/**+ * Why the embedding phase held a chunk back instead of embedding it+ * (chunk skip-and-hold, F2). One value today; a closed union so a second+ * hold reason - a per-request budget, say - cannot be added by string+ * comparison at the read sites.+ */+export const CHUNK_HOLD_REASON = Object.freeze({+ /** The estimator's FLOOR already exceeds the model's declared window. */+ overDeclaredWindow: "over_declared_window",+} as const);++/** Closed union over {@link CHUNK_HOLD_REASON}. */+export type ChunkHoldReason = (typeof CHUNK_HOLD_REASON)[keyof typeof CHUNK_HOLD_REASON];++/**+ * One chunk the embedding phase held back. `estimatedTokens` is the+ * estimator's floor - the conservative bound that made the overflow a+ * proof rather than a guess - and `windowTokens` is the effective+ * declared window it was measured against, prefix included.+ */+export interface HeldChunkRecord {+ readonly chunkId: number;+ readonly estimatedTokens: number;+ readonly windowTokens: number;+ readonly reason: ChunkHoldReason;+}++/**+ * One held chunk as the status snapshot reports it: the same facts as a+ * {@link HeldChunkRecord} plus the position that identifies it to an+ * operator - document path and chunk index - and no database id, which+ * means nothing outside the index file.+ */+export interface HeldChunkReportItem {+ readonly path: string;+ readonly chunkIndex: number;+ readonly estimatedTokens: number;+ readonly windowTokens: number;+ readonly reason: ChunkHoldReason;+}++/**+ * What the index currently holds under the ACTIVE embedding+ * configuration's (model, window, signature) key. `items` is capped by+ * the reporting bound; `total` is the unbounded count, so a reader can+ * see how many the cap left out.+ */+export interface HeldChunkReport {+ readonly total: number;+ readonly items: ReadonlyArray<HeldChunkReportItem>;+}+export interface IndexStats {readonly added: number;readonly updated: number;@@ -260,6 +312,19 @@ export interface IndexStats {*/readonly embeddingsReused: number;readonly embeddingsRetries: number;+ /**+ * Chunks the embedding phase held back as provably over the model's+ * declared input window (chunk skip-and-hold, F2): never sent, never+ * stored, never priced. Present only when the run held something, so+ * a run that embedded everything carries the shape it carried before+ * this field existed. {@link heldChunks} names which.+ */+ readonly embeddingsHeld?: number;+ /**+ * What was held, one {@link HeldChunkRecord} per held chunk. Present+ * exactly when {@link embeddingsHeld} is.+ */+ readonly heldChunks?: ReadonlyArray<HeldChunkRecord>;readonly errors: ReadonlyArray<{readonly path: string;readonly message: string;@@ -451,6 +516,16 @@ export interface IndexStatusSnapshot {* the check passed - without spending the warning channel on it.*/readonly chunkWindowUndeclared?: ChunkWindowUndeclared;+ /**+ * Chunks the index currently holds back from embedding (chunk+ * skip-and-hold, F2), read from the durable hold records under the+ * ACTIVE configuration's (model, window, signature) key. Present only+ * when the index holds something: a status with nothing held emits the+ * bytes it emitted before this field existed, and records whose key a+ * config change released are not a hold the next run would honour, so+ * they are absent too. The list is capped; `total` names the whole.+ */+ readonly heldChunks?: HeldChunkReport;readonly warnings: ReadonlyArray<string>;}diff --git a/src/core/vault.ts b/src/core/vault.tsindex c28990cf..da29696e 100644--- a/src/core/vault.ts+++ b/src/core/vault.ts@@ -65,11 +65,8 @@ import {} from "./integrity/degradation.ts";import { vaultRelative } from "./path-safety.ts";import { type TransportReach } from "./graph/transport-reach.ts";-import {- REMOTE_DENY_VISIBILITY_TOKEN,- isRemotelyReadable,- pageVisibility,-} from "./graph/visibility.ts";+import { remoteDenyVisibilityTokens } from "./graph/privacy-markers.ts";+import { REMOTE_DENY_VISIBILITY_TOKEN, isRemotelyReadable } from "./graph/visibility.ts";import type { FrontmatterMap, FrontmatterValue, VaultPage } from "./types.ts";/**@@ -677,6 +674,13 @@ const UNMEASURABLE_PAGE_VISIBILITY: ReadonlyArray<string> = Object.freeze([* showing you". A vault that never wrote the token is byte-identical at* both reaches.*+ * The reservation is read through {@link remoteDenyVisibilityTokens}, so+ * an informal privacy marker (`access: private`, `confidentiality:+ * secret`, `sensitivity: confidential`) withholds the page exactly as a+ * `visibility: private` declaration does. The marker is not a+ * caller-scope token and this walk knows no scopes; it answers the reach+ * question only.+ ** FAILS CLOSED on a page whose file could not be read, which is the one* input the three roots must not disagree about. The parsed map resolves* an unreadable file to `{}`, and reading that as "declares nothing"@@ -698,7 +702,7 @@ export function listVaultPages(vaultDir: string, opts: ListVaultPagesOptions): Vconst pages = walked.filter((w) =>isRemotelyReadable(- w.unreadable ? UNMEASURABLE_PAGE_VISIBILITY : pageVisibility(w.page.metadata),+ w.unreadable ? UNMEASURABLE_PAGE_VISIBILITY : remoteDenyVisibilityTokens(w.page.metadata),opts.reach,),)diff --git a/src/mcp/search-tools.ts b/src/mcp/search-tools.tsindex 2e439659..4c0a632b 100644--- a/src/mcp/search-tools.ts+++ b/src/mcp/search-tools.ts@@ -2000,8 +2000,17 @@ export async function buildSearchStatusBlock(ctx: ServerContext): Promise<Recordwarnings,...rest} = serializeIndexStatus(snap);+ // Held chunks travel as a COUNT, never as the item list: the items'+ // paths are an operator diagnostic (`o2b search status`, the doctor),+ // while this block reaches callers a transport may serve at `remote`+ // reach, and a held chunk's path is exactly the location a reservation+ // may want kept off the wire. The total is what an agent needs to+ // know - work is waiting that the active configuration will not spend.+ const heldChunks = rest["held_chunks"] as { total: number } | undefined;+ const trimmed =+ heldChunks === undefined ? rest : { ...rest, held_chunks: { total: heldChunks.total } };const reportable = Array.isArray(warnings) ? warnings : [];- return reportable.length > 0 ? { ...rest, warnings: reportable } : rest;+ return reportable.length > 0 ? { ...trimmed, warnings: reportable } : trimmed;} catch (e) {return { exists: false, error: e instanceof Error ? e.message : String(e) };}diff --git a/tests/cli/brain-hallways.test.ts b/tests/cli/brain-hallways.test.tsindex a52810cb..5940cea8 100644--- a/tests/cli/brain-hallways.test.ts+++ b/tests/cli/brain-hallways.test.ts@@ -264,6 +264,26 @@ describe("o2b brain hallways promote", () => {expect(page).not.toContain("[[ent-symbol-sessionstore]]");});+ test("a pair partner under a family the fold does not mine contributes nothing", async () => {+ seedPair(3);+ // A person-family row is not a mention the scan can re-reach (the+ // fold mines symbol/url/path/qualified-ident only), so a hand-planted+ // person key never becomes a pair partner: the promote neither fails+ // nor relates, whatever the entity registry holds under that spelling.+ for (let i = 1; i <= 3; i++) mention("person", "smith robert", `s${i}.jsonl`);+ await runCli(["brain", "entity", "set", "people", "Robert Smith", "--json"], { env: env() });++ const out = await runCli(["brain", "hallways", "promote", "symbol", "SessionStore", "--json"], {+ env: env(),+ });++ expect(out.returncode).toBe(0);+ const payload = JSON.parse(out.stdout) as Record<string, unknown>;+ expect(payload["created"]).toBe(true);+ expect(payload["related"]).toEqual([]);+ expect(entityFiles()).toContain("symbol/ent-symbol-sessionstore.md");+ });+test("promotes under the same bounded-wait deadline the mine fold carries", async () => {seedPair(3);// A clock that jumps a full day per reading, so any budget the safeguarddiff --git a/tests/cli/brain-help-write-honesty.test.ts b/tests/cli/brain-help-write-honesty.test.tsnew file mode 100644index 00000000..260342e7--- /dev/null+++ b/tests/cli/brain-help-write-honesty.test.ts@@ -0,0 +1,50 @@+/**+ * Help text for the write-honesty surfaces (F4, t_fb442a24; F1 restamp).+ *+ * `o2b brain <verb> --help` prints `VERB_HELP[verb]` and exits 0; a verb+ * with no entry dumps the whole `BRAIN_HELP` and exits 2, which reads as a+ * usage error. Both defects pinned here were discoverability dead ends:+ * the claims surface said nothing about the `undated-quantity` finding it+ * is the exit for, and `privacy-marker-restamp` had no entry at all.+ */++import { describe, expect, test } from "bun:test";++import { runCli } from "../helpers/run-cli.ts";++describe("o2b brain claims --help", () => {+ test("names the undated-quantity finding class and its structural hatches", async () => {+ const res = await runCli(["brain", "claims", "--help"]);+ expect(res.returncode).toBe(0);+ expect(res.stdout.startsWith("usage:")).toBe(true);+ expect(res.stdout).toContain("undated-quantity");+ expect(res.stdout).toContain("valid_from");+ expect(res.stdout).toContain("[Source:");+ expect(res.stdout).not.toContain("!");+ });+});++describe("o2b brain privacy-marker-restamp --help", () => {+ test("has its own entry rather than the whole brain help", async () => {+ const res = await runCli(["brain", "privacy-marker-restamp", "--help"]);+ expect(res.returncode).toBe(0);+ expect(res.stdout.startsWith("usage: o2b brain privacy-marker-restamp")).toBe(true);+ // Dry-run default, the applying flag, and the reserve token contract.+ expect(res.stdout).toContain("--apply");+ expect(res.stdout).toContain("dry-run");+ expect(res.stdout).toContain("visibility:");+ expect(res.stdout).toContain("idempotent");+ expect(res.stdout).toContain("never removes");+ // The three informal markers the verb looks for, named factually.+ expect(res.stdout).toContain("access");+ expect(res.stdout).toContain("confidentiality");+ expect(res.stdout).toContain("sensitivity");+ expect(res.stdout).not.toContain("!");+ });++ test("o2b brain --help lists the verb", async () => {+ const res = await runCli(["brain", "--help"]);+ expect(res.returncode).toBe(0);+ expect(res.stdout).toContain("privacy-marker-restamp");+ });+});diff --git a/tests/cli/terminal-state-census.test.ts b/tests/cli/terminal-state-census.test.tsindex 5df30f49..dc67dda8 100644--- a/tests/cli/terminal-state-census.test.ts+++ b/tests/cli/terminal-state-census.test.ts@@ -152,7 +152,7 @@ const CLASSIFIED_BY_HAND: Readonly<Record<string, StateNote>> = Object.freeze({* why an entry here owes a written reason, passes the gates a* hand-classified entry passes, and counts against the same cap.*- * Three of these five carry a second argument, about the EMPTY RESULT+ * Three of these four carry a second argument, about the EMPTY RESULT* each verb can return. That is a different question from the one the* three classes ask - it is a property of one path through a verb, not of* the verb's terminal states - and it is recorded here because the state@@ -186,6 +186,13 @@ const CLASSIFIED_BY_MODULE_WALK: Readonly<Record<string, StateNote>> = Object.fr"judgement over content that naming any one of would be a guess dressed as an " +"instruction.",},+ // `o2b brain privacy-marker-restamp` also classifies in its own file+ // now: its per-page apply refusal prints through `process.stderr.write`,+ // so hop 0 sees the refusal spelling and the walk is no longer what+ // classifies it. Its real vault refusal is still `brainVerbContext` in+ // `brain/helpers.ts`; the reason its EMPTY result names no exit travels+ // with the verb here - the dry-run default is the answer surface, and+ // zero candidates means the tightened posture already holds.// `o2b search check` sat here while its only refusal was the shared// `resolveConfig` one, a module hop away. It classifies in its own file// now: `--integrity` (what-the-index-already-knew, unit K) writes thediff --git a/tests/core/architecture/content-class-census.test.ts b/tests/core/architecture/content-class-census.test.tsnew file mode 100644index 00000000..141962a8--- /dev/null+++ b/tests/core/architecture/content-class-census.test.ts@@ -0,0 +1,360 @@+/**+ * Census over the content-class egress policy (F5, tasks E3 and E4).+ *+ * ## The defect+ *+ * The reserved `content_class: prompt-derived` stamp marks material an+ * agent produced from a prompt, and the default egress policy leaves such+ * pages out of every shared export. The exclusion is ONE call -+ * `egressContentClassSplit` at the guard seam+ * (`src/core/egress/guard.ts`), or the knowledge pack's block scan, which+ * composes nothing it blocks. A page-carrying export that skips the seam+ * composes agent-produced pages into the payload and lets the redactor+ * scan text the policy says should never have been composed at all: the+ * page leaves unless it happens to carry a credential shape, which is not+ * a policy, it is a coincidence.+ *+ * The same defect the egress census fixed for the guard call+ * (`egress-census.test.ts`), one policy later: wiring the seam into five+ * verbs fixes five verbs, and a declared census is what makes the sixth+ * impossible.+ *+ * ## The population, defined structurally+ *+ * A module is IN POPULATION - a page-carrying shared export - when all+ * three hold:+ *+ * 1. it declares an operator-named output destination (the same+ * unambiguous flag names the egress census reads);+ * 2. it redacts through the shared guard (`redactForEgress`);+ * 3. it composes vault page material: it imports a composer from the+ * portability surface or the brain-export modules.+ *+ * All three, because each alone over- or under-states: a destination+ * without the guard is the egress census's finding, not this one's; the+ * guard without a composer covers exports whose payload is not pages+ * (the config snapshot, continuity records, preference rows); and a+ * composer without a destination is an in-vault writer, which+ * `write-site-census.test.ts` owns.+ *+ * ## What it does not cover, stated rather than implied+ *+ * - The sharedRedactor sites whose payload is not vault PAGES: the+ * machine-config snapshot (`config-export`), the continuity export,+ * the explorer's preference rows, the credential bundle (ciphertext by+ * construction) and the decision-model routes. None imports a page+ * composer, so none is in population; the continuity and explorer+ * decisions are recorded in their registry reasons (commit 887cb142),+ * and the snapshot carries no vault content at all.+ * - A class filter delegated more than ONE hop. The knowledge-pack verb+ * composes through `selectKnowledgePack`, whose module applies the+ * filter inside `pageBlockReason`; that is the one named delegation+ * below, and a chain any longer stops being something a text census+ * can honestly claim to have checked.+ * - Per-SITE coverage inside a module. Like the guard-call census it+ * extends, the seam is asserted per declared page-carrying MODULE; two+ * collections behind one module and one split call still pass as a+ * pair (the bank bundle's graph nodes and pages are two calls today,+ * which is how the per-collection shape looks when it is done right).+ */++import { describe, expect, test } from "bun:test";+import { readFileSync } from "node:fs";+import { join } from "node:path";++import { EGRESS_REDACTION, EGRESS_SITES } from "../../../src/core/egress/registry.ts";+import { lexSource } from "../../helpers/source-lexer.ts";++const REPO_ROOT = join(import.meta.dir, "..", "..", "..");++/**+ * Where an operator is told what the class policy does. The census pins+ * the disclosure in the same style the egress census pins the network+ * disclosures: keyed on stable spellings, so an undisclosed surface is a+ * named failure rather than an implication.+ */+const OPERATOR_DOCS: ReadonlyArray<string> = Object.freeze([+ "docs/cli-reference.md",+ "SECURITY.md",+]);++/**+ * The unambiguous destination names, in the parameter-spec shape a CLI+ * flag spec and an MCP `inputSchema` property share. Read off the+ * comment-stripped view: the `"string"` is a literal, and its contents+ * must survive.+ */+const DESTINATION_NAMES: ReadonlyArray<string> = Object.freeze([+ "out",+ "output",+ "outfile",+ "outdir",+ "out-dir",+ "dest",+ "destination",+ "export",+]);++function namedStringParamRe(names: ReadonlyArray<string>): RegExp {+ return new RegExp(+ String.raw`(^|[^A-Za-z0-9_$"'\-])"?(${names.join("|")})"?\s*:\s*\{\s*type\s*:\s*"string"`,+ "m",+ );+}++const DESTINATION_RE = namedStringParamRe(DESTINATION_NAMES);++/** A composer import: the portability surface or the brain-export modules. */+const COMPOSER_IMPORT_RE =+ /from\s*"[^"]*(?:\/portability\/[^"]*|\/brain\/export(?:-transcripts)?\.ts)"/;++/** The shared egress guard. A module that calls it redacts on the way out. */+const GUARD_RE = /\bredactForEgress\s*\(/;++/** The class-filter seam: the policy runs BEFORE the payload is composed. */+const SEAM_RE = /\b(?:egressContentClassSplit|treeCarriesPromptDerived)\s*\(/;++/**+ * The one named delegation: the pack verb composes through+ * `selectKnowledgePack`, so its filter is the selection step's block+ * scan, not a call in the verb's own text.+ */+const DELEGATED_FILTER_CALL_RE = /\bselectKnowledgePack\s*\(/;+const DELEGATED_FILTER_OWNER = "src/core/brain/portability/knowledge-pack.ts";++/** The seam one delegation away: the selection step's block scan. */+function packSelectionAppliesFilter(text: string): boolean {+ const start = text.search(/^\s*function pageBlockReason\b/m);+ if (start < 0) return false;+ const end = text.indexOf("\n}\n", start);+ const body = text.slice(start, end < 0 ? undefined : end);+ return /\breadContentClass\s*\(/.test(body);+}++interface CensusFile {+ readonly path: string;+ readonly text: string;+}++/** Every `.ts` file under `src/`, as repo-relative POSIX path + text. */+function readSourceTree(): ReadonlyArray<CensusFile> {+ const glob = new Bun.Glob("src/**/*.ts");+ const files: CensusFile[] = [];+ for (const rel of glob.scanSync({ cwd: REPO_ROOT })) {+ const path = rel.split("\\").join("/");+ files.push({ path, text: readFileSync(join(REPO_ROOT, path), "utf8") });+ }+ files.sort((a, b) => (a.path < b.path ? -1 : a.path > b.path ? 1 : 0));+ return files;+}++/** True when the module is a page-carrying shared export (all three rules). */+function isPageCarryingRedactor(text: string): boolean {+ const views = lexSource(text);+ return (+ DESTINATION_RE.test(views.withoutComments) &&+ GUARD_RE.test(views.code) &&+ COMPOSER_IMPORT_RE.test(views.withoutComments)+ );+}++/**+ * The census over whatever tree it is handed: the page-carrying shared+ * exports that do NOT apply the class filter, in their own text or one+ * named delegation away. Taking the tree as an argument is what lets the+ * synthetic controls below fail - a real verb with its seam removed is+ * fed back through the same rule the live tree runs under.+ */+function classFilterCensus(files: ReadonlyArray<CensusFile>): ReadonlyArray<string> {+ const seamOwners = new Map<string, boolean>(+ files.map((f) => [f.path, SEAM_RE.test(lexSource(f.text).code)] as const),+ );+ const out: string[] = [];+ for (const file of files) {+ const views = lexSource(file.text);+ if (+ !DESTINATION_RE.test(views.withoutComments) ||+ !GUARD_RE.test(views.code) ||+ !COMPOSER_IMPORT_RE.test(views.withoutComments)+ ) {+ continue;+ }+ if (SEAM_RE.test(views.code)) continue;+ const delegated =+ DELEGATED_FILTER_CALL_RE.test(views.code) &&+ (seamOwners.get(DELEGATED_FILTER_OWNER) === true ||+ packSelectionAppliesFilter(+ files.find((f) => f.path === DELEGATED_FILTER_OWNER)?.text ?? "",+ ));+ if (!delegated) out.push(file.path);+ }+ return out;+}++const SOURCE_TREE = readSourceTree();+const POPULATION: ReadonlyArray<string> = SOURCE_TREE.filter((f) =>+ isPageCarryingRedactor(f.text),+).map((f) => f.path);++/**+ * The page-carrying shared exports, named. "Carries pages" is a property+ * of what a verb is FOR, not of its syntax, so the structural rule above+ * is checked against this list as an equality in both directions: a new+ * page-carrying export fails until it is named here, and a renamed or+ * retired one fails until the list follows it.+ */+const PAGE_CARRYING_SITES: ReadonlyArray<string> = Object.freeze([+ "src/cli/brain/verbs/bank-export.ts",+ "src/cli/brain/verbs/export.ts",+ "src/cli/brain/verbs/graph-export.ts",+ "src/cli/brain/verbs/knowledge-pack.ts",+ "src/cli/brain/verbs/okf-export.ts",+]);++const ENTRIES = Object.values(EGRESS_SITES);++describe("content-class egress census", () => {+ test("the derived population is the named page-carrying sites, as an equality", () => {+ expect([...POPULATION].toSorted()).toEqual([...PAGE_CARRYING_SITES].toSorted());+ });++ test("every page-carrying site is a declared sharedRedactor egress entry", () => {+ // The class filter extends the guard policy, so a module that filters+ // but is not declared as redacting would understate what it does -+ // the exact lie the egress census's own complement rule exists for.+ const undeclared = PAGE_CARRYING_SITES.filter(+ (path) =>+ !ENTRIES.some(+ (entry) => entry.module === path && entry.redaction === EGRESS_REDACTION.sharedRedactor,+ ),+ );+ expect(undeclared).toEqual([]);+ });++ test("every page-carrying site applies the class filter", () => {+ expect(classFilterCensus(SOURCE_TREE)).toEqual([]);+ });++ test("the pack verb's delegated filter is really there", () => {+ // The one hop the census credits. If the selection step's block scan+ // lost the class check, this fails by name AND the verb fails the+ // census above - the delegation is only as good as the delegate.+ const text = readFileSync(join(REPO_ROOT, DELEGATED_FILTER_OWNER), "utf8");+ expect(packSelectionAppliesFilter(text)).toBe(true);+ // The block list names the class beside the operator-set scopes.+ expect(text).toContain('| "prompt-derived";');+ });++ test("the named-omission sites document their class decision in the registry", () => {+ // The sharedRedactor entries that carry no page composer are out of+ // population by the structural rule, so the census cannot see them -+ // what pins them is the decision record: the registry reason says+ // what the site does about the class, in so many words. The+ // machine-config snapshot carries no vault content at all, so E3+ // wrote no class sentence for it and none is pinned here.+ const decisions: ReadonlyArray<readonly [string, string]> = Object.freeze([+ ["brain-continuity-export", "src/cli/brain/verbs/continuity.ts"],+ ["brain-explorer-export", "src/cli/brain/verbs/explorer.ts"],+ ]);+ for (const [id, module] of decisions) {+ const entry = ENTRIES.find((e) => e.id === id);+ expect(`${id}: ${entry !== undefined && entry.module === module}`).toBe(`${id}: true`);+ expect(`${id}: ${/\bcontent[-_]class\b|\bprompt-derived\b/.test(entry?.reason ?? "")}`).toBe(+ `${id}: true`,+ );+ }+ });+});++describe("the content-class census cannot pass by finding nothing", () => {+ test("the population is measured, not empty", () => {+ expect(POPULATION.length).toBeGreaterThanOrEqual(5);+ });++ test("a synthetic redacting page-carrying module without the filter is reported by name", () => {+ const intruder: CensusFile = {+ path: "src/cli/brain/verbs/synthetic-class-blind-export.ts",+ text:+ 'import { exportBundle } from "../../../core/brain/portability/okf.ts";\n' ++ 'import { redactForEgress } from "../../../core/egress/guard.ts";\n' ++ 'const flags = parse(argv, { out: { type: "string" } });\n' ++ ' const verdict = redactForEgress("synthetic-class-blind-export", bundle);\n',+ };+ // The intruder is IN population (the detector sees it), and the+ // census names it - a blinded detector or a vacuous census shows here.+ expect(isPageCarryingRedactor(intruder.text)).toBe(true);+ expect(classFilterCensus([...SOURCE_TREE, intruder])).toContain(intruder.path);+ }, 20000);++ test("removing the seam from a real verb is reported by name", () => {+ for (const path of [+ "src/cli/brain/verbs/bank-export.ts",+ "src/cli/brain/verbs/okf-export.ts",+ ]) {+ const stripped = readFileSync(join(REPO_ROOT, path), "utf8").replace(+ /\begressContentClassSplit\s*\(/g,+ "partitionByClass(",+ );+ expect(SEAM_RE.test(lexSource(stripped).code)).toBe(false);+ const tree = SOURCE_TREE.map((f) => (f.path === path ? { path, text: stripped } : f));+ expect(classFilterCensus(tree)).toContain(path);+ }+ }, 20000);++ test("removing the transcript naming from `brain export` is reported by name", () => {+ // The transcript branch's duty is refuse-or-name, never silently+ // empty; its seam is the tree tripwire, so losing it is the same+ // finding as losing a split call.+ const path = "src/cli/brain/verbs/export.ts";+ const stripped = readFileSync(join(REPO_ROOT, path), "utf8").replace(+ /\btreeCarriesPromptDerived\s*\(/g,+ "treeHasPromptMark(",+ );+ expect(SEAM_RE.test(lexSource(stripped).code)).toBe(false);+ const tree = SOURCE_TREE.map((f) => (f.path === path ? { path, text: stripped } : f));+ expect(classFilterCensus(tree)).toContain(path);+ });++ test("removing the delegated filter fails the pack verb by name", () => {+ const path = DELEGATED_FILTER_OWNER;+ const stripped = readFileSync(join(REPO_ROOT, path), "utf8").replace(+ /if \(readContentClass\(page\.frontmatter\) !== null\) return "prompt-derived";/,+ "return null;",+ );+ expect(packSelectionAppliesFilter(stripped)).toBe(false);+ const tree = SOURCE_TREE.map((f) => (f.path === path ? { path, text: stripped } : f));+ expect(classFilterCensus(tree)).toContain("src/cli/brain/verbs/knowledge-pack.ts");+ });++ test("a redacting destination without a page composer is not in population", () => {+ // The continuity export: guard call, destination flag, no composer+ // import - its class decision lives in the registry, not in a filter.+ const text = readFileSync(join(REPO_ROOT, "src/cli/brain/verbs/continuity.ts"), "utf8");+ expect(isPageCarryingRedactor(text)).toBe(false);+ });+});++describe("the class policy is disclosed where an operator reads", () => {+ // The registry reasons and the guard's stderr notices are source and+ // runtime surfaces. An operator deciding whether to hand an export to+ // someone else reads the CLI reference and the security policy, so the+ // default exclusion, the per-verb override and the era boundary must+ // appear there. Keyed on the stable spellings, the way the egress+ // census keys its network disclosures on the entry id: a disclosure+ // that stops naming the surface is a named failure here, and a new+ // export surface is undisclosed until someone writes it down.+ for (const doc of OPERATOR_DOCS) {+ test(`${doc} discloses the default, the override and the era`, () => {+ const text = readFileSync(join(REPO_ROOT, doc), "utf8");+ // The default-excluded class, named as the field and as the value.+ expect(text).toContain("content_class");+ expect(text).toContain("prompt-derived");+ // The per-verb operator override, spelled as the flag it is.+ expect(text).toContain("--include-prompt-derived");+ // The era boundary: absence of the mark is documented rather than+ // migrated, and a pre-stamp page keeps exporting.+ expect(text).toMatch(/pre-stamp era/);+ expect(text).toMatch(/never\s+rewritten|nothing\s+rewrites/);+ });+ }+});diff --git a/tests/core/architecture/verdict-vocabulary-census.test.ts b/tests/core/architecture/verdict-vocabulary-census.test.tsindex bcaa003e..7cb7cf5c 100644--- a/tests/core/architecture/verdict-vocabulary-census.test.ts+++ b/tests/core/architecture/verdict-vocabulary-census.test.ts@@ -541,6 +541,11 @@ import {BENCH_REFUSAL_KINDS,isBenchRefusalKind,} from "../../../src/core/bench/guards.ts";+import {+ CONTENT_CLASSES,+ CONTENT_CLASS,+ isContentClass,+} from "../../../src/core/brain/provenance/content-class.ts";interface VocabularyUnderCensus {/** Identifies the vocabulary in a failure message. */@@ -1701,6 +1706,21 @@ const CENSUS: ReadonlyArray<VocabularyUnderCensus> = Object.freeze([members: BENCH_REFUSAL_KINDS,guard: isBenchRefusalKind,},+ {+ // The write-honesty wave (F5, task E1, commit dca029dd). What kind of+ // process produced a page's text, stamped into `content_class`+ // frontmatter and read back by the egress filter at every shared+ // export - the value is an on-disk spelling an operator can query on+ // and hand-edit, so the guard is the boundary between this build's+ // one member and a spelling a later release or a text editor writes.+ // One member, because the stamp exists for exactly one decision+ // (shared exports leave such pages out by default); the vocabulary+ // grows the day a second decision ships.+ name: "CONTENT_CLASS",+ values: CONTENT_CLASS,+ members: CONTENT_CLASSES,+ guard: isContentClass,+ },]);// ---------------------------------------------------------------------------@@ -1976,7 +1996,11 @@ const SCANNED = scanVocabularies(SOURCE_TREE);// carry and so entering the scanned population.// 104: BENCH_REFUSAL (cycle 20261020, Lane E) - the bench guards'// closed refusal vocabulary, rendered by name in every bench refusal.-const VOCABULARY_POPULATION = 104;+// 105: CONTENT_CLASS (write-honesty wave, task E1, commit dca029dd) -+// the content-class stamp's one-member vocabulary, complete with its+// membership list and `unknown` guard from the start, enrolled here when+// the egress census that reads it landed (task E4).+const VOCABULARY_POPULATION = 105;const REGISTERED = new Map(CENSUS.map((entry) => [entry.name, entry] as const));describe("verdict vocabulary census", () => {diff --git a/tests/core/architecture/write-site-census.test.ts b/tests/core/architecture/write-site-census.test.tsindex c0edee44..d56c099c 100644--- a/tests/core/architecture/write-site-census.test.ts+++ b/tests/core/architecture/write-site-census.test.ts@@ -1506,8 +1506,18 @@ const DIRECT_WRITE_ROWS = 85;* commit `Brain/pledges/` pages through the shared atomic writer* (note-shaped pages by design; pledges.ts also carries the archive move* the direct-fs record excused, needing no exclusion for its write half).+ *+ * 128 -> 130, from the write-honesty wave and nothing else: the+ * content-class stamp's line insert+ * (`src/core/brain/provenance/content-class.ts`, commit dca029dd) rewrites+ * a just-written page through `atomicWriteFileSync` so the stamp cannot+ * lose the rest of the document, and the privacy-marker restamp verb+ * (`src/core/brain/privacy-marker-restamp.ts`, commit 571081ff) rewrites a+ * page's `visibility:` line through the same helper on its apply arm - a+ * dry-run-by-default repair pass over existing pages, writing no page of+ * its own.*/-const SHARED_HELPER_ROWS = 128;+const SHARED_HELPER_ROWS = 130;// ----- Origin-channel coverage boundary (Unit C) ----------------------------@@ -1661,8 +1671,16 @@ const UNSTAMPED_DIRECT_ROWS = 84;* `src/core/brain/pledge-intake.ts` and `src/core/brain/pledges.ts` both* commit pledge pages through the shared atomic writer (vault note pages,* and no origin stamp on the pledge lane).+ * 125 -> 127, from the write-honesty wave and nothing else: the+ * content-class stamp's line insert+ * (`src/core/brain/provenance/content-class.ts`, commit dca029dd) and the+ * privacy-marker restamp verb+ * (`src/core/brain/privacy-marker-restamp.ts`, commit 571081ff) both+ * rewrite existing pages through the shared atomic writer, and neither+ * stamps an origin channel - one writes a class the writer derives, the+ * other a visibility token the page's own informal marker declares.*/-const UNSTAMPED_SHARED_ROWS = 125;+const UNSTAMPED_SHARED_ROWS = 127;describe("in-vault write-site census", () => {test("every direct-fs write site carries a written exclusion", () => {diff --git a/tests/core/brain/applier-capability.test.ts b/tests/core/brain/applier-capability.test.tsindex 80315f4a..12c50ecd 100644--- a/tests/core/brain/applier-capability.test.ts+++ b/tests/core/brain/applier-capability.test.ts@@ -33,6 +33,7 @@ import {REPAIR_FIXER_CODES,applyRepair,} from "../../../src/core/brain/diagnostics.ts";+import { PRIVACY_MARKER_UNSTAMPED_CODE } from "../../../src/core/brain/doctor/privacy-marker-check.ts";import { LINT_CONSOLIDATE_KIND } from "../../../src/core/brain/lint-consolidate.ts";import { applyHygienePlan } from "../../../src/core/brain/hygiene/apply.ts";import {@@ -88,8 +89,9 @@ describe("applier capability table - shape", () => {if (entry.surface !== REPAIR_SURFACE.detectorOnly) continue;expect(entry.verdict).toBe(REPAIR_VERDICT.refused);}- // The four refusals reconnaissance verified as stated-and-deliberate.+ // The five refusals reconnaissance verified as stated-and-deliberate.expect(codesForSurface(REPAIR_SURFACE.detectorOnly)).toEqual([+ "chunk-holds-active","frontmatter-line-dropped","link_integrity","stale-lock",@@ -175,6 +177,18 @@ describe("applier capability table - cannot disagree with the fixers", () => {const kinds: string[] = Object.values(LINT_CONSOLIDATE_KIND);expect(kinds.toSorted()).toEqual(codesForSurface(REPAIR_SURFACE.lintConsolidate));});++ test("the privacy-marker restamp owns exactly its one code", () => {+ // The surface is an operator-run verb rather than a Finding router, so+ // the binding is by name in both directions: the table's row names the+ // restamp's planner, and the detector's code is the one it repairs.+ expect(codesForSurface(REPAIR_SURFACE.privacyMarkerRestamp)).toEqual([+ PRIVACY_MARKER_UNSTAMPED_CODE,+ ]);+ const entry = requireMechanicalRepair(PRIVACY_MARKER_UNSTAMPED_CODE);+ expect(entry.surface).toBe(REPAIR_SURFACE.privacyMarkerRestamp);+ expect(entry.fixer).toBe("privacy-marker-restamp.planPrivacyMarkerRestamp");+ });});function reportWithAction(action: HygieneProposedAction): HygieneScanReport {diff --git a/tests/core/brain/counterparty-brief.test.ts b/tests/core/brain/counterparty-brief.test.tsindex 41cda07b..8bf1f7ba 100644--- a/tests/core/brain/counterparty-brief.test.ts+++ b/tests/core/brain/counterparty-brief.test.ts@@ -351,6 +351,17 @@ describe("buildCounterpartyBrief abstentions", () => {expect(brief.entityId).toBe(publicId);});+ test("an entity page carrying only the informal access marker abstains", () => {+ const id = seedPerson("Ada Lovelace");+ // No `visibility:` key at all - the reservation is stated only under+ // the pre-convention informal key, which the brief's entity gate must+ // read through the same union the read roots use.+ addFrontmatter(entityRel(id), "access: private");+ expect(buildCounterpartyBrief(vault, { text: PROMPT, triggers: TRIGGERS, now: NOW })).toEqual({+ abstain: "no_match",+ });+ });+// A mode change cannot make a file unreadable to uid 0, nor on Windows.test.skipIf(CHMOD_CANNOT_DENY)("an entity page that cannot be read is treated as carrying the reserved token",@@ -479,6 +490,28 @@ describe("buildCounterpartyBrief items", () => {expect(rendered).not.toContain("pref-hidden");});+ test("a preference page carrying only the informal confidentiality marker never surfaces", () => {+ seedPerson("Ada Lovelace");+ const kept = writePref("kept", "Ada Lovelace reviewed the draft");+ const hidden = writePref(+ "hidden",+ "Ada Lovelace shared a confidence",+ "2026-05-02T00:00:00Z",+ "the sealed settlement terms",+ );+ // No `visibility:` key - the reservation lives only under the informal+ // key, so the preference walk must fold it through the same union the+ // read roots use or the page's text renders to the counterparty.+ addFrontmatter(hidden, "confidentiality: secret");+ const brief = expectBrief(+ buildCounterpartyBrief(vault, { text: PROMPT, triggers: TRIGGERS, now: NOW }),+ );+ expect(brief.items.map((i) => i.sourceRel)).toEqual([kept]);+ const rendered = renderCounterpartyBrief(brief);+ expect(rendered).not.toContain("the sealed settlement terms");+ expect(rendered).not.toContain("pref-hidden");+ });+test("a pledge whose cited source carries the reserved token is dropped", () => {const id = seedPerson("Ada Lovelace");const sourceRel = "notes/private-log.md";diff --git a/tests/core/brain/doctor-exit-census.test.ts b/tests/core/brain/doctor-exit-census.test.tsindex 236ecf65..0d833f0c 100644--- a/tests/core/brain/doctor-exit-census.test.ts+++ b/tests/core/brain/doctor-exit-census.test.ts@@ -62,6 +62,7 @@ import { existsSync, readdirSync, readFileSync } from "node:fs";import { dirname, join } from "node:path";import { DIAGNOSTIC_SIGNALS } from "../../../src/core/brain/diagnostics.ts";+import { PRIVACY_MARKER_UNSTAMPED_CODE } from "../../../src/core/brain/doctor/privacy-marker-check.ts";import {DOCTOR_EXIT_EXCLUSIONS,doctorExitReason,@@ -356,6 +357,15 @@ describe("the classification is readable at runtime, not only by this test", ()expect(doctorExitReason("not-a-doctor-code")).toBeNull();});+ test("the informal privacy marker finding publishes its no-command reason", () => {+ // The restamp is preview-first by contract, so the doctor names the+ // reason rather than an unconditional exit. Pinned by NAME: the code is+ // read from the check module, not retyped here.+ const reason = doctorExitReason(PRIVACY_MARKER_UNSTAMPED_CODE);+ expect(reason).toBe(DOCTOR_EXIT_EXCLUSIONS.get(PRIVACY_MARKER_UNSTAMPED_CODE)!);+ expect(reason!.trim().length).toBeGreaterThanOrEqual(MIN_REASON_LENGTH);+ });+test("reasons are collected once per code, in the order the codes were reported", () => {const reasons = noExitReasons(["config-missing",diff --git a/tests/core/brain/doctor.entities.test.ts b/tests/core/brain/doctor.entities.test.tsindex e15e938c..65530438 100644--- a/tests/core/brain/doctor.entities.test.ts+++ b/tests/core/brain/doctor.entities.test.ts@@ -7,6 +7,7 @@ import { bootstrapBrain } from "../../../src/core/brain/init.ts";import { runDoctor } from "../../../src/core/brain/doctor.ts";import { atomicWriteFileSync } from "../../../src/core/fs-atomic.ts";import { relateEntities, upsertEntity } from "../../../src/core/brain/entities/registry.ts";+import { PERSON_NICKNAMES_ARTIFACT_REL } from "../../../src/core/brain/entities/person-nicknames.ts";import { brainDirs } from "../../../src/core/brain/paths.ts";let vault: string;@@ -31,6 +32,22 @@ function entityFile(name: string, lines: string[]): void {atomicWriteFileSync(join(brainDirs(vault).entities, "people", name), lines.join("\n") + "\n");}+/** Write the operator nickname map the collision scan resolves. */+function writeNicknames(entries: Record<string, string[]>): void {+ atomicWriteFileSync(join(vault, PERSON_NICKNAMES_ARTIFACT_REL), JSON.stringify(entries));+}++function person(name: string, opts: { aliases?: string[]; untrusted?: boolean } = {}): string {+ return upsertEntity(vault, {+ category: "people",+ name,+ ...(opts.aliases !== undefined ? { aliases: opts.aliases } : {}),+ agent: "a",+ now: NOW,+ ...(opts.untrusted === true ? { untrustedOrigin: true } : {}),+ }).entity.id;+}+describe("doctor entity lints", () => {test("clean registry produces no entity issues", () => {upsertEntity(vault, { category: "people", name: "Ada", agent: "a", now: NOW });@@ -126,3 +143,116 @@ describe("doctor entity lints", () => {expect(broken[0]!.message).toContain("ent-systems-ghost");});});++describe("doctor derived-variant collisions (write-honesty-egress-consent F3/C5)", () => {+ function aliasCandidateWarnings() {+ return runDoctor(vault).warnings.filter((i) => i.code === "entity-alias-candidate");+ }++ test("a nickname derived from one person colliding with another person's name surfaces one warning", () => {+ const lovelace = person("Ada Lovelace");+ const ada = person("Ada");+ writeNicknames({ ada: ["Ada Lovelace"] });++ const warnings = aliasCandidateWarnings();+ expect(warnings).toHaveLength(1);+ expect(warnings[0]!.severity).toBe("warning");+ // The existing alias-merge candidate shape, naming both ids.+ expect(warnings[0]!.message).toContain("possible alias-merge:");+ expect(warnings[0]!.message).toContain(lovelace);+ expect(warnings[0]!.message).toContain(ada);+ expect(warnings[0]!.message).toContain("[person-variant 1]");+ expect(warnings[0]!.message).toContain("never auto-merged");+ });++ test("a collision through a declared alias counts the same", () => {+ const lovelace = person("Ada Lovelace");+ const prime = person("Adeline Prime", { aliases: ["Ada"] });+ writeNicknames({ ada: ["Ada Lovelace"] });++ const warnings = aliasCandidateWarnings();+ expect(warnings).toHaveLength(1);+ expect(warnings[0]!.message).toContain(lovelace);+ expect(warnings[0]!.message).toContain(prime);+ });++ test("several colliding variants of one pair still produce exactly one warning", () => {+ // Mutual: each record's declared name is the other's reversed+ // word-order variant, so both directions collide - the pair is+ // still one finding, not two.+ person("Grace Hopper");+ person("Hopper Grace");++ expect(aliasCandidateWarnings()).toHaveLength(1);+ });++ test("a variant shared as a variant is not a collision - only declared forms claim", () => {+ person("Robert Smith");+ person("Robert Jones");+ writeNicknames({ bob: ["Robert Smith", "Robert Jones"] });++ expect(aliasCandidateWarnings()).toEqual([]);+ });++ test("non-person records never collide on person variants", () => {+ upsertEntity(vault, { category: "org", name: "Ada Lovelace Ltd", agent: "a", now: NOW });+ person("Ada Lovelace");+ writeNicknames({ "ada lovelace ltd": ["Ada Lovelace"] });++ expect(aliasCandidateWarnings()).toEqual([]);+ });++ test("quarantined records are outside the collision scan", () => {+ person("Ada Lovelace");+ person("Ada", { untrusted: true });+ writeNicknames({ ada: ["Ada Lovelace"] });++ expect(aliasCandidateWarnings()).toEqual([]);+ });++ test("a malformed nickname file degrades the scan without losing the rest of the check", () => {+ person("Ada Lovelace");+ person("Ada");+ atomicWriteFileSync(join(vault, PERSON_NICKNAMES_ARTIFACT_REL), "{ not json");++ const out = runDoctor(vault);+ expect(out.warnings.filter((i) => i.code === "entity-alias-candidate")).toEqual([]);+ // The other entity lints of the same check still ran: the registry+ // is clean apart from the two distinct persons, so nothing else+ // fires - the point is that the pass did not swallow the check.+ expect(out.warnings.filter((i) => i.code === "duplicate-entity")).toEqual([]);+ });++ test("a malformed nickname file surfaces exactly one warning naming the path", () => {+ // The read lanes degrade to their documented postures, but the+ // doctor is the surfacing point: a present-yet-unusable operator+ // file must be visible, by name.+ person("Ada Lovelace");+ person("Ada");+ atomicWriteFileSync(join(vault, PERSON_NICKNAMES_ARTIFACT_REL), '{ "quoted-bytes" broken');++ const warnings = runDoctor(vault).warnings.filter(+ (i) => i.code === "person-nicknames-unreadable",+ );+ expect(warnings).toHaveLength(1);+ expect(warnings[0]!.severity).toBe("warning");+ expect(warnings[0]!.path).toContain("person-nicknames.json");+ expect(warnings[0]!.message).toContain("person-nicknames.json");+ // The detail is short and never quotes the file bytes the parser+ // choked on: the message must not become a byte-echo channel.+ expect(warnings[0]!.message).not.toContain("quoted-bytes");+ expect(warnings[0]!.message).not.toContain("Unexpected");+ });++ test("a well-formed nickname file raises no unreadable warning", () => {+ person("Ada Lovelace");+ person("Ada");+ atomicWriteFileSync(+ join(vault, PERSON_NICKNAMES_ARTIFACT_REL),+ JSON.stringify({ ada: ["Ada Lovelace"] }),+ );+ expect(+ runDoctor(vault).warnings.filter((i) => i.code === "person-nicknames-unreadable"),+ ).toEqual([]);+ });+});diff --git a/tests/core/brain/doctor/chunk-holds-check.test.ts b/tests/core/brain/doctor/chunk-holds-check.test.tsnew file mode 100644index 00000000..c0b0e8ca--- /dev/null+++ b/tests/core/brain/doctor/chunk-holds-check.test.ts@@ -0,0 +1,217 @@+/**+ * The held-chunks doctor check (chunk skip-and-hold, F2, task B4).+ *+ * A held chunk is invisible to every surface that counts pending work:+ * the pending census, the refresh estimate and the backfill all stop+ * seeing it BY DESIGN - the index does not owe it a vector. That is+ * correct for spend and wrong for awareness: nothing told the operator+ * which SOURCE DOCUMENTS carry chunks the active model cannot take.+ * This check speaks from the doctor pass, ONE aggregated warning per+ * affected document, naming the count, the largest estimate and the+ * window the proof ran against.+ *+ * The index facts are an injected probe, following+ * `embeddings-health-check`: every verdict is reachable from a test+ * without building an index. The read-failure states stay this check's+ * silence for a stated reason rather than a second uncertain stream -+ * the sibling check already reports "the index could not be read" for+ * the same file, and two findings for one condition is what its docblock+ * refuses.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { DIAGNOSTIC_SIGNALS } from "../../../../src/core/brain/diagnostics.ts";+import {+ APPLIER_CAPABILITY,+ REPAIR_VERDICT,+} from "../../../../src/core/brain/applier-capability.ts";+import type { DoctorCheckContext } from "../../../../src/core/brain/doctor/check.ts";+import {+ CHUNK_HOLDS_ACTIVE_CODE,+ chunkHoldsCheck,+ makeChunkHoldsCheck,+ type ChunkHoldsProbe,+ type HeldChunkDocRow,+} from "../../../../src/core/brain/doctor/chunk-holds-check.ts";+import { DOCTOR_EXIT_EXCLUSIONS } from "../../../../src/core/brain/doctor-exits.ts";+import { bootstrapBrain } from "../../../../src/core/brain/init.ts";+import { runDoctor } from "../../../../src/core/brain/doctor.ts";+import type { DoctorIssue } from "../../../../src/core/brain/types.ts";+import type { ChunkHoldExclusion } from "../../../../src/core/search/store/chunks.ts";+import { RECOMMENDED_EMBEDDING_MODEL } from "../../../../src/core/search/embeddings/presets.ts";++const NOW = new Date("2026-06-01T00:00:00.000Z");++let vault: string;+let configHome: string;+let configPath: string;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-chunk-holds-vault-"));+ configHome = mkdtempSync(join(tmpdir(), "o2b-chunk-holds-cfg-"));+ configPath = join(configHome, "config.yaml");+ bootstrapBrain(vault);+ configure(RECOMMENDED_EMBEDDING_MODEL, "8");+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+ rmSync(configHome, { recursive: true, force: true });+});++function configure(model: string, dimension: string): void {+ writeFileSync(+ configPath,+ [+ `vault: ${vault}`,+ `search_semantic_enabled: "true"`,+ `embedding_provider: "openai-compat"`,+ `embedding_base_url: "https://example.invalid/v1"`,+ `embedding_model: "${model}"`,+ `embedding_dimension: "${dimension}"`,+ ].join("\n") + "\n",+ );+}++function docRow(overrides: Partial<HeldChunkDocRow> = {}): HeldChunkDocRow {+ return { path: "Brain/big-note.md", heldChunks: 2, maxEstimatedTokens: 600, ...overrides };+}++function run(probe: ChunkHoldsProbe, extra: Partial<DoctorCheckContext> = {}): DoctorIssue[] {+ const issues: DoctorIssue[] = [];+ const uncertain: never[] = [];+ makeChunkHoldsCheck(probe).run(+ { vault, now: NOW, configPath, ...extra } as unknown as DoctorCheckContext,+ { issues, uncertain },+ );+ return issues;+}++function active(issues: DoctorIssue[]): DoctorIssue[] {+ return issues.filter((i) => i.code === CHUNK_HOLDS_ACTIVE_CODE);+}++describe("held documents become one warning each", () => {+ test("the warning names the count, the largest estimate and the window", () => {+ const issues = run(() => [docRow()]);+ expect(active(issues).length).toBe(1);+ const issue = active(issues)[0]!;+ expect(issue.severity).toBe("warning");+ expect(issue.path).toBe("Brain/big-note.md");+ expect(issue.message).toContain("2");+ expect(issue.message).toContain("600");+ expect(issue.message).toContain("512");+ expect(issue.message).toContain(RECOMMENDED_EMBEDDING_MODEL);+ });++ test("the probe is handed the active (model, window, signature) key", () => {+ const seen: ChunkHoldExclusion[] = [];+ run((_dbPath, hold) => {+ seen.push(hold);+ return [];+ });+ expect(seen.length).toBe(1);+ const hold = seen[0]!;+ expect(hold.model).toBe(RECOMMENDED_EMBEDDING_MODEL);+ expect(hold.windowTokens).toBe(512);+ expect(hold.embeddingSignature).toContain(RECOMMENDED_EMBEDDING_MODEL);+ expect(hold.embeddingSignature).toContain(":8");+ });++ test("affected documents warn separately", () => {+ const issues = run(() => [+ docRow({ path: "Brain/a.md" }),+ docRow({ path: "Brain/b.md", heldChunks: 1, maxEstimatedTokens: 900 }),+ ]);+ expect(active(issues).map((i) => i.path)).toEqual(["Brain/a.md", "Brain/b.md"]);+ });++ test("no held document, no finding", () => {+ expect(run(() => [])).toEqual([]);+ });+});++describe("states that are not this check's to report stay silent", () => {+ test("semantic search off: not configured, not probed", () => {+ writeFileSync(configPath, `vault: ${vault}\nsearch_semantic_enabled: "false"\n`);+ let probed = 0;+ const issues = run(() => {+ probed++;+ return [docRow()];+ });+ expect(probed).toBe(0);+ expect(issues).toEqual([]);+ });++ test("a model with no declared window can hold nothing: not probed", () => {+ configure("vendor-x/embed-v1", "8");+ let probed = 0;+ const issues = run(() => {+ probed++;+ return [docRow()];+ });+ expect(probed).toBe(0);+ expect(issues).toEqual([]);+ });++ test("an index that could not be read is the sibling check's finding, not a second one here", () => {+ // The embeddings-health check reports this file as census-unrecorded+ // or unmeasured; a null from this check's probe adds nothing to that.+ expect(run(() => null)).toEqual([]);+ });++ test("the ctx.dbPath the pass names is the index probed", () => {+ const seen: string[] = [];+ run(+ (dbPath) => {+ seen.push(dbPath);+ return [];+ },+ { dbPath: "/elsewhere/index.db" },+ );+ expect(seen).toEqual(["/elsewhere/index.db"]);+ });+});++describe("a caller who may not read a page does not learn it is held", () => {+ test("ctx.readable spends no warning slot and moves no count", () => {+ const probe: ChunkHoldsProbe = () => [+ docRow({ path: "Brain/a.md" }),+ docRow({ path: "Brain/b.md" }),+ ];+ const issues = run(probe, { readable: (rel: string) => rel !== "Brain/b.md" });+ expect(active(issues).map((i) => i.path)).toEqual(["Brain/a.md"]);+ });+});++describe("the registered instance and its exits", () => {+ test("the registered check is failSoft", () => {+ expect(chunkHoldsCheck.failSoft).toBe(true);+ });++ test("a semantic vault whose index is absent produces no held finding from runDoctor", () => {+ const result = runDoctor(vault, { now: NOW, configPath });+ expect(result.warnings.filter((i) => i.code === CHUNK_HOLDS_ACTIVE_CODE)).toEqual([]);+ });++ test("the warning code carries a written reason, not a command", () => {+ expect(DOCTOR_EXIT_EXCLUSIONS.has(CHUNK_HOLDS_ACTIVE_CODE)).toBe(true);+ expect(DIAGNOSTIC_SIGNALS.has(CHUNK_HOLDS_ACTIVE_CODE)).toBe(false);+ expect(+ DOCTOR_EXIT_EXCLUSIONS.get(CHUNK_HOLDS_ACTIVE_CODE)!.trim().length,+ ).toBeGreaterThanOrEqual(80);+ });++ test("the repair question is answered as a stated refusal", () => {+ const entry = APPLIER_CAPABILITY.get(CHUNK_HOLDS_ACTIVE_CODE);+ expect(entry?.verdict).toBe(REPAIR_VERDICT.refused);+ if (entry === undefined || entry.verdict !== REPAIR_VERDICT.refused) {+ throw new Error("chunk-holds-active must be a stated refusal");+ }+ expect(entry.reason.trim().length).toBeGreaterThan(0);+ });+});diff --git a/tests/core/brain/entities/entity-read-census.test.ts b/tests/core/brain/entities/entity-read-census.test.tsindex 320749ed..f53259f1 100644--- a/tests/core/brain/entities/entity-read-census.test.ts+++ b/tests/core/brain/entities/entity-read-census.test.ts@@ -120,11 +120,6 @@ const SCOPED_RE = /entityStatusInScope|ENTITY_STATUS_SCOPE|vaultPageInStatusScop* would be wrong, never because scoping it was inconvenient.*/const UNSCOPED_WITH_REASON: Readonly<Record<string, string>> = Object.freeze({- "src/core/brain/doctor/entity-checks.ts":- "registry-integrity surface. It reports duplicate identities and broken " +- "relation edges over what is STORED; scoping it would stop the doctor " +- "reporting on exactly the records most likely to be malformed - the ones " +- "a read path is already hiding.","src/core/brain/entities/label-hygiene.ts":"runs the label validator over every stored node (its own docblock says " +"'any status'), and the prune pass must see every survivor or it would " +diff --git a/tests/core/brain/entities/person-nicknames.test.ts b/tests/core/brain/entities/person-nicknames.test.tsnew file mode 100644index 00000000..668bbd8b--- /dev/null+++ b/tests/core/brain/entities/person-nicknames.test.ts@@ -0,0 +1,120 @@+/**+ * The operator nickname reader (write-honesty-egress-consent, F3).+ *+ * Characterization tests for the reader after its move from+ * `src/core/search/entity-alias.ts` to this brain-side leaf: the+ * registry's derived lookup lane (C4) and the doctor's collision scan+ * (C5) consume it from the brain side, and the brain must not import+ * from `src/core/search/`. The reader's behavior is unchanged - the search+ * suite keeps exercising the same class through its re-export.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import {+ PersonNicknameConfigError,+ PERSON_NICKNAMES_ARTIFACT_REL,+ resolvePersonNicknameMap,+} from "../../../../src/core/brain/entities/person-nicknames.ts";++let vault: string;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-nicknames-"));+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+});++/** Write a nickname artifact into the vault, creating its directories. */+function writeArtifact(text: string): void {+ mkdirSync(join(vault, "Brain", "entities"), { recursive: true });+ writeFileSync(join(vault, PERSON_NICKNAMES_ARTIFACT_REL), text);+}++describe("resolvePersonNicknameMap", () => {+ test("a missing file is the empty map, not an error", () => {+ const map = resolvePersonNicknameMap(vault);+ expect(map.size).toBe(0);+ });++ test("the artifact path is the Brain/entities neighbourhood of the vault", () => {+ expect(PERSON_NICKNAMES_ARTIFACT_REL).toContain("person-nicknames.json");+ });++ test("a present file parses to the normalized nickname map", () => {+ writeArtifact(+ JSON.stringify({ " BOB ": ["Robert Smith", "Robert Smith", ""], "": ["ignored"] }),+ );+ const map = resolvePersonNicknameMap(vault);+ expect([...map.keys()]).toEqual(["bob"]);+ expect([...map.get("bob")!]).toEqual(["robert smith"]);+ });++ test("a malformed file surfaces by name", () => {+ writeArtifact("{ not json");+ let err: unknown;+ try {+ resolvePersonNicknameMap(vault);+ } catch (e) {+ err = e;+ }+ expect(err).toBeInstanceOf(PersonNicknameConfigError);+ expect((err as PersonNicknameConfigError).name).toBe("PersonNicknameConfigError");+ expect((err as PersonNicknameConfigError).path).toContain("person-nicknames.json");+ });++ test("the parse-failure detail is position-only, never the parser's message", () => {+ // The parser's own message can quote the bytes it choked on, and the+ // detail reaches doctor output and logs - so only a position (or the+ // fixed sentence) may travel.+ writeArtifact('{ "quoted-bytes" from the file }');+ let err: unknown;+ try {+ resolvePersonNicknameMap(vault);+ } catch (e) {+ err = e;+ }+ expect(err).toBeInstanceOf(PersonNicknameConfigError);+ const detail = (err as PersonNicknameConfigError).detail;+ expect(detail).not.toContain("quoted-bytes");+ expect(detail).not.toContain("Unexpected");+ expect(detail).toMatch(+ /^not parseable JSON \((parse failed at (line \d+|offset \d+)|the parser reported a syntax error)\)$/,+ );+ });++ test("a non-array value surfaces by name", () => {+ writeArtifact(JSON.stringify({ bob: "Robert Smith" }));+ expect(() => resolvePersonNicknameMap(vault)).toThrow(PersonNicknameConfigError);+ });++ test("a shape-error detail is position-only: the operator key never travels", () => {+ // The key of a nickname entry is a person's name - the same kind of+ // data the parse-error path keeps out of doctor output and logs - so+ // the shape detail names the entry's POSITION, not the key's text.+ writeArtifact(JSON.stringify({ "Bobby Tables": "Robert" }));+ let err: unknown;+ try {+ resolvePersonNicknameMap(vault);+ } catch (e) {+ err = e;+ }+ expect(err).toBeInstanceOf(PersonNicknameConfigError);+ const detail = (err as PersonNicknameConfigError).detail;+ expect(detail).not.toContain("Bobby Tables");+ expect(detail).toMatch(/^value of entry #1 must be an array of names$/);+ });++ test("two raw keys that normalize alike merge instead of the last one winning", () => {+ // "Bob" and "bob " normalize to the same key; a silent replace makes+ // the expansion depend on the JSON key order. The value lists merge.+ writeArtifact(JSON.stringify({ Bob: ["Robert Smith"], "bob ": ["Bobby"] }));+ const map = resolvePersonNicknameMap(vault);+ expect(map.get("bob")).toEqual(["robert smith", "bobby"]);+ });+});diff --git a/tests/core/brain/entities/person-variants.test.ts b/tests/core/brain/entities/person-variants.test.tsnew file mode 100644index 00000000..e9a19ead--- /dev/null+++ b/tests/core/brain/entities/person-variants.test.ts@@ -0,0 +1,142 @@+/**+ * Person-name variant derivation (write-honesty-egress-consent, F3 / C1).+ *+ * The module is a leaf: it imports nothing, so these tests exercise the+ * pure kernel only - determinism, the form-length floor, script+ * neutrality, and the rule that nickname expansion comes exclusively+ * from the operator-supplied map argument.+ */++import { describe, expect, test } from "bun:test";++import {+ derivePersonNameVariants,+ isPersonCategory,+ MIN_PERSON_VARIANT_LENGTH,+ PERSON_ENTITY_CATEGORIES,+ type PersonNicknameMap,+} from "../../../../src/core/brain/entities/person-variants.ts";++function mapOf(entries: Record<string, string[]>): PersonNicknameMap {+ return Object.freeze(+ new Map(Object.entries(entries).map(([k, v]) => [k, Object.freeze([...v])])),+ );+}++describe("derivePersonNameVariants", () => {+ test("same input yields byte-identical frozen output", () => {+ const a = derivePersonNameVariants("ada lovelace");+ const b = derivePersonNameVariants("ada lovelace");+ expect(Object.isFrozen(a)).toBe(true);+ expect(JSON.stringify(a)).toBe(JSON.stringify(b));+ expect([...a]).toEqual([...b]);+ });++ test("two-token name derives first-initial-plus-surname and reversed order, in that order", () => {+ const out = derivePersonNameVariants("ada lovelace");+ expect([...out]).toEqual(["a lovelace", "lovelace ada"]);+ });++ test("three-token name keeps the middle tokens in both structural variants", () => {+ const out = derivePersonNameVariants("ada mary lovelace");+ expect([...out]).toEqual(["a mary lovelace", "lovelace mary ada"]);+ });++ test("a variant equal to the input name is never emitted", () => {+ const out = derivePersonNameVariants("ma li");+ expect([...out]).not.toContain("ma li");+ // A two-token name whose reversal equals itself stays single-form.+ const same = derivePersonNameVariants("li li");+ expect([...same]).toEqual(["l li"]);+ });++ test("a single-token name derives no structural variants", () => {+ expect([...derivePersonNameVariants("cher")]).toEqual([]);+ expect([...derivePersonNameVariants("smith")]).toEqual([]);+ });++ test("nickname expansion comes only from the operator map argument", () => {+ // No map: no nickname data exists, so nothing beyond structural forms.+ const bare = derivePersonNameVariants("robert smith");+ expect([...bare]).toEqual(["r smith", "smith robert"]);+ expect([...bare]).not.toContain("bob");+ // With the operator map (normalized keys and values, the shape the+ // config reader hands over): the nickname joins, sorted after the+ // structural forms.+ const mapped = derivePersonNameVariants("robert smith", mapOf({ bob: ["robert smith"] }));+ expect([...mapped]).toEqual(["r smith", "smith robert", "bob"]);+ });++ test("nickname matching is exact over the full normalized name", () => {+ const map = mapOf({ bob: ["robert jones", "roberta stone"] });+ expect([...derivePersonNameVariants("robert smith", map)]).toEqual(["r smith", "smith robert"]);+ expect([...derivePersonNameVariants("roberta stone", map)]).toEqual([+ "r stone",+ "stone roberta",+ "bob",+ ]);+ });++ test("multiple matching nicknames come out sorted", () => {+ const map = mapOf({+ roberto: ["robert smith"],+ bob: ["robert smith"],+ bobby: ["robert smith"],+ });+ const out = derivePersonNameVariants("robert smith", map);+ expect([...out]).toEqual(["r smith", "smith robert", "bob", "bobby", "roberto"]);+ });++ test(`forms shorter than MIN_PERSON_VARIANT_LENGTH (${MIN_PERSON_VARIANT_LENGTH}) are dropped`, () => {+ expect(MIN_PERSON_VARIANT_LENGTH).toBe(3);+ // A two-letter nickname is below the floor and never emitted.+ const map = mapOf({ ed: ["ma li"] });+ expect([...derivePersonNameVariants("ma li", map)]).toEqual(["m li", "li ma"]);+ });++ test("the floor is measured on the whole emitted form, not per token", () => {+ // "a li" is 4 characters and clears the floor even though the+ // initial itself is one character.+ const out = derivePersonNameVariants("ada li");+ expect([...out]).toEqual(["a li", "li ada"]);+ });++ test("unspaced-script names emit no word-order or initial variants", () => {+ expect([...derivePersonNameVariants("ç”°ä¸å¤ªéƒŽ")]).toEqual([]);+ // Even when the operator spells the family and given names with a+ // space, a CJK name gets no structural variants: the reversed and+ // initial forms are spaced-script conventions.+ expect([...derivePersonNameVariants("ç”°ä¸ å¤ªéƒŽ")]).toEqual([]);+ // A mixed-script name carries an unspaced-script token, so its+ // structural variants stay silent too.+ expect([...derivePersonNameVariants("ada ç”°ä¸")]).toEqual([]);+ });++ test("operator nicknames still apply to unspaced-script names", () => {+ const map = mapOf({ taro: ["ç”°ä¸ å¤ªéƒŽ"] });+ expect([...derivePersonNameVariants("ç”°ä¸ å¤ªéƒŽ", map)]).toEqual(["taro"]);+ });++ test("an empty or whitespace name derives nothing", () => {+ expect([...derivePersonNameVariants("")]).toEqual([]);+ expect([...derivePersonNameVariants(" ")]).toEqual([]);+ });+});++describe("isPersonCategory", () => {+ test("admits exactly the person-category folders", () => {+ expect(PERSON_ENTITY_CATEGORIES).toEqual(["people", "person"]);+ expect(isPersonCategory("people")).toBe(true);+ expect(isPersonCategory("person")).toBe(true);+ });++ test("rejects every non-person category", () => {+ expect(isPersonCategory("projects")).toBe(false);+ expect(isPersonCategory("org")).toBe(false);+ expect(isPersonCategory("organization")).toBe(false);+ expect(isPersonCategory("organisation")).toBe(false);+ expect(isPersonCategory("company")).toBe(false);+ expect(isPersonCategory("team")).toBe(false);+ expect(isPersonCategory("")).toBe(false);+ });+});diff --git a/tests/core/brain/entities/registry-derived-lookup.test.ts b/tests/core/brain/entities/registry-derived-lookup.test.tsnew file mode 100644index 00000000..0bcb87a9--- /dev/null+++ b/tests/core/brain/entities/registry-derived-lookup.test.ts@@ -0,0 +1,211 @@+/**+ * The registry's derived lookup lane (write-honesty-egress-consent, F3/C4).+ *+ * `getEntity` consults the derived person variants (the pure kernel in+ * `person-variants.ts`, fed by the operator nickname file) ONLY after an+ * exact name/alias miss. A unique derived match resolves; more than one+ * is an `EntityAmbiguityError` naming both ids. The WRITE lanes keep+ * their exact semantics: `resolveWriteTarget` never follows a variant,+ * so a trusted write cannot be captured by a quarantined namesake's+ * derived form and no write lands on a record through a form it never+ * declared.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { bootstrapBrain } from "../../../../src/core/brain/init.ts";+import { atomicWriteFileSync } from "../../../../src/core/fs-atomic.ts";+import {+ EntityAmbiguityError,+ archiveEntity,+ getEntity,+ listEntities,+ upsertEntity,+} from "../../../../src/core/brain/entities/registry.ts";+import { PERSON_NICKNAMES_ARTIFACT_REL } from "../../../../src/core/brain/entities/person-nicknames.ts";+import { BRAIN_ENTITY_STATUS } from "../../../../src/core/brain/entities/types.ts";++let vault: string;+let configHome: string;++const NOW = new Date("2026-06-13T12:00:00Z");++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-derived-lookup-"));+ configHome = mkdtempSync(join(tmpdir(), "o2b-derived-lookup-cfg-"));+ const configPath = join(configHome, "config.yaml");+ atomicWriteFileSync(configPath, `vault: ${vault}\n`);+ bootstrapBrain(vault, { configPath });+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+ rmSync(configHome, { recursive: true, force: true });+});++function upsertPerson(+ name: string,+ opts: { aliases?: string[]; untrusted?: boolean } = {},+): string {+ return upsertEntity(vault, {+ category: "people",+ name,+ ...(opts.aliases !== undefined ? { aliases: opts.aliases } : {}),+ agent: opts.untrusted === true ? "scraper" : "operator",+ now: NOW,+ ...(opts.untrusted === true ? { untrustedOrigin: true } : {}),+ }).entity.id;+}++function writeNicknames(entries: Record<string, string[]>): void {+ writeFileSync(join(vault, PERSON_NICKNAMES_ARTIFACT_REL), JSON.stringify(entries));+}++describe("getEntity resolves a unique derived match after the exact miss", () => {+ test("an operator nickname resolves the person it stands for", () => {+ const id = upsertPerson("Robert Smith");+ writeNicknames({ bob: ["Robert Smith"] });++ const found = getEntity(vault, { query: "Bob" });+ expect(found).not.toBeNull();+ expect(found!.id).toBe(id);+ expect(found!.status).toBe(BRAIN_ENTITY_STATUS.active);+ });++ test("the structural variants resolve too, scoped and unscoped", () => {+ const id = upsertPerson("Robert Smith");+ // No nickname data at all: the structural kernel alone must answer.+ expect(getEntity(vault, { query: "smith robert" })!.id).toBe(id);+ expect(getEntity(vault, { category: "people", query: "r smith" })!.id).toBe(id);+ });++ test("an exact declared form still wins before the derived lane is consulted", () => {+ const id = upsertPerson("Robert Smith", { aliases: ["Rob"] });+ writeNicknames({ bob: ["Robert Smith"], rob: ["Robert Smith"] });++ expect(getEntity(vault, { query: "robert smith" })!.id).toBe(id);+ expect(getEntity(vault, { query: "Rob" })!.id).toBe(id);+ });+});++describe("a shared derived form is an ambiguity, not a silent pick", () => {+ test("two persons sharing a nickname throw with both ids named", () => {+ const smith = upsertPerson("Robert Smith");+ const jones = upsertPerson("Robert Jones");+ writeNicknames({ bob: ["Robert Smith", "Robert Jones"] });++ let err: unknown;+ try {+ getEntity(vault, { query: "bob" });+ } catch (e) {+ err = e;+ }+ expect(err).toBeInstanceOf(EntityAmbiguityError);+ const amb = err as EntityAmbiguityError;+ expect(amb.name).toBe("EntityAmbiguityError");+ expect(amb.message).toContain(smith);+ expect(amb.message).toContain(jones);+ });++ test("the exact forms of the same two persons still resolve uniquely", () => {+ const smith = upsertPerson("Robert Smith");+ const jones = upsertPerson("Robert Jones");+ writeNicknames({ bob: ["Robert Smith", "Robert Jones"] });++ expect(getEntity(vault, { query: "robert smith" })!.id).toBe(smith);+ expect(getEntity(vault, { query: "robert jones" })!.id).toBe(jones);+ });+});++describe("the derived lane resolves canonical records only", () => {+ test("a quarantined person's variant never resolves a read", () => {+ upsertPerson("Robert Smith", { untrusted: true });+ writeNicknames({ bob: ["Robert Smith"] });++ expect(getEntity(vault, { query: "bob" })).toBeNull();+ });++ test("an archived person derives nothing", () => {+ upsertPerson("Robert Smith");+ archiveEntity(vault, { query: "Robert Smith" }, { now: NOW });+ writeNicknames({ bob: ["Robert Smith"] });++ expect(getEntity(vault, { query: "bob" })).toBeNull();+ expect(getEntity(vault, { category: "people", query: "Robert Smith" })).toBeNull();+ });++ test("a non-person category never resolves through a derived form", () => {+ upsertEntity(vault, {+ category: "org",+ name: "Ada Lovelace",+ agent: "operator",+ now: NOW,+ });+ writeNicknames({ ada: ["Ada Lovelace"] });++ expect(getEntity(vault, { query: "lovelace ada" })).toBeNull();+ expect(getEntity(vault, { query: "ada" })).toBeNull();+ });+});++describe("write lanes keep exact semantics", () => {+ test("a TRUSTED write is never captured by a quarantined namesake's variant", () => {+ upsertPerson("Robert Smith", { untrusted: true });+ writeNicknames({ bob: ["robert smith"] });+ const quarantinedBefore = readFileSync(+ listEntities(vault, { status: BRAIN_ENTITY_STATUS.quarantine })[0]!.path,+ "utf8",+ );++ const res = upsertEntity(vault, {+ category: "people",+ name: "Bob",+ agent: "operator",+ now: NOW,+ });++ expect(res.created).toBe(true);+ expect(res.entity.status).toBe(BRAIN_ENTITY_STATUS.active);+ const found = getEntity(vault, { query: "Bob" });+ expect(found!.id).toBe(res.entity.id);+ // The quarantined record is neither promoted nor rewritten.+ const quarantined = listEntities(vault, { status: BRAIN_ENTITY_STATUS.quarantine });+ expect(quarantined).toHaveLength(1);+ expect(quarantined[0]!.source_agent).toBe("scraper");+ expect(readFileSync(quarantined[0]!.path, "utf8")).toBe(quarantinedBefore);+ });++ test("a trusted write does not land inside a canonical person through a derived form", () => {+ const smith = upsertPerson("Robert Smith");+ writeNicknames({ bob: ["Robert Smith"] });++ const res = upsertEntity(vault, {+ category: "people",+ name: "Bob",+ agent: "operator",+ now: NOW,+ });++ expect(res.created).toBe(true);+ expect(res.entity.id).not.toBe(smith);+ expect(listEntities(vault, { category: "people" })).toHaveLength(2);+ // The derived lane stays a read aid: once a record DECLARES the+ // spelling, the exact lane answers for it, and the person the+ // nickname was derived from keeps answering to its declared name.+ expect(getEntity(vault, { query: "bob" })!.id).toBe(res.entity.id);+ expect(getEntity(vault, { query: "robert smith" })!.id).toBe(smith);+ });+});++describe("a malformed nickname file degrades reads to exact-only", () => {+ test("exact resolution holds and the derived lane stays silent without throwing", () => {+ const id = upsertPerson("Robert Smith");+ writeFileSync(join(vault, PERSON_NICKNAMES_ARTIFACT_REL), "{ not json");++ expect(getEntity(vault, { query: "robert smith" })!.id).toBe(id);+ expect(getEntity(vault, { query: "bob" })).toBeNull();+ });+});diff --git a/tests/core/brain/page-lint.test.ts b/tests/core/brain/page-lint.test.tsindex a5422509..b0b4a629 100644--- a/tests/core/brain/page-lint.test.ts+++ b/tests/core/brain/page-lint.test.ts@@ -23,6 +23,7 @@ import {PAGE_LINT_MAX_FINDINGS,PAGE_LINT_SKIP_REASON,PAGE_LINT_UNAVAILABLE_CODE,+ UNDATED_QUANTITY_CODE,comparePageLintFindings,lintPagesWithContext,lintWrittenPages,@@ -147,6 +148,103 @@ describe("lintWrittenPages - findings", () => {});});+/**+ * The fifth detector (write-honesty, F4, t_fb442a24). A count or amount+ * with no structural date is an advisory warning on the `body` path: the+ * page states a quantity a reader cannot place in time. The escape hatches+ * are structural, never natural language - `valid_from` in frontmatter, any+ * ISO date in the body, or a `[Source: <name>, YYYY-MM-DD]` marker.+ */+describe("lintWrittenPages - undated-quantity findings", () => {+ const BODY = "We onboarded 5,000 users.";+ const QUANTITY = "5,000 users";++ function undatedFindings(report: ReturnType<typeof lintWrittenPages>) {+ return report.findings.filter((f) => f.code === UNDATED_QUANTITY_CODE);+ }++ test("an undated numeric note is exactly one warning on the body path", () => {+ const rel = writeNote("Notes/Undated.md", `---\ntitle: U\n---\n\n${BODY}\n`);+ const report = lintWrittenPages(vault, [rel], READ_ALL);+ const findings = undatedFindings(report);+ expect(findings).toHaveLength(1);+ expect(findings[0]).toMatchObject({+ severity: "warning",+ code: UNDATED_QUANTITY_CODE,+ page: rel,+ path: "body",+ });+ // The finding quotes the lexeme and its offset, never the raw line.+ expect(findings[0]!.message).toContain(JSON.stringify(QUANTITY));+ expect(findings[0]!.message).toContain(`offset ${BODY.indexOf(QUANTITY)}`);+ expect(findings[0]!.message).not.toContain("onboarded");+ expect(report.total).toBe(1);+ expect(pageLintField(report)).toMatchObject({ [PAGE_LINT_KEY]: { total: 1 } });+ });++ test("valid_from in frontmatter silences the finding and keeps the key absent", () => {+ const rel = writeNote(+ "Notes/DatedFrontmatter.md",+ `---\ntitle: D\nvalid_from: 2026-10-01\n---\n\n${BODY}\n`,+ );+ const report = lintWrittenPages(vault, [rel], READ_ALL);+ expect(report.findings).toEqual([]);+ expect(pageLintField(report)).toEqual({});+ });++ test("a valid_from that names no date does not silence the finding", () => {+ // The hatch is structural: a value that places nothing in time (TBD,+ // unknown, soon) is not a date, and treating any non-empty string as+ // one let a placeholder boundary quiet the lint.+ const rel = writeNote(+ "Notes/UndatedFrontmatter.md",+ `---\ntitle: D\nvalid_from: TBD\n---\n\n${BODY}\n`,+ );+ const report = lintWrittenPages(vault, [rel], READ_ALL);+ expect(report.findings.map((f) => f.code)).toContain(UNDATED_QUANTITY_CODE);+ });++ test("an ISO date in the body silences the finding", () => {+ const rel = writeNote(+ "Notes/DatedBody.md",+ `---\ntitle: D\n---\n\nOn 2026-10-01 ${BODY.toLowerCase()}\n`,+ );+ const report = lintWrittenPages(vault, [rel], READ_ALL);+ expect(undatedFindings(report)).toEqual([]);+ expect(pageLintField(report)).toEqual({});+ });++ test("a [Source: <name>, YYYY-MM-DD] marker silences the finding", () => {+ const rel = writeNote(+ "Notes/Cited.md",+ `---\ntitle: C\n---\n\n${QUANTITY} [Source: annual report, 2026-10-01]\n`,+ );+ const report = lintWrittenPages(vault, [rel], READ_ALL);+ expect(undatedFindings(report)).toEqual([]);+ expect(pageLintField(report)).toEqual({});+ });++ test("a version token, a table row and a code span are not quantities", () => {+ const rel = writeNote(+ "Notes/NotQuantities.md",+ "---\ntitle: N\n---\n\nupgraded to v1.73.0\n\n| 5,000 users | 35% |\n\n`5,000 users`\n",+ );+ const report = lintWrittenPages(vault, [rel], READ_ALL);+ expect(undatedFindings(report)).toEqual([]);+ expect(pageLintField(report)).toEqual({});+ });++ test("the finding cap still counts what it drops", () => {+ const lines = Array.from({ length: PAGE_LINT_MAX_FINDINGS + 2 }, () => "5 users").join("\n");+ const rel = writeNote("Notes/ManyUndated.md", `---\ntitle: M\n---\n\n${lines}\n`);+ const report = lintWrittenPages(vault, [rel], READ_ALL);+ expect(report.total).toBe(PAGE_LINT_MAX_FINDINGS + 2);+ expect(report.returned).toBe(PAGE_LINT_MAX_FINDINGS);+ expect(report.truncated).toBe(true);+ expect(report.findings.every((f) => f.code === UNDATED_QUANTITY_CODE)).toBe(true);+ });+});+describe("lintWrittenPages - bounds", () => {test("a page over the artifact byte cap is skipped with a reason, never dropped", () => {const filler = "x".repeat(ARTIFACT_MAX_BYTES + 1024);diff --git a/tests/core/brain/pledge-intake.test.ts b/tests/core/brain/pledge-intake.test.tsindex 1b85ba2b..fd736116 100644--- a/tests/core/brain/pledge-intake.test.ts+++ b/tests/core/brain/pledge-intake.test.ts@@ -153,6 +153,20 @@ test("a created pledge inherits the source page's visibility", () => {expect(parseFrontmatter(page!.path)[0]["visibility"]).toEqual([REMOTE_DENY_VISIBILITY_TOKEN]);});+test("a created pledge inherits an informal privacy marker from the source", () => {+ // No `visibility:` key - the reservation is stated only under the+ // pre-convention informal key, and the evidence quotes the source, so+ // the derived page must inherit the reservation through the union.+ writeFileSync(+ join(vault, SOURCE_REL),+ `---\nconfidentiality: secret\n---\n\n${SOURCE_TEXT}`,+ "utf8",+ );+ intake([item()]);+ const page = showPledge(vault, { slug: "send-ada-the-migration-recap" });+ expect(parseFrontmatter(page!.path)[0]["visibility"]).toEqual([REMOTE_DENY_VISIBILITY_TOKEN]);+});+test("an empty intake is a no-op that reports nothing", () => {// The MCP tool refuses an empty array as INVALID_PARAMS in parseIntakeArgs// before the kernel runs, so no tool caller reaches this branch; the testdiff --git a/tests/core/brain/pledges.test.ts b/tests/core/brain/pledges.test.tsindex 673b768e..8f03fbbd 100644--- a/tests/core/brain/pledges.test.ts+++ b/tests/core/brain/pledges.test.ts@@ -314,6 +314,18 @@ test("addPledge inherits source visibility and omits it for public sources", ()]);});+test("addPledge inherits an informal privacy marker from the source", () => {+ // No `visibility:` key - the reservation is stated only under the+ // pre-convention informal key, and the evidence quotes the source, so+ // the derived page must inherit the reservation through the union.+ writeFileSync(join(vault, SOURCE_REL), `---\naccess: private\n---\n\n${SOURCE_TEXT}`, "utf8");+ const page = addPledge(vault, {+ ...cite(["Grace will review the incident timeline"]),+ title: "Marker-source pledge",+ });+ expect(parseFrontmatter(page.path)[0]["visibility"]).toEqual([REMOTE_DENY_VISIBILITY_TOKEN]);+});+test("evidence containment forgives presentation, never wording", () => {// The shared comparator strips Markdown emphasis and line markers, so a// quote over a list item or emphasis passes; wording and word boundariesdiff --git a/tests/core/brain/privacy-marker-check.test.ts b/tests/core/brain/privacy-marker-check.test.tsnew file mode 100644index 00000000..73b83db3--- /dev/null+++ b/tests/core/brain/privacy-marker-check.test.ts@@ -0,0 +1,95 @@+/**+ * The doctor's informal-privacy-marker finding (write-honesty, F1 unit 4).+ *+ * An informal marker (`access: private`, `confidentiality: secret`,+ * `sensitivity: confidential`) already withholds a page from a remote+ * read, but the reservation is invisible to every consumer that reads+ * `visibility:` directly. The doctor's job is to make that gap+ * discoverable: ONE warning naming how many pages are in that state and+ * which ones, silent when there are none.+ *+ * The count is the restamp's own candidate list, so the finding and the+ * preview-first verb can never disagree about the population.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { runDoctor } from "../../../src/core/brain/doctor.ts";+import { PRIVACY_MARKER_UNSTAMPED_CODE } from "../../../src/core/brain/doctor/privacy-marker-check.ts";+import { planPrivacyMarkerRestamp } from "../../../src/core/brain/privacy-marker-restamp.ts";+import { resetVaultIdentityPins } from "../../../src/core/brain/vault-identity.ts";++let tmp: string;+let vault: string;++function page(rel: string, frontmatter: string): void {+ const abs = join(vault, rel);+ mkdirSync(join(abs, ".."), { recursive: true });+ writeFileSync(abs, `---\n${frontmatter}\n---\n\nbody\n`);+}++beforeEach(() => {+ tmp = mkdtempSync(join(tmpdir(), "o2b-doctor-privacy-marker-"));+ vault = join(tmp, "vault");+ for (const d of ["preferences", "retired", "inbox", "processed", "log"]) {+ mkdirSync(join(vault, "Brain", d), { recursive: true });+ }+ writeFileSync(join(vault, "Brain", "_brain.yaml"), "schema_version: 1\n");+ resetVaultIdentityPins();+});++afterEach(() => {+ resetVaultIdentityPins();+ rmSync(tmp, { recursive: true, force: true });+});++async function findings() {+ const report = await runDoctor(vault);+ return report.warnings.filter((i) => i.code === PRIVACY_MARKER_UNSTAMPED_CODE);+}++describe("the privacy-marker check", () => {+ test("is silent on a vault with no marker pages", async () => {+ page("open.md", "title: Open");+ page("formal.md", "title: Formal\nvisibility: [private]");+ expect(await findings()).toEqual([]);+ });++ test("one warning counts every unstamped marker page and names the paths", async () => {+ page("open.md", "title: Open");+ page("access.md", "title: Access\naccess: private");+ page("nested/confidential.md", "title: Confidential\nconfidentiality: secret");+ page("team.md", "title: Team\nsensitivity: confidential\nvisibility: [team]");++ const found = await findings();+ expect(found).toHaveLength(1);+ expect(found[0]!.severity).toBe("warning");+ expect(found[0]!.path).toBe(vault);+ expect(found[0]!.message).toContain("3 page(s)");+ expect(found[0]!.message).toContain("access.md");+ expect(found[0]!.message).toContain("nested/confidential.md");+ expect(found[0]!.message).toContain("team.md");+ });++ test("a marker page already carrying the reserved token is not counted", async () => {+ page("already.md", "title: Already\naccess: private\nvisibility: [private]");+ page("pending.md", "title: Pending\naccess: private");++ const found = await findings();+ expect(found).toHaveLength(1);+ expect(found[0]!.message).toContain("1 page(s)");+ expect(found[0]!.message).toContain("pending.md");+ expect(found[0]!.message).not.toContain("already.md");+ });++ test("the finding clears once the restamp has run", async () => {+ page("access.md", "title: Access\naccess: private");+ expect(await findings()).toHaveLength(1);++ planPrivacyMarkerRestamp(vault, { apply: true });+ expect(await findings()).toEqual([]);+ });+});diff --git a/tests/core/brain/privacy-marker-folds.test.ts b/tests/core/brain/privacy-marker-folds.test.tsnew file mode 100644index 00000000..f3c5bcdf--- /dev/null+++ b/tests/core/brain/privacy-marker-folds.test.ts@@ -0,0 +1,251 @@+/**+ * Derived pages inherit an informal privacy marker's reservation+ * (write-honesty review, fold regression).+ *+ * The read roots deny a remote read for a page saying `access: private`+ * (`graph/privacy-markers.ts`), but every page DERIVED from such a page -+ * a preference the dream pass drafts from its signals, a cross-topic rule+ * promoted from its sources, a counterparty-brief item drawn from a+ * pledge - folds the `visibility:` tokens of its inputs. A fold fed only+ * `pageVisibility` never sees the informal claim, so the derived page+ * ships the source's words at default visibility. Each test below runs a+ * marker page through one fold and asks the derived artifact for the+ * reserved token.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdtempSync, readFileSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { dream } from "../../../src/core/brain/dream.ts";+import { emptyPlan, type ScanResult } from "../../../src/core/brain/dream-plan.ts";+import {+ planCrossTopicRules,+ type ResolvedCrossTopicConfig,+} from "../../../src/core/brain/dream-plan-cross-topic.ts";+import type { RefreshResult } from "../../../src/core/brain/dream-refresh.ts";+import {+ buildCounterpartyBrief,+ type CounterpartyBrief,+ type CounterpartyBriefAbstention,+} from "../../../src/core/brain/counterparty-brief.ts";+import { addPledge } from "../../../src/core/brain/pledges.ts";+import { bootstrapBrain } from "../../../src/core/brain/init.ts";+import { upsertEntity } from "../../../src/core/brain/entities/registry.ts";+import { brainConfigPath, preferencePath } from "../../../src/core/brain/paths.ts";+import { writePreference } from "../../../src/core/brain/preference.ts";+import { writeSignal } from "../../../src/core/brain/signal.ts";+import { scanBrain } from "../../../src/core/brain/dream-scan.ts";+import { atomicWriteFileSync } from "../../../src/core/fs-atomic.ts";+import {+ pageVisibility,+ REMOTE_DENY_VISIBILITY_TOKEN,+} from "../../../src/core/graph/visibility.ts";+import { parseFrontmatter } from "../../../src/core/vault.ts";++const NOW = new Date("2026-06-05T12:00:00Z");+const SIGNAL_STAMP = "2026-06-01T10:00:00Z";+const AGENT = "claude";+const TOPIC = "release-notes";+const PRINCIPLE = "Quote the budget figure in every release note.";++let tmp: string;++beforeEach(() => {+ tmp = mkdtempSync(join(tmpdir(), "o2b-marker-folds-"));+});++afterEach(() => {+ rmSync(tmp, { recursive: true, force: true });+});++/** Stamp an informal marker line into a page's frontmatter. */+function stampMarker(path: string, line: string): void {+ atomicWriteFileSync(path, readFileSync(path, "utf8").replace(/^---\n/, `---\n${line}\n`));+}++/** The visibility tokens of the one drafted preference, after a real pass. */+function draftedVisibility(vault: string): ReadonlyArray<string> {+ const report = dream(vault, { now: NOW, agentName: AGENT });+ expect(report.new_unconfirmed).toHaveLength(1);+ const path = preferencePath(vault, report.new_unconfirmed[0]!.slice("pref-".length));+ expect(readFileSync(path, "utf8")).toContain(PRINCIPLE);+ return pageVisibility(parseFrontmatter(path)[0]);+}++describe("the dream pass folds a marker into the preference it drafts", () => {+ function newVault(name: string): string {+ const vault = join(tmp, name);+ const configPath = join(tmp, `${name}.yaml`);+ atomicWriteFileSync(configPath, `vault: ${vault}\nagent_name: ${AGENT}\n`);+ bootstrapBrain(vault, { configPath });+ const yaml = readFileSync(brainConfigPath(vault), "utf8").replace(+ /^ {2}candidate_threshold: \d+$/m,+ " candidate_threshold: 3",+ );+ atomicWriteFileSync(brainConfigPath(vault), yaml);+ return vault;+ }++ function seed(vault: string, slug: string, marker: boolean): void {+ const { path } = writeSignal(vault, {+ topic: TOPIC,+ signal: "positive",+ agent: AGENT,+ principle: PRINCIPLE,+ created_at: SIGNAL_STAMP,+ date: SIGNAL_STAMP.slice(0, 10),+ slug,+ });+ if (marker) stampMarker(path, "access: private");+ }++ test("one marker signal in the cluster reserves the drafted preference", () => {+ const vault = newVault("promote");+ seed(vault, "a", false);+ seed(vault, "b", true);+ seed(vault, "c", false);+ expect(draftedVisibility(vault)).toEqual([REMOTE_DENY_VISIBILITY_TOKEN]);+ });++ test("rebutting from marker signals reserves the drafted preference", () => {+ const vault = newVault("rebut");+ writePreference(vault, {+ slug: "notes-rule",+ topic: TOPIC,+ principle: "The rule of record for release notes.",+ created_at: "2026-05-01T00:00:00Z",+ unconfirmed_until: "2026-05-15T00:00:00Z",+ confirmed_at: "2026-05-08T00:00:00Z",+ status: "confirmed",+ evidenced_by: [],+ applied_count: 2,+ violated_count: 0,+ last_evidence_at: "2026-06-01T00:00:00Z",+ confidence: "medium",+ });+ // No evidence resolves the active sign, so a unanimous batch reads as+ // a rebuttal and drafts a `-rebut` preference from the signals.+ seed(vault, "a", true);+ seed(vault, "b", true);+ seed(vault, "c", true);+ expect(draftedVisibility(vault)).toEqual([REMOTE_DENY_VISIBILITY_TOKEN]);+ });+});++describe("the cross-topic planner folds a marker into the rule it promotes", () => {+ test("a marker source reserves the promoted rule", () => {+ const vault = join(tmp, "cross-topic");+ const configPath = join(tmp, "cross-topic.yaml");+ atomicWriteFileSync(configPath, `vault: ${vault}\n`);+ bootstrapBrain(vault, { configPath });+ // Two confirmed sources, different topics, both ownerless, sharing+ // enough principle text to clear the jaccard floor.+ const sources: ReadonlyArray<[string, string, string, boolean]> = [+ ["alpha", "scripts", "prefer bun over node for scripts", true],+ ["beta", "tooling", "prefer bun over node for tooling", false],+ ];+ for (const [slug, topic, principle, marked] of sources) {+ const { path } = writePreference(vault, {+ slug,+ topic,+ principle,+ created_at: "2026-05-01T00:00:00Z",+ unconfirmed_until: "2026-05-15T00:00:00Z",+ confirmed_at: "2026-05-08T00:00:00Z",+ status: "confirmed",+ evidenced_by: [],+ applied_count: 1,+ violated_count: 0,+ last_evidence_at: "2026-06-01T00:00:00Z",+ confidence: "medium",+ });+ if (marked) stampMarker(path, "access: private");+ }+ const scan: ScanResult = scanBrain(vault);+ const refresh: RefreshResult = {+ confirmed: new Set(),+ updated: new Map(),+ bandDrops: [],+ outcomeRegressions: [],+ };+ const config: ResolvedCrossTopicConfig = { minTopics: 2, jaccard: 0.6 };+ const plan = emptyPlan();+ const result = planCrossTopicRules({ scan, refresh, config, ledger: null, now: NOW, plan });+ expect(result.fired).toBe(true);+ expect(plan.crossTopicRules).toHaveLength(1);+ expect(plan.crossTopicRules[0]!.visibility).toContain(REMOTE_DENY_VISIBILITY_TOKEN);+ });+});++describe("the counterparty brief folds a marker into the items it injects", () => {+ const PROMPT = "meet Ada Lovelace to prepare the review";+ let vault: string;+ let entityId: string;++ beforeEach(() => {+ vault = join(tmp, "brief");+ const configPath = join(tmp, "brief.yaml");+ atomicWriteFileSync(configPath, `vault: ${vault}\n`);+ bootstrapBrain(vault, { configPath });+ entityId = upsertEntity(vault, {+ category: "people",+ name: "Ada Lovelace",+ agent: AGENT,+ now: NOW,+ }).entity.id;+ });++ /** Seed one open pledge to the entity and stamp its page or its source. */+ function seedPledge(title: string, stamp: "pledge" | "source" | null): void {+ const sourceRel = "notes/scratch.md";+ const sourceAbs = join(vault, sourceRel);+ atomicWriteFileSync(+ sourceAbs,+ "---\ntitle: Scratch\n---\n\nThe promise was made in prose here.\n",+ );+ const page = addPledge(vault, {+ title,+ due: "2026-06-10",+ source: sourceRel,+ evidence: ["The promise was made in prose here."],+ counterparties: [entityId],+ agent: AGENT,+ now: NOW,+ });+ const pledgePath = join(vault, "Brain", "pledges", `${page.slug}.md`);+ if (stamp === "pledge") stampMarker(pledgePath, "access: private");+ if (stamp === "source") stampMarker(sourceAbs, "access: private");+ }++ function brief(): CounterpartyBrief | CounterpartyBriefAbstention {+ return buildCounterpartyBrief(vault, {+ text: PROMPT,+ triggers: ["meet"],+ now: NOW,+ });+ }++ test("a pledge page carrying a marker withholds its item", () => {+ seedPledge("Send the review notes", "pledge");+ const result = brief();+ if ("abstain" in result) throw new Error(`unexpected abstention: ${result.abstain}`);+ expect(result.items).toEqual([]);+ });++ test("a cited source page carrying a marker withholds the item", () => {+ seedPledge("Send the review notes", "source");+ const result = brief();+ if ("abstain" in result) throw new Error(`unexpected abstention: ${result.abstain}`);+ expect(result.items).toEqual([]);+ });++ test("control: a pledge whose marker value does not deny still surfaces", () => {+ seedPledge("Send the published notes", null);+ const result = brief();+ if ("abstain" in result) throw new Error(`unexpected abstention: ${result.abstain}`);+ expect(result.items).toHaveLength(1);+ expect(result.items[0]!.text).toContain("Send the published notes");+ });+});diff --git a/tests/core/brain/privacy-marker-restamp.test.ts b/tests/core/brain/privacy-marker-restamp.test.tsnew file mode 100644index 00000000..1e71d875--- /dev/null+++ b/tests/core/brain/privacy-marker-restamp.test.ts@@ -0,0 +1,373 @@+/**+ * Preview-first restamp of informal privacy markers (write-honesty, F1 unit 3).+ *+ * The read roots already deny a remote read for `access: private` and its+ * siblings, but the reservation lives only in the informal key: tooling+ * that reads `visibility:` directly (derived-page stamping, exports) does+ * not see it. This verb makes the claim formal - additively.+ *+ * The contract is the repair-verb family's, spelled at+ * `authored-at-backfill.ts`:+ *+ * - dry run by DEFAULT; `apply` is the caller's opt-in;+ * - the write guard is asserted only when the call will write;+ * - add-only: the reserved token is unioned into `visibility:`, and no+ * key or existing token is ever removed;+ * - idempotent: a re-run over a restamped vault is a byte-identical+ * no-op;+ * - a page already carrying `visibility: private` is not a candidate.+ */++import { afterEach, beforeEach, describe, expect, mock, test } from "bun:test";+import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { planPrivacyMarkerRestamp } from "../../../src/core/brain/privacy-marker-restamp.ts";+import * as vaultModule from "../../../src/core/vault.ts";+import {+ VaultIdentityMismatchError,+ assertVaultIdentityForWrite,+ resetVaultIdentityPins,+ vaultIdentityPath,+ writeVaultIdentity,+} from "../../../src/core/brain/vault-identity.ts";+import { atomicWriteFileSync } from "../../../src/core/fs-atomic.ts";++let vault: string;++function page(rel: string, frontmatter: string, body = "body text"): string {+ const abs = join(vault, rel);+ mkdirSync(join(abs, ".."), { recursive: true });+ writeFileSync(abs, `---\n${frontmatter}\n---\n\n${body}\n`);+ return abs;+}++beforeEach(() => {+ resetVaultIdentityPins();+ vault = mkdtempSync(join(tmpdir(), "privacy-marker-restamp-"));+ page("open.md", "title: Open");+ page("access.md", "title: Access\naccess: private");+ page("team-marked.md", "title: Team\nsensitivity: confidential\nvisibility: [team]");+ page("nested/array.md", "title: Array\nconfidentiality: [internal, Secret]");+ page("spelling.md", "title: Spelling\naccess: private\nvisibility: [Owners]");+ page("stamped.md", "title: Stamped\naccess: private\nvisibility: [private]");+ page("formal.md", "title: Formal\nvisibility: [team]");+ page("access.md.bak.md", "title: Backup\naccess: private");+});+afterEach(() => {+ resetVaultIdentityPins();+ rmSync(vault, { recursive: true, force: true });+});++function candidates(): ReadonlyArray<{ path: string; visibilityLine: string }> {+ return planPrivacyMarkerRestamp(vault).candidates.map((c) => ({+ path: c.path,+ visibilityLine: c.visibilityLine,+ }));+}++describe("a dry run writes nothing", () => {+ test("it reports the candidates and their would-be visibility line", () => {+ const result = planPrivacyMarkerRestamp(vault);+ expect(result.applied).toBe(false);+ expect(result.updated).toBe(0);+ expect(result.scanned).toBe(6);+ expect(candidates()).toEqual([+ { path: "access.md", visibilityLine: "visibility: [private]" },+ { path: "access.md.bak.md", visibilityLine: "visibility: [private]" },+ { path: "nested/array.md", visibilityLine: "visibility: [private]" },+ { path: "spelling.md", visibilityLine: "visibility: [Owners, private]" },+ { path: "team-marked.md", visibilityLine: "visibility: [team, private]" },+ ]);+ });++ test("every file is byte-identical after the preview", () => {+ const before = readFileSync(join(vault, "access.md"), "utf8");+ planPrivacyMarkerRestamp(vault);+ expect(readFileSync(join(vault, "access.md"), "utf8")).toBe(before);+ });++ test("a vault with no marker pages is silent and unchanged", () => {+ rmSync(join(vault, "access.md"));+ rmSync(join(vault, "access.md.bak.md"));+ rmSync(join(vault, "nested"), { recursive: true, force: true });+ rmSync(join(vault, "spelling.md"));+ rmSync(join(vault, "team-marked.md"));+ rmSync(join(vault, "stamped.md"));+ const result = planPrivacyMarkerRestamp(vault);+ expect(result.scanned).toBe(0);+ expect(result.candidates).toEqual([]);+ expect(result.updated).toBe(0);+ });+});++describe("apply unions the reserved token into visibility", () => {+ test("it writes the reserved token and keeps the declared one", () => {+ const result = planPrivacyMarkerRestamp(vault, { apply: true });+ expect(result.applied).toBe(true);+ expect(result.updated).toBe(result.candidates.length);++ const team = readFileSync(join(vault, "team-marked.md"), "utf8");+ expect(team).toContain("visibility: [team, private]");+ expect(team).toContain("sensitivity: confidential");+ expect(team).toContain("body text");++ const access = readFileSync(join(vault, "access.md"), "utf8");+ expect(access).toContain("visibility: [private]");+ expect(access).toContain("access: private");+ });++ test("existing tokens keep their declared spelling", () => {+ const result = planPrivacyMarkerRestamp(vault, { apply: true });+ expect(result.updated).toBe(5);+ const spelling = readFileSync(join(vault, "spelling.md"), "utf8");+ expect(spelling).toContain("visibility: [Owners, private]");+ // A marker-only page gets the reserved token with nothing beside it.+ const array = readFileSync(join(vault, "nested/array.md"), "utf8");+ expect(array).toContain("visibility: [private]");+ expect(array).toContain("confidentiality: [internal, Secret]");+ });++ test("a page already carrying the reserved token is untouched", () => {+ const before = readFileSync(join(vault, "stamped.md"), "utf8");+ planPrivacyMarkerRestamp(vault, { apply: true });+ expect(readFileSync(join(vault, "stamped.md"), "utf8")).toBe(before);+ });++ test("an unmarked page is untouched", () => {+ const formal = readFileSync(join(vault, "formal.md"), "utf8");+ const open = readFileSync(join(vault, "open.md"), "utf8");+ planPrivacyMarkerRestamp(vault, { apply: true });+ expect(readFileSync(join(vault, "formal.md"), "utf8")).toBe(formal);+ expect(readFileSync(join(vault, "open.md"), "utf8")).toBe(open);+ });++ test("a re-run is a byte-identical no-op", () => {+ planPrivacyMarkerRestamp(vault, { apply: true });+ const files = ["access.md", "team-marked.md", "nested/array.md", "spelling.md", "stamped.md"];+ const afterFirst = files.map((f) => readFileSync(join(vault, f), "utf8"));++ const second = planPrivacyMarkerRestamp(vault, { apply: true });+ expect(second.candidates).toEqual([]);+ expect(second.updated).toBe(0);+ expect(files.map((f) => readFileSync(join(vault, f), "utf8"))).toEqual(afterFirst);+ });+});++describe("apply refuses pages the scanner cannot round-trip", () => {+ test("a candidate carrying a dropped frontmatter line is skipped by name and left byte-identical", () => {+ // The stray dash item (no open key to land in) is a line the+ // line-based scanner drops with a notice: a rewrite through the+ // writer would silently lose it, breaking the no-key-removed promise.+ page("notice.md", "title: Notice\naccess: private\n- a stray dash item with no key");+ // The fixture is honest: the scanner really does report the line.+ const [, , notices] = vaultModule.parseFrontmatterWithNotices(join(vault, "notice.md"));+ expect(notices.length).toBe(1);+ const before = readFileSync(join(vault, "notice.md"), "utf8");+ const result = planPrivacyMarkerRestamp(vault, { apply: true });+ // The scan still nominates the page - the marker is real - but the+ // apply pass refuses it, names it, and says why.+ expect(result.candidates.map((c) => c.path)).toContain("notice.md");+ const skip = result.skipped.find((s) => s.path === "notice.md");+ expect(skip).toBeDefined();+ expect(skip!.reason).toContain("dropped");+ // Every other candidate is stamped; only the notice page is refused.+ expect(result.updated).toBe(5);+ expect(result.skipped).toHaveLength(1);+ // The bytes are untouched: the informal marker still denies remote+ // reads, and no line was lost to a rewrite.+ expect(readFileSync(join(vault, "notice.md"), "utf8")).toBe(before);+ });++ test("a dry run reports no skips and touches nothing", () => {+ page("notice.md", "title: Notice\naccess: private\n- a stray dash item with no key");+ const result = planPrivacyMarkerRestamp(vault);+ expect(result.applied).toBe(false);+ expect(result.skipped).toEqual([]);+ expect(result.updated).toBe(0);+ });++ test("a candidate whose frontmatter carries a comment is skipped by name and left byte-identical", () => {+ // A `#` line inside the block is skipped by the scanner WITHOUT a+ // notice (the format documents comments as content it discards), so+ // the refuse-on-notice guard above cannot see it - and a rewrite+ // through the writer would still delete it. The apply pass must+ // refuse the page the same way, by name.+ page("commented.md", "title: Commented\n# review before sharing\naccess: private");+ const before = readFileSync(join(vault, "commented.md"), "utf8");+ const result = planPrivacyMarkerRestamp(vault, { apply: true });+ expect(result.candidates.map((c) => c.path)).toContain("commented.md");+ const skip = result.skipped.find((s) => s.path === "commented.md");+ expect(skip).toBeDefined();+ expect(skip!.reason).toContain("comment");+ expect(result.updated).toBe(5);+ expect(readFileSync(join(vault, "commented.md"), "utf8")).toBe(before);+ });++ test("a candidate whose frontmatter carries an indented mapping is skipped by name and left byte-identical", () => {+ // The scanner trims each line before matching key/value, so an+ // indented nested mapping is flattened into top-level keys with no+ // notice; re-serialising the parsed map would write the flattening.+ page("nested-map.md", "title: Nested\naccess: private\nowner:\n team: core");+ const before = readFileSync(join(vault, "nested-map.md"), "utf8");+ const result = planPrivacyMarkerRestamp(vault, { apply: true });+ expect(result.candidates.map((c) => c.path)).toContain("nested-map.md");+ const skip = result.skipped.find((s) => s.path === "nested-map.md");+ expect(skip).toBeDefined();+ expect(skip!.reason).toContain("indented");+ expect(result.updated).toBe(5);+ expect(readFileSync(join(vault, "nested-map.md"), "utf8")).toBe(before);+ });++ test("a candidate that cannot be re-read is skipped as unreadable, not as a dropped line", () => {+ // A file that disappears (or becomes unreadable) between the scan and+ // the write surfaces as one `frontmatter-unreadable` notice; the+ // refusal must name the read failure, not point the operator at a+ // YAML problem the page does not have.+ withUnreadableAtWriteTime("access.md", () => {+ const result = planPrivacyMarkerRestamp(vault, { apply: true });+ const skip = result.skipped.find((s) => s.path === "access.md");+ expect(skip).toBeDefined();+ expect(skip!.reason).toContain("could not be re-read");+ expect(result.updated).toBe(4);+ });+ });+});++// --- The write reads the page it is about to rewrite ------------------------++/**+ * The scan and the write happen in one call, so the window between them is+ * simulated at the seam both passes read through: `parseFrontmatterWithNotices`.+ * The vault walk parses the page first (the scan), the apply loop re-parses+ * it second (the write-time recompute), so the mock below lands the+ * operator edit on disk the instant the page is parsed for the SECOND time -+ * the scan has seen the old bytes, the apply loop sees the new ones.+ */+// Captured at module load, before any `mock.module` call can replace the+// namespace's function: a capture inside the helper would pick up the+// previous test's mock and chain the mocks instead of restoring.+const realParseFrontmatterWithNotices = vaultModule.parseFrontmatterWithNotices;++function withEditAtWriteTime(rel: string, nextFrontmatter: string, run: () => void): void {+ const realParse = realParseFrontmatterWithNotices;+ const target = join(vault, rel);+ let parses = 0;+ mock.module("../../../src/core/vault.ts", () => ({+ ...vaultModule,+ parseFrontmatterWithNotices(path: string) {+ if (path === target) {+ parses += 1;+ if (parses === 2) {+ writeFileSync(target, `---\n${nextFrontmatter}\n---\n\nbody text\n`);+ }+ }+ return realParse(path);+ },+ }));+ try {+ run();+ } finally {+ mock.module("../../../src/core/vault.ts", () => vaultModule);+ }+}++/**+ * Same seam, the read-failure arm: the apply loop's re-parse reports the+ * page as unreadable, the way a file deleted (or moved to a failing+ * mount) between the scan and the write answers. The notice is built in+ * the shape `parseFrontmatterWithNotices` emits for a failed read, so the+ * core module's reason selection runs against real data.+ */+function withUnreadableAtWriteTime(rel: string, run: () => void): void {+ const realParse = realParseFrontmatterWithNotices;+ const target = join(vault, rel);+ let parses = 0;+ mock.module("../../../src/core/vault.ts", () => ({+ ...vaultModule,+ parseFrontmatterWithNotices(path: string) {+ if (path === target) {+ parses += 1;+ if (parses === 2) {+ return [+ {},+ "",+ [+ {+ code: "frontmatter-unreadable",+ site: "vault.parseFrontmatter",+ path,+ detail: "frontmatter read failed: ENOENT simulated by the test",+ },+ ],+ ];+ }+ }+ return realParse(path);+ },+ }));+ try {+ run();+ } finally {+ mock.module("../../../src/core/vault.ts", () => vaultModule);+ }+}++describe("apply recomputes the tokens from the page as it stands now", () => {+ test("a candidate edited between the scan and the write keeps its new declared token", () => {+ withEditAtWriteTime("access.md", "title: Access\naccess: private\nvisibility: [team]", () => {+ const result = planPrivacyMarkerRestamp(vault, { apply: true });+ // The page was rewritten, from the tokens the edit declared.+ expect(result.updated).toBe(5);+ const md = readFileSync(join(vault, "access.md"), "utf8");+ expect(md).toContain("visibility: [team, private]");+ expect(md).toContain("access: private");+ });+ });++ test("a candidate whose marker vanished between the scan and the write is skipped", () => {+ withEditAtWriteTime("access.md", "title: Access\nvisibility: [team]", () => {+ const result = planPrivacyMarkerRestamp(vault, { apply: true });+ expect(result.updated).toBe(4);+ // The reported list still shows what the scan found; the write did+ // not touch the page the scan can no longer nominate.+ expect(result.candidates.map((c) => c.path)).toContain("access.md");+ const md = readFileSync(join(vault, "access.md"), "utf8");+ // The operator's edit stands and no reservation was stamped.+ expect(md).toContain("visibility: [team]");+ expect(md).not.toContain("visibility: [private]");+ });+ });+});++// --- The write guard: at the entry point, only when the call writes --------++describe("vault identity", () => {+ function makeVaultIdentityMismatch(): void {+ writeVaultIdentity(vault);+ assertVaultIdentityForWrite(vault);+ atomicWriteFileSync(+ vaultIdentityPath(vault),+ `${JSON.stringify(+ { schema_version: 1, vault_id: "f".repeat(32), created_at: "2026-01-01T00:00:00.000Z" },+ null,+ 2,+ )}\n`,+ );+ }++ test("a preview still runs against a root the guard would refuse", () => {+ makeVaultIdentityMismatch();+ expect(planPrivacyMarkerRestamp(vault).applied).toBe(false);+ });++ test("an apply refuses before writing anything", () => {+ makeVaultIdentityMismatch();+ const before = readFileSync(join(vault, "access.md"), "utf8");+ expect(() => planPrivacyMarkerRestamp(vault, { apply: true })).toThrow(+ VaultIdentityMismatchError,+ );+ expect(readFileSync(join(vault, "access.md"), "utf8")).toBe(before);+ });+});diff --git a/tests/core/brain/provenance/content-class-paths.test.ts b/tests/core/brain/provenance/content-class-paths.test.tsnew file mode 100644index 00000000..06618d4f--- /dev/null+++ b/tests/core/brain/provenance/content-class-paths.test.ts@@ -0,0 +1,142 @@+/**+ * The deterministic prompt-derived write paths (F5, task E2).+ *+ * Three write paths are definitionally prompt-derived and stamp+ * `content_class: prompt-derived` at creation, server-side:+ * - a note created over the MCP transport (the mcp-tool origin channel),+ * - a signal replayed from a session by the import,+ * - a research report the in-process pipeline synthesised.+ *+ * An operator-authored CLI write stays unmarked - the class describes the+ * material, and the CLI is where an operator's own words enter the vault.+ * The writer-input pin at the bottom is the type-level half of the+ * origin-channel census carried over to this axis: no writer input may+ * declare the field, or an in-process caller could assert what only the+ * writer that knows the material may claim.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join, resolve } from "node:path";+import { readFileSync as readSource } from "node:fs";++import { brainDirs } from "../../../../src/core/brain/paths.ts";+import { DEFAULT_BRAIN_CONFIG_YAML } from "../../../../src/core/brain/config-template.ts";+import { atomicWriteFileSync } from "../../../../src/core/fs-atomic.ts";+import { importSession } from "../../../../src/core/brain/sessions/import.ts";+import { writeResearchReport } from "../../../../src/core/brain/research/research.ts";+import { bootstrapBrain } from "../../../../src/core/brain/init.ts";+import { createNote } from "../../../../src/core/brain/notes/create-note.ts";+import {+ CONTENT_CLASS_FIELD,+ CONTENT_CLASS,+} from "../../../../src/core/brain/provenance/content-class.ts";+import {+ clearOriginChannel,+ ORIGIN_CHANNEL,+ setOriginChannel,+} from "../../../../src/core/origin-channel.ts";++const CLAUDE = resolve("tests/fixtures/sessions/claude-minimal.jsonl");++let vault: string;+let configHome: string;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-content-class-paths-"));+ configHome = mkdtempSync(join(tmpdir(), "o2b-content-class-cfg-"));+ const dirs = brainDirs(vault);+ for (const d of [dirs.brain, dirs.inbox, dirs.processed, dirs.preferences, dirs.log]) {+ mkdirSync(d, { recursive: true });+ }+ atomicWriteFileSync(join(dirs.brain, "_brain.yaml"), DEFAULT_BRAIN_CONFIG_YAML);+ const configPath = join(configHome, "config.yaml");+ atomicWriteFileSync(configPath, `vault: ${vault}\nagent_name: claude\n`);+ bootstrapBrain(vault, { configPath });+});++afterEach(() => {+ clearOriginChannel();+ rmSync(vault, { recursive: true, force: true });+ rmSync(configHome, { recursive: true, force: true });+});++describe("session import stamps the class", () => {+ test("every signal a session replay writes carries content_class", async () => {+ const res = await importSession(vault, CLAUDE, { agent: "test" });+ expect(res.signals_created).toBeGreaterThan(0);+ const inbox = readdirSync(brainDirs(vault).inbox).filter((n) => n.startsWith("sig-"));+ expect(inbox.length).toBe(res.signals_created);+ for (const name of inbox) {+ const text = readFileSync(join(brainDirs(vault).inbox, name), "utf8");+ expect(`${name}: ${text.includes(`${CONTENT_CLASS_FIELD}: prompt-derived`)}`).toBe(+ `${name}: true`,+ );+ }+ });++ test("the stamp rides the signal through a re-run dedup unchanged", async () => {+ await importSession(vault, CLAUDE, { agent: "test" });+ const second = await importSession(vault, CLAUDE, { agent: "test" });+ expect(second.signals_created).toBe(0);+ const inbox = readdirSync(brainDirs(vault).inbox).filter((n) => n.startsWith("sig-"));+ for (const name of inbox) {+ const text = readFileSync(join(brainDirs(vault).inbox, name), "utf8");+ expect(text.includes(`${CONTENT_CLASS_FIELD}: prompt-derived`)).toBe(true);+ }+ });+});++describe("research reports stamp the class", () => {+ test("a written report page carries content_class: prompt-derived", () => {+ const res = writeResearchReport(+ vault,+ {+ title: "Class stamp survey",+ sources: ["Articles/a.md"],+ findings: [{ statement: "One finding", sources: ["Articles/a.md"] }],+ },+ { agent: "claude", now: new Date("2026-06-13T12:00:00Z") },+ );+ expect(res.created).toBe(true);+ const text = readFileSync(join(vault, res.reportPath), "utf8");+ expect(text).toContain(`${CONTENT_CLASS_FIELD}: ${CONTENT_CLASS.promptDerived}`);+ });+});++describe("operator-authored CLI writes stay unmarked", () => {+ test("a note created under a claimed cli channel carries no class", () => {+ setOriginChannel(ORIGIN_CHANNEL.cli);+ const res = createNote(vault, { path: "Notes/ByHand.md", content: "operator words" });+ expect(res.created).toBe(true);+ const text = readFileSync(join(vault, "Notes/ByHand.md"), "utf8");+ expect(text).not.toContain(CONTENT_CLASS_FIELD);+ });+});++describe("no writer input declares the class", () => {+ test("the caller-facing input interfaces have no content_class field", () => {+ // The type-level pin, in the shape the origin-channel census uses for+ // the channel: the one input interface a caller fills in per writer+ // family must not carry the field, or TypeScript could be talked into+ // letting an in-process caller assert its own class.+ const WRITER_INPUT_TYPES: ReadonlyArray<readonly [string, string]> = Object.freeze([+ ["src/core/brain/notes/create-note.ts", "CreateNoteInput"],+ ["src/core/brain/research/research.ts", "ResearchReportInput"],+ ["src/core/brain/sessions/import.ts", "ImportSessionOptions"],+ ]);+ const offenders: string[] = [];+ for (const [path, typeName] of WRITER_INPUT_TYPES) {+ const text = readSource(resolve(path), "utf8");+ const start = text.indexOf(`interface ${typeName} {`);+ expect(start).toBeGreaterThanOrEqual(0);+ const end = text.indexOf("\n}", start);+ const block = text.slice(start, end);+ if (new RegExp(`readonly\\s+${CONTENT_CLASS_FIELD}\\??\\s*:`).test(block)) {+ offenders.push(`${path}: ${typeName}.${CONTENT_CLASS_FIELD}`);+ }+ }+ expect(offenders).toEqual([]);+ });+});diff --git a/tests/core/brain/provenance/content-class.test.ts b/tests/core/brain/provenance/content-class.test.tsnew file mode 100644index 00000000..0c55da62--- /dev/null+++ b/tests/core/brain/provenance/content-class.test.ts@@ -0,0 +1,175 @@+/**+ * The content class stamp (F5, write-time honesty and egress consent).+ *+ * `content_class` is the sibling of `origin_channel` on a different axis:+ * not which process wrote the page, but what kind of material the page+ * carries. The vocabulary is closed with ONE member, `prompt-derived` -+ * text an agent produced from a prompt - and one member is enough because+ * the stamp exists for exactly one decision: shared exports leave such+ * pages out by default, and an operator override names them back in.+ *+ * The rules pinned here mirror the origin channel's, because they are the+ * same rules about a second axis:+ * - the value is server-derived at creation; a caller naming one is+ * ignored, never honored,+ * - the key is reserved against caller updates (forge and strip both),+ * - absence of the key is a meaning - "written before the stamp+ * existed" - and reads as unmarked, never as a guessed default,+ * - unknown values refuse at the stamp and read as unmarked on disk,+ * where a closed vocabulary met a hand editor.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { dirname, join } from "node:path";++import {+ CONTENT_CLASSES,+ CONTENT_CLASS_FIELD,+ CONTENT_CLASS,+ contentClassCreationStamp,+ ContentClassValueError,+ isContentClass,+ readContentClass,+ resolveContentClass,+} from "../../../../src/core/brain/provenance/content-class.ts";+import { applyWriteBatch, WriteBatchError } from "../../../../src/core/brain/write-batch.ts";+import { createNote } from "../../../../src/core/brain/notes/create-note.ts";+import {+ clearOriginChannel,+ ORIGIN_CHANNEL,+ setOriginChannel,+} from "../../../../src/core/origin-channel.ts";++let vault: string;+beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-content-class-"));+});+afterEach(() => {+ clearOriginChannel();+ rmSync(vault, { recursive: true, force: true });+});++describe("the content class vocabulary", () => {+ test("is closed, with prompt-derived as its one member", () => {+ expect(CONTENT_CLASSES).toEqual([CONTENT_CLASS.promptDerived]);+ expect(CONTENT_CLASS.promptDerived).toBe("prompt-derived");+ expect(isContentClass("prompt-derived")).toBe(true);+ expect(isContentClass("operator")).toBe(false);+ expect(isContentClass("prompt")).toBe(false);+ expect(isContentClass(undefined)).toBe(false);+ });++ test("the on-disk key avoids every origin-channel census spelling", () => {+ // The origin-channel census bans four spellings as caller-facing+ // property names; the new key must not collide with any of them, so+ // the two server-derived stamps never become one grep hit.+ for (const banned of ["origin_channel", "originChannel", "origin-channel", "channel"]) {+ expect(CONTENT_CLASS_FIELD.includes(banned)).toBe(false);+ }+ expect(CONTENT_CLASS_FIELD).toBe("content_class");+ });++ test("resolveContentClass refuses unknown values with a named error", () => {+ expect(resolveContentClass("prompt-derived")).toBe("prompt-derived");+ try {+ resolveContentClass("confidential");+ throw new Error("expected throw");+ } catch (err) {+ expect(err).toBeInstanceOf(ContentClassValueError);+ expect((err as ContentClassValueError).message).toContain("confidential");+ expect((err as ContentClassValueError).message).toContain("prompt-derived");+ }+ });+});++describe("the creation stamp", () => {+ test("an mcp-tool process stamps prompt-derived", () => {+ setOriginChannel(ORIGIN_CHANNEL.mcpTool);+ expect(contentClassCreationStamp(ORIGIN_CHANNEL.mcpTool)).toBe("prompt-derived");+ });++ test("a cli process, an import process, and an unclaimed process stay unmarked", () => {+ expect(contentClassCreationStamp(ORIGIN_CHANNEL.cli)).toBeNull();+ expect(contentClassCreationStamp(ORIGIN_CHANNEL.import)).toBeNull();+ expect(contentClassCreationStamp("unset")).toBeNull();+ });+});++describe("createNote merges the class server-side", () => {+ test("an mcp-tool write stamps the class and a caller-named class loses", () => {+ setOriginChannel(ORIGIN_CHANNEL.mcpTool);+ const res = createNote(vault, {+ path: "Notes/FromAgent.md",+ frontmatter: { title: "FromAgent", [CONTENT_CLASS_FIELD]: "operator" },+ content: "body",+ });+ expect(res.created).toBe(true);+ const written = readFileSync(join(vault, "Notes/FromAgent.md"), "utf8");+ expect(written).toContain(`${CONTENT_CLASS_FIELD}: prompt-derived`);+ expect(written).not.toContain("operator");+ });++ test("a cli write stays byte-honest: unmarked, even when the caller names a class", () => {+ const res = createNote(vault, {+ path: "Notes/ByOperator.md",+ frontmatter: { title: "ByOperator", [CONTENT_CLASS_FIELD]: "prompt-derived" },+ content: "body",+ });+ expect(res.created).toBe(true);+ const written = readFileSync(join(vault, "Notes/ByOperator.md"), "utf8");+ expect(written).not.toContain(CONTENT_CLASS_FIELD);+ });+});++describe("an unmarked page reads as unmarked", () => {+ test("absence of the key is the pre-stamp era, not a guess", () => {+ expect(readContentClass({})).toBeNull();+ expect(readContentClass({ title: "Old page" })).toBeNull();+ });++ test("the reserved member reads back; an unknown hand-written value does not mark", () => {+ expect(readContentClass({ [CONTENT_CLASS_FIELD]: "prompt-derived" })).toBe("prompt-derived");+ expect(readContentClass({ [CONTENT_CLASS_FIELD]: "hand-written" })).toBeNull();+ });+});++describe("an update cannot forge or strip the class", () => {+ function seed(rel: string, body: string, frontmatter: string): void {+ const abs = join(vault, rel);+ mkdirSync(dirname(abs), { recursive: true });+ writeFileSync(abs, `---\n${frontmatter}\n---\n\n${body}\n`, "utf8");+ }++ test("the write batch refuses a caller-named class and names the key", () => {+ seed("Notes/Marked.md", "body", "title: Marked\ncontent_class: prompt-derived");+ try {+ applyWriteBatch(vault, [+ {+ kind: "update_note",+ path: "Notes/Marked.md",+ frontmatter: { [CONTENT_CLASS_FIELD]: "operator" },+ },+ ]);+ throw new Error("expected throw");+ } catch (err) {+ expect(err).toBeInstanceOf(WriteBatchError);+ expect((err as WriteBatchError).code).toBe("reserved_frontmatter_key");+ expect((err as WriteBatchError).details).toMatchObject({ key: CONTENT_CLASS_FIELD });+ }+ expect(readFileSync(join(vault, "Notes/Marked.md"), "utf8")).toContain(+ `${CONTENT_CLASS_FIELD}: prompt-derived`,+ );+ });++ test("an update that touches other keys preserves the stamp in place", () => {+ seed("Notes/Marked2.md", "body", "title: Marked2\ncontent_class: prompt-derived");+ applyWriteBatch(vault, [+ { kind: "update_note", path: "Notes/Marked2.md", frontmatter: { status: "final" } },+ ]);+ const written = readFileSync(join(vault, "Notes/Marked2.md"), "utf8");+ expect(written).toContain("status: final");+ expect(written).toContain(`${CONTENT_CLASS_FIELD}: prompt-derived`);+ });+});diff --git a/tests/core/brain/provenance/session-stamp-error.test.ts b/tests/core/brain/provenance/session-stamp-error.test.tsnew file mode 100644index 00000000..596cacba--- /dev/null+++ b/tests/core/brain/provenance/session-stamp-error.test.ts@@ -0,0 +1,90 @@+/**+ * The session import's stamp-failure accounting.+ *+ * A signal a session replay writes is stamped `content_class:+ * prompt-derived` on the same run that wrote it (pinned in+ * `content-class-paths.test.ts`). When the stamp raises - the write+ * succeeded, the stamp did not - the import records the failure in its+ * `errors` list and keeps going, and the row must name the page the+ * failure is about: the freshly written signal page, not the session+ * transcript the import merely read. A row that names the transcript+ * sends the operator to reread a file with nothing wrong with it while+ * the unmarked page sits unexamined in the inbox.+ *+ * The stamp machinery is mocked at the module seam (the established+ * `mock.module` idiom) because `writeSignal` always emits a parseable+ * frontmatter block - there is no honest input that fails the stamp on+ * its own, and the accounting, not the stamp, is what is under test.+ */++import { afterEach, beforeEach, describe, expect, mock, test } from "bun:test";+import { mkdirSync, mkdtempSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join, resolve } from "node:path";++import { brainDirs } from "../../../../src/core/brain/paths.ts";+import { DEFAULT_BRAIN_CONFIG_YAML } from "../../../../src/core/brain/config-template.ts";+import { atomicWriteFileSync } from "../../../../src/core/fs-atomic.ts";+import { bootstrapBrain } from "../../../../src/core/brain/init.ts";++const CONTENT_CLASS_MODULE = "../../../../src/core/brain/provenance/content-class.ts";+const IMPORT_MODULE = "../../../../src/core/brain/sessions/import.ts";+const CLAUDE = resolve("tests/fixtures/sessions/claude-minimal.jsonl");++let vault: string;+let configHome: string;+let realContentClass: Record<string, unknown> | null = null;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-stamp-error-"));+ configHome = mkdtempSync(join(tmpdir(), "o2b-stamp-error-cfg-"));+ const dirs = brainDirs(vault);+ for (const d of [dirs.brain, dirs.inbox, dirs.processed, dirs.preferences, dirs.log]) {+ mkdirSync(d, { recursive: true });+ }+ atomicWriteFileSync(join(dirs.brain, "_brain.yaml"), DEFAULT_BRAIN_CONFIG_YAML);+ const configPath = join(configHome, "config.yaml");+ atomicWriteFileSync(configPath, `vault: ${vault}\nagent_name: claude\n`);+ bootstrapBrain(vault, { configPath });+});++afterEach(() => {+ // This runner's mock registration outlives mock.restore() across test+ // FILES, so the teardown re-registers the REAL module object instead:+ // whatever leaks past this file is content-class.ts itself, byte for+ // byte.+ if (realContentClass !== null) mock.module(CONTENT_CLASS_MODULE, () => realContentClass);+ mock.restore();+ realContentClass = null;+ rmSync(vault, { recursive: true, force: true });+ rmSync(configHome, { recursive: true, force: true });+});++describe("a stamp failure is reported against the signal page", () => {+ test("the error row names the written signal, not the session transcript", async () => {+ realContentClass = { ...(await import(CONTENT_CLASS_MODULE)) };+ mock.module(CONTENT_CLASS_MODULE, () => ({+ ...realContentClass,+ stampPromptDerivedLine: () => {+ throw new Error("stamp machinery exploded");+ },+ }));+ // Imported AFTER the mock so the import's binding is the stub.+ const { importSession } = await import(IMPORT_MODULE);++ const res = await importSession(vault, CLAUDE, { agent: "test" });+ // The writes themselves succeeded; every created signal then failed+ // to stamp, and every failure is named.+ expect(res.signals_created).toBeGreaterThan(0);+ expect(res.errors).toHaveLength(res.signals_created);++ const transcript = resolve(CLAUDE);+ const inbox = brainDirs(vault).inbox;+ for (const row of res.errors) {+ expect(row.message).toContain("content-class stamp failed");+ expect(`${row.path === transcript}: not the transcript`).toBe("false: not the transcript");+ expect(row.path.startsWith(inbox)).toBe(true);+ expect(row.path.endsWith(".md")).toBe(true);+ }+ });+});diff --git a/tests/core/brain/provenance/stamp-boundary.test.ts b/tests/core/brain/provenance/stamp-boundary.test.tsnew file mode 100644index 00000000..af59384b--- /dev/null+++ b/tests/core/brain/provenance/stamp-boundary.test.ts@@ -0,0 +1,113 @@+/**+ * Where {@link stampPromptDerivedLine} puts the stamp line.+ *+ * The stamp is a text insert: one `content_class: prompt-derived` line+ * added inside a page's frontmatter block, every other byte preserved.+ * The only question is WHERE the block ends, and the answer must be the+ * parser's answer - the same boundary `parseFrontmatterText` closes at -+ * because a stamp placed anywhere else is a page that reads unmarked no+ * matter what the bytes say. The cases here are the ones where a raw+ * `"\n---\n"` search disagrees with that boundary: a close fence that+ * carries trailing whitespace, with a bare `---` line further down that+ * the raw search then finds instead, and a close at end-of-file with no+ * trailing newline at all.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import {+ CONTENT_CLASS_FIELD,+ CONTENT_CLASS,+ readContentClass,+ stampPromptDerivedLine,+} from "../../../../src/core/brain/provenance/content-class.ts";+import { parseFrontmatterText } from "../../../../src/core/vault.ts";++let dir: string;+beforeEach(() => {+ dir = mkdtempSync(join(tmpdir(), "o2b-stamp-boundary-"));+});+afterEach(() => rmSync(dir, { recursive: true, force: true }));++function stamp(raw: string): string {+ const abs = join(dir, "page.md");+ writeFileSync(abs, raw, "utf8");+ stampPromptDerivedLine(abs);+ return readFileSync(abs, "utf8");+}++describe("stampPromptDerivedLine places the stamp at the parser's boundary", () => {+ test("a close the raw search misses stamps inside the block, not after a bare --- line", () => {+ // The close fence carries a trailing space, so the parser closes+ // there (`\n---` plus whitespace) while a raw `"\n---\n"` search+ // cannot see it and runs on to the bare `---` line below. A stamp+ // dropped at THAT line lands outside the parsed block: the page+ // reads unmarked, and the value region the author wrote is where+ // the marker went instead.+ const raw = [+ "---",+ "title: Checkpoint",+ "summary: a value the author continued past a rule",+ "--- ",+ "prose that names a rule:",+ "---",+ "",+ "body text",+ "",+ ].join("\n");+ const stamped = stamp(raw);++ // The page reads marked.+ expect(readContentClass(parseFrontmatterText(stamped)[0])).toBe(CONTENT_CLASS.promptDerived);+ // And the stamp is a frontmatter line, not body content: the body+ // the parser reads is unchanged.+ expect(parseFrontmatterText(stamped)[1]).toBe(parseFrontmatterText(raw)[1]);+ expect(stamped).toContain(+ `summary: a value the author continued past a rule\n` ++ `${CONTENT_CLASS_FIELD}: ${CONTENT_CLASS.promptDerived}\n`,+ );+ });++ test("a close at end-of-file without a trailing newline still stamps", () => {+ // The parser accepts this page (`\n---` at end-of-string closes the+ // block); the raw search found no `"\n---\n"` and refused it as+ // unterminated, leaving a written page unstampable.+ const stamped = stamp("---\ntitle: Final\ndeleted: no\n---");+ expect(readContentClass(parseFrontmatterText(stamped)[0])).toBe(CONTENT_CLASS.promptDerived);+ expect(stamped).toBe(+ `---\ntitle: Final\ndeleted: no\n${CONTENT_CLASS_FIELD}: ${CONTENT_CLASS.promptDerived}\n---`,+ );+ });++ test("a plain page stamps in place and a second stamp is a byte-level no-op", () => {+ const raw = "---\ntitle: Plain\nkind: note\n---\n\nbody\n";+ const once = stamp(raw);+ expect(once).toBe(+ `---\ntitle: Plain\nkind: note\n${CONTENT_CLASS_FIELD}: ${CONTENT_CLASS.promptDerived}\n---\n\nbody\n`,+ );+ const abs = join(dir, "page.md");+ stampPromptDerivedLine(abs);+ expect(readFileSync(abs, "utf8")).toBe(once);+ });++ test("a page already carrying an out-of-vocabulary class is refused, not double-stamped", () => {+ // The read side maps an unknown class value to unmarked, but the+ // STAMP must not read that as "no key": planting a second+ // `content_class:` line gives the frontmatter a duplicate key, which+ // strict YAML consumers reject outright.+ const abs = join(dir, "page.md");+ writeFileSync(+ abs,+ "---\ntitle: Hand\ntype: note\ncontent_class: agent-session-v2\n---\n\nbody\n",+ "utf8",+ );+ expect(() => stampPromptDerivedLine(abs)).toThrow(/content_class/);+ // The refusal never writes: the bytes are untouched.+ expect(readFileSync(abs, "utf8")).toBe(+ "---\ntitle: Hand\ntype: note\ncontent_class: agent-session-v2\n---\n\nbody\n",+ );+ });+});diff --git a/tests/core/brain/research/research-capture-scope.test.ts b/tests/core/brain/research/research-capture-scope.test.tsindex 28cc2d37..2562d015 100644--- a/tests/core/brain/research/research-capture-scope.test.ts+++ b/tests/core/brain/research/research-capture-scope.test.ts@@ -28,9 +28,9 @@ const NOW = new Date("2026-06-13T12:00:00Z");const LOCAL_SOURCE = "Articles/a.md";const URL_SOURCE = "https://example.test/articles/a";-/** The report page the base release (v1.66.0) wrote for one local source. */+/** The report page for one local source; the in-process generation stamp landed in the content-class wave. */const BASE_LOCAL_REPORT =- '---\nkind: brain-report\ntitle: T\nreport_date: 2026-06-13\nprovenance: stated\nsource_count: 1\ncreated_at: "2026-06-13T12:00:00Z"\nupdated_at: "2026-06-13T12:00:00Z"\ntags: [brain, brain/report]\n---\n\n# T\n\n## Findings\n\n- S (cites: [[Articles/a.md]])\n\n## Sources\n\n- [[Articles/a.md]]\n';+ '---\nkind: brain-report\ntitle: T\nreport_date: 2026-06-13\nprovenance: stated\nsource_count: 1\ncontent_class: prompt-derived\ncreated_at: "2026-06-13T12:00:00Z"\nupdated_at: "2026-06-13T12:00:00Z"\ntags: [brain, brain/report]\n---\n\n# T\n\n## Findings\n\n- S (cites: [[Articles/a.md]])\n\n## Sources\n\n- [[Articles/a.md]]\n';function seed(rel: string): void {mkdirSync(join(vault, rel, ".."), { recursive: true });diff --git a/tests/core/brain/tension-detect.test.ts b/tests/core/brain/tension-detect.test.tsindex c7e839e7..585ae1c5 100644--- a/tests/core/brain/tension-detect.test.ts+++ b/tests/core/brain/tension-detect.test.ts@@ -18,6 +18,7 @@ import {listTensions,TENSION_STATUS,} from "../../../src/core/brain/tensions.ts";+import { REMOTE_DENY_VISIBILITY_TOKEN } from "../../../src/core/graph/visibility.ts";let vault: string;@@ -85,6 +86,26 @@ describe("detectTensionsInVault", () => {expect(listTensions(vault)[0]!.detectedCount).toBe(2);});+ test("a tension drafted from a marker-only note carries the reserved token", () => {+ // One side reserves itself under the informal key only - no+ // `visibility:` line anywhere in the corpus - so the derived tension+ // page inherits the reservation through the marker union or not at all.+ writeConfig("\nnotes:\n read_paths:\n - Notes\n");+ writeMd(+ "Notes/tabs.md",+ "---\nid: note-tabs\naccess: private\n---\nAlways use tabs for indentation in source files.\n",+ );+ writeMd(+ "Notes/spaces.md",+ "---\nid: note-spaces\n---\nNever use tabs for indentation in source files.\n",+ );+ const res = detectTensionsInVault(vault, { agent: "tester" });+ expect(res.created).toBe(1);+ const persisted = listTensions(vault);+ expect(persisted.length).toBe(1);+ expect(persisted[0]!.visibility).toEqual([REMOTE_DENY_VISIBILITY_TOKEN]);+ });+test("an unconfigured vault (no read_paths) scans nothing and creates nothing", () => {// No notes block at all: default read_paths is empty.writeConfig("");diff --git a/tests/core/brain/undated-quantity-diagnostics.test.ts b/tests/core/brain/undated-quantity-diagnostics.test.tsnew file mode 100644index 00000000..52cd6ca7--- /dev/null+++ b/tests/core/brain/undated-quantity-diagnostics.test.ts@@ -0,0 +1,59 @@+/**+ * The undated-quantity diagnostic signal (write-honesty, F4, t_fb442a24).+ *+ * The finding is advisory, but an advisory with no registered exit is a+ * dead end: `nextCommandField` would contribute no key and the operator+ * would read a quantity warning with nowhere to go. The claims surface is+ * the place a claim's validity is read, so the code must resolve to it by+ * name. Pinned by NAME here so the producer and the registry cannot drift+ * into two spellings of one code.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { DIAGNOSTIC_SIGNALS } from "../../../src/core/brain/diagnostics.ts";+import { nextCommandField, resolveNextStep } from "../../../src/core/brain/next-step.ts";+import { UNDATED_QUANTITY_CODE, lintWrittenPages } from "../../../src/core/brain/page-lint.ts";+import { READ_ALL_REFS } from "../../../src/core/brain/near-duplicate.ts";++let vault: string;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-undated-diagnostics-"));+ for (const dir of ["preferences", "retired", "log", "inbox"]) {+ mkdirSync(join(vault, "Brain", dir), { recursive: true });+ }+ mkdirSync(join(vault, "Notes"), { recursive: true });+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+});++describe("the undated-quantity diagnostic signal", () => {+ test("the code is registered with the claims surface as its exit", () => {+ expect(UNDATED_QUANTITY_CODE).toBe("undated-quantity");+ expect(DIAGNOSTIC_SIGNALS.get(UNDATED_QUANTITY_CODE)).toMatchObject({+ code: UNDATED_QUANTITY_CODE,+ nextCommand: "o2b brain claims",+ autoRepairable: false,+ });+ expect(resolveNextStep(UNDATED_QUANTITY_CODE)).toMatchObject({+ code: UNDATED_QUANTITY_CODE,+ nextCommand: "o2b brain claims",+ });+ expect(nextCommandField(UNDATED_QUANTITY_CODE)).toEqual({ next_command: "o2b brain claims" });+ });++ test("an undated-quantity finding carries the resolved next command", () => {+ const rel = "Notes/Undated.md";+ writeFileSync(join(vault, rel), "---\ntitle: U\n---\n\nWe onboarded 5,000 users.\n", "utf8");+ const report = lintWrittenPages(vault, [rel], { readable: READ_ALL_REFS });+ const finding = report.findings.find((f) => f.code === UNDATED_QUANTITY_CODE);+ expect(finding).toBeDefined();+ expect(finding!.next_command).toBe(resolveNextStep(UNDATED_QUANTITY_CODE)!.nextCommand);+ });+});diff --git a/tests/core/brain/undated-quantity.test.ts b/tests/core/brain/undated-quantity.test.tsnew file mode 100644index 00000000..58f96211--- /dev/null+++ b/tests/core/brain/undated-quantity.test.ts@@ -0,0 +1,235 @@+/**+ * Undated-quantity recognizer (write-honesty, F4, t_fb442a24).+ *+ * A count or amount written with no structural date becomes a claim that+ * answers as true at every instant, because an absent `valid_from` is+ * unbounded. This recognizer owns only the high-precision half: find the+ * lexemes a writer would have to date, and say where they are.+ *+ * The corpus is table-driven in both directions. The negative half is as+ * load-bearing as the positive one: a version string, an ISO date, a+ * table cell, a code span and the frontmatter block are all places a+ * number appears without being an undated claim about the vault.+ */++import { describe, expect, test } from "bun:test";++import {+ containsIsoDate,+ findUndatedQuantities,+} from "../../../src/core/brain/undated-quantity.ts";++interface PositiveCase {+ readonly name: string;+ readonly body: string;+ /** Lexemes in ascending offset order, after overlap resolution. */+ readonly lexemes: ReadonlyArray<string>;+}++const POSITIVE: ReadonlyArray<PositiveCase> = [+ {+ name: "currency symbol with a magnitude suffix",+ body: "the migration cost $4.2M in total",+ lexemes: ["$4.2M"],+ },+ {+ name: "currency code after the amount",+ body: "the invoice totals 10 USD",+ lexemes: ["10 USD"],+ },+ {+ name: "percent",+ body: "35% of sessions were recalled",+ lexemes: ["35%"],+ },+ {+ name: "fractional percent",+ body: "2.5% growth against last quarter",+ lexemes: ["2.5%"],+ },+ {+ name: "thousands-separated count with a unit",+ body: "5,000 users signed up",+ lexemes: ["5,000 users"],+ },+ {+ name: "number with a time unit",+ body: "the pass took 359 ms",+ lexemes: ["359 ms"],+ },+ {+ name: "number with a count unit",+ body: "it served 1,200 requests per run",+ lexemes: ["1,200 requests"],+ },+ {+ name: "number with a data-size unit",+ body: "a 512 MB payload",+ lexemes: ["512 MB"],+ },+ {+ name: "bare thousands-separated count",+ body: "the counter reads 12,400 and stops",+ lexemes: ["12,400"],+ },+ {+ name: "several quantities keep ascending offsets",+ body: "35% of 5,000 users",+ lexemes: ["35%", "5,000 users"],+ },+ {+ name: "a thousands-separated count followed by a prose comma",+ body: "the counter reads 12,400, then stops",+ lexemes: ["12,400"],+ },+ {+ name: "a year-shaped number with a thousands separator is still a quantity",+ body: "2,024 users were migrated",+ lexemes: ["2,024 users"],+ },+];++const NEGATIVE: ReadonlyArray<PositiveCase> = [+ {+ name: "an ISO calendar date",+ body: "published 2026-10-22 by the team",+ lexemes: [],+ },+ {+ name: "an ISO instant",+ body: "the run ended at 2026-10-22T10:30:00Z",+ lexemes: [],+ },+ {+ name: "a version token with a v prefix",+ body: "upgraded to v1.73.0 today",+ lexemes: [],+ },+ {+ name: "a bare semantic version",+ body: "release 1.73.0 shipped",+ lexemes: [],+ },+ {+ name: "an IPv4 address",+ body: "bound to 192.168.1.1 here",+ lexemes: [],+ },+ {+ name: "ordered-list markers on their own",+ body: "1. first entry\n2. second entry",+ lexemes: [],+ },+ {+ name: "a table row",+ body: "| 5,000 users | 35% |",+ lexemes: [],+ },+ {+ name: "an inline code span",+ body: "the token `5,000 users` is quoted",+ lexemes: [],+ },+ {+ name: "a fenced code block",+ body: "```\n5,000 users\n35%\n```",+ lexemes: [],+ },+ {+ name: "a count in the frontmatter block",+ body: "---\ncount: 5,000\ntitle: X\n---\n\nplain prose",+ lexemes: [],+ },+ {+ name: "a count in a CRLF frontmatter block",+ body: "---\r\ncount: 5,000\r\n---\r\n\r\nplain prose",+ lexemes: [],+ },+ {+ name: "a decimal with no unit",+ body: "pi is 3.14159 here",+ lexemes: [],+ },+ {+ name: "a number followed by a word outside the unit set",+ body: "42 boxes arrived",+ lexemes: [],+ },+ {+ name: "a candidate span inside an ISO date with a later table row",+ body: "Date: 2026-10-22 days\n\n| table | column |\n",+ lexemes: [],+ },+ {+ name: "a version token overlapping a quantity trigger",+ body: "release 1.2.3 USD shipped",+ lexemes: [],+ },+ {+ name: "a four-digit year followed by a count unit is not a quantity",+ body: "In 2024 users doubled",+ lexemes: [],+ },+];++describe("findUndatedQuantities - the trigger classes", () => {+ for (const entry of POSITIVE) {+ test(entry.name, () => {+ const found = findUndatedQuantities(entry.body);+ expect(found.map((f) => f.lexeme)).toEqual([...entry.lexemes]);+ for (const item of found) {+ // The offset addresses the lexeme in the caller's own string.+ expect(entry.body.slice(item.offset, item.offset + item.lexeme.length)).toBe(item.lexeme);+ }+ });+ }+});++describe("findUndatedQuantities - the exclusions", () => {+ for (const entry of NEGATIVE) {+ test(entry.name, () => {+ expect(findUndatedQuantities(entry.body)).toEqual([]);+ });+ }++ test("an ordered-list marker is excluded without hiding its content", () => {+ const found = findUndatedQuantities("1. 5,000 users");+ expect(found.map((f) => f.lexeme)).toEqual(["5,000 users"]);+ expect(found[0]!.offset).toBe(3);+ });++ test("a CRLF frontmatter block masks only itself", () => {+ const head = "---\r\ncount: 5,000\r\n---\r\n\r\n";+ expect(findUndatedQuantities(`${head}plain prose`)).toEqual([]);++ // The closing fence on CRLF must not swallow the promoted body: a+ // quantity after it is still authored prose.+ const found = findUndatedQuantities(`${head}5,000 users`);+ expect(found.map((f) => f.lexeme)).toEqual(["5,000 users"]);+ expect(found[0]!.offset).toBe(head.length);+ });++ test("overlapping trigger classes yield the one longest lexeme", () => {+ expect(findUndatedQuantities("5,000 users").map((f) => f.lexeme)).toEqual(["5,000 users"]);+ });++ test("a pure scan leaves the input untouched and is repeatable", () => {+ const body = "35% of 5,000 users";+ expect(findUndatedQuantities(body)).toEqual(findUndatedQuantities(body));+ expect(body).toBe("35% of 5,000 users");+ });+});++describe("containsIsoDate", () => {+ test("recognises a date-only run anywhere in prose", () => {+ expect(containsIsoDate("published 2026-10-22 by the team")).toBe(true);+ });++ test("recognises an ISO instant", () => {+ expect(containsIsoDate("ended at 2026-10-22T10:30:00Z")).toBe(true);+ });++ test("does not read a version token as a date", () => {+ expect(containsIsoDate("upgraded to v1.73.0")).toBe(false);+ });+});diff --git a/tests/core/brain/vault-guard-census.test.ts b/tests/core/brain/vault-guard-census.test.tsindex d79b4dce..f1141709 100644--- a/tests/core/brain/vault-guard-census.test.ts+++ b/tests/core/brain/vault-guard-census.test.ts@@ -122,6 +122,11 @@ const UNGUARDED_WITH_REASON: Readonly<Record<string, string>> = Object.freeze({"the caller-supplied audit root. A host settings file has no vault " +"identity to assert; the executor is fail-closed on its own terms (a " +"failed backup denies the tool call).",+ "src/core/brain/provenance/content-class.ts":+ "the stamp helper composes no path of its own: the absolute page path arrives from " ++ "the caller, its one production call site (the session import loop, " ++ "sessions/import.ts) asserts vault identity ahead of the write, and the helper only " ++ "splices the class line into the frontmatter block the shared parser already delimited.","src/core/brain/provenance/source-archive.ts":"every path it writes or removes is derived inside this module from the " +"entry's own sha-256 and validated before the join, so the store cannot be " +diff --git a/tests/core/brain/write-batch.test.ts b/tests/core/brain/write-batch.test.tsindex cf24d11d..0313e89f 100644--- a/tests/core/brain/write-batch.test.ts+++ b/tests/core/brain/write-batch.test.ts@@ -45,6 +45,7 @@ import {WriteBatchError,type WriteOperation,} from "../../../src/core/brain/write-batch.ts";+import { PRIVACY_MARKER_KEYS } from "../../../src/core/graph/privacy-markers.ts";import { WriteRefusedError } from "../../../src/core/brain/pending/pending-lanes.ts";import { queryDecisionLedger } from "../../../src/core/brain/permissions/ledger.ts";import * as ledger from "../../../src/core/brain/idempotency-ledger.ts";@@ -211,6 +212,47 @@ describe("applyWriteBatch note operations", () => {expect(readFileSync(join(vault, "Notes/Other.md"), "utf8")).not.toContain("status: final");});+ test("update_note refuses the informal privacy marker keys, creation does not", () => {+ // The marker keys are boundary state like `visibility`: the read roots+ // resolve `access: private` (and its two siblings) to the reserved+ // remote-deny token, so a merged `access: ""` would clear the+ // reservation without the caller ever naming the boundary.+ for (const markerKey of PRIVACY_MARKER_KEYS) {+ const rel = `Notes/Marked-${markerKey}.md`;+ seedNote(rel, "body", `title: M\n${markerKey}: private`);+ let thrown: unknown;+ try {+ applyWriteBatch(vault, [+ { kind: "update_note", path: rel, frontmatter: { [markerKey]: "" } },+ ]);+ throw new Error("expected throw");+ } catch (err) {+ thrown = err;+ expect(err).toBeInstanceOf(WriteBatchError);+ expect((err as WriteBatchError).code).toBe("reserved_frontmatter_key");+ // The refusal names the key, in the visibility posture.+ expect((err as WriteBatchError).message).toContain(markerKey);+ expect((err as WriteBatchError).message).toContain("edit the note directly");+ expect((err as WriteBatchError).details).toMatchObject({ path: rel, key: markerKey });+ }+ // The marker survived the refused update byte-for-byte.+ expect(readFileSync(join(vault, rel), "utf8")).toContain(`${markerKey}: private`);+ }++ // Creation is not restricted: the guard lives at the update merge, so+ // a caller may still author a marker through create_note.+ const res = applyWriteBatch(vault, [+ {+ kind: "create_note",+ path: "Notes/Authored.md",+ frontmatter: { title: "A", access: "private" },+ content: "body",+ },+ ]);+ expect(res.applied).toBe(1);+ expect(readFileSync(join(vault, "Notes/Authored.md"), "utf8")).toContain("access: private");+ });+test("create_note refuses to clobber an existing note", () => {seedNote("Notes/Dup.md", "original");try {diff --git a/tests/core/content-class-egress.test.ts b/tests/core/content-class-egress.test.tsnew file mode 100644index 00000000..1be27440--- /dev/null+++ b/tests/core/content-class-egress.test.ts@@ -0,0 +1,349 @@+/**+ * The content-class egress policy (F5, task E3).+ *+ * The default at every shared export is exclusion: a page whose+ * `content_class` frontmatter carries the reserved `prompt-derived`+ * value is absent from the export, and every omission is NAMED on stderr+ * in the shape the knowledge pack's `blocked` list established. The+ * per-verb `--include-prompt-derived` override carries the pages back in+ * and says so. The transcript format has a contract to export session+ * material, so it never applies the default silently: marked records are+ * omitted and named, and a corpus the policy would empty is a refusal,+ * never an empty file.+ *+ * The vault pages here carry the class because they were WRITTEN with it+ * - the same bytes the deterministic prompt-derived writers put on disk.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import {+ existsSync,+ mkdirSync,+ mkdtempSync,+ readFileSync,+ readdirSync,+ rmSync,+ writeFileSync,+} from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { runCli } from "../helpers/run-cli.ts";+import {+ EGRESS_CONTENT_CLASS_INCLUDE_NOTICE,+ contentClassOmissionNotice,+ egressContentClassSplit,+} from "../../src/core/egress/guard.ts";+import {+ CONTENT_CLASS,+ CONTENT_CLASS_FIELD,+ treeCarriesPromptDerived,+} from "../../src/core/brain/provenance/content-class.ts";+import { spoolTranscriptCorpus, type SpoolOutcome } from "../../src/cli/brain/verbs/export.ts";++let tmp: string;+let vault: string;+let config: string;+const env = (): { OPEN_SECOND_BRAIN_CONFIG: string } => ({ OPEN_SECOND_BRAIN_CONFIG: config });++beforeEach(() => {+ tmp = mkdtempSync(join(tmpdir(), "o2b-class-egress-"));+ vault = join(tmp, "vault");+ config = join(tmp, "config.yaml");+});+afterEach(() => rmSync(tmp, { recursive: true, force: true }));++async function bootstrap(): Promise<void> {+ expect((await runCli(["init", "--vault", vault, "--name", "T"], { env: env() })).returncode).toBe(+ 0,+ );+ expect((await runCli(["brain", "init", "--vault", vault], { env: env() })).returncode).toBe(0);+ writeFileSync(+ join(vault, "Operator Page.md"),+ "---\ntitle: Operator Page\n---\n\nWritten by the operator.\n",+ "utf8",+ );+ writeFileSync(+ join(vault, "Agent Page.md"),+ `---\ntitle: Agent Page\n${CONTENT_CLASS_FIELD}: ${CONTENT_CLASS.promptDerived}\n---\n\nSynthesised from a prompt.\n`,+ "utf8",+ );+}++const MARKED = "Agent Page.md";++/**+ * A log day the bundle can fold into `log.md`; the export reads the+ * markdown shards of `Brain/log/` raw, so the block is written in the+ * shape `renderLogEntryBlock` produces.+ */+function seedLogDay(): void {+ const logDir = join(vault, "Brain", "log");+ mkdirSync(logDir, { recursive: true });+ writeFileSync(+ join(logDir, "2026-06-13.md"),+ "## 12:00:00Z — note\n- text: bundle fixture day\n- agent: test\n",+ "utf8",+ );+}++describe("the guard seam", () => {+ const items = ["a.md", "b.md", "c.md"];++ test("the default split excludes marked items and names each omission", () => {+ const split = egressContentClassSplit(+ items,+ {},+ {+ name: (entry) => entry,+ isPromptDerived: (entry) => entry === "b.md",+ },+ );+ expect(split.kept).toEqual(["a.md", "c.md"]);+ expect(split.omitted).toEqual(["b.md"]);+ });++ test("the override split is the identity with nothing to name", () => {+ const split = egressContentClassSplit(+ items,+ { contentClass: { include: true } },+ {+ name: (entry) => entry,+ isPromptDerived: () => true,+ },+ );+ expect(split.kept).toEqual(items);+ expect(split.omitted).toEqual([]);+ });++ test("the omission notice names every item with its reason and the flag", () => {+ const notice = contentClassOmissionNotice(["Notes/A.md", "Reports/B.md"]);+ expect(notice).toContain("2 item(s) left out as prompt-derived");+ expect(notice).toContain("Notes/A.md (content_class: prompt-derived)");+ expect(notice).toContain("Reports/B.md (content_class: prompt-derived)");+ expect(notice).toContain("--include-prompt-derived");+ expect(notice.endsWith("\n")).toBe(true);+ });+});++describe("okf-export applies the default and the override", () => {+ test("a marked page is absent from the bundle and named on stderr", async () => {+ await bootstrap();+ const out = join(tmp, "bundle");+ const r = await runCli(["brain", "okf-export", "--out", out], { env: env() });+ expect(r.returncode).toBe(0);+ expect(readdirSync(join(out, "concepts"))).toEqual(["Operator Page.md"]);+ expect(existsSync(join(out, "concepts", "Agent Page.md"))).toBe(false);+ expect(r.stderr).toContain(contentClassOmissionNotice([MARKED]));+ expect(r.stderr).not.toContain(EGRESS_CONTENT_CLASS_INCLUDE_NOTICE);+ });++ test("the override carries the marked page and says so", async () => {+ await bootstrap();+ const out = join(tmp, "bundle-override");+ const r = await runCli(["brain", "okf-export", "--out", out, "--include-prompt-derived"], {+ env: env(),+ });+ expect(r.returncode).toBe(0);+ expect(readFileSync(join(out, "concepts", "Agent Page.md"), "utf8")).toContain(+ "Synthesised from a prompt.",+ );+ expect(r.stderr).toContain(EGRESS_CONTENT_CLASS_INCLUDE_NOTICE);+ });+});++describe("okf-export writes the reserved bundle files", () => {+ test("a vault with zero prompt-derived pages still exports okf.json, index.md and log.md", async () => {+ await bootstrap();+ rmSync(join(vault, "Agent Page.md"));+ seedLogDay();+ const out = join(tmp, "bundle-plain");+ const r = await runCli(["brain", "okf-export", "--out", out], { env: env() });+ expect(r.returncode).toBe(0);+ // The reserved bundle-root files ride every bundle: the manifest, the+ // version-declaring index and the change log are the three files an+ // OKF reader expects at the root before it reads any page.+ expect(existsSync(join(out, "okf.json"))).toBe(true);+ expect(existsSync(join(out, "index.md"))).toBe(true);+ expect(existsSync(join(out, "log.md"))).toBe(true);+ expect(readFileSync(join(out, "log.md"), "utf8")).toContain("bundle fixture day");+ // The stdout page/log-day line stays truthful.+ expect(r.stdout).toContain("1 page(s), 1 log day(s)");+ });++ test("a vault with some prompt-derived pages exports them too, and the index lists only carried pages", async () => {+ await bootstrap();+ seedLogDay();+ const omitted = join(tmp, "bundle-default");+ const r = await runCli(["brain", "okf-export", "--out", omitted], { env: env() });+ expect(r.returncode).toBe(0);+ expect(existsSync(join(omitted, "okf.json"))).toBe(true);+ expect(existsSync(join(omitted, "index.md"))).toBe(true);+ expect(existsSync(join(omitted, "log.md"))).toBe(true);+ const index = readFileSync(join(omitted, "index.md"), "utf8");+ expect(index).toContain("concepts/Operator Page.md");+ expect(index).not.toContain("Agent");+ expect(r.stderr).toContain(contentClassOmissionNotice([MARKED]));++ const carried = join(tmp, "bundle-override");+ const ro = await runCli(["brain", "okf-export", "--out", carried, "--include-prompt-derived"], {+ env: env(),+ });+ expect(ro.returncode).toBe(0);+ expect(existsSync(join(carried, "okf.json"))).toBe(true);+ expect(existsSync(join(carried, "index.md"))).toBe(true);+ expect(existsSync(join(carried, "log.md"))).toBe(true);+ const indexBoth = readFileSync(join(carried, "index.md"), "utf8");+ expect(indexBoth).toContain("concepts/Operator Page.md");+ expect(indexBoth).toContain("concepts/Agent Page.md");+ });+});++describe("graph-export applies the default and the override", () => {+ test("a marked page has no node, and the omission is named", async () => {+ await bootstrap();+ const out = join(tmp, "graph.json");+ const r = await runCli(["brain", "graph-export", "--out", out], { env: env() });+ expect(r.returncode).toBe(0);+ const graph = JSON.parse(readFileSync(out, "utf8")) as {+ nodes: ReadonlyArray<{ path: string }>;+ };+ expect(graph.nodes.map((n) => n.path)).toEqual(["Operator Page.md"]);+ expect(r.stderr).toContain(contentClassOmissionNotice([MARKED]));+ });++ test("the override keeps the node and notices", async () => {+ await bootstrap();+ const out = join(tmp, "graph-override.json");+ const r = await runCli(["brain", "graph-export", "--out", out, "--include-prompt-derived"], {+ env: env(),+ });+ expect(r.returncode).toBe(0);+ const graph = JSON.parse(readFileSync(out, "utf8")) as {+ nodes: ReadonlyArray<{ path: string }>;+ };+ expect(graph.nodes.map((n) => n.path).toSorted()).toEqual([+ "Agent Page.md",+ "Operator Page.md",+ ]);+ expect(r.stderr).toContain(EGRESS_CONTENT_CLASS_INCLUDE_NOTICE);+ });+});++describe("bank-export applies the default and the override", () => {+ test("a marked page leaves both page sections, named once", async () => {+ await bootstrap();+ const out = join(tmp, "bank.json");+ const r = await runCli(["brain", "bank-export", "--out", out], { env: env() });+ expect(r.returncode).toBe(0);+ const bundle = JSON.parse(readFileSync(out, "utf8")) as {+ graph: { nodes: ReadonlyArray<{ path: string }> };+ pages: ReadonlyArray<{ path: string }>;+ };+ expect(bundle.graph.nodes.map((n) => n.path)).toEqual(["Operator Page.md"]);+ expect(bundle.pages.map((p) => p.path)).toEqual(["Operator Page.md"]);+ // One page omitted from two sections is ONE disclosure.+ expect(r.stderr.split(contentClassOmissionNotice([MARKED])).length - 1).toBe(1);+ });++ test("the override keeps both sections and notices", async () => {+ await bootstrap();+ const out = join(tmp, "bank-override.json");+ const r = await runCli(["brain", "bank-export", "--out", out, "--include-prompt-derived"], {+ env: env(),+ });+ expect(r.returncode).toBe(0);+ const bundle = JSON.parse(readFileSync(out, "utf8")) as {+ graph: { nodes: ReadonlyArray<{ path: string }> };+ pages: ReadonlyArray<{ path: string }>;+ };+ expect(bundle.graph.nodes).toHaveLength(2);+ expect(bundle.pages).toHaveLength(2);+ expect(r.stderr).toContain(EGRESS_CONTENT_CLASS_INCLUDE_NOTICE);+ });+});++/**+ * A stand-in corpus stream: the real one reads runtime session+ * directories; the loop under test only consumes records and a summary.+ */+async function* corpus(values: readonly unknown[]): AsyncGenerator<never, { scanned: number }> {+ for (const value of values) yield value as never;+ return { scanned: values.length };+}++async function collect(+ values: readonly unknown[],+ opts?: { readonly includePromptDerived?: boolean },+): Promise<{ outcome: SpoolOutcome; lines: string[] }> {+ const lines: string[] = [];+ const outcome = await spoolTranscriptCorpus(+ corpus(values) as unknown as Parameters<typeof spoolTranscriptCorpus>[0],+ (line: string) => lines.push(line),+ opts,+ );+ return { outcome, lines };+}++describe("the transcript corpus refuses or names, never silently empties", () => {+ const plainRecord = {+ session_id: "plain.jsonl",+ runtime: "claude",+ messages: [{ role: "user", content: "hello" }],+ };+ const markedRecord = {+ session_id: "marked.jsonl",+ runtime: "claude",+ provenance: { [CONTENT_CLASS_FIELD]: CONTENT_CLASS.promptDerived },+ messages: [{ role: "user", content: "from a prompt" }],+ };++ test("a marked record is omitted from the corpus and named", async () => {+ const { outcome, lines } = await collect([plainRecord, markedRecord]);+ expect(outcome.kind).toBe("written");+ if (outcome.kind !== "written") return;+ expect(outcome.written).toBe(1);+ expect(lines).toHaveLength(1);+ expect(lines[0]).toContain("plain.jsonl");+ expect(outcome.promptDerivedOmitted).toEqual(["marked.jsonl"]);+ });++ test("a corpus the policy would empty is a refusal that names what it held back", async () => {+ const { outcome, lines } = await collect([markedRecord]);+ expect(lines).toEqual([]);+ expect(outcome.kind).toBe("refused");+ if (outcome.kind !== "refused") return;+ expect(outcome.detail).toContain("content_class: prompt-derived");+ expect(outcome.detail).toContain("marked.jsonl");+ expect(outcome.detail).toContain("--include-prompt-derived");+ });++ test("the override carries every record", async () => {+ const { outcome, lines } = await collect([plainRecord, markedRecord], {+ includePromptDerived: true,+ });+ expect(outcome.kind).toBe("written");+ if (outcome.kind !== "written") return;+ expect(outcome.written).toBe(2);+ expect(outcome.promptDerivedOmitted).toEqual([]);+ expect(lines).toHaveLength(2);+ });+});++describe("the record tripwire", () => {+ test("finds a class mapping at any reasonable depth, and nothing else", () => {+ expect(treeCarriesPromptDerived({ [CONTENT_CLASS_FIELD]: CONTENT_CLASS.promptDerived })).toBe(+ true,+ );+ expect(+ treeCarriesPromptDerived({ a: [{ b: { [CONTENT_CLASS_FIELD]: "prompt-derived" } }] }),+ ).toBe(true);+ // A bare string that happens to spell the value is content, not a+ // claim about itself.+ expect(treeCarriesPromptDerived({ messages: ["prompt-derived"] })).toBe(false);+ expect(treeCarriesPromptDerived({ [CONTENT_CLASS_FIELD]: "hand-written" })).toBe(false);+ expect(treeCarriesPromptDerived("prompt-derived")).toBe(false);+ // Absence of the key - every pre-stamp-era record - marks nothing.+ expect(treeCarriesPromptDerived({})).toBe(false);+ });+});diff --git a/tests/core/content-class-knowledge-pack.test.ts b/tests/core/content-class-knowledge-pack.test.tsnew file mode 100644index 00000000..95215031--- /dev/null+++ b/tests/core/content-class-knowledge-pack.test.ts@@ -0,0 +1,132 @@+/**+ * The content class at the knowledge-pack seam (F5, tasks E4 + E5).+ *+ * A knowledge pack is a subset EXPORT built to be handed to someone else,+ * and it is the one export that filters by LABEL before it composes: the+ * selection step blocks private entries and names them instead of writing+ * them. The reserved `content_class: prompt-derived` stamp joins that+ * block list beside `visibility` and `owner` - agent-produced material is+ * default-excluded from a pack the same as it is from every other shared+ * export, and the exclusion is named per entry.+ *+ * The no-backfill era boundary lives here too: pages written before the+ * class shipped carry no key, nothing rewrites them, and absence reads as+ * unmarked - a pre-stamp page exports exactly as it did before this+ * release.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { dirname, join } from "node:path";++import {+ CONTENT_CLASS,+ CONTENT_CLASS_FIELD,+ stampPromptDerivedLine,+} from "../../src/core/brain/provenance/content-class.ts";+import { selectKnowledgePack } from "../../src/core/brain/portability/knowledge-pack.ts";+import { brainDirs } from "../../src/core/brain/paths.ts";++let src: string;++beforeEach(() => {+ src = mkdtempSync(join(tmpdir(), "o2b-kpack-class-"));+ mkdirSync(brainDirs(src).preferences, { recursive: true });+ mkdirSync(brainDirs(src).log, { recursive: true });+});++afterEach(() => rmSync(src, { recursive: true, force: true }));++function page(rel: string, frontmatter: ReadonlyArray<string>, body: string): void {+ const abs = join(src, rel);+ mkdirSync(dirname(abs), { recursive: true });+ const lines = frontmatter.length > 0 ? ["---", ...frontmatter, "---", ""] : [];+ writeFileSync(abs, `${lines.join("\n")}\n${body}`, "utf8");+}++describe("the knowledge-pack block list", () => {+ test("a prompt-derived page is blocked and named, beside visibility and owner", () => {+ page(+ "Notes/agent-draft.md",+ ["tags: [ops]", `${CONTENT_CLASS_FIELD}: ${CONTENT_CLASS.promptDerived}`],+ "Synthesised from a prompt.\n",+ );+ page("Notes/handbook.md", ["tags: [ops]"], "Written by the operator.\n");++ const selected = selectKnowledgePack(src, ["tag:ops"]);+ expect(selected.blocked).toEqual([+ { kind: "page", id: "Notes/agent-draft.md", reason: "prompt-derived" },+ ]);+ // The bundle renders an index beside each carried page; the marked+ // page's file is absent entirely.+ expect(selected.pageFiles.some((f) => f.path.endsWith("agent-draft.md"))).toBe(false);+ expect(selected.pageFiles.some((f) => f.path.endsWith("handbook.md"))).toBe(true);+ });++ test("the block reason survives the guard's narrowing shape", () => {+ // The class a hand editor wrote that this build does not know reads as+ // unmarked - a typo must not silently hide pages from every shared+ // export - so only the reserved member blocks.+ page(+ "Notes/future-class.md",+ ["tags: [ops]", `${CONTENT_CLASS_FIELD}: agent-summary`],+ "A spelling this build does not know.\n",+ );+ page("Notes/handbook.md", ["tags: [ops]"], "Written by the operator.\n");++ const selected = selectKnowledgePack(src, ["tag:ops"]);+ expect(selected.blocked).toEqual([]);+ expect(selected.pageFiles.some((f) => f.path.endsWith("future-class.md"))).toBe(true);+ expect(selected.pageFiles.some((f) => f.path.endsWith("handbook.md"))).toBe(true);+ });++ test("a visibility-scoped page keeps its own reason, not the class one", () => {+ // First match wins, and the operator-set scope is the older, more+ // specific decision; the class check never displaces it.+ page(+ "Notes/scoped.md",+ [+ "tags: [ops]",+ "visibility: private",+ `${CONTENT_CLASS_FIELD}: ${CONTENT_CLASS.promptDerived}`,+ ],+ "Scoped AND agent-produced.\n",+ );+ page("Notes/handbook.md", ["tags: [ops]"], "Written by the operator.\n");++ const selected = selectKnowledgePack(src, ["tag:ops"]);+ expect(selected.blocked).toEqual([+ { kind: "page", id: "Notes/scoped.md", reason: "visibility" },+ ]);+ });+});++describe("the no-backfill era boundary", () => {+ test("a pre-stamp page exports unchanged; absence of the mark is the documented era", () => {+ page("Notes/runbook.md", ["tags: [ops]"], "Written before the class shipped.\n");++ const selected = selectKnowledgePack(src, ["tag:ops"]);+ expect(selected.blocked).toEqual([]);+ // The carried page file (the bundle also renders an index.md for it).+ const packed = selected.pageFiles.find((f) => f.path.endsWith("runbook.md"))!;+ expect(packed).toBeDefined();+ // The packed page carries no content_class key: the export path adds+ // nothing, and the pre-stamp era is documented rather than migrated.+ expect(packed.contents).toContain("Written before the class shipped.");+ expect(packed.contents).not.toContain(CONTENT_CLASS_FIELD);+ });++ test("no surface of the stamp rewrites an existing unmarked page", () => {+ // The stamp is creation-time only. Reading the class back, resolving+ // it and stripping a caller-named one are all pure or read-only; the+ // one writer among them (the line stamp) refuses a file without+ // frontmatter rather than inventing one. There is no backfill entry+ // point: nothing here takes a vault and adds the key to what it finds.+ const abs = join(src, "Notes/runbook.md");+ mkdirSync(dirname(abs), { recursive: true });+ writeFileSync(abs, "no frontmatter at all\n", "utf8");+ expect(() => stampPromptDerivedLine(abs)).toThrow(/no frontmatter block to stamp/);+ expect(readFileSync(abs, "utf8")).toBe("no frontmatter at all\n");+ });+});diff --git a/tests/core/doctor-consent-probe.test.ts b/tests/core/doctor-consent-probe.test.tsnew file mode 100644index 00000000..60bfe35e--- /dev/null+++ b/tests/core/doctor-consent-probe.test.ts@@ -0,0 +1,161 @@+/**+ * The readiness probe that names the provider consent+ * (write-honesty-egress-consent, Lane F).+ *+ * Before this probe, an operator who wrote `search_semantic_enabled:+ * false` and an operator who never configured semantic search at all+ * read the same sentence on the same surface, and neither of them was+ * told what was in their environment. The probe splits those two+ * situations: a recorded decline is named as a decline, with the two+ * spellings it can be written in, and the provider environment variables+ * that are present are listed.+ *+ * BY NAME, never by value: a readiness detail travels into logs, into+ * the JSON payload of `o2b doctor --json --readiness` and into whatever+ * an operator pastes into a bug report. The file asserts the absence of+ * the value in the rendered JSON, not just in the detail string.+ *+ * No test here drops permission bits and none of them reaches the+ * network, so the file runs the same on every platform.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import {+ DEFAULT_PROBES,+ READINESS_PROBE,+ probeEmbeddingProvider,+ probeLlmKey,+ probeProviderConsent,+ runReadinessProbes,+} from "../../src/core/doctor-readiness.ts";++/** The environment variable whose NAME may be reported and whose VALUE may not. */+const PROVIDER_KEY_ENV = "OPEN_SECOND_BRAIN_EMBEDDING_KEY";++/**+ * A value that stands in for a credential. Assembled at runtime so no+ * source line carries a literal shaped like one, and named `PRIVATE_BODY`+ * so no identifier near it reads as a credential either.+ */+const PRIVATE_BODY = ["PRIVATE", "BODY"].join("-");++let tmp: string;+let configPath: string;++beforeEach(() => {+ tmp = mkdtempSync(join(tmpdir(), "o2b-consent-probe-"));+ configPath = join(tmp, "config.yaml");+});++afterEach(() => {+ rmSync(tmp, { recursive: true, force: true });+});++/** Write the machine configuration the probes read. */+function writeConfig(body: string): void {+ writeFileSync(configPath, `vault: "${tmp}"\n${body}`);+}++/** The probe options every case shares. */+function probeOpts(env: Record<string, string | undefined> = {}) {+ return { vault: tmp, config: configPath, env };+}++describe("the provider-consent readiness probe", () => {+ test("is registered in the probe registry and in the default set", () => {+ expect(READINESS_PROBE.providerConsent).toBe("provider_consent");+ const registered = DEFAULT_PROBES.map((probe) => probe.name);+ expect(registered).toContain(READINESS_PROBE.providerConsent);+ });++ test("a declined vault is named as a decline, with both spellings", async () => {+ writeConfig("search_semantic_enabled: false\n");+ const verdict = await probeProviderConsent(probeOpts());+ expect(verdict.status).toBe("skipped");+ expect(verdict.detail).toContain("search_semantic_enabled=false");+ expect(verdict.detail).toContain("embedding_provider=disabled");+ });++ test("the present provider environment variables are listed BY NAME", async () => {+ writeConfig("search_semantic_enabled: false\n");+ const verdict = await probeProviderConsent(probeOpts({ [PROVIDER_KEY_ENV]: PRIVATE_BODY }));+ expect(verdict.detail).toContain(PROVIDER_KEY_ENV);+ // …and the VALUE never appears, in the detail or anywhere else.+ expect(verdict.detail).not.toContain(PRIVATE_BODY);+ });++ test("a configuration that records no decision says so, without a decline", async () => {+ // No body at all: writeConfig already prepends the vault line, and a+ // second one would be a duplicate key the config reader never wrote.+ writeConfig("");+ const verdict = await probeProviderConsent(probeOpts());+ expect(verdict.status).toBe("skipped");+ expect(verdict.detail).not.toContain("search_semantic_enabled=false");+ expect(verdict.detail).not.toContain("embedding_provider=disabled");+ });++ test("the value of a present environment variable never reaches the JSON output", async () => {+ writeConfig("search_semantic_enabled: false\n");+ const report = await runReadinessProbes(probeOpts({ [PROVIDER_KEY_ENV]: PRIVATE_BODY }));+ const json = JSON.stringify(report);+ expect(json).toContain(PROVIDER_KEY_ENV);+ expect(json).not.toContain(PRIVATE_BODY);+ });++ test("every probe in the default set carries a non-empty detail", async () => {+ writeConfig("search_semantic_enabled: false\n");+ const report = await runReadinessProbes(probeOpts());+ for (const probe of report.probes) {+ expect(`${probe.name}: ${probe.detail.trim().length > 0}`).toBe(`${probe.name}: true`);+ }+ // The default entry point really consumes the default set: the report+ // the CLI's `--readiness` flag runs through lists exactly the default+ // probes, in order - a probe dropped from ( or reordered out of ) the+ // run fails here rather than vanishing silently.+ expect(report.probes.map((probe) => probe.name)).toEqual(+ DEFAULT_PROBES.map((probe) => probe.name),+ );+ });+});++describe("the skipped arm of the key probe is split", () => {+ test("a declined vault is reported as a decline, not as a bare 'disabled'", async () => {+ writeConfig("search_semantic_enabled: false\n");+ const verdict = await probeLlmKey(probeOpts());+ expect(verdict.status).toBe("skipped");+ expect(verdict.detail).toContain("provider consent declined");+ expect(verdict.detail).not.toBe("semantic search disabled; no model API key required");+ });++ test("a configuration that records no decision keeps the existing sentence", async () => {+ writeConfig("");+ const verdict = await probeLlmKey(probeOpts());+ expect(verdict.status).toBe("skipped");+ // Byte-identical to the sentence this probe has always reported.+ expect(verdict.detail).toBe("semantic search disabled; no model API key required");+ });+});++describe("the embedding probe reports a decline as skipped", () => {+ test("a declined vault reports skipped with the decline sentence", async () => {+ writeConfig("search_semantic_enabled: false\n");+ const verdict = await probeEmbeddingProvider(probeOpts());+ expect(verdict.status).toBe("skipped");+ expect(verdict.detail).toContain("provider consent declined");+ expect(verdict.detail).toContain("search_semantic_enabled=false");+ });++ test("a genuine provider load failure still reports fail", async () => {+ writeConfig(+ "search_semantic_enabled: true\nembedding_provider: openai-compat\nembedding_api_key: fake-key\n",+ );+ const verdict = await probeEmbeddingProvider(probeOpts());+ expect(verdict.status).toBe("fail");+ expect(verdict.detail).toContain("provider failed to load");+ expect(verdict.detail).toContain("embedding_base_url is required");+ });+});diff --git a/tests/core/egress-consent-stamp.test.ts b/tests/core/egress-consent-stamp.test.tsnew file mode 100644index 00000000..884b3911--- /dev/null+++ b/tests/core/egress-consent-stamp.test.ts@@ -0,0 +1,193 @@+/**+ * The egress consent stamp (write-honesty-egress-consent, Lane F).+ *+ * A stamp is a record of WHAT WOULD LEAVE the machine, made before+ * anything leaves. Two properties are asserted here and they are the+ * whole point of the primitive:+ *+ * 1. THE KILL SWITCH WINS. A machine whose operator set the kill+ * switch refuses every stamp, whatever any other environment value+ * says. There is no environment value that re-enables it, because a+ * kill switch that could be talked out of by the next variable in+ * the environment would not be one.+ * 2. A REFUSAL DISCLOSES THE PAYLOAD CLASS. An operator who is told+ * "egress refused" learns nothing; an operator who is told "page+ * bodies were withheld" can go and look at the thing that was+ * withheld.+ *+ * The stamp is also where the redaction happens: a stamped payload has+ * been through the shared egress guard, so what the stamp carries is+ * what would ACTUALLY leave, not what the caller meant to send. No+ * egress destination is declared in this release, so the primitive+ * declares none: the caller names the site it is stamping for.+ *+ * Every credential-shaped value below is assembled at runtime by the+ * fake-credentials helper, so no source line carries a literal shaped+ * like a real one.+ */++import { describe, expect, test } from "bun:test";++import {+ CONSENT_STAMP_ENV,+ CONSENT_STAMP_GRANT_VALUES,+ CONSENT_STAMP_KILL_ENV,+ CONSENT_STAMP_OUTCOME,+ stampEgressConsent,+} from "../../src/core/egress/consent-stamp.ts";+import { REDACTION_PLACEHOLDER } from "../../src/core/redactor.ts";+import { FAKE_VENDOR_KEY } from "../helpers/fake-credentials.ts";++/** The class of content this fixture offers to the boundary. */+const PAYLOAD_CLASS = "vault page bodies";++/** The site the stamp is for. No destination is declared in this release. */+const SITE = "brain-export";++describe("the egress consent stamp", () => {+ test("the kill switch refuses every stamp, whatever the grant says", () => {+ // The matrix the brief names: the kill switch set, plus EVERY value+ // that would otherwise grant consent.+ for (const grant of CONSENT_STAMP_GRANT_VALUES) {+ const verdict = stampEgressConsent({+ site: SITE,+ payloadClass: PAYLOAD_CLASS,+ payload: { body: "a page body" },+ env: { [CONSENT_STAMP_ENV]: grant, [CONSENT_STAMP_KILL_ENV]: "1" },+ });+ expect(`${grant}: ${verdict.outcome}`).toBe(`${grant}: ${CONSENT_STAMP_OUTCOME.withheld}`);+ }+ });++ test("the kill switch refuses even when the grant is absent", () => {+ const verdict = stampEgressConsent({+ site: SITE,+ payloadClass: PAYLOAD_CLASS,+ payload: { body: "a page body" },+ env: { [CONSENT_STAMP_KILL_ENV]: "1" },+ });+ expect(verdict.outcome).toBe(CONSENT_STAMP_OUTCOME.withheld);+ });++ test("the kill switch is a presence switch: no value re-enables it", () => {+ // Every non-empty value keeps the refusal in force, including the+ // ones that would read as "off" to a boolean parser and the ones+ // that are only whitespace - the raw value is what the operator set,+ // and nothing trims it away. An operator who has set the variable+ // has said no, and the next variable in the environment does not get+ // to say otherwise.+ for (const value of ["1", "true", "0", "false", "off", "please", " "]) {+ const verdict = stampEgressConsent({+ site: SITE,+ payloadClass: PAYLOAD_CLASS,+ payload: { body: "a page body" },+ env: { [CONSENT_STAMP_ENV]: "true", [CONSENT_STAMP_KILL_ENV]: value },+ });+ expect(`${value}: ${verdict.outcome}`).toBe(`${value}: ${CONSENT_STAMP_OUTCOME.withheld}`);+ }+ });++ test("a whitespace-only kill-switch value still refuses", () => {+ // Presence, not parse: the value is non-empty, so the switch is in+ // force even though trimming it would read as unset - the same+ // untrimmed presence rule the grant check applies to its own value.+ const verdict = stampEgressConsent({+ site: SITE,+ payloadClass: PAYLOAD_CLASS,+ payload: { body: "a page body" },+ env: { [CONSENT_STAMP_ENV]: "true", [CONSENT_STAMP_KILL_ENV]: " " },+ });+ expect(verdict.outcome).toBe(CONSENT_STAMP_OUTCOME.withheld);+ if (verdict.outcome !== CONSENT_STAMP_OUTCOME.withheld) return;+ expect(verdict.reason).toContain("kill switch");+ });++ test("a refusal discloses the payload class", () => {+ const verdict = stampEgressConsent({+ site: SITE,+ payloadClass: PAYLOAD_CLASS,+ payload: { body: "a page body" },+ env: { [CONSENT_STAMP_KILL_ENV]: "1" },+ });+ expect(verdict.outcome).toBe(CONSENT_STAMP_OUTCOME.withheld);+ if (verdict.outcome !== CONSENT_STAMP_OUTCOME.withheld) return;+ expect(verdict.detail).toContain(PAYLOAD_CLASS);+ expect(verdict.reason).toContain(PAYLOAD_CLASS);+ expect(verdict.detail.trim().length).toBeGreaterThan(0);+ });++ test("a grant with no kill switch issues a stamp", () => {+ const verdict = stampEgressConsent({+ site: SITE,+ payloadClass: PAYLOAD_CLASS,+ payload: { body: "a page body" },+ env: { [CONSENT_STAMP_ENV]: "true" },+ });+ expect(verdict.outcome).toBe(CONSENT_STAMP_OUTCOME.stamped);+ if (verdict.outcome !== CONSENT_STAMP_OUTCOME.stamped) return;+ expect(verdict.payload).toEqual({ body: "a page body" });+ expect(verdict.redacted).toBe(false);+ });++ test("no grant is a refusal, not a silent stamp", () => {+ const verdict = stampEgressConsent({+ site: SITE,+ payloadClass: PAYLOAD_CLASS,+ payload: { body: "a page body" },+ env: {},+ });+ expect(verdict.outcome).toBe(CONSENT_STAMP_OUTCOME.withheld);+ if (verdict.outcome !== CONSENT_STAMP_OUTCOME.withheld) return;+ expect(verdict.reason).toContain(CONSENT_STAMP_ENV);+ });++ test("a set-but-unrecognized grant value is named, not told to set what is already set", () => {+ // "no egress consent is recorded; set X to grant it" is a false+ // sentence when the variable IS set: the operator would re-run the+ // instruction that already failed. The refusal names the value as+ // the problem and lists what grants.+ const verdict = stampEgressConsent({+ site: SITE,+ payloadClass: PAYLOAD_CLASS,+ payload: { body: "a page body" },+ env: { [CONSENT_STAMP_ENV]: "yes" },+ });+ expect(verdict.outcome).toBe(CONSENT_STAMP_OUTCOME.withheld);+ if (verdict.outcome !== CONSENT_STAMP_OUTCOME.withheld) return;+ expect(verdict.reason).toContain("unrecognized");+ expect(verdict.reason).toContain(CONSENT_STAMP_ENV);+ for (const grant of CONSENT_STAMP_GRANT_VALUES) {+ expect(verdict.reason).toContain(grant);+ }+ });++ test("the stamped payload is the redacted one: what would leave, not what was meant to", () => {+ const verdict = stampEgressConsent({+ site: SITE,+ payloadClass: PAYLOAD_CLASS,+ payload: { body: `the key is ${FAKE_VENDOR_KEY}` },+ env: { [CONSENT_STAMP_ENV]: "true" },+ });+ expect(verdict.outcome).toBe(CONSENT_STAMP_OUTCOME.stamped);+ if (verdict.outcome !== CONSENT_STAMP_OUTCOME.stamped) return;+ expect(verdict.redacted).toBe(true);+ expect(JSON.stringify(verdict.payload)).not.toContain(FAKE_VENDOR_KEY);+ expect(JSON.stringify(verdict.payload)).toContain(REDACTION_PLACEHOLDER);+ });++ test("every refusal carries a non-empty reason and discloses the class", () => {+ const cases = [{ [CONSENT_STAMP_KILL_ENV]: "1" }, {}, { [CONSENT_STAMP_ENV]: "maybe" }];+ for (const env of cases) {+ const verdict = stampEgressConsent({+ site: SITE,+ payloadClass: PAYLOAD_CLASS,+ payload: { body: "a page body" },+ env,+ });+ expect(verdict.outcome).toBe(CONSENT_STAMP_OUTCOME.withheld);+ if (verdict.outcome !== CONSENT_STAMP_OUTCOME.withheld) return;+ expect(verdict.reason.trim().length).toBeGreaterThan(0);+ expect(verdict.reason).toContain(PAYLOAD_CLASS);+ }+ });+});diff --git a/tests/core/graph/privacy-markers.test.ts b/tests/core/graph/privacy-markers.test.tsnew file mode 100644index 00000000..8505fc98--- /dev/null+++ b/tests/core/graph/privacy-markers.test.ts@@ -0,0 +1,194 @@+/**+ * Informal privacy markers (write-honesty, F1 unit 1).+ *+ * A vault written before `visibility:` was the only reservation spelling+ * tends to say `access: private`, `confidentiality: secret` or+ * `sensitivity: confidential` instead. The visibility layer reads only the+ * `visibility` key (see `graph/visibility.ts`), so those pages resolve to+ * default visibility and are handed to remote readers.+ *+ * This module is the predicate that recognises the informal spelling. It+ * answers one question - does this frontmatter contain a privacy-denying+ * marker value - and returns the reserved token to union into the+ * `visibility:` set. It deliberately does NOT widen `pageVisibility` or+ * `isVisible`: an informal marker is not a caller-scope token, so a caller+ * asking for the `private` scope must not lift it. Enforcement happens at+ * the read roots (`result-filters.ts`, `vault.ts`) and in the indexed+ * visibility column.+ */++import { describe, expect, test } from "bun:test";++import {+ PRIVACY_DENY_MARKER_VALUES,+ PRIVACY_MARKER_KEYS,+ hasPrivacyDenyMarker,+ markerDenyTokens,+ remoteDenyVisibilityTokens,+} from "../../../src/core/graph/privacy-markers.ts";+import type { FrontmatterMap } from "../../../src/core/types.ts";+import { TRANSPORT_REACH } from "../../../src/core/graph/transport-reach.ts";+import {+ REMOTE_DENY_VISIBILITY_TOKEN,+ isRemotelyReadable,+ isVisible,+ normalizeVisibilityScope,+ normToken,+ pageVisibility,+} from "../../../src/core/graph/visibility.ts";++describe("the marker vocabulary", () => {+ test("names the three informal keys and the three deny values", () => {+ expect([...PRIVACY_MARKER_KEYS]).toEqual(["access", "confidentiality", "sensitivity"]);+ expect([...PRIVACY_DENY_MARKER_VALUES]).toEqual(["private", "secret", "confidential"]);+ });++ test("every deny value is already in the normal form the predicate compares", () => {+ for (const value of PRIVACY_DENY_MARKER_VALUES) expect(normToken(value)).toBe(value);+ });+});++describe("pages with no deny marker are unchanged", () => {+ test("keyless frontmatter declares nothing", () => {+ expect(hasPrivacyDenyMarker({})).toBe(false);+ expect(markerDenyTokens({})).toEqual([]);+ expect(remoteDenyVisibilityTokens({})).toEqual([]);+ });++ test("only-formal visibility is left exactly as it was", () => {+ const meta = { visibility: ["team", "ops"] };+ expect(hasPrivacyDenyMarker(meta)).toBe(false);+ expect(markerDenyTokens(meta)).toEqual([]);+ expect(remoteDenyVisibilityTokens(meta)).toEqual(["team", "ops"]);+ });++ test("a formal reservation is already the reserved token, not a marker", () => {+ const meta = { visibility: [REMOTE_DENY_VISIBILITY_TOKEN] };+ expect(hasPrivacyDenyMarker(meta)).toBe(false);+ expect(remoteDenyVisibilityTokens(meta)).toEqual([REMOTE_DENY_VISIBILITY_TOKEN]);+ });++ test("other keys carrying the same words are not privacy markers", () => {+ expect(hasPrivacyDenyMarker({ topic: "private" })).toBe(false);+ expect(hasPrivacyDenyMarker({ status: "confidential" })).toBe(false);+ expect(markerDenyTokens({ title: "secret" })).toEqual([]);+ });+});++describe("privacy-denying marker values are caught", () => {+ const cases: ReadonlyArray<{+ readonly name: string;+ readonly meta: FrontmatterMap;+ }> = [+ { name: "access: private", meta: { access: "private" } },+ { name: "access: secret", meta: { access: "secret" } },+ { name: "access: confidential", meta: { access: "confidential" } },+ { name: "confidentiality: private", meta: { confidentiality: "private" } },+ { name: "confidentiality: secret", meta: { confidentiality: "secret" } },+ { name: "confidentiality: confidential", meta: { confidentiality: "confidential" } },+ { name: "sensitivity: private", meta: { sensitivity: "private" } },+ { name: "sensitivity: secret", meta: { sensitivity: "secret" } },+ { name: "sensitivity: confidential", meta: { sensitivity: "confidential" } },+ { name: "a single-element array", meta: { access: ["private"] } },+ { name: "an array beside a non-deny value", meta: { sensitivity: ["internal", "secret"] } },+ ];++ for (const c of cases) {+ test(`${c.name} denies remote reads`, () => {+ expect(hasPrivacyDenyMarker(c.meta)).toBe(true);+ expect(markerDenyTokens(c.meta)).toEqual([REMOTE_DENY_VISIBILITY_TOKEN]);+ });+ }+});++describe("normalisation is the visibility module's own", () => {+ test("case and surrounding whitespace variants are the same value", () => {+ expect(hasPrivacyDenyMarker({ access: " PRIVATE " })).toBe(true);+ expect(hasPrivacyDenyMarker({ confidentiality: "Secret" })).toBe(true);+ expect(hasPrivacyDenyMarker({ sensitivity: ["CONFIDENTIAL"] })).toBe(true);+ });++ test("comparison happens after NFC composition, not by stripping marks", () => {+ // `private` + a combining acute composes to a DIFFERENT token under+ // NFC. A normaliser that stripped the mark would answer "private".+ const composed = `private${String.fromCodePoint(0x0301)}`;+ expect(normToken(composed)).not.toBe("private");+ expect(hasPrivacyDenyMarker({ access: composed })).toBe(false);+ // The same value already in NFC keeps its own spelling.+ expect(hasPrivacyDenyMarker({ access: composed.normalize("NFC") })).toBe(false);+ });++ test("an NFC value equal to a deny value is caught", () => {+ expect(hasPrivacyDenyMarker({ access: "private".normalize("NFC") })).toBe(true);+ });+});++describe("non-privacy values are not denied", () => {+ for (const value of ["public", "shared", "internal", "team", "draft"]) {+ test(`access: ${value} keeps the page reachable`, () => {+ expect(hasPrivacyDenyMarker({ access: value })).toBe(false);+ expect(markerDenyTokens({ access: value })).toEqual([]);+ });+ }++ test("a non-marker scalar type declares nothing", () => {+ expect(hasPrivacyDenyMarker({ access: 42 })).toBe(false);+ expect(hasPrivacyDenyMarker({ access: true })).toBe(false);+ });++ test("an empty string and an empty array declare nothing", () => {+ expect(hasPrivacyDenyMarker({ access: "" })).toBe(false);+ expect(hasPrivacyDenyMarker({ access: [] })).toBe(false);+ });+});++describe("the marker is not a caller-scope token", () => {+ test("pageVisibility does not see the informal key", () => {+ const meta = { access: "private" };+ expect(pageVisibility(meta)).toEqual([]);+ });++ test("a caller asking for the reserved scope does not lift a marker page", () => {+ const tags = pageVisibility({ access: "private" });+ // isVisible answers the caller-scope question only: the marker is not+ // in the tag list, so it cannot be lifted by naming it.+ expect(isVisible(tags, normalizeVisibilityScope([]))).toBe(true);+ expect(isVisible(tags, normalizeVisibilityScope(["private"]))).toBe(true);+ // And isRemotelyReadable is unchanged by the marker: the read roots,+ // not this pure predicate, are where the marker denies.+ expect(isRemotelyReadable(tags, TRANSPORT_REACH.remote)).toBe(true);+ });+});++describe("the tokens to union with the declared visibility", () => {+ test("the reserved token is appended after the declared tokens", () => {+ expect(remoteDenyVisibilityTokens({ access: "secret", visibility: ["team"] })).toEqual([+ "team",+ REMOTE_DENY_VISIBILITY_TOKEN,+ ]);+ });++ test("a declared token is never dropped", () => {+ const tokens = remoteDenyVisibilityTokens({+ confidentiality: "confidential",+ visibility: ["agent:foo", "team"],+ });+ expect(tokens).toEqual(["agent:foo", "team", REMOTE_DENY_VISIBILITY_TOKEN]);+ });++ test("a declaration already naming the reserved token is not duplicated", () => {+ expect(+ remoteDenyVisibilityTokens({+ sensitivity: "private",+ visibility: ["team", REMOTE_DENY_VISIBILITY_TOKEN],+ }),+ ).toEqual(["team", REMOTE_DENY_VISIBILITY_TOKEN]);+ });++ test("a string declaration is normalised like an array declaration", () => {+ expect(remoteDenyVisibilityTokens({ access: "private", visibility: "Team" })).toEqual([+ "team",+ REMOTE_DENY_VISIBILITY_TOKEN,+ ]);+ });+});diff --git a/tests/core/graph/remote-readability.test.ts b/tests/core/graph/remote-readability.test.tsindex c795db3f..281e8487 100644--- a/tests/core/graph/remote-readability.test.ts+++ b/tests/core/graph/remote-readability.test.ts@@ -12,7 +12,10 @@* natural-language phrase, and not a closed list of them.*/-import { describe, expect, test } from "bun:test";+import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";import {REMOTE_DENY_VISIBILITY_TOKEN,@@ -26,6 +29,8 @@ import {TRANSPORT_REACHES,isTransportReach,} from "../../../src/core/graph/transport-reach.ts";+import { isPathReadableAtReach } from "../../../src/core/search/result-filters.ts";+import { listVaultPages } from "../../../src/core/vault.ts";/** A token that is NOT the reserved one, for the "unaffected" arm. */const ORDINARY_TOKEN = "team";@@ -112,3 +117,81 @@ describe("the predicate is independent of the caller's requested scope", () => {expect(isVisible(tags, normalizeVisibilityScope([ORDINARY_TOKEN]))).toBe(true);});});++// --- The informal marker at the two path roots -------------------------------+//+// `access: private` and its siblings are not visibility tokens (see+// graph/privacy-markers.ts): no caller scope can name them, and they never+// enter `pageVisibility`. What they do is reserve the page against a remote+// read, at the same two roots that enforce the reserved token - the by-path+// verdict (`isPathReadableAtReach`, root A/C) and the vault walk+// (`listVaultPages`, root B).++describe("an informal privacy marker at the read roots", () => {+ let vault: string;++ beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "remote-readability-"));+ writeFileSync(join(vault, "open.md"), "---\ntitle: Open\n---\n\nordinary body");+ writeFileSync(+ join(vault, "access.md"),+ "---\ntitle: Access\ntags: [imported]\naccess: private\n---\n\nmarked body",+ );+ writeFileSync(+ join(vault, "confidentiality.md"),+ "---\ntitle: Confidentiality\nconfidentiality: [Secret]\n---\n\nmarked body",+ );+ writeFileSync(+ join(vault, "sensitivity.md"),+ "---\ntitle: Sensitivity\nsensitivity: confidential\nvisibility: [team]\n---\n\nmarked body",+ );+ });+ afterEach(() => rmSync(vault, { recursive: true, force: true }));++ const readable = (rel: string, reach: (typeof TRANSPORT_REACH)[keyof typeof TRANSPORT_REACH]) =>+ isPathReadableAtReach(vault, rel, reach, new Map());++ test("a marker-only page is withheld by path at remote reach", () => {+ expect(readable("access.md", TRANSPORT_REACH.remote)).toBe(false);+ expect(readable("confidentiality.md", TRANSPORT_REACH.remote)).toBe(false);+ });++ test("a marker-only page stays readable by path at local reach", () => {+ // Local reach is byte-identical to before the marker rule: the+ // operator can open the file directly anyway.+ for (const rel of ["open.md", "access.md", "confidentiality.md", "sensitivity.md"]) {+ expect(readable(rel, TRANSPORT_REACH.local)).toBe(true);+ }+ });++ test("a marker beside a caller-liftable token is still remote-denied", () => {+ // `visibility: [team]` would be readable at remote reach on its own;+ // the marker dominates, exactly as the reserved token does.+ expect(readable("sensitivity.md", TRANSPORT_REACH.remote)).toBe(false);+ });++ test("an unmarked non-reserved page keeps today's verdict at both reaches", () => {+ expect(readable("open.md", TRANSPORT_REACH.remote)).toBe(true);+ expect(readable("open.md", TRANSPORT_REACH.local)).toBe(true);+ });++ test("the indexed tokens are honoured beside a marker in the file", () => {+ // Either measurement reserving the page withholds it; the index+ // decides for bytes captured while reserved.+ expect(+ isPathReadableAtReach(vault, "open.md", TRANSPORT_REACH.remote, new Map(), [+ REMOTE_DENY_VISIBILITY_TOKEN,+ ]),+ ).toBe(false);+ });++ test("listVaultPages withholds every marker page at remote reach", () => {+ const titles = listVaultPages(vault, { reach: TRANSPORT_REACH.remote }).map((p) => p.title);+ expect(titles.toSorted()).toEqual(["Open"]);+ });++ test("listVaultPages keeps every page at local reach, marker or not", () => {+ const titles = listVaultPages(vault, { reach: TRANSPORT_REACH.local }).map((p) => p.title);+ expect(titles.toSorted()).toEqual(["Access", "Confidentiality", "Open", "Sensitivity"]);+ });+});diff --git a/tests/core/search/chunk-hold-persistence.test.ts b/tests/core/search/chunk-hold-persistence.test.tsnew file mode 100644index 00000000..6fe2a3c1--- /dev/null+++ b/tests/core/search/chunk-hold-persistence.test.ts@@ -0,0 +1,373 @@+/**+ * Chunk skip-and-hold, persistence and the pending exclusion (Lane B,+ * task B2): the hold a phase decides is a RECORD, not just a filter.+ *+ * The point of the persistent hold is that the index stops OWING the+ * provably-over chunk: it leaves the pending-vector anti-join, so the+ * status refresh estimate, the backfill dry run and the spend preview all+ * stop pricing work the run will refuse to spend anyway. The exclusion is+ * keyed by (model, window_tokens, embedding_signature): a window or model+ * change makes the stored key not match, which RELEASES the hold back+ * into pending with no migration and no delete pass - the next phase+ * decides fresh, and either embeds (releasing) or re-records (keeping).+ *+ * The release direction is what makes the key honest. An unchanged config+ * must keep the chunk held without any new provider contact - the second+ * run's census no longer even contains it.+ */++import { afterEach, beforeEach, expect, test } from "bun:test";+import { Database } from "bun:sqlite";++import { runEmbeddingPhase, type EmbeddingPhaseTally } from "../../../src/core/search/indexer.ts";+import { estimatePendingEmbeddingSpend } from "../../../src/core/search/indexer.ts";+import { Store } from "../../../src/core/search/store.ts";+import {+ chunkHoldExclusion,+ planEmbeddingSpend,+} from "../../../src/core/search/embedding-spend.ts";+import type { ChunkHoldExclusion } from "../../../src/core/search/store/chunks.ts";+import {+ declaredInputWindowTokens,+ RECOMMENDED_EMBEDDING_MODEL,+} from "../../../src/core/search/embeddings/presets.ts";+import {+ charLengthOverTokenBudget,+ embeddingSignature,+} from "../../../src/core/search/embeddings/signature.ts";+import type {+ ResolvedEmbeddingConfig,+ ResolvedSearchConfig,+} from "../../../src/core/search/types.ts";+import { createTempVault, makeConfig } from "../../helpers/search-fixtures.ts";+import { startFakeHttp, type FakeHttp } from "../../helpers/fake-http.ts";+import { FAKE_PROVIDER_KEY } from "../../helpers/fake-credentials.ts";++const WINDOW = declaredInputWindowTokens(RECOMMENDED_EMBEDDING_MODEL)!;++/** The LOW estimate exceeds the window: provably over, so it gets held. */+const OVER_CONTENT = "b".repeat(charLengthOverTokenBudget(WINDOW) + 1);+const OVER_FLOOR = Math.ceil(OVER_CONTENT.length / 4);+const FITTING = "small chunk of ordinary prose";++let vault: string;+let dbPath: string;+let cleanup: () => void;+let server: FakeHttp | null = null;++beforeEach(() => {+ ({ vault, dbPath, cleanup } = createTempVault("chunk-hold-persistence"));+});++afterEach(async () => {+ await server?.close();+ server = null;+ cleanup();+});++async function startProvider(): Promise<void> {+ server = await startFakeHttp();+}++function cfg(semantic: Partial<ResolvedEmbeddingConfig> = {}): ResolvedSearchConfig {+ return makeConfig({+ vault,+ dbPath,+ semantic: {+ enabled: true,+ provider: "openai-compat",+ baseUrl: server!.url,+ model: RECOMMENDED_EMBEDDING_MODEL,+ apiKey: FAKE_PROVIDER_KEY,+ ...semantic,+ },+ });+}++async function openSeeded(config: ResolvedSearchConfig, contents: string[]): Promise<Store> {+ const store = await Store.open(config, { mode: "write" });+ const docId = store.upsertDocument({+ path: "Brain/note.md",+ title: "note",+ contentHash: "h1",+ mtime: 1_700_000_000,+ size: 10,+ });+ store.replaceChunks(+ docId,+ contents.map((content, i) => ({+ chunkIndex: i,+ content,+ contentHash: `h1-${i}`,+ startLine: 1,+ endLine: 2,+ tokenCount: content.length,+ })),+ );+ return store;+}++function chunkIdByLength(length: number): number {+ const db = new Database(dbPath, { readonly: true });+ try {+ const row = db+ .query<{ id: number }, [number]>("SELECT id FROM chunks WHERE length(content) = ?")+ .get(length);+ if (!row) throw new Error(`no chunk of length ${length} in the seeded index`);+ return row.id;+ } finally {+ db.close();+ }+}++function embeddedChunkIds(): number[] {+ const db = new Database(dbPath, { readonly: true });+ try {+ return db+ .query<{ chunk_id: number }, []>("SELECT chunk_id FROM embeddings ORDER BY chunk_id")+ .all()+ .map((r) => r.chunk_id);+ } finally {+ db.close();+ }+}++const EMPTY_TALLY = (): EmbeddingPhaseTally => ({ embeddingsComputed: 0, embeddingsRetries: 0 });++/**+ * Seed one provably-over chunk plus one fitting chunk, run the phase to+ * the hold, and hand back the closed vault plus the held chunk's id.+ */+async function runOnceUntilHold(): Promise<{ heldId: number }> {+ await startProvider();+ const config = cfg();+ const store = await openSeeded(config, [OVER_CONTENT, FITTING]);+ try {+ await runEmbeddingPhase(store, config, EMPTY_TALLY(), {});+ } finally {+ await store.close();+ }+ return { heldId: chunkIdByLength(OVER_CONTENT.length) };+}++// ── the hold is a record ─────────────────────────────────────────────────────++test("the phase persists a hold row keyed by model, window and signature", async () => {+ const { heldId } = await runOnceUntilHold();+ const config = cfg();+ const rows = (() => {+ const db = new Database(dbPath, { readonly: true });+ try {+ return db+ .query<+ {+ chunk_id: number;+ model: string;+ window_tokens: number;+ embedding_signature: string;+ estimated_tokens: number;+ held_at: string;+ },+ []+ >("SELECT * FROM chunk_embedding_holds ORDER BY chunk_id")+ .all();+ } finally {+ db.close();+ }+ })();+ expect(rows.length).toBe(1);+ expect(rows[0]!.chunk_id).toBe(heldId);+ expect(rows[0]!.model).toBe(RECOMMENDED_EMBEDDING_MODEL);+ expect(rows[0]!.window_tokens).toBe(WINDOW);+ expect(rows[0]!.embedding_signature).toBe(+ embeddingSignature({+ provider: "openai-compat",+ model: RECOMMENDED_EMBEDDING_MODEL,+ dimension: config.semantic.dimension,+ }),+ );+ expect(rows[0]!.estimated_tokens).toBe(OVER_FLOOR);+ expect(Number.isNaN(Date.parse(rows[0]!.held_at))).toBe(false);+});++test("a held chunk leaves the pending census, the count, the plan and the preview", async () => {+ const { heldId } = await runOnceUntilHold();+ const config = cfg();+ const store = await Store.open(config, { mode: "write" });+ try {+ const hold = chunkHoldExclusion(config);+ expect(hold).not.toBeNull();+ // Config-aware reads exclude the held chunk...+ expect(+ store.findChunksWithoutEmbeddings(undefined, hold ?? undefined).map((p) => p.chunkId),+ ).toEqual([]);+ expect(store.countChunksWithoutEmbeddings(undefined, hold ?? undefined)).toBe(0);+ expect(planEmbeddingSpend(store, config).pending.map((p) => p.chunkId)).toEqual([]);+ // ...and the key-free diagnostic read still sees the missing vector:+ // the chunk genuinely has none, which is a fact the hold does not erase.+ expect(store.countChunksWithoutEmbeddings()).toBe(1);+ } finally {+ await store.close();+ }+ expect(embeddedChunkIds()).not.toContain(heldId);+ // The maintenance preview reads the plan: with nothing pending it+ // cannot spend, exactly as an embedded-clean vault reports.+ expect(await estimatePendingEmbeddingSpend(config)).toBeNull();+});++test("a second run with the same config never re-embeds and never re-spends", async () => {+ await runOnceUntilHold();+ const callsBefore = server!.callCount();+ const config = cfg();+ const store = await Store.open(config, { mode: "write" });+ const tally = EMPTY_TALLY();+ try {+ await runEmbeddingPhase(store, config, tally, {});+ } finally {+ await store.close();+ }+ // The held chunk is gone from the phase's own census: it is not even+ // re-decided, and the tally says nothing was held this run.+ expect(tally.embeddingsHeld).toBeUndefined();+ expect(tally.embeddingsComputed).toBe(0);+ expect(server!.callCount()).toBe(callsBefore);+});++// ── release on key change ────────────────────────────────────────────────────++test("raising the declared window releases the hold and the chunk embeds", async () => {+ const { heldId } = await runOnceUntilHold();+ const wider = cfg({ inputWindowTokens: WINDOW * 4 });+ const store = await Store.open(wider, { mode: "write" });+ try {+ // The stored key (window WINDOW) no longer matches: pending again.+ const hold = chunkHoldExclusion(wider);+ expect(store.countChunksWithoutEmbeddings(undefined, hold ?? undefined)).toBe(1);+ const before = server!.callCount();+ await runEmbeddingPhase(store, wider, EMPTY_TALLY(), {});+ expect(server!.callCount()).toBeGreaterThan(before);+ } finally {+ await store.close();+ }+ expect(embeddedChunkIds()).toContain(heldId);+});++test("a model change releases the hold under the old model's key", async () => {+ const { heldId } = await runOnceUntilHold();+ const other = cfg({ model: "BAAI/bge-m3" });+ const store = await Store.open(other, { mode: "write" });+ try {+ // A model change also trips the pre-existing clear-on-model-change+ // path, so the whole corpus is pending again; counting would not+ // isolate the hold's release. What the hold itself promises is+ // checked by id: the stored row survives under the OLD key and+ // excludes under it, and the NEW key's census contains the chunk.+ const [holdRow] = store.chunkHoldRowsFor(heldId);+ expect(holdRow).toBeDefined();+ const oldKey: ChunkHoldExclusion = {+ model: holdRow!.model,+ windowTokens: holdRow!.windowTokens,+ embeddingSignature: holdRow!.embeddingSignature,+ };+ const pendingUnderOldKey = store+ .findChunksWithoutEmbeddings(undefined, oldKey)+ .map((r) => r.chunkId);+ expect(pendingUnderOldKey).not.toContain(heldId);+ const hold = chunkHoldExclusion(other);+ const pendingUnderNewKey = store+ .findChunksWithoutEmbeddings(undefined, hold ?? undefined)+ .map((r) => r.chunkId);+ expect(pendingUnderNewKey).toContain(heldId);+ } finally {+ await store.close();+ }+ // bge-m3's declared window bounds the proof the other way: under the+ // new model the chunk simply fits, so the releasing run embeds it.+ expect(declaredInputWindowTokens("BAAI/bge-m3")! > OVER_FLOOR).toBe(true);+ const store2 = await Store.open(other, { mode: "write" });+ const tally = EMPTY_TALLY();+ try {+ await runEmbeddingPhase(store2, other, tally, {});+ } finally {+ await store2.close();+ }+ expect(embeddedChunkIds()).toContain(heldId);+ expect(tally.embeddingsHeld).toBeUndefined();+});++test("holding again refreshes the row rather than duplicating it", async () => {+ const { heldId } = await runOnceUntilHold();+ // No release and no re-decide under a new key: this test exercises only+ // the SAME-key path, re-recording the hold under the original+ // (model, embedding signature) key.+ const original = cfg();+ const store = await Store.open(original, { mode: "write" });+ try {+ // Same-key re-decision re-records through the primary key rather than+ // stacking a second row for one chunk.+ store.recordChunkHolds(+ [{ chunkId: heldId, estimatedTokens: OVER_FLOOR, windowTokens: WINDOW }],+ {+ model: RECOMMENDED_EMBEDDING_MODEL,+ embeddingSignature: chunkHoldExclusion(original)!.embeddingSignature,+ },+ );+ } finally {+ await store.close();+ }+ const db = new Database(dbPath, { readonly: true });+ try {+ expect(+ db.query<{ n: number }, []>("SELECT COUNT(*) AS n FROM chunk_embedding_holds").get()?.n,+ ).toBe(1);+ } finally {+ db.close();+ }+});++// ── no proof, no record ──────────────────────────────────────────────────────++test("an undeclared window holds nothing and records nothing", async () => {+ await startProvider();+ const config = cfg({ model: "acme/custom-embed-9000" });+ const store = await openSeeded(config, [OVER_CONTENT]);+ try {+ await runEmbeddingPhase(store, config, EMPTY_TALLY(), {});+ } finally {+ await store.close();+ }+ expect(chunkHoldExclusion(config)).toBeNull();+ const db = new Database(dbPath, { readonly: true });+ try {+ expect(+ db.query<{ n: number }, []>("SELECT COUNT(*) AS n FROM chunk_embedding_holds").get()?.n,+ ).toBe(0);+ } finally {+ db.close();+ }+});++test("deleting the chunk rows drops the hold rows with them", async () => {+ const { heldId } = await runOnceUntilHold();+ // The document's chunk rows die on a re-index that no longer holds+ // them; the FK cascade must take the hold with them. Exercised through+ // the Store connection, which is the one that enforces foreign keys.+ const config = cfg();+ const store = await Store.open(config, { mode: "write" });+ try {+ const docId = (() => {+ const db = new Database(dbPath);+ try {+ return db+ .query<{ document_id: number }, [number]>("SELECT document_id FROM chunks WHERE id = ?")+ .get(heldId)!.document_id;+ } finally {+ db.close();+ }+ })();+ store.replaceChunks(docId, []);+ expect(store.chunkHoldRowsFor(heldId)).toEqual([]);+ } finally {+ await store.close();+ }+});diff --git a/tests/core/search/chunk-hold-report.test.ts b/tests/core/search/chunk-hold-report.test.tsnew file mode 100644index 00000000..e47c2a5b--- /dev/null+++ b/tests/core/search/chunk-hold-report.test.ts@@ -0,0 +1,272 @@+/**+ * Chunk skip-and-hold, the bounded status report (Lane B, task B3).+ *+ * The phase tally says what one run held; the status snapshot must say+ * what the INDEX currently holds, read back from the durable record. Two+ * constraints shape the field:+ *+ * - presence/absence byte discipline (pinned for the census beside it+ * in chunk-window-census.test.ts): a status with nothing held emits+ * the bytes it emitted before this field existed - absent, never an+ * empty list, never a zero;+ * - a bound on the wire: a vault can hold thousands of chunks after a+ * model swap, and the status JSON is read by humans and by agents on+ * a token budget. The list is capped and the total discloses how+ * many the cap left out.+ *+ * Only holds under the CURRENT (model, window, signature) key are+ * reported: a row whose key no longer matches describes a decision the+ * active configuration already released, and reporting it would claim a+ * hold the next run will not honour.+ */++import { afterEach, beforeEach, expect, test } from "bun:test";+import { Database } from "bun:sqlite";++import { indexStatus, HELD_CHUNK_REPORT_CAP } from "../../../src/core/search/indexer.ts";+import { Store } from "../../../src/core/search/store.ts";+import { chunkHoldExclusion } from "../../../src/core/search/embedding-spend.ts";+import { serializeIndexStatus } from "../../../src/core/search/serialize.ts";+import {+ declaredInputWindowTokens,+ RECOMMENDED_EMBEDDING_MODEL,+} from "../../../src/core/search/embeddings/presets.ts";+import type { ChunkHoldExclusion } from "../../../src/core/search/store/chunks.ts";+import type {+ ResolvedEmbeddingConfig,+ ResolvedSearchConfig,+} from "../../../src/core/search/types.ts";+import { createTempVault, makeConfig } from "../../helpers/search-fixtures.ts";+import { FAKE_PROVIDER_KEY } from "../../helpers/fake-credentials.ts";++const WINDOW = declaredInputWindowTokens(RECOMMENDED_EMBEDDING_MODEL)!;++let vault: string;+let dbPath: string;+let cleanup: () => void;++beforeEach(() => {+ ({ vault, dbPath, cleanup } = createTempVault("chunk-hold-report"));+});+afterEach(() => cleanup());++function cfg(semantic: Partial<ResolvedEmbeddingConfig> = {}): ResolvedSearchConfig {+ return makeConfig({+ vault,+ dbPath,+ semantic: {+ enabled: true,+ provider: "openai-compat",+ // Never contacted: status reads the index, it does not embed.+ baseUrl: "https://hold-report.invalid/v1",+ model: RECOMMENDED_EMBEDDING_MODEL,+ apiKey: FAKE_PROVIDER_KEY,+ ...semantic,+ },+ });+}++/** Seed one document with one chunk per content entry, no phase involved. */+async function seedDocument(config: ResolvedSearchConfig, path: string, contents: string[]) {+ const store = await Store.open(config, { mode: "write" });+ try {+ const docId = store.upsertDocument({+ path,+ title: path,+ contentHash: `h-${path}`,+ mtime: 1_700_000_000,+ size: contents.reduce((n, c) => n + c.length, 0),+ });+ store.replaceChunks(+ docId,+ contents.map((content, i) => ({+ chunkIndex: i,+ content,+ contentHash: `h-${path}-${i}`,+ startLine: 1,+ endLine: 2,+ tokenCount: content.length,+ })),+ );+ } finally {+ await store.close();+ }+ const db = new Database(dbPath);+ try {+ return db+ .query<{ id: number }, [string]>(+ "SELECT c.id AS id FROM chunks c JOIN documents d ON d.id = c.document_id " ++ "WHERE d.path = ? ORDER BY c.chunk_index",+ )+ .all(path)+ .map((r) => r.id);+ } finally {+ db.close();+ }+}++/** Seed and hand back the chunk id at `index`, failing loudly if absent. */+async function seedOne(+ config: ResolvedSearchConfig,+ path: string,+ contents: string[],+ index = 0,+): Promise<number> {+ const ids = await seedDocument(config, path, contents);+ const id = ids[index];+ if (id === undefined) throw new Error(`seeded ${path} with no chunk at index ${index}`);+ return id;+}++/**+ * Record holds the way the phase would, under `key` (default: the+ * current config's key) so the test controls key match explicitly.+ */+async function recordHolds(+ config: ResolvedSearchConfig,+ entries: Array<{ chunkId: number; estimatedTokens: number }>,+ key: ChunkHoldExclusion | null = chunkHoldExclusion(config),+): Promise<void> {+ const store = await Store.open(config, { mode: "write" });+ try {+ store.recordChunkHolds(+ entries.map((e) => ({ ...e, windowTokens: WINDOW })),+ { model: key!.model, embeddingSignature: key!.embeddingSignature },+ );+ } finally {+ await store.close();+ }+}++test("the snapshot names every held chunk with its path, index, estimate and window", async () => {+ const config = cfg();+ expect(chunkHoldExclusion(config)).not.toBeNull();+ const bigId = await seedOne(config, "Brain/big-note.md", [+ "x".repeat(2048),+ "a modest paragraph",+ ]);+ const anotherId = await seedOne(config, "Brain/another.md", ["y".repeat(4096)]);+ await recordHolds(config, [+ { chunkId: bigId, estimatedTokens: 512 },+ { chunkId: anotherId, estimatedTokens: 1024 },+ ]);++ const status = await indexStatus(config);+ expect(status.heldChunks?.total).toBe(2);+ // Sorted by path, then chunk index: the reader gets a stable list.+ expect(status.heldChunks?.items).toEqual([+ {+ path: "Brain/another.md",+ chunkIndex: 0,+ estimatedTokens: 1024,+ windowTokens: WINDOW,+ reason: "over_declared_window",+ },+ {+ path: "Brain/big-note.md",+ chunkIndex: 0,+ estimatedTokens: 512,+ windowTokens: WINDOW,+ reason: "over_declared_window",+ },+ ]);+});++test("a status with nothing held emits the bytes from before the field existed", async () => {+ const config = cfg();+ await seedDocument(config, "Brain/plain.md", ["a modest paragraph"]);+ const status = await indexStatus(config);+ expect("heldChunks" in status).toBe(false);+ const wire = serializeIndexStatus(status);+ expect("held_chunks" in wire).toBe(false);+ expect(JSON.stringify(wire)).not.toContain("held_chunks");+});++test("the wire maps the report to snake_case with the cap's total disclosure", async () => {+ const config = cfg();+ const heldId = await seedOne(config, "Brain/big-note.md", ["x".repeat(2048)]);+ await recordHolds(config, [{ chunkId: heldId, estimatedTokens: 512 }]);+ const wire = serializeIndexStatus(await indexStatus(config));+ expect(wire["held_chunks"]).toEqual({+ total: 1,+ items: [+ {+ path: "Brain/big-note.md",+ chunk_index: 0,+ estimated_tokens: 512,+ window_tokens: WINDOW,+ reason: "over_declared_window",+ },+ ],+ });+});++test("the list is capped and the total discloses the remainder", async () => {+ const config = cfg();+ const contents = Array.from({ length: HELD_CHUNK_REPORT_CAP + 3 }, (_, i) => `d-${i}.md`);+ const ids: Array<{ chunkId: number; estimatedTokens: number }> = [];+ for (const [i, path] of contents.entries()) {+ ids.push({+ chunkId: await seedOne(config, `Brain/${path}`, [`chunk number ${i}`]),+ estimatedTokens: 512 + i,+ });+ }+ await recordHolds(config, ids);++ const status = await indexStatus(config);+ expect(status.heldChunks?.total).toBe(HELD_CHUNK_REPORT_CAP + 3);+ expect(status.heldChunks?.items.length).toBe(HELD_CHUNK_REPORT_CAP);+});++test("a hold row whose chunk has a vector is not reported as held", async () => {+ // Window up, chunk embedded, window back: the old row matches the+ // active key again, but the chunk has a stored vector and is not held+ // back from anything. Reporting it would warn an operator about work+ // that is already done - the report answers to the same fact the+ // pending anti-join answers to.+ const config = cfg();+ const chunkId = await seedOne(config, "Brain/embedded-anyway.md", ["x".repeat(2048)]);+ await recordHolds(config, [{ chunkId, estimatedTokens: 512 }]);+ const store = await Store.open(config, { mode: "write" });+ try {+ store.ensureEmbeddingModel(RECOMMENDED_EMBEDDING_MODEL, 4, { query: "", passage: "" });+ store.vecUpsert(chunkId, [1, 0, 0, 0], RECOMMENDED_EMBEDDING_MODEL, 4, "hash-1");+ } finally {+ await store.close();+ }+ const status = await indexStatus(config);+ expect(status.heldChunks).toBeUndefined();+});++test("a row under a released key is not reported", async () => {+ const config = cfg();+ const heldId = await seedOne(config, "Brain/big-note.md", ["x".repeat(2048)]);+ const oldKey = chunkHoldExclusion(config);+ // Record under the CURRENT key, then change the key underneath the row:+ // a wider declared window is a new key, and the phase would re-decide.+ await recordHolds(config, [{ chunkId: heldId, estimatedTokens: 512 }], oldKey);+ const wider = cfg({ inputWindowTokens: WINDOW * 2 });+ const status = await indexStatus(wider);+ expect("heldChunks" in status).toBe(false);+ // The row survives as inert history rather than being deleted on read.+ const db = new Database(dbPath, { readonly: true });+ try {+ expect(+ db.query<{ n: number }, []>("SELECT COUNT(*) AS n FROM chunk_embedding_holds").get()?.n,+ ).toBe(1);+ } finally {+ db.close();+ }+});++test("a config that cannot hold reports nothing even when rows exist", async () => {+ const config = cfg();+ const heldId = await seedOne(config, "Brain/big-note.md", ["x".repeat(2048)]);+ await recordHolds(config, [{ chunkId: heldId, estimatedTokens: 512 }]);+ // An uncurated model declares no window: there is no current key, so+ // there is nothing the active configuration holds.+ const uncurated = cfg({ model: "acme/custom-embed-9000" });+ expect(chunkHoldExclusion(uncurated)).toBeNull();+ const status = await indexStatus(uncurated);+ expect("heldChunks" in status).toBe(false);+});diff --git a/tests/core/search/chunk-window-census.test.ts b/tests/core/search/chunk-window-census.test.tsindex 3ea0e981..e4252d2a 100644--- a/tests/core/search/chunk-window-census.test.ts+++ b/tests/core/search/chunk-window-census.test.ts@@ -319,6 +319,13 @@ test("search status warns with the registered exit when chunks exceed the window// boundaries - is named too, because a rebuild alone re-cuts the same// chunks.expect(warning!).toContain("search_chunk_size");+ // F2: the consequence and the disclosure changed. Proved-over chunks+ // are HELD, not sent-and-truncated, and the warning points at the+ // held-chunks report that lists them.+ expect(warning!).toContain("held");+ expect(warning!).toContain("o2b search status");+ // The levers are named as two, not one implicit trim: split or exclude.+ expect(warning!).toContain("exclude");});test("an uncurated model is a status FIELD, never a status warning", async () => {@@ -376,6 +383,10 @@ test("the status wire shape carries the field only when the check could not run"// The bytes either side of where the key would have landed are the// pre-census bytes.expect(JSON.stringify(curated)).not.toContain("chunk_window");+ // The F2 held-chunk report follows the same discipline on the same+ // bytes: nothing held, so the key is absent rather than empty.+ expect("held_chunks" in curated).toBe(false);+ expect(JSON.stringify(curated)).not.toContain("held_chunks");});// ── every verdict names a registered exit ────────────────────────────────────diff --git a/tests/core/search/embed-window-hold.test.ts b/tests/core/search/embed-window-hold.test.tsnew file mode 100644index 00000000..f3a2d622--- /dev/null+++ b/tests/core/search/embed-window-hold.test.ts@@ -0,0 +1,358 @@+/**+ * F2 chunk skip-and-hold (Lane B, task B1): the embedding phase holds+ * chunks whose token estimate PROVABLY exceeds the model's effective+ * declared input window.+ *+ * "Provably" is the whole unit. The estimator is two-sided (floor+ * `ceil(codePoints/4)`, ceiling charging every non-ASCII code point a+ * full token), and only the floor clearing the window is a fact - the+ * undecided band between the two bounds stays embedded exactly as before+ * this feature existed, because holding there would act on a guess. A+ * model with no declared window (outside the curated preset table and no+ * operator `embedding_input_window_tokens`) proves nothing, so nothing+ * is held for it; declaring the window is what arms the hold.+ *+ * The hold is a FILTER before batching, so whatever the provider does+ * receive still answers 1:1 (the batch invariant in `runEmbeddingPhase`+ * compares vector counts against its own batch), and what is held never+ * reaches `vecUpsert` - no embeddings row, no vector, no spend. A run+ * that holds every pending chunk therefore also prices nothing: the cost+ * gate reads the embeddable census, not the held one, so held work can+ * neither trip a refusal nor appear on the receipt.+ */++import { afterEach, beforeEach, expect, test } from "bun:test";+import { Database } from "bun:sqlite";++import { runEmbeddingPhase, type EmbeddingPhaseTally } from "../../../src/core/search/indexer.ts";+import { Store } from "../../../src/core/search/store.ts";+import {+ declaredInputWindowTokens,+ RECOMMENDED_EMBEDDING_MODEL,+} from "../../../src/core/search/embeddings/presets.ts";+import { chunkHoldExclusion } from "../../../src/core/search/embedding-spend.ts";+import { signatureIdentityKnown } from "../../../src/core/search/embeddings/signature.ts";+import {+ charLengthOverTokenBudget,+ estimateTokens,+ LOCAL_EMBEDDING_MODEL,+} from "../../../src/core/search/embeddings/signature.ts";+import { CHUNK_HOLD_REASON } from "../../../src/core/search/types.ts";+import type {+ ResolvedEmbeddingConfig,+ ResolvedSearchConfig,+} from "../../../src/core/search/types.ts";+import { createTempVault, makeConfig } from "../../helpers/search-fixtures.ts";+import { startFakeHttp, type FakeHttp } from "../../helpers/fake-http.ts";+import { FAKE_PROVIDER_KEY } from "../../helpers/fake-credentials.ts";++/** The curated window of the shipped recommendation, read from the table. */+const WINDOW = declaredInputWindowTokens(RECOMMENDED_EMBEDDING_MODEL)!;++/** A model string deliberately outside the curated preset table. */+const UNCURATED_MODEL = "acme/custom-embed-9000";++/** Comfortably under any window. */+const FITTING = "small chunk of ordinary prose";++/** The LOW estimate already exceeds the window: provably over. */+const OVER_CONTENT = "b".repeat(charLengthOverTokenBudget(WINDOW) + 1);+const OVER_FLOOR = Math.ceil(OVER_CONTENT.length / 4);++/**+ * The undecided band: floor `ceil(2048/4)` equals the window (fits), the+ * ceiling charges each Cyrillic code point a full token (far over). The+ * census reports this band; the hold must never act on it.+ */+const UNDECIDED_CONTENT = "е".repeat(charLengthOverTokenBudget(WINDOW));++let vault: string;+let dbPath: string;+let cleanup: () => void;+let server: FakeHttp | null = null;++beforeEach(() => {+ ({ vault, dbPath, cleanup } = createTempVault("embed-window-hold"));+});++afterEach(async () => {+ await server?.close();+ server = null;+ cleanup();+});++/**+ * Start the fake provider capturing every input text it receives and+ * answering exactly one 4-dimensional vector per input.+ */+async function captureProvider(): Promise<string[]> {+ server = await startFakeHttp();+ const sent: string[] = [];+ server.setHandler((req) => {+ const body = (req.body ?? {}) as { input?: unknown };+ const inputs = Array.isArray(body.input) ? (body.input as string[]) : [];+ sent.push(...inputs);+ return {+ status: 200,+ body: {+ data: inputs.map((text, index) => ({+ object: "embedding",+ embedding: [text.length, index, 1, 1],+ index,+ })),+ model: "fake",+ },+ };+ });+ return sent;+}++function cfg(semantic: Partial<ResolvedEmbeddingConfig> = {}): ResolvedSearchConfig {+ return makeConfig({+ vault,+ dbPath,+ semantic: {+ enabled: true,+ provider: "openai-compat",+ // Never contacted by the local embedder; only the openai-compat+ // tests point it at the fake server.+ baseUrl: server?.url ?? "https://embeddings.invalid/v1",+ model: RECOMMENDED_EMBEDDING_MODEL,+ apiKey: FAKE_PROVIDER_KEY,+ ...semantic,+ },+ });+}++/** A write-mode store seeded with one document carrying `contents`. */+async function openSeeded(config: ResolvedSearchConfig, contents: string[]): Promise<Store> {+ const store = await Store.open(config, { mode: "write" });+ const docId = store.upsertDocument({+ path: "Brain/note.md",+ title: "note",+ contentHash: "h1",+ mtime: 1_700_000_000,+ size: 10,+ });+ store.replaceChunks(+ docId,+ contents.map((content, i) => ({+ chunkIndex: i,+ content,+ contentHash: `h1-${i}`,+ startLine: 1,+ endLine: 2,+ tokenCount: content.length,+ })),+ );+ return store;+}++/** The chunk row of exactly `length` code points (SQL counts code points). */+function chunkIdByLength(length: number): number {+ const db = new Database(dbPath, { readonly: true });+ try {+ const row = db+ .query<{ id: number }, [number]>("SELECT id FROM chunks WHERE length(content) = ?")+ .get(length);+ if (!row) throw new Error(`no chunk of length ${length} in the seeded index`);+ return row.id;+ } finally {+ db.close();+ }+}++/** Chunk ids that carry a stored vector after the run. */+function embeddedChunkIds(): number[] {+ const db = new Database(dbPath, { readonly: true });+ try {+ return db+ .query<{ chunk_id: number }, []>("SELECT chunk_id FROM embeddings ORDER BY chunk_id")+ .all()+ .map((r) => r.chunk_id);+ } finally {+ db.close();+ }+}++const EMPTY_TALLY = (): EmbeddingPhaseTally => ({ embeddingsComputed: 0, embeddingsRetries: 0 });++// ── the hold filter ──────────────────────────────────────────────────────────++test("a provably-over chunk is never sent while fitting and undecided-band chunks are", async () => {+ const sent = await captureProvider();+ const store = await openSeeded(cfg(), [FITTING, OVER_CONTENT, UNDECIDED_CONTENT]);+ const tally = EMPTY_TALLY();+ try {+ await runEmbeddingPhase(store, cfg(), tally, {});+ } finally {+ await store.close();+ }+ expect(sent).toContain(FITTING);+ expect(sent).toContain(UNDECIDED_CONTENT);+ expect(sent).not.toContain(OVER_CONTENT);+ // One vector per input survived the filter: the phase would have thrown+ // on a count mismatch, and the two texts it did send are stored.+ expect(tally.embeddingsComputed).toBe(2);+ const heldId = chunkIdByLength(OVER_CONTENT.length);+ expect(embeddedChunkIds()).not.toContain(heldId);+});++test("the held content reaches no vector store: embeddings rows exactly match what was sent", async () => {+ await captureProvider();+ const store = await openSeeded(cfg(), [FITTING, OVER_CONTENT, UNDECIDED_CONTENT]);+ try {+ await runEmbeddingPhase(store, cfg(), EMPTY_TALLY(), {});+ } finally {+ await store.close();+ }+ const ids = embeddedChunkIds().toSorted();+ expect(ids).toEqual(+ [chunkIdByLength(FITTING.length), chunkIdByLength(UNDECIDED_CONTENT.length)].toSorted(),+ );+});++// ── the tally ────────────────────────────────────────────────────────────────++test("the phase records each held chunk with its floor estimate, the window and the reason", async () => {+ await captureProvider();+ const store = await openSeeded(cfg(), [FITTING, OVER_CONTENT, UNDECIDED_CONTENT]);+ const tally = EMPTY_TALLY();+ try {+ await runEmbeddingPhase(store, cfg(), tally, {});+ } finally {+ await store.close();+ }+ expect(tally.embeddingsHeld).toBe(1);+ expect(tally.heldChunks).toEqual([+ {+ chunkId: chunkIdByLength(OVER_CONTENT.length),+ estimatedTokens: OVER_FLOOR,+ windowTokens: WINDOW,+ reason: CHUNK_HOLD_REASON.overDeclaredWindow,+ },+ ]);+});++test("a run that holds nothing records no hold tally at all", async () => {+ await captureProvider();+ const store = await openSeeded(cfg(), [FITTING]);+ const tally = EMPTY_TALLY();+ try {+ await runEmbeddingPhase(store, cfg(), tally, {});+ } finally {+ await store.close();+ }+ expect(tally.embeddingsHeld).toBeUndefined();+ expect(tally.heldChunks).toBeUndefined();+});++// ── holding is not spend ─────────────────────────────────────────────────────++test("everything held: no provider call, no receipt, no refusal under a gate the held census would blow", async () => {+ const sent = await captureProvider();+ // Priced so OVER_CONTENT's floor alone exceeds the gate: pre-feature+ // this census tripped EMBEDDING_COST_GATE, which is the false alarm a+ // held chunk must never cause.+ const config = cfg({+ costGateUsd: 0.000001,+ priceOverride: { model: RECOMMENDED_EMBEDDING_MODEL, usdPerMtok: 0.05 },+ });+ const store = await openSeeded(config, [OVER_CONTENT]);+ const tally = EMPTY_TALLY();+ try {+ await runEmbeddingPhase(store, config, tally, {});+ } finally {+ await store.close();+ }+ expect(sent).toEqual([]);+ expect(server!.callCount()).toBe(0);+ expect(tally.embeddingsComputed).toBe(0);+ expect(tally.embeddingsHeld).toBe(1);+ expect(tally.spend).toBeUndefined();+});++test("the gate and the receipt read the embeddable census, not the held one", async () => {+ await captureProvider();+ const config = cfg({+ costGateUsd: 0.000001,+ priceOverride: { model: RECOMMENDED_EMBEDDING_MODEL, usdPerMtok: 0.05 },+ });+ // The fitting chunk's own priced estimate stays under the gate; the+ // held chunk would have taken the run's estimate over it.+ const store = await openSeeded(config, [FITTING, OVER_CONTENT]);+ const tally = EMPTY_TALLY();+ try {+ await runEmbeddingPhase(store, config, tally, {});+ } finally {+ await store.close();+ }+ expect(tally.embeddingsHeld).toBe(1);+ expect(tally.spend).toBeDefined();+ expect(tally.spend!.tokens).toBe(estimateTokens([FITTING]));+ expect(tally.spend!.forced).toBe(false);+});++// ── the hold only acts on proof ──────────────────────────────────────────────++test("an undeclared window proves nothing: an oversized chunk is embedded as before", async () => {+ const sent = await captureProvider();+ const config = cfg({ model: UNCURATED_MODEL });+ const store = await openSeeded(config, [OVER_CONTENT]);+ const tally = EMPTY_TALLY();+ try {+ await runEmbeddingPhase(store, config, tally, {});+ } finally {+ await store.close();+ }+ expect(sent).toContain(OVER_CONTENT);+ expect(tally.embeddingsHeld).toBeUndefined();+ expect(tally.embeddingsComputed).toBe(1);+});++test("an operator-declared window arms the hold even for an uncurated model", async () => {+ const sent = await captureProvider();+ const config = cfg({ model: UNCURATED_MODEL, inputWindowTokens: WINDOW });+ const store = await openSeeded(config, [OVER_CONTENT]);+ const tally = EMPTY_TALLY();+ try {+ await runEmbeddingPhase(store, config, tally, {});+ } finally {+ await store.close();+ }+ expect(sent).not.toContain(OVER_CONTENT);+ expect(tally.embeddingsHeld).toBe(1);+ expect(tally.embeddingsComputed).toBe(0);+});++test("the hold key changes when the sent passage prefix changes", () => {+ // The hold's proof is measured over prefix + content, so the key the+ // proof is recorded under has to answer for the prefix too: a shortened+ // or removed prefix changes what fits the window, and a key that cannot+ // see the change would keep excluding a chunk that now fits.+ const withPrefix = cfg({ passagePrefix: "passage: ", dimension: 4 });+ const shortened = cfg({ passagePrefix: "", dimension: 4 });+ const otherPrefix = cfg({ passagePrefix: "represent passage: ", dimension: 4 });+ const a = chunkHoldExclusion(withPrefix)!;+ const b = chunkHoldExclusion(shortened)!;+ const c = chunkHoldExclusion(otherPrefix)!;+ expect(a.embeddingSignature).not.toBe(b.embeddingSignature);+ expect(a.embeddingSignature).not.toBe(c.embeddingSignature);+ // The composed key still reads as a signature from the end: the+ // trailing provider:model:dimension triple is intact.+ expect(signatureIdentityKnown(a.embeddingSignature)).toBe(true);+ expect(signatureIdentityKnown(b.embeddingSignature)).toBe(true);+});++test("the local embedder has no window and holds nothing", async () => {+ const config = cfg({ provider: "local", model: LOCAL_EMBEDDING_MODEL });+ const store = await openSeeded(config, [OVER_CONTENT]);+ const tally = EMPTY_TALLY();+ try {+ await runEmbeddingPhase(store, config, tally, {});+ } finally {+ await store.close();+ }+ expect(tally.embeddingsHeld).toBeUndefined();+ expect(tally.embeddingsComputed).toBe(1);+});diff --git a/tests/core/search/entity-alias.test.ts b/tests/core/search/entity-alias.test.tsindex e52f833d..0631fc2c 100644--- a/tests/core/search/entity-alias.test.ts+++ b/tests/core/search/entity-alias.test.ts@@ -13,7 +13,10 @@ import { join } from "node:path";import { indexVault, search } from "../../../src/core/search/index.ts";import { makeConfig, createTempVault } from "../../helpers/search-fixtures.ts";-import { expandQueryEntities } from "../../../src/core/search/entity-alias.ts";+import {+ expandQueryEntities,+ PERSON_NICKNAMES_ARTIFACT_REL,+} from "../../../src/core/search/entity-alias.ts";import { archiveEntity, upsertEntity } from "../../../src/core/brain/entities/registry.ts";let tmp: ReturnType<typeof createTempVault>;@@ -38,6 +41,22 @@ function seedRegistry(): void {});}+function seedPerson(name = "Robert Smith"): string {+ const { entity } = upsertEntity(tmp.vault, {+ category: "people",+ name,+ agent: "claude-dev-agent",+ now: NOW,+ });+ return entity.id;+}++/** Write the operator nickname map the config reader resolves. */+function writeNicknames(entries: Record<string, string[]>): void {+ mkdirSync(join(tmp.vault, "Brain", "entities"), { recursive: true });+ writeFileSync(join(tmp.vault, PERSON_NICKNAMES_ARTIFACT_REL), JSON.stringify(entries));+}+describe("expandQueryEntities", () => {test("query naming an alias gains the canonical name forms", () => {seedRegistry();@@ -69,6 +88,128 @@ describe("expandQueryEntities", () => {});});+describe("person-variant expansion", () => {+ test("query 'Bob' resolves against a Robert-only registry through operator nickname data", () => {+ const id = seedPerson();+ writeNicknames({ bob: ["Robert Smith"] });+ const out = expandQueryEntities(tmp.vault, ["bob"]);+ // The declared canonical name joins `added`; the derived structural+ // forms join `derived` ("bob" itself is already a query entity).+ expect(out.expanded).toEqual(["bob", "robert smith", "r smith", "smith robert"]);+ expect(out.added).toEqual(["robert smith"]);+ expect(out.derived).toEqual(["r smith", "smith robert"]);+ expect(out.sourceIds).toEqual([id]);+ });++ test("structural variants participate without any nickname data", () => {+ seedPerson();+ const out = expandQueryEntities(tmp.vault, ["smith robert"]);+ expect(out.added).toEqual(["robert smith"]);+ // The reversed form the query already carried is skipped; the+ // initial form is the one derived addition.+ expect(out.derived).toEqual(["r smith"]);+ expect(out.expanded).toEqual(["smith robert", "robert smith", "r smith"]);+ });++ test("declared forms always win attribution: a derived form another entity declared stays in added", () => {+ seedPerson();+ upsertEntity(tmp.vault, {+ category: "people",+ name: "Smith Robert",+ agent: "claude-dev-agent",+ now: NOW,+ });+ const out = expandQueryEntities(tmp.vault, ["smith robert"]);+ // "smith robert" is entity B's DECLARED name: it must never also+ // appear as a derived form.+ expect(out.added).toContain("robert smith");+ expect(out.derived).not.toContain("smith robert");+ expect(out.sourceIds).toContain("ent-people-smith-robert");+ });++ test("a malformed nickname file drops only the nickname lane: declared expansion keeps working", () => {+ // Identity expansion here would turn off the declared name/alias+ // expansion that never needed the file - one bad operator file must+ // not silence the whole registry.+ seedRegistry();+ mkdirSync(join(tmp.vault, "Brain", "entities"), { recursive: true });+ writeFileSync(join(tmp.vault, PERSON_NICKNAMES_ARTIFACT_REL), "{ not json");+ const out = expandQueryEntities(tmp.vault, ["osb"]);+ expect(out.added).toEqual(["open second brain", "the vault project"]);+ expect(out.expanded).toEqual(["osb", "open second brain", "the vault project"]);+ });++ test("a form two entities share is attributed to the declaration, never to both lanes", () => {+ // Entity A's derivation reaches "smith robert"; entity B DECLARES it+ // as an alias. Whichever entity the walk visits first, the form must+ // land in exactly one lane - the declared one - or candidate signals+ // count it twice and attribute it wrong.+ seedPerson("Robert Smith");+ upsertEntity(tmp.vault, {+ category: "people",+ name: "R Smith",+ aliases: ["Smith Robert"],+ agent: "claude-dev-agent",+ now: NOW,+ });+ const out = expandQueryEntities(tmp.vault, ["r smith"]);+ // The declared claim wins: the shared form is added, not derived.+ expect(out.added).toContain("smith robert");+ expect(out.derived).not.toContain("smith robert");+ // And the lanes are disjoint over the whole expansion.+ expect(new Set(out.expanded).size).toBe(out.expanded.length);+ });++ test("a vault with no person entities returns the identity expansion byte-identically", () => {+ seedRegistry();+ writeNicknames({ bob: ["ada lovelace"] });+ const out = expandQueryEntities(tmp.vault, ["osb"]);+ // Exactly the pre-variant expansion: every declared non-query form+ // in `added`, nothing derived.+ expect(out.expanded).toEqual(["osb", "open second brain", "the vault project"]);+ expect(out.added).toEqual(["open second brain", "the vault project"]);+ expect(out.derived).toEqual([]);+ expect(out.sourceIds).toEqual(["ent-projects-open-second-brain"]);+ });++ test("org-category entities never derive person variants", () => {+ upsertEntity(tmp.vault, {+ category: "org",+ name: "Acme Corp",+ agent: "claude-dev-agent",+ now: NOW,+ });+ const out = expandQueryEntities(tmp.vault, ["corp acme"]);+ expect(out.derived).toEqual([]);+ expect(out.added).toEqual([]);+ });++ test("a quarantined person does not derive", () => {+ upsertEntity(tmp.vault, {+ category: "people",+ name: "Robert Smith",+ agent: "claude-dev-agent",+ now: NOW,+ untrustedOrigin: true,+ });+ writeNicknames({ bob: ["Robert Smith"] });+ const out = expandQueryEntities(tmp.vault, ["bob"]);+ expect(out.expanded).toEqual(["bob"]);+ expect(out.added).toEqual([]);+ expect(out.derived).toEqual([]);+ });++ test("an unreadable or malformed nickname file degrades to the identity expansion", () => {+ seedPerson();+ mkdirSync(join(tmp.vault, "Brain", "entities"), { recursive: true });+ writeFileSync(join(tmp.vault, PERSON_NICKNAMES_ARTIFACT_REL), "{ not json");+ const out = expandQueryEntities(tmp.vault, ["bob"]);+ expect(out.expanded).toEqual(["bob"]);+ expect(out.added).toEqual([]);+ expect(out.derived).toEqual([]);+ });+});+describe("alias-aware search boost", () => {test("query naming an alias boosts the doc naming the canonical entity", async () => {seedRegistry();diff --git a/tests/core/search/entity-variant-rank.test.ts b/tests/core/search/entity-variant-rank.test.tsnew file mode 100644index 00000000..3de6c264--- /dev/null+++ b/tests/core/search/entity-variant-rank.test.ts@@ -0,0 +1,267 @@+/**+ * Exact-declared-first entity ranking (write-honesty-egress-consent,+ * F3 / C3).+ *+ * Declared-form matches keep the existing entity boost lane unchanged;+ * derived person-variant matches get a strictly smaller capped lane in+ * their own right. Both layers are re-ranking only - the candidate set+ * is never widened here - and both are omitted from reasons and+ * breakdown when they do not fire, so a vault where no derived form+ * exists ranks byte-identically.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdirSync, writeFileSync } from "node:fs";+import { join } from "node:path";++import { indexVault, search } from "../../../src/core/search/index.ts";+import { makeConfig, createTempVault } from "../../helpers/search-fixtures.ts";+import { PERSON_NICKNAMES_ARTIFACT_REL } from "../../../src/core/search/entity-alias.ts";+import { rankResults } from "../../../src/core/search/ranker.ts";+import { upsertEntity } from "../../../src/core/brain/entities/registry.ts";+import type { KeywordHit, HydratedChunk } from "../../../src/core/search/store.ts";++const NOW = 1_750_000_000_000;++function hyd(chunkId: number, docId: number): HydratedChunk {+ return Object.freeze({+ chunkId,+ documentId: docId,+ path: `doc${docId}.md`,+ title: `Doc ${docId}`,+ content: `chunk ${chunkId}`,+ startLine: 1,+ endLine: 1,+ mtime: 0,+ });+}++function baseInputs() {+ return {+ keyword: [+ { chunkId: 1, documentId: 10, bm25: -5 },+ { chunkId: 2, documentId: 11, bm25: -5 },+ { chunkId: 3, documentId: 12, bm25: -5 },+ ] as KeywordHit[],+ semantic: [],+ hydrated: new Map([+ [1, hyd(1, 10)],+ [2, hyd(2, 11)],+ [3, hyd(3, 12)],+ ]),+ inboundLinkSources: new Map(),+ tagsByDoc: new Map(),+ };+}++const RANK_OPTS = { keywordWeight: 0.6, semanticWeight: 0.4, limit: 10, nowMs: NOW };++describe("derived entity ranking lane (unit)", () => {+ test("a declared-form match outranks an equal derived-variant match", () => {+ const ranked = rankResults(+ {+ ...baseInputs(),+ entityMatchByChunk: new Map([[1, 1]]),+ derivedEntityMatchByChunk: new Map([[2, 1]]),+ },+ RANK_OPTS,+ );+ const declared = ranked.find((r) => r.chunkId === 1)!;+ const derived = ranked.find((r) => r.chunkId === 2)!;+ expect(declared.score).toBeGreaterThan(derived.score);+ expect(declared.breakdown!.entity).toBe(0.02);+ expect(derived.breakdown!.entity).toBe(0);+ expect(derived.breakdown!.entityDerived).toBe(0.01);+ expect(derived.reasons).toContain("entity_variant: 0.010");+ });++ test("declared, neutral and derived-only candidates sort transitively", () => {+ // A declared chunk with a tiny base score, a derived-only chunk with+ // a large one, and a neutral chunk in between: a comparator that+ // answers declared-before-derived BEFORE comparing scores is not a+ // strict weak ordering (A<B, B<C, C<A), so the sorted output depends+ // on the input order. The ranking must be a total order - declared,+ // then neutral, then derived-only, score within each tier - the same+ // for every input permutation.+ const inputsFor = (ids: number[]) => {+ const bm25 = new Map([+ [1, -1],+ [2, -9],+ [3, -5],+ ]);+ return {+ ...baseInputs(),+ keyword: ids.map(+ (id) => ({ chunkId: id, documentId: id * 10, bm25: bm25.get(id)! }) as KeywordHit,+ ),+ hydrated: new Map(ids.map((id) => [id, hyd(id, id * 10)])),+ entityMatchByChunk: new Map([[1, 1]]),+ derivedEntityMatchByChunk: new Map([[2, 1]]),+ };+ };+ const idsOf = (ranked: ReadonlyArray<{ chunkId: number }>) => ranked.map((r) => r.chunkId);+ const forward = idsOf(rankResults(inputsFor([1, 2, 3]), RANK_OPTS));+ const backward = idsOf(rankResults(inputsFor([3, 2, 1]), RANK_OPTS));+ expect(forward).toEqual(backward);+ expect(forward).toEqual([1, 3, 2]);+ });++ test("the derived lane is capped strictly below the declared cap", () => {+ const ranked = rankResults(+ {+ ...baseInputs(),+ entityMatchByChunk: new Map([[1, 50]]),+ derivedEntityMatchByChunk: new Map([[2, 50]]),+ },+ RANK_OPTS,+ );+ const declared = ranked.find((r) => r.chunkId === 1)!;+ const derived = ranked.find((r) => r.chunkId === 2)!;+ expect(declared.breakdown!.entity).toBe(0.04);+ expect(derived.breakdown!.entityDerived).toBe(0.02);+ expect(derived.breakdown!.entityDerived).toBeLessThan(declared.breakdown!.entity);+ });++ test("a chunk matching both a declared and a derived form outranks either alone", () => {+ const ranked = rankResults(+ {+ ...baseInputs(),+ entityMatchByChunk: new Map([[1, 1]]),+ derivedEntityMatchByChunk: new Map([[1, 1]]),+ },+ RANK_OPTS,+ );+ const both = ranked.find((r) => r.chunkId === 1)!;+ expect(both.breakdown!.entity).toBe(0.02);+ expect(both.breakdown!.entityDerived).toBe(0.01);+ });++ test("a zero derived match omits the layer from breakdown and reasons", () => {+ const ranked = rankResults(+ {+ ...baseInputs(),+ entityMatchByChunk: new Map([[1, 1]]),+ derivedEntityMatchByChunk: new Map<number, number>(),+ },+ RANK_OPTS,+ );+ const one = ranked.find((r) => r.chunkId === 1)!;+ const other = ranked.find((r) => r.chunkId === 2)!;+ expect(one.breakdown).not.toHaveProperty("entityDerived");+ expect(other.breakdown).not.toHaveProperty("entityDerived");+ expect(one.reasons.some((x) => x.startsWith("entity_variant"))).toBe(false);+ });++ test("an absent derived map ranks byte-identically to a run without one", () => {+ const without = rankResults(+ { ...baseInputs(), entityMatchByChunk: new Map([[1, 2]]) },+ RANK_OPTS,+ );+ const withEmpty = rankResults(+ {+ ...baseInputs(),+ entityMatchByChunk: new Map([[1, 2]]),+ derivedEntityMatchByChunk: undefined,+ },+ RANK_OPTS,+ );+ expect(withEmpty.map((r) => r.score)).toEqual(without.map((r) => r.score));+ });++ test("two otherwise-equal chunks, one with a single declared match, one with three derived matches: declared ranks first for several entityMul values", () => {+ for (const entityMul of [0.5, 1.0, 2.0]) {+ for (const [declaredId, derivedId] of [+ [1, 2],+ [2, 1],+ ] as const) {+ const ranked = rankResults(+ {+ ...baseInputs(),+ entityMatchByChunk: new Map([[declaredId, 1]]),+ derivedEntityMatchByChunk: new Map([[derivedId, 3]]),+ },+ {+ ...RANK_OPTS,+ weightProfile: { keywordMul: 1, semanticMul: 1, recencyMul: 1, entityMul },+ },+ );+ const declaredIndex = ranked.findIndex((r) => r.chunkId === declaredId);+ const derivedIndex = ranked.findIndex((r) => r.chunkId === derivedId);+ expect(declaredIndex).toBeGreaterThanOrEqual(0);+ expect(derivedIndex).toBeGreaterThanOrEqual(0);+ expect(declaredIndex).toBeLessThan(derivedIndex);+ }+ }+ });+});++let tmp: ReturnType<typeof createTempVault>;++beforeEach(() => {+ tmp = createTempVault("entity-variant-rank");+});++afterEach(() => {+ tmp.cleanup();+});++describe("declared-before-derived ranking (end to end)", () => {+ test("a declared-form note outranks a derived-variant note for otherwise-equal candidates", async () => {+ upsertEntity(tmp.vault, {+ category: "people",+ name: "Robert Smith",+ agent: "claude-dev-agent",+ now: new Date("2026-06-02T12:00:00Z"),+ });+ mkdirSync(join(tmp.vault, "Brain", "entities"), { recursive: true });+ writeFileSync(+ join(tmp.vault, PERSON_NICKNAMES_ARTIFACT_REL),+ JSON.stringify({ bob: ["Robert Smith"] }),+ );+ mkdirSync(join(tmp.vault, "notes"), { recursive: true });+ // All three notes carry identical query-term frequencies and word+ // counts (13 words, each query token once), so BM25 ties exactly;+ // only the entity lanes separate them.+ writeFileSync(+ join(tmp.vault, "notes", "declared.md"),+ "---\ntitle: Declared\n---\n\nThe payment flow Bob runs through Robert Smith to pay more memory costs.\n",+ );+ writeFileSync(+ join(tmp.vault, "notes", "derived.md"),+ "---\ntitle: Derived\n---\n\nThe payment flow Bob runs through Smith Robert to pay more memory costs.\n",+ );+ writeFileSync(+ join(tmp.vault, "notes", "plain.md"),+ "---\ntitle: Plain\n---\n\nThe payment flow Bob runs through other second things to pay memory costs.\n",+ );+ // mmrLambda 1 disables diversity reordering, so the final array+ // order is the ranking's order.+ const config = makeConfig({ vault: tmp.vault, dbPath: tmp.dbPath, maxHops: 0, mmrLambda: 1 });+ await indexVault(config, {});++ const out = await search(config, { query: "payment pay memory Bob", limit: 10 });+ const at = (name: string): number =>+ out.results.findIndex((r) => r.path === `notes/${name}.md`);+ const declared = at("declared");+ const derived = at("derived");+ const plain = at("plain");+ expect(declared).toBeGreaterThanOrEqual(0);+ expect(derived).toBeGreaterThanOrEqual(0);+ expect(plain).toBeGreaterThanOrEqual(0);+ // Declared-form match first, derived-variant match second.+ expect(declared).toBeLessThan(derived);+ expect(derived).toBeLessThan(plain);+ // The derived note discloses both lanes; the declared note keeps its+ // unchanged single lane; the plain note reports no variant layer.+ const derivedRow = out.results[derived]!;+ expect(derivedRow.breakdown?.entity).toBe(0.02);+ expect(derivedRow.breakdown?.entityDerived).toBe(0.01);+ expect(derivedRow.reasons).toContain("entity_variant: 0.010");+ const declaredRow = out.results[declared]!;+ expect(declaredRow.breakdown?.entity).toBe(0.04);+ expect(declaredRow.breakdown).not.toHaveProperty("entityDerived");+ const plainRow = out.results[plain]!;+ expect(plainRow.breakdown).not.toHaveProperty("entityDerived");+ expect(plainRow.reasons.some((x) => x.startsWith("entity_variant"))).toBe(false);+ });+});diff --git a/tests/core/search/hybrid-overlap.test.ts b/tests/core/search/hybrid-overlap.test.tsindex e3ce8e87..3ad4af4a 100644--- a/tests/core/search/hybrid-overlap.test.ts+++ b/tests/core/search/hybrid-overlap.test.ts@@ -528,7 +528,13 @@ test("a provider that never reports a dispatch keeps the pre-call interval as thtest("keyword sentences precede semantic sentences in the merged warnings", async () => {if (!sqliteVecLoadable()) return;await seedCorpus();- const cfg = semanticConfig({ inputWindowTokens: 8 });+ // The window must still CARRY the corpus now that provably over-window+ // chunks are held back from embedding (the skip-and-hold contract): a+ // window of 8 held every seeded chunk, so no embedding existed and the+ // query-side cut never ran. At 32 the seeded chunks embed and the+ // 200-character query still exceeds the window, which is the cut this+ // test exists to observe.+ const cfg = semanticConfig({ inputWindowTokens: 32 });await indexVault(cfg, { embeddings: true });// Keyword sentence: a desynced FTS table rebuilt at query time.const db = new Database(dbPath);diff --git a/tests/core/search/provider-consent.test.ts b/tests/core/search/provider-consent.test.tsnew file mode 100644index 00000000..81618e5e--- /dev/null+++ b/tests/core/search/provider-consent.test.ts@@ -0,0 +1,294 @@+/**+ * The provider-consent resolver: UNSET is not DECLINED, and a record that+ * cannot be read is a decline (write-honesty-egress-consent, Lane F).+ *+ * The defect this file exists for: an operator who wrote+ * `search_semantic_enabled: false` and an operator who never configured+ * semantic search at all reached the SAME capability tier, so every surface+ * reported the same sentence for two opposite situations - one of them a+ * decision, the other an absence. The resolver separates them, and it does+ * so WITHOUT inventing a third "off" spelling: the capability tier still+ * answers what the configuration can do, and the consent answers only+ * whether the operator wrote a decline.+ *+ * The fail-closed arm is the other half: a consent record that cannot be+ * read is not a record of consent. Reading it as "unset" would turn a+ * broken config back into a silent default, which is the same defect the+ * config reader itself refuses elsewhere ("an unreadable config is not+ * consent to print host paths", `src/mcp/vault-path-field.ts`).+ *+ * No test here reaches the network, and none of them drops permission+ * bits: the unreadable fixture is a DIRECTORY in the config's place, which+ * every platform refuses the same way.+ */++import { describe, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import {+ PROVIDER_CONSENT_KEY,+ PROVIDER_CONSENT_REENABLE_KEYS,+ PROVIDER_CONSENT_STATE,+ providerConsentIsDeclined,+ resolveProviderConsent,+} from "../../../src/core/search/provider-consent.ts";+import {+ resolveSemanticCapability,+ SEMANTIC_CAPABILITY_TIER,+} from "../../../src/core/search/capability-tier.ts";+import { BRAIN_CONFIG_TEMPLATE_OMISSIONS } from "../../../src/core/brain/config-template.ts";+import type { ResolvedEmbeddingConfig } from "../../../src/core/search/types.ts";++/** A resolved configuration with no provider named and the switch off. */+const UNSET_SEMANTIC: ResolvedEmbeddingConfig = Object.freeze({+ enabled: false,+ provider: "openai-compat",+ baseUrl: null,+ model: null,+ apiKey: null,+ dimension: null,+ timeoutMs: 10_000,+ concurrency: 4,+ batchSize: 32,+ costGateUsd: 0,+ maxRetries: 3,+});++/** A configuration that names the sentinel provider explicitly. */+const DECLINED_PROVIDER_SEMANTIC: ResolvedEmbeddingConfig = Object.freeze({+ ...UNSET_SEMANTIC,+ provider: "disabled",+});++/** A configuration with the switch on and a provider that needs a key. */+const KEYLESS_SEMANTIC: ResolvedEmbeddingConfig = Object.freeze({+ ...UNSET_SEMANTIC,+ enabled: true,+});++interface Fixture {+ readonly dir: string;+ readonly configPath: string;+ readonly cleanup: () => void;+}++/** A temporary directory with a config path inside it. */+function fixture(prefix: string): Fixture {+ const dir = mkdtempSync(join(tmpdir(), `o2b-${prefix}-`));+ return {+ dir,+ configPath: join(dir, "config.yaml"),+ cleanup: () => rmSync(dir, { recursive: true, force: true }),+ };+}++describe("the provider-consent resolver", () => {+ test("a configuration that records no decision is UNSET, not declined", () => {+ const f = fixture("consent-unset");+ try {+ writeFileSync(f.configPath, 'vault: "/tmp/v"\n');+ const consent = resolveProviderConsent({ configPath: f.configPath, env: {} });+ expect(consent.state).toBe(PROVIDER_CONSENT_STATE.unset);+ expect(providerConsentIsDeclined(consent)).toBe(false);+ expect(consent.declineKeys).toEqual([]);+ } finally {+ f.cleanup();+ }+ });++ test("an ABSENT config file is UNSET: nothing was written, so nothing was declined", () => {+ const f = fixture("consent-absent");+ try {+ const consent = resolveProviderConsent({ configPath: f.configPath, env: {} });+ expect(consent.state).toBe(PROVIDER_CONSENT_STATE.unset);+ } finally {+ f.cleanup();+ }+ });++ test("search_semantic_enabled=false is an explicit decline", () => {+ const f = fixture("consent-semantic-off");+ try {+ writeFileSync(f.configPath, "search_semantic_enabled: false\n");+ const consent = resolveProviderConsent({ configPath: f.configPath, env: {} });+ expect(consent.state).toBe(PROVIDER_CONSENT_STATE.declined);+ expect(providerConsentIsDeclined(consent)).toBe(true);+ expect(consent.declineKeys).toEqual([PROVIDER_CONSENT_KEY.semantic]);+ } finally {+ f.cleanup();+ }+ });++ test("embedding_provider=disabled is an explicit decline", () => {+ const f = fixture("consent-provider-disabled");+ try {+ writeFileSync(f.configPath, "embedding_provider: disabled\n");+ const consent = resolveProviderConsent({ configPath: f.configPath, env: {} });+ expect(consent.state).toBe(PROVIDER_CONSENT_STATE.declined);+ expect(consent.declineKeys).toEqual([PROVIDER_CONSENT_KEY.provider]);+ } finally {+ f.cleanup();+ }+ });++ test("both spellings at once are one decline carrying both keys", () => {+ const f = fixture("consent-both");+ try {+ writeFileSync(f.configPath, "search_semantic_enabled: false\nembedding_provider: disabled\n");+ const consent = resolveProviderConsent({ configPath: f.configPath, env: {} });+ expect(consent.state).toBe(PROVIDER_CONSENT_STATE.declined);+ expect(consent.declineKeys).toEqual([+ PROVIDER_CONSENT_KEY.semantic,+ PROVIDER_CONSENT_KEY.provider,+ ]);+ } finally {+ f.cleanup();+ }+ });++ test("the environment spelling records the same decline as the config spelling", () => {+ const f = fixture("consent-env");+ try {+ writeFileSync(f.configPath, "search_semantic_enabled: true\n");+ const consent = resolveProviderConsent({+ configPath: f.configPath,+ env: { OPEN_SECOND_BRAIN_SEARCH_SEMANTIC: "false" },+ });+ expect(consent.state).toBe(PROVIDER_CONSENT_STATE.declined);+ expect(consent.declineKeys).toEqual([PROVIDER_CONSENT_KEY.semantic]);+ } finally {+ f.cleanup();+ }+ });++ test("a config value that cannot be read is a DECLINE, not an absence", () => {+ // A DIRECTORY in the config's place: the reader refuses a path that+ // exists and is not a regular file, on every platform, with no+ // permission bits involved.+ const f = fixture("consent-unreadable");+ try {+ mkdirSync(f.configPath, { recursive: true });+ const consent = resolveProviderConsent({ configPath: f.configPath, env: {} });+ expect(consent.state).toBe(PROVIDER_CONSENT_STATE.declined);+ expect(providerConsentIsDeclined(consent)).toBe(true);+ expect(consent.detail).toContain(PROVIDER_CONSENT_KEY.semantic);+ } finally {+ f.cleanup();+ }+ });++ test("a consent value that cannot be parsed is a DECLINE, not an absence", () => {+ const f = fixture("consent-unparseable");+ try {+ writeFileSync(f.configPath, "search_semantic_enabled: maybe\n");+ const consent = resolveProviderConsent({ configPath: f.configPath, env: {} });+ expect(consent.state).toBe(PROVIDER_CONSENT_STATE.declined);+ expect(consent.declineKeys).toEqual([PROVIDER_CONSENT_KEY.semantic]);+ } finally {+ f.cleanup();+ }+ });++ test("the resolver extends the capability tier instead of replacing it", () => {+ const f = fixture("consent-tier");+ try {+ writeFileSync(f.configPath, 'vault: "/tmp/v"\n');+ // The tier still answers what the configuration can do, and the+ // consent carries that same tier rather than a second ladder.+ const unset = resolveProviderConsent({+ configPath: f.configPath,+ env: {},+ semantic: KEYLESS_SEMANTIC,+ });+ expect(unset.capability?.tier).toBe(SEMANTIC_CAPABILITY_TIER.credentialMissing);+ expect(unset.capability?.tier).toBe(resolveSemanticCapability(KEYLESS_SEMANTIC).tier);+ // A declined configuration sits on the SAME disabled rung - the+ // consent adds a fact beside the tier, it does not invent a rung.+ writeFileSync(f.configPath, "embedding_provider: disabled\n");+ const declined = resolveProviderConsent({+ configPath: f.configPath,+ env: {},+ semantic: DECLINED_PROVIDER_SEMANTIC,+ });+ expect(declined.capability?.tier).toBe(SEMANTIC_CAPABILITY_TIER.disabled);+ expect(declined.capability?.tier).toBe(+ resolveSemanticCapability(DECLINED_PROVIDER_SEMANTIC).tier,+ );+ expect(declined.state).toBe(PROVIDER_CONSENT_STATE.declined);+ } finally {+ f.cleanup();+ }+ });++ test("a resolved configuration is not a witness: only the record is", () => {+ // A configuration that was ASSEMBLED rather than read - a fixture, an+ // override - names the sentinel provider without any operator having+ // written a decision down. Claiming a decline there would report a+ // decision the operator never made.+ const f = fixture("consent-assembled");+ try {+ writeFileSync(f.configPath, 'vault: "/tmp/v"\n');+ const consent = resolveProviderConsent({+ configPath: f.configPath,+ env: {},+ semantic: DECLINED_PROVIDER_SEMANTIC,+ });+ expect(consent.state).toBe(PROVIDER_CONSENT_STATE.unset);+ expect(consent.capability?.tier).toBe(SEMANTIC_CAPABILITY_TIER.disabled);+ } finally {+ f.cleanup();+ }+ });++ test("the refusal names the keys that would re-enable the provider", () => {+ const f = fixture("consent-reenable");+ try {+ writeFileSync(f.configPath, "search_semantic_enabled: false\n");+ const consent = resolveProviderConsent({ configPath: f.configPath, env: {} });+ for (const key of PROVIDER_CONSENT_REENABLE_KEYS) {+ expect(consent.detail).toContain(key);+ }+ expect(consent.reenableKeys).toEqual([+ PROVIDER_CONSENT_KEY.semantic,+ PROVIDER_CONSENT_KEY.provider,+ ]);+ } finally {+ f.cleanup();+ }+ });++ test("every verdict carries a non-empty detail", () => {+ const f = fixture("consent-detail");+ try {+ writeFileSync(f.configPath, "search_semantic_enabled: false\n");+ for (const semantic of [UNSET_SEMANTIC, DECLINED_PROVIDER_SEMANTIC, KEYLESS_SEMANTIC]) {+ const consent = resolveProviderConsent({+ configPath: f.configPath,+ env: {},+ semantic,+ });+ expect(consent.detail.trim().length).toBeGreaterThan(0);+ }+ } finally {+ f.cleanup();+ }+ });+});++describe("the config surface of the record is declared", () => {+ test("the template declares where the decline lives, and names both spellings", () => {+ // The record is a MACHINE-configuration key, so the vault template+ // declares it as an omission with a written reason rather than+ // templating a knob the resolver never reads. The reason has to name+ // the two spellings, or it is a note to nobody.+ const omission = BRAIN_CONFIG_TEMPLATE_OMISSIONS.find(+ (entry) => entry.key === "provider_consent",+ );+ expect(omission).toBeDefined();+ expect(omission!.reason.trim().length).toBeGreaterThan(80);+ expect(omission!.reason).toContain(PROVIDER_CONSENT_KEY.semantic);+ expect(omission!.reason).toContain(PROVIDER_CONSENT_KEY.provider);+ });+});diff --git a/tests/core/search/provider-decline.test.ts b/tests/core/search/provider-decline.test.tsnew file mode 100644index 00000000..c6348db0--- /dev/null+++ b/tests/core/search/provider-decline.test.ts@@ -0,0 +1,388 @@+/**+ * The decline is honoured at provider construction, before any network+ * call (write-honesty-egress-consent, Lane F).+ *+ * `makeProvider` is the single construction point every embedding+ * provider in this tree goes through, so it is where a recorded decline+ * has to be honoured: a refusal there happens before an endpoint is+ * named, before a key is read and before a byte is sent. The file+ * asserts that with a REQUEST COUNTER on a live loopback server - the+ * count has to stay at zero, because "the provider was never built" is+ * the only claim that keeps a declined vault from reaching an endpoint.+ *+ * The other half of the file is the byte-identity requirement: a+ * configuration that records NO decision must build exactly the provider+ * it built before this feature existed, with no refusal and no changed+ * message, because the difference between "the operator declined" and+ * "the operator never configured this" is the whole point of the feature+ * and collapsing it back would undo it.+ *+ * No test here drops permission bits, and the fake server is a loopback+ * listener, so the file runs the same on every platform.+ */++import { describe, expect, test } from "bun:test";+import { mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { makeProvider } from "../../../src/core/search/embeddings/provider.ts";+import { providerProducesVectors } from "../../../src/core/search/embeddings/contract.ts";+import { resolveConfiguredEmbeddingProvider } from "../../../src/core/search/embeddings/configured-provider.ts";+import {+ PROVIDER_CONSENT_KEY,+ PROVIDER_CONSENT_REENABLE_KEYS,+ resolveProviderConsent,+} from "../../../src/core/search/provider-consent.ts";+import { runSemanticPhase } from "../../../src/core/search/semantic-phase.ts";+import { SearchError } from "../../../src/core/search/types.ts";+import type {+ ResolvedEmbeddingConfig,+ ResolvedSearchConfig,+} from "../../../src/core/search/types.ts";+import type { Store } from "../../../src/core/search/store.ts";+import { makeConfig } from "../../helpers/search-fixtures.ts";+import { startFakeHttp, type FakeHttp } from "../../helpers/fake-http.ts";+import { FAKE_PROVIDER_KEY } from "../../helpers/fake-credentials.ts";++/**+ * A store that holds vectors and has the extension, so the runtime+ * guards ahead of the configuration arm pass and what a case measures is+ * the configuration-derived arm alone.+ */+function readyStore(): Store {+ return {+ counts: () => ({ documents: 1, chunks: 1, embeddings: 1, staleEmbeddings: 0 }),+ vecLoaded: () => true,+ semanticTopK: () => [],+ } as unknown as Store;+}++/** A configuration that names a live endpoint and would reach it. */+function liveSemantic(baseUrl: string): ResolvedEmbeddingConfig {+ return {+ enabled: true,+ provider: "openai-compat",+ baseUrl,+ model: "fake-model",+ apiKey: FAKE_PROVIDER_KEY,+ dimension: 4,+ timeoutMs: 5_000,+ concurrency: 4,+ batchSize: 32,+ costGateUsd: 0,+ maxRetries: 0,+ };+}++interface Fixture {+ readonly dir: string;+ readonly configPath: string;+ readonly cleanup: () => void;+}++/** A temporary directory holding a config file. */+function fixture(prefix: string): Fixture {+ const dir = mkdtempSync(join(tmpdir(), `o2b-${prefix}-`));+ return {+ dir,+ configPath: join(dir, "config.yaml"),+ cleanup: () => rmSync(dir, { recursive: true, force: true }),+ };+}++/** The search configuration a declined vault resolves to. */+function searchConfig(configPath: string, semantic: ResolvedEmbeddingConfig): ResolvedSearchConfig {+ return {+ ...makeConfig({+ vault: "/tmp/does-not-matter",+ dbPath: "/tmp/does-not-matter/db.sqlite",+ semantic,+ }),+ configPath,+ };+}++/**+ * The consent env twins are saved and CLEARED around the tests that+ * resolve a provider without an explicit consent: the resolver reads+ * `process.env` and env values take precedence over the config file, so+ * a developer or CI runner exporting either of them would flip these+ * cases no matter what the fixtures write to disk.+ */+function saveAndClearConsentEnv(): () => void {+ const names = ["OPEN_SECOND_BRAIN_SEARCH_SEMANTIC", "OPEN_SECOND_BRAIN_EMBEDDING_PROVIDER"];+ const saved = names.map((name) => [name, process.env[name]] as const);+ for (const name of names) delete process.env[name];+ return () => {+ for (const [name, value] of saved) {+ if (value === undefined) delete process.env[name];+ else process.env[name] = value;+ }+ };+}++describe("the decline at provider construction", () => {+ test("a declined record refuses the construction and contacts nothing", async () => {+ const server: FakeHttp = await startFakeHttp();+ const f = fixture("decline-construction");+ try {+ writeFileSync(f.configPath, "search_semantic_enabled: false\n");+ const semantic = liveSemantic(server.url);+ const consent = resolveProviderConsent({ configPath: f.configPath, env: {}, semantic });+ let thrown: unknown;+ try {+ makeProvider(semantic, consent);+ } catch (e) {+ thrown = e;+ }+ expect(thrown instanceof SearchError).toBe(true);+ expect((thrown as SearchError).code).toBe("EMBEDDING_DISABLED");+ // The whole point: the refusal happened before a provider existed,+ // so no request was ever sent to the endpoint the configuration+ // named.+ expect(server.callCount()).toBe(0);+ } finally {+ f.cleanup();+ await server.close();+ }+ });++ test("the refusal names the keys that would re-enable the provider", async () => {+ const server: FakeHttp = await startFakeHttp();+ const f = fixture("decline-message");+ try {+ writeFileSync(f.configPath, "embedding_provider: disabled\n");+ const semantic = liveSemantic(server.url);+ const consent = resolveProviderConsent({ configPath: f.configPath, env: {}, semantic });+ let message = "";+ try {+ makeProvider(semantic, consent);+ } catch (e) {+ message = (e as SearchError).message;+ }+ for (const key of PROVIDER_CONSENT_REENABLE_KEYS) {+ expect(message).toContain(key);+ }+ expect(message).toContain(PROVIDER_CONSENT_KEY.provider);+ expect(server.callCount()).toBe(0);+ } finally {+ f.cleanup();+ await server.close();+ }+ });++ test("a caller that passes no consent still reads the record in force", async () => {+ // The indexer and the hygiene detectors call `makeProvider` with the+ // resolved configuration alone, so the record has to be read from the+ // configuration this process is pointed at - the same path the+ // operator named.+ const server: FakeHttp = await startFakeHttp();+ const f = fixture("decline-default-path");+ const saved = process.env["OPEN_SECOND_BRAIN_CONFIG"];+ const restoreConsentEnv = saveAndClearConsentEnv();+ process.env["OPEN_SECOND_BRAIN_CONFIG"] = f.configPath;+ try {+ writeFileSync(f.configPath, "search_semantic_enabled: false\n");+ let thrown: unknown;+ try {+ makeProvider(liveSemantic(server.url));+ } catch (e) {+ thrown = e;+ }+ expect(thrown instanceof SearchError).toBe(true);+ expect((thrown as SearchError).code).toBe("EMBEDDING_DISABLED");+ expect(server.callCount()).toBe(0);+ } finally {+ restoreConsentEnv();+ if (saved === undefined) delete process.env["OPEN_SECOND_BRAIN_CONFIG"];+ else process.env["OPEN_SECOND_BRAIN_CONFIG"] = saved;+ f.cleanup();+ await server.close();+ }+ });++ test("a decline in a named config file is honored when in force and ignored for another file", async () => {+ const server: FakeHttp = await startFakeHttp();+ const fDeclined = fixture("named-decline");+ const fActive = fixture("named-active");+ const saved = process.env["OPEN_SECOND_BRAIN_CONFIG"];+ const restoreConsentEnv = saveAndClearConsentEnv();+ try {+ writeFileSync(fDeclined.configPath, "search_semantic_enabled: false\n");+ writeFileSync(fActive.configPath, 'vault: "/tmp/v"\n');++ // Set ambient config to the declined file. A run pointed at fActive+ // must ignore the ambient decline and build the provider.+ process.env["OPEN_SECOND_BRAIN_CONFIG"] = fDeclined.configPath;+ const activeSemantic = { ...liveSemantic(server.url), configPath: fActive.configPath };+ const activeProvider = makeProvider(activeSemantic);+ expect(providerProducesVectors(activeProvider)).toBe(true);++ // Now set ambient config to the active file. A run pointed at fDeclined+ // must honor the decline in that named file and refuse.+ process.env["OPEN_SECOND_BRAIN_CONFIG"] = fActive.configPath;+ const declinedSemantic = { ...liveSemantic(server.url), configPath: fDeclined.configPath };+ let thrown: unknown;+ try {+ makeProvider(declinedSemantic);+ } catch (e) {+ thrown = e;+ }+ expect(thrown instanceof SearchError).toBe(true);+ expect((thrown as SearchError).code).toBe("EMBEDDING_DISABLED");+ expect(server.callCount()).toBe(0);+ } finally {+ restoreConsentEnv();+ if (saved === undefined) delete process.env["OPEN_SECOND_BRAIN_CONFIG"];+ else process.env["OPEN_SECOND_BRAIN_CONFIG"] = saved;+ fDeclined.cleanup();+ fActive.cleanup();+ await server.close();+ }+ });++ test("a caller that resolved a named config reads the consent from that file, not the ambient default", async () => {+ // `resolveConfiguredEmbeddingProvider` resolves the semantic+ // configuration from the config file it was handed. The consent that+ // gates construction must come from the SAME file: a decline written+ // in some other file the process happens to default to is not a+ // decision about this configuration, and reading it here let one+ // file's decline silently disable a provider another file enabled.+ const server: FakeHttp = await startFakeHttp();+ const fNamed = fixture("named-consent");+ const fAmbient = fixture("ambient-decline");+ const saved = process.env["OPEN_SECOND_BRAIN_CONFIG"];+ const restoreConsentEnv = saveAndClearConsentEnv();+ try {+ writeFileSync(+ fNamed.configPath,+ "search_semantic_enabled: true\n" ++ "embedding_provider: openai-compat\n" ++ `embedding_base_url: ${server.url}\n` ++ "embedding_model: fake-model\n" ++ `embedding_api_key: ${FAKE_PROVIDER_KEY}\n` ++ "embedding_dimension: 4\n",+ );+ writeFileSync(fAmbient.configPath, "search_semantic_enabled: false\n");+ process.env["OPEN_SECOND_BRAIN_CONFIG"] = fAmbient.configPath;+ const provider = resolveConfiguredEmbeddingProvider(fNamed.dir, {+ configPath: fNamed.configPath,+ });+ expect(provider !== null).toBe(true);+ expect(providerProducesVectors(provider!)).toBe(true);+ expect(server.callCount()).toBe(0);+ } finally {+ restoreConsentEnv();+ if (saved === undefined) delete process.env["OPEN_SECOND_BRAIN_CONFIG"];+ else process.env["OPEN_SECOND_BRAIN_CONFIG"] = saved;+ fNamed.cleanup();+ fAmbient.cleanup();+ await server.close();+ }+ });++ test("a configuration that records no decision builds the provider exactly as before", async () => {+ const server: FakeHttp = await startFakeHttp();+ const f = fixture("decline-unset");+ try {+ writeFileSync(f.configPath, 'vault: "/tmp/v"\n');+ const semantic = liveSemantic(server.url);+ const consent = resolveProviderConsent({ configPath: f.configPath, env: {}, semantic });+ // No refusal, and the provider is the one this tree has always+ // built for this configuration: same name, same model, same+ // dimension, and it produces vectors.+ const provider = makeProvider(semantic, consent);+ expect(provider.name).toBe("openai-compat");+ expect(provider.model).toBe("fake-model");+ expect(provider.dimension).toBe(4);+ expect(providerProducesVectors(provider)).toBe(true);+ expect(server.callCount()).toBe(0);+ } finally {+ f.cleanup();+ await server.close();+ }+ });+});++describe("the decline on a query-time embed request", () => {+ test("an explicit request is refused by name", async () => {+ const f = fixture("decline-explicit");+ try {+ writeFileSync(f.configPath, "search_semantic_enabled: false\n");+ const config = searchConfig(f.configPath, liveSemantic("http://127.0.0.1:9"));+ let thrown: unknown;+ try {+ await runSemanticPhase(readyStore(), config, "q", {+ limit: 10,+ pathPrefix: undefined,+ explicit: true,+ });+ } catch (e) {+ thrown = e;+ }+ expect(thrown instanceof SearchError).toBe(true);+ expect((thrown as SearchError).code).toBe("EMBEDDING_DISABLED");+ for (const key of PROVIDER_CONSENT_REENABLE_KEYS) {+ expect((thrown as SearchError).message).toContain(key);+ }+ } finally {+ f.cleanup();+ }+ });++ test("an implicit request degrades with the decline named, not with a bare 'disabled'", async () => {+ const f = fixture("decline-implicit");+ try {+ writeFileSync(f.configPath, "search_semantic_enabled: false\n");+ const config = searchConfig(f.configPath, liveSemantic("http://127.0.0.1:9"));+ const outcome = await runSemanticPhase(readyStore(), config, "q", {+ limit: 10,+ pathPrefix: undefined,+ explicit: false,+ });+ expect(outcome.attempted).toBe(false);+ expect(outcome.warnings.length).toBe(1);+ for (const key of PROVIDER_CONSENT_REENABLE_KEYS) {+ expect(outcome.warnings[0]).toContain(key);+ }+ } finally {+ f.cleanup();+ }+ });++ test("a configuration that records no decision keeps the existing behaviour", async () => {+ const f = fixture("decline-unset-phase");+ try {+ writeFileSync(f.configPath, 'vault: "/tmp/v"\n');+ // The switch is off and no provider is named: the capability tier+ // answers, exactly as it did before this feature existed.+ const config = searchConfig(f.configPath, {+ enabled: false,+ provider: "openai-compat",+ baseUrl: null,+ model: null,+ apiKey: null,+ dimension: null,+ timeoutMs: 10_000,+ concurrency: 4,+ batchSize: 32,+ costGateUsd: 0,+ maxRetries: 3,+ });+ const outcome = await runSemanticPhase(readyStore(), config, "q", {+ limit: 10,+ pathPrefix: undefined,+ explicit: false,+ });+ expect(outcome.attempted).toBe(false);+ expect(outcome.warnings.length).toBe(1);+ // The registry sentence, not a sentence assembled at the call site.+ expect(outcome.warnings[0]).toBe(+ "semantic search not configured (disabled by config or provider)",+ );+ } finally {+ f.cleanup();+ }+ });+});diff --git a/tests/core/search/relational-arm.test.ts b/tests/core/search/relational-arm.test.tsindex 0e5552d7..8af4832f 100644--- a/tests/core/search/relational-arm.test.ts+++ b/tests/core/search/relational-arm.test.ts@@ -175,6 +175,11 @@ function armFixtureStore(opts: {documentTitles() {return new Map([...byId].map(([id, path]) => [id, { path, title: null }]));},+ // The real Store always carries the measured-visibility reader; the+ // fixture pages carry no indexed tags, which is the empty map.+ indexedVisibilityByPaths(paths: ReadonlyArray<string>) {+ return new Map(paths.map((p) => [p, [] as string[]]));+ },typedRelationEdgesForDocuments(ids: ReadonlyArray<number>) {const set = new Set(ids);return opts.typeddiff --git a/tests/core/search/relational-path.test.ts b/tests/core/search/relational-path.test.tsindex 8ab07563..779da258 100644--- a/tests/core/search/relational-path.test.ts+++ b/tests/core/search/relational-path.test.ts@@ -120,6 +120,28 @@ test("path nodes unreadable at the caller's reach are omitted, never named and n}});+test("a page whose live file lost its reserved token but whose indexed column still carries it is not named by the relational arm at remote reach", async () => {+ await buildPathVault();+ writeMd(vault, "mid.md", '---\nvisibility: ["private"]\nextends: "[[far]]"\n---\n\nbeta mid.');+ await indexVault(makeConfig({ vault, dbPath }));+ // Remove the private reservation from the live file, without reindexing.+ writeMd(vault, "mid.md", '---\nextends: "[[far]]"\n---\n\nbeta mid.');+ const store = await Store.open(makeConfig({ vault, dbPath }), { mode: "write", loadVec: false });+ try {+ const remote = runRelationalArm(store, vault, "[[seed]] related extends", {+ reach: TRANSPORT_REACH.remote,+ });+ const farEntry = [...remote.reachByChunk].find(([, reach]) => reach.hops === 2);+ expect(farEntry).toBeDefined();+ // The middle node is still withheld because its indexed column carries the reserved token;+ // it must not be named in the path provenance at remote reach.+ const pathSteps = farEntry ? farEntry[1].path : [];+ expect(pathSteps).toEqual([{ documentId: farEntry ? farEntry[0] : 0, relation: "extends" }]);+ } finally {+ await store.close();+ }+});+test("an absent reach behaves like remote: provenance never names a page the row gate withholds", async () => {// The row-level filters resolve an absent reach to remote// (`resolvedTransportReach`), so the arm's per-node provenance gatediff --git a/tests/core/search/schema-migration-v10.test.ts b/tests/core/search/schema-migration-v10.test.tsindex 3b2b88a7..4211ea2c 100644--- a/tests/core/search/schema-migration-v10.test.ts+++ b/tests/core/search/schema-migration-v10.test.ts@@ -37,10 +37,11 @@ function hasColumn(db: Database, table: string, name: string): boolean {.some((c) => c.name === name);}-// v14 (recall-quality): the virtual chunks.chunk_kind column, the taxonomy-// indexes and the document_attributes side table joined the latest version.-test("latest schema version is 14", () => {- expect(LATEST_SCHEMA_VERSION).toBe(14);+// v15 (write-honesty): the chunk_embedding_holds table joined the latest+// version (v14 (recall-quality) had added the virtual chunks.chunk_kind+// column, the taxonomy indexes and the document_attributes side table).+test("latest schema version is 15", () => {+ expect(LATEST_SCHEMA_VERSION).toBe(15);});test("fresh migration reaches latest with the authored_at column", () => {diff --git a/tests/core/search/schema-v14.test.ts b/tests/core/search/schema-v14.test.tsindex cf06c09e..15d7fbfc 100644--- a/tests/core/search/schema-v14.test.ts+++ b/tests/core/search/schema-v14.test.ts@@ -214,10 +214,10 @@ function attributesFor(path: string): ReadonlyArray<{ key: string; value: string}describe("the v14 entry", () => {- test("appends to MIGRATIONS as the latest version", () => {- expect(LATEST_SCHEMA_VERSION).toBe(14);- expect(MIGRATIONS.length).toBe(14);- expect(MIGRATIONS[MIGRATIONS.length - 1]!.version).toBe(14);+ test("remains in the frozen array once the v15 entry appends after it", () => {+ expect(LATEST_SCHEMA_VERSION).toBe(15);+ expect(MIGRATIONS.length).toBe(15);+ expect(MIGRATIONS[MIGRATIONS.length - 2]!.version).toBe(14);});test("runs from a rewound v13 index and stamps the version", async () => {@@ -226,7 +226,7 @@ describe("the v14 entry", () => {expect(migrate()).toBe(LATEST_SCHEMA_VERSION);const db = new Database(dbPath, { readonly: true });try {- expect(readSchemaVersion(db)).toBe(14);+ expect(readSchemaVersion(db)).toBe(15);} finally {db.close();}diff --git a/tests/core/search/schema-v15.test.ts b/tests/core/search/schema-v15.test.tsnew file mode 100644index 00000000..35787cc8--- /dev/null+++ b/tests/core/search/schema-v15.test.ts@@ -0,0 +1,196 @@+/**+ * Schema v15 (chunk skip-and-hold, Lane B task B2): the persistent hold+ * record.+ *+ * The v15 entry appends to the frozen `MIGRATIONS` array and runs inside+ * `applyMigrations`' single open transaction. It adds one table:+ *+ * - `chunk_embedding_holds(chunk_id, model, window_tokens,+ * embedding_signature, estimated_tokens, held_at)` - one row per+ * chunk the embedding phase proved over a model's declared input+ * window, under the (model, window, signature) key that made the+ * proof. The pending-vector anti-join excludes only rows matching+ * the CURRENT key, so a window or model change releases the hold back+ * into pending without any migration; a chunk deleted or replaced at+ * its document loses its holds through the FK cascade, exactly like+ * the v14 attribute rows lose theirs with the document.+ *+ * The migration backfills NOTHING: a hold is a decision about a config+ * (window + signature), and a migration has no config to consult - an+ * index predating the feature simply keeps its chunks pending until the+ * next embedding pass decides (and records) for them.+ */++import { Database } from "bun:sqlite";+import { afterEach, beforeEach, describe, expect, test } from "bun:test";++import { indexVault } from "../../../src/core/search/indexer.ts";+import { Store } from "../../../src/core/search/store.ts";+import {+ CHUNK_HOLDS_TABLE,+ LATEST_SCHEMA_VERSION,+ MIGRATIONS,+ applyMigrations,+ findSchemaGaps,+ readSchemaVersion,+} from "../../../src/core/search/schema.ts";+import { createTempVault, makeConfig, writeMd } from "../../helpers/search-fixtures.ts";++const PRE_V15_SCHEMA_VERSION = 14;++let vault: string;+let dbPath: string;+let cleanup: () => void;++beforeEach(() => {+ ({ vault, dbPath, cleanup } = createTempVault("schema-v15"));+});+afterEach(() => cleanup());++async function seedVault(): Promise<void> {+ writeMd(vault, "note.md", "# Note\n\nlattice widget note body");+ await indexVault(makeConfig({ vault, dbPath }));+}++/** Drop the v15 artifact and stamp the version a pre-v15 binary wrote. */+function rewindToPreV15Schema(): void {+ const db = new Database(dbPath);+ try {+ db.exec(`DROP TABLE IF EXISTS ${CHUNK_HOLDS_TABLE}`);+ db.query("UPDATE index_state SET value = ? WHERE key = 'schema_version'").run(+ String(PRE_V15_SCHEMA_VERSION),+ );+ } finally {+ db.close();+ }+}++function migrate(): number {+ const db = new Database(dbPath);+ try {+ return applyMigrations(db);+ } finally {+ db.close();+ }+}++function tableSql(): string | null {+ const db = new Database(dbPath, { readonly: true });+ try {+ // The name is this module's own constant, not input.+ return (+ db+ .query<{ sql: string | null }, []>(+ `SELECT sql FROM sqlite_master WHERE type = 'table' AND name = '${CHUNK_HOLDS_TABLE}'`,+ )+ .get()?.sql ?? null+ );+ } finally {+ db.close();+ }+}++describe("the v15 entry", () => {+ test("appends to MIGRATIONS as the latest version", () => {+ expect(LATEST_SCHEMA_VERSION).toBe(15);+ expect(MIGRATIONS.length).toBe(15);+ expect(MIGRATIONS[MIGRATIONS.length - 1]!.version).toBe(15);+ });++ test("runs from a rewound v14 index and stamps the version", async () => {+ await seedVault();+ rewindToPreV15Schema();+ expect(migrate()).toBe(LATEST_SCHEMA_VERSION);+ const db = new Database(dbPath, { readonly: true });+ try {+ expect(readSchemaVersion(db)).toBe(15);+ } finally {+ db.close();+ }+ });++ test("a rewound index reports the hold table as a schema gap until migrated", async () => {+ await seedVault();+ rewindToPreV15Schema();+ const db = new Database(dbPath);+ try {+ const gapsBefore = findSchemaGaps(db).map((g) => g.table);+ expect(gapsBefore).toContain(CHUNK_HOLDS_TABLE);+ applyMigrations(db);+ expect(findSchemaGaps(db)).toEqual([]);+ } finally {+ db.close();+ }+ });++ test("idempotent: migrating a current index changes nothing", async () => {+ await seedVault();+ expect(migrate()).toBe(LATEST_SCHEMA_VERSION);+ const first = tableSql();+ expect(migrate()).toBe(LATEST_SCHEMA_VERSION);+ expect(tableSql()).toBe(first);+ });+});++describe("the hold table shape", () => {+ test("names the key triple, the estimate, the timestamp and the chunk FK", async () => {+ await seedVault();+ migrate();+ const db = new Database(dbPath, { readonly: true });+ try {+ const cols = db+ .query<{ name: string }, [string]>("SELECT name FROM pragma_table_info(?)")+ .all(CHUNK_HOLDS_TABLE)+ .map((c) => c.name);+ expect(cols).toContain("chunk_id");+ expect(cols).toContain("model");+ expect(cols).toContain("window_tokens");+ expect(cols).toContain("embedding_signature");+ expect(cols).toContain("estimated_tokens");+ expect(cols).toContain("held_at");+ } finally {+ db.close();+ }+ const sql = tableSql() ?? "";+ // The FK clause and its cascade are load-bearing: a re-index replaces+ // chunk rows, and a hold the chunk no longer has must vanish with it.+ expect(sql).toContain(`REFERENCES chunks(id) ON DELETE CASCADE`);+ expect(sql).toContain("PRIMARY KEY");+ });++ test("a replaced chunk row takes its holds with it (FK cascade)", async () => {+ await seedVault();+ migrate();+ const config = makeConfig({ vault, dbPath });+ const probe = new Database(dbPath);+ let chunkId = 0;+ let docId = 0;+ try {+ const row = probe+ .query<{ id: number; document_id: number }, []>(+ "SELECT id, document_id FROM chunks LIMIT 1",+ )+ .get();+ chunkId = row!.id;+ docId = row!.document_id;+ } finally {+ probe.close();+ }+ const store = await Store.open(config, { mode: "write" });+ try {+ store.recordChunkHolds([{ chunkId, estimatedTokens: 999, windowTokens: 512 }], {+ model: "test-model",+ embeddingSignature: "openai-compat:test-model:?",+ });+ expect(store.chunkHoldRowsFor(chunkId).length).toBe(1);+ // Re-indexing a document rewrites its chunks wholesale; the old rows+ // die, and their holds must die with them rather than key a ghost.+ // The cascade is enforced on the Store's own connection, which opens+ // with `PRAGMA foreign_keys = ON`.+ store.replaceChunks(docId, []);+ expect(store.chunkHoldRowsFor(chunkId)).toEqual([]);+ } finally {+ await store.close();+ }+ });+});diff --git a/tests/core/search/visibility-column-backfill.test.ts b/tests/core/search/visibility-column-backfill.test.tsindex ba397c31..ea8ca5ef 100644--- a/tests/core/search/visibility-column-backfill.test.ts+++ b/tests/core/search/visibility-column-backfill.test.ts@@ -165,6 +165,23 @@ describe("the migration over a pre-column index", () => {expect(migrate()).toBe(LATEST_SCHEMA_VERSION);expect(visibilityByPath()).toEqual(first);});++ test("a marker page's backfilled row keeps the reservation the indexer would stamp", async () => {+ // write-honesty F1: `access: private` and friends reserve a page+ // against remote reads. The indexer stamps that through+ // `remoteDenyVisibilityTokens`; a migration backfill reading only the+ // declared `visibility:` would silently UN-reserve the page for every+ // row it touches, which is the opposite of what the marker declares.+ writeMd(vault, "marked.md", "---\naccess: private\n---\n\n# Marked\n\nlattice widget marked");+ writeMd(vault, "plain.md", "# Plain\n\nlattice widget ordinary");+ await indexVault(makeConfig({ vault, dbPath }));+ rewindToPreColumnSchema();+ migrate();+ expect(visibilityByPath().get("marked.md")).toBe(+ JSON.stringify([REMOTE_DENY_VISIBILITY_TOKEN]),+ );+ expect(visibilityByPath().get("plain.md")).toBe("[]");+ });});describe("the indexer", () => {diff --git a/tests/core/search/visibility-filter.test.ts b/tests/core/search/visibility-filter.test.tsindex 7ea3277f..c6961217 100644--- a/tests/core/search/visibility-filter.test.ts+++ b/tests/core/search/visibility-filter.test.ts@@ -234,3 +234,109 @@ describe("visibility scoping in search", () => {expect(out.results.map((r) => r.path).toSorted()).toEqual(["a.md", "b.md"]);});});++// --- Informal privacy markers (write-honesty, F1) ----------------------------+//+// A page saying `access: private` used to resolve to default visibility and+// be served to a remote caller. The read roots now union the reserved token+// into the measured tokens, and the indexer stamps that union into the+// column, so the snippet is withheld even after the file changes.++describe("informal privacy markers in search", () => {+ let vault: string;+ let dbPath: string;+ let cleanup: () => void;++ beforeEach(() => {+ ({ vault, dbPath, cleanup } = createTempVault("privacy-markers"));+ });+ afterEach(() => cleanup());++ test("a marker-only page is remote-denied and locally readable", async () => {+ writeMd(vault, "open.md", "# Open\n\nlattice widget open");+ writeMd(vault, "access.md", "---\naccess: private\n---\n# Access\n\nlattice widget marked");+ const cfg = makeConfig({ vault, dbPath });+ await indexVault(cfg);++ const remote = await search(cfg, {+ query: "lattice widget",+ limit: 10,+ transportReach: TRANSPORT_REACH.remote,+ });+ expect(remote.results.map((r) => r.path).toSorted()).toEqual(["open.md"]);++ const local = await search(cfg, {+ query: "lattice widget",+ limit: 10,+ transportReach: TRANSPORT_REACH.local,+ });+ expect(local.results.map((r) => r.path).toSorted()).toEqual(["access.md", "open.md"]);+ });++ test("asking for the reserved scope does not lift an informal marker", async () => {+ writeMd(vault, "open.md", "# Open\n\nlattice widget open");+ writeMd(+ vault,+ "confidential.md",+ "---\nconfidentiality: Secret\n---\n# Confidential\n\nlattice widget marked",+ );+ const cfg = makeConfig({ vault, dbPath });+ await indexVault(cfg);++ const asked = await search(cfg, {+ query: "lattice widget",+ limit: 10,+ visibility: [REMOTE_DENY_VISIBILITY_TOKEN],+ transportReach: TRANSPORT_REACH.remote,+ });+ expect(asked.results.map((r) => r.path).toSorted()).toEqual(["open.md"]);+ });++ test("a marker beside visibility: team is still remote-denied", async () => {+ writeMd(vault, "open.md", "# Open\n\nlattice widget open");+ writeMd(+ vault,+ "team.md",+ "---\nsensitivity: confidential\nvisibility: [team]\n---\n# Team\n\nlattice widget marked",+ );+ const cfg = makeConfig({ vault, dbPath });+ await indexVault(cfg);++ const remote = await search(cfg, {+ query: "lattice widget",+ limit: 10,+ transportReach: TRANSPORT_REACH.remote,+ });+ expect(remote.results.map((r) => r.path).toSorted()).toEqual(["open.md"]);+ });++ test("the indexed union keeps the snippet reserved after the file is rewritten", async () => {+ // The page was reserved when its bytes were captured. A later rewrite+ // that drops the marker must not make the indexed snippet readable.+ writeMd(vault, "marked.md", "---\naccess: private\n---\n\nlattice widget captured");+ const cfg = makeConfig({ vault, dbPath });+ await indexVault(cfg);++ writeMd(vault, "marked.md", "# Rewritten\n\nlattice widget captured");+ const remote = await search(cfg, {+ query: "lattice widget",+ limit: 10,+ transportReach: TRANSPORT_REACH.remote,+ });+ expect(remote.results.map((r) => r.path)).toEqual([]);+ });++ test("a local-reach corpus without markers is byte-identical in ranking", async () => {+ // The control for the whole feature: the marker rule adds no row, no+ // score and no order on vaults that never wrote a marker key.+ writeMd(vault, "a.md", "# A\n\nlattice widget alpha");+ writeMd(vault, "b.md", "# B\n\nlattice widget beta");+ const cfg = makeConfig({ vault, dbPath });+ await indexVault(cfg);++ for (const reach of [TRANSPORT_REACH.local, TRANSPORT_REACH.remote] as const) {+ const out = await search(cfg, { query: "lattice widget", limit: 10, transportReach: reach });+ expect(out.results.map((r) => r.path).toSorted()).toEqual(["a.md", "b.md"]);+ }+ });+});diff --git a/tests/mcp/brain-create-note.test.ts b/tests/mcp/brain-create-note.test.tsindex a241fd25..32aadce4 100644--- a/tests/mcp/brain-create-note.test.ts+++ b/tests/mcp/brain-create-note.test.ts@@ -16,7 +16,7 @@ import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";import { NOTES_TOOLS } from "../../src/mcp/brain/notes-tools.ts";import { WRITE_BATCH_TOOLS } from "../../src/mcp/brain/write-batch-tools.ts";import { resolveNextStep } from "../../src/core/brain/next-step.ts";-import { PAGE_LINT_KEY } from "../../src/core/brain/page-lint.ts";+import { PAGE_LINT_KEY, UNDATED_QUANTITY_CODE } from "../../src/core/brain/page-lint.ts";import { resolveSearchConfig } from "../../src/core/search/index.ts";import { indexVault } from "../../src/core/search/indexer.ts";import { WRITE_BINDING_REFUSED_CODE } from "../../src/core/write-binding/index.ts";@@ -478,6 +478,55 @@ describe("brain_create_note - the lint attached to the receipt", () => {});});+/**+ * Undated-quantity lint at the receipt boundary (write-honesty, F4,+ * t_fb442a24). The write SUCCEEDS - the finding is advisory - and the+ * receipt carries the new code. The same write on a page that declares+ * `valid_from` is byte-identical to the clean receipt: no lint key at all.+ */+describe("brain_create_note - the undated-quantity lint", () => {+ const call = async (args: Record<string, unknown>): Promise<Record<string, unknown>> =>+ (await handler(ctx, args)) as Record<string, unknown>;++ test("an undated numeric write succeeds and carries the finding", async () => {+ const res = await call({+ path: "Notes/UndatedNumeric.md",+ frontmatter: { title: "Undated" },+ content: "We onboarded 5,000 users.",+ });+ expect(res).toMatchObject({ created: true, outcome: "created" });+ expect(existsSync(join(vault, "Notes/UndatedNumeric.md"))).toBe(true);+ const lint = res[PAGE_LINT_KEY] as { findings: ReadonlyArray<Record<string, unknown>> };+ expect(lint.findings).toEqual([+ expect.objectContaining({+ severity: "warning",+ code: UNDATED_QUANTITY_CODE,+ page: "Notes/UndatedNumeric.md",+ path: "body",+ }),+ ]);+ expect(typeof lint.findings[0]!["next_command"]).toBe("string");+ });++ test("the same write with valid_from keeps the byte-identical clean receipt", async () => {+ const res = await call({+ path: "Notes/DatedNumeric.md",+ frontmatter: { title: "Dated", valid_from: "2026-10-01" },+ content: "We onboarded 5,000 users.",+ });+ expect(res).toMatchObject({ created: true, outcome: "created" });+ expect(PAGE_LINT_KEY in res).toBe(false);+ expect(Object.keys(res)).toEqual([+ "created",+ "outcome",+ "path",+ "write_id",+ "index_refresh",+ "index_refresh_reason",+ ]);+ });+});+function enableWidening(): void {appendFileSync(ctx.configPath!, "near_duplicate_write_widening_enabled: true\n");}diff --git a/tests/mcp/brain-update-append-note.test.ts b/tests/mcp/brain-update-append-note.test.tsindex 482f9da1..3c5b884a 100644--- a/tests/mcp/brain-update-append-note.test.ts+++ b/tests/mcp/brain-update-append-note.test.ts@@ -24,7 +24,7 @@ import { bootstrapBrain } from "../../src/core/brain/init.ts";import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";import { NOTES_TOOLS, writeBatchErrorToMcp } from "../../src/mcp/brain/notes-tools.ts";import { WriteBatchError } from "../../src/core/brain/write-batch.ts";-import { PAGE_LINT_KEY } from "../../src/core/brain/page-lint.ts";+import { PAGE_LINT_KEY, UNDATED_QUANTITY_CODE } from "../../src/core/brain/page-lint.ts";import { INTERNAL_ERROR, INVALID_PARAMS, MCPError } from "../../src/mcp/protocol.ts";import { TRANSPORT_REACH } from "../../src/core/graph/transport-reach.ts";import { resolveSearchConfig } from "../../src/core/search/index.ts";@@ -623,3 +623,84 @@ describe("the lint attached to the update / append receipts", () => {expect(readFileSync(join(vault, "Notes/App.md"), "utf8")).toContain("[[pref-ghost]]");});});++/**+ * Undated-quantity lint on update and append receipts (write-honesty, F4,+ * t_fb442a24). Both writes succeed; the advisory rides the receipt with the+ * new code. A page already declaring `valid_from` keeps the clean receipt,+ * with no lint key at all.+ */+describe("the undated-quantity lint on update / append receipts", () => {+ const call = async (+ tool: typeof updateTool,+ args: Record<string, unknown>,+ ): Promise<Record<string, unknown>> => (await tool.handler(ctx, args)) as Record<string, unknown>;++ test("an undated numeric update succeeds and carries the finding", async () => {+ seedNote("Notes/UndatedUpdate.md", "old body", "title: U");+ const res = await call(updateTool, {+ path: "Notes/UndatedUpdate.md",+ content: "We onboarded 5,000 users.",+ });+ expect(res).toMatchObject({ updated: true, path: "Notes/UndatedUpdate.md" });+ const lint = res[PAGE_LINT_KEY] as { findings: ReadonlyArray<Record<string, unknown>> };+ expect(lint.findings).toEqual([+ expect.objectContaining({+ severity: "warning",+ code: UNDATED_QUANTITY_CODE,+ page: "Notes/UndatedUpdate.md",+ path: "body",+ }),+ ]);+ });++ test("an update onto a page with valid_from keeps the clean receipt", async () => {+ seedNote("Notes/DatedUpdate.md", "old body", "title: D\nvalid_from: 2026-10-01");+ const res = await call(updateTool, {+ path: "Notes/DatedUpdate.md",+ content: "We onboarded 5,000 users.",+ });+ expect(PAGE_LINT_KEY in res).toBe(false);+ expect(Object.keys(res)).toEqual([+ "updated",+ "path",+ "write_id",+ "index_refresh",+ "index_refresh_reason",+ ]);+ });++ test("an undated numeric append succeeds and carries the finding", async () => {+ seedNote("Notes/UndatedAppend.md", "first", "title: A");+ const res = await call(appendTool, {+ path: "Notes/UndatedAppend.md",+ content: "We onboarded 5,000 users.",+ });+ expect(res).toMatchObject({ appended: true, path: "Notes/UndatedAppend.md" });+ const lint = res[PAGE_LINT_KEY] as { findings: ReadonlyArray<Record<string, unknown>> };+ expect(lint.findings).toEqual([+ expect.objectContaining({+ severity: "warning",+ code: UNDATED_QUANTITY_CODE,+ page: "Notes/UndatedAppend.md",+ path: "body",+ }),+ ]);+ });++ test("an append onto a page with valid_from keeps the clean receipt", async () => {+ seedNote("Notes/DatedAppend.md", "first", "title: D\nvalid_from: 2026-10-01");+ const res = await call(appendTool, {+ path: "Notes/DatedAppend.md",+ content: "We onboarded 5,000 users.",+ });+ expect(PAGE_LINT_KEY in res).toBe(false);+ expect(Object.keys(res)).toEqual([+ "appended",+ "path",+ "write_id",+ "index_refresh",+ "index_refresh_reason",+ ]);+ });+});diff --git a/tests/mcp/brain-write-batch.test.ts b/tests/mcp/brain-write-batch.test.tsindex cd52f0ad..10fb31df 100644--- a/tests/mcp/brain-write-batch.test.ts+++ b/tests/mcp/brain-write-batch.test.ts@@ -39,7 +39,7 @@ import { WRITE_BATCH_TOOLS } from "../../src/mcp/brain/write-batch-tools.ts";import { NOTES_TOOLS } from "../../src/mcp/brain/notes-tools.ts";import { MAX_BATCH_OPERATIONS } from "../../src/core/brain/write-batch.ts";import * as ledger from "../../src/core/brain/idempotency-ledger.ts";-import { PAGE_LINT_KEY } from "../../src/core/brain/page-lint.ts";+import { PAGE_LINT_KEY, UNDATED_QUANTITY_CODE } from "../../src/core/brain/page-lint.ts";import { INVALID_PARAMS, MCPError } from "../../src/mcp/protocol.ts";import { TRANSPORT_REACH } from "../../src/core/graph/transport-reach.ts";import type { ServerContext } from "../../src/mcp/tool-contract.ts";@@ -392,6 +392,40 @@ describe("brain_write_batch - the lint attached to the receipt", () => {expect(second.results[0]).toMatchObject({ kind: "update_note", updated: false });expect(PAGE_LINT_KEY in second).toBe(false);});++ test("an undated numeric create reports the finding on the batch envelope", async () => {+ const res = await runBatch([+ {+ op: "create_note",+ path: "Notes/UndatedBatch.md",+ frontmatter: { title: "Undated" },+ content: "We onboarded 5,000 users.",+ },+ ]);+ expect(res["applied"]).toBe(1);+ const lint = res[PAGE_LINT_KEY]!;+ expect(lint.findings).toEqual([+ expect.objectContaining({+ severity: "warning",+ code: UNDATED_QUANTITY_CODE,+ page: "Notes/UndatedBatch.md",+ path: "body",+ }),+ ]);+ });++ test("the same batch create with valid_from keeps the clean envelope", async () => {+ const res = await runBatch([+ {+ op: "create_note",+ path: "Notes/DatedBatch.md",+ frontmatter: { title: "Dated", valid_from: "2026-10-01" },+ content: "We onboarded 5,000 users.",+ },+ ]);+ expect(Object.keys(res)).toEqual(["applied", "results", "done"]);+ expect(PAGE_LINT_KEY in res).toBe(false);+ });});/**diff --git a/tests/mcp/search-status-held-chunks.test.ts b/tests/mcp/search-status-held-chunks.test.tsnew file mode 100644index 00000000..8d0b1127--- /dev/null+++ b/tests/mcp/search-status-held-chunks.test.ts@@ -0,0 +1,98 @@+/**+ * The `search.*` status block over MCP names held chunks by COUNT, not+ * by path. The items list is an operator diagnostic - `o2b search status`+ * and the doctor show it at local reach - while `second_brain_status`+ * reaches callers a transport may serve at `remote` reach, and a held+ * chunk's path is exactly the location a reservation may want kept off+ * the wire. The total travels; the paths stay local.+ */++import { afterEach, beforeEach, expect, test } from "bun:test";+import { Database } from "bun:sqlite";+import { join } from "node:path";++import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";+import { indexStatus } from "../../src/core/search/indexer.ts";+import { resolveSearchConfig } from "../../src/core/search/index.ts";+import { Store } from "../../src/core/search/store.ts";+import { chunkHoldExclusion } from "../../src/core/search/embedding-spend.ts";+import {+ declaredInputWindowTokens,+ RECOMMENDED_EMBEDDING_MODEL,+} from "../../src/core/search/embeddings/presets.ts";+import { buildSearchStatusBlock } from "../../src/mcp/search-tools.ts";+import { createTempVault } from "../helpers/search-fixtures.ts";+import { FAKE_PROVIDER_KEY } from "../helpers/fake-credentials.ts";++const WINDOW = declaredInputWindowTokens(RECOMMENDED_EMBEDDING_MODEL)!;++let tmp: ReturnType<typeof createTempVault>;+let configPath: string;++beforeEach(() => {+ tmp = createTempVault("mcp-held-chunks");+ configPath = join(tmp.vault, "config.yaml");+ atomicWriteFileSync(+ configPath,+ `vault: ${tmp.vault}\n` ++ "search_semantic_enabled: true\n" ++ "embedding_provider: openai-compat\n" ++ "embedding_base_url: https://mcp-held.invalid/v1\n" ++ `embedding_model: ${RECOMMENDED_EMBEDDING_MODEL}\n` ++ `embedding_api_key: ${FAKE_PROVIDER_KEY}\n` ++ "embedding_dimension: 4\n",+ );+});++afterEach(() => tmp.cleanup());++test("the MCP status block reports held chunks as a total, without item paths", async () => {+ const config = resolveSearchConfig({ vault: tmp.vault, configPath });+ const store = await Store.open(config, { mode: "write" });+ let chunkId: number;+ try {+ const docId = store.upsertDocument({+ path: "Brain/note.md",+ title: "note",+ contentHash: "h1",+ mtime: 1_700_000_000,+ size: 10,+ });+ store.replaceChunks(docId, [+ {+ chunkIndex: 0,+ content: "a chunk",+ contentHash: "h1-0",+ startLine: 1,+ endLine: 2,+ tokenCount: 2,+ },+ ]);+ const db = new Database(config.dbPath);+ try {+ chunkId = db.query<{ id: number }, []>("SELECT id FROM chunks ORDER BY id LIMIT 1").get()!.id;+ } finally {+ db.close();+ }+ const exclusion = chunkHoldExclusion(config)!;+ store.recordChunkHolds([{ chunkId, estimatedTokens: WINDOW + 1, windowTokens: WINDOW }], {+ model: exclusion.model,+ embeddingSignature: exclusion.embeddingSignature,+ });+ } finally {+ await store.close();+ }+ // The operator surface still names the paths.+ const status = await indexStatus(config);+ expect(status.heldChunks?.total).toBe(1);+ expect(status.heldChunks?.items[0]!.path).toBe("Brain/note.md");++ const block = (await buildSearchStatusBlock({ vault: tmp.vault, configPath } as never)) as Record<+ string,+ unknown+ >;+ const held = block["held_chunks"] as { total: number; items?: unknown } | undefined;+ expect(held).toBeDefined();+ expect(held!.total).toBe(1);+ expect(held!.items).toBeUndefined();+});diff --git a/uv.lock b/uv.lockindex 7e1d4368..6228650a 100644--- a/uv.lock+++ b/uv.lock@@ -4,5 +4,5 @@ requires-python = ">=3.11"[[package]]name = "open-second-brain"-version = "1.91.0"+version = "1.92.0"source = { editable = "." }