Skip to content

Local-first 🧠 memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly 😴 dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.

TypeScript

feat: distillation pages that prove what they quote and say how much of their source the vault holds (v1.67.0)

Serg Eroshe400b20ea5315
diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json
index de5414ff..0afb2399 100644
--- a/.claude-plugin/plugin.json
+++ b/.claude-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "1.66.0",
+ "version": "1.67.0",
"description": "Plugin-first second brain package for AI agents and humans.",
"author": {
"name": "Open Second Brain contributors"
diff --git a/.codex-plugin/plugin.json b/.codex-plugin/plugin.json
index 5c2e3cd8..a03872b9 100644
--- a/.codex-plugin/plugin.json
+++ b/.codex-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "1.66.0",
+ "version": "1.67.0",
"description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.",
"author": {
"name": "Open Second Brain contributors",
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 61c1a053..3e3d2d5f 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,6 +5,46 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+## [1.67.0] - 2026-10-02
+
+A distillation page now proves what it quotes and says how much of its source the vault holds: every quoted span in a claim is checked against the block it cites, or the whole source, at write time, a span that does not verify loses its quotation marks and is named in the result (or, on request, refuses the write with `quote_unverified`), and distillation, ingest summary and research report pages carry a `capture_scope` of `full-local`, `bounded-local` or `url-only`, with a default-on hygiene detector that warns when active knowledge rests only on `url-only` sources. Every link and mention reader now masks a fenced block up to its own closing fence, and a source the caller cannot read at its reach is treated as absent wherever a page cites one.
+
+### Added
+
+- **Quotes in distilled claims are verified at write time.** `brain_distill_source` and `o2b brain distill` compare every span between quotation marks with the block the claim cites, or with the whole source when it cites none. Quotation marks are recognised by the Unicode `Quotation_Mark` property and paired by position, so no list of languages is involved: corner brackets in CJK running text pair, spaced guillemets pair across one no-break, narrow no-break or thin space, an apostrophe inside a word or closing a plural possessive (`the students' work`) is never a delimiter, an unclosed leading apostrophe (`'90s`) hides no later quote, and a quoted run with fewer than two letters or digits (`rock 'n' roll`) is not a quote. Both sides are normalised the same way for the comparison only (NFC, paired inline Markdown delimiters reduced to their text, a trailing `^id` removed, quote variants folded, whitespace collapsed), and line-leading block markers are removed on the evidence side only, so a span's own leading `3. `, `- ` or `> ` is wording to match; case, punctuation and wording are never folded. A span must start and end on a word boundary of the evidence by the Unicode word rules, so `"safe"` does not verify inside `unsafe`, and an ellipsis splits a span into fragments that must appear in order, with an edge next to the ellipsis free to cut a word. A block-cited claim is never checked against the whole source: an unknown block id is `block-not-found`, a duplicate `block-ambiguous`. The comparison and block resolution run in linear time, and a source's blocks are indexed once per write.
+- **A failed quote is unquoted and named.** A span that does not verify loses its two quotation marks, the words stay and the write lands; the result carries a `quotes` report (`checked`, `verified_in_block`, `verified_in_source`, `unquoted`, `unpaired`, capped `findings` with claim index, outcome and a bounded span preview). The page records `quotes_verified` and `quotes_unquoted` in its frontmatter when a claim holds a span. Pages without quoted spans are unchanged.
+- **Strict quote refusal.** `strict_quotes: true` on `brain_distill_source` and `--strict-quotes` on `o2b brain distill` refuse the whole write when any span would be unquoted, and nothing is written. The MCP refusal is `INVALID_PARAMS` with `error.data.code` set to the new registered code `quote_unverified`; its message names claim indices and outcomes, never claim text. The CLI exits 1.
+- **Capture scope on distilled, ingested and researched pages.** Distillation pages, ingest summary pages and research reports record how much of their source the vault holds: `full-local` (a file in the vault), `bounded-local` (no local bytes, but the page stores a verbatim excerpt with its digest) or `url-only` (a URL, a path with no file behind it, or a vault file the caller cannot read at its reach). An extensionless note source (`notes/meeting`, `[[Meeting]]`) is `full-local` when its `.md` note exists. The stamp (`capture_scope`, or `capture_scopes` per source on a research report) is written only when it is not `full-local`, so pages over local sources stay byte-identical. `brain_distill_source` and `brain_ingest_source` return `capture_scope`, `brain_research_report` returns `capture_scopes` in source order, and `o2b brain distill` prints `[bounded-local]` (a `url-only` source already prints `[untrusted_source]`) and `[quotes verified:V unquoted:U]`, with `--json` carrying `capture_scope` and `quotes`.
+- **A bounded excerpt for sources the vault does not hold.** `excerpt` on `brain_distill_source` (with a declared `maxLength`) and `--excerpt-file` on `o2b brain distill` store the text the caller read from a `url-only` source in a fenced `## Excerpt` section with an `excerpt_hash`, make the page `bounded-local`, and check its quotes against that excerpt. An excerpt for a local source, one that is blank or made only of control, format or whitespace characters, one holding NUL, or one over the byte cap is refused by name before anything is written; an `--excerpt-file` that is not valid UTF-8 exits 2 with `distill: excerpt file is not valid UTF-8`. A stored excerpt contributes no links or tags to the page that holds it.
+- **The `capture-scope` hygiene detector.** A default-on `brain_hygiene` detector warns (`proposed_action: review`) when a retrievable page of active knowledge (a distillation, an ingest summary, a research report or a canonical entity) rests only on sources that are `url-only` at scan time. It reads only the page's own last `## Sources` section, outside fenced blocks, and re-derives each cited source's scope from its identity at the caller's reach without reading bytes, so a deleted local source surfaces too; a `bounded-local` page counts as backed only while its excerpt still matches its digest, quarantined pages are skipped, and a cited source the filesystem refuses to stat counts as backing rather than ending the sweep. The `brain_status` hygiene count includes it.
+
+### Changed
+
+- **Quotation marks in distilled claims may now be removed.** A distillation whose claim quotes a span that the cited block or the source does not hold now lands with that span's quotation marks removed, and the result's `quotes` report names it; before, any quoted text was written as given. Set `strict_quotes` (or `--strict-quotes`) to refuse such a write instead.
+- **New frontmatter keys appear only on pages that are not `full-local`.** `capture_scope`, `capture_scopes` and `excerpt_hash` are written for a `bounded-local` or `url-only` source; `quotes_verified` and `quotes_unquoted` only when a claim holds a quoted span. An existing distillation, ingest summary or research report over local sources is written byte-identical to before.
+- **A claim or a research title that spans more than one line is refused.** `brain_distill_source` and `o2b brain distill` refuse a claim that contains a line break (a claim is one bullet line) and accept at most 1000 claims per call; `brain_research_report` refuses a title that holds a line break (a title is the page's one heading line). Both used to be written as given.
+- **A fenced block is masked up to its own closing fence.** Wikilink extraction (backlinks, unlinked mentions, the repair lane, the wikilink formatter) and search indexing of links and tags close a fenced block only on a run of its own character at least as long, the CommonMark rule, and stay linear on long runs. Links inside a tilde fence are no longer extracted, and a longer fence holding a shorter backtick run no longer leaks the rest of its content as links. An inline code span closes only on a backtick run of its own length, and a backtick run with no closer is plain text, so its last mark no longer masks the real links between it and an unrelated later backtick. A vault with such blocks may see different backlinks and link hits after the next index pass.
+- **The default hygiene sweep includes `capture-scope`.** A default `brain_hygiene` or `o2b brain hygiene` scan may report new warnings on an existing vault whose active knowledge cites only URLs; callers that pin the list of detectors a default scan runs see the new id at the end.
+- **A cited source is asked about at the caller's reach before it is read.** `brain_distill_source`, `brain_intake_entities`, `brain_ingest_source`, `brain_ingest_batch_plan` and `brain_research_report` decide whether the caller may read a cited source on its resolved vault-relative path before it is stat-ed or read; a source the caller cannot read at its reach is treated as absent (untrusted lane, no digest, `url-only`, not planned, not recorded in the content manifest or the plan checkpoint). A caller that may read it still gets the size-ceiling or read refusal. `brain_distill_source` checks quotes only against a source the caller may read (and, under `integrity.owner_scope_delivery: fail`, its owner scope).
+- **More tools treat a page the caller cannot read at its reach as absent.** The CLI is unchanged; at local reach every answer is as before.
+ - `brain_delete_by_source` leaves such an original out of the plan, `blast_radius`, `matched` and `recoverability`, and a confirmed run with `include_originals` leaves it on disk.
+ - `brain_note_lifecycle`, `brain_append_note`, `brain_update_note` and the update and append operations of `brain_write_batch` refuse such a page with the refusal a missing path gets (`source_missing`, `target_missing`), before any plan, reserved-key check or write; `brain_create_note` keeps its `note already exists` refusal on an occupied path and leaves the page untouched. At remote reach a note the host cannot read is also answered as missing; the `target_unreadable` diagnostic stays for local callers.
+ - `brain_note_lifecycle` names and counts only files the caller may read in its reference report (`inbound_files`, `files_scanned`, `files_rewritten`, `rewrite_failures`, the delete cascade's lists and the recoverability derived from the rewrite). A cascade never deletes a file the caller cannot read; an applied move or rename still rewrites the links inside such files and reports nothing about them.
+ - `brain_note_history` answers for such a page as for a path no commit touches (`commit_count: 0`, no phases), before any history is read.
+ - `brain_tiers check` lists drift rows only for pages the caller may read, and `restore` and `accept` refuse any other page as `not indexed` before reading or writing anything.
+ - `brain_event_trace` drops an event that names such a page before the `limit` and the `total` and `trace_total` counts, and drops an attached trace whose handoff reference is withheld; `brain_agent_query` and `brain_agent_diff` withhold the same contributions before the roster, topics and counts are built.
+ - `brain_diarize`, `brain_design_note` and the `brain_dream` rollup step leave such a page out of `link_candidates`, its `total` and the schema hint; `brain_idea_discovery` leaves it out of its walk before ranking and capping; `brain_procedural_memory` leaves such a procedure out of `list` and the `reconcile` counts, and `mark_used` and `mark_outcome` refuse its id as unknown before writing.
+ - `brain_writes plan_revert`, when it withholds an entry, returns a `digest` and `next_command` computed over the visible entries only; `o2b brain writes revert --apply` re-plans in full and refuses such a digest with `digest_mismatch`.
+ - The `brain_hygiene` scan's `link_integrity` block reports `measured: false` with reason `reach` below local reach, because its counts are taken over the whole index, and the `brain_status` hygiene problem count counts only the findings `brain_hygiene` would list for the same caller (owner scope and reach).
+ - `brain_doctor` and the `schema_inspect` `lint`, `orphans` and `stats` views leave out a Brain record the caller may not read, so a malformed private preference answers as an absent one; `brain_doctor` restates `ok`, `trust_verdict` and its tier-drift count over what it keeps.
+- **One source read feeds both the digest and the quote check.** A source is read through one descriptor that refuses a final-component symlink, a non-regular file and growth past the 8 MiB ceiling after the size check, so a distillation's `source_hash` and its quote verdicts always describe the same bytes; a source whose bytes are not valid UTF-8 verifies no quote (`source-not-text`) instead of being compared as mangled text.
+- **Docs:** `docs/mcp.md` gains a "Source distillation" section (inputs, the quote check and its tolerance, unquoting, strict refusal, the `quotes` report, capture scope and the excerpt) and the `quote_unverified` row in the error-code table; `docs/cli-reference.md` documents `o2b brain distill` with its flags and the new detector; `docs/how-it-works.md` gains "Checked quotes and capture scope"; `docs/updating.md` names what an existing vault sees after the upgrade; the README names this release.
+
+### Fixed
+
+- **`o2b brain distill` completions and `help --json` list its flags.** The command manifest modelled no flags for the verb, so shell completion and the machine-readable help offered none; all six are now listed, and the usage line names `--agent <name>`.
+- **A path that resolves outside the vault is answered as absent without being opened.** A caller-supplied path such as `../x.md` (for example to `brain_note_history` or `brain_tiers`), or a page reference a log entry carries (read by `brain_event_trace`, `brain_agent_query` and `brain_agent_diff`), is no longer read from beside the vault at remote reach; such a request used to wait for good on a named pipe. The answers are unchanged: such a path already read as one no commit touches.
+
## [1.66.0] - 2026-10-01
A failed MCP call now says what failed in a token a client can branch on, and the optional cross-encoder rerank says why it stepped aside: every JSON-RPC error answer carries `error.data.code` and every `isError` tool result carries `_meta["open-second-brain/error"].code`, both drawn from one closed registry with the error prose left byte for byte as it was; a failed rerank endpoint is recorded on the retrieval trail as `rerank-provider-unavailable` with a typed category and is never cached, a model past its announced decommission date is skipped without a request (`rerank-model-sunset`), and `o2b brain doctor` checks the rerank configuration before a search has to fail on it.
@@ -7952,6 +7992,7 @@ plugin config (vault field)`, and exits with a clear
- Sandbox vault and plugin manifest fixtures for tests.
- GitHub release workflow for tag-based and manually dispatched releases.
+[1.67.0]: https://github.com/itechmeat/open-second-brain/compare/v1.66.0...v1.67.0
[1.66.0]: https://github.com/itechmeat/open-second-brain/compare/v1.65.0...v1.66.0
[1.65.0]: https://github.com/itechmeat/open-second-brain/compare/v1.64.0...v1.65.0
[1.64.0]: https://github.com/itechmeat/open-second-brain/compare/v1.63.0...v1.64.0
diff --git a/README.md b/README.md
index 04442a9d..38dd5e09 100644
--- a/README.md
+++ b/README.md
@@ -113,7 +113,7 @@ The full router with readiness criteria is [`install.md`](install.md); native Wi
## What is new
-1.66.0 puts a stable error code on every failed MCP call, `error.data.code` on a JSON-RPC error and `_meta["open-second-brain/error"].code` on an `isError` result, drawn from one closed registry with the error prose unchanged, so a client branches on a token instead of matching English text. The optional cross-encoder rerank names why it stepped aside on the retrieval trail (`rerank-provider-unavailable` with a typed category, never cached, and `rerank-model-sunset` for a model past its announced decommission date), and `o2b brain doctor` checks the rerank configuration without sending a request. Every release is described in the [CHANGELOG](CHANGELOG.md).
+1.67.0 makes a distillation page prove what it quotes and say how much of its source the vault holds. Every quoted span in a claim is checked against the block it cites, or the whole source, when the page is written; a span that does not verify loses its quotation marks and is named in the result, or refuses the write with `quote_unverified` under `strict_quotes`. Distillation, ingest summary and research report pages record a `capture_scope` of `full-local`, `bounded-local` (a stored verbatim excerpt with its digest) or `url-only`, and a default-on `capture-scope` hygiene detector warns when active knowledge rests only on `url-only` sources. Every release is described in the [CHANGELOG](CHANGELOG.md).
## Documentation
diff --git a/docs/brainstorm/distilled-provenance/cli-output/claude.md b/docs/brainstorm/distilled-provenance/cli-output/claude.md
new file mode 100644
index 00000000..832955e9
--- /dev/null
+++ b/docs/brainstorm/distilled-provenance/cli-output/claude.md
@@ -0,0 +1,40 @@
+### Variant 1: Gate in distillSource, scope derived at classification
+
+- **Approach**: Extend `classifySourceOrigin` to keep the bytes it already reads and return `{trust, contentHash, bytes, captureScope}` in one pass, so the capture scope is a by-product of the existing trust classification (bytes present and complete gives `full-local`, bytes truncated by a ceiling gives `bounded-local`, URL or missing file gives `url-only`). `distillSource` scans each claim for `\p{Quotation_Mark}` pairs, normalises both sides for comparison only, and substring-matches against the whole source body; a failed span has its marks stripped and the receipt gains a `quote_fidelity` finding, while a `url-only` source yields a named `unavailable`. The health side is one hygiene detector that walks active pages to cited sources and warns when any cited source page carries `capture_scope: url-only`.
+- **Trade-offs**:
+ - Pro: smallest surface; one read of the source, no block resolver, no new module layout.
+ - Pro: the trust classifier is already the chokepoint for "no local bytes", so the shared vocabulary falls out naturally.
+ - Con: whole-source matching ignores the block id, so a quote that exists elsewhere in the source passes even when the cited block is wrong.
+ - Con: only guards the `distillSource` path; `brain_write_session` bytes and already-written pages are never checked.
+ - Con: `classifySourceOrigin` returning up to 8 MiB of bytes widens its contract for every other caller.
+- **Complexity**: small
+- **Risk**: medium
+
+### Variant 2: Shared provenance module, write-time gate plus lint detector
+
+- **Approach**: Add a `provenance` module holding the four-piece `capture_scope` vocabulary (registered in the census), a `readSourceEvidence()` that returns bytes, hash and scope and is called by ingest, distill and trust classification, a quote-span scanner over `\p{Quotation_Mark}` with a word-boundary rule for U+2019, a comparison-only normaliser separate from `QUOTE_VARIANT_RE`, and a best-effort block resolver that narrows the match window when the cited `^id` exists and falls back to the whole source otherwise. The verifier runs twice: at write time inside `distillSource` (failed span unquoted, receipt finding `quote_fidelity` with `verified | unquoted | unavailable`), and as a hygiene detector over existing distillations and session pages so caller-authored bytes and old pages surface the same finding. `capture_scope` is written to the frontmatter of new source, distillation and report pages; the url-only-backs-active-knowledge check is a second hygiene detector with a doctor code and exit-census row, deriving scope for pre-release pages from the existing `missing` sentinel.
+- **Trade-offs**:
+ - Pro: one vocabulary and one evidence reader serve both cards; "no local bytes" is a single enum member everywhere.
+ - Pro: covers the `brain_write_session` bypass and the already-written corpus through the existing detector plumbing that already reaches `brain_status`.
+ - Pro: block-scoped matching when ids exist, graceful whole-source fallback for the fixtures that cite `^abc` with no ids.
+ - Con: more files and tests; the block resolver is new code with its own edge cases (ids on list items, headings, tables).
+ - Con: new frontmatter key on new pages means those writers' byte-stability fixtures need regeneration (existing pages untouched).
+ - Con: the detector re-reads sources on every hygiene pass, which is a cost on large vaults.
+- **Complexity**: medium
+- **Risk**: low
+
+### Variant 3: Typed claims and a configurable quote policy
+
+- **Approach**: Change the claim contract so callers mark `kind: quote | paraphrase`; `distillSource` renders a quote only after verification and otherwise applies a configured policy (`distill_quote_policy: unquote | drop | reject`), where `reject` fails the whole write with a new coded error registered in tool-error-codes. `capture_scope` becomes an explicit ingest input validated against what was saved (mismatch is an error), and the url-only warning is a doctor check only. Failed spans under `unquote` are also recorded in a frontmatter list of unverified block ids on that page.
+- **Trade-offs**:
+ - Pro: strongest guarantee; a strict install can refuse to persist a paraphrase disguised as a quote.
+ - Pro: typed claims give downstream recall a clean quotable-or-not signal without re-scanning text.
+ - Con: breaks the current `{text, block?}` caller contract and the MCP schema, with the tool description at 280 of 300 characters.
+ - Con: three policies and an explicit scope input multiply the test matrix and the config surface for one release.
+ - Con: `drop` silently loses knowledge, and `reject` turns a model's punctuation choice into a hard write failure.
+ - Con: no detector over existing pages, so the corpus written before this release stays unchecked.
+- **Complexity**: large
+- **Risk**: high
+
+### Recommended: Variant 2
+**Rationale**: It is the only variant that satisfies both cards with one vocabulary and one evidence reader while covering the `brain_write_session` bypass and the already-written corpus through detector plumbing the project already has. Variant 1 leaves two write paths unguarded and silently widens the trust classifier's contract; Variant 3 breaks the claim and MCP contracts and adds policy surface this release does not need. The block resolver with whole-source fallback keeps the match deterministic and fixture-compatible, and the unquote-plus-named-finding fallback follows the existing receipt convention without losing text.
diff --git a/docs/brainstorm/distilled-provenance/cli-output/expected-output.md b/docs/brainstorm/distilled-provenance/cli-output/expected-output.md
new file mode 100644
index 00000000..cc6ea8c9
--- /dev/null
+++ b/docs/brainstorm/distilled-provenance/cli-output/expected-output.md
@@ -0,0 +1,92 @@
+# Expected output, before and after
+
+Illustrative vault: `Notes/talk.md` holds a paragraph ending in ` ^p1` that reads `The index is rebuilt nightly.`
+
+## `o2b brain distill` with a verbatim quote
+
+Claims: `[{"text": "The talk says “The index is rebuilt nightly.”", "block": "p1"}]`
+
+Before (3ec2334a):
+
+```
+distilled 1 claim(s) -> Brain/distillations/dist-notes-talk-md-<hash12>.md
+```
+
+After:
+
+```
+distilled 1 claim(s) -> Brain/distillations/dist-notes-talk-md-<hash12>.md [quotes verified:1 unquoted:0]
+```
+
+## `o2b brain distill` with a paraphrase in quotation marks
+
+Claims: `[{"text": "The talk says “The index is refreshed every night.”", "block": "p1"}]`
+
+Before: the same success line; the page shows the paraphrase as a quote.
+
+After (default): the page bullet reads `- The talk says The index is refreshed every night. ([[Notes/talk.md#^p1]])` and the line is:
+
+```
+distilled 1 claim(s) -> Brain/distillations/dist-notes-talk-md-<hash12>.md [quotes verified:0 unquoted:1]
+```
+
+After, with `--strict-quotes` (exit 1, nothing written):
+
+```
+distill: quoted spans failed verification: claim 0: not-in-block
+```
+
+`--json` (default mode) adds:
+
+```json
+{
+ "capture_scope": "full-local",
+ "quotes": {
+ "checked": 1,
+ "verified_in_block": 0,
+ "verified_in_source": 0,
+ "unquoted": 1,
+ "unpaired": 0,
+ "findings": [{ "claim": 0, "outcome": "not-in-block", "span": "The index is refreshed every night." }],
+ "total": 1,
+ "returned": 1,
+ "truncated": false
+ }
+}
+```
+
+## `o2b brain distill` from a URL with an excerpt
+
+`o2b brain distill https://example.test/post --claims-file c.json --excerpt-file excerpt.txt`
+
+Before: no `--excerpt-file` flag (usage error, exit 2).
+
+After:
+
+```
+distilled 1 claim(s) -> Brain/distillations/dist-https-example-test-post-<hash12>.md [untrusted_source] [bounded-local] [quotes verified:1 unquoted:0]
+```
+
+## `brain_distill_source` strict refusal on MCP
+
+After: a JSON-RPC error with `code: -32602`, `message: "brain_distill_source: quoted spans failed verification: claim 0: not-in-block"`, `data: { "code": "quote_unverified" }`. Before: the argument did not exist.
+
+## `o2b brain hygiene scan --json` with a research report that cites only URLs
+
+Before: no finding for the report.
+
+After, one more finding:
+
+```json
+{
+ "id": "capture-scope:<sha256-prefix>",
+ "detector": "capture-scope",
+ "severity": "warning",
+ "title": "Active knowledge rests on url-only sources",
+ "targets": ["Brain/reports/<date>-<slug>.md"],
+ "proposed_action": "review",
+ "evidence": { "sources": ["https://example.test/a", "https://example.test/b"], "stamped": null }
+}
+```
+
+A clean vault (every cited source is a vault file) prints exactly what it printed before.
diff --git a/docs/brainstorm/distilled-provenance/cli-output/prompt.md b/docs/brainstorm/distilled-provenance/cli-output/prompt.md
new file mode 100644
index 00000000..dee35c35
--- /dev/null
+++ b/docs/brainstorm/distilled-provenance/cli-output/prompt.md
@@ -0,0 +1,97 @@
+You are brainstorming architectural variants for the following task. Do not write code. Do not write a final design. Only produce variants and a recommendation.
+
+# Task
+
+Two tracker cards ship together as one release ("honest provenance for distilled knowledge"). The card bodies are reproduced verbatim, except that the project abbreviation is spelled out.
+
+## Card A: verify quotes on distilled brain pages verbatim against the source
+
+**Source**: https://github.com/garrytan/gbrain/releases/tag/v0.47.8.0
+
+### What
+When Open Second Brain distills a source or session into a brain page, any text wrapped in quotation marks should be checked word-for-word against the underlying transcript, and passed through only if it matches. Non-matching passages are either dropped or rendered unquoted rather than presented as a verbatim quote. This closes the gap where a distillation model emits a paraphrase inside quote marks that reads as something the user actually said but did not.
+
+### Why useful for Open Second Brain
+Distilled pages are trusted as a faithful record; a paraphrase disguised as a direct quote silently corrupts that record and propagates into downstream recall, claims, and synthesis. Verbatim verification raises quote trustworthiness and gives agents a hard signal for which passages are quotable source-of-truth versus model-generated summary.
+
+### Validator note (2026-09-12)
+premise_confirmed. No quote-fidelity check exists; distillSource validates claims structurally only (non-empty text plus a well-formed block id, src/core/brain/distill/distill-source.ts:11-13, :60-61). Every claim carries a block-id wikilink `- <text> ([[<source>#^<block>]])` (:161), and the source is pinned by a content sha256 (:236). The kernel runs no model here, so the check must be deterministic and run at write time. Unowned sources commit into a quarantine lane, which is a precedent for a failed span. One policy decision: matching tolerance and the fallback for a failed span.
+
+## Card B: capture_scope classification (full-local / bounded-local / url-only) plus a health warning when active knowledge rests on url-only sources
+
+**Source**: https://github.com/eugeniughelbur/obsidian-second-brain/releases/tag/v0.16.0
+
+### What
+Upstream's raw-source schema gains a `capture_scope` field with three values: `full-local` (the content is in the note body), `bounded-local` (a marked excerpt) or `url-only` (just the locator). Ingest sets it from what was actually saved, not what was intended, and health gains a check that reports when active knowledge (concepts, synthesis) rests on sources whose evidence is no longer locally readable - only a URL that may rot - instead of telling the user everything is fine.
+
+### Validator note (2026-09-29)
+premise_partly_false. capture_scope itself is absent (0 hits), but "nothing records how much of the source was captured" is false: the source_paths/source_hashes contract records per source whether evidence existed locally at derivation (`missing` sentinel, src/core/brain/freshness.ts:31) and freshness reports orphaned pages. Scope to the three-way distinction and the url-only-backs-active-knowledge warning.
+
+# Project context
+
+Open Second Brain: TypeScript on Bun 1.4, an agent-owned second brain in an Obsidian-compatible Markdown vault, exposed over MCP and the `o2b` CLI. Version 1.66.0.
+
+Recent commits:
+3ec2334a feat: stable error codes on the MCP wire, typed rerank degradation and a rerank doctor check (v1.66.0) (#225)
+a1ecfe7b feat: unattended maintenance recipes, install-owned lane tasks and an MCP fault guard (v1.65.0) (#224)
+4499698c feat: search that honours pins and event time, diagnostics that prove action, writes that name their residue (v1.64.0) (#223)
+a0b3c2e0 feat: exactly-one binding for mentions and imports, once-only write receipts, selectable search breadth and per-device ledgers (v1.63.0) (#222)
+f4f9e3e6 feat: silent failures surface by name in redaction, receipts, ingest, search store, doctor (v1.62.0) (#220)
+eb3c4c85 docs: concise README, ZCode client page, CLI reference and safety notes moved to child pages (#219)
+f22df2c5 chore(apb): zg search, code-ranker and OpenCodeReview before push, scope-scan hardening (#218)
+83979b9d fix: MCP startup off the upgrade path, fair ingest locks, inline plugin MCP manifest, inbox archive (v1.61.0) (#217)
+3e59f319 feat: decision-model uses, adapters and setup skill (v1.60.0) (#215)
+eacad953 feat: optional decision-model core and decision-model rerank kind (v1.59.0) (#214)
+0afa0c5a chore(deps): bump the bun-minor-and-patch group with 5 updates (#201)
+f6212cd2 fix: quiet one-line Stop reminder via Claude Code feedback channel (v1.58.2) (#202)
+534befdb chore(deps): bump the github-actions group with 3 updates (#200)
+5c550263 fix: session capture off the hook path, registry scanner hygiene (v1.58.1) (#199)
+6f33c424 feat: payload registry and knowledge packs, boundary hardening, CJK chunking (v1.58.0) (#197)
+a4d3ecf8 fix(codex): real plugin files and a truthful install check (v1.57.1) (#193)
+d7e6094a feat: a second platform (v1.57.0) (#191)
+54bb28d9 feat: what this install knows (v1.56.0) (#185)
+5f415267 feat: who wrote what, and how to take it back (v1.55.0) (#184)
+dc552590 feat: private is not a suggestion (v1.54.0) (#183)
+
+Verified facts from reconnaissance on the live source (3ec2334a):
+- `distillSource` (src/core/brain/distill/distill-source.ts:176-268): validate -> normalizeSourceIdentity -> classifySourceOrigin (source-trust.ts:351-381, returns {trust, contentHash?}, reads the file inside hashFile and throws the bytes away, 8 MiB ceiling, a URL or missing file gives untrusted with no bytes) -> render `## Claims` -> frontmatter -> idempotent byte-compare -> atomic write. Claims are caller-supplied `{text, block?}`; block is optional and most fixtures cite `^abc` against sources that contain no block ids.
+- Nothing resolves a block id to its text anywhere in src/ (parse-wikilink.ts only extracts the `^id`). A block resolver is new code.
+- entities/canonical.ts has `QUOTE_VARIANT_RE` (a fold over identity keys using Pi/Pf classes, pinned exhaustively by tests). It is not a span detector and misses U+0022, U+0027 (Po), U+201E and U+201A (Ps), CJK corner brackets (Ps/Pe), and `»…«` runs Pf to Pi. `\p{Quotation_Mark}` covers all of these without a language list. U+2019 is also the apostrophe inside words.
+- The retrieval trust gate (quarantine lane) is OFF by default (`search_trust_gate_enabled`), so quarantining a page does not stop a default search from returning it.
+- page-lint.ts refuses to normalise authored body bytes on write (receipts and byte-stability tests pin them); comparison-only normalisation is consistent with that.
+- MCP `brain_distill_source` description is 280 of 300 characters; property descriptions cap at 160; MCP errors become `invalid_params` unless a code-carrying class is registered in src/mcp/tool-error-codes.ts.
+- Pages that cite sources: ingest summaries (`kind: brain-source`), distillations (`brain-distillation`), research reports (`brain-report`, cite `[[https://...]]` URLs, no trust field), entities (quarantined when intake is untrusted; `restore` moves them to active and deletes the untrusted marker on purpose).
+- The only real bounded excerpt producer, `extractPage` in research/extract.ts (network fetch, 4000/500 char caps), is not wired up.
+- Hygiene detectors (`HYGIENE_DETECTOR_IDS`, findings with severity info|warning|action) reach `brain_status` through an existing count. Doctor codes need an exit-census row.
+- Writers add nothing to frontmatter for the default lane, so clean pages stay byte-identical (pinned by tests).
+- `brain_write_session` can also write caller-authored bytes into Brain/distillations without passing distillSource.
+
+Conventions:
+- Closed vocabularies ship as a four-piece (frozen object, member list, type, guard) and register in a census test.
+- Advisory findings in a write receipt: absent key means clean; a failure of the check is a named `unavailable`, never an absent key.
+- Errors surface by name; no silent fallbacks; no stubs.
+
+Constraints:
+- No hardcoded natural-language phrases in any language; other languages handled abstractly (Unicode properties, not word lists).
+- No model call in the kernel; checks are deterministic.
+- No network egress added to write paths in this release.
+- Do not widen `QUOTE_VARIANT_RE`.
+- Existing clean pages must stay byte-identical; no new external dependencies.
+- The two cards should share one vocabulary where they overlap ("no local bytes").
+
+# Required output format
+
+Produce exactly 3 distinct architectural variants. For each variant:
+
+### Variant N: <short name>
+- **Approach**: 2-3 sentences describing the variant.
+- **Trade-offs**: bullet list of pros and cons.
+- **Complexity**: small | medium | large
+- **Risk**: low | medium | high
+
+After the three variants, add exactly one recommendation:
+
+### Recommended: Variant N
+**Rationale**: 2-3 sentences explaining why this variant over the others, considering the project context and constraints above.
+
+Output nothing outside of these sections.
diff --git a/docs/brainstorm/distilled-provenance/design.md b/docs/brainstorm/distilled-provenance/design.md
new file mode 100644
index 00000000..054f8f28
--- /dev/null
+++ b/docs/brainstorm/distilled-provenance/design.md
@@ -0,0 +1,113 @@
+# Distilled provenance - quotes that are checked, sources that say how much was captured
+
+**Status:** draft (phase 1 spec review applied)
+**Author:** release orchestrator (via feature-release-playbook)
+**Audience:** implementation
+**Base:** origin/main 3ec2334a (v1.66.0), branch `feat/distilled-provenance`
+
+## Problem statement
+
+Open Second Brain presents a distillation page as a faithful record of its source, but `distillSource` validates claims structurally only (`src/core/brain/distill/distill-source.ts:10-15`, `validate()` at `:141-155`): a paraphrase wrapped in quotation marks is written exactly like a verbatim quote, and nothing compares it with the block the claim cites. Separately, no page records how much of its source was actually captured. A page whose only evidence is a URL looks the same as a page backed by a file the vault holds, and once the untrusted marker is removed (entity `restore`, `entities/registry.ts:669-721`) or was never written (research reports, `research/research.ts:158-210`), nothing reports that active knowledge rests on a locator that may rot.
+
+## Premise verification (live source, 3ec2334a)
+
+Full evidence: the two reconnaissance reports of this wave. The orchestrator re-opened the anchors this design builds on: `distill-source.ts:1-284`, `intake/source-trust.ts:94-135, 260-385`, `ingest/content-manifest.ts:81-87`, `ingest/ingest.ts:90-112, 156-200`, `research/research.ts:60-215`, `entities/canonical.ts:75-135`, `hygiene/types.ts:12-77`, `hygiene/scan.ts:29-36`, `hygiene/detectors/freshness.ts`, `mcp/tool-error-codes.ts:115-366`, `entities/status-scope.ts:34-76`, `trust/retrieval-gate.ts:38-104`.
+
+### Card A (verify quotes on distilled pages verbatim) - confirmed, two framing claims corrected
+
+1. Confirmed: no quote-fidelity check exists anywhere in `src/`. The kernel runs no model (`distill-source.ts:10-11`), so the check is deterministic and runs at write time inside `distillSource`, the one place both surfaces (MCP and CLI) pass through (`:172-174`).
+2. Corrected: "the substrate already exists (block ids)". Only the block-id grammar exists (`BLOCK_ID_RE`, `:61`; `parseWikilinkRich` extracts `^id`). Nothing in `src/` resolves a block id to its text. The block resolver is new code.
+3. Corrected (scope-scan claim): `entities/canonical.ts` does not find quotes. `QUOTE_VARIANT_RE` (`:79`) is a fold over identity keys built from `\p{Pi}\p{Pf}` plus U+02BC. It misses U+0022 and U+0027 (`Po`), U+201E and U+201A (`Ps`), the CJK corner brackets U+300C-U+300F and U+301D/U+301E (`Ps`/`Pe`), and `»…«` runs Pf to Pi. Its footprint is pinned over the whole code-point space (`tests/core/brain/entities/quote-variant-fold.test.ts`), so it is NOT widened. Span detection gets its own constant over the Unicode binary property `\p{Quotation_Mark}`; `foldQuoteVariantsByClass` (`:127-133`) is reused only in the comparison step.
+4. Corrected: the quarantine lane is not a usable answer for a failed span. The retrieval trust gate is off by default (`search_trust_gate_enabled`, pinned by `tests/cli/distill-trust-lane.test.ts:100-135`), so a quarantined fabricated quote is still served by a default search.
+5. Found: the source bytes are read inside `hashFile` and discarded (`source-trust.ts:369-372`, `content-manifest.ts:81-87`). A second read for the check could verify bytes that differ from the hashed ones, contradicting the module's own claim (`distill-source.ts:12-13`). One read must feed both.
+
+### Card B (capture_scope plus a url-only health warning) - partly false, narrowed
+
+1. Confirmed: `capture_scope`, `full-local`, `bounded-local` and `url-only` have zero hits in `src/`, `tests/` and `docs/`.
+2. False, as the validator said: "nothing records how much of the source was captured". The trust lane (`untrusted_source`, `trust/untrusted-provenance.ts:88-90`) is already a two-way local versus not-local verdict for a single source, `source_content_hash` records the hashed bytes, and the freshness `missing` sentinel (`freshness.ts:31`) records absence for handoff pages. The work is the third member (a bounded excerpt) and the warning.
+3. Corrected: the warning condition is not on the ingest happy path. Untrusted ingest and entity intake already land in the quarantine lane. The condition lives in three places that escape it: research reports (`provenance: stated`, URL wikilinks, no trust field), released entities (`restore` deletes the marker on purpose, `registry.ts:712-716`) and trusted pages whose local source later vanished (only handoff pages carry the freshness contract).
+4. Corrected anchors: `brain_sources` (`mcp/brain/query-tools.ts:1006`) is a signal dashboard, not a list of ingested sources; `vault_health` (`mcp/tools.ts:519`) runs the vault/config doctor, not the Brain doctor, and is the wrong home. The OKF walk is `collectOkfPages` at `portability/okf.ts:261`, not `:244`.
+5. Not usable as the base: the validator pointed at `MISSING_SOURCE_HASH`. Ingest, distill, research and entity pages do not carry the `source_paths`/`source_hashes` contract (only `handoff.ts:165` writes it), so the scope is derived from the source identity through the trust classifier, not from that sentinel.
+
+## Scope
+
+1. A shared capture-scope vocabulary `CAPTURE_SCOPE` (`full-local`, `bounded-local`, `url-only`) with a classifier over a source identity, a frontmatter helper that writes nothing for `full-local`, and the bounded-excerpt block format (render, read, digest).
+2. A quote-span finder over `\p{Quotation_Mark}` with position-based pairing and the between-letters apostrophe exclusion, and a comparison-only normaliser.
+3. A block resolver: block id to block text inside source bytes.
+4. The quote check inside `distillSource`: each quoted span in a claim is compared with the cited block (or, for a claim with no block, the whole source); a failed span is unquoted by default and named in the result; `strict_quotes` / `--strict-quotes` refuses the write with the registered code `quote_unverified`.
+5. One read of the source feeds both the content digest and the check.
+6. A `distillSource` `excerpt` argument that makes `bounded-local` reachable: for a source with no local bytes, the caller may supply the verbatim excerpt it read; the page stores it in a marked block with its digest, and quotes are checked against it.
+7. `capture_scope` stamped at write time on distillation, ingest summary and research report pages (only when not `full-local`), surfaced on each write result.
+8. A default-on hygiene detector `capture-scope`: a retrievable page of active knowledge whose every cited source is currently `url-only` gets a `warning` finding with `proposed_action: review`.
+9. Docs: new `brain_distill_source` and `o2b brain distill` reference sections (neither exists today), the detector in the hygiene lists, a capture-scope section.
+
+## Out of scope
+
+1. Checking pages that `brain_write_session` commits into `Brain/distillations` (`write-binding/index.ts:294`). Those bytes never pass `distillSource`; a doctor or hygiene re-check over existing distillation pages is a later slice.
+2. Re-checking quotes on pages already on disk (drift against changed source bytes). Same later slice.
+3. Wiring `research/extract.ts` `extractPage` (network egress on a write path). `bounded-local` is reachable through the explicit `excerpt` argument only.
+4. A `capture_scope` field on `listIngestedSources` / `IngestedSource`, an OKF `producer_meta` field, a "list ingested sources" surface.
+5. The unchecked `raw` quote on `brain_session_checkpoint` signals (same class of gap, different surface).
+6. Widening `QUOTE_VARIANT_RE`, or any change to `canonical.ts`.
+7. CHANGELOG, version bump, OpenClaw bundle rebuild (handled by PR preparation and the integrator).
+
+## Chosen approach
+
+Consultant variant 2 ("shared provenance module, write-time gate plus lint detector"), narrowed. Kept: one provenance vocabulary shared by both cards, one evidence read, a `\p{Quotation_Mark}` span scanner separate from `QUOTE_VARIANT_RE`, a comparison-only normaliser, the block resolver, unquote plus a named finding at write time, `capture_scope` on new pages, the url-only warning as a hygiene detector. Dropped: the second quote detector over existing and write-session pages (out of scope, item 1), the doctor code (a hygiene finding already reaches `brain_status` through the `hygiene-findings` count, `operator-snapshot.ts:166`), deriving scope from the `missing` sentinel (premise B.5), and the whole-source fallback when a cited block does not resolve (decision 6). Added: the `excerpt` argument, so `bounded-local` has a real producer.
+
+## Design decisions
+
+### Quote spans
+
+1. **Delimiters.** A quotation mark is any code point with the Unicode binary property `Quotation_Mark` (`/\p{Quotation_Mark}/u`). No language list. A new constant in the span module; `canonical.ts` is untouched.
+2. **Apostrophe exclusion.** A mark with a letter (`\p{L}`) immediately on both sides is never a delimiter (`don’t`, `l'eau`). U+02BC is not `Quotation_Mark` and needs no rule. Ps/Pe quotation marks are directional and never apostrophes.
+3. **Pairing by position, not by glyph.** An opener is a mark at the start of the text or after whitespace or opening punctuation (`\p{Ps}`, `\p{Pi}`, another opener), and followed by a non-space. A closer is a mark followed by the end, whitespace or punctuation, and preceded by a non-space. This pairs `"…"`, `“…”`, `„…“`, `»…«`, `«…»`, `「…」` and `『…』` without knowing which language uses which. Pairing is a stack; only outermost spans are checked, an inner quote is part of its outer span's text.
+4. **Unpaired marks** are not spans. They are counted (`unpaired`) in the result and left on the page untouched. An empty or whitespace-only span is not a quote.
+
+### Matching tolerance
+
+5. **Comparison-only normalisation, applied identically to both sides:** NFC; inline Markdown reduced to display text (emphasis and code markers `*`, `_`, `` ` ``, `~~`; `[text](url)` to `text`; `[[target|alias]]` to `alias`, `[[target]]` to `target`); line-leading block markers removed (`>`, list bullets, ordered-list numbers); the trailing ` ^id` removed; `foldQuoteVariantsByClass` for marks inside the span; whitespace runs collapsed to one space; trimmed. Then the span must be an exact substring of the normalised block. Never folded: case, punctuation other than quote width, wording. Evidence: `page-lint.ts:39-43` refuses normalisation of WRITTEN bytes only; the page bytes are never normalised by this check.
+6. **Ellipsis.** A span containing `…` (U+2026) or a run of three or more `.` is split there; every non-empty fragment must appear in the block, in order, without overlap. A bracketed insertion (`[sic]`, `[the protocol]`) is not interpreted and fails as a mismatch.
+7. **Comparison scope.** A claim with `block` is compared with the resolved block only. If the block does not resolve, the outcome is `block-not-found` (duplicate id: `block-ambiguous`), never a silent fall-back to the whole source: the citation is part of what the page asserts. A claim with no `block` is compared with the whole source and the outcome is `verified-in-source`, distinct from `verified-in-block`, so the weaker guarantee never reads as the stronger. Existing fixtures cite `^abc` against sources with no block ids but contain no quotation marks, so no claim in them is checked.
+8. **Block resolution.** Fenced code is skipped. A line ending in whitespace plus `^id` is a paragraph or list-item block (its paragraph, marker removed); a line holding only `^id` attaches to the block directly above it (table, blockquote, list). Two definitions of one id are `block-ambiguous`.
+9. **Source text.** The bytes from the single read are decoded as strict UTF-8 (`TextDecoder` with `fatal: true`); a decode failure gives every span of the page the outcome `source-not-text`.
+
+### Failed-span policy
+
+10. **Default: unquote and name it.** The delimiter pair of a failed span is removed, the words stay, the write lands, and the result names each failure. The page never shows an unverified quote. Dropping the claim was rejected (it loses caller-supplied knowledge unasked); quarantine was rejected (premise A.4). Unquoting is a named, deliberate edit of authored bytes stated in the result and the docs, unlike the silent normalisation `page-lint.ts` refuses.
+11. **Opt-in strict refusal.** `strict_quotes: true` (MCP) / `--strict-quotes` (CLI) refuses the whole write when any span would be unquoted, with `QuoteCheckError` carrying the registered wire code `quote_unverified`. Nothing is written. The message names claim indices and outcomes, never claim text or paths.
+12. **Unpaired marks never trigger strict refusal**; they are counted only.
+
+### Capture scope
+
+13. **One vocabulary for "no local bytes".** `CAPTURE_SCOPE.urlOnly = "url-only"` is the single name. The quote check's outcome for a span with no local bytes to check against IS that token (`QUOTE_CHECK_OUTCOME.urlOnly = CAPTURE_SCOPE.urlOnly`), not a second spelling.
+14. **Boundaries.** `full-local`: the identity is vault-shaped and resolves to a readable file (the trusted verdict of `classifySourceTrust` / `classifySourceOrigin`). `url-only`: the identity carries a URI scheme or authority shape, or is vault-shaped with no file behind it (the untrusted verdict), so the lane and the scope cannot disagree. `bounded-local`: a `url-only` identity whose page holds a verbatim excerpt the same writer stored, in the marked excerpt block, with its digest. Never inferred from prose: an ingest `summary` is a paraphrase, not a capture.
+15. **Oversized sources stay a refusal.** A file above `SOURCE_HASH_MAX_BYTES` keeps throwing `SourceTrustError`; it never becomes `bounded-local` by truncation (consultant variant 1's idea, rejected: a refusal turned into a degraded success).
+16. **Where `bounded-local` comes from.** Only `distillSource`, through the new `excerpt` argument, accepted only when the source is `url-only` (a local source with an excerpt is refused by name: the file is the evidence) and bounded by `CAPTURE_EXCERPT_MAX_BYTES`. Ingest and research keep two-valued scopes in this release; `extractPage` stays unwired (out of scope, item 3).
+17. **Stamp at write, re-derive on read.** Writers stamp `capture_scope` (single source) or `capture_scopes` (research reports, parallel to the report's `sources` order) only when a value is not `full-local`, so every full-local page stays byte-identical (pinned by `distill-source-trust.test.ts:278-309`). Absence means "full-local or written before this release" and is resolved by re-deriving, never by assuming. The detector re-derives the CURRENT scope from the identity (`classifySourceTrust`, no byte read), so a full-local page whose file vanished reports as `url-only`; a stamped `bounded-local` counts as backing only while the page's excerpt block is present and matches its digest.
+18. **The warning.** A default-on hygiene detector `capture-scope` (severity `warning`, `proposed_action: review`) over retrievable pages (`classifyRetrievalTrust(meta).quarantined === false`) of kind `brain-distillation`, `brain-source`, `brain-report`, and `brain-entity` in the canonical status scope (`entityStatusInScope(status, "canonical")`). It fires when every source the page cites is currently `url-only`. Quarantined pages are skipped: the gate already excludes them. A doctor check was rejected (it would need a `DOCTOR_EXIT_EXCLUSIONS` row and adds nothing the hygiene count does not already show in `brain_status`); `brain_health` was rejected (preference-centric fold).
+19. **Source extraction is not re-implemented.** `stringArrayField` and `wikilinkTarget` move from `source-cleanup.ts` into a small shared module with a `## Sources` link reader; `SourceCleanupEntry` and the `deleteBySource` payload (pinned by `gates/recoverability.test.ts`) are untouched.
+
+### Result and page shapes
+
+20. **Distill result.** Additive keys, absent when there is nothing to report: `quotes` (present only when at least one claim contains a span) and `captureScope` (always present, mirroring `trust`). The MCP payload uses `quotes` and `capture_scope`; the CLI `--json` the same. `quotes` = `{checked, verified_in_block, verified_in_source, unquoted, unpaired, findings, total, returned, truncated}`, findings only for unquoted spans, each `{claim, outcome, span}` with `claim` the 0-based index, `span` capped at `QUOTE_SPAN_PREVIEW_MAX_CHARS`, findings capped at the page-lint cap (25). Built from the claim's own text only, never a path.
+21. **Distill page.** `quotes_verified` and `quotes_unquoted` frontmatter integers only when spans exist; `capture_scope` and, for `bounded-local`, `excerpt_hash` only when not full-local; the excerpt in a `## Excerpt` section as a fenced block with the info string `excerpt`, so its bytes are verbatim and wikilink parsing skips it. A clean page stays byte-identical; the idempotent re-run stays inert.
+22. **CLI human line.** The existing suffix idiom (`untrustedNote`): ` [quotes verified:V unquoted:U]` when spans were checked, ` [bounded-local]` when an excerpt was stored. Exit 1 on a strict refusal, as for any validation failure.
+23. **Ingest and research results** gain `captureScope` / `captureScopes`, serialised as `capture_scope` / `capture_scopes` on MCP. None of these tools has an `outputSchema`, so no contract change.
+24. **Wire code.** `QuoteCheckError` is registered in `CLASSIFICATION` (`tool-error-codes.ts:340-366`) as `fixed(QuoteCheckError, TOOL_ERROR_CODE.quoteUnverified)`. `wrapToolErrors` drops data (`mcp/brain/shared.ts:95-110`), so `distill-tools.ts` maps this one class itself with `{code}` in the `MCPError` data; `shared.ts` is not changed.
+
+## File changes
+
+New: `src/core/brain/provenance/capture-scope.ts`, `src/core/brain/distill/quote-spans.ts`, `src/core/brain/distill/quote-verdict.ts`, `src/core/brain/distill/block-resolve.ts`, `src/core/brain/distill/quote-check.ts`, `src/core/brain/source-links.ts`, `src/core/brain/hygiene/detectors/capture-scope.ts`, plus their tests.
+
+Modified: `src/core/brain/intake/source-trust.ts`, `src/core/brain/ingest/content-manifest.ts`, `src/mcp/tool-error-codes.ts`, `src/core/brain/distill/distill-source.ts`, `src/core/brain/ingest/ingest.ts`, `src/core/brain/research/research.ts`, `src/core/brain/source-cleanup.ts`, `src/core/brain/hygiene/types.ts`, `src/core/brain/hygiene/scan.ts`, `src/mcp/brain/distill-tools.ts`, `src/mcp/brain/ingest-tools.ts`, `src/mcp/brain/research-tools.ts`, `src/mcp/brain/hygiene-tools.ts`, `src/mcp/registry-guard.ts`, `src/cli/brain/verbs/distill.ts`, `src/cli/command-manifest.ts`, `docs/mcp.md`, `docs/cli-reference.md`, `docs/how-it-works.md`, and the tests named in `plan.md`. Exact ownership is in `plan.md`.
+
+Not touched: `src/core/brain/entities/canonical.ts`, `src/mcp/brain/shared.ts`, `openclaw/index.js` (nothing distill- or hygiene-related is bundled; the integrator verifies with `bun run build:openclaw`), `plugins/codex/` (no skill text changes).
+
+## Risks and open questions
+
+1. **MCP description budget.** `brain_distill_source` is at 280 of 300 characters. The new `strict_quotes` and `excerpt` properties carry their own descriptions (cap 160); the tool description gains at most one short clause. If the clause does not fit, the property descriptions carry the behaviour and the docs section carries the rest.
+2. **Unquoting changes caller bytes.** A caller re-running the same claims gets the same unquoted page (idempotent), but a caller comparing its input with the page sees a difference. The result names every change; the docs state it.
+3. **Position pairing ambiguity.** Text like `5" screen` or `rock 'n' roll` can produce an unpaired mark or a short span. An unpaired mark is only counted; a short span that fails is unquoted, which is the honest outcome for an unverifiable quotation.
+4. **Detector cost.** One `stat` per cited source per page on every default hygiene sweep. Bounded by the page kinds above; no byte reads.
+5. **Census coupling.** The new four-piece vocabularies (`CAPTURE_SCOPE`, `QUOTE_CHECK_OUTCOME`) must be registered in `verdict-vocabulary-census.test.ts` in the same task that adds them, or the census is red for the other lanes.
diff --git a/docs/brainstorm/distilled-provenance/plan.md b/docs/brainstorm/distilled-provenance/plan.md
new file mode 100644
index 00000000..78bd063c
--- /dev/null
+++ b/docs/brainstorm/distilled-provenance/plan.md
@@ -0,0 +1,237 @@
+# Distilled provenance - implementation plan
+
+Design: `docs/brainstorm/distilled-provenance/design.md` (decision numbers below refer to its "Design decisions" list).
+
+## How to run this plan
+
+Four lanes with DISJOINT file ownership run concurrently in the one worktree. Lane A (shared substrate) commits first; every other lane waits only on lane A and, where named, on one upstream task. A lane edits only the files listed under its own tasks. Every task is TDD: write the named test first, run it and see it fail for the stated reason, implement, see it pass, then `oxfmt` and `oxlint` on the lane's own files only, then `git add <own paths>` and one conventional commit (`feat(distill): ...`, `feat(provenance): ...`, `feat(hygiene): ...`, `docs: ...`). Retry on `index.lock`. Gates per task: `bun test <the task's test files>` and `bun run typecheck`.
+
+CHANGELOG, the version bump and the OpenClaw bundle belong to no lane (PR preparation and the integrator handle them).
+
+Dependency summary:
+
+- A1 -> A2 -> A3 (A3 imports A1's `CAPTURE_SCOPE`; A1 and A3 both edit the census test, so they run in order); A4 has no dependency.
+- Lane A complete (A1..A4) -> B1, C1, D4.
+- B1 -> B2 -> B3 -> B4 (one lane, in order).
+- C1 -> C2 -> C3 -> C4 (one lane, in order).
+- B4 -> D1 -> D2; C4 -> D3. D1 and D2 run in lane D's order after B4; D3 waits on C4 alone.
+- D4 (docs) is drafted from the contract after lane A and finalised last.
+
+## Cross-lane contract (pinned now; no lane renames or re-spells any of this)
+
+### `src/core/brain/provenance/capture-scope.ts` (lane A, A1)
+
+- `CAPTURE_SCOPE = Object.freeze({ fullLocal: "full-local", boundedLocal: "bounded-local", urlOnly: "url-only" } as const)`
+- `CAPTURE_SCOPES: ReadonlyArray<CaptureScope>` (frozen member list), `type CaptureScope`, `isCaptureScope(value: unknown): value is CaptureScope`
+- `CAPTURE_SCOPE_KEY = "capture_scope"`, `CAPTURE_SCOPES_KEY = "capture_scopes"`, `EXCERPT_HASH_KEY = "excerpt_hash"`
+- `CAPTURE_EXCERPT_MAX_BYTES = 65_536`
+- `CAPTURE_EXCERPT_HEADING = "## Excerpt"`, `CAPTURE_EXCERPT_FENCE_INFO = "excerpt"`
+- `captureScopeForTrust(trust: IntakeTrust): CaptureScope` - trusted -> `full-local`, untrusted -> `url-only`
+- `classifyCaptureScope(vault: string, identity: string): CaptureScope` - via `classifySourceTrust`, never reads bytes
+- `captureScopeFrontmatter(scope: CaptureScope): FrontmatterMap` - `{}` for `full-local`, else `{ capture_scope: scope }`
+- `captureScopesFrontmatter(scopes: ReadonlyArray<CaptureScope>): FrontmatterMap` - `{}` when every member is `full-local`, else `{ capture_scopes: [...] }`
+- `excerptDigest(excerpt: string): string` - sha256 hex over the UTF-8 bytes of the excerpt as given
+- `renderExcerptSection(excerpt: string): string` - the heading, a blank line, then a fence long enough to contain any backtick run in the excerpt, info string `excerpt`, the excerpt verbatim, the closing fence
+- `readExcerptSection(body: string): string | null` - inverse of the render; `null` when the section is absent or malformed
+- `CaptureExcerptError extends Error` (`name = "CaptureExcerptError"`) - thrown by `assertExcerptAdmissible(scope: CaptureScope, excerpt: string)` when an excerpt is given for a non-`url-only` source, is empty after trim, or exceeds `CAPTURE_EXCERPT_MAX_BYTES`. Message: identifiers and integers only.
+
+### `src/core/brain/distill/quote-spans.ts` (lane A, A2)
+
+- `QUOTATION_MARK_RE = /\p{Quotation_Mark}/u` (separate from `canonical.ts`; `QUOTE_VARIANT_RE` is not touched)
+- `interface QuoteSpan { readonly open: number; readonly close: number; readonly inner: string }` - UTF-16 offsets of the opening and closing mark in the input
+- `interface QuoteSpanScan { readonly spans: ReadonlyArray<QuoteSpan>; readonly unpaired: number }`
+- `findQuoteSpans(text: string): QuoteSpanScan` - decisions 1-4, outermost spans only
+- `unquoteSpans(text: string, spans: ReadonlyArray<QuoteSpan>): string` - removes exactly the two marks of each given span
+- `normalizeForQuoteComparison(text: string): string` - decision 5, both sides
+- `splitEllipsisFragments(inner: string): ReadonlyArray<string>` - decision 6
+- `spanOccursIn(inner: string, haystackNormalized: string): boolean` - fragments in order, no overlap
+
+### `src/core/brain/distill/quote-verdict.ts` (lane A, A3)
+
+- `QUOTE_CHECK_OUTCOME = Object.freeze({ verifiedInBlock: "verified-in-block", verifiedInSource: "verified-in-source", notInBlock: "not-in-block", notInSource: "not-in-source", blockNotFound: "block-not-found", blockAmbiguous: "block-ambiguous", sourceNotText: "source-not-text", urlOnly: "url-only" } as const)`. The `urlOnly` value is written as the literal, not as `CAPTURE_SCOPE.urlOnly`: the verdict-vocabulary census only registers objects whose values are all literals. The one-name rule (decision 13) is kept by a test that asserts `QUOTE_CHECK_OUTCOME.urlOnly === CAPTURE_SCOPE.urlOnly`.
+- `QUOTE_CHECK_OUTCOMES`, `type QuoteCheckOutcome`, `isQuoteCheckOutcome(value: unknown)`
+- `VERIFIED_QUOTE_OUTCOMES: ReadonlySet<QuoteCheckOutcome>` = `{verified-in-block, verified-in-source}`
+- `QUOTE_FINDINGS_MAX = PAGE_LINT_MAX_FINDINGS` (imported from `page-lint.ts`), `QUOTE_SPAN_PREVIEW_MAX_CHARS = 120`
+- `interface QuoteFinding { readonly claim: number; readonly outcome: QuoteCheckOutcome; readonly span: string }`
+- `interface QuoteCheckReport { readonly checked: number; readonly verified_in_block: number; readonly verified_in_source: number; readonly unquoted: number; readonly unpaired: number; readonly findings: ReadonlyArray<QuoteFinding>; readonly total: number; readonly returned: number; readonly truncated: boolean }`
+- `QUOTE_UNVERIFIED_CODE = "quote_unverified"`
+- `QuoteCheckError extends Error` with `readonly code = QUOTE_UNVERIFIED_CODE`, `name = "QuoteCheckError"`, constructed from `ReadonlyArray<QuoteFinding>`; message is `quoted spans failed verification: ` followed by comma-separated `claim <i>: <outcome>` pairs, never span text
+- Frontmatter keys: `QUOTES_VERIFIED_KEY = "quotes_verified"`, `QUOTES_UNQUOTED_KEY = "quotes_unquoted"`
+
+### `src/core/brain/intake/source-trust.ts` and `src/core/brain/ingest/content-manifest.ts` (lane A, A4)
+
+- `hashBytes(bytes: Uint8Array): string` in `content-manifest.ts`; `hashFile` becomes `hashBytes(readFileSync(abs))` with identical output
+- `interface SourceOriginWithBytes extends SourceOrigin { readonly bytes?: Uint8Array }` - `bytes` present exactly when `contentHash` is
+- `readSourceOrigin(vault: string, sourcePath: string): SourceOriginWithBytes` - same verdicts, same refusals (`SourceTrustError`, the 8 MiB ceiling) as `classifySourceOrigin`, one read feeding the digest and the returned bytes. `classifySourceOrigin` keeps its signature and delegates.
+
+### `src/mcp/tool-error-codes.ts` (lane A, A3)
+
+- `TOOL_ERROR_CODE.quoteUnverified = "quote_unverified"` (equal to `QUOTE_UNVERIFIED_CODE`; the test asserts equality)
+- `fixed(QuoteCheckError, TOOL_ERROR_CODE.quoteUnverified)` in `CLASSIFICATION`
+
+### `src/core/brain/distill/block-resolve.ts` (lane B, B1)
+
+- `type BlockResolution = { kind: "found"; text: string } | { kind: "not-found" } | { kind: "ambiguous" }` - a plain discriminated union (no frozen object, so no census row), mapped onto `QUOTE_CHECK_OUTCOME` by B2
+- `resolveBlock(source: string, blockId: string): BlockResolution`
+
+### `src/core/brain/distill/quote-check.ts` (lane B, B2)
+
+- `interface QuoteCheckInput { readonly claims: ReadonlyArray<DistillClaim>; readonly evidence: QuoteEvidence }`
+- `type QuoteEvidence = { kind: "text"; text: string } | { kind: "not-text" } | { kind: "url-only" }`
+- `interface QuoteCheckResult { readonly claims: ReadonlyArray<DistillClaim>; readonly report: QuoteCheckReport | null }` - `claims` with failed spans unquoted; `report` `null` when no claim contains a span
+- `checkClaimQuotes(input: QuoteCheckInput): QuoteCheckResult` - pure, no I/O
+
+### `distillSource` (lane B, B3 and B4)
+
+- `DistillSourceInput` gains `readonly excerpt?: string`; `DistillSourceOptions` gains `readonly strictQuotes?: boolean`
+- `DistillSourceResult` gains `readonly captureScope: CaptureScope` (always) and `readonly quotes?: QuoteCheckReport` (only when spans exist)
+- Strict refusal throws `QuoteCheckError` before any write; an inadmissible excerpt throws `CaptureExcerptError` before any write
+
+### Ingest and research (lane C)
+
+- `IngestSourceResult.captureScope: CaptureScope` (always present)
+- `ResearchReportResult.captureScopes: ReadonlyArray<CaptureScope>` (always present, parallel to `input.sources`)
+
+### `src/core/brain/source-links.ts` (lane C, C3)
+
+- `stringArrayField(meta: FrontmatterMap, key: string): ReadonlyArray<string>` and `wikilinkTarget(raw: string): string` (moved verbatim from `source-cleanup.ts`, which imports them back)
+- `sourcesSectionTargets(body: string): ReadonlyArray<string>` - wikilink targets under the `## Sources` heading, the same rules `source-cleanup.ts` uses
+
+### Hygiene detector (lane C, C4)
+
+- Detector id `"capture-scope"` appended to `HYGIENE_DETECTOR_IDS` and to `DEFAULT_SCAN_IDS`
+- `CAPTURE_SCOPE_DETECTOR_ID = "capture-scope"`, `detectCaptureScope(vault: string): ReadonlyArray<HygieneFinding>`
+- Finding: `severity: "warning"`, `proposed_action: "review"`, `targets: [page]`, `title: "Active knowledge rests on url-only sources"`, `evidence: { sources: string[], stamped: CaptureScope | null }` where `stamped` is the page's own `capture_scope` value (null when absent, and always null on a report, whose per-source stamps live in `capture_scopes`)
+
+### Surface names (lane D)
+
+- MCP `brain_distill_source` input: `strict_quotes` (boolean), `excerpt` (string); output adds `capture_scope` and, when present, `quotes` (the `QuoteCheckReport` keys verbatim)
+- MCP `brain_ingest_source` output adds `capture_scope`; `brain_research_report` output adds `capture_scopes`
+- CLI `o2b brain distill`: `--strict-quotes`, `--excerpt-file <path>`; `--json` adds `capture_scope` and `quotes`; human suffixes ` [quotes verified:V unquoted:U]` and ` [bounded-local]`
+- A strict refusal on MCP: `MCPError(INVALID_PARAMS, "brain_distill_source: <message>", { code: "quote_unverified" })`
+
+## File ownership (every file this plan touches, exactly one owner)
+
+| Lane | Owns |
+| ---- | ---- |
+| A substrate | `src/core/brain/provenance/capture-scope.ts` (new), `src/core/brain/distill/quote-spans.ts` (new), `src/core/brain/distill/quote-verdict.ts` (new), `src/core/brain/intake/source-trust.ts`, `src/core/brain/ingest/content-manifest.ts`, `src/mcp/tool-error-codes.ts`, `tests/core/brain/provenance/capture-scope.test.ts` (new), `tests/core/brain/distill/quote-spans.test.ts` (new), `tests/core/brain/distill/quote-verdict.test.ts` (new), `tests/core/brain/intake/source-origin-bytes.test.ts` (new), `tests/mcp/tool-error-codes.test.ts`, `tests/core/architecture/verdict-vocabulary-census.test.ts` |
+| B quote check and distill write path | `src/core/brain/distill/block-resolve.ts` (new), `src/core/brain/distill/quote-check.ts` (new), `src/core/brain/distill/distill-source.ts`, `tests/core/brain/distill/block-resolve.test.ts` (new), `tests/core/brain/distill/quote-check.test.ts` (new), `tests/core/brain/distill/distill-source-quotes.test.ts` (new), `tests/core/brain/distill/distill-source-capture.test.ts` (new), `tests/core/brain/distill/distill-source.test.ts`, `tests/core/brain/distill/distill-source-trust.test.ts` |
+| C capture scope, ingest, research, detector | `src/core/brain/ingest/ingest.ts`, `src/core/brain/research/research.ts`, `src/core/brain/source-links.ts` (new), `src/core/brain/source-cleanup.ts`, `src/core/brain/hygiene/detectors/capture-scope.ts` (new), `src/core/brain/hygiene/types.ts`, `src/core/brain/hygiene/scan.ts`, `tests/core/brain/ingest/ingest-capture-scope.test.ts` (new), `tests/core/brain/research/research-capture-scope.test.ts` (new), `tests/core/brain/source-links.test.ts` (new), `tests/core/brain/hygiene-capture-scope.test.ts` (new), `tests/mcp/hygiene-link-integrity.test.ts` |
+| D surfaces and docs | `src/mcp/brain/distill-tools.ts`, `src/mcp/brain/ingest-tools.ts`, `src/mcp/brain/research-tools.ts`, `src/mcp/brain/hygiene-tools.ts`, `src/mcp/registry-guard.ts`, `src/cli/brain/verbs/distill.ts`, `src/cli/command-manifest.ts`, `tests/mcp/distill-tool.test.ts`, `tests/mcp/distill-quotes-envelope.test.ts` (new), `tests/mcp/ingest-tool.test.ts`, `tests/mcp/research-tool.test.ts`, `tests/cli/brain-distill.test.ts`, `tests/cli/brain-hygiene.test.ts`, `docs/mcp.md`, `docs/cli-reference.md`, `docs/how-it-works.md` |
+
+Shared-file rules:
+- `tests/core/architecture/verdict-vocabulary-census.test.ts` is lane A's. Both four-piece vocabularies (`CAPTURE_SCOPE`, `QUOTE_CHECK_OUTCOME`) are lane A's, so the census is green from A1/A3 onward. A lane that believes it needs a new four-piece vocabulary reports it instead of adding one.
+- `tests/mcp/hygiene-link-integrity.test.ts` pins `HYGIENE_DETECTOR_IDS`; lane C owns it so the id and the pin change in one commit (C4).
+- `src/core/brain/distill/distill-source.ts` is lane B's alone. The capture-scope stamp on distillation pages is done by B4 with lane A's helpers; lane C never edits it.
+- `src/mcp/brain/hygiene-tools.ts` derives its enum from `HYGIENE_DETECTOR_IDS`; lane D edits only its description text (D3).
+- `tests/core/brain/distill/distill-source-trust.test.ts:278-309` (byte identity of a clean trusted page) must stay green unchanged; it is listed under B only so that a deliberate change, if one is ever needed, has an owner. Expected: no edit.
+- Not touched by any lane: `src/core/brain/entities/canonical.ts`, `src/mcp/brain/shared.ts`, `src/core/brain/page-lint.ts` (only imported), `openclaw/index.js`, `plugins/codex/`, `CHANGELOG.md`, `package.json`.
+
+Size: 49 files in the table (20 new: 9 source, 11 test), within the 35-50 budget.
+
+## Tasks
+
+### Lane A - shared substrate (commits first)
+
+### Task A1: capture-scope vocabulary, classifier, frontmatter and excerpt block
+- **Files**: `src/core/brain/provenance/capture-scope.ts` (new), `tests/core/brain/provenance/capture-scope.test.ts` (new), `tests/core/architecture/verdict-vocabulary-census.test.ts`
+- **Test first**: (a) the four-piece shape: frozen object, frozen member list equal to its values, guard accepts each member and rejects `"full_local"`, `""`, `null`; (b) `classifyCaptureScope` over a temp vault: an existing vault file -> `full-local`; `https://x.test/a`, `//x.test/a`, `evil.com/x` -> `url-only`; a vault-shaped path with no file -> `url-only`; a directory -> `url-only`; (c) `captureScopeFrontmatter(full-local)` is `{}`, the others carry the key; `captureScopesFrontmatter` is `{}` for all-full-local and a list otherwise; (d) `renderExcerptSection` then `readExcerptSection` round-trips text containing a triple-backtick run, CJK text and a trailing newline exactly; `readExcerptSection` on a body without the heading is `null`; (e) `assertExcerptAdmissible` throws `CaptureExcerptError` for a `full-local` scope, a whitespace-only excerpt and one byte over the cap (multi-byte characters counted as bytes), and passes at the cap; (f) the census lists `CAPTURE_SCOPE`.
+- **Acceptance**: tests pass; census test passes; `bun run typecheck` clean.
+- **Depends on**: none
+
+### Task A2: quote-span finder and comparison normaliser
+- **Files**: `src/core/brain/distill/quote-spans.ts` (new), `tests/core/brain/distill/quote-spans.test.ts` (new)
+- **Test first**: cases are built from code points, not from word lists: (a) every one of U+0022, U+0027, U+00AB/U+00BB both directions, U+201C/U+201D, U+201E/U+201C, U+2018/U+2019, U+201A/U+2018, U+300C/U+300D, U+300E/U+300F, U+301D/U+301E pairs into exactly one span with the right `inner`; (b) a mark between two `\p{L}` letters (Latin, Cyrillic and Greek letters generated by code point) is never a delimiter, so `X’Y` alone yields zero spans and zero unpaired; (c) nested `“a ‘b’ c”` yields one outer span; (d) a lone opener yields `unpaired: 1`, zero spans; (e) an empty span `""` is not a span; (f) `unquoteSpans` removes exactly the two marks and keeps every other byte; (g) `normalizeForQuoteComparison` folds NFD to NFC, `**b**`, `_i_`, `` `c` ``, `~~s~~`, `[t](u)`, `[[p|a]]`, `[[p]]`, `> `, `- `, `1. ` and a trailing ` ^id`, collapses whitespace, and does NOT change case or punctuation; (h) `splitEllipsisFragments` splits on U+2026 and on three or more dots, drops empty fragments; `spanOccursIn` requires order and no overlap.
+- **Acceptance**: tests pass; `bun run typecheck` clean.
+- **Depends on**: A1 (ordering only)
+
+### Task A3: quote outcome vocabulary, report types, named error and wire code
+- **Files**: `src/core/brain/distill/quote-verdict.ts` (new), `src/mcp/tool-error-codes.ts`, `tests/core/brain/distill/quote-verdict.test.ts` (new), `tests/mcp/tool-error-codes.test.ts`, `tests/core/architecture/verdict-vocabulary-census.test.ts`
+- **Test first**: (a) four-piece shape of `QUOTE_CHECK_OUTCOME`; `QUOTE_CHECK_OUTCOME.urlOnly === CAPTURE_SCOPE.urlOnly` (one name for "no local bytes", decision 13); (b) `QuoteCheckError` carries `code === "quote_unverified"`, its message contains `claim 0: not-in-block` for a finding at index 0 and never the span text; (c) `codeForError(new QuoteCheckError([...]))` is `quote_unverified` and `TOOL_ERROR_CODE.quoteUnverified === QUOTE_UNVERIFIED_CODE`; (d) census lists `QUOTE_CHECK_OUTCOME`.
+- **Acceptance**: tests pass, including the existing `tool-error-codes` and census suites; `bun run typecheck` clean.
+- **Depends on**: A2
+
+### Task A4: one source read feeds the digest and the bytes
+- **Files**: `src/core/brain/intake/source-trust.ts`, `src/core/brain/ingest/content-manifest.ts`, `tests/core/brain/intake/source-origin-bytes.test.ts` (new)
+- **Test first**: (a) `readSourceOrigin` on a vault file returns `trust: trusted`, `contentHash === hashFile(abs)`, and `sha256(bytes) === contentHash`; (b) on a URL and on a missing file returns untrusted with neither `contentHash` nor `bytes`; (c) above `SOURCE_HASH_MAX_BYTES` throws `SourceTrustError`, as `classifySourceOrigin` does; (d) `hashBytes` equals `hashFile` for a file with a BOM and invalid UTF-8; (e) a spy on `readFileSync` (or a counting wrapper) proves one read per call.
+- **Acceptance**: new test passes; `tests/core/brain/intake/*.test.ts` and `tests/core/brain/ingest/content-manifest.test.ts` pass unchanged; `bun run typecheck` clean.
+- **Depends on**: none
+
+### Lane B - quote check and the distill write path
+
+### Task B1: block resolver
+- **Files**: `src/core/brain/distill/block-resolve.ts` (new), `tests/core/brain/distill/block-resolve.test.ts` (new)
+- **Test first**: (a) a paragraph line ending ` ^p1` resolves to the paragraph without the marker (multi-line paragraph included); (b) a list item ending ` ^li` resolves to that item; (c) a standalone `^t1` line after a table or blockquote resolves to that block; (d) an id inside a fenced code block is ignored (`not-found`); (e) two definitions give `ambiguous`; (f) an id that is a prefix of another (`^a` vs `^ab`) does not match; (g) CRLF sources resolve identically.
+- **Acceptance**: tests pass.
+- **Depends on**: lane A complete
+
+### Task B2: the pure quote check
+- **Files**: `src/core/brain/distill/quote-check.ts` (new), `tests/core/brain/distill/quote-check.test.ts` (new)
+- **Test first**: (a) a verbatim span in a claim citing an existing block -> `verified-in-block`, claim unchanged; (b) the same span differing in one word -> `not-in-block`, the claim's marks removed and words kept, finding `{claim: 0, outcome: "not-in-block", span}`; (c) a claim without `block` matched anywhere -> `verified-in-source`, missing -> `not-in-source`; (d) a cited block that does not resolve -> `block-not-found` even when the span appears elsewhere in the source (decision 7); duplicate -> `block-ambiguous`; (e) evidence `url-only` -> every span `url-only`, unquoted; evidence `not-text` -> `source-not-text`; (f) curly versus straight inner marks, NFD versus NFC, emphasis in the source and collapsed whitespace all verify; a case difference does not; an ellipsis span verifies only with fragments in order; `[sic]` fails; (g) no spans anywhere -> `report: null` and claims identical; (h) 30 failing spans -> `total: 30`, `returned: 25`, `truncated: true`; `span` capped at 120 characters; (i) unpaired marks counted and claims unchanged.
+- **Acceptance**: tests pass.
+- **Depends on**: B1
+
+### Task B3: quote check inside `distillSource`, strict refusal
+- **Files**: `src/core/brain/distill/distill-source.ts`, `tests/core/brain/distill/distill-source-quotes.test.ts` (new), `tests/core/brain/distill/distill-source.test.ts`
+- **Test first**: (a) a trusted source with a block-cited verbatim quote: page keeps the marks, frontmatter has `quotes_verified: 1` and `quotes_unquoted: 0`, result `quotes.verified_in_block === 1`; (b) a paraphrase in marks: page shows the words without marks, `quotes_unquoted: 1`, result names the finding; (c) `strictQuotes: true` with the paraphrase throws `QuoteCheckError` and writes nothing (no file, no directory created); (d) re-running the same input is inert (bytes and `updated_at` unchanged); (e) a page with no quoted spans is byte-identical to the 3ec2334a output (existing `distill-source.test.ts` and `distill-source-trust.test.ts` pass unchanged); (f) the check uses `readSourceOrigin` bytes: the source is rewritten between two calls and the verdict follows the bytes whose digest is on the page; (g) the header comment states the new contract (structural validation PLUS the quote check).
+- **Acceptance**: tests pass; existing distill suites pass unchanged.
+- **Depends on**: B2
+
+### Task B4: capture scope and the bounded excerpt on distillation pages
+- **Files**: `src/core/brain/distill/distill-source.ts`, `tests/core/brain/distill/distill-source-capture.test.ts` (new), `tests/core/brain/distill/distill-source-trust.test.ts` (expected: unchanged)
+- **Test first**: (a) a trusted source -> `captureScope: "full-local"`, no `capture_scope` key, page byte-identical to before; (b) a URL source -> `capture_scope: url-only` on the page beside `untrusted_source: true`, result `captureScope: "url-only"`, any quoted span unquoted with outcome `url-only`; (c) a URL source with `excerpt` -> `capture_scope: bounded-local`, `excerpt_hash` equal to `excerptDigest(excerpt)`, a `## Excerpt` section that `readExcerptSection` returns verbatim, and a quote present in the excerpt verifies (`verified-in-source`, or `verified-in-block` when the excerpt carries the cited `^id`); (d) `excerpt` with a trusted source throws `CaptureExcerptError` and writes nothing; an over-cap excerpt likewise; (e) idempotent re-run with the excerpt is inert.
+- **Acceptance**: tests pass; `bun run typecheck` clean.
+- **Depends on**: B3
+
+### Lane C - capture scope on ingest and research, the detector
+
+### Task C1: capture scope on ingest summary pages
+- **Files**: `src/core/brain/ingest/ingest.ts`, `tests/core/brain/ingest/ingest-capture-scope.test.ts` (new)
+- **Test first**: (a) an in-vault source -> result `captureScope: "full-local"`, page byte-identical to the 3ec2334a output; (b) a URL source -> page carries `capture_scope: url-only`, result `captureScope: "url-only"`; (c) re-ingest of an unchanged URL source is inert.
+- **Acceptance**: new test and `tests/core/brain/ingest/ingest.test.ts`, `tests/core/brain/intake/intake-trust.test.ts` pass.
+- **Depends on**: lane A complete
+
+### Task C2: per-source capture scopes on research reports
+- **Files**: `src/core/brain/research/research.ts`, `tests/core/brain/research/research-capture-scope.test.ts` (new)
+- **Test first**: (a) every source an in-vault file -> no `capture_scopes` key, result `captureScopes` all `full-local`; (b) one URL and one vault file -> `capture_scopes: [url-only, full-local]` in `sources` order; (c) a wikilinked source `[[x.md]]` is classified by its target.
+- **Acceptance**: new test and `tests/core/brain/research/research.test.ts` pass.
+- **Depends on**: C1 (ordering only; C1 and C2 touch different files)
+
+### Task C3: lift the source-link helpers
+- **Files**: `src/core/brain/source-links.ts` (new), `src/core/brain/source-cleanup.ts`, `tests/core/brain/source-links.test.ts` (new)
+- **Test first**: (a) `stringArrayField` and `wikilinkTarget` behave exactly as before (cases copied from `source-cleanup` behaviour: aliases, headings, block refs, `.md` suffix); (b) `sourcesSectionTargets` returns the targets under `## Sources` only and stops at the next heading.
+- **Acceptance**: new test passes; `tests/core/brain/source-cleanup.test.ts` and `tests/core/brain/gates/recoverability.test.ts` pass unchanged.
+- **Depends on**: C2 (ordering only)
+
+### Task C4: the `capture-scope` hygiene detector
+- **Files**: `src/core/brain/hygiene/detectors/capture-scope.ts` (new), `src/core/brain/hygiene/types.ts`, `src/core/brain/hygiene/scan.ts`, `tests/core/brain/hygiene-capture-scope.test.ts` (new), `tests/mcp/hygiene-link-integrity.test.ts`
+- **Test first**: (a) a research report citing only URLs -> one `warning` finding with `proposed_action: review` and the URLs in `evidence.sources`; (b) a report citing one URL and one vault file -> no finding; (c) a released entity (`status: active`, no marker, `## Sources` citing a URL) -> finding; the same entity in `quarantine` -> none; (d) an ingest page with `untrusted_source: true` -> none (gate excludes it); (e) a distillation page with no `capture_scope` whose vault source file was deleted -> finding with `evidence.stamped: null` (re-derived, decision 17); (f) a `bounded-local` page with an intact excerpt -> none; with the excerpt section removed or edited -> finding with `stamped: "bounded-local"`; (g) the detector runs in the default sweep; `HYGIENE_DETECTOR_IDS` pin updated; finding ids are deterministic.
+- **Acceptance**: tests pass; `tests/cli/brain-hygiene.test.ts` still passes (lane D adjusts it only if it pins the default list).
+- **Depends on**: C3
+
+### Lane D - surfaces, envelope tests and docs
+
+### Task D1: `brain_distill_source` surface
+- **Files**: `src/mcp/brain/distill-tools.ts`, `src/mcp/registry-guard.ts`, `tests/mcp/distill-tool.test.ts`, `tests/mcp/distill-quotes-envelope.test.ts` (new)
+- **Test first**: (a) `strict_quotes` and `excerpt` are declared with descriptions of 160 characters or fewer, the tool description stays within 300 and still contains `untrusted_source` and `search_trust_gate_enabled`; (b) the result carries `capture_scope` always and `quotes` only when spans exist, with the contract keys verbatim; (c) a strict refusal answers a JSON-RPC error whose `data.code` is `quote_unverified` and writes nothing; (d) an inadmissible excerpt answers `invalid_params` with a message naming the reason; (e) the budget rationale string for `brain_distill_source` names the quotes report.
+- **Acceptance**: tests pass, `tests/mcp/registry-guard.test.ts` and `tests/mcp/tool-error-envelope.test.ts` pass.
+- **Depends on**: B4
+
+### Task D2: `o2b brain distill` surface
+- **Files**: `src/cli/brain/verbs/distill.ts`, `src/cli/command-manifest.ts`, `tests/cli/brain-distill.test.ts`
+- **Test first**: (a) `--strict-quotes` on a paraphrase exits 1 with `distill: ` plus the `QuoteCheckError` message, and writes nothing; (b) `--excerpt-file` with a URL source writes a `bounded-local` page and the human line ends with ` [untrusted_source] [bounded-local]` (order pinned); (c) a checked quote adds ` [quotes verified:1 unquoted:0]`; (d) `--json` carries `capture_scope` and `quotes`; (e) the existing human line for a clean trusted run is unchanged; (f) the manifest entry lists the two new flags.
+- **Acceptance**: tests pass.
+- **Depends on**: B4
+
+### Task D3: ingest, research and hygiene surfaces
+- **Files**: `src/mcp/brain/ingest-tools.ts`, `src/mcp/brain/research-tools.ts`, `src/mcp/brain/hygiene-tools.ts`, `tests/mcp/ingest-tool.test.ts`, `tests/mcp/research-tool.test.ts`, `tests/cli/brain-hygiene.test.ts`
+- **Test first**: (a) `brain_ingest_source` returns `capture_scope`; (b) `brain_research_report` returns `capture_scopes` aligned with `sources`; (c) the `brain_hygiene` schema enum and description include `capture-scope` and stay within the registry caps; (d) the CLI hygiene default sweep lists the new detector where the test pins the list.
+- **Acceptance**: tests pass; `tests/mcp/registry-guard.test.ts` passes.
+- **Depends on**: C4
+
+### Task D4: reference docs
+- **Files**: `docs/mcp.md`, `docs/cli-reference.md`, `docs/how-it-works.md`
+- **Test first**: none new; the docs suites (`tests/docs/*.test.ts`) must pass.
+- **Content**: a `brain_distill_source` section in `docs/mcp.md` and an `o2b brain distill` entry in `docs/cli-reference.md` (neither exists): inputs, the quote check, the matching tolerance (decision 5-7), the unquote default and the strict refusal with `quote_unverified`, the `quotes` report keys, `capture_scope` and the excerpt; the `capture-scope` detector added to the hygiene detector lists (`docs/cli-reference.md:493`, `docs/mcp.md:379` and the detector table near `:1867`); a short capture-scope section in `docs/how-it-works.md` (the three values, stamp at write, re-derive on read, the warning). Keep `docs/mcp.md:1310-1318` true. English, "Open Second Brain" in full, no exclamation marks, no card ids.
+- **Acceptance**: `bun test tests/docs` passes; every flag and key named in the docs exists in the code.
+- **Depends on**: lane A complete (drafted from the contract), finalised after D1-D3.
diff --git a/docs/brainstorm/distilled-provenance/variants.md b/docs/brainstorm/distilled-provenance/variants.md
new file mode 100644
index 00000000..18bdc043
--- /dev/null
+++ b/docs/brainstorm/distilled-provenance/variants.md
@@ -0,0 +1,55 @@
+# Distilled provenance - variants (audit trail)
+
+Consultant: Claude Code (`claude -p`, primary), prompt in `cli-output/prompt.md`, raw output in `cli-output/claude.md`. It returned three parseable variants and exited 0, so the Codex fallback was not run.
+
+## Variants, verbatim
+
+### Variant 1: Gate in distillSource, scope derived at classification
+
+- **Approach**: Extend `classifySourceOrigin` to keep the bytes it already reads and return `{trust, contentHash, bytes, captureScope}` in one pass, so the capture scope is a by-product of the existing trust classification (bytes present and complete gives `full-local`, bytes truncated by a ceiling gives `bounded-local`, URL or missing file gives `url-only`). `distillSource` scans each claim for `\p{Quotation_Mark}` pairs, normalises both sides for comparison only, and substring-matches against the whole source body; a failed span has its marks stripped and the receipt gains a `quote_fidelity` finding, while a `url-only` source yields a named `unavailable`. The health side is one hygiene detector that walks active pages to cited sources and warns when any cited source page carries `capture_scope: url-only`.
+- **Trade-offs**:
+ - Pro: smallest surface; one read of the source, no block resolver, no new module layout.
+ - Pro: the trust classifier is already the chokepoint for "no local bytes", so the shared vocabulary falls out naturally.
+ - Con: whole-source matching ignores the block id, so a quote that exists elsewhere in the source passes even when the cited block is wrong.
+ - Con: only guards the `distillSource` path; `brain_write_session` bytes and already-written pages are never checked.
+ - Con: `classifySourceOrigin` returning up to 8 MiB of bytes widens its contract for every other caller.
+- **Complexity**: small
+- **Risk**: medium
+
+### Variant 2: Shared provenance module, write-time gate plus lint detector
+
+- **Approach**: Add a `provenance` module holding the four-piece `capture_scope` vocabulary (registered in the census), a `readSourceEvidence()` that returns bytes, hash and scope and is called by ingest, distill and trust classification, a quote-span scanner over `\p{Quotation_Mark}` with a word-boundary rule for U+2019, a comparison-only normaliser separate from `QUOTE_VARIANT_RE`, and a best-effort block resolver that narrows the match window when the cited `^id` exists and falls back to the whole source otherwise. The verifier runs twice: at write time inside `distillSource` (failed span unquoted, receipt finding `quote_fidelity` with `verified | unquoted | unavailable`), and as a hygiene detector over existing distillations and session pages so caller-authored bytes and old pages surface the same finding. `capture_scope` is written to the frontmatter of new source, distillation and report pages; the url-only-backs-active-knowledge check is a second hygiene detector with a doctor code and exit-census row, deriving scope for pre-release pages from the existing `missing` sentinel.
+- **Trade-offs**:
+ - Pro: one vocabulary and one evidence reader serve both cards; "no local bytes" is a single enum member everywhere.
+ - Pro: covers the `brain_write_session` bypass and the already-written corpus through the existing detector plumbing that already reaches `brain_status`.
+ - Pro: block-scoped matching when ids exist, graceful whole-source fallback for the fixtures that cite `^abc` with no ids.
+ - Con: more files and tests; the block resolver is new code with its own edge cases (ids on list items, headings, tables).
+ - Con: new frontmatter key on new pages means those writers' byte-stability fixtures need regeneration (existing pages untouched).
+ - Con: the detector re-reads sources on every hygiene pass, which is a cost on large vaults.
+- **Complexity**: medium
+- **Risk**: low
+
+### Variant 3: Typed claims and a configurable quote policy
+
+- **Approach**: Change the claim contract so callers mark `kind: quote | paraphrase`; `distillSource` renders a quote only after verification and otherwise applies a configured policy (`distill_quote_policy: unquote | drop | reject`), where `reject` fails the whole write with a new coded error registered in tool-error-codes. `capture_scope` becomes an explicit ingest input validated against what was saved (mismatch is an error), and the url-only warning is a doctor check only. Failed spans under `unquote` are also recorded in a frontmatter list of unverified block ids on that page.
+- **Trade-offs**:
+ - Pro: strongest guarantee; a strict install can refuse to persist a paraphrase disguised as a quote.
+ - Pro: typed claims give downstream recall a clean quotable-or-not signal without re-scanning text.
+ - Con: breaks the current `{text, block?}` caller contract and the MCP schema, with the tool description at 280 of 300 characters.
+ - Con: three policies and an explicit scope input multiply the test matrix and the config surface for one release.
+ - Con: `drop` silently loses knowledge, and `reject` turns a model's punctuation choice into a hard write failure.
+ - Con: no detector over existing pages, so the corpus written before this release stays unchecked.
+- **Complexity**: large
+- **Risk**: high
+
+### Recommended: Variant 2
+**Rationale**: It is the only variant that satisfies both cards with one vocabulary and one evidence reader while covering the `brain_write_session` bypass and the already-written corpus through detector plumbing the project already has. Variant 1 leaves two write paths unguarded and silently widens the trust classifier's contract; Variant 3 breaks the claim and MCP contracts and adds policy surface this release does not need. The block resolver with whole-source fallback keeps the match deterministic and fixture-compatible, and the unquote-plus-named-finding fallback follows the existing receipt convention without losing text.
+
+## Orchestrator decision
+
+Variant 2, narrowed and with one addition (design.md, "Chosen approach").
+
+- **Variant 1 lost** because it matches against the whole source and ignores the cited block, so a quote that exists elsewhere passes under a wrong citation; and because it makes `bounded-local` mean "truncated by the size ceiling", which would turn today's named `SourceTrustError` refusal into a degraded success. Its one-read idea is kept (premise A.5, task A4) through a new `readSourceOrigin` rather than by widening `classifySourceOrigin` for every caller.
+- **Variant 3 lost** because it breaks the `{text, block?}` claim contract and the MCP schema of a tool at 280 of 300 description characters, adds a three-way config policy, and its `drop` policy loses caller-supplied knowledge. Its strict refusal with a registered code survives as an opt-in argument (decision 11), not a config key.
+- **Variant 2, narrowed**: the second quote detector over existing and `brain_write_session` pages is deferred (out of scope by the orchestrator's steer); the doctor code is dropped because the hygiene finding already reaches `brain_status`; deriving scope from the freshness `missing` sentinel is replaced by the trust classifier, because distill, ingest, research and entity pages do not carry the freshness contract; the "fall back to the whole source when the block does not resolve" rule is replaced by a named `block-not-found` outcome, because the citation is part of what the page asserts.
+- **Addition**: an explicit `excerpt` argument on distillation, so `bounded-local` has a real producer without wiring network egress (`extractPage`). It is also where the two cards meet: an excerpt is the only local text a quote from a URL source can be checked against.
diff --git a/docs/cli-reference.md b/docs/cli-reference.md
index f7c8acba..db4e176c 100644
--- a/docs/cli-reference.md
+++ b/docs/cli-reference.md
@@ -490,7 +490,7 @@ o2b brain session-grep --query <text> [--session-id <id>] [--limit <n>] [
o2b brain session-describe --session-id <id> [--json]
o2b brain session-expand <record-id> [--raw-limit <n>] [--cursor <offset>] [--json]
o2b brain handoff <session-file> [--session-id <id>] [--format auto|claude|codex|hermes] [--json] - write Brain/handoffs/<date>-<scope>.md (since v0.37.0)
-o2b brain hygiene scan | apply --ids <id,...> [--detectors conflicts,dedup,freshness,usefulness,slug-collisions,tags] [--dry-run] [--json] - hygiene findings pipeline; review findings never execute (since v1.3.0); the default sweep runs every detector except the opt-in ones (since v1.64.0: `slug-collisions` is default-on and reports same-stem groups such as topic.md beside topic-2.md as info findings, `tags` is opt-in and audits inline body tags only - frontmatter tags arrays are out of scope)
+o2b brain hygiene scan | apply --ids <id,...> [--detectors conflicts,dedup,freshness,usefulness,slug-collisions,tags,capture-scope] [--dry-run] [--json] - hygiene findings pipeline; review findings never execute (since v1.3.0); the default sweep runs every detector except the opt-in ones (since v1.64.0: `slug-collisions` is default-on and reports same-stem groups such as topic.md beside topic-2.md as info findings, `tags` is opt-in and audits inline body tags only - frontmatter tags arrays are out of scope; since v1.67.0 `capture-scope` is default-on and warns, with proposed action review, when a retrievable page of active knowledge cites only sources that are currently url-only)
o2b brain hygiene scan: with the optional `dedup` decision-model use in enforce, dedup findings carry an advisory `decision_model` verdict and a confident `different` is listed last; `o2b brain doctor` annotates `entity-alias-candidate` warnings the same way; `apply` ignores verdicts
o2b brain refresh --stale [--dry-run] [--json] - targeted recompile of stale derived pages; orphans archive into Brain/.snapshots (since v1.3.0)
o2b brain anticipate --session <id> [--refresh] [--signal <text>] [--json] - read or warm the anticipatory context cache for the session's lineage root (since v1.3.0)
@@ -1141,6 +1141,54 @@ note: no conversation matched; <scanned> transcript file(s) scanned, <n> from an
The four reasons sum to `scanned` minus the exported records, so an empty
file under exit `0` can never be read as a machine that recorded nothing.
+### Source distillation
+
+```text
+o2b brain distill <source> (--claims <json> | --claims-file <path>) [--strict-quotes] [--excerpt-file <path>] [--agent <name>] [--vault <path>] [--json]
+```
+
+Writes one idempotent distillation page per source from atomic claims the
+calling agent supplies as a JSON array of `{ "text": "...", "block": "^abc" }`
+objects (or an object with a `claims` array), at most 1000 per call, each
+claim's text on one line. `<source>` is a vault-relative path or a URL. Open Second Brain runs no model: it validates the claims,
+checks every quoted span in them against the source, and writes the page.
+
+Since v1.67.0, every quoted span in a claim is compared with the block the
+claim cites, or with the whole source when the claim cites no block. A
+quotation mark is any character with the Unicode `Quotation_Mark` property,
+paired by position, and an apostrophe inside a word is never a delimiter. The
+comparison normalises both sides the same way (NFC, inline Markdown reduced
+to display text, block markers and a trailing ` ^id` removed, whitespace
+collapsed) and never folds case, punctuation or wording; an ellipsis splits
+a span into fragments that must appear in order. A cited block that does
+not resolve fails the span rather than falling back to the whole source.
+The full rules are in [the MCP reference](mcp.md#source-distillation).
+
+- A span that fails is unquoted: its quotation marks are removed, the words
+ stay, the page is written, and the result reports it.
+- `--strict-quotes` refuses the whole write instead, exits `1`, and writes
+ nothing. The message reads `distill: quoted spans failed verification:`
+ followed by `claim <index>: <outcome>` pairs, never claim text.
+- Since v1.67.0, `--excerpt-file <path>` stores the verbatim text read from
+ a source the vault does not hold (a URL, or a path with no file). The
+ page records `capture_scope: bounded-local` and an `excerpt_hash`, keeps
+ the text under a `## Excerpt` heading, and checks quotes against it. The excerpt is
+ refused, before anything is written, for a source the vault holds, when
+ it holds no text or contains NUL, or above 65,536 bytes. A file whose bytes are not valid
+ UTF-8 is a usage error (exit `2`, `distill: excerpt file is not valid
+ UTF-8`), so the stored excerpt is always the file's own bytes.
+
+The success line keeps its earlier form for a clean run over a local source
+and gains suffixes in this order: ` [untrusted_source]` when the page is
+in the untrusted lane, ` [bounded-local]` when an excerpt was stored, and
+` [quotes verified:V unquoted:U]` when spans were checked, where `V` is the
+number of spans verified in a block or in the source and `U` the number
+unquoted. `--json` adds `capture_scope` (always: `full-local`,
+`bounded-local` or `url-only`) and, when spans were checked, `quotes`:
+`checked`, `verified_in_block`, `verified_in_source`, `unquoted`,
+`unpaired`, `findings` (each `{ claim, outcome, span }`, capped at 25 with
+`total`, `returned` and `truncated`).
+
### Knowledge packs
A knowledge pack is a selected subset of Brain knowledge - rules and the
diff --git a/docs/how-it-works.md b/docs/how-it-works.md
index 94a884c3..b56e74ca 100644
--- a/docs/how-it-works.md
+++ b/docs/how-it-works.md
@@ -1397,6 +1397,48 @@ of the system follows - storage stays canonical UTC, conversion
happens at the boundary, and an unconfigured vault renders
byte-identically.
+## Checked quotes and capture scope (since v1.67.0)
+
+A distillation page presents itself as a faithful record of its source, so
+two things it asserts are now checked or recorded rather than assumed.
+
+**Quotes are verified at write time.** When an agent submits claims to
+`brain_distill_source` or `o2b brain distill`, every span between
+quotation marks is compared with the block the claim cites, or with the
+whole source when it cites none. The comparison is deterministic and
+language-neutral: quotation marks are recognised by their Unicode property,
+paired by position, and both sides are normalised the same way for the
+comparison only. A span that does not verify loses its quotation marks and
+is named in the result, so the page never shows an unverified quote; a
+caller that prefers a refusal passes `strict_quotes` and gets the
+registered error code `quote_unverified` with nothing written. The bytes
+checked are the bytes hashed into `source_hash`, read once.
+
+**Every page says how much of its source the vault holds.** The capture
+scope has three values:
+
+- `full-local`: the source is a file this vault holds;
+- `bounded-local`: the source has no local bytes, but the page stores a
+ verbatim excerpt the writer read, with its digest (`excerpt_hash`);
+- `url-only`: the page rests on a locator only - a URL, or a path with no
+ file behind it.
+
+The scope is stamped at write time on distillation, ingest summary and
+research report pages (`capture_scope`, or `capture_scopes` per source on a
+report), and only when it is not `full-local`, so pages over local sources
+stay byte-identical. An absent stamp means "full-local, or written before
+this release", and readers re-derive rather than assume: the default-on
+`capture-scope` hygiene detector re-classifies each cited source from its
+identity at scan time, without reading its bytes. A retrievable page of
+active knowledge - a distillation, an ingest summary, a research report or
+a canonical entity - whose every cited source is currently `url-only` gets
+a `warning` finding with `proposed_action: review`. A local source that
+was deleted therefore surfaces as `url-only`, a `bounded-local` page counts
+as backed only while its excerpt still matches its digest, and quarantined
+pages are skipped: they already carry the `untrusted_source` marker, which
+names the same condition (and which the retrieval trust gate excludes when
+`search_trust_gate_enabled` is on).
+
## Safety properties
These are invariants of the system, not configuration to enable.
diff --git a/docs/mcp.md b/docs/mcp.md
index 23f49aa1..de85814a 100644
--- a/docs/mcp.md
+++ b/docs/mcp.md
@@ -279,6 +279,7 @@ generic tokens:
| `unknown_argument` | the call carried an argument the tool does not declare (see "Argument contract" above) |
| `brain_artifact_unparseable` | a Brain artifact file (a preference or a retired rule, for example) could not be parsed |
| `argument_forbidden` | `brain_scaffold_stub` was given an argument that belongs to its other action |
+| `quote_unverified` | `brain_distill_source` with `strict_quotes: true` refused the write because a quoted span in a claim did not verify against its source (see "Source distillation" below) |
plus every member of the vocabularies the core already defines, passed
through unchanged:
@@ -376,7 +377,7 @@ flags for a narrower per-process full server.
| `brain_agenda` | Stateless agenda synthesis over caller-provided calendar events (the host fetches them; the Brain never calls a calendar API): overlap conflicts, free focus blocks (optionally clipped to a workday window), and events organised outside the operator's own email domain(s). No vault writes. | `events` |
| `brain_context_presets` | Show, suggest, or diff read-only context budget presets (`tight-context`, `long-context`) without writing config. | `operation` |
| `brain_pre_compact_extract` | Extract decision/commitment/outcome/rule/open-question records from bounded text into continuity storage. | `session_id`, `turn_start`, `turn_end`, `text` |
-| `brain_hygiene` | Memory hygiene: `scan` findings (conflicts, dedup, freshness, usefulness; since v1.64.0 `slug-collisions` is default-on — same-stem allocation residue such as `topic.md` beside `topic-2.md` — and `tags` is opt-in, auditing inline body tags only, never frontmatter tags arrays; the default is the sweep of every registered detector except the opt-in ones), `apply` selected ids, `refresh` stale pages. Resolver command comes from `_brain.yaml` only. With the optional `dedup` decision-model use in enforce, `scan` dedup findings carry an advisory `decision_model` verdict; `apply` never reads it. | `mode` |
+| `brain_hygiene` | Memory hygiene: `scan` findings (conflicts, dedup, freshness, usefulness; since v1.64.0 `slug-collisions` is default-on — same-stem allocation residue such as `topic.md` beside `topic-2.md` — and `tags` is opt-in, auditing inline body tags only, never frontmatter tags arrays; since v1.67.0 `capture-scope` is default-on and warns when active knowledge rests only on `url-only` sources; the default is the sweep of every registered detector except the opt-in ones), `apply` selected ids, `refresh` stale pages. Resolver command comes from `_brain.yaml` only. With the optional `dedup` decision-model use in enforce, `scan` dedup findings carry an advisory `decision_model` verdict; `apply` never reads it. | `mode` |
| `brain_anticipatory_context` | Turn-specific context bundle kept warm by lifecycle hooks, keyed by the session's lineage root; reports `cache_state` warm / stale / miss. | `session_id` |
| `brain_session_grep` | Search imported session recall raw turns and deterministic summary nodes. | `query` |
| `brain_session_describe` | Describe raw-turn counts and summary depths for one imported session recall DAG. | `session_id` |
@@ -1318,6 +1319,140 @@ and bypass the note-write path entirely. Log appends (`brain_note`,
`brain_apply_evidence`) name no note path and are not linted, because the
log line is machine-composed rather than authored.
+## Source distillation
+
+`brain_distill_source` writes one idempotent distillation page per source
+from atomic claims the calling agent supplies. Open Second Brain runs no
+model here: it validates the claims, checks every quotation inside them
+against the source, and writes the page. The page cites each claim as
+`[[source#^block]]` when the claim names a block.
+
+Inputs:
+
+| Argument | Type | Meaning |
+| --- | --- | --- |
+| `source_path` | string, required | the source identity: a vault-relative path or a URL |
+| `claims` | array, required | `{ text, block? }` items; `block` is the source block id (`^abc` or `abc`); at most 1000 per call, and a claim's text is one line |
+| `strict_quotes` | boolean, optional | since v1.67.0, refuse the whole write when any quoted span fails the check |
+| `excerpt` | string, optional | since v1.67.0, the verbatim text the caller read from a `url-only` source, stored on the page |
+| `agent` | string, optional | agent identity override |
+
+Result keys: `distillation_path`, `created`, `claim_count`, `source_hash`
+(absent when the source had no bytes to hash), `trust` (`trusted` or
+`untrusted`), and since v1.67.0 `capture_scope` (always present) and
+`quotes` (present only when at least one claim contains a quoted span).
+
+### The quote check
+
+Since v1.67.0, every quoted span in a claim is checked before the page is
+written. A quoted span is the text between a pair of quotation marks in a
+claim. A quotation mark is any character with the Unicode
+`Quotation_Mark` property, so straight, curly, low-high, guillemet and
+corner-bracket quotes all count without a list of languages. Marks pair
+by position: an opener follows the start of the text, whitespace or
+punctuation other than a mark that just closed, a closer is followed by
+the end, whitespace or punctuation. A mark whose Unicode category is
+opening or closing punctuation (low-9 marks, corner brackets) is
+directional by itself, so corner brackets pair even when glued to CJK
+letters. Any other mark with a letter on both sides (an apostrophe
+inside a word) is never a delimiter. A closer pairs only with an opener
+of its own width (single with single, double with double), so a
+possessive apostrophe never ends a double quote. Only the outermost
+closed span is checked; a quote inside it is part of its text. A mark
+that finds no partner is counted as `unpaired` and left on the page
+untouched, and it hides no later quote. A mark separated from its text by
+an ordinary space is counted as unpaired, not checked; a single no-break
+or thin space, as in spaced guillemets, is allowed. A pair holding fewer
+than two letters or digits (`rock 'n' roll`) is not a quote and counts
+as two unpaired marks. An empty span is not a quote.
+
+Each span is compared with the evidence after the same normalisation is
+applied to both sides: Unicode NFC, inline Markdown reduced to its
+display text (paired emphasis and code markers, so `snake_case` and
+`2*3` stay as written, `[text](url)`, `[[target|alias]]`,
+`[[target]]`), line-leading block markers (on the evidence side only: a
+span's own leading `3. `, `- ` or `> ` is wording) and a trailing ` ^id`
+removed, quotation-mark variants folded, whitespace runs collapsed. Case,
+punctuation and wording are never folded. The normalised span must then
+be an exact substring of the normalised evidence that starts and ends on
+a word boundary, by the Unicode word rules, so `"safe"` does not verify
+inside `unsafe`. A span holding an ellipsis (`…` or three or more dots)
+is split there, and every fragment must appear in order without overlap;
+an edge next to an ellipsis may fall inside a word. A bracketed insertion such as
+`[sic]` is not interpreted and fails as a mismatch. The page bytes are
+never normalised; normalisation is used for the comparison only.
+
+The evidence depends on the claim:
+
+- A claim with `block` is compared with that block only. A block id that
+ resolves nowhere gives `block-not-found`, an id defined twice gives
+ `block-ambiguous`; neither falls back to the whole source, because the
+ citation is part of what the page asserts.
+- A claim without `block` is compared with the whole source and verifies
+ as `verified-in-source`, a weaker outcome than `verified-in-block`.
+- A source whose bytes are not valid UTF-8 gives `source-not-text` for
+ every span. A source with no local bytes gives `url-only`.
+- A vault source the caller may not read at its transport reach (or, with
+ `integrity.owner_scope_delivery: fail`, another owner's page) is treated
+ exactly as an absent source: every span gives `url-only`, no
+ `source_hash` is returned or recorded on the page, the page lands in the
+ untrusted lane with `capture_scope: url-only`, and an `excerpt` is
+ accepted for it.
+
+The bytes checked are the bytes read once for `source_hash`, so the
+verdict and the digest on the page always describe the same content.
+
+**A failed span is unquoted by default.** Its two quotation marks are
+removed, the words stay, the write lands, and the result names the
+failure. The page never shows an unverified quote. Unquoting changes the
+bytes the caller supplied; this is deliberate, the change is reported in
+`quotes.findings`, and an identical re-run produces an identical page.
+
+**Strict refusal.** With `strict_quotes: true` the whole write is refused
+when any span would be unquoted, and nothing is written. The refusal is a
+JSON-RPC error with `INVALID_PARAMS` and `error.data.code` set to
+`quote_unverified`; its message names claim indices and outcomes
+(`claim 0: not-in-block`), never claim text or paths. Unpaired marks
+never trigger a refusal.
+
+The `quotes` report:
+
+| Key | Meaning |
+| --- | --- |
+| `checked` | quoted spans examined |
+| `verified_in_block` | spans found verbatim in the cited block |
+| `verified_in_source` | spans found verbatim in the whole source (claims without `block`) |
+| `unquoted` | spans that failed and lost their quotation marks |
+| `unpaired` | quotation marks without a partner, counted only |
+| `findings` | one `{ claim, outcome, span }` per unquoted span: `claim` is the 0-based claim index, `outcome` one of `not-in-block`, `not-in-source`, `block-not-found`, `block-ambiguous`, `source-not-text`, `url-only`, `span` the span text capped at 120 characters |
+| `total`, `returned`, `truncated` | the findings list is capped at 25 and declares its own truncation |
+
+A page whose claims held spans carries `quotes_verified` and
+`quotes_unquoted` in its frontmatter. A page without spans is
+byte-identical to the one earlier releases wrote.
+
+### Capture scope and the excerpt
+
+Since v1.67.0, `capture_scope` says how much of the source the vault
+actually holds:
+
+| Value | Meaning |
+| --- | --- |
+| `full-local` | the source is a file this vault holds |
+| `bounded-local` | the source has no local bytes, and the page stores a verbatim excerpt the caller read |
+| `url-only` | the source is a URL, or a vault-shaped path with no file behind it |
+
+The page records `capture_scope` in its frontmatter only when the value is
+not `full-local`, so a page over a local source stays byte-identical.
+`excerpt` makes `bounded-local` reachable: for a `url-only` source the
+caller passes the text it read, at most 65,536 bytes of UTF-8. The page
+stores it under a `## Excerpt` heading in a fenced block with the info
+string `excerpt`, records its sha256 as `excerpt_hash`, and quotes are
+checked against it. An excerpt is refused with `INVALID_PARAMS`, before
+anything is written, when the source is a file the vault holds (the file
+is the evidence), when it holds no text (only whitespace, control or
+format characters), when it contains NUL, or when it exceeds the cap.
+
## Safety notes
- The vault path is bound to the server instance at startup. Tools cannot
@@ -1893,3 +2028,37 @@ log line is machine-composed rather than authored.
group at its timeout or when the server exits. A duplicated
`retry_tasks` entry counts once, and the `status` notice names the env
override when that turned the tasks off.
+- Since v1.67.0 `brain_distill_source` checks quoted spans in claims
+ against the cited block or the source, unquotes a span that fails and
+ reports it under `quotes`, refuses the write with `quote_unverified`
+ under `strict_quotes: true`, accepts a bounded `excerpt` for a
+ `url-only` source, and always returns `capture_scope` (see "Source
+ distillation"). `brain_ingest_source` returns `capture_scope`;
+ `brain_intake_entities` and `brain_ingest_source` treat a cited vault
+ file the caller cannot read at its reach as an absent source (untrusted
+ lane, no `source_content_hash`, `url-only`);
+ `brain_research_report` returns `capture_scopes`, one per entry of
+ `sources` in the same order, and stamps a `capture_scopes` frontmatter
+ list when any source is not `full-local`; a source written without an
+ extension is matched against its `.md` note, and a source backed by a
+ vault file the caller cannot read at its reach is reported `url-only`,
+ the same answer as an absent file; a cited file the filesystem refuses
+ to stat never aborts the report and reads `url-only`; a title holding a
+ line break is refused. `brain_hygiene` gains the
+ default-on `capture-scope` detector: a retrievable page of active
+ knowledge whose every cited source is currently `url-only` gets a
+ `warning` finding with `proposed_action: review`. The scope is
+ re-derived from each source identity at scan time, so a page whose
+ local source was deleted reports as `url-only`, and a `bounded-local`
+ page counts as backed only while its excerpt still matches
+ `excerpt_hash`. Quarantined pages (those carrying the untrusted
+ marker) are skipped; the marker already names the condition. A cited
+ file the caller cannot read at its reach counts as `url-only` for that
+ caller, in the scan and in the `brain_status` hygiene count. Below
+ local reach the scan's `link_integrity` reports `measured: false` with
+ reason `reach`, because its counts are taken over the whole index.
+ `brain_ingest_batch_plan` plans a file the caller cannot read at its
+ reach as an absent one (visibility is a rule over Markdown pages, so the
+ plan's `unclassifiable` counts of other files are the same at every
+ reach), and `brain_ingest_source` records a source in
+ the content manifest only in the trusted lane.
diff --git a/docs/updating.md b/docs/updating.md
index c73047cc..8d0394f1 100644
--- a/docs/updating.md
+++ b/docs/updating.md
@@ -47,6 +47,62 @@ instruction files such as `CLAUDE.md`/`AGENTS.md`, installed
`.claude/skills/`) and warns with the exact replacement for any stale
reference it finds (`removed-tool-reference`).
+## Upgrading to 1.67.0
+
+No step below is required. Six changes are visible to an operator or a
+client.
+
+**A distilled quote that does not verify loses its quotation marks.**
+`brain_distill_source` and `o2b brain distill` now check every quoted span
+in a claim against the block it cites, or the whole source. A span the
+source does not hold is written without its two quotation marks and named
+in the result's `quotes` report; the words stay and the write lands. Pass
+`strict_quotes: true` (or `--strict-quotes`) to refuse such a write instead,
+with `error.data.code` `quote_unverified` over MCP and exit 1 on the CLI.
+See "Source distillation" in [`mcp.md`](mcp.md).
+
+**New frontmatter keys appear only on pages that are not `full-local`.**
+`capture_scope`, `capture_scopes` and `excerpt_hash` are written for a
+source that is `bounded-local` or `url-only`, and `quotes_verified` /
+`quotes_unquoted` only when a claim holds a quoted span. A distillation,
+ingest summary or research report over local sources is written
+byte-identical to before.
+
+**A claim or a research title that spans more than one line is refused.**
+A distillation claim holding a line break, or more than 1000 claims in one
+call, is refused, and so is a `brain_research_report` title holding a line
+break. Both used to be written as given.
+
+**Links inside a fenced block are masked up to its own closing fence.**
+Wikilink extraction and search indexing close a fenced block only on a run
+of its own character at least as long. Links inside a tilde fence are no
+longer extracted, and a longer fence holding a shorter backtick run no
+longer leaks the rest of its content as links. An inline code span closes
+only on a backtick run of its own length, and a backtick run with no closer
+is plain text that hides no link after it. Such a vault may show different
+backlinks and link hits after its next index pass.
+
+**The `capture-scope` hygiene detector is on by default.** A default
+`brain_hygiene` or `o2b brain hygiene` scan may report new warnings on an
+existing vault where a distillation, ingest summary, research report or
+canonical entity cites only URLs. The findings propose review and never
+change a page. Callers that pin the default detector list see the new id
+at the end. See [`cli-reference.md`](cli-reference.md).
+
+**More tools treat a page the caller cannot read at its reach as absent.**
+A remote client now gets, for such a page, exactly the answer an absent
+page gets: `brain_delete_by_source`, `brain_note_lifecycle`,
+`brain_append_note`, `brain_update_note`, `brain_write_batch`,
+`brain_note_history`, `brain_tiers`, `brain_event_trace`,
+`brain_agent_query`, `brain_agent_diff`, `brain_diarize`,
+`brain_design_note`, `brain_dream`, `brain_idea_discovery`,
+`brain_procedural_memory`, `brain_doctor`, `schema_inspect` and
+`brain_writes plan_revert` leave it out of their lists and counts or
+refuse it as missing, and the `brain_hygiene` `link_integrity` block
+reports `measured: false` with reason `reach`. A local caller and the CLI
+see no change. See the 1.67.0 entry in
+[`CHANGELOG.md`](../CHANGELOG.md).
+
## Upgrading to 1.66.0
No step below is required. Three changes are visible to a client or an
diff --git a/openclaw.plugin.json b/openclaw.plugin.json
index 11605f39..18e8eb7b 100644
--- a/openclaw.plugin.json
+++ b/openclaw.plugin.json
@@ -2,7 +2,7 @@
"id": "open-second-brain",
"name": "Open Second Brain",
"description": "Second brain for AI agents using Obsidian-compatible Markdown vaults.",
- "version": "1.66.0",
+ "version": "1.67.0",
"activation": { "onStartup": true },
"skills": ["./skills"],
"contracts": {
diff --git a/package.json b/package.json
index ed3349ad..43db28ad 100644
--- a/package.json
+++ b/package.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "1.66.0",
+ "version": "1.67.0",
"private": false,
"description": "Second brain for AI agents using Obsidian-compatible Markdown vaults. Works with Hermes, Claude Code, Codex, and OpenClaw.",
"keywords": [
diff --git a/plugin.yaml b/plugin.yaml
index 270880e1..72c0dcd5 100644
--- a/plugin.yaml
+++ b/plugin.yaml
@@ -1,5 +1,5 @@
name: open-second-brain
-version: "1.66.0"
+version: "1.67.0"
description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."
author: "Open Second Brain contributors"
memory_provider: true
diff --git a/plugins/codex/.codex-plugin/plugin.json b/plugins/codex/.codex-plugin/plugin.json
index a63a4aa9..9d936e99 100644
--- a/plugins/codex/.codex-plugin/plugin.json
+++ b/plugins/codex/.codex-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "1.66.0",
+ "version": "1.67.0",
"description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.",
"author": {
"name": "Open Second Brain contributors",
diff --git a/plugins/codex/README.md b/plugins/codex/README.md
index 197b2fe8..e707799e 100644
--- a/plugins/codex/README.md
+++ b/plugins/codex/README.md
@@ -113,7 +113,7 @@ The full router with readiness criteria is [`install.md`](https://github.com/ite
## What is new
-1.66.0 puts a stable error code on every failed MCP call, `error.data.code` on a JSON-RPC error and `_meta["open-second-brain/error"].code` on an `isError` result, drawn from one closed registry with the error prose unchanged, so a client branches on a token instead of matching English text. The optional cross-encoder rerank names why it stepped aside on the retrieval trail (`rerank-provider-unavailable` with a typed category, never cached, and `rerank-model-sunset` for a model past its announced decommission date), and `o2b brain doctor` checks the rerank configuration without sending a request. Every release is described in the [CHANGELOG](https://github.com/itechmeat/open-second-brain/blob/main/CHANGELOG.md).
+1.67.0 makes a distillation page prove what it quotes and say how much of its source the vault holds. Every quoted span in a claim is checked against the block it cites, or the whole source, when the page is written; a span that does not verify loses its quotation marks and is named in the result, or refuses the write with `quote_unverified` under `strict_quotes`. Distillation, ingest summary and research report pages record a `capture_scope` of `full-local`, `bounded-local` (a stored verbatim excerpt with its digest) or `url-only`, and a default-on `capture-scope` hygiene detector warns when active knowledge rests only on `url-only` sources. Every release is described in the [CHANGELOG](https://github.com/itechmeat/open-second-brain/blob/main/CHANGELOG.md).
## Documentation
diff --git a/plugins/hermes/plugin.yaml b/plugins/hermes/plugin.yaml
index 270880e1..72c0dcd5 100644
--- a/plugins/hermes/plugin.yaml
+++ b/plugins/hermes/plugin.yaml
@@ -1,5 +1,5 @@
name: open-second-brain
-version: "1.66.0"
+version: "1.67.0"
description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."
author: "Open Second Brain contributors"
memory_provider: true
diff --git a/pyproject.toml b/pyproject.toml
index dcb39841..e410c9b2 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -10,7 +10,7 @@ build-backend = "setuptools.build_meta"
# CLI entry points (those moved to `package.json` `bin`).
[project]
name = "open-second-brain"
-version = "1.66.0"
+version = "1.67.0"
description = "Hermes Python shim for Open Second Brain. Most of the project (CLI, MCP server, OpenClaw plugin) is TypeScript on Bun; see package.json."
readme = "README.md"
requires-python = ">=3.11"
diff --git a/src/cli/brain/verbs/distill.ts b/src/cli/brain/verbs/distill.ts
index 99be9b1d..4bbc052a 100644
--- a/src/cli/brain/verbs/distill.ts
+++ b/src/cli/brain/verbs/distill.ts
@@ -4,9 +4,14 @@
*
* Provider-agnostic: the agent supplies the atomic claims (and optional source
* block ids) as JSON - a `[{ "text": "...", "block": "^abc" }]` array via
- * `--claims` or `--claims-file`. OSB validates them and writes one idempotent
+ * `--claims` or `--claims-file`. Open Second Brain validates them, checks every
+ * quoted span against the cited block or the source, and writes one idempotent
* distillation page per source. No model, no extraction here.
*
+ * `--strict-quotes` turns an unverified span into a refusal (exit 1, nothing
+ * written); `--excerpt-file` stores the verbatim text read from a source the
+ * vault does not hold, making the page `bounded-local`.
+ *
* Exit codes: 0 on success, 1 on an operational failure, 2 on usage errors.
*/
@@ -17,7 +22,17 @@ import {
DistillValidationError,
parseDistillClaims,
type DistillClaim,
+ type DistillSourceResult,
} from "../../../core/brain/distill/distill-source.ts";
+import {
+ QuoteCheckError,
+ type QuoteCheckReport,
+} from "../../../core/brain/distill/quote-verdict.ts";
+import {
+ CAPTURE_SCOPE,
+ CaptureExcerptError,
+ type CaptureScope,
+} from "../../../core/brain/provenance/capture-scope.ts";
import { ResponseShapeError } from "../../../core/brain/response-shape.ts";
import {
INTAKE_TRUST,
@@ -27,7 +42,36 @@ import {
import { brainVerbContext, fail, ok, okJson, parse, resolveBrainAgent } from "../helpers.ts";
const USAGE =
- "usage: o2b brain distill <source> (--claims <json> | --claims-file <path>) [--vault <path>] [--json]";
+ "usage: o2b brain distill <source> (--claims <json> | --claims-file <path>) [--strict-quotes] [--excerpt-file <path>] [--agent <name>] [--vault <path>] [--json]";
+
+/** Refusal for an excerpt file whose bytes are not UTF-8 text. */
+const EXCERPT_NOT_UTF8_MESSAGE = "distill: excerpt file is not valid UTF-8";
+
+/**
+ * Strict UTF-8 decoder for the excerpt file: an invalid sequence throws
+ * instead of becoming U+FFFD, and a leading byte-order mark is kept, so the
+ * decoded text is exactly the file's bytes.
+ */
+const EXCERPT_DECODER = new TextDecoder("utf-8", { fatal: true, ignoreBOM: true });
+
+/** The excerpt file's text, or `null` when its bytes are not valid UTF-8. */
+function readExcerptFile(path: string): string | null {
+ const bytes = readFileSync(path);
+ try {
+ return EXCERPT_DECODER.decode(bytes);
+ } catch (err) {
+ if (err instanceof TypeError) return null;
+ throw err;
+ }
+}
+
+/** Errors that are the operator's input, reported as `distill: <message>`. */
+const OPERATOR_ERROR_CLASSES: ReadonlyArray<new (...args: never[]) => Error> = [
+ DistillValidationError,
+ ResponseShapeError,
+ QuoteCheckError,
+ CaptureExcerptError,
+];
/** What `--claims` accepts, named in the operator's own terms. */
const CLAIMS_SHAPE_HINT = "claims must be a JSON array of { text, block? } objects";
@@ -76,12 +120,41 @@ function untrustedNote(trust: IntakeTrust): string {
return trust === INTAKE_TRUST.untrusted ? ` [${UNTRUSTED_SOURCE_FRONTMATTER_KEY}]` : "";
}
+/**
+ * What the success line adds when an excerpt was stored. Only `bounded-local`
+ * earns a token: `url-only` is already named by the untrusted marker, and a
+ * `full-local` run keeps its line exactly as it was.
+ */
+function captureNote(scope: CaptureScope): string {
+ return scope === CAPTURE_SCOPE.boundedLocal ? ` [${CAPTURE_SCOPE.boundedLocal}]` : "";
+}
+
+/**
+ * What the success line adds when quoted spans were checked: the spans that
+ * verified (in a block or in the whole source) and the spans unquoted.
+ */
+function quotesNote(quotes: QuoteCheckReport | undefined): string {
+ if (quotes === undefined) return "";
+ const verified = quotes.verified_in_block + quotes.verified_in_source;
+ return ` [quotes verified:${verified} unquoted:${quotes.unquoted}]`;
+}
+
+/** The success line, suffixes in their pinned order. */
+function successLine(res: DistillSourceResult): string {
+ return (
+ `distilled ${res.claimCount} claim(s) -> ${res.distillationPath}${res.created ? "" : " (updated)"}` +
+ `${untrustedNote(res.trust)}${captureNote(res.captureScope)}${quotesNote(res.quotes)}`
+ );
+}
+
export async function cmdBrainDistill(argv: string[]): Promise<number> {
const { flags, positional } = parse(argv, {
vault: { type: "string" },
agent: { type: "string" },
claims: { type: "string" },
"claims-file": { type: "string" },
+ "strict-quotes": { type: "boolean" },
+ "excerpt-file": { type: "string" },
json: { type: "boolean" },
});
const source = positional[0];
@@ -105,10 +178,24 @@ export async function cmdBrainDistill(argv: string[]): Promise<number> {
? (flags["claims"] as string)
: readFileSync(flags["claims-file"] as string, "utf8");
const claims = parseClaims(claimsRaw);
+ // Read verbatim: the excerpt is stored byte for byte and its digest is
+ // taken over exactly these bytes.
+ const excerpt =
+ typeof flags["excerpt-file"] === "string"
+ ? readExcerptFile(flags["excerpt-file"] as string)
+ : undefined;
+ if (excerpt === null) {
+ process.stderr.write(`${EXCERPT_NOT_UTF8_MESSAGE}\n`);
+ return 2;
+ }
const res = distillSource(
vault,
- { sourcePath: source, claims },
- { agent: resolveBrainAgent(flags, config), now: new Date() },
+ { sourcePath: source, claims, ...(excerpt !== undefined ? { excerpt } : {}) },
+ {
+ agent: resolveBrainAgent(flags, config),
+ now: new Date(),
+ strictQuotes: flags["strict-quotes"] === true,
+ },
);
if (flags["json"]) {
okJson({
@@ -119,16 +206,16 @@ export async function cmdBrainDistill(argv: string[]): Promise<number> {
// "not recorded" rather than a sentinel that looks like a digest.
...(res.sourceHash !== undefined ? { source_hash: res.sourceHash } : {}),
trust: res.trust,
+ capture_scope: res.captureScope,
+ ...(res.quotes !== undefined ? { quotes: res.quotes } : {}),
});
return 0;
}
- ok(
- `distilled ${res.claimCount} claim(s) -> ${res.distillationPath}${res.created ? "" : " (updated)"}${untrustedNote(res.trust)}`,
- );
+ ok(successLine(res));
return 0;
} catch (err) {
- if (err instanceof DistillValidationError || err instanceof ResponseShapeError) {
- return fail(`distill: ${err.message}`);
+ if (OPERATOR_ERROR_CLASSES.some((cls) => err instanceof cls)) {
+ return fail(`distill: ${(err as Error).message}`);
}
return fail(`distill failed: ${(err as Error).message ?? err}`);
}
diff --git a/src/cli/command-manifest.ts b/src/cli/command-manifest.ts
index ef26e223..bee9b2f3 100644
--- a/src/cli/command-manifest.ts
+++ b/src/cli/command-manifest.ts
@@ -572,7 +572,15 @@ export const CLI_COMMAND_MANIFEST: CliRootManifest = Object.freeze({
),
command(
"distill",
- "Condense a source into agent-supplied atomic claims with block provenance",
+ "Condense a source into agent-supplied atomic claims with block provenance and checked quotes",
+ [
+ flag("vault", "string"),
+ flag("agent", "string"),
+ flag("claims", "string"),
+ flag("claims-file", "string"),
+ flag("strict-quotes", "boolean"),
+ flag("excerpt-file", "string"),
+ ],
),
command("session-summary", "Session-scoped structured digest: write, get, list"),
command(
diff --git a/src/core/brain/agent-source/query.ts b/src/core/brain/agent-source/query.ts
index 6b20b84d..519f932b 100644
--- a/src/core/brain/agent-source/query.ts
+++ b/src/core/brain/agent-source/query.ts
@@ -1,4 +1,5 @@
import { isOwnerVisible, normalizeAgentScope } from "../../graph/agent-scope.ts";
+import type { ArtifactRefView } from "../artifact-ref-view.ts";
import { collectAgentSourceContributions, summarizeAgentSources } from "./registry.ts";
import { summarizeAgentContributions } from "./summary.ts";
import type {
@@ -21,6 +22,15 @@ export interface AgentSourceQueryOptions {
* unscoped call is byte-identical to before.
*/
readonly ownerScope?: string;
+ /**
+ * A reference view each contribution must also pass, typically the
+ * caller's reach (`reachView`). It is asked over the contribution's id,
+ * the page a `note` contribution names and every string of an event body,
+ * which carries the written page's digest and size. Like the owner
+ * scope it runs before the roster is folded, so a withheld contribution
+ * is neither listed nor counted. Absent filters nothing.
+ */
+ readonly view?: ArtifactRefView;
}
export interface AgentSourceQueryFilters {
@@ -57,7 +67,9 @@ export function queryAgentSources(
// it withheld (context-integrity-gates, A5). An unscoped call filters
// nothing, so the roster is the same array it always was.
const visible = collectAgentSourceContributions(vault).filter(
- (contribution) => ownerScope === null || isOwnerVisible(contribution.owner ?? null, ownerScope),
+ (contribution) =>
+ (ownerScope === null || isOwnerVisible(contribution.owner ?? null, ownerScope)) &&
+ (opts.view === undefined || opts.view.row(...contributionRefs(contribution))),
);
const availableAgents = summarizeAgentSources(visible);
const availableIds = new Set(availableAgents.map((a) => a.id));
@@ -140,3 +152,23 @@ function matchesText(contribution: AgentSourceContribution, query: string): bool
];
return fields.some((field) => field.toLowerCase().includes(needle));
}
+
+/**
+ * The references a contribution answers for: its id (a preference or
+ * signal id resolves to that Brain page), the page a `note` contribution
+ * names, and every string of the event body it echoes.
+ */
+function contributionRefs(
+ contribution: AgentSourceContribution,
+): ReadonlyArray<string | undefined> {
+ const refs: Array<string | undefined> = [contribution.id, contribution.path];
+ const body = contribution.data["body"];
+ if (body !== null && typeof body === "object") {
+ for (const value of Object.values(body as Record<string, unknown>)) {
+ for (const item of Array.isArray(value) ? value : [value]) {
+ if (typeof item === "string" && item.length > 0) refs.push(item);
+ }
+ }
+ }
+ return refs;
+}
diff --git a/src/core/brain/artifact-ref-view.ts b/src/core/brain/artifact-ref-view.ts
index 94ed555d..8ec6dbe6 100644
--- a/src/core/brain/artifact-ref-view.ts
+++ b/src/core/brain/artifact-ref-view.ts
@@ -32,7 +32,7 @@
import { join } from "node:path";
import { existsSync } from "node:fs";
-import { vaultRelative } from "../path-safety.ts";
+import { pathIsInside, vaultRelative } from "../path-safety.ts";
import type { FrontmatterCache } from "../search/result-filters.ts";
import { BRAIN_SOURCES_REL, brainDirs } from "./paths.ts";
import { ANCHORED_WIKILINK_RE, stripWikilinkDecoration } from "./wikilink.ts";
@@ -202,7 +202,11 @@ export function artifactRefView(
const bare = unbracket(ref);
const rel =
bare.length === 0 ? null : bare.endsWith(MARKDOWN_EXT) ? bare : artifactPath(vault, bare);
- const verdict = rel === null ? true : pathVisible(rel, cache);
+ // A reference that resolves outside the vault names no page anyone
+ // could be shown, and is hidden before the rule reads anything: log
+ // bodies carry strings an earlier caller wrote.
+ const verdict =
+ rel === null ? true : pathIsInside(join(vault, rel), vault) && pathVisible(rel, cache);
verdicts.set(ref, verdict);
return verdict;
};
diff --git a/src/core/brain/design-note.ts b/src/core/brain/design-note.ts
index a2a4b265..33c712bf 100644
--- a/src/core/brain/design-note.ts
+++ b/src/core/brain/design-note.ts
@@ -206,6 +206,15 @@ export interface PlanDesignNoteOptions {
* it and leak in silence.
*/
readonly ownerScope: string | null;
+ /**
+ * May the caller read the page at this vault-relative path at its
+ * reach? ANDed into the link-candidate filter beside the owner scope, so
+ * a page withheld from the caller is not offered as a citation target
+ * nor counted in the manifest's total, exactly as an absent page. The
+ * MCP handler binds it to `readableAtContextReach(ctx)`; absent (the
+ * CLI) filters nothing beyond the owner scope.
+ */
+ readonly readable?: (rel: string) => boolean;
}
export interface CommitDesignNoteOptions {
@@ -378,9 +387,10 @@ export function planDesignNote(
const grounding = designNoteGrounding(vault, trimmed);
const slug = slugify(trimmed);
const targetPath = posix.join(DESIGN_NOTE_DIR_REL, noteBasename(slug, opts.now));
+ const ownerView = ownerScopeView(vault, opts.ownerScope);
const linkCandidates = buildLinkCandidateManifest(vault, {
query: trimmed,
- visible: ownerScopeView(vault, opts.ownerScope).visible,
+ visible: (rel) => ownerView.visible(rel) && (opts.readable?.(rel) ?? true),
});
return Object.freeze({
topic: trimmed,
diff --git a/src/core/brain/diarization.ts b/src/core/brain/diarization.ts
index 1360834e..3351d3b5 100644
--- a/src/core/brain/diarization.ts
+++ b/src/core/brain/diarization.ts
@@ -116,6 +116,15 @@ export interface DiarizationOptions {
* the one failure mode this option exists for.
*/
readonly ownerScope: string | null;
+ /**
+ * May the caller read the page at this vault-relative path at its
+ * reach? ANDed into the link-candidate filter beside the owner scope, so
+ * a page withheld from the caller is not offered as a citation target
+ * nor counted in the manifest's total, exactly as an absent page. The
+ * MCP handler binds it to `readableAtContextReach(ctx)`; absent (the
+ * CLI) filters nothing beyond the owner scope.
+ */
+ readonly readable?: (rel: string) => boolean;
}
function toPosixRel(vault: string, abs: string): string {
@@ -249,9 +258,10 @@ export function diarize(
// The subject and its aliases are the query: a profile cites the notes
// that are about its subject, and those are the ones an over-bound
// vault has to keep when the manifest truncates.
+ const ownerView = ownerScopeView(vault, opts.ownerScope);
const linkCandidates = buildLinkCandidateManifest(vault, {
query: [entity.name, ...entity.aliases].join(" "),
- visible: ownerScopeView(vault, opts.ownerScope).visible,
+ visible: (rel) => ownerView.visible(rel) && (opts.readable?.(rel) ?? true),
});
const llmStep: DiarizationLlmStep = buildNeedsLlmStep({
step: PROFILE_PROSE_STEP,
diff --git a/src/core/brain/distill/block-resolve.ts b/src/core/brain/distill/block-resolve.ts
new file mode 100644
index 00000000..17299659
--- /dev/null
+++ b/src/core/brain/distill/block-resolve.ts
@@ -0,0 +1,205 @@
+/**
+ * Block resolver: maps an Obsidian block id (the `^id` sigil) to the text of
+ * the block it marks inside a source. Nothing else in the codebase resolves a
+ * block id to its text; the distill quote check compares a quoted span with
+ * what this returns.
+ *
+ * Structural only, never over natural-language vocabulary:
+ *
+ * - Line endings are normalised (CRLF and CR to LF) before anything else, so a
+ * CRLF source resolves exactly as its LF twin.
+ * - Fenced code (``` or ~~~, three or more) is skipped: an id inside a fence is
+ * code, not a definition.
+ * - A line ending in whitespace plus `^id` defines a paragraph or list-item
+ * block: a list item resolves to that item (from its marker line), anything
+ * else to its paragraph (the run of non-blank lines up to the marker line).
+ * The marker itself is removed from the returned text. A line that looks
+ * like an item continues a paragraph unless it may start a list there (see
+ * {@link listItemLines}), so `1999) was a year` inside a paragraph is text.
+ * - The source is indexed once ({@link indexBlocks}) and every lookup after
+ * that is a map read; a block's text is built only when it is looked up.
+ * - A line holding only `^id` attaches to the block directly above it (a
+ * table, a blockquote, a list, a fenced block), skipping blank lines.
+ * - Two definitions of one id are ambiguous; the caller decides what that
+ * means, this module never picks one.
+ */
+
+/** Structural grammar of an Obsidian block id (the text after `#^`). */
+export const BLOCK_ID_RE = /^[A-Za-z0-9][A-Za-z0-9-]*$/;
+
+const BLOCK_ID_BODY = BLOCK_ID_RE.source.slice(1, -1);
+/** A trailing `^id` preceded by whitespace on a line with other content. */
+const TRAILING_MARKER_RE = new RegExp(`^(.*\\S)[ \\t]+\\^(${BLOCK_ID_BODY})[ \\t]*$`);
+/** A line that holds nothing but `^id`. */
+const STANDALONE_MARKER_RE = new RegExp(`^[ \\t]*\\^(${BLOCK_ID_BODY})[ \\t]*$`);
+/** Opening (or closing) fence: up to three spaces of indent, three or more backticks or tildes. */
+const FENCE_RE = /^ {0,3}(`{3,}|~{3,})(.*)$/;
+/** A list-item marker line: a bullet or an ordered-list number followed by whitespace. */
+export const LIST_ITEM_RE = /^[ \t]*(?:[-*+]|\d+[.)])[ \t]+/;
+/**
+ * An item that may interrupt a paragraph: a bullet, or an ordered item
+ * numbered 1 (the CommonMark rule; any other number continues the paragraph).
+ */
+const PARAGRAPH_INTERRUPTING_ITEM_RE = /^[ \t]*(?:[-*+]|0*1[.)])[ \t]+/;
+/** CRLF and lone CR, both folded to LF before anything else. */
+const LINE_ENDING_RE = /\r\n?/g;
+/** An ATX heading line; a heading is always a block of its own. */
+const HEADING_RE = /^ {0,3}#{1,6}(?:[ \t]|$)/;
+
+/** What a block id resolves to inside one source. */
+export type BlockResolution =
+ | { readonly kind: "found"; readonly text: string }
+ | { readonly kind: "not-found" }
+ | { readonly kind: "ambiguous" };
+
+/** One contiguous block of the source. */
+interface Segment {
+ /** For a fence, its inner lines; otherwise the segment's own lines. */
+ readonly text: ReadonlyArray<string>;
+ readonly fenced: boolean;
+}
+
+/**
+ * Split the source into segments: whole fenced blocks, headings, and runs of
+ * non-blank lines. Blank lines separate segments and belong to none.
+ */
+function segment(lines: ReadonlyArray<string>): Segment[] {
+ const out: Segment[] = [];
+ let i = 0;
+ while (i < lines.length) {
+ const line = lines[i]!;
+ const fence = FENCE_RE.exec(line);
+ if (fence !== null) {
+ const marker = fence[1]!;
+ let j = i + 1;
+ while (j < lines.length && !closesFence(lines[j]!, marker)) j++;
+ out.push({ text: lines.slice(i + 1, j), fenced: true });
+ i = j + 1;
+ continue;
+ }
+ if (line.trim().length === 0) {
+ i++;
+ continue;
+ }
+ if (HEADING_RE.test(line)) {
+ out.push({ text: [line], fenced: false });
+ i++;
+ continue;
+ }
+ let j = i + 1;
+ while (
+ j < lines.length &&
+ lines[j]!.trim().length > 0 &&
+ !FENCE_RE.test(lines[j]!) &&
+ !HEADING_RE.test(lines[j]!)
+ ) {
+ j++;
+ }
+ out.push({ text: lines.slice(i, j), fenced: false });
+ i = j;
+ }
+ return out;
+}
+
+function closesFence(line: string, opener: string): boolean {
+ const m = FENCE_RE.exec(line);
+ return (
+ m !== null &&
+ m[1]![0] === opener[0] &&
+ m[1]!.length >= opener.length &&
+ m[2]!.trim().length === 0
+ );
+}
+
+/**
+ * Which of `lines` start a list item. Blank lines end a run. A run whose
+ * first line is an item is a list, and every item line in it starts an
+ * item; inside a paragraph only a bullet or an item numbered 1 starts a
+ * list, after which the run is a list.
+ */
+export function listItemLines(lines: ReadonlyArray<string>): ReadonlyArray<boolean> {
+ let runStart = true;
+ let inList = false;
+ return lines.map((line) => {
+ if (line.trim().length === 0) {
+ runStart = true;
+ inList = false;
+ return false;
+ }
+ const item =
+ LIST_ITEM_RE.test(line) && (runStart || inList || PARAGRAPH_INTERRUPTING_ITEM_RE.test(line));
+ runStart = false;
+ if (item) inList = true;
+ return item;
+ });
+}
+
+/** Every block id of one source, mapped to its definitions, each built on demand. */
+export type BlockIndex = ReadonlyMap<string, ReadonlyArray<() => string>>;
+
+/**
+ * Index every block-id definition of `source` in one pass over its
+ * segments. A definition's text is built only when it is looked up, so a
+ * source with many ids costs one scan, not one per id.
+ */
+export function indexBlocks(source: string): BlockIndex {
+ const lines = source.replace(LINE_ENDING_RE, "\n").split("\n");
+ const segments = segment(lines);
+ const index = new Map<string, Array<() => string>>();
+ const define = (id: string, text: () => string): void => {
+ const list = index.get(id);
+ if (list === undefined) index.set(id, [text]);
+ else list.push(text);
+ };
+ segments.forEach((seg, segIndex) => {
+ if (seg.fenced) return;
+ seg.text.forEach((line, offset) => {
+ const standalone = STANDALONE_MARKER_RE.exec(line);
+ if (standalone !== null) {
+ // Lines of this run above the marker, else the whole segment above.
+ const above =
+ offset > 0
+ ? seg.text.slice(0, offset)
+ : segIndex > 0
+ ? segments[segIndex - 1]!.text
+ : null;
+ if (above !== null && above.length > 0) define(standalone[1]!, () => above.join("\n"));
+ return;
+ }
+ const trailing = TRAILING_MARKER_RE.exec(line);
+ if (trailing === null) return;
+ const content = trailing[1]!;
+ define(trailing[2]!, () => blockLines([...seg.text.slice(0, offset), content]).join("\n"));
+ });
+ });
+ return index;
+}
+
+/**
+ * The block a trailing marker on the LAST of `lines` closes: the list item it
+ * belongs to when the run is a list, else the whole paragraph.
+ */
+function blockLines(lines: ReadonlyArray<string>): ReadonlyArray<string> {
+ const items = listItemLines(lines);
+ for (let k = lines.length - 1; k >= 0; k--) {
+ if (items[k] === true) return lines.slice(k);
+ }
+ return lines;
+}
+
+/** Look `blockId` up in an index built by {@link indexBlocks}. */
+export function lookupBlock(index: BlockIndex, blockId: string): BlockResolution {
+ const found = index.get(blockId);
+ if (found === undefined || found.length === 0) return { kind: "not-found" };
+ if (found.length > 1) return { kind: "ambiguous" };
+ return { kind: "found", text: found[0]!() };
+}
+
+/**
+ * Resolve `blockId` (the id only, no `^`) to the text of the block it marks
+ * in `source`. Pure; never throws on any input. A caller resolving several
+ * ids in one source indexes it once with {@link indexBlocks} instead.
+ */
+export function resolveBlock(source: string, blockId: string): BlockResolution {
+ return lookupBlock(indexBlocks(source), blockId);
+}
diff --git a/src/core/brain/distill/claim.ts b/src/core/brain/distill/claim.ts
new file mode 100644
index 00000000..4b374ea9
--- /dev/null
+++ b/src/core/brain/distill/claim.ts
@@ -0,0 +1,13 @@
+/**
+ * The claim record of a distillation. A leaf module: the write path
+ * (`distill-source.ts`) and the pure quote check (`quote-check.ts`) both
+ * depend on it, and neither depends on the other for it.
+ */
+
+/** One atomic claim distilled from the source, with an optional block ref. */
+export interface DistillClaim {
+ /** The atomic claim text. */
+ readonly text: string;
+ /** Block id in the source the claim was drawn from (the `^abc` sigil, id only). */
+ readonly block?: string;
+}
diff --git a/src/core/brain/distill/distill-source.ts b/src/core/brain/distill/distill-source.ts
index eab5b9c3..3af279f5 100644
--- a/src/core/brain/distill/distill-source.ts
+++ b/src/core/brain/distill/distill-source.ts
@@ -9,10 +9,28 @@
*
* Provider-agnostic: the calling agent supplies the atomic claims and their
* block references; this core runs NO model. It validates the claims
- * structurally (non-empty text, well-formed block ids), stamps a content
- * sha256 the verifier can reproduce from the source file, and writes one
- * distillation page per source identity, rewritten in place on re-distill
- * (never duplicated). A byte-identical re-run is inert.
+ * structurally (non-empty text, well-formed block ids), checks every quoted
+ * span in a claim against the source ({@link checkClaimQuotes}), stamps a
+ * content sha256 the verifier can reproduce from the source file, and writes
+ * one distillation page per source identity, rewritten in place on
+ * re-distill (never duplicated). A byte-identical re-run is inert.
+ *
+ * QUOTES. A span in quotation marks is a claim that the source says exactly
+ * that. It is compared with the block the claim cites (or, without a block,
+ * the whole source) in the SAME bytes the content digest is computed over -
+ * one read feeds both. A span that does not verify loses its two marks on
+ * the page (its words stay) and is named in the result's `quotes` report, so
+ * the page never shows an unverified quote. `strictQuotes` refuses the whole
+ * write instead, with {@link QuoteCheckError}, before anything is written.
+ *
+ * CAPTURE SCOPE. Every result says how much of the source the vault holds
+ * (`captureScope`): a local file is `full-local` and adds nothing to the page;
+ * a source with no local bytes is `url-only` and the page says so. For such a
+ * source the caller may pass the verbatim `excerpt` it read: the page stores
+ * it in a fenced `## Excerpt` section with its digest, becomes
+ * `bounded-local`, and its quotes are checked against the excerpt. An excerpt
+ * beside a local source, an empty one or one past the byte ceiling is refused
+ * with {@link CaptureExcerptError} before anything is written.
*
* Language-agnostic: block-id validation is structural (the Obsidian `^id`
* grammar), never over natural-language vocabulary.
@@ -23,8 +41,9 @@
* and hashed whatever `join(vault, ...)` landed on, and wrote every page under
* `provenance: stated`, the top authority tier. It now asks the same question
* of the same classifier the other two claim-write paths ask
- * ({@link classifySourceOrigin}), so a source this vault does not own commits
- * in the quarantine lane the retrieval trust gate can actually see.
+ * ({@link readSourceOrigin}, the byte-keeping form of `classifySourceOrigin`),
+ * so a source this vault does not own commits in the quarantine lane the
+ * retrieval trust gate can actually see.
*/
import { existsSync, mkdirSync, readFileSync } from "node:fs";
@@ -34,7 +53,12 @@ import type { FrontmatterMap } from "../../types.ts";
import { atomicWriteFileSync } from "../../fs-atomic.ts";
import { canonicalNotePath } from "../../path-safety.ts";
import { formatFrontmatter, parseFrontmatter, slugify } from "../../vault.ts";
-import { classifySourceOrigin, normalizeSourceIdentity } from "../intake/source-trust.ts";
+import {
+ UNTRUSTED_ORIGIN,
+ isSourceHidden,
+ normalizeSourceIdentity,
+ readSourceOrigin,
+} from "../intake/source-trust.ts";
import { distillationPagePath } from "../paths.ts";
import {
DISTILL_CLAIMS_SHAPE,
@@ -53,20 +77,40 @@ import {
type Provenance,
} from "../provenance/provenance.ts";
import { isoSecond } from "../time.ts";
+import {
+ CAPTURE_SCOPE,
+ EXCERPT_HASH_KEY,
+ assertExcerptAdmissible,
+ captureScopeForTrust,
+ captureScopeFrontmatter,
+ excerptDigest,
+ renderExcerptSection,
+ type CaptureScope,
+} from "../provenance/capture-scope.ts";
+import { BLOCK_ID_RE } from "./block-resolve.ts";
+import type { DistillClaim } from "./claim.ts";
+import { checkClaimQuotes, type QuoteEvidence } from "./quote-check.ts";
+import {
+ QUOTES_UNQUOTED_KEY,
+ QUOTES_VERIFIED_KEY,
+ QUOTE_CHECK_OUTCOME,
+ QuoteCheckError,
+ type QuoteCheckReport,
+} from "./quote-verdict.ts";
/** Frontmatter `kind:` marker of a distillation page. */
export const BRAIN_DISTILLATION_KIND = "brain-distillation";
-/** Structural grammar of an Obsidian block id (the text after `#^`). */
-const BLOCK_ID_RE = /^[A-Za-z0-9][A-Za-z0-9-]*$/;
+export type { DistillClaim } from "./claim.ts";
-/** One atomic claim distilled from the source, with an optional block ref. */
-export interface DistillClaim {
- /** The atomic claim text. */
- readonly text: string;
- /** Block id in the source the claim was drawn from (the `^abc` sigil, id only). */
- readonly block?: string;
-}
+/**
+ * The most claims one call may carry. Every claim is checked against the
+ * source, so the cap bounds the work one request can ask for.
+ */
+export const DISTILL_CLAIMS_MAX = 1000;
+
+/** Line breaks a claim may not hold: a claim is one bullet line on the page. */
+const CLAIM_LINE_BREAK_RE = /[\r\n]/;
/**
* Normalize one agent-supplied claim record into a {@link DistillClaim}. Shared
@@ -98,11 +142,31 @@ export interface DistillSourceInput {
readonly sourcePath: string;
/** The atomic claims the agent distilled from the source (non-empty). */
readonly claims: readonly DistillClaim[];
+ /**
+ * The verbatim text the caller read from a source the vault holds no bytes
+ * of (`url-only`). Stored on the page and used as the quote evidence;
+ * refused for any other source.
+ */
+ readonly excerpt?: string;
}
export interface DistillSourceOptions {
readonly agent: string;
readonly now: Date;
+ /**
+ * Refuse the whole write with {@link QuoteCheckError} when any quoted span
+ * would be unquoted, instead of unquoting it. Unpaired marks never refuse.
+ */
+ readonly strictQuotes?: boolean;
+ /**
+ * May the caller read the vault file at this vault-relative path? Asked of
+ * the canonical source identity when the vault holds its bytes. A refusal
+ * makes the source answer exactly as an absent one: untrusted lane,
+ * `url-only` (an excerpt is admitted), every span `url-only`, and no digest
+ * in the result or on the page.
+ * Absent: the caller reads everything (the local CLI).
+ */
+ readonly readable?: (rel: string) => boolean;
}
export interface DistillSourceResult {
@@ -128,6 +192,56 @@ export interface DistillSourceResult {
* untrusted page from every scope.
*/
readonly trust: IntakeTrust;
+ /** How much of the source the page's evidence holds, as written. */
+ readonly captureScope: CaptureScope;
+ /**
+ * The quote check's account, present only when at least one claim contains
+ * a quoted span. Every finding is a span that was unquoted on the page.
+ */
+ readonly quotes?: QuoteCheckReport;
+}
+
+/** Strict UTF-8: a source that is not text verifies no quote. */
+const UTF8_STRICT = new TextDecoder("utf-8", { fatal: true });
+
+/** The evidence a quote is checked against: the bytes the digest covers. */
+function quoteEvidence(bytes: Uint8Array | undefined): QuoteEvidence {
+ if (bytes === undefined) return { kind: QUOTE_CHECK_OUTCOME.urlOnly };
+ try {
+ return { kind: "text", text: UTF8_STRICT.decode(bytes) };
+ } catch (cause) {
+ // A decode failure IS the verdict: these bytes are not text, so no span
+ // can be said to occur in them. It is reported per span, by name. The
+ // fatal decoder signals that with a TypeError; anything else is a fault.
+ if (!(cause instanceof TypeError)) throw cause;
+ return { kind: "not-text" };
+ }
+}
+
+/**
+ * The capture scope of this write and the evidence its quotes are checked
+ * against. An excerpt is admitted only for a `url-only` source, and then IS
+ * the evidence; otherwise the evidence is the bytes the digest covers.
+ */
+function captureOf(
+ trust: IntakeTrust,
+ bytes: Uint8Array | undefined,
+ excerpt: string | undefined,
+): { readonly scope: CaptureScope; readonly evidence: QuoteEvidence } {
+ const scope = captureScopeForTrust(trust);
+ if (excerpt === undefined) return { scope, evidence: quoteEvidence(bytes) };
+ assertExcerptAdmissible(scope, excerpt);
+ return { scope: CAPTURE_SCOPE.boundedLocal, evidence: { kind: "text", text: excerpt } };
+}
+
+/** The page's quote counters; nothing when no claim held a span. */
+function quotesFrontmatter(report: QuoteCheckReport | null): FrontmatterMap {
+ return report === null
+ ? {}
+ : {
+ [QUOTES_VERIFIED_KEY]: report.verified_in_block + report.verified_in_source,
+ [QUOTES_UNQUOTED_KEY]: report.unquoted,
+ };
}
/** A distillation failed structural validation; nothing was written. */
@@ -142,10 +256,18 @@ function validate(input: DistillSourceInput): void {
if (input.claims.length === 0) {
throw new DistillValidationError("distillation requires at least one claim");
}
+ if (input.claims.length > DISTILL_CLAIMS_MAX) {
+ throw new DistillValidationError(
+ `distillation accepts at most ${DISTILL_CLAIMS_MAX} claims per call, got ${input.claims.length}`,
+ );
+ }
input.claims.forEach((claim, i) => {
if (claim.text.trim().length === 0) {
throw new DistillValidationError(`claim ${i} has empty text`);
}
+ if (CLAIM_LINE_BREAK_RE.test(claim.text)) {
+ throw new DistillValidationError(`claim ${i} spans more than one line; a claim is one line`);
+ }
if (claim.block !== undefined && !BLOCK_ID_RE.test(claim.block)) {
throw new DistillValidationError(
`claim ${i} has a malformed block id ${JSON.stringify(claim.block)} - expected an Obsidian block id (alphanumerics and hyphens)`,
@@ -200,9 +322,28 @@ export function distillSource(
// `provenance.level` stays `stated` - that vocabulary bands how a conclusion
// was DERIVED, which is orthogonal to who was entitled to supply the
// material; the lane is carried by the marker below.
- const origin = classifySourceOrigin(vault, input.sourcePath);
+ //
+ // A source the caller may not read answers exactly as an absent one: the
+ // untrusted lane, no bytes, no digest. No span is checked against it, an
+ // excerpt is admitted for it, and nothing in the result or on the page
+ // tells the caller that the page exists or what it says. Asked first, so
+ // the source is never stat-ed or read for such a caller.
+ const origin = isSourceHidden(vault, input.sourcePath, opts.readable)
+ ? UNTRUSTED_ORIGIN
+ : readSourceOrigin(vault, input.sourcePath);
const sourceHash = origin.contentHash;
+ // The quote check runs on the bytes the digest above was computed over (or
+ // on the admitted excerpt), before any write, so a refusal of either kind
+ // leaves nothing behind.
+ const { excerpt } = input;
+ const capture = captureOf(origin.trust, origin.bytes, excerpt);
+ const checked = checkClaimQuotes({ claims: input.claims, evidence: capture.evidence });
+ const quotes = checked.report;
+ if (opts.strictQuotes === true && quotes !== null && quotes.unquoted > 0) {
+ throw new QuoteCheckError(quotes.findings);
+ }
+
const idHash = sourceIdentityHash([canonicalSource]);
const absPath = distillationPagePath(vault, `${slugify(canonicalSource)}-${idHash.slice(0, 12)}`);
const existed = existsSync(absPath);
@@ -213,9 +354,11 @@ export function distillSource(
const claimsSection = [
"## Claims",
"",
- ...input.claims.map((c) => renderClaim(c, canonicalSource)),
+ ...checked.claims.map((c) => renderClaim(c, canonicalSource)),
].join("\n");
- const body = [claimsSection, renderProvenanceSection(provenance)]
+ // The rendered section ends in a newline; the join supplies the separation.
+ const excerptSection = excerpt !== undefined ? renderExcerptSection(excerpt).trimEnd() : "";
+ const body = [claimsSection, excerptSection, renderProvenanceSection(provenance)]
.filter((section) => section.length > 0)
.join("\n\n");
@@ -239,9 +382,14 @@ export function distillSource(
// rather than ranking it beside the operator's own notes. A trusted
// source adds nothing, keeping its page byte-identical to before.
...untrustedSourceFrontmatter(origin.trust),
+ // How much of the source the page holds; nothing for a local file, so
+ // its page stays byte-identical to before.
+ ...captureScopeFrontmatter(capture.scope),
+ ...(excerpt !== undefined ? { [EXCERPT_HASH_KEY]: excerptDigest(excerpt) } : {}),
provenance: provenance.level,
agent: opts.agent,
claim_count: input.claims.length,
+ ...quotesFrontmatter(quotes),
created_at: createdAt,
updated_at: updatedAt,
tags: ["brain", "brain/distillation"],
@@ -264,6 +412,8 @@ export function distillSource(
claimCount: input.claims.length,
...(sourceHash !== undefined ? { sourceHash } : {}),
trust: origin.trust,
+ captureScope: capture.scope,
+ ...(quotes !== null ? { quotes } : {}),
};
}
diff --git a/src/core/brain/distill/quote-check.ts b/src/core/brain/distill/quote-check.ts
new file mode 100644
index 00000000..d68edd8f
--- /dev/null
+++ b/src/core/brain/distill/quote-check.ts
@@ -0,0 +1,190 @@
+/**
+ * The quote check of a distillation: every quoted span in a claim is compared
+ * with the evidence the claim rests on. Pure; no I/O.
+ *
+ * - A claim that cites a block is compared with that block only. A block that
+ * does not resolve is `block-not-found` (a duplicate id `block-ambiguous`),
+ * never a silent fall-back to the whole source: the citation is part of what
+ * the page asserts.
+ * - A claim without a block is compared with the whole evidence text and, when
+ * it matches, is `verified-in-source`, a weaker outcome than
+ * `verified-in-block` that never reads as the stronger one.
+ * - Evidence with no checkable text (`url-only`, or bytes that are not UTF-8)
+ * verifies nothing.
+ *
+ * A failed span loses exactly its two quotation marks; its words stay, and the
+ * failure is named in the report. Unpaired marks are counted and left alone.
+ * Matching runs over {@link normalizeForQuoteComparison} on both sides, so the
+ * claim and source bytes themselves are never normalised.
+ */
+
+import { indexBlocks, lookupBlock, type BlockIndex } from "./block-resolve.ts";
+import type { DistillClaim } from "./claim.ts";
+import {
+ findQuoteSpans,
+ normalizeForQuoteComparison,
+ quoteHaystack,
+ spanOccursIn,
+ unquoteSpans,
+ type QuoteHaystack,
+ type QuoteSpan,
+} from "./quote-spans.ts";
+import {
+ QUOTE_CHECK_OUTCOME,
+ QUOTE_FINDINGS_MAX,
+ QUOTE_SPAN_PREVIEW_MAX_CHARS,
+ VERIFIED_QUOTE_OUTCOMES,
+ type QuoteCheckOutcome,
+ type QuoteCheckReport,
+ type QuoteFinding,
+} from "./quote-verdict.ts";
+
+/** What the claims are checked against. */
+export type QuoteEvidence =
+ | { readonly kind: "text"; readonly text: string }
+ | { readonly kind: "not-text" }
+ | { readonly kind: typeof QUOTE_CHECK_OUTCOME.urlOnly };
+
+export interface QuoteCheckInput {
+ readonly claims: ReadonlyArray<DistillClaim>;
+ readonly evidence: QuoteEvidence;
+}
+
+export interface QuoteCheckResult {
+ /** The claims as they may be written: every failed span unquoted. */
+ readonly claims: ReadonlyArray<DistillClaim>;
+ /** `null` when no claim contains a quoted span. */
+ readonly report: QuoteCheckReport | null;
+}
+
+/** The outcome every span gets when the evidence holds no checkable text. */
+const TEXTLESS_OUTCOME: Readonly<
+ Record<Exclude<QuoteEvidence["kind"], "text">, QuoteCheckOutcome>
+> = Object.freeze({
+ "not-text": QUOTE_CHECK_OUTCOME.sourceNotText,
+ [QUOTE_CHECK_OUTCOME.urlOnly]: QUOTE_CHECK_OUTCOME.urlOnly,
+});
+
+/**
+ * The comparison target of one claim: normalised text to search, or an
+ * outcome that settles every span of the claim without a search.
+ */
+type Target =
+ | {
+ readonly kind: "haystack";
+ readonly haystack: QuoteHaystack;
+ readonly verified: QuoteCheckOutcome;
+ readonly failed: QuoteCheckOutcome;
+ }
+ | { readonly kind: "settled"; readonly outcome: QuoteCheckOutcome };
+
+/**
+ * Builds each claim's target. The whole source is normalised at most once,
+ * its blocks are indexed at most once, and each cited block is resolved and
+ * normalised at most once however many claims cite it.
+ */
+function targetResolver(evidence: QuoteEvidence): (claim: DistillClaim) => Target {
+ if (evidence.kind !== "text") {
+ const settled: Target = { kind: "settled", outcome: TEXTLESS_OUTCOME[evidence.kind] };
+ return () => settled;
+ }
+ let wholeSource: Target | undefined;
+ let blocks: BlockIndex | undefined;
+ const byBlock = new Map<string, Target>();
+ const blockTarget = (blockId: string): Target => {
+ blocks ??= indexBlocks(evidence.text);
+ const block = lookupBlock(blocks, blockId);
+ switch (block.kind) {
+ case "found":
+ return {
+ kind: "haystack",
+ haystack: quoteHaystack(normalizeForQuoteComparison(block.text)),
+ verified: QUOTE_CHECK_OUTCOME.verifiedInBlock,
+ failed: QUOTE_CHECK_OUTCOME.notInBlock,
+ };
+ case "not-found":
+ return { kind: "settled", outcome: QUOTE_CHECK_OUTCOME.blockNotFound };
+ case "ambiguous":
+ return { kind: "settled", outcome: QUOTE_CHECK_OUTCOME.blockAmbiguous };
+ }
+ };
+ return (claim) => {
+ if (claim.block === undefined) {
+ wholeSource ??= {
+ kind: "haystack",
+ haystack: quoteHaystack(normalizeForQuoteComparison(evidence.text)),
+ verified: QUOTE_CHECK_OUTCOME.verifiedInSource,
+ failed: QUOTE_CHECK_OUTCOME.notInSource,
+ };
+ return wholeSource;
+ }
+ let target = byBlock.get(claim.block);
+ if (target === undefined) {
+ target = blockTarget(claim.block);
+ byBlock.set(claim.block, target);
+ }
+ return target;
+ };
+}
+
+function spanOutcome(span: QuoteSpan, target: Target): QuoteCheckOutcome {
+ if (target.kind === "settled") return target.outcome;
+ return spanOccursIn(span.inner, target.haystack) ? target.verified : target.failed;
+}
+
+/** The span's own text, cut at the preview cap without splitting a code point. */
+function preview(inner: string): string {
+ const codePoints = Array.from(inner);
+ return codePoints.length <= QUOTE_SPAN_PREVIEW_MAX_CHARS
+ ? inner
+ : codePoints.slice(0, QUOTE_SPAN_PREVIEW_MAX_CHARS).join("");
+}
+
+/**
+ * Check every quoted span of every claim against the evidence. Returns the
+ * claims as they may be written (failed spans unquoted) and the report, or a
+ * `null` report when no claim contains a span.
+ */
+export function checkClaimQuotes(input: QuoteCheckInput): QuoteCheckResult {
+ const targetFor = targetResolver(input.evidence);
+ const findings: QuoteFinding[] = [];
+ let checked = 0;
+ let verifiedInBlock = 0;
+ let verifiedInSource = 0;
+ let unpaired = 0;
+
+ const claims = input.claims.map((claim, index) => {
+ const scan = findQuoteSpans(claim.text);
+ unpaired += scan.unpaired;
+ if (scan.spans.length === 0) return claim;
+ const target = targetFor(claim);
+ const failed: QuoteSpan[] = [];
+ for (const span of scan.spans) {
+ checked++;
+ const outcome = spanOutcome(span, target);
+ if (!VERIFIED_QUOTE_OUTCOMES.has(outcome)) {
+ failed.push(span);
+ findings.push({ claim: index, outcome, span: preview(span.inner) });
+ } else if (outcome === QUOTE_CHECK_OUTCOME.verifiedInBlock) verifiedInBlock++;
+ else verifiedInSource++;
+ }
+ return failed.length === 0 ? claim : { ...claim, text: unquoteSpans(claim.text, failed) };
+ });
+
+ if (checked === 0) return { claims: input.claims, report: null };
+ const returned = findings.slice(0, QUOTE_FINDINGS_MAX);
+ return {
+ claims,
+ report: {
+ checked,
+ verified_in_block: verifiedInBlock,
+ verified_in_source: verifiedInSource,
+ unquoted: findings.length,
+ unpaired,
+ findings: returned,
+ total: findings.length,
+ returned: returned.length,
+ truncated: returned.length < findings.length,
+ },
+ };
+}
diff --git a/src/core/brain/distill/quote-spans.ts b/src/core/brain/distill/quote-spans.ts
new file mode 100644
index 00000000..0522ed05
--- /dev/null
+++ b/src/core/brain/distill/quote-spans.ts
@@ -0,0 +1,509 @@
+/**
+ * Quoted spans in claim text, and the one normalisation used to compare them.
+ *
+ * A distillation claim that wraps words in quotation marks asserts that the
+ * source says exactly those words. Checking that needs two things this module
+ * owns: finding the spans, and a comparison that forgives presentation
+ * (Markdown, Unicode composition, quote-mark width, whitespace) while never
+ * forgiving wording, case or punctuation.
+ *
+ * DELIMITERS. A quotation mark is any code point with the Unicode binary
+ * property `Quotation_Mark`. No language list: the property already covers
+ * the ASCII marks, the curly marks, low-9 marks, guillemets and the CJK corner
+ * brackets. `QUOTE_VARIANT_RE` in `entities/canonical.ts` is a different
+ * population (identity-key folding, pinned over the whole code-point space)
+ * and is deliberately not reused for detection.
+ *
+ * DIRECTIONAL MARKS. A quotation mark whose general category is opening
+ * (`Ps`: low-9 marks, corner brackets) or closing (`Pe`) punctuation says
+ * which it is by itself: it opens before a non-space, closes after one, and
+ * is never an apostrophe, because CJK running text glues corner brackets to
+ * letters on both sides.
+ *
+ * PAIRING BY POSITION, NOT BY GLYPH. Which of the other marks opens and which
+ * closes differs between writing traditions (`»…«` and `«…»`, `„…“` and
+ * `“…”`), so such a mark is classified by what stands beside it: an opener
+ * follows the start, whitespace, punctuation (opening, dash or other, but not
+ * a mark that just closed) or another opener, and precedes a non-space; a
+ * closer precedes the end, whitespace or punctuation, and follows a
+ * non-space. One no-break, narrow no-break or thin space between a mark and
+ * its text is transparent (spaced guillemets); an ordinary space is not. A
+ * non-directional mark with a letter on both sides is an apostrophe in any
+ * script and is never a delimiter.
+ *
+ * PAIRING WITHIN ONE WIDTH. A closer pairs only with an opener of its own
+ * width class (double or single, by the shared quote fold; a directional mark
+ * pairs with either), so a trailing possessive apostrophe never ends a double
+ * quote: a single-width closer whose innermost opener is of the other class is
+ * not a delimiter, and a double-width closer skips (and counts as unpaired)
+ * any single-width openers left open inside it.
+ *
+ * WHAT IS REPORTED. Every closed pair is collected and only the outermost
+ * among the closed pairs are spans, so an opener that never closes (a leading
+ * apostrophe as in `'90s`) encloses nothing and hides no later quote; an
+ * inner quote is part of its outer span's text. A pair whose text holds fewer
+ * than two letters or digits (`rock 'n' roll`, an empty pair) is not a span.
+ *
+ * Unpaired marks are not spans: they are counted and left alone.
+ */
+
+import {
+ foldQuoteVariantsByClass,
+ QUOTE_VARIANT_FOLD_TARGET,
+ QUOTE_VARIANT_FOLD_TARGET_DOUBLE,
+} from "../entities/canonical.ts";
+import { listItemLines, LIST_ITEM_RE } from "./block-resolve.ts";
+
+/** A quotation mark, by Unicode property rather than by a list. */
+export const QUOTATION_MARK_RE = /\p{Quotation_Mark}/u;
+
+/** One quoted span: offsets of its two marks in the scanned text. */
+export interface QuoteSpan {
+ /** UTF-16 offset of the opening mark. */
+ readonly open: number;
+ /** UTF-16 offset of the closing mark. */
+ readonly close: number;
+ /** The text between the two marks, verbatim. */
+ readonly inner: string;
+}
+
+/** The spans found in one text, and the marks that paired with nothing. */
+export interface QuoteSpanScan {
+ readonly spans: ReadonlyArray<QuoteSpan>;
+ readonly unpaired: number;
+}
+
+const LETTER_RE = /\p{L}/u;
+const LETTER_OR_DIGIT_RE = /[\p{L}\p{N}]/u;
+const WHITESPACE_RE = /\s/u;
+/** What may stand before an opener: opening, dash or other punctuation. */
+const OPENING_CONTEXT_RE = /[\p{Ps}\p{Pi}\p{Pd}\p{Po}]/u;
+const PUNCTUATION_RE = /\p{P}/u;
+const DIRECTIONAL_OPEN_RE = /\p{Ps}/u;
+const DIRECTIONAL_CLOSE_RE = /\p{Pe}/u;
+/** No-break, narrow no-break and thin space: transparent once beside a mark. */
+const INNER_QUOTE_SPACE_RE = /[   ]/u;
+/** A span needs at least this many letters or digits between its marks. */
+const SPAN_MIN_LETTERS_OR_DIGITS = 2;
+/** The width class of a directional mark: it pairs with either width. */
+const ANY_WIDTH = "any";
+
+/** One code point and its UTF-16 offset. */
+interface CodePointAt {
+ readonly ch: string;
+ readonly at: number;
+}
+
+function codePoints(text: string): ReadonlyArray<CodePointAt> {
+ const out: CodePointAt[] = [];
+ let at = 0;
+ for (const ch of text) {
+ out.push({ ch, at });
+ at += ch.length;
+ }
+ return out;
+}
+
+const isLetter = (ch: string | undefined): boolean => ch !== undefined && LETTER_RE.test(ch);
+const isSpace = (ch: string | undefined): boolean => ch !== undefined && WHITESPACE_RE.test(ch);
+
+/** The width class a mark pairs within: the shared fold, or any for a directional mark. */
+function widthClass(ch: string): string {
+ return DIRECTIONAL_OPEN_RE.test(ch) || DIRECTIONAL_CLOSE_RE.test(ch)
+ ? ANY_WIDTH
+ : foldQuoteVariantsByClass(ch);
+}
+
+const sameWidth = (a: string, b: string): boolean => a === b || a === ANY_WIDTH || b === ANY_WIDTH;
+
+/** How a mark may act, decided by its neighbours; `null` for an apostrophe. */
+interface MarkRole {
+ readonly opens: boolean;
+ readonly closes: boolean;
+}
+
+/** The neighbour on one side, looking through one typographic inner space. */
+function neighbour(
+ points: ReadonlyArray<CodePointAt>,
+ i: number,
+ step: 1 | -1,
+): string | undefined {
+ const near = points[i + step]?.ch;
+ return near !== undefined && INNER_QUOTE_SPACE_RE.test(near) ? points[i + 2 * step]?.ch : near;
+}
+
+function markRole(
+ points: ReadonlyArray<CodePointAt>,
+ i: number,
+ prevOpened: boolean,
+ prevClosed: boolean,
+): MarkRole | null {
+ const ch = points[i]!.ch;
+ const prev = points[i - 1]?.ch;
+ const next = points[i + 1]?.ch;
+ const textAfter = neighbour(points, i, 1);
+ const textBefore = neighbour(points, i, -1);
+ if (DIRECTIONAL_OPEN_RE.test(ch)) {
+ return { opens: textAfter !== undefined && !isSpace(textAfter), closes: false };
+ }
+ if (DIRECTIONAL_CLOSE_RE.test(ch)) {
+ return { opens: false, closes: textBefore !== undefined && !isSpace(textBefore) };
+ }
+ if (isLetter(prev) && isLetter(next)) return null;
+ const opensAfter =
+ prev === undefined ||
+ isSpace(prev) ||
+ prevOpened ||
+ (!prevClosed && OPENING_CONTEXT_RE.test(prev));
+ const opens = opensAfter && textAfter !== undefined && !isSpace(textAfter);
+ const closes =
+ (next === undefined || isSpace(next) || PUNCTUATION_RE.test(next)) &&
+ textBefore !== undefined &&
+ !isSpace(textBefore);
+ return { opens, closes };
+}
+
+/** An opener still waiting for its closer. */
+interface OpenMark {
+ readonly at: number;
+ readonly width: string;
+}
+
+/** Find the outermost quoted spans of `text`. See the module docblock. */
+export function findQuoteSpans(text: string): QuoteSpanScan {
+ const points = codePoints(text);
+ const pairs: QuoteSpan[] = [];
+ const stack: OpenMark[] = [];
+ // Open marks per width class, so a closer with no opener of its class is
+ // answered without walking the stack.
+ const openByWidth = new Map<string, number>();
+ const bump = (width: string, by: number): void => {
+ openByWidth.set(width, (openByWidth.get(width) ?? 0) + by);
+ };
+ let unpaired = 0;
+ let prevOpened = false;
+ let prevClosed = false;
+
+ /** The stack index of the opener `width` closes, or -1. */
+ const openerFor = (width: string): number => {
+ const top = stack.length - 1;
+ if (top < 0) return -1;
+ // A single-width closer closes only the innermost opener: anything else
+ // there makes it an apostrophe (`"the students' work"`).
+ if (width === QUOTE_VARIANT_FOLD_TARGET) return sameWidth(stack[top]!.width, width) ? top : -1;
+ if (
+ width !== ANY_WIDTH &&
+ (openByWidth.get(width) ?? 0) + (openByWidth.get(ANY_WIDTH) ?? 0) === 0
+ ) {
+ return -1;
+ }
+ for (let k = top; k >= 0; k--) if (sameWidth(stack[k]!.width, width)) return k;
+ return -1;
+ };
+
+ for (let i = 0; i < points.length; i++) {
+ const point = points[i]!;
+ if (!QUOTATION_MARK_RE.test(point.ch)) {
+ prevOpened = false;
+ prevClosed = false;
+ continue;
+ }
+ const role = markRole(points, i, prevOpened, prevClosed);
+ prevOpened = false;
+ prevClosed = false;
+ if (role === null) continue;
+ const width = widthClass(point.ch);
+ if (role.closes && (stack.length > 0 || !role.opens)) {
+ const k = openerFor(width);
+ if (k !== -1) {
+ // Openers left open inside this pair (a leading apostrophe) are
+ // abandoned and counted once, here.
+ for (let j = stack.length - 1; j > k; j--) bump(stack[j]!.width, -1);
+ unpaired += stack.length - 1 - k;
+ const open = stack[k]!;
+ stack.length = k;
+ bump(open.width, -1);
+ const inner = text.slice(open.at + codePointLength(text, open.at), point.at);
+ pairs.push({ open: open.at, close: point.at, inner });
+ prevClosed = true;
+ continue;
+ }
+ if (!role.opens) {
+ unpaired += 1;
+ continue;
+ }
+ }
+ if (role.opens) {
+ stack.push({ at: point.at, width });
+ bump(width, 1);
+ prevOpened = true;
+ continue;
+ }
+ unpaired += 1;
+ }
+ unpaired += stack.length;
+
+ // Outermost among the CLOSED pairs. Pairs never cross (a stack made them),
+ // so in opening order a pair is enclosed exactly when an earlier one closes
+ // after it.
+ const spans: QuoteSpan[] = [];
+ let reach = -1;
+ for (const pair of pairs.toSorted((a, b) => a.open - b.open)) {
+ if (pair.close < reach) continue;
+ reach = pair.close;
+ if (!hasLettersOrDigits(pair.inner, SPAN_MIN_LETTERS_OR_DIGITS)) {
+ unpaired += 2;
+ continue;
+ }
+ spans.push(pair);
+ }
+ return { spans, unpaired };
+}
+
+/** Does `text` hold at least `min` letters or digits? Stops counting at `min`. */
+function hasLettersOrDigits(text: string, min: number): boolean {
+ let count = 0;
+ for (const ch of text) {
+ if (LETTER_OR_DIGIT_RE.test(ch) && ++count >= min) return true;
+ }
+ return false;
+}
+
+/** Length in UTF-16 units of the code point at `at`. */
+function codePointLength(text: string, at: number): number {
+ return String.fromCodePoint(text.codePointAt(at) ?? 0).length;
+}
+
+/** Remove exactly the two marks of each given span; every other byte stays. */
+export function unquoteSpans(text: string, spans: ReadonlyArray<QuoteSpan>): string {
+ const cuts = spans.flatMap((span) => [span.open, span.close]).toSorted((a, b) => b - a);
+ let out = text;
+ for (const at of cuts) out = out.slice(0, at) + out.slice(at + codePointLength(out, at));
+ return out;
+}
+
+// Every pattern below is linear on any input: a character class that cannot
+// hold the bracket that starts a match means a failed attempt never overlaps
+// the next one, and the line-anchored ones run once per line.
+
+/** `[[target|alias]]`, reduced to the alias. */
+const WIKILINK_ALIASED_RE = /\[\[[^[\]|]*\|([^[\]]*)\]\]/g;
+/** `[[target]]`, reduced to the target. */
+const WIKILINK_RE = /\[\[([^[\]]*)\]\]/g;
+/** `[text](url)`, reduced to the text. */
+const MARKDOWN_LINK_RE = /\[([^[\]]*)\]\([^()]*\)/g;
+/** Blockquote markers at the start of a line. */
+const BLOCKQUOTE_PREFIX_RE = /^[ \t]*(?:>[ \t]?)+/;
+/** An Obsidian block id: what follows the caret. */
+const BLOCK_ID_TAIL_RE = /^\^[A-Za-z0-9][A-Za-z0-9-]*$/;
+const WHITESPACE_RUN_RE = /\s+/gu;
+/** Bracket-type quotation marks (low-9 marks, corner brackets) for the comparison fold. */
+const BRACKET_PUNCTUATION_RE = /[\p{Ps}\p{Pe}]/gu;
+/** The emphasis, strike and code delimiter characters. */
+const INLINE_DELIMITER_CHARS: ReadonlySet<string> = new Set(["*", "_", "~", "`"]);
+const STRIKE_DELIMITER = "~~";
+const LINE_HORIZONTAL_SPACE = new Set([" ", "\t", "\r"]);
+
+/**
+ * `line` without an Obsidian block id ending it or standing alone on it. A
+ * caret glued to a word is text. Scans the line once, with no regex over the
+ * whitespace before the id.
+ */
+function stripTrailingBlockId(line: string): string {
+ let end = line.length;
+ while (end > 0 && LINE_HORIZONTAL_SPACE.has(line[end - 1]!)) end--;
+ const caret = line.lastIndexOf("^", end - 1);
+ if (caret === -1 || !BLOCK_ID_TAIL_RE.test(line.slice(caret, end))) return line;
+ let start = caret;
+ while (start > 0 && LINE_HORIZONTAL_SPACE.has(line[start - 1]!)) start--;
+ return start === caret && caret > 0 ? line : line.slice(0, start);
+}
+
+/**
+ * Block markers removed line by line: blockquote prefixes, list prefixes on
+ * the lines that start an item (the rule `block-resolve.ts` resolves blocks
+ * by), and trailing block ids.
+ */
+function stripLineMarkers(text: string): string {
+ const lines = text.split("\n").map((line) => line.replace(BLOCKQUOTE_PREFIX_RE, ""));
+ const items = listItemLines(lines);
+ return lines
+ .map((line, k) => stripTrailingBlockId(items[k] ? line.replace(LIST_ITEM_RE, "") : line))
+ .join("\n");
+}
+
+/**
+ * Inline emphasis, strike and code delimiters reduced to their content, in
+ * one linear pass. Only a PAIR is removed: a run of `*`, `_` or backticks
+ * (or `~~`) that can open (followed by a non-space) and a later run of the
+ * same characters that can close (preceded by a non-space). An underscore
+ * between two letters or digits never delimits, so `snake_case` and `2*3`
+ * keep their bytes.
+ */
+function stripInlineDelimiters(text: string): string {
+ const drop: Array<readonly [number, number]> = [];
+ const pending = new Map<string, number>();
+ let i = 0;
+ while (i < text.length) {
+ const ch = text[i]!;
+ if (!INLINE_DELIMITER_CHARS.has(ch)) {
+ i++;
+ continue;
+ }
+ let end = i + 1;
+ while (end < text.length && text[end] === ch) end++;
+ const run = text.slice(i, end);
+ if (ch === "~" && run !== STRIKE_DELIMITER) {
+ i = end;
+ continue;
+ }
+ const before = text[i - 1];
+ const after = text[end];
+ const intraword = ch === "_";
+ const canClose =
+ before !== undefined &&
+ !isSpace(before) &&
+ !(intraword && after !== undefined && LETTER_OR_DIGIT_RE.test(after));
+ const canOpen =
+ after !== undefined &&
+ !isSpace(after) &&
+ !(intraword && before !== undefined && LETTER_OR_DIGIT_RE.test(before));
+ const open = pending.get(run);
+ if (canClose && open !== undefined) {
+ drop.push([open, open + run.length], [i, end]);
+ pending.delete(run);
+ } else if (canOpen) {
+ pending.set(run, i);
+ }
+ i = end;
+ }
+ if (drop.length === 0) return text;
+ drop.sort((a, b) => a[0] - b[0]);
+ let out = "";
+ let from = 0;
+ for (const [start, stop] of drop) {
+ out += text.slice(from, start);
+ from = stop;
+ }
+ return out + text.slice(from);
+}
+
+/** Options of {@link normalizeForQuoteComparison}. */
+export interface QuoteNormalizeOptions {
+ /**
+ * Remove line-leading blockquote and list markers (default `true`). A span
+ * passes `false`: it is one run of the claim's own words, so a leading
+ * `3. ` or `> ` in it is wording to match, not a block marker.
+ */
+ readonly lineMarkers?: boolean;
+}
+
+/**
+ * The comparison form of a text, applied identically to a span and to the
+ * text it is checked against: NFC; block markers (on the evidence side only,
+ * see {@link QuoteNormalizeOptions.lineMarkers}), trailing block ids and
+ * paired inline Markdown reduced to display text; quote marks folded to their
+ * ASCII width (bracket-type quotation marks to `"`); whitespace collapsed.
+ * Case, punctuation and wording are untouched. Comparison only: no page byte
+ * is ever rewritten through this function. Linear in the length of `text`.
+ */
+export function normalizeForQuoteComparison(
+ text: string,
+ opts: QuoteNormalizeOptions = {},
+): string {
+ const nfc = text.normalize("NFC");
+ const lined =
+ opts.lineMarkers === false
+ ? nfc
+ .split("\n")
+ .map((line) => stripTrailingBlockId(line))
+ .join("\n")
+ : stripLineMarkers(nfc);
+ const reduced = stripInlineDelimiters(
+ lined
+ .replace(WIKILINK_ALIASED_RE, "$1")
+ .replace(WIKILINK_RE, "$1")
+ .replace(MARKDOWN_LINK_RE, "$1"),
+ );
+ return foldQuoteVariantsByClass(reduced)
+ .replace(BRACKET_PUNCTUATION_RE, (mark) =>
+ QUOTATION_MARK_RE.test(mark) ? QUOTE_VARIANT_FOLD_TARGET_DOUBLE : mark,
+ )
+ .replace(WHITESPACE_RUN_RE, " ")
+ .trim();
+}
+
+/** U+2026 or a run of three or more full stops. */
+const ELLIPSIS_RE = /…|\.{3,}/u;
+/** A span that opens with an ellipsis: its first fragment may start inside a word. */
+const LEADING_ELLIPSIS_RE = /^\s*(?:…|\.{3,})/u;
+/** A span that closes with an ellipsis: its last fragment may end inside a word. */
+const TRAILING_ELLIPSIS_RE = /(?:…|\.{3,})\s*$/u;
+
+/** The fragments an ellipsis separates, trimmed, empty ones dropped. */
+export function splitEllipsisFragments(inner: string): ReadonlyArray<string> {
+ return inner
+ .split(ELLIPSIS_RE)
+ .map((fragment) => fragment.trim())
+ .filter((fragment) => fragment.length > 0);
+}
+
+/** Word segmentation by the Unicode rules (dictionary-based for scripts written without spaces). */
+const WORD_SEGMENTER = new Intl.Segmenter(undefined, { granularity: "word" });
+
+/**
+ * An already-normalised text a span is searched in, with its word boundaries.
+ * The segmentation is built on the first boundary question and reused for
+ * every later one, so a haystack shared by many claims is segmented once.
+ */
+export interface QuoteHaystack {
+ readonly normalized: string;
+ /** Does a word boundary fall at `offset` (the two ends of the text always do)? */
+ isWordBoundary(offset: number): boolean;
+}
+
+/** Wrap a normalised text (from {@link normalizeForQuoteComparison}) as a haystack. */
+export function quoteHaystack(normalized: string): QuoteHaystack {
+ let segments: Intl.Segments | undefined;
+ return {
+ normalized,
+ isWordBoundary(offset: number): boolean {
+ if (offset <= 0 || offset >= normalized.length) return true;
+ segments ??= WORD_SEGMENTER.segment(normalized);
+ return segments.containing(offset)?.index === offset;
+ },
+ };
+}
+
+/**
+ * Does the quoted `inner` occur in an already-normalised haystack? Every
+ * ellipsis fragment must appear, in order, without overlapping the previous
+ * one. The outer ends of the span must fall on word boundaries of the
+ * haystack, so `"safe"` does not verify inside `unsafe`; an edge next to an
+ * ellipsis is free, since an ellipsis may cut a word on purpose. A span that
+ * is nothing but an ellipsis quotes nothing and fails.
+ */
+export function spanOccursIn(inner: string, haystack: string | QuoteHaystack): boolean {
+ const target = typeof haystack === "string" ? quoteHaystack(haystack) : haystack;
+ const text = target.normalized;
+ const fragments = splitEllipsisFragments(inner)
+ .map((fragment) => normalizeForQuoteComparison(fragment, { lineMarkers: false }))
+ .filter((fragment) => fragment.length > 0);
+ if (fragments.length === 0) return false;
+ const openStart = LEADING_ELLIPSIS_RE.test(inner);
+ const openEnd = TRAILING_ELLIPSIS_RE.test(inner);
+ const last = fragments.length - 1;
+ let from = 0;
+ for (const [k, fragment] of fragments.entries()) {
+ const startBound = k === 0 && !openStart;
+ const endBound = k === last && !openEnd;
+ const fits = (at: number): boolean =>
+ (!startBound || target.isWordBoundary(at)) &&
+ (!endBound || target.isWordBoundary(at + fragment.length));
+ let at = text.indexOf(fragment, from);
+ while (at !== -1 && !fits(at)) at = text.indexOf(fragment, at + 1);
+ if (at === -1) return false;
+ from = at + fragment.length;
+ }
+ return true;
+}
diff --git a/src/core/brain/distill/quote-verdict.ts b/src/core/brain/distill/quote-verdict.ts
new file mode 100644
index 00000000..9325d7e0
--- /dev/null
+++ b/src/core/brain/distill/quote-verdict.ts
@@ -0,0 +1,120 @@
+/**
+ * What the quote check decided about each quoted span of a distilled claim.
+ *
+ * A claim that wraps words in quotation marks asserts the source says exactly
+ * those words. Each such span gets one outcome from {@link QUOTE_CHECK_OUTCOME}:
+ * verified against the block the claim cites, verified against the whole
+ * source (the weaker guarantee, named apart so it never reads as the
+ * stronger), or one of the named reasons it could not be verified. A span
+ * whose source has no local bytes is `url-only`, the same token the
+ * capture-scope vocabulary uses for that fact (`CAPTURE_SCOPE.urlOnly`), so
+ * "no local bytes" has one spelling across both vocabularies. It is written
+ * as a literal because the verdict census reads only literal-valued
+ * vocabularies; the equality is pinned by the test suite.
+ *
+ * This module holds the vocabulary, the report shape and the named refusal.
+ * The check itself lives beside it in `quote-check.ts`.
+ */
+
+import { PAGE_LINT_MAX_FINDINGS } from "../page-lint.ts";
+
+/** The closed vocabulary of per-span outcomes. */
+export const QUOTE_CHECK_OUTCOME = Object.freeze({
+ /** The span occurs in the block the claim cites. */
+ verifiedInBlock: "verified-in-block",
+ /** The claim cites no block, and the span occurs in the source. */
+ verifiedInSource: "verified-in-source",
+ /** The cited block resolved, and the span is not in it. */
+ notInBlock: "not-in-block",
+ /** The claim cites no block, and the span is not in the source. */
+ notInSource: "not-in-source",
+ /** The cited block id is not defined in the source. */
+ blockNotFound: "block-not-found",
+ /** The cited block id is defined more than once in the source. */
+ blockAmbiguous: "block-ambiguous",
+ /** The source bytes are not valid UTF-8 text. */
+ sourceNotText: "source-not-text",
+ /** The source has no local bytes to check against. */
+ urlOnly: "url-only",
+} as const);
+
+/** Closed union over {@link QUOTE_CHECK_OUTCOME}. */
+export type QuoteCheckOutcome = (typeof QUOTE_CHECK_OUTCOME)[keyof typeof QUOTE_CHECK_OUTCOME];
+
+/** Membership list, verified outcomes first. */
+export const QUOTE_CHECK_OUTCOMES: ReadonlyArray<QuoteCheckOutcome> = Object.freeze(
+ Object.values(QUOTE_CHECK_OUTCOME),
+);
+
+/** Narrow an outcome read back off a payload or a fixture. */
+export function isQuoteCheckOutcome(value: unknown): value is QuoteCheckOutcome {
+ return (
+ typeof value === "string" && (QUOTE_CHECK_OUTCOMES as ReadonlyArray<string>).includes(value)
+ );
+}
+
+/** The outcomes under which a span keeps its quotation marks. */
+export const VERIFIED_QUOTE_OUTCOMES: ReadonlySet<QuoteCheckOutcome> = new Set([
+ QUOTE_CHECK_OUTCOME.verifiedInBlock,
+ QUOTE_CHECK_OUTCOME.verifiedInSource,
+]);
+
+/** Findings returned per check: the page-lint cap, one ceiling for both reports. */
+export const QUOTE_FINDINGS_MAX = PAGE_LINT_MAX_FINDINGS;
+
+/** Longest span preview a finding carries, in characters. */
+export const QUOTE_SPAN_PREVIEW_MAX_CHARS = 120;
+
+/** One span that failed verification and was unquoted. */
+export interface QuoteFinding {
+ /** 0-based index of the claim the span sits in. */
+ readonly claim: number;
+ readonly outcome: QuoteCheckOutcome;
+ /** The span's text, capped at {@link QUOTE_SPAN_PREVIEW_MAX_CHARS}. */
+ readonly span: string;
+}
+
+/** The quote check's account of one distillation. */
+export interface QuoteCheckReport {
+ /** Spans checked across every claim. */
+ readonly checked: number;
+ readonly verified_in_block: number;
+ readonly verified_in_source: number;
+ /** Spans that failed and lost their quotation marks. */
+ readonly unquoted: number;
+ /** Marks that paired with nothing; counted, never edited. */
+ readonly unpaired: number;
+ /** Failed spans, capped at {@link QUOTE_FINDINGS_MAX}. */
+ readonly findings: ReadonlyArray<QuoteFinding>;
+ /** Failed spans before the cap. */
+ readonly total: number;
+ /** Failed spans after the cap. */
+ readonly returned: number;
+ readonly truncated: boolean;
+}
+
+/** Wire code of a strict refusal; registered in the MCP error registry. */
+export const QUOTE_UNVERIFIED_CODE = "quote_unverified";
+
+/** Frontmatter key: spans that verified on this page. */
+export const QUOTES_VERIFIED_KEY = "quotes_verified";
+
+/** Frontmatter key: spans that failed and were unquoted on this page. */
+export const QUOTES_UNQUOTED_KEY = "quotes_unquoted";
+
+/**
+ * A strict distillation refused its write because a quoted span did not
+ * verify. The message names claim indices and outcomes only, never span text
+ * or paths, because it reaches an MCP caller.
+ */
+export class QuoteCheckError extends Error {
+ readonly code = QUOTE_UNVERIFIED_CODE;
+ readonly findings: ReadonlyArray<QuoteFinding>;
+
+ constructor(findings: ReadonlyArray<QuoteFinding>) {
+ const named = findings.map((finding) => `claim ${finding.claim}: ${finding.outcome}`);
+ super(`quoted spans failed verification: ${named.join(", ")}`);
+ this.name = "QuoteCheckError";
+ this.findings = findings;
+ }
+}
diff --git a/src/core/brain/doctor/store-integrity.ts b/src/core/brain/doctor/store-integrity.ts
index 7ba2002d..6d805880 100644
--- a/src/core/brain/doctor/store-integrity.ts
+++ b/src/core/brain/doctor/store-integrity.ts
@@ -46,15 +46,24 @@ export const tierDriftCheck: DoctorCheck = {
if (driftCount > 0) {
issues.push({
severity: "warning",
- code: "tier-drift",
- message:
- `${driftCount} identity-field hand-edit(s) staged - ` +
- "review with: o2b brain tiers check",
+ code: TIER_DRIFT_CODE,
+ message: tierDriftMessage(driftCount),
});
}
},
};
+/** The code of the {@link tierDriftCheck} warning. */
+export const TIER_DRIFT_CODE = "tier-drift";
+
+/**
+ * The {@link tierDriftCheck} warning text for `count` staged drift rows,
+ * shared with a surface that recounts the rows a caller may read.
+ */
+export function tierDriftMessage(count: number): string {
+ return `${count} identity-field hand-edit(s) staged - review with: o2b brain tiers check`;
+}
+
/**
* `dangling-workrun` (v0.12.0, Brain Integrity Suite): surfaces every
* dream-pass workrun JSONL whose last event is neither `finalized`
diff --git a/src/core/brain/dream-stage.ts b/src/core/brain/dream-stage.ts
index 20ea5a66..cc70878c 100644
--- a/src/core/brain/dream-stage.ts
+++ b/src/core/brain/dream-stage.ts
@@ -156,6 +156,8 @@ export interface DreamStageOptions {
*/
readonly onProgress?: DreamOptions["onProgress"];
readonly agentName?: string;
+ /** Forwarded to the dream pass's link-candidate filter; see `DreamOptions.readable`. */
+ readonly readable?: DreamOptions["readable"];
}
/** Sorted-copy helper keeping projections order-insensitive. */
@@ -323,6 +325,7 @@ export function stageDream(vault: string, opts: DreamStageOptions): DreamStageBu
...(opts.safeguard !== undefined ? { safeguard: opts.safeguard } : {}),
...(opts.onProgress !== undefined ? { onProgress: opts.onProgress } : {}),
...(opts.agentName !== undefined ? { agentName: opts.agentName } : {}),
+ ...(opts.readable !== undefined ? { readable: opts.readable } : {}),
});
const plan = projectDreamPlan(summary);
const salience = projectSalienceGate(summary.salience_gate);
diff --git a/src/core/brain/dream-types.ts b/src/core/brain/dream-types.ts
index 90602afc..12515fa8 100644
--- a/src/core/brain/dream-types.ts
+++ b/src/core/brain/dream-types.ts
@@ -215,6 +215,15 @@ export interface DreamOptions {
* threaded yet); the CLI always provides the value.
*/
readonly agentName?: string;
+ /**
+ * May the caller read the page at this vault-relative path at its
+ * reach? ANDed into the link-candidate filter beside the owner scope, so
+ * a page withheld from the caller is not offered as a citation target
+ * nor counted in the manifest's total, exactly as an absent page. The
+ * MCP handler binds it to `readableAtContextReach(ctx)`; absent (the
+ * CLI) filters nothing beyond the owner scope.
+ */
+ readonly readable?: (rel: string) => boolean;
/**
* Cooperative deadline (t_06784b8d). Checkpointed at exactly five
* points, in order: entry, before the pre-run snapshot, before the
diff --git a/src/core/brain/dream.ts b/src/core/brain/dream.ts
index b36d63b6..c8b02a3e 100644
--- a/src/core/brain/dream.ts
+++ b/src/core/brain/dream.ts
@@ -288,7 +288,7 @@ function dreamRun(
// the ladder is planned before the `changed` gate, and the runId
// collision path below re-plans - so an eager build paid that walk on
// every dry run that fires nothing, and twice on a collision.
- const linkCandidates = memoizedLinkCandidates(vault, opts.agentName);
+ const linkCandidates = memoizedLinkCandidates(vault, opts.agentName, opts.readable);
// Count-triggered fact rollup ladder (S3): pure counters over the
// ADMITTED fact artifacts (preferences) against the persisted per-tier
@@ -581,12 +581,17 @@ function buildRollupPlan(
function memoizedLinkCandidates(
vault: string,
agentName: string | undefined,
+ readable: ((rel: string) => boolean) | undefined,
): () => LinkCandidateManifest {
let manifest: LinkCandidateManifest | null = null;
- return () =>
- (manifest ??= buildLinkCandidateManifest(vault, {
- visible: gatedOwnerScopeView(vault, agentName).visible,
- }));
+ return () => {
+ if (manifest !== null) return manifest;
+ const ownerView = gatedOwnerScopeView(vault, agentName);
+ manifest = buildLinkCandidateManifest(vault, {
+ visible: (rel) => ownerView.visible(rel) && (readable?.(rel) ?? true),
+ });
+ return manifest;
+ };
}
function formatRunId(d: Date): string {
diff --git a/src/core/brain/event-trace.ts b/src/core/brain/event-trace.ts
index 49c7db19..0d0184da 100644
--- a/src/core/brain/event-trace.ts
+++ b/src/core/brain/event-trace.ts
@@ -35,6 +35,11 @@
*/
import { readLogDay } from "./log-jsonl.ts";
+import {
+ UNFILTERED_ARTIFACT_REFS,
+ everyArtifactRefView,
+ type ArtifactRefView,
+} from "./artifact-ref-view.ts";
import { ownerScopeView } from "./owner-scope-view.ts";
import type { BrainLogEntry, BrainLogEntryPayload } from "./log.ts";
import { loadNormalizedContinuityRecords } from "./continuity/read-model.ts";
@@ -98,6 +103,15 @@ export interface EventTraceSelector {
* artifact this caller may not see is a log line about that artifact.
*/
readonly ownerScope?: string | null;
+ /**
+ * A second reference view every event must also pass, typically the
+ * caller's reach (`reachView`). Asked over the same references as the
+ * owner scope, BEFORE the cap, so a withheld event is never counted and
+ * never takes a slot an absent one would leave free. An attached trace
+ * whose `handoffRef` the view hides is dropped from that event. Absent
+ * filters nothing.
+ */
+ readonly view?: ArtifactRefView;
}
/** One continuity record attached to a log event, with the join provenance. */
@@ -211,7 +225,10 @@ export function resolveLogEventTraces(
}
const { entries } = readLogDay(vault, date);
- const view = ownerScopeView(vault, selector.ownerScope ?? null);
+ const view = everyArtifactRefView(
+ ownerScopeView(vault, selector.ownerScope ?? null),
+ selector.view ?? UNFILTERED_ARTIFACT_REFS,
+ );
const records = loadNormalizedContinuityRecords(
vault,
selector.keepPrivate === true ? { keepPrivate: true } : {},
@@ -230,7 +247,9 @@ export function resolveLogEventTraces(
if (selector.sessionId !== undefined && correlation.sessionId !== selector.sessionId) continue;
if (!view.row(...correlation.artifacts, ...bodyStrings(entry.body))) continue;
if (cap !== undefined && results.length >= cap) break;
- const traces = attachTracesToEvent(records, correlation);
+ const traces = attachTracesToEvent(records, correlation).filter((trace) =>
+ view.row(trace.handoffRef),
+ );
results.push({
event: {
timestamp: entry.timestamp,
diff --git a/src/core/brain/hygiene/detectors/capture-scope.ts b/src/core/brain/hygiene/detectors/capture-scope.ts
new file mode 100644
index 00000000..8dc1a899
--- /dev/null
+++ b/src/core/brain/hygiene/detectors/capture-scope.ts
@@ -0,0 +1,196 @@
+/**
+ * Capture-scope detector: active knowledge that rests on url-only sources.
+ *
+ * A page whose every cited source has no local bytes behind it rests on a
+ * locator that may rot, and nothing on the page says so once the untrusted
+ * marker is gone (an entity released from quarantine) or was never written
+ * (a research report). This detector names those pages: one `warning` per
+ * page, `review` proposed action - deciding whether to capture the source,
+ * re-cite a local one or retire the page is an operator call.
+ *
+ * Inspected: retrievable pages (the retrieval trust gate does not
+ * quarantine them) of the four knowledge kinds - distillations, ingest
+ * summaries, research reports, and entities in the canonical status scope.
+ * A quarantined page is skipped: its untrusted marker already names the
+ * condition.
+ *
+ * Stamp at write, re-derive on read. The scope of every cited identity is
+ * re-derived on each sweep from its shape and one `stat` (never a byte
+ * read), so a full-local page whose file vanished reports as url-only. A
+ * stamped `bounded-local` counts as backing only while the page's excerpt
+ * block is present and matches its stored digest; an edited or removed
+ * excerpt is no longer a capture. A cited source the filesystem refuses to
+ * answer for (permission denied, a symlink loop) is not provably url-only,
+ * so it counts as backing: one unreadable citation never aborts the sweep
+ * and never hides the other pages' findings. A cited file the caller may not
+ * read at its reach counts as url-only, the answer an absent file gives, so
+ * a finding never tells a narrower caller that a hidden page exists.
+ *
+ * Cited sources are read with the shared source-link readers: the page's
+ * `source_path` plus the targets under its `## Sources` heading.
+ */
+
+import { existsSync } from "node:fs";
+import { join, posix } from "node:path";
+
+import { listVaultNotePaths, parseFrontmatter } from "../../../vault.ts";
+import type { FrontmatterMap } from "../../../types.ts";
+import {
+ BRAIN_DISTILLATIONS_REL,
+ BRAIN_ENTITIES_REL,
+ BRAIN_REPORTS_REL,
+ BRAIN_SOURCES_REL,
+} from "../../path-constants.ts";
+import { BRAIN_DISTILLATION_KIND } from "../../distill/distill-source.ts";
+import { BRAIN_SOURCE_KIND } from "../../ingest/ingest.ts";
+import { BRAIN_REPORT_KIND } from "../../research/research.ts";
+import { BRAIN_ENTITY_KIND } from "../../entities/types.ts";
+import { ENTITY_STATUS_SCOPE, entityStatusInScope } from "../../entities/status-scope.ts";
+import { classifyRetrievalTrust } from "../../trust/retrieval-gate.ts";
+import {
+ CAPTURE_SCOPE,
+ CAPTURE_SCOPE_KEY,
+ EXCERPT_HASH_KEY,
+ excerptDigest,
+ isCaptureScope,
+ readExcerptSection,
+ resolveCaptureScope,
+ type CaptureScope,
+} from "../../provenance/capture-scope.ts";
+import { SourceTrustError, isSourceHidden } from "../../intake/source-trust.ts";
+import { sourcesSectionTargets } from "../../source-links.ts";
+import { hygieneFindingId } from "./id.ts";
+import type { HygieneFinding } from "../types.ts";
+
+export const CAPTURE_SCOPE_DETECTOR_ID = "capture-scope" as const;
+
+/** One-line summary every finding of this detector carries. */
+export const CAPTURE_SCOPE_FINDING_TITLE = "Active knowledge rests on url-only sources";
+
+/** Frontmatter key holding a single-source page's source identity. */
+const SOURCE_PATH_KEY = "source_path";
+
+/** Frontmatter keys the inspection reads. */
+const KIND_KEY = "kind";
+const STATUS_KEY = "status";
+
+/** Directories the four inspected kinds are written to. */
+const INSPECTED_DIRS: ReadonlyArray<string> = Object.freeze([
+ BRAIN_DISTILLATIONS_REL,
+ BRAIN_SOURCES_REL,
+ BRAIN_REPORTS_REL,
+ BRAIN_ENTITIES_REL,
+]);
+
+/** Page kinds that carry knowledge built from cited sources. */
+const INSPECTED_KINDS: ReadonlySet<string> = new Set([
+ BRAIN_DISTILLATION_KIND,
+ BRAIN_SOURCE_KIND,
+ BRAIN_REPORT_KIND,
+ BRAIN_ENTITY_KIND,
+]);
+
+function stringValue(meta: FrontmatterMap, key: string): string | null {
+ const value = meta[key];
+ return typeof value === "string" && value.length > 0 ? value : null;
+}
+
+/** Is this page active knowledge a default recall would serve? */
+function isRetrievableKnowledge(meta: FrontmatterMap): boolean {
+ const kind = stringValue(meta, KIND_KEY);
+ if (kind === null || !INSPECTED_KINDS.has(kind)) return false;
+ if (kind === BRAIN_ENTITY_KIND) {
+ const status = stringValue(meta, STATUS_KEY);
+ if (status === null || !entityStatusInScope(status, ENTITY_STATUS_SCOPE.canonical)) {
+ return false;
+ }
+ }
+ return !classifyRetrievalTrust(meta).quarantined;
+}
+
+/** The page's cited source identities, deduplicated, in reading order. */
+function citedSources(meta: FrontmatterMap, body: string): ReadonlyArray<string> {
+ const sourcePath = stringValue(meta, SOURCE_PATH_KEY);
+ const cited = [...(sourcePath !== null ? [sourcePath] : []), ...sourcesSectionTargets(body)];
+ return [...new Set(cited)];
+}
+
+/** The page's own `capture_scope` stamp, `null` when absent or not a scope. */
+function stampedScope(meta: FrontmatterMap): CaptureScope | null {
+ const value = meta[CAPTURE_SCOPE_KEY];
+ return isCaptureScope(value) ? value : null;
+}
+
+/** Does the page still hold the excerpt its stored digest describes? */
+function excerptIntact(meta: FrontmatterMap, body: string): boolean {
+ const digest = stringValue(meta, EXCERPT_HASH_KEY);
+ if (digest === null) return false;
+ const excerpt = readExcerptSection(body);
+ return excerpt !== null && excerptDigest(excerpt) === digest;
+}
+
+/** May the caller read this vault-relative file? Always, without a predicate. */
+type Readable = ((rel: string) => boolean) | undefined;
+
+/**
+ * Is this cited source url-only now, as the caller may know it? A refused
+ * stat is not proof that it is, unless the caller may not read the file
+ * either, which an absent file would answer too; a backing file the caller
+ * may not read is.
+ */
+function isUrlOnly(vault: string, source: string, readable: Readable): boolean {
+ try {
+ const { scope, backing } = resolveCaptureScope(vault, source);
+ if (scope === CAPTURE_SCOPE.urlOnly) return true;
+ return backing !== null && readable !== undefined && !readable(backing);
+ } catch (err) {
+ if (err instanceof SourceTrustError) return isSourceHidden(vault, source, readable);
+ throw err;
+ }
+}
+
+function inspectPage(vault: string, page: string, readable: Readable): HygieneFinding | null {
+ const [meta, body] = parseFrontmatter(join(vault, page));
+ if (!isRetrievableKnowledge(meta)) return null;
+
+ const sources = citedSources(meta, body);
+ if (sources.length === 0) return null;
+ const allUrlOnly = sources.every((source) => isUrlOnly(vault, source, readable));
+ if (!allUrlOnly) return null;
+
+ const stamped = stampedScope(meta);
+ if (stamped === CAPTURE_SCOPE.boundedLocal && excerptIntact(meta, body)) return null;
+
+ const targets = Object.freeze([page]);
+ return Object.freeze({
+ id: hygieneFindingId(CAPTURE_SCOPE_DETECTOR_ID, targets),
+ detector: CAPTURE_SCOPE_DETECTOR_ID,
+ severity: "warning" as const,
+ title: CAPTURE_SCOPE_FINDING_TITLE,
+ targets,
+ proposed_action: "review" as const,
+ evidence: Object.freeze({ sources: Object.freeze([...sources]), stamped }),
+ });
+}
+
+/**
+ * Every finding in the vault. `readable` is the caller's reach (see
+ * `HygieneDetectorContext.readable`); a local caller passes nothing.
+ */
+export function detectCaptureScope(
+ vault: string,
+ readable?: (rel: string) => boolean,
+): ReadonlyArray<HygieneFinding> {
+ const pages: string[] = [];
+ for (const dir of INSPECTED_DIRS) {
+ const abs = join(vault, dir);
+ if (!existsSync(abs)) continue;
+ for (const rel of listVaultNotePaths(abs)) pages.push(posix.join(dir, rel));
+ }
+ const findings: HygieneFinding[] = [];
+ for (const page of pages.toSorted()) {
+ const finding = inspectPage(vault, page, readable);
+ if (finding !== null) findings.push(finding);
+ }
+ return Object.freeze(findings);
+}
diff --git a/src/core/brain/hygiene/scan.ts b/src/core/brain/hygiene/scan.ts
index 5b482933..7495cf23 100644
--- a/src/core/brain/hygiene/scan.ts
+++ b/src/core/brain/hygiene/scan.ts
@@ -10,6 +10,7 @@
* the vault; remediation lives in `apply.ts`.
*/
+import { detectCaptureScope } from "./detectors/capture-scope.ts";
import { detectConflicts } from "./detectors/conflicts.ts";
import { detectDedup } from "./detectors/dedup.ts";
import { detectFreshness } from "./detectors/freshness.ts";
@@ -33,6 +34,7 @@ const DETECTORS: Readonly<Record<HygieneDetectorId, HygieneDetector>> = Object.f
usefulness: (vault, ctx) => detectUsefulness(vault, ctx),
"slug-collisions": (vault) => detectSlugCollisions(vault),
tags: (vault) => detectTags(vault),
+ "capture-scope": (vault, ctx) => detectCaptureScope(vault, ctx.readable),
});
export interface RunHygieneScanOptions {
@@ -44,6 +46,8 @@ export interface RunHygieneScanOptions {
readonly detectors?: ReadonlyArray<HygieneDetectorId>;
/** Injected clock. */
readonly now: Date;
+ /** Handed to every detector; see `HygieneDetectorContext.readable`. */
+ readonly readable?: (rel: string) => boolean;
}
export function runHygieneScan(vault: string, opts: RunHygieneScanOptions): HygieneScanReport {
@@ -58,7 +62,10 @@ export function runHygieneScan(vault: string, opts: RunHygieneScanOptions): Hygi
for (const id of requested) {
try {
- const detected = DETECTORS[id](vault, { now: opts.now });
+ const detected = DETECTORS[id](vault, {
+ now: opts.now,
+ ...(opts.readable !== undefined ? { readable: opts.readable } : {}),
+ });
counts[id] = detected.length;
findings.push(...detected);
} catch (error) {
diff --git a/src/core/brain/hygiene/types.ts b/src/core/brain/hygiene/types.ts
index fb91b487..119e69e4 100644
--- a/src/core/brain/hygiene/types.ts
+++ b/src/core/brain/hygiene/types.ts
@@ -16,6 +16,7 @@ export const HYGIENE_DETECTOR_IDS = [
"usefulness",
"slug-collisions",
"tags",
+ "capture-scope",
] as const;
export type HygieneDetectorId = (typeof HYGIENE_DETECTOR_IDS)[number];
@@ -28,7 +29,8 @@ export type HygieneDetectorId = (typeof HYGIENE_DETECTOR_IDS)[number];
* mechanism, two explicit policies: `slug-collisions` is default-on
* (fires only on actual same-stem groups), `tags` is opt-in (noisy on
* vaults that tag loosely), and further noisy detectors register as
- * opt-in by staying out of this list.
+ * opt-in by staying out of this list. `capture-scope` is default-on: it
+ * fires only on retrievable knowledge whose every source is url-only.
*
* Members are compile-checked against the registered tuple; a registered
* id missing here is simply default-off, never an error.
@@ -39,6 +41,7 @@ export const DEFAULT_SCAN_IDS: ReadonlyArray<HygieneDetectorId> = Object.freeze(
"freshness",
"usefulness",
"slug-collisions",
+ "capture-scope",
]);
export type HygieneSeverity = "info" | "warning" | "action";
@@ -79,6 +82,13 @@ export interface HygieneFinding {
export interface HygieneDetectorContext {
/** Injected clock - detectors never read the wall clock themselves. */
readonly now: Date;
+ /**
+ * May the caller read the vault file at this vault-relative path? Supplied
+ * by a surface that answers at a reach narrower than the vault; a local
+ * caller passes nothing. A detector that derives an answer from another
+ * file's presence treats a refused file as absent.
+ */
+ readonly readable?: (rel: string) => boolean;
}
export type HygieneDetector = (
diff --git a/src/core/brain/idea-discovery.ts b/src/core/brain/idea-discovery.ts
index d3a37ae6..15af906e 100644
--- a/src/core/brain/idea-discovery.ts
+++ b/src/core/brain/idea-discovery.ts
@@ -43,6 +43,13 @@ export interface DiscoverIdeasOptions {
readonly lookbackDays?: number;
/** Injection point for tests; default reads the morning-brief scan. */
readonly openQuestions?: ReadonlyArray<MorningBriefOpenQuestion>;
+ /**
+ * May the caller read the page at this vault-relative path? A page it
+ * may not read is skipped before it is read or aged: it is neither a
+ * candidate nor a source of inbound links, so the ranking is the one a
+ * vault without that page would give. Absent, every page counts.
+ */
+ readonly include?: (rel: string) => boolean;
}
interface NoteFile {
@@ -161,8 +168,9 @@ export function discoverIdeas(
// Two passes: inbound links are collected from the FULL Brain tree
// (a reference from a handoff or trigger still counts), while orphan
// CANDIDATES come only from non-machine dirs.
- const allNotes = listBrainNotes(vault);
- const notes = listBrainNotes(vault, MACHINE_DIRS);
+ const included = (rel: string): boolean => opts.include?.(rel) ?? true;
+ const allNotes = listBrainNotes(vault).filter((note) => included(note.relPath));
+ const notes = listBrainNotes(vault, MACHINE_DIRS).filter((note) => included(note.relPath));
const inbound = new Set<string>();
for (const note of allNotes) {
let content: string;
@@ -201,6 +209,7 @@ export function discoverIdeas(
if (existsSync(inbox)) {
for (const name of readdirSync(inbox)) {
if (!name.startsWith("sig-") || !name.endsWith(".md")) continue;
+ if (!included(`Brain/inbox/${name}`)) continue;
const absPath = join(inbox, name);
const age = ageDays(absPath, opts.now);
// An unmeasurable age cannot clear the aging threshold and cannot
diff --git a/src/core/brain/ingest/batch-plan.ts b/src/core/brain/ingest/batch-plan.ts
index 7c663f09..ce71b06f 100644
--- a/src/core/brain/ingest/batch-plan.ts
+++ b/src/core/brain/ingest/batch-plan.ts
@@ -119,6 +119,14 @@ export interface BatchPlanOptions {
* overrides the repository. Absent/empty → no exclusion.
*/
readonly exclude?: readonly string[];
+ /**
+ * May the caller read this vault-relative file? Supplied by a surface that
+ * answers at a reach narrower than the vault; a local caller passes
+ * nothing. An ingestible file it refuses is treated as absent: never
+ * discovered, so it is in no batch, no total, no skip list and not in the
+ * plan id.
+ */
+ readonly include?: (rel: string) => boolean;
}
/** One file selected for (re-)ingest, with the reason it was selected. */
@@ -295,6 +303,7 @@ export function planBatches(vault: string, sourceDir: string, opts: BatchPlanOpt
out: discovered,
warnings: ignoreWarnings,
unclassifiable,
+ ...(opts.include !== undefined ? { include: opts.include } : {}),
});
}
const discoveredRel = discovered
@@ -549,6 +558,8 @@ interface WalkContext {
readonly warnings: IgnoreWarning[];
/** Counts files dropped for a non-ingestible extension, per extension. */
readonly unclassifiable: Map<string, number>;
+ /** See {@link BatchPlanOptions.include}. Absent: every file is included. */
+ readonly include?: (rel: string) => boolean;
}
/**
@@ -591,6 +602,9 @@ function collectIngestible(dir: string, scope: IgnoreScope, ctx: WalkContext): v
ctx.unclassifiable.set(ext, (ctx.unclassifiable.get(ext) ?? 0) + 1);
continue;
}
+ // After the extension test, so the predicate reads only files the plan
+ // could carry; a refused one is not this caller's to discover.
+ if (ctx.include !== undefined && !ctx.include(rel)) continue;
ctx.out.push(abs);
}
}
diff --git a/src/core/brain/ingest/content-manifest.ts b/src/core/brain/ingest/content-manifest.ts
index c4266e5b..537f44ce 100644
--- a/src/core/brain/ingest/content-manifest.ts
+++ b/src/core/brain/ingest/content-manifest.ts
@@ -73,6 +73,15 @@ export function manifestPath(vault: string): string {
return join(vault, MACHINE_ARTIFACT_DIR, MANIFEST_FILE);
}
+/**
+ * SHA-256 hex over raw bytes already in memory. The one digest every source
+ * record in this repository uses, so a caller that read a file once for
+ * another purpose records the same digest {@link hashFile} would.
+ */
+export function hashBytes(bytes: Uint8Array): string {
+ return createHash("sha256").update(bytes).digest("hex");
+}
+
/**
* SHA-256 hex over a single file's raw bytes. Timestamp-independent: only the
* content contributes to the digest. Throws if the path is a directory - use
@@ -83,7 +92,7 @@ export function hashFile(absPath: string): string {
if (stat.isDirectory()) {
throw new Error(`hashFile: path is a directory, use hashTree: ${absPath}`);
}
- return createHash("sha256").update(readFileSync(absPath)).digest("hex");
+ return hashBytes(readFileSync(absPath));
}
/**
diff --git a/src/core/brain/ingest/ingest.ts b/src/core/brain/ingest/ingest.ts
index 079cba4b..92c77750 100644
--- a/src/core/brain/ingest/ingest.ts
+++ b/src/core/brain/ingest/ingest.ts
@@ -36,7 +36,12 @@ import { sourcePagePath } from "../paths.ts";
import { assertVaultIdentityForWrite } from "../vault-identity.ts";
import { intakeExtraction, type ExtractionIntake } from "../intake/extract-intake.ts";
import { normalizeSourceIdentity } from "../intake/source-trust.ts";
-import { untrustedSourceFrontmatter } from "../trust/untrusted-provenance.ts";
+import { INTAKE_TRUST, untrustedSourceFrontmatter } from "../trust/untrusted-provenance.ts";
+import {
+ captureScopeForTrust,
+ captureScopeFrontmatter,
+ type CaptureScope,
+} from "../provenance/capture-scope.ts";
import {
renderProvenanceSection,
sourceIdentityHash,
@@ -88,6 +93,13 @@ export interface IngestSourceOptions {
* the result only, keeping the persisted page unchanged.
*/
readonly preExtract?: boolean;
+ /**
+ * May the caller read the vault file at this vault-relative path? Handed to
+ * the intake: a source the predicate refuses is classified as one with no
+ * local bytes (untrusted lane, no digest, `url-only`). A local caller passes
+ * nothing.
+ */
+ readonly readable?: (rel: string) => boolean;
}
export interface IngestSourceResult {
@@ -101,6 +113,14 @@ export interface IngestSourceResult {
readonly entitiesUpdated: readonly string[];
/** Pre-existing entity ids this source connected to (its connections). */
readonly connections: readonly string[];
+ /**
+ * How much of the source this page's knowledge was captured from: the
+ * intake's lane for the cited source, as a capture scope. `full-local` for
+ * a vault file, `url-only` for a source with no local bytes. Ingest never
+ * stores an excerpt, so it never answers `bounded-local`: the summary is a
+ * paraphrase, not a capture.
+ */
+ readonly captureScope: CaptureScope;
/**
* Code-structure pre-extraction seeds (P4), present only when the pass was
* requested via {@link IngestSourceOptions.preExtract}. `extracted: false`
@@ -152,8 +172,10 @@ export function ingestSource(
agent: opts.agent,
now: opts.now,
provenance,
+ ...(opts.readable !== undefined ? { readable: opts.readable } : {}),
});
const trust = intake.trust;
+ const captureScope = captureScopeForTrust(trust);
const connections = intake.entitiesUpdated;
const allEntities = [...intake.entitiesCreated, ...intake.entitiesUpdated];
@@ -179,6 +201,9 @@ export function ingestSource(
// reason rather than ranking it beside the operator's own notes. Trusted
// sources add nothing, keeping their page byte-identical to before.
...untrustedSourceFrontmatter(trust),
+ // Says how much of the source was captured when it is less than the
+ // whole file; a full-local source adds nothing, for the same reason.
+ ...captureScopeFrontmatter(captureScope),
created_at: createdAt,
updated_at: stamp,
tags: ["brain", "brain/source"],
@@ -212,7 +237,14 @@ export function ingestSource(
// an escaping identity must not reach `existsSync` either, or a path outside
// the vault would be hashed into the content manifest and keyed into a
// folder-plan checkpoint.
- if (resolvesInsideVault(vault, canonicalSource) && existsSync(join(vault, canonicalSource))) {
+ // Only in the trusted lane, which the intake grants only to a file the
+ // caller may read at its reach: a source answered as absent records no
+ // digest anywhere, the manifest included.
+ if (
+ trust === INTAKE_TRUST.trusted &&
+ resolvesInsideVault(vault, canonicalSource) &&
+ existsSync(join(vault, canonicalSource))
+ ) {
updateManifest(vault, [canonicalSource]);
// Record plan-scoped progress so an interrupted batch resumes at the item
// boundary (t_ba1fa5f6). Only for real vault files - a URL/identity-only
@@ -237,6 +269,7 @@ export function ingestSource(
entitiesCreated: intake.entitiesCreated,
entitiesUpdated: intake.entitiesUpdated,
connections,
+ captureScope,
...(preExtract !== undefined ? { preExtract } : {}),
};
}
diff --git a/src/core/brain/ingest/reconcile.ts b/src/core/brain/ingest/reconcile.ts
index f70e69e8..f1f47309 100644
--- a/src/core/brain/ingest/reconcile.ts
+++ b/src/core/brain/ingest/reconcile.ts
@@ -62,8 +62,17 @@ export interface ReconcileReport {
* nothing. A plan that dispatched nothing, or whose sources all completed,
* reports an empty (explicitly `complete`) gap.
*/
-export function reconcilePlan(vault: string, plan: BatchPlan): ReconcileReport {
- const completed = new Set(readCheckpoint(vault, plan.planId)?.completed ?? []);
+export function reconcilePlan(
+ vault: string,
+ plan: BatchPlan,
+ include?: (rel: string) => boolean,
+): ReconcileReport {
+ // The checkpoint may record a source the plan itself left out for this
+ // caller (see `BatchPlanOptions.include`); it is left out here too.
+ const recorded = readCheckpoint(vault, plan.planId)?.completed ?? [];
+ const completed = new Set(
+ include === undefined ? recorded : recorded.filter((path) => include(path)),
+ );
// Batch files are the sources actually dispatched for (re)ingest this run.
const batchFiles = new Set<string>();
diff --git a/src/core/brain/intake/extract-intake.ts b/src/core/brain/intake/extract-intake.ts
index 0b545275..1ff67a2e 100644
--- a/src/core/brain/intake/extract-intake.ts
+++ b/src/core/brain/intake/extract-intake.ts
@@ -37,7 +37,13 @@ import { validateEntityCategory, normalizeEntityName } from "../entities/canonic
import { relateEntities, upsertEntity } from "../entities/registry.ts";
import { renderProvenanceSection, type Provenance } from "../provenance/provenance.ts";
import { INTAKE_TRUST, type IntakeTrust } from "../trust/untrusted-provenance.ts";
-import { classifySourceOrigin, classifySourceTrust, type SourceOrigin } from "./source-trust.ts";
+import {
+ UNTRUSTED_ORIGIN,
+ classifySourceOrigin,
+ classifySourceTrust,
+ isSourceHidden,
+ type SourceOrigin,
+} from "./source-trust.ts";
/** One entity the agent extracted from a source. */
export interface IntakeEntity {
@@ -77,6 +83,14 @@ export interface IntakeOptions {
* itself, which nothing enforced for the next caller.
*/
readonly provenance: Provenance;
+ /**
+ * May the caller read the vault file at this vault-relative path? Supplied
+ * by a surface that answers at a reach narrower than the vault (the MCP
+ * tools at remote reach); a local caller passes nothing. A cited vault file
+ * the predicate refuses is classified as a source with no local bytes
+ * (untrusted lane, no digest), exactly like an absent file.
+ */
+ readonly readable?: (rel: string) => boolean;
}
export interface IntakeResult {
@@ -162,7 +176,11 @@ function validateIntake(intake: ExtractionIntake): void {
* itself the way in, while quarantining it punishes a caller for a question
* nobody asked. The remedy is to name a source, so the caller is told to.
*/
-function resolveIntakeOrigin(vault: string, provenance: Provenance): SourceOrigin {
+function resolveIntakeOrigin(
+ vault: string,
+ provenance: Provenance,
+ readable: ((rel: string) => boolean) | undefined,
+): SourceOrigin {
const sources = provenance.sources;
if (sources.length === 0) {
throw new IntakeValidationError(
@@ -176,12 +194,21 @@ function resolveIntakeOrigin(vault: string, provenance: Provenance): SourceOrigi
// each of them in full and then discarded every digest but the first - and
// discarded that one too the moment there was more than one source.
const only = sources.length === 1 ? sources[0] : undefined;
- if (only !== undefined) return classifySourceOrigin(vault, only);
+ // The reach question comes first, before any stat or read, so neither a
+ // refused stat nor the size ceiling can name a page the caller may not read.
+ if (only !== undefined) {
+ return isSourceHidden(vault, only, readable)
+ ? UNTRUSTED_ORIGIN
+ : classifySourceOrigin(vault, only);
+ }
- // `some` rather than a full map: the first source outside the vault settles
- // the whole intake, and nothing after it changes the answer.
+ // `some` rather than a full map: the first source outside the vault (or
+ // hidden from the caller) settles the whole intake, and nothing after it
+ // changes the answer.
const untrusted = sources.some(
- (source) => classifySourceTrust(vault, source) === INTAKE_TRUST.untrusted,
+ (source) =>
+ isSourceHidden(vault, source, readable) ||
+ classifySourceTrust(vault, source) === INTAKE_TRUST.untrusted,
);
return { trust: untrusted ? INTAKE_TRUST.untrusted : INTAKE_TRUST.trusted };
}
@@ -199,7 +226,7 @@ export function intakeExtraction(
validateIntake(intake);
// Before any write, like the payload validation above: an intake that
// cannot be classified must not leave half its entities behind.
- const origin = resolveIntakeOrigin(vault, opts.provenance);
+ const origin = resolveIntakeOrigin(vault, opts.provenance, opts.readable);
const provenanceSection = renderProvenanceSection(opts.provenance);
const untrustedOrigin = origin.trust === INTAKE_TRUST.untrusted;
diff --git a/src/core/brain/intake/source-trust.ts b/src/core/brain/intake/source-trust.ts
index fd9642bd..b3d26859 100644
--- a/src/core/brain/intake/source-trust.ts
+++ b/src/core/brain/intake/source-trust.ts
@@ -53,11 +53,24 @@
* that question is this module's to ask, and for the limit of the answer.
*/
-import { isAbsolute, join } from "node:path";
-import { statSync } from "node:fs";
-
-import { canonicalNotePath, ensureInsideVault, hasUriScheme } from "../../path-safety.ts";
-import { hashFile } from "../ingest/content-manifest.ts";
+import { isAbsolute, join, relative, sep } from "node:path";
+import {
+ closeSync,
+ constants as fsConstants,
+ fstatSync,
+ openSync,
+ readSync,
+ realpathSync,
+ statSync,
+} from "node:fs";
+
+import {
+ canonicalNotePath,
+ ensureInsideVault,
+ hasUriScheme,
+ realpathInsideVault,
+} from "../../path-safety.ts";
+import { hashBytes } from "../ingest/content-manifest.ts";
import { INTAKE_TRUST, type IntakeTrust } from "../trust/untrusted-provenance.ts";
/** `[[Articles/x.md]]` - the wikilink form the NER tool's `source` arrives in. */
@@ -76,6 +89,16 @@ const WIKILINK_ANCHOR_SEPARATOR = "#";
*/
const ERRNO_NO_SUCH_ENTRY = "ENOENT";
const ERRNO_NOT_A_DIRECTORY = "ENOTDIR";
+/** A symlink met where `O_NOFOLLOW` forbids one: not this vault's file. */
+const ERRNO_SYMLINK_LOOP = "ELOOP";
+
+/**
+ * How the one read opens a source: never through a final-component symlink,
+ * never blocking (a FIFO swapped in after the stat must not hang the read).
+ * Both flags are POSIX; where the platform lacks one it is simply absent.
+ */
+const SOURCE_OPEN_FLAGS =
+ fsConstants.O_RDONLY | (fsConstants.O_NOFOLLOW ?? 0) | (fsConstants.O_NONBLOCK ?? 0);
/** What a failure is called when it carries neither an errno nor a name. */
const UNNAMED_ERRNO = "unknown";
@@ -125,7 +148,23 @@ export interface SourceOrigin {
readonly contentHash?: string;
}
-const UNTRUSTED_ORIGIN: SourceOrigin = Object.freeze({ trust: INTAKE_TRUST.untrusted });
+/**
+ * A source origin together with the bytes its digest was computed over.
+ * `bytes` is present exactly when `contentHash` is: both come from one read,
+ * so a check that runs on these bytes runs on the bytes the page records.
+ */
+export interface SourceOriginWithBytes extends SourceOrigin {
+ readonly bytes?: Uint8Array;
+}
+
+/**
+ * The origin of a source with no local bytes: the untrusted lane, no digest.
+ * Also the answer for a vault file the caller may not read at its reach, so
+ * a hidden page and an absent one are indistinguishable.
+ */
+export const UNTRUSTED_ORIGIN: SourceOriginWithBytes = Object.freeze({
+ trust: INTAKE_TRUST.untrusted,
+});
/** Everything before the first occurrence of `separator`, or the whole string. */
function cutAt(value: string, separator: string): string {
@@ -238,6 +277,34 @@ function resolveVaultShapedPath(vault: string, canonical: string): string | null
}
}
+/**
+ * The vault-relative identity of a source SHAPED like a location inside this
+ * vault, in POSIX form with every `.` and `..` segment resolved, or `null`
+ * when the shape does not hold. Touches no file: it is the identity a reach
+ * predicate is asked about BEFORE anything is stat-ed or read, so a refusal
+ * of the stat or the size ceiling never names a page the caller may not read.
+ */
+export function vaultShapedIdentity(vault: string, sourcePath: string): string | null {
+ const abs = resolveVaultShapedPath(vault, normalizeSourceIdentity(sourcePath));
+ return abs === null ? null : relative(vault, abs).split(sep).join(PATH_SEPARATOR);
+}
+
+/**
+ * Does `readable` refuse this source? Asked on {@link vaultShapedIdentity},
+ * before any filesystem question about the source; a source with no
+ * vault-shaped identity is never asked about, and without a predicate
+ * nothing is refused.
+ */
+export function isSourceHidden(
+ vault: string,
+ sourcePath: string,
+ readable: ((rel: string) => boolean) | undefined,
+): boolean {
+ if (readable === undefined) return false;
+ const rel = vaultShapedIdentity(vault, sourcePath);
+ return rel !== null && !readable(rel);
+}
+
/** Is this the filesystem saying "nothing is there", rather than refusing? */
function isAbsenceErrno(cause: unknown): boolean {
const code = (cause as NodeJS.ErrnoException | null)?.code;
@@ -349,6 +416,18 @@ export function classifySourceTrust(vault: string, sourcePath: string): IntakeTr
* can be asked - and it belongs to the boundaries that can still ask.
*/
export function classifySourceOrigin(vault: string, sourcePath: string): SourceOrigin {
+ const { trust, contentHash } = readSourceOrigin(vault, sourcePath);
+ return contentHash === undefined ? UNTRUSTED_ORIGIN : { trust, contentHash };
+}
+
+/**
+ * {@link classifySourceOrigin}, keeping the bytes the digest was computed
+ * over. Same verdicts, same refusals ({@link SourceTrustError}, the
+ * {@link SOURCE_HASH_MAX_BYTES} ceiling), and ONE read of the file: the
+ * digest and the returned bytes cannot describe two different versions of
+ * the source, which a hash followed by a second read for the caller could.
+ */
+export function readSourceOrigin(vault: string, sourcePath: string): SourceOriginWithBytes {
const file = resolveVaultSourceFile(vault, sourcePath);
if (file === null) return UNTRUSTED_ORIGIN;
@@ -358,24 +437,67 @@ export function classifySourceOrigin(vault: string, sourcePath: string): SourceO
// case), so reusing it here would make an unrecordable source look like a
// multi-source intake and quietly drop the audit record this classifier
// exists to produce.
- throw new SourceTrustError(
- `source ${file.identity} is ${file.size} bytes, past the ${SOURCE_HASH_MAX_BYTES}-byte ` +
- "ceiling on a source this classifier will read to record its digest",
- );
+ throw ceilingRefusal(file.identity, file.size);
}
+ let bytes: Uint8Array | null;
try {
- // The one hasher in this repository, so the digest a summary page records
- // and the digest an entity page records cannot drift apart. It stats the
- // file again; that is the price of one hasher, and it is the READ this
- // ceiling was added to bound, not the stat.
- return { trust: INTAKE_TRUST.trusted, contentHash: hashFile(file.abs) };
+ bytes = readBoundedSource(vault, file);
} catch (cause) {
- // The file went away between the stat and the read. That is the same
- // answer the stat itself would have given a moment later - there is
- // nothing there - and a race is not a reason to fail an intake that a
- // retry would classify cleanly.
- if (isAbsenceErrno(cause)) return UNTRUSTED_ORIGIN;
+ if (cause instanceof SourceTrustError) throw cause;
+ // The file went away (or was swapped for a symlink) between the stat and
+ // the read. That is the same answer the stat itself would have given a
+ // moment later - there is nothing of ours there - and a race is not a
+ // reason to fail an intake that a retry would classify cleanly.
+ if (isAbsenceErrno(cause) || errnoCode(cause) === ERRNO_SYMLINK_LOOP) return UNTRUSTED_ORIGIN;
throw refusal(file.identity, cause);
}
+ if (bytes === null) return UNTRUSTED_ORIGIN;
+ // The one digest function in this repository, so the digest a summary page
+ // records and the digest an entity page records cannot drift apart.
+ return { trust: INTAKE_TRUST.trusted, contentHash: hashBytes(bytes), bytes };
+}
+
+/** The size refusal, composed from the identity and integers only. */
+function ceilingRefusal(identity: string, size: number): SourceTrustError {
+ return new SourceTrustError(
+ `source ${identity} is ${size} bytes, past the ${SOURCE_HASH_MAX_BYTES}-byte ` +
+ "ceiling on a source this classifier will read to record its digest",
+ );
+}
+
+/**
+ * Read a resolved source through ONE descriptor, so the checks that admitted
+ * it hold for the bytes read: the path is resolved to its real location and
+ * must still be inside the vault, the open refuses a symlink in the final
+ * component, the descriptor must be a regular file, and no more than the
+ * ceiling is read however large the file has grown since the stat. `null`
+ * when what is there now is not a regular file of this vault.
+ */
+function readBoundedSource(vault: string, file: VaultSourceFile): Uint8Array | null {
+ const real = realpathSync(file.abs);
+ if (!realpathInsideVault(real, vault)) return null;
+ const fd = openSync(real, SOURCE_OPEN_FLAGS);
+ try {
+ const stat = fstatSync(fd);
+ if (!stat.isFile()) return null;
+ if (stat.size > SOURCE_HASH_MAX_BYTES) throw ceilingRefusal(file.identity, stat.size);
+ // One byte past the expected size, so growth since the fstat is seen.
+ let buffer = Buffer.allocUnsafe(stat.size + 1);
+ let filled = 0;
+ for (;;) {
+ if (filled === buffer.length) {
+ if (filled > SOURCE_HASH_MAX_BYTES) throw ceilingRefusal(file.identity, filled);
+ const grown = Buffer.allocUnsafe(Math.min(buffer.length * 2, SOURCE_HASH_MAX_BYTES + 1));
+ buffer.copy(grown, 0, 0, filled);
+ buffer = grown;
+ }
+ const read = readSync(fd, buffer, filled, buffer.length - filled, null);
+ if (read === 0) break;
+ filled += read;
+ }
+ return buffer.subarray(0, filled);
+ } finally {
+ closeSync(fd);
+ }
}
diff --git a/src/core/brain/link-graph/format-wikilink.ts b/src/core/brain/link-graph/format-wikilink.ts
index b22481b8..3a2f7bfe 100644
--- a/src/core/brain/link-graph/format-wikilink.ts
+++ b/src/core/brain/link-graph/format-wikilink.ts
@@ -32,12 +32,9 @@ export function isWikiLinkFormat(value: string): value is WikiLinkFormat {
return (WIKI_LINK_FORMATS as ReadonlyArray<string>).includes(value);
}
-/** Mirrors the masks used by the link-graph parser, broadened for the
- * rewrite path: backtick fences of any length (3+), tilde fences, and
- * inline code spans all stay verbatim. */
-import { RICH_WIKILINK_RE } from "../wikilink.ts";
-
-const CODE_BLOCK_RE = /(`{3,}|~{3,})[\s\S]*?\1|`[^`]+`/g;
+/** Fenced blocks and inline code spans stay verbatim on the rewrite path:
+ * the regions every link reader masks. */
+import { RICH_WIKILINK_RE, codeRegions } from "../wikilink.ts";
export interface NormalizeResult {
readonly content: string;
@@ -199,10 +196,10 @@ export function normalizeWikilinks(
let out = "";
let last = 0;
- for (const m of content.matchAll(CODE_BLOCK_RE)) {
- out += transformSegment(content.slice(last, m.index));
- out += m[0];
- last = m.index + m[0].length;
+ for (const [start, end] of codeRegions(content)) {
+ out += transformSegment(content.slice(last, start));
+ out += content.slice(start, end);
+ last = end;
}
out += transformSegment(content.slice(last));
diff --git a/src/core/brain/link-graph/parse-wikilink.ts b/src/core/brain/link-graph/parse-wikilink.ts
index 4f0bc795..99d2776f 100644
--- a/src/core/brain/link-graph/parse-wikilink.ts
+++ b/src/core/brain/link-graph/parse-wikilink.ts
@@ -29,7 +29,7 @@
import { basename } from "node:path";
-import { ANCHORED_WIKILINK_RE, RICH_WIKILINK_RE } from "../wikilink.ts";
+import { ANCHORED_WIKILINK_RE, RICH_WIKILINK_RE, maskCodeRegions } from "../wikilink.ts";
/**
* Local regex for the rich extractor. Captures the full bracket
@@ -41,13 +41,6 @@ import { ANCHORED_WIKILINK_RE, RICH_WIKILINK_RE } from "../wikilink.ts";
* non-newline so a link that spans lines is not silently joined.
*/
-/**
- * Local mask for fenced and inline code spans. Mirrors the constant
- * used by `extractWikilinks` in `vault.ts` so a wikilink that lives
- * inside a code block does not pollute the index.
- */
-const CODE_BLOCK_RE = /```[\s\S]*?```|`[^`]+`/g;
-
/**
* Inclusive list of file-extension suffixes (lower-case) that
* disqualify a wikilink from the backlink index. Mirrors the media
@@ -182,7 +175,7 @@ export function parseWikilinkRich(value: string): WikilinkParse {
* input yields an empty array.
*/
export function extractWikilinkRichBodies(content: string): ReadonlyArray<string> {
- const masked = content.replace(CODE_BLOCK_RE, " ");
+ const masked = maskCodeRegions(content, () => " ");
const out: string[] = [];
for (const m of masked.matchAll(RICH_WIKILINK_RE)) {
const body = m[1]!;
diff --git a/src/core/brain/link-graph/repair-lane.ts b/src/core/brain/link-graph/repair-lane.ts
index a8ca3ac5..01da4b23 100644
--- a/src/core/brain/link-graph/repair-lane.ts
+++ b/src/core/brain/link-graph/repair-lane.ts
@@ -43,6 +43,7 @@ import { resolveUniqueMatch } from "../../graph/unique-match.ts";
import { assertVaultIdentityForWrite } from "../vault-identity.ts";
import { scaffoldStub } from "../notes/scaffold-stub.ts";
import { MAINTENANCE_LANE_REACH } from "../../graph/transport-reach.ts";
+import { maskCodeRegions } from "../wikilink.ts";
/** Identity-strength tiers, strongest first. `inferred` is opt-in. */
export const IDENTITY_STRENGTH = Object.freeze({
@@ -401,7 +402,6 @@ const CONFIDENCE_CEILING = 0.85;
*/
export const HUB_CANDIDATE_CONFIDENCE = 1.0;
-const CODE_SPAN_RE = /```[\s\S]*?```|`[^`]+`/g;
const WIKILINK_SPAN_RE = /\[\[[^\]\n]+\]\]/g;
const WORD_CHAR_RE = /[\p{L}\p{N}]/u;
@@ -415,9 +415,9 @@ function pairId(source: string, target: string): string {
/** Mask wikilink and code spans so a mention inside them is not counted. */
function maskSpans(text: string): string {
- return text
- .replace(CODE_SPAN_RE, (s) => " ".repeat(s.length))
- .replace(WIKILINK_SPAN_RE, (s) => " ".repeat(s.length));
+ return maskCodeRegions(text, (s) => " ".repeat(s.length)).replace(WIKILINK_SPAN_RE, (s) =>
+ " ".repeat(s.length),
+ );
}
function isWordEdge(ch: string | undefined): boolean {
diff --git a/src/core/brain/link-graph/unlinked-mentions.ts b/src/core/brain/link-graph/unlinked-mentions.ts
index 4cee06a4..70999e15 100644
--- a/src/core/brain/link-graph/unlinked-mentions.ts
+++ b/src/core/brain/link-graph/unlinked-mentions.ts
@@ -25,6 +25,7 @@ import { parseFrontmatter } from "../../vault.ts";
import { ownerScopeView } from "../owner-scope-view.ts";
import { UNFILTERED_ARTIFACT_REFS, type ArtifactRefView } from "../artifact-ref-view.ts";
import { brainDirs } from "../paths.ts";
+import { maskCodeRegions } from "../wikilink.ts";
/** One unlinked-mention occurrence. */
export interface MentionRef {
@@ -72,7 +73,6 @@ const MIN_TERM_CODEPOINTS = 2;
const MASK_CHAR = " ";
const BRACKET_RE = /\[\[[^\]\n]+\]\]/g;
-const CODE_BLOCK_RE = /```[\s\S]*?```|`[^`]+`/g;
const WORD_CHAR_RE = /[\p{L}\p{N}]/u;
/**
@@ -195,7 +195,9 @@ function scanBody(
// ignored. The mask preserves character offsets so line/column
// bookkeeping stays exact - replace each masked region with a
// run of MASK_CHAR of identical length.
- const masked = maskRegions(body, [BRACKET_RE, CODE_BLOCK_RE]);
+ const masked = maskCodeRegions(maskRegions(body, [BRACKET_RE]), (s) =>
+ MASK_CHAR.repeat(s.length),
+ );
const lines = masked.split("\n");
const originalLines = body.split("\n");
diff --git a/src/core/brain/note-history.ts b/src/core/brain/note-history.ts
index 789567b2..85a1070c 100644
--- a/src/core/brain/note-history.ts
+++ b/src/core/brain/note-history.ts
@@ -17,6 +17,8 @@
* returned for a path with no history.
*/
+import { relative, resolve, sep } from "node:path";
+
import { readCommits } from "./git/reader.ts";
import type { GitCommit } from "./git/reader.ts";
@@ -52,6 +54,14 @@ export interface DecomposeNoteHistoryOptions {
readonly repoPath?: string;
/** Bound the walk to the newest N commits touching the path. */
readonly maxCount?: number;
+ /**
+ * May the caller read the page at this vault-relative POSIX path? Asked
+ * before the history is read. A refused path answers exactly as a path
+ * no commit touches, because a page the caller may not read must not
+ * disclose its existence or edit timeline. Absent (the CLI) refuses
+ * nothing.
+ */
+ readonly include?: (rel: string) => boolean;
}
const DEFAULT_GAP_HOURS = 72;
@@ -74,29 +84,19 @@ export function decomposeNoteHistory(
}
const repoPath = opts.repoPath ?? vault;
+ if (opts.include !== undefined && !opts.include(vaultRelativePosix(vault, path))) {
+ // The repository's own availability is asked without the path, so the
+ // answer is the one a never-written path gets in this same vault.
+ const available = readCommits(repoPath, { maxCount: 1 }) !== null;
+ return available ? noCommits(path) : noHistory(path);
+ }
const commits = readCommits(repoPath, {
path,
...(opts.maxCount !== undefined ? { maxCount: opts.maxCount } : {}),
});
- if (commits === null) {
- return Object.freeze({
- notePath: path,
- available: false,
- reason: "no history available (not a git repository or git unavailable)",
- commitCount: 0,
- phases: Object.freeze([]),
- });
- }
- if (commits.length === 0) {
- return Object.freeze({
- notePath: path,
- available: true,
- reason: "no commits touch this path",
- commitCount: 0,
- phases: Object.freeze([]),
- });
- }
+ if (commits === null) return noHistory(path);
+ if (commits.length === 0) return noCommits(path);
const gapMs = Math.max(0, opts.gapHours ?? DEFAULT_GAP_HOURS) * HOUR_MS;
const phases = splitIntoPhases(commits, gapMs);
@@ -151,3 +151,28 @@ function parseMs(iso: string): number | null {
const ms = Date.parse(iso);
return Number.isNaN(ms) ? null : ms;
}
+
+function noHistory(notePath: string): NoteHistoryResult {
+ return Object.freeze({
+ notePath,
+ available: false,
+ reason: "no history available (not a git repository or git unavailable)",
+ commitCount: 0,
+ phases: Object.freeze([]),
+ });
+}
+
+function noCommits(notePath: string): NoteHistoryResult {
+ return Object.freeze({
+ notePath,
+ available: true,
+ reason: "no commits touch this path",
+ commitCount: 0,
+ phases: Object.freeze([]),
+ });
+}
+
+/** `path` as the vault-relative POSIX form the visibility predicate speaks. */
+function vaultRelativePosix(vault: string, path: string): string {
+ return relative(resolve(vault), resolve(vault, path)).split(sep).join("/");
+}
diff --git a/src/core/brain/notes/lifecycle.ts b/src/core/brain/notes/lifecycle.ts
index fcce634e..fe234c54 100644
--- a/src/core/brain/notes/lifecycle.ts
+++ b/src/core/brain/notes/lifecycle.ts
@@ -429,6 +429,20 @@ export interface NoteLifecycleInput {
readonly strict?: boolean;
/** Injected clock, forwarded to the recovery point so a run stays reproducible. */
readonly now?: Date;
+ /**
+ * May the caller read the subject note at this vault-relative path? One
+ * it may not read is refused as `source_missing`, the refusal a path
+ * with no note gets, before the note is stat-ed. The same test decides
+ * which files the reference report names and counts (`inboundFiles`,
+ * `filesScanned`, `filesRewritten`, `rewriteFailures`, the cascade
+ * lists and the recoverability derived from the rewrite): a file it
+ * rejects is reported as a vault without it would be, and a cascade
+ * never deletes it. The link rewrite of an applied relocation still
+ * edits it, so the readers who can see it are not left with links to a
+ * path that no longer exists. Absent, every file counts (the CLI, which
+ * reads the vault directly).
+ */
+ readonly readable?: (rel: string) => boolean;
}
export interface NoteLifecycleResult {
@@ -866,7 +880,7 @@ export async function noteLifecycle(
// alone let it through to `readFileSync`, which raised a bare `EISDIR`
// that left MCP as an internal fault. There is no note at that path,
// which is what `source_missing` says.
- const sourceStat = statOrNull(source.abs);
+ const sourceStat = input.readable?.(source.relPath) === false ? null : statOrNull(source.abs);
if (sourceStat === null || !sourceStat.isFile()) {
throw new NoteLifecycleError("source_missing", `no note file at ${source.relPath}`);
}
@@ -912,12 +926,17 @@ export async function noteLifecycle(
// nothing and is not counted. A relocation REWRITES its own file when
// the note names its own old path, so that file is one of the writes
// `--expect` guards and has to be in the number.
- const inboundFiles =
+ // A file the caller may not read is absent from every list and count
+ // below; see `NoteLifecycleInput.readable`.
+ const reported = (rel: string): boolean => input.readable?.(rel) ?? true;
+ const inboundFiles = (
action === NOTE_LIFECYCLE_ACTION.delete
? probe.matched.filter((rel) => rel !== source.relPath)
- : probe.matched;
+ : probe.matched
+ ).filter(reported);
+ const filesScanned = probe.files.filter(reported).length;
- const cascade = deleteLinked ? planCascade(vault, source.relPath, inboundFiles) : null;
+ const cascade = deleteLinked ? planCascade(vault, source.relPath, inboundFiles, reported) : null;
// The guard asserts what the operation will REMOVE. Without a cascade
// that is one file and the number worth guarding is the collateral -
@@ -999,7 +1018,7 @@ export async function noteLifecycle(
true,
gated.recoverability,
snapshot,
- probe.files.length,
+ filesScanned,
inboundFiles,
0,
[],
@@ -1029,8 +1048,8 @@ export async function noteLifecycle(
apply: true,
neverRewrite: APPEND_ONLY_REL,
});
- rewritten = pass.rewritten;
- rewriteFailures = pass.failed;
+ rewritten = pass.rewritten.filter(reported);
+ rewriteFailures = pass.failed.filter((failure) => reported(failure.path));
}
// The MOVE destroys nothing - the bytes are reachable under the
@@ -1065,7 +1084,7 @@ export async function noteLifecycle(
apply,
recoverability,
snapshot,
- probe.files.length,
+ filesScanned,
inboundFiles,
rewritten.length,
retargets.flatMap((r) => (r.to === undefined ? [] : [{ from: r.from, to: r.to }])),
@@ -1109,8 +1128,11 @@ function planCascade(
vault: string,
sourceRel: string,
inboundFiles: ReadonlyArray<string>,
+ readable: (rel: string) => boolean,
): NoteDeleteCascade {
- const traced = traceNoteDerivations(vault, sourceRel);
+ // A page the caller may not read is neither deleted nor reported, as in
+ // a vault without it.
+ const traced = traceNoteDerivations(vault, sourceRel).filter((entry) => readable(entry.path));
const derived = traced
.filter((entry) => entry.deletable && entry.structuredProvenance)
.map((entry) => entry.path)
diff --git a/src/core/brain/notes/revert.ts b/src/core/brain/notes/revert.ts
index 97feb011..cc5ca420 100644
--- a/src/core/brain/notes/revert.ts
+++ b/src/core/brain/notes/revert.ts
@@ -348,6 +348,19 @@ function hashOf(current: CurrentBytes): string {
return current.kind === "absent" ? NOTE_WRITE_NO_PRIOR : NOTE_REVERT_HASH_UNREADABLE;
}
+/**
+ * The digest that seals one plan: the selector with absent fields dropped
+ * and the entries, canonically serialised. Exported so a surface that
+ * shows only some of a plan's entries can seal exactly what it shows,
+ * spelled the same way the full plan is.
+ */
+export function noteRevertDigest(
+ selector: NoteRevertSelector,
+ entries: ReadonlyArray<NoteRevertEntry>,
+): string {
+ return sha256Hex(canonicalJson({ selector: normaliseSelector(selector), entries }));
+}
+
/** The selector with absent fields dropped, so the digest is stable. */
function normaliseSelector(selector: NoteRevertSelector): NoteRevertSelector {
return {
@@ -578,7 +591,7 @@ export function planNoteRevert(
return {
selector: normalised,
entries,
- digest: sha256Hex(canonicalJson({ selector: normalised, entries })),
+ digest: noteRevertDigest(normalised, entries),
planned_at: isoSecond(opts.now ?? new Date()),
warnings: universeRead.warnings,
};
diff --git a/src/core/brain/operator-snapshot.ts b/src/core/brain/operator-snapshot.ts
index 2ee43c0d..054694e1 100644
--- a/src/core/brain/operator-snapshot.ts
+++ b/src/core/brain/operator-snapshot.ts
@@ -28,6 +28,7 @@ import {
type SelfHealUpgradeFailure,
} from "../maintenance/self-heal-upgrade-state.ts";
import { runHygieneScan } from "./hygiene/scan.ts";
+import type { HygieneFinding } from "./hygiene/types.ts";
import { brainConfigPath, brainDirs } from "./paths.ts";
import { loadTemporalConfigSafe } from "./policy.ts";
import { listProfiles } from "./portability/profiles.ts";
@@ -98,6 +99,20 @@ export interface BuildOperatorSnapshotOptions {
readonly configPath?: string;
/** Wall clock for stale/review scans. Defaults to `new Date()`. */
readonly now?: Date;
+ /**
+ * May the caller read the vault file at this vault-relative path? Handed
+ * to the hygiene scan (see `HygieneDetectorContext.readable`); a local
+ * caller passes nothing.
+ */
+ readonly readable?: (rel: string) => boolean;
+ /**
+ * Does the caller see this finding? The hygiene problem line counts only
+ * the findings it keeps, so the count matches what `brain_hygiene`
+ * would list for the same caller rather than the whole vault's: a count
+ * over the unfiltered set would move with a page the caller may not
+ * read. A local caller passes nothing.
+ */
+ readonly keepFinding?: (finding: HygieneFinding) => boolean;
}
/**
@@ -161,9 +176,15 @@ export async function buildOperatorSnapshot(
// --- Hygiene ---
try {
- const hy = runHygieneScan(vault, { now });
- if (hy.findings.length > 0) {
- problem("hygiene-findings", `${hy.findings.length} hygiene finding(s)`);
+ const hy = runHygieneScan(vault, {
+ now,
+ ...(opts.readable !== undefined ? { readable: opts.readable } : {}),
+ });
+ const keepFinding = opts.keepFinding;
+ const findings =
+ keepFinding === undefined ? hy.findings : hy.findings.filter((f) => keepFinding(f));
+ if (findings.length > 0) {
+ problem("hygiene-findings", `${findings.length} hygiene finding(s)`);
}
} catch {
problem("hygiene-findings", "hygiene scan failed to run");
diff --git a/src/core/brain/page-dedup.ts b/src/core/brain/page-dedup.ts
index f5dc7eba..07f7c27f 100644
--- a/src/core/brain/page-dedup.ts
+++ b/src/core/brain/page-dedup.ts
@@ -30,6 +30,7 @@ import { compositeScopeKey, scopeFromFrontmatter } from "../scope-key.ts";
import { matchScope, mayDescend, resolveVaultScope } from "../vault-scope/index.ts";
import { pathCovers, type VaultScopeRules } from "../vault-scope/defaults.ts";
import { brainDirs, BRAIN_ROOT_REL } from "./paths.ts";
+import { codeRegions } from "./wikilink.ts";
import {
isMergeResolved,
mergePointerLookup,
@@ -278,20 +279,6 @@ const RETARGET_SKIP_DIRS: ReadonlySet<string> = new Set([
"node_modules",
]);
-/**
- * Fenced blocks (``` / ~~~, closing on the same run length) and inline
- * code spans, as one alternation.
- *
- * A `[[Projects/Old]]` inside either is a QUOTATION of a link - a
- * tutorial showing the syntax, a design note quoting a vault's contents,
- * a README explaining how a rename behaves - and rewriting it edits
- * documentation into describing a vault that never existed. The
- * precedent is `link-graph/format-wikilink.ts`, which has masked code
- * regions since the wikilink formatter shipped; it matters more here now
- * that this walk's root reaches every user note rather than `Brain/`.
- */
-const CODE_REGION_RE = /(`{3,}|~{3,})[\s\S]*?\1|`[^`]+`/g;
-
/** One stretch of a document, and whether it is code. */
interface DocumentSegment {
readonly text: string;
@@ -303,14 +290,22 @@ interface DocumentSegment {
* every `text` back together reproduces `raw` byte for byte, which is
* what lets the rewrite below transform the prose halves in place
* without needing a second opinion about where the code was.
+ *
+ * Code is a fenced block or an inline code span, the regions every link
+ * reader masks ({@link codeRegions}). A `[[Projects/Old]]` inside
+ * either is a QUOTATION of a link - a tutorial showing the syntax, a
+ * design note quoting a vault's contents, a README explaining how a
+ * rename behaves - and rewriting it edits documentation into describing a
+ * vault that never existed. It matters here because this walk's root
+ * reaches every user note rather than `Brain/`.
*/
function segmentCode(raw: string): DocumentSegment[] {
const out: DocumentSegment[] = [];
let last = 0;
- for (const m of raw.matchAll(CODE_REGION_RE)) {
- if (m.index > last) out.push({ text: raw.slice(last, m.index), code: false });
- out.push({ text: m[0], code: true });
- last = m.index + m[0].length;
+ for (const [start, end] of codeRegions(raw)) {
+ if (start > last) out.push({ text: raw.slice(last, start), code: false });
+ out.push({ text: raw.slice(start, end), code: true });
+ last = end;
}
out.push({ text: raw.slice(last), code: false });
return out;
diff --git a/src/core/brain/procedural-memory.ts b/src/core/brain/procedural-memory.ts
index 87e36d8a..80759a4f 100644
--- a/src/core/brain/procedural-memory.ts
+++ b/src/core/brain/procedural-memory.ts
@@ -20,6 +20,13 @@ interface UsageRecord {
export interface ProceduralReconcileOptions {
readonly roots: ReadonlyArray<string>;
+ /**
+ * May the caller read the page at this vault-relative path? The index
+ * is written in full either way; only the counts this call reports are
+ * taken over the pages the caller may read, so they match the counts a
+ * vault without the other pages would give. Absent, every page counts.
+ */
+ readonly include?: (rel: string) => boolean;
}
export interface ProceduralReconcileResult {
@@ -48,10 +55,13 @@ export function reconcileProceduralMemory(
});
const nextById = new Map(next.map((entry) => [entry.id, entry] as const));
+ const counted = (entry: ProceduralMemoryEntry): boolean =>
+ opts.include?.(entry.sourcePath) ?? true;
let added = 0;
let updated = 0;
for (const entry of next) {
+ if (!counted(entry)) continue;
const old = prev.get(entry.id);
if (!old) {
added++;
@@ -63,14 +73,14 @@ export function reconcileProceduralMemory(
}
let removed = 0;
- for (const id of prev.keys()) {
- if (!nextById.has(id)) removed++;
+ for (const [id, old] of prev) {
+ if (!nextById.has(id) && counted(old)) removed++;
}
writeIndex(vault, next);
const graph = rebuildProceduralGraph(vault);
rebuildProceduralHints(vault, { graph });
- return { total: next.length, added, updated, removed };
+ return { total: next.filter(counted).length, added, updated, removed };
}
export function listProceduralMemory(vault: string): ReadonlyArray<ProceduralMemoryEntry> {
diff --git a/src/core/brain/provenance/capture-scope.ts b/src/core/brain/provenance/capture-scope.ts
new file mode 100644
index 00000000..3624297e
--- /dev/null
+++ b/src/core/brain/provenance/capture-scope.ts
@@ -0,0 +1,264 @@
+/**
+ * How much of its source a page actually holds.
+ *
+ * A distillation, an ingest summary or a research report presents itself as a
+ * record of a source. Whether the vault can still show the source's bytes is
+ * a separate fact, and before this module nothing stated it: a page whose
+ * only evidence is a URL looked like one backed by a file the vault holds.
+ *
+ * Three members, decided STRUCTURALLY and never inferred from prose:
+ *
+ * - `full-local` - the identity is vault-shaped and resolves to a readable
+ * file. It is the trusted verdict of {@link classifySourceTrust}, so the
+ * trust lane and the capture scope never disagree on an identity as
+ * written; {@link classifyCaptureScope} also accepts the Obsidian
+ * spelling of a note without its extension (`[[notes/meeting]]`).
+ * - `url-only` - the identity carries a URI scheme or an authority shape,
+ * or is vault-shaped with nothing behind it: the untrusted verdict.
+ * - `bounded-local` - a `url-only` source whose page stores a verbatim
+ * excerpt the same writer captured, in the marked block rendered by
+ * {@link renderExcerptSection}, together with its digest. A summary is a
+ * paraphrase, not a capture, so nothing else earns this member.
+ *
+ * Writers stamp the scope only when it is not `full-local`, so every page
+ * backed by a vault file stays byte-identical to what it was before this
+ * vocabulary existed. Absence therefore means "full-local, or written before
+ * the stamp existed", and a reader resolves it by re-deriving the scope from
+ * the identity, never by assuming.
+ */
+
+import { createHash } from "node:crypto";
+import { posix } from "node:path";
+
+import type { FrontmatterMap } from "../../types.ts";
+import {
+ classifySourceTrust,
+ normalizeSourceIdentity,
+ vaultShapedIdentity,
+} from "../intake/source-trust.ts";
+import { INTAKE_TRUST, type IntakeTrust } from "../trust/untrusted-provenance.ts";
+
+/** The closed vocabulary. See the module docblock for what each member proves. */
+export const CAPTURE_SCOPE = Object.freeze({
+ /** The vault holds the source file. */
+ fullLocal: "full-local",
+ /** The vault holds a verbatim, digested excerpt of a non-local source. */
+ boundedLocal: "bounded-local",
+ /** The vault holds nothing of the source but its locator. */
+ urlOnly: "url-only",
+} as const);
+
+/** Closed union over {@link CAPTURE_SCOPE}. */
+export type CaptureScope = (typeof CAPTURE_SCOPE)[keyof typeof CAPTURE_SCOPE];
+
+/** Membership list, most-captured first. */
+export const CAPTURE_SCOPES: ReadonlyArray<CaptureScope> = Object.freeze([
+ CAPTURE_SCOPE.fullLocal,
+ CAPTURE_SCOPE.boundedLocal,
+ CAPTURE_SCOPE.urlOnly,
+]);
+
+/** Narrow a scope read back off a page's frontmatter. */
+export function isCaptureScope(value: unknown): value is CaptureScope {
+ return typeof value === "string" && (CAPTURE_SCOPES as ReadonlyArray<string>).includes(value);
+}
+
+/** Frontmatter key of a single-source page's scope. */
+export const CAPTURE_SCOPE_KEY = "capture_scope";
+
+/** Frontmatter key of a multi-source page's scopes, parallel to its sources. */
+export const CAPTURE_SCOPES_KEY = "capture_scopes";
+
+/** Frontmatter key of the stored excerpt's digest on a `bounded-local` page. */
+export const EXCERPT_HASH_KEY = "excerpt_hash";
+
+/**
+ * Largest excerpt a page will store, in UTF-8 bytes. An excerpt is the passage
+ * a writer read and cites, not a mirror of the source; 64 KiB holds many pages
+ * of text while keeping a page a page. A larger capture is refused by name
+ * rather than truncated, because a truncated excerpt would no longer be the
+ * verbatim passage its digest claims.
+ */
+export const CAPTURE_EXCERPT_MAX_BYTES = 65_536;
+
+/** The heading the excerpt section sits under. */
+export const CAPTURE_EXCERPT_HEADING = "## Excerpt";
+
+/**
+ * Info string of the excerpt's fenced block. A fence keeps the bytes verbatim
+ * and keeps wikilink parsing out of them.
+ */
+export const CAPTURE_EXCERPT_FENCE_INFO = "excerpt";
+
+/** The fence character; a backtick fence admits any info string. */
+const FENCE_CHAR = "`";
+
+/** The shortest fence CommonMark recognises. */
+const MIN_FENCE_LENGTH = 3;
+
+/** Every run of the fence character, to size a fence that contains them all. */
+const FENCE_CHAR_RUN_RE = /`+/g;
+
+/** Line separator of a rendered section. */
+const NEWLINE = "\n";
+
+/** The extension a note identity written the Obsidian way leaves off. */
+const NOTE_EXTENSION = ".md";
+
+/** What an excerpt with no text is made of: controls, format characters, whitespace. */
+const NON_TEXT_RE = /[\p{Cc}\p{Cf}\s]/gu;
+
+/** NUL, which a Markdown page never holds. */
+const NUL = "\u0000";
+
+/** The scope a trust verdict stands for: the two can never disagree. */
+export function captureScopeForTrust(trust: IntakeTrust): CaptureScope {
+ return trust === INTAKE_TRUST.trusted ? CAPTURE_SCOPE.fullLocal : CAPTURE_SCOPE.urlOnly;
+}
+
+/** A source identity's scope, and the vault file behind it when there is one. */
+export interface CaptureScopeResolution {
+ readonly scope: CaptureScope;
+ /**
+ * Vault-relative POSIX path of the backing file, every `.` and `..` segment
+ * resolved; `null` exactly when `url-only`.
+ */
+ readonly backing: string | null;
+}
+
+/**
+ * The current scope of a source identity and the file that backs it, from its
+ * shape and one `stat` (two for an identity with no extension, retried once as
+ * a `.md` note, the way Obsidian links a note). Never reads the source's
+ * bytes, so a sweep over many pages stays cheap. `bounded-local` is never
+ * returned: it is a property of a page, not of an identity. A refused `stat`
+ * propagates as the {@link classifySourceTrust} refusal.
+ */
+export function resolveCaptureScope(vault: string, identity: string): CaptureScopeResolution {
+ const target = normalizeSourceIdentity(identity);
+ if (classifySourceTrust(vault, identity) === INTAKE_TRUST.trusted) {
+ return { scope: CAPTURE_SCOPE.fullLocal, backing: vaultShapedIdentity(vault, identity) };
+ }
+ if (target.length > 0 && posix.extname(target) === "") {
+ const note = `${target}${NOTE_EXTENSION}`;
+ if (classifySourceTrust(vault, note) === INTAKE_TRUST.trusted) {
+ return { scope: CAPTURE_SCOPE.fullLocal, backing: vaultShapedIdentity(vault, note) };
+ }
+ }
+ return { scope: CAPTURE_SCOPE.urlOnly, backing: null };
+}
+
+/** The scope half of {@link resolveCaptureScope}. */
+export function classifyCaptureScope(vault: string, identity: string): CaptureScope {
+ return resolveCaptureScope(vault, identity).scope;
+}
+
+/** The frontmatter stamp of a single-source page: nothing for `full-local`. */
+export function captureScopeFrontmatter(scope: CaptureScope): FrontmatterMap {
+ return scope === CAPTURE_SCOPE.fullLocal ? {} : { [CAPTURE_SCOPE_KEY]: scope };
+}
+
+/**
+ * The frontmatter stamp of a multi-source page: nothing when every source is
+ * `full-local`, otherwise the whole list so positions stay aligned with the
+ * page's sources.
+ */
+export function captureScopesFrontmatter(scopes: ReadonlyArray<CaptureScope>): FrontmatterMap {
+ return scopes.every((scope) => scope === CAPTURE_SCOPE.fullLocal)
+ ? {}
+ : { [CAPTURE_SCOPES_KEY]: [...scopes] };
+}
+
+/** SHA-256 hex over the UTF-8 bytes of the excerpt exactly as given. */
+export function excerptDigest(excerpt: string): string {
+ return createHash("sha256").update(excerpt, "utf8").digest("hex");
+}
+
+/** A backtick fence longer than every backtick run in `text`. */
+function fenceFor(text: string): string {
+ let longest = 0;
+ for (const run of text.matchAll(FENCE_CHAR_RUN_RE)) longest = Math.max(longest, run[0].length);
+ return FENCE_CHAR.repeat(Math.max(MIN_FENCE_LENGTH, longest + 1));
+}
+
+/**
+ * The excerpt section: the heading, a blank line, then a fenced block whose
+ * fence is longer than any backtick run in the excerpt. One newline always
+ * separates the excerpt from the closing fence, so an excerpt that ends in a
+ * newline keeps it on the way back through {@link readExcerptSection}.
+ */
+export function renderExcerptSection(excerpt: string): string {
+ const fence = fenceFor(excerpt);
+ return (
+ `${CAPTURE_EXCERPT_HEADING}${NEWLINE}${NEWLINE}` +
+ `${fence}${CAPTURE_EXCERPT_FENCE_INFO}${NEWLINE}` +
+ `${excerpt}${NEWLINE}` +
+ `${fence}${NEWLINE}`
+ );
+}
+
+/** The opening of a rendered section: heading, blank line, fence, info string. */
+const EXCERPT_OPENING_RE = new RegExp(
+ `(?:^|${NEWLINE})${CAPTURE_EXCERPT_HEADING}${NEWLINE}${NEWLINE}` +
+ `(${FENCE_CHAR}{${MIN_FENCE_LENGTH},})${CAPTURE_EXCERPT_FENCE_INFO}${NEWLINE}`,
+);
+
+/**
+ * The inverse of {@link renderExcerptSection}. `null` when the section is
+ * absent or does not have the rendered shape (no fence, no closing fence),
+ * because a damaged section is not a capture.
+ */
+export function readExcerptSection(body: string): string | null {
+ const opening = EXCERPT_OPENING_RE.exec(body);
+ if (opening === null) return null;
+ const fence = opening[1] ?? "";
+ const start = opening.index + opening[0].length;
+ const closing = `${NEWLINE}${fence}`;
+ // The excerpt may be empty, in which case the closing fence follows the
+ // opening line directly; search from one character back to see it.
+ const at = body.indexOf(closing, start - NEWLINE.length);
+ if (at === -1) return null;
+ const after = body.slice(at + closing.length);
+ if (after.length > 0 && !after.startsWith(NEWLINE)) return null;
+ return at < start ? "" : body.slice(start, at);
+}
+
+/**
+ * An excerpt the caller supplied cannot be stored. The message carries
+ * identifiers and integers only, never excerpt text.
+ */
+export class CaptureExcerptError extends Error {
+ constructor(message: string) {
+ super(message);
+ this.name = "CaptureExcerptError";
+ }
+}
+
+/**
+ * Refuse an excerpt this page must not store: one for a source that is not
+ * `url-only` (a local file is already the evidence, and an excerpt beside it
+ * would be a second, weaker claim), an empty one (nothing but control,
+ * format or whitespace characters), one holding NUL, or one past
+ * {@link CAPTURE_EXCERPT_MAX_BYTES}.
+ */
+export function assertExcerptAdmissible(scope: CaptureScope, excerpt: string): void {
+ if (scope !== CAPTURE_SCOPE.urlOnly) {
+ throw new CaptureExcerptError(
+ `excerpt refused: the source is ${scope}; an excerpt is accepted only for a ` +
+ `${CAPTURE_SCOPE.urlOnly} source`,
+ );
+ }
+ if (excerpt.replace(NON_TEXT_RE, "").length === 0) {
+ throw new CaptureExcerptError("excerpt refused: the excerpt is empty");
+ }
+ if (excerpt.includes(NUL)) {
+ throw new CaptureExcerptError("excerpt refused: the excerpt contains NUL");
+ }
+ const bytes = Buffer.byteLength(excerpt, "utf8");
+ if (bytes > CAPTURE_EXCERPT_MAX_BYTES) {
+ throw new CaptureExcerptError(
+ `excerpt refused: the excerpt is ${bytes} bytes, past the ` +
+ `${CAPTURE_EXCERPT_MAX_BYTES}-byte ceiling`,
+ );
+ }
+}
diff --git a/src/core/brain/research/research.ts b/src/core/brain/research/research.ts
index 3022a120..2cf19215 100644
--- a/src/core/brain/research/research.ts
+++ b/src/core/brain/research/research.ts
@@ -32,6 +32,14 @@ import {
} from "../response-shape.ts";
import { assertVaultIdentityForWrite } from "../vault-identity.ts";
import { renderProvenanceSection, type Provenance } from "../provenance/provenance.ts";
+import {
+ CAPTURE_SCOPE,
+ captureScopesFrontmatter,
+ resolveCaptureScope,
+ type CaptureScope,
+ type CaptureScopeResolution,
+} from "../provenance/capture-scope.ts";
+import { SourceTrustError } from "../intake/source-trust.ts";
import {
ExternalFetchError,
createFetchTransport,
@@ -71,6 +79,14 @@ export interface ResearchReportInput {
export interface ResearchReportOptions {
readonly agent: string;
readonly now: Date;
+ /**
+ * May the caller read this vault-relative path? Supplied by a surface
+ * that answers at a reach narrower than the vault (the MCP tool at remote
+ * reach); a local caller passes nothing. A source backed by a file the
+ * caller cannot read is reported `url-only`, the same answer as an absent
+ * file, in the result and on the page.
+ */
+ readonly readable?: (rel: string) => boolean;
}
export interface ResearchReportResult {
@@ -78,6 +94,12 @@ export interface ResearchReportResult {
readonly reportPath: string;
readonly created: boolean;
readonly findingCount: number;
+ /**
+ * The current capture scope of every consulted source, parallel to
+ * `input.sources`. A report stores no excerpt, so each member is
+ * `full-local` (a vault file) or `url-only` (no local bytes).
+ */
+ readonly captureScopes: ReadonlyArray<CaptureScope>;
}
/** A research report failed validation; nothing was written. */
@@ -114,6 +136,9 @@ export function parseResearchReportInput(payload: unknown): ResearchReportInput
};
}
+/** Line breaks a title may not hold: it is rendered as the page's one heading line. */
+const TITLE_LINE_BREAK_RE = /[\r\n]/;
+
/** Wrap a bare source identifier in a wikilink; leave an existing one as-is. */
function asWikilink(source: string): string {
const trimmed = source.trim();
@@ -124,6 +149,9 @@ function validate(input: ResearchReportInput): void {
if (input.title.trim().length === 0) {
throw new ResearchValidationError("report title must not be empty");
}
+ if (TITLE_LINE_BREAK_RE.test(input.title)) {
+ throw new ResearchValidationError("report title spans more than one line; a title is one line");
+ }
if (input.sources.length === 0) {
throw new ResearchValidationError("a report must consult at least one source");
}
@@ -150,6 +178,39 @@ function validate(input: ResearchReportInput): void {
}
}
+/**
+ * The capture scope of one consulted source as the caller may know it. A
+ * `full-local` source whose backing file the caller cannot read answers
+ * `url-only`, so the scope never tells a narrower caller that a hidden page
+ * exists. The backing file is the identity's own path, or its `.md` note
+ * when the identity names no extension (see `resolveCaptureScope`).
+ *
+ * A source whose `stat` the filesystem refuses is inside this vault but
+ * unreadable. The scope is an annotation, so it never aborts the report, and
+ * it answers `url-only`: the vault can show none of the file's bytes, and the
+ * answer is the same at every reach, so it names nothing about the path.
+ * The `capture-scope` hygiene detector deliberately treats the same refused
+ * stat as backing, unless the caller may not read the file: this stamp is a
+ * snapshot of what the report could show, while the detector avoids warning
+ * about a file that may well exist.
+ */
+function captureScopeAtReach(
+ vault: string,
+ source: string,
+ readable: ((rel: string) => boolean) | undefined,
+): CaptureScope {
+ let resolved: CaptureScopeResolution;
+ try {
+ resolved = resolveCaptureScope(vault, source);
+ } catch (err) {
+ if (!(err instanceof SourceTrustError)) throw err;
+ return CAPTURE_SCOPE.urlOnly;
+ }
+ const { scope, backing } = resolved;
+ if (backing === null || readable === undefined) return scope;
+ return readable(backing) ? scope : CAPTURE_SCOPE.urlOnly;
+}
+
/**
* Write a dated, cited research report. Validates the citation contract first
* (throwing {@link ResearchValidationError} with no write on failure), then
@@ -178,6 +239,12 @@ export function writeResearchReport(
premises: [],
};
+ // Classified from the identity alone (no byte read), in the consulted
+ // order, so `capture_scopes[i]` always describes `sources[i]`.
+ const captureScopes = Object.freeze(
+ input.sources.map((source) => captureScopeAtReach(vault, source, opts.readable)),
+ );
+
const body = [
`# ${input.title.trim()}`,
["## Findings", "", ...findingLines].join("\n"),
@@ -190,6 +257,9 @@ export function writeResearchReport(
report_date: date,
provenance: provenance.level,
source_count: input.sources.length,
+ // Written only when some source is not a local file, so a report built
+ // from vault files alone stays byte-identical to before.
+ ...captureScopesFrontmatter(captureScopes),
created_at: stamp,
updated_at: stamp,
tags: ["brain", "brain/report"],
@@ -206,6 +276,7 @@ export function writeResearchReport(
reportPath: canonicalNotePath(relative(vault, absPath)),
created,
findingCount: input.findings.length,
+ captureScopes,
};
}
diff --git a/src/core/brain/source-cleanup.ts b/src/core/brain/source-cleanup.ts
index 30d68d01..a79ec928 100644
--- a/src/core/brain/source-cleanup.ts
+++ b/src/core/brain/source-cleanup.ts
@@ -69,6 +69,7 @@ import {
type RecoverabilityVerdict,
} from "./gates/recoverability.ts";
import { withDestructiveSnapshot } from "./snapshot-gate.ts";
+import { stringArrayField, wikilinkTarget } from "./source-links.ts";
import { BRAIN_SNAPSHOT_REASON } from "./types.ts";
import { assertVaultIdentityForWrite } from "./vault-identity.ts";
@@ -182,6 +183,13 @@ export interface DeleteBySourceOptions {
readonly now?: Date;
/** Agent identity recorded in the audit reason. */
readonly agent?: string;
+ /**
+ * May the caller read the original at this vault-relative path? Asked
+ * before the original is stat-ed, so one it may not read is planned,
+ * counted and left on disk exactly as an absent one would be. Absent
+ * includes every original (the CLI, which reads the vault directly).
+ */
+ readonly include?: (rel: string) => boolean;
}
/**
@@ -275,12 +283,6 @@ function stringField(meta: FrontmatterMap, key: string): string | null {
return typeof value === "string" && value.length > 0 ? value : null;
}
-function stringArrayField(meta: FrontmatterMap, key: string): ReadonlyArray<string> {
- const value = meta[key];
- if (!Array.isArray(value)) return [];
- return value.filter((item): item is string => typeof item === "string");
-}
-
/**
* A preference's evidence links. The writer stores the field under the
* managed `_evidenced_by` key (with a plain `evidenced_by` on legacy
@@ -291,12 +293,6 @@ function evidencedByLinks(meta: FrontmatterMap): ReadonlyArray<string> {
return managed.length > 0 ? managed : stringArrayField(meta, "evidenced_by");
}
-/** Strip a single enclosing `[[ … ]]` and any `|alias` / `#heading` tail. */
-function wikilinkTarget(raw: string): string {
- const m = /^\[\[([^\]|#]+)(?:[|#][^\]]*)?\]\]$/.exec(raw.trim());
- return m ? m[1]!.trim() : raw.trim();
-}
-
function classifyKind(vault: string, absPath: string): SourceCleanupKind {
const dirs = brainDirs(vault);
if (insideDir(absPath, join(vault, BRAIN_SOURCES_REL))) return "summary";
@@ -631,13 +627,21 @@ export function traceNoteDerivations(
* A source that lives inside `Brain/` (or that has no on-disk file — e.g. a
* URL identity) contributes no original.
*/
-function findOriginals(vault: string, canonical: string): string[] {
+function findOriginals(
+ vault: string,
+ canonical: string,
+ include: ((rel: string) => boolean) | undefined,
+): string[] {
let abs: string;
try {
abs = ensureInsideVault(join(vault, canonical), vault);
} catch {
return [];
}
+ // Before the stat: an original the caller may not read is answered as
+ // an absent one, so neither its existence nor its removal is decided by
+ // a caller that cannot see it.
+ if (include?.(canonical) === false) return [];
if (!existsSync(abs)) return [];
try {
if (!statSync(abs).isFile()) return [];
@@ -682,7 +686,7 @@ export function deleteBySource(
if (confirm) assertVaultIdentityForWrite(vault);
const { derived, mentions } = traceReferences(vault, identity);
- const originals = findOriginals(vault, canonical);
+ const originals = findOriginals(vault, canonical, opts.include);
const manifestKey = readManifest(vault).entries[canonical] !== undefined ? canonical : null;
const blastRadius = derived.length + mentions.length + originals.length;
diff --git a/src/core/brain/source-links.ts b/src/core/brain/source-links.ts
new file mode 100644
index 00000000..56697c01
--- /dev/null
+++ b/src/core/brain/source-links.ts
@@ -0,0 +1,122 @@
+/**
+ * Shared readers for the sources a Brain page cites.
+ *
+ * One implementation for every consumer that has to answer "which sources
+ * does this page rest on": the delete-by-source cleanup (`source-cleanup.ts`)
+ * and the capture-scope hygiene detector. Two readers would drift - a page
+ * the cleanup counts as derived from a source and the detector counts as
+ * citing nothing is exactly the kind of disagreement neither could explain.
+ *
+ * Structural only: frontmatter list members and wikilink targets. No prose
+ * is interpreted.
+ */
+
+import type { FrontmatterMap } from "../types.ts";
+
+/** Heading of the canonical provenance section (`provenance.ts`). */
+const SOURCES_HEADING_RE = /^##\s+Sources\s*$/;
+
+/** Any Markdown ATX heading: the line that ends a section. */
+const ANY_HEADING_RE = /^#{1,6}\s/;
+
+/** Every `[[ … ]]` wikilink on a line, captured whole. */
+const WIKILINK_RE = /\[\[[^\]]+\]\]/g;
+
+/** Line separator for LF and CRLF bodies alike. */
+const LINE_SPLIT_RE = /\r?\n/;
+
+/**
+ * Opening or closing CommonMark code fence: up to three spaces of indent,
+ * three or more backticks or tildes. Same rule as the block resolver
+ * (`distill/block-resolve.ts`), copied rather than imported so this reader
+ * does not depend on the distill module.
+ */
+const FENCE_RE = /^ {0,3}(`{3,}|~{3,})(.*)$/;
+
+/** True when `line` closes a fence opened by `opener`: same character, at least as long, nothing after it. */
+function closesFence(line: string, opener: string): boolean {
+ const m = FENCE_RE.exec(line);
+ return (
+ m !== null &&
+ m[1]![0] === opener[0] &&
+ m[1]!.length >= opener.length &&
+ m[2]!.trim().length === 0
+ );
+}
+
+/**
+ * Which lines sit inside a fenced code block, fence lines included. Only a
+ * closed fence is a fence: an opener with no closing line before the end of
+ * the body is plain text, as are the lines after it, so caller text that
+ * opens a fence and never closes it cannot hide the structure that follows.
+ * Two linear passes at most: the single pass that pairs fences, then the
+ * unmarking of the one span that can be left open.
+ */
+function fencedLineMask(lines: ReadonlyArray<string>): boolean[] {
+ const mask = Array.from({ length: lines.length }, () => false);
+ let fence: string | null = null;
+ let openedAt = 0;
+ for (const [i, line] of lines.entries()) {
+ if (fence !== null) {
+ mask[i] = true;
+ if (closesFence(line, fence)) fence = null;
+ continue;
+ }
+ const opened = FENCE_RE.exec(line);
+ if (opened !== null) {
+ fence = opened[1]!;
+ openedAt = i;
+ mask[i] = true;
+ }
+ }
+ if (fence !== null) mask.fill(false, openedAt);
+ return mask;
+}
+
+/** The string members of a frontmatter list field; anything else reads as empty. */
+export function stringArrayField(meta: FrontmatterMap, key: string): ReadonlyArray<string> {
+ const value = meta[key];
+ if (!Array.isArray(value)) return [];
+ return value.filter((item): item is string => typeof item === "string");
+}
+
+/** Strip a single enclosing `[[ … ]]` and any `|alias` / `#heading` tail. */
+export function wikilinkTarget(raw: string): string {
+ const m = /^\[\[([^\]|#]+)(?:[|#][^\]]*)?\]\]$/.exec(raw.trim());
+ return m ? m[1]!.trim() : raw.trim();
+}
+
+/**
+ * The wikilink targets listed under the page's LAST `## Sources` heading,
+ * in order, read up to the next heading of any level. Targets go through
+ * {@link wikilinkTarget}, the same rule the cleanup applies to a frontmatter
+ * `source` link. A body without the section cites nothing here.
+ *
+ * Lines inside fenced code blocks are content, not structure: a heading or
+ * link there neither opens, ends nor feeds the section. A fence that never
+ * closes is not a fence (see {@link fencedLineMask}). The last section
+ * wins because the writer renders the provenance section after everything
+ * else on the page, so an earlier `## Sources` line (quoted or planted in
+ * caller text) never stands in for it.
+ */
+export function sourcesSectionTargets(body: string): ReadonlyArray<string> {
+ let targets: string[] = [];
+ let inSection = false;
+ const lines = body.split(LINE_SPLIT_RE);
+ const fenced = fencedLineMask(lines);
+ for (const [i, line] of lines.entries()) {
+ if (fenced[i]) continue;
+ if (SOURCES_HEADING_RE.test(line)) {
+ inSection = true;
+ targets = [];
+ continue;
+ }
+ if (!inSection) continue;
+ if (ANY_HEADING_RE.test(line)) {
+ inSection = false;
+ continue;
+ }
+ for (const match of line.matchAll(WIKILINK_RE)) targets.push(wikilinkTarget(match[0]));
+ }
+ return targets;
+}
diff --git a/src/core/brain/wikilink.ts b/src/core/brain/wikilink.ts
index 3e1a8800..1bee9962 100644
--- a/src/core/brain/wikilink.ts
+++ b/src/core/brain/wikilink.ts
@@ -79,6 +79,123 @@ export const ANCHORED_WIKILINK_RE = /^\[\[([^\]]+)\]\]/;
/** Whole-string wikilink form; body in one capture. */
export const EXACT_WIKILINK_RE = /^\[\[([^\]]+)\]\]$/;
+/** The two characters a fence is built from. */
+const BACKTICK = "`";
+const TILDE = "~";
+
+/** The shortest run of fence characters that opens or closes a fence. */
+const FENCE_MIN_RUN = 3;
+
+/** A whole run of three or more fence characters: `[start, start + len)`. */
+interface FenceRun {
+ readonly start: number;
+ readonly len: number;
+}
+
+/**
+ * Every whole run of three or more `ch`, left to right, and for each the
+ * run that closes it: the next run of the same character at least as long
+ * (next-greater-or-equal, one monotonic-stack pass from the right).
+ */
+function fenceClosers(text: string, ch: string): Map<number, FenceRun> {
+ const runs: FenceRun[] = [];
+ let i = text.indexOf(ch);
+ while (i !== -1) {
+ let end = i + 1;
+ while (end < text.length && text[end] === ch) end++;
+ if (end - i >= FENCE_MIN_RUN) runs.push({ start: i, len: end - i });
+ i = text.indexOf(ch, end);
+ }
+ const closers = new Map<number, FenceRun>();
+ const stack: FenceRun[] = [];
+ for (let r = runs.length - 1; r >= 0; r--) {
+ const run = runs[r]!;
+ while (stack.length > 0 && stack.at(-1)!.len < run.len) stack.pop();
+ const closer = stack.at(-1);
+ if (closer !== undefined) closers.set(run.start, closer);
+ stack.push(run);
+ }
+ return closers;
+}
+
+/**
+ * Every whole run of one or two backticks, left to right, and for each the
+ * run that closes it as an inline code span: the next run of exactly the
+ * same length (the CommonMark rule). One pass from the right.
+ */
+function inlineClosers(text: string): Map<number, FenceRun> {
+ const runs: FenceRun[] = [];
+ let i = text.indexOf(BACKTICK);
+ while (i !== -1) {
+ let end = i + 1;
+ while (end < text.length && text[end] === BACKTICK) end++;
+ if (end - i < FENCE_MIN_RUN) runs.push({ start: i, len: end - i });
+ i = text.indexOf(BACKTICK, end);
+ }
+ const closers = new Map<number, FenceRun>();
+ const nextOfLength = new Map<number, FenceRun>();
+ for (let r = runs.length - 1; r >= 0; r--) {
+ const run = runs[r]!;
+ const closer = nextOfLength.get(run.len);
+ if (closer !== undefined) closers.set(run.start, closer);
+ nextOfLength.set(run.len, run);
+ }
+ return closers;
+}
+
+/**
+ * Code regions a wikilink or mention scan masks out, as `[start, end)`
+ * pairs left to right: a fenced block or an inline code span. A fence is a
+ * whole run of three or more backticks or tildes, and it closes on the next
+ * whole run of the same character at least as long (the CommonMark rule),
+ * consumed whole: a longer fence holding a shorter run stays masked.
+ * Elsewhere a run of one or two backticks opens an inline span that closes
+ * on the next run of exactly its length. A run with no closer is not a
+ * region, and none of its marks opens one.
+ *
+ * One pass over the text plus one pass over the runs, so the cost is
+ * linear whatever the runs look like. Shared by every link and mention
+ * reader, so a stored excerpt never contributes links anywhere.
+ */
+export function codeRegions(text: string): Array<[number, number]> {
+ const closers = new Map([
+ ...fenceClosers(text, BACKTICK),
+ ...fenceClosers(text, TILDE),
+ ...inlineClosers(text),
+ ]);
+ const regions: Array<[number, number]> = [];
+ let i = 0;
+ while (i < text.length) {
+ const ch = text[i];
+ if (ch !== BACKTICK && ch !== TILDE) {
+ i++;
+ continue;
+ }
+ const closer = closers.get(i);
+ if (closer !== undefined) {
+ const end = closer.start + closer.len;
+ regions.push([i, end]);
+ i = end;
+ continue;
+ }
+ // An unclosed run is literal text as a whole: skipping it keeps its last
+ // mark from opening an inline span against an unrelated later backtick.
+ while (i < text.length && text[i] === ch) i++;
+ }
+ return regions;
+}
+
+/** `text` with every {@link codeRegions} region replaced by `fill(region)`. */
+export function maskCodeRegions(text: string, fill: (region: string) => string): string {
+ let out = "";
+ let last = 0;
+ for (const [start, end] of codeRegions(text)) {
+ out += text.slice(last, start) + fill(text.slice(start, end));
+ last = end;
+ }
+ return out + text.slice(last);
+}
+
/**
* Strip wikilink decoration off `value` and return the bare target id.
*
diff --git a/src/core/brain/write-batch.ts b/src/core/brain/write-batch.ts
index a9c37c41..c8ef7cda 100644
--- a/src/core/brain/write-batch.ts
+++ b/src/core/brain/write-batch.ts
@@ -392,6 +392,14 @@ export interface ApplyWriteBatchOptions {
* before any write.
*/
readonly requestId?: string;
+ /**
+ * May the caller read the existing note at this vault-relative path?
+ * An update or append whose target it may not read is refused as
+ * `target_missing`, the refusal a path with no note gets, before the
+ * note is read or any other check runs on it. Absent, every existing
+ * note is a target (the CLI, which reads the vault directly).
+ */
+ readonly readable?: (rel: string) => boolean;
}
/**
@@ -715,7 +723,7 @@ function projectUpdateNote(
);
}
const target = reserveNoteTarget(vault, op.path, index, noteTargets);
- const state = readExistingNote(target.abs, target.relPath, index);
+ const state = readExistingNote(target.abs, target.relPath, index, opts.readable);
// The one seam the blank-overwrite guard is wired at - it covers both
// callers, `brain_update_note` and `brain_write_batch`'s update op,
// because both project through here.
@@ -803,7 +811,7 @@ function projectAppendNote(
);
}
const target = reserveNoteTarget(vault, op.path, index, noteTargets);
- const state = readExistingNote(target.abs, target.relPath, index);
+ const state = readExistingNote(target.abs, target.relPath, index, opts.readable);
const appended = op.content.trim();
const body = state.body.length > 0 ? `${state.body}${APPEND_SEPARATOR}${appended}` : appended;
const contents = formatFrontmatter(state.frontmatter, body);
@@ -1089,8 +1097,16 @@ function refuseReadOnlyAttribute(abs: string, relPath: string, index: number): v
* Both notices come off the same parse, and both mean the same thing:
* part of the content this write would replace is unknown to it.
*/
-function readExistingNote(abs: string, relPath: string, index: number): ExistingNote {
- if (!existsSync(abs)) {
+function readExistingNote(
+ abs: string,
+ relPath: string,
+ index: number,
+ readable: ((rel: string) => boolean) | undefined,
+): ExistingNote {
+ // A note the caller may not read is answered as a missing one, asked
+ // first so no later check (read-only attribute, parse, reserved key)
+ // can tell the two apart.
+ if (readable?.(relPath) === false || !existsSync(abs)) {
throw new WriteBatchError(
"target_missing",
index,
diff --git a/src/core/search/result-filters.ts b/src/core/search/result-filters.ts
index 81507e85..a144ec8e 100644
--- a/src/core/search/result-filters.ts
+++ b/src/core/search/result-filters.ts
@@ -9,6 +9,7 @@ import { statSync } from "node:fs";
import { join } from "node:path";
import { parseFrontmatterWithNotices } from "../vault.ts";
+import { pathIsInside } from "../path-safety.ts";
import { BRAIN_STATE_REL } from "../brain/paths.ts";
import { DEGRADATION_CODE } from "../integrity/degradation.ts";
import type { FrontmatterMap } from "../types.ts";
@@ -358,6 +359,12 @@ export function applyVisibilityScope(
* and writing a public one at its path made the stale private chunks
* pass this check - the file said "public" - and a remote search served
* the private body. Omitted by surfaces that serve the file itself.
+ *
+ * A path that resolves OUTSIDE the vault answers `false` at every reach,
+ * before any read: it names no page anyone could be shown, and some
+ * callers hand this function a path the caller of a tool chose, which
+ * must never make the server open a file beside the vault (a named pipe
+ * there would block the read for good).
*/
export function isPathReadableAtReach(
vault: string,
@@ -366,6 +373,7 @@ export function isPathReadableAtReach(
frontmatterCache: FrontmatterCache,
indexedTags: ReadonlyArray<string> = [],
): boolean {
+ if (!pathIsInside(join(vault, path), vault)) return false;
const entry = readCachedFrontmatterEntry(frontmatterCache, vault, path);
const tags = entry.unreadable ? UNMEASURABLE_VISIBILITY : pageVisibility(entry.meta);
return isRemotelyReadable(tags, reach) && isRemotelyReadable(indexedTags, reach);
diff --git a/src/core/search/visibility-surface-registry.ts b/src/core/search/visibility-surface-registry.ts
index 9ad5568f..ca8ef09f 100644
--- a/src/core/search/visibility-surface-registry.ts
+++ b/src/core/search/visibility-surface-registry.ts
@@ -252,11 +252,12 @@ export const VISIBILITY_SURFACE_REGISTRY: ReadonlyArray<VisibilitySurfaceEntry>
{
surface: "brain_idea_discovery",
kind: K.mcpTool,
- category: C.excluded,
+ category: C.covered,
reason:
- "discoverIdeas / ideaCandidates (core/brain/idea-discovery.ts) walk the vault via " +
- "readdirSync + parseFrontmatter to find orphan research notes, with no visibility check on " +
- "any candidate page.",
+ "discoverIdeas (core/brain/idea-discovery.ts) takes the handler's " +
+ "readableAtContextReach(ctx) as its include option and leaves a page the caller may not " +
+ "read at its reach out of the inbound-link walk and the candidates, before ranking and " +
+ "the cap, so a withheld research page is answered as an absent one.",
},
{
surface: "brain_dead_ends",
@@ -299,11 +300,62 @@ export const VISIBILITY_SURFACE_REGISTRY: ReadonlyArray<VisibilitySurfaceEntry>
{
surface: "brain_tiers",
kind: K.mcpTool,
+ category: C.covered,
+ reason:
+ "check keeps only the drift rows whose page passes the handler's " +
+ "readableAtContextReach(ctx), and restore and accept refuse any other page with the 'not " +
+ "indexed' error a page the index never saw gets, before the drift is read or anything is " +
+ "written.",
+ },
+ {
+ surface: "brain_doctor",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "the read-only report keeps every issue stream through the owner view ANDed with " +
+ "reachView, over the path, target, sources and message wikilinks each issue names, takes " +
+ "ok and trust_verdict again over the kept issues, and below local reach recounts the " +
+ "tier-drift warning over the rows readableAtContextReach(ctx) keeps. It stays EXCLUDED " +
+ "because the repair branch is bounded by the owner scope only, and its plan names the " +
+ "records it would fix, reserved ones included.",
+ },
+ {
+ surface: "schema_inspect",
+ kind: K.mcpTool,
+ category: C.covered,
+ reason:
+ "the lint and orphans views keep only the findings whose named pages (path, or the source " +
+ "and target of a blocked link) pass reachView, and below local reach stats counts the same " +
+ "kept findings; graph, explain_type, active_pack and packs read the schema pack and its " +
+ "token usage counts, never a page path, title or body. Swept in through schema-admin.ts.",
+ },
+ {
+ surface: "schema_apply_mutations",
+ kind: K.mcpTool,
category: C.excluded,
reason:
- "check returns store.listTierDrift() unscoped across the whole index - every drifted " +
- "document's path, with no visibility check - and restore additionally reads the named " +
- "page's full frontmatter and body via parseFrontmatter.",
+ "swept in for file-level completeness because schema-tools.ts also registers " +
+ "schema_inspect: it writes Brain/_brain.yaml and returns the resulting pack, its diff and " +
+ "the audit path, never a note path, title, or body.",
+ },
+ {
+ surface: "brain_health",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "runs runDoctor's semantic-health pass and names preferences by id in its contradiction, " +
+ "stale-claim and batch-inflation findings; those are kept through the gated owner view " +
+ "only, and no reach view is asked, so a reserved preference can be named at remote reach.",
+ },
+ {
+ surface: "brain_status",
+ kind: K.mcpTool,
+ category: C.excluded,
+ reason:
+ "its hygiene count asks the owner view ANDed with reachView and its cited pages answer " +
+ "readableAtContextReach(ctx), but the doctor error and warning counts and the preference " +
+ "counts are taken over the whole Brain layer, so they move with a reserved record; it " +
+ "returns counts and problem labels, never a note path, title, or body.",
},
{
surface: "brain_maintenance",
@@ -355,11 +407,13 @@ export const VISIBILITY_SURFACE_REGISTRY: ReadonlyArray<VisibilitySurfaceEntry>
{
surface: "brain_agent_query",
kind: K.mcpTool,
- category: C.excluded,
+ category: C.covered,
reason:
- "queryAgentSources (core/brain/agent-source/query.ts) returns, by the handler's own " +
- "comment, 'contribution rows - ids, topics and the record text' from the provenance fold, " +
- "gated by an explicit agent_scope argument only - never by visibility.",
+ "queryAgentSources (core/brain/agent-source/query.ts) takes the handler's " +
+ "reachView(ctx.vault, contextReach(ctx)) as its view option and drops, before the roster " +
+ "is folded, every contribution whose named page or event-body string the caller may not " +
+ "read at its reach, so a note-write row for a withheld page is answered as one for an " +
+ "absent page; agent_scope still gates ownership.",
},
{
surface: "brain_writes",
@@ -375,11 +429,12 @@ export const VISIBILITY_SURFACE_REGISTRY: ReadonlyArray<VisibilitySurfaceEntry>
{
surface: "brain_agent_diff",
kind: K.mcpTool,
- category: C.excluded,
+ category: C.covered,
reason:
- "diffAgentSources returns the same contribution rows brain_agent_query does, gated by the " +
- "GATED server identity (owner_scope_delivery, off by default) rather than an argument - " +
- "visibility is not part of either gate.",
+ "diffAgentSources folds queryAgentSources, and the handler passes the same " +
+ "reachView(ctx.vault, contextReach(ctx)) view brain_agent_query does, so a contribution " +
+ "naming a page the caller may not read at its reach is dropped before any count or topic " +
+ "map is built; ownership is gated by the GATED server identity.",
},
{
surface: "brain_anticipatory_context",
@@ -401,11 +456,12 @@ export const VISIBILITY_SURFACE_REGISTRY: ReadonlyArray<VisibilitySurfaceEntry>
{
surface: "brain_event_trace",
kind: K.mcpTool,
- category: C.excluded,
+ category: C.covered,
reason:
- "resolveLogEventTraces (core/brain/event-trace.ts) returns Brain log event bodies, which " +
- "can name a note path per event; gated by a keep_private argument over the event's own " +
- "flag, not by the named artifact's visibility.",
+ "resolveLogEventTraces (core/brain/event-trace.ts) takes the handler's " +
+ "reachView(ctx.vault, contextReach(ctx)) as its view option and drops, before the limit " +
+ "and the totals, every event whose artifacts or body strings name a page the caller may " +
+ "not read at its reach, and every attached trace whose handoff reference it hides.",
},
{
surface: "brain_foresight",
@@ -510,14 +566,12 @@ export const VISIBILITY_SURFACE_REGISTRY: ReadonlyArray<VisibilitySurfaceEntry>
{
surface: "brain_procedural_memory",
kind: K.mcpTool,
- category: C.excluded,
+ category: C.covered,
reason:
- "collectEntries walks the configured roots with parseFrontmatter and returns `sourcePath` " +
- "and `title` per entry, so this surface DOES disclose a page's path and title - it is " +
- "excluded rather than swept in for completeness, and the distinction is the point of the " +
- "row. It reads procedure-kind pages under caller-named roots rather than through any of " +
- "the three read roots, and closing it means giving that walk a reach the way listVaultPages " +
- "has one, which is a fourth root to build rather than a filter to add.",
+ "the handler filters list through readableAtContextReach(ctx) and recounts its total, " +
+ "reports reconcile counts over readable pages only, and refuses mark_used and " +
+ "mark_outcome on a withheld entry with the unknown-id error, before writing, so a " +
+ "procedure page the caller may not read at its reach is answered as an absent one.",
},
{
surface: "brain_procedural_graph",
diff --git a/src/core/tags.ts b/src/core/tags.ts
index 73a0a20e..0f19ef58 100644
--- a/src/core/tags.ts
+++ b/src/core/tags.ts
@@ -28,7 +28,15 @@
*/
export const TAG_RE = /(^|[^\w/])#([A-Za-z_][\w\-/]*)/g;
-const CODE_FENCE_RE = /(^|\n)(```|~~~)[^\n]*\n[\s\S]*?(?:\n(?:```|~~~)[^\n]*|$)/g;
+// A fenced block runs from an opening line of three or more backticks or
+// tildes to the first later line that starts with a run of the SAME
+// character at least as long (or to the end of the text), the CommonMark
+// rule: a shorter run, or a run of the other character, is content. A
+// stored excerpt is fenced one backtick longer than any run it holds, so
+// its lines never close it early. The opening run is a whole run, which
+// fixes its length and keeps the scan linear on a long line of marks.
+const CODE_FENCE_RE =
+ /(^|\n)(?:(`{3,})(?!`)[^\n]*\n[\s\S]*?(?:\n\2`*[^\n]*|$)|(~{3,})(?!~)[^\n]*\n[\s\S]*?(?:\n\3~*[^\n]*|$))/g;
const INLINE_CODE_RE = /`[^`\n]*`/g;
/**
diff --git a/src/core/vault.ts b/src/core/vault.ts
index cea0d223..98b0af0d 100644
--- a/src/core/vault.ts
+++ b/src/core/vault.ts
@@ -50,7 +50,7 @@ import { createHash } from "node:crypto";
import { mkdirSync, readFileSync, readdirSync, writeFileSync } from "node:fs";
import { dirname, join, relative } from "node:path";
-import { WIKILINK_TARGET_RE } from "./brain/wikilink.ts";
+import { WIKILINK_TARGET_RE, maskCodeRegions } from "./brain/wikilink.ts";
import {
FileAlreadyExistsError,
atomicCreateFileSyncExclusive,
@@ -91,7 +91,6 @@ const KEY_VALUE_RE = new RegExp(`^(${FRONTMATTER_KEY_PATTERN})\\s*:\\s*(.*?)\\s*
// list item) is NOT matched. Captures the item text after the whitespace.
const DASH_ITEM_RE = /^-(?:\s+(.*))?$/;
const PLAIN_SCALAR_RE = /^[A-Za-z0-9_./-](?:[A-Za-z0-9_./ -]*[A-Za-z0-9_./-])?$/;
-const CODE_BLOCK_RE = /```[\s\S]*?```|`[^`]+`/g;
const SLUG_INVALID_RE = /[^a-z0-9]+/g;
const SLUG_MAX_LEN = 64;
@@ -556,7 +555,7 @@ export function slugify(value: string): string {
* file extensions and links inside fenced or inline code blocks.
*/
export function extractWikilinks(content: string): string[] {
- const masked = content.replace(CODE_BLOCK_RE, " ");
+ const masked = maskCodeRegions(content, () => " ");
const seen = new Set<string>();
const result: string[] = [];
for (const m of masked.matchAll(WIKILINK_TARGET_RE)) {
diff --git a/src/mcp/brain/admin-tools.ts b/src/mcp/brain/admin-tools.ts
index d994568a..597cae26 100644
--- a/src/mcp/brain/admin-tools.ts
+++ b/src/mcp/brain/admin-tools.ts
@@ -28,6 +28,7 @@ import { loadSchemaPack } from "../../core/brain/schema-pack.ts";
import { listSecrets } from "../../core/brain/secrets/store.ts";
import { runWithSecret, SecretExecDeniedError } from "../../core/brain/secrets/exec.ts";
import type { ProgressSink } from "../../core/brain/progress.ts";
+import { readableAtContextReach } from "./reach-readable.ts";
import { requiredStringArg, toolSafeguard } from "./shared.ts";
import { currentLease } from "../../core/brain/maintenance/lease.ts";
import { listJournal, MAINTENANCE_JOURNAL_CAP } from "../../core/brain/maintenance/journal.ts";
@@ -183,9 +184,13 @@ async function toolBrainTiers(
// Fail-soft: a vault that was never indexed has no snapshots and
// therefore no drift - not an error.
if (!existsSync(searchConfig.dbPath)) return { findings: [] };
+ // A drift row names a page's path and both values of one of its
+ // identity fields, so only the rows of pages the caller may read at
+ // its reach are listed.
+ const readable = readableAtContextReach(ctx);
const store = await Store.open(searchConfig, { mode: "read" });
try {
- return { findings: store.listTierDrift() };
+ return { findings: store.listTierDrift().filter((row) => readable(row.path)) };
} finally {
await store.close();
}
@@ -214,7 +219,9 @@ async function toolBrainTiers(
}
const store = await Store.open(searchConfig, { mode: "write" });
try {
- const docId = store.getDocumentIdByPath(path);
+ // A page the caller may not read at its reach is refused as one the
+ // index never saw, before anything about its drift is read or written.
+ const docId = readableAtContextReach(ctx)(path) ? store.getDocumentIdByPath(path) : null;
if (docId === null) {
throw new MCPError(INVALID_PARAMS, `brain_tiers ${op}: not indexed: ${path}`);
}
diff --git a/src/mcp/brain/design-note-tools.ts b/src/mcp/brain/design-note-tools.ts
index 8aa10698..e2925795 100644
--- a/src/mcp/brain/design-note-tools.ts
+++ b/src/mcp/brain/design-note-tools.ts
@@ -19,6 +19,7 @@ import { coerceStr } from "../coerce.ts";
import { MCP_PREVIEW_BUDGET } from "../preview-budget.ts";
import type { ServerContext, ToolDefinition } from "../tool-contract.ts";
import { vaultRelativeSafe, wrapToolErrors } from "./shared.ts";
+import { readableAtContextReach } from "./reach-readable.ts";
const TOOL = "brain_design_note";
@@ -41,6 +42,7 @@ async function toolBrainDesignNote(
const report = planDesignNote(ctx.vault, topic, {
now: new Date(),
ownerScope: gatedOwnerScopeView(ctx.vault, ctx.agentName).scope,
+ readable: readableAtContextReach(ctx),
});
const g = report.grounding;
return {
diff --git a/src/mcp/brain/distill-tools.ts b/src/mcp/brain/distill-tools.ts
index 31868476..bbc9bd4e 100644
--- a/src/mcp/brain/distill-tools.ts
+++ b/src/mcp/brain/distill-tools.ts
@@ -3,23 +3,39 @@
*
* The calling agent reads a source and distills it into atomic claims, each
* with an optional block id pointing back to the source block it came from,
- * and submits them here. OSB runs no model - it validates the claims and writes
- * one idempotent distillation page per source, listing each claim with its
- * block-level citation and a provenance section.
+ * and submits them here. Open Second Brain runs no model - it validates the
+ * claims, checks every quoted span in them against the cited block or the
+ * source, and writes one idempotent distillation page per source, listing
+ * each claim with its block-level citation and a provenance section. The
+ * response names the capture scope and, when spans were checked, the quotes
+ * report; `strict_quotes` turns an unverified span into a refusal carrying
+ * the `quote_unverified` wire code.
*/
import {
+ DISTILL_CLAIMS_MAX,
distillSource,
DistillValidationError,
parseDistillClaims,
+ type DistillSourceInput,
+ type DistillSourceOptions,
+ type DistillSourceResult,
} from "../../core/brain/distill/distill-source.ts";
+import { QuoteCheckError } from "../../core/brain/distill/quote-verdict.ts";
+import {
+ CAPTURE_EXCERPT_MAX_BYTES,
+ CaptureExcerptError,
+} from "../../core/brain/provenance/capture-scope.ts";
import { ResponseShapeError } from "../../core/brain/response-shape.ts";
import { resolveAgentName } from "../../core/config.ts";
-import { coerceStr } from "../coerce.ts";
+import { coerceBoolOptional, coerceStr } from "../coerce.ts";
+import { INVALID_PARAMS, MCPError } from "../protocol.ts";
import type { ServerContext, ToolDefinition } from "../tool-contract.ts";
+import { readableAtContextReach } from "./reach-readable.ts";
import { wrapToolErrors } from "./shared.ts";
const TOOL = "brain_distill_source";
+const EXCERPT_ARG = "excerpt";
async function toolBrainDistillSource(
ctx: ServerContext,
@@ -32,26 +48,80 @@ async function toolBrainDistillSource(
? agentArg
: resolveAgentName(ctx.configPath ?? undefined);
- return wrapToolErrors(TOOL, [DistillValidationError, ResponseShapeError], async () => {
- // Shape first: the payload is validated before a single claim is
- // normalized, so a malformed item aborts the whole batch unwritten.
- const claims = parseDistillClaims(args["claims"]);
- const res = distillSource(ctx.vault, { sourcePath, claims }, { agent, now: new Date() });
- return {
- distillation_path: res.distillationPath,
- created: res.created,
- claim_count: res.claimCount,
- // Omitted rather than reported as a sentinel when the source had no
- // bytes to hash: an absent key reads as "not recorded", where the
- // `missing` string this used to return read as a digest until you knew
- // better.
- ...(res.sourceHash !== undefined ? { source_hash: res.sourceHash } : {}),
- // The lane the page ACTUALLY landed in. Classifying the source a second
- // time here would be a second answer to one question, free to disagree
- // with the write that already happened.
- trust: res.trust,
- };
- });
+ const excerpt = readExcerptArg(args);
+ const strictQuotes = coerceBoolOptional(args, "strict_quotes") ?? false;
+
+ return wrapToolErrors(
+ TOOL,
+ [DistillValidationError, ResponseShapeError, CaptureExcerptError],
+ async () => {
+ // Shape first: the payload is validated before a single claim is
+ // normalized, so a malformed item aborts the whole batch unwritten.
+ const claims = parseDistillClaims(args["claims"]);
+ const res = distillWithQuoteCode(
+ ctx.vault,
+ { sourcePath, claims, ...(excerpt !== undefined ? { excerpt } : {}) },
+ // The quote check reads the source's bytes on the caller's behalf, so
+ // it answers only for a page the caller may read at its reach.
+ { agent, now: new Date(), strictQuotes, readable: readableAtContextReach(ctx) },
+ );
+ return {
+ distillation_path: res.distillationPath,
+ created: res.created,
+ claim_count: res.claimCount,
+ // Omitted rather than reported as a sentinel when the source had no
+ // bytes to hash: an absent key reads as "not recorded", where the
+ // `missing` string this used to return read as a digest until you knew
+ // better.
+ ...(res.sourceHash !== undefined ? { source_hash: res.sourceHash } : {}),
+ // The lane the page ACTUALLY landed in. Classifying the source a second
+ // time here would be a second answer to one question, free to disagree
+ // with the write that already happened.
+ trust: res.trust,
+ // How much of the source the page holds. Always present, like `trust`.
+ capture_scope: res.captureScope,
+ // Only when a claim held a quoted span: absence means nothing was
+ // quoted, never "checked and clean" (that is `unquoted: 0`).
+ ...(res.quotes !== undefined ? { quotes: res.quotes } : {}),
+ };
+ },
+ );
+}
+
+/**
+ * The `excerpt` argument, verbatim. `coerceStr` folds a blank string into
+ * "absent", which would turn an empty excerpt into a silent no-op; the core
+ * refuses an empty one by name, so the bytes are passed through untouched
+ * and only a non-string is rejected here.
+ */
+function readExcerptArg(args: Record<string, unknown>): string | undefined {
+ const raw = args[EXCERPT_ARG];
+ if (raw === undefined || raw === null) return undefined;
+ if (typeof raw !== "string") {
+ throw new MCPError(INVALID_PARAMS, `${TOOL}: argument '${EXCERPT_ARG}' must be a string`);
+ }
+ return raw;
+}
+
+/**
+ * Run the distillation, answering a strict quote refusal with its registered
+ * wire code. `wrapToolErrors` folds a validation class into a bare
+ * `INVALID_PARAMS` and drops any data, so this one class is mapped here,
+ * where the code is still in hand; every other error keeps the shared path.
+ */
+function distillWithQuoteCode(
+ vault: string,
+ input: DistillSourceInput,
+ opts: DistillSourceOptions,
+): DistillSourceResult {
+ try {
+ return distillSource(vault, input, opts);
+ } catch (err) {
+ if (err instanceof QuoteCheckError) {
+ throw new MCPError(INVALID_PARAMS, `${TOOL}: ${err.message}`, { code: err.code });
+ }
+ throw err;
+ }
}
export const DISTILL_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([
@@ -83,7 +153,7 @@ export const DISTILL_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([
// by DEFAULT: it works through `status: quarantine` plus the entity page
// status scope, which nothing gates. The same words mean less here.
description:
- "Distill one source into atomic claims; runs no model. Writes one idempotent page citing each claim as `[[source#^block]]`. A source outside this vault is marked `untrusted_source`, which ordinary reads still return unless `search_trust_gate_enabled` is on. `trust` names the lane.",
+ "Distill one source into atomic claims; runs no model, checks quotes. Writes one idempotent page citing each claim as `[[source#^block]]`. A source outside this vault is marked `untrusted_source`, which ordinary reads still return unless `search_trust_gate_enabled` is on. `trust` names the lane.",
inputSchema: {
type: "object",
properties: {
@@ -93,7 +163,7 @@ export const DISTILL_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([
},
claims: {
type: "array",
- description: "Atomic claims distilled from the source (non-empty).",
+ description: `Atomic claims distilled from the source, each one line (1 to ${DISTILL_CLAIMS_MAX}).`,
items: {
type: "object",
properties: {
@@ -107,6 +177,17 @@ export const DISTILL_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([
additionalProperties: false,
},
},
+ strict_quotes: {
+ type: "boolean",
+ description:
+ "Refuse the write (code quote_unverified) when a quoted span fails the check; by default the span is unquoted and reported.",
+ },
+ excerpt: {
+ type: "string",
+ // Characters, a necessary bound on the byte cap the core enforces.
+ maxLength: CAPTURE_EXCERPT_MAX_BYTES,
+ description: `Verbatim text read from a url-only source; stored on the page as bounded-local and used to check quotes. At most ${CAPTURE_EXCERPT_MAX_BYTES} bytes.`,
+ },
agent: {
type: "string",
description: "Optional agent identity override; defaults to the server-resolved name.",
diff --git a/src/mcp/brain/feedback-tools.ts b/src/mcp/brain/feedback-tools.ts
index 445b3240..ec7f4036 100644
--- a/src/mcp/brain/feedback-tools.ts
+++ b/src/mcp/brain/feedback-tools.ts
@@ -91,6 +91,7 @@ import {
vaultRelativeSafe,
} from "./shared.ts";
import { OPERATION } from "../../core/brain/safeguard.ts";
+import { readableAtContextReach } from "./reach-readable.ts";
/**
* Build the slug used in the signal / preference filename. We never let
@@ -461,6 +462,9 @@ async function toolBrainDream(
// One view for the whole call: every branch below reports preference
// and signal ids, and the staged lifecycle reports them twice.
const dreamView = gatedOwnerScopeView(ctx.vault, ctx.agentName);
+ // The rollup step's link candidates leave out a page the caller may not
+ // read at its reach, exactly as an absent page.
+ const readable = readableAtContextReach(ctx);
// Single-step requests (no-dead-ends, Unit E - operator surface).
// Deliberately checked before any environment work: a step the pass
@@ -530,6 +534,7 @@ async function toolBrainDream(
const stageOpts = {
now,
safeguard: toolSafeguard(ctx, OPERATION.dream),
+ readable,
...(agent ? { agentName: agent } : {}),
...(onProgress ? { onProgress } : {}),
};
@@ -635,6 +640,7 @@ async function toolBrainDream(
const preview = dream(ctx.vault, {
dryRun: true,
safeguard,
+ readable,
...(nowDate ? { now: nowDate } : {}),
...(agent ? { agentName: agent } : {}),
// Preview the run being guarded, overrides and all.
@@ -658,6 +664,7 @@ async function toolBrainDream(
const summary = dream(ctx.vault, {
dryRun,
safeguard,
+ readable,
...(nowDate ? { now: nowDate } : {}),
...(agent ? { agentName: agent } : {}),
...(gates !== undefined ? { gates } : {}),
diff --git a/src/mcp/brain/health-tools.ts b/src/mcp/brain/health-tools.ts
index aafd57d7..c7f402ab 100644
--- a/src/mcp/brain/health-tools.ts
+++ b/src/mcp/brain/health-tools.ts
@@ -25,6 +25,15 @@ import type { DoctorIssue } from "../../core/brain/types.ts";
import type { ServerContext, ToolDefinition } from "../tool-contract.ts";
import { coerceBool, coerceFormat } from "../coerce.ts";
import { vaultRelativeSafe } from "./shared.ts";
+import { readableAtContextReach } from "./reach-readable.ts";
+import { findingRefs } from "./hygiene-tools.ts";
+import { everyArtifactRefView } from "../../core/brain/artifact-ref-view.ts";
+import { reachView } from "../../core/brain/reach-view.ts";
+import { contextReach } from "../tool-contract.ts";
+import { TRANSPORT_REACH } from "../../core/graph/transport-reach.ts";
+import { Store } from "../../core/search/store.ts";
+import { computeTrustVerdict } from "../../core/brain/trust/compute-trust-verdict.ts";
+import { TIER_DRIFT_CODE, tierDriftMessage } from "../../core/brain/doctor/store-integrity.ts";
/**
* One reported issue, as an MCP caller sees it.
@@ -53,6 +62,46 @@ function issueView(ctx: ServerContext, issue: DoctorIssue): Record<string, unkno
};
}
+/** The verification counts of a doctor run given no dream summary. */
+const NO_VERIFICATION_DELTA = Object.freeze({
+ confirmed: 0,
+ drift: 0,
+ regression: 0,
+ missing_evidence: 0,
+});
+
+/**
+ * The `tier-drift` warning restated over the drift rows the caller may
+ * read, the rows `brain_tiers check` lists for it.
+ *
+ * The warning carries no path - only the count of every staged row - so
+ * the reference view cannot judge it, and the count alone moves when a
+ * page the caller may not read drifts. Below local reach the count is
+ * taken again over the readable rows; none left drops the warning, as
+ * for a vault with no drift.
+ */
+async function recountTierDrift<T extends { readonly code: string; readonly message: string }>(
+ ctx: ServerContext,
+ warnings: ReadonlyArray<T>,
+): Promise<ReadonlyArray<T>> {
+ if (!warnings.some((w) => w.code === TIER_DRIFT_CODE)) return warnings;
+ const readable = readableAtContextReach(ctx);
+ const store = await Store.open(
+ resolveSearchConfig({ vault: ctx.vault, configPath: ctx.configPath ?? undefined }),
+ { mode: "read" },
+ );
+ let count: number;
+ try {
+ count = store.listTierDrift().filter((row) => readable(row.path)).length;
+ } finally {
+ await store.close();
+ }
+ return warnings.flatMap((w) => {
+ if (w.code !== TIER_DRIFT_CODE) return [w];
+ return count === 0 ? [] : [{ ...w, message: tierDriftMessage(count) }];
+ });
+}
+
async function toolBrainDoctor(
ctx: ServerContext,
args: Record<string, unknown>,
@@ -103,7 +152,15 @@ async function toolBrainDoctor(
// Filtered BEFORE `ok` is decided: an `ok: false` beside an empty error
// list would say a hidden artifact is broken without naming it, which
// is the existence leak with the evidence removed.
- const view = gatedOwnerScopeView(ctx.vault, ctx.agentName);
+ //
+ // The reach view joins the owner view for the same reason: below local
+ // reach a record the caller may not read is named by no issue, so a
+ // malformed private preference answers as an absent one.
+ const reach = contextReach(ctx);
+ const view = everyArtifactRefView(
+ gatedOwnerScopeView(ctx.vault, ctx.agentName),
+ reachView(ctx.vault, reach),
+ );
/** Generic over the three streams: they share the naming fields, not a type. */
const visibleIssues = <
T extends {
@@ -128,8 +185,24 @@ async function toolBrainDoctor(
...extractWikilinkRichBodies(i.message).map((b) => parseWikilinkRich(b).target),
]);
const errors = visibleIssues(result.errors);
- const warnings = visibleIssues(result.warnings);
+ const warnings =
+ reach === TRANSPORT_REACH.local
+ ? visibleIssues(result.warnings)
+ : await recountTierDrift(ctx, visibleIssues(result.warnings));
const uncertain = result.uncertain === undefined ? undefined : visibleIssues(result.uncertain);
+ // The verdict summarises the streams, so it is taken again over the
+ // streams this caller sees: a verdict computed over a dropped error
+ // would say a hidden record is broken without naming it. The dream and
+ // verification inputs are the ones `runDoctor` was given here (none).
+ const trustVerdict =
+ view.filtersNothing || result.trust_verdict === undefined
+ ? result.trust_verdict
+ : computeTrustVerdict({
+ doctorWarnings: warnings,
+ doctorErrors: errors,
+ dreamWarnings: [],
+ verification: result.verification_delta_summary ?? NO_VERIFICATION_DELTA,
+ });
// Decide a single ok flag — `strict` only changes the CLI exit code,
// so we mirror that semantic here: with `strict`, warnings demote ok
@@ -181,7 +254,7 @@ async function toolBrainDoctor(
// surface, so it stays absent here). `instruction_file_warnings`
// surfaces vault-root instruction files exceeding the configured
// ceiling.
- ...(result.trust_verdict !== undefined ? { trust_verdict: result.trust_verdict } : {}),
+ ...(trustVerdict !== undefined ? { trust_verdict: trustVerdict } : {}),
instruction_file_warnings: (result.instruction_file_warnings ?? []).map((w) => ({
path: w.path,
lines: w.lines,
@@ -295,10 +368,22 @@ async function toolBrainStatus(
args: Record<string, unknown>,
): Promise<Record<string, unknown>> {
const format = coerceFormat(args);
- const snapshot = await buildOperatorSnapshot(
- ctx.vault,
- ctx.configPath !== null ? { configPath: ctx.configPath } : {},
+ // The hygiene count keeps the findings `brain_hygiene` would list for
+ // this caller - the owner and reach views, ANDed over each finding's
+ // targets - so a page the caller may not read moves it no more than an
+ // absent page does.
+ const findingView = everyArtifactRefView(
+ gatedOwnerScopeView(ctx.vault, ctx.agentName),
+ reachView(ctx.vault, contextReach(ctx)),
);
+ const snapshot = await buildOperatorSnapshot(ctx.vault, {
+ ...(ctx.configPath !== null ? { configPath: ctx.configPath } : {}),
+ // A cited page the caller may not read at its reach answers as an absent one.
+ readable: readableAtContextReach(ctx),
+ ...(findingView.filtersNothing
+ ? {}
+ : { keepFinding: (finding) => findingView.row(...findingRefs(ctx.vault, finding)) }),
+ });
return { format, ...snapshot };
}
diff --git a/src/mcp/brain/hygiene-tools.ts b/src/mcp/brain/hygiene-tools.ts
index 9be6eeac..bcd6675b 100644
--- a/src/mcp/brain/hygiene-tools.ts
+++ b/src/mcp/brain/hygiene-tools.ts
@@ -25,6 +25,7 @@ import { gatedOwnerScopeView } from "../../core/brain/owner-scope-view.ts";
import { everyArtifactRefView } from "../../core/brain/artifact-ref-view.ts";
import { reachView } from "../../core/brain/reach-view.ts";
import { contextReach } from "../tool-contract.ts";
+import { TRANSPORT_REACH } from "../../core/graph/transport-reach.ts";
import { applyHygienePlan } from "../../core/brain/hygiene/apply.ts";
import { annotateDedupFindings } from "../../core/brain/hygiene/dedup-verdicts.ts";
import { verdictFields } from "../../core/decision-model/pair-verdict.ts";
@@ -45,12 +46,14 @@ import {
DANGLING_LINK_DEFINITION,
measureFromIndex,
type LinkRatchetMeasurement,
+ type LinkRatchetUnmeasurableReason,
} from "../../core/search/link-ratchet.ts";
import { coerceBool } from "../coerce.ts";
import { INVALID_PARAMS, MCPError } from "../protocol.ts";
import { MCP_PREVIEW_BUDGET } from "../preview-budget.ts";
import type { ServerContext, ToolDefinition } from "../tool-contract.ts";
import { enforceCountGuard, readCountGuardArgs, vaultRelativeSafe } from "./shared.ts";
+import { readableAtContextReach } from "./reach-readable.ts";
function coerceStringArray(args: Record<string, unknown>, key: string): string[] | undefined {
const raw = args[key];
@@ -70,13 +73,16 @@ function resolverCmdFromConfig(vault: string): string | undefined {
}
function scanWithResolver(
- vault: string,
+ ctx: ServerContext,
detectors: HygieneDetectorId[] | undefined,
now: Date,
): HygieneScanReport {
+ const vault = ctx.vault;
const report = runHygieneScan(vault, {
...(detectors !== undefined && detectors.length > 0 ? { detectors } : {}),
now,
+ // A cited page the caller may not read at its reach answers as an absent one.
+ readable: readableAtContextReach(ctx),
});
const resolverCmd = resolverCmdFromConfig(vault);
if (resolverCmd === undefined) return report;
@@ -116,6 +122,19 @@ function findingView(vault: string, finding: HygieneFinding): Record<string, unk
};
}
+/**
+ * The `link_integrity.reason` a scan reports below local reach, where the
+ * count is not taken at all (docs/mcp.md, `brain_hygiene`). It sits beside
+ * the reasons the measurement itself returns, in
+ * {@link LinkIntegrityReason}.
+ */
+export const LINK_INTEGRITY_REACH_REASON = "reach" as const;
+
+/** Every `link_integrity.reason` a scan can report. */
+export type LinkIntegrityReason =
+ | LinkRatchetUnmeasurableReason
+ | typeof LINK_INTEGRITY_REACH_REASON;
+
/**
* Vault-wide link integrity, reported beside the detector findings
* (context-integrity-gates, unit G).
@@ -135,6 +154,18 @@ function findingView(vault: string, finding: HygieneFinding): Record<string, unk
* flattened into a zero.
*/
async function linkIntegrityView(ctx: ServerContext): Promise<Record<string, unknown>> {
+ // The measurement is taken over the whole index, withheld pages
+ // included, so its counts would move when a page the caller may not
+ // read appears or goes. Below local reach the number is not taken at
+ // all, which is the same answer whether such a page exists or not.
+ if (contextReach(ctx) !== TRANSPORT_REACH.local) {
+ return {
+ definition: DANGLING_LINK_DEFINITION,
+ measured: false,
+ reason: LINK_INTEGRITY_REACH_REASON satisfies LinkIntegrityReason,
+ detail: "link integrity is measured over the whole index and is reported at local reach only",
+ };
+ }
let measurement: LinkRatchetMeasurement;
try {
measurement = await measureFromIndex(
@@ -148,7 +179,7 @@ async function linkIntegrityView(ctx: ServerContext): Promise<Record<string, unk
return {
definition: DANGLING_LINK_DEFINITION,
measured: false,
- reason: "index-unreadable",
+ reason: "index-unreadable" satisfies LinkIntegrityReason,
detail: e instanceof Error ? e.message : String(e),
};
}
@@ -156,7 +187,7 @@ async function linkIntegrityView(ctx: ServerContext): Promise<Record<string, unk
return {
definition: measurement.definition,
measured: false,
- reason: measurement.reason,
+ reason: measurement.reason satisfies LinkIntegrityReason,
detail: measurement.detail,
};
}
@@ -230,7 +261,7 @@ async function toolBrainHygiene(
`'detectors' entries must be: ${HYGIENE_DETECTOR_IDS.join(", ")}`,
);
}
- const scanned = scanWithResolver(ctx.vault, detectors, now);
+ const scanned = scanWithResolver(ctx, detectors, now);
// The owner boundary is applied to the REPORT, once, before EITHER mode
// reads it. `findings[].targets` are artifact ids (or absolute paths
@@ -330,7 +361,7 @@ export const HYGIENE_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([
name: "brain_hygiene",
previewBudget: MCP_PREVIEW_BUDGET,
description:
- "Memory hygiene pipeline. `scan`: read-only digest of contested truth slots, near-duplicate preferences, stale/orphaned pages, low-usefulness candidates. `apply`: execute selected finding ids (review findings never execute). `refresh`: targeted recompile of stale pages with dry-run.",
+ "Memory hygiene pipeline. `scan`: read-only digest of contested truth slots, near-duplicate preferences, stale/orphaned pages, low-usefulness candidates, knowledge resting on url-only sources. `apply`: run selected finding ids (review findings never run). `refresh`: recompile stale pages.",
inputSchema: {
type: "object",
properties: {
@@ -342,7 +373,7 @@ export const HYGIENE_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([
detectors: {
type: "array",
items: { type: "string", enum: [...HYGIENE_DETECTOR_IDS] },
- description: `Detector subset for scan/apply. Default: ${DEFAULT_SCAN_IDS.join(", ")} (every registered detector except the opt-in ones).`,
+ description: `Detector subset for scan/apply. Default: ${DEFAULT_SCAN_IDS.join(", ")} (all but the opt-in ones).`,
},
ids: {
type: "array",
diff --git a/src/mcp/brain/ingest-tools.ts b/src/mcp/brain/ingest-tools.ts
index 420743bb..a17d60e5 100644
--- a/src/mcp/brain/ingest-tools.ts
+++ b/src/mcp/brain/ingest-tools.ts
@@ -37,6 +37,7 @@ import { MCP_PREVIEW_BUDGET } from "../preview-budget.ts";
import { INVALID_PARAMS, MCPError } from "../protocol.ts";
import type { ServerContext, ToolDefinition } from "../tool-contract.ts";
import { parseExtractionIntakeArgs } from "./intake-args.ts";
+import { readableAtContextReach } from "./reach-readable.ts";
import { enforceCountGuard, readCountGuardArgs, wrapToolErrors } from "./shared.ts";
const TOOL = "brain_ingest_source";
@@ -86,6 +87,8 @@ async function toolBrainIngestSource(
{
agent,
now: new Date(),
+ // A page the caller may not read at its reach answers as an absent one.
+ readable: readableAtContextReach(ctx),
...(planId !== undefined ? { planId } : {}),
...(preExtract ? { preExtract: true } : {}),
},
@@ -96,6 +99,10 @@ async function toolBrainIngestSource(
entities_created: [...res.entitiesCreated],
entities_updated: [...res.entitiesUpdated],
connections: [...res.connections],
+ // How much of the source the vault holds, as the summary page records
+ // it. Always present: absence would read as "full-local" to a caller
+ // that never learned the key exists.
+ capture_scope: res.captureScope,
// Only emitted when the pre-extract pass ran, so a call without it is
// byte-identical to before (P4).
...(res.preExtract !== undefined ? { pre_extract: serializePreExtract(res.preExtract) } : {}),
@@ -206,9 +213,12 @@ async function toolBrainDeleteBySource(
const agent = coerceStr(args, "agent", false) ?? undefined;
const { expect, strict } = readCountGuardArgs(args);
const now = new Date();
+ // An original the caller may not read at its reach is planned and
+ // deleted exactly as an absent one: never confirmed, never removed.
const baseOpts = {
includeOriginals,
now,
+ include: readableAtContextReach(ctx),
...(agent !== undefined ? { agent } : {}),
};
@@ -316,16 +326,19 @@ async function toolBrainIngestBatchPlan(
const reconcile = coerceBoolOptional(args, "reconcile") ?? false;
const srcSubpath = coerceStr(args, "src_subpath", false) ?? undefined;
const exclude = coerceStrList(args, "exclude");
+ // A file the caller may not read at its reach is planned as an absent one.
+ const include = readableAtContextReach(ctx);
const plan = planBatches(ctx.vault, sourceDir, {
maxBatchBytes,
maxBatchFiles,
resume,
+ include,
...(srcSubpath !== undefined ? { srcSubpath } : {}),
...(exclude.length > 0 ? { exclude } : {}),
});
// Reconcile BEFORE any checkpoint clear below, so the gap report reads a live
// checkpoint. Read-only: it only diffs dispatched vs completed.
- const report = reconcile ? reconcilePlan(ctx.vault, plan) : undefined;
+ const report = reconcile ? reconcilePlan(ctx.vault, plan, include) : undefined;
// A resumed plan that comes back empty is fully drained: drop its checkpoint
// (the content manifest is the authoritative final state from here on).
if (resume && plan.batches.length === 0) {
diff --git a/src/mcp/brain/knowledge-tools.ts b/src/mcp/brain/knowledge-tools.ts
index 4cd2dd9b..c5df9b52 100644
--- a/src/mcp/brain/knowledge-tools.ts
+++ b/src/mcp/brain/knowledge-tools.ts
@@ -76,6 +76,7 @@ import {
unknownOperationError,
} from "../coerce.ts";
import { coercePositiveInteger, toolSafeguard } from "./shared.ts";
+import { readableAtContextReach } from "./reach-readable.ts";
/**
* Vault-relative locations these two handlers read BY PATH, rather than
@@ -498,7 +499,13 @@ async function toolBrainDiarize(
// filtered through the same gated view every other report surface
// here uses - an unscoped list would publish another owner's
// preference ids as citation targets.
- { now: new Date(), ownerScope: gatedOwnerScopeView(ctx.vault, ctx.agentName).scope },
+ // A page the caller may not read at its reach is withheld from the
+ // candidates exactly as an absent one.
+ {
+ now: new Date(),
+ ownerScope: gatedOwnerScopeView(ctx.vault, ctx.agentName).scope,
+ readable: readableAtContextReach(ctx),
+ },
);
return {
entity_id: report.entityId,
@@ -542,7 +549,13 @@ function toolBrainIdeaDiscovery(
// notes (a-label-is-not-a-boundary, U3). Filtered BEFORE the trigger
// pass so a hidden artifact cannot be enqueued either.
const view = gatedOwnerScopeView(ctx.vault, ctx.agentName);
- const ideas = view.keep(discoverIdeas(ctx.vault, { now, cap }), (i) => i.sourceArtifacts);
+ // A page the caller may not read at its reach is left out of the walk
+ // itself, before the ranking and the cap, so the list is the one a vault
+ // without that page would give.
+ const ideas = view.keep(
+ discoverIdeas(ctx.vault, { now, cap, include: readableAtContextReach(ctx) }),
+ (i) => i.sourceArtifacts,
+ );
let triggersCreated: number | undefined;
if (enqueue) {
const result = createTriggers(ctx.vault, ideaCandidates(ideas), {
diff --git a/src/mcp/brain/lifecycle-file-tools.ts b/src/mcp/brain/lifecycle-file-tools.ts
index 6820b1fe..ebbc4ecc 100644
--- a/src/mcp/brain/lifecycle-file-tools.ts
+++ b/src/mcp/brain/lifecycle-file-tools.ts
@@ -47,6 +47,7 @@ import { COUNT_GUARD_WIRE_CODE, TOOL_ERROR_CODE } from "../tool-error-codes.ts";
import type { ServerContext, ToolDefinition } from "../tool-contract.ts";
import { coerceBoolOptional, coerceStr, unknownOperationError } from "../coerce.ts";
import { coerceNonNegativeInteger, readCountGuardArgs } from "./shared.ts";
+import { readableAtContextReach } from "./reach-readable.ts";
const TOOL = "brain_note_lifecycle";
@@ -132,6 +133,9 @@ async function toolBrainNoteLifecycle(
...(deleteLinked !== undefined ? { deleteLinked } : {}),
expect,
strict,
+ // A note the caller may not read at its reach is refused exactly
+ // as a missing one.
+ readable: readableAtContextReach(ctx),
});
return renderResult(res);
} catch (err) {
diff --git a/src/mcp/brain/ner-tools.ts b/src/mcp/brain/ner-tools.ts
index 3fbffe84..2477372d 100644
--- a/src/mcp/brain/ner-tools.ts
+++ b/src/mcp/brain/ner-tools.ts
@@ -18,6 +18,7 @@ import { intakeExtraction, IntakeValidationError } from "../../core/brain/intake
import { resolveAgentName } from "../../core/config.ts";
import type { ServerContext, ToolDefinition } from "../tool-contract.ts";
import { parseExtractionIntakeArgs } from "./intake-args.ts";
+import { readableAtContextReach } from "./reach-readable.ts";
import { wrapToolErrors } from "./shared.ts";
const TOOL = "brain_intake_entities";
@@ -42,6 +43,8 @@ async function toolBrainIntakeEntities(
agent,
now: new Date(),
provenance: parsed.provenance,
+ // A page the caller may not read at its reach answers as an absent one.
+ readable: readableAtContextReach(ctx),
});
return {
entities_created: [...result.entitiesCreated],
diff --git a/src/mcp/brain/notes-tools.ts b/src/mcp/brain/notes-tools.ts
index c77e5052..207f1454 100644
--- a/src/mcp/brain/notes-tools.ts
+++ b/src/mcp/brain/notes-tools.ts
@@ -52,6 +52,7 @@ import { rethrowVaultFrozen } from "../frozen-refusal.ts";
import { INTERNAL_ERROR, INVALID_PARAMS, type JsonRpcErrorCode, MCPError } from "../protocol.ts";
import type { ServerContext, ToolDefinition } from "../tool-contract.ts";
import { coerceBoolOptional, coerceStr, coerceStringOptional } from "../coerce.ts";
+import { readableAtContextReach } from "./reach-readable.ts";
/**
* The two vocabularies a refused write speaks, and the bridge between
@@ -441,11 +442,12 @@ function runSingleWrite<K extends SingleNoteOperation["kind"]>(
): Extract<NoteOpResult, { readonly kind: K }> {
let batch;
try {
- batch = applyWriteBatch(
- ctx.vault,
- [op],
- ctx.configPath !== null ? { configPath: ctx.configPath } : {},
- );
+ // A target the caller may not read at its reach is refused exactly
+ // as a missing one.
+ batch = applyWriteBatch(ctx.vault, [op], {
+ readable: readableAtContextReach(ctx),
+ ...(ctx.configPath !== null ? { configPath: ctx.configPath } : {}),
+ });
} catch (err) {
throw writeBatchErrorToMcp(err, tool);
}
diff --git a/src/mcp/brain/pack-tools.ts b/src/mcp/brain/pack-tools.ts
index 103bb720..2a206d0d 100644
--- a/src/mcp/brain/pack-tools.ts
+++ b/src/mcp/brain/pack-tools.ts
@@ -530,6 +530,10 @@ async function toolBrainEventTrace(
// The event body is echoed verbatim and log payloads carry
// artifact paths (a-label-is-not-a-boundary, U3).
ownerScope: gatedOwnerScopeView(ctx.vault, ctx.agentName).scope,
+ // A note-write body carries the page's whole-file digest and byte
+ // size, so an event naming a page the caller may not read at its
+ // reach is dropped exactly as an event about an absent page is.
+ view: reachView(ctx.vault, contextReach(ctx)),
});
} catch (err) {
// A selector-validation error (bad date/at/kind, checked before any IO) is
diff --git a/src/mcp/brain/procedure-tools.ts b/src/mcp/brain/procedure-tools.ts
index db3656be..bc5f53d0 100644
--- a/src/mcp/brain/procedure-tools.ts
+++ b/src/mcp/brain/procedure-tools.ts
@@ -41,6 +41,7 @@ import { reachView } from "../../core/brain/reach-view.ts";
import type { ServerContext, ToolDefinition } from "../tool-contract.ts";
import { coerceStrList, unknownOperationError } from "../coerce.ts";
import { coercePositiveInteger, optionalStringArg, requiredStringArg } from "./shared.ts";
+import { readableAtContextReach } from "./reach-readable.ts";
async function toolBrainSkillProposals(
ctx: ServerContext,
@@ -191,6 +192,15 @@ async function toolBrainProceduralMemory(
args: Record<string, unknown>,
): Promise<Record<string, unknown>> {
const operation = requiredStringArg("brain_procedural_memory", args, "operation");
+ // An entry names its page's path, title and frontmatter lists, so a page
+ // the caller may not read at its reach is answered as an absent one: left
+ // out of every count and list, and an unknown id to the two mark
+ // operations, refused before anything is written.
+ const readable = readableAtContextReach(ctx);
+ const unknownId = (id: string): boolean => {
+ const entry = listProceduralMemory(ctx.vault).find((e) => e.id === id);
+ return entry === undefined || !readable(entry.sourcePath);
+ };
if (operation === "reconcile") {
const roots = coerceStrList(args, "roots");
const effectiveRoots =
@@ -202,18 +212,18 @@ async function toolBrainProceduralMemory(
join(ctx.vault, "runbooks"),
];
return {
- ...reconcileProceduralMemory(ctx.vault, { roots: effectiveRoots }),
+ ...reconcileProceduralMemory(ctx.vault, { roots: effectiveRoots, include: readable }),
};
}
if (operation === "list") {
- const entries = listProceduralMemory(ctx.vault);
+ const entries = listProceduralMemory(ctx.vault).filter((e) => readable(e.sourcePath));
// Opt-in success-rate ranking (t_703f7b18); default order unchanged.
const ordered = args["ranked"] === true ? rankProceduralMemory(entries) : entries;
return { total: ordered.length, entries: ordered };
}
if (operation === "mark_used") {
const id = requiredStringArg("brain_procedural_memory", args, "id");
- const updated = markProceduralMemoryUsed(ctx.vault, id);
+ const updated = unknownId(id) ? null : markProceduralMemoryUsed(ctx.vault, id);
if (!updated) {
throw new MCPError(INVALID_PARAMS, `brain_procedural_memory: unknown entry id: ${id}`);
}
@@ -228,7 +238,7 @@ async function toolBrainProceduralMemory(
"brain_procedural_memory: outcome must be 'success' or 'failure'",
);
}
- const updated = recordProceduralOutcome(ctx.vault, id, outcome);
+ const updated = unknownId(id) ? null : recordProceduralOutcome(ctx.vault, id, outcome);
if (!updated) {
throw new MCPError(INVALID_PARAMS, `brain_procedural_memory: unknown entry id: ${id}`);
}
diff --git a/src/mcp/brain/query-tools.ts b/src/mcp/brain/query-tools.ts
index 5cea603b..e7a2f6be 100644
--- a/src/mcp/brain/query-tools.ts
+++ b/src/mcp/brain/query-tools.ts
@@ -23,6 +23,7 @@ import { diffAgentSources, type AgentSourceDiffMode } from "../../core/brain/age
import { queryAgentSources } from "../../core/brain/agent-source/query.ts";
import {
NoteRevertError,
+ noteRevertDigest,
planNoteRevert,
type NoteRevertSelector,
} from "../../core/brain/notes/revert.ts";
@@ -294,6 +295,10 @@ async function toolBrainAgentQuery(
...(query !== null ? { query } : {}),
...(kind !== null ? { kind } : {}),
...(ownerScope !== undefined ? { ownerScope } : {}),
+ // A note contribution echoes the write event's body, which carries
+ // the page's digest and size: a page the caller may not read at its
+ // reach is withheld exactly as an absent one.
+ view: reachView(ctx.vault, contextReach(ctx)),
limit: coerceInt(args, "limit", 50, 1, 500),
}) as unknown as Record<string, unknown>;
}
@@ -410,21 +415,26 @@ function planRevert(ctx: ServerContext, args: Record<string, unknown>): Record<s
}
throw err;
}
- // Same reach rule as `list`. The digest still seals the WHOLE plan -
- // it is applied at the operator's terminal, where the full plan is
- // shown - so a remote caller sees the entries it may read and a digest
- // that names none of the others.
+ // Same reach rule as `list`. When an entry is withheld, the digest
+ // seals only the entries shown: the full plan's digest covers each
+ // hidden entry's current-bytes hash, so it would differ between a
+ // withheld page, a deleted one and a path never written, and change
+ // whenever the withheld page did. The CLI apply re-plans in full and
+ // refuses this digest with `digest_mismatch`, the safe outcome for a
+ // plan the caller could not fully see.
const view = reachView(ctx.vault, contextReach(ctx));
const entries = view.filtersNothing
? plan.entries
: plan.entries.filter((entry) => view.visible(entry.target));
+ const digest =
+ entries.length === plan.entries.length ? plan.digest : noteRevertDigest(plan.selector, entries);
return {
action: BRAIN_WRITES_ACTION.planRevert,
selector: { ...plan.selector },
entries: entries.map((entry) => ({ ...entry })),
- digest: plan.digest,
+ digest,
planned_at: plan.planned_at,
- next_command: `o2b brain writes revert --apply ${plan.digest}`,
+ next_command: `o2b brain writes revert --apply ${digest}`,
warnings: plan.warnings.map((w) => ({
path: w.path,
line: w.lineNumber,
@@ -460,6 +470,7 @@ async function toolBrainAgentDiff(
return diffAgentSources(ctx.vault, {
...(mode !== null ? { mode } : {}),
...(ownerScope !== null ? { ownerScope } : {}),
+ view: reachView(ctx.vault, contextReach(ctx)),
agents: coerceStrList(args, "agents"),
...(topic !== null ? { topic } : {}),
...(query !== null ? { query } : {}),
diff --git a/src/mcp/brain/reach-readable.ts b/src/mcp/brain/reach-readable.ts
new file mode 100644
index 00000000..bb684fbd
--- /dev/null
+++ b/src/mcp/brain/reach-readable.ts
@@ -0,0 +1,54 @@
+/**
+ * May the caller of one request read a vault page at this path?
+ *
+ * A tool that reads a page's bytes on the caller's behalf and answers
+ * something derived from them - whether a span occurs in it, which
+ * capture scope it backs - must answer at the caller's reach, or the
+ * derived answer discloses what the page itself would not. This module
+ * binds the two existing per-path rules to one request, and adds no rule
+ * of its own:
+ *
+ * - visibility: `isPathReadableAtReach` at the reach the transport
+ * minted (`contextReach`, fail closed when none was minted);
+ * - ownership: `isPathOwnerVisible` under the server-resolved identity,
+ * only when `integrity.owner_scope_delivery` is `fail` (the same gate
+ * `gatedOwnerScopeView` reads).
+ *
+ * The predicate is a plain vault-relative path test rather than an
+ * `ArtifactRefView`, because the reference view reads a path without a
+ * `.md` extension as a Brain artifact id, and a source path may name any
+ * vault file.
+ */
+
+import { resolve } from "node:path";
+
+import { TRANSPORT_REACH } from "../../core/graph/transport-reach.ts";
+import { pathIsInside } from "../../core/path-safety.ts";
+import { resolveOwnerScopeDelivery } from "../../core/brain/preferences-collect.ts";
+import {
+ isPathOwnerVisible,
+ isPathReadableAtReach,
+ type FrontmatterCache,
+} from "../../core/search/result-filters.ts";
+import { contextReach, type ServerContext } from "../tool-contract.ts";
+
+/** A vault-relative path test bound to one request's reach and owner. */
+export type ReadablePredicate = (rel: string) => boolean;
+
+/**
+ * The predicate for one request. Local reach with the ownership gate off
+ * answers `true` without touching the filesystem.
+ */
+export function readableAtContextReach(ctx: ServerContext): ReadablePredicate {
+ const reach = contextReach(ctx);
+ const scope = resolveOwnerScopeDelivery(ctx.vault, ctx.agentName).enforcedScope;
+ if (reach === TRANSPORT_REACH.local && scope === null) return () => true;
+ const root = resolve(ctx.vault);
+ const cache: FrontmatterCache = new Map();
+ return (rel) =>
+ // A path that leaves the vault names no page the caller could be
+ // shown, and is answered before anything is read.
+ pathIsInside(resolve(root, rel), root) &&
+ isPathReadableAtReach(ctx.vault, rel, reach, cache) &&
+ (scope === null || isPathOwnerVisible(ctx.vault, rel, scope, cache));
+}
diff --git a/src/mcp/brain/research-tools.ts b/src/mcp/brain/research-tools.ts
index 521d291e..01e84473 100644
--- a/src/mcp/brain/research-tools.ts
+++ b/src/mcp/brain/research-tools.ts
@@ -17,6 +17,7 @@ import { ResponseShapeError } from "../../core/brain/response-shape.ts";
import { resolveAgentName } from "../../core/config.ts";
import type { ServerContext, ToolDefinition } from "../tool-contract.ts";
import { coerceStr } from "../coerce.ts";
+import { readableAtContextReach } from "./reach-readable.ts";
import { wrapToolErrors } from "./shared.ts";
const TOOL = "brain_research_report";
@@ -35,11 +36,19 @@ async function toolBrainResearchReport(
// Shape first: the payload is validated before the citation contract is
// evaluated, so a malformed finding aborts the report unwritten.
const input = parseResearchReportInput(args);
- const res = writeResearchReport(ctx.vault, input, { agent, now: new Date() });
+ // Scopes answer at the caller's reach: a page the caller may not read
+ // is reported exactly as an absent one.
+ const res = writeResearchReport(ctx.vault, input, {
+ agent,
+ now: new Date(),
+ readable: readableAtContextReach(ctx),
+ });
return {
report_path: res.reportPath,
created: res.created,
finding_count: res.findingCount,
+ // One scope per consulted source, in the caller's `sources` order.
+ capture_scopes: [...res.captureScopes],
};
});
}
diff --git a/src/mcp/brain/synthesis-tools.ts b/src/mcp/brain/synthesis-tools.ts
index 730bdaa3..3e557f06 100644
--- a/src/mcp/brain/synthesis-tools.ts
+++ b/src/mcp/brain/synthesis-tools.ts
@@ -34,6 +34,7 @@ import { INVALID_PARAMS, MCPError } from "../protocol.ts";
import { TRANSPORT_REACH } from "../../core/graph/transport-reach.ts";
import { contextReach, type ServerContext, type ToolDefinition } from "../tool-contract.ts";
import { MCP_PREVIEW_BUDGET } from "../preview-budget.ts";
+import { readableAtContextReach } from "./reach-readable.ts";
const SUMMARY_TOOL = "brain_session_summary";
@@ -320,6 +321,9 @@ function toolBrainNoteHistory(
const result = decomposeNoteHistory(ctx.vault, pathRaw.trim(), {
...(gapHours !== undefined ? { gapHours } : {}),
...(maxCount !== undefined ? { maxCount } : {}),
+ // A page the caller may not read at its reach answers as a path no
+ // commit touches.
+ include: readableAtContextReach(ctx),
});
return {
note_path: result.notePath,
diff --git a/src/mcp/brain/write-batch-tools.ts b/src/mcp/brain/write-batch-tools.ts
index 2a083564..c806b946 100644
--- a/src/mcp/brain/write-batch-tools.ts
+++ b/src/mcp/brain/write-batch-tools.ts
@@ -34,6 +34,7 @@ import {
import { INVALID_PARAMS, MCPError } from "../protocol.ts";
import type { ServerContext, ToolDefinition } from "../tool-contract.ts";
import { noteWriteResult, parseFrontmatterArg, writeBatchErrorToMcp } from "./notes-tools.ts";
+import { readableAtContextReach } from "./reach-readable.ts";
import { vaultRelativeSafe } from "./shared.ts";
import { unknownOperationError } from "../coerce.ts";
@@ -221,7 +222,12 @@ async function toolBrainWriteBatch(
const operations = rawOps.map((raw, index) => mapOperation(raw, index, resolveAgent));
const requestId = optionalStr(args, "request_id");
- const baseOpts = ctx.configPath !== null ? { configPath: ctx.configPath } : {};
+ // An update or append whose target the caller may not read at its
+ // reach is refused exactly as a missing one.
+ const baseOpts = {
+ readable: readableAtContextReach(ctx),
+ ...(ctx.configPath !== null ? { configPath: ctx.configPath } : {}),
+ };
let batch: WriteBatchResult;
// Set exactly when a request ID was supplied: the core's receipted
// overload always carries the status, so nothing here invents one.
diff --git a/src/mcp/registry-guard.ts b/src/mcp/registry-guard.ts
index 063dcd02..b87040ca 100644
--- a/src/mcp/registry-guard.ts
+++ b/src/mcp/registry-guard.ts
@@ -350,8 +350,10 @@ export const PREVIEW_BUDGET_EXEMPT: Readonly<Record<string, string>> = Object.fr
brain_write_session: "lifecycle ops return one fixed-shape envelope; prompts are kernel-bounded",
brain_intake_entities: "write; returns created/updated id lists and a relation count",
brain_ingest_source: "write; returns the summary path plus bounded id lists",
- brain_distill_source: "write; returns the distillation path plus a claim count and source hash",
- brain_research_report: "write; returns the report path and a finding count",
+ brain_distill_source:
+ "write; returns the distillation path, a claim count, source hash, capture scope, and a quotes report whose finding list is capped and self-declaring its truncation",
+ brain_research_report:
+ "write; returns the report path, a finding count, and one capture scope per caller-supplied source",
brain_derive_fact: "write; returns one derived preference id, its level and premises",
brain_memory_bridge:
"write; returns a small fixed-shape receipt (recorded flag, kind, count, ids)",
diff --git a/src/mcp/schema-tools.ts b/src/mcp/schema-tools.ts
index 4bb9619a..89e4ce05 100644
--- a/src/mcp/schema-tools.ts
+++ b/src/mcp/schema-tools.ts
@@ -10,11 +10,44 @@ import {
reviewSchemaOrphans,
} from "../core/brain/schema-admin.ts";
import { previewSchemaMutations, type SchemaMutation } from "../core/brain/schema-mutate.ts";
+import { buildSchemaReport, type SchemaReportFinding } from "../core/brain/schema-report.ts";
+import { reachView } from "../core/brain/reach-view.ts";
+import { TRANSPORT_REACH } from "../core/graph/transport-reach.ts";
import { resolveSearchConfig } from "../core/search/index.ts";
import { INVALID_PARAMS, MCPError } from "./protocol.ts";
import { coerceBool, coerceStr } from "./coerce.ts";
import { MCP_PREVIEW_BUDGET } from "./preview-budget.ts";
-import type { ServerContext, ToolDefinition } from "./tool-contract.ts";
+import { contextReach, type ServerContext, type ToolDefinition } from "./tool-contract.ts";
+
+/**
+ * The pages a schema finding names, as references the reach view judges.
+ * A declaration nobody uses names no page.
+ */
+function findingRefs(finding: SchemaReportFinding): ReadonlyArray<string> {
+ switch (finding.kind) {
+ case "unknown-token":
+ case "unreadable-artifact":
+ return [finding.path];
+ case "link-constraint-violation":
+ return [finding.source, finding.target];
+ case "unused-declaration":
+ return [];
+ }
+}
+
+/**
+ * The findings a caller may see at its reach. Every finding that names a
+ * record names it by path, so below local reach a record the caller may
+ * not read is reported by no view, and `stats` counts what `lint` lists:
+ * a malformed private page answers as an absent one.
+ */
+function visibleFindings<T extends SchemaReportFinding>(
+ ctx: ServerContext,
+ findings: ReadonlyArray<T>,
+): ReadonlyArray<T> {
+ const view = reachView(ctx.vault, contextReach(ctx));
+ return view.filtersNothing ? findings : view.keep(findings, findingRefs);
+}
// Read-side handlers behind the consolidated `schema_inspect` views.
// The per-view alias tools were removed in 1.0.0 (tombstones in
@@ -23,13 +56,25 @@ const SCHEMA_INSPECT_VIEWS: Readonly<
Record<string, (ctx: ServerContext, args: Record<string, unknown>) => Promise<unknown> | unknown>
> = Object.freeze({
graph: (ctx: ServerContext) => buildSchemaGraph(ctx.vault),
- lint: (ctx: ServerContext) =>
- buildSchemaLint(ctx.vault, {
+ lint: (ctx: ServerContext) => {
+ const lint = buildSchemaLint(ctx.vault, {
dbPath: resolveSearchConfig({ vault: ctx.vault, configPath: ctx.configPath ?? undefined })
.dbPath,
- }),
- stats: (ctx: ServerContext) => buildSchemaStats(ctx.vault),
- orphans: (ctx: ServerContext) => reviewSchemaOrphans(ctx.vault),
+ });
+ return { ...lint, findings: visibleFindings(ctx, lint.findings) };
+ },
+ stats: (ctx: ServerContext) => {
+ const stats = buildSchemaStats(ctx.vault);
+ if (contextReach(ctx) === TRANSPORT_REACH.local) return stats;
+ return {
+ ...stats,
+ findings: visibleFindings(ctx, buildSchemaReport(ctx.vault).findings).length,
+ };
+ },
+ orphans: (ctx: ServerContext) => {
+ const orphans = reviewSchemaOrphans(ctx.vault);
+ return { ...orphans, orphans: visibleFindings(ctx, orphans.orphans) };
+ },
explain_type: (ctx: ServerContext, args: Record<string, unknown>) =>
explainSchemaToken(ctx.vault, coerceStr(args, "token")!),
active_pack: (ctx: ServerContext) => getActiveSchemaPack(ctx.vault),
diff --git a/src/mcp/tool-error-codes.ts b/src/mcp/tool-error-codes.ts
index d048f996..fd6bfe9b 100644
--- a/src/mcp/tool-error-codes.ts
+++ b/src/mcp/tool-error-codes.ts
@@ -27,6 +27,7 @@
import { BrainPreferenceNotFoundError } from "../core/brain/apply-evidence.ts";
import { CountGuardError } from "../core/brain/count-guard.ts";
+import { QuoteCheckError } from "../core/brain/distill/quote-verdict.ts";
import type { ExactStateErrorCode } from "../core/brain/exact-state.ts";
import { ExactStateError } from "../core/brain/exact-state.ts";
import {
@@ -125,6 +126,8 @@ export const TOOL_ERROR_CODE = Object.freeze({
brainArtifactUnparseable: "brain_artifact_unparseable",
/** An argument that belongs to another action of the same tool. */
argumentForbidden: "argument_forbidden",
+ /** A strict distillation refused a quoted span it could not verify. */
+ quoteUnverified: "quote_unverified",
} as const);
/** Closed union over {@link TOOL_ERROR_CODE}. */
@@ -366,6 +369,7 @@ const CLASSIFICATION: ReadonlyArray<ClassificationRule> = Object.freeze([
fixed(ExpirationValueError, EXPIRATION_REFUSAL_CODE.invalidValue),
fixed(ExpirationTargetNotFoundError, EXPIRATION_REFUSAL_CODE.targetNotFound),
fixed(InvalidExpirationTargetError, EXPIRATION_REFUSAL_CODE.invalidTarget),
+ fixed(QuoteCheckError, TOOL_ERROR_CODE.quoteUnverified),
]);
/**
diff --git a/tests/cli/brain-distill.test.ts b/tests/cli/brain-distill.test.ts
index 192cf905..d8cf247d 100644
--- a/tests/cli/brain-distill.test.ts
+++ b/tests/cli/brain-distill.test.ts
@@ -3,11 +3,22 @@
* with block-level provenance, supplied as JSON.
*/
-import { afterEach, beforeEach, expect, test } from "bun:test";
-import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
+import { afterEach, beforeEach, describe, expect, test } from "bun:test";
+import {
+ existsSync,
+ mkdirSync,
+ mkdtempSync,
+ readdirSync,
+ readFileSync,
+ rmSync,
+ writeFileSync,
+} from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
+import { nestedCommand } from "../../src/cli/command-manifest.ts";
+import { readExcerptSection } from "../../src/core/brain/provenance/capture-scope.ts";
+import { BRAIN_DISTILLATIONS_REL } from "../../src/core/brain/path-constants.ts";
import { runCli } from "../helpers/run-cli.ts";
let tmp: string;
@@ -132,3 +143,162 @@ test("missing <source> or --claims is a usage error (exit 2)", async () => {
const noClaims = await runCli(["brain", "distill", "Articles/src.md", "--vault", vault], { env });
expect(noClaims.returncode).toBe(2);
});
+
+const QUOTED_SOURCE = "Articles/quoted.md";
+
+/** A source whose one paragraph carries the block id `^p1`. */
+function seedQuotedSource(): void {
+ writeFileSync(
+ join(vault, QUOTED_SOURCE),
+ "# Quoted\n\nThe protocol settles every batch within one minute. ^p1\n",
+ "utf8",
+ );
+}
+
+/** Distillation pages on disk; the directory may not exist yet. */
+function distillationPages(): string[] {
+ const dir = join(vault, BRAIN_DISTILLATIONS_REL);
+ return existsSync(dir) ? readdirSync(dir).filter((name) => name.endsWith(".md")) : [];
+}
+
+/**
+ * The quote check and the capture scope reach the operator (distilled
+ * provenance, D2): two flags in, two suffixes and two `--json` members out.
+ */
+describe("o2b brain distill - quote check and capture scope", () => {
+ const VERBATIM = {
+ text: 'The author writes "settles every batch within one minute".',
+ block: "p1",
+ };
+ const PARAPHRASE = { text: 'The author writes "settles all batches quickly".', block: "p1" };
+
+ test("--strict-quotes refuses a paraphrase in quotation marks and writes nothing", async () => {
+ seedQuotedSource();
+ const res = await runCli(
+ [
+ "brain",
+ "distill",
+ QUOTED_SOURCE,
+ "--claims",
+ JSON.stringify([PARAPHRASE]),
+ "--strict-quotes",
+ "--vault",
+ vault,
+ ],
+ { env },
+ );
+ expect(res.returncode).toBe(1);
+ expect(res.stderr).toContain(
+ "distill: quoted spans failed verification: claim 0: not-in-block",
+ );
+ expect(distillationPages()).toEqual([]);
+ });
+
+ test("--excerpt-file stores a bounded-local page for a url source", async () => {
+ const excerptPath = join(tmp, "excerpt.txt");
+ writeFileSync(excerptPath, "The protocol settles every batch within one minute.\n", "utf8");
+ const res = await runCli(
+ [
+ "brain",
+ "distill",
+ "https://example.test/post",
+ "--claims",
+ JSON.stringify([{ text: "A claim." }]),
+ "--excerpt-file",
+ excerptPath,
+ "--vault",
+ vault,
+ ],
+ { env },
+ );
+ expect(res.returncode).toBe(0);
+ expect(res.stdout.trimEnd()).toEndWith(" [untrusted_source] [bounded-local]");
+ const [page] = distillationPages();
+ const md = readFileSync(join(vault, BRAIN_DISTILLATIONS_REL, page!), "utf8");
+ expect(md).toContain("capture_scope: bounded-local");
+ expect(readExcerptSection(md)).toBe(readFileSync(excerptPath, "utf8"));
+ });
+
+ test("an --excerpt-file that is not valid UTF-8 is refused before any write", async () => {
+ // Decoding leniently would store U+FFFD in place of the file's bytes,
+ // and the stored excerpt would no longer be the captured text.
+ const excerptPath = join(tmp, "latin1.txt");
+ writeFileSync(excerptPath, Buffer.from([0x63, 0x61, 0x66, 0xe9, 0x0a]));
+ const res = await runCli(
+ [
+ "brain",
+ "distill",
+ "https://example.test/post",
+ "--claims",
+ JSON.stringify([{ text: "A claim." }]),
+ "--excerpt-file",
+ excerptPath,
+ "--vault",
+ vault,
+ ],
+ { env },
+ );
+ expect(res.returncode).toBe(2);
+ expect(res.stderr).toContain("distill: excerpt file is not valid UTF-8");
+ expect(distillationPages()).toEqual([]);
+ });
+
+ test("a checked quote adds the quotes suffix to the human line", async () => {
+ seedQuotedSource();
+ const res = await runCli(
+ ["brain", "distill", QUOTED_SOURCE, "--claims", JSON.stringify([VERBATIM]), "--vault", vault],
+ { env },
+ );
+ expect(res.returncode).toBe(0);
+ expect(res.stdout.trimEnd()).toEndWith(" [quotes verified:1 unquoted:0]");
+ });
+
+ test("--json carries capture_scope and quotes", async () => {
+ seedQuotedSource();
+ const res = await runCli(
+ [
+ "brain",
+ "distill",
+ QUOTED_SOURCE,
+ "--claims",
+ JSON.stringify([VERBATIM, PARAPHRASE]),
+ "--vault",
+ vault,
+ "--json",
+ ],
+ { env },
+ );
+ expect(res.returncode).toBe(0);
+ const out = JSON.parse(res.stdout) as {
+ capture_scope: string;
+ quotes: { verified_in_block: number; unquoted: number; findings: unknown[] };
+ };
+ expect(out.capture_scope).toBe("full-local");
+ expect(out.quotes.verified_in_block).toBe(1);
+ expect(out.quotes.unquoted).toBe(1);
+ expect(out.quotes.findings).toHaveLength(1);
+ });
+
+ test("a clean trusted run keeps its human line exactly", async () => {
+ const res = await runCli(
+ [
+ "brain",
+ "distill",
+ "Articles/src.md",
+ "--claims",
+ JSON.stringify([{ text: "A claim." }]),
+ "--vault",
+ vault,
+ ],
+ { env },
+ );
+ expect(res.returncode).toBe(0);
+ expect(res.stdout.trimEnd()).toMatch(/^distilled 1 claim\(s\) -> \S+\.md$/);
+ });
+
+ test("the manifest entry lists the two new flags", () => {
+ const names = (nestedCommand("brain", "distill")?.flags ?? []).map((f) => f.name);
+ expect(names).toContain("strict-quotes");
+ expect(names).toContain("excerpt-file");
+ });
+});
diff --git a/tests/cli/brain-hygiene.test.ts b/tests/cli/brain-hygiene.test.ts
index 9ba422dd..245911d2 100644
--- a/tests/cli/brain-hygiene.test.ts
+++ b/tests/cli/brain-hygiene.test.ts
@@ -22,6 +22,8 @@ import { bootstrapBrain } from "../../src/core/brain/init.ts";
import { brainDirs } from "../../src/core/brain/paths.ts";
import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
import { resetVaultIdentityPins } from "../../src/core/brain/vault-identity.ts";
+import { HYGIENE_TOOLS } from "../../src/mcp/brain/hygiene-tools.ts";
+import { PROPERTY_DESCRIPTION_MAX, TOOL_DESCRIPTION_MAX } from "../../src/mcp/registry-guard.ts";
import { runCli } from "../helpers/run-cli.ts";
import { changedPaths, digestVaultFiles, digestVaultTree } from "../helpers/vault-digest.ts";
@@ -236,3 +238,39 @@ describe("o2b brain hygiene apply", () => {
expect(out.stderr).toContain("scan|apply");
});
});
+
+/**
+ * The `capture-scope` detector reaches both surfaces (distilled provenance,
+ * D3): it runs in the default sweep, it is a named subset member, and the
+ * MCP schema offers it within the registry caps.
+ */
+describe("the capture-scope detector on the hygiene surfaces", () => {
+ const DETECTOR = "capture-scope";
+
+ test("the default sweep runs it and an explicit subset accepts it", async () => {
+ expect((await scan()).detectors_run).toContain(DETECTOR);
+ expect((await scan("--detectors", DETECTOR)).detectors_run).toEqual([DETECTOR]);
+ });
+
+ test("an unknown detector refusal lists it", async () => {
+ const out = await runCli(
+ ["brain", "hygiene", "scan", "--vault", vault, "--detectors", "not-a-detector"],
+ { env: env() },
+ );
+ expect(out.returncode).not.toBe(0);
+ expect(out.stderr).toContain(DETECTOR);
+ });
+
+ test("the brain_hygiene schema offers it within the registry caps", () => {
+ const tool = HYGIENE_TOOLS[0]!;
+ const props = tool.inputSchema["properties"] as Record<
+ string,
+ { description: string; items?: { enum?: string[] } }
+ >;
+ expect(props["detectors"]!.items!.enum).toContain(DETECTOR);
+ expect(props["detectors"]!.description).toContain(DETECTOR);
+ expect(props["detectors"]!.description.length).toBeLessThanOrEqual(PROPERTY_DESCRIPTION_MAX);
+ expect(tool.description).toContain("url-only");
+ expect(tool.description.length).toBeLessThanOrEqual(TOOL_DESCRIPTION_MAX);
+ });
+});
diff --git a/tests/core/architecture/verdict-vocabulary-census.test.ts b/tests/core/architecture/verdict-vocabulary-census.test.ts
index a35c698f..9a756b9e 100644
--- a/tests/core/architecture/verdict-vocabulary-census.test.ts
+++ b/tests/core/architecture/verdict-vocabulary-census.test.ts
@@ -440,6 +440,16 @@ import {
TRANSPORT_REACHES,
isTransportReach,
} from "../../../src/core/graph/transport-reach.ts";
+import {
+ CAPTURE_SCOPE,
+ CAPTURE_SCOPES,
+ isCaptureScope,
+} from "../../../src/core/brain/provenance/capture-scope.ts";
+import {
+ isQuoteCheckOutcome,
+ QUOTE_CHECK_OUTCOME,
+ QUOTE_CHECK_OUTCOMES,
+} from "../../../src/core/brain/distill/quote-verdict.ts";
interface VocabularyUnderCensus {
/** Identifies the vocabulary in a failure message. */
@@ -1361,6 +1371,24 @@ const CENSUS: ReadonlyArray<VocabularyUnderCensus> = Object.freeze([
members: DISCLOSURE_MODES,
guard: isDisclosureMode,
},
+ {
+ // How much of its source a page holds. Stamped into page frontmatter
+ // and read back by the capture-scope hygiene detector, so a value the
+ // guard rejected would turn a stamped page into an unstamped one.
+ name: "CAPTURE_SCOPE",
+ values: CAPTURE_SCOPE,
+ members: CAPTURE_SCOPES,
+ guard: isCaptureScope,
+ },
+ {
+ // The verdict on one quoted span of a distilled claim. It reaches the
+ // MCP payload and the strict refusal message; its `url-only` member is
+ // the CAPTURE_SCOPE token, spelled once for "no local bytes".
+ name: "QUOTE_CHECK_OUTCOME",
+ values: QUOTE_CHECK_OUTCOME,
+ members: QUOTE_CHECK_OUTCOMES,
+ guard: isQuoteCheckOutcome,
+ },
]);
// ---------------------------------------------------------------------------
@@ -1625,7 +1653,7 @@ const SCANNED = scanVocabularies(SOURCE_TREE);
* How many four-piece vocabularies `src/` currently holds. Measured, and
* kept as an equality rather than a floor - see the population test.
*/
-const VOCABULARY_POPULATION = 81;
+const VOCABULARY_POPULATION = 83;
const REGISTERED = new Map(CENSUS.map((entry) => [entry.name, entry] as const));
describe("verdict vocabulary census", () => {
diff --git a/tests/core/architecture/visibility-surface-census.test.ts b/tests/core/architecture/visibility-surface-census.test.ts
index d0b22e91..adbf32e4 100644
--- a/tests/core/architecture/visibility-surface-census.test.ts
+++ b/tests/core/architecture/visibility-surface-census.test.ts
@@ -242,6 +242,13 @@ const NOTE_CONTENT_PRODUCERS: ReadonlyArray<ProducerRule> = Object.freeze([
identifiers: ["discoverIdeas", "ideaCandidates"],
},
{ specifierIncludes: "/brain/link-graph/moc-audit.ts", identifiers: ["auditMoc"] },
+ // Record diagnostics: the doctor and the schema report name a Brain
+ // record by path when it is malformed, so both hand paths to a caller.
+ { specifierIncludes: "/brain/doctor.ts", identifiers: ["runDoctor"] },
+ {
+ specifierIncludes: "/brain/schema-admin.ts",
+ identifiers: ["buildSchemaLint", "buildSchemaStats", "reviewSchemaOrphans"],
+ },
{ specifierIncludes: "/brain/dead-ends.ts", identifiers: ["listDeadEnds", "recordDeadEnd"] },
{
specifierIncludes: "/brain/claim-graph.ts",
@@ -448,8 +455,14 @@ function reasonProblems(entries: ReadonlyArray<VisibilitySurfaceEntry>): {
* `brain_skill_proposals`, which the sweep could not see and which were
* both leaking, plus the three tools that share `procedure-tools.ts` with
* the second of them and come in on the file-level rule.
+ *
+ * 49 before the vocabulary gained `runDoctor` and the schema-admin record
+ * views. The five new names are `brain_doctor` and `schema_inspect`,
+ * which named a malformed reserved record by path (the doctor's repair
+ * plan still does, so it is registered excluded), plus `brain_health`,
+ * `brain_status` and `schema_apply_mutations` on the file-level rule.
*/
-const MCP_TOOL_POPULATION_SIZE = 49;
+const MCP_TOOL_POPULATION_SIZE = 54;
/** Measured: MCP resources + templates, all excluded. */
const MCP_RESOURCE_POPULATION_SIZE = 8;
/** Measured: hand-enumerated CLI verb mirrors. */
@@ -507,20 +520,27 @@ describe("visibility surface census", () => {
.map((e) => e.surface)
.toSorted();
expect(covered).toEqual([
+ "brain_agent_diff",
+ "brain_agent_query",
"brain_backlinks",
"brain_bridges",
"brain_clusters",
"brain_deep_synthesis",
"brain_eval",
+ "brain_event_trace",
"brain_file_context",
"brain_hygiene",
+ "brain_idea_discovery",
+ "brain_procedural_memory",
"brain_query",
"brain_recall_feedback",
"brain_search",
"brain_search_expand",
"brain_skill_proposals",
+ "brain_tiers",
"brain_unlinked_mentions",
"brain_writes",
+ "schema_inspect",
"second_brain_query",
]);
});
diff --git a/tests/core/brain/artifact-ref-view.test.ts b/tests/core/brain/artifact-ref-view.test.ts
index ab70fb5a..5e3179b9 100644
--- a/tests/core/brain/artifact-ref-view.test.ts
+++ b/tests/core/brain/artifact-ref-view.test.ts
@@ -117,3 +117,19 @@ describe("what a log entry offers the rule", () => {
expect(hidingView().keep(rows, logEntryArtifactRefs)).toEqual(rows);
});
});
+
+describe("a reference that leaves the vault", () => {
+ test("is hidden without asking the rule, so nothing beside the vault is read", () => {
+ const asked: string[] = [];
+ const view = artifactRefView(vault, (rel) => {
+ asked.push(rel);
+ return true;
+ });
+ expect(view.visible("../outside.md")).toBe(false);
+ expect(view.visible("[[../../outside.md]]")).toBe(false);
+ expect(view.visible("Notes/../../outside.md")).toBe(false);
+ expect(asked).toEqual([]);
+ expect(view.visible("notes/inside.md")).toBe(true);
+ expect(asked).toEqual(["notes/inside.md"]);
+ });
+});
diff --git a/tests/core/brain/code-regions.test.ts b/tests/core/brain/code-regions.test.ts
new file mode 100644
index 00000000..e905bd1c
--- /dev/null
+++ b/tests/core/brain/code-regions.test.ts
@@ -0,0 +1,107 @@
+/**
+ * The code regions every link and mention reader masks out
+ * (`codeRegions` / `maskCodeRegions` in src/core/brain/wikilink.ts).
+ *
+ * The scanner replaced a regular expression with the same meaning. The
+ * expression is kept here as the reference, and the scanner must agree with
+ * it on every input; it must also stay linear where the expression did not
+ * (unclosed fence runs of strictly decreasing length).
+ */
+
+import { describe, expect, test } from "bun:test";
+
+import { codeRegions, maskCodeRegions } from "../../../src/core/brain/wikilink.ts";
+import { normalizeWikilinks } from "../../../src/core/brain/link-graph/format-wikilink.ts";
+import { extractWikilinks } from "../../../src/core/vault.ts";
+import { LINEAR_CEILING_MS } from "../../helpers/linear-time.ts";
+
+/**
+ * The reference the scanner is held to: the former definition, with an
+ * inline span that closes only on a backtick run of its own length (the
+ * CommonMark rule), so no mark of an unclosed run opens a span.
+ */
+const REFERENCE_RE =
+ /(?<!`)(`{3,})(?!`)[\s\S]*?(?<!`)\1`*(?!`)|(?<!~)(~{3,})(?!~)[\s\S]*?(?<!~)\2~*(?!~)|(?<!`)(`{1,2})(?!`)[\s\S]*?(?<!`)\3(?!`)/g;
+
+const ALPHABET = "`~x\n[]";
+const RANDOM_CASES = 5000;
+const RANDOM_MAX_LENGTH = 40;
+const SEED = 0x9e3779b9;
+
+/** A small deterministic PRNG (mulberry32), so a failure reproduces. */
+function prng(seed: number): () => number {
+ let a = seed >>> 0;
+ return () => {
+ a = (a + 0x6d2b79f5) >>> 0;
+ let t = a;
+ t = Math.imul(t ^ (t >>> 15), t | 1);
+ t ^= t + Math.imul(t ^ (t >>> 7), t | 61);
+ return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
+ };
+}
+
+function referenceRegions(text: string): Array<[number, number]> {
+ return [...text.matchAll(REFERENCE_RE)].map((m) => [m.index, m.index + m[0].length]);
+}
+
+/** Runs of `ch` of strictly decreasing length from `longest` down to 3, one per line. */
+function decreasingRuns(ch: string, longest: number): string {
+ const runs: string[] = [];
+ for (let len = longest; len >= 3; len--) runs.push(ch.repeat(len));
+ return runs.join("\nx\n");
+}
+
+/** About 4 MiB: the largest run length whose decreasing series fits. */
+const DECREASING_LONGEST = 2_896;
+
+describe("codeRegions", () => {
+ test("agrees with the reference expression on random short inputs", () => {
+ const next = prng(SEED);
+ for (let n = 0; n < RANDOM_CASES; n++) {
+ const length = Math.floor(next() * RANDOM_MAX_LENGTH);
+ let text = "";
+ for (let i = 0; i < length; i++) text += ALPHABET[Math.floor(next() * ALPHABET.length)];
+ expect({ text, regions: codeRegions(text) }).toEqual({
+ text,
+ regions: referenceRegions(text),
+ });
+ expect(maskCodeRegions(text, (s) => "#".repeat(s.length))).toBe(
+ text.replace(REFERENCE_RE, (s) => "#".repeat(s.length)),
+ );
+ }
+ });
+
+ test.each([
+ ["a longer closer ends the fence", "```\n[[in]]\n`````\n[[out]]"],
+ ["a shorter run inside a longer fence stays masked", "````\n```\n[[in]]\n````\n[[out]]"],
+ ["a tilde fence closes on a longer tilde run", "~~~~\n[[a]]\n~~~\n[[b]]\n~~~~ [[out]]"],
+ ["an unclosed fence is not a region", "~~~\n[[kept]]"],
+ ["no mark of an unclosed run opens an inline span", "```x`[[out]]"],
+ ["a double-backtick span closes on a double run", "``a `b` [[in]]``[[out]]"],
+ ["strikethrough is not a fence", "~~strike~~ [[out]]"],
+ ])("%s", (_name, text) => {
+ expect(codeRegions(text)).toEqual(referenceRegions(text));
+ });
+
+ test("an unclosed backtick run leaves the links after it unmasked", () => {
+ const text = "Intro ```\nsee [[Alpha]]\nthen `x`";
+ const single = text.indexOf("`x`");
+ expect(codeRegions(text)).toEqual([[single, single + 3]]);
+ expect(extractWikilinks(text)).toEqual(["Alpha"]);
+ });
+
+ test.each([
+ ["tilde", "~"],
+ ["backtick", "`"],
+ ])("stays linear on unclosed %s runs of decreasing length", (_name, ch) => {
+ const content = `${decreasingRuns(ch, DECREASING_LONGEST)}\n[[after]]`;
+ let started = performance.now();
+ const links = extractWikilinks(content);
+ expect(performance.now() - started).toBeLessThan(LINEAR_CEILING_MS);
+ expect(links).toEqual(["after"]);
+
+ started = performance.now();
+ normalizeWikilinks(content, "full", ["after.md"]);
+ expect(performance.now() - started).toBeLessThan(LINEAR_CEILING_MS);
+ });
+});
diff --git a/tests/core/brain/distill/block-resolve.test.ts b/tests/core/brain/distill/block-resolve.test.ts
new file mode 100644
index 00000000..919aeea9
--- /dev/null
+++ b/tests/core/brain/distill/block-resolve.test.ts
@@ -0,0 +1,117 @@
+/**
+ * Block resolver: an Obsidian block id (`^id`) to the text of the block it
+ * marks inside the source bytes. Pure; the quote check compares a quoted span
+ * against what this returns.
+ */
+
+import { describe, expect, test } from "bun:test";
+
+import {
+ indexBlocks,
+ lookupBlock,
+ resolveBlock,
+} from "../../../../src/core/brain/distill/block-resolve.ts";
+
+describe("resolveBlock", () => {
+ test("a paragraph whose last line ends in ` ^id` resolves to the whole paragraph without the marker", () => {
+ const source = [
+ "# Title",
+ "",
+ "First line of the paragraph",
+ "and its second line. ^p1",
+ "",
+ "Another paragraph.",
+ ].join("\n");
+ expect(resolveBlock(source, "p1")).toEqual({
+ kind: "found",
+ text: "First line of the paragraph\nand its second line.",
+ });
+ });
+
+ test("a list item ending in ` ^id` resolves to that item only", () => {
+ const source = ["- first item", "- second item ^li", "- third item"].join("\n");
+ expect(resolveBlock(source, "li")).toEqual({ kind: "found", text: "- second item" });
+ });
+
+ test("a numbered line inside a paragraph does not start a list item", () => {
+ const source = "Intro line\n1999) was a year ^p1";
+ expect(resolveBlock(source, "p1")).toEqual({
+ kind: "found",
+ text: "Intro line\n1999) was a year",
+ });
+ });
+
+ test("a bullet directly under a paragraph line still starts a list", () => {
+ const source = "Key points:\n- first\n- second ^li";
+ expect(resolveBlock(source, "li")).toEqual({ kind: "found", text: "- second" });
+ });
+
+ test("one index answers every id exactly as resolving each id does", () => {
+ const source = "Para one ^a\n\n- x ^b\n- y ^b\n\n| t |\n^c\n\n```\nz ^d\n```";
+ const index = indexBlocks(source);
+ for (const id of ["a", "b", "c", "d", "missing"]) {
+ expect(lookupBlock(index, id)).toEqual(resolveBlock(source, id));
+ }
+ });
+
+ test("a standalone `^id` line after a table resolves to the table", () => {
+ const table = ["| a | b |", "| - | - |", "| 1 | 2 |"].join("\n");
+ const source = ["Intro.", "", table, "", "^t1", "", "Outro."].join("\n");
+ expect(resolveBlock(source, "t1")).toEqual({ kind: "found", text: table });
+ });
+
+ test("a standalone `^id` line after a blockquote resolves to the blockquote", () => {
+ const quote = ["> quoted line one", "> quoted line two"].join("\n");
+ const source = [quote, "", "^q1"].join("\n");
+ expect(resolveBlock(source, "q1")).toEqual({ kind: "found", text: quote });
+ });
+
+ test("an id inside a fenced code block is not a definition", () => {
+ const source = ["```", "code line ^c1", "^c2", "```", "", "~~~", "tilde ^c3", "~~~"].join("\n");
+ expect(resolveBlock(source, "c1")).toEqual({ kind: "not-found" });
+ expect(resolveBlock(source, "c2")).toEqual({ kind: "not-found" });
+ expect(resolveBlock(source, "c3")).toEqual({ kind: "not-found" });
+ });
+
+ test("an id nobody defined is not found", () => {
+ expect(resolveBlock("Plain text with no ids.\n", "abc")).toEqual({ kind: "not-found" });
+ });
+
+ test("two definitions of one id are ambiguous", () => {
+ const source = ["One. ^dup", "", "Two. ^dup"].join("\n");
+ expect(resolveBlock(source, "dup")).toEqual({ kind: "ambiguous" });
+ });
+
+ test("an id that is a prefix of another does not match it", () => {
+ const source = ["Longer id here. ^ab", "", "Shorter id here. ^a"].join("\n");
+ expect(resolveBlock(source, "ab")).toEqual({ kind: "found", text: "Longer id here." });
+ expect(resolveBlock(source, "a")).toEqual({ kind: "found", text: "Shorter id here." });
+ expect(resolveBlock(["Only the long one. ^ab"].join("\n"), "a")).toEqual({
+ kind: "not-found",
+ });
+ });
+
+ test("a caret glued to a word is not a block marker", () => {
+ expect(resolveBlock("x^p1\n", "p1")).toEqual({ kind: "not-found" });
+ });
+
+ test("CRLF sources resolve exactly as LF sources do", () => {
+ const lf = ["Para line one", "line two. ^p1", "", "| a |", "| - |", "", "^t1"].join("\n");
+ const crlf = lf.replaceAll("\n", "\r\n");
+ expect(resolveBlock(crlf, "p1")).toEqual(resolveBlock(lf, "p1"));
+ expect(resolveBlock(crlf, "t1")).toEqual(resolveBlock(lf, "t1"));
+ expect(resolveBlock(crlf, "p1")).toEqual({ kind: "found", text: "Para line one\nline two." });
+ });
+
+ test("a standalone `^id` after a fenced block resolves to the fence's inner text", () => {
+ const source = ["```js", "const a = 1;", "", "const b = 2;", "```", "^code1"].join("\n");
+ expect(resolveBlock(source, "code1")).toEqual({
+ kind: "found",
+ text: "const a = 1;\n\nconst b = 2;",
+ });
+ });
+
+ test("a standalone `^id` with no block above it defines nothing", () => {
+ expect(resolveBlock("^lonely\n\nText.\n", "lonely")).toEqual({ kind: "not-found" });
+ });
+});
diff --git a/tests/core/brain/distill/distill-source-capture.test.ts b/tests/core/brain/distill/distill-source-capture.test.ts
new file mode 100644
index 00000000..78269ca6
--- /dev/null
+++ b/tests/core/brain/distill/distill-source-capture.test.ts
@@ -0,0 +1,161 @@
+/**
+ * Capture scope on distillation pages: how much of the source the vault
+ * actually holds. A local source adds nothing (byte-identical page); a URL
+ * adds `capture_scope: url-only`; a URL plus a caller-supplied verbatim
+ * excerpt is `bounded-local`, and its quotes are checked against the excerpt.
+ */
+
+import { afterEach, beforeEach, describe, expect, test } from "bun:test";
+import {
+ existsSync,
+ mkdirSync,
+ mkdtempSync,
+ readFileSync,
+ readdirSync,
+ rmSync,
+ writeFileSync,
+} from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { bootstrapBrain } from "../../../../src/core/brain/init.ts";
+import {
+ distillSource,
+ type DistillClaim,
+ type DistillSourceResult,
+} from "../../../../src/core/brain/distill/distill-source.ts";
+import { QUOTE_CHECK_OUTCOME } from "../../../../src/core/brain/distill/quote-verdict.ts";
+import { BRAIN_DISTILLATIONS_REL } from "../../../../src/core/brain/paths.ts";
+import {
+ CAPTURE_EXCERPT_MAX_BYTES,
+ CAPTURE_SCOPE,
+ CAPTURE_SCOPE_KEY,
+ CaptureExcerptError,
+ EXCERPT_HASH_KEY,
+ excerptDigest,
+ readExcerptSection,
+} from "../../../../src/core/brain/provenance/capture-scope.ts";
+import { hasUntrustedSourceMarker } from "../../../../src/core/brain/trust/untrusted-provenance.ts";
+import { parseFrontmatter } from "../../../../src/core/vault.ts";
+
+let vault: string;
+const NOW = new Date("2026-07-10T08:00:00Z");
+const LATER = new Date("2026-07-11T09:00:00Z");
+const LOCAL = "Articles/restaking.md";
+const URL_SOURCE = "https://example.test/restaking";
+const EXCERPT = "Restaking reuses staked capital to secure more services. ^abc\n\nIt adds risk.\n";
+
+beforeEach(() => {
+ vault = mkdtempSync(join(tmpdir(), "o2b-distill-capture-"));
+ bootstrapBrain(vault);
+ mkdirSync(join(vault, "Articles"), { recursive: true });
+ writeFileSync(join(vault, LOCAL), "# Restaking\n\nBody text.\n", "utf8");
+});
+afterEach(() => {
+ rmSync(vault, { recursive: true, force: true });
+});
+
+function distill(
+ sourcePath: string,
+ claims: ReadonlyArray<DistillClaim>,
+ extra: { excerpt?: string; now?: Date } = {},
+): DistillSourceResult {
+ return distillSource(
+ vault,
+ {
+ sourcePath,
+ claims,
+ ...(extra.excerpt !== undefined ? { excerpt: extra.excerpt } : {}),
+ },
+ { agent: "claude", now: extra.now ?? NOW },
+ );
+}
+
+function page(res: DistillSourceResult): string {
+ return readFileSync(join(vault, res.distillationPath), "utf8");
+}
+
+function meta(res: DistillSourceResult): Record<string, unknown> {
+ return parseFrontmatter(join(vault, res.distillationPath))[0];
+}
+
+/** Distillation pages on disk. An absent directory is zero pages, not a throw. */
+function distillationPages(): string[] {
+ const dir = join(vault, BRAIN_DISTILLATIONS_REL);
+ return existsSync(dir) ? readdirSync(dir) : [];
+}
+
+describe("distillSource - capture scope", () => {
+ test("a local source is full-local and adds no capture key", () => {
+ const res = distill(LOCAL, [{ text: "Plain claim.", block: "abc" }]);
+ expect(res.captureScope).toBe(CAPTURE_SCOPE.fullLocal);
+ expect(meta(res)[CAPTURE_SCOPE_KEY]).toBeUndefined();
+ expect(meta(res)[EXCERPT_HASH_KEY]).toBeUndefined();
+ expect(page(res)).not.toContain("## Excerpt");
+ });
+
+ test("a URL source is url-only beside the untrusted marker, and its quotes are unquoted", () => {
+ const res = distill(URL_SOURCE, [{ text: "It “reuses staked capital”." }]);
+ expect(res.captureScope).toBe(CAPTURE_SCOPE.urlOnly);
+ expect(meta(res)[CAPTURE_SCOPE_KEY]).toBe(CAPTURE_SCOPE.urlOnly);
+ expect(hasUntrustedSourceMarker(meta(res))).toBe(true);
+ expect(page(res)).toContain("- It reuses staked capital.");
+ expect(res.quotes?.findings).toEqual([
+ { claim: 0, outcome: QUOTE_CHECK_OUTCOME.urlOnly, span: "reuses staked capital" },
+ ]);
+ });
+
+ test("a URL source with an excerpt is bounded-local, stores the excerpt verbatim and checks against it", () => {
+ const res = distill(
+ URL_SOURCE,
+ [{ text: "It “reuses staked capital”.", block: "abc" }, { text: "Also “It adds risk.”" }],
+ { excerpt: EXCERPT },
+ );
+ expect(res.captureScope).toBe(CAPTURE_SCOPE.boundedLocal);
+ expect(meta(res)[CAPTURE_SCOPE_KEY]).toBe(CAPTURE_SCOPE.boundedLocal);
+ expect(meta(res)[EXCERPT_HASH_KEY]).toBe(excerptDigest(EXCERPT));
+ expect(hasUntrustedSourceMarker(meta(res))).toBe(true);
+ expect(readExcerptSection(page(res))).toBe(EXCERPT);
+ expect(res.quotes?.verified_in_block).toBe(1);
+ expect(res.quotes?.verified_in_source).toBe(1);
+ expect(res.quotes?.unquoted).toBe(0);
+ expect(page(res)).toContain("- It “reuses staked capital”.");
+ });
+
+ test("an excerpt for a local source is refused by name and writes nothing", () => {
+ expect(() => distill(LOCAL, [{ text: "Plain claim." }], { excerpt: EXCERPT })).toThrow(
+ CaptureExcerptError,
+ );
+ expect(distillationPages()).toEqual([]);
+ });
+
+ test("an excerpt over the byte cap is refused and writes nothing", () => {
+ const over = "é".repeat(CAPTURE_EXCERPT_MAX_BYTES / 2 + 1);
+ expect(() => distill(URL_SOURCE, [{ text: "Plain claim." }], { excerpt: over })).toThrow(
+ CaptureExcerptError,
+ );
+ expect(distillationPages()).toEqual([]);
+ });
+
+ test("re-running with the same excerpt is inert", () => {
+ const claims = [{ text: "It “reuses staked capital”.", block: "abc" }];
+ const first = distill(URL_SOURCE, claims, { excerpt: EXCERPT });
+ const before = page(first);
+ expect(readExcerptSection(before)).toBe(EXCERPT);
+ const second = distill(URL_SOURCE, claims, { excerpt: EXCERPT, now: LATER });
+ expect(second.created).toBe(false);
+ expect(second.captureScope).toBe(CAPTURE_SCOPE.boundedLocal);
+ expect(page(second)).toBe(before);
+ });
+
+ test("a CRLF excerpt holding a `---` line is stored verbatim and its digest still matches", () => {
+ const excerpt = "---\r\ntitle: not frontmatter\r\n---\r\nQuoted line here. ^q\r\n";
+ const res = distill(URL_SOURCE, [{ text: "It says “Quoted line here.”", block: "q" }], {
+ excerpt,
+ });
+ expect(res.quotes?.verified_in_block).toBe(1);
+ expect(meta(res)["title"]).toBeUndefined();
+ expect(readExcerptSection(page(res))).toBe(excerpt);
+ expect(meta(res)[EXCERPT_HASH_KEY]).toBe(excerptDigest(excerpt));
+ });
+});
diff --git a/tests/core/brain/distill/distill-source-quotes.test.ts b/tests/core/brain/distill/distill-source-quotes.test.ts
new file mode 100644
index 00000000..81b03a4f
--- /dev/null
+++ b/tests/core/brain/distill/distill-source-quotes.test.ts
@@ -0,0 +1,233 @@
+/**
+ * The quote check inside `distillSource`: a quoted span in a claim is
+ * compared with the source bytes the page's digest covers. A failed span is
+ * unquoted on the page and named in the result; `strictQuotes` refuses the
+ * whole write instead.
+ */
+
+import { afterEach, beforeEach, describe, expect, spyOn, test } from "bun:test";
+import * as fs from "node:fs";
+import {
+ existsSync,
+ mkdirSync,
+ mkdtempSync,
+ readFileSync,
+ readdirSync,
+ rmSync,
+ writeFileSync,
+} from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { bootstrapBrain } from "../../../../src/core/brain/init.ts";
+import {
+ distillSource,
+ type DistillClaim,
+ type DistillSourceResult,
+} from "../../../../src/core/brain/distill/distill-source.ts";
+import {
+ QUOTE_CHECK_OUTCOME,
+ QUOTE_UNVERIFIED_CODE,
+ QUOTES_UNQUOTED_KEY,
+ QUOTES_VERIFIED_KEY,
+ QuoteCheckError,
+} from "../../../../src/core/brain/distill/quote-verdict.ts";
+import { hashFile } from "../../../../src/core/brain/ingest/content-manifest.ts";
+import { BRAIN_DISTILLATIONS_REL } from "../../../../src/core/brain/paths.ts";
+import { parseFrontmatter } from "../../../../src/core/vault.ts";
+
+let vault: string;
+const NOW = new Date("2026-07-10T08:00:00Z");
+const LATER = new Date("2026-07-11T09:00:00Z");
+const SOURCE = "Articles/restaking.md";
+const SOURCE_BYTES =
+ "# Restaking\n\nRestaking reuses staked capital to secure more services. ^abc\n";
+
+const VERBATIM: DistillClaim = { text: "It “reuses staked capital” for more.", block: "abc" };
+const PARAPHRASE: DistillClaim = { text: "It “recycles staked capital” for more.", block: "abc" };
+
+beforeEach(() => {
+ vault = mkdtempSync(join(tmpdir(), "o2b-distill-quotes-"));
+ bootstrapBrain(vault);
+ mkdirSync(join(vault, "Articles"), { recursive: true });
+ writeFileSync(join(vault, SOURCE), SOURCE_BYTES, "utf8");
+});
+afterEach(() => {
+ rmSync(vault, { recursive: true, force: true });
+});
+
+function distill(
+ claims: ReadonlyArray<DistillClaim>,
+ opts: { now?: Date; strictQuotes?: boolean } = {},
+): DistillSourceResult {
+ return distillSource(
+ vault,
+ { sourcePath: SOURCE, claims },
+ {
+ agent: "claude",
+ now: opts.now ?? NOW,
+ ...(opts.strictQuotes !== undefined ? { strictQuotes: opts.strictQuotes } : {}),
+ },
+ );
+}
+
+function page(res: DistillSourceResult): string {
+ return readFileSync(join(vault, res.distillationPath), "utf8");
+}
+
+function meta(res: DistillSourceResult): Record<string, unknown> {
+ return parseFrontmatter(join(vault, res.distillationPath))[0];
+}
+
+describe("distillSource - quote check", () => {
+ test("a block-cited verbatim quote keeps its marks and is counted as verified", () => {
+ const res = distill([VERBATIM, { text: "No quote here." }]);
+ expect(page(res)).toContain(
+ "- It “reuses staked capital” for more. ([[Articles/restaking.md#^abc]])",
+ );
+ expect(page(res)).toContain(`${QUOTES_VERIFIED_KEY}: 1\n`);
+ expect(page(res)).toContain(`${QUOTES_UNQUOTED_KEY}: 0\n`);
+ expect(res.quotes?.verified_in_block).toBe(1);
+ expect(res.quotes?.unquoted).toBe(0);
+ });
+
+ test("a paraphrase in quotation marks is written without the marks and named", () => {
+ const res = distill([PARAPHRASE]);
+ const md = page(res);
+ expect(md).toContain("- It recycles staked capital for more. ([[Articles/restaking.md#^abc]])");
+ expect(md).not.toContain("“recycles");
+ expect(page(res)).toContain(`${QUOTES_VERIFIED_KEY}: 0\n`);
+ expect(page(res)).toContain(`${QUOTES_UNQUOTED_KEY}: 1\n`);
+ expect(res.quotes?.findings).toEqual([
+ { claim: 0, outcome: QUOTE_CHECK_OUTCOME.notInBlock, span: "recycles staked capital" },
+ ]);
+ });
+
+ test("strictQuotes refuses a paraphrase with QuoteCheckError and writes nothing", () => {
+ const dir = join(vault, BRAIN_DISTILLATIONS_REL);
+ const dirBefore = existsSync(dir);
+ let thrown: unknown;
+ try {
+ distill([PARAPHRASE], { strictQuotes: true });
+ } catch (err) {
+ thrown = err;
+ }
+ expect(thrown).toBeInstanceOf(QuoteCheckError);
+ expect((thrown as QuoteCheckError).code).toBe(QUOTE_UNVERIFIED_CODE);
+ expect((thrown as Error).message).toContain("claim 0: not-in-block");
+ expect(existsSync(dir)).toBe(dirBefore);
+ if (dirBefore) expect(readdirSync(dir)).toEqual([]);
+ });
+
+ test("strictQuotes lets a fully verified page through", () => {
+ const res = distill([VERBATIM], { strictQuotes: true });
+ expect(res.quotes?.verified_in_block).toBe(1);
+ });
+
+ test("re-running the same input is inert, unquoted page included", () => {
+ const first = distill([PARAPHRASE]);
+ const before = page(first);
+ expect(before).toContain("- It recycles staked capital for more.");
+ const second = distill([PARAPHRASE], { now: LATER });
+ expect(second.quotes?.unquoted).toBe(1);
+ expect(second.created).toBe(false);
+ expect(page(second)).toBe(before);
+ });
+
+ test("a page with no quoted spans carries no quote keys and no quotes result", () => {
+ const res = distill([{ text: "Plain claim.", block: "abc" }]);
+ expect(res.quotes).toBeUndefined();
+ expect(meta(res)[QUOTES_VERIFIED_KEY]).toBeUndefined();
+ expect(meta(res)[QUOTES_UNQUOTED_KEY]).toBeUndefined();
+ });
+
+ test("the verdict follows the bytes whose digest is on the page", () => {
+ const first = distill([VERBATIM]);
+ expect(first.quotes?.verified_in_block).toBe(1);
+
+ writeFileSync(join(vault, SOURCE), "# Restaking\n\nSomething else entirely. ^abc\n", "utf8");
+ const second = distill([VERBATIM], { now: LATER });
+ expect(second.quotes?.unquoted).toBe(1);
+ expect(second.sourceHash).toBe(hashFile(join(vault, SOURCE)));
+ expect(meta(second)["source_hash"]).toBe(second.sourceHash);
+ expect(page(second)).toContain("- It reuses staked capital for more.");
+ });
+
+ test("a source that is not valid UTF-8 gives every span source-not-text", () => {
+ writeFileSync(join(vault, SOURCE), Buffer.from([0x23, 0x20, 0xff, 0xfe, 0x0a]));
+ const res = distill([VERBATIM]);
+ expect(res.quotes?.findings.map((f) => f.outcome)).toEqual([QUOTE_CHECK_OUTCOME.sourceNotText]);
+ });
+});
+
+/** How many of the recorded fs calls named the source file, on any platform's separators. */
+const touches = (calls: ReadonlyArray<ReadonlyArray<unknown>>): number =>
+ calls.filter((args) => String(args[0]).replaceAll("\\", "/").endsWith(SOURCE)).length;
+
+describe("distillSource - one read of the source", () => {
+ test("the source file is opened or read exactly once per call", () => {
+ const open = spyOn(fs, "openSync");
+ const read = spyOn(fs, "readFileSync");
+ const bunRead = spyOn(Bun, "file");
+ try {
+ distill([VERBATIM]);
+ expect(
+ touches(open.mock.calls) + touches(read.mock.calls) + touches(bunRead.mock.calls),
+ ).toBe(1);
+ } finally {
+ open.mockRestore();
+ read.mockRestore();
+ bunRead.mockRestore();
+ }
+ });
+});
+
+/** One distillation of the source with a predicate that refuses it. */
+const hidden = (claims: ReadonlyArray<DistillClaim>, strictQuotes = false): DistillSourceResult =>
+ distillSource(
+ vault,
+ { sourcePath: SOURCE, claims },
+ { agent: "claude", now: NOW, strictQuotes, readable: (rel) => rel !== SOURCE },
+ );
+
+/**
+ * The caller's reach decides whether the source's bytes may answer the
+ * check. A source the predicate refuses is checked as one with no local
+ * bytes, and its digest is neither returned nor recorded.
+ */
+describe("distillSource - a source the caller may not read", () => {
+ test("every span settles url-only and no digest is returned or written", () => {
+ const res = hidden([VERBATIM, PARAPHRASE]);
+ expect(res.quotes?.findings.map((f) => f.outcome)).toEqual([
+ QUOTE_CHECK_OUTCOME.urlOnly,
+ QUOTE_CHECK_OUTCOME.urlOnly,
+ ]);
+ expect(res.sourceHash).toBeUndefined();
+ expect(meta(res)["source_hash"]).toBeUndefined();
+ expect(meta(res)["source_content_hash"]).toBeUndefined();
+ });
+
+ test("strict mode refuses a verbatim span and a paraphrase alike", () => {
+ let thrown: unknown;
+ try {
+ hidden([VERBATIM, PARAPHRASE], true);
+ } catch (err) {
+ thrown = err;
+ }
+ expect(thrown).toBeInstanceOf(QuoteCheckError);
+ expect((thrown as QuoteCheckError).findings.map((f) => f.outcome)).toEqual([
+ QUOTE_CHECK_OUTCOME.urlOnly,
+ QUOTE_CHECK_OUTCOME.urlOnly,
+ ]);
+ });
+
+ test("a predicate that admits the source changes nothing", () => {
+ const res = distillSource(
+ vault,
+ { sourcePath: SOURCE, claims: [VERBATIM] },
+ { agent: "claude", now: NOW, readable: () => true },
+ );
+ expect(res.quotes?.verified_in_block).toBe(1);
+ expect(res.sourceHash).toBe(hashFile(join(vault, SOURCE)));
+ });
+});
diff --git a/tests/core/brain/distill/distill-source.test.ts b/tests/core/brain/distill/distill-source.test.ts
index 7ee32820..d4b0b0e2 100644
--- a/tests/core/brain/distill/distill-source.test.ts
+++ b/tests/core/brain/distill/distill-source.test.ts
@@ -5,14 +5,24 @@
import { afterEach, beforeEach, describe, expect, test } from "bun:test";
import { createHash } from "node:crypto";
-import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
+import {
+ existsSync,
+ mkdirSync,
+ mkdtempSync,
+ readdirSync,
+ readFileSync,
+ rmSync,
+ writeFileSync,
+} from "node:fs";
import { join } from "node:path";
import { tmpdir } from "node:os";
import {
+ DISTILL_CLAIMS_MAX,
distillSource,
DistillValidationError,
} from "../../../../src/core/brain/distill/distill-source.ts";
+import { BRAIN_DISTILLATIONS_REL } from "../../../../src/core/brain/paths.ts";
import { bootstrapBrain } from "../../../../src/core/brain/init.ts";
let vault: string;
@@ -88,6 +98,45 @@ describe("distillSource", () => {
).toThrow(DistillValidationError);
});
+ test("refuses a claim that spans more than one line, writing nothing", () => {
+ for (const text of ["x\n## Sources\n- [[Notes/any.md]]", "one\rtwo"]) {
+ expect(() =>
+ distillSource(
+ vault,
+ { sourcePath: "Articles/restaking.md", claims: [{ text: "fine" }, { text }] },
+ { agent: "claude", now: NOW },
+ ),
+ ).toThrow(
+ new DistillValidationError("claim 1 spans more than one line; a claim is one line"),
+ );
+ }
+ const dir = join(vault, BRAIN_DISTILLATIONS_REL);
+ expect(existsSync(dir) ? readdirSync(dir) : []).toEqual([]);
+ });
+
+ test("refuses more claims than the per-call cap, naming the cap and the count", () => {
+ const claims = Array.from({ length: DISTILL_CLAIMS_MAX + 1 }, (_, i) => ({ text: `c${i}` }));
+ expect(() =>
+ distillSource(
+ vault,
+ { sourcePath: "Articles/restaking.md", claims },
+ { agent: "claude", now: NOW },
+ ),
+ ).toThrow(
+ new DistillValidationError(
+ `distillation accepts at most ${DISTILL_CLAIMS_MAX} claims per call, got ${DISTILL_CLAIMS_MAX + 1}`,
+ ),
+ );
+ const atCap = claims.slice(0, DISTILL_CLAIMS_MAX);
+ expect(
+ distillSource(
+ vault,
+ { sourcePath: "Articles/restaking.md", claims: atCap },
+ { agent: "claude", now: NOW },
+ ).claimCount,
+ ).toBe(DISTILL_CLAIMS_MAX);
+ });
+
test("rejects a malformed block id", () => {
expect(() =>
distillSource(
diff --git a/tests/core/brain/distill/quote-check.test.ts b/tests/core/brain/distill/quote-check.test.ts
new file mode 100644
index 00000000..7828ade6
--- /dev/null
+++ b/tests/core/brain/distill/quote-check.test.ts
@@ -0,0 +1,265 @@
+/**
+ * The pure quote check: every quoted span in a distilled claim is compared
+ * with the block the claim cites (or, without a block, the whole source). A
+ * failed span loses its marks and is named; nothing here touches the disk.
+ */
+
+import { describe, expect, spyOn, test } from "bun:test";
+
+import type { DistillClaim } from "../../../../src/core/brain/distill/distill-source.ts";
+import {
+ checkClaimQuotes,
+ type QuoteEvidence,
+} from "../../../../src/core/brain/distill/quote-check.ts";
+import {
+ QUOTE_CHECK_OUTCOME,
+ QUOTE_FINDINGS_MAX,
+ QUOTE_SPAN_PREVIEW_MAX_CHARS,
+} from "../../../../src/core/brain/distill/quote-verdict.ts";
+import * as blockResolve from "../../../../src/core/brain/distill/block-resolve.ts";
+
+const SOURCE = [
+ "# Restaking",
+ "",
+ "Restaking reuses staked capital to secure additional services. ^abc",
+ "",
+ "It introduces correlated **slashing** risk across",
+ "every service at once. ^risk",
+ "",
+ 'He said "stake it" twice. ^said',
+ "",
+ "The first step, then the second step, and the final step. ^steps",
+ "",
+ "Duplicate one. ^dup",
+ "",
+ "Duplicate two. ^dup",
+ "",
+ "Café prices rose.",
+].join("\n");
+
+const TEXT: QuoteEvidence = { kind: "text", text: SOURCE };
+
+function one(claim: DistillClaim, evidence: QuoteEvidence = TEXT) {
+ return checkClaimQuotes({ claims: [claim], evidence });
+}
+
+describe("checkClaimQuotes - block-cited claims", () => {
+ test("a verbatim span in a claim citing an existing block verifies in the block, claim unchanged", () => {
+ const claim = { text: "The article says “reuses staked capital” plainly.", block: "abc" };
+ const res = one(claim);
+ expect(res.claims[0]).toEqual(claim);
+ expect(res.report).not.toBeNull();
+ expect(res.report!.checked).toBe(1);
+ expect(res.report!.verified_in_block).toBe(1);
+ expect(res.report!.verified_in_source).toBe(0);
+ expect(res.report!.unquoted).toBe(0);
+ expect(res.report!.findings).toEqual([]);
+ });
+
+ test("a span differing in one word fails in the block: marks removed, words kept, finding named", () => {
+ const res = one({ text: "The article says “reuses pooled capital” plainly.", block: "abc" });
+ expect(res.claims[0]).toEqual({
+ text: "The article says reuses pooled capital plainly.",
+ block: "abc",
+ });
+ expect(res.report!.unquoted).toBe(1);
+ expect(res.report!.findings).toEqual([
+ { claim: 0, outcome: QUOTE_CHECK_OUTCOME.notInBlock, span: "reuses pooled capital" },
+ ]);
+ });
+
+ test("a span from elsewhere in the source does not verify against the cited block", () => {
+ const res = one({ text: "“correlated slashing risk”", block: "abc" });
+ expect(res.report!.findings[0]!.outcome).toBe(QUOTE_CHECK_OUTCOME.notInBlock);
+ });
+
+ test("a cited block that does not resolve is block-not-found even when the span is elsewhere", () => {
+ const res = one({ text: "“reuses staked capital”", block: "zzz" });
+ expect(res.report!.findings).toEqual([
+ { claim: 0, outcome: QUOTE_CHECK_OUTCOME.blockNotFound, span: "reuses staked capital" },
+ ]);
+ expect(res.claims[0]!.text).toBe("reuses staked capital");
+ });
+
+ test("a block id defined twice is block-ambiguous", () => {
+ const res = one({ text: "“Duplicate one.”", block: "dup" });
+ expect(res.report!.findings[0]!.outcome).toBe(QUOTE_CHECK_OUTCOME.blockAmbiguous);
+ });
+});
+
+describe("checkClaimQuotes - claims without a block", () => {
+ test("a span found anywhere in the source verifies in the source", () => {
+ const res = one({ text: "Risk is “correlated slashing risk” here." });
+ expect(res.report!.verified_in_source).toBe(1);
+ expect(res.report!.verified_in_block).toBe(0);
+ expect(res.report!.findings).toEqual([]);
+ });
+
+ test("a span inside a longer word of the source is not-in-source", () => {
+ const res = checkClaimQuotes({
+ claims: [{ text: 'It is "unsafe" here.' }, { text: 'It is "safe" here.' }],
+ evidence: { kind: "text", text: "The method is unsafe." },
+ });
+ expect(res.report!.verified_in_source).toBe(1);
+ expect(res.report!.unquoted).toBe(1);
+ expect(res.report!.findings).toEqual([
+ { claim: 1, outcome: QUOTE_CHECK_OUTCOME.notInSource, span: "safe" },
+ ]);
+ expect(res.claims[1]!.text).toBe("It is safe here.");
+ });
+
+ test("a span absent from the source is not-in-source", () => {
+ const res = one({ text: "Risk is “uncorrelated slashing risk” here." });
+ expect(res.report!.findings).toEqual([
+ { claim: 0, outcome: QUOTE_CHECK_OUTCOME.notInSource, span: "uncorrelated slashing risk" },
+ ]);
+ expect(res.claims[0]!.text).toBe("Risk is uncorrelated slashing risk here.");
+ });
+});
+
+describe("checkClaimQuotes - evidence without checkable text", () => {
+ test("url-only evidence: every span is unquoted with the url-only outcome", () => {
+ const res = one({ text: "“reuses staked capital” and “anything”" }, { kind: "url-only" });
+ expect(res.claims[0]!.text).toBe("reuses staked capital and anything");
+ expect(res.report!.findings.map((f) => f.outcome)).toEqual([
+ QUOTE_CHECK_OUTCOME.urlOnly,
+ QUOTE_CHECK_OUTCOME.urlOnly,
+ ]);
+ expect(res.report!.unquoted).toBe(2);
+ });
+
+ test("not-text evidence: every span is unquoted with source-not-text", () => {
+ const res = one({ text: "“reuses staked capital”", block: "abc" }, { kind: "not-text" });
+ expect(res.report!.findings[0]!.outcome).toBe(QUOTE_CHECK_OUTCOME.sourceNotText);
+ });
+});
+
+describe("checkClaimQuotes - matching tolerance", () => {
+ test("curly marks inside the span match straight marks in the source", () => {
+ const res = one({ text: "«He said “stake it” twice.»", block: "said" });
+ expect(res.report!.verified_in_block).toBe(1);
+ });
+
+ test("NFD in the claim matches NFC in the source", () => {
+ const nfd = "Café prices rose";
+ const res = one({ text: `“${nfd}”` });
+ expect(res.report!.verified_in_source).toBe(1);
+ });
+
+ test("emphasis in the source and collapsed whitespace both verify", () => {
+ const res = one({ text: "“correlated slashing risk across every service”", block: "risk" });
+ expect(res.report!.verified_in_block).toBe(1);
+ });
+
+ test("a case difference does not verify", () => {
+ const res = one({ text: "“Reuses staked capital”", block: "abc" });
+ expect(res.report!.findings[0]!.outcome).toBe(QUOTE_CHECK_OUTCOME.notInBlock);
+ });
+
+ test("an ellipsis span verifies only when its fragments appear in order", () => {
+ expect(
+ one({ text: "“The first step … the final step.”", block: "steps" }).report!.verified_in_block,
+ ).toBe(1);
+ expect(
+ one({ text: "“the final step … The first step”", block: "steps" }).report!.findings[0]!
+ .outcome,
+ ).toBe(QUOTE_CHECK_OUTCOME.notInBlock);
+ });
+
+ test("a bracketed insertion is not interpreted and fails", () => {
+ const res = one({ text: "“reuses [sic] staked capital”", block: "abc" });
+ expect(res.report!.findings[0]!.outcome).toBe(QUOTE_CHECK_OUTCOME.notInBlock);
+ });
+});
+
+describe("checkClaimQuotes - the report", () => {
+ test("no spans anywhere: report is null and claims are identical", () => {
+ const claims = [{ text: "No quotes here.", block: "abc" }, { text: "Nor here." }];
+ const res = checkClaimQuotes({ claims, evidence: TEXT });
+ expect(res.report).toBeNull();
+ expect(res.claims).toEqual(claims);
+ });
+
+ test("30 failing spans are all counted but only the cap is returned", () => {
+ const claims = Array.from({ length: 30 }, (_, i) => ({ text: `“missing phrase ${i}”` }));
+ const res = checkClaimQuotes({ claims, evidence: TEXT });
+ expect(res.report!.unquoted).toBe(30);
+ expect(res.report!.total).toBe(30);
+ expect(res.report!.returned).toBe(QUOTE_FINDINGS_MAX);
+ expect(res.report!.findings).toHaveLength(QUOTE_FINDINGS_MAX);
+ expect(res.report!.truncated).toBe(true);
+ expect(res.report!.findings[24]!.claim).toBe(24);
+ });
+
+ test("a finding's span preview is capped", () => {
+ const long = "word ".repeat(60).trim();
+ const res = one({ text: `“${long}”` });
+ expect(res.report!.findings[0]!.span).toHaveLength(QUOTE_SPAN_PREVIEW_MAX_CHARS);
+ expect(long.startsWith(res.report!.findings[0]!.span)).toBe(true);
+ });
+
+ test("unpaired marks are counted and leave the claim untouched", () => {
+ const claim = { text: 'A 5" screen and “reuses staked capital”.', block: "abc" };
+ const res = one(claim);
+ expect(res.claims[0]).toEqual(claim);
+ expect(res.report!.unpaired).toBe(1);
+ expect(res.report!.verified_in_block).toBe(1);
+ });
+
+ test("a fabricated quote in CJK running text is checked and unquoted", () => {
+ const res = one({ text: "他说「完全捏造」。" });
+ expect(res.report!.findings).toEqual([
+ { claim: 0, outcome: QUOTE_CHECK_OUTCOME.notInSource, span: "完全捏造" },
+ ]);
+ expect(res.claims[0]!.text).toBe("他说完全捏造。");
+ });
+
+ test("a CRLF source verifies a span that crosses its line break", () => {
+ const crlf = { kind: "text", text: SOURCE.replaceAll("\n", "\r\n") } as const;
+ const res = one(
+ { text: "“correlated slashing risk across every service”", block: "risk" },
+ crlf,
+ );
+ expect(res.report!.verified_in_block).toBe(1);
+ const whole = one({ text: "“correlated slashing risk across every service”" }, crlf);
+ expect(whole.report!.verified_in_source).toBe(1);
+ });
+
+ test("the preview cap never splits an astral code point", () => {
+ // An astral LETTER (CJK Extension B): a span must hold letters or digits.
+ const res = one({ text: `“${"\u{20000}".repeat(200)}”` });
+ expect(res.report!.findings[0]!.span).toBe("\u{20000}".repeat(QUOTE_SPAN_PREVIEW_MAX_CHARS));
+ });
+
+ test("findings index the claim they came from", () => {
+ const res = checkClaimQuotes({
+ claims: [{ text: "“reuses staked capital”", block: "abc" }, { text: "“not there”" }],
+ evidence: TEXT,
+ });
+ expect(res.report!.checked).toBe(2);
+ expect(res.report!.findings).toEqual([
+ { claim: 1, outcome: QUOTE_CHECK_OUTCOME.notInSource, span: "not there" },
+ ]);
+ expect(res.report!.total).toBe(1);
+ expect(res.report!.returned).toBe(1);
+ expect(res.report!.truncated).toBe(false);
+ });
+});
+
+describe("checkClaimQuotes - one block index per check", () => {
+ test("claims citing several blocks segment the source once", () => {
+ const index = spyOn(blockResolve, "indexBlocks");
+ try {
+ const claims = [
+ { text: `"reuses staked capital"`, block: "abc" },
+ { text: `"stake it"`, block: "said" },
+ { text: `"the second step"`, block: "steps" },
+ ];
+ const res = checkClaimQuotes({ claims, evidence: TEXT });
+ expect(res.report!.unquoted).toBe(0);
+ expect(index).toHaveBeenCalledTimes(1);
+ } finally {
+ index.mockRestore();
+ }
+ });
+});
diff --git a/tests/core/brain/distill/quote-spans.test.ts b/tests/core/brain/distill/quote-spans.test.ts
new file mode 100644
index 00000000..e398eaf8
--- /dev/null
+++ b/tests/core/brain/distill/quote-spans.test.ts
@@ -0,0 +1,367 @@
+/**
+ * The quote-span finder and the comparison-only normaliser.
+ *
+ * Every case is built from code points, never from a list of words or
+ * languages: a delimiter is any `\p{Quotation_Mark}` code point, pairing is
+ * decided by position (what stands on either side of the mark), and a mark
+ * with a letter on both sides is an apostrophe whatever script the letters
+ * come from.
+ */
+
+import { describe, expect, test } from "bun:test";
+
+import {
+ findQuoteSpans,
+ normalizeForQuoteComparison,
+ QUOTATION_MARK_RE,
+ spanOccursIn,
+ splitEllipsisFragments,
+ unquoteSpans,
+} from "../../../../src/core/brain/distill/quote-spans.ts";
+
+import { LINEAR_CEILING_MS } from "../../../helpers/linear-time.ts";
+
+const cp = (code: number): string => String.fromCodePoint(code);
+
+/** Opener and closer pairs, every one a `Quotation_Mark` code point. */
+const PAIRS: ReadonlyArray<readonly [number, number]> = [
+ [0x22, 0x22],
+ [0x27, 0x27],
+ [0xab, 0xbb],
+ [0xbb, 0xab],
+ [0x201c, 0x201d],
+ [0x201e, 0x201c],
+ [0x2018, 0x2019],
+ [0x201a, 0x2018],
+ [0x300c, 0x300d],
+ [0x300e, 0x300f],
+ [0x301d, 0x301e],
+];
+
+/** One letter from each of three scripts, generated by code point. */
+const LETTERS: ReadonlyArray<string> = [0x61, 0x430, 0x3b1].map(cp);
+
+describe("QUOTATION_MARK_RE", () => {
+ test("every pair member is a quotation mark", () => {
+ for (const [open, close] of PAIRS) {
+ expect(QUOTATION_MARK_RE.test(cp(open))).toBe(true);
+ expect(QUOTATION_MARK_RE.test(cp(close))).toBe(true);
+ }
+ });
+});
+
+describe("findQuoteSpans", () => {
+ test("every pair forms exactly one span with the right inner text", () => {
+ for (const [open, close] of PAIRS) {
+ const inner = `x${LETTERS[1]} y`;
+ const text = `said ${cp(open)}${inner}${cp(close)} today`;
+ const scan = findQuoteSpans(text);
+ expect(scan.unpaired).toBe(0);
+ expect(scan.spans).toHaveLength(1);
+ const span = scan.spans[0];
+ expect(span?.inner).toBe(inner);
+ expect(span?.open).toBe(5);
+ expect(span?.close).toBe(5 + 1 + inner.length);
+ }
+ });
+
+ test("a non-directional mark between two letters of any script is never a delimiter", () => {
+ // Corner brackets and low-9 marks carry their direction in their own
+ // category (Ps/Pe) and are never apostrophes; they are covered below.
+ const ambiguous = PAIRS.filter(([, close]) => !/[\p{Ps}\p{Pe}]/u.test(cp(close)));
+ expect(ambiguous.length).toBeGreaterThan(0);
+ for (const [, close] of ambiguous) {
+ for (const left of LETTERS) {
+ for (const right of LETTERS) {
+ const scan = findQuoteSpans(`${left}${cp(close)}${right}`);
+ expect(scan.spans).toHaveLength(0);
+ expect(scan.unpaired).toBe(0);
+ }
+ }
+ }
+ });
+
+ test("an apostrophe inside a quote does not end it", () => {
+ const scan = findQuoteSpans(`${cp(0x201c)}it${cp(0x2019)}s fine${cp(0x201d)}`);
+ expect(scan.spans.map((s) => s.inner)).toEqual([`it${cp(0x2019)}s fine`]);
+ });
+
+ test("a nested quote yields only the outer span", () => {
+ const text = `${cp(0x201c)}a ${cp(0x2018)}b${cp(0x2019)} c${cp(0x201d)}`;
+ const scan = findQuoteSpans(text);
+ expect(scan.spans).toHaveLength(1);
+ expect(scan.spans[0]?.inner).toBe(`a ${cp(0x2018)}b${cp(0x2019)} c`);
+ expect(scan.unpaired).toBe(0);
+ });
+
+ test("a lone opener is unpaired and not a span", () => {
+ const scan = findQuoteSpans(`he said ${cp(0x201c)}never closed`);
+ expect(scan.spans).toHaveLength(0);
+ expect(scan.unpaired).toBe(1);
+ });
+
+ test("an empty or whitespace-only pair is not a span", () => {
+ expect(findQuoteSpans(`x "" y`).spans).toHaveLength(0);
+ expect(findQuoteSpans(`x ${cp(0x201c)} ${cp(0x201d)} y`).spans).toHaveLength(0);
+ });
+
+ test("corner brackets glued to letters on both sides still pair (CJK running text)", () => {
+ const han = [0x4ed6, 0x8bf4].map(cp).join("");
+ const inner = [0x4f60, 0x597d].map(cp).join("");
+ for (const [open, close] of [
+ [0x300c, 0x300d],
+ [0x300e, 0x300f],
+ ] as const) {
+ const scan = findQuoteSpans(`${han}${cp(open)}${inner}${cp(close)}${cp(0x3002)}`);
+ expect(scan.spans.map((s) => s.inner)).toEqual([inner]);
+ expect(scan.unpaired).toBe(0);
+ }
+ });
+
+ test("an opener that never closes does not hide a later quote", () => {
+ const scan = findQuoteSpans(`the '90s were "great"`);
+ expect(scan.spans.map((s) => s.inner)).toEqual(["great"]);
+ expect(scan.unpaired).toBe(1);
+ });
+
+ test("a lone closer is unpaired and not a span", () => {
+ const scan = findQuoteSpans(`said today${cp(0x201d)}`);
+ expect(scan.spans).toHaveLength(0);
+ expect(scan.unpaired).toBe(1);
+ });
+
+ test("guillemets with inner spaces are counted as unpaired, never as a span", () => {
+ const scan = findQuoteSpans(`Il a dit ${cp(0xab)} bonjour ${cp(0xbb)} hier.`);
+ expect(scan.spans).toHaveLength(0);
+ expect(scan.unpaired).toBe(2);
+ });
+
+ test("corner brackets pair in CJK running text after letters and full-width punctuation", () => {
+ for (const [text, inner] of [
+ ["他说「你好」。", "你好"],
+ ["他说:「你好」。", "你好"],
+ ["「こんにちは」と言った", "こんにちは"],
+ ] as const) {
+ const scan = findQuoteSpans(text);
+ expect(scan.spans.map((s) => s.inner)).toEqual([inner]);
+ expect(scan.unpaired).toBe(0);
+ }
+ });
+
+ test("a possessive apostrophe never closes a double quote, and unquoting keeps it", () => {
+ const text = `"the students' work" is fine`;
+ const scan = findQuoteSpans(text);
+ expect(scan.spans.map((s) => s.inner)).toEqual(["the students' work"]);
+ expect(unquoteSpans(text, scan.spans)).toBe("the students' work is fine");
+ });
+
+ test("a single-width opener left open inside a double quote does not hide it", () => {
+ const scan = findQuoteSpans(`"back in the '90s it was" ok`);
+ expect(scan.spans.map((s) => s.inner)).toEqual(["back in the '90s it was"]);
+ expect(scan.unpaired).toBe(1);
+ });
+
+ test("an opener directly after a dash or a colon opens", () => {
+ expect(findQuoteSpans(`${cp(0x2014)}"quote" here`).spans.map((s) => s.inner)).toEqual([
+ "quote",
+ ]);
+ expect(findQuoteSpans(`He said:"xy"`).spans.map((s) => s.inner)).toEqual(["xy"]);
+ const scan = findQuoteSpans(
+ `${cp(0x201c)}Aa${cp(0x201d)}${cp(0x2014)}${cp(0x201c)}Bb${cp(0x201d)}`,
+ );
+ expect(scan.spans.map((s) => s.inner)).toEqual(["Aa", "Bb"]);
+ expect(scan.unpaired).toBe(0);
+ });
+
+ test("one no-break, narrow no-break or thin space inside guillemets is transparent", () => {
+ for (const space of [0xa0, 0x202f, 0x2009].map(cp)) {
+ const scan = findQuoteSpans(`Il a dit ${cp(0xab)}${space}bonjour${space}${cp(0xbb)} hier.`);
+ expect(scan.spans.map((s) => s.inner.trim())).toEqual(["bonjour"]);
+ expect(scan.unpaired).toBe(0);
+ }
+ });
+
+ test("a pair holding fewer than two letters or digits is not a span", () => {
+ const scan = findQuoteSpans(`rock 'n' roll`);
+ expect(scan.spans).toHaveLength(0);
+ expect(scan.unpaired).toBe(2);
+ expect(findQuoteSpans(`said "ok" twice`).spans.map((s) => s.inner)).toEqual(["ok"]);
+ });
+
+ test("stays linear on long runs of marks", () => {
+ const size = 256 * 1024;
+ for (const text of [
+ `${" 'a".repeat(size / 3)} "x"`,
+ `"ab" `.repeat(size / 5),
+ cp(0x201c).repeat(size),
+ `${" 'a".repeat(size / 6)}${'b" '.repeat(size / 6)}`,
+ ]) {
+ const started = performance.now();
+ findQuoteSpans(text);
+ expect(performance.now() - started).toBeLessThan(LINEAR_CEILING_MS);
+ }
+ });
+
+ test("two quotes in one text are two spans in order", () => {
+ const scan = findQuoteSpans(`"one" and "two".`);
+ expect(scan.spans.map((s) => s.inner)).toEqual(["one", "two"]);
+ });
+});
+
+describe("unquoteSpans", () => {
+ test("removes exactly the two marks of each span and keeps every other byte", () => {
+ const text = `A ${cp(0x201e)}b c${cp(0x201c)}, then "dd" and it${cp(0x2019)}s "e`;
+ const scan = findQuoteSpans(text);
+ expect(scan.spans).toHaveLength(2);
+ expect(scan.unpaired).toBe(1);
+ expect(unquoteSpans(text, scan.spans)).toBe(`A b c, then dd and it${cp(0x2019)}s "e`);
+ });
+
+ test("removing a subset leaves the other spans quoted", () => {
+ const text = `"one" and "two"`;
+ const scan = findQuoteSpans(text);
+ expect(unquoteSpans(text, scan.spans.slice(1))).toBe(`"one" and two`);
+ });
+});
+
+describe("normalizeForQuoteComparison", () => {
+ test("folds NFD to NFC", () => {
+ expect(normalizeForQuoteComparison("été")).toBe("été");
+ });
+
+ test("reduces inline Markdown to display text", () => {
+ expect(normalizeForQuoteComparison("a **b** _i_ `c` ~~s~~ [t](u) [[p|a2]] [[p]]")).toBe(
+ "a b i c s t a2 p",
+ );
+ });
+
+ test("removes line-leading block markers and the trailing block id", () => {
+ expect(normalizeForQuoteComparison("> quoted line\n- item one\n1. item two ^blk-1")).toBe(
+ "quoted line item one item two",
+ );
+ });
+
+ test("removes only paired emphasis delimiters, never an underscore or star inside a word", () => {
+ expect(normalizeForQuoteComparison("snake_case 2*3")).toBe("snake_case 2*3");
+ expect(normalizeForQuoteComparison("a *b* __c__ d_e_f")).toBe("a b c d_e_f");
+ });
+
+ test("a numbered line inside a paragraph is text; a list after a blank line is a list", () => {
+ expect(normalizeForQuoteComparison("Intro line\n1999) was a year")).toBe(
+ "Intro line 1999) was a year",
+ );
+ expect(normalizeForQuoteComparison("Intro line\n\n1999) was a year")).toBe(
+ "Intro line was a year",
+ );
+ expect(normalizeForQuoteComparison("Key points:\n- one\n2. two")).toBe("Key points: one two");
+ });
+
+ test("removes a trailing block id before a CR", () => {
+ expect(normalizeForQuoteComparison("first line ^p1\r\nsecond")).toBe("first line second");
+ expect(normalizeForQuoteComparison("glued^p1")).toBe("glued^p1");
+ });
+
+ test("bracket-type quotation marks fold to the double mark for comparison", () => {
+ const low9 = `Er sagte ${cp(0x201e)}ja${cp(0x201c)} dann`;
+ const curly = `Er sagte ${cp(0x201c)}ja${cp(0x201d)} dann`;
+ expect(normalizeForQuoteComparison(low9)).toBe(normalizeForQuoteComparison(curly));
+ expect(normalizeForQuoteComparison(`${cp(0x300c)}ja${cp(0x300d)} (x)`)).toBe(`"ja" (x)`);
+ });
+
+ test("stays linear on long runs of spaces and brackets", () => {
+ const size = 256 * 1024;
+ for (const text of [
+ `a${" ".repeat(size)}b`,
+ "[[".repeat(size / 2),
+ "[".repeat(size),
+ "*a ".repeat(size / 3),
+ `${"^".repeat(size)} x`,
+ ]) {
+ const started = performance.now();
+ normalizeForQuoteComparison(text);
+ expect(performance.now() - started).toBeLessThan(LINEAR_CEILING_MS);
+ }
+ });
+
+ test("collapses whitespace and trims", () => {
+ expect(normalizeForQuoteComparison(" a \n\n\t b ")).toBe("a b");
+ });
+
+ test("folds quote-mark width but keeps case and other punctuation", () => {
+ expect(normalizeForQuoteComparison(`It${cp(0x2019)}s ${cp(0x201c)}A${cp(0x201d)}, B!`)).toBe(
+ `It's "A", B!`,
+ );
+ expect(normalizeForQuoteComparison("Case Stays.")).toBe("Case Stays.");
+ });
+});
+
+describe("ellipsis fragments", () => {
+ test("splits on U+2026 and on three or more dots, dropping empty fragments", () => {
+ expect(splitEllipsisFragments(`a${cp(0x2026)} b .... c...`)).toEqual(["a", "b", "c"]);
+ expect(splitEllipsisFragments("no ellipsis. here..")).toEqual(["no ellipsis. here.."]);
+ expect(splitEllipsisFragments(`${cp(0x2026)}`)).toEqual([]);
+ });
+
+ test("fragments must occur in order and without overlap", () => {
+ const haystack = normalizeForQuoteComparison("the quick brown fox jumps over the dog");
+ expect(spanOccursIn("quick brown fox", haystack)).toBe(true);
+ expect(spanOccursIn(`quick${cp(0x2026)}jumps over`, haystack)).toBe(true);
+ expect(spanOccursIn(`jumps${cp(0x2026)}quick`, haystack)).toBe(false);
+ expect(spanOccursIn("brown fox...fox jumps", haystack)).toBe(false);
+ expect(spanOccursIn("quick [sic] brown", haystack)).toBe(false);
+ expect(spanOccursIn("Quick brown", haystack)).toBe(false);
+ expect(spanOccursIn(`${cp(0x2026)}`, haystack)).toBe(false);
+ });
+
+ test("the span side is normalised the same way as the haystack", () => {
+ const haystack = normalizeForQuoteComparison("he said **very** clearly");
+ expect(spanOccursIn("said very clearly", haystack)).toBe(true);
+ });
+});
+
+describe("word boundaries at the outer ends of a span", () => {
+ test("a span inside a longer word does not verify", () => {
+ const haystack = normalizeForQuoteComparison("The method is unsafe.");
+ expect(spanOccursIn("safe", haystack)).toBe(false);
+ expect(spanOccursIn("method i", haystack)).toBe(false);
+ });
+
+ test("a span that starts and ends on word boundaries verifies", () => {
+ const haystack = normalizeForQuoteComparison("The method is unsafe.");
+ expect(spanOccursIn("unsafe", haystack)).toBe(true);
+ expect(spanOccursIn("is unsafe.", haystack)).toBe(true);
+ expect(spanOccursIn("The method", haystack)).toBe(true);
+ });
+
+ test("a span of word-segmented text without spaces verifies inside running text", () => {
+ const sentence = [0x4ed6, 0x8bf4, 0x4f60, 0x597d, 0x5417, 0x3002].map(cp).join("");
+ const span = [0x4f60, 0x597d].map(cp).join("");
+ expect(spanOccursIn(span, normalizeForQuoteComparison(sentence))).toBe(true);
+ });
+
+ test("an ellipsis edge may cut a word, an outer edge may not", () => {
+ const haystack = normalizeForQuoteComparison("the reconfiguration was unsafe today");
+ expect(spanOccursIn(`the reconfig${cp(0x2026)}unsafe today`, haystack)).toBe(true);
+ expect(spanOccursIn(`${cp(0x2026)}configuration was`, haystack)).toBe(true);
+ expect(spanOccursIn(`was uns${cp(0x2026)}`, haystack)).toBe(true);
+ expect(spanOccursIn(`he reconfig${cp(0x2026)}today`, haystack)).toBe(false);
+ expect(spanOccursIn(`the${cp(0x2026)}safe today`, haystack)).toBe(true);
+ expect(spanOccursIn(`the${cp(0x2026)}today unsafe`, haystack)).toBe(false);
+ });
+});
+
+describe("line markers on the span side", () => {
+ test("a span's own leading list number is wording, not a marker", () => {
+ expect(spanOccursIn("3. the cat sat", normalizeForQuoteComparison("the cat sat"))).toBe(false);
+ expect(spanOccursIn("1) the cat sat", normalizeForQuoteComparison("the cat sat"))).toBe(false);
+ });
+
+ test("a list marker on the evidence side is still removed", () => {
+ expect(spanOccursIn("the cat sat", normalizeForQuoteComparison("3. the cat sat"))).toBe(true);
+ });
+
+ test("a span's own leading quote or bullet marker is wording, not a marker", () => {
+ expect(spanOccursIn("> the cat sat", normalizeForQuoteComparison("the cat sat"))).toBe(false);
+ expect(spanOccursIn("- the cat sat", normalizeForQuoteComparison("the cat sat"))).toBe(false);
+ });
+});
diff --git a/tests/core/brain/distill/quote-verdict.test.ts b/tests/core/brain/distill/quote-verdict.test.ts
new file mode 100644
index 00000000..09a2ba60
--- /dev/null
+++ b/tests/core/brain/distill/quote-verdict.test.ts
@@ -0,0 +1,76 @@
+/**
+ * The quote-check outcome vocabulary, the report shape constants and the
+ * named refusal a strict distillation throws.
+ */
+
+import { describe, expect, test } from "bun:test";
+
+import { PAGE_LINT_MAX_FINDINGS } from "../../../../src/core/brain/page-lint.ts";
+import { CAPTURE_SCOPE } from "../../../../src/core/brain/provenance/capture-scope.ts";
+import {
+ QUOTE_CHECK_OUTCOME,
+ QUOTE_FINDINGS_MAX,
+ QUOTE_SPAN_PREVIEW_MAX_CHARS,
+ QUOTE_UNVERIFIED_CODE,
+ QUOTES_UNQUOTED_KEY,
+ QUOTES_VERIFIED_KEY,
+ QuoteCheckError,
+ type QuoteFinding,
+ VERIFIED_QUOTE_OUTCOMES,
+} from "../../../../src/core/brain/distill/quote-verdict.ts";
+
+describe("QUOTE_CHECK_OUTCOME vocabulary", () => {
+ // Frozenness, membership and guard agreement are owned by
+ // tests/core/architecture/verdict-vocabulary-census.test.ts; this pins the
+ // wire spellings only.
+ test("the outcome tokens are the pinned wire spellings", () => {
+ expect(QUOTE_CHECK_OUTCOME).toEqual({
+ verifiedInBlock: "verified-in-block",
+ verifiedInSource: "verified-in-source",
+ notInBlock: "not-in-block",
+ notInSource: "not-in-source",
+ blockNotFound: "block-not-found",
+ blockAmbiguous: "block-ambiguous",
+ sourceNotText: "source-not-text",
+ urlOnly: "url-only",
+ });
+ });
+
+ test("a span with no local bytes is named by the one capture-scope token", () => {
+ expect(QUOTE_CHECK_OUTCOME.urlOnly).toBe(CAPTURE_SCOPE.urlOnly);
+ });
+
+ test("only the two verified outcomes count as verified", () => {
+ expect([...VERIFIED_QUOTE_OUTCOMES].toSorted()).toEqual([
+ "verified-in-block",
+ "verified-in-source",
+ ]);
+ });
+
+ test("the caps and keys are the pinned values", () => {
+ expect(QUOTE_FINDINGS_MAX).toBe(PAGE_LINT_MAX_FINDINGS);
+ expect(QUOTE_SPAN_PREVIEW_MAX_CHARS).toBe(120);
+ expect(QUOTES_VERIFIED_KEY).toBe("quotes_verified");
+ expect(QUOTES_UNQUOTED_KEY).toBe("quotes_unquoted");
+ });
+});
+
+describe("QuoteCheckError", () => {
+ const findings: ReadonlyArray<QuoteFinding> = [
+ { claim: 0, outcome: QUOTE_CHECK_OUTCOME.notInBlock, span: "secret paraphrase" },
+ { claim: 3, outcome: QUOTE_CHECK_OUTCOME.urlOnly, span: "another span" },
+ ];
+
+ test("carries the registered code and names claims and outcomes only", () => {
+ const error = new QuoteCheckError(findings);
+ expect(error).toBeInstanceOf(Error);
+ expect(error.name).toBe("QuoteCheckError");
+ expect(error.code).toBe(QUOTE_UNVERIFIED_CODE);
+ expect(QUOTE_UNVERIFIED_CODE).toBe("quote_unverified");
+ expect(error.message).toBe(
+ "quoted spans failed verification: claim 0: not-in-block, claim 3: url-only",
+ );
+ expect(error.message).not.toContain("secret");
+ expect(error.findings).toEqual(findings);
+ });
+});
diff --git a/tests/core/brain/format-wikilink.test.ts b/tests/core/brain/format-wikilink.test.ts
index a6054d23..9c6545c6 100644
--- a/tests/core/brain/format-wikilink.test.ts
+++ b/tests/core/brain/format-wikilink.test.ts
@@ -141,3 +141,18 @@ test("an already-canonical link is not counted as a change", () => {
expect(result.content).toBe(content);
expect(result.changed).toBe(0);
});
+
+test("a backtick or tilde fence closed by a longer run ends at the end of that run", () => {
+ for (const [open, close, tail] of [
+ ["```", "````", "`x`"],
+ ["~~~", "~~~~~~", "~~~"],
+ ]) {
+ const content = [open, "[[alpha]] in a fence stays", close, "After [[alpha]].", tail].join(
+ "\n",
+ );
+ const result = normalizeWikilinks(content, "full", PAGES);
+ expect(result.content).toContain("[[alpha]] in a fence stays");
+ expect(result.content).toContain("After [[Brain/notes/alpha]].");
+ expect(result.changed).toBe(1);
+ }
+});
diff --git a/tests/core/brain/hygiene-capture-scope.test.ts b/tests/core/brain/hygiene-capture-scope.test.ts
new file mode 100644
index 00000000..57f046c0
--- /dev/null
+++ b/tests/core/brain/hygiene-capture-scope.test.ts
@@ -0,0 +1,287 @@
+/**
+ * The `capture-scope` hygiene detector. A retrievable page of active
+ * knowledge whose every cited source is CURRENTLY `url-only` rests on a
+ * locator that may rot; the detector names it (`warning`, `review`). The
+ * scope is re-derived from each cited identity on every sweep, so a
+ * full-local page whose file vanished reports, and a stamped
+ * `bounded-local` counts as backing only while its excerpt block is present
+ * and matches its digest. Quarantined pages are skipped: their untrusted
+ * marker already names the condition.
+ */
+
+import { afterEach, beforeEach, describe, expect, test } from "bun:test";
+import { chmodSync, mkdirSync, mkdtempSync, rmSync, unlinkSync, writeFileSync } from "node:fs";
+import { dirname, join } from "node:path";
+import { tmpdir } from "node:os";
+
+import { bootstrapBrain } from "../../../src/core/brain/init.ts";
+import { atomicWriteFileSync } from "../../../src/core/fs-atomic.ts";
+import { formatFrontmatter, parseFrontmatter } from "../../../src/core/vault.ts";
+import type { FrontmatterMap } from "../../../src/core/types.ts";
+import { ingestSource } from "../../../src/core/brain/ingest/ingest.ts";
+import {
+ BRAIN_REPORT_KIND,
+ writeResearchReport,
+} from "../../../src/core/brain/research/research.ts";
+import { distillSource } from "../../../src/core/brain/distill/distill-source.ts";
+import {
+ CAPTURE_SCOPE,
+ excerptDigest,
+ renderExcerptSection,
+} from "../../../src/core/brain/provenance/capture-scope.ts";
+import {
+ CAPTURE_SCOPE_DETECTOR_ID,
+ detectCaptureScope,
+} from "../../../src/core/brain/hygiene/detectors/capture-scope.ts";
+import { runHygieneScan } from "../../../src/core/brain/hygiene/scan.ts";
+import {
+ UNTRUSTED_SOURCE_FRONTMATTER_KEY,
+ hasUntrustedSourceMarker,
+} from "../../../src/core/brain/trust/untrusted-provenance.ts";
+import { CHMOD_CANNOT_DENY } from "../../helpers/platform.ts";
+import { DEFAULT_SCAN_IDS, HYGIENE_DETECTOR_IDS } from "../../../src/core/brain/hygiene/types.ts";
+
+let vault: string;
+let configHome: string;
+
+const NOW = new Date("2026-06-13T12:00:00Z");
+const LOCAL_SOURCE = "Articles/a.md";
+const URL_A = "https://example.test/a";
+const URL_B = "https://example.test/b";
+const ENTITY_PAGE = "Brain/entities/concept/ent-concept-alpha.md";
+const BOUNDED_PAGE = "Brain/distillations/dist-bounded.md";
+const EXCERPT = "The protocol settles in two rounds.\n";
+
+function seed(rel: string): void {
+ mkdirSync(join(vault, dirname(rel)), { recursive: true });
+ writeFileSync(join(vault, rel), "bytes\n", "utf8");
+}
+
+function writePage(rel: string, meta: FrontmatterMap, body: string): void {
+ mkdirSync(join(vault, dirname(rel)), { recursive: true });
+ writeFileSync(join(vault, rel), formatFrontmatter(meta, body), "utf8");
+}
+
+function writeEntity(status: string, source: string): void {
+ writePage(
+ ENTITY_PAGE,
+ {
+ kind: "brain-entity",
+ entity_id: "ent-concept-alpha",
+ category: "concept",
+ name: "Alpha",
+ status,
+ tags: ["brain", "brain/entity"],
+ },
+ `# Alpha\n\n## Sources\n\n- [[${source}]]`,
+ );
+}
+
+function writeBounded(excerpt: string, stampedHashOf: string): void {
+ writePage(
+ BOUNDED_PAGE,
+ {
+ kind: "brain-distillation",
+ source_path: URL_A,
+ capture_scope: CAPTURE_SCOPE.boundedLocal,
+ excerpt_hash: excerptDigest(stampedHashOf),
+ provenance: "stated",
+ },
+ [
+ "## Claims",
+ "",
+ "- A claim.",
+ "",
+ "## Sources",
+ "",
+ `- [[${URL_A}]]`,
+ "",
+ renderExcerptSection(excerpt),
+ ].join("\n"),
+ );
+}
+
+function findingsFor(page: string) {
+ return detectCaptureScope(vault).filter((f) => f.targets.includes(page));
+}
+
+beforeEach(() => {
+ vault = mkdtempSync(join(tmpdir(), "o2b-hygiene-scope-vault-"));
+ configHome = mkdtempSync(join(tmpdir(), "o2b-hygiene-scope-cfg-"));
+ const configPath = join(configHome, "config.yaml");
+ atomicWriteFileSync(configPath, `vault: ${vault}\nagent_name: claude\n`);
+ bootstrapBrain(vault, { configPath });
+});
+
+afterEach(() => {
+ rmSync(vault, { recursive: true, force: true });
+ rmSync(configHome, { recursive: true, force: true });
+});
+
+describe("detectCaptureScope", () => {
+ test("a finding that opens a fence and never closes it does not hide the report", () => {
+ const report = writeResearchReport(
+ vault,
+ { title: "T", sources: [URL_B], findings: [{ statement: "x\n```", sources: [URL_B] }] },
+ { agent: "claude", now: NOW },
+ );
+
+ expect(findingsFor(report.reportPath)).toHaveLength(1);
+ });
+
+ test("a research report citing only URLs is one review warning naming the URLs", () => {
+ const report = writeResearchReport(
+ vault,
+ { title: "T", sources: [URL_A, URL_B], findings: [{ statement: "S", sources: [URL_A] }] },
+ { agent: "claude", now: NOW },
+ );
+
+ const findings = findingsFor(report.reportPath);
+ expect(findings).toHaveLength(1);
+ const finding = findings[0]!;
+ expect(finding.detector).toBe(CAPTURE_SCOPE_DETECTOR_ID);
+ expect(finding.severity).toBe("warning");
+ expect(finding.proposed_action).toBe("review");
+ expect(finding.title).toBe("Active knowledge rests on url-only sources");
+ expect(finding.targets).toEqual([report.reportPath]);
+ expect(finding.evidence).toEqual({ sources: [URL_A, URL_B], stamped: null });
+ });
+
+ test("a report citing one URL and one vault file is not reported", () => {
+ seed(LOCAL_SOURCE);
+ const report = writeResearchReport(
+ vault,
+ {
+ title: "T",
+ sources: [URL_A, LOCAL_SOURCE],
+ findings: [{ statement: "S", sources: [URL_A] }],
+ },
+ { agent: "claude", now: NOW },
+ );
+
+ expect(findingsFor(report.reportPath)).toEqual([]);
+ });
+
+ test("an active entity citing a URL is reported; the same entity in quarantine is not", () => {
+ writeEntity("active", URL_A);
+ expect(findingsFor(ENTITY_PAGE)).toHaveLength(1);
+
+ writeEntity("quarantine", URL_A);
+ expect(findingsFor(ENTITY_PAGE)).toEqual([]);
+ });
+
+ test("an archived entity is outside the canonical scope and is not reported", () => {
+ writeEntity("archived", URL_A);
+ expect(findingsFor(ENTITY_PAGE)).toEqual([]);
+ });
+
+ test("an untrusted ingest page is already gated out and is not reported", () => {
+ const ingest = ingestSource(
+ vault,
+ { sourcePath: URL_A, summary: "S.", extraction: { entities: [], relations: [] } },
+ { agent: "claude", now: NOW },
+ );
+
+ expect(findingsFor(ingest.summaryPath)).toEqual([]);
+
+ // The cause is the marker, not a page the detector cannot read: the
+ // same page without it is reported.
+ const [meta, body] = parseFrontmatter(join(vault, ingest.summaryPath));
+ expect(hasUntrustedSourceMarker(meta)).toBe(true);
+ const { [UNTRUSTED_SOURCE_FRONTMATTER_KEY]: _marker, ...released } = meta;
+ writePage(ingest.summaryPath, released, body);
+ expect(findingsFor(ingest.summaryPath)).toHaveLength(1);
+ });
+
+ test("a distillation whose vault source was deleted is re-derived as url-only", () => {
+ seed(LOCAL_SOURCE);
+ const distilled = distillSource(
+ vault,
+ { sourcePath: LOCAL_SOURCE, claims: [{ text: "A claim." }] },
+ { agent: "claude", now: NOW },
+ );
+ expect(findingsFor(distilled.distillationPath)).toEqual([]);
+
+ unlinkSync(join(vault, LOCAL_SOURCE));
+
+ const findings = findingsFor(distilled.distillationPath);
+ expect(findings).toHaveLength(1);
+ expect(findings[0]!.evidence).toEqual({ sources: [LOCAL_SOURCE], stamped: null });
+ });
+
+ test("a bounded-local page backs itself only while its excerpt matches the digest", () => {
+ writeBounded(EXCERPT, EXCERPT);
+ expect(findingsFor(BOUNDED_PAGE)).toEqual([]);
+
+ writeBounded("The protocol settles in three rounds.\n", EXCERPT);
+ const edited = findingsFor(BOUNDED_PAGE);
+ expect(edited).toHaveLength(1);
+ expect(edited[0]!.evidence).toEqual({ sources: [URL_A], stamped: CAPTURE_SCOPE.boundedLocal });
+
+ writePage(
+ BOUNDED_PAGE,
+ {
+ kind: "brain-distillation",
+ source_path: URL_A,
+ capture_scope: CAPTURE_SCOPE.boundedLocal,
+ excerpt_hash: excerptDigest(EXCERPT),
+ },
+ `## Claims\n\n- A claim.\n\n## Sources\n\n- [[${URL_A}]]`,
+ );
+ expect(findingsFor(BOUNDED_PAGE)).toHaveLength(1);
+ });
+
+ test("a page of another kind is never inspected", () => {
+ // Inside an inspected directory, so only the kind filter can exclude it.
+ writePage("Brain/distillations/n.md", { kind: "note" }, `## Sources\n\n- [[${URL_A}]]`);
+ expect(detectCaptureScope(vault)).toEqual([]);
+ });
+});
+
+describe("capture-scope when the filesystem refuses a source", () => {
+ test.skipIf(CHMOD_CANNOT_DENY)(
+ "an unreadable cited source is treated as backing and does not abort the sweep",
+ () => {
+ const locked = join(vault, "locked");
+ seed("locked/inner/a.md");
+ writePage("Brain/reports/a.md", { kind: BRAIN_REPORT_KIND }, `## Sources\n\n- [[${URL_A}]]`);
+ writePage(
+ "Brain/reports/b.md",
+ { kind: BRAIN_REPORT_KIND },
+ "## Sources\n\n- [[locked/inner/a.md]]",
+ );
+ chmodSync(locked, 0o000);
+ try {
+ const report = runHygieneScan(vault, { now: NOW, detectors: [CAPTURE_SCOPE_DETECTOR_ID] });
+ expect(report.errors).toEqual([]);
+ expect(report.counts[CAPTURE_SCOPE_DETECTOR_ID]).toBe(1);
+ expect(findingsFor("Brain/reports/a.md")).toHaveLength(1);
+ expect(findingsFor("Brain/reports/b.md")).toEqual([]);
+ } finally {
+ chmodSync(locked, 0o755);
+ }
+ },
+ );
+});
+
+describe("capture-scope in the hygiene sweep", () => {
+ test("the detector is registered and runs in the default sweep", () => {
+ expect(HYGIENE_DETECTOR_IDS).toContain(CAPTURE_SCOPE_DETECTOR_ID);
+ expect(DEFAULT_SCAN_IDS).toContain(CAPTURE_SCOPE_DETECTOR_ID);
+ writeEntity("active", URL_A);
+
+ const report = runHygieneScan(vault, { now: NOW });
+
+ expect(report.detectors_run).toContain(CAPTURE_SCOPE_DETECTOR_ID);
+ expect(report.counts[CAPTURE_SCOPE_DETECTOR_ID]).toBe(1);
+ });
+
+ test("finding ids are deterministic across sweeps", () => {
+ writeEntity("active", URL_A);
+ const first = detectCaptureScope(vault).map((f) => f.id);
+ const second = detectCaptureScope(vault).map((f) => f.id);
+ expect(first).toHaveLength(1);
+ expect(second).toEqual(first);
+ expect(first[0]!.startsWith(`${CAPTURE_SCOPE_DETECTOR_ID}:`)).toBe(true);
+ });
+});
diff --git a/tests/core/brain/hygiene-scan.test.ts b/tests/core/brain/hygiene-scan.test.ts
index 3ce2855a..7dac1556 100644
--- a/tests/core/brain/hygiene-scan.test.ts
+++ b/tests/core/brain/hygiene-scan.test.ts
@@ -148,6 +148,7 @@ describe("scan composition", () => {
"freshness",
"usefulness",
"slug-collisions",
+ "capture-scope",
]);
expect(Object.isFrozen(report)).toBe(true);
expect(Object.isFrozen(report.findings)).toBe(true);
diff --git a/tests/core/brain/ingest/ingest-capture-scope.test.ts b/tests/core/brain/ingest/ingest-capture-scope.test.ts
new file mode 100644
index 00000000..376a092e
--- /dev/null
+++ b/tests/core/brain/ingest/ingest-capture-scope.test.ts
@@ -0,0 +1,104 @@
+/**
+ * Capture scope on ingest summary pages. The scope follows the lane the
+ * intake already decided for the cited source: a vault file the summary was
+ * built from is `full-local` and writes nothing new (the page stays
+ * byte-identical to the pre-scope output), a source with no local bytes is
+ * `url-only` and the page says so in `capture_scope`.
+ */
+
+import { afterEach, beforeEach, describe, expect, test } from "bun:test";
+import { mkdirSync, mkdtempSync, readFileSync, rmSync, statSync, writeFileSync } from "node:fs";
+import { join } from "node:path";
+import { tmpdir } from "node:os";
+
+import { bootstrapBrain } from "../../../../src/core/brain/init.ts";
+import { atomicWriteFileSync } from "../../../../src/core/fs-atomic.ts";
+import { ingestSource } from "../../../../src/core/brain/ingest/ingest.ts";
+import {
+ CAPTURE_SCOPE,
+ CAPTURE_SCOPE_KEY,
+} from "../../../../src/core/brain/provenance/capture-scope.ts";
+import { parseFrontmatter } from "../../../../src/core/vault.ts";
+
+let vault: string;
+let configHome: string;
+
+const NOW = new Date("2026-06-13T12:00:00Z");
+const LATER = new Date("2026-06-14T09:00:00Z");
+const LOCAL_SOURCE = "Articles/a.md";
+const URL_SOURCE = "https://example.test/articles/a";
+
+/** The summary page the base release (v1.66.0) wrote for {@link LOCAL_SOURCE}. */
+const BASE_LOCAL_PAGE =
+ '---\nkind: brain-source\nsource_path: Articles/a.md\nsource_hash: f84f9777c1d1c4b660d50d094897c7b95dab0108683ad9343c481a99d277c20e\nprovenance: stated\ncreated_at: "2026-06-13T12:00:00Z"\nupdated_at: "2026-06-13T12:00:00Z"\ntags: [brain, brain/source]\n---\n\nA summary.\n\n## Sources\n\n- [[Articles/a.md]]\n\n## Entities\n\n- [[ent-concept-alpha]]\n';
+
+function input(sourcePath: string) {
+ return {
+ sourcePath,
+ summary: "A summary.",
+ extraction: { entities: [{ category: "concept", name: "Alpha" }], relations: [] },
+ };
+}
+
+/**
+ * A re-ingest of the same extraction lists its entity again as a connection
+ * (it now pre-exists), so the inertness case carries no entities: what it
+ * pins is that the stamp itself never churns the page.
+ */
+function bareInput(sourcePath: string) {
+ return { sourcePath, summary: "A summary.", extraction: { entities: [], relations: [] } };
+}
+
+beforeEach(() => {
+ vault = mkdtempSync(join(tmpdir(), "o2b-ingest-scope-vault-"));
+ configHome = mkdtempSync(join(tmpdir(), "o2b-ingest-scope-cfg-"));
+ const configPath = join(configHome, "config.yaml");
+ atomicWriteFileSync(configPath, `vault: ${vault}\nagent_name: claude\n`);
+ bootstrapBrain(vault, { configPath });
+});
+
+afterEach(() => {
+ rmSync(vault, { recursive: true, force: true });
+ rmSync(configHome, { recursive: true, force: true });
+});
+
+describe("ingestSource capture scope", () => {
+ test("an in-vault source is full-local and the page is byte-identical to the base output", () => {
+ mkdirSync(join(vault, "Articles"), { recursive: true });
+ writeFileSync(join(vault, LOCAL_SOURCE), "bytes\n", "utf8");
+
+ const result = ingestSource(vault, input(LOCAL_SOURCE), { agent: "claude", now: NOW });
+
+ expect(result.captureScope).toBe(CAPTURE_SCOPE.fullLocal);
+ expect(readFileSync(join(vault, result.summaryPath), "utf8")).toBe(BASE_LOCAL_PAGE);
+ });
+
+ test("a URL source stamps capture_scope: url-only and reports it", () => {
+ const result = ingestSource(vault, input(URL_SOURCE), { agent: "claude", now: NOW });
+
+ expect(result.captureScope).toBe(CAPTURE_SCOPE.urlOnly);
+ const [meta] = parseFrontmatter(join(vault, result.summaryPath));
+ expect(meta[CAPTURE_SCOPE_KEY]).toBe(CAPTURE_SCOPE.urlOnly);
+ });
+
+ test("re-ingesting an unchanged URL source is inert", () => {
+ const first = ingestSource(vault, bareInput(URL_SOURCE), { agent: "claude", now: NOW });
+ const abs = join(vault, first.summaryPath);
+ const bytes = readFileSync(abs, "utf8");
+ const mtime = statSync(abs).mtimeMs;
+
+ const second = ingestSource(vault, bareInput(URL_SOURCE), { agent: "claude", now: NOW });
+
+ expect(second.captureScope).toBe(CAPTURE_SCOPE.urlOnly);
+ expect(second.created).toBe(false);
+ expect(readFileSync(abs, "utf8")).toBe(bytes);
+ expect(statSync(abs).mtimeMs).toBe(mtime);
+ });
+
+ test("a later re-ingest keeps the stamp beside the bumped updated_at", () => {
+ ingestSource(vault, input(URL_SOURCE), { agent: "claude", now: NOW });
+ const again = ingestSource(vault, input(URL_SOURCE), { agent: "claude", now: LATER });
+ const [meta] = parseFrontmatter(join(vault, again.summaryPath));
+ expect(meta[CAPTURE_SCOPE_KEY]).toBe(CAPTURE_SCOPE.urlOnly);
+ });
+});
diff --git a/tests/core/brain/intake/intake-trust.test.ts b/tests/core/brain/intake/intake-trust.test.ts
index 4e07c823..dd855979 100644
--- a/tests/core/brain/intake/intake-trust.test.ts
+++ b/tests/core/brain/intake/intake-trust.test.ts
@@ -307,3 +307,55 @@ describe("ingestSource derives trust from the source identity", () => {
expect(scrapedMeta[UNTRUSTED_SOURCE_FRONTMATTER_KEY]).toBeDefined();
});
});
+
+/**
+ * A caller-supplied reach predicate: a cited vault file it refuses is
+ * classified as a source with no local bytes, in both intake shapes.
+ */
+describe("intakeExtraction with a reach predicate", () => {
+ const OTHER_SOURCE = "Articles/other.md";
+ const hide = (rel: string): boolean => rel !== TRUSTED_SOURCE;
+
+ test("a refused single source is untrusted, asked by its canonical identity", () => {
+ seed(TRUSTED_SOURCE);
+ const asked: string[] = [];
+ const res = intakeExtraction(vault, EXTRACTION, {
+ agent: "claude",
+ now: NOW,
+ provenance: provenanceCiting(TRUSTED_SOURCE),
+ readable: (rel) => {
+ asked.push(rel);
+ return hide(rel);
+ },
+ });
+ expect(res.trust).toBe(INTAKE_TRUST.untrusted);
+ expect(asked).toEqual([TRUSTED_SOURCE]);
+ });
+
+ test("a refused source among several makes the whole intake untrusted", () => {
+ seed(TRUSTED_SOURCE);
+ seed(OTHER_SOURCE);
+ const res = intakeExtraction(vault, EXTRACTION, {
+ agent: "claude",
+ now: NOW,
+ provenance: {
+ level: "stated",
+ sources: [`[[${OTHER_SOURCE}]]`, `[[${TRUSTED_SOURCE}]]`],
+ premises: [],
+ },
+ readable: hide,
+ });
+ expect(res.trust).toBe(INTAKE_TRUST.untrusted);
+ });
+
+ test("a predicate that admits every source changes nothing", () => {
+ seed(TRUSTED_SOURCE);
+ const res = intakeExtraction(vault, EXTRACTION, {
+ agent: "claude",
+ now: NOW,
+ provenance: provenanceCiting(TRUSTED_SOURCE),
+ readable: () => true,
+ });
+ expect(res.trust).toBe(INTAKE_TRUST.trusted);
+ });
+});
diff --git a/tests/core/brain/intake/source-origin-bytes.test.ts b/tests/core/brain/intake/source-origin-bytes.test.ts
new file mode 100644
index 00000000..2bcd778d
--- /dev/null
+++ b/tests/core/brain/intake/source-origin-bytes.test.ts
@@ -0,0 +1,122 @@
+/**
+ * One read of a source feeds both its digest and the bytes a check runs on.
+ *
+ * `classifySourceOrigin` hashed the source and discarded the bytes, so a
+ * caller that needed the text had to read the file a second time - and could
+ * check bytes that differ from the ones whose digest it records.
+ * `readSourceOrigin` returns the bytes the digest was computed over.
+ */
+
+import { afterEach, beforeEach, describe, expect, test } from "bun:test";
+import { mkdirSync, mkdtempSync, rmSync, symlinkSync, truncateSync, writeFileSync } from "node:fs";
+import { createHash } from "node:crypto";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { hashBytes, hashFile } from "../../../../src/core/brain/ingest/content-manifest.ts";
+import {
+ classifySourceOrigin,
+ readSourceOrigin,
+ SOURCE_HASH_MAX_BYTES,
+ SourceTrustError,
+} from "../../../../src/core/brain/intake/source-trust.ts";
+import { INTAKE_TRUST } from "../../../../src/core/brain/trust/untrusted-provenance.ts";
+import { IS_WINDOWS } from "../../../helpers/platform.ts";
+
+const sha256 = (bytes: Uint8Array): string => createHash("sha256").update(bytes).digest("hex");
+
+describe("readSourceOrigin", () => {
+ let vault: string;
+
+ beforeEach(() => {
+ vault = mkdtempSync(join(tmpdir(), "source-origin-bytes-"));
+ writeFileSync(join(vault, "note.md"), "# Note\n\nBody ^p1\n");
+ });
+
+ afterEach(() => {
+ rmSync(vault, { recursive: true, force: true });
+ });
+
+ test("a vault file returns the trusted verdict, its digest and the hashed bytes", () => {
+ const origin = readSourceOrigin(vault, "note.md");
+ expect(origin.trust).toBe(INTAKE_TRUST.trusted);
+ expect(origin.contentHash).toBe(hashFile(join(vault, "note.md")));
+ expect(origin.bytes).toBeDefined();
+ expect(sha256(origin.bytes ?? new Uint8Array())).toBe(origin.contentHash ?? "");
+ expect(new TextDecoder().decode(origin.bytes)).toBe("# Note\n\nBody ^p1\n");
+ });
+
+ test("agrees with classifySourceOrigin on the verdict and the digest", () => {
+ const origin = readSourceOrigin(vault, "note.md");
+ expect(classifySourceOrigin(vault, "note.md")).toEqual({
+ trust: origin.trust,
+ contentHash: origin.contentHash,
+ });
+ });
+
+ test("a URL and a missing file are untrusted with neither digest nor bytes", () => {
+ for (const identity of ["https://x.test/a", "missing.md"]) {
+ const origin = readSourceOrigin(vault, identity);
+ expect(origin).toEqual({ trust: INTAKE_TRUST.untrusted });
+ expect("bytes" in origin).toBe(false);
+ expect("contentHash" in origin).toBe(false);
+ }
+ });
+
+ test("a file past the ceiling is refused, as classifySourceOrigin refuses it", () => {
+ const big = join(vault, "big.md");
+ writeFileSync(big, "");
+ truncateSync(big, SOURCE_HASH_MAX_BYTES + 1);
+ expect(() => readSourceOrigin(vault, "big.md")).toThrow(SourceTrustError);
+ expect(() => classifySourceOrigin(vault, "big.md")).toThrow(SourceTrustError);
+ });
+});
+
+describe("readSourceOrigin - the one descriptor read", () => {
+ let vault: string;
+
+ beforeEach(() => {
+ vault = mkdtempSync(join(tmpdir(), "source-origin-fd-"));
+ mkdirSync(join(vault, "Notes"));
+ writeFileSync(join(vault, "Notes", "real.md"), "# Real\n\nBody.\n");
+ });
+
+ afterEach(() => {
+ rmSync(vault, { recursive: true, force: true });
+ });
+
+ test.skipIf(IS_WINDOWS)("a symlink to a note inside the vault reads that note's bytes", () => {
+ symlinkSync(join(vault, "Notes", "real.md"), join(vault, "Notes", "link.md"));
+ const origin = readSourceOrigin(vault, "Notes/link.md");
+ expect(origin.trust).toBe(INTAKE_TRUST.trusted);
+ expect(origin.contentHash).toBe(hashFile(join(vault, "Notes", "real.md")));
+ });
+
+ test("an empty file is read as zero bytes with the digest of nothing", () => {
+ writeFileSync(join(vault, "Notes", "empty.md"), "");
+ const origin = readSourceOrigin(vault, "Notes/empty.md");
+ expect(origin.trust).toBe(INTAKE_TRUST.trusted);
+ expect(origin.bytes?.byteLength).toBe(0);
+ expect(origin.contentHash).toBe(sha256(new Uint8Array()));
+ });
+});
+
+describe("hashBytes", () => {
+ let dir: string;
+
+ beforeEach(() => {
+ dir = mkdtempSync(join(tmpdir(), "hash-bytes-"));
+ });
+
+ afterEach(() => {
+ rmSync(dir, { recursive: true, force: true });
+ });
+
+ test("equals hashFile for a file with a BOM and invalid UTF-8", () => {
+ const bytes = new Uint8Array([0xef, 0xbb, 0xbf, 0x61, 0xff, 0xfe, 0xc3, 0x28, 0x0a]);
+ const path = join(dir, "odd.md");
+ writeFileSync(path, bytes);
+ expect(hashBytes(bytes)).toBe(hashFile(path));
+ expect(hashBytes(bytes)).toBe(sha256(bytes));
+ });
+});
diff --git a/tests/core/brain/link-graph/parse-wikilink.test.ts b/tests/core/brain/link-graph/parse-wikilink.test.ts
index 743a9342..7cf1245b 100644
--- a/tests/core/brain/link-graph/parse-wikilink.test.ts
+++ b/tests/core/brain/link-graph/parse-wikilink.test.ts
@@ -8,7 +8,11 @@
import { describe, expect, test } from "bun:test";
-import { parseWikilinkRich } from "../../../../src/core/brain/link-graph/parse-wikilink.ts";
+import {
+ extractWikilinkRichBodies,
+ parseWikilinkRich,
+} from "../../../../src/core/brain/link-graph/parse-wikilink.ts";
+import { renderExcerptSection } from "../../../../src/core/brain/provenance/capture-scope.ts";
describe("parseWikilinkRich - bare target", () => {
test("bare target inside brackets", () => {
@@ -109,3 +113,21 @@ describe("parseWikilinkRich - immutability", () => {
expect(Object.isFrozen(r)).toBe(true);
});
});
+
+describe("extractWikilinkRichBodies - code masking", () => {
+ test("a longer fence is not closed by a shorter backtick run inside it", () => {
+ const content = "````\nx ```\n[[Hidden/Note]]\n````\nReal: [[real-link]]";
+ expect(extractWikilinkRichBodies(content)).toEqual(["real-link"]);
+ });
+
+ test("a fence closed by a longer run ends at the end of that run", () => {
+ const content = "```\ncode\n````\nAfter: [[after-link]]\n`x`";
+ expect(extractWikilinkRichBodies(content)).toEqual(["after-link"]);
+ });
+
+ test("a stored excerpt never contributes links, whatever backtick runs it holds", () => {
+ for (const excerpt of ["x ```\n[[Secret/Note]]\n", "```` [[A]] ``` [[B]]", "`` [[C]] ```"]) {
+ expect(extractWikilinkRichBodies(renderExcerptSection(excerpt))).toEqual([]);
+ }
+ });
+});
diff --git a/tests/core/brain/link-graph/unlinked-mentions.test.ts b/tests/core/brain/link-graph/unlinked-mentions.test.ts
index 078ffafc..a3b4c9d1 100644
--- a/tests/core/brain/link-graph/unlinked-mentions.test.ts
+++ b/tests/core/brain/link-graph/unlinked-mentions.test.ts
@@ -245,6 +245,28 @@ describe("findUnlinkedMentions - code block exclusion", () => {
const out = findUnlinkedMentions(vault, "pref-foo");
expect(out.length).toBe(1);
});
+
+ test("a longer fence holding a shorter backtick run stays masked", () => {
+ writePref("pref-foo", {
+ kind: "preference",
+ topic: "foo",
+ status: "confirmed",
+ principle: "x",
+ title: "Token Bucket",
+ });
+ writePref(
+ "pref-linker",
+ {
+ kind: "preference",
+ topic: "l",
+ status: "confirmed",
+ principle: "y",
+ },
+ "````\nx ```\nToken Bucket in a fence\n````\n\nUse Token Bucket in real prose.",
+ );
+ const out = findUnlinkedMentions(vault, "pref-foo");
+ expect(out.length).toBe(1);
+ });
});
describe("findUnlinkedMentions - shape", () => {
diff --git a/tests/core/brain/page-dedup.test.ts b/tests/core/brain/page-dedup.test.ts
index c6ea1ad1..1aef62fb 100644
--- a/tests/core/brain/page-dedup.test.ts
+++ b/tests/core/brain/page-dedup.test.ts
@@ -9,6 +9,7 @@ import {
patchWikilinks,
} from "../../../src/core/brain/page-dedup.ts";
import { MergeChainError } from "../../../src/core/brain/page-meta/page-id.ts";
+import { LINEAR_CEILING_MS } from "../../helpers/linear-time.ts";
let vault: string;
@@ -182,6 +183,16 @@ describe("findDuplicateCandidates skips pages a merge already resolved", () => {
});
describe("patchWikilinks", () => {
+ test("stays linear on a long backtick run and rewrites the link after it", () => {
+ const log = join(vault, "Brain", "log", "2026-05-25.md");
+ writeFileSync(log, `${"`".repeat(256 * 1024)}\nsee [[pref-old]]\n`);
+ const started = performance.now();
+ const touched = patchWikilinks(vault, "pref-old", "pref-new");
+ expect(performance.now() - started).toBeLessThan(LINEAR_CEILING_MS);
+ expect(touched).toBe(1);
+ expect(readFileSync(log, "utf8")).toContain("[[pref-new]]");
+ });
+
test("rewrites plain [[oldTarget]] references", () => {
const log = join(vault, "Brain", "log", "2026-05-25.md");
writeFileSync(log, "see [[pref-old]] for context\n");
diff --git a/tests/core/brain/provenance/capture-scope.test.ts b/tests/core/brain/provenance/capture-scope.test.ts
new file mode 100644
index 00000000..bbba3690
--- /dev/null
+++ b/tests/core/brain/provenance/capture-scope.test.ts
@@ -0,0 +1,202 @@
+/**
+ * The capture-scope vocabulary: how much of a source a page actually holds.
+ *
+ * Three members, decided structurally. `full-local` is the trusted verdict of
+ * the intake classifier (a vault-shaped identity with a readable file behind
+ * it); `url-only` is the untrusted one, so the lane and the scope cannot
+ * disagree; `bounded-local` is a `url-only` source whose page stores a
+ * verbatim excerpt in the marked block this module renders and reads.
+ */
+
+import { afterEach, beforeEach, describe, expect, test } from "bun:test";
+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
+import { createHash } from "node:crypto";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import {
+ assertExcerptAdmissible,
+ CAPTURE_EXCERPT_FENCE_INFO,
+ CAPTURE_EXCERPT_HEADING,
+ CAPTURE_EXCERPT_MAX_BYTES,
+ CAPTURE_SCOPE,
+ CAPTURE_SCOPE_KEY,
+ CAPTURE_SCOPES_KEY,
+ CaptureExcerptError,
+ captureScopeForTrust,
+ captureScopeFrontmatter,
+ captureScopesFrontmatter,
+ classifyCaptureScope,
+ EXCERPT_HASH_KEY,
+ excerptDigest,
+ readExcerptSection,
+ renderExcerptSection,
+} from "../../../../src/core/brain/provenance/capture-scope.ts";
+import { INTAKE_TRUST } from "../../../../src/core/brain/trust/untrusted-provenance.ts";
+
+describe("CAPTURE_SCOPE vocabulary", () => {
+ // Frozenness, membership and guard agreement are owned by the vocabulary
+ // census; this pins the frontmatter spellings only.
+ test("the scope tokens are the pinned spellings", () => {
+ expect(CAPTURE_SCOPE).toEqual({
+ fullLocal: "full-local",
+ boundedLocal: "bounded-local",
+ urlOnly: "url-only",
+ });
+ });
+
+ test("the frontmatter keys are the pinned spellings", () => {
+ expect(CAPTURE_SCOPE_KEY).toBe("capture_scope");
+ expect(CAPTURE_SCOPES_KEY).toBe("capture_scopes");
+ expect(EXCERPT_HASH_KEY).toBe("excerpt_hash");
+ });
+});
+
+describe("classifyCaptureScope", () => {
+ let vault: string;
+
+ beforeEach(() => {
+ vault = mkdtempSync(join(tmpdir(), "capture-scope-"));
+ mkdirSync(join(vault, "Articles", "folder.md"), { recursive: true });
+ writeFileSync(join(vault, "Articles", "x.md"), "# X\n");
+ });
+
+ afterEach(() => {
+ rmSync(vault, { recursive: true, force: true });
+ });
+
+ test("an existing vault file is full-local", () => {
+ expect(classifyCaptureScope(vault, "Articles/x.md")).toBe(CAPTURE_SCOPE.fullLocal);
+ expect(classifyCaptureScope(vault, "[[Articles/x.md]]")).toBe(CAPTURE_SCOPE.fullLocal);
+ });
+
+ test("a URL, an authority shape and a scheme-less host are url-only", () => {
+ for (const identity of ["https://x.test/a", "//x.test/a", "evil.com/x"]) {
+ expect(classifyCaptureScope(vault, identity)).toBe(CAPTURE_SCOPE.urlOnly);
+ }
+ });
+
+ test("a vault-shaped path with no file and a directory are url-only", () => {
+ expect(classifyCaptureScope(vault, "Articles/missing.md")).toBe(CAPTURE_SCOPE.urlOnly);
+ expect(classifyCaptureScope(vault, "Articles/folder.md")).toBe(CAPTURE_SCOPE.urlOnly);
+ });
+
+ test("an extensionless note identity is matched against its .md file", () => {
+ for (const identity of ["Articles/x", "[[Articles/x]]", "[[Articles/x|Alias]]"]) {
+ expect(classifyCaptureScope(vault, identity)).toBe(CAPTURE_SCOPE.fullLocal);
+ }
+ writeFileSync(join(vault, "Meeting.md"), "# Meeting\n");
+ expect(classifyCaptureScope(vault, "[[Meeting]]")).toBe(CAPTURE_SCOPE.fullLocal);
+ for (const identity of [
+ "Articles/missing",
+ "https://x.test/a",
+ "evil.com/x",
+ "Articles/x.txt",
+ ]) {
+ expect(classifyCaptureScope(vault, identity)).toBe(CAPTURE_SCOPE.urlOnly);
+ }
+ });
+
+ test("captureScopeForTrust maps the two lanes", () => {
+ expect(captureScopeForTrust(INTAKE_TRUST.trusted)).toBe(CAPTURE_SCOPE.fullLocal);
+ expect(captureScopeForTrust(INTAKE_TRUST.untrusted)).toBe(CAPTURE_SCOPE.urlOnly);
+ });
+});
+
+describe("capture-scope frontmatter", () => {
+ test("full-local writes nothing, the others carry the key", () => {
+ expect(captureScopeFrontmatter(CAPTURE_SCOPE.fullLocal)).toEqual({});
+ expect(captureScopeFrontmatter(CAPTURE_SCOPE.urlOnly)).toEqual({ capture_scope: "url-only" });
+ expect(captureScopeFrontmatter(CAPTURE_SCOPE.boundedLocal)).toEqual({
+ capture_scope: "bounded-local",
+ });
+ });
+
+ test("a list writes nothing when every member is full-local, otherwise the whole list", () => {
+ expect(captureScopesFrontmatter([])).toEqual({});
+ expect(captureScopesFrontmatter([CAPTURE_SCOPE.fullLocal, CAPTURE_SCOPE.fullLocal])).toEqual(
+ {},
+ );
+ expect(captureScopesFrontmatter([CAPTURE_SCOPE.urlOnly, CAPTURE_SCOPE.fullLocal])).toEqual({
+ capture_scopes: ["url-only", "full-local"],
+ });
+ });
+});
+
+describe("the excerpt section", () => {
+ const TRICKY = "first line\n```js\ncode()\n```\n漢字の引用 and ````four````\n";
+
+ test("render then read round-trips backtick runs, CJK and a trailing newline", () => {
+ const rendered = renderExcerptSection(TRICKY);
+ expect(rendered.startsWith(`${CAPTURE_EXCERPT_HEADING}\n\n`)).toBe(true);
+ expect(rendered).toContain(`\`\`\`\`\`${CAPTURE_EXCERPT_FENCE_INFO}\n`);
+ expect(readExcerptSection(`# Page\n\nbody\n\n${rendered}`)).toBe(TRICKY);
+ });
+
+ test("a plain excerpt uses a three-backtick fence and round-trips", () => {
+ const plain = "no fences here";
+ const rendered = renderExcerptSection(plain);
+ expect(rendered).toContain("```excerpt\nno fences here\n```");
+ expect(readExcerptSection(rendered)).toBe(plain);
+ });
+
+ test("a body without the heading, or with a broken fence, reads as null", () => {
+ expect(readExcerptSection("# Page\n\nno excerpt\n")).toBeNull();
+ expect(readExcerptSection(`${CAPTURE_EXCERPT_HEADING}\n\nnot a fence\n`)).toBeNull();
+ expect(
+ readExcerptSection(`${CAPTURE_EXCERPT_HEADING}\n\n\`\`\`excerpt\nunclosed\n`),
+ ).toBeNull();
+ });
+
+ test("the digest is sha256 over the UTF-8 bytes as given", () => {
+ const expected = createHash("sha256").update(Buffer.from(TRICKY, "utf8")).digest("hex");
+ expect(excerptDigest(TRICKY)).toBe(expected);
+ });
+});
+
+describe("assertExcerptAdmissible", () => {
+ test("refuses an excerpt for a source that is not url-only", () => {
+ expect(() => assertExcerptAdmissible(CAPTURE_SCOPE.fullLocal, "text")).toThrow(
+ CaptureExcerptError,
+ );
+ expect(() => assertExcerptAdmissible(CAPTURE_SCOPE.boundedLocal, "text")).toThrow(
+ CaptureExcerptError,
+ );
+ });
+
+ test("refuses a whitespace-only excerpt", () => {
+ expect(() => assertExcerptAdmissible(CAPTURE_SCOPE.urlOnly, " \n\t ")).toThrow(
+ CaptureExcerptError,
+ );
+ });
+
+ test("refuses an excerpt of control or format characters only, as empty", () => {
+ for (const excerpt of ["\u0001\u001b", "\u200b\u00ad \n", "\u0000"]) {
+ expect(() => assertExcerptAdmissible(CAPTURE_SCOPE.urlOnly, excerpt)).toThrow(
+ new CaptureExcerptError("excerpt refused: the excerpt is empty"),
+ );
+ }
+ });
+
+ test("refuses an excerpt holding NUL", () => {
+ expect(() => assertExcerptAdmissible(CAPTURE_SCOPE.urlOnly, "text\u0000more")).toThrow(
+ new CaptureExcerptError("excerpt refused: the excerpt contains NUL"),
+ );
+ });
+
+ test("counts bytes, not characters, against the cap", () => {
+ // U+00E9 is two UTF-8 bytes: half the cap in characters is the cap in bytes.
+ const atCap = "é".repeat(CAPTURE_EXCERPT_MAX_BYTES / 2);
+ expect(() => assertExcerptAdmissible(CAPTURE_SCOPE.urlOnly, atCap)).not.toThrow();
+ let caught: unknown;
+ try {
+ assertExcerptAdmissible(CAPTURE_SCOPE.urlOnly, `${atCap}a`);
+ } catch (cause) {
+ caught = cause;
+ }
+ expect(caught).toBeInstanceOf(CaptureExcerptError);
+ expect((caught as Error).name).toBe("CaptureExcerptError");
+ expect((caught as Error).message).toContain(String(CAPTURE_EXCERPT_MAX_BYTES + 1));
+ expect((caught as Error).message).not.toContain("é");
+ });
+});
diff --git a/tests/core/brain/research/research-capture-scope.test.ts b/tests/core/brain/research/research-capture-scope.test.ts
new file mode 100644
index 00000000..28cc2d37
--- /dev/null
+++ b/tests/core/brain/research/research-capture-scope.test.ts
@@ -0,0 +1,206 @@
+/**
+ * Per-source capture scopes on research reports. A report cites several
+ * sources, so it records one scope per consulted source, parallel to the
+ * report's `sources` order, and only when at least one of them is not
+ * `full-local` (an all-local report stays byte-identical to the pre-scope
+ * output). The result always carries the list.
+ */
+
+import { afterEach, beforeEach, describe, expect, test } from "bun:test";
+import { chmodSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
+import { join } from "node:path";
+import { tmpdir } from "node:os";
+
+import { bootstrapBrain } from "../../../../src/core/brain/init.ts";
+import { atomicWriteFileSync } from "../../../../src/core/fs-atomic.ts";
+import { writeResearchReport } from "../../../../src/core/brain/research/research.ts";
+import {
+ CAPTURE_SCOPE,
+ CAPTURE_SCOPES_KEY,
+} from "../../../../src/core/brain/provenance/capture-scope.ts";
+import { parseFrontmatter } from "../../../../src/core/vault.ts";
+import { CHMOD_CANNOT_DENY } from "../../../helpers/platform.ts";
+
+let vault: string;
+let configHome: string;
+
+const NOW = new Date("2026-06-13T12:00:00Z");
+const LOCAL_SOURCE = "Articles/a.md";
+const URL_SOURCE = "https://example.test/articles/a";
+
+/** The report page the base release (v1.66.0) wrote for one local source. */
+const BASE_LOCAL_REPORT =
+ '---\nkind: brain-report\ntitle: T\nreport_date: 2026-06-13\nprovenance: stated\nsource_count: 1\ncreated_at: "2026-06-13T12:00:00Z"\nupdated_at: "2026-06-13T12:00:00Z"\ntags: [brain, brain/report]\n---\n\n# T\n\n## Findings\n\n- S (cites: [[Articles/a.md]])\n\n## Sources\n\n- [[Articles/a.md]]\n';
+
+function seed(rel: string): void {
+ mkdirSync(join(vault, rel, ".."), { recursive: true });
+ writeFileSync(join(vault, rel), "bytes\n", "utf8");
+}
+
+function report(sources: ReadonlyArray<string>) {
+ return writeResearchReport(
+ vault,
+ { title: "T", sources, findings: [{ statement: "S", sources: [sources[0]!] }] },
+ { agent: "claude", now: NOW },
+ );
+}
+
+beforeEach(() => {
+ vault = mkdtempSync(join(tmpdir(), "o2b-research-scope-vault-"));
+ configHome = mkdtempSync(join(tmpdir(), "o2b-research-scope-cfg-"));
+ const configPath = join(configHome, "config.yaml");
+ atomicWriteFileSync(configPath, `vault: ${vault}\nagent_name: claude\n`);
+ bootstrapBrain(vault, { configPath });
+});
+
+afterEach(() => {
+ rmSync(vault, { recursive: true, force: true });
+ rmSync(configHome, { recursive: true, force: true });
+});
+
+describe("writeResearchReport capture scopes", () => {
+ test("every source an in-vault file: no key, all full-local, byte-identical page", () => {
+ seed(LOCAL_SOURCE);
+
+ const result = report([LOCAL_SOURCE]);
+
+ expect(result.captureScopes).toEqual([CAPTURE_SCOPE.fullLocal]);
+ expect(readFileSync(join(vault, result.reportPath), "utf8")).toBe(BASE_LOCAL_REPORT);
+ });
+
+ test("one URL and one vault file: capture_scopes follows the sources order", () => {
+ seed(LOCAL_SOURCE);
+
+ const result = report([URL_SOURCE, LOCAL_SOURCE]);
+
+ const expected = [CAPTURE_SCOPE.urlOnly, CAPTURE_SCOPE.fullLocal];
+ expect(result.captureScopes).toEqual(expected);
+ const [meta] = parseFrontmatter(join(vault, result.reportPath));
+ expect(meta[CAPTURE_SCOPES_KEY]).toEqual(expected);
+ });
+
+ test("a wikilinked source is classified by its target", () => {
+ seed(LOCAL_SOURCE);
+
+ const present = report([`[[${LOCAL_SOURCE}]]`, "[[Articles/missing.md|alias]]"]);
+
+ expect(present.captureScopes).toEqual([CAPTURE_SCOPE.fullLocal, CAPTURE_SCOPE.urlOnly]);
+ });
+
+ test("a source the caller cannot read is url-only in the result and on the page", () => {
+ seed(LOCAL_SOURCE);
+ seed("Notes/open.md");
+ const asked: string[] = [];
+ const readable = (rel: string): boolean => {
+ asked.push(rel);
+ return rel !== LOCAL_SOURCE;
+ };
+
+ const result = writeResearchReport(
+ vault,
+ {
+ title: "T",
+ sources: [`[[${LOCAL_SOURCE}]]`, "Notes/open.md", URL_SOURCE],
+ findings: [{ statement: "S", sources: [URL_SOURCE] }],
+ },
+ { agent: "claude", now: NOW, readable },
+ );
+
+ // The hidden file answers exactly as an absent one would.
+ const expected = [CAPTURE_SCOPE.urlOnly, CAPTURE_SCOPE.fullLocal, CAPTURE_SCOPE.urlOnly];
+ expect(result.captureScopes).toEqual(expected);
+ const [meta] = parseFrontmatter(join(vault, result.reportPath));
+ expect(meta[CAPTURE_SCOPES_KEY]).toEqual(expected);
+ // The predicate is asked about the normalised vault path, never the URL.
+ expect(asked).toEqual([LOCAL_SOURCE, "Notes/open.md"]);
+ });
+
+ test("a source cited without its extension is the note beside it", () => {
+ seed("notes/meeting.md");
+
+ const result = report(["notes/meeting", "[[notes/meeting]]", "notes/absent"]);
+
+ expect(result.captureScopes).toEqual([
+ CAPTURE_SCOPE.fullLocal,
+ CAPTURE_SCOPE.fullLocal,
+ CAPTURE_SCOPE.urlOnly,
+ ]);
+ });
+
+ test("the reach check reads the note an extensionless source resolves to", () => {
+ seed("notes/meeting.md");
+ const asked: string[] = [];
+
+ const result = writeResearchReport(
+ vault,
+ {
+ title: "T",
+ sources: ["notes/meeting"],
+ findings: [{ statement: "S", sources: ["notes/meeting"] }],
+ },
+ {
+ agent: "claude",
+ now: NOW,
+ readable: (rel) => {
+ asked.push(rel);
+ return false;
+ },
+ },
+ );
+
+ expect(asked).toEqual(["notes/meeting.md"]);
+ expect(result.captureScopes).toEqual([CAPTURE_SCOPE.urlOnly]);
+ });
+
+ test("a directory beside an extensionless source does not stand in for its note", () => {
+ seed("notes/meeting.md");
+ seed("notes/meeting/agenda.md");
+ const asked: string[] = [];
+
+ const result = writeResearchReport(
+ vault,
+ {
+ title: "T",
+ sources: ["notes/meeting"],
+ findings: [{ statement: "S", sources: ["notes/meeting"] }],
+ },
+ {
+ agent: "claude",
+ now: NOW,
+ readable: (rel) => {
+ asked.push(rel);
+ return rel !== "notes/meeting.md";
+ },
+ },
+ );
+
+ expect(asked).toEqual(["notes/meeting.md"]);
+ expect(result.captureScopes).toEqual([CAPTURE_SCOPE.urlOnly]);
+ });
+});
+
+describe("writeResearchReport when the filesystem refuses a source", () => {
+ test.skipIf(CHMOD_CANNOT_DENY)(
+ "an unreadable cited source is url-only at every reach and never aborts the report",
+ () => {
+ const locked = join(vault, "locked");
+ seed("locked/inner/a.md");
+ const sources = ["locked/inner/a.md", URL_SOURCE];
+ const input = { title: "T", sources, findings: [{ statement: "S", sources: [URL_SOURCE] }] };
+ chmodSync(locked, 0o000);
+ try {
+ const local = writeResearchReport(vault, input, { agent: "claude", now: NOW });
+ expect(local.captureScopes).toEqual([CAPTURE_SCOPE.urlOnly, CAPTURE_SCOPE.urlOnly]);
+
+ const remote = writeResearchReport(
+ vault,
+ { ...input, title: "R" },
+ { agent: "claude", now: NOW, readable: () => true },
+ );
+ expect(remote.captureScopes).toEqual([CAPTURE_SCOPE.urlOnly, CAPTURE_SCOPE.urlOnly]);
+ } finally {
+ chmodSync(locked, 0o755);
+ }
+ },
+ );
+});
diff --git a/tests/core/brain/research/research.test.ts b/tests/core/brain/research/research.test.ts
index a550adb6..4d8c41de 100644
--- a/tests/core/brain/research/research.test.ts
+++ b/tests/core/brain/research/research.test.ts
@@ -5,7 +5,7 @@
*/
import { afterEach, beforeEach, describe, expect, test } from "bun:test";
-import { mkdtempSync, readFileSync, rmSync } from "node:fs";
+import { existsSync, mkdtempSync, readFileSync, rmSync } from "node:fs";
import { join } from "node:path";
import { tmpdir } from "node:os";
@@ -85,6 +85,16 @@ describe("writeResearchReport", () => {
);
});
+ test("rejects a title that spans more than one line and writes nothing", () => {
+ for (const title of ["one\n## Sources", "one\rtwo"]) {
+ const bad = { ...INPUT, title };
+ expect(() => writeResearchReport(vault, bad, { agent: "claude", now: NOW })).toThrow(
+ "report title spans more than one line; a title is one line",
+ );
+ }
+ expect(existsSync(join(vault, "Brain", "reports"))).toBe(false);
+ });
+
test("is idempotent on date+title: a re-run rewrites the same report page", () => {
const first = writeResearchReport(vault, INPUT, { agent: "claude", now: NOW });
const second = writeResearchReport(vault, INPUT, { agent: "claude", now: LATER });
diff --git a/tests/core/brain/source-links.test.ts b/tests/core/brain/source-links.test.ts
new file mode 100644
index 00000000..cc602645
--- /dev/null
+++ b/tests/core/brain/source-links.test.ts
@@ -0,0 +1,158 @@
+/**
+ * Shared source-link readers: the frontmatter list reader and the wikilink
+ * target extractor `source-cleanup.ts` has always used, plus the `## Sources`
+ * section reader the capture-scope detector needs. One implementation, so
+ * the cleanup and the detector cannot disagree about what a page cites.
+ */
+
+import { describe, expect, test } from "bun:test";
+
+import type { FrontmatterMap } from "../../../src/core/types.ts";
+
+import {
+ sourcesSectionTargets,
+ stringArrayField,
+ wikilinkTarget,
+} from "../../../src/core/brain/source-links.ts";
+
+describe("stringArrayField", () => {
+ test("returns the string members of a list field", () => {
+ // A hand-edited page can hold anything in a list; the parser's type does not.
+ const mixed = { source: ["[[a.md]]", 3, "b", null] } as unknown as FrontmatterMap;
+ expect(stringArrayField(mixed, "source")).toEqual(["[[a.md]]", "b"]);
+ });
+
+ test("a scalar or an absent field reads as empty", () => {
+ expect(stringArrayField({ source: "[[a.md]]" }, "source")).toEqual([]);
+ expect(stringArrayField({}, "source")).toEqual([]);
+ });
+});
+
+describe("wikilinkTarget", () => {
+ test.each([
+ ["[[Articles/a.md]]", "Articles/a.md"],
+ ["[[Articles/a.md|An alias]]", "Articles/a.md"],
+ ["[[Articles/a.md#Heading]]", "Articles/a.md"],
+ ["[[Articles/a.md#^block1]]", "Articles/a.md"],
+ [" [[ Articles/a ]] ", "Articles/a"],
+ ["https://example.test/a", "https://example.test/a"],
+ [" plain ", "plain"],
+ ])("%p -> %p", (raw, target) => {
+ expect(wikilinkTarget(raw)).toBe(target);
+ });
+});
+
+describe("sourcesSectionTargets", () => {
+ test("reads the bullets under ## Sources and stops at the next heading", () => {
+ const body = [
+ "# Title",
+ "",
+ "- [[Not/a-source.md]]",
+ "",
+ "## Sources",
+ "",
+ "- [[Articles/a.md]]",
+ "- [[https://example.test/x|x]]",
+ "",
+ "## Premises (stated)",
+ "",
+ "- [[fact-1]]",
+ ].join("\n");
+
+ expect(sourcesSectionTargets(body)).toEqual(["Articles/a.md", "https://example.test/x"]);
+ });
+
+ test("a body without the section has no targets", () => {
+ expect(sourcesSectionTargets("# Title\n\n- [[Articles/a.md]]\n")).toEqual([]);
+ });
+
+ test("CRLF bodies and several links on one line are read", () => {
+ const body = "## Sources\r\n\r\n- [[a.md]], [[b.md#h]]\r\n## Next\r\n- [[c.md]]\r\n";
+ expect(sourcesSectionTargets(body)).toEqual(["a.md", "b.md"]);
+ });
+
+ test("a Sources section inside a fenced block is not read", () => {
+ // An excerpt is caller-supplied text stored in a fence before the real
+ // provenance section; its lines are quoted content, not page structure.
+ const planted = [
+ "## Excerpt",
+ "",
+ "````text",
+ "## Sources",
+ "- [[notes/local.md]]",
+ "````",
+ "",
+ "## Sources",
+ "",
+ "- [[https://example.com/a]]",
+ ].join("\n");
+ expect(sourcesSectionTargets(planted)).toEqual(["https://example.com/a"]);
+ });
+
+ test("a heading inside a fence does not end the real section", () => {
+ const body = [
+ "## Excerpt",
+ "",
+ "~~~",
+ "## Sources",
+ "# Heading in quote",
+ "~~~",
+ "",
+ "## Sources",
+ "",
+ "- [[https://example.com/a]]",
+ ].join("\n");
+ expect(sourcesSectionTargets(body)).toEqual(["https://example.com/a"]);
+ });
+
+ test("a fence closes only on a run of the same character at least as long", () => {
+ const body = [
+ "````",
+ "```",
+ "## Sources",
+ "- [[inside.md]]",
+ "~~~~",
+ "````",
+ "## Sources",
+ "- [[real.md]]",
+ ].join("\n");
+ expect(sourcesSectionTargets(body)).toEqual(["real.md"]);
+ });
+
+ test("the last Sources section is the one read", () => {
+ // The writer renders the provenance section last; an earlier planted
+ // section (for example a multi-line claim) does not stand in for it.
+ const body = [
+ "## Claims",
+ "",
+ "- x",
+ "## Sources",
+ "- [[notes/any-real.md]]",
+ "",
+ "## Sources",
+ "",
+ "- [[https://example.com/a]]",
+ ].join("\n");
+ expect(sourcesSectionTargets(body)).toEqual(["https://example.com/a"]);
+ });
+
+ test("a fence that never closes is text and hides nothing after it", () => {
+ const body = [
+ "## Findings",
+ "",
+ "- x",
+ "```",
+ "",
+ "## Sources",
+ "",
+ "- [[https://example.com/b]]",
+ ];
+ expect(sourcesSectionTargets(body.join("\n"))).toEqual(["https://example.com/b"]);
+ });
+
+ test("a deeper heading inside the section ends it too", () => {
+ expect(sourcesSectionTargets("## Sources\n- [[a.md]]\n### Sub\n- [[b.md]]\n")).toEqual([
+ "a.md",
+ ]);
+ });
+});
diff --git a/tests/core/brain/vault-guard-census.test.ts b/tests/core/brain/vault-guard-census.test.ts
index f270628e..bcdcfdfd 100644
--- a/tests/core/brain/vault-guard-census.test.ts
+++ b/tests/core/brain/vault-guard-census.test.ts
@@ -66,6 +66,10 @@ const UNGUARDED_WITH_REASON: Readonly<Record<string, string>> = Object.freeze({
"machine-local derived cache OUTSIDE the vault (the user cache dir). It " +
"writes nothing under Brain/, and a refusal would only cost the next " +
"capture a full walk, so there is no vault identity to assert.",
+ "src/core/brain/intake/source-trust.ts":
+ "never writes. It names `openSync` only to open a source read-only " +
+ "(O_RDONLY | O_NOFOLLOW | O_NONBLOCK) through one descriptor so the read " +
+ "is bounded, so there is no write to guard.",
"src/core/brain/init.ts":
"bootstrap. This is the path that CREATES the tree and stamps the marker; " +
"gating it on the marker it is about to write would refuse `o2b brain init` " +
diff --git a/tests/core/search/links.test.ts b/tests/core/search/links.test.ts
index 16897b0e..8e4f49db 100644
--- a/tests/core/search/links.test.ts
+++ b/tests/core/search/links.test.ts
@@ -1,5 +1,7 @@
import { test, expect } from "bun:test";
+import { renderExcerptSection } from "../../../src/core/brain/provenance/capture-scope.ts";
import { extractLinks } from "../../../src/core/search/links.ts";
+import { LINEAR_CEILING_MS } from "../../helpers/linear-time.ts";
test("extracts wikilinks with and without alt text", () => {
const links = extractLinks("See [[foo-note]] and [[bar|the bar]].");
@@ -179,3 +181,28 @@ test("reference links inside code fences are ignored", () => {
expect(md.length).toBe(1);
expect(md[0]?.targetPath).toBe("./design.md");
});
+
+test("a fence closes only on a run of its own character at least as long", () => {
+ const longer = ["````", "x", "```", "[[inside-longer-fence]]", "````", "After [[real-link]]."];
+ const tilde = ["~~~", "```", "[[inside-tilde-fence]]", "```", "~~~", "After [[real-link]]."];
+ for (const lines of [longer, tilde]) {
+ const targets = extractLinks(lines.join("\n")).map((l) => l.targetPath);
+ expect(targets).toEqual(["real-link"]);
+ }
+});
+
+test("a stored excerpt contributes no links, whatever backtick lines it holds", () => {
+ for (const excerpt of ["x\n```\n[[Secret/Note]]\n", "```\n[[A]]\n```\n[[B]]"]) {
+ const page = `Intro [[real-link]].\n\n${renderExcerptSection(excerpt)}\nTail [[tail-link]].`;
+ const targets = extractLinks(page).map((l) => l.targetPath);
+ expect(targets).toEqual(["real-link", "tail-link"]);
+ }
+});
+
+test("link extraction stays linear on a long line of fence marks", () => {
+ for (const mark of ["`", "~"]) {
+ const started = performance.now();
+ extractLinks(`${mark.repeat(256 * 1024)} [[x]]`);
+ expect(performance.now() - started).toBeLessThan(LINEAR_CEILING_MS);
+ }
+});
diff --git a/tests/core/vault.test.ts b/tests/core/vault.test.ts
index dfc2b8b9..d2658db3 100644
--- a/tests/core/vault.test.ts
+++ b/tests/core/vault.test.ts
@@ -13,6 +13,8 @@ import {
import { tmpdir } from "node:os";
import { join } from "node:path";
+import { renderExcerptSection } from "../../src/core/brain/provenance/capture-scope.ts";
+import { LINEAR_CEILING_MS } from "../helpers/linear-time.ts";
import { DEGRADATION_CODE, type DegradationNotice } from "../../src/core/integrity/degradation.ts";
import {
FrontmatterKeyError,
@@ -385,6 +387,41 @@ describe("extractWikilinks", () => {
test("deduplicates", () => {
expect(extractWikilinks("[[A]] [[A]] [[B]]")).toEqual(["A", "B"]);
});
+
+ test("a longer fence is not closed by a shorter backtick run inside it", () => {
+ const content = "````\nx ```\n[[Hidden/Note.md]]\n````\nReal: [[real-link]]";
+ expect(extractWikilinks(content)).toEqual(["real-link"]);
+ });
+
+ test("a stored excerpt never contributes links, whatever backtick runs it holds", () => {
+ // The excerpt is fetched text; its wikilinks are quoted content, not
+ // edges of the page that stores it.
+ for (const excerpt of ["x ```\n[[Secret/Note.md]]\n", "```` [[A]] ``` [[B]]", "`` [[C]] ```"]) {
+ expect(extractWikilinks(renderExcerptSection(excerpt))).toEqual([]);
+ }
+ });
+
+ test("stays linear on long backtick runs", () => {
+ const largestExcerpt = renderExcerptSection("`".repeat(65_536));
+ for (const content of ["`".repeat(256 * 1024), `${largestExcerpt}\n[[after]]`]) {
+ const started = performance.now();
+ extractWikilinks(content);
+ expect(performance.now() - started).toBeLessThan(LINEAR_CEILING_MS);
+ }
+ expect(extractWikilinks(`${largestExcerpt}\n[[after]]`)).toEqual(["after"]);
+ });
+
+ test("an unclosed backtick run hides no link after it", () => {
+ const content = "Intro ```\nsee [[Alpha]] and [[Beta]]\nthen `x` and [[Gamma]]";
+ expect(extractWikilinks(content)).toEqual(["Alpha", "Beta", "Gamma"]);
+ expect(extractWikilinks("Intro ``\nsee [[Alpha]]\nthen `x`")).toEqual(["Alpha"]);
+ });
+
+ test("a tilde fence masks its links like a backtick fence", () => {
+ expect(extractWikilinks("~~~\n[[in-tilde-fence]]\n~~~\nReal: [[real-link]]")).toEqual([
+ "real-link",
+ ]);
+ });
});
/**
diff --git a/tests/docs/distilled-provenance-doc.test.ts b/tests/docs/distilled-provenance-doc.test.ts
new file mode 100644
index 00000000..6f72c17c
--- /dev/null
+++ b/tests/docs/distilled-provenance-doc.test.ts
@@ -0,0 +1,53 @@
+/**
+ * The distillation reference tables in docs/mcp.md stay in step with the
+ * vocabularies and the input schema they describe. Every expected row is
+ * derived from the source of truth, never copied, so a new outcome, scope or
+ * argument that the docs do not name fails here.
+ */
+
+import { expect, test } from "bun:test";
+import { readFileSync } from "node:fs";
+import { join } from "node:path";
+
+import {
+ QUOTE_CHECK_OUTCOMES,
+ QUOTE_UNVERIFIED_CODE,
+ VERIFIED_QUOTE_OUTCOMES,
+} from "../../src/core/brain/distill/quote-verdict.ts";
+import { CAPTURE_SCOPES } from "../../src/core/brain/provenance/capture-scope.ts";
+import { DISTILL_TOOLS } from "../../src/mcp/brain/distill-tools.ts";
+
+const MCP_DOC = readFileSync(join(import.meta.dir, "..", "..", "docs", "mcp.md"), "utf8");
+
+/** The text of one `## ` section, up to the next heading of the same depth. */
+function section(heading: string): string {
+ const start = MCP_DOC.indexOf(`\n## ${heading}`);
+ expect(start).toBeGreaterThan(-1);
+ const end = MCP_DOC.indexOf("\n## ", start + 4);
+ return MCP_DOC.slice(start, end === -1 ? undefined : end);
+}
+
+const DISTILL = section("Source distillation");
+
+test("every brain_distill_source input property has an argument row", () => {
+ const props = Object.keys(DISTILL_TOOLS[0]!.inputSchema["properties"] as object);
+ expect(props.length).toBeGreaterThan(0);
+ for (const name of props) expect(DISTILL, name).toContain(`| \`${name}\` |`);
+});
+
+test("every failing quote outcome is named in the findings row", () => {
+ const row = DISTILL.split("\n").find((line) => line.startsWith("| `findings` |"));
+ expect(row).toBeDefined();
+ const verified = new Set<string>(VERIFIED_QUOTE_OUTCOMES);
+ for (const outcome of QUOTE_CHECK_OUTCOMES) {
+ if (!verified.has(outcome)) expect(row, outcome).toContain(`\`${outcome}\``);
+ }
+});
+
+test("every capture scope has a row", () => {
+ for (const scope of CAPTURE_SCOPES) expect(DISTILL, scope).toContain(`| \`${scope}\` |`);
+});
+
+test("the strict refusal code has a row in the error-code table", () => {
+ expect(MCP_DOC).toContain(`| \`${QUOTE_UNVERIFIED_CODE}\` |`);
+});
diff --git a/tests/helpers/brain-pages.ts b/tests/helpers/brain-pages.ts
new file mode 100644
index 00000000..2bf7ae31
--- /dev/null
+++ b/tests/helpers/brain-pages.ts
@@ -0,0 +1,20 @@
+/**
+ * Test helper: the text of every Markdown page under a vault's `Brain/`,
+ * for assertions that must hold on whatever page a write produced (no page
+ * records a digest of a withheld source), not only on the page named in
+ * the result.
+ */
+
+import { readFileSync, readdirSync } from "node:fs";
+import { join } from "node:path";
+
+/** Every Markdown page under `<root>/Brain/`, as text. */
+export function brainPageTexts(root: string): string[] {
+ const out: string[] = [];
+ for (const entry of readdirSync(join(root, "Brain"), { recursive: true, withFileTypes: true })) {
+ if (entry.isFile() && entry.name.endsWith(".md")) {
+ out.push(readFileSync(join(entry.parentPath, entry.name), "utf8"));
+ }
+ }
+ return out;
+}
diff --git a/tests/helpers/linear-time.ts b/tests/helpers/linear-time.ts
new file mode 100644
index 00000000..d1d45d20
--- /dev/null
+++ b/tests/helpers/linear-time.ts
@@ -0,0 +1,8 @@
+/**
+ * Ceiling for the regression guards that prove a pass over a large input
+ * stays linear. A quadratic pass over 256 KiB takes tens of seconds (the old
+ * quote normaliser: 50 s, the old fence mask: 20 s); linear code takes well
+ * under 0.5 s even on a loaded runner, so the ceiling is generous on purpose
+ * and still discriminates by an order of magnitude.
+ */
+export const LINEAR_CEILING_MS = 2000;
diff --git a/tests/mcp/brain-update-append-note.test.ts b/tests/mcp/brain-update-append-note.test.ts
index 188b6476..29be408f 100644
--- a/tests/mcp/brain-update-append-note.test.ts
+++ b/tests/mcp/brain-update-append-note.test.ts
@@ -26,6 +26,7 @@ import { NOTES_TOOLS, writeBatchErrorToMcp } from "../../src/mcp/brain/notes-too
import { WriteBatchError } from "../../src/core/brain/write-batch.ts";
import { PAGE_LINT_KEY } from "../../src/core/brain/page-lint.ts";
import { INTERNAL_ERROR, INVALID_PARAMS, MCPError } from "../../src/mcp/protocol.ts";
+import { TRANSPORT_REACH } from "../../src/core/graph/transport-reach.ts";
import type { ServerContext } from "../../src/mcp/tool-contract.ts";
/**
@@ -180,8 +181,14 @@ describe("brain_update_note", () => {
// A directory where the note should be: it exists, and reading it
// raises EISDIR regardless of the running user.
mkdirSync(join(vault, "Notes/Doc.md"), { recursive: true });
+ // At local reach: a remote caller is answered as for a missing note,
+ // because a page nobody can measure is treated as one that reserved
+ // itself. The diagnostic is for the operator who can fix the file.
const err = await rejectedMcpError(
- updateTool.handler(ctx, { path: "Notes/Doc.md", content: "replacement" }),
+ updateTool.handler(
+ { ...ctx, reach: TRANSPORT_REACH.local },
+ { path: "Notes/Doc.md", content: "replacement" },
+ ),
);
expect(err.code).toBe(INTERNAL_ERROR);
expect(err.data).toMatchObject({ code: "target_unreadable", path: "Notes/Doc.md" });
@@ -224,8 +231,12 @@ describe("brain_append_note", () => {
test("an existing note the host cannot read is refused, not overwritten", async () => {
mkdirSync(join(vault, "Notes/Doc.md"), { recursive: true });
+ // At local reach, for the reason the update case above gives.
const err = await rejectedMcpError(
- appendTool.handler(ctx, { path: "Notes/Doc.md", content: "more" }),
+ appendTool.handler(
+ { ...ctx, reach: TRANSPORT_REACH.local },
+ { path: "Notes/Doc.md", content: "more" },
+ ),
);
expect(err.code).toBe(INTERNAL_ERROR);
expect(err.data).toMatchObject({ code: "target_unreadable", path: "Notes/Doc.md" });
diff --git a/tests/mcp/brain-write-batch.test.ts b/tests/mcp/brain-write-batch.test.ts
index fa9f7958..19df6451 100644
--- a/tests/mcp/brain-write-batch.test.ts
+++ b/tests/mcp/brain-write-batch.test.ts
@@ -41,6 +41,7 @@ import { MAX_BATCH_OPERATIONS } from "../../src/core/brain/write-batch.ts";
import * as ledger from "../../src/core/brain/idempotency-ledger.ts";
import { PAGE_LINT_KEY } from "../../src/core/brain/page-lint.ts";
import { INVALID_PARAMS, MCPError } from "../../src/mcp/protocol.ts";
+import { TRANSPORT_REACH } from "../../src/core/graph/transport-reach.ts";
import type { ServerContext } from "../../src/mcp/tool-contract.ts";
/**
@@ -249,10 +250,18 @@ describe("brain_write_batch", () => {
mkdirSync(join(vault, "Notes/Unreadable.md"), { recursive: true });
let thrown: unknown;
try {
- await runBatch([
- { op: "create_note", path: "Notes/First.md", content: "one" },
- { op: "update_note", path: "Notes/Unreadable.md", content: "two" },
- ]);
+ // At local reach: a remote caller is answered as for a missing
+ // note, because a page nobody can measure is treated as one that
+ // reserved itself. The diagnostic is for the operator.
+ await tool.handler(
+ { ...ctx, reach: TRANSPORT_REACH.local },
+ {
+ operations: [
+ { op: "create_note", path: "Notes/First.md", content: "one" },
+ { op: "update_note", path: "Notes/Unreadable.md", content: "two" },
+ ],
+ },
+ );
} catch (err) {
thrown = err;
}
diff --git a/tests/mcp/delete-by-source-reach.test.ts b/tests/mcp/delete-by-source-reach.test.ts
new file mode 100644
index 00000000..34f78ed6
--- /dev/null
+++ b/tests/mcp/delete-by-source-reach.test.ts
@@ -0,0 +1,103 @@
+/**
+ * `brain_delete_by_source` answers for an original the caller may not read
+ * at its reach exactly as for an absent one.
+ *
+ * The original named by `source_file` is a vault file outside `Brain/`.
+ * A page that reserved itself is not confirmed by the plan (`originals`,
+ * `blast_radius`, `matched`), and a confirmed run with
+ * `include_originals` leaves it on disk, because a caller that cannot
+ * read the page cannot learn that it exists by asking to delete it, and
+ * cannot remove it either. The same call at local reach plans and
+ * deletes it as before.
+ */
+
+import { afterEach, describe, expect, test } from "bun:test";
+import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { bootstrapBrain } from "../../src/core/brain/init.ts";
+import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
+import { TRANSPORT_REACH } from "../../src/core/graph/transport-reach.ts";
+import { MCPServer } from "../../src/mcp/server.ts";
+
+const PRIVATE_PATH = "Notes/secret.md";
+const SECRET_BODY = "---\nvisibility: private\n---\n# Secret plan\nThe PIN is 4711.\n";
+
+const bases: string[] = [];
+
+afterEach(() => {
+ for (const base of bases.splice(0)) rmSync(base, { recursive: true, force: true });
+});
+
+interface Fixture {
+ readonly vault: string;
+ readonly base: string;
+ readonly server: MCPServer;
+}
+
+/**
+ * One vault holding the private page, or one where it never existed. With
+ * no `reach`, the server mints none, which fails closed to remote.
+ */
+function fixture(withSecret: boolean, reach?: typeof TRANSPORT_REACH.local): Fixture {
+ const base = mkdtempSync(join(tmpdir(), "o2b-dbs-reach-"));
+ bases.push(base);
+ const vault = join(base, "vault");
+ mkdirSync(join(vault, "Notes"), { recursive: true });
+ const configPath = join(base, "config.yaml");
+ atomicWriteFileSync(configPath, `vault: ${vault}\nagent_name: claude\n`);
+ bootstrapBrain(vault, { configPath });
+ writeFileSync(join(vault, "Notes/open.md"), "# Open\n");
+ if (withSecret) writeFileSync(join(vault, PRIVATE_PATH), SECRET_BODY);
+ const server = new MCPServer({ vault, configPath }, reach !== undefined ? { reach } : undefined);
+ return { vault, base, server };
+}
+
+/** The structured answer with the per-vault parts (paths, run ids, timestamps) normalised. */
+async function answer(f: Fixture, args: Record<string, unknown>): Promise<string> {
+ let raw: string;
+ try {
+ raw = JSON.stringify(await f.server.callTool("brain_delete_by_source", args));
+ } catch (err) {
+ raw = `error: ${(err as Error).message}`;
+ }
+ return raw
+ .split(f.vault)
+ .join("<vault>")
+ .replace(/\d{4}-\d{2}-\d{2}T[\d:.]+Z?/g, "<ts>")
+ .replace(/"(snapshot_run_id|audit_record_id)":"[^"]*"/g, '"$1":"<id>"');
+}
+
+describe("an original the caller cannot read answers as an absent one", () => {
+ test("the dry run does not confirm it", async () => {
+ const hidden = fixture(true);
+ const absent = fixture(false);
+ const args = { source_file: PRIVATE_PATH };
+ expect(await answer(hidden, args)).toBe(await answer(absent, args));
+ });
+
+ test("a confirmed run with include_originals leaves it on disk", async () => {
+ const hidden = fixture(true);
+ const absent = fixture(false);
+ const args = { source_file: PRIVATE_PATH, confirm: true, include_originals: true };
+ expect(await answer(hidden, args)).toBe(await answer(absent, args));
+ expect(existsSync(join(hidden.vault, PRIVATE_PATH))).toBe(true);
+ });
+});
+
+describe("at local reach the original is planned and deleted", () => {
+ test("dry run and confirmed run", async () => {
+ const local = fixture(true, TRANSPORT_REACH.local);
+ const plan = await local.server.callTool("brain_delete_by_source", {
+ source_file: PRIVATE_PATH,
+ });
+ expect(JSON.stringify(plan)).toContain(`"originals":["${PRIVATE_PATH}"]`);
+ await local.server.callTool("brain_delete_by_source", {
+ source_file: PRIVATE_PATH,
+ confirm: true,
+ include_originals: true,
+ });
+ expect(existsSync(join(local.vault, PRIVATE_PATH))).toBe(false);
+ });
+});
diff --git a/tests/mcp/distill-quotes-envelope.test.ts b/tests/mcp/distill-quotes-envelope.test.ts
new file mode 100644
index 00000000..691145bd
--- /dev/null
+++ b/tests/mcp/distill-quotes-envelope.test.ts
@@ -0,0 +1,106 @@
+/**
+ * A strict quote refusal reaches the wire with its registered code
+ * (distilled provenance, D1).
+ *
+ * `wrapToolErrors` drops error data, so `brain_distill_source` maps
+ * `QuoteCheckError` itself. This suite drives the real server so the code
+ * is read where a client reads it - `error.data.code` on the JSON-RPC error
+ * - and proves the refusal wrote nothing.
+ */
+
+import { afterEach, beforeEach, describe, expect, test } from "bun:test";
+import { existsSync, mkdirSync, mkdtempSync, readdirSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { QUOTE_UNVERIFIED_CODE } from "../../src/core/brain/distill/quote-verdict.ts";
+import { bootstrapBrain } from "../../src/core/brain/init.ts";
+import { BRAIN_DISTILLATIONS_REL } from "../../src/core/brain/path-constants.ts";
+import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
+import {
+ INVALID_PARAMS,
+ JSONRPC_VERSION,
+ MCPServer,
+ PROTOCOL_VERSION,
+} from "../../src/mcp/index.ts";
+import { readRpcErrorCode } from "../helpers/tool-error-envelope.ts";
+
+const TOOL = "brain_distill_source";
+const SOURCE = "Articles/quoted.md";
+const PARAPHRASE = { text: 'The author writes "settles all batches quickly".', block: "p1" };
+
+let vault: string;
+let configHome: string;
+let configPath: string;
+
+beforeEach(() => {
+ vault = mkdtempSync(join(tmpdir(), "o2b-distill-envelope-vault-"));
+ configHome = mkdtempSync(join(tmpdir(), "o2b-distill-envelope-cfg-"));
+ configPath = join(configHome, "config.yaml");
+ atomicWriteFileSync(configPath, `vault: ${vault}\nagent_name: claude\n`);
+ bootstrapBrain(vault, { configPath });
+ mkdirSync(join(vault, "Articles"), { recursive: true });
+ writeFileSync(
+ join(vault, SOURCE),
+ "# Quoted\n\nThe protocol settles every batch within one minute. ^p1\n",
+ "utf8",
+ );
+});
+
+afterEach(() => {
+ rmSync(vault, { recursive: true, force: true });
+ rmSync(configHome, { recursive: true, force: true });
+});
+
+type Response = { result?: { content: ReadonlyArray<{ text: string }> }; error?: any };
+
+async function call(args: Record<string, unknown>): Promise<Response> {
+ const server = new MCPServer({ vault, configPath });
+ await server.handleRequest({
+ jsonrpc: JSONRPC_VERSION,
+ id: 1,
+ method: "initialize",
+ params: {
+ protocolVersion: PROTOCOL_VERSION,
+ capabilities: {},
+ clientInfo: { name: "distill-envelope-test", version: "0" },
+ },
+ });
+ await server.handleRequest({ jsonrpc: JSONRPC_VERSION, method: "notifications/initialized" });
+ return (await server.handleRequest({
+ jsonrpc: JSONRPC_VERSION,
+ id: 9,
+ method: "tools/call",
+ params: { name: TOOL, arguments: args },
+ })) as Response;
+}
+
+/** Distillation pages on disk; the directory may not exist yet. */
+function distillationPages(): string[] {
+ const dir = join(vault, BRAIN_DISTILLATIONS_REL);
+ return existsSync(dir) ? readdirSync(dir).filter((name) => name.endsWith(".md")) : [];
+}
+
+describe("brain_distill_source strict refusal on the wire", () => {
+ test("answers a JSON-RPC error carrying quote_unverified and writes nothing", async () => {
+ const res = await call({ source_path: SOURCE, claims: [PARAPHRASE], strict_quotes: true });
+ expect(res.result).toBeUndefined();
+ expect(res.error.code).toBe(INVALID_PARAMS);
+ expect(readRpcErrorCode(res)).toBe(QUOTE_UNVERIFIED_CODE);
+ expect(res.error.message).toStartWith(`${TOOL}: `);
+ expect(res.error.message).toContain("claim 0: not-in-block");
+ // The message names indices and outcomes, never the caller's words.
+ expect(res.error.message).not.toContain("settles all batches quickly");
+ expect(distillationPages()).toEqual([]);
+ });
+
+ test("without strict_quotes the same claim lands unquoted and is reported", async () => {
+ const res = await call({ source_path: SOURCE, claims: [PARAPHRASE] });
+ expect(res.error).toBeUndefined();
+ const payload = JSON.parse(res.result!.content[0]!.text) as {
+ quotes: { unquoted: number };
+ };
+ expect(payload.quotes.unquoted).toBe(1);
+ expect(distillationPages()).toHaveLength(1);
+ });
+});
diff --git a/tests/mcp/distill-tool.test.ts b/tests/mcp/distill-tool.test.ts
index 4deeaa82..8e3beedb 100644
--- a/tests/mcp/distill-tool.test.ts
+++ b/tests/mcp/distill-tool.test.ts
@@ -13,12 +13,21 @@ import { bootstrapBrain } from "../../src/core/brain/init.ts";
import { hashFile } from "../../src/core/brain/ingest/content-manifest.ts";
import {
INTAKE_TRUST,
+ SOURCE_CONTENT_HASH_FRONTMATTER_KEY,
UNTRUSTED_SOURCE_FRONTMATTER_KEY,
} from "../../src/core/brain/trust/untrusted-provenance.ts";
import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
import { DISTILL_TOOLS } from "../../src/mcp/brain/distill-tools.ts";
-import { MCPError } from "../../src/mcp/protocol.ts";
+import {
+ CAPTURE_EXCERPT_MAX_BYTES,
+ CAPTURE_SCOPE,
+} from "../../src/core/brain/provenance/capture-scope.ts";
+import { DISTILL_CLAIMS_MAX } from "../../src/core/brain/distill/distill-source.ts";
+import { QUOTE_CHECK_OUTCOME } from "../../src/core/brain/distill/quote-verdict.ts";
+import { INVALID_PARAMS, MCPError } from "../../src/mcp/protocol.ts";
+import { PROPERTY_DESCRIPTION_MAX, TOOL_DESCRIPTION_MAX } from "../../src/mcp/registry-guard.ts";
import type { ServerContext } from "../../src/mcp/tool-contract.ts";
+import { TRANSPORT_REACH } from "../../src/core/graph/transport-reach.ts";
let vault: string;
let configHome: string;
@@ -97,3 +106,211 @@ describe("brain_distill_source - the response names the lane it committed in", (
expect(DISTILL_TOOLS[0]!.description).toContain(UNTRUSTED_SOURCE_FRONTMATTER_KEY);
});
});
+
+/** A source whose one paragraph carries the block id `^p1`. */
+function seedQuotedSource(): void {
+ writeFileSync(
+ join(vault, "Articles", "quoted.md"),
+ "# Quoted\n\nThe protocol settles every batch within one minute. ^p1\n",
+ "utf8",
+ );
+}
+
+/**
+ * The quote check and the capture scope reach the MCP caller (distilled
+ * provenance, D1). The core decides; this surface declares the two new
+ * inputs, forwards them, and serialises the two new result members under
+ * their snake_case names.
+ */
+describe("brain_distill_source - quote check and capture scope", () => {
+ const VERBATIM = 'The author writes "settles every batch within one minute".';
+ const PARAPHRASE = 'The author writes "settles all batches quickly".';
+ const QUOTE_REPORT_KEYS = [
+ "checked",
+ "verified_in_block",
+ "verified_in_source",
+ "unquoted",
+ "unpaired",
+ "findings",
+ "total",
+ "returned",
+ "truncated",
+ ];
+
+ test("the two inputs are declared within the registry caps", () => {
+ const tool = DISTILL_TOOLS[0]!;
+ const props = tool.inputSchema["properties"] as Record<
+ string,
+ { type: string; description: string; maxLength?: number }
+ >;
+ expect(props["strict_quotes"]?.type).toBe("boolean");
+ expect(props["excerpt"]?.type).toBe("string");
+ expect(props["excerpt"]?.maxLength).toBe(CAPTURE_EXCERPT_MAX_BYTES);
+ expect(props["claims"]?.description).toContain(String(DISTILL_CLAIMS_MAX));
+ for (const name of ["strict_quotes", "excerpt"]) {
+ expect(props[name]!.description.length).toBeLessThanOrEqual(PROPERTY_DESCRIPTION_MAX);
+ }
+ expect(tool.description.length).toBeLessThanOrEqual(TOOL_DESCRIPTION_MAX);
+ expect(tool.description).toContain("search_trust_gate_enabled");
+ });
+
+ test("capture_scope is always present and quotes only when a claim holds a span", async () => {
+ const res = (await handler(ctx, {
+ source_path: "Articles/src.md",
+ claims: [{ text: "A claim without quotation marks." }],
+ })) as Record<string, unknown>;
+ expect(res["capture_scope"]).toBe(CAPTURE_SCOPE.fullLocal);
+ expect("quotes" in res).toBe(false);
+ });
+
+ test("a checked quote is reported under the contract keys verbatim", async () => {
+ seedQuotedSource();
+ const res = (await handler(ctx, {
+ source_path: "Articles/quoted.md",
+ claims: [
+ { text: VERBATIM, block: "p1" },
+ { text: PARAPHRASE, block: "p1" },
+ ],
+ })) as { quotes: Record<string, unknown>; distillation_path: string };
+ expect(Object.keys(res.quotes).toSorted()).toEqual(QUOTE_REPORT_KEYS.toSorted());
+ expect(res.quotes["verified_in_block"]).toBe(1);
+ expect(res.quotes["unquoted"]).toBe(1);
+ expect(res.quotes["findings"]).toEqual([
+ {
+ claim: 1,
+ outcome: QUOTE_CHECK_OUTCOME.notInBlock,
+ span: "settles all batches quickly",
+ },
+ ]);
+ });
+
+ test("a url-only source with an excerpt is bounded-local", async () => {
+ const res = (await handler(ctx, {
+ source_path: "https://example.test/post",
+ claims: [{ text: "A claim." }],
+ excerpt: "The protocol settles every batch within one minute.",
+ })) as Record<string, unknown>;
+ expect(res["capture_scope"]).toBe(CAPTURE_SCOPE.boundedLocal);
+ });
+
+ test("an excerpt for a source the vault holds is refused as invalid_params", async () => {
+ let caught: unknown;
+ try {
+ await handler(ctx, {
+ source_path: "Articles/src.md",
+ claims: [{ text: "A claim." }],
+ excerpt: "Body.",
+ });
+ } catch (err) {
+ caught = err;
+ }
+ expect(caught).toBeInstanceOf(MCPError);
+ expect((caught as MCPError).code).toBe(INVALID_PARAMS);
+ expect((caught as MCPError).message).toStartWith("brain_distill_source: ");
+ expect((caught as MCPError).message).toContain(CAPTURE_SCOPE.fullLocal);
+ });
+
+ test("a blank excerpt is refused by name, never dropped as absent", async () => {
+ await expect(
+ handler(ctx, {
+ source_path: "https://example.test/post",
+ claims: [{ text: "A claim." }],
+ excerpt: " ",
+ }),
+ ).rejects.toThrow("excerpt refused: the excerpt is empty");
+ });
+});
+
+/**
+ * A page the caller cannot read at its reach answers the quote check exactly
+ * as a source with no local bytes does: every span settles `url-only` and no
+ * digest is returned or recorded, so the check never tells a caller whether
+ * a guessed phrase occurs in a page it may not read.
+ */
+describe("brain_distill_source - a page withheld at the caller's reach checks nothing", () => {
+ const PRIVATE_PATH = "Notes/secret.md";
+ const GUESS_RIGHT = "“The code is ZX8”";
+ const GUESS_WRONG = "“The code is ZX9”";
+
+ beforeEach(() => {
+ mkdirSync(join(vault, "Notes"), { recursive: true });
+ writeFileSync(
+ join(vault, PRIVATE_PATH),
+ "---\nvisibility: private\n---\nThe code is ZX8 today. ^p1\n",
+ "utf8",
+ );
+ });
+
+ const claims = [{ text: GUESS_RIGHT }, { text: GUESS_WRONG, block: "p1" }];
+
+ test("by default every span is url-only and no digest is returned or written", async () => {
+ const res = (await handler(ctx, { source_path: PRIVATE_PATH, claims })) as Record<
+ string,
+ unknown
+ > & {
+ quotes: { findings: Array<{ outcome: string }>; verified_in_block: number };
+ distillation_path: string;
+ };
+ expect(res.quotes.findings.map((f) => f.outcome)).toEqual([
+ QUOTE_CHECK_OUTCOME.urlOnly,
+ QUOTE_CHECK_OUTCOME.urlOnly,
+ ]);
+ expect("source_hash" in res).toBe(false);
+ const md = readFileSync(join(vault, res.distillation_path), "utf8");
+ expect(md).not.toContain("source_hash");
+ expect(md).not.toContain(SOURCE_CONTENT_HASH_FRONTMATTER_KEY);
+ });
+
+ test("strict mode refuses every guess alike, naming url-only only", async () => {
+ let caught: unknown;
+ try {
+ await handler(ctx, { source_path: PRIVATE_PATH, claims, strict_quotes: true });
+ } catch (err) {
+ caught = err;
+ }
+ expect(caught).toBeInstanceOf(MCPError);
+ const message = (caught as MCPError).message;
+ expect(message).toContain(`claim 0: ${QUOTE_CHECK_OUTCOME.urlOnly}`);
+ expect(message).toContain(`claim 1: ${QUOTE_CHECK_OUTCOME.urlOnly}`);
+ for (const outcome of Object.values(QUOTE_CHECK_OUTCOME)) {
+ if (outcome !== QUOTE_CHECK_OUTCOME.urlOnly) expect(message).not.toContain(outcome);
+ }
+ });
+
+ test("the lane, scope and page match an absent source, and an excerpt is admitted", async () => {
+ const ABSENT = "Notes/absent.md";
+ const shape = (res: Record<string, unknown>) => ({
+ trust: res["trust"],
+ capture_scope: res["capture_scope"],
+ });
+ const hiddenRes = (await handler(ctx, { source_path: PRIVATE_PATH, claims })) as Record<
+ string,
+ unknown
+ > & { distillation_path: string };
+ const absentRes = (await handler(ctx, { source_path: ABSENT, claims })) as Record<
+ string,
+ unknown
+ >;
+ expect(shape(hiddenRes)).toEqual(shape(absentRes));
+ expect(hiddenRes["trust"]).toBe(INTAKE_TRUST.untrusted);
+ expect(hiddenRes["capture_scope"]).toBe(CAPTURE_SCOPE.urlOnly);
+ const md = readFileSync(join(vault, hiddenRes.distillation_path), "utf8");
+ expect(md).toContain(UNTRUSTED_SOURCE_FRONTMATTER_KEY);
+
+ const withExcerpt = (await handler(ctx, {
+ source_path: PRIVATE_PATH,
+ claims,
+ excerpt: "The code is ZX9 today.",
+ })) as Record<string, unknown>;
+ expect(withExcerpt["capture_scope"]).toBe(CAPTURE_SCOPE.boundedLocal);
+ });
+
+ test("at local reach the same page is checked", async () => {
+ const res = (await handler(
+ { ...ctx, reach: TRANSPORT_REACH.local },
+ { source_path: PRIVATE_PATH, claims },
+ )) as { quotes: { verified_in_source: number; findings: Array<{ outcome: string }> } };
+ expect(res.quotes.verified_in_source).toBe(1);
+ expect(res.quotes.findings.map((f) => f.outcome)).toEqual([QUOTE_CHECK_OUTCOME.notInBlock]);
+ });
+});
diff --git a/tests/mcp/doctor-schema-reach.test.ts b/tests/mcp/doctor-schema-reach.test.ts
new file mode 100644
index 00000000..67de99ba
--- /dev/null
+++ b/tests/mcp/doctor-schema-reach.test.ts
@@ -0,0 +1,109 @@
+/**
+ * `brain_doctor` and the `schema_inspect` record views answer for a page
+ * the caller may not read at its reach exactly as for an absent one.
+ *
+ * The fixture is a private Brain preference with no `status` field (a
+ * record the doctor and the schema report both flag by path) whose `id`
+ * was hand-edited after the first index (one tier-drift row). Vault A
+ * keeps it, vault B never had it; the same remote call must answer
+ * identically in both once the vault paths are normalised, and a local
+ * call must still name it, so an identical pair is not an empty surface.
+ */
+
+import { afterEach, describe, expect, test } from "bun:test";
+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { bootstrapBrain } from "../../src/core/brain/init.ts";
+import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
+import { TRANSPORT_REACH } from "../../src/core/graph/transport-reach.ts";
+import { resolveSearchConfig } from "../../src/core/search/index.ts";
+import { indexVault } from "../../src/core/search/indexer.ts";
+import { MCPServer } from "../../src/mcp/server.ts";
+
+const PREF = "Brain/preferences/pref-secret.md";
+
+/** A private preference with no `status`, whose identity field is `id`. */
+function prefBody(id: string, body: string): string {
+ return (
+ `---\nkind: brain-preference\nid: ${id}\ncreated_at: 2026-05-01T00:00:00Z\n` +
+ `topic: style\nvisibility: private\n---\n\n${body}`
+ );
+}
+
+const bases: string[] = [];
+const savedConfig = process.env["OPEN_SECOND_BRAIN_CONFIG"];
+
+afterEach(() => {
+ for (const base of bases.splice(0)) rmSync(base, { recursive: true, force: true });
+ if (savedConfig === undefined) delete process.env["OPEN_SECOND_BRAIN_CONFIG"];
+ else process.env["OPEN_SECOND_BRAIN_CONFIG"] = savedConfig;
+});
+
+interface Fixture {
+ readonly base: string;
+ readonly configPath: string;
+ readonly vault: string;
+}
+
+async function fixture(withSecret: boolean): Promise<Fixture> {
+ const base = mkdtempSync(join(tmpdir(), "o2b-doctor-schema-reach-"));
+ bases.push(base);
+ const vault = join(base, "vault");
+ mkdirSync(vault, { recursive: true });
+ const configPath = join(base, "config.yaml");
+ atomicWriteFileSync(configPath, `vault: ${vault}\nagent_name: claude\n`);
+ bootstrapBrain(vault, { configPath });
+ mkdirSync(join(vault, "Brain/preferences"), { recursive: true });
+ const config = resolveSearchConfig({ vault, configPath });
+ if (withSecret) writeFileSync(join(vault, PREF), prefBody("pref-secret", "Use spaces.\n"));
+ await indexVault(config);
+ if (withSecret) {
+ // A size change, so the indexer's mtime+size fast path re-reads it.
+ writeFileSync(join(vault, PREF), prefBody("pref-renamed", "Use considerably more spaces.\n"));
+ }
+ await indexVault(config);
+ return { base, configPath, vault };
+}
+
+async function answer(
+ f: Fixture,
+ tool: string,
+ args: Record<string, unknown>,
+ reach?: typeof TRANSPORT_REACH.local,
+): Promise<string> {
+ process.env["OPEN_SECOND_BRAIN_CONFIG"] = f.configPath;
+ const server = new MCPServer(
+ { vault: f.vault, configPath: f.configPath },
+ reach !== undefined ? { reach } : undefined,
+ );
+ const result = await server.callTool(tool, args);
+ return JSON.stringify(result).split(f.vault).join("<V>").split(f.base).join("<B>");
+}
+
+describe("a malformed private preference answers as an absent one at remote reach", () => {
+ for (const [tool, args] of [
+ ["brain_doctor", {}],
+ ["schema_inspect", { view: "lint" }],
+ ["schema_inspect", { view: "orphans" }],
+ ["schema_inspect", { view: "stats" }],
+ ] as const) {
+ test(`${tool} ${JSON.stringify(args)}`, async () => {
+ const withheld = await answer(await fixture(true), tool, args);
+ const absent = await answer(await fixture(false), tool, args);
+ expect(withheld).toBe(absent);
+ expect(withheld).not.toContain("pref-secret");
+ });
+ }
+
+ test("a local caller is still told about the record and the drift", async () => {
+ const a = await fixture(true);
+ const doctor = await answer(a, "brain_doctor", {}, TRANSPORT_REACH.local);
+ expect(doctor).toContain(PREF);
+ expect(doctor).toContain("tier-drift");
+ expect(await answer(a, "schema_inspect", { view: "orphans" }, TRANSPORT_REACH.local)).toContain(
+ PREF,
+ );
+ });
+});
diff --git a/tests/mcp/event-trace-tool.test.ts b/tests/mcp/event-trace-tool.test.ts
index 06a4550a..5e64c572 100644
--- a/tests/mcp/event-trace-tool.test.ts
+++ b/tests/mcp/event-trace-tool.test.ts
@@ -7,6 +7,7 @@ import { appendLogEvent } from "../../src/core/brain/log.ts";
import { appendContinuityRecord } from "../../src/core/brain/continuity/store.ts";
import { BRAIN_LOG_EVENT_KIND } from "../../src/core/brain/types.ts";
import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
+import { TRANSPORT_REACH } from "../../src/core/graph/transport-reach.ts";
import { JSONRPC_VERSION, MCPServer, PROTOCOL_VERSION } from "../../src/mcp/index.ts";
import { buildToolTable } from "../../src/mcp/tools.ts";
@@ -138,4 +139,22 @@ describe("brain_event_trace tool", () => {
})) as { error?: { code: number } };
expect(response.error?.code).toBe(-32603);
});
+
+ test("an event naming a page outside the vault is withheld at remote reach", async () => {
+ writeFileSync(join(tmp, "outside.md"), "---\ntitle: beside the vault\n---\nbody\n");
+ appendLogEvent(vault, {
+ timestamp: "2026-06-15T10:00:00Z",
+ eventType: BRAIN_LOG_EVENT_KIND.note,
+ agent: "tester",
+ body: { text: "../outside.md" },
+ });
+
+ const remote = new MCPServer({ vault, configPath });
+ await initialize(remote);
+ const local = new MCPServer({ vault, configPath }, { reach: TRANSPORT_REACH.local });
+ await initialize(local);
+
+ expect((await callEventTrace(remote, { date: "2026-06-15" }))["total"]).toBe(0);
+ expect((await callEventTrace(local, { date: "2026-06-15" }))["total"]).toBe(1);
+ });
});
diff --git a/tests/mcp/hygiene-capture-scope-reach.test.ts b/tests/mcp/hygiene-capture-scope-reach.test.ts
new file mode 100644
index 00000000..8aa6572f
--- /dev/null
+++ b/tests/mcp/hygiene-capture-scope-reach.test.ts
@@ -0,0 +1,108 @@
+/**
+ * `brain_hygiene` scan, `capture-scope` detector, at the caller's reach.
+ *
+ * The detector re-derives each cited source's scope from the filesystem.
+ * A cited vault page the caller may not read at its reach must count as
+ * url-only, so the finding a caller sees for a page citing it is the same
+ * finding it would see for a page citing a path that does not exist.
+ */
+
+import { afterEach, beforeEach, describe, expect, test } from "bun:test";
+import { chmodSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
+import { join } from "node:path";
+import { tmpdir } from "node:os";
+
+import { bootstrapBrain } from "../../src/core/brain/init.ts";
+import { CAPTURE_SCOPE_DETECTOR_ID } from "../../src/core/brain/hygiene/detectors/capture-scope.ts";
+import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
+import { TRANSPORT_REACH, type TransportReach } from "../../src/core/graph/transport-reach.ts";
+import { HEALTH_TOOLS } from "../../src/mcp/brain/health-tools.ts";
+import { HYGIENE_TOOLS } from "../../src/mcp/brain/hygiene-tools.ts";
+import { RESEARCH_TOOLS } from "../../src/mcp/brain/research-tools.ts";
+import type { ServerContext } from "../../src/mcp/tool-contract.ts";
+import { CHMOD_CANNOT_DENY } from "../helpers/platform.ts";
+
+const WITHHELD_SOURCE = "Notes/withheld.md";
+const ABSENT_SOURCE = "Notes/absent.md";
+
+let vault: string;
+let configHome: string;
+let ctx: ServerContext;
+
+beforeEach(() => {
+ vault = mkdtempSync(join(tmpdir(), "o2b-hygiene-reach-vault-"));
+ configHome = mkdtempSync(join(tmpdir(), "o2b-hygiene-reach-cfg-"));
+ const configPath = join(configHome, "config.yaml");
+ atomicWriteFileSync(configPath, `vault: ${vault}\nagent_name: claude\n`);
+ bootstrapBrain(vault, { configPath });
+ ctx = { vault, configPath, repoRoot: null };
+ mkdirSync(join(vault, "Notes"), { recursive: true });
+ writeFileSync(join(vault, WITHHELD_SOURCE), "---\nvisibility: private\n---\nbody\n");
+});
+
+afterEach(() => {
+ rmSync(vault, { recursive: true, force: true });
+ rmSync(configHome, { recursive: true, force: true });
+});
+
+const research = RESEARCH_TOOLS[0]!.handler;
+const hygiene = HYGIENE_TOOLS.find((tool) => tool.name === "brain_hygiene")!.handler;
+const status = HEALTH_TOOLS.find((tool) => tool.name === "brain_status")!.handler;
+
+/** The status snapshot's hygiene problem line at this reach, if any. */
+async function hygieneLineAt(reach: TransportReach): Promise<string | undefined> {
+ const res = (await status({ ...ctx, reach }, {})) as {
+ problems: ReadonlyArray<{ code: string; detail: string }>;
+ };
+ return res.problems.find((problem) => problem.code === "hygiene-findings")?.detail;
+}
+
+/** Write a report citing only `source`, and return its path. */
+async function reportCiting(title: string, source: string): Promise<string> {
+ const res = (await research(ctx, {
+ title,
+ sources: [source],
+ findings: [{ statement: "A point", sources: [source] }],
+ })) as Record<string, unknown>;
+ return res["report_path"] as string;
+}
+
+/** The report paths the capture-scope detector names at this reach. */
+async function flaggedAt(reach: TransportReach): Promise<ReadonlyArray<string>> {
+ const res = (await hygiene(
+ { ...ctx, reach },
+ { mode: "scan", detectors: [CAPTURE_SCOPE_DETECTOR_ID] },
+ )) as { findings: ReadonlyArray<{ targets: ReadonlyArray<string> }> };
+ return res.findings.flatMap((finding) => finding.targets).toSorted();
+}
+
+describe("brain_hygiene capture-scope at the caller's reach", () => {
+ test("a cited page withheld at the caller's reach counts as url-only, like an absent one", async () => {
+ const withheld = await reportCiting("Withheld", WITHHELD_SOURCE);
+ const absent = await reportCiting("Absent", ABSENT_SOURCE);
+
+ expect(await flaggedAt(TRANSPORT_REACH.remote)).toEqual([absent, withheld].toSorted());
+ expect(await flaggedAt(TRANSPORT_REACH.local)).toEqual([absent]);
+ });
+
+ test("the status snapshot counts a withheld citation as an absent one", async () => {
+ await reportCiting("Withheld", WITHHELD_SOURCE);
+ const withheldRemote = await hygieneLineAt(TRANSPORT_REACH.remote);
+ rmSync(join(vault, WITHHELD_SOURCE));
+ expect(await hygieneLineAt(TRANSPORT_REACH.remote)).toBe(withheldRemote);
+ });
+
+ test.skipIf(CHMOD_CANNOT_DENY)(
+ "a cited page whose stat is refused counts as url-only for a caller who may not read it",
+ async () => {
+ const withheld = await reportCiting("Withheld", WITHHELD_SOURCE);
+ chmodSync(join(vault, "Notes"), 0o000);
+ try {
+ expect(await flaggedAt(TRANSPORT_REACH.remote)).toEqual([withheld]);
+ expect(await flaggedAt(TRANSPORT_REACH.local)).toEqual([]);
+ } finally {
+ chmodSync(join(vault, "Notes"), 0o755);
+ }
+ },
+ );
+});
diff --git a/tests/mcp/hygiene-link-integrity.test.ts b/tests/mcp/hygiene-link-integrity.test.ts
index 2b9966ee..6618f6b0 100644
--- a/tests/mcp/hygiene-link-integrity.test.ts
+++ b/tests/mcp/hygiene-link-integrity.test.ts
@@ -19,7 +19,8 @@ import { HYGIENE_DETECTOR_IDS } from "../../src/core/brain/hygiene/types.ts";
import { indexVault } from "../../src/core/search/indexer.ts";
import { resolveSearchConfig } from "../../src/core/search/index.ts";
import { DANGLING_LINK_DEFINITION } from "../../src/core/search/link-ratchet.ts";
-import { HYGIENE_TOOLS } from "../../src/mcp/brain/hygiene-tools.ts";
+import { TRANSPORT_REACH, type TransportReach } from "../../src/core/graph/transport-reach.ts";
+import { HYGIENE_TOOLS, LINK_INTEGRITY_REACH_REASON } from "../../src/mcp/brain/hygiene-tools.ts";
import { JSONRPC_VERSION, MCPServer, PROTOCOL_VERSION } from "../../src/mcp/index.ts";
const SCAN_KEYS_BEFORE = [
@@ -42,8 +43,9 @@ afterEach(() => {
rmSync(vault, { recursive: true, force: true });
});
-async function scan(): Promise<any> {
- const server = new MCPServer({ vault });
+/** A scan through a server minted at `reach`, or with no reach minted when `null`. */
+async function scan(reach: TransportReach | null = TRANSPORT_REACH.local): Promise<any> {
+ const server = new MCPServer({ vault }, reach !== null ? { reach } : {});
await server.handleRequest({
jsonrpc: JSONRPC_VERSION,
id: 1,
@@ -72,6 +74,7 @@ test("the detector tuple and its input-schema enum stay in lockstep", () => {
"usefulness",
"slug-collisions",
"tags",
+ "capture-scope",
]);
const schema = HYGIENE_TOOLS[0]!.inputSchema as any;
expect(schema.properties.detectors.items.enum).toEqual([...HYGIENE_DETECTOR_IDS]);
@@ -103,3 +106,20 @@ test("after a full index run the dangling count is reported with its definition"
expect(s.link_integrity.dangling).toBeGreaterThanOrEqual(1);
expect(typeof s.link_integrity.links).toBe("number");
});
+
+test("at remote reach the count is not measured, so it cannot tell a withheld page from an absent one", async () => {
+ writeFileSync(join(vault, "Brain", "note-a.md"), "# A\n\n[[Brain/secret.md]] and [[gone]]\n");
+ writeFileSync(join(vault, "Brain", "secret.md"), "---\nvisibility: private\n---\n# S\n");
+ await indexVault(resolveSearchConfig({ vault }), { force: true });
+ const withheld = (await scan(null)).link_integrity;
+ expect((await scan()).link_integrity.measured).toBe(true);
+
+ rmSync(join(vault, "Brain", "secret.md"));
+ await indexVault(resolveSearchConfig({ vault }), { force: true });
+ const absent = (await scan(null)).link_integrity;
+
+ expect(withheld).toEqual(absent);
+ expect(withheld.measured).toBe(false);
+ expect(withheld.reason).toBe(LINK_INTEGRITY_REACH_REASON);
+ expect(withheld.dangling).toBeUndefined();
+});
diff --git a/tests/mcp/idea-discovery-reach.test.ts b/tests/mcp/idea-discovery-reach.test.ts
new file mode 100644
index 00000000..93b6a5b7
--- /dev/null
+++ b/tests/mcp/idea-discovery-reach.test.ts
@@ -0,0 +1,78 @@
+/**
+ * `brain_idea_discovery` ranks a vault the caller may read at its reach
+ * as a vault without the pages it may not read.
+ *
+ * An orphan-research idea names a Brain page's path, its basename and its
+ * age. A page that reserved itself is neither a candidate nor a source of
+ * inbound links (its links would otherwise decide which visible pages
+ * count as orphans), and it is left out before the ranking and the cap.
+ * At local reach the page is ranked as before.
+ */
+
+import { afterEach, describe, expect, test } from "bun:test";
+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { bootstrapBrain } from "../../src/core/brain/init.ts";
+import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
+import { TRANSPORT_REACH } from "../../src/core/graph/transport-reach.ts";
+import { MCPServer } from "../../src/mcp/server.ts";
+
+const PRIVATE_PATH = "Brain/research/secret-plan.md";
+const SECRET_BODY = "---\nvisibility: private\n---\n# Secret plan\nBuilds on [[open-study]].\n";
+
+const bases: string[] = [];
+
+afterEach(() => {
+ for (const base of bases.splice(0)) rmSync(base, { recursive: true, force: true });
+});
+
+interface Fixture {
+ readonly vault: string;
+ readonly server: MCPServer;
+}
+
+/**
+ * One vault holding the private research page, or one where it never
+ * existed. With no `reach`, the server mints none, which fails closed to
+ * remote.
+ */
+function fixture(withSecret: boolean, reach?: typeof TRANSPORT_REACH.local): Fixture {
+ const base = mkdtempSync(join(tmpdir(), "o2b-ideas-reach-"));
+ bases.push(base);
+ const vault = join(base, "vault");
+ mkdirSync(vault, { recursive: true });
+ const configPath = join(base, "config.yaml");
+ atomicWriteFileSync(configPath, `vault: ${vault}\nagent_name: claude\n`);
+ bootstrapBrain(vault, { configPath });
+ mkdirSync(join(vault, "Brain/research"), { recursive: true });
+ writeFileSync(join(vault, "Brain/research/open-study.md"), "# Open study\n");
+ if (withSecret) writeFileSync(join(vault, PRIVATE_PATH), SECRET_BODY);
+ const server = new MCPServer({ vault, configPath }, reach !== undefined ? { reach } : undefined);
+ return { vault, server };
+}
+
+/** The ranked ideas, with the age phrases (wall-clock dependent) normalised. */
+async function ideas(f: Fixture): Promise<string> {
+ const out = await f.server.callTool("brain_idea_discovery", { cap: 50 });
+ return JSON.stringify(out).replace(/\d+d old/g, "<age>");
+}
+
+describe("a page the caller cannot read is ranked as an absent one", () => {
+ test("neither a candidate nor an inbound link", async () => {
+ const hidden = fixture(true);
+ const absent = fixture(false);
+ const answer = await ideas(hidden);
+ expect(answer).toBe(await ideas(absent));
+ expect(answer).toContain("Brain/research/open-study.md");
+ });
+});
+
+describe("at local reach the page is ranked", () => {
+ test("it is a candidate and its link counts", async () => {
+ const answer = await ideas(fixture(true, TRANSPORT_REACH.local));
+ expect(answer).toContain(PRIVATE_PATH);
+ expect(answer).not.toContain("Brain/research/open-study.md");
+ });
+});
diff --git a/tests/mcp/ingest-tool.test.ts b/tests/mcp/ingest-tool.test.ts
index 7a2c8a37..004b35bb 100644
--- a/tests/mcp/ingest-tool.test.ts
+++ b/tests/mcp/ingest-tool.test.ts
@@ -12,11 +12,19 @@ import { tmpdir } from "node:os";
import { mkdirSync, writeFileSync } from "node:fs";
import { bootstrapBrain } from "../../src/core/brain/init.ts";
+import { CAPTURE_SCOPE } from "../../src/core/brain/provenance/capture-scope.ts";
import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
import { listEntities } from "../../src/core/brain/entities/registry.ts";
+import { readManifest } from "../../src/core/brain/ingest/content-manifest.ts";
+import {
+ SOURCE_CONTENT_HASH_FRONTMATTER_KEY,
+ UNTRUSTED_SOURCE_FRONTMATTER_KEY,
+} from "../../src/core/brain/trust/untrusted-provenance.ts";
+import { TRANSPORT_REACH } from "../../src/core/graph/transport-reach.ts";
import { INGEST_TOOLS } from "../../src/mcp/brain/ingest-tools.ts";
import { INVALID_PARAMS, MCPError } from "../../src/mcp/protocol.ts";
import type { ServerContext } from "../../src/mcp/tool-contract.ts";
+import { brainPageTexts } from "../helpers/brain-pages.ts";
let vault: string;
let configHome: string;
@@ -236,3 +244,131 @@ describe("brain_ingest_batch_plan reconcile (P5, t_d067a153)", () => {
expect(res["reconcile"]).toBeUndefined();
});
});
+
+/**
+ * The capture scope reaches the MCP caller (distilled provenance, D3): a
+ * source the vault holds is `full-local`, a URL is `url-only`, and the key
+ * is always present so a caller never infers it from absence.
+ */
+describe("brain_ingest_source - capture_scope", () => {
+ test("a source the vault holds is full-local", async () => {
+ seed("Articles/held.md");
+ const res = (await handler(ctx, {
+ source_path: "Articles/held.md",
+ summary: "A held source.",
+ entities: [{ category: "concept", name: "Held" }],
+ })) as Record<string, unknown>;
+ expect(res["capture_scope"]).toBe(CAPTURE_SCOPE.fullLocal);
+ });
+
+ test("a url source is url-only", async () => {
+ const res = (await handler(ctx, {
+ source_path: "https://example.test/post",
+ summary: "A remote source.",
+ entities: [{ category: "concept", name: "Remote" }],
+ })) as Record<string, unknown>;
+ expect(res["capture_scope"]).toBe(CAPTURE_SCOPE.urlOnly);
+ });
+});
+
+/**
+ * A page the caller cannot read at its reach is ingested exactly like an
+ * absent one: untrusted lane, `url-only`, and no digest on any page.
+ */
+describe("brain_ingest_source - a page withheld at the caller's reach", () => {
+ const PRIVATE_PATH = "Notes/secret.md";
+
+ beforeEach(() => {
+ mkdirSync(join(vault, "Notes"), { recursive: true });
+ writeFileSync(
+ join(vault, PRIVATE_PATH),
+ "---\nvisibility: private\n---\nThe code is ZX8.\n",
+ "utf8",
+ );
+ });
+
+ const ingest = (source: string, reachCtx: ServerContext = ctx) =>
+ handler(reachCtx, {
+ source_path: source,
+ summary: "Codes.",
+ entities: [{ category: "concept", name: "Codes" }],
+ }) as Promise<{ capture_scope: string; summary_path: string }>;
+
+ test("answers like an absent source and writes no digest", async () => {
+ const hidden = await ingest(PRIVATE_PATH);
+ const absent = await ingest("Notes/absent.md");
+ expect(hidden.capture_scope).toBe(CAPTURE_SCOPE.urlOnly);
+ expect(hidden.capture_scope).toBe(absent.capture_scope);
+ const summary = readFileSync(join(vault, hidden.summary_path), "utf8");
+ expect(summary).toContain(UNTRUSTED_SOURCE_FRONTMATTER_KEY);
+ for (const page of brainPageTexts(vault))
+ expect(page).not.toContain(SOURCE_CONTENT_HASH_FRONTMATTER_KEY);
+ });
+
+ test("at local reach the same page is full-local", async () => {
+ const res = await ingest(PRIVATE_PATH, { ...ctx, reach: TRANSPORT_REACH.local });
+ expect(res.capture_scope).toBe(CAPTURE_SCOPE.fullLocal);
+ });
+
+ test("records no content manifest entry for it, as for an absent source", async () => {
+ await ingest(PRIVATE_PATH);
+ expect(Object.keys(readManifest(vault).entries)).not.toContain(PRIVATE_PATH);
+ await ingest(PRIVATE_PATH, { ...ctx, reach: TRANSPORT_REACH.local });
+ expect(Object.keys(readManifest(vault).entries)).toContain(PRIVATE_PATH);
+ });
+});
+
+/**
+ * A file the caller cannot read at its reach is planned exactly like an
+ * absent one: in no batch, no total, no skip list and no reconcile list.
+ */
+describe("brain_ingest_batch_plan - a page withheld at the caller's reach", () => {
+ const batchPlan = INGEST_TOOLS.find((t) => t.name === "brain_ingest_batch_plan")!.handler;
+ const OPEN = "Notes/open.md";
+ const PRIVATE_PATH = "Notes/secret.md";
+
+ beforeEach(() => {
+ seed(OPEN, "open\n");
+ seed(PRIVATE_PATH, "---\nvisibility: private\n---\nThe code is ZX8.\n");
+ });
+
+ type Plan = {
+ plan_id: string;
+ total_files: number;
+ total_bytes: number;
+ batches: Array<{ files: Array<{ path: string }> }>;
+ reconcile?: { dispatched: string[]; ingested: string[]; missing: string[] };
+ };
+ const plan = (reachCtx: ServerContext, args: Record<string, unknown> = {}) =>
+ batchPlan(reachCtx, { source_dir: "Notes", ...args }) as Promise<Plan>;
+ const planned = (p: Plan) => p.batches.flatMap((b) => b.files.map((f) => f.path));
+
+ test("plans the vault as if the page were absent", async () => {
+ const remote = await plan(ctx);
+ expect(planned(remote)).toEqual([OPEN]);
+ expect(remote.total_files).toBe(1);
+
+ rmSync(join(vault, PRIVATE_PATH));
+ const absent = await plan(ctx);
+ expect(remote).toEqual(absent);
+ });
+
+ test("at local reach the page is planned", async () => {
+ const local = await plan({ ...ctx, reach: TRANSPORT_REACH.local });
+ expect(planned(local)).toEqual([OPEN, PRIVATE_PATH]);
+ });
+
+ test("a checkpoint entry for the page stays out of the reconcile lists", async () => {
+ const localCtx = { ...ctx, reach: TRANSPORT_REACH.local };
+ const local = await plan(localCtx);
+ await handler(localCtx, {
+ source_path: PRIVATE_PATH,
+ summary: "Codes.",
+ entities: [{ category: "concept", name: "Codes" }],
+ plan_id: local.plan_id,
+ });
+ const remote = await plan(ctx, { reconcile: true });
+ expect(remote.reconcile?.dispatched).toEqual([OPEN]);
+ expect(remote.reconcile?.ingested).toEqual([]);
+ });
+});
diff --git a/tests/mcp/ner-tool.test.ts b/tests/mcp/ner-tool.test.ts
index 9c477df3..54c24dac 100644
--- a/tests/mcp/ner-tool.test.ts
+++ b/tests/mcp/ner-tool.test.ts
@@ -16,9 +16,15 @@ import { join } from "node:path";
import { bootstrapBrain } from "../../src/core/brain/init.ts";
import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
import { getEntity, listEntities } from "../../src/core/brain/entities/registry.ts";
+import {
+ INTAKE_TRUST,
+ SOURCE_CONTENT_HASH_FRONTMATTER_KEY,
+} from "../../src/core/brain/trust/untrusted-provenance.ts";
+import { TRANSPORT_REACH } from "../../src/core/graph/transport-reach.ts";
import { NER_TOOLS } from "../../src/mcp/brain/ner-tools.ts";
import { MCPError } from "../../src/mcp/protocol.ts";
import type { ServerContext } from "../../src/mcp/tool-contract.ts";
+import { brainPageTexts } from "../helpers/brain-pages.ts";
import { CHMOD_CANNOT_DENY } from "../helpers/platform.ts";
let vault: string;
@@ -129,10 +135,15 @@ describe("brain_intake_entities", () => {
chmodSync(locked, 0o000);
let thrown: unknown;
try {
- await handler(ctx, {
- source: `[[${LOCKED_DIR}/note.md]]`,
- entities: [{ category: "concept", name: "Restaking" }],
- });
+ // At local reach: a caller that may not read the page is answered as
+ // for an absent one before the stat is made (source-reach-first.test.ts).
+ await handler(
+ { ...ctx, reach: TRANSPORT_REACH.local },
+ {
+ source: `[[${LOCKED_DIR}/note.md]]`,
+ entities: [{ category: "concept", name: "Restaking" }],
+ },
+ );
} catch (err) {
thrown = err;
}
@@ -162,3 +173,43 @@ describe("brain_intake_entities", () => {
expect(listEntities(vault)).toHaveLength(0);
});
});
+
+/**
+ * A page the caller cannot read at its reach is cited exactly like an absent
+ * one: the intake commits in the untrusted lane and records no digest of it.
+ */
+describe("brain_intake_entities - a page withheld at the caller's reach", () => {
+ const PRIVATE_PATH = "Notes/secret.md";
+ const ABSENT = "Notes/absent.md";
+
+ beforeEach(() => {
+ mkdirSync(join(vault, "Notes"), { recursive: true });
+ writeFileSync(
+ join(vault, PRIVATE_PATH),
+ "---\nvisibility: private\n---\nThe code is ZX8.\n",
+ "utf8",
+ );
+ });
+
+ const intake = (source: string, reachCtx: ServerContext = ctx) =>
+ handler(reachCtx, { source, entities: [{ category: "concept", name: "Codes" }] }) as Promise<{
+ trust: string;
+ }>;
+
+ test("answers like an absent source and writes no digest", async () => {
+ const hidden = await intake(PRIVATE_PATH);
+ const absent = await intake(ABSENT);
+ expect(hidden.trust).toBe(INTAKE_TRUST.untrusted);
+ expect(hidden.trust).toBe(absent.trust);
+ for (const page of brainPageTexts(vault))
+ expect(page).not.toContain(SOURCE_CONTENT_HASH_FRONTMATTER_KEY);
+ });
+
+ test("at local reach the same page is trusted and digested", async () => {
+ const res = await intake(PRIVATE_PATH, { ...ctx, reach: TRANSPORT_REACH.local });
+ expect(res.trust).toBe(INTAKE_TRUST.trusted);
+ expect(
+ brainPageTexts(vault).some((page) => page.includes(SOURCE_CONTENT_HASH_FRONTMATTER_KEY)),
+ ).toBe(true);
+ });
+});
diff --git a/tests/mcp/note-history-tool.test.ts b/tests/mcp/note-history-tool.test.ts
index eaab59a8..95a938b4 100644
--- a/tests/mcp/note-history-tool.test.ts
+++ b/tests/mcp/note-history-tool.test.ts
@@ -2,8 +2,9 @@ import { afterEach, beforeEach, describe, expect, test } from "bun:test";
import { execFileSync } from "node:child_process";
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
-import { join } from "node:path";
+import { basename, join } from "node:path";
+import { TRANSPORT_REACH } from "../../src/core/graph/transport-reach.ts";
import { JSONRPC_VERSION, MCPServer, PROTOCOL_VERSION } from "../../src/mcp/index.ts";
import { buildToolTable } from "../../src/mcp/tools.ts";
@@ -58,8 +59,11 @@ async function initialize(server: MCPServer): Promise<void> {
await server.handleRequest({ jsonrpc: JSONRPC_VERSION, method: "notifications/initialized" });
}
-async function call(args: Record<string, unknown>): Promise<{ result?: unknown; error?: unknown }> {
- const server = new MCPServer({ vault, configPath: null });
+async function call(
+ args: Record<string, unknown>,
+ reach?: typeof TRANSPORT_REACH.local,
+): Promise<{ result?: unknown; error?: unknown }> {
+ const server = new MCPServer({ vault, configPath: null }, reach !== undefined ? { reach } : {});
await initialize(server);
return (await server.handleRequest({
jsonrpc: JSONRPC_VERSION,
@@ -93,4 +97,38 @@ describe("brain_note_history tool", () => {
const response = await call({});
expect((response.error as { code: number }).code).toBe(-32602);
});
+
+ test("a page the caller may not read at its reach answers as a path no commit touches", async () => {
+ const privateBody = "---\nvisibility: private\n---\nv1\n";
+ commit("notes/secret.md", privateBody, "secret start", "2026-05-01T10:00:00Z");
+ commit("notes/secret.md", `${privateBody}v2\n`, "secret later", "2026-05-20T10:00:00Z");
+ commit("notes/open.md", "open", "open", "2026-05-21T10:00:00Z");
+
+ const withheld = payload(await call({ path: "notes/secret.md" }));
+ const local = payload(await call({ path: "notes/secret.md" }, TRANSPORT_REACH.local));
+ rmSync(join(vault, "notes/secret.md"));
+ const absent = payload(await call({ path: "notes/secret.md" }));
+ const neverWritten = payload(await call({ path: "notes/never.md" }));
+
+ expect(withheld).toEqual(absent);
+ expect(withheld).toEqual({ ...neverWritten, note_path: "notes/secret.md" });
+ expect(withheld["commit_count"]).toBe(0);
+ expect(JSON.stringify(withheld)).not.toContain("secret start");
+ expect(local["commit_count"]).toBe(2);
+ });
+
+ test("a path that leaves the vault answers as a path no commit touches at remote reach", async () => {
+ commit("notes/open.md", "open", "open", "2026-05-21T10:00:00Z");
+ const outside = mkdtempSync(join(tmpdir(), "o2b-note-history-outside-"));
+ try {
+ writeFileSync(join(outside, "x.md"), "---\ntitle: beside\n---\nbody\n");
+ const rel = `../${basename(outside)}/x.md`;
+ const escaped = payload(await call({ path: rel }));
+ const neverWritten = payload(await call({ path: "notes/never.md" }));
+ expect(escaped).toEqual({ ...neverWritten, note_path: escaped["note_path"] });
+ expect(escaped["commit_count"]).toBe(0);
+ } finally {
+ rmSync(outside, { recursive: true, force: true });
+ }
+ });
});
diff --git a/tests/mcp/note-lifecycle-references-reach.test.ts b/tests/mcp/note-lifecycle-references-reach.test.ts
new file mode 100644
index 00000000..f12d7111
--- /dev/null
+++ b/tests/mcp/note-lifecycle-references-reach.test.ts
@@ -0,0 +1,136 @@
+/**
+ * The reference report of `brain_note_lifecycle` counts only the pages the
+ * caller may read at its reach.
+ *
+ * A plan names every file that links to the note (`inbound_files`) and
+ * counts the files it scanned and rewrote. A page the caller may not read
+ * is left out of every one of those lists and counts, so the report is
+ * the one a vault without that page gives. The rewrite itself still edits
+ * the links inside that page when the plan is applied: leaving them
+ * pointing at the old path would break the vault for the readers who can
+ * see the page, and nothing about the edit is reported to this caller.
+ */
+
+import { afterEach, describe, expect, test } from "bun:test";
+import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { basename, dirname, join } from "node:path";
+
+import { bootstrapBrain } from "../../src/core/brain/init.ts";
+import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
+import { TRANSPORT_REACH } from "../../src/core/graph/transport-reach.ts";
+import { MCPServer } from "../../src/mcp/server.ts";
+
+const TARGET = "Notes/target.md";
+const PRIVATE_PATH = "Notes/secret.md";
+const SECRET_BODY = "---\nvisibility: private\n---\n# Secret\nBuilds on [[Notes/target]].\n";
+
+const bases: string[] = [];
+
+afterEach(() => {
+ for (const base of bases.splice(0)) rmSync(base, { recursive: true, force: true });
+});
+
+interface Fixture {
+ readonly vault: string;
+ readonly server: MCPServer;
+}
+
+/**
+ * A visible note linked from a visible page and, in one of the two vaults,
+ * from a private page. With no `reach`, the server mints none: remote.
+ */
+function fixture(withSecret: boolean, reach?: typeof TRANSPORT_REACH.local): Fixture {
+ const base = mkdtempSync(join(tmpdir(), "o2b-lifecycle-refs-reach-"));
+ bases.push(base);
+ const vault = join(base, "vault");
+ mkdirSync(join(vault, "Notes"), { recursive: true });
+ const configPath = join(base, "config.yaml");
+ atomicWriteFileSync(configPath, `vault: ${vault}\nagent_name: claude\n`);
+ bootstrapBrain(vault, { configPath });
+ writeFileSync(join(vault, TARGET), "# Target\n");
+ writeFileSync(join(vault, "Notes/open.md"), "# Open\nSee [[Notes/target]].\n");
+ if (withSecret) writeFileSync(join(vault, PRIVATE_PATH), SECRET_BODY);
+ const server = new MCPServer({ vault, configPath }, reach !== undefined ? { reach } : undefined);
+ return { vault, server };
+}
+
+/** The answer with the per-vault parts (paths, run ids, timestamps) normalised. */
+async function answer(f: Fixture, args: Record<string, unknown>): Promise<string> {
+ let raw: string;
+ try {
+ raw = `ok: ${JSON.stringify(await f.server.callTool("brain_note_lifecycle", args))}`;
+ } catch (err) {
+ const e = err as Error & { code?: unknown; data?: unknown };
+ raw = `error: ${e.message} ${JSON.stringify(e.code)} ${JSON.stringify(e.data)}`;
+ }
+ // The temp folder name carries no separators, so it also matches the
+ // JSON-escaped form of a Windows path, where `f.vault` itself does not.
+ return raw
+ .split(f.vault)
+ .join("<vault>")
+ .split(basename(dirname(f.vault)))
+ .join("<base>")
+ .replace(/\d{4}-\d{2}-\d{2}T[\d:.]+Z?/g, "<ts>")
+ .replace(/\d{4}-\d{2}-\d{2}-\d{6}/g, "<run>");
+}
+
+const CALLS: ReadonlyArray<Record<string, unknown>> = [
+ { action: "delete", path: TARGET },
+ { action: "delete", path: TARGET, delete_linked: true },
+ { action: "move", path: TARGET, to: "Other/target.md" },
+ { action: "rename", path: TARGET, to: "Notes/renamed.md" },
+ { action: "archive", path: TARGET },
+ { action: "move", path: TARGET, to: "Other/target.md", apply: true },
+ { action: "delete", path: TARGET, apply: true, confirm: true },
+];
+
+describe("a linking page the caller cannot read is left out of the report", () => {
+ for (const args of CALLS) {
+ test(JSON.stringify(args), async () => {
+ const hidden = fixture(true);
+ const absent = fixture(false);
+ expect(await answer(hidden, args)).toBe(await answer(absent, args));
+ });
+ }
+
+ test("an applied move still rewrites the link inside it", async () => {
+ const hidden = fixture(true);
+ await answer(hidden, { action: "move", path: TARGET, to: "Other/target.md", apply: true });
+ const secret = readFileSync(join(hidden.vault, PRIVATE_PATH), "utf8");
+ expect(secret).toContain("[[Other/target]]");
+ expect(secret).not.toContain("[[Notes/target]]");
+ });
+});
+
+describe("at local reach the linking page is reported", () => {
+ test("it is an inbound file", async () => {
+ const out = await answer(fixture(true, TRANSPORT_REACH.local), {
+ action: "delete",
+ path: TARGET,
+ });
+ expect(out).toContain(PRIVATE_PATH);
+ });
+});
+
+describe("a hidden page at the destination is a known one-bit limitation", () => {
+ // Pinned so a later change to it is deliberate: answering as absent
+ // would mean moving onto the page or reporting a move that did not
+ // happen, so the refusal stays and the page is left untouched.
+ const MOVED_ONTO = "Other/secret.md";
+ for (const args of [
+ { action: "rename", path: TARGET, to: PRIVATE_PATH },
+ { action: "move", path: TARGET, to: MOVED_ONTO, apply: true },
+ ]) {
+ test(JSON.stringify(args), async () => {
+ const hidden = fixture(true);
+ mkdirSync(join(hidden.vault, "Other"), { recursive: true });
+ writeFileSync(join(hidden.vault, MOVED_ONTO), SECRET_BODY);
+ const out = await answer(hidden, args);
+ expect(out).toContain("destination_occupied");
+ expect(out).not.toContain("Builds on");
+ expect(readFileSync(join(hidden.vault, String(args.to)), "utf8")).toBe(SECRET_BODY);
+ expect(readFileSync(join(hidden.vault, TARGET), "utf8")).toBe("# Target\n");
+ });
+ }
+});
diff --git a/tests/mcp/note-write-reach.test.ts b/tests/mcp/note-write-reach.test.ts
new file mode 100644
index 00000000..c97d539c
--- /dev/null
+++ b/tests/mcp/note-write-reach.test.ts
@@ -0,0 +1,119 @@
+/**
+ * The note write and lifecycle tools answer for a page the caller may not
+ * read at its reach exactly as for an absent one.
+ *
+ * `brain_note_lifecycle`, `brain_append_note`, `brain_update_note` and the
+ * update and append operations of `brain_write_batch` name an EXISTING
+ * page. A page that reserved itself raises the same refusal a missing
+ * path raises, before any plan is built or any byte is written, so a
+ * caller that cannot read the page can neither learn that it exists nor
+ * change it. `brain_create_note` keeps its occupied-target refusal: it
+ * cannot answer "created" without overwriting the page or reporting a
+ * write that did not happen, so it refuses and leaves the page alone.
+ */
+
+import { afterEach, describe, expect, test } from "bun:test";
+import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { bootstrapBrain } from "../../src/core/brain/init.ts";
+import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
+import { TRANSPORT_REACH } from "../../src/core/graph/transport-reach.ts";
+import { MCPServer } from "../../src/mcp/server.ts";
+
+const PRIVATE_PATH = "Notes/secret.md";
+const SECRET_BODY = "---\nvisibility: private\n---\n# Secret plan\nThe PIN is 4711.\n";
+
+const bases: string[] = [];
+
+afterEach(() => {
+ for (const base of bases.splice(0)) rmSync(base, { recursive: true, force: true });
+});
+
+interface Fixture {
+ readonly vault: string;
+ readonly server: MCPServer;
+}
+
+/**
+ * One vault holding the private page, or one where it never existed. With
+ * no `reach`, the server mints none, which fails closed to remote.
+ */
+function fixture(withSecret: boolean, reach?: typeof TRANSPORT_REACH.local): Fixture {
+ const base = mkdtempSync(join(tmpdir(), "o2b-note-write-reach-"));
+ bases.push(base);
+ const vault = join(base, "vault");
+ mkdirSync(join(vault, "Notes"), { recursive: true });
+ const configPath = join(base, "config.yaml");
+ atomicWriteFileSync(configPath, `vault: ${vault}\nagent_name: claude\n`);
+ bootstrapBrain(vault, { configPath });
+ writeFileSync(join(vault, "Notes/open.md"), "# Open\nSee [[secret]].\n");
+ writeFileSync(join(vault, "Notes/other.md"), "# Other\nSee [[Notes/secret]].\n");
+ if (withSecret) writeFileSync(join(vault, PRIVATE_PATH), SECRET_BODY);
+ const server = new MCPServer({ vault, configPath }, reach !== undefined ? { reach } : undefined);
+ return { vault, server };
+}
+
+/** The answer (or the refusal) with the vault path normalised. */
+async function answer(f: Fixture, tool: string, args: Record<string, unknown>): Promise<string> {
+ let raw: string;
+ try {
+ raw = `ok: ${JSON.stringify(await f.server.callTool(tool, args))}`;
+ } catch (err) {
+ const e = err as Error & { code?: unknown; data?: unknown };
+ raw = `error: ${e.message} ${JSON.stringify(e.code)} ${JSON.stringify(e.data)}`;
+ }
+ return raw.split(f.vault).join("<vault>");
+}
+
+const CALLS: ReadonlyArray<readonly [string, Record<string, unknown>]> = [
+ ["brain_note_lifecycle", { action: "delete", path: PRIVATE_PATH }],
+ ["brain_note_lifecycle", { action: "delete", path: PRIVATE_PATH, apply: true, confirm: true }],
+ ["brain_note_lifecycle", { action: "move", path: PRIVATE_PATH, to: "Other/secret.md" }],
+ ["brain_note_lifecycle", { action: "rename", path: PRIVATE_PATH, to: "Notes/renamed.md" }],
+ ["brain_note_lifecycle", { action: "archive", path: PRIVATE_PATH, apply: true }],
+ ["brain_append_note", { path: PRIVATE_PATH, content: "appended" }],
+ ["brain_update_note", { path: PRIVATE_PATH, frontmatter: { visibility: "public" } }],
+ ["brain_update_note", { path: PRIVATE_PATH, content: "replaced" }],
+ ["brain_write_batch", { operations: [{ op: "append_note", path: PRIVATE_PATH, content: "x" }] }],
+ [
+ "brain_write_batch",
+ {
+ operations: [
+ { op: "update_note", path: PRIVATE_PATH, frontmatter: { visibility: "public" } },
+ ],
+ },
+ ],
+];
+
+describe("a page the caller cannot read answers as an absent one", () => {
+ for (const [tool, args] of CALLS) {
+ test(`${tool} ${JSON.stringify(args)}`, async () => {
+ const hidden = fixture(true);
+ const absent = fixture(false);
+ expect(await answer(hidden, tool, args)).toBe(await answer(absent, tool, args));
+ expect(readFileSync(join(hidden.vault, PRIVATE_PATH), "utf8")).toBe(SECRET_BODY);
+ });
+ }
+
+ test("a create onto it is refused and leaves the page as it was", async () => {
+ const hidden = fixture(true);
+ const out = await answer(hidden, "brain_create_note", { path: PRIVATE_PATH, content: "mine" });
+ expect(out).toContain("note already exists");
+ expect(readFileSync(join(hidden.vault, PRIVATE_PATH), "utf8")).toBe(SECRET_BODY);
+ });
+});
+
+describe("at local reach the page is planned and written", () => {
+ test("lifecycle plan and append", async () => {
+ const local = fixture(true, TRANSPORT_REACH.local);
+ const plan = await local.server.callTool("brain_note_lifecycle", {
+ action: "delete",
+ path: PRIVATE_PATH,
+ });
+ expect(JSON.stringify(plan)).toContain('"from":"Notes/secret.md"');
+ await local.server.callTool("brain_append_note", { path: PRIVATE_PATH, content: "appended" });
+ expect(readFileSync(join(local.vault, PRIVATE_PATH), "utf8")).toContain("appended");
+ });
+});
diff --git a/tests/mcp/procedural-memory-reach.test.ts b/tests/mcp/procedural-memory-reach.test.ts
new file mode 100644
index 00000000..ddbe8500
--- /dev/null
+++ b/tests/mcp/procedural-memory-reach.test.ts
@@ -0,0 +1,101 @@
+/**
+ * `brain_procedural_memory` answers for a procedure page the caller may
+ * not read at its reach exactly as for an absent one.
+ *
+ * An entry names its page's vault-relative path, its title and its
+ * frontmatter lists. `list` leaves such an entry out and counts what it
+ * returns, `reconcile` reports counts over the pages the caller may read,
+ * and `mark_used` and `mark_outcome` refuse its id as unknown before
+ * anything is written. At local reach the entry is listed as before.
+ */
+
+import { afterEach, describe, expect, test } from "bun:test";
+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { bootstrapBrain } from "../../src/core/brain/init.ts";
+import { proceduralEntryId } from "../../src/core/brain/procedural-memory.ts";
+import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
+import { TRANSPORT_REACH } from "../../src/core/graph/transport-reach.ts";
+import { MCPServer } from "../../src/mcp/server.ts";
+
+const PRIVATE_PATH = "Brain/procedures/secret-deploy.md";
+const SECRET_BODY =
+ "---\nvisibility: private\ntriggers: [deploy]\n---\n# Secret deploy\nUse the PIN 4711.\n";
+const SECRET_ID = proceduralEntryId(PRIVATE_PATH);
+
+const bases: string[] = [];
+
+afterEach(() => {
+ for (const base of bases.splice(0)) rmSync(base, { recursive: true, force: true });
+});
+
+interface Fixture {
+ readonly vault: string;
+ readonly server: MCPServer;
+ readonly local: MCPServer;
+}
+
+/**
+ * One vault holding the private procedure, or one where it never existed.
+ * `server` mints no reach (remote); `local` is the same vault at local
+ * reach, used to seed the index the way an operator would.
+ */
+async function fixture(withSecret: boolean): Promise<Fixture> {
+ const base = mkdtempSync(join(tmpdir(), "o2b-pmem-reach-"));
+ bases.push(base);
+ const vault = join(base, "vault");
+ mkdirSync(join(vault, "Brain/procedures"), { recursive: true });
+ const configPath = join(base, "config.yaml");
+ atomicWriteFileSync(configPath, `vault: ${vault}\nagent_name: claude\n`);
+ bootstrapBrain(vault, { configPath });
+ writeFileSync(join(vault, "Brain/procedures/open-release.md"), "# Open release\nTag it.\n");
+ if (withSecret) writeFileSync(join(vault, PRIVATE_PATH), SECRET_BODY);
+ const local = new MCPServer({ vault, configPath }, { reach: TRANSPORT_REACH.local });
+ await local.callTool("brain_procedural_memory", { operation: "reconcile" });
+ return { vault, server: new MCPServer({ vault, configPath }), local };
+}
+
+async function answer(server: MCPServer, args: Record<string, unknown>): Promise<string> {
+ try {
+ return `ok: ${JSON.stringify(await server.callTool("brain_procedural_memory", args))}`;
+ } catch (err) {
+ const e = err as Error & { code?: unknown; data?: unknown };
+ return `error: ${e.message} ${JSON.stringify(e.code)} ${JSON.stringify(e.data)}`;
+ }
+}
+
+describe("a procedure the caller cannot read answers as an absent one", () => {
+ for (const args of [
+ { operation: "list" },
+ { operation: "list", ranked: true },
+ { operation: "reconcile" },
+ { operation: "mark_used", id: SECRET_ID },
+ { operation: "mark_outcome", id: SECRET_ID, outcome: "success" },
+ ]) {
+ test(JSON.stringify(args), async () => {
+ const hidden = await fixture(true);
+ const absent = await fixture(false);
+ expect(await answer(hidden.server, args)).toBe(await answer(absent.server, args));
+ });
+ }
+
+ test("a refused mark writes nothing", async () => {
+ const hidden = await fixture(true);
+ await answer(hidden.server, { operation: "mark_used", id: SECRET_ID });
+ const listed = await answer(hidden.local, { operation: "list" });
+ expect(listed).toContain(PRIVATE_PATH);
+ expect(listed).not.toContain('"usedCount":1');
+ });
+});
+
+describe("at local reach the procedure is listed and marked", () => {
+ test("list and mark_used", async () => {
+ const { local } = await fixture(true);
+ expect(await answer(local, { operation: "list" })).toContain(PRIVATE_PATH);
+ expect(await answer(local, { operation: "mark_used", id: SECRET_ID })).toContain(
+ '"usedCount":1',
+ );
+ });
+});
diff --git a/tests/mcp/reach-readable.test.ts b/tests/mcp/reach-readable.test.ts
new file mode 100644
index 00000000..31818db9
--- /dev/null
+++ b/tests/mcp/reach-readable.test.ts
@@ -0,0 +1,97 @@
+/**
+ * `readableAtContextReach` binds the visibility rule and the gated
+ * ownership rule to one request. It answers for vault files the caller
+ * names as a source, so a page withheld at the caller's reach must read
+ * as unreadable here exactly as it does on every read surface.
+ */
+
+import { beforeEach, describe, expect, test } from "bun:test";
+import { spawnSync } from "node:child_process";
+import { mkdirSync, writeFileSync } from "node:fs";
+import { basename, join } from "node:path";
+
+import { brainConfigPath } from "../../src/core/brain/paths.ts";
+import { TRANSPORT_REACH } from "../../src/core/graph/transport-reach.ts";
+import { readableAtContextReach } from "../../src/mcp/brain/reach-readable.ts";
+import type { ServerContext } from "../../src/mcp/tool-contract.ts";
+import { IS_WINDOWS } from "../helpers/platform.ts";
+import { pinHome, tempDirs } from "../helpers/temp-dir.ts";
+
+const mkTemp = tempDirs();
+pinHome("o2b-reach-readable-home-");
+
+let vault: string;
+
+beforeEach(() => {
+ vault = mkTemp("o2b-reach-readable-vault-");
+ mkdirSync(join(vault, "Notes"), { recursive: true });
+ mkdirSync(join(vault, "Brain"), { recursive: true });
+ writeFileSync(join(vault, "Notes", "open.md"), "---\ntitle: open\n---\nbody\n");
+ writeFileSync(join(vault, "Notes", "secret.md"), "---\nvisibility: private\n---\nbody\n");
+ writeFileSync(join(vault, "Notes", "theirs.md"), "---\nowner: other-agent\n---\nbody\n");
+});
+
+function ctx(reach: ServerContext["reach"], agentName = "me-agent"): ServerContext {
+ return { vault, configPath: null, repoRoot: null, agentName, ...(reach ? { reach } : {}) };
+}
+
+/** A public page in a fresh directory beside the vault, and its path from the vault. */
+function outside(): { dir: string; rel: string } {
+ const dir = mkTemp("o2b-reach-readable-outside-");
+ const rel = join("..", basename(dir), "sibling.md");
+ writeFileSync(join(dir, "sibling.md"), "---\ntitle: sibling\n---\nbody\n");
+ return { dir, rel };
+}
+
+function setOwnerGate(mode: string): void {
+ writeFileSync(
+ brainConfigPath(vault),
+ `schema_version: 1\nintegrity:\n owner_scope_delivery: ${mode}\n`,
+ );
+}
+
+describe("readableAtContextReach", () => {
+ test("a private page is unreadable at remote reach and readable at local reach", () => {
+ expect(readableAtContextReach(ctx(TRANSPORT_REACH.remote))("Notes/secret.md")).toBe(false);
+ expect(readableAtContextReach(ctx(TRANSPORT_REACH.local))("Notes/secret.md")).toBe(true);
+ expect(readableAtContextReach(ctx(TRANSPORT_REACH.remote))("Notes/open.md")).toBe(true);
+ });
+
+ test("a context with no minted reach answers as remote", () => {
+ expect(readableAtContextReach(ctx(undefined))("Notes/secret.md")).toBe(false);
+ });
+
+ test("another owner's page is unreadable only when the ownership gate fails closed", () => {
+ expect(readableAtContextReach(ctx(TRANSPORT_REACH.local))("Notes/theirs.md")).toBe(true);
+ setOwnerGate("warn");
+ expect(readableAtContextReach(ctx(TRANSPORT_REACH.local))("Notes/theirs.md")).toBe(true);
+ setOwnerGate("fail");
+ expect(readableAtContextReach(ctx(TRANSPORT_REACH.local))("Notes/theirs.md")).toBe(false);
+ expect(
+ readableAtContextReach(ctx(TRANSPORT_REACH.local, "other-agent"))("Notes/theirs.md"),
+ ).toBe(true);
+ });
+
+ describe("a path that leaves the vault", () => {
+ test("a public page beside the vault is not readable at remote reach", () => {
+ const { rel } = outside();
+ expect(readableAtContextReach(ctx(TRANSPORT_REACH.remote))(rel)).toBe(false);
+ });
+
+ test("an absolute path outside the vault is not readable at remote reach", () => {
+ const { dir } = outside();
+ expect(readableAtContextReach(ctx(TRANSPORT_REACH.remote))(join(dir, "sibling.md"))).toBe(
+ false,
+ );
+ });
+
+ test.skipIf(IS_WINDOWS)("a FIFO outside the vault is answered without reading it", () => {
+ const dir = mkTemp("o2b-reach-readable-fifo-");
+ const made = spawnSync("mkfifo", [join(dir, "pipe.md")]);
+ expect(made.status).toBe(0);
+ const rel = join("..", basename(dir), "pipe.md");
+ // Before the containment check this read blocked the process for good.
+ expect(readableAtContextReach(ctx(TRANSPORT_REACH.remote))(rel)).toBe(false);
+ });
+ });
+});
diff --git a/tests/mcp/reach-withheld-as-absent.test.ts b/tests/mcp/reach-withheld-as-absent.test.ts
new file mode 100644
index 00000000..bb57086c
--- /dev/null
+++ b/tests/mcp/reach-withheld-as-absent.test.ts
@@ -0,0 +1,264 @@
+/**
+ * A page the caller may not read at its reach answers exactly as an
+ * absent page would.
+ *
+ * Every case builds two vaults through the same local-reach setup: vault
+ * A keeps `Notes/secret.md` (`visibility: private`), vault B deletes it
+ * once the setup has run. The same call through a real `MCPServer` with
+ * no reach minted (so remote, fail closed) must then answer identically
+ * in both, once run ids, timestamps and the vault paths are normalised.
+ * A local-reach control shows the page is still there for a caller that
+ * may read it, so an identical pair is not an empty surface.
+ */
+
+import { afterEach, describe, expect, test } from "bun:test";
+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { bootstrapBrain } from "../../src/core/brain/init.ts";
+import { dream } from "../../src/core/brain/dream.ts";
+import { brainConfigPath } from "../../src/core/brain/paths.ts";
+import { writePreference } from "../../src/core/brain/preference.ts";
+import { BRAIN_PREFERENCE_STATUS } from "../../src/core/brain/types.ts";
+import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
+import { TRANSPORT_REACH } from "../../src/core/graph/transport-reach.ts";
+import { MCPServer } from "../../src/mcp/index.ts";
+import { readableAtContextReach } from "../../src/mcp/brain/reach-readable.ts";
+
+const SECRET_REL = "Notes/secret.md";
+const PRIVATE_BODY =
+ "---\nvisibility: private\ntitle: Secret plan\n---\n# Secret plan\nThe PIN is 4711. See [[open]].\n";
+const OPEN = "---\ntitle: Open page\n---\n# Open page\nLinks to [[secret]] and [[Notes/secret]].\n";
+
+type Call = readonly [string, Record<string, unknown>];
+
+interface Fixture {
+ readonly base: string;
+ readonly vault: string;
+ readonly configPath: string;
+}
+
+const made: string[] = [];
+const savedConfig = process.env["OPEN_SECOND_BRAIN_CONFIG"];
+
+afterEach(() => {
+ for (const dir of made.splice(0)) rmSync(dir, { recursive: true, force: true });
+ if (savedConfig === undefined) delete process.env["OPEN_SECOND_BRAIN_CONFIG"];
+ else process.env["OPEN_SECOND_BRAIN_CONFIG"] = savedConfig;
+});
+
+async function fixture(
+ setup: ReadonlyArray<Call>,
+ dropSecret: boolean,
+ prepare?: (vault: string) => void,
+): Promise<Fixture> {
+ const base = mkdtempSync(join(tmpdir(), "o2b-reach-absent-"));
+ made.push(base);
+ const vault = join(base, "vault");
+ mkdirSync(join(vault, "Notes"), { recursive: true });
+ const configPath = join(base, "config.yaml");
+ atomicWriteFileSync(configPath, `vault: ${vault}\nagent_name: claude\n`);
+ bootstrapBrain(vault, { configPath });
+ writeFileSync(join(vault, SECRET_REL), PRIVATE_BODY);
+ writeFileSync(join(vault, "Notes/open.md"), OPEN);
+ const local = new MCPServer({ vault, configPath }, { reach: TRANSPORT_REACH.local });
+ // Setup calls build on each other, so they run in order.
+ // eslint-disable-next-line no-await-in-loop
+ for (const [name, args] of setup) await local.callTool(name, args);
+ prepare?.(vault);
+ if (dropSecret) rmSync(join(vault, SECRET_REL));
+ return { base, vault, configPath };
+}
+
+function normalise(text: string, fx: Fixture): string {
+ return text
+ .split(fx.vault)
+ .join("<V>")
+ .split(fx.base)
+ .join("<B>")
+ .replace(/\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(\.\d+)?Z?([+-]\d{2}:?\d{2})?/g, "<TS>")
+ .replace(/"(\w*_ms|duration\w*|elapsed\w*|took\w*)":\s*[\d.]+/g, '"$1":<N>')
+ .replace(/run-\d+-[a-z0-9]+/g, "<RUN>")
+ .replace(/vault:\/\/[0-9a-f]{32}/g, "vault://<H>");
+}
+
+async function answer(
+ fx: Fixture,
+ call: Call,
+ reach?: (typeof TRANSPORT_REACH)[keyof typeof TRANSPORT_REACH],
+): Promise<string> {
+ process.env["OPEN_SECOND_BRAIN_CONFIG"] = fx.configPath;
+ const server = new MCPServer(
+ { vault: fx.vault, configPath: fx.configPath },
+ reach !== undefined ? { reach } : {},
+ );
+ try {
+ const res = (await server.callTool(call[0], call[1])) as Record<string, unknown>;
+ return normalise(JSON.stringify(res["structuredContent"] ?? res), fx);
+ } catch (err) {
+ return normalise(`ERR ${(err as Error).message}`, fx);
+ }
+}
+
+const CREATE_SECRET: Call = [
+ "brain_create_note",
+ { path: "Notes/private-write.md", frontmatter: { visibility: "private" }, content: "x\n" },
+];
+
+/** The remote answer with the page kept, the remote answer with it deleted, and the local answer. */
+async function pair(
+ setup: ReadonlyArray<Call>,
+ call: Call,
+ prepare?: (vault: string) => void,
+): Promise<{ withheld: string; absent: string; local: string }> {
+ const a = await fixture(setup, false, prepare);
+ const b = await fixture(setup, true, prepare);
+ return {
+ withheld: await answer(a, call),
+ absent: await answer(b, call),
+ local: await answer(a, call, TRANSPORT_REACH.local),
+ };
+}
+
+/** The remote answer with a privately written page kept, then deleted, and the local answer. */
+async function writePair(call: Call): Promise<{ withheld: string; absent: string; local: string }> {
+ const a = await fixture([CREATE_SECRET], false);
+ const b = await fixture([CREATE_SECRET], false);
+ rmSync(join(b.vault, "Notes/private-write.md"));
+ return {
+ withheld: await answer(a, call),
+ absent: await answer(b, call),
+ local: await answer(a, call, TRANSPORT_REACH.local),
+ };
+}
+
+describe("write events name a withheld page as they name an absent one", () => {
+ test("brain_event_trace", async () => {
+ const r = await writePair(["brain_event_trace", { kind: "note-write" }]);
+ expect(r.withheld).toBe(r.absent);
+ expect(r.withheld).not.toContain("private-write");
+ expect(r.local).toContain("Notes/private-write.md");
+ });
+
+ test("brain_agent_query", async () => {
+ const r = await writePair(["brain_agent_query", { kind: "note" }]);
+ expect(r.withheld).toBe(r.absent);
+ expect(r.withheld).not.toContain("private-write");
+ expect(r.local).toContain("Notes/private-write.md");
+ });
+
+ test("brain_agent_diff", async () => {
+ const r = await writePair(["brain_agent_diff", { kind: "note" }]);
+ expect(r.withheld).toBe(r.absent);
+ expect(r.withheld).not.toContain("private-write");
+ expect(r.local).toContain("Notes/private-write.md");
+ });
+});
+
+/** A vault whose dream pass fires one rollup rung, so its envelope carries link candidates. */
+function armRollup(vault: string): void {
+ atomicWriteFileSync(
+ brainConfigPath(vault),
+ "schema_version: 1\nrollup:\n fact_threshold: 1\n identity_threshold: 5\n",
+ );
+ writePreference(vault, {
+ slug: "one-fact",
+ topic: "one-fact",
+ principle: "principle for one fact",
+ created_at: "2026-01-01T00:00:00Z",
+ unconfirmed_until: "2026-01-08T00:00:00Z",
+ confirmed_at: new Date().toISOString(),
+ status: BRAIN_PREFERENCE_STATUS.confirmed,
+ evidenced_by: [],
+ applied_count: 0,
+ violated_count: 0,
+ last_evidence_at: null,
+ confidence_value: 0,
+ });
+}
+
+/** The rollup envelope's link candidates of a dream pass bound to the handler's predicate. */
+async function dreamCandidates(dropSecret: boolean, reach?: "local"): Promise<string> {
+ const fx = await fixture([], dropSecret, armRollup);
+ const ctx = {
+ vault: fx.vault,
+ configPath: fx.configPath,
+ repoRoot: null,
+ ...(reach !== undefined ? { reach: TRANSPORT_REACH.local } : {}),
+ };
+ const summary = dream(fx.vault, { dryRun: true, readable: readableAtContextReach(ctx) });
+ expect(summary.rollups).toHaveLength(1);
+ return JSON.stringify(summary.rollups[0]!.envelope.link_candidates);
+}
+
+describe("link candidates leave out a withheld page as they leave out an absent one", () => {
+ test("brain_design_note", async () => {
+ const r = await pair([], ["brain_design_note", { topic: "secret plan" }]);
+ expect(r.withheld).toBe(r.absent);
+ expect(r.withheld).not.toContain('"secret"');
+ expect(r.local).toContain('"secret"');
+ });
+
+ test("brain_diarize", async () => {
+ const setup: ReadonlyArray<Call> = [
+ [
+ "brain_ingest_source",
+ {
+ source_path: "Notes/open.md",
+ summary: "open summary",
+ entities: [{ category: "person", name: "Olga Open" }],
+ },
+ ],
+ ];
+ const r = await pair(setup, ["brain_diarize", { entity: "Olga Open" }]);
+ expect(r.withheld).toBe(r.absent);
+ expect(r.withheld).not.toContain('"secret"');
+ expect(r.local).toContain('"secret"');
+ });
+
+ // brain_dream does not project the rollup envelope onto the wire; the
+ // pass the handler runs is driven here with the predicate it binds.
+ test("brain_dream's rollup envelope", async () => {
+ const withheld = await dreamCandidates(false);
+ expect(withheld).toBe(await dreamCandidates(true));
+ expect(withheld).not.toContain('"secret"');
+ expect(await dreamCandidates(false, "local")).toContain('"secret"');
+ });
+});
+
+describe("a revert plan over a withheld page seals what an absent page's plan seals", () => {
+ test("brain_writes plan_revert", async () => {
+ const r = await writePair([
+ "brain_writes",
+ { action: "plan_revert", path: "Notes/private-write.md" },
+ ]);
+ expect(r.withheld.replace(/"planned_at":"[^"]*"/, "")).toBe(
+ r.absent.replace(/"planned_at":"[^"]*"/, ""),
+ );
+ expect(r.withheld).toContain('"entries":[]');
+ expect(r.local).toContain('"target":"Notes/private-write.md"');
+ });
+});
+
+/** The brain_status hygiene problem line, or a marker when there is none. */
+function hygieneLine(answerText: string): string {
+ const body = JSON.parse(answerText) as {
+ problems: ReadonlyArray<{ code: string; detail: string }>;
+ };
+ return body.problems.find((p) => p.code === "hygiene-findings")?.detail ?? "<none>";
+}
+
+// `secret-2.md` beside `secret.md` is a slug-collision finding that
+// names both pages.
+function collide(vault: string): void {
+ writeFileSync(join(vault, "Notes/secret-2.md"), "# Two\n");
+}
+
+describe("aggregate counts leave out a withheld page as they leave out an absent one", () => {
+ test("brain_status hygiene count", async () => {
+ const r = await pair([], ["brain_status", {}], collide);
+ expect(hygieneLine(r.withheld)).toBe(hygieneLine(r.absent));
+ expect(hygieneLine(r.local)).not.toBe(hygieneLine(r.withheld));
+ });
+});
diff --git a/tests/mcp/research-tool.test.ts b/tests/mcp/research-tool.test.ts
index b6fa04c9..c395244d 100644
--- a/tests/mcp/research-tool.test.ts
+++ b/tests/mcp/research-tool.test.ts
@@ -5,12 +5,14 @@
*/
import { afterEach, beforeEach, describe, expect, test } from "bun:test";
-import { mkdtempSync, rmSync } from "node:fs";
+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
import { join } from "node:path";
import { tmpdir } from "node:os";
import { bootstrapBrain } from "../../src/core/brain/init.ts";
+import { CAPTURE_SCOPE } from "../../src/core/brain/provenance/capture-scope.ts";
import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
+import { TRANSPORT_REACH } from "../../src/core/graph/transport-reach.ts";
import { RESEARCH_TOOLS } from "../../src/mcp/brain/research-tools.ts";
import { MCPError } from "../../src/mcp/protocol.ts";
import type { ServerContext } from "../../src/mcp/tool-contract.ts";
@@ -65,3 +67,44 @@ describe("brain_research_report", () => {
).rejects.toThrow(MCPError);
});
});
+
+/**
+ * Per-source capture scopes reach the MCP caller (distilled provenance, D3),
+ * aligned with the `sources` the caller passed, in the same order.
+ */
+describe("brain_research_report - capture_scopes", () => {
+ test("one scope per consulted source, in sources order", async () => {
+ mkdirSync(join(vault, "Articles"), { recursive: true });
+ writeFileSync(join(vault, "Articles", "held.md"), "# Held\n", "utf8");
+ const sources = ["https://example.test/post", "Articles/held.md"];
+ const res = (await handler(ctx, {
+ title: "Scoped survey",
+ sources,
+ findings: [{ statement: "A point", sources: ["Articles/held.md"] }],
+ })) as Record<string, unknown>;
+ expect(res["capture_scopes"]).toEqual([CAPTURE_SCOPE.urlOnly, CAPTURE_SCOPE.fullLocal]);
+ });
+
+ test("a vault page withheld at the caller's reach is reported as url-only", async () => {
+ mkdirSync(join(vault, "Notes"), { recursive: true });
+ writeFileSync(join(vault, "Notes", "secret.md"), "---\nvisibility: private\n---\nbody\n");
+ writeFileSync(join(vault, "Notes", "open.md"), "---\ntitle: open\n---\nbody\n");
+ const args = {
+ title: "Reach",
+ sources: ["Notes/secret.md", "Notes/open.md"],
+ findings: [{ statement: "A point", sources: ["Notes/open.md"] }],
+ };
+
+ const remote = (await handler({ ...ctx, reach: TRANSPORT_REACH.remote }, args)) as Record<
+ string,
+ unknown
+ >;
+ expect(remote["capture_scopes"]).toEqual([CAPTURE_SCOPE.urlOnly, CAPTURE_SCOPE.fullLocal]);
+
+ const local = (await handler({ ...ctx, reach: TRANSPORT_REACH.local }, args)) as Record<
+ string,
+ unknown
+ >;
+ expect(local["capture_scopes"]).toEqual([CAPTURE_SCOPE.fullLocal, CAPTURE_SCOPE.fullLocal]);
+ });
+});
diff --git a/tests/mcp/schema-tools.test.ts b/tests/mcp/schema-tools.test.ts
index 000764ee..8e8e3a2a 100644
--- a/tests/mcp/schema-tools.test.ts
+++ b/tests/mcp/schema-tools.test.ts
@@ -13,6 +13,7 @@ import { join } from "node:path";
import { bootstrapBrain } from "../../src/core/brain/init.ts";
import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
+import { TRANSPORT_REACH } from "../../src/core/graph/transport-reach.ts";
import { JSONRPC_VERSION, MCPServer, PROTOCOL_VERSION } from "../../src/mcp/index.ts";
import { INVALID_PARAMS } from "../../src/mcp/protocol.ts";
import { toPosix } from "../../src/core/path-safety.ts";
@@ -90,8 +91,8 @@ async function call(
});
}
-function makeServer(): MCPServer {
- return new MCPServer({ vault, configPath });
+function makeServer(reach?: typeof TRANSPORT_REACH.local): MCPServer {
+ return new MCPServer({ vault, configPath }, reach !== undefined ? { reach } : {});
}
describe("schema MCP tools", () => {
@@ -363,7 +364,9 @@ describe("schema_inspect reports the malformed artifact instead of dying on it",
for (const view of ["lint", "orphans"] as const) {
test(`view=${view} answers with findings, not an error`, async () => {
const planted = plantMalformedArtifacts();
- const server = makeServer();
+ // At local reach: below it, a record whose file cannot be read is
+ // treated as one the caller may not read, and is not named.
+ const server = makeServer(TRANSPORT_REACH.local);
await initialize(server);
const response = await call(server, "schema_inspect", { view });
diff --git a/tests/mcp/source-reach-first.test.ts b/tests/mcp/source-reach-first.test.ts
new file mode 100644
index 00000000..d74fb6eb
--- /dev/null
+++ b/tests/mcp/source-reach-first.test.ts
@@ -0,0 +1,141 @@
+/**
+ * The reach question is asked before any filesystem question about a cited
+ * source, on the vault-relative identity the classifier resolves.
+ *
+ * `brain_distill_source` and `brain_intake_entities` answer for a vault page
+ * the caller may not read at its reach exactly as for an absent one. That
+ * holds for a page the size ceiling or a refused read would otherwise stop:
+ * the caller is answered `untrusted`, and only a caller that may read the
+ * page is told why it cannot be classified. A predicate keyed on plain
+ * strings is asked about the resolved identity, never a spelling with `.`
+ * or `..` segments.
+ */
+
+import { afterEach, beforeEach, describe, expect, test } from "bun:test";
+import { chmodSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
+import { join } from "node:path";
+import { tmpdir } from "node:os";
+
+import { bootstrapBrain } from "../../src/core/brain/init.ts";
+import { distillSource } from "../../src/core/brain/distill/distill-source.ts";
+import { intakeExtraction } from "../../src/core/brain/intake/extract-intake.ts";
+import { SOURCE_HASH_MAX_BYTES } from "../../src/core/brain/intake/source-trust.ts";
+import { resolveCaptureScope } from "../../src/core/brain/provenance/capture-scope.ts";
+import { INTAKE_TRUST } from "../../src/core/brain/trust/untrusted-provenance.ts";
+import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
+import { TRANSPORT_REACH } from "../../src/core/graph/transport-reach.ts";
+import { DISTILL_TOOLS } from "../../src/mcp/brain/distill-tools.ts";
+import { NER_TOOLS } from "../../src/mcp/brain/ner-tools.ts";
+import type { ServerContext } from "../../src/mcp/tool-contract.ts";
+import { CHMOD_CANNOT_DENY } from "../helpers/platform.ts";
+
+const PRIVATE_PATH = "Notes/secret.md";
+const ABSENT = "Notes/absent.md";
+const OPEN = "Notes/open.md";
+const PRIVATE_HEAD = "---\nvisibility: private\n---\n";
+const NOW = new Date("2026-06-13T12:00:00Z");
+
+let vault: string;
+let configHome: string;
+let ctx: ServerContext;
+
+beforeEach(() => {
+ vault = mkdtempSync(join(tmpdir(), "o2b-reach-first-vault-"));
+ configHome = mkdtempSync(join(tmpdir(), "o2b-reach-first-cfg-"));
+ const configPath = join(configHome, "config.yaml");
+ atomicWriteFileSync(configPath, `vault: ${vault}\nagent_name: claude\n`);
+ bootstrapBrain(vault, { configPath });
+ ctx = { vault, configPath, repoRoot: null };
+ mkdirSync(join(vault, "Notes"), { recursive: true });
+ writeFileSync(join(vault, OPEN), "open\n");
+});
+
+afterEach(() => {
+ rmSync(vault, { recursive: true, force: true });
+ rmSync(configHome, { recursive: true, force: true });
+});
+
+const distillTool = DISTILL_TOOLS[0]!.handler;
+const intakeTool = NER_TOOLS[0]!.handler;
+const localCtx = (): ServerContext => ({ ...ctx, reach: TRANSPORT_REACH.local });
+
+/** The `trust` each of the two tools answers for `source` at `reachCtx`. */
+async function trustOf(source: string, reachCtx: ServerContext): Promise<unknown[]> {
+ const distilled = (await distillTool(reachCtx, {
+ source_path: source,
+ claims: [{ text: "A claim." }],
+ })) as Record<string, unknown>;
+ const intaken = (await intakeTool(reachCtx, {
+ source,
+ entities: [{ category: "concept", name: "Codes" }],
+ })) as Record<string, unknown>;
+ return [distilled["trust"], intaken["trust"]];
+}
+
+describe("a page withheld at the caller's reach is never stat-ed or read for it", () => {
+ test("a page past the size ceiling answers like an absent one", async () => {
+ writeFileSync(join(vault, PRIVATE_PATH), PRIVATE_HEAD + "a".repeat(SOURCE_HASH_MAX_BYTES + 1));
+ const untrusted = [INTAKE_TRUST.untrusted, INTAKE_TRUST.untrusted];
+ expect(await trustOf(PRIVATE_PATH, ctx)).toEqual(untrusted);
+ expect(await trustOf(ABSENT, ctx)).toEqual(untrusted);
+ await expect(trustOf(PRIVATE_PATH, localCtx())).rejects.toThrow(String(SOURCE_HASH_MAX_BYTES));
+ });
+
+ test.skipIf(CHMOD_CANNOT_DENY)("an unreadable page answers like an absent one", async () => {
+ writeFileSync(join(vault, PRIVATE_PATH), `${PRIVATE_HEAD}body\n`);
+ chmodSync(join(vault, PRIVATE_PATH), 0o000);
+ try {
+ expect(await trustOf(PRIVATE_PATH, ctx)).toEqual([
+ INTAKE_TRUST.untrusted,
+ INTAKE_TRUST.untrusted,
+ ]);
+ await expect(trustOf(PRIVATE_PATH, localCtx())).rejects.toThrow("EACCES");
+ } finally {
+ chmodSync(join(vault, PRIVATE_PATH), 0o644);
+ }
+ });
+});
+
+describe("the predicate is asked about the resolved identity", () => {
+ const SPELLINGS = ["Notes/./secret.md", "[[Notes/../Notes/secret.md]]"];
+ const byString = (rel: string): boolean => rel !== PRIVATE_PATH;
+
+ beforeEach(() => {
+ writeFileSync(join(vault, PRIVATE_PATH), "body\n");
+ });
+
+ test("a single-source and a multi-source intake", () => {
+ for (const spelling of SPELLINGS) {
+ for (const sources of [[spelling], [OPEN, spelling]]) {
+ const res = intakeExtraction(
+ vault,
+ { entities: [{ category: "concept", name: "Codes" }] },
+ {
+ agent: "claude",
+ now: NOW,
+ provenance: { level: "stated", sources, premises: [] },
+ readable: byString,
+ },
+ );
+ expect(res.trust).toBe(INTAKE_TRUST.untrusted);
+ }
+ }
+ });
+
+ test("a distillation", () => {
+ for (const spelling of SPELLINGS) {
+ const res = distillSource(
+ vault,
+ { sourcePath: spelling, claims: [{ text: "A claim." }] },
+ { agent: "claude", now: NOW, readable: byString },
+ );
+ expect(res.trust).toBe(INTAKE_TRUST.untrusted);
+ }
+ });
+
+ test("the capture-scope backing file", () => {
+ for (const spelling of SPELLINGS) {
+ expect(resolveCaptureScope(vault, spelling).backing).toBe(PRIVATE_PATH);
+ }
+ });
+});
diff --git a/tests/mcp/tiers-reach.test.ts b/tests/mcp/tiers-reach.test.ts
new file mode 100644
index 00000000..156b1fb6
--- /dev/null
+++ b/tests/mcp/tiers-reach.test.ts
@@ -0,0 +1,106 @@
+/**
+ * `brain_tiers` answers for a page the caller may not read at its reach
+ * exactly as for an absent one.
+ *
+ * A tier-drift row names a page's path, the drifted identity field, and
+ * both its expected and its actual value. `check` keeps only the rows of
+ * pages the caller may read, and `restore` and `accept` refuse a page it
+ * may not read with the refusal a page the index never saw gets, before
+ * anything is written. At local reach the drift is listed as before.
+ */
+
+import { afterEach, describe, expect, test } from "bun:test";
+import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import { bootstrapBrain } from "../../src/core/brain/init.ts";
+import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";
+import { TRANSPORT_REACH } from "../../src/core/graph/transport-reach.ts";
+import { resolveSearchConfig } from "../../src/core/search/index.ts";
+import { indexVault } from "../../src/core/search/indexer.ts";
+import { MCPServer } from "../../src/mcp/server.ts";
+
+const PREF = "Brain/preferences/pref-secret.md";
+
+/** A private preference whose identity field is `id`. */
+function prefBody(id: string, body: string): string {
+ return (
+ `---\nkind: brain-preference\nid: ${id}\ncreated_at: 2026-05-01T00:00:00Z\n` +
+ `topic: style\nvisibility: private\n---\n\n${body}`
+ );
+}
+
+const bases: string[] = [];
+
+afterEach(() => {
+ for (const base of bases.splice(0)) rmSync(base, { recursive: true, force: true });
+});
+
+interface Fixture {
+ readonly vault: string;
+ readonly server: MCPServer;
+}
+
+/**
+ * One indexed vault where the private preference's `id` was hand-edited
+ * after the first index (so one drift row stands), or one where the page
+ * never existed. With no `reach`, the server mints none: remote.
+ */
+async function fixture(
+ withSecret: boolean,
+ reach?: typeof TRANSPORT_REACH.local,
+): Promise<Fixture> {
+ const base = mkdtempSync(join(tmpdir(), "o2b-tiers-reach-"));
+ bases.push(base);
+ const vault = join(base, "vault");
+ mkdirSync(vault, { recursive: true });
+ const configPath = join(base, "config.yaml");
+ atomicWriteFileSync(configPath, `vault: ${vault}\nagent_name: claude\n`);
+ bootstrapBrain(vault, { configPath });
+ mkdirSync(join(vault, "Brain/preferences"), { recursive: true });
+ writeFileSync(join(vault, "Brain/preferences/pref-open.md"), "# Open\n");
+ const config = resolveSearchConfig({ vault, configPath });
+ if (withSecret) writeFileSync(join(vault, PREF), prefBody("pref-secret", "Use spaces.\n"));
+ await indexVault(config);
+ if (withSecret) {
+ // A size change, so the indexer's mtime+size fast path re-reads it.
+ writeFileSync(join(vault, PREF), prefBody("pref-renamed", "Use considerably more spaces.\n"));
+ }
+ await indexVault(config);
+ const server = new MCPServer({ vault, configPath }, reach !== undefined ? { reach } : undefined);
+ return { vault, server };
+}
+
+async function answer(f: Fixture, args: Record<string, unknown>): Promise<string> {
+ try {
+ return `ok: ${JSON.stringify(await f.server.callTool("brain_tiers", args))}`;
+ } catch (err) {
+ const e = err as Error & { code?: unknown; data?: unknown };
+ return `error: ${e.message} ${JSON.stringify(e.code)} ${JSON.stringify(e.data)}`;
+ }
+}
+
+describe("a page the caller cannot read answers as an absent one", () => {
+ for (const args of [
+ { operation: "check" },
+ { operation: "restore", path: PREF, apply: true },
+ { operation: "accept", path: PREF },
+ ]) {
+ test(JSON.stringify(args), async () => {
+ const hidden = await fixture(true);
+ const absent = await fixture(false);
+ expect(await answer(hidden, args)).toBe(await answer(absent, args));
+ expect(readFileSync(join(hidden.vault, PREF), "utf8")).toContain("id: pref-renamed");
+ });
+ }
+});
+
+describe("at local reach the drift is listed", () => {
+ test("check names the page and the field", async () => {
+ const local = await fixture(true, TRANSPORT_REACH.local);
+ const out = await answer(local, { operation: "check" });
+ expect(out).toContain(PREF);
+ expect(out).toContain("pref-renamed");
+ });
+});
diff --git a/tests/mcp/tool-error-codes.test.ts b/tests/mcp/tool-error-codes.test.ts
index 4f59c4a4..a151adb9 100644
--- a/tests/mcp/tool-error-codes.test.ts
+++ b/tests/mcp/tool-error-codes.test.ts
@@ -20,6 +20,11 @@ import {
ExpirationValueError,
InvalidExpirationTargetError,
} from "../../src/core/brain/expiration-set.ts";
+import {
+ QUOTE_CHECK_OUTCOME,
+ QUOTE_UNVERIFIED_CODE,
+ QuoteCheckError,
+} from "../../src/core/brain/distill/quote-verdict.ts";
import { BrainParseError } from "../../src/core/brain/parse-error.ts";
import { BrainStatusFolderMismatchError } from "../../src/core/brain/preference.ts";
import { PinnedBatchError } from "../../src/core/brain/pinned.ts";
@@ -82,6 +87,7 @@ const NEW_TOKENS = [
"unknown_argument",
"brain_artifact_unparseable",
"argument_forbidden",
+ "quote_unverified",
] as const;
/**
@@ -163,7 +169,7 @@ describe("TOOL_ERROR_CODES", () => {
expect(new Set(TOOL_ERROR_CODES).size).toBe(TOOL_ERROR_CODES.length);
});
- test("is exactly the imported vocabularies plus the 21 new tokens", () => {
+ test("is exactly the imported vocabularies plus the 22 new tokens", () => {
const expected = new Set<string>([
...NEW_TOKENS,
...SEARCH_ERROR_CODES,
@@ -286,6 +292,15 @@ describe("codeForError", () => {
expect(lines).toEqual([]);
});
+ test("a strict quote refusal is quote_unverified", () => {
+ const error = new QuoteCheckError([
+ { claim: 0, outcome: QUOTE_CHECK_OUTCOME.notInBlock, span: "x" },
+ ]);
+ expect(codeForError(error)).toBe("quote_unverified");
+ expect(TOOL_ERROR_CODE.quoteUnverified).toBe(QUOTE_UNVERIFIED_CODE);
+ expect(lines).toEqual([]);
+ });
+
test("walks the cause chain of a rethrown error", () => {
const original = new BrainPreferenceNotFoundError("pref-x", "Brain/preferences/pref-x.md");
const rethrown = new Error(original.message, { cause: original });
diff --git a/tests/mcp/visibility-matrix.test.ts b/tests/mcp/visibility-matrix.test.ts
index 7a45a741..1dfb553b 100644
--- a/tests/mcp/visibility-matrix.test.ts
+++ b/tests/mcp/visibility-matrix.test.ts
@@ -500,10 +500,6 @@ const COVERED_TOOLS: ReadonlySet<string> = new Set(
* queued and `brain_trigger operation=terminal` does not.
*/
const STILL_NAMED_AT_REMOTE: ReadonlySet<string> = new Set([
- "brain_agent_diff mode=browse",
- "brain_agent_diff mode=diff",
- "brain_agent_diff mode=map",
- "brain_agent_query",
"brain_analytics view=belief_evolution",
"brain_analytics view=concept_synthesis",
"brain_analytics view=timeline",
@@ -514,12 +510,7 @@ const STILL_NAMED_AT_REMOTE: ReadonlySet<string> = new Set([
"brain_claims operation=history",
"brain_context",
"brain_context_pack",
- "brain_design_note",
- "brain_doctor",
- "brain_doctor #1",
"brain_doctor #2",
- "brain_event_trace",
- "brain_idea_discovery",
"brain_pre_compress_pack",
"brain_retention",
"brain_scaffold_stub action=list",
diff --git a/uv.lock b/uv.lock
index fea388cd..fe041a30 100644
--- a/uv.lock
+++ b/uv.lock
@@ -4,5 +4,5 @@ requires-python = ">=3.11"
[[package]]
name = "open-second-brain"
-version = "1.66.0"
+version = "1.67.0"
source = { editable = "." }