Local-first 🧠memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly 😴 dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.
TypeScript
feat: silent failures surface by name in redaction, receipts, ingest, search store, doctor (v1.62.0) (#220)
Serg Eroshef4f9e3e622b5
diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.jsonindex e2aa09f1..d1be9021 100644--- a/.claude-plugin/plugin.json+++ b/.claude-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.61.0",+ "version": "1.62.0","description": "Plugin-first second brain package for AI agents and humans.","author": {"name": "Open Second Brain contributors"diff --git a/.codex-plugin/plugin.json b/.codex-plugin/plugin.jsonindex db6d7a3d..94008309 100644--- a/.codex-plugin/plugin.json+++ b/.codex-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.61.0",+ "version": "1.62.0","description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.","author": {"name": "Open Second Brain contributors",diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 9c0f453c..43ce846a 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -5,6 +5,31 @@ All notable changes to this project will be documented in this file.The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).+## [1.62.0] - 2026-09-29++Wider egress redaction, honest write receipts, an ingest plan that accounts for what it sets aside, a search store that keeps its journal mode honest on remote filesystems, and two more brain-doctor checks with a confirm-gated repair verb.++### Fixed++- **URL credentials whose username is empty are now redacted.** The URL pass required a username, so the password-only spelling connection strings use (`scheme://:pass@host`) was left verbatim wherever the pass runs, including the export boundary where it is on by default. The username half is optional now; the colon between the halves stays mandatory and adjacent, so a URL whose path carries an `@` but no userinfo colon is untouched, and the pass stays linear.+- **URL credentials whose password carries a slash are now redacted.** The password class stopped at a slash, and one unmatched character failed the whole match - the username was left verbatim too. The class now runs to the `@` anchor, so a generated database password with a slash is redacted wherever the URL pass runs. A colon followed by a port and a path is never read as the userinfo colon, so a URL such as `http://localhost:5173/@vite/client` or `https://registry.npmjs.org:443/@types/node` stays untouched (a slash password that itself begins with 1-5 digits is the known miss, as on main).+- **Bare JSON Web Tokens are redacted by default.** A JWT pass runs next to the bearer pass on every surface the shared redactor guards: it is anchored on the header prefix - compact JSON (`eyJ`) or a header with whitespace after the brace (`eyA`, `ewo`, `ew0`, `ewk`) - with segments bounded at 64 KiB each, so a bare token - not only a `Bearer`-prefixed one - is redacted.+- **A secret named beside its value is redacted in structured data.** The structured walk reads a `{name, value}` pair whose sibling name names a secret - an environment block such as `{"name": "DB_PASSWORD", "value": "..."}` - and redacts the value.+- **A byte-identical note rewrite no longer bumps the note's mtime.** The atomic rename landed a fresh inode on every write, so re-applying identical bytes moved the file's `mtime` while the receipt claimed `updated: true` for a write that changed nothing - poisoning recency ranking, validity windows and vault-delta counts. Both write chokepoints now skip a byte-identical rewrite, and `updated` / `appended` carry the write's own verdict: a skipped write stores no before-image and records no note-write event. The skip is decided before anything is written, so a rewrite that does change the bytes still stores its before-image before the rename, and a failed store leaves the note untouched.+- **The search index opens with journal mode DELETE on a remote filesystem, and says so once.** An index file on a network filesystem (on Linux nfs, cifs, SMB2/SMB3, smbfs, ceph, afs and 9p; on Windows a UNC path; macOS is not probed) opens without WAL, and the write open prints one clear warning naming the filesystem and the slower writes; every later open makes the same decision instead of enabling WAL there. Leaving WAL needs exclusive access, so while another connection has an index in WAL open the switch is kept for later instead of failing: the read open stays in WAL silently (never an `INDEX_UNREADABLE` that would trigger a reindex), and the write open keeps WAL, warns once that it stays until other connections close, and retries on its next open.+- **A batch write lints only the pages it actually committed.** `brain_write_batch` named a page for the write-time lint whenever a result carried a path, which included skipped rewrites; a skipped operation now names no page, matching the single-write receipts.++### Changed++- **Docs:** `docs/cli-reference.md` lists `o2b brain orphan-repair`; `docs/mcp.md` documents the plan's `skip_reason_counts` and `unclassifiable` aggregates, the optional `path_advisory` on note receipts, and the new doctor findings; `docs/observability.md` and `docs/how-it-works.md` describe the before-image ordering and the journal-mode decision as the code now makes them.++### Added++- **A note write receipt can carry a non-blocking `path_advisory`.** An authored note body that mentions an absolute host path is advised on the receipt (frontmatter values are never scanned) through the hygiene scan's grammar, as-is; the key is absent when the content is clean, log-line operations are never scanned, and the advisory never blocks the write.+- **The ingest plan accounts for what it sets aside.** Every file the extractable gate sets aside (`skipped_non_extractable`) carries a reason from a closed, typed set and the value behind it (the legacy free-text sentence still parses, through `parseSkippedPageReason`; the type guard accepts members only), while `skipped` stays a list of unchanged paths; the plan aggregates per-reason counts, and extensionless regular files nothing can classify are counted by extension; both aggregates appear on the MCP plan and the CLI rendering only when non-empty, so a plan with nothing set aside serializes exactly as before.+- **`o2b brain doctor` checks embedding health.** A fail-soft, read-only check reports an embedding backlog, an unrecorded census and an unmeasured health verdict; an unrecorded census lands in `uncertain`, never in healthy. The stale count resolves the dimension the way the indexer does (configured, then the local embedder's default, then the stored value) and compares by model alone when none resolves, so a model switch on a default configuration is counted - in `o2b search status` too; a provider with no configured model is `uncertain`, and the check reads the index the doctor pass was pointed at.+- **`o2b brain doctor` reports orphaned session references, and `o2b brain orphan-repair` detaches them.** An observation signal whose `session_ref` resolves against neither the continuity ledger nor the Brain log's session lifecycle is a warning that names the repair verb on its `fix` field. A ref written by `o2b brain sessions import` (`<transcript basename>#<turn>`) points at a transcript outside the vault that the import keeps no guaranteed record of, so it is never judged; the capture form and the extract-signals form are. A log day or shard the pass could not read makes the check report `uncertain` and no finding, because a session whose lifecycle lived there would otherwise read as an orphan. The verb removes only the `session_ref` key, and only while it still holds the value the scan resolved under the signal's per-file lock (a reference rewritten in between is reported `skip-changed`, a contended lock `skip-locked`) - the observation body, topic and every other field stay - dry-run by default, behind an exact confirmation phrase, with a hard per-run cap, and it is never reachable from the doctor pass itself.+## [1.61.0] - 2026-09-28MCP server start-up off the Brain upgrade path, fair ingest locks, the Claude Code MCP manifest inside `plugin.json`, and an inbox archive (issues #216, #198, #196, #195).@@ -7810,6 +7835,7 @@ plugin config (vault field)`, and exits with a clear- Sandbox vault and plugin manifest fixtures for tests.- GitHub release workflow for tag-based and manually dispatched releases.+[1.62.0]: https://github.com/itechmeat/open-second-brain/compare/v1.61.0...v1.62.0[1.61.0]: https://github.com/itechmeat/open-second-brain/compare/v1.60.0...v1.61.0[1.60.0]: https://github.com/itechmeat/open-second-brain/compare/v1.59.0...v1.60.0[1.59.0]: https://github.com/itechmeat/open-second-brain/compare/v1.58.2...v1.59.0diff --git a/docs/brainstorm/p4-silent-failure-hardening/cli-output/claude.md b/docs/brainstorm/p4-silent-failure-hardening/cli-output/claude.mdnew file mode 100644index 00000000..098aad50--- /dev/null+++ b/docs/brainstorm/p4-silent-failure-hardening/cli-output/claude.md@@ -0,0 +1,38 @@+### Variant 1: Nine local patches at the verified anchors+- **Approach**: Each card lands as an isolated change at its named anchor with card-local types and tests: two redactor passes, one walk tweak, one receipt field, two independent skip counters, two `skipIfUnchanged` call sites, one pragma gate, two doctor checks plus one repair verb. Cards share nothing new; they only register into the vocabularies that already exist (`DEGRADATION_CODE`, `DOCTOR_CHECKS`, the verdict census).+- **Trade-offs**:+ - Pro: every card is independently reviewable and revertable; nine small PR-sized diffs fold cleanly into one squash.+ - Pro: fully parallelizable across subagents with no shared file contention beyond the census test.+ - Pro: lowest chance of accidentally widening a public shape, since no new shared type crosses a wire contract.+ - Con: cards 4 and 5 both count ingest skips on the same BatchPlan lane and would ship two counter shapes and two serializer touches for one concept.+ - Con: cards 8 and 9 each hand-roll census plumbing and an uncertain stream, duplicating what `embeddingSunsetCheck` already does.+ - Con: naming drift across nine authors (reason strings, warning codes) is likely and only caught at review.+- **Complexity**: medium+- **Risk**: low++### Variant 2: Local patches with three shared seams where cards pair up+- **Approach**: Keep the nine anchors, but factor the three natural pairs onto one seam each: cards 4 and 5 share a typed ingest skip taxonomy (`SkippedPage.reason` membership list plus parser) whose per-reason counters also carry the unclassifiable-extension bucket, emitted once through `serializeBatchPlan`; cards 8 and 9 share a small read-only census helper modeled on `embeddingSunsetCheck` (probe, count, uncertain stream, embedded next command) and register two thin checks on top; cards 1 and 2 extend the redactor as one detector-table change with one census entry set. Cards 3, 6 and 7 stay strictly local, with 7 adding only a remoteness axis to `vault-backing.ts` that `applyPragmas` consults.+- **Trade-offs**:+ - Pro: one skip vocabulary and one serializer touch for the whole ingest lane, so CLI JSON and MCP cannot diverge.+ - Pro: doctor checks stay uniform in shape and failure isolation, and the repair verb for card 9 reuses the repair-lane discipline unchanged.+ - Pro: still additive and byte-identical-when-absent everywhere; no default flips; no new dependencies.+ - Con: the paired cards become sequential (taxonomy before counters, helper before checks), reducing parallelism for two of the nine.+ - Con: a shared helper is a small new internal surface that needs its own tests beyond the per-card ones.+ - Con: slightly larger diff on `batch-plan.ts` and `doctor.ts` than pure local patches.+- **Complexity**: medium+- **Risk**: low++### Variant 3: Unified silent-failure ledger+- **Approach**: Introduce one cross-cutting notice primitive (code, count, surface, severity) with a single serializer, and make all nine cards producers into it: redaction hits, sibling-pair hits, absolute-path advisories, skipped files and reasons, short-circuited rewrites, WAL downgrade, and both doctor findings. Doctor checks then become readers of the ledger rather than independent scanners, and every CLI verb and MCP tool emits the same notice block.+- **Trade-offs**:+ - Pro: one place to look for everything that degraded silently, fully aligned with the v1.52.0 principle.+ - Pro: future cards plug in with zero new wire plumbing.+ - Con: touches every wire contract in the wave at once and makes byte-identical-when-absent hard to prove across CLI JSON and MCP.+ - Con: the write-receipt, batch-plan and doctor surfaces have different lifetimes; forcing them through one ledger blurs read-only doctor checks with write-path advisories.+ - Con: the redactor is a hot egress path and should not gain a ledger dependency; the mtime-skip audit result already has its own `updated` semantics.+ - Con: far exceeds the verified, narrowed scope of the nine premises.+- **Complexity**: large+- **Risk**: high++### Recommended: Variant 2+**Rationale**: The premise verification already narrowed each card to a specific anchor, so the wave should stay anchor-local, but three pairs (4/5, 8/9, 1/2) genuinely describe one concept each, and shipping them as separate shapes would violate the one-serializer-per-wire-contract and closed-vocabulary conventions on day one. Variant 2 removes that duplication with two small internal seams and one detector-table change, keeps every change additive and opt-in, and preserves independent review for the six cards that do not overlap. Variant 3 is the right long-term shape but is a scope change the operator did not pick.diff --git a/docs/brainstorm/p4-silent-failure-hardening/cli-output/prompt.md b/docs/brainstorm/p4-silent-failure-hardening/cli-output/prompt.mdnew file mode 100644index 00000000..16effaac--- /dev/null+++ b/docs/brainstorm/p4-silent-failure-hardening/cli-output/prompt.md@@ -0,0 +1,69 @@+You are brainstorming architectural variants for the following task. Do not write code. Do not write a final design. Only produce variants and a recommendation.++# Task++Wave scope (operator-picked option 3, "P4 silent-failure hardening"): nine tracker cards, already verified against the live source at commit f22df2c5 by a premise-verification pass. Every card premise survived; the verification corrected anchors and narrowed scope. The wave ships the surviving premises as one merged, released increment. Per card, the verified premise and the landed scope:++1. t_5c777198 (redaction wire formats): the URL-credential regex `BASIC_AUTH_URL_RE` (src/core/redactor.ts:249) has password class `[^\s/@]+`, so a password containing `/` (common in generated secrets) defeats the whole match and both user and password leak through the egress redaction boundary; a bare three-segment JWT passes verbatim under default options and is only partially redacted under `redactTokens` (the 20-character header slips the 24-character HIGH_ENTROPY gate). Land: widen the password class within the documented bounded/linear regex constraint, and add a default-on three-segment JWT detector next to the existing `BEARER_RE` pass.+2. t_c70f3d94 (sibling {name, value} secret pairs): the structured redaction walk keys secrecy off the value's own key only, so `{environment: [{name: "DB_PASSWORD", value: "hunter2"}]}` passes the value verbatim; the existing `isSecretKeyName` matcher already matches the sibling name, the walk never consults it. Land: inspect plain-object array elements for `{name, value}` shape and reuse `isSecretKeyName` on the sibling name; ECS `valueFrom` is an ARN reference, out of scope.+3. t_796c1a9b (save-time absolute-path warning): note bodies are never scanned for absolute paths at write time, while a reusable grammar exists at src/core/hygiene/hardcoded-paths.ts (currently CI-only). Land: non-blocking advisory on the write receipt, following the page-lint precedent (`pageLintField` on `noteWriteResult`), advised not blocking (a WriteBatchError would abort the all-or-nothing batch).+4. t_9b30f48b (unclassifiable ingested files): two vault walks and the source-ingest discovery walk `collectIngestible` silently drop files whose extension is not ingestible; no count or notice exists. Land: aggregate counters for skipped extensions plumbed to a surface, reusing the closed `DEGRADATION_CODE` vocabulary.+5. t_09d52938 (typed skip taxonomy): `SkippedPage.reason` is free text; the session lane already has a dry run with counters, so the genuinely new ground is typing the source-ingest skip reasons and adding aggregate per-reason counters on the BatchPlan lane. No new preview. Land: type `SkippedPage.reason` following the `PageLintSkipReason` pattern (membership list plus narrowing parser, wire-compatible), aggregate per-reason counters surfaced through `serializeBatchPlan` and the CLI verb.+6. t_a77e35a2 (mtime-stable re-renders): `skipIfUnchanged` exists in fs-atomic but no production caller passes it; every identical rewrite lands a fresh inode and bumps mtime. Land: opt-in at exactly the two unguarded chokepoints, `commitNoteRewrite` (src/core/brain/write-batch.ts:594) and the `writeFrontmatterAtomic` overwrite branch (src/core/vault.ts:477); on short-circuit skip the before-image store and the note-write record, and make the audit result `updated` reflect whether bytes were written. No default flip in fs-atomic.+7. t_541083b3 (WAL on remote backing): `applyPragmas` sets `PRAGMA journal_mode = WAL` unconditionally on every index open; on NFS/CIFS/FUSE backing WAL is unreliable. The backing probe (src/core/vault-backing.ts) classifies nfs/cifs/fuse as `durable` and has no remoteness axis. Land: add a remoteness axis to the probe; on a remote backing, skip WAL (stay on the default journal mode) and warn, no hard refuse; non-Linux probe-unsupported stays silent, WSL/9p false-positive risk respected.+8. t_f99250aa (embeddings-health doctor check): signals for embeddings health exist and are scattered (pending-vector census, stale-embedding counts, provider probe in `o2b search check`) but nothing is consolidated into the brain doctor. Land: a new read-only failSoft doctor check modeled on `embeddingSunsetCheck`, registered in `DOCTOR_CHECKS`, message-embedded next command, uncertain stream for unrecorded census.+9. t_6cc80627 (orphan observation signals): observation signals carry `session_ref` and nothing resolves it against the sessions/continuity store; a signal whose parent session is gone is silently accepted. Land: a doctor check (failSoft, read-only) for unresolved `session_ref` signals, plus a repair verb following the repair-lane discipline (dry-run default, exact confirm phrase, hard write cap, never reachable from `runDoctor`).++# Project context++Project: Open Second Brain, TypeScript (ESM, NodeNext) on Bun 1.4.0; an Obsidian-compatible Markdown vault with a SQLite WAL search index, a deterministic CLI (`o2b`) and an MCP server.+Recent commits:+f22df2c5 chore(apb): zg search, code-ranker and OpenCodeReview before push, scope-scan hardening (#218)+83979b9d fix: MCP startup off the upgrade path, fair ingest locks, inline plugin MCP manifest, inbox archive (v1.61.0)+3e59f319 feat: decision-model uses, adapters and setup skill (v1.60.0)+eacad953 feat: optional decision-model core and decision-model rerank kind (v1.59.0)+5c550263 fix: session capture off the hook path, registry scanner hygiene (v1.58.1)+6f33c424 feat: payload registry and knowledge packs, boundary hardening, CJK chunking (v1.58.0)+54bb28d9 feat: what this install knows (v1.56.0)+5f415267 feat: who wrote what, and how to take it back (v1.55.0)+dc552590 feat: private is not a suggestion (v1.54.0)+323e2f09 feat: nothing writes silently, nothing degrades silently (v1.52.0)+Related files:+- src/core/redactor.ts, src/core/egress/guard.ts, src/core/brain/continuity/redaction.ts+- src/core/hygiene/hardcoded-paths.ts, src/core/hygiene/scan-repo.ts+- src/core/brain/write-batch.ts, src/core/brain/page-lint.ts, src/mcp/brain/write-batch-tools.ts, src/mcp/brain/notes-tools.ts+- src/core/vault.ts, src/core/fs-atomic.ts+- src/core/brain/ingest/batch-plan.ts, src/core/brain/ingest/extractable-gate.ts, src/core/integrity/degradation.ts+- src/mcp/brain/ingest-tools.ts, src/cli/brain/verbs/batch-plan.ts, src/core/brain/metrics.ts+- src/core/vault-backing.ts, src/core/search/store/lifecycle.ts, src/core/search/store.ts, src/core/search/paths.ts, src/core/install/ownership.ts+- src/core/brain/doctor.ts, src/core/brain/doctor/embedding-sunset-check.ts, src/core/brain/doctor/log-checks.ts, src/core/brain/types.ts+- src/core/search/indexer.ts (pending-vector census), src/core/brain/signal.ts, src/core/brain/link-graph/repair-lane.ts, src/cli/brain/verbs/repair-lane.ts+Conventions:+- Closed vocabularies with census tests: new verdict/reason codes must be registered in tests/core/architecture/verdict-vocabulary-census.test.ts or the census fails.+- Wire serializers follow a byte-identical-when-absent idiom: new fields are emitted only when non-empty; one serializer per wire contract shared by CLI JSON and MCP.+- Warnings are non-blocking by default; refusal paths are named errors with next-step advice (v1.52.0 rule: nothing writes silently, nothing degrades silently).+- Doctor checks are read-only and failSoft; one broken scan must not mask later checks; repairs live in dedicated verbs with dry-run default, exact confirm phrase and a hard write cap.+- Regexes in the redactor must stay bounded and linear (documented no-ReDoS constraint).+- Tests colocated under tests/ mirroring src/; TDD; formatter (oxfmt) and linter (oxlint) must pass before every commit.+Constraints:+- No new external dependencies.+- No changes to existing public API shapes except additive, byte-identical-when-absent fields.+- Do not flip global defaults (no fs-atomic default change, no config default changes); opt-in at named chokepoints.+- English only, no hardcoded natural-language phrases outside existing copy conventions; no AI-authorship markers in committed artifacts.++# Required output format++Produce exactly 3 distinct architectural variants. For each variant:++### Variant N: <short name>+- **Approach**: 2-3 sentences describing the variant.+- **Trade-offs**: bullet list of pros and cons.+- **Complexity**: small | medium | large+- **Risk**: low | medium | high++After the three variants, add exactly one recommendation:++### Recommended: Variant N+**Rationale**: 2-3 sentences explaining why this variant over the others, considering the project context and constraints above.++Output nothing outside of these sections.diff --git a/docs/brainstorm/p4-silent-failure-hardening/design.md b/docs/brainstorm/p4-silent-failure-hardening/design.mdnew file mode 100644index 00000000..85737e61--- /dev/null+++ b/docs/brainstorm/p4-silent-failure-hardening/design.md@@ -0,0 +1,189 @@+# P4 silent-failure hardening - nine verified premises landed where they fail quietly++**Status:** draft+**Author:** osb-developer (via feature-release-playbook)+**Audience:** implementation++## Problem statement++Nine tracker cards describe places where Open Second Brain fails silently: credentials that cross the+egress redaction boundary because a regex class excludes one character, secret values that ride past the+structured redaction walk under a sibling key, note bodies that embed absolute home paths with no+advisory at write time, ingested files that vanish between discovery and plan with no count, skip+reasons that are free text and cannot be aggregated, byte-identical rewrites that bump mtime and+poison recency and validity signals, a WAL journal mode forced onto filesystems where it is unreliable,+and health/orphan signals that exist but are never consolidated where an operator looks. Every premise+was verified against the live source at commit f22df2c5 before this design; all nine survived, each+with anchor corrections and a narrowed scope that this design folds in. The wave ships the surviving+premises as one merged, released increment, continuing the v1.52.0 rule: nothing writes silently,+nothing degrades silently.++## Premise verification++Verification ran against worktree branch `feat/p4-silent-failure-hardening` at commit f22df2c5, in five+cluster reports (`docs/brainstorm/p4-silent-failure-hardening/cli-output/` carries the brainstorm+consultant outputs; the cluster reports were consumed as design inputs). Load-bearing anchors were+spot-checked in the live tree during design (redactor.ts:249, write-batch.ts:584-602,+store/lifecycle.ts:51-60, extractable-gate.ts:29-32, batch-plan.ts:502-522, doctor.ts:139). Per-card+verdicts, all confirmed with corrections folded in:++| Card | Verdict | Evidence anchor | Correction folded in |+|---|---|---|---|+| t_5c777198 | confirmed | redactor.ts:249 (`BASIC_AUTH_URL_RE` password class `[^\s/@]+`), HIGH_ENTROPY gate drops a 20-char JWT header | `BASE64_SECRET_RE` sits at :451, not :324-360; credential-first pass order in `redactInfraTopology` (:487) and the bare-token-before-URL-pass interplay in `scanRawOutput` (:778) are preserved |+| t_c70f3d94 | confirmed | redactor.ts walk() object loop :1042-1049, array branch :1033-1037; `isSecretKeyName('DB_PASSWORD')` already true | anchor moved from :999-1006; ECS `valueFrom` is an ARN reference, out of scope |+| t_796c1a9b | confirmed | write-batch.ts:291/322/393, no redactor import in create-note.ts, `pageLintField` precedent notes-tools.ts:457-463 | paths are reported, never scrubbed, in the structured walk; the warning is non-blocking and advised onto the write receipt like page-lint |+| t_9b30f48b | confirmed | vault.ts:753/:812 bare `continue;` filters; batch-plan.ts:517 extension drop in `collectIngestible`; degradation.ts:47-120 | the drop exists in a second walk (`collectIngestible`); the design states which population is covered |+| t_09d52938 | confirmed, narrowed | extractable-gate.ts:29-32 free-text `reason`; session lane already has a dry run with counters (sessions/import.ts) | no new preview; the new ground is typing `SkippedPage.reason` plus aggregate counters on the source-ingest lane only; drop/cutoff reasons do not exist here |+| t_a77e35a2 | confirmed | fs-atomic.ts:35/:136/:151-158, no production caller passes `skipIfUnchanged`; write-batch.ts:594 and vault.ts:477 are the two unguarded chokepoints | validator-trimmed scope: inline-rewrite and log.ts already self-guard; no fs-atomic default flip; `updated: true` hardcoded at write-batch.ts:535/:569 must become honest |+| t_541083b3 | confirmed | lifecycle.ts:52 `PRAGMA journal_mode = WAL` on both open paths; vault-backing.ts:181-183 maps nfs/cifs/fuse to `durable`, no remoteness axis | lands as skip WAL + warn on remote backing, no hard refuse; ownership anchor drift corrected to :774/:777; WSL 9p false-positive risk respected |+| t_f99250aa | confirmed | doctor.ts:139 `DOCTOR_CHECKS`; `embeddingSunsetCheck` (doctor/embedding-sunset-check.ts:261) is the precedent | target is the brain doctor registry, not the install manifest doctor; `computeSemanticHealth` is rule-set-meaning health, not the embeddings lane |+| t_6cc80627 | confirmed | signal.ts:134/:479/:683 `session_ref` parsed and rendered, never resolved; repair-lane.ts:52-62 confirm-phrase/cap discipline | the concrete orphan is an unresolved `session_ref`; the repairable-vs-informational split is an additive optional field on `DoctorIssue` |++No card was refused this wave.++## Scope++- Egress redaction: widen the URL-credential password class within the documented bounded/linear regex+ constraint, and add a default-on three-segment JWT detector next to the existing `BEARER_RE` pass in+ `scanRawOutput`.+- Structured redaction: the walk inspects plain-object array elements for `{name, value}` shape and+ applies `isSecretKeyName` to the sibling `name`.+- Write receipts: a non-blocking absolute-path advisory, computed with the existing hygiene grammar,+ riding the write result like page-lint, covering both MCP write surfaces through the write-batch+ projection.+- Ingest visibility: typed `SkippedPage.reason` (closed union, membership list, narrowing parser,+ checkable detail) and aggregate per-reason counters plus per-extension unclassifiable counts on the+ source-ingest lane, surfaced through the one wire serializer and the CLI verb.+- mtime stability: `skipIfUnchanged` opt-in at exactly the two unguarded chokepoints; skipped writes+ store no before-image, record no note-write record, and report `updated`/`appended` honestly.+- Journal mode: a remoteness axis on the vault-backing probe; on a remote backing the index opens+ without WAL and warns once, non-Linux stays silent.+- Brain doctor: a read-only failSoft embeddings-health check consolidating the scattered signals, and a+ read-only failSoft orphan-observation check resolving `session_ref`; a repair verb for orphaned+ observations under the repair-lane discipline, plus an additive optional `fix` field on `DoctorIssue`.++## Out of scope++- The two generic vault walks (`walkBasenames`/`walk` in src/core/vault.ts) keep their silent+ extension filters: they serve listing surfaces where exclusion is expected behavior; only the+ source-ingest discovery lane gets counters.+- No new preview command for the ingest lane; the session lane already has one.+- A metrics JSONL surface for skipped extensions; the wire serializer and CLI render are the surfaces+ for this wave.+- No new member in the closed `DEGRADATION_CODE` vocabulary; the unclassifiable counts travel on the+ batch plan and its one serializer, not as per-entry notices.+- ECS `valueFrom` sibling handling (ARN reference, not a literal secret).+- A hard refusal to open the index on remote backing, and any escape-hatch env for one.+- The maintenance lease database (`src/core/brain/maintenance/lease.ts:44`) keeps unconditional WAL;+ the remoteness gate lands on the search index open path only.+- No fs-atomic default flip; the roughly thirty unrelated call sites stay untouched.+- No change to the structured walk's path-like-leaf decision (reported, never rewritten).++## Chosen approach++Variant 2 from the consultant review (see variants.md): keep the nine changes anchor-local, but factor+the three natural pairs onto one seam each. The redactor pair (JWT plus slash-password, sibling+`{name, value}`) lands as two passes in one module with one shared-file sequence. The ingest pair+(unclassifiable counts, typed skip taxonomy) shares one typed taxonomy whose aggregates carry the+unclassifiable bucket, emitted once through `serializeBatchPlan` so the CLI JSON and MCP surfaces+cannot diverge. The doctor pair (embeddings health, orphan observations) shares the check shape of+`embeddingSunsetCheck` and the additive `fix` field. The three singletons - write-path advisory, mtime+gate, WAL remoteness - stay strictly local to their anchors. The write-path advisory and the mtime gate+both touch `src/core/brain/write-batch.ts` and run as an explicit shared-file sequence. Everything is+additive and byte-identical-when-absent, no default flips, no new dependencies.++## Design decisions++- **Redaction pass order is preserved.** The bare-token pass runs before the URL-credential pass in+ `scanRawOutput`; widening the password class to `[^\s@]+` keeps the scheme and `@host` anchors and+ the bounded-repetition no-ReDoS constraint documented at redactor.ts:236-239. A vendor-token-shaped+ password is collapsed by the earlier pass, which is the existing ordering the design keeps.+- **JWT detection is default-on and text-lane only.** A three-segment base64url JWT detector joins+ `BEARER_RE` in `scanRawOutput`; the structured walk's decision to report rather than rewrite+ path-like leaves is unchanged, so a JWT-shaped string inside a path-like leaf stays a report. This+ changes the default byte-parity of egress output (continuity payloads flag it via `redacted`); that+ visibility is the point of the card, and the affected tests pin the new behavior.+- **The sibling-pair check reuses the existing matcher.** `isSecretKeyName` already matches+ `DB_PASSWORD`; the walk change consults it on the sibling `name` of a plain-object `{name, value}`+ array element. No new secret-name vocabulary.+- **The path advisory reuses the hygiene grammar as-is.** `scanText` from+ src/core/hygiene/hardcoded-paths.ts (unix-home and windows-home detectors, placeholder-segment+ filter, `hygiene:allow-path` escape hatch) is the single grammar; its home-prefix scope is the honest+ leak case and is stated in the advisory rather than widened here. The advisory is computed in the+ write-batch projection over authored note content only (create, update, append; log-line ops skip,+ following page-lint's authored-content rule), carried as an optional byte-identical-when-absent field+ on the write result, and is never a `WriteBatchError` - the batch stays all-or-nothing. Findings echo+ vault-relative paths, line numbers and detector names, never absolute paths (page-lint's+ identifiers-and-integers rule).+- **The mtime gate is chokepoint-level opt-in.** `commitNoteRewrite` passes `skipIfUnchanged: true` and+ branches on the returned wrote boolean to skip `storeBeforeImage` and `recordNoteWrite`; the update+ commit reports `updated: <wrote>` and the append commit `appended: <wrote>`. The+ `writeFrontmatterAtomic` overwrite branch opts in and returns the boolean additively; exclusive-create+ branches are untouched. No audit line exists for a write that did not happen.+- **Remoteness is a second axis, and the response is skip plus warn.** `vault-backing.ts` gains a+ remoteness classification alongside durability: named network magics (nfs, cifs) classify remote;+ fuse stays non-remote because its backing is unknowable and failing safe there would warn every+ user-space mount; unknown and probe-unsupported stay silent and keep WAL. `applyPragmas` consults the+ axis on both open paths and sets the default journal mode (DELETE) instead of WAL on a remote backing,+ emitting one stderr warning naming the filesystem and the reason, in the style of the existing+ restore-from-bak notice. The new classification vocabulary registers in the verdict-vocabulary census.+ The probe seam on the open path is an optional injectable statfs probe so tests can simulate NFS+ without a real mount.+- **Doctor checks are read-only, failSoft, and shaped like the sunset check.** The embeddings-health+ check reads the pending-vector census (preserving its unrecorded-is-not-zero contract), the index+ counts including `staleEmbeddings`, and the provider/vec-extension facts, emits warning issues with a+ message-embedded next command (`o2b search vector-backfill`, `o2b search check`), and sends an+ unrecorded census to the uncertain stream. The orphan-observation check parses signals and resolves+ `session_ref` against the sessions/continuity store; unreadable subtrees go to the uncertain stream.+ Both register in `DOCTOR_CHECKS` with codes in the diagnostics registry and the census tests.+- **The orphan repair detaches, never deletes.** The repair verb clears the dangling `session_ref` from+ a signal's frontmatter, keeping the observation content: dry-run default, exact confirm phrase, hard+ per-run write cap, idempotent rescan, vault-identity assert, atomic frontmatter writes, and never+ reachable from `runDoctor`. The repairable-vs-informational split is an optional additive `fix` field+ on `DoctorIssue` (mirroring the general doctor's `CheckResult.fix`), byte-identical-when-absent so+ existing JSON consumers and drift censuses stay green.++## File changes++New files:+- src/core/brain/write-path-advisory.ts (advisory over authored note content, wrapping the hygiene grammar)+- src/core/brain/doctor/embeddings-health-check.ts+- src/core/brain/doctor/orphan-session-check.ts+- src/core/brain/link-graph/orphan-repair.ts (repair-lane discipline applied to detached session references)+- src/cli/brain/verbs/orphan-repair.ts++Modified files:+- src/core/redactor.ts (password class, JWT pass, sibling-pair walk)+- src/core/brain/write-batch.ts (advisory field on the write result; skipIfUnchanged branch and honest updated flags)+- src/mcp/brain/notes-tools.ts, src/mcp/brain/write-batch-tools.ts (advisory pass-through on receipts)+- src/core/vault.ts (writeFrontmatterAtomic overwrite branch)+- src/core/brain/ingest/extractable-gate.ts (typed reason, membership, narrowing parser)+- src/core/brain/ingest/batch-plan.ts (per-reason and per-extension aggregates)+- src/mcp/brain/ingest-tools.ts, src/cli/brain/verbs/batch-plan.ts (serializer and render)+- src/core/vault-backing.ts (remoteness axis), src/core/search/store/lifecycle.ts (journal-mode decision)+- src/core/brain/doctor.ts (two registry entries), src/core/brain/types.ts (optional `fix` field)+- src/core/brain/diagnostics.ts (new codes; `parseSignal` is already exported for the orphan check)+- docs touched per house rules in the PR phase (mcp.md, cli-reference.md, observability.md as needed)++Test files mirror these under tests/ (redactor, redactor-scan-window, mcp/brain-write-batch,+core/brain/ingest/*, core/vault-backing, core/search store lifecycle, core/brain/doctor/*,+core/brain/link-graph/*, cli repair verb), plus the verdict-vocabulary and doctor censuses.++## Risks and open questions++- Redaction byte-parity: the default-on JWT pass and the wider password class change export and+ continuity output; tests that pin current verbatim leakage must be updated deliberately, and the+ EGRESS_REDACTION_NOTICE volume may grow. Bounded-repetition review is mandatory for any widened class.+- Wire compatibility on the ingest lane: typing `SkippedPage.reason` changes the serialized reason+ values; the narrowing parser accepts both the token and the legacy sentence, and every consumer in+ this repository is updated in the same commit.+- Audit-chain consumers: tests or callers asserting `updated: true` on a byte-identical update will+ move to `updated: false`; the honest flag is the deliverable, and the assertion updates are part of+ the task.+- mtime consumers (recency boost, validity-window fallback, vault-delta counts) change behavior on+ no-op reruns by design; any test pinning inflated counts moves with the task.+- WSL and overlay filesystems: the remoteness axis classifies only named network magics; a WSL 9p mount+ does not classify remote, so no WSL user sees the warning or loses WAL. If a future magic list grows,+ the census test is the gate.+- Doctor contract: the two new checks must stay failSoft and register their codes, or the exit census+ and drift tests fail; that failure is the designed guardrail, not a risk to work around.diff --git a/docs/brainstorm/p4-silent-failure-hardening/plan.md b/docs/brainstorm/p4-silent-failure-hardening/plan.mdnew file mode 100644index 00000000..621d6c79--- /dev/null+++ b/docs/brainstorm/p4-silent-failure-hardening/plan.md@@ -0,0 +1,183 @@+# P4 silent-failure hardening - implementation plan++Nine TDD tasks in five file-disjoint lanes. Tasks inside one lane share files and run as an explicit+sequence in the order given; lanes are independent of each other and can be implemented in any order+between lanes. Every task starts with the named failing test, implements until it passes, refactors,+runs the formatter and linter, and lands exactly one atomic conventional commit.++Shared-file sequences:+- Redactor lane: Task 1 then Task 2 (both edit `src/core/redactor.ts`).+- Write lane: Task 3 then Task 6 (both edit `src/core/brain/write-batch.ts`).+- Ingest lane: Task 4 then Task 5 (both edit `src/core/brain/ingest/batch-plan.ts`,+ `src/mcp/brain/ingest-tools.ts`, `src/cli/brain/verbs/batch-plan.ts`).+- Doctor lane: Task 8 then Task 9 (both edit `src/core/brain/doctor.ts`; Task 9 also adds the+ `fix` field Task 8's issue shape documents).+- Standalone lane: Task 7 (no shared files with any other task).++## Tasks++### Task 1: Redaction wire formats - slash-password URLs and bare JWTs+- **Files**: src/core/redactor.ts, tests/core/redactor.test.ts+- **Failing test first**: in tests/core/redactor.test.ts - (a) a URL whose password contains `/`+ (`postgres://admin:s3cr3t/[email protected]:5432/prod`) has both user and password redacted under+ default options and under `redactUrlCredentials: true`, keeping the scheme and host; (b) a bare+ three-segment JWT (`eyJ...hdr.eyJ...pld.sig`) is redacted under default options; (c) a+ Bearer-prefixed token stays redacted exactly as today (regression guard); (d) the slash-free control+ URL redacts as before.+- **Module**: src/core/redactor.ts - widen the `BASIC_AUTH_URL_RE` password class to `[^\s@]+` keeping+ the scheme and `@` anchors and bounded repetition; add a default-on three-segment base64url JWT+ detector pass in `scanRawOutput` next to the `BEARER_RE` pass. Preserve the existing pass order+ (bare-token pass before the URL-credential pass).+- **Acceptance**: the new tests pass; the existing redactor suites (redactor-base64-secret,+ redactor-scan-window, redactor-yaml-structure) stay green; continuity and egress-guard caller tests+ that pinned verbatim leakage are updated in the same commit.+- **Depends on**: none+- **Commit**: `fix(redaction): close the slash-password and bare JWT gaps in egress redaction`++### Task 2: Structured redaction - sibling {name, value} secret pairs+- **Files**: src/core/redactor.ts, tests/core/redactor-scan-window.test.ts+- **Failing test first**: in tests/core/redactor-scan-window.test.ts -+ `redactStructured({environment: [{name: "DB_PASSWORD", value: "hunter2s3cret"}]})` returns the value+ redacted with `redacted: true`, also under `redactTokens: true`; a `{name, value}` element whose name+ is not a secret key passes through; an ECS `valueFrom` ARN reference passes through untouched;+ non-object array elements are unaffected.+- **Module**: src/core/redactor.ts - in the walk() array branch (:1033-1037), inspect plain-object+ elements for a `{name, value}` shape and reuse `isSecretKeyName` (:162) on the sibling `name` when+ walking the value. No new secret-name vocabulary.+- **Acceptance**: the new tests pass; export-identifier-integrity tests stay green.+- **Depends on**: Task 1 (shared file src/core/redactor.ts, strict sequence)+- **Commit**: `fix(redaction): redact sibling name-value secret pairs in the structured walk`++### Task 3: Write receipt - non-blocking absolute-path advisory+- **Files**: src/core/brain/write-path-advisory.ts (new), src/core/brain/write-batch.ts,+ src/mcp/brain/notes-tools.ts, src/mcp/brain/write-batch-tools.ts,+ tests/core/brain/write-path-advisory.test.ts (new), tests/mcp/brain-write-batch.test.ts+- **Failing test first**: (a) new tests/core/brain/write-path-advisory.test.ts - the advisory scans+ authored note content and returns vault-relative page, line number and detector for a body containing+ `/home/alice/notes/x.md`, honors the `hygiene:allow-path` marker, and reports nothing for a clean+ body; (b) tests/mcp/brain-write-batch.test.ts - an update writing a body with an absolute home path+ yields a receipt carrying the advisory field, the batch still applies (non-blocking), and a clean+ write carries no advisory key at all (byte-identical-when-absent).+- **Module**: the advisory wraps `scanText` from src/core/hygiene/hardcoded-paths.ts; the write-batch+ projection computes it over authored note content only (create, update, append; log-line ops skip)+ and carries it as an optional field on the write result; the two MCP tools pass the field through on+ their receipts. Never a `WriteBatchError`; findings echo identifiers and integers, never absolute+ paths.+- **Acceptance**: the new tests pass; existing write-batch and notes-tools tests stay green.+- **Depends on**: none (first task of the write lane)+- **Commit**: `feat(write): advise absolute paths on the note write receipt`++### Task 4: Ingest lane - typed skip taxonomy and per-reason counts+- **Files**: src/core/brain/ingest/extractable-gate.ts, src/core/brain/ingest/batch-plan.ts,+ src/mcp/brain/ingest-tools.ts, src/cli/brain/verbs/batch-plan.ts,+ tests/core/brain/ingest/extractable-gate.test.ts, tests/core/brain/ingest/batch-plan.test.ts+- **Failing test first**: (a) extractable-gate.test.ts - a skipped page carries a typed reason token+ from a closed membership list plus a `detail` holding the schema_type value;+ `isSkippedPageReason` accepts the token and the legacy sentence and rejects an unknown string;+ (b) batch-plan.test.ts - the plan exposes per-reason counts and `serializeBatchPlan` emits them only+ when non-empty (byte-identical-when-absent for a plan with no skips).+- **Module**: extractable-gate.ts types `SkippedPage.reason` following the `PAGE_LINT_SKIP_REASON`+ pattern (token const, membership list, narrowing parser, checkable detail); batch-plan.ts aggregates+ skips per reason; ingest-tools.ts serializes; the CLI verb renders the counts. The discovered set,+ filter chain and `planId` are untouched.+- **Acceptance**: the new tests pass; the wire tests in tests/mcp/ingest-tool-ignore-warnings.test.ts+ are updated for the typed reason in the same commit.+- **Depends on**: none (first task of the ingest lane)+- **Commit**: `feat(ingest): type the skip reasons and count them per reason`++### Task 5: Ingest lane - unclassifiable-file counts+- **Files**: src/core/brain/ingest/batch-plan.ts, src/mcp/brain/ingest-tools.ts,+ src/cli/brain/verbs/batch-plan.ts, tests/core/brain/ingest/batch-plan.test.ts+- **Failing test first**: a vault containing a `.png` and a `.csv` beside ingestible pages yields a+ plan whose unclassifiable aggregate reports the total 2 and per-extension counts+ `{".png": 1, ".csv": 1}`; a vault with none carries no such key on the wire; `planId` is unchanged by+ the counting; the CLI render shows the counts.+- **Module**: `collectIngestible` (:502-522) counts files dropped for a non-ingestible extension+ (hidden-dot entries and ignore-rule matches are not counted; `DEFAULT_SKIP_DIRS`/`DEFAULT_SKIP_FILES`+ noise stays out); `WalkContext` carries the counters; `BatchPlan` exposes the aggregate and+ `serializeBatchPlan` emits it once for both surfaces. No new preview, no new `DEGRADATION_CODE` in+ this task, no change to the generic vault walks in src/core/vault.ts.+- **Acceptance**: the new tests pass; the scoping and git-discovery batch-plan suites stay green.+- **Depends on**: Task 4 (shared ingest-lane files, strict sequence)+- **Commit**: `feat(ingest): count unclassifiable files dropped by extension`++### Task 6: mtime stability - skipIfUnchanged at the two chokepoints+- **Files**: src/core/brain/write-batch.ts, src/core/vault.ts, tests/mcp/brain-write-batch.test.ts,+ tests/core/vault.test.ts+- **Failing test first**: (a) tests/mcp/brain-write-batch.test.ts - re-applying a byte-identical update+ leaves the note's mtime unchanged, reports `updated: false`, and records no new before-image and no+ new note-write record; a real change still bumps mtime and reports `updated: true`; an append with a+ fresh body reports `appended: true`; (b) tests/core/vault.test.ts - calling+ `writeFrontmatterAtomic` with overwrite on identical content leaves mtime unchanged, and a changed+ content still rewrites.+- **Module**: `commitNoteRewrite` (:584-602) passes `skipIfUnchanged: true` to `atomicWriteFileSync`,+ branches on the returned boolean to skip `storeBeforeImage` (:593) and `recordNoteWrite` (:595); the+ update commit (:535) reports `updated: <wrote>` and the append commit (:569) `appended: <wrote>`.+ The `writeFrontmatterAtomic` overwrite branch (:475-479) opts in and returns the boolean additively;+ exclusive-create branches are untouched; no fs-atomic default flip.+- **Acceptance**: the new tests pass; the fs-atomic primitive suite (tests/core/fs-atomic.test.ts) is+ untouched and green; audit-chain and active-note tests stay green.+- **Depends on**: Task 3 (shared file src/core/brain/write-batch.ts, strict sequence)+- **Commit**: `fix(write): stop phantom mtime bumps on byte-identical rewrites`++### Task 7: Journal mode - remoteness axis, skip WAL and warn on remote backing+- **Files**: src/core/vault-backing.ts, src/core/search/store/lifecycle.ts,+ tests/core/vault-backing.test.ts, tests/core/search/store-remoteness.test.ts (new),+ tests/core/architecture/verdict-vocabulary-census.test.ts+- **Failing test first**: (a) vault-backing.test.ts - the probe classifies nfs (`0x6969`) and cifs+ (`0xff534d42`) backing as remote on the new axis; fuse, ext4, tmpfs classify non-remote;+ probe-unsupported and fs-type-unknown stay silent and non-remote; (b) new+ tests/core/search/store-remoteness.test.ts - with an injected remote-classifying probe,+ `openWriteDatabase` opens with journal mode DELETE (read back via `PRAGMA journal_mode`), emits one+ stderr warning naming the filesystem, and the vault data is untouched; with the default local+ classification the journal mode stays WAL; (c) the census test fails until the new classification+ vocabulary is registered.+- **Module**: vault-backing.ts gains the remoteness axis beside durability (named network magics+ remote; fuse non-remote with the rationale documented; unknown/probe-unsupported silent);+ `applyPragmas` (:51-60) takes the journal-mode decision from the probe result - a pure decision+ function plus an optional injectable statfs probe parameter on the write-open path are the test seam.+ No refusal, no env gate, no change to the maintenance lease database.+- **Acceptance**: the new tests pass; the existing vault-backing and store-integrity suites stay green.+- **Depends on**: none (standalone lane)+- **Commit**: `fix(search): keep WAL off remote-backed index files and say so`++### Task 8: Brain doctor - embeddings-health check+- **Files**: src/core/brain/doctor/embeddings-health-check.ts (new), src/core/brain/doctor.ts,+ src/core/brain/diagnostics.ts, tests/core/brain/doctor/embeddings-health-check.test.ts (new),+ tests/core/brain/doctor-exit-census.test.ts+- **Failing test first**: in the new embeddings-health-check.test.ts - a healthy embedded index emits+ no issue; a census with pending or stale embeddings emits a warning whose message embeds+ `o2b search vector-backfill` and `o2b search check`; an unrecorded census emits to the uncertain+ stream and never reports a failed read as healthy; a throwing probe degrades to uncertain without+ failing the doctor run (failSoft); the exit-census test fails until the code is registered.+- **Module**: the check reads `readPendingVectorCensus` (indexer.ts:1706) and the index-count surface+ (:1280/:1406), uses `ctx.now`, and is shaped on `embeddingSunsetCheck` (:261); register in+ `DOCTOR_CHECKS` (doctor.ts:139) and add the code to the diagnostics registry. Strictly read-only.+- **Acceptance**: the new tests pass; doctor-drift and doctor-exit-census stay green.+- **Depends on**: none (first task of the doctor lane)+- **Commit**: `feat(doctor): consolidate embeddings health into the brain doctor`++### Task 9: Orphan observations - session_ref check and repair verb+- **Files**: src/core/brain/doctor/orphan-session-check.ts (new), src/core/brain/link-graph/orphan-repair.ts (new),+ src/cli/brain/verbs/orphan-repair.ts (new), src/core/brain/doctor.ts, src/core/brain/types.ts,+ tests/core/brain/doctor/orphan-session-check.test.ts (new), tests/core/brain/link-graph/orphan-repair.test.ts (new),+ tests/cli/brain-orphan-repair.test.ts (new), tests/core/brain/doctor-exit-census.test.ts+- **Failing test first**: (a) orphan-session-check.test.ts - an observation signal whose `session_ref`+ resolves to no session or continuity record emits a warning issue carrying the new optional `fix`+ field with the exact repair command; a resolvable `session_ref` emits nothing; an unreadable signals+ subtree goes to the uncertain stream; (b) orphan-repair.test.ts - dry-run default writes nothing and+ lists the detach actions; apply requires the exact confirm phrase; a run stops at the hard write cap;+ a rescan after apply is idempotent; (c) the CLI test proves `runDoctor` never repairs and the verb+ never runs from the doctor.+- **Module**: the check parses signals with `parseSignal` (signal.ts:683) and resolves `session_ref`+ (signal.ts:134/:479) against the sessions/continuity store, shaped on `danglingWorkrunCheck`+ (store-integrity.ts:62); `DoctorIssue` gains an optional additive `fix` field (mirroring the general+ doctor's `CheckResult.fix` at src/core/types.ts:27), byte-identical-when-absent; the repair module+ reuses the repair-lane discipline (REPAIR_CONFIRM_PHRASE-style exact phrase, hard per-run write cap,+ idempotent rescan, vault-identity assert, atomic frontmatter writes) and the repair action detaches+ the dangling reference from the signal frontmatter, keeping the observation content.+- **Acceptance**: the new tests pass; doctor-exit-census, doctor-drift and the repair-lane suites stay+ green with the new codes registered.+- **Depends on**: Task 8 (shared file src/core/brain/doctor.ts and the `fix` field Task 8 documents,+ strict sequence)+- **Commit**: `feat(brain): detect and repair orphaned session references on observations`diff --git a/docs/brainstorm/p4-silent-failure-hardening/variants.md b/docs/brainstorm/p4-silent-failure-hardening/variants.mdnew file mode 100644index 00000000..8070fb84--- /dev/null+++ b/docs/brainstorm/p4-silent-failure-hardening/variants.md@@ -0,0 +1,59 @@+# P4 silent-failure-hardening - variant audit trail++Consultant: primary CLI consultant (claude), phase 0 of the feature-release playbook. The+consultant prompt is cli-output/prompt.md; the verbatim session output is cli-output/claude.md.+All three variants are reproduced below verbatim, followed by the orchestrator decision.++## Variants (verbatim)++### Variant 1: Nine local patches at the verified anchors+- **Approach**: Each card lands as an isolated change at its named anchor with card-local types and tests: two redactor passes, one walk tweak, one receipt field, two independent skip counters, two `skipIfUnchanged` call sites, one pragma gate, two doctor checks plus one repair verb. Cards share nothing new; they only register into the vocabularies that already exist (`DEGRADATION_CODE`, `DOCTOR_CHECKS`, the verdict census).+- **Trade-offs**:+ - Pro: every card is independently reviewable and revertable; nine small PR-sized diffs fold cleanly into one squash.+ - Pro: fully parallelizable across subagents with no shared file contention beyond the census test.+ - Pro: lowest chance of accidentally widening a public shape, since no new shared type crosses a wire contract.+ - Con: cards 4 and 5 both count ingest skips on the same BatchPlan lane and would ship two counter shapes and two serializer touches for one concept.+ - Con: cards 8 and 9 each hand-roll census plumbing and an uncertain stream, duplicating what `embeddingSunsetCheck` already does.+ - Con: naming drift across nine authors (reason strings, warning codes) is likely and only caught at review.+- **Complexity**: medium+- **Risk**: low++### Variant 2: Local patches with three shared seams where cards pair up+- **Approach**: Keep the nine anchors, but factor the three natural pairs onto one seam each: cards 4 and 5 share a typed ingest skip taxonomy (`SkippedPage.reason` membership list plus parser) whose per-reason counters also carry the unclassifiable-extension bucket, emitted once through `serializeBatchPlan`; cards 8 and 9 share a small read-only census helper modeled on `embeddingSunsetCheck` (probe, count, uncertain stream, embedded next command) and register two thin checks on top; cards 1 and 2 extend the redactor as one detector-table change with one census entry set. Cards 3, 6 and 7 stay strictly local, with 7 adding only a remoteness axis to `vault-backing.ts` that `applyPragmas` consults.+- **Trade-offs**:+ - Pro: one skip vocabulary and one serializer touch for the whole ingest lane, so CLI JSON and MCP cannot diverge.+ - Pro: doctor checks stay uniform in shape and failure isolation, and the repair verb for card 9 reuses the repair-lane discipline unchanged.+ - Pro: still additive and byte-identical-when-absent everywhere; no default flips; no new dependencies.+ - Con: the paired cards become sequential (taxonomy before counters, helper before checks), reducing parallelism for two of the nine.+ - Con: a shared helper is a small new internal surface that needs its own tests beyond the per-card ones.+ - Con: slightly larger diff on `batch-plan.ts` and `doctor.ts` than pure local patches.+- **Complexity**: medium+- **Risk**: low++### Variant 3: Unified silent-failure ledger+- **Approach**: Introduce one cross-cutting notice primitive (code, count, surface, severity) with a single serializer, and make all nine cards producers into it: redaction hits, sibling-pair hits, absolute-path advisories, skipped files and reasons, short-circuited rewrites, WAL downgrade, and both doctor findings. Doctor checks then become readers of the ledger rather than independent scanners, and every CLI verb and MCP tool emits the same notice block.+- **Trade-offs**:+ - Pro: one place to look for everything that degraded silently, fully aligned with the v1.52.0 principle.+ - Pro: future cards plug in with zero new wire plumbing.+ - Con: touches every wire contract in the wave at once and makes byte-identical-when-absent hard to prove across CLI JSON and MCP.+ - Con: the write-receipt, batch-plan and doctor surfaces have different lifetimes; forcing them through one ledger blurs read-only doctor checks with write-path advisories.+ - Con: the redactor is a hot egress path and should not gain a ledger dependency; the mtime-skip audit result already has its own `updated` semantics.+ - Con: far exceeds the verified, narrowed scope of the nine premises.+- **Complexity**: large+- **Risk**: high++### Recommended: Variant 2+**Rationale**: The premise verification already narrowed each card to a specific anchor, so the wave should stay anchor-local, but three pairs (4/5, 8/9, 1/2) genuinely describe one concept each, and shipping them as separate shapes would violate the one-serializer-per-wire-contract and closed-vocabulary conventions on day one. Variant 2 removes that duplication with two small internal seams and one detector-table change, keeps every change additive and opt-in, and preserves independent review for the six cards that do not overlap. Variant 3 is the right long-term shape but is a scope change the operator did not pick.++## Orchestrator decision++Chosen: Variant 2, agreeing with the consultant recommendation. No override. The pairing matches+project context the consultant could weight only partially: the one-wire-serializer rule enforced by+the byte-identical-when-absent tests, the closed-vocabulary census gate, and the repair-lane+discipline that card 9 must slot into unchanged. Two refinements at design time, both recorded in+design.md: the redactor pair stays two atomic tasks in one shared-file sequence rather than one+detector-table change (each pass keeps its own failing test and revertable commit), and the doctor+pair shares a check shape and an additive `fix` field rather than a new census helper module, since+`embeddingSunsetCheck` already is that shape. Variant 1 was rejected for the duplicated counter+shapes on the ingest lane; Variant 3 was rejected as a scope change beyond the nine verified+premises.diff --git a/docs/cli-reference.md b/docs/cli-reference.mdindex 0024bb8b..ecd0f2bb 100644--- a/docs/cli-reference.md+++ b/docs/cli-reference.md@@ -717,6 +717,7 @@ o2b brain telegram-capture run | catchup - explicit inbound capture runner (feo2b brain inbox-drain [--apply] - walk staged captures, classify each structurally (URL-shaped body -> source reference, explicit obligation marker -> task, otherwise atomic idea), route on apply (source ingest, note in captured/, obligation open), archive processed captures, and report every item with action and reason; dry-run default writes nothing; rerun after apply is a no-opo2b brain diarize <entity> [--json] - entity profile skeleton plus one needs-llm-step envelope; the stated-vs-evidenced section is computed deterministically (stated claims versus evidence frequency and recency), every line carrying an evidence identity; unknown entity is a typed erroro2b brain repair-lane [--apply --confirm "apply repair"] - propose link-graph edges ordered by identity strength (explicit references, session continuity, same-topic evidence; inferred opt-in) under a confidence threshold and a hard per-run write cap; dry-run default; reruns after apply converge to zero writes. --apply plans first and then runs the paired graph-efficacy holdout harness over every edge that plan proposes, taking anchor and target from the edge itself: it reports graph lift (targets reachable only through the graph) apart from direct recall (targets already a 1-hop neighbour) and refuses the apply, writing nothing, when any target resolves to no durable-memory note or hydrates into no evidence. The refusal names the failing edges with their verdict and resolves its exit through the registered repair-holdout-unresolved diagnostic; --json carries the counts in an additive holdout object beside next_command. Dry-run does not evaluate the gate and its bytes are unchanged - it already names each unresolvable endpoint as a skip-missing-target decision+o2b brain orphan-repair [--apply --confirm "apply orphan repair"] - detach dangling session references from observation signals (the findings `o2b brain doctor` reports as `orphan-session-ref`, each naming this verb on its `fix` field); --apply removes ONLY the `session_ref` key from each signal's frontmatter, keeping the observation body, topic and every other field, and quotes the detached value in the report; dry-run default writes nothing, the exact confirmation phrase is required for apply, a hard per-run write cap bounds the run, and a rerun after apply converges to zero writes; the doctor pass never repairso2b brain design-note <topic> [--payload <json> | --payload-file <path>] [--agent <name>] [--json] - the one-shot sibling of `o2b brain panel`. Without a payload it is read-only: it grounds the topic in the vault's tension records, decision records and truth projections and prints the single needs-llm-step envelope the calling agent answers, naming any store the vault holds nothing in (which is not the same as a store that matched nothing). With a payload it validates the written note and commits it as Brain/decisions/design-<date>-<topic>.md. The note must weigh named alternatives and mark EXACTLY ONE recommended: zero and two-plus are both refused, and the refusal states the count. A second note for the same topic on the same day is refused, never overwritteno2b brain skill-proposals page-candidates [--json] - read-only: gate the vault's user pages on the page-meta trio (core tier, non-stale lifecycle, high confidence) and an observed-reuse floor, skip any page an installed skill already covers, and return one needs-llm-step envelope per admitted page plus every skip with its reasono2b brain skill-proposals page-draft <page> (--payload <json> | --payload-file <path>) [--json] - validate a returned SKILL.md draft and stage it as a pending mature_page proposal INSIDE the vault; accept is what materializes the SKILL.md under the configured skills root, through the write-ahead journaldiff --git a/docs/how-it-works.md b/docs/how-it-works.mdindex 2b341284..78c5824f 100644--- a/docs/how-it-works.md+++ b/docs/how-it-works.md@@ -682,7 +682,8 @@ integrity scanner and refuses a condemned store rather than archivingit over a good snapshot, checks the ceiling, then `VACUUM INTO`s atemporary file and compresses it to`Brain/.snapshots/<run_id>.store.sqlite.zst`. `VACUUM INTO` rather than-a file copy because the store runs in WAL mode and the runtime exposes+a file copy because the store is a live SQLite database - WAL mode on a+local backing, DELETE on a remote-backed file - and the runtime exposesno online-backup API. The archive sits **beside** the tar rather thaninside it: the extractor requires a `Brain/` root and the restore isdefined as "live `Brain/` equals archive `Brain/` minus the excluded@@ -1010,7 +1011,12 @@ Key behaviours, all driven from `Brain/_brain.yaml`-free `search_*` /alongside the body.- **Store boundary.** `src/core/search/store.ts` is the single SQLhome; every other module talks to it through a typed surface. WAL- mode for concurrent reads, `proper-lockfile` on the index path for+ mode for concurrent reads - journal mode DELETE instead, announced+ once on the write open, when the index file sits on a remote+ filesystem (an index already in WAL keeps it while another+ connection has the file open, since leaving WAL needs exclusive+ access: the read open stays silent, the write open says so once and+ retries on its next open) - `proper-lockfile` on the index path forwriter exclusivity (three attempts, 1 s backoff, then`INDEX_LOCKED`).- **Embedding providers (v0.36.0).** `embedding_provider` selectsdiff --git a/docs/mcp.md b/docs/mcp.mdindex 7434b802..d7efa5d8 100644--- a/docs/mcp.md+++ b/docs/mcp.md@@ -1560,3 +1560,19 @@ log line is machine-composed rather than authored.confidence threshold; `{ available: false, reason: "decision_model_off" }`while the use is off, and a private note is refused. None of these writes tothe vault, and `suggest` never assigns.+- Since v1.62.0 the ingest plan accounts for what it sets aside. Every+ `skipped_non_extractable` entry carries a `reason` from a closed, typed+ set and a `detail` naming the value behind it (the legacy free-text+ sentence still parses as input); `skipped` stays a list of path strings+ for unchanged files. The plan adds `skip_reason_counts` per reason, and extensionless regular files nothing+ can classify are counted in an `unclassifiable` object (`total`,+ `by_extension`); both keys are absent while empty, so a plan with nothing+ set aside serializes exactly as before. Note receipts gain an optional+ `path_advisory` when authored content mentions an absolute host path -+ absent when the content is clean, and never present for log-line+ operations; the advisory never blocks the write. `brain_doctor` reports+ `orphan-session-ref` warnings whose `fix` field names+ `o2b brain orphan-repair`, and an embeddings-health verdict+ (`embeddings-backlog`, `embeddings-census-unrecorded`,+ `embeddings-health-unmeasured`) that is fail-soft: an unrecorded census+ lands in `uncertain`, never in healthy.diff --git a/docs/observability.md b/docs/observability.mdindex f49e7601..faaa1804 100644--- a/docs/observability.md+++ b/docs/observability.md@@ -64,9 +64,9 @@ Every note mutation funnels through two seams - `createNote` and the write-batchThe device is NOT a payload field: it rides on the log shard name exactly as it does for every other event, and `origin_channel` is stamped by the appender. `listNoteWrites` (`src/core/brain/notes/write-log.ts`) reads the record back and reports the shard as the `device`, the empty string being the legacy un-sharded log rather than an unknown machine. `o2b brain writes` and the `brain_writes` MCP tool are the two surfaces over it, and `brain_agent_query --kind note` answers "which notes did this agent touch" off the same events.-**Order, and the gap the receipt names.** The bytes are written FIRST and the event appended second, so a log failure never fails a write that has already landed - the snapshot-event precedent, with the silence removed. `recordNoteWrite` never throws: it returns `write_id: null` with an `audit_reason`, and every note-write result and MCP receipt carries both. A skipped create (`if_exists: "skip"`) authored no bytes, so it records nothing and carries no `write_id` at all.+**Order, and the gap the receipt names.** The bytes are written FIRST and the event appended second, so a log failure never fails a write that has already landed - the snapshot-event precedent, with the silence removed. `recordNoteWrite` never throws: it returns `write_id: null` with an `audit_reason`, and every note-write result and MCP receipt carries both. A write that authored no bytes - a skipped create (`if_exists: "skip"`) or a byte-identical rewrite - records nothing and carries no `write_id` at all; the receipt reports the skip through the op's own flag (`updated: false` / `appended: false`).-**Before-image store.** The bytes an update or append replaced are kept at `Brain/.state/write-images/<sha256>`, written before the atomic write and skipped when an image of that content already exists - so a note toggled between two states costs two files, not two per write. The store is a `STATE_SURFACES` row inside the snapshot region; `o2b brain writes prune-images [--older-than-days N] [--dry-run]` bounds it by file age, default 30 days (`WRITE_IMAGE_RETENTION_DAYS`). An image is a copy of note content the snapshot region already archives, so removing one loses no content - it only narrows how far back a revert can reach, which a revert plan reports by name.+**Before-image store.** The bytes an update or append replaced are kept at `Brain/.state/write-images/<sha256>`, stored BEFORE the atomic write so a crash or a failed store never leaves the prior bytes replaced without their copy - a byte-identical rewrite is decided first and stores no image, because the prior bytes stay on disk and an image would claim a replace that never happened - and skipped when an image of that content already exists - so a note toggled between two states costs two files, not two per write. The store is a `STATE_SURFACES` row inside the snapshot region; `o2b brain writes prune-images [--older-than-days N] [--dry-run]` bounds it by file age, default 30 days (`WRITE_IMAGE_RETENTION_DAYS`). An image is a copy of note content the snapshot region already archives, so removing one loses no content - it only narrows how far back a revert can reach, which a revert plan reports by name.**Revert.** `o2b brain writes revert (--agent A | --device D | --path P) [--since --until]` reads the same events back and plans what undoing them would do; `brain_writes` action `plan_revert` returns the same plan and never applies it. A selector naming none of agent, device or path is refused by name (`unbounded_selector`): a time window alone selects every write by every agent on every machine, which is a vault rollback and has its own verb. Per target, over the selected writes S (oldest..newest) and every recorded write A on that target:diff --git a/openclaw.plugin.json b/openclaw.plugin.jsonindex d4969e22..415f6336 100644--- a/openclaw.plugin.json+++ b/openclaw.plugin.json@@ -2,7 +2,7 @@"id": "open-second-brain","name": "Open Second Brain","description": "Second brain for AI agents using Obsidian-compatible Markdown vaults.",- "version": "1.61.0",+ "version": "1.62.0","activation": { "onStartup": true },"skills": ["./skills"],"contracts": {diff --git a/openclaw/index.js b/openclaw/index.jsindex 9905307c..91edbe16 100644--- a/openclaw/index.js+++ b/openclaw/index.js@@ -1971,9 +1971,10 @@ var COLON_VALUE_RE = new RegExp(`(?<!")\\b(${KEY_PATTERN})([ \\t]*:[ \\t]*)("[^"var YAML_SECRET_BLOCK_RE = new RegExp(`^([ \\t]*)(${KEY_PATTERN})[ \\t]*:[ \\t]*(?:[|>][+-]?\\d{0,2})?[ \\t]*\\r?\\n` + "(?:\\1[ \\t]+[^\\r\\n]*\\r?\\n?)+", "gim");var JSON_ENTRY_RE = new RegExp(`("(?:${KEY_PATTERN})"\\s*:\\s*)("(?:[^"\\\\]|\\\\.)*"|true|false|null|-?\\d+(?:\\.\\d+)?)`, "gi");var BEARER_RE = /\b(Bearer\s+)([A-Za-z0-9._\-+/=]+)/gi;+var JWT_RE = /\b(?:eyJ|eyA|ewo|ew0|ewk)[A-Za-z0-9_-]{9,65533}(?:\.[A-Za-z0-9_-]{4,65536}){2}(?![A-Za-z0-9_-])/g;var IPV4_OCTET = "(?:25[0-5]|2[0-4]\\d|1\\d\\d|[1-9]?\\d)";var IPV4 = `${IPV4_OCTET}(?:\\.${IPV4_OCTET}){3}`;-var BASIC_AUTH_URL_RE = /\b([a-zA-Z][a-zA-Z0-9+.-]*:\/\/)([^\s/:@]+):([^\s/@]+)@/g;+var BASIC_AUTH_URL_RE = /\b([a-zA-Z][a-zA-Z0-9+.-]*:\/\/)([^\s/:@]*):(?!\d{1,5}\/)([^\s@]+)@/g;var IPV4_PORT_RE = new RegExp(`\\b${IPV4}:\\d{1,5}\\b`, "g");var FQDN_PORT_SOURCE_EXTS = "js|ts|tsx|jsx|py|json|rs|go|java|rb|php|c|cc|cpp|cxx|h|hpp|css|scss|sass|less|" + "html|htm|xml|yaml|yml|toml|ini|cfg|md|markdown|sh|bash|sql|vue|svelte|gradle|" + "kt|swift|scala|clj|ex|exs|erl|elm|dart|lua|pl|pm|r|jl|tf|lock|map|txt|csv|log";var FQDN_PORT_RE = new RegExp("\\b(?:[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?\\.)+(?!(" + FQDN_PORT_SOURCE_EXTS + "):\\d)[a-zA-Z]{2,63}:\\d{1,5}\\b", "g");@@ -2118,6 +2119,7 @@ function scanRawOutput(text, opts = {}) {return `${key}${sep}${PLACEHOLDER}`;});out = out.replace(BEARER_RE, (_match, prefix) => `${prefix}${PLACEHOLDER}`);+ out = out.replace(JWT_RE, PLACEHOLDER);out = out.replace(YAML_SECRET_BLOCK_RE, (_match, indent, key) => `${indent}${key}: "${PLACEHOLDER}"`);out = out.replace(COLON_VALUE_RE, (match, key, sep, value) => {@@ -2178,6 +2180,14 @@ function isPlainContainer(value) {const proto = Object.getPrototypeOf(value);return proto === Object.prototype || proto === null;}+function siblingPairDeclaresSecret(item) {+ if (typeof item !== "object" || item === null)+ return false;+ if (!isPlainContainer(item))+ return false;+ const record = item;+ return "value" in record && isSecretKeyName(record["name"]);+}function redactStructured(input, opts = {}) {let redacted = false;let truncated = false;@@ -2186,6 +2196,18 @@ function redactStructured(input, opts = {}) {if (secretIdentifiers.size < MAX_REPORTED_IDENTIFIERS)secretIdentifiers.add(location);};+ const walkEntries = (source, location, secretKey) => {+ const out = {};+ let index = 0;+ for (const [key, child] of Object.entries(source)) {+ if (keyNameCarriesSecret(key))+ record(`${location === "" ? "" : location}#${index}`);+ const childLocation = location === "" ? key : `${location}.${key}`;+ out[key] = walk(child, childLocation, secretKey(key), isIdentifierKeyName(key));+ index += 1;+ }+ return out;+ };const walk = (value, location, underSecretKey, underIdentifierKey) => {if (underSecretKey) {if (value === null || value === undefined)@@ -2211,21 +2233,18 @@ function redactStructured(input, opts = {}) {return scan.text;}if (Array.isArray(value)) {- return value.map((item, index) => walk(item, `${location}[${index}]`, false, underIdentifierKey));+ return value.map((item, index) => {+ const itemLocation = `${location}[${index}]`;+ if (!siblingPairDeclaresSecret(item)) {+ return walk(item, itemLocation, false, underIdentifierKey);+ }+ return walkEntries(item, itemLocation, (key) => key === "value" ? true : isSecretKeyName(key));+ });}if (typeof value === "object" && value !== null) {if (!isPlainContainer(value))return value;- const out = {};- let index = 0;- for (const [key, child] of Object.entries(value)) {- if (keyNameCarriesSecret(key))- record(`${location === "" ? "" : location}#${index}`);- const childLocation = location === "" ? key : `${location}.${key}`;- out[key] = walk(child, childLocation, isSecretKeyName(key), isIdentifierKeyName(key));- index += 1;- }- return out;+ return walkEntries(value, location, isSecretKeyName);}return value;};diff --git a/package.json b/package.jsonindex 721237f6..59ba534c 100644--- a/package.json+++ b/package.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.61.0",+ "version": "1.62.0","private": false,"description": "Second brain for AI agents using Obsidian-compatible Markdown vaults. Works with Hermes, Claude Code, Codex, and OpenClaw.","keywords": [diff --git a/plugin.yaml b/plugin.yamlindex 4cede0bc..c0d7f985 100644--- a/plugin.yaml+++ b/plugin.yaml@@ -1,5 +1,5 @@name: open-second-brain-version: "1.61.0"+version: "1.62.0"description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."author: "Open Second Brain contributors"memory_provider: truediff --git a/plugins/codex/.codex-plugin/plugin.json b/plugins/codex/.codex-plugin/plugin.jsonindex a81cf5ab..d6c97195 100644--- a/plugins/codex/.codex-plugin/plugin.json+++ b/plugins/codex/.codex-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.61.0",+ "version": "1.62.0","description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.","author": {"name": "Open Second Brain contributors",diff --git a/plugins/hermes/plugin.yaml b/plugins/hermes/plugin.yamlindex 4cede0bc..c0d7f985 100644--- a/plugins/hermes/plugin.yaml+++ b/plugins/hermes/plugin.yaml@@ -1,5 +1,5 @@name: open-second-brain-version: "1.61.0"+version: "1.62.0"description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."author: "Open Second Brain contributors"memory_provider: truediff --git a/pyproject.toml b/pyproject.tomlindex e6e7a89b..073a0618 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -10,7 +10,7 @@ build-backend = "setuptools.build_meta"# CLI entry points (those moved to `package.json` `bin`).[project]name = "open-second-brain"-version = "1.61.0"+version = "1.62.0"description = "Hermes Python shim for Open Second Brain. Most of the project (CLI, MCP server, OpenClaw plugin) is TypeScript on Bun; see package.json."readme = "README.md"requires-python = ">=3.11"diff --git a/src/cli/brain.ts b/src/cli/brain.tsindex 45f26643..2a4df38d 100644--- a/src/cli/brain.ts+++ b/src/cli/brain.ts@@ -144,6 +144,7 @@ import {cmdBrainTelegramCapture,cmdBrainInboxDrain,cmdBrainRepairLane,+ cmdBrainOrphanRepair,cmdBrainSignal,cmdBrainAttentionFlows,cmdBrainSessionDescribe,@@ -493,6 +494,8 @@ export async function handleBrainSubcommand(argv: ReadonlyArray<string>): Promisreturn await cmdBrainInboxDrain(rest);case "repair-lane":return await cmdBrainRepairLane(rest);+ case "orphan-repair":+ return await cmdBrainOrphanRepair(rest);case "session-grep":return await cmdBrainSessionGrep(rest);case "session-describe":diff --git a/src/cli/brain/help-text.ts b/src/cli/brain/help-text.tsindex 84521b0d..39252ad3 100644--- a/src/cli/brain/help-text.ts+++ b/src/cli/brain/help-text.ts@@ -144,6 +144,7 @@ Brain verbs (observing memory):telegram-capture Inbound Telegram capture bot: run (long-poll) | catchupinbox-drain Classify and route staged captures (dry-run; --apply to route)repair-lane Propose memory-graph edges (dry-run; --apply --confirm to write, holdout-gated)+ orphan-repair Detach dangling session references from observations (dry-run; --apply --confirm)session-grep Search imported session recall turns and summariessession-describe Describe an imported session recall DAGsession-expand Expand a session recall node to source turns@@ -1054,6 +1055,17 @@ export const VERB_HELP: Record<string, string> = {"the default and writes nothing; --apply routes each capture and archives\n" +"it, so a rerun is a no-op. Unroutable items are reported and left in\n" +"place. Each item names its action and reason.\n",+ "orphan-repair":+ 'usage: o2b brain orphan-repair [--apply --confirm "apply orphan repair"] [--vault <path>] [--json]\n' ++ "Detach dangling session references from observation signals. A signal whose\n" ++ "session_ref resolves to no session or continuity record is reported by the\n" ++ "doctor under orphan-session-ref with this verb as its fix. Dry-run is the\n" ++ "default and writes nothing; --apply removes ONLY the session_ref key from\n" ++ "each signal's frontmatter and requires the exact confirmation phrase via\n" ++ "--confirm. The observation body, topic and every other field are kept, and\n" ++ "the detached value is quoted in the report. A hard per-run cap bounds the\n" ++ "writes, and a rerun after apply converges to zero writes. The doctor pass\n" ++ "never repairs - it reports and names this verb.\n","repair-lane":'usage: o2b brain repair-lane [--apply --confirm "apply repair"] [--include-inferred] [--vault <path>] [--json]\n' +"Deterministic memory-graph repair lane. Collects candidate edges from\n" +diff --git a/src/cli/brain/verbs/batch-plan.ts b/src/cli/brain/verbs/batch-plan.tsindex b844b7f8..bb19bf41 100644--- a/src/cli/brain/verbs/batch-plan.ts+++ b/src/cli/brain/verbs/batch-plan.ts@@ -105,7 +105,20 @@ export async function cmdBrainBatchPlan(argv: string[]): Promise<number> {}if (plan.skippedNonExtractable.length > 0) {info(` ${plan.skippedNonExtractable.length} non-extractable page(s) skipped:`);- for (const s of plan.skippedNonExtractable) info(` - ${s.path} (${s.reason})`);+ for (const s of plan.skippedNonExtractable) {+ info(` - ${s.path} (${s.reason}: ${s.detail})`);+ }+ // Per-reason aggregate (P4): the taxonomy counted, not just listed.+ const reasonCounts = Object.entries(plan.skipReasonCounts);+ if (reasonCounts.length > 0) {+ info(` by reason: ${reasonCounts.map(([r, n]) => `${r}=${n}`).join(", ")}`);+ }+ }+ if (plan.unclassifiable.total > 0) {+ info(` ${plan.unclassifiable.total} unclassifiable file(s) not planned, by extension:`);+ for (const [ext, n] of Object.entries(plan.unclassifiable.byExtension)) {+ info(` - ${ext === "" ? "(no extension)" : ext}: ${n}`);+ }}if (plan.ignoreWarnings.length > 0) {info(` ${plan.ignoreWarnings.length} ignore warning(s):`);diff --git a/src/cli/brain/verbs/index.ts b/src/cli/brain/verbs/index.tsindex f17749cf..de793c67 100644--- a/src/cli/brain/verbs/index.ts+++ b/src/cli/brain/verbs/index.ts@@ -128,6 +128,7 @@ export { cmdBrainExpire } from "./expire.ts";export { cmdBrainTelegramCapture } from "./telegram-capture.ts";export { cmdBrainInboxDrain } from "./inbox-drain.ts";export { cmdBrainRepairLane } from "./repair-lane.ts";+export { cmdBrainOrphanRepair } from "./orphan-repair.ts";export { cmdBrainSignal } from "./signal.ts";export {cmdBrainSessionDescribe,diff --git a/src/cli/brain/verbs/orphan-repair.ts b/src/cli/brain/verbs/orphan-repair.tsnew file mode 100644index 00000000..0ce08d69--- /dev/null+++ b/src/cli/brain/verbs/orphan-repair.ts@@ -0,0 +1,85 @@+/**+ * `o2b brain orphan-repair` (t_6cc80627): detach dangling `session_ref`+ * values from observation signals, keeping the observations.+ *+ * The only door to the detach repair. Dry-run is the default and writes+ * nothing; --apply requires the exact confirmation phrase via --confirm,+ * checked before anything is scanned so that a refused apply leaves+ * stdout a pure error envelope under --json. A hard per-run write cap+ * bounds the run; a rescan after apply converges to zero writes. The+ * doctor pass reports orphans and names this verb on the issue's `fix`+ * field - it never runs it.+ */++import {+ ORPHAN_REPAIR_CONFIRM_PHRASE,+ OrphanRepairConfirmationError,+ runOrphanRepair,+ type OrphanRepairReport,+} from "../../../core/brain/link-graph/orphan-repair.ts";+import { brainVerbContext, ok, okJson, parse } from "../helpers.ts";+import { fail } from "../../output.ts";++function reportJson(report: OrphanRepairReport): Record<string, unknown> {+ return {+ mode: report.mode,+ detached: report.detached,+ decisions: report.decisions.map((decision) => ({+ path: decision.path,+ session_ref: decision.session_ref,+ action: decision.action,+ })),+ };+}++function renderReport(report: OrphanRepairReport, applied: boolean): void {+ ok(+ `orphan-repair (${report.mode}): ${report.decisions.length} orphan(s) found, ` ++ `${report.detached} reference(s) detached, observation(s) kept`,+ );+ for (const decision of report.decisions) {+ ok(` [${decision.action}] ${decision.path} (${decision.session_ref})`);+ }+ if (!applied && report.detached > 0) {+ ok(` re-run with --apply --confirm ${JSON.stringify(ORPHAN_REPAIR_CONFIRM_PHRASE)} to detach`);+ }+}++export async function cmdBrainOrphanRepair(argv: string[]): Promise<number> {+ const { flags } = parse(argv, {+ vault: { type: "string" },+ apply: { type: "boolean" },+ confirm: { type: "string" },+ json: { type: "boolean" },+ });+ const { vault } = brainVerbContext(flags);+ const asJson = flags["json"] === true;+ const apply = flags["apply"] === true;++ const refuse = (message: string): number => {+ if (asJson) {+ okJson({ ok: false, message });+ return 1;+ }+ return fail(message);+ };++ // The confirmation phrase is a precondition of the invocation, checked+ // before anything is scanned so that a refused apply never even reads+ // the store. `runOrphanRepair` re-checks it below; this is the same+ // typed error, not a second rule.+ if (apply && flags["confirm"] !== ORPHAN_REPAIR_CONFIRM_PHRASE) {+ const message = `${new OrphanRepairConfirmationError().message} (pass --confirm ${JSON.stringify(ORPHAN_REPAIR_CONFIRM_PHRASE)})`;+ return refuse(message);+ }++ let report;+ try {+ report = runOrphanRepair(vault, { apply, confirm: ORPHAN_REPAIR_CONFIRM_PHRASE });+ } catch (exc) {+ return refuse(`orphan repair failed: ${(exc as Error).message ?? String(exc)}`);+ }+ if (asJson) okJson(reportJson(report));+ else renderReport(report, apply);+ return 0;+}diff --git a/src/cli/command-manifest.ts b/src/cli/command-manifest.tsindex 1a75b6a1..d73a244c 100644--- a/src/cli/command-manifest.ts+++ b/src/cli/command-manifest.ts@@ -648,6 +648,11 @@ export const CLI_COMMAND_MANIFEST: CliRootManifest = Object.freeze({flag("include-inferred", "boolean"),],),+ command(+ "orphan-repair",+ "Detach dangling session references from observations (dry-run by default)",+ [flag("vault", "string"), flag("apply", "boolean"), flag("confirm", "string")],+ ),command("session-grep", "Search imported session recall turns and summaries"),command("session-describe", "Describe an imported session recall DAG"),command("session-expand", "Expand a session recall node to its source turns"),diff --git a/src/core/brain/diagnostics.ts b/src/core/brain/diagnostics.tsindex 8adfccc8..a12abc8f 100644--- a/src/core/brain/diagnostics.ts+++ b/src/core/brain/diagnostics.ts@@ -207,6 +207,17 @@ export const DIAGNOSTIC_SIGNALS: ReadonlyMap<string, DiagnosticSignal> = new MapnextCommand: "o2b brain audit",autoRepairable: false,},+ {+ // An observation signal whose session_ref resolves to no session+ // or continuity record. The exit is the detach verb, which is+ // dry-run by default and confirm-gated; `autoRepairable` stays+ // false because the doctor's repair planner has no fixer for it -+ // the repair lives behind the verb the issue's `fix` field names.+ code: "orphan-session-ref",+ issueClass: "observation whose session_ref resolves to no session",+ nextCommand: "o2b brain orphan-repair",+ autoRepairable: false,+ },{code: "broken-backlinks",issueClass: "broken Brain backlink",@@ -555,6 +566,19 @@ export const DIAGNOSTIC_SIGNALS: ReadonlyMap<string, DiagnosticSignal> = new MapnextCommand: "o2b search provider presets",autoRepairable: false,},+ {+ // The index behind its embedding configuration: chunks with no+ // vector, or stored vectors under a retired model/dimension. The+ // registry carries the backfill verb because it is the+ // condition's cheapest first act; the warning prose names the+ // re-embed remedy the stale half needs beside it, since a+ // backfill finds stale rows fully vectorised and leaves them+ // where they are.+ code: "embeddings-backlog",+ issueClass: "indexed chunks with no vector or vectors stale against the configured model",+ nextCommand: "o2b search vector-backfill",+ autoRepairable: false,+ },{code: "git-history-absent",issueClass: "no ingested git history",diff --git a/src/core/brain/doctor-exits.ts b/src/core/brain/doctor-exits.tsindex 5675d35f..ed516508 100644--- a/src/core/brain/doctor-exits.ts+++ b/src/core/brain/doctor-exits.ts@@ -221,6 +221,25 @@ const EXCLUSIONS: ReadonlyArray<DoctorExitExclusion> = ["The first is an edit to a config file whose intended value only the operator knows, and " +"the other two are defects in this build rather than in the vault being examined",},+ {+ code: "embeddings-census-unrecorded",+ reason:+ "the pending-vector count could not be taken: the index file is absent or would not open, " ++ "and the finding says which. What restores it depends on which - an absent index is a " ++ "first-run state an operator reaches deliberately, while a file that will not open is " ++ "replaced, not repaired in place, and replacing it is a decision about the derived copy " ++ "rather than about the vault it indexes. Reporting a count of zero here would read as a " ++ "fully embedded index, which is precisely the failed-read-as-healthy the check refuses",+ },+ {+ code: "embeddings-health-unmeasured",+ reason:+ "the check ran and could observe too little to say anything: the search configuration " ++ "would not resolve, the index probe threw, or the count half of the read failed after the " ++ "census half succeeded. Which half failed changes what restores it, and the finding names " ++ "the one that did - so the honest exit is the reason rather than a command guessed for a " ++ "state nobody observed",+ },{code: "payload-missing",reason:diff --git a/src/core/brain/doctor.ts b/src/core/brain/doctor.tsindex 20828030..e8b0d5e3 100644--- a/src/core/brain/doctor.ts+++ b/src/core/brain/doctor.ts@@ -51,6 +51,7 @@ import { inboxArchivableCheck } from "./doctor/inbox-archive-check.ts";import { capturePatternCheck, configCheck, vaultIgnoreCheck } from "./doctor/config-checks.ts";import { entityRegistryCheck } from "./doctor/entity-checks.ts";import { brokenBacklinkCheck } from "./doctor/link-checks.ts";+import { orphanSessionCheck } from "./doctor/orphan-session-check.ts";import { evidenceRangeCheck, logShardCheck, orphanEvidenceCheck } from "./doctor/log-checks.ts";import {contentHashDriftCheck,@@ -67,10 +68,12 @@ import {} from "./doctor/record-checks.ts";import {collectAllBasenames,- readAllLogRecords,+ readLogSnapshot,+ type LogSnapshot,readAllPreferenceRecords,} from "./doctor/records.ts";import { embeddingSunsetCheck } from "./doctor/embedding-sunset-check.ts";+import { embeddingsHealthCheck } from "./doctor/embeddings-health-check.ts";import { payloadRegistryCheck } from "./doctor/payload-checks.ts";import { recallChannelCoverageCheck } from "./doctor/recall-channel-coverage.ts";import { recoveryPointLivenessCheck } from "./doctor/recovery-point-liveness.ts";@@ -161,6 +164,7 @@ const DOCTOR_CHECKS: ReadonlyArray<DoctorCheck> = Object.freeze([activeBudgetPressureCheck,evidenceRangeCheck,orphanEvidenceCheck,+ orphanSessionCheck,entityRegistryCheck,capturePatternCheck,syncConflictLogCheck,@@ -176,6 +180,7 @@ const DOCTOR_CHECKS: ReadonlyArray<DoctorCheck> = Object.freeze([recallChannelCoverageCheck,recoveryPointLivenessCheck,embeddingSunsetCheck,+ embeddingsHealthCheck,payloadRegistryCheck,]);@@ -380,6 +385,9 @@ function resolveContext(} catch {config = undefined;}+ // Read in the literal's own order below, so the uncertainty stream keeps+ // the order it has always had.+ let logSnapshot: LogSnapshot;return {vault,now: opts.now ?? new Date(),@@ -403,13 +411,14 @@ function resolveContext("subset of the store",uncertain,}),- logs: readAllLogRecords(vault, {+ logs: (logSnapshot = readLogSnapshot(vault, {site: CONTEXT_SITE,consequence:"no log day in it was loaded, so the evidence and orphan lints below report on a subset " +"of the store",uncertain,- }),+ })).records,+ unreadableLogDays: logSnapshot.unreadableDays,};}diff --git a/src/core/brain/doctor/check.ts b/src/core/brain/doctor/check.tsindex 96996eef..e33f565e 100644--- a/src/core/brain/doctor/check.ts+++ b/src/core/brain/doctor/check.ts@@ -10,7 +10,7 @@*/import type { BrainConfig, DoctorIssue } from "../types.ts";-import type { LogRecord, PreferenceRecord } from "./records.ts";+import type { LogRecord, PreferenceRecord, UnreadableLogDay } from "./records.ts";import type { DoctorUncertainEntry } from "./report.ts";/**@@ -26,7 +26,11 @@ export interface DoctorCheckContext {readonly now: Date;/** Resolved `_brain.yaml`, or absent when it could not be loaded. */readonly config: BrainConfig | undefined;- /** Search-index path for index-backed checks; absent skips them. */+ /**+ * Search-index path for index-backed checks. When absent, the store+ * integrity check skips itself and the embeddings-health check falls+ * back to the path the search configuration resolves.+ */readonly dbPath: string | undefined;/*** The `o2b` config file the pass was invoked against (C1).@@ -49,6 +53,11 @@ export interface DoctorCheckContext {readonly idIndex: Map<string, string[]>;readonly preferences: ReadonlyArray<PreferenceRecord>;readonly logs: ReadonlyArray<LogRecord>;+ /**+ * Log days the snapshot above could not read. Optional so a context+ * built by hand stays valid; absent reads as none.+ */+ readonly unreadableLogDays?: ReadonlyArray<UnreadableLogDay>;}/**diff --git a/src/core/brain/doctor/embeddings-health-check.ts b/src/core/brain/doctor/embeddings-health-check.tsnew file mode 100644index 00000000..9d69a2a8--- /dev/null+++ b/src/core/brain/doctor/embeddings-health-check.ts@@ -0,0 +1,294 @@+/**+ * Consolidate the scattered embeddings-health signals into the brain+ * doctor (t_f99250aa).+ *+ * The facts already existed, each behind its own verb: the pending-vector+ * census (`readPendingVectorCensus`, with its unrecorded-is-not-zero+ * contract), the index counts including `staleEmbeddings`, the provider+ * readout `o2b search check` owns. Nothing read them together where an+ * operator looks, so a vault whose index lagged its configuration stayed+ * silent until a query came back thin. This check is the strictly earlier+ * signal: it speaks from the doctor pass an operator already runs.+ *+ * ## Why it lives in `o2b brain doctor`+ *+ * `o2b search check` owns the provider question and its exit table, but+ * it is a verb an operator must think to run; this doctor is the pass+ * they run to be told what they did not think to ask. The+ * `core/brain -> core/search` import edge already exists+ * (`embedding-sunset-check.ts`, this check's shape precedent), and the+ * check inherits the context clock (`ctx.now`) the age clause is+ * computed against rather than reading a clock of its own.+ *+ * ## The four answers, and why three of them are not silence+ *+ * | state | outcome |+ * |------------------------------|---------------------------------------------|+ * | measured, embedded, current | nothing - the condition does not hold |+ * | measured, backlog | warning naming both counts and the commands |+ * | census unrecorded | uncertain: NO statement was made |+ * | anything else unmeasurable | uncertain, naming which half could not run |+ *+ * `unrecorded` is deliberately not folded into healthy. An absent or+ * unreadable index proves nothing about how many chunks are waiting, and+ * zero is precisely the answer that reads as "nothing is waiting". The+ * same refusal covers a HALF read: the census opened the file but the+ * count half failed, so `staleEmbeddings` arrives as `null` - and a null+ * count may never collapse into the zero that reads as healthy.+ *+ * ## What the message may say+ *+ * The remedy clauses are per condition, because the conditions have+ * different verbs: a chunk with no vector is closed by+ * `o2b search vector-backfill` (the vector phase run alone), while+ * vectors stored under a retired model or dimension are incomparable to+ * new queries and need `o2b search reindex --embeddings` - a backfill+ * finds them fully vectorised and does nothing. `o2b search check` is+ * the shared readout for the provider facts behind either state. The+ * disabled state has its own reported class (`semantic-capability-disabled`)+ * and is not repeated here, following `embedding-sunset-check`.+ *+ * The check is strictly read-only: every fact it touches is a synchronous+ * read-only peek at the index file.+ */++import { resolveSearchConfig } from "../../search/index.ts";+import { readPendingVectorCensus } from "../../search/indexer.ts";+import { staleBaseline } from "../../search/store/counts.ts";+import { LAST_INDEXED_AT_STATE_KEY, withReadonlyIndex } from "../../search/store/state.ts";+import { staleEmbeddings } from "../../search/store/vectors.ts";+import type { PendingVectorCensus, ResolvedEmbeddingConfig } from "../../search/types.ts";+import type { DoctorIssue } from "../types.ts";+import type { DoctorCheck, DoctorCheckContext, DoctorFindings } from "./check.ts";+import { pushUncertain } from "./uncertain-stream.ts";++/** Indexed chunks wait for vectors, or stored vectors no longer match. */+export const EMBEDDINGS_BACKLOG_CODE = "embeddings-backlog";++/** The pending-vector census could not be taken; no statement was made. */+export const EMBEDDINGS_CENSUS_UNRECORDED_CODE = "embeddings-census-unrecorded";++/** The check ran and could observe too little to say anything at all. */+export const EMBEDDINGS_HEALTH_UNMEASURED_CODE = "embeddings-health-unmeasured";++const MS_PER_DAY = 24 * 60 * 60 * 1000;++/**+ * What one probe read off the index.+ *+ * `census` keeps the two-verdict shape of `readPendingVectorCensus` so+ * the unrecorded contract travels with the data rather than being+ * re-derived here. `staleEmbeddings` is `null` when the count half of+ * the read failed - NOT zero, for the reason in the header.+ */+export interface EmbeddingsHealthProbeSnapshot {+ readonly census: PendingVectorCensus;+ /** Vectors whose model/dimension no longer match the configured pair. */+ readonly staleEmbeddings: number | null;+ /** ISO instant of the last index run, when the index recorded one. */+ readonly lastIndexedAt: string | null;+}++/** The slice of the embedding configuration the stale count is resolved from. */+export type EmbeddingsHealthSemantic = Pick<+ ResolvedEmbeddingConfig,+ "provider" | "model" | "dimension"+>;++/**+ * The seam the check reads its facts through.+ *+ * Receives the resolved index path and the embedding configuration so+ * the default probe below can resolve the pair stored vectors are+ * compared against (`staleBaseline`: an unset dimension falls back to+ * the stored one, and with none the comparison is by model alone); a+ * test injects any snapshot it wants without building an index.+ */+export type EmbeddingsHealthProbe = (+ dbPath: string,+ semantic: EmbeddingsHealthSemantic,+) => EmbeddingsHealthProbeSnapshot;++/**+ * The default probe: the exported census fold, then one synchronous+ * read-only peek for the counts the census does not carry.+ *+ * Two peeks over one file, not one, because the census's unrecorded+ * contract is folded in `readPendingVectorCensus` and re-deriving it+ * here is exactly the drift the reuse exists to prevent. A file that+ * vanishes between the two peeks is a half read, and the check answers+ * that with uncertainty rather than a health claim.+ */+function indexProbe(+ dbPath: string,+ semantic: EmbeddingsHealthSemantic,+): EmbeddingsHealthProbeSnapshot {+ const census = readPendingVectorCensus(dbPath);+ if (census.verdict !== "measured") {+ return { census, staleEmbeddings: null, lastIndexedAt: null };+ }+ const counts = withReadonlyIndex(dbPath, (read, db) => {+ const baseline = staleBaseline(db, semantic);+ return {+ stale: staleEmbeddings(db, baseline.model, baseline.dimension),+ last: read(LAST_INDEXED_AT_STATE_KEY),+ };+ });+ return {+ census,+ staleEmbeddings: counts?.stale ?? null,+ lastIndexedAt: counts?.last ?? null,+ };+}++function unmeasurable(message: string): string {+ return `${message} No statement was made about the index, and none may be inferred from a read that did not happen.`;+}++function unrecordedMessage(reason: string): string {+ return (+ `${reason}. The pending-vector census could not be taken, so no embeddings-health ` ++ "statement was made: an absent or unreadable index is not an empty one, and zero is " ++ "precisely the count that reads as fully embedded."+ );+}++function backlogMessage(+ pending: number,+ chunks: number,+ stale: number,+ lastIndexedAt: string | null,+ now: Date,+): string {+ const remedies: string[] = [];+ if (pending > 0) {+ remedies.push(+ `${pending} of ${chunks} indexed chunk(s) have no vector; ` ++ "run `o2b search vector-backfill` to compute them",+ );+ }+ if (stale > 0) {+ remedies.push(+ `${stale} stored vector(s) no longer match the configured embedding model, so they are ` ++ "not comparable to new queries; run `o2b search reindex --embeddings` to re-embed",+ );+ }+ const age =+ lastIndexedAt !== null && Number.isFinite(Date.parse(lastIndexedAt))+ ? ` The index last ran ${daysBetween(Date.parse(lastIndexedAt), now.getTime())} ago.`+ : "";+ return (+ `the search index is behind its embedding configuration: ${remedies.join("; and ")}. ` ++ `Then run \`o2b search check\` to read the provider verdict behind either state.${age}`+ );+}++/** A whole-day age clause, floored, for the warning's context sentence. */+function daysBetween(fromMs: number, toMs: number): string {+ const days = Math.floor((toMs - fromMs) / MS_PER_DAY);+ return days < 1 ? "less than a day" : `${days} day(s)`;+}++/**+ * Build the check against `probe`, following `makeEmbeddingSunsetCheck`:+ * the index facts are data the check reads rather than logic it owns, so+ * every verdict is reachable from a test without building an index.+ */+export function makeEmbeddingsHealthCheck(probe: EmbeddingsHealthProbe = indexProbe): DoctorCheck {+ return {+ failSoft: true,+ run(ctx: DoctorCheckContext, out: DoctorFindings): void {+ let search;+ try {+ search = resolveSearchConfig({ vault: ctx.vault, configPath: ctx.configPath });+ } catch (err) {+ pushUncertain(out.uncertain, {+ code: EMBEDDINGS_HEALTH_UNMEASURED_CODE,+ path: ctx.configPath ?? ctx.vault,+ message: unmeasurable(+ `the search configuration could not be resolved, so the index could not be checked: ${+ err instanceof Error ? err.message : String(err)+ }`,+ ),+ });+ return;+ }+ // No semantic search, no index lane to be healthy or sick. The+ // disabled state has its own reported class+ // (`semantic-capability-disabled`) with its own exit; repeating it+ // here would meet an operator with two findings for one condition.+ if (!search.semantic.enabled || search.semantic.provider === "disabled") return;++ // The index the pass was pointed at wins over the one the search+ // configuration names, so every probe and every finding speak about+ // the same file the rest of the doctor pass reads.+ const dbPath = ctx.dbPath ?? search.dbPath;++ // The local embedder ships inside this build and carries no model+ // string in the configuration (`staleBaseline` names it the way the+ // index does). Any other provider with no configured model leaves+ // stored vectors nothing to be compared against, and a stale count+ // of zero from no baseline would read as healthy.+ if (search.semantic.provider !== "local" && !search.semantic.model) {+ pushUncertain(out.uncertain, {+ code: EMBEDDINGS_HEALTH_UNMEASURED_CODE,+ path: dbPath,+ message: unmeasurable(+ "no embedding model is configured, so stored vectors have no baseline to be " ++ "compared against",+ ),+ });+ return;+ }+ let snap;+ try {+ snap = probe(dbPath, search.semantic);+ } catch (err) {+ pushUncertain(out.uncertain, {+ code: EMBEDDINGS_HEALTH_UNMEASURED_CODE,+ path: dbPath,+ message: unmeasurable(+ `the index probe failed: ${err instanceof Error ? err.message : String(err)}`,+ ),+ });+ return;+ }++ if (snap.census.verdict === "unrecorded") {+ pushUncertain(out.uncertain, {+ code: EMBEDDINGS_CENSUS_UNRECORDED_CODE,+ path: dbPath,+ message: unrecordedMessage(snap.census.reason),+ });+ return;+ }+ const stale = snap.staleEmbeddings;+ if (stale === null) {+ // The census half measured, the count half did not: reporting the+ // pending count alone would read as a clean stale bill, and that+ // half-read-as-healthy is exactly the failure this check exists+ // to refuse.+ pushUncertain(out.uncertain, {+ code: EMBEDDINGS_HEALTH_UNMEASURED_CODE,+ path: dbPath,+ message: unmeasurable(+ "the index opened for the pending-vector census but its embedding counts could " ++ "not be read",+ ),+ });+ return;+ }+ const { pending, chunks } = snap.census;+ if (pending === 0 && stale === 0) return;+ out.issues.push({+ severity: "warning",+ code: EMBEDDINGS_BACKLOG_CODE,+ path: dbPath,+ message: backlogMessage(pending, chunks, stale, snap.lastIndexedAt, ctx.now),+ } satisfies DoctorIssue);+ },+ };+}++/** The registered instance, reading the live index. */+export const embeddingsHealthCheck: DoctorCheck = makeEmbeddingsHealthCheck();diff --git a/src/core/brain/doctor/orphan-session-check.ts b/src/core/brain/doctor/orphan-session-check.tsnew file mode 100644index 00000000..525d1dde--- /dev/null+++ b/src/core/brain/doctor/orphan-session-check.ts@@ -0,0 +1,320 @@+/**+ * Orphan observations: `session_ref` resolved, at last (t_6cc80627).+ *+ * An observation signal carries `session_ref` - the portable session+ * coordinates the capture writer (`session:<id>#<event>`) and the import+ * writer (`<transcript basename>#<turn>`) stamp - and nothing anywhere+ * resolved it. An observation whose parent session was gone was silently+ * accepted, which for a provenance field is the same as the field not+ * existing. This check is the read-only half of the answer: it parses+ * every signal and resolves the ref against the two halves of the+ * sessions/continuity store. The write half - a repair verb that DETACHES+ * the dangling reference, keeping the observation - is+ * `src/core/brain/link-graph/orphan-repair.ts`, reachable only through+ * the command the `fix` field carries. The doctor never repairs.+ *+ * ## What resolves+ *+ * A ref's identity half (everything before the first `#`, with the+ * writer's `session:` prefix dropped) is resolved against:+ *+ * - the continuity ledger, whose records carry `session_id` in their+ * payload - the surface both writers also stamp; and+ * - the Brain log's `session-lifecycle` events, whose body carries the+ * session id of a captured host session.+ *+ * An imported transcript ref (`<transcript basename>#<turn>`, the only+ * writer form with a turn half and no `session:` prefix) is NOT judged.+ * Its parent is a transcript file outside the vault, and the import+ * leaves no record it is guaranteed to keep: the resume checkpoint is+ * cleared when the file drains, and the recall, skill-invocation and+ * dedup records are each conditional. A missing record therefore proves+ * nothing about such a ref, and a repair run on that report would strip+ * valid provenance. The capture form (`session:<id>#<event>`) and the+ * extract-signals form (the bare raw-turn session id, whose turns live+ * in the continuity ledger) are judged.+ *+ * `session:unknown` is the capture writer's stand-in for a payload that+ * named no session at all. It is a statement that there was no session,+ * not a pointer to a missing one, so it resolves by vocabulary: detaching+ * it would erase the honesty rather than repair a dangle.+ *+ * ## What may never be judged+ *+ * A resolution surface this check could not read is not an empty one.+ * Reading it as empty would name every observation an orphan, and a+ * repair run on that report would strip provenance off healthy signals -+ * so an incomplete universe emits what it could not read into the+ * `uncertain` stream and raises NO finding. The same holds for the+ * signals subtree itself: an unreadable directory is swept uncertainty+ * (the shared `vault-walk-entry-skipped` code), never silence.+ *+ * Signal files that fail to parse are skipped: the record check already+ * reports them under `signal-invalid`, and saying it twice says nothing+ * new.+ */++import { join } from "node:path";++import { continuityLogDir, listContinuityRecords } from "../continuity/store.ts";+import { CONTINUITY_SESSION_ID_KEY } from "../continuity/types.ts";+import type { BrainLogEntry } from "../log.ts";+import { parseSignal } from "../signal.ts";+import { brainDirs } from "../paths.ts";+import type { DoctorIssue } from "../types.ts";+import { BRAIN_LOG_EVENT_KIND } from "../types.ts";+import type { DoctorCheck, DoctorCheckContext, DoctorFindings } from "./check.ts";+import type { DoctorUncertainEntry } from "./report.ts";+import { readLogSnapshot, type UnreadableLogDay } from "./records.ts";+import {+ readSweptDir,+ reportSweptFailure,+ reportSweptSkip,+ SWEEP_ORIGIN,+ type SweptPath,+} from "./unreadable-path.ts";+import { pushUncertain } from "./uncertain-stream.ts";++/** An observation's `session_ref` resolves to no session or continuity record. */+export const ORPHAN_SESSION_REF_CODE = "orphan-session-ref";++/**+ * The exact repair command, carried on the issue's `fix` field.+ *+ * The verb it names detaches the stale reference and keeps the+ * observation, under the repair-lane discipline (dry-run default, exact+ * confirm phrase, hard per-run write cap). It is a structural CLI+ * string, never prose, and nothing in this module runs it.+ */+export const ORPHAN_SESSION_REPAIR_COMMAND = "o2b brain orphan-repair";++/** Subsystem name the check's sweeps report an unreadable path under. */+export const ORPHAN_SESSION_SITE = "brain.doctor.orphanSession";++/** What the check can no longer claim for a signals subtree it could not list. */+const SIGNALS_CONSEQUENCE =+ "no observation in it had its session_ref resolved, so an orphaned one is missing from this report";++/** The capture writer's stand-in identity for a payload that named no session. */+const NO_SESSION_STAND_IN = "unknown";++/** Filename prefix of a signal file; every walked record carries it. */+const SIGNAL_FILE_PREFIX = "sig-";++/** The two halves of a `session_ref`, split the way both writers spell it. */+export interface ParsedSessionRef {+ /**+ * Everything before the first `#`, with the `session:` prefix the+ * capture writer puts on dropped. Empty only for a ref that never+ * named an identity at all.+ */+ readonly identity: string;+ /** The turn or event half, when the ref carried one. */+ readonly turn: string | null;+}++export function parseSessionRef(ref: string): ParsedSessionRef {+ const hash = ref.indexOf("#");+ const identityRaw = hash === -1 ? ref : ref.slice(0, hash);+ const turn = hash === -1 ? null : ref.slice(hash + 1);+ const identity = (+ identityRaw.startsWith("session:") ? identityRaw.slice("session:".length) : identityRaw+ ).trim();+ return { identity, turn: turn !== null && turn.trim().length > 0 ? turn.trim() : null };+}++/**+ * True for the import writer's `<transcript basename>#<turn>` form: a+ * turn half and no `session:` prefix. See the module docblock for why+ * such a ref is never judged.+ */+export function isImportedTranscriptRef(ref: string): boolean {+ const trimmed = ref.trim();+ return !trimmed.startsWith("session:") && parseSessionRef(trimmed).turn !== null;+}++/**+ * The session-id universe one vault's store holds.+ *+ * `complete` is the load-bearing half: a surface that could not be read+ * makes the set PARTIAL, and a partial universe resolves nothing - every+ * judgement taken from it would be a guess dressed as a finding. What+ * could not be read is named in `uncertain` (the shared swept-path+ * shape) so the caller reports the gap rather than inferring it.+ */+export interface SessionUniverse {+ readonly ids: ReadonlySet<string>;+ readonly complete: boolean;+ readonly uncertain: ReadonlyArray<DoctorUncertainEntry>;+}++/**+ * The pass's pre-parsed log snapshot, as the doctor context carries it:+ * the entries it read and the days it could not.+ */+export interface SessionLogSnapshot {+ readonly entries: ReadonlyArray<BrainLogEntry>;+ readonly unreadableDays: ReadonlyArray<UnreadableLogDay>;+}++/**+ * Read both resolution surfaces.+ *+ * `logSnapshot` - the doctor pass's pre-parsed log snapshot - stands in+ * for a second log walk when the caller has one. Without it the log is+ * read here, under the same sweep sink, so both paths report an+ * unreadable surface the same way and neither reads it as empty. A log+ * day that could not be read makes the universe partial, exactly like an+ * unreadable directory: the session whose lifecycle lived in it would+ * otherwise read as an orphan.+ */+export function collectKnownSessionIds(+ vault: string,+ logSnapshot?: SessionLogSnapshot,+): SessionUniverse {+ const uncertain: DoctorUncertainEntry[] = [];+ const swept: SweptPath = {+ site: ORPHAN_SESSION_SITE,+ consequence:+ "session_ref values could not be resolved against it, so no orphan statement about them is trustworthy",+ uncertain,+ };+ // The stand-in is vocabulary, not a session: see the module docblock.+ const ids = new Set<string>([NO_SESSION_STAND_IN]);+ let complete = true;+ // The listing is probed through the swept reader, NOT through+ // `listShardedFiles`' own `existsSync` gate: that gate answers false+ // for a permission denial exactly as it does for an absent directory,+ // and a denial read as "no continuity records" would name every+ // observation an orphan.+ const before = uncertain.length;+ const continuityEntries = readSweptDir(continuityLogDir(vault), swept, SWEEP_ORIGIN.root);+ if (continuityEntries !== null) {+ try {+ for (const record of listContinuityRecords(vault)) {+ const sid = record.payload[CONTINUITY_SESSION_ID_KEY];+ if (typeof sid === "string" && sid.trim().length > 0) ids.add(sid.trim());+ }+ } catch (err) {+ reportSweptFailure(+ continuityLogDir(vault),+ "continuity ledger read failed",+ err,+ swept,+ SWEEP_ORIGIN.root,+ );+ }+ }+ const log = logSnapshot ?? readLogEntries(vault, swept);+ for (const day of log.unreadableDays) {+ reportSweptSkip(+ join(brainDirs(vault).log, day.date),+ `log day ${day.date} could not be read: ${day.detail}`,+ swept,+ );+ }+ for (const entry of log.entries) {+ if (entry.eventType !== BRAIN_LOG_EVENT_KIND.sessionLifecycle) continue;+ const sid = entry.body["session_id"];+ if (typeof sid === "string" && sid.trim().length > 0) ids.add(sid.trim());+ }+ if (uncertain.length > before) complete = false;+ return { ids, complete, uncertain };+}++function readLogEntries(vault: string, swept: SweptPath): SessionLogSnapshot {+ const snapshot = readLogSnapshot(vault, swept);+ return {+ entries: snapshot.records.flatMap((record) => record.entries),+ unreadableDays: snapshot.unreadableDays,+ };+}++/** One observation whose parent session could not be found. */+export interface OrphanFinding {+ /** Vault-absolute path of the signal file. */+ readonly path: string;+ /** The dangling value, verbatim - the repair's decision quotes it. */+ readonly session_ref: string;+ readonly identity: string;+}++/**+ * Walk the signal store and resolve every `session_ref` against+ * `universe`. Unreadable signal directories report through `swept`;+ * unparseable signal files are skipped (the record check owns them).+ */+export function scanOrphanedSessionRefs(+ vault: string,+ universe: SessionUniverse,+ swept: SweptPath,+): ReadonlyArray<OrphanFinding> {+ const dirs = brainDirs(vault);+ const findings: OrphanFinding[] = [];+ for (const dir of [dirs.inbox, dirs.processed, dirs.archived]) {+ const entries = readSweptDir(dir, swept, SWEEP_ORIGIN.root);+ if (entries === null) continue;+ for (const entry of entries) {+ if (!entry.isFile()) continue;+ if (!entry.name.endsWith(".md") || !entry.name.startsWith(SIGNAL_FILE_PREFIX)) continue;+ const path = join(dir, entry.name);+ let sig;+ try {+ sig = parseSignal(path);+ } catch {+ // Parse failures are `signal-invalid`'s findings, not this+ // check's; a file with no readable frontmatter has no ref to+ // resolve either way.+ continue;+ }+ const ref = sig.session_ref;+ if (ref === undefined) continue;+ if (isImportedTranscriptRef(ref)) continue;+ const { identity } = parseSessionRef(ref);+ if (universe.ids.has(identity)) continue;+ findings.push({ path, session_ref: ref, identity });+ }+ }+ return findings;+}++/**+ * The registered check. Fail-soft: a store this module cannot read must+ * cost the pass an uncertainty, never a throw - and never a finding+ * either, for the reason the module docblock gives.+ */+export const orphanSessionCheck: DoctorCheck = {+ failSoft: true,+ run(ctx: DoctorCheckContext, out: DoctorFindings): void {+ const swept: SweptPath = {+ site: ORPHAN_SESSION_SITE,+ consequence: SIGNALS_CONSEQUENCE,+ uncertain: out.uncertain,+ };+ const universe = collectKnownSessionIds(ctx.vault, {+ entries: ctx.logs.flatMap((r) => r.entries),+ unreadableDays: ctx.unreadableLogDays ?? [],+ });+ if (!universe.complete) {+ // The id set is partial. Folding the gap into the stream and+ // reporting nothing is the only honest answer: a finding here+ // would be a resolution the check never managed to make.+ for (const entry of universe.uncertain) {+ pushUncertain(out.uncertain, entry, swept.consequence);+ }+ return;+ }+ for (const finding of scanOrphanedSessionRefs(ctx.vault, universe, swept)) {+ out.issues.push({+ severity: "warning",+ code: ORPHAN_SESSION_REF_CODE,+ path: finding.path,+ fix: ORPHAN_SESSION_REPAIR_COMMAND,+ message:+ `session_ref '${finding.session_ref}' resolves to no session or continuity record; ` ++ `the observation is orphaned. Run ${ORPHAN_SESSION_REPAIR_COMMAND} to detach the ` ++ "stale reference - the observation itself is kept",+ } satisfies DoctorIssue);+ }+ },+};diff --git a/src/core/brain/doctor/records.ts b/src/core/brain/doctor/records.tsindex 97fa5b09..2857e987 100644--- a/src/core/brain/doctor/records.ts+++ b/src/core/brain/doctor/records.ts@@ -130,22 +130,56 @@ export function readAllRetiredRecords(return out;}+/** One log day the snapshot could not read, with why. */+export interface UnreadableLogDay {+ readonly date: string;+ readonly detail: string;+}++/** The pre-parsed log snapshot plus the days it had to leave out. */+export interface LogSnapshot {+ readonly records: ReadonlyArray<LogRecord>;+ /**+ * Days whose read threw. Empty means the snapshot is complete; a check+ * that resolves identities against the log must not read a non-empty+ * list as "those days held nothing".+ */+ readonly unreadableDays: ReadonlyArray<UnreadableLogDay>;+}+/**- * Build a single pre-parsed snapshot of `Brain/log/` so the two- * log-walking lints don't each re-parse the directory.+ * Build a single pre-parsed snapshot of `Brain/log/` so the log-walking+ * lints don't each re-parse the directory, and name every day that could+ * not be read so no caller mistakes a partial snapshot for a whole one.*/-export function readAllLogRecords(vault: string, swept?: SweptPath): ReadonlyArray<LogRecord> {+export function readLogSnapshot(vault: string, swept?: SweptPath): LogSnapshot {// Shard-aware (Memory Integrity Suite): dates come from the single// discovery helper and entries arrive merged across device shards.- const out: LogRecord[] = [];+ const records: LogRecord[] = [];+ const unreadableDays: UnreadableLogDay[] = [];for (const date of listLogDatesSwept(vault, swept)) {try {- out.push({ date, entries: readLogDay(vault, date).entries });- } catch {- // parse error — surfaced separately by the log shard check+ const day = readLogDay(vault, date);+ records.push({ date, entries: day.entries });+ // A shard the listing showed but the read could not obtain comes+ // back as a line-0 warning beside the entries of the other shards:+ // the day is partial, and a partial day is an unreadable one here.+ const failed = day.warnings.filter((w) => w.lineNumber === 0);+ if (failed.length > 0) {+ unreadableDays.push({ date, detail: failed.map((w) => w.message).join("; ") });+ }+ } catch (err) {+ // The parse error itself is surfaced by the log shard check; the+ // day is named here so identity resolution knows it is missing.+ unreadableDays.push({ date, detail: err instanceof Error ? err.message : String(err) });}}- return out;+ return { records, unreadableDays };+}++/** {@link readLogSnapshot}'s records alone, for lints that only count what they read. */+export function readAllLogRecords(vault: string, swept?: SweptPath): ReadonlyArray<LogRecord> {+ return readLogSnapshot(vault, swept).records;}/**diff --git a/src/core/brain/ingest/batch-plan.ts b/src/core/brain/ingest/batch-plan.tsindex 675afe95..7c663f09 100644--- a/src/core/brain/ingest/batch-plan.ts+++ b/src/core/brain/ingest/batch-plan.ts@@ -64,7 +64,9 @@ import { classifyPaths, readManifest } from "./content-manifest.ts";import {extractableAllowlist,partitionExtractable,+ SKIPPED_PAGE_REASONS,type SkippedPage,+ type SkippedPageReason,} from "./extractable-gate.ts";/**@@ -139,6 +141,20 @@ export interface IngestBatch {readonly totalBytes: number;}+/**+ * Files the discovery walk dropped because their extension is not ingestible,+ * aggregated per extension (P4). Hidden entries and ignore-rule matches are+ * excluded by declaration, not unclassifiable, and never counted; a file with+ * no extension counts under the empty string. A `total` of 0 means every+ * walked file was ingestible (the aggregate is absent on the wire).+ */+export interface UnclassifiableFiles {+ /** How many files the walk dropped as non-ingestible. */+ readonly total: number;+ /** Per-extension counts, sorted by extension. "" keys extensionless files. */+ readonly byExtension: Readonly<Record<string, number>>;+}+export interface BatchPlan {/** Canonical vault-relative POSIX path of the planned source directory. */readonly sourceDir: string;@@ -154,6 +170,19 @@ export interface BatchPlan {* unset (the gate is off), keeping the plan byte-identical to before.*/readonly skippedNonExtractable: readonly SkippedPage[];+ /**+ * {@link BatchPlan.skippedNonExtractable} counted per typed reason token+ * (P4). Only non-zero reasons appear, in membership order; empty when+ * nothing was skipped (absent on the wire, byte-identical-when-absent).+ */+ readonly skipReasonCounts: Readonly<Partial<Record<SkippedPageReason, number>>>;+ /**+ * Files dropped during discovery because no ingestible extension matched+ * (P4), aggregated per extension. Exclusion by declaration (hidden entries,+ * ignore-rule matches) is not counted, and the counting never alters the+ * discovered set - hence not {@link BatchPlan.planId} either.+ */+ readonly unclassifiable: UnclassifiableFiles;/** Total number of files across all batches (`new` + `modified`). */readonly totalFiles: number;/** Total bytes across all batches. */@@ -257,6 +286,7 @@ export function planBatches(vault: string, sourceDir: string, opts: BatchPlanOpt// Discover ingestible files as canonical vault-relative paths, sorted. A walk// root inside an ignored subtree discovers nothing and says why on the plan.const discovered: string[] = [];+ const unclassifiable = new Map<string, number>();if (prunedWarning === null) {collectIngestible(walkRoot, walkScope, {vault,@@ -264,6 +294,7 @@ export function planBatches(vault: string, sourceDir: string, opts: BatchPlanOptexcludeLayer,out: discovered,warnings: ignoreWarnings,+ unclassifiable,});}const discoveredRel = discovered@@ -317,6 +348,8 @@ export function planBatches(vault: string, sourceDir: string, opts: BatchPlanOptbatches,skipped,skippedNonExtractable,+ skipReasonCounts: countSkipReasons(skippedNonExtractable),+ unclassifiable: aggregateUnclassifiable(unclassifiable),totalFiles: planned.length,totalBytes: planned.reduce((sum, f) => sum + f.bytes, 0),planId,@@ -333,6 +366,34 @@ function compareIgnoreWarnings(a: IgnoreWarning, b: IgnoreWarning): number {return a.line - b.line;}+/**+ * Count the plan's skipped pages per typed reason token, in membership order+ * so the aggregate (and the wire record built from it) is deterministic.+ */+function countSkipReasons(+ skipped: readonly SkippedPage[],+): Partial<Record<SkippedPageReason, number>> {+ const counts: Partial<Record<SkippedPageReason, number>> = {};+ for (const reason of SKIPPED_PAGE_REASONS) {+ const n = skipped.filter((s) => s.reason === reason).length;+ if (n > 0) counts[reason] = n;+ }+ return counts;+}++/**+ * Aggregate the walk's per-extension drop counts into the plan's shape,+ * sorted by extension so the record (and the wire object built from it) is+ * deterministic.+ */+function aggregateUnclassifiable(counts: ReadonlyMap<string, number>): UnclassifiableFiles {+ const entries = [...counts.entries()].toSorted(([a], [b]) => (a < b ? -1 : 1));+ return {+ total: entries.reduce((sum, [, n]) => sum + n, 0),+ byExtension: Object.fromEntries(entries),+ };+}+/*** Greedily pack sorted files into batches. A file joins the current batch unless* doing so would breach the byte cap or the count cap, in which case the current@@ -486,6 +547,8 @@ interface WalkContext {readonly out: string[];/** Accumulates malformed patterns found in the repository's own ignore files. */readonly warnings: IgnoreWarning[];+ /** Counts files dropped for a non-ingestible extension, per extension. */+ readonly unclassifiable: Map<string, number>;}/**@@ -498,6 +561,10 @@ interface WalkContext {* only what it governs and a nearer `!` re-include wins. The operator's* `--exclude` is applied on top for every match, keeping it above the whole* repository-declared stack no matter how deep the walk goes.+ *+ * A regular file the scope lets through whose extension is not ingestible is+ * counted on {@link WalkContext.unclassifiable} (P4) instead of vanishing:+ * the discovery outcome is unchanged, but the drop is no longer silent.*/function collectIngestible(dir: string, scope: IgnoreScope, ctx: WalkContext): void {const effective = ctx.excludeLayer === null ? scope : scope.extend(ctx.excludeLayer);@@ -514,8 +581,16 @@ function collectIngestible(dir: string, scope: IgnoreScope, ctx: WalkContext): vconst child = extendWithDirectoryIgnore(scope, abs, rel);ctx.warnings.push(...child.warnings);collectIngestible(abs, child.scope, ctx);- } else if (entry.isFile() && ctx.extensions.has(extname(entry.name).toLowerCase())) {+ } else if (entry.isFile()) {+ // An ignore-rule match is exclusion by DECLARATION - checked before the+ // extension test so an ignored binary never inflates the unclassifiable+ // count. The discovered set is untouched either way.if (effective.isIgnored(rel, false)) continue;+ const ext = extname(entry.name).toLowerCase();+ if (!ctx.extensions.has(ext)) {+ ctx.unclassifiable.set(ext, (ctx.unclassifiable.get(ext) ?? 0) + 1);+ continue;+ }ctx.out.push(abs);}}diff --git a/src/core/brain/ingest/extractable-gate.ts b/src/core/brain/ingest/extractable-gate.tsindex ed37ffa4..cdcf7f7e 100644--- a/src/core/brain/ingest/extractable-gate.ts+++ b/src/core/brain/ingest/extractable-gate.ts@@ -25,10 +25,66 @@ import { loadSchemaPack } from "../schema-pack.ts";/** The frontmatter field that carries a page's schema page-type token. */const PAGE_TYPE_FIELD = "schema_type";+/**+ * Why a page was skipped before extraction. A closed token union (the+ * `PAGE_LINT_SKIP_REASON` pattern): the reason crosses the MCP wire and the+ * CLI JSON verbatim, so a reader must be able to reject a value this build+ * does not understand instead of parsing free text.+ */+export const SKIPPED_PAGE_REASON = Object.freeze({+ /** The page's `schema_type` is not in the schema `extractable` allowlist. */+ notExtractable: "schema-type-not-extractable",+} as const);++/** The closed union of skip reason tokens. */+export type SkippedPageReason = (typeof SKIPPED_PAGE_REASON)[keyof typeof SKIPPED_PAGE_REASON];++/** Membership list of the closed union, in the order a page meets the gates. */+export const SKIPPED_PAGE_REASONS: ReadonlyArray<SkippedPageReason> = Object.freeze([+ SKIPPED_PAGE_REASON.notExtractable,+]);++/** The free-text sentence this gate emitted before the reason was typed (P4). */+const LEGACY_REASON_RE = /^schema_type "[^"]*" is not in the schema extractable allowlist$/;++/**+ * Membership guard of the closed union: true only for a token this build+ * emits. The legacy free-text sentence is NOT a member, so it is rejected+ * here; a reader of persisted or wire values uses+ * {@link parseSkippedPageReason}, which maps it to its token.+ */+export function isSkippedPageReason(value: unknown): value is SkippedPageReason {+ return (+ typeof value === "string" && (SKIPPED_PAGE_REASONS as ReadonlyArray<string>).includes(value)+ );+}++/**+ * Parse a reason read back across a tool boundary. Accepts the typed token+ * and the legacy free-text sentence pre-taxonomy builds serialized - that+ * sentence was the only reason this gate ever emitted, so it maps to+ * {@link SKIPPED_PAGE_REASON.notExtractable} - and returns null for+ * anything else, so a caller never misreads a value it does not understand.+ */+export function parseSkippedPageReason(value: unknown): SkippedPageReason | null {+ if (isSkippedPageReason(value)) return value;+ if (typeof value === "string" && LEGACY_REASON_RE.test(value)) {+ return SKIPPED_PAGE_REASON.notExtractable;+ }+ return null;+}+/** One page skipped by the gate, with the reason it was excluded. */export interface SkippedPage {readonly path: string;- readonly reason: string;+ /** Typed reason token from the closed {@link SKIPPED_PAGE_REASONS} union. */+ readonly reason: SkippedPageReason;+ /**+ * The value behind the skip - the page's declared `schema_type` - so the+ * reason is checkable without re-reading the page (the page-lint `detail`+ * pattern: identifiers cross the boundary, never prose).+ */+ readonly detail: string;}/** Discovered pages split into the extractable set and the skipped set. */@@ -78,10 +134,7 @@ export function partitionExtractable(extractable.push(path);continue;}- skipped.push({- path,- reason: `schema_type "${type}" is not in the schema extractable allowlist`,- });+ skipped.push({ path, reason: SKIPPED_PAGE_REASON.notExtractable, detail: type });}return { extractable, skipped };}diff --git a/src/core/brain/link-graph/orphan-repair.ts b/src/core/brain/link-graph/orphan-repair.tsnew file mode 100644index 00000000..5568cd6e--- /dev/null+++ b/src/core/brain/link-graph/orphan-repair.ts@@ -0,0 +1,210 @@+/**+ * Orphan-observation repair lane (t_6cc80627): detach dangling+ * `session_ref` values from observation signals, keeping the observation.+ *+ * The memory-graph repair lane (`repair-lane.ts`) writes ADDITIVE edges;+ * this lane's act looks destructive by contrast, which is exactly why it+ * inherits the whole discipline rather than part of it:+ *+ * - dry-run is the DEFAULT and writes nothing - the report is the plan;+ * - apply requires the exact confirmation phrase, checked before any+ * scan runs, so a refused apply is a pure refusal;+ * - a hard per-run write cap stops a run, because a repair loop that+ * never stops is a bug with write access;+ * - the vault-identity assertion fronts every write;+ * - a rescan after apply converges to zero writes: the reference is+ * gone from the frontmatter, so the second scan finds no finding.+ *+ * The one thing it does NOT do is delete. The detach removes the+ * `session_ref` key from the signal's frontmatter; the observation body,+ * its topic, and every other field are byte-identical, and the detached+ * value is quoted in the decision so the change is auditable and+ * recoverable by hand.+ *+ * ## Fail-closed on a partial store+ *+ * The lane resolves every candidate against the sessions/continuity+ * store first. A store that could not be read completely makes the id+ * set partial, and a partial set resolves nothing - so the lane REFUSES+ * rather than repair on a guess. The same holds for a signals subtree+ * the scan could not read. This is the narrower gate the design asks+ * for: an edge lane that mis-resolves writes a skip, a detach lane that+ * mis-resolves strips provenance.+ *+ * ## Never reachable from the doctor+ *+ * `runDoctor` reports orphans and carries the repair command on the+ * issue's `fix` field; this module is imported by the CLI verb only. A+ * scan-only pass and a write path share the detector+ * (`scanOrphanedSessionRefs`), never the invocation.+ */++import { acquireLockSync } from "../sync-lockfile.ts";+import { parseFrontmatter, writeFrontmatterAtomic } from "../../vault.ts";+import type { FrontmatterMap } from "../../types.ts";+import { vaultRelative } from "../../path-safety.ts";+import { assertVaultIdentityForWrite } from "../vault-identity.ts";+import { collectKnownSessionIds, scanOrphanedSessionRefs } from "../doctor/orphan-session-check.ts";+import type { DoctorUncertainEntry } from "../doctor/report.ts";+import type { SweptPath } from "../doctor/unreadable-path.ts";++/** Exact phrase an apply must supply as confirmation. */+export const ORPHAN_REPAIR_CONFIRM_PHRASE = "apply orphan repair";++/** Hard cap on signals detached in a single run. */+export const ORPHAN_REPAIR_WRITE_CAP = 25;++/** Raised when an apply is requested without the exact confirmation phrase. */+export class OrphanRepairConfirmationError extends Error {+ constructor() {+ super(+ `orphan repair requires the exact confirmation phrase: ${JSON.stringify(+ ORPHAN_REPAIR_CONFIRM_PHRASE,+ )}`,+ );+ this.name = "OrphanRepairConfirmationError";+ }+}++/** Raised when the store could not be read completely enough to act on. */+export class OrphanRepairStoreUnreadableError extends Error {+ constructor(detail: string) {+ super(+ `the session store could not be read completely, so no orphan statement is trustworthy: ` ++ `${detail}. Nothing was detached`,+ );+ this.name = "OrphanRepairStoreUnreadableError";+ }+}++/** What the lane decided about one finding. */+export type OrphanRepairAction = "detach" | "skip-cap" | "skip-changed" | "skip-locked";++export interface OrphanRepairDecision {+ /** Vault-relative path of the signal file. */+ readonly path: string;+ /** The dangling reference, verbatim, for audit and manual recovery. */+ readonly session_ref: string;+ readonly action: OrphanRepairAction;+}++export interface OrphanRepairReport {+ readonly mode: "dry-run" | "apply";+ /** Count of references detached (or, in dry-run, that would be). */+ readonly detached: number;+ readonly decisions: readonly OrphanRepairDecision[];+}++export interface OrphanRepairOptions {+ /** Apply the writes. Default false (dry-run). */+ readonly apply?: boolean;+ /** Exact confirmation phrase; required when `apply` is true. */+ readonly confirm?: string;+ /** Per-run cap override. Defaults to {@link ORPHAN_REPAIR_WRITE_CAP}. */+ readonly writeCap?: number;+}++/** What the repair's own signal sweep reports an unreadable path under. */+const REPAIR_SITE = "brain.orphan-repair";++/**+ * Run the lane over the vault's signals. Scans once, orders the findings+ * deterministically, and - only on a confirmed apply - detaches each one+ * until the cap stops the run.+ */+export function runOrphanRepair(vault: string, opts: OrphanRepairOptions = {}): OrphanRepairReport {+ const apply = opts.apply === true;+ if (apply && opts.confirm !== ORPHAN_REPAIR_CONFIRM_PHRASE) {+ throw new OrphanRepairConfirmationError();+ }+ // Vault-identity write guard (context-integrity-gates, Unit J).+ // A dry run reports decisions and writes nothing.+ if (apply) assertVaultIdentityForWrite(vault);++ const universe = collectKnownSessionIds(vault);+ if (!universe.complete) {+ throw new OrphanRepairStoreUnreadableError(+ universe.uncertain.map((entry) => entry.message).join("; ") ||+ "a resolution surface reported itself unreadable",+ );+ }++ const uncertain: DoctorUncertainEntry[] = [];+ const swept: SweptPath = {+ site: REPAIR_SITE,+ consequence:+ "signals in it were not scanned, so their session_ref values were not resolved and none " ++ "of them will be detached this run",+ uncertain,+ };+ const findings = [...scanOrphanedSessionRefs(vault, universe, swept)].toSorted((a, b) =>+ a.path < b.path ? -1 : a.path > b.path ? 1 : 0,+ );+ if (uncertain.length > 0) {+ // A subtree the scan could not read makes the finding set partial in+ // exactly the way the docblock refuses to act on.+ throw new OrphanRepairStoreUnreadableError(uncertain.map((entry) => entry.message).join("; "));+ }++ const cap = Math.max(0, Math.floor(opts.writeCap ?? ORPHAN_REPAIR_WRITE_CAP));+ const decisions: OrphanRepairDecision[] = [];+ let detached = 0;+ for (const finding of findings) {+ const decision: Omit<OrphanRepairDecision, "action"> = {+ path: vaultRelative(finding.path, vault),+ session_ref: finding.session_ref,+ };+ if (detached >= cap) {+ decisions.push({ ...decision, action: "skip-cap" });+ continue;+ }+ const outcome = apply ? detachSessionRef(finding.path, finding.session_ref) : "detached";+ if (outcome !== "detached") {+ decisions.push({ ...decision, action: outcome });+ continue;+ }+ detached += 1;+ decisions.push({ ...decision, action: "detach" });+ }++ return {+ mode: apply ? "apply" : "dry-run",+ detached,+ decisions: Object.freeze(decisions),+ };+}++/**+ * Remove the `session_ref` key from a signal's frontmatter, atomically,+ * keeping the body and every other field - but only while the key still+ * holds `expected`, the value the scan resolved and the decision quotes.+ * The read, the compare and the write run under the signal's per-file+ * lock, the one every other repair of a Brain record takes, so no writer+ * can change the file in between. A contended lock is `skip-locked`+ * (left for a later run); a reference rewritten since the scan was never+ * judged, so it is `skip-changed`; a key already gone (a concurrent+ * repair did the work) is not a write either.+ */+export function detachSessionRef(+ absPath: string,+ expected: string,+): "detached" | "skip-changed" | "skip-locked" {+ let handle: ReturnType<typeof acquireLockSync>;+ try {+ handle = acquireLockSync(absPath);+ } catch {+ return "skip-locked";+ }+ try {+ const [meta, body] = parseFrontmatter(absPath);+ const current = meta["session_ref"];+ if (current === undefined) return "detached";+ if (current !== expected) return "skip-changed";+ const next: FrontmatterMap = { ...meta };+ delete next["session_ref"];+ writeFrontmatterAtomic(absPath, next, body, { overwrite: true });+ return "detached";+ } finally {+ handle.release();+ }+}diff --git a/src/core/brain/types.ts b/src/core/brain/types.tsindex 2f81b8c7..9425983a 100644--- a/src/core/brain/types.ts+++ b/src/core/brain/types.ts@@ -2260,6 +2260,16 @@ export interface DoctorIssue {* per-state limit while the message carries the true total.*/readonly sources?: ReadonlyArray<string>;+ /**+ * The exact command that repairs this finding, when one mechanical+ * repair exists and the general doctor's `CheckResult.fix` shape+ * already models it (`orphan-session-ref`). Optional and additive:+ * every issue that does not carry one is byte-identical to what it+ * was, so JSON consumers and the drift censuses see no change. The+ * command is a structural CLI string - the verb that owns the repair,+ * never prose - and the doctor never runs it.+ */+ readonly fix?: string;}/**diff --git a/src/core/brain/write-batch.ts b/src/core/brain/write-batch.tsindex 814afa2a..e70c62b8 100644--- a/src/core/brain/write-batch.ts+++ b/src/core/brain/write-batch.ts@@ -41,6 +41,7 @@ import {} from "./notes/write-record.ts";import { formatFrontmatter, parseFrontmatterWithNotices } from "../vault.ts";import { DEGRADATION_CODE } from "../integrity/degradation.ts";+import { writePathAdvisoryField, type WritePathAdvisoryField } from "./write-path-advisory.ts";import {appendApplyEvidence,type AppendApplyEvidenceInput,@@ -237,23 +238,59 @@ export interface NoteWriteAudit {readonly audit_reason?: string;}-/** Per-operation outcome, discriminated by `kind`. */+/**+ * The advisory half every note-write result MAY carry (p4+ * silent-failure-hardening, Task 3): the absolute home paths the call's+ * authored content embeds. Byte-identical-when-absent - the key is simply+ * not there for a clean write - and never a refusal: the write lands and+ * the receipt says what it embeds. Log-line and evidence ops author no+ * note content and never carry it.+ */+export type NoteWriteAdvisory = WritePathAdvisoryField;++/**+ * Per-operation outcome, discriminated by `kind`.+ *+ * The update and append arms split on their flag so a caller cannot read+ * a write that did not happen as one that did: `updated: true` /+ * `appended: true` always carry the audit half (an id, or null plus the+ * reason there is none), while the `false` arms carry no audit fields at+ * all - a skipped write stored no before-image and recorded no event, so+ * there is no null that could be mistaken for a lost one, the same shape+ * the create skip spells. Both arms carry the advisory: the caller+ * authored that content either way.+ */export type WriteBatchOpResult =| ({readonly kind: "create_note";readonly path: string;readonly created: true;- } & NoteWriteAudit)+ } & NoteWriteAudit &+ NoteWriteAdvisory)| ({readonly kind: "update_note";readonly path: string;readonly updated: true;- } & NoteWriteAudit)+ } & NoteWriteAudit &+ NoteWriteAdvisory)+ | ({+ readonly kind: "update_note";+ readonly path: string;+ /** The target already held these bytes; nothing was written. */+ readonly updated: false;+ } & NoteWriteAdvisory)| ({readonly kind: "append_note";readonly path: string;readonly appended: true;- } & NoteWriteAudit)+ } & NoteWriteAudit &+ NoteWriteAdvisory)+ | ({+ readonly kind: "append_note";+ readonly path: string;+ /** The target already held the would-be result; nothing was written. */+ readonly appended: false;+ } & NoteWriteAdvisory)| { readonly kind: "apply_evidence"; readonly logged_at: string; readonly log_path: string }| { readonly kind: "append_log_line"; readonly logged_at: string; readonly log_path: string };@@ -432,13 +469,16 @@ function projectCreateNote();}// The create writer records its own event, so the batch carries- // its receipt through rather than appending a second one.+ // its receipt through rather than appending a second one. The+ // advisory was computed in the projection from the authored+ // content; it advises, it never refuses.return {kind: "create_note",path: res.path,created: true,write_id: res.write_id,...(res.audit_reason !== undefined ? { audit_reason: res.audit_reason } : {}),+ ...writePathAdvisoryField(op.content, res.path),};} catch (err) {throw envelopeError(err, index);@@ -532,7 +572,28 @@ function projectUpdateNote(NOTE_WRITE_OP.update,opts,);- return { kind: "update_note", path: target.relPath, updated: true, ...audit };+ // The flag is the write's own verdict, not a hardcoded success: a+ // byte-identical re-apply skipped the write and says so, carrying+ // no audit half because nothing was recorded.+ if (audit.wrote) {+ const { write_id, audit_reason } = audit;+ return {+ kind: "update_note",+ path: target.relPath,+ updated: true,+ write_id,+ ...(audit_reason !== undefined ? { audit_reason } : {}),+ // The caller's authored body, not the note's whole content: a+ // path already on disk was not this call's authorship.+ ...writePathAdvisoryField(op.body, target.relPath),+ };+ }+ return {+ kind: "update_note",+ path: target.relPath,+ updated: false,+ ...writePathAdvisoryField(op.body, target.relPath),+ };},};}@@ -566,20 +627,64 @@ function projectAppendNote(NOTE_WRITE_OP.append,opts,);- return { kind: "append_note", path: target.relPath, appended: true, ...audit };+ // Honest for the same reason the update's flag is: a rewrite whose+ // bytes already sit on disk (a hand-edited note that already+ // contains the would-be result) wrote nothing and says so, with no+ // audit half because nothing was recorded.+ if (audit.wrote) {+ const { write_id, audit_reason } = audit;+ return {+ kind: "append_note",+ path: target.relPath,+ appended: true,+ write_id,+ ...(audit_reason !== undefined ? { audit_reason } : {}),+ // The appended text is the call's authorship; the body it+ // joined is not.+ ...writePathAdvisoryField(op.content, target.relPath),+ };+ }+ return {+ kind: "append_note",+ path: target.relPath,+ appended: false,+ ...writePathAdvisoryField(op.content, target.relPath),+ };},};}/**- * Commit one note rewrite and attribute it, in the order the design- * fixes: keep the bytes this write replaces, write, record.+ * What one note rewrite actually did: either it wrote, and carries the+ * audit half (`write_id`, plus `audit_reason` exactly when that id is+ * null), or it skipped a byte-identical target and carries nothing. The+ * split - rather than a boolean beside optional fields - is what keeps a+ * skipped write from ever spelling a null write_id that a caller could+ * mistake for a lost record.+ */+type CommittedNoteRewrite = ({ readonly wrote: true } & NoteWriteAudit) | { readonly wrote: false };++/**+ * Commit one note rewrite and attribute it: for a rewrite that changes+ * the bytes, keep the bytes it replaces, write, and record the event.*- * The image goes FIRST because it is the only copy of the prior content- * that survives the rename - a process that dies between the rename and- * the record loses the audit line, which is recoverable, rather than the- * bytes, which are not. The record goes LAST and cannot fail the write:- * `recordNoteWrite` returns its failure, and this returns it too.+ * The write runs with `skipIfUnchanged`, so a byte-identical re-apply+ * leaves the target untouched - no temp file, no rename, no mtime bump+ * for the recency and validity consumers to misread - and the returned+ * verdict is the one source of truth for what happened. A skipped write+ * stores no before-image (the prior bytes are still the bytes on disk;+ * an image would claim a replace that never occurred) and records no+ * note-write event (no audit line exists for a write that did not+ * happen).+ *+ * The order is the design's: image, then write, then record. Whether an+ * image is owed is decided up front (`before !== contents`), so the image+ * lands BEFORE the rename and a crash or a failed store never leaves the+ * prior bytes replaced without their copy - a retry would otherwise see+ * the new bytes, skip as unchanged and never reach the ledger. The store+ * is content-addressed and idempotent, so an image stored for a write+ * that `skipIfUnchanged` then skips (the target changed underneath to+ * the new bytes) is harmless.*/function commitNoteRewrite(vault: string,@@ -588,17 +693,20 @@ function commitNoteRewrite(contents: string,op: NoteWriteOp,opts: ApplyWriteBatchOptions,-): NoteWriteAudit {- mkdirSync(dirname(target.abs), { recursive: true });+): CommittedNoteRewrite {+ if (before === contents) return { wrote: false };storeBeforeImage(vault, before);- atomicWriteFileSync(target.abs, contents);- return recordNoteWrite(vault, {+ mkdirSync(dirname(target.abs), { recursive: true });+ const wrote = atomicWriteFileSync(target.abs, contents, { skipIfUnchanged: true });+ if (!wrote) return { wrote: false };+ const audit = recordNoteWrite(vault, {op,target: target.relPath,before: { bytes: before },after: { bytes: contents },...(opts.configPath !== undefined ? { configPath: opts.configPath } : {}),});+ return { wrote: true, ...audit };}/** Accepted apply-evidence result values, for phase-1 validation. */diff --git a/src/core/brain/write-path-advisory.ts b/src/core/brain/write-path-advisory.tsnew file mode 100644index 00000000..42e97e60--- /dev/null+++ b/src/core/brain/write-path-advisory.ts@@ -0,0 +1,101 @@+/**+ * Write-time absolute-path advisory (p4-silent-failure-hardening, Task 3).+ *+ * A note body embedding an absolute home path used to land with no word+ * about it: the egress redactor works on OUTBOUND tool output, the+ * hygiene scanner runs CI-only over the repo tree, and the write seam+ * - the one moment the authoring agent is listening - said nothing. This+ * module closes that hole the same way the write-time page lint does:+ * an advisory computed at the write, carried on the receipt, never a+ * refusal. A write that embeds a home path still lands; the receipt says+ * so, and the next call can do better.+ *+ * The grammar is {@link scanText} from core/hygiene/hardcoded-paths.ts,+ * reused AS-IS rather than widened: its detectors (POSIX and Windows home+ * prefixes naming a specific account), its placeholder-segment filter+ * (`alice`, `user`, `you`, ... are intended examples, not leaks), its+ * home-prefix scope, and the `hygiene:allow-path` escape hatch are all+ * the hygiene module's decisions and stay there. Widening any of them is+ * a hygiene-module change that this lane must not shadow.+ *+ * Scope: the AUTHORED content of one call (a create's body, an update's+ * replacement body, an append's text) - the bytes this caller is asking+ * to put into the vault. Body only: frontmatter values are never+ * scanned, whether or not the caller named them, and bytes already on+ * disk are not this call's authorship either. Log-line and evidence ops are machine-composed, not authored,+ * and never reach this module.+ *+ * Shape, and the two rules the receipt inherits from page-lint:+ *+ * - byte-identical-when-absent - a clean scan contributes NO field,+ * so a receipt for a clean write is exactly the receipt that shipped+ * before this module existed;+ * - identifiers and integers only - a finding names the vault-relative+ * page, the 1-based line within the authored text, and the detector.+ * The matched path fragment (an operator home directory) is+ * deliberately dropped, the same leak `writeFrontmatterAtomic`'s+ * error re-wrap closes.+ *+ * The scan is over the authored string alone, so `line` indexes THAT+ * string - line 1 is the first line the caller sent, not the first line+ * of the file it lands in.+ */++import { scanText, type HardcodedPathDetector } from "../hygiene/hardcoded-paths.ts";++/** Key under which the advisory rides a note-write result or MCP receipt. */+export const WRITE_PATH_ADVISORY_KEY = "path_advisory";++/** One hit, composed from identifiers and integers only. */+export interface WritePathAdvisoryFinding {+ /** Vault-relative page the authored content lands in. */+ readonly page: string;+ /** 1-based line within the authored text (not within the note). */+ readonly line: number;+ /** Which hygiene detector fired. */+ readonly detector: HardcodedPathDetector;+}++/** The advisory payload: every hit, with the total a reader can count on. */+export interface WritePathAdvisoryReport {+ readonly total: number;+ readonly findings: ReadonlyArray<WritePathAdvisoryFinding>;+}++/**+ * The optional advisory field. Absent - the key itself, not an empty+ * report - when the authored content is clean, so consumers pinning the+ * receipt's key set see no change.+ */+export type WritePathAdvisoryField = Readonly<{+ readonly [WRITE_PATH_ADVISORY_KEY]?: WritePathAdvisoryReport;+}>;++/**+ * Scan one call's authored note content for absolute home paths and+ * render the advisory field for a write result targeting `page`.+ *+ * Returns the empty field (no key at all) when there is nothing authored+ * or nothing found, so callers spread this straight into the result:+ * `...writePathAdvisoryField(op.content, target.relPath)`. Never throws,+ * never gates the write - the caller keeps committing regardless of what+ * this finds.+ */+export function writePathAdvisoryField(+ authored: string | undefined,+ page: string,+): WritePathAdvisoryField {+ if (authored === undefined || authored.length === 0) return {};+ const findings = scanText(authored, page).map((finding) => ({+ page,+ line: finding.line,+ detector: finding.detector,+ }));+ if (findings.length === 0) return {};+ return Object.freeze({+ [WRITE_PATH_ADVISORY_KEY]: Object.freeze({+ total: findings.length,+ findings: Object.freeze(findings),+ }),+ });+}diff --git a/src/core/redactor.ts b/src/core/redactor.tsindex 55a171da..b7779459 100644--- a/src/core/redactor.ts+++ b/src/core/redactor.ts@@ -230,6 +230,27 @@ const JSON_ENTRY_RE = new RegExp(// and only replace the token portion.const BEARER_RE = /\b(Bearer\s+)([A-Za-z0-9._\-+/=]+)/gi;+// A bare JSON Web Token: three base64url segments joined by dots. The+// shape carries no key=value assignment for the passes above and no+// vendor prefix for the token pass, and the canonical 20-character+// header slips HIGH_ENTROPY_TOKEN_RE's 24-character gate - so under the+// token pass the payload and signature were eaten while the header+// stayed, announcing a credential and naming its algorithm. Default-on:+// a JWT is a bearer credential wherever it appears, not only where a+// key names it.+//+// The header prefix anchors the match, while matching any three+// dot-separated base64url runs would also claim ordinary dotted+// identifiers. A compact header `{"` encodes to `eyJ`; a header written+// with whitespace after the brace encodes to `eyA` (`{ `), `ewo` (`{\n`),+// `ew0` (`{\r`) or `ewk` (`{\t`), so all five open the match. The header+// segment must be at least 12 characters in total (a real header is far+// longer), which keeps short dotted words such as `ewok.a.b` prose.+// Segments are bounded (64 KiB each, room for a large claims payload),+// so the pass stays linear on large inputs.+const JWT_RE =+ /\b(?:eyJ|eyA|ewo|ew0|ewk)[A-Za-z0-9_-]{9,65533}(?:\.[A-Za-z0-9_-]{4,65536}){2}(?![A-Za-z0-9_-])/g;+// ----- Infra-topology detectors (opt-in via `redactInfra`) ------------------//// These scrub network coordinates that carry no key=value shape, so the@@ -246,7 +267,30 @@ const IPV4 = `${IPV4_OCTET}(?:\\.${IPV4_OCTET}){3}`;// `scheme://user:pass@host` — strip the embedded credentials but keep the// scheme and `@host` for readability. Run first so the host that follows// is still available to the host/port passes below.-const BASIC_AUTH_URL_RE = /\b([a-zA-Z][a-zA-Z0-9+.-]*:\/\/)([^\s/:@]+):([^\s/@]+)@/g;+//+// The password class is `[^\s@]+`, not `[^\s/@]+`: a generated database+// password routinely carries `/`, and with the old class the regex failed+// the match ENTIRELY - both the user and the password left verbatim, a+// wider leak than if the pass had never run. The username may be EMPTY -+// a password-only authority is a spelling connection strings really use+// (`scheme://:pass@host`) - while the colon between the halves stays+// mandatory and adjacent, so a URL whose path carries an `@` but no+// userinfo colon is untouched. Every class still cannot cross whitespace,+// the `://` scheme anchor and the `@` anchor are kept, and each run is+// bounded by those anchors (a `[^\s@]+` run ends at the first whitespace+// or `@`, deterministically), so the documented linear / no-ReDoS+// property above holds.+//+// Because the password class crosses `/`, a `host:port/path@x` URL would+// read the port colon as the userinfo colon and swallow the host, the port+// and half the path (`http://localhost:5173/@vite/client`). The lookahead+// rejects a colon followed by 1-5 digits and then `/`: that is a port and a+// path, not a password. Two accepted trade-offs, both as on main: a slash+// password that begins with 1-5 digits (`bob:8080/x@host`) is missed, and a+// query or fragment directly after a port with an `@` in it+// (`example.com:443?x@y`) is still read as userinfo - far rarer than a+// password such as `123?secret`, which this keeps redacted.+const BASIC_AUTH_URL_RE = /\b([a-zA-Z][a-zA-Z0-9+.-]*:\/\/)([^\s/:@]*):(?!\d{1,5}\/)([^\s@]+)@/g;// `ipv4:port` — a reachable service endpoint. Redacted whole regardless of// whether the address is public or private (the port is what leaks the@@ -751,6 +795,12 @@ export function scanRawOutput(text: string, opts: RedactRawOutputOptions = {}):// Bearer headers BEFORE the generic colon rule.out = out.replace(BEARER_RE, (_match, prefix: string) => `${prefix}${PLACEHOLDER}`);+ // Bare JWTs BEFORE the opt-in passes: the token pass would eat only the+ // payload and signature segments (the header slips the entropy gate) and+ // leave a half-standing credential, and the URL pass must see a+ // JWT-in-a-password already collapsed.+ out = out.replace(JWT_RE, PLACEHOLDER);+// Indented continuations BEFORE the single-line rule: `token: |` has a// same-line value the colon rule would consume, leaving its block behind.out = out.replace(@@ -955,6 +1005,25 @@ function isPlainContainer(value: object): boolean {return proto === Object.prototype || proto === null;}+/**+ * True for the sibling-pair shape configuration formats use for+ * environment entries: `{ name: "DB_PASSWORD", value: … }`. The credential+ * NAME sits in a SIBLING member, so the key-name rule - which reads only+ * the CURRENT key, and here sees `value`, which says nothing - never+ * fires, and a literal secret rode past the walk untouched.+ *+ * The matcher is the same {@link isSecretKeyName} the object branch+ * applies to a key; no new secret-name vocabulary. An entry without a+ * literal `value` member - the ECS `valueFrom` reference shape - is not a+ * literal secret and stays with the normal walk.+ */+function siblingPairDeclaresSecret(item: unknown): boolean {+ if (typeof item !== "object" || item === null) return false;+ if (!isPlainContainer(item)) return false;+ const record = item as Record<string, unknown>;+ return "value" in record && isSecretKeyName(record["name"]);+}+/*** Redact a JSON-shaped value tree: every string leaf through* {@link redactRawOutput}, and every value whose KEY NAME declares a@@ -984,6 +1053,31 @@ export function redactStructured(if (secretIdentifiers.size < MAX_REPORTED_IDENTIFIERS) secretIdentifiers.add(location);};+ /**+ * Walk a plain object's entries under `location`, marking each value+ * secret when `secretKey` says its key declares one. One loop for both+ * positions that walk an object - the object branch, and the array+ * branch's sibling `{name, value}` pair - so the key reporting and the+ * location spelling cannot drift between them.+ */+ const walkEntries = (+ source: Record<string, unknown>,+ location: string,+ secretKey: (key: string) => boolean,+ ): Record<string, unknown> => {+ const out: Record<string, unknown> = {};+ let index = 0;+ for (const [key, child] of Object.entries(source)) {+ // The key is reported by POSITION, never by name: the name is the+ // secret in this case.+ if (keyNameCarriesSecret(key)) record(`${location === "" ? "" : location}#${index}`);+ const childLocation = location === "" ? key : `${location}.${key}`;+ out[key] = walk(child, childLocation, secretKey(key), isIdentifierKeyName(key));+ index += 1;+ }+ return out;+ };+const walk = (value: unknown,location: string,@@ -1031,23 +1125,23 @@ export function redactStructured(return scan.text;}if (Array.isArray(value)) {- return value.map((item, index) =>- walk(item, `${location}[${index}]`, false, underIdentifierKey),- );+ return value.map((item, index) => {+ const itemLocation = `${location}[${index}]`;+ if (!siblingPairDeclaresSecret(item)) {+ return walk(item, itemLocation, false, underIdentifierKey);+ }+ // A `{name, value}` pair whose NAME declares a credential: the+ // literal rides in the sibling `value` member, so that member is+ // walked under a secret key. Every other member keeps the normal+ // per-key decision.+ return walkEntries(item as Record<string, unknown>, itemLocation, (key) =>+ key === "value" ? true : isSecretKeyName(key),+ );+ });}if (typeof value === "object" && value !== null) {if (!isPlainContainer(value)) return value;- const out: Record<string, unknown> = {};- let index = 0;- for (const [key, child] of Object.entries(value)) {- // The key is reported by POSITION, never by name: the name is the- // secret in this case.- if (keyNameCarriesSecret(key)) record(`${location === "" ? "" : location}#${index}`);- const childLocation = location === "" ? key : `${location}.${key}`;- out[key] = walk(child, childLocation, isSecretKeyName(key), isIdentifierKeyName(key));- index += 1;- }- return out;+ return walkEntries(value as Record<string, unknown>, location, isSecretKeyName);}return value;};diff --git a/src/core/search/indexer.ts b/src/core/search/indexer.tsindex 03f0e7db..b095aed8 100644--- a/src/core/search/indexer.ts+++ b/src/core/search/indexer.ts@@ -1702,8 +1702,12 @@ export async function indexCheck(* outcomes folded into the report's two. Both non-`read` outcomes are* UNRECORDED - they differ in their reason, not in what they prove -* and neither is a count of zero.+ *+ * Exported for the brain doctor's embeddings-health check, which+ * consolidates this census beside the index counts instead of deriving+ * a private copy whose unrecorded contract could drift from this one.*/-function readPendingVectorCensus(dbPath: string): PendingVectorCensus {+export function readPendingVectorCensus(dbPath: string): PendingVectorCensus {const peek = peekPendingVectorsSync(dbPath);if (peek.kind === "read") {return Object.freeze({diff --git a/src/core/search/store.ts b/src/core/search/store.tsindex cdf53c5f..a9fea1b8 100644--- a/src/core/search/store.ts+++ b/src/core/search/store.ts@@ -810,7 +810,8 @@ export class Store {// ── counts ─────────────────────────────────────────────────────────────────counts(): census.StoreCounts {- return census.counts(this.db, this.config.semantic.model, this.config.semantic.dimension);+ const baseline = census.staleBaseline(this.db, this.config.semantic);+ return census.counts(this.db, baseline.model, baseline.dimension);}/**diff --git a/src/core/search/store/counts.ts b/src/core/search/store/counts.tsindex cb50e9df..0830ebb1 100644--- a/src/core/search/store/counts.ts+++ b/src/core/search/store/counts.ts@@ -9,14 +9,22 @@ import { Database } from "bun:sqlite";import { SearchError } from "../search-error.ts";import { countChunks, countChunksWithoutEmbeddings } from "./chunks.ts";import { countDocuments } from "./documents.ts";-import { peekReadonlyIndex, type IndexPeek } from "./state.ts";+import {+ EMBEDDING_DIMENSION_STATE_KEY,+ getState,+ peekReadonlyIndex,+ type IndexPeek,+} from "./state.ts";import { countEmbeddings, staleEmbeddings } from "./vectors.ts";import {charLengthOverTokenBudget,NON_ASCII_CEILING_COEFFICIENT,textExtent,utf8ByteFloorUnderTokenBudget,+ LOCAL_EMBEDDING_MODEL,} from "../embeddings/signature.ts";+import { LOCAL_DEFAULT_DIMENSION } from "../embeddings/local-provider.ts";+import type { ResolvedEmbeddingConfig } from "../types.ts";export interface StoreCounts {readonly documents: number;@@ -26,6 +34,38 @@ export interface StoreCounts {readonly staleEmbeddings: number;}+/** The embedding pair stored vectors are compared against. */+export interface StaleBaseline {+ readonly model: string | null;+ readonly dimension: number | null;+}++/**+ * Resolve the pair {@link staleEmbeddings} compares against, the way the+ * indexer resolves the active signature: the local embedder is named by+ * its built-in model and default dimension; otherwise the configured+ * dimension wins and the stored one fills in when none is configured.+ * `embedding_dimension` is null unless configured, so reading the+ * configuration alone left the stale count at zero on a default setup.+ */+export function staleBaseline(+ db: Database,+ semantic: Pick<ResolvedEmbeddingConfig, "provider" | "model" | "dimension">,+): StaleBaseline {+ const local = semantic.provider === "local";+ const model = local ? LOCAL_EMBEDDING_MODEL : (semantic.model ?? null);+ if (semantic.dimension !== null && semantic.dimension !== undefined) {+ return { model, dimension: semantic.dimension };+ }+ if (local) return { model, dimension: LOCAL_DEFAULT_DIMENSION };+ const raw = getState(db, EMBEDDING_DIMENSION_STATE_KEY);+ const stored = raw === null ? null : Number(raw);+ return {+ model,+ dimension: stored !== null && Number.isFinite(stored) && stored > 0 ? stored : null,+ };+}+export function counts(db: Database,configuredModel: string | null,diff --git a/src/core/search/store/lifecycle.ts b/src/core/search/store/lifecycle.tsindex 5372d4c8..023e87f4 100644--- a/src/core/search/store/lifecycle.ts+++ b/src/core/search/store/lifecycle.ts@@ -35,6 +35,12 @@ import { loadVecExtension } from "./vectors.ts";import { acquireWriterLock, acquireWriterLockSync } from "./writer-lock.ts";import { closeDatabase } from "../../sqlite-close.ts";import { renameWithRetry } from "../../fs-atomic.ts";+import {+ probeVaultBacking,+ VAULT_BACKING_REMOTENESS,+ type StatfsProbe,+ type VaultBackingVerdict,+} from "../../vault-backing.ts";/** A connection on the index, plus the vec ABI it opened under. */export interface OpenedDatabase {@@ -48,8 +54,84 @@ export interface OpenedWriteDatabase extends OpenedDatabase {readonly release: () => Promise<void>;}-function applyPragmas(db: Database): void {- db.exec("PRAGMA journal_mode = WAL");+/** The journal modes an index open can set. */+type JournalMode = "wal" | "delete";++/** The journal-mode decision the backing verdict calls for. */+export interface JournalModeDecision {+ readonly journalMode: JournalMode;+ /**+ * One stderr line naming the filesystem and the reason WAL is skipped,+ * or `null` when the default mode stands.+ */+ readonly warning: string | null;+}++/**+ * The journal mode an index open sets, decided by the backing probe -+ * pure, so the whole decision is checkable without a real mount.+ *+ * WAL is the default everywhere EXCEPT a backing the probe classifies+ * remote (a named network filesystem): WAL's shared-memory index needs+ * advisory-lock semantics nfs/cifs do not provide, and a WAL file left on+ * such a mount is the corruption story, not a performance tradeoff. On+ * those the open keeps the DELETE mode a fresh sqlite file already has.+ * This is a skip, never a refusal: the index on someone's NFS server was+ * a deliberate deployment, and it must keep opening - slower, honest.+ * Every non-remote answer, including the silent undetermined ones+ * (unknown type, unreadable path, non-Linux), keeps WAL: the absence of a+ * network finding is not evidence for changing the default.+ */+export function journalModeForBacking(+ backing: VaultBackingVerdict,+ dbPath: string,+): JournalModeDecision {+ if (backing.remoteness !== VAULT_BACKING_REMOTENESS.remote) {+ return { journalMode: "wal", warning: null };+ }+ return {+ journalMode: "delete",+ warning:+ `search index at ${dbPath} is backed by ${backing.filesystem}, a network filesystem: ` ++ `WAL journaling is unreliable across hosts, so the index opens with journal mode DELETE ` ++ `(slower writes; no -wal sibling)`,+ };+}++/** Whether the journal-mode switch took, or another connection blocked it. */+type JournalModeSwitch = "set" | "busy";++/**+ * True for SQLite's "another connection holds the file" refusal. Leaving+ * WAL needs exclusive access, so `PRAGMA journal_mode = delete` on an+ * index in WAL fails at once while any other connection has it open -+ * the busy timeout does not apply to that switch.+ */+function isBusyError(e: unknown): boolean {+ const code = (e as { code?: unknown } | null)?.code;+ if (+ typeof code === "string" &&+ (code.startsWith("SQLITE_BUSY") || code.startsWith("SQLITE_LOCKED"))+ ) {+ return true;+ }+ const msg = e instanceof Error ? e.message : String(e);+ return /database is locked|database table is locked/i.test(msg);+}++/**+ * Set the journal mode, keeping the current one when another connection+ * blocks the switch. A blocked switch is a mode the open could not change,+ * not a broken index: every other error still throws.+ */+function applyPragmas(db: Database, journalMode: JournalMode): JournalModeSwitch {+ let switched: JournalModeSwitch = "set";+ try {+ db.exec(`PRAGMA journal_mode = ${journalMode}`);+ } catch (e) {+ if (!isBusyError(e)) throw e;+ switched = "busy";+ }db.exec("PRAGMA foreign_keys = ON");db.exec("PRAGMA synchronous = NORMAL");// Wait briefly for a concurrent writer (e.g. an indexer holding the WAL@@ -57,6 +139,18 @@ function applyPragmas(db: Database): void {// for the opportunistic query-cache writes a read-mode connection makes// during search (v0.20.0); search itself also degrades gracefully.db.exec("PRAGMA busy_timeout = 5000");+ return switched;+}++/** Index paths whose blocked journal-mode switch this process already announced. */+const busySwitchWarned = new Set<string>();++function busySwitchWarning(dbPath: string, journalMode: JournalMode): string {+ const current = journalMode === "delete" ? "WAL" : "DELETE";+ return (+ `search index at ${dbPath}: journal mode stays ${current} until other connections close ` ++ `(the switch to ${journalMode.toUpperCase()} needs exclusive access; it is retried on the next write open)`+ );}/**@@ -327,7 +421,11 @@ function runWriteOpenIntegrityGate(db: Database, dbPath: string): void {* older schema is refused rather than upgraded, because a reader holds* no writer lock and must not migrate under a concurrent writer.*/-export function openReadDatabase(config: ResolvedSearchConfig, loadVec: boolean): OpenedDatabase {+export function openReadDatabase(+ config: ResolvedSearchConfig,+ loadVec: boolean,+ opts: OpenWriteDatabaseOptions = {},+): OpenedDatabase {if (!existsSync(config.dbPath)) {throw new SearchError("INDEX_MISSING",@@ -342,7 +440,16 @@ export function openReadDatabase(config: ResolvedSearchConfig, loadVec: boolean)throw new SearchError("INDEX_UNREADABLE", `cannot open ${config.dbPath}: ${msg}`);}try {- applyPragmas(db);+ // The same decision the write open makes, and it MUST be the same:+ // journal_mode is persistent file state, so an unconditional WAL here+ // would flip a remote-backed index back to WAL behind the write+ // open's back. Silent by design: read opens happen per query, and the+ // skip was already announced by the write open that created or+ // upgraded the index. A switch another connection blocks keeps the+ // current mode: a read is correct in either, and the conversion is+ // the write open's to finish under the writer lock.+ const backing = probeVaultBacking(dirname(config.dbPath), { statfs: opts.statfs });+ applyPragmas(db, journalModeForBacking(backing, config.dbPath).journalMode);ensureFts5(db);assertSchemaIsCurrent(db, config.dbPath);assertNoRecordedIntegrityFault(db, config.dbPath);@@ -360,6 +467,15 @@ export function openReadDatabase(config: ResolvedSearchConfig, loadVec: boolean)}}+/** Options on the read and write opens, all defaulted; the seam tests need. */+export interface OpenWriteDatabaseOptions {+ /**+ * Injected statfs probe for the remoteness classification. Defaults to+ * `node:fs`'s `statfsSync`; simulating an NFS backing needs no mount.+ */+ readonly statfs?: StatfsProbe;+}+/*** Open the index for writing: create the file if absent, take the* exclusive writer lock, then migrate to the current schema. The@@ -368,8 +484,14 @@ export function openReadDatabase(config: ResolvedSearchConfig, loadVec: boolean)export async function openWriteDatabase(config: ResolvedSearchConfig,loadVec: boolean,+ opts: OpenWriteDatabaseOptions = {},): Promise<OpenedWriteDatabase> {mkdirSync(dirname(config.dbPath), { recursive: true });+ // The backing decides the journal mode before any connection exists.+ // The probed path is the directory: the index file itself may not exist+ // yet, and the mode decision is about the FILESYSTEM under it.+ const backing = probeVaultBacking(dirname(config.dbPath), { statfs: opts.statfs });+ const decision = journalModeForBacking(backing, config.dbPath);if (!existsSync(config.dbPath)) {const seed = new Database(config.dbPath);closeDatabase(seed);@@ -387,7 +509,16 @@ export async function openWriteDatabase(}try {- applyPragmas(db);+ const switched = applyPragmas(db, decision.journalMode);+ if (switched === "busy" && !busySwitchWarned.has(config.dbPath)) {+ busySwitchWarned.add(config.dbPath);+ // eslint-disable-next-line no-console+ console.error(busySwitchWarning(config.dbPath, decision.journalMode));+ }+ if (decision.warning !== null) {+ // eslint-disable-next-line no-console+ console.error(decision.warning);+ }applyMigrations(db, { ftsTokenize: config.ftsTokenize });runWriteOpenIntegrityGate(db, config.dbPath);ensureFts5(db);diff --git a/src/core/search/store/vectors.ts b/src/core/search/store/vectors.tsindex a3c46f31..cf3ea875 100644--- a/src/core/search/store/vectors.ts+++ b/src/core/search/store/vectors.ts@@ -236,15 +236,27 @@ export function countEmbeddings(db: Database): number {return db.query<{ n: number }, []>("SELECT COUNT(*) AS n FROM embeddings").get()?.n ?? 0;}-/** Embeddings whose `model`/`dimension` no longer match the current config. */+/**+ * Embeddings whose `model`/`dimension` no longer match the current config.+ *+ * With no model there is no baseline, and the count is 0 by convention -+ * a caller that must not read that as healthy checks the model first.+ * With a model but no dimension (none configured, none stored), the+ * vectors are compared by model alone: a model switch on a default+ * configuration is exactly the case a dimension-gated zero used to hide.+ */export function staleEmbeddings(db: Database,model: string | null,dimension: number | null,): number {- if (!model || !dimension) {- // No baseline to compare against: 0 stale by convention.- return 0;+ if (!model) return 0;+ if (!dimension) {+ return (+ db+ .query<{ c: number }, [string]>("SELECT count(*) AS c FROM embeddings WHERE model != ?")+ .get(model)?.c ?? 0+ );}const row = db.query<{ c: number }, [string, number]>(diff --git a/src/core/vault-backing.ts b/src/core/vault-backing.tsindex 9a5ca143..aa55449b 100644--- a/src/core/vault-backing.ts+++ b/src/core/vault-backing.ts@@ -32,10 +32,28 @@* {@link VAULT_BACKING.undetermined}, with a reason from a SEPARATE* vocabulary so a guard can never read a reason back as a verdict.*- * The probe is Linux-only by construction. `statfs(2)`'s `f_type` is a- * Linux filesystem magic number; the same field on macOS and the BSDs- * means something else, so every other platform answers- * `probe_unsupported` rather than being handed a number to misread.+ * Beside survival the probe answers one more axis, REMOTENESS: whether the+ * backing is a NAMED network filesystem. It is narrow by the same+ * discipline. Only the named network filesystems (nfs, cifs/smb2/smbfs,+ * ceph, afs, 9p) and, on Windows, a UNC path classify remote, because+ * those are the backings whose storage is provably shared with another host and whose+ * locking semantics SQLite's WAL journal mode depends on and does not+ * get. Fuse stays non-remote: what backs a fuse mount is unknowable from+ * its type, and failing safe there would warn every user-space mount.+ * Everything the probe could not read - unknown type, unreadable path,+ * platform without statfs - is {@link VAULT_BACKING_REMOTENESS.nonRemote}+ * and SILENT: the absence of a network finding is never rendered as a+ * finding that the storage is local.+ *+ * The DEFAULT probe is Linux-only by construction. `statfs(2)`'s `f_type`+ * is a Linux filesystem magic number; the same field on macOS and the BSDs+ * means something else, so a call that relies on the default `statfsSync`+ * answers `probe_unsupported` on every other platform rather than being+ * handed a number to misread. That veto guards the default facility's+ * meaning, not the probe itself: an explicitly injected `statfs` carries+ * its own meaning - the caller vouches for what it returns - so an+ * injection implies the probeable platform unless the caller pins+ * `platform` explicitly.** Pure and injectable: the platform and the `statfs` call are parameters,* so every state below is reachable in a unit test on any host.@@ -120,6 +138,48 @@ export function isVaultBackingUndeterminedReason();}+// ----- The second axis: remoteness ------------------------------------------++/**+ * Whether the filesystem under a path is a NAMED network filesystem.+ *+ * A second axis beside {@link VAULT_BACKING}, not a new member on it.+ * Durability answers "do the bytes survive this process and this reboot"+ * - on nfs and cifs they do, which is why both stay `durable`. Remoteness+ * answers "is this storage shared with another host", which is the+ * question a consumer of cross-host-unsafe machinery (SQLite's WAL+ * journal mode and its shared-memory index) needs and which no durability+ * verdict can answer.+ *+ * Two members, and `nonRemote` is the one that earns the vocabulary: it+ * is deliberately NOT named `local`, because unknown and probe-unsupported+ * backings answer non-remote too, and "we found no network filesystem" is+ * not evidence that the storage is local.+ */+export const VAULT_BACKING_REMOTENESS = Object.freeze({+ /** Backed by a named network filesystem (nfs, cifs). */+ remote: "remote",+ /** No network filesystem was identified. Not a claim that the storage is local. */+ nonRemote: "non_remote",+} as const);++/** Closed union over {@link VAULT_BACKING_REMOTENESS}. */+export type VaultBackingRemoteness =+ (typeof VAULT_BACKING_REMOTENESS)[keyof typeof VAULT_BACKING_REMOTENESS];++/** Membership list for {@link VAULT_BACKING_REMOTENESS}. */+export const VAULT_BACKING_REMOTENESS_STATES: ReadonlyArray<VaultBackingRemoteness> = Object.freeze(+ [VAULT_BACKING_REMOTENESS.remote, VAULT_BACKING_REMOTENESS.nonRemote],+);++/** Narrow a string read back off disk or across a tool boundary. */+export function isVaultBackingRemoteness(value: unknown): value is VaultBackingRemoteness {+ return (+ typeof value === "string" &&+ (VAULT_BACKING_REMOTENESS_STATES as ReadonlyArray<string>).includes(value)+ );+}+// ----- The probe ------------------------------------------------------------/**@@ -133,6 +193,12 @@ export interface VaultBackingVerdict {readonly filesystem: string | null;/** Why no verdict was reached, or `null` when one was. */readonly reason: VaultBackingUndeterminedReason | null;+ /**+ * The second axis: whether the identified filesystem is a named network+ * one. `nonRemote` whenever no filesystem was identified at all - the+ * absence of a network finding, never a claim of local storage.+ */+ readonly remoteness: VaultBackingRemoteness;/** One line naming what was probed and what came back. */readonly detail: string;}@@ -141,7 +207,11 @@ export interface VaultBackingVerdict {export type StatfsProbe = (path: string) => { readonly type: number | bigint };export interface VaultBackingOptions {- /** Defaults to `process.platform`, following the house injection style. */+ /**+ * Defaults to `process.platform` - or to the one probeable platform when+ * `statfs` is injected, since an injected probe carries its own meaning+ * wherever it runs. An explicit value always wins.+ */readonly platform?: string;/** Defaults to `node:fs`'s `statfsSync`. */readonly statfs?: StatfsProbe;@@ -151,7 +221,8 @@ export interface VaultBackingOptions {const PROBEABLE_PLATFORM = "linux";/**- * Linux filesystem magic numbers, each mapped to the state it proves.+ * Linux filesystem magic numbers, each mapped to the state and remoteness+ * they prove.** Only filesystems whose durability follows FROM THE TYPE are listed. A* type that is not here is `fs_type_unknown` rather than assumed durable -@@ -162,35 +233,137 @@ const PROBEABLE_PLATFORM = "linux";* Network filesystems are listed as durable deliberately. The vault living* on someone else's server is an availability question, not a survival* one: the bytes outlive this process and this reboot, which is the whole- * claim the state makes.+ * claim the state makes. Sharing that server with other hosts is a+ * DIFFERENT question, answered by the second axis: the network filesystems carry+ * `remote` because their storage is provably cross-host and their locking+ * semantics are the ones SQLite's WAL journal mode cannot rely on. Fuse+ * carries `nonRemote` deliberately too: what backs a fuse mount is+ * unknowable from the type, and failing safe there would put the warning+ * on every user-space mount rather than on the network ones.*/-const FS_MAGIC: ReadonlyMap<number, { readonly name: string; readonly state: VaultBackingState }> =- new Map([- // Memory-backed: provably lost at reboot.- [0x01021994, { name: "tmpfs", state: VAULT_BACKING.volatile }],- [0x858458f6, { name: "ramfs", state: VAULT_BACKING.volatile }],- // Union mount: a container's root by default, survival unknown.- [0x794c7630, { name: "overlayfs", state: VAULT_BACKING.layered }],- [0xaad7aaea, { name: "aufs", state: VAULT_BACKING.layered }],- // Storage that outlives the process and a reboot.- [0xef53, { name: "ext4", state: VAULT_BACKING.durable }],- [0x9123683e, { name: "btrfs", state: VAULT_BACKING.durable }],- [0x58465342, { name: "xfs", state: VAULT_BACKING.durable }],- [0x2fc12fc1, { name: "zfs", state: VAULT_BACKING.durable }],- [0xf2f52010, { name: "f2fs", state: VAULT_BACKING.durable }],- [0x6969, { name: "nfs", state: VAULT_BACKING.durable }],- [0xff534d42, { name: "cifs", state: VAULT_BACKING.durable }],- [0x65735546, { name: "fuse", state: VAULT_BACKING.durable }],- [0x4d44, { name: "vfat", state: VAULT_BACKING.durable }],- [0x52654973, { name: "reiserfs", state: VAULT_BACKING.durable }],- ]);+const FS_MAGIC: ReadonlyMap<+ number,+ {+ readonly name: string;+ readonly state: VaultBackingState;+ readonly remote: VaultBackingRemoteness;+ }+> = new Map([+ // Memory-backed: provably lost at reboot.+ [+ 0x01021994,+ { name: "tmpfs", state: VAULT_BACKING.volatile, remote: VAULT_BACKING_REMOTENESS.nonRemote },+ ],+ [+ 0x858458f6,+ { name: "ramfs", state: VAULT_BACKING.volatile, remote: VAULT_BACKING_REMOTENESS.nonRemote },+ ],+ // Union mount: a container's root by default, survival unknown.+ [+ 0x794c7630,+ { name: "overlayfs", state: VAULT_BACKING.layered, remote: VAULT_BACKING_REMOTENESS.nonRemote },+ ],+ [+ 0xaad7aaea,+ { name: "aufs", state: VAULT_BACKING.layered, remote: VAULT_BACKING_REMOTENESS.nonRemote },+ ],+ // Storage that outlives the process and a reboot.+ [+ 0xef53,+ { name: "ext4", state: VAULT_BACKING.durable, remote: VAULT_BACKING_REMOTENESS.nonRemote },+ ],+ [+ 0x9123683e,+ { name: "btrfs", state: VAULT_BACKING.durable, remote: VAULT_BACKING_REMOTENESS.nonRemote },+ ],+ [+ 0x58465342,+ { name: "xfs", state: VAULT_BACKING.durable, remote: VAULT_BACKING_REMOTENESS.nonRemote },+ ],+ [+ 0x2fc12fc1,+ { name: "zfs", state: VAULT_BACKING.durable, remote: VAULT_BACKING_REMOTENESS.nonRemote },+ ],+ [+ 0xf2f52010,+ { name: "f2fs", state: VAULT_BACKING.durable, remote: VAULT_BACKING_REMOTENESS.nonRemote },+ ],+ // Cross-host storage: durable AND remote, the two axes independently.+ [0x6969, { name: "nfs", state: VAULT_BACKING.durable, remote: VAULT_BACKING_REMOTENESS.remote }],+ [+ 0xff534d42,+ { name: "cifs", state: VAULT_BACKING.durable, remote: VAULT_BACKING_REMOTENESS.remote },+ ],+ // What a modern `mount -t cifs` (SMB2/SMB3) share reports.+ [+ 0xfe534d42,+ { name: "smb2", state: VAULT_BACKING.durable, remote: VAULT_BACKING_REMOTENESS.remote },+ ],+ [+ 0x517b,+ { name: "smbfs", state: VAULT_BACKING.durable, remote: VAULT_BACKING_REMOTENESS.remote },+ ],+ [+ 0x00c36400,+ { name: "ceph", state: VAULT_BACKING.durable, remote: VAULT_BACKING_REMOTENESS.remote },+ ],+ // AFS_SUPER_MAGIC and kAFS's AFS_FS_MAGIC.+ [+ 0x5346414f,+ { name: "afs", state: VAULT_BACKING.durable, remote: VAULT_BACKING_REMOTENESS.remote },+ ],+ [+ 0x6b414653,+ { name: "afs", state: VAULT_BACKING.durable, remote: VAULT_BACKING_REMOTENESS.remote },+ ],+ // 9p/v9fs, which also backs WSL2's drvfs mounts of the Windows drives.+ [+ 0x01021997,+ { name: "9p", state: VAULT_BACKING.durable, remote: VAULT_BACKING_REMOTENESS.remote },+ ],+ // Backing unknowable from the type; non-remote rather than every+ // user-space mount being warned about.+ [+ 0x65735546,+ { name: "fuse", state: VAULT_BACKING.durable, remote: VAULT_BACKING_REMOTENESS.nonRemote },+ ],+ [+ 0x4d44,+ { name: "vfat", state: VAULT_BACKING.durable, remote: VAULT_BACKING_REMOTENESS.nonRemote },+ ],+ [+ 0x52654973,+ { name: "reiserfs", state: VAULT_BACKING.durable, remote: VAULT_BACKING_REMOTENESS.nonRemote },+ ],+]);function hex(type: number): string {return `0x${type.toString(16)}`;}function undetermined(reason: VaultBackingUndeterminedReason, detail: string): VaultBackingVerdict {- return { state: VAULT_BACKING.undetermined, filesystem: null, reason, detail };+ return {+ state: VAULT_BACKING.undetermined,+ filesystem: null,+ reason,+ // Silent: nothing was read, so no remoteness finding is claimed.+ remoteness: VAULT_BACKING_REMOTENESS.nonRemote,+ detail,+ };+}++/** The platform whose UNC paths name a network share by their spelling. */+const WINDOWS_PLATFORM = "win32";++/**+ * A Windows UNC path: `\\server\share\...`, `//server/share/...` or the+ * extended-length `\\?\UNC\server\share\...`. The local+ * extended-length and device forms (`\\?\C:\...`, `\\.\C:\...`) are+ * not shares.+ */+function isUncPath(path: string): boolean {+ if (/^[\\/]{2}\?[\\/]UNC[\\/][^\\/]+[\\/][^\\/]+/i.test(path)) return true;+ return /^[\\/]{2}(?![?.][\\/])[^\\/]+[\\/][^\\/]+/.test(path);}/**@@ -204,7 +377,28 @@ export function probeVaultBacking(path: string,opts: VaultBackingOptions = {},): VaultBackingVerdict {- const platform = opts.platform ?? process.platform;+ // The veto below guards the DEFAULT facility's meaning, not the probe+ // itself: `statfsSync`'s f_type is a Linux magic number only on Linux, so+ // a call that relies on it answers probe_unsupported elsewhere rather than+ // being handed a number to misread. An injected statfs carries its own+ // meaning - the caller vouches for what it returns - so an injection+ // implies the probeable platform unless the caller pins `platform`.+ const platform =+ opts.platform ?? (opts.statfs !== undefined ? PROBEABLE_PLATFORM : process.platform);+ // Windows exposes no filesystem-type facility this build reads, but a+ // UNC path names a network share by its spelling alone: the one cheap,+ // certain remoteness signal there. Every other Windows path, and every+ // path on macOS (no stable `f_type`, no `f_fstypename` in Node), stays+ // probe_unsupported.+ if (platform === WINDOWS_PLATFORM && isUncPath(path)) {+ return {+ state: VAULT_BACKING.durable,+ filesystem: "unc-share",+ reason: null,+ remoteness: VAULT_BACKING_REMOTENESS.remote,+ detail: `${path} is a UNC path, which names a network share`,+ };+ }if (platform !== PROBEABLE_PLATFORM) {return undetermined(VAULT_BACKING_UNDETERMINED_REASON.probeUnsupported,@@ -235,6 +429,7 @@ export function probeVaultBacking(state: known.state,filesystem: known.name,reason: null,+ remoteness: known.remote,detail: `${path} is backed by ${known.name} (${hex(type)})`,};}diff --git a/src/core/vault.ts b/src/core/vault.tsindex 7af6b687..cea0d223 100644--- a/src/core/vault.ts+++ b/src/core/vault.ts@@ -449,6 +449,16 @@ export interface WriteFrontmatterAtomicOptions {* (CLI + MCP server, multiple agents) → exclusive `link(2)` instead* of the TOCTOU-prone `existsSync` + `writeFileSync` pair.*+ * Returns whether bytes were written. The overwrite branch skips a+ * byte-identical target: the rename pipeline lands a fresh inode on every+ * call, so a re-render that produced the exact bytes already on disk used+ * to bump the file's mtime anyway, and every mtime consumer downstream+ * (recency ranking, validity windows, vault-delta counts) read that+ * phantom bump as a change. Callers that ignore the return keep working+ * unchanged; the exclusive-create branch always writes or throws, so it+ * reports `true` and gains no compare step - a target that exists is a+ * refusal, never a skip.+ ** On a collision the function always throws a {@link FileAlreadyExistsError}* carrying `code === "EEXIST"`, the absolute path, and the `kind` when one* was supplied. Only the message differs between the two call shapes:@@ -471,14 +481,14 @@ export function writeFrontmatterAtomic(metadata: FrontmatterMap,body: string,opts: WriteFrontmatterAtomicOptions = {},-): void {+): boolean {const contents = formatFrontmatter(metadata, body);if (opts.overwrite) {- atomicWriteFileSync(path, contents);- return;+ return atomicWriteFileSync(path, contents, { skipIfUnchanged: true });}try {atomicCreateFileSyncExclusive(path, contents);+ return true;} catch (err) {if (opts.existsErrorKind && isFileAlreadyExists(err)) {// `vaultRelative` rather than a hand-rolled `startsWith(vault + "/")`diff --git a/src/mcp/brain/ingest-tools.ts b/src/mcp/brain/ingest-tools.tsindex 365fa71b..6589f484 100644--- a/src/mcp/brain/ingest-tools.ts+++ b/src/mcp/brain/ingest-tools.ts@@ -227,15 +227,33 @@ export function serializeBatchPlan(plan: BatchPlan): Record<string, unknown> {total_bytes: plan.totalBytes,skipped: [...plan.skipped],// Only emitted when the extractable gate skipped something, so a plan with- // no extractable declaration serializes byte-identically to before.+ // no extractable declaration serializes byte-identically to before. The+ // reason is the typed token (P4) and `detail` carries the schema_type+ // value behind it, so a reader can check the skip without re-reading....(plan.skippedNonExtractable.length > 0? {skipped_non_extractable: plan.skippedNonExtractable.map((s) => ({path: s.path,reason: s.reason,+ detail: s.detail,})),}: {}),+ // Per-reason counts (P4), emitted only when something was skipped.+ ...(Object.keys(plan.skipReasonCounts).length > 0+ ? { skip_reason_counts: plan.skipReasonCounts }+ : {}),+ // Per-extension unclassifiable counts (P4), emitted only when the walk+ // dropped at least one non-ingestible file, so a tree that is all+ // ingestible serializes byte-identically to before.+ ...(plan.unclassifiable.total > 0+ ? {+ unclassifiable: {+ total: plan.unclassifiable.total,+ by_extension: plan.unclassifiable.byExtension,+ },+ }+ : {}),// Only emitted when the repository's own ignore files carried a malformed// pattern (t_4b2bd8f7), so a tree that declares none - or declares only// well-formed patterns - serializes byte-identically to before.diff --git a/src/mcp/brain/notes-tools.ts b/src/mcp/brain/notes-tools.tsindex 77633701..8dd6cca8 100644--- a/src/mcp/brain/notes-tools.ts+++ b/src/mcp/brain/notes-tools.ts@@ -42,6 +42,10 @@ import {} from "../../core/brain/write-batch.ts";import { nextCommandField } from "../../core/brain/next-step.ts";import { lintWrittenPages, pageLintField, type PageLintField } from "../../core/brain/page-lint.ts";+import {+ WRITE_PATH_ADVISORY_KEY,+ type WritePathAdvisoryField,+} from "../../core/brain/write-path-advisory.ts";import { WRITE_BINDING_REFUSED_CODE } from "../../core/write-binding/index.ts";import { isFrontmatterKey } from "../../core/vault.ts";import { rethrowVaultFrozen } from "../frozen-refusal.ts";@@ -350,11 +354,14 @@ async function toolBrainUpdateNote(};const result = runSingleWrite(ctx, op, "brain_update_note");// The flag comes off the kernel result rather than being restated here:- // one fact, one source.- return noteWriteResult(ctx, [result.path], {+ // one fact, one source. A skipped update wrote no bytes, so it names no+ // page for the lint and carries no audit half - the same shape a+ // skipped create spells.+ return noteWriteResult(ctx, result.updated ? [result.path] : [], {updated: result.updated,path: result.path,- ...auditFields(result),+ ...(result.updated ? auditFields(result) : {}),+ ...pathAdvisoryFields(result),});}@@ -370,10 +377,13 @@ async function toolBrainAppendNote(const content = coerceStr(args, "content", true)!;const op: AppendNoteOperation = { kind: "append_note", path, content };const result = runSingleWrite(ctx, op, "brain_append_note");- return noteWriteResult(ctx, [result.path], {+ // Skipped-append receipts follow the skipped-update shape: no bytes+ // landed, so no page to lint and no audit half to mistake.+ return noteWriteResult(ctx, result.appended ? [result.path] : [], {appended: result.appended,path: result.path,- ...auditFields(result),+ ...(result.appended ? auditFields(result) : {}),+ ...pathAdvisoryFields(result),});}@@ -394,6 +404,18 @@ function auditFields(audit: NoteWriteAudit): Record<string, unknown> {};}+/**+ * The absolute-path advisory a kernel result computed over the call's+ * authored content, passed through under its own key. Byte-identical-+ * when-absent: a clean write contributes nothing, exactly as the page+ * lint does at the envelope level. The kernel already advises - this+ * only lets the receipt say it.+ */+function pathAdvisoryFields(result: NoteOpResult): WritePathAdvisoryField {+ const report = result[WRITE_PATH_ADVISORY_KEY];+ return report === undefined ? {} : { [WRITE_PATH_ADVISORY_KEY]: report };+}+/** The kernel results that name a note file: exactly the three note ops. */type NoteOpResult = Extract<WriteBatchOpResult, { readonly path: string }>;diff --git a/src/mcp/brain/write-batch-tools.ts b/src/mcp/brain/write-batch-tools.tsindex 32d62a3f..edab9e57 100644--- a/src/mcp/brain/write-batch-tools.ts+++ b/src/mcp/brain/write-batch-tools.ts@@ -166,6 +166,22 @@ function serializeResult(return { ...result };}+/**+ * The note ops that actually put bytes on disk: a create always wrote,+ * while an update or append whose flag says `false` skipped a+ * byte-identical target and carries a path but no commit. The receipt's+ * lint reads only the committed half.+ */+function committedNotePage(+ result: WriteBatchResult["results"][number],+): result is Extract<WriteBatchResult["results"][number], { readonly path: string }> {+ return (+ (result.kind === "create_note" && result.created) ||+ (result.kind === "update_note" && result.updated) ||+ (result.kind === "append_note" && result.appended)+ );+}+async function toolBrainWriteBatch(ctx: ServerContext,args: Record<string, unknown>,@@ -188,11 +204,13 @@ async function toolBrainWriteBatch(} catch (err) {throw writeBatchErrorToMcp(err, "brain_write_batch");}- // The pages this batch authored, in commit order. Log-writing ops name+ // The pages this batch committed, in commit order. Log-writing ops name// no note path: their line is machine-composed, not authored, so there- // is nothing for the page lint to judge. One report covers the whole- // batch, with the basename index and schema pack built ONCE.- const writtenPages = batch.results.filter((r) => "path" in r).map((r) => r.path);+ // is nothing for the page lint to judge. A byte-identical rewrite that+ // skipped wrote nothing either, so it names no page - the same rule the+ // single-write receipts follow. One report covers the whole batch, with+ // the basename index and schema pack built ONCE.+ const writtenPages = batch.results.filter(committedNotePage).map((r) => r.path);return noteWriteResult(ctx, writtenPages, {applied: batch.applied,results: batch.results.map((r) => serializeResult(ctx, r)),diff --git a/tests/cli/brain-batch-plan-ignore-warnings.test.ts b/tests/cli/brain-batch-plan-ignore-warnings.test.tsindex db91466f..68ccbfd3 100644--- a/tests/cli/brain-batch-plan-ignore-warnings.test.ts+++ b/tests/cli/brain-batch-plan-ignore-warnings.test.ts@@ -167,3 +167,78 @@ describe("the CLI and MCP batch-plan payloads come from one serializer", () => {expect(cli).toEqual(mcp);});});++/** A vault whose schema pack gates ingest to `paper` pages only. */+function gateToPaper(): void {+ mkdirSync(join(vault, "Brain"), { recursive: true });+ writeFileSync(+ join(vault, "Brain", "_brain.yaml"),+ "schema_version: 1\nschema:\n page_types:\n - paper\n - memo\n extractable:\n - paper\n",+ "utf8",+ );+}++describe("o2b brain batch-plan skip-reason counts (P4)", () => {+ test("the human surface renders the per-reason counts", async () => {+ write("mono/a.md", "---\nschema_type: paper\n---\nbody\n");+ write("mono/b.md", "---\nschema_type: memo\n---\nbody\n");+ gateToPaper();++ const res = await runCli(["brain", "batch-plan", "mono"], { env: ENV() });+ expect(res.returncode).toBe(0);+ expect(res.stdout).toContain("schema-type-not-extractable=1");+ expect(res.stdout).toContain("memo");+ });++ test("the --json payload carries the typed token and the counts", async () => {+ write("mono/a.md", "---\nschema_type: paper\n---\nbody\n");+ write("mono/b.md", "---\nschema_type: memo\n---\nbody\n");+ gateToPaper();++ const res = await runCli(["brain", "batch-plan", "mono", "--json"], { env: ENV() });+ expect(res.returncode).toBe(0);+ const plan = JSON.parse(res.stdout);+ expect(plan.skipped_non_extractable).toEqual([+ { path: "mono/b.md", reason: "schema-type-not-extractable", detail: "memo" },+ ]);+ expect(plan.skip_reason_counts).toEqual({ "schema-type-not-extractable": 1 });+ });+});++describe("o2b brain batch-plan unclassifiable counts (P4)", () => {+ test("the human surface renders the per-extension counts", async () => {+ write("mono/a.md");+ write("mono/pic.png", "binary");+ write("mono/data.csv", "a,b\n");++ const res = await runCli(["brain", "batch-plan", "mono"], { env: ENV() });+ expect(res.returncode).toBe(0);+ expect(res.stdout).toContain("2 unclassifiable file(s)");+ expect(res.stdout).toContain(".png: 1");+ expect(res.stdout).toContain(".csv: 1");+ });++ test("a tree with no unclassifiable files renders no count and carries no key on the wire", async () => {+ write("mono/a.md");++ const human = await runCli(["brain", "batch-plan", "mono"], { env: ENV() });+ expect(human.returncode).toBe(0);+ expect(human.stdout).not.toContain("unclassifiable");++ const json = await runCli(["brain", "batch-plan", "mono", "--json"], { env: ENV() });+ expect(json.returncode).toBe(0);+ const plan = JSON.parse(json.stdout);+ expect("unclassifiable" in plan).toBe(false);+ });++ test("the --json payload carries the total and per-extension counts", async () => {+ write("mono/a.md");+ write("mono/pic.png", "binary");+ write("mono/data.csv", "a,b\n");++ const res = await runCli(["brain", "batch-plan", "mono", "--json"], { env: ENV() });+ expect(res.returncode).toBe(0);+ const plan = JSON.parse(res.stdout);+ expect(plan.unclassifiable).toEqual({ total: 2, by_extension: { ".png": 1, ".csv": 1 } });+ });+});diff --git a/tests/cli/brain-orphan-repair.test.ts b/tests/cli/brain-orphan-repair.test.tsnew file mode 100644index 00000000..0813ec8c--- /dev/null+++ b/tests/cli/brain-orphan-repair.test.ts@@ -0,0 +1,155 @@+/**+ * `o2b brain orphan-repair` (t_6cc80627). The verb is the only door to+ * the detach repair: dry-run is the default and writes nothing; --apply+ * requires the exact --confirm phrase; a rerun converges to zero writes.+ *+ * The last describe pins the boundary the design states rather than+ * discovers: the doctor pass REPORTS the orphan and never repairs it,+ * and the verb is not wired into the pass - the repair is reachable only+ * by an operator typing the command.+ */++import { afterEach, beforeEach, expect, test } from "bun:test";+import { mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { runCli } from "../helpers/run-cli.ts";+import { ORPHAN_SESSION_REPAIR_COMMAND } from "../../src/core/brain/doctor/orphan-session-check.ts";+import { bootstrapBrain } from "../../src/core/brain/init.ts";+import { signalPath as brainSignalPath } from "../../src/core/brain/paths.ts";+import { writeSignal } from "../../src/core/brain/signal.ts";++/** Event date and slug of the single fixture signal; the production+ * path builder composes the same file name from them. */+const FIXTURE_DATE = "2026-06-01";+const FIXTURE_SLUG = "orphan";++let tmp: string;+let vault: string;+let config: string;++beforeEach(() => {+ tmp = mkdtempSync(join(tmpdir(), "o2b-orphan-repair-cli-"));+ vault = join(tmp, "vault");+ config = join(tmp, "config.yaml");+ writeFileSync(config, `vault: "${vault}"\n`);+ bootstrapBrain(vault, { configPath: config });+ writeOrphan();+});++afterEach(() => {+ rmSync(tmp, { recursive: true, force: true });+});++function signalPath(): string {+ // The production path builder derives the exact file name from the+ // date and slug the fixture writes with. Walking the inbox would be+ // fragile: readdirSync order is filesystem-defined, and the bootstrap+ // also places the `archived/` and `processed/` sibling directories+ // there, so on some filesystems the first entry is a directory.+ return brainSignalPath(vault, FIXTURE_DATE, FIXTURE_SLUG);+}++function writeOrphan(): void {+ // The canonical writer, so the doctor's own record check parses the+ // fixture cleanly and the only finding in play is the orphan itself.+ writeSignal(vault, {+ topic: FIXTURE_SLUG,+ signal: "positive",+ agent: "tester",+ principle: "observations keep their provenance",+ created_at: "2026-06-01T00:00:00Z",+ date: FIXTURE_DATE,+ slug: FIXTURE_SLUG,+ source_type: "session",+ source: ["[[session:sess-gone#turn-1]]"],+ session_ref: "session:sess-gone#turn-1",+ raw: "the observation body",+ });+}++const env = () => ({ OPEN_SECOND_BRAIN_CONFIG: config });++test("dry-run reports the detach and writes nothing", async () => {+ const before = readFileSync(signalPath(), "utf8");+ const res = await runCli(["brain", "orphan-repair", "--json"], { env: env() });+ expect(res.returncode).toBe(0);+ const report = JSON.parse(res.stdout) as { mode: string; detached: number; decisions: unknown[] };+ expect(report.mode).toBe("dry-run");+ expect(report.detached).toBe(1);+ expect(report.decisions.length).toBe(1);+ expect(readFileSync(signalPath(), "utf8")).toBe(before);+});++test("apply without the exact confirmation phrase is refused", async () => {+ const res = await runCli(["brain", "orphan-repair", "--apply", "--confirm", "nope"], {+ env: env(),+ });+ expect(res.returncode).not.toBe(0);+ expect(readFileSync(signalPath(), "utf8")).toContain("session_ref:");+});++test("a refused apply under --json emits a JSON error envelope, not plain text", async () => {+ const res = await runCli(["brain", "orphan-repair", "--apply", "--confirm", "nope", "--json"], {+ env: env(),+ });+ expect(res.returncode).toBe(1);+ const parsed = JSON.parse(res.stdout) as { ok: boolean; message: string };+ expect(parsed.ok).toBe(false);+ expect(parsed.message).toContain("confirmation phrase");+});++test("apply with the exact phrase detaches the reference, and a rerun is a no-op", async () => {+ const applied = await runCli(+ ["brain", "orphan-repair", "--apply", "--confirm", "apply orphan repair", "--json"],+ { env: env() },+ );+ expect(applied.returncode).toBe(0);+ const first = JSON.parse(applied.stdout) as { detached: number };+ expect(first.detached).toBe(1);+ const after = readFileSync(signalPath(), "utf8");+ expect(after).not.toContain("session_ref:");+ expect(after).toContain("the observation body");++ const rerun = await runCli(+ ["brain", "orphan-repair", "--apply", "--confirm", "apply orphan repair", "--json"],+ { env: env() },+ );+ const second = JSON.parse(rerun.stdout) as { detached: number };+ expect(second.detached).toBe(0);+ expect(readFileSync(signalPath(), "utf8")).toBe(after);+});++test("the doctor pass reports the orphan and never repairs it", async () => {+ const before = readFileSync(signalPath(), "utf8");+ const res = await runCli(["brain", "doctor", "--json"], { env: env() });+ expect(res.returncode).toBe(0);+ const report = JSON.parse(res.stdout) as { warnings: Array<{ code: string; fix?: string }> };+ const orphan = report.warnings.filter((w) => w.code === "orphan-session-ref");+ expect(orphan.length).toBe(1);+ expect(orphan[0]!.fix).toBe(ORPHAN_SESSION_REPAIR_COMMAND);+ // The pass is read-only: the dangling reference is still on disk after+ // the doctor ran, and the repair it names is the verb's to make.+ expect(readFileSync(signalPath(), "utf8")).toBe(before);+});++test("the repair is not wired into the doctor pass", () => {+ // A source-level boundary: no doctor module may import the repair or+ // reach the verb, so a future refactor cannot make `runDoctor` mutate.+ const doctorDir = join(import.meta.dir, "..", "..", "src", "core", "brain", "doctor");+ const modules = [join(doctorDir, "..", "doctor.ts")];+ for (const entry of readdirSync(doctorDir)) {+ if (entry.endsWith(".ts")) modules.push(join(doctorDir, entry));+ }+ // A module path containing the repair, on a `from` clause - never+ // prose: the check module's docblock names the repair module as the+ // write half it must not become, and must stay free to do so.+ const IMPORT_RE = /^[^\n]*\bfrom\s+"[^"]*orphan-repair[^"]*";/m;+ for (const path of modules) {+ const source = readFileSync(path, "utf8");+ expect(`${path} imports repair: ${IMPORT_RE.test(source)}`).toBe(+ `${path} imports repair: false`,+ );+ }+});diff --git a/tests/core/architecture/verdict-vocabulary-census.test.ts b/tests/core/architecture/verdict-vocabulary-census.test.tsindex 3e87b142..776c03b3 100644--- a/tests/core/architecture/verdict-vocabulary-census.test.ts+++ b/tests/core/architecture/verdict-vocabulary-census.test.ts@@ -210,6 +210,11 @@ import {PAGE_LINT_SKIP_REASON,PAGE_LINT_SKIP_REASONS,} from "../../../src/core/brain/page-lint.ts";+import {+ isSkippedPageReason,+ SKIPPED_PAGE_REASON,+ SKIPPED_PAGE_REASONS,+} from "../../../src/core/brain/ingest/extractable-gate.ts";import {EGRESS_REDACTION,EGRESS_REDACTION_STATUSES,@@ -297,9 +302,12 @@ import {isEmbeddingSunsetUndeterminedReason,} from "../../../src/core/search/embeddings/sunset.ts";import {+ isVaultBackingRemoteness,isVaultBackingState,isVaultBackingUndeterminedReason,VAULT_BACKING,+ VAULT_BACKING_REMOTENESS,+ VAULT_BACKING_REMOTENESS_STATES,VAULT_BACKING_STATES,VAULT_BACKING_UNDETERMINED_REASON,VAULT_BACKING_UNDETERMINED_REASONS,@@ -696,6 +704,16 @@ const CENSUS: ReadonlyArray<VocabularyUnderCensus> = Object.freeze([members: PAGE_LINT_SKIP_REASONS,guard: isPageLintSkipReason,},+ {+ // What the extractable gate names a skipped page. Registered for the+ // same reason the page-lint vocabulary is: the token crosses the MCP+ // wire and the CLI JSON verbatim, so a value added here and forgotten+ // in the membership list is a reason no reader can narrow.+ name: "SKIPPED_PAGE_REASON",+ values: SKIPPED_PAGE_REASON,+ members: SKIPPED_PAGE_REASONS,+ guard: isSkippedPageReason,+ },{// C1. What a given export path does about secrets on the way out.// Registered because the value is the load-bearing field of a@@ -943,6 +961,17 @@ const CENSUS: ReadonlyArray<VocabularyUnderCensus> = Object.freeze([members: VAULT_BACKING_UNDETERMINED_REASONS,guard: isVaultBackingUndeterminedReason,},+ {+ // A2, second axis. Whether the backing is a NAMED network filesystem,+ // beside - not inside - the survival verdict: nfs and cifs are both+ // durable AND remote, and `non_remote` deliberately is not named+ // `local`, because an unknown or unprobeable backing answers+ // non-remote too and "no network finding" is not evidence of local.+ name: "VAULT_BACKING_REMOTENESS",+ values: VAULT_BACKING_REMOTENESS,+ members: VAULT_BACKING_REMOTENESS_STATES,+ guard: isVaultBackingRemoteness,+ },{// U7. The runtimes whose session adapters ship in this tree. It was a// hand-written string union with a guard taking `string`, which is a@@ -1533,7 +1562,7 @@ const SCANNED = scanVocabularies(SOURCE_TREE);* How many four-piece vocabularies `src/` currently holds. Measured, and* kept as an equality rather than a floor - see the population test.*/-const VOCABULARY_POPULATION = 75;+const VOCABULARY_POPULATION = 77;const REGISTERED = new Map(CENSUS.map((entry) => [entry.name, entry] as const));describe("verdict vocabulary census", () => {diff --git a/tests/core/architecture/write-site-census.test.ts b/tests/core/architecture/write-site-census.test.tsindex bfccc84d..af0155db 100644--- a/tests/core/architecture/write-site-census.test.ts+++ b/tests/core/architecture/write-site-census.test.ts@@ -1267,8 +1267,11 @@ const DIRECT_WRITE_ROWS = 75;* 105 -> 106: `src/core/brain/dedup-index-cache.ts` persists the* machine-local dedup index cache through `atomicWriteFileSync`, outside* the vault (issue #195).+ *+ * 106 -> 107: `src/core/brain/link-graph/orphan-repair.ts` detaches a+ * signal's `session_ref` through `writeFrontmatterAtomic`.*/-const SHARED_HELPER_ROWS = 106;+const SHARED_HELPER_ROWS = 107;// ----- Origin-channel coverage boundary (Unit C) ----------------------------@@ -1349,9 +1352,10 @@ const UNSTAMPED_DIRECT_ROWS = 74;* The payload registry's `put()` is one of the unstamped sites counted, and* so is the knowledge-pack install fingerprint (100 -> 101), and the* failed-upgrade marker `self-heal-upgrade-state.ts` writes (101 -> 102),- * and the dedup index cache, a derived file outside the vault (102 -> 103).+ * and the dedup index cache, a derived file outside the vault (102 -> 103),+ * and the orphan-repair detach (103 -> 104).*/-const UNSTAMPED_SHARED_ROWS = 103;+const UNSTAMPED_SHARED_ROWS = 104;describe("in-vault write-site census", () => {test("every direct-fs write site carries a written exclusion", () => {diff --git a/tests/core/brain/continuity/reader-census.test.ts b/tests/core/brain/continuity/reader-census.test.tsindex 3387b0de..9c8dfc30 100644--- a/tests/core/brain/continuity/reader-census.test.ts+++ b/tests/core/brain/continuity/reader-census.test.ts@@ -91,6 +91,9 @@ const DIRECT_STORE_READERS: Readonly<Record<string, string>> = Object.freeze({"src/core/brain/generation-reports.ts": "reads `generation_report` for list, summary and show.","src/core/brain/idea-lineage.ts":"reads the WHOLE store to build id / turn-id / session-turn indexes.",+ "src/core/brain/doctor/orphan-session-check.ts":+ "reads the WHOLE store to collect every known session id, so a " ++ "session_ref resolves against the same universe the writers stamp.","src/core/brain/link-graph/repair-lane.ts":"reads the WHOLE store to harvest `sourceRefs` paths for link repair.","src/core/brain/mcp-route-metrics.ts": "reads `mcp_route_latency` for per-route rollups.",@@ -125,8 +128,8 @@ const DIRECT_STORE_READERS: Readonly<Record<string, string>> = Object.freeze({* these say how far the counts may fall before the split stops describing* anything.*- * Four modules read through the read-model and twenty-two read the store- * directly. Both numbers are EQUALITIES, not floors.+ * Four modules read through the read-model and twenty-three read the+ * store directly. Both numbers are EQUALITIES, not floors.** The direct floor used to sit at twenty against a reality of* twenty-one, on the reasoning that an exact pin would fail on another@@ -139,7 +142,7 @@ const DIRECT_STORE_READERS: Readonly<Record<string, string>> = Object.freeze({* nothing extra to keep true.*/const READ_MODEL_READER_COUNT = 4;-const DIRECT_STORE_READER_COUNT = 22;+const DIRECT_STORE_READER_COUNT = 23;/** A file the classifier must see, and that `grep` alone does not. */diff --git a/tests/core/brain/doctor-exit-census.test.ts b/tests/core/brain/doctor-exit-census.test.tsindex c5adcd4c..246fc0b7 100644--- a/tests/core/brain/doctor-exit-census.test.ts+++ b/tests/core/brain/doctor-exit-census.test.ts@@ -185,12 +185,14 @@ const DOCTOR_REGISTERED_CODES: ReadonlyArray<string> = ["dangling-workrun","duplicate-preferences","embedding-model-sunset-announced",+ "embeddings-backlog","entity-label-malformed","entity-quote-variant-collision","inbox-archivable","log-chain-broken","low-evidence-confirmed","orphan-evidence",+ "orphan-session-ref","payload-orphan","principle-corrupted","recall-channel-silent",diff --git a/tests/core/brain/doctor/embeddings-health-check.test.ts b/tests/core/brain/doctor/embeddings-health-check.test.tsnew file mode 100644index 00000000..516d20eb--- /dev/null+++ b/tests/core/brain/doctor/embeddings-health-check.test.ts@@ -0,0 +1,314 @@+/**+ * The embeddings-health check (t_f99250aa).+ *+ * The signals it consolidates already existed - the pending-vector census+ * with its unrecorded-is-not-zero contract, the index counts including+ * `staleEmbeddings`, the provider facts `o2b search check` owns - but+ * nothing looked at them together where an operator looks. What this file+ * pins is the check's own judgements: which states become a warning, which+ * reach the uncertain stream, which are silence, and that a failed read is+ * never reported as a healthy index.+ *+ * The index probe is injected, following `makeEmbeddingSunsetCheck`: the+ * facts are data the check reads rather than logic it owns, so every+ * verdict is reachable from a test without building a real index.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { DIAGNOSTIC_SIGNALS } from "../../../../src/core/brain/diagnostics.ts";+import type { DoctorCheckContext } from "../../../../src/core/brain/doctor/check.ts";+import {+ EMBEDDINGS_BACKLOG_CODE,+ EMBEDDINGS_CENSUS_UNRECORDED_CODE,+ EMBEDDINGS_HEALTH_UNMEASURED_CODE,+ embeddingsHealthCheck,+ makeEmbeddingsHealthCheck,+ type EmbeddingsHealthProbe,+ type EmbeddingsHealthProbeSnapshot,+} from "../../../../src/core/brain/doctor/embeddings-health-check.ts";+import type { DoctorUncertainEntry } from "../../../../src/core/brain/doctor/report.ts";+import { DOCTOR_EXIT_EXCLUSIONS } from "../../../../src/core/brain/doctor-exits.ts";+import { bootstrapBrain } from "../../../../src/core/brain/init.ts";+import { runDoctor } from "../../../../src/core/brain/doctor.ts";+import type { DoctorIssue } from "../../../../src/core/brain/types.ts";+import type { PendingVectorCensus } from "../../../../src/core/search/types.ts";++const MS_PER_DAY = 24 * 60 * 60 * 1000;+const NOW = new Date("2026-06-01T00:00:00.000Z");++let vault: string;+let configHome: string;+let configPath: string;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-embed-health-vault-"));+ configHome = mkdtempSync(join(tmpdir(), "o2b-embed-health-cfg-"));+ configPath = join(configHome, "config.yaml");+ bootstrapBrain(vault);+ configure("vendor-x/embed-v1", "8");+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+ rmSync(configHome, { recursive: true, force: true });+});++/** Semantic search on, an openai-compatible endpoint, `model` at `dim`. */+function configure(model: string, dimension: string): void {+ writeFileSync(+ configPath,+ [+ `vault: ${vault}`,+ `search_semantic_enabled: "true"`,+ `embedding_provider: "openai-compat"`,+ `embedding_base_url: "https://example.invalid/v1"`,+ `embedding_model: "${model}"`,+ `embedding_dimension: "${dimension}"`,+ ].join("\n") + "\n",+ );+}++function measured(pending: number, chunks: number): PendingVectorCensus {+ return { verdict: "measured", pending, chunks };+}++/** A fully embedded, current, recently indexed index - the healthy shape. */+function snapshot(+ overrides: Partial<EmbeddingsHealthProbeSnapshot> = {},+): EmbeddingsHealthProbeSnapshot {+ return {+ census: measured(0, 10),+ staleEmbeddings: 0,+ lastIndexedAt: new Date(NOW.getTime() - MS_PER_DAY).toISOString(),+ ...overrides,+ };+}++interface RunResult {+ readonly issues: ReadonlyArray<DoctorIssue>;+ readonly uncertain: ReadonlyArray<DoctorUncertainEntry>;+}++function run(+ probe: EmbeddingsHealthProbe = () => snapshot(),+ now: Date = NOW,+ configOverride: string | null = null,+): RunResult {+ const issues: DoctorIssue[] = [];+ const uncertain: DoctorUncertainEntry[] = [];+ makeEmbeddingsHealthCheck(probe).run(+ { vault, now, configPath: configOverride ?? configPath } as unknown as DoctorCheckContext,+ { issues, uncertain },+ );+ return { issues, uncertain };+}++function backlog(result: RunResult): ReadonlyArray<DoctorIssue> {+ return result.issues.filter((i) => i.code === EMBEDDINGS_BACKLOG_CODE);+}++describe("a fully embedded index is silent", () => {+ test("pending zero, stale zero, no issue on either stream", () => {+ const result = run();+ expect(result.issues).toEqual([]);+ expect(result.uncertain).toEqual([]);+ });++ test("semantic search switched off says nothing - there is no index to be sick", () => {+ writeFileSync(configPath, `vault: ${vault}\nsearch_semantic_enabled: "false"\n`);+ let probed = 0;+ const result = run(() => {+ probed++;+ return snapshot();+ });+ expect(probed).toBe(0);+ expect(result.issues).toEqual([]);+ expect(result.uncertain).toEqual([]);+ });+});++describe("a backlog is one warning naming both counts and both commands", () => {+ test("pending vectors warn with the backfill command", () => {+ const result = run(() => snapshot({ census: measured(3, 10) }));+ const issues = backlog(result);+ expect(issues.length).toBe(1);+ expect(issues[0]!.severity).toBe("warning");+ expect(issues[0]!.message).toContain("3");+ expect(issues[0]!.message).toContain("o2b search vector-backfill");+ });++ test("stale vectors warn with the provider check command", () => {+ const result = run(() => snapshot({ staleEmbeddings: 4 }));+ const issues = backlog(result);+ expect(issues.length).toBe(1);+ expect(issues[0]!.message).toContain("4");+ expect(issues[0]!.message).toContain("o2b search check");+ });++ test("both conditions at once stay one finding", () => {+ const result = run(() => snapshot({ census: measured(2, 10), staleEmbeddings: 5 }));+ expect(backlog(result).length).toBe(1);+ });++ test("the index age rides the message and the clock is injected", () => {+ const thirtyDays = new Date(NOW.getTime() - 30 * MS_PER_DAY).toISOString();+ const pending = measured(2, 10);+ const early = run(() => snapshot({ census: pending, lastIndexedAt: thirtyDays }), NOW);+ const later = run(+ () => snapshot({ census: pending, lastIndexedAt: thirtyDays }),+ new Date(NOW.getTime() + 10 * MS_PER_DAY),+ );+ expect(backlog(early)[0]!.message).toContain("30");+ // The same index a clock further on reads as older: the age is computed+ // against the context's clock, never one the check read for itself.+ expect(backlog(later)[0]!.message).toContain("40");+ });+});++describe("an unrecorded census is uncertain, never healthy", () => {+ test("the census reason reaches the uncertain stream verbatim", () => {+ const result = run(() =>+ snapshot({ census: { verdict: "unrecorded", reason: "no search index at /gone/index.db" } }),+ );+ expect(result.issues).toEqual([]);+ const entry = result.uncertain.find((e) => e.code === EMBEDDINGS_CENSUS_UNRECORDED_CODE);+ expect(entry).toBeDefined();+ expect(entry!.message).toContain("no search index at /gone/index.db");+ });++ test("an unrecorded census never shares the silence of a healthy one", () => {+ const healthy = run();+ const unrecorded = run(() =>+ snapshot({ census: { verdict: "unrecorded", reason: "no search index at /gone/index.db" } }),+ );+ expect(healthy.issues).toEqual([]);+ expect(healthy.uncertain).toEqual([]);+ expect(unrecorded.uncertain.length).toBeGreaterThan(0);+ });+});++describe("a half-read index is not a healthy one", () => {+ test("a measured census beside an unreadable count is uncertain, not silence", () => {+ // The census opened the file; the count half could not be read. Zero+ // stale is exactly the answer that reads as healthy, so a null count+ // may never collapse into it.+ const result = run(() => snapshot({ staleEmbeddings: null }));+ expect(result.issues).toEqual([]);+ expect(result.uncertain.some((e) => e.code === EMBEDDINGS_HEALTH_UNMEASURED_CODE)).toBe(true);+ });+});++describe("a throwing probe degrades to uncertain", () => {+ test("the throw is caught and reported, never propagated", () => {+ const result = run(() => {+ throw new Error("disk on fire");+ });+ expect(result.issues).toEqual([]);+ const entry = result.uncertain.find((e) => e.code === EMBEDDINGS_HEALTH_UNMEASURED_CODE);+ expect(entry).toBeDefined();+ expect(entry!.message).toContain("disk on fire");+ });++ test("the registered check is failSoft", () => {+ expect(embeddingsHealthCheck.failSoft).toBe(true);+ });+});++describe("a search configuration that will not resolve is uncertain", () => {+ test("a config file that fails to load reaches the uncertain stream", () => {+ // A path naming a directory: `discoverConfig` refuses it, which is the+ // cheapest way to make the resolver throw without mocking a module.+ const result = run(undefined, NOW, join(configHome));+ expect(result.issues).toEqual([]);+ expect(result.uncertain.some((e) => e.code === EMBEDDINGS_HEALTH_UNMEASURED_CODE)).toBe(true);+ });+});++describe("the probe reads the index the context names", () => {+ test("ctx.dbPath is the index probed and the path every finding names", () => {+ const seen: string[] = [];+ const issues: DoctorIssue[] = [];+ const uncertain: DoctorUncertainEntry[] = [];+ makeEmbeddingsHealthCheck((dbPath) => {+ seen.push(dbPath);+ return snapshot({ staleEmbeddings: 2 });+ }).run(+ {+ vault,+ now: NOW,+ configPath,+ dbPath: "/elsewhere/index.db",+ } as unknown as DoctorCheckContext,+ { issues, uncertain },+ );+ expect(seen).toEqual(["/elsewhere/index.db"]);+ expect(backlog({ issues, uncertain })[0]!.path).toBe("/elsewhere/index.db");+ });++ test("the probe is handed the configured embedding pair, dimension unset included", () => {+ writeFileSync(+ configPath,+ [+ `vault: ${vault}`,+ `search_semantic_enabled: "true"`,+ `embedding_provider: "openai-compat"`,+ `embedding_base_url: "https://example.invalid/v1"`,+ `embedding_model: "vendor-x/embed-v2"`,+ ].join("\n") + "\n",+ );+ const seen: unknown[] = [];+ run((_dbPath, semantic) => {+ seen.push({ model: semantic.model, dimension: semantic.dimension });+ return snapshot();+ });+ expect(seen).toEqual([{ model: "vendor-x/embed-v2", dimension: null }]);+ });+});++describe("no embedding model to compare against is uncertain, never healthy", () => {+ test("a remote provider with no configured model is unmeasured", () => {+ writeFileSync(+ configPath,+ [+ `vault: ${vault}`,+ `search_semantic_enabled: "true"`,+ `embedding_provider: "openai-compat"`,+ `embedding_base_url: "https://example.invalid/v1"`,+ ].join("\n") + "\n",+ );+ const result = run(() => snapshot());+ expect(result.issues).toEqual([]);+ expect(result.uncertain.some((e) => e.code === EMBEDDINGS_HEALTH_UNMEASURED_CODE)).toBe(true);+ });+});++describe("the check runs from runDoctor itself", () => {+ test("a semantic-enabled vault with no index reports its unrecorded census", () => {+ const result = runDoctor(vault, { now: NOW, configPath });+ expect((result.uncertain ?? []).some((e) => e.code === EMBEDDINGS_CENSUS_UNRECORDED_CODE)).toBe(+ true,+ );+ });+});++describe("every code is classified exactly once", () => {+ test("the warning code carries a registered command", () => {+ expect(DIAGNOSTIC_SIGNALS.has(EMBEDDINGS_BACKLOG_CODE)).toBe(true);+ expect(DOCTOR_EXIT_EXCLUSIONS.has(EMBEDDINGS_BACKLOG_CODE)).toBe(false);+ });++ test("both uncertain codes carry a written reason instead", () => {+ for (const code of [EMBEDDINGS_CENSUS_UNRECORDED_CODE, EMBEDDINGS_HEALTH_UNMEASURED_CODE]) {+ expect(`${code} excluded: ${DOCTOR_EXIT_EXCLUSIONS.has(code)}`).toBe(+ `${code} excluded: true`,+ );+ expect(`${code} registered: ${DIAGNOSTIC_SIGNALS.has(code)}`).toBe(+ `${code} registered: false`,+ );+ }+ });+});diff --git a/tests/core/brain/doctor/orphan-session-check.test.ts b/tests/core/brain/doctor/orphan-session-check.test.tsnew file mode 100644index 00000000..6b33533c--- /dev/null+++ b/tests/core/brain/doctor/orphan-session-check.test.ts@@ -0,0 +1,308 @@+/**+ * The orphan-observation check (t_6cc80627).+ *+ * An observation signal carries `session_ref` - the portable session+ * coordinates the capture and import writers stamp - and nothing anywhere+ * resolved it: an observation whose parent session is gone was silently+ * accepted. This check is the strictly read-only half of the answer; the+ * repair verb behind the `fix` field it carries is tested next door.+ *+ * What is pinned here: which signals are orphans, which resolve (through+ * either half of the sessions/continuity store), that the repair command+ * travels ON the issue rather than inside its prose, and that a subtree+ * the walk could not read is uncertainty rather than a clean bill.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { chmodSync, mkdtempSync, readdirSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { appendLogEvent } from "../../../../src/core/brain/log.ts";+import { appendContinuityRecord } from "../../../../src/core/brain/continuity/store.ts";+import { runDoctor } from "../../../../src/core/brain/doctor.ts";+import {+ collectKnownSessionIds,+ ORPHAN_SESSION_REF_CODE,+ ORPHAN_SESSION_REPAIR_COMMAND,+ orphanSessionCheck,+ isImportedTranscriptRef,+ parseSessionRef,+} from "../../../../src/core/brain/doctor/orphan-session-check.ts";+import type { DoctorUncertainEntry } from "../../../../src/core/brain/doctor/report.ts";+import { IS_WINDOWS } from "../../../helpers/platform.ts";+import { bootstrapBrain } from "../../../../src/core/brain/init.ts";+import { brainDirs } from "../../../../src/core/brain/paths.ts";+import { readAllLogRecords } from "../../../../src/core/brain/doctor/records.ts";+import { writeSignal } from "../../../../src/core/brain/signal.ts";+import type { DoctorIssue } from "../../../../src/core/brain/types.ts";++let tmp: string;+let vault: string;+let locked: string[];++beforeEach(() => {+ tmp = mkdtempSync(join(tmpdir(), "o2b-orphan-session-"));+ vault = join(tmp, "vault");+ bootstrapBrain(vault);+ locked = [];+});++afterEach(() => {+ for (const path of locked) chmodSync(path, 0o700);+ rmSync(tmp, { recursive: true, force: true });+});++function writeObservation(sessionRef: string | undefined): string {+ const result = writeSignal(vault, {+ topic: "orphan-lane",+ signal: "positive",+ agent: "tester",+ principle: "observations keep their provenance",+ created_at: "2026-06-01T00:00:00Z",+ date: "2026-06-01",+ slug: "orphan-lane",+ source_type: "session",+ ...(sessionRef !== undefined ? { session_ref: sessionRef } : {}),+ raw: "the observation body survives any repair",+ });+ return result.path;+}++function issuesFor(logs: LogRecords = readAllLogRecords(vault)): ReadonlyArray<DoctorIssue> {+ const issues: DoctorIssue[] = [];+ const uncertain: DoctorUncertainEntry[] = [];+ orphanSessionCheck.run(+ {+ vault,+ now: new Date("2026-06-02T00:00:00.000Z"),+ // The pre-parsed log snapshot, exactly as the pass's resolveContext+ // builds it: the lifecycle half of the resolution reads from here.+ logs,+ } as unknown as Parameters<typeof orphanSessionCheck.run>[0],+ { issues, uncertain },+ );+ return issues;+}++describe("a session_ref that resolves to nothing is an orphan", () => {+ test("the warning carries the exact repair command as its fix field", () => {+ const path = writeObservation("session:sess-gone#turn-1");+ const issues = issuesFor().filter((i) => i.code === ORPHAN_SESSION_REF_CODE);+ expect(issues.length).toBe(1);+ expect(issues[0]!.severity).toBe("warning");+ expect(issues[0]!.path).toBe(path);+ expect(issues[0]!.fix).toBe(ORPHAN_SESSION_REPAIR_COMMAND);+ expect(issues[0]!.message).toContain("session:sess-gone#turn-1");+ });++ test("a bare session id (the extract-signals form) with no raw turns is an orphan", () => {+ writeObservation("raw-sess-gone");+ const issues = issuesFor().filter((i) => i.code === ORPHAN_SESSION_REF_CODE);+ expect(issues.length).toBe(1);+ });+});++describe("an imported transcript ref is never judged", () => {+ // `o2b brain sessions import` stamps `<transcript basename>#<turn>`: the+ // parent is a transcript file outside the vault, and the import leaves+ // no record it is guaranteed to keep (the checkpoint is cleared, recall,+ // skill and dedup records are conditional). A missing record proves+ // nothing, and a repair run on such a finding would strip valid+ // provenance.+ test("an imported ref with no vault record raises no finding", () => {+ writeObservation("transcript.jsonl#t-9");+ expect(issuesFor().filter((i) => i.code === ORPHAN_SESSION_REF_CODE)).toEqual([]);+ });++ test("the import form is skipped before resolution, so the same identity in another form is judged", () => {+ // `transcript.jsonl` resolves against nothing. Under the import form the+ // ref is never judged; the same identity spelled in the extract-signals+ // form is, and is flagged - so the silence above is the form's skip,+ // not a resolution.+ writeObservation("transcript.jsonl#t-9");+ const bare = writeObservation("transcript.jsonl");+ const issues = issuesFor().filter((i) => i.code === ORPHAN_SESSION_REF_CODE);+ expect(issues.map((i) => i.path)).toEqual([bare]);+ expect(issues[0]!.message).toContain("'transcript.jsonl'");+ });++ test("isImportedTranscriptRef tells the three writer forms apart", () => {+ expect(isImportedTranscriptRef("transcript.jsonl#t-9")).toBe(true);+ expect(isImportedTranscriptRef("session:abc#turn-1")).toBe(false);+ expect(isImportedTranscriptRef("raw-session-id")).toBe(false);+ });+});++describe("a session_ref that resolves emits nothing", () => {+ test("a continuity record carries the session id", () => {+ appendContinuityRecord(vault, {+ kind: "session_turn",+ createdAt: "2026-06-01T00:00:00Z",+ payload: { session_id: "sess-live" },+ });+ writeObservation("session:sess-live#turn-2");+ expect(issuesFor().filter((i) => i.code === ORPHAN_SESSION_REF_CODE)).toEqual([]);+ });++ test("a session-lifecycle log event carries the session id", () => {+ appendLogEvent(vault, {+ timestamp: "2026-06-01T01:00:00Z",+ eventType: "session-lifecycle",+ agent: "tester",+ body: { event: "end", session_id: "sess-logged" },+ });+ writeObservation("session:sess-logged#end");+ expect(issuesFor().filter((i) => i.code === ORPHAN_SESSION_REF_CODE)).toEqual([]);+ });++ test("the writer's no-session stand-in is not an orphan", () => {+ // `session:unknown` is what the capture writer stamps when the host+ // payload carried no session id: a statement that there was no+ // session, not a pointer to a missing one. Detaching it would erase+ // the honesty rather than repair a dangle.+ writeObservation("session:unknown#turn-1");+ expect(issuesFor().filter((i) => i.code === ORPHAN_SESSION_REF_CODE)).toEqual([]);+ });++ test("a signal with no session_ref at all is none of this check's business", () => {+ writeObservation(undefined);+ expect(issuesFor().filter((i) => i.code === ORPHAN_SESSION_REF_CODE)).toEqual([]);+ });+});++describe("what the walk could not read is uncertainty", () => {+ // Windows chmod only toggles the read-only attribute and cannot deny a+ // read; the rest of this file passes there vacuously, so only this+ // subtree is skipped rather than guarded.+ test.skipIf(IS_WINDOWS)(+ "an unreadable signals directory reaches the uncertain stream, never silence",+ () => {+ const dirs = brainDirs(vault);+ const path = writeObservation("session:sess-gone#turn-1");+ chmodSync(dirs.inbox, 0o000);+ locked.push(dirs.inbox);+ expect(() => readdirSync(dirs.inbox)).toThrow();+ expect(issuesFor().filter((i) => i.code === ORPHAN_SESSION_REF_CODE)).toEqual([]);+ expect(uncertainFor().some((u) => u.path === dirs.inbox)).toBe(true);+ chmodSync(dirs.inbox, 0o700);+ locked.pop();+ expect(path).toBeDefined();+ },+ );++ test.skipIf(IS_WINDOWS)(+ "a continuity ledger that cannot be read is uncertainty, not an orphan flood",+ () => {+ const dirs = brainDirs(vault);+ writeObservation("session:sess-live#turn-2");+ appendContinuityRecord(vault, {+ kind: "session_turn",+ createdAt: "2026-06-01T00:00:00Z",+ payload: { session_id: "sess-live" },+ });+ chmodSync(dirs.log, 0o000);+ locked.push(dirs.log);+ // With Brain/log unreadable, the pass's own log read reports the+ // denial and hands the check an empty snapshot - which is the+ // context this assertion runs under. No issue may be raised off a+ // resolution surface the check could not read: the record naming+ // this session live was not seen.+ expect(issuesFor([]).filter((i) => i.code === ORPHAN_SESSION_REF_CODE)).toEqual([]);+ expect(uncertainFor([]).length).toBeGreaterThan(0);+ },+ );+});++describe("an unreadable log day is uncertainty, not an orphan", () => {+ test.skipIf(IS_WINDOWS)(+ "a session whose lifecycle lives in an unreadable day is never flagged",+ () => {+ const dirs = brainDirs(vault);+ appendLogEvent(vault, {+ timestamp: "2026-06-01T01:00:00Z",+ eventType: "session-lifecycle",+ agent: "tester",+ body: { event: "end", session_id: "sess-logged" },+ });+ writeObservation("session:sess-logged#end");+ const shards = readdirSync(dirs.log, { recursive: true, withFileTypes: true })+ .filter((e) => e.isFile() && e.name.includes("2026-06-01"))+ .map((e) => join(e.parentPath, e.name));+ expect(shards.length).toBeGreaterThan(0);+ for (const shard of shards) {+ chmodSync(shard, 0o000);+ locked.push(shard);+ }+ const result = runDoctor(vault);+ expect((result.warnings ?? []).filter((i) => i.code === ORPHAN_SESSION_REF_CODE)).toEqual([]);+ expect((result.uncertain ?? []).some((u) => u.message.includes("2026-06-01"))).toBe(true);+ const universe = collectKnownSessionIds(vault);+ expect(universe.complete).toBe(false);+ expect(universe.uncertain.length).toBeGreaterThan(0);+ },+ );+});++function uncertainFor(+ logs: LogRecords = readAllLogRecords(vault),+): ReadonlyArray<DoctorUncertainEntry> {+ const uncertain: DoctorUncertainEntry[] = [];+ orphanSessionCheck.run(+ {+ vault,+ now: new Date("2026-06-02T00:00:00.000Z"),+ logs,+ } as unknown as Parameters<typeof orphanSessionCheck.run>[0],+ { issues: [], uncertain },+ );+ return uncertain;+}++/** The log snapshot type the pass hands the check, as resolveContext builds it. */+type LogRecords = ReturnType<typeof readAllLogRecords>;++describe("the ref grammar, as the repair shares it", () => {+ test("the identity half drops the session prefix; the turn half is named", () => {+ expect(parseSessionRef("session:abc#turn-1")).toEqual({ identity: "abc", turn: "turn-1" });+ expect(parseSessionRef("transcript.jsonl#t-9")).toEqual({+ identity: "transcript.jsonl",+ turn: "t-9",+ });+ expect(parseSessionRef("session:abc")).toEqual({ identity: "abc", turn: null });+ });++ test("collectKnownSessionIds reads both halves of the store", () => {+ appendContinuityRecord(vault, {+ kind: "session_turn",+ createdAt: "2026-06-01T00:00:00Z",+ payload: { session_id: "sess-cont" },+ });+ appendLogEvent(vault, {+ timestamp: "2026-06-01T01:00:00Z",+ eventType: "session-lifecycle",+ agent: "tester",+ body: { event: "end", session_id: "sess-logged" },+ });+ const universe = collectKnownSessionIds(vault);+ expect(universe.ids.has("sess-cont")).toBe(true);+ expect(universe.ids.has("sess-logged")).toBe(true);+ expect(universe.complete).toBe(true);+ });+});++describe("the check is wired honestly", () => {+ test("runDoctor carries the orphan warning with its fix, and repairs nothing", () => {+ const path = writeObservation("session:sess-gone#turn-1");+ const before = issuesFor();+ expect(before.length).toBe(1);+ const result = runDoctor(vault);+ const orphan = (result.warnings ?? []).filter((i) => i.code === ORPHAN_SESSION_REF_CODE);+ expect(orphan.length).toBe(1);+ expect(orphan[0]!.fix).toBe(ORPHAN_SESSION_REPAIR_COMMAND);+ // Read-only: the doctor pass leaves the dangling reference exactly+ // where it was - the repair is the verb's to make, never the pass's.+ expect(issuesFor().length).toBe(1);+ expect(path).toContain("sig-");+ });+});diff --git a/tests/core/brain/ingest/batch-plan-extractable.test.ts b/tests/core/brain/ingest/batch-plan-extractable.test.tsindex dde22c8d..a414ed62 100644--- a/tests/core/brain/ingest/batch-plan-extractable.test.ts+++ b/tests/core/brain/ingest/batch-plan-extractable.test.ts@@ -62,7 +62,8 @@ describe("extractable gate in planBatches", () => {const plan = planBatches(vault, "Sources", CAPS);expect(plannedPaths(plan)).toEqual(["Sources/a.md", "Sources/c.md"]);expect(plan.skippedNonExtractable.map((s) => s.path)).toEqual(["Sources/b.md"]);- expect(plan.skippedNonExtractable[0]!.reason).toContain("memo");+ expect(plan.skippedNonExtractable[0]!.reason).toBe("schema-type-not-extractable");+ expect(plan.skippedNonExtractable[0]!.detail).toBe("memo");});test("with no allowlist the plan is byte-identical (empty gate report)", () => {diff --git a/tests/core/brain/ingest/batch-plan.test.ts b/tests/core/brain/ingest/batch-plan.test.tsindex 1259c781..c91bcd58 100644--- a/tests/core/brain/ingest/batch-plan.test.ts+++ b/tests/core/brain/ingest/batch-plan.test.ts@@ -15,8 +15,10 @@ import { tmpdir } from "node:os";import { bootstrapBrain } from "../../../../src/core/brain/init.ts";import { atomicWriteFileSync } from "../../../../src/core/fs-atomic.ts";import { updateManifest } from "../../../../src/core/brain/ingest/content-manifest.ts";-import { planBatches } from "../../../../src/core/brain/ingest/batch-plan.ts";+import { planBatches, type BatchPlan } from "../../../../src/core/brain/ingest/batch-plan.ts";import { recordCompleted } from "../../../../src/core/brain/ingest/checkpoint.ts";+import { SKIPPED_PAGE_REASON } from "../../../../src/core/brain/ingest/extractable-gate.ts";+import { serializeBatchPlan } from "../../../../src/mcp/brain/ingest-tools.ts";let vault: string;let configHome: string;@@ -45,6 +47,26 @@ function allPlannedPaths(plan: ReturnType<typeof planBatches>): string[] {return plan.batches.flatMap((b) => b.files.map((f) => f.path));}+const CAPS = { maxBatchBytes: 10_000, maxBatchFiles: 100 } as const;++/** Write a page carrying an optional `schema_type` frontmatter field. */+function page(rel: string, schemaType?: string): void {+ const abs = join(vault, rel);+ mkdirSync(join(abs, ".."), { recursive: true });+ const fm = schemaType === undefined ? "" : `schema_type: ${schemaType}\n`;+ writeFileSync(abs, `---\ntitle: ${rel}\n${fm}---\n\nbody text\n`, "utf8");+}++/** Point the schema pack at an `extractable` allowlist (gates discovery). */+function setExtractable(tokens: string[]): void {+ const block = tokens.map((t) => ` - ${t}`).join("\n");+ writeFileSync(+ join(vault, "Brain", "_brain.yaml"),+ `schema_version: 1\nschema:\n page_types:\n - paper\n - memo\n extractable:\n${block}\n`,+ "utf8",+ );+}+describe("planBatches — discovery + skip-unchanged", () => {test("skips `unchanged` files (via the manifest) and plans only new/modified", () => {writeSized("Inbox/keep.md", 100);@@ -227,3 +249,83 @@ describe("planBatches — resume (t_ba1fa5f6)", () => {}});});++describe("planBatches — typed skip reasons + per-reason counts (P4)", () => {+ test("the plan counts skipped pages per typed reason token", () => {+ page("Sources/a.md", "paper");+ page("Sources/b.md", "memo");+ page("Sources/c.md", "memo");+ page("Sources/d.md", "paper");+ setExtractable(["paper"]);++ const plan: BatchPlan = planBatches(vault, "Sources", CAPS);+ expect(plan.skippedNonExtractable).toHaveLength(2);+ expect(plan.skipReasonCounts).toEqual({ [SKIPPED_PAGE_REASON.notExtractable]: 2 });+ });++ test("a plan with no skips counts nothing", () => {+ page("Sources/a.md", "paper");+ page("Sources/b.md", "memo");+ const plan = planBatches(vault, "Sources", CAPS);+ expect(Object.keys(plan.skipReasonCounts)).toEqual([]);+ });++ test("serializeBatchPlan emits the counts only when non-empty (byte-identical when absent)", () => {+ writeSized("Docs/a.md", 50);+ const clean = planBatches(vault, "Docs", CAPS);+ expect("skip_reason_counts" in serializeBatchPlan(clean)).toBe(false);++ page("Sources/b.md", "memo");+ setExtractable(["paper"]);+ const gated = planBatches(vault, "Sources", CAPS);+ const wire = serializeBatchPlan(gated);+ expect(wire["skip_reason_counts"]).toEqual({ [SKIPPED_PAGE_REASON.notExtractable]: 1 });+ expect(wire["skipped_non_extractable"]).toEqual([+ { path: "Sources/b.md", reason: SKIPPED_PAGE_REASON.notExtractable, detail: "memo" },+ ]);+ });+});++describe("planBatches — unclassifiable files (P4)", () => {+ test("counts files dropped for a non-ingestible extension, per extension", () => {+ writeSized("Docs/a.md", 50);+ writeSized("Docs/pic.png", 50);+ writeSized("Docs/data.csv", 50);++ const plan = planBatches(vault, "Docs", CAPS);+ expect(plan.unclassifiable.total).toBe(2);+ expect(plan.unclassifiable.byExtension).toEqual({ ".png": 1, ".csv": 1 });++ // Counting must not alter the discovered set: the plan id (derived from+ // the ingestible set) is the same with and without the foreign files.+ rmSync(join(vault, "Docs", "pic.png"));+ rmSync(join(vault, "Docs", "data.csv"));+ expect(planBatches(vault, "Docs", CAPS).planId).toBe(plan.planId);+ });++ test("hidden entries and ignore-rule matches are excluded by declaration, not counted", () => {+ writeSized("Docs/a.md", 50);+ writeSized("Docs/.hidden.png", 50);+ writeSized("Docs/.hid/x.png", 50);+ writeSized("Docs/node_modules/lib/x.png", 50);+ writeFileSync(join(vault, "Docs", ".gitignore"), "node_modules/\n", "utf8");++ const plan = planBatches(vault, "Docs", CAPS);+ expect(plan.unclassifiable.total).toBe(0);+ expect(allPlannedPaths(plan)).toEqual(["Docs/a.md"]);+ });++ test("serializeBatchPlan emits the aggregate only when non-empty", () => {+ writeSized("Docs/a.md", 50);+ const clean = planBatches(vault, "Docs", CAPS);+ expect("unclassifiable" in serializeBatchPlan(clean)).toBe(false);++ writeSized("Docs/pic.png", 50);+ writeSized("Docs/data.csv", 50);+ const withDrops = planBatches(vault, "Docs", CAPS);+ expect(serializeBatchPlan(withDrops)["unclassifiable"]).toEqual({+ total: 2,+ by_extension: { ".png": 1, ".csv": 1 },+ });+ });+});diff --git a/tests/core/brain/ingest/extractable-gate.test.ts b/tests/core/brain/ingest/extractable-gate.test.tsindex bcfb475d..09c02069 100644--- a/tests/core/brain/ingest/extractable-gate.test.ts+++ b/tests/core/brain/ingest/extractable-gate.test.ts@@ -13,7 +13,11 @@ import { tmpdir } from "node:os";import {extractableAllowlist,+ isSkippedPageReason,+ parseSkippedPageReason,partitionExtractable,+ SKIPPED_PAGE_REASON,+ SKIPPED_PAGE_REASONS,} from "../../../../src/core/brain/ingest/extractable-gate.ts";let vault: string;@@ -41,7 +45,7 @@ describe("partitionExtractable", () => {const part = partitionExtractable(vault, paths, new Set(["paper"]));expect(part.extractable).toEqual(["a.md", "c.md"]);expect(part.skipped.map((s) => s.path)).toEqual(["b.md"]);- expect(part.skipped[0]!.reason).toContain("memo");+ expect(part.skipped[0]!.detail).toBe("memo");});test("a page with no declared type stays ungated (kept)", () => {@@ -59,6 +63,44 @@ describe("partitionExtractable", () => {});});+describe("typed skip reasons (P4)", () => {+ test("a skipped page carries a typed reason token from the closed list, plus the schema_type as detail", () => {+ const paths = [page("a.md", "paper"), page("b.md", "memo")];+ const part = partitionExtractable(vault, paths, new Set(["paper"]));+ expect(part.skipped).toHaveLength(1);+ const skip = part.skipped[0]!;+ expect(SKIPPED_PAGE_REASONS).toContain(skip.reason);+ expect(skip.reason).toBe(SKIPPED_PAGE_REASON.notExtractable);+ expect(skip.detail).toBe("memo");+ });++ test("isSkippedPageReason accepts only members of the closed union", () => {+ expect(isSkippedPageReason(SKIPPED_PAGE_REASON.notExtractable)).toBe(true);+ // The legacy sentence is not a member: a guard that accepted it would+ // narrow a string to a type it does not hold.+ expect(+ isSkippedPageReason('schema_type "memo" is not in the schema extractable allowlist'),+ ).toBe(false);+ expect(isSkippedPageReason("schema_type was rejected for some other reason")).toBe(false);+ expect(isSkippedPageReason("")).toBe(false);+ expect(isSkippedPageReason(42)).toBe(false);+ expect(isSkippedPageReason(null)).toBe(false);+ });++ test("parseSkippedPageReason maps the legacy sentence to its token and rejects the rest", () => {+ expect(parseSkippedPageReason(SKIPPED_PAGE_REASON.notExtractable)).toBe(+ SKIPPED_PAGE_REASON.notExtractable,+ );+ expect(+ parseSkippedPageReason('schema_type "memo" is not in the schema extractable allowlist'),+ ).toBe(SKIPPED_PAGE_REASON.notExtractable);+ expect(parseSkippedPageReason("schema_type was rejected for some other reason")).toBeNull();+ expect(parseSkippedPageReason("")).toBeNull();+ expect(parseSkippedPageReason(42)).toBeNull();+ expect(parseSkippedPageReason(null)).toBeNull();+ });+});+describe("extractableAllowlist", () => {test("is empty for a vault with no extractable schema declaration", () => {expect(extractableAllowlist(vault).size).toBe(0);diff --git a/tests/core/brain/link-graph/orphan-repair.test.ts b/tests/core/brain/link-graph/orphan-repair.test.tsnew file mode 100644index 00000000..5d3a6782--- /dev/null+++ b/tests/core/brain/link-graph/orphan-repair.test.ts@@ -0,0 +1,198 @@+/**+ * The orphan-observation repair lane (t_6cc80627).+ *+ * The repair DETACHES a dangling `session_ref` from a signal's+ * frontmatter and keeps the observation: nothing is deleted, no content+ * is rewritten, and the detached value is named in the decision so the+ * change is auditable. What this file pins is the lane discipline the+ * memory-graph repair lane established, applied to a destructive-looking+ * act that is in fact a one-key removal: dry-run default, exact confirm+ * phrase, hard per-run write cap, and an idempotent rescan.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { existsSync, mkdtempSync, readFileSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import {+ detachSessionRef,+ ORPHAN_REPAIR_CONFIRM_PHRASE,+ OrphanRepairConfirmationError,+ ORPHAN_REPAIR_WRITE_CAP,+ runOrphanRepair,+} from "../../../../src/core/brain/link-graph/orphan-repair.ts";+import { parseFrontmatter } from "../../../../src/core/vault.ts";+import { bootstrapBrain } from "../../../../src/core/brain/init.ts";+import { writeSignal } from "../../../../src/core/brain/signal.ts";+import { acquireLockSync } from "../../../../src/core/brain/sync-lockfile.ts";++let vault: string;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-orphan-repair-"));+ bootstrapBrain(vault);+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+});++function writeOrphan(sessionRef = "session:sess-gone#turn-1"): string {+ const result = writeSignal(vault, {+ topic: "orphan-repair",+ signal: "positive",+ agent: "tester",+ principle: "repairs detach, never delete",+ created_at: "2026-06-01T00:00:00Z",+ date: "2026-06-01",+ slug: "orphan-repair",+ source_type: "session",+ session_ref: sessionRef,+ raw: "the observation body",+ });+ return result.path;+}++describe("dry-run is the default and writes nothing", () => {+ test("the detach is listed, the file is byte-identical", () => {+ const path = writeOrphan();+ const before = readFileSync(path, "utf8");+ const report = runOrphanRepair(vault);+ expect(report.mode).toBe("dry-run");+ // `detached` counts the writes a dry run WOULD make, the same+ // convention `RepairReport.written` established: the report is the+ // plan, and the plan says one detach is pending.+ expect(report.detached).toBe(1);+ expect(report.decisions.length).toBe(1);+ expect(report.decisions[0]!.action).toBe("detach");+ expect(report.decisions[0]!.session_ref).toBe("session:sess-gone#turn-1");+ expect(readFileSync(path, "utf8")).toBe(before);+ });+});++describe("apply requires the exact confirmation phrase", () => {+ test("a wrong phrase throws and the file keeps its reference", () => {+ const path = writeOrphan();+ expect(() => runOrphanRepair(vault, { apply: true, confirm: "nope" })).toThrow(+ OrphanRepairConfirmationError,+ );+ const [, body] = parseFrontmatter(path);+ const [meta] = parseFrontmatter(path);+ expect(meta["session_ref"]).toBeDefined();+ expect(body).toContain("the observation body");+ });++ test("the exact phrase detaches the reference and keeps the observation", () => {+ const path = writeOrphan();+ const report = runOrphanRepair(vault, {+ apply: true,+ confirm: ORPHAN_REPAIR_CONFIRM_PHRASE,+ });+ expect(report.mode).toBe("apply");+ expect(report.detached).toBe(1);+ expect(report.decisions[0]!.action).toBe("detach");+ const [meta, body] = parseFrontmatter(path);+ expect(meta["session_ref"]).toBeUndefined();+ expect(body).toContain("the observation body");+ // The observation content is untouched: the topic frontmatter and the+ // raw body read exactly as before the detach.+ expect(meta["topic"]).toBe("orphan-repair");+ });+});++describe("the hard per-run write cap stops a run", () => {+ test("findings past the cap are skip-cap, and an apply writes no more than the cap", () => {+ for (let i = 0; i < ORPHAN_REPAIR_WRITE_CAP + 5; i++) {+ writeSignal(vault, {+ topic: `cap-${i}`,+ signal: "positive",+ agent: "tester",+ principle: "repairs detach, never delete",+ created_at: "2026-06-01T00:00:00Z",+ date: "2026-06-01",+ slug: `cap-${i}`,+ source_type: "session",+ session_ref: `session:sess-gone-${i}#turn-1`,+ });+ }+ const report = runOrphanRepair(vault, {+ apply: true,+ confirm: ORPHAN_REPAIR_CONFIRM_PHRASE,+ });+ expect(report.detached).toBe(ORPHAN_REPAIR_WRITE_CAP);+ expect(report.decisions.filter((d) => d.action === "skip-cap").length).toBe(5);+ });++ test("a caller-named cap bounds the run the same way", () => {+ writeOrphan("session:sess-gone-a#turn-1");+ writeSignal(vault, {+ topic: "second",+ signal: "negative",+ agent: "tester",+ principle: "repairs detach, never delete",+ created_at: "2026-06-01T00:00:00Z",+ date: "2026-06-01",+ slug: "second",+ source_type: "session",+ session_ref: "session:sess-gone-b#turn-1",+ });+ const report = runOrphanRepair(vault, {+ apply: true,+ confirm: ORPHAN_REPAIR_CONFIRM_PHRASE,+ writeCap: 1,+ });+ expect(report.detached).toBe(1);+ expect(report.decisions.filter((d) => d.action === "skip-cap").length).toBe(1);+ });+});++describe("a rescan after apply is idempotent", () => {+ test("the second run finds nothing to detach and writes nothing", () => {+ const path = writeOrphan();+ runOrphanRepair(vault, { apply: true, confirm: ORPHAN_REPAIR_CONFIRM_PHRASE });+ const afterFirst = readFileSync(path, "utf8");+ const again = runOrphanRepair(vault, {+ apply: true,+ confirm: ORPHAN_REPAIR_CONFIRM_PHRASE,+ });+ expect(again.detached).toBe(0);+ expect(again.decisions).toEqual([]);+ expect(readFileSync(path, "utf8")).toBe(afterFirst);+ });+});++describe("a reference that changed after the scan is not detached", () => {+ test("detachSessionRef removes only the value the scan quoted", () => {+ const path = writeOrphan("session:sess-new#turn-2");+ const before = readFileSync(path, "utf8");+ // The scan quoted an older value; the file now carries another one.+ expect(detachSessionRef(path, "session:sess-gone#turn-1")).toBe("skip-changed");+ expect(readFileSync(path, "utf8")).toBe(before);+ expect(detachSessionRef(path, "session:sess-new#turn-2")).toBe("detached");+ expect(parseFrontmatter(path)[0]["session_ref"]).toBeUndefined();+ });+});++describe("the detach runs under the signal's per-file lock", () => {+ test("a held lock leaves the file untouched and is reported skip-locked", () => {+ const path = writeOrphan();+ const before = readFileSync(path, "utf8");+ const held = acquireLockSync(path);+ try {+ expect(detachSessionRef(path, "session:sess-gone#turn-1")).toBe("skip-locked");+ const report = runOrphanRepair(vault, {+ apply: true,+ confirm: ORPHAN_REPAIR_CONFIRM_PHRASE,+ });+ expect(report.detached).toBe(0);+ expect(report.decisions.map((d) => d.action)).toEqual(["skip-locked"]);+ expect(readFileSync(path, "utf8")).toBe(before);+ } finally {+ held.release();+ }+ // Released, the same run detaches and leaves no lock behind.+ expect(detachSessionRef(path, "session:sess-gone#turn-1")).toBe("detached");+ expect(existsSync(`${path}.lock`)).toBe(false);+ });+});diff --git a/tests/core/brain/sync-lockfile.test.ts b/tests/core/brain/sync-lockfile.test.tsindex 276b55ea..8861157f 100644--- a/tests/core/brain/sync-lockfile.test.ts+++ b/tests/core/brain/sync-lockfile.test.ts@@ -234,6 +234,7 @@ describe("every module that takes a lock is under a scanned root", () => {["core/brain/idempotency-ledger.ts", "Brain"],["core/brain/ingest/content-manifest.ts", ".open-second-brain"],["core/brain/lineage/ledger.ts", "Brain"],+ ["core/brain/link-graph/orphan-repair.ts", "Brain"],["core/brain/payload-registry.ts", "Brain"],["core/brain/preference-txn.ts", "Brain"],["core/brain/query-demand.ts", "Brain"],diff --git a/tests/core/brain/write-batch.test.ts b/tests/core/brain/write-batch.test.tsindex 3225be6d..13535837 100644--- a/tests/core/brain/write-batch.test.ts+++ b/tests/core/brain/write-batch.test.ts@@ -37,7 +37,7 @@ import {writeFileSync,} from "node:fs";import { tmpdir } from "node:os";-import { join } from "node:path";+import { dirname, join } from "node:path";import { applyWriteBatch, WriteBatchError } from "../../../src/core/brain/write-batch.ts";import { listNoteWrites, type NoteWriteRecord } from "../../../src/core/brain/notes/write-log.ts";@@ -573,6 +573,24 @@ describe("applyWriteBatch note-write attribution", () => {expect(stored.toSorted()).toEqual([sha256Hex(first), sha256Hex(second)].toSorted());});+ test("the before-image is stored before the rewrite, so a failed store leaves the note as it was", () => {+ const abs = seedNote("Notes/Doc.md", "original body", "title: Doc");+ const before = readFileSync(abs, "utf8");+ // A file where the image directory belongs makes the store throw, the+ // way ENOSPC or EACCES under the images directory would.+ const imagesDir = dirname(writeImagePath(vault, sha256Hex(before)));+ mkdirSync(dirname(imagesDir), { recursive: true });+ writeFileSync(imagesDir, "not a directory");++ expect(() =>+ applyWriteBatch(vault, [{ kind: "update_note", path: "Notes/Doc.md", body: "next body" }]),+ ).toThrow();+ // The prior bytes were never replaced without their image, so a retry+ // is not mistaken for a byte-identical re-apply.+ expect(readFileSync(abs, "utf8")).toBe(before);+ expect(noteWrites()).toEqual([]);+ });+test("a refused batch records nothing, because no bytes were written", () => {seedNote("Notes/Doc.md", "original body", "title: Doc");expect(() =>diff --git a/tests/core/brain/write-path-advisory.test.ts b/tests/core/brain/write-path-advisory.test.tsnew file mode 100644index 00000000..22cc55bd--- /dev/null+++ b/tests/core/brain/write-path-advisory.test.ts@@ -0,0 +1,94 @@+/**+ * Unit tests for the write-path advisory (p4-silent-failure-hardening,+ * Task 3): a non-blocking, byte-identical-when-absent field on the note+ * write result that names the absolute home paths a call's authored+ * content embeds.+ *+ * The advisory reuses the hygiene grammar (`scanText` from+ * core/hygiene/hardcoded-paths.ts) AS-IS - its placeholder-segment filter+ * (`alice`, `user`, `you`, ... pass), its home-prefix scope, and its+ * `hygiene:allow-path` escape hatch. These tests pin that reuse rather+ * than a widened grammar: widening the detectors is a hygiene-module+ * decision, not a write-lane one. Path fragments are built by+ * concatenation so this file never trips the repo gate it tests against.+ */++import { describe, expect, test } from "bun:test";++import {+ WRITE_PATH_ADVISORY_KEY,+ writePathAdvisoryField,+} from "../../../src/core/brain/write-path-advisory.ts";++// The scanner is oblivious to string literals - compose the concrete+// segments so this file stays clean under the repo gate's grammar.+const HOME = "/home/";+const USERS = "/Users/";+const WIN_USERS = "C:\\Users\\";++describe("writePathAdvisoryField", () => {+ test("a body with a concrete home path yields page, line and detector", () => {+ const field = writePathAdvisoryField(+ ["intro", `see ${HOME}alice-dev/notes/x.md for details`, "outro"].join("\n"),+ "Notes/Leak.md",+ );+ const report = field[WRITE_PATH_ADVISORY_KEY]!;+ expect(report.total).toBe(1);+ expect(report.findings).toEqual([{ page: "Notes/Leak.md", line: 2, detector: "unix-home" }]);+ });++ test("the line number is the authored text's line, not the note's", () => {+ // The advisory scans the authored string on its own, so its line+ // numbers index that string: line 1 here even though the note this+ // content lands in has frontmatter above it.+ const field = writePathAdvisoryField(`path: ${USERS}devreal/docs/a.md`, "Notes/W.md");+ expect(field[WRITE_PATH_ADVISORY_KEY]!.findings).toEqual([+ { page: "Notes/W.md", line: 1, detector: "unix-home" },+ ]);+ });++ test("a windows home path names the windows detector", () => {+ const field = writePathAdvisoryField(`run ${WIN_USERS}devreal\\bin\\tool.exe`, "Notes/W.md");+ expect(field[WRITE_PATH_ADVISORY_KEY]!.findings[0]!.detector).toBe("windows-home");+ });++ test("every hit is counted, in line order", () => {+ const field = writePathAdvisoryField(+ [`${HOME}alice-dev/a.md`, "clean", `${HOME}bob-builder/b.md`].join("\n"),+ "Notes/Many.md",+ );+ const report = field[WRITE_PATH_ADVISORY_KEY]!;+ expect(report.total).toBe(2);+ expect(report.findings.map((f) => f.line)).toEqual([1, 3]);+ });++ test("a line carrying the hygiene allow marker is suppressed", () => {+ const marker = ["hygiene", "allow-path"].join(":");+ const field = writePathAdvisoryField(`${HOME}alice-dev/x.md ${marker}`, "Notes/W.md");+ expect(WRITE_PATH_ADVISORY_KEY in field).toBe(false);+ });++ test("a clean body contributes no field at all", () => {+ const field = writePathAdvisoryField("totally clean prose", "Notes/Clean.md");+ expect(WRITE_PATH_ADVISORY_KEY in field).toBe(false);+ expect(field).toEqual({});+ });++ test("no authored content contributes no field", () => {+ expect(WRITE_PATH_ADVISORY_KEY in writePathAdvisoryField(undefined, "Notes/W.md")).toBe(false);+ expect(WRITE_PATH_ADVISORY_KEY in writePathAdvisoryField("", "Notes/W.md")).toBe(false);+ });++ test("a placeholder segment passes - the grammar is reused, not widened", () => {+ // `alice` is one of the hygiene grammar's placeholder segments: an+ // intended example rather than a leak. The advisory keeps that rule+ // (and the plan's own example path) instead of widening the detector.+ const field = writePathAdvisoryField(`see ${HOME}alice/notes/x.md`, "Notes/W.md");+ expect(WRITE_PATH_ADVISORY_KEY in field).toBe(false);+ });++ test("findings never echo the matched path fragment", () => {+ const field = writePathAdvisoryField(`see ${HOME}alice-dev/notes/x.md`, "Notes/W.md");+ expect(JSON.stringify(field)).not.toContain(`${HOME}alice-dev`);+ });+});diff --git a/tests/core/install/ownership.test.ts b/tests/core/install/ownership.test.tsindex 046fba45..755b8d70 100644--- a/tests/core/install/ownership.test.ts+++ b/tests/core/install/ownership.test.ts@@ -58,6 +58,7 @@ import {} from "../../../src/core/install/ownership.ts";import {VAULT_BACKING,+ VAULT_BACKING_REMOTENESS,VAULT_BACKING_UNDETERMINED_REASON,} from "../../../src/core/vault-backing.ts";import { lexSource } from "../../helpers/source-lexer.ts";@@ -90,6 +91,7 @@ function ownership(overrides: Partial<DataOwnershipInput> = {}) {state: VAULT_BACKING.durable,filesystem: "ext4",reason: null,+ remoteness: VAULT_BACKING_REMOTENESS.nonRemote,detail: `${VAULT} is backed by ext4 (0xef53)`,},...overrides,@@ -130,6 +132,7 @@ describe("the statement is built from what was measured", () => {state: VAULT_BACKING.undetermined,filesystem: null,reason: VAULT_BACKING_UNDETERMINED_REASON.probeUnsupported,+ remoteness: VAULT_BACKING_REMOTENESS.nonRemote,detail: "the filesystem backing /tmp/some-vault was not probed",},}),@@ -147,6 +150,7 @@ describe("the statement is built from what was measured", () => {state: VAULT_BACKING.volatile,filesystem: "tmpfs",reason: null,+ remoteness: VAULT_BACKING_REMOTENESS.nonRemote,detail: "/tmp/some-vault is backed by tmpfs (0x1021994)",},}),diff --git a/tests/core/redactor-scan-window.test.ts b/tests/core/redactor-scan-window.test.tsindex 355fe9b4..c537ba33 100644--- a/tests/core/redactor-scan-window.test.ts+++ b/tests/core/redactor-scan-window.test.ts@@ -27,6 +27,7 @@ import { describe, expect, test } from "bun:test";import { EGRESS_OUTCOME, redactForEgress } from "../../src/core/egress/guard.ts";import {MAX_REDACTOR_INPUT,+ REDACTION_PLACEHOLDER,SCAN_TRUNCATED_MARKER,redactStructured,wasScanTruncated,@@ -81,3 +82,86 @@ describe("the window boundary is unchanged", () => {expect(verdict.outcome).toBe(EGRESS_OUTCOME.released);});});++describe("a sibling {name, value} secret pair is judged by the name", () => {+ // The four text passes and the walk's key-name rule all read the+ // credential's OWN key. Configuration formats that put the name in a+ // SIBLING member - the environment-entry shape ECS, Kubernetes and+ // docker-compose share - therefore passed whole: the walk saw the key+ // `value` (which says nothing) and never consulted the sibling+ // `DB_PASSWORD`, so the literal secret rode through the export boundary+ // with `redacted: false`.++ test("an environment-style secret pair has its value redacted", () => {+ const result = redactStructured({+ environment: [{ name: "DB_PASSWORD", value: "hunter2s3cret" }],+ });+ expect(result.redacted).toBe(true);+ const entry = (result.value as { environment: Array<{ name: string; value: string }> })+ .environment[0]!;+ expect(entry.name).toBe("DB_PASSWORD");+ expect(entry.value).toBe(REDACTION_PLACEHOLDER);+ });++ test("the same pair is redacted under redactTokens too", () => {+ const result = redactStructured(+ { environment: [{ name: "DB_PASSWORD", value: "hunter2s3cret" }] },+ { redactTokens: true },+ );+ expect(result.redacted).toBe(true);+ const entry = (result.value as { environment: Array<{ name: string; value: string }> })+ .environment[0]!;+ expect(entry.value).toBe(REDACTION_PLACEHOLDER);+ expect(entry.name).toBe("DB_PASSWORD");+ });++ test("extra members of the entry survive; only the value is a credential", () => {+ const result = redactStructured({+ environment: [{ name: "DB_PASSWORD", value: "hunter2s3cret", required: true }],+ });+ const entry = result.value as {+ environment: Array<{ name: string; value: string; required: boolean }>;+ };+ expect(entry.environment[0]).toEqual({+ name: "DB_PASSWORD",+ value: REDACTION_PLACEHOLDER,+ required: true,+ });+ });++ test("a pair whose name is not a secret key passes through", () => {+ const input = { environment: [{ name: "NODE_ENV", value: "production" }] };+ const result = redactStructured(input);+ expect(result.value).toEqual(input);+ expect(result.redacted).toBe(false);+ });++ test("an ECS valueFrom ARN reference passes through untouched", () => {+ // valueFrom names a REFERENCE, not a literal: there is no `value`+ // member to redact, and the ARN is an identifier.+ const input = {+ environment: [+ { name: "DB_PASSWORD", valueFrom: { arn: "arn:aws:ecs:us-east-1:123:task/c" } },+ ],+ };+ const result = redactStructured(input);+ expect(result.value).toEqual(input);+ expect(result.redacted).toBe(false);+ });++ test("non-object array elements are unaffected", () => {+ const input = { items: ["plain text", 42, null, true, ["nested", "elements"]] };+ const result = redactStructured(input);+ expect(result.value).toEqual(input);+ expect(result.redacted).toBe(false);+ });++ test("a secret-shaped VALUE in a pair is the placeholder, not a partial scan", () => {+ // A vendor-prefixed value is fully replaced by the sibling rule, the+ // way it would be under a secret KEY name.+ const token = fakeCredential("sk-", "live-SIBLINGPAIR0000");+ const result = redactStructured({ env: [{ name: "API_KEY", value: token }] });+ const entry = (result.value as { env: Array<{ value: string }> }).env[0]!;+ expect(entry.value).toBe(REDACTION_PLACEHOLDER);+ });+});diff --git a/tests/core/redactor.test.ts b/tests/core/redactor.test.tsindex 489f1838..cf6fb587 100644--- a/tests/core/redactor.test.ts+++ b/tests/core/redactor.test.ts@@ -10,6 +10,8 @@ import {stripPrivateRegions,wasScanTruncated,} from "../../src/core/redactor.ts";+import { EGRESS_OUTCOME, redactForEgress } from "../../src/core/egress/guard.ts";+import { fakeCredential } from "../helpers/fake-credentials.ts";describe("stripPrivateRegions", () => {test("strips balanced private regions across lines", () => {@@ -223,6 +225,179 @@ describe("redactRawOutput infra-topology pass (redactInfra)", () => {});});+describe("URL credentials whose password carries a slash", () => {+ // The password class was `[^\s/@]+`, which excluded the one character a+ // generated database password most often carries. The regex then failed+ // the match ENTIRELY, so both the user and the password left verbatim -+ // a wider leak than if the pass had never run.+ const SLASH_PASSWORD_URL = "postgres://admin:s3cr3t/[email protected]:5432/prod";++ test("the export-boundary default options redact user and password, keeping scheme and host", () => {+ const verdict = redactForEgress("brain-bank-export", { dsn: SLASH_PASSWORD_URL });+ expect(verdict.outcome).toBe(EGRESS_OUTCOME.released);+ if (verdict.outcome !== EGRESS_OUTCOME.released) throw new Error("unreachable");+ expect(verdict.payload.dsn).toBe("postgres://***REDACTED***@db.internal:5432/prod");+ expect(verdict.redacted).toBe(true);+ });++ test("redactUrlCredentials alone redacts user and password, keeping scheme and host", () => {+ const out = redactRawOutput(`connect ${SLASH_PASSWORD_URL} now`, {+ redactUrlCredentials: true,+ });+ expect(out).toBe("connect postgres://***REDACTED***@db.internal:5432/prod now");+ });++ test("the slash-free control URL redacts as before", () => {+ const out = redactRawOutput("git clone https://alice:[email protected]/x.git", {+ redactUrlCredentials: true,+ });+ expect(out).toBe("git clone https://***REDACTED***@github.com/x.git");+ });+});++describe("URL credentials with an empty username", () => {+ // The `user:pass@` spelling requires a username, so the sibling spelling+ // connection strings actually use for password-only authorities -+ // `scheme://:pass@host` - matched nothing: the password rode past the+ // pass verbatim. The user half is optional; the colon and `@` anchors+ // stay, so a URL whose path carries an `@` but no userinfo colon is+ // still untouched.+ const EMPTY_USER_URL = "postgres://:[email protected]:5432/prod";+ const EMPTY_USER_SLASH_PASSWORD_URL = "redis://:s3cr3t/[email protected]:6379/0";++ test("the export-boundary default options redact a password-only userinfo section", () => {+ const verdict = redactForEgress("brain-bank-export", { dsn: EMPTY_USER_URL });+ expect(verdict.outcome).toBe(EGRESS_OUTCOME.released);+ if (verdict.outcome !== EGRESS_OUTCOME.released) throw new Error("unreachable");+ expect(verdict.payload.dsn).toBe("postgres://***REDACTED***@db.internal:5432/prod");+ expect(verdict.redacted).toBe(true);+ });++ test("redactUrlCredentials alone redacts an empty username with a slash password", () => {+ const out = redactRawOutput(`connect ${EMPTY_USER_SLASH_PASSWORD_URL} now`, {+ redactUrlCredentials: true,+ });+ expect(out).toBe("connect redis://***REDACTED***@cache.internal:6379/0 now");+ });++ test("a path-only `@` without a userinfo colon is not rewritten", () => {+ const out = redactRawOutput("see https://mastodon.social/@someone for the thread", {+ redactUrlCredentials: true,+ });+ expect(out).toBe("see https://mastodon.social/@someone for the thread");+ });+});++describe("URL credentials never swallow a port and a path", () => {+ // The password class crosses `/`, so without a guard the `host:port`+ // colon read as the userinfo colon and a later `@` in the path closed+ // the match: the host, the port and half the path became "credentials".+ const UNTOUCHED = [+ "https://example.com:443/a@b",+ "http://localhost:5173/@vite/client",+ "https://registry.npmjs.org:443/@types/node",+ "https://example.com:8443/users/[email protected]",+ "see https://example.com:8080/users/@alice now",+ ];++ for (const url of UNTOUCHED) {+ test(`${url} stays byte-identical`, () => {+ expect(redactRawOutput(url, { redactUrlCredentials: true })).toBe(url);+ });+ }++ const REDACTED: Array<[string, string]> = [+ ["https://user:pa/ss@h/x", "https://***REDACTED***@h/x"],+ ["https://:secret@h", "https://***REDACTED***@h"],+ ["https://u:p?q@h", "https://***REDACTED***@h"],+ ["https://user:8080@h/x", "https://***REDACTED***@h/x"],+ ["https://alice:[email protected]", "https://***REDACTED***@host.example"],+ ["https://u:123#x@h", "https://***REDACTED***@h"],+ ];++ test("a slash password that begins with digits is the documented miss, as on main", () => {+ const url = "https://bob:8080/x@host";+ expect(redactRawOutput(url, { redactUrlCredentials: true })).toBe(url);+ });++ for (const [input, expected] of REDACTED) {+ test(`${input} still redacts its userinfo`, () => {+ expect(redactRawOutput(input, { redactUrlCredentials: true })).toBe(expected);+ });+ }+});++describe("bare JWT (three base64url segments)", () => {+ // A JWT's header is compact JSON, so it always base64s to the `eyJ`+ // prefix; the 20-character canonical header slips the 24-character+ // high-entropy gate, and a bare token carries no key=value shape for the+ // assignment passes. Nothing below default options saw it at all.+ const JWT = fakeCredential(+ "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9",+ ".eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIn0",+ ".SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c",+ );++ test("a bare three-segment JWT is redacted under default options", () => {+ const out = redactRawOutput(`token ${JWT} end`);+ expect(out).toBe(`token ${REDACTION_PLACEHOLDER} end`);+ });++ test("a JWT leaf is redacted at the export boundary by default", () => {+ const verdict = redactForEgress("brain-bank-export", { pasted: JWT });+ expect(verdict.outcome).toBe(EGRESS_OUTCOME.released);+ if (verdict.outcome !== EGRESS_OUTCOME.released) throw new Error("unreachable");+ expect(verdict.payload.pasted).toBe(REDACTION_PLACEHOLDER);+ expect(verdict.redacted).toBe(true);+ });++ test("a JWT is redacted whole under the token pass too, not left half-standing", () => {+ // The high-entropy pass alone ate the payload and signature segments+ // and left the header: `eyJ…J9.***REDACTED***.***REDACTED***` still+ // announces a credential and hands over its algorithm.+ const out = redactRawOutput(`token ${JWT}`, { redactTokens: true });+ expect(out).toBe(`token ${REDACTION_PLACEHOLDER}`);+ });++ test("a Bearer-prefixed JWT keeps the prefix and one placeholder (regression guard)", () => {+ const out = redactRawOutput(`Authorization: Bearer ${JWT}`);+ expect(out).toBe("Authorization: Bearer ***REDACTED***");+ });++ test("a JWT with a payload longer than 4096 characters is redacted whole", () => {+ const payload = Buffer.from(JSON.stringify({ sub: "x".repeat(3300) })).toString("base64url");+ expect(payload.length).toBeGreaterThan(4096);+ const token = fakeCredential(+ "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9",+ `.${payload}`,+ ".c2lnbmF0dXJlLXNlZ21lbnQ",+ );+ expect(redactRawOutput(`token ${token} end`)).toBe(`token ${REDACTION_PLACEHOLDER} end`);+ });++ test("a JWT whose header JSON is whitespace-formatted is redacted", () => {+ for (const header of [+ '{ "alg": "HS256", "typ": "JWT" }',+ '{\n "alg": "HS256"\n}',+ '{\t"alg":"HS256"}',+ '{\r\n"alg":"HS256"}',+ ]) {+ const encoded = Buffer.from(header).toString("base64url");+ const token = fakeCredential(+ encoded,+ ".eyJzdWIiOiIxMjM0NTY3ODkwIn0",+ ".c2lnbmF0dXJlLXNlZ21lbnQ",+ );+ expect(redactRawOutput(`token ${token} end`)).toBe(`token ${REDACTION_PLACEHOLDER} end`);+ }+ });++ test("short dotted words that merely start like a header stay prose", () => {+ const text = "see ewok.item.list and eyAb.cdef.ghij here";+ expect(redactRawOutput(text)).toBe(text);+ });+});+describe("normaliseTextField", () => {test("returns empty string for non-string input", () => {expect(normaliseTextField(123 as unknown, { maxLen: 10 })).toBe("");diff --git a/tests/core/search/stale-embeddings-baseline.test.ts b/tests/core/search/stale-embeddings-baseline.test.tsnew file mode 100644index 00000000..c4539b3a--- /dev/null+++ b/tests/core/search/stale-embeddings-baseline.test.ts@@ -0,0 +1,139 @@+/**+ * Which stored vectors count as stale when the configuration leaves the+ * embedding dimension unset (PR #220 review).+ *+ * `embedding_dimension` is null unless configured, and the stale count+ * used to return 0 whenever it was null - so on a default configuration+ * a model switch reported no stale vectors at all. The baseline now+ * resolves the dimension the way the indexer does (configured, then the+ * local embedder's default, then the stored value), and with no+ * dimension at all the vectors are compared by model alone.+ */++import { Database } from "bun:sqlite";+import { expect, test } from "bun:test";++import { LOCAL_DEFAULT_DIMENSION } from "../../../src/core/search/embeddings/local-provider.ts";+import { LOCAL_EMBEDDING_MODEL } from "../../../src/core/search/embeddings/signature.ts";+import { staleBaseline } from "../../../src/core/search/store/counts.ts";+import { staleEmbeddings } from "../../../src/core/search/store/vectors.ts";+import { indexVault } from "../../../src/core/search/indexer.ts";+import { Store } from "../../../src/core/search/store.ts";+import { createTempVault, makeConfig, writeMd } from "../../helpers/search-fixtures.ts";+import { sqliteVecLoadable } from "../../helpers/sqlite-vec.ts";++function db(rows: ReadonlyArray<[string, number]>, storedDim: string | null = null): Database {+ const d = new Database(":memory:");+ d.run("CREATE TABLE embeddings (chunk_id INTEGER PRIMARY KEY, model TEXT, dimension INTEGER)");+ d.run("CREATE TABLE index_state (key TEXT PRIMARY KEY, value TEXT, updated_at TEXT)");+ rows.forEach(([model, dim], i) =>+ d.run("INSERT INTO embeddings VALUES (?, ?, ?)", [i + 1, model, dim]),+ );+ if (storedDim !== null) {+ d.run("INSERT INTO index_state VALUES ('embedding_dimension', ?, '')", [storedDim]);+ }+ return d;+}++test("a model switch with no dimension anywhere is still counted as stale", () => {+ const d = db([+ ["old-model", 8],+ ["old-model", 8],+ ["new-model", 8],+ ]);+ expect(staleEmbeddings(d, "new-model", null)).toBe(2);+});++test("with no model there is no baseline and nothing is counted", () => {+ expect(staleEmbeddings(db([["old-model", 8]]), null, null)).toBe(0);+});++test("the baseline dimension falls back to the stored dimension", () => {+ const d = db([], "1536");+ expect(staleBaseline(d, { provider: "openai-compat", model: "m", dimension: null })).toEqual({+ model: "m",+ dimension: 1536,+ });+ expect(staleBaseline(d, { provider: "openai-compat", model: "m", dimension: 8 })).toEqual({+ model: "m",+ dimension: 8,+ });+});++test("the local embedder is named by its built-in model and default dimension", () => {+ expect(staleBaseline(db([]), { provider: "local", model: null, dimension: null })).toEqual({+ model: LOCAL_EMBEDDING_MODEL,+ dimension: LOCAL_DEFAULT_DIMENSION,+ });+});++test("no configured and no stored dimension leaves the dimension unresolved", () => {+ expect(staleBaseline(db([]), { provider: "openai-compat", model: "m", dimension: null })).toEqual(+ { model: "m", dimension: null },+ );+});++test("Store.counts reports vectors from another model as stale with no configured dimension", async () => {+ if (!sqliteVecLoadable()) return;+ const v = createTempVault("stale-baseline-store");+ try {+ writeMd(v.vault, "a.md", "# A\n\nA note embedded under the first model.");+ await indexVault(+ makeConfig({+ vault: v.vault,+ dbPath: v.dbPath,+ semantic: { enabled: true, provider: "local", dimension: 256 },+ }),+ { embeddings: true },+ );+ // The configuration now names another model and leaves the dimension+ // unset - the default setup, where the stale count used to read 0.+ const switched = makeConfig({+ vault: v.vault,+ dbPath: v.dbPath,+ semantic: {+ enabled: true,+ provider: "openai-compat",+ model: "vendor-x/embed-v2",+ dimension: null,+ },+ });+ const store = await Store.open(switched, { mode: "read" });+ try {+ const counts = store.counts();+ expect(counts.embeddings).toBeGreaterThan(0);+ expect(counts.staleEmbeddings).toBe(counts.embeddings);+ } finally {+ await store.close();+ }+ } finally {+ v.cleanup();+ }+});++test("Store.counts names the local embedder by its built-in model", async () => {+ if (!sqliteVecLoadable()) return;+ const v = createTempVault("stale-baseline-local");+ try {+ writeMd(v.vault, "a.md", "# A\n\nA note embedded at 256 dimensions.");+ const local = (dimension: number) =>+ makeConfig({+ vault: v.vault,+ dbPath: v.dbPath,+ semantic: { enabled: true, provider: "local", dimension },+ });+ await indexVault(local(256), { embeddings: true });+ // The local configuration carries no model string: without the+ // baseline naming it, a dimension switch counted nothing as stale.+ const store = await Store.open(local(128), { mode: "read" });+ try {+ const counts = store.counts();+ expect(counts.embeddings).toBeGreaterThan(0);+ expect(counts.staleEmbeddings).toBe(counts.embeddings);+ } finally {+ await store.close();+ }+ } finally {+ v.cleanup();+ }+});diff --git a/tests/core/search/store-remoteness.test.ts b/tests/core/search/store-remoteness.test.tsnew file mode 100644index 00000000..cfba90c4--- /dev/null+++ b/tests/core/search/store-remoteness.test.ts@@ -0,0 +1,229 @@+/**+ * The remoteness decision at the index open.+ *+ * A vault's index can live on a network filesystem: `search_db_path`+ * overrides the default in-vault location, and pointing it at an NFS or+ * SMB mount is precisely the deployment that breaks under WAL, whose+ * shared-memory index needs locking semantics network filesystems do not+ * provide. The response is deliberately the smaller hammer: on a backing+ * the vault-backing probe classifies remote, the write open skips WAL+ * (stays on the DELETE journal mode a fresh file already has) and says so+ * on stderr - no refusal, no escape-hatch env, no silent flip.+ *+ * The probe is injected: simulating an NFS backing needs no real mount.+ */++import { describe, expect, spyOn, test } from "bun:test";+import { Database } from "bun:sqlite";+import { readFileSync } from "node:fs";++import {+ journalModeForBacking,+ openReadDatabase,+ openWriteDatabase,+} from "../../../src/core/search/store/lifecycle.ts";+import { probeVaultBacking } from "../../../src/core/vault-backing.ts";+import { createTempVault, makeConfig, writeMd } from "../../helpers/search-fixtures.ts";++/** Linux NFS_SUPER_MAGIC. */+const NFS = 0x6969;+/** Linux CIFS_SUPER_MAGIC. */+const CIFS = 0xff534d42;+const EXT4 = 0xef53;+const TMPFS = 0x01021994;+const FUSE = 0x65735546;+/** A magic number no released filesystem uses; stands for "not in the table". */+const UNRECOGNISED = 0x0badf00d;++describe("journalModeForBacking, the pure decision", () => {+ const DB = "/vault/.open-second-brain/brain.sqlite";++ test("named network filesystems skip WAL and name themselves and the path", () => {+ for (const [magic, name] of [+ [NFS, "nfs"],+ [CIFS, "cifs"],+ ] as const) {+ const backing = probeVaultBacking("/vault/.open-second-brain", {+ platform: "linux",+ statfs: () => ({ type: magic }),+ });+ const decision = journalModeForBacking(backing, DB);+ expect(`${name}: ${decision.journalMode}`).toBe(`${name}: delete`);+ expect(decision.warning).toContain(`backed by ${name}`);+ expect(decision.warning).toContain(DB);+ }+ });++ test("every silent or known-local backing keeps WAL with no warning", () => {+ for (const magic of [EXT4, TMPFS, FUSE, UNRECOGNISED]) {+ const backing = probeVaultBacking("/vault/.open-second-brain", {+ platform: "linux",+ statfs: () => ({ type: magic }),+ });+ const decision = journalModeForBacking(backing, DB);+ expect(`${magic}: ${decision.journalMode}`).toBe(`${magic}: wal`);+ expect(decision.warning).toBeNull();+ }+ });++ test("a platform with no statfs keeps WAL and stays silent", () => {+ const backing = probeVaultBacking("/vault/.open-second-brain", {+ platform: "darwin",+ statfs: () => ({ type: NFS }),+ });+ const decision = journalModeForBacking(backing, DB);+ expect(decision.journalMode).toBe("wal");+ expect(decision.warning).toBeNull();+ });+});++/** The stderr lines a write open emits, with console restored afterwards. */+function captureStderr(): { lines: () => string[]; done: () => void } {+ const lines: string[] = [];+ const spy = spyOn(console, "error").mockImplementation((...args: unknown[]) => {+ lines.push(args.map(String).join(" "));+ });+ return { lines: () => lines, done: () => spy.mockRestore() };+}++describe("openWriteDatabase on a remote backing", () => {+ test("opens with WAL off, says so once, and leaves the vault working", async () => {+ const v = createTempVault("store-remote");+ const note = writeMd(v.vault, "Notes/note.md", "# Kept\n\nthe open must not touch this\n");+ const config = makeConfig({ vault: v.vault, dbPath: v.dbPath });+ const stderr = captureStderr();+ let opened: Awaited<ReturnType<typeof openWriteDatabase>> | null = null;+ try {+ opened = await openWriteDatabase(config, false, {+ statfs: () => ({ type: NFS }),+ });++ // The decision is read back from the file, not from memory: the+ // journal mode the connection actually runs under is DELETE.+ const mode = opened.db.query<{ journal_mode: string }, []>("PRAGMA journal_mode").get();+ expect(mode?.journal_mode).toBe("delete");++ // One stderr warning, naming the filesystem it classified.+ const warnings = stderr.lines().filter((line) => line.includes("network filesystem"));+ expect(warnings).toHaveLength(1);+ expect(warnings[0]).toContain("nfs");++ // The open is a working write open, not a refuse: data written+ // through it survives.+ opened.db.exec("CREATE TABLE remoteness_probe (v TEXT)");+ opened.db.exec("INSERT INTO remoteness_probe (v) VALUES ('kept')");+ opened.db.close(true);+ await opened.release();+ opened = null;++ // The vault's own data is untouched, and the row is readable fresh.+ expect(readFileSync(note, "utf8")).toContain("the open must not touch this");+ const reread = new Database(v.dbPath);+ try {+ const row = reread.query<{ v: string }, []>("SELECT v FROM remoteness_probe").get();+ expect(row?.v).toBe("kept");+ } finally {+ reread.close(true);+ }+ } finally {+ stderr.done();+ if (opened) {+ opened.db.close(true);+ await opened.release();+ }+ v.cleanup();+ }+ });++ test("the default local classification keeps WAL and stays silent", async () => {+ const v = createTempVault("store-local");+ const config = makeConfig({ vault: v.vault, dbPath: v.dbPath });+ const stderr = captureStderr();+ let opened: Awaited<ReturnType<typeof openWriteDatabase>> | null = null;+ try {+ // No injected probe: the real statfs answers for the temp vault,+ // which is a local disk or tmpfs - either way non-remote.+ opened = await openWriteDatabase(config, false);++ const mode = opened.db.query<{ journal_mode: string }, []>("PRAGMA journal_mode").get();+ expect(mode?.journal_mode).toBe("wal");+ expect(stderr.lines().join("\n")).not.toContain("network filesystem");+ } finally {+ stderr.done();+ if (opened) {+ opened.db.close(true);+ await opened.release();+ }+ v.cleanup();+ }+ });+});++/** A migrated index left in WAL, with a second connection holding it open. */+async function walIndex(label: string) {+ const v = createTempVault(label);+ const config = makeConfig({ vault: v.vault, dbPath: v.dbPath });+ const seeded = await openWriteDatabase(config, false);+ seeded.db.close(true);+ await seeded.release();+ const holder = new Database(v.dbPath);+ holder.query("SELECT count(*) FROM sqlite_master").get();+ return { v, config, holder };+}++describe("a journal-mode switch another connection blocks", () => {+ // `PRAGMA journal_mode = delete` on an index in WAL fails at once with+ // "database is locked" while any other connection has the file open,+ // busy timeout or not. That is a mode the open could not change, not a+ // broken index: the read open keeps WAL (a read is correct in it), and+ // the write open keeps it too and says so, instead of a raw error or an+ // INDEX_UNREADABLE that would trigger a full reindex.++ test("the read open keeps WAL, opens, and says nothing", async () => {+ const { v, config, holder } = await walIndex("store-busy-read");+ const stderr = captureStderr();+ try {+ const opened = openReadDatabase(config, false, { statfs: () => ({ type: NFS }) });+ try {+ const mode = opened.db.query<{ journal_mode: string }, []>("PRAGMA journal_mode").get();+ expect(mode?.journal_mode).toBe("wal");+ } finally {+ opened.db.close(true);+ }+ // Read opens happen per query: a blocked switch there is silent.+ expect(stderr.lines()).toEqual([]);+ } finally {+ stderr.done();+ holder.close(true);+ v.cleanup();+ }+ });++ test("the write open keeps WAL, opens, and warns once per path across opens", async () => {+ const { v, config, holder } = await walIndex("store-busy-write");+ const stderr = captureStderr();+ let opened: Awaited<ReturnType<typeof openWriteDatabase>> | null = null;+ try {+ opened = await openWriteDatabase(config, false, { statfs: () => ({ type: NFS }) });+ const mode = opened.db.query<{ journal_mode: string }, []>("PRAGMA journal_mode").get();+ expect(mode?.journal_mode).toBe("wal");+ opened.db.close(true);+ await opened.release();+ // The second write open meets the same blocked switch and stays quiet.+ opened = await openWriteDatabase(config, false, { statfs: () => ({ type: NFS }) });+ const again = opened.db.query<{ journal_mode: string }, []>("PRAGMA journal_mode").get();+ expect(again?.journal_mode).toBe("wal");+ const busy = stderr.lines().filter((line) => line.includes("journal mode stays WAL"));+ expect(busy).toHaveLength(1);+ expect(busy[0]).toContain(v.dbPath);+ } finally {+ stderr.done();+ if (opened) {+ opened.db.close(true);+ await opened.release();+ }+ holder.close(true);+ v.cleanup();+ }+ });+});diff --git a/tests/core/vault-backing.test.ts b/tests/core/vault-backing.test.tsindex 35231eea..0fcbe262 100644--- a/tests/core/vault-backing.test.ts+++ b/tests/core/vault-backing.test.ts@@ -23,10 +23,13 @@ import { dirname, join, resolve } from "node:path";import { fileURLToPath } from "node:url";import {+ isVaultBackingRemoteness,isVaultBackingState,isVaultBackingUndeterminedReason,probeVaultBacking,VAULT_BACKING,+ VAULT_BACKING_REMOTENESS,+ VAULT_BACKING_REMOTENESS_STATES,VAULT_BACKING_STATES,VAULT_BACKING_UNDETERMINED_REASON,VAULT_BACKING_UNDETERMINED_REASONS,@@ -38,6 +41,9 @@ const TMPFS = 0x01021994;const OVERLAYFS = 0x794c7630;/** A magic number no released filesystem uses; stands for "not in the table". */const UNRECOGNISED = 0x0badf00d;+const NFS = 0x6969;+const CIFS = 0xff534d42;+const FUSE = 0x65735546;function withFsType(type: number) {return () => ({ type });@@ -68,6 +74,121 @@ describe("the filesystem under the vault path decides the state", () => {});});+describe("the remoteness axis, beside durability", () => {+ test("named network filesystems classify remote without losing durability", () => {+ for (const [magic, name] of [+ [NFS, "nfs"],+ [CIFS, "cifs"],+ ] as const) {+ const verdict = probeVaultBacking("/vault", { platform: "linux", statfs: withFsType(magic) });+ expect(`${name} reads back as ${verdict.filesystem}`).toBe(`${name} reads back as ${name}`);+ expect(verdict.remoteness).toBe(VAULT_BACKING_REMOTENESS.remote);+ // The survival claim is unchanged: the bytes still outlive this+ // process and this reboot. Remoteness is a second axis answering a+ // different question (is this storage shared with other hosts), not+ // a different answer to the first.+ expect(verdict.state).toBe(VAULT_BACKING.durable);+ }+ });++ test("the other Linux network filesystems classify remote too", () => {+ // A modern `mount -t cifs` share reports SMB2_SUPER_MAGIC, not+ // CIFS_SUPER_MAGIC; ceph, afs, the legacy smbfs and 9p (WSL2 drvfs)+ // are cross-host storage in the same sense.+ for (const [magic, name] of [+ [0xfe534d42, "smb2"],+ [0x00c36400, "ceph"],+ [0x5346414f, "afs"],+ [0x6b414653, "afs"],+ [0x517b, "smbfs"],+ [0x01021997, "9p"],+ ] as const) {+ const verdict = probeVaultBacking("/vault", { platform: "linux", statfs: withFsType(magic) });+ expect(`${magic.toString(16)}: ${verdict.filesystem} ${verdict.remoteness}`).toBe(+ `${magic.toString(16)}: ${name} ${VAULT_BACKING_REMOTENESS.remote}`,+ );+ expect(verdict.state).toBe(VAULT_BACKING.durable);+ }+ });++ test("a Windows UNC path is remote without a statfs probe", () => {+ for (const path of [+ "\\\\server\\share\\vault\\.open-second-brain",+ "//server/share/vault",+ "\\\\?\\UNC\\server\\share\\vault",+ ]) {+ const verdict = probeVaultBacking(path, { platform: "win32" });+ expect(`${path}: ${verdict.remoteness}`).toBe(`${path}: ${VAULT_BACKING_REMOTENESS.remote}`);+ expect(verdict.filesystem).toBe("unc-share");+ }+ // A drive path and the extended-length local forms stay unprobed.+ for (const path of ["C:\\vault", "\\\\?\\C:\\vault", "\\\\.\\C:\\vault"]) {+ const verdict = probeVaultBacking(path, { platform: "win32" });+ expect(`${path}: ${verdict.reason}`).toBe(+ `${path}: ${VAULT_BACKING_UNDETERMINED_REASON.probeUnsupported}`,+ );+ expect(verdict.remoteness).toBe(VAULT_BACKING_REMOTENESS.nonRemote);+ }+ // A double slash on Linux is not a share.+ expect(+ probeVaultBacking("//server/share", { platform: "linux", statfs: withFsType(EXT4) })+ .remoteness,+ ).toBe(VAULT_BACKING_REMOTENESS.nonRemote);+ });++ test("fuse stays non-remote, and so does every disk and memory filesystem", () => {+ for (const magic of [FUSE, EXT4, TMPFS, OVERLAYFS]) {+ const verdict = probeVaultBacking("/vault", { platform: "linux", statfs: withFsType(magic) });+ expect(verdict.remoteness).toBe(VAULT_BACKING_REMOTENESS.nonRemote);+ }+ });++ test("probe-unsupported and fs-type-unknown stay silent and non-remote", () => {+ // Non-remote is the ABSENCE of a network finding, not a finding that+ // the storage is local: nothing was read, so nothing is claimed, and+ // a consumer keeps its default behaviour (WAL stays on).+ const unprobed = probeVaultBacking("/vault", { platform: "darwin", statfs: withFsType(NFS) });+ expect(unprobed.state).toBe(VAULT_BACKING.undetermined);+ expect(unprobed.reason).toBe(VAULT_BACKING_UNDETERMINED_REASON.probeUnsupported);+ expect(unprobed.remoteness).toBe(VAULT_BACKING_REMOTENESS.nonRemote);+ const unknown = probeVaultBacking("/vault", {+ platform: "linux",+ statfs: withFsType(UNRECOGNISED),+ });+ expect(unknown.reason).toBe(VAULT_BACKING_UNDETERMINED_REASON.fsTypeUnknown);+ expect(unknown.filesystem).toBeNull();+ expect(unknown.remoteness).toBe(VAULT_BACKING_REMOTENESS.nonRemote);+ });+});++describe("an injected statfs carries its own meaning", () => {+ test("it is honoured without an explicit platform, on any host", () => {+ // The platform veto exists because the DEFAULT statfsSync means+ // something else off Linux. A caller who injects a statfs vouches for+ // what it returns - the index open does exactly that to simulate a+ // network backing - so the injection implies the probeable platform.+ // This is the contract the Windows CI run turned on: there the veto+ // ran first, the injected nfs never applied, and the index opened+ // under WAL with no warning.+ const verdict = probeVaultBacking("/vault", { statfs: withFsType(NFS) });+ expect(verdict.filesystem).toBe("nfs");+ expect(verdict.remoteness).toBe(VAULT_BACKING_REMOTENESS.remote);+ expect(verdict.reason).toBeNull();+ });++ test("an explicitly pinned non-probeable platform still vetoes the injection", () => {+ // The guard on the rule above: an explicit `platform` is the caller+ // speaking about the host, and that word outranks any injection.+ const verdict = probeVaultBacking("/vault", {+ platform: "win32",+ statfs: withFsType(NFS),+ });+ expect(verdict.state).toBe(VAULT_BACKING.undetermined);+ expect(verdict.reason).toBe(VAULT_BACKING_UNDETERMINED_REASON.probeUnsupported);+ expect(verdict.remoteness).toBe(VAULT_BACKING_REMOTENESS.nonRemote);+ });+});+describe("everything the probe cannot establish says so, with a reason", () => {test("an unrecognised magic number is undetermined, NOT durable", () => {const verdict = probeVaultBacking("/vault", {@@ -137,6 +258,21 @@ describe("the vocabulary is closed and carries the could-not-tell member", () =>);}});++ test("the remoteness vocabulary is closed and unreadable as the other axes", () => {+ expect(VAULT_BACKING_REMOTENESS_STATES).toContain(VAULT_BACKING_REMOTENESS.remote);+ expect(VAULT_BACKING_REMOTENESS_STATES).toContain(VAULT_BACKING_REMOTENESS.nonRemote);+ for (const state of VAULT_BACKING_STATES) {+ expect(`${state} is a remoteness: ${isVaultBackingRemoteness(state)}`).toBe(+ `${state} is a remoteness: false`,+ );+ }+ for (const reason of VAULT_BACKING_UNDETERMINED_REASONS) {+ expect(`${reason} is a remoteness: ${isVaultBackingRemoteness(reason)}`).toBe(+ `${reason} is a remoteness: false`,+ );+ }+ });});describe("no container signal reaches this verdict", () => {diff --git a/tests/core/vault.test.ts b/tests/core/vault.test.tsindex 678f4139..dfc2b8b9 100644--- a/tests/core/vault.test.ts+++ b/tests/core/vault.test.ts@@ -6,6 +6,8 @@ import {mkdtempSync,readFileSync,rmSync,+ statSync,+ utimesSync,writeFileSync,} from "node:fs";import { tmpdir } from "node:os";@@ -23,6 +25,7 @@ import {parseFrontmatterWithNotices,slugify,writeFrontmatter,+ writeFrontmatterAtomic,} from "../../src/core/vault.ts";import { TRANSPORT_REACH } from "../../src/core/graph/transport-reach.ts";import { CHMOD_CANNOT_DENY } from "../helpers/platform.ts";@@ -492,3 +495,55 @@ describe("formatFrontmatter refuses a key the parser could not read back", () =>expect(existsSync(path)).toBe(false);});});++function ageMtime(path: string): number {+ const past = Date.now() / 1000 - 60;+ utimesSync(path, past, past);+ return statSync(path).mtimeMs;+}++/**+ * mtime stability on the overwrite branch (p4-silent-failure-hardening,+ * Task 6). `writeFrontmatterAtomic` goes through the atomic rename+ * pipeline, which lands a fresh inode on every call: a re-render that+ * produced the exact bytes already on disk still bumped the file's+ * mtime, and every mtime consumer downstream (recency ranking, validity+ * windows, vault-delta counts) read that phantom bump as a change. The+ * overwrite branch now skips byte-identical targets and reports whether+ * it wrote.+ */+describe("writeFrontmatterAtomic mtime stability", () => {+ test("an overwrite of identical content leaves the mtime alone and reports no write", () => {+ const path = join(tmp, "note.md");+ writeFrontmatterAtomic(path, { title: "T" }, "body", { overwrite: true });+ const mtime = ageMtime(path);++ const wrote = writeFrontmatterAtomic(path, { title: "T" }, "body", { overwrite: true });++ expect(wrote).toBe(false);+ expect(statSync(path).mtimeMs).toBe(mtime);+ expect(readFileSync(path, "utf8")).toContain("body");+ });++ test("an overwrite of changed content still rewrites and reports the write", () => {+ const path = join(tmp, "note.md");+ writeFrontmatterAtomic(path, { title: "T" }, "body", { overwrite: true });+ const mtime = ageMtime(path);++ const wrote = writeFrontmatterAtomic(path, { title: "T" }, "body2", { overwrite: true });++ expect(wrote).toBe(true);+ expect(statSync(path).mtimeMs).toBeGreaterThan(mtime);+ expect(readFileSync(path, "utf8")).toContain("body2");+ });++ test("the exclusive create is untouched by the gate and reports its write", () => {+ const path = join(tmp, "fresh.md");+ const wrote = writeFrontmatterAtomic(path, { title: "T" }, "body");+ expect(wrote).toBe(true);+ expect(existsSync(path)).toBe(true);+ // The no-overwrite branch keeps its collision semantics: it refuses,+ // it does not compare and skip.+ expect(() => writeFrontmatterAtomic(path, { title: "T" }, "other")).toThrow();+ });+});diff --git a/tests/mcp/brain-write-batch.test.ts b/tests/mcp/brain-write-batch.test.tsindex a5ef53bc..a32467fe 100644--- a/tests/mcp/brain-write-batch.test.ts+++ b/tests/mcp/brain-write-batch.test.ts@@ -16,13 +16,25 @@*/import { afterEach, beforeEach, describe, expect, test } from "bun:test";-import { existsSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs";+import {+ existsSync,+ mkdirSync,+ readdirSync,+ readFileSync,+ rmSync,+ statSync,+ utimesSync,+ writeFileSync,+} from "node:fs";import { mkdtempSync } from "node:fs";import { join } from "node:path";import { tmpdir } from "node:os";import { bootstrapBrain } from "../../src/core/brain/init.ts";import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";+import { readLogDay } from "../../src/core/brain/log-jsonl.ts";+import { writeImagesDir } from "../../src/core/brain/paths.ts";+import { BRAIN_LOG_EVENT_KIND } from "../../src/core/brain/types.ts";import { WRITE_BATCH_TOOLS } from "../../src/mcp/brain/write-batch-tools.ts";import { NOTES_TOOLS } from "../../src/mcp/brain/notes-tools.ts";import { MAX_BATCH_OPERATIONS } from "../../src/core/brain/write-batch.ts";@@ -354,4 +366,189 @@ describe("brain_write_batch - the lint attached to the receipt", () => {expect(res.applied).toBe(1);expect(PAGE_LINT_KEY in res).toBe(false);});++ test("a byte-identical rewrite that skipped names no page for the lint", async () => {+ // The write that happened carries its finding; the re-apply that+ // skipped carries the receipt but no lint, the same rule the+ // single-write receipts follow - nothing was committed, so the+ // envelope names no page as one the batch wrote.+ seedNote("Notes/SkipLint.md", "v1");+ const body = "see [[pref-ghost]]";+ const first = await runBatch([{ op: "update_note", path: "Notes/SkipLint.md", content: body }]);+ expect(PAGE_LINT_KEY in first).toBe(true);+ const second = await runBatch([+ { op: "update_note", path: "Notes/SkipLint.md", content: body },+ ]);+ expect(second.results[0]).toMatchObject({ kind: "update_note", updated: false });+ expect(PAGE_LINT_KEY in second).toBe(false);+ });+});++/**+ * The absolute-path advisory (p4-silent-failure-hardening, Task 3). A+ * note body embedding an absolute home path used to land with no word+ * about it. The advisory rides each note result like page-lint rides the+ * envelope: non-blocking, and absent entirely when there is nothing to+ * say. Path fragments are composed so this file stays clean under the+ * repo hygiene gate's grammar.+ */+describe("brain_write_batch - the absolute-path advisory on the receipt", () => {+ const HOME = "/home/";+ const leaky = `see ${HOME}alice-dev/notes/x.md for details`;++ test("an update authoring a home path carries the advisory and still applies", async () => {+ seedNote("Notes/Leaky.md", "old", "title: Leaky");+ const res = await runBatch([{ op: "update_note", path: "Notes/Leaky.md", content: leaky }]);+ // Non-blocking: the batch applies and the bytes land.+ expect(res.applied).toBe(1);+ expect(readFileSync(join(vault, "Notes/Leaky.md"), "utf8")).toContain("alice-dev");+ const only = res.results[0]!;+ const advisory = only["path_advisory"] as+ | { total: number; findings: ReadonlyArray<Record<string, unknown>> }+ | undefined;+ expect(advisory).toBeDefined();+ expect(advisory!.total).toBe(1);+ expect(advisory!.findings).toEqual([+ { page: "Notes/Leaky.md", line: 1, detector: "unix-home" },+ ]);+ // The advisory echoes identifiers and integers, never the path itself.+ expect(JSON.stringify(res)).not.toContain(`${HOME}alice-dev`);+ });++ test("a clean update carries no advisory key at all", async () => {+ seedNote("Notes/Clean.md", "old", "title: Clean");+ const res = await runBatch([{ op: "update_note", path: "Notes/Clean.md", content: "new" }]);+ const only = res.results[0]!;+ expect(Object.keys(only)).toEqual(["kind", "path", "updated", "write_id"]);+ expect("path_advisory" in only).toBe(false);+ });++ test("a frontmatter-only update authors no body and carries no advisory", async () => {+ seedNote("Notes/Fm.md", "prose with no leak", "title: Fm");+ const res = await runBatch([+ { op: "update_note", path: "Notes/Fm.md", frontmatter: { title: "Fm2" } },+ ]);+ expect(res.applied).toBe(1);+ expect("path_advisory" in res.results[0]!).toBe(false);+ });++ test("create and append ops carry the advisory; log-line ops do not", async () => {+ seedNote("Notes/AppTarget.md", "base", "title: AppTarget");+ const res = await runBatch([+ { op: "create_note", path: "Notes/Created.md", content: leaky },+ { op: "append_note", path: "Notes/AppTarget.md", content: leaky },+ { op: "append_log_line", text: "batch landed" },+ ]);+ expect(res.applied).toBe(3);+ const [created, appended, logLine] = res.results;+ expect("path_advisory" in created!).toBe(true);+ expect("path_advisory" in appended!).toBe(true);+ expect("path_advisory" in logLine!).toBe(false);+ });++ test("brain_update_note passes the advisory through on its receipt", async () => {+ seedNote("Notes/Single.md", "old", "title: Single");+ const updateTool = NOTES_TOOLS.find((t) => t.name === "brain_update_note")!;+ const leakyBody = `body with ${HOME}alice-dev/leak.md inside`;+ const res = (await updateTool.handler(ctx, {+ path: "Notes/Single.md",+ content: leakyBody,+ })) as Record<string, unknown>;+ expect(res["updated"]).toBe(true);+ const advisory = res["path_advisory"] as {+ total: number;+ findings: ReadonlyArray<Record<string, unknown>>;+ };+ expect(advisory.total).toBe(1);+ expect(advisory.findings[0]).toMatchObject({ page: "Notes/Single.md", line: 1 });+ expect(JSON.stringify(res)).not.toContain(`${HOME}alice-dev`);+ });++ test("brain_append_note carries no advisory key on a clean receipt", async () => {+ seedNote("Notes/App.md", "base", "title: App");+ const appendTool = NOTES_TOOLS.find((t) => t.name === "brain_append_note")!;+ const res = (await appendTool.handler(ctx, {+ path: "Notes/App.md",+ content: "clean addition",+ })) as Record<string, unknown>;+ expect(Object.keys(res)).toEqual(["appended", "path", "write_id"]);+ expect("path_advisory" in res).toBe(false);+ });+});++/** Note-write events recorded in today's log shard. */+function noteWriteEvents(): number {+ const day = readLogDay(vault, new Date().toISOString().slice(0, 10));+ return day.entries.filter((e) => e.eventType === BRAIN_LOG_EVENT_KIND.noteWrite).length;+}++function writeImages(): number {+ const dir = writeImagesDir(vault);+ return existsSync(dir) ? readdirSync(dir).length : 0;+}++/** Stamp a note's mtime one minute into the past, measurably. */+function ageMtime(abs: string): number {+ const past = Date.now() / 1000 - 60;+ utimesSync(abs, past, past);+ return statSync(abs).mtimeMs;+}++/**+ * mtime stability on byte-identical rewrites (p4-silent-failure-hardening,+ * Task 6). The atomic write pipeline lands a fresh inode on every call,+ * so re-applying a byte-identical update used to bump the note's mtime -+ * poisoning recency ranking, validity windows and vault-delta counts -+ * while storing a before-image and recording a note-write event for a+ * write that never happened. The commit now skips unchanged targets, and+ * the receipt says what actually occurred.+ */+describe("brain_write_batch - mtime stability on byte-identical rewrites", () => {+ test("a byte-identical update re-applied writes nothing and says updated: false", async () => {+ seedNote("Notes/Stable.md", "v1");+ const first = await runBatch([{ op: "update_note", path: "Notes/Stable.md", content: "v2" }]);+ expect(first.results[0]).toMatchObject({ kind: "update_note", updated: true });++ const abs = join(vault, "Notes/Stable.md");+ const mtime = ageMtime(abs);+ const events = noteWriteEvents();+ const images = writeImages();++ const second = await runBatch([{ op: "update_note", path: "Notes/Stable.md", content: "v2" }]);+ const only = second.results[0]!;+ expect(only).toMatchObject({ kind: "update_note", path: "Notes/Stable.md", updated: false });+ // The mtime is untouched: no temp file, no rename, no fresh inode.+ expect(statSync(abs).mtimeMs).toBe(mtime);+ // No audit line exists for a write that did not happen - no note-write+ // event, no before-image, and no null write_id that could be mistaken+ // for a lost one.+ expect(noteWriteEvents()).toBe(events);+ expect(writeImages()).toBe(images);+ expect("write_id" in only).toBe(false);+ expect("audit_reason" in only).toBe(false);+ });++ test("a real change still bumps the mtime and reports updated: true", async () => {+ seedNote("Notes/Real.md", "v1");+ const abs = join(vault, "Notes/Real.md");+ const mtime = ageMtime(abs);+ const res = await runBatch([{ op: "update_note", path: "Notes/Real.md", content: "v2" }]);+ const only = res.results[0]!;+ expect(only).toMatchObject({ kind: "update_note", path: "Notes/Real.md", updated: true });+ expect(statSync(abs).mtimeMs).toBeGreaterThan(mtime);+ // A write that happened stays attributed.+ expect("write_id" in only).toBe(true);+ });++ test("an append with a fresh body reports appended: true", async () => {+ seedNote("Notes/Appended.md", "base", "title: Appended");+ const res = await runBatch([{ op: "append_note", path: "Notes/Appended.md", content: "more" }]);+ expect(res.results[0]).toMatchObject({+ kind: "append_note",+ path: "Notes/Appended.md",+ appended: true,+ });+ expect(readFileSync(join(vault, "Notes/Appended.md"), "utf8")).toContain("base");+ expect(readFileSync(join(vault, "Notes/Appended.md"), "utf8")).toContain("more");+ });});diff --git a/tests/mcp/ingest-tool-ignore-warnings.test.ts b/tests/mcp/ingest-tool-ignore-warnings.test.tsindex f341a79d..8cac5773 100644--- a/tests/mcp/ingest-tool-ignore-warnings.test.ts+++ b/tests/mcp/ingest-tool-ignore-warnings.test.ts@@ -13,6 +13,7 @@ import { tmpdir } from "node:os";import { bootstrapBrain } from "../../src/core/brain/init.ts";import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";+import { SKIPPED_PAGE_REASON } from "../../src/core/brain/ingest/extractable-gate.ts";import { INGEST_TOOLS } from "../../src/mcp/brain/ingest-tools.ts";import type { ServerContext } from "../../src/mcp/tool-contract.ts";@@ -83,4 +84,28 @@ describe("brain_ingest_batch_plan ignore_warnings (t_4b2bd8f7 follow-through)",expect("ignore_warnings" in res).toBe(false);expect(res).not.toHaveProperty("ignore_warnings");});++ test("skipped_non_extractable crosses the wire as the typed reason token plus detail (P4)", async () => {+ write("Docs/a.md", "---\nschema_type: paper\n---\nbody\n");+ write("Docs/b.md", "---\nschema_type: memo\n---\nbody\n");+ write(+ "Brain/_brain.yaml",+ "schema_version: 1\nschema:\n page_types:\n - paper\n - memo\n extractable:\n - paper\n",+ );+ const res = (await batchPlan(ctx, { source_dir: "Docs" })) as Record<string, unknown>;++ expect(res["skipped_non_extractable"]).toEqual([+ { path: "Docs/b.md", reason: SKIPPED_PAGE_REASON.notExtractable, detail: "memo" },+ ]);+ expect(res["skip_reason_counts"]).toEqual({ [SKIPPED_PAGE_REASON.notExtractable]: 1 });+ });++ test("a plan with no skips carries no skip_reason_counts key (byte-identical when absent)", async () => {+ write("Docs/a.md", "---\nschema_type: memo\n---\nbody\n");++ const res = (await batchPlan(ctx, { source_dir: "Docs" })) as Record<string, unknown>;++ expect("skip_reason_counts" in res).toBe(false);+ expect("skipped_non_extractable" in res).toBe(false);+ });});diff --git a/uv.lock b/uv.lockindex b319c327..1783e9aa 100644--- a/uv.lock+++ b/uv.lock@@ -4,5 +4,5 @@ requires-python = ">=3.11"[[package]]name = "open-second-brain"-version = "1.61.0"+version = "1.62.0"source = { editable = "." }