Local-first π§ memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly π΄ dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.
TypeScript
feat(brain): memory-signal provenance and lifecycle integrity layer (v1.26.0) (#127)
Sol Aitken962c3e0ac8a9
diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.jsonindex 8bea12b9..b0b5eac1 100644--- a/.claude-plugin/plugin.json+++ b/.claude-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.25.0",+ "version": "1.26.0","description": "Plugin-first second brain package for AI agents and humans.","author": {"name": "Open Second Brain contributors"diff --git a/.codex-plugin/plugin.json b/.codex-plugin/plugin.jsonindex 839d8056..99315d83 100644--- a/.codex-plugin/plugin.json+++ b/.codex-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.25.0",+ "version": "1.26.0","description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.","skills": "./skills","hooks": "./hooks/hooks.json",diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 454c8ed4..beedf744 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -5,6 +5,26 @@ All notable changes to this project will be documented in this file.The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).+## [1.26.0] - 2026-07-08++A memory-signal provenance and lifecycle integrity layer that closes five gaps in one coherent scope. Every new surface is additive and byte-identical when off, and the kernel still calls no LLM.++### Added++- **Content-hash skip-unchanged manifest.** Ingest now classifies source files as new, modified, unchanged, or missing by SHA-256 over file bytes, distinct from identity dedup which hashes the source path. A machine manifest at `<vault>/.open-second-brain/ingest-manifest.json` records post-ingest hashes; a rerun where every source is unchanged skips the LLM extraction pass and rewrites nothing. Atomic write; idempotent.+- **Per-row event-time on batch remember and import.** `WriteSignalInput` gains optional `valid_from` and `recorded_at` that flow into the same frontmatter keys `readBiTemporalSlots()` already parses; session-import prefers `SessionTurn.timestamp` for `created_at` and `recorded_at` and falls back to `now` only when absent or unparseable, so backfilled signals are stamped with the original turn time instead of the import moment. `brain_feedback` and the batch import path forward an optional per-row `event_time`. Absent by default, existing records byte-identical.+- **Batch-plan step for large-folder ingest.** A new `planBatches` helper discovers ingestible files, skips unchanged ones through the content manifest, then splits the remainder into size- and count-bounded batches for the caller (agent or CLI) to dispatch as parallel subagents. Deterministic sort-by-path greedy fill; no parallel execution inside the kernel.+- **Idempotent writes via client idempotency keys.** `writeSignal`, `writePreference`, and `appendApplyEvidence` accept an optional `idempotency_key`; the writer hashes the would-be payload and consults a month-sharded JSONL ledger under `Brain/logs/idempotency/`. Same key and same hash is a deduped no-op; same key with a different hash is an explicit `payload_mismatch` error, never a silent overwrite. `brain_feedback` forwards an optional `idempotency_key`.+- **Dry-run extraction preview.** `extractPreCompactRecords` and the other extraction entry points accept an optional `dryRun` flag that returns the exact records the real path would append without calling `appendContinuityRecord` (no vault mutation, no log event, no dream or retire trigger). Mirrors the existing `opts.dryRun` idiom in `import.ts`; a parity test asserts dry-run candidates equal real-run records for the same input.+- **Bounded verbatim last-N-turns buffer.** A hard-capped ring buffer of the last N turns (default 20) persisted as a new `recent_turn` continuity kind (additive; legacy records read as v1). `appendRecentTurn` evicts oldest beyond N; reads are bounded. An opt-in post-compaction flag appends the buffer so the agent can recover recent verbatim wording after compaction. Default off; its own kind and reader, clearly separated from curated memory.+- **Batch checkpoint save for whole sessions.** A new `brain_session_checkpoint` MCP tool saves a whole session signals, learnings, and summary in one round-trip. Idempotency is session-id plus content-hash, reusing the idempotency ledger; a checkpoint that needs review returns a `{ status: "ok" | "mixed", partial: [...] }` result and never silently drops an item.+- **Caller-settable per-memory expiration date.** `WritePreferenceInput` and `WriteSignalInput` accept an optional ISO `expiration_date`. The read path silently drops anything past its expiration by default; an opt-in `showExpired` re-includes expired records for audit. Orthogonal to dream heuristic retirement: an expired-by-date memory is filtered, not moved to `Brain/retired/` (audit trail preserved).+- **Delete and search by exact source file.** Search accepts an exact `source_file` filter matching the source-ref frontmatter, and a new `delete_by_source` command is dry-run by default, reporting the blast radius (which derived signals, preferences, index entries, and summary pages trace back to the source) without deleting. An explicit `--confirm` deletes the derived entries and index artifacts; original user notes are removed only with `--include-originals`. Every cleanup is recorded as a continuity event.+- **Signed source-diversity grounding score.** A new `computeGroundingScore` projection over the same `ClaimEvent`s that feed `computeTruthState` returns `{ score, confidence, band, supportingSources, contradictingSources }`. Confirming vs contradicting evidence is counted across independent sources (distinct `source` and `agent`), not raw mention count, so N mentions in one document count far below N mentions across N independent sources. Signed: plus for confirming, minus for contradicting; confidence captures how much evidence backs the direction. Pure projection, order-insensitive, never mutates the append-only truth ledger; replaces nothing (the CONTESTED flag stays, the score quantifies it).+- **Note-position contradiction detection.** `detectNoteContradictions` pairs same-subject permanent notes asserting opposite stances, deriving position-sign from note prose rather than `evidenced_by` signals. Reuses the existing language-agnostic token-overlap machinery; emits `ask_user` findings quoting the relevant span from each note and never auto-resolves.+- **Declared-thesis register with new-note monitor.** A new `Brain/theses/` frontmatter kind records operator positions (statement, supporting and counter evidence, last-updated, falsification criterion). A monitor evaluates each newly-ingested note against active theses, flagging support or contradiction, reuses the obligations cadence machinery for a staleness check and a thesis-graveyard pass, and alerts when incoming evidence matches a thesis documented falsification scenario.+- **Session-bracketing memory wrapper for Aider.** A new `scripts/o2b-aider` wrapper (also `o2b aider wrap`) brackets an Aider session: it re-renders the static context sidecar at start, execs the Aider binary with the injected `read:` context, and at session end runs the write-back half (capture and persist the session into the Brain), mirroring Hermes `prefetch`, `sync_turn`, and `on_session_end`. The install-time sidecar adapter stays as the fallback for users who do not run through the wrapper.+## [1.25.0] - 2026-07-07An economics and observability layer for the context pack. Five newdiff --git a/README.md b/README.mdindex 0a368c60..80bec947 100644--- a/README.md+++ b/README.md@@ -8,7 +8,7 @@ Open Second Brain plugs into [Hermes Agent](https://github.com/NousResearch/herm## What is new-Open Second Brain 1.25.0 adds an optional economics and observability layer for the context pack, each surface off by default with byte-identical output when unconfigured. A value-per-token density ranking comparator breaks within-tier ties in the context pack after session focus and before recency, so a dense high-signal page can outrank a sparse recent one inside the same tier (tier stays the coarse gate). A durable token-impact ledger records the tokenizer-exact prompt-token delta every context pack saves (baseline minus packed), kept strictly separate from a modeled inference-avoidance estimate that outcomes can later calibrate, so a measured count is never averaged in with a model. An agent-operable context-pack outcome loop closes the quality feedback path: the agent posts first-pass, repair, and retry counters per carried sample, and three token signals stay separate rather than merged. Route-level MCP tool latency metrics show which endpoint blocks an agent turn by route rather than by aggregate benchmark, and an active-memory budget-pressure watermark surfaces proactive pressure before the pack is forced to trim. Every ledger is payload-safe (counts and opaque ids only, never prompts or recalled text), reads ignore the write gate, and the kernel still calls no LLM.+Open Second Brain 1.26.0 closes five memory-integrity gaps in one coherent scope, each surface additive and byte-identical when off. Ingest now classifies sources by SHA-256 content hash and skips the extraction pass for unchanged files, and batch remember and import stamp each signal with the original turn event time instead of the import moment. Writes are idempotent under retry through client idempotency keys, where the same key and hash is a no-op and the same key with a different hash is an explicit error; extraction offers a faithful dry-run preview, and a bounded verbatim last-N-turns buffer survives compaction. Preferences and signals accept a caller-settable expiration date, a dry-run-by-default delete-by-source command reports blast radius before removing derived entries, and a whole-session checkpoint saves signals, learnings, and summary in one round-trip. On the truth side, a signed source-diversity grounding score quantifies how much independent support a slot carries, note-position contradiction detection pairs same-subject notes with opposite stances, and a declared-thesis register evaluates incoming notes against standing positions. A new Aider wrapper brackets a session end to end, mirroring the Hermes lifecycle. The kernel still calls no LLM.## Whydiff --git a/docs/brainstorm/memory-signal-provenance-lifecycle/cli-output/claude.md b/docs/brainstorm/memory-signal-provenance-lifecycle/cli-output/claude.mdnew file mode 100644index 00000000..5166d509--- /dev/null+++ b/docs/brainstorm/memory-signal-provenance-lifecycle/cli-output/claude.md@@ -0,0 +1,29 @@+### Variant 1 β Shared-substrate first+Approach: Land the three cross-cutting primitives as standalone modules before any consumer: a `manifest` module (content-hash cache + classify), an `idempotency` keyβcontent-hash ledger under `Brain/`, and a single `dryRun` gate that wraps every continuity-append surface. All 13 cards then reduce to thin verticals that call into settled infrastructure.+Trade-offs:+- Maximal reuse and one canonical answer for each concern (one hashing path, one dedup notion, one preview switch) β no divergent re-implementations.+- Couples the p4 anchors (A1) and the shared ledger to the p2 capstones (A3, C4) through APIs designed before their only consumers exist, so the primitive shape is guessed, not validated; a wrong guess forces a rewrite that ripples across cards.+- A global dry-run wrapper contradicts the existing per-surface `opts.dryRun` idiom in `import.ts` and risks a leaky abstraction that must special-case dream/retire triggers anyway.+Complexity: large+Risk: high++### Variant 2 β Self-contained verticals+Approach: Treat every card as an independent vertical driven by priority then dependency, with no shared modules beyond the existing `sourceIdentityHash`/SHA-256 primitive. A1 grows its manifest inside ingest; C1 grows its ledger inside feedback-tools; C4 and A3 reach directly into whatever their parents left behind rather than a shared API.+Trade-offs:+- Lowest coupling and simplest per-card review; each commit is atomic and byte-identical-when-off is trivial to verify in isolation.+- The two hard composition edges (A1βA3, C1βC4) still force reuse, so "no shared module" is a fiction β A3 and C4 either duplicate parent logic or depend on internal shapes, producing drift (two idempotency notions, two hash-manifest readers) that the next release must reconcile.+- No structural place for the dry-run concern shared by C2 and any future preview, so C2's short-circuit logic is bespoke and hard to prove faithful to the real extraction path.+Complexity: medium+Risk: medium++### Variant 3 β Topological hybrid: shared seams only at composition edges+Approach: Extract shared infrastructure only where a hard dependency already mandates it β A1 ships the manifest as a real module because A3 consumes it, C1 ships the keyβcontent-hash ledger as a real module because C4 consumes it β and keep everything else a self-contained vertical, mirroring `import.ts`'s existing `opts.dryRun` as a per-surface parameter rather than a global wrapper. Drive in a topological order that serializes every shared-file collision: A1 β C1 β A2 β D2 β D1 β C2 β C3 β C5 β C6 β A3 β C4 β D3.+Trade-offs:+- Each shared module is designed against a concrete consumer (A3, C4) landed shortly after, so the API is validated by use, not guessed; the p4 anchors never wait on the p2 capstones.+- File collisions resolve by ordering, not merging: A1 before A3 on `ingest.ts`; A2 before C2 on `import.ts` (event-time settles the emit path, then dry-run gates it); D2 before D3 on `contradiction.ts`; D1 owns `truth/{fold,conflicts}.ts` alone as a pure projection over the append-only ledger.+- Two genuine modules (manifest, ledger) plus disciplined ordering is more upfront design than pure verticals, and the capstones (A3, C4, D3) landing last means the longest-pole cards gate branch completion.+Complexity: medium+Risk: low++### Recommended: Variant 3+The release's own constraints already dictate the answer: two hard composition edges (A1βA3, C1βC4) mean the manifest and idempotency ledger *must* be reusable, but nothing else composes, so building the full shared substrate of Variant 1 pays coupling and speculative-API cost for reuse the cards do not ask for. Variant 2's refusal to extract even those two modules is a false economy β the edges force reuse regardless, and doing it implicitly produces exactly the duplicate-hash / dual-idempotency drift this release exists to eliminate. Variant 3 extracts shared code precisely at the mandated seams and validates each module against a consumer landed immediately after, while a single topological drive order serializes all five shared-file collisions and honors all three parentβchild edges without coupling the p4 anchors to the p2 capstones. Keeping dry-run as a per-surface parameter that mirrors the existing `import.ts` idiom preserves the byte-identical-when-off invariant and keeps preview faithful to real extraction, which a global wrapper cannot guarantee.diff --git a/docs/brainstorm/memory-signal-provenance-lifecycle/cli-output/prompt.md b/docs/brainstorm/memory-signal-provenance-lifecycle/cli-output/prompt.mdnew file mode 100644index 00000000..24a860fa--- /dev/null+++ b/docs/brainstorm/memory-signal-provenance-lifecycle/cli-output/prompt.md@@ -0,0 +1,295 @@+# Consultant prompt β memory-signal provenance and lifecycle integrity++You are a senior backend architect. Produce architectural variants for a single+shippable release of **Open Second Brain (o2b)**. Read the context below+carefully; every design MUST respect o2b's hard invariants.++## Output contract (MANDATORY)++Output EXACTLY three variant sections then ONE recommendation. No code. Nothing+outside these sections. Each variant:++```+### Variant N β <short name>+Approach: <2-3 sentences>+Trade-offs:+- <bullet>+- <bullet>+Complexity: small | medium | large+Risk: low | medium | high+```++Then exactly:++```+### Recommended: Variant N+<rationale, 3-5 sentences>+```++Do not add an introduction, summary, or closing.++## Project++- Name: `open-second-brain` (o2b), v1.25.0.+- Language/runtime: TypeScript, ESM, runs on Bun + Node. `type: module`.+- What it is: an Obsidian-native memory layer for AI agents. Plain Markdown in a+ vault (`Brain/` directory). Deterministic CLI + MCP tools. **The kernel calls+ no LLM** β this is a core, load-bearing invariant. All scoring, detection, and+ lifecycle logic is deterministic (counting, hashing, token overlap, date math).+- Conventions (hard): deterministic, byte-identical-when-off for any new field+ or switch; pure functions do the work, IO shells stay thin; everything is+ language-agnostic (no English wordlists; negation/similarity are structural+ token-overlap); additive optional fields absent by default; append-only claim+ ledger is never mutated β derived projections fold over it; privacy-preserving+ (no raw prompts, recalled text, or secrets); persistence via the continuity+ store (append-only, month-sharded JSONL under `<vault>/Brain/logs/continuity/`,+ reloaded on startup) or as Markdown frontmatter; config flags are explicit+ opt-in. Atomic writes via `writeFrontmatterAtomic` / `atomicWriteFileSync`.+ A no-op rerun must not rewrite files (keeps Syncthing peers quiet).+- SHA-256 is the hashing primitive (`createHash("sha256")`); the provenance+ module already owns `sourceIdentityHash(parts)` (one source path β one digest).+- Multi-runtime: Hermes, Claude Code, Codex, opencode, Aider, etc. all read/write+ the same Brain. Writes can be concurrent or retried across runtimes.++## The release problem++This release hardens **provenance and lifecycle integrity** across four+orthogonal axes the codebase is currently weak on, cohered by a shared theme:+provenance (where a signal came from and when), lifecycle (how it expires and is+reconciled), and integrity (deterministic dedup, safe preview, and surgical+cleanup). The 13 cards below are dependency-closed leaves that ship together on+one branch, driven one at a time. Several share files and several compose (the+content-hash manifest feeds the batch-plan planner; the idempotency key feeds the+batch checkpoint). The binding architectural question is: **how much shared+infrastructure to introduce up front (a manifest module, an idempotency-key+ledger, a dry-run wrapper) vs. how much to keep each card a self-contained+vertical** β and in what order to drive the cards so the shared-file collisions+and the hard composition edges serialize cleanly without coupling the p4 anchors+to the p2 capstones.++### Axis A β provenance: source identity & event-time (4 cards)++#### Card A1 β `t_c5184fd8` (p4): content-hash skip-unchanged manifest+OSB already computes SHA-256 for source *identity* (`sourceIdentityHash` in+`src/core/brain/provenance/provenance.ts:91`) but has no content-hash skip-+unchanged manifest, so re-ingest is driven by mtimes/index revision, not byte+content β a `git checkout`/NFS touch forces re-ingest. Requirement: a content-+hash cache so re-ingest is driven by content changes: `cache-check` classifies+paths new/modified/unchanged/missing vs a `.manifest.json`; `cache-update`+records post-ingest hashes; `cache-hash` SHA-256s a file or dir tree. Reuse the+existing sha256 primitive; the new artifact is the per-source content-hash+manifest + classify step, distinct from identity dedup. Sequence A1 BEFORE the+batch-plan task (A3) so parallel ingest can skip unchanged sources. Consumer:+`src/core/brain/ingest/ingest.ts`. Change-watching is keyed on index revision+today (`src/core/brain/ingest/`).++#### Card A2 β `t_7526e8d3` (p4): per-row event-time on the batch remember/import API+OSB has the bi-temporal *concept* on the read-side/types+(`src/core/brain/signal.ts` `readBiTemporalSlots()` reads optional+`valid_from`/`valid_until`/`recorded_at`; `src/core/brain/reconcile-domains.ts`+uses `recorded_at if present else created_at`). But the WRITE side is weaker:+`WriteSignalInput` (`src/core/brain/signal.ts`) exposes only `created_at`/`date`+β no `valid_from`/`valid_until`/`recorded_at`, so no caller can set the original+event-time. Session-import discards the source turn time:+`src/core/brain/sessions/import.ts` stamps `const now = opts.now ?? new Date()`+and `created_at: isoSecond(now)` even though `SessionTurn.timestamp` already+carries the original turn's ISO-8601. Requirement: (1) add optional+`valid_from`/`recorded_at` (and/or `event_time`) fields to `WriteSignalInput`+that flow into frontmatter the read-side already parses; (2) have `import.ts`'s+`emit` prefer the turn's `SessionTurn.timestamp` for `created_at`/event-time,+falling back to `now` only when absent. Keep fields additive/optional so existing+signals stay byte-identical. Guard against future-dated or unparseable turn+timestamps during backfill.++### Axis B β ingest parallelism (2 cards, parentβchild)++#### Card A3 β `t_9eeb8ca2` (p2): batch-plan step to split large-folder ingest+A `batch-plan <vault> <source-dir>` step that discovers ingestible files, skips+unchanged via the content-hash cache (A1), and splits the remainder into+size+count-bounded batches that ingest dispatches as parallel subagents for large+directories. Today ingest handles sources one at a time (`src/core/brain/ingest/+ingest.ts`, per-source, agent-driven); the only bounded-batch primitive is for+context, not ingest (`runPinnedContextBatch` in `src/mcp/brain/context-+tools.ts`). **Depends on A1** (the content-hash manifest). Sequence A1 before A3.++### Axis C β write idempotency & lifecycle (6 cards)++#### Card C1 β `t_213f356b` (p3): idempotent feedback/signal writes via client idempotency keys+OSB is multi-runtime and feedback writes are file-appending Markdown; a retried+or double-delivered tool call can append a duplicate signal, and slug collisions+from genuinely different content are silently merged. Today dedupe is implicit+via `deriveSlug` (`src/mcp/brain/feedback-tools.ts`), not an explicit client key.+Requirement: an optional idempotency-key parameter on the write tools+(`brain_feedback`, `writeSignal`/`writePreference`/`appendApplyEvidence`) plus a+lightweight seen-key ledger (keyed by key β content hash) under `Brain/`; reject+reuse of the same key with a different payload (explicit error, not silent+overwrite). Keep the kernel deterministic (no LLM). This C1 ledger is the natural+dedup substrate for the batch checkpoint (C4) and the event-time backfill (A2).++#### Card C2 β `t_2c6cf3e2` (p3): dry-run extraction preview+OSB's extraction surfaces (`pre_compact_extract`, `derive_fact`, ingest) are+write-committing and fail-fast by design β there is no safe way to preview what+extraction will capture before it mutates `Brain/`. Requirement: a dry-run/preview+path that short-circuits ALL continuity-append writes (vault mutation, log events,+dream/retire triggers) while reusing the EXACT extraction logic, so preview output+faithfully predicts real extraction. Enables prompt/threshold tuning, eval+harnesses, operator review of low-confidence captures without polluting the vault,+and gives smoke-tests a non-destructive assertion target. Key files:+`src/core/brain/pre-compact-extract.ts` (calls `appendContinuityRecord`),+`src/core/brain/sessions/import.ts` (already has an `opts.dryRun` pattern to+mirror).++#### Card C3 β `t_92317f91` (p3): bounded verbatim last-N-turns buffer surviving compaction+After a compaction OSB only retains summarized/extracted forms (active.md,+pre-compact extract, session summary); the exact recent wording is lost.+Requirement: a small, bounded verbatim buffer of the last N conversation turns,+auto-captured each turn and preserved across host compaction, readable on demand+(and optionally re-surfaced right after a compaction). Must stay strictly bounded+and clearly separated from curated memory so it does not become verbatim hoarding+β ephemeral short-term scaffolding with a hard cap on N; surfacing post-compaction+is optional/opt-in. Searches for turn-history / conversation-history / last-n-turns+in `src/` returned ZERO matches today.++#### Card C4 β `t_1a3a9eba` (p3): batch checkpoint save for whole sessions+OSB captures session summaries and individual signals, but session-close ingestion+can involve several writes with partial-write states. Requirement: a batch+checkpoint tool that saves a whole session's extracted memories, decisions,+learnings, and optional diary/summary in ONE idempotent MCP round-trip. Define+idempotency around session id + content hash. If some writes need review, the+checkpoint reports a mixed result rather than silently dropping items. Session+summary write/get/list exists at `src/mcp/brain/synthesis-tools.ts`; learning+staging at `src/mcp/brain/feedback-tools.ts`. **Composes C1's idempotency-key+ledger** as the dedup substrate.++#### Card C5 β `t_a82b674e` (p3): caller-settable per-memory expiration date+OSB callers cannot time-box a memory: an agent that learns a deadline-bound rule+("use the staging endpoint until 2026-07-15") has no way to say when it should+stop applying β only dream's stale/rebutted heuristics can eventually retire it.+Requirement: an optional ISO `expiration_date` on+`WritePreferenceInput`/signal writes, honored by the read path and by dream;+default search/list silently drop anything past its date unless an opt-in+`showExpired` flag is set. Orthogonal to dream's heuristic retirement (an+expired-by-date memory need not move to `Brain/retired/` β it can simply be+filtered, preserving audit trail). `writePreference` is in+`src/core/brain/preference.ts`; retirement is dream-driven (`planAutoRetires` in+`src/core/brain/dream.ts`, reasons in `BRAIN_RETIRED_REASON`).++#### Card C6 β `t_edde2198` (p3): delete and search by exact source file+When benchmark files, logs, or accidental imports pollute a Brain, operators need+to find and surgically remove everything derived from a source without re-mining+the whole vault or manually chasing index artifacts. Requirement: exact+`source_file` filtering on search + a dry-run-by-default deletion/cleanup command+that removes all derived entries for a contaminated source file, including index+entries. Dry run MUST report blast radius first. Writes auditable; must NOT delete+original user notes unless explicitly requested.++### Axis D β intellectual-honesty detection (3 cards, parentβchild)++#### Card D1 β `t_4678a91a` (p4): signed source-diversity grounding score+The entity-truth ledger marks a slot CONTESTED (binary,+`resolution: ask_user`, never auto-resolved) when independent sources disagree+within a window (`src/core/brain/truth/conflicts.ts` `withinWindow`/+`CONFLICT_WINDOW_DAYS`). But CONTESTED is binary and confidence is unsigned bands+(high/medium/low, `src/core/brain/page-meta/confidence.ts`). It cannot say WHICH+side of a contest carries more independent support, nor distinguish "agreed by 1+source" from "agreed by 10". Requirement: a signed grounding score on a+β1.0..+1.0 scale, computed from the balance of confirming vs contradicting+evidence across INDEPENDENT sources (not raw mention count), weighted by+relationship strength, plus a separate confidence/sufficiency dimension. Labels+the band (Strongly supported β Mixed β Contested β Contradicted) and weights N+mentions in one document far below N mentions across N independent sources.+Deterministic (counting + weighting, no LLM). Derives from the append-only claim+ledger (`ClaimEvent` carries independent `source` + `agent` provenance); do NOT+mutate history β compute as a projection alongside the existing fold+(`src/core/brain/truth/fold.ts` `computeTruthState`). Source-diversity weighting+(independent sources > repeated mentions) is the part o2b lacks entirely.++#### Card D2 β `t_11e3db8b` (p3): note-position contradiction detection over prose+o2b detects contradictions only across confirmed preferences+(`src/core/brain/health/contradiction.ts` `detectContradictions`) and across fact+claim-slots (`computeTruthStateWithConflicts`). Neither surfaces the operator's+own contradictory *positions* held across note prose. Requirement: extend+contradiction detection to pair same-subject permanent notes that assert opposite+stances, quote the relevant span from each, and surface them as a reviewable+clarification prompt (ask_user), never auto-resolving. Reuse the language-agnostic+token-overlap + sign machinery (`similarity.ts`) but derive position-sign from+note prose rather than evidence signals; touch `src/core/brain/health/+contradiction.ts`. **Parent of D3.**++#### Card D3 β `t_74d29363` (p2): declared-thesis register with new-note monitor+o2b tracks preference contradictions and fact conflicts but has NO register of+operator-declared theses, and nothing evaluates each new note against standing+positions. Requirement: a declared-thesis register (statement, supporting+evidence, counter-evidence, last-updated) plus a monitor that flags newly-+ingested notes that support or contradict an active thesis β distinct from+D2 (this is incoming-note-vs-declared-position monitoring). Includes a staleness+check (reuse the cadence/obligation machinery `obligations.ts` for "not updated+in N days"), thesis-graveyard pass (flag theses with no supporting evidence in N+days for formal closing), and a per-thesis falsification field ("what would make+me wrong") monitored so the register alerts when incoming evidence matches the+documented failure scenario. Suppress mere added-complexity per the article.+**Child of D2** (both touch `src/core/brain/health/contradiction.ts`; D3 is the+specific incoming-note monitor on top of D2's note-position base).++### Axis E β runtime lifecycle (1 card)++#### Card E1 β `t_1c894c19` (p4): session-bracketing memory wrapper for Aider+OSB's Aider adapter (`src/core/install/adapters/aider.ts`) is a deliberate static+sidecar because "Aider has no native MCP client" β it covers the load half via a+regenerated one-time snapshot (`o2b install --target aider --apply`) but has NO+per-session capture/write-back loop for Aider. Requirement: scope a wrapper-based+bracketing mode that reuses the Hermes `prefetch`/`sync_turn` lifecycle shape+(load relevant memory at session start, persist the session back at session end),+implemented as a CLI wrapper process around Aider (since MCP/hooks aren't+available β the mechanically hard part Hindsight solved). Keep the static sidecar+as the fallback for users who don't run through the wrapper. The Hermes provider+shape to mirror is `plugins/hermes/provider.py` `prefetch()`/`sync_turn()`/+`on_session_end()`.++## Cross-cutting constraints (apply to every variant)++- **No LLM in any kernel logic.** Every score, detector, planner, and lifecycle+ rule is deterministic (hashing, token overlap, date math, counting).+- **Byte-identical-when-off / absent.** Any new optional field or switch must be+ absent by default so existing files and outputs stay byte-identical.+ Additive-only on `WriteSignalInput`/`WritePreferenceInput`.+- **Append-only ledger is never mutated.** D1's grounding score is a derived+ projection; truth history is never rewritten.+- **Shared-file collisions must be serialized by drive order.** Known collisions:+ - `src/core/brain/health/contradiction.ts` β D2 (note-position base) + D3+ (thesis monitor).+ - `src/core/brain/signal.ts` `WriteSignalInput` β A2 (event-time fields).+ - `src/core/brain/sessions/import.ts` β A2 (emit turn time) + C2 (dry-run).+ - `src/core/brain/truth/{fold,conflicts}.ts` β D1 (grounding projection).+ - `src/core/brain/ingest/ingest.ts` β A1 (manifest) + A3 (batch-plan).+- **Hard composition edges:**+ - A1 β A3 (batch-plan consumes the content-hash manifest).+ - C1 β C4 (checkpoint reuses the idempotency-key ledger).+ - D2 β D3 (thesis monitor builds on the note-position detector).+- **No silent overwrite or silent drop.** Idempotency-key mismatch is an explicit+ error; checkpoint mixed-result is reported; C6 dry-run reports blast radius.+- **Cards are driven ONE AT A TIME on the shared branch `feat/memory-signal-+ provenance-lifecycle`**; each worker builds on prior commits and must not+ duplicate sibling work. The variant's drive order must resolve every shared-file+ collision and honor every hard dependency.++## Recent git history (context for conventions)++```+fe2c0be2 feat(brain): context-pack economics and observability (1.25.0) (#126)+1cde572f fix(brain): harden reindex swap, self-heal, hot paths, and continuity contracts (1.24.0) (#125)+998e437f fix(windows): resolve 3 compatibility issues on Windows (#123)+ce5d5655 fix(windows): normalize vault path and resolve o2b command (#122)+67bcb71c refactor: DRY and decomposition (Phases 0-2) (#121)+b9bbcb16 feat(brain): semantic entity dedup and cross-encoder rerank (1.23.0) (#120)+b8d709ee fix: keep full MCP status output and normalize codegraph paths (#116)+a98bed1d feat(brain): retrieval precision and quality loop (v1.22.0) (#118)+42816058 feat(brain): integrity & safety hardening suite (1.21.0) (#115)+313d061e feat: configurable skills_dir + trigger-keyword auto-attach scoring (#114)+a3ea3151 fix: v1.19.1 - cross-vault cards, event-trace exit codes, registry-guard hygiene (#113)+bb5f3201 feat(brain): session-boundary capture durability and post-compaction pinned-anchor survival audit (v1.19.0) (#112)+33b4fba5 feat(brain): recall precision, coverage, and provenance hardening (v1.18.0) (#110)+da2e3ccd feat(brain): memory subsystem alignment - honest pinned budgets, atomic batch writes, on_memory_write host bridge (v1.16.0) (#107)+```++Now produce the three variants and the recommendation.diff --git a/docs/brainstorm/memory-signal-provenance-lifecycle/design.md b/docs/brainstorm/memory-signal-provenance-lifecycle/design.mdnew file mode 100644index 00000000..9b62d6aa--- /dev/null+++ b/docs/brainstorm/memory-signal-provenance-lifecycle/design.md@@ -0,0 +1,388 @@+# Design β memory-signal provenance and lifecycle integrity++Branch: `feat/memory-signal-provenance-lifecycle`+Slug: `memory-signal-provenance-lifecycle`++Cards (13), grouped by axis:++- **Axis A β provenance / event-time**+ - A1 `t_c5184fd8` (p4) β content-hash skip-unchanged manifest+ - A2 `t_7526e8d3` (p4) β per-row event-time on batch remember/import+- **Axis B β ingest parallelism**+ - A3 `t_9eeb8ca2` (p2) β batch-plan step (child of A1)+- **Axis C β write idempotency & lifecycle**+ - C1 `t_213f356b` (p3) β idempotent writes via client idempotency keys+ - C2 `t_2c6cf3e2` (p3) β dry-run extraction preview+ - C3 `t_92317f91` (p3) β bounded verbatim last-N-turns buffer+ - C4 `t_1a3a9eba` (p3) β batch checkpoint save for whole sessions+ - C5 `t_a82b674e` (p3) β caller-settable per-memory expiration date+ - C6 `t_edde2198` (p3) β delete and search by exact source file+- **Axis D β intellectual-honesty detection**+ - D1 `t_4678a91a` (p4) β signed source-diversity grounding score+ - D2 `t_11e3db8b` (p3) β note-position contradiction detection+ - D3 `t_74d29363` (p2) β declared-thesis register monitor (child of D2)+- **Axis E β runtime lifecycle**+ - E1 `t_1c894c19` (p4) β session-bracketing memory wrapper for Aider++## Problem++o2b's memory layer is strong on deterministic curation and recall precision but+weak on five integrity gaps that this release closes in one coherent scope:++1. **Re-ingest is mtime-driven, not content-driven.** `sourceIdentityHash`+ computes SHA-256 for source *identity* (one source path β one summary page)+ but there is no timestamp-independent "this source is byte-identical, skip the+ LLM pass" signal, so a `git checkout` / NFS touch / checkout-based restore+ forces full re-ingest of every source. (A1)+2. **Backfilled signals are mis-stamped with the import moment.** OSB has the+ bi-temporal *concept* on the read-side (`readBiTemporalSlots()` parses+ `valid_from`/`valid_until`/`recorded_at`; `reconcile-domains.ts` prefers+ `recorded_at if present else created_at`) but the WRITE side exposes only+ `created_at`/`date` on `WriteSignalInput`, and session-import stamps the+ import wall-clock even though `SessionTurn.timestamp` already carries the+ original turn's ISO-8601. Recency reasoning and bi-temporal reconciliation+ are therefore fed corrupt timestamps on backfill. (A2)+3. **Writes are not idempotent under retry, and dedupe is implicit.** Multi-+ runtime feedback writes are file-appending Markdown (`appendBrainNote`/+ `appendLogEvent`), so a retried/double-delivered tool call appends a+ duplicate; and the slug allocator (`allocateSlug` in `paths.ts`) disambiguates+ collisions with a `-2`/`-3` suffix rather than recognizing the same logical+ content, so a retried `writeSignal`/`writePreference` creates a distinct+ duplicate file rather than a no-op. There is no explicit client idempotency+ key and no payload-mismatch rejection. (C1)+4. **Extraction is write-committing with no safe preview, and there is no+ surgical cleanup of a contaminated source.** `pre_compact_extract` / ingest+ are fail-fast and mutate `Brain/` immediately β there is no faithful dry-run+ path (C2). Conversely there is no `delete_by_source` command to remove every+ derived entry for a polluting source file including index artifacts (C6).+ Compaction also loses the exact recent wording (C3). And there is no caller+ time-box on a memory (C5) nor a whole-session atomic checkpoint (C4).+5. **Disagreement and self-contradiction are preserved but unquantified and+ prose-blind.** The truth ledger marks a slot CONTESTED (binary,+ `resolution: ask_user`) when independent sources disagree within a window,+ but cannot say WHICH side carries more independent support, nor distinguish+ "agreed by 1 source" from "agreed by 10" β no signed, source-diversity-+ weighted grounding score (D1). Detection stops at confirmed preferences and+ fact claim-slots; the operator's own contradictory *positions* held across+ note prose are never surfaced (D2), and there is no declared-thesis register+ to evaluate incoming notes against standing positions (D3). Finally, the+ Aider runtime has no session write-back loop at all (E1).++This release closes all five gaps with deterministic, additive, byte-identical-+when-off capabilities that REUSE existing hashing, frontmatter, claim-ledger, and+lifecycle machinery.++## Scope++All 13 cards ship in this release, driven one at a time on the shared branch.+Four p4 anchors lead: A1, A2, D1, E1. The p3 cards cohere the lifecycle and+write-integrity layer; the two p2 capstones (A3 batch-plan, D3 thesis monitor)+land last as they depend on their parents.++## Out of scope++- **Any LLM inside kernel logic.** Every score, detector, planner, and lifecycle+ rule is deterministic (hashing, token overlap, date math, counting). An agent+ drives extraction/ingest; o2b owns the deterministic half.+- **Mutating the append-only claim ledger.** D1's grounding score is a derived+ projection over `ClaimEvent`s; truth history is never rewritten β exactly the+ discipline `conflicts.ts` already follows (`computeTruthStateWithConflicts`+ returns a projection, never mutates events).+- **Auto-resolution of any detected contradiction.** D1, D2, D3 all emit+ `ask_user`-style findings; the operator resolves, never the kernel.+- **Auto-deletion of original user notes.** C6 deletes only *derived* entries;+ original notes are removed only when an explicit flag requests it. Dry-run+ reports blast radius first.+- **A global dry-run wrapper.** C2 mirrors the existing per-surface+ `opts.dryRun` idiom already in `import.ts`; it does not introduce a+ process-wide preview switch that would risk a leaky abstraction across+ dream/retire triggers.+- **Migrating existing Markdown.** Every new frontmatter field is additive and+ optional; existing signals/preferences stay byte-identical (absent by default).+- **A new daemon or hosted service.** E1's Aider bracketing is a CLI wrapper+ process around the Aider binary (MCP/hooks are unavailable for Aider); C3's+ verbatim buffer is a bounded continuity-store artifact, not a running server.+- **The onboarding card `t_84500f39`** (off-theme) and the meta-blocked+ `t_2c8448bb` (blocked by umbrella `t_9935bd26`) β explicitly excluded from+ this release's scope selection.++## Chosen approach++**Variant 3 β topological hybrid: shared seams only at composition edges.**++Extract shared infrastructure ONLY where a hard dependency mandates it β A1 ships+the content-hash manifest as a real module because A3 consumes it, C1 ships the+keyβcontent-hash idempotency ledger as a real module because C4 consumes it β and+keep every other card a self-contained vertical. Drive in a topological order+that serializes every shared-file collision and honors all three parentβchild+edges (A1βA3, C1βC4, D2βD3). Dry-run (C2) stays a per-surface parameter mirroring+the existing `import.ts` `opts.dryRun` idiom, never a global wrapper.++This captures exactly the reuse that is real and validated by an immediate+consumer, without either front-loading a speculative full substrate (Variant 1 β+which couples the p4 anchors to p2 capstones through APIs designed before their+only consumers exist) or refusing to extract even the two mandated modules+(Variant 2 β a false economy, since the hard edges force reuse regardless and+implicit reuse produces exactly the dual-hash / dual-idempotency drift this+release exists to eliminate).++### Drive order: A1 β C1 β A2 β D2 β D1 β E1 β C2 β C3 β C5 β C6 β A3 β C4 β D3++Resolves all five shared-file collisions and all three composition edges:++- **A1 β A3** (`ingest.ts` + the manifest module): A1 lands the manifest; A3+ consumes `classifyPaths`/`cache-hash`. Hard dependency, serialized.+- **A2 β C2** (`import.ts`): A2 settles the `emit` path's event-time (prefers+ `SessionTurn.timestamp`), then C2 gates the same `emit` path with a dry-run+ short-circuit. Disjoint concerns in the same file, sequenced.+- **C1 β C4** (idempotency ledger): C1 lands the ledger module; C4's checkpoint+ reuses the keyβcontent-hash substrate for session-id+hash idempotency. Hard+ dependency.+- **D2 β D3** (`contradiction.ts`): D2 adds note-position detection; D3 builds+ the thesis register + incoming-note monitor on top. Hard dependency.+- **D1** owns `truth/{fold,conflicts}.ts` alone β it adds a pure projection+ (`computeGroundingScore`) alongside `computeTruthState`, never mutating the+ fold. No other card touches the truth family.+- **E1** is fully isolated (install adapter + a new wrapper script); no shared+ file with any other card. Driven in the p4 group.++The four p4 anchors (A1, A2, D1, E1) all land before the p3 cohering layer, so+the quick wins never wait on the capstones. The two p2 capstones (A3, D3) land+last, as they depend on their parents and gate branch completion.++## Design decisions++### Shared module 1 β content-hash manifest (lands in A1, consumed by A3)++- New module `src/core/brain/ingest/content-manifest.ts`:+ - `hashFile(absPath)` / `hashTree(dir)` β SHA-256 hex (reuses the same+ `createHash("sha256")` primitive as `sourceIdentityHash`, but over file+ *bytes*, not source-path identity strings). A content hash is intentionally a+ different artifact from the identity hash: identity dedups "is this the same+ logical source"; the content manifest answers "are its bytes unchanged since+ the last ingest".+ - `classifyPaths(vault, paths, manifest)` β `{ new, modified, unchanged,+ missing }` by comparing live hashes against the manifest.+ - `updateManifest(vault, paths)` records post-ingest hashes.+ - Manifest lives at `<vault>/.open-second-brain/ingest-manifest.json` (a+ machine artifact, NOT under `Brain/` β it is not curated memory; consistent+ with the existing `<vault>/.open-second-brain/` install-artifact location the+ Aider sidecar uses). Atomic write; a no-op rerun (all unchanged) rewrites+ nothing.+- `ingest.ts` calls `classifyPaths` before the extraction pass and skips the LLM+ for `unchanged` sources (still idempotently rewrites the summary page only if+ its bytes would change).++### Shared module 2 β idempotency-key ledger (lands in C1, consumed by C4)++- New module `src/core/brain/idempotency-ledger.ts`:+ - `rememberKey(vault, { key, contentHash })` β `inserted | duplicate_match |+ payload_mismatch`. Keyed by client-supplied key β content hash, stored under+ `<vault>/Brain/logs/idempotency/` (month-sharded JSONL, mirroring the+ continuity store's append/list model). Same key + same hash β deduped+ (no-op); same key + different hash β `payload_mismatch` (explicit throw,+ never silent overwrite).+ - `lookupKey(vault, key)` β `{ contentHash } | null` (read for audit/C6).+- `writeSignal` / `writePreference` / `appendApplyEvidence` gain an OPTIONAL+ `idempotency_key?: string` parameter (additive β absent by default, existing+ writes byte-identical). When present, the writer computes the content hash of+ the would-be payload and consults the ledger before writing.+- `brain_feedback` (`feedback-tools.ts`) forwards an optional `idempotency_key`+ argument to the underlying writers.++### A1 β content-hash skip-unchanged manifest (t_c5184fd8)+See shared module 1 above. The manifest is distinct from identity dedup.++### A2 β per-row event-time on batch remember/import (t_7526e8d3)+- `src/core/brain/signal.ts` `WriteSignalInput`: add optional `valid_from?`,+ `recorded_at?` (additive). These flow into the same frontmatter keys+ `readBiTemporalSlots()` already parses, so the read-side needs no change.+ Absent by default β existing signals byte-identical.+- `src/core/brain/sessions/import.ts` `emit`: prefer `SessionTurn.timestamp`+ (when present and parseable) for `created_at`/`recorded_at`, falling back to+ `now` only when absent or unparseable. Guard against future-dated turn+ timestamps during backfill (clamp or skip β deterministic, documented).+- `brain_feedback` / batch import path forwards an optional per-row `event_time`.++### A3 β batch-plan step (t_9eeb8ca2)+- New module `src/core/brain/ingest/batch-plan.ts`: `planBatches(vault,+ sourceDir, { maxBatchBytes, maxBatchFiles })` β discovers ingestible files,+ calls A1's `classifyPaths` to skip `unchanged`, then splits the remainder into+ size+count-bounded batches (deterministic: sort by path, fill greedily to the+ byte cap then the count cap). Returns the batch list for the caller (agent/+ CLI) to dispatch as parallel subagents.+- CLI/MCP reader surfaces the plan. No parallel execution inside the kernel β+ the caller dispatches; o2b only plans (deterministic, no LLM).++### C1 β idempotent writes via client idempotency keys (t_213f356b)+See shared module 2 above. Payload-mismatch is an explicit error.++### C2 β dry-run extraction preview (t_2c6cf3e2)+- Add an optional `dryRun?: boolean` parameter to `extractPreCompactRecords`+ (and the other extraction entry points) mirroring the EXISTING `opts.dryRun`+ pattern in `import.ts`. When true, the function returns the candidate records+ it *would* extract WITHOUT calling `appendContinuityRecord` (no vault mutation,+ no log event, no dream/retire trigger). The returned records are the exact+ objects the real path would append, so preview faithfully predicts real+ extraction.+- `import.ts` already has `opts.dryRun`; C2 generalizes the same short-circuit+ to `pre-compact-extract.ts`. No global wrapper.++### C3 β bounded verbatim last-N-turns buffer (t_92317f91)+- New module `src/core/brain/recent-turns.ts`: a bounded verbatim ring buffer of+ the last N turns (hard cap, default e.g. 20), persisted via the continuity+ store as a new `ContinuityRecordKind` (`recent_turn`, additive β legacy records+ read as v1). `appendRecentTurn(vault, turn)` evicts oldest beyond N; reads are+ bounded. Ephemeral scaffolding, NOT curated memory β clearly separated (its own+ kind, its own reader).+- Optional post-compaction re-surface (opt-in flag) appends the buffer to the+ post-compaction context so the agent can recover "what did the user literally+ say 2-3 turns ago". Default off β byte-identical.++### C4 β batch checkpoint save for whole sessions (t_1a3a9eba)+- New module/MCP tool `brain_session_checkpoint` (or extend `synthesis-tools.ts`)+ that saves a whole session's signals + learning + summary in ONE round-trip.+ Idempotency is session-id + content-hash, reusing C1's idempotency ledger+ (`rememberKey`). If some writes need review (payload mismatch / validation+ failure), the checkpoint returns a `{ status: "ok" | "mixed", partial: [...] }`+ result β never silently drops items.++### C5 β caller-settable per-memory expiration date (t_a82b674e)+- `WritePreferenceInput` / `WriteSignalInput`: add optional ISO+ `expiration_date?` (additive β absent by default, existing records byte-+ identical). Flow into frontmatter.+- Read path (search/list): default silently drops anything past its+ `expiration_date`; an opt-in `showExpired` flag re-includes them for audit.+ Orthogonal to dream's heuristic retirement β an expired-by-date memory is+ filtered, not moved to `Brain/retired/` (audit trail preserved).++### C6 β delete and search by exact source file (t_edde2198)+- Search: add exact `source_file` filter (matches the `[[source-ref]]` /+ `session_ref` / ingest source-link frontmatter).+- Delete: a new `delete_by_source` command that is **dry-run by default** β it+ reports the blast radius (which derived signals/preferences/index entries/+ summary pages trace back to the source) WITHOUT deleting. An explicit+ `--confirm` deletes the derived entries (and index artifacts); original user+ notes are removed ONLY with an explicit `--include-originals` flag. Writes are+ auditable (continuity record of the cleanup).++### D1 β signed source-diversity grounding score (t_4678a91a)+- New module `src/core/brain/truth/grounding.ts`:+ `computeGroundingScore(slot, events)` β `{ score: number (-1.0..+1.0),+ confidence: "high"|"medium"|"low", band: "strongly_supported"|"mixed"|+ "contested"|"contradicted", supportingSources: number,+ contradictingSources: number }`. Computed as a pure projection over the+ `ClaimEvent`s that feed the slot (the same events `computeTruthState` folds),+ so history is never mutated β exactly the `conflicts.ts` discipline.+ - Source-DIVERSITY weighting: confirming vs contradicting evidence is counted+ across INDEPENDENT sources (distinct `source` + `agent`), not raw mention+ count. N mentions in one document count far below N mentions across N+ independent sources (document a weight ceiling; relationship strength+ weights the contribution).+ - Signed: + confirming, β contradicting, magnitude = balance; the separate+ confidence/sufficiency dimension captures how much evidence backs the+ direction (distinct from the signed score).+ - Deterministic: counting + weighting, no LLM.+- Surfaced alongside `computeTruthStateWithConflicts` output; replaces nothing+ (CONTESTED flag stays; the score quantifies it).++### D2 β note-position contradiction detection (t_11e3db8b)+- Extend `src/core/brain/health/contradiction.ts`: add a+ `detectNoteContradictions(notes, opts)` that pairs same-subject permanent notes+ asserting opposite stances. Reuses the existing language-agnostic+ token-overlap (`similarity.ts` `findSimilarPairs`/`tokenise`) + sign machinery,+ but derives position-sign from note PROSE rather than `evidenced_by` signals.+ Emits `ask_user` findings (quote the relevant span from each note); never+ auto-resolves.++### D3 β declared-thesis register with new-note monitor (t_74d29363)+- New module `src/core/brain/health/thesis.ts`: a declared-thesis register+ (operator-recorded positions: statement, supporting evidence, counter-evidence,+ last-updated, falsification "what would make me wrong"). Stored as a dedicated+ frontmatter kind under `Brain/theses/`.+- A monitor evaluates each newly-ingested note against active theses, flagging+ support/contradiction (builds on D2's note-position base). Reuses+ `obligations.ts` cadence machinery for the "not updated in N days" staleness+ check and the thesis-graveyard pass. Suppresses mere added-complexity per the+ article. Alerts when incoming evidence matches a thesis's documented+ falsification scenario.++### E1 β session-bracketing memory wrapper for Aider (t_1c894c19)+- New CLI wrapper `scripts/o2b-aider` (or `o2b aider wrap`) that brackets an+ Aider session: at start it runs the load-half (re-render the static context+ sidecar via the Aider adapter's template path, mirroring Hermes'+ `prefetch()`), execs the Aider binary with the injected `read:` context, and at+ session end runs the write-back half (capture the session and persist it into+ the Brain, mirroring Hermes' `sync_turn()`/`on_session_end()`). Implemented as+ a wrapper process because Aider has no native MCP (the mechanically hard part+ Hindsight solved).+- The static sidecar adapter (`src/core/install/adapters/aider.ts`) is an+ install-time template renderer (writes `<vault>/.open-second-brain/aider-context.md`+ from `templates/install/aider-context.md.tmpl`); it has no runtime capture+ logic today. The wrapper owns both the load-half (re-render sidecar) and the+ write-back half; the adapter stays as the fallback for users who do not run+ through the wrapper. Lifecycle shape mirrors+ `plugins/hermes/provider.py` `prefetch`/`sync_turn`/`on_session_end`.++## File changes (summary; per-card detail in plan.md)++- `src/core/brain/ingest/content-manifest.ts` β NEW (A1, consumed by A3).+- `src/core/brain/ingest/ingest.ts` β classify-before-extract (A1).+- `src/core/brain/ingest/batch-plan.ts` β NEW (A3).+- `src/core/brain/signal.ts` β `WriteSignalInput` `valid_from`/`recorded_at`/+ `expiration_date`/`idempotency_key` (A2, C5, C1).+- `src/core/brain/sessions/import.ts` β emit turn-time (A2) + dry-run (C2).+- `src/core/brain/pre-compact-extract.ts` β `dryRun` parameter (C2).+- `src/core/brain/idempotency-ledger.ts` β NEW (C1, consumed by C4).+- `src/core/brain/preference.ts` β `expiration_date`/`idempotency_key` (C5, C1).+- `src/core/brain/apply-evidence.ts` β `idempotency_key` (C1).+- `src/core/brain/recent-turns.ts` β NEW (C3) + continuity kind.+- `src/core/brain/truth/grounding.ts` β NEW (D1).+- `src/core/brain/truth/{fold,conflicts}.ts` β projection surface (D1).+- `src/core/brain/health/contradiction.ts` β note-position detection (D2).+- `src/core/brain/health/thesis.ts` β NEW (D3).+- `src/core/brain/{search,...}` β `source_file` filter + expiration filter (C6,+ C5).+- `src/mcp/brain/feedback-tools.ts` β `idempotency_key`/`event_time` forwarding+ (A2, C1).+- `src/mcp/brain/synthesis-tools.ts` β batch checkpoint (C4).+- `src/core/install/adapters/aider.ts` + `scripts/o2b-aider` β wrapper (E1).+- `src/core/brain/continuity/types.ts` β `recent_turn` kind (C3).+- Tests under `tests/core/brain/` and `tests/mcp/` per card (TDD).+- `CHANGELOG.md`, relevant docs β document the new surfaces.++## Risks++- **Byte-identity regression (A2/C5/C1).** Any new `WriteSignalInput`/+ `WritePreferenceInput` field that fires by default rewrites every file and+ breaks the determinism + no-op-rerun contract. Mitigation: all new fields are+ optional and absent by default; a dedicated bit-identity test asserts a write+ with the field absent == a write on the old code path.+- **Idempotency-ledger correctness (C1/C4).** A buggy ledger could itself drop+ legitimate writes or false-positive on mismatch. Mitigation: same-key-same-hash+ = deduped no-op; same-key-different-hash = explicit throw; ledger is append-only+ and auditable; C4's checkpoint never silently drops (mixed result reported).+- **Dry-run faithfulness (C2).** A preview that diverges from real extraction is+ worse than no preview. Mitigation: dry-run returns the EXACT records the real+ path would append, short-circuiting ONLY the `appendContinuityRecord` call; a+ parity test asserts dry-run candidates == real-run records for the same input.+- **Grounding-score determinism / no-ledger-mutation (D1).** A score that+ accidentally mutates the append-only ledger, or that is order-sensitive,+ breaks the truth contract. Mitigation: pure projection over the same events+ `computeTruthState` folds; order-insensitive (sort internally like the fold);+ never writes history.+- **Shared-file conflict (import.ts for A2+C2; contradiction.ts for D2+D3;+ ingest.ts for A1+A3).** Mitigation: drive order serializes each pair (A2 then+ C2; D2 then D3; A1 then A3); each worker pulls before starting and builds on+ prior commits.+- **Verbatim hoarding drift (C3).** A buffer that grows unbounded or leaks into+ curated memory violates o2b's deterministic-curation philosophy. Mitigation:+ hard cap on N; its own continuity kind and reader, clearly separated from+ curated memory; post-compaction re-surface is opt-in and default-off.+- **Aider wrapper portability (E1).** A wrapper process that breaks on a future+ Aider release or a non-POSIX shell. Mitigation: keep the static sidecar as the+ documented fallback; the wrapper is opt-in and degrades to the sidecar.+- **Source-cleanup blast radius (C6).** Deleting derived entries for a source+ could remove more than intended. Mitigation: dry-run-by-default reports the+ exact blast radius first; `--confirm` required to delete; originals protected+ unless `--include-originals`.diff --git a/docs/brainstorm/memory-signal-provenance-lifecycle/plan.md b/docs/brainstorm/memory-signal-provenance-lifecycle/plan.mdnew file mode 100644index 00000000..e27f2172--- /dev/null+++ b/docs/brainstorm/memory-signal-provenance-lifecycle/plan.md@@ -0,0 +1,428 @@+# Implementation plan β memory-signal provenance and lifecycle integrity++Branch: `feat/memory-signal-provenance-lifecycle`. Cards are driven ONE AT A+TIME on this shared branch. Each worker MUST `git pull` / build on the commits+previously-driven in-scope cards already landed, and must not duplicate or+conflict with sibling tasks. Follow TDD: write the failing test first.++Combined design: `docs/brainstorm/memory-signal-provenance-lifecycle/design.md`.+Variants + rationale: `docs/brainstorm/memory-signal-provenance-lifecycle/variants.md`.++Drive order: **A1 β C1 β A2 β D2 β D1 β E1 β C2 β C3 β C5 β C6 β A3 β C4 β D3**++In-scope cards (ship together in this one release):+- `t_c5184fd8` (A1, p4) β content-hash skip-unchanged manifest+- `t_213f356b` (C1, p3) β idempotent writes via client idempotency keys+- `t_7526e8d3` (A2, p4) β per-row event-time on batch remember/import+- `t_11e3db8b` (D2, p3) β note-position contradiction detection+- `t_4678a91a` (D1, p4) β signed source-diversity grounding score+- `t_1c894c19` (E1, p4) β session-bracketing memory wrapper for Aider+- `t_2c6cf3e2` (C2, p3) β dry-run extraction preview+- `t_92317f91` (C3, p3) β bounded verbatim last-N-turns buffer surviving compaction+- `t_a82b674e` (C5, p3) β caller-settable per-memory expiration date+- `t_edde2198` (C6, p3) β delete and search by exact source file+- `t_9eeb8ca2` (A3, p2) β batch-plan step (depends on A1)+- `t_1a3a9eba` (C4, p3) β batch checkpoint save for whole sessions (depends on C1)+- `t_74d29363` (D3, p2) β declared-thesis register monitor (depends on D2)++Engineering rules: SOLID / KISS / DRY; no misleading fallbacks; no hardcoding;+English-only strings, abstract multi-language. The kernel calls no LLM.++---++## A1 β `t_c5184fd8` β Content-hash skip-unchanged manifest (driven first)++### Files+- `src/core/brain/ingest/content-manifest.ts` β NEW.+ `hashFile(absPath)` / `hashTree(dir)` β SHA-256 hex over file BYTES (reuses the+ `createHash("sha256")` primitive of `sourceIdentityHash`, but over content, not+ source-path identity β a distinct artifact from identity dedup).+ `readManifest(vault)` / `writeManifestAtomic(vault, entries)` against+ `<vault>/.open-second-brain/ingest-manifest.json` (machine artifact, NOT under+ `Brain/`). `classifyPaths(vault, paths, manifest)` β `{ new, modified,+ unchanged, missing }`. `updateManifest(vault, paths)` records post-ingest+ hashes; a no-op rerun (all unchanged) rewrites nothing.+- `src/core/brain/ingest/ingest.ts` β call `classifyPaths` before the extraction+ pass; skip the LLM/extraction for `unchanged` sources (still idempotently+ rewrite the summary page only if its own bytes would change).+- `tests/core/brain/ingest/content-manifest.test.ts` β NEW.++### Acceptance (passing test)+A test in `tests/core/brain/ingest/content-manifest.test.ts` that:+1. After ingesting a source, re-ingesting an UNCHANGED file (same bytes) classifies+ it `unchanged` and the summary page is NOT rewritten (no-op-rerun invariant).+2. Touching/restat-ing the file (mtime change, byte-identical) still classifies+ `unchanged` (timestamp-independent β the regression this card exists to fix).+3. Modifying bytes classifies `modified`; deleting classifies `missing`; a new+ path classifies `new`.+4. Asserts the manifest is written atomically and a no-op rerun leaves it+ byte-identical.++### Depends on+None. Lands the shared manifest module that A3 consumes. Driven first (opens the+ingest axis).++---++## C1 β `t_213f356b` β Idempotent writes via client idempotency keys++### Files+- `src/core/brain/idempotency-ledger.ts` β NEW.+ `rememberKey(vault, { key, contentHash })` β `inserted | duplicate_match |+ payload_mismatch`. Keyed by client key β content hash, stored under+ `<vault>/Brain/logs/idempotency/` (month-sharded JSONL, mirroring the+ continuity store's append/list model). Same key + same hash β deduped no-op;+ same key + different hash β `payload_mismatch` (explicit throw, never silent+ overwrite). `lookupKey(vault, key)` for audit/C6.+- `src/core/brain/signal.ts` `writeSignal` β add optional `idempotency_key?`+ (additive; absent β existing behavior byte-identical). When present, hash the+ would-be payload and consult the ledger before writing.+- `src/core/brain/preference.ts` `writePreference` β add optional+ `idempotency_key?` (same shape).+- `src/core/brain/apply-evidence.ts` `appendApplyEvidence` β add optional+ `idempotency_key?` (same shape).+- `src/mcp/brain/feedback-tools.ts` `toolBrainFeedback` β forward an optional+ `idempotency_key` arg to the underlying writers.+- `tests/core/brain/idempotency-ledger.test.ts` β NEW; extend writer tests.++### Acceptance (passing test)+A test that:+1. With NO `idempotency_key`, a write is byte-identical to the old path (the+ bit-identity guard).+2. Two writes with the SAME key + SAME payload β second is deduped (returns+ duplicate_match, no file appended, no duplicate signal).+3. Two writes with the SAME key + DIFFERENT payload β second THROWS+ payload_mismatch (explicit error, never silent overwrite).+4. A retried write (same key, same payload) after a simulated crash is safe+ (deduped, not duplicated) β the multi-runtime retry case.++### Depends on+None. Lands the shared idempotency-ledger module that C4 consumes. Driven second+(opens the write-integrity axis).++---++## A2 β `t_7526e8d3` β Per-row event-time on batch remember/import++### Files+- `src/core/brain/signal.ts` `WriteSignalInput` β add optional `valid_from?` and+ `recorded_at?` (additive; absent β existing signals byte-identical). These flow+ into the SAME frontmatter keys `readBiTemporalSlots()` already parses+ (`valid_from`/`valid_until`/`recorded_at`), so the read-side needs no change.+- `src/core/brain/sessions/import.ts` `emit` β prefer `SessionTurn.timestamp`+ (when present and parseable) for `created_at`/`recorded_at`, falling back to+ `now` only when absent or unparseable. Guard against future-dated turn+ timestamps during backfill (clamp/skip deterministically; document the choice).+- `src/mcp/brain/feedback-tools.ts` + batch import path β forward an optional+ per-row `event_time`.+- `tests/core/brain/sessions/import-event-time.test.ts` β NEW.++### Acceptance (passing test)+A test in the import test suite that:+1. Backfilling an old session log stamps each signal's `created_at`/`recorded_at`+ with the turn's ORIGINAL timestamp, NOT the import wall-clock.+2. A turn with NO timestamp falls back to `now` (backward-compatible).+3. A turn with an UNPARSEABLE or FUTURE-dated timestamp is handled deterministically+ (falls back to `now` or clamps β no throw, no corrupt future-dated signal).+4. Asserts existing imports (no `event_time`) are byte-identical.++### Depends on+None (orthogonal to C1). Driven third. NOTE: this card touches `import.ts`+`emit`; C2 (later) adds a dry-run short-circuit to the same path β sequenced.++---++## D2 β `t_11e3db8b` β Note-position contradiction detection over prose++### Files+- `src/core/brain/health/contradiction.ts` β add+ `detectNoteContradictions(notes, opts)` that pairs same-subject permanent notes+ asserting opposite stances. Reuses the existing language-agnostic machinery:+ `similarity.ts` `findSimilarPairs`/`tokenise` for token-overlap subject pairing+ and a sign derivation from note PROSE (not `evidenced_by` signals). Emits+ `ask_user` findings that quote the relevant span from each note; never+ auto-resolves. New result type `NoteContradictionFinding` alongside the+ existing `ContradictionFinding`.+- `tests/core/brain/health/contradiction-notes.test.ts` β NEW.++### Acceptance (passing test)+A test that:+1. Two same-subject permanent notes with opposite stances are paired and surfaced+ as a contradiction finding, each with its quoted span.+2. Notes that merely ADD complexity (same stance) are NOT flagged.+3. Unrelated-subject notes are not paired (token-overlap threshold).+4. Detection is deterministic and language-agnostic (no English wordlist); a+ negation/stance derived structurally.++### Depends on+None. Driven fourth. NOTE: this card touches `contradiction.ts`; D3 (later) adds+the thesis monitor on top β sequenced.++---++## D1 β `t_4678a91a` β Signed source-diversity grounding score++### Files+- `src/core/brain/truth/grounding.ts` β NEW.+ `computeGroundingScore(slot, events)` β `{ score: number (-1.0..+1.0),+ confidence: "high"|"medium"|"low", band: "strongly_supported"|"mixed"|+ "contested"|"contradicted", supportingSources, contradictingSources }`.+ Pure projection over the `ClaimEvent`s that feed the slot (same events+ `computeTruthState` folds). Source-DIVERSITY weighting: confirming vs+ contradicting evidence counted across INDEPENDENT sources (distinct `source` ++ `agent`), N mentions in one doc << N mentions across N independent sources+ (document a weight ceiling); relationship strength weights the contribution.+ Signed (+ confirming, β contradicting); separate confidence/sufficiency+ dimension. Deterministic, order-insensitive (sort internally like the fold).+ Never writes/mutates the append-only ledger.+- `src/core/brain/truth/{fold,conflicts}.ts` β surface the projection alongside+ `computeTruthStateWithConflicts` output (additive; CONTESTED flag stays, the+ score quantifies it). No mutation of the fold.+- `tests/core/brain/truth/grounding.test.ts` β NEW.++### Acceptance (passing test)+A test that:+1. A slot agreed by N independent sources scores strongly positive; agreed by 1+ source scores weaker (source-diversity weighting is the part o2b lacks).+2. A contested slot (independent sources disagree) gets a score whose SIGN points+ to the better-supported side and whose magnitude reflects the balance.+3. N mentions in ONE document weigh far less than N mentions across N independent+ sources.+4. The score is deterministic and order-insensitive (shuffled input events β+ identical score), and the append-only ledger is not mutated.++### Depends on+None (owns `truth/` alone). Driven fifth.++---++## E1 β `t_1c894c19` β Session-bracketing memory wrapper for Aider++### Files+- `scripts/o2b-aider` (or `o2b aider wrap`) β NEW CLI wrapper process. At start,+ runs the load-half (regenerate the context sidecar via the existing Aider+ adapter's snapshot logic, mirroring Hermes `prefetch()`); execs the Aider binary+ with the injected `read:` context; at session end runs the write-back half+ (capture the session and persist into the Brain, mirroring Hermes+ `sync_turn()`/`on_session_end()`). Wrapper-process design because Aider has no+ native MCP.+- `src/core/install/adapters/aider.ts` β keep as the documented fallback (static+ sidecar for users who do not run through the wrapper). Extract reusable+ snapshot/capture helpers so the wrapper and the adapter share one source of+ truth (DRY).+- `tests/` for the wrapper lifecycle (load/persist bracketing, fallback path).+- Lifecycle shape mirrors `plugins/hermes/provider.py`+ `prefetch`/`sync_turn`/`on_session_end`.++### Acceptance (passing test)+A test (and/or an integration smoke) that:+1. The wrapper loads the memory context at session start (the load-half fires+ before Aider is exec'd) and persists the session back at session end (the+ write-back half fires after Aider exits).+2. The static sidecar fallback path still works unchanged for users who do not+ run through the wrapper (byte-identical adapter output).+3. An interrupted session (non-zero exit / signal) is handled honestly (captured+ as interrupted, mirroring `pre_compact_extract`'s `interrupted` flag).++### Depends on+None (fully isolated; no shared file with any other card). Driven sixth.++---++## C2 β `t_2c6cf3e2` β Dry-run extraction preview++### Files+- `src/core/brain/pre-compact-extract.ts` `extractPreCompactRecords` β add+ optional `dryRun?: boolean` mirroring the EXISTING `opts.dryRun` idiom in+ `import.ts`. When true, return the candidate records it WOULD extract WITHOUT+ calling `appendContinuityRecord` (no vault mutation, no log event, no+ dream/retire trigger). The returned records are the exact objects the real path+ would append.+- `src/core/brain/sessions/import.ts` β already has `opts.dryRun`; ensure the+ pre-compact path generalizes the same short-circuit. No global wrapper.+- `src/mcp/brain/pack-tools.ts` (or the relevant extract surface) β expose the+ preview as a non-destructive MCP/CLI verb.+- `tests/core/brain/pre-compact-extract-dry-run.test.ts` β NEW.++### Acceptance (passing test)+A test that:+1. `dryRun: true` returns the candidate records but writes NOTHING to the vault+ (no `appendContinuityRecord` call, no dream/retire trigger).+2. A parity assertion: the dry-run candidate records are deeply equal to the+ records the real path appends for the same input (preview faithfully predicts+ real extraction).+3. `dryRun` absent/false is byte-identical to today.++### Depends on+A2 (touches the same `import.ts` `emit` path; A2 settles event-time first, then+C2 gates it). Driven seventh.++---++## C3 β `t_92317f91` β Bounded verbatim last-N-turns buffer surviving compaction++### Files+- `src/core/brain/continuity/types.ts` β add `recent_turn` to+ `ContinuityRecordKind` (additive; legacy records read as v1).+- `src/core/brain/recent-turns.ts` β NEW. Bounded verbatim ring buffer of the+ last N turns (hard cap, default 20). `appendRecentTurn(vault, turn)` evicts+ oldest beyond N; `listRecentTurns(vault, { limit })` for reads. Persisted via+ the continuity store. Ephemeral scaffolding β its own kind + reader, clearly+ separated from curated memory.+- Optional post-compaction re-surface (opt-in flag, default off β byte-identical)+ appends the buffer to the post-compaction context.+- `tests/core/brain/recent-turns.test.ts` β NEW.++### Acceptance (passing test)+A test that:+1. Appending N+1 turns evicts the oldest (hard cap enforced); the buffer never+ grows beyond N.+2. After a simulated compaction, `listRecentTurns` returns the exact recent+ wording (the continuity-store artifact survives).+3. The buffer is clearly separated from curated memory (its own kind; reads do+ not leak into preference/signal/provenance surfaces).+4. Post-compaction re-surface is default-off β byte-identical context when off.++### Depends on+None. Driven eighth.++---++## C5 β `t_a82b674e` β Caller-settable per-memory expiration date++### Files+- `src/core/brain/preference.ts` `WritePreferenceInput` β add optional ISO+ `expiration_date?` (additive; absent β byte-identical). Flow into frontmatter.+- `src/core/brain/signal.ts` `WriteSignalInput` β add optional `expiration_date?`+ (same shape).+- Read path (search/list) β default silently drops anything past its+ `expiration_date`; an opt-in `showExpired` flag re-includes them for audit.+- Orthogonal to dream's heuristic retirement: an expired-by-date memory is+ FILTERED, not moved to `Brain/retired/` (audit trail preserved).+- `tests/core/brain/{preference,signal}-expiration.test.ts` β NEW.++### Acceptance (passing test)+A test that:+1. Writing a preference with `expiration_date` stamps the frontmatter; a write+ WITHOUT it is byte-identical to today.+2. Default search/list DROPS memories past their `expiration_date`.+3. With `showExpired: true`, expired memories are included (audit path).+4. An expired-by-date memory is NOT moved to `Brain/retired/` (orthogonal to+ dream; audit trail preserved).++### Depends on+None. Driven ninth.++---++## C6 β `t_edde2198` β Delete and search by exact source file++### Files+- Search: add exact `source_file` filter (matches `[[source-ref]]` /+ `session_ref` / ingest source-link frontmatter).+- `src/core/brain/source-cleanup.ts` (or extend an existing surface) β NEW+ `deleteBySource(vault, sourceFile, { confirm, includeOriginals })` that is+ DRY-RUN BY DEFAULT: reports the blast radius (derived signals/preferences/+ index entries/summary pages tracing to the source) WITHOUT deleting. An explicit+ `confirm` deletes the derived entries (and index artifacts); original user notes+ removed ONLY with an explicit `includeOriginals` flag. Writes auditable+ (continuity record of the cleanup).+- CLI/MCP reader surfaces search-by-source and the dry-run blast-radius report.+- `tests/core/brain/source-cleanup.test.ts` β NEW.++### Acceptance (passing test)+A test that:+1. `source_file` search filter returns only entries derived from that exact source.+2. `deleteBySource` with NO `confirm` reports the blast radius and deletes NOTHING.+3. With `confirm` (no `includeOriginals`), derived entries + index artifacts are+ deleted but ORIGINAL user notes are preserved.+4. With `confirm` + `includeOriginals`, originals are also removed. Cleanup is+ auditable (a continuity record is written).++### Depends on+None. Driven tenth.++---++## A3 β `t_9eeb8ca2` β Batch-plan step (depends on A1)++### Files+- `src/core/brain/ingest/batch-plan.ts` β NEW. `planBatches(vault, sourceDir,+ { maxBatchBytes, maxBatchFiles })` discovers ingestible files, calls A1's+ `classifyPaths` to skip `unchanged`, then splits the remainder into+ size+count-bounded batches (deterministic: sort by path, fill greedily to the+ byte cap then the count cap). Returns the batch list for the caller (agent/CLI)+ to dispatch as parallel subagents. No parallel execution inside the kernel.+- CLI/MCP reader surfaces the plan.+- `tests/core/brain/ingest/batch-plan.test.ts` β NEW.++### Acceptance (passing test)+A test that:+1. `planBatches` skips `unchanged` files (via A1's manifest) and plans batches+ only over `new`/`modified` files.+2. Batches respect BOTH the byte cap and the file-count cap (no batch exceeds+ either).+3. The plan is deterministic (same dir β same batch list, stable ordering).+4. An empty/all-unchanged dir yields an empty plan (no spurious batches).++### Depends on+**A1** (`t_c5184fd8`) β consumes the content-hash manifest's `classifyPaths`.+Driven eleventh. Pull before starting (A1's commits must be present).++---++## C4 β `t_1a3a9eba` β Batch checkpoint save for whole sessions (depends on C1)++### Files+- `src/mcp/brain/synthesis-tools.ts` (or a new checkpoint module) β NEW+ `brain_session_checkpoint` tool that saves a whole session's signals + learning+ + summary in ONE idempotent MCP round-trip. Idempotency is session-id + content-+ hash, reusing C1's idempotency ledger (`rememberKey`). If some writes need+ review (payload mismatch / validation failure), returns+ `{ status: "ok" | "mixed", partial: [...] }` β never silently drops items.+- `tests/mcp/session-checkpoint.test.ts` β NEW.++### Acceptance (passing test)+A test that:+1. A checkpoint with a fresh session-id writes all items once; a RETRY with the+ same session-id + same content is deduped (idempotent, no duplicates) via C1's+ ledger.+2. A checkpoint where some items need review returns `status: "mixed"` with the+ partial list (never silently drops).+3. A retry with the same session-id but DIFFERENT content throws payload_mismatch+ (explicit error, no silent overwrite).++### Depends on+**C1** (`t_213f356b`) β reuses the idempotency-key ledger. Driven twelfth. Pull+before starting (C1's commits must be present).++---++## D3 β `t_74d29363` β Declared-thesis register with new-note monitor (depends on D2)++### Files+- `src/core/brain/health/thesis.ts` β NEW. A declared-thesis register+ (operator-recorded positions: statement, supporting evidence, counter-evidence,+ last-updated, falsification "what would make me wrong"), stored as a dedicated+ frontmatter kind under `Brain/theses/`. A monitor evaluates each newly-ingested+ note against active theses, flagging support/contradiction (builds on D2's+ note-position base). Reuses `obligations.ts` cadence machinery for the+ "not updated in N days" staleness check and the thesis-graveyard pass (flag+ theses with no supporting evidence in N days for formal closing). Suppresses+ mere added-complexity per the article. Alerts when incoming evidence matches a+ thesis's documented falsification scenario.+- `tests/core/brain/health/thesis.test.ts` β NEW.++### Acceptance (passing test)+A test that:+1. Recording a thesis and ingesting a note that CONTRADICTS it raises a conflict+ flag (quoting the thesis note); a note that SUPPORTS it raises a support flag.+2. A thesis not updated within the cadence is flagged stale (via the obligations+ machinery); a thesis with no supporting evidence in N days is flagged for the+ graveyard.+3. Incoming evidence matching a thesis's falsification field raises an alert.+4. A note that merely adds complexity is NOT flagged. Never auto-resolves.++### Depends on+**D2** (`t_11e3db8b`) β builds on the note-position detector. Driven last (closes+the honesty axis). Pull before starting (D2's commits must be present).diff --git a/docs/brainstorm/memory-signal-provenance-lifecycle/variants.md b/docs/brainstorm/memory-signal-provenance-lifecycle/variants.mdnew file mode 100644index 00000000..519542e5--- /dev/null+++ b/docs/brainstorm/memory-signal-provenance-lifecycle/variants.md@@ -0,0 +1,66 @@+# Variants β memory-signal provenance and lifecycle integrity++Verbatim consultant output (`claude-opus-4-8`) + final rationale.++## Variant 1 β Shared-substrate first+Approach: Land the three cross-cutting primitives as standalone modules before any consumer: a `manifest` module (content-hash cache + classify), an `idempotency` keyβcontent-hash ledger under `Brain/`, and a single `dryRun` gate that wraps every continuity-append surface. All 13 cards then reduce to thin verticals that call into settled infrastructure.+Trade-offs:+- Maximal reuse and one canonical answer for each concern (one hashing path, one dedup notion, one preview switch) β no divergent re-implementations.+- Couples the p4 anchors (A1) and the shared ledger to the p2 capstones (A3, C4) through APIs designed before their only consumers exist, so the primitive shape is guessed, not validated; a wrong guess forces a rewrite that ripples across cards.+- A global dry-run wrapper contradicts the existing per-surface `opts.dryRun` idiom in `import.ts` and risks a leaky abstraction that must special-case dream/retire triggers anyway.+Complexity: large+Risk: high++## Variant 2 β Self-contained verticals+Approach: Treat every card as an independent vertical driven by priority then dependency, with no shared modules beyond the existing `sourceIdentityHash`/SHA-256 primitive. A1 grows its manifest inside ingest; C1 grows its ledger inside feedback-tools; C4 and A3 reach directly into whatever their parents left behind rather than a shared API.+Trade-offs:+- Lowest coupling and simplest per-card review; each commit is atomic and byte-identical-when-off is trivial to verify in isolation.+- The two hard composition edges (A1βA3, C1βC4) still force reuse, so "no shared module" is a fiction β A3 and C4 either duplicate parent logic or depend on internal shapes, producing drift (two idempotency notions, two hash-manifest readers) that the next release must reconcile.+- No structural place for the dry-run concern shared by C2 and any future preview, so C2's short-circuit logic is bespoke and hard to prove faithful to the real extraction path.+Complexity: medium+Risk: medium++## Variant 3 β Topological hybrid: shared seams only at composition edges+Approach: Extract shared infrastructure only where a hard dependency already mandates it β A1 ships the manifest as a real module because A3 consumes it, C1 ships the keyβcontent-hash ledger as a real module because C4 consumes it β and keep everything else a self-contained vertical, mirroring `import.ts`'s existing `opts.dryRun` as a per-surface parameter rather than a global wrapper. Drive in a topological order that serializes every shared-file collision: A1 β C1 β A2 β D2 β D1 β C2 β C3 β C5 β C6 β A3 β C4 β D3.+Trade-offs:+- Each shared module is designed against a concrete consumer (A3, C4) landed shortly after, so the API is validated by use, not guessed; the p4 anchors never wait on the p2 capstones.+- File collisions resolve by ordering, not merging: A1 before A3 on `ingest.ts`; A2 before C2 on `import.ts` (event-time settles the emit path, then dry-run gates it); D2 before D3 on `contradiction.ts`; D1 owns `truth/{fold,conflicts}.ts` alone as a pure projection over the append-only ledger.+- Two genuine modules (manifest, ledger) plus disciplined ordering is more upfront design than pure verticals, and the capstones (A3, C4, D3) landing last means the longest-pole cards gate branch completion.+Complexity: medium+Risk: low++## Recommended: Variant 3+The release's own constraints already dictate the answer: two hard composition edges (A1βA3, C1βC4) mean the manifest and idempotency ledger *must* be reusable, but nothing else composes, so building the full shared substrate of Variant 1 pays coupling and speculative-API cost for reuse the cards do not ask for. Variant 2's refusal to extract even those two modules is a false economy β the edges force reuse regardless, and doing it implicitly produces exactly the duplicate-hash / dual-idempotency drift this release exists to eliminate. Variant 3 extracts shared code precisely at the mandated seams and validates each module against a consumer landed immediately after, while a single topological drive order serializes all five shared-file collisions and honors all three parentβchild edges without coupling the p4 anchors to the p2 capstones. Keeping dry-run as a per-surface parameter that mirrors the existing `import.ts` idiom preserves the byte-identical-when-off invariant and keeps preview faithful to real extraction, which a global wrapper cannot guarantee.++---++## Final decision: Variant 3++**Agreed with the consultant's recommendation.** Variant 3 is the right fit+because it preserves o2b's load-bearing invariants and resolves every concrete+constraint the 13 cards impose:++1. **Reuse only where a hard edge mandates it.** Two composition edges are real+ (A1βA3 the content-hash manifest; C1βC4 the idempotency-key ledger). Variant 3+ extracts exactly those two modules and validates each against a consumer+ landed immediately after β never guessing an API. Variant 1's speculative full+ substrate would couple the p4 anchors to p2 capstones through unvalidated+ shapes; Variant 2's refusal to extract would produce the dual-hash /+ dual-idempotency drift this release exists to eliminate.+2. **Byte-identity-when-off / absent-by-default.** Every new field is additive+ and optional; dry-run stays a per-surface parameter mirroring the existing+ `import.ts` `opts.dryRun` idiom (not a global wrapper), so each card keeps a+ locally-provable off-branch and preview stays faithful to real extraction.+3. **Shared-file collisions serialized by drive order.** The single topological+ order (A1 β C1 β A2 β D2 β D1 β E1 β C2 β C3 β C5 β C6 β A3 β C4 β D3)+ resolves all five collisions (`ingest.ts` for A1+A3; `import.ts` for A2+C2;+ `contradiction.ts` for D2+D3; `WriteSignalInput` for A2/C5/C1;+ `truth/{fold,conflicts}.ts` owned by D1 alone) and honors all three+ parentβchild edges (A1βA3, C1βC4, D2βD3).++One amendment to the consultant's drive order: it listed 12 of 13 cards and+omitted E1 (the isolated Aider wrapper). E1 has no shared-file collision and no+dependency, so it slots cleanly into the p4-anchor group (driven sixth, after the+other three p4 anchors). The variant choice is unaffected.++Drive order: **A1 β C1 β A2 β D2 β D1 β E1 β C2 β C3 β C5 β C6 β A3 β C4 β D3**.diff --git a/openclaw.plugin.json b/openclaw.plugin.jsonindex 2588efc9..d1378f3b 100644--- a/openclaw.plugin.json+++ b/openclaw.plugin.json@@ -2,7 +2,7 @@"id": "open-second-brain","name": "Open Second Brain","description": "Second brain for AI agents using Obsidian-compatible Markdown vaults.",- "version": "1.25.0",+ "version": "1.26.0","activation": { "onStartup": true },"skills": ["./skills"],"contracts": {diff --git a/package.json b/package.jsonindex 28ed0af3..77e5ae81 100644--- a/package.json+++ b/package.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.25.0",+ "version": "1.26.0","private": false,"description": "Second brain for AI agents using Obsidian-compatible Markdown vaults. Works with Hermes, Claude Code, Codex, and OpenClaw.","keywords": [diff --git a/plugin.yaml b/plugin.yamlindex e163cf42..0754f860 100644--- a/plugin.yaml+++ b/plugin.yaml@@ -1,5 +1,5 @@name: open-second-brain-version: "1.25.0"+version: "1.26.0"description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."author: "Open Second Brain contributors"memory_provider: truediff --git a/plugins/codex/.codex-plugin/plugin.json b/plugins/codex/.codex-plugin/plugin.jsonindex 120210a6..a3e7fe85 100644--- a/plugins/codex/.codex-plugin/plugin.json+++ b/plugins/codex/.codex-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.25.0",+ "version": "1.26.0","description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.","author": {"name": "Open Second Brain contributors",diff --git a/plugins/hermes/_schemas.py b/plugins/hermes/_schemas.pyindex be7f5432..d2616007 100644--- a/plugins/hermes/_schemas.py+++ b/plugins/hermes/_schemas.py@@ -64,7 +64,20 @@ STATIC_TOOL_SCHEMAS: tuple[dict[str, Any], ...] = ('alongside the inbox ''signal, skipping the ''dream-pass promotion '- 'step.'}},+ 'step.'},+ 'event_time': {'type': 'string',+ 'description': 'Optional ISO-8601 event-time for '+ 'a backfilled signal (when it '+ 'actually happened). Stamps '+ '`created_at`/`valid_from`/'+ '`recorded_at`; absent uses '+ 'wall-clock.'},+ 'idempotency_key': {'type': 'string',+ 'description': 'Optional client key that '+ 'dedupes retried calls: same '+ 'key + same payload is a '+ 'no-op; same key + different '+ 'payload is rejected.'}},'required': ['topic', 'signal', 'principle'],'additionalProperties': False}},{'name': 'brain_apply_evidence',@@ -165,6 +178,13 @@ STATIC_TOOL_SCHEMAS: tuple[dict[str, Any], ...] = ('description': 'Topic slug to aggregate signals + ''active/retired preference + log ''events.'},+ 'show_expired': {'type': 'boolean',+ 'description': 'Topic mode only: include '+ 'memories past their '+ '`expiration_date`. Default '+ 'false (expired memories are '+ 'silently dropped from the '+ 'result).'},'since': {'type': 'string','description': 'ISO-8601 timestamp; returns every ''Brain log event with timestamp >= '@@ -324,13 +344,23 @@ STATIC_TOOL_SCHEMAS: tuple[dict[str, Any], ...] = ('additionalProperties': False}},{'name': 'brain_recall_gate','description': 'Classify whether an automatic recall/surfacing attempt should run. '- 'Diagnostics only; does not search.',+ 'Diagnostics only; does not search. Pass `scores` (a recall attempt\'s '+ 'top-k relevance scores) to also get an adequacy verdict β sufficient '+ '(proceed) / weak (re_recall) / insufficient (abstain + escalate).','inputSchema': {'type': 'object','properties': {'prompt': {'type': 'string', 'minLength': 1, 'maxLength': 4000},'previous_prompt': {'type': 'string', 'maxLength': 4000},'explicit': {'type': 'boolean'},'telemetry_host': {'type': 'string', 'maxLength': 200},- 'session_id': {'type': 'string', 'maxLength': 512}},+ 'session_id': {'type': 'string', 'maxLength': 512},+ 'scores': {'type': 'array',+ 'maxItems': 200,+ 'items': {'type': 'number'},+ 'description': 'Optional top-k recall relevance '+ 'scores. When given, the gate adds '+ 'an adequacy verdict: '+ 'sufficient/proceed, weak/re_recall, '+ 'or insufficient/abstain.'}},'required': ['prompt'],'additionalProperties': False}},{'name': 'brain_context',@@ -406,6 +436,15 @@ STATIC_TOOL_SCHEMAS: tuple[dict[str, Any], ...] = ('description': 'Optional host/runtime name ''for emitted receipts; ''defaults to `mcp`.'},+ 'recall_scores': {'type': 'array',+ 'maxItems': 200,+ 'items': {'type': 'number'},+ 'description': 'Optional relevance scores '+ 'of the recall behind this '+ 'material. When given, the '+ 'response adds an adequacy '+ 'verdict (level + action) and '+ 'persists it in the receipt.'},'telemetry': {'type': 'boolean','description': 'When true, emit an opt-in recall ''telemetry record for this '@@ -451,7 +490,13 @@ STATIC_TOOL_SCHEMAS: tuple[dict[str, Any], ...] = ('interrupted close ''(SIGHUP/SIGTERM/force-quit/''restart-drain). Absent by '- 'default.'}},+ 'default.'},+ 'dry_run': {'type': 'boolean',+ 'description': 'Preview the candidate records '+ 'extraction would append WITHOUT '+ 'writing to the vault (no '+ 'continuity record, no dream/retire '+ 'trigger). Absent by default.'}},'required': ['session_id', 'turn_start', 'turn_end', 'text'],'additionalProperties': False}},)diff --git a/plugins/hermes/plugin.yaml b/plugins/hermes/plugin.yamlindex e163cf42..0754f860 100644--- a/plugins/hermes/plugin.yaml+++ b/plugins/hermes/plugin.yaml@@ -1,5 +1,5 @@name: open-second-brain-version: "1.25.0"+version: "1.26.0"description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."author: "Open Second Brain contributors"memory_provider: truediff --git a/pyproject.toml b/pyproject.tomlindex 6eda7ca6..3aeb5399 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -10,7 +10,7 @@ build-backend = "setuptools.build_meta"# CLI entry points (those moved to `package.json` `bin`).[project]name = "open-second-brain"-version = "1.25.0"+version = "1.26.0"description = "Hermes Python shim for Open Second Brain. Most of the project (CLI, MCP server, OpenClaw plugin) is TypeScript on Bun; see package.json."readme = "README.md"requires-python = ">=3.11"diff --git a/scripts/o2b-aider b/scripts/o2b-aidernew file mode 100755index 00000000..df1390c7--- /dev/null+++ b/scripts/o2b-aider@@ -0,0 +1,22 @@+#!/usr/bin/env bash+set -euo pipefail++# Session-bracketing memory wrapper for Aider (E1). Thin convenience entry+# for `o2b aider wrap` β regenerate the live context sidecar, run Aider with+# it injected, then capture the session back into the Brain. Everything passed+# here is forwarded to `o2b aider wrap` (use `-- <aider args...>` to reach+# Aider's own flags).++# Follow symlink so the repo root is correct even when invoked through+# a package-manager bin shim or a symlink in ~/.local/bin.+SELF=$(realpath -- "${BASH_SOURCE[0]}")+SCRIPT_DIR=$(cd -- "$(dirname -- "$SELF")" && pwd)+REPO_ROOT=$(cd -- "$SCRIPT_DIR/.." && pwd)++# shellcheck source=./_bun-precheck.sh+. "$SCRIPT_DIR/_bun-precheck.sh"++# shellcheck source=./_macos-sqlite.sh+. "$SCRIPT_DIR/_macos-sqlite.sh"++exec bun run "$REPO_ROOT/src/cli/main.ts" aider wrap "$@"diff --git a/src/cli/aider.ts b/src/cli/aider.tsnew file mode 100644index 00000000..56ea58f6--- /dev/null+++ b/src/cli/aider.ts@@ -0,0 +1,175 @@+/**+ * `o2b aider wrap` β session-bracketing memory wrapper for Aider (E1).+ *+ * o2b aider wrap [--session-id ID] [--aider-bin aider] [-- <aider args...>]+ *+ * Aider has no MCP client, so this wraps the Aider CLI as a process:+ * 1. LOAD-HALF: regenerate the live context sidecar and inject it into+ * Aider's read context (`--read <sidecar>`).+ * 2. Exec Aider (stdio inherited β a fully interactive session).+ * 3. WRITE-BACK HALF: capture the session transcript (the new tail of+ * Aider's `.aider.chat.history.md`) and persist it into the Brain via+ * the deterministic `pre_compact_extract` path β honestly flagged+ * `interrupted` when Aider exited non-zero / on a signal.+ *+ * The static `o2b install --target aider --apply` sidecar remains the+ * fallback for users who do not run through this wrapper. Both share one+ * snapshot renderer (`renderAiderSidecar`).+ *+ * Exit codes mirror the wrapped Aider process so shell callers see the real+ * result: Aider's own exit code passes through; a usage error is 2.+ */++import { spawnSync } from "node:child_process";+import { existsSync, mkdirSync, readFileSync, statSync } from "node:fs";+import { homedir } from "node:os";+import { dirname, join, resolve } from "node:path";++import { parseFlags } from "./argparse.ts";+import { defaultConfigPath, discoverConfig } from "../core/config.ts";+import { requireVault } from "./helpers.ts";+import { atomicWriteFileSync } from "../core/fs-atomic.ts";+import { buildPayload } from "../core/install/payload.ts";+import type { InstallEnv } from "../core/install/types.ts";+import {+ bracketAiderSession,+ renderAiderSidecar,+ resolveAiderSidecarPath,+ type AiderSpawnResult,+} from "../core/install/adapters/aider-wrapper.ts";+import { extractPreCompactRecords } from "../core/brain/pre-compact-extract.ts";++export async function handleAiderSubcommand(argv: string[]): Promise<number> {+ const verb = argv[0];+ if (verb === undefined || verb === "-h" || verb === "--help") {+ process.stdout.write(AIDER_HELP);+ return verb === undefined ? 2 : 0;+ }+ if (verb !== "wrap") {+ process.stderr.write(`error: unknown aider verb: ${verb}\n`);+ process.stderr.write(AIDER_HELP);+ return 2;+ }+ return await cmdAiderWrap(argv.slice(1));+}++const AIDER_HELP = `usage: o2b aider wrap [options] [-- <aider args...>]++Bracket an interactive Aider session with live memory load + write-back.++Options:+ --session-id ID Session id recorded on the write-back (default: auto).+ --aider-bin PATH Aider executable to run (default: aider).+ --chat-history PATH Aider chat history file to capture+ (default: <cwd>/.aider.chat.history.md).+ --vault PATH Vault whose Brain/ receives the write-back.+ --config PATH Config file to resolve vault/agent from.++Everything after -- is forwarded verbatim to Aider.+`;++async function cmdAiderWrap(argv: string[]): Promise<number> {+ const { flags, positional } = parseFlags(argv, {+ "session-id": { type: "string" },+ "aider-bin": { type: "string", default: "aider" },+ "chat-history": { type: "string" },+ vault: { type: "string" },+ config: { type: "string" },+ });++ const configPath = (flags["config"] as string | undefined) ?? defaultConfigPath();+ const vault = requireVault(flags["vault"] as string | undefined, configPath);+ const env = buildEnv(vault, configPath);+ const payload = buildPayload({+ vault,+ agent_name: env.env["VAULT_AGENT_NAME"] ?? null,+ timezone: env.env["VAULT_TIMEZONE"] ?? null,+ });++ const sidecarPath = resolveAiderSidecarPath(env, {});+ const aiderBin = flags["aider-bin"] as string;+ const chatHistoryPath = resolve(+ (flags["chat-history"] as string | undefined) ?? join(env.cwd, ".aider.chat.history.md"),+ );+ const sessionId = (flags["session-id"] as string | undefined) ?? `aider-${env.now.toISOString()}`;+ const aiderArgs = positional;++ // Capture the transcript by delta: record the chat-history size at load+ // time, then read only what Aider appended during the session.+ let historyOffsetAtStart = 0;++ const result = await bracketAiderSession(+ { sessionId, host: "aider" },+ {+ loadContext: () => {+ ensureParent(sidecarPath);+ atomicWriteFileSync(sidecarPath, renderAiderSidecar(env, payload));+ historyOffsetAtStart = fileSize(chatHistoryPath);+ return sidecarPath;+ },+ spawnAider: (sidecar) => Promise.resolve(runAider(aiderBin, sidecar, aiderArgs, env.cwd)),+ captureTranscript: () => readTail(chatHistoryPath, historyOffsetAtStart),+ persist: (input) => extractPreCompactRecords(vault, input),+ },+ );++ reportOutcome(result.exit, result.interrupted, result.persisted?.records.length ?? 0);++ // Pass Aider's own exit code through so shell callers see the real result.+ if (result.exit.signal !== null) return 1;+ return result.exit.code ?? 1;+}++function buildEnv(vault: string, configPath: string): InstallEnv {+ const cfg = discoverConfig(configPath).data;+ const merged = { ...process.env } as Record<string, string>;+ if (cfg["agent_name"]) merged["VAULT_AGENT_NAME"] = cfg["agent_name"];+ if (cfg["timezone"]) merged["VAULT_TIMEZONE"] = cfg["timezone"];+ return { vault, home: homedir(), cwd: process.cwd(), env: merged, now: new Date() };+}++function runAider(bin: string, sidecar: string, args: string[], cwd: string): AiderSpawnResult {+ const proc = spawnSync(bin, ["--read", sidecar, ...args], {+ cwd,+ stdio: "inherit",+ });+ if (proc.error) {+ // Missing binary / spawn failure β honest non-zero, not a silent success.+ process.stderr.write(`error: failed to launch aider (${bin}): ${proc.error.message}\n`);+ return { code: 127, signal: null };+ }+ return { code: proc.status, signal: proc.signal };+}++function reportOutcome(exit: AiderSpawnResult, interrupted: boolean, recordCount: number): void {+ const how = interrupted+ ? exit.signal !== null+ ? `interrupted by ${exit.signal}`+ : `interrupted (exit ${exit.code})`+ : "exited cleanly";+ process.stderr.write(`[aider-wrap] session ${how}; captured ${recordCount} memory record(s)\n`);+}++function ensureParent(path: string): void {+ const parent = dirname(path);+ if (!existsSync(parent)) mkdirSync(parent, { recursive: true });+}++function fileSize(path: string): number {+ try {+ return statSync(path).size;+ } catch {+ return 0;+ }+}++/** Read the bytes appended to `path` since `fromByte` (the session delta). */+function readTail(path: string, fromByte: number): string {+ if (!existsSync(path)) return "";+ // Slice by BYTES (statSync.size is bytes), so multi-byte UTF-8 offsets stay+ // correct. If the file shrank (rotated/truncated mid-session), capture the+ // whole current content rather than a bogus negative slice.+ const buf = readFileSync(path);+ const start = buf.length >= fromByte ? fromByte : 0;+ return buf.subarray(start).toString("utf8");+}diff --git a/src/cli/brain.ts b/src/cli/brain.tsindex d1b3a93c..d044c071 100644--- a/src/cli/brain.ts+++ b/src/cli/brain.ts@@ -73,6 +73,8 @@ import {cmdBrainScanInline,cmdBrainEntity,cmdBrainImportSession,+ cmdBrainForgetSource,+ cmdBrainBatchPlan,cmdBrainHandoff,cmdBrainIntention,cmdBrainProject,@@ -270,6 +272,10 @@ export async function handleBrainSubcommand(argv: ReadonlyArray<string>): Promisreturn await cmdBrainProject(rest);case "source":return await cmdBrainSource(rest);+ case "forget-source":+ return await cmdBrainForgetSource(rest);+ case "batch-plan":+ return await cmdBrainBatchPlan(rest);case "links":return await cmdBrainLinks(rest);case "profile":diff --git a/src/cli/brain/help-text.ts b/src/cli/brain/help-text.tsindex 25c90a7d..a5aef669 100644--- a/src/cli/brain/help-text.ts+++ b/src/cli/brain/help-text.ts@@ -111,6 +111,7 @@ Brain verbs (observing memory):weekly 7-day synthesis: transitions, retired, contradictions, vault deltaproject Link project directories to their owning vault (link/list/remove/status)source Read-only recall sources of the active vault (add/list/remove)+ forget-source Find/delete entries derived from an exact source file (dry-run by default)links Normalize wikilink path format (preserve/full/short); dry-run by defaultprofile Materialize Brain/profile.md digest + .o2bfs root marker (age-gated)sgrep Grep-shaped semantic search: o2b brain sgrep <query> [path]@@ -131,6 +132,14 @@ Common flags:`;export const VERB_HELP: Record<string, string> = {+ "forget-source":+ "usage: o2b brain forget-source <source> [--confirm] [--include-originals] [--vault <path>] [--json]\n" ++ "Find and delete every Brain entry derived from an EXACT source file. Dry-run by\n" ++ "default: reports the blast radius and deletes nothing. --confirm deletes the\n" ++ "single-purpose derived entries + ingest index artifacts (summary page + manifest\n" ++ "entry); shared/aggregate pages are reported, never deleted. --include-originals\n" ++ "also removes the original source file outside Brain/. Confirmed runs write an\n" ++ "auditable source_invalidation continuity record.\n",init:"usage: o2b brain init [--vault <path>] [--force] [--primary-agent <name>] [--json]\n" +"Bootstrap <vault>/Brain/. Requires `o2b init` to have run first.\n" +@@ -480,6 +489,7 @@ export const VERB_HELP: Record<string, string> = {" [--format auto|<registered-adapter>]\n" +" [--agent <name>] [--since <ISO>] [--dry-run] [--recall]\n" +" [--ingest-scope <label>] [--filter-role <role> ...] [--filter-text <substring>]\n" ++ " [--preserve-event-time]\n" +" [--recall-session-id <id>] [--recall-summary-group-size <n>] [--json]\n" +"Extract signals from a registered agent session .jsonl file (or\n" +"directory of .jsonl files). Two extraction paths run in parallel:\n" +@@ -487,6 +497,11 @@ export const VERB_HELP: Record<string, string> = {"tool_use calls. Dedup against the inbox by normalised payload hash.\n" +"With --recall, also stores normalized turns in the continuity-backed\n" +"session recall DAG.\n" ++ "With --preserve-event-time, each signal is stamped with its turn's\n" ++ "original timestamp (created_at/recorded_at/valid_from) instead of the\n" ++ "import wall-clock, so backfilling an old log stays historically\n" ++ "faithful. Turns with an absent/unparseable/future timestamp fall back\n" ++ "to now.\n" +"Autodetect failure exits 2 β pass --format to override.\n",merge:"usage: o2b brain merge <keep-pref-id> <drop-pref-id>\n" +@@ -580,8 +595,9 @@ export const VERB_HELP: Record<string, string> = {" o2b brain context-presets diff <preset-id> [current-value flags] [--override <path>...] [--json]\n" +"Dry-run model-aware context budget preset diagnostics. Never writes config.\n","pre-compact-extract":- "usage: o2b brain pre-compact-extract --vault <path> --session-id <id> --turn-start <id> --turn-end <id> --text <text> [--host <name>] [--max-chars <n>] [--json]\n" +- "Extract Decision/Commitment/Outcome/Rule/Open question lines into idempotent continuity records.\n",+ "usage: o2b brain pre-compact-extract --vault <path> --session-id <id> --turn-start <id> --turn-end <id> --text <text> [--host <name>] [--max-chars <n>] [--dry-run] [--json]\n" ++ "Extract Decision/Commitment/Outcome/Rule/Open question lines into idempotent continuity records.\n" ++ "--dry-run previews the candidate records without writing to the vault (predicts the real extraction byte-for-byte).\n","post-compact-audit":"usage: o2b brain post-compact-audit [--session-id <id>] [--no-reassert] [--force] [--vault <path>] [--json]\n" +"Reads a { session_id, messages } JSON document from stdin. Detects a Hermes compaction, audits which pinned anchors survived in the active (non-summary) region, and re-asserts only the drifted ones. Gated by post_compact_survival_audit (default off); --force overrides the gate.\n",diff --git a/src/cli/brain/verbs/batch-plan.ts b/src/cli/brain/verbs/batch-plan.tsnew file mode 100644index 00000000..3f4d75b8--- /dev/null+++ b/src/cli/brain/verbs/batch-plan.ts@@ -0,0 +1,83 @@+/**+ * `o2b brain batch-plan <source-dir>` (A3 / t_9eeb8ca2): plan a large-folder+ * ingest into bounded parallel batches.+ *+ * Discovers the ingestible (text-bearing) files under the directory, skips+ * everything the content-hash manifest classifies `unchanged` since the last+ * ingest, and packs the new/modified remainder into batches bounded by+ * `--max-bytes` and `--max-files`. Read-only and deterministic β the caller+ * dispatches each printed batch as a parallel ingest subagent; the kernel+ * spawns nothing itself.+ */++import { planBatches } from "../../../core/brain/ingest/batch-plan.ts";+import { brainVerbContext, fail, info, ok, okJson, parse } from "../helpers.ts";++/** Default caps, mirroring the MCP surface (1 MiB / 25 files per batch). */+const DEFAULT_MAX_BATCH_BYTES = 1024 * 1024;+const DEFAULT_MAX_BATCH_FILES = 25;++function parseCap(raw: unknown, name: string, fallback: number): number {+ if (raw === undefined) return fallback;+ const n = Number(raw);+ if (!Number.isInteger(n) || n <= 0) {+ throw new Error(`${name} must be a positive integer, got ${String(raw)}`);+ }+ return n;+}++export async function cmdBrainBatchPlan(argv: string[]): Promise<number> {+ const { flags, positional } = parse(argv, {+ vault: { type: "string" },+ "max-bytes": { type: "string" },+ "max-files": { type: "string" },+ json: { type: "boolean" },+ });+ const sourceDir = positional[0];+ if (!sourceDir) {+ return fail(+ "usage: o2b brain batch-plan <source-dir> [--max-bytes N] [--max-files N] [--json]",+ );+ }++ try {+ const { vault } = brainVerbContext(flags);+ const maxBatchBytes = parseCap(flags["max-bytes"], "--max-bytes", DEFAULT_MAX_BATCH_BYTES);+ const maxBatchFiles = parseCap(flags["max-files"], "--max-files", DEFAULT_MAX_BATCH_FILES);+ const plan = planBatches(vault, sourceDir, { maxBatchBytes, maxBatchFiles });++ if (flags["json"]) {+ okJson({+ source_dir: plan.sourceDir,+ max_batch_bytes: plan.maxBatchBytes,+ max_batch_files: plan.maxBatchFiles,+ total_files: plan.totalFiles,+ total_bytes: plan.totalBytes,+ skipped: [...plan.skipped],+ batches: plan.batches.map((b) => ({+ index: b.index,+ total_bytes: b.totalBytes,+ files: b.files.map((f) => ({ path: f.path, bytes: f.bytes, status: f.status })),+ })),+ });+ return 0;+ }++ ok(`batch-plan: ${plan.sourceDir}`);+ ok(+ ` ${plan.totalFiles} file(s) to ingest in ${plan.batches.length} batch(es); ` ++ `${plan.skipped.length} unchanged skipped`,+ );+ for (const b of plan.batches) {+ ok(` batch ${b.index}: ${b.files.length} file(s), ${b.totalBytes} byte(s)`);+ for (const f of b.files) ok(` - ${f.path} (${f.status}, ${f.bytes}B)`);+ }+ if (plan.skipped.length > 0) {+ info(` ${plan.skipped.length} unchanged file(s) skipped:`);+ for (const p of plan.skipped) info(` - ${p}`);+ }+ return 0;+ } catch (err) {+ return fail((err as Error).message ?? String(err));+ }+}diff --git a/src/cli/brain/verbs/forget-source.ts b/src/cli/brain/verbs/forget-source.tsnew file mode 100644index 00000000..a82ee05d--- /dev/null+++ b/src/cli/brain/verbs/forget-source.ts@@ -0,0 +1,97 @@+/**+ * `o2b brain forget-source <source>` (C6 / t_edde2198): find and surgically+ * remove everything derived from one EXACT source file.+ *+ * DRY-RUN BY DEFAULT β with no `--confirm` it prints the blast radius and+ * deletes nothing. `--confirm` deletes the single-purpose derived entries+ * and the ingest index artifacts (summary page + content-manifest entry);+ * shared-derivation pages and aggregate surfaces (logs, reports, MOCs) are+ * reported, never deleted. `--include-originals` additionally removes the+ * original source file(s) outside `Brain/`. Every confirmed cleanup writes+ * an auditable `source_invalidation` continuity record.+ */++import { deleteBySource } from "../../../core/brain/source-cleanup.ts";+import { brainVerbContext, fail, info, ok, okJson, parse } from "../helpers.ts";++export async function cmdBrainForgetSource(argv: string[]): Promise<number> {+ const { flags, positional } = parse(argv, {+ vault: { type: "string" },+ confirm: { type: "boolean" },+ "include-originals": { type: "boolean" },+ json: { type: "boolean" },+ });+ const sourceFile = positional[0];+ if (!sourceFile) {+ return fail(+ "usage: o2b brain forget-source <source> [--confirm] [--include-originals] [--json]",+ );+ }++ try {+ const { vault } = brainVerbContext(flags);+ const plan = deleteBySource(vault, sourceFile, {+ confirm: flags["confirm"] === true,+ includeOriginals: flags["include-originals"] === true,+ now: new Date(),+ });++ if (flags["json"]) {+ okJson({+ source: plan.source,+ confirmed: plan.confirmed,+ include_originals: plan.includeOriginals,+ blast_radius: plan.blastRadius,+ derived: plan.derived.map((e) => ({+ path: e.path,+ kind: e.kind,+ match: e.match,+ is_index_artifact: e.isIndexArtifact,+ })),+ mentions: plan.mentions.map((e) => ({ path: e.path, kind: e.kind, match: e.match })),+ originals: [...plan.originals],+ deleted: [...plan.deleted],+ manifest_entry_removed: plan.manifestEntryRemoved,+ audit_record_id: plan.auditRecordId,+ });+ return 0;+ }++ if (plan.confirmed) {+ ok(`forget-source: ${plan.source}`);+ ok(` deleted ${plan.deleted.length} file(s)`);+ for (const p of plan.deleted) ok(` - ${p}`);+ if (plan.manifestEntryRemoved) ok(" removed ingest manifest entry");+ if (plan.mentions.length > 0) {+ info(` ${plan.mentions.length} protected mention(s) NOT deleted (edit by hand):`);+ for (const e of plan.mentions) info(` - ${e.path} (${e.kind})`);+ }+ if (!plan.includeOriginals && plan.originals.length > 0) {+ info(` ${plan.originals.length} original(s) preserved (pass --include-originals):`);+ for (const p of plan.originals) info(` - ${p}`);+ }+ if (plan.auditRecordId) ok(` audit: ${plan.auditRecordId}`);+ return 0;+ }++ // Dry-run report.+ ok(`forget-source (DRY RUN): ${plan.source}`);+ ok(` blast radius: ${plan.blastRadius} referenced file(s) + originals`);+ ok(` ${plan.derived.length} derived entr(y/ies) WOULD be deleted:`);+ for (const e of plan.derived) {+ ok(` - ${e.path} (${e.kind}${e.isIndexArtifact ? ", index artifact" : ""})`);+ }+ if (plan.mentions.length > 0) {+ info(` ${plan.mentions.length} protected mention(s) reported, NOT deleted:`);+ for (const e of plan.mentions) info(` - ${e.path} (${e.kind})`);+ }+ if (plan.originals.length > 0) {+ info(` ${plan.originals.length} original(s) (removed only with --include-originals):`);+ for (const p of plan.originals) info(` - ${p}`);+ }+ info(" re-run with --confirm to delete");+ return 0;+ } catch (err) {+ return fail((err as Error).message ?? String(err));+ }+}diff --git a/src/cli/brain/verbs/import-session.ts b/src/cli/brain/verbs/import-session.tsindex 49634c92..4a575b97 100644--- a/src/cli/brain/verbs/import-session.ts+++ b/src/cli/brain/verbs/import-session.ts@@ -34,6 +34,7 @@ export async function cmdBrainImportSession(argv: string[]): Promise<number> {"ingest-scope": { type: "string" },"filter-role": { type: "string-array" },"filter-text": { type: "string" },+ "preserve-event-time": { type: "boolean" },json: { type: "boolean" },});if (positional.length < 1) return fail("brain import-session requires a <path> argument");@@ -65,6 +66,7 @@ export async function cmdBrainImportSession(argv: string[]): Promise<number> {}}const filterText = normalizeFlagString(flags["filter-text"] as string | undefined);+ const preserveEventTime = Boolean(flags["preserve-event-time"]);const formatRaw = flags["format"] as string | undefined;let format: SessionAdapterId | undefined;@@ -99,6 +101,7 @@ export async function cmdBrainImportSession(argv: string[]): Promise<number> {...(ingestScope !== null ? { ingestScope } : {}),...(filterRoles.length > 0 ? { filterRoles } : {}),...(filterText !== null ? { filterTextIncludes: filterText } : {}),+ ...(preserveEventTime ? { preserveEventTime } : {}),}): {files: [@@ -115,6 +118,7 @@ export async function cmdBrainImportSession(argv: string[]): Promise<number> {...(ingestScope !== null ? { ingestScope } : {}),...(filterRoles.length > 0 ? { filterRoles } : {}),...(filterText !== null ? { filterTextIncludes: filterText } : {}),+ ...(preserveEventTime ? { preserveEventTime } : {}),}),],warnings: [],diff --git a/src/cli/brain/verbs/index.ts b/src/cli/brain/verbs/index.tsindex fb90913b..65d84151 100644--- a/src/cli/brain/verbs/index.ts+++ b/src/cli/brain/verbs/index.ts@@ -60,6 +60,8 @@ export { cmdBrainHandoff } from "./handoff.ts";export { cmdBrainIntention } from "./intention.ts";export { cmdBrainProject } from "./project.ts";export { cmdBrainSource } from "./source.ts";+export { cmdBrainForgetSource } from "./forget-source.ts";+export { cmdBrainBatchPlan } from "./batch-plan.ts";export { cmdBrainLinks } from "./links.ts";export { cmdBrainProfile } from "./profile.ts";export { cmdBrainSgrep } from "./sgrep.ts";diff --git a/src/cli/brain/verbs/pre-compact-extract.ts b/src/cli/brain/verbs/pre-compact-extract.tsindex 5dfbd8a3..7a5419b6 100644--- a/src/cli/brain/verbs/pre-compact-extract.ts+++ b/src/cli/brain/verbs/pre-compact-extract.ts@@ -12,6 +12,7 @@ export async function cmdBrainPreCompactExtract(argv: string[]): Promise<number>text: { type: "string" },host: { type: "string" },"max-chars": { type: "string" },+ "dry-run": { type: "boolean" },});const vault = stringRequired(flags["vault"], "--vault");const sessionId = stringRequired(flags["session-id"], "--session-id");@@ -26,8 +27,12 @@ export async function cmdBrainPreCompactExtract(argv: string[]): Promise<number>text,...(stringOptional(flags["host"]) !== undefined ? { host: stringOptional(flags["host"]) } : {}),...(maxChars !== undefined ? { maxChars: positiveInteger(maxChars, "--max-chars") } : {}),+ ...(flags["dry-run"] === true ? { dryRun: true } : {}),});- writeOutput({ count: result.records.length, ...result }, flags["json"] === true);+ writeOutput(+ { count: result.records.length, dry_run: flags["dry-run"] === true, ...result },+ flags["json"] === true,+ );return 0;}diff --git a/src/cli/command-manifest.ts b/src/cli/command-manifest.tsindex e5f23f65..28c83e46 100644--- a/src/cli/command-manifest.ts+++ b/src/cli/command-manifest.ts@@ -80,6 +80,20 @@ export const CLI_COMMAND_MANIFEST: CliRootManifest = Object.freeze({flag("vault", "string"),flag("tool-arg", "string-array"),]),+ command(+ "aider",+ "Session-bracketing memory wrapper for Aider",+ [],+ [+ command("wrap", "Run Aider bracketed with live memory load + write-back", [+ flag("session-id", "string"),+ flag("aider-bin", "string"),+ flag("chat-history", "string"),+ flag("vault", "string"),+ flag("config", "string"),+ ]),+ ],+ ),command("brain","Brain memory verbs",@@ -181,6 +195,26 @@ export const CLI_COMMAND_MANIFEST: CliRootManifest = Object.freeze({command("intention", "Manage scoped current-intention chains"),command("project", "Link project directories to their owning vault"),command("source", "Manage read-only recall sources of the active vault"),+ command(+ "forget-source",+ "Find and delete entries derived from an exact source file (dry-run by default)",+ [+ flag("vault", "string"),+ flag("confirm", "boolean"),+ flag("include-originals", "boolean"),+ flag("json", "boolean"),+ ],+ ),+ command(+ "batch-plan",+ "Plan a large-folder ingest into bounded parallel batches (skips unchanged sources)",+ [+ flag("vault", "string"),+ flag("max-bytes", "string"),+ flag("max-files", "string"),+ flag("json", "boolean"),+ ],+ ),command("links", "Normalize wikilink path format across Brain notes"),command("bridges", "Propose, accept, or dismiss embedding-near bridge links"),command("clusters", "Detect link-graph communities and materialize cluster notes"),diff --git a/src/cli/main.ts b/src/cli/main.tsindex 2bc830bf..7ed4f1c8 100644--- a/src/cli/main.ts+++ b/src/cli/main.ts@@ -25,6 +25,7 @@ import { ensureVaultCurrent } from "../core/maintenance/ensure-current.ts";import { doctor } from "../core/doctor.ts";import { listVaultPages, writeFrontmatter } from "../core/vault.ts";import { CliError, parseFlags } from "./argparse.ts";+import { handleAiderSubcommand } from "./aider.ts";import { handleBrainSubcommand } from "./brain.ts";import { handleDisciplineSubcommand } from "./discipline.ts";import { handlePartnerSubcommand } from "./partner.ts";@@ -711,6 +712,9 @@ Commands:help Print this help text; --json prints command metadatacompletions Print shell completions for bash, zsh, fish, elvish, nushell, powershell+Aider (session-bracketing memory wrapper):+ aider wrap Run Aider bracketed with live memory load + write-back+Brain (observing memory):brain init Bootstrap <vault>/Brain/ skeleton (idempotent)brain feedback Record a taste signal into Brain/inbox/@@ -757,6 +761,7 @@ export async function main(argv: ReadonlyArray<string>): Promise<number> {if (rest.length === 1 &&(rest[0] === "-h" || rest[0] === "--help") &&+ command !== "aider" &&command !== "brain" &&command !== "vault") {@@ -835,6 +840,8 @@ async function dispatchCommand(command: string, rest: string[]): Promise<number>return cmdHelp(rest);case "completions":return cmdCompletions(rest);+ case "aider":+ return await handleAiderSubcommand(rest);case "brain":return await handleBrainSubcommand(rest);case "discipline":diff --git a/src/core/brain/apply-evidence.ts b/src/core/brain/apply-evidence.tsindex 9ce2b213..375e2eae 100644--- a/src/core/brain/apply-evidence.ts+++ b/src/core/brain/apply-evidence.ts@@ -30,8 +30,15 @@*/import { existsSync } from "node:fs";+import { join, relative } from "node:path";import { sanitiseTextField } from "../redactor.ts";+import {+ computePayloadHash,+ IdempotencyPayloadMismatchError,+ lookupKey,+ rememberKey,+} from "./idempotency-ledger.ts";import { appendLogEvent, type AppendLogEventResult, type BrainLogEntry } from "./log.ts";import { parsePreference } from "./preference.ts";import { preferencePath, validateSlug } from "./paths.ts";@@ -109,6 +116,16 @@ export interface AppendApplyEvidenceInput {* absent outcome, only `success`/`failure` persist in the log.*/readonly outcome?: BrainApplyOutcome;+ /**+ * Client-supplied idempotency key (C1 / t_213f356b). Additive and+ * optional: absent β byte-identical to the historical append path.+ * When present, the appender hashes the semantic payload (pref id,+ * artifact, result, agent, note, outcome) and consults the idempotency+ * ledger before appending. A repeat with the same key + same payload is+ * a deduped no-op (no second log row); the same key + a different+ * payload throws {@link IdempotencyPayloadMismatchError}.+ */+ readonly idempotency_key?: string;}export interface AppendApplyEvidenceOptions {@@ -131,6 +148,13 @@ export interface AppendApplyEvidenceResult {readonly logged_at: string;/** Absolute path of the log file the event landed in. */readonly log_path: string;+ /**+ * Set to `true` when an `idempotency_key` matched a prior append with an+ * identical payload, so this call was a deduped no-op (no second row).+ * The returned `logged_at`/`log_path` point at the ORIGINAL event.+ * Absent on a normal append.+ */+ readonly deduped?: boolean;}/**@@ -232,11 +256,44 @@ export function appendApplyEvidence(const trimmedNote = note?.trim();if (trimmedNote) body["note"] = trimmedNote;+ // Idempotency consult (C1): a repeat with the same key + same payload+ // dedupes (no second row); the same key + a different payload throws+ // before appending. The hash covers the rendered event body β exactly+ // what would land in the log.+ const idKey = input.idempotency_key?.trim();+ let idContentHash: string | undefined;+ if (idKey) {+ idContentHash = computePayloadHash(body);+ const existing = lookupKey(vault, idKey);+ if (existing) {+ if (existing.contentHash === idContentHash) {+ const ref = (existing.ref ?? {}) as { logged_at?: string; log_path?: string };+ return {+ logged_at: ref.logged_at ?? timestamp,+ log_path: ref.log_path ? join(vault, ref.log_path) : "",+ deduped: true,+ };+ }+ throw new IdempotencyPayloadMismatchError(idKey, existing.contentHash, idContentHash);+ }+ }+const entry: BrainLogEntry = {timestamp,eventType: BRAIN_LOG_EVENT_KIND.applyEvidence,body,};const result: AppendLogEventResult = appendLogEvent(vault, entry);++ // Record the key AFTER the row lands so a future retry dedupes.+ if (idKey && idContentHash !== undefined) {+ rememberKey(vault, {+ key: idKey,+ contentHash: idContentHash,+ createdAt: timestamp,+ ref: { logged_at: timestamp, log_path: relative(vault, result.logPath) },+ });+ }+return { logged_at: timestamp, log_path: result.logPath };}diff --git a/src/core/brain/continuity/store.ts b/src/core/brain/continuity/store.tsindex a1649717..ae18a95b 100644--- a/src/core/brain/continuity/store.ts+++ b/src/core/brain/continuity/store.ts@@ -100,7 +100,18 @@ export function appendContinuityRecord(vault: string,input: AppendContinuityRecordInput,): ContinuityRecord {- return appendRecord(vault, buildRecord(input));+ return appendRecord(vault, buildContinuityRecord(input));+}++/**+ * Build the exact continuity record {@link appendContinuityRecord} would+ * persist, WITHOUT touching disk. Same validation, payload sanitisation,+ * and dedup id as the append path β so a caller can preview a record and+ * trust it byte-for-byte predicts the real write. Used by the dry-run+ * extraction preview (C2 / t_2c6cf3e2).+ */+export function buildContinuityRecord(input: AppendContinuityRecordInput): ContinuityRecord {+ return buildRecord(input);}/**diff --git a/src/core/brain/continuity/types.ts b/src/core/brain/continuity/types.tsindex fc6aba13..ff671455 100644--- a/src/core/brain/continuity/types.ts+++ b/src/core/brain/continuity/types.ts@@ -16,6 +16,7 @@ export type ContinuityRecordKind =| "pre_compact_extract"| "post_compact_audit"| "session_turn"+ | "recent_turn"| "session_summary_node"| "session_summary_digest"| "generation_report"diff --git a/src/core/brain/expiration.ts b/src/core/brain/expiration.tsnew file mode 100644index 00000000..a20703a9--- /dev/null+++ b/src/core/brain/expiration.ts@@ -0,0 +1,115 @@+/**+ * Caller-settable per-memory expiration (C5 / t_a82b674e).+ *+ * A caller can stamp an explicit `expiration_date` on a preference or+ * signal at write time. The default read/list path silently drops+ * memories past their date; an opt-in `showExpired` flag re-includes+ * them for audit. This is orthogonal to dream's heuristic retirement β+ * an expired-by-date memory is FILTERED on read, never moved to+ * `Brain/retired/` (the audit trail is preserved on disk).+ *+ * LLM-free and deterministic: the whole feature is a frontmatter date+ * plus a date comparison. Two granularities are supported:+ *+ * - a date-only `YYYY-MM-DD` value keeps the memory live through the+ * END of that UTC day, so "use the staging endpoint until+ * 2026-07-15" stays live all of the 15th and lapses on the 16th;+ * - a full ISO-8601 timestamp expires at that exact instant.+ */++/** Frontmatter key carrying the caller-supplied expiration. */+export const EXPIRATION_DATE_FIELD = "expiration_date";++/** `YYYY-MM-DD` with no time component. */+const DATE_ONLY_RE = /^\d{4}-\d{2}-\d{2}$/;++/**+ * Validate + normalise a caller-supplied expiration date. Accepts a+ * date-only `YYYY-MM-DD` or a full ISO-8601 timestamp; returns the+ * trimmed value. Throws on an empty or unparseable value so a bad date+ * is rejected at write time rather than silently hiding a memory (or+ * silently never expiring one) on read.+ */+export function normalizeExpirationDate(raw: string): string {+ const trimmed = raw.trim();+ if (trimmed === "") {+ throw new Error("expiration_date must not be empty");+ }+ if (expirationBoundaryMs(trimmed) === null) {+ throw new Error(+ `expiration_date must be a YYYY-MM-DD date or ISO-8601 timestamp; got ${JSON.stringify(raw)}`,+ );+ }+ return trimmed;+}++/**+ * The last instant (epoch ms) at which a memory carrying this expiration+ * is still live. Date-only β end of that UTC day; full timestamp β the+ * parsed instant. Returns `null` when the value cannot be parsed as+ * either β the caller decides whether that is a hard error (write) or a+ * fail-open (read).+ */+function expirationBoundaryMs(expiration: string): number | null {+ const value = expiration.trim();+ const dateOnly = DATE_ONLY_RE.exec(value);+ if (dateOnly) {+ const year = Number(value.slice(0, 4));+ const month = Number(value.slice(5, 7));+ const day = Number(value.slice(8, 10));+ // Reject impossible calendar dates (e.g. 2026-13-40) β Date.UTC+ // would otherwise roll them over into an adjacent month.+ const utc = new Date(Date.UTC(year, month - 1, day));+ if (+ utc.getUTCFullYear() !== year ||+ utc.getUTCMonth() !== month - 1 ||+ utc.getUTCDate() !== day+ ) {+ return null;+ }+ return Date.UTC(year, month - 1, day, 23, 59, 59, 999);+ }+ const parsed = Date.parse(value);+ return Number.isNaN(parsed) ? null : parsed;+}++/**+ * True when a memory carrying `expiration_date` is past its expiration+ * relative to `now`. A value that cannot be parsed fails OPEN (returns+ * `false` β not expired) so a hand-corrupted date never silently hides a+ * memory; surfacing that corruption is the doctor's job, not this+ * filter's.+ */+export function isExpired(expiration_date: string, now: Date): boolean {+ const boundary = expirationBoundaryMs(expiration_date);+ if (boundary === null) return false;+ return now.getTime() > boundary;+}++/** A memory that may carry an optional expiration date. */+export interface ExpirableMemory {+ readonly expiration_date?: string;+}++export interface FilterExpiredOptions {+ /** Wall clock the expiration is compared against. Defaults to `new Date()`. */+ readonly now?: Date;+ /** When true, expired memories are kept (audit / recall of lapsed memories). */+ readonly showExpired?: boolean;+}++/**+ * Drop memories past their `expiration_date`. Memories with no+ * `expiration_date` are always kept. When `showExpired` is set, nothing+ * is dropped (the full list is returned, order-preserved).+ */+export function filterExpired<T extends ExpirableMemory>(+ items: ReadonlyArray<T>,+ options: FilterExpiredOptions = {},+): T[] {+ if (options.showExpired) return [...items];+ const now = options.now ?? new Date();+ return items.filter(+ (item) => item.expiration_date === undefined || !isExpired(item.expiration_date, now),+ );+}diff --git a/src/core/brain/health/contradiction.ts b/src/core/brain/health/contradiction.tsindex 5ae31317..613c7644 100644--- a/src/core/brain/health/contradiction.ts+++ b/src/core/brain/health/contradiction.ts@@ -50,6 +50,196 @@ export interface DetectContradictionsOptions {readonly jaccard: number;}+/**+ * A starter, deliberately multilingual negation lexicon.+ *+ * The confirmed-preference detector reads polarity from a structured+ * `signal` field and never inspects text, so it needs no lexicon. Note+ * prose carries no such field, so the stance sign must be derived from+ * the prose itself - and the only language-agnostic structural marker of+ * a reversed stance is a negation particle inserted into an otherwise+ * shared claim.+ *+ * This set is a documented DEFAULT, not a baked-in assumption: callers+ * pass `opts.negationMarkers` to override or extend it for their own+ * languages (mirroring `similarity.ts`, which keeps its stopword set+ * empty rather than bias toward English). It spans several space-+ * delimited scripts; single-glyph CJK negations are out of reach of the+ * whitespace tokeniser and left to a caller-supplied lexicon.+ */+export const DEFAULT_NEGATION_MARKERS: ReadonlySet<string> = new Set([+ // English+ "not",+ "no",+ "never",+ "none",+ "neither",+ "nor",+ "cannot",+ "without",+ // Romance (es / fr / pt / it)+ "non",+ "ne",+ "pas",+ "nunca",+ "sin",+ "sem",+ "senza",+ "sans",+ "nΓ£o",+ "nao",+ // German+ "nicht",+ "kein",+ "keine",+ "nie",+ "ohne",+ // Russian+ "Π½Π΅",+ "Π½Π΅Ρ",+ "Π½ΠΈ",+]);++/**+ * Narrow projection of a permanent note the detector needs. The caller+ * is responsible for restricting the input to permanent notes; the+ * detector operates on whatever prose it is handed.+ */+export interface NoteForContradiction {+ readonly id: string;+ /**+ * Optional subject bucket. When present, only notes sharing a subject+ * are compared (mirrors `scope` in the preference detector); absent+ * notes fall into a single bucket and are paired purely by prose+ * token overlap.+ */+ readonly subject?: string;+ readonly text: string;+}++export interface NoteContradictionFinding {+ readonly aId: string;+ readonly bId: string;+ /** Sorted shared subject tokens, space-joined, for display. */+ readonly subject: string;+ readonly jaccard: number;+ readonly aSign: BrainSignalSign;+ readonly bSign: BrainSignalSign;+ /** The subject-bearing span quoted verbatim from note A. */+ readonly aQuote: string;+ /** The subject-bearing span quoted verbatim from note B. */+ readonly bQuote: string;+ /** Always `ask_user`: contradictions are surfaced, never auto-resolved. */+ readonly action: "ask_user";+}++export interface DetectNoteContradictionsOptions {+ /** Minimum prose jaccard for two notes to count as the same subject. */+ readonly jaccard: number;+ /**+ * Negation markers used to derive each note's stance sign from prose.+ * Defaults to {@link DEFAULT_NEGATION_MARKERS}; pass a custom set to+ * override or extend it for a given language.+ */+ readonly negationMarkers?: ReadonlySet<string>;+}++/**+ * Derive a note's stance sign structurally from its prose: a note whose+ * tokens include any negation marker asserts the negative side of its+ * subject; otherwise it reads as positive. No sentiment analysis and no+ * per-language vocabulary beyond the injected marker set.+ */+export function deriveNoteStance(+ tokens: ReadonlySet<string>,+ negationMarkers: ReadonlySet<string>,+): BrainSignalSign {+ for (const t of tokens) {+ if (negationMarkers.has(t)) return "negative";+ }+ return "positive";+}++/**+ * Split prose into candidate spans on sentence terminators (Latin and+ * CJK) and hard line breaks, then return the span with the greatest+ * overlap with `subjectTokens`. Ties break to the earliest span; a note+ * with no scoring span falls back to its first non-empty span, or the+ * trimmed whole text. Deterministic for a given input.+ */+export function extractSpan(text: string, subjectTokens: ReadonlySet<string>): string {+ const segments = text+ .split(/(?<=[.!?γοΌοΌ])\s+|\n+/u)+ .map((s) => s.trim())+ .filter((s) => s.length > 0);+ if (segments.length === 0) return text.trim();+ let best = segments[0]!;+ let bestScore = -1;+ for (const segment of segments) {+ const segTokens = tokenise(segment);+ let score = 0;+ for (const t of subjectTokens) if (segTokens.has(t)) score++;+ if (score > bestScore) {+ bestScore = score;+ best = segment;+ }+ }+ return best;+}++/**+ * Detect contradictory note-position pairs over permanent-note prose.+ * Notes are bucketed by `subject` (single bucket when absent), paired by+ * prose token similarity at or above `jaccard`, and kept only when the+ * two stance signs - derived structurally from each note's prose -+ * disagree. Each finding quotes the subject-bearing span from both notes+ * and carries `action: "ask_user"`: the pair is surfaced for the+ * operator to clarify, never auto-resolved.+ */+export function detectNoteContradictions(+ notes: ReadonlyArray<NoteForContradiction>,+ opts: DetectNoteContradictionsOptions,+): NoteContradictionFinding[] {+ const negationMarkers = opts.negationMarkers ?? DEFAULT_NEGATION_MARKERS;+ const signById = new Map<string, BrainSignalSign>();+ const entries = [];+ for (const n of notes) {+ const tokens = tokenise(n.text);+ signById.set(n.id, deriveNoteStance(tokens, negationMarkers));+ entries.push({+ id: n.id,+ bucketKey: n.subject ?? "",+ tokens,+ source: n,+ });+ }++ const pairs = findSimilarPairs(entries, { threshold: opts.jaccard });+ const out: NoteContradictionFinding[] = [];+ for (const pair of pairs) {+ const aSign = signById.get(pair.a.id)!;+ const bSign = signById.get(pair.b.id)!;+ if (aSign === bSign) continue;+ const shared: string[] = [];+ for (const t of pair.a.tokens) if (pair.b.tokens.has(t)) shared.push(t);+ shared.sort();+ const sharedSet = new Set(shared);+ out.push({+ aId: pair.a.id,+ bId: pair.b.id,+ subject: shared.join(" "),+ jaccard: pair.jaccard,+ aSign,+ bSign,+ aQuote: extractSpan(pair.a.source.text, sharedSet),+ bQuote: extractSpan(pair.b.source.text, sharedSet),+ action: "ask_user",+ });+ }+ out.sort((x, y) => x.aId.localeCompare(y.aId) || x.bId.localeCompare(y.bId));+ return out;+}+/*** Detect contradictory confirmed-preference pairs. Preferences are* bucketed by scope (so only same-scope rules are compared), paired bydiff --git a/src/core/brain/health/thesis.ts b/src/core/brain/health/thesis.tsnew file mode 100644index 00000000..59f873a0--- /dev/null+++ b/src/core/brain/health/thesis.ts@@ -0,0 +1,574 @@+/**+ * Declared-thesis register with a new-note support/contradiction monitor (D3).+ *+ * The operator records STANDING POSITIONS - a statement, a supporting- and+ * counter-evidence summary, a review cadence, and a falsification field+ * ("what would make me wrong"). Each thesis is a Markdown page under+ * `Brain/theses/thesis-<slug>.md`: operator-readable in Obsidian,+ * greppable, versionable - mirroring the `obligations.ts` page model.+ *+ * The monitor evaluates each newly-ingested note against the ACTIVE+ * theses and raises one of three flags, all `ask_user` (never+ * auto-resolved):+ *+ * - `contradict` - the note asserts the OPPOSITE stance on the thesis+ * subject. This is note-vs-declared-position, distinct from D2's+ * note-vs-note detector, but built on the same structural,+ * language-agnostic stance derivation (a negation marker among the+ * note's tokens flips its sign).+ * - `support` - the note closely restates the thesis's own claim+ * (same stance AND high token overlap). A same-subject same-stance+ * note that merely ADDS COMPLEXITY (overlap below the support+ * threshold) is deliberately suppressed, per the article ("Don't flag+ * things that merely add complexity").+ * - `falsification` - the note's prose matches the thesis's documented+ * failure scenario (high coverage of the falsification's DISTINCTIVE+ * tokens - those not already in the statement). The article's "flag it+ * when live data starts matching the failure scenario you described".+ *+ * Two lifecycle checks reuse the shared cadence machinery:+ * - {@link detectStaleTheses} flags an active thesis whose `last_updated`+ * is past its cadence's next-due date (via `obligations.ts`+ * `nextDueDate`).+ * - {@link detectThesisGraveyard} flags an active thesis with no+ * supporting evidence in the last N days (mirroring `stale-claim.ts`),+ * suggesting a formal close.+ *+ * The kernel never calls a model: stance, overlap, and cadence arithmetic+ * are pure and deterministic, so the same inputs yield the same findings+ * on every Syncthing peer.+ */++import { existsSync, mkdirSync, readdirSync } from "node:fs";++import { atomicWriteFileSync } from "../../fs-atomic.ts";+import { parseFrontmatter, slugify } from "../../vault.ts";+import { nextDueDate, parseCadence, type ObligationCadence } from "../obligations.ts";+import { thesesDir, thesisPath, validateIsoDate } from "../paths.ts";+import { isoDate, isoSecond } from "../time.ts";+import { type BrainSignalSign } from "../types.ts";+import {+ DEFAULT_NEGATION_MARKERS,+ deriveNoteStance,+ extractSpan,+ type NoteForContradiction,+} from "./contradiction.ts";+import { jaccard, tokenise } from "../similarity.ts";++const DAY_MS = 24 * 60 * 60 * 1000;+const ISO_DATE_ONLY_RE = /^\d{4}-\d{2}-\d{2}$/;++/** A thesis is either an active standing position or formally closed. */+export const THESIS_STATUS = {+ active: "active",+ closed: "closed",+} as const;+export type ThesisStatus = (typeof THESIS_STATUS)[keyof typeof THESIS_STATUS];++export class ThesisError extends Error {}++export interface ThesisPage {+ readonly slug: string;+ /** The declared position, quoted verbatim when a finding cites it. */+ readonly statement: string;+ readonly supportingEvidence: string;+ readonly counterEvidence: string;+ /** "What would make me wrong" - the documented failure scenario. */+ readonly falsification: string;+ readonly status: ThesisStatus;+ /** Review cadence driving the staleness check. */+ readonly cadence: ObligationCadence;+ readonly createdAt: string;+ /** Last time the operator revised the thesis (`YYYY-MM-DD`). */+ readonly lastUpdated: string;+ /** Last time supporting evidence landed, or null if never. */+ readonly lastSupportAt: string | null;+ readonly agent: string;+ readonly notes: string;+ readonly path: string;+}++export interface RecordThesisInput {+ readonly statement: string;+ readonly supportingEvidence?: string;+ readonly counterEvidence?: string;+ readonly falsification?: string;+ /** Review cadence; defaults to `monthly`. */+ readonly cadence?: string;+ readonly agent: string;+ readonly notes?: string;+ /** Overrides `last_updated` (defaults to today, UTC). */+ readonly lastUpdated?: string;+ /** Seeds `last_support_at` (defaults to null). */+ readonly lastSupportAt?: string;+ readonly now?: Date;+}++export interface UpdateThesisInput {+ readonly slug: string;+ readonly statement?: string;+ readonly supportingEvidence?: string;+ readonly counterEvidence?: string;+ readonly falsification?: string;+ readonly cadence?: string;+ readonly notes?: string;+ readonly now?: Date;+}++// ----- Body-section (de)serialization --------------------------------------+//+// The statement and the machine fields live in frontmatter; the free-form+// prose fields live as body sections so multi-line operator notes survive a+// round-trip without straining the flat frontmatter parser.++const SECTION_SUPPORTING = "Supporting evidence";+const SECTION_COUNTER = "Counter-evidence";+const SECTION_FALSIFICATION = "Falsification";+const SECTION_NOTES = "Notes";++function renderBody(+ page: Pick<ThesisPage, "supportingEvidence" | "counterEvidence" | "falsification" | "notes">,+): string {+ const sections: Array<[string, string]> = [+ [SECTION_SUPPORTING, page.supportingEvidence],+ [SECTION_COUNTER, page.counterEvidence],+ [SECTION_FALSIFICATION, page.falsification],+ [SECTION_NOTES, page.notes],+ ];+ const parts: string[] = [];+ for (const [header, content] of sections) {+ const trimmed = content.trim();+ if (trimmed.length === 0) continue;+ parts.push(`## ${header}\n\n${trimmed}`);+ }+ return parts.join("\n\n");+}++/** Split a rendered body back into its `## `-headed sections. */+function parseSections(body: string): Map<string, string> {+ const out = new Map<string, string>();+ const lines = body.split("\n");+ let header: string | null = null;+ let buffer: string[] = [];+ const flush = () => {+ if (header !== null) out.set(header, buffer.join("\n").trim());+ buffer = [];+ };+ for (const line of lines) {+ const match = /^##\s+(.*)$/u.exec(line.trim());+ if (match) {+ flush();+ header = match[1]!.trim();+ } else if (header !== null) {+ buffer.push(line);+ }+ }+ flush();+ return out;+}++function render(page: Omit<ThesisPage, "path">): string {+ const lines = [+ "---",+ `statement: ${JSON.stringify(page.statement)}`,+ `status: ${page.status}`,+ `cadence: ${page.cadence}`,+ `created_at: ${page.createdAt}`,+ `last_updated: ${page.lastUpdated}`,+ `last_support_at: ${page.lastSupportAt ?? ""}`,+ `agent: ${JSON.stringify(page.agent)}`,+ "---",+ "",+ renderBody(page),+ "",+ ];+ return lines.join("\n");+}++function parseIsoOrEmpty(value: unknown): string {+ if (typeof value !== "string") return "";+ try {+ return validateIsoDate(value.trim());+ } catch {+ return "";+ }+}++function parseStatus(raw: unknown): ThesisStatus {+ return raw === THESIS_STATUS.closed ? THESIS_STATUS.closed : THESIS_STATUS.active;+}++function parsePage(vault: string, slug: string): ThesisPage | null {+ const path = thesisPath(vault, slug);+ if (!existsSync(path)) return null;+ const [meta, body] = parseFrontmatter(path);+ let cadence: ObligationCadence;+ try {+ cadence = parseCadence(typeof meta["cadence"] === "string" ? meta["cadence"] : "");+ } catch {+ cadence = "monthly";+ }+ const sections = parseSections(body);+ const lastSupport = parseIsoOrEmpty(meta["last_support_at"]);+ return Object.freeze({+ slug,+ statement: typeof meta["statement"] === "string" ? meta["statement"] : "",+ supportingEvidence: sections.get(SECTION_SUPPORTING) ?? "",+ counterEvidence: sections.get(SECTION_COUNTER) ?? "",+ falsification: sections.get(SECTION_FALSIFICATION) ?? "",+ status: parseStatus(meta["status"]),+ cadence,+ createdAt: typeof meta["created_at"] === "string" ? meta["created_at"] : "",+ lastUpdated: parseIsoOrEmpty(meta["last_updated"]),+ lastSupportAt: lastSupport.length > 0 ? lastSupport : null,+ agent: typeof meta["agent"] === "string" ? meta["agent"] : "",+ notes: sections.get(SECTION_NOTES) ?? "",+ path,+ });+}++/** Create a declared-thesis page. Refuses to clobber an existing slug. */+export function recordThesis(vault: string, input: RecordThesisInput): ThesisPage {+ const statement = input.statement.trim();+ if (statement.length === 0) throw new ThesisError("thesis statement must not be empty");+ const cadence = parseCadence(input.cadence ?? "monthly");+ const now = input.now ?? new Date();+ const slug = slugify(statement);+ if (existsSync(thesisPath(vault, slug))) {+ throw new ThesisError(`thesis already exists: ${slug} (remove it first to recreate)`);+ }+ const page = {+ slug,+ statement,+ supportingEvidence: (input.supportingEvidence ?? "").trim(),+ counterEvidence: (input.counterEvidence ?? "").trim(),+ falsification: (input.falsification ?? "").trim(),+ status: THESIS_STATUS.active,+ cadence,+ createdAt: isoSecond(now),+ lastUpdated: input.lastUpdated ? validateIsoDate(input.lastUpdated) : isoDate(now),+ lastSupportAt: input.lastSupportAt ? validateIsoDate(input.lastSupportAt) : null,+ agent: input.agent,+ notes: (input.notes ?? "").trim(),+ };+ mkdirSync(thesesDir(vault), { recursive: true });+ const path = thesisPath(vault, slug);+ atomicWriteFileSync(path, render(page));+ return Object.freeze({ ...page, path });+}++/** Revise a thesis; bumps `last_updated` to now (or the injected clock). */+export function updateThesis(vault: string, input: UpdateThesisInput): ThesisPage {+ const slug = slugify(input.slug);+ const prior = parsePage(vault, slug);+ if (prior === null) throw new ThesisError(`no thesis: ${slug}`);+ const now = input.now ?? new Date();+ const page = {+ slug: prior.slug,+ statement: input.statement?.trim() || prior.statement,+ supportingEvidence:+ input.supportingEvidence !== undefined+ ? input.supportingEvidence.trim()+ : prior.supportingEvidence,+ counterEvidence:+ input.counterEvidence !== undefined ? input.counterEvidence.trim() : prior.counterEvidence,+ falsification:+ input.falsification !== undefined ? input.falsification.trim() : prior.falsification,+ status: prior.status,+ cadence: input.cadence ? parseCadence(input.cadence) : prior.cadence,+ createdAt: prior.createdAt,+ lastUpdated: isoDate(now),+ lastSupportAt: prior.lastSupportAt,+ agent: prior.agent,+ notes: input.notes !== undefined ? input.notes.trim() : prior.notes,+ };+ atomicWriteFileSync(prior.path, render(page));+ return Object.freeze({ ...page, path: prior.path });+}++/**+ * Record that supporting evidence landed for a thesis: advances+ * `last_support_at` (resetting the graveyard clock) and `last_updated`.+ */+export function recordThesisSupport(+ vault: string,+ slug: string,+ opts: { date?: string; now?: Date } = {},+): ThesisPage {+ const normalized = slugify(slug);+ const prior = parsePage(vault, normalized);+ if (prior === null) throw new ThesisError(`no thesis: ${normalized}`);+ const now = opts.now ?? new Date();+ const date = opts.date ? validateIsoDate(opts.date) : isoDate(now);+ const page = { ...prior, lastSupportAt: date, lastUpdated: isoDate(now) };+ const { path: _path, ...rest } = page;+ atomicWriteFileSync(prior.path, render(rest));+ return Object.freeze(page);+}++/** Formally close a thesis (the graveyard's recommended action). */+export function closeThesis(vault: string, slug: string): ThesisPage {+ const normalized = slugify(slug);+ const prior = parsePage(vault, normalized);+ if (prior === null) throw new ThesisError(`no thesis: ${normalized}`);+ const page = { ...prior, status: THESIS_STATUS.closed };+ const { path: _path, ...rest } = page;+ atomicWriteFileSync(prior.path, render(rest));+ return Object.freeze(page);+}++/** One thesis, or null. */+export function showThesis(vault: string, slug: string): ThesisPage | null {+ return parsePage(vault, slugify(slug));+}++/** All theses sorted by slug; optionally only the active ones. */+export function listTheses(vault: string, options: { activeOnly?: boolean } = {}): ThesisPage[] {+ const dir = thesesDir(vault);+ if (!existsSync(dir)) return [];+ const out: ThesisPage[] = [];+ for (const name of readdirSync(dir)) {+ if (!name.endsWith(".md") || !name.startsWith("thesis-")) continue;+ const slug = name.replace(/^thesis-/u, "").replace(/\.md$/u, "");+ const page = parsePage(vault, slug);+ if (page === null) continue;+ if (options.activeOnly && page.status !== THESIS_STATUS.active) continue;+ out.push(page);+ }+ return out.toSorted((a, b) => a.slug.localeCompare(b.slug));+}++// ----- Monitor: new note vs declared positions -----------------------------++/**+ * Narrow projection the monitor needs. A full {@link ThesisPage}+ * satisfies it structurally, so callers pass parsed pages directly.+ */+export interface ThesisForMonitor {+ readonly slug: string;+ readonly status: ThesisStatus;+ readonly statement: string;+ readonly falsification?: string;+}++export type ThesisFindingKind = "support" | "contradict" | "falsification";++export interface ThesisNoteFinding {+ readonly thesisSlug: string;+ readonly noteId: string;+ readonly kind: ThesisFindingKind;+ /**+ * The deciding metric: subject jaccard for support/contradict, or the+ * distinctive-token coverage for falsification.+ */+ readonly score: number;+ readonly thesisStance: BrainSignalSign;+ readonly noteStance: BrainSignalSign;+ /** The declared position (or its failure scenario) quoted verbatim. */+ readonly thesisQuote: string;+ /** The subject-bearing span quoted verbatim from the incoming note. */+ readonly noteQuote: string;+ /** Always `ask_user`: findings are surfaced, never auto-resolved. */+ readonly action: "ask_user";+}++export interface MonitorThesesOptions {+ /** Minimum statement-vs-note jaccard for the note to address the thesis. */+ readonly subjectJaccard?: number;+ /** Jaccard at/above which a same-stance note counts as SUPPORT. */+ readonly supportJaccard?: number;+ /** Coverage of the falsification's distinctive tokens that raises an alert. */+ readonly falsificationCoverage?: number;+ /** Negation markers for structural stance derivation (mirrors D2). */+ readonly negationMarkers?: ReadonlySet<string>;+}++const DEFAULT_SUBJECT_JACCARD = 0.3;+const DEFAULT_SUPPORT_JACCARD = 0.6;+const DEFAULT_FALSIFICATION_COVERAGE = 0.6;++/** Directional coverage: fraction of `needle` tokens present in `haystack`. */+function coverage(needle: ReadonlySet<string>, haystack: ReadonlySet<string>): number {+ if (needle.size === 0) return 0;+ let hits = 0;+ for (const t of needle) if (haystack.has(t)) hits++;+ return hits / needle.size;+}++/**+ * Evaluate every note against every ACTIVE thesis, returning support /+ * contradiction / falsification findings. Deterministic and+ * language-agnostic (stance derived structurally from an injected marker+ * set). Never auto-resolves - all findings carry `action: "ask_user"`.+ */+export function monitorNotesAgainstTheses(+ theses: ReadonlyArray<ThesisForMonitor>,+ notes: ReadonlyArray<NoteForContradiction>,+ opts: MonitorThesesOptions = {},+): ThesisNoteFinding[] {+ const subjectJaccard = opts.subjectJaccard ?? DEFAULT_SUBJECT_JACCARD;+ const supportJaccard = opts.supportJaccard ?? DEFAULT_SUPPORT_JACCARD;+ const falsificationCoverage = opts.falsificationCoverage ?? DEFAULT_FALSIFICATION_COVERAGE;+ const markers = opts.negationMarkers ?? DEFAULT_NEGATION_MARKERS;++ const out: ThesisNoteFinding[] = [];+ for (const thesis of theses) {+ if (thesis.status !== THESIS_STATUS.active) continue;+ const statement = thesis.statement.trim();+ if (statement.length === 0) continue;+ const stmtTokens = tokenise(statement);+ if (stmtTokens.size === 0) continue;+ const thesisStance = deriveNoteStance(stmtTokens, markers);++ // Distinctive falsification tokens: those NOT already in the statement,+ // so the failure-scenario match cannot be triggered by generic subject+ // overlap.+ const falsification = (thesis.falsification ?? "").trim();+ const falsTokens = falsification.length > 0 ? tokenise(falsification) : new Set<string>();+ const distinctive = new Set<string>();+ for (const t of falsTokens) if (!stmtTokens.has(t)) distinctive.add(t);++ for (const note of notes) {+ // 1) Falsification: does the note match the documented failure scenario?+ if (distinctive.size > 0) {+ const falsSpan = extractSpan(note.text, falsTokens);+ const cov = coverage(distinctive, tokenise(falsSpan));+ if (cov >= falsificationCoverage) {+ out.push({+ thesisSlug: thesis.slug,+ noteId: note.id,+ kind: "falsification",+ score: cov,+ thesisStance,+ noteStance: deriveNoteStance(tokenise(falsSpan), markers),+ thesisQuote: falsification,+ noteQuote: falsSpan,+ action: "ask_user",+ });+ continue;+ }+ }++ // 2) Support / contradiction over the subject-bearing span.+ const span = extractSpan(note.text, stmtTokens);+ const spanTokens = tokenise(span);+ const subj = jaccard(stmtTokens, spanTokens);+ if (subj < subjectJaccard) continue;+ const noteStance = deriveNoteStance(spanTokens, markers);+ const base = {+ thesisSlug: thesis.slug,+ noteId: note.id,+ score: subj,+ thesisStance,+ noteStance,+ thesisQuote: statement,+ noteQuote: span,+ action: "ask_user" as const,+ };+ if (noteStance !== thesisStance) {+ out.push({ ...base, kind: "contradict" });+ } else if (subj >= supportJaccard) {+ out.push({ ...base, kind: "support" });+ }+ // Same stance below the support threshold merely adds complexity:+ // deliberately not flagged (per the article).+ }+ }++ out.sort(+ (x, y) =>+ x.thesisSlug.localeCompare(y.thesisSlug) ||+ x.noteId.localeCompare(y.noteId) ||+ x.kind.localeCompare(y.kind),+ );+ return out;+}++// ----- Lifecycle checks -----------------------------------------------------++export interface ThesisForStaleness {+ readonly slug: string;+ readonly status: ThesisStatus;+ readonly cadence: ObligationCadence;+ readonly lastUpdated: string;+}++export interface StaleThesisFinding {+ readonly slug: string;+ readonly lastUpdated: string;+ /** Next-due date derived from the cadence; already in the past. */+ readonly nextDue: string;+ readonly cadence: ObligationCadence;+}++/**+ * Flag active theses whose `last_updated` is past their cadence's next-due+ * date. Reuses `obligations.ts` `nextDueDate` for the cadence arithmetic.+ * Closed theses and those with an unparseable `last_updated` are skipped.+ */+export function detectStaleTheses(+ theses: ReadonlyArray<ThesisForStaleness>,+ opts: { now: Date },+): StaleThesisFinding[] {+ const today = isoDate(opts.now);+ const out: StaleThesisFinding[] = [];+ for (const t of theses) {+ if (t.status !== THESIS_STATUS.active) continue;+ let nextDue: string;+ try {+ nextDue = nextDueDate(t.cadence, t.lastUpdated);+ } catch {+ continue;+ }+ if (nextDue >= today) continue;+ out.push({ slug: t.slug, lastUpdated: t.lastUpdated, nextDue, cadence: t.cadence });+ }+ out.sort((a, b) => a.nextDue.localeCompare(b.nextDue) || a.slug.localeCompare(b.slug));+ return out;+}++export interface ThesisForGraveyard {+ readonly slug: string;+ readonly status: ThesisStatus;+ readonly createdAt: string;+ readonly lastSupportAt: string | null;+}++export interface GraveyardThesisFinding {+ readonly slug: string;+ /** The date support was last seen (or null β measured from `createdAt`). */+ readonly lastSupportAt: string | null;+ readonly ageDays: number;+ /** The recommended action; the operator confirms - never auto-applied. */+ readonly suggestion: "close";+}++function parseIsoUtc(value: string): number {+ const iso = ISO_DATE_ONLY_RE.test(value) ? `${value}T00:00:00Z` : value;+ return Date.parse(iso);+}++/**+ * Flag active theses with no supporting evidence in the last+ * `maxAgeDays` (the "thesis graveyard" pass). A thesis that never had+ * supporting evidence is measured from `createdAt`. Mirrors+ * `stale-claim.ts`; deterministic with an injected clock.+ */+export function detectThesisGraveyard(+ theses: ReadonlyArray<ThesisForGraveyard>,+ opts: { maxAgeDays: number; now: Date },+): GraveyardThesisFinding[] {+ const nowMs = opts.now.getTime();+ const out: GraveyardThesisFinding[] = [];+ for (const t of theses) {+ if (t.status !== THESIS_STATUS.active) continue;+ const anchor = t.lastSupportAt ?? t.createdAt;+ const anchorMs = parseIsoUtc(anchor);+ if (!Number.isFinite(anchorMs)) continue;+ const ageDays = Math.floor((nowMs - anchorMs) / DAY_MS);+ if (ageDays <= opts.maxAgeDays) continue;+ out.push({ slug: t.slug, lastSupportAt: t.lastSupportAt, ageDays, suggestion: "close" });+ }+ out.sort((a, b) => b.ageDays - a.ageDays || a.slug.localeCompare(b.slug));+ return out;+}diff --git a/src/core/brain/idempotency-ledger.ts b/src/core/brain/idempotency-ledger.tsnew file mode 100644index 00000000..d7b69069--- /dev/null+++ b/src/core/brain/idempotency-ledger.ts@@ -0,0 +1,238 @@+/**+ * Client-supplied idempotency-key ledger (C1 / t_213f356b).+ *+ * OSB is multi-runtime (Hermes, Claude Code, Codex, opencode) and the+ * feedback / signal write path is a file-append operation. A retried or+ * double-delivered tool call can therefore append a duplicate record,+ * and the implicit slug-based dedupe silently MERGES genuinely-different+ * content that happens to share a topic slug. This ledger gives those+ * writes a first-class idempotency key:+ *+ * - {@link rememberKey} records `key -> contentHash`. Same key + same+ * hash is a deduped no-op (`duplicate_match`); same key + different+ * hash is an explicit `payload_mismatch` (never a silent overwrite);+ * an unseen key is `inserted`.+ * - {@link lookupKey} reads the stored record for a key (audit / C6 /+ * the pre-write dedupe consult the writers perform).+ *+ * Storage mirrors the continuity store's append/list model: month-sharded+ * JSONL under `<vault>/Brain/logs/idempotency/<YYYY-MM>.jsonl`, appended+ * under a per-shard lock. Reads scan every shard because a retry may land+ * in a later month than the original write.+ *+ * Concurrency boundary (inherited from the continuity model): the+ * check-and-append is atomic within a single shard under its lock. Two+ * genuinely-concurrent FIRST writers of the same key can both observe+ * "absent" and both proceed β a rare race the append-only store does not+ * guard. The primary target β a SEQUENTIAL retry after a crash / double+ * delivery β is fully deduped because the prior write is already durable+ * on disk before the retry runs.+ *+ * The kernel stays deterministic: no LLM, no wall-clock beyond the+ * caller-supplied (or defaulted) `createdAt`.+ */++import { createHash } from "node:crypto";+import { existsSync, mkdirSync, readFileSync, readdirSync, writeFileSync } from "node:fs";+import { join } from "node:path";++import { BRAIN_ROOT_REL, ensureInsideVault } from "./paths.ts";+import { acquireLockSync } from "./sync-lockfile.ts";+import { isoSecond } from "./time.ts";++/** Month-sharded JSONL root, distinct from `Brain/log/` (the human trail). */+const IDEMPOTENCY_REL = `${BRAIN_ROOT_REL}/logs/idempotency`;++/** Hard cap on a client key. Long enough for `<session-id>:<slug>` joins. */+const KEY_MAX_LEN = 256;++export const REMEMBER_KEY_STATUS = Object.freeze({+ inserted: "inserted",+ duplicate_match: "duplicate_match",+ payload_mismatch: "payload_mismatch",+} as const);++export type RememberKeyStatus = (typeof REMEMBER_KEY_STATUS)[keyof typeof REMEMBER_KEY_STATUS];++/** One durable ledger entry. `ref` is an opaque, JSON-serialisable set of+ * coordinates the caller stores on insert so a later dedupe can return the+ * original write's identity (e.g. `{ id, path }`). */+export interface IdempotencyRecord {+ readonly key: string;+ readonly contentHash: string;+ readonly createdAt: string;+ readonly ref?: Readonly<Record<string, unknown>>;+}++export interface RememberKeyInput {+ readonly key: string;+ readonly contentHash: string;+ /** Canonical UTC ISO-8601; defaults to `isoSecond(new Date())`. Drives the+ * month shard, so a backfilled write lands in its real month. */+ readonly createdAt?: string;+ readonly ref?: Readonly<Record<string, unknown>>;+}++export interface RememberKeyResult {+ readonly status: RememberKeyStatus;+ /**+ * The effective record: the freshly-inserted one on `inserted`, or the+ * PRE-EXISTING stored record on `duplicate_match` / `payload_mismatch`+ * (so callers can read the original `contentHash` / `ref`).+ */+ readonly record: IdempotencyRecord;+}++/**+ * Thrown by the writers ({@link import('./signal.ts').writeSignal} et al.)+ * when a supplied idempotency key was already used with a DIFFERENT+ * payload. Carries both hashes so the caller can explain the conflict.+ * The ledger itself never throws this β it returns `payload_mismatch` and+ * the writer decides to surface it as an error (C4's batch path inspects+ * the status instead, without aborting the whole batch).+ */+export class IdempotencyPayloadMismatchError extends Error {+ readonly key: string;+ readonly existingHash: string;+ readonly attemptedHash: string;++ constructor(key: string, existingHash: string, attemptedHash: string) {+ super(+ `idempotency key '${key}' was already used with a different payload ` ++ `(stored ${existingHash.slice(0, 12)}β¦, attempted ${attemptedHash.slice(0, 12)}β¦). ` ++ "Reusing a key with different content is rejected to avoid a silent overwrite.",+ );+ this.name = "IdempotencyPayloadMismatchError";+ this.key = key;+ this.existingHash = existingHash;+ this.attemptedHash = attemptedHash;+ }+}++/**+ * SHA-256 (hex) over a canonical, key-order-insensitive JSON encoding of+ * `fields`. `undefined`-valued keys are dropped so an optional field left+ * unset hashes identically to it being absent. Callers pass the SEMANTIC+ * payload of a write (never timestamps or allocated slugs) so a retry with+ * the same content produces the same hash.+ */+export function computePayloadHash(fields: Readonly<Record<string, unknown>>): string {+ return createHash("sha256").update(canonicalJson(fields), "utf8").digest("hex");+}++/** Deterministic JSON: object keys sorted recursively, arrays kept in order,+ * `undefined` object entries omitted. */+function canonicalJson(value: unknown): string {+ if (value === null || typeof value !== "object") return JSON.stringify(value) ?? "null";+ if (Array.isArray(value)) return `[${value.map((v) => canonicalJson(v)).join(",")}]`;+ const entries = Object.entries(value as Record<string, unknown>)+ .filter(([, v]) => v !== undefined)+ .sort(([a], [b]) => (a < b ? -1 : a > b ? 1 : 0));+ return `{${entries.map(([k, v]) => `${JSON.stringify(k)}:${canonicalJson(v)}`).join(",")}}`;+}++export function idempotencyLogPath(vault: string, month: string): string {+ if (!/^\d{4}-\d{2}$/.test(month)) throw new Error(`invalid idempotency month: ${month}`);+ return ensureInsideVault(join(vault, IDEMPOTENCY_REL, `${month}.jsonl`), vault);+}++/**+ * Look up the stored record for a client key across all month shards.+ * Returns the first-written record for the key (shards read in ascending+ * month order, lines in append order) or `null` when the key is unseen.+ */+export function lookupKey(vault: string, key: string): IdempotencyRecord | null {+ const normalised = normaliseKey(key);+ for (const record of readAllRecords(vault)) {+ if (record.key === normalised) return record;+ }+ return null;+}++/**+ * Record `key -> contentHash`. See {@link RememberKeyResult} for the three+ * outcomes. Only `inserted` appends a line; `duplicate_match` and+ * `payload_mismatch` write nothing.+ */+export function rememberKey(vault: string, input: RememberKeyInput): RememberKeyResult {+ const key = normaliseKey(input.key);+ const contentHash = requireHash(input.contentHash);+ const createdAt = input.createdAt ?? isoSecond(new Date());+ const month = monthOf(createdAt);++ mkdirSync(join(vault, IDEMPOTENCY_REL), { recursive: true });+ const shardPath = idempotencyLogPath(vault, month);+ const handle = acquireLockSync(shardPath);+ try {+ // Re-scan under the lock so a same-shard concurrent insert is caught.+ const existing = lookupKey(vault, key);+ if (existing) {+ return {+ status:+ existing.contentHash === contentHash+ ? REMEMBER_KEY_STATUS.duplicate_match+ : REMEMBER_KEY_STATUS.payload_mismatch,+ record: existing,+ };+ }+ const record: IdempotencyRecord = Object.freeze({+ key,+ contentHash,+ createdAt,+ ...(input.ref !== undefined ? { ref: input.ref } : {}),+ });+ writeFileSync(shardPath, `${JSON.stringify(record)}\n`, { encoding: "utf8", flag: "a" });+ return { status: REMEMBER_KEY_STATUS.inserted, record };+ } finally {+ handle.release();+ }+}++// ----- internals -----------------------------------------------------------++function normaliseKey(key: unknown): string {+ if (typeof key !== "string" || key.trim() === "") {+ throw new Error("idempotency key must be a non-empty string");+ }+ const trimmed = key.trim();+ if (trimmed.length > KEY_MAX_LEN) {+ throw new Error(`idempotency key too long (max ${KEY_MAX_LEN} chars)`);+ }+ return trimmed;+}++function requireHash(hash: unknown): string {+ if (typeof hash !== "string" || hash.trim() === "") {+ throw new Error("idempotency contentHash must be a non-empty string");+ }+ return hash.trim();+}++function monthOf(createdAt: string): string {+ const month = createdAt.slice(0, 7);+ if (!/^\d{4}-\d{2}$/.test(month)) {+ throw new Error(+ `idempotency createdAt must start with YYYY-MM; got ${JSON.stringify(createdAt)}`,+ );+ }+ return month;+}++function readAllRecords(vault: string): IdempotencyRecord[] {+ const dir = ensureInsideVault(join(vault, IDEMPOTENCY_REL), vault);+ if (!existsSync(dir)) return [];+ const records: IdempotencyRecord[] = [];+ for (const name of readdirSync(dir).toSorted()) {+ if (!name.endsWith(".jsonl")) continue;+ const path = ensureInsideVault(join(dir, name), vault);+ for (const line of readFileSync(path, "utf8").split("\n")) {+ if (!line.trim()) continue;+ try {+ records.push(JSON.parse(line) as IdempotencyRecord);+ } catch {+ continue;+ }+ }+ }+ return records;+}diff --git a/src/core/brain/ingest/batch-plan.ts b/src/core/brain/ingest/batch-plan.tsnew file mode 100644index 00000000..97f2e379--- /dev/null+++ b/src/core/brain/ingest/batch-plan.ts@@ -0,0 +1,226 @@+/**+ * Batch-plan step for large-folder ingest (A3, t_9eeb8ca2).+ *+ * OSB's ingest pipeline ({@link ingestSource}) handles ONE source at a time and+ * has no planner to shard a large folder into bounded parallel work. This module+ * supplies that planner: {@link planBatches} discovers the ingestible files under+ * a source directory, consults A1's content-hash manifest to skip everything that+ * is `unchanged` since the last ingest, and splits the `new`/`modified` remainder+ * into size+count-bounded batches. The caller (an agent or the CLI) dispatches+ * each batch as a parallel subagent - the kernel runs NO ingestion and spawns NO+ * subagents itself. Planning stays deterministic and model-free.+ *+ * Determinism: files are discovered by a stable recursive walk, sorted by their+ * canonical vault-relative path, then packed greedily - each batch is filled up+ * to the byte cap and the file-count cap, whichever binds first. The same+ * directory (same bytes on disk) always yields the same batch list.+ *+ * Ingestibility mirrors the pipeline's "text-bearing sources only" contract (no+ * OCR, binary, or media): a small, extensible set of text/document extensions.+ * The match is language-agnostic - it is over file extension and bytes, never+ * over natural-language content. Hidden files and dot-directories (`.git`,+ * `.open-second-brain`, ...) are skipped so machine artifacts never masquerade+ * as ingestible sources.+ */++import { existsSync, readdirSync, statSync } from "node:fs";+import { join, posix, relative } from "node:path";++import { canonicalNotePath, ensureInsideVault } from "../../path-safety.ts";+import { classifyPaths, readManifest } from "./content-manifest.ts";++/**+ * Default set of ingestible (text-bearing) extensions, lowercase with the dot.+ * Deliberately conservative - the pipeline runs no OCR/binary path, so only+ * plain-text and lightweight-markup document formats qualify. Callers may+ * override via {@link BatchPlanOptions.extensions}.+ */+export const DEFAULT_INGESTIBLE_EXTENSIONS: readonly string[] = Object.freeze([+ ".md",+ ".markdown",+ ".txt",+ ".text",+ ".rst",+ ".org",+]);++export interface BatchPlanOptions {+ /** Hard upper bound on the summed bytes of one batch (must be > 0). */+ readonly maxBatchBytes: number;+ /** Hard upper bound on the file count of one batch (must be a positive int). */+ readonly maxBatchFiles: number;+ /**+ * Override the ingestible extension set (lowercase, dot-prefixed). Absent β+ * {@link DEFAULT_INGESTIBLE_EXTENSIONS}.+ */+ readonly extensions?: readonly string[];+}++/** One file selected for (re-)ingest, with the reason it was selected. */+export interface PlannedFile {+ /** Canonical vault-relative POSIX path (the id ingest addresses). */+ readonly path: string;+ /** File size in bytes (drives the byte-cap packing). */+ readonly bytes: number;+ /** Why the file is being ingested: absent from / differing in the manifest. */+ readonly status: "new" | "modified";+}++/** One bounded batch the caller dispatches as a single parallel unit. */+export interface IngestBatch {+ /** Zero-based position of the batch within the plan. */+ readonly index: number;+ /** Files in this batch, in sorted-by-path order. */+ readonly files: readonly PlannedFile[];+ /** Summed bytes of the batch's files. */+ readonly totalBytes: number;+}++export interface BatchPlan {+ /** Canonical vault-relative POSIX path of the planned source directory. */+ readonly sourceDir: string;+ readonly maxBatchBytes: number;+ readonly maxBatchFiles: number;+ /** The bounded batches, in packing order. Empty when nothing needs ingest. */+ readonly batches: readonly IngestBatch[];+ /** Canonical paths classified `unchanged` and therefore skipped, sorted. */+ readonly skipped: readonly string[];+ /** Total number of files across all batches (`new` + `modified`). */+ readonly totalFiles: number;+ /** Total bytes across all batches. */+ readonly totalBytes: number;+}++/**+ * Plan the ingest of a source directory into bounded parallel batches.+ *+ * Discovers the ingestible files under `sourceDir` (a vault-relative path),+ * skips those the content-hash manifest classifies `unchanged`, and packs the+ * `new`/`modified` remainder greedily into batches that respect BOTH caps. A+ * single file larger than the byte cap cannot be split, so it forms its own+ * singleton batch (documented, honest - never silently dropped or truncated).+ *+ * Throws when a cap is not a positive number, or when `sourceDir` does not+ * resolve to an existing directory inside the vault (no misleading empty-plan+ * fallback for a typo'd or escaping path).+ */+export function planBatches(vault: string, sourceDir: string, opts: BatchPlanOptions): BatchPlan {+ if (!Number.isFinite(opts.maxBatchBytes) || opts.maxBatchBytes <= 0) {+ throw new Error(`planBatches: maxBatchBytes must be > 0, got ${opts.maxBatchBytes}`);+ }+ if (!Number.isInteger(opts.maxBatchFiles) || opts.maxBatchFiles <= 0) {+ throw new Error(+ `planBatches: maxBatchFiles must be a positive integer, got ${opts.maxBatchFiles}`,+ );+ }++ const dirRel = canonicalNotePath(sourceDir);+ const dirAbs = ensureInsideVault(join(vault, dirRel), vault);+ if (!existsSync(dirAbs) || !statSync(dirAbs).isDirectory()) {+ throw new Error(`planBatches: source dir is not an existing directory: ${sourceDir}`);+ }++ const extensions = normalizeExtensions(opts.extensions ?? DEFAULT_INGESTIBLE_EXTENSIONS);++ // Discover ingestible files as canonical vault-relative paths, sorted.+ const discovered: string[] = [];+ collectIngestible(dirAbs, extensions, discovered);+ const relPaths = discovered.map((abs) => canonicalNotePath(toPosixRel(vault, abs))).toSorted();++ // Consult A1's manifest: only `new`/`modified` sources are worth ingesting.+ const classification = classifyPaths(vault, relPaths, readManifest(vault));+ const skipped = [...classification.unchanged].toSorted();++ // Build the planned-file list, tagging each with why it was selected, in+ // sorted-by-path order so packing (and the whole plan) is deterministic.+ const status = new Map<string, "new" | "modified">();+ for (const p of classification.new) status.set(p, "new");+ for (const p of classification.modified) status.set(p, "modified");+ const planned: PlannedFile[] = [...status.keys()].toSorted().map((path) => ({+ path,+ bytes: statSync(join(vault, path)).size,+ status: status.get(path)!,+ }));++ const batches = packBatches(planned, opts.maxBatchBytes, opts.maxBatchFiles);++ return {+ sourceDir: dirRel,+ maxBatchBytes: opts.maxBatchBytes,+ maxBatchFiles: opts.maxBatchFiles,+ batches,+ skipped,+ totalFiles: planned.length,+ totalBytes: planned.reduce((sum, f) => sum + f.bytes, 0),+ };+}++/**+ * Greedily pack sorted files into batches. A file joins the current batch unless+ * doing so would breach the byte cap or the count cap, in which case the current+ * batch is flushed and the file opens a new one. The `current.length > 0` guard+ * ensures a lone file that itself exceeds the byte cap is never deferred forever+ * - it becomes its own (oversize, unsplittable) singleton batch.+ */+function packBatches(+ files: readonly PlannedFile[],+ maxBatchBytes: number,+ maxBatchFiles: number,+): IngestBatch[] {+ const batches: IngestBatch[] = [];+ let current: PlannedFile[] = [];+ let currentBytes = 0;++ const flush = (): void => {+ if (current.length === 0) return;+ batches.push({ index: batches.length, files: current, totalBytes: currentBytes });+ current = [];+ currentBytes = 0;+ };++ for (const file of files) {+ const wouldExceedBytes = current.length > 0 && currentBytes + file.bytes > maxBatchBytes;+ const wouldExceedCount = current.length >= maxBatchFiles;+ if (wouldExceedBytes || wouldExceedCount) flush();+ current.push(file);+ currentBytes += file.bytes;+ }+ flush();++ return batches;+}++/** Recursively collect ingestible regular-file absolute paths, skipping hidden entries. */+function collectIngestible(dir: string, extensions: ReadonlySet<string>, out: string[]): void {+ for (const entry of readdirSync(dir, { withFileTypes: true })) {+ if (entry.name.startsWith(".")) continue; // hidden file or dot-directory+ const abs = join(dir, entry.name);+ if (entry.isDirectory()) {+ collectIngestible(abs, extensions, out);+ } else if (entry.isFile() && extensions.has(extname(entry.name))) {+ out.push(abs);+ }+ }+}++/** Lowercase, dot-prefixed extension of a filename, or "" when it has none. */+function extname(name: string): string {+ const dot = name.lastIndexOf(".");+ return dot <= 0 ? "" : name.slice(dot).toLowerCase();+}++/** Normalize an extension list to a lowercase, dot-prefixed lookup set. */+function normalizeExtensions(exts: readonly string[]): ReadonlySet<string> {+ const set = new Set<string>();+ for (const raw of exts) {+ const trimmed = raw.trim().toLowerCase();+ if (trimmed.length === 0) continue;+ set.add(trimmed.startsWith(".") ? trimmed : `.${trimmed}`);+ }+ return set;+}++/** Vault-relative POSIX path for an absolute path inside the vault. */+function toPosixRel(vault: string, abs: string): string {+ return relative(vault, abs).split(/[\\/]/).join(posix.sep);+}diff --git a/src/core/brain/ingest/content-manifest.ts b/src/core/brain/ingest/content-manifest.tsnew file mode 100644index 00000000..7b8bb87f--- /dev/null+++ b/src/core/brain/ingest/content-manifest.ts@@ -0,0 +1,248 @@+/**+ * Content-hash skip-unchanged manifest for incremental ingest.+ *+ * The source-ingest pipeline already computes a source-IDENTITY hash+ * ({@link sourceIdentityHash}) that answers "is this the same logical source" -+ * a stable key for idempotent dedup, hashed over the source-path string. This+ * module answers a DIFFERENT question: "are the source's bytes unchanged since+ * we last ingested it?" - hashed over the file CONTENT, not its identity. The+ * two are deliberately distinct artifacts:+ *+ * - identity hash β which summary page a source maps to (path-derived),+ * - content hash β whether a re-ingest can skip the expensive extraction+ * pass because nothing actually changed (byte-derived).+ *+ * The manifest is TIMESTAMP-INDEPENDENT by design. A `git checkout`, an `rsync`,+ * or an NFS restat bumps a file's mtime without changing its bytes; an mtime-+ * driven cache would force a needless re-ingest (and a needless LLM pass) in+ * that case. Comparing content hashes instead means only a real byte change+ * re-triggers ingestion.+ *+ * The manifest lives at `<vault>/.open-second-brain/ingest-manifest.json`, a+ * MACHINE artifact - not curated memory, so NOT under `Brain/`. This mirrors the+ * existing `<vault>/.open-second-brain/` location used by the search index and+ * the Aider install sidecar. Writes are atomic and a no-op rerun (every source+ * unchanged) rewrites nothing, so the file's bytes and mtime stay stable.+ *+ * Language-agnostic: hashing is over raw bytes; no natural-language content is+ * inspected.+ */++import { createHash } from "node:crypto";+import { existsSync, readdirSync, readFileSync, statSync } from "node:fs";+import { join } from "node:path";++import { atomicWriteFileSync } from "../../fs-atomic.ts";+import { canonicalNotePath } from "../../path-safety.ts";++/** Only schema version currently understood. Unknown versions are refused. */+const SCHEMA_VERSION = 1 as const;++/** Vault-relative directory holding OSB machine artifacts (not curated memory). */+const MACHINE_ARTIFACT_DIR = ".open-second-brain";++/** Basename of the ingest content-hash manifest. */+const MANIFEST_FILE = "ingest-manifest.json";++/**+ * The persisted manifest: a map from a canonical vault-relative source path to+ * the SHA-256 hex of its content at last ingest.+ */+export interface ContentManifest {+ readonly schema_version: typeof SCHEMA_VERSION;+ /** Canonical vault-relative path β 64-char lowercase SHA-256 hex of content. */+ readonly entries: Readonly<Record<string, string>>;+}++/** Result of classifying a set of paths against a manifest. */+export interface PathClassification {+ /** On disk, absent from the manifest. */+ readonly new: string[];+ /** On disk, present in the manifest, bytes differ. */+ readonly modified: string[];+ /** On disk, present in the manifest, bytes identical. */+ readonly unchanged: string[];+ /** In the manifest (or requested) but no longer on disk. */+ readonly missing: string[];+}++/** Absolute path of the content-hash manifest for a vault. */+export function manifestPath(vault: string): string {+ return join(vault, MACHINE_ARTIFACT_DIR, MANIFEST_FILE);+}++/**+ * SHA-256 hex over a single file's raw bytes. Timestamp-independent: only the+ * content contributes to the digest. Throws if the path is a directory - use+ * {@link hashTree} for those.+ */+export function hashFile(absPath: string): string {+ const stat = statSync(absPath);+ if (stat.isDirectory()) {+ throw new Error(`hashFile: path is a directory, use hashTree: ${absPath}`);+ }+ return createHash("sha256").update(readFileSync(absPath)).digest("hex");+}++/**+ * SHA-256 hex over a directory TREE, deterministic and content-only. Every+ * regular file under `dir` is enumerated, sorted by its POSIX relative path,+ * and folded into one digest as `<relPath>\0<fileContentHash>\n`. Because the+ * relative path is part of the digest, moving a file changes the tree hash even+ * if its bytes are unchanged; because file contents are hashed (not their+ * mtimes), a touch that leaves bytes identical does not.+ */+export function hashTree(dir: string): string {+ const rels: string[] = [];+ collectFilesRel(dir, "", rels);+ rels.sort();+ const hash = createHash("sha256");+ for (const rel of rels) {+ hash.update(rel);+ hash.update("\0");+ hash.update(hashFile(join(dir, rel)));+ hash.update("\n");+ }+ return hash.digest("hex");+}++/** SHA-256 over a file or a directory tree, dispatching on the path's type. */+export function hashPath(absPath: string): string {+ return statSync(absPath).isDirectory() ? hashTree(absPath) : hashFile(absPath);+}++/**+ * Read the manifest for a vault. A missing file is an empty manifest (every+ * path then classifies `new`). A corrupted file or an unknown `schema_version`+ * is a hard error - never a silent reset that would masquerade every source as+ * unchanged or force a full re-ingest without saying so.+ */+export function readManifest(vault: string): ContentManifest {+ const path = manifestPath(vault);+ if (!existsSync(path)) {+ return { schema_version: SCHEMA_VERSION, entries: {} };+ }+ const raw = readFileSync(path, "utf8");+ let parsed: unknown;+ try {+ parsed = JSON.parse(raw);+ } catch (e) {+ throw new Error(`ingest manifest is corrupted JSON: ${path}`, { cause: e });+ }+ if (parsed === null || typeof parsed !== "object" || Array.isArray(parsed)) {+ throw new Error(`ingest manifest is not an object: ${path}`);+ }+ const obj = parsed as Record<string, unknown>;+ const sv = obj["schema_version"];+ if (sv !== SCHEMA_VERSION) {+ throw new Error(+ `ingest manifest schema_version ${String(sv)} not supported (expected ${SCHEMA_VERSION}): ${path}`,+ );+ }+ const rawEntries = obj["entries"];+ const entries: Record<string, string> = {};+ if (rawEntries !== null && typeof rawEntries === "object" && !Array.isArray(rawEntries)) {+ for (const [key, value] of Object.entries(rawEntries as Record<string, unknown>)) {+ if (typeof value === "string") entries[key] = value;+ }+ }+ return { schema_version: SCHEMA_VERSION, entries };+}++/**+ * Serialize `entries` to the canonical manifest bytes: keys sorted so the+ * output is deterministic regardless of insertion order, `schema_version`+ * first, trailing newline.+ */+function serializeManifest(entries: Record<string, string>): string {+ const sorted: Record<string, string> = {};+ for (const key of Object.keys(entries).toSorted()) {+ sorted[key] = entries[key]!;+ }+ return JSON.stringify({ schema_version: SCHEMA_VERSION, entries: sorted }, null, 2) + "\n";+}++/**+ * Atomically write the manifest, but ONLY if its serialized bytes differ from+ * what is already on disk. Returns `true` when a write happened, `false` on a+ * no-op. The byte-identity check is what makes an all-unchanged rerun rewrite+ * nothing (and leaves the file's mtime alone).+ */+export function writeManifestAtomic(vault: string, entries: Record<string, string>): boolean {+ const path = manifestPath(vault);+ const next = serializeManifest(entries);+ if (existsSync(path) && readFileSync(path, "utf8") === next) {+ return false;+ }+ atomicWriteFileSync(path, next);+ return true;+}++/**+ * Classify each requested path against the manifest by comparing its LIVE+ * content hash to the recorded one:+ * - not on disk β `missing`,+ * - on disk, not recorded β `new`,+ * - on disk, hash matches β `unchanged`,+ * - on disk, hash differs β `modified`.+ * Paths are canonicalized before lookup so they match the keys written by+ * {@link updateManifest}. Order within each bucket follows the input order.+ */+export function classifyPaths(+ vault: string,+ paths: readonly string[],+ manifest: ContentManifest,+): PathClassification {+ const result: PathClassification = { new: [], modified: [], unchanged: [], missing: [] };+ for (const path of paths) {+ const canonical = canonicalNotePath(path);+ const abs = join(vault, canonical);+ if (!existsSync(abs)) {+ result.missing.push(canonical);+ continue;+ }+ const recorded = manifest.entries[canonical];+ const live = hashPath(abs);+ if (recorded === undefined) {+ result.new.push(canonical);+ } else if (recorded === live) {+ result.unchanged.push(canonical);+ } else {+ result.modified.push(canonical);+ }+ }+ return result;+}++/**+ * Record post-ingest content hashes for `paths`, merging into the existing+ * manifest. A path still on disk gets its current hash recorded; a path that+ * has been deleted is dropped from the manifest (so it will re-classify as+ * `new` if it ever reappears). Writes atomically and skips the write entirely+ * when nothing changed. Returns `true` when the manifest was rewritten.+ */+export function updateManifest(vault: string, paths: readonly string[]): boolean {+ const entries: Record<string, string> = { ...readManifest(vault).entries };+ for (const path of paths) {+ const canonical = canonicalNotePath(path);+ const abs = join(vault, canonical);+ if (existsSync(abs)) {+ entries[canonical] = hashPath(abs);+ } else {+ delete entries[canonical];+ }+ }+ return writeManifestAtomic(vault, entries);+}++/** Recursively collect regular-file relative paths under `dir` (POSIX slashes). */+function collectFilesRel(dir: string, prefix: string, out: string[]): void {+ for (const entry of readdirSync(dir, { withFileTypes: true })) {+ const rel = prefix === "" ? entry.name : `${prefix}/${entry.name}`;+ if (entry.isDirectory()) {+ collectFilesRel(join(dir, entry.name), rel, out);+ } else if (entry.isFile()) {+ out.push(rel);+ }+ }+}diff --git a/src/core/brain/ingest/ingest.ts b/src/core/brain/ingest/ingest.tsindex 5a01cdca..4c668510 100644--- a/src/core/brain/ingest/ingest.ts+++ b/src/core/brain/ingest/ingest.ts@@ -19,12 +19,17 @@* genuine connection to prior material; a freshly created entity is not.*/-import { existsSync, mkdirSync } from "node:fs";-import { dirname, relative } from "node:path";+import { existsSync, mkdirSync, readFileSync } from "node:fs";+import { dirname, join, relative } from "node:path";import type { FrontmatterMap } from "../../types.ts";import { canonicalNotePath } from "../../path-safety.ts";-import { parseFrontmatter, slugify, writeFrontmatterAtomic } from "../../vault.ts";+import {+ formatFrontmatter,+ parseFrontmatter,+ slugify,+ writeFrontmatterAtomic,+} from "../../vault.ts";import { isoSecond } from "../time.ts";import { sourcePagePath } from "../paths.ts";import { intakeExtraction, type ExtractionIntake } from "../intake/extract-intake.ts";@@ -33,6 +38,7 @@ import {sourceIdentityHash,type Provenance,} from "../provenance/provenance.ts";+import { updateManifest } from "./content-manifest.ts";/** Frontmatter `kind:` marker of an ingested source summary page. */export const BRAIN_SOURCE_KIND = "brain-source";@@ -122,8 +128,24 @@ export function ingestSource(.filter((section) => section.length > 0).join("\n\n");- mkdirSync(dirname(absPath), { recursive: true });- writeFrontmatterAtomic(absPath, meta, body, { overwrite: true });+ // Idempotent no-op: only rewrite the summary page when its bytes would+ // actually change. A byte-identical rewrite would churn the mtime and wake+ // the index watcher for nothing; skipping it keeps a re-ingest of an+ // unchanged source truly inert.+ const nextContents = formatFrontmatter(meta, body);+ const unchanged = existed && readFileSync(absPath, "utf8") === nextContents;+ if (!unchanged) {+ mkdirSync(dirname(absPath), { recursive: true });+ writeFrontmatterAtomic(absPath, meta, body, { overwrite: true });+ }++ // Record the source's CONTENT hash so a future re-ingest can classify it+ // `unchanged` and skip the extraction pass. Only when the source resolves to+ // a real file inside the vault - URL and other identity-only sources have no+ // bytes to hash and must leave the manifest untouched (backward-compatible).+ if (existsSync(join(vault, canonicalSource))) {+ updateManifest(vault, [canonicalSource]);+ }return {summaryPath: canonicalNotePath(relative(vault, absPath)),diff --git a/src/core/brain/paths.ts b/src/core/brain/paths.tsindex ee51ac4f..8f8b1c18 100644--- a/src/core/brain/paths.ts+++ b/src/core/brain/paths.ts@@ -52,6 +52,8 @@ export const BRAIN_PROCEDURES_REL = posix.join(BRAIN_ROOT_REL, "procedures");export const BRAIN_PROCEDURAL_MEMORY_REL = posix.join(BRAIN_ROOT_REL, "procedural-memory");export const BRAIN_ATTENTION_REL = posix.join(BRAIN_ROOT_REL, "attention");export const BRAIN_OBLIGATIONS_REL = posix.join(BRAIN_ROOT_REL, "obligations");+/** Declared-thesis register pages: `Brain/theses/thesis-<slug>.md` (D3). */+export const BRAIN_THESES_REL = posix.join(BRAIN_ROOT_REL, "theses");export const BRAIN_LOG_REL = posix.join(BRAIN_ROOT_REL, "log");export const BRAIN_ENTITIES_REL = posix.join(BRAIN_ROOT_REL, "entities");/** Obsidian Bases view definitions: `Brain/bases/<view>.base` (v1.15.0). */@@ -273,6 +275,17 @@ export function obligationPath(vault: string, slug: string): string {return ensureInsideVault(join(vault, BRAIN_OBLIGATIONS_REL, `${slug}.md`), vault);}+/** Declared-thesis register pages dir: `Brain/theses/`. */+export function thesesDir(vault: string): string {+ return ensureInsideVault(join(vault, BRAIN_THESES_REL), vault);+}++/** A single thesis page: `Brain/theses/thesis-<slug>.md`. */+export function thesisPath(vault: string, slug: string): string {+ const s = validateSlug(slug);+ return ensureInsideVault(join(vault, BRAIN_THESES_REL, `thesis-${s}.md`), vault);+}+/** Proposal scan watermark path: `Brain/procedural-memory/proposal-watermark.json`. */export function proposalWatermarkPath(vault: string): string {return ensureInsideVault(diff --git a/src/core/brain/pre-compact-extract.ts b/src/core/brain/pre-compact-extract.tsindex ef91540c..137fba8a 100644--- a/src/core/brain/pre-compact-extract.ts+++ b/src/core/brain/pre-compact-extract.ts@@ -1,7 +1,15 @@import { createHash } from "node:crypto";-import { appendContinuityRecord, listContinuityRecords } from "./continuity/store.ts";-import type { ContinuityRecord, ContinuitySourceRef } from "./continuity/types.ts";+import {+ appendContinuityRecord,+ buildContinuityRecord,+ listContinuityRecords,+} from "./continuity/store.ts";+import type {+ AppendContinuityRecordInput,+ ContinuityRecord,+ ContinuitySourceRef,+} from "./continuity/types.ts";export type PreCompactExtractType =| "decision"@@ -25,6 +33,16 @@ export interface PreCompactExtractInput {* one. Absent by default - omitted records stay byte-identical (t_c181f92b).*/readonly interrupted?: boolean;+ /**+ * Preview mode (C2 / t_2c6cf3e2). When true, return the candidate+ * records extraction WOULD append WITHOUT touching the vault β no+ * `appendContinuityRecord`, no log event, no dream/retire trigger. Each+ * returned record is built through the SAME `buildContinuityRecord`+ * path the real write uses, so the preview predicts the real extraction+ * byte-for-byte. Absent/false β existing write-committing behavior,+ * byte-identical. Mirrors the `opts.dryRun` idiom in session `import.ts`.+ */+ readonly dryRun?: boolean;}export interface PreCompactExtractResult {@@ -60,6 +78,12 @@ export function extractPreCompactRecords(const extracted = extractLines(sanitizePreCompactText(boundedText));const createdAt = input.createdAt ?? new Date().toISOString();const host = input.host?.trim();+ // Preview mode reuses the exact record builder but never writes, so the+ // candidate output predicts the real extraction byte-for-byte (C2).+ const persist: (recordInput: AppendContinuityRecordInput) => ContinuityRecord =+ input.dryRun === true+ ? buildContinuityRecord+ : (recordInput) => appendContinuityRecord(vault, recordInput);for (const item of extracted) {try {@@ -77,7 +101,7 @@ export function extractPreCompactRecords(continue;}records.push(- appendContinuityRecord(vault, {+ persist({kind: "pre_compact_extract",createdAt,sourceRefs,diff --git a/src/core/brain/preference.ts b/src/core/brain/preference.tsindex b6848595..c3433ee2 100644--- a/src/core/brain/preference.ts+++ b/src/core/brain/preference.ts@@ -34,7 +34,7 @@*/import { existsSync, readFileSync, unlinkSync } from "node:fs";-import { basename, dirname, join } from "node:path";+import { basename, dirname, join, relative } from "node:path";import type { FrontmatterMap } from "../types.ts";import { formatFrontmatter, parseFrontmatter, writeFrontmatterAtomic } from "../vault.ts";@@ -66,6 +66,13 @@ import {import type { PageLifecycle } from "./page-meta/lifecycle.ts";import type { PageTier } from "./page-meta/tier.ts";import { computeContentHash } from "./content-hash.ts";+import { normalizeExpirationDate } from "./expiration.ts";+import {+ computePayloadHash,+ IdempotencyPayloadMismatchError,+ lookupKey,+ rememberKey,+} from "./idempotency-ledger.ts";import { appendPrefAudit } from "./pref-audit.ts";import { PREF_AUDIT_OP } from "./types.ts";@@ -184,10 +191,32 @@ export interface WritePreferenceInput {*/readonly valid_from?: string;readonly valid_until?: string;+ /**+ * Caller-settable expiration (C5 / t_a82b674e). ISO date+ * (`YYYY-MM-DD`) or full timestamp. Additive and optional: absent β+ * the write is byte-identical to the historical path. When present it+ * is validated and stamped into frontmatter; the default read/list+ * path drops a preference past this date unless the caller opts into+ * `showExpired`. Orthogonal to dream retirement β the file is never+ * moved to `Brain/retired/` on expiry.+ */+ readonly expiration_date?: string;/** Optional extra tags merged after the canonical set. */readonly extraTags?: ReadonlyArray<string>;/** Free-form "How to apply" prose (rendered as a section). */readonly howToApply?: string;+ /**+ * Client-supplied idempotency key (C1 / t_213f356b). Additive and+ * optional: absent β the write is byte-identical to the historical+ * path. When present, the writer hashes the identity-defining fields+ * (slug, topic, principle, scope, status, evidenced_by β never the+ * timestamps or dream-owned counters) and consults the idempotency+ * ledger. A repeat with the same key + same payload is a deduped no-op;+ * the same key + a different payload throws+ * {@link IdempotencyPayloadMismatchError}. Used by the force-confirmed+ * feedback path and by C4's session checkpoint.+ */+ readonly idempotency_key?: string;/*** Recent `apply-evidence applied` rows for this pref, derived from* `Brain/log/`. Newest first. Used by {@link renderPreferenceBody}@@ -213,6 +242,12 @@ export interface ParsePreferenceOptions {export interface WritePreferenceResult {readonly path: string;readonly id: string;+ /**+ * Set to `true` when an `idempotency_key` matched a prior write with an+ * identical payload, so this call was a deduped no-op. Absent on a+ * normal write, so existing callers are unaffected.+ */+ readonly deduped?: boolean;}export interface MoveToRetiredOptions {@@ -291,6 +326,21 @@ export function writePreference(const path = preferencePath(vault, slug);const id = `pref-${slug}`;+ // Idempotency consult (C1): a repeat with the same key + same payload+ // dedupes; the same key + a different payload throws before any write.+ const idKey = input.idempotency_key?.trim();+ let idContentHash: string | undefined;+ if (idKey) {+ idContentHash = computePayloadHash(preferencePayloadFields(input, slug));+ const existing = lookupKey(vault, idKey);+ if (existing) {+ if (existing.contentHash === idContentHash) {+ return { path, id, deduped: true };+ }+ throw new IdempotencyPayloadMismatchError(idKey, existing.contentHash, idContentHash);+ }+ }+let metadata = preferenceFrontmatter(input, id);const body = renderPreferenceBody(input);@@ -318,7 +368,10 @@ export function writePreference(try {const next = formatFrontmatter(metadata, body);const prev = readFileSync(path, "utf8");- if (next === prev) return { path, id };+ if (next === prev) {+ recordPreferenceKey(vault, idKey, idContentHash, input.created_at, id, path);+ return { path, id };+ }} catch {// Fall through to the atomic write β a read failure should not// prevent a legitimate update.@@ -331,9 +384,51 @@ export function writePreference(vaultForRelativePath: vault,});+ recordPreferenceKey(vault, idKey, idContentHash, input.created_at, id, path);return { path, id };}+/**+ * The identity-defining fields of a preference for idempotency hashing.+ * Timestamps, dream-owned counters, and confidence are excluded so a+ * retried write (or a force-confirmed re-record) with the same rule+ * dedupes regardless of when it arrives.+ */+function preferencePayloadFields(+ input: WritePreferenceInput,+ slug: string,+): Record<string, unknown> {+ return {+ slug,+ topic: input.topic.trim(),+ principle: sanitisePrinciple(input.principle),+ scope: input.scope?.trim(),+ status: input.status,+ evidenced_by: [...input.evidenced_by],+ };+}++/**+ * Record the idempotency key after a successful preference write so a+ * future retry dedupes. No-op when no key was supplied.+ */+function recordPreferenceKey(+ vault: string,+ idKey: string | undefined,+ contentHash: string | undefined,+ createdAt: string,+ id: string,+ path: string,+): void {+ if (!idKey || contentHash === undefined) return;+ rememberKey(vault, {+ key: idKey,+ contentHash,+ createdAt,+ ref: { id, path: relative(vault, path) },+ });+}+/*** Predicate twin of {@link writePreference}'s content-equality* short-circuit. Returns `true` iff calling `writePreference(vault,@@ -457,6 +552,12 @@ function preferenceFrontmatter(input: WritePreferenceInput, id: string): Frontma// callers stay byte-identical.if (input.valid_from?.trim()) metadata["valid_from"] = input.valid_from.trim();if (input.valid_until?.trim()) metadata["valid_until"] = input.valid_until.trim();+ // Caller-settable expiration (C5): validated + emitted only when+ // supplied so legacy callers stay byte-identical. Reader-side support+ // is via parsePreference's optional-scalar read + the expiration filter.+ if (input.expiration_date?.trim()) {+ metadata["expiration_date"] = normalizeExpirationDate(input.expiration_date);+ }// Freshness trend (t_ee09a6ce): stamped by the dream refresh pass;// emitted only when supplied so legacy callers stay byte-identical.if (input.freshness_trend?.trim()) metadata["freshness_trend"] = input.freshness_trend.trim();@@ -680,6 +781,9 @@ export function parsePreference(...(optionalScalarString(meta, "freshness_trend") !== undefined? { freshness_trend: optionalScalarString(meta, "freshness_trend") }: {}),+ ...(optionalScalarString(meta, "expiration_date") !== undefined+ ? { expiration_date: optionalScalarString(meta, "expiration_date") }+ : {}),...(optionalScalarString(meta, "supersedes") !== undefined? { supersedes: optionalScalarString(meta, "supersedes") }: {}),diff --git a/src/core/brain/query.ts b/src/core/brain/query.tsindex e6fa4652..00494dbe 100644--- a/src/core/brain/query.ts+++ b/src/core/brain/query.ts@@ -30,6 +30,7 @@ import { existsSync, readdirSync } from "node:fs";import { join } from "node:path";import { brainDirs } from "./paths.ts";+import { filterExpired, isExpired } from "./expiration.ts";import type { BrainLogEntry } from "./log.ts";import { listLogDates, readLogDay } from "./log-jsonl.ts";import { parsePreference, parseRetired } from "./preference.ts";@@ -99,6 +100,18 @@ export interface QueryByTopicResult {readonly all_log_events: ReadonlyArray<BrainLogEntry>;}+/**+ * Read-path options for the list surfaces (C5 / t_a82b674e). Default+ * behaviour drops any signal / preference past its `expiration_date`;+ * `showExpired` re-includes them for audit / recall of lapsed memories.+ */+export interface QueryByTopicOptions {+ /** When true, expired memories are kept in the result. Default false. */+ readonly showExpired?: boolean;+ /** Wall clock the expiration is compared against. Defaults to `new Date()`. */+ readonly now?: Date;+}+// ----- Public API -----------------------------------------------------------/**@@ -186,7 +199,11 @@ export function queryByPreference(vault: string, pref_id: string): QueryByPrefer* retired), and every log entry mentioning the preference id (active* or retired form). Unknown topic β empty result with `preference: null`.*/-export function queryByTopic(vault: string, topic: string): QueryByTopicResult {+export function queryByTopic(+ vault: string,+ topic: string,+ options: QueryByTopicOptions = {},+): QueryByTopicResult {const want = topic.trim();if (!want) {return Object.freeze({@@ -196,8 +213,11 @@ export function queryByTopic(vault: string, topic: string): QueryByTopicResult {});}+ const now = options.now ?? new Date();+ const showExpired = options.showExpired ?? false;+const dirs = brainDirs(vault);- const signals: BrainSignal[] = [];+ let signals: BrainSignal[] = [];// Collect signals from both inbox/ and inbox/processed/. We do not// recurse β `inbox/processed/` is the only allowed sub-folder of@@ -206,6 +226,10 @@ export function queryByTopic(vault: string, topic: string): QueryByTopicResult {collectSignals(dirs.inbox, want, signals);collectSignals(dirs.processed, want, signals);signals.sort((a, b) => a.created_at.localeCompare(b.created_at));+ // Expiration filter (C5): drop signals past their expiration_date+ // unless the caller opts into showExpired. The files stay on disk β+ // this is a read-path filter, orthogonal to dream retirement.+ signals = filterExpired(signals, { now, showExpired });// Current rule β prefer the active preference; fall back to retired.let preference: BrainPreference | BrainRetired | null = null;@@ -213,6 +237,19 @@ export function queryByTopic(vault: string, topic: string): QueryByTopicResult {if (!preference) {preference = findPreferenceForTopic(dirs.retired, want, "retired");}+ // Expiration filter (C5): an active preference past its expiration_date+ // is dropped from the default result (surfaced only under showExpired).+ // A retired preference carries no live expiration and is never filtered+ // here β it is already out of the loop.+ if (+ !showExpired &&+ preference !== null &&+ preference.kind === "brain-preference" &&+ preference.expiration_date !== undefined &&+ isExpired(preference.expiration_date, now)+ ) {+ preference = null;+ }// All log events whose `preference` payload resolves to the topic's// pref id (both `pref-` and `ret-` forms, since the slug stem isdiff --git a/src/core/brain/recent-turns.ts b/src/core/brain/recent-turns.tsnew file mode 100644index 00000000..b5cf3126--- /dev/null+++ b/src/core/brain/recent-turns.ts@@ -0,0 +1,156 @@+/**+ * Bounded verbatim last-N-turns buffer (C3 / t_92317f91).+ *+ * Short-term continuity scaffolding β NOT curated long-term memory. After a+ * host compaction, OSB retains only summarized/extracted forms (active.md,+ * pre-compact extract, session summary); the exact recent wording is lost.+ * This buffer keeps a small, strictly-bounded ring of the last N conversation+ * turns verbatim so the agent can recover "what did the user literally say+ * 2-3 turns ago" immediately post-compaction.+ *+ * Design choices that keep it from becoming verbatim hoarding:+ * - Its OWN continuity kind (`recent_turn`) and its OWN reader β it never+ * leaks into the curated preference/signal/provenance surfaces.+ * - A hard read cap `RECENT_TURNS_CAP` (default 20): the observable buffer+ * never exceeds N turns. The continuity log is append-only by invariant+ * (`store.ts`: files are never rewritten/migrated), so eviction is a+ * bounded READ VIEW over the newest N β the older `recent_turn` rows share+ * the continuity log's existing retention lifecycle rather than being+ * rewritten in place. This is stated plainly, not disguised as a delete.+ * - Per-turn verbatim text is truncated to `MAX_TURN_CHARS`, so no single+ * row can grow without bound.+ *+ * Post-compaction re-surface is opt-in (default off β nothing surfaced), so an+ * unchanged install's context is byte-identical.+ */++import { appendContinuityRecord, listContinuityRecords } from "./continuity/store.ts";+import type { ContinuityRecord, ContinuitySourceRef } from "./continuity/types.ts";++/**+ * Hard cap on the observable buffer size (N). The last N turns are kept; a+ * read never returns more than this many, regardless of the requested limit.+ */+export const RECENT_TURNS_CAP = 20;++/** Per-turn verbatim text ceiling β bounds each stored row (no hoarding). */+export const MAX_TURN_CHARS = 8_000;++export interface RecentTurnInput {+ /** Speaker role, e.g. `user` | `assistant` (free-form, host-defined). */+ readonly role: string;+ /** Verbatim turn text. Truncated to {@link MAX_TURN_CHARS} on store. */+ readonly text: string;+ readonly sessionId?: string;+ readonly turnId?: string;+ /** Canonical UTC ISO-8601 timestamp; defaults to now when absent. */+ readonly createdAt?: string;+}++export interface RecentTurn {+ readonly role: string;+ readonly text: string;+ readonly createdAt: string;+ readonly seq: number;+ readonly sessionId?: string;+ readonly turnId?: string;+}++export interface ListRecentTurnsOptions {+ /** Max turns to return; clamped to [1, {@link RECENT_TURNS_CAP}]. */+ readonly limit?: number;+}++export interface ResurfaceOptions {+ /**+ * Opt-in gate. Default/absent β OFF: {@link resurfaceRecentTurns} returns+ * null and nothing is added to the post-compaction context (byte-identical).+ */+ readonly enabled?: boolean;+ /** Max turns to surface; clamped like {@link ListRecentTurnsOptions.limit}. */+ readonly limit?: number;+}++/**+ * Append one verbatim turn to the bounded buffer. The turn is persisted as a+ * `recent_turn` continuity record. A monotonic `seq` (the count of existing+ * `recent_turn` rows) stamps insertion order so same-timestamp turns keep a+ * deterministic, stable order on read.+ */+export function appendRecentTurn(vault: string, input: RecentTurnInput): ContinuityRecord {+ const seq = listContinuityRecords(vault, { kind: "recent_turn" }).length;+ const createdAt = input.createdAt ?? new Date().toISOString();+ const text = input.text.slice(0, MAX_TURN_CHARS);+ const sourceRefs: ContinuitySourceRef[] = [];+ if (input.sessionId !== undefined && input.sessionId.length > 0) {+ sourceRefs.push({ id: input.sessionId, kind: "session" });+ }+ return appendContinuityRecord(vault, {+ kind: "recent_turn",+ createdAt,+ sourceRefs,+ payload: {+ role: input.role,+ text,+ seq,+ ...(input.sessionId !== undefined && input.sessionId.length > 0+ ? { session_id: input.sessionId }+ : {}),+ ...(input.turnId !== undefined && input.turnId.length > 0 ? { turn_id: input.turnId } : {}),+ truncated: text.length < input.text.length,+ },+ });+}++/**+ * Read the newest turns in chronological order (oldest β newest). Never+ * returns more than {@link RECENT_TURNS_CAP}; the older overflow is evicted+ * from this view (the hard cap). Reads only the `recent_turn` kind β the+ * buffer is isolated from every curated surface.+ */+export function listRecentTurns(vault: string, opts: ListRecentTurnsOptions = {}): RecentTurn[] {+ const requested = opts.limit ?? RECENT_TURNS_CAP;+ const limit = Math.max(1, Math.min(RECENT_TURNS_CAP, Math.floor(requested)));+ const turns = listContinuityRecords(vault, { kind: "recent_turn" })+ .map(toRecentTurn)+ .filter((turn): turn is RecentTurn => turn !== null)+ .toSorted(compareTurns);+ return turns.slice(Math.max(0, turns.length - limit));+}++/**+ * Optional post-compaction re-surface. Returns a formatted verbatim block of+ * the recent buffer ONLY when opted in AND the buffer is non-empty; otherwise+ * null (nothing surfaced β byte-identical context). Callers gate `enabled`+ * with the `recent_turns_resurface` config flag (default off).+ */+export function resurfaceRecentTurns(vault: string, opts: ResurfaceOptions = {}): string | null {+ if (opts.enabled !== true) return null;+ const turns = listRecentTurns(vault, opts.limit !== undefined ? { limit: opts.limit } : {});+ if (turns.length === 0) return null;+ const lines = turns.map((turn) => `- **${turn.role}**: ${turn.text}`);+ return [`## Recent turns (verbatim, last ${turns.length})`, ...lines].join("\n");+}++function toRecentTurn(record: ContinuityRecord): RecentTurn | null {+ const payload = record.payload;+ const role = payload["role"];+ const text = payload["text"];+ if (typeof role !== "string" || typeof text !== "string") return null;+ const seq = typeof payload["seq"] === "number" ? payload["seq"] : 0;+ const sessionId = typeof payload["session_id"] === "string" ? payload["session_id"] : undefined;+ const turnId = typeof payload["turn_id"] === "string" ? payload["turn_id"] : undefined;+ return {+ role,+ text,+ createdAt: record.createdAt,+ seq,+ ...(sessionId !== undefined ? { sessionId } : {}),+ ...(turnId !== undefined ? { turnId } : {}),+ };+}++/** Deterministic chronological order: seq (insertion), then timestamp. */+function compareTurns(a: RecentTurn, b: RecentTurn): number {+ return a.seq - b.seq || a.createdAt.localeCompare(b.createdAt);+}diff --git a/src/core/brain/session-checkpoint.ts b/src/core/brain/session-checkpoint.tsnew file mode 100644index 00000000..c81a3f3e--- /dev/null+++ b/src/core/brain/session-checkpoint.ts@@ -0,0 +1,333 @@+/**+ * Batch session checkpoint (C4 / t_1a3a9eba).+ *+ * A whole session's extracted memories (taste signals), decisions,+ * learnings, an optional request/next-steps summary, and an optional+ * diary note, persisted in ONE call. OSB already exposes per-signal+ * writes and a session-summary digest, but closing a session cleanly+ * means several independent writes; a partial failure mid-way leaves the+ * vault in a half-captured state and a naive retry double-appends.+ *+ * This module makes an end-of-session save atomic FROM THE CALLER'S+ * PERSPECTIVE by giving the whole checkpoint a single idempotency key+ * (session id) over the C1 idempotency ledger:+ *+ * - A first checkpoint for a session writes every item and records the+ * session key -> whole-checkpoint content hash.+ * - A retry with the SAME session id + SAME content short-circuits to a+ * deduped no-op (`deduped: true`) BEFORE touching disk β the+ * multi-runtime retry / double-delivery case.+ * - A retry with the SAME session id + DIFFERENT content is an explicit+ * {@link IdempotencyPayloadMismatchError} (never a silent overwrite).+ *+ * Within a single checkpoint, each item is written independently and a+ * per-item failure (a malformed signal, an empty summary) is COLLECTED+ * into `partial` rather than aborting the batch β the result is+ * `status: "mixed"` and nothing is silently dropped. A mixed checkpoint+ * does NOT record the session key, so a corrected retry re-attempts the+ * items that need review while the already-written items dedupe through+ * their own per-item keys.+ *+ * The kernel stays deterministic: no LLM, no wall-clock beyond the+ * caller-supplied (or defaulted) `createdAt`.+ */++import { relative } from "node:path";++import { appendBrainNote } from "./note.ts";+import {+ computePayloadHash,+ IdempotencyPayloadMismatchError,+ lookupKey,+ rememberKey,+} from "./idempotency-ledger.ts";+import { appendSessionSummary, type SessionSummaryDigest } from "./session-summary.ts";+import { writeSignal, type WriteSignalOptions } from "./signal.ts";+import { isoDate, isoSecond } from "./time.ts";+import { slugify } from "../vault.ts";+import type { BrainSignalSign } from "./types.ts";++export const SESSION_CHECKPOINT_STATUS = Object.freeze({+ ok: "ok",+ mixed: "mixed",+} as const);++export type SessionCheckpointStatus =+ (typeof SESSION_CHECKPOINT_STATUS)[keyof typeof SESSION_CHECKPOINT_STATUS];++/** One extracted memory (taste signal) in a checkpoint batch. */+export interface CheckpointSignalInput {+ readonly topic: string;+ readonly signal: BrainSignalSign;+ readonly principle: string;+ readonly scope?: string;+ readonly raw?: string;+ readonly source?: ReadonlyArray<string>;+}++export interface SessionCheckpointInput {+ readonly sessionId: string;+ /** Identity stamped on every written signal / diary note. */+ readonly agent: string;+ /** Extracted memories to persist as `sig-*` signals. */+ readonly signals?: ReadonlyArray<CheckpointSignalInput>;+ /** Session summary digest fields (folded into one digest record). */+ readonly request?: string;+ readonly decisions?: ReadonlyArray<string>;+ readonly learnings?: ReadonlyArray<string>;+ readonly nextSteps?: ReadonlyArray<string>;+ readonly sourceTurnIds?: ReadonlyArray<string>;+ readonly host?: string;+ /** Optional narrative diary line appended to today's Brain log. */+ readonly diary?: string;+ /**+ * Canonical UTC ISO-8601 write time. Defaults to `isoSecond(new Date())`;+ * supplied by tests for deterministic fixtures.+ */+ readonly createdAt?: string;+}++/** One item that needs review β never silently dropped. */+export interface CheckpointPartialItem {+ readonly kind: "signal" | "summary" | "diary";+ readonly index?: number;+ readonly topic?: string;+ readonly reason: string;+}++export interface CheckpointSignalResult {+ readonly id: string;+ /** Vault-relative POSIX path of the written (or deduped) signal. */+ readonly path: string;+ /** True when this signal deduped against a prior write of the same key. */+ readonly deduped: boolean;+}++export interface SessionCheckpointResult {+ readonly status: SessionCheckpointStatus;+ readonly sessionId: string;+ /** True when the whole checkpoint deduped at the session key (a retry). */+ readonly deduped: boolean;+ readonly signals: ReadonlyArray<CheckpointSignalResult>;+ readonly summary: { readonly id: string } | null;+ readonly diaryWritten: boolean;+ readonly partial: ReadonlyArray<CheckpointPartialItem>;+}++export class SessionCheckpointError extends Error {+ constructor(message: string) {+ super(message);+ this.name = "SessionCheckpointError";+ }+}++/**+ * The whole-checkpoint idempotency key. Scoped to the session id so a+ * retry of the same session collides here, and DISTINCT from the C1+ * feedback keys (which are `<session>:<slug>` shaped) by the `checkpoint:`+ * prefix.+ */+function checkpointKey(sessionId: string): string {+ return `checkpoint:${sessionId}`;+}++/** Per-signal key so a mixed-then-corrected retry dedupes items already+ * written by the earlier attempt. Index-scoped: a stable slot within the+ * batch, disambiguated by the ledger's content hash. */+function signalKey(sessionId: string, index: number): string {+ return `${checkpointKey(sessionId)}:sig:${index}`;+}++/** The semantic identity of a whole checkpoint for idempotency hashing:+ * every item's content, never timestamps or allocated slugs. */+function checkpointPayloadFields(input: SessionCheckpointInput): Record<string, unknown> {+ return {+ session_id: input.sessionId.trim(),+ signals: (input.signals ?? []).map((s) => ({+ topic: s.topic,+ signal: s.signal,+ principle: s.principle,+ scope: s.scope,+ raw: s.raw,+ source: s.source ? [...s.source] : undefined,+ })),+ request: input.request,+ decisions: input.decisions ? [...input.decisions] : undefined,+ learnings: input.learnings ? [...input.learnings] : undefined,+ next_steps: input.nextSteps ? [...input.nextSteps] : undefined,+ diary: input.diary,+ };+}++function hasSummaryContent(input: SessionCheckpointInput): boolean {+ return (+ (input.request?.trim().length ?? 0) > 0 ||+ (input.decisions?.length ?? 0) > 0 ||+ (input.learnings?.length ?? 0) > 0 ||+ (input.nextSteps?.length ?? 0) > 0+ );+}++/**+ * Save a whole session's memories + summary (+ optional diary) in one+ * idempotent batch. See the module header for the idempotency contract.+ *+ * @throws SessionCheckpointError when the session id is empty or the+ * checkpoint carries no items at all.+ * @throws IdempotencyPayloadMismatchError when the session id was already+ * checkpointed with different content.+ */+export function saveSessionCheckpoint(+ vault: string,+ input: SessionCheckpointInput,+ writeOptions: WriteSignalOptions = {},+): SessionCheckpointResult {+ const sessionId = input.sessionId.trim();+ if (sessionId.length === 0) {+ throw new SessionCheckpointError("session checkpoint requires a non-empty session id");+ }+ const signals = input.signals ?? [];+ const summaryWanted = hasSummaryContent(input);+ if (signals.length === 0 && !summaryWanted && !input.diary?.trim()) {+ throw new SessionCheckpointError(+ "session checkpoint requires at least one signal, summary field, or diary line",+ );+ }++ const createdAt = input.createdAt ?? isoSecond(new Date());+ const key = checkpointKey(sessionId);+ const contentHash = computePayloadHash(checkpointPayloadFields(input));++ // Whole-checkpoint idempotency consult BEFORE any write: a matching+ // retry short-circuits to a deduped no-op; a differing payload for the+ // same session throws rather than silently overwriting the checkpoint.+ const existing = lookupKey(vault, key);+ if (existing) {+ if (existing.contentHash === contentHash) {+ const ref = (existing.ref ?? {}) as {+ signals?: ReadonlyArray<CheckpointSignalResult>;+ summary?: { id: string } | null;+ diaryWritten?: boolean;+ };+ return Object.freeze({+ status: SESSION_CHECKPOINT_STATUS.ok,+ sessionId,+ deduped: true,+ signals: Object.freeze([...(ref.signals ?? [])]),+ summary: ref.summary ?? null,+ diaryWritten: ref.diaryWritten ?? false,+ partial: Object.freeze([]),+ });+ }+ throw new IdempotencyPayloadMismatchError(key, existing.contentHash, contentHash);+ }++ const signalDate = isoDate(new Date(createdAt));+ const signalResults: CheckpointSignalResult[] = [];+ const partial: CheckpointPartialItem[] = [];++ signals.forEach((sig, index) => {+ try {+ const res = writeSignal(+ vault,+ {+ topic: sig.topic,+ signal: sig.signal,+ agent: input.agent,+ principle: sig.principle,+ created_at: createdAt,+ date: signalDate,+ slug: slugify(sig.topic),+ ...(sig.scope ? { scope: sig.scope } : {}),+ ...(sig.raw ? { raw: sig.raw } : {}),+ ...(sig.source && sig.source.length > 0 ? { source: [...sig.source] } : {}),+ idempotency_key: signalKey(sessionId, index),+ },+ writeOptions,+ );+ signalResults.push(+ Object.freeze({+ id: res.id,+ path: res.path ? relative(vault, res.path) : "",+ deduped: res.deduped === true,+ }),+ );+ } catch (err) {+ // A malformed signal or a per-item payload mismatch needs review β+ // collect it, never abort the whole batch.+ partial.push(+ Object.freeze({+ kind: "signal",+ index,+ topic: sig.topic,+ reason: err instanceof Error ? err.message : String(err),+ }),+ );+ }+ });++ let summary: { id: string } | null = null;+ if (summaryWanted) {+ try {+ const digest: SessionSummaryDigest = appendSessionSummary(vault, {+ sessionId,+ ...(input.request !== undefined ? { request: input.request } : {}),+ ...(input.decisions !== undefined ? { decisions: input.decisions } : {}),+ ...(input.learnings !== undefined ? { learnings: input.learnings } : {}),+ ...(input.nextSteps !== undefined ? { nextSteps: input.nextSteps } : {}),+ ...(input.sourceTurnIds !== undefined ? { sourceTurnIds: input.sourceTurnIds } : {}),+ ...(input.host !== undefined ? { host: input.host } : {}),+ createdAt,+ });+ summary = Object.freeze({ id: digest.id });+ } catch (err) {+ partial.push(+ Object.freeze({+ kind: "summary",+ reason: err instanceof Error ? err.message : String(err),+ }),+ );+ }+ }++ let diaryWritten = false;+ if (input.diary?.trim()) {+ try {+ appendBrainNote({ vault, text: input.diary, agent: input.agent });+ diaryWritten = true;+ } catch (err) {+ partial.push(+ Object.freeze({+ kind: "diary",+ reason: err instanceof Error ? err.message : String(err),+ }),+ );+ }+ }++ const status =+ partial.length === 0 ? SESSION_CHECKPOINT_STATUS.ok : SESSION_CHECKPOINT_STATUS.mixed;++ // Record the session key ONLY on a clean checkpoint. A mixed result+ // leaves the key unset so a corrected retry re-attempts the items that+ // needed review; the items that already landed dedupe via their own+ // per-item keys.+ if (status === SESSION_CHECKPOINT_STATUS.ok) {+ rememberKey(vault, {+ key,+ contentHash,+ createdAt,+ ref: { signals: signalResults, summary, diaryWritten },+ });+ }++ return Object.freeze({+ status,+ sessionId,+ deduped: false,+ signals: Object.freeze(signalResults),+ summary,+ diaryWritten,+ partial: Object.freeze(partial),+ });+}diff --git a/src/core/brain/sessions/import.ts b/src/core/brain/sessions/import.tsindex 7a7d2544..f94c887d 100644--- a/src/core/brain/sessions/import.ts+++ b/src/core/brain/sessions/import.ts@@ -88,6 +88,46 @@ export interface ImportSessionOptions {readonly filterRoles?: ReadonlyArray<SessionTurn["role"]>;/** Optional case-insensitive substring filter on turn text. */readonly filterTextIncludes?: string;+ /**+ * Per-row event-time backfill (A2 / t_7526e8d3). When true, each+ * emitted signal is stamped with its turn's ORIGINAL+ * `SessionTurn.timestamp` (`created_at` / `recorded_at` / `valid_from`+ * and the filename calendar day) instead of the import wall-clock, so+ * backfilling an old session log stays historically faithful and+ * recency-based reconciliation can trust the timestamps. A turn whose+ * timestamp is absent, unparseable, at/before the Unix epoch (the+ * adapter's "no timestamp" sentinel), or future-dated relative to+ * `now` falls back to `now` deterministically β no throw, no+ * future-dated or epoch signal. Default off β byte-identical to the+ * wall-clock path.+ */+ readonly preserveEventTime?: boolean;+}++/**+ * Resolve the effective stamp instant for an emitted signal under the+ * per-row event-time backfill (A2). Returns whether the instant came+ * from the turn (so the caller stamps the bi-temporal slots) or fell+ * back to `now` (so the file stays byte-identical to the wall-clock+ * path β no `recorded_at` / `valid_from` keys).+ *+ * The usable window is `epoch < ts <= now`: a timestamp at or before the+ * Unix epoch is the adapter's synthesized "no timestamp" sentinel+ * (`new Date(0)`), a future timestamp would mint a nonsensical+ * forward-dated signal during backfill, and an unparseable value is+ * `NaN` β all three fall back to `now`.+ */+export function resolveEventInstant(+ turnTimestamp: string | undefined,+ now: Date,+ preserve: boolean,+): { readonly instant: Date; readonly fromTurn: boolean } {+ if (!preserve || !turnTimestamp) return { instant: now, fromTurn: false };+ const ms = Date.parse(turnTimestamp);+ if (!Number.isFinite(ms) || ms <= 0 || ms > now.getTime()) {+ return { instant: now, fromTurn: false };+ }+ return { instant: new Date(ms), fromTurn: true };}export interface ImportSessionResult {@@ -202,6 +242,8 @@ export async function importSession(agent: string;note?: string;turnId: string;+ /** Origin turn's ISO timestamp; used for per-row event-time backfill. */+ eventTimestamp?: string;dedupHash: string;}): void => {if (dedup.has(input.dedupHash)) {@@ -215,15 +257,24 @@ export async function importSession(return;}const sessionRef = `${sessionKey}#${input.turnId}`;+ // Per-row event-time (A2): stamp the turn's original instant when+ // preservation is on and the timestamp is usable; otherwise fall+ // back to `now` and emit NO bi-temporal slots (byte-identical path).+ const { instant, fromTurn } = resolveEventInstant(+ input.eventTimestamp,+ now,+ opts.preserveEventTime === true,+ );try {const res = writeSignal(vault, {topic: input.topic,signal: input.signal,agent: input.agent,principle: input.principle,- created_at: isoSecond(now),- date: isoDate(now),+ created_at: isoSecond(instant),+ date: isoDate(instant),slug: input.topic,+ ...(fromTurn ? { recorded_at: isoSecond(instant), valid_from: isoSecond(instant) } : {}),...(input.scope ? { scope: input.scope } : {}),source: [`[[${sessionRef}]]`],source_type: BRAIN_SIGNAL_SOURCE_TYPE.session,@@ -334,6 +385,7 @@ export async function importSession(agent: m.agent ?? agentLabelForTurn(turn, adapter.id, opts.agent),...(m.note ? { note: m.note } : {}),turnId: turn.turnId,+ eventTimestamp: turn.timestamp,dedupHash: hash,});}@@ -364,6 +416,7 @@ export async function importSession(agent: v.agent ?? agentLabelForTurn(turn, adapter.id, opts.agent),...(v.raw ? { note: v.raw } : {}),turnId: call.id ?? turn.turnId,+ eventTimestamp: turn.timestamp,dedupHash: hash,});}diff --git a/src/core/brain/signal.ts b/src/core/brain/signal.tsindex 966efc5c..f698d5e2 100644--- a/src/core/brain/signal.ts+++ b/src/core/brain/signal.ts@@ -24,9 +24,18 @@* identical files β a property exercised by the roundtrip test.*/+import { join, relative } from "node:path";+import type { FrontmatterMap } from "../types.ts";import { sanitiseTextField } from "../redactor.ts";+import {+ computePayloadHash,+ IdempotencyPayloadMismatchError,+ lookupKey,+ rememberKey,+} from "./idempotency-ledger.ts";import { sanitisePrinciple } from "./text/sanitize-principle.ts";+import { normalizeExpirationDate } from "./expiration.ts";import { writeFrontmatterAtomic, parseFrontmatter } from "../vault.ts";import { compress, expand, CODEC_VERSION } from "./portability/codec.ts";import { allocateSlug, brainDirs, validateIsoDate } from "./paths.ts";@@ -91,6 +100,29 @@ export interface WriteSignalInput {* so existing signals stay byte-identical.*/readonly origin_vault?: string;+ /**+ * Bi-temporal event-time slots (A2 / t_7526e8d3). Additive and+ * optional: absent β the written file is byte-identical to the+ * historical path. When supplied they flow into the SAME frontmatter+ * keys the read-side (`readBiTemporal`) already parses, so no reader+ * change is needed. Backfill / batch-import stamps these with the+ * record's ORIGINAL event-time (`valid_from` = when it happened;+ * `recorded_at` = the record time recency reconciliation trusts) so+ * an imported old session stays historically faithful instead of+ * being mis-stamped with the import wall-clock.+ */+ readonly valid_from?: string;+ readonly recorded_at?: string;+ /**+ * Caller-settable expiration (C5 / t_a82b674e). ISO date+ * (`YYYY-MM-DD`) or full timestamp. Additive and optional: absent β+ * the write is byte-identical to the historical path. When present it+ * is validated and stamped into frontmatter; the default read/list+ * path drops a signal past this date unless the caller opts into+ * `showExpired`. Orthogonal to dream retirement β the file is never+ * deleted or moved on expiry.+ */+ readonly expiration_date?: string;/*** Vault portability suite (v0.22.0). Opt-in: when true and `raw` is* present, the body is stored through the deterministic codec and a@@ -98,6 +130,18 @@ export interface WriteSignalInput {* Default (absent/false) writes the raw body verbatim - byte-identical.*/readonly rawCodec?: boolean;+ /**+ * Client-supplied idempotency key (C1 / t_213f356b). Additive and+ * optional: absent β the write is byte-identical to the historical+ * path. When present, the writer hashes the SEMANTIC payload (topic,+ * sign, principle, scope, source, raw, capture fields β never the+ * timestamp/slug) and consults the idempotency ledger before writing.+ * A repeat with the same key + same payload is a deduped no-op; the+ * same key + a different payload throws+ * {@link IdempotencyPayloadMismatchError} rather than silently+ * appending a second, conflicting signal.+ */+ readonly idempotency_key?: string;}export interface WriteSignalOptions {@@ -136,6 +180,13 @@ export function resolveEffectiveScope(export interface WriteSignalResult {readonly path: string;readonly id: string;+ /**+ * Set to `true` when an `idempotency_key` matched a prior write with an+ * identical payload, so this call was a deduped no-op (no new file). The+ * returned `path`/`id` point at the ORIGINAL signal. Absent on a normal+ * (first) write, so existing callers are unaffected.+ */+ readonly deduped?: boolean;}export interface ParseSignalOptions {@@ -203,6 +254,29 @@ export function writeSignal();}+ // Idempotency consult (C1): when a client key is supplied, hash the+ // semantic payload and check the ledger BEFORE allocating a slug or+ // touching disk. A prior write with the same key short-circuits here β+ // matching payload dedupes (return the original coordinates), a+ // differing payload throws rather than writing a conflicting signal.+ const idKey = sanitised.idempotency_key?.trim();+ let contentHash: string | undefined;+ if (idKey) {+ contentHash = computePayloadHash(signalPayloadFields(sanitised));+ const existing = lookupKey(vault, idKey);+ if (existing) {+ if (existing.contentHash === contentHash) {+ const ref = (existing.ref ?? {}) as { id?: string; path?: string };+ return {+ path: ref.path ? join(vault, ref.path) : "",+ id: ref.id ?? "",+ deduped: true,+ };+ }+ throw new IdempotencyPayloadMismatchError(idKey, existing.contentHash, contentHash);+ }+ }+const dirs = brainDirs(vault);const allocated = allocateSlug({vault,@@ -259,6 +333,21 @@ export function writeSignal(if (sanitised.origin_vault && sanitised.origin_vault.trim()) {metadata["origin_vault"] = sanitised.origin_vault.trim();}+ // Bi-temporal event-time slots (A2): emitted only when supplied, so+ // legacy / live writes stay byte-identical. The read-side already+ // parses these keys via readBiTemporal.+ if (sanitised.valid_from && sanitised.valid_from.trim()) {+ metadata["valid_from"] = sanitised.valid_from.trim();+ }+ if (sanitised.recorded_at && sanitised.recorded_at.trim()) {+ metadata["recorded_at"] = sanitised.recorded_at.trim();+ }+ // Caller-settable expiration (C5): validated + emitted only when+ // supplied so legacy / live writes stay byte-identical. Reader-side+ // support is via parseSignal's optional read + the expiration filter.+ if (sanitised.expiration_date && sanitised.expiration_date.trim()) {+ metadata["expiration_date"] = normalizeExpirationDate(sanitised.expiration_date);+ }// Opt-in codec (v0.22.0): store the raw body compressed and stamp a// `_raw_codec` marker so the reader expands it. Default path is verbatim.@@ -278,9 +367,42 @@ export function writeSignal(vaultForRelativePath: vault,});+ // Record the key AFTER the file lands so a future retry dedupes. `ref`+ // stores the vault-relative path + id so the dedupe branch above can+ // return the original coordinates without re-deriving them.+ if (idKey && contentHash !== undefined) {+ rememberKey(vault, {+ key: idKey,+ contentHash,+ createdAt: sanitised.created_at,+ ref: { id, path: relative(vault, allocated.path) },+ });+ }+return { path: allocated.path, id };}+/**+ * The semantic identity of a signal for idempotency hashing: the fields+ * that make two signals "the same content". Timestamps, the calendar+ * date, the allocated slug, and portability markers are intentionally+ * excluded so a retry with a later wall-clock still dedupes.+ */+function signalPayloadFields(input: WriteSignalInput): Record<string, unknown> {+ return {+ topic: input.topic.trim(),+ signal: input.signal,+ agent: input.agent.trim(),+ principle: input.principle.trim(),+ scope: input.scope?.trim(),+ source: input.source ? [...input.source] : undefined,+ raw: input.raw,+ source_type: input.source_type,+ schema_type: input.schema_type?.trim(),+ session_ref: input.session_ref?.trim(),+ };+}+// ----- Sanitisation --------------------------------------------------------/**@@ -437,6 +559,9 @@ export function parseSignal(path: string, options: ParseSignalOptions = {}): Bra...(dedup_hash !== undefined ? { dedup_hash } : {}),...(session_ref !== undefined ? { session_ref } : {}),...readBiTemporal(meta, path),+ ...(readOptionalTrimmedString(meta, "expiration_date", path) !== undefined+ ? { expiration_date: readOptionalTrimmedString(meta, "expiration_date", path) }+ : {}),};return Object.freeze(result);}diff --git a/src/core/brain/source-cleanup.ts b/src/core/brain/source-cleanup.tsnew file mode 100644index 00000000..c753a66f--- /dev/null+++ b/src/core/brain/source-cleanup.ts@@ -0,0 +1,546 @@+/**+ * Delete and search by exact source file (C6 / t_edde2198).+ *+ * When a benchmark file, log, or accidental import pollutes a Brain, the+ * operator needs to (a) find every derived entry that traces back to one+ * EXACT source path and (b) surgically remove those entries β including+ * the ingest index artifacts (the per-source summary page and the+ * content-manifest entry) β without re-mining the vault or hand-chasing+ * summary pages.+ *+ * Two surfaces, one shared tracer:+ * - {@link searchBySourceFile} β read-only. Every Brain page that traces+ * to the exact source.+ * - {@link deleteBySource} β DRY-RUN BY DEFAULT. Reports the blast radius+ * and deletes nothing. With `confirm`, removes the derived entries and+ * the index artifacts; original user notes are removed ONLY with an+ * explicit `includeOriginals`. Every confirmed cleanup writes a+ * `source_invalidation` continuity record (auditable).+ *+ * Provenance a page can carry back to a source:+ * - frontmatter `source_path` (the ingest summary page, kind+ * `brain-source`),+ * - frontmatter `session_ref` (a session-derived signal),+ * - a `[[source]]` wikilink in the frontmatter `source` array or in the+ * page body's `## Sources` provenance section,+ * - a preference `evidenced_by` link to a signal that itself derives+ * from the source (transitive fold).+ *+ * Blast-radius safety. A page is only auto-DELETED when it is a+ * single-purpose derived file that traces SOLELY to this source:+ * - a per-item derivation page (summary / signal / preference / entity /+ * retired), AND+ * - it co-references no OTHER source (a signal citing a second source, or+ * a preference folded from a second signal, is REPORTED, never+ * deleted).+ * Everything else that merely mentions the source β daily logs, research+ * reports, MOCs, any aggregate surface β is reported as a protected+ * mention so the operator can hand-edit it. This is the "never remove more+ * than intended" guarantee the dry-run report exists to make good on.+ */++import { existsSync, readFileSync, readdirSync, rmSync, statSync } from "node:fs";+import { join, sep } from "node:path";++import type { FrontmatterMap } from "../types.ts";+import { canonicalNotePath, ensureInsideVault, vaultRelative } from "../path-safety.ts";+import { parseFrontmatter } from "../vault.ts";+import {+ BRAIN_ENTITIES_REL,+ BRAIN_INBOX_REL,+ BRAIN_PREFERENCES_REL,+ BRAIN_REPORTS_REL,+ BRAIN_RETIRED_REL,+ BRAIN_SOURCES_REL,+ brainDirs,+} from "./paths.ts";+import { BRAIN_SOURCE_KIND } from "./ingest/ingest.ts";+import { manifestPath, readManifest, writeManifestAtomic } from "./ingest/content-manifest.ts";+import { appendContinuitySourceInvalidation } from "./continuity/store.ts";+import { isoSecond } from "./time.ts";++/** How a page was found to trace back to the source. */+export type SourceCleanupMatch = "source_path" | "session_ref" | "wikilink" | "evidenced_by";++/** Coarse classification of the referencing page, by its home directory. */+export type SourceCleanupKind =+ | "summary"+ | "signal"+ | "preference"+ | "entity"+ | "retired"+ | "report"+ | "log"+ | "other";++export interface SourceCleanupEntry {+ /** Vault-relative POSIX path of the referencing page. */+ readonly path: string;+ /** Frontmatter `id` when present, else the file basename (no extension). */+ readonly id: string;+ readonly kind: SourceCleanupKind;+ readonly match: SourceCleanupMatch;+ /** True for the ingest summary page β an index artifact, always purged. */+ readonly isIndexArtifact: boolean;+ /**+ * True when the page is a single-purpose derived file tracing SOLELY to+ * this source and is therefore safe to auto-delete on `confirm`. False+ * for aggregate surfaces and shared-derivation pages (reported only).+ */+ readonly deletable: boolean;+}++export interface SourceCleanupPlan {+ /** Canonical form of the queried source. */+ readonly source: string;+ /** False on a dry run (the default); true only when `confirm` was set. */+ readonly confirmed: boolean;+ readonly includeOriginals: boolean;+ /** Single-purpose derived entries removed (or that would be) on confirm. */+ readonly derived: ReadonlyArray<SourceCleanupEntry>;+ /** Referencing pages reported but NOT auto-deleted (aggregates / shared). */+ readonly mentions: ReadonlyArray<SourceCleanupEntry>;+ /** Original source file(s) outside `Brain/`; removed only with includeOriginals. */+ readonly originals: ReadonlyArray<string>;+ /** Manifest key for the source, when the content manifest tracks it. */+ readonly manifestEntry: string | null;+ /** Paths actually removed this run (empty on a dry run). */+ readonly deleted: ReadonlyArray<string>;+ readonly manifestEntryRemoved: boolean;+ /** Id of the audit continuity record, when one was written. */+ readonly auditRecordId: string | null;+ /** Total referencing pages + originals (the reported blast radius). */+ readonly blastRadius: number;+}++export interface DeleteBySourceOptions {+ /** Required true to actually delete; absent/false = dry run. */+ readonly confirm?: boolean;+ /** Also remove the original source file(s) outside `Brain/`. */+ readonly includeOriginals?: boolean;+ /** Injected clock for the audit record's timestamp. */+ readonly now?: Date;+ /** Agent identity recorded in the audit reason. */+ readonly agent?: string;+}++/** Directories whose pages are per-item, single-source derived artifacts. */+const DERIVATION_DIRS: ReadonlyArray<string> = Object.freeze([+ BRAIN_SOURCES_REL,+ BRAIN_INBOX_REL,+ BRAIN_PREFERENCES_REL,+ BRAIN_ENTITIES_REL,+ BRAIN_RETIRED_REL,+]);++function escapeRegExp(value: string): string {+ return value.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");+}++/**+ * A wikilink to exactly `target`: `[[target]]`, `[[target|alias]]`, or+ * `[[target#heading]]`. Anchored on the `[[` boundary so `[[foo]]` never+ * matches `[[foobar]]`.+ */+function wikilinkRegExp(target: string): RegExp {+ return new RegExp(`\\[\\[${escapeRegExp(target)}(?:[|#][^\\]]*)?\\]\\]`);+}++function stringField(meta: FrontmatterMap, key: string): string | null {+ const value = meta[key];+ return typeof value === "string" && value.length > 0 ? value : null;+}++function stringArrayField(meta: FrontmatterMap, key: string): ReadonlyArray<string> {+ const value = meta[key];+ if (!Array.isArray(value)) return [];+ return value.filter((item): item is string => typeof item === "string");+}++/**+ * A preference's evidence links. The writer stores the field under the+ * managed `_evidenced_by` key (with a plain `evidenced_by` on legacy+ * pages), so consult both.+ */+function evidencedByLinks(meta: FrontmatterMap): ReadonlyArray<string> {+ const managed = stringArrayField(meta, "_evidenced_by");+ return managed.length > 0 ? managed : stringArrayField(meta, "evidenced_by");+}++/** Strip a single enclosing `[[ β¦ ]]` and any `|alias` / `#heading` tail. */+function wikilinkTarget(raw: string): string {+ const m = /^\[\[([^\]|#]+)(?:[|#][^\]]*)?\]\]$/.exec(raw.trim());+ return m ? m[1]!.trim() : raw.trim();+}++function classifyKind(vault: string, absPath: string): SourceCleanupKind {+ const dirs = brainDirs(vault);+ if (insideDir(absPath, join(vault, BRAIN_SOURCES_REL))) return "summary";+ if (insideDir(absPath, dirs.retired)) return "retired";+ if (insideDir(absPath, dirs.preferences)) return "preference";+ if (insideDir(absPath, dirs.inbox)) return "signal";+ if (insideDir(absPath, dirs.entities)) return "entity";+ if (insideDir(absPath, join(vault, BRAIN_REPORTS_REL))) return "report";+ if (insideDir(absPath, dirs.log)) return "log";+ return "other";+}++function insideDir(path: string, dir: string): boolean {+ return path === dir || path.startsWith(`${dir}${sep}`);+}++function inDerivationDir(vault: string, absPath: string): boolean {+ return DERIVATION_DIRS.some((rel) => insideDir(absPath, join(vault, rel)));+}++function readId(meta: FrontmatterMap, absPath: string): string {+ const id = stringField(meta, "id") ?? stringField(meta, "entity_id");+ if (id) return id;+ const base = absPath.slice(absPath.lastIndexOf(sep) + 1);+ return base.endsWith(".md") ? base.slice(0, -3) : base;+}++interface RawMatch {+ readonly absPath: string;+ readonly path: string;+ readonly id: string;+ readonly kind: SourceCleanupKind;+ readonly match: SourceCleanupMatch;+ readonly isIndexArtifact: boolean;+ /** Structured source links this page carries (signal `source` array). */+ readonly sourceLinks: ReadonlyArray<string>;+ /** Structured evidence links this page carries (preference `evidenced_by`). */+ readonly evidencedBy: ReadonlyArray<string>;+}++/**+ * Classify one Brain page against the canonical source. Returns null when+ * the page does not reference the source at all. Only DIRECT references are+ * detected here (source_path / session_ref / `[[source]]` wikilink); the+ * transitive `evidenced_by` fold is resolved in a second pass once the set+ * of directly-derived signals is known.+ */+function directMatch(+ vault: string,+ absPath: string,+ canonical: string,+ sourceFile: string,+): RawMatch | null {+ let text: string;+ try {+ text = readFileSync(absPath, "utf8");+ } catch {+ return null;+ }+ let meta: FrontmatterMap = {};+ try {+ [meta] = parseFrontmatter(absPath);+ } catch {+ // A page we cannot parse is scanned as raw text only.+ meta = {};+ }++ const kind = classifyKind(vault, absPath);+ const sourceLinks = stringArrayField(meta, "source");+ const evidencedBy = stringArrayField(meta, "evidenced_by");+ const base = {+ absPath,+ path: vaultRelative(absPath, vault),+ id: readId(meta, absPath),+ kind,+ sourceLinks,+ evidencedBy,+ };++ const sourcePath = stringField(meta, "source_path");+ if (sourcePath !== null && canonicalNotePath(sourcePath) === canonical) {+ return {+ ...base,+ match: "source_path",+ isIndexArtifact: meta["kind"] === BRAIN_SOURCE_KIND,+ };+ }++ const sessionRef = stringField(meta, "session_ref");+ if (sessionRef !== null && (sessionRef === canonical || sessionRef === sourceFile)) {+ return { ...base, match: "session_ref", isIndexArtifact: false };+ }++ if (wikilinkRegExp(canonical).test(text)) {+ return { ...base, match: "wikilink", isIndexArtifact: false };+ }++ return null;+}++/**+ * A directly-derived page traces SOLELY to this source when it declares no+ * OTHER structured source link. A signal whose `source` array names a+ * second source is a shared observation β reported, never auto-deleted.+ */+function tracesSolelyToSource(raw: RawMatch, canonical: string): boolean {+ for (const link of raw.sourceLinks) {+ if (canonicalNotePath(wikilinkTarget(link)) !== canonical) return false;+ }+ return true;+}++interface Traced {+ readonly derived: ReadonlyArray<SourceCleanupEntry>;+ readonly mentions: ReadonlyArray<SourceCleanupEntry>;+}++/**+ * Walk `Brain/` and classify every page referencing the source into the+ * deletable-derived set and the reported-only mentions set. Deterministic:+ * both lists are sorted by vault-relative path.+ */+function traceReferences(vault: string, sourceFile: string): Traced {+ const canonical = canonicalNotePath(sourceFile);+ const root = brainDirs(vault).brain;+ const rawMatches: RawMatch[] = [];+ if (existsSync(root)) {+ for (const absPath of walkBrainMarkdown(root)) {+ const m = directMatch(vault, absPath, canonical, sourceFile);+ if (m !== null) rawMatches.push(m);+ }+ }++ // Second pass: a preference folded SOLELY from directly-derived signals+ // is itself a derived entry. One evidenced by any foreign signal is a+ // shared fold β reported, never auto-deleted.+ const derivedSignalIds = new Set(rawMatches.filter((m) => m.kind === "signal").map((m) => m.id));+ const alreadyMatched = new Set(rawMatches.map((m) => m.absPath));+ if (existsSync(root)) {+ for (const absPath of walkBrainMarkdown(root)) {+ if (alreadyMatched.has(absPath)) continue;+ const kind = classifyKind(vault, absPath);+ if (kind !== "preference") continue;+ let meta: FrontmatterMap;+ try {+ [meta] = parseFrontmatter(absPath);+ } catch {+ continue;+ }+ const evidencedBy = evidencedByLinks(meta);+ const targets = evidencedBy.map((link) => wikilinkTarget(link));+ if (targets.length === 0) continue;+ if (!targets.some((t) => derivedSignalIds.has(t))) continue;+ rawMatches.push({+ absPath,+ path: vaultRelative(absPath, vault),+ id: readId(meta, absPath),+ kind,+ match: "evidenced_by",+ isIndexArtifact: false,+ sourceLinks: [],+ evidencedBy,+ });+ }+ }++ const derived: SourceCleanupEntry[] = [];+ const mentions: SourceCleanupEntry[] = [];+ for (const raw of rawMatches) {+ const deletable = computeDeletable(vault, raw, canonical, derivedSignalIds);+ const entry: SourceCleanupEntry = {+ path: raw.path,+ id: raw.id,+ kind: raw.kind,+ match: raw.match,+ isIndexArtifact: raw.isIndexArtifact,+ deletable,+ };+ (deletable ? derived : mentions).push(entry);+ }+ return { derived: derived.toSorted(byPath), mentions: mentions.toSorted(byPath) };+}++function byPath(a: SourceCleanupEntry, b: SourceCleanupEntry): number {+ return a.path.localeCompare(b.path);+}++function computeDeletable(+ vault: string,+ raw: RawMatch,+ canonical: string,+ derivedSignalIds: ReadonlySet<string>,+): boolean {+ // The ingest summary page is always an index artifact to purge.+ if (raw.isIndexArtifact) return true;+ // Only per-item derivation pages are ever auto-deleted.+ if (!inDerivationDir(vault, raw.absPath)) return false;+ // A signal citing a second source is a shared observation β report it.+ if (!tracesSolelyToSource(raw, canonical)) return false;+ // A preference folded from any foreign signal is a shared fold β report it.+ if (raw.match === "evidenced_by") {+ const targets = raw.evidencedBy.map((link) => wikilinkTarget(link));+ if (!targets.every((t) => derivedSignalIds.has(t))) return false;+ }+ return true;+}++function walkBrainMarkdown(root: string): string[] {+ const out: string[] = [];+ const visit = (dir: string): void => {+ let names: string[];+ try {+ names = readdirSync(dir);+ } catch {+ return;+ }+ for (const name of names) {+ if (name.startsWith(".")) continue;+ const full = join(dir, name);+ let st;+ try {+ st = statSync(full);+ } catch {+ continue;+ }+ if (st.isDirectory()) {+ visit(full);+ continue;+ }+ if (st.isFile() && name.endsWith(".md")) out.push(full);+ }+ };+ visit(root);+ return out.toSorted();+}++/**+ * Every Brain page tracing back to the EXACT source β the derived entries+ * and the protected mentions, merged and sorted by path. Read-only.+ */+export function searchBySourceFile(+ vault: string,+ sourceFile: string,+): ReadonlyArray<SourceCleanupEntry> {+ const { derived, mentions } = traceReferences(vault, sourceFile);+ return Object.freeze([...derived, ...mentions].toSorted(byPath));+}++/**+ * Original source file(s) outside `Brain/`: the imported artifact itself.+ * A source that lives inside `Brain/` (or that has no on-disk file β e.g. a+ * URL identity) contributes no original.+ */+function findOriginals(vault: string, canonical: string): string[] {+ let abs: string;+ try {+ abs = ensureInsideVault(join(vault, canonical), vault);+ } catch {+ return [];+ }+ if (!existsSync(abs)) return [];+ try {+ if (!statSync(abs).isFile()) return [];+ } catch {+ return [];+ }+ if (insideDir(abs, brainDirs(vault).brain)) return [];+ return [canonical];+}++function removeFile(vault: string, relPath: string): boolean {+ let abs: string;+ try {+ abs = ensureInsideVault(join(vault, relPath), vault);+ } catch {+ return false;+ }+ try {+ rmSync(abs);+ return true;+ } catch (err) {+ if ((err as NodeJS.ErrnoException)?.code === "ENOENT") return false;+ throw err;+ }+}++/**+ * Plan and (with `confirm`) execute the removal of everything derived from+ * one exact source file. Dry-run by default.+ */+export function deleteBySource(+ vault: string,+ sourceFile: string,+ opts: DeleteBySourceOptions = {},+): SourceCleanupPlan {+ const canonical = canonicalNotePath(sourceFile);+ const confirm = opts.confirm === true;+ const includeOriginals = opts.includeOriginals === true;++ const { derived, mentions } = traceReferences(vault, sourceFile);+ const originals = findOriginals(vault, canonical);+ const manifestKey = readManifest(vault).entries[canonical] !== undefined ? canonical : null;+ const blastRadius = derived.length + mentions.length + originals.length;++ if (!confirm) {+ return Object.freeze({+ source: canonical,+ confirmed: false,+ includeOriginals,+ derived: Object.freeze(derived),+ mentions: Object.freeze(mentions),+ originals: Object.freeze(originals),+ manifestEntry: manifestKey,+ deleted: Object.freeze([]),+ manifestEntryRemoved: false,+ auditRecordId: null,+ blastRadius,+ });+ }++ // Confirmed path β delete the derived entries first, then (opt-in) the+ // originals, then drop the manifest entry (an index artifact).+ const deleted: string[] = [];+ for (const entry of derived) {+ if (removeFile(vault, entry.path)) deleted.push(entry.path);+ }+ if (includeOriginals) {+ for (const original of originals) {+ if (removeFile(vault, original)) deleted.push(original);+ }+ }++ let manifestEntryRemoved = false;+ if (manifestKey !== null && existsSync(manifestPath(vault))) {+ const entries = { ...readManifest(vault).entries };+ if (entries[manifestKey] !== undefined) {+ delete entries[manifestKey];+ manifestEntryRemoved = writeManifestAtomic(vault, entries);+ }+ }++ // Audit only a run that actually changed something; a no-op re-run writes+ // no record (idempotent).+ let auditRecordId: string | null = null;+ if (deleted.length > 0 || manifestEntryRemoved) {+ const agent = opts.agent?.trim() ? opts.agent.trim() : "delete_by_source";+ const record = appendContinuitySourceInvalidation(vault, {+ createdAt: isoSecond(opts.now ?? new Date()),+ source: { id: canonical, path: canonical, kind: "source" },+ reason:+ `${agent}: removed ${deleted.length} derived/original entr${deleted.length === 1 ? "y" : "ies"}` ++ `${manifestEntryRemoved ? " and the ingest manifest entry" : ""}` ++ `${includeOriginals ? " (originals included)" : ""}`,+ });+ auditRecordId = record.id;+ }++ return Object.freeze({+ source: canonical,+ confirmed: true,+ includeOriginals,+ derived: Object.freeze(derived),+ mentions: Object.freeze(mentions),+ originals: Object.freeze(originals),+ manifestEntry: manifestKey,+ deleted: Object.freeze(deleted),+ manifestEntryRemoved,+ auditRecordId,+ blastRadius,+ });+}diff --git a/src/core/brain/truth/grounding.ts b/src/core/brain/truth/grounding.tsnew file mode 100644index 00000000..f7c82c64--- /dev/null+++ b/src/core/brain/truth/grounding.ts@@ -0,0 +1,201 @@+/**+ * Signed source-diversity grounding score (t_4678a91a / D1).+ *+ * A pure projection over the same `ClaimEvent`s that `computeTruthState`+ * folds. Where `conflicts.ts` marks a slot with a BINARY `contested`+ * flag, this module quantifies it: a signed score on [-1, +1] whose+ * SIGN points to the better-supported side (+ = the slot's current+ * value is confirmed, β = a contesting alternative is better supported)+ * and whose MAGNITUDE reflects both the balance of confirming vs+ * contradicting evidence AND how many INDEPENDENT sources back that+ * direction. A separate `confidence` band captures sufficiency.+ *+ * Kappa Graph's core insight (aaronsb/knowledge-graph-system): evidence+ * is weighed by SOURCE DIVERSITY, not raw mention count. N mentions in+ * one document weigh far below N mentions across N independent sources.+ * We honour that with a per-source repeat-mention ceiling: a single+ * source can never outweigh two independent ones.+ *+ * Deterministic (counting + weighting, no LLM), order-insensitive, and+ * never mutates the append-only ledger - exactly the `conflicts.ts`+ * discipline.+ */++import { CONFLICT_WINDOW_DAYS } from "./conflicts.ts";+import { normalizeClaimValue, slotKey } from "./fold.ts";+import type { ClaimEvent, ClaimSlot, TruthState } from "./types.ts";++const DAY_MS = 24 * 60 * 60 * 1000;++/**+ * Saturation constant for the diversity factor: `n / (n + K)`. With+ * K = 2, two independent confirming sources reach 0.5 and the factor+ * approaches 1 as sources accumulate - so uncontested agreement grows+ * with diversity but stays bounded below 1.+ */+export const SOURCE_SATURATION = 2;++/**+ * Ceiling on the bonus a single source earns by repeating itself. One+ * source asserting a value M times contributes `1 + CEILINGΒ·(1 β 1/M)`,+ * i.e. at most `1 + CEILING`. With CEILING = 0.5 a source maxes at 1.5,+ * so it can never outweigh two independent sources (weight 2) - the+ * source-diversity guarantee.+ */+export const REPEAT_MENTION_CEILING = 0.5;++/** Score >= this bands as `strongly_supported`. */+export const STRONGLY_SUPPORTED_MIN = 0.5;+/** Score in [MIXED_MIN, STRONGLY_SUPPORTED_MIN) bands as `mixed`. */+export const MIXED_MIN = 0.15;+/** Score in [CONTESTED_MIN, MIXED_MIN) bands as `contested`; below β `contradicted`. */+export const CONTESTED_MIN = -0.15;++/** Independent-source counts that earn each confidence band (sufficiency). */+export const CONFIDENCE_HIGH_MIN_SOURCES = 3;+export const CONFIDENCE_MEDIUM_MIN_SOURCES = 2;++export type GroundingBand = "strongly_supported" | "mixed" | "contested" | "contradicted";+export type GroundingConfidence = "high" | "medium" | "low";++export interface GroundingScore {+ /** Signed grounding on [-1, +1]; + confirms the current value, β contradicts it. */+ readonly score: number;+ /** Sufficiency band over the count of independent sources. */+ readonly confidence: GroundingConfidence;+ /** Kappa band derived from {@link score}. */+ readonly band: GroundingBand;+ /** Distinct (source, agent) pairs asserting the slot's current value. */+ readonly supportingSources: number;+ /** Distinct (source, agent) pairs asserting a contesting value within the window. */+ readonly contradictingSources: number;+}++export interface SlotGrounding {+ readonly entity: string;+ readonly aspect: string;+ readonly grounding: GroundingScore;+}++export interface GroundingOptions {+ /** Window (days) within which an opposing value counts as contradiction. */+ readonly windowDays?: number;+}++function withinWindow(aTs: string, bTs: string, windowDays: number): boolean {+ const delta = Math.abs(Date.parse(aTs) - Date.parse(bTs));+ return Number.isFinite(delta) && delta <= windowDays * DAY_MS;+}++/** Identity of one independent source: its provenance path plus asserting agent. */+function sourceIdentity(e: ClaimEvent): string {+ return `${e.source}\n${e.agent}`;+}++/** Diminishing per-source weight: 1 for a single mention, capped at 1 + ceiling. */+function sourceWeight(mentionCount: number): number {+ return 1 + REPEAT_MENTION_CEILING * (1 - 1 / mentionCount);+}++function bandFor(score: number): GroundingBand {+ if (score >= STRONGLY_SUPPORTED_MIN) return "strongly_supported";+ if (score >= MIXED_MIN) return "mixed";+ if (score >= CONTESTED_MIN) return "contested";+ return "contradicted";+}++function confidenceFor(independentSources: number): GroundingConfidence {+ if (independentSources >= CONFIDENCE_HIGH_MIN_SOURCES) return "high";+ if (independentSources >= CONFIDENCE_MEDIUM_MIN_SOURCES) return "medium";+ return "low";+}++/** Sum weights over sources in a stable (sorted-key) order so FP addition is order-insensitive. */+function totalWeight(mentionsBySource: Map<string, number>): number {+ let sum = 0;+ for (const key of [...mentionsBySource.keys()].toSorted()) {+ sum += sourceWeight(mentionsBySource.get(key)!);+ }+ return sum;+}++/**+ * Compute the signed source-diversity grounding score for one slot from+ * the raw claim events that feed it. Confirming events (matching the+ * slot's current value) count as support; opposing events asserted+ * within `windowDays` of the current value count as contradiction - a+ * value superseded outside the window is normal fact evolution, not a+ * contradiction (mirroring `conflicts.ts`). A source that ever asserts+ * the current value is a supporter, never a contradictor (self-+ * correction, not conflict). Pure and order-insensitive; the ledger is+ * never touched.+ */+export function computeGroundingScore(+ slot: ClaimSlot,+ events: ReadonlyArray<ClaimEvent>,+ opts: GroundingOptions = {},+): GroundingScore {+ const windowDays = opts.windowDays ?? CONFLICT_WINDOW_DAYS;+ const key = slotKey(slot.entity, slot.aspect);+ const currentNorm = normalizeClaimValue(slot.current.value);+ const currentTs = slot.current.ts;++ const supportMentions = new Map<string, number>();+ const contradictMentions = new Map<string, number>();++ for (const e of events) {+ if (slotKey(e.entity, e.aspect) !== key) continue;+ const id = sourceIdentity(e);+ if (normalizeClaimValue(e.value) === currentNorm) {+ supportMentions.set(id, (supportMentions.get(id) ?? 0) + 1);+ } else if (withinWindow(e.ts, currentTs, windowDays)) {+ contradictMentions.set(id, (contradictMentions.get(id) ?? 0) + 1);+ }+ }++ // A source that also supports the current value is self-correcting,+ // not a contradictor - drop it from the contradiction side.+ for (const id of supportMentions.keys()) contradictMentions.delete(id);++ const supportingSources = supportMentions.size;+ const contradictingSources = contradictMentions.size;+ const supportWeight = totalWeight(supportMentions);+ const contradictWeight = totalWeight(contradictMentions);+ const totalW = supportWeight + contradictWeight;++ const balance = totalW === 0 ? 0 : (supportWeight - contradictWeight) / totalW;+ const independentSources = supportingSources + contradictingSources;+ const diversityFactor = independentSources / (independentSources + SOURCE_SATURATION);+ const raw = balance * diversityFactor;+ // Round to tame FP noise so shuffled inputs compare exactly equal.+ const score = Math.round(raw * 1e6) / 1e6;++ return Object.freeze({+ score,+ confidence: confidenceFor(independentSources),+ band: bandFor(score),+ supportingSources,+ contradictingSources,+ });+}++/**+ * Surface the grounding projection alongside a folded {@link TruthState}:+ * one {@link SlotGrounding} per slot, in slot order. Additive - the fold+ * and its CONTESTED flags are unchanged; the score merely quantifies them.+ */+export function computeGroundings(+ state: TruthState,+ events: ReadonlyArray<ClaimEvent>,+ opts: GroundingOptions = {},+): ReadonlyArray<SlotGrounding> {+ return Object.freeze(+ state.slots.map((slot) =>+ Object.freeze({+ entity: slot.entity,+ aspect: slot.aspect,+ grounding: computeGroundingScore(slot, events, opts),+ }),+ ),+ );+}diff --git a/src/core/brain/types.ts b/src/core/brain/types.tsindex a2f60f2f..a62e7f8a 100644--- a/src/core/brain/types.ts+++ b/src/core/brain/types.ts@@ -396,6 +396,14 @@ export interface BrainSignal {readonly valid_until?: string;/** Bi-temporal transaction-time (additive optional, v0.10.18). */readonly recorded_at?: string;+ /**+ * Caller-settable expiration (C5 / t_a82b674e). ISO date (`YYYY-MM-DD`)+ * or full timestamp. Additive optional β absent on legacy signals. The+ * default read/list path drops a signal past this date unless the+ * caller opts into `showExpired`. Orthogonal to dream retirement: a+ * lapsed signal is filtered on read, never deleted or moved.+ */+ readonly expiration_date?: string;}/**@@ -511,6 +519,15 @@ export interface BrainPreference {* which is the dream-pass promotion moment). Additive optional.*/readonly recorded_at?: string;+ /**+ * Caller-settable expiration (C5 / t_a82b674e). ISO date (`YYYY-MM-DD`)+ * or full timestamp. Additive optional β absent on legacy preferences.+ * The default read/list path drops a preference past this date unless+ * the caller opts into `showExpired`. Orthogonal to dream's heuristic+ * retirement: a lapsed preference is filtered on read, never moved to+ * `Brain/retired/` (audit trail preserved).+ */+ readonly expiration_date?: string;}/**diff --git a/src/core/config.ts b/src/core/config.tsindex 1b241db8..3a1a85e5 100644--- a/src/core/config.ts+++ b/src/core/config.ts@@ -383,6 +383,21 @@ export function resolvePostCompactSurvivalAudit(configPath?: string): boolean {);}+/**+ * Post-compaction verbatim recent-turns re-surface gate (C3 / t_92317f91).+ * Default OFF: the bounded last-N-turns buffer is captured and readable on+ * demand, but nothing is auto-surfaced into the post-compaction context+ * unless `recent_turns_resurface: "true"`, so unchanged installs stay+ * byte-identical.+ */+export function resolveRecentTurnsResurface(configPath?: string): boolean {+ return resolveConfigFlag(+ "OPEN_SECOND_BRAIN_RECENT_TURNS_RESURFACE",+ "recent_turns_resurface",+ configPath,+ );+}+/*** SessionEnd handoff-note gate (Agent Surface Suite, t_28afa4d2).* Default OFF: lifecycle capture writes no handoff note unlessdiff --git a/src/core/install/adapters/aider-wrapper.ts b/src/core/install/adapters/aider-wrapper.tsnew file mode 100644index 00000000..8ec71823--- /dev/null+++ b/src/core/install/adapters/aider-wrapper.ts@@ -0,0 +1,181 @@+/**+ * Aider session-bracketing memory wrapper (E1).+ *+ * Aider has no native MCP client, so the static adapter (`aider.ts`) can only+ * inject a one-time context sidecar at install time and captures nothing back.+ * This module adds the live lifecycle Hindsight's Aider integration solved with+ * a wrapper process: it BRACKETS an interactive Aider session β+ *+ * 1. LOAD-HALF (mirrors Hermes `provider.py` `prefetch()`): regenerate the+ * context sidecar fresh at session start and inject it into Aider's read+ * context β live load, not a stale install-time snapshot.+ * 2. Exec the Aider binary bracketed between load and persist.+ * 3. WRITE-BACK HALF (mirrors `sync_turn()` / `on_session_end()`): capture the+ * session transcript at session end and persist it into the Brain via the+ * deterministic `pre_compact_extract` path β closing the write-back half+ * the static adapter omits for Aider entirely.+ *+ * The static sidecar (`aider.ts`) stays the documented fallback for users who+ * do not run through the wrapper. Both share ONE snapshot renderer+ * (`renderAiderSidecar`) so their sidecar bytes never drift (DRY).+ *+ * The orchestrator (`bracketAiderSession`) takes injected deps so the+ * lifecycle is unit-testable without a real Aider binary; the CLI+ * (`src/cli/aider.ts`) supplies the real spawn/capture/persist wiring.+ */++import { readFileSync } from "node:fs";+import { dirname, join, resolve } from "node:path";+import { fileURLToPath } from "node:url";++import type {+ PreCompactExtractInput,+ PreCompactExtractResult,+} from "../../brain/pre-compact-extract.ts";+import type { InstallEnv, McpPayload } from "../types.ts";++// ββ Shared snapshot logic (single source of truth for the sidecar) βββββββββββ++function repoRoot(): string {+ return resolve(dirname(fileURLToPath(import.meta.url)), "..", "..", "..", "..");+}++/** Path to the Aider context sidecar template. */+export function aiderTemplatePath(): string {+ return join(repoRoot(), "templates", "install", "aider-context.md.tmpl");+}++/** Resolve the agent identity stamped into the sidecar. */+export function resolveAiderAgentName(env: InstallEnv, payload: McpPayload): string {+ return payload.full.env?.["VAULT_AGENT_NAME"] ?? env.env["VAULT_AGENT_NAME"] ?? "agent";+}++/**+ * Render the Aider context sidecar from the shipped template. This is the ONE+ * place the sidecar bytes are produced β the static adapter and the live+ * wrapper both call it, so their output can never drift.+ */+export function renderAiderSidecar(env: InstallEnv, payload: McpPayload): string {+ const tpl = readFileSync(aiderTemplatePath(), "utf8");+ return tpl+ .replace(/\{\{VAULT\}\}/g, env.vault)+ .replace(/\{\{AGENT_NAME\}\}/g, resolveAiderAgentName(env, payload));+}++/** Default sidecar location inside the vault's machine-artifact dir. */+export function resolveAiderSidecarPath(+ env: InstallEnv,+ opts: { aiderContextPath?: string },+): string {+ if (opts.aiderContextPath) return opts.aiderContextPath;+ return join(env.vault, ".open-second-brain", "aider-context.md");+}++// ββ Session-bracketing lifecycle βββββββββββββββββββββββββββββββββββββββββββββ++export interface AiderSpawnResult {+ /** Process exit code, or null if the process was killed by a signal. */+ readonly code: number | null;+ /** Signal that killed the process, or null on a normal exit. */+ readonly signal: string | null;+}++/** Which lifecycle phases actually ran (for observability + tests). */+export type BracketPhase = "load" | "spawn" | "persist";++export interface BracketAiderDeps {+ /**+ * LOAD-HALF: regenerate + inject the live context sidecar and return its+ * path. Mirrors `prefetch()`. Runs before Aider is spawned.+ */+ readonly loadContext: () => string;+ /**+ * Spawn the Aider binary with the injected read-context and resolve on exit.+ * Mirrors execing the wrapped CLI.+ */+ readonly spawnAider: (sidecarPath: string) => Promise<AiderSpawnResult>;+ /**+ * WRITE-BACK: capture the session transcript produced during the run.+ * Mirrors the `sync_turn()` buffer drained at session end.+ */+ readonly captureTranscript: () => string;+ /**+ * Persist the captured transcript into the Brain. Mirrors+ * `on_session_end()` β `brain_pre_compact_extract`.+ */+ readonly persist: (input: PreCompactExtractInput) => PreCompactExtractResult;+}++export interface BracketAiderConfig {+ readonly sessionId: string;+ readonly turnStart?: string;+ readonly turnEnd?: string;+ readonly host?: string;+}++export interface BracketAiderResult {+ readonly sidecarPath: string;+ readonly exit: AiderSpawnResult;+ readonly interrupted: boolean;+ /** Null when the session produced no capturable transcript. */+ readonly persisted: PreCompactExtractResult | null;+ readonly phases: ReadonlyArray<BracketPhase>;+}++/**+ * An Aider exit is "interrupted" when it did not exit cleanly: killed by a+ * signal (SIGTERM/SIGINT/force-quit), or a non-zero exit code, or an unknown+ * exit (both null). Mirrors the honest `interrupted` flag the Hermes+ * `on_session_end` hook forwards on a non-clean close.+ */+export function isInterruptedExit(exit: AiderSpawnResult): boolean {+ if (exit.signal !== null) return true;+ return exit.code !== 0;+}++/**+ * Bracket one interactive Aider session: load context, run Aider, persist the+ * session back. The write-back fires even on an interrupted close so no session+ * is lost β the record is just honestly flagged `interrupted`.+ */+export async function bracketAiderSession(+ config: BracketAiderConfig,+ deps: BracketAiderDeps,+): Promise<BracketAiderResult> {+ const phases: BracketPhase[] = [];++ // 1. LOAD-HALF β regenerate + inject the live context before Aider starts.+ const sidecarPath = deps.loadContext();+ phases.push("load");++ // 2. Run Aider, bracketed between the load and the write-back.+ const exit = await deps.spawnAider(sidecarPath);+ phases.push("spawn");+ const interrupted = isInterruptedExit(exit);++ // 3. WRITE-BACK HALF β capture + persist the session. Skip only when the+ // session produced nothing capturable (mirrors the empty-buffer early return+ // in Hermes `_flush_buffer`), so an empty run writes no spurious records.+ const transcript = deps.captureTranscript();+ let persisted: PreCompactExtractResult | null = null;+ if (transcript.trim().length > 0) {+ const input: PreCompactExtractInput = {+ sessionId: config.sessionId,+ turnStart: config.turnStart ?? "0",+ turnEnd: config.turnEnd ?? "0",+ text: transcript,+ ...(config.host !== undefined ? { host: config.host } : {}),+ ...(interrupted ? { interrupted: true } : {}),+ };+ persisted = deps.persist(input);+ phases.push("persist");+ }++ return Object.freeze({+ sidecarPath,+ exit,+ interrupted,+ persisted,+ phases: Object.freeze(phases),+ });+}diff --git a/src/core/install/adapters/aider.ts b/src/core/install/adapters/aider.tsindex ac44f3af..55d9c666 100644--- a/src/core/install/adapters/aider.ts+++ b/src/core/install/adapters/aider.ts@@ -21,8 +21,7 @@*/import { existsSync, mkdirSync, readFileSync, rmSync, statSync } from "node:fs";-import { dirname, join, resolve } from "node:path";-import { fileURLToPath } from "node:url";+import { dirname, join } from "node:path";import { atomicWriteFileSync } from "../../fs-atomic.ts";import {@@ -46,18 +45,13 @@ import {type VerifyResult,} from "../types.ts";import { defaultRegistry } from "../registry.ts";+// The sidecar snapshot logic is shared with the live session-bracketing+// wrapper (E1) so the static and wrapped paths render byte-identical sidecars.+import { renderAiderSidecar, resolveAiderSidecarPath } from "./aider-wrapper.ts";const TARGET = "aider";const LABEL = "Aider";-function repoRoot(): string {- return resolve(dirname(fileURLToPath(import.meta.url)), "..", "..", "..", "..");-}--function templatePath(): string {- return join(repoRoot(), "templates", "install", "aider-context.md.tmpl");-}-function resolveConfPath(env: InstallEnv): string {if (env.env["AIDER_CONFIG"]) return env.env["AIDER_CONFIG"];const local = join(env.cwd, ".aider.conf.yml");@@ -66,8 +60,7 @@ function resolveConfPath(env: InstallEnv): string {}function resolveSidecarPath(env: InstallEnv, opts: { aiderContextPath?: string }): string {- if (opts.aiderContextPath) return opts.aiderContextPath;- return join(env.vault, ".open-second-brain", "aider-context.md");+ return resolveAiderSidecarPath(env, opts);}function readSafe(path: string): string {@@ -86,17 +79,6 @@ function fileMtimeMs(path: string): number | null {}}-function resolveAgentName(env: InstallEnv, payload: McpPayload): string {- return payload.full.env?.["VAULT_AGENT_NAME"] ?? env.env["VAULT_AGENT_NAME"] ?? "agent";-}--function renderTemplate(env: InstallEnv, payload: McpPayload): string {- const tpl = readFileSync(templatePath(), "utf8");- return tpl- .replace(/\{\{VAULT\}\}/g, env.vault)- .replace(/\{\{AGENT_NAME\}\}/g, resolveAgentName(env, payload));-}-function renderManagedBody(sidecarPath: string): string {return ["# Open Second Brain β agent context.",@@ -195,7 +177,7 @@ export const aiderAdapter: InstallAdapter = {}// Render sidecar- const sidecarContent = renderTemplate(env, payload);+ const sidecarContent = renderAiderSidecar(env, payload);if (!opts.dryRun) {ensureParent(sidecar);atomicWriteFileSync(sidecar, sidecarContent);diff --git a/src/mcp/brain/feedback-tools.ts b/src/mcp/brain/feedback-tools.tsindex 016d7226..40936071 100644--- a/src/mcp/brain/feedback-tools.ts+++ b/src/mcp/brain/feedback-tools.ts@@ -84,9 +84,24 @@ async function toolBrainFeedback(const agent =normalizeAgentArgument(validated.value.agent ?? null) ??resolveAgentName(ctx.configPath ?? undefined);+ // Per-row event-time (A2 / t_7526e8d3): an optional caller-supplied+ // `event_time` lets a backfilled / imported "remember" carry when it+ // actually happened instead of the wall-clock. When present and valid,+ // it drives the signal's `created_at` / filename day and the+ // bi-temporal `valid_from` / `recorded_at` slots. The audit log and any+ // force-confirmed preference stay stamped at the real write moment+ // (`now`) so the audit chronology is not rewritten. Absent β+ // byte-identical to the historical wall-clock path.+ const eventTime = coerceIsoDate(args, "event_time");+ // Optional client idempotency key (C1 / t_213f356b): forwarded to the+ // signal writer so a retried / double-delivered feedback call dedupes+ // instead of appending a second signal. Absent β historical behaviour.+ const idempotencyKey = coerceStr(args, "idempotency_key", false);const now = new Date();- const date = isoDate(now);const createdAt = isoSecond(now);+ const signalStamp = eventTime ?? now;+ const signalCreatedAt = isoSecond(signalStamp);+ const signalDate = isoDate(signalStamp);const slug = deriveSlug(topic);// Vault-configured fallback scope (`feedback.default_scope`). Applied@@ -106,14 +121,34 @@ async function toolBrainFeedback(signal: signalRaw as BrainSignalSign,agent,principle,- created_at: createdAt,- date,+ created_at: signalCreatedAt,+ date: signalDate,slug,...(scope ? { scope } : {}),...(source && source.length > 0 ? { source: [...source] } : {}),...(raw ? { raw } : {}),+ // Stamp the bi-temporal slots only when an explicit event_time was+ // supplied, so a live "remember" stays byte-identical.+ ...(eventTime ? { valid_from: signalCreatedAt, recorded_at: signalCreatedAt } : {}),+ ...(idempotencyKey ? { idempotency_key: idempotencyKey } : {}),};const sigResult = writeSignal(ctx.vault, signalInput, writeOpts);+ // A deduped signal means this whole feedback call is a retry of one+ // already recorded β skip the log event, the shared-namespace mirror,+ // and any force-confirmed pref so the retry stays a true no-op.+ if (sigResult.deduped) {+ return {+ kind: "signal",+ deduped: true,+ signal_path: vaultRelativeSafe(ctx.vault, sigResult.path),+ signal_absolute_path: resolve(sigResult.path),+ signal_id: sigResult.id,+ path: vaultRelativeSafe(ctx.vault, sigResult.path),+ absolute_path: resolve(sigResult.path),+ id: sigResult.id,+ agent,+ };+ }// t_936a1a61: fail-soft mirror into the shared namespace AFTER the// primary write; `mirror` is reported only when the key is configured.const sharedNamespace = resolveSharedNamespace(ctx.configPath);@@ -487,6 +522,16 @@ export const FEEDBACK_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([description:"When true, also creates an immediately-active confirmed `pref-*` alongside the inbox signal, skipping the dream-pass promotion step.",},+ event_time: {+ type: "string",+ description:+ "Optional ISO-8601 event-time for a backfilled signal (when it actually happened). Stamps `created_at`/`valid_from`/`recorded_at`; absent uses wall-clock.",+ },+ idempotency_key: {+ type: "string",+ description:+ "Optional client key that dedupes retried calls: same key + same payload is a no-op; same key + different payload is rejected.",+ },},required: ["topic", "signal", "principle"],additionalProperties: false,diff --git a/src/mcp/brain/ingest-tools.ts b/src/mcp/brain/ingest-tools.tsindex fdbf6c34..d28165d2 100644--- a/src/mcp/brain/ingest-tools.ts+++ b/src/mcp/brain/ingest-tools.ts@@ -9,15 +9,30 @@* Idempotent on the source path.*/+import { planBatches, type BatchPlan } from "../../core/brain/ingest/batch-plan.ts";import { ingestSource } from "../../core/brain/ingest/ingest.ts";import { IntakeValidationError } from "../../core/brain/intake/extract-intake.ts";+import {+ deleteBySource,+ searchBySourceFile,+ type SourceCleanupEntry,+ type SourceCleanupPlan,+} from "../../core/brain/source-cleanup.ts";import { resolveAgentName } from "../../core/config.ts";+import { coerceBoolOptional, coerceInt, coerceStr } from "../coerce.ts";+import { MCP_PREVIEW_BUDGET } from "../preview-budget.ts";import type { ServerContext, ToolDefinition } from "../tools.ts";-import { coerceStr } from "../coerce.ts";import { parseExtractionIntakeArgs } from "./intake-args.ts";import { wrapToolErrors } from "./shared.ts";const TOOL = "brain_ingest_source";+const SEARCH_TOOL = "brain_search_by_source";+const DELETE_TOOL = "brain_delete_by_source";+const BATCH_PLAN_TOOL = "brain_ingest_batch_plan";++/** Default batch caps when the caller omits them (1 MiB / 25 files per batch). */+const DEFAULT_MAX_BATCH_BYTES = 1024 * 1024;+const DEFAULT_MAX_BATCH_FILES = 25;async function toolBrainIngestSource(ctx: ServerContext,@@ -47,6 +62,102 @@ async function toolBrainIngestSource(});}+function serializeEntry(entry: SourceCleanupEntry): Record<string, unknown> {+ return {+ path: entry.path,+ id: entry.id,+ kind: entry.kind,+ match: entry.match,+ is_index_artifact: entry.isIndexArtifact,+ deletable: entry.deletable,+ };+}++function serializePlan(plan: SourceCleanupPlan): Record<string, unknown> {+ return {+ source: plan.source,+ confirmed: plan.confirmed,+ include_originals: plan.includeOriginals,+ blast_radius: plan.blastRadius,+ derived: plan.derived.map(serializeEntry),+ mentions: plan.mentions.map(serializeEntry),+ originals: [...plan.originals],+ manifest_entry: plan.manifestEntry,+ deleted: [...plan.deleted],+ manifest_entry_removed: plan.manifestEntryRemoved,+ audit_record_id: plan.auditRecordId,+ };+}++async function toolBrainSearchBySource(+ ctx: ServerContext,+ args: Record<string, unknown>,+): Promise<Record<string, unknown>> {+ const sourceFile = coerceStr(args, "source_file", true)!;+ const hits = searchBySourceFile(ctx.vault, sourceFile);+ return {+ source_file: sourceFile,+ total: hits.length,+ entries: hits.map(serializeEntry),+ };+}++async function toolBrainDeleteBySource(+ ctx: ServerContext,+ args: Record<string, unknown>,+): Promise<Record<string, unknown>> {+ const sourceFile = coerceStr(args, "source_file", true)!;+ const confirm = coerceBoolOptional(args, "confirm") ?? false;+ const includeOriginals = coerceBoolOptional(args, "include_originals") ?? false;+ const agent = coerceStr(args, "agent", false) ?? undefined;+ const plan = deleteBySource(ctx.vault, sourceFile, {+ confirm,+ includeOriginals,+ now: new Date(),+ ...(agent !== undefined ? { agent } : {}),+ });+ return serializePlan(plan);+}++function serializeBatchPlan(plan: BatchPlan): Record<string, unknown> {+ return {+ source_dir: plan.sourceDir,+ max_batch_bytes: plan.maxBatchBytes,+ max_batch_files: plan.maxBatchFiles,+ total_files: plan.totalFiles,+ total_bytes: plan.totalBytes,+ skipped: [...plan.skipped],+ batches: plan.batches.map((b) => ({+ index: b.index,+ total_bytes: b.totalBytes,+ files: b.files.map((f) => ({ path: f.path, bytes: f.bytes, status: f.status })),+ })),+ };+}++async function toolBrainIngestBatchPlan(+ ctx: ServerContext,+ args: Record<string, unknown>,+): Promise<Record<string, unknown>> {+ const sourceDir = coerceStr(args, "source_dir", true)!;+ const maxBatchBytes = coerceInt(+ args,+ "max_batch_bytes",+ DEFAULT_MAX_BATCH_BYTES,+ 1,+ Number.MAX_SAFE_INTEGER,+ );+ const maxBatchFiles = coerceInt(+ args,+ "max_batch_files",+ DEFAULT_MAX_BATCH_FILES,+ 1,+ Number.MAX_SAFE_INTEGER,+ );+ const plan = planBatches(ctx.vault, sourceDir, { maxBatchBytes, maxBatchFiles });+ return serializeBatchPlan(plan);+}+export const INGEST_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([{name: TOOL,@@ -104,4 +215,80 @@ export const INGEST_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([},handler: toolBrainIngestSource,},+ {+ name: SEARCH_TOOL,+ description:+ "Find every Brain page derived from one EXACT source file (`source_file`: a vault path or URL): the ingest summary page, session-derived signals, `[[source]]` provenance wikilinks, and preferences folded from those signals. Read-only; each entry flags deletable vs protected mention.",+ inputSchema: {+ type: "object",+ properties: {+ source_file: {+ type: "string",+ description: "Exact source identity to trace: a vault-relative path or a URL.",+ },+ },+ required: ["source_file"],+ additionalProperties: false,+ },+ handler: toolBrainSearchBySource,+ previewBudget: MCP_PREVIEW_BUDGET,+ },+ {+ name: DELETE_TOOL,+ description:+ "Remove everything derived from one EXACT source file. DRY-RUN BY DEFAULT: no `confirm` reports the blast radius, deletes nothing. `confirm` deletes derived entries + ingest index artifacts (shared/aggregate pages only reported); originals outside Brain/ only with `include_originals`. Auditable.",+ inputSchema: {+ type: "object",+ properties: {+ source_file: {+ type: "string",+ description: "Exact source identity to purge: a vault-relative path or a URL.",+ },+ confirm: {+ type: "boolean",+ description: "Required true to actually delete; absent/false is a dry run.",+ },+ include_originals: {+ type: "boolean",+ description: "Also remove the original source file(s) outside Brain/. Default false.",+ },+ agent: {+ type: "string",+ description: "Optional agent identity recorded in the audit reason.",+ },+ },+ required: ["source_file"],+ additionalProperties: false,+ },+ handler: toolBrainDeleteBySource,+ previewBudget: MCP_PREVIEW_BUDGET,+ },+ {+ name: BATCH_PLAN_TOOL,+ description:+ "Plan a folder ingest into bounded parallel batches. Walks `source_dir` for ingestible files, skips unchanged ones via the content-hash manifest, and packs the new/modified remainder into batches bounded by `max_batch_bytes`/`max_batch_files`. Read-only; the caller dispatches each batch.",+ inputSchema: {+ type: "object",+ properties: {+ source_dir: {+ type: "string",+ description: "Vault-relative directory to plan ingest over.",+ },+ max_batch_bytes: {+ type: "integer",+ minimum: 1,+ description: `Byte cap per batch. Default ${DEFAULT_MAX_BATCH_BYTES}.`,+ },+ max_batch_files: {+ type: "integer",+ minimum: 1,+ description: `File-count cap per batch. Default ${DEFAULT_MAX_BATCH_FILES}.`,+ },+ },+ required: ["source_dir"],+ additionalProperties: false,+ },+ handler: toolBrainIngestBatchPlan,+ previewBudget: MCP_PREVIEW_BUDGET,+ },]);diff --git a/src/mcp/brain/pack-tools.ts b/src/mcp/brain/pack-tools.tsindex 3b73e64c..83e0cfb4 100644--- a/src/mcp/brain/pack-tools.ts+++ b/src/mcp/brain/pack-tools.ts@@ -457,8 +457,9 @@ async function toolBrainPreCompactExtract(: {}),...(maxChars !== undefined ? { maxChars } : {}),...(coerceBool(args, "interrupted") === true ? { interrupted: true } : {}),+ ...(coerceBool(args, "dry_run") === true ? { dryRun: true } : {}),});- return { count: result.records.length, ...result };+ return { count: result.records.length, dry_run: coerceBool(args, "dry_run") === true, ...result };}// ----- session recall DAG --------------------------------------------------@@ -825,6 +826,11 @@ export const PACK_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([description:"When true, mark the extracted records as flushed by an interrupted close (SIGHUP/SIGTERM/force-quit/restart-drain). Absent by default.",},+ dry_run: {+ type: "boolean",+ description:+ "Preview the candidate records extraction would append WITHOUT writing to the vault (no continuity record, no dream/retire trigger). Absent by default.",+ },},required: ["session_id", "turn_start", "turn_end", "text"],additionalProperties: false,diff --git a/src/mcp/brain/query-tools.ts b/src/mcp/brain/query-tools.tsindex 2ca7e701..cfe0c4a7 100644--- a/src/mcp/brain/query-tools.ts+++ b/src/mcp/brain/query-tools.ts@@ -135,7 +135,10 @@ async function toolBrainQuery(}if (topic !== null) {- const res = queryByTopic(ctx.vault, topic);+ // Expiration opt-in (C5): default drops memories past their+ // expiration_date; show_expired re-includes them for audit.+ const showExpired = coerceBoolOptional(args, "show_expired") ?? false;+ const res = queryByTopic(ctx.vault, topic, { showExpired });const resultCount = res.signals.length + res.all_log_events.length;emitQueryTelemetry(resultCount > 0 ? "ok" : "empty", resultCount);const topicPrefOwner =@@ -380,6 +383,7 @@ function serializeSignal(s: BrainSignal): Record<string, unknown> {...(s.scope !== undefined ? { scope: s.scope } : {}),...(s.source !== undefined ? { source: [...s.source] } : {}),...(s.raw !== undefined ? { raw: s.raw } : {}),+ ...(s.expiration_date !== undefined ? { expiration_date: s.expiration_date } : {}),};}@@ -428,6 +432,7 @@ function serializePreference(p: BrainPreference | BrainRetired): Record<string,tags: [...p.tags],...(p.supersedes !== undefined ? { supersedes: p.supersedes } : {}),...(p.aliases !== undefined ? { aliases: [...p.aliases] } : {}),+ ...(p.expiration_date !== undefined ? { expiration_date: p.expiration_date } : {}),};}@@ -479,6 +484,11 @@ export const QUERY_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([type: "string",description: "Topic slug to aggregate signals + active/retired preference + log events.",},+ show_expired: {+ type: "boolean",+ description:+ "Topic mode only: include memories past their `expiration_date`. Default false (expired memories are silently dropped from the result).",+ },since: {type: "string",description: "ISO-8601 timestamp; returns every Brain log event with timestamp >= since.",diff --git a/src/mcp/brain/synthesis-tools.ts b/src/mcp/brain/synthesis-tools.tsindex 5372461b..f1410dbb 100644--- a/src/mcp/brain/synthesis-tools.ts+++ b/src/mcp/brain/synthesis-tools.ts@@ -13,6 +13,16 @@ import {SessionSummaryError,type SessionSummaryDigest,} from "../../core/brain/session-summary.ts";+import {+ saveSessionCheckpoint,+ SessionCheckpointError,+ type CheckpointSignalInput,+ type SessionCheckpointResult,+} from "../../core/brain/session-checkpoint.ts";+import { IdempotencyPayloadMismatchError } from "../../core/brain/idempotency-ledger.ts";+import { resolveAgentName } from "../../core/config.ts";+import { normalizeAgentArgument } from "../../core/agent-identity.ts";+import type { BrainSignalSign } from "../../core/brain/types.ts";import {traceIdeaLineage,IdeaLineageError,@@ -110,6 +120,115 @@ async function toolBrainSessionSummary(return { count: digests.length, digests: digests.map(serializeDigest) };}+const CHECKPOINT_TOOL = "brain_session_checkpoint";++/** Parse the `signals` array into checkpoint signal inputs. Only STRUCTURAL+ * problems (not-an-array, an item that is not an object, a mistyped+ * scalar) are protocol errors here. SEMANTIC validity β a bad sign, a+ * missing topic / principle β is deliberately passed through so the kernel+ * attempts the write and collects the failure as a `partial` item (status+ * "mixed"), rather than aborting the whole batch. */+function parseCheckpointSignals(+ args: Record<string, unknown>,+): ReadonlyArray<CheckpointSignalInput> {+ const value = args["signals"];+ if (value === undefined || value === null) return [];+ if (!Array.isArray(value)) {+ throw new MCPError(INVALID_PARAMS, `${CHECKPOINT_TOOL}: signals must be an array`);+ }+ return value.map((raw, index) => {+ if (typeof raw !== "object" || raw === null) {+ throw new MCPError(INVALID_PARAMS, `${CHECKPOINT_TOOL}: signals[${index}] must be an object`);+ }+ const item = raw as Record<string, unknown>;+ const topic = typeof item["topic"] === "string" ? item["topic"] : "";+ const principle = typeof item["principle"] === "string" ? item["principle"] : "";+ // Sign passed through verbatim; the writer rejects an out-of-enum value+ // and the kernel surfaces it as a `partial` item.+ const sign = item["signal"] as BrainSignalSign;+ const scope = typeof item["scope"] === "string" ? item["scope"] : undefined;+ const rawText = typeof item["raw"] === "string" ? item["raw"] : undefined;+ const source = stringArrayField(item["source"], `signals[${index}].source`);+ return {+ topic,+ signal: sign,+ principle,+ ...(scope !== undefined ? { scope } : {}),+ ...(rawText !== undefined ? { raw: rawText } : {}),+ ...(source !== undefined ? { source } : {}),+ } satisfies CheckpointSignalInput;+ });+}++function stringArrayField(value: unknown, name: string): ReadonlyArray<string> | undefined {+ if (value === undefined || value === null) return undefined;+ if (!Array.isArray(value) || value.some((v) => typeof v !== "string")) {+ throw new MCPError(INVALID_PARAMS, `${CHECKPOINT_TOOL}: ${name} must be an array of strings`);+ }+ return value as ReadonlyArray<string>;+}++function serializeCheckpoint(result: SessionCheckpointResult): Record<string, unknown> {+ return {+ status: result.status,+ session_id: result.sessionId,+ deduped: result.deduped,+ signals: result.signals.map((s) => ({ id: s.id, path: s.path, deduped: s.deduped })),+ summary: result.summary === null ? null : { id: result.summary.id },+ diary_written: result.diaryWritten,+ partial: result.partial.map((p) => ({+ kind: p.kind,+ ...(p.index !== undefined ? { index: p.index } : {}),+ ...(p.topic !== undefined ? { topic: p.topic } : {}),+ reason: p.reason,+ })),+ };+}++async function toolBrainSessionCheckpoint(+ ctx: ServerContext,+ args: Record<string, unknown>,+): Promise<Record<string, unknown>> {+ const sessionIdRaw = args["session_id"];+ if (typeof sessionIdRaw !== "string" || sessionIdRaw.trim().length === 0) {+ throw new MCPError(INVALID_PARAMS, `${CHECKPOINT_TOOL}: session_id is required`);+ }+ const agentArg = typeof args["agent"] === "string" ? args["agent"] : null;+ const agent = normalizeAgentArgument(agentArg) ?? resolveAgentName(ctx.configPath ?? undefined);+ const signals = parseCheckpointSignals(args);+ const request = typeof args["request"] === "string" ? args["request"] : undefined;+ const decisions = stringArrayField(args["decisions"], "decisions");+ const learnings = stringArrayField(args["learnings"], "learnings");+ const nextSteps = stringArrayField(args["next_steps"], "next_steps");+ const sourceTurnIds = stringArrayField(args["source_turn_ids"], "source_turn_ids");+ const host = typeof args["host"] === "string" ? args["host"] : undefined;+ const diary = typeof args["diary"] === "string" ? args["diary"] : undefined;++ try {+ const result = saveSessionCheckpoint(ctx.vault, {+ sessionId: sessionIdRaw.trim(),+ agent,+ signals,+ ...(request !== undefined ? { request } : {}),+ ...(decisions !== undefined ? { decisions } : {}),+ ...(learnings !== undefined ? { learnings } : {}),+ ...(nextSteps !== undefined ? { nextSteps } : {}),+ ...(sourceTurnIds !== undefined ? { sourceTurnIds } : {}),+ ...(host !== undefined ? { host } : {}),+ ...(diary !== undefined ? { diary } : {}),+ });+ return serializeCheckpoint(result);+ } catch (error) {+ if (+ error instanceof SessionCheckpointError ||+ error instanceof IdempotencyPayloadMismatchError+ ) {+ throw new MCPError(INVALID_PARAMS, error.message);+ }+ throw error;+ }+}+const LINEAGE_TOOL = "brain_idea_lineage";function serializeLineage(result: IdeaLineageResult): Record<string, unknown> {@@ -232,6 +351,58 @@ export const SYNTHESIS_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([handler: toolBrainSessionSummary,previewBudget: MCP_PREVIEW_BUDGET,},+ {+ name: CHECKPOINT_TOOL,+ description:+ "Batch-save a whole session's signals + a decisions/learnings/next_steps summary + optional diary in one idempotent round-trip, keyed by session_id. A same-content retry dedupes; a different-content retry is payload_mismatch. Items needing review return in `partial` (status 'mixed'), never dropped.",+ inputSchema: {+ type: "object",+ properties: {+ session_id: { type: "string", description: "Session id; the idempotency key." },+ agent: { type: "string", description: "Identity stamped on written signals (optional)." },+ signals: {+ type: "array",+ description: "Extracted memories to persist as signals.",+ items: {+ type: "object",+ properties: {+ topic: { type: "string", description: "Stable kebab-slug topic." },+ signal: {+ type: "string",+ enum: ["positive", "negative"],+ description: "positive = rule to follow; negative = to avoid.",+ },+ principle: { type: "string", description: "One-line rule formulation." },+ scope: { type: "string", description: "Soft category (optional)." },+ raw: { type: "string", description: "Verbatim source quote (optional)." },+ source: {+ type: "array",+ items: { type: "string" },+ description: "Wikilinks to triggering artifacts (optional).",+ },+ },+ required: ["topic", "signal", "principle"],+ additionalProperties: false,+ },+ },+ request: { type: "string", description: "One-line session goal." },+ decisions: { type: "array", items: { type: "string" }, description: "Decisions made." },+ learnings: { type: "array", items: { type: "string" }, description: "Things learned." },+ next_steps: { type: "array", items: { type: "string" }, description: "Follow-up actions." },+ source_turn_ids: {+ type: "array",+ items: { type: "string" },+ description: "Turn ids the summary was distilled from.",+ },+ host: { type: "string", description: "Originating runtime (claude, codex, ...)." },+ diary: { type: "string", description: "Optional narrative diary line for the Brain log." },+ },+ required: ["session_id"],+ additionalProperties: false,+ },+ handler: toolBrainSessionCheckpoint,+ previewBudget: MCP_PREVIEW_BUDGET,+ },{name: LINEAGE_TOOL,description:diff --git a/tests/cli/brain-pre-compact-extract.test.ts b/tests/cli/brain-pre-compact-extract.test.tsindex 2f994336..df550c89 100644--- a/tests/cli/brain-pre-compact-extract.test.ts+++ b/tests/cli/brain-pre-compact-extract.test.ts@@ -43,3 +43,34 @@ test("brain pre-compact-extract emits typed records without writing raw media",).toEqual(["decision", "open_question"]);expect(result.stdout).not.toContain("QUJDREVGR0hJSktMTU5PUFFSU1RVVldYWVo");});++test("brain pre-compact-extract --dry-run previews without writing to the vault", async () => {+ const argv = [+ "brain",+ "pre-compact-extract",+ "--vault",+ vault,+ "--session-id",+ "session-cli-dry",+ "--turn-start",+ "turn-1",+ "--turn-end",+ "turn-1",+ "--text",+ "Decision: Preview only, do not persist.",+ "--json",+ ];+ const preview = await runCli([...argv, "--dry-run"]);+ expect(preview.returncode).toBe(0);+ const previewBody = JSON.parse(preview.stdout);+ expect(previewBody).toMatchObject({ count: 1, dry_run: true, errors: [] });++ // The preview's timestamp-independent dedupe_key matches the record the+ // real run appends β the preview predicts real extraction. (record `id`+ // folds in the ms-precise createdAt, which differs across two un-pinned+ // calls, so compare dedupe_key.)+ const real = await runCli(argv);+ const realBody = JSON.parse(real.stdout);+ expect(realBody).toMatchObject({ count: 1, dry_run: false });+ expect(previewBody.records[0].payload.dedupe_key).toBe(realBody.records[0].payload.dedupe_key);+});diff --git a/tests/core/brain/expiration.test.ts b/tests/core/brain/expiration.test.tsnew file mode 100644index 00000000..ff61b6fd--- /dev/null+++ b/tests/core/brain/expiration.test.ts@@ -0,0 +1,93 @@+/**+ * C5 (t_a82b674e): caller-settable per-memory expiration date.+ *+ * The pure, LLM-free expiration model: validate a caller-supplied date on+ * write, decide whether a memory is past its expiration on read, and drop+ * expired memories from a list unless an opt-in `showExpired` flag is set.+ *+ * Date-granular semantics: a date-only `YYYY-MM-DD` value keeps the memory+ * live through the END of that UTC day (so "until 2026-07-15" stays live+ * all of the 15th and lapses on the 16th); a full ISO timestamp expires at+ * that exact instant.+ */++import { describe, expect, test } from "bun:test";++import {+ filterExpired,+ isExpired,+ normalizeExpirationDate,+} from "../../../src/core/brain/expiration.ts";++describe("normalizeExpirationDate", () => {+ test("accepts a date-only value and returns it trimmed", () => {+ expect(normalizeExpirationDate(" 2026-07-15 ")).toBe("2026-07-15");+ });++ test("accepts a full ISO-8601 timestamp", () => {+ expect(normalizeExpirationDate("2026-07-15T09:30:00Z")).toBe("2026-07-15T09:30:00Z");+ });++ test("throws on an unparseable value", () => {+ expect(() => normalizeExpirationDate("not-a-date")).toThrow();+ });++ test("throws on an impossible calendar date", () => {+ expect(() => normalizeExpirationDate("2026-13-40")).toThrow();+ });+});++describe("isExpired β date-only granularity", () => {+ test("still live during the expiration day (UTC)", () => {+ expect(isExpired("2026-07-15", new Date("2026-07-15T23:00:00Z"))).toBe(false);+ });++ test("expired the following day", () => {+ expect(isExpired("2026-07-15", new Date("2026-07-16T00:00:01Z"))).toBe(true);+ });++ test("not expired well before the date", () => {+ expect(isExpired("2026-07-15", new Date("2026-01-01T00:00:00Z"))).toBe(false);+ });+});++describe("isExpired β full timestamp granularity", () => {+ test("live just before the instant", () => {+ expect(isExpired("2026-07-15T09:30:00Z", new Date("2026-07-15T09:29:59Z"))).toBe(false);+ });++ test("expired just after the instant", () => {+ expect(isExpired("2026-07-15T09:30:00Z", new Date("2026-07-15T09:30:01Z"))).toBe(true);+ });+});++describe("isExpired β malformed value fails open", () => {+ test("an unparseable stored date is treated as not-expired (never silently hides a memory)", () => {+ expect(isExpired("garbage", new Date("2999-01-01T00:00:00Z"))).toBe(false);+ });+});++describe("filterExpired", () => {+ const now = new Date("2026-08-01T00:00:00Z");+ const items = [+ { id: "a" }, // no expiration β always kept+ { id: "b", expiration_date: "2026-07-15" }, // expired before `now`+ { id: "c", expiration_date: "2026-12-31" }, // still live+ ];++ test("drops expired items by default", () => {+ const kept = filterExpired(items, { now }).map((i) => i.id);+ expect(kept).toEqual(["a", "c"]);+ });++ test("keeps expired items when showExpired is set", () => {+ const kept = filterExpired(items, { now, showExpired: true }).map((i) => i.id);+ expect(kept).toEqual(["a", "b", "c"]);+ });++ test("items without an expiration_date are always kept", () => {+ const noExpiry: { id: string; expiration_date?: string }[] = [{ id: "a" }];+ const kept = filterExpired(noExpiry, { now }).map((i) => i.id);+ expect(kept).toEqual(["a"]);+ });+});diff --git a/tests/core/brain/health/contradiction-notes.test.ts b/tests/core/brain/health/contradiction-notes.test.tsnew file mode 100644index 00000000..7500c000--- /dev/null+++ b/tests/core/brain/health/contradiction-notes.test.ts@@ -0,0 +1,191 @@+/**+ * Note-position contradiction detector (D2).+ *+ * Unlike the confirmed-preference detector, whose polarity comes from+ * each preference's `evidenced_by` signals, this one derives each note's+ * stance sign structurally from its own PROSE: a note carrying a+ * negation marker among its tokens asserts the negative side of an+ * otherwise-shared subject. Two same-subject notes (high token overlap)+ * with opposite stances are surfaced as an `ask_user` clarification -+ * quoting the relevant span from each - and never auto-resolved.+ */++import { describe, expect, test } from "bun:test";++import { BRAIN_SIGNAL_SIGN } from "../../../../src/core/brain/types.ts";+import {+ detectNoteContradictions,+ type NoteForContradiction,+} from "../../../../src/core/brain/health/contradiction.ts";++function note(+ over: Partial<NoteForContradiction> & Pick<NoteForContradiction, "id" | "text">,+): NoteForContradiction {+ return { ...over };+}++describe("detectNoteContradictions", () => {+ test("pairs same-subject notes with opposite stances and quotes each span", () => {+ const findings = detectNoteContradictions(+ [+ note({+ id: "notes/remote-work-good.md",+ text: "Remote work boosts long-term team productivity.",+ }),+ note({+ id: "notes/remote-work-bad.md",+ text: "Remote work does not boost long-term team productivity.",+ }),+ ],+ { jaccard: 0.5 },+ );+ expect(findings.length).toBe(1);+ const f = findings[0]!;+ expect(f.aId).toBe("notes/remote-work-bad.md");+ expect(f.bId).toBe("notes/remote-work-good.md");+ // aId sorts first; its prose carries the negation marker -> negative.+ expect(f.aSign).toBe(BRAIN_SIGNAL_SIGN.negative);+ expect(f.bSign).toBe(BRAIN_SIGNAL_SIGN.positive);+ expect(f.jaccard).toBeGreaterThanOrEqual(0.5);+ // Each finding quotes the relevant span from each note verbatim.+ expect(f.aQuote).toBe("Remote work does not boost long-term team productivity.");+ expect(f.bQuote).toBe("Remote work boosts long-term team productivity.");+ // Never auto-resolves: it is a clarification prompt.+ expect(f.action).toBe("ask_user");+ });++ test("notes that merely add complexity (same stance) are not flagged", () => {+ const findings = detectNoteContradictions(+ [+ note({+ id: "notes/a.md",+ text: "Remote work boosts long-term team productivity.",+ }),+ note({+ id: "notes/b.md",+ text: "Remote work boosts long-term team productivity, though it demands discipline.",+ }),+ ],+ { jaccard: 0.5 },+ );+ expect(findings).toEqual([]);+ });++ test("unrelated-subject notes are not paired (token-overlap threshold)", () => {+ const findings = detectNoteContradictions(+ [+ note({+ id: "notes/a.md",+ text: "Deploy releases on friday afternoon.",+ }),+ note({+ id: "notes/b.md",+ text: "We should not write unit tests before merging code.",+ }),+ ],+ { jaccard: 0.5 },+ );+ expect(findings).toEqual([]);+ });++ test("stance is derived structurally and is language-agnostic", () => {+ // German prose with a caller-supplied negation lexicon: no English+ // wordlist is consulted, proving the derivation is not hardcoded.+ const findings = detectNoteContradictions(+ [+ note({+ id: "notes/de-pro.md",+ text: "Fernarbeit steigert die langfristige Produktivitaet des Teams.",+ }),+ note({+ id: "notes/de-contra.md",+ text: "Fernarbeit steigert nicht die langfristige Produktivitaet des Teams.",+ }),+ ],+ { jaccard: 0.5, negationMarkers: new Set(["nicht"]) },+ );+ expect(findings.length).toBe(1);+ expect(findings[0]!.aId).toBe("notes/de-contra.md");+ expect(findings[0]!.aSign).toBe(BRAIN_SIGNAL_SIGN.negative);+ expect(findings[0]!.bSign).toBe(BRAIN_SIGNAL_SIGN.positive);+ });++ test("an English-only stance is NOT assumed: a non-marker language yields no false pair", () => {+ // Default lexicon does not know this token; both notes read as the+ // same (positive) stance, so no spurious contradiction is emitted.+ const findings = detectNoteContradictions(+ [+ note({ id: "notes/x.md", text: "Fernarbeit steigert die Produktivitaet des Teams." }),+ note({+ id: "notes/y.md",+ text: "Fernarbeit steigert nichtxx die Produktivitaet des Teams.",+ }),+ ],+ { jaccard: 0.5 },+ );+ expect(findings).toEqual([]);+ });++ test("subject buckets are compared apart when provided", () => {+ const findings = detectNoteContradictions(+ [+ note({+ id: "notes/a.md",+ subject: "remote-work",+ text: "Remote work boosts long-term team productivity.",+ }),+ note({+ id: "notes/b.md",+ subject: "office-work",+ text: "Remote work does not boost long-term team productivity.",+ }),+ ],+ { jaccard: 0.5 },+ );+ expect(findings).toEqual([]);+ });++ test("output is deterministically ordered with aId < bId", () => {+ const findings = detectNoteContradictions(+ [+ note({+ id: "notes/zebra.md",+ text: "Remote work does not boost long-term team productivity.",+ }),+ note({+ id: "notes/alpha.md",+ text: "Remote work boosts long-term team productivity.",+ }),+ ],+ { jaccard: 0.5 },+ );+ expect(findings.length).toBe(1);+ expect(findings[0]!.aId).toBe("notes/alpha.md");+ expect(findings[0]!.bId).toBe("notes/zebra.md");+ // alpha carries no negation marker (positive); zebra does (negative).+ expect(findings[0]!.aSign).toBe(BRAIN_SIGNAL_SIGN.positive);+ expect(findings[0]!.bSign).toBe(BRAIN_SIGNAL_SIGN.negative);+ });++ test("relevant span is the subject-bearing sentence in a multi-sentence note", () => {+ const findings = detectNoteContradictions(+ [+ note({+ id: "notes/a.md",+ text: "I keep changing my mind. Remote work boosts long-term team productivity. That is my current view.",+ }),+ note({+ id: "notes/b.md",+ text: "Some background first. Remote work does not boost long-term team productivity.",+ }),+ ],+ // Framing sentences dilute whole-note overlap; the span extractor+ // still isolates the subject-bearing sentence for the quote.+ { jaccard: 0.2 },+ );+ expect(findings.length).toBe(1);+ // aId (notes/a.md) sorts first and holds the positive "boosts" sentence.+ expect(findings[0]!.aQuote).toBe("Remote work boosts long-term team productivity.");+ expect(findings[0]!.bQuote).toBe("Remote work does not boost long-term team productivity.");+ });+});diff --git a/tests/core/brain/health/thesis.test.ts b/tests/core/brain/health/thesis.test.tsnew file mode 100644index 00000000..4aace614--- /dev/null+++ b/tests/core/brain/health/thesis.test.ts@@ -0,0 +1,348 @@+/**+ * Declared-thesis register + new-note monitor (D3).+ *+ * The operator records standing positions (statement, supporting /+ * counter-evidence, a falsification "what would make me wrong", a review+ * cadence). The monitor evaluates each newly-ingested note against the+ * ACTIVE theses and flags support / contradiction / falsification-match -+ * distinct from note-vs-note contradiction (D2): this is+ * incoming-note-vs-declared-position. It builds on D2's structural,+ * language-agnostic stance derivation and never auto-resolves. Two+ * lifecycle checks reuse the cadence machinery: a staleness flag for+ * theses not updated within their cadence, and a graveyard flag for+ * active theses with no supporting evidence in N days.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, readFileSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { BRAIN_SIGNAL_SIGN } from "../../../../src/core/brain/types.ts";+import {+ closeThesis,+ detectStaleTheses,+ detectThesisGraveyard,+ listTheses,+ monitorNotesAgainstTheses,+ recordThesis,+ recordThesisSupport,+ showThesis,+ ThesisError,+ updateThesis,+ type ThesisForMonitor,+} from "../../../../src/core/brain/health/thesis.ts";++const NOW = new Date("2026-07-08T12:00:00Z");++let vault: string;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "osb-thesis-"));+ mkdirSync(join(vault, "Brain"), { recursive: true });+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+});++const REMOTE_THESIS: ThesisForMonitor = {+ slug: "remote-work-boosts-productivity",+ status: "active",+ statement: "Remote work boosts long-term team productivity.",+ falsification: "Productivity gains disappear once headcount passes fifty people.",+};++describe("thesis register (persistence)", () => {+ test("recordThesis writes a page and round-trips through showThesis", () => {+ const page = recordThesis(vault, {+ statement: "Remote work boosts long-term team productivity.",+ supportingEvidence: "Two internal cohorts shipped faster after going remote.",+ counterEvidence: "Onboarding new hires slowed in the first quarter.",+ falsification: "Productivity gains disappear once headcount passes fifty people.",+ cadence: "monthly",+ agent: "test-agent",+ now: NOW,+ });+ expect(page.slug).toBe("remote-work-boosts-long-term-team-productivity");+ expect(page.status).toBe("active");+ expect(page.cadence).toBe("monthly");+ expect(page.lastUpdated).toBe("2026-07-08");+ expect(page.lastSupportAt).toBeNull();++ const read = showThesis(vault, page.slug);+ expect(read).not.toBeNull();+ expect(read!.statement).toBe("Remote work boosts long-term team productivity.");+ expect(read!.supportingEvidence).toBe(+ "Two internal cohorts shipped faster after going remote.",+ );+ expect(read!.counterEvidence).toBe("Onboarding new hires slowed in the first quarter.");+ expect(read!.falsification).toBe(+ "Productivity gains disappear once headcount passes fifty people.",+ );+ });++ test("recordThesis rejects an empty statement and refuses to clobber", () => {+ expect(() => recordThesis(vault, { statement: " ", agent: "a", now: NOW })).toThrow(+ ThesisError,+ );+ recordThesis(vault, { statement: "A clear position.", agent: "a", now: NOW });+ expect(() =>+ recordThesis(vault, { statement: "A clear position.", agent: "a", now: NOW }),+ ).toThrow(ThesisError);+ });++ test("updateThesis bumps last_updated; recordThesisSupport bumps last_support_at", () => {+ const created = recordThesis(vault, {+ statement: "Remote work boosts long-term team productivity.",+ agent: "a",+ now: new Date("2026-01-01T00:00:00Z"),+ });+ expect(created.lastUpdated).toBe("2026-01-01");++ const touched = updateThesis(vault, {+ slug: created.slug,+ counterEvidence: "A dissenting cohort report landed.",+ now: NOW,+ });+ expect(touched.lastUpdated).toBe("2026-07-08");+ expect(touched.counterEvidence).toBe("A dissenting cohort report landed.");++ const supported = recordThesisSupport(vault, created.slug, { date: "2026-07-08", now: NOW });+ expect(supported.lastSupportAt).toBe("2026-07-08");+ expect(supported.lastUpdated).toBe("2026-07-08");+ });++ test("closeThesis formally closes; listTheses can filter to active only", () => {+ recordThesis(vault, { statement: "First position.", agent: "a", now: NOW });+ const second = recordThesis(vault, { statement: "Second position.", agent: "a", now: NOW });+ closeThesis(vault, second.slug);+ expect(showThesis(vault, second.slug)!.status).toBe("closed");++ const active = listTheses(vault, { activeOnly: true });+ expect(active.map((t) => t.slug)).toEqual(["first-position"]);+ expect(listTheses(vault).length).toBe(2);+ });+});++describe("monitorNotesAgainstTheses", () => {+ test("a contradicting note raises a conflict flag quoting the thesis", () => {+ const findings = monitorNotesAgainstTheses(+ [REMOTE_THESIS],+ [+ {+ id: "notes/incoming.md",+ text: "Remote work does not boost long-term team productivity.",+ },+ ],+ {},+ );+ expect(findings.length).toBe(1);+ const f = findings[0]!;+ expect(f.kind).toBe("contradict");+ expect(f.thesisSlug).toBe(REMOTE_THESIS.slug);+ expect(f.noteId).toBe("notes/incoming.md");+ expect(f.thesisStance).toBe(BRAIN_SIGNAL_SIGN.positive);+ expect(f.noteStance).toBe(BRAIN_SIGNAL_SIGN.negative);+ // Quotes the declared position verbatim (article: quote the thesis note).+ expect(f.thesisQuote).toBe("Remote work boosts long-term team productivity.");+ expect(f.noteQuote).toBe("Remote work does not boost long-term team productivity.");+ expect(f.action).toBe("ask_user");+ });++ test("a closely-matching same-stance note raises a support flag", () => {+ const findings = monitorNotesAgainstTheses(+ [REMOTE_THESIS],+ [+ {+ id: "notes/support.md",+ text: "Remote work boosts long-term team productivity.",+ },+ ],+ {},+ );+ expect(findings.length).toBe(1);+ expect(findings[0]!.kind).toBe("support");+ expect(findings[0]!.thesisStance).toBe(BRAIN_SIGNAL_SIGN.positive);+ expect(findings[0]!.noteStance).toBe(BRAIN_SIGNAL_SIGN.positive);+ });++ test("a note that merely adds complexity is NOT flagged", () => {+ const findings = monitorNotesAgainstTheses(+ [REMOTE_THESIS],+ [+ {+ id: "notes/nuance.md",+ text: "Remote work boosts long-term team productivity, though it demands strong written communication, deliberate onboarding rituals, and careful timezone overlap planning.",+ },+ ],+ {},+ );+ expect(findings).toEqual([]);+ });++ test("an unrelated note is not paired with the thesis", () => {+ const findings = monitorNotesAgainstTheses(+ [REMOTE_THESIS],+ [{ id: "notes/x.md", text: "Deploy releases on friday afternoon." }],+ {},+ );+ expect(findings).toEqual([]);+ });++ test("incoming evidence matching the falsification scenario raises an alert", () => {+ const findings = monitorNotesAgainstTheses(+ [REMOTE_THESIS],+ [+ {+ id: "notes/failure.md",+ text: "Our metrics show the productivity gains disappear once headcount passes fifty people.",+ },+ ],+ {},+ );+ expect(findings.length).toBe(1);+ const f = findings[0]!;+ expect(f.kind).toBe("falsification");+ expect(f.thesisQuote).toBe("Productivity gains disappear once headcount passes fifty people.");+ expect(f.noteQuote).toBe(+ "Our metrics show the productivity gains disappear once headcount passes fifty people.",+ );+ expect(f.action).toBe("ask_user");+ });++ test("closed theses are not monitored", () => {+ const findings = monitorNotesAgainstTheses(+ [{ ...REMOTE_THESIS, status: "closed" }],+ [+ {+ id: "notes/incoming.md",+ text: "Remote work does not boost long-term team productivity.",+ },+ ],+ {},+ );+ expect(findings).toEqual([]);+ });++ test("stance derivation is language-agnostic with a caller-supplied lexicon", () => {+ const findings = monitorNotesAgainstTheses(+ [+ {+ slug: "fernarbeit",+ status: "active",+ statement: "Fernarbeit steigert die langfristige Produktivitaet des Teams.",+ },+ ],+ [+ {+ id: "notes/de.md",+ text: "Fernarbeit steigert nicht die langfristige Produktivitaet des Teams.",+ },+ ],+ { negationMarkers: new Set(["nicht"]) },+ );+ expect(findings.length).toBe(1);+ expect(findings[0]!.kind).toBe("contradict");+ expect(findings[0]!.noteStance).toBe(BRAIN_SIGNAL_SIGN.negative);+ });++ test("findings are deterministically ordered by thesis then note then kind", () => {+ const theses: ThesisForMonitor[] = [+ {+ slug: "zebra",+ status: "active",+ statement: "Remote work boosts long-term team productivity.",+ },+ {+ slug: "alpha",+ status: "active",+ statement: "Remote work boosts long-term team productivity.",+ },+ ];+ const notes = [+ { id: "notes/b.md", text: "Remote work boosts long-term team productivity." },+ { id: "notes/a.md", text: "Remote work does not boost long-term team productivity." },+ ];+ const findings = monitorNotesAgainstTheses(theses, notes, {});+ const keys = findings.map((f) => `${f.thesisSlug}/${f.noteId}/${f.kind}`);+ const sorted = keys.toSorted();+ expect(keys).toEqual(sorted);+ });+});++describe("thesis lifecycle checks", () => {+ test("detectStaleTheses flags a thesis past its cadence via nextDueDate", () => {+ // Updated 2026-01-01 on a monthly cadence; by NOW (2026-07-08) it is+ // long past next-due. A fresh thesis updated today is not stale.+ const stale = detectStaleTheses(+ [+ {+ slug: "old",+ status: "active",+ cadence: "monthly",+ lastUpdated: "2026-01-01",+ },+ {+ slug: "fresh",+ status: "active",+ cadence: "monthly",+ lastUpdated: "2026-07-08",+ },+ {+ slug: "closed-old",+ status: "closed",+ cadence: "monthly",+ lastUpdated: "2020-01-01",+ },+ ],+ { now: NOW },+ );+ expect(stale.map((s) => s.slug)).toEqual(["old"]);+ expect(stale[0]!.nextDue).toBe("2026-02-01");+ });++ test("detectThesisGraveyard flags active theses with no support in N days", () => {+ const graveyard = detectThesisGraveyard(+ [+ {+ slug: "abandoned",+ status: "active",+ createdAt: "2026-01-01T00:00:00Z",+ lastSupportAt: "2026-02-01",+ },+ {+ slug: "recently-supported",+ status: "active",+ createdAt: "2026-01-01T00:00:00Z",+ lastSupportAt: "2026-07-01",+ },+ {+ slug: "never-supported-old",+ status: "active",+ createdAt: "2026-01-01T00:00:00Z",+ lastSupportAt: null,+ },+ ],+ { maxAgeDays: 30, now: NOW },+ );+ // Both the long-unsupported and the never-supported (falls back to+ // createdAt) are flagged; the recently-supported one is not.+ expect(graveyard.map((g) => g.slug).toSorted()).toEqual(["abandoned", "never-supported-old"]);+ for (const g of graveyard) expect(g.suggestion).toBe("close");+ });++ test("the register persists as an operator-readable markdown page", () => {+ const page = recordThesis(vault, {+ statement: "Remote work boosts long-term team productivity.",+ falsification: "Productivity gains disappear once headcount passes fifty people.",+ agent: "test-agent",+ now: NOW,+ });+ const raw = readFileSync(page.path, "utf8");+ expect(raw).toContain("status: active");+ expect(raw).toContain("cadence: monthly");+ expect(raw).toContain("Remote work boosts long-term team productivity.");+ expect(raw).toContain("Productivity gains disappear once headcount passes fifty people.");+ });+});diff --git a/tests/core/brain/idempotency-ledger.test.ts b/tests/core/brain/idempotency-ledger.test.tsnew file mode 100644index 00000000..5cd64ff3--- /dev/null+++ b/tests/core/brain/idempotency-ledger.test.ts@@ -0,0 +1,121 @@+/**+ * C1 (t_213f356b): client-supplied idempotency-key ledger.+ *+ * The ledger maps a client key -> content hash under+ * `Brain/logs/idempotency/<YYYY-MM>.jsonl` (month-sharded JSONL,+ * mirroring the continuity store's append/list model). It answers three+ * outcomes: `inserted` (first time), `duplicate_match` (same key + same+ * hash), and `payload_mismatch` (same key + different hash β never a+ * silent overwrite).+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { existsSync, mkdtempSync, readFileSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import {+ computePayloadHash,+ idempotencyLogPath,+ lookupKey,+ REMEMBER_KEY_STATUS,+ rememberKey,+} from "../../../src/core/brain/idempotency-ledger.ts";++let vault: string;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-idempotency-"));+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+});++describe("idempotency ledger", () => {+ test("first remember inserts and persists under the month shard", () => {+ const r = rememberKey(vault, {+ key: "client-key-1",+ contentHash: "hash-a",+ createdAt: "2026-05-14T10:00:00Z",+ });+ expect(r.status).toBe(REMEMBER_KEY_STATUS.inserted);+ expect(r.record.key).toBe("client-key-1");+ expect(r.record.contentHash).toBe("hash-a");++ const shard = idempotencyLogPath(vault, "2026-05");+ expect(existsSync(shard)).toBe(true);+ expect(readFileSync(shard, "utf8")).toContain("client-key-1");+ });++ test("same key + same hash is a deduped no-op (no second line)", () => {+ rememberKey(vault, { key: "k", contentHash: "hash-a", createdAt: "2026-05-14T10:00:00Z" });+ const shard = idempotencyLogPath(vault, "2026-05");+ const before = readFileSync(shard, "utf8");++ const r = rememberKey(vault, {+ key: "k",+ contentHash: "hash-a",+ createdAt: "2026-05-14T10:05:00Z",+ });+ expect(r.status).toBe(REMEMBER_KEY_STATUS.duplicate_match);+ // No new line appended β the shard is byte-identical.+ expect(readFileSync(shard, "utf8")).toBe(before);+ });++ test("same key + different hash reports payload_mismatch and writes nothing", () => {+ rememberKey(vault, { key: "k", contentHash: "hash-a", createdAt: "2026-05-14T10:00:00Z" });+ const shard = idempotencyLogPath(vault, "2026-05");+ const before = readFileSync(shard, "utf8");++ const r = rememberKey(vault, {+ key: "k",+ contentHash: "hash-b",+ createdAt: "2026-05-14T10:05:00Z",+ });+ expect(r.status).toBe(REMEMBER_KEY_STATUS.payload_mismatch);+ // The stored (original) hash is surfaced, never overwritten.+ expect(r.record.contentHash).toBe("hash-a");+ expect(readFileSync(shard, "utf8")).toBe(before);+ });++ test("lookupKey finds a stored key across shards and returns null otherwise", () => {+ rememberKey(vault, {+ key: "k",+ contentHash: "hash-a",+ createdAt: "2026-04-30T23:59:00Z",+ ref: { id: "sig-2026-04-30-topic" },+ });+ const found = lookupKey(vault, "k");+ expect(found?.contentHash).toBe("hash-a");+ expect(found?.ref).toEqual({ id: "sig-2026-04-30-topic" });+ expect(lookupKey(vault, "absent")).toBeNull();+ });++ test("a dedupe survives across a fresh read (the retry-after-crash case)", () => {+ rememberKey(vault, {+ key: "retry-key",+ contentHash: "hash-a",+ createdAt: "2026-05-14T10:00:00Z",+ });+ // A brand-new call (no in-memory state) still sees the prior key.+ const retry = rememberKey(vault, {+ key: "retry-key",+ contentHash: "hash-a",+ createdAt: "2026-05-14T11:00:00Z",+ });+ expect(retry.status).toBe(REMEMBER_KEY_STATUS.duplicate_match);+ });++ test("an empty key is rejected", () => {+ expect(() => rememberKey(vault, { key: " ", contentHash: "hash-a" })).toThrow();+ });++ test("computePayloadHash is deterministic and key-order-insensitive", () => {+ const a = computePayloadHash({ topic: "t", principle: "p", scope: "s" });+ const b = computePayloadHash({ scope: "s", principle: "p", topic: "t" });+ expect(a).toBe(b);+ const c = computePayloadHash({ topic: "t", principle: "different", scope: "s" });+ expect(c).not.toBe(a);+ });+});diff --git a/tests/core/brain/ingest/batch-plan.test.ts b/tests/core/brain/ingest/batch-plan.test.tsnew file mode 100644index 00000000..81e2cc66--- /dev/null+++ b/tests/core/brain/ingest/batch-plan.test.ts@@ -0,0 +1,168 @@+/**+ * Batch-plan step (A3, t_9eeb8ca2). `planBatches` discovers ingestible files+ * under a source dir, consults A1's content-hash manifest to skip `unchanged`+ * sources, then splits the `new`/`modified` remainder into size+count-bounded+ * batches an agent/CLI can dispatch as parallel subagents. The kernel runs no+ * ingestion here - it only plans. The plan is deterministic (sort by path, fill+ * greedily to the byte cap then the count cap).+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { join } from "node:path";+import { tmpdir } from "node:os";++import { bootstrapBrain } from "../../../../src/core/brain/init.ts";+import { atomicWriteFileSync } from "../../../../src/core/fs-atomic.ts";+import { updateManifest } from "../../../../src/core/brain/ingest/content-manifest.ts";+import { planBatches } from "../../../../src/core/brain/ingest/batch-plan.ts";++let vault: string;+let configHome: string;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-batchplan-vault-"));+ configHome = mkdtempSync(join(tmpdir(), "o2b-batchplan-cfg-"));+ const configPath = join(configHome, "config.yaml");+ atomicWriteFileSync(configPath, `vault: ${vault}\nagent_name: claude\n`);+ bootstrapBrain(vault, { configPath });+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+ rmSync(configHome, { recursive: true, force: true });+});++/** Write a source file of exactly `bytes` bytes (ASCII "x" fill). */+function writeSized(rel: string, bytes: number): void {+ const abs = join(vault, rel);+ mkdirSync(join(abs, ".."), { recursive: true });+ writeFileSync(abs, "x".repeat(bytes), "utf8");+}++function allPlannedPaths(plan: ReturnType<typeof planBatches>): string[] {+ return plan.batches.flatMap((b) => b.files.map((f) => f.path));+}++describe("planBatches β discovery + skip-unchanged", () => {+ test("skips `unchanged` files (via the manifest) and plans only new/modified", () => {+ writeSized("Inbox/keep.md", 100);+ writeSized("Inbox/edit.md", 100);+ writeSized("Inbox/fresh.md", 100);+ // Seed the manifest for keep + edit, then modify edit so it re-classifies.+ updateManifest(vault, ["Inbox/keep.md", "Inbox/edit.md"]);+ writeSized("Inbox/edit.md", 120);++ const plan = planBatches(vault, "Inbox", { maxBatchBytes: 10_000, maxBatchFiles: 100 });++ // keep is unchanged β skipped, never batched.+ expect(plan.skipped).toEqual(["Inbox/keep.md"]);+ const planned = allPlannedPaths(plan);+ expect(planned).toContain("Inbox/edit.md"); // modified+ expect(planned).toContain("Inbox/fresh.md"); // new+ expect(planned).not.toContain("Inbox/keep.md");+ expect(plan.totalFiles).toBe(2);+ });++ test("only ingestible (text-bearing) files are discovered; binaries/media ignored", () => {+ writeSized("Docs/a.md", 50);+ writeSized("Docs/b.txt", 50);+ writeSized("Docs/image.png", 50);+ writeSized("Docs/archive.zip", 50);++ const plan = planBatches(vault, "Docs", { maxBatchBytes: 10_000, maxBatchFiles: 100 });+ const planned = allPlannedPaths(plan).toSorted();+ expect(planned).toEqual(["Docs/a.md", "Docs/b.txt"]);+ });++ test("hidden files and dot-directories are not discovered", () => {+ writeSized("Notes/real.md", 50);+ writeSized("Notes/.secret.md", 50);+ writeSized("Notes/.git/config.md", 50);++ const plan = planBatches(vault, "Notes", { maxBatchBytes: 10_000, maxBatchFiles: 100 });+ expect(allPlannedPaths(plan)).toEqual(["Notes/real.md"]);+ });+});++describe("planBatches β bounded batches", () => {+ test("no batch exceeds the byte cap or the file-count cap", () => {+ // 6 files of 100 bytes each. Byte cap 250 β at most 2 per batch (200 ok,+ // 300 would exceed). File cap 5 is looser, so the byte cap governs here.+ for (let i = 0; i < 6; i++) writeSized(`Big/f${i}.md`, 100);++ const plan = planBatches(vault, "Big", { maxBatchBytes: 250, maxBatchFiles: 5 });+ expect(plan.totalFiles).toBe(6);+ for (const b of plan.batches) {+ expect(b.totalBytes).toBeLessThanOrEqual(250);+ expect(b.files.length).toBeLessThanOrEqual(5);+ }+ // Every file lands in exactly one batch.+ expect(allPlannedPaths(plan).length).toBe(6);+ });++ test("file-count cap governs when it is tighter than the byte cap", () => {+ for (let i = 0; i < 5; i++) writeSized(`Small/f${i}.md`, 10);++ const plan = planBatches(vault, "Small", { maxBatchBytes: 1_000_000, maxBatchFiles: 2 });+ // 5 files, cap 2 β batches of 2, 2, 1.+ expect(plan.batches.map((b) => b.files.length)).toEqual([2, 2, 1]);+ });++ test("a single file larger than the byte cap forms its own singleton batch", () => {+ writeSized("Huge/small.md", 10);+ writeSized("Huge/whale.md", 5_000);++ const plan = planBatches(vault, "Huge", { maxBatchBytes: 1_000, maxBatchFiles: 100 });+ // The oversize file cannot be split, so it lands alone; the small file is+ // its own batch. No batch mixes them.+ const whaleBatch = plan.batches.find((b) => b.files.some((f) => f.path === "Huge/whale.md"));+ expect(whaleBatch!.files.length).toBe(1);+ expect(whaleBatch!.totalBytes).toBe(5_000);+ });++ test("rejects non-positive caps (no silent fallback)", () => {+ writeSized("X/a.md", 10);+ expect(() => planBatches(vault, "X", { maxBatchBytes: 0, maxBatchFiles: 10 })).toThrow();+ expect(() => planBatches(vault, "X", { maxBatchBytes: 100, maxBatchFiles: 0 })).toThrow();+ });+});++describe("planBatches β determinism + empty", () => {+ test("same dir β identical plan, stable path ordering", () => {+ // Write out of lexical order to prove the plan sorts.+ writeSized("D/zebra.md", 100);+ writeSized("D/alpha.md", 100);+ writeSized("D/mango.md", 100);++ const p1 = planBatches(vault, "D", { maxBatchBytes: 250, maxBatchFiles: 10 });+ const p2 = planBatches(vault, "D", { maxBatchBytes: 250, maxBatchFiles: 10 });+ expect(p1).toEqual(p2);++ // Files are batched in sorted-by-path order.+ expect(allPlannedPaths(p1)).toEqual(["D/alpha.md", "D/mango.md", "D/zebra.md"]);+ });++ test("an empty dir yields an empty plan (no spurious batches)", () => {+ mkdirSync(join(vault, "Empty"), { recursive: true });+ const plan = planBatches(vault, "Empty", { maxBatchBytes: 100, maxBatchFiles: 10 });+ expect(plan.batches).toEqual([]);+ expect(plan.skipped).toEqual([]);+ expect(plan.totalFiles).toBe(0);+ });++ test("an all-unchanged dir yields an empty plan but reports the skipped files", () => {+ writeSized("Stable/a.md", 100);+ writeSized("Stable/b.md", 100);+ updateManifest(vault, ["Stable/a.md", "Stable/b.md"]);++ const plan = planBatches(vault, "Stable", { maxBatchBytes: 100, maxBatchFiles: 10 });+ expect(plan.batches).toEqual([]);+ expect(plan.totalFiles).toBe(0);+ expect(plan.skipped.toSorted()).toEqual(["Stable/a.md", "Stable/b.md"]);+ });++ test("a non-existent source dir is a hard error, not an empty plan", () => {+ expect(() => planBatches(vault, "Nope", { maxBatchBytes: 100, maxBatchFiles: 10 })).toThrow();+ });+});diff --git a/tests/core/brain/ingest/content-manifest.test.ts b/tests/core/brain/ingest/content-manifest.test.tsnew file mode 100644index 00000000..7f6c9357--- /dev/null+++ b/tests/core/brain/ingest/content-manifest.test.ts@@ -0,0 +1,198 @@+/**+ * Content-hash skip-unchanged manifest (A1). A re-ingest is driven by whether a+ * source's BYTES changed, not by its mtime: a `git checkout` or an NFS touch+ * that leaves bytes identical must classify `unchanged` and rewrite nothing.+ * The manifest is a machine artifact at `<vault>/.open-second-brain/`, NOT under+ * `Brain/`.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, readFileSync, rmSync, utimesSync, writeFileSync } from "node:fs";+import { join } from "node:path";+import { tmpdir } from "node:os";++import { bootstrapBrain } from "../../../../src/core/brain/init.ts";+import { atomicWriteFileSync } from "../../../../src/core/fs-atomic.ts";+import { ingestSource } from "../../../../src/core/brain/ingest/ingest.ts";+import {+ classifyPaths,+ hashFile,+ hashTree,+ manifestPath,+ readManifest,+ updateManifest,+ writeManifestAtomic,+} from "../../../../src/core/brain/ingest/content-manifest.ts";++let vault: string;+let configHome: string;++const NOW = new Date("2026-06-13T12:00:00Z");++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-manifest-vault-"));+ configHome = mkdtempSync(join(tmpdir(), "o2b-manifest-cfg-"));+ const configPath = join(configHome, "config.yaml");+ atomicWriteFileSync(configPath, `vault: ${vault}\nagent_name: claude\n`);+ bootstrapBrain(vault, { configPath });+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+ rmSync(configHome, { recursive: true, force: true });+});++function writeSource(rel: string, contents: string): void {+ const abs = join(vault, rel);+ mkdirSync(join(abs, ".."), { recursive: true });+ writeFileSync(abs, contents, "utf8");+}++describe("hashFile / hashTree", () => {+ test("hashFile is a 64-char lowercase hex SHA-256 over file bytes", () => {+ writeSource("a.md", "hello");+ const h = hashFile(join(vault, "a.md"));+ expect(h).toMatch(/^[0-9a-f]{64}$/);+ // Same bytes β same hash; different bytes β different hash.+ writeSource("b.md", "hello");+ expect(hashFile(join(vault, "b.md"))).toBe(h);+ writeSource("c.md", "hello!");+ expect(hashFile(join(vault, "c.md"))).not.toBe(h);+ });++ test("hashFile is timestamp-independent (mtime change, same bytes)", () => {+ writeSource("a.md", "hello");+ const h1 = hashFile(join(vault, "a.md"));+ utimesSync(join(vault, "a.md"), new Date("2000-01-01"), new Date("2000-01-01"));+ expect(hashFile(join(vault, "a.md"))).toBe(h1);+ });++ test("hashTree is deterministic over a directory and content-sensitive", () => {+ writeSource("dir/x.md", "one");+ writeSource("dir/y.md", "two");+ const h1 = hashTree(join(vault, "dir"));+ expect(h1).toMatch(/^[0-9a-f]{64}$/);+ // Recomputing over the same bytes is stable.+ expect(hashTree(join(vault, "dir"))).toBe(h1);+ // Changing a nested file changes the tree hash.+ writeSource("dir/y.md", "three");+ expect(hashTree(join(vault, "dir"))).not.toBe(h1);+ });+});++describe("classifyPaths", () => {+ test("classifies new / modified / unchanged / missing against the manifest", () => {+ writeSource("keep.md", "kept");+ writeSource("edit.md", "before");+ writeSource("gone.md", "temp");+ // Seed the manifest with post-ingest hashes for the three known paths.+ updateManifest(vault, ["keep.md", "edit.md", "gone.md"]);++ // Now: keep unchanged, edit modified, gone deleted, fresh is new.+ writeSource("edit.md", "after");+ rmSync(join(vault, "gone.md"));+ writeSource("fresh.md", "brand new");++ const manifest = readManifest(vault);+ const res = classifyPaths(vault, ["keep.md", "edit.md", "gone.md", "fresh.md"], manifest);++ expect(res.unchanged).toEqual(["keep.md"]);+ expect(res.modified).toEqual(["edit.md"]);+ expect(res.missing).toEqual(["gone.md"]);+ expect(res.new).toEqual(["fresh.md"]);+ });++ test("a byte-identical file with a bumped mtime is still `unchanged`", () => {+ writeSource("touched.md", "stable bytes");+ updateManifest(vault, ["touched.md"]);+ // Simulate a `git checkout` / NFS touch: mtime moves, bytes do not.+ utimesSync(join(vault, "touched.md"), new Date("2030-01-01"), new Date("2030-01-01"));++ const res = classifyPaths(vault, ["touched.md"], readManifest(vault));+ expect(res.unchanged).toEqual(["touched.md"]);+ expect(res.modified).toEqual([]);+ });+});++describe("manifest persistence", () => {+ test("manifest lives under .open-second-brain, not under Brain/", () => {+ expect(manifestPath(vault)).toBe(join(vault, ".open-second-brain", "ingest-manifest.json"));+ });++ test("writeManifestAtomic sorts keys and a no-op rerun is byte-identical", () => {+ writeSource("z.md", "z");+ writeSource("a.md", "a");+ updateManifest(vault, ["z.md", "a.md"]);++ const bytes1 = readFileSync(manifestPath(vault), "utf8");+ // Keys are serialized in sorted order for deterministic bytes.+ expect(bytes1.indexOf('"a.md"')).toBeLessThan(bytes1.indexOf('"z.md"'));++ // A no-op rerun (nothing changed) rewrites nothing β byte-identical.+ updateManifest(vault, ["z.md", "a.md"]);+ const bytes2 = readFileSync(manifestPath(vault), "utf8");+ expect(bytes2).toBe(bytes1);+ });++ test("updateManifest drops entries whose file was deleted", () => {+ writeSource("temp.md", "temp");+ updateManifest(vault, ["temp.md"]);+ expect(readManifest(vault).entries["temp.md"]).toMatch(/^[0-9a-f]{64}$/);++ rmSync(join(vault, "temp.md"));+ updateManifest(vault, ["temp.md"]);+ expect(readManifest(vault).entries["temp.md"]).toBeUndefined();+ });++ test("writeManifestAtomic reports whether it wrote", () => {+ expect(writeManifestAtomic(vault, { "a.md": "x".repeat(64) })).toBe(true);+ // Same entries β no write.+ expect(writeManifestAtomic(vault, { "a.md": "x".repeat(64) })).toBe(false);+ // Different entries β write.+ expect(writeManifestAtomic(vault, { "a.md": "y".repeat(64) })).toBe(true);+ });+});++describe("ingestSource integration", () => {+ const extraction = {+ entities: [{ category: "concept", name: "Restaking" }],+ relations: [],+ };++ test("re-ingesting a byte-identical source is a no-op: classified unchanged, summary not rewritten", () => {+ writeSource("Articles/primer.md", "the source bytes");+ const input = {+ sourcePath: "Articles/primer.md",+ summary: "An overview.",+ extraction,+ };++ const first = ingestSource(vault, input, { agent: "claude", now: NOW });+ const summaryAbs = join(vault, first.summaryPath);++ // The manifest now records the source's content hash.+ const res = classifyPaths(vault, ["Articles/primer.md"], readManifest(vault));+ expect(res.unchanged).toEqual(["Articles/primer.md"]);++ // Second ingest reaches steady state: the entity created by the first pass+ // now already exists, so it moves into the connections list β a legitimate+ // body change. Capture the steady-state bytes AFTER this pass.+ ingestSource(vault, input, { agent: "claude", now: NOW });+ const steadyBytes = readFileSync(summaryAbs, "utf8");++ // A further re-ingest of the byte-identical source is a true no-op: the+ // summary page is not rewritten (bytes only change when they would differ).+ ingestSource(vault, input, { agent: "claude", now: NOW });+ expect(readFileSync(summaryAbs, "utf8")).toBe(steadyBytes);+ });++ test("a source with no on-disk file (e.g. a URL) does not touch the manifest", () => {+ // Backward-compat: identity-only sources must not error or write a manifest.+ ingestSource(+ vault,+ { sourcePath: "https://example.com/post", summary: "s", extraction },+ { agent: "claude", now: NOW },+ );+ expect(readManifest(vault).entries).toEqual({});+ });+});diff --git a/tests/core/brain/pre-compact-extract-dry-run.test.ts b/tests/core/brain/pre-compact-extract-dry-run.test.tsnew file mode 100644index 00000000..8a6e1419--- /dev/null+++ b/tests/core/brain/pre-compact-extract-dry-run.test.ts@@ -0,0 +1,79 @@+import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdtempSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { listContinuityRecords } from "../../../src/core/brain/continuity/store.ts";+import { extractPreCompactRecords } from "../../../src/core/brain/pre-compact-extract.ts";++let vault: string;++const INPUT = Object.freeze({+ createdAt: "2026-05-20T17:00:00.000Z",+ host: "unit-test",+ sessionId: "session-preview",+ turnStart: "turn-1",+ turnEnd: "turn-3",+ text: [+ "Decision: Ship the dry-run preview.",+ "Commitment: Add a parity test.",+ "Outcome: Vault untouched.",+ "Rule: Preview must not mutate.",+ "Open question: Should this be default off?",+ ].join("\n"),+});++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-pre-compact-dry-run-"));+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+});++describe("pre-compact extraction dry-run preview", () => {+ test("dryRun: true returns candidate records but writes nothing to the vault", () => {+ const result = extractPreCompactRecords(vault, { ...INPUT, dryRun: true });++ expect(result.errors).toEqual([]);+ expect(result.records.map((record) => record.payload["extract_type"])).toEqual([+ "decision",+ "commitment",+ "outcome",+ "rule",+ "open_question",+ ]);+ // Nothing persisted β no continuity record, no dream/retire trigger.+ expect(listContinuityRecords(vault)).toHaveLength(0);+ });++ test("dry-run candidate records are deeply equal to the records the real path appends", () => {+ const dryVault = mkdtempSync(join(tmpdir(), "o2b-pre-compact-dry-"));+ try {+ const preview = extractPreCompactRecords(dryVault, { ...INPUT, dryRun: true });+ const real = extractPreCompactRecords(vault, INPUT);+ expect(preview.records).toEqual(real.records);+ expect(preview.skipped).toBe(real.skipped);+ expect(preview.errors).toEqual(real.errors);+ // The real path persisted; the dry-run path did not.+ expect(listContinuityRecords(vault)).toHaveLength(real.records.length);+ expect(listContinuityRecords(dryVault)).toHaveLength(0);+ } finally {+ rmSync(dryVault, { recursive: true, force: true });+ }+ });++ test("dryRun absent or false is byte-identical to today (records persisted)", () => {+ const absent = extractPreCompactRecords(vault, INPUT);+ expect(listContinuityRecords(vault)).toHaveLength(absent.records.length);++ const falseVault = mkdtempSync(join(tmpdir(), "o2b-pre-compact-false-"));+ try {+ const explicitFalse = extractPreCompactRecords(falseVault, { ...INPUT, dryRun: false });+ expect(explicitFalse.records).toEqual(absent.records);+ expect(listContinuityRecords(falseVault)).toHaveLength(explicitFalse.records.length);+ } finally {+ rmSync(falseVault, { recursive: true, force: true });+ }+ });+});diff --git a/tests/core/brain/preference-expiration.test.ts b/tests/core/brain/preference-expiration.test.tsnew file mode 100644index 00000000..6803bff1--- /dev/null+++ b/tests/core/brain/preference-expiration.test.ts@@ -0,0 +1,127 @@+/**+ * C5 (t_a82b674e): caller-settable per-memory expiration on preferences.+ *+ * A caller can stamp an explicit `expiration_date` on a preference at+ * write time. The default read/list path silently drops preferences past+ * their date; an opt-in `showExpired` flag re-includes them for audit.+ * Expiration is orthogonal to dream's heuristic retirement β an+ * expired-by-date preference is FILTERED, never moved to `Brain/retired/`.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import {+ parsePreference,+ writePreference,+ type WritePreferenceInput,+} from "../../../src/core/brain/preference.ts";+import { writeSignal } from "../../../src/core/brain/signal.ts";+import { queryByTopic } from "../../../src/core/brain/query.ts";+import { BRAIN_CONFIDENCE, BRAIN_PREFERENCE_STATUS } from "../../../src/core/brain/types.ts";++let vault: string;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-pref-expiry-"));+ mkdirSync(join(vault, "Brain", "preferences"), { recursive: true });+ mkdirSync(join(vault, "Brain", "inbox"), { recursive: true });+});+afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+});++function basePref(+ slug: string,+ overrides: Partial<WritePreferenceInput> = {},+): WritePreferenceInput {+ return {+ slug,+ topic: "deploy",+ principle: `Principle for ${slug}`,+ created_at: "2026-05-26T12:00:00Z",+ unconfirmed_until: "2026-06-02T12:00:00Z",+ status: BRAIN_PREFERENCE_STATUS.confirmed,+ evidenced_by: [],+ pinned: false,+ confidence: BRAIN_CONFIDENCE.low,+ ...overrides,+ };+}++describe("writePreference β expiration_date frontmatter", () => {+ test("stamps expiration_date into the frontmatter when supplied", () => {+ const res = writePreference(vault, basePref("with-expiry", { expiration_date: "2026-07-15" }));+ const text = readFileSync(res.path, "utf8");+ expect(text).toContain("expiration_date: 2026-07-15");+ });++ test("does NOT emit expiration_date when the caller omits it (byte-identical)", () => {+ const res = writePreference(vault, basePref("no-expiry"));+ const text = readFileSync(res.path, "utf8");+ expect(text).not.toContain("expiration_date");+ });++ test("parsePreference reads the expiration_date back", () => {+ const res = writePreference(vault, basePref("roundtrip", { expiration_date: "2026-07-15" }));+ const parsed = parsePreference(res.path);+ expect(parsed.expiration_date).toBe("2026-07-15");+ });++ test("a preference without an expiration parses with expiration_date undefined", () => {+ const res = writePreference(vault, basePref("no-expiry-parse"));+ const parsed = parsePreference(res.path);+ expect(parsed.expiration_date).toBeUndefined();+ });++ test("rejects an unparseable expiration_date on write", () => {+ expect(() => writePreference(vault, basePref("bad", { expiration_date: "someday" }))).toThrow();+ });+});++describe("queryByTopic β default drops expired, showExpired re-includes", () => {+ const now = new Date("2026-08-01T00:00:00Z");++ function seedExpiredPreference(): void {+ writePreference(vault, basePref("expired-rule", { expiration_date: "2026-07-15" }));+ // A signal so the topic is populated regardless of the preference filter.+ writeSignal(vault, {+ topic: "deploy",+ signal: "positive",+ agent: "tester",+ principle: "use the staging endpoint",+ created_at: "2026-05-01T00:00:00Z",+ date: "2026-05-01",+ slug: "deploy-live",+ });+ }++ test("default query drops a preference past its expiration_date", () => {+ seedExpiredPreference();+ const res = queryByTopic(vault, "deploy", { now });+ expect(res.preference).toBeNull();+ });++ test("showExpired: true re-includes the expired preference (audit path)", () => {+ seedExpiredPreference();+ const res = queryByTopic(vault, "deploy", { now, showExpired: true });+ expect(res.preference).not.toBeNull();+ expect(res.preference?.id).toBe("pref-expired-rule");+ });++ test("a still-live preference is returned by the default query", () => {+ writePreference(vault, basePref("live-rule", { expiration_date: "2026-12-31" }));+ const res = queryByTopic(vault, "deploy", { now });+ expect(res.preference?.id).toBe("pref-live-rule");+ });++ test("an expired preference is FILTERED, not moved to Brain/retired/ (audit trail preserved)", () => {+ seedExpiredPreference();+ // A read that drops the expired preference must not touch disk.+ queryByTopic(vault, "deploy", { now });+ expect(existsSync(join(vault, "Brain", "preferences", "pref-expired-rule.md"))).toBe(true);+ expect(existsSync(join(vault, "Brain", "retired", "ret-expired-rule.md"))).toBe(false);+ });+});diff --git a/tests/core/brain/recent-turns.test.ts b/tests/core/brain/recent-turns.test.tsnew file mode 100644index 00000000..b2e8f59c--- /dev/null+++ b/tests/core/brain/recent-turns.test.ts@@ -0,0 +1,135 @@+import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdtempSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { listContinuityRecords } from "../../../src/core/brain/continuity/store.ts";+import { extractPreCompactRecords } from "../../../src/core/brain/pre-compact-extract.ts";+import {+ RECENT_TURNS_CAP,+ appendRecentTurn,+ listRecentTurns,+ resurfaceRecentTurns,+} from "../../../src/core/brain/recent-turns.ts";++let vault: string;++/** Stamp turn N with a distinct, monotonically-increasing canonical UTC timestamp. */+function stampAt(index: number): string {+ const minute = String(index).padStart(2, "0");+ return `2026-05-20T17:${minute}:00.000Z`;+}++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-recent-turns-"));+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+});++describe("bounded verbatim last-N-turns buffer", () => {+ test("appending N+1 turns evicts the oldest β buffer never grows beyond N", () => {+ for (let index = 0; index <= RECENT_TURNS_CAP; index += 1) {+ appendRecentTurn(vault, {+ role: index % 2 === 0 ? "user" : "assistant",+ text: `turn number ${index}`,+ createdAt: stampAt(index),+ });+ }++ const turns = listRecentTurns(vault);+ // Hard cap enforced: exactly N turns, never more.+ expect(turns).toHaveLength(RECENT_TURNS_CAP);+ // The oldest turn (index 0) has been evicted from the buffer view.+ expect(turns.some((turn) => turn.text === "turn number 0")).toBe(false);+ // The newest turn survives and is last (chronological order preserved).+ expect(turns.at(-1)?.text).toBe(`turn number ${RECENT_TURNS_CAP}`);+ expect(turns.at(0)?.text).toBe("turn number 1");+ });++ test("a read limit above the hard cap is still clamped to N", () => {+ for (let index = 0; index < RECENT_TURNS_CAP + 5; index += 1) {+ appendRecentTurn(vault, { role: "user", text: `t${index}`, createdAt: stampAt(index) });+ }+ expect(listRecentTurns(vault, { limit: 9999 })).toHaveLength(RECENT_TURNS_CAP);+ expect(listRecentTurns(vault, { limit: 3 })).toHaveLength(3);+ });++ test("after a simulated compaction the exact recent wording survives", () => {+ appendRecentTurn(vault, {+ role: "user",+ text: "please refactor the parser to be streaming",+ sessionId: "sess-1",+ turnId: "turn-42",+ createdAt: stampAt(1),+ });+ appendRecentTurn(vault, {+ role: "assistant",+ text: "done β the parser now streams line by line",+ sessionId: "sess-1",+ turnId: "turn-43",+ createdAt: stampAt(2),+ });++ // Simulate compaction: nothing in-memory is retained; we re-read the+ // durable continuity-store artifact from disk in a fresh call.+ const survived = listRecentTurns(vault);+ expect(survived.map((turn) => turn.text)).toEqual([+ "please refactor the parser to be streaming",+ "done β the parser now streams line by line",+ ]);+ expect(survived[0]?.role).toBe("user");+ expect(survived[0]?.turnId).toBe("turn-42");+ expect(survived[0]?.sessionId).toBe("sess-1");+ });++ test("the buffer is clearly separated from other curated/continuity surfaces", () => {+ appendRecentTurn(vault, { role: "user", text: "verbatim scaffolding", createdAt: stampAt(1) });+ extractPreCompactRecords(vault, {+ createdAt: stampAt(2),+ sessionId: "sess-2",+ turnStart: "turn-1",+ turnEnd: "turn-1",+ text: "Decision: keep the two surfaces separate.",+ });++ // The recent-turn buffer has its own kind and does not leak into other+ // continuity kinds' reads.+ const recentKind = listContinuityRecords(vault, { kind: "recent_turn" });+ expect(recentKind).toHaveLength(1);+ expect(recentKind[0]?.kind).toBe("recent_turn");++ const extracts = listContinuityRecords(vault, { kind: "pre_compact_extract" });+ expect(extracts.every((record) => record.kind === "pre_compact_extract")).toBe(true);++ // Reading the buffer never surfaces the curated extract's wording.+ const turns = listRecentTurns(vault);+ expect(turns).toHaveLength(1);+ expect(turns.every((turn) => turn.text === "verbatim scaffolding")).toBe(true);+ });++ test("post-compaction re-surface is default-off β nothing surfaced", () => {+ appendRecentTurn(vault, { role: "user", text: "hello there", createdAt: stampAt(1) });++ // Off (default / explicit): byte-identical β nothing added to context.+ expect(resurfaceRecentTurns(vault, { enabled: false })).toBeNull();+ expect(resurfaceRecentTurns(vault, {})).toBeNull();++ // On (opt-in): the verbatim buffer is surfaced for recovery.+ const surfaced = resurfaceRecentTurns(vault, { enabled: true });+ expect(surfaced).not.toBeNull();+ expect(surfaced).toContain("hello there");+ });++ test("re-surface on an empty buffer returns null even when enabled", () => {+ expect(resurfaceRecentTurns(vault, { enabled: true })).toBeNull();+ });++ test("per-turn verbatim text is bounded (no unbounded hoarding)", () => {+ const huge = "x".repeat(1_000_000);+ const record = appendRecentTurn(vault, { role: "user", text: huge, createdAt: stampAt(1) });+ const stored = String(record.payload["text"]);+ expect(stored.length).toBeLessThan(huge.length);+ });+});diff --git a/tests/core/brain/sessions/import-event-time.test.ts b/tests/core/brain/sessions/import-event-time.test.tsnew file mode 100644index 00000000..ae241f38--- /dev/null+++ b/tests/core/brain/sessions/import-event-time.test.ts@@ -0,0 +1,232 @@+/**+ * Per-row event-time on batch remember/import (A2 / t_7526e8d3).+ *+ * Backfilling an old session log used to stamp every derived signal with+ * the import wall-clock, corrupting recency-based reconciliation. This+ * suite locks the historically-faithful behaviour:+ *+ * - With `preserveEventTime`, each signal carries its turn's ORIGINAL+ * timestamp in `created_at` / `recorded_at` / `valid_from`.+ * - A turn with no / unparseable / future-dated timestamp falls back to+ * `now` deterministically (never mints a future-dated or epoch signal).+ * - Without the flag, output is byte-identical to the wall-clock path.+ *+ * It also exercises the additive `WriteSignalInput` bi-temporal slots that+ * the read-side (`readBiTemporal`) already parses.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, readdirSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { brainDirs } from "../../../../src/core/brain/paths.ts";+import { DEFAULT_BRAIN_CONFIG_YAML } from "../../../../src/core/brain/policy.ts";+import { atomicWriteFileSync } from "../../../../src/core/fs-atomic.ts";+import { importSession } from "../../../../src/core/brain/sessions/import.ts";+import {+ parseSignal,+ writeSignal,+ type WriteSignalInput,+} from "../../../../src/core/brain/signal.ts";+import { isoSecond } from "../../../../src/core/brain/time.ts";+import type { BrainSignal } from "../../../../src/core/brain/types.ts";++let tmp: string;++beforeEach(() => {+ tmp = mkdtempSync(join(tmpdir(), "o2b-event-time-"));+ const dirs = brainDirs(tmp);+ for (const d of [+ dirs.brain,+ dirs.inbox,+ dirs.processed,+ dirs.preferences,+ dirs.retired,+ dirs.log,+ dirs.snapshots,+ ]) {+ mkdirSync(d, { recursive: true });+ }+ atomicWriteFileSync(join(dirs.brain, "_brain.yaml"), DEFAULT_BRAIN_CONFIG_YAML);+});++afterEach(() => {+ rmSync(tmp, { recursive: true, force: true });+});++/** Build a one-turn claude-format session with an `@osb` marker. */+function claudeSession(opts: { topic: string; principle: string; timestamp?: string }): string {+ const line: Record<string, unknown> = {+ parentUuid: null,+ sessionId: "s",+ entrypoint: "sdk-cli",+ type: "user",+ message: {+ role: "user",+ content: `@osb feedback positive topic=${opts.topic} principle="${opts.principle}"`,+ },+ uuid: "u1",+ };+ if (opts.timestamp !== undefined) line["timestamp"] = opts.timestamp;+ return JSON.stringify(line) + "\n";+}++function writeFixture(name: string, body: string): string {+ const p = join(tmp, name);+ writeFileSync(p, body);+ return p;+}++/** Parse the single signal the import produced. */+function onlySignal(vault: string): { sig: BrainSignal; file: string } {+ const inbox = brainDirs(vault).inbox;+ const files = readdirSync(inbox).filter((n) => n.startsWith("sig-") && n.endsWith(".md"));+ expect(files.length).toBe(1);+ const file = files[0]!;+ return { sig: parseSignal(join(inbox, file)), file };+}++const NOW = new Date("2026-07-07T12:00:00Z");++describe("importSession β per-row event-time (preserveEventTime)", () => {+ test("backfills created_at/recorded_at/valid_from with the turn's ORIGINAL timestamp", async () => {+ const fixture = writeFixture(+ "old.jsonl",+ claudeSession({+ topic: "evttime",+ principle: "stamp the original event time",+ timestamp: "2020-03-15T08:30:00Z",+ }),+ );++ const res = await importSession(tmp, fixture, {+ agent: "test",+ now: NOW,+ preserveEventTime: true,+ });+ expect(res.signals_created).toBe(1);++ const { sig, file } = onlySignal(tmp);+ expect(sig.created_at).toBe("2020-03-15T08:30:00Z");+ expect(sig.recorded_at).toBe("2020-03-15T08:30:00Z");+ expect(sig.valid_from).toBe("2020-03-15T08:30:00Z");+ // The filename calendar day reflects the event, not the import moment.+ expect(file.startsWith("sig-2020-03-15")).toBe(true);+ });++ test("a turn with NO timestamp falls back to now (backward-compatible)", async () => {+ const fixture = writeFixture(+ "no-ts.jsonl",+ claudeSession({ topic: "evttime", principle: "fall back to now when absent" }),+ );++ const res = await importSession(tmp, fixture, {+ agent: "test",+ now: NOW,+ preserveEventTime: true,+ });+ expect(res.signals_created).toBe(1);++ const { sig } = onlySignal(tmp);+ expect(sig.created_at).toBe(isoSecond(NOW));+ expect(sig.recorded_at).toBeUndefined();+ expect(sig.valid_from).toBeUndefined();+ });++ test("a FUTURE-dated turn timestamp falls back to now (never mints a future signal)", async () => {+ const fixture = writeFixture(+ "future.jsonl",+ claudeSession({+ topic: "evttime",+ principle: "clamp future dated turns",+ timestamp: "2099-01-01T00:00:00Z",+ }),+ );++ const res = await importSession(tmp, fixture, {+ agent: "test",+ now: NOW,+ preserveEventTime: true,+ });+ expect(res.signals_created).toBe(1);++ const { sig } = onlySignal(tmp);+ expect(sig.created_at).toBe(isoSecond(NOW));+ expect(sig.recorded_at).toBeUndefined();+ expect(sig.valid_from).toBeUndefined();+ });++ test("an UNPARSEABLE turn timestamp falls back to now (no throw)", async () => {+ const fixture = writeFixture(+ "bad-ts.jsonl",+ claudeSession({+ topic: "evttime",+ principle: "fall back to now when unparseable",+ timestamp: "not-a-timestamp",+ }),+ );++ const res = await importSession(tmp, fixture, {+ agent: "test",+ now: NOW,+ preserveEventTime: true,+ });+ expect(res.signals_created).toBe(1);++ const { sig } = onlySignal(tmp);+ expect(sig.created_at).toBe(isoSecond(NOW));+ expect(sig.recorded_at).toBeUndefined();+ expect(sig.valid_from).toBeUndefined();+ });++ test("without preserveEventTime, the wall-clock path is byte-identical (no event-time fields)", async () => {+ const fixture = writeFixture(+ "default.jsonl",+ claudeSession({+ topic: "evttime",+ principle: "ignore turn time by default",+ timestamp: "2020-03-15T08:30:00Z",+ }),+ );++ const res = await importSession(tmp, fixture, { agent: "test", now: NOW });+ expect(res.signals_created).toBe(1);++ const { sig } = onlySignal(tmp);+ // The old turn time is ignored entirely β stamped with the import clock.+ expect(sig.created_at).toBe(isoSecond(NOW));+ expect(sig.recorded_at).toBeUndefined();+ expect(sig.valid_from).toBeUndefined();+ });+});++describe("writeSignal β additive bi-temporal event-time slots", () => {+ const base: WriteSignalInput = {+ topic: "evttime",+ signal: "positive",+ agent: "test",+ principle: "carry the original event time",+ created_at: "2020-03-15T08:30:00Z",+ date: "2020-03-15",+ slug: "evttime",+ };++ test("valid_from + recorded_at flow into frontmatter the read-side parses", () => {+ const { path } = writeSignal(tmp, {+ ...base,+ valid_from: "2020-03-15T08:30:00Z",+ recorded_at: "2020-03-15T08:30:00Z",+ });+ const sig = parseSignal(path);+ expect(sig.valid_from).toBe("2020-03-15T08:30:00Z");+ expect(sig.recorded_at).toBe("2020-03-15T08:30:00Z");+ });++ test("absent slots leave the file free of event-time keys (byte-identity guard)", () => {+ const { path } = writeSignal(tmp, base);+ const sig = parseSignal(path);+ expect(sig.valid_from).toBeUndefined();+ expect(sig.recorded_at).toBeUndefined();+ });+});diff --git a/tests/core/brain/signal-expiration.test.ts b/tests/core/brain/signal-expiration.test.tsnew file mode 100644index 00000000..25b6374e--- /dev/null+++ b/tests/core/brain/signal-expiration.test.ts@@ -0,0 +1,93 @@+/**+ * C5 (t_a82b674e): caller-settable per-memory expiration on signals.+ *+ * A caller can stamp an explicit `expiration_date` on a signal at write+ * time. The default read/list path silently drops signals past their+ * date; an opt-in `showExpired` flag re-includes them for audit. An+ * expired-by-date signal is FILTERED on read, never deleted or moved.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { parseSignal, writeSignal, type WriteSignalInput } from "../../../src/core/brain/signal.ts";+import { queryByTopic } from "../../../src/core/brain/query.ts";++let vault: string;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-sig-expiry-"));+ mkdirSync(join(vault, "Brain", "inbox"), { recursive: true });+});+afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+});++function baseSignal(slug: string, overrides: Partial<WriteSignalInput> = {}): WriteSignalInput {+ return {+ topic: "deploy",+ signal: "positive",+ agent: "tester",+ principle: `Principle for ${slug}`,+ created_at: "2026-05-01T00:00:00Z",+ date: "2026-05-01",+ slug,+ ...overrides,+ };+}++describe("writeSignal β expiration_date frontmatter", () => {+ test("stamps expiration_date into the frontmatter when supplied", () => {+ const res = writeSignal(vault, baseSignal("with-expiry", { expiration_date: "2026-07-15" }));+ const text = readFileSync(res.path, "utf8");+ expect(text).toContain("expiration_date: 2026-07-15");+ });++ test("does NOT emit expiration_date when the caller omits it (byte-identical)", () => {+ const res = writeSignal(vault, baseSignal("no-expiry"));+ const text = readFileSync(res.path, "utf8");+ expect(text).not.toContain("expiration_date");+ });++ test("parseSignal reads the expiration_date back", () => {+ const res = writeSignal(vault, baseSignal("roundtrip", { expiration_date: "2026-07-15" }));+ const parsed = parseSignal(res.path);+ expect(parsed.expiration_date).toBe("2026-07-15");+ });++ test("rejects an unparseable expiration_date on write", () => {+ expect(() => writeSignal(vault, baseSignal("bad", { expiration_date: "whenever" }))).toThrow();+ });+});++describe("queryByTopic β default drops expired signals, showExpired re-includes", () => {+ const now = new Date("2026-08-01T00:00:00Z");++ function seed(): void {+ writeSignal(vault, baseSignal("live-sig", { expiration_date: "2026-12-31" }));+ writeSignal(vault, baseSignal("expired-sig", { expiration_date: "2026-07-15" }));+ writeSignal(vault, baseSignal("no-expiry-sig"));+ }++ test("default query drops signals past their expiration_date", () => {+ seed();+ const res = queryByTopic(vault, "deploy", { now });+ const principles = res.signals.map((s) => s.principle).sort();+ // The expired one is dropped; the live one and the never-expiring one stay.+ expect(principles).toEqual(["Principle for live-sig", "Principle for no-expiry-sig"]);+ });++ test("showExpired: true re-includes the expired signal (audit path)", () => {+ seed();+ const res = queryByTopic(vault, "deploy", { now, showExpired: true });+ expect(res.signals.length).toBe(3);+ });++ test("an expired signal is FILTERED, not deleted from the inbox", () => {+ const res = writeSignal(vault, baseSignal("expired-sig", { expiration_date: "2026-07-15" }));+ queryByTopic(vault, "deploy", { now });+ expect(existsSync(res.path)).toBe(true);+ });+});diff --git a/tests/core/brain/source-cleanup.test.ts b/tests/core/brain/source-cleanup.test.tsnew file mode 100644index 00000000..0bef256c--- /dev/null+++ b/tests/core/brain/source-cleanup.test.ts@@ -0,0 +1,252 @@+/**+ * C6 (t_edde2198): delete and search by exact source file.+ *+ * When a benchmark file, log, or accidental import pollutes a Brain,+ * operators need to (a) find every derived entry that traces back to an+ * exact source path and (b) surgically remove them, including index+ * artifacts, WITHOUT re-mining the vault or hand-chasing summary pages.+ *+ * The contract:+ * - `searchBySourceFile` returns only entries derived from that exact+ * source (frontmatter `source_path` / `session_ref`, a `[[source]]`+ * wikilink in prose, or a preference evidenced by a derived signal).+ * - `deleteBySource` is DRY-RUN BY DEFAULT: it reports the blast radius+ * and deletes NOTHING.+ * - With `confirm`, derived entries + index artifacts are deleted, but+ * original user notes are preserved unless `includeOriginals` is set.+ * - Every confirmed cleanup is auditable (a `source_invalidation`+ * continuity record is written).+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { ingestSource } from "../../../src/core/brain/ingest/ingest.ts";+import { readManifest } from "../../../src/core/brain/ingest/content-manifest.ts";+import { writeSignal } from "../../../src/core/brain/signal.ts";+import { writePreference } from "../../../src/core/brain/preference.ts";+import { listContinuityRecords } from "../../../src/core/brain/continuity/store.ts";+import { deleteBySource, searchBySourceFile } from "../../../src/core/brain/source-cleanup.ts";++let vault: string;++const SOURCE = "imports/benchmark.md";+const OTHER_SOURCE = "imports/legit-note.md";+const NOW = new Date("2026-06-01T00:00:00Z");++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-source-cleanup-"));+ mkdirSync(join(vault, "imports"), { recursive: true });+ mkdirSync(join(vault, "Brain", "inbox"), { recursive: true });+ mkdirSync(join(vault, "Brain", "preferences"), { recursive: true });+ mkdirSync(join(vault, "Brain", "log"), { recursive: true });+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+});++/**+ * Build a realistic contaminated Brain: an ingested source (summary page+ * + entity page), a signal citing the source, a preference evidenced by+ * that signal, plus unrelated material and a shared daily log that merely+ * mentions the source. Returns the derived signal id for assertions.+ */+function seedContaminatedVault(): { signalId: string } {+ // The original imported file itself.+ writeFileSync(join(vault, SOURCE), "benchmark rows\n", "utf8");+ writeFileSync(join(vault, OTHER_SOURCE), "a real note\n", "utf8");++ // Ingest the contaminated source: a summary page (kind: brain-source,+ // source_path === SOURCE) plus an entity page carrying `[[SOURCE]]`+ // provenance.+ ingestSource(+ vault,+ {+ sourcePath: SOURCE,+ summary: "Benchmark import summary.",+ extraction: {+ entities: [{ category: "concept", name: "BenchWidget" }],+ },+ },+ { agent: "tester", now: NOW },+ );++ // Ingest an unrelated legit source so isolation is testable.+ ingestSource(+ vault,+ {+ sourcePath: OTHER_SOURCE,+ summary: "A legitimate note.",+ extraction: { entities: [{ category: "concept", name: "RealThing" }] },+ },+ { agent: "tester", now: NOW },+ );++ // A signal that directly cites the contaminated source.+ const sig = writeSignal(vault, {+ topic: "bench",+ signal: "positive",+ agent: "tester",+ principle: "Derived from the contaminated benchmark import.",+ created_at: "2026-06-01T00:00:00Z",+ date: "2026-06-01",+ slug: "bench-derived",+ source: [`[[${SOURCE}]]`],+ });++ // An unrelated signal citing the legit source.+ writeSignal(vault, {+ topic: "legit",+ signal: "positive",+ agent: "tester",+ principle: "Derived from a legitimate note.",+ created_at: "2026-06-01T00:00:00Z",+ date: "2026-06-01",+ slug: "legit-derived",+ source: [`[[${OTHER_SOURCE}]]`],+ });++ // A preference folded from the contaminated signal (transitive).+ writePreference(vault, {+ slug: "bench-pref",+ topic: "bench",+ principle: "A preference folded from the contaminated signal.",+ created_at: "2026-06-01T00:00:00Z",+ unconfirmed_until: "2026-06-08T00:00:00Z",+ status: "confirmed",+ evidenced_by: [`[[${sig.id}]]`],+ });++ // A shared daily log that merely MENTIONS the source. Deleting the whole+ // log would destroy unrelated lines, so it must be reported, not removed.+ writeFileSync(+ join(vault, "Brain", "log", "2026-06-01.md"),+ `# Log\n\n- ingested [[${SOURCE}]]\n- unrelated line about other work\n`,+ "utf8",+ );++ return { signalId: sig.id };+}++describe("searchBySourceFile β exact source filter", () => {+ test("returns only entries derived from the exact source", () => {+ seedContaminatedVault();+ const hits = searchBySourceFile(vault, SOURCE);+ const paths = hits.map((h) => h.path);++ // The summary page, the entity page, the citing signal, the folded+ // preference, and the mentioning log all trace to the source.+ expect(paths.some((p) => p.startsWith("Brain/sources/"))).toBe(true);+ expect(paths.some((p) => p.startsWith("Brain/entities/"))).toBe(true);+ expect(paths.some((p) => p.startsWith("Brain/inbox/") && p.includes("bench-derived"))).toBe(+ true,+ );+ expect(paths.some((p) => p.startsWith("Brain/preferences/"))).toBe(true);+ expect(paths.some((p) => p.startsWith("Brain/log/"))).toBe(true);++ // Nothing derived from the OTHER source leaks in.+ expect(paths.some((p) => p.includes("legit-derived"))).toBe(false);+ expect(paths.some((p) => p.includes("RealThing") || p.includes("legit-note"))).toBe(false);+ });++ test("an unknown source yields no hits", () => {+ seedContaminatedVault();+ expect(searchBySourceFile(vault, "imports/never-ingested.md")).toEqual([]);+ });+});++describe("deleteBySource β dry-run by default", () => {+ test("with no confirm, reports the blast radius and deletes nothing", () => {+ seedContaminatedVault();+ const plan = deleteBySource(vault, SOURCE, { now: NOW });++ expect(plan.confirmed).toBe(false);+ expect(plan.deleted).toEqual([]);+ expect(plan.blastRadius).toBeGreaterThan(0);+ // Derived entries + at least one index artifact are reported.+ expect(plan.derived.length).toBeGreaterThan(0);+ expect(plan.derived.some((e) => e.isIndexArtifact)).toBe(true);+ // The shared log is reported as a protected mention, not a deletion.+ expect(plan.mentions.some((e) => e.path.startsWith("Brain/log/"))).toBe(true);+ // The original file is reported as an original.+ expect(plan.originals).toContain(SOURCE);+ // Nothing was written: no audit record, no manifest change.+ expect(plan.auditRecordId).toBeNull();+ expect(plan.manifestEntryRemoved).toBe(false);++ // Every referenced file still exists on disk.+ for (const entry of [...plan.derived, ...plan.mentions]) {+ expect(existsSync(join(vault, entry.path))).toBe(true);+ }+ expect(existsSync(join(vault, SOURCE))).toBe(true);+ // No continuity audit record was written.+ expect(listContinuityRecords(vault, { kind: "source_invalidation" })).toEqual([]);+ });+});++describe("deleteBySource β confirmed cleanup", () => {+ test("confirm (no includeOriginals) removes derived + index, preserves originals", () => {+ seedContaminatedVault();+ const plan = deleteBySource(vault, SOURCE, { confirm: true, now: NOW });++ expect(plan.confirmed).toBe(true);+ expect(plan.deleted.length).toBeGreaterThan(0);++ // Derived entries and index artifacts are gone.+ for (const entry of plan.derived) {+ expect(existsSync(join(vault, entry.path))).toBe(false);+ }+ // The manifest entry for the source (an index artifact) is dropped.+ expect(plan.manifestEntryRemoved).toBe(true);+ expect(readManifest(vault).entries[SOURCE]).toBeUndefined();++ // The ORIGINAL imported file is preserved.+ expect(existsSync(join(vault, SOURCE))).toBe(true);+ // The shared log (a protected mention) is preserved.+ expect(existsSync(join(vault, "Brain", "log", "2026-06-01.md"))).toBe(true);+ // Unrelated derived material survives.+ expect(existsSync(join(vault, OTHER_SOURCE))).toBe(true);++ // The cleanup is auditable.+ expect(plan.auditRecordId).not.toBeNull();+ const audit = listContinuityRecords(vault, { kind: "source_invalidation" });+ expect(audit.length).toBe(1);+ expect(audit[0]!.sourceRefs.some((r) => r.id === SOURCE)).toBe(true);+ });++ test("confirm + includeOriginals also removes the original file", () => {+ seedContaminatedVault();+ const plan = deleteBySource(vault, SOURCE, {+ confirm: true,+ includeOriginals: true,+ now: NOW,+ });++ expect(plan.confirmed).toBe(true);+ expect(plan.includeOriginals).toBe(true);+ // The original imported file is removed.+ expect(existsSync(join(vault, SOURCE))).toBe(false);+ expect(plan.deleted).toContain(SOURCE);+ // Unrelated originals are untouched.+ expect(existsSync(join(vault, OTHER_SOURCE))).toBe(true);+ // Still auditable.+ expect(plan.auditRecordId).not.toBeNull();+ });++ test("re-running a confirmed cleanup is a safe no-op (idempotent)", () => {+ seedContaminatedVault();+ deleteBySource(vault, SOURCE, { confirm: true, includeOriginals: true, now: NOW });+ const second = deleteBySource(vault, SOURCE, {+ confirm: true,+ includeOriginals: true,+ now: NOW,+ });+ expect(second.deleted).toEqual([]);+ expect(second.derived).toEqual([]);+ // A no-op confirmed run writes no audit record.+ expect(second.auditRecordId).toBeNull();+ });+});diff --git a/tests/core/brain/truth/grounding.test.ts b/tests/core/brain/truth/grounding.test.tsnew file mode 100644index 00000000..3d09f901--- /dev/null+++ b/tests/core/brain/truth/grounding.test.ts@@ -0,0 +1,278 @@+/**+ * Signed source-diversity grounding score (t_4678a91a / D1): a pure+ * projection over the same claim events `computeTruthState` folds. It+ * upgrades the binary CONTESTED flag into a signed -1..+1 measure whose+ * SIGN points to the better-supported side and whose MAGNITUDE reflects+ * both the balance of confirming vs contradicting INDEPENDENT sources+ * and how many independent sources back that direction. N mentions in+ * one document weigh far below N mentions across N independent sources.+ * Deterministic (counting + weighting, no LLM); never mutates history.+ */++import { describe, expect, test } from "bun:test";++import {+ computeGroundingScore,+ computeGroundings,+} from "../../../../src/core/brain/truth/grounding.ts";+import { computeTruthState } from "../../../../src/core/brain/truth/fold.ts";+import { computeTruthStateWithConflicts } from "../../../../src/core/brain/truth/conflicts.ts";+import type { ClaimEvent } from "../../../../src/core/brain/truth/types.ts";++function claim(over: Partial<ClaimEvent> = {}): ClaimEvent {+ return {+ v: 1,+ ts: "2026-06-01T10:00:00Z",+ agent: "claude-dev-agent",+ entity: "alice mason",+ aspect: "employer",+ value: "Google",+ valueKind: "text",+ source: "[[Brain/notes/standup.md]]",+ ...over,+ };+}++/** Convenience: fold events, then grounding for the (single) slot. */+function groundingOf(events: ClaimEvent[]) {+ const state = computeTruthState(events);+ const slot = state.slots[0]!;+ return computeGroundingScore(slot, events);+}++/** N distinct independent sources all asserting the same value. */+function independentAgreement(n: number, value = "Google"): ClaimEvent[] {+ return Array.from({ length: n }, (_, i) =>+ claim({+ value,+ ts: `2026-06-0${(i % 9) + 1}T10:00:00Z`,+ source: `[[Brain/notes/src-${i}.md]]`,+ agent: `agent-${i}`,+ }),+ );+}++describe("computeGroundingScore", () => {+ test("agreement by many independent sources scores strongly positive; one source is weaker", () => {+ const many = groundingOf(independentAgreement(6));+ const one = groundingOf([claim()]);++ expect(many.score).toBeGreaterThan(one.score);+ expect(many.score).toBeGreaterThan(0.5);+ expect(many.band).toBe("strongly_supported");+ expect(many.confidence).toBe("high");+ expect(many.supportingSources).toBe(6);+ expect(many.contradictingSources).toBe(0);++ expect(one.score).toBeGreaterThan(0);+ expect(one.score).toBeLessThan(0.5);+ expect(one.confidence).toBe("low");+ expect(one.supportingSources).toBe(1);+ });++ test("score is bounded to the -1..+1 interval", () => {+ const g = groundingOf(independentAgreement(50));+ expect(g.score).toBeLessThanOrEqual(1);+ expect(g.score).toBeGreaterThanOrEqual(-1);+ });++ test("a contested slot leans to the better-supported side with balance-scaled magnitude", () => {+ // Current value "Google" (latest) backed by 3 independent sources;+ // "Meta" contests from 1 independent source within the window.+ const events = [+ claim({+ value: "Meta",+ ts: "2026-06-02T10:00:00Z",+ source: "[[Brain/notes/m.md]]",+ agent: "a-m",+ }),+ claim({+ value: "Google",+ ts: "2026-06-05T10:00:00Z",+ source: "[[Brain/notes/g1.md]]",+ agent: "a-1",+ }),+ claim({+ value: "Google",+ ts: "2026-06-06T10:00:00Z",+ source: "[[Brain/notes/g2.md]]",+ agent: "a-2",+ }),+ claim({+ value: "Google",+ ts: "2026-06-07T10:00:00Z",+ source: "[[Brain/notes/g3.md]]",+ agent: "a-3",+ }),+ ];+ const g = groundingOf(events);+ expect(g.score).toBeGreaterThan(0); // sign points to the better-supported (current) side+ expect(g.supportingSources).toBe(3);+ expect(g.contradictingSources).toBe(1);+ });++ test("the sign points AWAY from the current value when an alternative is better supported", () => {+ // Current value "Google" (latest) has 1 source; "Meta" has 3 β contradicted.+ const events = [+ claim({+ value: "Meta",+ ts: "2026-06-01T10:00:00Z",+ source: "[[Brain/notes/m1.md]]",+ agent: "a-1",+ }),+ claim({+ value: "Meta",+ ts: "2026-06-02T10:00:00Z",+ source: "[[Brain/notes/m2.md]]",+ agent: "a-2",+ }),+ claim({+ value: "Meta",+ ts: "2026-06-03T10:00:00Z",+ source: "[[Brain/notes/m3.md]]",+ agent: "a-3",+ }),+ claim({+ value: "Google",+ ts: "2026-06-05T10:00:00Z",+ source: "[[Brain/notes/g.md]]",+ agent: "a-g",+ }),+ ];+ const g = groundingOf(events);+ expect(g.score).toBeLessThan(0);+ expect(g.band).toBe("contradicted");+ expect(g.supportingSources).toBe(1);+ expect(g.contradictingSources).toBe(3);+ });++ test("a genuinely balanced contest scores near zero and bands as contested", () => {+ const events = [+ claim({+ value: "Meta",+ ts: "2026-06-01T10:00:00Z",+ source: "[[Brain/notes/m1.md]]",+ agent: "a-1",+ }),+ claim({+ value: "Meta",+ ts: "2026-06-02T10:00:00Z",+ source: "[[Brain/notes/m2.md]]",+ agent: "a-2",+ }),+ claim({+ value: "Google",+ ts: "2026-06-05T10:00:00Z",+ source: "[[Brain/notes/g1.md]]",+ agent: "a-3",+ }),+ claim({+ value: "Google",+ ts: "2026-06-06T10:00:00Z",+ source: "[[Brain/notes/g2.md]]",+ agent: "a-4",+ }),+ ];+ const g = groundingOf(events);+ expect(Math.abs(g.score)).toBeLessThan(0.15);+ expect(g.band).toBe("contested");+ });++ test("N mentions in ONE document weigh far below N mentions across N independent sources", () => {+ // One source repeats "Google" 5 times, contradicted by one "Meta" source.+ const oneDoc = [+ ...Array.from({ length: 5 }, (_, i) =>+ claim({ value: "Google", ts: `2026-06-0${i + 4}T10:00:00Z` }),+ ),+ claim({+ value: "Meta",+ ts: "2026-06-02T10:00:00Z",+ source: "[[Brain/notes/m.md]]",+ agent: "a-m",+ }),+ ];+ // Five distinct sources each assert "Google" once, same lone "Meta".+ const manyDocs = [+ ...independentAgreement(5),+ claim({+ value: "Meta",+ ts: "2026-06-02T10:00:00Z",+ source: "[[Brain/notes/m.md]]",+ agent: "a-m",+ }),+ ];++ const gOne = groundingOf(oneDoc);+ const gMany = groundingOf(manyDocs);++ expect(gOne.supportingSources).toBe(1);+ expect(gMany.supportingSources).toBe(5);+ expect(gMany.score).toBeGreaterThan(gOne.score);+ });++ test("a source that later confirms is self-correction, not a contradictor", () => {+ // Same (source, agent) asserts "Meta" then "Google": counts as support only.+ const events = [+ claim({ value: "Meta", ts: "2026-06-01T10:00:00Z" }),+ claim({ value: "Google", ts: "2026-06-05T10:00:00Z" }),+ ];+ const g = groundingOf(events);+ expect(g.contradictingSources).toBe(0);+ expect(g.supportingSources).toBe(1);+ });++ test("a value superseded outside the conflict window does not drag the score", () => {+ const events = [+ claim({+ value: "Meta",+ ts: "2026-01-01T10:00:00Z",+ source: "[[Brain/notes/old.md]]",+ agent: "a-old",+ }),+ claim({+ value: "Google",+ ts: "2026-06-05T10:00:00Z",+ source: "[[Brain/notes/new.md]]",+ agent: "a-new",+ }),+ ];+ const g = groundingOf(events);+ expect(g.contradictingSources).toBe(0);+ expect(g.score).toBeGreaterThan(0);+ });++ test("is deterministic and order-insensitive; never mutates its input", () => {+ const events = [+ ...independentAgreement(4),+ claim({+ value: "Meta",+ ts: "2026-06-02T10:00:00Z",+ source: "[[Brain/notes/m.md]]",+ agent: "a-m",+ }),+ ];+ const snapshot = JSON.stringify(events);+ const forward = groundingOf(events);+ const shuffled = groundingOf([...events].toReversed());++ expect(shuffled).toEqual(forward);+ expect(JSON.stringify(events)).toBe(snapshot); // input untouched+ });++ test("computeGroundings surfaces one grounding per slot alongside the fold", () => {+ const events = [+ claim(),+ claim({ entity: "bob hale", value: "Acme", source: "[[Brain/notes/bob.md]]", agent: "a-b" }),+ ];+ const state = computeTruthStateWithConflicts(events);+ const groundings = computeGroundings(state, events);+ expect(groundings).toHaveLength(state.slots.length);+ expect(groundings.map((g) => g.entity)).toEqual(state.slots.map((s) => s.entity));+ for (const g of groundings) {+ expect(g.grounding.score).toBeGreaterThanOrEqual(-1);+ expect(g.grounding.score).toBeLessThanOrEqual(1);+ }+ // The projection leaves the fold byte-identical.+ expect(computeTruthState(events)).toEqual(computeTruthState(events));+ });+});diff --git a/tests/core/brain/writer-idempotency.test.ts b/tests/core/brain/writer-idempotency.test.tsnew file mode 100644index 00000000..52c46293--- /dev/null+++ b/tests/core/brain/writer-idempotency.test.ts@@ -0,0 +1,192 @@+/**+ * C1 (t_213f356b): the write path honours an optional client-supplied+ * `idempotency_key` on writeSignal / writePreference / appendApplyEvidence.+ *+ * Invariants under test:+ * 1. No key -> the written file is byte-identical to the historical path.+ * 2. Same key + same payload -> deduped (no second file / no second row).+ * 3. Same key + different payload -> throws IdempotencyPayloadMismatchError.+ * 4. A retried write (same key, same payload, fresh call) is deduped β+ * the multi-runtime retry-after-crash case.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdtempSync, readFileSync, readdirSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { bootstrapBrain } from "../../../src/core/brain/init.ts";+import { writeSignal, type WriteSignalInput } from "../../../src/core/brain/signal.ts";+import { writePreference } from "../../../src/core/brain/preference.ts";+import { appendApplyEvidence } from "../../../src/core/brain/apply-evidence.ts";+import { brainDirs } from "../../../src/core/brain/paths.ts";+import { IdempotencyPayloadMismatchError } from "../../../src/core/brain/idempotency-ledger.ts";++let vault: string;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-writer-idem-"));+ bootstrapBrain(vault);+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+});++function baseSignal(overrides: Partial<WriteSignalInput> = {}): WriteSignalInput {+ return {+ topic: "no-abbrev",+ signal: "positive",+ agent: "tester",+ principle: "spell it out",+ created_at: "2026-05-14T10:00:00Z",+ date: "2026-05-14",+ slug: "no-abbrev",+ ...overrides,+ };+}++function inboxFiles(): string[] {+ return readdirSync(brainDirs(vault).inbox).filter((f) => f.endsWith(".md"));+}++describe("writeSignal idempotency key", () => {+ test("no key writes byte-identically to the historical path", () => {+ writeSignal(vault, baseSignal());+ const withoutKey = readFileSync(join(brainDirs(vault).inbox, inboxFiles()[0]!), "utf8");++ // A separate vault, same input, no key at all.+ const vault2 = mkdtempSync(join(tmpdir(), "o2b-writer-idem2-"));+ bootstrapBrain(vault2);+ const r2 = writeSignal(vault2, baseSignal());+ const other = readFileSync(r2.path, "utf8");+ rmSync(vault2, { recursive: true, force: true });++ expect(other).toBe(withoutKey);+ });++ test("same key + same payload dedupes to a single signal file", () => {+ const first = writeSignal(vault, baseSignal({ idempotency_key: "abc" }));+ expect(first.deduped).toBeUndefined();++ const second = writeSignal(vault, baseSignal({ idempotency_key: "abc" }));+ expect(second.deduped).toBe(true);+ expect(second.id).toBe(first.id);+ expect(inboxFiles()).toHaveLength(1);+ });++ test("same key + different payload throws payload_mismatch", () => {+ writeSignal(vault, baseSignal({ idempotency_key: "abc" }));+ expect(() =>+ writeSignal(vault, baseSignal({ idempotency_key: "abc", principle: "totally different" })),+ ).toThrow(IdempotencyPayloadMismatchError);+ // No partial second file.+ expect(inboxFiles()).toHaveLength(1);+ });++ test("a retried write after a simulated crash is deduped, not duplicated", () => {+ writeSignal(+ vault,+ baseSignal({ idempotency_key: "retry", created_at: "2026-05-14T10:00:00Z" }),+ );+ // Fresh call, later wall-clock β a redelivery of the same logical write.+ const retry = writeSignal(+ vault,+ baseSignal({ idempotency_key: "retry", created_at: "2026-05-14T12:30:00Z" }),+ );+ expect(retry.deduped).toBe(true);+ expect(inboxFiles()).toHaveLength(1);+ });+});++describe("writePreference idempotency key", () => {+ test("same key + same payload dedupes", () => {+ const base = {+ slug: "rule-a",+ topic: "coding",+ principle: "prefer X",+ created_at: "2026-05-14T10:00:00Z",+ unconfirmed_until: "2026-05-14T10:00:00Z",+ status: "confirmed" as const,+ evidenced_by: ["[[sig-1]]"],+ idempotency_key: "pref-key",+ };+ const first = writePreference(vault, base);+ const second = writePreference(vault, { ...base, created_at: "2026-05-14T12:00:00Z" });+ expect(second.deduped).toBe(true);+ expect(second.id).toBe(first.id);+ });++ test("same key + different payload throws", () => {+ const base = {+ slug: "rule-b",+ topic: "coding",+ principle: "prefer X",+ created_at: "2026-05-14T10:00:00Z",+ unconfirmed_until: "2026-05-14T10:00:00Z",+ status: "confirmed" as const,+ evidenced_by: ["[[sig-1]]"],+ idempotency_key: "pref-key-b",+ };+ writePreference(vault, base);+ expect(() => writePreference(vault, { ...base, principle: "prefer Y" })).toThrow(+ IdempotencyPayloadMismatchError,+ );+ });+});++describe("appendApplyEvidence idempotency key", () => {+ beforeEach(() => {+ writePreference(vault, {+ slug: "rule-c",+ topic: "coding",+ principle: "prefer X",+ created_at: "2026-05-14T10:00:00Z",+ unconfirmed_until: "2026-05-14T10:00:00Z",+ status: "confirmed",+ evidenced_by: ["[[sig-1]]"],+ });+ });++ test("same key + same payload dedupes to a single log row", () => {+ const first = appendApplyEvidence(vault, {+ pref_id: "rule-c",+ artifact: "[[work]]",+ result: "applied",+ agent: "tester",+ idempotency_key: "ev-key",+ });+ const logBody = readFileSync(first.log_path, "utf8");+ const occurrences = (body: string): number => body.split("apply-evidence").length - 1;+ const before = occurrences(logBody);++ const second = appendApplyEvidence(vault, {+ pref_id: "rule-c",+ artifact: "[[work]]",+ result: "applied",+ agent: "tester",+ idempotency_key: "ev-key",+ });+ expect(second.deduped).toBe(true);+ expect(occurrences(readFileSync(first.log_path, "utf8"))).toBe(before);+ });++ test("same key + different payload throws", () => {+ appendApplyEvidence(vault, {+ pref_id: "rule-c",+ artifact: "[[work]]",+ result: "applied",+ agent: "tester",+ idempotency_key: "ev-key-2",+ });+ expect(() =>+ appendApplyEvidence(vault, {+ pref_id: "rule-c",+ artifact: "[[other]]",+ result: "applied",+ agent: "tester",+ idempotency_key: "ev-key-2",+ }),+ ).toThrow(IdempotencyPayloadMismatchError);+ });+});diff --git a/tests/core/install/adapters/aider-wrapper.test.ts b/tests/core/install/adapters/aider-wrapper.test.tsnew file mode 100644index 00000000..602123e2--- /dev/null+++ b/tests/core/install/adapters/aider-wrapper.test.ts@@ -0,0 +1,161 @@+import { describe, expect, test, beforeEach, afterEach } from "bun:test";+import { mkdtempSync, rmSync, readFileSync, existsSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";+import { Writable } from "node:stream";++import {+ bracketAiderSession,+ isInterruptedExit,+ renderAiderSidecar,+ type AiderSpawnResult,+ type BracketAiderDeps,+} from "../../../../src/core/install/adapters/aider-wrapper.ts";+import { aiderAdapter } from "../../../../src/core/install/adapters/aider.ts";+import { buildPayload } from "../../../../src/core/install/payload.ts";+import { extractPreCompactRecords } from "../../../../src/core/brain/pre-compact-extract.ts";+import { listContinuityRecords } from "../../../../src/core/brain/continuity/store.ts";++let vault: string;+let home: string;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "osb-aiderwrap-v-"));+ home = mkdtempSync(join(tmpdir(), "osb-aiderwrap-h-"));+});+afterEach(() => {+ for (const p of [vault, home]) {+ try {+ rmSync(p, { recursive: true, force: true });+ } catch {}+ }+});++function env(now = new Date("2026-07-07T12:00:00.000Z"), envVars: Record<string, string> = {}) {+ return { vault, home, cwd: home, env: envVars, now };+}++function applyOpts(overrides: Record<string, unknown> = {}) {+ const sink = new Writable({+ write(_c, _e, cb) {+ cb();+ },+ });+ return {+ dryRun: false,+ force: false,+ stdout: sink as unknown as NodeJS.WriteStream,+ stderr: sink as unknown as NodeJS.WriteStream,+ ...overrides,+ };+}++const payload = buildPayload({ vault: "/v", agent_name: "claude-vps", timezone: "UTC" });++/** Deps that record phase ordering and let each test override behaviour. */+function makeDeps(+ over: Partial<BracketAiderDeps> & { exit?: AiderSpawnResult; transcript?: string } = {},+): { deps: BracketAiderDeps; log: string[]; persistInputs: unknown[] } {+ const log: string[] = [];+ const persistInputs: unknown[] = [];+ const exit = over.exit ?? { code: 0, signal: null };+ const transcript = over.transcript ?? "decision: adopt the wrapper\ncommitment: land E1\n";+ const deps: BracketAiderDeps = {+ loadContext:+ over.loadContext ??+ (() => {+ log.push("load");+ return join(vault, ".open-second-brain", "aider-context.md");+ }),+ spawnAider:+ over.spawnAider ??+ (async () => {+ log.push("spawn");+ return exit;+ }),+ captureTranscript: over.captureTranscript ?? (() => transcript),+ persist:+ over.persist ??+ ((input) => {+ log.push("persist");+ persistInputs.push(input);+ return extractPreCompactRecords(vault, input);+ }),+ };+ return { deps, log, persistInputs };+}++describe("aider session-bracketing wrapper", () => {+ test("load-half fires before Aider is spawned; persist-half fires after it exits", async () => {+ const { deps, log, persistInputs } = makeDeps();+ const result = await bracketAiderSession({ sessionId: "s1" }, deps);++ expect(log).toEqual(["load", "spawn", "persist"]);+ expect(result.phases).toEqual(["load", "spawn", "persist"]);+ // The load-half ran, then Aider, then the write-back persisted the session.+ expect(result.interrupted).toBe(false);+ expect((persistInputs[0] as { text: string }).text).toContain("decision: adopt the wrapper");+ // The captured session was persisted into the Brain (write-back closed).+ const records = listContinuityRecords(vault, { kind: "pre_compact_extract" });+ expect(records.length).toBeGreaterThan(0);+ expect(records.some((r) => r.payload["extract_type"] === "decision")).toBe(true);+ });++ test("live load-half regenerates the sidecar each session start", async () => {+ // The real load-half writes a fresh sidecar (live), unlike the static+ // install-time snapshot. renderAiderSidecar is the shared source of truth.+ const content = renderAiderSidecar(env(), payload);+ expect(content).toContain("@claude-vps");+ expect(content).toContain(vault);+ });++ test("static sidecar fallback path stays byte-identical (adapter output unchanged)", () => {+ // The wrapper reuses the adapter's snapshot logic (DRY). The adapter's+ // written sidecar must equal the shared renderer's output byte-for-byte.+ aiderAdapter.apply(aiderAdapter.plan(payload, env()), payload, env(), applyOpts());+ const sidecarPath = join(vault, ".open-second-brain", "aider-context.md");+ expect(existsSync(sidecarPath)).toBe(true);+ const written = readFileSync(sidecarPath, "utf8");+ expect(written).toBe(renderAiderSidecar(env(), payload));+ });++ test("interrupted session (non-zero exit) is captured honestly as interrupted", async () => {+ const { deps, persistInputs } = makeDeps({ exit: { code: 130, signal: null } });+ const result = await bracketAiderSession({ sessionId: "s-int" }, deps);++ expect(result.interrupted).toBe(true);+ expect((persistInputs[0] as { interrupted?: boolean }).interrupted).toBe(true);+ const records = listContinuityRecords(vault, { kind: "pre_compact_extract" });+ expect(records.length).toBeGreaterThan(0);+ expect(records.every((r) => r.payload["interrupted"] === true)).toBe(true);+ });++ test("interrupted session (signal) is captured honestly as interrupted", async () => {+ const { deps } = makeDeps({ exit: { code: null, signal: "SIGTERM" } });+ const result = await bracketAiderSession({ sessionId: "s-sig" }, deps);+ expect(result.interrupted).toBe(true);+ });++ test("clean exit persists without an interrupted flag (byte-identical continuity record)", async () => {+ const { deps, persistInputs } = makeDeps({ exit: { code: 0, signal: null } });+ await bracketAiderSession({ sessionId: "s-clean" }, deps);+ expect((persistInputs[0] as { interrupted?: boolean }).interrupted).toBeUndefined();+ const records = listContinuityRecords(vault, { kind: "pre_compact_extract" });+ expect(records.every((r) => r.payload["interrupted"] === undefined)).toBe(true);+ });++ test("an empty transcript persists nothing (no spurious records)", async () => {+ const { deps, log } = makeDeps({ transcript: " \n " });+ const result = await bracketAiderSession({ sessionId: "s-empty" }, deps);+ expect(log).toEqual(["load", "spawn"]);+ expect(result.persisted).toBeNull();+ expect(listContinuityRecords(vault, { kind: "pre_compact_extract" }).length).toBe(0);+ });++ test("isInterruptedExit: non-zero code or a signal is interrupted; clean zero is not", () => {+ expect(isInterruptedExit({ code: 0, signal: null })).toBe(false);+ expect(isInterruptedExit({ code: 1, signal: null })).toBe(true);+ expect(isInterruptedExit({ code: null, signal: "SIGINT" })).toBe(true);+ expect(isInterruptedExit({ code: null, signal: null })).toBe(true);+ });+});diff --git a/tests/mcp/brain-tools-parity.test.ts b/tests/mcp/brain-tools-parity.test.tsindex 6872220d..323babbd 100644--- a/tests/mcp/brain-tools-parity.test.ts+++ b/tests/mcp/brain-tools-parity.test.ts@@ -30,7 +30,11 @@* separate from a modeled, outcome-calibrated inference-avoidance estimate)* and `brain_context_pack_outcome` (agent-operable outcome loop: a compact* per-sample outcome row keeping the exact/modeled/observed token signals- * strictly separate, composing the token-impact ledger's calibration).+ * strictly separate, composing the token-impact ledger's calibration); the+ * memory-signal-provenance-lifecycle release added `brain_ingest_batch_plan`+ * (deterministic large-folder ingest planner: skips unchanged sources via the+ * content-hash manifest and shards the remainder into size+count-bounded+ * batches for parallel-subagent dispatch).*/import { describe, expect, test } from "bun:test";@@ -59,6 +63,7 @@ const FROZEN_BRAIN_TOOL_NAMES = ["brain_create_note","brain_dead_ends","brain_deep_synthesis",+ "brain_delete_by_source","brain_derive_fact","brain_doctor","brain_dream",@@ -71,6 +76,7 @@ const FROZEN_BRAIN_TOOL_NAMES = ["brain_hygiene","brain_idea_discovery","brain_idea_lineage",+ "brain_ingest_batch_plan","brain_ingest_source","brain_intake_entities","brain_intent_review",@@ -96,7 +102,9 @@ const FROZEN_BRAIN_TOOL_NAMES = ["brain_retention","brain_review_candidates","brain_route_metrics",+ "brain_search_by_source","brain_secrets",+ "brain_session_checkpoint","brain_session_describe","brain_session_expand","brain_session_grep",diff --git a/tests/mcp/mcp.test.ts b/tests/mcp/mcp.test.tsindex f8f1c4b6..6be23e64 100644--- a/tests/mcp/mcp.test.ts+++ b/tests/mcp/mcp.test.ts@@ -248,6 +248,8 @@ describe("tool listing", () => {"brain_file_context",// Session Knowledge Synthesis Suite (v1.11.0)."brain_session_summary",+ // Batch checkpoint save (memory-signal-provenance-lifecycle C4).+ "brain_session_checkpoint","brain_idea_lineage","brain_note_history",// Schema admin + watchdog recovery probes.@@ -287,6 +289,11 @@ describe("tool listing", () => {// Knowledge Provenance Suite (v1.7.0)."brain_intake_entities","brain_ingest_source",+ // Large-folder ingest planner (memory-signal-provenance-lifecycle A3).+ "brain_ingest_batch_plan",+ // Source cleanup: search + delete by exact source file (C6, v1.x).+ "brain_search_by_source",+ "brain_delete_by_source","brain_research_report","brain_derive_fact",// Calendar integration: recurring obligations + agenda synthesis.@@ -575,7 +582,12 @@ describe("stdio loop", () => {// + brain_route_metrics (route-level MCP latency, context-pack-economics-observability) = 90.// + brain_token_impact (durable token-impact ledger, context-pack-economics-observability) = 91.// + brain_context_pack_outcome (agent-operable outcome loop, context-pack-economics-observability) = 92.- expect(list.result.tools.length).toBe(92);+ // + brain_session_checkpoint (batch checkpoint save, memory-signal-provenance-lifecycle C4) = 93.+ // + brain_search_by_source / brain_delete_by_source (delete & search by exact+ // source file, memory-signal-provenance-lifecycle C6) = 95.+ // + brain_ingest_batch_plan (large-folder ingest planner,+ // memory-signal-provenance-lifecycle A3) = 96.+ expect(list.result.tools.length).toBe(96);});test("returns parse error for invalid JSON", async () => {diff --git a/tests/mcp/pre-compact-extract-tool.test.ts b/tests/mcp/pre-compact-extract-tool.test.tsindex f05cbcd9..cfc57848 100644--- a/tests/mcp/pre-compact-extract-tool.test.ts+++ b/tests/mcp/pre-compact-extract-tool.test.ts@@ -74,4 +74,25 @@ describe("brain_pre_compact_extract tool", () => {);expect(JSON.stringify(first)).not.toContain("QUJDREVGR0hJSktMTU5PUFFSU1RVVldYWVo");});++ test("dry_run previews candidate records without persisting", async () => {+ const args = {+ session_id: "session-preview",+ turn_start: "turn-1",+ turn_end: "turn-1",+ text: "Decision: Preview only.",+ };+ const preview = await callExtract({ ...args, dry_run: true });+ expect(preview).toMatchObject({ count: 1, dry_run: true, errors: [] });++ // The preview's logical identity (timestamp-independent dedupe_key)+ // matches the record the real path later appends β the preview predicts+ // real extraction. (record `id` folds in the ms-precise createdAt, which+ // differs between two un-pinned calls, so compare dedupe_key.)+ const real = await callExtract(args);+ expect(real).toMatchObject({ count: 1, dry_run: false });+ const key = (recs: unknown): string =>+ (recs as Array<{ payload: { dedupe_key: string } }>)[0]!.payload.dedupe_key;+ expect(key(preview.records)).toBe(key(real.records));+ });});diff --git a/tests/mcp/removed-tools.test.ts b/tests/mcp/removed-tools.test.tsindex 7579c521..a9d4a984 100644--- a/tests/mcp/removed-tools.test.ts+++ b/tests/mcp/removed-tools.test.ts@@ -152,5 +152,10 @@ test("the shadow surface is gone: no hidden tools, removed names unlisted", asyn// + brain_route_metrics (route-level MCP latency, context-pack-economics-observability) = 90.// + brain_token_impact (durable token-impact ledger, context-pack-economics-observability) = 91.// + brain_context_pack_outcome (agent-operable outcome loop, context-pack-economics-observability) = 92.- expect(list.result.tools.length).toBe(92);+ // + brain_session_checkpoint (batch checkpoint save, memory-signal-provenance-lifecycle C4) = 93.+ // + brain_search_by_source / brain_delete_by_source (delete & search by exact+ // source file, memory-signal-provenance-lifecycle C6) = 95.+ // + brain_ingest_batch_plan (large-folder ingest planner,+ // memory-signal-provenance-lifecycle A3) = 96.+ expect(list.result.tools.length).toBe(96);});diff --git a/tests/mcp/session-checkpoint.test.ts b/tests/mcp/session-checkpoint.test.tsnew file mode 100644index 00000000..c41204cf--- /dev/null+++ b/tests/mcp/session-checkpoint.test.ts@@ -0,0 +1,135 @@+import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { existsSync, mkdtempSync, readdirSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { JSONRPC_VERSION, MCPServer, PROTOCOL_VERSION } from "../../src/mcp/index.ts";+import { buildToolTable } from "../../src/mcp/tools.ts";++let vault: string;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-session-checkpoint-tool-"));+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+});++async function initialize(server: MCPServer): Promise<void> {+ await server.handleRequest({+ jsonrpc: JSONRPC_VERSION,+ id: 1,+ method: "initialize",+ params: {+ protocolVersion: PROTOCOL_VERSION,+ capabilities: {},+ clientInfo: { name: "session-checkpoint-test", version: "0" },+ },+ });+ await server.handleRequest({+ jsonrpc: JSONRPC_VERSION,+ method: "notifications/initialized",+ });+}++async function call(args: Record<string, unknown>): Promise<{ result?: unknown; error?: unknown }> {+ const server = new MCPServer({ vault, configPath: null });+ await initialize(server);+ return (await server.handleRequest({+ jsonrpc: JSONRPC_VERSION,+ id: 9,+ method: "tools/call",+ params: { name: "brain_session_checkpoint", arguments: args },+ })) as { result?: unknown; error?: unknown };+}++function payload(response: { result?: unknown }): Record<string, unknown> {+ const result = response.result as { content: ReadonlyArray<{ type: string; text: string }> };+ return JSON.parse(result.content[0]!.text);+}++function inboxSignalCount(): number {+ const inbox = join(vault, "Brain", "inbox");+ if (!existsSync(inbox)) return 0;+ return readdirSync(inbox).filter((n) => n.startsWith("sig-") && n.endsWith(".md")).length;+}++const CHECKPOINT_A = {+ session_id: "s1",+ signals: [+ { topic: "prefer-tabs", signal: "positive", principle: "Use tabs for indentation." },+ { topic: "no-force-push", signal: "negative", principle: "Never force-push shared branches." },+ ],+ request: "Wire up the checkpoint tool",+ decisions: ["Reuse the C1 idempotency ledger"],+ learnings: ["Checkpoint = signals + summary in one round-trip"],+ next_steps: ["Ship it"],+};++describe("brain_session_checkpoint tool", () => {+ test("registered in the full tool table only", () => {+ expect(+ buildToolTable("full").find((tool) => tool.name === "brain_session_checkpoint"),+ ).toBeDefined();+ expect(+ buildToolTable("writer").find((tool) => tool.name === "brain_session_checkpoint"),+ ).toBeUndefined();+ });++ test("fresh session writes all items once; retry with same content is idempotent", async () => {+ const first = payload(await call(CHECKPOINT_A));+ expect(first["status"]).toBe("ok");+ expect(first["deduped"]).toBe(false);+ expect((first["signals"] as unknown[]).length).toBe(2);+ expect((first["summary"] as Record<string, unknown>)["id"]).toBeDefined();+ expect((first["partial"] as unknown[]).length).toBe(0);+ expect(inboxSignalCount()).toBe(2);++ // Retry the exact same checkpoint: deduped at the session key, no+ // duplicate signals appended and no second summary digest.+ const retry = payload(await call(CHECKPOINT_A));+ expect(retry["status"]).toBe("ok");+ expect(retry["deduped"]).toBe(true);+ expect(inboxSignalCount()).toBe(2);+ });++ test("a checkpoint with an item needing review returns status mixed with the partial list", async () => {+ const mixed = payload(+ await call({+ session_id: "s2",+ signals: [+ { topic: "good-one", signal: "positive", principle: "Keep it green." },+ // Invalid sign -> this item needs review, must not silently drop.+ { topic: "bad-one", signal: "sideways", principle: "This one is malformed." },+ ],+ }),+ );+ expect(mixed["status"]).toBe("mixed");+ const partial = mixed["partial"] as Array<Record<string, unknown>>;+ expect(partial.length).toBe(1);+ expect(partial[0]!["index"]).toBe(1);+ // The valid signal still landed.+ expect((mixed["signals"] as unknown[]).length).toBe(1);+ expect(inboxSignalCount()).toBe(1);+ });++ test("retry with the same session id but different content is a payload_mismatch error", async () => {+ const ok = payload(await call(CHECKPOINT_A));+ expect(ok["status"]).toBe("ok");++ const response = await call({+ ...CHECKPOINT_A,+ learnings: ["A materially different learning changes the checkpoint content"],+ });+ expect(response.error).toBeDefined();+ expect((response.error as { code: number }).code).toBe(-32602);+ expect((response.error as { message: string }).message).toContain("different payload");+ });++ test("missing session_id is an invalid-params error", async () => {+ const response = await call({ signals: [] });+ expect(response.error).toBeDefined();+ expect((response.error as { code: number }).code).toBe(-32602);+ });+});