Local-first 🧠 memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly 😴 dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.
TypeScript
feat: Memory Integrity Suite - canonical entities, conflict-free log shards, capture boundaries, fact extraction (#66)
Sol Aitkendee5ab721abe
diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.jsonindex 5646169e..1c89c52b 100644--- a/.claude-plugin/plugin.json+++ b/.claude-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "0.34.0",+ "version": "0.35.0","description": "Plugin-first second brain package for AI agents and humans.","author": {"name": "Open Second Brain contributors"diff --git a/.codex-plugin/plugin.json b/.codex-plugin/plugin.jsonindex c8064d4a..4acf6587 100644--- a/.codex-plugin/plugin.json+++ b/.codex-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "0.34.0",+ "version": "0.35.0","description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.","skills": "./skills","hooks": "./hooks/hooks.json",diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 19a18404..eccf4dfe 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -5,6 +5,82 @@ All notable changes to this project will be documented in this file.The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).+## [0.35.0] - 2026-06-02++Memory Integrity Suite: four changes that make the memory itself+trustworthy - named things get one canonical home, the daily log stops+sync-conflicting across devices, runtime noise stops entering memory, and+structured facts from user turns are captured in real time with canonical+anchors.++### Added++- **Canonical entity registry.** One canonical entity per+ `(category, normalized name)` lives at `Brain/entities/<category>/<id>.md`+ as plain Obsidian Markdown with frontmatter identity (`entity_id`,+ `category`, `name`, `aliases`, `status`, lifecycle stamps). The identity+ index rebuilds from the Markdown files on every read - nothing to persist,+ nothing to sync-conflict. `o2b brain entity set | get | list | relate |+ archive` drives the registry from the CLI (upsert resolves names AND+ aliases before creating anything and refuses duplicate claims; archive+ removes from active lookup, `--restore` returns); the read-only+ `brain_entity` MCP tool (`view: get | list`) serves lookups to agents.+ Relations reuse the typed relation vocabulary, so graph export and+ relation polarity pick entities up without new plumbing. Search expands+ query entities through the registry: a query naming an alias boosts+ documents naming the canonical entity and explains the hop with an+ `entity_canonical` reason - vaults without a registry rank+ bit-identically. Doctor lints duplicate identity claims+ (`duplicate-entity`) and dangling relations (`broken-entity-relation`).+- **Per-device Brain log shards.** `appendLogEvent` writes+ `Brain/log/<date>.<deviceId>.jsonl` + `.md`, so two Syncthing devices+ never touch the same file on the same day - the write-conflict class+ observed live on 2026-06-01 is gone. The device id is a stable+ per-install value in the DEVICE-LOCAL config, generated once on first+ use (`O2B_DEVICE_ID` overrides; identity-resolution failure falls back+ to the legacy pair - an append never fails on identity). `readLogDay`+ merges every shard of a day sorted by (timestamp, shardId, line) with+ per-shard JSONL-over-markdown preference, and `listLogDates` becomes the+ single date-discovery helper behind every reader (doctor, digest,+ query, dream, evidence, status, backlinks, most-applied, temporal index,+ MCP log resource). Legacy single-file days keep reading forever - no+ migration. Doctor flags leftover Syncthing conflict copies+ (`sync-conflict-log`).+- **Capture boundaries.** A `sessions:` block in `_brain.yaml` declares+ what may become memory: `ignore_patterns` (sessions that produce+ nothing), `stateless_patterns` (sessions that read but never write),+ and `ignore_message_patterns` (message text that never reaches+ extraction). Session patterns are anchored globs, message patterns are+ regexes; an invalid regex degrades to a doctor warning+ (`invalid-capture-pattern`), never an error. Machine-local config can+ ADD patterns (comma-separated `sessions_*` keys) but never remove vault+ policy. Both ingestion seams consult the boundary FIRST - live hooks+ (`captureSessionLifecycleEvent`) and batch import (`importSession`) -+ and suppression is counted in results and audit rows, never stored raw.+ An unconfigured vault captures bit-identically to before.+- **Regex fact extraction.** Seven precision-first pattern families+ (identity, preference, possession, location, url, email, confirmation)+ capture structured facts from USER turns in real time, without an LLM+ call. Every family requires an explicit first-person or confirmation+ frame; code blocks and quoted lines are stripped; bare assistant output+ is never auto-extracted (the HANDOFF carve-out's conservative core).+ Facts land as `source_type: extracted` signals with family-scoped dedup+ hashes - repeats and re-imports dedup identically - and a fact naming a+ registered canonical entity (or alias) carries the canonical id, the+ shared canonicalization kernel at work. Extraction runs strictly BEHIND+ the capture boundary; the pipeline order is pinned by tests at both+ seams.++### Process wins++- The capture pipeline order (classify/suppress -> extract -> route) is a+ tested contract, not a convention - suppressed input can never become+ evidence.+- The entity identity index is a pure projection of the Markdown files;+ "rebuildable" is satisfied by never persisting it.+- Every reader refactor shipped behind golden tests: legacy-only vaults+ read byte-identically through the new shard-merging path.+## [0.34.0] - 2026-06-02Token Diet: six changes that cut what an agent pays in context before it does@@ -3988,6 +4064,7 @@ plugin config (vault field)`, and exits with a clear- Sandbox vault and plugin manifest fixtures for tests.- GitHub release workflow for tag-based and manually dispatched releases.+[0.35.0]: https://github.com/itechmeat/open-second-brain/compare/v0.34.0...v0.35.0[0.34.0]: https://github.com/itechmeat/open-second-brain/compare/v0.33.0...v0.34.0[0.33.0]: https://github.com/itechmeat/open-second-brain/compare/v0.32.1...v0.33.0[0.32.1]: https://github.com/itechmeat/open-second-brain/compare/v0.32.0...v0.32.1diff --git a/README.md b/README.mdindex a9729512..006792b4 100644--- a/README.md+++ b/README.md@@ -114,7 +114,7 @@ The capabilities you actually feel day to day:These are the headline capabilities. The full surface also includes: importing Claude Code memory directories, daily logging-discipline cron with a complexity-to-thinking warning, cross-project pointers for shared vaults, transient pinned session context through `brain_pinned_context` / `Brain/pinned.md`, source-agent query and comparison (`brain_agent_query`, `brain_agent_diff`, `o2b brain agent-query`, `o2b brain agent-diff`), the codegraph partner skill, vault hygiene lints, per-MOC coverage audit, concept synthesis, an operator dashboard, configurable Markdown-link presentation output, MCP output-contract checks for core agent-facing tools, private-region stripping with `<private>...</private>`, the v0.12.0 Brain Integrity Suite (content-hash drift detection, durable workrun checkpoints, destructive-from-confirmed retire gate, and the `brain_review_candidates` MCP tool that previews what the next dream pass would do without writing anything), the v0.13.0 Hybrid Search and Recall Quality suite (explainable recall, MMR diversity, link-graph traversal, entity-boosted retrieval, header-anchored chunking), the v0.14.0 Semantic Brain Health and Self-Maintenance suite (cross-preference contradiction detection, concept-gap and stale-claim surfacing, per-preference edit-history, a clean/watch/investigate reconciliation verdict via `brain_health`, and dependency-ordered `doctor --remediate`), the v0.17.0 Brain Lifecycle Review Suite (`intent-review`, `retention`, `monthly`, lifecycle schemas, and discipline complexity ratio), and the v0.18.0 MCP context economy (a per-tool preview budget that returns a bounded inline preview and parks the full payload in a vault-local artifact the agent fetches on demand with `brain_artifact_get`, plus a `recall_hint` one-liner on search results), and the v0.19.0 typed graph semantics (typed frontmatter relationships - `related` / `extends` / `contradicts` / `superseded_by` - that surface inline on search results, a `visibility:` scoping axis for `brain_search`, and a `brain_mcp_landscape` view of the Model Context Protocol servers configured across the vault), and the v0.20.0 Recall and Ranking Quality suite (a configurable Weibull recency-decay curve, structural query-intent classification that re-weights ranking by query shape, language-agnostic synonym expansion via local co-occurrence, per-memory and total recall character budgets for `brain_context_pack`, a persistent query cache that self-invalidates on a corpus-generation change, and a `brain_pre_compress_pack` tool that returns a budgeted top-preferences addendum for a host runtime to inject before context compression), and the v0.21.0 Brain Lifecycle Suite (a per-preference mutation audit trail keyed by preference id - create / promote / update / retire / merge with revision + content-hash before/after, via `o2b brain audit` and `brain_audit`; a multi-phase `dream` pipeline that names its close / reconcile / synthesize / heal / log seams and reports a structured per-phase summary without rewriting the proven internals; deterministic reconcile-phase domain classification that buckets contradictions into claims / entity / decisions / source-freshness, auto-resolves only the decisive source-freshness case, and surfaces the rest as open questions instead of forcing a merge; a budgeted `o2b brain morning-brief` / `brain_morning_brief` session-start summary of top preferences, recent open questions, and recent notes; language-agnostic ISO-8601 temporal extraction that fills preference validity windows from signal text; and opt-in heal-phase vault enrichment that links exact title mentions across user pages), and the v0.22.0 Vault portability + session economy suite (a deterministic lossless session codec - `expand(compress(x)) === x` for all input, opt-in on the signal store and exposed as `o2b brain codec`; an `o2b brain sources` / `brain_sources` read-only dashboard of signals by agent and source type; vault-map `{{role}}` tokens resolved via an optional `Brain/_vault-map.yaml` and wired into scan-inline read paths and graph import; named multi-vault profiles with `o2b vault profile` + a `brain_switch_vault` tool, activated by a pointer in config; and `o2b brain graph-export` / `graph-import` for moving a vault's knowledge graph between layouts with skip/overwrite/merge conflict modes). Browse [`docs/cli-reference.md`](docs/cli-reference.md) for every verb and [`docs/how-it-works.md`](docs/how-it-works.md) for the mental model.-The v0.26.0 surface adds audited schema mutation/admin tools, real-time lifecycle capture hooks for prompt markers and `brain_feedback` tool calls, CJK-aware FTS recall, and `brain_watchdog` / `o2b brain watchdog` recovery probes. The v0.27.0 surface adds recall-control and trust tools: structured search query documents, session focus, MCP `brain_recall_gate`, polarity-aware context lanes, and opt-in evidence packs for CLI/MCP search. The v0.28.0 surface adds a deterministic prompt-injection guard for automatically surfaced Brain context, agent-blind `$secret:NAME` references, and preview foundations for source-scoped forget, privacy-scanned knowledge packs, and oversized payload externalization. The v0.29.0 surface adds opt-in context receipts and recall telemetry, read-only context budget presets, pre-compaction extraction, and a continuity-backed session recall DAG. The v0.33.0 surface adds the Recall Trust Suite: relation-aware recall polarity for typed memory edges, an explicit retrieval feedback loop with bounded learned weights (`o2b search feedback` / `weights`, MCP `brain_recall_feedback`), time-aware recall (`--since` / `--until` with natural-language ranges), verified multi-record recall (per-token union records, IDF-weighted coverage, rare-term abstention), and a search-completeness verdict with a false-absence guard. The v0.34.0 surface adds the Token Diet: a budgeted SessionStart injection of `Brain/active.md` (default 8,000 chars, `active.inject_budget_chars`) with the post-compaction path moved to the supported SessionStart `compact` matcher, a once-per-session post-write reminder cadence, frontmatter escape-amplification fixed at the parser with a one-shot `brain upgrade --apply` repair for already-corrupted preference files, three consolidated read tools (`brain_brief`, `brain_analytics`, `schema_inspect`) with 18 hidden deprecated aliases, registry description caps with a contract guard, preview budgets by default, and a `scripts/measure-token-surface.ts` report.+The v0.26.0 surface adds audited schema mutation/admin tools, real-time lifecycle capture hooks for prompt markers and `brain_feedback` tool calls, CJK-aware FTS recall, and `brain_watchdog` / `o2b brain watchdog` recovery probes. The v0.27.0 surface adds recall-control and trust tools: structured search query documents, session focus, MCP `brain_recall_gate`, polarity-aware context lanes, and opt-in evidence packs for CLI/MCP search. The v0.28.0 surface adds a deterministic prompt-injection guard for automatically surfaced Brain context, agent-blind `$secret:NAME` references, and preview foundations for source-scoped forget, privacy-scanned knowledge packs, and oversized payload externalization. The v0.29.0 surface adds opt-in context receipts and recall telemetry, read-only context budget presets, pre-compaction extraction, and a continuity-backed session recall DAG. The v0.33.0 surface adds the Recall Trust Suite: relation-aware recall polarity for typed memory edges, an explicit retrieval feedback loop with bounded learned weights (`o2b search feedback` / `weights`, MCP `brain_recall_feedback`), time-aware recall (`--since` / `--until` with natural-language ranges), verified multi-record recall (per-token union records, IDF-weighted coverage, rare-term abstention), and a search-completeness verdict with a false-absence guard. The v0.34.0 surface adds the Token Diet: a budgeted SessionStart injection of `Brain/active.md` (default 8,000 chars, `active.inject_budget_chars`) with the post-compaction path moved to the supported SessionStart `compact` matcher, a once-per-session post-write reminder cadence, frontmatter escape-amplification fixed at the parser with a one-shot `brain upgrade --apply` repair for already-corrupted preference files, three consolidated read tools (`brain_brief`, `brain_analytics`, `schema_inspect`) with 18 hidden deprecated aliases, registry description caps with a contract guard, preview budgets by default, and a `scripts/measure-token-surface.ts` report. The v0.35.0 surface adds the Memory Integrity Suite: a canonical entity registry under `Brain/entities/` (one Markdown entity per `(category, name)` with alias resolution, archive/restore, typed relations, `o2b brain entity` CLI verbs, a read-only `brain_entity` MCP tool, doctor duplicate/relation lints, and alias-aware search boost with `entity_canonical` explanations), per-device Brain log shards (`Brain/log/<date>.<deviceId>.jsonl` + `.md` keyed by a device-local id, merged reads through `readLogDay`/`listLogDates`, no more Syncthing write conflicts, legacy days untouched), capture boundaries (a `sessions:` block with ignored/stateless session globs and message suppression regexes, applied before any extraction at both live-hook and import seams, counted but never stored raw; keep suppression regexes simple - they run on every message, and a catastrophic-backtracking pattern slows capture), and deterministic regex fact extraction (seven precision-first families over user turns, `source_type: extracted` signals with family-scoped dedup and canonical entity anchors).## CLI@@ -130,6 +130,8 @@ o2b brain monthly --month 2026-05 --json # Month-level Brain synthesiso2b brain audit pref-foo --json # Full mutation trail for one preferenceo2b brain morning-brief --json # Session-start summary (prefs, open questions, notes)o2b brain sources --json # Per-source signal dashboard+o2b brain entity set people "Ada Lovelace" --alias "the operator" # Canonical entity registry+o2b brain entity get "the operator" --json # Alias resolves to the canonical recordo2b brain schema lint --json # Runtime schema vocabulary + usage/admin findingso2b brain watchdog --json # Brain probes + safe recovery recommendationso2b brain graph-export --out graph.json # Export the vault knowledge graphdiff --git a/docs/brainstorm/memory-integrity/cli-output/claude.md b/docs/brainstorm/memory-integrity/cli-output/claude.mdnew file mode 100644index 00000000..ce4944da--- /dev/null+++ b/docs/brainstorm/memory-integrity/cli-output/claude.md@@ -0,0 +1,37 @@+### Variant 1: Four independent vertical slices+- **Approach**: Land each feature as a self-contained module against its own existing seam, sharing only primitives that already exist (config chain, `brainDirs`, doctor, `writeSignal`). Sharding edits `log.ts`/`log-jsonl.ts` and the directory-scanning readers; capture boundaries wrap `session-lifecycle.ts` + `import.ts`; regex extraction is a new fail-soft hook emitting `brain_feedback` candidates; the entity registry is a standalone `Brain/entities/` subsystem with its own index and doctor checks. No new cross-feature abstraction is introduced.+- **Trade-offs**:+ - Pro: maximally parallelizable; four small PRs-worth of work bundled into one CHANGELOG, each independently testable under TDD.+ - Pro: smallest blast radius on the hot capture path; fail-soft is local to each slice.+ - Pro: lowest risk of speculative generality — matches the tasks' explicit "no over-engineering" warnings.+ - Con: regex extraction (task 3) and message suppression (task 4) independently re-walk the same `SessionTurn` stream and duplicate turn-iteration + pattern-compile + malformed-pattern handling.+ - Con: name normalization/alias resolution gets implemented twice — once in the entity registry, once (informally) wherever extracted identity/possession facts are routed — so extracted facts never cleanly resolve to canonical entities.+ - Con: four separate diagnostics surfaces; no single place to reason about "what entered/was kept out of memory."+- **Complexity**: medium+- **Risk**: low++### Variant 2: Capture-boundary pipeline + canonicalization kernel+- **Approach**: Pair the two regex-over-turns features into one ordered, deterministic capture pipeline at the shared ingestion seam (`session-lifecycle.ts` live hook and `import.ts` batch): source-classify/suppress (task 4) → extract facts (task 3) → route to signals, with suppression and stateless mode gating extraction so suppressed/read-only turns never produce evidence. Build the entity registry (task 1) as its own subsystem but factor out a small canonicalization primitive (normalized `(category, name)` + alias index) that the extraction router reuses, so identity/possession facts can resolve to canonical entities. Keep device sharding (task 2) fully independent as a write-layer/file-strategy change, since it shares nothing semantic with the other three.+- **Trade-offs**:+ - Pro: eliminates the real duplication (one turn walk, one pattern-compile/error-handling path) and makes the suppression→extraction ordering explicit rather than emergent.+ - Pro: extracted facts gain a precise anchor via the shared canonicalization kernel — the entity task's core value proposition, delivered without coupling the two subsystems.+ - Pro: single coherent capture-diagnostics surface (counters, active patterns, foreground/side-channel, unknown-source) for `brain sources`/doctor.+ - Pro: sharding stays decoupled and shippable on its own, respecting its "decided design."+ - Con: the pipeline couples tasks 3+4 on the hot path; staged fail-soft must be carefully preserved (one stage's crash must not block the others or the runtime).+ - Con: more upfront design than Variant 1; the 3+4 pairing must land together rather than as two trivially independent slices.+- **Complexity**: medium+- **Risk**: medium++### Variant 3: Full integrity substrate+- **Approach**: Extract a foundational layer first — a generalized shardable append-only Brain store (sharding the log becomes one instance, and signals/entities could shard too), a reusable rebuildable-index-as-cache primitive shared by the log-shard reader and the entity identity index, and a single turn-scan/filter primitive — then implement all four features as thin consumers on top. Everything routes through common substrate contracts for write/merge, index rebuild, and turn processing.+- **Trade-offs**:+ - Pro: maximal reuse and a uniform index-is-cache/append-only contract across the whole Brain; future features inherit the substrate.+ - Pro: one consistent merged-read and rebuild story everywhere.+ - Con: directly contradicts the tasks' explicit anti-over-engineering guidance (sharding warns against a daemon; entity task insists the index is a cache, not a new storage model).+ - Con: speculative generality — entity writes are rare and operator-governed, so they don't need device sharding; generalizing the store buys little and risks regressing the many directory-scanning log readers (`readAllLogRecords`, temporal index).+ - Con: largest blast radius, slowest to ship, hardest to keep deterministic and backward-compatible across all existing readers in one release.+- **Complexity**: large+- **Risk**: high++### Recommended: Variant 2+**Rationale**: It captures the only genuine synergies — the two regex-over-turns features share one turn seam and a suppress-then-extract ordering, and a small canonicalization kernel lets extracted facts resolve to canonical entities — while leaving the orthogonal sharding work decoupled and independently shippable, exactly as its "decided design" expects. This avoids both Variant 1's duplicated turn-walking and normalization logic and Variant 3's over-engineered substrate that the tasks explicitly warn against, and it keeps each piece testable, fail-soft, and backward-compatible within a single bundled release.diff --git a/docs/brainstorm/memory-integrity/cli-output/prompt.md b/docs/brainstorm/memory-integrity/cli-output/prompt.mdnew file mode 100644index 00000000..3d00a9de--- /dev/null+++ b/docs/brainstorm/memory-integrity/cli-output/prompt.md@@ -0,0 +1,195 @@+You are brainstorming architectural variants for the following multi-task scope. Do not write code. Do not write a final design. Only produce variants and a recommendation.++# Task++One release ("Memory integrity suite") ships four related features for Open Second Brain - an Obsidian-native, filesystem-first, agent-owned second brain written in TypeScript on Bun. The four kanban tasks below are the scope. Variants should address the OVERALL architecture of the suite: how the four features compose, where each one's seam is, and what shared primitives they should reuse.++## Task t_95d7a31b (priority 4): [upstream:Sibyl-Memory] Canonical entity memory with single-source-of-truth registry++**Source**: https://github.com/Sibyl-Labs/Sibyl-Memory+**Upstream files studied**: `sibyl-memory-client/src/sibyl_memory_client/schema.sql`, `sibyl-memory-client/README.md`++## What+Sibyl-Memory separates durable entities from journal events and reference documents. Its WARM tier stores one canonical entity per `(tenant_id, category, name)` and enforces that uniqueness at the schema level. It also has typed `entity_relations` and an archive tier for frozen entities. The important idea is not the SQLite implementation itself; it is the single-source-of-truth contract for remembered people, projects, systems, accounts, and other named things.++## Why useful for OSB+OSB currently has strong preference lifecycle management, typed page relationships, graph export/import, entity-boosted search, and page-level deduplication. It does not have a first-class canonical entity memory where agents can say: "this project/person/system is represented here, this is the current structured state, and duplicates should resolve to this identity." A canonical entity layer would improve Second Brain behavior by making project/person/tool facts stable across sessions, reducing drift between notes, and giving recall a precise anchor that is not just a search-time extracted entity string.++## Current OSB status+- **Verdict**: present_weaker+- **Local evidence**: CodeGraph shows OSB entity handling mainly in search (`src/core/search/entities.ts`, `chunk_entities`, entity boost). Page dedup, typed links, schema packs, and graph export exist, but they do not provide an operator-governed entity registry with uniqueness and archive semantics.+- **Related existing work, not a duplicate**: Page-level deduplication and typed graph semantics work on vault pages/relations. This task is about a canonical memory surface for named entities and their current structured state.++## Proposal+Add a filesystem-first OSB entity registry under `Brain/entities/` with deterministic identity rules, typed relations, archive support, and search/graph integration. The registry should stay Obsidian-native: each entity is still Markdown with frontmatter, but OSB owns an identity index that can detect or prevent duplicate canonical entities.++## Suggested design+- Introduce `Brain/entities/<category>/<slug>.md` or a similarly explicit layout for canonical entities.+- Define frontmatter fields such as `entity_id`, `category`, `name`, `aliases`, `status`, `source_agent`, `confidence`, `updated_at`, and `archived_at`.+- Maintain a rebuildable identity index mapping `(category, normalized_name)` and aliases to the canonical file.+- Add read/write CLI verbs for `entity set`, `entity get`, `entity list`, `entity relate`, and `entity archive`; expose only the safe subset through MCP.+- Connect entity IDs to typed relationships and search ranking so search results can explain when a hit came through a canonical entity rather than only raw text/entity extraction.+- Add conflict reporting when two files claim the same canonical `(category, name)` or alias.++## Acceptance criteria+- [ ] OSB can create or update a canonical entity without creating a duplicate for the same normalized `(category, name)`.+- [ ] Entity files remain plain Markdown and are readable/editable in Obsidian.+- [ ] Aliases resolve to the canonical entity during lookup and search.+- [ ] Entity relations are typed and appear in graph export/search explanation output.+- [ ] Archiving an entity removes it from active lookup while preserving audit/history.+- [ ] Doctor or schema lint reports duplicate entity claims and broken entity relations.+- [ ] Tests cover canonicalization, alias resolution, duplicate detection, relation validation, archive/restore semantics, and search integration.++## Out of scope+- Replacing user-authored vault notes with entity files.+- Moving OSB memory storage into SQLite.+- Importing Sibyl's tenant/account/tier model.++## Task t_6d52641f (priority 3): Brain log: per-device shards to eliminate Syncthing write-conflicts++## Problem+Brain/log/<date>.jsonl (+ .md) is a single shared per-day file written by every runtime. On one device concurrent writers are serialized by proper-lockfile (no corruption, no conflict - one device = one file). But the vault is Syncthing-synced across devices, so when two DEVICES write the same <date>.jsonl the same day, Syncthing cannot merge and emits a .sync-conflict-* copy (observed 2026-06-01: claude-mac-agent + claude-vps-agent; reconciled manually). No in-process lock/daemon fixes this - the lock cannot span the sync boundary.++Note: agents already do NOT write the log directly - they go through the o2b writer (o2b-hook session-capture script + brain_note/brain_feedback/brain_apply_evidence MCP tools, proper-lockfile). So the fix is the FILE STRATEGY, not adding a serializer/daemon (that would be over-engineering; the lockfile already serializes same-device writers).++## Design (decided)+Shard the log by DEVICE, mirroring the conflict-free pattern inbox/ already uses (one file per signal):+- Writer appends to Brain/log/<date>.<deviceId>.jsonl (+ .md), where deviceId is a stable per-install id stored in the PER-DEVICE config (~/.config/open-second-brain/, NOT in the synced vault - else all devices share one id and the sharding is defeated). Generate the id on first use if absent.+- One device -> only its own shard file -> Syncthing never conflicts. Multiple agents/Claude-Code instances on one device share the device shard via the existing serialized writer (lock), no races.+- Readers (digest, timeline, daily/morning/weekly briefs, anything reading <date>.jsonl/.md) glob <date>*.jsonl across shards and sort by ts. The human-facing <date>.md becomes a derived merged view regenerated on read (or also sharded).+- Migration: existing single-file Brain/log/<date>.jsonl/.md keep working (treat as an implicit shard / fold into the merged read); a one-time reconcile of any leftover .sync-conflict-* copies (union+dedup by ts+content) - see the manual merge done for 2026-06-01.++## Scope / acceptance+- Single writer path appends to a per-device shard; deviceId resolved from per-device config (not vault).+- All log readers merge shards (sorted by ts) and produce the same results as before.+- Concurrent writers across devices produce NO Syncthing conflict; same-device concurrency stays serialized.+- TDD; ship via feature-release-playbook.++## Out of scope+- A logging daemon (the existing lockfile writer is sufficient).+- Excluding the log from Syncthing (loses cross-device visibility).++## Task t_d0782ab2 (priority 3): [upstream:yantrikdb-hermes-plugin] Regex-based fact extraction from conversation turns++**Source**: https://github.com/yantrikos/yantrikdb-hermes-plugin/releases/tag/v0.5.0+**Repo**: yantrikos/yantrikdb-hermes-plugin (27★)+**Released**: v0.5.0 (2026-05-31T07:10:31Z)++## What+YantrikDB v0.5 introduced 7 high-precision regex patterns that automatically extract facts from user conversation turns: preference, possession, identity, location, URL, email, and confirmation patterns. Only USER turns and user-confirmed assistant assertions are extracted (HANDOFF carve-out); bare LLM output is never auto-extracted. A recall filter and yantrikdb_extraction_stats tool allow tuning.++## Why useful for OSB+OSB has brain_pre_compact_extract for structured Decision/Commitment/Outcome/Rule/Open question extraction from session turns, but it is LLM-driven and runs at compaction time. Lightweight regex-based fact extraction would capture structured identity/preference/possession facts in real-time without an LLM call, complementing the heavier pre-compact extraction. The HANDOFF carve-out (only user turns + confirmed assistant assertions) is a sound guard against hallucinated memory.++## Status in OSB+- **Verdict**: not_in_osb_useful+- **Codegraph hints**: src/core/brain/pre-compact-extract.ts:29 (ExtractedLine), src/core/brain/pre-compact-extract.ts:110 (extractLines), hooks/lib/transcript.ts:30 (TurnSignal). Pre-compact extraction is LLM-based and runs at session boundary. No regex-based real-time fact extraction exists.++## Notes+Could be implemented as a lightweight hook (PostToolUse or UserPromptSubmit) that runs regex patterns against user turns and produces candidate brain_feedback entries. The extraction_stats tuning tool concept could map to a brain extraction diagnostics command.++## Task t_0532ed5a (priority 2): [upstream:hermes-lcm] Source-aware session and message noise boundaries++**Source**: https://github.com/stephenschoettler/hermes-lcm+**Upstream files studied**: `README.md`, `config.py`, `engine.py`, `session_patterns.py`, `message_patterns.py`, `command.py`, `tools.py`++## What+Hermes-LCM has explicit session and message boundary controls: ignored session globs are excluded from storage, stateless session globs can read carried-over state without writing new rows, and message-level regex patterns suppress noisy content before it reaches the store. The engine tracks foreground versus side-channel sessions, exposes filter state in `lcm_status`, and keeps source lineage (`source`, `unknown`, legacy blank-source normalization) visible in search and diagnostics.++## Why useful for OSB+OSB now captures lifecycle events through hooks and imports sessions from multiple agents. That is powerful, but Second Brain quality depends on what does *not* enter memory: cron chatter, test probes, heartbeat messages, generated progress pings, temporary side-channel sessions, or tool noise that would pollute search and synthesis. OSB has vault ignore paths and private-region stripping, but it does not appear to have a first-class capture boundary for agent sessions and individual messages. Adding one would make long-running multi-agent memory cleaner and easier to trust.++## Current OSB status+- **Verdict**: not_in_osb_useful+- **Local evidence**: OSB has `brain session-hook`, source-agent query/diff, private-region stripping, vault ignore paths, and redaction for secret-shaped content. Existing tasks include lifecycle hooks (`t_9eaebcad`) and privacy-aware capture (`t_c12e0e9c`). CodeGraph/readme/CLI context did not show session glob filters, stateless capture mode, message regex suppression, or foreground/side-channel diagnostics for session ingestion.+- **Related existing work, not a duplicate**: Vault ignore paths decide which vault files walkers read. This task decides which runtime sessions/messages become Brain evidence at all.++## Proposal+Add OSB capture-boundary configuration for session imports and live hooks. Operators should be able to declare ignored sessions, stateless/read-only sessions, and message suppression patterns in `_brain.yaml` or machine-local config. Every skipped item should be counted and diagnosable, but skipped content should not become searchable memory or preference evidence.++## Suggested design+- Add config keys such as `sessions.ignore_patterns`, `sessions.stateless_patterns`, and `sessions.ignore_message_patterns`, with clear source precedence between machine-local and vault-portable config.+- Apply the filters in `brain session-hook` and `brain import-session` before signal extraction or narrative logging.+- Store skip counters and non-sensitive reasons in `Brain/log/<date>.jsonl` or doctor diagnostics; avoid writing the skipped message text itself.+- Add a `brain sources` / `brain doctor` section showing active patterns, match counts, unknown-source counts, and recently suppressed categories.+- Treat stateless sessions as allowed to read `brain_context` / `brain_morning_brief` but not allowed to write signals, notes, extracted decisions, or apply-evidence rows unless explicitly overridden.++## Acceptance criteria+- [ ] Session import and live hook capture both honor ignored session patterns and message-level suppression patterns.+- [ ] Stateless sessions can receive context but do not mutate Brain state by default.+- [ ] Suppression is auditable through counters, pattern names, and source/session metadata without persisting suppressed raw content.+- [ ] Existing imports behave unchanged when no filters are configured.+- [ ] Tests cover ignored sessions, stateless sessions, message regex suppression, source attribution, and malformed pattern handling.++## Out of scope+- Building a general moderation or classification system.+- Deleting already-imported historical Brain records automatically.+- Making regex filtering block the host runtime; invalid or expensive patterns should degrade safely.+++# Project context++Open Second Brain: TypeScript, Bun runtime, bun:test, oxlint/oxfmt. MCP server + o2b CLI + lifecycle hooks. Core principle: the Brain core is deterministic - no LLM calls inside core state transitions. All durable state is Markdown with YAML-like frontmatter inside an Obsidian vault; the vault is Syncthing-synced across devices.++Recent commits:+5066e71 feat: Token Diet - budgeted injection, reminder cadence, consolidated MCP surface (#65)+c494fd8 feat(search): Recall Trust Suite - relation polarity, learned weights, time-aware and verified recall (#64)+23ff4fb refactor(hermes): single intentional self-bootstrap for the plugin entrypoint (#63)+0952dfc feat: become a native Hermes memory provider (#62)+6fbab0b feat: hands-off post-upgrade migration (v0.31.2) (#61)+496dd2d fix: make plugin updates self-healing (v0.31.1) (#60)+09c0592 chore(release): v0.31.0 (#59)+b81335c Feat/procedural attention suite (#58)+1f3a218 Feat/self learning skill proposals (#57)+0162d13 feat(brain): add context continuity and receipts suite (#56)++Related files (verified seams):+- src/core/brain/log.ts:237 appendLogEvent - the ONLY write point for Brain/log/<date>.jsonl + .md; per-day directory lock via proper-lockfile; JSONL written first (machine-authoritative), markdown second (human view).+- src/core/brain/log-jsonl.ts:47 readLogDay - the single reader entry point (JSONL preferred, markdown fallback). Used by morning-brief, digest, timeline.+- src/core/brain/doctor.ts:905 readAllLogRecords - reads the Brain/log/ DIRECTORY listing directly (not via readLogDay); temporal/build-index.ts collectLogEvents similar. Directory-scanning readers must stay correct under any new file layout.+- src/core/config.ts - per-device config chain: OPEN_SECOND_BRAIN_CONFIG env -> XDG_CONFIG_HOME -> ~/.config/open-second-brain/config.yaml. This is the per-device (NOT synced) store.+- src/core/brain/session-lifecycle.ts captureSessionLifecycleEvent - live hook capture seam (hooks/session-capture.ts feeds it; fail-soft, never blocks runtime).+- src/core/brain/sessions/import.ts:127 importSession - batch session import seam; adapters claude/codex/hermes normalise to SessionTurn {turnId, timestamp, role, text, toolCalls}.+- src/core/brain/pre-compact-extract.ts - precedent for regex-label extraction (LABELS table, sanitize, dedupe_key, continuity records).+- src/core/brain/signal.ts writeSignal - the one way signals are born; supports source_type, dedup_hash, session_ref.+- src/core/brain/policy.ts - _brain.yaml config parsing (vault.ignore_paths, notes.read_paths, active.inject_budget_chars precedents); 1789 lines, strict per-key validation with warnings.+- src/core/search/entities.ts extractEntities - deterministic, language-agnostic entity extraction used as a ranking boost (entityMatchByChunk in src/core/search/ranker.ts).+- src/core/graph/frontmatter-relations.ts extractFrontmatterRelations - typed relation edges from frontmatter fields.+- src/core/brain/paths.ts brainDirs - all Brain/ subdirectories composed here; ensureInsideVault guards.+- src/core/brain/doctor.ts - lint surface for new integrity checks.+- src/mcp/brain-tools.ts + src/mcp/tools.ts - MCP tool definitions (300-char description cap, preview budgets by default, registry guard contract tests).++Conventions:+- TDD; every feature lands with failing tests first. 3,198 tests green on main.+- Determinism: given the same vault bytes + injected clock, every core function returns identical output. No Date.now() in core paths - callers inject.+- Append-only logs; atomic writes (atomicWriteFileSync); idempotent operations with dedup hashes.+- Conventional commits; one PR = one CHANGELOG version bundling the suite.+- Contract tests pin public behavior (tool counts, description caps, hook allowlists).+- Everything in the vault must remain readable/editable in Obsidian (plain Markdown + frontmatter).+- Per-feature config lives in Brain/_brain.yaml (vault-portable) or ~/.config/open-second-brain/ (device-local). The split matters: device identity MUST NOT sync; policy SHOULD sync.++Constraints:+- No new external dependencies unless unavoidable (proper-lockfile, sharp etc. already present).+- No LLM calls inside core; extraction must be deterministic regex/structural.+- Syncthing safety: two devices must never write the same file on the same day.+- Existing readers (digest, timeline, morning brief, doctor, temporal index) must keep producing the same results over merged data after the log-shard change.+- Backward compatibility: existing single-file logs keep working without migration; existing MCP tool names stay callable.+- The entity registry must stay rebuildable from the Markdown files alone (the index is a cache, not a source of truth).+- Fail-soft hooks: a hook crash must never block the host runtime.++# Required output format++Produce exactly 3 distinct architectural variants. For each variant:++### Variant N: <short name>+- **Approach**: 2-3 sentences describing the variant.+- **Trade-offs**: bullet list of pros and cons.+- **Complexity**: small | medium | large+- **Risk**: low | medium | high++After the three variants, add exactly one recommendation:++### Recommended: Variant N+**Rationale**: 2-3 sentences explaining why this variant over the others, considering the project context and constraints above.++Output nothing outside of these sections.diff --git a/docs/brainstorm/memory-integrity/design.md b/docs/brainstorm/memory-integrity/design.mdnew file mode 100644index 00000000..10e8e055--- /dev/null+++ b/docs/brainstorm/memory-integrity/design.md@@ -0,0 +1,83 @@+# Memory Integrity Suite - canonical entities, conflict-free log, capture hygiene++**Status:** draft+**Author:** claude-dev-agent (via feature-release-playbook)+**Audience:** implementation+**Kanban:** epic t_e7eb3224; children t_95d7a31b (entities), t_6d52641f (log shards), t_d0782ab2 (fact extraction), t_0532ed5a (capture boundaries)++## Problem statement++Second Brain trust depends on three properties the vault does not yet guarantee. Named things (people, projects, systems) have no canonical home - facts about them drift across notes and search-time entity strings. The daily Brain log is one shared file per day, so two Syncthing devices writing the same day produce `.sync-conflict-*` copies (observed live 2026-06-01). And everything a runtime emits can become memory: cron chatter, heartbeats, and side-channel sessions enter the same capture path as real operator signals, with no boundary and no real-time structured fact capture for the turns that do matter.++## Scope++- **Canonical entity registry** (t_95d7a31b): `Brain/entities/<category>/<slug>.md` Markdown entities with frontmatter identity (`entity_id`, `category`, `name`, `aliases`, `status`, lifecycle stamps), a rebuildable in-memory identity index over `(category, normalized name)` + aliases, duplicate refusal at write time, typed relations through the existing frontmatter-relations vocabulary, archive/restore semantics, CLI verbs (`entity set/get/list/relate/archive`), a read-only MCP surface, doctor lints (duplicate claims, broken relations), and alias-aware search boost.+- **Per-device log shards** (t_6d52641f): `appendLogEvent` writes `Brain/log/<date>.<deviceId>.jsonl` + `.md`, where `deviceId` is a stable per-install id from the device-local config (`~/.config/open-second-brain/config.yaml`, never the vault). `readLogDay` merges legacy single files plus all shards sorted by timestamp; a new `listLogDates` helper replaces every ad-hoc directory scan (doctor x2, digest x2, digest-agent-summary, temporal index - the last already reads through `readLogDay` but discovers dates from raw filenames).+- **Capture boundaries** (t_0532ed5a): `sessions.ignore_patterns`, `sessions.stateless_patterns`, `sessions.ignore_message_patterns` in `_brain.yaml`; ignored sessions produce nothing, stateless sessions read but never write, suppressed messages never reach marker/fact extraction. Applied at both seams - live (`captureSessionLifecycleEvent`) and batch (`importSession`). Suppression is counted, never stored raw.+- **Regex fact extraction** (t_d0782ab2): deterministic high-precision patterns (preference, possession, identity, location, URL, email, confirmation) over USER turns only, emitting candidate signals through `writeSignal` with a new `source_type: extracted`, dedup-hashed, gated BEHIND the capture boundary (suppressed/ignored/stateless input is never extracted). Extracted identity/possession facts that name a registered canonical entity (or alias) get the canonical `entity_id` stamped into the signal note - the canonicalization kernel shared with the registry.++## Out of scope++- A logging daemon or any serializer beyond the existing per-directory lockfile (the lock already serializes same-device writers).+- Excluding `Brain/log/` from Syncthing.+- Moving entity or log storage into SQLite; importing Sibyl's tenant/tier model.+- Replacing user-authored notes with entity files; auto-creating entities from extracted facts (extraction only ANCHORS to existing entities, creation stays operator/agent-explicit).+- Extracting facts from assistant turns (the HANDOFF carve-out's conservative core: user turns only in v1).+- Automatic deletion of already-imported historical records.+- A general moderation/classification system; LLM calls anywhere in these paths.++## Chosen approach++Variant 2 of the consultant round (see `variants.md`): a capture-boundary pipeline plus a canonicalization kernel. The two regex-over-turns features compose into one ordered, deterministic pipeline at both ingestion seams - classify/suppress first, extract second, route third - so suppressed input can never become evidence and the turn walk/pattern-compile/malformed-pattern handling exists once. The entity registry is its own subsystem, but its canonicalization primitive (normalize, identity key, alias resolution) is exported and reused by the fact router. Log sharding stays a fully independent write-layer change, as its decided design expects.++## Design decisions++- **Identity index is built on read, never persisted.** Entity counts are small (tens to low hundreds); walking `Brain/entities/` per operation is cheap, deterministic, and avoids a cache file that could itself sync-conflict. The acceptance criterion "rebuildable" is satisfied trivially: there is nothing to rebuild.+- **Entity name normalization = NFC, lowercase, whitespace-collapse** - the same shape `extractEntities` already produces, so search alias expansion and fact anchoring compare like with like.+- **Duplicate policy: refuse at write, lint at rest.** `entity set` refuses to create a second file claiming an existing `(category, normalized name)` or alias (update goes to the canonical file); doctor reports duplicates that arrived by hand-editing or sync.+- **Entity relations reuse the frontmatter-relations vocabulary** (`relates_to`, `part_of`, ... exactly the fields `extractFrontmatterRelations` already maps), so graph export and relation polarity pick entities up without new edge plumbing.+- **MCP surface is read-only and single-tool**: `brain_entity` with `view: get | list` (registry-guard caps respected). Writes stay on the CLI where the operator drives them; this honours the task's "safe subset" requirement and the Token Diet tool-count discipline.+- **Device id lives in device-local config** (`device_id:` in `~/.config/open-second-brain/config.yaml`), generated once (8 hex chars from crypto randomness) on first write-path use, validated as `[a-z0-9-]{1,32}`. Putting it in the vault would defeat sharding (all devices would share one id).+- **Shard merge order: timestamp, then shard id, then line order** - deterministic across devices regardless of read order; two events in the same second on different devices have a stable total order.+- **Legacy files are implicit shards.** `<date>.jsonl` / `<date>.md` (no device id) keep being read forever; no migration. The writer simply stops appending to them once sharding ships.+- **Markdown shards mirror JSONL shards** (`<date>.<deviceId>.md`). A regenerated merged `.md` view would itself be a cross-device write target - the exact conflict class this task removes. Obsidian users see one file per device per day; each is valid standalone.+- **`listLogDates` becomes the only way readers discover log days.** It returns sorted unique dates from any known filename shape (`<date>.jsonl`, `<date>.<id>.jsonl`, `.md` variants), and the five direct `readdirSync` scans in doctor/digest/digest-agent-summary route through it plus `readLogDay`-based merge reads. `parseLogDay` (markdown parser) gains a shard-aware sibling used by doctor's markdown lint so warnings still carry exact paths/lines.+- **Boundary config is vault-portable policy; matching is glob for sessions, regex for messages.** Session globs match against `session_id` and (when present) the transcript path; message regexes compile defensively - an invalid pattern degrades to a doctor warning and is skipped, never thrown. Machine-local config may ADD patterns (union semantics); it cannot remove vault policy.+- **Stateless sessions write nothing, read everything.** The gate sits at the top of `captureSessionLifecycleEvent` (and per-file in `importSession`): ignored -> return early with counters only; stateless -> skip signal writes, lifecycle log writes, and fact extraction but still return success so hosts keep functioning; suppressed message -> the text never reaches marker or fact extraction, only a counter survives.+- **Extraction patterns are a closed, ordered table** (same shape as `pre-compact-extract.ts` LABELS): 7 pattern families, English-first with structural cues, each capturing a single fact span. Only `role === "user"` turns are scanned. Every emitted signal carries `source_type: extracted` (new closed-enum member), a dedup hash over `(family, normalized fact)`, and the session ref - so re-imports and repeated prompts dedup exactly like markers do today.+- **Extraction counters ride existing result surfaces** (`CaptureSessionLifecycleResult`, `ImportSessionResult` gain `facts_extracted` / `facts_deduped`; capture results also gain `suppressed_messages`, `ignored_session`, `stateless_session` as applicable). The audit JSONL row carries the same counters - no new diagnostics file.+- **No new dependencies.** Globs are matched with a tiny anchored-translate helper (the codebase already avoids minimatch); randomness via `node:crypto`.++## File changes++New:+- `src/core/brain/entities/types.ts` - entity record, category/status enums, frontmatter contract.+- `src/core/brain/entities/canonical.ts` - normalize, identity key, alias resolution (the shared kernel).+- `src/core/brain/entities/registry.ts` - read/walk/upsert/relate/archive + duplicate refusal.+- `src/core/brain/entities/index-builder.ts` - in-memory identity index from a vault walk.+- `src/core/brain/capture-boundary.ts` - config matcher (session globs, message regexes), decision type, counters.+- `src/core/brain/fact-extract.ts` - pattern table + `extractFacts(turnText)` + router (`writeSignal` + entity anchoring).+- `src/cli/brain/verbs/entity.ts` - `o2b brain entity set|get|list|relate|archive`.+- `tests/core/brain/entities.*.test.ts`, `tests/core/brain/capture-boundary.test.ts`, `tests/core/brain/fact-extract.test.ts`, `tests/core/brain/log-shards.test.ts` (+ reader-merge, doctor, CLI, MCP tests).++Modified:+- `src/core/config.ts` - `device_id` accessor with first-use generation.+- `src/core/brain/log.ts` - `appendLogEvent` writes shard paths; `paths.ts` gains shard path helpers.+- `src/core/brain/log-jsonl.ts` - `readLogDay` merges shards; new `listLogDates`.+- `src/core/brain/doctor.ts` - scanners -> `listLogDates`; new lints: `duplicate-entity`, `broken-entity-relation`, `invalid-capture-pattern`, `sync-conflict-log`.+- `src/core/brain/digest.ts`, `digest-agent-summary.ts`, `temporal/build-index.ts` - date discovery via `listLogDates`.+- `src/core/brain/policy.ts` + `types.ts` - `sessions:` config block; `BRAIN_SIGNAL_SOURCE_TYPE.extracted`.+- `src/core/brain/session-lifecycle.ts` - boundary gate + fact extraction stage + counters.+- `src/core/brain/sessions/import.ts` - boundary filters + per-turn fact extraction + counters.+- `src/core/search/search.ts` (+ `entities.ts` seam) - alias-aware entity expansion at query time; `why_retrieved` notes canonical-entity hits.+- `src/mcp/brain-tools.ts` - `brain_entity` read tool (view: get|list).+- `src/cli/brain.ts`, `command-manifest.ts`, `help-text.ts` - verb registration.+- `README.md`, `CHANGELOG.md`, `docs/` - phase 5.++## Risks and open questions++- **Reader refactor regression risk** (highest): five scanners change date discovery; mitigated by golden tests asserting merged reads equal pre-shard reads on legacy fixtures, plus mixed legacy+shard fixtures.+- **Pattern precision**: the 7 fact families must stay high-precision (upstream reports precision-first design); each family ships with negative fixtures (assistant turns, code blocks, quoted text) proving non-matches.+- **Hot-path cost**: capture gate + extraction run per prompt event; both are regex-over-one-prompt with compiled-once patterns - measured in microseconds, and the whole hook stays fail-soft.+- **Glob semantics**: kept deliberately small (`*`, `?`, `**`); documented in the config template so operators do not expect full minimatch.+- **`.md` shard naming vs Obsidian daily-note plugins**: shard files keep the `YYYY-MM-DD` prefix so date-sorted views stay usable; documented in README.diff --git a/docs/brainstorm/memory-integrity/plan.md b/docs/brainstorm/memory-integrity/plan.mdnew file mode 100644index 00000000..cc53ed01--- /dev/null+++ b/docs/brainstorm/memory-integrity/plan.md@@ -0,0 +1,50 @@+# Memory Integrity Suite - implementation plan++TDD order: each task starts with failing tests. Features land sequentially (entities -> log shards -> capture boundaries -> fact extraction) because the fact router consumes both the entity kernel and the boundary gate. Format + lint before every commit.++## Tasks++### Task 1: Entity canonicalization kernel and registry core+- **Files**: `src/core/brain/entities/{types,canonical,registry,index-builder}.ts`, `src/core/brain/paths.ts` (entities dir), `tests/core/brain/entities.canonical.test.ts`, `tests/core/brain/entities.registry.test.ts`+- **Acceptance**: normalize/identity-key/alias resolution pass property fixtures (NFC, case, whitespace); upsert refuses duplicate `(category, normalized name)` and duplicate alias claims; archive removes from active lookup, restore returns it; registry rebuilds identically from Markdown alone; files stay Obsidian-valid frontmatter.+- **Depends on**: none++### Task 2: Entity surfaces - CLI verbs, MCP read tool, doctor lints, search alias boost, graph relations+- **Files**: `src/cli/brain/verbs/entity.ts`, `src/cli/brain.ts`, `src/cli/command-manifest.ts`, `src/cli/brain/help-text.ts`, `src/mcp/brain-tools.ts`, `src/core/brain/doctor.ts`, `src/core/search/search.ts`, `tests/cli/brain.entity.test.ts`, `tests/mcp/entity-tool.test.ts`, `tests/core/brain/doctor.entities.test.ts`, `tests/core/search/entity-alias.test.ts`+- **Acceptance**: `o2b brain entity set/get/list/relate/archive` round-trip with `--json`; `brain_entity` (view get|list) read-only, registry-guard caps green; doctor flags duplicate claims and broken relations; a query naming an alias boosts chunks naming the canonical entity and `why_retrieved` says so; relations appear in graph export through existing frontmatter-relations.+- **Depends on**: Task 1++### Task 3: Device identity and sharded log writer+- **Files**: `src/core/config.ts`, `src/core/brain/paths.ts` (shard helpers), `src/core/brain/log.ts`, `tests/core/config.device-id.test.ts`, `tests/core/brain/log-shards.write.test.ts`+- **Acceptance**: `device_id` generated once into device-local config, validated `[a-z0-9-]{1,32}`, never read from the vault; `appendLogEvent` writes `<date>.<id>.jsonl` + `<date>.<id>.md` under the existing per-dir lock; two writers with different ids never touch the same file; byte-deterministic output for fixed inputs.+- **Depends on**: none++### Task 4: Shard-merging readers and scanner refactor+- **Files**: `src/core/brain/log-jsonl.ts` (`readLogDay` merge, `listLogDates`), `src/core/brain/doctor.ts` (x2 scanners + `sync-conflict-log` lint), `src/core/brain/digest.ts` (x2), `src/core/brain/digest-agent-summary.ts`, `src/core/brain/temporal/build-index.ts`, `tests/core/brain/log-shards.read.test.ts`, golden-equivalence updates in existing log/digest/doctor tests+- **Acceptance**: merged read over legacy-only fixtures is byte-identical to pre-change output; mixed legacy+shard fixtures merge in (ts, shardId, line) order; all five scanners produce identical results through `listLogDates`; doctor warns on `.sync-conflict-*` files under `Brain/log/`.+- **Depends on**: Task 3++### Task 5: Capture-boundary config and matcher+- **Files**: `src/core/brain/policy.ts` (+ config template), `src/core/brain/types.ts`, `src/core/brain/capture-boundary.ts`, `tests/core/brain/policy.sessions.test.ts`, `tests/core/brain/capture-boundary.test.ts`+- **Acceptance**: `sessions.{ignore_patterns,stateless_patterns,ignore_message_patterns}` parse with per-key warnings (policy.ts conventions); glob matcher covers `*`/`?`/`**` anchored semantics; invalid message regex degrades to a warning and is skipped; machine-local config unions with vault policy; unconfigured vault behaves bit-identically to today.+- **Depends on**: none++### Task 6: Boundary wiring at both seams+- **Files**: `src/core/brain/session-lifecycle.ts`, `src/core/brain/sessions/import.ts`, `src/core/brain/doctor.ts` (`invalid-capture-pattern` lint), `tests/core/brain/session-lifecycle.boundary.test.ts`, `tests/core/brain/import-session.boundary.test.ts`+- **Acceptance**: ignored session produces no signals/log/markers, only counters in the audit row; stateless session reads fine but writes nothing; suppressed message text never reaches marker extraction and is never persisted; existing behavior unchanged when no patterns configured; counters surface in both result types.+- **Depends on**: Task 5++### Task 7: Regex fact extraction module+- **Files**: `src/core/brain/fact-extract.ts`, `src/core/brain/types.ts` (`source_type: extracted`), `tests/core/brain/fact-extract.test.ts`+- **Acceptance**: 7 pattern families extract from positive fixtures and reject negative fixtures (assistant text, code blocks, quoted speech); only `role: user` turns scanned; emitted signals carry `source_type: extracted`, family-scoped dedup hash, session ref; second run is a no-op (dedup).+- **Depends on**: none++### Task 8: Fact routing - live + batch wiring, entity anchoring, counters+- **Files**: `src/core/brain/session-lifecycle.ts`, `src/core/brain/sessions/import.ts`, `src/core/brain/fact-extract.ts` (router), `tests/core/brain/fact-extract.routing.test.ts`+- **Acceptance**: extraction runs only on turns that pass the boundary gate (pipeline order pinned by a test: suppressed input never extracts); a fact naming a registered entity or alias gets the canonical `entity_id` stamped in the signal note; `facts_extracted`/`facts_deduped` counters in capture and import results and audit rows.+- **Depends on**: Tasks 1, 6, 7++### Task 9: Docs+- **Files**: `README.md`, `CHANGELOG.md` (one v0.35.0 entry bundling the suite), `docs/` pages the design names+- **Acceptance**: every new surface (entity verbs, `brain_entity`, `sessions:` config, shard layout, extraction families) documented; CHANGELOG follows Keep a Changelog with the release narrative.+- **Depends on**: Tasks 1-8diff --git a/docs/brainstorm/memory-integrity/variants.md b/docs/brainstorm/memory-integrity/variants.mdnew file mode 100644index 00000000..b3eff31b--- /dev/null+++ b/docs/brainstorm/memory-integrity/variants.md@@ -0,0 +1,47 @@+# Memory Integrity Suite - brainstorm audit trail++Consultant: Claude CLI (`claude -p`), prompt in `cli-output/prompt.md`, raw output in `cli-output/claude.md`. Fallback consultant (Codex) was not needed - the primary returned three parseable variants on the first run.++## Variants (verbatim from the consultant)++### Variant 1: Four independent vertical slices+- **Approach**: Land each feature as a self-contained module against its own existing seam, sharing only primitives that already exist (config chain, `brainDirs`, doctor, `writeSignal`). Sharding edits `log.ts`/`log-jsonl.ts` and the directory-scanning readers; capture boundaries wrap `session-lifecycle.ts` + `import.ts`; regex extraction is a new fail-soft hook emitting `brain_feedback` candidates; the entity registry is a standalone `Brain/entities/` subsystem with its own index and doctor checks. No new cross-feature abstraction is introduced.+- **Trade-offs**:+ - Pro: maximally parallelizable; four small PRs-worth of work bundled into one CHANGELOG, each independently testable under TDD.+ - Pro: smallest blast radius on the hot capture path; fail-soft is local to each slice.+ - Pro: lowest risk of speculative generality - matches the tasks' explicit "no over-engineering" warnings.+ - Con: regex extraction (task 3) and message suppression (task 4) independently re-walk the same `SessionTurn` stream and duplicate turn-iteration + pattern-compile + malformed-pattern handling.+ - Con: name normalization/alias resolution gets implemented twice - once in the entity registry, once (informally) wherever extracted identity/possession facts are routed - so extracted facts never cleanly resolve to canonical entities.+ - Con: four separate diagnostics surfaces; no single place to reason about "what entered/was kept out of memory."+- **Complexity**: medium+- **Risk**: low++### Variant 2: Capture-boundary pipeline + canonicalization kernel+- **Approach**: Pair the two regex-over-turns features into one ordered, deterministic capture pipeline at the shared ingestion seam (`session-lifecycle.ts` live hook and `import.ts` batch): source-classify/suppress (task 4) -> extract facts (task 3) -> route to signals, with suppression and stateless mode gating extraction so suppressed/read-only turns never produce evidence. Build the entity registry (task 1) as its own subsystem but factor out a small canonicalization primitive (normalized `(category, name)` + alias index) that the extraction router reuses, so identity/possession facts can resolve to canonical entities. Keep device sharding (task 2) fully independent as a write-layer/file-strategy change, since it shares nothing semantic with the other three.+- **Trade-offs**:+ - Pro: eliminates the real duplication (one turn walk, one pattern-compile/error-handling path) and makes the suppression->extraction ordering explicit rather than emergent.+ - Pro: extracted facts gain a precise anchor via the shared canonicalization kernel - the entity task's core value proposition, delivered without coupling the two subsystems.+ - Pro: single coherent capture-diagnostics surface (counters, active patterns, foreground/side-channel, unknown-source) for `brain sources`/doctor.+ - Pro: sharding stays decoupled and shippable on its own, respecting its "decided design."+ - Con: the pipeline couples tasks 3+4 on the hot path; staged fail-soft must be carefully preserved (one stage's crash must not block the others or the runtime).+ - Con: more upfront design than Variant 1; the 3+4 pairing must land together rather than as two trivially independent slices.+- **Complexity**: medium+- **Risk**: medium++### Variant 3: Full integrity substrate+- **Approach**: Extract a foundational layer first - a generalized shardable append-only Brain store (sharding the log becomes one instance, and signals/entities could shard too), a reusable rebuildable-index-as-cache primitive shared by the log-shard reader and the entity identity index, and a single turn-scan/filter primitive - then implement all four features as thin consumers on top. Everything routes through common substrate contracts for write/merge, index rebuild, and turn processing.+- **Trade-offs**:+ - Pro: maximal reuse and a uniform index-is-cache/append-only contract across the whole Brain; future features inherit the substrate.+ - Pro: one consistent merged-read and rebuild story everywhere.+ - Con: directly contradicts the tasks' explicit anti-over-engineering guidance (sharding warns against a daemon; entity task insists the index is a cache, not a new storage model).+ - Con: speculative generality - entity writes are rare and operator-governed, so they don't need device sharding; generalizing the store buys little and risks regressing the many directory-scanning log readers (`readAllLogRecords`, temporal index).+ - Con: largest blast radius, slowest to ship, hardest to keep deterministic and backward-compatible across all existing readers in one release.+- **Complexity**: large+- **Risk**: high++### Consultant recommendation: Variant 2+**Rationale (verbatim)**: It captures the only genuine synergies - the two regex-over-turns features share one turn seam and a suppress-then-extract ordering, and a small canonicalization kernel lets extracted facts resolve to canonical entities - while leaving the orthogonal sharding work decoupled and independently shippable, exactly as its "decided design" expects. This avoids both Variant 1's duplicated turn-walking and normalization logic and Variant 3's over-engineered substrate that the tasks explicitly warn against, and it keeps each piece testable, fail-soft, and backward-compatible within a single bundled release.++## Orchestrator decision++Variant 2, accepted as recommended. Project context confirms both synergies the consultant identified: `captureSessionLifecycleEvent` and `importSession` are verifiably the only two ingestion seams (so one pipeline covers everything), and the entity task's own acceptance criteria require alias resolution during lookup AND search - the same normalization the fact router needs. Variant 3 was additionally ruled out by the live-verified reader inventory (five independent directory scanners would all sit on top of a new substrate in one release - exactly the regression surface the constraints forbid). One refinement over the consultant's framing: the in-memory identity index is built per read rather than maintained as a cache file, because a persisted index file under Syncthing would reintroduce the conflict class the log-shard task removes.diff --git a/openclaw.plugin.json b/openclaw.plugin.jsonindex 7e1c9f7b..195cee2c 100644--- a/openclaw.plugin.json+++ b/openclaw.plugin.json@@ -2,7 +2,7 @@"id": "open-second-brain","name": "Open Second Brain","description": "Second brain for AI agents using Obsidian-compatible Markdown vaults.",- "version": "0.34.0",+ "version": "0.35.0","activation": { "onStartup": true },"skills": ["./skills"],"contracts": {diff --git a/package.json b/package.jsonindex 877965d7..41d5461d 100644--- a/package.json+++ b/package.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "0.34.0",+ "version": "0.35.0","private": false,"description": "Second brain for AI agents using Obsidian-compatible Markdown vaults. Works with Hermes, Claude Code, Codex, and OpenClaw.","keywords": [diff --git a/plugin.yaml b/plugin.yamlindex a3039c8c..c1fdcd75 100644--- a/plugin.yaml+++ b/plugin.yaml@@ -1,5 +1,5 @@name: open-second-brain-version: "0.34.0"+version: "0.35.0"description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."author: "Open Second Brain contributors"memory_provider: truediff --git a/plugins/codex/.codex-plugin/plugin.json b/plugins/codex/.codex-plugin/plugin.jsonindex b81802b9..a510a7cd 100644--- a/plugins/codex/.codex-plugin/plugin.json+++ b/plugins/codex/.codex-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "0.34.0",+ "version": "0.35.0","description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.","author": {"name": "Open Second Brain contributors",diff --git a/plugins/hermes/plugin.yaml b/plugins/hermes/plugin.yamlindex a3039c8c..c1fdcd75 100644--- a/plugins/hermes/plugin.yaml+++ b/plugins/hermes/plugin.yaml@@ -1,5 +1,5 @@name: open-second-brain-version: "0.34.0"+version: "0.35.0"description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."author: "Open Second Brain contributors"memory_provider: truediff --git a/pyproject.toml b/pyproject.tomlindex dea514f4..d101a99e 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -10,7 +10,7 @@ build-backend = "setuptools.build_meta"# CLI entry points (those moved to `package.json` `bin`).[project]name = "open-second-brain"-version = "0.34.0"+version = "0.35.0"description = "Hermes Python shim for Open Second Brain. Most of the project (CLI, MCP server, OpenClaw plugin) is TypeScript on Bun; see package.json."readme = "README.md"requires-python = ">=3.11"diff --git a/src/cli/brain.ts b/src/cli/brain.tsindex de4e8a8b..6f02f8bf 100644--- a/src/cli/brain.ts+++ b/src/cli/brain.ts@@ -48,6 +48,7 @@ import {cmdBrainUpgrade,handleBrainSnapshotSubcommand,cmdBrainScanInline,+ cmdBrainEntity,cmdBrainImportSession,cmdBrainSessionHook,cmdBrainImportClaudeMemory,@@ -171,6 +172,8 @@ export async function handleBrainSubcommand(argv: ReadonlyArray<string>): Promisreturn await cmdBrainScanInline(rest);case "import-session":return await cmdBrainImportSession(rest);+ case "entity":+ return await cmdBrainEntity(rest);case "session-hook":return await cmdBrainSessionHook(rest);case "import-claude-memory":diff --git a/src/cli/brain/help-text.ts b/src/cli/brain/help-text.tsindex 09e9d73f..32233a94 100644--- a/src/cli/brain/help-text.ts+++ b/src/cli/brain/help-text.ts@@ -51,6 +51,7 @@ Brain verbs (observing memory):mcp-landscape List MCP servers configured across the vault (packages, env names)scan-inline Capture @osb markers from folders listed under notes.read_paths in _brain.yamlimport-session Replay signals from a registered agent session .jsonl (or directory)+ entity Canonical entity registry: set, get, list, relate, archivesession-hook Capture one runtime hook payload from stdin (internal hook bridge)import-claude-memory Import metadata.type:feedback MEMORY entries as confirmed preferencespage-dedup Detect (and optionally merge) near-duplicate vault pages@@ -264,6 +265,14 @@ export const VERB_HELP: Record<string, string> = {"accumulated evidence fields. CONFLICT (preference exists without a manifest\n" +"entry) exits 2 — never silent overwrites.\n" +"Default is --dry-run; --apply requires --yes in non-interactive mode.\n",+ entity:+ "usage: o2b brain entity <set|get|list|relate|archive> [args]\n" ++ " set <category> <name> [--alias <a>]... [--body <md>] [--confidence <c>] [--json]\n" ++ " get <name-or-alias> [--category <c>] [--json] exit 2 when not found\n" ++ " list [--category <c>] [--status active|archived] [--json]\n" ++ " relate <from> <relation> <to> [--from-category <c>] [--to-category <c>] [--json]\n" ++ " archive <name-or-alias> [--restore] [--category <c>] [--json]\n" ++ "One canonical entity per (category, name); aliases resolve to the canonical record.","import-session":"usage: o2b brain import-session <path> [--vault <vault>]\n" +" [--format auto|<registered-adapter>]\n" +diff --git a/src/cli/brain/verbs/entity.ts b/src/cli/brain/verbs/entity.tsnew file mode 100644index 00000000..a25d4454--- /dev/null+++ b/src/cli/brain/verbs/entity.ts@@ -0,0 +1,227 @@+/**+ * `o2b brain entity <set|get|list|relate|archive>` - canonical entity+ * registry verbs (Memory Integrity Suite). Writes stay on the CLI;+ * the MCP surface exposes the read-only subset (`brain_entity`).+ */++import { defaultConfigPath, resolveAgentName } from "../../../core/config.ts";+import {+ archiveEntity,+ getEntity,+ listEntities,+ relateEntities,+ upsertEntity,+} from "../../../core/brain/entities/registry.ts";+import { BRAIN_ENTITY_STATUS, type BrainEntity } from "../../../core/brain/entities/types.ts";+import { parse, fail, ok, okJson, normalizeFlagString, resolveBrainVault } from "../helpers.ts";++function entityPayload(entity: BrainEntity): Record<string, unknown> {+ return {+ id: entity.id,+ category: entity.category,+ name: entity.name,+ aliases: [...entity.aliases],+ status: entity.status,+ ...(entity.source_agent !== undefined ? { source_agent: entity.source_agent } : {}),+ ...(entity.confidence !== undefined ? { confidence: entity.confidence } : {}),+ created_at: entity.created_at,+ updated_at: entity.updated_at,+ ...(entity.archived_at !== undefined ? { archived_at: entity.archived_at } : {}),+ relations: entity.relations.map((r) => ({ relation: r.relation, target: r.target })),+ path: entity.path,+ };+}++function entityLine(entity: BrainEntity): string {+ const aliases = entity.aliases.length > 0 ? ` aliases: ${entity.aliases.join(", ")}` : "";+ return `${entity.id} [${entity.status}] ${entity.category}/${entity.name}${aliases}`;+}++export async function cmdBrainEntity(argv: string[]): Promise<number> {+ const sub = argv[0];+ const rest = argv.slice(1);+ switch (sub) {+ case "set":+ return entitySet(rest);+ case "get":+ return entityGet(rest);+ case "list":+ return entityList(rest);+ case "relate":+ return entityRelate(rest);+ case "archive":+ return entityArchive(rest);+ default:+ return fail(+ "brain entity requires a subcommand: set <category> <name> | get <name> | list | " ++ "relate <from> <relation> <to> | archive <name> [--restore]",+ );+ }+}++function entitySet(argv: string[]): number {+ const { flags, positional } = parse(argv, {+ vault: { type: "string" },+ alias: { type: "string-array" },+ body: { type: "string" },+ confidence: { type: "string" },+ json: { type: "boolean" },+ });+ if (positional.length < 2) {+ return fail("brain entity set requires <category> and <name> arguments");+ }+ const config = defaultConfigPath();+ const vault = resolveBrainVault(flags["vault"] as string | undefined, config);+ const agent = resolveAgentName(config);+ const body = normalizeFlagString(flags["body"]);+ const confidence = normalizeFlagString(flags["confidence"]);+ try {+ const out = upsertEntity(vault, {+ category: positional[0]!,+ name: positional[1]!,+ aliases: (flags["alias"] as string[] | undefined) ?? [],+ agent,+ now: new Date(),+ ...(body !== null ? { body } : {}),+ ...(confidence !== null ? { confidence } : {}),+ });+ if (flags["json"]) {+ okJson({ ...entityPayload(out.entity), created: out.created });+ } else {+ ok(`${out.created ? "created" : "updated"}: ${out.entity.id}`);+ }+ return 0;+ } catch (exc) {+ return fail(`brain entity set failed: ${(exc as Error).message}`);+ }+}++function entityGet(argv: string[]): number {+ const { flags, positional } = parse(argv, {+ vault: { type: "string" },+ category: { type: "string" },+ json: { type: "boolean" },+ });+ if (positional.length < 1) return fail("brain entity get requires a <name-or-alias> argument");+ const vault = resolveBrainVault(flags["vault"] as string | undefined, defaultConfigPath());+ const category = normalizeFlagString(flags["category"]);+ try {+ const entity = getEntity(vault, {+ query: positional[0]!,+ ...(category !== null ? { category } : {}),+ });+ if (entity === null) {+ process.stderr.write(`entity not found: ${positional[0]!}\n`);+ return 2;+ }+ if (flags["json"]) {+ okJson(entityPayload(entity));+ } else {+ ok(entityLine(entity));+ if (entity.body.trim().length > 0) ok(entity.body.trim());+ }+ return 0;+ } catch (exc) {+ return fail(`brain entity get failed: ${(exc as Error).message}`);+ }+}++function entityList(argv: string[]): number {+ const { flags } = parse(argv, {+ vault: { type: "string" },+ category: { type: "string" },+ status: { type: "string" },+ json: { type: "boolean" },+ });+ const vault = resolveBrainVault(flags["vault"] as string | undefined, defaultConfigPath());+ const category = normalizeFlagString(flags["category"]);+ const statusRaw = normalizeFlagString(flags["status"]);+ if (+ statusRaw !== null &&+ statusRaw !== BRAIN_ENTITY_STATUS.active &&+ statusRaw !== BRAIN_ENTITY_STATUS.archived+ ) {+ return fail("--status must be 'active' or 'archived'");+ }+ try {+ const entities = listEntities(vault, {+ ...(category !== null ? { category } : {}),+ ...(statusRaw !== null ? { status: statusRaw } : {}),+ });+ if (flags["json"]) {+ okJson({ entities: entities.map(entityPayload), total: entities.length });+ } else if (entities.length === 0) {+ ok("no entities");+ } else {+ for (const entity of entities) ok(entityLine(entity));+ }+ return 0;+ } catch (exc) {+ return fail(`brain entity list failed: ${(exc as Error).message}`);+ }+}++function entityRelate(argv: string[]): number {+ const { flags, positional } = parse(argv, {+ vault: { type: "string" },+ "from-category": { type: "string" },+ "to-category": { type: "string" },+ json: { type: "boolean" },+ });+ if (positional.length < 3) {+ return fail("brain entity relate requires <from> <relation> <to> arguments");+ }+ const vault = resolveBrainVault(flags["vault"] as string | undefined, defaultConfigPath());+ const fromCategory = normalizeFlagString(flags["from-category"]);+ const toCategory = normalizeFlagString(flags["to-category"]);+ try {+ const entity = relateEntities(vault, {+ from: { query: positional[0]!, ...(fromCategory !== null ? { category: fromCategory } : {}) },+ relation: positional[1]!,+ to: { query: positional[2]!, ...(toCategory !== null ? { category: toCategory } : {}) },+ now: new Date(),+ });+ if (flags["json"]) {+ okJson(entityPayload(entity));+ } else {+ ok(`${entity.id} ${positional[1]!} -> ${positional[2]!}`);+ }+ return 0;+ } catch (exc) {+ return fail(`brain entity relate failed: ${(exc as Error).message}`);+ }+}++function entityArchive(argv: string[]): number {+ const { flags, positional } = parse(argv, {+ vault: { type: "string" },+ category: { type: "string" },+ restore: { type: "boolean" },+ json: { type: "boolean" },+ });+ if (positional.length < 1) {+ return fail("brain entity archive requires a <name-or-alias> argument");+ }+ const vault = resolveBrainVault(flags["vault"] as string | undefined, defaultConfigPath());+ const category = normalizeFlagString(flags["category"]);+ try {+ const entity = archiveEntity(+ vault,+ { query: positional[0]!, ...(category !== null ? { category } : {}) },+ { now: new Date(), ...(flags["restore"] ? { restore: true } : {}) },+ );+ if (flags["json"]) {+ okJson(entityPayload(entity));+ } else {+ ok(`${entity.id} -> ${entity.status}`);+ }+ return 0;+ } catch (exc) {+ const message = (exc as Error).message;+ if (message.includes("not found")) {+ process.stderr.write(`${message}\n`);+ return 2;+ }+ return fail(`brain entity archive failed: ${message}`);+ }+}diff --git a/src/cli/brain/verbs/index.ts b/src/cli/brain/verbs/index.tsindex 29bf7165..efb61af7 100644--- a/src/cli/brain/verbs/index.ts+++ b/src/cli/brain/verbs/index.ts@@ -36,6 +36,7 @@ export { cmdBrainUpgrade } from "./upgrade.ts";export { cmdBrainSnapshotDiff, handleBrainSnapshotSubcommand } from "./snapshot.ts";export { cmdBrainScanInline } from "./scan-inline.ts";export { cmdBrainImportSession } from "./import-session.ts";+export { cmdBrainEntity } from "./entity.ts";export { cmdBrainSessionHook } from "./session-hook.ts";export { cmdBrainImportClaudeMemory } from "./import-claude-memory.ts";export { cmdBrainPageDedup } from "./page-dedup.ts";diff --git a/src/cli/command-manifest.ts b/src/cli/command-manifest.tsindex 3186f267..952d26df 100644--- a/src/cli/command-manifest.ts+++ b/src/cli/command-manifest.ts@@ -149,6 +149,7 @@ export const CLI_COMMAND_MANIFEST: CliRootManifest = Object.freeze({command("mcp-landscape", "List MCP servers configured across the vault"),command("scan-inline", "Capture inline @osb markers"),command("import-session", "Replay registered agent sessions"),+ command("entity", "Canonical entity registry: set, get, list, relate, archive"),command("session-hook", "Capture runtime lifecycle hook payloads"),command("import-claude-memory", "Import Claude memory feedback"),],diff --git a/src/core/brain/backlinks.ts b/src/core/brain/backlinks.tsindex e1cd1909..47926fa6 100644--- a/src/core/brain/backlinks.ts+++ b/src/core/brain/backlinks.ts@@ -33,7 +33,7 @@ import { relationFromFrontmatterField } from "../graph/relation-vocab.ts";import { normalizeRelationTarget } from "../graph/frontmatter-relations.ts";import { buildAliasIndex } from "./link-graph/alias-index.ts";import { extractWikilinkRichBodies, parseWikilinkRich } from "./link-graph/parse-wikilink.ts";-import { parseLogDay } from "./log.ts";+import { listLogDates, readLogDay } from "./log-jsonl.ts";import { brainDirs } from "./paths.ts";import { normalizeDerivedKeys } from "./preference.ts";import { normaliseWikilinkTarget } from "./wikilink.ts";@@ -264,14 +264,12 @@ function collectLog(push: (target: string, ref: BacklinkRef) => void,): void {if (!existsSync(dir)) return;- for (const name of readdirSync(dir)) {- if (!name.endsWith(".md")) continue;- const date = name.slice(0, -".md".length);- if (!/^\d{4}-\d{2}-\d{2}$/.test(date)) continue;+ // Shard-aware (Memory Integrity Suite): merged per-day reads.+ for (const date of listLogDates(vault)) {const source = `log-${date}`;let entries;try {- entries = parseLogDay(vault, date).entries;+ entries = readLogDay(vault, date).entries;} catch {continue;}diff --git a/src/core/brain/capture-boundary.ts b/src/core/brain/capture-boundary.tsnew file mode 100644index 00000000..2d6d6b27--- /dev/null+++ b/src/core/brain/capture-boundary.ts@@ -0,0 +1,141 @@+/**+ * Capture-boundary matcher (Memory Integrity Suite, t_0532ed5a).+ *+ * The single decision point for what runtime output may become Brain+ * evidence. Both ingestion seams consult it BEFORE any extraction:+ * `session-lifecycle.ts` (live hooks) and `sessions/import.ts`+ * (batch import). The pipeline order is fixed - classify/suppress+ * first, extract second - so suppressed or ignored input can never+ * produce signals, facts, or log entries.+ *+ * Session patterns are anchored globs (`*` any run, `?` one char,+ * everything else literal) matched against the session id and, when+ * known, the transcript path. Message patterns are regexes; an+ * invalid one degrades to a warning and is skipped - a typo in+ * `_brain.yaml` must never take capture down. Everything fails soft+ * to "capture" because losing real evidence is worse than admitting+ * noise the doctor can surface later.+ */++import { defaultConfigPath, discoverConfig } from "../config.ts";+import { BRAIN_SESSIONS_DEFAULTS, loadBrainConfigDetailed, resolveSessions } from "./policy.ts";+import type { ResolvedBrainSessionsConfig } from "./types.ts";++export type SessionCaptureDecision = "capture" | "ignore" | "stateless";++export interface CaptureBoundary {+ /**+ * Decide what a session may do: `ignore` produces nothing,+ * `stateless` reads but never writes, `capture` is full capture.+ * Ignore outranks stateless when both match.+ */+ sessionDecision(sessionId: string | undefined, transcriptPath?: string): SessionCaptureDecision;+ /** True when the message text must not reach extraction or storage. */+ suppressMessage(text: string): boolean;+ /** Invalid patterns skipped at compile time (doctor surfaces these). */+ readonly warnings: ReadonlyArray<string>;+ /** The resolved pattern lists, for diagnostics surfaces. */+ readonly policy: ResolvedBrainSessionsConfig;+}++/** Translate one anchored glob (`*`, `?`) to a RegExp. */+function globToRegExp(glob: string): RegExp {+ let out = "^";+ for (const ch of glob) {+ if (ch === "*") out += ".*";+ else if (ch === "?") out += ".";+ else out += ch.replace(/[.+^${}()|[\]\\]/g, "\\$&");+ }+ return new RegExp(out + "$");+}++function matchesAny(regexps: ReadonlyArray<RegExp>, id?: string, path?: string): boolean {+ return regexps.some(+ (re) => (id !== undefined && re.test(id)) || (path !== undefined && re.test(path)),+ );+}++export function compileCaptureBoundary(policy: ResolvedBrainSessionsConfig): CaptureBoundary {+ const warnings: string[] = [];+ const ignore = policy.ignore_patterns.map(globToRegExp);+ const stateless = policy.stateless_patterns.map(globToRegExp);+ const suppress: RegExp[] = [];+ for (const pattern of policy.ignore_message_patterns) {+ try {+ suppress.push(new RegExp(pattern));+ } catch (err) {+ warnings.push(+ `invalid sessions.ignore_message_patterns regex ${JSON.stringify(pattern)}: ` ++ `${(err as Error).message} - pattern skipped`,+ );+ }+ }++ return Object.freeze({+ sessionDecision(sessionId: string | undefined, transcriptPath?: string) {+ if (matchesAny(ignore, sessionId, transcriptPath)) return "ignore" as const;+ if (matchesAny(stateless, sessionId, transcriptPath)) return "stateless" as const;+ return "capture" as const;+ },+ suppressMessage(text: string) {+ return suppress.some((re) => re.test(text));+ },+ warnings: Object.freeze(warnings),+ policy,+ });+}++export interface BuildCaptureBoundaryOptions {+ /** Device-local config override for tests; defaults to the standard chain. */+ readonly localConfigPath?: string;+}++/** Comma-separated machine-local additions from the device config. */+function localAdditions(localConfigPath: string | undefined, key: string): string[] {+ try {+ const data = discoverConfig(localConfigPath ?? defaultConfigPath()).data;+ const raw = data[key];+ if (!raw) return [];+ return raw+ .split(",")+ .map((s) => s.trim())+ .filter((s) => s.length > 0);+ } catch {+ return [];+ }+}++/**+ * Resolve the effective boundary for a vault: the vault-portable+ * `sessions:` policy unioned with machine-local additions+ * (`sessions_ignore_patterns` / `sessions_stateless_patterns` /+ * `sessions_ignore_message_patterns`, comma-separated, in the+ * device config). Local config can ADD patterns, never remove+ * vault policy. Any load failure resolves to capture-everything.+ */+export function buildCaptureBoundary(+ vault: string,+ opts: BuildCaptureBoundaryOptions = {},+): CaptureBoundary {+ let vaultPolicy: ResolvedBrainSessionsConfig;+ try {+ vaultPolicy = resolveSessions(loadBrainConfigDetailed(vault).config);+ } catch {+ vaultPolicy = BRAIN_SESSIONS_DEFAULTS;+ }+ const merged: ResolvedBrainSessionsConfig = Object.freeze({+ ignore_patterns: Object.freeze([+ ...vaultPolicy.ignore_patterns,+ ...localAdditions(opts.localConfigPath, "sessions_ignore_patterns"),+ ]),+ stateless_patterns: Object.freeze([+ ...vaultPolicy.stateless_patterns,+ ...localAdditions(opts.localConfigPath, "sessions_stateless_patterns"),+ ]),+ ignore_message_patterns: Object.freeze([+ ...vaultPolicy.ignore_message_patterns,+ ...localAdditions(opts.localConfigPath, "sessions_ignore_message_patterns"),+ ]),+ });+ return compileCaptureBoundary(merged);+}diff --git a/src/core/brain/digest-agent-summary.ts b/src/core/brain/digest-agent-summary.tsindex 6468a70b..694a2c8a 100644--- a/src/core/brain/digest-agent-summary.ts+++ b/src/core/brain/digest-agent-summary.ts@@ -6,11 +6,12 @@* apply-evidence / note), and attribution to confirmed/retired rules.*/-import { existsSync, readdirSync, readFileSync } from "node:fs";+import { existsSync, readFileSync } from "node:fs";import { join } from "node:path";import { brainDirs } from "./paths.ts";-import { parseLogDay, type BrainLogEntry } from "./log.ts";+import type { BrainLogEntry } from "./log.ts";+import { listLogDates, readLogDay } from "./log-jsonl.ts";import { BRAIN_LOG_EVENT_KIND } from "./types.ts";export interface AgentSummaryEntry {@@ -121,22 +122,16 @@ export function computeAgentSummary(}function readLogsInWindow(vault: string, since: Date, until: Date): BrainLogEntry[] {- const dirs = brainDirs(vault);- if (!existsSync(dirs.log)) return [];const sinceIso = since.toISOString();const untilIso = until.toISOString();const sinceDay = sinceIso.slice(0, 10);const untilDay = untilIso.slice(0, 10);const out: BrainLogEntry[] = [];- const dates = readdirSync(dirs.log, { withFileTypes: true })- .filter((d) => d.isFile() && d.name.endsWith(".md"))- .map((d) => d.name.slice(0, -3))- .filter((n) => /^\d{4}-\d{2}-\d{2}$/.test(n))- .toSorted();- for (const date of dates) {+ // Shard-aware (Memory Integrity Suite): one discovery helper, merged reads.+ for (const date of listLogDates(vault)) {if (date < addDays(sinceDay, -1)) continue;if (date > addDays(untilDay, 1)) continue;- const { entries } = parseLogDay(vault, date);+ const { entries } = readLogDay(vault, date);for (const e of entries) {if (e.timestamp >= sinceIso && e.timestamp < untilIso) {out.push(e);diff --git a/src/core/brain/digest.ts b/src/core/brain/digest.tsindex f3995c11..1f60ed13 100644--- a/src/core/brain/digest.ts+++ b/src/core/brain/digest.ts@@ -55,7 +55,8 @@ import {} from "./policy.ts";import { normaliseWikilinkTarget, renderPrefLink } from "./wikilink.ts";import { parsePreference, parseRetired } from "./preference.ts";-import { parseLogDay, type BrainLogEntry } from "./log.ts";+import type { BrainLogEntry } from "./log.ts";+import { listLogDates, readLogDay } from "./log-jsonl.ts";import {BRAIN_APPLY_RESULT,BRAIN_LOG_EVENT_KIND,@@ -735,24 +736,19 @@ function readAllRetired(vault: string): ReadonlyArray<RetiredWithPath> {}function readLogsInWindow(vault: string, since: Date, until: Date): ReadonlyArray<BrainLogEntry> {- const dirs = brainDirs(vault);- if (!existsSync(dirs.log)) return [];const sinceIso = since.toISOString();const untilIso = until.toISOString();// Restrict scans to dates intersecting the window — but we err// permissive (one day before / after) to avoid TZ off-by-ones.+ // Shard-aware (Memory Integrity Suite): dates come from listLogDates+ // and entries arrive merged across device shards via readLogDay.const sinceDay = sinceIso.slice(0, 10);const untilDay = untilIso.slice(0, 10);const out: BrainLogEntry[] = [];- const dates = readdirSync(dirs.log, { withFileTypes: true })- .filter((d) => d.isFile() && d.name.endsWith(".md"))- .map((d) => d.name.slice(0, -".md".length))- .filter((n) => /^\d{4}-\d{2}-\d{2}$/.test(n))- .toSorted();- for (const date of dates) {+ for (const date of listLogDates(vault)) {if (date < addDays(sinceDay, -1)) continue;if (date > addDays(untilDay, 1)) continue;- const { entries } = parseLogDay(vault, date);+ const { entries } = readLogDay(vault, date);for (const e of entries) {if (e.timestamp >= sinceIso && e.timestamp < untilIso) {out.push(e);@@ -776,15 +772,8 @@ function addDays(day: string, delta: number): string {* without ever being applied. This matches design doc §8.2 example.*/function findFirstAppliedArtifact(vault: string, prefId: string): string | null {- const dirs = brainDirs(vault);- if (!existsSync(dirs.log)) return null;- const dates = readdirSync(dirs.log, { withFileTypes: true })- .filter((d) => d.isFile() && d.name.endsWith(".md"))- .map((d) => d.name.slice(0, -".md".length))- .filter((n) => /^\d{4}-\d{2}-\d{2}$/.test(n))- .toSorted();- for (const date of dates) {- const { entries } = parseLogDay(vault, date);+ for (const date of listLogDates(vault)) {+ const { entries } = readLogDay(vault, date);for (const e of entries) {if (e.eventType !== BRAIN_LOG_EVENT_KIND.applyEvidence) continue;if (e.body["result"] !== BRAIN_APPLY_RESULT.applied) continue;diff --git a/src/core/brain/doctor.ts b/src/core/brain/doctor.tsindex 561286fd..3706acb1 100644--- a/src/core/brain/doctor.ts+++ b/src/core/brain/doctor.ts@@ -29,7 +29,7 @@* ISO-8601 timestamps (`null` / missing acceptable for the* optional ones). Bad → error.* 7. **Log header parsing.** Every malformed `## <HH:MM:SS>Z — kind`- * block surfaced by `parseLogDay` as a warning is forwarded here.+ * block surfaced by `parseLogDayFile` (per shard) is forwarded here.** The function never mutates state. It will gracefully tolerate a* vault that has no Brain layer yet (returns clean) — same shape as@@ -42,9 +42,17 @@ import { join } from "node:path";import { extractWikilinks, listVaultBasenames, parseFrontmatter } from "../vault.ts";import { resolveVaultScope } from "../vault-scope/index.ts";import { buildBacklinkIndex } from "./backlinks.ts";+import { buildEntityIndex } from "./entities/index-builder.ts";+import { buildCaptureBoundary } from "./capture-boundary.ts";import { verifyContentHash } from "./content-hash.ts";import { scanDanglingWorkruns } from "./dream-workrun.ts";-import { parseLogDay } from "./log.ts";+import { parseLogDayFile } from "./log.ts";+import {+ listLogDates,+ listLogMarkdownFiles,+ listLogSyncConflicts,+ readLogDay,+} from "./log-jsonl.ts";import {BRAIN_CONFIG_SUPPORTED_VERSIONS,BRAIN_GUARDRAIL_DEFAULTS,@@ -254,7 +262,7 @@ export function runDoctor(vault: string, opts: RunDoctorOptions = {}): RunDoctor}}- // 7. Log header parsing — surface warnings from `parseLogDay`.+ // 7. Log header parsing — surface warnings from every markdown shard.checkLogs(vault, issues);// 8. Broken-backlinks lint — any preference / retired / log entry@@ -328,6 +336,46 @@ export function runDoctor(vault: string, opts: RunDoctorOptions = {}): RunDoctor} catch {/* doctor never throws */}+ // Memory Integrity Suite: canonical entity registry hygiene. Write+ // seams refuse duplicates, so anything reported here arrived through+ // hand edits or sync merges - observable, never auto-deleted.+ try {+ checkEntities(vault, issues);+ } catch {+ /* doctor never throws */+ }+ // Memory Integrity Suite: capture-boundary patterns that failed to+ // compile (invalid regex in sessions.ignore_message_patterns or the+ // machine-local additions). Capture itself degrades gracefully; the+ // doctor makes the skipped pattern visible.+ try {+ for (const warning of buildCaptureBoundary(vault).warnings) {+ issues.push({+ severity: "warning",+ code: "invalid-capture-pattern",+ message: warning,+ });+ }+ } catch {+ /* doctor never throws */+ }+ // Memory Integrity Suite: leftover Syncthing conflict copies under+ // Brain/log/. The per-device shard layout prevents new ones; old+ // copies need a manual union+dedup merge into the day's log.+ try {+ for (const path of listLogSyncConflicts(vault)) {+ issues.push({+ severity: "warning",+ code: "sync-conflict-log",+ path,+ message:+ `Syncthing sync-conflict copy under Brain/log/: ${path}. ` ++ "Merge its rows into the day's log (union + dedup by ts and content), then delete it.",+ });+ }+ } catch {+ /* doctor never throws */+ }// v0.14.0 semantic-health pass. Best-effort like every other lint:// a failure here must not poison the structural warning / error@@ -742,14 +790,11 @@ function checkRetired(// ----- Log check ------------------------------------------------------------function checkLogs(vault: string, issues: DoctorIssue[]): void {- const dirs = brainDirs(vault);- if (!existsSync(dirs.log)) return;- const dates = readdirSync(dirs.log, { withFileTypes: true })- .filter((d) => d.isFile() && d.name.endsWith(".md"))- .map((d) => d.name.slice(0, -".md".length))- .filter((n) => /^\d{4}-\d{2}-\d{2}$/.test(n));- for (const date of dates) {- const { warnings } = parseLogDay(vault, date);+ // Per-device shards (Memory Integrity Suite): lint every markdown+ // log file - legacy `<date>.md` and sharded `<date>.<deviceId>.md` -+ // so warnings keep their exact file paths and line numbers.+ for (const { date, path } of listLogMarkdownFiles(vault)) {+ const { warnings } = parseLogDayFile(vault, date, path);for (const w of warnings) {issues.push({severity: "warning",@@ -903,15 +948,12 @@ function readAllPreferenceRecords(vault: string): ReadonlyArray<PreferenceRecord* log-walking lints don't each re-parse the directory.*/function readAllLogRecords(vault: string): ReadonlyArray<LogRecord> {- const dirs = brainDirs(vault);- if (!existsSync(dirs.log)) return [];+ // Shard-aware (Memory Integrity Suite): dates come from the single+ // discovery helper and entries arrive merged across device shards.const out: LogRecord[] = [];- for (const name of readdirSync(dirs.log)) {- if (!name.endsWith(".md")) continue;- const date = name.slice(0, -".md".length);- if (!/^\d{4}-\d{2}-\d{2}$/.test(date)) continue;+ for (const date of listLogDates(vault)) {try {- out.push({ date, entries: parseLogDay(vault, date).entries });+ out.push({ date, entries: readLogDay(vault, date).entries });} catch {// parse error — surfaced separately by checkLogs}@@ -1160,6 +1202,43 @@ function checkOrphanEvidence(}}+/**+ * `duplicate-entity` / `broken-entity-relation`: canonical entity+ * registry hygiene (Memory Integrity Suite). Duplicates come straight+ * from the index builder's conflict report; broken relations are edges+ * whose target resolves to no entity id in the registry.+ */+function checkEntities(vault: string, issues: DoctorIssue[]): void {+ const index = buildEntityIndex(vault);+ if (index.entities.length === 0 && index.conflicts.length === 0) return;++ for (const conflict of index.conflicts) {+ issues.push({+ severity: "warning",+ code: "duplicate-entity",+ message:+ `${conflict.kind === "duplicate-name" ? "identity" : "alias"} '${conflict.key}' is ` ++ `claimed by ${conflict.paths.length} entity files: ${conflict.paths.join(", ")}. ` ++ "Merge them or archive the duplicates - lookups resolve to the first claimant only.",+ });+ }++ const knownIds = new Set(index.entities.map((e) => e.id));+ for (const entity of index.entities) {+ for (const edge of entity.relations) {+ if (knownIds.has(edge.target)) continue;+ issues.push({+ severity: "warning",+ code: "broken-entity-relation",+ path: entity.path,+ message:+ `${entity.id} declares '${edge.relation}: [[${edge.target}]]' but no entity ` ++ "with that id exists in the registry.",+ });+ }+ }+}+function collectAllBasenames(vault: string): ReadonlySet<string> {const out = new Set<string>();const dirs = brainDirs(vault);diff --git a/src/core/brain/dream.ts b/src/core/brain/dream.tsindex b9113cae..b677cf6f 100644--- a/src/core/brain/dream.ts+++ b/src/core/brain/dream.ts@@ -51,7 +51,8 @@ import { runHealEnrichment } from "./heal-run.ts";import { collectEvidenceForSlug } from "./evidence.ts";import { buildIntentReview, type BrainIntentReviewEntry } from "./intent-review.ts";import { writePreferenceTxn } from "./preference-txn.ts";-import { appendLogEvent, parseLogDay, type BrainLogEntry } from "./log.ts";+import { appendLogEvent, type BrainLogEntry } from "./log.ts";+import { listLogDates, readLogDay } from "./log-jsonl.ts";import { moveToRetired, parsePreference, wouldRewritePreference } from "./preference.ts";import { parseSignal } from "./signal.ts";import { isPinned } from "./pin.ts";@@ -1607,15 +1608,11 @@ interface ApplyEvidenceEntry {}function scanApplyEvidence(vault: string): ApplyEvidenceEntry[] {- const dirs = brainDirs(vault);- if (!existsSync(dirs.log)) return [];const out: ApplyEvidenceEntry[] = [];const mergeAliases = new Map<string, string>();- for (const name of readdirSync(dirs.log)) {- if (!name.endsWith(".md")) continue;- const date = name.slice(0, -3);- if (!/^\d{4}-\d{2}-\d{2}$/.test(date)) continue;- const { entries } = parseLogDay(vault, date);+ // Shard-aware (Memory Integrity Suite): merged per-day reads.+ for (const date of listLogDates(vault)) {+ const { entries } = readLogDay(vault, date);for (const e of entries) {if (e.eventType === BRAIN_LOG_EVENT_KIND.merge) {const keep = parseWikilinkFromBodyValue(e.body["keep"]);diff --git a/src/core/brain/entities/canonical.ts b/src/core/brain/entities/canonical.tsnew file mode 100644index 00000000..6c1ded30--- /dev/null+++ b/src/core/brain/entities/canonical.ts@@ -0,0 +1,38 @@+/**+ * Canonicalization kernel - the one place entity identity is computed.+ *+ * Shared by the registry (duplicate refusal, alias resolution), the+ * doctor lints, search alias expansion, and the fact-extraction router,+ * so every consumer compares like with like. Same normalization shape+ * as `extractEntities` in src/core/search/entities.ts: NFC, lowercase,+ * collapsed whitespace.+ */++/** NFC-normalise, trim, collapse whitespace runs, lowercase. */+export function normalizeEntityName(raw: string): string {+ return raw.normalize("NFC").trim().replace(/\s+/g, " ").toLowerCase();+}++/**+ * Validate an entity category: a lowercase kebab-ish slug with no path+ * separators, traversal, or whitespace. Lowercases the input so+ * `People` and `people` are the same category.+ */+export function validateEntityCategory(raw: string): string {+ const category = raw.normalize("NFC").trim().toLowerCase();+ if (!category) throw new Error("entity category must not be empty");+ if (!/^[a-z0-9][a-z0-9-]*$/.test(category)) {+ throw new Error(+ `entity category must be a lowercase slug ([a-z0-9-], starting alphanumeric): ${JSON.stringify(raw)}`,+ );+ }+ return category;+}++/**+ * The identity key one canonical entity owns: `<category>:<normalized name>`.+ * Two files claiming the same key are duplicates by definition.+ */+export function entityIdentityKey(category: string, name: string): string {+ return `${validateEntityCategory(category)}:${normalizeEntityName(name)}`;+}diff --git a/src/core/brain/entities/index-builder.ts b/src/core/brain/entities/index-builder.tsnew file mode 100644index 00000000..2750d0abBinary files /dev/null and b/src/core/brain/entities/index-builder.ts differdiff --git a/src/core/brain/entities/registry.ts b/src/core/brain/entities/registry.tsnew file mode 100644index 00000000..f5b7e90c--- /dev/null+++ b/src/core/brain/entities/registry.ts@@ -0,0 +1,403 @@+/**+ * Canonical entity registry operations (Memory Integrity Suite).+ *+ * Write-side contract: one canonical entity per `(category, normalized+ * name)`. `upsertEntity` resolves through names AND aliases before+ * creating anything, so duplicates are refused at the write seam;+ * doctor lints catch the ones that arrive by hand-editing or sync.+ * All operations are deterministic - the caller injects the clock.+ *+ * Files stay plain Obsidian Markdown. Every rewrite preserves unknown+ * frontmatter keys the operator may have added by hand.+ */++import { dirname } from "node:path";+import { mkdirSync } from "node:fs";++import type { FrontmatterMap } from "../../types.ts";+import { parseFrontmatter, slugify, writeFrontmatterAtomic } from "../../vault.ts";+import { isKnownRelation, normalizeRelation } from "../../graph/relation-vocab.ts";+import { normalizeRelationTarget } from "../../graph/frontmatter-relations.ts";+import { isoSecond } from "../time.ts";+import { entityPath } from "../paths.ts";+import { entityIdentityKey, normalizeEntityName, validateEntityCategory } from "./canonical.ts";+import { buildEntityIndex, parseEntityFile, type EntityIndex } from "./index-builder.ts";+import {+ BRAIN_ENTITY_ID_PREFIX,+ BRAIN_ENTITY_KIND,+ BRAIN_ENTITY_STATUS,+ type BrainEntity,+ type BrainEntityStatus,+ type EntityRef,+} from "./types.ts";++export interface UpsertEntityInput {+ readonly category: string;+ readonly name: string;+ readonly aliases?: ReadonlyArray<string>;+ /** Agent identity stamped as `source_agent`. */+ readonly agent: string;+ /** Injected clock for deterministic stamps. */+ readonly now: Date;+ /** Optional confidence label passed through verbatim. */+ readonly confidence?: string;+ /** Markdown body (current structured state). Replaces on update. */+ readonly body?: string;+}++export interface UpsertEntityResult {+ readonly entity: BrainEntity;+ readonly created: boolean;+}++export interface ListEntitiesOptions {+ readonly category?: string;+ readonly status?: BrainEntityStatus;+}++export interface RelateEntitiesInput {+ readonly from: EntityRef;+ readonly relation: string;+ readonly to: EntityRef;+ readonly now: Date;+}++export interface ArchiveEntityOptions {+ readonly now: Date;+ /** When true, return an archived entity to active lookup. */+ readonly restore?: boolean;+}++// ----- Lookup ----------------------------------------------------------------++/** A category-less reference matched entities in several categories. */+export class EntityAmbiguityError extends Error {+ constructor(query: string, ids: ReadonlyArray<string>) {+ super(+ `entity reference '${query}' is ambiguous across categories: ${ids.join(", ")} - pass a category`,+ );+ this.name = "EntityAmbiguityError";+ }+}++/** Resolve a ref against ACTIVE entities: canonical name first, then alias. */+function resolveActive(index: EntityIndex, ref: EntityRef): BrainEntity | null {+ const query = normalizeEntityName(ref.query);+ if (!query) return null;+ if (ref.category !== undefined) {+ const category = validateEntityCategory(ref.category);+ const byName = index.byKey.get(`${category}:${query}`);+ if (byName) return byName;+ const byAlias = index.byAlias.get(query);+ return byAlias && byAlias.category === category ? byAlias : null;+ }+ const matches = index.entities.filter(+ (e) =>+ e.status === BRAIN_ENTITY_STATUS.active &&+ (normalizeEntityName(e.name) === query ||+ e.aliases.some((a) => normalizeEntityName(a) === query)),+ );+ if (matches.length > 1) {+ throw new EntityAmbiguityError(+ ref.query,+ matches.map((m) => m.id),+ );+ }+ return matches[0] ?? null;+}++export function getEntity(vault: string, ref: EntityRef): BrainEntity | null {+ return resolveActive(buildEntityIndex(vault), ref);+}++export function listEntities(vault: string, opts: ListEntitiesOptions = {}): BrainEntity[] {+ const category = opts.category !== undefined ? validateEntityCategory(opts.category) : undefined;+ return buildEntityIndex(vault).entities.filter(+ (e) =>+ (category === undefined || e.category === category) &&+ (opts.status === undefined || e.status === opts.status),+ );+}++// ----- Write helpers ---------------------------------------------------------++const ENTITY_FIELD_ORDER = [+ "kind",+ "entity_id",+ "category",+ "name",+ "aliases",+ "status",+ "source_agent",+ "confidence",+ "created_at",+ "updated_at",+ "archived_at",+ "tags",+] as const;++const ENTITY_OWN_FIELDS: ReadonlySet<string> = new Set(ENTITY_FIELD_ORDER);++/**+ * Rewrite an entity file: known fields in canonical order, then the+ * operator's extra frontmatter keys (relations among them) verbatim.+ */+function writeEntityFile(+ path: string,+ fields: FrontmatterMap,+ extras: FrontmatterMap,+ body: string,+ opts: { overwrite: boolean },+): void {+ const meta: FrontmatterMap = {};+ for (const key of ENTITY_FIELD_ORDER) {+ const value = fields[key];+ if (value !== undefined) meta[key] = value;+ }+ for (const [key, value] of Object.entries(extras)) {+ if (!ENTITY_OWN_FIELDS.has(key)) meta[key] = value;+ }+ mkdirSync(dirname(path), { recursive: true });+ writeFrontmatterAtomic(path, meta, body, {+ overwrite: opts.overwrite,+ existsErrorKind: "entity",+ });+}++/** Validate requested aliases against the rest of the registry. */+function checkAliasClaims(+ index: EntityIndex,+ category: string,+ selfId: string | null,+ ownName: string,+ aliases: ReadonlyArray<string>,+): string[] {+ const kept: string[] = [];+ const seen = new Set<string>();+ for (const raw of aliases) {+ const alias = raw.trim();+ const normalized = normalizeEntityName(alias);+ if (!normalized || normalized === normalizeEntityName(ownName)) continue;+ if (seen.has(normalized)) continue;+ seen.add(normalized);+ const aliasHolder = index.byAlias.get(normalized);+ if (aliasHolder && aliasHolder.id !== selfId) {+ throw new Error(+ `alias '${alias}' is already claimed by ${aliasHolder.id} (${aliasHolder.path})`,+ );+ }+ const nameHolder = index.byKey.get(`${category}:${normalized}`);+ if (nameHolder && nameHolder.id !== selfId) {+ throw new Error(+ `alias '${alias}' collides with the canonical name of ${nameHolder.id} (${nameHolder.path})`,+ );+ }+ kept.push(alias);+ }+ return kept;+}++/** Allocate an unused `ent-<category>-<slug>` id (suffix -2, -3, ... on collision). */+function allocateEntityId(index: EntityIndex, category: string, name: string): string {+ const base = `${BRAIN_ENTITY_ID_PREFIX}${category}-${slugify(name)}`;+ const taken = new Set(index.entities.map((e) => e.id));+ if (!taken.has(base)) return base;+ for (let i = 2; ; i++) {+ const candidate = `${base}-${i}`;+ if (!taken.has(candidate)) return candidate;+ }+}++// ----- Operations ------------------------------------------------------------++export function upsertEntity(vault: string, input: UpsertEntityInput): UpsertEntityResult {+ const category = validateEntityCategory(input.category);+ const name = input.name.trim();+ if (!normalizeEntityName(name)) throw new Error("entity name must not be empty");+ const index = buildEntityIndex(vault);+ const stamp = isoSecond(input.now);++ // Resolve to an existing canonical entity: name key first, alias second.+ const key = entityIdentityKey(category, name);+ let target = index.byKey.get(key) ?? null;+ if (target === null) {+ const viaAlias = index.byAlias.get(normalizeEntityName(name));+ if (viaAlias && viaAlias.category === category) target = viaAlias;+ }++ if (target === null) {+ // The name may be held by an archived entity - refuse with the remedy+ // instead of silently forking a second file for the same identity.+ const archivedHolder = index.entities.find(+ (e) =>+ e.status === BRAIN_ENTITY_STATUS.archived && entityIdentityKey(e.category, e.name) === key,+ );+ if (archivedHolder) {+ throw new Error(+ `entity '${name}' (${category}) exists but is archived: ${archivedHolder.id}. ` ++ "Restore it (entity archive --restore) or choose another name.",+ );+ }+ }++ const aliases = checkAliasClaims(+ index,+ category,+ target?.id ?? null,+ target?.name ?? name,+ input.aliases ?? [],+ );++ if (target !== null) {+ const [meta, existingBody] = parseFrontmatter(target.path);+ const mergedAliases = [...target.aliases];+ for (const alias of aliases) {+ if (!mergedAliases.some((a) => normalizeEntityName(a) === normalizeEntityName(alias))) {+ mergedAliases.push(alias);+ }+ }+ const fields: FrontmatterMap = {+ kind: BRAIN_ENTITY_KIND,+ entity_id: target.id,+ category: target.category,+ name: target.name,+ ...(mergedAliases.length > 0 ? { aliases: mergedAliases } : {}),+ status: target.status,+ source_agent: input.agent,+ ...(input.confidence !== undefined+ ? { confidence: input.confidence }+ : target.confidence !== undefined+ ? { confidence: target.confidence }+ : {}),+ created_at: target.created_at,+ updated_at: stamp,+ tags: ["brain", "brain/entity"],+ };+ writeEntityFile(target.path, fields, meta, input.body ?? existingBody, { overwrite: true });+ const entity = parseEntityFile(target.path);+ if (entity === null) throw new Error(`entity file unreadable after write: ${target.path}`);+ return { entity, created: false };+ }++ const id = allocateEntityId(index, category, name);+ const path = entityPath(vault, category, id);+ const fields: FrontmatterMap = {+ kind: BRAIN_ENTITY_KIND,+ entity_id: id,+ category,+ name,+ ...(aliases.length > 0 ? { aliases } : {}),+ status: BRAIN_ENTITY_STATUS.active,+ source_agent: input.agent,+ ...(input.confidence !== undefined ? { confidence: input.confidence } : {}),+ created_at: stamp,+ updated_at: stamp,+ tags: ["brain", "brain/entity"],+ };+ writeEntityFile(path, fields, {}, input.body ?? `# ${name}`, { overwrite: false });+ const entity = parseEntityFile(path);+ if (entity === null) throw new Error(`entity file unreadable after write: ${path}`);+ return { entity, created: true };+}++export function relateEntities(vault: string, input: RelateEntitiesInput): BrainEntity {+ const relation = normalizeRelation(input.relation);+ if (!isKnownRelation(relation)) {+ throw new Error(+ `unknown relation '${input.relation}' - the relation vocabulary is defined in relation-vocab.ts`,+ );+ }+ const index = buildEntityIndex(vault);+ const from = resolveActive(index, input.from);+ if (from === null) throw new Error(`entity not found: ${JSON.stringify(input.from)}`);+ const to = resolveActive(index, input.to);+ if (to === null) throw new Error(`entity not found: ${JSON.stringify(input.to)}`);+ if (from.id === to.id) throw new Error("an entity cannot relate to itself");++ const [meta, body] = parseFrontmatter(from.path);+ const existingRaw = meta[relation];+ const targets: string[] = Array.isArray(existingRaw)+ ? [...existingRaw]+ : typeof existingRaw === "string" && existingRaw.trim().length > 0+ ? [existingRaw]+ : [];+ const alreadyLinked = targets.some((t) => normalizeRelationTarget(String(t)) === to.id);+ if (!alreadyLinked) targets.push(`[[${to.id}]]`);++ const fields: FrontmatterMap = {+ kind: BRAIN_ENTITY_KIND,+ entity_id: from.id,+ category: from.category,+ name: from.name,+ ...(from.aliases.length > 0 ? { aliases: [...from.aliases] } : {}),+ status: from.status,+ ...(from.source_agent !== undefined ? { source_agent: from.source_agent } : {}),+ ...(from.confidence !== undefined ? { confidence: from.confidence } : {}),+ created_at: from.created_at,+ updated_at: isoSecond(input.now),+ tags: ["brain", "brain/entity"],+ };+ const extras: FrontmatterMap = { ...meta, [relation]: targets };+ writeEntityFile(from.path, fields, extras, body, { overwrite: true });+ const entity = parseEntityFile(from.path);+ if (entity === null) throw new Error(`entity file unreadable after write: ${from.path}`);+ return entity;+}++export function archiveEntity(+ vault: string,+ ref: EntityRef,+ opts: ArchiveEntityOptions,+): BrainEntity {+ const index = buildEntityIndex(vault);+ let target: BrainEntity | null;+ if (opts.restore) {+ const query = normalizeEntityName(ref.query);+ const matches = index.entities.filter(+ (e) =>+ e.status === BRAIN_ENTITY_STATUS.archived &&+ (ref.category === undefined || e.category === validateEntityCategory(ref.category)) &&+ (normalizeEntityName(e.name) === query ||+ e.aliases.some((a) => normalizeEntityName(a) === query)),+ );+ if (matches.length > 1) {+ throw new Error(+ `archived entity reference '${ref.query}' is ambiguous: ${matches.map((m) => m.id).join(", ")}`,+ );+ }+ target = matches[0] ?? null;+ } else {+ target = resolveActive(index, ref);+ }+ if (target === null) {+ throw new Error(+ `${opts.restore ? "archived " : ""}entity not found: ${JSON.stringify(ref.query)}`,+ );+ }++ const nextStatus = opts.restore ? BRAIN_ENTITY_STATUS.active : BRAIN_ENTITY_STATUS.archived;+ if (target.status === nextStatus) return target;++ const [meta, body] = parseFrontmatter(target.path);+ const stamp = isoSecond(opts.now);+ const fields: FrontmatterMap = {+ kind: BRAIN_ENTITY_KIND,+ entity_id: target.id,+ category: target.category,+ name: target.name,+ ...(target.aliases.length > 0 ? { aliases: [...target.aliases] } : {}),+ status: nextStatus,+ ...(target.source_agent !== undefined ? { source_agent: target.source_agent } : {}),+ ...(target.confidence !== undefined ? { confidence: target.confidence } : {}),+ created_at: target.created_at,+ updated_at: stamp,+ ...(nextStatus === BRAIN_ENTITY_STATUS.archived ? { archived_at: stamp } : {}),+ tags: ["brain", "brain/entity"],+ };+ // `archived_at` must disappear on restore: writeEntityFile only emits+ // the keys present in `fields`, and extras never override own fields.+ writeEntityFile(target.path, fields, meta, body, { overwrite: true });+ const entity = parseEntityFile(target.path);+ if (entity === null) throw new Error(`entity file unreadable after write: ${target.path}`);+ return entity;+}diff --git a/src/core/brain/entities/types.ts b/src/core/brain/entities/types.tsnew file mode 100644index 00000000..36016493--- /dev/null+++ b/src/core/brain/entities/types.ts@@ -0,0 +1,70 @@+/**+ * Canonical entity registry types (Memory Integrity Suite, design:+ * docs/brainstorm/memory-integrity/design.md).+ *+ * One canonical entity per `(category, normalized name)` lives at+ * `Brain/entities/<category>/<entity-id>.md` as plain Markdown with+ * frontmatter. The Markdown files are the single source of truth; every+ * index over them is rebuilt on read and never persisted.+ */++import type { RelationEdge } from "../../graph/frontmatter-relations.ts";++export const BRAIN_ENTITY_STATUS = {+ active: "active",+ archived: "archived",+} as const;++export type BrainEntityStatus = (typeof BRAIN_ENTITY_STATUS)[keyof typeof BRAIN_ENTITY_STATUS];++const STATUS_VALUES: ReadonlyArray<string> = Object.values(BRAIN_ENTITY_STATUS);++export function isBrainEntityStatus(value: unknown): value is BrainEntityStatus {+ return typeof value === "string" && STATUS_VALUES.includes(value);+}++/** Frontmatter `kind:` marker of an entity file. */+export const BRAIN_ENTITY_KIND = "brain-entity";++/** Filename/id prefix: `ent-<category>-<name-slug>`. */+export const BRAIN_ENTITY_ID_PREFIX = "ent-";++/** A parsed canonical entity. */+export interface BrainEntity {+ /** Stable id, identical to the file basename: `ent-<category>-<slug>`. */+ readonly id: string;+ readonly category: string;+ /** Display name verbatim as the operator wrote it. */+ readonly name: string;+ readonly aliases: ReadonlyArray<string>;+ readonly status: BrainEntityStatus;+ readonly source_agent?: string;+ readonly confidence?: string;+ readonly created_at: string;+ readonly updated_at: string;+ /** Present only while `status` is `archived`. */+ readonly archived_at?: string;+ /** Typed relation edges from the entity's frontmatter (relation-vocab). */+ readonly relations: ReadonlyArray<RelationEdge>;+ /** Absolute file path. */+ readonly path: string;+ /** Markdown body - the entity's current structured state. */+ readonly body: string;+}++export type EntityConflictKind = "duplicate-name" | "duplicate-alias";++/** A duplicate identity claim discovered while building the index. */+export interface EntityConflict {+ readonly kind: EntityConflictKind;+ /** The contested key: `category:name` or the normalized alias. */+ readonly key: string;+ /** Absolute paths of every file claiming the key, in walk order. */+ readonly paths: ReadonlyArray<string>;+}++/** Reference used by lookup verbs: optional category plus name-or-alias. */+export interface EntityRef {+ readonly category?: string;+ readonly query: string;+}diff --git a/src/core/brain/evidence.ts b/src/core/brain/evidence.tsindex 4529454b..59455080 100644--- a/src/core/brain/evidence.ts+++ b/src/core/brain/evidence.ts@@ -8,10 +8,7 @@* Pure read: no I/O outside reading log files.*/-import { existsSync, readdirSync } from "node:fs";--import { brainDirs } from "./paths.ts";-import { parseLogDay } from "./log.ts";+import { listLogDates, readLogDay } from "./log-jsonl.ts";import {BRAIN_APPLY_RESULT,BRAIN_LOG_EVENT_KIND,@@ -32,22 +29,14 @@ function eqId(eventPrefRef: string, slug: string): boolean {/** Sorted list of `YYYY-MM-DD` log days actually on disk. Newest first. */export function listLogDays(vault: string): string[] {- const dir = brainDirs(vault).log;- if (!existsSync(dir)) return [];- let entries: string[];+ // Shard-aware (Memory Integrity Suite): one discovery helper for+ // every log filename shape, legacy and sharded.+ let days: string[];try {- entries = readdirSync(dir);+ days = [...listLogDates(vault)];} catch {return [];}- const days: string[] = [];- for (const name of entries) {- if (!name.endsWith(".md")) continue;- const day = name.slice(0, -3);- if (!/^\d{4}-\d{2}-\d{2}$/.test(day)) continue;- days.push(day);- }- days.sort();days.reverse();return days;}@@ -96,8 +85,8 @@ export function collectEvidenceForSlug(for (const day of listLogDays(vault)) {if (cutoff && day < cutoff.slice(0, 10)) break;- const { entries } = parseLogDay(vault, day);- // parseLogDay returns entries in file order (oldest first). We want+ const { entries } = readLogDay(vault, day);+ // readLogDay returns entries in merged order (oldest first). We want// newest first across the whole vault, so iterate in reverse.for (let i = entries.length - 1; i >= 0; i--) {const e = entries[i]!;diff --git a/src/core/brain/fact-extract.ts b/src/core/brain/fact-extract.tsnew file mode 100644index 00000000..4219fab3Binary files /dev/null and b/src/core/brain/fact-extract.ts differdiff --git a/src/core/brain/log-jsonl.ts b/src/core/brain/log-jsonl.tsindex 262fc172..94280e78 100644--- a/src/core/brain/log-jsonl.ts+++ b/src/core/brain/log-jsonl.ts@@ -1,19 +1,29 @@/**- * Brain log reader (§23, v0.10.8). Prefers the JSONL sidecar written by- * `appendLogEvent`; falls back to parsing the markdown on days without- * a sidecar (pre-v0.10.8 history, or sidecar deleted by hand).+ * Brain log reader (§23, v0.10.8; per-device shards in the Memory+ * Integrity Suite). A day's events live in one or more files:*- * Every machine consumer of `Brain/log/` (discipline-report today,- * future tooling tomorrow) reads through this entry point so the- * fallback logic never duplicates. Malformed JSONL lines surface as+ * - `<date>.jsonl` / `<date>.md` - legacy single pair,+ * treated as the shard with the empty shard id;+ * - `<date>.<deviceId>.jsonl` / `.md` - one pair per device.+ *+ * `readLogDay` merges every shard of a day sorted by+ * (timestamp, shardId, line) - deterministic across devices regardless+ * of Syncthing arrival order. Per shard the JSONL sidecar is preferred+ * and the markdown is the fallback, so pre-v0.10.8 markdown history+ * keeps reading even when new shards exist for the same date.+ *+ * Every machine consumer of `Brain/log/` discovers dates through+ * `listLogDates` and reads through `readLogDay`, so the shard layout+ * lives in exactly one module. Malformed JSONL lines surface as* `warnings` instead of aborting the read, matching `parseLogDay`'s* tolerance contract.*/-import { existsSync, readFileSync } from "node:fs";+import { existsSync, readdirSync, readFileSync } from "node:fs";+import { join } from "node:path";-import { logJsonlPath, logPath, validateIsoDate } from "./paths.ts";-import { parseLogDay, type BrainLogEntry, type BrainLogParseWarning } from "./log.ts";+import { brainDirs, validateIsoDate } from "./paths.ts";+import { parseLogDayFile, type BrainLogEntry, type BrainLogParseWarning } from "./log.ts";import { BRAIN_LOG_EVENT_KIND_SET, type BrainLogEventKind } from "./types.ts";export interface ReadLogDayResult {@@ -37,47 +47,158 @@ export interface ReadLogDayResult {// break the contract of the other surface.const ISO_UTC_TS_RE = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?Z$/;+// One log file name: `<date>.jsonl`, `<date>.md`, or the sharded+// `<date>.<deviceId>.jsonl` / `.md`. Device ids are lowercase slugs+// (see `resolveDeviceId`); Syncthing conflict copies and anything else+// that does not match are NOT log shards.+const LOG_FILE_RE = /^(\d{4}-\d{2}-\d{2})(?:\.([a-z0-9-]{1,32}))?\.(jsonl|md)$/;++interface LogShardFile {+ readonly date: string;+ /** Empty string for the legacy un-sharded pair. */+ readonly shardId: string;+ readonly ext: "jsonl" | "md";+ readonly path: string;+ readonly name: string;+}++/** Every recognised log file under `Brain/log/`, sorted by name. */+function listLogShardFiles(vault: string): LogShardFile[] {+ const dir = brainDirs(vault).log;+ if (!existsSync(dir)) return [];+ const out: LogShardFile[] = [];+ for (const entry of readdirSync(dir, { withFileTypes: true }).toSorted((a, b) =>+ a.name.localeCompare(b.name),+ )) {+ if (!entry.isFile()) continue;+ const m = LOG_FILE_RE.exec(entry.name);+ if (!m) continue;+ const shardId = m[2] ?? "";+ // Defensive: a hand-renamed Syncthing conflict copy could match the+ // shard-id shape; conflict copies are never log shards.+ if (shardId.startsWith("sync-conflict")) continue;+ out.push({+ date: m[1]!,+ shardId,+ ext: m[3] as "jsonl" | "md",+ path: join(dir, entry.name),+ name: entry.name,+ });+ }+ return out;+}++/**+ * Sorted unique dates that have at least one log file (any shard, any+ * extension). The single date-discovery helper for every reader that+ * used to scan the directory itself.+ */+export function listLogDates(vault: string): string[] {+ const dates = new Set<string>();+ for (const f of listLogShardFiles(vault)) dates.add(f.date);+ return [...dates].toSorted();+}++/**+ * `Brain/log/*.sync-conflict-*` copies left behind by Syncthing. The+ * shard layout prevents NEW conflicts; doctor surfaces any leftovers+ * for a manual union+dedup merge.+ */+export function listLogSyncConflicts(vault: string): string[] {+ const dir = brainDirs(vault).log;+ if (!existsSync(dir)) return [];+ return readdirSync(dir, { withFileTypes: true })+ .filter((e) => e.isFile() && e.name.includes(".sync-conflict-"))+ .map((e) => join(dir, e.name))+ .toSorted();+}++/** Markdown log files (legacy + shards) for doctor's per-file lint. */+export function listLogMarkdownFiles(vault: string): Array<{ date: string; path: string }> {+ return listLogShardFiles(vault)+ .filter((f) => f.ext === "md")+ .map((f) => ({ date: f.date, path: f.path }));+}+/**- * Read one day of Brain log events. Picks the JSONL sidecar when it- * exists; otherwise falls back to parsing the markdown. When neither- * file exists, returns an empty result with `source: "jsonl"` (a- * convenient default — the caller treats it as "no events for that- * date" regardless of source).+ * Read one day of Brain log events, merged across every shard sorted+ * by (timestamp, shardId, line). Per shard the JSONL sidecar wins;+ * markdown is the fallback. When no file exists, returns an empty+ * result with `source: "jsonl"` (a convenient default — the caller+ * treats it as "no events for that date" regardless of source).*/export function readLogDay(vault: string, date: string): ReadLogDayResult {const validDate = validateIsoDate(date);- const mdPath = logPath(vault, validDate);- const jsonlPath = logJsonlPath(vault, validDate);+ const shards = listLogShardFiles(vault).filter((f) => f.date === validDate);+ if (shards.length === 0) return { entries: [], source: "jsonl", warnings: [] };- if (existsSync(jsonlPath)) {- return readJsonl(jsonlPath);+ // Group by shard id; per shard prefer .jsonl over .md.+ const byShard = new Map<string, { jsonl?: LogShardFile; md?: LogShardFile }>();+ for (const f of shards) {+ const slot = byShard.get(f.shardId) ?? {};+ slot[f.ext] = f;+ byShard.set(f.shardId, slot);}- if (existsSync(mdPath)) {- const parsed = parseLogDay(vault, validDate);- return {- entries: parsed.entries,- source: "markdown-fallback",- warnings: parsed.warnings,- };++ interface Tagged {+ readonly entry: BrainLogEntry;+ readonly shardId: string;+ readonly line: number;+ }+ const tagged: Tagged[] = [];+ const warnings: BrainLogParseWarning[] = [];+ let usedMarkdown = false;++ for (const shardId of [...byShard.keys()].toSorted()) {+ const slot = byShard.get(shardId)!;+ if (slot.jsonl) {+ const r = readJsonl(slot.jsonl.path);+ r.entries.forEach((entry, i) => tagged.push({ entry, shardId, line: i }));+ warnings.push(...r.warnings);+ continue;+ }+ if (slot.md) {+ usedMarkdown = true;+ const r = parseLogDayFile(vault, validDate, slot.md.path);+ r.entries.forEach((entry, i) => tagged.push({ entry, shardId, line: i }));+ warnings.push(...r.warnings);+ }}- return { entries: [], source: "jsonl", warnings: [] };++ tagged.sort((a, b) => {+ if (a.entry.timestamp !== b.entry.timestamp) {+ return a.entry.timestamp < b.entry.timestamp ? -1 : 1;+ }+ if (a.shardId !== b.shardId) return a.shardId < b.shardId ? -1 : 1;+ return a.line - b.line;+ });++ return {+ entries: tagged.map((t) => t.entry),+ source: usedMarkdown ? "markdown-fallback" : "jsonl",+ warnings,+ };+}++interface ReadJsonlResult {+ readonly entries: ReadonlyArray<BrainLogEntry>;+ readonly warnings: ReadonlyArray<BrainLogParseWarning>;}-function readJsonl(path: string): ReadLogDayResult {+function readJsonl(path: string): ReadJsonlResult {let text: string;try {text = readFileSync(path, "utf8");} catch (err) {- // §23 (v0.10.8): we got here because `existsSync(jsonlPath)`- // returned true, so a subsequent read error is anomalous (race- // with rotation, permission flip, fs-transient) and worth- // surfacing rather than reporting an empty day. Discipline-report- // and any future doctor can flag it without falling back to a- // silently incomplete count.+ // §23 (v0.10.8): we got here because the directory listing showed+ // the file, so a subsequent read error is anomalous (race with+ // rotation, permission flip, fs-transient) and worth surfacing+ // rather than reporting an empty shard. Discipline-report and any+ // future doctor can flag it without falling back to a silently+ // incomplete count.const message = (err as NodeJS.ErrnoException).message ?? String(err);return {entries: [],- source: "jsonl",warnings: [{ path, lineNumber: 0, message: `failed to read JSONL file: ${message}` }],};}@@ -102,7 +223,7 @@ function readJsonl(path: string): ReadLogDayResult {const entry = coerceEntry(parsed, path, i + 1, warnings);if (entry !== null) entries.push(entry);}- return { entries, source: "jsonl", warnings };+ return { entries, warnings };}function coerceEntry(@@ -158,9 +279,13 @@ function coerceEntry(});}}+ // Mirror `parseLogDay`: surface the payload agent at the top level so+ // jsonl- and markdown-sourced entries are interchangeable downstream.+ const agent = typeof body["agent"] === "string" ? body["agent"] : undefined;return {timestamp: ts,eventType: kind as BrainLogEventKind,+ ...(agent !== undefined ? { agent } : {}),body: Object.freeze(body),};}diff --git a/src/core/brain/log.ts b/src/core/brain/log.tsindex b02b8923..9ce17bcf 100644--- a/src/core/brain/log.ts+++ b/src/core/brain/log.ts@@ -30,7 +30,8 @@ import { existsSync, mkdirSync, readFileSync } from "node:fs";import lockfile from "proper-lockfile";import { atomicWriteFileSync } from "../fs-atomic.ts";-import { brainDirs, logJsonlPath, logPath, validateIsoDate } from "./paths.ts";+import { brainDirs, logPath, logShardJsonlPath, logShardPath, validateIsoDate } from "./paths.ts";+import { isValidDeviceId, resolveDeviceId } from "../config.ts";import { BRAIN_LOG_EVENT_KIND, BRAIN_LOG_EVENT_KIND_SET, type BrainLogEventKind } from "./types.ts";// ----- Public types ---------------------------------------------------------@@ -98,7 +99,17 @@ const SUB_BULLET_RE = /^(?:\s{2,}|\t+)-\s+(.+)$/;*/export function parseLogDay(vault: string, date: string): ParseLogDayResult {const validDate = validateIsoDate(date);- const path = logPath(vault, validDate);+ return parseLogDayFile(vault, validDate, logPath(vault, validDate));+}++/**+ * Parse one specific log markdown file for `date`. The per-device+ * shard layout (Memory Integrity Suite) means a day can have several+ * markdown files (`<date>.md`, `<date>.<deviceId>.md`); the shard+ * readers in log-jsonl.ts call this once per file and merge.+ */+export function parseLogDayFile(vault: string, date: string, path: string): ParseLogDayResult {+ const validDate = validateIsoDate(date);if (!existsSync(path)) {return { entries: [], warnings: [] };}@@ -234,7 +245,22 @@ function acquireLogLock(logDir: string): () => void {throw lastErr;}-export function appendLogEvent(vault: string, event: BrainLogEntry): AppendLogEventResult {+export interface AppendLogEventOptions {+ /**+ * Per-device shard id (Memory Integrity Suite). Defaults to+ * `resolveDeviceId()` from the device-local config; the empty string+ * forces the legacy un-sharded `<date>.jsonl` / `<date>.md` pair.+ * Resolution failure (read-only config home, missing HOME) falls back+ * to the legacy pair too - an append must never fail on identity.+ */+ readonly deviceId?: string;+}++export function appendLogEvent(+ vault: string,+ event: BrainLogEntry,+ opts: AppendLogEventOptions = {},+): AppendLogEventResult {if (!BRAIN_LOG_EVENT_KIND_SET.has(event.eventType)) {throw new Error(`appendLogEvent: unknown event kind '${event.eventType}' — must be one of ${Object.values(@@ -242,13 +268,29 @@ export function appendLogEvent(vault: string, event: BrainLogEntry): AppendLogEv).join(", ")}`,);}+ let deviceId: string;+ if (opts.deviceId !== undefined) {+ if (opts.deviceId !== "" && !isValidDeviceId(opts.deviceId)) {+ throw new Error(+ `appendLogEvent: invalid deviceId ${JSON.stringify(opts.deviceId)} - ` ++ "expected a lowercase slug matching the device_id config shape",+ );+ }+ deviceId = opts.deviceId;+ } else {+ try {+ deviceId = resolveDeviceId();+ } catch {+ deviceId = ""; // fail-soft: legacy un-sharded pair+ }+ }// Parse the timestamp once to extract date + HHMMSS deterministically.// We deliberately do not accept Date objects: the caller controls the// canonical representation so two calls with identical input produce// byte-identical output (idempotency requirement).const ts = parseIsoUtc(event.timestamp);- const path = logPath(vault, ts.date);- const jsonlPath = logJsonlPath(vault, ts.date);+ const path = logShardPath(vault, ts.date, deviceId);+ const jsonlPath = logShardJsonlPath(vault, ts.date, deviceId);const logDir = brainDirs(vault).log;const topLevelAgent = (event as { agent?: unknown }).agent;const eventBody =diff --git a/src/core/brain/most-applied.ts b/src/core/brain/most-applied.tsindex 80273769..b134e604 100644--- a/src/core/brain/most-applied.ts+++ b/src/core/brain/most-applied.ts@@ -7,10 +7,10 @@* module never re-walks the preferences directory.*/-import { existsSync, readdirSync } from "node:fs";+import { existsSync } from "node:fs";import { brainDirs } from "./paths.ts";-import { readLogDay } from "./log-jsonl.ts";+import { listLogDates, readLogDay } from "./log-jsonl.ts";import { MOST_APPLIED_LIMIT_DEFAULT, MOST_APPLIED_WINDOW_DAYS_DEFAULT } from "./policy.ts";import { isoDate } from "./time.ts";import { normaliseWikilinkTarget } from "./wikilink.ts";@@ -22,8 +22,6 @@ import {} from "./types.ts";const DAY_MS = 24 * 3600 * 1000;-const LOG_FILENAME_RE = /^(\d{4}-\d{2}-\d{2})\.md$/;-export interface MostAppliedEntry {readonly preference: BrainPreference;readonly applied_30d: number;@@ -81,10 +79,9 @@ export function computeMostApplied(const earliestDayPrefix = isoDate(new Date(windowStartMs - DAY_MS));const counts = new Map<string, number>();- for (const name of readdirSync(dirs.log)) {- const m = LOG_FILENAME_RE.exec(name);- if (!m) continue;- const datePrefix = m[1]!;+ // Shard-aware (Memory Integrity Suite): one discovery pass over every+ // log filename shape; readLogDay merges the day's shards.+ for (const datePrefix of listLogDates(vault)) {if (datePrefix < earliestDayPrefix) continue;let parsed;diff --git a/src/core/brain/paths.ts b/src/core/brain/paths.tsindex 32bf7626..99764ba6 100644--- a/src/core/brain/paths.ts+++ b/src/core/brain/paths.ts@@ -53,6 +53,7 @@ export const BRAIN_PROCEDURES_REL = posix.join(BRAIN_ROOT_REL, "procedures");export const BRAIN_PROCEDURAL_MEMORY_REL = posix.join(BRAIN_ROOT_REL, "procedural-memory");export const BRAIN_ATTENTION_REL = posix.join(BRAIN_ROOT_REL, "attention");export const BRAIN_LOG_REL = posix.join(BRAIN_ROOT_REL, "log");+export const BRAIN_ENTITIES_REL = posix.join(BRAIN_ROOT_REL, "entities");export const BRAIN_SNAPSHOTS_REL = posix.join(BRAIN_ROOT_REL, ".snapshots");/*** Ephemeral MCP tool-result artifacts (v0.18.0). Dot-directory so the@@ -88,6 +89,8 @@ export interface BrainDirs {readonly preferences: string;readonly retired: string;readonly log: string;+ /** Canonical entity registry root: `Brain/entities/<category>/`. */+ readonly entities: string;/** Pre-`dream` archive directory. Never recursed into by `dream`. */readonly snapshots: string;}@@ -106,6 +109,7 @@ export function brainDirs(vault: string): BrainDirs {preferences: ensureInsideVault(join(vault, BRAIN_PREFERENCES_REL), vault),retired: ensureInsideVault(join(vault, BRAIN_RETIRED_REL), vault),log: ensureInsideVault(join(vault, BRAIN_LOG_REL), vault),+ entities: ensureInsideVault(join(vault, BRAIN_ENTITIES_REL), vault),snapshots: ensureInsideVault(join(vault, BRAIN_SNAPSHOTS_REL), vault),};}@@ -239,6 +243,17 @@ export function logPath(vault: string, date: string): string {return ensureInsideVault(join(brainDirs(vault).log, `${d}.md`), vault);}+/**+ * Canonical entity file path: `Brain/entities/<category>/<id>.md`+ * (Memory Integrity Suite). Both segments are slug-validated; the id+ * is the entity's stable identifier and the file basename.+ */+export function entityPath(vault: string, category: string, id: string): string {+ const c = validateSlug(category);+ const i = validateSlug(id);+ return ensureInsideVault(join(brainDirs(vault).entities, c, `${i}.md`), vault);+}+/*** Structured JSONL sidecar that accompanies each `<date>.md` log* file (§23, v0.10.8). Every machine consumer reads through this@@ -251,6 +266,27 @@ export function logJsonlPath(vault: string, date: string): string {return ensureInsideVault(join(brainDirs(vault).log, `${d}.jsonl`), vault);}+/**+ * Per-device markdown log shard: `Brain/log/<date>.<deviceId>.md`+ * (Memory Integrity Suite). The empty device id resolves to the+ * legacy un-sharded path so pre-shard call sites keep working.+ */+export function logShardPath(vault: string, date: string, deviceId: string): string {+ if (deviceId === "") return logPath(vault, date);+ const d = validateIsoDate(date);+ return ensureInsideVault(join(brainDirs(vault).log, `${d}.${validateSlug(deviceId)}.md`), vault);+}++/** Per-device JSONL log shard: `Brain/log/<date>.<deviceId>.jsonl`. */+export function logShardJsonlPath(vault: string, date: string, deviceId: string): string {+ if (deviceId === "") return logJsonlPath(vault, date);+ const d = validateIsoDate(date);+ return ensureInsideVault(+ join(brainDirs(vault).log, `${d}.${validateSlug(deviceId)}.jsonl`),+ vault,+ );+}+/*** Brain Integrity Suite (v0.12.0). Workrun directory used by the* durable dream-pass checkpoint surface: `Brain/log/dream-runs/`.diff --git a/src/core/brain/policy.ts b/src/core/brain/policy.tsindex 08726799..29848992 100644--- a/src/core/brain/policy.ts+++ b/src/core/brain/policy.ts@@ -28,6 +28,7 @@ import type {BrainHealthConfig,BrainLinkGraphConfig,BrainNotesConfig,+ BrainSessionsConfig,BrainSchemaConfig,BrainMostAppliedConfig,BrainTemporalConfig,@@ -37,6 +38,7 @@ import type {ResolvedBrainHealthConfig,ResolvedBrainLinkGraphConfig,ResolvedBrainNotesConfig,+ ResolvedBrainSessionsConfig,ResolvedBrainTemporalConfig,} from "./types.ts";import {@@ -276,6 +278,32 @@ export function resolveNotes(cfg: BrainConfig): ResolvedBrainNotesConfig {}) as ResolvedBrainNotesConfig;}+/**+ * Default `sessions:` block (Memory Integrity Suite). Empty lists+ * mean every session and message is captured - the pre-boundary+ * behaviour, bit-identical.+ */+export const BRAIN_SESSIONS_DEFAULTS: ResolvedBrainSessionsConfig = Object.freeze({+ ignore_patterns: Object.freeze([]) as ReadonlyArray<string>,+ stateless_patterns: Object.freeze([]) as ReadonlyArray<string>,+ ignore_message_patterns: Object.freeze([]) as ReadonlyArray<string>,+}) as ResolvedBrainSessionsConfig;++/** Merge a parsed `sessions` block (or `undefined`) with the defaults. */+export function resolveSessions(cfg: BrainConfig): ResolvedBrainSessionsConfig {+ const block = cfg.sessions;+ if (block === undefined) return BRAIN_SESSIONS_DEFAULTS;+ return Object.freeze({+ ignore_patterns: Object.freeze([...(block.ignore_patterns ?? [])]) as ReadonlyArray<string>,+ stateless_patterns: Object.freeze([+ ...(block.stateless_patterns ?? []),+ ]) as ReadonlyArray<string>,+ ignore_message_patterns: Object.freeze([+ ...(block.ignore_message_patterns ?? []),+ ]) as ReadonlyArray<string>,+ }) as ResolvedBrainSessionsConfig;+}+/*** Load + resolve the `notes:` block, falling back to* `BRAIN_NOTES_DEFAULTS` when the config file is missing, malformed,@@ -1287,6 +1315,58 @@ export function validateBrainConfigDetailed(notes = Object.keys(partialNotes).length > 0 ? (partialNotes as BrainNotesConfig) : {};}+ // Optional `sessions:` block (Memory Integrity Suite). Shape:+ // sessions:+ // ignore_patterns: ["cron-*"]+ // stateless_patterns: ["probe-*"]+ // ignore_message_patterns: ["^\\[heartbeat\\]"]+ // Session patterns are anchored globs; message patterns are regexes+ // (validated lazily at compile time - an invalid regex degrades to a+ // capture-boundary warning, never a config error, so a typo cannot+ // take the whole Brain config down).+ let sessions: BrainSessionsConfig | undefined;+ if ("sessions" in obj) {+ const rawSessions = obj["sessions"];+ if (typeof rawSessions !== "object" || rawSessions === null || Array.isArray(rawSessions)) {+ throw new BrainConfigError("sessions must be a mapping", "sessions", source);+ }+ const sessionsObj = rawSessions as Record<string, unknown>;+ const partial: Record<string, unknown> = {};+ const LIST_KEYS = ["ignore_patterns", "stateless_patterns", "ignore_message_patterns"];+ for (const key of LIST_KEYS) {+ if (!(key in sessionsObj)) continue;+ const v = sessionsObj[key];+ if (!Array.isArray(v)) {+ throw new BrainConfigError(+ "must be an array of pattern strings",+ `sessions.${key}`,+ source,+ );+ }+ const cleaned: string[] = [];+ v.forEach((entry, idx) => {+ if (typeof entry !== "string" || entry.trim().length === 0) {+ throw new BrainConfigError(+ `must be a non-empty string; got ${describe(entry)}`,+ `sessions.${key}[${idx}]`,+ source,+ );+ }+ cleaned.push(entry.trim());+ });+ partial[key] = cleaned;+ }+ for (const key of Object.keys(sessionsObj)) {+ if (!LIST_KEYS.includes(key)) {+ warnings.push({+ path: source ?? "<config>",+ message: `sessions.${key}: unknown field ignored (forward-compat)`,+ });+ }+ }+ sessions = partial as BrainSessionsConfig;+ }+// Optional `schema:` block (runtime schema-pack foundation). Shape:// schema:// preference_types: [research, decision]@@ -1403,6 +1483,7 @@ export function validateBrainConfigDetailed("link_graph","temporal","notes",+ "sessions","schema",]);for (const key of Object.keys(obj)) {@@ -1444,6 +1525,7 @@ export function validateBrainConfigDetailed(...(linkGraph !== undefined ? { link_graph: linkGraph } : {}),...(temporal !== undefined ? { temporal } : {}),...(notes !== undefined ? { notes } : {}),+ ...(sessions !== undefined ? { sessions } : {}),...(health !== undefined ? { health } : {}),...(schema !== undefined ? { schema } : {}),};diff --git a/src/core/brain/query.ts b/src/core/brain/query.tsindex 0b561396..e6fa4652 100644--- a/src/core/brain/query.ts+++ b/src/core/brain/query.ts@@ -30,7 +30,8 @@ import { existsSync, readdirSync } from "node:fs";import { join } from "node:path";import { brainDirs } from "./paths.ts";-import { parseLogDay, type BrainLogEntry } from "./log.ts";+import type { BrainLogEntry } from "./log.ts";+import { listLogDates, readLogDay } from "./log-jsonl.ts";import { parsePreference, parseRetired } from "./preference.ts";import { parseSignal } from "./signal.ts";import { normaliseWikilinkTarget } from "./wikilink.ts";@@ -263,16 +264,11 @@ export function queryByLogSince(vault: string, since: Date): ReadonlyArray<Brain* → empty array.*/export function readAllLogEntries(vault: string): BrainLogEntry[] {- const dirs = brainDirs(vault);- if (!existsSync(dirs.log)) return [];- const names = readdirSync(dirs.log, { withFileTypes: true })- .filter((d) => d.isFile() && d.name.endsWith(".md"))- .map((d) => d.name.slice(0, -".md".length))- .filter((n) => /^\d{4}-\d{2}-\d{2}$/.test(n))- .toSorted();+ // Shard-aware (Memory Integrity Suite): dates from the single+ // discovery helper, entries merged across device shards.const out: BrainLogEntry[] = [];- for (const date of names) {- const { entries } = parseLogDay(vault, date);+ for (const date of listLogDates(vault)) {+ const { entries } = readLogDay(vault, date);for (const e of entries) out.push(e);}return out;diff --git a/src/core/brain/session-lifecycle.ts b/src/core/brain/session-lifecycle.tsindex 435edb7e..1d6fb75d 100644--- a/src/core/brain/session-lifecycle.ts+++ b/src/core/brain/session-lifecycle.ts@@ -3,6 +3,8 @@ import { join } from "node:path";import { appendAuditRecord } from "../reliability/audit.ts";import { appendLogEvent } from "./log.ts";import { brainDirs } from "./paths.ts";+import { buildCaptureBoundary, type SessionCaptureDecision } from "./capture-boundary.ts";+import { extractFacts, routeExtractedFacts } from "./fact-extract.ts";import { buildDedupIndex, computeDedupHash, type DedupIndexEntry } from "./dedup-hash.ts";import { discoverMarkersDetailed } from "./inline.ts";import { writeSignal } from "./signal.ts";@@ -23,6 +25,12 @@ export interface CaptureSessionLifecycleResult {readonly signals_deduped: number;readonly tool_replays: number;readonly malformed: number;+ /** Capture-boundary verdict for this event (Memory Integrity Suite). */+ readonly boundary_decision: SessionCaptureDecision;+ /** Messages whose text was suppressed before any extraction. */+ readonly suppressed_messages: number;+ readonly facts_extracted: number;+ readonly facts_deduped: number;readonly audit_path: string;readonly log_path?: string;}@@ -30,6 +38,7 @@ export interface CaptureSessionLifecycleResult {interface NormalizedPayload {readonly event: string;readonly sessionId?: string;+ readonly transcriptPath?: string;readonly promptText?: string;readonly toolName?: string;readonly toolInput?: unknown;@@ -53,18 +62,47 @@ export async function captureSessionLifecycleEvent(signals_deduped: 0,tool_replays: 0,malformed: normalized.malformed,+ suppressed_messages: 0,+ facts_extracted: 0,+ facts_deduped: 0,};- if (normalized.promptText !== undefined) {- captureMarkers(vault, normalized, normalized.promptText, opts, now, ensureDedup(), counters);+ // Capture boundary (Memory Integrity Suite): classify FIRST, before+ // any extraction. Ignored sessions produce nothing but the audit+ // row; stateless sessions read but never write; suppressed message+ // text never reaches marker or fact extraction.+ const boundary = buildCaptureBoundary(vault);+ const decision = boundary.sessionDecision(normalized.sessionId, normalized.transcriptPath);+ const mayWrite = decision === "capture";++ let promptText = normalized.promptText;+ if (mayWrite && promptText !== undefined && boundary.suppressMessage(promptText)) {+ counters.suppressed_messages++;+ promptText = undefined;+ }++ if (mayWrite && promptText !== undefined) {+ captureMarkers(vault, normalized, promptText, opts, now, ensureDedup(), counters);+ // Fact extraction runs strictly AFTER the boundary: only captured,+ // unsuppressed user text reaches the pattern table.+ const routed = routeExtractedFacts(vault, {+ facts: extractFacts(promptText),+ agent: opts.agent,+ now,+ sessionRef: sessionReference(normalized),+ dedup: ensureDedup(),+ ...(opts.dryRun ? { dryRun: true } : {}),+ });+ counters.facts_extracted += routed.created;+ counters.facts_deduped += routed.deduped;}- if (normalized.toolName === "brain_feedback") {+ if (mayWrite && normalized.toolName === "brain_feedback") {captureToolFeedback(vault, normalized, opts, now, ensureDedup(), counters);}let logPath: string | undefined;- if (!opts.dryRun) {+ if (mayWrite && !opts.dryRun) {logPath = appendLifecycleLog(vault, normalized, opts.agent, now, counters);}@@ -78,6 +116,7 @@ export async function captureSessionLifecycleEvent(event: normalized.event,...(normalized.sessionId ? { session_id: normalized.sessionId } : {}),dry_run: opts.dryRun === true,+ boundary_decision: decision,...counters,},});@@ -89,6 +128,10 @@ export async function captureSessionLifecycleEvent(signals_deduped: counters.signals_deduped,tool_replays: counters.tool_replays,malformed: counters.malformed,+ boundary_decision: decision,+ suppressed_messages: counters.suppressed_messages,+ facts_extracted: counters.facts_extracted,+ facts_deduped: counters.facts_deduped,audit_path: auditPath,...(logPath ? { log_path: logPath } : {}),};@@ -104,9 +147,11 @@ function normalizePayload(payload: unknown): NormalizedPayload {readNonEmptyString(record["event"]) ??"unknown";const sessionId = readNonEmptyString(record["session_id"]);+ const transcriptPath = readNonEmptyString(record["transcript_path"]);return {event,...(sessionId ? { sessionId } : {}),+ ...(transcriptPath ? { transcriptPath } : {}),...(extractPromptText(record) ? { promptText: extractPromptText(record)! } : {}),...(readNonEmptyString(record["tool_name"])? { toolName: readNonEmptyString(record["tool_name"])! }@@ -205,6 +250,9 @@ interface MutableCounters {signals_deduped: number;tool_replays: number;malformed: number;+ suppressed_messages: number;+ facts_extracted: number;+ facts_deduped: number;}function emitSignal(@@ -259,6 +307,9 @@ function appendLifecycleLog(signals_deduped: String(counters.signals_deduped),tool_replays: String(counters.tool_replays),malformed: String(counters.malformed),+ suppressed_messages: String(counters.suppressed_messages),+ facts_extracted: String(counters.facts_extracted),+ facts_deduped: String(counters.facts_deduped),},}).logPath;}diff --git a/src/core/brain/sessions/import.ts b/src/core/brain/sessions/import.tsindex 16a1336b..2c1f60f5 100644--- a/src/core/brain/sessions/import.ts+++ b/src/core/brain/sessions/import.ts@@ -42,6 +42,8 @@ import {type SessionTurn,} from "./types.ts";import { validateBrainFeedbackInput } from "./validate-feedback.ts";+import { buildCaptureBoundary, type SessionCaptureDecision } from "../capture-boundary.ts";+import { extractFacts, routeExtractedFacts } from "../fact-extract.ts";export interface ImportSessionOptions {/** Agent identity stamped on signals when the turn has no own agent. */@@ -88,6 +90,12 @@ export interface ImportSessionResult {readonly tool_replays: number;readonly malformed: number;readonly filtered_turns: number;+ /** Capture-boundary verdict for this file (Memory Integrity Suite). */+ readonly boundary_decision: SessionCaptureDecision;+ /** Turns whose text was suppressed before any extraction. */+ readonly suppressed_turns: number;+ readonly facts_extracted: number;+ readonly facts_deduped: number;readonly recall_turns_imported: number;readonly recall_summary_nodes: number;readonly errors: ReadonlyArray<{ path: string; message: string }>;@@ -148,6 +156,9 @@ export async function importSession(let toolReplays = 0;let malformed = 0;let filteredTurns = 0;+ let suppressedTurns = 0;+ let factsExtracted = 0;+ let factsDeduped = 0;const recallTurns: SessionTurn[] = [];const filterRoles =opts.filterRoles && opts.filterRoles.length > 0 ? new Set(opts.filterRoles) : null;@@ -209,6 +220,32 @@ export async function importSession(}};+ // Capture boundary (Memory Integrity Suite): classify the session+ // FIRST. An ignored file imports nothing; a stateless file is+ // scanned read-only (no signals, no recall records, no facts).+ const boundary = buildCaptureBoundary(vault);+ const boundaryDecision = boundary.sessionDecision(opts.recallSessionId, absPath);+ const mayWrite = boundaryDecision === "capture";+ const emptyResult = (): ImportSessionResult =>+ Object.freeze({+ file: absPath,+ format: adapter.id,+ turns_scanned: 0,+ signals_created: 0,+ signals_deduped: 0,+ tool_replays: 0,+ malformed: 0,+ filtered_turns: 0,+ boundary_decision: boundaryDecision,+ suppressed_turns: 0,+ facts_extracted: 0,+ facts_deduped: 0,+ recall_turns_imported: 0,+ recall_summary_nodes: 0,+ errors: Object.freeze([]),+ });+ if (boundaryDecision === "ignore") return emptyResult();+for await (const turn of adapter.iterate(path)) {turnsScanned++;if (sinceMs !== undefined) {@@ -226,10 +263,31 @@ export async function importSession(continue;}}- if (opts.recall === true) recallTurns.push(turn);+ // Message-level boundary: suppressed text never reaches marker or+ // fact extraction (and is not stored for recall).+ if (turn.text && boundary.suppressMessage(turn.text)) {+ suppressedTurns++;+ continue;+ }+ if (opts.recall === true && mayWrite) recallTurns.push(turn);++ // Facts from USER turns only (the HANDOFF carve-out's conservative+ // core: bare assistant output is never auto-extracted).+ if (mayWrite && turn.text && turn.role === "user") {+ const routed = routeExtractedFacts(vault, {+ facts: extractFacts(turn.text),+ agent: agentLabelForTurn(turn, adapter.id, opts.agent),+ now,+ sessionRef: `${absPath}#${turn.turnId}`,+ dedup,+ ...(opts.dryRun === true ? { dryRun: true } : {}),+ });+ factsExtracted += routed.created;+ factsDeduped += routed.deduped;+ }// Path A — markers in text.- if (turn.text && (turn.role === "user" || turn.role === "assistant")) {+ if (mayWrite && turn.text && (turn.role === "user" || turn.role === "assistant")) {const discovery = discoverMarkersDetailed(turn.text);malformed += discovery.malformed;const markers = discovery.markers;@@ -254,6 +312,7 @@ export async function importSession(}// Path B — brain_feedback tool_use replay.+ if (!mayWrite) continue;for (const call of turn.toolCalls ?? []) {if (call.name !== "brain_feedback") continue;const validated = validateBrainFeedbackInput(call.input);@@ -312,6 +371,10 @@ export async function importSession(signals_deduped: signalsDeduped,tool_replays: toolReplays,malformed,+ boundary_decision: boundaryDecision,+ suppressed_turns: suppressedTurns,+ facts_extracted: factsExtracted,+ facts_deduped: factsDeduped,filtered_turns: filteredTurns,recall_turns_imported: recallTurnsImported,recall_summary_nodes: recallSummaryNodes,diff --git a/src/core/brain/status.ts b/src/core/brain/status.tsindex 93797897..31ca84d2 100644--- a/src/core/brain/status.ts+++ b/src/core/brain/status.ts@@ -21,7 +21,7 @@ import { join } from "node:path";import { parseFrontmatter } from "../vault.ts";import { brainDirs } from "./paths.ts";-import { parseLogDay } from "./log.ts";+import { listLogDates, readLogDay } from "./log-jsonl.ts";import { loadBrainConfig } from "./policy.ts";import { parseSignal } from "./signal.ts";import { BRAIN_LOG_EVENT_KIND, BRAIN_PREFERENCE_STATUS } from "./types.ts";@@ -105,7 +105,8 @@ function countArtifacts(vault: string): BrainStatusCounts {const inbox = countMd(dirs.inbox);const inbox_processed = countMd(dirs.processed);const retired = countMd(dirs.retired);- const log_days = countMd(dirs.log);+ // Shard-aware: count distinct DAYS, not files (several shards share a day).+ const log_days = listLogDates(vault).length;const snapshots = countZst(dirs.snapshots);// Per-status preference counts: read frontmatter line `status:` only@@ -179,11 +180,7 @@ function scanLogTimestamps(vault: string): {} {const dirs = brainDirs(vault);if (!existsSync(dirs.log)) return { lastDreamAt: null, lastApplyEvidenceAt: null };- const days = readdirSync(dirs.log)- .filter((n) => n.endsWith(".md") && /^\d{4}-\d{2}-\d{2}\.md$/.test(n))- .map((n) => n.slice(0, -".md".length))- .toSorted()- .toReversed(); // newest day first+ const days = listLogDates(vault).toReversed(); // newest day firstlet lastDreamAt: string | null = null;let lastApplyEvidenceAt: string | null = null;@@ -192,7 +189,7 @@ function scanLogTimestamps(vault: string): {// both are set we can stop scanning older files.let entries;try {- entries = parseLogDay(vault, date).entries;+ entries = readLogDay(vault, date).entries;} catch {continue;}diff --git a/src/core/brain/temporal/build-index.ts b/src/core/brain/temporal/build-index.tsindex ab4b7c2c..2de65064 100644--- a/src/core/brain/temporal/build-index.ts+++ b/src/core/brain/temporal/build-index.ts@@ -19,7 +19,7 @@ import { existsSync, readdirSync } from "node:fs";import { join, relative } from "node:path";import type { BrainLogEntry } from "./../log.ts";-import { readLogDay } from "./../log-jsonl.ts";+import { listLogDates, readLogDay } from "./../log-jsonl.ts";import { brainDirs } from "./../paths.ts";import { parseFrontmatter } from "../../vault.ts";import { parseWikilink } from "./../wikilink.ts";@@ -171,13 +171,9 @@ function collectLogEvents(vault: string, window: TimelineWindow, out: TemporalEv// same convention `digest.ts:readLogsInWindow` uses).const lowerBound = addDays(sinceDay, -1);const upperBound = addDays(untilDay, 1);- const dates = new Set<string>();- for (const entry of readdirSync(logDir, { withFileTypes: true })) {- if (!entry.isFile()) continue;- const base = stripKnownLogExtension(entry.name);- if (base !== null && ISO_DATE_ONLY_RE.test(base)) dates.add(base);- }- const sortedDates = [...dates].toSorted();+ // Shard-aware (Memory Integrity Suite): listLogDates recognises the+ // legacy pair AND `<date>.<deviceId>.jsonl` / `.md` shard names.+ const sortedDates = listLogDates(vault);for (const date of sortedDates) {if (date < lowerBound) continue;if (date > upperBound) continue;@@ -201,12 +197,6 @@ function collectLogEvents(vault: string, window: TimelineWindow, out: TemporalEv}}-function stripKnownLogExtension(name: string): string | null {- if (name.endsWith(".jsonl")) return name.slice(0, -".jsonl".length);- if (name.endsWith(".md")) return name.slice(0, -".md".length);- return null;-}-/*** Turn one `BrainLogEntry` into one `TemporalEvent`. Optional slots* (`prefId`, `topic`, `result`, `artifact`, `reason`, `text`) comediff --git a/src/core/brain/types.ts b/src/core/brain/types.tsindex dfd576e1..7bfea45b 100644--- a/src/core/brain/types.ts+++ b/src/core/brain/types.ts@@ -39,6 +39,8 @@ export const BRAIN_SIGNAL_SOURCE_TYPE = {live: "live",inline: "inline",session: "session",+ /** Regex fact extraction (Memory Integrity Suite). */+ extracted: "extracted",} as const;export type BrainSignalSourceType =(typeof BRAIN_SIGNAL_SOURCE_TYPE)[keyof typeof BRAIN_SIGNAL_SOURCE_TYPE];@@ -996,6 +998,12 @@ export interface BrainConfig {* to these paths - the type is `read_paths` for a reason.*/readonly notes?: BrainNotesConfig;+ /**+ * Optional `sessions:` block (Memory Integrity Suite). Capture+ * boundaries for session/message ingestion. Absent: every session+ * is captured - bit-identical to pre-boundary behaviour.+ */+ readonly sessions?: BrainSessionsConfig;/*** Optional `health:` block (v0.14.0). Tunes the semantic-health* detectors and the remediation step cap. Absent: callers fall back@@ -1028,6 +1036,25 @@ export interface ResolvedBrainNotesConfig {readonly read_paths: ReadonlyArray<string>;}+/**+ * Optional `sessions:` block (Memory Integrity Suite). Capture+ * boundaries for runtime session ingestion: ignored sessions produce+ * nothing, stateless sessions read but never write, and suppressed+ * messages never become Brain evidence. Session patterns are anchored+ * globs (`*`, `?`); message patterns are regexes.+ */+export interface BrainSessionsConfig {+ readonly ignore_patterns?: ReadonlyArray<string>;+ readonly stateless_patterns?: ReadonlyArray<string>;+ readonly ignore_message_patterns?: ReadonlyArray<string>;+}++export interface ResolvedBrainSessionsConfig {+ readonly ignore_patterns: ReadonlyArray<string>;+ readonly stateless_patterns: ReadonlyArray<string>;+ readonly ignore_message_patterns: ReadonlyArray<string>;+}+/*** Concrete (fully-resolved) guardrail config. Returned by* `resolveGuardrails(cfg)` so consumers do not have to handlediff --git a/src/core/config.ts b/src/core/config.tsindex 78c88aa7..a0eb04d2 100644--- a/src/core/config.ts+++ b/src/core/config.ts@@ -7,9 +7,12 @@* via parallel suites in tests/core/config.test.ts.*/-import { readFileSync } from "node:fs";+import { mkdirSync, readFileSync } from "node:fs";+import { randomBytes } from "node:crypto";import { homedir } from "node:os";-import { join } from "node:path";+import { dirname, join } from "node:path";++import lockfile from "proper-lockfile";import { atomicWriteFileSync } from "./fs-atomic.ts";import { isFile } from "./fs-utils.ts";@@ -179,6 +182,70 @@ export function resolveAgentName(configPath?: string): string {return "agent";}+/**+ * Stable per-install device identity (Memory Integrity Suite). Keys the+ * per-device Brain log shards (`Brain/log/<date>.<deviceId>.jsonl`), so+ * it MUST live in the device-local config and never in the synced+ * vault - all devices sharing one id would defeat the sharding.+ *+ * Generated once (8 hex chars) on first use and persisted. An invalid+ * hand-edited value self-heals to a fresh generated id; the+ * `sync-conflict` prefix is reserved so a renamed Syncthing conflict+ * copy can never masquerade as a shard.+ */+export const DEVICE_ID_RE = /^[a-z0-9][a-z0-9-]{0,31}$/;++export function isValidDeviceId(value: string): boolean {+ return DEVICE_ID_RE.test(value) && !value.startsWith("sync-conflict");+}++export function resolveDeviceId(configPath?: string): string {+ // Env override: a valid id wins outright; the empty string opts out+ // of sharding (legacy single-file log pair). The test preload pins+ // this to "" so the suite stays deterministic; an invalid value+ // falls through to config resolution.+ const env = process.env["O2B_DEVICE_ID"];+ if (env !== undefined && (env === "" || isValidDeviceId(env))) return env;+ const resolved = configPath ?? defaultConfigPath();++ const read = (): string | null => {+ const value = discoverConfig(resolved).data["device_id"];+ return value && isValidDeviceId(value) ? value : null;+ };++ const existing = read();+ if (existing !== null) return existing;++ // First-use generation. Two processes racing here could each persist+ // a different id and split one device's logs across two shards, so+ // the read-generate-write sequence holds a directory lock (same+ // bounded-retry shape as the log writer's `acquireLogLock`). A+ // lock failure (read-only config home, exotic fs) falls through to+ // the unlocked path - identity resolution must never fail.+ const dir = dirname(resolved);+ mkdirSync(dir, { recursive: true });+ let release: (() => void) | undefined;+ try {+ for (let attempt = 0; attempt < 10; attempt++) {+ try {+ release = lockfile.lockSync(dir, { stale: 10_000, realpath: false });+ break;+ } catch (err) {+ if ((err as NodeJS.ErrnoException).code !== "ELOCKED") break;+ Bun.sleepSync(50);+ }+ }+ // Re-check under the lock: the racing process may have just won.+ const won = read();+ if (won !== null) return won;+ const generated = randomBytes(4).toString("hex");+ setConfigValue("device_id", generated, resolved);+ return generated;+ } finally {+ release?.();+ }+}+export function resolveLinkOutputFormat(configPath?: string): LinkOutputFormat {const env = process.env["OBSIDIAN_LINK_FORMAT"]?.trim();const data = env ? {} : discoverConfig(configPath).data;diff --git a/src/core/search/entity-alias.ts b/src/core/search/entity-alias.tsnew file mode 100644index 00000000..719a6bb4--- /dev/null+++ b/src/core/search/entity-alias.ts@@ -0,0 +1,76 @@+/**+ * Registry-aware query entity expansion (Memory Integrity Suite).+ *+ * `extractEntities` pulls entity strings out of the raw query; this+ * module folds the canonical entity registry over that set so a query+ * naming an ALIAS also matches chunks naming the CANONICAL entity and+ * vice versa. Expansion is additive and fail-soft: no registry (or an+ * unreadable one) leaves the query entity set untouched, so vaults+ * without `Brain/entities/` rank bit-identically to pre-registry+ * behaviour.+ */++import { buildEntityIndex } from "../brain/entities/index-builder.ts";+import { normalizeEntityName } from "../brain/entities/canonical.ts";+import { BRAIN_ENTITY_STATUS } from "../brain/entities/types.ts";++export interface QueryEntityExpansion {+ /** Original query entities plus every form added by the registry. */+ readonly expanded: ReadonlyArray<string>;+ /** Only the forms the registry added (for canonical-hop attribution). */+ readonly added: ReadonlyArray<string>;+ /** Ids of the canonical entities whose forms were added, sorted. */+ readonly sourceIds: ReadonlyArray<string>;+}++/**+ * Expand query entities with canonical names and aliases from the+ * registry. A registry entity participates when ANY of its forms+ * (name or alias) appears among the query entities; all its other+ * forms are then added.+ */+export function expandQueryEntities(+ vault: string,+ queryEntities: ReadonlyArray<string>,+): QueryEntityExpansion {+ const identity: QueryEntityExpansion = Object.freeze({+ expanded: queryEntities,+ added: Object.freeze([] as string[]),+ sourceIds: Object.freeze([] as string[]),+ });+ if (queryEntities.length === 0) return identity;++ let index;+ try {+ index = buildEntityIndex(vault);+ } catch {+ return identity; // fail-soft: search never breaks on a bad registry+ }+ if (index.entities.length === 0) return identity;++ const querySet = new Set(queryEntities.map((e) => normalizeEntityName(e)));+ const added: string[] = [];+ const addedSet = new Set<string>();+ const sourceIds: string[] = [];++ for (const entity of index.entities) {+ if (entity.status !== BRAIN_ENTITY_STATUS.active) continue;+ const forms = [entity.name, ...entity.aliases].map((f) => normalizeEntityName(f));+ if (!forms.some((f) => querySet.has(f))) continue;+ let contributed = false;+ for (const form of forms) {+ if (querySet.has(form) || addedSet.has(form) || form.length < 2) continue;+ addedSet.add(form);+ added.push(form);+ contributed = true;+ }+ if (contributed) sourceIds.push(entity.id);+ }++ if (added.length === 0) return identity;+ return Object.freeze({+ expanded: Object.freeze([...queryEntities, ...added]),+ added: Object.freeze(added),+ sourceIds: Object.freeze(sourceIds.toSorted()),+ });+}diff --git a/src/core/search/search.ts b/src/core/search/search.tsindex 613da30c..d7200f3e 100644--- a/src/core/search/search.ts+++ b/src/core/search/search.ts@@ -22,6 +22,7 @@ import {readLearnedWeights,} from "./feedback.ts";import { extractEntities } from "./entities.ts";+import { expandQueryEntities } from "./entity-alias.ts";import { buildCoverageReport, significantTerms, termIncludedIn } from "./coverage.ts";import { buildEvidencePack, downrankTerminalEvidenceResults } from "./evidence-pack.ts";import type { EvidenceUnionRecord, EvidenceVerification } from "./evidence-pack.ts";@@ -369,9 +370,22 @@ export async function search(// Entity-boosted retrieval (v0.13.0): extract entities from the// query and count overlaps with each candidate chunk. Empty when the// query names no entities or the index predates the entity store.+ // The canonical entity registry (Memory Integrity Suite) expands the+ // set so a query naming an alias also matches chunks naming the+ // canonical entity; identity expansion (no registry) keeps ranking+ // bit-identical to pre-registry behaviour.const queryEntities = extractEntities(query);+ const entityExpansion = expandQueryEntities(config.vault, queryEntities);const entityMatchByChunk =- queryEntities.length > 0 ? store.chunkEntityMatches(idsList, queryEntities) : undefined;+ entityExpansion.expanded.length > 0+ ? store.chunkEntityMatches(idsList, entityExpansion.expanded)+ : undefined;+ // Canonical-hop attribution: chunks matching a registry-ADDED form+ // carry an explicit reason naming the canonical entity ids below.+ const canonicalMatchByChunk =+ entityExpansion.added.length > 0+ ? store.chunkEntityMatches(idsList, entityExpansion.added)+ : undefined;// When a property filter is active, overfetch the ranked// candidates so the post-filter result set still has a chance@@ -514,10 +528,27 @@ export async function search(return rels && rels.length > 0 ? { ...r, relations: Object.freeze(rels) } : r;});const withStructuredReasons = addStructuredReasons(withRelations, structured);+ // Canonical-entity attribution (Memory Integrity Suite): a hit whose+ // chunk matched a registry-added form explains the alias hop. Vaults+ // without a registry never reach this branch.+ const withCanonicalReasons =+ canonicalMatchByChunk !== undefined+ ? withStructuredReasons.map((r) =>+ (canonicalMatchByChunk.get(r.chunkId) ?? 0) > 0+ ? Object.freeze({+ ...r,+ reasons: Object.freeze([+ ...r.reasons,+ `entity_canonical: ${entityExpansion.sourceIds.join(", ")}`,+ ]),+ })+ : r,+ )+ : withStructuredReasons;const finalResults =opts.evidencePack === true- ? downrankTerminalEvidenceResults(withStructuredReasons)- : withStructuredReasons;+ ? downrankTerminalEvidenceResults(withCanonicalReasons)+ : withCanonicalReasons;const evidencePack =opts.evidencePack === true? buildEvidencePack(diff --git a/src/mcp/brain-tools.ts b/src/mcp/brain-tools.tsindex c38d7b93..f55e3f10 100644--- a/src/mcp/brain-tools.ts+++ b/src/mcp/brain-tools.ts@@ -50,6 +50,8 @@ import {type AppendApplyEvidenceInput,} from "../core/brain/apply-evidence.ts";import { buildBacklinkIndex } from "../core/brain/backlinks.ts";+import { EntityAmbiguityError, getEntity, listEntities } from "../core/brain/entities/registry.ts";+import { validateEntityCategory } from "../core/brain/entities/canonical.ts";import { readPrefAudit } from "../core/brain/pref-audit.ts";import { buildMorningBrief } from "../core/brain/morning-brief.ts";import { aggregateSources } from "../core/brain/portability/sources.ts";@@ -910,6 +912,76 @@ function coerceAgentDiffMode(// ----- brain_backlinks -----------------------------------------------------+/** Serialize one entity for the MCP payload (no absolute paths). */+function entityToPayload(+ entity: import("../core/brain/entities/types.ts").BrainEntity,+): Record<string, unknown> {+ return {+ id: entity.id,+ category: entity.category,+ name: entity.name,+ aliases: [...entity.aliases],+ status: entity.status,+ ...(entity.source_agent !== undefined ? { source_agent: entity.source_agent } : {}),+ ...(entity.confidence !== undefined ? { confidence: entity.confidence } : {}),+ created_at: entity.created_at,+ updated_at: entity.updated_at,+ ...(entity.archived_at !== undefined ? { archived_at: entity.archived_at } : {}),+ relations: entity.relations.map((r) => ({ relation: r.relation, target: r.target })),+ body: entity.body,+ };+}++async function toolBrainEntity(+ ctx: ServerContext,+ args: Record<string, unknown>,+): Promise<Record<string, unknown>> {+ // Argument validation happens at the boundary so INVALID_PARAMS is+ // reserved for genuine client-input errors; unexpected registry+ // failures (malformed entity file, I/O) surface as INTERNAL_ERROR.+ const view = coerceStr(args, "view", true)!;+ if (view !== "get" && view !== "list") {+ throw new MCPError(INVALID_PARAMS, `brain_entity: unknown view '${view}' (get | list)`);+ }+ const category = coerceStr(args, "category", false);+ if (category !== null) {+ try {+ validateEntityCategory(category);+ } catch (err) {+ throw new MCPError(INVALID_PARAMS, `brain_entity: ${(err as Error).message}`);+ }+ }+ const status = coerceStr(args, "status", false);+ if (status !== null && status !== "active" && status !== "archived") {+ throw new MCPError(INVALID_PARAMS, "brain_entity: status must be 'active' or 'archived'");+ }+ const query = view === "get" ? coerceStr(args, "query", true)! : null;++ try {+ if (view === "get") {+ const entity = getEntity(ctx.vault, {+ query: query!,+ ...(category !== null ? { category } : {}),+ });+ if (entity === null) return { found: false, query: query! };+ return { found: true, ...entityToPayload(entity) };+ }+ const entities = listEntities(ctx.vault, {+ ...(category !== null ? { category } : {}),+ ...(status !== null ? { status } : {}),+ });+ return { entities: entities.map(entityToPayload), total: entities.length };+ } catch (err) {+ if (err instanceof MCPError) throw err;+ if (err instanceof EntityAmbiguityError) {+ // A category-less query hitting several categories is a+ // client-resolvable input problem, not a server fault.+ throw new MCPError(INVALID_PARAMS, `brain_entity: ${err.message}`);+ }+ throw new MCPError(INTERNAL_ERROR, `brain_entity: ${(err as Error).message}`);+ }+}+async function toolBrainBacklinks(ctx: ServerContext,args: Record<string, unknown>,@@ -2968,6 +3040,38 @@ export const BRAIN_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([},handler: toolMcpLandscape,},+ {+ name: "brain_entity",+ previewBudget: MCP_PREVIEW_BUDGET,+ description:+ "Look up (view: get) or list (view: list) canonical entities from the Brain/entities/ registry. Aliases resolve to the canonical record. Read-only; writes go through the o2b CLI.",+ inputSchema: {+ type: "object",+ properties: {+ view: {+ type: "string",+ enum: ["get", "list"],+ description: "get: resolve one entity by name or alias. list: enumerate the registry.",+ },+ query: {+ type: "string",+ description: "Name or alias to resolve (required for view: get).",+ },+ category: {+ type: "string",+ description: "Optional category filter (slug, e.g. `people`).",+ },+ status: {+ type: "string",+ enum: ["active", "archived"],+ description: "Optional status filter for view: list.",+ },+ },+ required: ["view"],+ additionalProperties: false,+ },+ handler: toolBrainEntity,+ },{name: "brain_backlinks",previewBudget: MCP_PREVIEW_BUDGET,diff --git a/src/mcp/resources.ts b/src/mcp/resources.tsindex 5d3bd5cb..955cf92e 100644--- a/src/mcp/resources.ts+++ b/src/mcp/resources.ts@@ -34,6 +34,7 @@*/import { existsSync, readFileSync } from "node:fs";+import { listLogMarkdownFiles } from "../core/brain/log-jsonl.ts";import { regenerateActive } from "../core/brain/active.ts";import { buildBacklinkIndex } from "../core/brain/backlinks.ts";@@ -42,7 +43,6 @@ import { renderDigest } from "../core/brain/digest.ts";import { computeBrainStatus, type BrainStatusSnapshot } from "../core/brain/status.ts";import {brainActivePath,- logPath,preferencePath,retiredPath,validateIsoDate,@@ -313,11 +313,17 @@ function readTopic(ctx: ResourceContext, uri: string, rawSlug: string): Resourcefunction readLog(ctx: ResourceContext, uri: string, rawDate: string): ResourceContent {validateIsoDate(rawDate);- const path = logPath(ctx.vault, rawDate);- if (!existsSync(path)) {+ // Per-device shards (Memory Integrity Suite): a day's human view can+ // span several markdown files. The legacy single file is returned+ // verbatim (byte-identical to pre-shard behaviour); multiple shards+ // are concatenated in shard order.+ const files = listLogMarkdownFiles(ctx.vault).filter((f) => f.date === rawDate);+ if (files.length === 0) {throw new MCPError(INTERNAL_ERROR, `no log file for date '${rawDate}'`);}- return readMarkdown(uri, path);+ if (files.length === 1) return readMarkdown(uri, files[0]!.path);+ const text = files.map((f) => readFileSync(f.path, "utf8").trimEnd()).join("\n\n");+ return { uri, mimeType: MIME_MARKDOWN, text };}function readBacklinks(ctx: ResourceContext, uri: string, rawId: string): ResourceContent {diff --git a/tests/cli/brain-entity.test.ts b/tests/cli/brain-entity.test.tsnew file mode 100644index 00000000..40f90bb9--- /dev/null+++ b/tests/cli/brain-entity.test.ts@@ -0,0 +1,123 @@+import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { bootstrapBrain } from "../../src/core/brain/init.ts";+import { runCli } from "../helpers/run-cli.ts";++let tmp: string;+let vault: string;+let configPath: string;++beforeEach(() => {+ tmp = mkdtempSync(join(tmpdir(), "o2b-entity-cli-"));+ vault = join(tmp, "vault");+ configPath = join(tmp, "config.yaml");+ writeFileSync(configPath, `vault: ${vault}\nagent_name: test-agent\n`);+ bootstrapBrain(vault, { configPath });+});++afterEach(() => {+ rmSync(tmp, { recursive: true, force: true });+});++const env = () => ({ OPEN_SECOND_BRAIN_CONFIG: configPath });++async function seed(): Promise<void> {+ const out = await runCli(+ [+ "brain",+ "entity",+ "set",+ "people",+ "Ada",+ "--alias",+ "the operator",+ "--body",+ "Vault operator.",+ ],+ { env: env() },+ );+ expect(out.returncode).toBe(0);+}++describe("o2b brain entity", () => {+ test("set creates and reports the entity id", async () => {+ const out = await runCli(["brain", "entity", "set", "people", "Ada", "--json"], {+ env: env(),+ });+ expect(out.returncode).toBe(0);+ const payload = JSON.parse(out.stdout) as Record<string, unknown>;+ expect(payload["id"]).toBe("ent-people-ada");+ expect(payload["created"]).toBe(true);+ });++ test("get resolves by alias and prints the record", async () => {+ await seed();+ const out = await runCli(["brain", "entity", "get", "the operator", "--json"], {+ env: env(),+ });+ expect(out.returncode).toBe(0);+ const payload = JSON.parse(out.stdout) as Record<string, unknown>;+ expect(payload["id"]).toBe("ent-people-ada");+ expect(payload["name"]).toBe("Ada");+ });++ test("get returns exit 2 for an unknown entity", async () => {+ await seed();+ const out = await runCli(["brain", "entity", "get", "nobody", "--json"], { env: env() });+ expect(out.returncode).toBe(2);+ });++ test("list filters by category", async () => {+ await seed();+ await runCli(["brain", "entity", "set", "projects", "Open Second Brain"], { env: env() });+ const out = await runCli(["brain", "entity", "list", "--category", "people", "--json"], {+ env: env(),+ });+ expect(out.returncode).toBe(0);+ const payload = JSON.parse(out.stdout) as { entities: Array<{ id: string }> };+ expect(payload.entities.map((e) => e.id)).toEqual(["ent-people-ada"]);+ });++ test("relate writes a typed relation", async () => {+ await seed();+ await runCli(["brain", "entity", "set", "projects", "Open Second Brain"], { env: env() });+ const out = await runCli(+ ["brain", "entity", "relate", "Ada", "related", "Open Second Brain", "--json"],+ { env: env() },+ );+ expect(out.returncode).toBe(0);+ const payload = JSON.parse(out.stdout) as { relations: Array<{ target: string }> };+ expect(payload.relations[0]!.target).toBe("ent-projects-open-second-brain");+ });++ test("archive and restore round-trip", async () => {+ await seed();+ const archived = await runCli(["brain", "entity", "archive", "Ada", "--json"], {+ env: env(),+ });+ expect(archived.returncode).toBe(0);+ expect((JSON.parse(archived.stdout) as Record<string, unknown>)["status"]).toBe("archived");++ const gone = await runCli(["brain", "entity", "get", "Ada"], { env: env() });+ expect(gone.returncode).toBe(2);++ const restored = await runCli(["brain", "entity", "archive", "Ada", "--restore", "--json"], {+ env: env(),+ });+ expect(restored.returncode).toBe(0);+ expect((JSON.parse(restored.stdout) as Record<string, unknown>)["status"]).toBe("active");+ });++ test("duplicate alias claim fails with a clear error", async () => {+ await seed();+ const out = await runCli(+ ["brain", "entity", "set", "people", "Imposter", "--alias", "the operator"],+ { env: env() },+ );+ expect(out.returncode).not.toBe(0);+ expect(out.stderr).toContain("alias");+ });+});diff --git a/tests/core/brain/capture-boundary.test.ts b/tests/core/brain/capture-boundary.test.tsnew file mode 100644index 00000000..afa47dd9--- /dev/null+++ b/tests/core/brain/capture-boundary.test.ts@@ -0,0 +1,166 @@+/**+ * Capture-boundary matcher (Memory Integrity Suite, t_0532ed5a).+ *+ * Compiles the `sessions:` policy into a decision object: session ids+ * (and transcript paths) match anchored globs; message suppression is+ * regex; an invalid regex degrades to a warning and is skipped, never+ * thrown. Ignore outranks stateless. Machine-local config may ADD+ * patterns (comma-separated union) but never remove vault policy.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdtempSync, readFileSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { bootstrapBrain } from "../../../src/core/brain/init.ts";+import { atomicWriteFileSync } from "../../../src/core/fs-atomic.ts";+import { brainConfigPath } from "../../../src/core/brain/paths.ts";+import {+ buildCaptureBoundary,+ compileCaptureBoundary,+} from "../../../src/core/brain/capture-boundary.ts";++let vault: string;+let configHome: string;+let configPath: string;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-capture-boundary-"));+ configHome = mkdtempSync(join(tmpdir(), "o2b-capture-boundary-cfg-"));+ configPath = join(configHome, "config.yaml");+ atomicWriteFileSync(configPath, `vault: ${vault}\n`);+ bootstrapBrain(vault, { configPath });+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+ rmSync(configHome, { recursive: true, force: true });+});++function appendBrainConfig(block: string): void {+ const path = brainConfigPath(vault);+ atomicWriteFileSync(path, readFileSync(path, "utf8") + "\n" + block + "\n");+}++describe("compileCaptureBoundary", () => {+ test("empty policy captures everything and suppresses nothing", () => {+ const b = compileCaptureBoundary({+ ignore_patterns: [],+ stateless_patterns: [],+ ignore_message_patterns: [],+ });+ expect(b.sessionDecision("any-session")).toBe("capture");+ expect(b.suppressMessage("any text")).toBe(false);+ expect(b.warnings).toEqual([]);+ });++ test("anchored glob semantics: *, ?, full-match only", () => {+ const b = compileCaptureBoundary({+ ignore_patterns: ["cron-*", "probe-?"],+ stateless_patterns: [],+ ignore_message_patterns: [],+ });+ expect(b.sessionDecision("cron-nightly")).toBe("ignore");+ expect(b.sessionDecision("a-cron-nightly")).toBe("capture"); // anchored+ expect(b.sessionDecision("probe-1")).toBe("ignore");+ expect(b.sessionDecision("probe-12")).toBe("capture"); // ? is one char+ });++ test("glob metacharacters from regex never leak", () => {+ const b = compileCaptureBoundary({+ ignore_patterns: ["a.b"],+ stateless_patterns: [],+ ignore_message_patterns: [],+ });+ expect(b.sessionDecision("a.b")).toBe("ignore");+ expect(b.sessionDecision("axb")).toBe("capture"); // dot is literal+ });++ test("ignore outranks stateless", () => {+ const b = compileCaptureBoundary({+ ignore_patterns: ["dual-*"],+ stateless_patterns: ["dual-*"],+ ignore_message_patterns: [],+ });+ expect(b.sessionDecision("dual-x")).toBe("ignore");+ });++ test("transcript path participates in session matching", () => {+ const b = compileCaptureBoundary({+ ignore_patterns: ["*side-channel*"],+ stateless_patterns: [],+ ignore_message_patterns: [],+ });+ expect(b.sessionDecision("sess-1", "/tmp/side-channel/sess-1.jsonl")).toBe("ignore");+ expect(b.sessionDecision("sess-1", "/tmp/main/sess-1.jsonl")).toBe("capture");+ });++ test("message suppression is regex", () => {+ const b = compileCaptureBoundary({+ ignore_patterns: [],+ stateless_patterns: [],+ ignore_message_patterns: ["^\\[heartbeat\\]", "ping{2,}"],+ });+ expect(b.suppressMessage("[heartbeat] still alive")).toBe(true);+ expect(b.suppressMessage("pingg")).toBe(true);+ expect(b.suppressMessage("normal message")).toBe(false);+ });++ test("an invalid message regex degrades to a warning and is skipped", () => {+ const b = compileCaptureBoundary({+ ignore_patterns: [],+ stateless_patterns: [],+ ignore_message_patterns: ["([unclosed", "^ok$"],+ });+ expect(b.warnings.length).toBe(1);+ expect(b.warnings[0]).toContain("([unclosed");+ expect(b.suppressMessage("ok")).toBe(true); // valid pattern still active+ });++ test("undefined session id only matches via transcript path", () => {+ const b = compileCaptureBoundary({+ ignore_patterns: ["cron-*"],+ stateless_patterns: [],+ ignore_message_patterns: [],+ });+ expect(b.sessionDecision(undefined)).toBe("capture");+ expect(b.sessionDecision(undefined, "cron-nightly")).toBe("ignore");+ });+});++describe("buildCaptureBoundary (vault + machine-local union)", () => {+ test("reads the vault sessions policy", () => {+ appendBrainConfig(["sessions:", " ignore_patterns:", ' - "cron-*"'].join("\n"));+ const b = buildCaptureBoundary(vault, { localConfigPath: configPath });+ expect(b.sessionDecision("cron-x")).toBe("ignore");+ });++ test("machine-local comma-separated patterns union with vault policy", () => {+ appendBrainConfig(["sessions:", " ignore_patterns:", ' - "cron-*"'].join("\n"));+ atomicWriteFileSync(+ configPath,+ `vault: ${vault}\nsessions_ignore_patterns: "local-*, scratch-?"\n`,+ );+ const b = buildCaptureBoundary(vault, { localConfigPath: configPath });+ expect(b.sessionDecision("cron-x")).toBe("ignore"); // vault policy intact+ expect(b.sessionDecision("local-y")).toBe("ignore"); // local addition+ expect(b.sessionDecision("scratch-1")).toBe("ignore");+ });++ test("a broken vault config fails soft to capture-everything", () => {+ atomicWriteFileSync(brainConfigPath(vault), "schema_version: [broken\n");+ const b = buildCaptureBoundary(vault, { localConfigPath: configPath });+ expect(b.sessionDecision("anything")).toBe("capture");+ });+});++describe("doctor invalid-capture-pattern lint", () => {+ test("an invalid message regex in the vault policy surfaces in doctor", async () => {+ appendBrainConfig(["sessions:", " ignore_message_patterns:", ' - "([unclosed"'].join("\n"));+ const { runDoctor } = await import("../../../src/core/brain/doctor.ts");+ const hits = runDoctor(vault).warnings.filter((i) => i.code === "invalid-capture-pattern");+ expect(hits).toHaveLength(1);+ expect(hits[0]!.message).toContain("([unclosed");+ });+});diff --git a/tests/core/brain/doctor.entities.test.ts b/tests/core/brain/doctor.entities.test.tsnew file mode 100644index 00000000..09ef40da--- /dev/null+++ b/tests/core/brain/doctor.entities.test.ts@@ -0,0 +1,97 @@+import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdtempSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { bootstrapBrain } from "../../../src/core/brain/init.ts";+import { runDoctor } from "../../../src/core/brain/doctor.ts";+import { atomicWriteFileSync } from "../../../src/core/fs-atomic.ts";+import { relateEntities, upsertEntity } from "../../../src/core/brain/entities/registry.ts";+import { brainDirs } from "../../../src/core/brain/paths.ts";++let vault: string;+let configHome: string;++const NOW = new Date("2026-06-02T12:00:00Z");++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-doctor-entities-"));+ configHome = mkdtempSync(join(tmpdir(), "o2b-doctor-entities-cfg-"));+ const configPath = join(configHome, "config.yaml");+ atomicWriteFileSync(configPath, `vault: ${vault}\n`);+ bootstrapBrain(vault, { configPath });+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+ rmSync(configHome, { recursive: true, force: true });+});++function entityFile(name: string, lines: string[]): void {+ atomicWriteFileSync(join(brainDirs(vault).entities, "people", name), lines.join("\n") + "\n");+}++describe("doctor entity lints", () => {+ test("clean registry produces no entity issues", () => {+ upsertEntity(vault, { category: "people", name: "Ada", agent: "a", now: NOW });+ const out = runDoctor(vault);+ const all = [...out.warnings, ...out.errors];+ expect(all.filter((i) => i.code.includes("entity"))).toEqual([]);+ });++ test("duplicate identity claims surface as duplicate-entity warnings", () => {+ upsertEntity(vault, { category: "people", name: "Ada", agent: "a", now: NOW });+ entityFile("ent-people-ada-dup.md", [+ "---",+ "kind: brain-entity",+ "entity_id: ent-people-ada-dup",+ "category: people",+ "name: ADA",+ "status: active",+ "created_at: 2026-06-02T12:30:00Z",+ "updated_at: 2026-06-02T12:30:00Z",+ "---",+ "",+ "# dup",+ ]);+ const out = runDoctor(vault);+ const dup = out.warnings.filter((i) => i.code === "duplicate-entity");+ expect(dup).toHaveLength(1);+ expect(dup[0]!.severity).toBe("warning");+ expect(dup[0]!.message).toContain("people:ada");+ });++ test("a relation pointing at a missing entity surfaces as broken-entity-relation", () => {+ upsertEntity(vault, { category: "people", name: "Ada", agent: "a", now: NOW });+ upsertEntity(vault, { category: "projects", name: "Open Second Brain", agent: "a", now: NOW });+ relateEntities(vault, {+ from: { category: "people", query: "Ada" },+ relation: "related",+ to: { category: "projects", query: "Open Second Brain" },+ now: NOW,+ });+ // Healthy relation: no issue.+ expect(runDoctor(vault).warnings.filter((i) => i.code === "broken-entity-relation")).toEqual(+ [],+ );++ // Hand-author a relation to a non-existent entity.+ entityFile("ent-people-orphan.md", [+ "---",+ "kind: brain-entity",+ "entity_id: ent-people-orphan",+ "category: people",+ "name: Orphan Author",+ "status: active",+ "created_at: 2026-06-02T12:30:00Z",+ "updated_at: 2026-06-02T12:30:00Z",+ 'related: ["[[ent-systems-ghost]]"]',+ "---",+ "",+ "# orphan",+ ]);+ const broken = runDoctor(vault).warnings.filter((i) => i.code === "broken-entity-relation");+ expect(broken).toHaveLength(1);+ expect(broken[0]!.message).toContain("ent-systems-ghost");+ });+});diff --git a/tests/core/brain/entities.canonical.test.ts b/tests/core/brain/entities.canonical.test.tsnew file mode 100644index 00000000..52e1d6df--- /dev/null+++ b/tests/core/brain/entities.canonical.test.ts@@ -0,0 +1,69 @@+import { describe, expect, test } from "bun:test";++import {+ entityIdentityKey,+ normalizeEntityName,+ validateEntityCategory,+} from "../../../src/core/brain/entities/canonical.ts";++describe("normalizeEntityName", () => {+ test("lowercases and trims", () => {+ expect(normalizeEntityName(" Open Second Brain ")).toBe("open second brain");+ });++ test("collapses internal whitespace runs", () => {+ expect(normalizeEntityName("Open\t Second Brain")).toBe("open second brain");+ });++ test("applies NFC so composed and decomposed forms agree", () => {+ const composed = "café"; // é as one code point+ const decomposed = "café"; // e + combining acute+ expect(normalizeEntityName(composed)).toBe(normalizeEntityName(decomposed));+ });++ test("keeps non-latin scripts intact", () => {+ expect(normalizeEntityName("Ада")).toBe("ада");+ });+});++describe("entityIdentityKey", () => {+ test("composes category and normalized name", () => {+ expect(entityIdentityKey("people", " Ada ")).toBe("people:ada");+ });++ test("same key for case and whitespace variants", () => {+ expect(entityIdentityKey("projects", "Open Second Brain")).toBe(+ entityIdentityKey("projects", "open second brain"),+ );+ });++ test("different categories never collide", () => {+ expect(entityIdentityKey("people", "mercury")).not.toBe(+ entityIdentityKey("systems", "mercury"),+ );+ });+});++describe("validateEntityCategory", () => {+ test("accepts kebab-case slugs", () => {+ expect(validateEntityCategory("people")).toBe("people");+ expect(validateEntityCategory("payment-systems")).toBe("payment-systems");+ });++ test("lowercases on the way through", () => {+ expect(validateEntityCategory("People")).toBe("people");+ });++ test("rejects path separators and traversal", () => {+ expect(() => validateEntityCategory("a/b")).toThrow();+ expect(() => validateEntityCategory("..")).toThrow();+ });++ test("rejects empty and whitespace-only", () => {+ expect(() => validateEntityCategory(" ")).toThrow();+ });++ test("rejects spaces inside the category", () => {+ expect(() => validateEntityCategory("payment systems")).toThrow();+ });+});diff --git a/tests/core/brain/entities.registry.test.ts b/tests/core/brain/entities.registry.test.tsnew file mode 100644index 00000000..321b5397--- /dev/null+++ b/tests/core/brain/entities.registry.test.ts@@ -0,0 +1,340 @@+import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { existsSync, mkdtempSync, readFileSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { bootstrapBrain } from "../../../src/core/brain/init.ts";+import { atomicWriteFileSync } from "../../../src/core/fs-atomic.ts";+import { buildEntityIndex } from "../../../src/core/brain/entities/index-builder.ts";+import {+ archiveEntity,+ getEntity,+ listEntities,+ relateEntities,+ upsertEntity,+} from "../../../src/core/brain/entities/registry.ts";++let vault: string;+let configHome: string;++const NOW = new Date("2026-06-02T12:00:00Z");+const LATER = new Date("2026-06-02T13:00:00Z");++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-entities-vault-"));+ configHome = mkdtempSync(join(tmpdir(), "o2b-entities-cfg-"));+ const configPath = join(configHome, "config.yaml");+ atomicWriteFileSync(configPath, `vault: ${vault}\n`);+ bootstrapBrain(vault, { configPath });+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+ rmSync(configHome, { recursive: true, force: true });+});++function seedAda() {+ return upsertEntity(vault, {+ category: "people",+ name: "Ada",+ aliases: ["Ада", "S.E."],+ agent: "claude-dev-agent",+ now: NOW,+ });+}++describe("upsertEntity", () => {+ test("creates a Markdown entity file under Brain/entities/<category>/", () => {+ const { entity, created } = seedAda();+ expect(created).toBe(true);+ expect(entity.id).toBe("ent-people-ada");+ expect(entity.path).toContain(join("Brain", "entities", "people", "ent-people-ada.md"));+ expect(existsSync(entity.path)).toBe(true);+ const raw = readFileSync(entity.path, "utf8");+ expect(raw).toContain("kind: brain-entity");+ expect(raw).toContain("name: Ada");+ expect(raw.startsWith("---\n")).toBe(true);+ });++ test("is idempotent: same input updates in place, never duplicates", () => {+ seedAda();+ const second = upsertEntity(vault, {+ category: "people",+ name: "ada", // case variant resolves to the same identity+ agent: "claude-dev-agent",+ now: LATER,+ });+ expect(second.created).toBe(false);+ expect(second.entity.id).toBe("ent-people-ada");+ expect(listEntities(vault)).toHaveLength(1);+ });++ test("upsert via an alias updates the canonical entity", () => {+ seedAda();+ const viaAlias = upsertEntity(vault, {+ category: "people",+ name: "Ада",+ agent: "claude-dev-agent",+ now: LATER,+ body: "## Current state\nOperator of the vault.",+ });+ expect(viaAlias.created).toBe(false);+ expect(viaAlias.entity.id).toBe("ent-people-ada");+ expect(viaAlias.entity.name).toBe("Ada");+ expect(readFileSync(viaAlias.entity.path, "utf8")).toContain("Operator of the vault.");+ });++ test("merges new aliases into the existing set", () => {+ seedAda();+ const updated = upsertEntity(vault, {+ category: "people",+ name: "Ada",+ aliases: ["the operator"],+ agent: "claude-dev-agent",+ now: LATER,+ });+ expect(updated.entity.aliases).toContain("Ада");+ expect(updated.entity.aliases).toContain("the operator");+ });++ test("refuses an alias already claimed by another active entity in the category", () => {+ seedAda();+ expect(() =>+ upsertEntity(vault, {+ category: "people",+ name: "Adam Petrov",+ aliases: ["Ада"],+ agent: "claude-dev-agent",+ now: LATER,+ }),+ ).toThrow(/alias/i);+ });++ test("same name in a different category is a distinct entity", () => {+ seedAda();+ const sys = upsertEntity(vault, {+ category: "systems",+ name: "Ada",+ agent: "claude-dev-agent",+ now: NOW,+ });+ expect(sys.created).toBe(true);+ expect(sys.entity.id).toBe("ent-systems-ada");+ expect(listEntities(vault)).toHaveLength(2);+ });++ test("distinct names mapping to one file slug get a suffixed filename", () => {+ const a = upsertEntity(vault, {+ category: "projects",+ name: "Foo Bar",+ agent: "claude-dev-agent",+ now: NOW,+ });+ const b = upsertEntity(vault, {+ category: "projects",+ name: "Foo/Bar", // different identity, same slugified stem+ agent: "claude-dev-agent",+ now: NOW,+ });+ expect(a.created).toBe(true);+ expect(b.created).toBe(true);+ expect(a.entity.id).not.toBe(b.entity.id);+ expect(listEntities(vault)).toHaveLength(2);+ });++ test("rejects empty names", () => {+ expect(() =>+ upsertEntity(vault, {+ category: "people",+ name: " ",+ agent: "claude-dev-agent",+ now: NOW,+ }),+ ).toThrow(/name/i);+ });+});++describe("getEntity", () => {+ test("resolves by canonical name, case-insensitively", () => {+ seedAda();+ const hit = getEntity(vault, { category: "people", query: "ADA" });+ expect(hit?.id).toBe("ent-people-ada");+ });++ test("resolves by alias", () => {+ seedAda();+ const hit = getEntity(vault, { category: "people", query: "s.e." });+ expect(hit?.id).toBe("ent-people-ada");+ });++ test("resolves without category when unambiguous", () => {+ seedAda();+ const hit = getEntity(vault, { query: "Ада" });+ expect(hit?.id).toBe("ent-people-ada");+ });++ test("returns null for unknown names", () => {+ seedAda();+ expect(getEntity(vault, { category: "people", query: "nobody" })).toBeNull();+ });+});++describe("listEntities", () => {+ test("filters by category and sorts by id", () => {+ seedAda();+ upsertEntity(vault, {+ category: "projects",+ name: "Open Second Brain",+ agent: "claude-dev-agent",+ now: NOW,+ });+ const all = listEntities(vault);+ expect(all.map((e) => e.id)).toEqual(["ent-people-ada", "ent-projects-open-second-brain"]);+ const people = listEntities(vault, { category: "people" });+ expect(people).toHaveLength(1);+ });+});++describe("relateEntities", () => {+ test("writes a typed relation edge readable from frontmatter", () => {+ seedAda();+ upsertEntity(vault, {+ category: "projects",+ name: "Open Second Brain",+ agent: "claude-dev-agent",+ now: NOW,+ });+ const updated = relateEntities(vault, {+ from: { category: "people", query: "Ada" },+ relation: "related",+ to: { category: "projects", query: "Open Second Brain" },+ now: LATER,+ });+ expect(updated.relations).toEqual([+ { relation: "related", target: "ent-projects-open-second-brain" },+ ]);+ const raw = readFileSync(updated.path, "utf8");+ expect(raw).toContain("ent-projects-open-second-brain");+ });++ test("rejects relations outside the known vocabulary", () => {+ seedAda();+ upsertEntity(vault, {+ category: "projects",+ name: "Open Second Brain",+ agent: "claude-dev-agent",+ now: NOW,+ });+ expect(() =>+ relateEntities(vault, {+ from: { category: "people", query: "Ada" },+ relation: "loves",+ to: { category: "projects", query: "Open Second Brain" },+ now: LATER,+ }),+ ).toThrow(/relation/i);+ });++ test("relating twice is idempotent", () => {+ seedAda();+ upsertEntity(vault, {+ category: "projects",+ name: "Open Second Brain",+ agent: "claude-dev-agent",+ now: NOW,+ });+ const args = {+ from: { category: "people", query: "Ada" },+ relation: "related",+ to: { category: "projects", query: "Open Second Brain" },+ now: LATER,+ } as const;+ relateEntities(vault, args);+ const second = relateEntities(vault, args);+ expect(second.relations).toHaveLength(1);+ });+});++describe("archiveEntity", () => {+ test("removes the entity from active lookup but keeps the file", () => {+ const { entity } = seedAda();+ const archived = archiveEntity(vault, { category: "people", query: "Ada" }, { now: LATER });+ expect(archived.status).toBe("archived");+ expect(archived.archived_at).toBe("2026-06-02T13:00:00Z");+ expect(existsSync(entity.path)).toBe(true);+ expect(getEntity(vault, { category: "people", query: "Ada" })).toBeNull();+ expect(listEntities(vault, { status: "archived" })).toHaveLength(1);+ });++ test("restore returns the entity to active lookup", () => {+ seedAda();+ archiveEntity(vault, { category: "people", query: "Ada" }, { now: LATER });+ const restored = archiveEntity(+ vault,+ { category: "people", query: "Ada" },+ { now: LATER, restore: true },+ );+ expect(restored.status).toBe("active");+ expect(restored.archived_at).toBeUndefined();+ expect(getEntity(vault, { category: "people", query: "Ada" })?.id).toBe("ent-people-ada");+ });++ test("upsert refuses a name held by an archived entity and names the remedy", () => {+ seedAda();+ archiveEntity(vault, { category: "people", query: "Ada" }, { now: LATER });+ expect(() => seedAda()).toThrow(/archived/i);+ });+});++describe("buildEntityIndex", () => {+ test("rebuilds identically from the Markdown files alone", () => {+ seedAda();+ upsertEntity(vault, {+ category: "projects",+ name: "Open Second Brain",+ aliases: ["o2b vault"],+ agent: "claude-dev-agent",+ now: NOW,+ });+ const first = buildEntityIndex(vault);+ const second = buildEntityIndex(vault);+ expect(second.entities.map((e) => e.id)).toEqual(first.entities.map((e) => e.id));+ expect([...second.byAlias.keys()].toSorted()).toEqual([...first.byAlias.keys()].toSorted());+ expect(first.byKey.get("people:ada")?.id).toBe("ent-people-ada");+ expect(first.byAlias.get("o2b vault")?.id).toBe("ent-projects-open-second-brain");+ });++ test("reports duplicate identity claims from hand-authored files as conflicts", () => {+ const { entity } = seedAda();+ // Simulate a sync/hand-edit duplicate claiming the same (category, name).+ const dupPath = join(entity.path, "..", "ent-people-ada-dup.md");+ atomicWriteFileSync(+ dupPath,+ [+ "---",+ "kind: brain-entity",+ "entity_id: ent-people-ada-dup",+ "category: people",+ "name: ada",+ "status: active",+ "created_at: 2026-06-02T12:30:00Z",+ "updated_at: 2026-06-02T12:30:00Z",+ "---",+ "",+ "# ada",+ "",+ ].join("\n"),+ );+ const index = buildEntityIndex(vault);+ expect(index.conflicts.length).toBeGreaterThanOrEqual(1);+ expect(index.conflicts[0]!.kind).toBe("duplicate-name");+ expect(index.conflicts[0]!.paths).toHaveLength(2);+ });++ test("skips malformed entity files without aborting the walk", () => {+ const { entity } = seedAda();+ atomicWriteFileSync(join(entity.path, "..", "broken.md"), "not frontmatter at all");+ const index = buildEntityIndex(vault);+ expect(index.entities).toHaveLength(1);+ });+});diff --git a/tests/core/brain/fact-extract.test.ts b/tests/core/brain/fact-extract.test.tsnew file mode 100644index 00000000..5a9e6003--- /dev/null+++ b/tests/core/brain/fact-extract.test.ts@@ -0,0 +1,116 @@+/**+ * Regex fact extraction (Memory Integrity Suite, t_d0782ab2).+ *+ * Seven precision-first pattern families over USER turn text:+ * identity, preference, possession, location, url, email,+ * confirmation. Deterministic - no LLM, no clock. Negative fixtures+ * pin the precision contract: code blocks, quoted lines, and+ * assistant-shaped prose never extract.+ */++import { describe, expect, test } from "bun:test";++import { extractFacts, factDedupHash } from "../../../src/core/brain/fact-extract.ts";++function families(text: string): string[] {+ return extractFacts(text).map((f) => f.family);+}++describe("extractFacts - positive fixtures", () => {+ test("identity: my name is ...", () => {+ const out = extractFacts("Hi, my name is Ada Lovelace and I work on the vault.");+ expect(out.some((f) => f.family === "identity" && f.text.includes("Ada Lovelace"))).toBe(true);+ });++ test("preference: I prefer / I always use", () => {+ const out = extractFacts("For diagrams I prefer the vintage blueprint style.");+ expect(out.some((f) => f.family === "preference")).toBe(true);+ const out2 = extractFacts("I always use bun for this repository.");+ expect(out2.some((f) => f.family === "preference")).toBe(true);+ });++ test("possession: my <thing> is <value>", () => {+ const out = extractFacts("my editor is Neovim with a custom config");+ expect(out.some((f) => f.family === "possession" && f.text.includes("Neovim"))).toBe(true);+ });++ test("location: I live in / I am based in", () => {+ const out = extractFacts("These days I live in Lisbon most of the year.");+ expect(out.some((f) => f.family === "location" && f.text.includes("Lisbon"))).toBe(true);+ });++ test("url: possessively introduced site", () => {+ const out = extractFacts("my blog is https://techmeat.dev and it runs on a static site");+ expect(out.some((f) => f.family === "url" && f.text.includes("https://techmeat.dev"))).toBe(+ true,+ );+ });++ test("email: my email is ...", () => {+ const out = extractFacts("my email is [email protected] for anything urgent");+ expect(out.some((f) => f.family === "email" && f.text.includes("[email protected]"))).toBe(true);+ });++ test("confirmation: yes, the X is Y", () => {+ const out = extractFacts("Correct, the production vault is on the VPS at /srv/vault.");+ expect(out.some((f) => f.family === "confirmation")).toBe(true);+ });+});++describe("extractFacts - precision (negative fixtures)", () => {+ test("plain assistant-shaped prose extracts nothing", () => {+ expect(+ families(+ "The function resolves the vault path and returns the merged entries sorted by timestamp.",+ ),+ ).toEqual([]);+ });++ test("code blocks never extract", () => {+ const text = [+ "Try this:",+ "```ts",+ 'const x = "my name is Bob";',+ "// I prefer tabs",+ "```",+ ].join("\n");+ expect(families(text)).toEqual([]);+ });++ test("quoted lines never extract", () => {+ expect(families("> my name is Bob, said the old book")).toEqual([]);+ });++ test("bare URLs without a possessive introduction never extract", () => {+ expect(families("see https://example.com/docs for details")).toEqual([]);+ });++ test("bare 'yes' without a restated fact never extracts", () => {+ expect(families("yes")).toEqual([]);+ expect(families("yes, please do that now")).toEqual([]);+ });++ test("empty and whitespace input", () => {+ expect(extractFacts("")).toEqual([]);+ expect(extractFacts(" \n ")).toEqual([]);+ });+});++describe("fact dedup hashing", () => {+ test("same fact hashes identically across whitespace and case variants", () => {+ const a = extractFacts("my name is Ada")[0]!;+ const b = extractFacts("My name is ada")[0]!;+ expect(factDedupHash(a)).toBe(factDedupHash(b));+ });++ test("different families never collide on the same text", () => {+ const id = extractFacts("my name is Lisbon")[0]!;+ const loc = extractFacts("I live in Lisbon")[0]!;+ expect(factDedupHash(id)).not.toBe(factDedupHash(loc));+ });++ test("extraction is deterministic", () => {+ const text = "my name is Ada. I prefer dark themes. my email is [email protected]";+ expect(extractFacts(text)).toEqual(extractFacts(text));+ });+});diff --git a/tests/core/brain/import-session.boundary.test.ts b/tests/core/brain/import-session.boundary.test.tsnew file mode 100644index 00000000..5032ae4c--- /dev/null+++ b/tests/core/brain/import-session.boundary.test.ts@@ -0,0 +1,143 @@+/**+ * Capture-boundary and fact-extraction wiring at the BATCH seam+ * (Memory Integrity Suite, t_0532ed5a + t_d0782ab2).+ *+ * importSession honors session ignore globs (matched against the+ * file path), suppresses messages before any extraction, and+ * extracts facts from user turns only.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { existsSync, mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { bootstrapBrain } from "../../../src/core/brain/init.ts";+import { atomicWriteFileSync } from "../../../src/core/fs-atomic.ts";+import { brainConfigPath, brainDirs } from "../../../src/core/brain/paths.ts";+import { importSession } from "../../../src/core/brain/sessions/import.ts";++let vault: string;+let sessionsDir: string;++const NOW = new Date("2026-06-02T10:00:00Z");++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-import-boundary-"));+ sessionsDir = mkdtempSync(join(tmpdir(), "o2b-import-boundary-sessions-"));+ bootstrapBrain(vault);+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+ rmSync(sessionsDir, { recursive: true, force: true });+});++function setSessionsPolicy(lines: string[]): void {+ const path = brainConfigPath(vault);+ atomicWriteFileSync(path, readFileSync(path, "utf8") + "\n" + lines.join("\n") + "\n");+}++function inboxSignals(): string[] {+ const dir = brainDirs(vault).inbox;+ if (!existsSync(dir)) return [];+ return readdirSync(dir).filter((n) => n.startsWith("sig-") && n.endsWith(".md"));+}++/** Minimal Claude Code session fixture: meta line + user turns. */+function writeClaudeSession(name: string, userTexts: string[]): string {+ const lines = [+ JSON.stringify({ type: "summary", summary: "fixture", leafUuid: "0" }),+ ...userTexts.map((text, i) =>+ JSON.stringify({+ type: "user",+ uuid: `u-${i}`,+ timestamp: "2026-06-02T09:00:00.000Z",+ message: { role: "user", content: [{ type: "text", text }] },+ }),+ ),+ ];+ const path = join(sessionsDir, name);+ writeFileSync(path, lines.join("\n") + "\n");+ return path;+}++describe("session-level boundary at import", () => {+ test("a file matching an ignore glob imports nothing", async () => {+ setSessionsPolicy(["sessions:", " ignore_patterns:", ' - "*cron-export*"']);+ const path = writeClaudeSession("cron-export-1.jsonl", ["my name is Ada"]);+ const result = await importSession(vault, path, {+ agent: "tester",+ now: NOW,+ format: "claude",+ });+ expect(result.boundary_decision).toBe("ignore");+ expect(result.turns_scanned).toBe(0);+ expect(result.signals_created).toBe(0);+ expect(result.facts_extracted).toBe(0);+ expect(inboxSignals()).toEqual([]);+ });++ test("a stateless file scans but writes nothing", async () => {+ setSessionsPolicy(["sessions:", " stateless_patterns:", ' - "*probe*"']);+ const path = writeClaudeSession("probe-2.jsonl", ["my name is Ada"]);+ const result = await importSession(vault, path, {+ agent: "tester",+ now: NOW,+ format: "claude",+ });+ expect(result.boundary_decision).toBe("stateless");+ expect(result.signals_created).toBe(0);+ expect(result.facts_extracted).toBe(0);+ expect(inboxSignals()).toEqual([]);+ });+});++describe("message-level boundary at import", () => {+ test("suppressed turns never reach marker or fact extraction", async () => {+ setSessionsPolicy(["sessions:", " ignore_message_patterns:", ' - "^\\[heartbeat\\]"']);+ const path = writeClaudeSession("normal.jsonl", [+ "[heartbeat] my name is Ada",+ "I prefer dark themes",+ ]);+ const result = await importSession(vault, path, {+ agent: "tester",+ now: NOW,+ format: "claude",+ });+ expect(result.boundary_decision).toBe("capture");+ expect(result.suppressed_turns).toBe(1);+ expect(result.facts_extracted).toBe(1); // only the preference fact+ const names = inboxSignals();+ expect(names.some((n) => n.includes("fact-identity"))).toBe(false);+ expect(names.some((n) => n.includes("fact-preference"))).toBe(true);+ });+});++describe("fact extraction at import", () => {+ test("user-turn facts import with dedup across re-imports", async () => {+ const path = writeClaudeSession("facts.jsonl", ["my email is [email protected]"]);+ const first = await importSession(vault, path, { agent: "tester", now: NOW, format: "claude" });+ expect(first.facts_extracted).toBe(1);+ const second = await importSession(vault, path, {+ agent: "tester",+ now: NOW,+ format: "claude",+ });+ expect(second.facts_extracted).toBe(0);+ expect(second.facts_deduped).toBe(1);+ });++ test("unconfigured vault imports exactly as before plus facts", async () => {+ const path = writeClaudeSession("plain.jsonl", ["just a question about the codebase"]);+ const result = await importSession(vault, path, {+ agent: "tester",+ now: NOW,+ format: "claude",+ });+ expect(result.boundary_decision).toBe("capture");+ expect(result.suppressed_turns).toBe(0);+ expect(result.facts_extracted).toBe(0);+ expect(result.signals_created).toBe(0);+ });+});diff --git a/tests/core/brain/log-shards.read.test.ts b/tests/core/brain/log-shards.read.test.tsnew file mode 100644index 00000000..72ab4e7b--- /dev/null+++ b/tests/core/brain/log-shards.read.test.ts@@ -0,0 +1,140 @@+/**+ * Shard-merging log readers (Memory Integrity Suite, t_6d52641f).+ *+ * `Brain/log/<date>.<deviceId>.jsonl` (+ `.md`) shards join the legacy+ * `<date>.jsonl` / `<date>.md` pair as equal sources of one day's+ * events. `readLogDay` merges every shard sorted by (ts, shardId, line)+ * and `listLogDates` is the single date-discovery helper every+ * directory-scanning reader routes through.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { bootstrapBrain } from "../../../src/core/brain/init.ts";+import { atomicWriteFileSync } from "../../../src/core/fs-atomic.ts";+import { appendLogEvent } from "../../../src/core/brain/log.ts";+import { listLogDates, readLogDay } from "../../../src/core/brain/log-jsonl.ts";+import { brainDirs } from "../../../src/core/brain/paths.ts";+import { runDoctor } from "../../../src/core/brain/doctor.ts";++let vault: string;+let configHome: string;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-log-shards-"));+ configHome = mkdtempSync(join(tmpdir(), "o2b-log-shards-cfg-"));+ const configPath = join(configHome, "config.yaml");+ atomicWriteFileSync(configPath, `vault: ${vault}\n`);+ bootstrapBrain(vault, { configPath });+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+ rmSync(configHome, { recursive: true, force: true });+});++function logDir(): string {+ return brainDirs(vault).log;+}++function jsonlRow(ts: string, text: string): string {+ return JSON.stringify({ ts, kind: "note", payload: { text, agent: "test" } });+}++function writeShard(name: string, rows: string[]): void {+ mkdirSync(logDir(), { recursive: true });+ writeFileSync(join(logDir(), name), rows.join("\n") + "\n");+}++describe("readLogDay shard merge", () => {+ test("legacy-only day reads exactly as before", () => {+ appendLogEvent(vault, {+ timestamp: "2026-06-01T10:00:00Z",+ eventType: "note",+ body: { text: "legacy row", agent: "test" },+ });+ const out = readLogDay(vault, "2026-06-01");+ expect(out.entries).toHaveLength(1);+ expect(out.entries[0]!.body["text"]).toBe("legacy row");+ expect(out.source).toBe("jsonl");+ });++ test("merges legacy file and device shards sorted by timestamp", () => {+ writeShard("2026-06-01.jsonl", [jsonlRow("2026-06-01T10:00:02Z", "legacy")]);+ writeShard("2026-06-01.mac1234.jsonl", [+ jsonlRow("2026-06-01T10:00:01Z", "mac-early"),+ jsonlRow("2026-06-01T10:00:03Z", "mac-late"),+ ]);+ writeShard("2026-06-01.vps5678.jsonl", [jsonlRow("2026-06-01T10:00:02Z", "vps")]);+ const out = readLogDay(vault, "2026-06-01");+ expect(out.entries.map((e) => e.body["text"])).toEqual([+ "mac-early",+ "legacy", // ts tie with vps: legacy shard ("") sorts before "vps5678"+ "vps",+ "mac-late",+ ]);+ });++ test("same-timestamp rows within one shard keep line order", () => {+ writeShard("2026-06-01.mac1234.jsonl", [+ jsonlRow("2026-06-01T10:00:00Z", "first"),+ jsonlRow("2026-06-01T10:00:00Z", "second"),+ ]);+ const out = readLogDay(vault, "2026-06-01");+ expect(out.entries.map((e) => e.body["text"])).toEqual(["first", "second"]);+ });++ test("a malformed shard row surfaces as a warning without dropping other shards", () => {+ writeShard("2026-06-01.mac1234.jsonl", ["{not json"]);+ writeShard("2026-06-01.vps5678.jsonl", [jsonlRow("2026-06-01T10:00:00Z", "good")]);+ const out = readLogDay(vault, "2026-06-01");+ expect(out.entries).toHaveLength(1);+ expect(out.warnings.length).toBeGreaterThanOrEqual(1);+ });++ test("markdown-only shards fall back to markdown parsing", () => {+ // No JSONL at all; one legacy md + one shard md.+ mkdirSync(logDir(), { recursive: true });+ writeFileSync(+ join(logDir(), "2026-06-01.md"),+ "---\nkind: brain-log\ndate: 2026-06-01\ntags: [brain, brain/log]\n---\n\n# Brain log — 2026-06-01\n\n## 10:00:02Z — note\n\n- text: legacy md\n- agent: test\n",+ );+ writeFileSync(+ join(logDir(), "2026-06-01.mac1234.md"),+ "---\nkind: brain-log\ndate: 2026-06-01\ntags: [brain, brain/log]\n---\n\n# Brain log — 2026-06-01\n\n## 10:00:01Z — note\n\n- text: shard md\n- agent: test\n",+ );+ const out = readLogDay(vault, "2026-06-01");+ expect(out.source).toBe("markdown-fallback");+ expect(out.entries.map((e) => e.body["text"])).toEqual(["shard md", "legacy md"]);+ });+});++describe("listLogDates", () => {+ test("collects unique dates across legacy and shard file names", () => {+ writeShard("2026-06-01.jsonl", [jsonlRow("2026-06-01T10:00:00Z", "a")]);+ writeShard("2026-06-01.mac1234.jsonl", [jsonlRow("2026-06-01T10:00:01Z", "b")]);+ writeShard("2026-06-02.vps5678.jsonl", [jsonlRow("2026-06-02T10:00:00Z", "c")]);+ mkdirSync(logDir(), { recursive: true });+ writeFileSync(join(logDir(), "2026-06-03.md"), "---\nkind: brain-log\n---\n");+ writeFileSync(join(logDir(), "not-a-date.md"), "junk");+ writeFileSync(join(logDir(), "2026-06-04.sync-conflict-x.jsonl"), "junk");+ expect(listLogDates(vault)).toEqual(["2026-06-01", "2026-06-02", "2026-06-03"]);+ });++ test("empty or absent log dir lists nothing", () => {+ expect(listLogDates(vault)).toEqual([]);+ });+});++describe("doctor sync-conflict lint", () => {+ test("flags .sync-conflict-* files under Brain/log", () => {+ mkdirSync(logDir(), { recursive: true });+ writeFileSync(join(logDir(), "2026-06-01.sync-conflict-20260601-120000-ABCDEF.jsonl"), "{}");+ const issues = runDoctor(vault).warnings.filter((i) => i.code === "sync-conflict-log");+ expect(issues).toHaveLength(1);+ expect(issues[0]!.message).toContain("sync-conflict");+ });+});diff --git a/tests/core/brain/log-shards.write.test.ts b/tests/core/brain/log-shards.write.test.tsnew file mode 100644index 00000000..2b3d4c6d--- /dev/null+++ b/tests/core/brain/log-shards.write.test.ts@@ -0,0 +1,105 @@+/**+ * Sharded Brain log writer (Memory Integrity Suite, t_6d52641f).+ *+ * `appendLogEvent` writes `Brain/log/<date>.<deviceId>.jsonl` + `.md`+ * so two devices never touch the same file on the same day. The+ * device id comes from the device-local config; tests inject it.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { existsSync, mkdtempSync, readFileSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { bootstrapBrain } from "../../../src/core/brain/init.ts";+import { atomicWriteFileSync } from "../../../src/core/fs-atomic.ts";+import { appendLogEvent } from "../../../src/core/brain/log.ts";+import { readLogDay } from "../../../src/core/brain/log-jsonl.ts";+import { brainDirs } from "../../../src/core/brain/paths.ts";++let vault: string;+let configHome: string;+let configPath: string;+const savedEnv: Record<string, string | undefined> = {};++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-shard-write-"));+ configHome = mkdtempSync(join(tmpdir(), "o2b-shard-write-cfg-"));+ configPath = join(configHome, "config.yaml");+ savedEnv["OPEN_SECOND_BRAIN_CONFIG"] = process.env["OPEN_SECOND_BRAIN_CONFIG"];+ savedEnv["O2B_DEVICE_ID"] = process.env["O2B_DEVICE_ID"];+ process.env["OPEN_SECOND_BRAIN_CONFIG"] = configPath;+ delete process.env["O2B_DEVICE_ID"];+ atomicWriteFileSync(configPath, `vault: ${vault}\ndevice_id: "testdev1"\n`);+ bootstrapBrain(vault, { configPath });+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+ rmSync(configHome, { recursive: true, force: true });+ for (const k of ["OPEN_SECOND_BRAIN_CONFIG", "O2B_DEVICE_ID"]) {+ const v = savedEnv[k];+ if (v === undefined) delete process.env[k];+ else process.env[k] = v;+ }+});++function note(ts: string, text: string) {+ return {+ timestamp: ts,+ eventType: "note",+ body: { text, agent: "test" },+ } as const;+}++describe("appendLogEvent shard writing", () => {+ test("writes the device shard pair, not the legacy files", () => {+ appendLogEvent(vault, note("2026-06-01T10:00:00Z", "row"));+ const dir = brainDirs(vault).log;+ expect(existsSync(join(dir, "2026-06-01.testdev1.jsonl"))).toBe(true);+ expect(existsSync(join(dir, "2026-06-01.testdev1.md"))).toBe(true);+ expect(existsSync(join(dir, "2026-06-01.jsonl"))).toBe(false);+ expect(existsSync(join(dir, "2026-06-01.md"))).toBe(false);+ });++ test("an explicit deviceId option overrides config resolution", () => {+ appendLogEvent(vault, note("2026-06-01T10:00:00Z", "row"), { deviceId: "otherdev" });+ const dir = brainDirs(vault).log;+ expect(existsSync(join(dir, "2026-06-01.otherdev.jsonl"))).toBe(true);+ expect(existsSync(join(dir, "2026-06-01.testdev1.jsonl"))).toBe(false);+ });++ test("two appends land in one shard in order and read back merged", () => {+ appendLogEvent(vault, note("2026-06-01T10:00:00Z", "one"));+ appendLogEvent(vault, note("2026-06-01T10:00:01Z", "two"));+ const dir = brainDirs(vault).log;+ const jsonl = readFileSync(join(dir, "2026-06-01.testdev1.jsonl"), "utf8");+ expect(jsonl.trim().split("\n")).toHaveLength(2);+ const out = readLogDay(vault, "2026-06-01");+ expect(out.entries.map((e) => e.body["text"])).toEqual(["one", "two"]);+ });++ test("new shard writes merge with a pre-existing legacy file on read", () => {+ // Legacy single-file day written before the shard layout shipped.+ const dir = brainDirs(vault).log;+ appendLogEvent(vault, note("2026-06-01T10:00:00Z", "legacy"), { deviceId: "" });+ expect(existsSync(join(dir, "2026-06-01.jsonl"))).toBe(true);+ appendLogEvent(vault, note("2026-06-01T10:00:01Z", "sharded"));+ const out = readLogDay(vault, "2026-06-01");+ expect(out.entries.map((e) => e.body["text"])).toEqual(["legacy", "sharded"]);+ });++ test("markdown shard carries the canonical header and the event block", () => {+ appendLogEvent(vault, note("2026-06-01T10:00:00Z", "row"));+ const md = readFileSync(join(brainDirs(vault).log, "2026-06-01.testdev1.md"), "utf8");+ expect(md).toContain("kind: brain-log");+ expect(md).toContain("## 10:00:00Z");+ expect(md).toContain("row");+ });++ test("rejects an invalid explicit deviceId", () => {+ expect(() =>+ appendLogEvent(vault, note("2026-06-01T10:00:00Z", "row"), { deviceId: "BAD/ID" }),+ ).toThrow(/device/i);+ });+});diff --git a/tests/core/brain/policy.sessions.test.ts b/tests/core/brain/policy.sessions.test.tsnew file mode 100644index 00000000..85c047a5--- /dev/null+++ b/tests/core/brain/policy.sessions.test.ts@@ -0,0 +1,92 @@+/**+ * `sessions:` config block (Memory Integrity Suite, t_0532ed5a).+ *+ * Vault-portable capture-boundary policy: ignored session globs,+ * stateless session globs, and message suppression regexes. Absent+ * block resolves to empty lists - bit-identical capture behaviour.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdtempSync, readFileSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { bootstrapBrain } from "../../../src/core/brain/init.ts";+import { atomicWriteFileSync } from "../../../src/core/fs-atomic.ts";+import {+ BRAIN_SESSIONS_DEFAULTS,+ loadBrainConfigDetailed,+ resolveSessions,+} from "../../../src/core/brain/policy.ts";+import { brainConfigPath } from "../../../src/core/brain/paths.ts";++let vault: string;+let configHome: string;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-policy-sessions-"));+ configHome = mkdtempSync(join(tmpdir(), "o2b-policy-sessions-cfg-"));+ const configPath = join(configHome, "config.yaml");+ atomicWriteFileSync(configPath, `vault: ${vault}\n`);+ bootstrapBrain(vault, { configPath });+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+ rmSync(configHome, { recursive: true, force: true });+});++function appendConfig(block: string): void {+ // bootstrap wrote the canonical template; append our block.+ const path = brainConfigPath(vault);+ const text = readFileSync(path, "utf8");+ atomicWriteFileSync(path, text + "\n" + block + "\n");+}++describe("sessions block parsing", () => {+ test("absent block resolves to empty defaults", () => {+ const { config } = loadBrainConfigDetailed(vault);+ expect(config.sessions).toBeUndefined();+ const resolved = resolveSessions(config);+ expect(resolved).toEqual(BRAIN_SESSIONS_DEFAULTS);+ expect(resolved.ignore_patterns).toEqual([]);+ expect(resolved.stateless_patterns).toEqual([]);+ expect(resolved.ignore_message_patterns).toEqual([]);+ });++ test("parses all three lists", () => {+ appendConfig(+ [+ "sessions:",+ " ignore_patterns:",+ ' - "cron-*"',+ ' - "*heartbeat*"',+ " stateless_patterns:",+ ' - "probe-*"',+ " ignore_message_patterns:",+ ' - "^[ping]"',+ ].join("\n"),+ );+ const { config } = loadBrainConfigDetailed(vault);+ const resolved = resolveSessions(config);+ expect(resolved.ignore_patterns).toEqual(["cron-*", "*heartbeat*"]);+ expect(resolved.stateless_patterns).toEqual(["probe-*"]);+ expect(resolved.ignore_message_patterns).toEqual(["^[ping]"]);+ });++ test("rejects a non-array value", () => {+ appendConfig(["sessions:", " ignore_patterns: nope"].join("\n"));+ expect(() => loadBrainConfigDetailed(vault)).toThrow(/ignore_patterns/);+ });++ test("rejects empty-string entries", () => {+ appendConfig(["sessions:", " ignore_patterns:", ' - ""'].join("\n"));+ expect(() => loadBrainConfigDetailed(vault)).toThrow(/ignore_patterns\[0\]/);+ });++ test("unknown sub-keys warn instead of failing (forward-compat)", () => {+ appendConfig(["sessions:", " shiny_future_knob: 5"].join("\n"));+ const { warnings } = loadBrainConfigDetailed(vault);+ expect(warnings.some((w) => w.message.includes("shiny_future_knob"))).toBe(true);+ });+});diff --git a/tests/core/brain/session-lifecycle.boundary.test.ts b/tests/core/brain/session-lifecycle.boundary.test.tsnew file mode 100644index 00000000..b57cabcb--- /dev/null+++ b/tests/core/brain/session-lifecycle.boundary.test.ts@@ -0,0 +1,181 @@+/**+ * Capture-boundary and fact-extraction wiring at the LIVE seam+ * (Memory Integrity Suite, t_0532ed5a + t_d0782ab2).+ *+ * Pipeline order is the contract: classify/suppress first, extract+ * second. Ignored sessions produce nothing but the audit row;+ * stateless sessions write no Brain state; suppressed messages never+ * reach marker or fact extraction; facts extract only from text that+ * passed the boundary.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { existsSync, mkdtempSync, readFileSync, readdirSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { bootstrapBrain } from "../../../src/core/brain/init.ts";+import { atomicWriteFileSync } from "../../../src/core/fs-atomic.ts";+import { brainConfigPath, brainDirs } from "../../../src/core/brain/paths.ts";+import { captureSessionLifecycleEvent } from "../../../src/core/brain/session-lifecycle.ts";+import { upsertEntity } from "../../../src/core/brain/entities/registry.ts";+import { parseSignal } from "../../../src/core/brain/signal.ts";++let vault: string;++const NOW = new Date("2026-06-02T10:00:00Z");++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-lifecycle-boundary-"));+ bootstrapBrain(vault);+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+});++function setSessionsPolicy(lines: string[]): void {+ const path = brainConfigPath(vault);+ atomicWriteFileSync(path, readFileSync(path, "utf8") + "\n" + lines.join("\n") + "\n");+}++function inboxSignals(): string[] {+ const dir = brainDirs(vault).inbox;+ if (!existsSync(dir)) return [];+ return readdirSync(dir).filter((n) => n.startsWith("sig-") && n.endsWith(".md"));+}++const MARKER_PROMPT =+ '@osb feedback positive topic=boundary-check principle="capture only what passed the boundary"';++describe("ignored sessions", () => {+ test("produce no signals, no lifecycle log, only the audit row", async () => {+ setSessionsPolicy(["sessions:", " ignore_patterns:", ' - "cron-*"']);+ const result = await captureSessionLifecycleEvent(+ vault,+ { hook_event_name: "UserPromptSubmit", session_id: "cron-nightly", prompt: MARKER_PROMPT },+ { agent: "tester", now: NOW },+ );+ expect(result.boundary_decision).toBe("ignore");+ expect(result.signals_created).toBe(0);+ expect(result.log_path).toBeUndefined();+ expect(result.audit_path).toContain("session-lifecycle");+ expect(inboxSignals()).toEqual([]);+ expect(existsSync(join(brainDirs(vault).log, "2026-06-02.md"))).toBe(false);+ });+});++describe("stateless sessions", () => {+ test("write no signals and no lifecycle log", async () => {+ setSessionsPolicy(["sessions:", " stateless_patterns:", ' - "probe-*"']);+ const result = await captureSessionLifecycleEvent(+ vault,+ { hook_event_name: "UserPromptSubmit", session_id: "probe-7", prompt: MARKER_PROMPT },+ { agent: "tester", now: NOW },+ );+ expect(result.boundary_decision).toBe("stateless");+ expect(result.signals_created).toBe(0);+ expect(result.log_path).toBeUndefined();+ expect(inboxSignals()).toEqual([]);+ });+});++describe("message suppression", () => {+ test("a suppressed prompt never reaches marker or fact extraction", async () => {+ setSessionsPolicy(["sessions:", " ignore_message_patterns:", ' - "^.osb feedback"']);+ const result = await captureSessionLifecycleEvent(+ vault,+ { hook_event_name: "UserPromptSubmit", session_id: "real-session", prompt: MARKER_PROMPT },+ { agent: "tester", now: NOW },+ );+ expect(result.boundary_decision).toBe("capture");+ expect(result.suppressed_messages).toBe(1);+ expect(result.signals_created).toBe(0);+ expect(inboxSignals()).toEqual([]);+ // The lifecycle log still records the event (the suppression is+ // counted, the content is not persisted).+ expect(result.log_path).toBeDefined();+ });+});++describe("unconfigured vault", () => {+ test("behaves exactly as before the boundary existed", async () => {+ const result = await captureSessionLifecycleEvent(+ vault,+ { hook_event_name: "UserPromptSubmit", session_id: "s1", prompt: MARKER_PROMPT },+ { agent: "tester", now: NOW },+ );+ expect(result.boundary_decision).toBe("capture");+ expect(result.suppressed_messages).toBe(0);+ expect(result.signals_created).toBe(1);+ });+});++describe("fact extraction at the live seam", () => {+ test("facts extract from a captured user prompt and dedupe on repeat", async () => {+ const payload = {+ hook_event_name: "UserPromptSubmit",+ session_id: "s1",+ prompt: "By the way, my name is Ada and I prefer dark themes.",+ };+ const first = await captureSessionLifecycleEvent(vault, payload, {+ agent: "tester",+ now: NOW,+ });+ expect(first.facts_extracted).toBe(2);+ expect(first.facts_deduped).toBe(0);+ const names = inboxSignals();+ expect(names.some((n) => n.includes("fact-identity"))).toBe(true);+ expect(names.some((n) => n.includes("fact-preference"))).toBe(true);++ const second = await captureSessionLifecycleEvent(vault, payload, {+ agent: "tester",+ now: new Date("2026-06-02T10:00:05Z"),+ });+ expect(second.facts_extracted).toBe(0);+ expect(second.facts_deduped).toBe(2);+ });++ test("extracted signals carry source_type extracted", async () => {+ await captureSessionLifecycleEvent(+ vault,+ { hook_event_name: "UserPromptSubmit", session_id: "s1", prompt: "my name is Ada" },+ { agent: "tester", now: NOW },+ );+ const name = inboxSignals().find((n) => n.includes("fact-identity"))!;+ const sig = parseSignal(join(brainDirs(vault).inbox, name));+ expect(sig.source_type).toBe("extracted");+ expect(sig.principle).toContain("Ada");+ });++ test("a fact naming a registered entity gets the canonical anchor", async () => {+ upsertEntity(vault, {+ category: "projects",+ name: "Open Second Brain",+ aliases: ["the vault project"],+ agent: "tester",+ now: NOW,+ });+ await captureSessionLifecycleEvent(+ vault,+ {+ hook_event_name: "UserPromptSubmit",+ session_id: "s1",+ prompt: "I prefer the vault project release diagrams in blueprint style",+ },+ { agent: "tester", now: NOW },+ );+ const name = inboxSignals().find((n) => n.includes("fact-preference"))!;+ const raw = readFileSync(join(brainDirs(vault).inbox, name), "utf8");+ expect(raw).toContain("ent-projects-open-second-brain");+ });++ test("assistant-shaped events without prompt text extract nothing", async () => {+ const result = await captureSessionLifecycleEvent(+ vault,+ { hook_event_name: "Stop", session_id: "s1" },+ { agent: "tester", now: NOW },+ );+ expect(result.facts_extracted).toBe(0);+ });+});diff --git a/tests/core/config.device-id.test.ts b/tests/core/config.device-id.test.tsnew file mode 100644index 00000000..700b4bab--- /dev/null+++ b/tests/core/config.device-id.test.ts@@ -0,0 +1,65 @@+/**+ * Per-install device identity (Memory Integrity Suite, t_6d52641f).+ *+ * The device id keys the per-device Brain log shards. It lives in the+ * DEVICE-LOCAL config (never the synced vault - all devices sharing+ * one id would defeat the sharding), is generated once on first use,+ * and self-heals to a valid value when hand-edited into an invalid one.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdtempSync, readFileSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { resolveDeviceId } from "../../src/core/config.ts";+import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";++let configHome: string;+let configPath: string;+let savedEnvId: string | undefined;++beforeEach(() => {+ configHome = mkdtempSync(join(tmpdir(), "o2b-device-id-"));+ configPath = join(configHome, "config.yaml");+ savedEnvId = process.env["O2B_DEVICE_ID"];+ delete process.env["O2B_DEVICE_ID"];+});++afterEach(() => {+ rmSync(configHome, { recursive: true, force: true });+ if (savedEnvId === undefined) delete process.env["O2B_DEVICE_ID"];+ else process.env["O2B_DEVICE_ID"] = savedEnvId;+});++describe("resolveDeviceId", () => {+ test("generates an 8-hex id on first use and persists it", () => {+ const id = resolveDeviceId(configPath);+ expect(id).toMatch(/^[a-f0-9]{8}$/);+ expect(readFileSync(configPath, "utf8")).toContain(`device_id: "${id}"`);+ });++ test("returns the same id on every subsequent call", () => {+ const first = resolveDeviceId(configPath);+ const second = resolveDeviceId(configPath);+ expect(second).toBe(first);+ });++ test("respects an existing valid value", () => {+ atomicWriteFileSync(configPath, 'device_id: "vps-main"\n');+ expect(resolveDeviceId(configPath)).toBe("vps-main");+ });++ test("regenerates when the stored value is invalid", () => {+ atomicWriteFileSync(configPath, 'device_id: "NOT/valid id!"\n');+ const id = resolveDeviceId(configPath);+ expect(id).toMatch(/^[a-f0-9]{8}$/);+ expect(readFileSync(configPath, "utf8")).toContain(`device_id: "${id}"`);+ });++ test("rejects a stored value shaped like a sync-conflict marker", () => {+ atomicWriteFileSync(configPath, 'device_id: "sync-conflict-x"\n');+ const id = resolveDeviceId(configPath);+ expect(id).toMatch(/^[a-f0-9]{8}$/);+ });+});diff --git a/tests/core/search/entity-alias.test.ts b/tests/core/search/entity-alias.test.tsnew file mode 100644index 00000000..a70534c0--- /dev/null+++ b/tests/core/search/entity-alias.test.ts@@ -0,0 +1,116 @@+/**+ * Registry-aware entity alias expansion (Memory Integrity Suite).+ *+ * Unit level: a query entity matching a registered alias expands to the+ * canonical name forms (and vice versa). End to end: a query naming an+ * ALIAS boosts the document that names the CANONICAL entity, and the+ * result explains the canonical hop in its reasons.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdirSync, writeFileSync } from "node:fs";+import { join } from "node:path";++import { indexVault, search } from "../../../src/core/search/index.ts";+import { makeConfig, createTempVault } from "../../helpers/search-fixtures.ts";+import { expandQueryEntities } from "../../../src/core/search/entity-alias.ts";+import { archiveEntity, upsertEntity } from "../../../src/core/brain/entities/registry.ts";++let tmp: ReturnType<typeof createTempVault>;++const NOW = new Date("2026-06-02T12:00:00Z");++beforeEach(() => {+ tmp = createTempVault("entity-alias");+});++afterEach(() => {+ tmp.cleanup();+});++function seedRegistry(): void {+ upsertEntity(tmp.vault, {+ category: "projects",+ name: "Open Second Brain",+ aliases: ["OSB", "the vault project"],+ agent: "claude-dev-agent",+ now: NOW,+ });+}++describe("expandQueryEntities", () => {+ test("query naming an alias gains the canonical name forms", () => {+ seedRegistry();+ const out = expandQueryEntities(tmp.vault, ["osb"]);+ expect(out.expanded).toContain("osb");+ expect(out.expanded).toContain("open second brain");+ expect(out.added).toContain("open second brain");+ expect(out.sourceIds).toEqual(["ent-projects-open-second-brain"]);+ });++ test("query naming the canonical name gains the alias forms", () => {+ seedRegistry();+ const out = expandQueryEntities(tmp.vault, ["open second brain"]);+ expect(out.added).toContain("osb");+ });++ test("no registry means identity expansion", () => {+ const out = expandQueryEntities(tmp.vault, ["osb"]);+ expect(out.expanded).toEqual(["osb"]);+ expect(out.added).toEqual([]);+ expect(out.sourceIds).toEqual([]);+ });++ test("archived entities do not expand", () => {+ seedRegistry();+ archiveEntity(tmp.vault, { category: "projects", query: "Open Second Brain" }, { now: NOW });+ const out = expandQueryEntities(tmp.vault, ["osb"]);+ expect(out.added).toEqual([]);+ });+});++describe("alias-aware search boost", () => {+ test("query naming an alias boosts the doc naming the canonical entity", async () => {+ seedRegistry();+ mkdirSync(join(tmp.vault, "notes"), { recursive: true });+ // Both notes carry identical query-term frequencies and identical+ // word counts so their BM25 scores tie exactly (min-max then maps+ // both to 1.0); entity boost is re-ranking only, never retrieval.+ // Only canonical.md names the CANONICAL entity - without registry+ // expansion both docs would get the same one-entity boost from the+ // literal alias and stay tied.+ writeFileSync(+ join(tmp.vault, "notes", "canonical.md"),+ "---\ntitle: Canonical\n---\n\nThe payment flow OSB runs through Open Second Brain to pay memory costs.\n",+ );+ writeFileSync(+ join(tmp.vault, "notes", "plain.md"),+ "---\ntitle: Plain\n---\n\nThe payment flow OSB runs through other second things to pay memory costs.\n",+ );+ const config = makeConfig({ vault: tmp.vault, dbPath: tmp.dbPath, maxHops: 0 });+ await indexVault(config, {});++ const out = await search(config, { query: "payment pay memory OSB", limit: 10 });+ const canonical = out.results.findIndex((r) => r.path === "notes/canonical.md");+ const plain = out.results.findIndex((r) => r.path === "notes/plain.md");+ expect(canonical).toBeGreaterThanOrEqual(0);+ expect(plain).toBeGreaterThanOrEqual(0);+ expect(canonical).toBeLessThan(plain);+ const top = out.results[canonical]!;+ expect(top.reasons.some((x) => x.startsWith("entity_match"))).toBe(true);+ expect(top.reasons.some((x) => x.includes("ent-projects-open-second-brain"))).toBe(true);+ });++ test("search without a registry behaves exactly as before", async () => {+ mkdirSync(join(tmp.vault, "notes"), { recursive: true });+ writeFileSync(+ join(tmp.vault, "notes", "a.md"),+ "---\ntitle: A\n---\n\nThe payment flow runs through Pay Memory to pay memory costs.\n",+ );+ const config = makeConfig({ vault: tmp.vault, dbPath: tmp.dbPath, maxHops: 0 });+ await indexVault(config, {});+ const out = await search(config, { query: "payment Pay Memory", limit: 5 });+ expect(out.results.length).toBeGreaterThan(0);+ expect(out.results[0]!.reasons.some((x) => x.includes("ent-"))).toBe(false);+ });+});diff --git a/tests/mcp/entity-tool.test.ts b/tests/mcp/entity-tool.test.tsnew file mode 100644index 00000000..637aca00--- /dev/null+++ b/tests/mcp/entity-tool.test.ts@@ -0,0 +1,153 @@+/**+ * MCP integration tests for the read-only `brain_entity` tool+ * (Memory Integrity Suite). Writes stay on the CLI; the MCP surface+ * exposes lookup (`view: get`) and listing (`view: list`) only.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdtempSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { JSONRPC_VERSION, MCPServer, PROTOCOL_VERSION } from "../../src/mcp/index.ts";+import { buildToolTable } from "../../src/mcp/tools.ts";+import { bootstrapBrain } from "../../src/core/brain/init.ts";+import { upsertEntity } from "../../src/core/brain/entities/registry.ts";+import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";++let tmp: string;+let vault: string;+let configHome: string;+let configPath: string;+const savedEnv: Record<string, string | undefined> = {};++const NOW = new Date("2026-06-02T12:00:00Z");++beforeEach(() => {+ tmp = mkdtempSync(join(tmpdir(), "o2b-mcp-entity-"));+ vault = join(tmp, "vault");+ configHome = mkdtempSync(join(tmpdir(), "o2b-mcp-entity-cfg-"));+ configPath = join(configHome, "config.yaml");+ for (const k of ["VAULT_AGENT_NAME", "VAULT_TIMEZONE", "VAULT_DIR", "OPEN_SECOND_BRAIN_CONFIG"]) {+ savedEnv[k] = process.env[k];+ delete process.env[k];+ }+ process.env["OPEN_SECOND_BRAIN_CONFIG"] = configPath;+ atomicWriteFileSync(configPath, `vault: ${vault}\nagent_name: claude\n`);+ bootstrapBrain(vault, { configPath });+ upsertEntity(vault, {+ category: "people",+ name: "Ada",+ aliases: ["the operator"],+ agent: "claude",+ now: NOW,+ });+ upsertEntity(vault, {+ category: "projects",+ name: "Open Second Brain",+ agent: "claude",+ now: NOW,+ });+});++afterEach(() => {+ rmSync(tmp, { recursive: true, force: true });+ rmSync(configHome, { recursive: true, force: true });+ for (const [k, v] of Object.entries(savedEnv)) {+ if (v === undefined) delete process.env[k];+ else process.env[k] = v;+ }+});++async function initialize(server: MCPServer): Promise<void> {+ await server.handleRequest({+ jsonrpc: JSONRPC_VERSION,+ id: 1,+ method: "initialize",+ params: {+ protocolVersion: PROTOCOL_VERSION,+ capabilities: {},+ clientInfo: { name: "entity-tool-test", version: "0" },+ },+ });+ await server.handleRequest({+ jsonrpc: JSONRPC_VERSION,+ method: "notifications/initialized",+ });+}++async function callEntity(+ server: MCPServer,+ args: Record<string, unknown>,+): Promise<{ ok: boolean; payload: Record<string, unknown>; error?: string }> {+ const r = (await server.handleRequest({+ jsonrpc: JSONRPC_VERSION,+ id: 7,+ method: "tools/call",+ params: { name: "brain_entity", arguments: args },+ })) as {+ result?: { content: ReadonlyArray<{ type: string; text: string }>; isError?: boolean };+ error?: { message: string };+ };+ if (r.error) return { ok: false, payload: {}, error: r.error.message };+ const text = r.result!.content[0]!.text;+ if (r.result!.isError) return { ok: false, payload: {}, error: text };+ return { ok: true, payload: JSON.parse(text) as Record<string, unknown> };+}++describe("brain_entity registration", () => {+ test("present in the full scope, absent from the writer scope", () => {+ expect(buildToolTable("full").find((t) => t.name === "brain_entity")).toBeDefined();+ expect(buildToolTable("writer").find((t) => t.name === "brain_entity")).toBeUndefined();+ });+});++describe("brain_entity view: get", () => {+ test("resolves an alias to the canonical record", async () => {+ const server = new MCPServer({ vault, configPath });+ await initialize(server);+ const out = await callEntity(server, { view: "get", query: "the operator" });+ expect(out.ok).toBe(true);+ expect(out.payload["id"]).toBe("ent-people-ada");+ expect(out.payload["name"]).toBe("Ada");+ });++ test("reports found:false for an unknown name", async () => {+ const server = new MCPServer({ vault, configPath });+ await initialize(server);+ const out = await callEntity(server, { view: "get", query: "nobody" });+ expect(out.ok).toBe(true);+ expect(out.payload["found"]).toBe(false);+ });++ test("requires query", async () => {+ const server = new MCPServer({ vault, configPath });+ await initialize(server);+ const out = await callEntity(server, { view: "get" });+ expect(out.ok).toBe(false);+ });+});++describe("brain_entity view: list", () => {+ test("lists all entities sorted by id, with category filter", async () => {+ const server = new MCPServer({ vault, configPath });+ await initialize(server);+ const all = await callEntity(server, { view: "list" });+ expect(all.ok).toBe(true);+ expect((all.payload["entities"] as Array<{ id: string }>).map((e) => e.id)).toEqual([+ "ent-people-ada",+ "ent-projects-open-second-brain",+ ]);+ const people = await callEntity(server, { view: "list", category: "people" });+ expect((people.payload["entities"] as Array<{ id: string }>).map((e) => e.id)).toEqual([+ "ent-people-ada",+ ]);+ });++ test("rejects an unknown view", async () => {+ const server = new MCPServer({ vault, configPath });+ await initialize(server);+ const out = await callEntity(server, { view: "explode" });+ expect(out.ok).toBe(false);+ });+});diff --git a/tests/mcp/mcp.test.ts b/tests/mcp/mcp.test.tsindex 38d87f4b..ac6bc233 100644--- a/tests/mcp/mcp.test.ts+++ b/tests/mcp/mcp.test.ts@@ -199,6 +199,8 @@ describe("tool listing", () => {"brain_doctor","brain_health","brain_mcp_landscape",+ // Canonical entity registry read surface (Memory Integrity Suite).+ "brain_entity","brain_backlinks",// Per-preference mutation audit + morning brief (added in v0.21.0)."brain_audit",@@ -492,7 +494,7 @@ describe("stdio loop", () => {// + brain_brief / brain_analytics / schema_inspect (token-diet) = 74// - 18 predecessors hidden as deprecated aliases (callable via// tools/call, not advertised) = 56.- expect(list.result.tools.length).toBe(56);+ expect(list.result.tools.length).toBe(57);});test("returns parse error for invalid JSON", async () => {diff --git a/tests/setup.ts b/tests/setup.tsindex 10d35445..e1b045e0 100644--- a/tests/setup.ts+++ b/tests/setup.ts@@ -20,6 +20,14 @@ import { mkdtempSync, mkdirSync, writeFileSync } from "node:fs";import { tmpdir } from "node:os";import { join } from "node:path";+// Per-device log sharding (Memory Integrity Suite): pin the device id+// to the empty string so the suite writes the legacy un-sharded log+// pair deterministically. Shard-specific tests clear / override+// O2B_DEVICE_ID themselves.+if (process.env["O2B_DEVICE_ID"] === undefined) {+ process.env["O2B_DEVICE_ID"] = "";+}+if (!process.env["OPEN_SECOND_BRAIN_CONFIG"]) {const root = mkdtempSync(join(tmpdir(), "osb-test-default-"));const vault = join(root, "vault");