Local-first 🧠memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly 😴 dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.
TypeScript
feat: Memory Observability Suite - versioned continuity contract, lazy telemetry, ATOF/ATIF export, recall benchmark (#70)
Sol Aitken8e8c0bc35acb
diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.jsonindex 5123ad94..42aaee40 100644--- a/.claude-plugin/plugin.json+++ b/.claude-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "0.38.0",+ "version": "0.39.0","description": "Plugin-first second brain package for AI agents and humans.","author": {"name": "Open Second Brain contributors"diff --git a/.codex-plugin/plugin.json b/.codex-plugin/plugin.jsonindex cc10ad21..dc593c1d 100644--- a/.codex-plugin/plugin.json+++ b/.codex-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "0.38.0",+ "version": "0.39.0","description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.","skills": "./skills","hooks": "./hooks/hooks.json",diff --git a/.gitignore b/.gitignoreindex 2085cc2a..3bdbf047 100644--- a/.gitignore+++ b/.gitignore@@ -25,3 +25,4 @@ sandbox-vault/.codegraph/.orphaned_at.claude/scheduled_tasks.lock+.open-second-brain/bench-runs/diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 5f284c7a..851eb114 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -5,6 +5,70 @@ All notable changes to this project will be documented in this file.The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).+## [0.39.0] - 2026-06-03++Memory Observability Suite: the continuity store becomes a documented,+versioned, exportable contract, and recall quality becomes a measured+number instead of an intuition. Every new continuity record carries a+contract-wide schema version, gated telemetry surfaces route through+one lazy emit kernel that makes "no consumer, no payload work" and+"telemetry never fails the primary operation" structural guarantees, a+read-model normalizes legacy and stamped records identically for every+consumer, trajectories export as standard ATOF/ATIF formats for replay+and eval tooling, and a deterministic memory benchmark reports+quality, latency, and context cost as separate families with+checkpoint/resume. One contract document enumerates every event kind,+gate, correlation id, and safety rule. Everything additive; telemetry+stays default-off.++### Added++- **Versioned continuity schema.** Every new continuity record is+ stamped `schema: "o2b.continuity.v1"`; legacy records without the+ field read as v1 and existing JSONL files are never migrated. The+ dedup id deliberately excludes the stamp, so identical records keep+ identical ids across the transition (locked by a known-answer+ test). Evolution rule documented: additive fields never bump the+ version, renames/removals/semantic changes do.+- **Lazy gated telemetry emit kernel.** One `emitGatedTelemetry(gate,+ build)` helper now carries every gated telemetry surface+ (context-pack receipts and telemetry, pre-compress, `brain_search`+ telemetry, recall-gate telemetry): with the gate off the payload+ thunk is never invoked and nothing reaches the continuity store; a+ throwing thunk or write is swallowed, so telemetry can no longer+ fail a pack, a search, or a gate decision. No-consumer regression+ tests pin the property per surface; session-recall import and+ pre-compact extract stay deliberately fail-fast (their write IS the+ operation).+- **Continuity read-model.** `loadNormalizedContinuityRecords()`+ absorbs schema-version dispatch, lifts `session_id`/`turn_id` into+ first-class fields, drops `private` records by default, and never+ un-masks redacted text - the single normalization layer every+ read-side consumer shares.+- **ATOF/ATIF trajectory export.** `o2b brain continuity export+ --format atof|atif [--session <id>] [--month YYYY-MM]` renders the+ store as standard trajectory formats: ATOF JSONL events (recall+ telemetry as `retriever` scope pairs with a marked synthetic start,+ other kinds as marks) and ATIF v1.7 documents (one per session,+ memory-layer events as `llm_call_count: 0` system steps). Read-only+ over the read-model; private records never reach an export file.+- **Memory quality benchmark.** `o2b brain bench memory --fixture+ <name|path>` ingests a fixture into a disposable vault inside the+ runs directory (never the configured vault), indexes, retrieves,+ evaluates, and reports - quality (pass rate per category), latency,+ and context cost as separate families, never one collapsed score.+ Checkpointed phases resume by run id with a fixture-hash guard;+ shipped fixtures cover single-hop recall, temporal supersession (a+ stale-fact regression catcher), contradiction visibility,+ multi-record evidence, session handoff, and context budget+ truncation. Deterministic and network-free; `bench_judge_cmd` arms+ an optional external judge that is advisory and fail-open.+- **Observability contract.** `docs/observability.md` enumerates every+ Brain log event kind and continuity record kind verified against+ source, the always-on vs opt-in matrix, correlation ids, payload+ safety guarantees, fail-open rules, and the schema version with its+ evolution rule.+## [0.38.0] - 2026-06-03Workspace Insight Suite: eight changes in two themes. Theme A makes@@ -4271,6 +4335,7 @@ plugin config (vault field)`, and exits with a clear- Sandbox vault and plugin manifest fixtures for tests.- GitHub release workflow for tag-based and manually dispatched releases.+[0.39.0]: https://github.com/itechmeat/open-second-brain/compare/v0.38.0...v0.39.0[0.38.0]: https://github.com/itechmeat/open-second-brain/compare/v0.37.0...v0.38.0[0.37.0]: https://github.com/itechmeat/open-second-brain/compare/v0.36.0...v0.37.0[0.36.0]: https://github.com/itechmeat/open-second-brain/compare/v0.35.0...v0.36.0diff --git a/README.md b/README.mdindex b4045ac6..2e76776d 100644--- a/README.md+++ b/README.md@@ -114,7 +114,7 @@ The capabilities you actually feel day to day:These are the headline capabilities. The full surface also includes: importing Claude Code memory directories, daily logging-discipline cron with a complexity-to-thinking warning, cross-project pointers for shared vaults, transient pinned session context through `brain_pinned_context` / `Brain/pinned.md`, source-agent query and comparison (`brain_agent_query`, `brain_agent_diff`, `o2b brain agent-query`, `o2b brain agent-diff`), the codegraph partner skill, vault hygiene lints, per-MOC coverage audit, concept synthesis, an operator dashboard, configurable Markdown-link presentation output, MCP output-contract checks for core agent-facing tools, private-region stripping with `<private>...</private>`, the v0.12.0 Brain Integrity Suite (content-hash drift detection, durable workrun checkpoints, destructive-from-confirmed retire gate, and the `brain_review_candidates` MCP tool that previews what the next dream pass would do without writing anything), the v0.13.0 Hybrid Search and Recall Quality suite (explainable recall, MMR diversity, link-graph traversal, entity-boosted retrieval, header-anchored chunking), the v0.14.0 Semantic Brain Health and Self-Maintenance suite (cross-preference contradiction detection, concept-gap and stale-claim surfacing, per-preference edit-history, a clean/watch/investigate reconciliation verdict via `brain_health`, and dependency-ordered `doctor --remediate`), the v0.17.0 Brain Lifecycle Review Suite (`intent-review`, `retention`, `monthly`, lifecycle schemas, and discipline complexity ratio), and the v0.18.0 MCP context economy (a per-tool preview budget that returns a bounded inline preview and parks the full payload in a vault-local artifact the agent fetches on demand with `brain_artifact_get`, plus a `recall_hint` one-liner on search results), and the v0.19.0 typed graph semantics (typed frontmatter relationships - `related` / `extends` / `contradicts` / `superseded_by` - that surface inline on search results, a `visibility:` scoping axis for `brain_search`, and a `brain_mcp_landscape` view of the Model Context Protocol servers configured across the vault), and the v0.20.0 Recall and Ranking Quality suite (a configurable Weibull recency-decay curve, structural query-intent classification that re-weights ranking by query shape, language-agnostic synonym expansion via local co-occurrence, per-memory and total recall character budgets for `brain_context_pack`, a persistent query cache that self-invalidates on a corpus-generation change, and a `brain_pre_compress_pack` tool that returns a budgeted top-preferences addendum for a host runtime to inject before context compression), and the v0.21.0 Brain Lifecycle Suite (a per-preference mutation audit trail keyed by preference id - create / promote / update / retire / merge with revision + content-hash before/after, via `o2b brain audit` and `brain_audit`; a multi-phase `dream` pipeline that names its close / reconcile / synthesize / heal / log seams and reports a structured per-phase summary without rewriting the proven internals; deterministic reconcile-phase domain classification that buckets contradictions into claims / entity / decisions / source-freshness, auto-resolves only the decisive source-freshness case, and surfaces the rest as open questions instead of forcing a merge; a budgeted `o2b brain morning-brief` / `brain_morning_brief` session-start summary of top preferences, recent open questions, and recent notes; language-agnostic ISO-8601 temporal extraction that fills preference validity windows from signal text; and opt-in heal-phase vault enrichment that links exact title mentions across user pages), and the v0.22.0 Vault portability + session economy suite (a deterministic lossless session codec - `expand(compress(x)) === x` for all input, opt-in on the signal store and exposed as `o2b brain codec`; an `o2b brain sources` / `brain_sources` read-only dashboard of signals by agent and source type; vault-map `{{role}}` tokens resolved via an optional `Brain/_vault-map.yaml` and wired into scan-inline read paths and graph import; named multi-vault profiles with `o2b vault profile` + a `brain_switch_vault` tool, activated by a pointer in config; and `o2b brain graph-export` / `graph-import` for moving a vault's knowledge graph between layouts with skip/overwrite/merge conflict modes). Browse [`docs/cli-reference.md`](docs/cli-reference.md) for every verb and [`docs/how-it-works.md`](docs/how-it-works.md) for the mental model.-The v0.26.0 surface adds audited schema mutation/admin tools, real-time lifecycle capture hooks for prompt markers and `brain_feedback` tool calls, CJK-aware FTS recall, and `brain_watchdog` / `o2b brain watchdog` recovery probes. The v0.27.0 surface adds recall-control and trust tools: structured search query documents, session focus, MCP `brain_recall_gate`, polarity-aware context lanes, and opt-in evidence packs for CLI/MCP search. The v0.28.0 surface adds a deterministic prompt-injection guard for automatically surfaced Brain context, agent-blind `$secret:NAME` references, and preview foundations for source-scoped forget, privacy-scanned knowledge packs, and oversized payload externalization. The v0.29.0 surface adds opt-in context receipts and recall telemetry, read-only context budget presets, pre-compaction extraction, and a continuity-backed session recall DAG. The v0.33.0 surface adds the Recall Trust Suite: relation-aware recall polarity for typed memory edges, an explicit retrieval feedback loop with bounded learned weights (`o2b search feedback` / `weights`, MCP `brain_recall_feedback`), time-aware recall (`--since` / `--until` with natural-language ranges), verified multi-record recall (per-token union records, IDF-weighted coverage, rare-term abstention), and a search-completeness verdict with a false-absence guard. The v0.34.0 surface adds the Token Diet: a budgeted SessionStart injection of `Brain/active.md` (default 8,000 chars, `active.inject_budget_chars`) with the post-compaction path moved to the supported SessionStart `compact` matcher, a once-per-session post-write reminder cadence, frontmatter escape-amplification fixed at the parser with a one-shot `o2b brain upgrade --apply` repair for already-corrupted preference files, three consolidated read tools (`brain_brief`, `brain_analytics`, `schema_inspect`) with 18 hidden deprecated aliases, registry description caps with a contract guard, preview budgets by default, and a `scripts/measure-token-surface.ts` report. The v0.35.0 surface adds the Memory Integrity Suite: a canonical entity registry under `Brain/entities/` (one Markdown entity per `(category, name)` with alias resolution, archive/restore, typed relations, `o2b brain entity` CLI verbs, a read-only `brain_entity` MCP tool, doctor duplicate/relation lints, and alias-aware search boost with `entity_canonical` explanations), per-device Brain log shards (`Brain/log/<date>.<deviceId>.jsonl` + `.md` keyed by a device-local id, merged reads through `readLogDay`/`listLogDates`, no more Syncthing write conflicts, legacy days untouched), capture boundaries (a `sessions:` block with ignored/stateless session globs and message suppression regexes, applied before any extraction at both live-hook and import seams, counted but never stored raw; keep suppression regexes simple - they run on every message, and a catastrophic-backtracking pattern slows capture), and deterministic regex fact extraction (seven precision-first families over user turns, `source_type: extracted` signals with family-scoped dedup and canonical entity anchors). The v0.36.0 surface adds the Embedding Provider Suite: an offline `local` embedding provider (deterministic feature hashing, no cloud, no key, no model download - set `embedding_provider: local` for a privacy-first recall path), a CLI provider registry (`o2b search provider add | list | show | remove`, persisted to `Brain/search/embedding-providers.json` storing only environment-variable names, resolved after the built-ins so a configured key is never shadowed), an embedding cost gate (`embedding_cost_gate_usd`, default 0 = off; refuses an over-budget run unless `o2b search index --force-cost`) with the active `<provider>:<model>:<dimension>` signature and a refresh-cost estimate in `o2b search status`, and an opt-in Reciprocal Rank Fusion mode (`search_fusion_mode: rrf`, `search_rrf_k` default 60) that fuses the keyword and semantic lanes by rank position instead of weighted magnitude while the default `linear` mode stays bit-identical and storage remains SQLite + sqlite-vec + FTS5. The v0.37.0 surface adds the Agent Surface Suite: skills as callable MCP tools (`list_skills` / `get_skill` over the shipped `skills/` plus vault-local `Brain/skills/`), two-pass tool catalog hydration (a compact `catalog` scope where `tool_hydrate` returns the sorted catalog and per-name full schemas while every tool stays callable), five named tool-surface profiles with fail-open fallback (`mcp_tool_profile` / `o2b mcp --tool-profile`), deterministic BM25-style skill auto-attach (`skills_attach`, gated by `skill_auto_attach`), config-level capture role filtering (`session_capture_roles`), session-scoped search focus with SessionEnd auto-clear and gated context-pack promotion (`--session`, `focus_session`, `search_focus_context_pack`), operator-readable handoff notes (`o2b brain handoff`, gated SessionEnd generation via `session_handoff`), and scoped current-intention chains with a move-to-history lifecycle (`o2b brain intention`, MCP `brain_intention`). The v0.38.0 surface adds the Workspace Insight Suite: project vault pointers (`o2b brain project link | list | remove | status` writes a `.o2b-vault.json` into any project directory and `resolveVault` honours it from anywhere under that tree), read-only recall sources (`o2b brain source add | list | remove` with self/duplicate/circular validation and BROKEN flagging), cross-vault union search (`o2b search <query> --global`, `brain_search { global: true }` - origin-labelled results, external vaults never written to), a configurable wikilink path format (`wiki_link_format`: preserve/full/short plus `o2b brain links normalize`), a shell-native surface (`o2b brain profile` materializes `Brain/profile.md` + a `.o2bfs` root marker; `o2b brain sgrep` is grep-shaped semantic search), a grounded trigger queue with an anti-nag lifecycle (`o2b brain trigger scan | list | ack | dismiss | act | history`, MCP `brain_trigger`, cooldown-key dedup, morning-brief delivery once per `trigger_cooldown_days`), deep vault synthesis (`o2b brain deep-synthesis <topic>`, MCP `brain_deep_synthesis` - a deterministic dossier of agreements, contradictions, stale claims, and knowledge gaps), idea discovery (`o2b brain ideas`, MCP `brain_idea_discovery`), and opt-in recall-gate telemetry (`recall_gate_telemetry`, hashed prompts only, gate_list / gate_summary operations).+The v0.26.0 surface adds audited schema mutation/admin tools, real-time lifecycle capture hooks for prompt markers and `brain_feedback` tool calls, CJK-aware FTS recall, and `brain_watchdog` / `o2b brain watchdog` recovery probes. The v0.27.0 surface adds recall-control and trust tools: structured search query documents, session focus, MCP `brain_recall_gate`, polarity-aware context lanes, and opt-in evidence packs for CLI/MCP search. The v0.28.0 surface adds a deterministic prompt-injection guard for automatically surfaced Brain context, agent-blind `$secret:NAME` references, and preview foundations for source-scoped forget, privacy-scanned knowledge packs, and oversized payload externalization. The v0.29.0 surface adds opt-in context receipts and recall telemetry, read-only context budget presets, pre-compaction extraction, and a continuity-backed session recall DAG. The v0.33.0 surface adds the Recall Trust Suite: relation-aware recall polarity for typed memory edges, an explicit retrieval feedback loop with bounded learned weights (`o2b search feedback` / `weights`, MCP `brain_recall_feedback`), time-aware recall (`--since` / `--until` with natural-language ranges), verified multi-record recall (per-token union records, IDF-weighted coverage, rare-term abstention), and a search-completeness verdict with a false-absence guard. The v0.34.0 surface adds the Token Diet: a budgeted SessionStart injection of `Brain/active.md` (default 8,000 chars, `active.inject_budget_chars`) with the post-compaction path moved to the supported SessionStart `compact` matcher, a once-per-session post-write reminder cadence, frontmatter escape-amplification fixed at the parser with a one-shot `o2b brain upgrade --apply` repair for already-corrupted preference files, three consolidated read tools (`brain_brief`, `brain_analytics`, `schema_inspect`) with 18 hidden deprecated aliases, registry description caps with a contract guard, preview budgets by default, and a `scripts/measure-token-surface.ts` report. The v0.35.0 surface adds the Memory Integrity Suite: a canonical entity registry under `Brain/entities/` (one Markdown entity per `(category, name)` with alias resolution, archive/restore, typed relations, `o2b brain entity` CLI verbs, a read-only `brain_entity` MCP tool, doctor duplicate/relation lints, and alias-aware search boost with `entity_canonical` explanations), per-device Brain log shards (`Brain/log/<date>.<deviceId>.jsonl` + `.md` keyed by a device-local id, merged reads through `readLogDay`/`listLogDates`, no more Syncthing write conflicts, legacy days untouched), capture boundaries (a `sessions:` block with ignored/stateless session globs and message suppression regexes, applied before any extraction at both live-hook and import seams, counted but never stored raw; keep suppression regexes simple - they run on every message, and a catastrophic-backtracking pattern slows capture), and deterministic regex fact extraction (seven precision-first families over user turns, `source_type: extracted` signals with family-scoped dedup and canonical entity anchors). The v0.36.0 surface adds the Embedding Provider Suite: an offline `local` embedding provider (deterministic feature hashing, no cloud, no key, no model download - set `embedding_provider: local` for a privacy-first recall path), a CLI provider registry (`o2b search provider add | list | show | remove`, persisted to `Brain/search/embedding-providers.json` storing only environment-variable names, resolved after the built-ins so a configured key is never shadowed), an embedding cost gate (`embedding_cost_gate_usd`, default 0 = off; refuses an over-budget run unless `o2b search index --force-cost`) with the active `<provider>:<model>:<dimension>` signature and a refresh-cost estimate in `o2b search status`, and an opt-in Reciprocal Rank Fusion mode (`search_fusion_mode: rrf`, `search_rrf_k` default 60) that fuses the keyword and semantic lanes by rank position instead of weighted magnitude while the default `linear` mode stays bit-identical and storage remains SQLite + sqlite-vec + FTS5. The v0.37.0 surface adds the Agent Surface Suite: skills as callable MCP tools (`list_skills` / `get_skill` over the shipped `skills/` plus vault-local `Brain/skills/`), two-pass tool catalog hydration (a compact `catalog` scope where `tool_hydrate` returns the sorted catalog and per-name full schemas while every tool stays callable), five named tool-surface profiles with fail-open fallback (`mcp_tool_profile` / `o2b mcp --tool-profile`), deterministic BM25-style skill auto-attach (`skills_attach`, gated by `skill_auto_attach`), config-level capture role filtering (`session_capture_roles`), session-scoped search focus with SessionEnd auto-clear and gated context-pack promotion (`--session`, `focus_session`, `search_focus_context_pack`), operator-readable handoff notes (`o2b brain handoff`, gated SessionEnd generation via `session_handoff`), and scoped current-intention chains with a move-to-history lifecycle (`o2b brain intention`, MCP `brain_intention`). The v0.38.0 surface adds the Workspace Insight Suite: project vault pointers (`o2b brain project link | list | remove | status` writes a `.o2b-vault.json` into any project directory and `resolveVault` honours it from anywhere under that tree), read-only recall sources (`o2b brain source add | list | remove` with self/duplicate/circular validation and BROKEN flagging), cross-vault union search (`o2b search <query> --global`, `brain_search { global: true }` - origin-labelled results, external vaults never written to), a configurable wikilink path format (`wiki_link_format`: preserve/full/short plus `o2b brain links normalize`), a shell-native surface (`o2b brain profile` materializes `Brain/profile.md` + a `.o2bfs` root marker; `o2b brain sgrep` is grep-shaped semantic search), a grounded trigger queue with an anti-nag lifecycle (`o2b brain trigger scan | list | ack | dismiss | act | history`, MCP `brain_trigger`, cooldown-key dedup, morning-brief delivery once per `trigger_cooldown_days`), deep vault synthesis (`o2b brain deep-synthesis <topic>`, MCP `brain_deep_synthesis` - a deterministic dossier of agreements, contradictions, stale claims, and knowledge gaps), idea discovery (`o2b brain ideas`, MCP `brain_idea_discovery`), and opt-in recall-gate telemetry (`recall_gate_telemetry`, hashed prompts only, gate_list / gate_summary operations). The v0.39.0 surface adds the Memory Observability Suite: a contract-wide continuity schema version (`o2b.continuity.v1`, legacy records read as v1, dedup ids unchanged), a lazy gated telemetry emit kernel with structural no-consumer and fail-open guarantees across every gated surface, a continuity read-model with private-record masking shared by all read-side consumers, ATOF/ATIF trajectory export (`o2b brain continuity export --format atof|atif` - read-only, redaction honoured), a deterministic memory quality benchmark (`o2b brain bench memory` with checkpoint/resume and separate quality/latency/context-cost reporting), and the `docs/observability.md` contract enumerating every event kind, gate, and safety rule.## CLIdiff --git a/docs/brainstorm/memory-observability-suite/atof-atif-mapping.md b/docs/brainstorm/memory-observability-suite/atof-atif-mapping.mdnew file mode 100644index 00000000..b42081f2--- /dev/null+++ b/docs/brainstorm/memory-observability-suite/atof-atif-mapping.md@@ -0,0 +1,68 @@+# ATOF/ATIF mapping for Open Second Brain continuity records++**Status:** assessed - GO+**Task:** t_51959aeb (Memory Observability Suite)+**Specs matched:** ATOF event format (NVIDIA NeMo Agent Toolkit, `packages/nvidia_nat_atif/atof-event-format.md`, version family 0.x); ATIF v1.7 (harbor-framework RFC 0001)++## Why a mapping document first++Open Second Brain is a memory layer, not the agent loop. The trajectory it can faithfully export is not "agent thought, tool ran" but "turn happened, recall gate decided, context pack was served, receipts were written". The question this document answers: do ATOF and ATIF have room for that shape without abusing their vocabulary? Verdict: yes, with the deliberate interpretations listed below.++## Source material: continuity record kinds++| O2B kind | Content | Notes |+|---|---|---|+| `session_turn` | session_id, turn_id, role, text, text_hash | imported conversation turns |+| `session_summary_node` | depth, summary, source_record_ids | derived summary tree |+| `recall_telemetry` | host, mode, status, duration_ms, result_count, top_artifacts, gaps | opt-in per call |+| `gate_telemetry` | host, decision, reason, prompt_hash, prompt_chars | config-gated; raw prompt never stored |+| `context_receipt` | trigger, host, items, budget | opt-in per call |+| `pre_compact_extract` | extracted lines | explicit operation |+| `source_invalidation` | source ref, reason | always-on |++All records share `id` (sha-256 dedup), `createdAt`, `sourceRefs`, `private`/`redacted` flags, and (since this suite) `schema: o2b.continuity.v1`.++## ATOF mapping (JSONL event stream)++ATOF events require `kind` (`scope`/`mark`), `atof_version`, `uuid`, `timestamp`, `name`; scope events add `category` (closed vocabulary) and `attributes`, plus paired start/end sharing one `uuid`.++| O2B kind | ATOF shape | category | Rationale |+|---|---|---|---|+| `recall_telemetry` | `scope` start/end pair | `retriever` | the one record with a duration; start synthesized as `createdAt - duration_ms` |+| `gate_telemetry` | `mark` | `guardrail` | point-in-time decision |+| `context_receipt` | `mark` | `retriever` | snapshot of served context |+| `session_turn` | `mark` | `custom` (`category_profile.subtype: "o2b.session_turn"`) | turns are events to a memory layer, not scopes it owns |+| `session_summary_node`, `pre_compact_extract`, `source_invalidation` | `mark` | `custom` with subtype | derived/maintenance events |++Deliberate interpretations, all marked in event `attributes`:++- **Synthetic scope start**: O2B records one row per recall with `duration_ms`; the exporter back-computes the start timestamp. Events carry `o2b.synthetic_start: true` so consumers can tell recorded facts from interpolation.+- **Flat hierarchy**: `parent_uuid` stays null - the memory layer does not know the agent's call tree. Correlation rides `session_id` / `turn_id` attributes.+- **UUIDs** derive deterministically from the record `id` (UUIDv5-style sha-256 fold), so re-exports are reproducible and diffable.+- `atof_version: "0.1"`; the exporter emits the lowest version whose fields it uses.++## ATIF mapping (one JSON document per session)++ATIF v1.7 requires `schema_version`, `agent`, `steps[]`; steps require `step_id` (1-based), `source` (`system`/`user`/`agent`), `message`.++- One trajectory per `session_id`; records without a session are skipped by the ATIF renderer (they have no step ordering) - the export summary counts them.+- `session_turn` becomes a step: `role: "user"` maps to `source: "user"`, everything else to `source: "agent"`; `message` carries the turn text (already redaction-masked at write time).+- Memory-layer events (`gate_telemetry`, `recall_telemetry`, `context_receipt`) become `source: "system"` steps with `llm_call_count: 0` - the spec's explicit marker for deterministic dispatch - and the full payload under `extra.o2b`.+- `agent`: `{ name: "open-second-brain", version: <package version> }`; the memory layer is the recording agent, the host agent is unknown to it.+- Steps order by `createdAt`, then record id (the store's stable order); `step_id` re-numbers from 1.+- `subagent_trajectories` unused - O2B sessions are flat.+- `schema_version: "ATIF-v1.7"`.++## Privacy and redaction++The exporter consumes the continuity read-model, never the raw store:++- records flagged `private` are dropped before any rendering (the read-model default);+- payload text was redaction-masked at write time (`***REDACTED***`, `***PRIVATE***`) and the read-model never un-masks;+- `gate_telemetry` carries only `prompt_hash`/`prompt_chars` by construction - no raw prompt can reach an export because none is stored.++## Go / no-go++**GO.** Both formats accommodate memory-layer traces without vocabulary abuse: ATOF's `mark` + `custom` subtype carries every kind, scope pairs fit the one duration-bearing kind, and ATIF's `llm_call_count: 0` system steps are exactly the deterministic-dispatch shape O2B produces. The deliberate interpretations (synthetic scope start, flat hierarchy, memory layer as recording agent) are documented above and marked in the output itself.++Golden-file tests pin the exporter's output; this document records the spec revisions matched, so a future spec bump is a conscious re-assessment, not silent drift.diff --git a/docs/brainstorm/memory-observability-suite/cli-output/claude.md b/docs/brainstorm/memory-observability-suite/cli-output/claude.mdnew file mode 100644index 00000000..4b9a102a--- /dev/null+++ b/docs/brainstorm/memory-observability-suite/cli-output/claude.md@@ -0,0 +1,37 @@+### Variant 1: Single observability kernel+- **Approach**: Introduce one shared `src/core/brain/observability/` kernel that owns the schema-version constant (`o2b.continuity.v1`), the `emitTelemetry(kind, gate, lazyPayload)` gate helper, and a record-normalization function. Tasks 1+2 land entirely inside it; every existing writer (`context-receipts`, `session-recall`, `recall-telemetry`, `gate-telemetry`, lifecycle) is refactored to route through the helper. Tasks 4 (export) and 5 (bench) become read-only consumers that import the kernel's normalize/version logic so all three surfaces agree on record shape.+- **Trade-offs**:+ - Pro: Matches the established kernel-first convention (suites #64–#69); the lazy-after-gate shape becomes structurally enforced, not conventional.+ - Pro: Single source of truth for schema version + evolution rule, which Task 3's doc and Task 4's export both depend on.+ - Pro: Easiest place to hang the no-consumer regression tests and fail-open guarantees.+ - Con: Refactoring six call sites in one PR widens the blast radius and risks touching the lifecycle hot path.+ - Con: Up-front abstraction cost before Task 4's fit assessment is even known to pay off.+- **Complexity**: large+- **Risk**: medium++### Variant 2: Thin additive per-task patches+- **Approach**: Keep each task local with no new shared module. Add the `schema` field directly in `buildRecord()`, add `emitTelemetry` as a small helper beside the continuity store, write the doc, and add `export` and `bench` as independent CLI verbs that each read the JSONL store directly. Call sites are audited and fixed in place but not rerouted through a unifying layer.+- **Trade-offs**:+ - Pro: Smallest diff, lowest chance of regressing the lifecycle hot path or dedup hashing.+ - Pro: Task 4 can be cheaply abandoned (write-up only) without unwinding shared infrastructure.+ - Pro: Fast to land; each task is independently reviewable.+ - Con: The "no consumer => no payload work" property stays convention-enforced at most sites; drift is likely to recur in future suites.+ - Con: Export and bench each reimplement JSONL reading and version handling, duplicating normalization logic.+ - Con: Cuts against the project's kernel-first direction.+- **Complexity**: medium+- **Risk**: low++### Variant 3: Split write-emit kernel + read-model layer+- **Approach**: Build two narrow layers instead of one. A write-side emit kernel covers Tasks 1+2 (schema stamping at `buildRecord`, `emitTelemetry` thunk gate, fail-open). A separate read-side "trajectory read-model" normalizes JSONL records into a canonical in-memory shape with schema-version dispatch, and both Task 4 (ATOF/ATIF export) and Task 5 (bench retrieval scoring) consume only that read-model — never the raw store.+- **Trade-offs**:+ - Pro: Clean separation of fail-fast write concerns from fail-soft read concerns, matching the project's read/write posture.+ - Pro: The read-model absorbs schema-version branching once, so export and bench can't disagree on legacy-record handling or private/redacted masking.+ - Pro: Gated emit kernel is small and low-risk; the larger read-model is purely additive and read-only.+ - Con: Two kernels to design and document instead of one; some conceptual overlap (version constant referenced on both sides).+ - Con: The read-model is partly speculative until Task 4's fit assessment confirms ATOF/ATIF is worth building.+ - Con: More moving parts than a single kernel for reviewers to hold.+- **Complexity**: large+- **Risk**: medium++### Recommended: Variant 3+**Rationale**: The five tasks split cleanly along the project's own fail-fast-write / fail-soft-read seam, and a shared read-model is the one place that guarantees export and bench handle legacy unversioned records and private/redacted masking identically — the highest-leverage correctness invariant in the epic. The write-emit kernel stays small enough to keep the lifecycle hot path safe, while the read layer remains purely additive and can be scoped down to a write-up artifact if Task 4's ATOF/ATIF fit assessment comes back negative, without stranding the rest of the suite.diff --git a/docs/brainstorm/memory-observability-suite/cli-output/prompt.md b/docs/brainstorm/memory-observability-suite/cli-output/prompt.mdnew file mode 100644index 00000000..10b20bb3--- /dev/null+++ b/docs/brainstorm/memory-observability-suite/cli-output/prompt.md@@ -0,0 +1,106 @@+You are brainstorming architectural variants for the following task. Do not write code. Do not write a final design. Only produce variants and a recommendation.++# Task++Epic: Memory Observability Suite for Open Second Brain (target v0.39.0). Five kanban tasks ship in one PR:++## Task 1 (t_26040ee8, p2): Versioned telemetry schema for continuity records++The continuity store (`src/core/brain/continuity/store.ts`) writes JSONL records to `Brain/log/continuity/<month>.jsonl` with `id`, `kind`, `createdAt`, `sourceRefs`, `payload`, `private`, `redacted` - but no schema version. Consumers cannot detect when a record kind's payload shape evolves.++- Add a `schema` field to `ContinuityRecord` (`src/core/brain/continuity/types.ts`), stamped at `buildRecord()` - e.g. `o2b.continuity.v1`.+- Old records without the field are implicitly v1; readers treat `undefined` as v1 (no migration of existing JSONL).+- Decide versioning granularity: one version for the whole store vs per-kind versions. Upstream (hermes-agent PR #38232) uses one contract-wide version - simplest, recommended.+- Document the evolution rule: additive fields do not bump the version; renames/removals/semantic changes do.+- Tests: new records carry the version; readers accept legacy records without it; `recordId()` hash either includes or deliberately excludes the version (pick one, test it - including it would change dedup IDs for otherwise-identical records).++## Task 2 (t_5d7aa7c5, p1): Lazy gated telemetry emit helper with no-consumer regression tests++Telemetry opt-in gates live at each call site (`if (resolveRecallGateTelemetry(...))` in `src/mcp/search-tools.ts`, `telemetry` options in `src/core/brain/context-pack.ts` / `pre-compress-pack.ts`, lifecycle capture in `src/core/brain/session-lifecycle.ts`). The property "no consumer => no payload work" is enforced by convention, not by structure or tests.++1. Audit every `appendContinuityRecord` / `captureSessionLifecycleEvent` / telemetry-emit call site for eager payload construction ahead of the gate. Fix any found.+2. Centralize the gate: a small `emitTelemetry(kind, gate, lazyPayload)` helper (payload as a thunk) so the lazy-after-gate shape is the default.+3. No-consumer regression tests: for each gated surface, a test asserting that with telemetry off, the payload thunk is never invoked and no continuity write happens.+4. Keep fail-open semantics: a throwing payload thunk or write must never fail the primary operation - assert that too.++Known call sites of appendContinuityRecord: context-receipts.ts, pre-compact-extract.ts, session-recall.ts (2 sites), recall-telemetry.ts (emitRecallTelemetry), gate-telemetry.ts (emitGateTelemetry).++## Task 3 (t_66545537, p2): Observability contract doc++Write `docs/observability.md`: event families (Brain/log/<date>.md log events via appendLogEvent in src/core/brain/log.ts; continuity record kinds; session lifecycle capture), correlation IDs (session_id, turn_id, createdAt, record id sha-256 dedup, sourceRefs), payload safety (private-region stripping, secret redaction in continuity/redaction.ts, deep-freeze, private/redacted flags), fail-open rules, always-on vs opt-in status per event kind verified against source. Doc only - no behavior changes, but it documents the schema version + evolution rule from Task 1.++## Task 4 (t_51959aeb, p1): ATOF/ATIF trajectory export for continuity session records (exploration first)++1. Assess fit: map continuity record kinds (session_turn, recall_telemetry, gate_telemetry, context_receipt) onto ATOF event kinds (scope/mark) and ATIF steps/observations. O2B is a memory layer, not the agent loop - the interesting trajectory is "turn -> recall gate decision -> context pack served -> receipts". Write up the mapping before any code.+2. If the fit is real: `o2b continuity export --format atof|atif [--session <id>] [--month <m>]` CLI producing spec-conformant files. Read-only over the JSONL store; no new emission paths.+3. Honor private/redacted record flags in the export (drop or mask, never leak).+4. Validate output against the published format specs (golden-file tests with sanitized fixtures).+If the mapping write-up shows the formats are a bad fit, close with the write-up as the artifact.++## Task 5 (t_882c396a, p2): Memory quality benchmark harness with MemScore-style reporting++Add an OSB-local benchmark harness for memory quality regression testing (inspired by supermemory MemoryBench):++- `o2b brain bench memory` with phases: ingest fixture, build index/dream, run retrieval/context-pack, answer with configured judge/model or deterministic expected-context mode, evaluate, report.+- Store runs under a local ignored directory such as `.open-second-brain/bench-runs/` with `checkpoint.json`, per-question results, and `report.json`.+- Track quality/pass rate, recall latency, average context tokens/chars, missing-ground-truth cases, stale-fact regressions. Keep quality / latency / context-cost as separate numbers (MemScore triple), never one opaque score.+- Fixture categories: single-hop preference recall, temporal updates, contradiction/supersession, multi-record evidence, session handoff, context budget truncation.+- Reports diffable so PRs can show whether recall quality improved or regressed.+- A benchmark run can resume from a checkpointed run ID without repeating completed phases.+- The harness can run in a deterministic no-network mode for CI smoke coverage.+- Optional judge-model evaluation clearly gated by config and skipped when unavailable.++# Project context++Open Second Brain - TypeScript on Bun, plus a Python provider layer. Markdown/Obsidian vault as storage, SQLite FTS5 index for search, MCP server (65 advertised tools) + `o2b` CLI.++Recent commits:+eb56c9f feat: Workspace Insight Suite - project pointers, cross-vault recall, trigger queue, proactive synthesis (#69)+707197a feat: Agent Surface Suite - skills over MCP, two-pass tool catalog, surface profiles, session lifecycle (#68)+eda202d feat: Embedding Provider Suite - local embedder, provider registry, cost gate, RRF fusion (#67)+dee5ab7 feat: Memory Integrity Suite - canonical entities, conflict-free log shards, capture boundaries, fact extraction (#66)+5066e71 feat: Token Diet - budgeted injection, reminder cadence, consolidated MCP surface (#65)+c494fd8 feat(search): Recall Trust Suite - relation polarity, learned weights, time-aware and verified recall (#64)++Related files:+- src/core/brain/continuity/{store,types,redaction}.ts - JSONL continuity store, buildRecord, safeContinuityPayload+- src/core/brain/recall-telemetry.ts, gate-telemetry.ts - emit/list/summarize over continuity records+- src/core/brain/session-lifecycle.ts - per-tool-use lifecycle capture (hot path)+- src/core/brain/context-receipts.ts, pre-compact-extract.ts, session-recall.ts - other continuity writers+- src/core/brain/context-pack.ts (packContext), src/core/search/search.ts (search) - recall surfaces the bench would drive+- src/core/brain/log.ts (appendLogEvent) - daily Markdown log + JSONL sidecar+- src/cli/brain/verbs/ - one file per CLI verb, registration in verbs/index.ts + help-text.ts + command-manifest.ts+- src/mcp/{tools,brain-tools,search-tools}.ts - MCP tool table; registry-guard test caps description length at 160 chars; advertised-tool-count contract test++Conventions:+- Deterministic core: no LLM calls inside Brain core; judge-model steps must be external/optional+- Everything additive: existing CLI verbs, MCP tool shapes, search result fields never change shape; new behavior off by default or per-call explicit+- Fail-soft on read paths, fail-fast on mutations; external/secondary writes never break the primary operation+- TDD with bun test (3576 tests), oxlint baseline 111 warnings, oxfmt formatter, tsc strict, no `as` casts to silence types+- Kernel-first: shared kernels per suite so features cannot drift (see recent suites)+- snake_case keys in on-disk JSON/frontmatter and MCP payloads; camelCase in TS++Constraints:+- No new external dependencies unless unavoidable+- No migration of existing JSONL continuity files+- recordId() dedup-hash compatibility must be considered explicitly (Task 1)+- Telemetry stays default-off; raw prompts never persisted+- Benchmark must not write into a user vault; it runs against disposable fixture vaults+- MCP contract: new tools must keep description properties under 160 chars; advertised count test updated deliberately++# Required output format++Produce exactly 3 distinct architectural variants. For each variant:++### Variant N: <short name>+- **Approach**: 2-3 sentences describing the variant.+- **Trade-offs**: bullet list of pros and cons.+- **Complexity**: small | medium | large+- **Risk**: low | medium | high++After the three variants, add exactly one recommendation:++### Recommended: Variant N+**Rationale**: 2-3 sentences explaining why this variant over the others, considering the project context and constraints above.++Output nothing outside of these sections.diff --git a/docs/brainstorm/memory-observability-suite/design.md b/docs/brainstorm/memory-observability-suite/design.mdnew file mode 100644index 00000000..fe702a32--- /dev/null+++ b/docs/brainstorm/memory-observability-suite/design.md@@ -0,0 +1,94 @@+# Memory Observability Suite - versioned continuity contract, lazy telemetry, trajectory export, recall benchmark++**Status:** draft+**Author:** claude-dev-agent (via feature-release-playbook)+**Audience:** implementation++## Problem statement++The continuity store is the de-facto observability surface of the Brain - recall telemetry, gate decisions, context receipts, session turns all land in `Brain/log/continuity/<month>.jsonl` - but the contract is implicit. Records carry no schema version, the "no consumer means no payload work" property is enforced by convention only, no document says which events exist or which are opt-in, traces are locked in a proprietary shape, and there is no way to measure whether a recall change improved or regressed memory quality.++## Scope++- **Versioned continuity schema** (t_26040ee8): every new record stamped `schema: "o2b.continuity.v1"`; legacy records read as v1; evolution rule documented.+- **Lazy gated emit kernel** (t_5d7aa7c5): one `emitGatedTelemetry` helper with payload-as-thunk, fail-open semantics, no-consumer regression tests across all gated surfaces.+- **Continuity read-model**: one normalization layer (schema-version dispatch, legacy defaults, private/redacted masking policy) consumed by export and bench - never the raw store.+- **ATOF/ATIF trajectory export** (t_51959aeb): mapping write-up first; `o2b brain continuity export --format atof|atif` read-only over the JSONL store; private/redacted honored; golden-file tests.+- **Memory quality benchmark harness** (t_882c396a): `o2b brain bench memory` - ingest fixture vault, index, retrieve, evaluate, report; checkpoint/resume by run ID; quality/latency/context-cost reported separately; deterministic no-network mode for CI; optional external judge command.+- **Observability contract doc** (t_66545537): `docs/observability.md` - event families, correlation IDs, payload safety, fail-open rules, opt-in matrix, schema version + evolution rule.++## Out of scope++- Migration or rewriting of existing JSONL continuity files.+- New telemetry emission paths (export and bench are read-only over the store; bench writes only to its own run directory and disposable fixture vaults).+- Hosted telemetry, dashboards, or public benchmark score claims.+- A judge LLM inside the Brain core - judge evaluation is an external optional command.+- Per-origin score normalization, MCP export tools (export produces files; CLI is the right surface).++## Chosen approach++Consultant Variant 3: split write-emit kernel + read-model layer.++Two narrow layers along the project's fail-fast-write / fail-soft-read seam. The write-side emit kernel lives inside `src/core/brain/continuity/` and owns the schema-version constant (stamped in `buildRecord`) and the lazy gate helper (`emit.ts`): payload thunks run only after the gate passes, and a throwing thunk or write never fails the primary operation. The read-side read-model (`read-model.ts`) normalizes raw JSONL records into one canonical shape - schema-version dispatch with `undefined` read as v1, plus a masking policy for `private`/`redacted` records - and both the ATOF/ATIF exporter and the bench harness consume only that read-model, so they cannot disagree on legacy-record handling or redaction. The bench harness is a separate `src/core/bench/` module driving the public `search`/`packContext` APIs against disposable fixture vaults.++Refinements over the consultant's sketch:++- The read-model is not speculative: the bench harness needs normalized continuity records for its session-handoff fixtures regardless of the ATOF/ATIF go/no-go, so the layer pays off even in the no-go branch.+- The export verb lives in the `o2b brain` namespace (`o2b brain continuity export`) like every other Brain verb, not as a new top-level command.+- The ATOF/ATIF fit assessment ran during phase 0 against the published specs: the mapping is viable (see `docs/brainstorm/memory-observability-suite/atof-atif-mapping.md` summary below) - go.++## ATOF/ATIF fit assessment (go)++Verified against the published specs (NVIDIA NeMo ATOF event format; harbor ATIF RFC 0001 v1.7):++- **ATOF** is JSONL with `scope` (paired start/end, shared `uuid`) and `mark` (point-in-time) events; required fields `kind`, `atof_version`, `uuid`, `timestamp`, `name`; scope events add `category` (closed vocabulary including `retriever`, `custom`) and `attributes`. Mapping: `recall_telemetry` becomes a `retriever` scope pair (start synthesized as `createdAt - duration_ms`, end at `createdAt`); `gate_telemetry`, `context_receipt`, `pre_compact_extract`, and `source_invalidation` become `mark` events with `category_profile.subtype` carrying the O2B kind; `session_turn` becomes a `custom` mark. `parent_uuid` left null - O2B records are flat; correlation rides `session_id` in attributes.+- **ATIF v1.7** is a JSON document with `schema_version`, `agent`, `steps[]`; steps have `step_id`, `source` (`system`/`user`/`agent`), `message`, optional `observation`/`extra`. Mapping: one trajectory per `session_id`; `session_turn` records become steps with `source` mapped from `role` (`user` to `user`, everything else to `agent`); memory-layer events (gate decision, recall, receipt) become `system` steps with `llm_call_count: 0` - the spec's explicit marker for deterministic dispatch - and payload in `extra`. Subagent embedding unused (flat sessions).+- Both formats honor redaction by construction: the exporter consumes the read-model with masking applied, and records flagged `private` are dropped while `redacted` payload text passes through already masked.++## Design decisions++- **One contract-wide version, not per-kind** - `o2b.continuity.v1`, matching the upstream pattern; simplest rule for consumers; per-kind granularity can be added later inside the payload if ever needed.+- **`recordId()` deliberately EXCLUDES the schema field** - identical records must dedupe identically across the version-stamp transition; including it would silently change every dedup ID with no migration. Locked by test.+- **Constant lives in `continuity/types.ts`** (`CONTINUITY_SCHEMA_VERSION`) so both the write kernel and the read-model import one symbol.+- **Evolution rule** (documented in `docs/observability.md`): additive optional fields do not bump the version; renames, removals, or semantic changes bump to `o2b.continuity.v2`; readers must accept unknown newer minor shapes fail-soft.+- **Lazy emit helper shape**: `emitGatedTelemetry(gate, build)` where `gate` is a boolean (already-resolved config) and `build` is a thunk performing the emission and returning the record. Gate off: thunk never invoked, returns `null`. Thunk throws: error swallowed, returns `null` (fail-open). Call sites that need the record id (`packContext` telemetryId enrichment) handle `null` by omitting the field.+- **Fail-open is a deliberate semantic change**: today a throwing `appendContinuityRecord` inside `packContext` would fail the pack; after this suite telemetry can never fail the primary operation. This is exactly what t_5d7aa7c5 demands; asserted by tests on every surface.+- **No-consumer regression tests spy on the thunk and on the continuity directory** - with the gate off, the thunk must never run and `Brain/log/continuity/` must stay absent/untouched. Covered gated surfaces: recall telemetry (context-pack, pre-compress), gate telemetry (MCP recall gate), context receipts (both pack paths).+- **Audit verdict on the non-gated writers**: `session-recall.ts` and `pre-compact-extract.ts` write continuity records as their PRIMARY operation (explicit import/extract commands) - they are always-on by design, keep fail-fast semantics, and do not route through the gated kernel. Session lifecycle capture writes audit rows and Brain log events, not continuity records; a regression test pins that boundary (lifecycle capture with defaults produces zero continuity writes).+- **Read-model masking policy is caller-explicit**: `normalizeContinuityRecords(records, { dropPrivate: true })` - export always drops `private` records; bench keeps them (fixture vaults contain no real private data) but the default is the safe drop.+- **Export is CLI-only, no MCP tool** - it produces files on disk; MCP surface gains nothing and the 65-tool contract stays untouched this release.+- **Bench fixtures are repo-local JSON** under `tests/fixtures/bench/` (checked in, sanitized); each fixture declares notes to materialize into a disposable vault plus questions with expected artifact paths. Categories shipped: single-hop preference recall, temporal update/stale-fact supersession, contradiction, multi-record evidence, session handoff, context budget truncation.+- **Deterministic evaluation is the default**: expected-context mode checks retrieval hits (hit@k, reciprocal rank) and context-pack evidence inclusion within budget - no network, CI-safe. Judge mode is an optional external command (config key `bench_judge_cmd`) receiving JSON on stdin and returning a verdict JSON on stdout; absent config means judge phase is skipped and marked `skipped` in the report.+- **Run store** under `.open-second-brain/bench-runs/<run-id>/` (gitignored): `checkpoint.json` (completed phases + fixture hash), per-question results, `report.json`. Resume validates the fixture hash before skipping phases - a changed fixture invalidates the checkpoint rather than silently mixing runs.+- **MemScore-style triple stays a triple**: `quality` (pass rate + per-category breakdown), `latency_ms` (avg + max), `context_cost` (avg chars + estimated tokens) - never collapsed into one number. `report.json` keys are stable and sorted for diffability.+- **Bench never touches a user vault**: the harness materializes its fixture vault under the run directory and resolves config strictly inside it; a guard refuses to run phases against any path outside the run directory.++## File changes++New:+- `src/core/brain/continuity/emit.ts` - lazy gated emit kernel.+- `src/core/brain/continuity/read-model.ts` - normalized record shape + masking policy.+- `src/core/brain/continuity/export-atof.ts`, `export-atif.ts` - format renderers over the read-model.+- `src/core/bench/` - `types.ts`, `fixture.ts` (load/validate/materialize), `run-store.ts` (checkpoint/resume), `phases.ts` (ingest/index/retrieve/evaluate/report), `judge.ts` (external command bridge), `report.ts` (diffable report).+- `src/cli/brain/verbs/continuity.ts` - `o2b brain continuity export`.+- `src/cli/brain/verbs/bench.ts` - `o2b brain bench memory`.+- `tests/fixtures/bench/*.json` - fixture suite.+- `docs/observability.md` - the contract doc (t_66545537).+- `docs/brainstorm/memory-observability-suite/atof-atif-mapping.md` - full mapping write-up (audit trail).+- Test suites: continuity-schema-version, continuity-emit, continuity-read-model, continuity-export-atof, continuity-export-atif, bench-fixture, bench-run-store, bench-phases, bench-report, CLI verb tests, e2e.++Modified:+- `src/core/brain/continuity/types.ts` (+`schema` field, +constant), `store.ts` (stamp in `buildRecord`, recordId untouched).+- `src/core/brain/recall-telemetry.ts`, `gate-telemetry.ts`, `context-receipts.ts`, `pre-compact-extract.ts`, `session-recall.ts`, `src/core/brain/context-pack.ts`, `pre-compress-pack.ts`, `src/mcp/search-tools.ts` - route gated emissions through the kernel.+- `src/cli/brain/verbs/index.ts`, `src/cli/brain.ts`, `command-manifest.ts`, `help-text.ts` - verb registration.+- `src/core/config.ts` - `resolveBenchJudgeCmd` (+ env override), test env scrub list.+- `.gitignore` - `.open-second-brain/bench-runs/`.+- `README.md`, `CHANGELOG.md`, `docs/cli-reference.md`, `docs/how-it-works.md` (phase 5).++## Risks and open questions++- **Fail-open conversion changes error propagation** at existing call sites; mitigated by explicit tests asserting both the old happy path and the new swallow path.+- **Synthetic ATOF scope start times** (createdAt minus duration) are an interpretation, not a recorded fact; documented in the mapping write-up and marked via an attribute so consumers can tell.+- **Golden-file tests vs spec drift**: specs are external; goldens pin our output shape, and the mapping doc records which spec revisions we matched.+- **Bench latency numbers are environment-sensitive**; the report separates them from quality precisely so CI can assert quality while treating latency as informative.+- **Session lifecycle hot path**: the audit found gating already happens before payload construction at the MCP gate site; lifecycle's audit row is the primary operation (not telemetry) and stays untouched - the no-consumer tests pin the boundary.diff --git a/docs/brainstorm/memory-observability-suite/plan.md b/docs/brainstorm/memory-observability-suite/plan.mdnew file mode 100644index 00000000..2c2279ce--- /dev/null+++ b/docs/brainstorm/memory-observability-suite/plan.md@@ -0,0 +1,43 @@+# Memory Observability Suite - implementation plan++## Tasks++### Task 1: Versioned continuity schema (t_26040ee8)+- **Files**: `src/core/brain/continuity/types.ts` (`schema` field on `ContinuityRecord`, `CONTINUITY_SCHEMA_VERSION` constant), `src/core/brain/continuity/store.ts` (stamp in `buildRecord`), `tests/core/brain/continuity-schema-version.test.ts`+- **Acceptance**: new records carry `schema: "o2b.continuity.v1"`; readers accept legacy records without the field; `recordId()` provably excludes the schema field (same id for stamped and unstamped identical input)+- **Depends on**: none++### Task 2: Lazy gated emit kernel + call-site audit (t_5d7aa7c5)+- **Files**: `src/core/brain/continuity/emit.ts` (new), call-site reroutes in `src/core/brain/context-pack.ts`, `src/core/brain/pre-compress-pack.ts`, `src/mcp/search-tools.ts` (gate telemetry); `tests/core/brain/continuity-emit.test.ts` (helper semantics), no-consumer regression tests per gated surface (extend `tests/core/brain/{context-pack,context-receipts}.test.ts`, `tests/mcp/recall-gate-telemetry.test.ts`), plus a lifecycle boundary test (capture with defaults produces zero continuity writes)+- **Acceptance**: with gate off, payload thunk never invoked and no continuity write happens (asserted per gated surface); a throwing thunk or write never fails the primary operation; gate on keeps existing record shapes byte-identical; `session-recall.ts` / `pre-compact-extract.ts` documented as always-on primary writes and left fail-fast+- **Depends on**: Task 1++### Task 3: Continuity read-model+- **Files**: `src/core/brain/continuity/read-model.ts` (new), `tests/core/brain/continuity-read-model.test.ts`+- **Acceptance**: raw JSONL lines (stamped, legacy, malformed) normalize into one canonical shape; legacy records report schema v1; masking policy drops `private` records by default and masks per option; unknown kinds pass through fail-soft+- **Depends on**: Task 1++### Task 4: ATOF/ATIF mapping write-up + export verb (t_51959aeb)+- **Files**: `docs/brainstorm/memory-observability-suite/atof-atif-mapping.md` (mapping, spec revisions, go decision), `src/core/brain/continuity/export-atof.ts`, `src/core/brain/continuity/export-atif.ts`, `src/cli/brain/verbs/continuity.ts`, verb registration (`verbs/index.ts`, `brain.ts`, `command-manifest.ts`, `help-text.ts`), `tests/core/brain/continuity-export-atof.test.ts`, `continuity-export-atif.test.ts`, `tests/cli/continuity-export.test.ts` (golden files with sanitized fixtures)+- **Acceptance**: `o2b brain continuity export --format atof|atif [--session <id>] [--month <m>] --out <dir>` produces spec-conformant files; `private` records dropped, `redacted` text stays masked; golden-file tests pass; read-only over the store+- **Depends on**: Task 3++### Task 5: Bench harness core (t_882c396a, part 1)+- **Files**: `src/core/bench/types.ts`, `src/core/bench/fixture.ts`, `src/core/bench/run-store.ts`, `src/core/bench/phases.ts`, `src/core/bench/report.ts`, `src/core/bench/judge.ts`, `tests/core/bench/{fixture,run-store,phases,report}.test.ts`+- **Acceptance**: fixture JSON validates and materializes a disposable vault inside the run directory; phases ingest/index/retrieve/evaluate/report run end-to-end deterministically with no network; checkpoint/resume by run ID skips completed phases and invalidates on fixture-hash change; report keeps quality/latency/context-cost separate with stable sorted keys; guard refuses paths outside the run dir+- **Depends on**: Task 3 (session-handoff fixture reads normalized continuity records)++### Task 6: Bench CLI verb + fixture suite (t_882c396a, part 2)+- **Files**: `src/cli/brain/verbs/bench.ts`, verb registration, `tests/fixtures/bench/*.json` (6 categories), `src/core/config.ts` (`resolveBenchJudgeCmd` + env override), `tests/helpers/run-cli.ts` (env scrub), `.gitignore`, `tests/cli/bench.test.ts`+- **Acceptance**: `o2b brain bench memory --fixture <name> [--resume <run-id>] [--json]` runs the full pipeline on every shipped fixture; at least one fixture catches stale/superseded recall regressions; at least one measures expected evidence within context budget; judge phase skipped cleanly when `bench_judge_cmd` unset+- **Depends on**: Task 5++### Task 7: e2e integration test+- **Files**: `tests/e2e/memory-observability.integration.test.ts`+- **Acceptance**: one flow exercises schema-stamped emission with gates on, export of the produced records to both formats, and a bench run over a small fixture - all green in one test file+- **Depends on**: Tasks 2, 4, 6++### Task 8: Observability contract doc (t_66545537)+- **Files**: `docs/observability.md`, cross-links from `docs/how-it-works.md` and `docs/mcp.md`+- **Acceptance**: every emitted event kind enumerated and verified against source (log events, continuity kinds, session lifecycle); opt-in vs always-on correct per kind; correlation IDs, payload safety, fail-open rules, schema version + evolution rule documented; repo writing prefs honored+- **Depends on**: Tasks 1, 2 (documents their behavior)diff --git a/docs/brainstorm/memory-observability-suite/variants.md b/docs/brainstorm/memory-observability-suite/variants.mdnew file mode 100644index 00000000..77909760--- /dev/null+++ b/docs/brainstorm/memory-observability-suite/variants.md@@ -0,0 +1,51 @@+# Memory Observability Suite - consultant variants (audit trail)++Consultant: Claude Code (`claude -p`), primary, exit 0, one pass.+Prompt: `cli-output/prompt.md`. Raw output: `cli-output/claude.md` (verbatim below).++## Variant 1: Single observability kernel++- **Approach**: Introduce one shared `src/core/brain/observability/` kernel that owns the schema-version constant (`o2b.continuity.v1`), the `emitTelemetry(kind, gate, lazyPayload)` gate helper, and a record-normalization function. Tasks 1+2 land entirely inside it; every existing writer (`context-receipts`, `session-recall`, `recall-telemetry`, `gate-telemetry`, lifecycle) is refactored to route through the helper. Tasks 4 (export) and 5 (bench) become read-only consumers that import the kernel's normalize/version logic so all three surfaces agree on record shape.+- **Trade-offs**:+ - Pro: Matches the established kernel-first convention (suites #64-#69); the lazy-after-gate shape becomes structurally enforced, not conventional.+ - Pro: Single source of truth for schema version + evolution rule, which Task 3's doc and Task 4's export both depend on.+ - Pro: Easiest place to hang the no-consumer regression tests and fail-open guarantees.+ - Con: Refactoring six call sites in one PR widens the blast radius and risks touching the lifecycle hot path.+ - Con: Up-front abstraction cost before Task 4's fit assessment is even known to pay off.+- **Complexity**: large+- **Risk**: medium++## Variant 2: Thin additive per-task patches++- **Approach**: Keep each task local with no new shared module. Add the `schema` field directly in `buildRecord()`, add `emitTelemetry` as a small helper beside the continuity store, write the doc, and add `export` and `bench` as independent CLI verbs that each read the JSONL store directly. Call sites are audited and fixed in place but not rerouted through a unifying layer.+- **Trade-offs**:+ - Pro: Smallest diff, lowest chance of regressing the lifecycle hot path or dedup hashing.+ - Pro: Task 4 can be cheaply abandoned (write-up only) without unwinding shared infrastructure.+ - Pro: Fast to land; each task is independently reviewable.+ - Con: The "no consumer => no payload work" property stays convention-enforced at most sites; drift is likely to recur in future suites.+ - Con: Export and bench each reimplement JSONL reading and version handling, duplicating normalization logic.+ - Con: Cuts against the project's kernel-first direction.+- **Complexity**: medium+- **Risk**: low++## Variant 3: Split write-emit kernel + read-model layer++- **Approach**: Build two narrow layers instead of one. A write-side emit kernel covers Tasks 1+2 (schema stamping at `buildRecord`, `emitTelemetry` thunk gate, fail-open). A separate read-side "trajectory read-model" normalizes JSONL records into a canonical in-memory shape with schema-version dispatch, and both Task 4 (ATOF/ATIF export) and Task 5 (bench retrieval scoring) consume only that read-model - never the raw store.+- **Trade-offs**:+ - Pro: Clean separation of fail-fast write concerns from fail-soft read concerns, matching the project's read/write posture.+ - Pro: The read-model absorbs schema-version branching once, so export and bench can't disagree on legacy-record handling or private/redacted masking.+ - Pro: Gated emit kernel is small and low-risk; the larger read-model is purely additive and read-only.+ - Con: Two kernels to design and document instead of one; some conceptual overlap (version constant referenced on both sides).+ - Con: The read-model is partly speculative until Task 4's fit assessment confirms ATOF/ATIF is worth building.+ - Con: More moving parts than a single kernel for reviewers to hold.+- **Complexity**: large+- **Risk**: medium++## Consultant recommendation: Variant 3++**Rationale (verbatim)**: The five tasks split cleanly along the project's own fail-fast-write / fail-soft-read seam, and a shared read-model is the one place that guarantees export and bench handle legacy unversioned records and private/redacted masking identically - the highest-leverage correctness invariant in the epic. The write-emit kernel stays small enough to keep the lifecycle hot path safe, while the read layer remains purely additive and can be scoped down to a write-up artifact if Task 4's ATOF/ATIF fit assessment comes back negative, without stranding the rest of the suite.++## Orchestrator decision: accept Variant 3, two refinements++1. The read-model's "partly speculative" con is resolved during phase 0 rather than left open: the ATOF/ATIF fit assessment was performed against the published specs before implementation (verdict: go, see design.md), and the bench harness consumes the read-model for its session-handoff fixtures regardless of that verdict - so the layer is justified on both branches.+2. The export verb lands inside the existing `o2b brain` namespace (`o2b brain continuity export`) instead of a new top-level `o2b continuity` command, matching every other Brain verb and the help/manifest registration convention.diff --git a/docs/cli-reference.md b/docs/cli-reference.mdindex 3b03eca4..2b1d43de 100644--- a/docs/cli-reference.md+++ b/docs/cli-reference.md@@ -136,6 +136,15 @@ o2b search <query> --global Cross-vault union over profiles + read-only sourceTrigger generation, brief delivery, and gate telemetry are pull-based or config-gated; pointer resolution activates only when a pointer file exists.+### Memory observability (since v0.39.0)++```text+o2b brain continuity export --format atof|atif [--session <id>] [--month YYYY-MM] [--out <dir>] [--json] - read-only trajectory export of the continuity store; private records dropped, redacted text stays masked+o2b brain bench memory --fixture <name|path> [--resume <run-id>] [--runs-dir <dir>] [--json] - memory quality benchmark over a disposable fixture vault; quality, latency, and context cost reported separately; checkpoint/resume by run id; exit 1 on any failed question+```++The benchmark never touches the configured vault - the fixture materializes into `<runs-dir>/<run-id>/vault` (default runs dir `.open-second-brain/bench-runs/`, gitignored). The optional `bench_judge_cmd` config key (env `OPEN_SECOND_BRAIN_BENCH_JUDGE_CMD`) arms an advisory external judge; absent means the judge phase is skipped. The full observability contract (event kinds, gates, correlation ids, payload safety, schema version) lives in `docs/observability.md`.+## Vault scopeSingle exclusion policy for every vault walker.diff --git a/docs/how-it-works.md b/docs/how-it-works.mdindex 7ad28fe5..6ad64ce6 100644--- a/docs/how-it-works.md+++ b/docs/how-it-works.md@@ -866,6 +866,33 @@ enqueue their findings as triggers. Opt-in recall-gate telemetryrecords every automatic-recall decision with a hashed prompt fortuning, never the raw text.+## Memory observability (since v0.39.0)++The continuity store is the Brain's flight recorder, and the Memory+Observability Suite turns it into a contract. Every new record is+stamped with a contract-wide schema version (`o2b.continuity.v1`);+legacy records read as v1, nothing is migrated, and the dedup id+deliberately excludes the stamp so identical records keep identical+ids. Gated telemetry surfaces (context-pack receipts and telemetry,+pre-compress, search, the recall gate) route through one lazy emit+kernel: with the gate off the payload thunk never runs, and a broken+continuity store can never fail the operation it was observing. A+read-model normalizes stamped and legacy records identically and+drops `private` records by default, so every read-side consumer+agrees on masking.++Two consumers ship with the suite. `o2b brain continuity export`+renders the store as standard trajectory formats - ATOF JSONL events+or ATIF v1.7 documents (one per session, memory-layer events marked+as deterministic dispatch) - so recall traces feed existing replay+and eval tooling. `o2b brain bench memory` measures recall quality+against disposable fixture vaults: checkpointed phases with resume by+run id, deterministic no-network evaluation, and a report that keeps+quality, latency, and context cost as separate families so a ranking+change cannot hide a token-cost regression behind a quality win. The+full contract - every event kind, its gate, correlation ids, payload+safety, fail-open rules - lives in `docs/observability.md`.+## Safety propertiesThese are invariants of the system, not configuration to enable.@@ -963,6 +990,10 @@ re-renders it from the current template.## Where to go next+- **`docs/observability.md`** — the observability contract: every+ log and continuity event kind, the always-on vs opt-in matrix,+ correlation IDs, payload safety, fail-open rules, and the+ `o2b.continuity.v1` schema version with its evolution rule.- **`docs/architecture.md`** — layered system architecture beyondBrain (vault model, runtime adapters, configuration model).- **`docs/plans/2026-05-15-brain-observing-memory.md`** — the Braindiff --git a/docs/mcp.md b/docs/mcp.mdindex 57b74ed8..e8f5b1b1 100644--- a/docs/mcp.md+++ b/docs/mcp.md@@ -305,6 +305,10 @@ decision when the `recall_gate_telemetry` config key is `"true"`never the raw prompt. `brain_recall_telemetry` gains `gate_list` /`gate_summary` operations.+The full observability contract behind these tools - event kinds,+always-on vs opt-in status, correlation IDs, payload safety, and the+continuity schema version - lives in `docs/observability.md`.+## Hermes integrationHermes discovers MCP servers from `~/.hermes/config.yaml` under thediff --git a/docs/observability.md b/docs/observability.mdnew file mode 100644index 00000000..2da4ca2c--- /dev/null+++ b/docs/observability.md@@ -0,0 +1,89 @@+# Observability contract++Open Second Brain records what it did - learning events, recall decisions, served context, session activity - across a small set of file-based surfaces inside the vault. This document is the contract for anyone consuming those surfaces: which events exist, when they fire, what correlates them, what safety guarantees their payloads carry, and how the record shapes evolve. Everything here is verified against the source files named in each section.++## Surfaces at a glance++| Surface | Location | Writer | Format |+|---|---|---|---|+| Brain log | `Brain/log/<date>.md` + JSONL sidecar | `appendLogEvent()` in `src/core/brain/log.ts` | Markdown line + JSONL row per event |+| Continuity store | `Brain/log/continuity/<month>.jsonl` | `appendContinuityRecord()` in `src/core/brain/continuity/store.ts` | one JSON record per line |+| Session lifecycle audit | `Brain/log/session-lifecycle/` | `captureSessionLifecycleEvent()` in `src/core/brain/session-lifecycle.ts` | JSONL audit rows |+| Bench runs | `<runs-dir>/<run-id>/` (default `.open-second-brain/bench-runs/`, gitignored) | `runMemoryBench()` in `src/core/bench/phases.ts` | `checkpoint.json`, per-question results, `report.json` |++## Brain log events++`appendLogEvent()` writes one human-readable line to the daily Markdown log plus a structured JSONL sidecar row. These events are always-on: each one is the audit trail of the operation that produced it, never optional telemetry. Kinds (from `BRAIN_LOG_EVENT_KIND` in `src/core/brain/types.ts`):++| Kind | Fires when |+|---|---|+| `dream`, `promote`, `retire`, `noted-redundant`, `signal-suppressed`, `skip-corrupted-frontmatter`, `reconcile` | the deterministic learning pass runs |+| `feedback`, `apply-evidence`, `force-confirmed`, `reject` | a taste signal or evidence event is recorded |+| `pin`, `unpin`, `rollback`, `merge`, `upgrade` | operator-facing vault maintenance |+| `scan-inline`, `import-session`, `import-claude-memory` | capture and import operations |+| `note` | a narrative milestone is recorded (`brain_note`) |+| `session-lifecycle` | a captured lifecycle event also produced Brain writes |++## Continuity record kinds++Every continuity record shares one envelope: `schema`, `id`, `kind`, `createdAt`, `sourceRefs`, `payload`, `private`, `redacted`. The table marks how each kind is gated - this is the always-on vs opt-in matrix, verified against the call sites named in the right column.++| Kind | Gate | Emitted by |+|---|---|---|+| `recall_telemetry` | opt-in per call (`telemetry` option/param) | `packContext` (`mode: context_pack`), `buildPreCompressPack` (`mode: pre_compress`), `brain_search` MCP handler (`mode: search`) |+| `context_receipt` | opt-in per call (`receipt` option/param) | `packContext`, `buildPreCompressPack` |+| `gate_telemetry` | config key `recall_gate_telemetry` (default off) | `brain_recall_gate` MCP handler |+| `session_turn`, `session_summary_node` | always-on within its operation | session-recall import (`src/core/brain/session-recall.ts`) - the write IS the operation |+| `pre_compact_extract` | always-on within its operation | explicit pre-compact extraction (`src/core/brain/pre-compact-extract.ts`) |+| `source_invalidation` | always-on within its operation | source drift detection (`appendContinuitySourceInvalidation`) |++Session lifecycle capture (`captureSessionLifecycleEvent`) writes audit rows and Brain log events but ZERO continuity records - that boundary is pinned by a regression test (`tests/core/brain/continuity-emit.test.ts`).++## Schema version and evolution rule++Since v0.39.0 every new continuity record is stamped `schema: "o2b.continuity.v1"` (`CONTINUITY_SCHEMA_VERSION` in `src/core/brain/continuity/types.ts`).++- Records written before the stamp existed carry no `schema` field; every reader treats `undefined` as v1. Existing JSONL files are never migrated.+- Additive optional fields do NOT bump the version. Renames, removals, or semantic changes bump to `o2b.continuity.v2`.+- The record id (`recordId()`, sha-256 over kind + createdAt + sourceRefs + payload) deliberately EXCLUDES the schema field, so identical records keep identical dedup ids across the stamp transition. Locked by a known-answer test.+- Consumers should go through the read-model (`src/core/brain/continuity/read-model.ts`), which absorbs the version dispatch, lifts `session_id`/`turn_id` into first-class fields, and applies the masking policy below.++## Correlation IDs++| Field | Where | Stable join? |+|---|---|---|+| `id` (`ctn_<stamp>_<sha16>`) | every continuity record | yes - content-hash dedup id |+| `createdAt` | every continuity record | ordering key (store sorts by `createdAt`, then `id`) |+| `sourceRefs[].id` / `.path` | continuity records | joins records to vault artifacts |+| `session_id`, `turn_id` | payloads of session-scoped kinds; lifted by the read-model | yes - the cross-surface session join |+| `timestamp` + `agent` | Brain log events | per-event attribution |++Multi-agent vaults attribute writers via per-agent identity (`brain_agent_query`); continuity records do not yet thread parent/child agent ids - if delegated-subagent correlation becomes needed, it lands as an additive payload field (no version bump).++## Payload safety++Every continuity payload passes `safeContinuityPayload()` (`src/core/brain/continuity/redaction.ts`) before it reaches disk:++- `<private>...</private>` regions are stripped and replaced; the record is flagged `private: true`.+- Secret-shaped tokens are redacted to `***REDACTED***`; the record is flagged `redacted: true`.+- The sanitized payload is deep-frozen; nothing mutates it after the flags are computed.+- `gate_telemetry` never stores the raw prompt by construction - only a SHA-256 prefix (`prompt_hash`) and the length (`prompt_chars`). The test suite asserts this against the persisted files, not just returned values.+- Read-side consumers (trajectory export, bench) get the masking policy from the read-model: records flagged `private` are dropped by default and kept only on explicit request; masked text is never un-masked.++## Fail-open rules++Telemetry must never block or fail the primary operation. The guarantee is structural, not conventional: gated surfaces route through `emitGatedTelemetry(gate, build)` (`src/core/brain/continuity/emit.ts`):++- gate off (`undefined`/`false`/`null`): the build thunk is NEVER invoked - no payload object, no hash, no write on the no-consumer path;+- a throwing thunk or store write is swallowed and reported as `null`; the context pack, pre-compress pack, search response, and recall-gate decision all complete normally with the telemetry field simply absent.++No-consumer regression tests cover each gated surface, so a future call site cannot silently regress to eager payload construction. Writers whose continuity append IS the primary operation (session-recall import, pre-compact extract) stay fail-fast on purpose - failing silently there would lose user data, not telemetry.++## Reading the surfaces++- `brain_recall_telemetry` (MCP) and `o2b brain recall-telemetry list|summary|gate-list|gate-summary` aggregate recall and gate records.+- `brain_context_receipts` and `o2b brain context-receipts` inspect served-context receipts.+- `o2b brain continuity export --format atof|atif` renders the store as standard trajectory formats (ATOF JSONL event stream; ATIF v1.7 one document per session) for replay and eval tooling. Read-only; `private` records never reach an export file. Mapping decisions: `docs/brainstorm/memory-observability-suite/atof-atif-mapping.md`.+- `o2b brain bench memory` measures recall quality over disposable fixture vaults and reports quality, latency, and context cost as separate families (`o2b.bench.v1` report schema).++Consumers should treat unknown record kinds and unknown payload fields as fine - skip what you do not recognize (the store and read-model already behave this way).diff --git a/openclaw.plugin.json b/openclaw.plugin.jsonindex 09db5d21..264d44bf 100644--- a/openclaw.plugin.json+++ b/openclaw.plugin.json@@ -2,7 +2,7 @@"id": "open-second-brain","name": "Open Second Brain","description": "Second brain for AI agents using Obsidian-compatible Markdown vaults.",- "version": "0.38.0",+ "version": "0.39.0","activation": { "onStartup": true },"skills": ["./skills"],"contracts": {diff --git a/package.json b/package.jsonindex f4e203e3..73db8335 100644--- a/package.json+++ b/package.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "0.38.0",+ "version": "0.39.0","private": false,"description": "Second brain for AI agents using Obsidian-compatible Markdown vaults. Works with Hermes, Claude Code, Codex, and OpenClaw.","keywords": [diff --git a/plugin.yaml b/plugin.yamlindex 8a85fc0e..ce9d4de7 100644--- a/plugin.yaml+++ b/plugin.yaml@@ -1,5 +1,5 @@name: open-second-brain-version: "0.38.0"+version: "0.39.0"description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."author: "Open Second Brain contributors"memory_provider: truediff --git a/plugins/codex/.codex-plugin/plugin.json b/plugins/codex/.codex-plugin/plugin.jsonindex a3b892ff..f5839eea 100644--- a/plugins/codex/.codex-plugin/plugin.json+++ b/plugins/codex/.codex-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "0.38.0",+ "version": "0.39.0","description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.","author": {"name": "Open Second Brain contributors",diff --git a/plugins/hermes/plugin.yaml b/plugins/hermes/plugin.yamlindex 8a85fc0e..ce9d4de7 100644--- a/plugins/hermes/plugin.yaml+++ b/plugins/hermes/plugin.yaml@@ -1,5 +1,5 @@name: open-second-brain-version: "0.38.0"+version: "0.39.0"description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."author: "Open Second Brain contributors"memory_provider: truediff --git a/pyproject.toml b/pyproject.tomlindex 4d3abcf9..ce000456 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -10,7 +10,7 @@ build-backend = "setuptools.build_meta"# CLI entry points (those moved to `package.json` `bin`).[project]name = "open-second-brain"-version = "0.38.0"+version = "0.39.0"description = "Hermes Python shim for Open Second Brain. Most of the project (CLI, MCP server, OpenClaw plugin) is TypeScript on Bun; see package.json."readme = "README.md"requires-python = ">=3.11"diff --git a/src/cli/brain.ts b/src/cli/brain.tsindex ad202f3b..6269fc4b 100644--- a/src/cli/brain.ts+++ b/src/cli/brain.ts@@ -57,6 +57,8 @@ import {cmdBrainLinks,cmdBrainProfile,cmdBrainSgrep,+ cmdBrainContinuity,+ cmdBrainBench,cmdBrainTrigger,cmdBrainDeepSynthesis,cmdBrainIdeas,@@ -196,6 +198,10 @@ export async function handleBrainSubcommand(argv: ReadonlyArray<string>): Promisreturn await cmdBrainProfile(rest);case "sgrep":return await cmdBrainSgrep(rest);+ case "continuity":+ return await cmdBrainContinuity(rest);+ case "bench":+ return await cmdBrainBench(rest);case "trigger":return await cmdBrainTrigger(rest);case "deep-synthesis":diff --git a/src/cli/brain/help-text.ts b/src/cli/brain/help-text.tsindex 9c59b506..7beb512c 100644--- a/src/cli/brain/help-text.ts+++ b/src/cli/brain/help-text.ts@@ -88,6 +88,8 @@ Brain verbs (observing memory):trigger Proactive trigger queue with anti-nag lifecycle (scan/list/ack/dismiss/act/history)deep-synthesis Topic dossier: notes, agreements, contradictions, stale claims, gapsideas Ranked next-direction candidates from open loops (--triggers to enqueue)+ continuity Export continuity records as ATOF/ATIF trajectories (read-only)+ bench Memory quality benchmark over a disposable fixture vaultCommon flags:--vault <path> Override the configured vault@@ -537,4 +539,16 @@ export const VERB_HELP: Record<string, string> = {"usage: o2b brain ideas [--cap <n>] [--triggers] [--vault <path>] [--json]\n" +"Ranked next-direction candidates from open questions, orphan notes,\n" +"and aging inbox signals. --triggers enqueues the ranked ideas.\n",+ continuity:+ "usage: o2b brain continuity export --format atof|atif [--session <id>] [--month YYYY-MM] [--out <dir>] [--json]\n" ++ "Read-only trajectory export of the continuity store. atof renders one\n" ++ "JSONL event stream; atif renders one trajectory document per session.\n" ++ "Records flagged private are dropped; redacted text stays masked.\n",+ bench:+ "usage: o2b brain bench memory --fixture <name|path> [--resume <run-id>] [--runs-dir <dir>] [--json]\n" ++ "Memory quality benchmark over a disposable fixture vault under the\n" ++ "runs directory (never the configured vault). Reports quality,\n" ++ "latency, and context cost separately; checkpoint/resume by run id;\n" ++ "bench_judge_cmd arms the optional external judge. Exit 1 on any\n" ++ "failed question.\n",};diff --git a/src/cli/brain/verbs/bench.ts b/src/cli/brain/verbs/bench.tsnew file mode 100644index 00000000..bceb35c3--- /dev/null+++ b/src/cli/brain/verbs/bench.ts@@ -0,0 +1,86 @@+/**+ * `o2b brain bench memory` (Memory Observability Suite, t_882c396a):+ * memory quality benchmark over a disposable fixture vault.+ *+ * The harness NEVER touches the operator's configured vault: it+ * materializes the fixture into `<runs-dir>/<run-id>/vault` and runs+ * the public search/context-pack APIs there. Deterministic and+ * network-free by default; `bench_judge_cmd` arms the optional+ * external judge. Quality, latency, and context cost stay separate+ * report families (the MemScore lesson).+ */++import { existsSync } from "node:fs";+import { join, resolve } from "node:path";++import { loadBenchFixture } from "../../../core/bench/fixture.ts";+import { runMemoryBench } from "../../../core/bench/phases.ts";+import { defaultConfigPath, resolveBenchJudgeCmd } from "../../../core/config.ts";+import { fail, parse } from "../helpers.ts";++const USAGE =+ "usage: o2b brain bench memory --fixture <name|path> [--resume <run-id>] [--runs-dir <dir>] [--json]";++const DEFAULT_RUNS_DIR = join(".open-second-brain", "bench-runs");+const REPO_FIXTURE_DIR = join("tests", "fixtures", "bench");++export async function cmdBrainBench(argv: string[]): Promise<number> {+ if (argv[0] !== "memory") return fail(USAGE);+ const { flags } = parse(argv.slice(1), {+ fixture: { type: "string" },+ resume: { type: "string" },+ "runs-dir": { type: "string" },+ json: { type: "boolean" },+ });+ const fixtureFlag = typeof flags["fixture"] === "string" ? flags["fixture"].trim() : "";+ if (fixtureFlag === "") return fail(USAGE);+ const fixturePath = resolveFixturePath(fixtureFlag);+ if (fixturePath === null) {+ return fail(+ `brain bench memory: fixture not found: ${fixtureFlag} (tried the literal path and ${REPO_FIXTURE_DIR}/${fixtureFlag}.json)`,+ );+ }++ const fixture = loadBenchFixture(fixturePath);+ const runsDir = resolve(+ typeof flags["runs-dir"] === "string" && flags["runs-dir"].trim() !== ""+ ? flags["runs-dir"].trim()+ : DEFAULT_RUNS_DIR,+ );+ const resume = typeof flags["resume"] === "string" ? flags["resume"].trim() : undefined;+ const judgeCmd = resolveBenchJudgeCmd(defaultConfigPath());++ const report = await runMemoryBench({+ fixture,+ runsDir,+ ...(resume !== undefined && resume !== "" ? { resume } : {}),+ ...(judgeCmd !== undefined ? { judgeCmd } : {}),+ });++ if (flags["json"] === true) {+ process.stdout.write(`${JSON.stringify(report, null, 2)}\n`);+ return report.quality.passed === report.quality.total ? 0 : 1;+ }+ process.stdout.write(`bench ${report.fixture} run ${report.run_id}\n`);+ process.stdout.write(+ `quality: ${report.quality.passed}/${report.quality.total} (pass rate ${report.quality.pass_rate})\n`,+ );+ process.stdout.write(`latency_ms: avg ${report.latency_ms.avg} max ${report.latency_ms.max}\n`);+ process.stdout.write(+ `context_cost: avg_chars ${report.context_cost.avg_chars} est_tokens ${report.context_cost.est_tokens}\n`,+ );+ process.stdout.write(`judge: ${report.judge.status}\n`);+ for (const question of report.questions) {+ process.stdout.write(+ ` ${question.pass ? "PASS" : "FAIL"} ${question.id} (${question.category})${question.failure ? ` - ${question.failure}` : ""}\n`,+ );+ }+ return report.quality.passed === report.quality.total ? 0 : 1;+}++function resolveFixturePath(flag: string): string | null {+ if (existsSync(flag)) return flag;+ const repoLocal = join(REPO_FIXTURE_DIR, `${flag}.json`);+ if (existsSync(repoLocal)) return repoLocal;+ return null;+}diff --git a/src/cli/brain/verbs/continuity.ts b/src/cli/brain/verbs/continuity.tsnew file mode 100644index 00000000..42cce519--- /dev/null+++ b/src/cli/brain/verbs/continuity.ts@@ -0,0 +1,103 @@+/**+ * `o2b brain continuity export` (Memory Observability Suite,+ * t_51959aeb): read-only trajectory export of the continuity store as+ * ATOF JSONL or ATIF v1.7 documents. Mapping decisions live in+ * docs/brainstorm/memory-observability-suite/atof-atif-mapping.md.+ *+ * Privacy: the exporter consumes the continuity read-model, which+ * drops `private` records and never un-masks redacted text. No new+ * emission paths - this verb only reads JSONL and writes export files+ * into `--out`.+ */++import { mkdirSync, writeFileSync } from "node:fs";+import { join, resolve } from "node:path";++import { defaultConfigPath } from "../../../core/config.ts";+import { loadNormalizedContinuityRecords } from "../../../core/brain/continuity/read-model.ts";+import { renderAtofEvents } from "../../../core/brain/continuity/export-atof.ts";+import {+ countSessionlessRecords,+ renderAtifTrajectories,+} from "../../../core/brain/continuity/export-atif.ts";+import packageJson from "../../../../package.json" with { type: "json" };+import { fail, parse, resolveBrainVault } from "../helpers.ts";++const CLI_VERSION: string = packageJson.version;++const USAGE =+ "usage: o2b brain continuity export --format atof|atif [--session <id>] [--month YYYY-MM] [--out <dir>] [--json]";++export async function cmdBrainContinuity(argv: string[]): Promise<number> {+ const subcommand = argv[0];+ if (subcommand !== "export") return fail(USAGE);+ return exportContinuity(argv.slice(1));+}++function exportContinuity(argv: string[]): number {+ const { flags } = parse(argv, {+ vault: { type: "string" },+ format: { type: "string" },+ session: { type: "string" },+ month: { type: "string" },+ out: { type: "string" },+ json: { type: "boolean" },+ });+ const format = typeof flags["format"] === "string" ? flags["format"].trim() : "";+ if (format !== "atof" && format !== "atif") return fail(USAGE);+ const month = typeof flags["month"] === "string" ? flags["month"].trim() : undefined;+ if (month !== undefined && !/^\d{4}-\d{2}$/.test(month)) {+ return fail("brain continuity export: --month must look like YYYY-MM");+ }+ const session = typeof flags["session"] === "string" ? flags["session"].trim() : undefined;+ const outDir = resolve(typeof flags["out"] === "string" ? flags["out"].trim() : ".");++ const vault = resolveBrainVault(flags["vault"] as string | undefined, defaultConfigPath());+ const records = loadNormalizedContinuityRecords(vault, {+ ...(session !== undefined && session !== "" ? { sessionId: session } : {}),+ ...(month !== undefined ? { since: `${month}-01`, until: `${month}-31T23:59:59.999Z` } : {}),+ });++ mkdirSync(outDir, { recursive: true });+ const written: string[] = [];+ let skippedSessionless = 0;+ if (format === "atof") {+ const lines = renderAtofEvents(records);+ const path = join(outDir, `o2b-continuity-${month ?? "all"}.atof.jsonl`);+ writeFileSync(path, lines.length > 0 ? `${lines.join("\n")}\n` : "", "utf8");+ written.push(path);+ } else {+ skippedSessionless = countSessionlessRecords(records);+ for (const trajectory of renderAtifTrajectories(records, { agentVersion: CLI_VERSION })) {+ const path = join(outDir, `o2b-continuity-${safeName(trajectory.session_id)}.atif.json`);+ writeFileSync(path, `${JSON.stringify(trajectory, null, 2)}\n`, "utf8");+ written.push(path);+ }+ }++ if (flags["json"] === true) {+ process.stdout.write(+ `${JSON.stringify(+ {+ format,+ records: records.length,+ skipped_sessionless: skippedSessionless,+ files: written,+ },+ null,+ 2,+ )}\n`,+ );+ return 0;+ }+ process.stdout.write(`exported ${records.length} record(s) as ${format}\n`);+ for (const path of written) process.stdout.write(` ${path}\n`);+ if (skippedSessionless > 0) {+ process.stdout.write(` skipped ${skippedSessionless} record(s) without a session id\n`);+ }+ return 0;+}++function safeName(value: string): string {+ return value.replace(/[^A-Za-z0-9._-]/g, "_").slice(0, 120);+}diff --git a/src/cli/brain/verbs/index.ts b/src/cli/brain/verbs/index.tsindex 2e923e9a..0358671c 100644--- a/src/cli/brain/verbs/index.ts+++ b/src/cli/brain/verbs/index.ts@@ -43,6 +43,8 @@ export { cmdBrainSource } from "./source.ts";export { cmdBrainLinks } from "./links.ts";export { cmdBrainProfile } from "./profile.ts";export { cmdBrainSgrep } from "./sgrep.ts";+export { cmdBrainContinuity } from "./continuity.ts";+export { cmdBrainBench } from "./bench.ts";export { cmdBrainTrigger } from "./trigger.ts";export { cmdBrainDeepSynthesis } from "./deep-synthesis.ts";export { cmdBrainIdeas } from "./ideas.ts";diff --git a/src/cli/command-manifest.ts b/src/cli/command-manifest.tsindex ec1df713..553cd1bc 100644--- a/src/cli/command-manifest.ts+++ b/src/cli/command-manifest.ts@@ -157,6 +157,8 @@ export const CLI_COMMAND_MANIFEST: CliRootManifest = Object.freeze({command("links", "Normalize wikilink path format across Brain notes"),command("profile", "Materialize the compact Brain/profile.md digest"),command("sgrep", "Grep-shaped semantic Brain search (path:line: output)"),+ command("continuity", "Export continuity records as ATOF/ATIF trajectories"),+ command("bench", "Memory quality benchmark over a disposable fixture vault"),command("trigger", "Grounded proactive trigger queue (scan/list/ack/dismiss/act/history)"),command("deep-synthesis", "Topic dossier: agreements, contradictions, stale claims, gaps"),command("ideas", "Ranked next-direction candidates from open loops"),diff --git a/src/core/bench/fixture.ts b/src/core/bench/fixture.tsnew file mode 100644index 00000000..15d98083--- /dev/null+++ b/src/core/bench/fixture.ts@@ -0,0 +1,227 @@+/**+ * Bench fixture loading, validation, hashing, and materialization+ * (Memory Observability Suite, t_882c396a).+ *+ * Fixtures are repo-local JSON documents. Validation fails fast with+ * named errors (mutation posture); materialization writes ONLY into+ * the caller-supplied disposable directory - note paths are checked+ * against traversal and absolutes at parse time.+ */++import { createHash } from "node:crypto";+import { mkdirSync, readFileSync, writeFileSync } from "node:fs";+import { dirname, isAbsolute, join, normalize } from "node:path";++import { appendContinuityRecord } from "../brain/continuity/store.ts";+import {+ BENCH_CATEGORIES,+ type BenchCategory,+ type BenchFixture,+ type BenchFixtureContinuity,+ type BenchFixtureNote,+ type BenchQuestion,+ RETRIEVAL_CATEGORIES,+} from "./types.ts";++/** Parse and validate a raw fixture document. Throws named errors. */+export function parseBenchFixture(raw: unknown): BenchFixture {+ if (raw === null || typeof raw !== "object" || Array.isArray(raw)) {+ throw new Error("bench fixture: expected a JSON object");+ }+ const record = raw as Record<string, unknown>;+ const name = nonEmptyString(record["name"]);+ if (name === undefined) throw new Error("bench fixture: missing or empty name");+ const description = nonEmptyString(record["description"]);++ const notesRaw = record["notes"];+ if (!Array.isArray(notesRaw) || notesRaw.length === 0) {+ throw new Error("bench fixture: notes must be a non-empty array");+ }+ const notes = notesRaw.map(parseNote);++ const continuityRaw = record["continuity"];+ const continuity =+ continuityRaw === undefined+ ? []+ : Array.isArray(continuityRaw)+ ? continuityRaw.map(parseContinuity)+ : (() => {+ throw new Error("bench fixture: continuity must be an array");+ })();++ const questionsRaw = record["questions"];+ if (!Array.isArray(questionsRaw) || questionsRaw.length === 0) {+ throw new Error("bench fixture: at least one question is required");+ }+ const questions = questionsRaw.map(parseQuestion);+ const ids = new Set(questions.map((question) => question.id));+ if (ids.size !== questions.length) {+ throw new Error("bench fixture: question ids must be unique");+ }++ return Object.freeze({+ name,+ ...(description !== undefined ? { description } : {}),+ notes: Object.freeze(notes),+ continuity: Object.freeze(continuity),+ questions: Object.freeze(questions),+ });+}++/** Load a fixture from a JSON file. Fail-fast on unreadable or broken input. */+export function loadBenchFixture(path: string): BenchFixture {+ return parseBenchFixture(JSON.parse(readFileSync(path, "utf8")));+}++/** Canonical content hash - stable across key order, sensitive to content. */+export function fixtureHash(fixture: BenchFixture): string {+ return createHash("sha256").update(canonicalJson(fixture)).digest("hex").slice(0, 16);+}++/** Write the fixture's notes and continuity records into a disposable vault. */+export function materializeBenchVault(fixture: BenchFixture, vaultDir: string): void {+ mkdirSync(join(vaultDir, "Brain"), { recursive: true });+ for (const note of fixture.notes) {+ const target = join(vaultDir, note.path);+ mkdirSync(dirname(target), { recursive: true });+ writeFileSync(target, note.body, "utf8");+ }+ for (const record of fixture.continuity) {+ appendContinuityRecord(vaultDir, {+ kind: record.kind,+ createdAt: record.created_at,+ payload: record.payload,+ });+ }+}++function parseNote(raw: unknown, index: number): BenchFixtureNote {+ if (raw === null || typeof raw !== "object") {+ throw new Error(`bench fixture: note ${index} must be an object`);+ }+ const record = raw as Record<string, unknown>;+ const path = nonEmptyString(record["path"]);+ const body = typeof record["body"] === "string" ? record["body"] : undefined;+ if (path === undefined || body === undefined) {+ throw new Error(`bench fixture: note ${index} needs path and body`);+ }+ if (isAbsolute(path) || normalize(path).split(/[\\/]/).includes("..")) {+ throw new Error(`bench fixture: note path escapes the vault: ${path}`);+ }+ return Object.freeze({ path, body });+}++function parseContinuity(raw: unknown, index: number): BenchFixtureContinuity {+ if (raw === null || typeof raw !== "object") {+ throw new Error(`bench fixture: continuity ${index} must be an object`);+ }+ const record = raw as Record<string, unknown>;+ if (record["kind"] !== "session_turn") {+ throw new Error(`bench fixture: continuity ${index} kind must be session_turn`);+ }+ const createdAt = nonEmptyString(record["created_at"]);+ const payload = record["payload"];+ if (createdAt === undefined || payload === null || typeof payload !== "object") {+ throw new Error(`bench fixture: continuity ${index} needs created_at and payload`);+ }+ return Object.freeze({+ kind: "session_turn",+ created_at: createdAt,+ payload: Object.freeze({ ...(payload as Record<string, unknown>) }),+ });+}++function parseQuestion(raw: unknown, index: number): BenchQuestion {+ if (raw === null || typeof raw !== "object") {+ throw new Error(`bench fixture: question ${index} must be an object`);+ }+ const record = raw as Record<string, unknown>;+ const id = nonEmptyString(record["id"]);+ if (id === undefined) throw new Error(`bench fixture: question ${index} needs an id`);+ const category = nonEmptyString(record["category"]);+ if (category === undefined || !isBenchCategory(category)) {+ throw new Error(+ `bench fixture: question ${id} has an unknown category (expected one of ${BENCH_CATEGORIES.join(", ")})`,+ );+ }+ const question: BenchQuestion = Object.freeze({+ id,+ category,+ ...(nonEmptyString(record["query"]) !== undefined+ ? { query: nonEmptyString(record["query"]) }+ : {}),+ ...(positiveInt(record["top_k"]) !== undefined ? { top_k: positiveInt(record["top_k"]) } : {}),+ ...(stringArray(record["expected_paths"]) !== undefined+ ? { expected_paths: stringArray(record["expected_paths"]) }+ : {}),+ ...(stringArray(record["not_expected_above"]) !== undefined+ ? { not_expected_above: stringArray(record["not_expected_above"]) }+ : {}),+ ...(nonEmptyString(record["session_id"]) !== undefined+ ? { session_id: nonEmptyString(record["session_id"]) }+ : {}),+ ...(positiveInt(record["expected_turns"]) !== undefined+ ? { expected_turns: positiveInt(record["expected_turns"]) }+ : {}),+ ...(nonEmptyString(record["expected_text"]) !== undefined+ ? { expected_text: nonEmptyString(record["expected_text"]) }+ : {}),+ ...(stringArray(record["expected_ids"]) !== undefined+ ? { expected_ids: stringArray(record["expected_ids"]) }+ : {}),+ ...(positiveInt(record["max_tokens"]) !== undefined+ ? { max_tokens: positiveInt(record["max_tokens"]) }+ : {}),+ ...(positiveInt(record["max_total_chars"]) !== undefined+ ? { max_total_chars: positiveInt(record["max_total_chars"]) }+ : {}),+ });+ validateQuestionShape(question);+ return question;+}++function validateQuestionShape(question: BenchQuestion): void {+ if (RETRIEVAL_CATEGORIES.has(question.category)) {+ if (question.query === undefined || (question.expected_paths ?? []).length === 0) {+ throw new Error(+ `bench fixture: question ${question.id} (${question.category}) needs query and expected_paths`,+ );+ }+ }+ if (question.category === "session_handoff" && question.session_id === undefined) {+ throw new Error(`bench fixture: question ${question.id} needs session_id`);+ }+ if (question.category === "budget" && (question.expected_ids ?? []).length === 0) {+ throw new Error(`bench fixture: question ${question.id} needs expected_ids`);+ }+}++function isBenchCategory(value: string): value is BenchCategory {+ return (BENCH_CATEGORIES as ReadonlyArray<string>).includes(value);+}++function nonEmptyString(value: unknown): string | undefined {+ return typeof value === "string" && value.trim().length > 0 ? value : undefined;+}++function positiveInt(value: unknown): number | undefined {+ return typeof value === "number" && Number.isInteger(value) && value > 0 ? value : undefined;+}++function stringArray(value: unknown): ReadonlyArray<string> | undefined {+ if (!Array.isArray(value)) return undefined;+ if (!value.every((item) => typeof item === "string" && item.length > 0)) return undefined;+ return Object.freeze([...value]);+}++/** JSON with recursively sorted object keys - the canonical hash input. */+function canonicalJson(value: unknown): string {+ if (Array.isArray(value)) return `[${value.map(canonicalJson).join(",")}]`;+ if (value !== null && typeof value === "object") {+ const entries = Object.entries(value as Record<string, unknown>)+ .toSorted(([a], [b]) => (a < b ? -1 : a > b ? 1 : 0))+ .map(([key, child]) => `${JSON.stringify(key)}:${canonicalJson(child)}`);+ return `{${entries.join(",")}}`;+ }+ return JSON.stringify(value) ?? "null";+}diff --git a/src/core/bench/judge.ts b/src/core/bench/judge.tsnew file mode 100644index 00000000..a9483af1--- /dev/null+++ b/src/core/bench/judge.ts@@ -0,0 +1,54 @@+/**+ * Optional external judge bridge (Memory Observability Suite,+ * t_882c396a).+ *+ * The Brain core stays deterministic: no LLM ever runs inside the+ * harness. When `bench_judge_cmd` is configured, the command receives+ * the evaluated questions as JSON on stdin and may return+ * `{ "verdicts": { "<question-id>": true|false } }` on stdout. The+ * judge is advisory - deterministic evaluation stays canonical, the+ * verdicts land in the report for comparison. Absent config means the+ * judge phase is skipped and marked as such; a failing command marks+ * `error` without failing the run (fail-open, like all telemetry).+ */++import { spawnSync } from "node:child_process";++import type { BenchQuestionResult, BenchReport } from "./types.ts";++export interface JudgeOutcome {+ readonly status: BenchReport["judge"]["status"];+ readonly detail?: string;+ readonly verdicts?: Readonly<Record<string, boolean>>;+}++export function runJudge(+ cmd: string | undefined,+ questions: ReadonlyArray<BenchQuestionResult>,+): JudgeOutcome {+ if (cmd === undefined || cmd.trim() === "") return Object.freeze({ status: "skipped" });+ try {+ const proc = spawnSync("sh", ["-c", cmd], {+ input: JSON.stringify({ questions }),+ encoding: "utf8",+ timeout: 60_000,+ });+ if (proc.status !== 0) {+ return Object.freeze({+ status: "error",+ detail: `judge command exited ${proc.status ?? "by signal"}`,+ });+ }+ const parsed = JSON.parse(proc.stdout) as { verdicts?: Record<string, unknown> };+ const verdicts: Record<string, boolean> = {};+ for (const [id, value] of Object.entries(parsed.verdicts ?? {})) {+ if (typeof value === "boolean") verdicts[id] = value;+ }+ return Object.freeze({ status: "ran", verdicts: Object.freeze(verdicts) });+ } catch (error) {+ return Object.freeze({+ status: "error",+ detail: error instanceof Error ? error.message : "judge command failed",+ });+ }+}diff --git a/src/core/bench/phases.ts b/src/core/bench/phases.tsnew file mode 100644index 00000000..b2a8d948--- /dev/null+++ b/src/core/bench/phases.ts@@ -0,0 +1,312 @@+/**+ * Bench phase pipeline (Memory Observability Suite, t_882c396a):+ * ingest -> index -> retrieve -> evaluate -> report.+ *+ * Each phase checkpoints on completion, so a resumed run (same run id,+ * same fixture hash) skips finished work - including the searches:+ * the retrieve phase persists per-question raw results to disk and the+ * evaluate phase works ONLY from those files, never from the live+ * vault. The pipeline drives the public `search` / `packContext` APIs+ * against a disposable vault inside the run directory and never+ * resolves the operator's configured vault. Deterministic and+ * network-free: keyword-only search, no embedding providers, judge+ * optional and external.+ */++import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";+import { join } from "node:path";++import { packContext } from "../brain/context-pack.ts";+import { loadNormalizedContinuityRecords } from "../brain/continuity/read-model.ts";+import { resolveSearchConfig, search } from "../search/index.ts";+import { fixtureHash, materializeBenchVault } from "./fixture.ts";+import { runJudge } from "./judge.ts";+import {+ benchResultsDir,+ benchVaultDir,+ completeBenchPhase,+ createBenchRun,+ loadBenchRun,+ phaseDone,+ type BenchRunHandle,+} from "./run-store.ts";+import {+ BENCH_REPORT_SCHEMA,+ RETRIEVAL_CATEGORIES,+ type BenchFixture,+ type BenchQuestion,+ type BenchQuestionResult,+ type BenchReport,+} from "./types.ts";++export interface BenchRunOptions {+ readonly fixture: BenchFixture;+ readonly runsDir: string;+ /** Resume an existing run id; validates the fixture hash first. */+ readonly resume?: string;+ /** Optional external judge command (config bench_judge_cmd). */+ readonly judgeCmd?: string;+ readonly now?: Date;+}++/** Raw per-question record persisted by the retrieve phase. */+interface RetrievedQuestion {+ readonly id: string;+ readonly latency_ms: number;+ /** Result paths in rank order (retrieval categories). */+ readonly paths?: ReadonlyArray<string>;+ /** Normalized turn texts (session_handoff). */+ readonly turn_texts?: ReadonlyArray<string>;+ /** Pack item ids and character count (budget). */+ readonly item_ids?: ReadonlyArray<string>;+ readonly context_chars?: number;+}++export async function runMemoryBench(opts: BenchRunOptions): Promise<BenchReport> {+ const run: BenchRunHandle =+ opts.resume !== undefined+ ? loadBenchRun(opts.runsDir, opts.resume, { expectFixture: opts.fixture })+ : createBenchRun(opts.runsDir, opts.fixture, opts.now ? { now: opts.now } : {});+ let checkpoint = run.checkpoint;+ const vault = benchVaultDir(run.runDir);++ // Phase: ingest - materialize the disposable vault.+ if (!phaseDone(checkpoint, "ingest")) {+ materializeBenchVault(opts.fixture, vault);+ checkpoint = completeBenchPhase(run.runDir, checkpoint, "ingest");+ }++ // Phase: index - one warmup search triggers the store self-heal so+ // the FTS index exists before any timed retrieval.+ if (!phaseDone(checkpoint, "index")) {+ const config = resolveSearchConfig({ vault });+ await search(config, { query: "warmup", limit: 1 });+ checkpoint = completeBenchPhase(run.runDir, checkpoint, "index");+ }++ // Phase: retrieve - run every question once, persist raw results.+ if (!phaseDone(checkpoint, "retrieve")) {+ const config = resolveSearchConfig({ vault });+ mkdirSync(benchResultsDir(run.runDir), { recursive: true });+ // Deliberately sequential: per-question latency_ms must not include+ // contention from sibling searches running on the same store.+ // oxlint-disable-next-line no-await-in-loop+ for (const question of opts.fixture.questions) {+ // oxlint-disable-next-line no-await-in-loop+ const retrieved = await retrieveQuestion(vault, config, question);+ writeFileSync(+ join(benchResultsDir(run.runDir), `${question.id}.json`),+ `${JSON.stringify(retrieved, null, 2)}\n`,+ "utf8",+ );+ }+ checkpoint = completeBenchPhase(run.runDir, checkpoint, "retrieve");+ }++ // Phase: evaluate - pure over the persisted results; no vault access.+ const results = opts.fixture.questions+ .map((question) => evaluateQuestion(question, readRetrieved(run.runDir, question.id)))+ .toSorted((a, b) => (a.id < b.id ? -1 : a.id > b.id ? 1 : 0));+ if (!phaseDone(checkpoint, "evaluate")) {+ checkpoint = completeBenchPhase(run.runDir, checkpoint, "evaluate");+ }++ // Optional judge (advisory, fail-open), then report.+ const judge = runJudge(opts.judgeCmd, results);+ const report = buildReport(run, opts.fixture, results, judge);+ writeFileSync(join(run.runDir, "report.json"), `${JSON.stringify(report, null, 2)}\n`, "utf8");+ if (!phaseDone(checkpoint, "report")) {+ completeBenchPhase(run.runDir, checkpoint, "report");+ }+ return report;+}++async function retrieveQuestion(+ vault: string,+ config: ReturnType<typeof resolveSearchConfig>,+ question: BenchQuestion,+): Promise<RetrievedQuestion> {+ const startedAt = Date.now();+ if (RETRIEVAL_CATEGORIES.has(question.category)) {+ const outcome = await search(config, {+ query: question.query ?? "",+ limit: question.top_k ?? 5,+ });+ return {+ id: question.id,+ latency_ms: Date.now() - startedAt,+ paths: outcome.results.map((result) => result.path),+ };+ }+ if (question.category === "session_handoff") {+ const turns = loadNormalizedContinuityRecords(vault, {+ kind: "session_turn",+ ...(question.session_id !== undefined ? { sessionId: question.session_id } : {}),+ });+ return {+ id: question.id,+ latency_ms: Date.now() - startedAt,+ turn_texts: turns.map((turn) =>+ typeof turn.payload["text"] === "string" ? turn.payload["text"] : "",+ ),+ };+ }+ // budget+ const pack = packContext(vault, {+ maxTokens: question.max_tokens ?? 1000,+ ...(question.max_total_chars !== undefined ? { maxTotalChars: question.max_total_chars } : {}),+ });+ return {+ id: question.id,+ latency_ms: Date.now() - startedAt,+ item_ids: pack.items.map((item) => item.id),+ context_chars: pack.items.reduce((sum, item) => sum + item.body.length, 0),+ };+}++function readRetrieved(runDir: string, questionId: string): RetrievedQuestion {+ const path = join(benchResultsDir(runDir), `${questionId}.json`);+ if (!existsSync(path)) {+ throw new Error(`bench run is missing retrieve results for question ${questionId}`);+ }+ return JSON.parse(readFileSync(path, "utf8")) as RetrievedQuestion;+}++function evaluateQuestion(+ question: BenchQuestion,+ retrieved: RetrievedQuestion,+): BenchQuestionResult {+ const base = {+ id: question.id,+ category: question.category,+ latency_ms: retrieved.latency_ms,+ ...(retrieved.context_chars !== undefined ? { context_chars: retrieved.context_chars } : {}),+ };+ if (RETRIEVAL_CATEGORIES.has(question.category)) {+ const paths = retrieved.paths ?? [];+ const missing = (question.expected_paths ?? []).filter((path) => !paths.includes(path));+ if (missing.length > 0) {+ return Object.freeze({+ ...base,+ pass: false,+ failure: `missing expected: ${missing.join(", ")}`,+ });+ }+ const bestExpected = Math.min(+ ...(question.expected_paths ?? []).map((path) => paths.indexOf(path)),+ );+ for (const stale of question.not_expected_above ?? []) {+ const rank = paths.indexOf(stale);+ if (rank !== -1 && rank < bestExpected) {+ return Object.freeze({+ ...base,+ pass: false,+ failure: `stale path ranked above expected: ${stale}`,+ });+ }+ }+ return Object.freeze({ ...base, pass: true });+ }+ if (question.category === "session_handoff") {+ const texts = retrieved.turn_texts ?? [];+ if (question.expected_turns !== undefined && texts.length < question.expected_turns) {+ return Object.freeze({+ ...base,+ pass: false,+ failure: `expected ${question.expected_turns} turns, found ${texts.length}`,+ });+ }+ if (+ question.expected_text !== undefined &&+ !texts.some((text) => text.includes(question.expected_text ?? ""))+ ) {+ return Object.freeze({+ ...base,+ pass: false,+ failure: `expected text not found in session turns: ${question.expected_text}`,+ });+ }+ return Object.freeze({ ...base, pass: true });+ }+ // budget+ const ids = retrieved.item_ids ?? [];+ const missingIds = (question.expected_ids ?? []).filter((id) => !ids.includes(id));+ if (missingIds.length > 0) {+ return Object.freeze({+ ...base,+ pass: false,+ failure: `expected evidence missing from budgeted pack: ${missingIds.join(", ")}`,+ });+ }+ return Object.freeze({ ...base, pass: true });+}++function buildReport(+ run: BenchRunHandle,+ fixture: BenchFixture,+ results: ReadonlyArray<BenchQuestionResult>,+ judge: ReturnType<typeof runJudge>,+): BenchReport {+ const byCategory: Record<string, { passed: number; total: number }> = {};+ let passed = 0;+ for (const result of results) {+ const bucket = (byCategory[result.category] ??= { passed: 0, total: 0 });+ bucket.total += 1;+ if (result.pass) {+ bucket.passed += 1;+ passed += 1;+ }+ }+ const latencies = results.map((result) => result.latency_ms);+ const contextChars = results+ .map((result) => result.context_chars)+ .filter((value): value is number => typeof value === "number");+ const avgChars =+ contextChars.length > 0+ ? Math.round(contextChars.reduce((sum, value) => sum + value, 0) / contextChars.length)+ : 0;+ return Object.freeze({+ schema: BENCH_REPORT_SCHEMA,+ run_id: run.runId,+ fixture: fixture.name,+ fixture_hash: fixtureHash(fixture),+ created_at: run.checkpoint.created_at,+ quality: Object.freeze({+ passed,+ total: results.length,+ pass_rate: results.length > 0 ? round3(passed / results.length) : 0,+ by_category: Object.freeze(+ Object.fromEntries(Object.entries(byCategory).toSorted(([a], [b]) => (a < b ? -1 : 1))),+ ),+ }),+ latency_ms: Object.freeze({+ avg:+ latencies.length > 0+ ? Math.round(latencies.reduce((sum, value) => sum + value, 0) / latencies.length)+ : 0,+ max: latencies.length > 0 ? Math.max(...latencies) : 0,+ }),+ context_cost: Object.freeze({+ avg_chars: avgChars,+ est_tokens: Math.ceil(avgChars / 4),+ }),+ judge: Object.freeze({+ status: judge.status,+ ...(judge.detail !== undefined ? { detail: judge.detail } : {}),+ }),+ questions: Object.freeze(+ results.map((result) =>+ Object.freeze({+ ...result,+ ...(judge.verdicts && result.id in judge.verdicts+ ? { judge_pass: judge.verdicts[result.id] }+ : {}),+ }),+ ),+ ),+ });+}++function round3(value: number): number {+ return Math.round(value * 1000) / 1000;+}diff --git a/src/core/bench/run-store.ts b/src/core/bench/run-store.tsnew file mode 100644index 00000000..ee3b3a3e--- /dev/null+++ b/src/core/bench/run-store.ts@@ -0,0 +1,149 @@+/**+ * Bench run store (Memory Observability Suite, t_882c396a).+ *+ * Checkpointed runs under a caller-supplied runs directory:+ * `<runsDir>/<run-id>/checkpoint.json` plus per-question results and+ * the final report. Resume validates the fixture hash before skipping+ * phases - a changed fixture invalidates the checkpoint rather than+ * silently mixing two runs. A path guard refuses run ids that resolve+ * outside the runs directory.+ */++import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";+import { join, resolve, sep } from "node:path";++import { fixtureHash } from "./fixture.ts";+import { BENCH_PHASES, type BenchCheckpoint, type BenchFixture, type BenchPhase } from "./types.ts";++export interface BenchRunHandle {+ readonly runId: string;+ readonly runDir: string;+ readonly checkpoint: BenchCheckpoint;+}++const CHECKPOINT_FILE = "checkpoint.json";++/** Disposable fixture vault location inside one run. */+export function benchVaultDir(runDir: string): string {+ return join(runDir, "vault");+}++/** Per-question raw results location inside one run. */+export function benchResultsDir(runDir: string): string {+ return join(runDir, "results");+}++export function createBenchRun(+ runsDir: string,+ fixture: BenchFixture,+ opts: { now?: Date } = {},+): BenchRunHandle {+ const now = opts.now ?? new Date();+ const hash = fixtureHash(fixture);+ const stamp = now+ .toISOString()+ .replace(/[^0-9]/g, "")+ .slice(0, 14);+ const runId = `run-${stamp}-${hash.slice(0, 6)}`;+ const runDir = guardedRunDir(runsDir, runId);+ mkdirSync(runDir, { recursive: true });+ const checkpoint: BenchCheckpoint = Object.freeze({+ run_id: runId,+ fixture_name: fixture.name,+ fixture_hash: hash,+ created_at: now.toISOString(),+ completed_phases: Object.freeze([]),+ });+ persist(runDir, checkpoint);+ return Object.freeze({ runId, runDir, checkpoint });+}++export function loadBenchRun(+ runsDir: string,+ runId: string,+ opts: { expectFixture?: BenchFixture } = {},+): BenchRunHandle {+ const runDir = guardedRunDir(runsDir, runId);+ const path = join(runDir, CHECKPOINT_FILE);+ if (!existsSync(path)) {+ throw new Error(`bench run not found: ${runId}`);+ }+ const checkpoint = parseCheckpoint(JSON.parse(readFileSync(path, "utf8")));+ if (opts.expectFixture !== undefined) {+ const expected = fixtureHash(opts.expectFixture);+ if (expected !== checkpoint.fixture_hash) {+ throw new Error(+ `bench run ${runId}: fixture hash mismatch (checkpoint ${checkpoint.fixture_hash}, current ${expected}) - the fixture changed since this run started`,+ );+ }+ }+ return Object.freeze({ runId, runDir, checkpoint });+}++/** Mark a phase complete (idempotent) and persist the checkpoint. */+export function completeBenchPhase(+ runDir: string,+ checkpoint: BenchCheckpoint,+ phase: BenchPhase,+): BenchCheckpoint {+ if (checkpoint.completed_phases.includes(phase)) return checkpoint;+ const updated: BenchCheckpoint = Object.freeze({+ ...checkpoint,+ completed_phases: Object.freeze([...checkpoint.completed_phases, phase]),+ });+ persist(runDir, updated);+ return updated;+}++export function phaseDone(checkpoint: BenchCheckpoint, phase: BenchPhase): boolean {+ return checkpoint.completed_phases.includes(phase);+}++function persist(runDir: string, checkpoint: BenchCheckpoint): void {+ writeFileSync(join(runDir, CHECKPOINT_FILE), `${JSON.stringify(checkpoint, null, 2)}\n`, "utf8");+}++function parseCheckpoint(raw: unknown): BenchCheckpoint {+ if (raw === null || typeof raw !== "object") {+ throw new Error("bench checkpoint: malformed JSON");+ }+ const record = raw as Record<string, unknown>;+ const runId = record["run_id"];+ const fixtureName = record["fixture_name"];+ const hash = record["fixture_hash"];+ const createdAt = record["created_at"];+ const phases = record["completed_phases"];+ if (+ typeof runId !== "string" ||+ typeof fixtureName !== "string" ||+ typeof hash !== "string" ||+ typeof createdAt !== "string" ||+ !Array.isArray(phases)+ ) {+ throw new Error("bench checkpoint: missing required fields");+ }+ const completed = phases.filter(+ (phase): phase is BenchPhase =>+ typeof phase === "string" && (BENCH_PHASES as ReadonlyArray<string>).includes(phase),+ );+ return Object.freeze({+ run_id: runId,+ fixture_name: fixtureName,+ fixture_hash: hash,+ created_at: createdAt,+ completed_phases: Object.freeze(completed),+ });+}++/** Resolve a run directory and refuse anything outside the runs dir. */+function guardedRunDir(runsDir: string, runId: string): string {+ if (!/^[A-Za-z0-9._-]+$/.test(runId)) {+ throw new Error(`bench run id contains unsafe characters: ${runId}`);+ }+ const root = resolve(runsDir);+ const dir = resolve(root, runId);+ if (dir !== root && !dir.startsWith(`${root}${sep}`)) {+ throw new Error(`bench run directory escapes the runs dir: ${runId}`);+ }+ return dir;+}diff --git a/src/core/bench/types.ts b/src/core/bench/types.tsnew file mode 100644index 00000000..dfba85b4--- /dev/null+++ b/src/core/bench/types.ts@@ -0,0 +1,112 @@+/**+ * Memory quality benchmark types (Memory Observability Suite,+ * t_882c396a).+ *+ * MemoryBench-inspired harness for recall regression testing. The+ * MemScore lesson is encoded in the report type: quality, latency, and+ * context cost are SEPARATE metric families - never one collapsed+ * number. Everything is deterministic and network-free by default;+ * judge-model evaluation is an optional external command.+ */++export const BENCH_REPORT_SCHEMA = "o2b.bench.v1";++export const BENCH_PHASES = ["ingest", "index", "retrieve", "evaluate", "report"] as const;+export type BenchPhase = (typeof BENCH_PHASES)[number];++export const BENCH_CATEGORIES = [+ "single_hop",+ "temporal",+ "contradiction",+ "multi_evidence",+ "session_handoff",+ "budget",+] as const;+export type BenchCategory = (typeof BENCH_CATEGORIES)[number];++/** Question categories answered by running the search pipeline. */+export const RETRIEVAL_CATEGORIES: ReadonlySet<BenchCategory> = new Set([+ "single_hop",+ "temporal",+ "contradiction",+ "multi_evidence",+]);++export interface BenchFixtureNote {+ /** Vault-relative path; validated against traversal and absolutes. */+ readonly path: string;+ readonly body: string;+}++export interface BenchFixtureContinuity {+ readonly kind: "session_turn";+ readonly created_at: string;+ readonly payload: Readonly<Record<string, unknown>>;+}++export interface BenchQuestion {+ readonly id: string;+ readonly category: BenchCategory;+ /** Search query (retrieval categories). */+ readonly query?: string;+ readonly top_k?: number;+ readonly expected_paths?: ReadonlyArray<string>;+ /**+ * Stale-fact guard: none of these paths may rank above the best+ * expected path. Catches superseded-recall regressions.+ */+ readonly not_expected_above?: ReadonlyArray<string>;+ /** session_handoff: which session's turns must be readable. */+ readonly session_id?: string;+ readonly expected_turns?: number;+ readonly expected_text?: string;+ /** budget: pack item ids that must fit inside the budget. */+ readonly expected_ids?: ReadonlyArray<string>;+ readonly max_tokens?: number;+ readonly max_total_chars?: number;+}++export interface BenchFixture {+ readonly name: string;+ readonly description?: string;+ readonly notes: ReadonlyArray<BenchFixtureNote>;+ readonly continuity: ReadonlyArray<BenchFixtureContinuity>;+ readonly questions: ReadonlyArray<BenchQuestion>;+}++export interface BenchQuestionResult {+ readonly id: string;+ readonly category: BenchCategory;+ readonly pass: boolean;+ /** One-line reason when pass is false. */+ readonly failure?: string;+ readonly latency_ms: number;+ /** Pack character count (budget questions). */+ readonly context_chars?: number;+}++export interface BenchReport {+ readonly schema: string;+ readonly run_id: string;+ readonly fixture: string;+ readonly fixture_hash: string;+ readonly created_at: string;+ readonly quality: {+ readonly passed: number;+ readonly total: number;+ readonly pass_rate: number;+ readonly by_category: Readonly<Record<string, { passed: number; total: number }>>;+ };+ readonly latency_ms: { readonly avg: number; readonly max: number };+ readonly context_cost: { readonly avg_chars: number; readonly est_tokens: number };+ readonly judge: { readonly status: "skipped" | "ran" | "error"; readonly detail?: string };+ readonly questions: ReadonlyArray<BenchQuestionResult>;+}++export interface BenchCheckpoint {+ readonly run_id: string;+ readonly fixture_name: string;+ readonly fixture_hash: string;+ readonly created_at: string;+ readonly completed_phases: ReadonlyArray<BenchPhase>;+}diff --git a/src/core/brain/context-pack.ts b/src/core/brain/context-pack.tsindex 67fdc048..430de204 100644--- a/src/core/brain/context-pack.ts+++ b/src/core/brain/context-pack.ts@@ -28,6 +28,7 @@ import { estimateTokens } from "./text/tokenizer.ts";import { normalizeForDedup } from "./text/normalize.ts";import { applyCharBudget } from "./recall-budget.ts";import { emitContextReceipt, type ContextReceiptOptions } from "./context-receipts.ts";+import { emitGatedTelemetry } from "./continuity/emit.ts";import { emitRecallTelemetry, type RecallTelemetryOptions } from "./recall-telemetry.ts";import {applyContextTransforms,@@ -395,9 +396,12 @@ function finalizeContextPackReport(startedAtMs: number,): ContextPackReport {let enriched = report;- if (opts.receipt) {- const receipt = emitContextReceipt(vault, {- options: opts.receipt,+ // Gated emissions route through the lazy emit kernel (t_5d7aa7c5):+ // with the option absent the thunk never runs, and a broken+ // continuity store can no longer fail the pack (fail-open).+ const receipt = emitGatedTelemetry(opts.receipt, (receiptOptions) =>+ emitContextReceipt(vault, {+ options: receiptOptions,items: report.items.map((item) => ({id: item.id,path: item.path,@@ -413,15 +417,15 @@ function finalizeContextPackReport(skipped_count: report.skipped.length,lanes: opts.includeLanes === true,},- });- enriched = { ...enriched, receiptId: receipt.id };- }- if (opts.telemetry) {- const telemetry = emitRecallTelemetry(vault, {- createdAt: opts.telemetry.createdAt,- host: opts.telemetry.host,- sessionId: opts.telemetry.sessionId,- turnId: opts.telemetry.turnId,+ }),+ );+ if (receipt) enriched = { ...enriched, receiptId: receipt.id };+ const telemetry = emitGatedTelemetry(opts.telemetry, (telemetryOptions) =>+ emitRecallTelemetry(vault, {+ createdAt: telemetryOptions.createdAt,+ host: telemetryOptions.host,+ sessionId: telemetryOptions.sessionId,+ turnId: telemetryOptions.turnId,mode: "context_pack",status: report.items.length > 0 ? "ok" : "empty",durationMs: Date.now() - startedAtMs,@@ -429,15 +433,15 @@ function finalizeContextPackReport(topArtifacts: report.items.slice(0, 10).map((item) => ({ id: item.id, path: item.path })),gaps: contextPackGaps(report),metadata: {- ...(opts.telemetry.metadata ?? {}),+ ...(telemetryOptions.metadata ?? {}),...contextPackBudgetMetadata(opts, report),skipped_count: report.skipped.length,lanes: opts.includeLanes === true,...(enriched.receiptId ? { receipt_id: enriched.receiptId } : {}),},- });- enriched = { ...enriched, telemetryId: telemetry.id };- }+ }),+ );+ if (telemetry) enriched = { ...enriched, telemetryId: telemetry.id };return Object.freeze(enriched);}diff --git a/src/core/brain/continuity/emit.ts b/src/core/brain/continuity/emit.tsnew file mode 100644index 00000000..a31a0ab5--- /dev/null+++ b/src/core/brain/continuity/emit.ts@@ -0,0 +1,37 @@+/**+ * Lazy gated telemetry emit kernel (Memory Observability Suite,+ * t_5d7aa7c5).+ *+ * One helper enforces the two telemetry invariants structurally instead+ * of by convention at every call site:+ *+ * 1. No consumer, no payload work - with the gate off (`undefined`,+ * `false`, or `null`) the build thunk is NEVER invoked, so no+ * payload object, hash, or continuity write happens on the+ * no-listener path.+ * 2. Fail-open - telemetry must never fail the primary operation. A+ * throwing thunk (or a throwing continuity write inside it) is+ * swallowed and reported as `null`.+ *+ * The gate doubles as the payload-options carrier: passing+ * `opts.telemetry` directly keeps TypeScript narrowing inside the thunk+ * without non-null assertions. Boolean config gates pass `true`.+ *+ * Scope note: this kernel covers GATED telemetry surfaces (context-pack+ * receipts/telemetry, pre-compress receipts/telemetry, recall-gate+ * telemetry). Writers whose continuity append IS the primary operation+ * (session-recall import, pre-compact extract) stay fail-fast and do+ * not route through here.+ */++export function emitGatedTelemetry<G, T>(+ gate: G | false | null | undefined,+ build: (gate: G) => T,+): T | null {+ if (gate === false || gate === null || gate === undefined) return null;+ try {+ return build(gate);+ } catch {+ return null;+ }+}diff --git a/src/core/brain/continuity/export-atif.ts b/src/core/brain/continuity/export-atif.tsnew file mode 100644index 00000000..f6820c3c--- /dev/null+++ b/src/core/brain/continuity/export-atif.ts@@ -0,0 +1,110 @@+/**+ * ATIF renderer (Memory Observability Suite, t_51959aeb).+ *+ * Renders normalized continuity records as ATIF v1.7 trajectories+ * (harbor-framework RFC 0001) - one trajectory document per session.+ * Mapping decisions live in+ * docs/brainstorm/memory-observability-suite/atof-atif-mapping.md:+ *+ * - `session_turn` records become steps; `role: "user"` maps to+ * `source: "user"`, everything else to `source: "agent"`;+ * - memory-layer events (gate decisions, recall telemetry, receipts)+ * become `source: "system"` steps with `llm_call_count: 0` - the+ * spec's explicit deterministic-dispatch marker - and the full+ * payload under `extra.o2b`;+ * - records without a session id cannot be ordered into a trajectory+ * and are skipped (the CLI surfaces the skipped count);+ * - the memory layer itself is the recording agent.+ *+ * Read-only over the read-model; `private` records never reach this+ * module (the read-model drops them by default).+ */++import type { NormalizedContinuityRecord } from "./read-model.ts";++export const ATIF_SCHEMA_VERSION = "ATIF-v1.7";++export interface AtifStep {+ readonly step_id: number;+ readonly source: "system" | "user" | "agent";+ readonly message: string;+ readonly timestamp?: string;+ readonly llm_call_count?: number;+ readonly extra?: Readonly<Record<string, unknown>>;+}++export interface AtifTrajectory {+ readonly schema_version: string;+ readonly session_id: string;+ readonly agent: { readonly name: string; readonly version: string };+ readonly steps: ReadonlyArray<AtifStep>;+}++export interface AtifRenderOptions {+ readonly agentVersion: string;+}++/** Render one ATIF trajectory per session, ordered by record time. */+export function renderAtifTrajectories(+ records: ReadonlyArray<NormalizedContinuityRecord>,+ opts: AtifRenderOptions,+): ReadonlyArray<AtifTrajectory> {+ const bySession = new Map<string, NormalizedContinuityRecord[]>();+ for (const record of records) {+ if (record.sessionId === undefined) continue;+ const bucket = bySession.get(record.sessionId) ?? [];+ bucket.push(record);+ bySession.set(record.sessionId, bucket);+ }+ const trajectories: AtifTrajectory[] = [];+ for (const sessionId of [...bySession.keys()].toSorted()) {+ const sessionRecords = bySession+ .get(sessionId)!+ .toSorted((a, b) => a.createdAt.localeCompare(b.createdAt) || a.id.localeCompare(b.id));+ trajectories.push(+ Object.freeze({+ schema_version: ATIF_SCHEMA_VERSION,+ session_id: sessionId,+ agent: Object.freeze({ name: "open-second-brain", version: opts.agentVersion }),+ steps: Object.freeze(sessionRecords.map((record, index) => step(record, index + 1))),+ }),+ );+ }+ return Object.freeze(trajectories);+}++/** Count of records a render would skip (no session id to order by). */+export function countSessionlessRecords(+ records: ReadonlyArray<NormalizedContinuityRecord>,+): number {+ return records.filter((record) => record.sessionId === undefined).length;+}++function step(record: NormalizedContinuityRecord, stepId: number): AtifStep {+ if (record.kind === "session_turn") {+ const role = record.payload["role"];+ const text = record.payload["text"];+ return Object.freeze({+ step_id: stepId,+ source: role === "user" ? ("user" as const) : ("agent" as const),+ message: typeof text === "string" ? text : "",+ timestamp: record.createdAt,+ });+ }+ return Object.freeze({+ step_id: stepId,+ source: "system" as const,+ message: `o2b:${record.kind}`,+ timestamp: record.createdAt,+ // Deterministic dispatch: the memory layer made no LLM call.+ llm_call_count: 0,+ extra: Object.freeze({+ o2b: Object.freeze({+ record_id: record.id,+ schema: record.schema,+ kind: record.kind,+ payload: record.payload,+ }),+ }),+ });+}diff --git a/src/core/brain/continuity/export-atof.ts b/src/core/brain/continuity/export-atof.tsnew file mode 100644index 00000000..d9d88b89--- /dev/null+++ b/src/core/brain/continuity/export-atof.ts@@ -0,0 +1,131 @@+/**+ * ATOF renderer (Memory Observability Suite, t_51959aeb).+ *+ * Renders normalized continuity records as an ATOF (Agent Trajectory+ * Observability Format) JSONL event stream. Mapping decisions and the+ * spec revision matched are documented in+ * docs/brainstorm/memory-observability-suite/atof-atif-mapping.md:+ *+ * - `recall_telemetry` is the one duration-bearing kind and becomes a+ * `retriever` scope pair; the start timestamp is SYNTHESIZED as+ * `createdAt - duration_ms` and flagged `o2b.synthetic_start`;+ * - `gate_telemetry` maps to a `guardrail` mark, `context_receipt`+ * to a `retriever` mark, every other kind to a `custom` mark with+ * `category_profile.subtype: "o2b.<kind>"`;+ * - `parent_uuid` stays null - the memory layer does not know the+ * host agent's call tree; correlation rides the namespaced `o2b`+ * object (session_id / turn_id / record id);+ * - uuids derive deterministically from the record id so re-exports+ * are reproducible and diffable.+ *+ * Read-only over the read-model; `private` records never reach this+ * module (the read-model drops them by default).+ */++import { createHash } from "node:crypto";++import type { NormalizedContinuityRecord } from "./read-model.ts";++export const ATOF_VERSION = "0.1";++interface AtofEvent {+ readonly kind: "scope" | "mark";+ readonly atof_version: string;+ readonly uuid: string;+ readonly timestamp: string;+ readonly name: string;+ readonly category?: string;+ readonly scope_category?: "start" | "end";+ readonly attributes?: ReadonlyArray<string>;+ readonly category_profile?: Readonly<Record<string, unknown>>;+ readonly parent_uuid?: null;+ readonly o2b?: Readonly<Record<string, unknown>>;+}++/** Render records as ATOF JSONL lines (one JSON object per line). */+export function renderAtofEvents(+ records: ReadonlyArray<NormalizedContinuityRecord>,+): ReadonlyArray<string> {+ const events: AtofEvent[] = [];+ for (const record of records) {+ if (record.kind === "recall_telemetry") {+ events.push(...recallScopePair(record));+ } else {+ events.push(markEvent(record));+ }+ }+ return Object.freeze(events.map((event) => JSON.stringify(event)));+}++function recallScopePair(record: NormalizedContinuityRecord): AtofEvent[] {+ const uuid = deterministicUuid(record.id);+ const endMs = Date.parse(record.createdAt);+ const duration = readNonNegativeNumber(record.payload["duration_ms"]) ?? 0;+ const start = Number.isFinite(endMs)+ ? new Date(endMs - duration).toISOString()+ : record.createdAt;+ const end = Number.isFinite(endMs) ? new Date(endMs).toISOString() : record.createdAt;+ const name = `recall:${typeof record.payload["mode"] === "string" ? record.payload["mode"] : "unknown"}`;+ const base = {+ atof_version: ATOF_VERSION,+ uuid,+ name,+ category: "retriever",+ attributes: Object.freeze(["o2b.synthetic_start"]),+ parent_uuid: null,+ o2b: correlation(record),+ } as const;+ return [+ { kind: "scope", scope_category: "start", timestamp: start, ...base },+ { kind: "scope", scope_category: "end", timestamp: end, ...base },+ ];+}++function markEvent(record: NormalizedContinuityRecord): AtofEvent {+ const category = markCategory(record.kind);+ return {+ kind: "mark",+ atof_version: ATOF_VERSION,+ uuid: deterministicUuid(record.id),+ timestamp: record.createdAt,+ name: `o2b:${record.kind}`,+ category,+ ...(category === "custom"+ ? { category_profile: Object.freeze({ subtype: `o2b.${record.kind}` }) }+ : {}),+ parent_uuid: null,+ o2b: correlation(record),+ };+}++function markCategory(kind: string): string {+ if (kind === "gate_telemetry") return "guardrail";+ if (kind === "context_receipt") return "retriever";+ return "custom";+}++function correlation(record: NormalizedContinuityRecord): Readonly<Record<string, unknown>> {+ return Object.freeze({+ record_id: record.id,+ schema: record.schema,+ ...(record.sessionId !== undefined ? { session_id: record.sessionId } : {}),+ ...(record.turnId !== undefined ? { turn_id: record.turnId } : {}),+ payload: record.payload,+ });+}++/** Deterministic uuid-shaped id derived from the record id (reproducible re-exports). */+function deterministicUuid(recordId: string): string {+ const hex = createHash("sha256").update(`o2b-atof:${recordId}`).digest("hex");+ return [+ hex.slice(0, 8),+ hex.slice(8, 12),+ hex.slice(12, 16),+ hex.slice(16, 20),+ hex.slice(20, 32),+ ].join("-");+}++function readNonNegativeNumber(value: unknown): number | undefined {+ return typeof value === "number" && Number.isFinite(value) && value >= 0 ? value : undefined;+}diff --git a/src/core/brain/continuity/read-model.ts b/src/core/brain/continuity/read-model.tsnew file mode 100644index 00000000..aeac237f--- /dev/null+++ b/src/core/brain/continuity/read-model.ts@@ -0,0 +1,108 @@+/**+ * Continuity read-model (Memory Observability Suite kernel).+ *+ * The single normalization layer between the raw JSONL continuity+ * store and every read-side consumer (ATOF/ATIF export, bench+ * harness). It absorbs three concerns exactly once so consumers+ * cannot disagree on them:+ *+ * - schema-version dispatch: records written before the stamp+ * existed carry no `schema` field and read as v1 (`legacy: true`);+ * - masking policy: `private` records are DROPPED by default and+ * kept only on explicit request; payload text is already+ * redaction-masked at write time and is never un-masked here;+ * - fail-soft reads: malformed rows normalize to null and unknown+ * kinds stay readable (the evolution rule is additive).+ *+ * Read-only by construction - this module never writes to the store.+ */++import { listContinuityRecords } from "./store.ts";+import { CONTINUITY_SCHEMA_VERSION } from "./types.ts";++export interface NormalizedContinuityRecord {+ /** Effective schema version - legacy records report v1. */+ readonly schema: string;+ /** True when the on-disk record predates the version stamp. */+ readonly legacy: boolean;+ readonly id: string;+ readonly kind: string;+ readonly createdAt: string;+ readonly sourceRefs: ReadonlyArray<Readonly<Record<string, unknown>>>;+ readonly payload: Readonly<Record<string, unknown>>;+ readonly private: boolean;+ readonly redacted: boolean;+ /** Correlation ids lifted out of the payload when present. */+ readonly sessionId?: string;+ readonly turnId?: string;+}++export interface ContinuityReadModelFilter {+ readonly kind?: string;+ readonly sessionId?: string;+ readonly since?: string;+ readonly until?: string;+ /** Keep records flagged `private` (default: drop them). */+ readonly keepPrivate?: boolean;+}++/** Normalize one raw record (parsed JSONL row). Fail-soft: null on malformed input. */+export function normalizeContinuityRecord(raw: unknown): NormalizedContinuityRecord | null {+ if (raw === null || typeof raw !== "object" || Array.isArray(raw)) return null;+ const record = raw as Record<string, unknown>;+ const id = readString(record["id"]);+ const kind = readString(record["kind"]);+ const createdAt = readString(record["createdAt"]);+ if (id === undefined || kind === undefined || createdAt === undefined) return null;+ const schema = readString(record["schema"]);+ const payload = isPlainObject(record["payload"])+ ? (record["payload"] as Record<string, unknown>)+ : {};+ const sourceRefs = Array.isArray(record["sourceRefs"])+ ? (record["sourceRefs"] as unknown[]).filter(isPlainObject)+ : [];+ const sessionId = readString(payload["session_id"]);+ const turnId = readString(payload["turn_id"]);+ return Object.freeze({+ schema: schema ?? CONTINUITY_SCHEMA_VERSION,+ legacy: schema === undefined,+ id,+ kind,+ createdAt,+ sourceRefs: Object.freeze(sourceRefs.map((ref) => Object.freeze({ ...ref }))),+ payload: Object.freeze({ ...payload }),+ private: record["private"] === true,+ redacted: record["redacted"] === true,+ ...(sessionId !== undefined ? { sessionId } : {}),+ ...(turnId !== undefined ? { turnId } : {}),+ });+}++/** Load, normalize, and filter the whole store. Private records drop unless kept. */+export function loadNormalizedContinuityRecords(+ vault: string,+ filter: ContinuityReadModelFilter = {},+): ReadonlyArray<NormalizedContinuityRecord> {+ const records = listContinuityRecords(vault, {+ ...(filter.since !== undefined ? { since: filter.since } : {}),+ ...(filter.until !== undefined ? { until: filter.until } : {}),+ });+ const normalized: NormalizedContinuityRecord[] = [];+ for (const record of records) {+ const entry = normalizeContinuityRecord(record);+ if (entry === null) continue;+ if (entry.private && filter.keepPrivate !== true) continue;+ if (filter.kind !== undefined && entry.kind !== filter.kind) continue;+ if (filter.sessionId !== undefined && entry.sessionId !== filter.sessionId) continue;+ normalized.push(entry);+ }+ return Object.freeze(normalized);+}++function readString(value: unknown): string | undefined {+ return typeof value === "string" && value.length > 0 ? value : undefined;+}++function isPlainObject(value: unknown): value is Record<string, unknown> {+ return value !== null && typeof value === "object" && !Array.isArray(value);+}diff --git a/src/core/brain/continuity/store.ts b/src/core/brain/continuity/store.tsindex 9a0a48f3..1caf5d2a 100644--- a/src/core/brain/continuity/store.ts+++ b/src/core/brain/continuity/store.ts@@ -5,6 +5,7 @@ import { join } from "node:path";import { BRAIN_LOG_REL, ensureInsideVault } from "../paths.ts";import { acquireLockSync } from "../sync-lockfile.ts";import { safeContinuityPayload } from "./redaction.ts";+import { CONTINUITY_SCHEMA_VERSION } from "./types.ts";import type {AppendContinuityRecordInput,ContinuityRecord,@@ -96,8 +97,11 @@ function buildRecord(): ContinuityRecord {const payloadResult = safeContinuityPayload(input.payload ?? {});const sourceRefs = Object.freeze([...(input.sourceRefs ?? [])]);+ // `schema` stays OUT of recordId(): identical records must keep+ // identical dedup ids across the version-stamp transition.const id = recordId(input.kind, input.createdAt, sourceRefs, payloadResult.payload);return Object.freeze({+ schema: CONTINUITY_SCHEMA_VERSION,id,kind: input.kind,createdAt: input.createdAt,diff --git a/src/core/brain/continuity/types.ts b/src/core/brain/continuity/types.tsindex db77fc71..b7f40c22 100644--- a/src/core/brain/continuity/types.ts+++ b/src/core/brain/continuity/types.ts@@ -1,3 +1,14 @@+/**+ * Contract-wide continuity schema version (Memory Observability Suite,+ * t_26040ee8), stamped on every new record at `buildRecord()`.+ *+ * Evolution rule: additive optional fields do NOT bump the version;+ * renames, removals, or semantic changes bump to `o2b.continuity.v2`.+ * Records written before the stamp existed carry no `schema` field and+ * are read as v1. Existing JSONL files are never migrated.+ */+export const CONTINUITY_SCHEMA_VERSION = "o2b.continuity.v1";+export type ContinuityRecordKind =| "context_receipt"| "recall_telemetry"@@ -17,6 +28,13 @@ export interface ContinuitySourceRef {}export interface ContinuityRecord {+ /**+ * Schema version of the record's on-disk shape. `undefined` on legacy+ * records written before the stamp existed - readers treat that as v1.+ * Deliberately EXCLUDED from `recordId()` so identical records dedupe+ * identically across the stamp transition.+ */+ readonly schema?: string;readonly id: string;readonly kind: ContinuityRecordKind;readonly createdAt: string;diff --git a/src/core/brain/pre-compress-pack.ts b/src/core/brain/pre-compress-pack.tsindex 75aac699..ee80559e 100644--- a/src/core/brain/pre-compress-pack.ts+++ b/src/core/brain/pre-compress-pack.ts@@ -22,6 +22,7 @@ import { brainActivePath, brainDirs } from "./paths.ts";import { parsePreference } from "./preference.ts";import { applyCharBudget } from "./recall-budget.ts";import { emitContextReceipt, type ContextReceiptOptions } from "./context-receipts.ts";+import { emitGatedTelemetry } from "./continuity/emit.ts";import { emitRecallTelemetry, type RecallTelemetryOptions } from "./recall-telemetry.ts";import {contextSafetyReport,@@ -170,66 +171,69 @@ export function buildPreCompressPack(vault: string, opts: PreCompressOptions): P}const text = sections.join("\n\n");- const receipt = opts.receipt- ? emitContextReceipt(vault, {- options: opts.receipt,- items: [- ...(activeText !== null- ? [- {- id: ACTIVE_ID,- path: brainActivePath(vault),- text: activeText,- },- ]- : []),- ...items.map((item) => ({- id: item.id,- text: item.principle,- trimmed: item.trimmed,- safetyFiltered: item.safety?.filtered,- })),- ],- finalText: text,- budget: {- top_k: opts.topK,- ...(opts.maxCharsPerMemory !== undefined- ? { max_chars_per_memory: opts.maxCharsPerMemory }- : {}),- ...(opts.maxTotalChars !== undefined ? { max_total_chars: opts.maxTotalChars } : {}),- },- extra: { active_head_included: activeText !== null },- })- : null;-- const telemetry = opts.telemetry- ? emitRecallTelemetry(vault, {- createdAt: opts.telemetry.createdAt,- host: opts.telemetry.host,- sessionId: opts.telemetry.sessionId,- turnId: opts.telemetry.turnId,- mode: "pre_compress",- status: activeText !== null || items.length > 0 ? "ok" : "empty",- durationMs: Date.now() - startedAtMs,- resultCount: items.length + (activeText !== null ? 1 : 0),- topArtifacts: [- ...(activeText !== null ? [{ id: ACTIVE_ID, path: brainActivePath(vault) }] : []),- ...items.slice(0, 10).map((item) => ({ id: item.id })),- ],- gaps: activeText === null && items.length === 0 ? ["no_matching_context"] : [],- metadata: {- ...opts.telemetry.metadata,- top_k: opts.topK,- total_chars: budgeted.totalChars,- active_head_included: activeText !== null,- ...(opts.maxCharsPerMemory !== undefined- ? { max_chars_per_memory: opts.maxCharsPerMemory }- : {}),- ...(opts.maxTotalChars !== undefined ? { max_total_chars: opts.maxTotalChars } : {}),- ...(receipt ? { receipt_id: receipt.id } : {}),- },- })- : null;+ // Gated emissions route through the lazy emit kernel (t_5d7aa7c5):+ // option absent means the thunk never runs; a broken continuity+ // store can no longer fail the pack (fail-open).+ const receipt = emitGatedTelemetry(opts.receipt, (receiptOptions) =>+ emitContextReceipt(vault, {+ options: receiptOptions,+ items: [+ ...(activeText !== null+ ? [+ {+ id: ACTIVE_ID,+ path: brainActivePath(vault),+ text: activeText,+ },+ ]+ : []),+ ...items.map((item) => ({+ id: item.id,+ text: item.principle,+ trimmed: item.trimmed,+ safetyFiltered: item.safety?.filtered,+ })),+ ],+ finalText: text,+ budget: {+ top_k: opts.topK,+ ...(opts.maxCharsPerMemory !== undefined+ ? { max_chars_per_memory: opts.maxCharsPerMemory }+ : {}),+ ...(opts.maxTotalChars !== undefined ? { max_total_chars: opts.maxTotalChars } : {}),+ },+ extra: { active_head_included: activeText !== null },+ }),+ );++ const telemetry = emitGatedTelemetry(opts.telemetry, (telemetryOptions) =>+ emitRecallTelemetry(vault, {+ createdAt: telemetryOptions.createdAt,+ host: telemetryOptions.host,+ sessionId: telemetryOptions.sessionId,+ turnId: telemetryOptions.turnId,+ mode: "pre_compress",+ status: activeText !== null || items.length > 0 ? "ok" : "empty",+ durationMs: Date.now() - startedAtMs,+ resultCount: items.length + (activeText !== null ? 1 : 0),+ topArtifacts: [+ ...(activeText !== null ? [{ id: ACTIVE_ID, path: brainActivePath(vault) }] : []),+ ...items.slice(0, 10).map((item) => ({ id: item.id })),+ ],+ gaps: activeText === null && items.length === 0 ? ["no_matching_context"] : [],+ metadata: {+ ...telemetryOptions.metadata,+ top_k: opts.topK,+ total_chars: budgeted.totalChars,+ active_head_included: activeText !== null,+ ...(opts.maxCharsPerMemory !== undefined+ ? { max_chars_per_memory: opts.maxCharsPerMemory }+ : {}),+ ...(opts.maxTotalChars !== undefined ? { max_total_chars: opts.maxTotalChars } : {}),+ ...(receipt ? { receipt_id: receipt.id } : {}),+ },+ }),+ );return Object.freeze({text,diff --git a/src/core/config.ts b/src/core/config.tsindex f92b5175..851f6eb8 100644--- a/src/core/config.ts+++ b/src/core/config.ts@@ -366,6 +366,17 @@ export function resolveRecallGateTelemetry(configPath?: string): boolean {return raw === "true" || raw === "1";}+/**+ * Optional external judge command for the memory benchmark (Memory+ * Observability Suite, t_882c396a). Unset (the default) means the+ * judge phase is skipped - the harness itself never calls an LLM.+ */+export function resolveBenchJudgeCmd(configPath?: string): string | undefined {+ const env = process.env["OPEN_SECOND_BRAIN_BENCH_JUDGE_CMD"]?.trim();+ const raw = env || discoverConfig(configPath).data["bench_judge_cmd"]?.trim();+ return raw !== undefined && raw !== "" ? raw : undefined;+}+export const SESSION_CAPTURE_ROLES = ["user", "assistant", "system", "tool", "meta"] as const;export type SessionCaptureRole = (typeof SESSION_CAPTURE_ROLES)[number];diff --git a/src/mcp/search-tools.ts b/src/mcp/search-tools.tsindex baf6b144..fbd52b94 100644--- a/src/mcp/search-tools.ts+++ b/src/mcp/search-tools.ts@@ -30,6 +30,7 @@ import { MCP_PREVIEW_BUDGET } from "./preview-budget.ts";import { deriveRecallHint } from "../core/search/recall-hint.ts";import { emitRecallTelemetry } from "../core/brain/recall-telemetry.ts";import { emitGateTelemetry } from "../core/brain/gate-telemetry.ts";+import { emitGatedTelemetry } from "../core/brain/continuity/emit.ts";const MCP_LIMIT_MAX = 50;const MCP_CONTENT_MAX = 600;@@ -342,7 +343,9 @@ async function toolBrainSearch(searchTimeoutError,);} catch (e) {- if (telemetry) {+ // Lazy emit kernel (t_5d7aa7c5): a throwing telemetry write inside+ // this catch can no longer mask the original search error.+ emitGatedTelemetry(telemetry || undefined, () =>emitRecallTelemetry(ctx.vault, {host: telemetryHost,...(telemetrySessionId !== undefined ? { sessionId: telemetrySessionId } : {}),@@ -361,40 +364,40 @@ async function toolBrainSearch(keyword_only: keywordOnly,semantic: semantic ?? null,},- });- }+ }),+ );if (e instanceof SearchError) throw searchErrorToMcp(e);if (e instanceof MCPError) throw e;throw new MCPError(INTERNAL_ERROR, e instanceof Error ? e.message : String(e));}const recallHint = deriveRecallHint(outcome.results, outcome.total);- const telemetryRecord = telemetry- ? emitRecallTelemetry(ctx.vault, {- host: telemetryHost,- ...(telemetrySessionId !== undefined ? { sessionId: telemetrySessionId } : {}),- ...(telemetryTurnId !== undefined ? { turnId: telemetryTurnId } : {}),- mode: "search",- status: outcome.results.length > 0 ? "ok" : "empty",- durationMs: Date.now() - startedAtMs,- resultCount: outcome.results.length,- topArtifacts: outcome.results.slice(0, 10).map((result) => ({- id: `${result.documentId}:${result.chunkId}`,- path: result.path,- score: result.score,- })),- gaps: searchTelemetryGaps(outcome),- metadata: {- limit,- total: outcome.total,- keyword_only: keywordOnly,- semantic: semantic ?? null,- evidence_pack: evidencePack,- warnings_count: outcome.warnings.length,- ...(pathPrefix !== undefined ? { path_prefix: pathPrefix } : {}),- },- })- : null;+ const telemetryRecord = emitGatedTelemetry(telemetry || undefined, () =>+ emitRecallTelemetry(ctx.vault, {+ host: telemetryHost,+ ...(telemetrySessionId !== undefined ? { sessionId: telemetrySessionId } : {}),+ ...(telemetryTurnId !== undefined ? { turnId: telemetryTurnId } : {}),+ mode: "search",+ status: outcome.results.length > 0 ? "ok" : "empty",+ durationMs: Date.now() - startedAtMs,+ resultCount: outcome.results.length,+ topArtifacts: outcome.results.slice(0, 10).map((result) => ({+ id: `${result.documentId}:${result.chunkId}`,+ path: result.path,+ score: result.score,+ })),+ gaps: searchTelemetryGaps(outcome),+ metadata: {+ limit,+ total: outcome.total,+ keyword_only: keywordOnly,+ semantic: semantic ?? null,+ evidence_pack: evidencePack,+ warnings_count: outcome.warnings.length,+ ...(pathPrefix !== undefined ? { path_prefix: pathPrefix } : {}),+ },+ }),+ );return {results: outcome.results.map((r: BrainSearchResult) => ({path: r.path,@@ -496,23 +499,21 @@ async function toolBrainRecallGate(previousPrompt: previousPrompt ?? null,explicit,});- // Gate telemetry (t_65036e02): default off; fail-soft so a broken- // continuity store never breaks the gate's pure-diagnostic contract.- if (resolveRecallGateTelemetry(ctx.configPath ?? undefined)) {- try {- const host = coerceStringOptional(args, "telemetry_host", 200) ?? "mcp";- const sessionId = coerceStringOptional(args, "session_id", 512);- emitGateTelemetry(ctx.vault, {- host,- prompt,- retrieve: decision.retrieve,- reason: decision.reason,- ...(sessionId !== undefined ? { sessionId } : {}),- });- } catch {- // never let telemetry break the gate- }- }+ // Gate telemetry (t_65036e02): default off. Routed through the lazy+ // emit kernel (t_5d7aa7c5) - the payload thunk never runs with the+ // config off, and a broken continuity store never breaks the gate's+ // pure-diagnostic contract (fail-open).+ emitGatedTelemetry(resolveRecallGateTelemetry(ctx.configPath ?? undefined), () => {+ const host = coerceStringOptional(args, "telemetry_host", 200) ?? "mcp";+ const sessionId = coerceStringOptional(args, "session_id", 512);+ return emitGateTelemetry(ctx.vault, {+ host,+ prompt,+ retrieve: decision.retrieve,+ reason: decision.reason,+ ...(sessionId !== undefined ? { sessionId } : {}),+ });+ });return { ...decision };}diff --git a/tests/cli/brain-bench.test.ts b/tests/cli/brain-bench.test.tsnew file mode 100644index 00000000..3f9c4696--- /dev/null+++ b/tests/cli/brain-bench.test.ts@@ -0,0 +1,86 @@+/**+ * `o2b brain bench memory` (Memory Observability Suite, t_882c396a):+ * the CLI surface over the bench pipeline. The harness materializes a+ * disposable vault inside the runs directory and never resolves the+ * operator's configured vault.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { existsSync, mkdtempSync, readdirSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { runCli } from "../helpers/run-cli.ts";++let tmp: string;+let runsDir: string;++beforeEach(() => {+ tmp = mkdtempSync(join(tmpdir(), "o2b-bench-cli-"));+ runsDir = join(tmp, "bench-runs");+});++afterEach(() => {+ rmSync(tmp, { recursive: true, force: true });+});++describe("o2b brain bench memory", () => {+ test("runs the repo core-recall fixture green and reports separate metric families", async () => {+ const out = await runCli(+ ["brain", "bench", "memory", "--fixture", "core-recall", "--runs-dir", runsDir, "--json"],+ {},+ );+ expect(out.returncode).toBe(0);+ const report = JSON.parse(out.stdout) as {+ schema: string;+ run_id: string;+ quality: { passed: number; total: number };+ latency_ms: { avg: number };+ context_cost: { avg_chars: number };+ judge: { status: string };+ };+ expect(report.schema).toBe("o2b.bench.v1");+ expect(report.quality.total).toBe(6);+ expect(report.quality.passed).toBe(6);+ expect(report.judge.status).toBe("skipped");+ // The disposable vault lives inside the run directory.+ expect(existsSync(join(runsDir, report.run_id, "vault", "Brain"))).toBe(true);+ expect(existsSync(join(runsDir, report.run_id, "report.json"))).toBe(true);+ }, 60_000);++ test("resume reuses the checkpoint instead of creating a second run", async () => {+ const first = await runCli(+ ["brain", "bench", "memory", "--fixture", "stale-recall", "--runs-dir", runsDir, "--json"],+ {},+ );+ expect(first.returncode).toBe(0);+ const report = JSON.parse(first.stdout) as { run_id: string };+ const resumed = await runCli(+ [+ "brain",+ "bench",+ "memory",+ "--fixture",+ "stale-recall",+ "--runs-dir",+ runsDir,+ "--resume",+ report.run_id,+ "--json",+ ],+ {},+ );+ expect(resumed.returncode).toBe(0);+ expect((JSON.parse(resumed.stdout) as { run_id: string }).run_id).toBe(report.run_id);+ expect(readdirSync(runsDir)).toHaveLength(1);+ }, 60_000);++ test("an unknown fixture fails with a clear error", async () => {+ const out = await runCli(+ ["brain", "bench", "memory", "--fixture", "no-such-fixture", "--runs-dir", runsDir],+ {},+ );+ expect(out.returncode).not.toBe(0);+ expect(out.stderr).toContain("fixture not found");+ });+});diff --git a/tests/cli/brain-continuity.test.ts b/tests/cli/brain-continuity.test.tsnew file mode 100644index 00000000..c941dbf7--- /dev/null+++ b/tests/cli/brain-continuity.test.ts@@ -0,0 +1,100 @@+/**+ * `o2b brain continuity export` (Memory Observability Suite,+ * t_51959aeb): read-only ATOF/ATIF export of the continuity store.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { bootstrapBrain } from "../../src/core/brain/init.ts";+import { appendContinuityRecord } from "../../src/core/brain/continuity/store.ts";+import { runCli } from "../helpers/run-cli.ts";++let tmp: string;+let vault: string;+let configPath: string;+let outDir: string;++beforeEach(() => {+ tmp = mkdtempSync(join(tmpdir(), "o2b-continuity-cli-"));+ vault = join(tmp, "vault");+ configPath = join(tmp, "config.yaml");+ outDir = join(tmp, "out");+ writeFileSync(configPath, `vault: ${vault}\nagent_name: test-agent\n`);+ bootstrapBrain(vault, { configPath });+ appendContinuityRecord(vault, {+ kind: "session_turn",+ createdAt: "2026-06-03T10:00:00.000Z",+ payload: { session_id: "s-1", turn_id: "t-1", role: "user", text: "hello" },+ });+ appendContinuityRecord(vault, {+ kind: "gate_telemetry",+ createdAt: "2026-06-03T10:00:01.000Z",+ payload: { host: "mcp", session_id: "s-1", decision: "skip", reason: "smalltalk" },+ });+ // A private record must never reach an export file.+ appendContinuityRecord(vault, {+ kind: "session_turn",+ createdAt: "2026-06-03T10:00:02.000Z",+ payload: {+ session_id: "s-1",+ turn_id: "t-2",+ role: "user",+ text: "carries a <private>secret plan</private> region",+ },+ });+});++afterEach(() => {+ rmSync(tmp, { recursive: true, force: true });+});++const env = () => ({ OPEN_SECOND_BRAIN_CONFIG: configPath });++describe("o2b brain continuity export", () => {+ test("atof export writes one JSONL stream and drops private records", async () => {+ const out = await runCli(+ ["brain", "continuity", "export", "--format", "atof", "--out", outDir, "--json"],+ { env: env() },+ );+ expect(out.returncode).toBe(0);+ const parsed = JSON.parse(out.stdout) as { records: number; files: string[] };+ expect(parsed.records).toBe(2);+ expect(parsed.files).toHaveLength(1);+ const content = readFileSync(parsed.files[0]!, "utf8");+ expect(content).toContain('"atof_version"');+ expect(content).not.toContain("secret plan");+ });++ test("atif export writes one trajectory per session", async () => {+ const out = await runCli(+ ["brain", "continuity", "export", "--format", "atif", "--out", outDir],+ { env: env() },+ );+ expect(out.returncode).toBe(0);+ const path = join(outDir, "o2b-continuity-s-1.atif.json");+ expect(existsSync(path)).toBe(true);+ const trajectory = JSON.parse(readFileSync(path, "utf8")) as {+ schema_version: string;+ steps: Array<{ source: string }>;+ };+ expect(trajectory.schema_version).toBe("ATIF-v1.7");+ expect(trajectory.steps).toHaveLength(2);+ expect(trajectory.steps.some((s) => s.source === "system")).toBe(true);+ });++ test("unknown format and bad month fail with usage errors", async () => {+ const bad = await runCli(+ ["brain", "continuity", "export", "--format", "csv", "--out", outDir],+ { env: env() },+ );+ expect(bad.returncode).not.toBe(0);+ const badMonth = await runCli(+ ["brain", "continuity", "export", "--format", "atof", "--month", "June", "--out", outDir],+ { env: env() },+ );+ expect(badMonth.returncode).not.toBe(0);+ });+});diff --git a/tests/core/bench/fixture.test.ts b/tests/core/bench/fixture.test.tsnew file mode 100644index 00000000..72e47e48--- /dev/null+++ b/tests/core/bench/fixture.test.ts@@ -0,0 +1,113 @@+/**+ * Bench fixture loading and materialization (Memory Observability+ * Suite, t_882c396a). Fixtures are repo-local JSON; materialization+ * writes ONLY into the caller-supplied disposable directory.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import {+ fixtureHash,+ loadBenchFixture,+ materializeBenchVault,+ parseBenchFixture,+} from "../../../src/core/bench/fixture.ts";+import { loadNormalizedContinuityRecords } from "../../../src/core/brain/continuity/read-model.ts";++let tmp: string;++beforeEach(() => {+ tmp = mkdtempSync(join(tmpdir(), "o2b-bench-fixture-"));+});++afterEach(() => {+ rmSync(tmp, { recursive: true, force: true });+});++const FIXTURE = {+ name: "mini",+ description: "smallest viable fixture",+ notes: [+ { path: "Brain/notes/coffee.md", body: "# Coffee\n\nThe operator prefers flat white.\n" },+ ],+ continuity: [+ {+ kind: "session_turn",+ created_at: "2026-06-01T10:00:00.000Z",+ payload: { session_id: "s-1", turn_id: "t-1", role: "user", text: "remember flat white" },+ },+ ],+ questions: [+ {+ id: "q1",+ category: "single_hop",+ query: "flat white coffee preference",+ top_k: 5,+ expected_paths: ["Brain/notes/coffee.md"],+ },+ ],+};++describe("parseBenchFixture", () => {+ test("accepts a well-formed fixture and rejects malformed ones", () => {+ const fixture = parseBenchFixture(FIXTURE);+ expect(fixture.name).toBe("mini");+ expect(fixture.questions).toHaveLength(1);++ expect(() => parseBenchFixture(null)).toThrow();+ expect(() => parseBenchFixture({ ...FIXTURE, name: "" })).toThrow("name");+ expect(() => parseBenchFixture({ ...FIXTURE, questions: [] })).toThrow("question");+ expect(() =>+ parseBenchFixture({+ ...FIXTURE,+ notes: [{ path: "../escape.md", body: "x" }],+ }),+ ).toThrow("path");+ expect(() =>+ parseBenchFixture({+ ...FIXTURE,+ notes: [{ path: "/abs/escape.md", body: "x" }],+ }),+ ).toThrow("path");+ expect(() =>+ parseBenchFixture({+ ...FIXTURE,+ questions: [{ id: "q1", category: "unknown_kind", query: "x" }],+ }),+ ).toThrow("category");+ });++ test("fixtureHash is stable across key order and changes with content", () => {+ const a = fixtureHash(parseBenchFixture(FIXTURE));+ const reordered = JSON.parse(JSON.stringify(FIXTURE)) as typeof FIXTURE;+ const b = fixtureHash(parseBenchFixture(reordered));+ expect(a).toBe(b);+ const c = fixtureHash(parseBenchFixture({ ...FIXTURE, name: "other" }));+ expect(c).not.toBe(a);+ });++ test("loadBenchFixture reads a file and fails fast on broken JSON", () => {+ const path = join(tmp, "fixture.json");+ writeFileSync(path, JSON.stringify(FIXTURE));+ expect(loadBenchFixture(path).name).toBe("mini");+ writeFileSync(path, "{broken");+ expect(() => loadBenchFixture(path)).toThrow();+ });+});++describe("materializeBenchVault", () => {+ test("writes notes and continuity records into the disposable vault", () => {+ const vault = join(tmp, "vault");+ materializeBenchVault(parseBenchFixture(FIXTURE), vault);+ expect(existsSync(join(vault, "Brain", "notes", "coffee.md"))).toBe(true);+ expect(readFileSync(join(vault, "Brain", "notes", "coffee.md"), "utf8")).toContain(+ "flat white",+ );+ const turns = loadNormalizedContinuityRecords(vault, { kind: "session_turn" });+ expect(turns).toHaveLength(1);+ expect(turns[0]!.sessionId).toBe("s-1");+ });+});diff --git a/tests/core/bench/phases.test.ts b/tests/core/bench/phases.test.tsnew file mode 100644index 00000000..02192407--- /dev/null+++ b/tests/core/bench/phases.test.ts@@ -0,0 +1,186 @@+/**+ * Bench phase pipeline (Memory Observability Suite, t_882c396a):+ * ingest -> index -> retrieve -> evaluate -> report over a disposable+ * fixture vault. Deterministic, no network; quality / latency /+ * context cost stay separate numbers; resume skips completed phases;+ * a stale-fact fixture catches superseded-recall regressions.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { existsSync, mkdtempSync, readFileSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { parseBenchFixture } from "../../../src/core/bench/fixture.ts";+import { runMemoryBench } from "../../../src/core/bench/phases.ts";+import { BENCH_REPORT_SCHEMA } from "../../../src/core/bench/types.ts";++let runsDir: string;++beforeEach(() => {+ runsDir = mkdtempSync(join(tmpdir(), "o2b-bench-phases-"));+});++afterEach(() => {+ rmSync(runsDir, { recursive: true, force: true });+});++const FIXTURE = parseBenchFixture({+ name: "phase-suite",+ notes: [+ {+ path: "Brain/notes/coffee.md",+ body: "# Coffee preference\n\nCoffee preference of the operator: flat white every morning. The coffee preference is stable.\n",+ },+ {+ path: "Brain/notes/coffee-old.md",+ body: "# Superseded note\n\nOld coffee preference: espresso. Replaced by the flat white note.\n",+ },+ {+ path: "Brain/notes/deploy-a.md",+ body: "# Deploy step A\n\nDeployment requires the staging gate to pass first.\n",+ },+ {+ path: "Brain/notes/deploy-b.md",+ body: "# Deploy step B\n\nDeployment finishes with the production smoke checklist.\n",+ },+ {+ path: "Brain/preferences/pref-tabs.md",+ body: "---\nid: pref-tabs\ntopic: tabs\nstatus: confirmed\nprinciple: Use tabs for indentation in Go files\ncreated_at: 2026-05-01T10:00:00\n---\n\nUse tabs for indentation in Go files\n",+ },+ ],+ continuity: [+ {+ kind: "session_turn",+ created_at: "2026-06-01T10:00:00.000Z",+ payload: { session_id: "s-1", turn_id: "t-1", role: "user", text: "handoff: flat white" },+ },+ {+ kind: "session_turn",+ created_at: "2026-06-01T10:01:00.000Z",+ payload: { session_id: "s-1", turn_id: "t-2", role: "assistant", text: "noted" },+ },+ ],+ questions: [+ {+ id: "q-single",+ category: "single_hop",+ query: "flat white",+ top_k: 5,+ expected_paths: ["Brain/notes/coffee.md"],+ },+ {+ id: "q-stale",+ category: "temporal",+ query: "coffee preference",+ top_k: 5,+ expected_paths: ["Brain/notes/coffee.md"],+ not_expected_above: ["Brain/notes/coffee-old.md"],+ },+ {+ id: "q-multi",+ category: "multi_evidence",+ query: "deployment",+ top_k: 5,+ expected_paths: ["Brain/notes/deploy-a.md", "Brain/notes/deploy-b.md"],+ },+ {+ id: "q-handoff",+ category: "session_handoff",+ session_id: "s-1",+ expected_turns: 2,+ expected_text: "flat white",+ },+ {+ id: "q-budget",+ category: "budget",+ expected_ids: ["pref-tabs"],+ max_tokens: 500,+ },+ ],+});++describe("runMemoryBench", () => {+ test("full pipeline produces a diffable report with separate metric families", async () => {+ const report = await runMemoryBench({ fixture: FIXTURE, runsDir });+ expect(report.schema).toBe(BENCH_REPORT_SCHEMA);+ expect(report.fixture).toBe("phase-suite");+ expect(report.quality.total).toBe(5);+ expect(report.quality.passed).toBe(5);+ expect(report.quality.pass_rate).toBe(1);+ expect(report.quality.by_category["single_hop"]).toEqual({ passed: 1, total: 1 });+ // Metric families stay separate - never one collapsed score.+ expect(report.latency_ms.avg).toBeGreaterThanOrEqual(0);+ expect(report.latency_ms.max).toBeGreaterThanOrEqual(report.latency_ms.avg);+ expect(report.context_cost.avg_chars).toBeGreaterThan(0);+ expect(report.context_cost.est_tokens).toBeGreaterThan(0);+ expect(report.judge.status).toBe("skipped");+ // Stable question order by id for diffability.+ expect(report.questions.map((q) => q.id)).toEqual(report.questions.map((q) => q.id).toSorted());+ // report.json lands in the run directory.+ const onDisk = JSON.parse(+ readFileSync(join(runsDir, report.run_id, "report.json"), "utf8"),+ ) as {+ schema: string;+ };+ expect(onDisk.schema).toBe(BENCH_REPORT_SCHEMA);+ }, 30_000);++ test("a stale-fact regression is caught: superseded note ranking above fails the question", async () => {+ const regression = parseBenchFixture({+ name: "stale-regression",+ notes: [+ {+ path: "Brain/notes/new.md",+ body: "# Current\n\nThe coffee preference today is flat white, replacing every older note about it.\n",+ },+ {+ path: "Brain/notes/old.md",+ body: "# Old\n\nCoffee preference coffee preference coffee preference: espresso coffee preference, morning coffee preference.\n",+ },+ ],+ questions: [+ {+ id: "q-stale",+ category: "temporal",+ query: "coffee preference",+ top_k: 5,+ expected_paths: ["Brain/notes/new.md"],+ not_expected_above: ["Brain/notes/old.md"],+ },+ ],+ });+ const report = await runMemoryBench({ fixture: regression, runsDir });+ expect(report.quality.passed).toBe(0);+ const q = report.questions.find((entry) => entry.id === "q-stale")!;+ expect(q.pass).toBe(false);+ expect(q.failure).toContain("Brain/notes/old.md");+ }, 30_000);++ test("resume by run id skips completed phases and reuses stored results", async () => {+ const first = await runMemoryBench({ fixture: FIXTURE, runsDir });+ // Remove the disposable vault: a resumed run must NOT need to+ // re-ingest or re-search - evaluation reuses stored results.+ rmSync(join(runsDir, first.run_id, "vault"), { recursive: true, force: true });+ const resumed = await runMemoryBench({+ fixture: FIXTURE,+ runsDir,+ resume: first.run_id,+ });+ expect(resumed.run_id).toBe(first.run_id);+ expect(resumed.quality).toEqual(first.quality);+ expect(existsSync(join(runsDir, first.run_id, "vault"))).toBe(false);+ }, 30_000);++ test("resume with a changed fixture fails fast on the hash guard", async () => {+ const first = await runMemoryBench({ fixture: FIXTURE, runsDir });+ const changed = parseBenchFixture({+ ...JSON.parse(JSON.stringify(FIXTURE)),+ name: "phase-suite",+ notes: [{ path: "Brain/notes/x.md", body: "different" }],+ });+ await expect(+ runMemoryBench({ fixture: changed, runsDir, resume: first.run_id }),+ ).rejects.toThrow("hash");+ }, 30_000);+});diff --git a/tests/core/bench/run-store.test.ts b/tests/core/bench/run-store.test.tsnew file mode 100644index 00000000..dac5db3b--- /dev/null+++ b/tests/core/bench/run-store.test.ts@@ -0,0 +1,88 @@+/**+ * Bench run store (Memory Observability Suite, t_882c396a):+ * checkpointed runs under a runs directory, resume by run id, fixture+ * hash validation, and a path guard that refuses to operate outside+ * the runs directory.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { existsSync, mkdtempSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { parseBenchFixture } from "../../../src/core/bench/fixture.ts";+import {+ benchVaultDir,+ completeBenchPhase,+ createBenchRun,+ loadBenchRun,+} from "../../../src/core/bench/run-store.ts";++let runsDir: string;++beforeEach(() => {+ runsDir = mkdtempSync(join(tmpdir(), "o2b-bench-runs-"));+});++afterEach(() => {+ rmSync(runsDir, { recursive: true, force: true });+});++const FIXTURE = parseBenchFixture({+ name: "mini",+ notes: [{ path: "Brain/notes/a.md", body: "alpha" }],+ questions: [+ {+ id: "q1",+ category: "single_hop",+ query: "alpha",+ expected_paths: ["Brain/notes/a.md"],+ },+ ],+});++describe("bench run store", () => {+ test("createBenchRun writes a checkpoint and loadBenchRun round-trips it", () => {+ const run = createBenchRun(runsDir, FIXTURE, { now: new Date("2026-06-03T12:00:00Z") });+ expect(run.runId).toStartWith("run-20260603120000-");+ expect(existsSync(join(run.runDir, "checkpoint.json"))).toBe(true);+ expect(run.checkpoint.completed_phases).toEqual([]);++ const loaded = loadBenchRun(runsDir, run.runId);+ expect(loaded.checkpoint.run_id).toBe(run.runId);+ expect(loaded.checkpoint.fixture_name).toBe("mini");+ expect(benchVaultDir(loaded.runDir)).toBe(join(loaded.runDir, "vault"));+ });++ test("completeBenchPhase persists progress exactly once per phase", () => {+ const run = createBenchRun(runsDir, FIXTURE, { now: new Date("2026-06-03T12:00:00Z") });+ const afterIngest = completeBenchPhase(run.runDir, run.checkpoint, "ingest");+ expect(afterIngest.completed_phases).toEqual(["ingest"]);+ const again = completeBenchPhase(run.runDir, afterIngest, "ingest");+ expect(again.completed_phases).toEqual(["ingest"]);+ const reloaded = loadBenchRun(runsDir, run.runId);+ expect(reloaded.checkpoint.completed_phases).toEqual(["ingest"]);+ });++ test("a fixture-hash mismatch invalidates resume instead of mixing runs", () => {+ const run = createBenchRun(runsDir, FIXTURE, { now: new Date("2026-06-03T12:00:00Z") });+ const changed = parseBenchFixture({+ name: "mini",+ notes: [{ path: "Brain/notes/a.md", body: "CHANGED" }],+ questions: [+ {+ id: "q1",+ category: "single_hop",+ query: "alpha",+ expected_paths: ["Brain/notes/a.md"],+ },+ ],+ });+ expect(() => loadBenchRun(runsDir, run.runId, { expectFixture: changed })).toThrow("hash");+ });++ test("unknown run ids and path-escaping ids are rejected", () => {+ expect(() => loadBenchRun(runsDir, "run-nope")).toThrow();+ expect(() => loadBenchRun(runsDir, "../escape")).toThrow();+ });+});diff --git a/tests/core/brain/continuity-emit.test.ts b/tests/core/brain/continuity-emit.test.tsnew file mode 100644index 00000000..154b0b17--- /dev/null+++ b/tests/core/brain/continuity-emit.test.ts@@ -0,0 +1,129 @@+/**+ * Lazy gated telemetry emit kernel (Memory Observability Suite,+ * t_5d7aa7c5).+ *+ * The structural guarantee: "no consumer means no payload work". With+ * the gate off the payload thunk is never invoked and nothing reaches+ * the continuity store; a throwing thunk or write never fails the+ * primary operation (fail-open). Gated surfaces route through this one+ * helper instead of re-implementing the shape by convention.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { emitGatedTelemetry } from "../../../src/core/brain/continuity/emit.ts";+import { packContext } from "../../../src/core/brain/context-pack.ts";+import { buildPreCompressPack } from "../../../src/core/brain/pre-compress-pack.ts";+import { captureSessionLifecycleEvent } from "../../../src/core/brain/session-lifecycle.ts";++let vault: string;++const CONTINUITY_DIR = ["Brain", "log", "continuity"];++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-continuity-emit-"));+ mkdirSync(join(vault, "Brain", "preferences"), { recursive: true });+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+});++function continuityDir(): string {+ return join(vault, ...CONTINUITY_DIR);+}++function writePref(slug: string, principle: string): void {+ writeFileSync(+ join(vault, "Brain", "preferences", `pref-${slug}.md`),+ `---\nid: pref-${slug}\ntopic: ${slug}\nprinciple: ${principle}\n---\n\n${principle}\n`,+ );+}++/** Force every continuity append to throw: the store's directory path exists as a FILE. */+function blockContinuityWrites(): void {+ mkdirSync(join(vault, "Brain", "log"), { recursive: true });+ writeFileSync(continuityDir(), "not a directory");+}++describe("emitGatedTelemetry", () => {+ test("gate off: the thunk is never invoked and null comes back", () => {+ let invoked = 0;+ for (const gate of [undefined, false, null]) {+ const result = emitGatedTelemetry(gate, () => {+ invoked++;+ return "never";+ });+ expect(result).toBeNull();+ }+ expect(invoked).toBe(0);+ });++ test("gate on: the gate value flows into the thunk and the result comes back", () => {+ const result = emitGatedTelemetry({ host: "cli" }, (gate) => `host=${gate.host}`);+ expect(result).toBe("host=cli");+ expect(emitGatedTelemetry(true, () => 42)).toBe(42);+ });++ test("a throwing thunk is swallowed: fail-open returns null", () => {+ const result = emitGatedTelemetry(true, () => {+ throw new Error("continuity store on fire");+ });+ expect(result).toBeNull();+ });+});++describe("no-consumer regression: gated surfaces stay silent with gates off", () => {+ test("packContext without receipt/telemetry options writes nothing to continuity", () => {+ writePref("alpha", "Always alpha");+ const report = packContext(vault, { maxTokens: 500 });+ expect(report.items.length).toBe(1);+ expect(existsSync(continuityDir())).toBe(false);+ });++ test("buildPreCompressPack without receipt/telemetry options writes nothing to continuity", () => {+ writePref("beta", "Always beta");+ const pack = buildPreCompressPack(vault, { topK: 3 });+ expect(pack.items.length).toBeGreaterThanOrEqual(0);+ expect(existsSync(continuityDir())).toBe(false);+ });++ test("session lifecycle capture with defaults writes zero continuity records", async () => {+ const result = await captureSessionLifecycleEvent(+ vault,+ { hook_event_name: "UserPromptSubmit", session_id: "s-1", prompt: "hello world" },+ { agent: "test-agent" },+ );+ expect(result.event).toBe("UserPromptSubmit");+ expect(existsSync(continuityDir())).toBe(false);+ });+});++describe("fail-open: a broken continuity store never fails the primary operation", () => {+ test("packContext with telemetry requested still returns a pack, without telemetryId", () => {+ writePref("gamma", "Always gamma");+ blockContinuityWrites();+ const report = packContext(vault, {+ maxTokens: 500,+ telemetry: { host: "test" },+ receipt: { host: "test", trigger: "context_pack" },+ });+ expect(report.items.length).toBe(1);+ expect(report.telemetryId).toBeUndefined();+ expect(report.receiptId).toBeUndefined();+ });++ test("buildPreCompressPack with telemetry requested still returns a pack", () => {+ writePref("delta", "Always delta");+ blockContinuityWrites();+ const pack = buildPreCompressPack(vault, {+ topK: 3,+ telemetry: { host: "test" },+ });+ expect(pack.text.length).toBeGreaterThanOrEqual(0);+ expect(pack.telemetryId).toBeUndefined();+ });+});diff --git a/tests/core/brain/continuity-export.test.ts b/tests/core/brain/continuity-export.test.tsnew file mode 100644index 00000000..fb9eaef3--- /dev/null+++ b/tests/core/brain/continuity-export.test.ts@@ -0,0 +1,160 @@+/**+ * ATOF/ATIF trajectory export (Memory Observability Suite, t_51959aeb).+ *+ * Golden-shape tests over the continuity read-model: ATOF JSONL events+ * (scope pairs for duration-bearing recalls, marks for the rest) and+ * one ATIF trajectory per session. Mapping decisions live in+ * docs/brainstorm/memory-observability-suite/atof-atif-mapping.md.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdtempSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import {+ appendContinuityRecord,+ appendContinuitySourceInvalidation,+} from "../../../src/core/brain/continuity/store.ts";+import { loadNormalizedContinuityRecords } from "../../../src/core/brain/continuity/read-model.ts";+import { ATOF_VERSION, renderAtofEvents } from "../../../src/core/brain/continuity/export-atof.ts";+import {+ ATIF_SCHEMA_VERSION,+ renderAtifTrajectories,+} from "../../../src/core/brain/continuity/export-atif.ts";++let vault: string;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-continuity-export-"));+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+});++function seedRecords(): void {+ appendContinuityRecord(vault, {+ kind: "session_turn",+ createdAt: "2026-06-03T10:00:00.000Z",+ payload: { session_id: "s-1", turn_id: "t-1", role: "user", text: "find the index notes" },+ });+ appendContinuityRecord(vault, {+ kind: "gate_telemetry",+ createdAt: "2026-06-03T10:00:01.000Z",+ payload: {+ host: "mcp",+ session_id: "s-1",+ decision: "retrieve",+ reason: "question_shape",+ prompt_hash: "abcdef0123456789",+ prompt_chars: 20,+ },+ });+ appendContinuityRecord(vault, {+ kind: "recall_telemetry",+ createdAt: "2026-06-03T10:00:02.500Z",+ payload: {+ host: "mcp",+ session_id: "s-1",+ mode: "search",+ status: "ok",+ duration_ms: 1500,+ result_count: 2,+ top_artifacts: [],+ gaps: [],+ },+ });+ appendContinuityRecord(vault, {+ kind: "session_turn",+ createdAt: "2026-06-03T10:00:03.000Z",+ payload: { session_id: "s-1", turn_id: "t-2", role: "assistant", text: "found two notes" },+ });+}++describe("renderAtofEvents", () => {+ test("recall telemetry becomes a retriever scope pair with one shared uuid", () => {+ seedRecords();+ const records = loadNormalizedContinuityRecords(vault);+ const lines = renderAtofEvents(records);+ const events = lines.map((line) => JSON.parse(line) as Record<string, unknown>);++ // Every event carries the required ATOF fields.+ for (const event of events) {+ expect(event["atof_version"]).toBe(ATOF_VERSION);+ expect(typeof event["uuid"]).toBe("string");+ expect(typeof event["timestamp"]).toBe("string");+ expect(typeof event["name"]).toBe("string");+ const kind = event["kind"];+ expect(kind === "scope" || kind === "mark").toBe(true);+ }++ const scopes = events.filter((event) => event["kind"] === "scope");+ expect(scopes).toHaveLength(2);+ expect(scopes[0]!["scope_category"]).toBe("start");+ expect(scopes[1]!["scope_category"]).toBe("end");+ expect(scopes[0]!["uuid"]).toBe(scopes[1]!["uuid"]);+ expect(scopes[0]!["category"]).toBe("retriever");+ // The synthesized start sits duration_ms before the recorded end.+ expect(scopes[0]!["timestamp"]).toBe("2026-06-03T10:00:01.000Z");+ expect(scopes[1]!["timestamp"]).toBe("2026-06-03T10:00:02.500Z");+ const attrs = scopes[0]!["attributes"];+ expect(Array.isArray(attrs)).toBe(true);+ expect(attrs as string[]).toContain("o2b.synthetic_start");++ // Marks: gate decision is a guardrail, turns are custom with subtype.+ const gate = events.find((event) => (event["name"] as string).includes("gate"));+ expect(gate!["kind"]).toBe("mark");+ expect(gate!["category"]).toBe("guardrail");+ const turn = events.find((event) => (event["name"] as string).includes("session_turn"));+ expect(turn!["category"]).toBe("custom");+ expect((turn!["category_profile"] as Record<string, unknown>)["subtype"]).toBe(+ "o2b.session_turn",+ );+ });++ test("the export is deterministic: same records, same lines", () => {+ seedRecords();+ const records = loadNormalizedContinuityRecords(vault);+ expect(renderAtofEvents(records)).toEqual(renderAtofEvents(records));+ });+});++describe("renderAtifTrajectories", () => {+ test("one trajectory per session with renumbered steps and llm_call_count 0 system steps", () => {+ seedRecords();+ appendContinuityRecord(vault, {+ kind: "session_turn",+ createdAt: "2026-06-03T11:00:00.000Z",+ payload: { session_id: "s-2", turn_id: "t-1", role: "user", text: "other session" },+ });+ const records = loadNormalizedContinuityRecords(vault);+ const trajectories = renderAtifTrajectories(records, { agentVersion: "0.39.0" });+ expect(trajectories).toHaveLength(2);++ const first = trajectories.find((t) => t.session_id === "s-1")!;+ expect(first.schema_version).toBe(ATIF_SCHEMA_VERSION);+ expect(first.agent).toEqual({ name: "open-second-brain", version: "0.39.0" });+ const steps = first.steps;+ expect(steps.map((s) => s.step_id)).toEqual([1, 2, 3, 4]);+ expect(steps[0]!.source).toBe("user");+ expect(steps[0]!.message).toBe("find the index notes");+ // Memory-layer events are deterministic system steps.+ const gateStep = steps.find((s) => s.source === "system")!;+ expect(gateStep.llm_call_count).toBe(0);+ expect((gateStep.extra as Record<string, unknown>)["o2b"]).toBeDefined();+ // Assistant turns map to agent steps.+ expect(steps[3]!.source).toBe("agent");+ });++ test("records without a session id are skipped and counted", () => {+ appendContinuitySourceInvalidation(vault, {+ createdAt: "2026-06-03T10:00:00.000Z",+ source: { id: "note-x" },+ reason: "drift",+ });+ const records = loadNormalizedContinuityRecords(vault);+ const trajectories = renderAtifTrajectories(records, { agentVersion: "0.39.0" });+ expect(trajectories).toHaveLength(0);+ });+});diff --git a/tests/core/brain/continuity-read-model.test.ts b/tests/core/brain/continuity-read-model.test.tsnew file mode 100644index 00000000..76033814--- /dev/null+++ b/tests/core/brain/continuity-read-model.test.ts@@ -0,0 +1,127 @@+/**+ * Continuity read-model (Memory Observability Suite kernel).+ *+ * One normalization layer between the raw JSONL store and every+ * read-side consumer (ATOF/ATIF export, bench harness): schema-version+ * dispatch with legacy records reading as v1, a caller-explicit masking+ * policy for `private` records, and fail-soft handling of unknown kinds+ * and malformed rows - so consumers cannot disagree on any of it.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdtempSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { appendContinuityRecord } from "../../../src/core/brain/continuity/store.ts";+import {+ loadNormalizedContinuityRecords,+ normalizeContinuityRecord,+} from "../../../src/core/brain/continuity/read-model.ts";+import { CONTINUITY_SCHEMA_VERSION } from "../../../src/core/brain/continuity/types.ts";++let vault: string;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-read-model-"));+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+});++const LEGACY_RAW = {+ id: "ctn_20260101120000_aaaaaaaaaaaaaaaa",+ kind: "recall_telemetry",+ createdAt: "2026-01-01T12:00:00Z",+ sourceRefs: [{ id: "note-a" }],+ payload: { status: "ok" },+ private: false,+ redacted: false,+};++describe("normalizeContinuityRecord", () => {+ test("a stamped record keeps its version; a legacy record reads as v1", () => {+ const stamped = normalizeContinuityRecord({+ ...LEGACY_RAW,+ schema: CONTINUITY_SCHEMA_VERSION,+ });+ expect(stamped).not.toBeNull();+ expect(stamped!.schema).toBe(CONTINUITY_SCHEMA_VERSION);+ expect(stamped!.legacy).toBe(false);++ const legacy = normalizeContinuityRecord(LEGACY_RAW);+ expect(legacy).not.toBeNull();+ expect(legacy!.schema).toBe(CONTINUITY_SCHEMA_VERSION);+ expect(legacy!.legacy).toBe(true);+ });++ test("session/turn correlation ids surface as first-class fields", () => {+ const normalized = normalizeContinuityRecord({+ ...LEGACY_RAW,+ kind: "session_turn",+ payload: { session_id: "s-1", turn_id: "t-3", role: "user", text: "hello" },+ });+ expect(normalized!.sessionId).toBe("s-1");+ expect(normalized!.turnId).toBe("t-3");+ });++ test("malformed rows and non-record values normalize to null, fail-soft", () => {+ expect(normalizeContinuityRecord(null)).toBeNull();+ expect(normalizeContinuityRecord("garbage")).toBeNull();+ expect(normalizeContinuityRecord({ kind: "recall_telemetry" })).toBeNull();+ // Unknown kinds stay readable - the evolution rule says additive.+ const unknown = normalizeContinuityRecord({ ...LEGACY_RAW, kind: "future_kind" });+ expect(unknown).not.toBeNull();+ expect(unknown!.kind).toBe("future_kind");+ });+});++describe("loadNormalizedContinuityRecords", () => {+ test("private records are dropped by default and kept only on request", () => {+ appendContinuityRecord(vault, {+ kind: "context_receipt",+ createdAt: "2026-06-03T10:00:00Z",+ payload: { text: "open detail" },+ });+ appendContinuityRecord(vault, {+ kind: "context_receipt",+ createdAt: "2026-06-03T10:01:00Z",+ payload: { text: "carries a <private>secret plan</private> region" },+ });++ const safe = loadNormalizedContinuityRecords(vault);+ expect(safe).toHaveLength(1);+ expect(JSON.stringify(safe)).not.toContain("secret plan");++ const all = loadNormalizedContinuityRecords(vault, { keepPrivate: true });+ expect(all).toHaveLength(2);+ // Even kept records stay redaction-masked: the store stripped the+ // region at write time and the read-model never un-masks.+ expect(JSON.stringify(all)).not.toContain("secret plan");+ });++ test("kind and session filters narrow the load", () => {+ appendContinuityRecord(vault, {+ kind: "session_turn",+ createdAt: "2026-06-03T10:00:00Z",+ payload: { session_id: "s-1", turn_id: "t-1", role: "user", text: "alpha" },+ });+ appendContinuityRecord(vault, {+ kind: "session_turn",+ createdAt: "2026-06-03T10:01:00Z",+ payload: { session_id: "s-2", turn_id: "t-1", role: "user", text: "beta" },+ });+ appendContinuityRecord(vault, {+ kind: "recall_telemetry",+ createdAt: "2026-06-03T10:02:00Z",+ payload: { status: "ok", session_id: "s-1" },+ });++ const turns = loadNormalizedContinuityRecords(vault, { kind: "session_turn" });+ expect(turns).toHaveLength(2);+ const sessionOne = loadNormalizedContinuityRecords(vault, { sessionId: "s-1" });+ expect(sessionOne).toHaveLength(2);+ expect(sessionOne.every((r) => r.sessionId === "s-1")).toBe(true);+ });+});diff --git a/tests/core/brain/continuity-schema-version.test.ts b/tests/core/brain/continuity-schema-version.test.tsnew file mode 100644index 00000000..ba015b1e--- /dev/null+++ b/tests/core/brain/continuity-schema-version.test.ts@@ -0,0 +1,103 @@+/**+ * Versioned continuity schema (Memory Observability Suite, t_26040ee8).+ *+ * Every new continuity record is stamped with one contract-wide schema+ * version; legacy records without the field read as v1; the dedup id+ * deliberately EXCLUDES the version so identical records keep identical+ * ids across the stamp transition (no migration of existing JSONL).+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { appendFileSync, mkdirSync, mkdtempSync, readFileSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { dirname } from "node:path";+import { join } from "node:path";++import {+ appendContinuityRecord,+ appendContinuitySourceInvalidation,+ continuityLogPath,+ listContinuityRecords,+} from "../../../src/core/brain/continuity/store.ts";+import { CONTINUITY_SCHEMA_VERSION } from "../../../src/core/brain/continuity/types.ts";++let vault: string;++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-continuity-schema-"));+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+});++describe("continuity schema version", () => {+ test("the contract-wide constant is o2b.continuity.v1", () => {+ expect(CONTINUITY_SCHEMA_VERSION).toBe("o2b.continuity.v1");+ });++ test("new records are stamped with the schema version, in memory and on disk", () => {+ const record = appendContinuityRecord(vault, {+ kind: "recall_telemetry",+ createdAt: "2026-06-03T10:00:00Z",+ sourceRefs: [{ id: "note-a" }],+ payload: { status: "ok" },+ });+ expect(record.schema).toBe(CONTINUITY_SCHEMA_VERSION);++ const raw = readFileSync(continuityLogPath(vault, "2026-06"), "utf8").trim();+ const parsed = JSON.parse(raw) as Record<string, unknown>;+ expect(parsed["schema"]).toBe(CONTINUITY_SCHEMA_VERSION);+ });++ test("source invalidation records carry the version too", () => {+ const record = appendContinuitySourceInvalidation(vault, {+ createdAt: "2026-06-03T10:00:00Z",+ source: { id: "note-b", path: "Brain/notes/b.md" },+ reason: "content drift",+ });+ expect(record.schema).toBe(CONTINUITY_SCHEMA_VERSION);+ });++ test("legacy records without the field are still read and report no version", () => {+ const path = continuityLogPath(vault, "2026-05");+ mkdirSync(dirname(path), { recursive: true });+ const legacy = {+ id: "ctn_20260531120000_aaaaaaaaaaaaaaaa",+ kind: "recall_telemetry",+ createdAt: "2026-05-31T12:00:00Z",+ sourceRefs: [],+ payload: { status: "ok" },+ private: false,+ redacted: false,+ };+ appendFileSync(path, `${JSON.stringify(legacy)}\n`, "utf8");++ const listed = listContinuityRecords(vault, { kind: "recall_telemetry" });+ expect(listed).toHaveLength(1);+ expect(listed[0]!.schema).toBeUndefined();+ });++ test("recordId excludes the schema field: stamped id matches the legacy formula", () => {+ // Two records with identical kind/createdAt/sourceRefs/payload must get+ // the same id regardless of the version stamp. The stamped record's id+ // is reproducible from the pre-stamp input shape - locked here so a+ // future change that folds `schema` into the hash fails loudly.+ const a = appendContinuityRecord(vault, {+ kind: "context_receipt",+ createdAt: "2026-06-03T11:00:00Z",+ sourceRefs: [{ id: "note-c" }],+ payload: { query: "alpha" },+ });+ const b = appendContinuityRecord(vault, {+ kind: "context_receipt",+ createdAt: "2026-06-03T11:00:00Z",+ sourceRefs: [{ id: "note-c" }],+ payload: { query: "alpha" },+ });+ expect(a.id).toBe(b.id);+ // Known-answer id computed with the pre-suite formula (sha-256 over+ // kind/createdAt/sourceRefs/payload only).+ expect(a.id).toBe("ctn_20260603110000_4eecda1810fdbf02");+ });+});diff --git a/tests/e2e/memory-observability.integration.test.ts b/tests/e2e/memory-observability.integration.test.tsnew file mode 100644index 00000000..cca86e30--- /dev/null+++ b/tests/e2e/memory-observability.integration.test.ts@@ -0,0 +1,125 @@+/**+ * Memory Observability Suite end-to-end flow (epic t_d75dc6a2).+ *+ * One scenario exercises the whole suite: gated telemetry emission+ * writes schema-stamped continuity records, the read-model serves them+ * to both trajectory exporters through the CLI, and a bench run over a+ * repo fixture reports quality / latency / context cost separately.+ */++import { afterEach, beforeEach, expect, test } from "bun:test";+import { existsSync, mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { bootstrapBrain } from "../../src/core/brain/init.ts";+import { packContext } from "../../src/core/brain/context-pack.ts";+import { CONTINUITY_SCHEMA_VERSION } from "../../src/core/brain/continuity/types.ts";+import { appendContinuityRecord } from "../../src/core/brain/continuity/store.ts";+import { runCli } from "../helpers/run-cli.ts";++let tmp: string;+let vault: string;+let configPath: string;++beforeEach(() => {+ tmp = mkdtempSync(join(tmpdir(), "o2b-mos-e2e-"));+ vault = join(tmp, "vault");+ configPath = join(tmp, "config.yaml");+ writeFileSync(configPath, `vault: ${vault}\nagent_name: e2e-agent\n`);+ bootstrapBrain(vault, { configPath });+});++afterEach(() => {+ rmSync(tmp, { recursive: true, force: true });+});++const env = () => ({ OPEN_SECOND_BRAIN_CONFIG: configPath });++test("telemetry -> schema-stamped records -> ATOF/ATIF export -> bench report", async () => {+ // 1. Gated telemetry: a context pack with receipt+telemetry writes+ // schema-stamped continuity records; without options it stays silent.+ writeFileSync(+ join(vault, "Brain", "preferences", "pref-e2e.md"),+ "---\nid: pref-e2e\ntopic: e2e\nstatus: confirmed\nprinciple: Keep e2e flows deterministic\ncreated_at: 2026-06-01T10:00:00\n---\n\nKeep e2e flows deterministic\n",+ );+ const report = packContext(vault, {+ maxTokens: 500,+ receipt: { host: "e2e", trigger: "context_pack", sessionId: "e2e-session" },+ telemetry: { host: "e2e", sessionId: "e2e-session" },+ });+ expect(report.receiptId).toBeDefined();+ expect(report.telemetryId).toBeDefined();++ // A session turn for the trajectory exporters.+ appendContinuityRecord(vault, {+ kind: "session_turn",+ createdAt: new Date().toISOString(),+ payload: { session_id: "e2e-session", turn_id: "t-1", role: "user", text: "ship the suite" },+ });++ // Every persisted record carries the contract-wide schema version.+ const continuityDir = join(vault, "Brain", "log", "continuity");+ const months = readdirSync(continuityDir).filter((name) => name.endsWith(".jsonl"));+ expect(months.length).toBeGreaterThan(0);+ for (const month of months) {+ for (const line of readFileSync(join(continuityDir, month), "utf8").split("\n")) {+ if (!line.trim()) continue;+ const record = JSON.parse(line) as { schema?: string };+ expect(record.schema).toBe(CONTINUITY_SCHEMA_VERSION);+ }+ }++ // 2. Export both trajectory formats through the CLI.+ const outDir = join(tmp, "export");+ const atof = await runCli(+ ["brain", "continuity", "export", "--format", "atof", "--out", outDir, "--json"],+ { env: env() },+ );+ expect(atof.returncode).toBe(0);+ const atofParsed = JSON.parse(atof.stdout) as { records: number; files: string[] };+ expect(atofParsed.records).toBeGreaterThanOrEqual(3);+ expect(readFileSync(atofParsed.files[0]!, "utf8")).toContain('"atof_version"');++ const atif = await runCli(+ [+ "brain",+ "continuity",+ "export",+ "--format",+ "atif",+ "--session",+ "e2e-session",+ "--out",+ outDir,+ ],+ { env: env() },+ );+ expect(atif.returncode).toBe(0);+ const trajectoryPath = join(outDir, "o2b-continuity-e2e-session.atif.json");+ expect(existsSync(trajectoryPath)).toBe(true);+ const trajectory = JSON.parse(readFileSync(trajectoryPath, "utf8")) as {+ schema_version: string;+ steps: Array<{ source: string; llm_call_count?: number }>;+ };+ expect(trajectory.schema_version).toBe("ATIF-v1.7");+ expect(trajectory.steps.some((s) => s.source === "system" && s.llm_call_count === 0)).toBe(true);++ // 3. Bench run over the repo fixture: separate metric families, green.+ const runsDir = join(tmp, "bench-runs");+ const bench = await runCli(+ ["brain", "bench", "memory", "--fixture", "core-recall", "--runs-dir", runsDir, "--json"],+ { env: env() },+ );+ expect(bench.returncode).toBe(0);+ const benchReport = JSON.parse(bench.stdout) as {+ quality: { passed: number; total: number };+ latency_ms: { avg: number };+ context_cost: { est_tokens: number };+ };+ expect(benchReport.quality.passed).toBe(benchReport.quality.total);+ expect(benchReport.latency_ms.avg).toBeGreaterThanOrEqual(0);+ expect(benchReport.context_cost.est_tokens).toBeGreaterThan(0);+ // The bench vault is disposable and separate from the operator vault.+ expect(existsSync(join(vault, "Brain", "notes", "coffee.md"))).toBe(false);+}, 120_000);diff --git a/tests/fixtures/bench/core-recall.json b/tests/fixtures/bench/core-recall.jsonnew file mode 100644index 00000000..cfa32459--- /dev/null+++ b/tests/fixtures/bench/core-recall.json@@ -0,0 +1,101 @@+{+ "name": "core-recall",+ "description": "Core memory quality fixture: one question per category - single-hop preference recall, temporal supersession, contradiction visibility, multi-record evidence, session handoff, and context budget truncation.",+ "notes": [+ {+ "path": "Brain/notes/coffee.md",+ "body": "# Coffee preference\n\nCoffee preference of the operator: flat white every morning. The coffee preference is stable.\n"+ },+ {+ "path": "Brain/notes/coffee-old.md",+ "body": "# Superseded note\n\nOld coffee preference: espresso. Replaced by the flat white note.\n"+ },+ {+ "path": "Brain/notes/timezone-a.md",+ "body": "# Timezone claim A\n\nThe team standup timezone is Berlin time according to the calendar invite.\n"+ },+ {+ "path": "Brain/notes/timezone-b.md",+ "body": "# Timezone claim B\n\nThe team standup timezone is Lisbon time according to the latest announcement.\n"+ },+ {+ "path": "Brain/notes/deploy-a.md",+ "body": "# Deploy step A\n\nDeployment requires the staging gate to pass first.\n"+ },+ {+ "path": "Brain/notes/deploy-b.md",+ "body": "# Deploy step B\n\nDeployment finishes with the production smoke checklist.\n"+ },+ {+ "path": "Brain/preferences/pref-tabs.md",+ "body": "---\nid: pref-tabs\ntopic: tabs\nstatus: confirmed\nprinciple: Use tabs for indentation in Go files\ncreated_at: 2026-05-01T10:00:00\n---\n\nUse tabs for indentation in Go files\n"+ }+ ],+ "continuity": [+ {+ "kind": "session_turn",+ "created_at": "2026-06-01T10:00:00.000Z",+ "payload": {+ "session_id": "bench-session",+ "turn_id": "t-1",+ "role": "user",+ "text": "handoff context: remember the flat white decision"+ }+ },+ {+ "kind": "session_turn",+ "created_at": "2026-06-01T10:01:00.000Z",+ "payload": {+ "session_id": "bench-session",+ "turn_id": "t-2",+ "role": "assistant",+ "text": "recorded the decision"+ }+ }+ ],+ "questions": [+ {+ "id": "q-single-hop",+ "category": "single_hop",+ "query": "flat white",+ "top_k": 5,+ "expected_paths": ["Brain/notes/coffee.md"]+ },+ {+ "id": "q-temporal",+ "category": "temporal",+ "query": "coffee preference",+ "top_k": 5,+ "expected_paths": ["Brain/notes/coffee.md"],+ "not_expected_above": ["Brain/notes/coffee-old.md"]+ },+ {+ "id": "q-contradiction",+ "category": "contradiction",+ "query": "standup timezone",+ "top_k": 5,+ "expected_paths": ["Brain/notes/timezone-a.md", "Brain/notes/timezone-b.md"]+ },+ {+ "id": "q-multi-evidence",+ "category": "multi_evidence",+ "query": "deployment",+ "top_k": 5,+ "expected_paths": ["Brain/notes/deploy-a.md", "Brain/notes/deploy-b.md"]+ },+ {+ "id": "q-session-handoff",+ "category": "session_handoff",+ "session_id": "bench-session",+ "expected_turns": 2,+ "expected_text": "flat white decision"+ },+ {+ "id": "q-budget",+ "category": "budget",+ "expected_ids": ["pref-tabs"],+ "max_tokens": 500,+ "max_total_chars": 2000+ }+ ]+}diff --git a/tests/fixtures/bench/stale-recall.json b/tests/fixtures/bench/stale-recall.jsonnew file mode 100644index 00000000..a61f65c6--- /dev/null+++ b/tests/fixtures/bench/stale-recall.json@@ -0,0 +1,24 @@+{+ "name": "stale-recall",+ "description": "Focused stale-fact regression fixture: a keyword-stuffed superseded note must never outrank the current note for the same topic. A ranking change that resurfaces stale facts fails this fixture.",+ "notes": [+ {+ "path": "Brain/notes/current.md",+ "body": "# Current\n\nThe coffee preference today is flat white, replacing every older note about it. Coffee preference confirmed.\n"+ },+ {+ "path": "Brain/notes/superseded.md",+ "body": "# Old\n\nCoffee preference: espresso. Espresso was the choice before the change.\n"+ }+ ],+ "questions": [+ {+ "id": "q-stale-guard",+ "category": "temporal",+ "query": "coffee preference",+ "top_k": 5,+ "expected_paths": ["Brain/notes/current.md"],+ "not_expected_above": ["Brain/notes/superseded.md"]+ }+ ]+}diff --git a/tests/helpers/run-cli.ts b/tests/helpers/run-cli.tsindex 7ea16a7b..f7b96e3b 100644--- a/tests/helpers/run-cli.ts+++ b/tests/helpers/run-cli.ts@@ -39,6 +39,7 @@ const RUNTIME_OVERRIDABLE_ENV = ["OPEN_SECOND_BRAIN_TRIGGER_COOLDOWN_DAYS","OPEN_SECOND_BRAIN_WIKI_LINK_FORMAT","OPEN_SECOND_BRAIN_RECALL_GATE_TELEMETRY",+ "OPEN_SECOND_BRAIN_BENCH_JUDGE_CMD",] as const;export async function runCli(diff --git a/tests/mcp/recall-gate-telemetry.test.ts b/tests/mcp/recall-gate-telemetry.test.tsindex 4ce91a57..96cf7238 100644--- a/tests/mcp/recall-gate-telemetry.test.ts+++ b/tests/mcp/recall-gate-telemetry.test.ts@@ -87,3 +87,15 @@ test("with recall_gate_telemetry on, decisions land as continuity records", asyn})) as { total: number; records: unknown[] };expect(listed.total).toBe(1);});++test("fail-open: a broken continuity store never breaks the gate decision", async () => {+ writeFileSync(configPath, `vault: "${vault}"\nrecall_gate_telemetry: "true"\n`);+ // The continuity directory path exists as a FILE, so every append throws.+ mkdirSync(join(vault, "Brain", "log"), { recursive: true });+ writeFileSync(join(vault, "Brain", "log", "continuity"), "not a directory");+ const decision = (await tool("brain_recall_gate").handler(ctx(), {+ prompt: "what did we decide about the index?",+ })) as { retrieve: boolean; reason: string };+ expect(typeof decision.retrieve).toBe("boolean");+ expect(typeof decision.reason).toBe("string");+});