Local-first 🧠memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly 😴 dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.
Storage lifecycle - implementation plan
Six features across five parallel lanes. Each lane owns a disjoint file set; the only shared files are the pinned contracts listed below, and every contract names its owner and landing order. Tasks are TDD-ready: the acceptance test is written first and must fail with the expected error before implementation.
Pinned cross-lane contracts
| # | File(s) | Exported symbols | Owner | Landing |
|---|---|---|---|---|
| C1 | src/core/brain/storage/usage.ts, forecast.ts, gate.ts, capacity-error.ts |
measureStorageUsage, sampleStorageUsage, forecastStorageUsage, STORAGE_CAPACITY_TIER, StorageForecast, StorageUsage, evaluateCapacityGate, CapacityGateDecision, StorageCapacityError, STORAGE_ERROR_CODES |
Lane 1 | Lands first; Lane 2 and Lane 3 consume read-only |
| C2 | src/core/brain/types.ts, src/core/brain/policy/blocks/storage.ts, policy/load.ts, policy/defaults.ts, policy/validate.ts, src/core/brain/config-template.ts |
BrainStorageConfig, ResolvedBrainStorageConfig, BRAIN_STORAGE_DEFAULTS, resolveStorage |
Lane 1 | Lands first; no other lane edits any config file |
| C3 | src/mcp/tool-error-codes.ts |
STORAGE_ERROR_CODES registered via exhaustiveMembers |
Lane 1 | Lands first |
| C4 | src/core/brain/doctor.ts, src/core/brain/diagnostics.ts |
DOCTOR_CHECKS row for storageCapacityCheck then receiptLedgerCheck; DIAGNOSTIC_SIGNALS codes |
Lane 1 first, Lane 3 appends | Lane 3 depends on Lane 1 for these two files |
| C5 | src/mcp/brain-tools.ts |
STORAGE_TOOLS aggregation then RECEIPT_TOOLS |
Lane 1 first, Lane 3 appends | Lane 3 depends on Lane 1 for this file |
| C6 | src/core/brain/source-cleanup.ts |
widened export removeFile (plus Lane 5's own loop decomposition) |
Lane 5 | Lands before Lane 4 consumes removeFile; Lane 4 imports it, never edits the file |
| C7 | src/core/brain/ingest/*, src/cli/brain/verbs/extract.ts |
watermark internals | Lane 5 | Lane 2 must not touch ingest.ts; the admission refusal lives in src/mcp/brain/ingest-tools.ts (Lane 2's file), so the two never meet |
| C8 | src/core/brain/hygiene/types.ts, scan.ts, apply.ts, src/core/brain/applier-capability.ts |
document-dedup detector id, retire action, applier routing |
Lane 4 | Exclusive to Lane 4 |
| C9 | src/core/brain/maintenance/lane.ts, journal.ts, src/cli/brain/verbs/maintenance.ts, src/mcp/brain/admin-tools.ts |
capacity gate wiring, MAINTENANCE_VERDICT.skippedCapacity |
Lane 2 | Exclusive to Lane 2 |
| C10 | src/core/brain/continuity/types.ts |
additive receipt_compaction record kind |
Lane 3 | Exclusive to Lane 3 |
No lane edits src/core/brain/continuity/store.ts (Lane 3 keeps the append path generic; Lane 4 imports the existing exported appendContinuitySourceInvalidation). No new STATE_SURFACES rows are expected: samples ride the metrics sink, the audit rides continuity, and ledger rewrites touch existing rows; the state-surface census run in CI verifies this per lane.
Lane 1 - storage substrate (lands first)
Pure modules first, then registration, then surfaces.
Task 1.1: usage meter
- Files: new
src/core/brain/storage/usage.ts; newtests/core/brain/storage-usage.test.ts - Acceptance:
measureStorageUsage(vault, opts)on a sandbox vault returns per-tier byte totals (vault-content, derived) that match a hand-computed fixture (page files,brain.sqliteplus WAL/SHM sidecars, artifacts, snapshots, ledgers); a surface made unreadable (chmod fixture gated onCHMOD_CANNOT_DENY) appears inunmeasuredwith its named reason and never as zero bytes; the walker's per-dot-dir policy (artifacts, archives, snapshots all counted) is asserted; deterministic via injectednowonly. - Depends on: none.
Task 1.2: forecast and tiers
- Files: new
src/core/brain/storage/forecast.ts; newtests/core/brain/storage-forecast.test.ts - Acceptance:
forecastStorageUsageover seededstorage_usagemetric samples returns the linear slope,days_to_capand tier; tier boundaries hit exactly at the default constants (warn 80 percent / 30 days, critical 95 percent / 7 days, module-level named constants); absentcap_bytesyieldsunbounded; an unreadable store yieldsunmeasurable; fewer than two samples yields the named no-forecast state;sampleStorageUsageappends onestorage_usagemetrics record per call, fail-soft, and is asserted idempotent per run id. - Depends on: Task 1.1.
Task 1.3: capacity error leaf and code registration
- Files: new
src/core/brain/storage/capacity-error.ts; modifiedsrc/mcp/tool-error-codes.ts(C3); newtests/mcp/storage-capacity-errors.test.ts - Acceptance:
StorageCapacityErrorcarries the closedSTORAGE_ERROR_CODESunion (capacity_unmeasurable,capacity_exhausted); registration compiles throughexhaustiveMembers; the MCP envelope test asserts the codes map by name and never leak vault paths at remote reach. - Depends on: none (parallel with 1.1/1.2 within the lane).
Task 1.4: storage policy block
- Files: C2 set - new
src/core/brain/policy/blocks/storage.ts, modifiedsrc/core/brain/types.ts,policy/load.ts,policy/defaults.ts,policy/validate.ts,src/core/brain/config-template.ts; newtests/core/brain/storage-policy.test.ts - Acceptance:
resolveStorageparsescap_bytes,warn_percent,critical_percent,warn_days,critical_dayswith bounded field checks that hard-error out-of-range values and acritical_percentthat must exceedwarn_percent; absent block resolves to the documented defaults withcap_bytesabsent (unbounded); the template ratchet appends the new block to an existing_brain.yamlwithout rewriting existing values (mergeBrainYaml fixture). - Depends on: none within the lane.
Task 1.5: capacity gate decision
- Files: new
src/core/brain/storage/gate.ts; extendtests/core/brain/storage-forecast.test.tsor a siblingtests/core/brain/storage-gate.test.ts - Acceptance:
evaluateCapacityGatereturnsrun/skipwith the tier and reading attached; unbounded andunmeasurablestores never skip; the decision is computed from a freshmeasureStorageUsagecall at decision time, never from the stored aggregate (asserted by mutating the aggregate between sample and decision). - Depends on: Tasks 1.2 and 1.4.
Task 1.6: storage-capacity doctor check
- Files: new
src/core/brain/doctor/storage-capacity.ts; modifiedsrc/core/brain/doctor.ts,src/core/brain/diagnostics.ts(C4, first landing); newtests/core/brain/doctor-storage-capacity.test.ts - Acceptance:
runDoctoron a hand-built skeleton vault (doctor-drift pattern) emits a warning issue at the warn tier, an error issue at critical, and anuncertainentry for an unmeasurable store, readingctx.dbPathfail-soft; both issue codes are registered inDIAGNOSTIC_SIGNALSandtests/core/brain/doctor-exit-census.test.tspasses unchanged in mechanism; reach filtering follows the doctor-counts-reach pattern. - Depends on: Tasks 1.2 and 1.3.
Task 1.7: report surfaces (CLI verb and MCP tool)
- Files: new
src/cli/brain/verbs/storage.ts,src/mcp/brain/storage-tools.ts; modifiedsrc/mcp/brain-tools.ts(C5, first landing); newtests/cli/brain/verbs/storage.test.ts,tests/mcp/brain-storage.test.ts,tests/mcp/storage-reach.test.ts - Acceptance:
o2b brain storageandbrain_storagereturn the measurement, forecast, tier and per-surfaceunmeasuredreasons; below local reach the report answersmeasured: falsewith the named reason (link-integrity model); each invocation appends one usage sample; the brain-tools parity test derives both surfaces without a CLI_ONLY/MCP_ONLY entry. - Depends on: Tasks 1.5 and 1.6.
Lane 2 - background write backpressure
Task 2.1: maintenance lane capacity gate
- Files: modified
src/core/brain/maintenance/lane.ts(C9),src/core/brain/maintenance/journal.ts(C9); newtests/core/brain/maintenance-capacity-gate.test.ts - Acceptance:
evaluateGatesgains the capacity check ordered after host pressure, active only when the vault's storage block configurescap_bytes; a critical tier returns the newMAINTENANCE_VERDICT.skippedCapacitywith the reading carried alongside (the pressure-reading model);skipped:capacityjournal rows parse and render;--forcebypasses it, the lease is never bypassed; the usage sample is taken before the skip returns (asserted: a skipped run still appends a sample); unmeasurable leaves the gate open with the reason journalled. - Depends on: Lane 1 Tasks 1.2, 1.4, 1.5 (C1, C2).
Task 2.2: lane surfaces carry the verdict
- Files: modified
src/cli/brain/verbs/maintenance.ts(C9),src/mcp/brain/admin-tools.ts(C9); extendtests/core/brain/maintenance-capacity-gate.test.ts; updatetests/mcp/maintenance-parity-census.test.tsexpectations - Acceptance: CLI flags and MCP properties derive the new verdict and reading identically from source; the parity census passes in both directions with no new exclusion; a skipped-capacity run reports the tier and the remedy path in both surfaces.
- Depends on: Task 2.1.
Task 2.3: ingest admission refusal
- Files: modified
src/mcp/brain/ingest-tools.ts(C7 boundary - this file only); newtests/mcp/ingest-capacity-refusal.test.ts - Acceptance: at the critical tier
brain_ingest_sourcerefuses before any write with thecapacity_exhaustedenvelope naming the remedy (compaction, retirement, cap raise); zero bytes written to the vault (asserted withbrainPageTexts); an unbounded or sub-critical vault ingests unchanged; the check is skipped entirely when no cap is configured (no measurement cost). - Depends on: Lane 1 Tasks 1.3, 1.5 (C1, C3).
Lane 3 - receipt compaction
Task 3.1: ledger scan
- Files: new
src/core/brain/receipts/types.ts,src/core/brain/receipts/scan.ts; newtests/core/brain/receipts-scan.test.ts - Acceptance:
scanReceiptLedgersinventories continuity and metrics shards with record counts, byte sizes, foldable counts above the age threshold and protected-record counts; the protected vocabulary (source_invalidation, pre/post-compact kinds, clip-protected-key carriers, unknown kinds) is asserted never counted foldable; the scan is read-only (byte-identical vault after run). - Depends on: none.
Task 3.2: fold engine
- Files: new
src/core/brain/receipts/compact.ts; newtests/core/brain/receipts-compact.test.ts - Acceptance:
compactReceiptLedgerswith explicit shard ids folds records older thanolder_than_daysinto one aggregate line per (kind, month) preserving count, first/last timestamps and byte totals; protected records survive verbatim in order; unknown kinds fail closed to keep; shrink is verified (output strictly smaller, protected content byte-present); the rewrite is atomic per shard insidewithDestructiveSnapshotand per-shard fail-soft (a poisoned shard is collected, the run completes); re-run is idempotent (nothing left foldable, no write); the continuity record kindreceipt_compaction(C10, additive tosrc/core/brain/continuity/types.ts) is appended exactly once per confirmed run and never on a dry run;expect/count-guard mismatch aborts before any write. - Depends on: Task 3.1.
Task 3.3: receipt-ledger-growth doctor check
- Files: new
src/core/brain/doctor/receipt-ledger.ts; modifiedsrc/core/brain/doctor.ts,src/core/brain/diagnostics.ts(C4, appended after Lane 1); newtests/core/brain/doctor-receipt-ledger.test.ts - Acceptance: a ledger above the growth threshold yields a warning issue naming the receipts command; a compacted ledger yields no issue; codes registered in
DIAGNOSTIC_SIGNALS; doctor-exit census passes. - Depends on: Task 3.1; Lane 1 Task 1.6 (C4 order).
Task 3.4: receipt surfaces (CLI verb and MCP tool)
- Files: new
src/cli/brain/verbs/receipts.ts,src/mcp/brain/receipt-tools.ts; modifiedsrc/mcp/brain-tools.ts(C5, appended after Lane 1); newtests/cli/brain/verbs/receipts.test.ts,tests/mcp/brain-receipts.test.ts - Acceptance:
o2b brain receipts scanis the default and writes nothing;compactrequires explicit shard ids from a scan and enforces the count guard before mutating; the deny refusal (refuseDocumentDeny) fires on the mutation lane; the tool and verb derive from one source (parity test); output names the operation receipt compaction, disjoint from the context-compaction vocabulary. - Depends on: Task 3.2, Task 3.3; Lane 1 Task 1.7 (C5 order).
Lane 4 - near-duplicate document retirement
Task 4.1: document-dedup detector
- Files: new
src/core/brain/hygiene/detectors/document-dedup.ts; newtests/core/brain/document-dedup-detector.test.ts - Acceptance: the detector groups ingest summary pages (kind
brain-source) by content similarity at the lexical threshold (0.8) and, when a provider is configured, the embedding threshold (0.97), excluding merge-resolved pairs and keeping scope keys folded; canonical is the newestcreated_at; a group containing any member withoutcreated_atis emitted ineligible with the reason in evidence (mtime instability); finding ids are deterministic viahygieneFindingId. - Depends on: none.
Task 4.2: retire action vocabulary and routing
- Files: modified
src/core/brain/hygiene/types.ts,src/core/brain/hygiene/apply.ts,src/core/brain/applier-capability.ts(C8); newtests/core/brain/hygiene-retire-routing.test.ts - Acceptance:
retirejoins the closed action vocabulary and the applier-capability table;buildHygienePlanroutes retire findings through the review path (excluded from unattended apply);executeFindingdispatches retire to the retirement core with the count guard semantics hygiene apply already enforces. - Depends on: Task 4.1.
Task 4.3: retirement applier core
- Files: new
src/core/brain/document-retire.ts; newtests/core/brain/document-retire.test.ts - Acceptance: per stale copy, the applier retargets inbound wikilinks to the canonical (retargetWikilinks report asserted), removes the page, drops the content-manifest entry round-tripping the read contract, and appends one
appendContinuitySourceInvalidationrecord naming canonical id, retired id, dropped manifest key and the retained source-archive hash; a re-ingest of the retired copy's source does not resurrect a page (manifest-drop assertion); per-copy failure is collected ({path, message}), the failed copy stays on disk, and the run completes; the whole span sits insidewithDestructiveSnapshotwith a recoverability verdict; the source archive file is asserted still present. - Depends on: Task 4.2; Lane 5 Task 5.1 (C6 - consumes the exported
removeFile).
Lane 5 - residues: decomposition and incremental extraction
Task 5.1: deleteBySource per-file decomposition
- Files: modified
src/core/brain/source-cleanup.ts(C6, lands first - widens theremoveFileexport for Lane 4); extendtests/core/brain/source-cleanup.test.ts - Acceptance: a per-entry removal failure is collected as
{path, message}, the remaining entries are still removed, the partial-progress audit and the final audit carry honest removed/failed counts, and the result shape gains thefailedarray; a run with zero failures is byte-identical in behavior to today; regression test asserts the pre-fix code fails this test (one injected failing file aborts the run today). - Depends on: none.
Task 5.2: page-dedup and collect loop decomposition
- Files: modified
src/cli/brain/verbs/page-dedup.ts,src/core/brain/maintenance/collect.ts; newtests/cli/brain/verbs/page-dedup-apply-fail-soft.test.ts, extend the maintenance collect tests - Acceptance: a MergeChainError on one cluster is collected (
{key, message}), later clusters still apply, the report lists failures, and the CLI exits 1 when failures are non-empty; collect loops collect per-item errors in the module's existing report shape; write-batch all-or-nothing behavior is asserted unchanged. - Depends on: none (parallel with 5.1 inside the lane).
Task 5.3: tail read primitive
- Files: new
src/core/brain/ingest/tail-read.ts; newtests/core/brain/ingest/tail-read.test.ts - Acceptance: reading from a byte offset returns only appended text snapped forward to the first UTF-8-safe newline boundary; a file that shrank below the offset falls back to the full content with the shrink named; the bounded-read ceiling is enforced with the existing refusal vocabulary; multi-byte characters split across the boundary decode cleanly.
- Depends on: none.
Task 5.4: manifest watermark
- Files: modified
src/core/brain/ingest/content-manifest.ts; extendtests/core/brain/ingest/content-manifest.test.ts - Acceptance:
parseEntryValueaccepts entries carrying additiveextracted_bytes/extracted_atand preserves them throughupdateManifestround-trips; an entry without them degrades to byte 0; schema_version stays 3 (additive fields never bump); unknown field shapes still drop toward reprocessing exactly as today. - Depends on: none.
Task 5.5: incremental extraction plumbing
- Files: modified
src/core/brain/ingest/ingest.ts,src/cli/brain/verbs/extract.ts(C7); newtests/core/brain/ingest/incremental-extract.test.ts - Acceptance: an ingest of a source whose manifest record carries
extracted_bytesreports the appended delta{from_byte, to_byte}onIngestSourceResultand stamps the new watermark only when the run covered those bytes; the CLI extract preview reports the appended slice metadata for a previously-extracted source; appending bytes to a fixture source twice yields two correct deltas with no overlap; a rewritten (not appended) source classifies modified and reprocesses fully, resetting the watermark;EXTRACT_CONTRACT_VERSIONis asserted unchanged at 2 and no mass reprocess is triggered. - Depends on: Tasks 5.3 and 5.4.
Landing order and integration
- Lane 1 lands whole (substrate first: Tasks 1.1-1.4 may interleave, 1.5-1.7 in order).
- Lane 2 and Lane 3 start once Lane 1's dependency tasks land; Lane 5 starts immediately (independent); Lane 4 starts once Task 5.1 lands.
- Integration order at the end of the wave: Lane 3's C4/C5 appends rebase onto Lane 1's landed registry rows; the full suite, both censuses (
doctor-exit-census,maintenance-parity-census,brain-tools-parity, write-site and destructive-site) and the formatter run before each lane's commits per the playbook's commit discipline.