Local-first 🧠memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly 😴 dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.
TypeScript
/**
* Census over the in-vault state population.
*
* `STATE_SURFACES` claims to be the complete list of durable locations
* this tool keeps inside a vault. A hand-written list cannot vouch for its
* own completeness, and the failure mode is silent: a new module joins a
* path under `<vault>/.open-second-brain/`, ships, and the inventory an
* operator reads before migrating their vault simply does not mention it.
*
* So the derived-store half of the population is DERIVED, from the source:
* every module under `src/` that names {@link DERIVED_STORE_DIR} - by the
* constant or by its value - has to be attributed to a row through
* {@link StateSurface.sources} or excused in
* {@link STATE_SURFACE_SWEEP_EXCLUSIONS} with a written reason. That is
* the same instrument `OUT_OF_VAULT_SWEEP_EXCLUSIONS` already applies to
* the outside-the-vault half, pointed at the other root.
*
* ## What the sweep can and cannot see, stated rather than implied
*
* It lexes through the shared lexer (`tests/helpers/source-lexer.ts`) and
* matches on the CODE view, so a directory named in a docblock is not a
* path builder and a `"{"` inside a value cannot move a block. Four
* private copies of that lexer produced a real defect in v1.49.0; there is
* no fifth here.
*
* - `src/` only. `hooks/lib/session-state.ts` builds a real surface
* under the same root and is out of the swept tree, so its row names
* it explicitly and `tests/core/state/surfaces.test.ts` asserts that
* the file it names still exists.
* - The `Brain/` half of the population carries no single anchor token -
* `BRAIN_LOG_REL`, `brainDirs(...).log` and a bare `"Brain"` are all
* the same root - so it is held true by the resolver-binding block in
* the unit test rather than by this sweep. Saying which half is swept
* is the difference between a census and a claim about a census. That
* delegation is only worth what the block covers: it bound 26 of 40
* rows when this sentence first appeared, and a deliberately wrong
* value in one of the other fourteen changed no test. It now binds
* every row bar a counted, reasoned exception list, and the count is
* itself an assertion there.
* - This sweep NEVER checks that a row's declared path is the path its
* owner builds. `attributed()` asks only whether a module is claimed
* by some row's `sources`, which is a question about attribution and
* not about correctness.
* - A path whose root arrives as an argument carries no token to match
* on. `writer-lock.ts` and `session-focus.ts` are exactly that shape:
* both build under the store directory and neither names it.
*
* ## The alarm on the parser
*
* A block matcher that goes wrong reports a clean tree, so the population
* is cross-checked against a second, dumber search that cannot mis-parse
* because it does not parse - a line regex for a `join(` whose arguments
* name the store directory. Every file it finds must be in the population.
*/
import { describe, expect, test } from "bun:test";
import { readdirSync, readFileSync } from "node:fs";
import { join, relative } from "node:path";
import { DERIVED_STORE_DIR } from "../../../src/core/brain/path-constants.ts";
import {
STATE_SURFACE_SWEEP_EXCLUSIONS,
STATE_SURFACES,
STATE_SURFACE_ID,
} from "../../../src/core/state/surfaces.ts";
import { lexSource } from "../../helpers/source-lexer.ts";
const REPO_ROOT = join(import.meta.dir, "..", "..", "..");
/** The one root the sweep walks. See the docblock for what living outside costs. */
const SWEPT_ROOT = "src";
/** The identifier the constant is imported under everywhere it is used. */
const STORE_DIR_IDENTIFIER = "DERIVED_STORE_DIR";
/**
* Modules in the swept tree today. An equality rather than a floor: a
* glob that stopped matching would sweep an empty set clean and every
* assertion below would pass over nothing.
*/
// 24: the bootstrap receipt writer joins the population with its row
// (write-side-trust Task 14), the same join install.lock.json made.
// 24 -> 25: the session mining ledger joins the population with its row
// (session-mining-economy), the same join the session import ledger made.
// 25 -> 26: the storage usage meter joins, excused (storage lifecycle
// external review, M3): it joins the directory to MEASURE the store and the
// state beside it as census surfaces, and writes nothing.
// 26 -> 27: the pending mining surveys join the population with their row
// (v1.86.1 review fixes), the same join the session mining ledger made.
// 27 -> 28, net, from the trust-and-responsiveness-hardening wave: the
// gitignore guard and its doctor tracked-state probe join the population,
// excused (one seeds the vault `.gitignore` that covers the directory, one
// probes git's view of it; neither creates state inside it), while the bench
// verb LEFT the population - its runs dir was hoisted to `BENCH_RUNS_REL` in
// `path-constants.ts`, so it no longer spells the directory - taking its now
// stale exclusion with it.
// 30 -> 32, net, from the agent-presence wave: the approvals ledger writer
// (`src/core/brain/lessons-promotion.ts`, t_1e952e43) spells the directory
// for `lessons-approvals.json` and is answered by that surface's row, and
// the instruction-block installer (`src/core/install/instruction-block.ts`,
// t_ba41a78a) spells it for `instruction-block.lock.json` and is answered
// by the `instruction_block_lock` row's `sources`.
// 32 -> 33: the lessons-block readiness check
// (`src/core/brain/doctor/lessons-block-check.ts`, t_1e952e43) joins with
// the same wave, answered by the `lesson_approvals` row's `sources`.
// 33 -> 32: the readiness check names the ledger through
// `lessonsApprovalsPath` (device-local once switched) instead of joining
// the directory itself.
// 32 -> 33: machine-state joins, excused: it copies a bare vault's own
// `.open-second-brain/` once into the device-local root (task 2.10 review).
const SWEPT_POPULATION_SIZE = 33;
/** Declared surfaces today. Pinned for the same reason. */
// 50: the source-archives row (answer-trust Task C1) joins the population,
// the same join the decisions ledger and the bootstrap receipt made.
// 50 -> 51: the session mining ledger joins the population with its row
// (session-mining-economy), the same join the session import ledger made.
// 51 -> 52: the pending mining surveys row (v1.86.1 review fixes).
// 52 -> 53: the cross-topic rule ledger row (t_642bc2b8 substrate), the
// same join the rollup ladder ledger made.
// 53 -> 55: the delivery-trust wave's attestation kernel joins the
// population with two rows - the signed attestation records (attestations)
// and the operator's Ed25519 signing keypair (attestation_keys), the same
// derived-store join the ingest checkpoints row made.
// 55 -> 56: the instruction-block lock row (t_ba41a78a Task B4) joins the
// catalogue - the ownership record uninstall reads to remove exactly the
// recorded pair, the same derived-store join the install manifest made.
// 56 -> 57: the skill-benchmarks row (t_635e3e72 Task A5) joins the
// catalogue - the preregistration a staged scored_rewrite proposal is
// audited against, the same vault-content join the skills tree made.
// 57 -> 58: the lesson_approvals row joins the catalogue with the lessons lane's final
// commit (t_1e952e43), the same join the session-mining ledger made.
// 58 -> 61: the embedding and rerank provider registries and the search
// activation history join the catalogue as device-local rows, so the
// migration moves them (task 2.10 review).
// 61 -> 64: the procedural usage counters, graph and hints join the
// catalogue as device-local rows, so a usage tick appends no store record
// and the migration moves the counters (task 3.10).
const DECLARED_SURFACE_COUNT = 64;
/** An exclusion reason has to be an argument, not a label. */
const MIN_REASON_LENGTH = 80;
interface SourceFile {
readonly path: string;
readonly text: string;
}
function readSourceTree(): ReadonlyArray<SourceFile> {
const files: SourceFile[] = [];
const walk = (dir: string): void => {
for (const entry of readdirSync(dir, { withFileTypes: true })) {
const abs = join(dir, entry.name);
if (entry.isDirectory()) walk(abs);
else if (entry.name.endsWith(".ts")) {
files.push({
path: relative(REPO_ROOT, abs).split("\\").join("/"),
text: readFileSync(abs, "utf8"),
});
}
}
};
walk(join(REPO_ROOT, SWEPT_ROOT));
return files.toSorted((a, b) => (a.path < b.path ? -1 : a.path > b.path ? 1 : 0));
}
/**
* Whether `file` names the derived-store directory in CODE.
*
* Two shapes, because both ship: the imported constant, and the bare
* literal a dozen modules spell privately. The literal is read off the
* comment-stripped view - which keeps string CONTENTS, so `".open-second-
* brain"` is still readable - and confirmed against the code view, where a
* quote delimiter survives but a comment does not. Reading it off the code
* view alone would find nothing, because that view blanks literal
* contents; reading it off the raw text would enrol every docblock that
* mentions the directory by name.
*/
function namesStoreDir(file: SourceFile): boolean {
const { withoutComments, code } = lexSource(file.text);
if (new RegExp(`\\b${STORE_DIR_IDENTIFIER}\\b`).test(code)) return true;
for (const quote of ['"', "'"]) {
const needle = `${quote}${DERIVED_STORE_DIR}${quote}`;
for (let at = withoutComments.indexOf(needle); at !== -1;) {
if (code[at] === quote) return true;
at = withoutComments.indexOf(needle, at + 1);
}
}
return false;
}
/** Every swept module that names the store directory. */
function sweepPopulation(tree: ReadonlyArray<SourceFile> = TREE): ReadonlyArray<string> {
return tree.filter(namesStoreDir).map((file) => file.path);
}
/**
* The naive cross-check: a `join(` whose arguments name the store
* directory, matched on raw text one line at a time. It cannot mis-parse
* because it does not parse, and it is strictly narrower than the rule
* above, so every file it finds must be in the population.
*/
const NAIVE_RE = new RegExp(
`join\\([^)\\n]*(?:"${DERIVED_STORE_DIR.replace(/\./g, "\\.")}"|${STORE_DIR_IDENTIFIER})`,
);
function naivePathBuilders(tree: ReadonlyArray<SourceFile> = TREE): ReadonlyArray<string> {
return tree.filter((file) => NAIVE_RE.test(file.text)).map((file) => file.path);
}
/**
* Whether any declared row claims `path` through its `sources`.
*
* A trailing-slash source names the files IN that directory and not the
* tree beneath it. Unbounded, it was an absolution: a row declaring
* `sources: ["src/core/"]` would have answered for most of this
* repository, and the "every declared source module exists" check in
* `tests/core/state/surfaces.test.ts` would have waved it through
* because `src/core` does exist. No row spells one today, which is
* exactly when the bound is cheap to put on.
*/
function attributed(
path: string,
surfaces: ReadonlyArray<{ readonly sources: ReadonlyArray<string> }> = STATE_SURFACES,
): boolean {
return surfaces.some((surface) =>
surface.sources.some((source) =>
source.endsWith("/")
? path.startsWith(source) && !path.slice(source.length).includes("/")
: path === source,
),
);
}
const TREE = readSourceTree();
describe("state surface census", () => {
test("every module that names the derived store is attributed to a row or excused", () => {
// ONE folded expectation whose value NAMES the work: asserting per
// module would stop at the first and hide the rest.
const unaccounted = sweepPopulation().filter(
(path) => !attributed(path) && !STATE_SURFACE_SWEEP_EXCLUSIONS.has(path),
);
expect(unaccounted.toSorted().join("\n")).toBe("");
});
test("no exclusion outlives the module it excuses, and every reason says something", () => {
const swept = new Set(sweepPopulation());
const stale: string[] = [];
const thin: string[] = [];
for (const [path, reason] of STATE_SURFACE_SWEEP_EXCLUSIONS) {
if (!swept.has(path)) stale.push(path);
// A placeholder ("todo", "n/a", "") cannot satisfy this.
if (reason.trim().length < MIN_REASON_LENGTH) thin.push(path);
if (/\bTODO\b|\bfor now\b|\blater\b/i.test(reason)) thin.push(path);
}
expect(stale.toSorted().join("\n")).toBe("");
expect(thin.toSorted().join("\n")).toBe("");
});
test("no module is both attributed and excused", () => {
// Two answers to one question is how an exclusion survives the row
// that replaced it, still excusing a module nobody reads it for.
const both = sweepPopulation().filter(
(path) => attributed(path) && STATE_SURFACE_SWEEP_EXCLUSIONS.has(path),
);
expect(both.toSorted().join("\n")).toBe("");
});
test("the naive cross-check finds nothing the population missed", () => {
// The alarm on the parser itself. A lexer that blanked the wrong span
// would report a clean sweep over a shrunken population; this search
// cannot, because it never looks at structure.
const population = new Set(sweepPopulation());
const unseen = naivePathBuilders().filter((path) => !population.has(path));
expect(unseen.toSorted().join("\n")).toBe("");
// …and it has to have found something, or it is no alarm at all.
expect(naivePathBuilders().length).toBeGreaterThan(5);
});
test("the lesson approvals row attributes the ledger writer (t_1e952e43)", () => {
const row = STATE_SURFACES.find((r) => r.id === STATE_SURFACE_ID.lessonApprovals);
expect(row).toBeDefined();
expect(row?.sources).toContain("src/core/brain/lessons-promotion.ts");
// ...and the writer sits inside the swept population it answers.
expect(sweepPopulation()).toContain("src/core/brain/lessons-promotion.ts");
});
test("the census is not vacuous", () => {
expect(TREE.length).toBeGreaterThan(850);
expect(sweepPopulation().length).toBe(SWEPT_POPULATION_SIZE);
expect(STATE_SURFACES.length).toBe(DECLARED_SURFACE_COUNT);
});
});
describe("the census can fail", () => {
/** The intruder joins the real tree, so the rule runs under real conditions. */
const intruder = (path: string, text: string): ReadonlyArray<SourceFile> => [
...TREE,
{ path, text },
];
/** Only the intruder's own row, so an unrelated module cannot answer for it. */
const unaccountedFrom = (path: string, text: string): ReadonlyArray<string> =>
sweepPopulation(intruder(path, text)).filter(
(p) => p === path && !attributed(p) && !STATE_SURFACE_SWEEP_EXCLUSIONS.has(p),
);
test("a new path builder under the store directory is unaccounted for", () => {
const path = "src/core/synthetic-store-writer.ts";
const text =
'import { join } from "node:path";\n' +
"export function spoolPath(vault: string): string {\n" +
` return join(vault, "${DERIVED_STORE_DIR}", "synthetic-spool.jsonl");\n` +
"}\n";
expect(unaccountedFrom(path, text)).toEqual([path]);
});
test("a builder reaching the directory through the constant is caught too", () => {
const path = "src/core/synthetic-constant-writer.ts";
const text =
'import { DERIVED_STORE_DIR } from "./brain/path-constants.ts";\n' +
'export const dir = (vault: string): string => join(vault, DERIVED_STORE_DIR, "x");\n';
expect(unaccountedFrom(path, text)).toEqual([path]);
expect(naivePathBuilders(intruder(path, text))).toContain(path);
});
test("a directory source answers for its own files and not for the tree below it", () => {
// Driven against a synthetic row rather than a real one, because the
// point is what the RULE would allow: an unbounded prefix let one
// source line absolve every module under it, and the population is
// only as honest as the narrowest claim it accepts.
const sweeping = [{ sources: ["src/core/"] }];
expect(attributed("src/core/doctor.ts", sweeping)).toBe(true);
expect(attributed("src/core/state/migrate.ts", sweeping)).toBe(false);
expect(attributed("src/core/brain/anticipatory-cache.ts", sweeping)).toBe(false);
});
test("a docblock that merely names the directory is not a path builder", () => {
// The false positive the lexing exists to prevent. Seven modules in
// the real tree mention the directory only in prose.
const path = "src/core/synthetic-prose.ts";
const text = `/** Nothing here writes to ${DERIVED_STORE_DIR}. */\nexport const x = 1;\n`;
expect(sweepPopulation(intruder(path, text))).not.toContain(path);
});
});