Local-first 🧠memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly 😴 dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.
TypeScript
/**
* The shared per-session mining record (session-mining-economy,
* t_93617709).
*
* `planExtractSignals` is a pure read with no cross-run memory, so a
* session that was mined once is rebuilt and re-offered to the calling
* agent's model on every re-scan - repeated model spend, whether the mine
* would yield anything or not. This ledger is the memory: every commit that
* answers a plan records its outcome breakdown for the session, and the
* plan gate consults the record before any decision-model work. A zero-
* yield record (nothing written, nothing staged) is a tombstone that
* suppresses the session by name; a yield record is the distilled state the
* mining census reports, and suppresses nothing.
*
* ## The entry is keyed on content the mining lane actually holds
*
* The session import ledger (`sessions/discover.ts`) hashes the BYTES of a
* source log and keys the hash on the log's absolute machine path. The
* mining lane holds neither: `session_turn` continuity records carry no
* source path and no hash, and the lane's session id is the portable
* `sessionRefIdentity` basename, not a path. So the tombstone hashes the
* mined turn material itself - exactly the (turn id, text, timestamp)
* triples the plan embeds - under the portable id. The invalidation
* semantics fall out for free: a re-import that changes a session's turns
* changes the hash and re-opens the session with no clearing step, and a
* session whose turns vanish leaves nothing for the gate to honor.
*
* ## The house ledger discipline, mirrored
*
* The shape is the import ledger's: schema-versioned, read fail-closed on
* corrupt JSON or an unknown schema version (a silent reset would unsuppress
* every barren session at once and send them all back to the model), written
* under the same file lock with the vault-identity assertion, serialized
* with sorted keys so a no-op rerun is byte-identical, and pruned of dead
* entries at write time only, because a read that rewrote the state it read
* would not be a read. The pruning here drops sessions with no imported
* turns rather than files that no longer exist - the mining lane's
* equivalent of a vanished path.
*
* The ledger is deterministic local state: no model, no network, no config
* knob. Like the import ledger it is always on, because the spend it
* prevents is the point of the feature.
*/
import { existsSync, readFileSync } from "node:fs";
import { atomicWriteFileSync } from "../fs-atomic.ts";
import { hashBytes } from "./ingest/content-manifest.ts";
import { DERIVED_STORE_DIR } from "./path-constants.ts";
import { buildSessionTurnIndex } from "./session-recall.ts";
import { acquireLockSyncWithRetry, ingestLockRemedy } from "./sync-lockfile.ts";
import { isoSecond } from "./time.ts";
import { assertVaultIdentityForWrite } from "./vault-identity.ts";
import { machineStatePath } from "../state/machine-state.ts";
/** Only ledger schema this build understands. Anything else is refused. */
export const SESSION_MINING_LEDGER_SCHEMA_VERSION = 1 as const;
/** Basename of the mining ledger inside the derived store. */
const SESSION_MINING_LEDGER_FILE = "session-mining-ledger.json";
/** Absolute path of the session mining ledger for a vault. */
export function sessionMiningLedgerPath(vault: string): string {
return machineStatePath(vault, DERIVED_STORE_DIR, SESSION_MINING_LEDGER_FILE);
}
/** How the mined session ended, in the commit result's own vocabulary. */
export interface SessionMiningOutcome {
/** Signals written straight to the inbox. */
readonly written: number;
/** Of those, the ones staged into `Brain/pending/` for approval. */
readonly staged: number;
/** Items refused because their dedup hash was already in the vault. */
readonly deduped: number;
/** Items the durability gate refused. */
readonly durabilityRejected: number;
}
/**
* The turn material the plan embeds, as far as the hash is concerned:
* the turn's id, its trimmed text and its stored timestamp verbatim.
* Structurally the plan's `MinedTurn`, kept separate so this module does
* not import the mining lane that records into it.
*/
export interface MinedTurnMaterial {
readonly turnId: string;
readonly text: string;
readonly timestamp: string;
}
/** What was recorded about one mined session. */
export interface SessionMiningEntry {
/**
* The portable session id. Carried inside the entry as well as under its
* key so a record is self-describing: a census or an operator reading the
* file needs no key context to attribute it, and an entry misfiled under
* another session's key is unreadable rather than misattributed.
*/
readonly sessionId: string;
/** {@link minedTurnsHash} of the turn material the mine covered. */
readonly minedContentHash: string;
/**
* The outcome breakdown the last commit over this material recorded.
* Zero written and zero staged makes the entry a tombstone (the plan
* gate's evidence); a yield makes it the distilled state the census
* reports - the same field, read through two contracts.
*/
readonly outcome: SessionMiningOutcome;
/** When the commit recorded it, to the second. */
readonly recordedAt: string;
}
/** Portable session id → what was recorded about mining it. */
export interface SessionMiningLedger {
readonly schema_version: typeof SESSION_MINING_LEDGER_SCHEMA_VERSION;
readonly entries: Readonly<Record<string, SessionMiningEntry>>;
}
/**
* The remedy every file-level refusal names: the refusal stops every plan,
* the census and the nudge, so the operator must learn what removing the
* file costs (every session re-opens for mining) and that nothing else is
* lost.
*/
const CORRUPT_LEDGER_REMEDY =
"; to recover, move the file aside or delete it - every imported session then becomes " +
"eligible for mining again, and the next commits rebuild the ledger";
/**
* Read the ledger. A missing file is an empty ledger - no session is
* tombstoned, which is the correct answer for a vault that has mined
* nothing. A CORRUPT or unknown-schema file is a hard error: a silent reset
* would unsuppress every barren session at once and re-offer all of them to
* the model on the next scan. The entries container is held to the same
* contract: a container that is absent, null, an array or a primitive is
* file-level corruption and refuses by name, while damage to a SINGLE entry
* drops that entry only.
*/
export function readSessionMiningLedger(vault: string): SessionMiningLedger {
const path = sessionMiningLedgerPath(vault);
if (!existsSync(path)) {
return { schema_version: SESSION_MINING_LEDGER_SCHEMA_VERSION, entries: {} };
}
let parsed: unknown;
try {
parsed = JSON.parse(readFileSync(path, "utf8"));
} catch (e) {
throw new Error(`session mining ledger is corrupted JSON: ${path}${CORRUPT_LEDGER_REMEDY}`, {
cause: e,
});
}
if (parsed === null || typeof parsed !== "object" || Array.isArray(parsed)) {
throw new Error(`session mining ledger is not an object: ${path}${CORRUPT_LEDGER_REMEDY}`);
}
const obj = parsed as Record<string, unknown>;
if (obj["schema_version"] !== SESSION_MINING_LEDGER_SCHEMA_VERSION) {
throw new Error(
`session mining ledger schema_version ${String(obj["schema_version"])} not supported ` +
`(expected ${SESSION_MINING_LEDGER_SCHEMA_VERSION}): ${path}${CORRUPT_LEDGER_REMEDY}`,
);
}
const entries: Record<string, SessionMiningEntry> = {};
const raw = obj["entries"];
// Container shape is file-level corruption, the same class as a corrupt
// body or an unknown schema: a string, array, null or absent container
// read as an empty ledger would silently unsuppress every barren session
// at once, which is the outcome the hard errors exist to prevent. This is
// different from per-ENTRY damage, which is dropped rather than guessed
// at, because a dropped entry can only re-open one session for mining.
if (raw === null || typeof raw !== "object" || Array.isArray(raw)) {
throw new Error(
`session mining ledger entries is not an object: ${path}${CORRUPT_LEDGER_REMEDY}`,
);
}
for (const [key, value] of Object.entries(raw as Record<string, unknown>)) {
const entry = readEntry(key, value);
if (entry !== null) entries[key] = entry;
}
return { schema_version: SESSION_MINING_LEDGER_SCHEMA_VERSION, entries };
}
/**
* One entry, or `null` when it is not one this build can read. An
* unreadable entry is dropped rather than guessed at: treated as absent it
* can only re-open a session for mining (the safe direction), while
* suppressing a session on a half-readable record would hide material
* behind corrupted state. An entry filed under a DIFFERENT session's key is
* unreadable too - attributing it to the key it sits under would suppress
* one session on another session's evidence.
*/
function readEntry(key: string, value: unknown): SessionMiningEntry | null {
if (value === null || typeof value !== "object" || Array.isArray(value)) return null;
const row = value as Record<string, unknown>;
const sessionId = row["sessionId"];
const hash = row["minedContentHash"];
const recordedAt = row["recordedAt"];
if (typeof sessionId !== "string" || sessionId.length === 0 || sessionId !== key) return null;
if (typeof hash !== "string" || hash.length === 0) return null;
const outcome = readOutcome(row["outcome"]);
if (outcome === null) return null;
return {
sessionId,
minedContentHash: hash,
outcome,
recordedAt: typeof recordedAt === "string" ? recordedAt : "",
};
}
function readOutcome(value: unknown): SessionMiningOutcome | null {
if (value === null || typeof value !== "object" || Array.isArray(value)) return null;
const row = value as Record<string, unknown>;
const written = row["written"];
const staged = row["staged"];
const deduped = row["deduped"];
const durabilityRejected = row["durabilityRejected"];
if (
typeof written !== "number" ||
typeof staged !== "number" ||
typeof deduped !== "number" ||
typeof durabilityRejected !== "number"
) {
return null;
}
return { written, staged, deduped, durabilityRejected };
}
/** Canonical ledger bytes: keys sorted, so a no-op rerun is byte-identical. */
function serializeLedger(entries: Record<string, SessionMiningEntry>): string {
const sorted: Record<string, SessionMiningEntry> = {};
for (const key of Object.keys(entries).toSorted()) sorted[key] = entries[key]!;
return (
JSON.stringify(
{ schema_version: SESSION_MINING_LEDGER_SCHEMA_VERSION, entries: sorted },
null,
2,
) + "\n"
);
}
/** One commit's mining outcome to record. */
export interface SessionMiningRecord {
readonly sessionId: string;
/** {@link minedTurnsHash} of the turn material the commit covered. */
readonly minedContentHash: string;
readonly outcome: SessionMiningOutcome;
}
/**
* Record one session's mining outcome, keyed on its portable id. Returns
* `true` when the ledger was rewritten, `false` on a byte-identical no-op.
*
* The read, the merge and the write are one critical section under the same
* lock every other Brain read-modify-write takes: two mining commits for
* different sessions can be running in two processes at once, and two
* snapshots written back in sequence would each lose the other's entry.
*
* Every OTHER entry whose session no longer has turns is dropped in the
* same pass, and so is the recorded one should its own session have none.
* A session the plan refuses (zero imported turns) has nothing for the gate
* to suppress, so its entry has no left reason to exist - and dropping only
* never-mined-again sessions would have meant one pass that hears a
* session's turn count and still leaves a dead key behind. The pruning
* happens here and not in the read on purpose: the read stays a read. The
* cost is one continuity-store pass per record building the set of sessions
* that hold turns directly, plus a full turn listing only for an entry
* whose session holds none of them - the rare root-ref-mined-over-segments
* case, where the turns are reachable only through the lineage scope the
* plan itself reads.
*/
export function recordSessionMiningOutcome(
vault: string,
record: SessionMiningRecord,
now: Date = new Date(),
): boolean {
assertVaultIdentityForWrite(vault);
const path = sessionMiningLedgerPath(vault);
const handle = acquireLockSyncWithRetry(path, undefined, { remedy: ingestLockRemedy });
try {
const entries: Record<string, SessionMiningEntry> = {
...readSessionMiningLedger(vault).entries,
};
entries[record.sessionId] = {
sessionId: record.sessionId,
minedContentHash: record.minedContentHash,
outcome: { ...record.outcome },
recordedAt: isoSecond(now),
};
// One store pass for the whole prune, mirroring the import ledger's
// one-existsSync-per-entry idiom: a full turn listing per entry here
// would be a whole-store read plus a lineage build per entry, growing
// with the ledger's history instead of with this record. The index
// applies the lineage scope, so a root ref mined over segment-recorded
// turns keeps its entry.
const turns = buildSessionTurnIndex(vault);
for (const key of Object.keys(entries)) {
if (turns.turnsFor(key).length === 0) delete entries[key];
}
const next = serializeLedger(entries);
if (existsSync(path) && readFileSync(path, "utf8") === next) return false;
atomicWriteFileSync(path, next);
return true;
} finally {
handle.release();
}
}
/**
* The mined-content hash: a SHA-256 over the turn material in list order,
* each turn as its (turn id, text, timestamp) triple. JSON is the canonical
* encoding, so the field boundaries are unambiguous no matter what the turn
* text contains. A turn that changes in any field, or a list that changes
* order, changes the hash - which is exactly the invalidation the gate
* wants: the hash covers what the plan would embed, so a matching hash
* means the recorded outcome was earned by the turns the caller is about to
* be offered again.
*/
export function minedTurnsHash(turns: ReadonlyArray<MinedTurnMaterial>): string {
const canonical = turns.map((turn) => ({
turnId: turn.turnId,
text: turn.text,
timestamp: turn.timestamp,
}));
return hashBytes(new TextEncoder().encode(JSON.stringify(canonical)));
}
/**
* The plan gate's answer: the recorded TOMBSTONE whose hash still matches
* `turns`, or `null`. `null` means no entry (never mined), a changed turn
* list (re-mined material, re-opened), a session id that has no record - or
* an entry whose recorded outcome carried a yield: a yielding commit writes
* the distilled state the census reports, not a suppression, so the session
* stays minable and only the zero-yield record suppresses. A hit means the
* session's CURRENT material was mined to nothing accepted; the caller
* suppresses with {@link SessionMiningEntry.outcome} as evidence.
*
* A corrupt or unknown-schema ledger throws, by the read's contract: a gate
* that could not verify the ledger must not wave the session through as if
* the ledger were empty.
*/
export function readMiningTombstone(
vault: string,
sessionId: string,
turns: ReadonlyArray<MinedTurnMaterial>,
): SessionMiningEntry | null {
const hash = minedTurnsHash(turns);
const entry = readSessionMiningLedger(vault).entries[sessionId.trim()];
if (entry === undefined || entry.minedContentHash !== hash) return null;
if (entry.outcome.written + entry.outcome.staged > 0) return null;
return entry;
}