Local-first 🧠memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly 😴 dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.
TypeScript
/**
* Per-file session-import checkpoint (nothing-writes-silently, Unit F).
*
* Resume is NOT introduced here. `ingest/checkpoint.ts` has shipped
* plan-scoped resume for the source-ingest lane since the Ingestion & Import
* Robustness suite; this module extends that substrate
* ({@link ../checkpoint-store.ts}) to the sessions lane, which had none.
*
* What the sessions lane had instead was DEDUP: `importSession` rebuilt a
* dedup index from the inbox at the start of every run, so a second run over
* the same transcript found every hash already present and discarded its
* writes. That is re-do-and-discard, not resume - an interrupted 50k-turn
* import re-read and re-hashed all fifty thousand turns to conclude it had
* nothing to write. This checkpoint records the turn boundary instead, so the
* next run skips straight past the turns a previous one finished.
*
* ## The key: (file identity, turn boundary)
*
* The turn boundary is a count of turns the adapter has already yielded and
* the import has already processed. The file identity is what makes that
* count meaningful, and it is two cheap facts rather than a content hash: the
* SHA-256 of the transcript's first line, and its size in bytes at the moment
* the boundary was taken. Hashing the whole file would defeat the purpose -
* re-reading every byte is the cost being avoided - and would invalidate the
* checkpoint on every append to a log that is append-only by construction.
*
* Stated precisely, because the guarantee is narrower than "the file is
* unchanged": a transcript whose first line differs, or which has SHRUNK
* since the boundary, is a different log and is refused. A log rewritten in
* place to the same-or-greater length with the same first line would not be
* caught. Every session adapter this build ships reads an append-only JSONL
* whose first line carries the session's own identifiers, so that case is not
* one the format produces; the limit is written down rather than implied.
*
* ## Scoped to the import's question, not just the file
*
* The id folds the import's filters in - role and text filters, `since`,
* event-time preservation, recall, the ingest scope label, the agent. Two
* runs asking different questions of one transcript reach different turn sets
* and must never resume off each other's boundary; giving them different ids
* is what makes that structural rather than remembered.
*
* Language-agnostic: the id is a hex digest over paths and option values; no
* natural-language content is inspected.
*/
import { createHash } from "node:crypto";
import {
checkpointFilePath,
checkpointingEnabled,
readCheckpointObject,
removeCheckpointFile,
withCheckpointLock,
writeCheckpointObject,
} from "../checkpoint-store.ts";
import { isoSecond } from "../time.ts";
/** Only schema version currently understood. Unknown versions are refused. */
const SCHEMA_VERSION = 1 as const;
/** Subdirectory holding one JSON checkpoint per scoped session import. */
const CHECKPOINT_DIR = "session-import-checkpoints";
/** How a refusal names this lane's id and its file. */
const ID_LABEL = "session import";
const FILE_LABEL = "session import checkpoint";
/**
* Turns between boundary writes.
*
* A boundary per turn would take the sync lock and land an atomic write fifty
* thousand times on a large transcript, which costs more than the extraction
* it protects. A boundary every 250 turns bounds an interruption's lost work
* to at most 250 turns while keeping the whole run to a couple of hundred
* checkpoint writes.
*/
export const SESSION_CHECKPOINT_TURN_INTERVAL = 250;
/** The persisted per-import checkpoint. */
export interface SessionImportCheckpoint {
readonly schema_version: typeof SCHEMA_VERSION;
/** Stable id derived from the session file and the import's scope. */
readonly import_id: string;
/**
* The session log this boundary belongs to, as this machine addressed it.
* Machine-local by design and for the same reason the session import ledger
* is: a transcript lives outside the vault, under a per-runtime root that
* differs on every device, so there is no portable spelling of it.
*/
readonly session_file: string;
/** SHA-256 of the transcript's first line when the boundary was taken. */
readonly head_hash: string;
/** Size of the transcript in bytes when the boundary was taken. */
readonly bytes: number;
/** Turns the import had fully processed at the boundary. */
readonly turns_completed: number;
readonly updated_at: string;
}
/** The subset of import options that changes which turns a run processes. */
export interface SessionImportScope {
readonly agent: string;
readonly format?: string;
readonly since?: Date;
readonly filterRoles?: ReadonlyArray<string>;
readonly filterTextIncludes?: string;
readonly preserveEventTime?: boolean;
readonly recall?: boolean;
readonly recallSessionId?: string;
readonly recallSummaryGroupSize?: number;
readonly ingestScope?: string;
readonly rawCodec?: boolean;
}
/** Why a checkpoint that existed was refused instead of resumed from. */
export const SESSION_RESUME_DISCARD = Object.freeze({
/** The transcript's first line is not the one the boundary was taken against. */
headChanged: "head_changed",
/** The transcript is smaller than it was, so the boundary is past its end. */
fileShrank: "file_shrank",
/**
* The file parsed, and its payload is not one this writer could have
* written - a `turns_completed` that is not a whole number, or a negative
* one. Coercing it to zero would present finished turns as pending and be
* indistinguishable from "there was no checkpoint".
*/
payloadInvalid: "payload_invalid",
} as const);
export type SessionResumeDiscard =
(typeof SESSION_RESUME_DISCARD)[keyof typeof SESSION_RESUME_DISCARD];
/** What a run measured about the transcript it is about to read. */
export interface SessionFileIdentity {
readonly headHash: string;
readonly bytes: number;
}
/** The resume decision: where to start, and why a checkpoint was rejected. */
export interface SessionResumeDecision {
/** Turns to skip. Zero when there is no usable checkpoint. */
readonly turns: number;
/** Set only when a checkpoint existed and was refused. */
readonly discarded: SessionResumeDiscard | null;
}
/** SHA-256 hex of a transcript's first line - the cheap half of file identity. */
export function computeSessionHeadHash(firstLine: string): string {
return createHash("sha256").update(firstLine).digest("hex");
}
/**
* Deterministic import id: a short SHA-256 hex over the session file's path
* and every option that changes which turns the run processes. Options are
* folded in a fixed order with NUL separators so no two distinct scopes can
* collide by concatenation.
*/
export function computeSessionImportId(sessionFile: string, scope: SessionImportScope): string {
const parts: string[] = [
sessionFile,
scope.agent,
scope.format ?? "",
scope.since ? String(scope.since.getTime()) : "",
[...(scope.filterRoles ?? [])].toSorted().join(","),
scope.filterTextIncludes ?? "",
scope.preserveEventTime === true ? "1" : "",
scope.recall === true ? "1" : "",
scope.recallSessionId ?? "",
scope.recallSummaryGroupSize !== undefined ? String(scope.recallSummaryGroupSize) : "",
scope.ingestScope ?? "",
scope.rawCodec === true ? "1" : "",
];
return createHash("sha256").update(parts.join("\u0000")).digest("hex").slice(0, 16);
}
/** Absolute path of one scoped import's checkpoint file. */
export function sessionCheckpointPath(vault: string, importId: string): string {
return checkpointFilePath(vault, CHECKPOINT_DIR, ID_LABEL, importId);
}
function serialize(cp: SessionImportCheckpoint): string {
return (
JSON.stringify(
{
schema_version: cp.schema_version,
import_id: cp.import_id,
session_file: cp.session_file,
head_hash: cp.head_hash,
bytes: cp.bytes,
turns_completed: cp.turns_completed,
updated_at: cp.updated_at,
},
null,
2,
) + "\n"
);
}
function requireNonNegativeInteger(field: string, value: number): void {
if (!Number.isInteger(value) || value < 0) {
throw new Error(
`session import checkpoint: ${field} must be a non-negative integer, got ${value}`,
);
}
}
function requireHex64(field: string, value: string): void {
if (!/^[0-9a-f]{64}$/.test(value)) {
throw new Error(
`session import checkpoint: ${field} must be a 64-character lowercase hex digest`,
);
}
}
/**
* A checkpoint file parsed, and carried a payload the writer could not have
* produced. Its own class so {@link resolveSessionResume} can turn exactly
* this condition into a named discard and let every other read failure -
* corrupt bytes, an unknown `schema_version` - keep propagating.
*/
export class SessionCheckpointPayloadError extends Error {
constructor(message: string) {
super(message);
this.name = "SessionCheckpointPayloadError";
}
}
/**
* Read one scoped import's checkpoint. A missing file (or checkpointing
* disabled) returns `null`. Corrupt bytes and an unknown `schema_version` are
* hard errors - never a silent reset, which would present finished turns as
* pending and re-hash every one of them.
*
* The PAYLOAD is held to the same rule as the envelope, and to exactly the
* rule {@link recordSessionProgress} enforces on the way in: a
* `turns_completed` or `bytes` that is not a non-negative integer raises
* {@link SessionCheckpointPayloadError} rather than being coerced to zero.
* The coercion was the silent reset by another route - zero turns completed
* reads back as "there was no checkpoint" - and a negative count, which no
* writer here can produce, was honoured verbatim.
*/
export function readSessionCheckpoint(
vault: string,
importId: string,
): SessionImportCheckpoint | null {
if (!checkpointingEnabled()) return null;
const path = sessionCheckpointPath(vault, importId);
const obj = readCheckpointObject(path, FILE_LABEL, SCHEMA_VERSION);
if (obj === null) return null;
return Object.freeze({
schema_version: SCHEMA_VERSION,
import_id: importId,
session_file: typeof obj["session_file"] === "string" ? obj["session_file"] : "",
head_hash: typeof obj["head_hash"] === "string" ? obj["head_hash"] : "",
bytes: requireStoredCount(path, "bytes", obj["bytes"]),
turns_completed: requireStoredCount(path, "turns_completed", obj["turns_completed"]),
updated_at: typeof obj["updated_at"] === "string" ? obj["updated_at"] : "",
});
}
function requireStoredCount(path: string, field: string, value: unknown): number {
if (!Number.isInteger(value) || (value as number) < 0) {
throw new SessionCheckpointPayloadError(
`${FILE_LABEL}: ${field} must be a non-negative integer, got ${JSON.stringify(value)}: ${path}`,
);
}
return value as number;
}
/** What {@link recordSessionProgress} is told about the boundary just reached. */
export interface SessionProgressInput {
readonly sessionFile: string;
readonly headHash: string;
readonly bytes: number;
readonly turnsCompleted: number;
}
/**
* Record the turn boundary a run has reached, atomically. Returns `true` when
* the checkpoint was written, `false` on a no-op - checkpointing disabled, or
* the serialized bytes unchanged.
*
* The read and the write are ONE critical section under the sync lock, for
* the reason the shared substrate states: two runs over one transcript that
* both wrote back the state they read would leave the lower boundary, silently
* turning finished turns back into pending work.
*/
export function recordSessionProgress(
vault: string,
importId: string,
input: SessionProgressInput,
now: Date,
): boolean {
requireNonNegativeInteger("turns_completed", input.turnsCompleted);
requireNonNegativeInteger("bytes", input.bytes);
requireHex64("head_hash", input.headHash);
if (!checkpointingEnabled()) return false;
const path = sessionCheckpointPath(vault, importId);
return withCheckpointLock(path, () =>
writeCheckpointObject(
vault,
path,
serialize({
schema_version: SCHEMA_VERSION,
import_id: importId,
session_file: input.sessionFile,
head_hash: input.headHash,
bytes: input.bytes,
turns_completed: input.turnsCompleted,
updated_at: isoSecond(now),
}),
),
);
}
/**
* Decide where a run should start. Returns `turns: 0` with `discarded: null`
* when there is no checkpoint at all, and `turns: 0` with a named reason when
* one exists but was taken against a transcript this is no longer. The caller
* surfaces that reason: a boundary quietly ignored is a full re-import the
* operator was never told about.
*/
export function resolveSessionResume(
vault: string,
importId: string,
live: SessionFileIdentity,
): SessionResumeDecision {
let cp: SessionImportCheckpoint | null;
try {
cp = readSessionCheckpoint(vault, importId);
} catch (err) {
// A payload the writer could not have produced is a boundary this run
// cannot use - reported as a discard, the same as a boundary taken
// against another transcript, so the re-import is the operator's to see.
// Every other read failure (corrupt bytes, an unknown schema_version)
// still propagates: those say the substrate is broken, not that one
// boundary is unusable.
if (err instanceof SessionCheckpointPayloadError) {
return { turns: 0, discarded: SESSION_RESUME_DISCARD.payloadInvalid };
}
throw err;
}
if (cp === null) return { turns: 0, discarded: null };
if (cp.head_hash !== live.headHash) {
return { turns: 0, discarded: SESSION_RESUME_DISCARD.headChanged };
}
if (live.bytes < cp.bytes) {
return { turns: 0, discarded: SESSION_RESUME_DISCARD.fileShrank };
}
return { turns: cp.turns_completed, discarded: null };
}
/**
* Drop one scoped import's checkpoint once the run has drained it - the same
* settle the ingest lane performs on a fully drained plan, where the content
* manifest (here, the dedup index) is the authoritative final state from then
* on. Returns `true` when a file was removed.
*/
export function clearSessionCheckpoint(vault: string, importId: string): boolean {
return removeCheckpointFile(vault, sessionCheckpointPath(vault, importId));
}