Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

fix(transcripts): take a coherent VACUUM INTO snapshot of the Hermes store (#7067)

garrytana3e4e29cfcd4
diff --git a/docs/architecture/key-files/core-services-3.md b/docs/architecture/key-files/core-services-3.md
index d7ba74c5f7..543c684a8a 100644
--- a/docs/architecture/key-files/core-services-3.md
+++ b/docs/architecture/key-files/core-services-3.md
@@ -13,7 +13,7 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL
- `src/core/think/intent.ts` + `src/core/think/entity-extract.ts` — pure `classifyIntent(question)` returns `'temporal' | 'knowledge_update' | 'other'` (regex-first, no LLM, `'other'` fast path short-circuits with zero SQL). `extractCandidateEntities(question, retrievedSlugs)` pulls high-precision candidates from retrieved entity-prefix slugs (`people/`, `companies/`, `organizations/`) and medium-precision noun phrases. Stop-word boundaries + leading-verb stripper handle "When did I last meet Marco" → `marco`. Both consumed by `runThink` and the LongMemEval harness so the two paths cannot drift. Pinned by `test/think-intent.test.ts` and `test/think-entity-extract.test.ts`.
-- `src/core/transcripts/` (directory) — the transcript-adapter seam. `types.ts`: the `TranscriptAdapter` contract — `parse(path): AsyncGenerator<ParsedSession, FileDiagnostics>` (one FILE may hold many sessions; the generator RETURN value carries bytes/skipped-lines/zero-session diagnostics so an empty file explains itself), format-specific byte caps (JSONL 50MB hard cap — the 10MB default in `claude-code-jsonl.ts` belongs to the hook-lane tail reader, not imports; monolithic export JSON rejects-not-truncates at 200MB), and the ONE `buildTranscriptSlug` helper (per-provider dirs matching the conversation-archive layout; id suffixes are sha256 prefixes of the session id — hash12 in slugs, hash16 for the dedup identity — never a cleaned prefix of the source id, which would let same-prefix session ids collide). `detect.ts`: adapter registry + head-sample detection (filesystem admission first for every path, explicit format included: symlinks lstat-rejected, an unreadable path carries its `errno`, which ingest copies to the file outcome with a per-cause next step; then the explicit format wins without sniffing) + injectable `harnessRoots` (the confined discovery surface). Adapters, each with a DATED `SPEC_TARGET` + scrubbed fixture + `bytes>0 && sessions==0` drift alarm: `claude-code.ts` (thin wrapper over the shipped `claude-code-jsonl.ts`, which also exposes the full-file `parseClaudeSessionFile` with real per-message timestamps — hook-lane `parseTranscript` output is pinned byte-identical; owns `isClaudeCodeSubagentFile` (a literal `subagents` path segment at ANY depth + `agent-*.jsonl` basename — both `<session>/subagents/agent-*.jsonl` and the workflow-nested `<session>/subagents/workflows/<wf>/agent-*.jsonl` are all-sidechain, zero-turn files carrying the PARENT session id; a project slug containing "subagents" is one segment and never matches) and `isClaudeCodeWorkflowArtifactFile` (everything under `<session>/subagents/workflows/<wf>/` — the agent logs plus the `journal.jsonl` run log that matches no adapter format; a UUID-named session directly under `subagents/` stays discoverable); `discover.ts` and the CLI's `expandPaths` skip both, so they never read as gap-table backlog, drift, or `unknown format` errors that hold the since-last watermark frozen; the adapter sets `expectedEmpty` when zero accepted turn-shaped records and zero skipped lines remain after root metadata/sidechain and explicit non-human user-text filtering. Metadata-only, non-human-text-only, and all-subagent files are understood empties and do not freeze the watermark; accepted turn records that yield no text and unparseable lines still signal drift. Pinned by `test/claude-code-jsonl-origin.test.ts`), `codex.ts` (turn selection is STRUCTURAL: user turns from `event_msg` user_message (through 0.152) or `event_msg` item_completed with item.type UserMessage, text blocks only (0.153+, #5163; a turn recorded in both shapes is kept once, other item types incl. AgentMessage skipped), assistant from `response_item` output_text; a file with assistant turns and zero user turns sets `FileDiagnostics.userTurnsMissing` (codex and claude-code; E-N4), which ingest counts as drift and the session-end hook heartbeats as `degraded`/`no_user_turns` on whole-file reads (pinned by `test/codex-rollout-0154.serial.test.ts`); role user/developer response_items are injected preambles and never leak; identity is `payload.id` — the per-thread id codex embeds in the rollout filename — with `payload.session_id` (the ROOT session shared by every forked/subagent thread) only as the legacy fallback, and the FIRST `session_meta` header wins because a child rollout carries its inherited parent header later in the file; the hook lane's `parseCodexHookTranscript` follows the same two rules), `openclaw.ts` (session header + message lines; `.checkpoint.*.jsonl` snapshots rejected), `hermes.ts` (COPY-THEN-READ of state.db + wal/shm sidecars — readonly WAL opens need -shm write access and lock against a live writer; schema from the installed hermes-agent source; `json:`-prefixed and bare block arrays unwrap to text, image blocks to `[image]`; #7396: the answered responses of a `clarify` tool result (`role=tool`, `tool_name=clarify`; `status: answered` under `outcome: submitted` or `timed_out`, whose locked answers Hermes keeps, or the earlier batch shape without `timed_out`) import as user turns at the result's timestamp, each after its question as an assistant turn, with the question and tool-call id as flat `clarify_<n>_*` raw keys, while offered choices, unanswered responses, cancelled/undelivered forms, the legacy single-question shape and other tool rows never do; pinned by `test/hermes-clarify-transcripts-7396.test.ts`; PROVISIONAL), `grok.ts` (Grok Build session dirs — structural turn selection, sidecar-aware discovery; see its own entry), `chatgpt-export.ts` (the `mapping`-tree current_node walk; branches dropped by design, orphaned parents terminate quietly, latest-leaf fallback; extracted conversations.json only) + `claude-export.ts` (flat chat_messages); both export adapters load through the shared `export-json.ts` (monolithic JSON over the cap REJECTS with a split hint; a zip or wrong-shape file gets the unzip-first hint) and give id-less conversations content-derived fallback ids so two id-less exports cannot dedup-skip or overwrite each other. `render.ts`: session → part pages — imessage-slack line format with the regex IMPORTED from `conversation-parser/builtins.ts` (round-trip pinned), REAL timestamps (missing ones carry forward; zero-timestamp sessions refused — provenance is never fabricated), anchor-shaped BODY lines backslash-escaped (hostile message content cannot forge speakers), quoted facts/takes fence-marker tokens backslash-escaped at the same per-message seam (`gbrain\:facts:begin` — a session that read another page cannot plant a live or unbalanced fence on the transcript page; every fence consumer is a substring matcher, so the escape lands inside the token), fail-closed redaction (secret-scan + `harvest-private-patterns.txt` user patterns, slack-channel default excluded because it eats issue refs; imperatives COUNTED into hash-covered `transcript_import` frontmatter, never content_flag), message-boundary splitting at `PART_TARGET_BYTES` = 45,000 bytes (0.9 × the 50,000-byte content-sanity warn line, so no part takes the oversize-warn branch; far under the embed-skip threshold) with 2-message overlap, part 1 keeps the base slug, `frontmatter.id` unique per part. `ingest.ts`: the engine-facing core — a session whose part-1 `frontmatter.id` already has a live page imports at that page's slug (a renamed or re-dated conversation updates in place instead of being skipped as a cross-slug duplicate), SESSION atomicity (a failed session records its `classifyItemFailure` code and the run continues; a part the coordinator accepted but has not published leaves the session pending under a request id derived from source, slug, revision and content, so a rerun waits on that write and a write that ended without committing is retried under the next attempt id; only run-scoped failures and the `item_failures_systemic` breaker abort the run, GBRA-130 A-2, pinned by `test/pathological/a2-transcripts-ingest.test.ts`), stale-part reconciliation (deletes `part > of` leftovers), session metadata banked in the base page's raw_data as the REDACTED copy and healed by content-compare even on hash-skipped re-runs (a run that died before `putRawData`, or a private pattern added after first import, repairs on the next pass), `cleanScan`/`maxSessionTs` for the watermark. `ingest-facts.ts`: ONE `runExtractConversationFactsCore` invocation (batch `slugs` selector) inside ONE `withBudgetTracker`, `isFactsExtractionEnabled` pre-checked. Pinned by `test/transcript-adapters.test.ts`, `test/transcript-render.test.ts`, `test/transcripts-retitle.test.ts`, `test/transcripts-ingest-pglite.test.ts`, `test/e2e/transcripts-writeback-fidelity.test.ts` (raw files through the adapters into the gold-extractor facts path).
+- `src/core/transcripts/` (directory) — the transcript-adapter seam. `types.ts`: the `TranscriptAdapter` contract — `parse(path): AsyncGenerator<ParsedSession, FileDiagnostics>` (one FILE may hold many sessions; the generator RETURN value carries bytes/skipped-lines/zero-session diagnostics so an empty file explains itself), format-specific byte caps (JSONL 50MB hard cap — the 10MB default in `claude-code-jsonl.ts` belongs to the hook-lane tail reader, not imports; monolithic export JSON rejects-not-truncates at 200MB), and the ONE `buildTranscriptSlug` helper (per-provider dirs matching the conversation-archive layout; id suffixes are sha256 prefixes of the session id — hash12 in slugs, hash16 for the dedup identity — never a cleaned prefix of the source id, which would let same-prefix session ids collide). `detect.ts`: adapter registry + head-sample detection (filesystem admission first for every path, explicit format included: symlinks lstat-rejected, an unreadable path carries its `errno`, which ingest copies to the file outcome with a per-cause next step; then the explicit format wins without sniffing) + injectable `harnessRoots` (the confined discovery surface). Adapters, each with a DATED `SPEC_TARGET` + scrubbed fixture + `bytes>0 && sessions==0` drift alarm: `claude-code.ts` (thin wrapper over the shipped `claude-code-jsonl.ts`, which also exposes the full-file `parseClaudeSessionFile` with real per-message timestamps — hook-lane `parseTranscript` output is pinned byte-identical; owns `isClaudeCodeSubagentFile` (a literal `subagents` path segment at ANY depth + `agent-*.jsonl` basename — both `<session>/subagents/agent-*.jsonl` and the workflow-nested `<session>/subagents/workflows/<wf>/agent-*.jsonl` are all-sidechain, zero-turn files carrying the PARENT session id; a project slug containing "subagents" is one segment and never matches) and `isClaudeCodeWorkflowArtifactFile` (everything under `<session>/subagents/workflows/<wf>/` — the agent logs plus the `journal.jsonl` run log that matches no adapter format; a UUID-named session directly under `subagents/` stays discoverable); `discover.ts` and the CLI's `expandPaths` skip both, so they never read as gap-table backlog, drift, or `unknown format` errors that hold the since-last watermark frozen; the adapter sets `expectedEmpty` when zero accepted turn-shaped records and zero skipped lines remain after root metadata/sidechain and explicit non-human user-text filtering. Metadata-only, non-human-text-only, and all-subagent files are understood empties and do not freeze the watermark; accepted turn records that yield no text and unparseable lines still signal drift. Pinned by `test/claude-code-jsonl-origin.test.ts`), `codex.ts` (turn selection is STRUCTURAL: user turns from `event_msg` user_message (through 0.152) or `event_msg` item_completed with item.type UserMessage, text blocks only (0.153+, #5163; a turn recorded in both shapes is kept once, other item types incl. AgentMessage skipped), assistant from `response_item` output_text; a file with assistant turns and zero user turns sets `FileDiagnostics.userTurnsMissing` (codex and claude-code; E-N4), which ingest counts as drift and the session-end hook heartbeats as `degraded`/`no_user_turns` on whole-file reads (pinned by `test/codex-rollout-0154.serial.test.ts`); role user/developer response_items are injected preambles and never leak; identity is `payload.id` — the per-thread id codex embeds in the rollout filename — with `payload.session_id` (the ROOT session shared by every forked/subagent thread) only as the legacy fallback, and the FIRST `session_meta` header wins because a child rollout carries its inherited parent header later in the file; the hook lane's `parseCodexHookTranscript` follows the same two rules), `openclaw.ts` (session header + message lines; `.checkpoint.*.jsonl` snapshots rejected), `hermes.ts` (SNAPSHOT-THEN-READ: a read-only connection writes one coherent `VACUUM INTO` copy of state.db, WAL frames included, to a temp dir that is removed after the read; copying the db and its sidecars separately tore under a concurrent checkpoint, pinned by `test/hermes-snapshot-7067.serial.test.ts`; schema from the installed hermes-agent source; `json:`-prefixed and bare block arrays unwrap to text, image blocks to `[image]`; #7396: the answered responses of a `clarify` tool result (`role=tool`, `tool_name=clarify`; `status: answered` under `outcome: submitted` or `timed_out`, whose locked answers Hermes keeps, or the earlier batch shape without `timed_out`) import as user turns at the result's timestamp, each after its question as an assistant turn, with the question and tool-call id as flat `clarify_<n>_*` raw keys, while offered choices, unanswered responses, cancelled/undelivered forms, the legacy single-question shape and other tool rows never do; pinned by `test/hermes-clarify-transcripts-7396.test.ts`; PROVISIONAL), `grok.ts` (Grok Build session dirs — structural turn selection, sidecar-aware discovery; see its own entry), `chatgpt-export.ts` (the `mapping`-tree current_node walk; branches dropped by design, orphaned parents terminate quietly, latest-leaf fallback; extracted conversations.json only) + `claude-export.ts` (flat chat_messages); both export adapters load through the shared `export-json.ts` (monolithic JSON over the cap REJECTS with a split hint; a zip or wrong-shape file gets the unzip-first hint) and give id-less conversations content-derived fallback ids so two id-less exports cannot dedup-skip or overwrite each other. `render.ts`: session → part pages — imessage-slack line format with the regex IMPORTED from `conversation-parser/builtins.ts` (round-trip pinned), REAL timestamps (missing ones carry forward; zero-timestamp sessions refused — provenance is never fabricated), anchor-shaped BODY lines backslash-escaped (hostile message content cannot forge speakers), quoted facts/takes fence-marker tokens backslash-escaped at the same per-message seam (`gbrain\:facts:begin` — a session that read another page cannot plant a live or unbalanced fence on the transcript page; every fence consumer is a substring matcher, so the escape lands inside the token), fail-closed redaction (secret-scan + `harvest-private-patterns.txt` user patterns, slack-channel default excluded because it eats issue refs; imperatives COUNTED into hash-covered `transcript_import` frontmatter, never content_flag), message-boundary splitting at `PART_TARGET_BYTES` = 45,000 bytes (0.9 × the 50,000-byte content-sanity warn line, so no part takes the oversize-warn branch; far under the embed-skip threshold) with 2-message overlap, part 1 keeps the base slug, `frontmatter.id` unique per part. `ingest.ts`: the engine-facing core — a session whose part-1 `frontmatter.id` already has a live page imports at that page's slug (a renamed or re-dated conversation updates in place instead of being skipped as a cross-slug duplicate), SESSION atomicity (a failed session records its `classifyItemFailure` code and the run continues; a part the coordinator accepted but has not published leaves the session pending under a request id derived from source, slug, revision and content, so a rerun waits on that write and a write that ended without committing is retried under the next attempt id; only run-scoped failures and the `item_failures_systemic` breaker abort the run, GBRA-130 A-2, pinned by `test/pathological/a2-transcripts-ingest.test.ts`), stale-part reconciliation (deletes `part > of` leftovers), session metadata banked in the base page's raw_data as the REDACTED copy and healed by content-compare even on hash-skipped re-runs (a run that died before `putRawData`, or a private pattern added after first import, repairs on the next pass), `cleanScan`/`maxSessionTs` for the watermark. `ingest-facts.ts`: ONE `runExtractConversationFactsCore` invocation (batch `slugs` selector) inside ONE `withBudgetTracker`, `isFactsExtractionEnabled` pre-checked. Pinned by `test/transcript-adapters.test.ts`, `test/transcript-render.test.ts`, `test/transcripts-retitle.test.ts`, `test/transcripts-ingest-pglite.test.ts`, `test/e2e/transcripts-writeback-fidelity.test.ts` (raw files through the adapters into the gold-extractor facts path).
- `src/core/verbs.ts` — MEMORY_VERBS v1: the four frozen protocol verbs (`remember`, `entity`, `synthesize`, `forget`) as first-class Operations, plus `MEMORY_VERBS_VERSION` (single source of truth, =1), `VERB_NAMES`, the hand-authored `RESPONSE_SCHEMAS` registry (Operation carries input params only; response shapes live here and conformance validates LIVE responses against them), and `ERROR_SCHEMA`. The extended `recall` op is defined in ops/facts.ts and registered through operations.ts. Its response registry advertises optional `budget_packing` policy/applied/reason and per-arm integer accounting without changing required default fields or protocol version; `test/protocol-document.test.ts` pins discovery and malformed metadata rejection. The `context_pack` and `delta` fact items advertise optional `fact_id` and `provenance` (#6146). The recall registry advertises optional `facts_order`, `facts_degraded` and per-fact `relevance` (question ranking). RUNTIME LEAF invariant: operations.ts spreads `verbOperations` into its array at module-eval time, so this file must never statically import operations.ts VALUES (type-only imports fine; handlers use dynamic import) — violating it causes a TDZ crash on whichever module evaluates second. Every verb error carries a populated `suggestion` + `protocol_version` (via `verbError` in operations.ts). The `forget` verb deliberately has NO cliHints (CLI_ONLY `forget` dispatches first and would shadow it). Frozen contract: docs/protocol/MEMORY_VERBS_v1.md; pinned by test/memory-verbs-conformance.test.ts. The starter list leaves out remember's `expected_revision`/`force`/`valid_from`/`infer_entity`/`visibility` and, in `ops/facts.ts`, recall's `budget_policy`/`include_expired`/`supersessions`/`limit`/`grep`/`include_pending`/`return_unit`/`return_window`, context_pack's `since`/`session_id`/`include_private` and delta's `since_slug`/`entities`/`include_private` (`fullSurfaceOnly`); the verbs surface lists every param.
- `src/core/verbs/conformance.ts` + `src/core/verbs/conformance-fixtures.ts` — the conformance runner core (transport-agnostic: minimal `ConformanceClient` = list_tools + call_tool) and the embedded fixture set. Deterministic by construction: shape/enum/behavior/round-trip checks only, never ranking quality. Validation is NON-STRICT on extra fields (additive-forever means unknown fields are always legal). Entity-card cases seed via put_page when the target exposes it and skip honestly on verbs-only targets; synthesize is cost-gated behind --synthesize. `validateAgainstSchema` is a minimal JSON-Schema-subset validator (type unions, required, properties, enum, const, items). Fixtures mirror to `test/fixtures/memory-verbs/cases.json` (BrainBench seeds; drift-guarded by the conformance test). The negative self-test (test/memory-verbs-conformance.test.ts) proves the runner FAILS a lying server.
diff --git a/scripts/serial-files.tsv b/scripts/serial-files.tsv
index 2a818722ac..587589a499 100644
--- a/scripts/serial-files.tsv
+++ b/scripts/serial-files.tsv
@@ -449,3 +449,4 @@ test/embed-once-per-content-version.serial.test.ts global-state global-state: pr
test/page-embedding-lease.serial.test.ts global-state global-state: process-wide test seams __setEmbedTransportForTests (each case swaps the embed transport, one to a refusing provider) and pageEmbeddingLeaseProbe (the module-wide lease contention hook); withEnv scopes GBRAIN_HOME
test/migrate-search-path-best-effort.serial.test.ts R2 R2: mock.module(v277 migration module) replaces modules for every file in the process
test/agent-session-speaker-8064.serial.test.ts R1 R1: sets GBRAIN_HOME and GBRAIN_SESSION_AGENT and clears DATABASE_URL and the hook env on process.env
+test/hermes-snapshot-7067.serial.test.ts R2 R2: mock.module(node:fs) wraps copyFileSync to inject a writer and WAL checkpoint mid-snapshot
diff --git a/src/core/transcripts/hermes.ts b/src/core/transcripts/hermes.ts
index 02e8c0c2d2..9378b15a43 100644
--- a/src/core/transcripts/hermes.ts
+++ b/src/core/transcripts/hermes.ts
@@ -2,11 +2,12 @@
* hermes.ts — Hermes state.db (SQLite) adapter (cathedral-4).
*
* ONE store file holds MANY sessions (hermes-agent DEFAULT_DB_PATH =
- * <hermes home>/state.db). Reads are COPY-THEN-READ by default: readonly
- * opens of a WAL-mode SQLite database require write access to the -shm
- * sidecar and can intermittently lock against a live writer, so the adapter
- * copies the DB (+ -wal/-shm sidecars when present) to a temp dir and reads
- * the copy — deterministic, zero lock races, cleaned up in finally.
+ * <hermes home>/state.db). Reads are SNAPSHOT-THEN-READ: the adapter opens
+ * the live store read-only just long enough for `VACUUM INTO` to write a
+ * transactionally coherent copy (main file + un-checkpointed WAL frames) to
+ * a temp dir, then reads the copy and removes it in finally. Copying the
+ * main file and its -wal/-shm sidecars as separate files tears when a live
+ * writer checkpoints between the copies (#7067).
*
* Schema verified against the INSTALLED hermes-agent v0.20.0 source
* (hermes_state_common.py SCHEMA_SQL) — sessions(id, source, display_name,
@@ -17,7 +18,7 @@
* the runtime backstop.
*/
-import { copyFileSync, existsSync, mkdtempSync, rmSync, statSync } from 'node:fs';
+import { existsSync, mkdtempSync, rmSync, statSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { basename, join } from 'node:path';
import { Database } from 'bun:sqlite';
@@ -156,9 +157,9 @@ export const hermesAdapter: TranscriptAdapter = {
async *parse(path: string, opts: ParseSessionsOpts = {}): AsyncGenerator<ParsedSession, FileDiagnostics> {
const cap = opts.maxBytes ?? HERMES_DB_HARD_CAP;
const size = statSync(path).size;
- // The cap bounds the TOTAL copied (db + sidecars) — a runaway WAL can
- // dwarf the main file, and only capping the db would let the copy blow
- // through temp storage while advertising a 512MB bound.
+ // The cap bounds the TOTAL store (db + sidecars) — a runaway WAL can
+ // dwarf the main file, and only capping the db would let the snapshot
+ // blow through temp storage while advertising a 512MB bound.
let totalBytes = size;
for (const suffix of ['-wal', '-shm']) {
if (existsSync(path + suffix)) totalBytes += statSync(path + suffix).size;
@@ -169,18 +170,19 @@ export const hermesAdapter: TranscriptAdapter = {
);
}
- // Copy-then-read: DB plus WAL/SHM sidecars so un-checkpointed writes are
- // visible in the copy. A live writer can checkpoint BETWEEN the copies —
- // the resulting torn snapshot surfaces as a schema/corruption error from
- // the sessions query below, lands in the drift lane, and (because drift
- // freezes the watermark) is safely retried by the next run.
+ // VACUUM INTO runs inside one read transaction, so the snapshot holds
+ // exactly one committed state (WAL frames included) even while a live
+ // writer commits and checkpoints; the source connection is read-only.
const tmp = mkdtempSync(join(tmpdir(), 'gbrain-hermes-'));
const copyPath = join(tmp, basename(path));
let sessions = 0;
try {
- copyFileSync(path, copyPath);
- for (const suffix of ['-wal', '-shm']) {
- if (existsSync(path + suffix)) copyFileSync(path + suffix, copyPath + suffix);
+ const source = new Database(path, { readonly: true });
+ try {
+ source.exec('PRAGMA busy_timeout = 5000');
+ source.query('VACUUM INTO ?').run(copyPath);
+ } finally {
+ source.close();
}
const db = new Database(copyPath, { readonly: true });
diff --git a/test/hermes-snapshot-7067.serial.test.ts b/test/hermes-snapshot-7067.serial.test.ts
new file mode 100644
index 0000000000..cf763458aa
--- /dev/null
+++ b/test/hermes-snapshot-7067.serial.test.ts
@@ -0,0 +1,114 @@
+/**
+ * #7067: the Hermes adapter snapshotted a live WAL-mode store by copying the
+ * main file and then the -wal/-shm sidecars as separate filesystem copies. A
+ * writer that checkpoints between those copies leaves a torn snapshot: the
+ * copied WAL refers to pages the copied main file never had, and the read
+ * fails (`disk I/O error`) even though the original store is healthy.
+ *
+ * Protects: a writer that commits, checkpoints (TRUNCATE) and writes again
+ * between the old copy steps cannot tear the snapshot; the adapter reads one
+ * coherent, complete session; un-checkpointed WAL writes are visible; the
+ * original store stays healthy and no temp snapshot is left behind.
+ * Seams: mock.module('node:fs') wraps copyFileSync to inject the writer after
+ * the main-file copy (the issue's synthetic repro shape), hence serial.
+ */
+import { afterAll, describe, expect, mock, test } from 'bun:test';
+import { Database } from 'bun:sqlite';
+import * as fs from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+
+const realCopyFileSync = fs.copyFileSync;
+let activePath = '';
+let writer: Database | null = null;
+let injected = false;
+
+function injectWriterAndCheckpoint(db: Database): void {
+ const ins = db.prepare('INSERT INTO messages (session_id, role, content, timestamp) VALUES (?, ?, ?, ?)');
+ db.transaction(() => {
+ for (let i = 0; i < 400; i++) ins.run('session-a', 'assistant', `synthetic reply ${i} ${'x'.repeat(1000)}`, 1786000001 + i);
+ })();
+ db.exec('PRAGMA wal_checkpoint(TRUNCATE)');
+ db.prepare('UPDATE messages SET content = ? WHERE id = 1').run(`synthetic question updated ${'z'.repeat(50000)}`);
+}
+
+mock.module('node:fs', () => ({
+ ...fs,
+ copyFileSync: (src: string, dest: string, ...rest: unknown[]) => {
+ (realCopyFileSync as (...a: unknown[]) => void)(src, dest, ...rest);
+ if (!injected && writer && src === activePath) {
+ injected = true;
+ injectWriterAndCheckpoint(writer);
+ }
+ },
+}));
+
+const { hermesAdapter } = await import('../src/core/transcripts/hermes.ts');
+
+const root = fs.mkdtempSync(join(tmpdir(), 'gbrain-hermes-7067-'));
+afterAll(() => {
+ writer?.close();
+ fs.rmSync(root, { recursive: true, force: true });
+});
+
+function openLiveStore(name: string): Database {
+ activePath = join(root, name);
+ const db = new Database(activePath, { create: true });
+ db.exec(`
+ PRAGMA page_size = 512;
+ PRAGMA journal_mode = WAL;
+ PRAGMA wal_autocheckpoint = 0;
+ CREATE TABLE sessions (id TEXT PRIMARY KEY, title TEXT, display_name TEXT, started_at REAL, cwd TEXT, model TEXT, source TEXT);
+ CREATE TABLE messages (id INTEGER PRIMARY KEY, session_id TEXT, role TEXT, content TEXT, timestamp REAL);
+ `);
+ db.prepare('INSERT INTO sessions VALUES (?, ?, ?, ?, ?, ?, ?)').run('session-a', 'synthetic session', null, 1786000000, '/tmp/acme-example', null, 'cli');
+ db.prepare('INSERT INTO messages (session_id, role, content, timestamp) VALUES (?, ?, ?, ?)').run('session-a', 'user', 'synthetic question', 1786000000);
+ db.exec('PRAGMA wal_checkpoint(TRUNCATE)');
+ return db;
+}
+
+async function drain(path: string) {
+ const sessions = [];
+ const it = hermesAdapter.parse(path, { maxBytes: 64 * 1024 * 1024 });
+ while (true) {
+ const step = await it.next();
+ if (step.done) return { sessions, diag: step.value };
+ sessions.push(step.value);
+ }
+}
+
+function hermesTempDirs(): number {
+ return fs.readdirSync(tmpdir()).filter((n) => n.startsWith('gbrain-hermes-') && !n.startsWith('gbrain-hermes-7067-')).length;
+}
+
+describe('hermes snapshot under a concurrent checkpoint (#7067)', () => {
+ test('a writer that checkpoints mid-snapshot cannot tear it: one coherent, complete session', async () => {
+ writer = openLiveStore('race.db');
+ injected = false;
+ const { sessions, diag } = await drain(activePath);
+ expect(diag.zeroSessionsReason).toBeUndefined();
+ expect(sessions).toHaveLength(1);
+
+ const texts = sessions[0].messages.map((m) => m.text);
+ const afterWriter = writer.query<{ content: string }, []>('SELECT content FROM messages ORDER BY timestamp, id').all().map((r) => r.content);
+ expect([['synthetic question'], afterWriter]).toContainEqual(texts);
+ expect(writer.query<{ integrity_check: string }, []>('PRAGMA integrity_check').get()?.integrity_check).toBe('ok');
+ writer.close();
+ writer = null;
+ });
+
+ test('un-checkpointed WAL writes are in the snapshot, and no temp snapshot survives', async () => {
+ writer = openLiveStore('wal-only.db');
+ injected = true;
+ writer.prepare('INSERT INTO messages (session_id, role, content, timestamp) VALUES (?, ?, ?, ?)').run('session-a', 'assistant', 'synthetic answer still in the WAL', 1786000001);
+ expect(fs.statSync(`${activePath}-wal`).size).toBeGreaterThan(0);
+
+ const tmpBefore = hermesTempDirs();
+ const { sessions } = await drain(activePath);
+ expect(sessions).toHaveLength(1);
+ expect(sessions[0].messages.map((m) => m.text)).toEqual(['synthetic question', 'synthetic answer still in the WAL']);
+ expect(hermesTempDirs()).toBeLessThanOrEqual(tmpBefore);
+ writer.close();
+ writer = null;
+ });
+});