Skip to content

armgit / ruflo

Public Imported from GitHub
0

🌊 The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated

fix(memory): make the WAL-sidecar safety gate liveness-based, not presence-based (#3161)

Nicholas Ruest0a23ae4487fb
diff --git a/v3/@claude-flow/cli/__tests__/issue-2735-memory-wal-sidecar-guard.test.ts b/v3/@claude-flow/cli/__tests__/issue-2735-memory-wal-sidecar-guard.test.ts
index 3d12907f2d..7e4100c67f 100644
--- a/v3/@claude-flow/cli/__tests__/issue-2735-memory-wal-sidecar-guard.test.ts
+++ b/v3/@claude-flow/cli/__tests__/issue-2735-memory-wal-sidecar-guard.test.ts
@@ -7,21 +7,34 @@
* native writer (daemon, MCP server, another CLI invocation) held the file
* open.
*
- * This suite covers the scoped fix actually shipped: refuse the sql.js
- * fallback (`success: false`, typed error, never a whole-image write) when
- * `-wal`/`-shm` sidecar files are present next to the database β€” strong
- * evidence of a live native WAL connection, since a native connection keeps
- * its sidecars on disk for its entire lifetime. It does NOT cover the
- * fuller "scan live process holders" design also discussed in the issue β€”
- * that remains a documented follow-up, not part of this fix.
+ * #3161 found the original presence-only version of this guard ("-wal"/
+ * "-shm" exists β†’ refuse) was over-eager: `ruflo doctor`'s readonly
+ * diagnostic connections leave orphaned sidecars behind after a plain
+ * `.close()`, with no live holder at all, permanently blocking every later
+ * write. The guard now probes LIVENESS (`PRAGMA wal_checkpoint(TRUNCATE)`,
+ * then a post-close re-existence check β€” see `hasNativeWalSidecars`'s own
+ * doc comment in memory-initializer.ts for why both steps are needed)
+ * instead of treating mere sidecar presence as proof of a live connection.
+ *
+ * This file's cases below were rewritten accordingly: the "sidecar present"
+ * cases now use a REAL WAL-mode database with a genuinely orphaned WAL (no
+ * live holder) β€” the exact #3161 scenario β€” and assert the write now
+ * PROCEEDS (the bug fix). A new case asserts the original #2735 guarantee
+ * is intact: a WAL connection with a genuinely open, uncommitted transaction
+ * still blocks the write. (Previously this file used empty hand-touched
+ * `-wal`/`-shm` files as a presence proxy; that no longer exercises a
+ * meaningful scenario once the guard is liveness-based, since such a file
+ * isn't attached to any real WAL-mode database and the probe correctly
+ * treats it as inert.)
*
* The bridge is force-disabled via CLAUDE_FLOW_DISABLE_BRIDGE=1 (the
* package's own documented switch, dist/src/memory/memory-initializer.js)
* so every case below exercises the sql.js fallback path directly,
- * deterministically, without needing a real second native connection.
+ * deterministically.
*/
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
-import { mkdtempSync, rmSync, writeFileSync, existsSync } from 'node:fs';
+import { mkdtempSync, rmSync, existsSync, statSync } from 'node:fs';
+import { createRequire } from 'node:module';
import { tmpdir } from 'node:os';
import path from 'node:path';
@@ -29,6 +42,32 @@ let dir: string;
let dbPath: string;
const ORIGINAL_ENV = process.env.CLAUDE_FLOW_DISABLE_BRIDGE;
+const sidecarsExist = () => existsSync(`${dbPath}-wal`) || existsSync(`${dbPath}-shm`);
+
+function openNative(opts?: Record<string, unknown>) {
+ const Database = createRequire(import.meta.url)('better-sqlite3');
+ return new Database(dbPath, opts);
+}
+
+/**
+ * Puts the database into real WAL mode with a committed write, then leaves
+ * a genuinely orphaned WAL behind via a readonly open + plain `.close()` β€”
+ * the same pattern `doctor.ts`'s diagnostic checks use. No connection is
+ * left attached afterward; this is the exact #3161 scenario.
+ */
+function makeGenuineOrphan(): void {
+ const writer = openNative();
+ writer.pragma('journal_mode = WAL');
+ writer.exec('CREATE TABLE IF NOT EXISTS _wal_probe_marker(x)');
+ writer.close();
+ expect(sidecarsExist()).toBe(false); // clean close, nothing else attached β€” sidecars gone
+
+ const reader = openNative({ readonly: true, fileMustExist: true });
+ reader.prepare('SELECT 1').get();
+ reader.close();
+ expect(sidecarsExist()).toBe(true); // orphaned by the readonly close, no live holder
+}
+
beforeEach(async () => {
dir = mkdtempSync(path.join(tmpdir(), 'ruflo-2735-'));
dbPath = path.join(dir, 'memory.db');
@@ -47,21 +86,15 @@ afterEach(() => {
rmSync(dir, { recursive: true, force: true });
});
-describe('memory sql.js fallback WAL-sidecar guard β€” issue #2735', () => {
- it('storeEntry refuses the whole-image write when -wal sidecar is present', async () => {
- writeFileSync(`${dbPath}-wal`, '');
+describe('memory sql.js fallback WAL-sidecar guard β€” issue #2735 / #3161', () => {
+ it('storeEntry proceeds when a genuinely orphaned -wal/-shm pair is present (#3161)', async () => {
+ makeGenuineOrphan();
const { storeEntry } = await import('../src/memory/memory-initializer.js');
- const result = await storeEntry({ key: 'k1', value: 'v1', dbPath });
- expect(result.success).toBe(false);
- expect(result.error).toMatch(/native WAL connection/i);
- });
-
- it('storeEntry refuses the whole-image write when -shm sidecar is present', async () => {
- writeFileSync(`${dbPath}-shm`, '');
- const { storeEntry } = await import('../src/memory/memory-initializer.js');
- const result = await storeEntry({ key: 'k1', value: 'v1', dbPath });
- expect(result.success).toBe(false);
- expect(result.error).toMatch(/native WAL connection/i);
+ const result = await storeEntry({ key: 'k1', value: 'v1', dbPath, generateEmbeddingFlag: false });
+ expect(result.success).toBe(true);
+ expect(result.id).not.toBe('');
+ // The liveness probe's own checkpoint clears the orphan as a side effect.
+ expect(sidecarsExist()).toBe(false);
});
it('storeEntry proceeds normally with no sidecars present', async () => {
@@ -71,16 +104,17 @@ describe('memory sql.js fallback WAL-sidecar guard β€” issue #2735', () => {
expect(result.id).not.toBe('');
});
- it('getEntry refuses the whole-image access_count-bump write when sidecars are present', async () => {
+ it('getEntry proceeds through the access_count-bump write past a genuine orphan (#3161)', async () => {
const { storeEntry } = await import('../src/memory/memory-initializer.js');
const stored = await storeEntry({ key: 'k2', value: 'v2', dbPath, generateEmbeddingFlag: false });
expect(stored.success).toBe(true);
- writeFileSync(`${dbPath}-wal`, '');
+ makeGenuineOrphan();
const { getEntry } = await import('../src/memory/memory-initializer.js');
const result = await getEntry({ key: 'k2', dbPath });
- expect(result.success).toBe(false);
- expect(result.error).toMatch(/native WAL connection/i);
+ expect(result.success).toBe(true);
+ expect(result.found).toBe(true);
+ expect(sidecarsExist()).toBe(false);
});
it('getEntry proceeds normally with no sidecars present', async () => {
@@ -91,15 +125,16 @@ describe('memory sql.js fallback WAL-sidecar guard β€” issue #2735', () => {
expect(result.found).toBe(true);
});
- it('deleteEntry refuses the whole-image write when sidecars are present', async () => {
+ it('deleteEntry proceeds past a genuinely orphaned sidecar pair (#3161)', async () => {
const { storeEntry } = await import('../src/memory/memory-initializer.js');
await storeEntry({ key: 'k4', value: 'v4', dbPath, generateEmbeddingFlag: false });
- writeFileSync(`${dbPath}-shm`, '');
+ makeGenuineOrphan();
const { deleteEntry } = await import('../src/memory/memory-initializer.js');
const result = await deleteEntry({ key: 'k4', dbPath });
- expect(result.success).toBe(false);
- expect(result.error).toMatch(/native WAL connection/i);
+ expect(result.success).toBe(true);
+ expect(result.deleted).toBe(true);
+ expect(sidecarsExist()).toBe(false);
});
it('deleteEntry proceeds normally with no sidecars present', async () => {
@@ -110,13 +145,49 @@ describe('memory sql.js fallback WAL-sidecar guard β€” issue #2735', () => {
expect(result.deleted).toBe(true);
});
+ it('storeEntry still refuses while a native connection genuinely holds an open transaction', async () => {
+ const live = openNative();
+ live.pragma('journal_mode = WAL');
+ live.exec('BEGIN');
+ live.exec('CREATE TABLE _wal_probe_marker(x)');
+ live.exec('INSERT INTO _wal_probe_marker VALUES (1)');
+ expect(sidecarsExist()).toBe(true);
+
+ try {
+ const { storeEntry } = await import('../src/memory/memory-initializer.js');
+ const result = await storeEntry({ key: 'k7', value: 'v7', dbPath, generateEmbeddingFlag: false });
+ expect(result.success).toBe(false);
+ expect(result.error).toMatch(/native WAL connection/i);
+ } finally {
+ live.exec('COMMIT');
+ live.close();
+ }
+ });
+
it('a refused store never touches the database file (no whole-image write occurred)', async () => {
- const before = existsSync(dbPath) ? require('node:fs').statSync(dbPath).mtimeMs : 0;
- writeFileSync(`${dbPath}-wal`, '');
- const { storeEntry } = await import('../src/memory/memory-initializer.js');
- const result = await storeEntry({ key: 'k6', value: 'v6', dbPath });
- expect(result.success).toBe(false);
- const after = require('node:fs').statSync(dbPath).mtimeMs;
- expect(after).toBe(before);
+ const live = openNative();
+ live.pragma('journal_mode = WAL');
+ // Keep the table creation itself inside the uncommitted transaction β€”
+ // if it were a separate, already-committed statement, the liveness
+ // probe's own checkpoint would legitimately flush that committed frame
+ // into the main file (a safe, incremental WAL checkpoint, not a
+ // whole-image write) while still correctly reporting busy for the
+ // uncommitted insert, which would change dbPath's mtime and produce a
+ // false failure here even though nothing unsafe happened.
+ live.exec('BEGIN');
+ live.exec('CREATE TABLE _wal_probe_marker(x)');
+ live.exec('INSERT INTO _wal_probe_marker VALUES (1)');
+
+ try {
+ const before = existsSync(dbPath) ? statSync(dbPath).mtimeMs : 0;
+ const { storeEntry } = await import('../src/memory/memory-initializer.js');
+ const result = await storeEntry({ key: 'k6', value: 'v6', dbPath });
+ expect(result.success).toBe(false);
+ const after = statSync(dbPath).mtimeMs;
+ expect(after).toBe(before);
+ } finally {
+ live.exec('COMMIT');
+ live.close();
+ }
});
});
diff --git a/v3/@claude-flow/cli/__tests__/issue-3161-doctor-leak-recovery.test.ts b/v3/@claude-flow/cli/__tests__/issue-3161-doctor-leak-recovery.test.ts
new file mode 100644
index 0000000000..e6502e85ba
--- /dev/null
+++ b/v3/@claude-flow/cli/__tests__/issue-3161-doctor-leak-recovery.test.ts
@@ -0,0 +1,133 @@
+/**
+ * Regression coverage for issue #3161: `ruflo doctor`'s readonly diagnostic
+ * better-sqlite3 connections against `.swarm/memory.db` (see
+ * `src/commands/doctor.ts`, e.g. the structural `quick_check` probe) call
+ * `.close()` with no checkpoint. WAL mode does not guarantee a checkpoint on
+ * a plain close, so this can leave orphaned, non-empty `-wal`/`-shm`
+ * sidecars on disk β€” and the #2735 guard (`hasNativeWalSidecars` in
+ * `src/memory/memory-initializer.ts`) used to treat sidecar *presence*
+ * alone as proof of a live native holder, permanently refusing every later
+ * `memory store`/`memory list` in the project until someone manually
+ * deleted the two sidecar files. Running the tool users are told to run when
+ * something's wrong (`doctor`) is what caused the lockout. (Empirically the
+ * doctor-leaked `-wal` file is present but zero-byte β€” there is no pending
+ * WAL data to lose, which is exactly why the old presence-only check was
+ * overbroad and the #3161 fix probes liveness instead of presence.)
+ *
+ * The #3161 fix makes the guard probe liveness directly
+ * (`PRAGMA wal_checkpoint(TRUNCATE)`, refusing only when the checkpoint
+ * reports `busy`) instead of trusting sidecar presence. This suite
+ * reproduces the bug's exact mechanism end-to-end: a real doctor-style
+ * readonly open+close that leaks real, non-empty sidecars, then a real
+ * subsequent write through `storeEntry`'s sql.js fallback path β€” confirming
+ * it now self-heals instead of failing permanently.
+ *
+ * The bridge is force-disabled via CLAUDE_FLOW_DISABLE_BRIDGE=1 (same
+ * convention as the #2735/#3397 suites) so the write goes through the
+ * sql.js fallback the guard protects, deterministically.
+ */
+import { afterEach, beforeEach, describe, expect, it } from 'vitest';
+import { existsSync, mkdtempSync, rmSync, statSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import path from 'node:path';
+
+let dir: string;
+let dbPath: string;
+const ORIGINAL_ENV = process.env.CLAUDE_FLOW_DISABLE_BRIDGE;
+
+const walSize = () => (existsSync(`${dbPath}-wal`) ? statSync(`${dbPath}-wal`).size : 0);
+
+beforeEach(async () => {
+ dir = mkdtempSync(path.join(tmpdir(), 'ruflo-3161-'));
+ dbPath = path.join(dir, 'memory.db');
+ process.env.CLAUDE_FLOW_DISABLE_BRIDGE = '1';
+ const { initializeMemoryDatabase } = await import('../src/memory/memory-initializer.js');
+ const initResult = await initializeMemoryDatabase({ dbPath, verbose: false });
+ expect(initResult.success).toBe(true);
+ expect(existsSync(`${dbPath}-wal`)).toBe(false);
+ expect(existsSync(`${dbPath}-shm`)).toBe(false);
+});
+
+afterEach(() => {
+ if (ORIGINAL_ENV === undefined) delete process.env.CLAUDE_FLOW_DISABLE_BRIDGE;
+ else process.env.CLAUDE_FLOW_DISABLE_BRIDGE = ORIGINAL_ENV;
+ rmSync(dir, { recursive: true, force: true });
+});
+
+describe('doctor-leaked WAL sidecar self-heals on next write β€” issue #3161', () => {
+ it('a doctor-style readonly open+close really does leak non-empty sidecars (precondition)', async () => {
+ const { loadBetterSqlite3 } = await import('../src/memory/shared-sqlite.js');
+ const Database = await loadBetterSqlite3();
+
+ // Exact pattern from src/commands/doctor.ts's structural quick_check probe:
+ // readonly, fileMustExist, one pragma query, then a plain close() in a
+ // finally block β€” no checkpoint anywhere in that path.
+ const probe = new Database(dbPath, { readonly: true, fileMustExist: true });
+ try {
+ probe.pragma('quick_check');
+ } finally {
+ try { probe.close(); } catch { /* best-effort, matches doctor.ts */ }
+ }
+
+ expect(existsSync(`${dbPath}-wal`)).toBe(true);
+ });
+
+ it('a subsequent memory store self-heals the leaked sidecars instead of failing permanently', async () => {
+ const { storeEntry } = await import('../src/memory/memory-initializer.js');
+ const { loadBetterSqlite3 } = await import('../src/memory/shared-sqlite.js');
+ const Database = await loadBetterSqlite3();
+
+ // Reproduce the doctor.ts leak.
+ const probe = new Database(dbPath, { readonly: true, fileMustExist: true });
+ try {
+ probe.pragma('quick_check');
+ } finally {
+ try { probe.close(); } catch { /* best-effort, matches doctor.ts */ }
+ }
+ expect(existsSync(`${dbPath}-wal`)).toBe(true);
+
+ // Before #3161: this failed with "active native WAL connection" forever,
+ // until someone manually deleted the sidecar files.
+ const result = await storeEntry({
+ key: 'after-doctor-leak',
+ value: 'survives the leak',
+ dbPath,
+ generateEmbeddingFlag: false,
+ });
+ expect(result.error).toBeUndefined();
+ expect(result.success).toBe(true);
+
+ // Self-healed: the checkpoint that proved liveness ran as a side effect,
+ // truncating the WAL β€” no manual sidecar deletion required.
+ expect(walSize()).toBe(0);
+
+ // And the lockout doesn't recur on the next call either.
+ const again = await storeEntry({
+ key: 'after-doctor-leak-2',
+ value: 'still fine',
+ dbPath,
+ generateEmbeddingFlag: false,
+ });
+ expect(again.success).toBe(true);
+ });
+
+ it('a subsequent memory list also self-heals the leaked sidecars', async () => {
+ const { storeEntry, listEntries } = await import('../src/memory/memory-initializer.js');
+ await storeEntry({ key: 'seed', value: 'v0', dbPath, generateEmbeddingFlag: false });
+
+ const { loadBetterSqlite3 } = await import('../src/memory/shared-sqlite.js');
+ const Database = await loadBetterSqlite3();
+ const probe = new Database(dbPath, { readonly: true, fileMustExist: true });
+ try {
+ probe.pragma('quick_check');
+ } finally {
+ try { probe.close(); } catch { /* best-effort */ }
+ }
+ expect(existsSync(`${dbPath}-wal`)).toBe(true);
+
+ const result = await listEntries({ dbPath });
+ expect(result.error).toBeUndefined();
+ expect(result.success).toBe(true);
+ expect(walSize()).toBe(0);
+ });
+});
diff --git a/v3/@claude-flow/cli/__tests__/issue-3161-wal-liveness-probe.test.ts b/v3/@claude-flow/cli/__tests__/issue-3161-wal-liveness-probe.test.ts
new file mode 100644
index 0000000000..3da9442ab4
--- /dev/null
+++ b/v3/@claude-flow/cli/__tests__/issue-3161-wal-liveness-probe.test.ts
@@ -0,0 +1,195 @@
+/**
+ * Issue #3161 β€” liveness-probe behavior of `hasNativeWalSidecars`
+ * (`src/memory/memory-initializer.ts`). The #2735 guard used to treat
+ * `-wal`/`-shm` sidecar *presence* as proof of a live native writer; #3161
+ * found that's false (a doctor-style readonly open+close leaks them too,
+ * see `issue-3161-doctor-leak-recovery.test.ts`) and replaced it with a
+ * direct `PRAGMA wal_checkpoint(TRUNCATE)` liveness probe: refuse only when
+ * the checkpoint reports `busy`.
+ *
+ * This file covers what that change must still get right beyond the
+ * doctor-leak repro itself:
+ * - the safety property #2735 existed for β€” a GENUINE live writer (mid
+ * transaction, pending WAL frames) must still block the sql.js write;
+ * - fail-closed behavior on a probe error (corrupted sidecar, not merely
+ * absent);
+ * - a nonexistent dbPath still can't crash the guard's callers;
+ * - the no-sidecars fast path never opens a database connection at all.
+ *
+ * The bridge is force-disabled via CLAUDE_FLOW_DISABLE_BRIDGE=1, same
+ * convention as the #2735/#3397/#3161 suites, so every case exercises the
+ * sql.js fallback path the guard protects.
+ */
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
+import { createRequire } from 'node:module';
+import { tmpdir } from 'node:os';
+import path from 'node:path';
+
+let dir: string;
+let dbPath: string;
+const ORIGINAL_ENV = process.env.CLAUDE_FLOW_DISABLE_BRIDGE;
+
+beforeEach(async () => {
+ dir = mkdtempSync(path.join(tmpdir(), 'ruflo-3161-liveness-'));
+ dbPath = path.join(dir, 'memory.db');
+ process.env.CLAUDE_FLOW_DISABLE_BRIDGE = '1';
+});
+
+afterEach(() => {
+ vi.doUnmock('better-sqlite3');
+ vi.resetModules();
+ if (ORIGINAL_ENV === undefined) delete process.env.CLAUDE_FLOW_DISABLE_BRIDGE;
+ else process.env.CLAUDE_FLOW_DISABLE_BRIDGE = ORIGINAL_ENV;
+ rmSync(dir, { recursive: true, force: true });
+});
+
+describe('#3161 β€” genuine live writer is still refused (the #2735 property)', () => {
+ it('refuses the sql.js write while a foreign connection holds an uncommitted transaction', async () => {
+ const { initializeMemoryDatabase, storeEntry } = await import('../src/memory/memory-initializer.js');
+ expect((await initializeMemoryDatabase({ dbPath, verbose: false })).success).toBe(true);
+
+ const Database = createRequire(import.meta.url)('better-sqlite3');
+ const foreign = new Database(dbPath);
+ try {
+ foreign.pragma('journal_mode = WAL');
+ // A genuine, still-open write transaction β€” pending WAL frames that
+ // wal_checkpoint(TRUNCATE) cannot flush while it's held.
+ foreign.exec('BEGIN IMMEDIATE');
+ foreign.prepare(
+ "INSERT INTO memory_entries (id, key, namespace, content) VALUES ('live-1', 'live-1', 'default', 'x')",
+ ).run();
+ expect(existsSync(`${dbPath}-wal`)).toBe(true);
+
+ const result = await storeEntry({ key: 'blocked', value: 'v', dbPath, generateEmbeddingFlag: false });
+ expect(result.success).toBe(false);
+ expect(result.error).toMatch(/native WAL connection/i);
+ } finally {
+ try { foreign.exec('ROLLBACK'); } catch { /* best-effort cleanup */ }
+ foreign.close();
+ }
+ });
+
+ it('allows the write again once the foreign writer commits and closes', async () => {
+ const { initializeMemoryDatabase, storeEntry } = await import('../src/memory/memory-initializer.js');
+ expect((await initializeMemoryDatabase({ dbPath, verbose: false })).success).toBe(true);
+
+ const Database = createRequire(import.meta.url)('better-sqlite3');
+ const foreign = new Database(dbPath);
+ foreign.pragma('journal_mode = WAL');
+ foreign.exec('BEGIN IMMEDIATE');
+ foreign.prepare(
+ "INSERT INTO memory_entries (id, key, namespace, content) VALUES ('live-2', 'live-2', 'default', 'x')",
+ ).run();
+
+ const blocked = await storeEntry({ key: 'still-blocked', value: 'v', dbPath, generateEmbeddingFlag: false });
+ expect(blocked.success).toBe(false);
+
+ foreign.exec('COMMIT');
+ foreign.close();
+
+ const result = await storeEntry({ key: 'now-allowed', value: 'v', dbPath, generateEmbeddingFlag: false });
+ expect(result.success).toBe(true);
+ });
+
+ it('two concurrent sql.js callers racing the guard both correctly refuse while the real holder is attached', async () => {
+ const { initializeMemoryDatabase, storeEntry } = await import('../src/memory/memory-initializer.js');
+ expect((await initializeMemoryDatabase({ dbPath, verbose: false })).success).toBe(true);
+
+ const Database = createRequire(import.meta.url)('better-sqlite3');
+ const foreign = new Database(dbPath);
+ foreign.pragma('journal_mode = WAL');
+ foreign.exec('BEGIN IMMEDIATE');
+ foreign.prepare(
+ "INSERT INTO memory_entries (id, key, namespace, content) VALUES ('live-3', 'live-3', 'default', 'x')",
+ ).run();
+
+ try {
+ // Two callers race the guard concurrently while the real holder is
+ // still attached β€” the real holder's uncommitted write must keep
+ // winning and both checkers must independently refuse, not just
+ // whichever ran first.
+ const [a, b] = await Promise.all([
+ storeEntry({ key: 'racer-a', value: 'v', dbPath, generateEmbeddingFlag: false }),
+ storeEntry({ key: 'racer-b', value: 'v', dbPath, generateEmbeddingFlag: false }),
+ ]);
+ expect(a.success).toBe(false);
+ expect(b.success).toBe(false);
+ } finally {
+ try { foreign.exec('ROLLBACK'); } catch { /* best-effort cleanup */ }
+ foreign.close();
+ }
+
+ // Once the real holder is gone, a racer-style retry succeeds normally.
+ const after = await storeEntry({ key: 'racer-after', value: 'v', dbPath, generateEmbeddingFlag: false });
+ expect(after.success).toBe(true);
+ });
+});
+
+describe('#3161 β€” fail-closed on probe error (corrupted sidecar, not merely absent)', () => {
+ it('garbage (but structurally valid-length) bytes in -wal are safely ignored by SQLite\'s own WAL validation, not a probe error', async () => {
+ // Documents actual behavior so it isn't mistaken for a regression: SQLite
+ // validates the WAL header/frame checksums itself. Bytes that fail that
+ // validation are treated as "no real pending WAL data", not as an error β€”
+ // wal_checkpoint succeeds (busy: 0) and the write proceeds. This is
+ // correct: a WAL file whose own integrity check fails has nothing
+ // recoverable to lose.
+ const { initializeMemoryDatabase, storeEntry } = await import('../src/memory/memory-initializer.js');
+ expect((await initializeMemoryDatabase({ dbPath, verbose: false })).success).toBe(true);
+ writeFileSync(`${dbPath}-wal`, Buffer.from([0xde, 0xad, 0xbe, 0xef, 0x00, 0x01, 0x02, 0x03]));
+
+ const result = await storeEntry({ key: 'corrupt-wal', value: 'v', dbPath, generateEmbeddingFlag: false });
+ expect(result.success).toBe(true);
+ });
+
+ it('refuses the write when a sidecar path cannot be opened at all (probe genuinely errors)', async () => {
+ const { initializeMemoryDatabase, storeEntry } = await import('../src/memory/memory-initializer.js');
+ expect((await initializeMemoryDatabase({ dbPath, verbose: false })).success).toBe(true);
+
+ // A directory where SQLite needs to open/create a file β€” the probe
+ // (opening the main db, which touches its -wal companion) must fail
+ // closed rather than crash or silently proceed.
+ mkdirSync(`${dbPath}-wal`);
+
+ const result = await storeEntry({ key: 'unopenable-wal', value: 'v', dbPath, generateEmbeddingFlag: false });
+ expect(result.success).toBe(false);
+ expect(result.error).toMatch(/native WAL connection/i);
+ });
+});
+
+describe('#3161 β€” a nonexistent dbPath cannot crash the guard\'s callers', () => {
+ it('storeEntry reports a clean error instead of throwing when dbPath does not exist', async () => {
+ const { storeEntry } = await import('../src/memory/memory-initializer.js');
+ const missing = path.join(dir, 'never-initialized.db');
+ await expect(storeEntry({ key: 'k', value: 'v', dbPath: missing, generateEmbeddingFlag: false }))
+ .resolves.toMatchObject({ success: false });
+ });
+
+ it('getEntry reports a clean error instead of throwing when dbPath does not exist', async () => {
+ const { getEntry } = await import('../src/memory/memory-initializer.js');
+ const missing = path.join(dir, 'never-initialized.db');
+ await expect(getEntry({ key: 'k', dbPath: missing })).resolves.toMatchObject({ success: false, found: false });
+ });
+});
+
+describe('#3161 β€” no-sidecars fast path never opens a database connection', () => {
+ it('storeEntry succeeds with no sidecars even if constructing a native connection would throw', async () => {
+ vi.doMock('better-sqlite3', () => ({
+ default: class {
+ constructor() {
+ throw new Error('fast path must not reach here β€” no sidecars means no probe connection');
+ }
+ },
+ }));
+ vi.resetModules();
+
+ const { initializeMemoryDatabase, storeEntry } = await import('../src/memory/memory-initializer.js');
+ expect((await initializeMemoryDatabase({ dbPath, verbose: false })).success).toBe(true);
+ expect(existsSync(`${dbPath}-wal`)).toBe(false);
+ expect(existsSync(`${dbPath}-shm`)).toBe(false);
+
+ const result = await storeEntry({ key: 'fast-path', value: 'v', dbPath, generateEmbeddingFlag: false });
+ expect(result.error).toBeUndefined();
+ expect(result.success).toBe(true);
+ });
+});
diff --git a/v3/@claude-flow/cli/__tests__/memory-cli-mirror-removal.test.ts b/v3/@claude-flow/cli/__tests__/memory-cli-mirror-removal.test.ts
index c52f560a2b..03cdccf329 100644
--- a/v3/@claude-flow/cli/__tests__/memory-cli-mirror-removal.test.ts
+++ b/v3/@claude-flow/cli/__tests__/memory-cli-mirror-removal.test.ts
@@ -1,5 +1,6 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { mkdtempSync, rmSync, writeFileSync, readFileSync } from 'node:fs';
+import { createRequire } from 'node:module';
import { join } from 'node:path';
import { tmpdir } from 'node:os';
@@ -71,20 +72,56 @@ describe('CLI removal of default memory and its AgentDB mirror', () => {
} finally { db.close(); }
});
- it.each(['delete', 'purge', 'preview'])('%s refuses to rewrite a mirror with native WAL sidecars', async command => {
+ // The guard is liveness-based (#3161), not presence-based: a hand-touched
+ // `-wal` file next to a non-WAL-mode db is inert under the probe and no
+ // longer proves anything, so these simulate a genuinely live native
+ // holder instead β€” a real WAL-mode connection with an open, uncommitted
+ // transaction on the mirror. `setup` first stabilizes the mirror in real
+ // WAL mode with a committed baseline (closed cleanly, so no sidecars
+ // remain) so enabling WAL mode itself doesn't show up as an unrelated
+ // byte diff in the `before`/`after` comparison below.
+ it.each(['delete', 'purge', 'preview'])('%s refuses to rewrite a mirror while a native WAL writer holds an open transaction', async command => {
+ const Database = createRequire(import.meta.url)('better-sqlite3');
+ const setup = new Database(mirror);
+ setup.pragma('journal_mode = WAL');
+ setup.exec('CREATE TABLE IF NOT EXISTS _wal_probe_marker(x)');
+ setup.close();
+
const before = readFileSync(mirror);
- writeFileSync(`${mirror}-wal`, 'owned by another native process');
- expect(await invoke(command === 'preview' ? 'purge' : command, { dryRun: command === 'preview' }))
- .toMatchObject({ success: false, exitCode: 1 });
- expect(readFileSync(mirror)).toEqual(before);
+ const live = new Database(mirror);
+ live.pragma('journal_mode = WAL');
+ live.exec('BEGIN');
+ live.exec('INSERT INTO _wal_probe_marker VALUES (1)');
+ try {
+ expect(await invoke(command === 'preview' ? 'purge' : command, { dryRun: command === 'preview' }))
+ .toMatchObject({ success: false, exitCode: 1 });
+ expect(readFileSync(mirror)).toEqual(before);
+ } finally {
+ live.exec('ROLLBACK');
+ live.close();
+ }
});
- it('refuses native WAL sidecars in the purge fallback after preview', async () => {
+ it('refuses a native WAL writer with an open transaction in the purge fallback after preview', async () => {
+ const Database = createRequire(import.meta.url)('better-sqlite3');
+ const setup = new Database(mirror);
+ setup.pragma('journal_mode = WAL');
+ setup.exec('CREATE TABLE IF NOT EXISTS _wal_probe_marker(x)');
+ setup.close();
+
const before = readFileSync(mirror);
- writeFileSync(`${mirror}-wal`, 'native writer acquired after preview');
- expect(await purgeNamespace({ namespace: 'scratch', dbPath: mirror, encryptWrites: false }))
- .toMatchObject({ success: false });
- expect(readFileSync(mirror)).toEqual(before);
+ const live = new Database(mirror);
+ live.pragma('journal_mode = WAL');
+ live.exec('BEGIN');
+ live.exec('INSERT INTO _wal_probe_marker VALUES (1)');
+ try {
+ expect(await purgeNamespace({ namespace: 'scratch', dbPath: mirror, encryptWrites: false }))
+ .toMatchObject({ success: false });
+ expect(readFileSync(mirror)).toEqual(before);
+ } finally {
+ live.exec('ROLLBACK');
+ live.close();
+ }
});
it('refuses an unconfirmed purge without changing either store', async () => {
diff --git a/v3/@claude-flow/cli/src/memory/memory-initializer.ts b/v3/@claude-flow/cli/src/memory/memory-initializer.ts
index 72584961bc..8afc75f408 100644
--- a/v3/@claude-flow/cli/src/memory/memory-initializer.ts
+++ b/v3/@claude-flow/cli/src/memory/memory-initializer.ts
@@ -63,25 +63,85 @@ function isRuvectorCoreResolvable(): boolean {
}
/**
- * #2735 β€” before a whole-image sql.js read-modify-persist (export() +
- * rename over the live database path), refuse if there is evidence of a
- * live native (better-sqlite3) WAL connection: `-wal`/`-shm` sidecar files
- * on disk. A native connection in WAL mode keeps its sidecars present for
- * its entire lifetime (removed only on the last connection's clean close),
- * so their presence is strong evidence of a live native holder β€” and their
- * absence means the image is a clean, checkpointed, standalone file that
- * sql.js can safely read-modify-write.
+ * #2735 / #3161 β€” before a whole-image sql.js read-modify-persist (export() +
+ * rename over the live database path), refuse if a native (better-sqlite3)
+ * WAL connection is actually LIVE. `-wal`/`-shm` sidecar presence alone is
+ * not proof of that: #3161 found `ruflo doctor`'s readonly diagnostic
+ * connections leave orphaned sidecars behind after a plain `.close()` (WAL
+ * mode does not checkpoint on close unless the closing connection was the
+ * last holder in a state that triggers one), permanently blocking every
+ * later write in that project.
*
- * This is a scoped-down version of the fuller "scan live process holders"
- * design discussed in #2735: it does not close the narrow assess-then-write
- * race (a native opener could still attach in the gap between this check
- * and the write), but it directly closes the demonstrated corruption
- * mechanism β€” a whole-image write proceeding while an ALREADY-OPEN native
- * connection's sidecars are on disk β€” with no platform-specific process
- * scanning. Fails closed (treats a stat error as "unsafe") because this is
- * a safety gate, not a best-effort probe.
+ * Liveness is probed in two steps, because a single `wal_checkpoint`
+ * busy-check is not sufficient on its own:
+ *
+ * 1. Open our own probe connection and run `PRAGMA wal_checkpoint(TRUNCATE)`.
+ * `busy !== 0` means another connection genuinely holds a lock blocking
+ * the checkpoint RIGHT NOW (an open transaction) β€” unsafe, refuse
+ * immediately. This also truncates a genuinely-orphaned WAL as a side
+ * effect.
+ * 2. If `busy === 0`, close our probe connection and re-check whether the
+ * sidecars are still on disk. SQLite only deletes `-wal`/`-shm` when the
+ * LAST connection to the database closes. If nothing else is attached,
+ * OUR close is that last close and the sidecars vanish β€” safe, proceed.
+ * If they are still present after our own close, some OTHER connection
+ * is still attached (even idle, holding no lock β€” e.g. a foreign native
+ * handle that already finished its last statement), so step 1's
+ * busy-check alone would have missed it. Refuse.
+ *
+ * Step 2 is what makes this safe against an idle-but-attached native
+ * connection (an open handle holding no lock does not block checkpointing,
+ * so step 1 alone would report "safe" even though a real holder is still
+ * attached) without resorting to platform-specific process scanning.
+ *
+ * The common case (no sidecars at all) stays a couple of cheap `existsSync`
+ * calls β€” a DB connection is only opened when a sidecar is actually present
+ * to investigate.
+ *
+ * This is still a scoped-down version of the fuller "scan live process
+ * holders" design discussed in #2735: it does not close the narrow
+ * assess-then-write race (a native opener could still attach in the gap
+ * between this check and the write), but it directly closes the
+ * demonstrated corruption mechanism β€” a whole-image write proceeding while
+ * an ALREADY-OPEN native connection genuinely holds the file β€” with no
+ * platform-specific process scanning. Fails closed (treats any probe error
+ * as "unsafe") because this is a safety gate, not a best-effort probe.
*/
-function hasNativeWalSidecars(dbPath: string): boolean {
+async function hasNativeWalSidecars(dbPath: string): Promise<boolean> {
+ try {
+ if (!fs.existsSync(`${dbPath}-wal`) && !fs.existsSync(`${dbPath}-shm`)) {
+ return false;
+ }
+ } catch {
+ return true;
+ }
+
+ let db: { pragma(sql: string): unknown; close(): void } | undefined;
+ try {
+ const Database = await loadBetterSqlite3();
+ const opened = new Database(dbPath);
+ db = opened;
+ // Report busy immediately instead of retrying for the driver's default
+ // (multi-second) busy_timeout β€” a live holder's checkpoint-blocking
+ // lock is a steady-state fact at probe time, not a transient we should
+ // wait out, and this is a safety gate on a hot write path.
+ opened.pragma('busy_timeout = 0');
+ const [{ busy }] = opened.pragma('wal_checkpoint(TRUNCATE)') as Array<{ busy: number }>;
+ if (busy !== 0) return true;
+ } catch {
+ return true;
+ } finally {
+ try {
+ db?.close();
+ } catch {
+ // best-effort
+ }
+ }
+
+ // Nothing was mid-transaction at probe time, and our own (now-closed)
+ // probe connection just truncated the WAL. If the sidecars are still
+ // here, another connection is still attached and kept them alive through
+ // our close.
try {
return fs.existsSync(`${dbPath}-wal`) || fs.existsSync(`${dbPath}-shm`);
} catch {
@@ -2966,7 +3026,7 @@ export async function storeEntry(options: {
// race). This check gates ensureSchemaColumns()'s own whole-image
// write below too, not just this function's.
await releaseOwnNativeHandle(dbPath);
- if (hasNativeWalSidecars(dbPath)) {
+ if (await hasNativeWalSidecars(dbPath)) {
return {
success: false,
id: '',
@@ -3487,7 +3547,7 @@ export async function listEntries(options: {
// Listing can migrate/backfill the schema, so it is also a whole-image
// writer. The newly selected native mirror may still have a live WAL.
await releaseOwnNativeHandle(dbPath);
- if (hasNativeWalSidecars(dbPath)) {
+ if (await hasNativeWalSidecars(dbPath)) {
return { success: false, entries: [], total: 0, error: await walRefusalError('read/write') };
}
@@ -3659,7 +3719,7 @@ export async function getEntry(options: {
// bump is itself a whole-image write, not a lightweight read, even
// though this function's contract reads as a "get".
await releaseOwnNativeHandle(dbPath);
- if (hasNativeWalSidecars(dbPath)) {
+ if (await hasNativeWalSidecars(dbPath)) {
return {
success: false,
found: false,
@@ -3812,7 +3872,7 @@ export async function deleteEntry(options: {
// #2735 β€” see storeEntry's identical gate for the corruption mechanism
// this closes.
await releaseOwnNativeHandle(dbPath);
- if (hasNativeWalSidecars(dbPath)) {
+ if (await hasNativeWalSidecars(dbPath)) {
return {
success: false,
deleted: false,
@@ -4056,7 +4116,7 @@ export async function purgeNamespace(options: {
// Recheck at mutation time even when the CLI already read a preview.
await releaseOwnNativeHandle(dbPath);
- if (hasNativeWalSidecars(dbPath)) {
+ if (await hasNativeWalSidecars(dbPath)) {
return { success: false, deletedCount: 0, remainingEntries: 0, error: await walRefusalError('write') };
}