Skip to content

armgit / ruflo

Public Imported from GitHub
0

🌊 The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated

fix(policy): authenticate retained ledgers and stream full audits with fenced writers (#4216)

Dragan Spiridonovf18ff4c427f2
diff --git a/scripts/audit-env-var-precedence.mjs b/scripts/audit-env-var-precedence.mjs
index b5fefce01e..93b9198be9 100644
--- a/scripts/audit-env-var-precedence.mjs
+++ b/scripts/audit-env-var-precedence.mjs
@@ -124,6 +124,8 @@ const KNOWN_ESCAPE_HATCHES = new Set([
'CLAUDE_FLOW_FORCE_UPDATE', // Set by --force flag internally, then cleared β€” not external
'CLAUDE_FLOW_AUTO_UPDATE', // Auto-update cadence β€” env-only documented design
'RUFLO_DAEMON_LOCK_WAIT_MS', // How long `daemon start` waits on another start's spawn lock; operator tuning knob, env-only by design (no `daemon start` flag)
+ 'CLAUDE_FLOW_POLICY_LEDGER_RETENTION', // Operator policy-writer retention window, shared by CLI and long-lived MCP transactions; no per-request flag owns the writer lifecycle. Pruned receipts remain archived and authenticated (#3164/#3892).
+ 'CLAUDE_FLOW_POLICY_AUDIT_MAX_RECORD_BYTES', // Operator memory-safety bound shared by archive preflight and full-audit replay, including MCP calls. Intentionally process-wide, not caller-overridable; validated 1 KiB–64 MiB with an unchanged 16 MiB default (#3892).
// ── Logging / diagnostics ───────────────────────────────────────────────────
'CLAUDE_FLOW_LOG_LEVEL',
diff --git a/v3/@claude-flow/cli/__tests__/helpers/isolated-policy-trust.ts b/v3/@claude-flow/cli/__tests__/helpers/isolated-policy-trust.ts
new file mode 100644
index 0000000000..0d74affac7
--- /dev/null
+++ b/v3/@claude-flow/cli/__tests__/helpers/isolated-policy-trust.ts
@@ -0,0 +1,16 @@
+import { afterAll, beforeEach } from 'vitest';
+import { mkdirSync, mkdtempSync, rmSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+
+const previous = process.env.XDG_CONFIG_HOME;
+const config = mkdtempSync(join(tmpdir(), 'ruflo-policy-test-config-'));
+// policyTrustRoot preserves an existing legacy root unless the preferred
+// XDG root exists. Never fall back to the developer's account home.
+mkdirSync(join(config, 'ruflo', 'policy-trust'), { recursive: true });
+beforeEach(() => { process.env.XDG_CONFIG_HOME = config; });
+afterAll(() => {
+ if (previous === undefined) delete process.env.XDG_CONFIG_HOME;
+ else process.env.XDG_CONFIG_HOME = previous;
+ rmSync(config, { recursive: true, force: true });
+});
diff --git a/v3/@claude-flow/cli/__tests__/policy-ledger-anchor-3602.test.ts b/v3/@claude-flow/cli/__tests__/policy-ledger-anchor-3602.test.ts
index beb41f05dc..f0480a2d40 100644
--- a/v3/@claude-flow/cli/__tests__/policy-ledger-anchor-3602.test.ts
+++ b/v3/@claude-flow/cli/__tests__/policy-ledger-anchor-3602.test.ts
@@ -1,3 +1,5 @@
+import './helpers/isolated-policy-trust.js';
+import { policyTrustRoot } from '../src/services/policy-ledger-anchor.js';
import { afterEach, describe, expect, it } from 'vitest';
import { existsSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
@@ -19,7 +21,7 @@ function project(): string {
const root = mkdtempSync(join(tmpdir(), 'ruflo-policy-3602-'));
mkdirSync(join(root, '.claude-flow'), { recursive: true });
const id = createHash('sha256').update(realpathSync(root)).digest('hex');
- roots.push({ root, trust: join(userInfo().homedir, '.config', 'ruflo', 'policy-trust', id) });
+ roots.push({ root, trust: join(policyTrustRoot(), id) });
return root;
}
afterEach(() => {
@@ -35,7 +37,7 @@ const statePath = (root: string) => join(root, '.claude-flow', 'policy', 'state.
const logPath = (root: string) => join(root, '.claude-flow', 'policy', 'ledger-anchors.json');
const mirrorPath = (root: string) => {
const id = createHash('sha256').update(realpathSync(root)).digest('hex');
- return join(userInfo().homedir, '.config', 'ruflo', 'policy-trust', id, 'ledger-anchor-head.json');
+ return join(policyTrustRoot(), id, 'ledger-anchor-head.json');
};
const readState = (root: string) => JSON.parse(readFileSync(statePath(root), 'utf8'));
const writeState = (root: string, state: unknown) => writeFileSync(statePath(root), JSON.stringify(state, null, 2));
diff --git a/v3/@claude-flow/cli/__tests__/policy-ledger-archive-anchor-3892-review.test.ts b/v3/@claude-flow/cli/__tests__/policy-ledger-archive-anchor-3892-review.test.ts
new file mode 100644
index 0000000000..8c0b3696e9
--- /dev/null
+++ b/v3/@claude-flow/cli/__tests__/policy-ledger-archive-anchor-3892-review.test.ts
@@ -0,0 +1,474 @@
+import './helpers/isolated-policy-trust.js';
+import { policyTrustRoot } from '../src/services/policy-ledger-anchor.js';
+import { afterEach, describe, expect, it } from 'vitest';
+import { spawn } from 'node:child_process';
+import {
+ appendFileSync,
+ existsSync,
+ mkdirSync,
+ mkdtempSync,
+ readFileSync,
+ realpathSync,
+ rmSync,
+ truncateSync,
+ writeFileSync,
+} from 'node:fs';
+import { join } from 'node:path';
+import { tmpdir } from 'node:os';
+import { createHash } from 'node:crypto';
+import { AgenticPolicyEngine } from '@claude-flow/security';
+import {
+ autoMigratePolicyStateIfNeeded,
+ evaluatePolicyRequest,
+ verifyPolicyLedger,
+} from '../src/services/policy-runtime.js';
+import { anchorLogPath } from '../src/services/policy-ledger-anchor.js';
+
+// PR #3892 maintainer review (ruvnet nightly-triage, 2026-10-11): three
+// confirmed blockers on top of #3164 (ledger retention/archive) + #3602
+// (external anchor). This file closes all three:
+// 1. Default (non-`--full`) verify never looked at the archive at all,
+// so pruning + archive tampering went undetected unless `--full` was
+// explicitly requested.
+// 2. `dedupeArchiveReceipts` only compared `hash` strings, never
+// recomputed a receipt's hash from its own content.
+// 3. `--full` held the policy-state lock for the entire O(total-history)
+// replay, reintroducing the lock contention #3164 fixed on the hot
+// path.
+
+const roots: Array<{ root: string; trust: string }> = [];
+function project(): string {
+ const root = mkdtempSync(join(tmpdir(), 'ruflo-policy-3892-'));
+ mkdirSync(join(root, '.claude-flow'), { recursive: true });
+ const id = createHash('sha256').update(realpathSync(root)).digest('hex');
+ roots.push({ root, trust: join(policyTrustRoot(), id) });
+ return root;
+}
+afterEach(() => {
+ for (const item of roots.splice(0)) {
+ rmSync(item.trust, { recursive: true, force: true });
+ rmSync(item.root, { recursive: true, force: true });
+ }
+ delete process.env.CLAUDE_FLOW_POLICY_LEDGER_RETENTION;
+});
+
+const statePath = (root: string) => join(root, '.claude-flow', 'policy', 'state.json');
+const archivePath = (root: string) => join(root, '.claude-flow', 'policy', 'receipts.ledger.jsonl');
+const lockPath = (root: string) => join(root, '.claude-flow', 'policy', 'state.lock');
+const mirrorPath = (root: string) => {
+ const id = createHash('sha256').update(realpathSync(root)).digest('hex');
+ return join(policyTrustRoot(), id, 'ledger-anchor-head.json');
+};
+const readState = (root: string) => JSON.parse(readFileSync(statePath(root), 'utf8'));
+const writeState = (root: string, state: unknown) => writeFileSync(statePath(root), JSON.stringify(state, null, 2));
+const readLog = (root: string) => JSON.parse(readFileSync(anchorLogPath(root), 'utf8'));
+const writeLog = (root: string, log: unknown) => writeFileSync(anchorLogPath(root), JSON.stringify(log, null, 2));
+const readMirror = (root: string) => JSON.parse(readFileSync(mirrorPath(root), 'utf8'));
+const writeMirror = (root: string, entry: unknown) => writeFileSync(mirrorPath(root), JSON.stringify(entry, null, 2));
+const readArchiveLines = (root: string): string[] => readFileSync(archivePath(root), 'utf8')
+ .split('\n')
+ .filter((line) => line.trim().length > 0);
+
+async function decide(root: string, i: number) {
+ return evaluatePolicyRequest({
+ identity: { id: `agent:${i}`, type: 'agent' },
+ action: { type: 'code.read', resource: `file-${i}` },
+ }, root);
+}
+
+async function withRetention<T>(value: string, fn: () => Promise<T>): Promise<T> {
+ const previous = process.env.CLAUDE_FLOW_POLICY_LEDGER_RETENTION;
+ process.env.CLAUDE_FLOW_POLICY_LEDGER_RETENTION = value;
+ try {
+ return await fn();
+ } finally {
+ if (previous === undefined) delete process.env.CLAUDE_FLOW_POLICY_LEDGER_RETENTION;
+ else process.env.CLAUDE_FLOW_POLICY_LEDGER_RETENTION = previous;
+ }
+}
+
+async function prunedLedger(retain: string, count: number): Promise<string> {
+ const root = project();
+ await autoMigratePolicyStateIfNeeded(root);
+ await withRetention(retain, async () => {
+ for (let i = 0; i < count; i++) await decide(root, i);
+ });
+ return root;
+}
+
+/**
+ * Builds a large pruned ledger in-memory (bypassing the per-transaction
+ * file-locking path, which would make a 20k-receipt fixture far too slow
+ * for a test) so the --full replay has measurable, non-trivial cost for
+ * the Blocker 3 cross-process lock test.
+ */
+async function bigPrunedLedger(total: number, retain: number): Promise<string> {
+ const root = project();
+ mkdirSync(join(root, '.claude-flow', 'policy'), { recursive: true });
+ const engine = new AgenticPolicyEngine({});
+ for (let i = 0; i < total; i++) {
+ engine.evaluate({ identity: { id: `agent:${i}`, type: 'agent' }, action: { type: 'code.read', resource: `f-${i}` } });
+ }
+ engine.verifyLedger({ establishAnchor: true });
+ const pruned = engine.pruneReceipts(retain);
+ writeState(root, engine.exportState());
+ writeFileSync(archivePath(root), `${pruned.map((receipt) => JSON.stringify(receipt)).join('\n')}\n`);
+ // One normal call establishes the #3602 second anchor (log + mirror) to
+ // match this hand-built state.
+ expect((await verifyPolicyLedger(root)).valid).toBe(true);
+ return root;
+}
+
+// #3892: a hash computed the same way `entryHash` computes it for an entry
+// that has no archive fields at all β€” i.e. exactly what pre-this-fix code
+// wrote. Used to hand-craft a "transitional" ledger without depending on
+// code that no longer exists.
+function oldFormatHash(entry: {
+ seq: number; length: number; headHash: string | null; prevAnchorHash: string | null;
+ ts: number; event: string; by?: string;
+}): string {
+ return createHash('sha256').update(JSON.stringify([
+ entry.seq, entry.length, entry.headHash, entry.prevAnchorHash, entry.ts, entry.event, entry.by ?? null,
+ ])).digest('hex');
+}
+
+function stripArchiveFields(entry: Record<string, unknown>): Record<string, unknown> {
+ const { archivedThrough, archiveHeadHash, hash, ...rest } = entry;
+ return { ...rest, hash: oldFormatHash(rest as never) };
+}
+
+describe('#3892 Blocker 1: default verify must check the archive\'s tip, not just the hot tail', () => {
+ it('reproduction: deleting the archive is caught by default (non-full) verify', async () => {
+ const root = await prunedLedger('3', 10);
+ expect((await verifyPolicyLedger(root)).valid).toBe(true);
+ rmSync(archivePath(root));
+ const result = await verifyPolicyLedger(root);
+ expect(result).toMatchObject({ valid: false, error: 'policy-ledger-archive-tampered' });
+ });
+
+ it('reproduction: emptying the archive is caught by default verify', async () => {
+ const root = await prunedLedger('3', 10);
+ truncateSync(archivePath(root), 0);
+ const result = await verifyPolicyLedger(root);
+ expect(result).toMatchObject({ valid: false, error: 'policy-ledger-archive-tampered' });
+ });
+
+ it('reproduction: truncating (dropping the trailing line of) the archive is caught by default verify', async () => {
+ const root = await prunedLedger('3', 10);
+ const lines = readArchiveLines(root);
+ lines.pop();
+ writeFileSync(archivePath(root), `${lines.join('\n')}\n`);
+ const result = await verifyPolicyLedger(root);
+ expect(result).toMatchObject({ valid: false, error: 'policy-ledger-archive-tampered' });
+ });
+
+ it('reproduction: editing decision.outcome in the archive\'s last receipt is caught by default verify', async () => {
+ const root = await prunedLedger('3', 10);
+ const lines = readArchiveLines(root);
+ const tampered = JSON.parse(lines[lines.length - 1]!);
+ tampered.payload.decision.outcome = 'denied';
+ // The stale `hash` field is deliberately left untouched β€” the naive
+ // tamper a reviewer would actually write by hand.
+ lines[lines.length - 1] = JSON.stringify(tampered);
+ writeFileSync(archivePath(root), `${lines.join('\n')}\n`);
+ const result = await verifyPolicyLedger(root);
+ expect(result).toMatchObject({ valid: false, error: 'policy-ledger-archive-tampered' });
+ });
+
+ it('the fast hot-tail checks are unaffected: truncating the archive does not flag a healthy hot tail as truncated', async () => {
+ const root = await prunedLedger('3', 10);
+ rmSync(archivePath(root));
+ const result = await verifyPolicyLedger(root);
+ expect(result.error).toBe('policy-ledger-archive-tampered');
+ expect(result.error).not.toBe('policy-ledger-truncated');
+ });
+
+ it('KNOWN, DISCLOSED coverage boundary (tester-6): a self-consistent INTERIOR edit that leaves the tip untouched is NOT caught by default verify, and IS caught by --full', async () => {
+ // This is not a bug to fix β€” it is the inherent trade-off of a cheap
+ // tip check over an O(total-history) replay (see the file-level and
+ // archiveHasTailReceipt doc comments in policy-ledger-anchor.ts). This
+ // test pins the boundary so it can't silently widen OR silently
+ // "improve" without someone noticing the hot-path cost implications.
+ const root = await prunedLedger('3', 10);
+ const lines = readArchiveLines(root);
+
+ // Tamper an interior receipt (sequence 2, not the tip at sequence 6)
+ // and recompute ONLY its own hash, so it is self-consistent in
+ // isolation. Its `previousReceiptHash`/chain-forward relationship to
+ // later receipts is now broken, but nothing downstream re-checks that
+ // on the default path β€” only a full replay walks the chain.
+ const tampered = JSON.parse(lines[2]!);
+ tampered.payload.decision.reason = 'surgically-tampered-interior-receipt';
+ const { policyHash } = await import('@claude-flow/security');
+ tampered.hash = policyHash(tampered.payload);
+ lines[2] = JSON.stringify(tampered);
+ writeFileSync(archivePath(root), `${lines.join('\n')}\n`);
+
+ // Default verify: the tip (sequence 6) is untouched and still matches
+ // the anchor, so this reports valid β€” the disclosed gap.
+ expect(await verifyPolicyLedger(root)).toEqual({ valid: true, length: 10 });
+
+ // --full replays the whole chain and catches the broken link.
+ const full = await verifyPolicyLedger(root, { full: true });
+ expect(full.valid).toBe(false);
+ });
+});
+
+describe('#3892 Blocker 1: backward compatibility with pre-archive-anchoring ledgers', () => {
+ it('a ledger pruned by the old anchor shape (no archivedThrough) is not failed outright, and gets caught up', async () => {
+ const root = await prunedLedger('3', 10);
+
+ // Simulate a ledger pruned before this fix existed: strip the archive
+ // fields from both the log's last entry and the mirror, recomputing
+ // their hash the way the old code did.
+ const log = readLog(root);
+ const lastIndex = log.entries.length - 1;
+ log.entries[lastIndex] = stripArchiveFields(log.entries[lastIndex]);
+ writeLog(root, log);
+ writeMirror(root, stripArchiveFields(readMirror(root)));
+
+ // Must NOT fail outright.
+ const first = await verifyPolicyLedger(root);
+ expect(first).toEqual({ valid: true, length: 10 });
+
+ // The transitional anchor must have been caught up with a fresh entry.
+ const updatedLog = readLog(root);
+ expect(updatedLog.entries.at(-1)).toMatchObject({ archivedThrough: 7 });
+ expect(readMirror(root)).toMatchObject({ archivedThrough: 7 });
+
+ // The archive is now actually protected going forward.
+ rmSync(archivePath(root));
+ const second = await verifyPolicyLedger(root);
+ expect(second).toMatchObject({ valid: false, error: 'policy-ledger-archive-tampered' });
+ });
+
+ it('accepts a genuine older mirror after a failed best-effort write, and rejects a forged nonmember mirror', async () => {
+ const root = await prunedLedger('3', 10);
+
+ // Transitional ledger: strip archive fields from both copies.
+ const log = readLog(root);
+ const preTransitionEntry = log.entries[log.entries.length - 1];
+ log.entries[log.entries.length - 1] = stripArchiveFields(preTransitionEntry);
+ writeLog(root, log);
+ const oldMirror = stripArchiveFields(readMirror(root));
+ writeMirror(root, oldMirror);
+
+ // One real transaction catches both copies up with archive fields β€”
+ // this is the normal, correct path (already covered above).
+ await decide(root, 999);
+
+ // Simulate a mirror write that failed on that transaction (home
+ // directory read-only, per recordAnchor's documented degradation): the
+ // mirror is left pointing at the OLD, pre-catch-up entry, which is
+ // still a real, present entry in the log (mirror and log entries are
+ // written together in the non-degraded case, so this is the only way
+ // they can legitimately disagree about archive fields once the ledger
+ // has already been pruned).
+ writeMirror(root, oldMirror);
+
+ const result = await verifyPolicyLedger(root);
+ // The mirror still authenticates an exact entry of the validated log;
+ // state-ahead publication plus a best-effort mirror failure is legitimate.
+ expect(result).toMatchObject({ valid: true, length: 11 });
+ const forged = stripArchiveFields({ ...oldMirror, length: oldMirror.length + 100 });
+ writeMirror(root, forged);
+ expect(await verifyPolicyLedger(root)).toMatchObject({ valid: false, error: 'policy-anchor-log-rolled-back' });
+ });
+
+ it('a ledger that has never been pruned stays byte-identical: no archive fields are ever written', async () => {
+ const root = project();
+ await autoMigratePolicyStateIfNeeded(root);
+ for (let i = 0; i < 5; i++) await decide(root, i);
+
+ const log = readLog(root);
+ for (const entry of log.entries) {
+ expect(Object.hasOwn(entry, 'archivedThrough')).toBe(false);
+ expect(Object.hasOwn(entry, 'archiveHeadHash')).toBe(false);
+ // The hash must match the pre-existing (7-field) formula exactly.
+ expect(entry.hash).toBe(oldFormatHash(entry));
+ }
+ expect(await verifyPolicyLedger(root)).toEqual({ valid: true, length: 5 });
+ });
+});
+
+describe('#3892 Blocker 2: dedupeArchiveReceipts must verify a receipt\'s own hash, not just compare hash strings', () => {
+ it('a lone archive receipt whose stored hash does not match its own payload is rejected, even with no duplicate', async () => {
+ const root = await prunedLedger('3', 10);
+ const lines = readArchiveLines(root);
+ const tampered = JSON.parse(lines[0]!);
+ tampered.payload.decision.reason = 'tampered-without-recomputing-hash';
+ // hash left stale on purpose.
+ lines[0] = JSON.stringify(tampered);
+ writeFileSync(archivePath(root), `${lines.join('\n')}\n`);
+
+ const result = await verifyPolicyLedger(root, { full: true });
+ expect(result).toMatchObject({ valid: false, error: 'policy-ledger-archive-conflict' });
+ });
+
+ it('two same-sequence candidates with a copied hash string but different payload content are rejected', async () => {
+ const root = await prunedLedger('3', 10);
+ const lines = readArchiveLines(root);
+ const original = JSON.parse(lines[0]!);
+ const forged = { ...original, payload: { ...original.payload, decision: { ...original.payload.decision, reason: 'forged' } } };
+ // forged.hash is copied from `original` verbatim β€” it will not match
+ // forged's own recomputed content hash, so the per-record
+ // self-consistency check (not the duplicate-specific deep-equal check)
+ // is what actually catches this: a genuine SHA-256 collision would be
+ // needed to reach the deep-equal branch, which is cryptographically
+ // infeasible to construct in a test.
+ writeFileSync(archivePath(root), `${[...lines, JSON.stringify(forged)].join('\n')}\n`);
+
+ const result = await verifyPolicyLedger(root, { full: true });
+ expect(result).toMatchObject({ valid: false, error: 'policy-ledger-archive-conflict' });
+ });
+
+ it('still dedupes a genuine crash-duplicate (identical re-appended line) as harmless', async () => {
+ const root = await prunedLedger('3', 10);
+ const lines = readArchiveLines(root);
+ appendFileSync(archivePath(root), `${lines[0]}\n`);
+ expect(await verifyPolicyLedger(root, { full: true })).toEqual({ valid: true, length: 10 });
+ });
+});
+
+describe('#3892 follow-up (team-lead review): a torn read must fail clean, not throw', () => {
+ it('a torn/unparseable MIDDLE line during --full fails clean instead of throwing', async () => {
+ const root = await prunedLedger('3', 10);
+ const lines = readArchiveLines(root);
+ // Torn a line other than the tail (sequence 6, index 6): the tail stays
+ // intact and self-consistent, so Blocker 1's fast archive-tip check
+ // (which runs first, inside reconcileLedger, even when options.full is
+ // set) passes and the call actually reaches the --full replay path β€”
+ // this is what distinguishes this test from simply tearing the tail,
+ // which the fast check alone would already catch as
+ // 'policy-ledger-archive-tampered' before ever reaching the parser
+ // this test is targeting. This is exactly what a concurrent
+ // `appendFileSync` mid-write (or a crash) could leave behind: a
+ // truncated fragment of a JSON line that cannot parse.
+ lines[2] = lines[2]!.slice(0, 15);
+ writeFileSync(archivePath(root), `${lines.join('\n')}\n`);
+
+ await expect(verifyPolicyLedger(root, { full: true })).resolves.toMatchObject({
+ valid: false,
+ error: 'policy-ledger-archive-parse-error',
+ });
+ // The fast (default) path never reads the full archive content, so a
+ // torn middle line that doesn't disturb the tail is invisible to it.
+ expect(await verifyPolicyLedger(root)).toEqual({ valid: true, length: 10 });
+ });
+});
+
+describe('#3892 Blocker 3: --full must not hold the policy-state lock for the whole replay', () => {
+ it('a transaction started immediately after a --full verify is not starved waiting on a lock the replay no longer holds', async () => {
+ const root = await prunedLedger('4', 30);
+ // Sanity: the lock file does not outlive the call at all (covers both
+ // the fast-path reconcile and the full replay).
+ const verification = await verifyPolicyLedger(root, { full: true });
+ expect(verification.valid).toBe(true);
+ expect(existsSync(lockPath(root))).toBe(false);
+ await withRetention('4', () => decide(root, 999));
+ expect(await verifyPolicyLedger(root)).toEqual({ valid: true, length: 31 });
+ });
+
+ it('a SEPARATE process observes the lock held only for the cheap reconcile, not for the whole slow --full replay (real cross-process concurrency)', async () => {
+ // 20k receipts makes the O(total-history) replay (dedupe + full chain
+ // verify) take tens to low-hundreds of ms β€” long enough to give a
+ // waiting process a meaningfully larger window to lose, if the lock
+ // were (as before this fix) held for the whole call.
+ const root = await bigPrunedLedger(20_000, 10);
+ const lock = lockPath(root);
+ expect(existsSync(lock)).toBe(false);
+
+ // A genuinely separate OS process (not just another promise in this
+ // same single-threaded event loop, which a CPU-bound synchronous
+ // replay would starve regardless of the lock) that busy-polls for the
+ // lock file's existence, then busy-polls to re-acquire it the instant
+ // it is removed β€” measuring exactly how long the lock was HELD, not
+ // how long the whole parent call took. Uses writeSync(1, ...) rather
+ // than process.stdout.write so signals are flushed synchronously and
+ // are not deferred behind the busy loop.
+ const probeScript = `
+ const { openSync, closeSync, existsSync, writeSync } = require('node:fs');
+ const lockPath = process.argv[1];
+ writeSync(1, 'READY\\n');
+ const deadline1 = Date.now() + 10000;
+ while (!existsSync(lockPath)) {
+ if (Date.now() > deadline1) { writeSync(1, 'TIMEOUT\\n'); process.exit(1); }
+ }
+ const heldSince = Date.now();
+ const deadline2 = heldSince + 10000;
+ for (;;) {
+ try {
+ const fd = openSync(lockPath, 'wx', 0o600);
+ closeSync(fd);
+ writeSync(1, 'ELAPSED:' + (Date.now() - heldSince) + '\\n');
+ break;
+ } catch {
+ if (Date.now() > deadline2) { writeSync(1, 'TIMEOUT\\n'); process.exit(1); }
+ }
+ }
+ `;
+ const child = spawn(process.execPath, ['-e', probeScript, lock], { stdio: ['ignore', 'pipe', 'ignore'] });
+ let buffer = '';
+ const seen = { ready: false };
+ const readyPromise = new Promise<void>((resolveReady) => {
+ child.stdout.on('data', (chunk: Buffer) => {
+ buffer += chunk.toString();
+ if (!seen.ready && buffer.includes('READY')) { seen.ready = true; resolveReady(); }
+ });
+ });
+ const elapsedPromise = new Promise<number>((resolveElapsed, rejectElapsed) => {
+ child.stdout.on('data', () => {
+ const match = buffer.match(/ELAPSED:(\d+)/);
+ if (match) resolveElapsed(Number(match[1]));
+ else if (buffer.includes('TIMEOUT')) rejectElapsed(new Error('probe process timed out'));
+ });
+ });
+
+ try {
+ await readyPromise;
+ const start = Date.now();
+ const result = await verifyPolicyLedger(root, { full: true });
+ const totalDurationMs = Date.now() - start;
+ expect(result).toEqual({ valid: true, length: 20_000 });
+
+ const childHeldForMs = await elapsedPromise;
+ // The lock should be released long before the full replay finishes β€”
+ // held only for the cheap reconcile, a small fraction of the total
+ // call duration (and in absolute terms, well under 50ms).
+ expect(childHeldForMs).toBeLessThan(Math.max(50, totalDurationMs * 0.5));
+ expect(totalDurationMs).toBeGreaterThan(childHeldForMs);
+ } finally {
+ // The probe created the lock file and never released it (no
+ // transaction to pair the release with in this test).
+ rmSync(lock, { force: true });
+ if (!child.killed) child.kill();
+ }
+ }, 20_000);
+});
+
+describe('#3892 regression guard: existing #3568/#3602/#3164 behavior is unchanged', () => {
+ it('a plain unpruned ledger with intact anchors verifies valid with no archive-specific errors', async () => {
+ const root = project();
+ await autoMigratePolicyStateIfNeeded(root);
+ for (let i = 0; i < 4; i++) await decide(root, i);
+ expect(await verifyPolicyLedger(root)).toEqual({ valid: true, length: 4 });
+ expect(await verifyPolicyLedger(root, { full: true })).toEqual({ valid: true, length: 4 });
+ });
+
+ it('truncation of the hot tail after pruning is still caught (#3568 under retention)', async () => {
+ const root = await prunedLedger('3', 8);
+ const state = readState(root);
+ state.receipts.pop();
+ writeState(root, state);
+ expect(await verifyPolicyLedger(root)).toMatchObject({ valid: false, error: 'policy-ledger-truncated' });
+ });
+
+ it('a tampered receipt in the hot tail is still caught (#3164 regression)', async () => {
+ const root = await prunedLedger('3', 8);
+ const state = readState(root);
+ state.receipts[0].payload.decision.reason = 'tampered';
+ writeState(root, state);
+ const result = await verifyPolicyLedger(root);
+ expect(result).toMatchObject({ valid: false, error: 'receipt-hash-mismatch' });
+ });
+});
diff --git a/v3/@claude-flow/cli/__tests__/policy-ledger-integrity-durability-3892.test.ts b/v3/@claude-flow/cli/__tests__/policy-ledger-integrity-durability-3892.test.ts
new file mode 100644
index 0000000000..b02d60654c
--- /dev/null
+++ b/v3/@claude-flow/cli/__tests__/policy-ledger-integrity-durability-3892.test.ts
@@ -0,0 +1,255 @@
+import './helpers/isolated-policy-trust.js';
+import { afterEach, describe, expect, it, vi } from 'vitest';
+import { appendFileSync, existsSync, mkdtempSync, mkdirSync, readFileSync, realpathSync, rmSync, symlinkSync, unlinkSync, utimesSync, writeFileSync } from 'node:fs';
+import { execFileSync, spawn } from 'node:child_process';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { createHash } from 'node:crypto';
+import { policyTrustRoot } from '../src/services/policy-ledger-anchor.js';
+import { acquireLock, autoMigratePolicyStateIfNeeded, evaluatePolicyRequest, verifyPolicyLedger, withPolicyTransaction } from '../src/services/policy-runtime.js';
+
+const io = vi.hoisted(() => ({ events: [] as string[], files: new Map<number, string>(), failSync: false, failState: false, failRenameSuffix: '', failStateDirSync: false, failAnchorStream: false }));
+vi.mock('node:fs', async (importOriginal) => {
+ const actual = await importOriginal<typeof import('node:fs')>();
+ return {
+ ...actual,
+ openSync: (...args: Parameters<typeof actual.openSync>) => { const fd = actual.openSync(...args); io.files.set(fd, String(args[0])); return fd; },
+ closeSync: (fd: number) => { io.files.delete(fd); return actual.closeSync(fd); },
+ fsyncSync: (fd: number) => {
+ const file = io.files.get(fd) ?? '?'; io.events.push(`sync:${file}`);
+ if (io.failStateDirSync && io.events[io.events.length - 2] === `rename:${join(file, 'state.json')}`) throw Object.assign(new Error('injected state parent sync failure'), { code: 'EACCES' });
+ if (io.failSync && file.endsWith('receipts.ledger.jsonl')) throw Object.assign(new Error('injected archive sync failure'), { code: 'EIO' });
+ return actual.fsyncSync(fd);
+ },
+ writeSync: (fd: number, ...args: any[]) => {
+ const file = io.files.get(fd) ?? '';
+ if (io.failAnchorStream && file.includes('ledger-anchors.json.') && file.endsWith('.tmp')) throw new Error('injected anchor stream failure');
+ return (actual.writeSync as (...args: any[]) => number)(fd, ...args);
+ },
+ renameSync: (from: string, to: string) => {
+ io.events.push(`rename:${to}`);
+ if (io.failRenameSuffix && to.endsWith(io.failRenameSuffix)) throw new Error('injected anchor publication failure');
+ if (io.failState && to.endsWith('state.json')) throw Object.assign(new Error('injected state publication failure'), { code: 'EIO' });
+ return actual.renameSync(from, to);
+ },
+ };
+});
+
+const roots: string[] = [];
+const dir = (root: string) => join(root, '.claude-flow', 'policy');
+const archive = (root: string) => join(dir(root), 'receipts.ledger.jsonl');
+const state = (root: string) => join(dir(root), 'state.json');
+function root() { const result = mkdtempSync(join(tmpdir(), 'ruflo-3892-integrity-')); roots.push(result); mkdirSync(join(result, '.claude-flow')); return result; }
+async function decide(project: string, id: number) {
+ return evaluatePolicyRequest({ identity: { id: `agent:${id}`, type: 'agent' }, action: { type: 'code.read', resource: `cafΓ©-${id}` } }, project);
+}
+async function ledger(signed = false) {
+ if (signed) { process.env.CLAUDE_FLOW_POLICY_SIGNING_KEY = 'disposable-regression-key'; process.env.CLAUDE_FLOW_POLICY_KEY_ID = 'original-custom-id'; }
+ process.env.CLAUDE_FLOW_POLICY_LEDGER_RETENTION = '2';
+ const project = root(); await autoMigratePolicyStateIfNeeded(project);
+ for (let i = 0; i < 6; i++) await decide(project, i);
+ return project;
+}
+function records(project: string) { return readFileSync(archive(project), 'utf8').trim().split('\n').map(line => JSON.parse(line)); }
+afterEach(() => {
+ io.failSync = io.failState = io.failStateDirSync = io.failAnchorStream = false; io.failRenameSuffix = ''; io.events.length = 0;
+ for (const project of roots.splice(0)) {
+ const id = createHash('sha256').update(realpathSync(project)).digest('hex');
+ rmSync(join(policyTrustRoot(), id), { recursive: true, force: true });
+ rmSync(project, { recursive: true, force: true });
+ }
+ delete process.env.CLAUDE_FLOW_POLICY_SIGNING_KEY; delete process.env.CLAUDE_FLOW_POLICY_KEY_ID; delete process.env.CLAUDE_FLOW_POLICY_LEDGER_RETENTION;
+});
+
+describe('authenticate every physical archive record before deduplication', () => {
+ it('authenticates an unpruned staging archive instead of skipping its physical records', async () => {
+ process.env.CLAUDE_FLOW_POLICY_SIGNING_KEY = 'disposable-regression-key'; process.env.CLAUDE_FLOW_POLICY_KEY_ID = 'original-custom-id';
+ const project = root(); await autoMigratePolicyStateIfNeeded(project); await decide(project, 0);
+ const hot = JSON.parse(readFileSync(state(project), 'utf8')).receipts[0];
+ writeFileSync(archive(project), `${JSON.stringify(hot)}\n${JSON.stringify({ ...hot, signature: 'bad' })}\n`);
+ expect(await verifyPolicyLedger(project, { full: true })).toMatchObject({ valid: false, error: 'policy-ledger-archive-conflict' });
+ writeFileSync(archive(project), 'null\n');
+ expect(await verifyPolicyLedger(project, { full: true })).toMatchObject({ valid: false, error: 'policy-ledger-archive-conflict' });
+ writeFileSync(archive(project), '{"pending":');
+ expect(await verifyPolicyLedger(project, { full: true })).toMatchObject({ valid: false, error: 'policy-ledger-archive-parse-error' });
+ });
+ it.skipIf(process.platform === 'win32')('an unpruned FIFO snapshot cannot block full verification', async () => {
+ const project = root(); await autoMigratePolicyStateIfNeeded(project); await decide(project, 0);
+ execFileSync('mkfifo', [archive(project)]);
+ // A regressed blocking read is released by a child so the test remains
+ // bounded even though JavaScript timers cannot interrupt readFileSync.
+ const writer = spawn('sh', ['-c', 'sleep 2; printf null > "$1"', 'sh', archive(project)], { stdio: 'ignore' });
+ try {
+ const started = Date.now(); expect(await verifyPolicyLedger(project, { full: true })).toMatchObject({ valid: false, error: 'policy-ledger-archive-not-regular' });
+ expect(Date.now() - started).toBeLessThan(1_000);
+ } finally { writer.kill('SIGKILL'); }
+ });
+ it.each(['valid-first', 'invalid-first'])('rejects a corrupted signature in either duplicate order (%s)', async order => {
+ const project = await ledger(true); const lines = records(project); const damaged = { ...lines[0], signature: 'bad' };
+ if (order === 'valid-first') appendFileSync(archive(project), `${JSON.stringify(damaged)}\n`);
+ else writeFileSync(archive(project), `${[damaged, ...lines].map(record => JSON.stringify(record)).join('\n')}\n`);
+ expect(await verifyPolicyLedger(project, { full: true })).toMatchObject({ valid: false, error: 'policy-ledger-archive-conflict' });
+ });
+ it('rejects same-key wrong-ID duplicates and passes matching custom-ID history', async () => {
+ const project = await ledger(true); expect(await verifyPolicyLedger(project, { full: true })).toMatchObject({ valid: true });
+ appendFileSync(archive(project), `${JSON.stringify({ ...records(project)[0], keyId: 'other-id' })}\n`);
+ expect(await verifyPolicyLedger(project, { full: true })).toMatchObject({ valid: false });
+ });
+ it('rejects a forged overlap with the retained hot tail', async () => {
+ const project = await ledger(true); const hot = JSON.parse(readFileSync(state(project), 'utf8')).receipts[0];
+ appendFileSync(archive(project), `${JSON.stringify({ ...hot, signature: 'bad' })}\n`);
+ expect(await verifyPolicyLedger(project, { full: true })).toMatchObject({ valid: false });
+ });
+ it.each([null, {}, { payload: null }, { payload: { sequence: '0' } }])('reports malformed JSON record cleanly (%j)', async record => {
+ const project = await ledger(); appendFileSync(archive(project), `${JSON.stringify(record)}\n`);
+ expect(await verifyPolicyLedger(project, { full: true })).toMatchObject({ valid: false, error: 'policy-ledger-archive-conflict' });
+ });
+});
+
+describe('state-backed torn append recovery', () => {
+ it.each(['ascii', 'split-utf8'])('recovers exact retry receipt prefix (%s)', async mode => {
+ const project = await ledger(true); const hot = JSON.parse(readFileSync(state(project), 'utf8')).receipts[0];
+ const bytes = Buffer.from(JSON.stringify(hot));
+ const cut = mode === 'ascii' ? 40 : bytes.indexOf(Buffer.from('Γ©')) + 1;
+ expect(cut).toBeGreaterThan(0); appendFileSync(archive(project), bytes.subarray(0, cut));
+ await decide(project, 6);
+ expect(await verifyPolicyLedger(project, { full: true })).toMatchObject({ valid: true, length: 7 });
+ });
+ it.each(['{"unknown":', 'null', '{}', '{"payload":null}', '{"payload":{"sequence":"bad"}}'])('preserves unknown/complete malformed trailing bytes (%s)', async suffix => {
+ const project = await ledger(); appendFileSync(archive(project), suffix);
+ const before = readFileSync(archive(project)); const stateBefore = readFileSync(state(project));
+ await expect(decide(project, 6)).rejects.toThrow(/policy-ledger-archive/);
+ expect(readFileSync(archive(project))).toEqual(before); expect(readFileSync(state(project))).toEqual(stateBefore);
+ });
+ it('preserves a valid complete final receipt without a newline', async () => {
+ const project = await ledger(); const hot = JSON.parse(readFileSync(state(project), 'utf8')).receipts[0];
+ appendFileSync(archive(project), JSON.stringify(hot)); const before = readFileSync(archive(project));
+ await expect(decide(project, 6)).rejects.toThrow('policy-ledger-archive-unterminated-record');
+ expect(readFileSync(archive(project))).toEqual(before);
+ });
+ it('does not recover an interior malformed record', async () => {
+ const project = await ledger(); appendFileSync(archive(project), '{"bad":\n');
+ const before = readFileSync(archive(project)); await expect(decide(project, 6)).rejects.toThrow('policy-ledger-archive-parse-error');
+ expect(readFileSync(archive(project))).toEqual(before);
+ });
+ it('does not truncate a symlink target', async () => {
+ const project = await ledger(); const outside = join(project, 'outside-ledger');
+ writeFileSync(outside, readFileSync(archive(project))); appendFileSync(outside, '{"unknown":');
+ unlinkSync(archive(project)); symlinkSync(outside, archive(project)); const before = readFileSync(outside);
+ await expect(decide(project, 6)).rejects.toThrow(); expect(readFileSync(outside)).toEqual(before);
+ });
+});
+
+describe('durable publication and retry', () => {
+ it('syncs archive before state publication and syncs renamed state and anchor parents', async () => {
+ const project = await ledger(); io.events.length = 0; await decide(project, 6);
+ const archiveSync = io.events.indexOf(`sync:${archive(project)}`); const stateRename = io.events.indexOf(`rename:${state(project)}`);
+ expect(archiveSync).toBeGreaterThanOrEqual(0); expect(stateRename).toBeGreaterThan(archiveSync);
+ expect(io.events.slice(0, stateRename).some(event => event.startsWith(`sync:${state(project)}.`))).toBe(true);
+ expect(io.events[stateRename + 1]).toBe(`sync:${dir(project)}`);
+ const anchorRename = io.events.indexOf(`rename:${join(dir(project), 'ledger-anchors.json')}`);
+ expect(anchorRename).toBeGreaterThan(stateRename); expect(io.events[anchorRename + 1]).toBe(`sync:${dir(project)}`);
+ const mirrorRename = io.events.findIndex(event => event.endsWith('/ledger-anchor-head.json') && event.startsWith('rename:'));
+ expect(mirrorRename).toBeGreaterThan(anchorRename);
+ expect(io.events[mirrorRename + 1]).toBe(`sync:${io.events[mirrorRename].slice(7).replace(/\/ledger-anchor-head.json$/, '')}`);
+ });
+ it.each(['complete', 'missing-extension'])('a lagging mirror requires the entire newly archived extension (%s)', async mode => {
+ const project = await ledger(); io.failRenameSuffix = 'ledger-anchor-head.json';
+ const warning = vi.spyOn(process.stderr, 'write').mockImplementation(() => true);
+ try {
+ await withPolicyTransaction(project, engine => {
+ for (let i = 6; i < 12; i++) engine.evaluate({ identity: { id: `agent:${i}`, type: 'agent' }, action: { type: 'code.read', resource: `f-${i}` } });
+ });
+ await decide(project, 12); io.failRenameSuffix = '';
+ if (mode === 'missing-extension') {
+ const lines = records(project).filter(record => record.payload.sequence !== 7);
+ writeFileSync(archive(project), `${lines.map(record => JSON.stringify(record)).join('\n')}\n`);
+ const log = readFileSync(join(dir(project), 'ledger-anchors.json'));
+ expect(await verifyPolicyLedger(project)).toMatchObject({ valid: false });
+ expect(readFileSync(join(dir(project), 'ledger-anchors.json'))).toEqual(log);
+ } else expect(await verifyPolicyLedger(project, { full: true })).toMatchObject({ valid: true, length: 13 });
+ } finally { warning.mockRestore(); }
+ });
+ it.each(['zero-retention', 'large-batch'])('archives only committed bytes before a failed first publication (%s)', async mode => {
+ const project = await ledger(); const acknowledged = records(project);
+ if (mode === 'zero-retention') process.env.CLAUDE_FLOW_POLICY_LEDGER_RETENTION = '0';
+ io.failState = true;
+ const batch = () => withPolicyTransaction(project, engine => {
+ for (let i = 6; i < (mode === 'large-batch' ? 12 : 7); i++) engine.evaluate({ identity: { id: `retry-agent:${i}`, type: 'agent' }, action: { type: 'code.read', resource: `retry-${i}` } });
+ });
+ await expect(batch()).rejects.toThrow('injected');
+ expect(records(project).every(record => record.payload.sequence < 6)).toBe(true);
+ io.failState = false;
+ // A later retry creates fresh timestamps; no uncommitted physical record
+ // from the failed operation can collide with its same sequence.
+ await new Promise(resolve => setTimeout(resolve, 5)); await batch();
+ expect(await verifyPolicyLedger(project, { full: true })).toMatchObject({ valid: true, length: mode === 'large-batch' ? 12 : 7 });
+ expect(records(project).slice(0, acknowledged.length)).toEqual(acknowledged);
+ expect(JSON.parse(readFileSync(state(project), 'utf8')).receipts).toHaveLength(mode === 'large-batch' ? 6 : 1);
+ });
+ it.each(['state-parent-sync', 'anchor-log-rename'])('state-ahead %s failure preserves historical receipts and catches up on verify', async failure => {
+ const project = await ledger(); const old = records(project);
+ io.failStateDirSync = failure === 'state-parent-sync';
+ io.failRenameSuffix = failure === 'anchor-log-rename' ? 'ledger-anchors.json' : '';
+ await expect(decide(project, 6)).rejects.toThrow('injected');
+ // Publication may precede failure; an API failure does not imply rollback.
+ expect(JSON.parse(readFileSync(state(project), 'utf8')).ledgerLength).toBe(7);
+ io.failStateDirSync = false; io.failRenameSuffix = '';
+ expect(await verifyPolicyLedger(project, { full: true })).toMatchObject({ valid: true, length: 7 });
+ expect(records(project).slice(0, old.length)).toEqual(old);
+ });
+ it('an anchor serialization callback failure preserves the old log and supports state-ahead recovery', async () => {
+ const project = await ledger(); const file = join(dir(project), 'ledger-anchors.json'); const old = readFileSync(file);
+ io.failAnchorStream = true; await expect(decide(project, 6)).rejects.toThrow('injected anchor stream failure');
+ expect(readFileSync(file)).toEqual(old); io.failAnchorStream = false;
+ expect(await verifyPolicyLedger(project, { full: true })).toMatchObject({ valid: true, length: 7 });
+ });
+ it('a mirror failure at the first prune keeps its authentic prearchive checkpoint usable', async () => {
+ process.env.CLAUDE_FLOW_POLICY_LEDGER_RETENTION = '2';
+ const project = root(); await autoMigratePolicyStateIfNeeded(project); await decide(project, 0); await decide(project, 1);
+ io.failRenameSuffix = 'ledger-anchor-head.json';
+ const warning = vi.spyOn(process.stderr, 'write').mockImplementation(() => true);
+ try {
+ await decide(project, 2); expect(warning).toHaveBeenCalled(); io.failRenameSuffix = '';
+ expect(await verifyPolicyLedger(project, { full: true })).toMatchObject({ valid: true, length: 3 });
+ await decide(project, 3);
+ expect(await verifyPolicyLedger(project, { full: true })).toMatchObject({ valid: true, length: 4 });
+ } finally { warning.mockRestore(); }
+ });
+ it('a best-effort mirror publication failure preserves a usable durable log and catches up later', async () => {
+ const project = await ledger(); io.failRenameSuffix = 'ledger-anchor-head.json';
+ const warning = vi.spyOn(process.stderr, 'write').mockImplementation(() => true);
+ try {
+ await decide(project, 6); // mirror is explicitly best effort in ADR-475
+ expect(warning).toHaveBeenCalled(); io.failRenameSuffix = '';
+ expect(await verifyPolicyLedger(project, { full: true })).toMatchObject({ valid: true, length: 7 });
+ await decide(project, 7);
+ expect(await verifyPolicyLedger(project, { full: true })).toMatchObject({ valid: true, length: 8 });
+ } finally { warning.mockRestore(); }
+ });
+ it.each(['archive-sync', 'state-rename'])('keeps acknowledged state on %s failure and permits exact duplicate retry', async failure => {
+ const project = await ledger(); const before = readFileSync(state(project));
+ io.failSync = failure === 'archive-sync'; io.failState = failure === 'state-rename';
+ await expect(decide(project, 6)).rejects.toThrow('injected'); expect(readFileSync(state(project))).toEqual(before);
+ io.failSync = io.failState = false; await decide(project, 6);
+ expect(await verifyPolicyLedger(project, { full: true })).toMatchObject({ valid: true, length: 7 });
+ });
+});
+
+describe('existing lock safety defect', () => {
+ it('does not reclaim an aged live owner', async () => {
+ const project = root(); const lock = join(project, 'lock'); const release = await acquireLock(lock);
+ utimesSync(lock, new Date(0), new Date(0));
+ await expect(acquireLock(lock)).rejects.toThrow('policy-state-lock-timeout'); expect(existsSync(lock)).toBe(true); release();
+ }, 10_000);
+ it('an old release does not remove a replacement owner lock', async () => {
+ const project = root(); const lock = join(project, 'lock'); const release = await acquireLock(lock);
+ unlinkSync(lock); const newerRelease = await acquireLock(lock); const newer = readFileSync(lock);
+ release(); expect(readFileSync(lock)).toEqual(newer); newerRelease();
+ });
+ it('serializes concurrent owners', async () => {
+ const project = root(); const lock = join(project, 'lock'); let active = 0; let max = 0;
+ await Promise.all(Array.from({ length: 8 }, async () => { const release = await acquireLock(lock); active++; max = Math.max(active, max); await new Promise(resolve => setTimeout(resolve, 10)); active--; release(); }));
+ expect(max).toBe(1);
+ });
+});
diff --git a/v3/@claude-flow/cli/__tests__/policy-ledger-retention-3164.test.ts b/v3/@claude-flow/cli/__tests__/policy-ledger-retention-3164.test.ts
new file mode 100644
index 0000000000..6c5c83249a
--- /dev/null
+++ b/v3/@claude-flow/cli/__tests__/policy-ledger-retention-3164.test.ts
@@ -0,0 +1,288 @@
+import './helpers/isolated-policy-trust.js';
+import { policyTrustRoot } from '../src/services/policy-ledger-anchor.js';
+import { afterEach, describe, expect, it } from 'vitest';
+import {
+ appendFileSync,
+ existsSync,
+ mkdirSync,
+ mkdtempSync,
+ readFileSync,
+ realpathSync,
+ rmSync,
+ writeFileSync,
+} from 'node:fs';
+import { join } from 'node:path';
+import { tmpdir } from 'node:os';
+import { createHash } from 'node:crypto';
+import {
+ autoMigratePolicyStateIfNeeded,
+ evaluatePolicyRequest,
+ verifyPolicyLedger,
+ withPolicyTransaction,
+} from '../src/services/policy-runtime.js';
+import { policyCommand } from '../src/commands/policy.js';
+import { policyTools } from '../src/mcp-tools/policy-tools.js';
+import { anchorLogPath } from '../src/services/policy-ledger-anchor.js';
+
+// #3164: `state.receipts` and `.claude-flow/policy/state.json` grew forever β€”
+// every MCP tool call paid O(all receipts ever issued) to verify/clone/
+// serialize the ledger, eventually exceeding the 5s policy lock and timing
+// out every tool call. This file covers the runtime-layer retention fix:
+// pruning the hot tail, archiving the rest, and still catching tampering and
+// truncation (#3568/#3602 must keep working, not just the new behavior).
+
+const policyStatusTool = policyTools.find((tool) => tool.name === 'policy_status')!;
+
+const roots: Array<{ root: string; trust: string }> = [];
+function project(): string {
+ const root = mkdtempSync(join(tmpdir(), 'ruflo-policy-3164-'));
+ mkdirSync(join(root, '.claude-flow'), { recursive: true });
+ const projectId = createHash('sha256').update(realpathSync(root)).digest('hex');
+ roots.push({ root, trust: join(policyTrustRoot(), projectId) });
+ return root;
+}
+
+afterEach(() => {
+ for (const item of roots.splice(0)) {
+ rmSync(item.trust, { recursive: true, force: true });
+ rmSync(item.root, { recursive: true, force: true });
+ }
+ delete process.env.CLAUDE_FLOW_POLICY_LEDGER_RETENTION;
+});
+
+const statePath = (root: string) => join(root, '.claude-flow', 'policy', 'state.json');
+const archivePath = (root: string) => join(root, '.claude-flow', 'policy', 'receipts.ledger.jsonl');
+const readState = (root: string) => JSON.parse(readFileSync(statePath(root), 'utf8'));
+const writeState = (root: string, state: unknown) => writeFileSync(statePath(root), JSON.stringify(state, null, 2));
+const readArchiveLines = (root: string): string[] => readFileSync(archivePath(root), 'utf8')
+ .split('\n')
+ .filter((line) => line.trim().length > 0);
+
+async function decide(root: string, i: number) {
+ return evaluatePolicyRequest({
+ identity: { id: `agent:${i}`, type: 'agent' },
+ action: { type: 'code.read', resource: `file-${i}` },
+ }, root);
+}
+
+async function withRetention<T>(value: string, fn: () => Promise<T>): Promise<T> {
+ const previous = process.env.CLAUDE_FLOW_POLICY_LEDGER_RETENTION;
+ process.env.CLAUDE_FLOW_POLICY_LEDGER_RETENTION = value;
+ try {
+ return await fn();
+ } finally {
+ if (previous === undefined) delete process.env.CLAUDE_FLOW_POLICY_LEDGER_RETENTION;
+ else process.env.CLAUDE_FLOW_POLICY_LEDGER_RETENTION = previous;
+ }
+}
+
+async function ledgerWithRetention(root: string, retain: string, count: number): Promise<void> {
+ await withRetention(retain, async () => {
+ for (let i = 0; i < count; i++) await decide(root, i);
+ });
+}
+
+describe('policy ledger retention bounds state.json growth and archives pruned receipts (#3164)', () => {
+ it('prunes past the retention threshold, keeps state.json bounded, and archives the rest', async () => {
+ const root = project();
+ await autoMigratePolicyStateIfNeeded(root);
+ await ledgerWithRetention(root, '5', 12);
+
+ const state = readState(root);
+ expect(state.receipts).toHaveLength(5);
+ expect(state.retainedFrom).toBe(7);
+ expect(state.prunedHead).toBe(state.receipts[0].payload.previousReceiptHash);
+
+ const archived = readArchiveLines(root).map((line) => JSON.parse(line));
+ expect(archived).toHaveLength(7);
+ expect(archived.map((r) => r.payload.sequence)).toEqual([0, 1, 2, 3, 4, 5, 6]);
+
+ // a later, normal transaction still succeeds against the pruned ledger
+ await withRetention('5', () => decide(root, 100));
+ expect(await verifyPolicyLedger(root)).toEqual({ valid: true, length: 13 });
+ });
+
+ it('fast verify (the hot path every MCP call takes) stays valid across many prune cycles', async () => {
+ const root = project();
+ await autoMigratePolicyStateIfNeeded(root);
+ await ledgerWithRetention(root, '3', 20);
+ expect(await verifyPolicyLedger(root)).toEqual({ valid: true, length: 20 });
+ expect(readState(root).receipts).toHaveLength(3);
+ });
+
+ it('verifyPolicyLedger({ full: true }) validates the complete historical chain across archive + hot tail', async () => {
+ const root = project();
+ await autoMigratePolicyStateIfNeeded(root);
+ await ledgerWithRetention(root, '4', 15);
+
+ expect(await verifyPolicyLedger(root)).toEqual({ valid: true, length: 15 });
+ expect(await verifyPolicyLedger(root, { full: true })).toEqual({ valid: true, length: 15 });
+ });
+
+ it('wires through `ruflo policy verify --full` and MCP policy_status({ full: true })', async () => {
+ const root = project();
+ await autoMigratePolicyStateIfNeeded(root);
+ await ledgerWithRetention(root, '4', 9);
+
+ const cli = await policyCommand.action!({ args: ['verify'], flags: { projectRoot: root, full: true } } as never);
+ expect(cli).toMatchObject({ success: true, exitCode: 0, data: { valid: true, length: 9 } });
+
+ const status = await policyStatusTool.handler({ full: true }, { projectRoot: root });
+ expect(status).toMatchObject({ ledger: { valid: true, length: 9 } });
+
+ const statusFast = await policyStatusTool.handler({}, { projectRoot: root });
+ expect(statusFast).toMatchObject({ ledger: { valid: true, length: 9 } });
+ });
+
+ it('a state.json with no retainedFrom/prunedHead at all (pre-#3164 shape) verifies and operates identically', async () => {
+ const root = project();
+ await autoMigratePolicyStateIfNeeded(root);
+ // Default retention (2000) is far above this count, so nothing prunes β€”
+ // this is today's on-disk shape, byte for byte.
+ for (let i = 0; i < 5; i++) await decide(root, i);
+ const state = readState(root);
+ expect(state.retainedFrom).toBeUndefined();
+ expect(state.prunedHead).toBeUndefined();
+
+ expect(await verifyPolicyLedger(root)).toEqual({ valid: true, length: 5 });
+ await decide(root, 50);
+ expect(await verifyPolicyLedger(root)).toEqual({ valid: true, length: 6 });
+ });
+});
+
+describe('#3164 crash-duplicate archive window', () => {
+ it('dedupes an identical re-appended line (archive-append landed, state write of the smaller retainedFrom did not, then the next prune re-wrote the same receipts)', async () => {
+ const root = project();
+ await autoMigratePolicyStateIfNeeded(root);
+ await ledgerWithRetention(root, '3', 10);
+ const before = await verifyPolicyLedger(root, { full: true });
+ expect(before).toEqual({ valid: true, length: 10 });
+
+ const lines = readArchiveLines(root);
+ // Simulate the documented crash: the same already-archived line appended
+ // again, byte for byte (what a re-prune of the un-shrunk state produces).
+ appendFileSync(archivePath(root), `${lines[0]}\n`);
+
+ expect(await verifyPolicyLedger(root, { full: true })).toEqual({ valid: true, length: 10 });
+ });
+
+ it('detects a genuine conflict β€” same sequence, different hash β€” as tampering, not a harmless duplicate', async () => {
+ const root = project();
+ await autoMigratePolicyStateIfNeeded(root);
+ await ledgerWithRetention(root, '3', 10);
+
+ const lines = readArchiveLines(root);
+ const conflicting = JSON.parse(lines[0]);
+ conflicting.hash = `${'0'.repeat(63)}1`;
+ appendFileSync(archivePath(root), `${JSON.stringify(conflicting)}\n`);
+
+ const result = await verifyPolicyLedger(root, { full: true });
+ expect(result).toMatchObject({ valid: false, error: 'policy-ledger-archive-conflict' });
+ // the fast path (hot tail only) is unaffected by archive corruption
+ expect(await verifyPolicyLedger(root)).toEqual({ valid: true, length: 10 });
+ });
+
+ it('detects a genuine gap in the archive as policy-ledger-archive-gap', async () => {
+ const root = project();
+ await autoMigratePolicyStateIfNeeded(root);
+ await ledgerWithRetention(root, '3', 10);
+
+ const lines = readArchiveLines(root);
+ lines.splice(3, 1);
+ writeFileSync(archivePath(root), `${lines.join('\n')}\n`);
+
+ const result = await verifyPolicyLedger(root, { full: true });
+ expect(result).toMatchObject({ valid: false, error: 'policy-ledger-archive-gap' });
+ expect(await verifyPolicyLedger(root)).toEqual({ valid: true, length: 10 });
+ });
+});
+
+describe('#3164 pruning does not weaken tamper/truncation detection in the retained hot tail (regression)', () => {
+ it('still catches a tampered receipt that survived pruning into the hot tail', async () => {
+ const root = project();
+ await autoMigratePolicyStateIfNeeded(root);
+ await ledgerWithRetention(root, '3', 8);
+
+ const state = readState(root);
+ expect(state.receipts).toHaveLength(3);
+ state.receipts[0].payload.decision.reason = 'tampered';
+ writeState(root, state);
+
+ const result = await verifyPolicyLedger(root);
+ expect(result).toMatchObject({ valid: false, error: 'receipt-hash-mismatch' });
+ // reports the absolute index within the FULL historical ledger (retainedFrom + 0),
+ // not the hot-array index
+ expect(result.length).toBe(5);
+
+ await expect(withRetention('3', () => decide(root, 999))).rejects.toThrow();
+ });
+
+ it('still catches truncation of the hot tail after pruning (#3568 regression under retention)', async () => {
+ const root = project();
+ await autoMigratePolicyStateIfNeeded(root);
+ await ledgerWithRetention(root, '3', 8);
+
+ const state = readState(root);
+ state.receipts.pop();
+ writeState(root, state);
+
+ expect(await verifyPolicyLedger(root)).toMatchObject({ valid: false, error: 'policy-ledger-truncated' });
+ });
+});
+
+describe('#3164 retention keeps the #3602 second anchor in agreement with hot state across prune boundaries', () => {
+ it('normal sequential operation never regresses into anchor-missing/anchor-mismatch across many prune cycles', async () => {
+ const root = project();
+ await autoMigratePolicyStateIfNeeded(root);
+ await ledgerWithRetention(root, '2', 30);
+
+ expect(await verifyPolicyLedger(root)).toEqual({ valid: true, length: 30 });
+ const log = JSON.parse(readFileSync(anchorLogPath(root), 'utf8'));
+ expect(log.entries).toHaveLength(30);
+ expect(log.entries.at(-1).length).toBe(30);
+ });
+
+ it('a concurrent transaction burst with pruning enabled keeps a valid, chained second anchor', async () => {
+ const root = project();
+ await autoMigratePolicyStateIfNeeded(root);
+ await ledgerWithRetention(root, '5', 3);
+
+ await withRetention('5', () => Promise.all(Array.from({ length: 10 }, (_, i) => decide(root, 100 + i))));
+
+ expect(await verifyPolicyLedger(root)).toEqual({ valid: true, length: 13 });
+ const log = JSON.parse(readFileSync(anchorLogPath(root), 'utf8'));
+ expect(log.entries.at(-1).length).toBe(13);
+ log.entries.forEach((entry: { seq: number; prevAnchorHash: string | null }, i: number) => {
+ expect(entry.seq).toBe(i);
+ expect(entry.prevAnchorHash).toBe(i === 0 ? null : log.entries[i - 1].hash);
+ });
+ });
+});
+
+describe('#3164 CLAUDE_FLOW_POLICY_LEDGER_RETENTION overrides', () => {
+ it('retain=0 archives committed receipts on the next transaction and keeps the latest transaction until then', async () => {
+ const root = project();
+ await autoMigratePolicyStateIfNeeded(root);
+ await ledgerWithRetention(root, '0', 3);
+
+ const state = readState(root);
+ expect(state.receipts).toHaveLength(1);
+ expect(state.retainedFrom).toBe(2);
+ await withRetention('0', () => withPolicyTransaction(root, () => undefined));
+ expect(readState(root).receipts).toHaveLength(0);
+ expect(readState(root).retainedFrom).toBe(3);
+ expect(await verifyPolicyLedger(root)).toEqual({ valid: true, length: 3 });
+ expect(await verifyPolicyLedger(root, { full: true })).toEqual({ valid: true, length: 3 });
+ });
+
+ it('a negative or non-integer override falls back to the default retention (no pruning for a small ledger)', async () => {
+ const root = project();
+ await autoMigratePolicyStateIfNeeded(root);
+ await ledgerWithRetention(root, '-5', 10);
+ expect(readState(root).retainedFrom).toBeUndefined();
+
+ await ledgerWithRetention(root, 'not-a-number', 1);
+ expect(readState(root).retainedFrom).toBeUndefined();
+ expect(await verifyPolicyLedger(root)).toEqual({ valid: true, length: 11 });
+ });
+});
diff --git a/v3/@claude-flow/cli/__tests__/policy-ledger-streaming-3892.test.ts b/v3/@claude-flow/cli/__tests__/policy-ledger-streaming-3892.test.ts
new file mode 100644
index 0000000000..fa7cacaffb
--- /dev/null
+++ b/v3/@claude-flow/cli/__tests__/policy-ledger-streaming-3892.test.ts
@@ -0,0 +1,210 @@
+import './helpers/isolated-policy-trust.js';
+import { afterEach, describe, expect, it } from 'vitest';
+import { appendFileSync, closeSync, ftruncateSync, mkdirSync, mkdtempSync, openSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { AgenticPolicyEngine, policyHash, type PolicyReceipt, type PolicyState } from '@claude-flow/security';
+import { ATOMIC_JSON_WRITER, anchorEntries } from '../src/services/policy-anchor-log.js';
+import { assessAnchors, recordAnchor } from '../src/services/policy-ledger-anchor.js';
+import { autoMigratePolicyStateIfNeeded, evaluatePolicyRequest, verifyPolicyLedger } from '../src/services/policy-runtime.js';
+import { captureArchiveSnapshot, verifyFullPolicyLedger } from '../src/services/policy-ledger-audit.js';
+
+const scratch: string[] = [];
+const originalXdg = process.env.XDG_CONFIG_HOME;
+function fixture(options: { signed?: boolean; count?: number; retain?: number } = {}) {
+ const dir = mkdtempSync(join(tmpdir(), 'ruflo-stream-test-')); scratch.push(dir);
+ const engineOptions = options.signed ? { signingKey: 'disposable-streaming-key', keyId: 'stream-test' } : {};
+ const engine = new AgenticPolicyEngine({ ...engineOptions, now: () => 1000 });
+ for (let i = 0; i < (options.count ?? 8); i++) engine.evaluate({ identity: { id: `agent:${i}`, type: 'agent' }, action: { type: 'code.read', resource: `cafΓ©-πŸ™‚-${i}` } });
+ const archive = engine.pruneReceipts(options.retain ?? 2);
+ const state = engine.exportState(); const file = join(dir, 'receipts.jsonl');
+ writeFileSync(file, `${archive.map(receipt => JSON.stringify(receipt)).join('\n')}\n`);
+ return { dir, file, archive, state, engineOptions };
+}
+async function audit(file: string, state: PolicyState, engineOptions: { signingKey?: string; keyId?: string } = {}) {
+ const snapshot = captureArchiveSnapshot(file);
+ try { return await verifyFullPolicyLedger(snapshot, state, engineOptions); }
+ finally { if (snapshot) closeSync(snapshot.fd); }
+}
+afterEach(() => { for (const dir of scratch.splice(0)) rmSync(dir, { recursive: true, force: true }); delete process.env.CLAUDE_FLOW_POLICY_AUDIT_MAX_RECORD_BYTES; delete process.env.CLAUDE_FLOW_POLICY_LEDGER_RETENTION;
+ if (originalXdg === undefined) delete process.env.XDG_CONFIG_HOME; else process.env.XDG_CONFIG_HOME = originalXdg; });
+
+describe('bounded full ledger replay', () => {
+ it('replays arbitrary physical order and exact duplicates without a history-sized receipt map', async () => {
+ const f = fixture({ signed: true });
+ writeFileSync(f.file, `${[...f.archive].reverse().concat([f.archive[0], f.archive[3], f.state.receipts[0]]).map(record => JSON.stringify(record)).join('\n')}\n`);
+ expect(await audit(f.file, f.state, f.engineOptions)).toEqual({ valid: true, length: 8 });
+ });
+ it('compares full canonical records even when a duplicate keeps the same valid payload/signature', async () => {
+ const f = fixture({ signed: true });
+ appendFileSync(f.file, `${JSON.stringify({ ...f.archive[0], untrustedExtra: true })}\n`);
+ expect(await audit(f.file, f.state, f.engineOptions)).toMatchObject({ valid: false, error: 'policy-ledger-archive-conflict' });
+ });
+ it('authenticates and dedupes valid huge-sequence extras without huge sparse index offsets', async () => {
+ const f = fixture(); const original = f.archive[0];
+ const { receiptId, ...unsigned } = original.payload; unsigned.sequence = Number.MAX_SAFE_INTEGER;
+ const payload = { ...unsigned, receiptId: policyHash(unsigned) }; const extra = { payload, hash: policyHash(payload) };
+ appendFileSync(f.file, `${JSON.stringify(extra)}\n${JSON.stringify(extra)}\n`);
+ expect(await audit(f.file, f.state)).toEqual({ valid: true, length: 8 });
+ appendFileSync(f.file, `${JSON.stringify({ ...extra, untrustedExtra: true })}\n`);
+ expect(await audit(f.file, f.state)).toMatchObject({ valid: false, error: 'policy-ledger-archive-conflict' });
+ });
+ it('compares canonical staging duplicates even when finite exponent numbers expand during canonicalization', async () => {
+ const f = fixture(); process.env.CLAUDE_FLOW_POLICY_AUDIT_MAX_RECORD_BYTES = '11000';
+ const { receiptId, ...unsigned } = f.archive[0].payload; unsigned.sequence = 8;
+ const payload = { ...unsigned, receiptId: policyHash(unsigned) }; const extra = { payload, hash: policyHash(payload), metadata: Array(2000).fill(1e20) };
+ const compact = JSON.stringify(extra).replaceAll('100000000000000000000', '1e20');
+ expect(Buffer.byteLength(compact)).toBeLessThan(11000);
+ expect(Buffer.byteLength(JSON.stringify(extra))).toBeGreaterThan(44000);
+ appendFileSync(f.file, `${compact}\n${compact}\n`);
+ expect(await audit(f.file, f.state)).toEqual({ valid: true, length: 8 });
+ });
+ it('checks hot overlaps and malformed staging records even with retainedFrom=0', async () => {
+ const f = fixture({ count: 2, retain: 2, signed: true });
+ writeFileSync(f.file, `${JSON.stringify(f.state.receipts[0])}\nnull\n`);
+ expect(await audit(f.file, f.state, f.engineOptions)).toMatchObject({ valid: false, error: 'policy-ledger-archive-conflict' });
+ });
+ it('fails on a missing claimed sequence and on a broken previous-hash link', async () => {
+ const f = fixture();
+ writeFileSync(f.file, `${f.archive.filter(receipt => receipt.payload.sequence !== 2).map(receipt => JSON.stringify(receipt)).join('\n')}\n`);
+ expect(await audit(f.file, f.state)).toMatchObject({ valid: false, error: 'policy-ledger-archive-gap' });
+ const { receiptId, ...unsigned } = f.archive[2].payload; unsigned.previousReceiptHash = `sha256:${'0'.repeat(64)}`;
+ const payload = { ...unsigned, receiptId: policyHash(unsigned) }; const broken = { payload, hash: policyHash(payload) };
+ writeFileSync(f.file, `${f.archive.map(receipt => JSON.stringify(receipt.payload.sequence === 2 ? broken : receipt)).join('\n')}\n`);
+ expect(await audit(f.file, f.state)).toMatchObject({ valid: false, error: 'receipt-chain-mismatch' });
+ });
+ it('does not read subsequent appends beyond the captured byte boundary', async () => {
+ const f = fixture(); const snapshot = captureArchiveSnapshot(f.file)!;
+ appendFileSync(f.file, 'null\n');
+ try { expect(await verifyFullPolicyLedger(snapshot, f.state, {})).toEqual({ valid: true, length: 8 }); }
+ finally { closeSync(snapshot.fd); }
+ expect(await audit(f.file, f.state)).toMatchObject({ valid: false, error: 'policy-ledger-archive-conflict' });
+ });
+ it('fails cleanly on truncation after capture', async () => {
+ const f = fixture(); const snapshot = captureArchiveSnapshot(f.file)!; writeFileSync(f.file, '');
+ try { expect(await verifyFullPolicyLedger(snapshot, f.state, {})).toMatchObject({ valid: false, error: 'policy-ledger-archive-changed' }); }
+ finally { closeSync(snapshot.fd); }
+ });
+ it.each(['{"torn":', 'null', '{}'])('requires the snapshot to end with a complete newline-delimited record (%s)', async suffix => {
+ const f = fixture(); appendFileSync(f.file, suffix);
+ expect(await audit(f.file, f.state)).toMatchObject({ valid: false, error: 'policy-ledger-archive-parse-error' });
+ });
+ it('rejects malformed UTF8 explicitly instead of replacing bytes', async () => {
+ const f = fixture(); appendFileSync(f.file, Buffer.from([0xff, 10]));
+ expect(await audit(f.file, f.state)).toMatchObject({ valid: false, error: 'policy-ledger-archive-utf8-error' });
+ });
+ it('handles multibyte UTF8 split across the 64KiB read boundary', async () => {
+ const f = fixture(); const receipt = f.archive[0];
+ const base = Buffer.from(JSON.stringify({ padding: '', ...receipt }));
+ const padding = 'x'.repeat(65536 - base.indexOf(Buffer.from('Γ©')) - 1);
+ const record = { padding, ...receipt };
+ const bytes = Buffer.from(`${JSON.stringify(record)}\n`);
+ expect(bytes.indexOf(Buffer.from('Γ©'))).toBe(65535);
+ writeFileSync(f.file, Buffer.concat([bytes, Buffer.from(`${f.archive.slice(1).map(receipt => JSON.stringify(receipt)).join('\n')}\n`)]));
+ expect(await audit(f.file, f.state)).toEqual({ valid: true, length: 8 });
+ });
+ it('reports the configured record-byte limit without skipping or changing ledger bytes', async () => {
+ const f = fixture(); process.env.CLAUDE_FLOW_POLICY_AUDIT_MAX_RECORD_BYTES = '1024';
+ const first = { ...f.archive[0], padding: 'x'.repeat(2048) };
+ writeFileSync(f.file, `${[first, ...f.archive.slice(1)].map(record => JSON.stringify(record)).join('\n')}\n`);
+ const before = readFileSync(f.file);
+ expect(await audit(f.file, f.state)).toMatchObject({ valid: false, error: 'policy-ledger-archive-record-too-large:1024' });
+ expect(readFileSync(f.file)).toEqual(before);
+ });
+ it('rejects an invalid limit and unsafe claimed state length before indexing', async () => {
+ const f = fixture(); process.env.CLAUDE_FLOW_POLICY_AUDIT_MAX_RECORD_BYTES = 'Infinity';
+ expect(await audit(f.file, f.state)).toMatchObject({ valid: false, error: 'policy-ledger-audit-invalid-record-limit' });
+ delete process.env.CLAUDE_FLOW_POLICY_AUDIT_MAX_RECORD_BYTES;
+ expect(await audit(f.file, { ...f.state, retainedFrom: Number.MAX_SAFE_INTEGER })).toMatchObject({ valid: false, error: 'policy-ledger-archive-gap' });
+ });
+});
+
+describe('policy state format 2 writer fence', () => {
+ it('accepts version 1 history and migrates rules/budgets/approvals/usage/receipt bytes without re-signing', () => {
+ const f = fixture({ signed: true }); const old = { ...f.state, version: 1 };
+ const migrated = AgenticPolicyEngine.fromState(old, f.engineOptions).exportState();
+ expect(migrated.version).toBe(2);
+ for (const field of ['rules', 'budgets', 'approvals', 'usage', 'receipts', 'retainedFrom', 'prunedHead', 'ledgerHead', 'ledgerLength'] as const) expect(migrated[field]).toEqual(old[field]);
+ expect(AgenticPolicyEngine.fromState(migrated, f.engineOptions).verifyLedger()).toEqual({ valid: true, length: 8 });
+ });
+ it('rejects unknown future versions instead of silently downgrading', () => {
+ const f = fixture(); expect(() => AgenticPolicyEngine.fromState({ ...f.state, version: 3 })).toThrow('unsupported-policy-state-version:3');
+ });
+});
+
+
+describe('legacy anchor JSON streaming contract', () => {
+ it('preserves property order, escaped strings, nested values and unknown valid root fields', () => {
+ const f = fixture(); const entries = [{ text: 'quotes " and backslash \\ and { [ cafΓ© πŸ™‚', nested: { values: [1, 'two', true] } }];
+ const value = { metadata: { ignored: [false, null] }, entries, version: 1 };
+ writeFileSync(f.file, JSON.stringify(value, null, 2)); expect([...anchorEntries(f.file)]).toEqual(entries);
+ });
+ it.each([
+ '{"version":1,"version":1,"entries":[]}', '{"version":1,"entries":[],"entries":[]}',
+ '{"version":1,"entries":[{}', '{"version":1,"entries":[{},]}', '{"version":1,"entries":[]}true',
+ '{"version":2,"entries":[]}', '{"version":1,"entries":null}', '{"version":1,"entries":[],"unknown":[1,]}',
+ '{\u00a0"version":1,"entries":[]}', '{"version":1,"entries":[]}\u00a0',
+ ])('rejects malformed/ambiguous legacy anchor JSON (%s)', text => {
+ const f = fixture(); writeFileSync(f.file, text); expect(() => [...anchorEntries(f.file)]).toThrow();
+ });
+ it('binds reused assessments to exact source count and tip before publishing', async () => {
+ const f = fixture({ retain: 8 }); process.env.XDG_CONFIG_HOME = join(f.dir, 'scratch-home');
+ mkdirSync(join(process.env.XDG_CONFIG_HOME, 'ruflo', 'policy-trust'), { recursive: true });
+ const write = async (file: string, value: unknown) => { mkdirSync(join(file, '..'), { recursive: true }); writeFileSync(file, JSON.stringify(value)); };
+ await recordAnchor(f.dir, f.state, write);
+ const assessed = assessAnchors(f.dir, f.state); expect(assessed.kind).toBe('ok');
+ if (assessed.kind !== 'ok') throw new Error('assessment required');
+ const engine = AgenticPolicyEngine.fromState(f.state); engine.evaluate({ identity: { id: 'new', type: 'agent' }, action: { type: 'code.read' } });
+ await recordAnchor(f.dir, engine.exportState(), write);
+ const log = join(f.dir, '.claude-flow/policy/ledger-anchors.json'); const before = readFileSync(log);
+ let published = false;
+ const streaming = async (_file: string, source: { [ATOMIC_JSON_WRITER]: (fd: number) => void }) => {
+ const temporary = join(f.dir, 'temp-anchor'); const fd = openSync(temporary, 'w');
+ try { source[ATOMIC_JSON_WRITER](fd); published = true; } finally { closeSync(fd); }
+ };
+ engine.evaluate({ identity: { id: 'newer', type: 'agent' }, action: { type: 'code.read' } });
+ await expect(recordAnchor(f.dir, engine.exportState(), write, 'append', undefined, streaming, assessed.last)).rejects.toThrow('policy-anchor-log-corrupt');
+ expect(published).toBe(false); expect(readFileSync(log)).toEqual(before);
+ });
+ it('does not grant private assessment reuse to plain JSON or another project', async () => {
+ const f = fixture({ retain: 8 }); const other = fixture({ retain: 8 }); process.env.XDG_CONFIG_HOME = join(f.dir, 'scratch-home');
+ mkdirSync(join(process.env.XDG_CONFIG_HOME, 'ruflo', 'policy-trust'), { recursive: true });
+ const write = async (file: string, value: unknown) => { mkdirSync(join(file, '..'), { recursive: true }); writeFileSync(file, JSON.stringify(value)); };
+ await recordAnchor(f.dir, f.state, write); await recordAnchor(other.dir, other.state, write);
+ const assessed = assessAnchors(f.dir, f.state); if (assessed.kind !== 'ok') throw new Error('assessment required');
+ const engine = AgenticPolicyEngine.fromState(f.state); engine.evaluate({ identity: { id: 'new', type: 'agent' }, action: { type: 'code.read' } });
+ const streaming = async () => { throw new Error('corrupt log must be rejected before serialization'); };
+ writeFileSync(join(other.dir, '.claude-flow/policy/ledger-anchors.json'), 'null');
+ await expect(recordAnchor(other.dir, engine.exportState(), write, 'append', undefined, streaming, assessed.last)).rejects.toThrow('policy-anchor-log-corrupt');
+ writeFileSync(join(f.dir, '.claude-flow/policy/ledger-anchors.json'), 'null');
+ await expect(recordAnchor(f.dir, engine.exportState(), write, 'append', undefined, streaming, JSON.parse(JSON.stringify(assessed.last)))).rejects.toThrow('policy-anchor-log-corrupt');
+ });
+ it('retains the exported custom AnchorWriter JSON-object contract', async () => {
+ const f = fixture(); const writes: Array<{ file: string; value: unknown }> = [];
+ await recordAnchor(f.dir, f.state, async (file, value) => { writes.push({ file, value: JSON.parse(JSON.stringify(value)) }); });
+ expect(writes[0].value).toMatchObject({ version: 1, entries: [expect.objectContaining({ length: 8 })] });
+ expect(writes[1].value).toMatchObject({ length: 8 });
+ });
+});
+
+describe('public full-audit preflight is bounded and fails closed', () => {
+ async function project() {
+ const f = fixture(); process.env.XDG_CONFIG_HOME = join(f.dir, 'private-config');
+ mkdirSync(join(process.env.XDG_CONFIG_HOME, 'ruflo', 'policy-trust'), { recursive: true });
+ mkdirSync(join(f.dir, '.claude-flow')); process.env.CLAUDE_FLOW_POLICY_LEDGER_RETENTION = '2';
+ await autoMigratePolicyStateIfNeeded(f.dir);
+ for (let i = 0; i < 4; i++) await evaluatePolicyRequest({ identity: { id: `public:${i}`, type: 'agent' }, action: { type: 'code.read' } }, f.dir);
+ return { ...f, archivePath: join(f.dir, '.claude-flow', 'policy', 'receipts.ledger.jsonl') };
+ }
+ it('rejects a sparse oversize unterminated suffix before any whole-file allocation', async () => {
+ const f = await project(); process.env.CLAUDE_FLOW_POLICY_AUDIT_MAX_RECORD_BYTES = '1024';
+ const before = readFileSync(join(f.dir, '.claude-flow', 'policy', 'state.json'));
+ const fd = openSync(f.archivePath, 'r+'); try { ftruncateSync(fd, readFileSync(f.archivePath).length + 4096); } finally { closeSync(fd); }
+ expect(await verifyPolicyLedger(f.dir, { full: true })).toMatchObject({ valid: false, error: 'policy-ledger-archive-record-too-large:1024' });
+ expect(readFileSync(join(f.dir, '.claude-flow', 'policy', 'state.json'))).toEqual(before);
+ });
+ it('reports malformed UTF8 from the tip preflight through the public API', async () => {
+ const f = await project(); appendFileSync(f.archivePath, Buffer.from([0xff, 10]));
+ expect(await verifyPolicyLedger(f.dir, { full: true })).toMatchObject({ valid: false, error: 'policy-ledger-archive-utf8-error' });
+ });
+});
diff --git a/v3/@claude-flow/cli/__tests__/policy-ledger-truncation-3568.test.ts b/v3/@claude-flow/cli/__tests__/policy-ledger-truncation-3568.test.ts
index 316e7eaed0..e714aa0625 100644
--- a/v3/@claude-flow/cli/__tests__/policy-ledger-truncation-3568.test.ts
+++ b/v3/@claude-flow/cli/__tests__/policy-ledger-truncation-3568.test.ts
@@ -1,7 +1,9 @@
+import './helpers/isolated-policy-trust.js';
+import { policyTrustRoot } from '../src/services/policy-ledger-anchor.js';
import { afterEach, describe, expect, it } from 'vitest';
import { mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
-import { tmpdir, userInfo } from 'node:os';
+import { tmpdir } from 'node:os';
import { createHash } from 'node:crypto';
import {
autoMigratePolicyStateIfNeeded,
@@ -18,7 +20,7 @@ function project(): string {
const root = mkdtempSync(join(tmpdir(), 'ruflo-policy-3568-'));
mkdirSync(join(root, '.claude-flow'), { recursive: true });
const projectId = createHash('sha256').update(realpathSync(root)).digest('hex');
- roots.push({ root, trust: join(userInfo().homedir, '.config', 'ruflo', 'policy-trust', projectId) });
+ roots.push({ root, trust: join(policyTrustRoot(), projectId) });
return root;
}
diff --git a/v3/@claude-flow/cli/__tests__/policy-lock-dead-owner.test.ts b/v3/@claude-flow/cli/__tests__/policy-lock-dead-owner.test.ts
index 2266e01a55..39a82f9396 100644
--- a/v3/@claude-flow/cli/__tests__/policy-lock-dead-owner.test.ts
+++ b/v3/@claude-flow/cli/__tests__/policy-lock-dead-owner.test.ts
@@ -17,8 +17,8 @@
* written by this OS, on this kernel boot and in this PID namespace, treats
* only ESRCH as dead, reads only small regular files (no symlinks, FIFOs or
* devices), and answers only for the lock it read, by re-reading its bytes
- * (not a lock a new owner created meanwhile). Every other lock keeps the old
- * mtime rule.
+ * (not a lock a new owner created meanwhile). Unknown/foreign/nonregular owners require explicit inspection; mtime
+ * alone never grants takeover.
*/
import { afterEach, describe, expect, it, vi } from 'vitest';
@@ -289,16 +289,15 @@ describe('policy state.lock that must still be waited for', () => {
}, 15_000);
});
-describe('policy state.lock mtime fallback', () => {
- it('still reclaims an old lock with no parsable pid', async () => {
+describe('policy state.lock unsafe age-only takeover is refused', () => {
+ it('preserves an old lock with no parsable pid', async () => {
const { root, lock } = await setup();
plantLock(lock, '{"pid":');
const old = (Date.now() - LOCK_STALE_MS - 5_000) / 1000;
utimesSync(lock, old, old);
- const started = Date.now();
- expect((await authorizeRead(root)).enforcedOutcome).toBe('allowed');
- expect(Date.now() - started).toBeLessThan(LOCK_WAIT_MS / 5);
+ await expect(authorizeRead(root)).rejects.toThrow('policy-state-lock-timeout');
+ expect(readFileSync(lock, 'utf8')).toBe('{"pid":');
}, 15_000);
// POSIX only: needs mkfifo and symlinks. Opening a FIFO for reading blocks
@@ -319,9 +318,37 @@ describe('policy state.lock mtime fallback', () => {
children.push(spawn('sh', ['-c', 'sleep 2; : > "$1"', 'sh', fifo], { stdio: 'ignore' }));
const started = Date.now();
- expect((await authorizeRead(root)).enforcedOutcome).toBe('allowed');
- expect(Date.now() - started).toBeLessThan(1_000);
- // The mtime rule removed the symlink, not its target.
+ await expect(authorizeRead(root)).rejects.toThrow('policy-state-lock-timeout');
+ expect(Date.now() - started).toBeLessThan(LOCK_WAIT_MS + 1_000);
+ // No nonregular owner or target is removed.
expect(existsSync(fifo)).toBe(true);
}, 15_000);
});
+
+
+describe('serialized dead-owner reclamation', () => {
+ it('a competing reclaimer sees the guard before it can remove a new owner', async () => {
+ const { root, lock, owner } = await setup(); const dead = await deadPid();
+ plantLock(lock, JSON.stringify({ ...owner, pid: dead }));
+ let competitor: Promise<unknown> | undefined; let active = 0; let max = 0; let probed = false;
+ const run = () => withPolicyTransaction(root, async () => { active++; max = Math.max(max, active); await sleep(30); active--; });
+ const realKill = process.kill.bind(process);
+ const probe = vi.spyOn(process, 'kill').mockImplementation(((pid: number, signal?: string | number) => {
+ if (pid === dead && signal === 0 && !probed) {
+ probed = true; expect(existsSync(`${lock}.reclaim`)).toBe(true);
+ competitor = run();
+ }
+ return realKill(pid, signal);
+ }) as typeof process.kill);
+ try { await run(); await competitor; expect(probed).toBe(true); expect(max).toBe(1); }
+ finally { probe.mockRestore(); }
+ });
+ it('preserves a dead-owner lock when its reclamation guard is orphaned', async () => {
+ const { root, lock, owner } = await setup(); const dead = await deadPid();
+ const content = JSON.stringify({ ...owner, pid: dead }); plantLock(lock, content);
+ const guard = JSON.stringify({ ...owner, pid: dead, nonce: 'orphan-regression' });
+ writeFileSync(`${lock}.reclaim`, guard); utimesSync(`${lock}.reclaim`, new Date(0), new Date(0));
+ await expect(authorizeRead(root)).rejects.toThrow('inspect exact owner');
+ expect(readFileSync(lock, 'utf8')).toBe(content); expect(readFileSync(`${lock}.reclaim`, 'utf8')).toBe(guard);
+ }, 15_000);
+});
diff --git a/v3/@claude-flow/cli/__tests__/policy-runtime-import-graph-3892.test.ts b/v3/@claude-flow/cli/__tests__/policy-runtime-import-graph-3892.test.ts
new file mode 100644
index 0000000000..6aeec7f1fe
--- /dev/null
+++ b/v3/@claude-flow/cli/__tests__/policy-runtime-import-graph-3892.test.ts
@@ -0,0 +1,30 @@
+import { readFileSync } from 'node:fs';
+import { dirname, resolve } from 'node:path';
+import { fileURLToPath } from 'node:url';
+import ts from 'typescript';
+import { describe, expect, it } from 'vitest';
+
+describe('policy runtime eager dependency graph (#3989 / #3892)', () => {
+ it('keeps the security barrel out of all transitively imported CLI helpers', () => {
+ const visited = new Set<string>();
+ const violations: string[] = [];
+ function visit(file: string): void {
+ if (visited.has(file)) return;
+ visited.add(file);
+ const source = ts.createSourceFile(file, readFileSync(file, 'utf8'), ts.ScriptTarget.Latest, true);
+ for (const node of source.statements) {
+ if (!ts.isImportDeclaration(node) && !ts.isExportDeclaration(node)) continue;
+ if (ts.isImportDeclaration(node) && node.importClause?.isTypeOnly) continue;
+ if (ts.isExportDeclaration(node) && node.isTypeOnly) continue;
+ const specifier = node.moduleSpecifier;
+ if (!specifier || !ts.isStringLiteral(specifier)) continue;
+ const name = specifier.text;
+ if (name === '@claude-flow/security') violations.push(`${file}: ${name}`);
+ if (name.startsWith('.')) visit(resolve(dirname(file), name.replace(/\.js$/, '.ts')));
+ }
+ }
+ visit(fileURLToPath(new URL('../src/services/policy-runtime.ts', import.meta.url)));
+ expect(visited.size).toBeGreaterThan(5);
+ expect(violations).toEqual([]);
+ });
+});
diff --git a/v3/@claude-flow/cli/__tests__/policy-runtime.test.ts b/v3/@claude-flow/cli/__tests__/policy-runtime.test.ts
index c98b82ebb4..ea4372e25b 100644
--- a/v3/@claude-flow/cli/__tests__/policy-runtime.test.ts
+++ b/v3/@claude-flow/cli/__tests__/policy-runtime.test.ts
@@ -1,7 +1,9 @@
+import './helpers/isolated-policy-trust.js';
+import { policyTrustRoot } from '../src/services/policy-ledger-anchor.js';
import { afterEach, describe, expect, it } from 'vitest';
import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, symlinkSync, unlinkSync, writeFileSync, realpathSync } from 'node:fs';
import { join } from 'node:path';
-import { tmpdir, userInfo } from 'node:os';
+import { tmpdir } from 'node:os';
import { createHash, createHmac } from 'node:crypto';
import { execFileSync } from 'node:child_process';
import {
@@ -34,7 +36,7 @@ function project(): string {
const projectId = createHash('sha256').update(realpathSync(root)).digest('hex');
roots.push({
root,
- trust: join(userInfo().homedir, '.config', 'ruflo', 'policy-trust', projectId),
+ trust: join(policyTrustRoot(), projectId),
});
return root;
}
diff --git a/v3/@claude-flow/cli/scripts/benchmark-policy-ledger-audit.mjs b/v3/@claude-flow/cli/scripts/benchmark-policy-ledger-audit.mjs
new file mode 100644
index 0000000000..520811aaaa
--- /dev/null
+++ b/v3/@claude-flow/cli/scripts/benchmark-policy-ledger-audit.mjs
@@ -0,0 +1,73 @@
+#!/usr/bin/env node
+// Synthetic, source-bound ledger benchmark. Operates only in an explicitly
+// supplied scratch fixture; no automatic project discovery or live data paths.
+import { createRequire } from 'node:module';
+import { createHash } from 'node:crypto';
+import { closeSync, existsSync, mkdirSync, openSync, readFileSync, realpathSync, statSync, writeFileSync, writeSync } from 'node:fs';
+import { resolve, join } from 'node:path';
+import { performance } from 'node:perf_hooks';
+const require = createRequire(import.meta.url);
+const [mode, suppliedRoot, suppliedBundle, suppliedCount] = process.argv.slice(2);
+if (!['generate', 'audit', 'preflight', 'transaction', 'prepare'].includes(mode) || !suppliedRoot || !suppliedBundle) throw new Error('usage: generate|prepare|audit|transaction <owned-scratch-root> <source-bound-bundle> [count]');
+const root = resolve(suppliedRoot);
+if (!root.startsWith('/tmp/ruflo-3892-benchmark-')) throw new Error('benchmark requires an owned /tmp/ruflo-3892-benchmark-* fixture');
+process.env.HOME = join(root, 'scratch-home');
+mkdirSync(process.env.HOME, { recursive: true, mode: 0o700 });
+process.env.XDG_CONFIG_HOME = join(root, 'scratch-config');
+process.env.RUFLO_DAEMON_AUTOSTART = '0';
+const runtime = require(resolve(suppliedBundle));
+const policy = join(root, '.claude-flow', 'policy');
+const trustRoot = join(root, 'scratch-config', 'ruflo', 'policy-trust');
+process.env.XDG_CONFIG_HOME = join(root, 'scratch-config');
+const digestFile = file => createHash('sha256').update(readFileSync(file)).digest('hex');
+function bufferedWriter(file, header = '') {
+ const fd = openSync(file, 'wx', 0o600); let buffered = header;
+ return { put(text) { buffered += text; if (buffered.length >= 65536) { writeSync(fd, buffered); buffered = ''; } }, close(footer = '') { buffered += footer; if (buffered) writeSync(fd, buffered); closeSync(fd); } };
+}
+if (mode === 'generate') {
+ if (existsSync(policy)) throw new Error('refuse to overwrite an existing fixture');
+ const count = Number(suppliedCount ?? 170000); const retention = 2000;
+ if (!Number.isSafeInteger(count) || count <= retention) throw new Error('invalid synthetic count');
+ mkdirSync(policy, { recursive: true, mode: 0o700 });
+ const archiveFile = join(policy, 'receipts.ledger.jsonl'); const anchorFile = join(policy, 'ledger-anchors.json');
+ const archive = bufferedWriter(archiveFile); const anchors = bufferedWriter(anchorFile, '{"version":1,"entries":[');
+ const hot = []; let previous = null; let prevAnchor = null; let archivedHead = null; let lastAnchor;
+ const policyHash = runtime.policyHash([]);
+ for (let sequence = 0; sequence < count; sequence++) {
+ const unsigned = { previousReceiptHash: previous, sequence, issuedAt: 1000 + sequence,
+ request: { identity: { id: `synthetic-agent:${sequence % 16}`, type: 'agent' }, action: { type: 'code.read', resource: `synthetic-${sequence}-${'x'.repeat(600)}` } },
+ decision: { outcome: 'allowed', enforcedOutcome: 'allowed', reason: 'synthetic benchmark fixture, not production evidence' }, policyHash };
+ const payload = { ...unsigned, receiptId: runtime.policyHash(unsigned) }; const receipt = { payload, hash: runtime.policyHash(payload) };
+ hot.push(receipt); if (hot.length > retention) { const removed = hot.shift(); archive.put(`${JSON.stringify(removed)}\n`); archivedHead = removed.hash; }
+ previous = receipt.hash;
+ const retainedFrom = Math.max(0, sequence + 1 - retention);
+ const entry = { seq: sequence, length: sequence + 1, headHash: receipt.hash, prevAnchorHash: prevAnchor, ts: 1000 + sequence, event: 'append',
+ ...(retainedFrom > 0 ? { archivedThrough: retainedFrom, archiveHeadHash: archivedHead } : {}) };
+ const fields = [entry.seq, entry.length, entry.headHash, entry.prevAnchorHash, entry.ts, entry.event, null];
+ if (entry.archivedThrough !== undefined) fields.push(entry.archivedThrough, entry.archiveHeadHash);
+ lastAnchor = { ...entry, hash: createHash('sha256').update(JSON.stringify(fields)).digest('hex') };
+ if (sequence) anchors.put(','); anchors.put(JSON.stringify(lastAnchor)); prevAnchor = lastAnchor.hash;
+ }
+ archive.close(); anchors.close(']}\n');
+ const state = { version: 1, mode: 'legacy', rules: [], budgets: [], usage: [], approvals: [], receipts: hot, retainedFrom: count - retention,
+ prunedHead: archivedHead, ledgerLength: count, ledgerHead: previous };
+ writeFileSync(join(policy, 'state.json'), JSON.stringify(state));
+ const id = createHash('sha256').update(realpathSync(root)).digest('hex'); mkdirSync(join(trustRoot, id), { recursive: true, mode: 0o700 });
+ writeFileSync(join(trustRoot, id, 'ledger-anchor-head.json'), JSON.stringify(lastAnchor));
+ const manifest = { schema: 'ruflo.synthetic-policy-ledger-benchmark/v1', synthetic: true, receipts: count, retention, anchorEntries: count,
+ archiveBytes: statSync(archiveFile).size, anchorBytes: statSync(anchorFile).size, archiveSha256: digestFile(archiveFile), anchorSha256: digestFile(anchorFile),
+ stateSha256: digestFile(join(policy, 'state.json')), finalReceiptHash: previous, signing: 'unsigned legacy fixture', fixtureBundleSha256: digestFile(resolve(suppliedBundle)) };
+ writeFileSync(join(root, 'synthetic-manifest.json'), JSON.stringify(manifest, null, 2)); console.log(JSON.stringify(manifest));
+} else {
+ if (!existsSync(join(root, 'synthetic-manifest.json'))) throw new Error('missing synthetic fixture marker');
+ const start = performance.now();
+ let outcome;
+ try {
+ if (mode === 'preflight') outcome = await runtime.verifyPolicyLedger(root);
+ if (mode === 'audit') outcome = await runtime.verifyPolicyLedger(root, { full: true });
+ if (mode === 'transaction') outcome = await runtime.evaluatePolicyRequest({ identity: { id: `synthetic-client:${process.pid}`, type: 'agent' }, action: { type: 'code.read', resource: `synthetic-transaction:${process.pid}` } }, root);
+ if (mode === 'prepare') outcome = await runtime.withPolicyTransaction(root, () => undefined);
+ } catch (error) { outcome = { error: error.message }; process.exitCode = 1; }
+ console.log(JSON.stringify({ mode, node: process.version, synthetic: true, runtimeBundleSha256: digestFile(resolve(suppliedBundle)), elapsedMs: performance.now() - start,
+ maxRssKiB: process.resourceUsage().maxRSS, userCpuMs: process.resourceUsage().userCPUTime / 1000, systemCpuMs: process.resourceUsage().systemCPUTime / 1000, outcome, stateVersion: JSON.parse(readFileSync(join(policy, 'state.json'))).version }));
+}
diff --git a/v3/@claude-flow/cli/src/commands/policy.ts b/v3/@claude-flow/cli/src/commands/policy.ts
index 9589fddca4..d940d7f20c 100644
--- a/v3/@claude-flow/cli/src/commands/policy.ts
+++ b/v3/@claude-flow/cli/src/commands/policy.ts
@@ -46,6 +46,7 @@ export const policyCommand: Command = {
{ name: 'mode', type: 'string', description: 'Policy mode: legacy | observe | enforce' },
{ name: 'project-root', type: 'string', description: 'Project root containing .claude-flow/policy' },
{ name: 'establish-anchor', type: 'boolean', description: 'verify: explicitly anchor a ledger that has receipts but no anchor (interactive, logged with user and time)' },
+ { name: 'full', type: 'boolean', description: 'verify: additionally replay the archived receipts (receipts.ledger.jsonl) to verify the complete historical chain, not just the retained hot tail' },
],
async action(context: CommandContext): Promise<CommandResult> {
const args = (context as { args?: string[] }).args ?? [];
@@ -108,8 +109,12 @@ export const policyCommand: Command = {
}
if (operation === 'verify') {
const establishAnchor = flags['establish-anchor'] === true || flags.establishAnchor === true;
+ const full = flags.full === true;
if (establishAnchor) requireInteractiveAdministrator();
- const ledger = await verifyPolicyLedger(root, { establishAnchor });
+ const ledger = await verifyPolicyLedger(root, { establishAnchor, full });
+ if (ledger.error === 'receipt-key-id-mismatch') {
+ output.printError('Restore the original CLAUDE_FLOW_POLICY_KEY_ID for this ledger; do not rewrite or re-sign historical receipts.');
+ }
const result = print({ ...ledger, anchors: anchorPosture(root), scope: LEDGER_TRUST_SCOPE });
return ledger.valid ? result : { ...result, success: false, exitCode: 1 };
}
diff --git a/v3/@claude-flow/cli/src/mcp-tools/policy-tools.ts b/v3/@claude-flow/cli/src/mcp-tools/policy-tools.ts
index 3c1fdeb0a3..a9986790fd 100644
--- a/v3/@claude-flow/cli/src/mcp-tools/policy-tools.ts
+++ b/v3/@claude-flow/cli/src/mcp-tools/policy-tools.ts
@@ -48,9 +48,11 @@ export const policyTools: MCPTool[] = [
category: 'security',
inputSchema: {
type: 'object',
- properties: {},
+ properties: {
+ full: { type: 'boolean', description: 'Also replay the archived receipts to verify the complete historical ledger, not just the retained hot tail (#3164)' },
+ },
},
- handler: async (_input, context) => {
+ handler: async (input, context) => {
const root = typeof context?.projectRoot === 'string' ? context.projectRoot : process.cwd();
const state = loadPolicyState(root);
return {
@@ -63,7 +65,7 @@ export const policyTools: MCPTool[] = [
approvals: state.approvals.length,
receipts: state.receipts.length,
},
- ledger: await verifyPolicyLedger(root),
+ ledger: await verifyPolicyLedger(root, { full: input.full === true }),
};
},
},
diff --git a/v3/@claude-flow/cli/src/services/policy-anchor-log.ts b/v3/@claude-flow/cli/src/services/policy-anchor-log.ts
new file mode 100644
index 0000000000..8d6404dfba
--- /dev/null
+++ b/v3/@claude-flow/cli/src/services/policy-anchor-log.ts
@@ -0,0 +1,159 @@
+/** Streaming reader/writer for the existing version-1 anchor JSON object.
+ * Every entry is still parsed and authenticated by the caller. This bounds
+ * memory without weakening historical validation or changing the log format.
+ */
+import { closeSync, constants, fstatSync, openSync, readSync, writeSync } from 'node:fs';
+import { TextDecoder } from 'node:util';
+
+export const ATOMIC_JSON_WRITER = Symbol('policy-atomic-json-writer');
+export interface AtomicJsonSource { [ATOMIC_JSON_WRITER](fd: number): void }
+const CHUNK_BYTES = 64 * 1024;
+const VALUE_LIMIT = 16 * 1024 * 1024;
+const STRING_STRUCTURE = /["\\]/g;
+const VALUE_STRUCTURE = /["\\{}\[\]]/g;
+
+class JsonReader {
+ private readonly fd: number;
+ private readonly buffer = Buffer.allocUnsafe(CHUNK_BYTES);
+ private readonly decoder = new TextDecoder('utf-8', { fatal: true });
+ private text = '';
+ private cursor = 0;
+ private position = 0;
+ private ended = false;
+ constructor(file: string) {
+ this.fd = openSync(file, constants.O_RDONLY | (constants.O_NOFOLLOW ?? 0) | (constants.O_NONBLOCK ?? 0));
+ try { if (!fstatSync(this.fd).isFile()) throw new Error('policy-anchor-log-corrupt'); }
+ catch (error) { closeSync(this.fd); throw error; }
+ }
+ close(): void { closeSync(this.fd); }
+ peek(): string {
+ while (this.cursor >= this.text.length && !this.ended) {
+ const count = readSync(this.fd, this.buffer, 0, this.buffer.length, this.position);
+ this.position += count;
+ this.text = this.decoder.decode(this.buffer.subarray(0, count), { stream: count !== 0 });
+ this.cursor = 0;
+ if (!count) this.ended = true;
+ }
+ return this.text[this.cursor] ?? '';
+ }
+ next(): string { const character = this.peek(); if (character) this.cursor++; return character; }
+ whitespace(): void { while (/[ \t\r\n]/.test(this.peek()) && this.peek()) this.next(); }
+ expect(character: string): void { this.whitespace(); if (this.next() !== character) throw new Error('policy-anchor-log-corrupt'); }
+ value(): unknown {
+ this.whitespace(); const first = this.peek();
+ if (!first) throw new Error('policy-anchor-log-corrupt');
+ const compound = first === '{' || first === '[';
+ const string = first === '"';
+ const parts: string[] = [];
+ let bytes = 0; let depth = 0; let quoted = false; let escaped = false;
+ for (;;) {
+ if (!this.peek()) {
+ if (quoted || depth !== 0 || !parts.length) throw new Error('policy-anchor-log-corrupt');
+ break;
+ }
+ const start = this.cursor;
+ let end = start; let complete = false;
+ // Scan the available segment directly. Avoid per-character method,
+ // regular-expression and string-concatenation overhead on mature logs.
+ for (; end < this.text.length; end++) {
+ if (!escaped && (quoted || compound)) {
+ const markers = quoted ? STRING_STRUCTURE : VALUE_STRUCTURE;
+ markers.lastIndex = end;
+ const next = markers.exec(this.text);
+ if (!next) { end = this.text.length; break; }
+ end = next.index;
+ }
+ const code = this.text.charCodeAt(end);
+ if (!quoted && depth === 0 && (parts.length > 0 || end > start) && !compound && !string
+ && (code === 32 || code === 9 || code === 10 || code === 13 || code === 44 || code === 125 || code === 93)) {
+ complete = true; break;
+ }
+ if (quoted) {
+ if (escaped) escaped = false;
+ else if (code === 92) escaped = true;
+ else if (code === 34) { quoted = false; if (string && depth === 0) { end++; complete = true; break; } }
+ } else if (code === 34) quoted = true;
+ else if (code === 123 || code === 91) depth++;
+ else if (code === 125 || code === 93) {
+ depth--; if (depth < 0) throw new Error('policy-anchor-log-corrupt');
+ if (compound && depth === 0) { end++; complete = true; break; }
+ }
+ }
+ const segment = this.text.slice(start, end);
+ bytes += Buffer.byteLength(segment);
+ if (bytes > VALUE_LIMIT) throw new Error('policy-anchor-log-record-too-large');
+ if (segment) parts.push(segment);
+ this.cursor = end;
+ if (complete) break;
+ }
+ // JSON.parse validates the complete bounded token's grammar, including
+ // bracket kinds, escapes, control characters and scalar spellings.
+ return JSON.parse(parts.join(''));
+ }
+}
+
+/** Supports property order/whitespace from ordinary JSON writers. Unknown
+ * root values remain bounded and syntax-checked; duplicate known keys fail. */
+export function* anchorEntries(file: string): Generator<unknown> {
+ const reader = new JsonReader(file);
+ let version: unknown; let seenVersion = false; let seenEntries = false;
+ try {
+ reader.expect('{'); reader.whitespace();
+ if (reader.peek() === '}') throw new Error('policy-anchor-log-corrupt');
+ for (;;) {
+ const key = reader.value(); if (typeof key !== 'string') throw new Error('policy-anchor-log-corrupt');
+ reader.expect(':');
+ if (key === 'entries') {
+ if (seenEntries) throw new Error('policy-anchor-log-corrupt'); seenEntries = true;
+ reader.expect('['); reader.whitespace();
+ if (reader.peek() !== ']') {
+ for (;;) {
+ yield reader.value(); reader.whitespace();
+ if (reader.peek() === ']') break;
+ reader.expect(',');
+ }
+ }
+ reader.expect(']');
+ } else {
+ const value = reader.value();
+ if (key === 'version') { if (seenVersion) throw new Error('policy-anchor-log-corrupt'); seenVersion = true; version = value; }
+ }
+ reader.whitespace(); if (reader.peek() === '}') break; reader.expect(',');
+ }
+ reader.expect('}'); reader.whitespace();
+ if (reader.peek() || !seenEntries || !seenVersion || version !== 1) throw new Error('policy-anchor-log-corrupt');
+ } finally { reader.close(); }
+}
+function writeText(fd: number, text: string): void {
+ const bytes = Buffer.from(text); let count = 0;
+ while (count < bytes.length) {
+ const next = writeSync(fd, bytes, count, bytes.length - count);
+ if (!next) throw new Error('policy-anchor-log-short-write'); count += next;
+ }
+}
+/** Serialization callback runs inside the runtime's existing durable atomic
+ * writer. Failure removes the temporary file; no partial log is published. */
+export function appendedAnchorLog(file: string | undefined, next: unknown, validate: (value: unknown, sequence: number, previous: string | null) => value is { hash: string }): AtomicJsonSource {
+ return { [ATOMIC_JSON_WRITER](fd: number): void {
+ let pending = ''; let pendingBytes = 0;
+ const flush = () => { if (pendingBytes) writeText(fd, pending); pending = ''; pendingBytes = 0; };
+ const emit = (text: string) => {
+ const bytes = Buffer.byteLength(text);
+ if (pendingBytes + bytes > CHUNK_BYTES) flush();
+ if (bytes >= CHUNK_BYTES) writeText(fd, text);
+ else { pending += text; pendingBytes += bytes; }
+ };
+ emit('{"version":1,"entries":[\n');
+ let count = 0; let previous: string | null = null;
+ if (file) {
+ for (const entry of anchorEntries(file)) {
+ if (!validate(entry, count, previous)) throw new Error('policy-anchor-log-corrupt');
+ if (count) emit(',\n'); emit(JSON.stringify(entry));
+ count++; previous = entry.hash;
+ }
+ }
+ if (!validate(next, count, previous)) throw new Error('policy-anchor-log-corrupt');
+ if (count) emit(',\n'); emit(JSON.stringify(next));
+ emit('\n]}\n'); flush();
+ } };
+}
diff --git a/v3/@claude-flow/cli/src/services/policy-archive-scan.ts b/v3/@claude-flow/cli/src/services/policy-archive-scan.ts
new file mode 100644
index 0000000000..c5be99d391
--- /dev/null
+++ b/v3/@claude-flow/cli/src/services/policy-archive-scan.ts
@@ -0,0 +1,39 @@
+/** Fixed-chunk reverse archive scan used by the synchronous anchor gate.
+ * Memory follows the largest single line, never the entire archive/window.
+ */
+import { readSync } from 'node:fs';
+const CHUNK_BYTES = 64 * 1024;
+export interface ReverseRecord { bytes: Buffer; offset: number; terminated: boolean }
+export function* reverseArchiveRecords(fd: number, size: number, limit: number): Generator<ReverseRecord> {
+ let position = size;
+ let parts: Buffer[] = [];
+ let partBytes = 0;
+ let terminated = false;
+ while (position > 0) {
+ const count = Math.min(CHUNK_BYTES, position);
+ position -= count;
+ const chunk = Buffer.allocUnsafe(count);
+ let read = 0;
+ while (read < count) {
+ const next = readSync(fd, chunk, read, count - read, position + read);
+ if (!next) throw new Error('policy-ledger-archive-changed'); read += next;
+ }
+ let end = chunk.length;
+ for (;;) {
+ const newline = chunk.lastIndexOf(10, end - 1);
+ if (newline < 0) break;
+ const piece = chunk.subarray(newline + 1, end);
+ if (piece.length + partBytes > limit) throw new Error(`policy-ledger-archive-record-too-large:${limit}`);
+ const bytes = parts.length ? Buffer.concat([piece, ...parts], piece.length + partBytes) : Buffer.from(piece);
+ yield { bytes, offset: position + newline + 1, terminated };
+ terminated = true;
+ parts = []; partBytes = 0; end = newline;
+ if (end === 0) break;
+ }
+ if (end > 0) {
+ if (partBytes + end > limit) throw new Error(`policy-ledger-archive-record-too-large:${limit}`);
+ const piece = Buffer.from(chunk.subarray(0, end)); parts.unshift(piece); partBytes += piece.length;
+ }
+ }
+ if (partBytes > 0) yield { bytes: Buffer.concat(parts, partBytes), offset: 0, terminated };
+}
diff --git a/v3/@claude-flow/cli/src/services/policy-ledger-anchor.ts b/v3/@claude-flow/cli/src/services/policy-ledger-anchor.ts
index 9dc6968790..9626871498 100644
--- a/v3/@claude-flow/cli/src/services/policy-ledger-anchor.ts
+++ b/v3/@claude-flow/cli/src/services/policy-ledger-anchor.ts
@@ -10,12 +10,29 @@
* Threat model: this detects any tampering confined to state.json, and any
* tampering confined to the project directory. It does NOT stop an attacker
* who can rewrite both the project directory and the user's home directory.
+ *
+ * Archive coverage (#3892): once receipts have been pruned, the default
+ * (non-`--full`) verify path checks the archive's TIP only β€” it confirms
+ * the archive's newest entry still matches what the anchor recorded, in
+ * bounded time, catching deletion, truncation, and tampering of that tip
+ * entry. It does NOT walk `previousReceiptHash` back through the rest of
+ * the archive, so a surgical edit to an INTERIOR archived receipt that
+ * leaves the tip entry (and everything chained after it) byte-for-byte
+ * untouched is NOT detected by default verify. Only
+ * `verifyPolicyLedger({ full: true })` (`ruflo policy verify --full`),
+ * which replays the entire archive, closes that gap. This is the inherent
+ * trade-off of a cheap tip check versus an O(total-history) replay, not an
+ * oversight β€” closing it fully on the default path would reintroduce the
+ * unbounded per-transaction cost #3164 removed.
*/
+import { decodeLedgerRecord, policyLedgerRecordLimit } from './policy-ledger-record.js';
+import { reverseArchiveRecords } from './policy-archive-scan.js';
+import { anchorEntries, appendedAnchorLog, type AtomicJsonSource } from './policy-anchor-log.js';
import { createHash } from 'node:crypto';
-import { existsSync, mkdirSync, readFileSync, realpathSync } from 'node:fs';
-import { dirname, isAbsolute, join, resolve } from 'node:path';
-import { homedir, userInfo } from 'node:os';
-import type { PolicyState } from '@claude-flow/security';
+import { closeSync, constants as fsConstants, fstatSync, existsSync, mkdtempSync, chmodSync, openSync, readFileSync, readSync, realpathSync, rmSync, writeFileSync } from 'node:fs';
+import { isAbsolute, join, resolve } from 'node:path';
+import { homedir, tmpdir, userInfo } from 'node:os';
+import { AgenticPolicyEngine, canonicalizePolicy, type PolicyState } from '@claude-flow/security/policy/index';
export type AnchorEvent = 'append' | 'migrated-from-state' | 'establish-anchor';
@@ -27,6 +44,15 @@ export interface AnchorEntry {
ts: number;
event: AnchorEvent;
by?: string;
+ /**
+ * How much of the archive this anchor claims (`state.retainedFrom` at
+ * anchor time) and the hash of its newest archived receipt (#3892 Blocker
+ * 1). Absent on anchors written before this field existed, or for a
+ * ledger that has never been pruned β€” both resolve to a vacuous pass so
+ * the unpruned baseline stays byte-for-byte identical (see `entryHash`).
+ */
+ archivedThrough?: number;
+ archiveHeadHash?: string | null;
hash: string;
}
@@ -36,6 +62,7 @@ export type AnchorAssessment =
| { kind: 'fail'; error: string };
export type AnchorWriter = (file: string, value: unknown) => Promise<void>;
+export type StreamingAnchorWriter = (file: string, value: AtomicJsonSource) => Promise<void>;
const LOG_FILE = 'ledger-anchors.json';
const MIRROR_FILE = 'ledger-anchor-head.json';
@@ -76,10 +103,167 @@ function mirrorPath(projectRoot: string): string {
return join(policyTrustRoot(), id, MIRROR_FILE);
}
+// Duplicated from policy-runtime.ts's RECEIPT_ARCHIVE_FILE rather than
+// imported, to avoid a circular dependency (policy-runtime.ts imports this
+// module) β€” same pattern already used by anchorLogPath above.
+const ARCHIVE_FILE = 'receipts.ledger.jsonl';
+function archiveLedgerPath(projectRoot: string): string {
+ return join(resolve(projectRoot), '.claude-flow', 'policy', ARCHIVE_FILE);
+}
+
+const TAIL_READ_INITIAL_BYTES = 64 * 1024;
+
+/**
+ * Whether the archive still contains a self-consistent copy of the receipt
+ * at `expectedSequence` whose own stored `hash` is `expectedHash` (#3892
+ * Blocker 1) β€” i.e. the archive's TIP, nothing more. Scans a trailing
+ * suffix in fixed 64KiB chunks until a match is found or the
+ * whole file has been scanned β€” NOT just the physical last line: a harmless
+ * crash-duplicate (#3164) can legitimately land after the true tail, so
+ * truncation/tampering is "the matching receipt is nowhere in the file",
+ * not "it isn't the very last line". "Self-consistent" means
+ * `policyHash(payload) === hash` β€” a tamper that edits a receipt's content
+ * but leaves its stale `hash` field untouched must not read back as a
+ * match.
+ *
+ * Coverage boundary (disclosed at the top of this file): this checks only
+ * the tip record itself, never `previousReceiptHash` back through the rest
+ * of the archive. An interior receipt tampered with its OWN hash
+ * recomputed to stay self-consistent, while the tip and everything after
+ * it are left untouched, passes this check β€” only `--full` catches that.
+ *
+ * Cost: bounded on the common (untampered) path, where the match is almost
+ * always inside the first 64KB suffix; a truncation/deletion attack (where
+ * no match exists anywhere) scans O(archive size) bytes with bounded
+ * per-record memory before reporting tampering β€” an acceptable cost on that rare failure path. On
+ * the healthy path this runs on every `withPolicyTransaction` call once
+ * `retainedFrom > 0` (via `checkArchiveClaim` β†’ `assessAnchors` β†’
+ * `reconcileLedger`), not only on explicit verify calls.
+ * This PR originally measured its un-synced tip reader separately; those
+ * timings do not describe the durable writer protocol added here.
+ * Crash catch-up checks only the newly archived extension from a trusted
+ * head; this rare path costs O(extension plus the trailing archive window).
+ */
+function archiveHasTailReceipt(filePath: string, expectedSequence: number, expectedHash: string, from?: { sequence: number; hash: string }): boolean {
+ if (!existsSync(filePath) || !Number.isSafeInteger(expectedSequence) || expectedSequence < 0) return false;
+ let fd: number;
+ try { fd = openSync(filePath, fsConstants.O_RDONLY | (fsConstants.O_NOFOLLOW ?? 0) | (fsConstants.O_NONBLOCK ?? 0)); } catch { return false; }
+ let scratch: string | undefined;
+ try {
+ const stat = fstatSync(fd);
+ if (!stat.isFile() || stat.size === 0) return false;
+ if (from && (!Number.isSafeInteger(from.sequence) || from.sequence >= expectedSequence || expectedSequence - from.sequence > stat.size)) return false;
+ const verifier = new AgenticPolicyEngine({ signingKey: process.env.CLAUDE_FLOW_POLICY_SIGNING_KEY, keyId: process.env.CLAUDE_FLOW_POLICY_KEY_ID });
+ if (from) { scratch = mkdtempSync(join(tmpdir(), 'ruflo-policy-extension-')); chmodSync(scratch, 0o700); }
+ let found: string | undefined; let extensionCount = 0;
+ for (const physical of reverseArchiveRecords(fd, stat.size, policyLedgerRecordLimit())) {
+ let line: string;
+ try { line = decodeLedgerRecord(physical.bytes); }
+ catch (error) {
+ // An unacknowledged final append may end inside a UTF8 code point.
+ // Preserve it here; the writer can repair only an exact state-backed
+ // byte prefix. Newline-complete records must always decode strictly.
+ if (!physical.terminated && (error as Error).message === 'policy-ledger-archive-utf8-error') continue;
+ throw error;
+ }
+ if (!line.trim()) continue;
+ let parsed: any;
+ try { parsed = JSON.parse(line); } catch { continue; }
+ const sequence = parsed?.payload?.sequence;
+ if (sequence === expectedSequence) {
+ if (verifier.verifyReceipt(parsed) || parsed.hash !== expectedHash) return false;
+ const canonical = canonicalizePolicy(parsed);
+ if (found !== undefined && found !== canonical) return false; found = canonical;
+ }
+ if (from && Number.isSafeInteger(sequence) && sequence > from.sequence && sequence <= expectedSequence) {
+ if (verifier.verifyReceipt(parsed)) return false;
+ const canonical = canonicalizePolicy(parsed); const file = join(scratch!, `receipt-${sequence}`);
+ try { writeFileSync(file, canonical, { mode: 0o600, flag: 'wx' }); extensionCount++; }
+ catch (error) {
+ if ((error as NodeJS.ErrnoException).code !== 'EEXIST') throw error;
+ const other = openSync(file, fsConstants.O_RDONLY | (fsConstants.O_NOFOLLOW ?? 0) | (fsConstants.O_NONBLOCK ?? 0));
+ try { if (!fstatSync(other).isFile() || readFileSync(other, 'utf8') !== canonical) return false; }
+ finally { closeSync(other); }
+ }
+ }
+ // Inspect at least the original healthy 64KiB suffix, including all
+ // matching duplicates there, before accepting a found tip.
+ if (found !== undefined && physical.offset <= Math.max(0, stat.size - TAIL_READ_INITIAL_BYTES)) {
+ if (!from) return true;
+ if (extensionCount === expectedSequence - from.sequence) break;
+ }
+ }
+ if (found === undefined) return false;
+ if (!from) return true;
+ if (extensionCount !== expectedSequence - from.sequence) return false;
+ let previous = from.hash;
+ for (let sequence = from.sequence + 1; sequence <= expectedSequence; sequence++) {
+ const other = openSync(join(scratch!, `receipt-${sequence}`), fsConstants.O_RDONLY | (fsConstants.O_NOFOLLOW ?? 0) | (fsConstants.O_NONBLOCK ?? 0));
+ try {
+ if (!fstatSync(other).isFile()) return false;
+ const receipt = JSON.parse(readFileSync(other, 'utf8'));
+ if (verifier.verifyReceipt(receipt) || receipt.payload.previousReceiptHash !== previous) return false;
+ previous = receipt.hash;
+ } finally { closeSync(other); }
+ }
+ return previous === expectedHash;
+ } catch (error) {
+ if ((error as Error).message.startsWith('policy-ledger-archive-record-too-large:')
+ || (error as Error).message === 'policy-ledger-archive-utf8-error'
+ || (error as Error).message === 'policy-ledger-audit-invalid-record-limit') throw error;
+ return false;
+ }
+ finally { closeSync(fd); if (scratch !== undefined) rmSync(scratch, { recursive: true, force: true }); }
+}
+
+/** #3164: `state.receipts` is only the hot tail once pruning is in play β€” the
+ * anchor log always speaks in terms of the full historical ledger length. */
+function fullLength(state: PolicyState): number {
+ return (state.retainedFrom ?? 0) + state.receipts.length;
+}
+
+/**
+ * Hash of the receipt at absolute (historical) 1-based length
+ * `absoluteLength`, or `undefined` if it is neither in the hot tail nor
+ * exactly the boundary receipt this transaction just pruned (#3164).
+ * `recordAnchor` runs every transaction, so the anchor it reads here is
+ * always at most one transaction behind the current state β€” never further
+ * back than the just-pruned boundary.
+ */
+function receiptHashAt(state: PolicyState, absoluteLength: number): string | undefined {
+ const retainedFrom = state.retainedFrom ?? 0;
+ const index = absoluteLength - 1;
+ if (index === retainedFrom - 1) return state.prunedHead ?? undefined;
+ if (index >= retainedFrom) return state.receipts[index - retainedFrom]?.hash;
+ return undefined;
+}
+
+/**
+ * The two archive fields are appended to the hashed array ONLY when at
+ * least one of them is set on this entry (#3892 Blocker 1). An anchor
+ * written before the fields existed, or for a ledger that was never
+ * pruned, therefore hashes over the exact same array as before this
+ * change β€” appending `null, null` unconditionally would change the
+ * hashed byte string and break every anchor already on disk.
+ */
+function anchorHeadMatches(projectRoot: string, state: PolicyState, anchor: AnchorEntry): boolean {
+ const hotHash = receiptHashAt(state, anchor.length);
+ if (hotHash !== undefined) return hotHash === anchor.headHash;
+ const retainedFrom = state.retainedFrom ?? 0;
+ return anchor.length < retainedFrom && typeof anchor.headHash === 'string' && typeof state.prunedHead === 'string'
+ && archiveHasTailReceipt(archiveLedgerPath(projectRoot), anchor.length - 1, anchor.headHash)
+ && archiveHasTailReceipt(archiveLedgerPath(projectRoot), retainedFrom - 1, state.prunedHead,
+ { sequence: anchor.length - 1, hash: anchor.headHash });
+}
+
function entryHash(entry: Omit<AnchorEntry, 'hash'>): string {
- return createHash('sha256').update(JSON.stringify([
+ const fields: unknown[] = [
entry.seq, entry.length, entry.headHash, entry.prevAnchorHash, entry.ts, entry.event, entry.by ?? null,
- ])).digest('hex');
+ ];
+ if (entry.archivedThrough !== undefined || entry.archiveHeadHash !== undefined) {
+ fields.push(entry.archivedThrough ?? null, entry.archiveHeadHash ?? null);
+ }
+ return createHash('sha256').update(JSON.stringify(fields)).digest('hex');
}
function validEntry(value: unknown, seq: number, prev: string | null): value is AnchorEntry {
@@ -91,25 +275,30 @@ function validEntry(value: unknown, seq: number, prev: string | null): value is
&& e.prevAnchorHash === prev
&& typeof e.ts === 'number'
&& ['append', 'migrated-from-state', 'establish-anchor'].includes(e.event)
+ && (e.archivedThrough === undefined || (Number.isInteger(e.archivedThrough) && e.archivedThrough >= 0))
+ && (e.archiveHeadHash === undefined || e.archiveHeadHash === null || typeof e.archiveHeadHash === 'string')
&& typeof e.hash === 'string'
&& e.hash === entryHash({ ...e, hash: undefined } as never);
}
-function readLog(projectRoot: string): AnchorEntry[] | 'corrupt' {
+interface LogSummary { count: number; last?: AnchorEntry; mirrorIncluded: boolean }
+// Identity-only, single-use assessment metadata. Plain JSON cannot grant reuse.
+// Callers hold the policy lock across assessment and publication; the append
+// serializer independently authenticates the entire source and binds count/tip.
+const assessedLogs = new WeakMap<AnchorEntry, { file: string; summary: LogSummary }>();
+function readLog(projectRoot: string, mirrorHash?: string): LogSummary | 'corrupt' {
const file = anchorLogPath(projectRoot);
- if (!existsSync(file)) return [];
+ if (!existsSync(file)) return { count: 0, mirrorIncluded: false };
try {
- const parsed = JSON.parse(readFileSync(file, 'utf8')) as { version?: number; entries?: unknown[] };
- if (parsed.version !== 1 || !Array.isArray(parsed.entries)) return 'corrupt';
let prev: string | null = null;
- for (let i = 0; i < parsed.entries.length; i++) {
- if (!validEntry(parsed.entries[i], i, prev)) return 'corrupt';
- prev = (parsed.entries[i] as AnchorEntry).hash;
+ const summary: LogSummary = { count: 0, mirrorIncluded: false };
+ for (const value of anchorEntries(file)) {
+ if (!validEntry(value, summary.count, prev)) return 'corrupt';
+ summary.count++; summary.last = value; prev = value.hash;
+ if (mirrorHash !== undefined && value.hash === mirrorHash) summary.mirrorIncluded = true;
}
- return parsed.entries as AnchorEntry[];
- } catch {
- return 'corrupt';
- }
+ return summary;
+ } catch { return 'corrupt'; }
}
function readMirror(projectRoot: string): AnchorEntry | null | 'corrupt' {
@@ -125,43 +314,100 @@ function readMirror(projectRoot: string): AnchorEntry | null | 'corrupt' {
}
}
+/**
+ * Cross-checks one anchor's claim about the archive against current state
+ * and, when that claim is self-consistent, against the live archive file
+ * on disk (#3892 Blocker 1). `undefined` means no mismatch was found β€”
+ * including the legitimate transitional case of an anchor written before
+ * archive-anchoring existed, which `behind` catches up separately.
+ *
+ * Tip check only β€” see `archiveHasTailReceipt`'s doc comment and the
+ * file-level comment for the coverage boundary (an interior-archive
+ * tamper that leaves the tip untouched is not caught here) and the
+ * measured per-transaction cost this adds once `retainedFrom > 0`.
+ */
+function checkArchiveClaim(projectRoot: string, state: PolicyState, anchor: AnchorEntry): string | undefined {
+ const retainedFrom = state.retainedFrom ?? 0;
+ if (retainedFrom === 0) return undefined;
+ const archiveHeadHash = state.prunedHead ?? null;
+ if (archiveHeadHash === null
+ || !archiveHasTailReceipt(archiveLedgerPath(projectRoot), retainedFrom - 1, archiveHeadHash)) {
+ return 'policy-ledger-archive-tampered';
+ }
+ if (anchor.archivedThrough === undefined) return undefined;
+ if (anchor.archivedThrough > retainedFrom
+ || (anchor.archivedThrough === retainedFrom && (anchor.archiveHeadHash ?? null) !== archiveHeadHash)) {
+ return 'policy-ledger-anchor-mismatch';
+ }
+ // State is published before the secondary anchors. A crash can therefore
+ // leave an older, durable archive claim. Preserve and authenticate that
+ // older tip plus the state's current tip before catching anchors up.
+ if (anchor.archivedThrough > 0 && (!anchor.archiveHeadHash
+ || !archiveHasTailReceipt(archiveLedgerPath(projectRoot), anchor.archivedThrough - 1, anchor.archiveHeadHash))) {
+ return 'policy-ledger-archive-tampered';
+ }
+ return undefined;
+}
+
/**
* How the stored anchors relate to `state`; read-only. `rebuild` is the
* explicit-establish path: an unreadable log or mirror is treated as absent, but
* whatever readable anchor remains must still agree with the chain, so it can
* never bless a truncation the anchors still remember.
*/
-export function assessAnchors(projectRoot: string, state: PolicyState, rebuild = false): AnchorAssessment {
- let entries = readLog(projectRoot);
- if (entries === 'corrupt') {
- if (!rebuild) return { kind: 'fail', error: 'policy-anchor-log-corrupt' };
- entries = [];
- }
+function assessAnchorsChecked(projectRoot: string, state: PolicyState, rebuild = false): AnchorAssessment {
let mirror = readMirror(projectRoot);
if (mirror === 'corrupt') {
if (!rebuild) return { kind: 'fail', error: 'policy-anchor-mirror-corrupt' };
mirror = null;
}
- if (entries.length === 0) {
+ let entries = readLog(projectRoot, mirror?.hash);
+ if (entries === 'corrupt') {
+ if (!rebuild) return { kind: 'fail', error: 'policy-anchor-log-corrupt' };
+ entries = { count: 0, mirrorIncluded: false };
+ }
+ const retainedFrom = state.retainedFrom ?? 0;
+ if (entries.count === 0) {
if (!mirror) return { kind: 'absent' };
if (!rebuild) return { kind: 'fail', error: 'policy-anchor-log-missing' };
- if (state.receipts.length < mirror.length) return { kind: 'fail', error: 'policy-ledger-truncated' };
- if (mirror.length > 0 && state.receipts[mirror.length - 1]!.hash !== mirror.headHash) {
+ if (fullLength(state) < mirror.length) return { kind: 'fail', error: 'policy-ledger-truncated' };
+ if (mirror.length > 0 && !anchorHeadMatches(projectRoot, state, mirror)) {
return { kind: 'fail', error: 'policy-ledger-anchor-mismatch' };
}
+ const archiveError = checkArchiveClaim(projectRoot, state, mirror);
+ if (archiveError) return { kind: 'fail', error: archiveError };
return { kind: 'absent' };
}
- if (mirror && !entries.some((entry) => entry.hash === mirror.hash)) {
+ if (mirror && !entries.mirrorIncluded) {
return { kind: 'fail', error: 'policy-anchor-log-rolled-back' };
}
- const last = entries[entries.length - 1]!;
+ const last = entries.last!;
+ // A failed best-effort mirror write may leave a genuine earlier entry,
+ // including an entry from before the first prune. Membership in the fully
+ // validated log above plus the trusted-head/extension checks below grant
+ // catch-up authority; asymmetric archive fields alone do not prove tampering.
for (const anchor of mirror ? [last, mirror] : [last]) {
- if (state.receipts.length < anchor.length) return { kind: 'fail', error: 'policy-ledger-truncated' };
- if (anchor.length > 0 && state.receipts[anchor.length - 1]!.hash !== anchor.headHash) {
+ if (fullLength(state) < anchor.length) return { kind: 'fail', error: 'policy-ledger-truncated' };
+ if (anchor.length > 0 && !anchorHeadMatches(projectRoot, state, anchor)) {
return { kind: 'fail', error: 'policy-ledger-anchor-mismatch' };
}
+ const archiveError = checkArchiveClaim(projectRoot, state, anchor);
+ if (archiveError) return { kind: 'fail', error: archiveError };
+ }
+ const behind = fullLength(state) > last.length
+ || (retainedFrom > 0 && last.archivedThrough === undefined);
+ Object.freeze(last);
+ assessedLogs.set(last, { file: anchorLogPath(projectRoot), summary: entries });
+ return { kind: 'ok', last, behind };
+}
+
+/** Convert archive record-limit/UTF8 failures into structured gate failures. */
+export function assessAnchors(projectRoot: string, state: PolicyState, rebuild = false): AnchorAssessment {
+ try { return assessAnchorsChecked(projectRoot, state, rebuild); }
+ catch (error) {
+ const message = (error as Error).message;
+ return { kind: 'fail', error: message.startsWith('policy-ledger-') ? message : 'policy-ledger-archive-read-error' };
}
- return { kind: 'ok', last, behind: state.receipts.length > last.length };
}
/**
@@ -175,31 +421,55 @@ export async function recordAnchor(
write: AnchorWriter,
event: AnchorEvent = 'append',
by?: string,
+ writeLog?: StreamingAnchorWriter,
+ assessedTip?: AnchorEntry,
): Promise<void> {
- if (state.receipts.length === 0) return;
- const loaded = readLog(projectRoot);
- const entries = loaded === 'corrupt' ? [] : loaded;
+ const length = fullLength(state);
+ if (length === 0) return;
+ const cached = assessedTip && assessedLogs.get(assessedTip);
+ if (assessedTip) assessedLogs.delete(assessedTip);
+ // Reuse only for an actual append through the production streaming writer.
+ // Unchanged checkpoints still read the live log before refreshing the mirror.
+ const stateHead = receiptHashAt(state, length);
+ const changed = assessedTip && (assessedTip.length !== length || assessedTip.headHash !== stateHead
+ || assessedTip.archivedThrough !== ((state.retainedFrom ?? 0) > 0 ? state.retainedFrom : undefined)
+ || (assessedTip.archiveHeadHash ?? undefined) !== ((state.retainedFrom ?? 0) > 0 ? state.prunedHead ?? null : undefined));
+ const loaded = writeLog && changed && cached?.file === anchorLogPath(projectRoot)
+ ? cached.summary : readLog(projectRoot);
+ const entries = loaded === 'corrupt' ? { count: 0, mirrorIncluded: false } : loaded;
if (loaded === 'corrupt' && event !== 'establish-anchor') throw new Error('policy-anchor-log-corrupt');
- const last = entries[entries.length - 1];
- const length = state.receipts.length;
- const headHash = state.receipts[length - 1]!.hash;
+ const last = entries.last;
+ const headHash = receiptHashAt(state, length);
+ if (headHash === undefined) throw new Error('policy-ledger-anchor-head-unresolvable');
+ const retainedFrom = state.retainedFrom ?? 0;
+ const archivedThrough = retainedFrom > 0 ? retainedFrom : undefined;
+ const archiveHeadHash = retainedFrom > 0 ? (state.prunedHead ?? null) : undefined;
let tip = last;
- if (!last || last.length !== length || last.headHash !== headHash) {
+ // Deferred archival can advance the archive boundary without a new receipt.
+ if (!last || last.length !== length || last.headHash !== headHash
+ || last.archivedThrough !== archivedThrough || (last.archiveHeadHash ?? undefined) !== archiveHeadHash) {
const base = {
- seq: entries.length,
+ seq: entries.count,
length,
headHash,
prevAnchorHash: last?.hash ?? null,
ts: Date.now(),
event,
...(by ? { by } : {}),
+ ...(archivedThrough !== undefined ? { archivedThrough, archiveHeadHash } : {}),
};
tip = { ...base, hash: entryHash(base) };
- await write(anchorLogPath(projectRoot), { version: 1, entries: [...entries, tip] });
+ if (writeLog) {
+ await writeLog(anchorLogPath(projectRoot), appendedAnchorLog(loaded === 'corrupt' || entries.count === 0 ? undefined : anchorLogPath(projectRoot), tip, validEntry));
+ } else {
+ // Preserve the exported custom-writer contract: ordinary version-1 JSON.
+ // The production runtime supplies the bounded streaming writer above.
+ const history = loaded === 'corrupt' || entries.count === 0 ? [] : [...anchorEntries(anchorLogPath(projectRoot))];
+ await write(anchorLogPath(projectRoot), { version: 1, entries: [...history, tip] });
+ }
}
try {
const mirror = mirrorPath(projectRoot);
- mkdirSync(dirname(mirror), { recursive: true, mode: 0o700 });
await write(mirror, tip);
} catch (error) {
// A read-only home must not stop the ledger; verify reports the weaker
diff --git a/v3/@claude-flow/cli/src/services/policy-ledger-audit.ts b/v3/@claude-flow/cli/src/services/policy-ledger-audit.ts
new file mode 100644
index 0000000000..cc4f935ae6
--- /dev/null
+++ b/v3/@claude-flow/cli/src/services/policy-ledger-audit.ts
@@ -0,0 +1,206 @@
+/** Bounded-memory full policy audit. The caller owns the captured archive FD.
+ * Only bytes below its locked snapshot boundary are read; normal later appends
+ * do not join the audit. Disk scratch is private, disposable, never ledger state.
+ */
+import { AgenticPolicyEngine, canonicalizePolicy, type LedgerVerification, type PolicyReceipt, type PolicyState } from '@claude-flow/security/policy/index';
+import { createHash } from 'node:crypto';
+import { closeSync, chmodSync, constants, fstatSync, mkdtempSync, openSync, read, readSync, rmSync, writeSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { decodeLedgerRecord, policyLedgerRecordLimit } from './policy-ledger-record.js';
+
+export interface ArchiveSnapshot { fd: number; size: number }
+const CHUNK_BYTES = 64 * 1024;
+const INDEX_BYTES = 48;
+function fail(error: string): never { throw new Error(error); }
+
+/** Must run under the policy writer lock. O(1), with no whole-file read. */
+export function captureArchiveSnapshot(file: string): ArchiveSnapshot | undefined {
+ let fd: number;
+ try { fd = openSync(file, constants.O_RDONLY | (constants.O_NOFOLLOW ?? 0) | (constants.O_NONBLOCK ?? 0)); }
+ catch (error) {
+ if ((error as NodeJS.ErrnoException).code === 'ENOENT') return undefined;
+ fail('policy-ledger-archive-not-regular');
+ }
+ try {
+ const stat = fstatSync(fd);
+ if (!stat.isFile() || !Number.isSafeInteger(stat.size) || stat.size < 0) fail('policy-ledger-archive-not-regular');
+ return { fd, size: stat.size };
+ } catch (error) { closeSync(fd); throw error; }
+}
+
+function readAt(fd: number, buffer: Buffer, offset: number, length: number, position: number): Promise<number> {
+ return new Promise((resolve, reject) => read(fd, buffer, offset, length, position, (error, count) => error ? reject(error) : resolve(count)));
+}
+async function readExact(snapshot: ArchiveSnapshot, offset: number, length: number, limit: number): Promise<Buffer> {
+ if (!Number.isSafeInteger(offset) || offset < 0 || !Number.isSafeInteger(length) || length <= 0
+ || length > limit || offset > snapshot.size - length) fail('policy-ledger-archive-changed');
+ const buffer = Buffer.allocUnsafe(length);
+ let count = 0;
+ while (count < length) {
+ const next = await readAt(snapshot.fd, buffer, count, length - count, offset + count);
+ if (!next) fail('policy-ledger-archive-changed');
+ count += next;
+ }
+ return buffer;
+}
+interface PhysicalRecord { bytes: Buffer; offset: number }
+async function* records(snapshot: ArchiveSnapshot, limit: number): AsyncGenerator<PhysicalRecord> {
+ const chunk = Buffer.allocUnsafe(CHUNK_BYTES);
+ let position = 0;
+ let recordStart = 0;
+ let parts: Buffer[] = [];
+ let partBytes = 0;
+ while (position < snapshot.size) {
+ const count = await readAt(snapshot.fd, chunk, 0, Math.min(chunk.length, snapshot.size - position), position);
+ if (!count) fail('policy-ledger-archive-changed');
+ let cursor = 0;
+ while (cursor < count) {
+ const newline = chunk.indexOf(10, cursor);
+ const end = newline < 0 || newline >= count ? count : newline;
+ const piece = chunk.subarray(cursor, end);
+ partBytes += piece.length;
+ if (partBytes > limit) fail(`policy-ledger-archive-record-too-large:${limit}`);
+ if (end === count) { parts.push(Buffer.from(piece)); break; }
+ const bytes = parts.length ? Buffer.concat([...parts, piece], partBytes) : Buffer.from(piece);
+ yield { bytes, offset: recordStart };
+ recordStart = position + end + 1;
+ parts = [];
+ partBytes = 0;
+ cursor = end + 1;
+ }
+ position += count;
+ }
+ if (partBytes !== 0) fail('policy-ledger-archive-parse-error');
+}
+function parsed(bytes: Buffer): PolicyReceipt | undefined {
+ const text = decodeLedgerRecord(bytes);
+ if (!text.trim()) return undefined;
+ try { return JSON.parse(text) as PolicyReceipt; } catch { fail('policy-ledger-archive-parse-error'); }
+}
+const digest = (canonical: string): Buffer => createHash('sha256').update(canonical).digest();
+function writeAll(fd: number, bytes: Buffer, position: number): void {
+ let count = 0;
+ while (count < bytes.length) {
+ const next = writeSync(fd, bytes, count, bytes.length - count, position + count);
+ if (!next) fail('policy-ledger-audit-index-write-error');
+ count += next;
+ }
+}
+function readIndex(fd: number, sequence: number): Buffer {
+ const position = sequence * INDEX_BYTES;
+ if (!Number.isSafeInteger(position)) fail('policy-ledger-archive-gap');
+ const row = Buffer.alloc(INDEX_BYTES);
+ const count = readSync(fd, row, 0, row.length, position);
+ if (count !== 0 && count !== row.length) fail('policy-ledger-audit-index-read-error');
+ return row;
+}
+async function observedRecord(snapshot: ArchiveSnapshot, row: Buffer, limit: number, verifier: AgenticPolicyEngine): Promise<{ receipt: PolicyReceipt; canonical: string }> {
+ const offsetBig = row.readBigUInt64LE(0);
+ if (offsetBig > BigInt(Number.MAX_SAFE_INTEGER)) fail('policy-ledger-audit-index-read-error');
+ const bytes = await readExact(snapshot, Number(offsetBig), row.readUInt32LE(8), limit);
+ const receipt = parsed(bytes);
+ if (!receipt || verifier.verifyReceipt(receipt)) fail('policy-ledger-archive-conflict');
+ const canonical = canonicalizePolicy(receipt);
+ if (!digest(canonical).equals(row.subarray(16))) fail('policy-ledger-archive-changed');
+ return { receipt, canonical };
+}
+
+/** Every physical record is authenticated before deduplication, including
+ * unclaimed staging records. The fixed-width index only addresses the bounded
+ * snapshot's claimed prefix; arbitrary extra sequence numbers never set sparse
+ * file offsets. Exact full canonical equality is checked on duplicate rereads.
+ * Scratch I/O is disposable and does not need fsync or mutate the ledger.
+ */
+export async function verifyFullPolicyLedger(
+ snapshot: ArchiveSnapshot | undefined,
+ state: PolicyState,
+ engineOptions: { signingKey?: string; keyId?: string },
+): Promise<LedgerVerification> {
+ const retainedFrom = state.retainedFrom ?? 0;
+ const length = retainedFrom + state.receipts.length;
+ let scratch: string | undefined;
+ let index: number | undefined;
+ try {
+ const limit = policyLedgerRecordLimit();
+ if (!Number.isSafeInteger(retainedFrom) || retainedFrom < 0 || !Number.isSafeInteger(length)) fail('policy-ledger-archive-gap');
+ if (!snapshot) {
+ if (retainedFrom > 0) fail('policy-ledger-archive-gap');
+ return AgenticPolicyEngine.fromState(state, engineOptions).verifyLedger();
+ }
+ // At least one byte must exist for each claimed receipt. This also rejects
+ // malicious sparse-index lengths before creating scratch or replay loops.
+ if (retainedFrom > snapshot.size) fail('policy-ledger-archive-gap');
+ if (fstatSync(snapshot.fd).size < snapshot.size) fail('policy-ledger-archive-changed');
+ scratch = mkdtempSync(join(tmpdir(), 'ruflo-policy-audit-'));
+ chmodSync(scratch, 0o700);
+ index = openSync(join(scratch, 'index'), 'wx+', 0o600);
+ const verifier = new AgenticPolicyEngine(engineOptions);
+ let claimedUnique = 0;
+ for await (const physical of records(snapshot, limit)) {
+ const receipt = parsed(physical.bytes);
+ if (receipt === undefined) continue;
+ if (verifier.verifyReceipt(receipt)) fail('policy-ledger-archive-conflict');
+ const sequence = receipt.payload.sequence;
+ const canonical = canonicalizePolicy(receipt);
+ if (sequence < retainedFrom) {
+ const row = readIndex(index, sequence);
+ if (row.readUInt32LE(8) !== 0) {
+ const first = await observedRecord(snapshot, row, limit, verifier);
+ if (first.canonical !== canonical) fail('policy-ledger-archive-conflict');
+ } else {
+ row.writeBigUInt64LE(BigInt(physical.offset), 0);
+ row.writeUInt32LE(physical.bytes.length, 8);
+ digest(canonical).copy(row, 16);
+ writeAll(index, row, sequence * INDEX_BYTES);
+ claimedUnique++;
+ }
+ } else if (sequence < length) {
+ if (canonicalizePolicy(state.receipts[sequence - retainedFrom]) !== canonical) fail('policy-ledger-archive-conflict');
+ } else {
+ // Rare, unclaimed staging extras are still authenticated and deduped;
+ // decimal safe-integer names never choose a path or a sparse offset.
+ const file = join(scratch, `extra-${sequence}`);
+ const canonicalBytes = Buffer.from(canonical);
+ let fd: number;
+ try { fd = openSync(file, 'wx', 0o600); }
+ catch (error) {
+ if ((error as NodeJS.ErrnoException).code !== 'EEXIST') throw error;
+ fd = openSync(file, constants.O_RDONLY | (constants.O_NOFOLLOW ?? 0) | (constants.O_NONBLOCK ?? 0));
+ try {
+ const stat = fstatSync(fd);
+ if (!stat.isFile()) fail('policy-ledger-audit-index-read-error');
+ // Canonical finite numbers can expand beyond a fixed multiple of
+ // raw input bytes. Compare the already-bounded incoming value's
+ // exact encoded length before allocating any scratch read buffer.
+ if (stat.size !== canonicalBytes.length) fail('policy-ledger-archive-conflict');
+ const stored = Buffer.alloc(stat.size);
+ if (readSync(fd, stored, 0, stored.length, 0) !== stored.length || !stored.equals(canonicalBytes)) fail('policy-ledger-archive-conflict');
+ } finally { closeSync(fd); }
+ continue;
+ }
+ try { writeAll(fd, canonicalBytes, 0); } finally { closeSync(fd); }
+ }
+ }
+ if (claimedUnique !== retainedFrom) fail('policy-ledger-archive-gap');
+ let previous: string | null = null;
+ for (let sequence = 0; sequence < retainedFrom; sequence++) {
+ const row = readIndex(index, sequence);
+ if (!row.readUInt32LE(8)) fail('policy-ledger-archive-gap');
+ const { receipt } = await observedRecord(snapshot, row, limit, verifier);
+ if (receipt.payload.sequence !== sequence || receipt.payload.previousReceiptHash !== previous) fail('receipt-chain-mismatch');
+ previous = receipt.hash;
+ }
+ if (retainedFrom > 0 && previous !== state.prunedHead) fail('receipt-chain-mismatch');
+ const hot = AgenticPolicyEngine.fromState(state, engineOptions).verifyLedger();
+ if (!hot.valid) return hot;
+ if (fstatSync(snapshot.fd).size < snapshot.size) fail('policy-ledger-archive-changed');
+ return hot;
+ } catch (error) {
+ const message = (error as Error).message;
+ return { valid: false, length, error: message.startsWith('policy-ledger-') || message === 'receipt-chain-mismatch' ? message : 'policy-ledger-audit-io-error' };
+ } finally {
+ if (index !== undefined) closeSync(index);
+ if (scratch !== undefined) rmSync(scratch, { recursive: true, force: true });
+ }
+}
diff --git a/v3/@claude-flow/cli/src/services/policy-ledger-record.ts b/v3/@claude-flow/cli/src/services/policy-ledger-record.ts
new file mode 100644
index 0000000000..207c310b2f
--- /dev/null
+++ b/v3/@claude-flow/cli/src/services/policy-ledger-record.ts
@@ -0,0 +1,14 @@
+import { TextDecoder } from 'node:util';
+const DEFAULT_LIMIT = 16 * 1024 * 1024;
+const MAX_LIMIT = 64 * 1024 * 1024;
+const decoder = new TextDecoder('utf-8', { fatal: true });
+export function policyLedgerRecordLimit(): number {
+ const raw = process.env.CLAUDE_FLOW_POLICY_AUDIT_MAX_RECORD_BYTES;
+ if (!raw) return DEFAULT_LIMIT;
+ const value = Number(raw);
+ if (!Number.isSafeInteger(value) || value < 1024 || value > MAX_LIMIT) throw new Error('policy-ledger-audit-invalid-record-limit');
+ return value;
+}
+export function decodeLedgerRecord(bytes: Buffer): string {
+ try { return decoder.decode(bytes); } catch { throw new Error('policy-ledger-archive-utf8-error'); }
+}
diff --git a/v3/@claude-flow/cli/src/services/policy-runtime.ts b/v3/@claude-flow/cli/src/services/policy-runtime.ts
index 1cc9a8a699..951968b604 100644
--- a/v3/@claude-flow/cli/src/services/policy-runtime.ts
+++ b/v3/@claude-flow/cli/src/services/policy-runtime.ts
@@ -5,12 +5,14 @@
// runtime never uses.
import {
AgenticPolicyEngine,
+ POLICY_STATE_VERSION,
createLegacyCompatibleState,
type BudgetLimit,
type CapabilityEnvelope,
type PolicyApproval,
type PolicyDecision,
type PolicyEvidence,
+ type PolicyReceipt,
type PolicyRequest,
type PolicyRule,
type PolicyState,
@@ -28,6 +30,9 @@ import {
constants as fsConstants,
existsSync,
fstatSync,
+ fsyncSync,
+ ftruncateSync,
+ writeSync,
lstatSync,
mkdirSync,
openSync,
@@ -36,20 +41,32 @@ import {
readlinkSync,
realpathSync,
renameSync,
- statSync,
unlinkSync,
writeFileSync,
} from 'node:fs';
import { dirname, join, resolve } from 'node:path';
import { hostname, userInfo } from 'node:os';
import { syncPolicyProjection } from '../mods/policy-projection.js';
-import { assessAnchors, policyTrustRoot, recordAnchor, type AnchorEvent } from './policy-ledger-anchor.js';
+import { ATOMIC_JSON_WRITER, type AtomicJsonSource } from './policy-anchor-log.js';
+import { captureArchiveSnapshot, verifyFullPolicyLedger, type ArchiveSnapshot } from './policy-ledger-audit.js';
+import { assessAnchors, policyTrustRoot, recordAnchor, type AnchorEvent, type AnchorEntry } from './policy-ledger-anchor.js';
const POLICY_DIR = join('.claude-flow', 'policy');
const POLICY_FILE = 'state.json';
const LOCK_FILE = 'state.lock';
-const LOCK_STALE_MS = 30_000;
+const RECEIPT_ARCHIVE_FILE = 'receipts.ledger.jsonl';
const LOCK_WAIT_MS = 5_000;
+// #3164: without a bound, `state.receipts` grows forever and every
+// transaction's verify/clone/serialize cost is O(all receipts ever issued) β€”
+// eventually exceeding LOCK_WAIT_MS and timing out every MCP tool call.
+// Pruned receipts are archived to RECEIPT_ARCHIVE_FILE, never discarded.
+const DEFAULT_LEDGER_RETENTION = 2_000;
+function ledgerRetention(): number {
+ const raw = process.env.CLAUDE_FLOW_POLICY_LEDGER_RETENTION;
+ if (!raw) return DEFAULT_LEDGER_RETENTION;
+ const parsed = Number(raw);
+ return Number.isInteger(parsed) && parsed >= 0 ? parsed : DEFAULT_LEDGER_RETENTION;
+}
// A lock is ~170 bytes plus the hostname (at most 255 bytes).
const LOCK_MAX_BYTES = 1_024;
// Linux PID namespace of this process, e.g. `pid:[4026531836]`; undefined on
@@ -66,10 +83,10 @@ const BOOT_ID = (() => {
try { return readFileSync('/proc/sys/kernel/random/boot_id', 'utf8').trim(); } catch { return undefined; }
})();
-function paths(projectRoot: string): { dir: string; state: string; lock: string } {
+function paths(projectRoot: string): { dir: string; state: string; lock: string; archive: string } {
const root = resolve(projectRoot);
const dir = join(root, POLICY_DIR);
- return { dir, state: join(dir, POLICY_FILE), lock: join(dir, LOCK_FILE) };
+ return { dir, state: join(dir, POLICY_FILE), lock: join(dir, LOCK_FILE), archive: join(dir, RECEIPT_ARCHIVE_FILE) };
}
function sleep(ms: number): Promise<void> {
@@ -94,7 +111,7 @@ function isPidAlive(pid: number): boolean {
/**
* Reads a lock only if it is a small regular file: never follows a symlink,
* never blocks on a FIFO or device, never reads more than LOCK_MAX_BYTES.
- * Anything else returns undefined and the caller keeps the mtime rule.
+ * Anything else returns undefined; unknown owners are preserved.
*/
function readLockFile(lockPath: string): string | undefined {
if (!lstatSync(lockPath).isFile()) return undefined;
@@ -121,8 +138,8 @@ function readLockFile(lockPath: string): string | undefined {
* boot and in this PID namespace: anywhere else (a bwrap --unshare-pid
* sandbox, a container, another machine on a shared filesystem) a live owner
* also answers ESRCH. Locks without that identity (older ruflo, a crash
- * between create and write, anything unparsable) return false and keep the
- * mtime rule.
+ * between create and write, anything unparsable) return false and require
+ * explicit operator inspection rather than age-only takeover.
*/
function lockOwnerIsDead(lockPath: string): boolean {
try {
@@ -151,6 +168,26 @@ function lockOwnerIsDead(lockPath: string): boolean {
}
}
+/** Serialize dead-owner reclamation across processes. A normal acquirer can
+ * create state.lock after deletion, but the next reclaimer must re-probe that
+ * new owner under this guard. Guard orphans fail closed, requiring explicit
+ * operator recovery after validating their exact recorded owner and nonce.
+ * Unknown/foreign owners are never stolen merely because their mtime aged. */
+function reclaimDeadLock(lockPath: string): void {
+ const guard = `${lockPath}.reclaim`;
+ let fd: number;
+ try { fd = openSync(guard, 'wx', 0o600); } catch { return; }
+ const ownership = JSON.stringify({ pid: process.pid, nonce: randomUUID(), host: hostname(), platform: process.platform, pidns: PID_NAMESPACE, boot: BOOT_ID });
+ try {
+ writeFileSync(fd, ownership);
+ if (lockOwnerIsDead(lockPath)) unlinkSync(lockPath);
+ } catch { /* lock changed, unknown identity, or an inaccessible path */ }
+ finally {
+ closeSync(fd);
+ try { if (readLockFile(guard) === ownership) unlinkSync(guard); } catch { /* preserve an unknown guard */ }
+ }
+}
+
/**
* Exported for ADR-406 mission storage, which takes the same per-file lock
* (pid, pid namespace and boot id identify a dead owner) rather than a copy.
@@ -160,27 +197,27 @@ export async function acquireLock(lockPath: string): Promise<() => void> {
while (Date.now() - started < LOCK_WAIT_MS) {
try {
const fd = openSync(lockPath, 'wx', 0o600);
- writeFileSync(fd, JSON.stringify({
+ const ownership = JSON.stringify({
pid: process.pid,
acquiredAt: Date.now(),
host: hostname(),
platform: process.platform,
pidns: PID_NAMESPACE,
boot: BOOT_ID,
- }));
- closeSync(fd);
+ nonce: randomUUID(),
+ });
+ try { writeFileSync(fd, ownership); } finally { closeSync(fd); }
return () => {
- try { unlinkSync(lockPath); } catch { /* already released */ }
+ try { if (readLockFile(lockPath) === ownership) unlinkSync(lockPath); } catch { /* already released */ }
};
} catch {
- try {
- if (lockOwnerIsDead(lockPath)
- || Date.now() - statSync(lockPath).mtimeMs > LOCK_STALE_MS) unlinkSync(lockPath);
- } catch { /* another process changed the lock */ }
+ reclaimDeadLock(lockPath);
await sleep(10);
}
}
- throw new Error('policy-state-lock-timeout');
+ throw new Error(lockOwnerIsDead(lockPath)
+ ? 'policy-state-lock-timeout: dead owner; inspect exact owner of state.lock.reclaim before explicit recovery'
+ : 'policy-state-lock-timeout');
}
// #3398: Windows rename over a file another process holds open fails with
@@ -190,23 +227,100 @@ export async function acquireLock(lockPath: string): Promise<() => void> {
const RENAME_RETRY_DELAYS_MS = [25, 50, 100, 150, 250, 300, 400];
const TRANSIENT_RENAME_CODES = new Set(['EPERM', 'EBUSY', 'EACCES']);
+function mkdirDurably(dir: string): void {
+ if (existsSync(dir)) return;
+ const parent = dirname(dir);
+ if (parent !== dir) mkdirDurably(parent);
+ try { mkdirSync(dir, { mode: 0o700 }); }
+ catch (error) { if ((error as NodeJS.ErrnoException).code !== 'EEXIST') throw error; }
+ syncDirectory(parent);
+}
+
+function syncDirectory(dir: string): void {
+ // Windows does not expose directory fsync through Node. File sync and
+ // atomic rename still apply there; POSIX failures must reach the caller.
+ if (process.platform === 'win32') return;
+ const fd = openSync(dir, fsConstants.O_RDONLY | (fsConstants.O_DIRECTORY ?? 0));
+ try { fsyncSync(fd); } finally { closeSync(fd); }
+}
+
+/** Recover only an incomplete, unacknowledged final append. All inspection,
+ * truncation, append and sync use the same nofollow regular-file descriptor. */
+function appendArchiveDurably(file: string, receipts: PolicyReceipt[], verifier: AgenticPolicyEngine): void {
+ const fd = openSync(file, fsConstants.O_RDWR | fsConstants.O_CREAT
+ | (fsConstants.O_NOFOLLOW ?? 0) | (fsConstants.O_NONBLOCK ?? 0), 0o600);
+ try {
+ const stat = fstatSync(fd);
+ if (!stat.isFile()) throw new Error('policy-ledger-archive-not-regular');
+ const limit = 64 * 1024;
+ const start = Math.max(0, stat.size - limit);
+ const buffer = Buffer.alloc(stat.size - start);
+ if (readSync(fd, buffer, 0, buffer.length, start) !== buffer.length) throw new Error('policy-ledger-archive-short-read');
+ let end = stat.size;
+ const lastNewline = buffer.lastIndexOf(10);
+ if (buffer.length && buffer[buffer.length - 1] !== 10) {
+ if (lastNewline < 0 && start > 0) throw new Error('policy-ledger-archive-tail-too-large');
+ const fragmentBytes = buffer.subarray(lastNewline + 1);
+ const fragment = fragmentBytes.toString('utf8');
+ try { JSON.parse(fragment); throw new Error('policy-ledger-archive-unterminated-record'); }
+ catch (error) { if (!(error instanceof SyntaxError)) throw error; }
+ // A parse failure alone grants no repair authority: the bytes must
+ // be an exact prefix of a receipt still available in the verified hot
+ // state and being retried by this prune.
+ if (!fragmentBytes.length || !receipts.some(receipt => {
+ const expected = Buffer.from(JSON.stringify(receipt));
+ return fragmentBytes.length < expected.length && expected.subarray(0, fragmentBytes.length).equals(fragmentBytes);
+ })) {
+ throw new Error('policy-ledger-archive-unrecoverable-tail');
+ }
+ end = start + lastNewline + 1;
+ }
+ const complete = buffer.subarray(0, end - start).toString('utf8');
+ const usable = start === 0 ? complete : complete.slice(complete.indexOf('\n') + 1);
+ for (const line of usable.split('\n')) {
+ if (!line.trim()) continue;
+ let record: unknown;
+ try { record = JSON.parse(line); } catch { throw new Error('policy-ledger-archive-parse-error'); }
+ if (verifier.verifyReceipt(record)) throw new Error('policy-ledger-archive-conflict');
+ }
+ if (end < stat.size) { ftruncateSync(fd, end); fsyncSync(fd); }
+ const bytes = Buffer.from(`${receipts.map(receipt => JSON.stringify(receipt)).join('\n')}\n`);
+ let written = 0;
+ while (written < bytes.length) {
+ const count = writeSync(fd, bytes, written, bytes.length - written, end + written);
+ if (!count) throw new Error('policy-ledger-archive-short-write');
+ written += count;
+ }
+ fsyncSync(fd);
+ syncDirectory(dirname(file));
+ } finally { closeSync(fd); }
+}
+
async function writeJsonAtomic(file: string, value: unknown): Promise<void> {
- mkdirSync(dirname(file), { recursive: true, mode: 0o700 });
+ mkdirDurably(dirname(file));
const temporary = `${file}.${process.pid}.${randomUUID()}.tmp`;
let renamed = false;
try {
- writeFileSync(temporary, `${JSON.stringify(value, null, 2)}\n`, { mode: 0o600 });
+ const fd = openSync(temporary, 'wx', 0o600);
+ try {
+ if (value && typeof (value as AtomicJsonSource)[ATOMIC_JSON_WRITER] === 'function') (value as AtomicJsonSource)[ATOMIC_JSON_WRITER](fd);
+ else writeFileSync(fd, `${JSON.stringify(value, null, 2)}\n`);
+ fsyncSync(fd);
+ } finally { closeSync(fd); }
for (let attempt = 0; ; attempt++) {
try {
renameSync(temporary, file);
renamed = true;
- return;
+ break;
} catch (error) {
const code = (error as NodeJS.ErrnoException)?.code ?? '';
if (!TRANSIENT_RENAME_CODES.has(code) || attempt >= RENAME_RETRY_DELAYS_MS.length) throw error;
await sleep(RENAME_RETRY_DELAYS_MS[attempt]);
}
}
+ // A post-rename sync failure is a publication failure, not a rename
+ // failure. Never retry with a temporary file already consumed by rename.
+ syncDirectory(dirname(file));
} finally {
if (!renamed) {
try { unlinkSync(temporary); } catch { /* never created or already gone */ }
@@ -225,8 +339,10 @@ function trustKey(projectRoot: string, create: boolean): Buffer | undefined {
const { key } = trustPaths(projectRoot);
if (!existsSync(key)) {
if (!create) return undefined;
- mkdirSync(dirname(key), { recursive: true, mode: 0o700 });
- writeFileSync(key, randomBytes(32), { mode: 0o600, flag: 'wx' });
+ mkdirDurably(dirname(key));
+ const fd = openSync(key, 'wx', 0o600);
+ try { writeFileSync(fd, randomBytes(32)); fsyncSync(fd); } finally { closeSync(fd); }
+ syncDirectory(dirname(key));
}
const material = readFileSync(key);
if (material.length !== 32) throw new Error('invalid-policy-trust-key');
@@ -258,11 +374,12 @@ async function writePolicyState(
state: PolicyState,
anchorEvent: AnchorEvent = 'append',
by?: string,
+ assessedTip?: AnchorEntry,
): Promise<void> {
await writePolicyStateFiles(projectRoot, statePath, state);
// #3602: the second anchor is written after the state, so a crash leaves the
// state AHEAD of the log (caught up on the next write), never behind it.
- await recordAnchor(projectRoot, state, writeJsonAtomic, anchorEvent, by);
+ await recordAnchor(projectRoot, state, writeJsonAtomic, anchorEvent, by, writeJsonAtomic, assessedTip);
// ADR-404: the ruflo mod reads Claude Code tool rules from a small
// projection, never from state.json. Written only after the state and its
// anchor are safely down; a failure here never fails the state write. A
@@ -337,7 +454,7 @@ export function loadPolicyState(projectRoot = process.cwd()): PolicyState {
return createLegacyCompatibleState(detectLegacyCapabilities(projectRoot));
}
const parsed = JSON.parse(readFileSync(target.state, 'utf8')) as PolicyState;
- if (parsed.version !== 1 || !Array.isArray(parsed.rules) || !Array.isArray(parsed.receipts)) {
+ if ((parsed.version !== 1 && parsed.version !== POLICY_STATE_VERSION) || !Array.isArray(parsed.rules) || !Array.isArray(parsed.receipts)) {
throw new Error(`unsupported-policy-state-version:${String(parsed.version)}`);
}
verifyStateAnchor(projectRoot, parsed);
@@ -362,7 +479,7 @@ export async function autoMigratePolicyStateIfNeeded(projectRoot = process.cwd()
// acquire policy state merely because `ruflo --version` ran there.
if (!existsSync(join(resolve(projectRoot), '.claude-flow'))
&& !existsSync(join(resolve(projectRoot), '.swarm'))) return { migrated: false };
- mkdirSync(target.dir, { recursive: true, mode: 0o700 });
+ mkdirDurably(target.dir);
const release = await acquireLock(target.lock);
try {
if (!existsSync(target.state)) {
@@ -388,7 +505,7 @@ export async function withPolicyTransaction<T>(
} = {},
): Promise<T> {
const target = paths(projectRoot);
- mkdirSync(target.dir, { recursive: true, mode: 0o700 });
+ mkdirDurably(target.dir);
const release = await acquireLock(target.lock);
try {
const loaded = loadPolicyState(projectRoot);
@@ -407,9 +524,29 @@ export async function withPolicyTransaction<T>(
: gate.result.error ?? 'policy-ledger-verification-failed');
}
const result = await operation(engine);
- if (!engine.verifyLedger().valid) throw new Error('policy-ledger-verification-failed');
+ const verified = engine.verifyLedger();
+ if (!verified.valid) throw new Error('policy-ledger-verification-failed');
+ // Only archive the prefix already committed in loaded state. Archiving
+ // newly generated receipts before their first state publication would
+ // leave conflicting same-sequence records when a failed operation is
+ // retried with a different timestamp/request. Retention=0 and large
+ // batches therefore retain this transaction's new receipts until a
+ // subsequent transaction can archive their exact committed bytes.
+ const newlyAppended = verified.length - ((loaded.retainedFrom ?? 0) + loaded.receipts.length);
+ const pruned = engine.pruneReceipts(Math.max(ledgerRetention(), newlyAppended));
+ // #3164: archive BEFORE writing the shrunk state. If the process crashes
+ // between the two writes, the archive append already landed but
+ // `retainedFrom` on disk still reflects the pre-prune length, so the next
+ // prune recomputes and re-appends the SAME receipts β€” a harmless,
+ // detectable duplicate (same sequence, identical content), deduped by
+ // `verifyPolicyLedger({ full: true })`. The reverse order (state first)
+ // would instead risk a permanent gap if the process died in between,
+ // which is strictly worse for an audit ledger β€” never reorder this.
+ if (pruned.length > 0) {
+ appendArchiveDurably(target.archive, pruned, engine);
+ }
const nextState = engine.exportState();
- await writePolicyState(projectRoot, target.state, nextState, gate.event);
+ await writePolicyState(projectRoot, target.state, nextState, gate.event, undefined, gate.anchorTip);
return result;
} finally {
release();
@@ -484,6 +621,7 @@ interface Reconciled {
result: PolicyLedgerVerification;
persist: boolean;
event?: AnchorEvent;
+ anchorTip?: AnchorEntry;
}
/**
@@ -498,7 +636,9 @@ function reconcileLedger(
options: { establishAnchor?: boolean },
): Reconciled {
const state = engine.exportState();
- const length = state.receipts.length;
+ // #3164: the historical length includes anything pruned out of the hot
+ // array β€” `length` here must keep meaning "total receipts ever issued".
+ const length = (state.retainedFrom ?? 0) + state.receipts.length;
const primary = typeof state.ledgerLength === 'number';
const base = engine.verifyLedger();
const fail = (error: string): Reconciled => ({ result: { valid: false, length, error }, persist: false });
@@ -511,7 +651,7 @@ function reconcileLedger(
? { result: { ...base, secondaryAnchor: 'established-explicitly' }, persist: true, event: 'establish-anchor' }
: { result: { ...base, secondaryAnchor: 'recorded-from-state' }, persist: true, event: 'migrated-from-state' };
}
- return { result: base, persist: assessed.kind === 'ok' && assessed.behind, event: 'append' };
+ return { result: base, persist: assessed.kind === 'ok' && assessed.behind, event: 'append', anchorTip: assessed.kind === 'ok' ? assessed.last : undefined };
}
// No primary anchor.
if (length === 0) return { result: { valid: true, length, state: 'empty' }, persist: false };
@@ -519,7 +659,7 @@ function reconcileLedger(
// The second anchor is authentic and agrees with the chain: the deleted
// primary can be restored from it. Truncation was already rejected above.
engine.verifyLedger({ establishAnchor: true });
- return { result: { valid: true, length, secondaryAnchor: 'restored-primary' }, persist: true, event: 'append' };
+ return { result: { valid: true, length, secondaryAnchor: 'restored-primary' }, persist: true, event: 'append', anchorTip: assessed.last };
}
if (options.establishAnchor) {
const established = engine.verifyLedger({ establishAnchor: true });
@@ -540,25 +680,44 @@ function reconcileLedger(
*/
export async function verifyPolicyLedger(
projectRoot = process.cwd(),
- options: { establishAnchor?: boolean } = {},
+ options: { establishAnchor?: boolean; full?: boolean } = {},
): Promise<PolicyLedgerVerification> {
const target = paths(projectRoot);
- mkdirSync(target.dir, { recursive: true, mode: 0o700 });
+ mkdirDurably(target.dir);
+ const engineOptions = {
+ signingKey: process.env.CLAUDE_FLOW_POLICY_SIGNING_KEY,
+ keyId: process.env.CLAUDE_FLOW_POLICY_KEY_ID,
+ };
+ let outcomeResult: PolicyLedgerVerification;
+ let snapshotState: PolicyState;
+ let archiveSnapshot: ArchiveSnapshot | undefined;
+ // Capture the verified hot state and a checked regular archive descriptor
+ // plus immutable byte boundary under the writer lock. Historical read,
+ // authentication, deduplication and replay all happen after release, using
+ // bounded chunks and private disposable disk scratch instead of O(history)
+ // strings, receipt arrays or maps. The descriptor pins path replacements.
const release = await acquireLock(target.lock);
try {
- const engine = AgenticPolicyEngine.fromState(loadPolicyState(projectRoot), {
- signingKey: process.env.CLAUDE_FLOW_POLICY_SIGNING_KEY,
- keyId: process.env.CLAUDE_FLOW_POLICY_KEY_ID,
- });
+ const engine = AgenticPolicyEngine.fromState(loadPolicyState(projectRoot), engineOptions);
const outcome = reconcileLedger(projectRoot, engine, options);
if (outcome.persist) {
const by = outcome.event === 'establish-anchor' ? userInfo().username : undefined;
- await writePolicyState(projectRoot, target.state, engine.exportState(), outcome.event, by);
+ await writePolicyState(projectRoot, target.state, engine.exportState(), outcome.event, by, outcome.anchorTip);
+ }
+ outcomeResult = outcome.result;
+ snapshotState = engine.exportState();
+ if (options.full && outcomeResult.valid) {
+ try { archiveSnapshot = captureArchiveSnapshot(target.archive); }
+ catch (error) { outcomeResult = { valid: false, length: outcomeResult.length, error: (error as Error).message }; }
}
- return outcome.result;
} finally {
release();
}
+ if (!options.full || !outcomeResult.valid) return outcomeResult;
+ try {
+ const full = await verifyFullPolicyLedger(archiveSnapshot, snapshotState, engineOptions);
+ return { ...full, secondaryAnchor: outcomeResult.secondaryAnchor };
+ } finally { if (archiveSnapshot !== undefined) closeSync(archiveSnapshot.fd); }
}
/**
diff --git a/v3/@claude-flow/security/__tests__/agentic-policy-engine.test.ts b/v3/@claude-flow/security/__tests__/agentic-policy-engine.test.ts
index 53969caa8b..2ee2e05fb1 100644
--- a/v3/@claude-flow/security/__tests__/agentic-policy-engine.test.ts
+++ b/v3/@claude-flow/security/__tests__/agentic-policy-engine.test.ts
@@ -207,6 +207,84 @@ describe('AgenticPolicyEngine', () => {
}))).toThrow('invalid-policy-action-tokens');
});
+ it('prunes receipts below the retain count, keeping full historical length (#3164)', () => {
+ const engine = new AgenticPolicyEngine({ mode: 'legacy' });
+ for (let i = 0; i < 5; i++) engine.evaluate(request({ action: { type: 'code.read', resource: `f${i}.ts` } }));
+ expect(engine.verifyLedger()).toEqual({ valid: true, length: 5 });
+
+ const removed = engine.pruneReceipts(2);
+ expect(removed).toHaveLength(3);
+ expect(removed.map((r) => r.payload.sequence)).toEqual([0, 1, 2]);
+
+ const state = engine.exportState();
+ expect(state.receipts).toHaveLength(2);
+ expect(state.retainedFrom).toBe(3);
+ expect(state.prunedHead).toBe(removed[2]!.hash);
+ expect((state.retainedFrom ?? 0) + state.receipts.length).toBe(state.ledgerLength);
+
+ const verification = engine.verifyLedger();
+ expect(verification).toEqual({ valid: true, length: 5 });
+ });
+
+ it('is a no-op when retain is at or above the current receipt count', () => {
+ const engine = new AgenticPolicyEngine({ mode: 'legacy' });
+ engine.evaluate(request());
+ engine.evaluate(request({ action: { type: 'code.read', resource: 'b.ts' } }));
+ expect(engine.pruneReceipts(5)).toEqual([]);
+ expect(engine.pruneReceipts(2)).toEqual([]);
+ expect(engine.exportState().receipts).toHaveLength(2);
+ expect(engine.exportState().retainedFrom).toBeUndefined();
+ });
+
+ it('rejects an invalid retain count', () => {
+ const engine = new AgenticPolicyEngine({ mode: 'legacy' });
+ engine.evaluate(request());
+ expect(() => engine.pruneReceipts(-1)).toThrow('invalid-prune-retention');
+ expect(() => engine.pruneReceipts(1.5)).toThrow('invalid-prune-retention');
+ });
+
+ it('refuses to prune a ledger with no established anchor', () => {
+ const engine = new AgenticPolicyEngine({ mode: 'legacy' });
+ engine.evaluate(request());
+ const unanchored = AgenticPolicyEngine.fromState({ ...engine.exportState(), ledgerLength: undefined, ledgerHead: undefined });
+ expect(() => unanchored.pruneReceipts(0)).toThrow('policy-ledger-anchor-missing');
+ });
+
+ it('continues the chain and sequence numbers correctly after pruning', () => {
+ const engine = new AgenticPolicyEngine({ mode: 'legacy', signingKey: 'test-secret' });
+ for (let i = 0; i < 4; i++) engine.evaluate(request({ action: { type: 'code.read', resource: `f${i}.ts` } }));
+ engine.pruneReceipts(1);
+ engine.evaluate(request({ action: { type: 'code.read', resource: 'f4.ts' } }));
+ const state = engine.exportState();
+ expect(state.receipts.map((r) => r.payload.sequence)).toEqual([3, 4]);
+ expect(state.ledgerLength).toBe(5);
+ expect(engine.verifyLedger()).toEqual({ valid: true, length: 5 });
+ });
+
+ it('still detects hash and sequence tampering in the retained tail after pruning', () => {
+ const engine = new AgenticPolicyEngine({ mode: 'legacy' });
+ for (let i = 0; i < 5; i++) engine.evaluate(request({ action: { type: 'code.read', resource: `f${i}.ts` } }));
+ engine.pruneReceipts(2);
+ const tamperedState = engine.exportState();
+ tamperedState.receipts[0]!.payload.decision.reason = 'tampered';
+ const tampered = AgenticPolicyEngine.fromState(tamperedState);
+ const result = tampered.verifyLedger();
+ expect(result.valid).toBe(false);
+ expect(result.error).toBe('receipt-hash-mismatch');
+ // length reflects position within the FULL historical ledger, not the hot array
+ expect(result.length).toBe(3);
+ });
+
+ it('fails closed when retainedFrom is set but prunedHead is missing', () => {
+ const engine = new AgenticPolicyEngine({ mode: 'legacy' });
+ for (let i = 0; i < 3; i++) engine.evaluate(request({ action: { type: 'code.read', resource: `f${i}.ts` } }));
+ engine.pruneReceipts(1);
+ const corrupted = engine.exportState();
+ delete corrupted.prunedHead;
+ const reloaded = AgenticPolicyEngine.fromState(corrupted);
+ expect(reloaded.verifyLedger().error).toBe('policy-ledger-prune-state-missing');
+ });
+
it('fails closed when a constrained action omits required metering', () => {
const engine = new AgenticPolicyEngine({
mode: 'enforce',
@@ -289,3 +367,17 @@ describe('CapabilityEnvelope', () => {
expect(decision.reason).toBe('network-outside-envelope');
});
});
+
+
+it('creates request IDs without the optional global WebCrypto API (Node 18)', () => {
+ const descriptor = Object.getOwnPropertyDescriptor(globalThis, 'crypto');
+ try {
+ Object.defineProperty(globalThis, 'crypto', { value: undefined, configurable: true });
+ const engine = new AgenticPolicyEngine({ mode: 'legacy' });
+ engine.evaluate({ identity: { id: 'node18-agent', type: 'agent' }, action: { type: 'code.read' } });
+ expect(engine.exportState().receipts[0].payload.request.requestId).toMatch(/^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/);
+ } finally {
+ if (descriptor) Object.defineProperty(globalThis, 'crypto', descriptor);
+ else Reflect.deleteProperty(globalThis, 'crypto');
+ }
+});
diff --git a/v3/@claude-flow/security/__tests__/policy-ledger-retention-3164-perf.test.ts b/v3/@claude-flow/security/__tests__/policy-ledger-retention-3164-perf.test.ts
new file mode 100644
index 0000000000..94b8a057e7
--- /dev/null
+++ b/v3/@claude-flow/security/__tests__/policy-ledger-retention-3164-perf.test.ts
@@ -0,0 +1,174 @@
+import { describe, expect, it, vi } from 'vitest';
+import { performance } from 'node:perf_hooks';
+import { AgenticPolicyEngine, type PolicyState } from '../src/policy/index.js';
+
+// #3164 engine hot-tail regression: verification/export must work on retained
+// receipts, independently of how much history was issued. Real receipt checks
+// and clone-input volume are the gates; shared-host wall time is diagnostic.
+// These controls do not certify a latency SLA or mature-writer contention:
+// authenticating/rewriting the external anchor log remains O(anchor count).
+const TOTAL_RECEIPTS = 60_000;
+const RETENTION = 2_000;
+const SIGNING = { signingKey: 'disposable-retention-test-key', keyId: 'retention-test' };
+
+function seedEngine(count: number): AgenticPolicyEngine {
+ const engine = new AgenticPolicyEngine({ mode: 'legacy', ...SIGNING });
+ for (let i = 0; i < count; i++) {
+ engine.evaluate({
+ identity: { id: `agent:${i % 50}`, type: 'agent' },
+ action: { type: 'code.read', resource: `file-${i}` },
+ });
+ }
+ return engine;
+}
+
+function measured<T>(run: () => T) {
+ const start = performance.now();
+ const cpuStart = process.cpuUsage();
+ const value = run();
+ const cpu = process.cpuUsage(cpuStart);
+ return {
+ value,
+ timing: { wallMs: performance.now() - start, cpuMs: (cpu.user + cpu.system) / 1_000 },
+ };
+}
+
+function receiptChecks(engine: AgenticPolicyEngine) {
+ let count = 0;
+ const original = engine.verifyReceipt.bind(engine);
+ engine.verifyReceipt = (receipt) => { count++; return original(receipt); };
+ return { count: () => count, reset: () => { count = 0; } };
+}
+
+// Count physical receipt slots across clone inputs, including extra historical
+// arrays a regression might clone and then discard before returning the tail.
+// This observes clone work without requiring a particular number of calls.
+function receiptSlots(value: unknown): number {
+ const seen = new WeakSet<object>();
+ const pending = [value];
+ let count = 0;
+ while (pending.length) {
+ const item = pending.pop();
+ if (!item || typeof item !== 'object' || seen.has(item)) continue;
+ seen.add(item);
+ if (Array.isArray(item)) {
+ for (const child of item) {
+ if (child && typeof child === 'object' && 'payload' in child && 'hash' in child) count++;
+ pending.push(child);
+ }
+ } else {
+ pending.push(...Object.values(item));
+ }
+ }
+ return count;
+}
+
+function measuredExport(engine: AgenticPolicyEngine) {
+ const clone = globalThis.structuredClone;
+ const inputs: unknown[] = [];
+ const observer = vi.spyOn(globalThis, 'structuredClone').mockImplementation((value, options) => {
+ inputs.push(value);
+ return clone(value, options);
+ });
+ try {
+ const result = measured(() => {
+ const state = engine.exportState();
+ return { state, serializedBytes: Buffer.byteLength(JSON.stringify(state)) };
+ });
+ // Inspect outside the timed region and before any pruning/mutation of the
+ // observed inputs. Real structuredClone and JSON serialization ran above.
+ return { ...result, clonedReceiptSlots: inputs.reduce<number>((sum, input) => sum + receiptSlots(input), 0) };
+ } finally {
+ observer.mockRestore();
+ }
+}
+
+function expectTail(state: PolicyState, issued: number) {
+ expect(state.receipts).toHaveLength(RETENTION);
+ expect(state.retainedFrom).toBe(issued - RETENTION);
+ expect(state.ledgerLength).toBe(issued);
+ expect(state.receipts[0]!.payload.sequence).toBe(issued - RETENTION);
+ expect(state.receipts[0]!.payload.previousReceiptHash).toBe(state.prunedHead);
+ expect(state.ledgerHead).toBe(state.receipts.at(-1)!.hash);
+}
+
+describe('#3164 engine hot-tail verification and export work', () => {
+ it(`bounds authenticated receipts and cloning after pruning ${TOTAL_RECEIPTS} receipts to ${RETENTION}`, () => {
+ const engine = seedEngine(TOTAL_RECEIPTS);
+ const checks = receiptChecks(engine);
+ const verifyBefore = measured(() => engine.verifyLedger());
+ expect(verifyBefore.value).toEqual({ valid: true, length: TOTAL_RECEIPTS });
+ expect(checks.count()).toBe(TOTAL_RECEIPTS);
+ const exportBefore = measuredExport(engine);
+ expect(exportBefore.value.state.receipts).toHaveLength(TOTAL_RECEIPTS);
+ expect(exportBefore.clonedReceiptSlots).toBe(TOTAL_RECEIPTS);
+
+ const removed = engine.pruneReceipts(RETENTION);
+ expect(removed).toHaveLength(TOTAL_RECEIPTS - RETENTION);
+ checks.reset();
+ const verifyAfter = measured(() => engine.verifyLedger());
+ expect(verifyAfter.value).toEqual({ valid: true, length: TOTAL_RECEIPTS });
+ expect(checks.count()).toBe(RETENTION);
+ const exportAfter = measuredExport(engine);
+ const { state } = exportAfter.value;
+ expectTail(state, TOTAL_RECEIPTS);
+ expect(state.prunedHead).toBe(removed.at(-1)!.hash);
+ expect(exportAfter.clonedReceiptSlots).toBeLessThanOrEqual(RETENTION);
+ expect(AgenticPolicyEngine.fromState(state, SIGNING).verifyLedger())
+ .toEqual({ valid: true, length: TOTAL_RECEIPTS });
+
+ // A cheap alias/shallow copy must not satisfy the bounded-clone guard.
+ const original = JSON.stringify(state);
+ for (const offset of [0, Math.floor(RETENTION / 2), RETENTION - 1]) {
+ state.receipts[offset]!.payload.request.action.resource = `changed-only-in-export-${offset}`;
+ }
+ expect(JSON.stringify(engine.exportState())).toBe(original);
+ expect(engine.verifyLedger()).toEqual({ valid: true, length: TOTAL_RECEIPTS });
+
+ console.log(JSON.stringify({
+ schema: 'ruflo.policy-ledger-retention-work/v2',
+ scope: 'synthetic engine hot tail; timings are diagnostic, not a latency or contention gate',
+ totalReceipts: TOTAL_RECEIPTS,
+ retention: RETENTION,
+ authenticatedReceipts: { unpruned: TOTAL_RECEIPTS, pruned: RETENTION },
+ clonedReceiptSlots: { unpruned: exportBefore.clonedReceiptSlots, pruned: exportAfter.clonedReceiptSlots },
+ verify: { unpruned: verifyBefore.timing, pruned: verifyAfter.timing },
+ exportStateAndSerialize: { unpruned: exportBefore.timing, pruned: exportAfter.timing },
+ serializedBytes: { unpruned: exportBefore.value.serializedBytes, pruned: exportAfter.value.serializedBytes },
+ }, null, 2));
+ }, 60_000);
+
+ it('authenticates and clones the same bounded tail with 5,000 or 60,000 issued receipts', () => {
+ const diagnostics = [];
+ for (const issued of [5_000, TOTAL_RECEIPTS]) {
+ const engine = seedEngine(issued);
+ const removed = engine.pruneReceipts(RETENTION);
+ expect(removed).toHaveLength(issued - RETENTION);
+ const checks = receiptChecks(engine);
+ const verify = measured(() => engine.verifyLedger());
+ expect(verify.value).toEqual({ valid: true, length: issued });
+ expect(checks.count()).toBe(RETENTION);
+ const exported = measuredExport(engine);
+ expectTail(exported.value.state, issued);
+ expect(exported.clonedReceiptSlots).toBeLessThanOrEqual(RETENTION);
+ diagnostics.push({ issued, authenticatedReceipts: checks.count(), clonedReceiptSlots: exported.clonedReceiptSlots, verify: verify.timing, exportStateAndSerialize: exported.timing });
+ }
+ console.log(JSON.stringify({ schema: 'ruflo.policy-ledger-retention-scaling/v2', diagnosticTimingsOnly: true, histories: diagnostics }, null, 2));
+ }, 60_000);
+
+ it.each(['middle-payload', 'final-payload', 'final-signature'] as const)('rejects %s corruption inside the retained signed tail', (corruption) => {
+ const issued = RETENTION + 10;
+ const engine = seedEngine(issued);
+ engine.pruneReceipts(RETENTION);
+ const state = engine.exportState();
+ const offset = corruption === 'middle-payload' ? Math.floor(RETENTION / 2) : RETENTION - 1;
+ const receipt = state.receipts[offset]!;
+ if (corruption === 'final-signature') receipt.signature = 'invalid-signature';
+ else receipt.payload.request.action.resource = 'corrupted-retained-receipt';
+ expect(AgenticPolicyEngine.fromState(state, SIGNING).verifyLedger()).toEqual({
+ valid: false,
+ length: issued - RETENTION + offset,
+ error: corruption === 'final-signature' ? 'receipt-signature-invalid' : 'receipt-hash-mismatch',
+ });
+ });
+});
diff --git a/v3/@claude-flow/security/src/policy/engine.ts b/v3/@claude-flow/security/src/policy/engine.ts
index e46a74bbea..b3df421dd4 100644
--- a/v3/@claude-flow/security/src/policy/engine.ts
+++ b/v3/@claude-flow/security/src/policy/engine.ts
@@ -1,3 +1,4 @@
+import { randomUUID } from 'node:crypto';
import { policyHash, signPolicyHash, verifyPolicySignature } from './canonical.js';
import { evaluatePolicy } from './evaluator.js';
import type {
@@ -47,6 +48,9 @@ export class AgenticPolicyEngine {
}
static fromState(state: PolicyState, options: Omit<PolicyEngineOptions, 'mode' | 'rules' | 'budgets' | 'usage' | 'approvals' | 'receipts'> = {}): AgenticPolicyEngine {
+ if (state.version !== 1 && state.version !== POLICY_STATE_VERSION) {
+ throw new Error(`unsupported-policy-state-version:${String(state.version)}`);
+ }
const engine = new AgenticPolicyEngine({
...options,
mode: state.mode,
@@ -63,6 +67,10 @@ export class AgenticPolicyEngine {
engine.state.ledgerLength = state.ledgerLength;
engine.state.ledgerHead = state.ledgerHead ?? null;
}
+ if ((state.retainedFrom ?? 0) > 0) {
+ engine.state.retainedFrom = state.retainedFrom;
+ engine.state.prunedHead = state.prunedHead ?? null;
+ }
return engine;
}
@@ -136,7 +144,7 @@ export class AgenticPolicyEngine {
evaluate(request: PolicyRequest): PolicyDecision {
const normalized: PolicyRequest = {
...request,
- requestId: request.requestId ?? crypto.randomUUID(),
+ requestId: request.requestId ?? randomUUID(),
// Caller time is evidence only; expiry, budgets, and receipts always use
// the authority's clock.
context: { ...request.context, now: this.now() },
@@ -153,6 +161,39 @@ export class AgenticPolicyEngine {
return { ...decision, receiptId: receipt.payload.receiptId };
}
+ /** Authenticate each physical record before callers deduplicate it.
+ * Receipts use one configured key and key ID for this ledger. For existing
+ * custom-ID history, restore the original CLAUDE_FLOW_POLICY_KEY_ID when
+ * verifying; never rewrite or re-sign history to change its identity.
+ * Key rotation/multi-key historical verification is not supported here. */
+ verifyReceipt(receipt: unknown): string | undefined {
+ if (!receipt || typeof receipt !== 'object' || Array.isArray(receipt)) return 'receipt-malformed';
+ const r = receipt as PolicyReceipt;
+ if (!r.payload || typeof r.payload !== 'object' || Array.isArray(r.payload)
+ || !Number.isSafeInteger(r.payload.sequence) || r.payload.sequence < 0
+ || typeof r.payload.receiptId !== 'string'
+ || (r.payload.previousReceiptHash !== null && typeof r.payload.previousReceiptHash !== 'string')
+ || !Number.isFinite(r.payload.issuedAt)
+ || typeof r.payload.policyHash !== 'string'
+ || !r.payload.request || typeof r.payload.request !== 'object' || Array.isArray(r.payload.request)
+ || !r.payload.decision || typeof r.payload.decision !== 'object' || Array.isArray(r.payload.decision)
+ || typeof r.hash !== 'string'
+ || (r.signature !== undefined && typeof r.signature !== 'string')
+ || (r.keyId !== undefined && typeof r.keyId !== 'string')) return 'receipt-malformed';
+ let hash: string;
+ try {
+ hash = policyHash(r.payload);
+ const { receiptId, ...unsignedIdPayload } = r.payload;
+ if (hash !== r.hash || receiptId !== policyHash(unsignedIdPayload)) return 'receipt-hash-mismatch';
+ } catch { return 'receipt-malformed'; }
+ if (this.signingKey && !r.signature) return 'receipt-signature-missing';
+ if (r.signature && !this.signingKey) return 'receipt-signing-key-required';
+ if (r.signature && r.keyId !== (this.keyId ?? 'local')) return 'receipt-key-id-mismatch';
+ if (!r.signature && r.keyId !== undefined) return 'receipt-key-id-mismatch';
+ if (r.signature && this.signingKey && !verifyPolicySignature(hash, r.signature, this.signingKey)) return 'receipt-signature-invalid';
+ return undefined;
+ }
+
/**
* `establishAnchor` is the ONLY way to create an anchor over receipts that
* already exist (#3602). A caller must have decided, outside the state file,
@@ -162,29 +203,28 @@ export class AgenticPolicyEngine {
* anchor protects would otherwise let verify bless a truncated chain.
*/
verifyLedger(options: { establishAnchor?: boolean } = {}): LedgerVerification {
- let previous: string | null = null;
+ const retainedFrom = this.state.retainedFrom ?? 0;
+ // #3164: pruning only ever removes a verified prefix, so the retained
+ // tail's chain must start from the hash pruning recorded, not genesis.
+ if (retainedFrom > 0 && this.state.prunedHead == null) {
+ return { valid: false, length: retainedFrom + this.state.receipts.length, error: 'policy-ledger-prune-state-missing' };
+ }
+ let previous: string | null = retainedFrom > 0 ? (this.state.prunedHead ?? null) : null;
for (let i = 0; i < this.state.receipts.length; i++) {
const receipt = this.state.receipts[i]!;
- if (receipt.payload.sequence !== i || receipt.payload.previousReceiptHash !== previous) {
- return { valid: false, length: i, error: 'receipt-chain-mismatch' };
- }
- const hash = policyHash(receipt.payload);
- const { receiptId, ...unsignedIdPayload } = receipt.payload;
- if (hash !== receipt.hash || receiptId !== policyHash(unsignedIdPayload)) {
- return { valid: false, length: i, error: 'receipt-hash-mismatch' };
+ if (!receipt || typeof receipt !== 'object' || !receipt.payload || typeof receipt.payload !== 'object') {
+ return { valid: false, length: i + retainedFrom, error: 'receipt-malformed' };
}
- if (this.signingKey && !receipt.signature) {
- return { valid: false, length: i, error: 'receipt-signature-missing' };
- }
- if (receipt.signature && !this.signingKey) {
- return { valid: false, length: i, error: 'receipt-signing-key-required' };
- }
- if (receipt.signature && this.signingKey && !verifyPolicySignature(hash, receipt.signature, this.signingKey)) {
- return { valid: false, length: i, error: 'receipt-signature-invalid' };
+ if (receipt.payload.sequence !== i + retainedFrom || receipt.payload.previousReceiptHash !== previous) {
+ return { valid: false, length: i + retainedFrom, error: 'receipt-chain-mismatch' };
}
+ const error = this.verifyReceipt(receipt);
+ if (error) return { valid: false, length: i + retainedFrom, error };
previous = receipt.hash;
}
- const length = this.state.receipts.length;
+ // The anchor's `length` means "total receipts ever issued", which stays
+ // true of the hot tail plus whatever was pruned out from under it (#3164).
+ const length = retainedFrom + this.state.receipts.length;
// #3568: a prefix of a valid chain is itself a valid chain, so the chain
// alone cannot tell "the last receipts were deleted" from "they were never
// written". The anchor records how long the chain is and where it ends.
@@ -208,6 +248,26 @@ export class AgenticPolicyEngine {
return { valid: true, length, anchor: 'established-now' };
}
+ /**
+ * Removes the oldest receipts from the hot `receipts` array, keeping only
+ * `retain` of the most recent ones in memory (#3164). Pure and in-memory β€”
+ * this class stays side-effect-free; the runtime layer archives the
+ * returned receipts to disk and persists the resulting state. Requires an
+ * established anchor first: pruning a ledger whose historical length isn't
+ * externally anchored would make a future `verifyLedger` unable to tell
+ * truncation-by-attacker apart from legitimate pruning.
+ */
+ pruneReceipts(retain: number): PolicyReceipt[] {
+ if (!Number.isInteger(retain) || retain < 0) throw new Error('invalid-prune-retention');
+ if (this.state.receipts.length <= retain) return [];
+ if (typeof this.state.ledgerLength !== 'number') throw new Error('policy-ledger-anchor-missing');
+ const removedCount = this.state.receipts.length - retain;
+ const removed = this.state.receipts.splice(0, removedCount);
+ this.state.retainedFrom = (this.state.retainedFrom ?? 0) + removedCount;
+ this.state.prunedHead = removed[removed.length - 1]!.hash;
+ return structuredClone(removed);
+ }
+
private applyBudget(request: PolicyRequest, decision: Omit<PolicyDecision, 'receiptId'>): Omit<PolicyDecision, 'receiptId'> {
if (decision.outcome === 'denied') return decision;
const now = request.context?.now ?? this.now();
@@ -279,21 +339,23 @@ export class AgenticPolicyEngine {
}
private appendReceipt(request: PolicyRequest, decision: Omit<PolicyDecision, 'receiptId'>): PolicyReceipt {
- const previous = this.state.receipts.at(-1)?.hash ?? null;
+ const retainedFrom = this.state.retainedFrom ?? 0;
+ const previous = this.state.receipts.at(-1)?.hash ?? (retainedFrom > 0 ? this.state.prunedHead ?? null : null);
+ const totalLength = retainedFrom + this.state.receipts.length;
// #3602: appending to receipts that have no anchor would anchor them,
// which is the silent re-establishment verify refuses to do.
- if (typeof this.state.ledgerLength !== 'number' && this.state.receipts.length > 0) {
+ if (typeof this.state.ledgerLength !== 'number' && totalLength > 0) {
throw new Error('policy-ledger-anchor-missing');
}
// Never extend a chain that no longer matches its anchor: appending would
// re-anchor onto the truncated chain and erase the evidence of truncation.
if (typeof this.state.ledgerLength === 'number'
- && (this.state.ledgerLength !== this.state.receipts.length || (this.state.ledgerHead ?? null) !== previous)) {
- throw new Error(this.state.receipts.length < this.state.ledgerLength
+ && (this.state.ledgerLength !== totalLength || (this.state.ledgerHead ?? null) !== previous)) {
+ throw new Error(totalLength < this.state.ledgerLength
? 'policy-ledger-truncated'
: 'policy-ledger-anchor-mismatch');
}
- const sequence = this.state.receipts.length;
+ const sequence = totalLength;
const payloadWithoutId = {
previousReceiptHash: previous,
sequence,
@@ -313,7 +375,7 @@ export class AgenticPolicyEngine {
};
this.state.receipts.push(receipt);
this.state.ledgerHead = hash;
- this.state.ledgerLength = this.state.receipts.length;
+ this.state.ledgerLength = retainedFrom + this.state.receipts.length;
return receipt;
}
diff --git a/v3/@claude-flow/security/src/policy/types.ts b/v3/@claude-flow/security/src/policy/types.ts
index d33c46f0f9..accbca978f 100644
--- a/v3/@claude-flow/security/src/policy/types.ts
+++ b/v3/@claude-flow/security/src/policy/types.ts
@@ -1,4 +1,6 @@
-export const POLICY_STATE_VERSION = 1;
+// Version 2 is a writer fence for the archived/retained ledger protocol.
+// Version-1 binaries reject it instead of silently rewriting a pruned ledger.
+export const POLICY_STATE_VERSION = 2;
export type PolicyMode = 'legacy' | 'observe' | 'enforce';
export type PolicyEffect = 'allow' | 'deny' | 'require_approval';
@@ -184,6 +186,19 @@ export interface PolicyState {
*/
ledgerHead?: string | null;
ledgerLength?: number;
+ /**
+ * Sequence number of the oldest receipt still physically present in
+ * `receipts[0]` (#3164). 0 or absent means nothing has ever been pruned.
+ * Together with `receipts.length`, `(retainedFrom ?? 0) + receipts.length
+ * === ledgerLength` must always hold whenever `ledgerLength` is defined.
+ */
+ retainedFrom?: number;
+ /**
+ * Hash of the last receipt pruned out of `receipts` (#3164) β€” the value
+ * `receipts[0].previousReceiptHash` must equal when `retainedFrom > 0`.
+ * null when `retainedFrom` is 0/absent (chain starts at genesis).
+ */
+ prunedHead?: string | null;
}
export interface LedgerVerification {
diff --git a/v3/docs/adr/ADR-475-policy-ledger-anchor.md b/v3/docs/adr/ADR-475-policy-ledger-anchor.md
index c14bffa54d..f28e853f60 100644
--- a/v3/docs/adr/ADR-475-policy-ledger-anchor.md
+++ b/v3/docs/adr/ADR-475-policy-ledger-anchor.md
@@ -95,3 +95,98 @@ correct in what they decide; these are the places where the code is more specifi
`policy-anchor-log-missing`; only `--establish-anchor` rebuilds the log, and only when the receipts still agree with the mirror.
Nothing in the threat model (section 3) is contradicted: the mirror and log are under the same user's control, so the stated limit stands.
+
+## Update 2026-10-14: format 2 writer fence and bounded full audit
+
+Policy state format 2 prevents older format-1 writers from silently rewriting a
+retained ledger using obsolete archive/durability rules. The current loader accepts
+existing format-1 state, including pruned history. The engine preserves its rules,
+approvals, budgets, usage, receipt payloads/signatures and anchors, and writes format
+2 on the next state publication. Unknown future versions fail closed. This is a
+state-format migration, not receipt re-signing or signing-key rotation.
+
+Before rolling out a format-2 writer, stop older MCP servers and other policy
+writers for the project. Older released CLI loaders reject format 2 with
+`unsupported-policy-state-version:2`; do not restart them against migrated state.
+A binary rollback by restoring an exact pre-migration backup is supported only
+when an operator verifies that no receipts, approval uses, budget consumption or
+other acknowledged semantic changes occurred after that backup, and all writers
+are stopped. The state, archive and project/external trust anchors must be restored
+as the exact matching set. Restoring an older snapshot after acknowledged changes
+would discard authority history and could permit replay; that rollback is not
+supported. Keep a compatible writer and fix forward, or require an operator-approved
+data-preserving migration plan. There is no automatic downgrade, history rewrite
+or deletion path. These instructions do not authorize live backups, restores or
+filesystem changes.
+
+`policy verify --full` captures a checked, nofollow regular archive descriptor,
+its byte boundary, the hot state and signing configuration under the writer lock.
+It releases the lock before reading the historical prefix in 64 KiB chunks. Later
+appends are outside that audit snapshot; replacing its pathname does not replace
+the pinned descriptor. Truncation, short reads, invalid UTF-8, unterminated records
+and authentication/canonical duplicate conflicts fail verification explicitly.
+
+Memory scales with one bounded record and its parsed/canonical representations
+plus the retained hot state; canonical finite-number formatting can expand beyond
+the raw JSON input length. This is not a fixed RSS ceiling for every record. A private 0700
+scratch directory holds a 0600 disk index; it addresses only the snapshot's claimed
+sequences and is removed after success or failure. Every physical record is
+authenticated before deduplication. Duplicate checks reread and compare the entire
+canonical receipt, including signature and key ID. A receipt digest in the index
+detects changes to that observed canonical record; it does not replace exact
+canonical duplicate comparison. Unclaimed staging extras are authenticated and
+compared in separate numeric scratch files, never enormous sparse offsets.
+
+The default audit record limit is 16 MiB. Set
+`CLAUDE_FLOW_POLICY_AUDIT_MAX_RECORD_BYTES` to an integer from 1024 through 67108864
+to change the bound; invalid configuration and larger records fail with an explicit
+error, without skipping or truncating ledger bytes. This is a new full-audit
+compatibility limit and also bounds the archive-tip preflight. The audit byte
+limit and `CLAUDE_FLOW_POLICY_LEDGER_RETENTION` are operator process-level settings
+shared by CLI and long-lived MCP writers, not per-request flags. The retention
+window defaults to 2,000 receipts; pruning preserves the authenticated archive.
+Neither setting changes policy rules, budgets, approvals or receipt verification.
+Complete records
+use strict UTF-8. An incomplete EOF fragment is preserved by preflight so the
+writer can recover only its exact verified-state byte prefix; full audit rejects
+that incomplete snapshot. Legacy anchor JSON is streamed without changing its
+version-1 format or historical hash/mirror checks; each bounded JSON value is
+limited to 16 MiB of encoded UTF-8. Temporary-disk or descriptor failures also fail
+the audit. Anchor verification CPU and atomic log-rewrite I/O still scale with the
+number of anchors under the writer lock; bounded memory does not eliminate that
+separate mature-ledger contention limit.
+The default hot-path check remains tip-based, with the existing disclosed historical
+interior coverage limit. This protocol requires neither a full historical array nor
+a history-sized in-memory map, and makes no new guarantee against full local write
+access or platforms lacking directory fsync.
+
+
+### Local synthetic measurement, 2026-10-14
+
+The final bounded candidate was compared with the integrated pre-streaming
+source on the same shared Linux host using Node 24.21.0, 170,000 synthetic
+unsigned receipts, 2,000 retained receipts, a 209,900,711-byte archive and
+170,000 authenticated anchor entries. These are local measurements, not
+production capacity guarantees. The first write starts from an 84,632,655-byte
+pretty legacy anchor log. A priming transaction produces each writer's own
+format: the baseline remains pretty (84,633,165 bytes), while the candidate
+writes compact JSON (70,895,073 bytes). All prior entries remain authenticated.
+
+| Actual transaction | Baseline | Bounded candidate |
+| --- | --- | --- |
+| First legacy-log write, elapsed | 2.488 s | 3.341 s |
+| First write, process peak RSS | 636.5 MiB | 124.1 MiB |
+| Subsequent single write, elapsed | 1.795 s | 1.812 s |
+| Subsequent write, process peak RSS | 637.9 MiB | 141.2 MiB |
+| First-write simultaneous clients 1 / 4 / 16, successful returns | 1 / 3 / 2 | 1 / 3 / 2 |
+| Primed simultaneous clients 1 / 4 / 16, successful returns | 1 / 3 / 2 | 1 / 3 / 3 |
+
+The remaining clients failed with the existing policy-state lock timeout.
+Anchor validation CPU and complete atomic anchor-log rewrites remain
+O(anchor count) under the writer lock. This inherited mature-ledger contention
+issue remains unresolved under #3164; bounded full-audit memory does not make
+sixteen concurrent writers reliable. Shared-host load affects these timings;
+an earlier loaded run measured a 7.170-second first candidate write and
+4.399-second primed write. Its process RSS was additionally affected by the
+benchmark controller's inherited JSON-generation allocation, so that run is
+not used for the memory comparison above.