Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
fix(openclaw): declare transcriptSemantics and implement commitTurn (#3442)
Radhakrishnan Pachyappana28dd4eab70a
diff --git a/plugins/openclaw/src/advancement-key-store.ts b/plugins/openclaw/src/advancement-key-store.tsnew file mode 100644index 000000000..e9801e6d6--- /dev/null+++ b/plugins/openclaw/src/advancement-key-store.ts@@ -0,0 +1,240 @@+/**+ * Durable, atomic, restart-safe, cross-process-safe record of committed+ * turn advancements.+ *+ * OpenClaw's `atomic-idempotent-v1` transcriptSemantics contract requires+ * that a retried `commitTurn` call report "duplicate" for a key that was+ * already committed -- including after this process (or the host) restarts+ * -- and that "the accepted turn and key advance together" (the committed+ * messages are persisted atomically with the key, not discarded).+ *+ * The default commit-log path is shared by every gateway process using the+ * same Headroom workspace, not just every store instance in one process --+ * an in-process mutex (a JS `Map`/promise queue) does nothing to stop two+ * separate OS processes from each reading the same file, adding a different+ * key, and racing their renames, with the loser's accepted turn silently+ * discarded. The read/check/write transaction is therefore protected by a+ * real inter-process lock: an exclusive lock FILE (`<path>.lock`), acquired+ * via atomic exclusive creation (`open(..., "wx")`, which is atomic at the+ * OS level on POSIX and Windows alike) so only one process -- in this one or+ * any other -- can be inside the critical section at a time.+ *+ * The lock is never force-broken by age. An age-based "the holder must have+ * crashed" heuristic cannot tell a dead holder apart from one that is simply+ * slow (a large write, a GC pause, disk contention): a second process that+ * reclaims a lock the first still holds can commit and release before the+ * first resumes and overwrites that commit with its own now-stale snapshot+ * -- both processes report "committed", but only the second process's write+ * survives, and the first process's genuinely-accepted turn silently+ * vanishes. Node has no built-in binding for a real OS-mediated lock+ * (POSIX `flock`/Windows `LockFileEx`, where the kernel itself releases the+ * lock when the holder's process exits, crash included, so no staleness+ * guess is needed at all), so this fails closed instead: a lock that is+ * never released is never reclaimed, and a caller that can't acquire it+ * within the timeout gets a clear error rather than a silent lost write. A+ * lock file orphaned by a genuinely crashed process requires a human (or an+ * operator script) to remove `<path>.lock` before commits can resume --+ * that operational cost is the trade for never silently dropping an+ * acknowledged commit. Lock ownership is also verified before release (see+ * `withFileLock`): a process only ever deletes the lock file it created.+ *+ * Every `tryCommit` re-reads the file fresh from disk inside the lock (no+ * long-lived in-memory cache), and the in-memory return value is only ever+ * "committed" once the atomic (temp file + rename) write recording the key+ * has actually succeeded -- a failed write is never remembered as seen, so+ * retrying the same key attempts the commit again instead of returning a+ * phantom "duplicate" for a commit that never reached disk.+ *+ * Keys are never evicted. OpenClaw may retry at any point after a restart or+ * a long stall, so pruning by count or age would risk re-accepting an+ * already-committed turn -- exactly the failure this store exists to avoid.+ */++import { mkdir, open, readFile, rename, stat, unlink, writeFile } from "node:fs/promises";+import { dirname, join } from "node:path";+import * as os from "node:os";++/**+ * True if `p` exists and is something other than a directory (e.g. a file+ * sitting where a directory needs to be). Some platforms (observed on+ * Windows) report `ENOENT` -- not a distinct "not a directory" code -- for+ * `readFile` when a path component like this blocks the rest of the path,+ * which is indistinguishable from "the commit log just doesn't exist yet"+ * by error code alone. Checking the parent directly is what tells the two+ * apart: silently treating a broken storage path as an empty store would+ * make every future commit re-accept an already-committed key.+ */+async function isBlockingNonDirectory(p: string): Promise<boolean> {+ try {+ const info = await stat(p);+ return !info.isDirectory();+ } catch {+ return false; // Doesn't exist -- not blocking anything.+ }+}++/** Default commit-log location, honoring the same workspace override the+ * rest of the Headroom workspace convention uses (`HEADROOM_WORKSPACE_DIR`). */+export function defaultCommitLogPath(): string {+ const workspaceDir = process.env.HEADROOM_WORKSPACE_DIR?.trim() || join(os.homedir(), ".headroom");+ return join(workspaceDir, "openclaw", "commit-log.json");+}++/** One durably-committed turn: the accepted messages plus when they landed. */+export interface CommittedTurn {+ messages: unknown;+ committedAt: string;+}++type CommitLog = Record<string, CommittedTurn>;++/** Give up (rather than poll forever) if the lock still can't be acquired after this long. */+const LOCK_ACQUIRE_TIMEOUT_MS = 60_000;+const LOCK_POLL_BASE_MS = 20;++function sleep(ms: number): Promise<void> {+ return new Promise((resolve) => setTimeout(resolve, ms));+}++/**+ * Run `fn` while holding an exclusive lock on `path` (a sibling `<path>.lock`+ * file). Serializes every caller -- this process or another -- against the+ * same file, not just calls on one object or one process.+ *+ * The lock is held until `fn` returns, however long that takes, and is+ * never reclaimed by another caller on a staleness guess -- see the module+ * docstring for why an age-based heuristic can silently drop a genuinely+ * accepted commit. A caller that can't acquire the lock within+ * `LOCK_ACQUIRE_TIMEOUT_MS` gets a clear timeout error instead of the+ * transaction being allowed to proceed unsafely.+ */+async function withFileLock<T>(path: string, fn: () => Promise<T>): Promise<T> {+ const lockPath = `${path}.lock`;+ await mkdir(dirname(path), { recursive: true });+ // Unique per acquisition, so release only ever removes the lock file this+ // exact call created -- never one a different (later) holder created.+ const token = `${process.pid}-${Date.now()}-${Math.random().toString(36).slice(2)}`;+ const deadline = Date.now() + LOCK_ACQUIRE_TIMEOUT_MS;+ let attempt = 0;+ for (;;) {+ try {+ const handle = await open(lockPath, "wx");+ try {+ await handle.writeFile(`${token}\n${process.pid}\n${new Date().toISOString()}\n`);+ } finally {+ await handle.close();+ }+ break;+ } catch (error) {+ if ((error as NodeJS.ErrnoException)?.code !== "EEXIST") {+ throw error;+ }+ if (Date.now() > deadline) {+ throw new Error(+ `Timed out waiting for the advancement-key-store lock on ${path}. If the ` ++ `process that created ${lockPath} is no longer running, remove it manually.`,+ );+ }+ attempt += 1;+ await sleep(LOCK_POLL_BASE_MS + Math.random() * LOCK_POLL_BASE_MS * Math.min(attempt, 10));+ }+ }+ try {+ return await fn();+ } finally {+ // Verify the lock file is still the one this call created before+ // removing it -- if it doesn't start with our token, some other process+ // (or a human) replaced or is using it, and it must not be touched.+ try {+ const current = await readFile(lockPath, "utf8");+ if (current.startsWith(`${token}\n`)) {+ await unlink(lockPath);+ }+ } catch (error) {+ if ((error as NodeJS.ErrnoException)?.code !== "ENOENT") {+ throw error;+ }+ }+ }+}++export class DurableAdvancementKeyStore {+ constructor(private readonly path: string) {}++ private async readAll(): Promise<CommitLog> {+ let raw: string;+ try {+ raw = await readFile(this.path, "utf8");+ } catch (error) {+ const code = (error as NodeJS.ErrnoException)?.code;+ if (code !== "ENOENT" && code !== "ENOTDIR") {+ throw error;+ }+ if (await isBlockingNonDirectory(dirname(this.path))) {+ throw error;+ }+ return {};+ }+ const parsed = JSON.parse(raw) as unknown;+ if (parsed && typeof parsed === "object" && !Array.isArray(parsed)) {+ return parsed as CommitLog;+ }+ if (Array.isArray(parsed)) {+ // Migrate the earlier key-only array format (no message payload) so+ // upgrading from it never re-accepts an already-committed key.+ const migrated: CommitLog = {};+ for (const entry of parsed) {+ if (typeof entry === "string") {+ migrated[entry] = { messages: null, committedAt: "" };+ }+ }+ return migrated;+ }+ return {};+ }++ private async writeAll(entries: CommitLog): Promise<void> {+ await mkdir(dirname(this.path), { recursive: true });+ const tmpPath = `${this.path}.${process.pid}.${Math.random().toString(36).slice(2)}.tmp`;+ await writeFile(tmpPath, JSON.stringify(entries), "utf8");+ await rename(tmpPath, this.path);+ }++ /**+ * Atomically check-and-record `key` together with the `messages` it+ * advances. Holds the cross-process file lock (see `withFileLock`) for+ * the full read/check/write transaction, so two processes -- or two+ * instances in this one -- can't each read the same starting state and+ * race their writes.+ *+ * Throws if the write fails -- the caller sees a rejected commit, not a+ * false "committed"/"duplicate", and a retry re-reads the (still+ * unchanged) on-disk state and attempts the write again.+ */+ async tryCommit(key: string, messages: unknown): Promise<"committed" | "duplicate"> {+ return withFileLock(this.path, async () => {+ const entries = await this.readAll();+ if (key in entries) {+ return "duplicate";+ }+ const next: CommitLog = {+ ...entries,+ [key]: { messages, committedAt: new Date().toISOString() },+ };+ await this.writeAll(next);+ return "committed";+ });+ }++ /** Test-only: check membership without recording. */+ async has(key: string): Promise<boolean> {+ const entries = await this.readAll();+ return key in entries;+ }++ /** Test-only: read back a committed entry (messages + commit time) without recording. */+ async get(key: string): Promise<CommittedTurn | undefined> {+ const entries = await this.readAll();+ return entries[key];+ }+}diff --git a/plugins/openclaw/src/engine.ts b/plugins/openclaw/src/engine.tsindex 71a3836fb..7b2a8f5ed 100644--- a/plugins/openclaw/src/engine.ts+++ b/plugins/openclaw/src/engine.ts@@ -10,6 +10,7 @@import { compress } from "headroom-ai";import { ProxyManager, defaultLogger, type ProxyManagerConfig, type ProxyManagerLogger } from "./proxy-manager.js";import { agentToOpenAI, normalizeAgentMessages, openAIToAgent } from "./convert.js";+import { DurableAdvancementKeyStore, defaultCommitLogPath } from "./advancement-key-store.js";import {delegateCompactionToRuntime,type OpenClawCompactParams,@@ -32,6 +33,9 @@ export interface HeadroomEngineConfig extends ProxyManagerConfig {requestTimeoutMs?: number;circuitBreakerThreshold?: number;circuitBreakerCooldownMs?: number;+ /** Where to durably record committed turn-advancement keys (see+ * `DurableAdvancementKeyStore`). Defaults to `defaultCommitLogPath()`. */+ commitLogPath?: string;}export class HeadroomContextEngine {@@ -40,8 +44,17 @@ export class HeadroomContextEngine {name: "Headroom Context Compression",version: "0.1.0",ownsCompaction: false,+ transcriptSemantics: {+ currentTurnFence: "before-current-turn-entry-v1",+ turnAdvancementIdempotency: "atomic-idempotent-v1",+ },};+ // Durable, restart-safe record of committed advancement keys, for+ // commitTurn's idempotent-retry check. See advancement-key-store.ts for+ // why this must survive a process restart and must not evict entries.+ private advancementKeyStore: DurableAdvancementKeyStore;+private proxyManager: ProxyManager;private proxyUrl: string | null = null;private config: HeadroomEngineConfig;@@ -61,6 +74,9 @@ export class HeadroomContextEngine {this.config = config;this.logger = logger ?? defaultLogger;this.proxyManager = new ProxyManager(config, this.logger);+ this.advancementKeyStore = new DurableAdvancementKeyStore(+ config.commitLogPath ?? defaultCommitLogPath(),+ );}// === ContextEngine Lifecycle ===@@ -189,6 +205,23 @@ export class HeadroomContextEngine {return result;}+ /**+ * Durable turn-advancement commit — required by OpenClaw's transcriptSemantics+ * contract. Called only for the accepted, successful turn; failed or aborted+ * turns never reach here. Must be an atomic, idempotent write of the+ * accepted `messages` keyed by `advancementKey` so a host retry with the+ * same key reports "duplicate" instead of re-applying the advancement —+ * including a retry that arrives after this process restarted, which is+ * why the record lives on disk (see `DurableAdvancementKeyStore`) rather+ * than in memory, and includes the messages rather than just the key.+ */+ async commitTurn(params: { advancementKey: string; messages: any[] }): Promise<{+ status: "committed" | "duplicate";+ }> {+ const status = await this.advancementKeyStore.tryCommit(params.advancementKey, params.messages);+ return { status };+ }+async afterTurn?(params: {sessionId: string;messages: any[];diff --git a/plugins/openclaw/test/advancement-key-store.test.ts b/plugins/openclaw/test/advancement-key-store.test.tsnew file mode 100644index 000000000..765e66bb1--- /dev/null+++ b/plugins/openclaw/test/advancement-key-store.test.ts@@ -0,0 +1,350 @@+import { execFile } from "node:child_process";+import * as fs from "node:fs/promises";+import * as os from "node:os";+import * as path from "node:path";+import { fileURLToPath } from "node:url";+import { promisify } from "node:util";++import { afterEach, beforeEach, describe, expect, it } from "vitest";++import { DurableAdvancementKeyStore, defaultCommitLogPath } from "../src/advancement-key-store.js";++const execFileAsync = promisify(execFile);++// Node's native TypeScript support (needed to run the worker fixture, which+// imports the real DurableAdvancementKeyStore, as a plain `node` child+// process with no build step) requires `--experimental-transform-types`,+// added in Node 22.6. Skip the cross-process tests rather than fail on an+// older Node -- the failure mode being tested (a lost write across two OS+// processes) is orthogonal to which Node version happens to run the test.+const NODE_SUPPORTS_TRANSFORM_TYPES = (() => {+ const [major, minor] = process.versions.node.split(".").map(Number);+ return major > 22 || (major === 22 && minor >= 6);+})();++const WORKER_PATH = fileURLToPath(new URL("./fixtures/commit-turn-worker.ts", import.meta.url));+const HOLD_LOCK_WORKER_PATH = fileURLToPath(+ new URL("./fixtures/hold-lock-worker.ts", import.meta.url),+);++/** Run the commit-turn worker as a real child process; returns its parsed JSON result. */+async function runCommitWorker(+ storePath: string,+ key: string,+ goPath: string,+ resultPath: string,+): Promise<{ status?: string; error?: string }> {+ await execFileAsync(process.execPath, [+ "--experimental-transform-types",+ WORKER_PATH,+ storePath,+ key,+ goPath,+ resultPath,+ ]);+ return JSON.parse(await fs.readFile(resultPath, "utf8"));+}++async function waitForFile(p: string): Promise<void> {+ for (;;) {+ try {+ await fs.access(p);+ return;+ } catch {+ await new Promise((r) => setTimeout(r, 5));+ }+ }+}++describe("DurableAdvancementKeyStore", () => {+ let dir: string;+ let storePath: string;++ beforeEach(async () => {+ dir = await fs.mkdtemp(path.join(os.tmpdir(), "headroom-advancement-key-store-"));+ storePath = path.join(dir, "nested", "commit-log.json");+ });++ afterEach(async () => {+ await fs.rm(dir, { recursive: true, force: true });+ });++ it("commits a new key and creates the parent directory", async () => {+ const store = new DurableAdvancementKeyStore(storePath);++ await expect(store.tryCommit("turn-1", [{ role: "user", content: "hi" }])).resolves.toBe(+ "committed",+ );+ await expect(fs.stat(storePath)).resolves.toBeTruthy();+ });++ it("reports duplicate for an already-committed key", async () => {+ const store = new DurableAdvancementKeyStore(storePath);++ await expect(store.tryCommit("turn-1", [])).resolves.toBe("committed");+ await expect(store.tryCommit("turn-1", [])).resolves.toBe("duplicate");+ });++ it("persists across a simulated restart (a fresh store instance, same path)", async () => {+ const before = new DurableAdvancementKeyStore(storePath);+ await expect(before.tryCommit("turn-1", [])).resolves.toBe("committed");++ const after = new DurableAdvancementKeyStore(storePath);+ await expect(after.tryCommit("turn-1", [])).resolves.toBe("duplicate");+ // A key that was never committed is still accepted normally.+ await expect(after.tryCommit("turn-2", [])).resolves.toBe("committed");+ });++ it("treats a missing commit-log file as an empty store rather than an error", async () => {+ const store = new DurableAdvancementKeyStore(storePath);++ await expect(store.has("turn-1")).resolves.toBe(false);+ await expect(store.tryCommit("turn-1", [])).resolves.toBe("committed");+ });++ it("fails loudly on a corrupt commit-log file instead of silently forgetting its contents", async () => {+ await fs.mkdir(path.dirname(storePath), { recursive: true });+ await fs.writeFile(storePath, "{not valid json", "utf8");+ const store = new DurableAdvancementKeyStore(storePath);++ await expect(store.tryCommit("turn-1", [])).rejects.toThrow();+ });++ it("serializes concurrent commits of the same key so exactly one wins", async () => {+ const store = new DurableAdvancementKeyStore(storePath);++ const results = await Promise.all([+ store.tryCommit("turn-race", []),+ store.tryCommit("turn-race", []),+ store.tryCommit("turn-race", []),+ ]);++ expect(results.filter((r) => r === "committed")).toHaveLength(1);+ expect(results.filter((r) => r === "duplicate")).toHaveLength(2);+ });++ it(+ "never evicts a key regardless of how many others were committed since",+ async () => {+ const store = new DurableAdvancementKeyStore(storePath);++ for (let i = 0; i < 600; i++) {+ await store.tryCommit(`turn-${i}`, []);+ }++ await expect(store.tryCommit("turn-0", [])).resolves.toBe("duplicate");+ },+ 20_000,+ );++ // ---------------------------------------------------------------------+ // PR #3442 review, round 2: three additional atomic-commit failures.+ // ---------------------------------------------------------------------++ it("persists the accepted messages together with the key, not just the key", async () => {+ const store = new DurableAdvancementKeyStore(storePath);+ const messages = [+ { role: "user", content: "what's the weather?" },+ { role: "assistant", content: "Let me check." },+ ];++ await store.tryCommit("turn-1", messages);++ const entry = await store.get("turn-1");+ expect(entry).toBeDefined();+ expect(entry?.messages).toEqual(messages);+ expect(typeof entry?.committedAt).toBe("string");+ expect(entry?.committedAt.length).toBeGreaterThan(0);++ // Survives a restart too -- the messages aren't an in-memory-only echo.+ const after = new DurableAdvancementKeyStore(storePath);+ const reloaded = await after.get("turn-1");+ expect(reloaded?.messages).toEqual(messages);+ });++ it("does not remember a key when storage is inaccessible, so a retry can still succeed", async () => {+ // Block the parent directory with a file. Both the commit's initial+ // read and a membership lookup must propagate the filesystem error.+ const blockedParent = path.join(dir, "blocked");+ await fs.writeFile(blockedParent, "not a directory", "utf8");+ const blockedPath = path.join(blockedParent, "commit-log.json");+ const store = new DurableAdvancementKeyStore(blockedPath);++ await expect(store.tryCommit("turn-1", [])).rejects.toThrow();+ // The failed write must not be remembered in memory: retrying the same+ // key must attempt the commit again (and fail again, for the same+ // reason) rather than silently reporting "duplicate" for a commit that+ // never reached disk.+ await expect(store.tryCommit("turn-1", [])).rejects.toThrow();+ await expect(store.has("turn-1")).rejects.toThrow();++ // Once the underlying problem is fixed, the key is still absent and+ // the same key commits normally.+ await fs.rm(blockedParent, { force: true });+ await expect(store.has("turn-1")).resolves.toBe(false);+ await expect(store.tryCommit("turn-1", [])).resolves.toBe("committed");+ await expect(store.has("turn-1")).resolves.toBe(true);+ });++ it("serializes across store instances sharing the same file, not only within one object", async () => {+ const storeA = new DurableAdvancementKeyStore(storePath);+ const storeB = new DurableAdvancementKeyStore(storePath);++ // Interleaved commits through two independent instances against the+ // same file must not lose either one to a read-modify-write race.+ const [resultA, resultB] = await Promise.all([+ storeA.tryCommit("turn-a", ["from a"]),+ storeB.tryCommit("turn-b", ["from b"]),+ ]);++ expect(resultA).toBe("committed");+ expect(resultB).toBe("committed");++ // A third, fresh instance (simulating a restart) must see both.+ const after = new DurableAdvancementKeyStore(storePath);+ await expect(after.has("turn-a")).resolves.toBe(true);+ await expect(after.has("turn-b")).resolves.toBe(true);+ await expect(after.tryCommit("turn-a", [])).resolves.toBe("duplicate");+ await expect(after.tryCommit("turn-b", [])).resolves.toBe("duplicate");+ });++ // -------------------------------------------------------------------+ // PR #3442 review, round 3: an in-process lock (a JS Map/promise queue)+ // does nothing to stop two separate OS processes -- e.g. two gateway+ // processes sharing one Headroom workspace -- from racing the same+ // commit-log file. These spawn the real DurableAdvancementKeyStore in+ // real child processes (not Worker threads, which share the parent's+ // memory and wouldn't exercise inter-process file locking) and release+ // them via a shared "go" file so their read/check/write transactions+ // contend for the file lock as close to simultaneously as possible.+ // -------------------------------------------------------------------++ it.skipIf(!NODE_SUPPORTS_TRANSFORM_TYPES)(+ "serializes commits from two separate OS processes sharing the same file",+ async () => {+ const goPath = path.join(dir, "go");+ const resultAPath = path.join(dir, "result-a.json");+ const resultBPath = path.join(dir, "result-b.json");++ const workerA = runCommitWorker(storePath, "turn-a", goPath, resultAPath);+ const workerB = runCommitWorker(storePath, "turn-b", goPath, resultBPath);+ // Give both worker processes time to spawn and reach their poll loop+ // before releasing them together.+ await new Promise((r) => setTimeout(r, 300));+ await fs.writeFile(goPath, "go", "utf8");++ const [resultA, resultB] = await Promise.all([workerA, workerB]);++ expect(resultA).toEqual({ status: "committed" });+ expect(resultB).toEqual({ status: "committed" });++ // Neither commit was lost to a cross-process race.+ const after = new DurableAdvancementKeyStore(storePath);+ await expect(after.has("turn-a")).resolves.toBe(true);+ await expect(after.has("turn-b")).resolves.toBe(true);+ },+ 30_000,+ );++ it.skipIf(!NODE_SUPPORTS_TRANSFORM_TYPES)(+ "reports duplicate for the same key committed from two separate OS processes",+ async () => {+ const goPath = path.join(dir, "go");+ const resultAPath = path.join(dir, "result-a.json");+ const resultBPath = path.join(dir, "result-b.json");++ const workerA = runCommitWorker(storePath, "turn-same", goPath, resultAPath);+ const workerB = runCommitWorker(storePath, "turn-same", goPath, resultBPath);+ await new Promise((r) => setTimeout(r, 300));+ await fs.writeFile(goPath, "go", "utf8");++ const [resultA, resultB] = await Promise.all([workerA, workerB]);++ // Exactly one process's transaction should win the race and see+ // "committed"; the other must see "duplicate", never both+ // "committed" (which would mean the file lock let them interleave).+ const statuses = [resultA.status, resultB.status].sort();+ expect(statuses).toEqual(["committed", "duplicate"]);+ },+ 30_000,+ );++ it.skipIf(!NODE_SUPPORTS_TRANSFORM_TYPES)(+ "never lets a second writer commit while a live holder still holds the lock",+ async () => {+ // Regression for PR #3442 review, round 4: a lock reclaimed on an+ // age-based staleness guess let a second process commit and release+ // while the first (merely slow, not dead) holder still believed it+ // owned the lock -- the first then resumed and overwrote the+ // second's accepted commit with its own stale snapshot. This proves+ // the invariant that replaces that heuristic: a second writer must+ // never proceed while ANY holder -- however long it takes -- still+ // holds the lock. Deterministic and fast: the "holder" is a worker+ // that occupies the real lock file and only releases it when told+ // to, rather than relying on a real multi-second sleep.+ const readyPath = path.join(dir, "holder-ready");+ const releasePath = path.join(dir, "holder-release");++ const holder = execFileAsync(process.execPath, [+ "--experimental-transform-types",+ HOLD_LOCK_WORKER_PATH,+ storePath,+ readyPath,+ releasePath,+ ]);++ // Wait until the holder has actually acquired the lock before racing+ // a commit against it.+ await waitForFile(readyPath);++ const store = new DurableAdvancementKeyStore(storePath);+ let secondCommitSettled = false;+ const secondCommit = store.tryCommit("turn-b", ["b"]).finally(() => {+ secondCommitSettled = true;+ });++ // Give the second commit ample opportunity to (wrongly) proceed+ // while the holder is still alive and has not released.+ try {+ await new Promise((r) => setTimeout(r, 500));+ expect(secondCommitSettled).toBe(false);+ } finally {+ // Release even if the assertion fails, so the child cannot leak.+ await fs.writeFile(releasePath, "go", "utf8");+ await holder;+ }+ // The second commit must now complete successfully, with nothing lost.+ await expect(secondCommit).resolves.toBe("committed");+ await expect(store.has("turn-b")).resolves.toBe(true);+ },+ 15_000,+ );+});++describe("defaultCommitLogPath", () => {+ const originalWorkspaceDir = process.env.HEADROOM_WORKSPACE_DIR;++ afterEach(() => {+ if (originalWorkspaceDir === undefined) {+ delete process.env.HEADROOM_WORKSPACE_DIR;+ } else {+ process.env.HEADROOM_WORKSPACE_DIR = originalWorkspaceDir;+ }+ });++ it("honors HEADROOM_WORKSPACE_DIR when set", () => {+ process.env.HEADROOM_WORKSPACE_DIR = "/custom/workspace";++ expect(defaultCommitLogPath()).toBe(+ path.join("/custom/workspace", "openclaw", "commit-log.json"),+ );+ });++ it("falls back to ~/.headroom when unset", () => {+ delete process.env.HEADROOM_WORKSPACE_DIR;++ expect(defaultCommitLogPath()).toBe(+ path.join(os.homedir(), ".headroom", "openclaw", "commit-log.json"),+ );+ });+});diff --git a/plugins/openclaw/test/engine.test.ts b/plugins/openclaw/test/engine.test.tsindex d9886f707..786717727 100644--- a/plugins/openclaw/test/engine.test.ts+++ b/plugins/openclaw/test/engine.test.ts@@ -1,4 +1,8 @@-import { afterEach, describe, expect, it, vi } from "vitest";+import * as fs from "node:fs/promises";+import * as os from "node:os";+import * as path from "node:path";++import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";const mocked = vi.hoisted(() => ({compress: vi.fn(),@@ -325,3 +329,102 @@ describe("HeadroomContextEngine proxy startup helpers", () => {);});});++describe("HeadroomContextEngine transcriptSemantics contract", () => {+ let commitLogDir: string;+ let commitLogPath: string;++ beforeEach(async () => {+ commitLogDir = await fs.mkdtemp(path.join(os.tmpdir(), "headroom-commit-log-"));+ commitLogPath = path.join(commitLogDir, "commit-log.json");+ });++ afterEach(async () => {+ await fs.rm(commitLogDir, { recursive: true, force: true });+ });++ it("declares the durable-commit transcript semantics OpenClaw requires", () => {+ const engine = new HeadroomContextEngine({ commitLogPath });++ expect(engine.info.transcriptSemantics).toEqual({+ currentTurnFence: "before-current-turn-entry-v1",+ turnAdvancementIdempotency: "atomic-idempotent-v1",+ });+ });++ it("commits a new advancement key", async () => {+ const engine = new HeadroomContextEngine({ commitLogPath });++ await expect(+ engine.commitTurn({ advancementKey: "turn-1", messages: [] }),+ ).resolves.toEqual({ status: "committed" });+ });++ it("reports duplicate on a retried advancement key", async () => {+ const engine = new HeadroomContextEngine({ commitLogPath });++ await expect(+ engine.commitTurn({ advancementKey: "turn-1", messages: [] }),+ ).resolves.toEqual({ status: "committed" });+ await expect(+ engine.commitTurn({ advancementKey: "turn-1", messages: [] }),+ ).resolves.toEqual({ status: "duplicate" });+ });++ it("treats distinct advancement keys independently", async () => {+ const engine = new HeadroomContextEngine({ commitLogPath });++ await expect(+ engine.commitTurn({ advancementKey: "turn-1", messages: [] }),+ ).resolves.toEqual({ status: "committed" });+ await expect(+ engine.commitTurn({ advancementKey: "turn-2", messages: [] }),+ ).resolves.toEqual({ status: "committed" });+ });++ it("reports duplicate for a key committed before a process restart", async () => {+ // Simulate a restart: a brand new engine instance (no shared in-memory+ // state) pointed at the same durable commit-log path.+ const before = new HeadroomContextEngine({ commitLogPath });+ await expect(+ before.commitTurn({ advancementKey: "turn-restart", messages: [] }),+ ).resolves.toEqual({ status: "committed" });++ const after = new HeadroomContextEngine({ commitLogPath });+ await expect(+ after.commitTurn({ advancementKey: "turn-restart", messages: [] }),+ ).resolves.toEqual({ status: "duplicate" });+ });++ it(+ "never forgets a key regardless of how many other keys were committed since",+ async () => {+ // Regression: the old implementation evicted the oldest key past a+ // 512-entry cap, so a retry of an early key was wrongly re-accepted as+ // new instead of reported as a duplicate. There is no such cap now.+ const engine = new HeadroomContextEngine({ commitLogPath });++ for (let i = 0; i < 600; i++) {+ await engine.commitTurn({ advancementKey: `turn-${i}`, messages: [] });+ }++ await expect(+ engine.commitTurn({ advancementKey: "turn-0", messages: [] }),+ ).resolves.toEqual({ status: "duplicate" });+ },+ 20_000,+ );++ it("persists the accepted messages together with the advancement key", async () => {+ const engine = new HeadroomContextEngine({ commitLogPath });+ const messages = [{ role: "user", content: "hello" }];++ await expect(+ engine.commitTurn({ advancementKey: "turn-1", messages }),+ ).resolves.toEqual({ status: "committed" });++ const raw = await fs.readFile(commitLogPath, "utf8");+ const entries = JSON.parse(raw) as Record<string, { messages: unknown }>;+ expect(entries["turn-1"].messages).toEqual(messages);+ });+});diff --git a/plugins/openclaw/test/fixtures/commit-turn-worker.ts b/plugins/openclaw/test/fixtures/commit-turn-worker.tsnew file mode 100644index 000000000..440d8c10b--- /dev/null+++ b/plugins/openclaw/test/fixtures/commit-turn-worker.ts@@ -0,0 +1,59 @@+/**+ * Child-process worker for the cross-process DurableAdvancementKeyStore+ * regression tests (PR #3442 review, round 3).+ *+ * Run as a real, separate OS process (not a Worker thread, which shares the+ * parent's memory and wouldn't exercise inter-process file locking) via+ * `node --experimental-transform-types commit-turn-worker.ts <storePath>+ * <key> <goPath> <resultPath>`.+ *+ * Blocks until `goPath` exists (the parent test creates it only after every+ * worker has been spawned and had time to reach this poll loop), then+ * commits `key` through the real `DurableAdvancementKeyStore` and writes+ * the outcome to `resultPath` as JSON -- this is what lets two independently+ * launched processes race their read/check/write transactions against the+ * same commit-log file as closely to simultaneously as possible.+ */++import { access, writeFile } from "node:fs/promises";++// Explicit `.ts` extension: this file runs directly under Node's native+// TypeScript support (`--experimental-transform-types`), which -- unlike+// the TypeScript compiler / bundler used to build the plugin itself --+// resolves imports literally and does not map a `.js` specifier onto a+// sibling `.ts` file.+import { DurableAdvancementKeyStore } from "../../src/advancement-key-store.ts";++async function waitForGo(goPath: string): Promise<void> {+ for (;;) {+ try {+ await access(goPath);+ return;+ } catch {+ await new Promise((resolveDelay) => setTimeout(resolveDelay, 5));+ }+ }+}++async function main(): Promise<void> {+ const [, , storePath, key, goPath, resultPath] = process.argv;+ if (!storePath || !key || !goPath || !resultPath) {+ throw new Error("usage: commit-turn-worker.ts <storePath> <key> <goPath> <resultPath>");+ }++ await waitForGo(goPath);++ const store = new DurableAdvancementKeyStore(storePath);+ try {+ const status = await store.tryCommit(key, [key]);+ await writeFile(resultPath, JSON.stringify({ status }), "utf8");+ } catch (error) {+ await writeFile(+ resultPath,+ JSON.stringify({ error: error instanceof Error ? error.message : String(error) }),+ "utf8",+ );+ }+}++main();diff --git a/plugins/openclaw/test/fixtures/hold-lock-worker.ts b/plugins/openclaw/test/fixtures/hold-lock-worker.tsnew file mode 100644index 000000000..9845ce110--- /dev/null+++ b/plugins/openclaw/test/fixtures/hold-lock-worker.ts@@ -0,0 +1,52 @@+/**+ * Simulates a live-but-slow lock holder for the DurableAdvancementKeyStore+ * cross-process regression tests (PR #3442 review, round 4): acquires the+ * exact same lock file the real store uses, signals readiness, then holds+ * it until told to release -- proving a real, still-alive holder is never+ * force-preempted no matter how long its critical section runs, unlike the+ * round-3 age-based staleness heuristic this replaces.+ *+ * Deliberately does not import `DurableAdvancementKeyStore`: it only needs+ * to occupy the lock file at the path the real class would use, using the+ * same `open(path, "wx")` protocol, so the *other* worker's real+ * `tryCommit` call is what gets exercised and observed.+ *+ * Usage: node --experimental-transform-types hold-lock-worker.ts+ * <storePath> <readyPath> <releasePath>+ */++import { access, mkdir, open, unlink, utimes, writeFile } from "node:fs/promises";+import { dirname } from "node:path";++async function waitFor(p: string): Promise<void> {+ for (;;) {+ try {+ await access(p);+ return;+ } catch {+ await new Promise((resolveDelay) => setTimeout(resolveDelay, 5));+ }+ }+}++async function main(): Promise<void> {+ const [, , storePath, readyPath, releasePath] = process.argv;+ if (!storePath || !readyPath || !releasePath) {+ throw new Error("usage: hold-lock-worker.ts <storePath> <readyPath> <releasePath>");+ }++ const lockPath = `${storePath}.lock`;+ await mkdir(dirname(storePath), { recursive: true });+ const handle = await open(lockPath, "wx");+ await handle.close();+ // Exercise the former 30-second reclaim threshold without a slow test.+ const oldTime = new Date(Date.now() - 60_000);+ await utimes(lockPath, oldTime, oldTime);++ await writeFile(readyPath, "ready", "utf8");++ await waitFor(releasePath);+ await unlink(lockPath).catch(() => undefined);+}++main();