Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.
Python agent-contextagent-memoryagent-pluginsagentic-ragcontext-databasedsh-pluginknowledge-basememorymemory-managementself-evolvingvector-database
fix(codex-plugin): exclude host startup context from captured sessions (#5392)
t0saki276dfffc8017
diff --git a/examples/codex-memory-plugin/.codex-plugin/plugin.json b/examples/codex-memory-plugin/.codex-plugin/plugin.jsonindex 122744a78..27a5da640 100644--- a/examples/codex-memory-plugin/.codex-plugin/plugin.json+++ b/examples/codex-memory-plugin/.codex-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "openviking-memory",- "version": "0.10.4",+ "version": "0.10.5","description": "Long-term semantic memory for Codex, powered by OpenViking. Recall on UserPromptSubmit, incremental add_message on Stop (per turn), commit on SessionEnd (Codex >= 0.145) and on PreCompact, with an idle/ended sweep on SessionStart as the fallback for signals, crashes and older Codex builds. Includes a local stdio MCP proxy that reads OpenViking credentials from env/ovcli.conf.","author": {"name": "OpenViking",diff --git a/examples/codex-memory-plugin/DESIGN.md b/examples/codex-memory-plugin/DESIGN.mdindex 43ede546e..4f8bfe456 100644--- a/examples/codex-memory-plugin/DESIGN.md+++ b/examples/codex-memory-plugin/DESIGN.md@@ -18,7 +18,7 @@ events imply "context for a particular codex `session_id` is gone".memory extractor) at session-end-equivalent moments. `/messages`auto-creates the OV session, so the plugin does not call session create.- **State file** — `~/.openviking/codex-plugin-state/<safe-codex-session-id>.json`,- shape `{ codexSessionId, ovSessionId, transcriptPath, capturedTurnCount, createdAt, lastUpdatedAt }`.+ shape `{ codexSessionId, ovSessionId, transcriptPath, capturedTurnCount, captureFormatVersion, createdAt, lastUpdatedAt }`.- **End marker** — `<safe-codex-session-id>.ended.<timestamp>`, a sidecar written by the`SessionEnd` parent hook, containing the timestamp at which it waswritten. Its presence means "the thread ended and its commit has not been@@ -210,8 +210,12 @@ a state file that never captured a turn is identical to no state file at all.Every `Stop` reads `transcript_path`, slices to `[capturedTurnCount, end)`,and appends each new user/assistant turn to the OV session for this codex`session_id` (the `/messages` endpoint auto-creates it on first append).-State is updated:-`{ovSessionId, capturedTurnCount, lastUpdatedAt: now}`.+Codex's startup-only user message is removed before slicing. Only complete+host blocks before the first `turn_context` qualify: plugin recommendations,+`AGENTS.md instructions`, and `<environment_context>`. If a message also+contains a real prompt, only those blocks are removed. Normal conversation+may quote the same labels without being filtered. State is updated:+`{ovSessionId, capturedTurnCount, captureFormatVersion, lastUpdatedAt: now}`.The transcript and persisted cursor own retries. Failed messages are not alsoput in the shared pending queue: replaying both sources would duplicate them.@@ -359,6 +363,7 @@ OV session id, while commits create additional archives under that session."ovSessionId": "cx-0193af...-or-null", // null means "committed, awaiting next Stop or retirement""transcriptPath": "/path/rollout.jsonl", // last rollout seen; lets the sweep catch up"capturedTurnCount": 7, // turns from transcript already appended+ "captureFormatVersion": 2, // cursor counts turns after Codex startup filtering"createdAt": 1715000000000,"lastUpdatedAt": 1715000300000}@@ -369,6 +374,12 @@ Legacy state files from earlier plugin versions may still contain a UUIDnext resolve. The migration window for preserving old UUID sessions hasclosed.+A state file without `captureFormatVersion` uses the old extraction count.+On the first readable rollout under the session lock, the plugin subtracts+startup turns before the old cursor and persists version 2 before appending.+Unreadable rollouts leave the old cursor untouched for a later retry. Already+written OV sessions and extracted memories are not changed.+State files are atomic-write (tmpfile + rename) to survive crash mid-write.Two sidecars live next to `<safe-codex-session-id>.json`:diff --git a/examples/codex-memory-plugin/scripts/capture-utils.mjs b/examples/codex-memory-plugin/scripts/capture-utils.mjsindex 508907d35..1bcbe5915 100644--- a/examples/codex-memory-plugin/scripts/capture-utils.mjs+++ b/examples/codex-memory-plugin/scripts/capture-utils.mjs@@ -353,9 +353,93 @@ function normalizeCodexCompletedToolEvents(rolloutEntries) {});}-export function extractCaptureTurns(rolloutEntries, cfg = {}) {+// Codex writes these host-provided blocks as a user message before its first+// turn_context. Match the complete wrapper, not a tag mentioned by a user.+const STARTUP_BLOCKS = [+ /^<recommended_plugins>\r?\nHere is a list of plugins that are available but not installed\.[\s\S]*?\r?\n<\/recommended_plugins>$/,+ /^# AGENTS\.md instructions(?: for [^\r\n]+)?\r?\n\r?\n<INSTRUCTIONS>\r?\n[\s\S]*?\r?\n<\/INSTRUCTIONS>$/,+ /^<environment_context>\r?\n\s*<cwd>[^\r\n]*<\/cwd>\r?\n\s*<shell>[^\r\n]*<\/shell>\r?\n\s*<current_date>[^\r\n]*<\/current_date>\r?\n\s*<timezone>[^\r\n]*<\/timezone>[\s\S]*?<\/environment_context>$/,+];++function stripStartupBlocks(text) {+ const source = String(text);+ // Blocks can share an input_text part with a real prompt. A complete host+ // block starts on its own line and ends there or at end of text.+ const candidates = [+ /<recommended_plugins>\r?\nHere is a list of plugins that are available but not installed\.[\s\S]*?\r?\n<\/recommended_plugins>/g,+ /# AGENTS\.md instructions(?: for [^\r\n]+)?\r?\n\r?\n<INSTRUCTIONS>\r?\n[\s\S]*?\r?\n<\/INSTRUCTIONS>/g,+ /<environment_context>\r?\n\s*<cwd>[^\r\n]*<\/cwd>\r?\n\s*<shell>[^\r\n]*<\/shell>\r?\n\s*<current_date>[^\r\n]*<\/current_date>\r?\n\s*<timezone>[^\r\n]*<\/timezone>[\s\S]*?<\/environment_context>/g,+ ];+ const matches = [];+ for (let i = 0; i < candidates.length; i += 1) {+ for (const match of source.matchAll(candidates[i])) {+ const start = match.index;+ const end = start + match[0].length;+ if (start > 0 && !/\r?\n$/.test(source.slice(0, start))) continue;+ if (end < source.length && !/^\r?\n/.test(source.slice(end))) continue;+ if (STARTUP_BLOCKS[i].test(match[0])) matches.push({ start, end });+ }+ }+ if (!matches.length) return source;+ matches.sort((a, b) => a.start - b.start);+ let output = "";+ let cursor = 0;+ for (const { start, end } of matches) {+ if (start < cursor) continue;+ output += source.slice(cursor, start);+ cursor = end;+ }+ return (output + source.slice(cursor)).trim();+}++function withoutStartupContext(entries, cfg) {+ const filtered = [];+ const excludedLegacyTurnIndices = [];+ let beforeFirstTurn = true;+ for (const [index, entry] of entries.entries()) {+ if (entry?.type === "turn_context") beforeFirstTurn = false;+ const payload = entry?.payload;+ if (+ !beforeFirstTurn || entry?.type !== "response_item"+ || payload?.type !== "message" || payload.role !== "user"+ || !Array.isArray(payload.content)+ ) {+ filtered.push(entry);+ continue;+ }+ let changed = false;+ const content = payload.content.flatMap((part) => {+ if (part?.type !== "input_text" || typeof part.text !== "string") return [part];+ const text = stripStartupBlocks(part.text);+ if (text === part.text) return [part];+ changed = true;+ return text ? [{ ...part, text }] : [];+ });+ if (!changed) {+ filtered.push(entry);+ } else if (content.length) {+ filtered.push({ ...entry, payload: { ...payload, content } });+ } else {+ // Use the old extractor to locate this message in the old cursor's+ // coordinate system. Capture filters may already have dropped it.+ const prefix = extractSharedCaptureTurns(entries.slice(0, index), cfg).length;+ if (extractSharedCaptureTurns(entries.slice(0, index + 1), cfg).length > prefix) {+ excludedLegacyTurnIndices.push(prefix);+ }+ }+ }+ return { entries: filtered, excludedLegacyTurnIndices };+}++export function extractCaptureTranscript(rolloutEntries, cfg = {}) {const expanded = expandCodexRolloutEntries(rolloutEntries);const deduplicated = deduplicateCodexToolEvents(expanded);const normalized = normalizeCodexNativeToolEvents(deduplicated);- return extractSharedCaptureTurns(normalizeCodexCompletedToolEvents(normalized), cfg);+ const completed = normalizeCodexCompletedToolEvents(normalized);+ const { entries, excludedLegacyTurnIndices } = withoutStartupContext(completed, cfg);+ return { turns: extractSharedCaptureTurns(entries, cfg), excludedLegacyTurnIndices };+}++export function extractCaptureTurns(rolloutEntries, cfg = {}) {+ return extractCaptureTranscript(rolloutEntries, cfg).turns;}diff --git a/examples/codex-memory-plugin/scripts/capture-utils.test.mjs b/examples/codex-memory-plugin/scripts/capture-utils.test.mjsindex b95d6bbea..c6c0adb15 100644--- a/examples/codex-memory-plugin/scripts/capture-utils.test.mjs+++ b/examples/codex-memory-plugin/scripts/capture-utils.test.mjs@@ -1,7 +1,7 @@import assert from "node:assert/strict";import test from "node:test";-import { extractCaptureTurns } from "./capture-utils.mjs";+import { extractCaptureTranscript, extractCaptureTurns } from "./capture-utils.mjs";const CAPTURE_CONFIG = {captureAssistantTurns: true,@@ -9,6 +9,77 @@ const CAPTURE_CONFIG = {captureMaxLength: 24000,};+const startupBlocks = [+ "<recommended_plugins>\nHere is a list of plugins that are available but not installed.\n\n- Example (example@marketplace)\n</recommended_plugins>",+ "# AGENTS.md instructions\n\n<INSTRUCTIONS>\nUse Chinese for answers.\n</INSTRUCTIONS>",+ "<environment_context>\n <cwd>/tmp/project</cwd>\n <shell>zsh</shell>\n <current_date>2026-09-25</current_date>\n <timezone>Asia/Singapore</timezone>\n <filesystem><permission_profile type=\"disabled\" /></filesystem>\n</environment_context>",+];++function userEntry(...texts) {+ return {+ type: "response_item",+ payload: { type: "message", role: "user", content: texts.map((text) => ({ type: "input_text", text })) },+ };+}++test("excludes complete Codex startup blocks and records their old turn positions", () => {+ const { turns, excludedLegacyTurnIndices } = extractCaptureTranscript([+ { type: "session_meta", payload: {} },+ userEntry(...startupBlocks),+ { type: "turn_context", payload: {} },+ userEntry("How does capture work?"),+ ], CAPTURE_CONFIG);+ assert.deepEqual(excludedLegacyTurnIndices, [0]);+ assert.deepEqual(turns.map((turn) => turn.text), ["How does capture work?"]);+});++test("records separate excluded startup messages at their old indices", () => {+ const { turns, excludedLegacyTurnIndices } = extractCaptureTranscript([+ userEntry(startupBlocks[0]),+ userEntry(startupBlocks[1], startupBlocks[2]),+ { type: "turn_context", payload: {} },+ userEntry("Real question"),+ ], CAPTURE_CONFIG);+ assert.deepEqual(excludedLegacyTurnIndices, [0, 1]);+ assert.deepEqual(turns.map((turn) => turn.text), ["Real question"]);+});++test("excludes the legacy path-qualified AGENTS.md startup block", () => {+ const legacyAgents = "# AGENTS.md instructions for /tmp/project\n\n<INSTRUCTIONS>\nUse Chinese for answers.\n</INSTRUCTIONS>";+ const { turns, excludedLegacyTurnIndices } = extractCaptureTranscript([+ userEntry(legacyAgents),+ { type: "turn_context", payload: {} },+ userEntry("Real question"),+ ], CAPTURE_CONFIG);+ assert.deepEqual(excludedLegacyTurnIndices, [0]);+ assert.deepEqual(turns.map((turn) => turn.text), ["Real question"]);+});++test("preserves user text mixed into a startup message", () => {+ const { turns, excludedLegacyTurnIndices } = extractCaptureTranscript([+ userEntry(`${startupBlocks[0]}\nPlease explain this plugin list.`, startupBlocks[1], startupBlocks[2]),+ { type: "turn_context", payload: {} },+ userEntry("Why does the rollout contain AGENTS.md instructions and <environment_context>?"),+ ], CAPTURE_CONFIG);+ assert.deepEqual(excludedLegacyTurnIndices, []);+ assert.deepEqual(turns.map((turn) => turn.text), [+ "Please explain this plugin list.",+ "Why does the rollout contain AGENTS.md instructions and <environment_context>?",+ ]);+});++test("keeps incomplete wrappers and normal-turn quotations", () => {+ const quoted = `${startupBlocks[0]}\n\n${startupBlocks[1]}`;+ const { turns, excludedLegacyTurnIndices } = extractCaptureTranscript([+ userEntry("<recommended_plugins>\nmissing closing tag"),+ { type: "turn_context", payload: {} },+ userEntry(quoted),+ ], CAPTURE_CONFIG);+ assert.deepEqual(excludedLegacyTurnIndices, []);+ assert.equal(turns.length, 2);+ assert.match(turns[1].text, /AGENTS\.md instructions/);+});+function toolParts(turns, toolName) {return turns.flatMap((turn) => turn.parts)diff --git a/examples/codex-memory-plugin/scripts/ov-session.mjs b/examples/codex-memory-plugin/scripts/ov-session.mjsindex 7872157b9..e762c9448 100644--- a/examples/codex-memory-plugin/scripts/ov-session.mjs+++ b/examples/codex-memory-plugin/scripts/ov-session.mjs@@ -11,8 +11,8 @@*/import { readFile } from "node:fs/promises";-import { extractCaptureTurns, findLastHumanTurnIndex } from "./capture-utils.mjs";-import { resolveOvSessionId, saveState } from "./session-state.mjs";+import { extractCaptureTranscript, findLastHumanTurnIndex } from "./capture-utils.mjs";+import { CAPTURE_FORMAT_VERSION, resolveOvSessionId, saveState } from "./session-state.mjs";import { makeAgentFetchJSON } from "./shared/agent-hook-runtime.mjs";import { sendSessionMessages } from "./shared/batch-send.mjs";@@ -66,14 +66,14 @@ function parseTranscript(content) {* cursor on a transient read error replays the whole session.*/export async function readTranscriptTurns(transcriptPath, cfg, logError) {- if (!transcriptPath) return { turns: [], ok: false };+ if (!transcriptPath) return { turns: [], excludedLegacyTurnIndices: [], ok: false };try {const raw = await readFile(transcriptPath, "utf-8");- if (!raw.trim()) return { turns: [], ok: true };- return { turns: extractCaptureTurns(parseTranscript(raw), cfg), ok: true };+ if (!raw.trim()) return { turns: [], excludedLegacyTurnIndices: [], ok: true };+ return { ...extractCaptureTranscript(parseTranscript(raw), cfg), ok: true };} catch (err) {logError?.("transcript_read", err);- return { turns: [], ok: false };+ return { turns: [], excludedLegacyTurnIndices: [], ok: false };}}@@ -113,7 +113,21 @@ export async function catchUpTurns({// a session whose Stop/SessionEnd workers never ran.if (transcriptPath) state.transcriptPath = transcriptPath;- const { turns, ok } = await readTranscriptTurns(transcriptPath, cfg, logError);+ const { turns, excludedLegacyTurnIndices, ok } = await readTranscriptTurns(transcriptPath, cfg, logError);++ if (ok && state.captureFormatVersion !== CAPTURE_FORMAT_VERSION) {+ const oldCursor = Math.max(0, Number(state.capturedTurnCount) || 0);+ state.capturedTurnCount = oldCursor - excludedLegacyTurnIndices.filter((index) => index < oldCursor).length;+ state.captureFormatVersion = CAPTURE_FORMAT_VERSION;+ // The caller holds the session lock. Persist the corrected cursor before+ // any append; a failed send must retry from this new coordinate system.+ await saveState(state, { touch: false });+ log?.("capture_format_migrated", {+ oldCursor,+ correctedCursor: state.capturedTurnCount,+ excludedLegacyTurnIndices,+ });+ }if (!ok || turns.length === 0) {log?.("transcript_empty", {diff --git a/examples/codex-memory-plugin/scripts/pre-compact-capture.test.mjs b/examples/codex-memory-plugin/scripts/pre-compact-capture.test.mjsindex a3eb32584..72cbabe69 100644--- a/examples/codex-memory-plugin/scripts/pre-compact-capture.test.mjs+++ b/examples/codex-memory-plugin/scripts/pre-compact-capture.test.mjs@@ -145,6 +145,34 @@ test("pre-compact catches up, commits and keeps the cursor", async () => {}});+test("pre-compact does not create or commit a startup-only session", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-pre-compact-startup-"));+ const transcriptPath = join(stateDir, "transcript.jsonl");+ const calls = [];+ try {+ await writeFile(transcriptPath, [+ JSON.stringify({ type: "response_item", payload: {+ type: "message", role: "user", content: [{ type: "input_text",+ text: "# AGENTS.md instructions\n\n<INSTRUCTIONS>\nUse Chinese.\n</INSTRUCTIONS>" }],+ } }),+ JSON.stringify({ type: "turn_context", payload: {} }),+ ].join("\n"));+ await withMockOpenViking(mockHandler(calls), async (baseUrl) => {+ await runPreCompact(+ { session_id: "startup-only", transcript_path: transcriptPath, trigger: "manual" },+ baseEnv(baseUrl, stateDir),+ );+ });+ assert.equal(calls.some((call) => call.path.endsWith("/messages/batch")), false);+ assert.equal(calls.some((call) => call.path.endsWith("/commit")), false);+ const state = JSON.parse(await readFile(join(stateDir, "startup-only.json"), "utf-8"));+ assert.equal(state.ovSessionId, null);+ assert.equal(state.capturedTurnCount, 0);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});+test("pre-compact leaves state untouched when the session lock is held", async () => {const stateDir = await mkdtemp(join(tmpdir(), "ov-pre-compact-lock-"));const transcriptPath = join(stateDir, "transcript.jsonl");diff --git a/examples/codex-memory-plugin/scripts/session-end.test.mjs b/examples/codex-memory-plugin/scripts/session-end.test.mjsindex b3f3775ba..b2cd28e87 100644--- a/examples/codex-memory-plugin/scripts/session-end.test.mjs+++ b/examples/codex-memory-plugin/scripts/session-end.test.mjs@@ -168,6 +168,41 @@ test("session-end catches up the missing turns then commits", async () => {}});+test("session-end migrates a startup-inclusive cursor before catch-up and commit", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-session-end-startup-"));+ const transcriptPath = join(stateDir, "transcript.jsonl");+ const calls = [];+ const startup = {+ type: "response_item",+ payload: { type: "message", role: "user", content: [+ { type: "input_text", text: "# AGENTS.md instructions\n\n<INSTRUCTIONS>\nAnswer in Chinese.\n</INSTRUCTIONS>" },+ { type: "input_text", text: "<environment_context>\n <cwd>/tmp/project</cwd>\n <shell>zsh</shell>\n <current_date>2026-09-25</current_date>\n <timezone>Asia/Singapore</timezone>\n</environment_context>" },+ ] },+ };+ try {+ await writeState(stateDir, "startup-end", { capturedTurnCount: 2 });+ await writeFile(transcriptPath, [+ JSON.stringify(startup),+ JSON.stringify({ type: "turn_context", payload: { turn_id: "first" } }),+ turn("user", "Previously captured"),+ turn("user", "New question"),+ ].join("\n"));+ await withMockOpenViking(mockHandler(calls), async (baseUrl) => {+ await runSessionEnd(+ { session_id: "startup-end", transcript_path: transcriptPath },+ workerEnv(baseUrl, stateDir),+ );+ });+ assert.deepEqual(sentMessages(calls).map((item) => item.parts?.[0]?.text ?? item.content), ["New question"]);+ assert.equal(calls.filter((call) => call.path.endsWith("/commit")).length, 1);+ const state = await readState(stateDir, "startup-end");+ assert.equal(state.captureFormatVersion, 2);+ assert.equal(state.capturedTurnCount, 2);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});+test("a second session-end on an unchanged transcript neither sends nor commits", async () => {const stateDir = await mkdtemp(join(tmpdir(), "ov-session-end-idem-"));const transcriptPath = join(stateDir, "transcript.jsonl");diff --git a/examples/codex-memory-plugin/scripts/session-state.mjs b/examples/codex-memory-plugin/scripts/session-state.mjsindex 38da75456..24678ed90 100644--- a/examples/codex-memory-plugin/scripts/session-state.mjs+++ b/examples/codex-memory-plugin/scripts/session-state.mjs@@ -115,6 +115,7 @@ function lockPath(codexSessionId) {// outlive that rule; bump this whenever the derivation changes. The rest of the// state (ovSessionId, capture progress) stays valid across a bump and is kept.export const PEER_PIN_VERSION = 3;+export const CAPTURE_FORMAT_VERSION = 2;function defaultState(codexSessionId) {const now = Date.now();@@ -127,6 +128,7 @@ function defaultState(codexSessionId) {// catch up turns for a session whose own workers never ran.transcriptPath: null,capturedTurnCount: 0,+ captureFormatVersion: CAPTURE_FORMAT_VERSION,createdAt: now,lastUpdatedAt: now,};@@ -137,6 +139,9 @@ export async function loadState(codexSessionId) {const raw = await readFile(statePath(codexSessionId), "utf-8");const parsed = JSON.parse(raw);const state = { ...defaultState(codexSessionId), ...parsed };+ // A missing marker means capturedTurnCount still counts Codex's startup+ // context as a turn. Migration needs the rollout and happens under lock.+ if (parsed?.captureFormatVersion == null) state.captureFormatVersion = 1;if (parsed?.peerPinVersion !== PEER_PIN_VERSION) {state.workspacePeerId = "";state.peerPinVersion = PEER_PIN_VERSION;diff --git a/examples/codex-memory-plugin/scripts/startup-capture.test.mjs b/examples/codex-memory-plugin/scripts/startup-capture.test.mjsnew file mode 100644index 000000000..edbea8977--- /dev/null+++ b/examples/codex-memory-plugin/scripts/startup-capture.test.mjs@@ -0,0 +1,145 @@+import assert from "node:assert/strict";+import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";+import { tmpdir } from "node:os";+import { join } from "node:path";+import test from "node:test";++const stateDir = await mkdtemp(join(tmpdir(), "ov-codex-startup-"));+process.env.OPENVIKING_CODEX_STATE_DIR = stateDir;++const { catchUpTurns } = await import("./ov-session.mjs");+const { CAPTURE_FORMAT_VERSION, loadState, saveState, withSessionLock } = await import("./session-state.mjs");++const cfg = { captureAssistantTurns: true, captureMaxLength: 24000 };+const startup = [+ "<recommended_plugins>\nHere is a list of plugins that are available but not installed.\n\n- Example (example@marketplace)\n</recommended_plugins>",+ "# AGENTS.md instructions\n\n<INSTRUCTIONS>\nAnswer in Chinese.\n</INSTRUCTIONS>",+ "<environment_context>\n <cwd>/tmp/project</cwd>\n <shell>zsh</shell>\n <current_date>2026-09-25</current_date>\n <timezone>Asia/Singapore</timezone>\n <filesystem><permission_profile type=\"disabled\" /></filesystem>\n</environment_context>",+];++function message(role, ...texts) {+ return { type: "response_item", payload: {+ type: "message", role,+ content: texts.map((text) => ({ type: role === "user" ? "input_text" : "output_text", text })),+ } };+}++function rollout(...conversation) {+ return [+ { type: "session_meta", payload: { id: "example" } },+ message("user", ...startup),+ { type: "turn_context", payload: { turn_id: "turn-1" } },+ ...conversation,+ ];+}++async function writeRollout(id, entries) {+ const path = join(stateDir, `${id}.jsonl`);+ await writeFile(path, entries.map((entry) => JSON.stringify(entry)).join("\n"));+ return path;+}++async function catchUp(id, transcriptPath, sent, { oldCursor, fail = false } = {}) {+ const outcome = await withSessionLock(id, async () => {+ const state = await loadState(id);+ if (oldCursor !== undefined) {+ state.capturedTurnCount = oldCursor;+ state.captureFormatVersion = 1;+ await saveState(state);+ }+ const result = await catchUpTurns({+ state, transcriptPath, cfg,+ fetchJSONRes: async (_path, init) => {+ const persisted = await readFile(join(stateDir, `${id}.json`), "utf-8").then(JSON.parse, () => null);+ if (oldCursor !== undefined) {+ assert.equal(persisted.captureFormatVersion, CAPTURE_FORMAT_VERSION,+ "migration must be durable before the first send");+ }+ if (fail) return { ok: false, status: 500 };+ sent.push(...JSON.parse(init.body).messages);+ return { ok: true, result: {} };+ },+ });+ return { result, state };+ });+ return outcome.value;+}++test("startup-only rollout sends nothing and never opens an OV session", async () => {+ const id = "startup-only";+ const sent = [];+ const path = await writeRollout(id, rollout());+ const { result, state } = await catchUp(id, path, sent);+ assert.equal(result.added, 0);+ assert.deepEqual(sent, []);+ assert.equal(state.ovSessionId, null);+ assert.equal(state.capturedTurnCount, 0);+});++test("first real prompt is sent and resumed capture does not replay it", async () => {+ const id = "fresh";+ const sent = [];+ const path = await writeRollout(id, rollout(message("user", "First question"), message("assistant", "First answer")));+ const first = await catchUp(id, path, sent);+ assert.equal(first.result.added, 2);+ assert.deepEqual(sent.map((item) => item.parts[0].text), ["First question", "First answer"]);+ const second = await catchUp(id, path, sent);+ assert.equal(second.result.added, 0);+ assert.equal(sent.length, 2);+ await writeRollout(id, rollout(+ message("user", "First question"), message("assistant", "First answer"),+ message("user", "Quoted <environment_context> in a real prompt"),+ ));+ const resumed = await catchUp(id, path, sent);+ assert.equal(resumed.result.added, 1);+ assert.equal(sent.at(-1).parts[0].text, "Quoted <environment_context> in a real prompt");+});++test("legacy cursor is corrected by excluded old positions before sending", async () => {+ const id = "legacy";+ const sent = [];+ const path = await writeRollout(id, rollout(+ message("user", "Already sent"), message("assistant", "Also sent"),+ message("user", "New question"),+ ));+ const { result, state } = await catchUp(id, path, sent, { oldCursor: 3 });+ assert.equal(result.added, 1);+ assert.deepEqual(sent.map((item) => item.parts[0].text), ["New question"]);+ assert.equal(state.capturedTurnCount, 3);+ assert.equal((await catchUp(id, path, sent)).result.added, 0);+});++test("failed send keeps the corrected legacy cursor for retry", async () => {+ const id = "failure";+ const sent = [];+ const path = await writeRollout(id, rollout(message("user", "Retry me")));+ const failed = await catchUp(id, path, sent, { oldCursor: 1, fail: true });+ assert.equal(failed.result.added, 0);+ assert.equal((await loadState(id)).capturedTurnCount, 0);+ assert.equal((await catchUp(id, path, sent)).result.added, 1);+ assert.equal(sent[0].parts[0].text, "Retry me");+});++test("legacy startup-only cursor migrates without an empty batch", async () => {+ const id = "legacy-startup-only";+ const sent = [];+ const path = await writeRollout(id, rollout());+ const { result, state } = await catchUp(id, path, sent, { oldCursor: 1 });+ assert.equal(result.added, 0);+ assert.equal(state.capturedTurnCount, 0);+ assert.deepEqual(sent, []);+});++test("shortened transcript resumes at the latest human turn", async () => {+ const id = "shortened";+ const sent = [];+ const path = await writeRollout(id, rollout(+ message("user", "Current question"), message("assistant", "Current answer"),+ ));+ const { result, state } = await catchUp(id, path, sent, { oldCursor: 9 });+ assert.equal(result.added, 2);+ assert.deepEqual(sent.map((item) => item.parts[0].text), ["Current question", "Current answer"]);+ assert.equal(state.capturedTurnCount, 2);+});++test.after(async () => { await rm(stateDir, { recursive: true, force: true }); });diff --git a/examples/codex-memory-plugin/skills/ov-memory-doctor/reference.md b/examples/codex-memory-plugin/skills/ov-memory-doctor/reference.mdindex 1ce749321..d5e30e632 100644--- a/examples/codex-memory-plugin/skills/ov-memory-doctor/reference.md+++ b/examples/codex-memory-plugin/skills/ov-memory-doctor/reference.md@@ -17,7 +17,7 @@ and `CODEX_CONFIG_FILE` relocate individual pieces.| `~/.codex/config.toml` | `[features] hooks` (or legacy `plugin_hooks`), `[plugins."openviking-memory@openviking"] enabled`, `[marketplaces.openviking]` (source, ref), `[hooks.state."openviking-memory@openviking:hooks/hooks.json:<event>:0:0"] trusted_hash` for `session_start`, `user_prompt_submit`, `pre_tool_use`, `stop`, `session_end`, `pre_compact`. || `~/.codex/plugins/cache/openviking/openviking-memory/<version>/` | The copy Codex runs hooks from. Keyed by `plugin.json` version. || `~/.codex/.tmp/marketplaces/openviking/` | Git clone of the marketplace (GitHub/TOS dist); `examples/codex-memory-plugin` inside it is what `codex plugin list` reports as the source path. |-| `~/.openviking/codex-plugin-state/<session_id>.json` | Per-session state: `ovSessionId` (`cx-<session_id>`, null once committed), `transcriptPath` (last rollout seen, used by the SessionStart sweep to catch up unsent turns), `capturedTurnCount`, `lastUpdatedAt`. |+| `~/.openviking/codex-plugin-state/<session_id>.json` | Per-session state: `ovSessionId` (`cx-<session_id>`, null once committed), `transcriptPath` (last rollout seen, used by the SessionStart sweep to catch up unsent turns), `capturedTurnCount`, `captureFormatVersion` (2; absent means the old cursor format and is migrated on the next readable rollout), `lastUpdatedAt`. || `~/.openviking/codex-plugin-state/<session_id>.ended.<timestamp>` / `.lock` | Sidecars: `.ended.<timestamp>` marks a thread whose SessionEnd fired but whose commit has not succeeded yet (the timestamp is in the filename so a conditional removal cannot delete a newer exit's marker; a bare `.ended` is a pre-0.8.1 leftover); `.lock` is the directory lock serializing the capture hooks. || `~/.openviking/codex-plugin-state/recall-compressor-profile.json` | Cached local-compressor detection (`profile.enabled`, `model`, `source`). || `~/.openviking/logs/codex-hooks.log` | JSONL hook + proxy log; written only when `OPENVIKING_DEBUG=1` or `codex.debug: true`. |