Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.
Python agent-contextagent-memoryagent-pluginsagentic-ragcontext-databasedsh-pluginknowledge-basememorymemory-managementself-evolvingvector-database
fix(codex): preserve transcript cursors across commit and compaction (#4191)
t0sakia61c57bf5979
diff --git a/docs/en/agent-integrations/04-codex.md b/docs/en/agent-integrations/04-codex.mdindex 7737e04ec..8388e72dc 100644--- a/docs/en/agent-integrations/04-codex.md+++ b/docs/en/agent-integrations/04-codex.md@@ -78,6 +78,7 @@ Credential source: env vars win by default — when any `OPENVIKING_*` credentia| `OPENVIKING_PROFILE_TOKEN_BUDGET` | `10000` | CJK-aware token budget for `profile.md` plus `preferences/` and `entities/` indexes || `OPENVIKING_CODEX_ACTIVE_WINDOW_MS` | `120000` | SessionStart active-window threshold || `OPENVIKING_CODEX_IDLE_TTL_MS` | `1800000` | SessionStart idle-TTL sweep threshold |+| `OPENVIKING_CODEX_COMMITTED_TTL_MS` | `2592000000` | How long a committed session's transcript cursor is kept before its state file is retired || `OPENVIKING_DEBUG` | `false` | Write logs to `~/.openviking/logs/codex-hooks.log` |If recall latency matters most, see [Low-latency recall](./01-overview.md#low-latency-recall) for the environment-variable and `ovcli.conf` settings that disable query expansion and Codex's local result compression.diff --git a/docs/zh/agent-integrations/04-codex.md b/docs/zh/agent-integrations/04-codex.mdindex b0dbc1c20..a5e07f886 100644--- a/docs/zh/agent-integrations/04-codex.md+++ b/docs/zh/agent-integrations/04-codex.md@@ -77,6 +77,7 @@ TraeCode CLI 2.0 用户启动 `trae-cli`,并可用 `trae-cli plugin list` 确| `OPENVIKING_PROFILE_TOKEN_BUDGET` | `10000` | `profile.md` 及 `preferences/`、`entities/` 索引共用的 CJK-aware token 预算 || `OPENVIKING_CODEX_ACTIVE_WINDOW_MS` | `120000` | `SessionStart` 活动窗口阈值(毫秒) || `OPENVIKING_CODEX_IDLE_TTL_MS` | `1800000` | `SessionStart` 闲置 TTL 清理阈值(毫秒) |+| `OPENVIKING_CODEX_COMMITTED_TTL_MS` | `2592000000` | 已提交会话的转录游标保留时长(毫秒),过期后删除状态文件 || `OPENVIKING_DEBUG` | `false` | 是否将日志写入 `~/.openviking/logs/codex-hooks.log` |如果更看重召回响应速度,请参阅[低延迟召回](./01-overview.md#低延迟召回),其中说明了如何通过环境变量或 `ovcli.conf` 关闭查询扩展与 Codex 本地结果压缩。diff --git a/examples/codex-memory-plugin/.codex-plugin/plugin.json b/examples/codex-memory-plugin/.codex-plugin/plugin.jsonindex 2481cdd6b..ac62b72e4 100644--- a/examples/codex-memory-plugin/.codex-plugin/plugin.json+++ b/examples/codex-memory-plugin/.codex-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "openviking-memory",- "version": "0.7.5",+ "version": "0.7.6","description": "Long-term semantic memory for Codex, powered by OpenViking. Recall on UserPromptSubmit, incremental add_message on Stop (per turn), commit on PreCompact, and active-window heuristic + idle-TTL sweep on SessionStart (source=startup|clear). Includes a local stdio MCP proxy that reads OpenViking credentials from env/ovcli.conf.","author": {"name": "OpenViking",diff --git a/examples/codex-memory-plugin/DESIGN.md b/examples/codex-memory-plugin/DESIGN.mdindex 19254cb19..501df7a08 100644--- a/examples/codex-memory-plugin/DESIGN.md+++ b/examples/codex-memory-plugin/DESIGN.md@@ -90,6 +90,12 @@ fired and bumped `lastUpdatedAt`; we commit it. The multi-recent caseimplies concurrent codex sessions are active; we can't tell which one (ifany) ended, so we defer to idle TTL to clean up genuinely-dead ones.+The count is over *activity*, including cursor-only states: a session+`PreCompact` just committed has no live `ovSessionId` but may well still be+running, and counting it keeps the `≥2` branch from sealing a sibling+session mid-flight. Only a state that still has a live `ovSessionId` is+actually committed; the single-recent case with nothing live is a no-op.+### 4. `SessionStart` source=`resume` — never commits, optional archive injectShort reconnects and `/resume` re-fire `SessionStart` for the same@@ -120,11 +126,16 @@ transcript.### 5. Idle TTL sweep — fallback-State files whose `lastUpdatedAt` is older than `IDLE_TTL_MS` (default 30-min) get committed and cleared. Mental model: a session not touched for-30 min is "temporarily concluded"; if the user resumes later, subsequent-turns append under the same deterministic OV session id, and the next-commit creates another archive there.+State files with a live `ovSessionId` whose `lastUpdatedAt` is older than+`IDLE_TTL_MS` (default 30 min) get committed. Their transcript cursor is+preserved while `ovSessionId` is cleared. Mental model: a session not touched+for 30 min is "temporarily concluded"; if the user resumes later, subsequent+turns append under the same deterministic OV session id, and the next commit+creates another archive there.++Releasing the session id instead of deleting the file writes state without+touching `lastUpdatedAt`, so a committed session doesn't look freshly active+to the next `SessionStart`.This covers:- SIGTERM / Ctrl+C / `/exit` (no hook fires; state file rots)@@ -145,6 +156,21 @@ machine, no sweep ever runs and the OV session stays open server-sideforever. Accepted. Future work could add an MCP tool`openviking_commit_pending` so the model can commit explicitly.+### 6. Cursor retention — same pass++A committed state file keeps living as a cursor (`ovSessionId: null`) so a+later resume appends instead of replaying. Kept forever that would leak one+file per codex session and make `listStates()` — which reads every file on+every `SessionStart` — slower over time, so the same pass retires them:++| State | Retired after |+|---|---|+| cursor-only, `capturedTurnCount > 0` | `COMMITTED_TTL_MS` (default 30 days) |+| cursor-only, nothing ever captured | `IDLE_TTL_MS` (default 30 min) |++A cursor that outlives its codex rollout has nothing left to resume from, and+a state file that never captured a turn is identical to no state file at all.+## Stop hook — append + threshold commitEvery `Stop` reads `transcript_path`, slices to `[capturedTurnCount, end)`,@@ -194,9 +220,19 @@ compatibility fallbacks.Codex's `/compact` may rewrite or truncate `transcript_path`. Aftercompaction, if `allTurns.length < state.capturedTurnCount`, our slice-math underflows and we silently drop new turns. Defensive fix: when this-inequality is detected on `Stop`, reset `capturedTurnCount = 0` so the-next slice captures everything in the new transcript.+math underflows and we silently drop new turns. When this inequality is+detected on `Stop`, move `capturedTurnCount` to the latest human turn+so the current interaction is captured without replaying compacted history.++"Human turn" is not just `role === "user"` — `normalizeCaptureRole()` maps+tool results onto the user role as well, so `findLastHumanTurnIndex()`+additionally requires a `text` part. If the rewrite left no human turn at+all (index `-1`), we fall back to capturing the whole transcript and log+`fallback: "full_transcript"`; replaying beats losing the interaction.++**Trade-off**: turns older than that human turn are assumed captured. If+earlier `Stop` hooks failed to reach OV and compaction happened before they+were retried, those turns are dropped rather than duplicated.### Commit failure@@ -226,7 +262,7 @@ OV session id, while commits create additional archives under that session.```json{"codexSessionId": "0193af...", // codex thread id- "ovSessionId": "cx-0193af...-or-null", // null means "committed, awaiting next Stop"+ "ovSessionId": "cx-0193af...-or-null", // null means "committed, awaiting next Stop or retirement""capturedTurnCount": 7, // turns from transcript already appended"createdAt": 1715000000000,"lastUpdatedAt": 1715000300000@@ -249,6 +285,7 @@ Env var overrides for tuning without rebuilding:| `OPENVIKING_CODEX_STATE_DIR` | `~/.openviking/codex-plugin-state` | state file dir || `OPENVIKING_CODEX_ACTIVE_WINDOW_MS` | `120000` (2 min) | rule-3 active window || `OPENVIKING_CODEX_IDLE_TTL_MS` | `1800000` (30 min) | idle sweep TTL |+| `OPENVIKING_CODEX_COMMITTED_TTL_MS` | `2592000000` (30 days) | how long a committed cursor is kept for resume || `OPENVIKING_RECALL_TIMEOUT_MS` | `120000` (2 min) | whole UserPromptSubmit auto-recall deadline || `OPENVIKING_RECALL_COMPRESS` | `1` | set `0` / `off` to skip `codex exec` compression || `OPENVIKING_RECALL_COMPRESS_MODEL` | unset | custom first-choice compressor model; `off` disables compression |diff --git a/examples/codex-memory-plugin/README.md b/examples/codex-memory-plugin/README.mdindex 88670017b..db2dd066d 100644--- a/examples/codex-memory-plugin/README.md+++ b/examples/codex-memory-plugin/README.md@@ -198,11 +198,12 @@ On `startup` or `clear`, the script:1. Counts state files (excluding the new session_id) whose `lastUpdatedAt` is within `OPENVIKING_CODEX_ACTIVE_WINDOW_MS` (default 2 min) of "now":- **0 active** → no-op (no orphan to commit)- - **1 active** → commit it (the just-ended session)+ - **1 active** → commit it (the just-ended session), unless it has no live `ovSessionId` left to commit- **≥2 active** → skip; rely on idle TTL (we can't tell which one ended)-2. **Idle-TTL sweep at the tail**: any state file (regardless of session_id) older than `OPENVIKING_CODEX_IDLE_TTL_MS` (default 30 min) gets committed and cleared.+2. **Idle-TTL sweep at the tail**: any live session state older than `OPENVIKING_CODEX_IDLE_TTL_MS` (default 30 min) gets committed while preserving its transcript cursor for resume.+3. **Cursor retention in the same pass**: a state file with no live OV session is kept as a resume cursor until `OPENVIKING_CODEX_COMMITTED_TTL_MS` (default 30 days), or dropped after the idle TTL if it never captured a turn.-On any /commit failure (OV unreachable, non-2xx, timeout) we **preserve state** (don't `clearState`) so the next sweep can retry.+On any /commit failure (OV unreachable, non-2xx, timeout) we **preserve state** (keep `ovSessionId` set) so the next sweep can retry.On `resume`, the script skips commit/sweep. It still injects the profile block. If local state has no live `ovSessionId`, it also reads `/api/v1/sessions/{cx-session-id}/context` and combines the latest committed archive overview into the same `SessionStart` output. The archive block includes a `viking://~/sessions/{cx-session-id}/history/` URI and tells the model to use the OpenViking MCP `read`/`search` tools for exact prior commands, file paths, tool outputs, or messages. Set `OPENVIKING_RESUME_ARCHIVE_INJECT=0` to disable the archive half without disabling profile injection.diff --git a/examples/codex-memory-plugin/VERIFICATION.md b/examples/codex-memory-plugin/VERIFICATION.mdindex 22f3cccd5..79ff12ac4 100644--- a/examples/codex-memory-plugin/VERIFICATION.md+++ b/examples/codex-memory-plugin/VERIFICATION.md@@ -196,7 +196,7 @@ files are still present — the skip path does not clear them.# Backdate one of the state files to be older than IDLE_TTL_MS (default 30 min).OLD=$(node -e 'console.log(Date.now() - 60*60*1000)') # 1 hour agocat > "$STATE_DIR/state/sess-aaa.json" <<EOF-{"codexSessionId":"sess-aaa","ovSessionId":null,"capturedTurnCount":0,"createdAt":$OLD,"lastUpdatedAt":$OLD}+{"codexSessionId":"sess-aaa","ovSessionId":"cx-sess-aaa","capturedTurnCount":2,"createdAt":$OLD,"lastUpdatedAt":$OLD}EOFecho '{"session_id":"sess-ddd","source":"startup","cwd":"/tmp","model":"x","permission_mode":"default","transcript_path":null,"hook_event_name":"SessionStart"}' \@@ -206,12 +206,31 @@ echo '{"session_id":"sess-ddd","source":"startup","cwd":"/tmp","model":"x","permnode $PLUGIN/scripts/session-start-commit.mjs```-Expect: log shows `idle_sweep` for `sess-aaa` (committed and cleared).-`sess-bbb.json` is still present (still fresh). `sess-aaa.json` is gone.+Expect: log shows `idle_sweep` for `sess-aaa` (committed).+`sess-bbb.json` is still present (still fresh). `sess-aaa.json` is also+present with `ovSessionId: null` and `capturedTurnCount: 2` for resume.If `sess-bbb` was in `≥2 active` from 6c, the heuristic on this call seesjust `sess-bbb` (1 active) and commits it — that's expected and shows theheuristic + sweep working together.+### 6d-2. Cursor retention++```bash+# sess-aaa is now cursor-only (ovSessionId: null, capturedTurnCount: 2).+# Re-run the same SessionStart with a 1 s committed TTL.+echo '{"session_id":"sess-eee","source":"startup","cwd":"/tmp","model":"x","permission_mode":"default","transcript_path":null,"hook_event_name":"SessionStart"}' \+ | OPENVIKING_CONFIG_FILE=$OV_CONF \+ OPENVIKING_CODEX_STATE_DIR=$STATE_DIR/state \+ OPENVIKING_CODEX_COMMITTED_TTL_MS=1000 \+ CODEX_PLUGIN_ROOT=$PLUGIN \+ OPENVIKING_DEBUG=1 \+ node $PLUGIN/scripts/session-start-commit.mjs+```++Expect: log shows `state_retire` for `sess-aaa` and the file is gone. With+the default 30-day TTL it stays, and no `/commit` is issued for it either+way — a cursor-only state has nothing left to commit.+### 6e. `source=resume` → no commit/sweep; optional archive inject```bashdiff --git a/examples/codex-memory-plugin/scripts/auto-capture.mjs b/examples/codex-memory-plugin/scripts/auto-capture.mjsindex 5dcd5a7cf..c1b3b7f16 100644--- a/examples/codex-memory-plugin/scripts/auto-capture.mjs+++ b/examples/codex-memory-plugin/scripts/auto-capture.mjs@@ -28,6 +28,7 @@import { readFile } from "node:fs/promises";import {extractCaptureTurns,+ findLastHumanTurnIndex,} from "./capture-utils.mjs";import { loadConfig } from "./config.mjs";import { createLogger } from "./debug-log.mjs";@@ -226,15 +227,20 @@ async function main() {// Post-compact transcript-shrink defense: codex's /compact may rewrite or// truncate transcript_path. If allTurns has fewer entries than we cached,- // our slice math would underflow and silently drop turns. Reset the- // counter so the next slice captures everything in the new transcript.- // See DESIGN.md "Post-compact transcript shrink".+ // our slice math would underflow and silently drop turns. Resume at the+ // latest human turn so the current interaction is captured without replaying+ // compacted history. See DESIGN.md "Post-compact transcript shrink".if (allTurns.length < state.capturedTurnCount) {+ const humanIndex = findLastHumanTurnIndex(allTurns);log("transcript_shrink_detected", {cached: state.capturedTurnCount,observed: allTurns.length,+ resumeFrom: Math.max(0, humanIndex),+ // -1 means no human turn survived the rewrite; capturing the whole+ // transcript is the only way not to lose the interaction.+ fallback: humanIndex < 0 ? "full_transcript" : undefined,});- state.capturedTurnCount = 0;+ state.capturedTurnCount = Math.max(0, humanIndex);}const newTurns = allTurns.slice(state.capturedTurnCount);diff --git a/examples/codex-memory-plugin/scripts/auto-capture.test.mjs b/examples/codex-memory-plugin/scripts/auto-capture.test.mjsindex 0e1ac0de4..8aee2c283 100644--- a/examples/codex-memory-plugin/scripts/auto-capture.test.mjs+++ b/examples/codex-memory-plugin/scripts/auto-capture.test.mjs@@ -385,3 +385,81 @@ test("auto-capture logs a commit error trace_id", async () => {await rm(stateDir, { recursive: true, force: true });}});++test("auto-capture skips compacted history after transcript shrink", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-auto-capture-compact-"));+ const transcriptPath = join(stateDir, "transcript.jsonl");+ const batches = [];+ const now = Date.now();++ try {+ await writeFile(join(stateDir, "compaction.json"), JSON.stringify({+ codexSessionId: "compaction",+ ovSessionId: "cx-compaction",+ capturedTurnCount: 8,+ createdAt: now - 1000,+ lastUpdatedAt: now,+ }));+ await writeFile(+ transcriptPath,+ [+ { payload: { message: { role: "user", content: "compacted historical summary" } } },+ { payload: { message: { role: "assistant", content: "prior assistant tail" } } },+ { payload: { message: { role: "user", content: "current user request" } } },+ { payload: { type: "function_call", id: "call-1", name: "shell", arguments: "{}" } },+ { payload: { type: "function_call_output", call_id: "call-1", output: "tool result" } },+ { payload: { message: { role: "assistant", content: "current assistant response" } } },+ ].map((entry) => JSON.stringify(entry)).join("\n"),+ );++ await withMockOpenViking(async (req, res) => {+ const url = new URL(req.url, "http://127.0.0.1");+ if (req.method === "GET" && url.pathname === "/health") {+ writeJson(res, { status: "ok", result: { ok: true } });+ return;+ }+ if (req.method === "POST" && url.pathname.endsWith("/messages/batch")) {+ batches.push(await readRequestBody(req));+ writeJson(res, { status: "ok", result: { ok: true } });+ return;+ }+ if (req.method === "GET" && url.pathname === "/api/v1/sessions/cx-compaction") {+ writeJson(res, {+ status: "ok",+ result: { pending_tokens: 0, commit_count: 0, total_message_count: 4 },+ });+ return;+ }+ res.writeHead(404, { "Content-Type": "application/json" });+ res.end(JSON.stringify({ status: "error", error: "not found" }));+ }, async (baseUrl) => {+ await runAutoCapture(+ { session_id: "compaction", transcript_path: transcriptPath },+ {+ OPENVIKING_AUTO_CAPTURE: "1",+ OPENVIKING_CAPTURE_ASSISTANT_TURNS: "1",+ OPENVIKING_CODEX_STATE_DIR: stateDir,+ OPENVIKING_CONFIG_FILE: join(stateDir, "missing-ov.conf"),+ OPENVIKING_CLI_CONFIG_FILE: join(stateDir, "missing-ovcli.conf"),+ OPENVIKING_CREDENTIAL_SOURCE: "env",+ OPENVIKING_MIN_QUERY_LENGTH: "1",+ OPENVIKING_WRITE_PATH_ASYNC: "0",+ OPENVIKING_TIMEOUT_MS: "5000",+ OPENVIKING_URL: baseUrl,+ },+ );+ });++ const messages = batches.flatMap((batch) => batch.messages || []);+ assert.equal(messages.length, 4);+ assert.equal(messages[0].parts[0].text, "current user request");+ assert.equal(+ messages.some((message) =>+ message.parts?.some((part) => part.text === "compacted historical summary")+ ),+ false,+ );+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});diff --git a/examples/codex-memory-plugin/scripts/capture-utils.mjs b/examples/codex-memory-plugin/scripts/capture-utils.mjsindex 4f01546f3..6fcd5f747 100644--- a/examples/codex-memory-plugin/scripts/capture-utils.mjs+++ b/examples/codex-memory-plugin/scripts/capture-utils.mjs@@ -176,3 +176,21 @@ export function extractCaptureTurns(rolloutEntries, cfg = {}) {const normalized = normalizeCodexNativeToolEvents(deduplicated);return extractSharedCaptureTurns(normalizeCodexMcpToolEvents(normalized), cfg);}++/**+ * Index of the last turn that came from a human prompt, or -1.+ *+ * `role === "user"` alone is not enough: normalizeCaptureRole() maps tool+ * results onto the user role too, and those carry `tool` parts rather than+ * `text` parts. Used by the post-compact shrink path to find where the current+ * interaction starts.+ */+export function findLastHumanTurnIndex(turns) {+ const list = Array.isArray(turns) ? turns : [];+ for (let i = list.length - 1; i >= 0; i -= 1) {+ const turn = list[i];+ if (turn?.role !== "user") continue;+ if (turn.parts?.some((part) => part?.type === "text")) return i;+ }+ return -1;+}diff --git a/examples/codex-memory-plugin/scripts/capture-utils.test.mjs b/examples/codex-memory-plugin/scripts/capture-utils.test.mjsindex 0c7c68498..6bfe64a5c 100644--- a/examples/codex-memory-plugin/scripts/capture-utils.test.mjs+++ b/examples/codex-memory-plugin/scripts/capture-utils.test.mjs@@ -1,7 +1,7 @@import assert from "node:assert/strict";import test from "node:test";-import { extractCaptureTurns } from "./capture-utils.mjs";+import { extractCaptureTurns, findLastHumanTurnIndex } from "./capture-utils.mjs";const CAPTURE_CONFIG = {captureAssistantTurns: true,@@ -497,3 +497,35 @@ test("captureToolMaxChars still caps tool output when an operator lowers it", ()assert.ok(completed.tool_output.length <= 1000);assert.match(completed.tool_output, /\[truncated\]$/);});++test("findLastHumanTurnIndex skips tool results mapped onto the user role", () => {+ const turns = extractCaptureTurns(+ [+ { payload: { message: { role: "user", content: "compacted historical summary" } } },+ { payload: { message: { role: "user", content: "current user request" } } },+ { payload: { type: "function_call", id: "call-1", name: "shell", arguments: "{}" } },+ { payload: { type: "function_call_output", call_id: "call-1", output: "tool result" } },+ { payload: { message: { role: "assistant", content: "current assistant response" } } },+ ],+ CAPTURE_CONFIG,+ );++ const index = findLastHumanTurnIndex(turns);+ assert.equal(turns[index].text, "current user request");+ assert.equal(turns.at(-2).role, "user");+ assert.equal(turns.at(-2).parts[0].type, "tool");+});++test("findLastHumanTurnIndex reports -1 when no human turn survives", () => {+ const turns = extractCaptureTurns(+ [+ { payload: { type: "function_call", id: "call-1", name: "shell", arguments: "{}" } },+ { payload: { type: "function_call_output", call_id: "call-1", output: "tool result" } },+ { payload: { message: { role: "assistant", content: "assistant only" } } },+ ],+ CAPTURE_CONFIG,+ );++ assert.equal(findLastHumanTurnIndex(turns), -1);+ assert.equal(findLastHumanTurnIndex([]), -1);+});diff --git a/examples/codex-memory-plugin/scripts/session-start-commit.mjs b/examples/codex-memory-plugin/scripts/session-start-commit.mjsindex 111ef731b..70b774334 100644--- a/examples/codex-memory-plugin/scripts/session-start-commit.mjs+++ b/examples/codex-memory-plugin/scripts/session-start-commit.mjs@@ -21,17 +21,24 @@* - 1 recently-active → commit it (the just-ended session)* - ≥2 recently-active → skip; rely on idle TTL* "Recently-active" means lastUpdatedAt within ACTIVE_WINDOW_MS (default 2 min).+ * Concurrency is judged on activity alone; only a state that still has a live+ * ovSessionId is actually committed.** At the tail (regardless of which branch above ran), run an idle-TTL sweep:- * any state file (including the new session_id, but in practice it's just- * been created and is fresh) older than IDLE_TTL_MS (default 30 min) gets- * committed and cleared. This catches SIGTERM/Ctrl+C/`/exit` exits and- * crashes that left state files orphaned.+ * any live OV session state older than IDLE_TTL_MS (default 30 min) gets+ * committed while retaining its transcript cursor. This catches+ * SIGTERM/Ctrl+C/`/exit` exits and crashes that left sessions orphaned.+ *+ * The same pass retires cursor-only states (no live OV session): a cursor+ * that was never used is dropped after IDLE_TTL_MS, and a real cursor is+ * kept for resume until COMMITTED_TTL_MS (default 30 days). Without this the+ * state directory grows one file per codex session forever and listStates()+ * reads all of them on every SessionStart.** Commit failure handling:- * On any /commit failure (OV unreachable, non-2xx, timeout) we DO NOT call- * clearState — we keep the state file with ovSessionId still set so the- * next sweep retries. A transient OV outage shouldn't lose memory.+ * On any /commit failure (OV unreachable, non-2xx, timeout) we keep the state+ * file with ovSessionId still set so the next sweep retries. A transient OV+ * outage shouldn't lose memory.** Output may contain hookSpecificOutput.additionalContext for profile/archive* injection and systemMessage for commit status at the same time.@@ -58,6 +65,11 @@ const IDLE_TTL_MS = (() => {return Number.isFinite(v) && v > 0 ? Math.floor(v) : 1_800_000;})();+const COMMITTED_TTL_MS = (() => {+ const v = Number(process.env.OPENVIKING_CODEX_COMMITTED_TTL_MS);+ return Number.isFinite(v) && v > 0 ? Math.floor(v) : 2_592_000_000;+})();+function output(obj) {process.stdout.write(JSON.stringify(obj) + "\n");}@@ -232,59 +244,69 @@ async function buildResumeArchiveContext(newSessionId) {}/**- * Commit and clear a single state file. On commit failure, preserve state- * (don't call clearState) so the next sweep retries.+ * Commit a live OV session and release it, keeping the transcript cursor so a+ * later resume appends instead of replaying (DESIGN.md "Commit-then-resume").+ * On commit failure, keep the live session id so the next sweep retries.*- * Returns { committed: bool, ovSessionId: string|null }.+ * Returns { committed: bool, ovSessionId: string|null, traceId: string }.*/-async function commitAndClear(state, reason) {- if (state.ovSessionId) {- const ovSessionId = state.ovSessionId;- const commit = await commitOvSession(state.ovSessionId);- if (!commit?.ok) {- log("commit", {- reason,- codexSessionId: state.codexSessionId,- ovSessionId: state.ovSessionId,- ok: false,- status: commit?.status,- trace_id: commit?.traceId,- error: commit?.error?.message || commit?.error?.code,- });- return { committed: false, ovSessionId: null, traceId: commit?.traceId || "" };- }- const traceId = commit.traceId || commit.result?.trace_id || "";+async function commitAndRelease(state, reason) {+ const ovSessionId = state.ovSessionId;+ const commit = await commitOvSession(ovSessionId);+ if (!commit?.ok) {log("commit", {reason,codexSessionId: state.codexSessionId,ovSessionId,- archived: commit.result?.archived ?? false,- taskId: commit.result?.task_id,- status: commit.result?.status,- trace_id: traceId || undefined,+ ok: false,+ status: commit?.status,+ trace_id: commit?.traceId,+ error: commit?.error?.message || commit?.error?.code,});- await clearState(state.codexSessionId);- return { committed: true, ovSessionId, traceId };+ return { committed: false, ovSessionId: null, traceId: commit?.traceId || "" };}- // No OV session attached — nothing to commit on the server, but the local- // state file is still stale and should be removed.- log("clear_no_ov", { reason, codexSessionId: state.codexSessionId });+ const traceId = commit.traceId || commit.result?.trace_id || "";+ log("commit", {+ reason,+ codexSessionId: state.codexSessionId,+ ovSessionId,+ archived: commit.result?.archived ?? false,+ taskId: commit.result?.task_id,+ status: commit.result?.status,+ trace_id: traceId || undefined,+ });+ state.ovSessionId = null;+ await saveState(state, { touch: false });+ return { committed: true, ovSessionId, traceId };+}++/**+ * Retire a state file with no live OV session. A cursor that never captured+ * anything carries no resume value, so it goes on the idle schedule; a real+ * cursor is kept until COMMITTED_TTL_MS in case the codex session is resumed.+ */+async function maybeRetireCursorState(state, ageMs) {+ const hasCursor = Number(state.capturedTurnCount) > 0;+ const ttl = hasCursor ? COMMITTED_TTL_MS : IDLE_TTL_MS;+ if (ageMs <= ttl) return false;+ log("state_retire", {+ codexSessionId: state.codexSessionId,+ capturedTurnCount: state.capturedTurnCount,+ ageMs,+ ttlMs: ttl,+ });await clearState(state.codexSessionId);- return { committed: true, ovSessionId: null, traceId: "" };+ return true;}function describeCommittedSessions(commits) {+ const traceIds = commits.map((item) => item.traceId).filter(Boolean);if (commits.length === 1) {return `OpenViking session ${commits[0].ovSessionId} is committed` +- (commits[0].traceId ? ` (trace_id=${commits[0].traceId})` : "");+ (traceIds.length ? ` (trace_id=${traceIds[0]})` : "");}- const ovSessionIds = commits.map((item) => item.ovSessionId);- if (ovSessionIds.length > 1) {- const traceIds = commits.map((item) => item.traceId).filter(Boolean);- return `OpenViking sessions ${ovSessionIds.join(", ")} are committed` +- (traceIds.length ? ` (trace_ids=${traceIds.join(",")})` : "");- }- return "OpenViking session state is cleared";+ return `OpenViking sessions ${commits.map((item) => item.ovSessionId).join(", ")} are committed` ++ (traceIds.length ? ` (trace_ids=${traceIds.join(",")})` : "");}async function main() {@@ -367,12 +389,15 @@ async function main() {(s) => s?.codexSessionId && s.codexSessionId !== newSessionId,);+ // Concurrency is judged on activity, not on whether an OV session is live:+ // a session that PreCompact just committed is cursor-only yet may still be+ // running, and counting it keeps the ≥2 branch from committing a sibling+ // session mid-flight.const recentlyActive = otherStates.filter((s) => typeof s.lastUpdatedAt === "number"&& (now - s.lastUpdatedAt) <= ACTIVE_WINDOW_MS,);- let heuristicCommitted = 0;const heuristicCommits = [];const skippedSessionIds = new Set();@@ -380,16 +405,22 @@ async function main() {log("heuristic", { branch: "0_active", action: "noop", otherStates: otherStates.length });} else if (recentlyActive.length === 1) {const target = recentlyActive[0];- log("heuristic", {- branch: "1_active",- action: "commit",- codexSessionId: target.codexSessionId,- ovSessionId: target.ovSessionId,- });- const r = await commitAndClear(target, "heuristic_1_active");- if (r.committed) {- heuristicCommitted += 1;- if (r.ovSessionId) heuristicCommits.push(r);+ if (!target.ovSessionId) {+ log("heuristic", {+ branch: "1_active",+ action: "skip",+ reason: "no live OV session",+ codexSessionId: target.codexSessionId,+ });+ } else {+ log("heuristic", {+ branch: "1_active",+ action: "commit",+ codexSessionId: target.codexSessionId,+ ovSessionId: target.ovSessionId,+ });+ const r = await commitAndRelease(target, "heuristic_1_active");+ if (r.committed) heuristicCommits.push(r);}} else {log("heuristic", {@@ -402,44 +433,46 @@ async function main() {}// -------------------------------------------------------------------------- // Idle TTL sweep (tail) — applies to ALL state files including ones we just- // skipped above (≥2 active path). We re-list because the heuristic branch- // may have removed entries.+ // Idle TTL sweep + cursor retention (tail) — applies to ALL state files+ // including ones we just skipped above (≥2 active path). We re-list because+ // the heuristic branch may have changed entries.// -------------------------------------------------------------------------const postHeuristic = await listStates();- let idleCommitted = 0;const idleCommits = [];+ let retired = 0;for (const s of postHeuristic) {if (!s?.codexSessionId) continue;if (typeof s.lastUpdatedAt !== "number") continue;- if ((now - s.lastUpdatedAt) <= IDLE_TTL_MS) continue;+ const ageMs = now - s.lastUpdatedAt;+ if (!s.ovSessionId) {+ if (await maybeRetireCursorState(s, ageMs)) retired += 1;+ continue;+ }+ if (ageMs <= IDLE_TTL_MS) continue;log("idle_sweep", {codexSessionId: s.codexSessionId,ovSessionId: s.ovSessionId,- ageMs: now - s.lastUpdatedAt,+ ageMs,});- const r = await commitAndClear(s, "idle_ttl");- if (r.committed) {- idleCommitted += 1;- if (r.ovSessionId) idleCommits.push(r);- }+ const r = await commitAndRelease(s, "idle_ttl");+ if (r.committed) idleCommits.push(r);}- const totalCommitted = heuristicCommitted + idleCommitted;const commits = [...heuristicCommits, ...idleCommits];const ovSessionIds = commits.map((item) => item.ovSessionId);log("done", {source,- heuristicCommitted,- idleCommitted,- totalCommitted,+ heuristicCommitted: heuristicCommits.length,+ idleCommitted: idleCommits.length,+ totalCommitted: commits.length,+ retired,ovSessionIds,skipped: [...skippedSessionIds],});- if (totalCommitted > 0) {+ if (commits.length > 0) {emitSessionStartOutput({contexts: [profileContext],systemMessage: describeCommittedSessions(commits),diff --git a/examples/codex-memory-plugin/scripts/session-start-commit.test.mjs b/examples/codex-memory-plugin/scripts/session-start-commit.test.mjsindex 2ae20f475..81567f073 100644--- a/examples/codex-memory-plugin/scripts/session-start-commit.test.mjs+++ b/examples/codex-memory-plugin/scripts/session-start-commit.test.mjs@@ -1,6 +1,6 @@import assert from "node:assert/strict";import { spawn } from "node:child_process";-import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";+import { mkdir, mkdtemp, readFile, readdir, rm, writeFile } from "node:fs/promises";import http from "node:http";import { tmpdir } from "node:os";import { dirname, join } from "node:path";@@ -224,6 +224,149 @@ test("startup preserves the existing commit systemMessage alongside profile contrequest.method === "POST"&& request.path === "/api/v1/sessions/cx-old-session/commit"));+ const state = JSON.parse(await readFile(join(stateDir, "old-session.json"), "utf-8"));+ assert.equal(state.ovSessionId, null);+ assert.equal(state.capturedTurnCount, 2);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("startup ignores committed cursor-only states", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-codex-cursor-only-"));+ const requests = [];+ try {+ const now = Date.now();+ await Promise.all([+ writeFile(join(stateDir, "recent.json"), JSON.stringify({+ codexSessionId: "recent",+ ovSessionId: null,+ capturedTurnCount: 4,+ createdAt: now - 500,+ lastUpdatedAt: now,+ })),+ writeFile(join(stateDir, "stale.json"), JSON.stringify({+ codexSessionId: "stale",+ ovSessionId: null,+ capturedTurnCount: 6,+ createdAt: now - 20_000,+ lastUpdatedAt: now - 10_000,+ })),+ ]);++ await withMockOpenViking(profileHandler(requests), async (baseUrl) => {+ await runSessionStart(+ { session_id: "new-session", source: "startup", cwd: "/tmp/codex-cursor-only" },+ {+ ...baseEnv(baseUrl, stateDir),+ OPENVIKING_CODEX_ACTIVE_WINDOW_MS: "1000",+ OPENVIKING_CODEX_IDLE_TTL_MS: "5000",+ },+ );+ });++ const [recent, stale] = await Promise.all([+ readFile(join(stateDir, "recent.json"), "utf-8"),+ readFile(join(stateDir, "stale.json"), "utf-8"),+ ]);+ assert.equal(JSON.parse(recent).capturedTurnCount, 4);+ assert.equal(JSON.parse(stale).capturedTurnCount, 6);+ assert.equal(requests.some((request) => request.path.endsWith("/commit")), false);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("startup retires cursor-only states once their retention window closes", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-codex-cursor-retention-"));+ const requests = [];+ try {+ const now = Date.now();+ await Promise.all([+ writeFile(join(stateDir, "expired.json"), JSON.stringify({+ codexSessionId: "expired",+ ovSessionId: null,+ capturedTurnCount: 6,+ createdAt: now - 20_000,+ lastUpdatedAt: now - 10_000,+ })),+ writeFile(join(stateDir, "keeper.json"), JSON.stringify({+ codexSessionId: "keeper",+ ovSessionId: null,+ capturedTurnCount: 4,+ createdAt: now - 8_000,+ lastUpdatedAt: now - 6_000,+ })),+ writeFile(join(stateDir, "empty.json"), JSON.stringify({+ codexSessionId: "empty",+ ovSessionId: null,+ capturedTurnCount: 0,+ createdAt: now - 8_000,+ lastUpdatedAt: now - 6_000,+ })),+ ]);++ await withMockOpenViking(profileHandler(requests), async (baseUrl) => {+ await runSessionStart(+ { session_id: "new-session", source: "startup", cwd: "/tmp/codex-cursor-retention" },+ {+ ...baseEnv(baseUrl, stateDir),+ OPENVIKING_CODEX_ACTIVE_WINDOW_MS: "500",+ OPENVIKING_CODEX_IDLE_TTL_MS: "5000",+ OPENVIKING_CODEX_COMMITTED_TTL_MS: "8000",+ },+ );+ });++ const files = await readdir(stateDir);+ assert.equal(files.includes("expired.json"), false);+ assert.equal(files.includes("empty.json"), false);+ assert.equal(JSON.parse(await readFile(join(stateDir, "keeper.json"), "utf-8")).capturedTurnCount, 4);+ assert.equal(requests.some((request) => request.path.endsWith("/commit")), false);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("startup defers to idle TTL when a cursor-only session is still active", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-codex-active-cursor-"));+ const requests = [];+ try {+ const now = Date.now();+ await Promise.all([+ writeFile(join(stateDir, "live.json"), JSON.stringify({+ codexSessionId: "live",+ ovSessionId: "cx-live",+ capturedTurnCount: 2,+ createdAt: now - 2_000,+ lastUpdatedAt: now - 100,+ })),+ // Just committed by PreCompact, so cursor-only — but still running.+ writeFile(join(stateDir, "compacted.json"), JSON.stringify({+ codexSessionId: "compacted",+ ovSessionId: null,+ capturedTurnCount: 5,+ createdAt: now - 2_000,+ lastUpdatedAt: now - 100,+ })),+ ]);++ await withMockOpenViking(profileHandler(requests), async (baseUrl) => {+ const { output } = await runSessionStart(+ { session_id: "new-session", source: "startup", cwd: "/tmp/codex-active-cursor" },+ {+ ...baseEnv(baseUrl, stateDir),+ OPENVIKING_CODEX_ACTIVE_WINDOW_MS: "60000",+ OPENVIKING_CODEX_IDLE_TTL_MS: "1800000",+ },+ );+ assert.equal(output.systemMessage, undefined);+ });++ assert.equal(requests.some((request) => request.path.endsWith("/commit")), false);+ const live = JSON.parse(await readFile(join(stateDir, "live.json"), "utf-8"));+ assert.equal(live.ovSessionId, "cx-live");+ assert.equal(JSON.parse(await readFile(join(stateDir, "compacted.json"), "utf-8")).capturedTurnCount, 5);} finally {await rm(stateDir, { recursive: true, force: true });}diff --git a/examples/codex-memory-plugin/scripts/session-state.mjs b/examples/codex-memory-plugin/scripts/session-state.mjsindex 9837c4768..def36b832 100644--- a/examples/codex-memory-plugin/scripts/session-state.mjs+++ b/examples/codex-memory-plugin/scripts/session-state.mjs@@ -64,10 +64,21 @@ export async function loadState(codexSessionId) {}}-export async function saveState(state) {+/**+ * Persist state. `touch: false` keeps the existing `lastUpdatedAt` so a write+ * that isn't transcript activity (e.g. releasing `ovSessionId` after a commit)+ * doesn't make a dead session look freshly used to the active-window heuristic+ * and the retention sweep.+ */+export async function saveState(state, { touch = true } = {}) {if (!state || !state.codexSessionId) return;await mkdir(getStateDir(), { recursive: true });- const next = { ...state, lastUpdatedAt: Date.now() };+ const next = {+ ...state,+ lastUpdatedAt: touch || typeof state.lastUpdatedAt !== "number"+ ? Date.now()+ : state.lastUpdatedAt,+ };// Atomic write (tmpfile + rename) so a crash mid-write can't leave a// truncated/corrupt state file. See DESIGN.md "State file schema".const final = statePath(state.codexSessionId);