Skip to content

Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.

fix(codex): preserve transcript cursors across commit and compaction (#4191)

t0sakia61c57bf5979
diff --git a/docs/en/agent-integrations/04-codex.md b/docs/en/agent-integrations/04-codex.md
index 7737e04ec..8388e72dc 100644
--- a/docs/en/agent-integrations/04-codex.md
+++ b/docs/en/agent-integrations/04-codex.md
@@ -78,6 +78,7 @@ Credential source: env vars win by default — when any `OPENVIKING_*` credentia
| `OPENVIKING_PROFILE_TOKEN_BUDGET` | `10000` | CJK-aware token budget for `profile.md` plus `preferences/` and `entities/` indexes |
| `OPENVIKING_CODEX_ACTIVE_WINDOW_MS` | `120000` | SessionStart active-window threshold |
| `OPENVIKING_CODEX_IDLE_TTL_MS` | `1800000` | SessionStart idle-TTL sweep threshold |
+| `OPENVIKING_CODEX_COMMITTED_TTL_MS` | `2592000000` | How long a committed session's transcript cursor is kept before its state file is retired |
| `OPENVIKING_DEBUG` | `false` | Write logs to `~/.openviking/logs/codex-hooks.log` |
If recall latency matters most, see [Low-latency recall](./01-overview.md#low-latency-recall) for the environment-variable and `ovcli.conf` settings that disable query expansion and Codex's local result compression.
diff --git a/docs/zh/agent-integrations/04-codex.md b/docs/zh/agent-integrations/04-codex.md
index b0dbc1c20..a5e07f886 100644
--- a/docs/zh/agent-integrations/04-codex.md
+++ b/docs/zh/agent-integrations/04-codex.md
@@ -77,6 +77,7 @@ TraeCode CLI 2.0 用户启动 `trae-cli`,并可用 `trae-cli plugin list` 确
| `OPENVIKING_PROFILE_TOKEN_BUDGET` | `10000` | `profile.md` 及 `preferences/`、`entities/` 索引共用的 CJK-aware token 预算 |
| `OPENVIKING_CODEX_ACTIVE_WINDOW_MS` | `120000` | `SessionStart` 活动窗口阈值(毫秒) |
| `OPENVIKING_CODEX_IDLE_TTL_MS` | `1800000` | `SessionStart` 闲置 TTL 清理阈值(毫秒) |
+| `OPENVIKING_CODEX_COMMITTED_TTL_MS` | `2592000000` | 已提交会话的转录游标保留时长(毫秒),过期后删除状态文件 |
| `OPENVIKING_DEBUG` | `false` | 是否将日志写入 `~/.openviking/logs/codex-hooks.log` |
如果更看重召回响应速度,请参阅[低延迟召回](./01-overview.md#低延迟召回),其中说明了如何通过环境变量或 `ovcli.conf` 关闭查询扩展与 Codex 本地结果压缩。
diff --git a/examples/codex-memory-plugin/.codex-plugin/plugin.json b/examples/codex-memory-plugin/.codex-plugin/plugin.json
index 2481cdd6b..ac62b72e4 100644
--- a/examples/codex-memory-plugin/.codex-plugin/plugin.json
+++ b/examples/codex-memory-plugin/.codex-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "openviking-memory",
- "version": "0.7.5",
+ "version": "0.7.6",
"description": "Long-term semantic memory for Codex, powered by OpenViking. Recall on UserPromptSubmit, incremental add_message on Stop (per turn), commit on PreCompact, and active-window heuristic + idle-TTL sweep on SessionStart (source=startup|clear). Includes a local stdio MCP proxy that reads OpenViking credentials from env/ovcli.conf.",
"author": {
"name": "OpenViking",
diff --git a/examples/codex-memory-plugin/DESIGN.md b/examples/codex-memory-plugin/DESIGN.md
index 19254cb19..501df7a08 100644
--- a/examples/codex-memory-plugin/DESIGN.md
+++ b/examples/codex-memory-plugin/DESIGN.md
@@ -90,6 +90,12 @@ fired and bumped `lastUpdatedAt`; we commit it. The multi-recent case
implies concurrent codex sessions are active; we can't tell which one (if
any) ended, so we defer to idle TTL to clean up genuinely-dead ones.
+The count is over *activity*, including cursor-only states: a session
+`PreCompact` just committed has no live `ovSessionId` but may well still be
+running, and counting it keeps the `≥2` branch from sealing a sibling
+session mid-flight. Only a state that still has a live `ovSessionId` is
+actually committed; the single-recent case with nothing live is a no-op.
+
### 4. `SessionStart` source=`resume` — never commits, optional archive inject
Short reconnects and `/resume` re-fire `SessionStart` for the same
@@ -120,11 +126,16 @@ transcript.
### 5. Idle TTL sweep — fallback
-State files whose `lastUpdatedAt` is older than `IDLE_TTL_MS` (default 30
-min) get committed and cleared. Mental model: a session not touched for
-30 min is "temporarily concluded"; if the user resumes later, subsequent
-turns append under the same deterministic OV session id, and the next
-commit creates another archive there.
+State files with a live `ovSessionId` whose `lastUpdatedAt` is older than
+`IDLE_TTL_MS` (default 30 min) get committed. Their transcript cursor is
+preserved while `ovSessionId` is cleared. Mental model: a session not touched
+for 30 min is "temporarily concluded"; if the user resumes later, subsequent
+turns append under the same deterministic OV session id, and the next commit
+creates another archive there.
+
+Releasing the session id instead of deleting the file writes state without
+touching `lastUpdatedAt`, so a committed session doesn't look freshly active
+to the next `SessionStart`.
This covers:
- SIGTERM / Ctrl+C / `/exit` (no hook fires; state file rots)
@@ -145,6 +156,21 @@ machine, no sweep ever runs and the OV session stays open server-side
forever. Accepted. Future work could add an MCP tool
`openviking_commit_pending` so the model can commit explicitly.
+### 6. Cursor retention — same pass
+
+A committed state file keeps living as a cursor (`ovSessionId: null`) so a
+later resume appends instead of replaying. Kept forever that would leak one
+file per codex session and make `listStates()` — which reads every file on
+every `SessionStart` — slower over time, so the same pass retires them:
+
+| State | Retired after |
+|---|---|
+| cursor-only, `capturedTurnCount > 0` | `COMMITTED_TTL_MS` (default 30 days) |
+| cursor-only, nothing ever captured | `IDLE_TTL_MS` (default 30 min) |
+
+A cursor that outlives its codex rollout has nothing left to resume from, and
+a state file that never captured a turn is identical to no state file at all.
+
## Stop hook — append + threshold commit
Every `Stop` reads `transcript_path`, slices to `[capturedTurnCount, end)`,
@@ -194,9 +220,19 @@ compatibility fallbacks.
Codex's `/compact` may rewrite or truncate `transcript_path`. After
compaction, if `allTurns.length < state.capturedTurnCount`, our slice
-math underflows and we silently drop new turns. Defensive fix: when this
-inequality is detected on `Stop`, reset `capturedTurnCount = 0` so the
-next slice captures everything in the new transcript.
+math underflows and we silently drop new turns. When this inequality is
+detected on `Stop`, move `capturedTurnCount` to the latest human turn
+so the current interaction is captured without replaying compacted history.
+
+"Human turn" is not just `role === "user"` — `normalizeCaptureRole()` maps
+tool results onto the user role as well, so `findLastHumanTurnIndex()`
+additionally requires a `text` part. If the rewrite left no human turn at
+all (index `-1`), we fall back to capturing the whole transcript and log
+`fallback: "full_transcript"`; replaying beats losing the interaction.
+
+**Trade-off**: turns older than that human turn are assumed captured. If
+earlier `Stop` hooks failed to reach OV and compaction happened before they
+were retried, those turns are dropped rather than duplicated.
### Commit failure
@@ -226,7 +262,7 @@ OV session id, while commits create additional archives under that session.
```json
{
"codexSessionId": "0193af...", // codex thread id
- "ovSessionId": "cx-0193af...-or-null", // null means "committed, awaiting next Stop"
+ "ovSessionId": "cx-0193af...-or-null", // null means "committed, awaiting next Stop or retirement"
"capturedTurnCount": 7, // turns from transcript already appended
"createdAt": 1715000000000,
"lastUpdatedAt": 1715000300000
@@ -249,6 +285,7 @@ Env var overrides for tuning without rebuilding:
| `OPENVIKING_CODEX_STATE_DIR` | `~/.openviking/codex-plugin-state` | state file dir |
| `OPENVIKING_CODEX_ACTIVE_WINDOW_MS` | `120000` (2 min) | rule-3 active window |
| `OPENVIKING_CODEX_IDLE_TTL_MS` | `1800000` (30 min) | idle sweep TTL |
+| `OPENVIKING_CODEX_COMMITTED_TTL_MS` | `2592000000` (30 days) | how long a committed cursor is kept for resume |
| `OPENVIKING_RECALL_TIMEOUT_MS` | `120000` (2 min) | whole UserPromptSubmit auto-recall deadline |
| `OPENVIKING_RECALL_COMPRESS` | `1` | set `0` / `off` to skip `codex exec` compression |
| `OPENVIKING_RECALL_COMPRESS_MODEL` | unset | custom first-choice compressor model; `off` disables compression |
diff --git a/examples/codex-memory-plugin/README.md b/examples/codex-memory-plugin/README.md
index 88670017b..db2dd066d 100644
--- a/examples/codex-memory-plugin/README.md
+++ b/examples/codex-memory-plugin/README.md
@@ -198,11 +198,12 @@ On `startup` or `clear`, the script:
1. Counts state files (excluding the new session_id) whose `lastUpdatedAt` is within `OPENVIKING_CODEX_ACTIVE_WINDOW_MS` (default 2 min) of "now":
- **0 active** → no-op (no orphan to commit)
- - **1 active** → commit it (the just-ended session)
+ - **1 active** → commit it (the just-ended session), unless it has no live `ovSessionId` left to commit
- **≥2 active** → skip; rely on idle TTL (we can't tell which one ended)
-2. **Idle-TTL sweep at the tail**: any state file (regardless of session_id) older than `OPENVIKING_CODEX_IDLE_TTL_MS` (default 30 min) gets committed and cleared.
+2. **Idle-TTL sweep at the tail**: any live session state older than `OPENVIKING_CODEX_IDLE_TTL_MS` (default 30 min) gets committed while preserving its transcript cursor for resume.
+3. **Cursor retention in the same pass**: a state file with no live OV session is kept as a resume cursor until `OPENVIKING_CODEX_COMMITTED_TTL_MS` (default 30 days), or dropped after the idle TTL if it never captured a turn.
-On any /commit failure (OV unreachable, non-2xx, timeout) we **preserve state** (don't `clearState`) so the next sweep can retry.
+On any /commit failure (OV unreachable, non-2xx, timeout) we **preserve state** (keep `ovSessionId` set) so the next sweep can retry.
On `resume`, the script skips commit/sweep. It still injects the profile block. If local state has no live `ovSessionId`, it also reads `/api/v1/sessions/{cx-session-id}/context` and combines the latest committed archive overview into the same `SessionStart` output. The archive block includes a `viking://~/sessions/{cx-session-id}/history/` URI and tells the model to use the OpenViking MCP `read`/`search` tools for exact prior commands, file paths, tool outputs, or messages. Set `OPENVIKING_RESUME_ARCHIVE_INJECT=0` to disable the archive half without disabling profile injection.
diff --git a/examples/codex-memory-plugin/VERIFICATION.md b/examples/codex-memory-plugin/VERIFICATION.md
index 22f3cccd5..79ff12ac4 100644
--- a/examples/codex-memory-plugin/VERIFICATION.md
+++ b/examples/codex-memory-plugin/VERIFICATION.md
@@ -196,7 +196,7 @@ files are still present — the skip path does not clear them.
# Backdate one of the state files to be older than IDLE_TTL_MS (default 30 min).
OLD=$(node -e 'console.log(Date.now() - 60*60*1000)') # 1 hour ago
cat > "$STATE_DIR/state/sess-aaa.json" <<EOF
-{"codexSessionId":"sess-aaa","ovSessionId":null,"capturedTurnCount":0,"createdAt":$OLD,"lastUpdatedAt":$OLD}
+{"codexSessionId":"sess-aaa","ovSessionId":"cx-sess-aaa","capturedTurnCount":2,"createdAt":$OLD,"lastUpdatedAt":$OLD}
EOF
echo '{"session_id":"sess-ddd","source":"startup","cwd":"/tmp","model":"x","permission_mode":"default","transcript_path":null,"hook_event_name":"SessionStart"}' \
@@ -206,12 +206,31 @@ echo '{"session_id":"sess-ddd","source":"startup","cwd":"/tmp","model":"x","perm
node $PLUGIN/scripts/session-start-commit.mjs
```
-Expect: log shows `idle_sweep` for `sess-aaa` (committed and cleared).
-`sess-bbb.json` is still present (still fresh). `sess-aaa.json` is gone.
+Expect: log shows `idle_sweep` for `sess-aaa` (committed).
+`sess-bbb.json` is still present (still fresh). `sess-aaa.json` is also
+present with `ovSessionId: null` and `capturedTurnCount: 2` for resume.
If `sess-bbb` was in `≥2 active` from 6c, the heuristic on this call sees
just `sess-bbb` (1 active) and commits it — that's expected and shows the
heuristic + sweep working together.
+### 6d-2. Cursor retention
+
+```bash
+# sess-aaa is now cursor-only (ovSessionId: null, capturedTurnCount: 2).
+# Re-run the same SessionStart with a 1 s committed TTL.
+echo '{"session_id":"sess-eee","source":"startup","cwd":"/tmp","model":"x","permission_mode":"default","transcript_path":null,"hook_event_name":"SessionStart"}' \
+ | OPENVIKING_CONFIG_FILE=$OV_CONF \
+ OPENVIKING_CODEX_STATE_DIR=$STATE_DIR/state \
+ OPENVIKING_CODEX_COMMITTED_TTL_MS=1000 \
+ CODEX_PLUGIN_ROOT=$PLUGIN \
+ OPENVIKING_DEBUG=1 \
+ node $PLUGIN/scripts/session-start-commit.mjs
+```
+
+Expect: log shows `state_retire` for `sess-aaa` and the file is gone. With
+the default 30-day TTL it stays, and no `/commit` is issued for it either
+way — a cursor-only state has nothing left to commit.
+
### 6e. `source=resume` → no commit/sweep; optional archive inject
```bash
diff --git a/examples/codex-memory-plugin/scripts/auto-capture.mjs b/examples/codex-memory-plugin/scripts/auto-capture.mjs
index 5dcd5a7cf..c1b3b7f16 100644
--- a/examples/codex-memory-plugin/scripts/auto-capture.mjs
+++ b/examples/codex-memory-plugin/scripts/auto-capture.mjs
@@ -28,6 +28,7 @@
import { readFile } from "node:fs/promises";
import {
extractCaptureTurns,
+ findLastHumanTurnIndex,
} from "./capture-utils.mjs";
import { loadConfig } from "./config.mjs";
import { createLogger } from "./debug-log.mjs";
@@ -226,15 +227,20 @@ async function main() {
// Post-compact transcript-shrink defense: codex's /compact may rewrite or
// truncate transcript_path. If allTurns has fewer entries than we cached,
- // our slice math would underflow and silently drop turns. Reset the
- // counter so the next slice captures everything in the new transcript.
- // See DESIGN.md "Post-compact transcript shrink".
+ // our slice math would underflow and silently drop turns. Resume at the
+ // latest human turn so the current interaction is captured without replaying
+ // compacted history. See DESIGN.md "Post-compact transcript shrink".
if (allTurns.length < state.capturedTurnCount) {
+ const humanIndex = findLastHumanTurnIndex(allTurns);
log("transcript_shrink_detected", {
cached: state.capturedTurnCount,
observed: allTurns.length,
+ resumeFrom: Math.max(0, humanIndex),
+ // -1 means no human turn survived the rewrite; capturing the whole
+ // transcript is the only way not to lose the interaction.
+ fallback: humanIndex < 0 ? "full_transcript" : undefined,
});
- state.capturedTurnCount = 0;
+ state.capturedTurnCount = Math.max(0, humanIndex);
}
const newTurns = allTurns.slice(state.capturedTurnCount);
diff --git a/examples/codex-memory-plugin/scripts/auto-capture.test.mjs b/examples/codex-memory-plugin/scripts/auto-capture.test.mjs
index 0e1ac0de4..8aee2c283 100644
--- a/examples/codex-memory-plugin/scripts/auto-capture.test.mjs
+++ b/examples/codex-memory-plugin/scripts/auto-capture.test.mjs
@@ -385,3 +385,81 @@ test("auto-capture logs a commit error trace_id", async () => {
await rm(stateDir, { recursive: true, force: true });
}
});
+
+test("auto-capture skips compacted history after transcript shrink", async () => {
+ const stateDir = await mkdtemp(join(tmpdir(), "ov-auto-capture-compact-"));
+ const transcriptPath = join(stateDir, "transcript.jsonl");
+ const batches = [];
+ const now = Date.now();
+
+ try {
+ await writeFile(join(stateDir, "compaction.json"), JSON.stringify({
+ codexSessionId: "compaction",
+ ovSessionId: "cx-compaction",
+ capturedTurnCount: 8,
+ createdAt: now - 1000,
+ lastUpdatedAt: now,
+ }));
+ await writeFile(
+ transcriptPath,
+ [
+ { payload: { message: { role: "user", content: "compacted historical summary" } } },
+ { payload: { message: { role: "assistant", content: "prior assistant tail" } } },
+ { payload: { message: { role: "user", content: "current user request" } } },
+ { payload: { type: "function_call", id: "call-1", name: "shell", arguments: "{}" } },
+ { payload: { type: "function_call_output", call_id: "call-1", output: "tool result" } },
+ { payload: { message: { role: "assistant", content: "current assistant response" } } },
+ ].map((entry) => JSON.stringify(entry)).join("\n"),
+ );
+
+ await withMockOpenViking(async (req, res) => {
+ const url = new URL(req.url, "http://127.0.0.1");
+ if (req.method === "GET" && url.pathname === "/health") {
+ writeJson(res, { status: "ok", result: { ok: true } });
+ return;
+ }
+ if (req.method === "POST" && url.pathname.endsWith("/messages/batch")) {
+ batches.push(await readRequestBody(req));
+ writeJson(res, { status: "ok", result: { ok: true } });
+ return;
+ }
+ if (req.method === "GET" && url.pathname === "/api/v1/sessions/cx-compaction") {
+ writeJson(res, {
+ status: "ok",
+ result: { pending_tokens: 0, commit_count: 0, total_message_count: 4 },
+ });
+ return;
+ }
+ res.writeHead(404, { "Content-Type": "application/json" });
+ res.end(JSON.stringify({ status: "error", error: "not found" }));
+ }, async (baseUrl) => {
+ await runAutoCapture(
+ { session_id: "compaction", transcript_path: transcriptPath },
+ {
+ OPENVIKING_AUTO_CAPTURE: "1",
+ OPENVIKING_CAPTURE_ASSISTANT_TURNS: "1",
+ OPENVIKING_CODEX_STATE_DIR: stateDir,
+ OPENVIKING_CONFIG_FILE: join(stateDir, "missing-ov.conf"),
+ OPENVIKING_CLI_CONFIG_FILE: join(stateDir, "missing-ovcli.conf"),
+ OPENVIKING_CREDENTIAL_SOURCE: "env",
+ OPENVIKING_MIN_QUERY_LENGTH: "1",
+ OPENVIKING_WRITE_PATH_ASYNC: "0",
+ OPENVIKING_TIMEOUT_MS: "5000",
+ OPENVIKING_URL: baseUrl,
+ },
+ );
+ });
+
+ const messages = batches.flatMap((batch) => batch.messages || []);
+ assert.equal(messages.length, 4);
+ assert.equal(messages[0].parts[0].text, "current user request");
+ assert.equal(
+ messages.some((message) =>
+ message.parts?.some((part) => part.text === "compacted historical summary")
+ ),
+ false,
+ );
+ } finally {
+ await rm(stateDir, { recursive: true, force: true });
+ }
+});
diff --git a/examples/codex-memory-plugin/scripts/capture-utils.mjs b/examples/codex-memory-plugin/scripts/capture-utils.mjs
index 4f01546f3..6fcd5f747 100644
--- a/examples/codex-memory-plugin/scripts/capture-utils.mjs
+++ b/examples/codex-memory-plugin/scripts/capture-utils.mjs
@@ -176,3 +176,21 @@ export function extractCaptureTurns(rolloutEntries, cfg = {}) {
const normalized = normalizeCodexNativeToolEvents(deduplicated);
return extractSharedCaptureTurns(normalizeCodexMcpToolEvents(normalized), cfg);
}
+
+/**
+ * Index of the last turn that came from a human prompt, or -1.
+ *
+ * `role === "user"` alone is not enough: normalizeCaptureRole() maps tool
+ * results onto the user role too, and those carry `tool` parts rather than
+ * `text` parts. Used by the post-compact shrink path to find where the current
+ * interaction starts.
+ */
+export function findLastHumanTurnIndex(turns) {
+ const list = Array.isArray(turns) ? turns : [];
+ for (let i = list.length - 1; i >= 0; i -= 1) {
+ const turn = list[i];
+ if (turn?.role !== "user") continue;
+ if (turn.parts?.some((part) => part?.type === "text")) return i;
+ }
+ return -1;
+}
diff --git a/examples/codex-memory-plugin/scripts/capture-utils.test.mjs b/examples/codex-memory-plugin/scripts/capture-utils.test.mjs
index 0c7c68498..6bfe64a5c 100644
--- a/examples/codex-memory-plugin/scripts/capture-utils.test.mjs
+++ b/examples/codex-memory-plugin/scripts/capture-utils.test.mjs
@@ -1,7 +1,7 @@
import assert from "node:assert/strict";
import test from "node:test";
-import { extractCaptureTurns } from "./capture-utils.mjs";
+import { extractCaptureTurns, findLastHumanTurnIndex } from "./capture-utils.mjs";
const CAPTURE_CONFIG = {
captureAssistantTurns: true,
@@ -497,3 +497,35 @@ test("captureToolMaxChars still caps tool output when an operator lowers it", ()
assert.ok(completed.tool_output.length <= 1000);
assert.match(completed.tool_output, /\[truncated\]$/);
});
+
+test("findLastHumanTurnIndex skips tool results mapped onto the user role", () => {
+ const turns = extractCaptureTurns(
+ [
+ { payload: { message: { role: "user", content: "compacted historical summary" } } },
+ { payload: { message: { role: "user", content: "current user request" } } },
+ { payload: { type: "function_call", id: "call-1", name: "shell", arguments: "{}" } },
+ { payload: { type: "function_call_output", call_id: "call-1", output: "tool result" } },
+ { payload: { message: { role: "assistant", content: "current assistant response" } } },
+ ],
+ CAPTURE_CONFIG,
+ );
+
+ const index = findLastHumanTurnIndex(turns);
+ assert.equal(turns[index].text, "current user request");
+ assert.equal(turns.at(-2).role, "user");
+ assert.equal(turns.at(-2).parts[0].type, "tool");
+});
+
+test("findLastHumanTurnIndex reports -1 when no human turn survives", () => {
+ const turns = extractCaptureTurns(
+ [
+ { payload: { type: "function_call", id: "call-1", name: "shell", arguments: "{}" } },
+ { payload: { type: "function_call_output", call_id: "call-1", output: "tool result" } },
+ { payload: { message: { role: "assistant", content: "assistant only" } } },
+ ],
+ CAPTURE_CONFIG,
+ );
+
+ assert.equal(findLastHumanTurnIndex(turns), -1);
+ assert.equal(findLastHumanTurnIndex([]), -1);
+});
diff --git a/examples/codex-memory-plugin/scripts/session-start-commit.mjs b/examples/codex-memory-plugin/scripts/session-start-commit.mjs
index 111ef731b..70b774334 100644
--- a/examples/codex-memory-plugin/scripts/session-start-commit.mjs
+++ b/examples/codex-memory-plugin/scripts/session-start-commit.mjs
@@ -21,17 +21,24 @@
* - 1 recently-active → commit it (the just-ended session)
* - ≥2 recently-active → skip; rely on idle TTL
* "Recently-active" means lastUpdatedAt within ACTIVE_WINDOW_MS (default 2 min).
+ * Concurrency is judged on activity alone; only a state that still has a live
+ * ovSessionId is actually committed.
*
* At the tail (regardless of which branch above ran), run an idle-TTL sweep:
- * any state file (including the new session_id, but in practice it's just
- * been created and is fresh) older than IDLE_TTL_MS (default 30 min) gets
- * committed and cleared. This catches SIGTERM/Ctrl+C/`/exit` exits and
- * crashes that left state files orphaned.
+ * any live OV session state older than IDLE_TTL_MS (default 30 min) gets
+ * committed while retaining its transcript cursor. This catches
+ * SIGTERM/Ctrl+C/`/exit` exits and crashes that left sessions orphaned.
+ *
+ * The same pass retires cursor-only states (no live OV session): a cursor
+ * that was never used is dropped after IDLE_TTL_MS, and a real cursor is
+ * kept for resume until COMMITTED_TTL_MS (default 30 days). Without this the
+ * state directory grows one file per codex session forever and listStates()
+ * reads all of them on every SessionStart.
*
* Commit failure handling:
- * On any /commit failure (OV unreachable, non-2xx, timeout) we DO NOT call
- * clearState — we keep the state file with ovSessionId still set so the
- * next sweep retries. A transient OV outage shouldn't lose memory.
+ * On any /commit failure (OV unreachable, non-2xx, timeout) we keep the state
+ * file with ovSessionId still set so the next sweep retries. A transient OV
+ * outage shouldn't lose memory.
*
* Output may contain hookSpecificOutput.additionalContext for profile/archive
* injection and systemMessage for commit status at the same time.
@@ -58,6 +65,11 @@ const IDLE_TTL_MS = (() => {
return Number.isFinite(v) && v > 0 ? Math.floor(v) : 1_800_000;
})();
+const COMMITTED_TTL_MS = (() => {
+ const v = Number(process.env.OPENVIKING_CODEX_COMMITTED_TTL_MS);
+ return Number.isFinite(v) && v > 0 ? Math.floor(v) : 2_592_000_000;
+})();
+
function output(obj) {
process.stdout.write(JSON.stringify(obj) + "\n");
}
@@ -232,59 +244,69 @@ async function buildResumeArchiveContext(newSessionId) {
}
/**
- * Commit and clear a single state file. On commit failure, preserve state
- * (don't call clearState) so the next sweep retries.
+ * Commit a live OV session and release it, keeping the transcript cursor so a
+ * later resume appends instead of replaying (DESIGN.md "Commit-then-resume").
+ * On commit failure, keep the live session id so the next sweep retries.
*
- * Returns { committed: bool, ovSessionId: string|null }.
+ * Returns { committed: bool, ovSessionId: string|null, traceId: string }.
*/
-async function commitAndClear(state, reason) {
- if (state.ovSessionId) {
- const ovSessionId = state.ovSessionId;
- const commit = await commitOvSession(state.ovSessionId);
- if (!commit?.ok) {
- log("commit", {
- reason,
- codexSessionId: state.codexSessionId,
- ovSessionId: state.ovSessionId,
- ok: false,
- status: commit?.status,
- trace_id: commit?.traceId,
- error: commit?.error?.message || commit?.error?.code,
- });
- return { committed: false, ovSessionId: null, traceId: commit?.traceId || "" };
- }
- const traceId = commit.traceId || commit.result?.trace_id || "";
+async function commitAndRelease(state, reason) {
+ const ovSessionId = state.ovSessionId;
+ const commit = await commitOvSession(ovSessionId);
+ if (!commit?.ok) {
log("commit", {
reason,
codexSessionId: state.codexSessionId,
ovSessionId,
- archived: commit.result?.archived ?? false,
- taskId: commit.result?.task_id,
- status: commit.result?.status,
- trace_id: traceId || undefined,
+ ok: false,
+ status: commit?.status,
+ trace_id: commit?.traceId,
+ error: commit?.error?.message || commit?.error?.code,
});
- await clearState(state.codexSessionId);
- return { committed: true, ovSessionId, traceId };
+ return { committed: false, ovSessionId: null, traceId: commit?.traceId || "" };
}
- // No OV session attached — nothing to commit on the server, but the local
- // state file is still stale and should be removed.
- log("clear_no_ov", { reason, codexSessionId: state.codexSessionId });
+ const traceId = commit.traceId || commit.result?.trace_id || "";
+ log("commit", {
+ reason,
+ codexSessionId: state.codexSessionId,
+ ovSessionId,
+ archived: commit.result?.archived ?? false,
+ taskId: commit.result?.task_id,
+ status: commit.result?.status,
+ trace_id: traceId || undefined,
+ });
+ state.ovSessionId = null;
+ await saveState(state, { touch: false });
+ return { committed: true, ovSessionId, traceId };
+}
+
+/**
+ * Retire a state file with no live OV session. A cursor that never captured
+ * anything carries no resume value, so it goes on the idle schedule; a real
+ * cursor is kept until COMMITTED_TTL_MS in case the codex session is resumed.
+ */
+async function maybeRetireCursorState(state, ageMs) {
+ const hasCursor = Number(state.capturedTurnCount) > 0;
+ const ttl = hasCursor ? COMMITTED_TTL_MS : IDLE_TTL_MS;
+ if (ageMs <= ttl) return false;
+ log("state_retire", {
+ codexSessionId: state.codexSessionId,
+ capturedTurnCount: state.capturedTurnCount,
+ ageMs,
+ ttlMs: ttl,
+ });
await clearState(state.codexSessionId);
- return { committed: true, ovSessionId: null, traceId: "" };
+ return true;
}
function describeCommittedSessions(commits) {
+ const traceIds = commits.map((item) => item.traceId).filter(Boolean);
if (commits.length === 1) {
return `OpenViking session ${commits[0].ovSessionId} is committed` +
- (commits[0].traceId ? ` (trace_id=${commits[0].traceId})` : "");
+ (traceIds.length ? ` (trace_id=${traceIds[0]})` : "");
}
- const ovSessionIds = commits.map((item) => item.ovSessionId);
- if (ovSessionIds.length > 1) {
- const traceIds = commits.map((item) => item.traceId).filter(Boolean);
- return `OpenViking sessions ${ovSessionIds.join(", ")} are committed` +
- (traceIds.length ? ` (trace_ids=${traceIds.join(",")})` : "");
- }
- return "OpenViking session state is cleared";
+ return `OpenViking sessions ${commits.map((item) => item.ovSessionId).join(", ")} are committed` +
+ (traceIds.length ? ` (trace_ids=${traceIds.join(",")})` : "");
}
async function main() {
@@ -367,12 +389,15 @@ async function main() {
(s) => s?.codexSessionId && s.codexSessionId !== newSessionId,
);
+ // Concurrency is judged on activity, not on whether an OV session is live:
+ // a session that PreCompact just committed is cursor-only yet may still be
+ // running, and counting it keeps the ≥2 branch from committing a sibling
+ // session mid-flight.
const recentlyActive = otherStates.filter(
(s) => typeof s.lastUpdatedAt === "number"
&& (now - s.lastUpdatedAt) <= ACTIVE_WINDOW_MS,
);
- let heuristicCommitted = 0;
const heuristicCommits = [];
const skippedSessionIds = new Set();
@@ -380,16 +405,22 @@ async function main() {
log("heuristic", { branch: "0_active", action: "noop", otherStates: otherStates.length });
} else if (recentlyActive.length === 1) {
const target = recentlyActive[0];
- log("heuristic", {
- branch: "1_active",
- action: "commit",
- codexSessionId: target.codexSessionId,
- ovSessionId: target.ovSessionId,
- });
- const r = await commitAndClear(target, "heuristic_1_active");
- if (r.committed) {
- heuristicCommitted += 1;
- if (r.ovSessionId) heuristicCommits.push(r);
+ if (!target.ovSessionId) {
+ log("heuristic", {
+ branch: "1_active",
+ action: "skip",
+ reason: "no live OV session",
+ codexSessionId: target.codexSessionId,
+ });
+ } else {
+ log("heuristic", {
+ branch: "1_active",
+ action: "commit",
+ codexSessionId: target.codexSessionId,
+ ovSessionId: target.ovSessionId,
+ });
+ const r = await commitAndRelease(target, "heuristic_1_active");
+ if (r.committed) heuristicCommits.push(r);
}
} else {
log("heuristic", {
@@ -402,44 +433,46 @@ async function main() {
}
// -------------------------------------------------------------------------
- // Idle TTL sweep (tail) — applies to ALL state files including ones we just
- // skipped above (≥2 active path). We re-list because the heuristic branch
- // may have removed entries.
+ // Idle TTL sweep + cursor retention (tail) — applies to ALL state files
+ // including ones we just skipped above (≥2 active path). We re-list because
+ // the heuristic branch may have changed entries.
// -------------------------------------------------------------------------
const postHeuristic = await listStates();
- let idleCommitted = 0;
const idleCommits = [];
+ let retired = 0;
for (const s of postHeuristic) {
if (!s?.codexSessionId) continue;
if (typeof s.lastUpdatedAt !== "number") continue;
- if ((now - s.lastUpdatedAt) <= IDLE_TTL_MS) continue;
+ const ageMs = now - s.lastUpdatedAt;
+ if (!s.ovSessionId) {
+ if (await maybeRetireCursorState(s, ageMs)) retired += 1;
+ continue;
+ }
+ if (ageMs <= IDLE_TTL_MS) continue;
log("idle_sweep", {
codexSessionId: s.codexSessionId,
ovSessionId: s.ovSessionId,
- ageMs: now - s.lastUpdatedAt,
+ ageMs,
});
- const r = await commitAndClear(s, "idle_ttl");
- if (r.committed) {
- idleCommitted += 1;
- if (r.ovSessionId) idleCommits.push(r);
- }
+ const r = await commitAndRelease(s, "idle_ttl");
+ if (r.committed) idleCommits.push(r);
}
- const totalCommitted = heuristicCommitted + idleCommitted;
const commits = [...heuristicCommits, ...idleCommits];
const ovSessionIds = commits.map((item) => item.ovSessionId);
log("done", {
source,
- heuristicCommitted,
- idleCommitted,
- totalCommitted,
+ heuristicCommitted: heuristicCommits.length,
+ idleCommitted: idleCommits.length,
+ totalCommitted: commits.length,
+ retired,
ovSessionIds,
skipped: [...skippedSessionIds],
});
- if (totalCommitted > 0) {
+ if (commits.length > 0) {
emitSessionStartOutput({
contexts: [profileContext],
systemMessage: describeCommittedSessions(commits),
diff --git a/examples/codex-memory-plugin/scripts/session-start-commit.test.mjs b/examples/codex-memory-plugin/scripts/session-start-commit.test.mjs
index 2ae20f475..81567f073 100644
--- a/examples/codex-memory-plugin/scripts/session-start-commit.test.mjs
+++ b/examples/codex-memory-plugin/scripts/session-start-commit.test.mjs
@@ -1,6 +1,6 @@
import assert from "node:assert/strict";
import { spawn } from "node:child_process";
-import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
+import { mkdir, mkdtemp, readFile, readdir, rm, writeFile } from "node:fs/promises";
import http from "node:http";
import { tmpdir } from "node:os";
import { dirname, join } from "node:path";
@@ -224,6 +224,149 @@ test("startup preserves the existing commit systemMessage alongside profile cont
request.method === "POST"
&& request.path === "/api/v1/sessions/cx-old-session/commit"
));
+ const state = JSON.parse(await readFile(join(stateDir, "old-session.json"), "utf-8"));
+ assert.equal(state.ovSessionId, null);
+ assert.equal(state.capturedTurnCount, 2);
+ } finally {
+ await rm(stateDir, { recursive: true, force: true });
+ }
+});
+
+test("startup ignores committed cursor-only states", async () => {
+ const stateDir = await mkdtemp(join(tmpdir(), "ov-codex-cursor-only-"));
+ const requests = [];
+ try {
+ const now = Date.now();
+ await Promise.all([
+ writeFile(join(stateDir, "recent.json"), JSON.stringify({
+ codexSessionId: "recent",
+ ovSessionId: null,
+ capturedTurnCount: 4,
+ createdAt: now - 500,
+ lastUpdatedAt: now,
+ })),
+ writeFile(join(stateDir, "stale.json"), JSON.stringify({
+ codexSessionId: "stale",
+ ovSessionId: null,
+ capturedTurnCount: 6,
+ createdAt: now - 20_000,
+ lastUpdatedAt: now - 10_000,
+ })),
+ ]);
+
+ await withMockOpenViking(profileHandler(requests), async (baseUrl) => {
+ await runSessionStart(
+ { session_id: "new-session", source: "startup", cwd: "/tmp/codex-cursor-only" },
+ {
+ ...baseEnv(baseUrl, stateDir),
+ OPENVIKING_CODEX_ACTIVE_WINDOW_MS: "1000",
+ OPENVIKING_CODEX_IDLE_TTL_MS: "5000",
+ },
+ );
+ });
+
+ const [recent, stale] = await Promise.all([
+ readFile(join(stateDir, "recent.json"), "utf-8"),
+ readFile(join(stateDir, "stale.json"), "utf-8"),
+ ]);
+ assert.equal(JSON.parse(recent).capturedTurnCount, 4);
+ assert.equal(JSON.parse(stale).capturedTurnCount, 6);
+ assert.equal(requests.some((request) => request.path.endsWith("/commit")), false);
+ } finally {
+ await rm(stateDir, { recursive: true, force: true });
+ }
+});
+
+test("startup retires cursor-only states once their retention window closes", async () => {
+ const stateDir = await mkdtemp(join(tmpdir(), "ov-codex-cursor-retention-"));
+ const requests = [];
+ try {
+ const now = Date.now();
+ await Promise.all([
+ writeFile(join(stateDir, "expired.json"), JSON.stringify({
+ codexSessionId: "expired",
+ ovSessionId: null,
+ capturedTurnCount: 6,
+ createdAt: now - 20_000,
+ lastUpdatedAt: now - 10_000,
+ })),
+ writeFile(join(stateDir, "keeper.json"), JSON.stringify({
+ codexSessionId: "keeper",
+ ovSessionId: null,
+ capturedTurnCount: 4,
+ createdAt: now - 8_000,
+ lastUpdatedAt: now - 6_000,
+ })),
+ writeFile(join(stateDir, "empty.json"), JSON.stringify({
+ codexSessionId: "empty",
+ ovSessionId: null,
+ capturedTurnCount: 0,
+ createdAt: now - 8_000,
+ lastUpdatedAt: now - 6_000,
+ })),
+ ]);
+
+ await withMockOpenViking(profileHandler(requests), async (baseUrl) => {
+ await runSessionStart(
+ { session_id: "new-session", source: "startup", cwd: "/tmp/codex-cursor-retention" },
+ {
+ ...baseEnv(baseUrl, stateDir),
+ OPENVIKING_CODEX_ACTIVE_WINDOW_MS: "500",
+ OPENVIKING_CODEX_IDLE_TTL_MS: "5000",
+ OPENVIKING_CODEX_COMMITTED_TTL_MS: "8000",
+ },
+ );
+ });
+
+ const files = await readdir(stateDir);
+ assert.equal(files.includes("expired.json"), false);
+ assert.equal(files.includes("empty.json"), false);
+ assert.equal(JSON.parse(await readFile(join(stateDir, "keeper.json"), "utf-8")).capturedTurnCount, 4);
+ assert.equal(requests.some((request) => request.path.endsWith("/commit")), false);
+ } finally {
+ await rm(stateDir, { recursive: true, force: true });
+ }
+});
+
+test("startup defers to idle TTL when a cursor-only session is still active", async () => {
+ const stateDir = await mkdtemp(join(tmpdir(), "ov-codex-active-cursor-"));
+ const requests = [];
+ try {
+ const now = Date.now();
+ await Promise.all([
+ writeFile(join(stateDir, "live.json"), JSON.stringify({
+ codexSessionId: "live",
+ ovSessionId: "cx-live",
+ capturedTurnCount: 2,
+ createdAt: now - 2_000,
+ lastUpdatedAt: now - 100,
+ })),
+ // Just committed by PreCompact, so cursor-only — but still running.
+ writeFile(join(stateDir, "compacted.json"), JSON.stringify({
+ codexSessionId: "compacted",
+ ovSessionId: null,
+ capturedTurnCount: 5,
+ createdAt: now - 2_000,
+ lastUpdatedAt: now - 100,
+ })),
+ ]);
+
+ await withMockOpenViking(profileHandler(requests), async (baseUrl) => {
+ const { output } = await runSessionStart(
+ { session_id: "new-session", source: "startup", cwd: "/tmp/codex-active-cursor" },
+ {
+ ...baseEnv(baseUrl, stateDir),
+ OPENVIKING_CODEX_ACTIVE_WINDOW_MS: "60000",
+ OPENVIKING_CODEX_IDLE_TTL_MS: "1800000",
+ },
+ );
+ assert.equal(output.systemMessage, undefined);
+ });
+
+ assert.equal(requests.some((request) => request.path.endsWith("/commit")), false);
+ const live = JSON.parse(await readFile(join(stateDir, "live.json"), "utf-8"));
+ assert.equal(live.ovSessionId, "cx-live");
+ assert.equal(JSON.parse(await readFile(join(stateDir, "compacted.json"), "utf-8")).capturedTurnCount, 5);
} finally {
await rm(stateDir, { recursive: true, force: true });
}
diff --git a/examples/codex-memory-plugin/scripts/session-state.mjs b/examples/codex-memory-plugin/scripts/session-state.mjs
index 9837c4768..def36b832 100644
--- a/examples/codex-memory-plugin/scripts/session-state.mjs
+++ b/examples/codex-memory-plugin/scripts/session-state.mjs
@@ -64,10 +64,21 @@ export async function loadState(codexSessionId) {
}
}
-export async function saveState(state) {
+/**
+ * Persist state. `touch: false` keeps the existing `lastUpdatedAt` so a write
+ * that isn't transcript activity (e.g. releasing `ovSessionId` after a commit)
+ * doesn't make a dead session look freshly used to the active-window heuristic
+ * and the retention sweep.
+ */
+export async function saveState(state, { touch = true } = {}) {
if (!state || !state.codexSessionId) return;
await mkdir(getStateDir(), { recursive: true });
- const next = { ...state, lastUpdatedAt: Date.now() };
+ const next = {
+ ...state,
+ lastUpdatedAt: touch || typeof state.lastUpdatedAt !== "number"
+ ? Date.now()
+ : state.lastUpdatedAt,
+ };
// Atomic write (tmpfile + rename) so a crash mid-write can't leave a
// truncated/corrupt state file. See DESIGN.md "State file schema".
const final = statePath(state.codexSessionId);