Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.
Python agent-contextagent-memoryagent-pluginsagentic-ragcontext-databasedsh-pluginknowledge-basememorymemory-managementself-evolvingvector-database
fix(codex): derive ov session ids from codex sessions (#2039)
Zayn Jarvisace6efcbe2ea
diff --git a/examples/codex-memory-plugin/DESIGN.md b/examples/codex-memory-plugin/DESIGN.mdindex 6ba56ebc2..a1fa36c74 100644--- a/examples/codex-memory-plugin/DESIGN.md+++ b/examples/codex-memory-plugin/DESIGN.md@@ -11,9 +11,11 @@ events imply "context for a particular codex `session_id` is gone".- **codex `session_id`** — the codex thread/session id. Stable acrossprocess restarts when zouk-daemon resumes the same thread; replaced when`/clear`, `/new`, fresh codex startup, or zouk reset occurs.-- **OV session** — `viking://session/<uuid>`. We open one per codex- `session_id`, append messages on every `Stop`, and commit it (which- triggers OV's memory extractor) at session-end-equivalent moments.+- **OV session** — `viking://session/cx-<codex-session-id>`. New captures+ derive the OV session id from the codex `session_id` with a `cx-` prefix,+ append messages on every `Stop`, and commit it (which triggers OV's+ memory extractor) at session-end-equivalent moments. `/messages`+ auto-creates the OV session, so the plugin does not call session create.- **State file** — `~/.openviking/codex-plugin-state/<safe-codex-session-id>.json`,shape `{ codexSessionId, ovSessionId, capturedTurnCount, createdAt, lastUpdatedAt }`.- **Active window** — state files whose `lastUpdatedAt` is within@@ -47,10 +49,10 @@ or append-only.Codex fires `PreCompact` before summarizing. We catch up with anyunappended turns from the transcript, commit the OV session for this codex-`session_id`, and clear `ovSessionId` so the next `Stop` opens a fresh OV-session for the post-compact half. `capturedTurnCount` is preserved unless-the transcript was truncated by compaction (see "Post-compact transcript-shrink" below).+`session_id`, and clear `ovSessionId` so the next `Stop` re-derives the+same `cx-<codex-session-id>` OV session id for the post-compact half.+`capturedTurnCount` is preserved unless the transcript was truncated by+compaction (see "Post-compact transcript shrink" below).### 2. `SessionStart` source=`clear` — heuristic, same shape as `startup`@@ -98,9 +100,9 @@ Short reconnects and `/resume` re-fire `SessionStart` for the sameState files whose `lastUpdatedAt` is older than `IDLE_TTL_MS` (default 30min) get committed and cleared. Mental model: a session not touched for-30 min is "temporarily concluded"; if the user resumes later, they get a-fresh OV session for the new turns (memory will be split, but each chunk-gets extracted).+30 min is "temporarily concluded"; if the user resumes later, subsequent+turns append under the same deterministic OV session id, and the next+commit creates another archive there.This covers:- SIGTERM / Ctrl+C / `/exit` (no hook fires; state file rots)@@ -125,7 +127,8 @@ forever. Accepted. Future work could add an MCP toolEvery `Stop` reads `transcript_path`, slices to `[capturedTurnCount, end)`,and appends each new user/assistant turn to the OV session for this codex-`session_id` (creating one on first append). State is updated:+`session_id` (the `/messages` endpoint auto-creates it on first append).+State is updated:`{ovSessionId, capturedTurnCount, lastUpdatedAt: now}`. Never commits.## Edge cases handled@@ -155,24 +158,28 @@ we can't commit it deterministically — idle TTL will catch it later.### Commit-then-resume-After PreCompact (or idle sweep, or rule-3 commit) we set `ovSessionId =-null` but keep `capturedTurnCount`. The next `Stop` for the same codex-`session_id` opens a fresh OV session and starts appending from-`capturedTurnCount`. Memory ends up split across two OV sessions; each-gets extracted independently. Acceptable.+After PreCompact we set `ovSessionId = null` but keep+`capturedTurnCount`. The next `Stop` for the same codex `session_id`+re-derives the same `cx-<codex-session-id>` OV session id and starts+appending from `capturedTurnCount`. Memory remains grouped under the same+OV session id, while commits create additional archives under that session.## State file schema```json{"codexSessionId": "0193af...", // codex thread id- "ovSessionId": "uuid-or-null", // null means "committed, awaiting next Stop"+ "ovSessionId": "cx-0193af...-or-null", // null means "committed, awaiting next Stop""capturedTurnCount": 7, // turns from transcript already appended"createdAt": 1715000000000,"lastUpdatedAt": 1715000300000}```+Legacy state files from earlier plugin versions may still contain a UUID+`ovSessionId`; the hook preserves that value until the next commit so+already-captured turns are not orphaned.+State files are atomic-write (tmpfile + rename) to survive crash mid-write.## Configuration@@ -196,7 +203,7 @@ continuity unless the most recent committed memories are surfaced.Proposed flow:1. Load state for the resumed `session_id`. If `ovSessionId` is non-null,no action — the session is still appendable.-2. Otherwise list `viking://session/<codex-session-id>/history/archive_*/`+2. Otherwise list `viking://session/cx-<codex-session-id>/history/archive_*/`on the OV server, take the most recent.3. Read its abstract (L0) / overview (L1).4. Emit via `hookSpecificOutput.additionalContext` so codex injects thediff --git a/examples/codex-memory-plugin/README.md b/examples/codex-memory-plugin/README.mdindex faff3b6a4..b3585e09c 100644--- a/examples/codex-memory-plugin/README.md+++ b/examples/codex-memory-plugin/README.md@@ -5,7 +5,7 @@ Long-term semantic memory for [Codex](https://developers.openai.com/codex), poweThis is the Codex counterpart to [`claude-code-memory-plugin`](../claude-code-memory-plugin). It hooks Codex's lifecycle to:- **Auto-recall** relevant memories on every `UserPromptSubmit` and inject them via `hookSpecificOutput.additionalContext`-- **Incremental capture on `Stop`** (turn end): append the new user/assistant turns to a single long-lived OpenViking session keyed by Codex `session_id`. No commit per turn.+- **Incremental capture on `Stop`** (turn end): append the new user/assistant turns to a deterministic OpenViking session id `cx-<codex_session_id>`. No commit per turn.- **Commit on `PreCompact`**: trigger OpenViking's memory extractor on the full pre-compact transcript before Codex summarizes it.- **Commit on `SessionStart` (source=startup|clear)**: active-window heuristic — if exactly one *other* state file was touched within the last 2 min, commit it (the just-ended session). On `≥2`, defer to idle-TTL sweep at the tail. `source=resume` is a hard no-op (short reconnects re-fire `resume` and we don't want to commit a still-active session). See `DESIGN.md` for the full decision tree.@@ -180,7 +180,7 @@ Codex injects `additionalContext` into the model turn, so memories arrive withou### Stop (turn end → `add_message`, NOT `commit`)-`auto-capture.mjs` keeps one long-lived OpenViking session per Codex `session_id` and incrementally appends every new user/assistant turn via `/api/v1/sessions/{id}/messages`. Per-codex-session state lives at `~/.openviking/codex-plugin-state/<safe-session-id>.json`. No `/commit` per turn — that would over-fragment memory extraction.+`auto-capture.mjs` derives one long-lived OpenViking session id per Codex `session_id` as `cx-<safe-session-id>` and incrementally appends every new user/assistant turn via `/api/v1/sessions/{id}/messages`. The `/messages` endpoint auto-creates the session on first append. Per-codex-session state lives at `~/.openviking/codex-plugin-state/<safe-session-id>.json`. No `/commit` per turn — that would over-fragment memory extraction.### PreCompact (deterministic commit)@@ -188,7 +188,7 @@ Codex injects `additionalContext` into the model turn, so memories arrive withou1. Catch-up append for any turns Stop hasn't captured yet (race-safe via `capturedTurnCount`)2. Commit the long-lived OV session so the extractor runs against the full pre-compact transcript-3. Reset state so the next `Stop` opens a fresh OV session for the post-compact half+3. Reset `ovSessionId` to `null` so the next `Stop` re-derives the same `cx-<safe-session-id>` and appends the post-compact half under that deterministic OV session id### Known gap: SIGTERM / Ctrl+C / `/exit` are silentdiff --git a/examples/codex-memory-plugin/VERIFICATION.md b/examples/codex-memory-plugin/VERIFICATION.mdindex 2b0fda05a..da73dd912 100644--- a/examples/codex-memory-plugin/VERIFICATION.md+++ b/examples/codex-memory-plugin/VERIFICATION.md@@ -34,17 +34,17 @@ echo '{"session_id":"verify-sess","transcript_path":"'"$STATE_DIR"'/transcript.j| node $PLUGIN/scripts/auto-capture.mjs```-Expect: `{"systemMessage":"appended 2 turn(s) to OpenViking session <UUID>"}`.+Expect: `{"systemMessage":"appended 2 turn(s) to OpenViking session cx-verify-sess"}`.State file:```bashcat $STATE_DIR/state/verify-sess.json-# {"codexSessionId":"verify-sess","ovSessionId":"<UUID>","capturedTurnCount":2,...}+# {"codexSessionId":"verify-sess","ovSessionId":"cx-verify-sess","capturedTurnCount":2,...}```OV side:```bash-OPENVIKING_CONFIG_FILE=$OV_CONF ov read viking://session/<UUID>/messages.jsonl+OPENVIKING_CONFIG_FILE=$OV_CONF ov read viking://session/cx-verify-sess/messages.jsonl# 2 JSONL records: user "fuchsia", assistant "noted"```@@ -78,7 +78,7 @@ echo '{"session_id":"verify-sess","transcript_path":"'"$STATE_DIR"'/transcript.j```Expect: `appended 2 turn(s)` (only the new ones). Re-read-`viking://session/<UUID>/messages.jsonl` — 4 records now.+`viking://session/cx-verify-sess/messages.jsonl` — 4 records now.## 4. PreCompact — commit + reset@@ -90,21 +90,21 @@ echo '{"session_id":"verify-sess","transcript_path":"'"$STATE_DIR"'/transcript.jnode $PLUGIN/scripts/pre-compact-capture.mjs```-Expect: `OpenViking session <UUID> is committed`.+Expect: `OpenViking session cx-verify-sess is committed`.State file: `ovSessionId` is now `null`, `capturedTurnCount` stays at 4.OV side:```bash-OPENVIKING_CONFIG_FILE=$OV_CONF ov ls viking://session/<UUID>+OPENVIKING_CONFIG_FILE=$OV_CONF ov ls viking://session/cx-verify-sess# messages.jsonl is now size 0 (archived)# history/archive_001/ exists with the committed messages-OPENVIKING_CONFIG_FILE=$OV_CONF ov read viking://session/<UUID>/history/archive_001/messages.jsonl+OPENVIKING_CONFIG_FILE=$OV_CONF ov read viking://session/cx-verify-sess/history/archive_001/messages.jsonl```-## 5. Post-compact Stop — fresh OV session+## 5. Post-compact Stop — same deterministic OV session id-Append more turns and run Stop. A new OV session UUID should appear:+Append more turns and run Stop. The same OV session id should appear:```bashcat >> "$STATE_DIR/transcript.jsonl" <<'EOF'@@ -119,8 +119,7 @@ echo '{"session_id":"verify-sess","transcript_path":"'"$STATE_DIR"'/transcript.jnode $PLUGIN/scripts/auto-capture.mjs```-Expect: `appended 2 turn(s) to OpenViking session <NEW_UUID>` — different-from step 4's UUID.+Expect: `appended 2 turn(s) to OpenViking session cx-verify-sess`.## 6. SessionStart — active-window heuristic + idle-TTL sweep@@ -142,7 +141,7 @@ echo '{"session_id":"new-after-verify","source":"startup","cwd":"/tmp","model":"node $PLUGIN/scripts/session-start-commit.mjs```-Expect: `OpenViking session <UUID> is committed`.+Expect: `OpenViking session cx-verify-sess is committed`.After this `verify-sess.json` is gone from `$STATE_DIR/state`.### 6b. `0 active` → no-opdiff --git a/examples/codex-memory-plugin/scripts/auto-capture.mjs b/examples/codex-memory-plugin/scripts/auto-capture.mjsindex 3862c0d80..180af46b6 100644--- a/examples/codex-memory-plugin/scripts/auto-capture.mjs+++ b/examples/codex-memory-plugin/scripts/auto-capture.mjs@@ -7,8 +7,9 @@* last_assistant_message. Stop fires per turn — NOT at session end.** Strategy:- * 1. For this codex session_id, lazily create one long-lived OpenViking- * session and remember it in state. Do NOT commit per turn.+ * 1. For this codex session_id, derive one long-lived OpenViking session+ * id (`cx-<codex-session-id>`) and remember it in state. Do NOT commit+ * per turn.* 2. Read transcript_path, parse JSONL rollout, append every new* user/assistant turn since last capture via add_message.*@@ -27,7 +28,7 @@import { readFile } from "node:fs/promises";import { loadConfig } from "./config.mjs";import { createLogger } from "./debug-log.mjs";-import { loadState, saveState } from "./session-state.mjs";+import { loadState, resolveOvSessionId, saveState } from "./session-state.mjs";const cfg = loadConfig();const { log, logError } = createLogger("auto-capture");@@ -137,21 +138,6 @@ async function readTranscriptTurns(transcriptPath) {}}-// ----------------------------------------------------------------------------// OpenViking session ops-// -----------------------------------------------------------------------------async function ensureOvSession(state) {- if (state.ovSessionId) return state.ovSessionId;- const created = await fetchJSON("/api/v1/sessions", {- method: "POST",- body: JSON.stringify({}),- });- if (!created?.session_id) return null;- state.ovSessionId = created.session_id;- return state.ovSessionId;-}-async function appendTurns(ovSessionId, turns) {let appended = 0;for (const turn of turns) {@@ -231,9 +217,9 @@ async function main() {let added = 0;if (newTurns.length > 0) {- const ovSessionId = await ensureOvSession(state);+ const ovSessionId = resolveOvSessionId(state);if (!ovSessionId) {- logError("ensure_ov_session", "failed to create OV session");+ logError("resolve_ov_session", "failed to derive OV session id");} else {added = await appendTurns(ovSessionId, newTurns);state.capturedTurnCount += added;diff --git a/examples/codex-memory-plugin/scripts/pre-compact-capture.mjs b/examples/codex-memory-plugin/scripts/pre-compact-capture.mjsindex f40968773..280a68db7 100644--- a/examples/codex-memory-plugin/scripts/pre-compact-capture.mjs+++ b/examples/codex-memory-plugin/scripts/pre-compact-capture.mjs@@ -13,8 +13,9 @@** After commit, we clear ovSessionId from state but keep* capturedTurnCount so post-compact Stop hooks don't re-capture pre-- * compact turns. The next Stop will create a fresh OV session for the- * post-compact half of the conversation.+ * compact turns. The next Stop will append to the same deterministic+ * `cx-<codex-session-id>` OV session id; `/messages` auto-creates it if+ * needed.** PreCompact output schema accepts {} as a no-op.*/@@ -22,7 +23,7 @@import { readFile } from "node:fs/promises";import { loadConfig } from "./config.mjs";import { createLogger } from "./debug-log.mjs";-import { loadState, saveState } from "./session-state.mjs";+import { loadState, resolveOvSessionId, saveState } from "./session-state.mjs";const cfg = loadConfig();const { log, logError } = createLogger("pre-compact");@@ -128,17 +129,6 @@ async function readTranscriptTurns(transcriptPath) {}}-async function ensureOvSession(state) {- if (state.ovSessionId) return state.ovSessionId;- const created = await fetchJSON("/api/v1/sessions", {- method: "POST",- body: JSON.stringify({}),- });- if (!created?.session_id) return null;- state.ovSessionId = created.session_id;- return state.ovSessionId;-}-async function appendTurns(ovSessionId, turns) {let appended = 0;for (const turn of turns) {@@ -206,9 +196,9 @@ async function main() {}if (newTurns.length > 0) {- const ovSessionId = await ensureOvSession(state);+ const ovSessionId = resolveOvSessionId(state);if (!ovSessionId) {- logError("ensure_ov_session", "failed to create OV session for catch-up");+ logError("resolve_ov_session", "failed to derive OV session id for catch-up");noop();return;}diff --git a/examples/codex-memory-plugin/scripts/session-state.mjs b/examples/codex-memory-plugin/scripts/session-state.mjsindex da4d89fc5..58f8d12ca 100644--- a/examples/codex-memory-plugin/scripts/session-state.mjs+++ b/examples/codex-memory-plugin/scripts/session-state.mjs@@ -2,8 +2,9 @@* Per-codex-session state for the OpenViking memory plugin.** One state file per codex session_id, holding the long-lived OpenViking- * session that we incrementally append turns to via the Stop hook. The- * OV session is committed (which extracts memories) by the PreCompact+ * session id that we incrementally append turns to via the Stop hook. The+ * OV session id is derived as `cx-<codex-session-id>` for new captures.+ * The OV session is committed (which extracts memories) by the PreCompact* hook or by the idle-sweep that runs at the tail of each Stop.** State directory: $OPENVIKING_CODEX_STATE_DIR or ~/.openviking/codex-plugin-state@@ -23,6 +24,18 @@ function safeId(codexSessionId) {return String(codexSessionId).replace(/[^a-zA-Z0-9_-]/g, "_");}+export function deriveOvSessionId(codexSessionId) {+ return `cx-${safeId(codexSessionId || "unknown")}`;+}++export function resolveOvSessionId(state) {+ // Keep legacy persisted UUIDs so already-captured turns are not orphaned+ // before their next commit. Fresh or cleared state derives the cx-* id.+ if (state.ovSessionId) return state.ovSessionId;+ state.ovSessionId = deriveOvSessionId(state.codexSessionId);+ return state.ovSessionId;+}+function statePath(codexSessionId) {return join(getStateDir(), `${safeId(codexSessionId)}.json`);}