Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.
Python agent-contextagent-memoryagent-pluginsagentic-ragcontext-databasedsh-pluginknowledge-basememorymemory-managementself-evolvingvector-database
feat(codex): commit on SessionEnd hook, keep SessionStart sweep as fallback (#4429)
t0saki7200cdb176c3
diff --git a/.github/workflows/pr.yml b/.github/workflows/pr.ymlindex 4dffb1eee..9fed7e2a5 100644--- a/.github/workflows/pr.yml+++ b/.github/workflows/pr.yml@@ -41,6 +41,9 @@ jobs:examples/codex-memory-plugin/scripts/recall-compressor-profile.test.mjs \examples/codex-memory-plugin/scripts/auto-recall.test.mjs \examples/codex-memory-plugin/scripts/session-start-commit.test.mjs \+ examples/codex-memory-plugin/scripts/session-end.test.mjs \+ examples/codex-memory-plugin/scripts/pre-compact-capture.test.mjs \+ examples/codex-memory-plugin/scripts/session-state.test.mjs \examples/claude-code-memory-plugin/scripts/auto-capture.test.mjs \examples/claude-code-memory-plugin/scripts/config.test.mjs \examples/claude-code-memory-plugin/scripts/marketplace.test.mjs \diff --git a/docs/en/agent-integrations/04-codex.md b/docs/en/agent-integrations/04-codex.mdindex 8388e72dc..5f0ccb4ff 100644--- a/docs/en/agent-integrations/04-codex.md+++ b/docs/en/agent-integrations/04-codex.md@@ -57,9 +57,9 @@ For TraeCode CLI 2.0, launch `trae-cli` and use `trae-cli plugin list` to confir## How it works-The plugin integrates with Codex's lifecycle by hooking into key events. On `SessionStart` (`startup`, `clear`, or `resume`), it injects `profile.md` plus URI and abstract indexes for `preferences/` and `entities/` through the same shared, CJK-aware profile builder used by the other coding-agent integrations. It then searches OpenViking and injects relevant memories before every prompt (`UserPromptSubmit`), appends new turns to the session after each response (`Stop`), and commits the full transcript before compaction (`PreCompact`) to ensure memory extraction processes the entire conversation. Upon starting a fresh session, it also cleans up any orphaned sessions from previous runs. A resumed session may combine the fixed profile block with its latest archive digest.+The plugin integrates with Codex's lifecycle by hooking into key events. On `SessionStart` (`startup`, `clear`, or `resume`), it injects `profile.md` plus URI and abstract indexes for `preferences/` and `entities/` through the same shared, CJK-aware profile builder used by the other coding-agent integrations. It then searches OpenViking and injects relevant memories before every prompt (`UserPromptSubmit`), appends new turns to the session after each response (`Stop`), commits the full transcript before compaction (`PreCompact`), and commits the session when the thread shuts down (`SessionEnd`) so memory extraction processes the entire conversation. Upon starting a fresh session, it also sweeps any orphaned sessions left by previous runs. A resumed session may combine the fixed profile block with its latest archive digest.-> **Known limitation**: Codex does not fire a hook upon `SIGTERM`, `Ctrl+C`, or `/exit`. Orphaned sessions are recovered during the next `SessionStart` via the idle-TTL sweep (30 minutes) or the active-window heuristic.+> **Known limitation**: `SessionEnd` requires Codex 0.145 or newer, and it only fires on a graceful exit (`/quit`, `/exit`, double `Ctrl-C`, EOF, end of a `codex exec` run). It does not fire on `SIGTERM`, a closed terminal, `kill -9`, or a crash, and it is deferred when the TUI runs against a `codex app-server` daemon. Those sessions — and every session on Codex older than 0.145, and any TraeCode CLI build without it — are recovered by the idle-TTL sweep (30 minutes) at the next `SessionStart`.Tool calls and results are captured as dedicated `tool` parts, and `tool_output` is reported verbatim. Truncation is the server's job: output larger than `tool_output_externalization.threshold_chars` (default `20000`) is written to the session's tool-result store, and the part keeps a synopsis stub plus `tool_output_ref`, so the original stays readable through [`/api/v1/sessions/{id}/tool-results`](../api/05-sessions.md#read_tool_result).@@ -76,8 +76,8 @@ Credential source: env vars win by default — when any `OPENVIKING_*` credentia| `OPENVIKING_CREDENTIAL_SOURCE` | `auto` | `auto` prefers env-var credentials when any are set; `cli` forces the active ovcli config, `env` forces env vars || `OPENVIKING_NO_AUTO_INJECT` | `false` | Disable fixed session-start profile/background injection without disabling per-prompt recall || `OPENVIKING_PROFILE_TOKEN_BUDGET` | `10000` | CJK-aware token budget for `profile.md` plus `preferences/` and `entities/` indexes |-| `OPENVIKING_CODEX_ACTIVE_WINDOW_MS` | `120000` | SessionStart active-window threshold || `OPENVIKING_CODEX_IDLE_TTL_MS` | `1800000` | SessionStart idle-TTL sweep threshold |+| `OPENVIKING_CODEX_LOCK_WAIT_MS` | `120000` (SessionEnd), `40000` (PreCompact) | How long a capture hook waits for the per-session state lock || `OPENVIKING_CODEX_COMMITTED_TTL_MS` | `2592000000` | How long a committed session's transcript cursor is kept before its state file is retired || `OPENVIKING_DEBUG` | `false` | Write logs to `~/.openviking/logs/codex-hooks.log` |diff --git a/docs/en/agent-integrations/16-capability-reference.md b/docs/en/agent-integrations/16-capability-reference.mdindex 897d118f6..9c635761c 100644--- a/docs/en/agent-integrations/16-capability-reference.md+++ b/docs/en/agent-integrations/16-capability-reference.md@@ -320,7 +320,7 @@ To prevent injected content from being captured a second time, the injection pro| harness | turn-level threshold | explicit / boundary trigger | compaction trigger ||---|---|---|---|| claude-code | Stop: `pending_tokens ≥ 20000` (reads the server value), keep 10 | SessionEnd: unconditional; SubagentStop: unconditional (no threshold); SessionStart: replays pending | PreCompact: unconditional (runs synchronously, no detach) |-| codex / trae-cli | Stop: same as above, 20000 / keep 10 | SessionStart(startup\|clear): an active-window heuristic (exactly 1 state within 2min → commit; ≥2 → skip) + an idle-TTL sweep (>30min always commits and cleans up) | PreCompact: full commit (sends an empty body `{}`), then sets `ovSessionId=null` |+| codex / trae-cli | Stop: same as above, 20000 / keep 10 | SessionEnd (Codex ≥ 0.145): unconditional, after catching up the turns Stop missed — the parent hook writes an `.ended` marker and detaches the worker (Codex budgets it at 1s, clamped to 3s); SessionStart(startup\|clear): a fallback sweep committing states that carry an `.ended` marker or have been idle >30min. a trae-cli build without `SessionEnd` falls back to the sweep and relies on the sweep | PreCompact: full commit (sends an empty body `{}`), then sets `ovSessionId=null` || cursor | stop: `capturedSinceCommit ≥ 8` (counted in messages, ~4 Q&A turns; purely client-side counting), keep 0 | sessionEnd: registered (but never reached in practice, see [§3.3.3](#_3-3-3-shutdown-method-×-harness-outcome-matrix)) | preCompact: unconditional || trae / trae-cn | Every Stop with content commits (no threshold), keep 0 | — | None (no PreCompact event upstream) || zcode | Same as trae (every Stop commits, keep 0; the rollout incremental cursor advances conservatively, so any missed turns are caught up on the next Stop in the same session) | — | None (no PreCompact event upstream) |@@ -341,7 +341,8 @@ Legend: **C** = commits; **C\*** = commits, with a precondition (see notes); **| harness | normal exit | Ctrl+C | SIGTERM | SIGHUP / close terminal / close window·tab | kill -9 / crash | recovery path ||---|---|---|---|---|---|---|| claude-code | **C** (SessionEnd → a detached child process commits, so the user does not wait) | **C** | **C** | **C** (the detached worker forms its own process group and is unaffected by SIGHUP) | **—** | Next Stop over the threshold / `/compact` / next SessionEnd |-| codex / trae-cli | **—** (no SessionEnd-style event upstream) | **—** | **—** | **—** | **—** | The heuristic on the next `SessionStart(startup\|clear)` (exactly 1 active state within 2min → commit) or the 30min idle-TTL sweep; with ≥2 concurrent sessions, the heuristic defers to the TTL sweep |+| codex | **C** (SessionEnd → a detached worker commits, so the user does not wait; Codex ≥ 0.145) | **C\*** | **—** | **—** | **—** | C\* precondition: double `Ctrl-C` is a graceful quit and fires SessionEnd; a single one is not. Anything that does not commit is recovered at the next `SessionStart(startup\|clear)`: `ended_retry` when the marker survives, otherwise the 30min idle-TTL sweep |+| trae-cli | **—** (unless the TraeCode CLI build ships `SessionEnd`) | **—** | **—** | **—** | **—** | The 30min idle-TTL sweep at the next `SessionStart(startup\|clear)` || cursor | **—** (closing a chat or opening a new chat fires no event) | **—** | **—** | **—** (`sessionEnd` is registered and only fires on window_close, but by then the host has destroyed the shell-exec host, causing the hook to abort before spawn) | **—** | A session ending below the 8-message watermark leaves its tail waiting for later messages in the same session to trigger a commit || trae / trae-cn | **—** (no session-end-style event) | **—** | **—** | **—** | **—** | Every Stop has already committed, meaning the most data left to archive equals the last in-flight turn || zcode | **—** (no session-end-style event) | **C\*** | **—** | **—** | **—** | C\* precondition: The Stop for that turn had already fired when Ctrl+C arrived (the detached worker finishes writing as usual); every Stop has already committed, and missed turns are recovered by the rollout cursor on the next Stop in the same session |@@ -498,9 +499,9 @@ Each card serves as a quick-reference entry point. It records only the facts and## codex- **Integration docs**: [Codex Memory Plugin](./04-codex.md)-- **Form**: A Codex plugin (marketplace). Features 4 hooks (`SessionStart` 70s / `UserPromptSubmit` 130s / `Stop` 30s / `PreCompact` 60s), an MCP proxy, and 1 experience skill. Version 0.7.5.-- **Capability highlights**: A local recall-compression pipeline (`codex exec`, [§3.2.5](#_3-2-5-recall-digest)). The `SessionStart` active-window heuristic, combined with an idle-TTL scan, automatically reclaims messages left unarchived by earlier sessions ([§3.3.3](#_3-3-3-shutdown-method-×-harness-outcome-matrix)).-- **Behavior notes**: The session ID format is `cx-<safeId>` (derived deterministically without reading state). Lacks a shutdown hook (upstream provides no `SessionEnd`), so commits rely on the reclaim path during the next startup. Has no on-disk pending queue (while offline, the cursor simply doesn't advance, and the next turn resends to compensate, [§3.3.4](#_3-3-4-pending-queue-offline-compensation-comparison)). Active-window is 120000ms / idle-TTL is 1800000ms (configured via env). `Stop` detaches by default (the `appended N turn(s)` notice is hidden by default).+- **Form**: A Codex plugin (marketplace). Features 5 hooks (`SessionStart` 70s / `UserPromptSubmit` 130s / `Stop` 30s / `SessionEnd` 3s / `PreCompact` 60s), an MCP proxy, and 1 experience skill. Version 0.8.0.+- **Capability highlights**: A local recall-compression pipeline (`codex exec`, [§3.2.5](#_3-2-5-recall-digest)). `SessionEnd` (Codex ≥ 0.145) commits the session on a graceful exit, catching up any turns `Stop` missed, and a `SessionStart` fallback sweep reclaims messages left unarchived by exits that never fired it ([§3.3.3](#_3-3-3-shutdown-method-×-harness-outcome-matrix)).+- **Behavior notes**: The session ID format is `cx-<safeId>` (derived deterministically without reading state). `SessionEnd` fires only on a graceful exit and only on Codex 0.145+, so signals, crashes, older builds and `codex app-server` deferral fall to the sweep. Has no on-disk pending queue (while offline, the cursor simply doesn't advance, and the next turn resends to compensate, [§3.3.4](#_3-3-4-pending-queue-offline-compensation-comparison)). Idle-TTL is 1800000ms, lock wait 120000ms (configured via env). `Stop` and `SessionEnd` detach by default (the `appended N turn(s)` notice is hidden by default); a per-session mkdir lock serializes the Stop worker, PreCompact, the SessionEnd worker and the sweep. A newly registered hook event has no Codex trust record, so `/hooks` must approve `SessionEnd` after an upgrade.- **Configuration**: Configured via environment variables, `ovcli.conf` (`plugin.codex`), and `ov.conf` (`codex`). Hooks transmit both `Bearer` and the `X-API-Key` compatibility headers ([§3.1.3](#_3-1-3-credential-systems)).- **Dimension index**: tool surface [§2.1](#_2-1-server-side-mcp-tool-surface) | recall [§3.2](#_3-2-automatic-recall-and-injection) | commit [§3.3.2](#_3-3-2-regular-commit-triggers)/[§3.3.3](#_3-3-3-shutdown-method-×-harness-outcome-matrix) | degradation [§3.6](#_3-6-degradation-and-fault-tolerance).@@ -508,7 +509,7 @@ Each card serves as a quick-reference entry point. It records only the facts and- **Integration docs**: [TRAE Memory Integration](./13-trae.md)- **Form**: TraeCode CLI 2.0 is a Codex-family CLI (binary `traecli`, user config `~/.trae/traecli.toml`, TUI support for `/plugins`, `/skills`, and `/mcp`). OpenViking integrates via a **codex plugin alias install**: using `--harness trae-cli` reuses the Codex installation flow, redirecting only the install parameters (binary, home, and config paths) to TraeCode CLI.-- **Capability surface**: Identical to Codex: 4 hooks, an MCP proxy, an experience skill, local recall compression, active-window/idle-TTL commit reclamation, and resume-archive injection. Refer to the Codex profile card.+- **Capability surface**: Same plugin as Codex: 5 registered hooks, an MCP proxy, an experience skill, local recall compression, idle-TTL commit reclamation, and resume-archive injection. Refer to the Codex profile card. If the TraeCode CLI build lacks `SessionEnd`, the registration is ignored and every commit at shutdown goes through the idle-TTL sweep.- **Version support**: TraeCode CLI 2.0 only. 1.0 and 2.0 are not the same CLI — only 2.0 is Codex-family, and only 2.0 can use the codex plugin alias install. The earlier standalone plugin for 1.0, `examples/trae-cli-memory-hooks` (the `~/.trae/cli/hooks.json` + `[mcp_servers."openviking-memory"]` approach), is deprecated.- **Dimension index**: Same as the Codex card.diff --git a/docs/zh/agent-integrations/04-codex.md b/docs/zh/agent-integrations/04-codex.mdindex a5e07f886..10b02da8b 100644--- a/docs/zh/agent-integrations/04-codex.md+++ b/docs/zh/agent-integrations/04-codex.md@@ -56,9 +56,9 @@ TraeCode CLI 2.0 用户启动 `trae-cli`,并可用 `trae-cli plugin list` 确## 工作原理-本插件深度挂载于 Codex 的生命周期之中:在 `SessionStart`(`startup`、`clear` 或 `resume`)阶段,它会复用其他 coding-agent 集成共用的 CJK-aware profile 构建逻辑,注入 `profile.md`,以及 `preferences/`、`entities/` 的 URI 和摘要索引;在每次用户输入前,它会搜索 OpenViking 并注入相关的记忆(触发 `UserPromptSubmit`);在每轮对话结束后,会将新的对话追加至当前会话(触发 `Stop`);在上下文压缩前,补齐并提交(commit)完整的对话记录(触发 `PreCompact`),以确保记忆抽取器能够在完整的上下文环境中运行。此外,在启动新会话时,插件还会自动清理前次运行遗留的孤儿会话(orphan session)。恢复已有会话时,固定 profile 背景还会与最新的 archive digest 合并注入。+本插件深度挂载于 Codex 的生命周期之中:在 `SessionStart`(`startup`、`clear` 或 `resume`)阶段,它会复用其他 coding-agent 集成共用的 CJK-aware profile 构建逻辑,注入 `profile.md`,以及 `preferences/`、`entities/` 的 URI 和摘要索引;在每次用户输入前,它会搜索 OpenViking 并注入相关的记忆(触发 `UserPromptSubmit`);在每轮对话结束后,会将新的对话追加至当前会话(触发 `Stop`);在上下文压缩前,补齐并提交(commit)完整的对话记录(触发 `PreCompact`);在线程正常退出时提交整段会话(触发 `SessionEnd`),以确保记忆抽取器能够在完整的上下文环境中运行。此外,在启动新会话时,插件还会清扫前次运行遗留的孤儿会话(orphan session)。恢复已有会话时,固定 profile 背景还会与最新的 archive digest 合并注入。-> **已知局限**:当通过 `SIGTERM`、`Ctrl+C` 或输入 `/exit` 退出 Codex 时,不会触发任何 hook(钩子)。遗留的孤儿会话将在下一次触发 `SessionStart` 时,通过闲置 TTL(生存时间,默认为 30 分钟)机制或活动窗口启发式策略进行回收清理。+> **已知局限**:`SessionEnd` 需要 Codex 0.145 及以上版本,且只在正常退出时触发(`/quit`、`/exit`、连按两次 `Ctrl-C`、EOF、`codex exec` 运行结束)。`SIGTERM`、直接关闭终端、`kill -9` 或崩溃都不会触发;当 TUI 挂在 `codex app-server` 守护进程上时,该事件会被延后。这些会话——以及 Codex 低于 0.145 的所有会话(以及没有该事件的 TraeCode CLI 版本)——由下一次 `SessionStart` 的闲置 TTL(生存时间,默认为 30 分钟)清扫回收。工具调用和结果会作为独立的 `tool` part 捕获,`tool_output` 原样上报。截断由服务端负责:超过 `tool_output_externalization.threshold_chars`(默认 `20000`)的输出会写入 session 的 tool-result 存储,part 中只保留 synopsis stub 和 `tool_output_ref`,原文仍可通过 [`/api/v1/sessions/{id}/tool-results`](../api/05-sessions.md#read_tool_result) 读回。@@ -75,8 +75,8 @@ TraeCode CLI 2.0 用户启动 `trae-cli`,并可用 `trae-cli plugin list` 确| `OPENVIKING_CREDENTIAL_SOURCE` | `auto` | `auto` 下已设置的凭据环境变量优先;设为 `cli` 强制使用激活的 ovcli 配置,设为 `env` 强制使用环境变量 || `OPENVIKING_NO_AUTO_INJECT` | `false` | 关闭会话启动阶段的固定 profile/背景注入,但不关闭逐 prompt 语义召回 || `OPENVIKING_PROFILE_TOKEN_BUDGET` | `10000` | `profile.md` 及 `preferences/`、`entities/` 索引共用的 CJK-aware token 预算 |-| `OPENVIKING_CODEX_ACTIVE_WINDOW_MS` | `120000` | `SessionStart` 活动窗口阈值(毫秒) || `OPENVIKING_CODEX_IDLE_TTL_MS` | `1800000` | `SessionStart` 闲置 TTL 清理阈值(毫秒) |+| `OPENVIKING_CODEX_LOCK_WAIT_MS` | `120000`(SessionEnd)、`40000`(PreCompact) | 捕获类 hook 等待单会话状态锁的时长(毫秒) || `OPENVIKING_CODEX_COMMITTED_TTL_MS` | `2592000000` | 已提交会话的转录游标保留时长(毫秒),过期后删除状态文件 || `OPENVIKING_DEBUG` | `false` | 是否将日志写入 `~/.openviking/logs/codex-hooks.log` |diff --git a/docs/zh/agent-integrations/16-capability-reference.md b/docs/zh/agent-integrations/16-capability-reference.mdindex b85f611b7..4c1ccfa4f 100644--- a/docs/zh/agent-integrations/16-capability-reference.md+++ b/docs/zh/agent-integrations/16-capability-reference.md@@ -299,7 +299,7 @@ JS 系 harness 的召回逻辑均由 `recall-core.mjs` 中的三级降级链处| harness | 轮内阈值触发 | 显式/边界触发 | 压缩触发 ||---|---|---|---|| claude-code | Stop:`pending_tokens ≥ 20000`(读服务端值),keep 10 | SessionEnd:无条件触发;SubagentStop:无条件触发(无阈值);SessionStart:重放 pending | PreCompact:无条件触发(同步执行,不 detach) |-| codex / trae-cli | Stop:同上,20000 / keep 10 | SessionStart(startup\|clear):基于 active-window 的启发式(2min 内恰好 1 个 state → commit;≥2 个则跳过) + idle-TTL 扫描(>30min 一律 commit 并清理) | PreCompact:全量 commit(发空 body `{}`),随后置 `ovSessionId=null` |+| codex / trae-cli | Stop:同上,20000 / keep 10 | SessionEnd(Codex ≥ 0.145):无条件触发,先补齐 Stop 漏掉的轮次再 commit——父 hook 只写 `.ended` 标记并 detach worker(Codex 默认给 1s,`timeout` 上限 3s);SessionStart(startup\|clear):兜底扫描,提交带 `.ended` 标记或闲置超过 30min 的 state。trae-cli 若无 `SessionEnd` 则只走扫描 | PreCompact:全量 commit(发空 body `{}`),随后置 `ovSessionId=null` || cursor | stop:`capturedSinceCommit ≥ 8`(按消息条数计算,≈4 轮问答;纯客户端计数),keep 0 | sessionEnd:已注册该事件(但实践中不触达,见 [§3.3.3](#_3-3-3-关闭方式-×-harness-终局矩阵)) | preCompact:无条件触发 || trae / trae-cn | 每个有内容的 Stop 都 commit(无阈值),keep 0 | — | 无(上游无 PreCompact 事件) || zcode | 同 trae(每 Stop 都 commit,keep 0;rollout 增量游标保守推进,若有漏掉的轮次,将在同会话的下个 Stop 补齐) | — | 无(上游无 PreCompact 事件) |@@ -320,7 +320,8 @@ JS 系 harness 的召回逻辑均由 `recall-core.mjs` 中的三级降级链处| harness | 正常退出 | Ctrl+C | SIGTERM | SIGHUP/关终端/关窗口·tab | kill -9/崩溃 | 补救路径 ||---|---|---|---|---|---|---|| claude-code | **C**(SessionEnd → detach 子进程 commit,用户不等待) | **C** | **C** | **C**(detached worker 自成进程组,不受 SIGHUP 波及) | **—** | 下次 Stop 越阈值 / `/compact` / 下次 SessionEnd |-| codex / trae-cli | **—**(上游不提供 SessionEnd 类事件) | **—** | **—** | **—** | **—** | 下次 `SessionStart(startup\|clear)` 的启发式(2min 内恰好 1 个活跃 → commit)或 30min idle-TTL 扫描;≥2 并发会话时启发式让位于 TTL 扫描 |+| codex | **C**(SessionEnd → detach worker commit,用户不等待;需 Codex ≥ 0.145) | **C\*** | **—** | **—** | **—** | C\* 前提:连按两次 `Ctrl-C` 属于正常退出、会触发 SessionEnd,单次不会。未 commit 的一律由下次 `SessionStart(startup\|clear)` 回收:标记仍在则 `ended_retry`,否则等 30min idle-TTL 扫描 |+| trae-cli | **—**(除非 TraeCode CLI 版本已带 `SessionEnd`) | **—** | **—** | **—** | **—** | 下次 `SessionStart(startup\|clear)` 的 30min idle-TTL 扫描 || cursor | **—**(chat 关闭 / new-chat 无事件) | **—** | **—** | **—**(`sessionEnd` 已注册且仅 window_close 触发,但此时宿主已销毁 shell-exec host,hook 在 spawn 前中止) | **—** | 结束在 <8 条消息水位的会话,尾部依赖同一会话的后续消息触发 commit || trae / trae-cn | **—**(无 session-end 类事件) | **—** | **—** | **—** | **—** | 每 Stop 已 commit,最大待归档量 = 最后一轮 in-flight || zcode | **—**(无 session-end 类事件) | **C\*** | **—** | **—** | **—** | C\* 前提:Ctrl+C 时该轮 Stop 已触发(detached worker 照常写完);每 Stop 已 commit,漏掉的轮次靠 rollout 游标在同一会话的下个 Stop 补回 |@@ -479,9 +480,9 @@ MCP `write` / REST `content/write` 的三道 guard(`content_write.py`):可## codex- **集成文档**:[Codex 记忆插件](./04-codex.md)-- **形态**:Codex 插件(marketplace),4 hook(SessionStart 70s / UserPromptSubmit 130s / Stop 30s / PreCompact 60s)+ MCP 代理 + 1 experience skill。版本 0.7.5。-- **能力亮点**:本地召回压缩管线(`codex exec`,[§3.2.5](#_3-2-5-召回再摘要));SessionStart 的 active-window 启发式 + idle-TTL 扫描,可自动回收历史会话中未归档的消息([§3.3.3](#_3-3-3-关闭方式-×-harness-终局矩阵))。-- **行为要点**:session id 为 `cx-<safeId>`(确定性推导,不读 state);关闭时无 hook(上游不提供 SessionEnd),commit 依赖下次启动的回收路径;不使用磁盘 pending queue(离线靠游标不推进、下一轮重发补偿,[§3.3.4](#_3-3-4-pending-queue-离线补偿对照));active-window 120000ms / idle-TTL 1800000ms(env 配置);Stop 默认 detach(`appended N turn(s)` 提示默认不展示)。+- **形态**:Codex 插件(marketplace),5 hook(SessionStart 70s / UserPromptSubmit 130s / Stop 30s / SessionEnd 3s / PreCompact 60s)+ MCP 代理 + 1 experience skill。版本 0.8.0。+- **能力亮点**:本地召回压缩管线(`codex exec`,[§3.2.5](#_3-2-5-召回再摘要));SessionEnd(Codex ≥ 0.145)在正常退出时补齐 Stop 漏掉的轮次并 commit,SessionStart 的兜底扫描回收那些没触发 SessionEnd 的退出所遗留的未归档消息([§3.3.3](#_3-3-3-关闭方式-×-harness-终局矩阵))。+- **行为要点**:session id 为 `cx-<safeId>`(确定性推导,不读 state);SessionEnd 只在正常退出、且 Codex 0.145+ 时触发,信号、崩溃、旧版本以及 `codex app-server` 延后的场景都落到扫描路径;不使用磁盘 pending queue(离线靠游标不推进、下一轮重发补偿,[§3.3.4](#_3-3-4-pending-queue-离线补偿对照));idle-TTL 1800000ms、锁等待 120000ms(env 配置);Stop 与 SessionEnd 默认 detach(`appended N turn(s)` 提示默认不展示);单会话 mkdir 锁串行化 Stop worker、PreCompact、SessionEnd worker 与扫描。新增的 hook 事件在 Codex 侧没有信任记录,升级后需在 `/hooks` 中批准 SessionEnd。- **配置**:env + ovcli.conf `plugin.codex` + ov.conf `codex`;hooks 同时发送 Bearer 与 `X-API-Key` 兼容头([§3.1.3](#_3-1-3-凭据体系))。- **维度索引**:工具面 [§2.1](#_2-1-服务端-mcp-工具面) |召回 [§3.2](#_3-2-自动召回与注入) |commit [§3.3.2](#_3-3-2-常规-commit-触发条件)/[§3.3.3](#_3-3-3-关闭方式-×-harness-终局矩阵) |降级 [§3.6](#_3-6-降级与容错)。@@ -489,7 +490,7 @@ MCP `write` / REST `content/write` 的三道 guard(`content_write.py`):可- **集成文档**:[TRAE 记忆集成](./13-trae.md)- **形态**:TraeCode CLI 2.0 是 Codex 系 CLI(binary `traecli`,用户配置 `~/.trae/traecli.toml`,TUI 支持 `/plugins` `/skills` `/mcp`)。OpenViking 经 **codex 插件别名安装**接入:`--harness trae-cli` 复用 codex 安装流程,仅安装参数(binary / home / 配置路径)指向 TraeCode CLI。-- **能力面**:与 codex 完全一致——4 hook + MCP 代理 + experience skill、本地召回压缩、active-window/idle-TTL commit 回收、resume archive 注入等,详见 codex 档案卡。+- **能力面**:与 codex 同一套插件——5 个已注册 hook + MCP 代理 + experience skill、本地召回压缩、idle-TTL commit 回收、resume archive 注入等,详见 codex 档案卡。若 TraeCode CLI 所基于的 Codex 版本没有 `SessionEnd`,该 hook 会被忽略,关闭时的 commit 全部依赖 idle-TTL 扫描。- **版本支持**:仅支持 TraeCode CLI 2.0。1.0 与 2.0 不是同一套 CLI,2.0 才是 Codex 系、才能走 codex 插件别名安装;早期面向 1.0 的独立插件 `examples/trae-cli-memory-hooks`(`~/.trae/cli/hooks.json` + `[mcp_servers."openviking-memory"]` 方案)已废弃。- **维度索引**:同 codex 卡。diff --git a/examples/codex-memory-plugin/.codex-plugin/plugin.json b/examples/codex-memory-plugin/.codex-plugin/plugin.jsonindex f0e06cae7..7225cd17e 100644--- a/examples/codex-memory-plugin/.codex-plugin/plugin.json+++ b/examples/codex-memory-plugin/.codex-plugin/plugin.json@@ -1,7 +1,7 @@{"name": "openviking-memory",- "version": "0.7.7",- "description": "Long-term semantic memory for Codex, powered by OpenViking. Recall on UserPromptSubmit, incremental add_message on Stop (per turn), commit on PreCompact, and active-window heuristic + idle-TTL sweep on SessionStart (source=startup|clear). Includes a local stdio MCP proxy that reads OpenViking credentials from env/ovcli.conf.",+ "version": "0.8.1",+ "description": "Long-term semantic memory for Codex, powered by OpenViking. Recall on UserPromptSubmit, incremental add_message on Stop (per turn), commit on SessionEnd (Codex >= 0.145) and on PreCompact, with an idle/ended sweep on SessionStart as the fallback for signals, crashes and older Codex builds. Includes a local stdio MCP proxy that reads OpenViking credentials from env/ovcli.conf.","author": {"name": "OpenViking","url": "https://github.com/volcengine/OpenViking"@@ -22,7 +22,7 @@"interface": {"displayName": "OpenViking Memory","shortDescription": "Long-term semantic memory for Codex",- "longDescription": "Hooks Codex's lifecycle to keep an external semantic memory: recall relevant memories on each UserPromptSubmit; on every Stop (turn end) append the new user/assistant turns to a long-lived OpenViking session keyed by codex session_id; on PreCompact commit that session so OV's extractor produces durable memories before context is summarized; on SessionStart(source=startup|clear) commit eligible orphaned sessions using the active-window and idle-TTL rules. Also exposes OpenViking's native MCP tools through a local stdio proxy to the streamable-HTTP /mcp endpoint for manual use.",+ "longDescription": "Hooks Codex's lifecycle to keep an external semantic memory: recall relevant memories on each UserPromptSubmit; on every Stop (turn end) append the new user/assistant turns to a long-lived OpenViking session keyed by codex session_id; on SessionEnd (Codex >= 0.145) catch up any turns Stop missed and commit the session so OV's extractor produces durable memories when the thread finishes; on PreCompact commit before context is summarized; on SessionStart(source=startup|clear) sweep sessions that ended without a SessionEnd or went idle. Also exposes OpenViking's native MCP tools through a local stdio proxy to the streamable-HTTP /mcp endpoint for manual use.","developerName": "OpenViking","category": "Productivity","capabilities": [diff --git a/examples/codex-memory-plugin/DESIGN.md b/examples/codex-memory-plugin/DESIGN.mdindex 501df7a08..d9493c836 100644--- a/examples/codex-memory-plugin/DESIGN.md+++ b/examples/codex-memory-plugin/DESIGN.md@@ -2,8 +2,9 @@This document records *why* the plugin commits when it commits. The commitshape (which OpenViking session is sealed by which hook event) is the part-worth understanding before reading code: the codex hook surface gives us-**no clean SessionEnd signal**, so we have to reason about which observable+worth understanding before reading code: `SessionEnd` gives us a+deterministic end signal for graceful exits, but it does not cover signals,+crashes or older Codex builds, so we still reason about which observableevents imply "context for a particular codex `session_id` is gone".## Vocabulary@@ -17,10 +18,17 @@ events imply "context for a particular codex `session_id` is gone".memory extractor) at session-end-equivalent moments. `/messages`auto-creates the OV session, so the plugin does not call session create.- **State file** — `~/.openviking/codex-plugin-state/<safe-codex-session-id>.json`,- shape `{ codexSessionId, ovSessionId, capturedTurnCount, createdAt, lastUpdatedAt }`.-- **Active window** — state files whose `lastUpdatedAt` is within- `ACTIVE_WINDOW_MS` (default 2 min) of "now". Used to detect "the codex- session that just ended".+ shape `{ codexSessionId, ovSessionId, transcriptPath, capturedTurnCount, createdAt, lastUpdatedAt }`.+- **End marker** — `<safe-codex-session-id>.ended.<timestamp>`, a sidecar written by the+ `SessionEnd` parent hook, containing the timestamp at which it was+ written. Its presence means "the thread ended and its commit has not been+ confirmed yet"; its timestamp is the token that tells one exit's marker+ from another's, and it lives in the filename so a conditional removal+ targets an immutable path. A bare `<safe-codex-session-id>.ended` is a+ pre-0.8.1 marker and is still read back.+- **Session lock** — `<safe-codex-session-id>.lock`, an exclusive mkdir lock+ serializing the four writers that persist the whole state object. It holds+ an `owner` file so a holder only ever releases its own lock.## Codex hook surface (what we observe)@@ -33,12 +41,13 @@ events imply "context for a particular codex `session_id` is gone".| `Stop` | end of every model turn (NOT end of session) | append turns to OV session || `PreCompact` | `/compact` or auto-compact | context is about to be summarized || `PostCompact` | after compaction | (unused) |-| SIGTERM / SIGINT / Ctrl+C / `/exit` | process killed | **no hook fires** — confirmed in `codex-rs/hooks/src/events/` |+| `SessionEnd` | graceful thread shutdown: `/quit`, `/exit`, double Ctrl-C, EOF, end of a `codex exec` run | this thread's context is gone; commit it |+| SIGTERM / SIGHUP / terminal close / `kill -9` / crash | process killed | **no hook fires** — tui/exec install no signal handlers |-Verified against codex-rs `main` 2026-05-10. Upstream issues #17421, #20374-have requested a `SessionEnd` hook; OpenAI rejected with two reasons:-"threads can always be resumed" and "/exit only makes sense in TUI". Not-landing.+Verified against codex-rs `main` @ 6be2a6ca, 2026-08-28. `SessionEnd` shipped+in `rust-v0.145.0` and is present in every stable since; older Codex builds+(and TraeCode CLI builds without it) ignore the unknown event name, which is why+the fallback sweep stays.## Commit triggers@@ -54,47 +63,47 @@ same `cx-<codex-session-id>` OV session id for the post-compact half.`capturedTurnCount` is preserved unless the transcript was truncated bycompaction (see "Post-compact transcript shrink" below).-### 2. `SessionStart` source=`clear` — heuristic, same shape as `startup`--`/clear` creates a brand-new codex `session_id` and orphans the previous-in-memory thread (preserved on disk). Naively committing "every state file-whose `codexSessionId` ≠ new id" would falsely commit concurrent codex-processes' still-active sessions on the same machine.--Instead, we treat `clear` and `startup` identically: both run the-**active-window heuristic** below. `/clear` only invalidates the current-codex process's *previous* session; the heuristic correctly catches that-session (a single recently-touched orphan) without trampling unrelated-parallel codex processes.--### 3. `SessionStart` source=`startup` — heuristic, active-window--Triggered by `/new`, fresh codex CLI startup, and zouk daemon+### 2. `SessionEnd` — deterministic, graceful exits++This is the primary commit path. Codex fires `SessionEnd` when a thread+shuts down gracefully: `/quit`, `/exit`, double Ctrl-C, EOF on stdin, and+the end of a `codex exec` run. In a single TUI process every thread the+process touched via `/new` or `/resume` gets its own `SessionEnd`, all in a+burst at process exit — so `/new` on its own does not end the previous+thread; its end event arrives later, when the process leaves.++The rollout is flushed to disk before the hook runs, so the hook sees the+complete transcript. `session-end.mjs` therefore catches up whatever turns+the last `Stop` never sent, then commits the OV session and clears+`ovSessionId` with `touch: false`.++The budget forces the work off the hook's own process. Codex allows 1s by+default, clamps a `timeout` in `hooks.json` to 3s, forces `async: true`+hooks to run synchronously, and ignores stdout — nowhere near enough for a+catch-up append plus a commit. So the parent hook does two cheap things and+exits: write the `.ended` sidecar (lock-free, before anything else, so the+sweep can still recover if the worker never runs) and detach a worker,+regardless of `writePathAsync`. Codex deliberately leaves cleanly detached+helpers running after a hook exits and only kills the process group on a+timeout, and the detached worker holds no inherited stdout/stderr, so Codex+does not wait on it.++`SessionEnd` does **not** fire on SIGTERM, SIGHUP, a closed terminal,+`kill -9`, or a crash. When the TUI is attached to a `codex app-server`+daemon, it is deferred until the thread is unloaded (30 min) or the daemon+shuts down. Those cases fall to the fallback sweep.++### 3. `SessionStart` source=`startup` / `clear` — fallback sweep entry++Triggered by `/new`, `/clear`, fresh codex CLI startup, and zouk daemonspawn-without-sessionId (including zouk's "reset codex" UI action).-The hook script gates internally on `source ∈ {startup, clear}`. On a-match, it iterates state files (excluding the new `session_id` itself) and-counts how many were touched within `ACTIVE_WINDOW_MS`:--```-recently-active count ⇒ action-─────────────────────────────────-0 ⇒ no-op (no orphan to commit)-1 ⇒ commit it (the just-ended session)-≥2 ⇒ skip; rely on idle TTL-```--The single-recent case captures the common path: user runs codex, hits-`/new` or `/clear` after a turn or two; the previous session's `Stop` just-fired and bumped `lastUpdatedAt`; we commit it. The multi-recent case-implies concurrent codex sessions are active; we can't tell which one (if-any) ended, so we defer to idle TTL to clean up genuinely-dead ones.--The count is over *activity*, including cursor-only states: a session-`PreCompact` just committed has no live `ovSessionId` but may well still be-running, and counting it keeps the `≥2` branch from sealing a sibling-session mid-flight. Only a state that still has a live `ovSessionId` is-actually committed; the single-recent case with nothing live is a no-op.+`/clear` creates a brand-new codex `session_id` and orphans the previous+in-memory thread (preserved on disk); `startup` may or may not follow an+exit that already committed. Either way the hook does not try to guess+which session just ended — that is `SessionEnd`'s job. It gates internally+on `source ∈ {startup, clear}` and runs the fallback sweep of rule 5 over+every state file except the new `session_id`.### 4. `SessionStart` source=`resume` — never commits, optional archive inject@@ -124,37 +133,58 @@ If local state still has a live `ovSessionId`, resume injection is skipped:the session is appendable and Codex should already be resuming its owntranscript.-### 5. Idle TTL sweep — fallback+### 5. Fallback sweep — everything `SessionEnd` could not reach++On `startup` and `clear`, every state file except the new `session_id` is+examined. A state that still has a live `ovSessionId` is committed for one+of two reasons:++| Reason | Condition | What it means |+|---|---|---|+| `ended_retry` | an `.ended.<timestamp>` sidecar is present | `SessionEnd` fired but its commit never completed: OV was unreachable, `/commit` failed, or the worker was killed (Windows job objects kill detached children) |+| `idle_ttl` | no marker and `lastUpdatedAt` older than `IDLE_TTL_MS` (default 30 min) | no `SessionEnd` was ever going to arrive: signals, crashes, Codex older than 0.145 / TraeCode CLI, app-server deferral, or a mid-turn zouk reset that cancelled `Stop` |++Before committing, the sweep appends whatever the state's recorded+`transcriptPath` still holds past `capturedTurnCount`, so a session whose+own workers never ran is not archived without its tail turns. If part of+that append fails, the sweep keeps `ovSessionId` and the marker and skips+the commit; the next sweep retries from the advanced cursor.-State files with a live `ovSessionId` whose `lastUpdatedAt` is older than-`IDLE_TTL_MS` (default 30 min) get committed. Their transcript cursor is-preserved while `ovSessionId` is cleared. Mental model: a session not touched-for 30 min is "temporarily concluded"; if the user resumes later, subsequent-turns append under the same deterministic OV session id, and the next commit-creates another archive there.+A marker enters the lock even when the state carries no live `ovSessionId`. `PreCompact` releases the id but leaves the cursor and `transcriptPath` behind, so the tail turns of a thread whose `SessionEnd` worker was killed are only reachable through that catch-up — the catch-up derives a live id by itself as soon as it has something to send. Only when the catch-up finds nothing new and there is still no live id is the marker cleared.++An unreadable transcript is never treated as an empty one. If the state records a `transcriptPath` that cannot be read, the sweep logs `transcript_unreadable`, keeps the live id and the marker, and skips the commit, exactly like a partial append — otherwise a transient read failure would archive a session without turns nobody has seen yet. `SessionEnd` and `PreCompact` apply the same guard.++Their transcript cursor is preserved while `ovSessionId` is cleared. Mental+model for the idle case: a session not touched for 30 min is "temporarily+concluded"; if the user resumes later, subsequent turns append under the+same deterministic OV session id, and the next commit creates another+archive there.Releasing the session id instead of deleting the file writes state withouttouching `lastUpdatedAt`, so a committed session doesn't look freshly active-to the next `SessionStart`.--This covers:-- SIGTERM / Ctrl+C / `/exit` (no hook fires; state file rots)-- Crashes-- Mid-turn zouk reset where `Stop` got cancelled before bumping- `lastUpdatedAt`-- The `≥2 recently-active` skip from rule 3--**Sweep trigger**: at the tail of `session-start-commit.mjs` only. We do-not sweep on every `Stop` because state-write-on-every-turn already gives-us the freshness signal we need; running the sweep once per session start-is the right cadence. The Stop hook contains a comment marking the option-to add sweep there if codex's session creation rate is low enough that-arbitrarily-orphaned state files accumulate.--**Known limitation**: if the user never starts another codex on this-machine, no sweep ever runs and the OV session stays open server-side-forever. Accepted. Future work could add an MCP tool-`openviking_commit_pending` so the model can commit explicitly.+to the next `SessionStart`. A successful commit also removes the `.ended`+marker; a failed one keeps both marker and `ovSessionId` so the next sweep+retries.++Each candidate is committed under its session lock as a **try-lock**: a lock+we cannot take immediately means a `SessionEnd` or `Stop` worker already+owns that session (the user quit and relaunched within seconds), so the+sweep logs the skip and leaves the state alone. State is reloaded inside the+lock before the commit, so the sweep never writes back a snapshot the worker+has since superseded. The `.ended` marker is re-read there too: if an+`ended_retry` candidate's marker is gone (the thread was resumed) or newer+than the snapshot (a later exit whose own worker will commit), the sweep+falls back to the idle rule and leaves anything younger than `IDLE_TTL_MS`+alone.++**Sweep trigger**: at the tail of `session-start-commit.mjs` only. `Stop`+does not sweep; state-write-on-every-turn already gives us the freshness+signal, and once per session start is the right cadence.++**Known limitation**: if a session's `SessionEnd` never fired and the user+never starts another codex on this machine, no sweep ever runs and the OV+session stays open server-side forever. Accepted. Future work could add an+MCP tool `openviking_commit_pending` so the model can commit explicitly.### 6. Cursor retention — same pass@@ -236,18 +266,72 @@ were retried, those turns are dropped rather than duplicated.### Commit failure-When OV `/commit` returns non-2xx or times out, we currently log and treat-the result as null. We must NOT call `clearState` on failure — keep the-state file so the next sweep / SessionStart can retry. A transient OV-outage shouldn't lose a session's worth of memory.--### Race: SIGTERM before Stop completes--Codex's tokio runtime cancels in-flight async tasks on SIGTERM, so the last-turn's `Stop` hook may be aborted before it bumps `lastUpdatedAt`. This-makes the state look older than it actually is. Consequence: that session-may fall outside the 2 min active window when the user respawns codex and-we can't commit it deterministically — idle TTL will catch it later.+When OV `/commit` returns non-2xx or times out, we log the `trace_id` and+keep `ovSessionId` set. We must NOT call `clearState` on failure — keep the+state file, and in the `SessionEnd` path keep the `.ended` marker too, so+the next sweep retries. A transient OV outage shouldn't lose a session's+worth of memory. An unreadable transcript is handled the same way: no+commit, state and marker preserved.++### Race: exit before Stop completes++Codex's tokio runtime cancels in-flight async tasks when the process goes+away, so the last turn's `Stop` hook may be aborted before it appends its+turns and bumps `lastUpdatedAt`. On a graceful exit this costs nothing: the+`SessionEnd` worker reads the flushed rollout and appends everything past+the cursor before committing. On a signal or a crash there is no+`SessionEnd`, the state looks older than it really is, and the idle TTL+sweep commits it at the next `SessionStart` — with the same catch-up, from+the `transcriptPath` and cursor the last completed `Stop` recorded. A+session whose very first `Stop` never completed has no recorded transcript,+so there is nothing for the sweep to catch up and it commits whatever the+OV session already holds.++### Race: concurrent writers of the same state file++The `Stop` worker, `PreCompact`, the `SessionEnd` worker and the sweep all+persist the whole state object, so without serialization the last writer+wins and can resurrect a committed `ovSessionId` or rewind the cursor. All+four therefore run under `withSessionLock`, and all four load state *inside*+the lock. The lock is a directory (`mkdir` is atomic on every platform we+run on); a holder that dies leaves a lock that is abandoned once its mtime+is `staleMs` (5 min) old, and a live holder refreshes the mtime from the+batch-send callback so a long catch-up never looks stale. Wait budgets:+120s for the `Stop` and `SessionEnd` workers, 40s for `PreCompact` (which+must still answer inside its 60s hook budget, and on timeout emits `{}` and+touches nothing), and 0 for the sweep.++Ownership makes the lock safe to abandon. The holder writes an `owner` file+inside the directory containing `<pid>:<uuid>`, and only releases (or+refreshes) a lock whose `owner` still matches its own — otherwise a taker+that lost a race would release a lock somebody else now holds. Takeover+happens in place, on that same `owner` file: the taker renames it aside and+then creates its own exclusively, two atomic steps that exactly one racer can+complete, and a racer that loses either one leaves the winner's lock intact.+The directory itself is never moved or removed during a takeover, because a+lock path that is momentarily absent would let another racer's `mkdir` succeed+alongside the taker.++### Race: a marker and a worker that outlive each other++The `.ended` marker's timestamp is its identity. The `SessionEnd` parent+passes the timestamp it wrote to the detached worker through+`OPENVIKING_SESSION_END_TOKEN`; the worker, after taking the lock and before+any network call, re-reads the marker and returns without committing unless+it still matches — a cleared marker means the thread was resumed, a+different one belongs to a newer exit whose own worker will commit. A+commit only clears the marker it verified. Because each marker's timestamp+is part of its filename, a conditional removal enumerates the markers older+than its cutoff and unlinks exactly those paths — a marker written between+the enumeration and the unlink is a different file and survives untouched.++`Stop` and `PreCompact` clear the `.ended` marker at entry, because a turn+for this session proves the thread is alive again after a resume. They clear+it only if it is older than the hook's own start time, so a detached `Stop`+worker that boots after the next exit cannot erase that exit's fresh marker;+the `Stop` parent forwards its start time to its worker through+`OPENVIKING_HOOK_STARTED_AT`. `SessionStart` `source=resume` clears with its+own start time for the same reason.### Commit-then-resume@@ -263,6 +347,7 @@ OV session id, while commits create additional archives under that session.{"codexSessionId": "0193af...", // codex thread id"ovSessionId": "cx-0193af...-or-null", // null means "committed, awaiting next Stop or retirement"+ "transcriptPath": "/path/rollout.jsonl", // last rollout seen; lets the sweep catch up"capturedTurnCount": 7, // turns from transcript already appended"createdAt": 1715000000000,"lastUpdatedAt": 1715000300000@@ -276,6 +361,18 @@ closed.State files are atomic-write (tmpfile + rename) to survive crash mid-write.+Two sidecars live next to `<safe-codex-session-id>.json`:++| Path | Written by | Meaning |+|---|---|---|+| `<safe-id>.ended.<timestamp>` | `SessionEnd` parent hook (timestamp in the name, also the content; created exclusively, and bumped by a millisecond until that succeeds, so two exits in the same millisecond still get distinct markers) | the thread ended; its commit is not confirmed. Read back by `listStates()` as `endedAt`, which takes the largest timestamp when several markers exist. Removed by a commit that verified this exact timestamp, or by a `Stop` / `PreCompact` / `resume` that started after it was written — each removal unlinks the exact marker paths older than its cutoff, so it can never take out a newer exit's marker. A bare `<safe-id>.ended` (pre-0.8.1) is still honoured, with the timestamp read from its content |+| `<safe-id>.lock` | whichever writer currently holds the session | exclusive `mkdir` lock holding an `owner` file; abandoned when its mtime is older than 5 min, and taken over in place by claiming that `owner` file |++Keeping the end marker out of the JSON is deliberate: a whole-object+`saveState` from a concurrent worker cannot clobber a separate file, and the+parent hook can write it lock-free in about a millisecond. Neither sidecar+is picked up by `listStates()` or the doctor, which read `.json` only.+## ConfigurationEnv var overrides for tuning without rebuilding:@@ -283,8 +380,8 @@ Env var overrides for tuning without rebuilding:| Var | Default | Purpose ||---|---|---|| `OPENVIKING_CODEX_STATE_DIR` | `~/.openviking/codex-plugin-state` | state file dir |-| `OPENVIKING_CODEX_ACTIVE_WINDOW_MS` | `120000` (2 min) | rule-3 active window || `OPENVIKING_CODEX_IDLE_TTL_MS` | `1800000` (30 min) | idle sweep TTL |+| `OPENVIKING_CODEX_LOCK_WAIT_MS` | `120000` (`SessionEnd` worker), `40000` (`PreCompact`) | how long a writer waits for the session lock || `OPENVIKING_CODEX_COMMITTED_TTL_MS` | `2592000000` (30 days) | how long a committed cursor is kept for resume || `OPENVIKING_RECALL_TIMEOUT_MS` | `120000` (2 min) | whole UserPromptSubmit auto-recall deadline || `OPENVIKING_RECALL_COMPRESS` | `1` | set `0` / `off` to skip `codex exec` compression |@@ -359,6 +456,27 @@ Fallback order:Configured `off` (`OPENVIKING_RECALL_COMPRESS=0`, model `off`, or thinking`off`) skips all probing and writes a disabled profile.+## What changed vs 0.7.x++- `SessionEnd` (Codex ≥ 0.145) is registered and is now the primary commit+ path; `session-end.mjs` marks the session ended, then a detached worker+ catches up missed turns and commits.+- The active-window heuristic and `OPENVIKING_CODEX_ACTIVE_WINDOW_MS` are+ gone. `SessionStart` `startup|clear` now only sweeps: `ended_retry` for+ states with an `.ended` marker, `idle_ttl` for states past the idle TTL.+- A per-session `.lock` directory serializes the `Stop` worker,+ `PreCompact`, the `SessionEnd` worker and the sweep, with a new+ `OPENVIKING_CODEX_LOCK_WAIT_MS` wait budget and a try-lock for the sweep.+- The `.ended` sidecar is cleared by any `Stop`, `PreCompact` or+ `source=resume` for that session.+- `ov-session.mjs` holds the OV HTTP client, transcript reader and catch-up+ logic that `Stop`, `PreCompact` and `SessionEnd` used to duplicate;+ `PreCompact` gains the post-compact shrink defense it lacked, and an+ unreadable or empty transcript can no longer reset the cursor to 0.+- Behaviour change: after `/new` the abandoned thread is committed when the+ process exits (the `SessionEnd` burst) or after the idle TTL — no longer+ at the next `SessionStart`.+## What changed vs v0.3.1- `SessionStart` matcher widened from `"clear"` to `"clear|startup|resume"`@@ -387,8 +505,6 @@ Configured `off` (`OPENVIKING_RECALL_COMPRESS=0`, model `off`, or thinking- **Subagent hook events**: kimicode has them, codex doesn't yet.When codex adds them, we should hook to keep subagent memory threadsseparate from main session.-- **Upstream `SessionEnd`**: rejected by OpenAI. If they reverse, idle- TTL becomes redundant — replace with deterministic SessionEnd commit.## Verified hook payload reference@@ -436,6 +552,15 @@ Configured `off` (`OPENVIKING_RECALL_COMPRESS=0`, model `off`, or thinking"model": "gpt-5.5","hook_event_name": "PreCompact"}++// SessionEnd input (Codex >= 0.145; `reason` is a constant)+{+ "session_id": "0193af...",+ "transcript_path": "/path/to/rollout.jsonl",+ "cwd": "/path/to/cwd",+ "hook_event_name": "SessionEnd",+ "reason": "other"+}```Output schema for SessionStart / UserPromptSubmit supports@@ -443,4 +568,5 @@ Output schema for SessionStart / UserPromptSubmit supportsthat field together with `systemMessage`, allowing profile injection and orphancommit status to coexist. Stop / PreCompact only support`{ continue, stopReason, suppressOutput, systemMessage }` — `{}` is a-valid no-op.+valid no-op. SessionEnd output is ignored entirely; the hook cannot block+and cannot inject, so `session-end.mjs` prints `{}` for symmetry.diff --git a/examples/codex-memory-plugin/README.md b/examples/codex-memory-plugin/README.mdindex 3aec0fd67..b25cc9190 100644--- a/examples/codex-memory-plugin/README.md+++ b/examples/codex-memory-plugin/README.md@@ -13,7 +13,8 @@ This is the Codex counterpart to [`claude-code-memory-plugin`](../claude-code-me- **Auto-recall** relevant memories on every `UserPromptSubmit` and inject them via `hookSpecificOutput.additionalContext`- **Incremental capture on `Stop`** (turn end): append the new user/assistant turns to a deterministic OpenViking session id `cx-<codex_session_id>`. When `pending_tokens` reaches `OPENVIKING_COMMIT_TOKEN_THRESHOLD`, commit while keeping a recent live tail.- **Commit on `PreCompact`**: trigger OpenViking's memory extractor on the full pre-compact transcript before Codex summarizes it.-- **Commit on `SessionStart` (source=startup|clear)**: active-window heuristic — if exactly one *other* state file was touched within the last 2 min, commit it (the just-ended session). On `≥2`, defer to idle-TTL sweep at the tail. `source=resume` never commits or sweeps; if the live OV session was already committed, it combines the profile block with the latest archive summary for continuity. See `DESIGN.md` for the full decision tree.+- **Commit on `SessionEnd`** (Codex ≥ 0.145): when a thread shuts down gracefully, catch up any turns `Stop` never sent and commit the OV session, so the extractor runs on the whole conversation the moment you leave.+- **Fallback sweep on `SessionStart` (source=startup|clear)**: commit state files that carry an end marker whose commit did not go through, or that have been idle past `OPENVIKING_CODEX_IDLE_TTL_MS`. `source=resume` never commits or sweeps; if the live OV session was already committed, it combines the profile block with the latest archive summary for continuity. See `DESIGN.md` for the full decision tree.It also starts a local stdio MCP proxy that forwards to OpenViking's native `/mcp` endpoint with credentials resolved from env / `ovcli.conf`, so the model has direct access to the server's retrieval, memory, resource, watch, filesystem, and code-navigation tools.@@ -152,28 +153,28 @@ Earlier plugin versions configured tuning fields under a `codex` block in `~/.op## Architecture```- ┌──────────────────────────────────────────────────────────────┐- │ Codex │- └──┬─────────────────┬────────────────┬───────────────────┬────┘- │ │ │ │- SessionStart UserPromptSubmit Stop PreCompact- (startup|clear|resume) │ (per turn) │- │ │ │ │- ┌────▼──────────┐ ┌────▼──────┐ ┌──────▼──────┐ ┌──────────▼──────┐- │ session-start │ │ auto- │ │ auto- │ │ pre-compact- │- │ -commit.mjs │ │ recall.mjs│ │ capture.mjs │ │ capture.mjs │- │ (profile + │ │ (search + │ │ (append + │ │ (commit + reset │- │ active-win + │ │ compress) │ │ threshold │ │ ovSessionId) │- │ idle TTL + │ │ │ │ │ │ │- │ resume archive)││ │ │ │ │ │- └────┬──────────┘ └────┬──────┘ └──────┬──────┘ └──────────┬──────┘- │ │ │ │- │ ┌───▼────────────────▼───────────────────▼──┐- └────────────►│ OpenViking REST API │- │ /api/v1/search/{recall,search} │- │ /api/v1/sessions [+/{id}/{messages,commit}]│- │ /api/v1/content/read │- └─────────────────┬─────────────────────────┘+ ┌────────────────────────────────────────────────────────────────────────┐+ │ Codex │+ └──┬─────────────────┬────────────────┬──────────────────┬───────────┬───┘+ │ │ │ │ │+ SessionStart UserPromptSubmit Stop PreCompact SessionEnd+ (startup|clear|resume) │ (per turn) │ (graceful exit)+ │ │ │ │ │+ ┌────▼──────────┐ ┌────▼──────┐ ┌──────▼──────┐ ┌─────────▼──────┐ ┌───▼─────────┐+ │ session-start │ │ auto- │ │ auto- │ │ pre-compact- │ │ session- │+ │ -commit.mjs │ │ recall.mjs│ │ capture.mjs │ │ capture.mjs │ │ end.mjs │+ │ (profile + │ │ (search + │ │ (append + │ │ (commit + reset│ │ (mark + │+ │ fallback sweep│ │ compress) │ │ threshold) │ │ ovSessionId) │ │ catch-up + │+ │ + resume │ │ │ │ │ │ │ │ commit) │+ │ archive) │ │ │ │ │ │ │ │ │+ └────┬──────────┘ └────┬──────┘ └──────┬──────┘ └─────────┬──────┘ └───┬─────────┘+ │ │ │ │ │+ │ ┌───▼────────────────▼──────────────────▼───────────▼──┐+ └────────────►│ OpenViking REST API │+ │ /api/v1/search/{recall,search} │+ │ /api/v1/sessions [+/{id}/{messages,commit}] │+ │ /api/v1/content/read │+ └─────────────────┬───────────────────────────────────┘│Codex ◄── stdio MCP proxy ──► /mcp (find, search, read,(env/ovcli.conf) remember, resources, watches,@@ -188,24 +189,23 @@ For details on OpenViking's MCP endpoint, tools, and protocol, see the [MCP Inte> See [`DESIGN.md`](./DESIGN.md) for the commit decision tree — it's the source of truth for *which* OpenViking session is sealed by *which* hook event.-### SessionStart profile injection and commit logic+### SessionStart profile injection and fallback sweep-Codex fires `SessionStart` with one of three `source` values: `startup` (fresh process / `/new` / zouk daemon spawn-without-sessionId), `resume` (`/resume` or short reconnect), and `clear` (`/clear` — the previous transcript is orphaned and a new session_id is created). `resume` never commits or sweeps; on `startup` and `clear` we run the same active-window heuristic.+Codex fires `SessionStart` with one of three `source` values: `startup` (fresh process / `/new` / zouk daemon spawn-without-sessionId), `resume` (`/resume` or short reconnect), and `clear` (`/clear` — the previous transcript is orphaned and a new session_id is created). `resume` never commits or sweeps; on `startup` and `clear` the hook runs the fallback sweep.-`hooks.json` registers `SessionStart` with `matcher: "clear|startup|resume"` so codex's dispatcher invokes the script on all three relevant sources. `session-start-commit.mjs` gates internally so only `startup` and `clear` commit/sweep.+`hooks.json` registers `SessionStart` with `matcher: "clear|startup|resume"` so codex's dispatcher invokes the script on all three relevant sources. `session-start-commit.mjs` gates internally so only `startup` and `clear` sweep.On all three sources, the hook uses the same shared `buildProfileBlock()` implementation as the Claude Code, OpenCode, and pi integrations. It reads the user's `profile.md` and adds URI plus abstract indexes for `preferences/` and `entities/`, with a CJK-aware token budget. The default budget is `10000`; set `OPENVIKING_PROFILE_TOKEN_BUDGET` or `plugin.codex.profileTokenBudget` to change it. Set `OPENVIKING_NO_AUTO_INJECT=1` or `plugin.codex.noAutoInject=true` to disable only this fixed profile/background injection; per-prompt semantic recall remains controlled separately by `OPENVIKING_AUTO_RECALL`.-On `startup` or `clear`, the script:+On `startup` or `clear`, the script walks every state file except the new session_id and, for each one that still holds a live `ovSessionId` or carries an end marker:-1. Counts state files (excluding the new session_id) whose `lastUpdatedAt` is within `OPENVIKING_CODEX_ACTIVE_WINDOW_MS` (default 2 min) of "now":- - **0 active** → no-op (no orphan to commit)- - **1 active** → commit it (the just-ended session), unless it has no live `ovSessionId` left to commit- - **≥2 active** → skip; rely on idle TTL (we can't tell which one ended)-2. **Idle-TTL sweep at the tail**: any live session state older than `OPENVIKING_CODEX_IDLE_TTL_MS` (default 30 min) gets committed while preserving its transcript cursor for resume.+1. **`ended_retry`**: an `.ended.<timestamp>` marker is present, meaning `SessionEnd` fired but its commit never completed (server down, worker killed). Commit it now. A marker is swept even when the state has no live `ovSessionId`: `PreCompact` releases the id but leaves the cursor behind, so the catch-up under the lock is the only way the tail turns are ever sent, and it derives a live id by itself as soon as it has something to send.+2. **`idle_ttl`**: no marker, but the state has been idle for more than `OPENVIKING_CODEX_IDLE_TTL_MS` (default 30 min). This is the path for exits that never fire `SessionEnd` — signals, crashes, Codex older than 0.145, and app-server threads whose `SessionEnd` is deferred.3. **Cursor retention in the same pass**: a state file with no live OV session is kept as a resume cursor until `OPENVIKING_CODEX_COMMITTED_TTL_MS` (default 30 days), or dropped after the idle TTL if it never captured a turn.-On any /commit failure (OV unreachable, non-2xx, timeout) we **preserve state** (keep `ovSessionId` set) so the next sweep can retry.+Each candidate is committed under its per-session lock with no waiting; a lock the sweep cannot take means a `SessionEnd` or `Stop` worker already owns that session, and the sweep logs the skip and moves on. Under the lock it first appends whatever the state's recorded `transcriptPath` still holds past the cursor, so a session whose own workers never ran is not archived without its tail turns; if part of that append fails it keeps the live session and the marker and leaves the commit to the next sweep. It also re-reads the `.ended` marker there: an `ended_retry` candidate whose marker is now gone (the thread was resumed) or newer than the snapshot (a later exit will commit it) falls back to the idle rule. A recorded `transcriptPath` that cannot be read is never mistaken for an empty transcript: the sweep logs `transcript_unreadable`, keeps the live session and the marker, and skips the commit. Commits preserve the transcript cursor for resume.++On any /commit failure (OV unreachable, non-2xx, timeout) we **preserve state** (keep `ovSessionId` set, and keep the `.ended` marker) so the next sweep can retry. `SessionEnd` and `PreCompact` apply the same unreadable-transcript guard as the sweep, so neither commits a session whose transcript it could not read.On `resume`, the script skips commit/sweep. It still injects the profile block. If local state has no live `ovSessionId`, it also reads `/api/v1/sessions/{cx-session-id}/context` and combines the latest committed archive overview into the same `SessionStart` output. The archive block includes a `viking://~/sessions/{cx-session-id}/history/` URI and tells the model to use the OpenViking MCP `read`/`search` tools for exact prior commands, file paths, tool outputs, or messages. Set `OPENVIKING_RESUME_ARCHIVE_INJECT=0` to disable the archive half without disabling profile injection.@@ -273,12 +273,19 @@ After a successful append, Stop reads the session meta and commits when `pending2. Commit the long-lived OV session so the extractor runs against the full pre-compact transcript3. Reset `ovSessionId` to `null` so the next `Stop` re-derives the same `cx-<safe-session-id>` and appends the post-compact half under that deterministic OV session id-### Known gap: SIGTERM / Ctrl+C / `/exit` are silent+### Session end++`SessionEnd` exists since Codex `rust-v0.145.0`. It fires when a thread shuts down gracefully — `/quit`, `/exit`, double Ctrl-C, EOF, and the end of a `codex exec` run — and at TUI exit every thread the process touched gets one, as a burst. `/new` on its own does not end the previous thread; its `SessionEnd` arrives when the process exits.++`session-end.mjs` catches up whatever turns the last `Stop` never sent, then commits the OV session so the extractor runs on the whole conversation. If any of those turns fail to land it keeps the live session and the marker instead of committing, so the sweep retries rather than archiving a conversation without its tail. Codex budgets the hook at 1s by default and clamps `timeout` in `hooks.json` to 3s, forces `async: true` hooks to run synchronously, and ignores their stdout — far too little for a commit. So the parent hook only writes an `.ended` marker next to the session state (lock-free, a millisecond) and detaches a worker that does the catch-up and the commit; Codex deliberately leaves cleanly detached helpers running after a hook exits.++`SessionEnd` does not fire on `SIGTERM`, `SIGHUP`, a closed terminal, `kill -9`, or a crash. When the TUI is attached to a `codex app-server` daemon, it is deferred to thread unload (30 min) or daemon shutdown. Those cases, and Codex older than 0.145 (and any TraeCode CLI build without it), are covered by the fallback sweep at the next `SessionStart`.++The `.ended.<timestamp>` marker and the per-session `.lock` directory live beside the state file. The timestamp it was written at is the marker's identity, and it is part of the filename: the `SessionEnd` parent hands it to its worker, which verifies the marker still matches before committing and returns untouched if it does not, and `Stop` / `PreCompact` / `resume` only clear markers older than their own start time. Because each removal unlinks the exact marker paths below its cutoff, a marker written while a removal is in flight is a different file and survives, so a late worker cannot erase a fresh exit's marker. `Date.now()` is only the starting point for that name: the marker is created exclusively and its timestamp bumped until that succeeds, so two exits within one millisecond cannot share a path. A bare `<id>.ended` written by an older build is still read back.-Codex fires no hook on process exit. `/compact` is the only fully-deterministic "context disappearing" signal. If you `/exit` without `/compact`, the OV session for that codex session_id stays open. Two fallbacks recover the orphan:+The lock serializes the four writers that persist the whole state object — the `Stop` worker, `PreCompact`, the `SessionEnd` worker, and the sweep — so none of them can clobber another's cursor or `ovSessionId`. The holder stamps an `owner` file inside the lock directory and releases only while it still owns it; a stale lock is taken over in place by claiming that `owner` file — an atomic rename aside followed by an exclusive create, so exactly one taker wins and the lock path is never momentarily absent. Its wait budget is `OPENVIKING_CODEX_LOCK_WAIT_MS` (default 120s for `SessionEnd`, 40s for `PreCompact`, which must answer inside a 60s hook budget); the sweep never waits.-1. The idle-TTL sweep at the next `SessionStart` commits any state file older than 30 min-2. The active-window heuristic catches it if you `/new` or `/clear` shortly after+> **Upgrading from 0.7.x**: `SessionEnd` is a newly registered hook event, and Codex has no trust record for it. Run `/hooks` in Codex after updating and approve it, otherwise it silently never runs and every session falls back to the sweep.## Codex hook output schema@@ -290,6 +297,7 @@ Codex's hook output schema differs from Claude Code's. Notably:| `UserPromptSubmit` | `prompt`, `session_id` | `hookSpecificOutput.additionalContext` || `Stop` | `last_assistant_message`, `transcript_path`, `session_id` | `systemMessage` (only) || `PreCompact` | `trigger` (`manual`/`auto`), `transcript_path`, `session_id` | `systemMessage` (only) |+| `SessionEnd` | `session_id`, `transcript_path`, `cwd`, `reason` (constant `other`) | none — Codex ignores the output; the script prints `{}` for symmetry |Unlike Claude Code, **Codex does not support `decision: "approve"`**; only `decision: "block"`. A no-op is `{}` (which is what these scripts emit when there's nothing to add).@@ -310,9 +318,9 @@ codex-memory-plugin/├── .codex-plugin/│ └── plugin.json # Plugin manifest (hooks + mcp wiring)├── hooks/-│ └── hooks.json # SessionStart + UserPromptSubmit + Stop + PreCompact-│ (uses Codex's native ${PLUGIN_ROOT} token; no-│ rendering needed on modern Codex)+│ └── hooks.json # SessionStart + UserPromptSubmit + Stop + SessionEnd+│ + PreCompact (uses Codex's native ${PLUGIN_ROOT}+│ token; no rendering needed on modern Codex)├── skills/│ ├── openviking-memory/ # How to use the memory tools│ ├── ov-experience-memory/@@ -323,11 +331,14 @@ codex-memory-plugin/│ ├── capture-utils.mjs # Transcript text extraction, filtering, tool compression│ ├── debug-log.mjs # Structured JSONL logger│ ├── recall-compressor-profile.mjs # Compressor profile detection/cache-│ ├── session-state.mjs # Per-codex-session OV session state+│ ├── session-state.mjs # Per-codex-session OV session state (+ .ended.<ts> / .lock sidecars)+│ ├── ov-session.mjs # Shared OV HTTP + transcript catch-up helpers│ ├── auto-recall.mjs # UserPromptSubmit hook (REST /search/search)│ ├── auto-capture.mjs # Stop hook (append + threshold commit)-│ ├── session-start-commit.mjs # SessionStart hook (profile + active-window + idle TTL + resume archive)-│ └── pre-compact-capture.mjs # PreCompact hook+│ ├── session-start-commit.mjs # SessionStart hook (profile + fallback sweep + resume archive)+│ ├── session-end.mjs # SessionEnd hook (mark + detached catch-up + commit)+│ ├── pre-compact-capture.mjs # PreCompact hook+│ └── *.test.mjs # node --test suites (session-end, pre-compact, ...)├── servers/│ ├── mcp-proxy.mjs # stdio -> OpenViking /mcp bridge│ └── mcp-proxy.test.mjs # proxy contract testsdiff --git a/examples/codex-memory-plugin/VERIFICATION.md b/examples/codex-memory-plugin/VERIFICATION.mdindex 79ff12ac4..e2e9d82b0 100644--- a/examples/codex-memory-plugin/VERIFICATION.md+++ b/examples/codex-memory-plugin/VERIFICATION.md@@ -121,63 +121,74 @@ echo '{"session_id":"verify-sess","transcript_path":"'"$STATE_DIR"'/transcript.jExpect: `appended 2 turn(s) to OpenViking session cx-verify-sess`.-## 6. SessionStart — active-window heuristic + idle-TTL sweep+## 6. SessionEnd commit + SessionStart fallback sweep-`source=startup` and `source=clear` both run the same logic+`SessionEnd` (Codex ≥ 0.145) is the primary commit path; `SessionStart`+`source=startup|clear` sweeps only what `SessionEnd` could not reach(matcher = `clear|startup|resume`). `source=resume` never commits or sweeps;all three sources inject the shared profile/background block by default, andresume may additionally inject latest archive context if a committed archiveexists. Set `OPENVIKING_NO_AUTO_INJECT=1` when a cleanup-only smoke test needsthe historical `{}` output.-See `DESIGN.md` §3 + §5 for the full decision tree.+See `DESIGN.md` §2 + §5 for the full decision tree.-### 6a. `1 active` → commit+### 6a. SessionEnd — catch-up append + commit-After step 5, the `verify-sess` state file is fresh (touched within the-last 2 min) and is the only state file other than the new session_id.-Heuristic should commit it.+After step 5, `verify-sess` has a live `cx-verify-sess` and a cursor of 6.+Append two more turns that `Stop` never saw, then run the worker path+directly (`OV_HOOK_WORKER=1` skips the detach, so the run is observable).```bash-echo '{"session_id":"new-after-verify","source":"startup","cwd":"/tmp","model":"x","permission_mode":"default","transcript_path":null,"hook_event_name":"SessionStart"}' \+cat >> "$STATE_DIR/transcript.jsonl" <<'EOF'+{"payload":{"role":"user","content":"One last thing: I ship on Fridays."}}+{"payload":{"role":"assistant","content":"Noted — Friday releases."}}+EOF++echo '{"session_id":"verify-sess","transcript_path":"'"$STATE_DIR"'/transcript.jsonl","cwd":"/tmp","hook_event_name":"SessionEnd","reason":"other"}' \| OPENVIKING_CONFIG_FILE=$OV_CONF \OPENVIKING_CODEX_STATE_DIR=$STATE_DIR/state \CODEX_PLUGIN_ROOT=$PLUGIN \- node $PLUGIN/scripts/session-start-commit.mjs+ OV_HOOK_WORKER=1 \+ OPENVIKING_DEBUG=1 \+ node $PLUGIN/scripts/session-end.mjs```-Expect: `hookSpecificOutput.additionalContext` contains the OpenViking profile-block, and `systemMessage` is-`OpenViking session cx-verify-sess is committed`.-After this `verify-sess.json` is gone from `$STATE_DIR/state`.+Expect: `{}` on stdout (SessionEnd output is ignored by Codex). In+`~/.openviking/logs/codex-hooks.log`: `appended_catchup` with `added: 2`,+then `commit` with `"reason":"session_end"`.-### 6b. `0 active` → no-op+```bash+cat $STATE_DIR/state/verify-sess.json+# ovSessionId is null, capturedTurnCount is 8 (cursor preserved for resume)+ls $STATE_DIR/state/verify-sess.ended.* # no such file — the marker was cleared+```++### 6b. SessionEnd parent — marker first, work detached```bash-# State dir empty (after 6a). Fire SessionStart-startup again.-echo '{"session_id":"another-fresh","source":"startup","cwd":"/tmp","model":"x","permission_mode":"default","transcript_path":null,"hook_event_name":"SessionStart"}' \+time (echo '{"session_id":"verify-sess","transcript_path":"'"$STATE_DIR"'/transcript.jsonl","cwd":"/tmp","hook_event_name":"SessionEnd","reason":"other"}' \| OPENVIKING_CONFIG_FILE=$OV_CONF \OPENVIKING_CODEX_STATE_DIR=$STATE_DIR/state \CODEX_PLUGIN_ROOT=$PLUGIN \- node $PLUGIN/scripts/session-start-commit.mjs-# Expect: profile context in hookSpecificOutput.additionalContext.-# Add OPENVIKING_NO_AUTO_INJECT=1 to expect {} (no orphan to commit).+ node $PLUGIN/scripts/session-end.mjs)```-### 6c. `≥2 active` → skip; rely on idle TTL+Expect: `{}` well under 1 s — the parent only writes+`$STATE_DIR/state/verify-sess.ended.<timestamp>` and detaches the worker. The worker+finds nothing live to commit and removes the marker again shortly after.++### 6c. `.ended.<ts>` marker → next SessionStart commits immediately```bash-# Manufacture two fresh state files for different session_ids (no ovSessionId-# so no real commit needed, just exercise the skip-path log).+# A live session whose SessionEnd worker never finished: fresh timestamp,+# so only the marker can make the sweep pick it up.NOW=$(node -e 'console.log(Date.now())')mkdir -p "$STATE_DIR/state"-cat > "$STATE_DIR/state/sess-aaa.json" <<EOF-{"codexSessionId":"sess-aaa","ovSessionId":null,"capturedTurnCount":0,"createdAt":$NOW,"lastUpdatedAt":$NOW}-EOF-cat > "$STATE_DIR/state/sess-bbb.json" <<EOF-{"codexSessionId":"sess-bbb","ovSessionId":null,"capturedTurnCount":0,"createdAt":$NOW,"lastUpdatedAt":$NOW}+cat > "$STATE_DIR/state/sess-ended.json" <<EOF+{"codexSessionId":"sess-ended","ovSessionId":"cx-sess-ended","capturedTurnCount":2,"createdAt":$NOW,"lastUpdatedAt":$NOW}EOF+printf '%s' "$NOW" > "$STATE_DIR/state/sess-ended.ended.$NOW"-OPENVIKING_DEBUG=1 \echo '{"session_id":"sess-ccc","source":"startup","cwd":"/tmp","model":"x","permission_mode":"default","transcript_path":null,"hook_event_name":"SessionStart"}' \| OPENVIKING_CONFIG_FILE=$OV_CONF \OPENVIKING_CODEX_STATE_DIR=$STATE_DIR/state \@@ -186,14 +197,40 @@ echo '{"session_id":"sess-ccc","source":"startup","cwd":"/tmp","model":"x","permnode $PLUGIN/scripts/session-start-commit.mjs```-Expect: profile context on stdout. In `~/.openviking/logs/codex-hooks.log` look for-`"branch":">=2_active","action":"skip; rely on idle TTL"`. The two state-files are still present — the skip path does not clear them.+Expect: `systemMessage` reports `cx-sess-ended` committed, and the log shows+`"reason":"ended_retry"` despite the fresh `lastUpdatedAt`. Afterwards+`sess-ended.json` has `ovSessionId: null` with `capturedTurnCount: 2`, and+`sess-ended.ended.$NOW` is gone.++### 6c-2. Held lock → sweep skips instead of racing++```bash+# Re-arm the same session, then hold its lock as a concurrent worker would.+cat > "$STATE_DIR/state/sess-ended.json" <<EOF+{"codexSessionId":"sess-ended","ovSessionId":"cx-sess-ended","capturedTurnCount":2,"createdAt":$NOW,"lastUpdatedAt":$NOW}+EOF+printf '%s' "$NOW" > "$STATE_DIR/state/sess-ended.ended.$NOW"+mkdir "$STATE_DIR/state/sess-ended.lock"++echo '{"session_id":"sess-fff","source":"startup","cwd":"/tmp","model":"x","permission_mode":"default","transcript_path":null,"hook_event_name":"SessionStart"}' \+ | OPENVIKING_CONFIG_FILE=$OV_CONF \+ OPENVIKING_CODEX_STATE_DIR=$STATE_DIR/state \+ CODEX_PLUGIN_ROOT=$PLUGIN \+ OPENVIKING_DEBUG=1 \+ node $PLUGIN/scripts/session-start-commit.mjs++rmdir "$STATE_DIR/state/sess-ended.lock"+```++Expect: the log shows `sweep_skip` with `"reason":"locked by another writer"`,+no `/commit` is issued, and `sess-ended.json` keeps `ovSessionId` and its+marker. (A lock older than 5 min is treated as stale and taken over instead.)-### 6d. Idle-TTL sweep at the tail+### 6d. Idle-TTL sweep```bash-# Backdate one of the state files to be older than IDLE_TTL_MS (default 30 min).+# A live state with no marker, backdated past IDLE_TTL_MS (default 30 min):+# the signal/crash/old-Codex path.OLD=$(node -e 'console.log(Date.now() - 60*60*1000)') # 1 hour agocat > "$STATE_DIR/state/sess-aaa.json" <<EOF{"codexSessionId":"sess-aaa","ovSessionId":"cx-sess-aaa","capturedTurnCount":2,"createdAt":$OLD,"lastUpdatedAt":$OLD}@@ -206,12 +243,11 @@ echo '{"session_id":"sess-ddd","source":"startup","cwd":"/tmp","model":"x","permnode $PLUGIN/scripts/session-start-commit.mjs```-Expect: log shows `idle_sweep` for `sess-aaa` (committed).-`sess-bbb.json` is still present (still fresh). `sess-aaa.json` is also-present with `ovSessionId: null` and `capturedTurnCount: 2` for resume.-If `sess-bbb` was in `≥2 active` from 6c, the heuristic on this call sees-just `sess-bbb` (1 active) and commits it — that's expected and shows the-heuristic + sweep working together.+Expect: log shows `sweep` for `sess-aaa` with `"reason":"idle_ttl"`, followed+by the commit. `sess-aaa.json` is still present, with `ovSessionId: null` and+`capturedTurnCount: 2` for resume. Any other state file that is both fresh and+unmarked is left untouched — that is the whole point of dropping the old+active-window heuristic.### 6d-2. Cursor retentiondiff --git a/examples/codex-memory-plugin/hooks/hooks.json b/examples/codex-memory-plugin/hooks/hooks.jsonindex 4ac2caaf6..4e58859a5 100644--- a/examples/codex-memory-plugin/hooks/hooks.json+++ b/examples/codex-memory-plugin/hooks/hooks.json@@ -36,6 +36,18 @@]}],+ "SessionEnd": [+ {+ "matcher": "*",+ "hooks": [+ {+ "type": "command",+ "command": "node ${PLUGIN_ROOT}/scripts/session-end.mjs",+ "timeout": 3+ }+ ]+ }+ ],"PreCompact": [{"matcher": "*",diff --git a/examples/codex-memory-plugin/scripts/auto-capture.mjs b/examples/codex-memory-plugin/scripts/auto-capture.mjsindex c1b3b7f16..9a2e23236 100644--- a/examples/codex-memory-plugin/scripts/auto-capture.mjs+++ b/examples/codex-memory-plugin/scripts/auto-capture.mjs@@ -14,33 +14,36 @@* 3. If session pending_tokens crosses commitTokenThreshold, commit while* keeping a recent live tail for continuity.*- * PreCompact still commits deterministically before context compaction, and- * SessionStart still handles orphaned sessions / idle TTL sweep.+ * A Stop for this session also proves the thread is alive again, so the+ * SessionEnd marker (if any) is cleared before anything else. Committing is+ * SessionEnd's job; PreCompact still commits before context compaction and+ * the SessionStart sweep remains the fallback for exits that never fire it.** Stop output schema accepts {} as a no-op.- *- * Note: we deliberately do NOT run an idle-TTL sweep here. State-write-on-- * every-turn already gives us the freshness signal we need; running the- * sweep once per session start (in session-start-commit.mjs) is the right- * cadence. See DESIGN.md §5 ("Sweep trigger").*/-import { readFile } from "node:fs/promises";-import {- extractCaptureTurns,- findLastHumanTurnIndex,-} from "./capture-utils.mjs";import { loadConfig } from "./config.mjs";import { createLogger } from "./debug-log.mjs";-import { loadState, resolveOvSessionId, saveState } from "./session-state.mjs";+import { catchUpTurns, hasCaptureKeyword, makeFetchJSON } from "./ov-session.mjs";+import { clearEnded, loadState, saveState, withSessionLock } from "./session-state.mjs";import { maybeDetach, readHookStdin } from "./shared/async-writer.mjs";-import { sendSessionMessages } from "./shared/batch-send.mjs";import { resolveEffectivePeerId } from "./shared/workspace-peer.mjs";const cfg = loadConfig();-const { log, logError } = createLogger("auto-capture");+const { log, logError } = createLogger("auto-capture", cfg);let activePeerId = cfg.peerId || "";+const LOCK_WAIT_MS = 120_000;++// A detached worker can boot long after the turn ended, so it clears end+// markers against the parent's start time rather than its own.+const HOOK_STARTED_AT = (() => {+ const inherited = Number(process.env.OPENVIKING_HOOK_STARTED_AT);+ return Number.isFinite(inherited) && inherited > 0 ? inherited : Date.now();+})();++const { fetchJSONRes, fetchJSON } = makeFetchJSON(cfg, { getActorPeerId: () => activePeerId });+function output(obj) {process.stdout.write(JSON.stringify(obj) + "\n");}@@ -49,107 +52,15 @@ function noop(message) {output(message ? { systemMessage: message } : {});}-function makeHeaders() {- const headers = { "Content-Type": "application/json" };- if (cfg.apiKey) {- headers["Authorization"] = `Bearer ${cfg.apiKey}`;- headers["X-API-Key"] = cfg.apiKey;- }- if (cfg.sendIdentityHeaders && cfg.account) headers["X-OpenViking-Account"] = cfg.account;- if (cfg.sendIdentityHeaders && cfg.user) headers["X-OpenViking-User"] = cfg.user;- if (activePeerId) headers["X-OpenViking-Actor-Peer"] = activePeerId;- if (cfg.userAgent) headers["User-Agent"] = cfg.userAgent;- return headers;-}--function responseTraceId(body) {- return body?.result?.trace_id || body?.error?.trace_id || body?.trace_id || undefined;-}--async function fetchJSONRes(path, init = {}) {- const controller = new AbortController();- const timer = setTimeout(() => controller.abort(), cfg.captureTimeoutMs);- try {- const res = await fetch(`${cfg.baseUrl}${path}`, { ...init, headers: makeHeaders(), signal: controller.signal });- const body = await res.json().catch(() => null);- if (!body) return { ok: false, status: res.status, error: { message: "empty or invalid JSON response" } };- const traceId = responseTraceId(body);- if (!res.ok || body.status === "error") {- return { ok: false, status: res.status, error: body.error || body, traceId };- }- return { ok: true, status: res.status, result: body.result ?? body, traceId };- } catch (err) {- return { ok: false, status: 0, error: { message: err?.message || String(err) } };- } finally {- clearTimeout(timer);- }-}--async function fetchJSON(path, init = {}) {- const r = await fetchJSONRes(path, init);- return r.ok ? (r.result ?? null) : null;-}--// ----------------------------------------------------------------------------// Transcript parsing (JSONL rollout)-// -----------------------------------------------------------------------------function parseTranscript(content) {- try {- const data = JSON.parse(content);- if (Array.isArray(data)) return data;- } catch { /* not a JSON array */ }- const lines = content.split("\n").filter((l) => l.trim());- const out = [];- for (const line of lines) {- try { out.push(JSON.parse(line)); } catch { /* skip */ }- }- return out;-}--function extractTurns(rolloutEntries) {- return extractCaptureTurns(rolloutEntries, cfg);-}--async function readTranscriptTurns(transcriptPath) {- if (!transcriptPath) return [];- try {- const raw = await readFile(transcriptPath, "utf-8");- if (!raw.trim()) return [];- return extractTurns(parseTranscript(raw));- } catch (err) {- logError("transcript_read", err);- return [];- }-}--async function appendTurns(ovSessionId, turns, state) {- const payloads = turns.map((turn) => {- const body = turn.parts?.length- ? { role: turn.role, parts: turn.parts }- : { role: turn.role, content: turn.text };- if (activePeerId) body.peer_id = activePeerId;- return body;- });- const r = await sendSessionMessages(fetchJSONRes, ovSessionId, payloads, {- onSent: async (n) => {- state.capturedTurnCount += n;- await saveState(state);- },- });- return r.sent;-}-async function maybeCommitByThreshold(ovSessionId, added) {- if (added <= 0) {- return {- committed: false,- pendingTokens: 0,- commitCount: 0,- totalMessageCount: 0,- traceId: "",- };- }+ const empty = {+ committed: false,+ pendingTokens: 0,+ commitCount: 0,+ totalMessageCount: 0,+ traceId: "",+ };+ if (added <= 0) return empty;const meta = await fetchJSON(`/api/v1/sessions/${encodeURIComponent(ovSessionId)}`);const pendingTokens = Number(meta?.pending_tokens || 0);const commitCount = Number(meta?.commit_count || 0);@@ -186,9 +97,43 @@ async function maybeCommitByThreshold(ovSessionId, added) {};}-// ----------------------------------------------------------------------------// Main-// ---------------------------------------------------------------------------+async function capture(sessionId, transcriptPath, heartbeat) {+ const state = await loadState(sessionId);+ activePeerId = cfg.peerId || state.workspacePeerId || resolveEffectivePeerId({ cfg, cwd: process.cwd() }).peerId;+ log("start", { sessionId, transcriptPath, hasPeer: Boolean(activePeerId) });++ const health = await fetchJSON("/health");+ if (!health) {+ logError("health_check", "server unreachable or unhealthy");+ return "";+ }++ const { added, ovSessionId } = await catchUpTurns({+ state,+ transcriptPath,+ fetchJSONRes,+ activePeerId,+ cfg,+ log,+ logError,+ heartbeat,+ shouldSend: (turns) => cfg.captureMode !== "keyword" || hasCaptureKeyword(turns),+ });++ let commitInfo = { committed: false, traceId: "" };+ if (added > 0) {+ log("appended", { ovSessionId, added });+ commitInfo = await maybeCommitByThreshold(ovSessionId, added);+ }++ await saveState(state);++ if (added <= 0) return "";+ return `appended ${added} turn(s) to OpenViking session ${state.ovSessionId}` ++ (commitInfo.committed+ ? ` (committed${commitInfo.traceId ? `; trace_id=${commitInfo.traceId}` : ""})`+ : "");+}async function main() {if (!cfg.autoCapture) {@@ -199,6 +144,7 @@ async function main() {// Async write mode returns a no-op response immediately; worker stdout is// intentionally discarded, so appended-count systemMessage is sync-only.+ process.env.OPENVIKING_HOOK_STARTED_AT = String(HOOK_STARTED_AT);if (await maybeDetach(cfg, { approve: () => output({}) })) return;let input;@@ -212,90 +158,22 @@ async function main() {const sessionId = input.session_id || "unknown";const transcriptPath = input.transcript_path || null;- const state = await loadState(sessionId);- activePeerId = cfg.peerId || state.workspacePeerId || resolveEffectivePeerId({ cfg, cwd: process.cwd() }).peerId;- log("start", { sessionId, transcriptPath, hasPeer: Boolean(activePeerId) });- const health = await fetchJSON("/health");- if (!health) {- logError("health_check", "server unreachable or unhealthy");+ // A turn ended for this session, so it is alive again after any resume — but+ // only for markers older than this hook run.+ await clearEnded(sessionId, { before: HOOK_STARTED_AT });++ const outcome = await withSessionLock(+ sessionId,+ ({ heartbeat }) => capture(sessionId, transcriptPath, heartbeat),+ { waitMs: LOCK_WAIT_MS },+ );+ if (outcome.skipped) {+ logError("lock_timeout", `another writer holds ${sessionId}; leaving state untouched`);noop();return;}-- const allTurns = await readTranscriptTurns(transcriptPath);-- // Post-compact transcript-shrink defense: codex's /compact may rewrite or- // truncate transcript_path. If allTurns has fewer entries than we cached,- // our slice math would underflow and silently drop turns. Resume at the- // latest human turn so the current interaction is captured without replaying- // compacted history. See DESIGN.md "Post-compact transcript shrink".- if (allTurns.length < state.capturedTurnCount) {- const humanIndex = findLastHumanTurnIndex(allTurns);- log("transcript_shrink_detected", {- cached: state.capturedTurnCount,- observed: allTurns.length,- resumeFrom: Math.max(0, humanIndex),- // -1 means no human turn survived the rewrite; capturing the whole- // transcript is the only way not to lose the interaction.- fallback: humanIndex < 0 ? "full_transcript" : undefined,- });- state.capturedTurnCount = Math.max(0, humanIndex);- }-- const newTurns = allTurns.slice(state.capturedTurnCount);-- log("transcript_parse", {- totalTurns: allTurns.length,- previouslyCaptured: state.capturedTurnCount,- newTurns: newTurns.length,- });-- if (cfg.captureMode === "keyword" && newTurns.length > 0 && !hasCaptureKeyword(newTurns)) {- log("skip", { stage: "capture_mode", reason: "keyword mode without capture trigger" });- await saveState(state);- noop();- return;- }-- let added = 0;- let ovSessionId = "";- let commitInfo = {- committed: false,- pendingTokens: 0,- commitCount: 0,- totalMessageCount: 0,- traceId: "",- };- if (newTurns.length > 0) {- ovSessionId = resolveOvSessionId(state);- if (!ovSessionId) {- logError("resolve_ov_session", "failed to derive OV session id");- } else {- added = await appendTurns(ovSessionId, newTurns, state);- log("appended", { ovSessionId, added });- commitInfo = await maybeCommitByThreshold(ovSessionId, added);- }- }-- await saveState(state);-- // could also sweep here, deliberately not — see header comment + DESIGN.md §5.-- if (added > 0) {- noop(- `appended ${added} turn(s) to OpenViking session ${state.ovSessionId}` +- (commitInfo.committed- ? ` (committed${commitInfo.traceId ? `; trace_id=${commitInfo.traceId}` : ""})`- : ""),- );- } else {- noop();- }-}--function hasCaptureKeyword(turns) {- return turns.some((turn) => /\b(remember|memorize|store|save|capture|note|record)\b|记住|保存|记录|记忆/i.test(turn.text));+ noop(outcome.value);}main().catch((err) => { logError("uncaught", err); noop(); });diff --git a/examples/codex-memory-plugin/scripts/auto-capture.test.mjs b/examples/codex-memory-plugin/scripts/auto-capture.test.mjsindex 8aee2c283..cb5c51856 100644--- a/examples/codex-memory-plugin/scripts/auto-capture.test.mjs+++ b/examples/codex-memory-plugin/scripts/auto-capture.test.mjs@@ -1,6 +1,6 @@import assert from "node:assert/strict";import { spawn } from "node:child_process";-import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";+import { mkdtemp, readFile, readdir, rm, stat, writeFile } from "node:fs/promises";import http from "node:http";import { tmpdir } from "node:os";import { dirname, join } from "node:path";@@ -9,6 +9,24 @@ import { fileURLToPath } from "node:url";const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url));+async function endedMarkerStamps(dir, id) {+ const prefix = `${id}.ended.`;+ const files = await readdir(dir).catch(() => []);+ return files+ .filter((name) => name.startsWith(prefix))+ .map((name) => Number(name.slice(prefix.length)))+ .filter((ts) => Number.isFinite(ts));+}++async function endedMarkerExists(dir, id) {+ return (await endedMarkerStamps(dir, id)).length > 0;+}++function writeEndedMarker(dir, id, ts) {+ return writeFile(join(dir, `${id}.ended.${ts}`), String(ts));+}++function readRequestBody(req) {return new Promise((resolve, reject) => {const chunks = [];@@ -463,3 +481,146 @@ test("auto-capture skips compacted history after transcript shrink", async () =>await rm(stateDir, { recursive: true, force: true });}});++test("auto-capture clears a stale session-end marker and never resets the cursor", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-auto-capture-ended-"));+ const transcriptPath = join(stateDir, "transcript.jsonl");+ const batches = [];+ const now = Date.now();++ try {+ await writeFile(join(stateDir, "resumed.json"), JSON.stringify({+ codexSessionId: "resumed",+ ovSessionId: "cx-resumed",+ capturedTurnCount: 1,+ createdAt: now - 1000,+ lastUpdatedAt: now,+ }));+ await writeEndedMarker(stateDir, "resumed", now);+ await writeFile(+ transcriptPath,+ [+ { payload: { message: { role: "user", content: "first request" } } },+ { payload: { message: { role: "user", content: "second request" } } },+ ].map((entry) => JSON.stringify(entry)).join("\n"),+ );++ const env = (baseUrl) => ({+ OPENVIKING_AUTO_CAPTURE: "1",+ OPENVIKING_CODEX_STATE_DIR: stateDir,+ OPENVIKING_CONFIG_FILE: join(stateDir, "missing-ov.conf"),+ OPENVIKING_CLI_CONFIG_FILE: join(stateDir, "missing-ovcli.conf"),+ OPENVIKING_CREDENTIAL_SOURCE: "env",+ OPENVIKING_MIN_QUERY_LENGTH: "1",+ OPENVIKING_WRITE_PATH_ASYNC: "0",+ OPENVIKING_TIMEOUT_MS: "5000",+ OPENVIKING_URL: baseUrl,+ });++ await withMockOpenViking(async (req, res) => {+ const url = new URL(req.url, "http://127.0.0.1");+ if (req.method === "GET" && url.pathname === "/health") {+ writeJson(res, { status: "ok", result: { ok: true } });+ return;+ }+ if (req.method === "POST" && url.pathname.endsWith("/messages/batch")) {+ batches.push(await readRequestBody(req));+ writeJson(res, { status: "ok", result: { ok: true } });+ return;+ }+ if (req.method === "GET" && url.pathname === "/api/v1/sessions/cx-resumed") {+ writeJson(res, { status: "ok", result: { pending_tokens: 0, commit_count: 0, total_message_count: 2 } });+ return;+ }+ res.writeHead(404, { "Content-Type": "application/json" });+ res.end(JSON.stringify({ status: "error", error: "not found" }));+ }, async (baseUrl) => {+ await runAutoCapture({ session_id: "resumed", transcript_path: transcriptPath }, env(baseUrl));++ const marker = await endedMarkerExists(stateDir, "resumed");+ assert.equal(marker, false, "a Stop proves the thread is alive again");+ assert.equal(+ JSON.parse(await readFile(join(stateDir, "resumed.json"), "utf-8")).capturedTurnCount,+ 2,+ );++ // An unreadable transcript must not look like a shrink.+ await runAutoCapture(+ { session_id: "resumed", transcript_path: join(stateDir, "gone.jsonl") },+ env(baseUrl),+ );+ assert.equal(+ JSON.parse(await readFile(join(stateDir, "resumed.json"), "utf-8")).capturedTurnCount,+ 2,+ );+ });++ assert.equal(batches.flatMap((batch) => batch.messages || []).length, 1);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("a Stop clears only end markers older than the hook run", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-auto-capture-marker-"));+ const transcriptPath = join(stateDir, "transcript.jsonl");+ const startedAt = Date.now();++ try {+ await writeFile(transcriptPath, JSON.stringify({+ payload: { message: { role: "user", content: "hello" } },+ }));++ const env = (baseUrl, extra) => ({+ OPENVIKING_AUTO_CAPTURE: "1",+ OPENVIKING_CODEX_STATE_DIR: stateDir,+ OPENVIKING_CONFIG_FILE: join(stateDir, "missing-ov.conf"),+ OPENVIKING_CLI_CONFIG_FILE: join(stateDir, "missing-ovcli.conf"),+ OPENVIKING_CREDENTIAL_SOURCE: "env",+ OPENVIKING_MIN_QUERY_LENGTH: "1",+ OPENVIKING_WRITE_PATH_ASYNC: "0",+ OPENVIKING_TIMEOUT_MS: "5000",+ OPENVIKING_URL: baseUrl,+ OPENVIKING_HOOK_STARTED_AT: String(startedAt),+ ...extra,+ });++ await withMockOpenViking(async (req, res) => {+ const url = new URL(req.url, "http://127.0.0.1");+ if (req.method === "GET" && url.pathname === "/health") {+ writeJson(res, { status: "ok", result: { ok: true } });+ return;+ }+ if (req.method === "POST" && url.pathname.endsWith("/messages/batch")) {+ await readRequestBody(req);+ writeJson(res, { status: "ok", result: { ok: true } });+ return;+ }+ if (req.method === "GET" && url.pathname.startsWith("/api/v1/sessions/")) {+ writeJson(res, { status: "ok", result: { pending_tokens: 0 } });+ return;+ }+ res.writeHead(404, { "Content-Type": "application/json" });+ res.end(JSON.stringify({ status: "error", error: "not found" }));+ }, async (baseUrl) => {+ // A marker written after this hook started belongs to a later exit.+ await writeEndedMarker(stateDir, "newer", startedAt + 10_000);+ await runAutoCapture({ session_id: "newer", transcript_path: transcriptPath }, env(baseUrl));+ assert.deepEqual(+ await endedMarkerStamps(stateDir, "newer"),+ [startedAt + 10_000],+ "a fresher marker survives a late Stop worker",+ );++ await writeEndedMarker(stateDir, "older", startedAt - 10_000);+ await runAutoCapture({ session_id: "older", transcript_path: transcriptPath }, env(baseUrl));+ assert.equal(+ await endedMarkerExists(stateDir, "older"),+ false,+ "an older marker is cleared: the thread is alive again",+ );+ });+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});diff --git a/examples/codex-memory-plugin/scripts/marketplace.test.mjs b/examples/codex-memory-plugin/scripts/marketplace.test.mjsindex e6d527bd4..0e5c75692 100644--- a/examples/codex-memory-plugin/scripts/marketplace.test.mjs+++ b/examples/codex-memory-plugin/scripts/marketplace.test.mjs@@ -156,12 +156,21 @@ test("hooks.json uses Codex's native ${PLUGIN_ROOT}, not the legacy placeholder".flat().flatMap((group) => group.hooks || []).map((h) => h.command || "");- assert.ok(commands.length >= 4, "expected at least 4 hook commands (SessionStart/UserPromptSubmit/Stop/PreCompact)");+ assert.ok(commands.length >= 5, "expected at least 5 hook commands (SessionStart/UserPromptSubmit/Stop/SessionEnd/PreCompact)");for (const cmd of commands) {assert.ok(cmd.includes("${PLUGIN_ROOT}/scripts/"), `hook command must be rooted at \${PLUGIN_ROOT}: ${cmd}`);}});+test("hooks.json registers SessionEnd within Codex's clamped budget", () => {+ const parsed = JSON.parse(readFileSync(join(pluginDir, "hooks", "hooks.json"), "utf-8"));+ const entries = (parsed.hooks?.SessionEnd || []).flatMap((group) => group.hooks || []);+ assert.equal(entries.length, 1, "expected exactly one SessionEnd hook");+ assert.match(entries[0].command, /scripts\/session-end\.mjs/);+ // Codex clamps SessionEnd to 3s; anything larger is silently ignored.+ assert.ok(entries[0].timeout <= 3, `SessionEnd timeout must be <= 3, got ${entries[0].timeout}`);+});+test(".mcp.json starts the stdio MCP proxy from the plugin root", () => {const mcp = readJson(join(pluginDir, ".mcp.json"));const server = mcp.mcpServers?.[PLUGIN_NAME];diff --git a/examples/codex-memory-plugin/scripts/ov-memory-doctor.mjs b/examples/codex-memory-plugin/scripts/ov-memory-doctor.mjsindex fa149ba86..0c9991789 100644--- a/examples/codex-memory-plugin/scripts/ov-memory-doctor.mjs+++ b/examples/codex-memory-plugin/scripts/ov-memory-doctor.mjs@@ -55,9 +55,9 @@ const LEGACY_MARKETPLACE = "openviking-plugins-local";const CODEX_DIR = join(homedir(), ".codex");const CODEX_CONFIG = process.env.CODEX_CONFIG_FILE || join(CODEX_DIR, "config.toml");const CACHE_DIR = join(CODEX_DIR, "plugins", "cache", MARKETPLACE, PLUGIN_NAME);-const HOOK_EVENTS = ["session_start", "user_prompt_submit", "stop", "pre_compact"];+const HOOK_EVENTS = ["session_start", "user_prompt_submit", "stop", "session_end", "pre_compact"];const RC_MARKERS = ["# >>> openviking-codex-plugin >>>", "codex-plugin.rc.sh"];-const REQUIRED_PLUGIN_FILES = [".codex-plugin/plugin.json", "hooks/hooks.json", ".mcp.json", "servers/mcp-proxy.mjs", "scripts/config.mjs", "scripts/auto-recall.mjs", "scripts/auto-capture.mjs"];+const REQUIRED_PLUGIN_FILES = [".codex-plugin/plugin.json", "hooks/hooks.json", ".mcp.json", "servers/mcp-proxy.mjs", "scripts/config.mjs", "scripts/auto-recall.mjs", "scripts/auto-capture.mjs", "scripts/session-end.mjs", "scripts/ov-session.mjs"];function parseArgs(argv) {const opts = { json: false, offline: false, timeoutMs: 5000, color: process.stdout.isTTY && !process.env.NO_COLOR };@@ -226,8 +226,8 @@ function checkInstall(report, { codexOnPath }) {else trusted.push(event);}if (disabled.length) report.fail(`hooks disabled in [hooks.state]: ${disabled.join(", ")}`, "", `remove enabled = false from those [hooks.state] sections in ${homeShort(CODEX_CONFIG)}`);- if (untrusted.length) report.info(`hooks without a trust record yet: ${untrusted.join(", ")} (Codex records trusted_hash the first time a hook is approved; a changed hooks.json needs re-approval)`);- if (trusted.length === HOOK_EVENTS.length) report.ok("all four hooks have trust records in config.toml");+ if (untrusted.length) report.info(`hooks without a trust record yet: ${untrusted.join(", ")} (Codex records trusted_hash the first time a hook is approved; a changed hooks.json — including a newly added event such as session_end — needs re-approval)`);+ if (trusted.length === HOOK_EVENTS.length) report.ok(`all ${HOOK_EVENTS.length} hooks have trust records in config.toml`);const legacyKeys = Object.keys(toml).filter((k) => k.includes(LEGACY_MARKETPLACE));if (legacyKeys.length) report.info(`config.toml still has ${legacyKeys.length} section(s) for the legacy id ${LEGACY_MARKETPLACE} (harmless)`);}@@ -382,9 +382,16 @@ function checkActivity(report, cfg, connection) {report.info(`${states.length} session state file(s) in ${homeShort(stateDir)}; newest ${fmtAge(newest.mtimeMs)}: ${d.ovSessionId || "(committed)"} captured ${d.capturedTurnCount ?? "?"} turns`);if (states.length > 3 && states.every((s) => (s.data?.capturedTurnCount ?? 0) === 0)) report.warn("no session has ever captured a turn", "the Stop hook runs but never appends messages", "check the Connection section; enable OPENVIKING_DEBUG=1 and read the hook log");const idleTtl = Number(process.env.OPENVIKING_CODEX_IDLE_TTL_MS) || 30 * 60 * 1000;- const orphans = states.filter((s) => s.data?.ovSessionId && Date.now() - (s.data.lastUpdatedAt || s.mtimeMs) > idleTtl);- if (orphans.length > 10) report.warn(`${orphans.length} idle sessions still uncommitted`, "the idle sweep at SessionStart commits them; a growing pile usually means commits are failing", "check the Connection section, then start a new Codex session to trigger the sweep");- else if (orphans.length) report.info(`${orphans.length} idle session(s) waiting for the SessionStart sweep`);+ // Markers are `<id>.ended.<ts>`; the bare `<id>.ended` is a pre-0.8.1 leftover.+ const ended = new Set(+ readdirSync(stateDir)+ .map((n) => /^(.*)\.ended(?:\.\d+)?$/.exec(n)?.[1])+ .filter(Boolean),+ );+ const orphans = states.filter((s) => s.data?.ovSessionId+ && (ended.has(s.name.slice(0, -5)) || Date.now() - (s.data.lastUpdatedAt || s.mtimeMs) > idleTtl));+ if (orphans.length > 10) report.warn(`${orphans.length} sessions still uncommitted`, "SessionEnd commits a thread when it exits; the SessionStart sweep retries ended and idle ones, so a growing pile usually means commits are failing", "check the Connection section, then start a new Codex session to trigger the sweep");+ else if (orphans.length) report.info(`${orphans.length} session(s) waiting for the SessionStart sweep`);}const profile = tryJson(join(stateDir, "recall-compressor-profile.json"));if (profile?.profile) {diff --git a/examples/codex-memory-plugin/scripts/ov-session.mjs b/examples/codex-memory-plugin/scripts/ov-session.mjsnew file mode 100644index 000000000..bc3f94b0b--- /dev/null+++ b/examples/codex-memory-plugin/scripts/ov-session.mjs@@ -0,0 +1,209 @@+/**+ * Shared OpenViking HTTP + transcript helpers for the Codex capture hooks.+ *+ * Stop (auto-capture), PreCompact and SessionEnd all do the same three things:+ * talk to the OV session API with the plugin's headers, parse the JSONL+ * rollout at `transcript_path`, and append everything past the state cursor.+ * This module owns that logic so the three hooks cannot drift apart.+ *+ * Codex-local on purpose: it depends on Codex rollout shapes and on+ * `session-state.mjs`, so it does not belong under `scripts/shared/`.+ */++import { readFile } from "node:fs/promises";+import { extractCaptureTurns, findLastHumanTurnIndex } from "./capture-utils.mjs";+import { resolveOvSessionId, saveState } from "./session-state.mjs";+import { sendSessionMessages } from "./shared/batch-send.mjs";++function responseTraceId(body) {+ return body?.result?.trace_id || body?.error?.trace_id || body?.trace_id || undefined;+}++/**+ * Build the `{ fetchJSONRes, fetchJSON }` pair used by every capture hook.+ * The peer id is read through a getter because callers only know it after+ * loading state (which happens under the session lock).+ */+export function makeFetchJSON(cfg, { getActorPeerId = () => "" } = {}) {+ function makeHeaders() {+ const headers = { "Content-Type": "application/json" };+ if (cfg.apiKey) {+ headers["Authorization"] = `Bearer ${cfg.apiKey}`;+ headers["X-API-Key"] = cfg.apiKey;+ }+ if (cfg.sendIdentityHeaders && cfg.account) headers["X-OpenViking-Account"] = cfg.account;+ if (cfg.sendIdentityHeaders && cfg.user) headers["X-OpenViking-User"] = cfg.user;+ const actorPeerId = getActorPeerId();+ if (actorPeerId) headers["X-OpenViking-Actor-Peer"] = actorPeerId;+ if (cfg.userAgent) headers["User-Agent"] = cfg.userAgent;+ return headers;+ }++ async function fetchJSONRes(path, init = {}) {+ const controller = new AbortController();+ const timer = setTimeout(() => controller.abort(), cfg.captureTimeoutMs);+ try {+ const res = await fetch(`${cfg.baseUrl}${path}`, {+ ...init,+ headers: makeHeaders(),+ signal: controller.signal,+ });+ const body = await res.json().catch(() => null);+ if (!body) return { ok: false, status: res.status, error: { message: "empty or invalid JSON response" } };+ const traceId = responseTraceId(body);+ if (!res.ok || body.status === "error") {+ return { ok: false, status: res.status, error: body.error || body, traceId };+ }+ return { ok: true, status: res.status, result: body.result ?? body, traceId };+ } catch (err) {+ return { ok: false, status: 0, error: { message: err?.message || String(err) } };+ } finally {+ clearTimeout(timer);+ }+ }++ async function fetchJSON(path, init = {}) {+ const r = await fetchJSONRes(path, init);+ return r.ok ? (r.result ?? null) : null;+ }++ return { fetchJSONRes, fetchJSON };+}++export function commitOvSession(fetchJSONRes, ovSessionId, body = {}) {+ return fetchJSONRes(`/api/v1/sessions/${encodeURIComponent(ovSessionId)}/commit`, {+ method: "POST",+ body: JSON.stringify(body),+ });+}++function parseTranscript(content) {+ try {+ const data = JSON.parse(content);+ if (Array.isArray(data)) return data;+ } catch { /* not a JSON array */ }+ const out = [];+ for (const line of content.split("\n")) {+ if (!line.trim()) continue;+ try { out.push(JSON.parse(line)); } catch { /* skip */ }+ }+ return out;+}++/**+ * Read and normalize the rollout at `transcriptPath`.+ *+ * `ok` distinguishes "read it, it has N turns" from "could not read it".+ * Callers must not treat `ok: false` as a transcript shrink — resetting the+ * cursor on a transient read error replays the whole session.+ */+export async function readTranscriptTurns(transcriptPath, cfg, logError) {+ if (!transcriptPath) return { turns: [], ok: false };+ try {+ const raw = await readFile(transcriptPath, "utf-8");+ if (!raw.trim()) return { turns: [], ok: true };+ return { turns: extractCaptureTurns(parseTranscript(raw), cfg), ok: true };+ } catch (err) {+ logError?.("transcript_read", err);+ return { turns: [], ok: false };+ }+}++export function hasCaptureKeyword(turns) {+ return turns.some((turn) =>+ /\b(remember|memorize|store|save|capture|note|record)\b|记住|保存|记录|记忆/i.test(turn.text)+ );+}++/**+ * Append every transcript turn past `state.capturedTurnCount`.+ *+ * Advances the cursor (and persists it) after each durably sent batch, so a+ * failure halfway through never replays what already landed. Mutates `state`;+ * the caller owns the session lock.+ *+ * `shouldSend(newTurns)` lets a caller veto the send (keyword capture mode)+ * without disturbing the cursor.+ *+ * Returns `{ newTurns, added, ovSessionId, skipped, unreadable }`. `unreadable`+ * separates "the transcript is empty" from "the transcript could not be read":+ * committing on the latter would archive a session whose tail turns were never+ * seen, so callers must keep the session live instead.+ */+export async function catchUpTurns({+ state,+ transcriptPath,+ fetchJSONRes,+ activePeerId = "",+ cfg,+ log,+ logError,+ heartbeat,+ shouldSend,+}) {+ // Remember the rollout path so the SessionStart sweep can catch up turns for+ // a session whose Stop/SessionEnd workers never ran.+ if (transcriptPath) state.transcriptPath = transcriptPath;++ const { turns, ok } = await readTranscriptTurns(transcriptPath, cfg, logError);++ if (!ok || turns.length === 0) {+ log?.("transcript_empty", {+ readable: ok,+ previouslyCaptured: state.capturedTurnCount,+ });+ return { newTurns: [], added: 0, ovSessionId: "", unreadable: Boolean(transcriptPath) && !ok };+ }++ // Post-compact transcript-shrink defense: codex's /compact may rewrite or+ // truncate transcript_path. Resume at the latest human turn so the current+ // interaction is captured without replaying compacted history.+ if (turns.length < state.capturedTurnCount) {+ const humanIndex = findLastHumanTurnIndex(turns);+ log?.("transcript_shrink_detected", {+ cached: state.capturedTurnCount,+ observed: turns.length,+ resumeFrom: Math.max(0, humanIndex),+ fallback: humanIndex < 0 ? "full_transcript" : undefined,+ });+ state.capturedTurnCount = Math.max(0, humanIndex);+ }++ const newTurns = turns.slice(state.capturedTurnCount);+ log?.("transcript_parse", {+ totalTurns: turns.length,+ previouslyCaptured: state.capturedTurnCount,+ newTurns: newTurns.length,+ });+ if (newTurns.length === 0) return { newTurns, added: 0, ovSessionId: "" };++ if (shouldSend && !shouldSend(newTurns)) {+ log?.("skip", { stage: "capture_mode", reason: "keyword mode without capture trigger" });+ return { newTurns, added: 0, ovSessionId: "", skipped: "capture_mode" };+ }++ // Only revive the OV session id when there is something to send; otherwise+ // an already-committed session would look live again on every hook run.+ const ovSessionId = resolveOvSessionId(state);+ if (!ovSessionId) {+ logError?.("resolve_ov_session", "failed to derive OV session id");+ return { newTurns, added: 0, ovSessionId: "" };+ }++ const payloads = newTurns.map((turn) => {+ const body = turn.parts?.length+ ? { role: turn.role, parts: turn.parts }+ : { role: turn.role, content: turn.text };+ if (activePeerId) body.peer_id = activePeerId;+ return body;+ });++ const r = await sendSessionMessages(fetchJSONRes, ovSessionId, payloads, {+ onSent: async (n) => {+ state.capturedTurnCount += n;+ await saveState(state);+ await heartbeat?.();+ },+ });+ return { newTurns, added: r.sent, ovSessionId };+}diff --git a/examples/codex-memory-plugin/scripts/pre-compact-capture.mjs b/examples/codex-memory-plugin/scripts/pre-compact-capture.mjsindex dddf205fa..91fdbc8e3 100644--- a/examples/codex-memory-plugin/scripts/pre-compact-capture.mjs+++ b/examples/codex-memory-plugin/scripts/pre-compact-capture.mjs@@ -17,23 +17,33 @@* `cx-<codex-session-id>` OV session id; `/messages` auto-creates it if* needed.*+ * This hook runs synchronously under a 60s budget, so it waits only a+ * bounded time for the per-session lock; if another writer holds it, we+ * leave everything alone and let SessionEnd or the sweep commit later.+ ** PreCompact output schema accepts {} as a no-op.*/-import { readFile } from "node:fs/promises";-import {- extractCaptureTurns,-} from "./capture-utils.mjs";import { loadConfig } from "./config.mjs";import { createLogger } from "./debug-log.mjs";-import { loadState, resolveOvSessionId, saveState } from "./session-state.mjs";-import { sendSessionMessages } from "./shared/batch-send.mjs";+import { catchUpTurns, commitOvSession, hasCaptureKeyword, makeFetchJSON } from "./ov-session.mjs";+import { clearEnded, loadState, saveState, withSessionLock } from "./session-state.mjs";import { resolveEffectivePeerId } from "./shared/workspace-peer.mjs";const cfg = loadConfig();-const { log, logError } = createLogger("pre-compact");+const { log, logError } = createLogger("pre-compact", cfg);let activePeerId = cfg.peerId || "";+// Well inside the 60s hook budget, leaving room for the commit itself.+const LOCK_WAIT_MS = (() => {+ const v = Number(process.env.OPENVIKING_CODEX_LOCK_WAIT_MS);+ return Number.isFinite(v) && v >= 0 ? Math.floor(v) : 40_000;+})();++const HOOK_STARTED_AT = Date.now();++const { fetchJSONRes, fetchJSON } = makeFetchJSON(cfg, { getActorPeerId: () => activePeerId });+function output(obj) {process.stdout.write(JSON.stringify(obj) + "\n");}@@ -42,109 +52,7 @@ function noop(message) {output(message ? { systemMessage: message } : {});}-function makeHeaders() {- const headers = { "Content-Type": "application/json" };- if (cfg.apiKey) {- headers["Authorization"] = `Bearer ${cfg.apiKey}`;- headers["X-API-Key"] = cfg.apiKey;- }- if (cfg.sendIdentityHeaders && cfg.account) headers["X-OpenViking-Account"] = cfg.account;- if (cfg.sendIdentityHeaders && cfg.user) headers["X-OpenViking-User"] = cfg.user;- if (activePeerId) headers["X-OpenViking-Actor-Peer"] = activePeerId;- if (cfg.userAgent) headers["User-Agent"] = cfg.userAgent;- return headers;-}--function responseTraceId(body) {- return body?.result?.trace_id || body?.error?.trace_id || body?.trace_id || undefined;-}--async function fetchJSONRes(path, init = {}) {- const controller = new AbortController();- const timer = setTimeout(() => controller.abort(), cfg.captureTimeoutMs);- try {- const res = await fetch(`${cfg.baseUrl}${path}`, { ...init, headers: makeHeaders(), signal: controller.signal });- const body = await res.json().catch(() => null);- if (!body) return { ok: false, status: res.status, error: { message: "empty or invalid JSON response" } };- const traceId = responseTraceId(body);- if (!res.ok || body.status === "error") {- return { ok: false, status: res.status, error: body.error || body, traceId };- }- return { ok: true, status: res.status, result: body.result ?? body, traceId };- } catch (err) {- return { ok: false, status: 0, error: { message: err?.message || String(err) } };- } finally {- clearTimeout(timer);- }-}--async function fetchJSON(path, init = {}) {- const r = await fetchJSONRes(path, init);- return r.ok ? (r.result ?? null) : null;-}--function parseTranscript(content) {- try {- const data = JSON.parse(content);- if (Array.isArray(data)) return data;- } catch { /* not array */ }- const lines = content.split("\n").filter((l) => l.trim());- const out = [];- for (const line of lines) {- try { out.push(JSON.parse(line)); } catch { /* skip */ }- }- return out;-}--function extractTurns(entries) {- return extractCaptureTurns(entries, cfg);-}--async function readTranscriptTurns(transcriptPath) {- if (!transcriptPath) return [];- try {- const raw = await readFile(transcriptPath, "utf-8");- if (!raw.trim()) return [];- return extractTurns(parseTranscript(raw));- } catch (err) {- logError("transcript_read", err);- return [];- }-}--async function appendTurns(ovSessionId, turns) {- const payloads = turns.map((turn) => {- const body = turn.parts?.length- ? { role: turn.role, parts: turn.parts }- : { role: turn.role, content: turn.text };- if (activePeerId) body.peer_id = activePeerId;- return body;- });- const r = await sendSessionMessages(fetchJSONRes, ovSessionId, payloads);- return r.sent;-}--async function main() {- if (!cfg.autoCommitOnCompact) {- log("skip", { stage: "init", reason: "autoCommitOnCompact disabled" });- noop();- return;- }-- let input;- try {- const chunks = [];- for await (const chunk of process.stdin) chunks.push(chunk);- input = JSON.parse(Buffer.concat(chunks).toString());- } catch {- log("skip", { stage: "stdin_parse", reason: "invalid input" });- noop();- return;- }-- const sessionId = input.session_id || "unknown";- const transcriptPath = input.transcript_path || null;- const trigger = input.trigger || "auto";+async function compact(sessionId, transcriptPath, trigger, heartbeat) {const state = await loadState(sessionId);activePeerId = cfg.peerId || state.workspacePeerId || resolveEffectivePeerId({ cfg, cwd: process.cwd() }).peerId;log("start", { sessionId, transcriptPath, trigger, hasPeer: Boolean(activePeerId) });@@ -152,86 +60,68 @@ async function main() {const health = await fetchJSON("/health");if (!health) {logError("health_check", "server unreachable");- noop();- return;+ return "";}- const allTurns = await readTranscriptTurns(transcriptPath);- const newTurns = allTurns.slice(state.capturedTurnCount);-- log("transcript_parse", {- totalTurns: allTurns.length,- previouslyCaptured: state.capturedTurnCount,- newTurns: newTurns.length,+ const { newTurns, added, ovSessionId, skipped, unreadable } = await catchUpTurns({+ state,+ transcriptPath,+ fetchJSONRes,+ activePeerId,+ cfg,+ log,+ logError,+ heartbeat,+ // Keyword mode only gates sessions that have nothing live yet; once an OV+ // session exists we always finish it before compaction.+ shouldSend: (turns) =>+ Boolean(state.ovSessionId) || cfg.captureMode !== "keyword" || hasCaptureKeyword(turns),});- if (allTurns.length === 0 && !state.ovSessionId) {- log("skip", { stage: "nothing_to_commit", reason: "no transcript and no open OV session" });- noop();- return;- }+ if (added > 0) log("appended_catchup", { ovSessionId, added });- if (newTurns.length > 0 && !state.ovSessionId && cfg.captureMode === "keyword" && !hasCaptureKeyword(newTurns)) {- log("skip", { stage: "capture_mode", reason: "keyword mode without capture trigger" });- await saveState(state);- noop();- return;+ // An unreadable transcript is not an empty one: the tail turns may still be+ // there. Keep the live id and the marker so a later commit retries.+ if (unreadable) {+ logError("transcript_unreadable", { ovSessionId: state.ovSessionId, transcriptPath });+ await saveState(state, { touch: false });+ return `pre-compact transcript unreadable for ${state.ovSessionId || sessionId}; state preserved for retry`;}- if (newTurns.length > 0) {- const ovSessionId = resolveOvSessionId(state);- if (!ovSessionId) {- logError("resolve_ov_session", "failed to derive OV session id for catch-up");- noop();- return;- }- const added = await appendTurns(ovSessionId, newTurns);- state.capturedTurnCount += added;- log("appended_catchup", { ovSessionId, added });- if (added < newTurns.length) {- logError("append_failed_keep_state", { ovSessionId, attempted: newTurns.length, added });- await saveState(state);- noop(`pre-compact catch-up append incomplete for ${ovSessionId}; state preserved for retry`);- return;- }+ if (newTurns.length > 0 && !skipped && added < newTurns.length) {+ logError("append_failed_keep_state", { ovSessionId, attempted: newTurns.length, added });+ await saveState(state);+ return `pre-compact catch-up append incomplete for ${ovSessionId}; state preserved for retry`;}if (!state.ovSessionId) {log("skip", { stage: "commit", reason: "no OV session for this codex session" });await saveState(state);- noop();- return;+ return "";}- const ovSessionId = state.ovSessionId;- const commit = await fetchJSONRes(- `/api/v1/sessions/${encodeURIComponent(ovSessionId)}/commit`,- { method: "POST", body: JSON.stringify({}) },- );+ const liveOvSessionId = state.ovSessionId;+ const commit = await commitOvSession(fetchJSONRes, liveOvSessionId);// Commit failure handling (see DESIGN.md "Commit failure"): if /commit// fails (server unreachable, non-2xx, timeout) we MUST NOT reset- // ovSessionId — keep state intact so the next sweep / SessionStart can- // retry. A transient OV outage shouldn't lose a session's memory.+ // ovSessionId — keep state intact so SessionEnd / the sweep can retry.if (!commit.ok) {log("commit", {- ovSessionId,+ ovSessionId: liveOvSessionId,ok: false,status: commit.status,trace_id: commit.traceId,error: commit.error?.message || commit.error?.code,});- await saveState(state); // bumps lastUpdatedAt only, keeps ovSessionId- noop(- `pre-compact commit attempted on ${ovSessionId}; result unavailable` +- `${commit.traceId ? ` (trace_id=${commit.traceId})` : ""} (state preserved for retry)`,- );- return;+ await saveState(state);+ return `pre-compact commit attempted on ${liveOvSessionId}; result unavailable` ++ `${commit.traceId ? ` (trace_id=${commit.traceId})` : ""} (state preserved for retry)`;}const traceId = commit.traceId || commit.result?.trace_id || "";log("commit", {- ovSessionId,+ ovSessionId: liveOvSessionId,archived: commit.result?.archived ?? false,taskId: commit.result?.task_id,status: commit.result?.status,@@ -243,14 +133,45 @@ async function main() {state.ovSessionId = null;await saveState(state);- noop(- `OpenViking session ${ovSessionId} is committed` +- (traceId ? ` (trace_id=${traceId})` : ""),- );+ return `OpenViking session ${liveOvSessionId} is committed` + (traceId ? ` (trace_id=${traceId})` : "");}-function hasCaptureKeyword(turns) {- return turns.some((turn) => /\b(remember|memorize|store|save|capture|note|record)\b|记住|保存|记录|记忆/i.test(turn.text));+async function main() {+ if (!cfg.autoCommitOnCompact) {+ log("skip", { stage: "init", reason: "autoCommitOnCompact disabled" });+ noop();+ return;+ }++ let input;+ try {+ const chunks = [];+ for await (const chunk of process.stdin) chunks.push(chunk);+ input = JSON.parse(Buffer.concat(chunks).toString());+ } catch {+ log("skip", { stage: "stdin_parse", reason: "invalid input" });+ noop();+ return;+ }++ const sessionId = input.session_id || "unknown";+ const transcriptPath = input.transcript_path || null;+ const trigger = input.trigger || "auto";++ // Compaction means the thread is running, so any earlier end marker is stale.+ await clearEnded(sessionId, { before: HOOK_STARTED_AT });++ const outcome = await withSessionLock(+ sessionId,+ ({ heartbeat }) => compact(sessionId, transcriptPath, trigger, heartbeat),+ { waitMs: LOCK_WAIT_MS },+ );+ if (outcome.skipped) {+ logError("lock_timeout", `another writer holds ${sessionId}; leaving state untouched`);+ noop();+ return;+ }+ noop(outcome.value);}main().catch((err) => { logError("uncaught", err); noop(); });diff --git a/examples/codex-memory-plugin/scripts/pre-compact-capture.test.mjs b/examples/codex-memory-plugin/scripts/pre-compact-capture.test.mjsnew file mode 100644index 000000000..4ba82a0ad--- /dev/null+++ b/examples/codex-memory-plugin/scripts/pre-compact-capture.test.mjs@@ -0,0 +1,259 @@+import assert from "node:assert/strict";+import { spawn } from "node:child_process";+import { mkdir, mkdtemp, readFile, readdir, rm, stat, writeFile } from "node:fs/promises";+import http from "node:http";+import { tmpdir } from "node:os";+import { dirname, join } from "node:path";+import test from "node:test";+import { fileURLToPath } from "node:url";++const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url));++async function endedMarkerStamps(dir, id) {+ const prefix = `${id}.ended.`;+ const files = await readdir(dir).catch(() => []);+ return files+ .filter((name) => name.startsWith(prefix))+ .map((name) => Number(name.slice(prefix.length)))+ .filter((ts) => Number.isFinite(ts));+}++async function endedMarkerExists(dir, id) {+ return (await endedMarkerStamps(dir, id)).length > 0;+}++function writeEndedMarker(dir, id, ts) {+ return writeFile(join(dir, `${id}.ended.${ts}`), String(ts));+}+++function readRequestBody(req) {+ return new Promise((resolve, reject) => {+ const chunks = [];+ req.on("data", (chunk) => chunks.push(chunk));+ req.on("end", () => {+ const raw = Buffer.concat(chunks).toString("utf-8");+ try { resolve(raw ? JSON.parse(raw) : null); } catch (err) { reject(err); }+ });+ req.on("error", reject);+ });+}++function writeJson(res, value) {+ res.writeHead(200, { "Content-Type": "application/json" });+ res.end(JSON.stringify(value));+}++async function withMockOpenViking(handler, fn) {+ const server = http.createServer((req, res) => {+ Promise.resolve(handler(req, res)).catch((err) => {+ res.writeHead(500, { "Content-Type": "application/json" });+ res.end(JSON.stringify({ status: "error", error: String(err?.stack || err) }));+ });+ });+ await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve));+ try {+ const { port } = server.address();+ return await fn(`http://127.0.0.1:${port}`);+ } finally {+ await new Promise((resolve) => server.close(resolve));+ }+}++function runPreCompact(input, env) {+ return new Promise((resolve, reject) => {+ const cleanEnv = { ...process.env };+ for (const key of Object.keys(cleanEnv)) {+ if (key.startsWith("OPENVIKING_")) delete cleanEnv[key];+ }+ const child = spawn(process.execPath, [join(SCRIPT_DIR, "pre-compact-capture.mjs")], {+ env: { ...cleanEnv, ...env },+ stdio: ["pipe", "pipe", "pipe"],+ });+ let stdout = "";+ let stderr = "";+ child.stdout.on("data", (chunk) => { stdout += chunk.toString(); });+ child.stderr.on("data", (chunk) => { stderr += chunk.toString(); });+ child.on("error", reject);+ child.on("close", (code) => {+ if (code !== 0) {+ reject(new Error(`pre-compact-capture exited ${code}: ${stderr}`));+ return;+ }+ resolve({ output: JSON.parse(stdout.trim()), stderr });+ });+ child.stdin.end(JSON.stringify(input));+ });+}++function baseEnv(baseUrl, stateDir, extra = {}) {+ return {+ OPENVIKING_URL: baseUrl,+ OPENVIKING_AUTO_CAPTURE: "1",+ OPENVIKING_CAPTURE_ASSISTANT_TURNS: "1",+ OPENVIKING_CODEX_STATE_DIR: stateDir,+ OPENVIKING_CONFIG_FILE: join(stateDir, "missing-ov.conf"),+ OPENVIKING_CLI_CONFIG_FILE: join(stateDir, "missing-ovcli.conf"),+ OPENVIKING_CREDENTIAL_SOURCE: "env",+ OPENVIKING_MIN_QUERY_LENGTH: "1",+ OPENVIKING_TIMEOUT_MS: "5000",+ OPENVIKING_CAPTURE_TIMEOUT_MS: "5000",+ ...extra,+ };+}++function turn(role, content) {+ return JSON.stringify({ payload: { message: { role, content } } });+}++async function exists(path) {+ try { await stat(path); return true; } catch { return false; }+}++function mockHandler(calls) {+ return async (req, res) => {+ const url = new URL(req.url, "http://127.0.0.1");+ const call = { method: req.method, path: url.pathname, body: null };+ calls.push(call);+ if (req.method === "GET" && url.pathname === "/health") {+ writeJson(res, { status: "ok", result: { ok: true } });+ return;+ }+ if (req.method === "POST" && url.pathname.endsWith("/messages/batch")) {+ call.body = await readRequestBody(req);+ writeJson(res, { status: "ok", result: { ok: true } });+ return;+ }+ if (req.method === "POST" && url.pathname.endsWith("/commit")) {+ call.body = await readRequestBody(req);+ writeJson(res, { status: "ok", result: { archived: true, task_id: "task-pc", trace_id: "trace-pc" } });+ return;+ }+ res.writeHead(404, { "Content-Type": "application/json" });+ res.end(JSON.stringify({ status: "error", error: "not found" }));+ };+}++test("pre-compact catches up, commits and keeps the cursor", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-pre-compact-"));+ const transcriptPath = join(stateDir, "transcript.jsonl");+ const calls = [];+ try {+ const now = Date.now();+ await writeFile(join(stateDir, "pc1.json"), JSON.stringify({+ codexSessionId: "pc1",+ ovSessionId: "cx-pc1",+ capturedTurnCount: 2,+ createdAt: now - 1000,+ lastUpdatedAt: now,+ }));+ await writeEndedMarker(stateDir, "pc1", now);+ await writeFile(transcriptPath, [+ turn("user", "turn-0"),+ turn("assistant", "turn-1"),+ turn("user", "turn-2"),+ turn("assistant", "turn-3"),+ ].join("\n"));++ await withMockOpenViking(mockHandler(calls), async (baseUrl) => {+ const { output } = await runPreCompact(+ { session_id: "pc1", transcript_path: transcriptPath, trigger: "auto" },+ baseEnv(baseUrl, stateDir),+ );+ assert.match(output.systemMessage, /cx-pc1 is committed \(trace_id=trace-pc\)/);+ });++ const messages = calls+ .filter((c) => c.path.endsWith("/messages/batch"))+ .flatMap((c) => c.body?.messages ?? []);+ assert.equal(messages.length, 2);+ assert.ok(calls.some((c) => c.path === "/api/v1/sessions/cx-pc1/commit"));++ const state = JSON.parse(await readFile(join(stateDir, "pc1.json"), "utf-8"));+ assert.equal(state.ovSessionId, null);+ assert.equal(state.capturedTurnCount, 4);+ assert.equal(await endedMarkerExists(stateDir, "pc1"), false, "compaction proves the thread is alive");+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("pre-compact leaves state untouched when the session lock is held", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-pre-compact-lock-"));+ const transcriptPath = join(stateDir, "transcript.jsonl");+ const calls = [];+ try {+ const now = Date.now();+ await writeFile(join(stateDir, "pc2.json"), JSON.stringify({+ codexSessionId: "pc2",+ ovSessionId: "cx-pc2",+ capturedTurnCount: 1,+ createdAt: now - 1000,+ lastUpdatedAt: now,+ }));+ await writeFile(transcriptPath, [turn("user", "turn-0"), turn("assistant", "turn-1")].join("\n"));+ await mkdir(join(stateDir, "pc2.lock"), { recursive: true });++ await withMockOpenViking(mockHandler(calls), async (baseUrl) => {+ const { output } = await runPreCompact(+ { session_id: "pc2", transcript_path: transcriptPath, trigger: "manual" },+ baseEnv(baseUrl, stateDir, { OPENVIKING_CODEX_LOCK_WAIT_MS: "300" }),+ );+ assert.deepEqual(output, {});+ });++ assert.equal(calls.length, 0, "a held lock must stop every HTTP call");+ const state = JSON.parse(await readFile(join(stateDir, "pc2.json"), "utf-8"));+ assert.equal(state.ovSessionId, "cx-pc2");+ assert.equal(state.capturedTurnCount, 1);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("pre-compact does not commit when the catch-up append fails entirely", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-pre-compact-fail-"));+ const transcriptPath = join(stateDir, "transcript.jsonl");+ const calls = [];+ const handler = async (req, res) => {+ const url = new URL(req.url, "http://127.0.0.1");+ calls.push({ method: req.method, path: url.pathname });+ if (req.method === "GET" && url.pathname === "/health") {+ writeJson(res, { status: "ok", result: { ok: true } });+ return;+ }+ res.writeHead(500, { "Content-Type": "application/json" });+ res.end(JSON.stringify({ status: "error", error: { message: "boom" } }));+ };+ try {+ const now = Date.now();+ await writeFile(join(stateDir, "pc3.json"), JSON.stringify({+ codexSessionId: "pc3",+ ovSessionId: "cx-pc3",+ capturedTurnCount: 2,+ createdAt: now - 1000,+ lastUpdatedAt: now,+ }));+ await writeFile(transcriptPath, [+ turn("user", "turn-0"),+ turn("assistant", "turn-1"),+ turn("user", "turn-2"),+ turn("assistant", "turn-3"),+ ].join("\n"));++ await withMockOpenViking(handler, async (baseUrl) => {+ const { output } = await runPreCompact(+ { session_id: "pc3", transcript_path: transcriptPath, trigger: "manual" },+ baseEnv(baseUrl, stateDir),+ );+ assert.match(output.systemMessage, /catch-up append incomplete for cx-pc3/);+ });++ assert.ok(!calls.some((c) => c.path.endsWith("/commit")), "must not commit with turns still unsent");+ const state = JSON.parse(await readFile(join(stateDir, "pc3.json"), "utf-8"));+ assert.equal(state.ovSessionId, "cx-pc3");+ assert.equal(state.capturedTurnCount, 2);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});diff --git a/examples/codex-memory-plugin/scripts/session-end.mjs b/examples/codex-memory-plugin/scripts/session-end.mjsnew file mode 100644index 000000000..21c251ff9--- /dev/null+++ b/examples/codex-memory-plugin/scripts/session-end.mjs@@ -0,0 +1,205 @@+#!/usr/bin/env node++/**+ * SessionEnd hook for Codex (Codex >= 0.145).+ *+ * Fires when a codex thread shuts down gracefully (`/quit`, `/exit`, double+ * Ctrl-C, EOF, `codex exec` completion); at TUI exit every thread touched in+ * the process gets one in a burst. This is the deterministic commit point:+ * catch up whatever turns the last Stop never sent, then commit the OV+ * session so the extractor runs on the whole conversation.+ *+ * Signals, crashes, Codex < 0.145 and app-server thread deferral never fire+ * it — the SessionStart sweep is the fallback for those.+ *+ * Budget: Codex allows 1s by default and clamps `timeout` to 3s, so the+ * parent only writes the `.ended` marker (lock-free) and detaches a worker,+ * regardless of `writePathAsync`. Codex deliberately leaves cleanly detached+ * helpers running after the hook exits.+ *+ * SessionEnd output is ignored; we print `{}` for symmetry with the other hooks.+ */++import { loadConfig } from "./config.mjs";+import { createLogger } from "./debug-log.mjs";+import { catchUpTurns, commitOvSession, hasCaptureKeyword, makeFetchJSON } from "./ov-session.mjs";+import {+ clearEnded,+ loadState,+ markEnded,+ readEndedAt,+ saveState,+ withSessionLock,+} from "./session-state.mjs";+import { maybeDetach, readHookStdin } from "./shared/async-writer.mjs";+import { resolveEffectivePeerId } from "./shared/workspace-peer.mjs";++const cfg = loadConfig();+const { log, logError } = createLogger("session-end", cfg);+let activePeerId = cfg.peerId || "";++const LOCK_WAIT_MS = (() => {+ const v = Number(process.env.OPENVIKING_CODEX_LOCK_WAIT_MS);+ return Number.isFinite(v) && v >= 0 ? Math.floor(v) : 120_000;+})();++const { fetchJSONRes, fetchJSON } = makeFetchJSON(cfg, { getActorPeerId: () => activePeerId });++function output(obj) {+ process.stdout.write(JSON.stringify(obj) + "\n");+}++async function finish(sessionId, transcriptPath, endToken, heartbeat) {+ // Verify the marker this worker was launched for before doing any work. A+ // marker that is gone means the thread was resumed; a different one belongs+ // to a later exit whose own worker will commit.+ const marker = await readEndedAt(sessionId);+ if (marker !== endToken) {+ log("superseded", { sessionId, token: endToken, marker: marker ?? null });+ return;+ }++ const state = await loadState(sessionId);+ activePeerId = cfg.peerId || state.workspacePeerId || resolveEffectivePeerId({ cfg, cwd: process.cwd() }).peerId;+ log("start", { sessionId, transcriptPath, hasPeer: Boolean(activePeerId) });++ const health = await fetchJSON("/health");+ if (!health) {+ // Keep the state and the end marker; the sweep retries once OV is back.+ logError("health_check", "server unreachable; end marker kept for the sweep");+ return;+ }++ const { newTurns, added, skipped, unreadable } = await catchUpTurns({+ state,+ transcriptPath,+ fetchJSONRes,+ activePeerId,+ cfg,+ log,+ logError,+ heartbeat,+ shouldSend: (turns) =>+ Boolean(state.ovSessionId) || cfg.captureMode !== "keyword" || hasCaptureKeyword(turns),+ });+ if (added > 0) log("appended_catchup", { ovSessionId: state.ovSessionId, added });++ // An unreadable transcript is not an empty one: the tail turns may still be+ // there. Keep the live id and the marker so the sweep retries.+ if (unreadable) {+ logError("transcript_unreadable", { ovSessionId: state.ovSessionId, transcriptPath });+ await saveState(state, { touch: false });+ return;+ }++ // Committing now would archive a session missing its tail turns and release+ // the live id. Keep the live id and the marker so the sweep retries; the+ // cursor already advanced past the prefix that landed.+ if (newTurns.length > 0 && !skipped && added < newTurns.length) {+ logError("append_incomplete", {+ ovSessionId: state.ovSessionId,+ attempted: newTurns.length,+ added,+ });+ await saveState(state, { touch: false });+ return;+ }++ if (!state.ovSessionId) {+ log("skip", { stage: "commit", reason: "no live OV session for this codex session" });+ if (added > 0) await saveState(state, { touch: false });+ await clearEnded(sessionId, { before: endToken + 1 });+ return;+ }++ const ovSessionId = state.ovSessionId;+ const commit = await commitOvSession(fetchJSONRes, ovSessionId);+ if (!commit.ok) {+ log("commit", {+ reason: "session_end",+ ovSessionId,+ ok: false,+ status: commit.status,+ trace_id: commit.traceId,+ error: commit.error?.message || commit.error?.code,+ });+ // Keep ovSessionId and the end marker so the sweep retries this session.+ await saveState(state, { touch: false });+ return;+ }++ const traceId = commit.traceId || commit.result?.trace_id || "";+ log("commit", {+ reason: "session_end",+ ovSessionId,+ archived: commit.result?.archived ?? false,+ taskId: commit.result?.task_id,+ status: commit.result?.status,+ trace_id: traceId || undefined,+ });+ state.ovSessionId = null;+ await saveState(state, { touch: false });+ await clearEnded(sessionId, { before: endToken + 1 });+}++async function main() {+ if (!cfg.autoCapture) {+ log("skip", { stage: "init", reason: "autoCapture disabled" });+ output({});+ return;+ }++ let raw;+ let input;+ try {+ raw = await readHookStdin();+ input = JSON.parse(raw);+ } catch {+ log("skip", { stage: "stdin_parse", reason: "invalid input" });+ output({});+ return;+ }++ const sessionId = input.session_id;+ const transcriptPath = input.transcript_path || null;+ if (!sessionId) {+ log("skip", { stage: "init", reason: "no session_id" });+ output({});+ return;+ }++ // Lock-free and first: even if the worker never runs (job-object kill on+ // Windows, crash, disabled server) the sweep can still find and commit this+ // session at the next SessionStart. The marker's timestamp is the token the+ // worker verifies before committing, inherited through the environment.+ const inherited = Number(process.env.OPENVIKING_SESSION_END_TOKEN);+ const endToken = process.env.OV_HOOK_WORKER === "1" && Number.isFinite(inherited) && inherited > 0+ ? inherited+ : await markEnded(sessionId);+ process.env.OPENVIKING_SESSION_END_TOKEN = String(endToken);++ if (process.env.OV_HOOK_WORKER !== "1") {+ // stdin is already drained; hand the payload to the worker through the+ // shared cache env var.+ process.env.OPENVIKING_HOOK_STDIN_CACHE = raw;+ const detached = await maybeDetach(+ { ...cfg, writePathAsync: true },+ { approve: () => output({}) },+ );+ if (detached) return;+ delete process.env.OPENVIKING_HOOK_STDIN_CACHE;+ logError("detach_failed", "running the commit inline; Codex may kill it at the timeout");+ }++ const outcome = await withSessionLock(+ sessionId,+ ({ heartbeat }) => finish(sessionId, transcriptPath, endToken, heartbeat),+ { waitMs: LOCK_WAIT_MS },+ );+ if (outcome.skipped) {+ logError("lock_timeout", `another writer holds ${sessionId}; end marker left for the sweep`);+ }+ output({});+}++main().catch((err) => { logError("uncaught", err); output({}); });diff --git a/examples/codex-memory-plugin/scripts/session-end.test.mjs b/examples/codex-memory-plugin/scripts/session-end.test.mjsnew file mode 100644index 000000000..8e3623c14--- /dev/null+++ b/examples/codex-memory-plugin/scripts/session-end.test.mjs@@ -0,0 +1,573 @@+import assert from "node:assert/strict";+import { spawn } from "node:child_process";+import { mkdir, mkdtemp, readFile, readdir, rm, stat, utimes, writeFile } from "node:fs/promises";+import http from "node:http";+import { tmpdir } from "node:os";+import { dirname, join } from "node:path";+import test from "node:test";+import { fileURLToPath } from "node:url";++const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url));++async function endedMarkerStamps(dir, id) {+ const prefix = `${id}.ended.`;+ const files = await readdir(dir).catch(() => []);+ return files+ .filter((name) => name.startsWith(prefix))+ .map((name) => Number(name.slice(prefix.length)))+ .filter((ts) => Number.isFinite(ts));+}++async function endedMarkerExists(dir, id) {+ return (await endedMarkerStamps(dir, id)).length > 0;+}++function writeEndedMarker(dir, id, ts) {+ return writeFile(join(dir, `${id}.ended.${ts}`), String(ts));+}+++function readRequestBody(req) {+ return new Promise((resolve, reject) => {+ const chunks = [];+ req.on("data", (chunk) => chunks.push(chunk));+ req.on("end", () => {+ const raw = Buffer.concat(chunks).toString("utf-8");+ try { resolve(raw ? JSON.parse(raw) : null); } catch (err) { reject(err); }+ });+ req.on("error", reject);+ });+}++function writeJson(res, value) {+ res.writeHead(200, { "Content-Type": "application/json" });+ res.end(JSON.stringify(value));+}++async function withMockOpenViking(handler, fn) {+ const server = http.createServer((req, res) => {+ Promise.resolve(handler(req, res)).catch((err) => {+ res.writeHead(500, { "Content-Type": "application/json" });+ res.end(JSON.stringify({ status: "error", error: String(err?.stack || err) }));+ });+ });+ await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve));+ try {+ const { port } = server.address();+ return await fn(`http://127.0.0.1:${port}`);+ } finally {+ await new Promise((resolve) => server.close(resolve));+ }+}++function runSessionEnd(input, env) {+ return new Promise((resolve, reject) => {+ const cleanEnv = { ...process.env };+ for (const key of Object.keys(cleanEnv)) {+ if (key.startsWith("OPENVIKING_") || key === "OV_HOOK_WORKER") delete cleanEnv[key];+ }+ const child = spawn(process.execPath, [join(SCRIPT_DIR, "session-end.mjs")], {+ env: { ...cleanEnv, ...env },+ stdio: ["pipe", "pipe", "pipe"],+ });+ let stdout = "";+ let stderr = "";+ child.stdout.on("data", (chunk) => { stdout += chunk.toString(); });+ child.stderr.on("data", (chunk) => { stderr += chunk.toString(); });+ child.on("error", reject);+ child.on("close", (code) => {+ if (code !== 0) {+ reject(new Error(`session-end exited ${code}: ${stderr}`));+ return;+ }+ resolve({ stdout, stderr });+ });+ child.stdin.end(JSON.stringify(input));+ });+}++function workerEnv(baseUrl, stateDir, extra = {}) {+ return {+ OV_HOOK_WORKER: "1",+ OPENVIKING_URL: baseUrl,+ OPENVIKING_AUTO_CAPTURE: "1",+ OPENVIKING_CAPTURE_ASSISTANT_TURNS: "1",+ OPENVIKING_CODEX_STATE_DIR: stateDir,+ OPENVIKING_CONFIG_FILE: join(stateDir, "missing-ov.conf"),+ OPENVIKING_CLI_CONFIG_FILE: join(stateDir, "missing-ovcli.conf"),+ OPENVIKING_CREDENTIAL_SOURCE: "env",+ OPENVIKING_MIN_QUERY_LENGTH: "1",+ OPENVIKING_TIMEOUT_MS: "5000",+ OPENVIKING_CAPTURE_TIMEOUT_MS: "5000",+ ...extra,+ };+}++function turn(role, content) {+ return JSON.stringify({ payload: { message: { role, content } } });+}++async function writeTranscript(path, count) {+ const lines = [];+ for (let i = 0; i < count; i += 1) {+ lines.push(turn(i % 2 === 0 ? "user" : "assistant", `turn-${i}`));+ }+ await writeFile(path, lines.join("\n"));+}++async function writeState(stateDir, id, patch = {}) {+ const now = Date.now();+ await mkdir(stateDir, { recursive: true });+ await writeFile(join(stateDir, `${id}.json`), JSON.stringify({+ codexSessionId: id,+ ovSessionId: `cx-${id}`,+ capturedTurnCount: 0,+ createdAt: now - 1000,+ lastUpdatedAt: now,+ ...patch,+ }));+}++function readState(stateDir, id) {+ return readFile(join(stateDir, `${id}.json`), "utf-8").then(JSON.parse);+}++async function exists(path) {+ try { await stat(path); return true; } catch { return false; }+}++function mockHandler(calls, { commitStatus = 200 } = {}) {+ return async (req, res) => {+ const url = new URL(req.url, "http://127.0.0.1");+ const call = { method: req.method, path: url.pathname, body: null };+ calls.push(call);+ if (req.method === "GET" && url.pathname === "/health") {+ writeJson(res, { status: "ok", result: { ok: true } });+ return;+ }+ if (req.method === "POST" && url.pathname.endsWith("/messages/batch")) {+ call.body = await readRequestBody(req);+ writeJson(res, { status: "ok", result: { ok: true } });+ return;+ }+ if (req.method === "POST" && url.pathname.endsWith("/commit")) {+ call.body = await readRequestBody(req);+ if (commitStatus !== 200) {+ res.writeHead(commitStatus, { "Content-Type": "application/json" });+ res.end(JSON.stringify({ status: "error", error: { code: "INTERNAL", message: "commit failed", trace_id: "trace-end-error" } }));+ return;+ }+ writeJson(res, { status: "ok", result: { archived: true, task_id: "task-end", trace_id: "trace-end" } });+ return;+ }+ res.writeHead(404, { "Content-Type": "application/json" });+ res.end(JSON.stringify({ status: "error", error: "not found" }));+ };+}++function sentMessages(calls) {+ return calls+ .filter((c) => c.path.endsWith("/messages/batch") || c.path.endsWith("/messages"))+ .flatMap((c) => c.body?.messages ?? (c.body ? [c.body] : []));+}++test("session-end catches up the missing turns then commits", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-session-end-"));+ const transcriptPath = join(stateDir, "transcript.jsonl");+ const calls = [];+ try {+ await writeState(stateDir, "s1", { capturedTurnCount: 2 });+ await writeTranscript(transcriptPath, 4);+ await withMockOpenViking(mockHandler(calls), async (baseUrl) => {+ await runSessionEnd(+ { session_id: "s1", transcript_path: transcriptPath, hook_event_name: "SessionEnd" },+ workerEnv(baseUrl, stateDir),+ );+ });++ const messages = sentMessages(calls);+ assert.equal(messages.length, 2);+ assert.equal(messages[0].parts?.[0]?.text ?? messages[0].content, "turn-2");+ assert.equal(messages[1].parts?.[0]?.text ?? messages[1].content, "turn-3");+ assert.ok(calls.some((c) => c.method === "POST" && c.path === "/api/v1/sessions/cx-s1/commit"));+ assert.deepEqual(calls.find((c) => c.path.endsWith("/commit")).body, {});++ const state = await readState(stateDir, "s1");+ assert.equal(state.ovSessionId, null);+ assert.equal(state.capturedTurnCount, 4);+ assert.equal(await endedMarkerExists(stateDir, "s1"), false);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("a second session-end on an unchanged transcript neither sends nor commits", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-session-end-idem-"));+ const transcriptPath = join(stateDir, "transcript.jsonl");+ const calls = [];+ try {+ await writeState(stateDir, "s2", { capturedTurnCount: 4, ovSessionId: null });+ await writeTranscript(transcriptPath, 4);+ await withMockOpenViking(mockHandler(calls), async (baseUrl) => {+ await runSessionEnd(+ { session_id: "s2", transcript_path: transcriptPath },+ workerEnv(baseUrl, stateDir),+ );+ });++ assert.equal(sentMessages(calls).length, 0);+ assert.equal(calls.some((c) => c.path.endsWith("/commit")), false);+ const state = await readState(stateDir, "s2");+ assert.equal(state.capturedTurnCount, 4);+ assert.equal(state.ovSessionId, null);+ assert.equal(await endedMarkerExists(stateDir, "s2"), false);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("a failed commit keeps the live session and the end marker", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-session-end-fail-"));+ const transcriptPath = join(stateDir, "transcript.jsonl");+ const calls = [];+ try {+ await writeState(stateDir, "s3", { capturedTurnCount: 0 });+ await writeTranscript(transcriptPath, 2);+ await withMockOpenViking(mockHandler(calls, { commitStatus: 500 }), async (baseUrl) => {+ await runSessionEnd(+ { session_id: "s3", transcript_path: transcriptPath },+ workerEnv(baseUrl, stateDir),+ );+ });++ const state = await readState(stateDir, "s3");+ assert.equal(state.ovSessionId, "cx-s3");+ assert.equal(state.capturedTurnCount, 2);+ assert.equal(await endedMarkerExists(stateDir, "s3"), true);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("an unreachable server leaves the cursor and the end marker alone", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-session-end-down-"));+ const transcriptPath = join(stateDir, "transcript.jsonl");+ try {+ await writeState(stateDir, "s4", { capturedTurnCount: 2 });+ await writeTranscript(transcriptPath, 6);+ const closedPort = await withMockOpenViking(() => {}, async (baseUrl) => baseUrl);+ await runSessionEnd(+ { session_id: "s4", transcript_path: transcriptPath },+ workerEnv(closedPort, stateDir, { OPENVIKING_CAPTURE_TIMEOUT_MS: "1500" }),+ );++ const state = await readState(stateDir, "s4");+ assert.equal(state.capturedTurnCount, 2);+ assert.equal(state.ovSessionId, "cx-s4");+ assert.equal(await endedMarkerExists(stateDir, "s4"), true);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("a missing transcript never resets the cursor", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-session-end-notranscript-"));+ const calls = [];+ try {+ await writeState(stateDir, "s5", { capturedTurnCount: 8, ovSessionId: null });+ await withMockOpenViking(mockHandler(calls), async (baseUrl) => {+ await runSessionEnd(+ { session_id: "s5", transcript_path: join(stateDir, "gone.jsonl") },+ workerEnv(baseUrl, stateDir),+ );+ });++ assert.equal(sentMessages(calls).length, 0);+ const state = await readState(stateDir, "s5");+ assert.equal(state.capturedTurnCount, 8);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("an unreadable transcript never commits the live session", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-session-end-unreadable-"));+ const calls = [];+ try {+ await writeState(stateDir, "s13", { capturedTurnCount: 3 });+ await withMockOpenViking(mockHandler(calls), async (baseUrl) => {+ await runSessionEnd(+ { session_id: "s13", transcript_path: join(stateDir, "gone.jsonl") },+ workerEnv(baseUrl, stateDir),+ );+ });++ assert.equal(sentMessages(calls).length, 0);+ assert.equal(calls.some((c) => c.path.endsWith("/commit")), false);+ const state = await readState(stateDir, "s13");+ assert.equal(state.ovSessionId, "cx-s13");+ assert.equal(state.capturedTurnCount, 3);+ assert.equal(await endedMarkerExists(stateDir, "s13"), true);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("a shrunk transcript resumes at the last human turn", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-session-end-shrink-"));+ const transcriptPath = join(stateDir, "transcript.jsonl");+ const calls = [];+ try {+ await writeState(stateDir, "s6", { capturedTurnCount: 8 });+ // 6 turns, last user turn at index 2.+ await writeFile(transcriptPath, [+ turn("user", "old-a"),+ turn("assistant", "old-b"),+ turn("user", "current request"),+ turn("assistant", "current reply"),+ turn("assistant", "more"),+ turn("assistant", "tail"),+ ].join("\n"));++ await withMockOpenViking(mockHandler(calls), async (baseUrl) => {+ await runSessionEnd(+ { session_id: "s6", transcript_path: transcriptPath },+ workerEnv(baseUrl, stateDir),+ );+ });++ const messages = sentMessages(calls);+ assert.equal(messages.length, 4);+ assert.equal(messages[0].parts?.[0]?.text ?? messages[0].content, "current request");+ const state = await readState(stateDir, "s6");+ assert.equal(state.capturedTurnCount, 6);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("a fresh lock blocks the worker; a stale one does not", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-session-end-lock-"));+ const transcriptPath = join(stateDir, "transcript.jsonl");+ const calls = [];+ try {+ await writeState(stateDir, "s7", { capturedTurnCount: 0 });+ await writeTranscript(transcriptPath, 2);+ const lockDir = join(stateDir, "s7.lock");+ await mkdir(lockDir, { recursive: true });++ await withMockOpenViking(mockHandler(calls), async (baseUrl) => {+ await runSessionEnd(+ { session_id: "s7", transcript_path: transcriptPath },+ workerEnv(baseUrl, stateDir, { OPENVIKING_CODEX_LOCK_WAIT_MS: "300" }),+ );++ assert.equal(calls.length, 0, "a held lock must stop every HTTP call");+ assert.equal((await readState(stateDir, "s7")).capturedTurnCount, 0);++ // Backdate the lock past the stale window so the next run takes it over.+ const old = new Date(Date.now() - 10 * 60_000);+ await utimes(lockDir, old, old);+ await runSessionEnd(+ { session_id: "s7", transcript_path: transcriptPath },+ workerEnv(baseUrl, stateDir, { OPENVIKING_CODEX_LOCK_WAIT_MS: "300" }),+ );+ });++ assert.equal(sentMessages(calls).length, 2);+ const state = await readState(stateDir, "s7");+ assert.equal(state.ovSessionId, null);+ assert.equal(state.capturedTurnCount, 2);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("a concurrent Stop worker and session-end worker never double-send a turn", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-session-end-race-"));+ const transcriptPath = join(stateDir, "transcript.jsonl");+ const calls = [];+ try {+ await writeState(stateDir, "s8", { capturedTurnCount: 0 });+ await writeTranscript(transcriptPath, 6);++ await withMockOpenViking(async (req, res) => {+ const url = new URL(req.url, "http://127.0.0.1");+ if (url.pathname.endsWith("/messages/batch")) {+ await new Promise((resolve) => setTimeout(resolve, 500));+ }+ return mockHandler(calls)(req, res);+ }, async (baseUrl) => {+ const env = workerEnv(baseUrl, stateDir, { OPENVIKING_WRITE_PATH_ASYNC: "0" });+ const stop = new Promise((resolve, reject) => {+ const cleanEnv = { ...process.env };+ for (const key of Object.keys(cleanEnv)) {+ if (key.startsWith("OPENVIKING_") || key === "OV_HOOK_WORKER") delete cleanEnv[key];+ }+ const child = spawn(process.execPath, [join(SCRIPT_DIR, "auto-capture.mjs")], {+ env: { ...cleanEnv, ...env },+ stdio: ["pipe", "ignore", "ignore"],+ });+ child.on("error", reject);+ child.on("close", resolve);+ child.stdin.end(JSON.stringify({ session_id: "s8", transcript_path: transcriptPath }));+ });+ const end = runSessionEnd({ session_id: "s8", transcript_path: transcriptPath }, env);+ await Promise.all([stop, end]);+ });++ assert.equal(sentMessages(calls).length, 6, "each transcript turn must be sent exactly once");+ const state = await readState(stateDir, "s8");+ assert.equal(state.capturedTurnCount, 6);+ assert.equal(state.ovSessionId, null);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("session-end is inert when auto-capture is disabled", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-session-end-off-"));+ const calls = [];+ try {+ await withMockOpenViking(mockHandler(calls), async (baseUrl) => {+ const { stdout } = await runSessionEnd(+ { session_id: "s9", transcript_path: null },+ workerEnv(baseUrl, stateDir, { OPENVIKING_AUTO_CAPTURE: "0" }),+ );+ assert.deepEqual(JSON.parse(stdout.trim()), {});+ });+ assert.equal(calls.length, 0);+ assert.equal(await endedMarkerExists(stateDir, "s9"), false);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("the parent hook returns immediately and leaves the end marker behind", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-session-end-parent-"));+ const transcriptPath = join(stateDir, "transcript.jsonl");+ const calls = [];+ try {+ await writeState(stateDir, "s10", { capturedTurnCount: 0 });+ await writeTranscript(transcriptPath, 2);+ await withMockOpenViking(mockHandler(calls), async (baseUrl) => {+ const env = workerEnv(baseUrl, stateDir);+ delete env.OV_HOOK_WORKER;+ const started = Date.now();+ const { stdout } = await runSessionEnd(+ { session_id: "s10", transcript_path: transcriptPath },+ env,+ );+ const elapsed = Date.now() - started;+ assert.deepEqual(JSON.parse(stdout.trim()), {});+ assert.ok(elapsed < 1000, `parent hook took ${elapsed}ms; Codex budgets 1s`);+ assert.equal(await endedMarkerExists(stateDir, "s10"), true);+ });+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("session-end without a session_id writes nothing", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-session-end-noid-"));+ const calls = [];+ try {+ await withMockOpenViking(mockHandler(calls), async (baseUrl) => {+ const { stdout } = await runSessionEnd(+ { transcript_path: null, hook_event_name: "SessionEnd" },+ workerEnv(baseUrl, stateDir),+ );+ assert.deepEqual(JSON.parse(stdout.trim()), {});+ });+ assert.equal(calls.length, 0);+ assert.deepEqual((await readdir(stateDir)).filter((n) => n !== "recall-compressor-profile.json"), []);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("a partial catch-up keeps the live session and the end marker instead of committing", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-session-end-partial-"));+ const transcriptPath = join(stateDir, "transcript.jsonl");+ const calls = [];+ try {+ await writeState(stateDir, "s11", { capturedTurnCount: 0 });+ await writeTranscript(transcriptPath, 2);++ // Batch is unavailable, so the sender falls back to serial; the second+ // message then fails, leaving the tail turn unsent.+ let serial = 0;+ await withMockOpenViking(async (req, res) => {+ const url = new URL(req.url, "http://127.0.0.1");+ calls.push({ method: req.method, path: url.pathname, body: null });+ if (req.method === "GET" && url.pathname === "/health") {+ writeJson(res, { status: "ok", result: { ok: true } });+ return;+ }+ if (req.method === "POST" && url.pathname.endsWith("/messages/batch")) {+ res.writeHead(404, { "Content-Type": "application/json" });+ res.end(JSON.stringify({ status: "error", error: "no batch endpoint" }));+ return;+ }+ if (req.method === "POST" && url.pathname.endsWith("/messages")) {+ calls[calls.length - 1].body = await readRequestBody(req);+ serial += 1;+ if (serial === 1) {+ writeJson(res, { status: "ok", result: { ok: true } });+ return;+ }+ res.writeHead(500, { "Content-Type": "application/json" });+ res.end(JSON.stringify({ status: "error", error: { message: "boom" } }));+ return;+ }+ writeJson(res, { status: "ok", result: {} });+ }, async (baseUrl) => {+ await runSessionEnd(+ { session_id: "s11", transcript_path: transcriptPath },+ workerEnv(baseUrl, stateDir),+ );+ });++ assert.equal(sentMessages(calls).length, 2, "both serial attempts are made");+ assert.equal(calls.some((c) => c.path.endsWith("/commit")), false, "an incomplete append must not commit");+ const state = await readState(stateDir, "s11");+ assert.equal(state.ovSessionId, "cx-s11");+ assert.equal(state.capturedTurnCount, 1, "the cursor advances only past what landed");+ assert.equal(await endedMarkerExists(stateDir, "s11"), true);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("a worker whose end token no longer matches the marker does nothing", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-session-end-superseded-"));+ const transcriptPath = join(stateDir, "transcript.jsonl");+ const calls = [];+ try {+ await writeState(stateDir, "s12", { capturedTurnCount: 0 });+ await writeTranscript(transcriptPath, 2);+ const markerAt = Date.now();+ await writeEndedMarker(stateDir, "s12", markerAt);++ await withMockOpenViking(mockHandler(calls), async (baseUrl) => {+ await runSessionEnd(+ { session_id: "s12", transcript_path: transcriptPath },+ workerEnv(baseUrl, stateDir, {+ OPENVIKING_SESSION_END_TOKEN: String(markerAt - 5_000),+ }),+ );+ });++ assert.equal(calls.length, 0, "a superseded worker makes no HTTP calls at all");+ const state = await readState(stateDir, "s12");+ assert.equal(state.ovSessionId, "cx-s12");+ assert.equal(state.capturedTurnCount, 0);+ assert.deepEqual(+ await endedMarkerStamps(stateDir, "s12"),+ [markerAt],+ "the newer marker survives",+ );+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});diff --git a/examples/codex-memory-plugin/scripts/session-start-commit.mjs b/examples/codex-memory-plugin/scripts/session-start-commit.mjsindex 70b774334..8fdfcfef6 100644--- a/examples/codex-memory-plugin/scripts/session-start-commit.mjs+++ b/examples/codex-memory-plugin/scripts/session-start-commit.mjs@@ -14,20 +14,20 @@* abstract-annotated indexes of preferences/ and entities/, capped by* OPENVIKING_PROFILE_TOKEN_BUDGET with the shared CJK-aware estimator.*- * Behavior (see DESIGN.md §3 — "SessionStart source=startup, heuristic"):- * On `startup` or `clear`, run the active-window heuristic over state files- * excluding the new session_id:- * - 0 recently-active → no-op- * - 1 recently-active → commit it (the just-ended session)- * - ≥2 recently-active → skip; rely on idle TTL- * "Recently-active" means lastUpdatedAt within ACTIVE_WINDOW_MS (default 2 min).- * Concurrency is judged on activity alone; only a state that still has a live- * ovSessionId is actually committed.+ * Behavior (see DESIGN.md — "Fallback sweep"):+ * Committing a finished thread is SessionEnd's job. On `startup` or `clear`+ * this hook only sweeps what SessionEnd could not reach: a state file with a+ * live OV session is committed when it carries an `.ended` marker (the+ * SessionEnd worker was killed or the server was down) or when it has been+ * idle for more than IDLE_TTL_MS (default 30 min) — signals, crashes, Codex+ * older than 0.145, and app-server threads whose SessionEnd is deferred.*- * At the tail (regardless of which branch above ran), run an idle-TTL sweep:- * any live OV session state older than IDLE_TTL_MS (default 30 min) gets- * committed while retaining its transcript cursor. This catches- * SIGTERM/Ctrl+C/`/exit` exits and crashes that left sessions orphaned.+ * Each candidate is committed under its session lock with no waiting: a lock+ * we cannot take means a SessionEnd or Stop worker already owns the session.+ * Under the lock the sweep first appends whatever the state's recorded+ * transcript still holds past the cursor, so a session whose workers never+ * ran is not archived without its tail turns; an incomplete append keeps the+ * session live for the next sweep instead of committing.** The same pass retires cursor-only states (no live OV session): a cursor* that was never used is dropped after IDLE_TTL_MS, and a real cursor is@@ -46,8 +46,18 @@import { loadConfig } from "./config.mjs";import { createLogger } from "./debug-log.mjs";+import { catchUpTurns, makeFetchJSON } from "./ov-session.mjs";import { detectRecallCompressorProfile } from "./recall-compressor-profile.mjs";-import { clearState, deriveOvSessionId, listStates, loadState, saveState } from "./session-state.mjs";+import {+ clearEnded,+ clearState,+ deriveOvSessionId,+ listStates,+ loadState,+ readEndedAt,+ saveState,+ withSessionLock,+} from "./session-state.mjs";import { buildProfileBlock } from "./shared/profile-inject.mjs";import { resolveEffectivePeerId } from "./shared/workspace-peer.mjs";@@ -55,16 +65,17 @@ const cfg = loadConfig();const { log, logError } = createLogger("session-start");let activePeerId = cfg.peerId || "";-const ACTIVE_WINDOW_MS = (() => {- const v = Number(process.env.OPENVIKING_CODEX_ACTIVE_WINDOW_MS);- return Number.isFinite(v) && v > 0 ? Math.floor(v) : 120_000;-})();-const IDLE_TTL_MS = (() => {const v = Number(process.env.OPENVIKING_CODEX_IDLE_TTL_MS);return Number.isFinite(v) && v > 0 ? Math.floor(v) : 1_800_000;})();+const HOOK_STARTED_AT = Date.now();++// The sweep catches up unsent turns before committing, so it needs the same+// HTTP helper the capture hooks use.+const { fetchJSONRes } = makeFetchJSON(cfg, { getActorPeerId: () => activePeerId });+const COMMITTED_TTL_MS = (() => {const v = Number(process.env.OPENVIKING_CODEX_COMMITTED_TTL_MS);return Number.isFinite(v) && v > 0 ? Math.floor(v) : 2_592_000_000;@@ -250,7 +261,7 @@ async function buildResumeArchiveContext(newSessionId) {** Returns { committed: bool, ovSessionId: string|null, traceId: string }.*/-async function commitAndRelease(state, reason) {+async function commitAndRelease(state, reason, endToken) {const ovSessionId = state.ovSessionId;const commit = await commitOvSession(ovSessionId);if (!commit?.ok) {@@ -277,6 +288,12 @@ async function commitAndRelease(state, reason) {});state.ovSessionId = null;await saveState(state, { touch: false });+ // Only retire the marker this pass verified; a newer one belongs to an exit+ // that happened while we were committing.+ await clearEnded(+ state.codexSessionId,+ typeof endToken === "number" ? { before: endToken + 1 } : {},+ );return { committed: true, ovSessionId, traceId };}@@ -336,7 +353,6 @@ async function main() {log("start", {source,newSessionId,- activeWindowMs: ACTIVE_WINDOW_MS,idleTtlMs: IDLE_TTL_MS,peerSource: effectivePeer.source,});@@ -348,6 +364,9 @@ async function main() {}if (source === "resume") {+ // Earliest liveness signal after `codex resume`: the thread is running+ // again, so a marker left by its previous exit must not trigger the sweep.+ await clearEnded(newSessionId, { before: HOOK_STARTED_AT });const health = await fetchJSON("/health");if (!health) {logError("health_check", "server unreachable; skipping profile + archive injection");@@ -380,96 +399,120 @@ async function main() {const profileContext = await buildSessionProfileContext();const now = Date.now();- const states = await listStates();-- // -------------------------------------------------------------------------- // Active-window heuristic (DESIGN.md §3)- // -------------------------------------------------------------------------- const otherStates = states.filter(- (s) => s?.codexSessionId && s.codexSessionId !== newSessionId,- );-- // Concurrency is judged on activity, not on whether an OV session is live:- // a session that PreCompact just committed is cursor-only yet may still be- // running, and counting it keeps the ≥2 branch from committing a sibling- // session mid-flight.- const recentlyActive = otherStates.filter(- (s) => typeof s.lastUpdatedAt === "number"- && (now - s.lastUpdatedAt) <= ACTIVE_WINDOW_MS,- );-- const heuristicCommits = [];- const skippedSessionIds = new Set();-- if (recentlyActive.length === 0) {- log("heuristic", { branch: "0_active", action: "noop", otherStates: otherStates.length });- } else if (recentlyActive.length === 1) {- const target = recentlyActive[0];- if (!target.ovSessionId) {- log("heuristic", {- branch: "1_active",- action: "skip",- reason: "no live OV session",- codexSessionId: target.codexSessionId,- });- } else {- log("heuristic", {- branch: "1_active",- action: "commit",- codexSessionId: target.codexSessionId,- ovSessionId: target.ovSessionId,- });- const r = await commitAndRelease(target, "heuristic_1_active");- if (r.committed) heuristicCommits.push(r);- }- } else {- log("heuristic", {- branch: ">=2_active",- action: "skip; rely on idle TTL",- activeCount: recentlyActive.length,- activeIds: recentlyActive.map((s) => s.codexSessionId),- });- for (const s of recentlyActive) skippedSessionIds.add(s.codexSessionId);- }-- // -------------------------------------------------------------------------- // Idle TTL sweep + cursor retention (tail) — applies to ALL state files- // including ones we just skipped above (≥2 active path). We re-list because- // the heuristic branch may have changed entries.- // -------------------------------------------------------------------------- const postHeuristic = await listStates();- const idleCommits = [];+ const commits = [];let retired = 0;+ let lockSkipped = 0;- for (const s of postHeuristic) {+ for (const s of await listStates()) {if (!s?.codexSessionId) continue;+ if (s.codexSessionId === newSessionId) continue;if (typeof s.lastUpdatedAt !== "number") continue;const ageMs = now - s.lastUpdatedAt;- if (!s.ovSessionId) {++ // A marker must always enter the lock, even with no live id: PreCompact+ // releases the id while leaving a cursor behind, so the tail turns of a+ // session whose SessionEnd worker died are only reachable through a+ // catch-up under the lock.+ if (!s.ovSessionId && !s.endedAt) {if (await maybeRetireCursorState(s, ageMs)) retired += 1;continue;}- if (ageMs <= IDLE_TTL_MS) continue;- log("idle_sweep", {- codexSessionId: s.codexSessionId,- ovSessionId: s.ovSessionId,- ageMs,- });- const r = await commitAndRelease(s, "idle_ttl");- if (r.committed) idleCommits.push(r);+ if (!s.endedAt && ageMs <= IDLE_TTL_MS) continue;++ const reason = s.endedAt ? "ended_retry" : "idle_ttl";+ log("sweep", { codexSessionId: s.codexSessionId, ovSessionId: s.ovSessionId, ageMs, reason });+ // Try-lock only: a held lock means a SessionEnd or Stop worker is already+ // committing this session (user quit and relaunched within seconds).+ const outcome = await withSessionLock(s.codexSessionId, async ({ heartbeat }) => {+ const fresh = await loadState(s.codexSessionId);++ // Re-read the marker under the lock: it may have been cleared by a+ // resume or replaced by a newer exit since listStates() sampled it.+ const endToken = await readEndedAt(s.codexSessionId);+ if (reason === "ended_retry" && (endToken === undefined || endToken > s.endedAt)) {+ if (ageMs <= IDLE_TTL_MS) {+ log("sweep_skip", {+ codexSessionId: s.codexSessionId,+ reason: endToken === undefined ? "marker cleared under the lock" : "newer end marker",+ });+ return null;+ }+ }++ // Turns the session's own workers never sent would be lost by an+ // archive-now commit, so catch them up first and keep the session live+ // for the next sweep if any of them failed to land.+ const { newTurns, added, skipped, unreadable } = await catchUpTurns({+ state: fresh,+ transcriptPath: fresh.transcriptPath,+ fetchJSONRes,+ activePeerId: fresh.workspacePeerId || activePeerId,+ cfg,+ log,+ logError,+ heartbeat,+ });+ if (added > 0) log("appended_catchup", { ovSessionId: fresh.ovSessionId, added });++ // An unreadable transcript is not an empty one: the tail turns may still+ // be there. Keep the live id and the marker for the next sweep.+ if (unreadable) {+ logError("transcript_unreadable", {+ codexSessionId: s.codexSessionId,+ ovSessionId: fresh.ovSessionId,+ transcriptPath: fresh.transcriptPath,+ });+ await saveState(fresh, { touch: false });+ return null;+ }++ if (newTurns.length > 0 && !skipped && added < newTurns.length) {+ logError("append_incomplete", {+ codexSessionId: s.codexSessionId,+ ovSessionId: fresh.ovSessionId,+ attempted: newTurns.length,+ added,+ });+ await saveState(fresh, { touch: false });+ return null;+ }++ // The catch-up derives a live id whenever it sends something; still+ // having none means there is genuinely nothing to commit, so the marker+ // can go.+ if (!fresh.ovSessionId) {+ if (added > 0) await saveState(fresh, { touch: false });+ await clearEnded(+ s.codexSessionId,+ typeof endToken === "number" ? { before: endToken + 1 } : {},+ );+ log("skip", {+ stage: "commit",+ codexSessionId: s.codexSessionId,+ reason: "no live OV session for this codex session",+ });+ return null;+ }++ return commitAndRelease(fresh, reason, endToken);+ }, { waitMs: 0 });++ if (outcome.skipped) {+ lockSkipped += 1;+ log("sweep_skip", { codexSessionId: s.codexSessionId, reason: "locked by another writer" });+ continue;+ }+ if (outcome.value?.committed) commits.push(outcome.value);}- const commits = [...heuristicCommits, ...idleCommits];const ovSessionIds = commits.map((item) => item.ovSessionId);log("done", {source,- heuristicCommitted: heuristicCommits.length,- idleCommitted: idleCommits.length,- totalCommitted: commits.length,+ committed: commits.length,retired,+ lockSkipped,ovSessionIds,- skipped: [...skippedSessionIds],});if (commits.length > 0) {diff --git a/examples/codex-memory-plugin/scripts/session-start-commit.test.mjs b/examples/codex-memory-plugin/scripts/session-start-commit.test.mjsindex 81567f073..f36d20b10 100644--- a/examples/codex-memory-plugin/scripts/session-start-commit.test.mjs+++ b/examples/codex-memory-plugin/scripts/session-start-commit.test.mjs@@ -1,6 +1,6 @@import assert from "node:assert/strict";import { spawn } from "node:child_process";-import { mkdir, mkdtemp, readFile, readdir, rm, writeFile } from "node:fs/promises";+import { mkdir, mkdtemp, readFile, readdir, rm, stat, writeFile } from "node:fs/promises";import http from "node:http";import { tmpdir } from "node:os";import { dirname, join } from "node:path";@@ -9,6 +9,24 @@ import { fileURLToPath } from "node:url";const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url));+async function endedMarkerStamps(dir, id) {+ const prefix = `${id}.ended.`;+ const files = await readdir(dir).catch(() => []);+ return files+ .filter((name) => name.startsWith(prefix))+ .map((name) => Number(name.slice(prefix.length)))+ .filter((ts) => Number.isFinite(ts));+}++async function endedMarkerExists(dir, id) {+ return (await endedMarkerStamps(dir, id)).length > 0;+}++function writeEndedMarker(dir, id, ts) {+ return writeFile(join(dir, `${id}.ended.${ts}`), String(ts));+}++function writeJson(res, value) {res.writeHead(200, { "Content-Type": "application/json" });res.end(JSON.stringify(value));@@ -188,7 +206,11 @@ test("startup injects the shared profile block with workspace peer routing", asy}});-test("startup preserves the existing commit systemMessage alongside profile context", async () => {+async function exists(path) {+ try { await stat(path); return true; } catch { return false; }+}++test("startup commits a session whose SessionEnd marker is still present", async () => {const stateDir = await mkdtemp(join(tmpdir(), "ov-codex-session-commit-"));const requests = [];try {@@ -201,6 +223,7 @@ test("startup preserves the existing commit systemMessage alongside profile contcreatedAt: now - 1000,lastUpdatedAt: now,}));+ await writeEndedMarker(stateDir, "old-session", now);await withMockOpenViking(profileHandler(requests), async (baseUrl) => {const { output } = await runSessionStart(@@ -227,6 +250,7 @@ test("startup preserves the existing commit systemMessage alongside profile contconst state = JSON.parse(await readFile(join(stateDir, "old-session.json"), "utf-8"));assert.equal(state.ovSessionId, null);assert.equal(state.capturedTurnCount, 2);+ assert.equal(await endedMarkerExists(stateDir, "old-session"), false);} finally {await rm(stateDir, { recursive: true, force: true });}@@ -259,7 +283,6 @@ test("startup ignores committed cursor-only states", async () => {{ session_id: "new-session", source: "startup", cwd: "/tmp/codex-cursor-only" },{...baseEnv(baseUrl, stateDir),- OPENVIKING_CODEX_ACTIVE_WINDOW_MS: "1000",OPENVIKING_CODEX_IDLE_TTL_MS: "5000",},);@@ -311,7 +334,6 @@ test("startup retires cursor-only states once their retention window closes", as{ session_id: "new-session", source: "startup", cwd: "/tmp/codex-cursor-retention" },{...baseEnv(baseUrl, stateDir),- OPENVIKING_CODEX_ACTIVE_WINDOW_MS: "500",OPENVIKING_CODEX_IDLE_TTL_MS: "5000",OPENVIKING_CODEX_COMMITTED_TTL_MS: "8000",},@@ -328,45 +350,365 @@ test("startup retires cursor-only states once their retention window closes", as}});-test("startup defers to idle TTL when a cursor-only session is still active", async () => {- const stateDir = await mkdtemp(join(tmpdir(), "ov-codex-active-cursor-"));+test("startup leaves a fresh live session alone until it ends or goes idle", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-codex-active-live-"));const requests = [];try {const now = Date.now();- await Promise.all([- writeFile(join(stateDir, "live.json"), JSON.stringify({- codexSessionId: "live",- ovSessionId: "cx-live",- capturedTurnCount: 2,- createdAt: now - 2_000,- lastUpdatedAt: now - 100,- })),- // Just committed by PreCompact, so cursor-only — but still running.- writeFile(join(stateDir, "compacted.json"), JSON.stringify({- codexSessionId: "compacted",- ovSessionId: null,- capturedTurnCount: 5,- createdAt: now - 2_000,- lastUpdatedAt: now - 100,- })),- ]);+ await writeFile(join(stateDir, "live.json"), JSON.stringify({+ codexSessionId: "live",+ ovSessionId: "cx-live",+ capturedTurnCount: 2,+ createdAt: now - 2_000,+ lastUpdatedAt: now - 100,+ }));await withMockOpenViking(profileHandler(requests), async (baseUrl) => {const { output } = await runSessionStart(- { session_id: "new-session", source: "startup", cwd: "/tmp/codex-active-cursor" },- {- ...baseEnv(baseUrl, stateDir),- OPENVIKING_CODEX_ACTIVE_WINDOW_MS: "60000",- OPENVIKING_CODEX_IDLE_TTL_MS: "1800000",- },+ { session_id: "new-session", source: "startup", cwd: "/tmp/codex-active-live" },+ { ...baseEnv(baseUrl, stateDir), OPENVIKING_CODEX_IDLE_TTL_MS: "1800000" },+ );+ assert.equal(output.systemMessage, undefined);+ });++ assert.equal(requests.some((request) => request.path.endsWith("/commit")), false);+ assert.equal(JSON.parse(await readFile(join(stateDir, "live.json"), "utf-8")).ovSessionId, "cx-live");+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("startup commits a live session once it passes the idle TTL", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-codex-idle-commit-"));+ const requests = [];+ try {+ const now = Date.now();+ await writeFile(join(stateDir, "idle.json"), JSON.stringify({+ codexSessionId: "idle",+ ovSessionId: "cx-idle",+ capturedTurnCount: 3,+ createdAt: now - 60_000,+ lastUpdatedAt: now - 30_000,+ }));++ await withMockOpenViking(profileHandler(requests), async (baseUrl) => {+ const { output } = await runSessionStart(+ { session_id: "new-session", source: "startup", cwd: "/tmp/codex-idle-commit" },+ { ...baseEnv(baseUrl, stateDir), OPENVIKING_CODEX_IDLE_TTL_MS: "5000" },+ );+ assert.match(output.systemMessage, /cx-idle is committed/);+ });++ const state = JSON.parse(await readFile(join(stateDir, "idle.json"), "utf-8"));+ assert.equal(state.ovSessionId, null);+ assert.equal(state.capturedTurnCount, 3);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("startup skips a session another writer already holds the lock for", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-codex-sweep-lock-"));+ const requests = [];+ try {+ const now = Date.now();+ await writeFile(join(stateDir, "locked.json"), JSON.stringify({+ codexSessionId: "locked",+ ovSessionId: "cx-locked",+ capturedTurnCount: 2,+ createdAt: now - 2_000,+ lastUpdatedAt: now - 100,+ }));+ await writeEndedMarker(stateDir, "locked", now);+ await mkdir(join(stateDir, "locked.lock"), { recursive: true });++ await withMockOpenViking(profileHandler(requests), async (baseUrl) => {+ const { output } = await runSessionStart(+ { session_id: "new-session", source: "startup", cwd: "/tmp/codex-sweep-lock" },+ baseEnv(baseUrl, stateDir),);assert.equal(output.systemMessage, undefined);});assert.equal(requests.some((request) => request.path.endsWith("/commit")), false);- const live = JSON.parse(await readFile(join(stateDir, "live.json"), "utf-8"));- assert.equal(live.ovSessionId, "cx-live");- assert.equal(JSON.parse(await readFile(join(stateDir, "compacted.json"), "utf-8")).capturedTurnCount, 5);+ const state = JSON.parse(await readFile(join(stateDir, "locked.json"), "utf-8"));+ assert.equal(state.ovSessionId, "cx-locked");+ assert.equal(await endedMarkerExists(stateDir, "locked"), true);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("resume clears a stale SessionEnd marker for the resumed session", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-codex-resume-ended-"));+ const requests = [];+ try {+ const now = Date.now();+ await writeFile(join(stateDir, "resumed.json"), JSON.stringify({+ codexSessionId: "resumed",+ ovSessionId: null,+ capturedTurnCount: 3,+ createdAt: now - 2_000,+ lastUpdatedAt: now - 100,+ }));+ await writeEndedMarker(stateDir, "resumed", now);++ await withMockOpenViking(profileHandler(requests), async (baseUrl) => {+ await runSessionStart(+ { session_id: "resumed", source: "resume", cwd: "/tmp/codex-resume-ended" },+ baseEnv(baseUrl, stateDir),+ );+ });++ assert.equal(await endedMarkerExists(stateDir, "resumed"), false);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++function readRequestBody(req) {+ return new Promise((resolve, reject) => {+ const chunks = [];+ req.on("data", (chunk) => chunks.push(chunk));+ req.on("end", () => {+ const raw = Buffer.concat(chunks).toString("utf-8");+ try { resolve(raw ? JSON.parse(raw) : null); } catch (error) { reject(error); }+ });+ req.on("error", reject);+ });+}++function turn(role, content) {+ return JSON.stringify({ payload: { message: { role, content } } });+}++/** profileHandler plus the session message endpoints the sweep catch-up needs. */+function sweepHandler(requests, { onBatch, batchStatus = 200 } = {}) {+ const base = profileHandler(requests);+ return async (req, res) => {+ const url = new URL(req.url, "http://127.0.0.1");+ if (req.method === "POST" && url.pathname.endsWith("/messages/batch")) {+ const body = await readRequestBody(req);+ requests.push({ method: req.method, path: url.pathname, body });+ await onBatch?.(body);+ if (batchStatus !== 200) {+ res.writeHead(batchStatus, { "Content-Type": "application/json" });+ res.end(JSON.stringify({ status: "error", error: { message: "batch failed" } }));+ return;+ }+ writeJson(res, { status: "ok", result: { ok: true } });+ return;+ }+ return base(req, res);+ };+}++test("the sweep catches up the recorded transcript before committing", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-codex-sweep-catchup-"));+ const transcriptPath = join(stateDir, "orphan.jsonl");+ const requests = [];+ try {+ const now = Date.now();+ await writeFile(transcriptPath, [+ turn("user", "first request"),+ turn("assistant", "first reply"),+ turn("user", "second request"),+ ].join("\n"));+ await writeFile(join(stateDir, "orphan.json"), JSON.stringify({+ codexSessionId: "orphan",+ ovSessionId: "cx-orphan",+ transcriptPath,+ capturedTurnCount: 1,+ createdAt: now - 5_000,+ lastUpdatedAt: now,+ }));+ await writeEndedMarker(stateDir, "orphan", now);++ await withMockOpenViking(sweepHandler(requests), async (baseUrl) => {+ const { output } = await runSessionStart(+ { session_id: "new-session", source: "startup", cwd: "/tmp/codex-sweep-catchup" },+ { ...baseEnv(baseUrl, stateDir), OPENVIKING_CAPTURE_ASSISTANT_TURNS: "1" },+ );+ assert.match(output.systemMessage, /cx-orphan is committed/);+ });++ const sent = requests+ .filter((r) => r.path === "/api/v1/sessions/cx-orphan/messages/batch")+ .flatMap((r) => r.body?.messages ?? []);+ assert.equal(sent.length, 2);+ assert.equal(sent[0].parts?.[0]?.text ?? sent[0].content, "first reply");+ assert.ok(requests.some((r) => r.path === "/api/v1/sessions/cx-orphan/commit"));++ const state = JSON.parse(await readFile(join(stateDir, "orphan.json"), "utf-8"));+ assert.equal(state.ovSessionId, null);+ assert.equal(state.capturedTurnCount, 3);+ assert.equal(await endedMarkerExists(stateDir, "orphan"), false);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("the sweep catches up an ended session that PreCompact already released", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-codex-sweep-released-"));+ const transcriptPath = join(stateDir, "released.jsonl");+ const requests = [];+ try {+ const now = Date.now();+ await writeFile(transcriptPath, [+ turn("user", "first request"),+ turn("user", "second request"),+ turn("user", "third request"),+ turn("user", "fourth request"),+ ].join("\n"));+ await writeFile(join(stateDir, "released.json"), JSON.stringify({+ codexSessionId: "released",+ ovSessionId: null,+ transcriptPath,+ capturedTurnCount: 2,+ createdAt: now - 5_000,+ lastUpdatedAt: now,+ }));+ await writeEndedMarker(stateDir, "released", now);++ await withMockOpenViking(sweepHandler(requests), async (baseUrl) => {+ const { output } = await runSessionStart(+ { session_id: "new-session", source: "startup", cwd: "/tmp/codex-sweep-released" },+ baseEnv(baseUrl, stateDir),+ );+ assert.match(output.systemMessage, /cx-released is committed/);+ });++ const sent = requests+ .filter((r) => r.path === "/api/v1/sessions/cx-released/messages/batch")+ .flatMap((r) => r.body?.messages ?? []);+ assert.equal(sent.length, 2, "the tail turns left by PreCompact must still be sent");+ assert.equal(sent[0].parts?.[0]?.text ?? sent[0].content, "third request");+ assert.ok(requests.some((r) => r.path === "/api/v1/sessions/cx-released/commit"));++ const state = JSON.parse(await readFile(join(stateDir, "released.json"), "utf-8"));+ assert.equal(state.ovSessionId, null);+ assert.equal(state.capturedTurnCount, 4);+ assert.equal(await endedMarkerExists(stateDir, "released"), false);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("the sweep never commits a session whose transcript cannot be read", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-codex-sweep-unreadable-"));+ const requests = [];+ try {+ const now = Date.now();+ await writeFile(join(stateDir, "unreadable.json"), JSON.stringify({+ codexSessionId: "unreadable",+ ovSessionId: "cx-unreadable",+ transcriptPath: join(stateDir, "gone.jsonl"),+ capturedTurnCount: 3,+ createdAt: now - 5_000,+ lastUpdatedAt: now,+ }));+ await writeEndedMarker(stateDir, "unreadable", now);++ await withMockOpenViking(sweepHandler(requests), async (baseUrl) => {+ const { output } = await runSessionStart(+ { session_id: "new-session", source: "startup", cwd: "/tmp/codex-sweep-unreadable" },+ baseEnv(baseUrl, stateDir),+ );+ assert.equal(output.systemMessage, undefined);+ });++ assert.equal(requests.some((r) => r.path.endsWith("/commit")), false);+ const state = JSON.parse(await readFile(join(stateDir, "unreadable.json"), "utf-8"));+ assert.equal(state.ovSessionId, "cx-unreadable");+ assert.equal(state.capturedTurnCount, 3);+ assert.equal(await endedMarkerExists(stateDir, "unreadable"), true);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("the sweep keeps a session live when its catch-up is incomplete", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-codex-sweep-partial-"));+ const transcriptPath = join(stateDir, "partial.jsonl");+ const requests = [];+ try {+ const now = Date.now();+ await writeFile(transcriptPath, [+ turn("user", "first request"),+ turn("user", "second request"),+ ].join("\n"));+ await writeFile(join(stateDir, "partial.json"), JSON.stringify({+ codexSessionId: "partial",+ ovSessionId: "cx-partial",+ transcriptPath,+ capturedTurnCount: 0,+ createdAt: now - 5_000,+ lastUpdatedAt: now,+ }));+ await writeEndedMarker(stateDir, "partial", now);++ await withMockOpenViking(sweepHandler(requests, { batchStatus: 500 }), async (baseUrl) => {+ const { output } = await runSessionStart(+ { session_id: "new-session", source: "startup", cwd: "/tmp/codex-sweep-partial" },+ baseEnv(baseUrl, stateDir),+ );+ assert.equal(output.systemMessage, undefined);+ });++ assert.equal(requests.some((r) => r.path.endsWith("/commit")), false);+ const state = JSON.parse(await readFile(join(stateDir, "partial.json"), "utf-8"));+ assert.equal(state.ovSessionId, "cx-partial");+ assert.equal(state.capturedTurnCount, 0);+ assert.equal(await endedMarkerExists(stateDir, "partial"), true);+ } finally {+ await rm(stateDir, { recursive: true, force: true });+ }+});++test("a marker that disappears under the lock falls back to the idle rule", async () => {+ const stateDir = await mkdtemp(join(tmpdir(), "ov-codex-sweep-marker-gone-"));+ const transcriptPath = join(stateDir, "twins.jsonl");+ const requests = [];+ try {+ const now = Date.now();+ await writeFile(transcriptPath, turn("user", "only request"));+ for (const id of ["twin-a", "twin-b"]) {+ await writeFile(join(stateDir, `${id}.json`), JSON.stringify({+ codexSessionId: id,+ ovSessionId: `cx-${id}`,+ transcriptPath,+ capturedTurnCount: 0,+ createdAt: now - 5_000,+ lastUpdatedAt: now,+ }));+ await writeEndedMarker(stateDir, id, now);+ }++ // Whichever twin the sweep reaches first wipes both markers mid-catch-up;+ // the other one is fresh, so with its marker gone it must be left alone.+ const onBatch = async () => {+ await Promise.all(+ ["twin-a", "twin-b"].map(async (id) => {+ for (const ts of await endedMarkerStamps(stateDir, id)) {+ await rm(join(stateDir, `${id}.ended.${ts}`), { force: true });+ }+ }),+ );+ };++ await withMockOpenViking(sweepHandler(requests, { onBatch }), async (baseUrl) => {+ await runSessionStart(+ { session_id: "new-session", source: "startup", cwd: "/tmp/codex-sweep-marker-gone" },+ { ...baseEnv(baseUrl, stateDir), OPENVIKING_CODEX_IDLE_TTL_MS: "1800000" },+ );+ });++ const commits = requests.filter((r) => r.path.endsWith("/commit"));+ assert.equal(commits.length, 1, "only the twin that still had its marker commits");+ const live = await Promise.all(["twin-a", "twin-b"].map(async (id) =>+ JSON.parse(await readFile(join(stateDir, `${id}.json`), "utf-8")).ovSessionId+ ));+ assert.equal(live.filter(Boolean).length, 1, "the other twin stays live for a later sweep");} finally {await rm(stateDir, { recursive: true, force: true });}diff --git a/examples/codex-memory-plugin/scripts/session-state.mjs b/examples/codex-memory-plugin/scripts/session-state.mjsindex def36b832..b93dac435 100644--- a/examples/codex-memory-plugin/scripts/session-state.mjs+++ b/examples/codex-memory-plugin/scripts/session-state.mjs@@ -4,13 +4,25 @@* One state file per codex session_id, holding the long-lived OpenViking* session id that we incrementally append turns to via the Stop hook. The* OV session id is derived as `cx-<codex-session-id>` for new captures.- * The OV session is committed (which extracts memories) by the PreCompact- * hook or by the idle-sweep that runs at the tail of each Stop.+ * The OV session is committed (which extracts memories) by SessionEnd, by+ * PreCompact, or by the fallback sweep at SessionStart.+ *+ * Two sidecars live next to `<safeId>.json`:+ * - `<safeId>.ended.<ts>` — written by the SessionEnd parent hook, lock-free,+ * so the sweep can still commit the session if the worker never ran. A+ * whole-object saveState from a concurrent worker cannot clobber it. The+ * timestamp is in the name so a conditional removal always targets an+ * immutable path and can never delete a marker written by a later exit.+ * - `<safeId>.lock` — an exclusive mkdir lock serializing the writers+ * (Stop worker, PreCompact, SessionEnd worker, SessionStart sweep) that+ * all persist the whole state object. A stale lock is taken over in place,+ * by claiming the `owner` file inside it.** State directory: $OPENVIKING_CODEX_STATE_DIR or ~/.openviking/codex-plugin-state*/-import { mkdir, readFile, readdir, rename, rm, writeFile } from "node:fs/promises";+import { randomUUID } from "node:crypto";+import { mkdir, readFile, readdir, rename, rm, rmdir, stat, utimes, writeFile } from "node:fs/promises";import { homedir } from "node:os";import { join } from "node:path";import { deriveCodexSessionId } from "./shared/session-model.mjs";@@ -42,12 +54,71 @@ function statePath(codexSessionId) {return join(getStateDir(), `${safeId(codexSessionId)}.json`);}+function endedPath(codexSessionId, ts) {+ return join(getStateDir(), `${safeId(codexSessionId)}.ended.${ts}`);+}++// Pre-0.8.1 marker: a bare `<safeId>.ended` whose content is the timestamp.+function legacyEndedPath(codexSessionId) {+ return join(getStateDir(), `${safeId(codexSessionId)}.ended`);+}++/**+ * Marker files for one session, newest first, from an optional pre-read dir+ * listing (`listStates()` already has one).+ */+function endedMarkersFrom(files, codexSessionId) {+ const prefix = `${safeId(codexSessionId)}.ended.`;+ const out = [];+ for (const file of files) {+ if (!file.startsWith(prefix)) continue;+ const ts = Number(file.slice(prefix.length));+ if (Number.isFinite(ts) && ts > 0) out.push({ file, ts });+ }+ return out.sort((a, b) => b.ts - a.ts);+}++async function endedMarkers(codexSessionId) {+ try {+ return endedMarkersFrom(await readdir(getStateDir()), codexSessionId);+ } catch {+ return [];+ }+}++/** Newest marker timestamp for one session, from a pre-read dir listing. */+async function endedAtFrom(files, codexSessionId) {+ const stamps = endedMarkersFrom(files, codexSessionId).map((m) => m.ts);+ if (files.includes(`${safeId(codexSessionId)}.ended`)) {+ const legacy = await legacyEndedAt(codexSessionId);+ if (legacy !== undefined) stamps.push(legacy);+ }+ return stamps.length ? Math.max(...stamps) : undefined;+}++async function legacyEndedAt(codexSessionId) {+ try {+ const raw = await readFile(legacyEndedPath(codexSessionId), "utf-8");+ const ts = Number(raw.trim());+ return Number.isFinite(ts) && ts > 0 ? ts : Date.now();+ } catch {+ return undefined;+ }+}++function lockPath(codexSessionId) {+ return join(getStateDir(), `${safeId(codexSessionId)}.lock`);+}+function defaultState(codexSessionId) {const now = Date.now();return {codexSessionId,ovSessionId: null,workspacePeerId: "",+ // Last rollout path seen by a capture hook, so the SessionStart sweep can+ // catch up turns for a session whose own workers never ran.+ transcriptPath: null,capturedTurnCount: 0,createdAt: now,lastUpdatedAt: now,@@ -67,8 +138,8 @@ export async function loadState(codexSessionId) {/*** Persist state. `touch: false` keeps the existing `lastUpdatedAt` so a write* that isn't transcript activity (e.g. releasing `ovSessionId` after a commit)- * doesn't make a dead session look freshly used to the active-window heuristic- * and the retention sweep.+ * doesn't make a dead session look freshly used to the idle-TTL sweep or to+ * the doctor's orphan count.*/export async function saveState(state, { touch = true } = {}) {if (!state || !state.codexSessionId) return;@@ -91,6 +162,175 @@ export async function clearState(codexSessionId) {try {await rm(statePath(codexSessionId), { force: true });} catch { /* best effort */ }+ await clearEnded(codexSessionId);+}++/**+ * Record that the codex thread ended. Written lock-free by the SessionEnd+ * parent. The returned timestamp is the marker's identity: workers carry it as+ * a token and only act on a marker that still matches it.+ *+ * `Date.now()` is only the starting point. Two exits within the same+ * millisecond would otherwise share a marker path, and the first one's+ * conditional removal would take the second one's marker with it, so the+ * timestamp is bumped until an exclusive create succeeds. Bumping rather than+ * randomizing keeps the names ordered, which is what `before` compares.+ */+export async function markEnded(codexSessionId) {+ if (!codexSessionId) return 0;+ let ts = Date.now();+ try {+ await mkdir(getStateDir(), { recursive: true });+ while (true) {+ try {+ await writeFile(endedPath(codexSessionId, ts), String(ts), { flag: "wx" });+ break;+ } catch (err) {+ if (err?.code !== "EEXIST") throw err;+ ts += 1;+ }+ }+ } catch { /* best effort */ }+ return ts;+}++/**+ * Drop the end marker: the thread is alive again, or its commit succeeded.+ *+ * `before` makes the removal conditional: a marker at or after that timestamp+ * belongs to a later exit than the caller and is left in place, so a late Stop+ * worker cannot erase a fresh marker.+ */+export async function clearEnded(codexSessionId, { before } = {}) {+ if (!codexSessionId) return;+ const cutoff = typeof before === "number" ? before : Infinity;+ const dir = getStateDir();+ for (const marker of await endedMarkers(codexSessionId)) {+ if (marker.ts >= cutoff) continue;+ // Each marker path carries its own timestamp, so removing one can never+ // take out a marker a later exit wrote.+ await rm(join(dir, marker.file), { force: true }).catch(() => {});+ }+ const legacy = await legacyEndedAt(codexSessionId);+ if (legacy !== undefined && legacy < cutoff) {+ // Pre-0.8.1 leftover: still read-then-remove, races included.+ await rm(legacyEndedPath(codexSessionId), { force: true }).catch(() => {});+ }+}++export async function readEndedAt(codexSessionId) {+ try {+ return await endedAtFrom(await readdir(getStateDir()), codexSessionId);+ } catch {+ return undefined;+ }+}++const LOCK_POLL_MS = 100;++/**+ * Take a stale lock over by claiming its `owner` file.+ *+ * Both steps are atomic and exactly one racer can complete them: the rename+ * removes the dead holder's stamp, and the exclusive create installs ours.+ * A racer that loses either step leaves the winner's lock intact.+ */+async function claimStaleLock(ownerFile, token) {+ const aside = `${ownerFile}.taken-${randomUUID()}`;+ try {+ await rename(ownerFile, aside);+ } catch {+ // No stamp to displace: the holder died before writing one, or a racer+ // already displaced it. The exclusive create alone decides the winner.+ try {+ await writeFile(ownerFile, token, { flag: "wx" });+ return true;+ } catch {+ return false;+ }+ }+ try {+ await writeFile(ownerFile, token, { flag: "wx" });+ } catch {+ await rm(aside, { force: true }).catch(() => {});+ return false;+ }+ await rm(aside, { force: true }).catch(() => {});+ return true;+}++/**+ * Run `fn` while holding an exclusive per-session lock.+ *+ * The lock is a directory (mkdir is atomic everywhere we run). A lock whose+ * mtime is older than `staleMs` is abandoned, so a killed holder cannot wedge+ * a session forever; a live holder keeps it fresh through `heartbeat()`.+ * `waitMs: 0` makes this a try-lock that returns `{ skipped: true }` instead+ * of waiting. Callers must always load state *inside* `fn`.+ *+ * The holder stamps an `owner` file inside the directory and only ever+ * releases (or refreshes) a lock whose owner is still its own, so a stale+ * takeover cannot make one taker drop the lock another taker now holds.+ * Takeover never removes or moves the directory — the lock path must never be+ * momentarily absent, or a racer's `mkdir` would succeed alongside the taker.+ * Instead the takers race for the `owner` file itself; see `claimStaleLock`.+ */+export async function withSessionLock(codexSessionId, fn, { waitMs = 0, staleMs = 300_000 } = {}) {+ const dir = lockPath(codexSessionId);+ const ownerFile = join(dir, "owner");+ const token = `${process.pid}:${randomUUID()}`;+ await mkdir(getStateDir(), { recursive: true });+ const deadline = Date.now() + Math.max(0, waitMs);+ let held = false;+ while (true) {+ try {+ await mkdir(dir);+ await writeFile(ownerFile, token);+ held = true;+ break;+ } catch (err) {+ if (err?.code !== "EEXIST") throw err;+ let ageMs = 0;+ try {+ ageMs = Date.now() - (await stat(dir)).mtimeMs;+ } catch {+ continue; // holder released between mkdir and stat; retry immediately+ }+ if (ageMs > staleMs) {+ if (await claimStaleLock(ownerFile, token)) {+ // Stop the other waiters from reading this lock as stale too.+ const now = new Date();+ await utimes(dir, now, now).catch(() => {});+ held = true;+ break;+ }+ continue; // another taker claimed it; re-read the lock from the top+ }+ if (Date.now() >= deadline) break;+ await new Promise((resolve) => setTimeout(resolve, LOCK_POLL_MS));+ }+ }+ if (!held) return { skipped: true };+ const owned = async () => {+ try {+ return (await readFile(ownerFile, "utf-8")) === token;+ } catch {+ return false;+ }+ };+ const heartbeat = async () => {+ if (!(await owned())) return;+ const now = new Date();+ await utimes(dir, now, now).catch(() => {});+ };+ try {+ return { skipped: false, value: await fn({ heartbeat }) };+ } finally {+ if (await owned()) {+ await rm(ownerFile, { force: true }).catch(() => {});+ await rmdir(dir).catch(() => {});+ }+ }}export async function listStates() {@@ -105,7 +345,11 @@ export async function listStates() {try {const raw = await readFile(join(dir, file), "utf-8");const parsed = JSON.parse(raw);- if (parsed?.codexSessionId) out.push(parsed);+ if (!parsed?.codexSessionId) continue;+ // Resolve the marker from the listing we already have instead of+ // re-reading the directory once per state file.+ const endedAt = await endedAtFrom(files, parsed.codexSessionId);+ out.push(endedAt ? { ...parsed, endedAt } : parsed);} catch { /* skip */ }}return out;diff --git a/examples/codex-memory-plugin/scripts/session-state.test.mjs b/examples/codex-memory-plugin/scripts/session-state.test.mjsnew file mode 100644index 000000000..519182079--- /dev/null+++ b/examples/codex-memory-plugin/scripts/session-state.test.mjs@@ -0,0 +1,107 @@+import assert from "node:assert/strict";+import { mkdir, mkdtemp, readFile, rm, stat, utimes, writeFile } from "node:fs/promises";+import { tmpdir } from "node:os";+import { join } from "node:path";+import test from "node:test";++const STATE_DIR = await mkdtemp(join(tmpdir(), "ov-session-state-"));+process.env.OPENVIKING_CODEX_STATE_DIR = STATE_DIR;++const { clearEnded, markEnded, readEndedAt, withSessionLock } = await import("./session-state.mjs");++async function exists(path) {+ try { await stat(path); return true; } catch { return false; }+}++test("releasing a lock taken over by someone else leaves it alone", async () => {+ const dir = join(STATE_DIR, "takeover.lock");+ const outcome = await withSessionLock("takeover", async () => {+ // Simulate another taker that renamed our lock aside and re-created it.+ await rm(dir, { recursive: true, force: true });+ await mkdir(dir);+ await writeFile(join(dir, "owner"), "9999:other-owner");+ return "done";+ }, { waitMs: 0 });++ assert.equal(outcome.skipped, false);+ assert.equal(outcome.value, "done");+ assert.equal(await exists(dir), true, "the new holder's lock must survive our release");+ assert.equal(await readFile(join(dir, "owner"), "utf-8"), "9999:other-owner");+ await rm(dir, { recursive: true, force: true });+});++test("concurrent stale takeovers leave exactly one holder", async () => {+ const dir = join(STATE_DIR, "stale.lock");+ await mkdir(dir, { recursive: true });+ const old = new Date(Date.now() - 600_000);+ await utimes(dir, old, old);++ let inside = 0;+ let maxInside = 0;+ const run = () => withSessionLock("stale", async () => {+ inside += 1;+ maxInside = Math.max(maxInside, inside);+ await new Promise((resolve) => setTimeout(resolve, 120));+ inside -= 1;+ return true;+ }, { waitMs: 5_000, staleMs: 300_000 });++ const outcomes = await Promise.all([run(), run(), run()]);+ assert.equal(maxInside, 1, "a stale takeover must never hand the lock to two takers");+ assert.equal(outcomes.filter((o) => o.value === true).length, 3);+ assert.equal(await exists(dir), false, "the last holder releases the lock");+});++test("clearEnded honours the `before` cutoff", async () => {+ const at = await markEnded("cutoff");+ assert.ok(at > 0);++ await clearEnded("cutoff", { before: at });+ assert.equal(await readEndedAt("cutoff"), at, "a marker at the cutoff is kept");++ await clearEnded("cutoff", { before: at - 1 });+ assert.equal(await readEndedAt("cutoff"), at, "a marker newer than the cutoff is kept");++ await clearEnded("cutoff", { before: at + 1 });+ assert.equal(await readEndedAt("cutoff"), undefined, "an older marker is removed");++ await markEnded("cutoff");+ await clearEnded("cutoff");+ assert.equal(await readEndedAt("cutoff"), undefined, "no cutoff clears unconditionally");+});++test("a clearEnded racing a newer markEnded leaves the newer marker", async () => {+ for (let i = 0; i < 500; i += 1) {+ const id = `race-${i}`;+ const at = await markEnded(id);+ // No artificial gap: the second exit lands in the same millisecond, while+ // the old caller's removal of everything up to `at` is in flight.+ const [, newer] = await Promise.all([+ clearEnded(id, { before: at + 1 }),+ markEnded(id),+ ]);+ const left = await readEndedAt(id);+ assert.equal(left, newer, "the newer marker must survive the concurrent clear");+ await clearEnded(id);+ }+});++test("markEnded / readEndedAt still honour a pre-0.8.1 bare marker", async () => {+ const at = Date.now() - 5_000;+ await writeFile(join(STATE_DIR, "legacy.ended"), String(at));+ assert.equal(await readEndedAt("legacy"), at);++ const newer = await markEnded("legacy");+ assert.equal(await readEndedAt("legacy"), newer, "the newest marker wins");++ await clearEnded("legacy", { before: at + 1 });+ assert.equal(await readEndedAt("legacy"), newer, "only the legacy marker was old enough");+ assert.equal(await exists(join(STATE_DIR, "legacy.ended")), false);++ await clearEnded("legacy");+ assert.equal(await readEndedAt("legacy"), undefined);+});++test.after(async () => {+ await rm(STATE_DIR, { recursive: true, force: true });+});diff --git a/examples/codex-memory-plugin/skills/ov-memory-doctor/reference.md b/examples/codex-memory-plugin/skills/ov-memory-doctor/reference.mdindex e7f255e4a..0bb3eb795 100644--- a/examples/codex-memory-plugin/skills/ov-memory-doctor/reference.md+++ b/examples/codex-memory-plugin/skills/ov-memory-doctor/reference.md@@ -12,10 +12,11 @@ and `CODEX_CONFIG_FILE` relocate individual pieces.| `~/.openviking/ovcli.conf` | Client connection: `url`, `api_key`, `account`, `user`, optional `plugin.codex.*` tuning. Mode 0600. || `~/.openviking/ov.conf` | Server config. The plugin reads only `server.url/host/port`, `server.root_api_key` (last-resort key) and the legacy `codex` block. || `~/.openviking/ovcli.conf.<name>` | Saved CLI profiles (`ov config switch` copies one over `ovcli.conf`). `ovcli.conf.bak.<epoch>` are installer backups. |-| `~/.codex/config.toml` | `[features] plugin_hooks`, `[plugins."openviking-memory@openviking"] enabled`, `[marketplaces.openviking]` (source, ref), `[hooks.state."openviking-memory@openviking:hooks/hooks.json:<event>:0:0"] trusted_hash` for `session_start`, `user_prompt_submit`, `stop`, `pre_compact`. |+| `~/.codex/config.toml` | `[features] plugin_hooks`, `[plugins."openviking-memory@openviking"] enabled`, `[marketplaces.openviking]` (source, ref), `[hooks.state."openviking-memory@openviking:hooks/hooks.json:<event>:0:0"] trusted_hash` for `session_start`, `user_prompt_submit`, `stop`, `session_end`, `pre_compact`. || `~/.codex/plugins/cache/openviking/openviking-memory/<version>/` | The copy Codex runs hooks from. Keyed by `plugin.json` version. || `~/.codex/.tmp/marketplaces/openviking/` | Git clone of the marketplace (GitHub/TOS dist); `examples/codex-memory-plugin` inside it is what `codex plugin list` reports as the source path. |-| `~/.openviking/codex-plugin-state/<session_id>.json` | Per-session state: `ovSessionId` (`cx-<session_id>`, null once committed), `capturedTurnCount`, `lastUpdatedAt`. |+| `~/.openviking/codex-plugin-state/<session_id>.json` | Per-session state: `ovSessionId` (`cx-<session_id>`, null once committed), `transcriptPath` (last rollout seen, used by the SessionStart sweep to catch up unsent turns), `capturedTurnCount`, `lastUpdatedAt`. |+| `~/.openviking/codex-plugin-state/<session_id>.ended.<timestamp>` / `.lock` | Sidecars: `.ended.<timestamp>` marks a thread whose SessionEnd fired but whose commit has not succeeded yet (the timestamp is in the filename so a conditional removal cannot delete a newer exit's marker; a bare `.ended` is a pre-0.8.1 leftover); `.lock` is the directory lock serializing the capture hooks. || `~/.openviking/codex-plugin-state/recall-compressor-profile.json` | Cached local-compressor detection (`profile.enabled`, `model`, `source`). || `~/.openviking/logs/codex-hooks.log` | JSONL hook + proxy log; written only when `OPENVIKING_DEBUG=1` or `codex.debug: true`. || `~/.openviking/codex-plugin.rc.sh`, `~/.openviking/codex-memory-plugin/runtime/` | Residue of the pre-marketplace installer. The rc script, if still sourced, exports `OPENVIKING_*` and pins credentials to env mode. |@@ -42,7 +43,8 @@ Codex plugin. Disable features with `OPENVIKING_AUTO_RECALL=0`,`codex plugin remove openviking-memory@openviking` / `enabled = false`.Hook budgets in `hooks/hooks.json`: SessionStart 70s, UserPromptSubmit 130s,-Stop 30s, PreCompact 60s. `recallTimeoutMs` (default 120000) must stay below+Stop 30s, SessionEnd 3s (Codex clamps it there; the hook detaches a worker),+PreCompact 60s. `recallTimeoutMs` (default 120000) must stay below130s and `captureTimeoutMs` (default 30000) at or below 30s.Sent headers: `Authorization: Bearer <key>`, `X-OpenViking-Account/User` (trusted@@ -94,7 +96,7 @@ Plugin MCP proxy (what Codex shows for a failing tool call):| `-32003` | `OpenViking MCP upstream returned an empty response` | 2xx with blank/non-JSON body — captive portal or proxy interstitial |Hook log (`codex-hooks.log`) hooks: `session-start`, `auto-recall`,-`auto-capture`, `pre-compact`, `mcp-proxy`. Stages worth grepping:+`auto-capture`, `session-end`, `pre-compact`, `mcp-proxy`. Stages worth grepping:`health_check`, `appended`, `pending_tokens`, `commit`, `recall_context_assembled`,`mcp-proxy` `start` (resolved `mcpUrl`, `hasApiKey` = present, not valid),`uncaught`. Ordinary failed proxy requests are not logged — the JSON-RPC error