Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.
Python agent-contextagent-memoryagent-pluginsagentic-ragcontext-databasedsh-pluginknowledge-basememorymemory-managementself-evolvingvector-database
feat(dsh-plugin): commit pending messages on compaction/start (#5254)
Almaz400b70eeee1e
diff --git a/docs/en/agent-integrations/16-capability-reference.md b/docs/en/agent-integrations/16-capability-reference.mdindex 71ea45f3a..0fb5d0c9f 100644--- a/docs/en/agent-integrations/16-capability-reference.md+++ b/docs/en/agent-integrations/16-capability-reference.md@@ -54,7 +54,7 @@ Long-term memories are extracted from a session only after a **commit**. Capture| ZCode | Yes, session-aware | Profile, memory index, skills | Every turn | No exit event; turns are already committed | No pre-compaction event || Kimi Code | Yes, session-aware | Profile, memory index, skills, at the first prompt | Every 8 captured messages | Only when `SessionEnd` captures new messages | Commits newly captured messages || OpenCode | Yes, session-aware | Profile, memory index, skills, indexed repositories | At 20,000 pending tokens, when idle | Yes, within the host's cleanup time | Commits around compaction |-| DSH | Yes, session-aware | Profile, memory index, skills, once per session | At 20,000 pending tokens | Yes, with a 3-second budget | Not observed |+| DSH | Yes, session-aware | Profile, memory index, skills, once per session | At 20,000 pending tokens | Yes, with a 3-second budget | Commits, then the host summarizes || pi | Yes, session-aware | Profile, memory index, skills, every turn | Takeover on: about 30,000 tokens; off: 20,000 | Takeover on: no; off: yes | Takeover replaces pi's summary || OpenClaw | Yes, session-aware | Profile, every turn | At half the token budget (64,000 by default) | No | The plugin owns compaction || Hermes (bundled) | Yes, session-aware with a fallback | Profile and memory listings | No; only at session boundaries | Yes, if pending uploads finish within 10 seconds | Commits at fork-style compaction |@@ -319,7 +319,7 @@ Thresholds in this table are client-side. They read the server's pending-token c| ZCode | Every `Stop`; turns a missed `Stop` skipped are caught up from the rollout file at the next `Stop` | None | No pre-compaction event || Kimi Code | `Stop` after 8 captured messages since the last commit | `SessionEnd` and `Interrupt` commit when they capture new messages | `PreCompact` commits when it captures new messages || OpenCode | v1 `session.idle`, v2 end of execution: at 20,000 pending tokens | Session deletion, v1 `session.error`, v1 dispose, and v2 cleanup force a commit | v1 before and after compaction; v2 once after compaction ends |-| DSH | `turn/end` at 20,000 pending tokens | Cordis teardown commits each session | None |+| DSH | `turn/end` at 20,000 pending tokens | Cordis teardown commits each session | `compaction/start` commits without the threshold check || pi, takeover on (default) | Locally estimated 30,000 tokens with more than 3 user turns | `/viking commit` | `session_before_compact` || pi, takeover off | 20,000 pending tokens after each sync | `session_shutdown` and `/viking commit` | `session_before_compact` || OpenClaw | After a turn, at `tokenBudget × commitTokenThresholdRatio` (128,000 × 0.5 by default) | `/new`, `/reset`, and `memory_store` commit and wait | `compact()` commits and waits up to 5 minutes for extraction |@@ -396,7 +396,7 @@ When the host shortens its context, most integrations make sure the dropped mess| TRAE, TRAE CN, ZCode | Host summarizes | No pre-compaction event | Nothing || Kimi Code | Host summarizes | `PreCompact` commits newly captured messages | Nothing || OpenCode | Host summarizes | v1 flushes and commits; v2 captures the transcript | v1 commits again on `session.compacted`; v2 commits after `session.compaction.ended` |-| DSH | Not observed | Nothing. Injected context is a user message and shrinks with the host's compaction; the profile is not sent again | Nothing |+| DSH | Host summarizes | `compaction/start` commits regardless of the token threshold. Injected context is a user message and shrinks with the host's compaction; the profile is not sent again | Nothing || pi | Takeover, on by default | Commits and waits for the archive overview | The overview replaces pi's summary; on failure pi compacts as usual || OpenClaw | The plugin owns compaction | `compact()` commits and waits | The next context assembly rebuilds history from the server || Hermes (bundled) | Host summarizes | Fork-style compaction commits the old session; in-place compaction does nothing | Nothing |diff --git a/docs/en/agent-integrations/17-dsh.md b/docs/en/agent-integrations/17-dsh.mdindex 66257b85a..e6217cc32 100644--- a/docs/en/agent-integrations/17-dsh.md+++ b/docs/en/agent-integrations/17-dsh.md@@ -52,7 +52,7 @@ For MCP proxy diagnostics, set `OPENVIKING_DEBUG=1` and `OPENVIKING_DEBUG_LOG=/t## How it works-The bundle runs inside DSH as a Cordis plugin rather than as external hooks, so it follows the session in-process. At session start it injects your OpenViking profile block, an index of available memories, and an `<available-skills>` catalog of your OpenViking skills. When a model step takes new user input, it searches OpenViking with the text the user typed and appends what it finds to that step as a durable message, so the injection replays with the session and is visible to compaction. Context that DSH or other plugins inject (for example `time-context` or job notices) and tool results neither trigger recall nor enter the query. It captures user, assistant, and (optionally) tool-result messages straight from DSH's event stream, skipping injected context, and commits to OpenViking once pending tokens cross the threshold, archiving every captured message. Writes that fail land in a pending queue and replay at the next session start.+The bundle runs inside DSH as a Cordis plugin rather than as external hooks, so it follows the session in-process. At session start it injects your OpenViking profile block, an index of available memories, and an `<available-skills>` catalog of your OpenViking skills. When a model step takes new user input, it searches OpenViking with the text the user typed and appends what it finds to that step as a durable message, so the injection replays with the session and is visible to compaction. Context that DSH or other plugins inject (for example `time-context` or job notices) and tool results neither trigger recall nor enter the query. It captures user, assistant, and (optionally) tool-result messages straight from DSH's event stream, skipping injected context, and commits to OpenViking once pending tokens cross the threshold or when DSH begins a compaction, archiving every captured message. Writes that fail land in a pending queue and replay at the next session start.Each DSH session maps to `dsh-<session-id>` in OpenViking, and every subagent gets its own session.@@ -110,7 +110,7 @@ Credentials given in the patch win over the environment. Behavior knobs resolve| Recall is empty | `curl "<your OpenViking URL>/health"`; check the endpoint and that the prompt is longer than the minimum query length (3 characters) || 401 / 403 from OpenViking | Verify `OPENVIKING_API_KEY`; for trusted-mode deployments also verify `OPENVIKING_ACCOUNT` and `OPENVIKING_USER` || Memories from other projects leak in | Set `OPENVIKING_RECALL_PEER_SCOPE=actor` to limit peer memories to the active peer; user-level memories remain shared |-| Nothing committed after a crash | Commit runs on a token threshold and at teardown; queued writes replay at the next session start |+| Nothing committed after a crash | Commit runs on a token threshold, at compaction start, and at teardown; queued writes replay at the next session start |## See alsodiff --git a/docs/zh/agent-integrations/16-capability-reference.md b/docs/zh/agent-integrations/16-capability-reference.mdindex facd95368..f3610674f 100644--- a/docs/zh/agent-integrations/16-capability-reference.md+++ b/docs/zh/agent-integrations/16-capability-reference.md@@ -54,7 +54,7 @@| ZCode | 是,带会话 | profile、记忆索引、skill | 每轮 | 没有退出事件;每轮已提交 | 没有压缩前事件 || Kimi Code | 是,带会话 | profile、记忆索引、skill,在首次提问时 | 每捕获 8 条消息 | 仅当 `SessionEnd` 捕获到新消息 | 提交新捕获的消息 || OpenCode | 是,带会话 | profile、记忆索引、skill、已索引的仓库 | 空闲时,待提交 token 达到 20,000 | 是,在宿主的清理时限内 | 在压缩前后提交 |-| DSH | 是,带会话 | profile、记忆索引、skill,每个会话一次 | 待提交 token 达到 20,000 | 是,预算 3 秒 | 未观察到 |+| DSH | 是,带会话 | profile、记忆索引、skill,每个会话一次 | 待提交 token 达到 20,000 | 是,预算 3 秒 | 先提交,再由宿主摘要 || pi | 是,带会话 | profile、记忆索引、skill,每轮 | takeover 开:约 30,000 token;关:20,000 | takeover 开:否;关:是 | takeover 替换 pi 的摘要 || OpenClaw | 是,带会话 | profile,每轮 | token 预算的一半(默认 64,000) | 否 | 插件接管压缩 || Hermes(内置) | 是,带会话,有回退路径 | profile 与记忆清单 | 否;只在会话边界 | 是,前提是待上传内容在 10 秒内完成 | fork 型压缩时提交 |@@ -318,7 +318,7 @@ Claude Code 和 Codex 默认 `auto`,其他集成默认 `off`。支持服务端| ZCode | 每个 `Stop`;漏掉的 `Stop` 对应的轮次,在下一个 `Stop` 从 rollout 文件补齐 | 无 | 没有压缩前事件 || Kimi Code | 距上次提交捕获满 8 条消息时,在 `Stop` 提交 | `SessionEnd` 和 `Interrupt` 捕获到新消息时提交 | `PreCompact` 捕获到新消息时提交 || OpenCode | v1 `session.idle`、v2 执行结束时:待提交 token 达到 20,000 | 删除会话、v1 `session.error`、v1 dispose 和 v2 cleanup 强制提交 | v1 在压缩前后各一次;v2 在压缩结束后一次 |-| DSH | `turn/end` 时待提交 token 达到 20,000 | Cordis teardown 提交每个会话 | 无 |+| DSH | `turn/end` 时待提交 token 达到 20,000 | Cordis teardown 提交每个会话 | `compaction/start` 提交,跳过阈值检查 || pi,takeover 开(默认) | 本地估算达到 30,000 token,且用户轮多于 3 个 | `/viking commit` | `session_before_compact` || pi,takeover 关 | 每次同步后待提交 token 达到 20,000 | `session_shutdown` 和 `/viking commit` | `session_before_compact` || OpenClaw | 每轮结束后达到 `tokenBudget × commitTokenThresholdRatio`(默认 128,000 × 0.5) | `/new`、`/reset` 和 `memory_store` 提交并等待 | `compact()` 提交,并最多等待 5 分钟完成抽取 |@@ -395,7 +395,7 @@ Claude Code 和 Codex 默认 `auto`,其他集成默认 `off`。支持服务端| TRAE、TRAE CN、ZCode | 宿主摘要 | 没有压缩前事件 | 无 || Kimi Code | 宿主摘要 | `PreCompact` 提交新捕获的消息 | 无 || OpenCode | 宿主摘要 | v1 刷新并提交;v2 捕获 transcript | v1 在 `session.compacted` 时再提交一次;v2 在 `session.compaction.ended` 后提交 |-| DSH | 未观察到 | 无。注入的上下文是一条用户消息,随宿主压缩一起缩减;profile 不再发送 | 无 |+| DSH | 宿主摘要 | `compaction/start` 无论 token 阈值如何都会提交。注入的上下文是一条用户消息,随宿主压缩一起缩减;profile 不再发送 | 无 || pi | takeover,默认开启 | 提交并等待归档概览 | 用概览替换 pi 的摘要;失败时 pi 照常压缩 || OpenClaw | 插件接管压缩 | `compact()` 提交并等待 | 下一次上下文组装从服务端重建历史 || Hermes(内置) | 宿主摘要 | fork 型压缩提交旧会话;原地压缩不做任何处理 | 无 |diff --git a/docs/zh/agent-integrations/17-dsh.md b/docs/zh/agent-integrations/17-dsh.mdindex 96c8b6977..04e0e093c 100644--- a/docs/zh/agent-integrations/17-dsh.md+++ b/docs/zh/agent-integrations/17-dsh.md@@ -52,7 +52,7 @@ curl -fsSL https://openviking.ai/install | bash## 工作方式-插件以 Cordis 插件的形式跑在 DSH 进程内,而不是外挂 hook,因此能贴着会话走。会话开始时注入 OpenViking 画像块、可用记忆索引和 OpenViking 技能清单 `<available-skills>`;模型步骤带有新的用户输入时,用用户输入的文本做语义检索,把结果作为持久消息追加到同一步骤,因此注入会随会话重放,也对压缩可见。DSH 或其他插件注入的上下文(例如 `time-context`、job 通知)和工具结果既不触发召回,也不拼进检索文本。它直接从 DSH 的事件流捕获 user、assistant 以及(可选的)工具结果消息,并跳过注入的上下文,待同步 token 超过阈值即 commit,每次 commit 都归档全部已捕获的消息。写入失败会进入待写队列,在下次会话开始时重放。+插件以 Cordis 插件的形式跑在 DSH 进程内,而不是外挂 hook,因此能贴着会话走。会话开始时注入 OpenViking 画像块、可用记忆索引和 OpenViking 技能清单 `<available-skills>`;模型步骤带有新的用户输入时,用用户输入的文本做语义检索,把结果作为持久消息追加到同一步骤,因此注入会随会话重放,也对压缩可见。DSH 或其他插件注入的上下文(例如 `time-context`、job 通知)和工具结果既不触发召回,也不拼进检索文本。它直接从 DSH 的事件流捕获 user、assistant 以及(可选的)工具结果消息,并跳过注入的上下文,待同步 token 超过阈值或 DSH 开始压缩时即 commit,每次 commit 都归档全部已捕获的消息。写入失败会进入待写队列,在下次会话开始时重放。每个 DSH 会话映射为 OpenViking 中的 `dsh-<session-id>`,子 agent 各自拥有独立会话。@@ -110,7 +110,7 @@ patch 中写的凭证优先于环境变量。行为配置按优先级从高到| 召不回任何内容 | `curl "<OpenViking 服务地址>/health"`;检查端点配置,以及 prompt 是否长于最小查询长度(3 个字符) || OpenViking 返回 401 / 403 | 检查 `OPENVIKING_API_KEY`;可信模式部署还要检查 `OPENVIKING_ACCOUNT` 与 `OPENVIKING_USER` || 串入了其他项目的记忆 | 设置 `OPENVIKING_RECALL_PEER_SCOPE=actor`,将 peer 记忆限定为当前 peer;用户级记忆仍会共享 |-| 崩溃后没有 commit | commit 由 token 阈值和 teardown 触发;排队的写入会在下次会话开始时重放 |+| 崩溃后没有 commit | commit 由 token 阈值、压缩开始和 teardown 触发;排队的写入会在下次会话开始时重放 |## 延伸阅读diff --git a/examples/dsh-memory-plugin/runtime.mjs b/examples/dsh-memory-plugin/runtime.mjsindex 4137f8d1f..636e4b783 100644--- a/examples/dsh-memory-plugin/runtime.mjs+++ b/examples/dsh-memory-plugin/runtime.mjs@@ -179,22 +179,27 @@ export class OpenVikingRuntime {}maybeCommit(session, event) {- if (event.type !== "turn/end") return;+ // DSH appends compaction/start before it summarizes and rewrites history,+ // so commit there regardless of the threshold, like PreCompact elsewhere.+ const boundary = event.type === "compaction/start";+ if (event.type !== "turn/end" && !boundary) return;const state = this.stateFor(session);if (!isCaptureEnabled(state.config)) return;this.enqueueWrite(state, async () => {if (state.hasPendingWrites) return;if (!state.ready && !(await this.ensureState(state)).ready) return;- const metadata = await this.client.getSession(- state.ovSessionId,- state.config.peerId,- );- if (Number(metadata?.pending_tokens || 0) < state.config.commitTokenThreshold) return;+ if (!boundary) {+ const metadata = await this.client.getSession(+ state.ovSessionId,+ state.config.peerId,+ );+ if (Number(metadata?.pending_tokens || 0) < state.config.commitTokenThreshold) return;+ }const response = await this.client.commitSession(state.ovSessionId,state.config.peerId,);- this.log("commit", {+ this.log(boundary ? "compaction_commit" : "commit", {sessionId: state.ovSessionId,ok: response.ok,trace_id: response.result?.trace_id || response.traceId,diff --git a/examples/dsh-memory-plugin/runtime.test.mjs b/examples/dsh-memory-plugin/runtime.test.mjsindex 9efceb5b0..7bdaaa987 100644--- a/examples/dsh-memory-plugin/runtime.test.mjs+++ b/examples/dsh-memory-plugin/runtime.test.mjs@@ -127,6 +127,38 @@ test("a retryable threshold commit failure is queued", async () => {]);});+test("compaction/start commits below the token threshold without reading session metadata", async () => {+ const pendingDir = await mkdtemp(join(tmpdir(), "dsh-memory-compaction-"));+ tempDirs.push(pendingDir);+ process.env.OPENVIKING_PENDING_DIR = pendingDir;+ let getSessionCalls = 0;+ let commitCalls = 0;+ const runtime = new OpenVikingRuntime({+ async getSession() {+ getSessionCalls += 1;+ return { pending_tokens: 5 };+ },+ async commitSession() {+ commitCalls += 1;+ return { ok: true };+ },+ }, config(), { debug() {} });+ const session = { id: "compaction-commit", header: { cwd: "/workspace" } };+ runtime.stateFor(session).ready = true;++ runtime.maybeCommit(session, { type: "compaction/start" });+ await runtime.flush(session);++ assert.equal(getSessionCalls, 0);+ assert.equal(commitCalls, 1);++ runtime.maybeCommit(session, { type: "compaction/end" });+ await runtime.flush(session);++ assert.equal(getSessionCalls, 0);+ assert.equal(commitCalls, 1);+});+test("once a write is queued, later messages and the final commit stay ordered on disk", async () => {const pendingDir = await mkdtemp(join(tmpdir(), "dsh-memory-order-"));tempDirs.push(pendingDir);