Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.
Python agent-contextagent-memoryagent-pluginsagentic-ragcontext-databasedsh-pluginknowledge-basememorymemory-managementself-evolvingvector-database
fix(openclaw): preserve capture across host turn protocols (#4900)
Zayn Jarvis6b9d54900f93
diff --git a/examples/openclaw-plugin/README.md b/examples/openclaw-plugin/README.mdindex f84feee85..6128587fa 100644--- a/examples/openclaw-plugin/README.md+++ b/examples/openclaw-plugin/README.md@@ -23,7 +23,7 @@ The agent runs install → setup → restart → verify automatically. See [INST| Stage | What happens ||-------|-------------|-| **Every turn** (`afterTurn`) | New messages are appended to an OpenViking session; commit/extraction is threshold-triggered |+| **Every turn** (`afterTurn` or `commitTurn`, depending on host/runner) | New messages are appended to an OpenViking session; commit/extraction is threshold-triggered || **Explicit remember** (`memory_store`) | Important long-term facts can be written and committed immediately || **On `/compact`** (`compact`) | Pending session messages are committed and extracted into long-term memories || **Before each reply** (`assemble`) | Relevant memories are auto-retrieved and injected into context |@@ -202,6 +202,17 @@ That means OpenClaw sees "compressed history summary + archive index + active me`afterTurn()` has a narrower job: append only the new turn into the OpenViking session.+Capture ownership depends on the **host** version from `api.runtime.version`:++- Before OpenClaw 2026.9.3, `afterTurn` captures messages; `commitTurn` only acknowledges the turn to avoid writing those messages twice.+- Starting with 2026.9.3, admitted/deferred turns are captured inside `commitTurn` before acknowledgment. Standalone runners still capture via `afterTurn`.+- The minimum supported OpenClaw version remains 2026.5.27. Missing/unparseable versions and ambiguous 2026.9.3 prereleases reject `commitTurn` rather than silently discarding queued messages; `afterTurn` remains available. Numeric packaging revisions such as `2026.9.3-1` use the 9.3 behavior.++Durable capture propagates client/write/commit failures to the host for retry and shares one in-flight write between concurrent deliveries of the same advancement key. Disabled capture, heartbeats, bypassed sessions, and empty turns are intentional skips. The legacy `afterTurn` callback retains best-effort error handling.++The host's durable delivery does not supply the model token budget or sender runtime context: this path uses the 128,000-token fallback and cannot attach the runtime sender ID to user messages. Message writes remain non-transactional: replay after partial writes or a process crash can duplicate messages. The in-memory advancement-key cache does not provide server-side exactly-once delivery.++- it slices only the newly added messages- it keeps only `user` / `assistant` capture text- it preserves `toolCall` / `toolResult` content in the serialized turn text@@ -214,7 +225,7 @@ After that, the plugin checks `pending_tokens`. Once it reaches `commitTokenThre- the current turn is not blocked waiting for extraction- if `logFindRequests` is enabled, the logs include the task id and follow-up extraction detail-This automatic path is best-effort and commit-dependent. Short but important facts can stay only in the live session until a threshold commit, `/compact`, or an explicit store happens.+Memory extraction on this automatic path is commit-dependent. Short but important facts can stay only in the live session until a threshold commit, `/compact`, or an explicit store happens.### Explicit long-term memory writesdiff --git a/examples/openclaw-plugin/context-engine.ts b/examples/openclaw-plugin/context-engine.tsindex 5bcd71503..b421bf252 100644--- a/examples/openclaw-plugin/context-engine.ts+++ b/examples/openclaw-plugin/context-engine.ts@@ -245,10 +245,34 @@ function validTokenBudget(raw: unknown): number | undefined {return undefined;}+// OpenClaw 9.1/9.2 still capture via afterTurn even when commitTurn exists.+// 9.3 (#140024) defers admitted turns to commitTurn; standalone runners keep+// afterTurn. The registration API's runtime.version is the HOST version.+function usesDeferredTurnCapture(version: string | undefined): boolean | undefined {+ const match = version?.trim().match(+ /^(\d+)\.(\d+)\.(\d+)(?:-([0-9A-Za-z.-]+))?(?:\+[0-9A-Za-z.-]+)?$/,+ );+ if (!match) return undefined;+ const parts = match.slice(1, 4).map(Number);+ if (!parts.every(Number.isSafeInteger)) return undefined;+ const [year, month, day] = parts;+ // OpenClaw uses 0.0.0 when version metadata cannot be resolved. More generally,+ // versions outside our supported range cannot prove who owns capture.+ if (year < 2026 || (year === 2026 && (month < 5 || (month === 5 && day < 27)))) {+ return undefined;+ }+ const boundary = year === 2026 && month === 9 && day === 3;+ // The 9.3 prerelease train can straddle the host change. Numeric packaging+ // revisions (-1, -2, ...) refer to the stable release; do not guess for betas.+ if (boundary && match[4] && !/^\d+$/.test(match[4])) return undefined;+ return year > 2026 || (year === 2026 && (month > 9 || (month === 9 && day >= 3)));+}+export function createMemoryOpenVikingContextEngine(params: {id: string;name: string;version?: string;+ hostVersion?: string;cfg: ParsedMemoryOpenVikingConfig;logger: Logger;getClient: () => Promise<OpenVikingClient>;@@ -333,6 +357,8 @@ export function createMemoryOpenVikingContextEngine(params: {}const committedTurnKeys = new Set<string>();+ const inFlightTurns = new Map<string, Promise<void>>();+ const deferredTurnCapture = usesDeferredTurnCapture(params.hostVersion);return {info: {@@ -388,20 +414,56 @@ export function createMemoryOpenVikingContextEngine(params: {});},- // Capture still happens in afterTurn (host calls it per LLM call + on finalize);- // commitTurn only acknowledges the accepted turn so OpenClaw drains its outbox.- // ponytail: in-memory key set, not durable across restarts — the host outbox is.- async commitTurn({ advancementKey, sessionId }): Promise<{ status: "committed" | "duplicate" }> {+ async commitTurn(commitParams): Promise<{ status: "committed" | "duplicate" }> {+ const { advancementKey, sessionId } = commitParams;+ if (deferredTurnCapture === undefined) {+ // ACKing with an unknown capture owner can discard the only copy in the+ // host outbox. Reject instead; legacy afterTurn remains available.+ throw new Error(`openviking: cannot select turn capture for OpenClaw version ${params.hostVersion ?? "(missing)"}`);+ }if (committedTurnKeys.has(advancementKey)) {diag("commitTurn_duplicate", sessionId, { advancementKey });return { status: "duplicate" };}- committedTurnKeys.add(advancementKey);- if (committedTurnKeys.size > 1024) {- committedTurnKeys.delete(committedTurnKeys.values().next().value as string);+ const pending = inFlightTurns.get(advancementKey);+ if (pending) {+ await pending;+ return { status: "duplicate" };+ }+ // Publish the promise before capture begins so concurrent deliveries share+ // both success and failure. Completion keys are intentionally process-local.+ const capture = Promise.resolve().then(async () => {+ if (deferredTurnCapture) {+ await afterTurnOpenVikingSession({+ sessionId,+ sessionKey: resolveSessionKey(commitParams),+ messages: commitParams.messages,+ prePromptMessageCount: 0, // Host already supplies the closed-turn range.+ isHeartbeat: commitParams.isHeartbeat,+ tokenBudget: 128_000, // Durable delivery does not carry a token budget.+ throwOnError: true,+ cfg,+ getClient,+ logger,+ resolveAgentId,+ rememberSessionAgentId,+ isBypassedSession,+ diag,+ });+ }+ committedTurnKeys.add(advancementKey);+ if (committedTurnKeys.size > 1024) {+ committedTurnKeys.delete(committedTurnKeys.values().next().value as string);+ }+ diag("commitTurn", sessionId, { advancementKey, deferredTurnCapture });+ });+ inFlightTurns.set(advancementKey, capture);+ try {+ await capture;+ return { status: "committed" };+ } finally {+ inFlightTurns.delete(advancementKey);}- diag("commitTurn", sessionId, { advancementKey });- return { status: "committed" };},async afterTurn(afterTurnParams): Promise<void> {diff --git a/examples/openclaw-plugin/index.ts b/examples/openclaw-plugin/index.tsindex 822856854..2ba0c911b 100644--- a/examples/openclaw-plugin/index.ts+++ b/examples/openclaw-plugin/index.ts@@ -95,6 +95,7 @@ type ToolContext = {};type OpenClawPluginApi = {+ runtime?: { version?: string };pluginConfig?: unknown;openVikingTransport?: HttpTransport;logger: PluginLogger;diff --git a/examples/openclaw-plugin/plugin/openviking-context-engine-registration.ts b/examples/openclaw-plugin/plugin/openviking-context-engine-registration.tsindex b79ae28e5..db97949c3 100644--- a/examples/openclaw-plugin/plugin/openviking-context-engine-registration.ts+++ b/examples/openclaw-plugin/plugin/openviking-context-engine-registration.ts@@ -1,4 +1,5 @@export type OpenVikingContextEngineRegistrationApi = {+ runtime?: { version?: string };registerContextEngine?: (id: string, factory: () => unknown) => void;};@@ -22,6 +23,7 @@ export type OpenVikingContextEngineCreateParams<id: string;name: string;version: string;+ hostVersion?: string;cfg: TCfg;logger: TLogger;getClient: () => Promise<TClient>;@@ -80,6 +82,7 @@ export function registerOpenVikingContextEngine<id: deps.plugin.id,name: deps.plugin.name,version: deps.version,+ hostVersion: deps.api.runtime?.version,cfg: deps.cfg,logger: deps.logger,getClient: deps.getClient,diff --git a/examples/openclaw-plugin/services/context-lifecycle-service.ts b/examples/openclaw-plugin/services/context-lifecycle-service.tsindex 0dbc8f3c9..53826d6a5 100644--- a/examples/openclaw-plugin/services/context-lifecycle-service.ts+++ b/examples/openclaw-plugin/services/context-lifecycle-service.ts@@ -125,6 +125,8 @@ export type CompactOpenVikingSessionParams = {type AfterTurnClient = Pick<OpenVikingClient, "addSessionMessage" | "getSession" | "commitSession" | "getTask">;export type AfterTurnOpenVikingSessionParams = {+ /** Durable delivery must reject failed writes so the host retains its outbox row. */+ throwOnError?: boolean;sessionId: string;sessionKey?: string;messages?: AgentMessage[];@@ -786,6 +788,7 @@ function messageDigest(messages: AgentMessage[], maxCharsPerMsg = 2000): Array<{}export async function afterTurnOpenVikingSession({+ throwOnError = false,sessionId,sessionKey,messages: rawMessages,@@ -976,6 +979,9 @@ export async function afterTurnOpenVikingSession({senderIdFound: sender.found,senderId: sender.senderId ?? null,});+ if (throwOnError) {+ throw err;+ }}}diff --git a/examples/openclaw-plugin/tests/ut/context-engine-commit-turn.test.ts b/examples/openclaw-plugin/tests/ut/context-engine-commit-turn.test.tsindex 6370049d3..030c530eb 100644--- a/examples/openclaw-plugin/tests/ut/context-engine-commit-turn.test.ts+++ b/examples/openclaw-plugin/tests/ut/context-engine-commit-turn.test.ts@@ -4,34 +4,175 @@ import type { OpenVikingClient } from "../../client.js";import { memoryOpenVikingConfigSchema } from "../../config.js";import { createMemoryOpenVikingContextEngine } from "../../context-engine.js";-function makeEngine() {+// Mock the transport boundary, not the lifecycle service: swallowed client+// failures must make these tests fail, even when the callback itself resolves.+function makeEngine(hostVersion: string | undefined, overrides = {}) {const logger = { info: vi.fn(), warn: vi.fn(), error: vi.fn() };- return createMemoryOpenVikingContextEngine({+ const client = {+ addSessionMessage: vi.fn().mockResolvedValue(undefined),+ getSession: vi.fn().mockResolvedValue({ pending_tokens: 100_000 }),+ commitSession: vi.fn().mockResolvedValue({ status: "accepted" }),+ };+ const getClient = vi.fn().mockResolvedValue(client);+ const engine = createMemoryOpenVikingContextEngine({id: "openviking",name: "Context Engine (OpenViking)",version: "test",- cfg: memoryOpenVikingConfigSchema.parse({ mode: "remote", baseUrl: "http://127.0.0.1:1933" }),+ hostVersion,+ cfg: memoryOpenVikingConfigSchema.parse({+ mode: "remote",+ baseUrl: "http://127.0.0.1:1933",+ autoCapture: true,+ ...overrides,+ }),logger,- getClient: vi.fn().mockResolvedValue({} as OpenVikingClient),- resolveAgentId: vi.fn(() => "agent"),+ getClient: getClient as () => Promise<OpenVikingClient>,+ resolveAgentId: () => "agent",});+ return { engine, client, getClient, logger };}-// OpenClaw >=2026.8.1 degrades the engine to "legacy" every turn unless both-// transcript semantics are declared and commitTurn exists.-describe("context-engine durable turn contract (OpenClaw 2026.8.1)", () => {- it("declares transcript semantics", () => {- expect(makeEngine().info.transcriptSemantics).toEqual({+const messages = [+ { role: "user", content: "Remember that my preferred editor is Vim." },+ { role: "assistant", content: "I will remember your editor preference." },+];+const turn = { advancementKey: "k1", sessionId: "session-1", messages };+const afterTurn = { ...turn, sessionFile: "unused", prePromptMessageCount: 0 };++function deferred() {+ let resolve!: () => void;+ let reject!: (error: Error) => void;+ const promise = new Promise<void>((yes, no) => { resolve = yes; reject = no; });+ return { promise, resolve, reject };+}++describe("context-engine capture ownership", () => {+ it("keeps the transcript declarations required by 8.1+ hosts", () => {+ expect(makeEngine("2026.9.3").engine.info.transcriptSemantics).toEqual({currentTurnFence: "before-current-turn-entry-v1",turnAdvancementIdempotency: "atomic-idempotent-v1",});});- it("commitTurn is idempotent per advancementKey", async () => {- const engine = makeEngine();- const params = { advancementKey: "k1", sessionId: "s", messages: [] };+ it.each(["2026.5.27"])("keeps afterTurn-only hosts working (%s)", async (version) => {+ const { engine, client } = makeEngine(version);+ await engine.afterTurn!(afterTurn);+ expect(client.addSessionMessage).toHaveBeenCalledTimes(2);+ expect(client.commitSession).toHaveBeenCalledTimes(1);+ });++ it.each(["2026.9.2"])(+ "does not recapture afterTurn messages in commitTurn on %s", async (version) => {+ const { engine, client } = makeEngine(version);+ // Old tool-loop checkpoint, then finalize checkpoint, then durable ACK.+ await engine.afterTurn!({ ...afterTurn, messages: messages.slice(0, 1) });+ await engine.afterTurn!({ ...afterTurn, prePromptMessageCount: 1 });+ await expect(engine.commitTurn(turn)).resolves.toEqual({ status: "committed" });+ expect(client.addSessionMessage).toHaveBeenCalledTimes(2);+ expect(client.getSession).toHaveBeenCalledTimes(2);+ await expect(engine.commitTurn(turn)).resolves.toEqual({ status: "duplicate" });+ expect(client.addSessionMessage).toHaveBeenCalledTimes(2);+ },+ );++ it.each(["2026.9.3", "2026.9.3-1"])(+ "captures closed turns before ACK on %s", async (version) => {+ const { engine, client } = makeEngine(version);+ await expect(engine.commitTurn(turn)).resolves.toEqual({ status: "committed" });+ expect(client.addSessionMessage).toHaveBeenCalledTimes(2);+ expect(client.addSessionMessage.mock.calls.map((call) => call[1])).toEqual(["user", "assistant"]);+ expect(client.commitSession).toHaveBeenCalledTimes(1);+ await expect(engine.commitTurn(turn)).resolves.toEqual({ status: "duplicate" });+ expect(client.addSessionMessage).toHaveBeenCalledTimes(2);+ },+ );++ it("keeps standalone afterTurn capture on 9.3 alongside durable sessions", async () => {+ const { engine, client } = makeEngine("2026.9.3");+ await engine.afterTurn!({ ...afterTurn, sessionId: "standalone" });+ await engine.commitTurn(turn);+ expect(client.addSessionMessage).toHaveBeenCalledTimes(4);+ expect(client.commitSession).toHaveBeenCalledTimes(2);+ });++ it.each([undefined, "0.0.0", "2026.9.3-beta.1"])("does not ACK an unknown capture owner (%s)", async (version) => {+ const { engine, client } = makeEngine(version);+ await expect(engine.commitTurn(turn)).rejects.toThrow("cannot select turn capture");+ await expect(engine.commitTurn(turn)).rejects.toThrow("cannot select turn capture");+ expect(client.addSessionMessage).not.toHaveBeenCalled();+ await engine.afterTurn!(afterTurn);+ expect(client.addSessionMessage).toHaveBeenCalledTimes(2);+ });+});++describe("durable capture failures and skips", () => {+ it.each(["addSessionMessage", "commitSession"] as const)(+ "rejects %s failure and permits retry with the same key", async (stage) => {+ const { engine, client } = makeEngine("2026.9.3");+ const operation = client[stage];+ operation.mockRejectedValueOnce(new Error(`${stage} unavailable`));+ await expect(engine.commitTurn(turn)).rejects.toThrow(`${stage} unavailable`);+ await expect(engine.commitTurn(turn)).resolves.toEqual({ status: "committed" });+ await expect(engine.commitTurn(turn)).resolves.toEqual({ status: "duplicate" });+ expect(client.commitSession).toHaveBeenCalled();+ },+ );++ it("preserves best-effort errors in the legacy callback", async () => {+ const { engine, client, logger } = makeEngine("2026.9.1");+ client.addSessionMessage.mockRejectedValueOnce(new Error("offline"));+ await expect(engine.afterTurn!(afterTurn)).resolves.toBeUndefined();+ expect(logger.warn).toHaveBeenCalledWith(expect.stringContaining("offline"));+ });++ it.each([+ { config: { autoCapture: false }, params: turn },+ { config: {}, params: { ...turn, isHeartbeat: true } },+ { config: { bypassSessionPatterns: ["session-1"] }, params: turn },+ ])("ACKs intentional capture skips", async ({ config, params }) => {+ const { engine, getClient } = makeEngine("2026.9.3", config);await expect(engine.commitTurn(params)).resolves.toEqual({ status: "committed" });- await expect(engine.commitTurn(params)).resolves.toEqual({ status: "duplicate" });- await expect(engine.commitTurn({ ...params, advancementKey: "k2" })).resolves.toEqual({ status: "committed" });+ expect(getClient).not.toHaveBeenCalled();+ });+});++describe("concurrent advancement-key delivery", () => {+ it("waits for one capture before either caller ACKs", async () => {+ const { engine, client } = makeEngine("2026.9.3");+ const gate = deferred();+ const entered = deferred();+ client.addSessionMessage.mockImplementationOnce(() => { entered.resolve(); return gate.promise; });+ const first = engine.commitTurn(turn);+ await entered.promise;+ const second = engine.commitTurn(turn);+ let secondSettled = false;+ void second.then(() => { secondSettled = true; });+ await Promise.resolve();+ expect(secondSettled).toBe(false);+ expect(client.addSessionMessage).toHaveBeenCalledTimes(1);+ gate.resolve();+ await expect(Promise.all([first, second])).resolves.toEqual([+ { status: "committed" }, { status: "duplicate" },+ ]);+ expect(client.addSessionMessage).toHaveBeenCalledTimes(2);+ expect(client.commitSession).toHaveBeenCalledTimes(1);+ });++ it("rejects every waiter on failure and permits a later retry", async () => {+ const { engine, client } = makeEngine("2026.9.3");+ const gate = deferred();+ const entered = deferred();+ client.addSessionMessage.mockImplementationOnce(() => { entered.resolve(); return gate.promise; });+ const first = engine.commitTurn(turn);+ await entered.promise;+ const second = engine.commitTurn(turn);+ const results = Promise.allSettled([first, second]);+ const error = new Error("write failed");+ gate.reject(error);+ expect(await results).toEqual([+ { status: "rejected", reason: error }, { status: "rejected", reason: error },+ ]);+ await expect(engine.commitTurn(turn)).resolves.toEqual({ status: "committed" });+ expect(client.addSessionMessage).toHaveBeenCalledTimes(3);});});diff --git a/examples/openclaw-plugin/tests/ut/plugin-modules.test.ts b/examples/openclaw-plugin/tests/ut/plugin-modules.test.tsindex 7576bf847..eafc49f0e 100644--- a/examples/openclaw-plugin/tests/ut/plugin-modules.test.ts+++ b/examples/openclaw-plugin/tests/ut/plugin-modules.test.ts@@ -468,7 +468,7 @@ describe("plugin module seams", () => {it("registers the context engine through a dedicated plugin module", () => {const engine = { id: "openviking", commitOVSession: vi.fn() };- const api = { registerContextEngine: vi.fn() };+ const api = { runtime: { version: "2026.9.3" }, registerContextEngine: vi.fn() };const logger = { info: vi.fn(), warn: vi.fn() };const getClient = vi.fn();const resolveAgentId = vi.fn();@@ -502,6 +502,7 @@ describe("plugin module seams", () => {id: "openviking",name: "OpenViking",version: "0.1.0",+ hostVersion: "2026.9.3",cfg: { baseUrl: "http://127.0.0.1:1933" },logger,getClient,