Skip to content

Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.

fix(plugins): honor cloud recall compression across harnesses (#5240)

Zayn Jarvise44ea6e11add
diff --git a/docs/en/agent-integrations/04-codex.md b/docs/en/agent-integrations/04-codex.md
index 073fc240f..9aa413964 100644
--- a/docs/en/agent-integrations/04-codex.md
+++ b/docs/en/agent-integrations/04-codex.md
@@ -115,3 +115,9 @@ Change it with `OPENVIKING_PEER_SOURCE`, with `plugin.peerSource` in `ovcli.conf
- [DESIGN.md](https://github.com/volcengine/OpenViking/blob/main/examples/codex-memory-plugin/DESIGN.md) — Commit decision tree.
- [MCP Clients](./06-mcp-clients.md) — MCP protocol, tools, and other clients.
- [Deployment Guide → CLI](../guides/03-deployment.md#cli) — `ovcli.conf` setup instructions.
+
+### Recall compression
+
+Set `OPENVIKING_RECALL_COMPRESS=server` to compress recalled context on the OpenViking server without launching a local Codex compressor. `client` uses local compression only; `auto` (the default) uses the server when the local compressor is unavailable; `off` disables compression. Existing server digests are used directly, and an explicit no-relevant result injects nothing.
+
+Codex calls the shared `buildRecallBlockDetailed()` pipeline for retrieval, ranking, budgets and old-server fallback. Only session mapping, model execution and hook output remain host-specific. Local compression failures preserve bounded retrieved context. Without local compression, raw fallback now honors `recallPreferAbstract` instead of always reading every leaf in full. Budgets include body text, URIs and wrapper text. See the [shared plugin configuration](https://github.com/volcengine/OpenViking/blob/main/examples/memory-plugin-shared/README.md#cloud-recall-compression).
diff --git a/docs/zh/agent-integrations/04-codex.md b/docs/zh/agent-integrations/04-codex.md
index e07077e28..8367e11a7 100644
--- a/docs/zh/agent-integrations/04-codex.md
+++ b/docs/zh/agent-integrations/04-codex.md
@@ -114,3 +114,9 @@ TraeCode CLI 2.0 用户启动 `trae-cli`,并可用 `trae-cli plugin list` 确
- [DESIGN.md](https://github.com/volcengine/OpenViking/blob/main/examples/codex-memory-plugin/DESIGN.md) — 提交(commit)决策树
- [MCP 客户端](./06-mcp-clients.md) — MCP 协议、工具列表及其他客户端
- [部署指南 → CLI](../guides/03-deployment.md#cli) — `ovcli.conf` 配置说明
+
+### 召回压缩
+
+设置 `OPENVIKING_RECALL_COMPRESS=server` 可让 OpenViking 服务端压缩召回内容,Codex 不启动本地压缩进程。`client` 仅用本地压缩,`auto`(默认)在本地压缩器不可用时走服务端,`off` 关闭压缩。服务端已有摘要时直接使用;明确返回无相关记忆时不注入。
+
+Codex 通过共享 `buildRecallBlockDetailed()` 执行召回、排序、预算和旧服务端回退,仅保留会话映射、模型调用与 hook 输出适配。本地压缩失败保留预算内的检索结果;原始检索回退在不使用本地压缩时遵循 `recallPreferAbstract`,不再固定读取所有叶子全文。预算包含正文、URI 和包装文本。配置详见 [共享插件说明](https://github.com/volcengine/OpenViking/blob/main/examples/memory-plugin-shared/README.md#cloud-recall-compression)。
diff --git a/examples/agent-hook-plugin/hosts/cursor/openviking.integration.json b/examples/agent-hook-plugin/hosts/cursor/openviking.integration.json
index d49be2c23..0937e5460 100644
--- a/examples/agent-hook-plugin/hosts/cursor/openviking.integration.json
+++ b/examples/agent-hook-plugin/hosts/cursor/openviking.integration.json
@@ -1,7 +1,7 @@
{
"schemaVersion": 1,
"id": "openviking-memory",
- "version": "0.3.2",
+ "version": "0.3.3",
"clients": [
"cursor"
],
diff --git a/examples/agent-hook-plugin/hosts/trae/openviking.integration.json b/examples/agent-hook-plugin/hosts/trae/openviking.integration.json
index d888f4728..73d45d6cc 100644
--- a/examples/agent-hook-plugin/hosts/trae/openviking.integration.json
+++ b/examples/agent-hook-plugin/hosts/trae/openviking.integration.json
@@ -1,7 +1,7 @@
{
"schemaVersion": 1,
"id": "openviking-memory",
- "version": "0.3.2",
+ "version": "0.3.3",
"clients": [
"trae",
"trae-cn"
diff --git a/examples/agent-hook-plugin/hosts/zcode/openviking.integration.json b/examples/agent-hook-plugin/hosts/zcode/openviking.integration.json
index d70465589..7bf872e09 100644
--- a/examples/agent-hook-plugin/hosts/zcode/openviking.integration.json
+++ b/examples/agent-hook-plugin/hosts/zcode/openviking.integration.json
@@ -1,7 +1,7 @@
{
"schemaVersion": 1,
"id": "openviking-memory",
- "version": "0.3.2",
+ "version": "0.3.3",
"clients": [
"zcode"
],
diff --git a/examples/agent-hook-plugin/plugin.json b/examples/agent-hook-plugin/plugin.json
index d03af30dd..d2e378e8f 100644
--- a/examples/agent-hook-plugin/plugin.json
+++ b/examples/agent-hook-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "agent-hook-plugin",
- "version": "0.3.2",
+ "version": "0.3.3",
"description": "Long-term semantic memory for the config-driven hook hosts — Cursor, TRAE, TRAE CN and ZCode — powered by OpenViking. Auto-recall relevant memories at session start and capture the conversation as it happens.",
"author": {
"name": "OpenViking",
diff --git a/examples/claude-code-memory-plugin/.claude-plugin/plugin.json b/examples/claude-code-memory-plugin/.claude-plugin/plugin.json
index 60bc1dec9..5e7f0e85e 100644
--- a/examples/claude-code-memory-plugin/.claude-plugin/plugin.json
+++ b/examples/claude-code-memory-plugin/.claude-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "openviking-memory",
- "version": "0.5.2",
+ "version": "0.5.3",
"description": "Long-term semantic memory for Claude Code, powered by OpenViking. Auto-recall relevant memories at session start and capture important information during conversations.",
"author": {
"name": "OpenViking",
diff --git a/examples/claude-code-memory-plugin/scripts/auto-recall.test.mjs b/examples/claude-code-memory-plugin/scripts/auto-recall.test.mjs
index 97b4faa3b..017321712 100644
--- a/examples/claude-code-memory-plugin/scripts/auto-recall.test.mjs
+++ b/examples/claude-code-memory-plugin/scripts/auto-recall.test.mjs
@@ -273,7 +273,8 @@ test("the client-ranked fallback degrades past the budget to URI hints", async (
assert.equal(state.count, 2);
assert.equal(state.content_items, 1);
assert.equal(state.hint_items, 1);
- assert.equal(state.tokens_used, Math.ceil(`- [memory 90%] ${first}`.length / 4));
+ assert.equal(state.tokens_used, Math.ceil(block.length / 4));
+ assert.ok(state.tokens_used <= state.tokens_budget);
assert.equal(state.tokens_budget, 200);
});
diff --git a/examples/claude-code-memory-plugin/scripts/config.mjs b/examples/claude-code-memory-plugin/scripts/config.mjs
index 74a99f730..42d869eb0 100644
--- a/examples/claude-code-memory-plugin/scripts/config.mjs
+++ b/examples/claude-code-memory-plugin/scripts/config.mjs
@@ -28,7 +28,7 @@ import { readFileSync } from "node:fs";
import { homedir } from "node:os";
import { join, resolve as resolvePath } from "node:path";
-import { buildPluginConfig, normalizeRewriteMode } from "./shared/plugin-config.mjs";
+import { buildPluginConfig } from "./shared/plugin-config.mjs";
const DEFAULT_OV_CONF_PATH = join(homedir(), ".openviking", "ov.conf");
const DEFAULT_OVCLI_CONF_PATH = join(homedir(), ".openviking", "ovcli.conf");
@@ -121,17 +121,5 @@ export function loadConfig(cwd = process.cwd(), { env = process.env } = {}) {
configPath: config.ovPath || config.cliPath || null,
credentialPath: config.credentialPath || null,
- // Digest compression defaults to auto: prefer the local host CLI and fall
- // back to the server when it is unavailable. A failed digest still falls
- // back to the uncompressed context block. `recallRewrite` keeps the
- // internal field name because the shared core maps this mode to the
- // server's `rewrite` request field; OPENVIKING_RECALL_REWRITE is the older
- // env spelling and still works.
- recallRewrite: normalizeRewriteMode(
- env.OPENVIKING_RECALL_COMPRESS
- ?? env.OPENVIKING_RECALL_REWRITE
- ?? config.recallCompress,
- "auto",
- ),
};
}
diff --git a/examples/claude-code-memory-plugin/scripts/shared/config-schema.mjs b/examples/claude-code-memory-plugin/scripts/shared/config-schema.mjs
index 4a1a7c9c4..55342f2c4 100644
--- a/examples/claude-code-memory-plugin/scripts/shared/config-schema.mjs
+++ b/examples/claude-code-memory-plugin/scripts/shared/config-schema.mjs
@@ -98,7 +98,7 @@ export const KNOBS = [
// off/client/server/auto through `normalizeRewriteMode`; Codex reads the same
// key as on/off. Both spellings stay declared so neither harness's users are
// told their config is a typo.
- { name: "recallCompress", type: "string", default: "auto", env: "OPENVIKING_RECALL_COMPRESS", aliases: ["recallRewrite"], capability: "recall" },
+ { name: "recallCompress", type: "string", default: "off", harness: { claude_code: "auto", codex: "auto" }, env: "OPENVIKING_RECALL_COMPRESS", aliases: ["recallRewrite"], capability: "recall" },
{ name: "recallCompressModel", type: "string", default: "", env: "OPENVIKING_RECALL_COMPRESS_MODEL", capability: "recall" },
{ name: "recallCompressBaseUrl", type: "string", default: "", env: "OPENVIKING_RECALL_COMPRESS_BASE_URL", capability: "recall" },
{ name: "recallCompressThinking", type: "string", default: "", env: "OPENVIKING_RECALL_COMPRESS_THINKING", aliases: ["recallCompressReasoningEffort"], capability: "recall" },
diff --git a/examples/claude-code-memory-plugin/scripts/shared/plugin-config.mjs b/examples/claude-code-memory-plugin/scripts/shared/plugin-config.mjs
index 214a287f9..f60c3bca8 100644
--- a/examples/claude-code-memory-plugin/scripts/shared/plugin-config.mjs
+++ b/examples/claude-code-memory-plugin/scripts/shared/plugin-config.mjs
@@ -29,7 +29,7 @@
import { homedir } from "node:os";
import { join } from "node:path";
-import { HARNESS_KEYS, KNOBS, harnessKey, resolveKnobs } from "./config-schema.mjs";
+import { HARNESS_KEYS, KNOBS, harnessKey, knobDefault, resolveKnobs } from "./config-schema.mjs";
import {
buildUserAgent,
loadCredentialFiles,
@@ -224,9 +224,15 @@ export function buildPluginConfig(harness, {
env,
});
+ const recallRewrite = normalizeRewriteMode(
+ env.OPENVIKING_RECALL_COMPRESS ?? env.OPENVIKING_RECALL_REWRITE ?? settings.recallCompress,
+ knobDefault(KNOBS.find((knob) => knob.name === "recallCompress"), harnessKey(name)),
+ );
const config = {
...settings,
harness: name,
+ recallCompress: recallRewrite,
+ recallRewrite,
clientVersion,
userAgent: buildUserAgent(name, clientVersion),
diff --git a/examples/claude-code-memory-plugin/scripts/shared/recall-core.mjs b/examples/claude-code-memory-plugin/scripts/shared/recall-core.mjs
index 4695a887b..b893078b0 100644
--- a/examples/claude-code-memory-plugin/scripts/shared/recall-core.mjs
+++ b/examples/claude-code-memory-plugin/scripts/shared/recall-core.mjs
@@ -309,21 +309,46 @@ async function resolveTargetUri(fetchJSON, targetUri, actorPeerId = "") {
return `viking://user/${space}/${parts.join("/")}`;
}
-async function searchOneSource(fetchJSON, query, source, limit, actorPeerId = "") {
- const resolvedUri = await resolveTargetUri(fetchJSON, source.uri, actorPeerId);
- const body = { query, target_uri: resolvedUri, limit, score_threshold: 0 };
- const res = await fetchJSON("/api/v1/search/find", {
- method: "POST",
- body: JSON.stringify(body),
- }, { actorPeerId });
- if (!res.ok) return [];
- const items = res.result?.[source.bucket] || [];
- return items.map((item) => ({ ...item, _sourceType: source.type }));
+async function searchOneSource(fetchJSON, cfg, query, source, limit, options) {
+ const actorPeerId = options.actorPeerId || "";
+ const home = await resolveTargetUri(fetchJSON, source.uri, actorPeerId);
+ const targets = [...new Set(cfg.user
+ ? [`viking://user/${cfg.user}/${source.bucket}`, home] : [home])];
+ const sessionId = String(options.sessionId || "").trim();
+ const search = async (target, session) => {
+ const body = { query, target_uri: target, limit, score_threshold: 0 };
+ if (session) body.session_id = session;
+ const init = { method: "POST", body: JSON.stringify(body) };
+ // Session-aware retrieval may decide no context is needed. Only after all
+ // targets are empty do we retry without the session. Old servers still
+ // support find; an unsupported search route can fall back to it.
+ let res = await fetchJSON(sessionId ? "/api/v1/search/search" : "/api/v1/search/find", init, { actorPeerId });
+ if (sessionId && !res.ok && (
+ !res.status || res.status === 404 || res.status === 405 || res.status >= 500
+ || ((res.status === 400 || res.status === 422) && looksLikeUnknownField(res))
+ )) {
+ const { session_id, ...findBody } = body;
+ res = await fetchJSON("/api/v1/search/find", { method: "POST", body: JSON.stringify(findBody) }, { actorPeerId });
+ }
+ const items = res.ok && Array.isArray(res.result?.[source.bucket]) ? res.result[source.bucket] : [];
+ return items.map((item) => ({ ...item, _sourceType: source.type }));
+ };
+ for (const target of targets) {
+ const items = await search(target, sessionId);
+ if (items.length) return items;
+ }
+ if (sessionId) {
+ for (const target of targets) {
+ const items = await search(target, "");
+ if (items.length) return items;
+ }
+ }
+ return [];
}
-async function searchAllSources(fetchJSON, query, perSourceLimit, actorPeerId = "", log = () => {}) {
+async function searchAllSources(fetchJSON, cfg, query, perSourceLimit, options, log = () => {}) {
const results = await Promise.all(
- SOURCES.map((src) => searchOneSource(fetchJSON, query, src, perSourceLimit, actorPeerId)),
+ SOURCES.map((src) => searchOneSource(fetchJSON, cfg, query, src, perSourceLimit, options)),
);
const all = results.flat();
log("recall_search_summary", {
@@ -359,50 +384,30 @@ async function resolveItemContent(fetchJSON, item, cfg, actorPeerId = "") {
return content;
}
-async function buildFallbackInjectionBlock(fetchJSON, items, cfg, actorPeerId = "", log = () => {}) {
+function formatFallback(items, cfg, log = () => {}) {
const budgetTotal = Math.max(200, Number(cfg.recallTokenBudget || 2000));
- let budgetRemaining = budgetTotal;
- const lines = [
- "<openviking-context>",
- "Relevant context from OpenViking. Use the read MCP tool to expand URIs.",
- ];
+ const lines = [];
let contentCount = 0;
let hintCount = 0;
-
+ const maxChars = Math.max(50, Number(cfg.recallMaxContentChars || 500));
for (const item of items) {
const score = (clampScore(item.score) * 100).toFixed(0);
- const uriLine = `- [${item._sourceType} ${score}%] ${item.uri}`;
-
- if (budgetRemaining > 0) {
- const content = await resolveItemContent(fetchJSON, item, cfg, actorPeerId);
- const contentLine = `- [${item._sourceType} ${score}%] ${content}`;
- const lineTokens = estimateTokens(contentLine);
-
- if (lineTokens > budgetRemaining && contentCount > 0) {
- lines.push(uriLine);
- hintCount++;
- } else {
- lines.push(contentLine);
- budgetRemaining -= lineTokens;
- contentCount++;
- }
- } else {
+ const kind = item._sourceType || item.category || "memory";
+ const uriLine = `- [${kind} ${score}%] ${item.uri}`;
+ const text = String(item.text || item.uri);
+ const content = text.length > maxChars ? `${text.slice(0, maxChars)}...` : text;
+ const contentLine = `- [${kind} ${score}%] ${content}`;
+ if (estimateTokens(wrapContext([...lines, uriLine, contentLine].join("\n"))) <= budgetTotal) {
+ lines.push(uriLine, contentLine);
+ contentCount += 1;
+ } else if (estimateTokens(wrapContext([...lines, uriLine].join("\n"))) <= budgetTotal) {
lines.push(uriLine);
- hintCount++;
+ hintCount += 1;
}
}
-
- lines.push("</openviking-context>");
-
- const budgetUsed = budgetTotal - budgetRemaining;
- log("recall_injection_built", {
- contentItems: contentCount,
- hintItems: hintCount,
- budgetUsed,
- budgetTotal,
- });
-
- return { block: lines.join("\n"), contentCount, hintCount, budgetUsed, stage: "ranked" };
+ const budgetUsed = lines.length ? estimateTokens(wrapContext(lines.join("\n"))) : 0;
+ log("recall_injection_built", { contentItems: contentCount, hintItems: hintCount, budgetUsed, budgetTotal });
+ return { rendered: lines.join("\n"), contentCount, hintCount, budgetUsed };
}
const LEGACY_CACHE_TTL_MS = 6 * 60 * 60 * 1000;
@@ -469,10 +474,14 @@ function looksLikeUnknownField(res) {
}
function wrapContext(body) {
+ // Retrieved text cannot introduce a second capture delimiter inside ours.
+ const text = String(body)
+ .replace(/<\/?relevant-memor(?:y|ies)\b[^>]*>/gi, "legacy memory wrapper")
+ .replace(/<\/?openviking-context\b[^>]*>/gi, "openviking context marker");
return [
"<openviking-context>",
"Relevant memory from OpenViking. Use the search/read MCP tools to expand URIs.",
- body,
+ text,
"</openviking-context>",
].join("\n");
}
@@ -488,7 +497,7 @@ export async function buildServerAssembledBlock(fetchJSON, cfg, query, options =
const block = await recallViaContextFace(fetchJSON, cfg, query, { ...options, actorPeerId }, log);
if (block !== null) return block;
- return recallViaEndpoint(fetchJSON, cfg, query, actorPeerId, log);
+ return recallViaEndpoint(fetchJSON, cfg, query, { ...options, actorPeerId }, log);
}
/**
@@ -552,53 +561,65 @@ export function normalizeContextEntry(entry = {}) {
};
}
-async function recallViaContextFace(fetchJSON, cfg, query, options, log) {
- const assembled = await fetchAssembledContext(fetchJSON, cfg, query, { ...options, log });
- if (assembled === null) return null;
+/** Server relevance decisions and digest precedence are shared by every host. */
+export function selectRecallContent(result = {}) {
+ if (String(result.stats?.rewrite || "").toLowerCase() === "no_relevant") return "";
+ return String(result.digest || "").trim() || String(result.rendered || "").trim();
+}
- const { rendered, entries } = assembled;
- let digest = assembled.digest;
+function wantsLocalCompression(cfg, options) {
const mode = String(cfg.recallRewrite || "off").toLowerCase();
- if (String(assembled.stats?.rewrite || "").toLowerCase() === "no_relevant") {
- log("recall_server_compression", { status: "empty" });
- return "";
- }
- const wantsLocal = mode === "client" || (mode === "auto" && !digest);
- if (wantsLocal && rendered && typeof options.runCompressor === "function") {
- try {
- const compression = await compressRecallContext({
- query,
- rendered,
- entries,
- cfg,
- runCompressor: options.runCompressor,
- cachePath: options.digestCachePath || stateFile("recall-digest.json"),
- now: Date.now(),
- });
- log("recall_local_compression", { status: compression.status });
- if (compression.status === "ok") digest = compression.context;
- if (compression.status === "empty") return "";
- } catch (err) {
- log("recall_local_compression_failed", { error: String(err?.message || err) });
- }
+ return (mode === "client" || mode === "auto")
+ && options.localCompressorAvailable !== false
+ && typeof options.runCompressor === "function";
+}
+
+// Context, legacy recall, and raw retrieval share the same result policy.
+// The host callback only runs its model; it never chooses a fallback or digest.
+async function finalizeRecall(result, cfg, query, options, log, shortContext) {
+ const selected = selectRecallContent(result);
+ if (String(result.stats?.rewrite || "").toLowerCase() === "no_relevant") return "";
+ const entries = (result.entries || []).map(normalizeContextEntry).filter((entry) => entry.uri);
+ const rendered = String(result.rendered || "").trim();
+ if (String(result.digest || "").trim() || !wantsLocalCompression(cfg, options)) return selected;
+ const fallback = shortContext ?? (entries.length ? formatFallback(entries, cfg).rendered : rendered);
+ const input = rendered || entries.map((entry) => `${entry.uri}\n${entry.text}`).join("\n");
+ if (!input) return "";
+ try {
+ const compression = await compressRecallContext({
+ query, rendered: input, shortContext: shortContext ?? (rendered || fallback),
+ entries, cfg, runCompressor: options.runCompressor,
+ cachePath: options.digestCachePath || stateFile("recall-digest.json"), now: Date.now(),
+ });
+ log("recall_local_compression", { status: compression.status });
+ if (compression.status === "ok") return compression.context;
+ if (compression.status === "empty") return "";
+ } catch (err) {
+ log("recall_local_compression_failed", { error: String(err?.message || err) });
}
+ return fallback;
+}
- const injected = digest || rendered;
- if (!injected) return "";
- return wrapContext(injected);
+async function recallViaContextFace(fetchJSON, cfg, query, options, log) {
+ const assembled = await fetchAssembledContext(fetchJSON, cfg, query, { ...options, log });
+ if (assembled === null) return null;
+ const text = await finalizeRecall(assembled, cfg, query, options, log);
+ return text ? wrapContext(text) : "";
}
-async function recallViaEndpoint(fetchJSON, cfg, query, actorPeerId = "", log = () => {}) {
+async function recallViaEndpoint(fetchJSON, cfg, query, options, log) {
const body = buildRecallEndpointBody(cfg);
body.query = query;
- const res = await postRecall(fetchJSON, body, { actorPeerId, log });
+ if (wantsLocalCompression(cfg, options)) {
+ body.max_chars = Math.max(1000, Number(cfg.recallCompressMaxInputChars || 18000));
+ }
+ const res = await postRecall(fetchJSON, body, { actorPeerId: options.actorPeerId, log });
if (!res.ok) {
log("recall_endpoint_fallback", { status: res.status || 0 });
return null;
}
- const rendered = String(res.result?.rendered || "").trim();
- if (!rendered) return "";
- return wrapContext(rendered);
+ const text = await finalizeRecall(res.result || {}, cfg, query, options, log);
+ return text ? wrapContext(text) : "";
}
export async function postRecall(fetchJSON, body, opts = {}) {
@@ -711,7 +732,7 @@ async function recallForPeer(fetchJSON, cfg, query, options = {}) {
const recallLimit = Math.max(1, Number(cfg.recallLimit || DEFAULT_CONTEXT_LIMIT));
const perSourceLimit = Math.max(recallLimit * 2, 8);
- const raw = await searchAllSources(fetchJSON, trimmed, perSourceLimit, actorPeerId, log);
+ const raw = await searchAllSources(fetchJSON, cfg, trimmed, perSourceLimit, { ...options, actorPeerId }, log);
if (raw.length === 0) return emptyRecall("no_results");
const profile = buildQueryProfile(trimmed);
@@ -727,5 +748,25 @@ async function recallForPeer(fetchJSON, cfg, query, options = {}) {
});
if (picked.length === 0) return emptyRecall("filtered_out");
- return buildFallbackInjectionBlock(fetchJSON, picked, cfg, actorPeerId, log);
+ const local = wantsLocalCompression(cfg, options);
+ const inputLimit = Math.max(1000, Number(cfg.recallCompressMaxInputChars || 18000));
+ const contentCfg = local ? { ...cfg, recallPreferAbstract: false, recallMaxContentChars: inputLimit } : cfg;
+ const entries = await Promise.all(picked.map(async (item) => ({
+ ...item, text: await resolveItemContent(fetchJSON, item, contentCfg, actorPeerId),
+ })));
+ const fallback = formatFallback(entries, cfg, log);
+ let text = await finalizeRecall({
+ rendered: local ? entries.map((item) => `${item.uri}\n${item.text}`).join("\n").slice(0, inputLimit) : fallback.rendered,
+ entries,
+ }, cfg, trimmed, options, log, fallback.rendered);
+ if (estimateTokens(wrapContext(text)) > Math.max(200, Number(cfg.recallTokenBudget || 2000))) {
+ text = fallback.rendered;
+ }
+ if (!text) return emptyRecall("no_results");
+ const unchanged = text === fallback.rendered;
+ return {
+ block: wrapContext(text), contentCount: unchanged ? fallback.contentCount : 1,
+ hintCount: unchanged ? fallback.hintCount : 0,
+ budgetUsed: estimateTokens(wrapContext(text)), stage: "ranked",
+ };
}
diff --git a/examples/codex-memory-plugin/.codex-plugin/plugin.json b/examples/codex-memory-plugin/.codex-plugin/plugin.json
index 827e484c8..098849d02 100644
--- a/examples/codex-memory-plugin/.codex-plugin/plugin.json
+++ b/examples/codex-memory-plugin/.codex-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "openviking-memory",
- "version": "0.9.2",
+ "version": "0.9.3",
"description": "Long-term semantic memory for Codex, powered by OpenViking. Recall on UserPromptSubmit, incremental add_message on Stop (per turn), commit on SessionEnd (Codex >= 0.145) and on PreCompact, with an idle/ended sweep on SessionStart as the fallback for signals, crashes and older Codex builds. Includes a local stdio MCP proxy that reads OpenViking credentials from env/ovcli.conf.",
"author": {
"name": "OpenViking",
diff --git a/examples/codex-memory-plugin/README.md b/examples/codex-memory-plugin/README.md
index 090a175d4..64077c35f 100644
--- a/examples/codex-memory-plugin/README.md
+++ b/examples/codex-memory-plugin/README.md
@@ -283,13 +283,30 @@ On `resume`, the script skips commit/sweep. It still injects the profile block.
### Auto-recall (every UserPromptSubmit)
-`auto-recall.mjs` reads `prompt` and `session_id` from stdin. It first asks `/api/v1/search/recall` for bounded, type-quota candidates and passes those entries through the same relevance compressor used by the fallback path. If that endpoint is unavailable, the hook derives the long-lived OpenViking session id (`cx-<safe-session-id>`) directly from the Codex session id (no plugin state read, so a corrupt state file can't crash recall), calls `/api/v1/search/search` with that `session_id`, ranks results, and reads full content for top-ranked leaves before compression.
+`auto-recall.mjs` adapts the Codex prompt/session payload and calls the shared
+`buildRecallBlockDetailed()` pipeline. The shared core owns context search,
+legacy `/recall`, raw-search fallback, ranking, injection budgets, digest selection,
+compression caching and URI repair. Codex owns the `cx-<safe-session-id>` mapping,
+model/profile selection, CLI execution and the hook deadline.
```json
-{ "hookSpecificOutput": { "hookEventName": "UserPromptSubmit", "additionalContext": "<openviking-context source=\"auto-recall\" format=\"digest\">\nOpenViking memory digest:\n- ...\n</openviking-context>" } }
+{ "hookSpecificOutput": { "hookEventName": "UserPromptSubmit", "additionalContext": "<openviking-context>\n...\n</openviking-context>" } }
```
-Codex injects `additionalContext` into the model turn, so memories arrive without an extra tool call. By default, recalled context below `OPENVIKING_RECALL_COMPRESS_MIN_INPUT_CHARS` is injected directly; larger blocks pass through the shared relevance compressor, and an identical query/context pair reuses its cached digest. If the compressor returns `NO_RELEVANT_MEMORY`, empty text, or non-digest chatter, the hook emits `{}` and injects nothing. The whole hook has its own `OPENVIKING_RECALL_TIMEOUT_MS` deadline (default 120s); the bundled `hooks.json` gives Codex 130s so the script can return `{}` before Codex kills it. Digests keep validated `viking://` source URIs and point the model at the OpenViking MCP `read`/`search` tools for details when the inline bullet is intentionally short. The outer `<openviking-context ...>` wrapper is deterministic, not compressor-generated; capture strips it to distinguish recalled context from the user's prompt. Set `OPENVIKING_RECALL_COMPRESS=0` to fall back to deterministic short formatting.
+The shared core prefers a server digest and suppresses injection for
+`no_relevant` / `NO_RELEVANT_MEMORY`. Local compressor failures retain bounded
+retrieved context. Raw fallback uses session-aware search when a session exists,
+then retries without the session if all targets are empty; unavailable search
+can fall back to `find`. Explicit user targets retain the home-alias fallback.
+Local compression receives bounded full leaf content before injection truncation.
+Without local compression, `recallPreferAbstract` and the shared token budget
+control the fallback (including URI and wrapper overhead).
+
+The hook has an `OPENVIKING_RECALL_TIMEOUT_MS` deadline (default 120s); the bundled
+hook allows 130s. Nested compressor calls disable automatic memory hooks and are
+killed on timeout. A failed compressor is not restarted for a legacy-peer pass
+within the same turn. Capture recognizes the shared `<openviking-context>` wrapper
+and removes injected context from newly captured messages.
The compressor profile is recreated on every `SessionStart` and cached under `OPENVIKING_CODEX_STATE_DIR` so cross-session config changes are picked up but each `UserPromptSubmit` does not probe models. Default fallback order:
@@ -303,8 +320,8 @@ Config knobs:
| Env var | Default | Meaning |
|---|---|---|
| `OPENVIKING_RECALL_LIMIT` | `10` | Legacy quota-scaling input; explicit values are converted to six coding quotas, not enforced as a final result cap. |
-| `OPENVIKING_RECALL_COMPRESS` | `1` | Set `0` / `off` to disable `codex exec` compression. |
-| `OPENVIKING_RECALL_COMPRESS_MODEL` | unset | Custom first-choice compressor model. Set `off` to disable compression. |
+| `OPENVIKING_RECALL_COMPRESS` | `auto` | `server`: cloud rewrite, never launches `codex exec`; `client`: local only; `auto`: local when available, otherwise cloud; `off` / `0`: uncompressed. `1` aliases `auto`. |
+| `OPENVIKING_RECALL_COMPRESS_MODEL` | unset | Custom first-choice compressor model. Set `off` to disable the local compressor (`auto` then uses cloud compression). |
| `OPENVIKING_RECALL_COMPRESS_THINKING` | unset | Custom `model_reasoning_effort`; `default` omits the Codex config override. Alias: `OPENVIKING_RECALL_COMPRESS_REASONING_EFFORT`. |
| `OPENVIKING_RECALL_COMPRESS_BASE_URL` | unset | Base URL for the nested compressor's provider. Use this when `--ignore-user-config` prevents the compressor from reading the main Codex provider configuration. |
| `OPENVIKING_RECALL_COMPRESS_MIN_INPUT_CHARS` | `1500` | Skip the nested compressor below this recalled-context size. Set `0` to compress every non-empty result. |
@@ -487,3 +504,16 @@ The Codex marketplace catalog that exposes this plugin for `codex plugin marketp
## License
Apache-2.0 — same as [OpenViking](https://github.com/volcengine/OpenViking).
+
+
+### Cloud recall compression
+
+Set `OPENVIKING_RECALL_COMPRESS=server` to request `POST /api/v1/search/search`
+with `mode: "context", rewrite: true`. This also disables local startup compressor
+probes. A returned server digest is injected without a second local compression
+pass; a server `no_relevant` result injects nothing. If rewrite is unavailable,
+the hook preserves the existing raw-context / legacy retrieval fallback.
+
+`auto` uses `rewrite: "auto"` when the Codex executable or its compressor profile
+is unavailable (including a cached runtime failure). A first local failure still
+uses the deterministic fallback for that turn; later turns use the server.
diff --git a/examples/codex-memory-plugin/scripts/auto-recall.mjs b/examples/codex-memory-plugin/scripts/auto-recall.mjs
index 9e745c482..909a5b203 100644
--- a/examples/codex-memory-plugin/scripts/auto-recall.mjs
+++ b/examples/codex-memory-plugin/scripts/auto-recall.mjs
@@ -13,28 +13,14 @@
* is just `{}`.
*/
-import { mkdtemp, readFile, rm } from "node:fs/promises";
-import { tmpdir } from "node:os";
import { join } from "node:path";
import { loadConfig } from "./config.mjs";
-import { trySpawnCodex } from "./codex-launch.mjs";
import { createLogger } from "./debug-log.mjs";
-import {
- buildCodexExecArgs,
- fallbackRecallCompressorProfile,
- loadCachedRecallCompressorProfile,
- markRecallCompressorRuntimeFailed,
-} from "./recall-compressor-profile.mjs";
-import { deriveOvSessionId, getStateDir } from "./session-state.mjs";
-import {
- buildRecallEndpointBody,
- fetchAssembledContext,
- normalizeContextEntry,
- postRecall,
-} from "./shared/recall-core.mjs";
+import { getStateDir, deriveOvSessionId } from "./session-state.mjs";
+import { createCodexCompressor } from "./host-compressor.mjs";
+import { buildRecallBlockDetailed } from "./shared/recall-core.mjs";
import { runHookStage } from "./shared/agent-hook-runtime.mjs";
import { createOvHttp } from "./shared/ov-http.mjs";
-import { compressRecallContext } from "./shared/recall-compress-core.mjs";
import { applyInputFilters, compileInputFilters } from "./shared/input-filters.mjs";
import { resolveEffectivePeerId } from "./shared/workspace-peer.mjs";
@@ -45,7 +31,6 @@ let effectivePeer = { peerId: "" };
let emitted = false;
let activeCompressor = null;
let recallDeadline = null;
-const DEFAULT_FINAL_RECALL_CHARS = 6500;
const RECALL_DIGEST_CACHE_PATH = join(getStateDir(), "recall-digest.json");
function output(obj, exitAfter = false) {
@@ -60,22 +45,12 @@ function output(obj, exitAfter = false) {
process.stdout.write(line);
}
-function wrapRecallContext(additionalContext) {
- const body = sanitizeInjectedText(additionalContext).trim();
- if (!body) return "";
- return [
- '<openviking-context source="auto-recall" format="digest">',
- body,
- "</openviking-context>",
- ].join("\n");
-}
-
function emit(additionalContext) {
if (!additionalContext) {
output({});
return;
}
- const wrappedContext = wrapRecallContext(additionalContext);
+ const wrappedContext = String(additionalContext).trim();
if (!wrappedContext) {
output({});
return;
@@ -108,434 +83,6 @@ function makeFetchJSON() {
let fetchJSON = makeFetchJSON();
-// ---------------------------------------------------------------------------
-// Ranking
-// ---------------------------------------------------------------------------
-
-function clampScore(v) {
- if (typeof v !== "number" || Number.isNaN(v)) return 0;
- return Math.max(0, Math.min(1, v));
-}
-
-const PREFERENCE_QUERY_RE = /prefer|preference|favorite|favourite|like|偏好|喜欢|爱好|更倾向/i;
-const TEMPORAL_QUERY_RE = /when|what time|date|day|month|year|yesterday|today|tomorrow|last|next|什么时候|何时|哪天|几月|几年|昨天|今天|明天/i;
-const QUERY_TOKEN_RE = /[a-z0-9一-龥]{2,}/gi;
-const STOPWORDS = new Set([
- "what", "when", "where", "which", "who", "whom", "whose", "why", "how", "did", "does",
- "is", "are", "was", "were", "the", "and", "for", "with", "from", "that", "this", "your", "you",
-]);
-
-function buildQueryProfile(query) {
- const text = query.trim();
- const allTokens = text.toLowerCase().match(QUERY_TOKEN_RE) || [];
- const tokens = allTokens.filter((t) => !STOPWORDS.has(t));
- return {
- tokens,
- wantsPreference: PREFERENCE_QUERY_RE.test(text),
- wantsTemporal: TEMPORAL_QUERY_RE.test(text),
- };
-}
-
-function lexicalOverlapBoost(tokens, text) {
- if (tokens.length === 0 || !text) return 0;
- const haystack = ` ${text.toLowerCase()} `;
- let matched = 0;
- for (const token of tokens.slice(0, 8)) {
- if (haystack.includes(token)) matched += 1;
- }
- return Math.min(0.2, (matched / Math.min(tokens.length, 4)) * 0.2);
-}
-
-function getRankingBreakdown(item, profile) {
- const base = clampScore(item.score);
- const abstract = (item.abstract || item.overview || "").trim();
- const cat = (item.category || "").toLowerCase();
- const uri = item.uri.toLowerCase();
- const leafBoost = (item.level === 2 || uri.endsWith(".md")) ? 0.12 : 0;
- const eventBoost = profile.wantsTemporal && (cat === "events" || uri.includes("/events/")) ? 0.1 : 0;
- const prefBoost = profile.wantsPreference && (cat === "preferences" || uri.includes("/preferences/")) ? 0.08 : 0;
- const overlapBoost = lexicalOverlapBoost(profile.tokens, `${item.uri} ${abstract}`);
- return {
- baseScore: base,
- leafBoost,
- eventBoost,
- prefBoost,
- overlapBoost,
- finalScore: base + leafBoost + eventBoost + prefBoost + overlapBoost,
- };
-}
-
-function rankForInjection(item, profile) {
- return getRankingBreakdown(item, profile).finalScore;
-}
-
-function dedupeByAbstract(items) {
- const seen = new Set();
- return items.filter((item) => {
- const key = (item.abstract || item.overview || "").trim().toLowerCase() || item.uri;
- if (seen.has(key)) return false;
- seen.add(key);
- return true;
- });
-}
-
-function pickMemories(items, limit, queryText) {
- if (items.length === 0 || limit <= 0) return [];
- const profile = buildQueryProfile(queryText);
- const sorted = [...items].sort((a, b) => rankForInjection(b, profile) - rankForInjection(a, profile));
- const deduped = dedupeByAbstract(sorted);
- const leaves = deduped.filter((m) => m.level === 2 || m.uri.endsWith(".md"));
- if (leaves.length >= limit) return leaves.slice(0, limit);
- const picked = [...leaves];
- const used = new Set(picked.map((m) => m.uri));
- for (const item of deduped) {
- if (picked.length >= limit) break;
- if (used.has(item.uri)) continue;
- picked.push(item);
- }
- return picked;
-}
-
-function postProcess(items, limit, threshold) {
- const seen = new Set();
- const sorted = [...items].sort((a, b) => clampScore(b.score) - clampScore(a.score));
- const result = [];
- for (const item of sorted) {
- if (item.level !== 2) continue;
- if (clampScore(item.score) < threshold) continue;
- const cat = (item.category || "").toLowerCase() || "unknown";
- const abs = (item.abstract || item.overview || "").trim().toLowerCase();
- const key = abs ? `${cat}:${abs}` : `uri:${item.uri}`;
- if (seen.has(key)) continue;
- seen.add(key);
- result.push(item);
- if (result.length >= limit) break;
- }
- return result;
-}
-
-async function searchScope(query, targetUri, limit, bucket = "memories", sessionId = null) {
- const body = { query, target_uri: targetUri, limit, score_threshold: 0 };
- if (sessionId) body.session_id = sessionId;
- const result = await fetchJSON("/api/v1/search/search", {
- method: "POST",
- body: JSON.stringify(body),
- });
- return result.ok ? (result.result?.[bucket] || []) : [];
-}
-
-// Candidate target URIs for a bucket, most-specific first. In trusted mode a
-// user's memories live under viking://user/<user>/<bucket>; in api_key mode the
-// configured user may be unknown, so the viking://~ home alias lets the server
-// resolve the authenticated caller's own space. Trying the user-scoped path
-// first with the home alias as a fallback recalls correctly in both modes.
-// De-duped so a missing/blank user never doubles the request count.
-function userScopedTargets(kind) {
- const suffix = kind.replace(/^\/+/, "");
- const targets = [`viking://~/${suffix}`];
- if (cfg.user) {
- targets.unshift(`viking://user/${cfg.user}/${suffix}`);
- }
- return [...new Set(targets)];
-}
-
-// Two-phase, short-circuiting search over the candidate targets:
-// 1) a session-scoped pass (uses OpenViking's session-aware planner);
-// 2) only if the entire session pass is empty, a single session-independent
-// pass (the planner can legitimately decide that no extra context is
-// needed for this session, but auto-recall still needs a memory lookup).
-// Each phase stops at the first non-empty target, so a warm user costs one
-// request and the worst case is bounded by (targets x 2) — instead of running
-// a per-target session+fallback for every target.
-async function searchBucket(query, targetUris, limit, bucket, sessionId = null) {
- for (const targetUri of targetUris) {
- const items = await searchScope(query, targetUri, limit, bucket, sessionId);
- if (items.length > 0) return items;
- }
- if (!sessionId) return [];
- for (const targetUri of targetUris) {
- const items = await searchScope(query, targetUri, limit, bucket, null);
- if (items.length > 0) return items;
- }
- return [];
-}
-
-async function searchAll(query, limit, sessionId = null) {
- const [userMems, userSkills] = await Promise.all([
- searchBucket(query, userScopedTargets("memories"), limit, "memories", sessionId),
- searchBucket(query, userScopedTargets("skills"), limit, "skills", sessionId),
- ]);
- log("search_complete", { scope: "user", rawCount: userMems.length, topScores: userMems.slice(0, 3).map((m) => m.score) });
- log("search_complete", { scope: "skills", rawCount: userSkills.length, topScores: userSkills.slice(0, 3).map((m) => m.score) });
- const all = [...userMems, ...userSkills];
- const seen = new Set();
- return all.filter((m) => {
- if (seen.has(m.uri)) return false;
- seen.add(m.uri);
- return true;
- });
-}
-
-function resolveRecallSessionId(codexSessionId) {
- if (!codexSessionId) return null;
- // Derive directly: the OV session id is deterministic (cx-<safe-id>), so
- // recall does not need to read plugin state. This keeps the recall hook
- // crash-free even if the state file is corrupt/missing, and stays in sync
- // with capture, which now also derives cx-* unconditionally.
- return deriveOvSessionId(codexSessionId);
-}
-
-async function readMemoryContent(uri) {
- try {
- const result = await fetchJSON(`/api/v1/content/read?uri=${encodeURIComponent(uri)}`);
- if (result.ok && typeof result.result === "string" && result.result.trim()) return result.result.trim();
- } catch { /* fallback */ }
- return null;
-}
-
-function assembledToRecallResult(rendered, entries) {
- const items = entries
- .map(normalizeContextEntry)
- .map((entry) => ({ ...entry, score: clampScore(entry.score) }))
- .filter((entry) => entry.uri && entry.text);
- const context = rendered
- ? [
- "OpenViking memory digest:",
- rendered,
- "",
- "More detail: use the OpenViking MCP recall/read/search tools with cited viking:// URIs if needed.",
- ].join("\n")
- : "";
- return { context, items };
-}
-
-async function recallViaServerAssembly(query, ovSessionId = "") {
- const maxInputChars = cfg.recallCompress
- ? cfg.recallCompressMaxInputChars
- : DEFAULT_FINAL_RECALL_CHARS;
- const assembleCfg = {
- ...cfg,
- // Local compression happens below, so ask the server for the assembled
- // block only. The server budget stays independent from the compressor's
- // input-character ceiling.
- recallRewrite: "off",
- };
-
- const assembled = await fetchAssembledContext(fetchJSON, assembleCfg, query, {
- actorPeerId: effectivePeer.peerId,
- sessionId: ovSessionId,
- log,
- });
- if (assembled) {
- // `peer_scope: "all"` already sweeps every peer under this user; only
- // under "actor" does the pre-git peer need asking separately.
- const legacyPeerId = effectivePeer.legacyPeerId;
- if (cfg.recallPeerScope === "actor" && legacyPeerId && legacyPeerId !== effectivePeer.peerId) {
- const legacy = await fetchAssembledContext(fetchJSON, assembleCfg, query, {
- actorPeerId: legacyPeerId,
- sessionId: ovSessionId,
- log,
- });
- if (legacy) {
- log("recall_legacy_peer_hit", { legacyPeerId });
- return assembledToRecallResult(
- [assembled.rendered, legacy.rendered].filter(Boolean).join("\n"),
- [...(assembled.entries || []), ...(legacy.entries || [])],
- );
- }
- }
- return assembledToRecallResult(assembled.rendered, assembled.entries);
- }
-
- const body = buildRecallEndpointBody(cfg);
- body.query = query;
- body.max_chars = maxInputChars;
- const result = await postRecall(fetchJSON, body, { actorPeerId: effectivePeer.peerId, log });
- if (!result.ok) {
- log("recall_endpoint_fallback", { status: result.status || 0 });
- return null;
- }
- return assembledToRecallResult(
- String(result.result?.rendered || "").trim(),
- Array.isArray(result.result?.entries) ? result.result.entries : [],
- );
-}
-
-function truncateText(text, maxChars) {
- const value = String(text || "").trim();
- if (value.length <= maxChars) return value;
- return `${value.slice(0, Math.max(0, maxChars - 20)).trimEnd()}\n[truncated]`;
-}
-
-function sanitizeInjectedText(text) {
- return String(text || "")
- .replace(/<\/?relevant-memor(?:y|ies)\b[^>]*>/gi, "legacy memory wrapper")
- .replace(/<\/?openviking-context\b[^>]*>/gi, "openviking context marker");
-}
-
-function appendMcpRetrievalHint(text) {
- const value = String(text || "").trim();
- if (!/\bviking:\/\//i.test(value) || /OpenViking MCP/i.test(value)) return value;
- return `${value}\n\nMore detail: use the OpenViking MCP read/search tools with the cited viking:// URI if needed.`;
-}
-
-function fallbackDigest(items) {
- const lines = items.slice(0, cfg.recallCompressMaxBullets).map((item) => {
- const text = sanitizeInjectedText(truncateText(item.text, 260)).replace(/\s+/g, " ");
- return `- [${item.category || "memory"}] ${text} (${item.uri})`;
- });
- return lines.length > 0 ? appendMcpRetrievalHint(`OpenViking memory digest:\n${lines.join("\n")}`) : "";
-}
-
-function fallbackCompressionInput(items) {
- const perItemChars = Math.max(
- 500,
- Math.floor(cfg.recallCompressMaxInputChars / Math.max(1, items.length)),
- );
- return JSON.stringify({
- memories: items.map((item) => ({
- uri: item.uri,
- category: item.category || "memory",
- score: item.score,
- text: truncateText(item.text, perItemChars),
- })),
- }, null, 2);
-}
-
-async function getRecallCompressorProfile() {
- const cached = await loadCachedRecallCompressorProfile(cfg);
- if (cached) return cached;
- const fallback = fallbackRecallCompressorProfile(cfg);
- log("compress_profile_cache_miss", fallback);
- return fallback;
-}
-
-async function runCodexCompressor(prompt, profile) {
- const tmp = await mkdtemp(join(tmpdir(), "ov-recall-compress-"));
- const outputPath = join(tmp, "last-message.txt");
- const args = buildCodexExecArgs(profile, outputPath, cfg);
-
- try {
- return await new Promise((resolve) => {
- const env = {
- ...process.env,
- OPENVIKING_AUTO_RECALL: "0",
- OPENVIKING_AUTO_CAPTURE: "0",
- OPENVIKING_RECALL_COMPRESS: "0",
- };
- let child = null;
- let timer = null;
- let done = false;
- let timedOut = false;
- let stderr = "";
- const finish = (value, { runtimeFailed = false } = {}) => {
- if (done) return;
- done = true;
- if (activeCompressor === child) activeCompressor = null;
- clearTimeout(timer);
- if (runtimeFailed) {
- // Mark the profile as runtime_failed so subsequent UPS calls in
- // this same codex session skip compress (avoids burning
- // ~recallCompressTimeoutMs per turn on a guaranteed-to-fail
- // spawn). Next SessionStart's cache-first detect treats this
- // marker as a cache miss and re-resolves against the current
- // catalogue, so a transient failure self-recovers across codex
- // restarts. Best-effort write; failure is non-fatal.
- markRecallCompressorRuntimeFailed(cfg, { failedModel: profile.model || "" })
- .catch(() => {});
- }
- resolve(value);
- };
- const launch = trySpawnCodex(args, { env, stdio: ["pipe", "ignore", "pipe"] });
- if (launch.error) {
- logError("compress_spawn", launch.error);
- finish(null, { runtimeFailed: true });
- return;
- }
- child = launch.child;
- activeCompressor = child;
- timer = setTimeout(() => {
- timedOut = true;
- logError("compress_timeout", `timed out after ${cfg.recallCompressTimeoutMs}ms`);
- try {
- child.kill("SIGKILL");
- } catch { /* best effort */ }
- }, cfg.recallCompressTimeoutMs);
-
- child.stderr.on("data", (chunk) => {
- stderr += chunk.toString();
- if (stderr.length > 4000) stderr = stderr.slice(-4000);
- });
- child.on("error", (err) => {
- logError("compress_spawn", err);
- finish(null, { runtimeFailed: true });
- });
- child.on("close", async (code) => {
- if (timedOut) {
- finish(null, { runtimeFailed: true });
- return;
- }
- if (code !== 0) {
- logError("compress_exit", {
- profile,
- error: stderr.trim().slice(-1000) || `codex exited ${code}`,
- });
- finish(null, { runtimeFailed: true });
- return;
- }
- try {
- finish(await readFile(outputPath, "utf-8"));
- } catch (err) {
- logError("compress_read", err);
- finish(null, { runtimeFailed: true });
- }
- });
- child.stdin.end(prompt);
- });
- } finally {
- await rm(tmp, { recursive: true, force: true }).catch(() => {});
- }
-}
-
-async function compressMemoryContext(userPrompt, rendered, items, shortContext = rendered) {
- if (!cfg.recallCompress) return null;
- const input = String(rendered || "").trim() || fallbackDigest(items);
- if (!input) return "";
-
- let profile = null;
- try {
- const compression = await compressRecallContext({
- query: userPrompt,
- rendered: input,
- shortContext,
- entries: items,
- cfg,
- cachePath: RECALL_DIGEST_CACHE_PATH,
- now: Date.now(),
- // Short inputs and cache hits return before host model state is needed.
- runCompressor: async (prompt) => {
- profile = await getRecallCompressorProfile();
- if (!profile.enabled) {
- log("compress_skip", { reason: "profile disabled", profile });
- return "";
- }
- return await runCodexCompressor(prompt, profile) ?? "";
- },
- });
- log("compressed", {
- status: compression.status,
- inputCount: items.length,
- chars: compression.context.length,
- profile,
- });
- if (compression.status === "failed") return null;
- return appendMcpRetrievalHint(compression.context);
- } catch (err) {
- logError("compress", err);
- return null;
- }
-}
-
runHookStage({
loadConfig,
gates: { enabled: (reloaded) => reloaded.autoRecall },
@@ -549,7 +96,7 @@ runHookStage({
let userPrompt = (input.prompt || "").trim();
const codexSessionId = typeof input.session_id === "string" ? input.session_id.trim() : "";
- const recallSessionId = resolveRecallSessionId(codexSessionId);
+ const recallSessionId = codexSessionId ? deriveOvSessionId(codexSessionId) : "";
log("start", {
codexSessionId: codexSessionId || null,
recallSessionId,
@@ -589,89 +136,15 @@ runHookStage({
return;
}
- const endpointRecall = await recallViaServerAssembly(userPrompt, recallSessionId || "");
- if (endpointRecall !== null) {
- if (!endpointRecall.context && endpointRecall.items.length === 0) {
- log("skip", { stage: "recall_endpoint", reason: "no results" });
- return;
- }
- const compressedContext = endpointRecall.items.length > 0
- ? await compressMemoryContext(userPrompt, endpointRecall.context, endpointRecall.items)
- : null;
- const endpointFallback = cfg.recallCompress && endpointRecall.items.length > 0
- ? fallbackDigest(endpointRecall.items)
- : endpointRecall.context;
- const memoryContext = compressedContext === null
- ? endpointFallback
- : compressedContext;
- if (!memoryContext) {
- log("skip", { stage: "recall_endpoint", reason: "compressor found no relevant memory" });
- return;
- }
- log("recall_endpoint", {
- chars: memoryContext.length,
- compressed: compressedContext !== null,
- entryCount: endpointRecall.items.length,
- });
- return memoryContext;
- }
-
- const candidateLimit = Math.max(cfg.recallLimit * 4, 20);
- const allMemories = await searchAll(userPrompt, candidateLimit, recallSessionId);
- if (allMemories.length === 0) {
- log("skip", { stage: "search", reason: "no results" });
- return;
- }
-
- const processed = postProcess(allMemories, candidateLimit, cfg.scoreThreshold);
- log("post_process", { beforeCount: allMemories.length, afterCount: processed.length });
-
- const profile = buildQueryProfile(userPrompt);
- const ranked = [...processed]
- .map((item) => ({ item, breakdown: getRankingBreakdown(item, profile) }))
- .sort((a, b) => b.breakdown.finalScore - a.breakdown.finalScore);
-
- if (cfg.logRankingDetails) {
- for (const entry of ranked) {
- log("ranking_detail", { uri: entry.item.uri, ...entry.breakdown });
- }
- } else {
- log("ranking_summary", {
- candidateCount: processed.length,
- topCandidates: ranked.slice(0, 5).map((entry) => ({ uri: entry.item.uri, finalScore: entry.breakdown.finalScore })),
- });
- }
-
- const memories = pickMemories(processed, cfg.recallLimit, userPrompt);
- if (memories.length === 0) {
- log("skip", { stage: "pick", reason: "no memories survived ranking" });
- return;
- }
-
- log("picked", { pickedCount: memories.length, uris: memories.map((m) => m.uri) });
-
- const memoryItems = await Promise.all(
- memories.map(async (item) => {
- let text = (item.abstract || item.overview || item.uri).trim();
- if (item.level === 2) {
- const content = await readMemoryContent(item.uri);
- if (content) text = content;
- }
- return {
- uri: item.uri,
- category: item.category || "memory",
- score: clampScore(item.score),
- text,
- };
- }),
- );
-
- const fallbackContext = fallbackDigest(memoryItems);
- const compressedContext = await compressMemoryContext(
- userPrompt,
- fallbackCompressionInput(memoryItems),
- memoryItems,
- fallbackContext,
- );
- return compressedContext === null ? fallbackContext : compressedContext;
+ const runCompressor = await createCodexCompressor(cfg, {
+ log, logError, onActiveChild: (child) => { activeCompressor = child; },
+ });
+ const recalled = await buildRecallBlockDetailed(fetchJSON, cfg, userPrompt, {
+ actorPeerId: effectivePeer.peerId, legacyPeerId: effectivePeer.legacyPeerId,
+ sessionId: recallSessionId || "", runCompressor,
+ localCompressorAvailable: Boolean(runCompressor),
+ digestCachePath: RECALL_DIGEST_CACHE_PATH, log,
+ });
+ log("recall_complete", { stage: recalled.stage, chars: recalled.block.length });
+ return recalled.block;
}).catch((err) => { logError("uncaught", err); emit(); });
diff --git a/examples/codex-memory-plugin/scripts/auto-recall.test.mjs b/examples/codex-memory-plugin/scripts/auto-recall.test.mjs
index a4cbdea6b..ea9a6f61f 100644
--- a/examples/codex-memory-plugin/scripts/auto-recall.test.mjs
+++ b/examples/codex-memory-plugin/scripts/auto-recall.test.mjs
@@ -331,7 +331,7 @@ test("auto-recall prefers the server recall endpoint when available", async () =
);
const output = JSON.parse(result.stdout.trim());
- assert.match(output.hookSpecificOutput.additionalContext, /OpenViking memory digest/);
+ assert.match(output.hookSpecificOutput.additionalContext, /^<openviking-context>/);
assert.match(output.hookSpecificOutput.additionalContext, /Launch summary/);
});
@@ -340,7 +340,7 @@ test("auto-recall prefers the server recall endpoint when available", async () =
"/api/v1/search/recall",
]);
assert.equal(Object.values(requests[1].body.quotas).reduce((sum, quota) => sum + quota, 0), 3);
- assert.equal(requests[1].body.max_chars, 6500);
+ assert.equal(requests[1].body.max_chars, 1000);
} finally {
await rm(stateDir, { recursive: true, force: true });
}
@@ -655,6 +655,7 @@ test("auto-recall expands configured user in memory search target", async () =>
OPENVIKING_CLI_CONFIG_FILE: join(stateDir, "missing-ovcli.conf"),
OPENVIKING_CREDENTIAL_SOURCE: "env",
OPENVIKING_USER: "zeus",
+ OPENVIKING_RECALL_PREFER_ABSTRACT: "0",
OPENVIKING_RECALL_COMPRESS: "0",
OPENVIKING_RECALL_LIMIT: "1",
OPENVIKING_RECALL_TIMEOUT_MS: "10000",
@@ -741,6 +742,7 @@ test("auto-recall preserves explicit default user memory target", async () => {
OPENVIKING_CLI_CONFIG_FILE: join(stateDir, "missing-ovcli.conf"),
OPENVIKING_CREDENTIAL_SOURCE: "env",
OPENVIKING_USER: "default",
+ OPENVIKING_RECALL_PREFER_ABSTRACT: "0",
OPENVIKING_RECALL_COMPRESS: "0",
OPENVIKING_RECALL_LIMIT: "1",
OPENVIKING_RECALL_TIMEOUT_MS: "10000",
@@ -1049,3 +1051,90 @@ test("auto-recall authenticates with Bearer alone and gates the identity headers
await rm(stateDir, { recursive: true, force: true });
}
});
+
+for (const scenario of [
+ {mode: "server", rewrite: true, digest: "Cloud digest", calls: 0},
+ {mode: "server", rewrite: true, digest: "", calls: 0},
+ {mode: "auto", rewrite: undefined, digest: "", calls: 1},
+ {mode: "client", rewrite: undefined, digest: "", calls: 1},
+ {mode: "off", rewrite: undefined, digest: "", calls: 0},
+ {mode: "auto", model: "off", rewrite: "auto", digest: "Cloud digest", calls: 0},
+ {mode: "auto", missingCli: true, rewrite: "auto", digest: "Cloud digest", calls: 0},
+ {mode: "auto", rewrite: undefined, digest: "Cloud digest", calls: 0},
+ {mode: "server", rewrite: true, digest: "", noRelevant: true, calls: 0},
+]) {
+ test(`context rewrite routing ${JSON.stringify(scenario)}`, async () => {
+ const stateDir = await mkdtemp(join(tmpdir(), "ov-rewrite-mode-"));
+ try {
+ await withFakeCodex("- Local digest [viking://user/test/memories/example.md]", async ({env, callLog}) => {
+ let body;
+ const rendered = "Raw context. ".repeat(200);
+ const output = await withMockOpenViking(async (req,res) => {
+ if (req.url === "/health") return writeJson(res,{status:"ok", result:{ok:true}});
+ if (req.url === "/api/v1/search/search") {
+ body = await readRequestBody(req);
+ return writeJson(res,{status:"ok",result:{
+ rendered, digest:scenario.digest,
+ entries:[{uri:"viking://user/test/memories/example.md",text:rendered,score:0.9}],
+ stats:{rewrite:scenario.noRelevant ? "no_relevant" : "ok"},
+ }});
+ }
+ writeStatusJson(res,404,{status:"error"});
+ },baseUrl => runAutoRecall({prompt:"recall the project conventions",session_id:"rewrite-test"},{
+ ...env,
+ ...(scenario.missingCli ? {PATH: stateDir} : {}),
+ OPENVIKING_URL:baseUrl, OPENVIKING_CREDENTIAL_SOURCE:"env",
+ OPENVIKING_CONFIG_FILE:join(stateDir,"missing.conf"),
+ OPENVIKING_CLI_CONFIG_FILE:join(stateDir,"missing-cli.conf"),
+ OPENVIKING_CODEX_STATE_DIR:stateDir,OPENVIKING_STATE_DIR:stateDir,
+ OPENVIKING_HOME:stateDir,
+ OPENVIKING_RECALL_COMPRESS:scenario.mode,
+ OPENVIKING_RECALL_COMPRESS_MODEL:scenario.model || "test-model",
+ OPENVIKING_RECALL_COMPRESS_MIN_INPUT_CHARS:"0",
+ OPENVIKING_RECALL_TIMEOUT_MS:"15000",OPENVIKING_RECALL_COMPRESS_TIMEOUT_MS:"5000",
+ }));
+ assert.equal(body.rewrite,scenario.rewrite);
+ const calls = (await readFile(callLog,"utf8").catch(()=>"")).trim().split("\n").filter(Boolean).length;
+ assert.equal(calls,scenario.calls);
+ const result = JSON.parse(output.stdout);
+ if (scenario.noRelevant) assert.deepEqual(result,{});
+ else {
+ const context = result.hookSpecificOutput.additionalContext;
+ assert.match(context,scenario.digest ? /Cloud digest/ : scenario.calls ? /Local digest/ : /Raw context/);
+ if (scenario.digest) assert.doesNotMatch(context,/Raw context/);
+ }
+ });
+ } finally { await rm(stateDir,{recursive:true,force:true}); }
+ });
+}
+
+test("a failed local compressor is not relaunched for another peer in the same hook", async () => {
+ const stateDir = await mkdtemp(join(tmpdir(), "ov-compressor-latch-"));
+ try {
+ await withFakeCodex("unused", async ({ env, callLog }) => {
+ const script = `
+ const { loadConfig } = await import(${JSON.stringify(new URL('./config.mjs', import.meta.url).href)});
+ const { createCodexCompressor } = await import(${JSON.stringify(new URL('./host-compressor.mjs', import.meta.url).href)});
+ const run = await createCodexCompressor(loadConfig());
+ if (!run) throw new Error("missing compressor");
+ if (await run("primary peer") !== null) throw new Error("expected failure");
+ if (await run("legacy peer") !== null) throw new Error("expected disabled compressor");
+ `;
+ await new Promise((resolve, reject) => {
+ const child = spawn(process.execPath, ['--input-type=module', '-e', script], {
+ env: { ...process.env, ...env,
+ OPENVIKING_CODEX_STATE_DIR: stateDir, OPENVIKING_HOME: stateDir,
+ OPENVIKING_CONFIG_FILE: join(stateDir, 'missing'), OPENVIKING_CLI_CONFIG_FILE: join(stateDir, 'missing-cli'),
+ OPENVIKING_RECALL_COMPRESS: 'client', OPENVIKING_RECALL_COMPRESS_MODEL: 'test-model',
+ OPENVIKING_RECALL_COMPRESS_TIMEOUT_MS: '1000',
+ }, stdio: ['ignore', 'pipe', 'pipe'],
+ });
+ let stderr = '';
+ child.stderr.on('data', data => { stderr += data; });
+ child.on('error', reject);
+ child.on('close', code => code === 0 ? resolve() : reject(new Error(stderr)));
+ });
+ assert.equal((await readFile(callLog, 'utf8')).trim().split('\n').length, 1);
+ }, { exitCode: 1 });
+ } finally { await rm(stateDir, { recursive: true, force: true }); }
+});
diff --git a/examples/codex-memory-plugin/scripts/codex-launch.mjs b/examples/codex-memory-plugin/scripts/codex-launch.mjs
index 98dd06506..881e35344 100644
--- a/examples/codex-memory-plugin/scripts/codex-launch.mjs
+++ b/examples/codex-memory-plugin/scripts/codex-launch.mjs
@@ -1,6 +1,6 @@
import { spawn } from "node:child_process";
-import { existsSync } from "node:fs";
-import { win32 } from "node:path";
+import { accessSync, constants, existsSync } from "node:fs";
+import { delimiter, join, win32 } from "node:path";
function stripPathQuotes(value) {
const entry = String(value || "").trim();
@@ -73,3 +73,18 @@ export function trySpawnCodex(args, options, {
return { ...launch, child: null, error };
}
}
+
+/** Check the executable without starting a model or a CLI subprocess. */
+export function isCodexAvailable({
+ platform = process.platform,
+ pathValue = process.env.PATH || "",
+ canExecute = (path) => {
+ try { accessSync(path, constants.X_OK); return true; } catch { return false; }
+ },
+ pathExists = existsSync,
+} = {}) {
+ if (platform === "win32") {
+ return resolveCodexLaunch({ platform, pathValue, pathExists }).command !== "codex";
+ }
+ return pathValue.split(delimiter).some((entry) => canExecute(join(entry || ".", "codex")));
+}
diff --git a/examples/codex-memory-plugin/scripts/config.mjs b/examples/codex-memory-plugin/scripts/config.mjs
index d77fbfc7d..cbb9e3c97 100644
--- a/examples/codex-memory-plugin/scripts/config.mjs
+++ b/examples/codex-memory-plugin/scripts/config.mjs
@@ -9,8 +9,7 @@
* registry → ovcli.conf `plugin.codex` → ovcli.conf `plugin` → ov.conf's
* `codex` section (legacy) → the schema's defaults
*
- * What stays here is what only this harness knows: how it reads the digest
- * switch, and what counts as having configured a compressor.
+ * What stays here is what only this harness knows: what counts as having configured a compressor.
*
* Credential source:
* - Default (auto): env-var credentials win when any credential env var is
@@ -44,15 +43,6 @@ import { buildPluginConfig } from "./shared/plugin-config.mjs";
const MANIFEST_URL = new URL("../.codex-plugin/plugin.json", import.meta.url);
-function configBool(value, fallback) {
- if (typeof value === "boolean") return value;
- const lower = String(value ?? "").trim().toLowerCase();
- if (lower === "0" || lower === "false" || lower === "no" || lower === "off") return false;
- if (lower === "1" || lower === "true" || lower === "yes" || lower === "on"
- || lower === "auto" || lower === "client") return true;
- return fallback;
-}
-
/**
* `cwd` selects the workspace layer (`.openviking/config.json` and the registry
* entry for that directory). It defaults to this process's directory, which is
@@ -71,9 +61,6 @@ export function loadConfig(cwd = process.cwd(), { env = process.env } = {}) {
return {
...config,
- // Codex reads the compression knob as on/off; "auto" and "client" are the
- // Claude Code spellings of on, and mean the same thing here.
- recallCompress: configBool(config.recallCompress, true),
// Not `configured.has`: what makes a compressor configured here is having
// been told which model to run, not having named the switch.
recallCompressConfigured: Boolean(config.recallCompressModel || config.recallCompressThinking),
diff --git a/examples/codex-memory-plugin/scripts/config.test.mjs b/examples/codex-memory-plugin/scripts/config.test.mjs
index af2878a76..b9bbc0e18 100644
--- a/examples/codex-memory-plugin/scripts/config.test.mjs
+++ b/examples/codex-memory-plugin/scripts/config.test.mjs
@@ -8,7 +8,7 @@ import { isBypassed } from "./shared/session-model.mjs";
// The layers, the knobs and the peer are the shared loader's, and
// memory-plugin-shared/plugin-config.test.mjs holds this harness to them. What
-// is left here is the compression switch this harness reads as a boolean, and
+// is left here is the compression modes this harness preserves, and
// the bypass patterns it is the only harness to resolve from every layer.
const OVERRIDES = [
"OPENVIKING_CONFIG_FILE",
@@ -18,6 +18,7 @@ const OVERRIDES = [
"OPENVIKING_BYPASS_SESSION",
"OPENVIKING_BYPASS_SESSION_PATTERNS",
"OPENVIKING_RECALL_COMPRESS",
+ "OPENVIKING_RECALL_REWRITE",
"OPENVIKING_RECALL_COMPRESS_MODEL",
"OPENVIKING_RECALL_COMPRESS_THINKING",
"OPENVIKING_URL",
@@ -114,21 +115,21 @@ test("ov.conf's codex section still supplies the patterns when no env var does",
});
});
-// This harness reads the shared compression knob as a boolean, and calls a
-// compressor configured only once it has been told what to run.
-test("the compression switch is a boolean here, and a model is what configures one", () => {
+// Naming the mode must not count as configuring a local model.
+test("compression modes survive config resolution, and a model configures the local compressor", () => {
withConfigs({
cli: { url: "http://127.0.0.1:1933", api_key: "sk-cli", plugin: { recallCompress: "auto" } },
}, ({ otherDir }) => {
const cfg = loadConfig(otherDir);
- assert.equal(cfg.recallCompress, true, "\"auto\" is the Claude Code spelling of on");
+ assert.equal(cfg.recallCompress, "auto");
+ assert.equal(cfg.recallRewrite, "auto");
assert.equal(cfg.recallCompressConfigured, false, "naming the switch is not naming a compressor");
});
withConfigs({
cli: { url: "http://127.0.0.1:1933", api_key: "sk-cli", plugin: { recallCompress: "off" } },
}, ({ otherDir }) => {
- assert.equal(loadConfig(otherDir).recallCompress, false);
+ assert.equal(loadConfig(otherDir).recallCompress, "off");
});
withConfigs({
@@ -205,3 +206,12 @@ test("a comma survives in a configured rule but splits an env one", () => {
assert.deepEqual(loadConfig().captureFilters, ["s/a{2", "}/X/"]);
});
});
+
+for (const mode of ["off", "client", "server", "auto", "0", "1"]) {
+ test(`compression env preserves ${mode}`, () => {
+ withConfigs({env: {OPENVIKING_RECALL_COMPRESS: mode}}, ({otherDir}) => {
+ const expected = mode === "0" ? "off" : mode === "1" ? "auto" : mode;
+ assert.equal(loadConfig(otherDir).recallRewrite, expected);
+ });
+ });
+}
diff --git a/examples/codex-memory-plugin/scripts/host-compressor.mjs b/examples/codex-memory-plugin/scripts/host-compressor.mjs
new file mode 100644
index 000000000..addc899b1
--- /dev/null
+++ b/examples/codex-memory-plugin/scripts/host-compressor.mjs
@@ -0,0 +1,108 @@
+import { mkdtemp, readFile, rm } from "node:fs/promises";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import { isCodexAvailable, trySpawnCodex } from "./codex-launch.mjs";
+import { buildCodexExecArgs, fallbackRecallCompressorProfile, loadCachedRecallCompressorProfile, markRecallCompressorRuntimeFailed } from "./recall-compressor-profile.mjs";
+
+export async function createCodexCompressor(cfg, { log = () => {}, logError = () => {}, onActiveChild = () => {} } = {}) {
+ if (!["client", "auto"].includes(cfg.recallRewrite) || !isCodexAvailable()) return null;
+ const profile = await loadCachedRecallCompressorProfile(cfg) || fallbackRecallCompressorProfile(cfg);
+ if (!profile.enabled) return null;
+ log("compress_profile", profile);
+ let available = true;
+ return async (prompt) => {
+ if (!available) return null;
+ const output = await runCodexCompressor(prompt, profile, cfg, { logError, onActiveChild });
+ // A legacy-peer pass in this hook must not launch a model that just failed.
+ if (output === null) available = false;
+ return output;
+ };
+}
+
+async function runCodexCompressor(prompt, profile, cfg, { logError, onActiveChild }) {
+ const tmp = await mkdtemp(join(tmpdir(), "ov-recall-compress-"));
+ const outputPath = join(tmp, "last-message.txt");
+ const args = buildCodexExecArgs(profile, outputPath, cfg);
+
+ try {
+ return await new Promise((resolve) => {
+ const env = {
+ ...process.env,
+ OPENVIKING_AUTO_RECALL: "0",
+ OPENVIKING_AUTO_CAPTURE: "0",
+ OPENVIKING_RECALL_COMPRESS: "0",
+ };
+ let child = null;
+ let timer = null;
+ let done = false;
+ let timedOut = false;
+ let stderr = "";
+ const finish = (value, { runtimeFailed = false } = {}) => {
+ if (done) return;
+ done = true;
+ onActiveChild(null);
+ clearTimeout(timer);
+ if (runtimeFailed) {
+ // Mark the profile as runtime_failed so subsequent UPS calls in
+ // this same codex session skip compress (avoids burning
+ // ~recallCompressTimeoutMs per turn on a guaranteed-to-fail
+ // spawn). Next SessionStart's cache-first detect treats this
+ // marker as a cache miss and re-resolves against the current
+ // catalogue, so a transient failure self-recovers across codex
+ // restarts. Best-effort write; failure is non-fatal.
+ markRecallCompressorRuntimeFailed(cfg, { failedModel: profile.model || "" })
+ .catch(() => {});
+ }
+ resolve(value);
+ };
+ const launch = trySpawnCodex(args, { env, stdio: ["pipe", "ignore", "pipe"] });
+ if (launch.error) {
+ logError("compress_spawn", launch.error);
+ finish(null, { runtimeFailed: true });
+ return;
+ }
+ child = launch.child;
+ onActiveChild(child);
+ timer = setTimeout(() => {
+ timedOut = true;
+ logError("compress_timeout", `timed out after ${cfg.recallCompressTimeoutMs}ms`);
+ try {
+ child.kill("SIGKILL");
+ } catch { /* best effort */ }
+ }, cfg.recallCompressTimeoutMs);
+
+ child.stderr.on("data", (chunk) => {
+ stderr += chunk.toString();
+ if (stderr.length > 4000) stderr = stderr.slice(-4000);
+ });
+ child.on("error", (err) => {
+ logError("compress_spawn", err);
+ finish(null, { runtimeFailed: true });
+ });
+ child.on("close", async (code) => {
+ if (timedOut) {
+ finish(null, { runtimeFailed: true });
+ return;
+ }
+ if (code !== 0) {
+ logError("compress_exit", {
+ profile,
+ error: stderr.trim().slice(-1000) || `codex exited ${code}`,
+ });
+ finish(null, { runtimeFailed: true });
+ return;
+ }
+ try {
+ finish(await readFile(outputPath, "utf-8"));
+ } catch (err) {
+ logError("compress_read", err);
+ finish(null, { runtimeFailed: true });
+ }
+ });
+ child.stdin.end(prompt);
+ });
+ } finally {
+ await rm(tmp, { recursive: true, force: true }).catch(() => {});
+ }
+}
+
diff --git a/examples/codex-memory-plugin/scripts/ov-memory-doctor.mjs b/examples/codex-memory-plugin/scripts/ov-memory-doctor.mjs
index 035c02ce3..ab7c52c1f 100644
--- a/examples/codex-memory-plugin/scripts/ov-memory-doctor.mjs
+++ b/examples/codex-memory-plugin/scripts/ov-memory-doctor.mjs
@@ -320,7 +320,7 @@ function checkConfig(report, cfg, host) {
const peer = reportPeer(report, cfg);
reportTimeouts(report, cfg, host);
- const toggles = [`auto-inject ${cfg.noAutoInject ? "OFF" : "on"}`, `auto-recall ${cfg.autoRecall ? "on" : "OFF"}`, `auto-capture ${cfg.autoCapture ? "on" : "OFF"}`, `commit on compact ${cfg.autoCommitOnCompact ? "on" : "OFF"}`, `recall compress ${cfg.recallCompress ? "on" : "off"}`, `write path ${cfg.writePathAsync ? "async" : "sync"}`];
+ const toggles = [`auto-inject ${cfg.noAutoInject ? "OFF" : "on"}`, `auto-recall ${cfg.autoRecall ? "on" : "OFF"}`, `auto-capture ${cfg.autoCapture ? "on" : "OFF"}`, `commit on compact ${cfg.autoCommitOnCompact ? "on" : "OFF"}`, `recall compress ${cfg.recallRewrite}`, `write path ${cfg.writePathAsync ? "async" : "sync"}`];
reportToggles(report, cfg, host, toggles);
sweepEnv(report, cfg, host, (r, env) => {
diff --git a/examples/codex-memory-plugin/scripts/recall-compressor-profile.mjs b/examples/codex-memory-plugin/scripts/recall-compressor-profile.mjs
index 21e1f0ea6..aeaaea0c5 100644
--- a/examples/codex-memory-plugin/scripts/recall-compressor-profile.mjs
+++ b/examples/codex-memory-plugin/scripts/recall-compressor-profile.mjs
@@ -2,6 +2,7 @@ import { readFile, rm } from "node:fs/promises";
import { homedir } from "node:os";
import { join } from "node:path";
import { mkdir, rename, writeFile } from "node:fs/promises";
+import { normalizeRewriteMode } from "./shared/plugin-config.mjs";
import { getStateDir } from "./session-state.mjs";
const DEFAULT_PRIMARY = { model: "gpt-5.3-codex-spark", thinking: "default", source: "default_primary" };
@@ -25,7 +26,9 @@ function normalizeModel(value) {
}
export function recallCompressionExplicitlyOff(cfg) {
- return !cfg.recallCompress || isOff(cfg.recallCompressModel) || isOff(cfg.recallCompressThinking);
+ const mode = normalizeRewriteMode(cfg.recallRewrite ?? cfg.recallCompress, "off");
+ return mode === "off" || mode === "server"
+ || isOff(cfg.recallCompressModel) || isOff(cfg.recallCompressThinking);
}
export function buildCodexExecArgs(profile, outputPath, cfg = {}) {
diff --git a/examples/codex-memory-plugin/scripts/shared/config-schema.mjs b/examples/codex-memory-plugin/scripts/shared/config-schema.mjs
index 4a1a7c9c4..55342f2c4 100644
--- a/examples/codex-memory-plugin/scripts/shared/config-schema.mjs
+++ b/examples/codex-memory-plugin/scripts/shared/config-schema.mjs
@@ -98,7 +98,7 @@ export const KNOBS = [
// off/client/server/auto through `normalizeRewriteMode`; Codex reads the same
// key as on/off. Both spellings stay declared so neither harness's users are
// told their config is a typo.
- { name: "recallCompress", type: "string", default: "auto", env: "OPENVIKING_RECALL_COMPRESS", aliases: ["recallRewrite"], capability: "recall" },
+ { name: "recallCompress", type: "string", default: "off", harness: { claude_code: "auto", codex: "auto" }, env: "OPENVIKING_RECALL_COMPRESS", aliases: ["recallRewrite"], capability: "recall" },
{ name: "recallCompressModel", type: "string", default: "", env: "OPENVIKING_RECALL_COMPRESS_MODEL", capability: "recall" },
{ name: "recallCompressBaseUrl", type: "string", default: "", env: "OPENVIKING_RECALL_COMPRESS_BASE_URL", capability: "recall" },
{ name: "recallCompressThinking", type: "string", default: "", env: "OPENVIKING_RECALL_COMPRESS_THINKING", aliases: ["recallCompressReasoningEffort"], capability: "recall" },
diff --git a/examples/codex-memory-plugin/scripts/shared/plugin-config.mjs b/examples/codex-memory-plugin/scripts/shared/plugin-config.mjs
index 214a287f9..f60c3bca8 100644
--- a/examples/codex-memory-plugin/scripts/shared/plugin-config.mjs
+++ b/examples/codex-memory-plugin/scripts/shared/plugin-config.mjs
@@ -29,7 +29,7 @@
import { homedir } from "node:os";
import { join } from "node:path";
-import { HARNESS_KEYS, KNOBS, harnessKey, resolveKnobs } from "./config-schema.mjs";
+import { HARNESS_KEYS, KNOBS, harnessKey, knobDefault, resolveKnobs } from "./config-schema.mjs";
import {
buildUserAgent,
loadCredentialFiles,
@@ -224,9 +224,15 @@ export function buildPluginConfig(harness, {
env,
});
+ const recallRewrite = normalizeRewriteMode(
+ env.OPENVIKING_RECALL_COMPRESS ?? env.OPENVIKING_RECALL_REWRITE ?? settings.recallCompress,
+ knobDefault(KNOBS.find((knob) => knob.name === "recallCompress"), harnessKey(name)),
+ );
const config = {
...settings,
harness: name,
+ recallCompress: recallRewrite,
+ recallRewrite,
clientVersion,
userAgent: buildUserAgent(name, clientVersion),
diff --git a/examples/codex-memory-plugin/scripts/shared/recall-core.mjs b/examples/codex-memory-plugin/scripts/shared/recall-core.mjs
index 4695a887b..b893078b0 100644
--- a/examples/codex-memory-plugin/scripts/shared/recall-core.mjs
+++ b/examples/codex-memory-plugin/scripts/shared/recall-core.mjs
@@ -309,21 +309,46 @@ async function resolveTargetUri(fetchJSON, targetUri, actorPeerId = "") {
return `viking://user/${space}/${parts.join("/")}`;
}
-async function searchOneSource(fetchJSON, query, source, limit, actorPeerId = "") {
- const resolvedUri = await resolveTargetUri(fetchJSON, source.uri, actorPeerId);
- const body = { query, target_uri: resolvedUri, limit, score_threshold: 0 };
- const res = await fetchJSON("/api/v1/search/find", {
- method: "POST",
- body: JSON.stringify(body),
- }, { actorPeerId });
- if (!res.ok) return [];
- const items = res.result?.[source.bucket] || [];
- return items.map((item) => ({ ...item, _sourceType: source.type }));
+async function searchOneSource(fetchJSON, cfg, query, source, limit, options) {
+ const actorPeerId = options.actorPeerId || "";
+ const home = await resolveTargetUri(fetchJSON, source.uri, actorPeerId);
+ const targets = [...new Set(cfg.user
+ ? [`viking://user/${cfg.user}/${source.bucket}`, home] : [home])];
+ const sessionId = String(options.sessionId || "").trim();
+ const search = async (target, session) => {
+ const body = { query, target_uri: target, limit, score_threshold: 0 };
+ if (session) body.session_id = session;
+ const init = { method: "POST", body: JSON.stringify(body) };
+ // Session-aware retrieval may decide no context is needed. Only after all
+ // targets are empty do we retry without the session. Old servers still
+ // support find; an unsupported search route can fall back to it.
+ let res = await fetchJSON(sessionId ? "/api/v1/search/search" : "/api/v1/search/find", init, { actorPeerId });
+ if (sessionId && !res.ok && (
+ !res.status || res.status === 404 || res.status === 405 || res.status >= 500
+ || ((res.status === 400 || res.status === 422) && looksLikeUnknownField(res))
+ )) {
+ const { session_id, ...findBody } = body;
+ res = await fetchJSON("/api/v1/search/find", { method: "POST", body: JSON.stringify(findBody) }, { actorPeerId });
+ }
+ const items = res.ok && Array.isArray(res.result?.[source.bucket]) ? res.result[source.bucket] : [];
+ return items.map((item) => ({ ...item, _sourceType: source.type }));
+ };
+ for (const target of targets) {
+ const items = await search(target, sessionId);
+ if (items.length) return items;
+ }
+ if (sessionId) {
+ for (const target of targets) {
+ const items = await search(target, "");
+ if (items.length) return items;
+ }
+ }
+ return [];
}
-async function searchAllSources(fetchJSON, query, perSourceLimit, actorPeerId = "", log = () => {}) {
+async function searchAllSources(fetchJSON, cfg, query, perSourceLimit, options, log = () => {}) {
const results = await Promise.all(
- SOURCES.map((src) => searchOneSource(fetchJSON, query, src, perSourceLimit, actorPeerId)),
+ SOURCES.map((src) => searchOneSource(fetchJSON, cfg, query, src, perSourceLimit, options)),
);
const all = results.flat();
log("recall_search_summary", {
@@ -359,50 +384,30 @@ async function resolveItemContent(fetchJSON, item, cfg, actorPeerId = "") {
return content;
}
-async function buildFallbackInjectionBlock(fetchJSON, items, cfg, actorPeerId = "", log = () => {}) {
+function formatFallback(items, cfg, log = () => {}) {
const budgetTotal = Math.max(200, Number(cfg.recallTokenBudget || 2000));
- let budgetRemaining = budgetTotal;
- const lines = [
- "<openviking-context>",
- "Relevant context from OpenViking. Use the read MCP tool to expand URIs.",
- ];
+ const lines = [];
let contentCount = 0;
let hintCount = 0;
-
+ const maxChars = Math.max(50, Number(cfg.recallMaxContentChars || 500));
for (const item of items) {
const score = (clampScore(item.score) * 100).toFixed(0);
- const uriLine = `- [${item._sourceType} ${score}%] ${item.uri}`;
-
- if (budgetRemaining > 0) {
- const content = await resolveItemContent(fetchJSON, item, cfg, actorPeerId);
- const contentLine = `- [${item._sourceType} ${score}%] ${content}`;
- const lineTokens = estimateTokens(contentLine);
-
- if (lineTokens > budgetRemaining && contentCount > 0) {
- lines.push(uriLine);
- hintCount++;
- } else {
- lines.push(contentLine);
- budgetRemaining -= lineTokens;
- contentCount++;
- }
- } else {
+ const kind = item._sourceType || item.category || "memory";
+ const uriLine = `- [${kind} ${score}%] ${item.uri}`;
+ const text = String(item.text || item.uri);
+ const content = text.length > maxChars ? `${text.slice(0, maxChars)}...` : text;
+ const contentLine = `- [${kind} ${score}%] ${content}`;
+ if (estimateTokens(wrapContext([...lines, uriLine, contentLine].join("\n"))) <= budgetTotal) {
+ lines.push(uriLine, contentLine);
+ contentCount += 1;
+ } else if (estimateTokens(wrapContext([...lines, uriLine].join("\n"))) <= budgetTotal) {
lines.push(uriLine);
- hintCount++;
+ hintCount += 1;
}
}
-
- lines.push("</openviking-context>");
-
- const budgetUsed = budgetTotal - budgetRemaining;
- log("recall_injection_built", {
- contentItems: contentCount,
- hintItems: hintCount,
- budgetUsed,
- budgetTotal,
- });
-
- return { block: lines.join("\n"), contentCount, hintCount, budgetUsed, stage: "ranked" };
+ const budgetUsed = lines.length ? estimateTokens(wrapContext(lines.join("\n"))) : 0;
+ log("recall_injection_built", { contentItems: contentCount, hintItems: hintCount, budgetUsed, budgetTotal });
+ return { rendered: lines.join("\n"), contentCount, hintCount, budgetUsed };
}
const LEGACY_CACHE_TTL_MS = 6 * 60 * 60 * 1000;
@@ -469,10 +474,14 @@ function looksLikeUnknownField(res) {
}
function wrapContext(body) {
+ // Retrieved text cannot introduce a second capture delimiter inside ours.
+ const text = String(body)
+ .replace(/<\/?relevant-memor(?:y|ies)\b[^>]*>/gi, "legacy memory wrapper")
+ .replace(/<\/?openviking-context\b[^>]*>/gi, "openviking context marker");
return [
"<openviking-context>",
"Relevant memory from OpenViking. Use the search/read MCP tools to expand URIs.",
- body,
+ text,
"</openviking-context>",
].join("\n");
}
@@ -488,7 +497,7 @@ export async function buildServerAssembledBlock(fetchJSON, cfg, query, options =
const block = await recallViaContextFace(fetchJSON, cfg, query, { ...options, actorPeerId }, log);
if (block !== null) return block;
- return recallViaEndpoint(fetchJSON, cfg, query, actorPeerId, log);
+ return recallViaEndpoint(fetchJSON, cfg, query, { ...options, actorPeerId }, log);
}
/**
@@ -552,53 +561,65 @@ export function normalizeContextEntry(entry = {}) {
};
}
-async function recallViaContextFace(fetchJSON, cfg, query, options, log) {
- const assembled = await fetchAssembledContext(fetchJSON, cfg, query, { ...options, log });
- if (assembled === null) return null;
+/** Server relevance decisions and digest precedence are shared by every host. */
+export function selectRecallContent(result = {}) {
+ if (String(result.stats?.rewrite || "").toLowerCase() === "no_relevant") return "";
+ return String(result.digest || "").trim() || String(result.rendered || "").trim();
+}
- const { rendered, entries } = assembled;
- let digest = assembled.digest;
+function wantsLocalCompression(cfg, options) {
const mode = String(cfg.recallRewrite || "off").toLowerCase();
- if (String(assembled.stats?.rewrite || "").toLowerCase() === "no_relevant") {
- log("recall_server_compression", { status: "empty" });
- return "";
- }
- const wantsLocal = mode === "client" || (mode === "auto" && !digest);
- if (wantsLocal && rendered && typeof options.runCompressor === "function") {
- try {
- const compression = await compressRecallContext({
- query,
- rendered,
- entries,
- cfg,
- runCompressor: options.runCompressor,
- cachePath: options.digestCachePath || stateFile("recall-digest.json"),
- now: Date.now(),
- });
- log("recall_local_compression", { status: compression.status });
- if (compression.status === "ok") digest = compression.context;
- if (compression.status === "empty") return "";
- } catch (err) {
- log("recall_local_compression_failed", { error: String(err?.message || err) });
- }
+ return (mode === "client" || mode === "auto")
+ && options.localCompressorAvailable !== false
+ && typeof options.runCompressor === "function";
+}
+
+// Context, legacy recall, and raw retrieval share the same result policy.
+// The host callback only runs its model; it never chooses a fallback or digest.
+async function finalizeRecall(result, cfg, query, options, log, shortContext) {
+ const selected = selectRecallContent(result);
+ if (String(result.stats?.rewrite || "").toLowerCase() === "no_relevant") return "";
+ const entries = (result.entries || []).map(normalizeContextEntry).filter((entry) => entry.uri);
+ const rendered = String(result.rendered || "").trim();
+ if (String(result.digest || "").trim() || !wantsLocalCompression(cfg, options)) return selected;
+ const fallback = shortContext ?? (entries.length ? formatFallback(entries, cfg).rendered : rendered);
+ const input = rendered || entries.map((entry) => `${entry.uri}\n${entry.text}`).join("\n");
+ if (!input) return "";
+ try {
+ const compression = await compressRecallContext({
+ query, rendered: input, shortContext: shortContext ?? (rendered || fallback),
+ entries, cfg, runCompressor: options.runCompressor,
+ cachePath: options.digestCachePath || stateFile("recall-digest.json"), now: Date.now(),
+ });
+ log("recall_local_compression", { status: compression.status });
+ if (compression.status === "ok") return compression.context;
+ if (compression.status === "empty") return "";
+ } catch (err) {
+ log("recall_local_compression_failed", { error: String(err?.message || err) });
}
+ return fallback;
+}
- const injected = digest || rendered;
- if (!injected) return "";
- return wrapContext(injected);
+async function recallViaContextFace(fetchJSON, cfg, query, options, log) {
+ const assembled = await fetchAssembledContext(fetchJSON, cfg, query, { ...options, log });
+ if (assembled === null) return null;
+ const text = await finalizeRecall(assembled, cfg, query, options, log);
+ return text ? wrapContext(text) : "";
}
-async function recallViaEndpoint(fetchJSON, cfg, query, actorPeerId = "", log = () => {}) {
+async function recallViaEndpoint(fetchJSON, cfg, query, options, log) {
const body = buildRecallEndpointBody(cfg);
body.query = query;
- const res = await postRecall(fetchJSON, body, { actorPeerId, log });
+ if (wantsLocalCompression(cfg, options)) {
+ body.max_chars = Math.max(1000, Number(cfg.recallCompressMaxInputChars || 18000));
+ }
+ const res = await postRecall(fetchJSON, body, { actorPeerId: options.actorPeerId, log });
if (!res.ok) {
log("recall_endpoint_fallback", { status: res.status || 0 });
return null;
}
- const rendered = String(res.result?.rendered || "").trim();
- if (!rendered) return "";
- return wrapContext(rendered);
+ const text = await finalizeRecall(res.result || {}, cfg, query, options, log);
+ return text ? wrapContext(text) : "";
}
export async function postRecall(fetchJSON, body, opts = {}) {
@@ -711,7 +732,7 @@ async function recallForPeer(fetchJSON, cfg, query, options = {}) {
const recallLimit = Math.max(1, Number(cfg.recallLimit || DEFAULT_CONTEXT_LIMIT));
const perSourceLimit = Math.max(recallLimit * 2, 8);
- const raw = await searchAllSources(fetchJSON, trimmed, perSourceLimit, actorPeerId, log);
+ const raw = await searchAllSources(fetchJSON, cfg, trimmed, perSourceLimit, { ...options, actorPeerId }, log);
if (raw.length === 0) return emptyRecall("no_results");
const profile = buildQueryProfile(trimmed);
@@ -727,5 +748,25 @@ async function recallForPeer(fetchJSON, cfg, query, options = {}) {
});
if (picked.length === 0) return emptyRecall("filtered_out");
- return buildFallbackInjectionBlock(fetchJSON, picked, cfg, actorPeerId, log);
+ const local = wantsLocalCompression(cfg, options);
+ const inputLimit = Math.max(1000, Number(cfg.recallCompressMaxInputChars || 18000));
+ const contentCfg = local ? { ...cfg, recallPreferAbstract: false, recallMaxContentChars: inputLimit } : cfg;
+ const entries = await Promise.all(picked.map(async (item) => ({
+ ...item, text: await resolveItemContent(fetchJSON, item, contentCfg, actorPeerId),
+ })));
+ const fallback = formatFallback(entries, cfg, log);
+ let text = await finalizeRecall({
+ rendered: local ? entries.map((item) => `${item.uri}\n${item.text}`).join("\n").slice(0, inputLimit) : fallback.rendered,
+ entries,
+ }, cfg, trimmed, options, log, fallback.rendered);
+ if (estimateTokens(wrapContext(text)) > Math.max(200, Number(cfg.recallTokenBudget || 2000))) {
+ text = fallback.rendered;
+ }
+ if (!text) return emptyRecall("no_results");
+ const unchanged = text === fallback.rendered;
+ return {
+ block: wrapContext(text), contentCount: unchanged ? fallback.contentCount : 1,
+ hintCount: unchanged ? fallback.hintCount : 0,
+ budgetUsed: estimateTokens(wrapContext(text)), stage: "ranked",
+ };
}
diff --git a/examples/dsh-memory-plugin/config.mjs b/examples/dsh-memory-plugin/config.mjs
index 7aecb5d91..de1463ed1 100644
--- a/examples/dsh-memory-plugin/config.mjs
+++ b/examples/dsh-memory-plugin/config.mjs
@@ -1,7 +1,7 @@
import { buildPluginConfig } from "./shared/plugin-config.mjs";
import { loadCredentialFiles } from "./shared/credentials.mjs";
-export const PLUGIN_VERSION = "0.4.3";
+export const PLUGIN_VERSION = "0.4.4";
/**
* Namespace for the bridged OpenViking MCP tools. DSH publishes every MCP tool
diff --git a/examples/dsh-memory-plugin/package-lock.json b/examples/dsh-memory-plugin/package-lock.json
index 4aff24e29..c70651bc8 100644
--- a/examples/dsh-memory-plugin/package-lock.json
+++ b/examples/dsh-memory-plugin/package-lock.json
@@ -1,12 +1,12 @@
{
"name": "@openviking/dsh-memory-plugin",
- "version": "0.4.3",
+ "version": "0.4.4",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@openviking/dsh-memory-plugin",
- "version": "0.4.3",
+ "version": "0.4.4",
"license": "Apache-2.0",
"devDependencies": {
"@deepseek-ai/dsh-llm": "0.1.0-rc.6",
diff --git a/examples/dsh-memory-plugin/package.json b/examples/dsh-memory-plugin/package.json
index 826fb1de6..32e7633b6 100644
--- a/examples/dsh-memory-plugin/package.json
+++ b/examples/dsh-memory-plugin/package.json
@@ -1,6 +1,6 @@
{
"name": "@openviking/dsh-memory-plugin",
- "version": "0.4.3",
+ "version": "0.4.4",
"description": "OpenViking memory and context bundle for DeepSeek Harness",
"type": "module",
"main": "index.mjs",
diff --git a/examples/hermes-plugin/README.md b/examples/hermes-plugin/README.md
index bc394a12d..d92d2f7f6 100644
--- a/examples/hermes-plugin/README.md
+++ b/examples/hermes-plugin/README.md
@@ -230,3 +230,17 @@ are also allowed because OpenViking supports them. The tool rejects directories,
resources, skills, sessions, generated summary files, and URIs with query
strings or fragments. Use OpenViking's MCP, CLI, or admin APIs for broader
resource and directory cleanup.
+
+
+### Cloud recall compression
+
+Set `OPENVIKING_RECALL_COMPRESS=server` to enable cloud recall compression, or
+`auto` to let the server decide whether to rewrite. Both use search
+`mode=context`; `server` sends `rewrite=true`, and `auto` sends `rewrite="auto"`.
+The server digest takes precedence over raw rendered context, and `no_relevant`
+suppresses injection. The default remains `off`; no local compressor is launched.
+
+The Hermes config equivalent is `memory.openviking.recall_compress: server`.
+When enabled, the default request and total recall deadlines become 55 seconds;
+explicit recall timeout settings still take precedence. Older servers fall back
+to the existing search path within that deadline.
diff --git a/examples/hermes-plugin/__init__.py b/examples/hermes-plugin/__init__.py
index ad8a96256..07331b3fa 100644
--- a/examples/hermes-plugin/__init__.py
+++ b/examples/hermes-plugin/__init__.py
@@ -88,6 +88,12 @@ _CONFIG_SCHEMA = [
_cfg_field("account", "Advanced local identity override (leave blank for user API keys)"),
_cfg_field("user", "Advanced local user override (leave blank for user API keys)"),
_cfg_field("agent", "Optional peer ID for separate assistant context. Uses user memory when no peer is configured.", default=_DEFAULT_AGENT),
+ _cfg_field(
+ "recall_compress",
+ "Cloud recall compression: off, server or auto (no local compressor)",
+ type="string",
+ default="off",
+ ),
_cfg_field("recall_limit", "Maximum memories injected by automatic recall", type="integer", minimum=1, maximum=100, default=6),
_cfg_field("recall_score_threshold", "Minimum relevance score for automatic recall", type="number", minimum=0.0, maximum=1.0, step=0.01, default=0.15),
_cfg_field("recall_max_injected_chars", "Maximum total characters injected by recall", type="integer", minimum=100, maximum=50000, default=4000),
@@ -1595,11 +1601,53 @@ class OpenVikingMemoryProvider(MemoryProvider):
try:
cfg = self._recall_config()
deadline = time.monotonic() + cfg["timeout_seconds"]
- result = self._unwrap_result(self._post_prefetch_search(
- client, query_text, session_id, limit=max(cfg["limit"] * 4, 20),
- context_type=["memory", "resource"] if cfg["resources"] else "memory",
- deadline=deadline, request_timeout=cfg["request_timeout_seconds"],
- ))
+ if cfg["compress"] in ("server", "auto"):
+ payload = {
+ "query": query_text,
+ "mode": "context",
+ "purpose": "coding",
+ "rewrite": True if cfg["compress"] == "server" else "auto",
+ "score_threshold": cfg["score_threshold"],
+ "max_tokens": max(64, min(32000, cfg["max_injected_chars"] // 4)),
+ "context_type": ["memory", "resource"] if cfg["resources"] else "memory",
+ }
+ if session_id:
+ payload["session_id"] = session_id
+ try:
+ assembled = self._unwrap_result(
+ client.post(
+ "/api/v1/search/search",
+ payload,
+ timeout=self._remaining_recall_timeout(
+ deadline, cfg["request_timeout_seconds"]
+ ),
+ )
+ )
+ if isinstance(assembled, dict) and any(
+ k in assembled for k in ("rendered", "digest", "entries")
+ ):
+ if (assembled.get("stats") or {}).get("rewrite") == "no_relevant":
+ return ""
+ return str(
+ assembled.get("digest") or assembled.get("rendered") or ""
+ ).strip()
+ except TimeoutError:
+ raise
+ except Exception as e:
+ logger.debug(
+ "OpenViking context rewrite unavailable, falling back to search: %s", e
+ )
+ result = self._unwrap_result(
+ self._post_prefetch_search(
+ client,
+ query_text,
+ session_id,
+ limit=max(cfg["limit"] * 4, 20),
+ context_type=["memory", "resource"] if cfg["resources"] else "memory",
+ deadline=deadline,
+ request_timeout=cfg["request_timeout_seconds"],
+ )
+ )
if not isinstance(result, dict):
return ""
candidates = [item for ctx_type in ("memories", "resources") for item in (result.get(ctx_type, []) or []) if isinstance(item, dict)]
@@ -1615,8 +1663,15 @@ class OpenVikingMemoryProvider(MemoryProvider):
# -- typed settings ------------------------------------------------------
@staticmethod
- def _parse_setting_value(value: Any, kind: str) -> Optional[bool | int | float]:
+ def _parse_setting_value(value: Any, kind: str) -> Optional[bool | int | float | str]:
"""Parse per schema ``kind`` (boolean / integer / number); None when invalid."""
+ if kind == "string":
+ normalized = str(value).strip().lower()
+ if normalized in {"1", "true", "yes"}:
+ return "auto"
+ if normalized in {"0", "false", "no"}:
+ return "off"
+ return normalized if normalized in {"off", "server", "auto"} else None
if kind == "boolean":
if isinstance(value, bool):
return value
@@ -1656,7 +1711,16 @@ class OpenVikingMemoryProvider(MemoryProvider):
def _recall_config(self) -> Dict[str, Any]:
cfg = _load_hermes_openviking_config()
- return {key.removeprefix("recall_"): self._setting(key, cfg) for key in _RECALL_SETTING_KEYS}
+ resolved = {
+ key.removeprefix("recall_"): self._setting(key, cfg) for key in _RECALL_SETTING_KEYS
+ }
+ if resolved["compress"] in ("server", "auto"):
+ # Retrieval plus server rewrite has a longer fuse. Keep explicit
+ # user deadlines authoritative and the default off path unchanged.
+ for key in ("recall_timeout_seconds", "recall_request_timeout_seconds"):
+ if key not in cfg and not os.environ.get(_SETTING_SPECS[key]["env_var"]):
+ resolved[key.removeprefix("recall_")] = 55.0
+ return resolved
def _profile_token_budget(self) -> int:
return self._setting("profile_token_budget", _load_hermes_openviking_config())
diff --git a/examples/hermes-plugin/plugin.yaml b/examples/hermes-plugin/plugin.yaml
index 18b8ea787..348d8deb1 100644
--- a/examples/hermes-plugin/plugin.yaml
+++ b/examples/hermes-plugin/plugin.yaml
@@ -1,5 +1,5 @@
name: openviking
-version: 2.0.0
+version: 2.0.1
description: "OpenViking context database — session-managed memory with automatic extraction, tiered retrieval, and filesystem-style knowledge browsing."
pip_dependencies:
- httpx
diff --git a/examples/hermes-plugin/pyproject.toml b/examples/hermes-plugin/pyproject.toml
index 5d12aa0ff..f66de1d31 100644
--- a/examples/hermes-plugin/pyproject.toml
+++ b/examples/hermes-plugin/pyproject.toml
@@ -1,6 +1,6 @@
[project]
name = "hermes-plugin-openviking"
-version = "2.0.0"
+version = "2.0.1"
description = "Hermes Agent memory provider plugin: openviking"
requires-python = ">=3.11"
license = { text = "MIT" }
diff --git a/examples/hermes-plugin/tests/test_standalone.py b/examples/hermes-plugin/tests/test_standalone.py
index efe6b2d50..4fd3b98a4 100644
--- a/examples/hermes-plugin/tests/test_standalone.py
+++ b/examples/hermes-plugin/tests/test_standalone.py
@@ -136,3 +136,46 @@ def test_cancelled_external_setup_keeps_existing_config(external_provider, monke
provider.post_setup(str(home), {"memory": {"provider": "openviking"}})
assert config_path.read_bytes() == before
assert env_path.read_text(encoding="utf-8") == "UNRELATED_SETTING=keep\n"
+
+
[email protected]("mode,rewrite", [("server", True), ("auto", "auto")])
+ "response,expected",
+ [
+ ({"rendered": "raw", "digest": "compressed", "entries": []}, "compressed"),
+ ({"rendered": "raw", "entries": []}, "raw"),
+ ({"rendered": "raw", "stats": {"rewrite": "no_relevant"}}, ""),
+ ],
+)
+def test_cloud_recall(external_provider, monkeypatch, mode, rewrite, response, expected):
+ from unittest.mock import Mock
+
+ _, provider, _, _ = external_provider("cloud-recall")
+ monkeypatch.setenv("OPENVIKING_RECALL_COMPRESS", mode)
+ client = Mock()
+ client.post.return_value = {"result": response}
+ assert (
+ provider._search_prefetch_context(
+ "remember deployment preferences", session_id="session", client=client
+ )
+ == expected
+ )
+ client.post.assert_called_once()
+ path, body = client.post.call_args.args
+ assert path == "/api/v1/search/search"
+ assert body["mode"] == "context"
+ assert body["rewrite"] == rewrite
+ assert body["session_id"] == "session"
+ assert 50 < client.post.call_args.kwargs["timeout"] <= 55
+
+
+def test_cloud_recall_legacy_fallback(external_provider, monkeypatch):
+ from unittest.mock import Mock
+
+ _, provider, _, _ = external_provider("cloud-fallback")
+ monkeypatch.setenv("OPENVIKING_RECALL_COMPRESS", "server")
+ client = Mock()
+ client.post.side_effect = [RuntimeError("unsupported mode"), {"result": {"memories": []}}]
+ assert provider._search_prefetch_context("deployment preferences", client=client) == ""
+ assert client.post.call_count == 2
+ assert "rewrite" not in client.post.call_args.args[1]
diff --git a/examples/memory-plugin-shared/README.md b/examples/memory-plugin-shared/README.md
index e33b04908..01b11e547 100644
--- a/examples/memory-plugin-shared/README.md
+++ b/examples/memory-plugin-shared/README.md
@@ -85,3 +85,21 @@ Every file declares `version: 1`; one declaring another version is skipped with
Workspace files are trusted without a prompt: a hook is non-interactive, and an approval gate would degrade into one command per workspace. What is refused instead is structural — connection and credential keys (`url`, `api_key`, `account`, `user`, `extra_headers`, …) are stripped with a warning and `${VAR}` is never expanded in these files. What a committed file switches off is announced in `ov-memory-doctor` rather than blocked.
`.gitignore` must not ignore all of `.openviking/`, or `config.json` can never be committed. Narrow the rule to `.openviking/media/` and `.openviking/downloads/`; doctor warns while it is still blanket.
+
+## Cloud recall compression
+
+`OPENVIKING_RECALL_COMPRESS=server` enables cloud compression across Claude Code,
+Codex, OpenCode, DSH, Pi, the Cursor/Trae/Trae CN/ZCode hooks, OpenClaw, and Hermes.
+The context-search request sends `rewrite: true`; the server digest is preferred
+over raw rendered context, and `stats.rewrite: "no_relevant"` suppresses injection.
+
+Claude Code and Codex also support `client` for local-only compression. Their
+`auto` mode uses local compression when available and otherwise sends
+`rewrite: "auto"`. Harnesses without local compressors send `rewrite: "auto"`
+directly. Their existing default remains `off`; cloud compression is opt-in.
+`off` omits rewrite. The legacy boolean aliases `1` and `0` mean `auto` and `off`.
+
+These settings apply to automatic recall. Explicit MCP search calls retain the
+arguments supplied by the caller. Cloud compression requires a server that
+supports the context-search rewrite API; older-server fallback behavior remains
+specific to each integration.
diff --git a/examples/memory-plugin-shared/install-agent-hooks.test.mjs b/examples/memory-plugin-shared/install-agent-hooks.test.mjs
index e7a415c35..7c411f20c 100644
--- a/examples/memory-plugin-shared/install-agent-hooks.test.mjs
+++ b/examples/memory-plugin-shared/install-agent-hooks.test.mjs
@@ -363,7 +363,8 @@ test("combined hook-host install preserves unrelated hooks and is idempotent", (
[join(home, ".openviking", "agent-integrations", "cursor", "scripts", "ov-memory-doctor.mjs"), "cursor", "--offline", "--no-color"],
{ env: { ...process.env, HOME: home }, encoding: "utf8" },
);
- assert.match(doctor.stdout, /version 0\.3\.2, client cursor/);
+ const version = JSON.parse(readFileSync(join(checkout, "examples", "agent-hook-plugin", "plugin.json"), "utf8")).version;
+ assert.ok(doctor.stdout.includes(`version ${version}, client cursor`));
// A hooks.json entry that names a script the install did not put on disk
// fails only when the host first runs it, so the rendered commands are
// checked against the tree they were rendered for.
diff --git a/examples/memory-plugin-shared/lib/config-schema.mjs b/examples/memory-plugin-shared/lib/config-schema.mjs
index 5cd9f7c11..cce7198c3 100644
--- a/examples/memory-plugin-shared/lib/config-schema.mjs
+++ b/examples/memory-plugin-shared/lib/config-schema.mjs
@@ -97,7 +97,7 @@ export const KNOBS = [
// off/client/server/auto through `normalizeRewriteMode`; Codex reads the same
// key as on/off. Both spellings stay declared so neither harness's users are
// told their config is a typo.
- { name: "recallCompress", type: "string", default: "auto", env: "OPENVIKING_RECALL_COMPRESS", aliases: ["recallRewrite"], capability: "recall" },
+ { name: "recallCompress", type: "string", default: "off", harness: { claude_code: "auto", codex: "auto" }, env: "OPENVIKING_RECALL_COMPRESS", aliases: ["recallRewrite"], capability: "recall" },
{ name: "recallCompressModel", type: "string", default: "", env: "OPENVIKING_RECALL_COMPRESS_MODEL", capability: "recall" },
{ name: "recallCompressBaseUrl", type: "string", default: "", env: "OPENVIKING_RECALL_COMPRESS_BASE_URL", capability: "recall" },
{ name: "recallCompressThinking", type: "string", default: "", env: "OPENVIKING_RECALL_COMPRESS_THINKING", aliases: ["recallCompressReasoningEffort"], capability: "recall" },
diff --git a/examples/memory-plugin-shared/lib/plugin-config.mjs b/examples/memory-plugin-shared/lib/plugin-config.mjs
index 030fd52a3..3ae139992 100644
--- a/examples/memory-plugin-shared/lib/plugin-config.mjs
+++ b/examples/memory-plugin-shared/lib/plugin-config.mjs
@@ -28,7 +28,7 @@
import { homedir } from "node:os";
import { join } from "node:path";
-import { HARNESS_KEYS, KNOBS, harnessKey, resolveKnobs } from "./config-schema.mjs";
+import { HARNESS_KEYS, KNOBS, harnessKey, knobDefault, resolveKnobs } from "./config-schema.mjs";
import {
buildUserAgent,
loadCredentialFiles,
@@ -223,9 +223,15 @@ export function buildPluginConfig(harness, {
env,
});
+ const recallRewrite = normalizeRewriteMode(
+ env.OPENVIKING_RECALL_COMPRESS ?? env.OPENVIKING_RECALL_REWRITE ?? settings.recallCompress,
+ knobDefault(KNOBS.find((knob) => knob.name === "recallCompress"), harnessKey(name)),
+ );
const config = {
...settings,
harness: name,
+ recallCompress: recallRewrite,
+ recallRewrite,
clientVersion,
userAgent: buildUserAgent(name, clientVersion),
diff --git a/examples/memory-plugin-shared/lib/recall-core.d.mts b/examples/memory-plugin-shared/lib/recall-core.d.mts
index 1b91f025b..a3f1793d2 100644
--- a/examples/memory-plugin-shared/lib/recall-core.d.mts
+++ b/examples/memory-plugin-shared/lib/recall-core.d.mts
@@ -51,6 +51,10 @@ export type RecallOptions = {
legacyPeerId?: string;
sessionId?: string;
log?: (stage: string, data?: any) => void;
+ localCompressorAvailable?: boolean;
+ runCompressor?: ((prompt: string) => Promise<string | null>) | null;
+ digestCachePath?: string;
+ legacyCachePath?: string;
};
export type DetailedRecall = {
@@ -78,3 +82,5 @@ export function buildRecallBlock(
export function buildRecallEndpointBody(cfg?: Record<string, any>): Record<string, any>;
export function estimateTokens(text: string): number;
export function isRecallEnabled(cfg?: Record<string, any>): boolean;
+
+export function selectRecallContent(result?: { rendered?: string; digest?: string; stats?: Record<string, unknown> }): string;
diff --git a/examples/memory-plugin-shared/lib/recall-core.mjs b/examples/memory-plugin-shared/lib/recall-core.mjs
index 5f9b2f4ec..1c7409f00 100644
--- a/examples/memory-plugin-shared/lib/recall-core.mjs
+++ b/examples/memory-plugin-shared/lib/recall-core.mjs
@@ -308,21 +308,46 @@ async function resolveTargetUri(fetchJSON, targetUri, actorPeerId = "") {
return `viking://user/${space}/${parts.join("/")}`;
}
-async function searchOneSource(fetchJSON, query, source, limit, actorPeerId = "") {
- const resolvedUri = await resolveTargetUri(fetchJSON, source.uri, actorPeerId);
- const body = { query, target_uri: resolvedUri, limit, score_threshold: 0 };
- const res = await fetchJSON("/api/v1/search/find", {
- method: "POST",
- body: JSON.stringify(body),
- }, { actorPeerId });
- if (!res.ok) return [];
- const items = res.result?.[source.bucket] || [];
- return items.map((item) => ({ ...item, _sourceType: source.type }));
+async function searchOneSource(fetchJSON, cfg, query, source, limit, options) {
+ const actorPeerId = options.actorPeerId || "";
+ const home = await resolveTargetUri(fetchJSON, source.uri, actorPeerId);
+ const targets = [...new Set(cfg.user
+ ? [`viking://user/${cfg.user}/${source.bucket}`, home] : [home])];
+ const sessionId = String(options.sessionId || "").trim();
+ const search = async (target, session) => {
+ const body = { query, target_uri: target, limit, score_threshold: 0 };
+ if (session) body.session_id = session;
+ const init = { method: "POST", body: JSON.stringify(body) };
+ // Session-aware retrieval may decide no context is needed. Only after all
+ // targets are empty do we retry without the session. Old servers still
+ // support find; an unsupported search route can fall back to it.
+ let res = await fetchJSON(sessionId ? "/api/v1/search/search" : "/api/v1/search/find", init, { actorPeerId });
+ if (sessionId && !res.ok && (
+ !res.status || res.status === 404 || res.status === 405 || res.status >= 500
+ || ((res.status === 400 || res.status === 422) && looksLikeUnknownField(res))
+ )) {
+ const { session_id, ...findBody } = body;
+ res = await fetchJSON("/api/v1/search/find", { method: "POST", body: JSON.stringify(findBody) }, { actorPeerId });
+ }
+ const items = res.ok && Array.isArray(res.result?.[source.bucket]) ? res.result[source.bucket] : [];
+ return items.map((item) => ({ ...item, _sourceType: source.type }));
+ };
+ for (const target of targets) {
+ const items = await search(target, sessionId);
+ if (items.length) return items;
+ }
+ if (sessionId) {
+ for (const target of targets) {
+ const items = await search(target, "");
+ if (items.length) return items;
+ }
+ }
+ return [];
}
-async function searchAllSources(fetchJSON, query, perSourceLimit, actorPeerId = "", log = () => {}) {
+async function searchAllSources(fetchJSON, cfg, query, perSourceLimit, options, log = () => {}) {
const results = await Promise.all(
- SOURCES.map((src) => searchOneSource(fetchJSON, query, src, perSourceLimit, actorPeerId)),
+ SOURCES.map((src) => searchOneSource(fetchJSON, cfg, query, src, perSourceLimit, options)),
);
const all = results.flat();
log("recall_search_summary", {
@@ -358,50 +383,30 @@ async function resolveItemContent(fetchJSON, item, cfg, actorPeerId = "") {
return content;
}
-async function buildFallbackInjectionBlock(fetchJSON, items, cfg, actorPeerId = "", log = () => {}) {
+function formatFallback(items, cfg, log = () => {}) {
const budgetTotal = Math.max(200, Number(cfg.recallTokenBudget || 2000));
- let budgetRemaining = budgetTotal;
- const lines = [
- "<openviking-context>",
- "Relevant context from OpenViking. Use the read MCP tool to expand URIs.",
- ];
+ const lines = [];
let contentCount = 0;
let hintCount = 0;
-
+ const maxChars = Math.max(50, Number(cfg.recallMaxContentChars || 500));
for (const item of items) {
const score = (clampScore(item.score) * 100).toFixed(0);
- const uriLine = `- [${item._sourceType} ${score}%] ${item.uri}`;
-
- if (budgetRemaining > 0) {
- const content = await resolveItemContent(fetchJSON, item, cfg, actorPeerId);
- const contentLine = `- [${item._sourceType} ${score}%] ${content}`;
- const lineTokens = estimateTokens(contentLine);
-
- if (lineTokens > budgetRemaining && contentCount > 0) {
- lines.push(uriLine);
- hintCount++;
- } else {
- lines.push(contentLine);
- budgetRemaining -= lineTokens;
- contentCount++;
- }
- } else {
+ const kind = item._sourceType || item.category || "memory";
+ const uriLine = `- [${kind} ${score}%] ${item.uri}`;
+ const text = String(item.text || item.uri);
+ const content = text.length > maxChars ? `${text.slice(0, maxChars)}...` : text;
+ const contentLine = `- [${kind} ${score}%] ${content}`;
+ if (estimateTokens(wrapContext([...lines, uriLine, contentLine].join("\n"))) <= budgetTotal) {
+ lines.push(uriLine, contentLine);
+ contentCount += 1;
+ } else if (estimateTokens(wrapContext([...lines, uriLine].join("\n"))) <= budgetTotal) {
lines.push(uriLine);
- hintCount++;
+ hintCount += 1;
}
}
-
- lines.push("</openviking-context>");
-
- const budgetUsed = budgetTotal - budgetRemaining;
- log("recall_injection_built", {
- contentItems: contentCount,
- hintItems: hintCount,
- budgetUsed,
- budgetTotal,
- });
-
- return { block: lines.join("\n"), contentCount, hintCount, budgetUsed, stage: "ranked" };
+ const budgetUsed = lines.length ? estimateTokens(wrapContext(lines.join("\n"))) : 0;
+ log("recall_injection_built", { contentItems: contentCount, hintItems: hintCount, budgetUsed, budgetTotal });
+ return { rendered: lines.join("\n"), contentCount, hintCount, budgetUsed };
}
const LEGACY_CACHE_TTL_MS = 6 * 60 * 60 * 1000;
@@ -468,10 +473,14 @@ function looksLikeUnknownField(res) {
}
function wrapContext(body) {
+ // Retrieved text cannot introduce a second capture delimiter inside ours.
+ const text = String(body)
+ .replace(/<\/?relevant-memor(?:y|ies)\b[^>]*>/gi, "legacy memory wrapper")
+ .replace(/<\/?openviking-context\b[^>]*>/gi, "openviking context marker");
return [
"<openviking-context>",
"Relevant memory from OpenViking. Use the search/read MCP tools to expand URIs.",
- body,
+ text,
"</openviking-context>",
].join("\n");
}
@@ -487,7 +496,7 @@ export async function buildServerAssembledBlock(fetchJSON, cfg, query, options =
const block = await recallViaContextFace(fetchJSON, cfg, query, { ...options, actorPeerId }, log);
if (block !== null) return block;
- return recallViaEndpoint(fetchJSON, cfg, query, actorPeerId, log);
+ return recallViaEndpoint(fetchJSON, cfg, query, { ...options, actorPeerId }, log);
}
/**
@@ -551,53 +560,65 @@ export function normalizeContextEntry(entry = {}) {
};
}
-async function recallViaContextFace(fetchJSON, cfg, query, options, log) {
- const assembled = await fetchAssembledContext(fetchJSON, cfg, query, { ...options, log });
- if (assembled === null) return null;
+/** Server relevance decisions and digest precedence are shared by every host. */
+export function selectRecallContent(result = {}) {
+ if (String(result.stats?.rewrite || "").toLowerCase() === "no_relevant") return "";
+ return String(result.digest || "").trim() || String(result.rendered || "").trim();
+}
- const { rendered, entries } = assembled;
- let digest = assembled.digest;
+function wantsLocalCompression(cfg, options) {
const mode = String(cfg.recallRewrite || "off").toLowerCase();
- if (String(assembled.stats?.rewrite || "").toLowerCase() === "no_relevant") {
- log("recall_server_compression", { status: "empty" });
- return "";
- }
- const wantsLocal = mode === "client" || (mode === "auto" && !digest);
- if (wantsLocal && rendered && typeof options.runCompressor === "function") {
- try {
- const compression = await compressRecallContext({
- query,
- rendered,
- entries,
- cfg,
- runCompressor: options.runCompressor,
- cachePath: options.digestCachePath || stateFile("recall-digest.json"),
- now: Date.now(),
- });
- log("recall_local_compression", { status: compression.status });
- if (compression.status === "ok") digest = compression.context;
- if (compression.status === "empty") return "";
- } catch (err) {
- log("recall_local_compression_failed", { error: String(err?.message || err) });
- }
+ return (mode === "client" || mode === "auto")
+ && options.localCompressorAvailable !== false
+ && typeof options.runCompressor === "function";
+}
+
+// Context, legacy recall, and raw retrieval share the same result policy.
+// The host callback only runs its model; it never chooses a fallback or digest.
+async function finalizeRecall(result, cfg, query, options, log, shortContext) {
+ const selected = selectRecallContent(result);
+ if (String(result.stats?.rewrite || "").toLowerCase() === "no_relevant") return "";
+ const entries = (result.entries || []).map(normalizeContextEntry).filter((entry) => entry.uri);
+ const rendered = String(result.rendered || "").trim();
+ if (String(result.digest || "").trim() || !wantsLocalCompression(cfg, options)) return selected;
+ const fallback = shortContext ?? (entries.length ? formatFallback(entries, cfg).rendered : rendered);
+ const input = rendered || entries.map((entry) => `${entry.uri}\n${entry.text}`).join("\n");
+ if (!input) return "";
+ try {
+ const compression = await compressRecallContext({
+ query, rendered: input, shortContext: shortContext ?? (rendered || fallback),
+ entries, cfg, runCompressor: options.runCompressor,
+ cachePath: options.digestCachePath || stateFile("recall-digest.json"), now: Date.now(),
+ });
+ log("recall_local_compression", { status: compression.status });
+ if (compression.status === "ok") return compression.context;
+ if (compression.status === "empty") return "";
+ } catch (err) {
+ log("recall_local_compression_failed", { error: String(err?.message || err) });
}
+ return fallback;
+}
- const injected = digest || rendered;
- if (!injected) return "";
- return wrapContext(injected);
+async function recallViaContextFace(fetchJSON, cfg, query, options, log) {
+ const assembled = await fetchAssembledContext(fetchJSON, cfg, query, { ...options, log });
+ if (assembled === null) return null;
+ const text = await finalizeRecall(assembled, cfg, query, options, log);
+ return text ? wrapContext(text) : "";
}
-async function recallViaEndpoint(fetchJSON, cfg, query, actorPeerId = "", log = () => {}) {
+async function recallViaEndpoint(fetchJSON, cfg, query, options, log) {
const body = buildRecallEndpointBody(cfg);
body.query = query;
- const res = await postRecall(fetchJSON, body, { actorPeerId, log });
+ if (wantsLocalCompression(cfg, options)) {
+ body.max_chars = Math.max(1000, Number(cfg.recallCompressMaxInputChars || 18000));
+ }
+ const res = await postRecall(fetchJSON, body, { actorPeerId: options.actorPeerId, log });
if (!res.ok) {
log("recall_endpoint_fallback", { status: res.status || 0 });
return null;
}
- const rendered = String(res.result?.rendered || "").trim();
- if (!rendered) return "";
- return wrapContext(rendered);
+ const text = await finalizeRecall(res.result || {}, cfg, query, options, log);
+ return text ? wrapContext(text) : "";
}
export async function postRecall(fetchJSON, body, opts = {}) {
@@ -710,7 +731,7 @@ async function recallForPeer(fetchJSON, cfg, query, options = {}) {
const recallLimit = Math.max(1, Number(cfg.recallLimit || DEFAULT_CONTEXT_LIMIT));
const perSourceLimit = Math.max(recallLimit * 2, 8);
- const raw = await searchAllSources(fetchJSON, trimmed, perSourceLimit, actorPeerId, log);
+ const raw = await searchAllSources(fetchJSON, cfg, trimmed, perSourceLimit, { ...options, actorPeerId }, log);
if (raw.length === 0) return emptyRecall("no_results");
const profile = buildQueryProfile(trimmed);
@@ -726,5 +747,25 @@ async function recallForPeer(fetchJSON, cfg, query, options = {}) {
});
if (picked.length === 0) return emptyRecall("filtered_out");
- return buildFallbackInjectionBlock(fetchJSON, picked, cfg, actorPeerId, log);
+ const local = wantsLocalCompression(cfg, options);
+ const inputLimit = Math.max(1000, Number(cfg.recallCompressMaxInputChars || 18000));
+ const contentCfg = local ? { ...cfg, recallPreferAbstract: false, recallMaxContentChars: inputLimit } : cfg;
+ const entries = await Promise.all(picked.map(async (item) => ({
+ ...item, text: await resolveItemContent(fetchJSON, item, contentCfg, actorPeerId),
+ })));
+ const fallback = formatFallback(entries, cfg, log);
+ let text = await finalizeRecall({
+ rendered: local ? entries.map((item) => `${item.uri}\n${item.text}`).join("\n").slice(0, inputLimit) : fallback.rendered,
+ entries,
+ }, cfg, trimmed, options, log, fallback.rendered);
+ if (estimateTokens(wrapContext(text)) > Math.max(200, Number(cfg.recallTokenBudget || 2000))) {
+ text = fallback.rendered;
+ }
+ if (!text) return emptyRecall("no_results");
+ const unchanged = text === fallback.rendered;
+ return {
+ block: wrapContext(text), contentCount: unchanged ? fallback.contentCount : 1,
+ hintCount: unchanged ? fallback.hintCount : 0,
+ budgetUsed: estimateTokens(wrapContext(text)), stage: "ranked",
+ };
}
diff --git a/examples/memory-plugin-shared/plugin-config.test.mjs b/examples/memory-plugin-shared/plugin-config.test.mjs
index 864c13a47..689dbb4a0 100644
--- a/examples/memory-plugin-shared/plugin-config.test.mjs
+++ b/examples/memory-plugin-shared/plugin-config.test.mjs
@@ -5,6 +5,7 @@ import { join } from "node:path";
import test from "node:test";
import { HARNESS_KEYS, KNOBS } from "./lib/config-schema.mjs";
+import { buildContextSearchBody } from "./lib/recall-core.mjs";
import { buildPluginConfig } from "./lib/plugin-config.mjs";
import { loadAgentHookConfig } from "./lib/agent-hook-runtime.mjs";
import { loadConfig as loadClaudeCode } from "../claude-code-memory-plugin/scripts/config.mjs";
@@ -29,7 +30,7 @@ const LOADERS = {
harness: "codex",
load: (cwd) => loadCodex(cwd),
options: { manifestUrl: new URL("../codex-memory-plugin/.codex-plugin/plugin.json", import.meta.url), logFile: "codex-hooks.log" },
- owns: ["recallCompress"],
+ owns: [],
},
opencode: {
harness: "opencode",
@@ -349,3 +350,18 @@ test("ovcli.conf's plugin section outranks ov.conf, under ovcli.conf's own key",
rmSync(dir, { recursive: true, force: true });
}
});
+
+for (const [name, loader] of Object.entries(LOADERS)) {
+ test(`${name} explicit cloud compression reaches the shared HTTP contract`, () => {
+ withFixture({}, ({otherDir}) => {
+ process.env.OPENVIKING_RECALL_COMPRESS = "server";
+ const cfg = loader.load(otherDir);
+ assert.equal(cfg.recallRewrite, "server");
+ assert.equal(buildContextSearchBody(cfg).rewrite, true);
+ process.env.OPENVIKING_RECALL_COMPRESS = "auto";
+ assert.equal(buildContextSearchBody(loader.load(otherDir), { localCompressorAvailable: false }).rewrite, "auto");
+ process.env.OPENVIKING_RECALL_COMPRESS = "off";
+ assert.equal(buildContextSearchBody(loader.load(otherDir)).rewrite, undefined);
+ });
+ });
+}
diff --git a/examples/memory-plugin-shared/recall-core.test.mjs b/examples/memory-plugin-shared/recall-core.test.mjs
index e3565d5f2..8cb45c217 100644
--- a/examples/memory-plugin-shared/recall-core.test.mjs
+++ b/examples/memory-plugin-shared/recall-core.test.mjs
@@ -452,9 +452,12 @@ test("the ranked fallback reports what fitted the budget and what degraded", asy
}, "what happened", { legacyCachePath: await tempPath("context-face.json") });
assert.equal(detailed.stage, "ranked");
- assert.equal(detailed.contentCount, 1);
- assert.equal(detailed.hintCount, 1);
- assert.equal(detailed.budgetUsed, estimateTokens(`- [memory 90%] ${"x".repeat(500)}`));
+ const visibleContent = ["x".repeat(500), "y".repeat(500)].filter((text) => detailed.block.includes(text));
+ const citedUris = [...detailed.block.matchAll(/viking:\/\/[^\s<>]+/g)].map((match) => match[0]);
+ assert.equal(detailed.contentCount, visibleContent.length);
+ assert.equal(detailed.hintCount, citedUris.length - visibleContent.length);
+ assert.equal(detailed.budgetUsed, estimateTokens(detailed.block));
+ assert.ok(detailed.budgetUsed <= 200, "budget includes the wrapper, citations, and body");
});
test("an empty recall says whether the server had nothing or the threshold took it", async () => {
@@ -469,3 +472,171 @@ test("an empty recall says whether the server had nothing or the threshold took
assert.equal(belowThreshold.stage, "filtered_out");
assert.equal(await buildRecallBlock(fallbackFetch([]), {}, "hello", { legacyCachePath }), null);
});
+
+for (const source of ["context", "legacy"]) {
+ test(`${source} recall keeps a server digest in client mode without recompressing`, async () => {
+ const paths = [];
+ let compressorCalls = 0;
+ const recalled = await buildRecallBlockDetailed(async (path) => {
+ paths.push(path);
+ if (source === "legacy" && path === "/api/v1/search/search") return { ok: false, status: 404 };
+ return { ok: true, result: {
+ rendered: "RAW_CONTENT_SHOULD_NOT_APPEAR",
+ digest: "- server fact viking://user/default/memories/a.md",
+ entries: [{ uri: "viking://user/default/memories/a.md", text: "raw fact" }],
+ stats: { rewrite: "ok" },
+ } };
+ }, { recallRewrite: "client", recallCompressMinInputChars: 0 }, "remember fact", {
+ legacyCachePath: await tempPath("context-face.json"),
+ digestCachePath: await tempPath("digest.json"),
+ localCompressorAvailable: true,
+ runCompressor: async () => { compressorCalls++; return "- unwanted local rewrite"; },
+ });
+ assert.equal(compressorCalls, 0);
+ assert.match(recalled.block, /server fact/);
+ assert.doesNotMatch(recalled.block, /RAW_CONTENT_SHOULD_NOT_APPEAR/);
+ assert.equal(paths.length, source === "context" ? 1 : 2);
+ });
+
+ test(`${source} no_relevant neither compresses nor falls back to raw entries`, async () => {
+ const paths = [];
+ let compressorCalls = 0;
+ const recalled = await buildRecallBlockDetailed(async (path) => {
+ paths.push(path);
+ if (source === "legacy" && path === "/api/v1/search/search") return { ok: false, status: 404 };
+ return { ok: true, result: {
+ rendered: "RAW_IRRELEVANT_CONTENT",
+ entries: [{ uri: "viking://user/default/memories/a.md", text: "raw fact" }],
+ stats: { rewrite: "no_relevant" },
+ } };
+ }, { recallRewrite: "client", recallCompressMinInputChars: 0 }, "unrelated question", {
+ legacyCachePath: await tempPath("context-face.json"),
+ localCompressorAvailable: true,
+ runCompressor: async () => { compressorCalls++; return "- unwanted local rewrite"; },
+ });
+ assert.equal(compressorCalls, 0);
+ assert.equal(recalled.block, "");
+ assert.equal(paths.length, source === "context" ? 1 : 2);
+ });
+}
+
+test("legacy recall feeds normalized entries to the shared local compressor", async () => {
+ const uri = "viking://user/default/memories/editor.md";
+ const prompts = [];
+ const recalled = await buildRecallBlockDetailed(async (path) => {
+ if (path === "/api/v1/search/search") return { ok: false, status: 404 };
+ assert.equal(path, "/api/v1/search/recall");
+ return { ok: true, result: {
+ rendered: `LEGACY_FULL_CONTENT ${uri}`,
+ entries: [{ uri, type: "preferences", mode: "summary", summary: "Vim" }],
+ } };
+ }, { recallRewrite: "client", recallCompressMinInputChars: 0 }, "preferred editor", {
+ legacyCachePath: await tempPath("context-face.json"),
+ digestCachePath: await tempPath("digest.json"),
+ localCompressorAvailable: true,
+ runCompressor: async (prompt) => { prompts.push(prompt); return `- Choose Vim ${uri}`; },
+ });
+ assert.equal(prompts.length, 1);
+ assert.match(prompts[0], /LEGACY_FULL_CONTENT/);
+ assert.match(recalled.block, /Choose Vim/);
+ assert.match(recalled.block, /viking:\/\/user\/default\/memories\/editor\.md/);
+});
+
+test("raw fallback compresses bounded full content before injection truncation", async () => {
+ const uri = "viking://user/default/memories/detail.md";
+ const prompts = [];
+ const recalled = await buildRecallBlockDetailed(async (path, init) => {
+ if (path === "/api/v1/search/search" || path === "/api/v1/search/recall") return { ok: false, status: 404 };
+ if (path === "/api/v1/search/find") {
+ const body = JSON.parse(init.body);
+ return { ok: true, result: { memories: body.target_uri.endsWith("/memories")
+ ? [{ uri, level: 2, score: 0.9, abstract: "short summary" }] : [], skills: [] } };
+ }
+ if (path.startsWith("/api/v1/content/read?")) return { ok: true,
+ result: `${"x".repeat(320)}DETAIL_AFTER_INJECTION_LIMIT${"y".repeat(5000)}OUTSIDE_COMPRESSION_BUDGET` };
+ assert.fail(`unexpected request ${path}`);
+ }, {
+ recallRewrite: "client", recallCompressMinInputChars: 0,
+ recallMaxContentChars: 80, recallCompressMaxInputChars: 1200, recallTokenBudget: 200,
+ }, "use the full detail", {
+ legacyCachePath: await tempPath("context-face.json"),
+ digestCachePath: await tempPath("digest.json"),
+ localCompressorAvailable: true,
+ runCompressor: async (prompt) => { prompts.push(prompt); return `- retained detail ${uri}`; },
+ });
+ assert.equal(prompts.length, 1);
+ assert.match(prompts[0], /DETAIL_AFTER_INJECTION_LIMIT/);
+ assert.doesNotMatch(prompts[0], /OUTSIDE_COMPRESSION_BUDGET/);
+ assert.match(recalled.block, /retained detail/);
+});
+
+for (const user of ["zeus", "default"]) {
+ test(`raw session recall retries without session and keeps explicit ${user} then home alias`, async () => {
+ const requests = [];
+ const uri = `viking://user/${user}/memories/fact.md`;
+ const recalled = await buildRecallBlockDetailed(async (path, init) => {
+ const body = init?.body ? JSON.parse(init.body) : {};
+ if (path === "/api/v1/search/recall" || body.mode === "context") return { ok: false, status: 404 };
+ assert.equal(path, "/api/v1/search/search");
+ requests.push(body);
+ const hit = !body.session_id && body.target_uri === "viking://~/memories";
+ return { ok: true, result: { memories: hit
+ ? [{ uri, score: 0.9, level: 1, abstract: "recovered memory" }] : [], skills: [] } };
+ }, { user, recallPreferAbstract: true }, "recover the fact", {
+ sessionId: "cx-shared-session", legacyCachePath: await tempPath("context-face.json"),
+ });
+ const memoryRequests = requests.filter((body) => body.target_uri.endsWith("/memories"));
+ assert.deepEqual(memoryRequests.map((body) => [body.target_uri, body.session_id || ""]), [
+ [`viking://user/${user}/memories`, "cx-shared-session"],
+ ["viking://~/memories", "cx-shared-session"],
+ [`viking://user/${user}/memories`, ""],
+ ["viking://~/memories", ""],
+ ]);
+ assert.match(recalled.block, /recovered memory/);
+ });
+}
+
+test("raw session recall uses find when the search endpoint is unsupported", async () => {
+ const requests = [];
+ const recalled = await buildRecallBlockDetailed(async (path, init) => {
+ const body = init?.body ? JSON.parse(init.body) : {};
+ if (body.mode === "context" || path === "/api/v1/search/recall") return { ok: false, status: 404 };
+ requests.push({ path, body });
+ if (path === "/api/v1/search/search") return { ok: false, status: 405 };
+ assert.equal(path, "/api/v1/search/find");
+ return { ok: true, result: { memories: body.target_uri.endsWith("/memories")
+ ? [{ uri: "viking://user/default/memories/a.md", score: 0.9, level: 1, abstract: "old server fact" }]
+ : [], skills: [] } };
+ }, { recallPreferAbstract: true }, "old server fact", {
+ sessionId: "cx-old-server", legacyCachePath: await tempPath("context-face.json"),
+ });
+ assert.ok(requests.some(({ path, body }) => path.endsWith("/search") && body.session_id === "cx-old-server"));
+ assert.ok(requests.some(({ path }) => path.endsWith("/find")));
+ assert.match(recalled.block, /old server fact/);
+});
+
+
+for (const textLength of [20, 500]) {
+ test(`raw fallback charges long URI citations and ${textLength}-character bodies to its whole-block budget`, async () => {
+ const items = Array.from({ length: 10 }, (_, index) => ({
+ uri: `viking://user/default/memories/${"long-path/".repeat(40)}${index}.md`,
+ abstract: `body-${index}-${"x".repeat(textLength)}`,
+ score: 0.9,
+ level: 1,
+ }));
+ const recalled = await buildRecallBlockDetailed(fallbackFetch(items), {
+ recallTokenBudget: 200, recallLimit: 10, recallMaxContentChars: 1000,
+ recallPreferAbstract: true,
+ }, "use remembered details", { legacyCachePath: await tempPath("context-face.json") });
+ assert.ok(recalled.block, "at least one complete URI fits the budget");
+ assert.ok(estimateTokens(recalled.block) <= 200, "URI-only hints cannot bypass the budget");
+ assert.equal(recalled.budgetUsed, estimateTokens(recalled.block));
+ const citedUris = [...recalled.block.matchAll(/viking:\/\/[^\s<>]+/g)].map((match) => match[0]);
+ const visibleContent = items.filter((item) => recalled.block.includes(item.abstract));
+ assert.ok(citedUris.length < items.length, "entries that do not fit must be omitted");
+ assert.ok(citedUris.every((uri) => items.some((item) => item.uri === uri)), "never truncate a URI to make it fit");
+ assert.ok(visibleContent.every((item) => citedUris.includes(item.uri)), "every included body retains its full source URI");
+ assert.equal(recalled.contentCount, visibleContent.length);
+ assert.equal(recalled.hintCount, citedUris.length - visibleContent.length);
+ });
+}
diff --git a/examples/openclaw-plugin/README.md b/examples/openclaw-plugin/README.md
index 290612a9a..7dfe99b04 100644
--- a/examples/openclaw-plugin/README.md
+++ b/examples/openclaw-plugin/README.md
@@ -385,3 +385,14 @@ ov tui
For installation, upgrade, and uninstall operations, use [INSTALL.md](./INSTALL.md).
</details>
+
+
+### Cloud recall compression
+
+Set `OPENVIKING_RECALL_COMPRESS=server` to enable cloud recall compression, or
+`auto` to let the server decide whether to rewrite. Both use search
+`mode=context`; `server` sends `rewrite=true`, and `auto` sends `rewrite="auto"`.
+The server digest takes precedence over raw rendered context, and `no_relevant`
+suppresses injection. The default remains `off`; no local compressor is launched.
+
+The plugin config equivalent is `recallCompress: "server"`.
diff --git a/examples/openclaw-plugin/auto-recall.ts b/examples/openclaw-plugin/auto-recall.ts
index a00f7ec29..fa1f80bff 100644
--- a/examples/openclaw-plugin/auto-recall.ts
+++ b/examples/openclaw-plugin/auto-recall.ts
@@ -12,6 +12,7 @@ import {
type RecallTraceResult,
} from "./recall-trace.js";
import { sanitizeUserTextForCapture } from "./text-utils.js";
+import { selectRecallContent } from "./shared/recall-core.mjs";
import { estimateTextTokens } from "./token-estimator.js";
const RECALL_QUERY_MAX_CHARS = 4_000;
@@ -369,6 +370,7 @@ export async function buildAutoRecallContext(params: {
contextResult = await client.searchContext(queryText, {
sessionId: params.ovSessionId,
limit: recallLimit,
+ recallCompress: cfg.recallCompress,
scoreThreshold,
contextType: contextTypes.length === 1 ? contextTypes[0] : contextTypes,
queryExpansion: "auto",
@@ -475,14 +477,15 @@ export async function buildAutoRecallContext(params: {
params.traceRecorder?.record(entry);
};
- const rendered = contextResult?.rendered?.trim() ?? "";
- if (requestError || entries.length === 0 || !rendered) {
+ const digest = contextResult?.digest?.trim() ?? "";
+ const rendered = selectRecallContent(contextResult);
+ if (requestError || !rendered) {
await recordTrace([], 0, 0);
return { memoryCount: 0, estimatedTokens: 0 };
}
const usedTokens = contextResult?.stats?.used_tokens;
- const estimatedTokens = typeof usedTokens === "number" && Number.isFinite(usedTokens)
+ const estimatedTokens = !digest && typeof usedTokens === "number" && Number.isFinite(usedTokens)
? Math.max(0, Math.ceil(usedTokens))
: estimateTokenCount(rendered);
const block = buildRecallContextBlock([rendered]);
diff --git a/examples/openclaw-plugin/client.ts b/examples/openclaw-plugin/client.ts
index ebd8f7d66..6860cd1a9 100644
--- a/examples/openclaw-plugin/client.ts
+++ b/examples/openclaw-plugin/client.ts
@@ -48,6 +48,7 @@ export type SearchContextOptions = {
scoreThreshold?: number;
contextType?: string | string[];
queryExpansion?: "off" | "auto";
+ recallCompress?: "off" | "server" | "auto";
maxTokens?: number;
detail?: "abstract" | "overview" | "full";
dedupTurns?: number;
@@ -496,6 +497,7 @@ export class OpenVikingClient {
): Promise<SearchContextResult> {
const contractConfig = {
recallLimit: options.limit,
+ recallRewrite: options.recallCompress,
recallLimitConfigured: options.limit !== undefined,
recallMaxTokens: options.maxTokens,
recallMaxTokensConfigured: options.maxTokens !== undefined,
diff --git a/examples/openclaw-plugin/config.ts b/examples/openclaw-plugin/config.ts
index dc9cc4469..725a2ae3b 100644
--- a/examples/openclaw-plugin/config.ts
+++ b/examples/openclaw-plugin/config.ts
@@ -48,6 +48,8 @@ export type MemoryOpenVikingConfig = {
autoRecall?: boolean;
/** Outer time budget for the whole server-assembled auto-recall flow. */
autoRecallTimeoutMs?: number;
+ /** Cloud recall compression; no local compressor is shipped. Default off. */
+ recallCompress?: "off" | "server" | "auto";
/** Include resources in auto-recall and default memory_recall search. Default false. */
recallResources?: boolean;
recallLimit?: number;
@@ -531,6 +533,7 @@ export const memoryOpenVikingConfigSchema = {
"captureMaxLength",
"autoRecall",
"autoRecallTimeoutMs",
+ "recallCompress",
"recallResources",
"recallLimit",
"recallScoreThreshold",
@@ -628,6 +631,12 @@ export const memoryOpenVikingConfigSchema = {
),
),
);
+ const rawCompress = String(getEnv("OPENVIKING_RECALL_COMPRESS") ?? cfg.recallCompress ?? "off").trim().toLowerCase();
+ const recallCompress = ["1", "true", "yes"].includes(rawCompress) ? "auto"
+ : ["0", "false", "no"].includes(rawCompress) ? "off" : rawCompress;
+ if (recallCompress !== "off" && recallCompress !== "server" && recallCompress !== "auto") {
+ throw new Error("openviking recallCompress must be server, auto or off (no local compressor)");
+ }
const recallResources = cfg.recallResources === true || envFlag("OPENVIKING_RECALL_RESOURCES");
const recallTargetTypes = normalizeRecallTargetTypes(
cfg.recallTargetTypes,
@@ -751,6 +760,7 @@ export const memoryOpenVikingConfigSchema = {
traceRecallIncludeContentByDefault: cfg.traceRecallIncludeContentByDefault === true,
traceRecallIncludeRawUserPreview: cfg.traceRecallIncludeRawUserPreview === true,
recallTargetTypes,
+ recallCompress,
enableAddResourceTool: cfg.enableAddResourceTool === true,
enabledTools,
disabledTools,
diff --git a/examples/openclaw-plugin/install-manifest.json b/examples/openclaw-plugin/install-manifest.json
index de6eee496..13a5b0ae3 100644
--- a/examples/openclaw-plugin/install-manifest.json
+++ b/examples/openclaw-plugin/install-manifest.json
@@ -1,6 +1,6 @@
{
"manifestVersion": "1.0",
- "pluginVersion": "2026.6.18",
+ "pluginVersion": "2026.9.21",
"plugin": {
"id": "openviking",
"kind": "context-engine",
diff --git a/examples/openclaw-plugin/openclaw.plugin.json b/examples/openclaw-plugin/openclaw.plugin.json
index 67457bffe..f76b6b1df 100644
--- a/examples/openclaw-plugin/openclaw.plugin.json
+++ b/examples/openclaw-plugin/openclaw.plugin.json
@@ -272,6 +272,12 @@
"type": "object",
"additionalProperties": false,
"properties": {
+ "recallCompress": {
+ "type": "string",
+ "enum": ["off", "server", "auto"],
+ "default": "off",
+ "description": "Cloud recall compression. auto delegates to the server; off preserves raw context."
+ },
"mode": {
"type": "string",
"enum": ["local", "remote"],
diff --git a/examples/openclaw-plugin/package-lock.json b/examples/openclaw-plugin/package-lock.json
index b9f0dde30..adb5f4320 100644
--- a/examples/openclaw-plugin/package-lock.json
+++ b/examples/openclaw-plugin/package-lock.json
@@ -1,12 +1,12 @@
{
"name": "@openviking/openclaw-plugin",
- "version": "2026.6.18",
+ "version": "2026.9.21",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@openviking/openclaw-plugin",
- "version": "2026.6.18",
+ "version": "2026.9.21",
"license": "MIT",
"dependencies": {
"@sinclair/typebox": "0.34.48",
diff --git a/examples/openclaw-plugin/package.json b/examples/openclaw-plugin/package.json
index f821bc6bb..fba361244 100644
--- a/examples/openclaw-plugin/package.json
+++ b/examples/openclaw-plugin/package.json
@@ -1,6 +1,6 @@
{
"name": "@openviking/openclaw-plugin",
- "version": "2026.6.18",
+ "version": "2026.9.21",
"displayName": "OpenViking",
"description": "OpenClaw context-engine plugin — OpenViking AI memory, long-term memory, knowledge base, semantic search, RAG, context engine. 长期记忆 · 知识库 · 语义检索 · 上下文引擎",
"type": "module",
diff --git a/examples/openclaw-plugin/shared/recall-core.d.mts b/examples/openclaw-plugin/shared/recall-core.d.mts
index ed28bc750..490b6901b 100644
--- a/examples/openclaw-plugin/shared/recall-core.d.mts
+++ b/examples/openclaw-plugin/shared/recall-core.d.mts
@@ -52,6 +52,10 @@ export type RecallOptions = {
legacyPeerId?: string;
sessionId?: string;
log?: (stage: string, data?: any) => void;
+ localCompressorAvailable?: boolean;
+ runCompressor?: ((prompt: string) => Promise<string | null>) | null;
+ digestCachePath?: string;
+ legacyCachePath?: string;
};
export type DetailedRecall = {
@@ -79,3 +83,5 @@ export function buildRecallBlock(
export function buildRecallEndpointBody(cfg?: Record<string, any>): Record<string, any>;
export function estimateTokens(text: string): number;
export function isRecallEnabled(cfg?: Record<string, any>): boolean;
+
+export function selectRecallContent(result?: { rendered?: string; digest?: string; stats?: Record<string, unknown> }): string;
diff --git a/examples/openclaw-plugin/shared/recall-core.mjs b/examples/openclaw-plugin/shared/recall-core.mjs
index 4695a887b..b893078b0 100644
--- a/examples/openclaw-plugin/shared/recall-core.mjs
+++ b/examples/openclaw-plugin/shared/recall-core.mjs
@@ -309,21 +309,46 @@ async function resolveTargetUri(fetchJSON, targetUri, actorPeerId = "") {
return `viking://user/${space}/${parts.join("/")}`;
}
-async function searchOneSource(fetchJSON, query, source, limit, actorPeerId = "") {
- const resolvedUri = await resolveTargetUri(fetchJSON, source.uri, actorPeerId);
- const body = { query, target_uri: resolvedUri, limit, score_threshold: 0 };
- const res = await fetchJSON("/api/v1/search/find", {
- method: "POST",
- body: JSON.stringify(body),
- }, { actorPeerId });
- if (!res.ok) return [];
- const items = res.result?.[source.bucket] || [];
- return items.map((item) => ({ ...item, _sourceType: source.type }));
+async function searchOneSource(fetchJSON, cfg, query, source, limit, options) {
+ const actorPeerId = options.actorPeerId || "";
+ const home = await resolveTargetUri(fetchJSON, source.uri, actorPeerId);
+ const targets = [...new Set(cfg.user
+ ? [`viking://user/${cfg.user}/${source.bucket}`, home] : [home])];
+ const sessionId = String(options.sessionId || "").trim();
+ const search = async (target, session) => {
+ const body = { query, target_uri: target, limit, score_threshold: 0 };
+ if (session) body.session_id = session;
+ const init = { method: "POST", body: JSON.stringify(body) };
+ // Session-aware retrieval may decide no context is needed. Only after all
+ // targets are empty do we retry without the session. Old servers still
+ // support find; an unsupported search route can fall back to it.
+ let res = await fetchJSON(sessionId ? "/api/v1/search/search" : "/api/v1/search/find", init, { actorPeerId });
+ if (sessionId && !res.ok && (
+ !res.status || res.status === 404 || res.status === 405 || res.status >= 500
+ || ((res.status === 400 || res.status === 422) && looksLikeUnknownField(res))
+ )) {
+ const { session_id, ...findBody } = body;
+ res = await fetchJSON("/api/v1/search/find", { method: "POST", body: JSON.stringify(findBody) }, { actorPeerId });
+ }
+ const items = res.ok && Array.isArray(res.result?.[source.bucket]) ? res.result[source.bucket] : [];
+ return items.map((item) => ({ ...item, _sourceType: source.type }));
+ };
+ for (const target of targets) {
+ const items = await search(target, sessionId);
+ if (items.length) return items;
+ }
+ if (sessionId) {
+ for (const target of targets) {
+ const items = await search(target, "");
+ if (items.length) return items;
+ }
+ }
+ return [];
}
-async function searchAllSources(fetchJSON, query, perSourceLimit, actorPeerId = "", log = () => {}) {
+async function searchAllSources(fetchJSON, cfg, query, perSourceLimit, options, log = () => {}) {
const results = await Promise.all(
- SOURCES.map((src) => searchOneSource(fetchJSON, query, src, perSourceLimit, actorPeerId)),
+ SOURCES.map((src) => searchOneSource(fetchJSON, cfg, query, src, perSourceLimit, options)),
);
const all = results.flat();
log("recall_search_summary", {
@@ -359,50 +384,30 @@ async function resolveItemContent(fetchJSON, item, cfg, actorPeerId = "") {
return content;
}
-async function buildFallbackInjectionBlock(fetchJSON, items, cfg, actorPeerId = "", log = () => {}) {
+function formatFallback(items, cfg, log = () => {}) {
const budgetTotal = Math.max(200, Number(cfg.recallTokenBudget || 2000));
- let budgetRemaining = budgetTotal;
- const lines = [
- "<openviking-context>",
- "Relevant context from OpenViking. Use the read MCP tool to expand URIs.",
- ];
+ const lines = [];
let contentCount = 0;
let hintCount = 0;
-
+ const maxChars = Math.max(50, Number(cfg.recallMaxContentChars || 500));
for (const item of items) {
const score = (clampScore(item.score) * 100).toFixed(0);
- const uriLine = `- [${item._sourceType} ${score}%] ${item.uri}`;
-
- if (budgetRemaining > 0) {
- const content = await resolveItemContent(fetchJSON, item, cfg, actorPeerId);
- const contentLine = `- [${item._sourceType} ${score}%] ${content}`;
- const lineTokens = estimateTokens(contentLine);
-
- if (lineTokens > budgetRemaining && contentCount > 0) {
- lines.push(uriLine);
- hintCount++;
- } else {
- lines.push(contentLine);
- budgetRemaining -= lineTokens;
- contentCount++;
- }
- } else {
+ const kind = item._sourceType || item.category || "memory";
+ const uriLine = `- [${kind} ${score}%] ${item.uri}`;
+ const text = String(item.text || item.uri);
+ const content = text.length > maxChars ? `${text.slice(0, maxChars)}...` : text;
+ const contentLine = `- [${kind} ${score}%] ${content}`;
+ if (estimateTokens(wrapContext([...lines, uriLine, contentLine].join("\n"))) <= budgetTotal) {
+ lines.push(uriLine, contentLine);
+ contentCount += 1;
+ } else if (estimateTokens(wrapContext([...lines, uriLine].join("\n"))) <= budgetTotal) {
lines.push(uriLine);
- hintCount++;
+ hintCount += 1;
}
}
-
- lines.push("</openviking-context>");
-
- const budgetUsed = budgetTotal - budgetRemaining;
- log("recall_injection_built", {
- contentItems: contentCount,
- hintItems: hintCount,
- budgetUsed,
- budgetTotal,
- });
-
- return { block: lines.join("\n"), contentCount, hintCount, budgetUsed, stage: "ranked" };
+ const budgetUsed = lines.length ? estimateTokens(wrapContext(lines.join("\n"))) : 0;
+ log("recall_injection_built", { contentItems: contentCount, hintItems: hintCount, budgetUsed, budgetTotal });
+ return { rendered: lines.join("\n"), contentCount, hintCount, budgetUsed };
}
const LEGACY_CACHE_TTL_MS = 6 * 60 * 60 * 1000;
@@ -469,10 +474,14 @@ function looksLikeUnknownField(res) {
}
function wrapContext(body) {
+ // Retrieved text cannot introduce a second capture delimiter inside ours.
+ const text = String(body)
+ .replace(/<\/?relevant-memor(?:y|ies)\b[^>]*>/gi, "legacy memory wrapper")
+ .replace(/<\/?openviking-context\b[^>]*>/gi, "openviking context marker");
return [
"<openviking-context>",
"Relevant memory from OpenViking. Use the search/read MCP tools to expand URIs.",
- body,
+ text,
"</openviking-context>",
].join("\n");
}
@@ -488,7 +497,7 @@ export async function buildServerAssembledBlock(fetchJSON, cfg, query, options =
const block = await recallViaContextFace(fetchJSON, cfg, query, { ...options, actorPeerId }, log);
if (block !== null) return block;
- return recallViaEndpoint(fetchJSON, cfg, query, actorPeerId, log);
+ return recallViaEndpoint(fetchJSON, cfg, query, { ...options, actorPeerId }, log);
}
/**
@@ -552,53 +561,65 @@ export function normalizeContextEntry(entry = {}) {
};
}
-async function recallViaContextFace(fetchJSON, cfg, query, options, log) {
- const assembled = await fetchAssembledContext(fetchJSON, cfg, query, { ...options, log });
- if (assembled === null) return null;
+/** Server relevance decisions and digest precedence are shared by every host. */
+export function selectRecallContent(result = {}) {
+ if (String(result.stats?.rewrite || "").toLowerCase() === "no_relevant") return "";
+ return String(result.digest || "").trim() || String(result.rendered || "").trim();
+}
- const { rendered, entries } = assembled;
- let digest = assembled.digest;
+function wantsLocalCompression(cfg, options) {
const mode = String(cfg.recallRewrite || "off").toLowerCase();
- if (String(assembled.stats?.rewrite || "").toLowerCase() === "no_relevant") {
- log("recall_server_compression", { status: "empty" });
- return "";
- }
- const wantsLocal = mode === "client" || (mode === "auto" && !digest);
- if (wantsLocal && rendered && typeof options.runCompressor === "function") {
- try {
- const compression = await compressRecallContext({
- query,
- rendered,
- entries,
- cfg,
- runCompressor: options.runCompressor,
- cachePath: options.digestCachePath || stateFile("recall-digest.json"),
- now: Date.now(),
- });
- log("recall_local_compression", { status: compression.status });
- if (compression.status === "ok") digest = compression.context;
- if (compression.status === "empty") return "";
- } catch (err) {
- log("recall_local_compression_failed", { error: String(err?.message || err) });
- }
+ return (mode === "client" || mode === "auto")
+ && options.localCompressorAvailable !== false
+ && typeof options.runCompressor === "function";
+}
+
+// Context, legacy recall, and raw retrieval share the same result policy.
+// The host callback only runs its model; it never chooses a fallback or digest.
+async function finalizeRecall(result, cfg, query, options, log, shortContext) {
+ const selected = selectRecallContent(result);
+ if (String(result.stats?.rewrite || "").toLowerCase() === "no_relevant") return "";
+ const entries = (result.entries || []).map(normalizeContextEntry).filter((entry) => entry.uri);
+ const rendered = String(result.rendered || "").trim();
+ if (String(result.digest || "").trim() || !wantsLocalCompression(cfg, options)) return selected;
+ const fallback = shortContext ?? (entries.length ? formatFallback(entries, cfg).rendered : rendered);
+ const input = rendered || entries.map((entry) => `${entry.uri}\n${entry.text}`).join("\n");
+ if (!input) return "";
+ try {
+ const compression = await compressRecallContext({
+ query, rendered: input, shortContext: shortContext ?? (rendered || fallback),
+ entries, cfg, runCompressor: options.runCompressor,
+ cachePath: options.digestCachePath || stateFile("recall-digest.json"), now: Date.now(),
+ });
+ log("recall_local_compression", { status: compression.status });
+ if (compression.status === "ok") return compression.context;
+ if (compression.status === "empty") return "";
+ } catch (err) {
+ log("recall_local_compression_failed", { error: String(err?.message || err) });
}
+ return fallback;
+}
- const injected = digest || rendered;
- if (!injected) return "";
- return wrapContext(injected);
+async function recallViaContextFace(fetchJSON, cfg, query, options, log) {
+ const assembled = await fetchAssembledContext(fetchJSON, cfg, query, { ...options, log });
+ if (assembled === null) return null;
+ const text = await finalizeRecall(assembled, cfg, query, options, log);
+ return text ? wrapContext(text) : "";
}
-async function recallViaEndpoint(fetchJSON, cfg, query, actorPeerId = "", log = () => {}) {
+async function recallViaEndpoint(fetchJSON, cfg, query, options, log) {
const body = buildRecallEndpointBody(cfg);
body.query = query;
- const res = await postRecall(fetchJSON, body, { actorPeerId, log });
+ if (wantsLocalCompression(cfg, options)) {
+ body.max_chars = Math.max(1000, Number(cfg.recallCompressMaxInputChars || 18000));
+ }
+ const res = await postRecall(fetchJSON, body, { actorPeerId: options.actorPeerId, log });
if (!res.ok) {
log("recall_endpoint_fallback", { status: res.status || 0 });
return null;
}
- const rendered = String(res.result?.rendered || "").trim();
- if (!rendered) return "";
- return wrapContext(rendered);
+ const text = await finalizeRecall(res.result || {}, cfg, query, options, log);
+ return text ? wrapContext(text) : "";
}
export async function postRecall(fetchJSON, body, opts = {}) {
@@ -711,7 +732,7 @@ async function recallForPeer(fetchJSON, cfg, query, options = {}) {
const recallLimit = Math.max(1, Number(cfg.recallLimit || DEFAULT_CONTEXT_LIMIT));
const perSourceLimit = Math.max(recallLimit * 2, 8);
- const raw = await searchAllSources(fetchJSON, trimmed, perSourceLimit, actorPeerId, log);
+ const raw = await searchAllSources(fetchJSON, cfg, trimmed, perSourceLimit, { ...options, actorPeerId }, log);
if (raw.length === 0) return emptyRecall("no_results");
const profile = buildQueryProfile(trimmed);
@@ -727,5 +748,25 @@ async function recallForPeer(fetchJSON, cfg, query, options = {}) {
});
if (picked.length === 0) return emptyRecall("filtered_out");
- return buildFallbackInjectionBlock(fetchJSON, picked, cfg, actorPeerId, log);
+ const local = wantsLocalCompression(cfg, options);
+ const inputLimit = Math.max(1000, Number(cfg.recallCompressMaxInputChars || 18000));
+ const contentCfg = local ? { ...cfg, recallPreferAbstract: false, recallMaxContentChars: inputLimit } : cfg;
+ const entries = await Promise.all(picked.map(async (item) => ({
+ ...item, text: await resolveItemContent(fetchJSON, item, contentCfg, actorPeerId),
+ })));
+ const fallback = formatFallback(entries, cfg, log);
+ let text = await finalizeRecall({
+ rendered: local ? entries.map((item) => `${item.uri}\n${item.text}`).join("\n").slice(0, inputLimit) : fallback.rendered,
+ entries,
+ }, cfg, trimmed, options, log, fallback.rendered);
+ if (estimateTokens(wrapContext(text)) > Math.max(200, Number(cfg.recallTokenBudget || 2000))) {
+ text = fallback.rendered;
+ }
+ if (!text) return emptyRecall("no_results");
+ const unchanged = text === fallback.rendered;
+ return {
+ block: wrapContext(text), contentCount: unchanged ? fallback.contentCount : 1,
+ hintCount: unchanged ? fallback.hintCount : 0,
+ budgetUsed: estimateTokens(wrapContext(text)), stage: "ranked",
+ };
}
diff --git a/examples/openclaw-plugin/tests/ut/client.test.ts b/examples/openclaw-plugin/tests/ut/client.test.ts
index 7065155d9..72eeaea75 100644
--- a/examples/openclaw-plugin/tests/ut/client.test.ts
+++ b/examples/openclaw-plugin/tests/ut/client.test.ts
@@ -826,3 +826,15 @@ it("does not report reset success when an older server ignores reset_context", a
const client = new OpenVikingClient("http://127.0.0.1:1933", "", "agent", 5000, "", "", undefined, false, true, { transport });
await expect(client.commitSession("s", { resetContext: true })).rejects.toThrow("did not confirm reset_context");
});
+
+describe("cloud recall compression", () => {
+ it.each([['server', true], ['auto', 'auto'], ['off', undefined]] as const)(
+ "forwards %s and preserves server digest", async (mode, rewrite) => {
+ const transport = vi.fn().mockResolvedValue(okResponse({ entries: [], rendered: 'raw', digest: 'compressed', stats: {} }));
+ const client = new OpenVikingClient('http://127.0.0.1:1933', '', 'agent', 5000, '', '', undefined, false, true, { transport });
+ const result = await client.searchContext('deployment preferences', { recallCompress: mode });
+ expect(JSON.parse(String(transport.mock.calls[0][1].body)).rewrite).toBe(rewrite);
+ expect(result.digest).toBe('compressed');
+ },
+ );
+});
diff --git a/examples/openclaw-plugin/tests/ut/config.test.ts b/examples/openclaw-plugin/tests/ut/config.test.ts
index a04bf0cd9..d196ecde7 100644
--- a/examples/openclaw-plugin/tests/ut/config.test.ts
+++ b/examples/openclaw-plugin/tests/ut/config.test.ts
@@ -539,3 +539,13 @@ describe("memoryOpenVikingConfigSchema.parse() — apiKey SecretRef (#3522)", ()
expect(cfg2.apiKey).toBe("fallback-key");
});
});
+
+describe("cloud compression configuration", () => {
+ afterEach(() => vi.unstubAllEnvs());
+ it("accepts server configuration and environment override", () => {
+ vi.stubEnv('OPENVIKING_RECALL_COMPRESS', undefined);
+ expect(memoryOpenVikingConfigSchema.parse({ recallCompress: 'server' }).recallCompress).toBe('server');
+ vi.stubEnv('OPENVIKING_RECALL_COMPRESS', 'auto');
+ expect(memoryOpenVikingConfigSchema.parse({ recallCompress: 'server' }).recallCompress).toBe('auto');
+ });
+});
diff --git a/examples/openclaw-plugin/tests/ut/context-engine-assemble.test.ts b/examples/openclaw-plugin/tests/ut/context-engine-assemble.test.ts
index fbe58e92e..99e973574 100644
--- a/examples/openclaw-plugin/tests/ut/context-engine-assemble.test.ts
+++ b/examples/openclaw-plugin/tests/ut/context-engine-assemble.test.ts
@@ -929,3 +929,18 @@ describe("context-engine assemble()", () => {
expect(emptyContentMsg).toBeUndefined();
});
});
+
+
+describe("cloud recall digest injection", () => {
+ it.each([false, true])("honors server no_relevant=%s", async (noRelevant) => {
+ const { engine, client } = makeEngine({}, { cfgOverrides: { autoRecall: true, recallCompress: "server" } });
+ client.searchContext.mockResolvedValueOnce({
+ entries: [], rendered: "RAW SHOULD NOT APPEAR", digest: "Prefer Rust for backend tasks.",
+ stats: { rewrite: noRelevant ? "no_relevant" : "rewritten" },
+ });
+ const result = await engine.assemble({ sessionId: "cloud-digest", messages: [{ role: "user", content: "what backend language should we use?" }] });
+ expect(client.searchContext).toHaveBeenCalledWith(expect.any(String), expect.objectContaining({ recallCompress: "server" }));
+ expect(JSON.stringify(result.messages)).not.toContain("RAW SHOULD NOT APPEAR");
+ expect(JSON.stringify(result.messages).includes("Prefer Rust")).toBe(!noRelevant);
+ });
+});
diff --git a/examples/opencode-plugin/package.json b/examples/opencode-plugin/package.json
index c4cb449a1..f7323d502 100644
--- a/examples/opencode-plugin/package.json
+++ b/examples/opencode-plugin/package.json
@@ -1,6 +1,6 @@
{
"name": "@openviking/opencode-plugin",
- "version": "0.3.2",
+ "version": "0.3.3",
"author": "@openviking",
"description": "Unified OpenCode plugin for OpenViking repository retrieval and long-term memory",
"type": "module",
diff --git a/examples/pi-coding-agent-extension/package.json b/examples/pi-coding-agent-extension/package.json
index 431b71d59..30658ab8b 100644
--- a/examples/pi-coding-agent-extension/package.json
+++ b/examples/pi-coding-agent-extension/package.json
@@ -1,6 +1,6 @@
{
"name": "openviking-pi-extension",
- "version": "0.3.2",
+ "version": "0.3.3",
"description": "OpenViking memory and context extension for the pi coding agent",
"private": true,
"type": "module"