Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.
Python agent-contextagent-memoryagent-pluginsagentic-ragcontext-databasedsh-pluginknowledge-basememorymemory-managementself-evolvingvector-database
feat(openclaw-plugin): SecretRef support for config.apiKey (#3522) (#3618)
Ziyang Guoabc387955d40
diff --git a/examples/openclaw-plugin/INSTALL-ZH.md b/examples/openclaw-plugin/INSTALL-ZH.mdindex eb0b8ef85..2e8c9e1d2 100644--- a/examples/openclaw-plugin/INSTALL-ZH.md+++ b/examples/openclaw-plugin/INSTALL-ZH.md@@ -232,7 +232,15 @@ openclaw openviking setup --base-url <OPENVIKING_URL> --api-key <API_KEY> --peer- 如果配置中已有 `plugins.allow`,把 `openviking` 追加进去;如果没有,不要仅为本插件新建 allowlist。- `contextEngine` 是独占 slot;若已有其他 context engine,只有确认替换后再修改该字段。使用 root API key 时,还需在 `config` 中设置 `accountId` 和 `userId`。- 容器连接其他服务时,`baseUrl` 应使用容器内可访问的服务地址,而不是 `127.0.0.1`。-- `apiKey` 会以明文保存;请限制文件权限或使用受控的 Secret 卷。修改后重启 Gateway、容器或 Pod。+- 推荐使用 `SecretRef` 对象作为 `apiKey`(而不是明文字符串),避免密钥直接落盘写入 `openclaw.json`。支持的形式与 OpenClaw 核心中 LLM/TTS/MCP 等 provider 配置使用的标准 `SecretRef` 一致:++ | 类型 | 示例 | 说明 |+ | --- | --- | --- |+ | `env` | `{ "source": "env", "id": "OPENVIKING_API_KEY" }` | 启动时读取同名环境变量。 |+ | `file` | `{ "source": "file", "id": "/etc/secrets/openviking.key" }` | 以 UTF-8 读取并去除首尾空白;`~` 可展开,适配 Kubernetes `secretKeyRef` 卷挂载、0600 权限文件。 |+ | `exec` | `{ "source": "exec", "provider": "op", "id": "op://vault/openviking/credential" }` | 运行 `<provider> <id>`,去除 stdout 首尾空白(1Password `op`、Vault、gopass 等均符合此形态)。 |++ 仍可使用纯字符串形式(含 `${ENV_VAR}` 插值)作为向后兼容路径;此时请限制文件权限或通过受控 Secret 卷提供,并在修改后重启 Gateway、容器或 Pod。### 3. 重启 OpenClaw Gatewaydiff --git a/examples/openclaw-plugin/INSTALL.md b/examples/openclaw-plugin/INSTALL.mdindex 15f51dbc5..ca7528dab 100644--- a/examples/openclaw-plugin/INSTALL.md+++ b/examples/openclaw-plugin/INSTALL.md@@ -176,7 +176,15 @@ If the `openclaw` CLI cannot run inside the container, merge the following field- If `plugins.allow` already exists, append `openviking`; otherwise, do not create an allowlist only for this plugin.- `contextEngine` is an exclusive slot. If another context engine is configured, change it only after confirming the replacement. Root API keys also require `accountId` and `userId` in `config`.- When connecting to another service from a container, use a `baseUrl` that is reachable from the container rather than `127.0.0.1`.-- `apiKey` is stored as plaintext. Restrict file permissions or provide the file through a managed Secret volume, then restart the Gateway, container, or Pod.+- Prefer a `SecretRef` for `apiKey` instead of a plaintext string so the key is never stored inside `openclaw.json`. Supported shapes match OpenClaw's standard `SecretRef` used by LLM/TTS/MCP provider configs:++ | Shape | Example | Notes |+ | --- | --- | --- |+ | `env` | `{ "source": "env", "id": "OPENVIKING_API_KEY" }` | Reads the named env var at plugin load time. |+ | `file` | `{ "source": "file", "id": "/etc/secrets/openviking.key" }` | Reads UTF-8, trims whitespace. `~` is expanded; ideal for Kubernetes `secretKeyRef` volumes and 0600-managed files. |+ | `exec` | `{ "source": "exec", "provider": "op", "id": "op://vault/openviking/credential" }` | Runs `<provider> <id>` and trims stdout (1Password `op`, Vault, gopass, etc. all match this shape). |++ A plain `string` (including `${ENV_VAR}` interpolation) is still accepted, but only as a backward-compatibility path; in that case, restrict file permissions or provide the file through a managed Secret volume, then restart the Gateway, container, or Pod.### 3. Restart OpenClaw Gatewaydiff --git a/examples/openclaw-plugin/config.ts b/examples/openclaw-plugin/config.tsindex 1f86c9d2a..3356a9fad 100644--- a/examples/openclaw-plugin/config.ts+++ b/examples/openclaw-plugin/config.ts@@ -1,13 +1,38 @@import { homedir } from "node:os";+import { readFileSync } from "node:fs";import { getEnv } from "./runtime-utils.js";+/**+ * Subset of OpenClaw's standard `SecretRef` shape supported by the OpenViking+ * plugin. The full runtime SDK also recognises `source: "exec"` with a+ * provider plugin, but the plugin self-hosts its own env/file resolution so+ * users on pluginSdkVersion < 2026.6.x (no host-level secret-resolver) still+ * get vault-free secret management.+ *+ * source: "env" -> `process.env[id]` (same as the bare `${ENV}` string+ * interpolation, but declarative and visible to UI hints).+ * source: "file" -> contents of `id` path (``~`` expansion supported),+ * leading/trailing whitespace trimmed — convenient for+ * Kubernetes `secretKeyRef` mount volumes and+ * file-permission-only deployments.+ * source: "exec" -> delegate to host `openclaw` secret resolver when the+ * runtime exposes it; today the plugin performs a+ * best-effort `provider + id` CLI exec call so that+ * 1Password / Vault / gopass users can share OpenViking+ * credentials with their other providers.+ */+export type OpenVikingSecretRef =+ | { source: "env"; id: string }+ | { source: "file"; id: string }+ | { source: "exec"; provider: string; id: string };+export type MemoryOpenVikingConfig = {mode?: "remote";baseUrl?: string;peer_role?: "none" | "assistant" | "person";peer_prefix?: string;- apiKey?: string;+ apiKey?: string | OpenVikingSecretRef;/** Optional HTTP headers merged into every OpenViking request. */headers?: Record<string, string>;/** Advanced option. Only needed when explicitly sending tenant identity headers. With a user key the server derives identity from the key. */@@ -188,6 +213,105 @@ function resolvePeerPrefix(configured: unknown): string {return DEFAULT_PEER_PREFIX;}+/**+ * Resolve an {@link OpenVikingSecretRef} to the actual secret string. Plain+ * strings pass through untouched so callers can compose this with the legacy+ * `resolveEnvVars` pipeline without branching.+ *+ * Intentional behaviours:+ * * `env`: uses `getEnv` (process.env but exposed for vitest overrides). An+ * unset / empty env var is an error — the user asked for a named secret+ * so silent empty-string fallback would mask misconfigurations.+ * * `file`: ~ expanded, then read as UTF-8, then leading/trailing whitespace+ * stripped (de-facto standard for files written with `echo xxx > key`).+ * Missing / unreadable files propagate the original node error with an+ * error prefix that names the OpenViking field, so the user knows which+ * secretRef failed to load.+ * * `exec`: shells out `<provider> <id>`. Providers commonly expose this+ * shape (1Password CLI, gopass, HashiCorp Vault), which matches how+ * OpenClaw's own `@transmitt0r/openclaw-plugin-onepassword` provider+ * resolves `{source:"exec", provider:"onepassword", id:"..."}`. Exec is+ * best-effort; failures surface with an error prefix so users can+ * distinguish `provider not installed` from `provider configured but id+ * missing`.+ */+function resolveSecret(+ value: string | OpenVikingSecretRef | undefined | null,+ label: string,+): string {+ if (value === undefined || value === null) return "";+ if (typeof value === "string") return value;+ if (!value || typeof value !== "object") {+ throw new Error(+ `OpenViking ${label} must be a plain string or a SecretRef object ` ++ `({source:"env"|"file"|"exec", id})`,+ );+ }+ const obj = value as Record<string, unknown>;+ if (typeof obj.id !== "string" || !obj.id) {+ throw new Error(`OpenViking ${label} SecretRef requires a non-empty string "id"`);+ }+ const id = obj.id;+ switch (obj.source) {+ case "env": {+ const envValue = getEnv(id);+ if (!envValue) {+ throw new Error(+ `OpenViking ${label} SecretRef env source: environment variable ${id} is not set or empty`,+ );+ }+ return envValue;+ }+ case "file": {+ const resolvedPath = expandHomeDir(id);+ try {+ const contents = readFileSync(resolvedPath, "utf8");+ return contents.trim();+ } catch (err) {+ const msg = err instanceof Error ? err.message : String(err);+ throw new Error(+ `OpenViking ${label} SecretRef file source: cannot read ${resolvedPath} (${msg})`,+ );+ }+ }+ case "exec": {+ const provider = typeof (obj as Record<string, unknown>).provider === "string"+ ? (obj as Record<string, unknown>).provider as string+ : "";+ if (!provider) {+ throw new Error(+ `OpenViking ${label} SecretRef exec source requires a "provider" field naming the secret-manager CLI on PATH`,+ );+ }+ try {+ // Avoid top-level `import "node:child_process"` — this branch is rare+ // and keeping the require lazy makes startup marginally faster for+ // the 95% of users who pick env/file.+ // eslint-disable-next-line @typescript-eslint/no-var-requires+ const { execFileSync } = require("node:child_process") as typeof import("node:child_process");+ const out = execFileSync(provider, [id], {+ encoding: "utf8",+ stdio: ["ignore", "pipe", "pipe"],+ timeout: 15_000,+ env: process.env,+ }) as string;+ return out.trim();+ } catch (err) {+ const msg = err instanceof Error ? err.message : String(err);+ throw new Error(+ `OpenViking ${label} SecretRef exec source: provider "${provider}" failed for id "${id}" (${msg})`,+ );+ }+ }+ default:+ throw new Error(+ `OpenViking ${label} SecretRef has unknown source "${String(+ (obj as Record<string, unknown>).source,+ )}". Supported: "env" | "file" | "exec".`,+ );+ }+}+function resolvePeerRole(configured: unknown) {if (typeof configured === "string") {const role = configured.trim().toLowerCase();@@ -465,7 +589,14 @@ export const memoryOpenVikingConfigSchema = {const peerPrefix = resolvePeerPrefix(cfg.peer_prefix);const rawBaseUrl = typeof cfg.baseUrl === "string" ? cfg.baseUrl : resolveDefaultBaseUrl();const resolvedBaseUrl = resolveEnvVars(rawBaseUrl).replace(/\/+$/, "");- const rawApiKey = typeof cfg.apiKey === "string" ? cfg.apiKey : getEnv("OPENVIKING_API_KEY");+ // Support plain string, SecretRef object, and OPENVIKING_API_KEY fallback.+ // A user writing a SecretRef has explicitly opted out of the bare env+ // interpolation path, so the fallback kicks in only when nothing is+ // configured at all (matches existing behaviour).+ const rawApiKey: string | OpenVikingSecretRef | undefined =+ cfg.apiKey !== undefined && cfg.apiKey !== null+ ? (cfg.apiKey as string | OpenVikingSecretRef)+ : getEnv("OPENVIKING_API_KEY") || undefined;const captureMode = cfg.captureMode;if (typeof captureMode !== "undefined" &&@@ -508,7 +639,7 @@ export const memoryOpenVikingConfigSchema = {baseUrl: resolvedBaseUrl,peer_role: peerRole,peer_prefix: peerPrefix,- apiKey: rawApiKey ? resolveEnvVars(rawApiKey) : "",+ apiKey: rawApiKey ? resolveEnvVars(resolveSecret(rawApiKey, "config.apiKey")) : "",headers: toStringRecord(cfg.headers, "openviking config headers"),accountId,userId,@@ -678,7 +809,10 @@ export const memoryOpenVikingConfigSchema = {label: "OpenViking API Key",sensitive: true,placeholder: "${OPENVIKING_API_KEY}",- help: "Optional API key for OpenViking server",+ help:+ "Optional API key for OpenViking server. Accepts a plain string, " ++ "${ENV_VAR} interpolation, or a SecretRef object ({source: env/file/exec, id}). " ++ "Prefer the SecretRef shapes so the key never sits as plaintext in openclaw.json.",},headers: {label: "Headers",diff --git a/examples/openclaw-plugin/openclaw.plugin.json b/examples/openclaw-plugin/openclaw.plugin.jsonindex 775e271c5..cc0eeb20a 100644--- a/examples/openclaw-plugin/openclaw.plugin.json+++ b/examples/openclaw-plugin/openclaw.plugin.json@@ -98,7 +98,7 @@"label": "OpenViking API Key","sensitive": true,"placeholder": "${OPENVIKING_API_KEY}",- "help": "Optional API key for OpenViking server"+ "help": "Optional API key for OpenViking server. Accepts a plain string, ${ENV_VAR} interpolation, or a SecretRef object ({source: env/file/exec, id}). Prefer a SecretRef so the key is never stored as plaintext in openclaw.json."},"headers": {"label": "Headers",@@ -292,7 +292,46 @@"description": "Deprecated and ignored. Tenant identity headers are controlled by explicit accountId/userId."},"apiKey": {- "type": "string"+ "oneOf": [+ {+ "type": "string",+ "description": "Plain API key or ${ENV_VAR} interpolation. For secret-manager-managed keys, prefer the SecretRef object shape below."+ },+ {+ "type": "object",+ "title": "SecretRef (env)",+ "description": "Read the API key from an environment variable.",+ "additionalProperties": false,+ "required": ["source", "id"],+ "properties": {+ "source": { "type": "string", "const": "env", "description": "Look up the key in process.env[id] at plugin load time. Equivalent to ${ENV} interpolation but declarative and visible in the UI." },+ "id": { "type": "string", "description": "Name of the environment variable. Example: OPENVIKING_API_KEY." }+ }+ },+ {+ "type": "object",+ "title": "SecretRef (file)",+ "description": "Read the API key from a file on disk, e.g. a Kubernetes secretKeyRef mount volume or a 0600-permission managed file. ~ is expanded and trailing whitespace is trimmed.",+ "additionalProperties": false,+ "required": ["source", "id"],+ "properties": {+ "source": { "type": "string", "const": "file" },+ "id": { "type": "string", "description": "Absolute file path or ~/relative path. Example: /etc/secrets/openviking.key." }+ }+ },+ {+ "type": "object",+ "title": "SecretRef (exec)",+ "description": "Read the API key by shelling out to a secret-manager CLI. Matches the shape used by OpenClaw's core provider configs so 1Password, HashiCorp Vault, gopass, etc. can manage the OpenViking key alongside LLM/TTS/MCP credentials.",+ "additionalProperties": false,+ "required": ["source", "provider", "id"],+ "properties": {+ "source": { "type": "string", "const": "exec" },+ "provider": { "type": "string", "description": "CLI binary name on PATH. Examples: op, vault, gopass." },+ "id": { "type": "string", "description": "Argument passed to the provider CLI to select the specific secret, e.g. 'vault kv get -field=key secret/openviking' in plain-arg form; this implementation passes id as a single arg." }+ }+ }+ ]},"headers": {"type": "object",diff --git a/examples/openclaw-plugin/tests/ut/config.test.ts b/examples/openclaw-plugin/tests/ut/config.test.tsindex 8fcf0b4a7..693d674eb 100644--- a/examples/openclaw-plugin/tests/ut/config.test.ts+++ b/examples/openclaw-plugin/tests/ut/config.test.ts@@ -1,4 +1,4 @@-import { describe, expect, it, vi, afterEach } from "vitest";+import { afterEach, describe, expect, it, vi } from "vitest";import { memoryOpenVikingConfigSchema } from "../../config.js";@@ -432,3 +432,110 @@ describe("memoryOpenVikingConfigSchema.parse()", () => {expect(cfg.recallTargetTypes).toEqual(["resource"]);});});++describe("memoryOpenVikingConfigSchema.parse() — apiKey SecretRef (#3522)", () => {+ const originalEnv = { ...process.env };+ let tmpDir: string | undefined;++ afterEach(async () => {+ process.env = { ...originalEnv };+ vi.restoreAllMocks();+ if (tmpDir) {+ await import("node:fs/promises").then((fs) =>+ fs.rm(tmpDir, { recursive: true, force: true }).catch(() => void 0),+ );+ }+ });++ it("plain string + ${ENV_VAR} interpolation still works (backward compat)", () => {+ process.env.OV_KEY = "plain-old-key";+ const cfg = memoryOpenVikingConfigSchema.parse({ apiKey: "${OV_KEY}" });+ expect(cfg.apiKey).toBe("plain-old-key");+ });++ it("SecretRef env source reads the named env var", () => {+ process.env.MY_OV_KEY = "from-env-secret";+ const cfg = memoryOpenVikingConfigSchema.parse({+ apiKey: { source: "env", id: "MY_OV_KEY" },+ });+ expect(cfg.apiKey).toBe("from-env-secret");+ });++ it("SecretRef env source errors when the variable is missing instead of silently empty", () => {+ expect(() =>+ memoryOpenVikingConfigSchema.parse({+ apiKey: { source: "env", id: "UNDEFINED_VAR_XYZ" },+ }),+ ).toThrow(/UNDEFINED_VAR_XYZ.*not set or empty/);+ });++ it("SecretRef file source reads the file, expands ~, and trims whitespace", async () => {+ const fs = await import("node:fs/promises");+ const os = await import("node:os");+ const path = await import("node:path");+ tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), "ov-secret-"));+ const keyPath = path.join(tmpDir, "ov.key");+ await fs.writeFile(keyPath, " the-file-secret\n\n");++ const cfg = memoryOpenVikingConfigSchema.parse({ apiKey: { source: "file", id: keyPath } });+ expect(cfg.apiKey).toBe("the-file-secret");+ });++ it("SecretRef file source surfaces a readable error when the file is missing", () => {+ expect(() =>+ memoryOpenVikingConfigSchema.parse({+ apiKey: { source: "file", id: "/no/such/path/openviking-3522.key" },+ }),+ ).toThrow(/file source.*no\/such\/path/);+ });++ it("SecretRef exec source runs <provider> <id> and trims stdout", () => {+ // eslint-disable-next-line @typescript-eslint/no-var-requires+ const cp = require("node:child_process") as typeof import("node:child_process");+ const spy = vi+ .spyOn(cp, "execFileSync")+ .mockImplementation(((cmd: string, args: readonly string[]): unknown => {+ expect(cmd).toBe("my-vault");+ expect(args).toEqual(["secret/openviking/apiKey"]);+ return " provider-returned-key \n";+ }) as typeof cp.execFileSync);++ const cfg = memoryOpenVikingConfigSchema.parse({+ apiKey: { source: "exec", provider: "my-vault", id: "secret/openviking/apiKey" },+ });+ expect(cfg.apiKey).toBe("provider-returned-key");+ expect(spy).toHaveBeenCalledTimes(1);+ });++ it("SecretRef exec source errors without a provider field", () => {+ expect(() =>+ memoryOpenVikingConfigSchema.parse({+ apiKey: { source: "exec", id: "some-id" } as unknown as { source: "exec"; provider: string; id: string },+ }),+ ).toThrow(/exec source requires a "provider" field/);+ });++ it("SecretRef validates required fields (unknown source, missing id)", () => {+ expect(() =>+ memoryOpenVikingConfigSchema.parse({+ apiKey: { source: "magic", id: "x" } as unknown as import("../../config.js").OpenVikingSecretRef,+ }),+ ).toThrow(/unknown source.*magic/);+ expect(() =>+ memoryOpenVikingConfigSchema.parse({+ apiKey: { source: "env" } as unknown as import("../../config.js").OpenVikingSecretRef,+ }),+ ).toThrow(/SecretRef requires a non-empty string "id"/);+ });++ it("falls back to OPENVIKING_API_KEY env var only when apiKey key is absent", () => {+ process.env.OPENVIKING_API_KEY = "fallback-key";+ // Explicitly present but empty string → still used as-is, NOT replaced by env fallback.+ const cfg1 = memoryOpenVikingConfigSchema.parse({ apiKey: "" });+ expect(cfg1.apiKey).toBe("");+ // Completely missing → env fallback kicks in.+ const cfg2 = memoryOpenVikingConfigSchema.parse({});+ expect(cfg2.apiKey).toBe("fallback-key");+ });+});+