Skip to content

Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.

feat(plugins): configurable regex input filters for recall queries and captured turns (#4858)

t0saki708dba60bd24
diff --git a/.github/workflows/pr.yml b/.github/workflows/pr.yml
index 9543934f5..b7a9c2b80 100644
--- a/.github/workflows/pr.yml
+++ b/.github/workflows/pr.yml
@@ -47,6 +47,7 @@ jobs:
examples/codex-memory-plugin/scripts/session-state.test.mjs \
examples/codex-memory-plugin/scripts/config.test.mjs \
examples/claude-code-memory-plugin/scripts/auto-capture.test.mjs \
+ examples/claude-code-memory-plugin/scripts/auto-recall-filters.test.mjs \
examples/claude-code-memory-plugin/scripts/config.test.mjs \
examples/claude-code-memory-plugin/scripts/marketplace.test.mjs \
examples/claude-code-memory-plugin/scripts/skill-experience.test.mjs \
@@ -83,6 +84,7 @@ jobs:
examples/memory-plugin-shared/plugin-known-keys.test.mjs \
examples/memory-plugin-shared/uri-guard.test.mjs \
examples/memory-plugin-shared/capture-utils.test.mjs \
+ examples/memory-plugin-shared/input-filters.test.mjs \
examples/dsh-memory-plugin/*.test.mjs \
examples/opencode-plugin/tests/*.test.mjs \
examples/opencode-plugin/servers/mcp-proxy.test.mjs \
diff --git a/docs/en/agent-integrations/02-claude-code.md b/docs/en/agent-integrations/02-claude-code.md
index bc7db80e5..032e3872b 100644
--- a/docs/en/agent-integrations/02-claude-code.md
+++ b/docs/en/agent-integrations/02-claude-code.md
@@ -87,9 +87,13 @@ Configuration priority: Environment variables > `ovcli.conf` > `ov.conf` > Built
| `OPENVIKING_AUTO_CAPTURE` | `true` | Auto-capture after each turn |
| `OPENVIKING_BYPASS_SESSION` | `false` | Skip all hooks for this session |
| `OPENVIKING_BYPASS_SESSION_PATTERNS` | `""` | CSV glob patterns to auto-bypass |
+| `OPENVIKING_RECALL_QUERY_FILTERS` | `""` | CSV of sed-style regex rules applied to the prompt before it becomes a query ([grammar and examples](https://github.com/volcengine/OpenViking/blob/main/examples/claude-code-memory-plugin/README.md#input-filters)) |
+| `OPENVIKING_CAPTURE_FILTERS` | `""` | CSV of sed-style regex rules applied to every captured turn (same grammar) |
| `OPENVIKING_MEMORY_ENABLED` | (auto) | Force on/off |
| `OPENVIKING_DEBUG` | `false` | Write logs to `~/.openviking/logs/cc-hooks.log` |
+Most of these knobs can also live in `ovcli.conf` under `plugin` — see [Plugin Settings](../configuration/02-client.md#plugin-settings). The two filter knobs are better written there, as JSON arrays, because the environment form is split on commas.
+
If recall latency matters most, see [Low-latency recall](./01-overview.md#low-latency-recall) for the environment-variable and `ovcli.conf` settings that disable query expansion and result compression.
For multi-tenant deployments, configure `OPENVIKING_ACCOUNT` and `OPENVIKING_USER`. The complete list of environment variables is available in the [plugin README](https://github.com/volcengine/OpenViking/blob/main/examples/claude-code-memory-plugin/README.md#configuration).
diff --git a/docs/en/agent-integrations/04-codex.md b/docs/en/agent-integrations/04-codex.md
index dbd8c364c..033ea8ffa 100644
--- a/docs/en/agent-integrations/04-codex.md
+++ b/docs/en/agent-integrations/04-codex.md
@@ -79,8 +79,12 @@ Credential source: env vars win by default — when any `OPENVIKING_*` credentia
| `OPENVIKING_CODEX_IDLE_TTL_MS` | `1800000` | SessionStart idle-TTL sweep threshold |
| `OPENVIKING_CODEX_LOCK_WAIT_MS` | `120000` (SessionEnd), `40000` (PreCompact) | How long a capture hook waits for the per-session state lock |
| `OPENVIKING_CODEX_COMMITTED_TTL_MS` | `2592000000` | How long a committed session's transcript cursor is kept before its state file is retired |
+| `OPENVIKING_RECALL_QUERY_FILTERS` | `""` | CSV of sed-style regex rules applied to the prompt before it becomes a query ([grammar and examples](https://github.com/volcengine/OpenViking/blob/main/examples/codex-memory-plugin/README.md#input-filters)) |
+| `OPENVIKING_CAPTURE_FILTERS` | `""` | CSV of sed-style regex rules applied to every captured turn (same grammar) |
| `OPENVIKING_DEBUG` | `false` | Write logs to `~/.openviking/logs/codex-hooks.log` |
+Most of these knobs can also live in `ovcli.conf` under `plugin` — see [Plugin Settings](../configuration/02-client.md#plugin-settings). The two filter knobs are better written there, as JSON arrays, because the environment form is split on commas.
+
If recall latency matters most, see [Low-latency recall](./01-overview.md#low-latency-recall) for the environment-variable and `ovcli.conf` settings that disable query expansion and Codex's local result compression.
Additional tuning options (e.g., `OPENVIKING_RECALL_LIMIT`, `OPENVIKING_CAPTURE_ASSISTANT_TURNS`) are documented in the [plugin README](https://github.com/volcengine/OpenViking/blob/main/examples/codex-memory-plugin/README.md#tuning-the-plugin).
diff --git a/docs/en/agent-integrations/16-capability-reference.md b/docs/en/agent-integrations/16-capability-reference.md
index fcf2febbf..d4bbc187d 100644
--- a/docs/en/agent-integrations/16-capability-reference.md
+++ b/docs/en/agent-integrations/16-capability-reference.md
@@ -110,9 +110,9 @@ Supporting mechanisms:
## 2.2 The memory-plugin-shared layer
-The `examples/memory-plugin-shared/lib/` directory contains 23 `.mjs` modules and serves as the single source of truth for all JS-based harnesses. These are consumed in two ways:
+The `examples/memory-plugin-shared/lib/` directory contains 24 `.mjs` modules and serves as the single source of truth for all JS-based harnesses. These are consumed in two ways:
-1. **Vendoring (copying)**: The `sync.mjs` script distributes these modules to 7 targets, prefixing every file with `// GENERATED FROM ... DO NOT EDIT.`. Because of this added line, a vendored copy's line number will be exactly one line greater than the library source (keep this in mind when cross-referencing line numbers). Every target ships exactly the modules it imports, so the breakdown follows the import graph rather than the harness. Over the 12-module hook set (`HOOK_SHARED_FILES` in `sync.mjs`) that every hook-driven plugin takes: pi adds `setup-wizard` (13); dsh adds the two `mcp-proxy-*` modules it needs for the stdio proxy (14); opencode adds `setup-wizard`, the two `mcp-proxy-*`, and `batch-send` (16); zcode adds the two `mcp-proxy-*`, `batch-send`, `async-writer`, `agent-hook-runtime`, and `agent-uri-guard` (18); claude-code and codex add `setup-wizard`, the two `mcp-proxy-*`, `batch-send`, `async-writer`, `plugin-config`, `workspace-config`, `workspace-registry`, and `doctor-core` (21 each). agent-plugins takes none of the hook set — it has no hooks — only `credentials`, `debug-log`, and the two `mcp-proxy-*` (4). At the current HEAD, every target has zero drift from the original library source.
+1. **Vendoring (copying)**: The `sync.mjs` script distributes these modules to 7 targets, prefixing every file with `// GENERATED FROM ... DO NOT EDIT.`. Because of this added line, a vendored copy's line number will be exactly one line greater than the library source (keep this in mind when cross-referencing line numbers). Every target ships exactly the modules it imports, so the breakdown follows the import graph rather than the harness. Over the 13-module hook set (`HOOK_SHARED_FILES` in `sync.mjs`) that every hook-driven plugin takes: pi adds `setup-wizard` and `batch-send` (15); dsh adds the two `mcp-proxy-*` modules it needs for the stdio proxy (15); opencode adds `setup-wizard`, the two `mcp-proxy-*`, and `batch-send` (17); zcode adds the two `mcp-proxy-*`, `batch-send`, `async-writer`, `agent-hook-runtime`, and `agent-uri-guard` (19); claude-code and codex add `setup-wizard`, the two `mcp-proxy-*`, `batch-send`, `async-writer`, `plugin-config`, `workspace-config`, `workspace-registry`, and `doctor-core` (22 each). agent-plugins takes none of the hook set — it has no hooks — only `credentials`, `debug-log`, and the two `mcp-proxy-*` (4). At the current HEAD, every target has zero drift from the original library source.
2. **Direct import via relative path (no copying)**: Integrations like cursor, trae, and trae-cn directly `import "../../memory-plugin-shared/lib/..."`. To ensure this works, the installer copies the package alongside the 15 shared modules those hooks transitively import into `~/.openviking/agent-integrations/{<client>,memory-plugin-shared}/`, preserving the relative folder layout. That installed set is closed under imports and excludes the workspace configuration layer, which no hook runtime reaches. At runtime, these harnesses share this directory, meaning reinstalling any one of them will overwrite the shared directory wholesale.
Core modules at a glance (detailed further in the per-dimension sections):
@@ -126,7 +126,8 @@ Core modules at a glance (detailed further in the per-dimension sections):
| `batch-send.mjs` | Executes writes in batches of 100, handles per-message degradation on 404/405 errors, and queues the leading contiguous prefix | cc / codex / opencode + the agent-hook family |
| `profile-inject.mjs` | Injects the profile and available-memory index at session start | 9 harnesses (all but openclaw / hermes) |
| `recall-compress-core.mjs` | Manages the recall compression prompt, URI edit-distance repair, and caching | claude-code |
-| `capture-utils.mjs` | Handles message normalization, injection back-flow guarding, and capture filtering | codex / opencode / dsh / pi / zcode |
+| `capture-utils.mjs` | Handles message normalization, injection back-flow guarding, and the built-in capture heuristics (acks, slash commands, signal floor) | codex / opencode / dsh / pi / zcode |
+| `input-filters.mjs` | Compiles and applies the operator's own sed-style rules — `s` substitute, `d` drop, `k` keep-only, optionally scoped to one role — over the recall query and every captured turn. A rule that fails to parse is reported and skipped, never thrown ([grammar](https://github.com/volcengine/OpenViking/blob/main/examples/claude-code-memory-plugin/README.md#input-filters)) | Vendored by every hook plugin; the `recallQueryFilters` / `captureFilters` knobs are supplied by claude-code / codex |
| `credentials.mjs` | Credential resolution chain (see [§3.1.3](#_3-1-3-credential-systems)) | All JS-based |
| `session-model.mjs` | Session ID prefix derivation and bypass globbing | All JS-based |
| `async-writer.mjs` | Detaches the write path (drains stdin → spawn → approve → write → unref). Falls back to synchronous writing if spawn fails | cc / codex / zcode |
diff --git a/docs/en/configuration/02-client.md b/docs/en/configuration/02-client.md
index aeb35d709..5bcfa8883 100644
--- a/docs/en/configuration/02-client.md
+++ b/docs/en/configuration/02-client.md
@@ -42,7 +42,10 @@ export OPENVIKING_CLI_CONFIG_FILE=/path/to/ovcli.conf
"extra_headers": {
"X-Tenant": "acme"
},
- "gateway_token": "<gateway-token>"
+ "gateway_token": "<gateway-token>",
+ "plugin": {
+ "recallCompress": "off"
+ }
}
```
@@ -131,6 +134,30 @@ Command-line options such as `--profile`, `--progress`, `--no-progress`, and `--
Local directory uploads also honor `.gitignore`. Command-line `--include` and `--exclude` rules are merged with the configuration.
+## Plugin Settings
+
+Behaviour knobs for the memory plugins live under `plugin`. Keys directly under it apply to every harness; a nested object named after a harness — `claude_code` or `codex` — overrides them for that one.
+
+```json
+{
+ "url": "https://openviking.example.com",
+ "api_key": "<user-or-admin-key>",
+ "plugin": {
+ "recallCompress": "off",
+ "bypassSessionPatterns": ["/tmp/**", "**/scratch/**"],
+ "claude_code": {
+ "captureAssistantTurns": false
+ }
+ }
+}
+```
+
+Each key is the camelCase counterpart of an `OPENVIKING_*` tuning variable — `OPENVIKING_RECALL_LIMIT` is `recallLimit`, `OPENVIKING_CAPTURE_ASSISTANT_TURNS` is `captureAssistantTurns`. The mapping does not go both ways: a few variables are deliberately environment-only, such as the one-shot `OPENVIKING_BYPASS_SESSION`. The full list lives in the plugin READMEs: [Claude Code](https://github.com/volcengine/OpenViking/blob/main/examples/claude-code-memory-plugin/README.md#configuration) and [Codex](https://github.com/volcengine/OpenViking/blob/main/examples/codex-memory-plugin/README.md#tuning-the-plugin). List-valued knobs — `bypassSessionPatterns`, `recallQueryFilters`, `captureFilters` — are JSON arrays here, while their environment counterparts are comma-separated strings, so a value containing a literal comma can only be written in the array.
+
+Resolution order, highest first: environment variables → the [workspace layers](#workspace-configuration) → `plugin.<harness>` → `plugin` → the legacy per-harness block in `ov.conf` → built-in defaults. Hook processes read the file on every invocation, so an edit takes effect on the next turn; changing an `OPENVIKING_*` variable instead needs the agent restarted, since hooks inherit its environment.
+
+Only the Claude Code and Codex plugins read this section today, so an entry named after any other harness is inert. `ov-memory-doctor` prints what it resolved and warns about keys it does not recognise, naming the closest real one.
+
## Workspace Configuration
A repository can carry its own plugin settings, so the memory behavior of a project travels with the checkout instead of living in each contributor's home directory. Two files sit under the workspace root, and a third layer is kept per machine:
@@ -153,7 +180,7 @@ Highest first:
| `~/.openviking/workspaces/<slot>.json` | This machine, this workspace |
| `<repo-root>/.openviking/config.local.json` | This checkout, private |
| `<repo-root>/.openviking/config.json` | This repository, committed |
-| `ovcli.conf` `plugin.<harness>` | This machine, one harness |
+| `ovcli.conf` [`plugin.<harness>`](#plugin-settings) | This machine, one harness |
| `ovcli.conf` `plugin` | This machine, every harness |
| `ov.conf` harness section | Compatibility layer for older deployments |
| Built-in defaults | |
diff --git a/docs/zh/agent-integrations/02-claude-code.md b/docs/zh/agent-integrations/02-claude-code.md
index 137127cff..4d1df2322 100644
--- a/docs/zh/agent-integrations/02-claude-code.md
+++ b/docs/zh/agent-integrations/02-claude-code.md
@@ -87,9 +87,13 @@ bash <(curl -fsSL https://ovrelease.tos-cn-beijing.volces.com/memory-plugin-shar
| `OPENVIKING_AUTO_CAPTURE` | `true` | 每轮对话结束后自动捕获新记忆 |
| `OPENVIKING_BYPASS_SESSION` | `false` | 禁用当前会话的所有 Hook |
| `OPENVIKING_BYPASS_SESSION_PATTERNS` | `""` | 通过 CSV 格式的 glob 模式匹配并自动跳过特定会话 |
+| `OPENVIKING_RECALL_QUERY_FILTERS` | `""` | CSV 格式的 sed 风格正则规则,在 prompt 变成检索 query 前生效([语法与示例](https://github.com/volcengine/OpenViking/blob/main/examples/claude-code-memory-plugin/README.md#input-filters)) |
+| `OPENVIKING_CAPTURE_FILTERS` | `""` | CSV 格式的 sed 风格正则规则,作用于每个被捕获的回合(同一套语法) |
| `OPENVIKING_MEMORY_ENABLED` | (auto) | 强制开启或关闭插件 |
| `OPENVIKING_DEBUG` | `false` | 将调试日志输出至 `~/.openviking/logs/cc-hooks.log` |
+这些旋钮大多也可以写在 `ovcli.conf` 的 `plugin` 段下——见[插件配置](../configuration/02-client.md#插件配置)。两个过滤器 knob 尤其建议写在那里,用 JSON 数组,因为环境变量形式会按逗号切分。
+
如果更看重召回响应速度,请参阅[低延迟召回](./01-overview.md#低延迟召回),其中说明了如何通过环境变量或 `ovcli.conf` 关闭查询扩展与结果压缩。
在多租户场景下,请额外配置 `OPENVIKING_ACCOUNT` 和 `OPENVIKING_USER`。完整的环境变量列表请参阅 [插件 README](https://github.com/volcengine/OpenViking/blob/main/examples/claude-code-memory-plugin/README.md#configuration)。
diff --git a/docs/zh/agent-integrations/04-codex.md b/docs/zh/agent-integrations/04-codex.md
index fa0dce475..5045982db 100644
--- a/docs/zh/agent-integrations/04-codex.md
+++ b/docs/zh/agent-integrations/04-codex.md
@@ -78,8 +78,12 @@ TraeCode CLI 2.0 用户启动 `trae-cli`,并可用 `trae-cli plugin list` 确
| `OPENVIKING_CODEX_IDLE_TTL_MS` | `1800000` | `SessionStart` 闲置 TTL 清理阈值(毫秒) |
| `OPENVIKING_CODEX_LOCK_WAIT_MS` | `120000`(SessionEnd)、`40000`(PreCompact) | 捕获类 hook 等待单会话状态锁的时长(毫秒) |
| `OPENVIKING_CODEX_COMMITTED_TTL_MS` | `2592000000` | 已提交会话的转录游标保留时长(毫秒),过期后删除状态文件 |
+| `OPENVIKING_RECALL_QUERY_FILTERS` | `""` | CSV 格式的 sed 风格正则规则,在 prompt 变成检索 query 前生效([语法与示例](https://github.com/volcengine/OpenViking/blob/main/examples/codex-memory-plugin/README.md#input-filters)) |
+| `OPENVIKING_CAPTURE_FILTERS` | `""` | CSV 格式的 sed 风格正则规则,作用于每个被捕获的回合(同一套语法) |
| `OPENVIKING_DEBUG` | `false` | 是否将日志写入 `~/.openviking/logs/codex-hooks.log` |
+这些旋钮大多也可以写在 `ovcli.conf` 的 `plugin` 段下——见[插件配置](../configuration/02-client.md#插件配置)。两个过滤器 knob 尤其建议写在那里,用 JSON 数组,因为环境变量形式会按逗号切分。
+
如果更看重召回响应速度,请参阅[低延迟召回](./01-overview.md#低延迟召回),其中说明了如何通过环境变量或 `ovcli.conf` 关闭查询扩展与 Codex 本地结果压缩。
更多调参说明(如 `OPENVIKING_RECALL_LIMIT`、`OPENVIKING_CAPTURE_ASSISTANT_TURNS` 等),请参考 [插件 README](https://github.com/volcengine/OpenViking/blob/main/examples/codex-memory-plugin/README.md#tuning-the-plugin)。
diff --git a/docs/zh/agent-integrations/16-capability-reference.md b/docs/zh/agent-integrations/16-capability-reference.md
index d94ca98ee..f83033d43 100644
--- a/docs/zh/agent-integrations/16-capability-reference.md
+++ b/docs/zh/agent-integrations/16-capability-reference.md
@@ -111,9 +111,9 @@ per-harness 章节(档案卡)只写差异;所有共享事实均在本章
## 2.2 memory-plugin-shared 共享层
-`examples/memory-plugin-shared/lib/` 下共 23 个 `.mjs` 模块,是 JS 系 harness 的唯一事实源。两种消费形态:
+`examples/memory-plugin-shared/lib/` 下共 24 个 `.mjs` 模块,是 JS 系 harness 的唯一事实源。两种消费形态:
-1. **Vendoring(复制)**:由 `sync.mjs` 分发到 7 个目标,每个文件首行加 `// GENERATED FROM ... DO NOT EDIT.`(因此 vendored 副本行号 = lib 源行号 + 1,交叉读行号引用时要换算)。每个目标只发它真正 import 的模块,因此分发清单跟的是 import 图而不是 harness。在每个 hook 型插件都会拿的 12 个 hook 模块(`sync.mjs` 里的 `HOOK_SHARED_FILES`)之上:pi 加 setup-wizard(13 个);dsh 加 stdio 代理需要的两个 mcp-proxy-*(14 个);opencode 加 setup-wizard、两个 mcp-proxy-* 与 batch-send(16 个);zcode 加两个 mcp-proxy-*、batch-send、async-writer、agent-hook-runtime 与 agent-uri-guard(18 个);claude-code / codex 加 setup-wizard、两个 mcp-proxy-*、batch-send、async-writer、plugin-config、workspace-config、workspace-registry 与 doctor-core(各 21 个)。agent-plugins 没有 hook,因此一个 hook 模块都不拿,只有 credentials、debug-log 与两个 mcp-proxy-*(4 个)。当前 HEAD 各目标与 lib 源零漂移。
+1. **Vendoring(复制)**:由 `sync.mjs` 分发到 7 个目标,每个文件首行加 `// GENERATED FROM ... DO NOT EDIT.`(因此 vendored 副本行号 = lib 源行号 + 1,交叉读行号引用时要换算)。每个目标只发它真正 import 的模块,因此分发清单跟的是 import 图而不是 harness。在每个 hook 型插件都会拿的 13 个 hook 模块(`sync.mjs` 里的 `HOOK_SHARED_FILES`)之上:pi 加 setup-wizard 与 batch-send(15 个);dsh 加 stdio 代理需要的两个 mcp-proxy-*(15 个);opencode 加 setup-wizard、两个 mcp-proxy-* 与 batch-send(17 个);zcode 加两个 mcp-proxy-*、batch-send、async-writer、agent-hook-runtime 与 agent-uri-guard(19 个);claude-code / codex 加 setup-wizard、两个 mcp-proxy-*、batch-send、async-writer、plugin-config、workspace-config、workspace-registry 与 doctor-core(各 22 个)。agent-plugins 没有 hook,因此一个 hook 模块都不拿,只有 credentials、debug-log 与两个 mcp-proxy-*(4 个)。当前 HEAD 各目标与 lib 源零漂移。
2. **相对路径直接 import(不复制)**:cursor / trae / trae-cn 直接 `import "../../memory-plugin-shared/lib/..."`;安装器把包与这些 hook 传递 import 到的 15 个共享模块一起复制到 `~/.openviking/agent-integrations/{<client>,memory-plugin-shared}/`,使相对层级成立。这份安装集合对 import 闭合,不含任何 hook 运行时都够不到的 workspace 配置层。运行期这个共享目录被这几个 harness 共用,任一重装都会整体覆盖。
核心模块速览(细节在各维度章展开):
@@ -127,7 +127,8 @@ per-harness 章节(档案卡)只写差异;所有共享事实均在本章
| `batch-send.mjs` | 100 条/批写入 + 404/405 逐条降级 + 连续前缀入队 | cc / codex / opencode + agent-hook 系 |
| `profile-inject.mjs` | session-start 的 profile + 可用记忆清单注入 | 9 个 harness(openclaw / hermes 除外) |
| `recall-compress-core.mjs` | 召回压缩 prompt + URI 编辑距离修复 + 缓存 | claude-code |
-| `capture-utils.mjs` | 消息归一 + 注入回流防护 + 捕获过滤 | codex / opencode / dsh / pi / zcode |
+| `capture-utils.mjs` | 消息归一 + 注入回流防护 + 内置捕获启发式(应答语、slash 命令、信息量下限) | codex / opencode / dsh / pi / zcode |
+| `input-filters.mjs` | 编译并执行操作员自己配的 sed 风格规则 —— `s` 替换、`d` 丢弃、`k` 仅保留,可限定角色 —— 作用于召回 query 与每个被捕获的回合;解析失败的规则会被报告并跳过,不抛异常([语法](https://github.com/volcengine/OpenViking/blob/main/examples/claude-code-memory-plugin/README_CN.md#输入过滤器)) | 每个 hook 插件都会 vendoring;`recallQueryFilters` / `captureFilters` 两个 knob 目前由 claude-code / codex 提供 |
| `credentials.mjs` | 凭据解析链(详见 [§3.1.3](#_3-1-3-凭据体系)) | 全部 JS 系 |
| `session-model.mjs` | 会话 id 前缀派生 + bypass glob | 全部 JS 系 |
| `async-writer.mjs` | 写路径 detach(drain stdin → spawn → approve → write → unref;spawn 失败回落同步) | cc / codex / zcode |
diff --git a/docs/zh/configuration/02-client.md b/docs/zh/configuration/02-client.md
index 1e9c10938..1f226e22e 100644
--- a/docs/zh/configuration/02-client.md
+++ b/docs/zh/configuration/02-client.md
@@ -131,6 +131,30 @@ export OPENVIKING_CLI_CONFIG_FILE=/path/to/ovcli.conf
本地目录上传还会遵循 `.gitignore`。命令行 `--include`、`--exclude` 会与配置文件中的规则合并。
+## 插件配置
+
+记忆插件的行为旋钮写在 `plugin` 段下。直接挂在它下面的键对所有 harness 生效;以 harness 命名的嵌套对象(`claude_code` 或 `codex`)只覆盖那一个。
+
+```json
+{
+ "url": "https://openviking.example.com",
+ "api_key": "<user 或 admin key>",
+ "plugin": {
+ "recallCompress": "off",
+ "bypassSessionPatterns": ["/tmp/**", "**/scratch/**"],
+ "claude_code": {
+ "captureAssistantTurns": false
+ }
+ }
+}
+```
+
+每个键都是某个 `OPENVIKING_*` 调优变量的 camelCase 对应写法——`OPENVIKING_RECALL_LIMIT` 对应 `recallLimit`,`OPENVIKING_CAPTURE_ASSISTANT_TURNS` 对应 `captureAssistantTurns`。反过来不成立:少数变量刻意只认环境变量,比如一次性的 `OPENVIKING_BYPASS_SESSION`。完整列表在插件 README 里:[Claude Code](https://github.com/volcengine/OpenViking/blob/main/examples/claude-code-memory-plugin/README.md#configuration)、[Codex](https://github.com/volcengine/OpenViking/blob/main/examples/codex-memory-plugin/README.md#tuning-the-plugin)。取列表值的旋钮(`bypassSessionPatterns`、`recallQueryFilters`、`captureFilters`)在这里是 JSON 数组,而它们的环境变量对应物是逗号分隔的字符串,所以值里带字面逗号的只能写进数组。
+
+优先级从高到低:环境变量 → [工作区各层](#工作区配置) → `plugin.<harness>` → `plugin` → `ov.conf` 里遗留的按 harness 分块 → 内置默认值。hook 每次触发都会重新读文件,所以改完下一轮就生效;改 `OPENVIKING_*` 变量则需要重启 agent,因为 hook 继承的是它的环境。
+
+目前只有 Claude Code 和 Codex 插件会读这一段,用其它 harness 名字建的条目不会生效。`ov-memory-doctor` 会打印它解析到的结果,并对不认识的键给出告警和最接近的正确键名。
+
## 工作区配置
仓库可以自带插件配置,这样项目的记忆行为跟着代码走,而不是散落在每位协作者的 home 目录里。工作区根目录下有两个文件,另有一层按机器保存:
@@ -153,7 +177,7 @@ export OPENVIKING_CLI_CONFIG_FILE=/path/to/ovcli.conf
| `~/.openviking/workspaces/<slot>.json` | 本机的这个工作区 |
| `<repo-root>/.openviking/config.local.json` | 本地这份 checkout,私有 |
| `<repo-root>/.openviking/config.json` | 整个仓库,随代码提交 |
-| `ovcli.conf` `plugin.<harness>` | 本机的单个 harness |
+| `ovcli.conf` [`plugin.<harness>`](#插件配置) | 本机的单个 harness |
| `ovcli.conf` `plugin` | 本机的所有 harness |
| `ov.conf` harness 段 | 旧部署的兼容层 |
| 内置默认值 | |
diff --git a/examples/claude-code-memory-plugin/.claude-plugin/plugin.json b/examples/claude-code-memory-plugin/.claude-plugin/plugin.json
index a64d33f42..f0c3f5acb 100644
--- a/examples/claude-code-memory-plugin/.claude-plugin/plugin.json
+++ b/examples/claude-code-memory-plugin/.claude-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "openviking-memory",
- "version": "0.4.6",
+ "version": "0.4.7",
"description": "Long-term semantic memory for Claude Code, powered by OpenViking. Auto-recall relevant memories at session start and capture important information during conversations.",
"author": {
"name": "OpenViking",
diff --git a/examples/claude-code-memory-plugin/README.md b/examples/claude-code-memory-plugin/README.md
index 70b5c97c6..9ab12e5f0 100644
--- a/examples/claude-code-memory-plugin/README.md
+++ b/examples/claude-code-memory-plugin/README.md
@@ -180,6 +180,7 @@ Upgrading from the path-derived peer needs no action: memories written under the
| `OPENVIKING_RECALL_COMPRESS_MAX_BULLETS` | `6` | Digest bullet ceiling |
| `OPENVIKING_SCORE_THRESHOLD` | `0.35` | Min relevance score (0–1) |
| `OPENVIKING_MIN_QUERY_LENGTH` | `3` | Skip recall for very short queries |
+| `OPENVIKING_RECALL_QUERY_FILTERS` | `""` | CSV of regex rules applied to the prompt before it becomes a search query — see [Input filters](#input-filters) |
Recall defaults to the broad mode: global memory, the current workspace, and other workspace memories can all be recalled, with other workspaces penalized and rendered later. Set `OPENVIKING_RECALL_PEER_SCOPE=actor` for the isolation mode, which only sees global memory plus the current workspace. In deployments where one bot serves multiple real people, such as zouk, vikingbot, or AstrBot, use the isolation mode with an explicit actor peer so one person's memories are not recalled into another person's session.
| `OPENVIKING_LOG_RANKING_DETAILS` | `false` | Per-candidate scoring logs (verbose) |
@@ -195,6 +196,7 @@ Recall defaults to the broad mode: global memory, the current workspace, and oth
| `OPENVIKING_CAPTURE_TOOL_MAX_CHARS` | `1000000` | Guard cap on one tool part's `tool_output`; oversized output is externalized server-side |
| `OPENVIKING_COMMIT_TOKEN_THRESHOLD` | `20000` | Pending-token threshold for client-driven commit |
| `OPENVIKING_RESUME_CONTEXT_BUDGET` | `32000` | Token budget when fetching archive overview on session resume |
+| `OPENVIKING_CAPTURE_FILTERS` | `""` | CSV of regex rules applied to every captured turn — see [Input filters](#input-filters) |
#### Lifecycle / behavior / misc
@@ -243,6 +245,51 @@ OPENVIKING_BYPASS_SESSION=1 claude
When bypass is active, every hook approves immediately without contacting OpenViking.
+### Input filters
+
+Two knobs put an ordered list of regex rules in front of the text the plugin sends:
+
+- `recallQueryFilters` / `OPENVIKING_RECALL_QUERY_FILTERS` — the prompt, before it becomes a search query.
+- `captureFilters` / `OPENVIKING_CAPTURE_FILTERS` — every turn on the write path (`Stop`, `PreCompact`, `SessionEnd`, `SubagentStop`), before it is stored.
+
+Rules are sed-style strings applied in order to one piece of text:
+
+| Form | Meaning |
+|------|---------|
+| `s<d>pattern<d>replacement<d>[flags]` | substitute; `$1`, `$&`, `$$` work in the replacement |
+| `d<d>pattern<d>[flags]` | drop the text when the pattern matches |
+| `k<d>pattern<d>[flags]` | keep the text only when the pattern matches (chain them for AND) |
+| `user:` / `assistant:` prefix | apply the rule to that role only |
+
+`<d>` is any punctuation delimiter — `/`, `|`, `#`, `:` — and `\` escapes it inside the pattern. Flags are `i`, `m`, `s`, `u` and `g` (`g` replaces every match; it means nothing on `d`/`k` and is dropped there).
+
+| Rule | Effect |
+|------|--------|
+| `s/^\s*(ultrathink\|think harder?)\s+//i` | strip a thinking-keyword prefix from the query |
+| `d\|^\s*[/!]\|` | skip recall for slash commands and `!` bash-mode prompts (a `\|` delimiter keeps the `/` unescaped) |
+| `k/^\?ov\b/` then `s/^\?ov\s*//` | opt-in recall: only prompts starting with `?ov`, with the trigger stripped |
+| `s/\b(sk\|ghp\|xoxb)_[A-Za-z0-9_-]+/[redacted]/g` | redact tokens before they are stored |
+| `user:d/^\s*\/(clear\|compact)\b/` | never store those command turns, user role only |
+| `s/^(请\|麻烦)(你\|帮我)?//` | strip a Chinese politeness prefix |
+
+In `ovcli.conf` the rules are a JSON array, so backslashes are doubled:
+
+```json
+{
+ "plugin": {
+ "claude_code": {
+ "recallQueryFilters": ["s/^\\s*ultrathink\\s+//i", "d|^\\s*[/!]|"],
+ "captureFilters": ["s/\\b(sk|ghp)_[A-Za-z0-9_-]{10,}/[redacted]/g"]
+ }
+ }
+}
+```
+
+- **The env vars are comma-separated lists**, split before parsing, so a rule that needs a literal comma — a bounded `{10,}`, say — belongs in the array above. (`\x2c` covers a literal comma elsewhere in a pattern, but it is not quantifier syntax.)
+- **Order matters and drops win.** The first `d` that matches, or `k` that does not, ends the decision. Filters run before `OPENVIKING_MIN_QUERY_LENGTH` and before the built-in ack / slash-command heuristics, so a prefix stripped down to `ok` is discarded as an ack. Text a substitution empties is not a drop — an emptied query is simply too short to recall on.
+- **Filters shape what is sent, not what is stored.** Adding a `d`/`k` rule mid-session also shortens the turn list the capture cursor counts, which reads as a transcript rewrite and replays from the last user turn — the same thing toggling `OPENVIKING_CAPTURE_ASSISTANT_TURNS` does.
+- **A bad rule is skipped, never fatal.** `ov-memory-doctor` lists the active rules and reports the exact parse or RegExp error for the ones it could not compile.
+
### Plugin settings in `ovcli.conf`
Client-side tuning belongs in `~/.openviking/ovcli.conf` under a `plugin` section. Shared keys apply to every harness; a per-harness object overrides them:
diff --git a/examples/claude-code-memory-plugin/README_CN.md b/examples/claude-code-memory-plugin/README_CN.md
index eb5d87959..78a88f1a2 100644
--- a/examples/claude-code-memory-plugin/README_CN.md
+++ b/examples/claude-code-memory-plugin/README_CN.md
@@ -148,6 +148,7 @@ claude
| `OPENVIKING_RECALL_PREFER_ABSTRACT` | `true` | 有 abstract 时优先用 abstract 而非完整 body |
| `OPENVIKING_SCORE_THRESHOLD` | `0.35` | 最低相关度得分(0–1) |
| `OPENVIKING_MIN_QUERY_LENGTH` | `3` | 短于此长度的 query 跳过召回 |
+| `OPENVIKING_RECALL_QUERY_FILTERS` | `""` | 逗号分隔的正则规则,在 prompt 变成检索 query 前生效 —— 见[输入过滤器](#输入过滤器) |
| `OPENVIKING_LOG_RANKING_DETAILS` | `false` | 每候选打分日志(很啰嗦) |
| `OPENVIKING_RECALL_MAX_TOKENS` | `1600` | 服务端组装上下文块的 token 预算(与本地压缩输入上限相互独立) |
| `OPENVIKING_RECALL_DEDUP_TURNS` | `5` | 跨轮冷却:最近 N 轮已注入过的 URI 本轮跳过 |
@@ -165,6 +166,7 @@ claude
| `OPENVIKING_CAPTURE_ASSISTANT_TURNS` | `true` | 捕获 assistant 回合(文本 + tool 输入/输出)。设为 `0` 可退回仅用户 |
| `OPENVIKING_COMMIT_TOKEN_THRESHOLD` | `20000` | client-driven commit 的 pending-token 阈值 |
| `OPENVIKING_RESUME_CONTEXT_BUDGET` | `32000` | resume 时拉取 archive overview 的 token 预算 |
+| `OPENVIKING_CAPTURE_FILTERS` | `""` | 逗号分隔的正则规则,作用于每个被捕获的回合 —— 见[输入过滤器](#输入过滤器) |
#### 生命周期 / 行为 / 杂项
@@ -213,6 +215,51 @@ OPENVIKING_BYPASS_SESSION=1 claude
bypass 命中时所有 hook 直接放行,不联系 OpenViking。
+### 输入过滤器
+
+两个配置项在插件送出文本之前加了一层有序的正则规则:
+
+- `recallQueryFilters` / `OPENVIKING_RECALL_QUERY_FILTERS` —— 作用于 prompt,在它变成检索 query 之前。
+- `captureFilters` / `OPENVIKING_CAPTURE_FILTERS` —— 作用于写路径(`Stop`、`PreCompact`、`SessionEnd`、`SubagentStop`)上的每个回合,在它被存下来之前。
+
+规则是 sed 风格的字符串,按顺序作用于同一段文本:
+
+| 写法 | 含义 |
+|------|------|
+| `s<d>模式<d>替换<d>[flags]` | 替换;替换串里可以用 `$1`、`$&`、`$$` |
+| `d<d>模式<d>[flags]` | 命中则丢弃这段文本 |
+| `k<d>模式<d>[flags]` | 只有命中才保留(多条串联即 AND) |
+| `user:` / `assistant:` 前缀 | 该规则只对这个角色生效 |
+
+`<d>` 是任意标点分隔符(`/`、`|`、`#`、`:`),模式里用 `\` 转义它。flags 支持 `i`、`m`、`s`、`u`、`g`(`g` 表示替换全部;对 `d`/`k` 没有意义,会被去掉)。
+
+| 规则 | 效果 |
+|------|------|
+| `s/^\s*(ultrathink\|think harder?)\s+//i` | 去掉 query 前面的思考关键词 |
+| `d\|^\s*[/!]\|` | slash 命令和 `!` bash 模式的 prompt 不触发召回(用 `\|` 当分隔符,`/` 就不必转义) |
+| `k/^\?ov\b/` 配 `s/^\?ov\s*//` | 改成显式触发:只有以 `?ov` 开头才召回,并去掉这个触发词 |
+| `s/\b(sk\|ghp\|xoxb)_[A-Za-z0-9_-]+/[redacted]/g` | 存进记忆前把 token 打码 |
+| `user:d/^\s*\/(clear\|compact)\b/` | 这类命令回合永不入库,且只针对用户侧 |
+| `s/^(请\|麻烦)(你\|帮我)?//` | 去掉中文客套前缀 |
+
+写进 `ovcli.conf` 时是 JSON 数组,所以反斜杠要写两遍:
+
+```json
+{
+ "plugin": {
+ "claude_code": {
+ "recallQueryFilters": ["s/^\\s*ultrathink\\s+//i", "d|^\\s*[/!]|"],
+ "captureFilters": ["s/\\b(sk|ghp)_[A-Za-z0-9_-]{10,}/[redacted]/g"]
+ }
+ }
+}
+```
+
+- **环境变量是逗号分隔的列表**,先按逗号切分再解析,所以需要字面逗号的规则(比如带下界的 `{10,}`)只能写进上面的数组。(`\x2c` 能表示模式里其它位置的字面逗号,但它不是量词语法。)
+- **顺序有意义,丢弃优先。** 第一条命中的 `d`(或未命中的 `k`)就决定了结果。过滤发生在 `OPENVIKING_MIN_QUERY_LENGTH` 和内置的应答语/slash 命令启发式之前,所以剥掉前缀后只剩「好的」的文本会按应答语丢弃。被替换成空串不算丢弃 —— query 空了只是长度不够。
+- **过滤只管送出去的内容,管不到已经存下的。** 会话中途新增 `d`/`k` 规则还会让捕获游标计数的回合列表变短,这会被当成 transcript 被改写、从最后一个用户回合重放 —— 和切换 `OPENVIKING_CAPTURE_ASSISTANT_TURNS` 是同一个现象。
+- **坏规则只会被跳过,不会让 hook 挂掉。** `ov-memory-doctor` 会列出生效的规则,并对编译失败的那条给出确切的解析或 RegExp 报错。
+
### 插件配置放在 `ovcli.conf`
客户端侧的调优应写在 `~/.openviking/ovcli.conf` 的 `plugin` 区域。共享键对所有 harness 生效,分 harness 的对象可覆盖它:
diff --git a/examples/claude-code-memory-plugin/package.json b/examples/claude-code-memory-plugin/package.json
index d4d7c2cbf..9fb3f932a 100644
--- a/examples/claude-code-memory-plugin/package.json
+++ b/examples/claude-code-memory-plugin/package.json
@@ -1,6 +1,6 @@
{
"name": "claude-code-openviking-memory",
- "version": "0.4.6",
+ "version": "0.4.7",
"description": "OpenViking memory plugin for Claude Code — semantic long-term memory",
"type": "module",
"scripts": {
diff --git a/examples/claude-code-memory-plugin/scripts/auto-capture.mjs b/examples/claude-code-memory-plugin/scripts/auto-capture.mjs
index 42c7ee1ae..9e45bbaf1 100644
--- a/examples/claude-code-memory-plugin/scripts/auto-capture.mjs
+++ b/examples/claude-code-memory-plugin/scripts/auto-capture.mjs
@@ -37,6 +37,7 @@ import { maybeDetach, readHookStdin } from "./lib/async-writer.mjs";
import { readJsonState, writeJsonState } from "./lib/state.mjs";
import { getEffectivePeerId } from "./lib/workspace-peer.mjs";
import { sendSessionMessages } from "./shared/batch-send.mjs";
+import { filterCaptureParts } from "./shared/capture-utils.mjs";
if (!isPluginEnabled()) {
process.stdout.write(JSON.stringify({ decision: "approve" }) + "\n");
@@ -421,8 +422,10 @@ function formatTurnsAsText(turns) {
// Strip plugin-injected blocks from text parts (tool parts pass through), and
// drop parts that become empty. Mirrors the old content-path stripInjectedBlocks
-// + trim, but per text part so tool I/O is never collapsed.
-function sanitizePartsForSend(parts) {
+// + trim, but per text part so tool I/O is never collapsed. The configured
+// capture filters run last, here at the send site rather than in the extractor,
+// because the cursor CC advances is an index into the extracted turn list.
+function sanitizePartsForSend(parts, role = "") {
const out = [];
for (const p of parts || []) {
if (p.type === "text") {
@@ -432,7 +435,8 @@ function sanitizePartsForSend(parts) {
out.push(p);
}
}
- return out;
+ const shaped = filterCaptureParts(out, role, cfg);
+ return shaped.dropped ? [] : shaped.parts;
}
async function pushTurnsToOv(ovSessionId, turns, peerId = "") {
@@ -440,7 +444,7 @@ async function pushTurnsToOv(ovSessionId, turns, peerId = "") {
for (const turn of turns) {
// Send structured parts: tool calls/results are dedicated `tool` parts, not
// inlined into content, so the server can process them separately.
- const parts = sanitizePartsForSend(turn.parts);
+ const parts = sanitizePartsForSend(turn.parts, turn.role);
if (parts.length === 0) continue;
const payload = { role: turn.role, parts };
@@ -463,7 +467,7 @@ async function enqueueTurnsToPending(ovSessionId, turns, peerId = "") {
let queued = 0;
let failed = 0;
for (const turn of turns) {
- const parts = sanitizePartsForSend(turn.parts);
+ const parts = sanitizePartsForSend(turn.parts, turn.role);
if (parts.length === 0) continue;
const payload = { role: turn.role, parts };
diff --git a/examples/claude-code-memory-plugin/scripts/auto-capture.test.mjs b/examples/claude-code-memory-plugin/scripts/auto-capture.test.mjs
index 2b5c9937f..e9db0a09a 100644
--- a/examples/claude-code-memory-plugin/scripts/auto-capture.test.mjs
+++ b/examples/claude-code-memory-plugin/scripts/auto-capture.test.mjs
@@ -384,3 +384,56 @@ test("the workspace that decides capture is the payload's, not the hook process'
await rm(root, { recursive: true, force: true });
}
});
+
+test("capture filters rewrite and drop turns at the send site", async () => {
+ const root = await mkdtemp(join(tmpdir(), "ov-cc-capture-filters-"));
+ const transcriptPath = join(root, "transcript.jsonl");
+ const sessionId = "capture-filters";
+ const batches = [];
+
+ try {
+ await writeFile(
+ transcriptPath,
+ [
+ JSON.stringify({ role: "user", content: "the token is sk_LIVE_ABCDEF, remember it" }),
+ JSON.stringify({ role: "user", content: "scratch: ignore this throwaway note" }),
+ JSON.stringify({ role: "assistant", content: "noted, I will retain that context" }),
+ ].join("\n"),
+ );
+ await withMockOpenViking(async (req, res) => {
+ const url = new URL(req.url, "http://127.0.0.1");
+ if (req.method === "GET" && url.pathname === "/health") {
+ writeJson(res, 200, { status: "ok", result: { healthy: true } });
+ return;
+ }
+ if (req.method === "POST" && url.pathname.endsWith("/messages/batch")) {
+ const body = await readRequestBody(req);
+ batches.push(body);
+ writeJson(res, 200, { status: "ok", result: { added: body.messages.length } });
+ return;
+ }
+ writeJson(res, 200, { status: "ok", result: {} });
+ }, async (baseUrl) => {
+ await runAutoCapture({ session_id: sessionId, transcript_path: transcriptPath, cwd: root }, {
+ ...hookEnv(root, baseUrl),
+ OPENVIKING_CAPTURE_FILTERS: "s/sk_[A-Za-z0-9_]+/[redacted]/g,user:d/^scratch:/",
+ });
+ });
+
+ assert.equal(batches.length, 1);
+ const texts = batches[0].messages.flatMap(
+ (message) => message.parts.filter((p) => p.type === "text").map((p) => p.text),
+ );
+ assert.deepEqual(texts, [
+ "the token is [redacted], remember it",
+ "noted, I will retain that context",
+ ]);
+ const state = JSON.parse(
+ await readFile(join(root, "openviking-cc-capture-state", `${sessionId}.json`), "utf-8"),
+ );
+ // The cursor counts extracted turns, so the dropped one still advances it.
+ assert.equal(state.capturedTurnCount, 3);
+ } finally {
+ await rm(root, { recursive: true, force: true });
+ }
+});
diff --git a/examples/claude-code-memory-plugin/scripts/auto-recall-filters.test.mjs b/examples/claude-code-memory-plugin/scripts/auto-recall-filters.test.mjs
new file mode 100644
index 000000000..ee0e0ffe4
--- /dev/null
+++ b/examples/claude-code-memory-plugin/scripts/auto-recall-filters.test.mjs
@@ -0,0 +1,147 @@
+import assert from "node:assert/strict";
+import { spawn } from "node:child_process";
+import { mkdtemp, readFile, rm } from "node:fs/promises";
+import http from "node:http";
+import { tmpdir } from "node:os";
+import { dirname, join } from "node:path";
+import test from "node:test";
+import { fileURLToPath } from "node:url";
+
+const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url));
+
+function writeJson(res, statusCode, value) {
+ res.writeHead(statusCode, { "Content-Type": "application/json" });
+ res.end(JSON.stringify(value));
+}
+
+function readRequestBody(req) {
+ return new Promise((resolve) => {
+ const chunks = [];
+ req.on("data", (chunk) => chunks.push(chunk));
+ req.on("end", () => {
+ const raw = Buffer.concat(chunks).toString("utf-8");
+ try {
+ resolve(raw ? JSON.parse(raw) : null);
+ } catch {
+ resolve(null);
+ }
+ });
+ });
+}
+
+/** Mock server that records every request the hook makes. */
+async function withRecordedServer(fn) {
+ const seen = [];
+ const server = http.createServer((req, res) => {
+ readRequestBody(req).then((body) => {
+ seen.push({ method: req.method, path: req.url.split("?")[0], body });
+ if (req.url.startsWith("/health")) {
+ writeJson(res, 200, { status: "ok", result: { healthy: true } });
+ return;
+ }
+ writeJson(res, 200, { status: "ok", result: {} });
+ });
+ });
+ await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve));
+ try {
+ const { port } = server.address();
+ return await fn(`http://127.0.0.1:${port}`, seen);
+ } finally {
+ await new Promise((resolve) => server.close(resolve));
+ }
+}
+
+function runAutoRecall(input, env) {
+ return new Promise((resolve, reject) => {
+ const cleanEnv = { ...process.env };
+ for (const key of Object.keys(cleanEnv)) {
+ if (key.startsWith("OPENVIKING_")) delete cleanEnv[key];
+ }
+ const child = spawn(process.execPath, [join(SCRIPT_DIR, "auto-recall.mjs")], {
+ env: { ...cleanEnv, ...env },
+ stdio: ["pipe", "pipe", "pipe"],
+ });
+ let stdout = "";
+ let stderr = "";
+ child.stdout.on("data", (chunk) => { stdout += chunk.toString(); });
+ child.stderr.on("data", (chunk) => { stderr += chunk.toString(); });
+ child.on("error", reject);
+ child.on("close", (code) => {
+ if (code !== 0) reject(new Error(`auto-recall exited ${code}: ${stderr}`));
+ else resolve({ stdout, stderr });
+ });
+ child.stdin.end(JSON.stringify(input));
+ });
+}
+
+function hookEnv(root, baseUrl, extra) {
+ return {
+ HOME: root,
+ TMPDIR: root,
+ OPENVIKING_HOME: join(root, ".openviking"),
+ OPENVIKING_MEMORY_ENABLED: "1",
+ OPENVIKING_AUTO_RECALL: "1",
+ OPENVIKING_TIMEOUT_MS: "5000",
+ OPENVIKING_URL: baseUrl,
+ ...extra,
+ };
+}
+
+async function lastRecall(root) {
+ return JSON.parse(await readFile(join(root, ".openviking", "state", "last-recall.json"), "utf-8"));
+}
+
+test("a drop rule ends the turn before any request is made", async () => {
+ const root = await mkdtemp(join(tmpdir(), "ov-cc-recall-filter-drop-"));
+ try {
+ await withRecordedServer(async (baseUrl, seen) => {
+ const { stdout } = await runAutoRecall(
+ { session_id: "filter-drop", prompt: "/help me out here", cwd: root },
+ hookEnv(root, baseUrl, { OPENVIKING_RECALL_QUERY_FILTERS: "d|^\\s*[/!]|" }),
+ );
+ assert.deepEqual(JSON.parse(stdout), { decision: "approve" });
+ assert.deepEqual(seen, []);
+ });
+ const state = await lastRecall(root);
+ assert.equal(state.reason, "query_filtered");
+ assert.equal(state.count, 0);
+ assert.equal(state.cc_session_id, "filter-drop");
+ } finally {
+ await rm(root, { recursive: true, force: true });
+ }
+});
+
+test("a substitute rule rewrites the query the server is asked for", async () => {
+ const root = await mkdtemp(join(tmpdir(), "ov-cc-recall-filter-sub-"));
+ try {
+ await withRecordedServer(async (baseUrl, seen) => {
+ await runAutoRecall(
+ { session_id: "filter-sub", prompt: "ultrathink what did we decide about retries", cwd: root },
+ hookEnv(root, baseUrl, { OPENVIKING_RECALL_QUERY_FILTERS: "s/^\\s*ultrathink\\s+//i" }),
+ );
+ const queries = seen.map((call) => call.body?.query).filter(Boolean);
+ assert.ok(queries.length > 0, `no query was sent: ${JSON.stringify(seen)}`);
+ for (const query of queries) {
+ assert.equal(query, "what did we decide about retries");
+ }
+ });
+ } finally {
+ await rm(root, { recursive: true, force: true });
+ }
+});
+
+test("an unparsable rule is skipped and the turn proceeds", async () => {
+ const root = await mkdtemp(join(tmpdir(), "ov-cc-recall-filter-bad-"));
+ try {
+ await withRecordedServer(async (baseUrl, seen) => {
+ await runAutoRecall(
+ { session_id: "filter-bad", prompt: "what did we decide about retries", cwd: root },
+ hookEnv(root, baseUrl, { OPENVIKING_RECALL_QUERY_FILTERS: "s/(/x/" }),
+ );
+ assert.ok(seen.some((call) => call.path === "/health"), "the turn should have proceeded");
+ });
+ assert.notEqual((await lastRecall(root)).reason, "query_filtered");
+ } finally {
+ await rm(root, { recursive: true, force: true });
+ }
+});
diff --git a/examples/claude-code-memory-plugin/scripts/auto-recall.mjs b/examples/claude-code-memory-plugin/scripts/auto-recall.mjs
index 43225d0d0..184c2c9a0 100644
--- a/examples/claude-code-memory-plugin/scripts/auto-recall.mjs
+++ b/examples/claude-code-memory-plugin/scripts/auto-recall.mjs
@@ -21,6 +21,7 @@ import { writeJsonState } from "./lib/state.mjs";
import { createHostCompressor } from "./lib/host-compressor.mjs";
import { getEffectivePeerId } from "./lib/workspace-peer.mjs";
import { buildServerAssembledBlock } from "./shared/recall-core.mjs";
+import { applyInputFilters, compileInputFilters } from "./shared/input-filters.mjs";
if (!isPluginEnabled()) {
process.stdout.write(JSON.stringify({ decision: "approve" }) + "\n");
@@ -348,7 +349,7 @@ async function main() {
return;
}
- const userPrompt = (input.prompt || "").trim();
+ let userPrompt = (input.prompt || "").trim();
const sessionId = input.session_id;
const cwd = input.cwd;
// The workspace layer belongs to the session's directory, which only the
@@ -384,6 +385,22 @@ async function main() {
return;
}
+ // Filters run before the length gate, so a prompt whose only content was a
+ // stripped prefix is short_query rather than a search for the empty string.
+ const queryFilters = compileInputFilters(cfg.recallQueryFilters);
+ if (queryFilters.rules.length) {
+ const verdict = applyInputFilters(userPrompt, queryFilters.rules, { role: "user" });
+ if (verdict.dropped) {
+ log("skip", { reason: "query_filter", rule: verdict.ruleIndex, op: verdict.op });
+ writeRecallState({ count: 0, reason: "query_filtered", cc_session_id: sessionId });
+ approve();
+ return;
+ }
+ if (verdict.changed) log("query_filter", { rawLength: userPrompt.length, length: verdict.text.length });
+ userPrompt = verdict.text;
+ }
+ if (queryFilters.errors.length) log("query_filter_errors", queryFilters.errors);
+
if (!userPrompt || userPrompt.length < cfg.minQueryLength) {
log("skip", { reason: "query too short or empty" });
writeRecallState({ count: 0, reason: "short_query", cc_session_id: sessionId });
diff --git a/examples/claude-code-memory-plugin/scripts/config.mjs b/examples/claude-code-memory-plugin/scripts/config.mjs
index 2ee3ec602..e3bd76a82 100644
--- a/examples/claude-code-memory-plugin/scripts/config.mjs
+++ b/examples/claude-code-memory-plugin/scripts/config.mjs
@@ -21,11 +21,12 @@
* OPENVIKING_AUTO_RECALL, OPENVIKING_RECALL_LIMIT, OPENVIKING_RECALL_TOKEN_BUDGET,
* OPENVIKING_RECALL_MAX_CONTENT_CHARS, OPENVIKING_RECALL_PREFER_ABSTRACT,
* OPENVIKING_SCORE_THRESHOLD, OPENVIKING_MIN_QUERY_LENGTH, OPENVIKING_LOG_RANKING_DETAILS,
- * OPENVIKING_RECALL_PEER_SCOPE, OPENVIKING_RECALL_COMPRESS
+ * OPENVIKING_RECALL_PEER_SCOPE, OPENVIKING_RECALL_COMPRESS,
+ * OPENVIKING_RECALL_QUERY_FILTERS (CSV)
* Capture tuning:
* OPENVIKING_AUTO_CAPTURE, OPENVIKING_CAPTURE_MODE, OPENVIKING_CAPTURE_MAX_LENGTH,
* OPENVIKING_CAPTURE_ASSISTANT_TURNS, OPENVIKING_COMMIT_TOKEN_THRESHOLD,
- * OPENVIKING_RESUME_CONTEXT_BUDGET
+ * OPENVIKING_RESUME_CONTEXT_BUDGET, OPENVIKING_CAPTURE_FILTERS (CSV)
* Lifecycle / behavior:
* OPENVIKING_TIMEOUT_MS, OPENVIKING_CAPTURE_TIMEOUT_MS, OPENVIKING_WRITE_PATH_ASYNC,
* OPENVIKING_BYPASS_SESSION, OPENVIKING_BYPASS_SESSION_PATTERNS (CSV),
@@ -252,6 +253,18 @@ export function loadConfig(cwd = process.cwd()) {
? cc.bypassSessionPatterns.filter((p) => typeof p === "string" && p.trim())
: []);
+ // Input filter rule lists: env CSV overrides the configured array entirely,
+ // same shape as bypassSessionPatterns. Both paths trim, so a rule reads the
+ // same however it was configured — and a rule that needs a literal comma has
+ // to come from the array, since the env value is split on it.
+ const filterList = (envName, configured) => {
+ const raw = str(process.env[envName], null);
+ const list = raw !== null ? raw.split(",") : (Array.isArray(configured) ? configured : []);
+ return list.filter((r) => typeof r === "string").map((r) => r.trim()).filter(Boolean);
+ };
+ const recallQueryFilters = filterList("OPENVIKING_RECALL_QUERY_FILTERS", cc.recallQueryFilters);
+ const captureFilters = filterList("OPENVIKING_CAPTURE_FILTERS", cc.captureFilters);
+
return {
configPath,
credentialSource,
@@ -408,6 +421,11 @@ export function loadConfig(cwd = process.cwd()) {
bypassSessionPatterns,
bypassSession: envBool("OPENVIKING_BYPASS_SESSION") ?? false,
+ // Ordered sed-style rules over the text the plugin sends: the recall query
+ // on UserPromptSubmit, and every captured turn on the write path.
+ recallQueryFilters,
+ captureFilters,
+
// Write-path async: auto-capture / session-end / subagent-stop fire-and-
// forget via a detached child process, so the hook returns to CC instantly.
// pre-compact stays sync regardless (CC rewrites transcript right after).
diff --git a/examples/claude-code-memory-plugin/scripts/config.test.mjs b/examples/claude-code-memory-plugin/scripts/config.test.mjs
index b85a2630c..f7fc0115f 100644
--- a/examples/claude-code-memory-plugin/scripts/config.test.mjs
+++ b/examples/claude-code-memory-plugin/scripts/config.test.mjs
@@ -14,6 +14,8 @@ const OVERRIDES = [
"OPENVIKING_AUTO_CAPTURE",
"OPENVIKING_API_KEY",
"OPENVIKING_BEARER_TOKEN",
+ "OPENVIKING_RECALL_QUERY_FILTERS",
+ "OPENVIKING_CAPTURE_FILTERS",
];
/**
@@ -163,3 +165,67 @@ test("no api_key anywhere reports no source", () => {
assert.equal(cfg.credentialPath, null);
});
});
+
+test("input filter rules default to empty and read from the legacy ov.conf block", () => {
+ withConfigs({ ov: { server: { host: "127.0.0.1" } } }, () => {
+ assert.deepEqual(loadConfig().recallQueryFilters, []);
+ assert.deepEqual(loadConfig().captureFilters, []);
+ });
+ withConfigs({
+ ov: { server: { host: "127.0.0.1" }, claude_code: { recallQueryFilters: ["s/^hi //"] } },
+ }, () => {
+ assert.deepEqual(loadConfig().recallQueryFilters, ["s/^hi //"]);
+ });
+});
+
+test("plugin.claude_code input filters beat the shared plugin section", () => {
+ withConfigs({
+ ov: { server: { host: "127.0.0.1" } },
+ cli: {
+ url: "http://127.0.0.1:1933",
+ plugin: {
+ recallQueryFilters: ["s/shared//"],
+ captureFilters: ["d/^shared/"],
+ claude_code: { recallQueryFilters: ["s/harness//"] },
+ },
+ },
+ }, () => {
+ const cfg = loadConfig();
+ assert.deepEqual(cfg.recallQueryFilters, ["s/harness//"]);
+ assert.deepEqual(cfg.captureFilters, ["d/^shared/"]);
+ });
+});
+
+test("an env CSV replaces the configured filter list entirely", () => {
+ withConfigs({
+ ov: { server: { host: "127.0.0.1" }, claude_code: { recallQueryFilters: ["s/^hi //"] } },
+ env: { OPENVIKING_RECALL_QUERY_FILTERS: " s/^a// , d/^b/ " },
+ }, () => {
+ assert.deepEqual(loadConfig().recallQueryFilters, ["s/^a//", "d/^b/"]);
+ });
+});
+
+test("filter entries are trimmed and non-strings dropped on both paths", () => {
+ withConfigs({
+ ov: {
+ server: { host: "127.0.0.1" },
+ claude_code: { captureFilters: [" s/^a// ", "", 7, null, "d/^b/"] },
+ },
+ }, () => {
+ assert.deepEqual(loadConfig().captureFilters, ["s/^a//", "d/^b/"]);
+ });
+});
+
+test("a comma survives in a configured rule but splits an env one", () => {
+ withConfigs({
+ ov: { server: { host: "127.0.0.1" }, claude_code: { captureFilters: ["s/a{2,}/X/"] } },
+ }, () => {
+ assert.deepEqual(loadConfig().captureFilters, ["s/a{2,}/X/"]);
+ });
+ withConfigs({
+ ov: { server: { host: "127.0.0.1" } },
+ env: { OPENVIKING_CAPTURE_FILTERS: "s/a{2,}/X/" },
+ }, () => {
+ assert.deepEqual(loadConfig().captureFilters, ["s/a{2", "}/X/"]);
+ });
+});
diff --git a/examples/claude-code-memory-plugin/scripts/ov-memory-doctor.mjs b/examples/claude-code-memory-plugin/scripts/ov-memory-doctor.mjs
index ea7357d9c..dbed4fedc 100644
--- a/examples/claude-code-memory-plugin/scripts/ov-memory-doctor.mjs
+++ b/examples/claude-code-memory-plugin/scripts/ov-memory-doctor.mjs
@@ -52,6 +52,7 @@ import {
lintPeerScopeDowngrade,
WORKSPACE_PEER_HINT,
} from "./shared/doctor-core.mjs";
+import { describeInputFilters } from "./shared/input-filters.mjs";
import { isBypassed } from "./shared/session-model.mjs";
import { resolveEffectivePeerId } from "./shared/workspace-peer.mjs";
@@ -409,6 +410,20 @@ function checkConfig(report, cfg) {
const hit = isBypassed(cfg, { cwd: process.cwd() });
report[hit ? "warn" : "info"](`bypass patterns: ${cfg.bypassSessionPatterns.join(", ")}${hit ? " — MATCH the current cwd" : ""}`, hit ? "recall/capture are skipped in this directory" : "", hit ? "narrow OPENVIKING_BYPASS_SESSION_PATTERNS" : "");
}
+ for (const filters of describeInputFilters(cfg)) {
+ if (!filters.total) continue;
+ report.info(`${filters.label} ${filters.summary}`);
+ for (const e of filters.errors) {
+ const where = `${filters.env} or ovcli.conf plugin.claude_code.${filters.key}`;
+ report.warn(
+ `${filters.key}[${e.index}]: ${e.message}`,
+ e.source ? `rule: ${e.source}` : "this rule is skipped, the rest still apply",
+ e.message.startsWith("invalid regular expression")
+ ? `fix the pattern in ${where} (the u flag rejects escapes that are legal without it)`
+ : `fix the rule in ${where}`,
+ );
+ }
+ }
report.info(`debug log ${cfg.debug ? "on" : "off"} → ${homeShort(cfg.debugLogPath)}${cfg.debug ? "" : " (set OPENVIKING_DEBUG=1 in Claude Code's environment to record hook errors)"}`);
// Environment sweep
diff --git a/examples/claude-code-memory-plugin/scripts/shared/capture-utils.mjs b/examples/claude-code-memory-plugin/scripts/shared/capture-utils.mjs
index 38e0baecc..22bc1b63c 100644
--- a/examples/claude-code-memory-plugin/scripts/shared/capture-utils.mjs
+++ b/examples/claude-code-memory-plugin/scripts/shared/capture-utils.mjs
@@ -1,4 +1,6 @@
// GENERATED FROM examples/memory-plugin-shared/lib. DO NOT EDIT.
+import { applyInputFilters, compileInputFilters } from "./input-filters.mjs";
+
const TEXT_BLOCK_TYPES = new Set(["text", "input_text", "output_text"]);
const TOOL_CALL_TYPES = new Set([
"tool_call",
@@ -354,6 +356,53 @@ export function extractPartsFromPayload(payload, options = {}) {
return parts;
}
+/**
+ * Prune blank text parts and run `cfg.captureFilters` over the ones that remain.
+ *
+ * One drop verdict is taken per turn, on the aggregate of its text parts; the
+ * individual parts are then rewritten with the substitutions only, so a rule can
+ * never both keep and drop the same turn. Tool call/result parts are carried
+ * through untouched — filters match conversation text, not tool payloads — but a
+ * turn dropped on its text takes its tool parts with it.
+ */
+export function filterCaptureParts(parts, role, cfg = {}) {
+ const kept = [];
+ for (const part of parts || []) {
+ if (!part) continue;
+ if (part.type !== "text") {
+ kept.push(part);
+ continue;
+ }
+ const text = typeof part.text === "string" ? part.text.trim() : "";
+ if (!text) continue;
+ kept.push(text === part.text ? part : { ...part, text });
+ }
+
+ const blank = { parts: kept, dropped: false };
+ const compiled = compileInputFilters(cfg?.captureFilters);
+ if (!compiled.rules.length || kept.length === 0) return blank;
+
+ const textParts = kept.filter((part) => part.type === "text");
+ if (textParts.length === 0) return blank;
+
+ const verdict = applyInputFilters(textParts.map((part) => part.text).join("\n\n"), compiled.rules, {
+ role,
+ });
+ if (verdict.dropped) return { parts: [], dropped: true };
+
+ const out = [];
+ for (const part of kept) {
+ if (part.type !== "text") {
+ out.push(part);
+ continue;
+ }
+ const shaped = applyInputFilters(part.text, compiled.rules, { role, substituteOnly: true });
+ if (!shaped.text) continue;
+ out.push(shaped.text === part.text ? part : { ...part, text: shaped.text });
+ }
+ return { parts: out, dropped: out.length === 0 };
+}
+
export function extractCaptureTurns(rolloutEntries, cfg = {}) {
const toolNameById = collectToolNamesByIdFromEntries(rolloutEntries);
const turns = [];
@@ -371,10 +420,15 @@ export function extractCaptureTurns(rolloutEntries, cfg = {}) {
toolMaxChars: cfg.captureToolMaxChars,
toolNameById,
});
- const decision = shouldCaptureText(rawText, role, cfg);
- if (!decision.shouldCapture && parts.length === 0) continue;
+ const shaped = filterCaptureParts(parts, role, cfg);
+ if (shaped.dropped) continue;
+ // With parts on the wire the drop decision was already taken above, so the
+ // text path only runs the filters for the `content` fallback. That also
+ // keeps `turn.text` faithful for callers that scan it for trigger words.
+ const decision = shouldCaptureText(rawText, role, cfg, { filters: shaped.parts.length === 0 });
+ if (!decision.shouldCapture && shaped.parts.length === 0) continue;
const text = decision.shouldCapture ? decision.text : "";
- turns.push({ role, text, parts });
+ turns.push({ role, text, parts: shaped.parts });
}
return turns;
}
@@ -486,12 +540,21 @@ function isPunctuationOnly(text) {
return !/[a-z0-9\u3400-\u9fff]/i.test(text);
}
-export function shouldCaptureText(text, role, cfg = {}) {
+export function shouldCaptureText(text, role, cfg = {}, { filters = true } = {}) {
const maxLength = cfg.captureMaxLength || 24000;
const sanitized = sanitizeCapturedText(text);
if (!sanitized) return { shouldCapture: false, reason: "empty", text: "" };
- const capped = truncateCaptureText(sanitized, maxLength);
+ let capped = truncateCaptureText(sanitized, maxLength);
+ if (filters) {
+ const compiled = compileInputFilters(cfg?.captureFilters);
+ if (compiled.rules.length) {
+ const verdict = applyInputFilters(capped, compiled.rules, { role });
+ if (verdict.dropped) return { shouldCapture: false, reason: "filtered", text: "" };
+ capped = verdict.text;
+ if (!capped) return { shouldCapture: false, reason: "empty", text: "" };
+ }
+ }
const compact = oneLine(capped);
const isToolSummary = /^\[tool-(?:call|result)\b/i.test(compact);
diff --git a/examples/claude-code-memory-plugin/scripts/shared/doctor-core.mjs b/examples/claude-code-memory-plugin/scripts/shared/doctor-core.mjs
index 7c9650607..47887e6e8 100644
--- a/examples/claude-code-memory-plugin/scripts/shared/doctor-core.mjs
+++ b/examples/claude-code-memory-plugin/scripts/shared/doctor-core.mjs
@@ -326,6 +326,7 @@ export const KNOWN_PLUGIN_KEYS = new Set([
"recallCompressMinInputChars", "recallCompressBaseUrl", "recallCompressModel",
"recallCompressThinking", "recallCompressReasoningEffort", "recallCompressTimeoutMs",
"recallCompressDetectOnStartup", "recallCompressDetectTimeoutMs", "recallCompressDetectTtlMs",
+ "recallQueryFilters", "captureFilters",
"captureMode", "captureMaxLength", "captureTimeoutMs", "captureToolMaxChars",
"captureAssistantTurns", "captureLastAssistantOnStop", "logRankingDetails",
"commitTokenThreshold", "commitKeepRecentCount", "autoCommitOnCompact",
diff --git a/examples/claude-code-memory-plugin/scripts/shared/input-filters.mjs b/examples/claude-code-memory-plugin/scripts/shared/input-filters.mjs
new file mode 100644
index 000000000..e20bb03a7
--- /dev/null
+++ b/examples/claude-code-memory-plugin/scripts/shared/input-filters.mjs
@@ -0,0 +1,219 @@
+// GENERATED FROM examples/memory-plugin-shared/lib. DO NOT EDIT.
+/**
+ * Ordered, operator-configured regex filters for memory plugin input.
+ *
+ * Rules are sed-style strings, applied in order to a single piece of text:
+ * s/^\s*ultrathink\s+//i substitute (add `g` to replace every match)
+ * d|^\s*[/!]| drop the text when the pattern matches
+ * k/^\?ov\b/ keep the text only when the pattern matches
+ * user:d/^\s*\/clear\b/ apply to one role only
+ *
+ * A malformed rule becomes an entry in `errors` and is skipped rather than
+ * throwing, so a typo in a config file cannot take a hook down.
+ */
+
+const OPS = new Set(["s", "d", "k"]);
+const SCOPES = ["user", "assistant"];
+const ALLOWED_FLAGS = "imsug";
+
+export const INPUT_FILTER_KNOBS = [
+ {
+ key: "recallQueryFilters",
+ env: "OPENVIKING_RECALL_QUERY_FILTERS",
+ label: "recall query filters",
+ },
+ {
+ key: "captureFilters",
+ env: "OPENVIKING_CAPTURE_FILTERS",
+ label: "capture filters",
+ },
+];
+
+/**
+ * Read one delimited field. `\` always consumes the next character, and only
+ * `\<delim>` is un-escaped, so `\d` / `\b` / `\x2c` reach RegExp verbatim.
+ * Returns the index of the closing delimiter, or -1 when it is missing.
+ */
+function scanField(source, start, delim) {
+ let out = "";
+ let i = start;
+ while (i < source.length) {
+ const c = source[i];
+ if (c === "\\") {
+ const next = source[i + 1];
+ if (next === undefined) {
+ out += "\\";
+ i += 1;
+ continue;
+ }
+ out += next === delim ? next : "\\" + next;
+ i += 2;
+ continue;
+ }
+ if (c === delim) return { value: out, end: i };
+ out += c;
+ i += 1;
+ }
+ return { value: out, end: -1 };
+}
+
+export function parseInputFilterRule(source) {
+ if (typeof source !== "string") {
+ return { source: String(source ?? ""), error: "rule must be a string" };
+ }
+ const raw = source.trim();
+ if (!raw) return { source: raw, error: "rule is empty" };
+
+ let scope = "";
+ let rest = raw;
+ for (const candidate of SCOPES) {
+ if (rest.startsWith(`${candidate}:`)) {
+ scope = candidate;
+ rest = rest.slice(candidate.length + 1);
+ break;
+ }
+ }
+
+ const op = rest[0] || "";
+ if (!OPS.has(op)) {
+ return {
+ source: raw,
+ error: `unknown operation "${op}" — expected s (substitute), d (drop) or k (keep)`,
+ };
+ }
+ const delim = rest[1];
+ if (delim === undefined) {
+ return { source: raw, error: `missing delimiter after "${op}"` };
+ }
+ if (/[A-Za-z0-9\s\\]/.test(delim)) {
+ return {
+ source: raw,
+ error: `invalid delimiter ${JSON.stringify(delim)} — use punctuation such as / | # or :`,
+ };
+ }
+
+ const first = scanField(rest, 2, delim);
+ const pattern = first.value;
+ let replacement = null;
+ let flagsRaw = "";
+ if (op === "s") {
+ if (first.end < 0) {
+ return { source: raw, error: `missing ${JSON.stringify(delim)} after the pattern` };
+ }
+ const second = scanField(rest, first.end + 1, delim);
+ if (second.end < 0) {
+ return {
+ source: raw,
+ error: `missing ${JSON.stringify(delim)} after the replacement — write s${delim}pattern${delim}replacement${delim}`,
+ };
+ }
+ replacement = second.value;
+ flagsRaw = rest.slice(second.end + 1);
+ } else {
+ flagsRaw = first.end < 0 ? "" : rest.slice(first.end + 1);
+ }
+
+ if (!pattern) return { source: raw, error: "pattern is empty" };
+
+ const flags = new Set();
+ for (const flag of flagsRaw) {
+ if (!ALLOWED_FLAGS.includes(flag)) {
+ return { source: raw, error: `unknown flag "${flag}" — allowed flags are i, m, s, u, g` };
+ }
+ if (flags.has(flag)) return { source: raw, error: `duplicate flag "${flag}"` };
+ flags.add(flag);
+ }
+ // d/k decide with .test(), which advances lastIndex on a global regex and
+ // would then alternate between calls. `g` means nothing there, so drop it.
+ if (op !== "s") flags.delete("g");
+
+ let re;
+ try {
+ re = new RegExp(pattern, [...flags].join(""));
+ } catch (err) {
+ // V8 already says "Invalid regular expression: <pattern>: <why>"; keep that
+ // detail without stuttering the prefix the doctors key off.
+ const message = err?.message || String(err);
+ return {
+ source: raw,
+ error: /^invalid regular expression/i.test(message)
+ ? `invalid${message.slice("Invalid".length)}`
+ : `invalid regular expression: ${message}`,
+ };
+ }
+
+ return { op, scope, re, replacement, source: raw };
+}
+
+export function compileInputFilters(rules) {
+ const compiled = [];
+ const errors = [];
+ const list = Array.isArray(rules) ? rules : [];
+ list.forEach((entry, index) => {
+ if (typeof entry !== "string") {
+ errors.push({ index, source: String(entry ?? ""), message: "rule must be a string" });
+ return;
+ }
+ if (!entry.trim()) return;
+ const parsed = parseInputFilterRule(entry);
+ if (parsed.error) errors.push({ index, source: parsed.source, message: parsed.error });
+ else compiled.push({ ...parsed, index });
+ });
+ return { rules: compiled, errors };
+}
+
+/**
+ * Run compiled rules over `text`.
+ *
+ * `substituteOnly` skips the d/k operations, so a caller that already took one
+ * drop decision for a turn can rewrite its individual pieces without taking a
+ * second, possibly contradictory one.
+ */
+export function applyInputFilters(text, compiled, { role = "", substituteOnly = false } = {}) {
+ const original = typeof text === "string" ? text : String(text ?? "");
+ let current = original;
+ for (const rule of Array.isArray(compiled) ? compiled : []) {
+ if (rule.scope && rule.scope !== role) continue;
+ if (rule.op === "s") {
+ current = current.replace(rule.re, rule.replacement);
+ continue;
+ }
+ if (substituteOnly) continue;
+ const matched = rule.re.test(current);
+ if ((rule.op === "d" && matched) || (rule.op === "k" && !matched)) {
+ return { text: "", changed: false, dropped: true, ruleIndex: rule.index, op: rule.op };
+ }
+ }
+ const finalText = current.trim();
+ return {
+ text: finalText,
+ changed: finalText !== original,
+ dropped: false,
+ ruleIndex: -1,
+ op: "",
+ };
+}
+
+/** Doctor-facing summary of both filter knobs on a resolved plugin config. */
+export function describeInputFilters(cfg = {}) {
+ return INPUT_FILTER_KNOBS.map(({ key, env, label }) => {
+ const configured = Array.isArray(cfg?.[key]) ? cfg[key] : [];
+ const compiled = compileInputFilters(configured);
+ const ops = { s: 0, d: 0, k: 0 };
+ for (const rule of compiled.rules) ops[rule.op] += 1;
+ const bits = [];
+ if (ops.s) bits.push(`${ops.s} substitute`);
+ if (ops.d) bits.push(`${ops.d} drop`);
+ if (ops.k) bits.push(`${ops.k} keep-only`);
+ const active = compiled.rules.length;
+ return {
+ key,
+ env,
+ label,
+ total: configured.length,
+ active,
+ summary: `${active} rule${active === 1 ? "" : "s"}${bits.length ? ` (${bits.join(", ")})` : ""}`,
+ errors: compiled.errors,
+ };
+ });
+}
diff --git a/examples/claude-code-memory-plugin/scripts/subagent-stop.mjs b/examples/claude-code-memory-plugin/scripts/subagent-stop.mjs
index f94a9cb81..b237c030a 100644
--- a/examples/claude-code-memory-plugin/scripts/subagent-stop.mjs
+++ b/examples/claude-code-memory-plugin/scripts/subagent-stop.mjs
@@ -32,6 +32,7 @@ import {
import { maybeDetach, readHookStdin } from "./lib/async-writer.mjs";
import { getEffectivePeerId } from "./lib/workspace-peer.mjs";
import { sendSessionMessages } from "./shared/batch-send.mjs";
+import { filterCaptureParts } from "./shared/capture-utils.mjs";
if (!isPluginEnabled()) {
process.stdout.write(JSON.stringify({ decision: "approve" }) + "\n");
@@ -240,9 +241,8 @@ async function pushTurns(ovSessionId, turns, { peerId = null, enqueueOnly = fals
for (const turn of turns) {
// Send structured parts: tool calls/results are dedicated `tool` parts, not
// inlined into content, so the server can process them separately.
- const parts = (turn.parts || []).filter(
- (p) => p.type !== "text" || (p.text && p.text.trim()),
- );
+ // Blank text parts are pruned and the configured capture filters applied.
+ const parts = filterCaptureParts(turn.parts, turn.role, cfg).parts;
if (parts.length === 0) continue;
const payload = { role: turn.role, parts };
if (peerId) payload.peer_id = peerId;
diff --git a/examples/codex-memory-plugin/.codex-plugin/plugin.json b/examples/codex-memory-plugin/.codex-plugin/plugin.json
index de349c661..9c8911ece 100644
--- a/examples/codex-memory-plugin/.codex-plugin/plugin.json
+++ b/examples/codex-memory-plugin/.codex-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "openviking-memory",
- "version": "0.8.2",
+ "version": "0.8.3",
"description": "Long-term semantic memory for Codex, powered by OpenViking. Recall on UserPromptSubmit, incremental add_message on Stop (per turn), commit on SessionEnd (Codex >= 0.145) and on PreCompact, with an idle/ended sweep on SessionStart as the fallback for signals, crashes and older Codex builds. Includes a local stdio MCP proxy that reads OpenViking credentials from env/ovcli.conf.",
"author": {
"name": "OpenViking",
diff --git a/examples/codex-memory-plugin/README.md b/examples/codex-memory-plugin/README.md
index ba80926fb..a6a6f9c81 100644
--- a/examples/codex-memory-plugin/README.md
+++ b/examples/codex-memory-plugin/README.md
@@ -162,6 +162,33 @@ export OPENVIKING_DEBUG=1
Full list: see the `Misc env vars` block in `scripts/config.mjs`. Tuning fields have `OPENVIKING_*` counterparts and env vars win for those tuning fields.
+#### Input filters
+
+Two knobs put an ordered list of regex rules in front of the text the plugin sends: `recallQueryFilters` / `OPENVIKING_RECALL_QUERY_FILTERS` shapes the prompt before it becomes a search query, and `captureFilters` / `OPENVIKING_CAPTURE_FILTERS` shapes every turn on the write path before it is stored.
+
+Rules are sed-style strings applied in order to one piece of text: `s<d>pattern<d>replacement<d>[flags]` substitutes, `d<d>pattern<d>[flags]` drops the text on a match, and `k<d>pattern<d>[flags]` keeps it only on a match (chain them for AND). `<d>` is any punctuation delimiter — `/`, `|`, `#`, `:` — escaped with `\` inside the pattern; flags are `i`, `m`, `s`, `u`, `g`. A `user:` or `assistant:` prefix limits a rule to that role.
+
+```sh
+# strip a thinking-keyword prefix, and don't recall on slash / bash-mode prompts
+export OPENVIKING_RECALL_QUERY_FILTERS='s/^\s*(ultrathink|think harder?)\s+//i,d|^\s*[/!]|'
+# redact tokens before they are stored, and never store /clear or /compact turns
+export OPENVIKING_CAPTURE_FILTERS='s/\b(sk|ghp|xoxb)_[A-Za-z0-9_-]+/[redacted]/g,user:d/^\s*\/(clear|compact)\b/'
+```
+
+The env vars are comma-separated lists, split before parsing, so a rule needing a literal comma — a bounded `{10,}` quantifier, say — belongs in the `ovcli.conf` array instead, where only trimming happens:
+
+```json
+{
+ "plugin": {
+ "codex": {
+ "captureFilters": ["s/\\b(sk|ghp)_[A-Za-z0-9_-]{10,}/[redacted]/g"]
+ }
+ }
+}
+```
+
+Rules run top to bottom and the first `d` that matches (or `k` that does not) ends the decision; text a substitution empties is not a drop, just too short to recall on. Filters run before `OPENVIKING_MIN_QUERY_LENGTH` and before the built-in ack / slash-command heuristics. A capture rule shapes what is sent, not what is already stored, and anything already in the pending queue carries the rules that were in effect when it was enqueued; adding a `d`/`k` rule mid-session also shortens the turn list the cursor counts, which reads as a transcript rewrite and replays from the last user turn. A rule that fails to compile is skipped, never fatal — `ov-memory-doctor` lists the active rules and reports the exact error for the ones it could not parse.
+
#### Workspace configuration files
A repository can carry its own plugin settings in `<repo-root>/.openviking/config.json`, which the team commits, and `<repo-root>/.openviking/config.local.json`, which stays private and gitignored. A third layer, this machine's entry under `~/.openviking/workspaces/`, outranks both, and all three outrank `ovcli.conf`.
@@ -277,6 +304,7 @@ Config knobs:
| `OPENVIKING_RECALL_MAX_TOKENS` | `1600` | Token budget the server assembles the context block within, independent of the local compressor input limit. |
| `OPENVIKING_RECALL_DEDUP_TURNS` | `5` | Cross-turn cooldown: URIs served in the last N turns are skipped. |
| `OPENVIKING_RECALL_QUERY_EXPANSION` | `auto` | `auto` lets the server widen short prompts using session context; `off` disables it. |
+| `OPENVIKING_RECALL_QUERY_FILTERS` | `""` | Comma-separated regex rules applied to the prompt before it becomes a query — see [Input filters](#input-filters). |
Recall now asks the server to assemble the context block in one request
(`POST /api/v1/search/search` with `mode="context"`), so budgeting, detail tiers
@@ -297,7 +325,7 @@ layers; resolution order is env vars → the workspace layers → `plugin.codex`
### Stop (turn end → `add_message`, threshold commit)
-`auto-capture.mjs` derives one long-lived OpenViking session id per Codex `session_id` as `cx-<safe-session-id>` and incrementally appends every new user/assistant turn via `/api/v1/sessions/{id}/messages`. The `/messages` endpoint auto-creates the session on first append. Per-codex-session state lives at `~/.openviking/codex-plugin-state/<safe-session-id>.json`. Capture sanitizes obvious hook noise, metadata wrappers, and plugin-injected `<openviking-context ...>` blocks before append. Tool calls and results become dedicated `tool` parts and `tool_output` is reported verbatim — the server externalizes anything larger than `tool_output_externalization.threshold_chars` (default `20000`) and leaves a synopsis stub plus `tool_output_ref`, so the original stays readable via `/api/v1/sessions/{id}/tool-results`. `OPENVIKING_CAPTURE_TOOL_MAX_CHARS` (default `1000000`) is only a guard against pathological payloads.
+`auto-capture.mjs` derives one long-lived OpenViking session id per Codex `session_id` as `cx-<safe-session-id>` and incrementally appends every new user/assistant turn via `/api/v1/sessions/{id}/messages`. The `/messages` endpoint auto-creates the session on first append. Per-codex-session state lives at `~/.openviking/codex-plugin-state/<safe-session-id>.json`. Capture sanitizes obvious hook noise, metadata wrappers, and plugin-injected `<openviking-context ...>` blocks before append. Tool calls and results become dedicated `tool` parts and `tool_output` is reported verbatim — the server externalizes anything larger than `tool_output_externalization.threshold_chars` (default `20000`) and leaves a synopsis stub plus `tool_output_ref`, so the original stays readable via `/api/v1/sessions/{id}/tool-results`. `OPENVIKING_CAPTURE_TOOL_MAX_CHARS` (default `1000000`) is only a guard against pathological payloads. Configured `captureFilters` rules run last, just before the payload is sent — see [Input filters](#input-filters).
After a successful append, Stop reads the session meta and commits when `pending_tokens >= OPENVIKING_COMMIT_TOKEN_THRESHOLD` (default `20000`). Threshold commits pass `keep_recent_count=OPENVIKING_COMMIT_KEEP_RECENT_COUNT` (default `10`) so the newest turns remain live for continuity while older context is archived and extracted. `PreCompact` still commits everything before compaction.
diff --git a/examples/codex-memory-plugin/scripts/auto-capture.test.mjs b/examples/codex-memory-plugin/scripts/auto-capture.test.mjs
index 353b470af..4a9c7ce4a 100644
--- a/examples/codex-memory-plugin/scripts/auto-capture.test.mjs
+++ b/examples/codex-memory-plugin/scripts/auto-capture.test.mjs
@@ -696,3 +696,59 @@ test("the workspace that decides capture is the payload's, not the hook process'
await rm(stateDir, { recursive: true, force: true });
}
});
+
+test("capture filters rewrite and drop turns without stranding the cursor", async () => {
+ const stateDir = await mkdtemp(join(tmpdir(), "ov-auto-capture-filters-"));
+ const transcriptPath = join(stateDir, "transcript.jsonl");
+ const batches = [];
+
+ try {
+ await writeFile(
+ transcriptPath,
+ [
+ JSON.stringify({ payload: { message: { role: "user", content: "the token is sk_LIVE_ABCDEF, remember it" } } }),
+ JSON.stringify({ payload: { message: { role: "user", content: "scratch: ignore this throwaway note" } } }),
+ JSON.stringify({ payload: { message: { role: "assistant", content: "noted for future sessions" } } }),
+ ].join("\n"),
+ );
+
+ await withMockOpenViking(async (req, res) => {
+ const url = new URL(req.url, "http://127.0.0.1");
+ if (req.method === "POST" && url.pathname.endsWith("/messages/batch")) {
+ batches.push(await readRequestBody(req));
+ writeJson(res, { status: "ok", result: { ok: true } });
+ return;
+ }
+ writeJson(res, { status: "ok", result: { ok: true } });
+ }, async (baseUrl) => {
+ const env = {
+ OPENVIKING_AUTO_CAPTURE: "1",
+ OPENVIKING_CAPTURE_ASSISTANT_TURNS: "1",
+ OPENVIKING_CODEX_STATE_DIR: stateDir,
+ OPENVIKING_CONFIG_FILE: join(stateDir, "missing-ov.conf"),
+ OPENVIKING_CLI_CONFIG_FILE: join(stateDir, "missing-ovcli.conf"),
+ OPENVIKING_CREDENTIAL_SOURCE: "env",
+ OPENVIKING_WRITE_PATH_ASYNC: "0",
+ OPENVIKING_TIMEOUT_MS: "5000",
+ OPENVIKING_URL: baseUrl,
+ OPENVIKING_CAPTURE_FILTERS: "s/sk_[A-Za-z0-9_]+/[redacted]/g,user:d/^scratch:/",
+ };
+ const input = { session_id: "codex:filters", transcript_path: transcriptPath };
+ await runAutoCapture(input, env);
+ // A second run over the same transcript must find nothing new: the
+ // dropped turn still counts against the cursor.
+ await runAutoCapture(input, env);
+ });
+
+ assert.equal(batches.length, 1);
+ const texts = batches[0].messages.flatMap(
+ (message) => (message.parts || []).filter((p) => p.type === "text").map((p) => p.text),
+ );
+ assert.deepEqual(texts, [
+ "the token is [redacted], remember it",
+ "noted for future sessions",
+ ]);
+ } finally {
+ await rm(stateDir, { recursive: true, force: true });
+ }
+});
diff --git a/examples/codex-memory-plugin/scripts/auto-recall.mjs b/examples/codex-memory-plugin/scripts/auto-recall.mjs
index 768744341..fdceb4342 100644
--- a/examples/codex-memory-plugin/scripts/auto-recall.mjs
+++ b/examples/codex-memory-plugin/scripts/auto-recall.mjs
@@ -33,6 +33,7 @@ import {
postRecall,
} from "./shared/recall-core.mjs";
import { compressRecallContext } from "./shared/recall-compress-core.mjs";
+import { applyInputFilters, compileInputFilters } from "./shared/input-filters.mjs";
import { resolveEffectivePeerId } from "./shared/workspace-peer.mjs";
let cfg = loadConfig();
@@ -576,7 +577,7 @@ async function main() {
return;
}
- const userPrompt = (input.prompt || "").trim();
+ let userPrompt = (input.prompt || "").trim();
const codexSessionId = typeof input.session_id === "string" ? input.session_id.trim() : "";
const recallSessionId = resolveRecallSessionId(codexSessionId);
log("start", {
@@ -592,6 +593,21 @@ async function main() {
},
});
+ // Filters run before the length gate, so a prompt whose only content was a
+ // stripped prefix is a short query rather than a search for the empty string.
+ const queryFilters = compileInputFilters(cfg.recallQueryFilters);
+ if (queryFilters.rules.length) {
+ const verdict = applyInputFilters(userPrompt, queryFilters.rules, { role: "user" });
+ if (verdict.dropped) {
+ log("skip", { stage: "query_filter", reason: "query_filtered", rule: verdict.ruleIndex, op: verdict.op });
+ emit();
+ return;
+ }
+ if (verdict.changed) log("query_filter", { rawLength: userPrompt.length, length: verdict.text.length });
+ userPrompt = verdict.text;
+ }
+ if (queryFilters.errors.length) log("query_filter_errors", { errors: queryFilters.errors });
+
if (!userPrompt || userPrompt.length < cfg.minQueryLength) {
log("skip", { stage: "query_check", reason: "query too short or empty" });
emit();
diff --git a/examples/codex-memory-plugin/scripts/auto-recall.test.mjs b/examples/codex-memory-plugin/scripts/auto-recall.test.mjs
index e365c32ed..a6aa7fd5f 100644
--- a/examples/codex-memory-plugin/scripts/auto-recall.test.mjs
+++ b/examples/codex-memory-plugin/scripts/auto-recall.test.mjs
@@ -834,3 +834,86 @@ test("the actor peer comes from the workspace named by the payload's cwd", async
await rm(stateDir, { recursive: true, force: true });
}
});
+
+function filterEnv(stateDir, baseUrl, filters) {
+ return {
+ OPENVIKING_AUTO_RECALL: "1",
+ OPENVIKING_CODEX_STATE_DIR: stateDir,
+ OPENVIKING_STATE_DIR: stateDir,
+ OPENVIKING_CONFIG_FILE: join(stateDir, "missing-ov.conf"),
+ OPENVIKING_CLI_CONFIG_FILE: join(stateDir, "missing-ovcli.conf"),
+ OPENVIKING_CREDENTIAL_SOURCE: "env",
+ OPENVIKING_RECALL_COMPRESS: "0",
+ OPENVIKING_RECALL_TIMEOUT_MS: "10000",
+ OPENVIKING_MIN_QUERY_LENGTH: "3",
+ OPENVIKING_SCORE_THRESHOLD: "0",
+ OPENVIKING_TIMEOUT_MS: "5000",
+ OPENVIKING_URL: baseUrl,
+ OPENVIKING_RECALL_QUERY_FILTERS: filters,
+ };
+}
+
+test("a recall query filter drop ends the turn before any request", async () => {
+ const stateDir = await mkdtemp(join(tmpdir(), "ov-auto-recall-filter-drop-"));
+ const requests = [];
+ try {
+ await withMockOpenViking(async (req, res) => {
+ requests.push(req.url);
+ writeJson(res, { status: "ok", result: {} });
+ }, async (baseUrl) => {
+ const result = await runAutoRecall(
+ { prompt: "/status please", session_id: "codex:filter-drop" },
+ filterEnv(stateDir, baseUrl, "d|^\\s*[/!]|"),
+ );
+ assert.deepEqual(JSON.parse(result.stdout.trim()), {});
+ });
+ assert.deepEqual(requests, []);
+ } finally {
+ await rm(stateDir, { recursive: true, force: true });
+ }
+});
+
+test("a recall query filter rewrites the query the server is asked for", async () => {
+ const stateDir = await mkdtemp(join(tmpdir(), "ov-auto-recall-filter-sub-"));
+ const queries = [];
+ try {
+ await withMockOpenViking(async (req, res) => {
+ const url = new URL(req.url, "http://127.0.0.1");
+ if (req.method === "GET" && url.pathname === "/health") {
+ writeJson(res, { status: "ok", result: { ok: true } });
+ return;
+ }
+ const body = await readRequestBody(req);
+ if (body?.query) queries.push(body.query);
+ writeJson(res, { status: "ok", result: { entries: [], rendered: "", stats: {} } });
+ }, async (baseUrl) => {
+ await runAutoRecall(
+ { prompt: "ultrathink what did we decide about retries", session_id: "codex:filter-sub" },
+ filterEnv(stateDir, baseUrl, "s/^\\s*ultrathink\\s+//i"),
+ );
+ });
+ assert.ok(queries.length > 0, "the hook made no query request");
+ for (const query of queries) assert.equal(query, "what did we decide about retries");
+ } finally {
+ await rm(stateDir, { recursive: true, force: true });
+ }
+});
+
+test("a query stripped to nothing never reaches the server", async () => {
+ const stateDir = await mkdtemp(join(tmpdir(), "ov-auto-recall-filter-empty-"));
+ const requests = [];
+ try {
+ await withMockOpenViking(async (req, res) => {
+ requests.push(req.url);
+ writeJson(res, { status: "ok", result: {} });
+ }, async (baseUrl) => {
+ await runAutoRecall(
+ { prompt: "ultrathink", session_id: "codex:filter-empty" },
+ filterEnv(stateDir, baseUrl, "s/^ultrathink$//"),
+ );
+ });
+ assert.deepEqual(requests, []);
+ } finally {
+ await rm(stateDir, { recursive: true, force: true });
+ }
+});
diff --git a/examples/codex-memory-plugin/scripts/config.mjs b/examples/codex-memory-plugin/scripts/config.mjs
index 90761af0d..72a29ac7b 100644
--- a/examples/codex-memory-plugin/scripts/config.mjs
+++ b/examples/codex-memory-plugin/scripts/config.mjs
@@ -39,6 +39,7 @@
* OPENVIKING_RECALL_LIMIT, OPENVIKING_SCORE_THRESHOLD
* OPENVIKING_WORKSPACE_PEER, OPENVIKING_RECALL_PEER_SCOPE
* OPENVIKING_NO_AUTO_INJECT, OPENVIKING_PROFILE_TOKEN_BUDGET
+ * OPENVIKING_RECALL_QUERY_FILTERS (CSV), OPENVIKING_CAPTURE_FILTERS (CSV)
* OPENVIKING_DEBUG=1, OPENVIKING_DEBUG_LOG
*/
@@ -90,6 +91,18 @@ function hasOwn(obj, key) {
return Object.prototype.hasOwnProperty.call(obj || {}, key);
}
+/**
+ * A configured list of input filter rules. An env CSV replaces the configured
+ * array entirely; both paths trim, so a rule reads the same however it was
+ * configured — and a rule needing a literal comma has to come from the array,
+ * since the env value is split on one.
+ */
+function filterList(envName, configured) {
+ const raw = str(process.env[envName], null);
+ const list = raw !== null ? raw.split(",") : (Array.isArray(configured) ? configured : []);
+ return list.filter((r) => typeof r === "string").map((r) => r.trim()).filter(Boolean);
+}
+
function normalizeAuthMode(val) {
const mode = str(val, "").toLowerCase();
return ["trusted", "api_key"].includes(mode) ? mode : "";
@@ -214,6 +227,10 @@ export function loadConfig(cwd = process.cwd()) {
process.env.OPENVIKING_MIN_QUERY_LENGTH,
num(cx.minQueryLength, 3),
))),
+ // Ordered sed-style rules over the text the plugin sends: the recall query
+ // on the UserPromptSubmit hook, and every captured turn on the write path.
+ recallQueryFilters: filterList("OPENVIKING_RECALL_QUERY_FILTERS", cx.recallQueryFilters),
+ captureFilters: filterList("OPENVIKING_CAPTURE_FILTERS", cx.captureFilters),
logRankingDetails: envBool("OPENVIKING_LOG_RANKING_DETAILS") ?? (cx.logRankingDetails === true),
recallPeerScope,
recallCompress: envBool("OPENVIKING_RECALL_COMPRESS") ?? configBool(cx.recallCompress, true),
diff --git a/examples/codex-memory-plugin/scripts/config.test.mjs b/examples/codex-memory-plugin/scripts/config.test.mjs
index d9c769f72..4032538e7 100644
--- a/examples/codex-memory-plugin/scripts/config.test.mjs
+++ b/examples/codex-memory-plugin/scripts/config.test.mjs
@@ -19,6 +19,8 @@ const OVERRIDES = [
"OPENVIKING_BASE_URL",
"OPENVIKING_API_KEY",
"OPENVIKING_BEARER_TOKEN",
+ "OPENVIKING_RECALL_QUERY_FILTERS",
+ "OPENVIKING_CAPTURE_FILTERS",
];
/**
@@ -140,3 +142,67 @@ test("an omitted cwd falls back to this process's directory", () => {
}
});
});
+
+test("input filter rules default to empty and read from the legacy ov.conf block", () => {
+ withConfigs({ ov: { server: { host: "127.0.0.1" } } }, () => {
+ assert.deepEqual(loadConfig().recallQueryFilters, []);
+ assert.deepEqual(loadConfig().captureFilters, []);
+ });
+ withConfigs({
+ ov: { server: { host: "127.0.0.1" }, codex: { captureFilters: ["d/^scratch:/"] } },
+ }, () => {
+ assert.deepEqual(loadConfig().captureFilters, ["d/^scratch:/"]);
+ });
+});
+
+test("plugin.codex input filters beat the shared plugin section", () => {
+ withConfigs({
+ ov: { server: { host: "127.0.0.1" } },
+ cli: {
+ url: "http://127.0.0.1:1933",
+ plugin: {
+ recallQueryFilters: ["s/shared//"],
+ captureFilters: ["d/^shared/"],
+ codex: { recallQueryFilters: ["s/harness//"] },
+ },
+ },
+ }, () => {
+ const cfg = loadConfig();
+ assert.deepEqual(cfg.recallQueryFilters, ["s/harness//"]);
+ assert.deepEqual(cfg.captureFilters, ["d/^shared/"]);
+ });
+});
+
+test("an env CSV replaces the configured filter list, trimming every entry", () => {
+ withConfigs({
+ ov: { server: { host: "127.0.0.1" }, codex: { recallQueryFilters: ["s/^hi //"] } },
+ env: { OPENVIKING_RECALL_QUERY_FILTERS: " s/^a// , d/^b/ " },
+ }, () => {
+ assert.deepEqual(loadConfig().recallQueryFilters, ["s/^a//", "d/^b/"]);
+ });
+});
+
+test("configured filter entries are trimmed and non-strings dropped", () => {
+ withConfigs({
+ ov: {
+ server: { host: "127.0.0.1" },
+ codex: { captureFilters: [" s/^a// ", "", 7, null, "d/^b/"] },
+ },
+ }, () => {
+ assert.deepEqual(loadConfig().captureFilters, ["s/^a//", "d/^b/"]);
+ });
+});
+
+test("a comma survives in a configured rule but splits an env one", () => {
+ withConfigs({
+ ov: { server: { host: "127.0.0.1" }, codex: { captureFilters: ["s/a{2,}/X/"] } },
+ }, () => {
+ assert.deepEqual(loadConfig().captureFilters, ["s/a{2,}/X/"]);
+ });
+ withConfigs({
+ ov: { server: { host: "127.0.0.1" } },
+ env: { OPENVIKING_CAPTURE_FILTERS: "s/a{2,}/X/" },
+ }, () => {
+ assert.deepEqual(loadConfig().captureFilters, ["s/a{2", "}/X/"]);
+ });
+});
diff --git a/examples/codex-memory-plugin/scripts/ov-memory-doctor.mjs b/examples/codex-memory-plugin/scripts/ov-memory-doctor.mjs
index c44ff35f7..1262e90ab 100644
--- a/examples/codex-memory-plugin/scripts/ov-memory-doctor.mjs
+++ b/examples/codex-memory-plugin/scripts/ov-memory-doctor.mjs
@@ -49,6 +49,7 @@ import {
lintPeerScopeDowngrade,
WORKSPACE_PEER_HINT,
} from "./shared/doctor-core.mjs";
+import { describeInputFilters } from "./shared/input-filters.mjs";
import { resolveEffectivePeerId } from "./shared/workspace-peer.mjs";
const PLUGIN_ROOT = resolvePath(dirname(fileURLToPath(import.meta.url)), "..");
@@ -455,6 +456,20 @@ function checkConfig(report, cfg) {
const toggles = [`auto-inject ${cfg.noAutoInject ? "OFF" : "on"}`, `auto-recall ${cfg.autoRecall ? "on" : "OFF"}`, `auto-capture ${cfg.autoCapture ? "on" : "OFF"}`, `commit on compact ${cfg.autoCommitOnCompact ? "on" : "OFF"}`, `recall compress ${cfg.recallCompress ? "on" : "off"}`, `write path ${cfg.writePathAsync ? "async" : "sync"}`];
report.info(`toggles ${toggles.join(", ")}`);
if (!cfg.autoRecall || !cfg.autoCapture || cfg.noAutoInject) report.warn("one or more injection paths are switched off", toggles.join(", "), "check OPENVIKING_AUTO_RECALL / OPENVIKING_AUTO_CAPTURE / OPENVIKING_NO_AUTO_INJECT and ovcli.conf plugin.codex");
+ for (const filters of describeInputFilters(cfg)) {
+ if (!filters.total) continue;
+ report.info(`${filters.label} ${filters.summary}`);
+ for (const e of filters.errors) {
+ const where = `${filters.env} or ovcli.conf plugin.codex.${filters.key}`;
+ report.warn(
+ `${filters.key}[${e.index}]: ${e.message}`,
+ e.source ? `rule: ${e.source}` : "this rule is skipped, the rest still apply",
+ e.message.startsWith("invalid regular expression")
+ ? `fix the pattern in ${where} (the u flag rejects escapes that are legal without it)`
+ : `fix the rule in ${where}`,
+ );
+ }
+ }
report.info(`debug log ${cfg.debug ? "on" : "off"} → ${homeShort(cfg.debugLogPath)}${cfg.debug ? "" : " (set OPENVIKING_DEBUG=1 in Codex's environment to record hook errors)"}`);
const env = collectEnv();
diff --git a/examples/codex-memory-plugin/scripts/shared/capture-utils.mjs b/examples/codex-memory-plugin/scripts/shared/capture-utils.mjs
index 38e0baecc..22bc1b63c 100644
--- a/examples/codex-memory-plugin/scripts/shared/capture-utils.mjs
+++ b/examples/codex-memory-plugin/scripts/shared/capture-utils.mjs
@@ -1,4 +1,6 @@
// GENERATED FROM examples/memory-plugin-shared/lib. DO NOT EDIT.
+import { applyInputFilters, compileInputFilters } from "./input-filters.mjs";
+
const TEXT_BLOCK_TYPES = new Set(["text", "input_text", "output_text"]);
const TOOL_CALL_TYPES = new Set([
"tool_call",
@@ -354,6 +356,53 @@ export function extractPartsFromPayload(payload, options = {}) {
return parts;
}
+/**
+ * Prune blank text parts and run `cfg.captureFilters` over the ones that remain.
+ *
+ * One drop verdict is taken per turn, on the aggregate of its text parts; the
+ * individual parts are then rewritten with the substitutions only, so a rule can
+ * never both keep and drop the same turn. Tool call/result parts are carried
+ * through untouched — filters match conversation text, not tool payloads — but a
+ * turn dropped on its text takes its tool parts with it.
+ */
+export function filterCaptureParts(parts, role, cfg = {}) {
+ const kept = [];
+ for (const part of parts || []) {
+ if (!part) continue;
+ if (part.type !== "text") {
+ kept.push(part);
+ continue;
+ }
+ const text = typeof part.text === "string" ? part.text.trim() : "";
+ if (!text) continue;
+ kept.push(text === part.text ? part : { ...part, text });
+ }
+
+ const blank = { parts: kept, dropped: false };
+ const compiled = compileInputFilters(cfg?.captureFilters);
+ if (!compiled.rules.length || kept.length === 0) return blank;
+
+ const textParts = kept.filter((part) => part.type === "text");
+ if (textParts.length === 0) return blank;
+
+ const verdict = applyInputFilters(textParts.map((part) => part.text).join("\n\n"), compiled.rules, {
+ role,
+ });
+ if (verdict.dropped) return { parts: [], dropped: true };
+
+ const out = [];
+ for (const part of kept) {
+ if (part.type !== "text") {
+ out.push(part);
+ continue;
+ }
+ const shaped = applyInputFilters(part.text, compiled.rules, { role, substituteOnly: true });
+ if (!shaped.text) continue;
+ out.push(shaped.text === part.text ? part : { ...part, text: shaped.text });
+ }
+ return { parts: out, dropped: out.length === 0 };
+}
+
export function extractCaptureTurns(rolloutEntries, cfg = {}) {
const toolNameById = collectToolNamesByIdFromEntries(rolloutEntries);
const turns = [];
@@ -371,10 +420,15 @@ export function extractCaptureTurns(rolloutEntries, cfg = {}) {
toolMaxChars: cfg.captureToolMaxChars,
toolNameById,
});
- const decision = shouldCaptureText(rawText, role, cfg);
- if (!decision.shouldCapture && parts.length === 0) continue;
+ const shaped = filterCaptureParts(parts, role, cfg);
+ if (shaped.dropped) continue;
+ // With parts on the wire the drop decision was already taken above, so the
+ // text path only runs the filters for the `content` fallback. That also
+ // keeps `turn.text` faithful for callers that scan it for trigger words.
+ const decision = shouldCaptureText(rawText, role, cfg, { filters: shaped.parts.length === 0 });
+ if (!decision.shouldCapture && shaped.parts.length === 0) continue;
const text = decision.shouldCapture ? decision.text : "";
- turns.push({ role, text, parts });
+ turns.push({ role, text, parts: shaped.parts });
}
return turns;
}
@@ -486,12 +540,21 @@ function isPunctuationOnly(text) {
return !/[a-z0-9\u3400-\u9fff]/i.test(text);
}
-export function shouldCaptureText(text, role, cfg = {}) {
+export function shouldCaptureText(text, role, cfg = {}, { filters = true } = {}) {
const maxLength = cfg.captureMaxLength || 24000;
const sanitized = sanitizeCapturedText(text);
if (!sanitized) return { shouldCapture: false, reason: "empty", text: "" };
- const capped = truncateCaptureText(sanitized, maxLength);
+ let capped = truncateCaptureText(sanitized, maxLength);
+ if (filters) {
+ const compiled = compileInputFilters(cfg?.captureFilters);
+ if (compiled.rules.length) {
+ const verdict = applyInputFilters(capped, compiled.rules, { role });
+ if (verdict.dropped) return { shouldCapture: false, reason: "filtered", text: "" };
+ capped = verdict.text;
+ if (!capped) return { shouldCapture: false, reason: "empty", text: "" };
+ }
+ }
const compact = oneLine(capped);
const isToolSummary = /^\[tool-(?:call|result)\b/i.test(compact);
diff --git a/examples/codex-memory-plugin/scripts/shared/doctor-core.mjs b/examples/codex-memory-plugin/scripts/shared/doctor-core.mjs
index 7c9650607..47887e6e8 100644
--- a/examples/codex-memory-plugin/scripts/shared/doctor-core.mjs
+++ b/examples/codex-memory-plugin/scripts/shared/doctor-core.mjs
@@ -326,6 +326,7 @@ export const KNOWN_PLUGIN_KEYS = new Set([
"recallCompressMinInputChars", "recallCompressBaseUrl", "recallCompressModel",
"recallCompressThinking", "recallCompressReasoningEffort", "recallCompressTimeoutMs",
"recallCompressDetectOnStartup", "recallCompressDetectTimeoutMs", "recallCompressDetectTtlMs",
+ "recallQueryFilters", "captureFilters",
"captureMode", "captureMaxLength", "captureTimeoutMs", "captureToolMaxChars",
"captureAssistantTurns", "captureLastAssistantOnStop", "logRankingDetails",
"commitTokenThreshold", "commitKeepRecentCount", "autoCommitOnCompact",
diff --git a/examples/codex-memory-plugin/scripts/shared/input-filters.mjs b/examples/codex-memory-plugin/scripts/shared/input-filters.mjs
new file mode 100644
index 000000000..e20bb03a7
--- /dev/null
+++ b/examples/codex-memory-plugin/scripts/shared/input-filters.mjs
@@ -0,0 +1,219 @@
+// GENERATED FROM examples/memory-plugin-shared/lib. DO NOT EDIT.
+/**
+ * Ordered, operator-configured regex filters for memory plugin input.
+ *
+ * Rules are sed-style strings, applied in order to a single piece of text:
+ * s/^\s*ultrathink\s+//i substitute (add `g` to replace every match)
+ * d|^\s*[/!]| drop the text when the pattern matches
+ * k/^\?ov\b/ keep the text only when the pattern matches
+ * user:d/^\s*\/clear\b/ apply to one role only
+ *
+ * A malformed rule becomes an entry in `errors` and is skipped rather than
+ * throwing, so a typo in a config file cannot take a hook down.
+ */
+
+const OPS = new Set(["s", "d", "k"]);
+const SCOPES = ["user", "assistant"];
+const ALLOWED_FLAGS = "imsug";
+
+export const INPUT_FILTER_KNOBS = [
+ {
+ key: "recallQueryFilters",
+ env: "OPENVIKING_RECALL_QUERY_FILTERS",
+ label: "recall query filters",
+ },
+ {
+ key: "captureFilters",
+ env: "OPENVIKING_CAPTURE_FILTERS",
+ label: "capture filters",
+ },
+];
+
+/**
+ * Read one delimited field. `\` always consumes the next character, and only
+ * `\<delim>` is un-escaped, so `\d` / `\b` / `\x2c` reach RegExp verbatim.
+ * Returns the index of the closing delimiter, or -1 when it is missing.
+ */
+function scanField(source, start, delim) {
+ let out = "";
+ let i = start;
+ while (i < source.length) {
+ const c = source[i];
+ if (c === "\\") {
+ const next = source[i + 1];
+ if (next === undefined) {
+ out += "\\";
+ i += 1;
+ continue;
+ }
+ out += next === delim ? next : "\\" + next;
+ i += 2;
+ continue;
+ }
+ if (c === delim) return { value: out, end: i };
+ out += c;
+ i += 1;
+ }
+ return { value: out, end: -1 };
+}
+
+export function parseInputFilterRule(source) {
+ if (typeof source !== "string") {
+ return { source: String(source ?? ""), error: "rule must be a string" };
+ }
+ const raw = source.trim();
+ if (!raw) return { source: raw, error: "rule is empty" };
+
+ let scope = "";
+ let rest = raw;
+ for (const candidate of SCOPES) {
+ if (rest.startsWith(`${candidate}:`)) {
+ scope = candidate;
+ rest = rest.slice(candidate.length + 1);
+ break;
+ }
+ }
+
+ const op = rest[0] || "";
+ if (!OPS.has(op)) {
+ return {
+ source: raw,
+ error: `unknown operation "${op}" — expected s (substitute), d (drop) or k (keep)`,
+ };
+ }
+ const delim = rest[1];
+ if (delim === undefined) {
+ return { source: raw, error: `missing delimiter after "${op}"` };
+ }
+ if (/[A-Za-z0-9\s\\]/.test(delim)) {
+ return {
+ source: raw,
+ error: `invalid delimiter ${JSON.stringify(delim)} — use punctuation such as / | # or :`,
+ };
+ }
+
+ const first = scanField(rest, 2, delim);
+ const pattern = first.value;
+ let replacement = null;
+ let flagsRaw = "";
+ if (op === "s") {
+ if (first.end < 0) {
+ return { source: raw, error: `missing ${JSON.stringify(delim)} after the pattern` };
+ }
+ const second = scanField(rest, first.end + 1, delim);
+ if (second.end < 0) {
+ return {
+ source: raw,
+ error: `missing ${JSON.stringify(delim)} after the replacement — write s${delim}pattern${delim}replacement${delim}`,
+ };
+ }
+ replacement = second.value;
+ flagsRaw = rest.slice(second.end + 1);
+ } else {
+ flagsRaw = first.end < 0 ? "" : rest.slice(first.end + 1);
+ }
+
+ if (!pattern) return { source: raw, error: "pattern is empty" };
+
+ const flags = new Set();
+ for (const flag of flagsRaw) {
+ if (!ALLOWED_FLAGS.includes(flag)) {
+ return { source: raw, error: `unknown flag "${flag}" — allowed flags are i, m, s, u, g` };
+ }
+ if (flags.has(flag)) return { source: raw, error: `duplicate flag "${flag}"` };
+ flags.add(flag);
+ }
+ // d/k decide with .test(), which advances lastIndex on a global regex and
+ // would then alternate between calls. `g` means nothing there, so drop it.
+ if (op !== "s") flags.delete("g");
+
+ let re;
+ try {
+ re = new RegExp(pattern, [...flags].join(""));
+ } catch (err) {
+ // V8 already says "Invalid regular expression: <pattern>: <why>"; keep that
+ // detail without stuttering the prefix the doctors key off.
+ const message = err?.message || String(err);
+ return {
+ source: raw,
+ error: /^invalid regular expression/i.test(message)
+ ? `invalid${message.slice("Invalid".length)}`
+ : `invalid regular expression: ${message}`,
+ };
+ }
+
+ return { op, scope, re, replacement, source: raw };
+}
+
+export function compileInputFilters(rules) {
+ const compiled = [];
+ const errors = [];
+ const list = Array.isArray(rules) ? rules : [];
+ list.forEach((entry, index) => {
+ if (typeof entry !== "string") {
+ errors.push({ index, source: String(entry ?? ""), message: "rule must be a string" });
+ return;
+ }
+ if (!entry.trim()) return;
+ const parsed = parseInputFilterRule(entry);
+ if (parsed.error) errors.push({ index, source: parsed.source, message: parsed.error });
+ else compiled.push({ ...parsed, index });
+ });
+ return { rules: compiled, errors };
+}
+
+/**
+ * Run compiled rules over `text`.
+ *
+ * `substituteOnly` skips the d/k operations, so a caller that already took one
+ * drop decision for a turn can rewrite its individual pieces without taking a
+ * second, possibly contradictory one.
+ */
+export function applyInputFilters(text, compiled, { role = "", substituteOnly = false } = {}) {
+ const original = typeof text === "string" ? text : String(text ?? "");
+ let current = original;
+ for (const rule of Array.isArray(compiled) ? compiled : []) {
+ if (rule.scope && rule.scope !== role) continue;
+ if (rule.op === "s") {
+ current = current.replace(rule.re, rule.replacement);
+ continue;
+ }
+ if (substituteOnly) continue;
+ const matched = rule.re.test(current);
+ if ((rule.op === "d" && matched) || (rule.op === "k" && !matched)) {
+ return { text: "", changed: false, dropped: true, ruleIndex: rule.index, op: rule.op };
+ }
+ }
+ const finalText = current.trim();
+ return {
+ text: finalText,
+ changed: finalText !== original,
+ dropped: false,
+ ruleIndex: -1,
+ op: "",
+ };
+}
+
+/** Doctor-facing summary of both filter knobs on a resolved plugin config. */
+export function describeInputFilters(cfg = {}) {
+ return INPUT_FILTER_KNOBS.map(({ key, env, label }) => {
+ const configured = Array.isArray(cfg?.[key]) ? cfg[key] : [];
+ const compiled = compileInputFilters(configured);
+ const ops = { s: 0, d: 0, k: 0 };
+ for (const rule of compiled.rules) ops[rule.op] += 1;
+ const bits = [];
+ if (ops.s) bits.push(`${ops.s} substitute`);
+ if (ops.d) bits.push(`${ops.d} drop`);
+ if (ops.k) bits.push(`${ops.k} keep-only`);
+ const active = compiled.rules.length;
+ return {
+ key,
+ env,
+ label,
+ total: configured.length,
+ active,
+ summary: `${active} rule${active === 1 ? "" : "s"}${bits.length ? ` (${bits.join(", ")})` : ""}`,
+ errors: compiled.errors,
+ };
+ });
+}
diff --git a/examples/dsh-memory-plugin/shared/capture-utils.mjs b/examples/dsh-memory-plugin/shared/capture-utils.mjs
index 38e0baecc..22bc1b63c 100644
--- a/examples/dsh-memory-plugin/shared/capture-utils.mjs
+++ b/examples/dsh-memory-plugin/shared/capture-utils.mjs
@@ -1,4 +1,6 @@
// GENERATED FROM examples/memory-plugin-shared/lib. DO NOT EDIT.
+import { applyInputFilters, compileInputFilters } from "./input-filters.mjs";
+
const TEXT_BLOCK_TYPES = new Set(["text", "input_text", "output_text"]);
const TOOL_CALL_TYPES = new Set([
"tool_call",
@@ -354,6 +356,53 @@ export function extractPartsFromPayload(payload, options = {}) {
return parts;
}
+/**
+ * Prune blank text parts and run `cfg.captureFilters` over the ones that remain.
+ *
+ * One drop verdict is taken per turn, on the aggregate of its text parts; the
+ * individual parts are then rewritten with the substitutions only, so a rule can
+ * never both keep and drop the same turn. Tool call/result parts are carried
+ * through untouched — filters match conversation text, not tool payloads — but a
+ * turn dropped on its text takes its tool parts with it.
+ */
+export function filterCaptureParts(parts, role, cfg = {}) {
+ const kept = [];
+ for (const part of parts || []) {
+ if (!part) continue;
+ if (part.type !== "text") {
+ kept.push(part);
+ continue;
+ }
+ const text = typeof part.text === "string" ? part.text.trim() : "";
+ if (!text) continue;
+ kept.push(text === part.text ? part : { ...part, text });
+ }
+
+ const blank = { parts: kept, dropped: false };
+ const compiled = compileInputFilters(cfg?.captureFilters);
+ if (!compiled.rules.length || kept.length === 0) return blank;
+
+ const textParts = kept.filter((part) => part.type === "text");
+ if (textParts.length === 0) return blank;
+
+ const verdict = applyInputFilters(textParts.map((part) => part.text).join("\n\n"), compiled.rules, {
+ role,
+ });
+ if (verdict.dropped) return { parts: [], dropped: true };
+
+ const out = [];
+ for (const part of kept) {
+ if (part.type !== "text") {
+ out.push(part);
+ continue;
+ }
+ const shaped = applyInputFilters(part.text, compiled.rules, { role, substituteOnly: true });
+ if (!shaped.text) continue;
+ out.push(shaped.text === part.text ? part : { ...part, text: shaped.text });
+ }
+ return { parts: out, dropped: out.length === 0 };
+}
+
export function extractCaptureTurns(rolloutEntries, cfg = {}) {
const toolNameById = collectToolNamesByIdFromEntries(rolloutEntries);
const turns = [];
@@ -371,10 +420,15 @@ export function extractCaptureTurns(rolloutEntries, cfg = {}) {
toolMaxChars: cfg.captureToolMaxChars,
toolNameById,
});
- const decision = shouldCaptureText(rawText, role, cfg);
- if (!decision.shouldCapture && parts.length === 0) continue;
+ const shaped = filterCaptureParts(parts, role, cfg);
+ if (shaped.dropped) continue;
+ // With parts on the wire the drop decision was already taken above, so the
+ // text path only runs the filters for the `content` fallback. That also
+ // keeps `turn.text` faithful for callers that scan it for trigger words.
+ const decision = shouldCaptureText(rawText, role, cfg, { filters: shaped.parts.length === 0 });
+ if (!decision.shouldCapture && shaped.parts.length === 0) continue;
const text = decision.shouldCapture ? decision.text : "";
- turns.push({ role, text, parts });
+ turns.push({ role, text, parts: shaped.parts });
}
return turns;
}
@@ -486,12 +540,21 @@ function isPunctuationOnly(text) {
return !/[a-z0-9\u3400-\u9fff]/i.test(text);
}
-export function shouldCaptureText(text, role, cfg = {}) {
+export function shouldCaptureText(text, role, cfg = {}, { filters = true } = {}) {
const maxLength = cfg.captureMaxLength || 24000;
const sanitized = sanitizeCapturedText(text);
if (!sanitized) return { shouldCapture: false, reason: "empty", text: "" };
- const capped = truncateCaptureText(sanitized, maxLength);
+ let capped = truncateCaptureText(sanitized, maxLength);
+ if (filters) {
+ const compiled = compileInputFilters(cfg?.captureFilters);
+ if (compiled.rules.length) {
+ const verdict = applyInputFilters(capped, compiled.rules, { role });
+ if (verdict.dropped) return { shouldCapture: false, reason: "filtered", text: "" };
+ capped = verdict.text;
+ if (!capped) return { shouldCapture: false, reason: "empty", text: "" };
+ }
+ }
const compact = oneLine(capped);
const isToolSummary = /^\[tool-(?:call|result)\b/i.test(compact);
diff --git a/examples/dsh-memory-plugin/shared/input-filters.mjs b/examples/dsh-memory-plugin/shared/input-filters.mjs
new file mode 100644
index 000000000..e20bb03a7
--- /dev/null
+++ b/examples/dsh-memory-plugin/shared/input-filters.mjs
@@ -0,0 +1,219 @@
+// GENERATED FROM examples/memory-plugin-shared/lib. DO NOT EDIT.
+/**
+ * Ordered, operator-configured regex filters for memory plugin input.
+ *
+ * Rules are sed-style strings, applied in order to a single piece of text:
+ * s/^\s*ultrathink\s+//i substitute (add `g` to replace every match)
+ * d|^\s*[/!]| drop the text when the pattern matches
+ * k/^\?ov\b/ keep the text only when the pattern matches
+ * user:d/^\s*\/clear\b/ apply to one role only
+ *
+ * A malformed rule becomes an entry in `errors` and is skipped rather than
+ * throwing, so a typo in a config file cannot take a hook down.
+ */
+
+const OPS = new Set(["s", "d", "k"]);
+const SCOPES = ["user", "assistant"];
+const ALLOWED_FLAGS = "imsug";
+
+export const INPUT_FILTER_KNOBS = [
+ {
+ key: "recallQueryFilters",
+ env: "OPENVIKING_RECALL_QUERY_FILTERS",
+ label: "recall query filters",
+ },
+ {
+ key: "captureFilters",
+ env: "OPENVIKING_CAPTURE_FILTERS",
+ label: "capture filters",
+ },
+];
+
+/**
+ * Read one delimited field. `\` always consumes the next character, and only
+ * `\<delim>` is un-escaped, so `\d` / `\b` / `\x2c` reach RegExp verbatim.
+ * Returns the index of the closing delimiter, or -1 when it is missing.
+ */
+function scanField(source, start, delim) {
+ let out = "";
+ let i = start;
+ while (i < source.length) {
+ const c = source[i];
+ if (c === "\\") {
+ const next = source[i + 1];
+ if (next === undefined) {
+ out += "\\";
+ i += 1;
+ continue;
+ }
+ out += next === delim ? next : "\\" + next;
+ i += 2;
+ continue;
+ }
+ if (c === delim) return { value: out, end: i };
+ out += c;
+ i += 1;
+ }
+ return { value: out, end: -1 };
+}
+
+export function parseInputFilterRule(source) {
+ if (typeof source !== "string") {
+ return { source: String(source ?? ""), error: "rule must be a string" };
+ }
+ const raw = source.trim();
+ if (!raw) return { source: raw, error: "rule is empty" };
+
+ let scope = "";
+ let rest = raw;
+ for (const candidate of SCOPES) {
+ if (rest.startsWith(`${candidate}:`)) {
+ scope = candidate;
+ rest = rest.slice(candidate.length + 1);
+ break;
+ }
+ }
+
+ const op = rest[0] || "";
+ if (!OPS.has(op)) {
+ return {
+ source: raw,
+ error: `unknown operation "${op}" — expected s (substitute), d (drop) or k (keep)`,
+ };
+ }
+ const delim = rest[1];
+ if (delim === undefined) {
+ return { source: raw, error: `missing delimiter after "${op}"` };
+ }
+ if (/[A-Za-z0-9\s\\]/.test(delim)) {
+ return {
+ source: raw,
+ error: `invalid delimiter ${JSON.stringify(delim)} — use punctuation such as / | # or :`,
+ };
+ }
+
+ const first = scanField(rest, 2, delim);
+ const pattern = first.value;
+ let replacement = null;
+ let flagsRaw = "";
+ if (op === "s") {
+ if (first.end < 0) {
+ return { source: raw, error: `missing ${JSON.stringify(delim)} after the pattern` };
+ }
+ const second = scanField(rest, first.end + 1, delim);
+ if (second.end < 0) {
+ return {
+ source: raw,
+ error: `missing ${JSON.stringify(delim)} after the replacement — write s${delim}pattern${delim}replacement${delim}`,
+ };
+ }
+ replacement = second.value;
+ flagsRaw = rest.slice(second.end + 1);
+ } else {
+ flagsRaw = first.end < 0 ? "" : rest.slice(first.end + 1);
+ }
+
+ if (!pattern) return { source: raw, error: "pattern is empty" };
+
+ const flags = new Set();
+ for (const flag of flagsRaw) {
+ if (!ALLOWED_FLAGS.includes(flag)) {
+ return { source: raw, error: `unknown flag "${flag}" — allowed flags are i, m, s, u, g` };
+ }
+ if (flags.has(flag)) return { source: raw, error: `duplicate flag "${flag}"` };
+ flags.add(flag);
+ }
+ // d/k decide with .test(), which advances lastIndex on a global regex and
+ // would then alternate between calls. `g` means nothing there, so drop it.
+ if (op !== "s") flags.delete("g");
+
+ let re;
+ try {
+ re = new RegExp(pattern, [...flags].join(""));
+ } catch (err) {
+ // V8 already says "Invalid regular expression: <pattern>: <why>"; keep that
+ // detail without stuttering the prefix the doctors key off.
+ const message = err?.message || String(err);
+ return {
+ source: raw,
+ error: /^invalid regular expression/i.test(message)
+ ? `invalid${message.slice("Invalid".length)}`
+ : `invalid regular expression: ${message}`,
+ };
+ }
+
+ return { op, scope, re, replacement, source: raw };
+}
+
+export function compileInputFilters(rules) {
+ const compiled = [];
+ const errors = [];
+ const list = Array.isArray(rules) ? rules : [];
+ list.forEach((entry, index) => {
+ if (typeof entry !== "string") {
+ errors.push({ index, source: String(entry ?? ""), message: "rule must be a string" });
+ return;
+ }
+ if (!entry.trim()) return;
+ const parsed = parseInputFilterRule(entry);
+ if (parsed.error) errors.push({ index, source: parsed.source, message: parsed.error });
+ else compiled.push({ ...parsed, index });
+ });
+ return { rules: compiled, errors };
+}
+
+/**
+ * Run compiled rules over `text`.
+ *
+ * `substituteOnly` skips the d/k operations, so a caller that already took one
+ * drop decision for a turn can rewrite its individual pieces without taking a
+ * second, possibly contradictory one.
+ */
+export function applyInputFilters(text, compiled, { role = "", substituteOnly = false } = {}) {
+ const original = typeof text === "string" ? text : String(text ?? "");
+ let current = original;
+ for (const rule of Array.isArray(compiled) ? compiled : []) {
+ if (rule.scope && rule.scope !== role) continue;
+ if (rule.op === "s") {
+ current = current.replace(rule.re, rule.replacement);
+ continue;
+ }
+ if (substituteOnly) continue;
+ const matched = rule.re.test(current);
+ if ((rule.op === "d" && matched) || (rule.op === "k" && !matched)) {
+ return { text: "", changed: false, dropped: true, ruleIndex: rule.index, op: rule.op };
+ }
+ }
+ const finalText = current.trim();
+ return {
+ text: finalText,
+ changed: finalText !== original,
+ dropped: false,
+ ruleIndex: -1,
+ op: "",
+ };
+}
+
+/** Doctor-facing summary of both filter knobs on a resolved plugin config. */
+export function describeInputFilters(cfg = {}) {
+ return INPUT_FILTER_KNOBS.map(({ key, env, label }) => {
+ const configured = Array.isArray(cfg?.[key]) ? cfg[key] : [];
+ const compiled = compileInputFilters(configured);
+ const ops = { s: 0, d: 0, k: 0 };
+ for (const rule of compiled.rules) ops[rule.op] += 1;
+ const bits = [];
+ if (ops.s) bits.push(`${ops.s} substitute`);
+ if (ops.d) bits.push(`${ops.d} drop`);
+ if (ops.k) bits.push(`${ops.k} keep-only`);
+ const active = compiled.rules.length;
+ return {
+ key,
+ env,
+ label,
+ total: configured.length,
+ active,
+ summary: `${active} rule${active === 1 ? "" : "s"}${bits.length ? ` (${bits.join(", ")})` : ""}`,
+ errors: compiled.errors,
+ };
+ });
+}
diff --git a/examples/memory-plugin-shared/capture-utils.test.mjs b/examples/memory-plugin-shared/capture-utils.test.mjs
index 3086abc63..d0f310396 100644
--- a/examples/memory-plugin-shared/capture-utils.test.mjs
+++ b/examples/memory-plugin-shared/capture-utils.test.mjs
@@ -1,6 +1,11 @@
import test from "node:test"
import assert from "node:assert/strict"
-import { extractPartsFromPayload } from "./lib/capture-utils.mjs"
+import {
+ extractCaptureTurns,
+ extractPartsFromPayload,
+ filterCaptureParts,
+ shouldCaptureText,
+} from "./lib/capture-utils.mjs"
function toolPart(parts) {
return parts.find((part) => part?.type === "tool")
@@ -57,3 +62,101 @@ test("toolStatus stays completed for a successful camelCase result", () => {
}, { toolNameById: { "call-3": "bash" } })
assert.equal(toolPart(parts).tool_status, "completed")
})
+
+const TOOL_PART = { type: "tool", tool_name: "bash", tool_input: "ls" }
+
+function userEntry(text) {
+ return { payload: { role: "user", content: [{ type: "text", text }] } }
+}
+
+test("filterCaptureParts prunes blank text parts even with no rules configured", () => {
+ const shaped = filterCaptureParts(
+ [{ type: "text", text: " " }, { type: "text", text: " kept " }, TOOL_PART],
+ "user",
+ {},
+ )
+ assert.equal(shaped.dropped, false)
+ assert.deepEqual(shaped.parts, [{ type: "text", text: "kept" }, TOOL_PART])
+})
+
+test("filterCaptureParts substitutes in every text part and leaves tool parts alone", () => {
+ const shaped = filterCaptureParts(
+ [{ type: "text", text: "token sk_ABCDEF here" }, TOOL_PART, { type: "text", text: "and sk_ZZZ" }],
+ "user",
+ { captureFilters: ["s/sk_[A-Za-z0-9]+/[redacted]/g"] },
+ )
+ assert.equal(shaped.dropped, false)
+ assert.deepEqual(shaped.parts, [
+ { type: "text", text: "token [redacted] here" },
+ TOOL_PART,
+ { type: "text", text: "and [redacted]" },
+ ])
+})
+
+test("filterCaptureParts takes one drop verdict on the aggregate, tool parts included", () => {
+ const cfg = { captureFilters: ["d/^internal notes/"] }
+ const shaped = filterCaptureParts(
+ [{ type: "text", text: "internal notes" }, TOOL_PART, { type: "text", text: "trailing" }],
+ "user",
+ cfg,
+ )
+ assert.equal(shaped.dropped, true)
+ assert.deepEqual(shaped.parts, [])
+
+ // The aggregate is what a keep-only rule sees, so a match anywhere keeps the turn.
+ const keep = filterCaptureParts(
+ [{ type: "text", text: "one" }, { type: "text", text: "keeper" }],
+ "user",
+ { captureFilters: ["k/keeper/"] },
+ )
+ assert.equal(keep.dropped, false)
+ assert.equal(keep.parts.length, 2)
+})
+
+test("filterCaptureParts never judges a turn that carries no text", () => {
+ const shaped = filterCaptureParts([TOOL_PART], "user", { captureFilters: ["k/never matches/"] })
+ assert.equal(shaped.dropped, false)
+ assert.deepEqual(shaped.parts, [TOOL_PART])
+ assert.equal(filterCaptureParts([], "user", { captureFilters: ["k/x/"] }).dropped, false)
+})
+
+test("shouldCaptureText reports a filtered drop and can be asked to skip filters", () => {
+ const cfg = { captureFilters: ["user:d/^\\/compact\\b/", "s/^ultrathink\\s+//"] }
+ assert.deepEqual(shouldCaptureText("/compact the thread", "user", cfg), {
+ shouldCapture: false,
+ reason: "filtered",
+ text: "",
+ })
+ assert.equal(shouldCaptureText("/compact the thread", "assistant", cfg).shouldCapture, true)
+ assert.equal(shouldCaptureText("ultrathink about the retry policy", "user", cfg).text,
+ "about the retry policy")
+ assert.equal(
+ shouldCaptureText("ultrathink about the retry policy", "user", cfg, { filters: false }).text,
+ "ultrathink about the retry policy",
+ )
+})
+
+test("extractCaptureTurns drops filtered turns and keeps turn.text unfiltered", () => {
+ const entries = [userEntry("please remember the retry policy"), userEntry("scratch: throwaway note")]
+ const cfg = { captureFilters: ["d/^scratch:/", "s/^please\\s+//"] }
+ const turns = extractCaptureTurns(entries, cfg)
+ assert.equal(turns.length, 1)
+ assert.deepEqual(turns[0].parts, [{ type: "text", text: "remember the retry policy" }])
+ // parts are on the wire, so the text path stayed faithful for keyword scanning
+ assert.equal(turns[0].text, "please remember the retry policy")
+})
+
+test("extractCaptureTurns falls back to the text path when a turn has no parts", () => {
+ const entries = [{ payload: { role: "user", content: [{ type: "unknown", value: 1 }], text: "" } }]
+ assert.deepEqual(extractCaptureTurns(entries, { captureFilters: ["d/x/"] }), [])
+})
+
+test("extractCaptureTurns honours the role scope of a capture rule", () => {
+ const cfg = { captureAssistantTurns: true, captureFilters: ["assistant:d/^ok so/"] }
+ const entries = [
+ userEntry("ok so what did we decide"),
+ { payload: { role: "assistant", content: [{ type: "text", text: "ok so here is the plan" }] } },
+ ]
+ const turns = extractCaptureTurns(entries, cfg)
+ assert.deepEqual(turns.map((turn) => turn.role), ["user"])
+})
diff --git a/examples/memory-plugin-shared/input-filters.test.mjs b/examples/memory-plugin-shared/input-filters.test.mjs
new file mode 100644
index 000000000..7dab25e1d
--- /dev/null
+++ b/examples/memory-plugin-shared/input-filters.test.mjs
@@ -0,0 +1,207 @@
+import test from "node:test"
+import assert from "node:assert/strict"
+import {
+ applyInputFilters,
+ compileInputFilters,
+ describeInputFilters,
+ parseInputFilterRule,
+} from "./lib/input-filters.mjs"
+
+function compile(rules) {
+ const compiled = compileInputFilters(rules)
+ assert.deepEqual(compiled.errors, [], `unexpected errors: ${JSON.stringify(compiled.errors)}`)
+ return compiled.rules
+}
+
+function apply(rules, text, options) {
+ return applyInputFilters(text, compile(rules), options)
+}
+
+test("parses a substitute rule with flags", () => {
+ const rule = parseInputFilterRule(String.raw`s/^\s*ultrathink\s+//i`)
+ assert.equal(rule.error, undefined)
+ assert.equal(rule.op, "s")
+ assert.equal(rule.scope, "")
+ assert.equal(rule.replacement, "")
+ assert.equal(rule.re.source, String.raw`^\s*ultrathink\s+`)
+ assert.equal(rule.re.flags, "i")
+})
+
+test("accepts any punctuation delimiter, including a colon", () => {
+ for (const rule of ["s|a|b|", "s#a#b#", "s:a:b:", "s,a,b,", "s!a!b!"]) {
+ const parsed = parseInputFilterRule(rule)
+ assert.equal(parsed.error, undefined, `${rule} should parse`)
+ assert.equal(parsed.re.source, "a")
+ assert.equal(parsed.replacement, "b")
+ }
+})
+
+test("rejects alphanumeric, whitespace and backslash delimiters", () => {
+ for (const rule of ["sxaxbx", "s a b ", "s\\a\\b\\"]) {
+ assert.match(parseInputFilterRule(rule).error, /invalid delimiter|missing/)
+ }
+})
+
+test("only the delimiter is un-escaped; every other escape reaches RegExp", () => {
+ const parsed = parseInputFilterRule(String.raw`s/a\/b\d\\c/x/`)
+ assert.equal(parsed.re.test("a/b7\\c"), true)
+ assert.equal(parsed.re.test(String.raw`a\/b7\\c`), false)
+ assert.equal(apply([String.raw`s/a\/b\d\\c/x/`], "a/b7\\c!").text, "x!")
+})
+
+test("\\x2c is an env-safe literal comma, but not a quantifier bound", () => {
+ assert.equal(apply([String.raw`s/hey\x2c\s*//i`], "Hey, what changed").text, "what changed")
+ // `{2\x2c}` is not quantifier syntax: RegExp falls back to matching it
+ // literally. Bounded quantifiers have to come from the ovcli.conf array.
+ assert.equal(apply([String.raw`s/a{2\x2c}/X/`], "aaa").text, "aaa")
+ assert.equal(apply([String.raw`s/a{2\x2c}/X/`], "a{2,}").text, "X")
+})
+
+test("drop and keep rules take optional trailing delimiter and flags", () => {
+ for (const rule of ["d/foo", "d/foo/", "d/foo/i"]) {
+ assert.equal(parseInputFilterRule(rule).error, undefined, `${rule} should parse`)
+ }
+ assert.equal(parseInputFilterRule("d/foo/i").re.flags, "i")
+})
+
+test("g is stripped from d/k rules so .test() stays stateless", () => {
+ const rules = compile(["d/foo/g"])
+ assert.equal(rules[0].re.flags, "")
+ // With `g` kept, .test() would advance lastIndex and alternate between calls.
+ assert.equal(applyInputFilters("foo", rules).dropped, true)
+ assert.equal(applyInputFilters("foo", rules).dropped, true)
+})
+
+test("recognises the role scope prefixes", () => {
+ assert.equal(parseInputFilterRule("user:d/foo/").scope, "user")
+ assert.equal(parseInputFilterRule("assistant:s/foo/bar/").scope, "assistant")
+ assert.equal(parseInputFilterRule("s/user:/x/").scope, "")
+})
+
+test("reports every malformed rule instead of throwing", () => {
+ const cases = [
+ ["", /rule is empty/],
+ [" ", /rule is empty/],
+ ["strip the prefix", /invalid delimiter "t"/],
+ ["x/a/b/", /unknown operation "x"/],
+ ["s", /missing delimiter/],
+ ["s/abc", /missing "\/" after the pattern/],
+ ["s/secret/", /missing "\/" after the replacement/],
+ ["s///", /pattern is empty/],
+ ["d//", /pattern is empty/],
+ ["d/foo/x", /unknown flag "x"/],
+ ["d/foo/ii", /duplicate flag "i"/],
+ ["s/(/x/", /invalid regular expression/],
+ [String.raw`d/[\w-.]/u`, /invalid regular expression/],
+ ]
+ for (const [rule, expected] of cases) {
+ assert.match(parseInputFilterRule(rule).error || "", expected, `for rule ${JSON.stringify(rule)}`)
+ }
+ assert.match(parseInputFilterRule(42).error, /rule must be a string/)
+})
+
+test("non-array and non-string entries never throw", () => {
+ assert.deepEqual(compileInputFilters(undefined).rules, [])
+ assert.deepEqual(compileInputFilters("s/a/b/").rules, [])
+ const compiled = compileInputFilters([null, "", " ", "s/a/b/"])
+ assert.equal(compiled.rules.length, 1)
+ assert.equal(compiled.errors.length, 1)
+ assert.match(compiled.errors[0].message, /rule must be a string/)
+})
+
+test("substitutions run in order, once unless g is set", () => {
+ assert.equal(apply(["s/a/b/"], "aaa").text, "baa")
+ assert.equal(apply(["s/a/b/g"], "aaa").text, "bbb")
+ assert.equal(apply(["s/a/b/g", "s/b/c/"], "aa").text, "cb")
+})
+
+test("replacement honours $1, $& and $$", () => {
+ assert.equal(apply(["s/(\\w+) (\\w+)/$2 $1/"], "hello world").text, "world hello")
+ assert.equal(apply(["s/world/[$&]/"], "hello world").text, "hello [world]")
+ assert.equal(apply(["s/world/$$/"], "hello world").text, "hello $")
+})
+
+test("d drops on a match and k drops when there is no match", () => {
+ const dropped = apply(["d/^\\//"], "/help")
+ assert.equal(dropped.dropped, true)
+ assert.equal(dropped.op, "d")
+ assert.equal(dropped.ruleIndex, 0)
+ assert.equal(dropped.text, "")
+
+ assert.equal(apply(["d/^\\//"], "how do I").dropped, false)
+ assert.equal(apply(["k/^\\?ov\\b/"], "?ov what did we decide").dropped, false)
+ assert.equal(apply(["k/^\\?ov\\b/"], "what did we decide").dropped, true)
+})
+
+test("chained keep rules are an AND, and the reported index is the config index", () => {
+ const rules = compile(["s/^x //", "k/alpha/", "k/beta/"])
+ assert.equal(applyInputFilters("x alpha beta", rules).dropped, false)
+ const verdict = applyInputFilters("x alpha only", rules)
+ assert.equal(verdict.dropped, true)
+ assert.equal(verdict.ruleIndex, 2)
+ assert.equal(verdict.op, "k")
+})
+
+test("scoped rules only fire for their role", () => {
+ const rules = compile(["user:d/secret/"])
+ assert.equal(applyInputFilters("secret", rules, { role: "user" }).dropped, true)
+ assert.equal(applyInputFilters("secret", rules, { role: "assistant" }).dropped, false)
+ assert.equal(applyInputFilters("secret", rules, { role: "" }).dropped, false)
+})
+
+test("text emptied by a substitution is not a drop", () => {
+ const verdict = apply(["s/^ultrathink$//"], "ultrathink")
+ assert.equal(verdict.dropped, false)
+ assert.equal(verdict.text, "")
+ assert.equal(verdict.changed, true)
+})
+
+test("substituteOnly rewrites but never drops", () => {
+ const rules = compile(["s/secret/[redacted]/g", "d/^drop me/"])
+ const verdict = applyInputFilters("drop me: secret secret", rules, { substituteOnly: true })
+ assert.equal(verdict.dropped, false)
+ assert.equal(verdict.text, "drop me: [redacted] [redacted]")
+})
+
+test("describeInputFilters summarises both knobs for the doctors", () => {
+ const described = describeInputFilters({
+ recallQueryFilters: ["s/^hi //", "d/^\\//", "k/x/"],
+ captureFilters: ["s/(/x/"],
+ })
+ assert.deepEqual(described.map((entry) => entry.key), ["recallQueryFilters", "captureFilters"])
+ assert.equal(described[0].env, "OPENVIKING_RECALL_QUERY_FILTERS")
+ assert.equal(described[0].summary, "3 rules (1 substitute, 1 drop, 1 keep-only)")
+ assert.equal(described[0].errors.length, 0)
+ assert.equal(described[1].total, 1)
+ assert.equal(described[1].active, 0)
+ assert.equal(described[1].errors.length, 1)
+ assert.deepEqual(describeInputFilters({}).map((entry) => entry.summary), ["0 rules", "0 rules"])
+})
+
+// Every rule printed in the plugin READMEs, so a broken example cannot ship.
+test("the documented examples parse and behave as documented", () => {
+ const strip = String.raw`s/^\s*(ultrathink|think harder?)\s+//i`
+ assert.equal(apply([strip], "Ultrathink what did we decide").text, "what did we decide")
+
+ const skipCommands = String.raw`d|^\s*[/!]|`
+ assert.equal(apply([skipCommands], "/help").dropped, true)
+ assert.equal(apply([skipCommands], "!ls -la").dropped, true)
+ assert.equal(apply([skipCommands], "how do I run it").dropped, false)
+
+ const optIn = [String.raw`k/^\?ov\b/`, String.raw`s/^\?ov\s*//`]
+ assert.equal(apply(optIn, "?ov the retry policy").text, "the retry policy")
+ assert.equal(apply(optIn, "the retry policy").dropped, true)
+
+ const redact = String.raw`s/\b(sk|ghp|xoxb)_[A-Za-z0-9_-]+/[redacted]/g`
+ assert.equal(
+ apply([redact], "use sk_live_ABCDEFGHIJ and ghp_0123456789abcdef now").text,
+ "use [redacted] and [redacted] now",
+ )
+
+ const skipCommandTurns = String.raw`user:d/^\s*\/(clear|compact)\b/`
+ assert.equal(apply([skipCommandTurns], "/compact please", { role: "user" }).dropped, true)
+ assert.equal(apply([skipCommandTurns], "/compact please", { role: "assistant" }).dropped, false)
+
+ const politeness = String.raw`s/^(请|麻烦)(你|帮我)?//`
+ assert.equal(apply([politeness], "请帮我看看这个报错").text, "看看这个报错")
+})
diff --git a/examples/memory-plugin-shared/lib/capture-utils.mjs b/examples/memory-plugin-shared/lib/capture-utils.mjs
index 75e76bbad..74e886e46 100644
--- a/examples/memory-plugin-shared/lib/capture-utils.mjs
+++ b/examples/memory-plugin-shared/lib/capture-utils.mjs
@@ -1,3 +1,5 @@
+import { applyInputFilters, compileInputFilters } from "./input-filters.mjs";
+
const TEXT_BLOCK_TYPES = new Set(["text", "input_text", "output_text"]);
const TOOL_CALL_TYPES = new Set([
"tool_call",
@@ -353,6 +355,53 @@ export function extractPartsFromPayload(payload, options = {}) {
return parts;
}
+/**
+ * Prune blank text parts and run `cfg.captureFilters` over the ones that remain.
+ *
+ * One drop verdict is taken per turn, on the aggregate of its text parts; the
+ * individual parts are then rewritten with the substitutions only, so a rule can
+ * never both keep and drop the same turn. Tool call/result parts are carried
+ * through untouched — filters match conversation text, not tool payloads — but a
+ * turn dropped on its text takes its tool parts with it.
+ */
+export function filterCaptureParts(parts, role, cfg = {}) {
+ const kept = [];
+ for (const part of parts || []) {
+ if (!part) continue;
+ if (part.type !== "text") {
+ kept.push(part);
+ continue;
+ }
+ const text = typeof part.text === "string" ? part.text.trim() : "";
+ if (!text) continue;
+ kept.push(text === part.text ? part : { ...part, text });
+ }
+
+ const blank = { parts: kept, dropped: false };
+ const compiled = compileInputFilters(cfg?.captureFilters);
+ if (!compiled.rules.length || kept.length === 0) return blank;
+
+ const textParts = kept.filter((part) => part.type === "text");
+ if (textParts.length === 0) return blank;
+
+ const verdict = applyInputFilters(textParts.map((part) => part.text).join("\n\n"), compiled.rules, {
+ role,
+ });
+ if (verdict.dropped) return { parts: [], dropped: true };
+
+ const out = [];
+ for (const part of kept) {
+ if (part.type !== "text") {
+ out.push(part);
+ continue;
+ }
+ const shaped = applyInputFilters(part.text, compiled.rules, { role, substituteOnly: true });
+ if (!shaped.text) continue;
+ out.push(shaped.text === part.text ? part : { ...part, text: shaped.text });
+ }
+ return { parts: out, dropped: out.length === 0 };
+}
+
export function extractCaptureTurns(rolloutEntries, cfg = {}) {
const toolNameById = collectToolNamesByIdFromEntries(rolloutEntries);
const turns = [];
@@ -370,10 +419,15 @@ export function extractCaptureTurns(rolloutEntries, cfg = {}) {
toolMaxChars: cfg.captureToolMaxChars,
toolNameById,
});
- const decision = shouldCaptureText(rawText, role, cfg);
- if (!decision.shouldCapture && parts.length === 0) continue;
+ const shaped = filterCaptureParts(parts, role, cfg);
+ if (shaped.dropped) continue;
+ // With parts on the wire the drop decision was already taken above, so the
+ // text path only runs the filters for the `content` fallback. That also
+ // keeps `turn.text` faithful for callers that scan it for trigger words.
+ const decision = shouldCaptureText(rawText, role, cfg, { filters: shaped.parts.length === 0 });
+ if (!decision.shouldCapture && shaped.parts.length === 0) continue;
const text = decision.shouldCapture ? decision.text : "";
- turns.push({ role, text, parts });
+ turns.push({ role, text, parts: shaped.parts });
}
return turns;
}
@@ -485,12 +539,21 @@ function isPunctuationOnly(text) {
return !/[a-z0-9\u3400-\u9fff]/i.test(text);
}
-export function shouldCaptureText(text, role, cfg = {}) {
+export function shouldCaptureText(text, role, cfg = {}, { filters = true } = {}) {
const maxLength = cfg.captureMaxLength || 24000;
const sanitized = sanitizeCapturedText(text);
if (!sanitized) return { shouldCapture: false, reason: "empty", text: "" };
- const capped = truncateCaptureText(sanitized, maxLength);
+ let capped = truncateCaptureText(sanitized, maxLength);
+ if (filters) {
+ const compiled = compileInputFilters(cfg?.captureFilters);
+ if (compiled.rules.length) {
+ const verdict = applyInputFilters(capped, compiled.rules, { role });
+ if (verdict.dropped) return { shouldCapture: false, reason: "filtered", text: "" };
+ capped = verdict.text;
+ if (!capped) return { shouldCapture: false, reason: "empty", text: "" };
+ }
+ }
const compact = oneLine(capped);
const isToolSummary = /^\[tool-(?:call|result)\b/i.test(compact);
diff --git a/examples/memory-plugin-shared/lib/doctor-core.mjs b/examples/memory-plugin-shared/lib/doctor-core.mjs
index 71079de18..8045a47f0 100644
--- a/examples/memory-plugin-shared/lib/doctor-core.mjs
+++ b/examples/memory-plugin-shared/lib/doctor-core.mjs
@@ -325,6 +325,7 @@ export const KNOWN_PLUGIN_KEYS = new Set([
"recallCompressMinInputChars", "recallCompressBaseUrl", "recallCompressModel",
"recallCompressThinking", "recallCompressReasoningEffort", "recallCompressTimeoutMs",
"recallCompressDetectOnStartup", "recallCompressDetectTimeoutMs", "recallCompressDetectTtlMs",
+ "recallQueryFilters", "captureFilters",
"captureMode", "captureMaxLength", "captureTimeoutMs", "captureToolMaxChars",
"captureAssistantTurns", "captureLastAssistantOnStop", "logRankingDetails",
"commitTokenThreshold", "commitKeepRecentCount", "autoCommitOnCompact",
diff --git a/examples/memory-plugin-shared/lib/input-filters.mjs b/examples/memory-plugin-shared/lib/input-filters.mjs
new file mode 100644
index 000000000..d50ce78c4
--- /dev/null
+++ b/examples/memory-plugin-shared/lib/input-filters.mjs
@@ -0,0 +1,218 @@
+/**
+ * Ordered, operator-configured regex filters for memory plugin input.
+ *
+ * Rules are sed-style strings, applied in order to a single piece of text:
+ * s/^\s*ultrathink\s+//i substitute (add `g` to replace every match)
+ * d|^\s*[/!]| drop the text when the pattern matches
+ * k/^\?ov\b/ keep the text only when the pattern matches
+ * user:d/^\s*\/clear\b/ apply to one role only
+ *
+ * A malformed rule becomes an entry in `errors` and is skipped rather than
+ * throwing, so a typo in a config file cannot take a hook down.
+ */
+
+const OPS = new Set(["s", "d", "k"]);
+const SCOPES = ["user", "assistant"];
+const ALLOWED_FLAGS = "imsug";
+
+export const INPUT_FILTER_KNOBS = [
+ {
+ key: "recallQueryFilters",
+ env: "OPENVIKING_RECALL_QUERY_FILTERS",
+ label: "recall query filters",
+ },
+ {
+ key: "captureFilters",
+ env: "OPENVIKING_CAPTURE_FILTERS",
+ label: "capture filters",
+ },
+];
+
+/**
+ * Read one delimited field. `\` always consumes the next character, and only
+ * `\<delim>` is un-escaped, so `\d` / `\b` / `\x2c` reach RegExp verbatim.
+ * Returns the index of the closing delimiter, or -1 when it is missing.
+ */
+function scanField(source, start, delim) {
+ let out = "";
+ let i = start;
+ while (i < source.length) {
+ const c = source[i];
+ if (c === "\\") {
+ const next = source[i + 1];
+ if (next === undefined) {
+ out += "\\";
+ i += 1;
+ continue;
+ }
+ out += next === delim ? next : "\\" + next;
+ i += 2;
+ continue;
+ }
+ if (c === delim) return { value: out, end: i };
+ out += c;
+ i += 1;
+ }
+ return { value: out, end: -1 };
+}
+
+export function parseInputFilterRule(source) {
+ if (typeof source !== "string") {
+ return { source: String(source ?? ""), error: "rule must be a string" };
+ }
+ const raw = source.trim();
+ if (!raw) return { source: raw, error: "rule is empty" };
+
+ let scope = "";
+ let rest = raw;
+ for (const candidate of SCOPES) {
+ if (rest.startsWith(`${candidate}:`)) {
+ scope = candidate;
+ rest = rest.slice(candidate.length + 1);
+ break;
+ }
+ }
+
+ const op = rest[0] || "";
+ if (!OPS.has(op)) {
+ return {
+ source: raw,
+ error: `unknown operation "${op}" — expected s (substitute), d (drop) or k (keep)`,
+ };
+ }
+ const delim = rest[1];
+ if (delim === undefined) {
+ return { source: raw, error: `missing delimiter after "${op}"` };
+ }
+ if (/[A-Za-z0-9\s\\]/.test(delim)) {
+ return {
+ source: raw,
+ error: `invalid delimiter ${JSON.stringify(delim)} — use punctuation such as / | # or :`,
+ };
+ }
+
+ const first = scanField(rest, 2, delim);
+ const pattern = first.value;
+ let replacement = null;
+ let flagsRaw = "";
+ if (op === "s") {
+ if (first.end < 0) {
+ return { source: raw, error: `missing ${JSON.stringify(delim)} after the pattern` };
+ }
+ const second = scanField(rest, first.end + 1, delim);
+ if (second.end < 0) {
+ return {
+ source: raw,
+ error: `missing ${JSON.stringify(delim)} after the replacement — write s${delim}pattern${delim}replacement${delim}`,
+ };
+ }
+ replacement = second.value;
+ flagsRaw = rest.slice(second.end + 1);
+ } else {
+ flagsRaw = first.end < 0 ? "" : rest.slice(first.end + 1);
+ }
+
+ if (!pattern) return { source: raw, error: "pattern is empty" };
+
+ const flags = new Set();
+ for (const flag of flagsRaw) {
+ if (!ALLOWED_FLAGS.includes(flag)) {
+ return { source: raw, error: `unknown flag "${flag}" — allowed flags are i, m, s, u, g` };
+ }
+ if (flags.has(flag)) return { source: raw, error: `duplicate flag "${flag}"` };
+ flags.add(flag);
+ }
+ // d/k decide with .test(), which advances lastIndex on a global regex and
+ // would then alternate between calls. `g` means nothing there, so drop it.
+ if (op !== "s") flags.delete("g");
+
+ let re;
+ try {
+ re = new RegExp(pattern, [...flags].join(""));
+ } catch (err) {
+ // V8 already says "Invalid regular expression: <pattern>: <why>"; keep that
+ // detail without stuttering the prefix the doctors key off.
+ const message = err?.message || String(err);
+ return {
+ source: raw,
+ error: /^invalid regular expression/i.test(message)
+ ? `invalid${message.slice("Invalid".length)}`
+ : `invalid regular expression: ${message}`,
+ };
+ }
+
+ return { op, scope, re, replacement, source: raw };
+}
+
+export function compileInputFilters(rules) {
+ const compiled = [];
+ const errors = [];
+ const list = Array.isArray(rules) ? rules : [];
+ list.forEach((entry, index) => {
+ if (typeof entry !== "string") {
+ errors.push({ index, source: String(entry ?? ""), message: "rule must be a string" });
+ return;
+ }
+ if (!entry.trim()) return;
+ const parsed = parseInputFilterRule(entry);
+ if (parsed.error) errors.push({ index, source: parsed.source, message: parsed.error });
+ else compiled.push({ ...parsed, index });
+ });
+ return { rules: compiled, errors };
+}
+
+/**
+ * Run compiled rules over `text`.
+ *
+ * `substituteOnly` skips the d/k operations, so a caller that already took one
+ * drop decision for a turn can rewrite its individual pieces without taking a
+ * second, possibly contradictory one.
+ */
+export function applyInputFilters(text, compiled, { role = "", substituteOnly = false } = {}) {
+ const original = typeof text === "string" ? text : String(text ?? "");
+ let current = original;
+ for (const rule of Array.isArray(compiled) ? compiled : []) {
+ if (rule.scope && rule.scope !== role) continue;
+ if (rule.op === "s") {
+ current = current.replace(rule.re, rule.replacement);
+ continue;
+ }
+ if (substituteOnly) continue;
+ const matched = rule.re.test(current);
+ if ((rule.op === "d" && matched) || (rule.op === "k" && !matched)) {
+ return { text: "", changed: false, dropped: true, ruleIndex: rule.index, op: rule.op };
+ }
+ }
+ const finalText = current.trim();
+ return {
+ text: finalText,
+ changed: finalText !== original,
+ dropped: false,
+ ruleIndex: -1,
+ op: "",
+ };
+}
+
+/** Doctor-facing summary of both filter knobs on a resolved plugin config. */
+export function describeInputFilters(cfg = {}) {
+ return INPUT_FILTER_KNOBS.map(({ key, env, label }) => {
+ const configured = Array.isArray(cfg?.[key]) ? cfg[key] : [];
+ const compiled = compileInputFilters(configured);
+ const ops = { s: 0, d: 0, k: 0 };
+ for (const rule of compiled.rules) ops[rule.op] += 1;
+ const bits = [];
+ if (ops.s) bits.push(`${ops.s} substitute`);
+ if (ops.d) bits.push(`${ops.d} drop`);
+ if (ops.k) bits.push(`${ops.k} keep-only`);
+ const active = compiled.rules.length;
+ return {
+ key,
+ env,
+ label,
+ total: configured.length,
+ active,
+ summary: `${active} rule${active === 1 ? "" : "s"}${bits.length ? ` (${bits.join(", ")})` : ""}`,
+ errors: compiled.errors,
+ };
+ });
+}
diff --git a/examples/memory-plugin-shared/sync.mjs b/examples/memory-plugin-shared/sync.mjs
index 3d2c58534..98e236c27 100644
--- a/examples/memory-plugin-shared/sync.mjs
+++ b/examples/memory-plugin-shared/sync.mjs
@@ -15,6 +15,7 @@ export const SHARED_DIR = join(ROOT, "examples", "memory-plugin-shared", "lib");
const HOOK_SHARED_FILES = [
"credentials.mjs",
"capture-utils.mjs",
+ "input-filters.mjs",
"session-model.mjs",
"pending-queue.mjs",
"debug-log.mjs",
diff --git a/examples/opencode-plugin/lib/shared/capture-utils.mjs b/examples/opencode-plugin/lib/shared/capture-utils.mjs
index 38e0baecc..22bc1b63c 100644
--- a/examples/opencode-plugin/lib/shared/capture-utils.mjs
+++ b/examples/opencode-plugin/lib/shared/capture-utils.mjs
@@ -1,4 +1,6 @@
// GENERATED FROM examples/memory-plugin-shared/lib. DO NOT EDIT.
+import { applyInputFilters, compileInputFilters } from "./input-filters.mjs";
+
const TEXT_BLOCK_TYPES = new Set(["text", "input_text", "output_text"]);
const TOOL_CALL_TYPES = new Set([
"tool_call",
@@ -354,6 +356,53 @@ export function extractPartsFromPayload(payload, options = {}) {
return parts;
}
+/**
+ * Prune blank text parts and run `cfg.captureFilters` over the ones that remain.
+ *
+ * One drop verdict is taken per turn, on the aggregate of its text parts; the
+ * individual parts are then rewritten with the substitutions only, so a rule can
+ * never both keep and drop the same turn. Tool call/result parts are carried
+ * through untouched — filters match conversation text, not tool payloads — but a
+ * turn dropped on its text takes its tool parts with it.
+ */
+export function filterCaptureParts(parts, role, cfg = {}) {
+ const kept = [];
+ for (const part of parts || []) {
+ if (!part) continue;
+ if (part.type !== "text") {
+ kept.push(part);
+ continue;
+ }
+ const text = typeof part.text === "string" ? part.text.trim() : "";
+ if (!text) continue;
+ kept.push(text === part.text ? part : { ...part, text });
+ }
+
+ const blank = { parts: kept, dropped: false };
+ const compiled = compileInputFilters(cfg?.captureFilters);
+ if (!compiled.rules.length || kept.length === 0) return blank;
+
+ const textParts = kept.filter((part) => part.type === "text");
+ if (textParts.length === 0) return blank;
+
+ const verdict = applyInputFilters(textParts.map((part) => part.text).join("\n\n"), compiled.rules, {
+ role,
+ });
+ if (verdict.dropped) return { parts: [], dropped: true };
+
+ const out = [];
+ for (const part of kept) {
+ if (part.type !== "text") {
+ out.push(part);
+ continue;
+ }
+ const shaped = applyInputFilters(part.text, compiled.rules, { role, substituteOnly: true });
+ if (!shaped.text) continue;
+ out.push(shaped.text === part.text ? part : { ...part, text: shaped.text });
+ }
+ return { parts: out, dropped: out.length === 0 };
+}
+
export function extractCaptureTurns(rolloutEntries, cfg = {}) {
const toolNameById = collectToolNamesByIdFromEntries(rolloutEntries);
const turns = [];
@@ -371,10 +420,15 @@ export function extractCaptureTurns(rolloutEntries, cfg = {}) {
toolMaxChars: cfg.captureToolMaxChars,
toolNameById,
});
- const decision = shouldCaptureText(rawText, role, cfg);
- if (!decision.shouldCapture && parts.length === 0) continue;
+ const shaped = filterCaptureParts(parts, role, cfg);
+ if (shaped.dropped) continue;
+ // With parts on the wire the drop decision was already taken above, so the
+ // text path only runs the filters for the `content` fallback. That also
+ // keeps `turn.text` faithful for callers that scan it for trigger words.
+ const decision = shouldCaptureText(rawText, role, cfg, { filters: shaped.parts.length === 0 });
+ if (!decision.shouldCapture && shaped.parts.length === 0) continue;
const text = decision.shouldCapture ? decision.text : "";
- turns.push({ role, text, parts });
+ turns.push({ role, text, parts: shaped.parts });
}
return turns;
}
@@ -486,12 +540,21 @@ function isPunctuationOnly(text) {
return !/[a-z0-9\u3400-\u9fff]/i.test(text);
}
-export function shouldCaptureText(text, role, cfg = {}) {
+export function shouldCaptureText(text, role, cfg = {}, { filters = true } = {}) {
const maxLength = cfg.captureMaxLength || 24000;
const sanitized = sanitizeCapturedText(text);
if (!sanitized) return { shouldCapture: false, reason: "empty", text: "" };
- const capped = truncateCaptureText(sanitized, maxLength);
+ let capped = truncateCaptureText(sanitized, maxLength);
+ if (filters) {
+ const compiled = compileInputFilters(cfg?.captureFilters);
+ if (compiled.rules.length) {
+ const verdict = applyInputFilters(capped, compiled.rules, { role });
+ if (verdict.dropped) return { shouldCapture: false, reason: "filtered", text: "" };
+ capped = verdict.text;
+ if (!capped) return { shouldCapture: false, reason: "empty", text: "" };
+ }
+ }
const compact = oneLine(capped);
const isToolSummary = /^\[tool-(?:call|result)\b/i.test(compact);
diff --git a/examples/opencode-plugin/lib/shared/input-filters.mjs b/examples/opencode-plugin/lib/shared/input-filters.mjs
new file mode 100644
index 000000000..e20bb03a7
--- /dev/null
+++ b/examples/opencode-plugin/lib/shared/input-filters.mjs
@@ -0,0 +1,219 @@
+// GENERATED FROM examples/memory-plugin-shared/lib. DO NOT EDIT.
+/**
+ * Ordered, operator-configured regex filters for memory plugin input.
+ *
+ * Rules are sed-style strings, applied in order to a single piece of text:
+ * s/^\s*ultrathink\s+//i substitute (add `g` to replace every match)
+ * d|^\s*[/!]| drop the text when the pattern matches
+ * k/^\?ov\b/ keep the text only when the pattern matches
+ * user:d/^\s*\/clear\b/ apply to one role only
+ *
+ * A malformed rule becomes an entry in `errors` and is skipped rather than
+ * throwing, so a typo in a config file cannot take a hook down.
+ */
+
+const OPS = new Set(["s", "d", "k"]);
+const SCOPES = ["user", "assistant"];
+const ALLOWED_FLAGS = "imsug";
+
+export const INPUT_FILTER_KNOBS = [
+ {
+ key: "recallQueryFilters",
+ env: "OPENVIKING_RECALL_QUERY_FILTERS",
+ label: "recall query filters",
+ },
+ {
+ key: "captureFilters",
+ env: "OPENVIKING_CAPTURE_FILTERS",
+ label: "capture filters",
+ },
+];
+
+/**
+ * Read one delimited field. `\` always consumes the next character, and only
+ * `\<delim>` is un-escaped, so `\d` / `\b` / `\x2c` reach RegExp verbatim.
+ * Returns the index of the closing delimiter, or -1 when it is missing.
+ */
+function scanField(source, start, delim) {
+ let out = "";
+ let i = start;
+ while (i < source.length) {
+ const c = source[i];
+ if (c === "\\") {
+ const next = source[i + 1];
+ if (next === undefined) {
+ out += "\\";
+ i += 1;
+ continue;
+ }
+ out += next === delim ? next : "\\" + next;
+ i += 2;
+ continue;
+ }
+ if (c === delim) return { value: out, end: i };
+ out += c;
+ i += 1;
+ }
+ return { value: out, end: -1 };
+}
+
+export function parseInputFilterRule(source) {
+ if (typeof source !== "string") {
+ return { source: String(source ?? ""), error: "rule must be a string" };
+ }
+ const raw = source.trim();
+ if (!raw) return { source: raw, error: "rule is empty" };
+
+ let scope = "";
+ let rest = raw;
+ for (const candidate of SCOPES) {
+ if (rest.startsWith(`${candidate}:`)) {
+ scope = candidate;
+ rest = rest.slice(candidate.length + 1);
+ break;
+ }
+ }
+
+ const op = rest[0] || "";
+ if (!OPS.has(op)) {
+ return {
+ source: raw,
+ error: `unknown operation "${op}" — expected s (substitute), d (drop) or k (keep)`,
+ };
+ }
+ const delim = rest[1];
+ if (delim === undefined) {
+ return { source: raw, error: `missing delimiter after "${op}"` };
+ }
+ if (/[A-Za-z0-9\s\\]/.test(delim)) {
+ return {
+ source: raw,
+ error: `invalid delimiter ${JSON.stringify(delim)} — use punctuation such as / | # or :`,
+ };
+ }
+
+ const first = scanField(rest, 2, delim);
+ const pattern = first.value;
+ let replacement = null;
+ let flagsRaw = "";
+ if (op === "s") {
+ if (first.end < 0) {
+ return { source: raw, error: `missing ${JSON.stringify(delim)} after the pattern` };
+ }
+ const second = scanField(rest, first.end + 1, delim);
+ if (second.end < 0) {
+ return {
+ source: raw,
+ error: `missing ${JSON.stringify(delim)} after the replacement — write s${delim}pattern${delim}replacement${delim}`,
+ };
+ }
+ replacement = second.value;
+ flagsRaw = rest.slice(second.end + 1);
+ } else {
+ flagsRaw = first.end < 0 ? "" : rest.slice(first.end + 1);
+ }
+
+ if (!pattern) return { source: raw, error: "pattern is empty" };
+
+ const flags = new Set();
+ for (const flag of flagsRaw) {
+ if (!ALLOWED_FLAGS.includes(flag)) {
+ return { source: raw, error: `unknown flag "${flag}" — allowed flags are i, m, s, u, g` };
+ }
+ if (flags.has(flag)) return { source: raw, error: `duplicate flag "${flag}"` };
+ flags.add(flag);
+ }
+ // d/k decide with .test(), which advances lastIndex on a global regex and
+ // would then alternate between calls. `g` means nothing there, so drop it.
+ if (op !== "s") flags.delete("g");
+
+ let re;
+ try {
+ re = new RegExp(pattern, [...flags].join(""));
+ } catch (err) {
+ // V8 already says "Invalid regular expression: <pattern>: <why>"; keep that
+ // detail without stuttering the prefix the doctors key off.
+ const message = err?.message || String(err);
+ return {
+ source: raw,
+ error: /^invalid regular expression/i.test(message)
+ ? `invalid${message.slice("Invalid".length)}`
+ : `invalid regular expression: ${message}`,
+ };
+ }
+
+ return { op, scope, re, replacement, source: raw };
+}
+
+export function compileInputFilters(rules) {
+ const compiled = [];
+ const errors = [];
+ const list = Array.isArray(rules) ? rules : [];
+ list.forEach((entry, index) => {
+ if (typeof entry !== "string") {
+ errors.push({ index, source: String(entry ?? ""), message: "rule must be a string" });
+ return;
+ }
+ if (!entry.trim()) return;
+ const parsed = parseInputFilterRule(entry);
+ if (parsed.error) errors.push({ index, source: parsed.source, message: parsed.error });
+ else compiled.push({ ...parsed, index });
+ });
+ return { rules: compiled, errors };
+}
+
+/**
+ * Run compiled rules over `text`.
+ *
+ * `substituteOnly` skips the d/k operations, so a caller that already took one
+ * drop decision for a turn can rewrite its individual pieces without taking a
+ * second, possibly contradictory one.
+ */
+export function applyInputFilters(text, compiled, { role = "", substituteOnly = false } = {}) {
+ const original = typeof text === "string" ? text : String(text ?? "");
+ let current = original;
+ for (const rule of Array.isArray(compiled) ? compiled : []) {
+ if (rule.scope && rule.scope !== role) continue;
+ if (rule.op === "s") {
+ current = current.replace(rule.re, rule.replacement);
+ continue;
+ }
+ if (substituteOnly) continue;
+ const matched = rule.re.test(current);
+ if ((rule.op === "d" && matched) || (rule.op === "k" && !matched)) {
+ return { text: "", changed: false, dropped: true, ruleIndex: rule.index, op: rule.op };
+ }
+ }
+ const finalText = current.trim();
+ return {
+ text: finalText,
+ changed: finalText !== original,
+ dropped: false,
+ ruleIndex: -1,
+ op: "",
+ };
+}
+
+/** Doctor-facing summary of both filter knobs on a resolved plugin config. */
+export function describeInputFilters(cfg = {}) {
+ return INPUT_FILTER_KNOBS.map(({ key, env, label }) => {
+ const configured = Array.isArray(cfg?.[key]) ? cfg[key] : [];
+ const compiled = compileInputFilters(configured);
+ const ops = { s: 0, d: 0, k: 0 };
+ for (const rule of compiled.rules) ops[rule.op] += 1;
+ const bits = [];
+ if (ops.s) bits.push(`${ops.s} substitute`);
+ if (ops.d) bits.push(`${ops.d} drop`);
+ if (ops.k) bits.push(`${ops.k} keep-only`);
+ const active = compiled.rules.length;
+ return {
+ key,
+ env,
+ label,
+ total: configured.length,
+ active,
+ summary: `${active} rule${active === 1 ? "" : "s"}${bits.length ? ` (${bits.join(", ")})` : ""}`,
+ errors: compiled.errors,
+ };
+ });
+}
diff --git a/examples/pi-coding-agent-extension/shared/capture-utils.mjs b/examples/pi-coding-agent-extension/shared/capture-utils.mjs
index 38e0baecc..22bc1b63c 100644
--- a/examples/pi-coding-agent-extension/shared/capture-utils.mjs
+++ b/examples/pi-coding-agent-extension/shared/capture-utils.mjs
@@ -1,4 +1,6 @@
// GENERATED FROM examples/memory-plugin-shared/lib. DO NOT EDIT.
+import { applyInputFilters, compileInputFilters } from "./input-filters.mjs";
+
const TEXT_BLOCK_TYPES = new Set(["text", "input_text", "output_text"]);
const TOOL_CALL_TYPES = new Set([
"tool_call",
@@ -354,6 +356,53 @@ export function extractPartsFromPayload(payload, options = {}) {
return parts;
}
+/**
+ * Prune blank text parts and run `cfg.captureFilters` over the ones that remain.
+ *
+ * One drop verdict is taken per turn, on the aggregate of its text parts; the
+ * individual parts are then rewritten with the substitutions only, so a rule can
+ * never both keep and drop the same turn. Tool call/result parts are carried
+ * through untouched — filters match conversation text, not tool payloads — but a
+ * turn dropped on its text takes its tool parts with it.
+ */
+export function filterCaptureParts(parts, role, cfg = {}) {
+ const kept = [];
+ for (const part of parts || []) {
+ if (!part) continue;
+ if (part.type !== "text") {
+ kept.push(part);
+ continue;
+ }
+ const text = typeof part.text === "string" ? part.text.trim() : "";
+ if (!text) continue;
+ kept.push(text === part.text ? part : { ...part, text });
+ }
+
+ const blank = { parts: kept, dropped: false };
+ const compiled = compileInputFilters(cfg?.captureFilters);
+ if (!compiled.rules.length || kept.length === 0) return blank;
+
+ const textParts = kept.filter((part) => part.type === "text");
+ if (textParts.length === 0) return blank;
+
+ const verdict = applyInputFilters(textParts.map((part) => part.text).join("\n\n"), compiled.rules, {
+ role,
+ });
+ if (verdict.dropped) return { parts: [], dropped: true };
+
+ const out = [];
+ for (const part of kept) {
+ if (part.type !== "text") {
+ out.push(part);
+ continue;
+ }
+ const shaped = applyInputFilters(part.text, compiled.rules, { role, substituteOnly: true });
+ if (!shaped.text) continue;
+ out.push(shaped.text === part.text ? part : { ...part, text: shaped.text });
+ }
+ return { parts: out, dropped: out.length === 0 };
+}
+
export function extractCaptureTurns(rolloutEntries, cfg = {}) {
const toolNameById = collectToolNamesByIdFromEntries(rolloutEntries);
const turns = [];
@@ -371,10 +420,15 @@ export function extractCaptureTurns(rolloutEntries, cfg = {}) {
toolMaxChars: cfg.captureToolMaxChars,
toolNameById,
});
- const decision = shouldCaptureText(rawText, role, cfg);
- if (!decision.shouldCapture && parts.length === 0) continue;
+ const shaped = filterCaptureParts(parts, role, cfg);
+ if (shaped.dropped) continue;
+ // With parts on the wire the drop decision was already taken above, so the
+ // text path only runs the filters for the `content` fallback. That also
+ // keeps `turn.text` faithful for callers that scan it for trigger words.
+ const decision = shouldCaptureText(rawText, role, cfg, { filters: shaped.parts.length === 0 });
+ if (!decision.shouldCapture && shaped.parts.length === 0) continue;
const text = decision.shouldCapture ? decision.text : "";
- turns.push({ role, text, parts });
+ turns.push({ role, text, parts: shaped.parts });
}
return turns;
}
@@ -486,12 +540,21 @@ function isPunctuationOnly(text) {
return !/[a-z0-9\u3400-\u9fff]/i.test(text);
}
-export function shouldCaptureText(text, role, cfg = {}) {
+export function shouldCaptureText(text, role, cfg = {}, { filters = true } = {}) {
const maxLength = cfg.captureMaxLength || 24000;
const sanitized = sanitizeCapturedText(text);
if (!sanitized) return { shouldCapture: false, reason: "empty", text: "" };
- const capped = truncateCaptureText(sanitized, maxLength);
+ let capped = truncateCaptureText(sanitized, maxLength);
+ if (filters) {
+ const compiled = compileInputFilters(cfg?.captureFilters);
+ if (compiled.rules.length) {
+ const verdict = applyInputFilters(capped, compiled.rules, { role });
+ if (verdict.dropped) return { shouldCapture: false, reason: "filtered", text: "" };
+ capped = verdict.text;
+ if (!capped) return { shouldCapture: false, reason: "empty", text: "" };
+ }
+ }
const compact = oneLine(capped);
const isToolSummary = /^\[tool-(?:call|result)\b/i.test(compact);
diff --git a/examples/pi-coding-agent-extension/shared/input-filters.mjs b/examples/pi-coding-agent-extension/shared/input-filters.mjs
new file mode 100644
index 000000000..e20bb03a7
--- /dev/null
+++ b/examples/pi-coding-agent-extension/shared/input-filters.mjs
@@ -0,0 +1,219 @@
+// GENERATED FROM examples/memory-plugin-shared/lib. DO NOT EDIT.
+/**
+ * Ordered, operator-configured regex filters for memory plugin input.
+ *
+ * Rules are sed-style strings, applied in order to a single piece of text:
+ * s/^\s*ultrathink\s+//i substitute (add `g` to replace every match)
+ * d|^\s*[/!]| drop the text when the pattern matches
+ * k/^\?ov\b/ keep the text only when the pattern matches
+ * user:d/^\s*\/clear\b/ apply to one role only
+ *
+ * A malformed rule becomes an entry in `errors` and is skipped rather than
+ * throwing, so a typo in a config file cannot take a hook down.
+ */
+
+const OPS = new Set(["s", "d", "k"]);
+const SCOPES = ["user", "assistant"];
+const ALLOWED_FLAGS = "imsug";
+
+export const INPUT_FILTER_KNOBS = [
+ {
+ key: "recallQueryFilters",
+ env: "OPENVIKING_RECALL_QUERY_FILTERS",
+ label: "recall query filters",
+ },
+ {
+ key: "captureFilters",
+ env: "OPENVIKING_CAPTURE_FILTERS",
+ label: "capture filters",
+ },
+];
+
+/**
+ * Read one delimited field. `\` always consumes the next character, and only
+ * `\<delim>` is un-escaped, so `\d` / `\b` / `\x2c` reach RegExp verbatim.
+ * Returns the index of the closing delimiter, or -1 when it is missing.
+ */
+function scanField(source, start, delim) {
+ let out = "";
+ let i = start;
+ while (i < source.length) {
+ const c = source[i];
+ if (c === "\\") {
+ const next = source[i + 1];
+ if (next === undefined) {
+ out += "\\";
+ i += 1;
+ continue;
+ }
+ out += next === delim ? next : "\\" + next;
+ i += 2;
+ continue;
+ }
+ if (c === delim) return { value: out, end: i };
+ out += c;
+ i += 1;
+ }
+ return { value: out, end: -1 };
+}
+
+export function parseInputFilterRule(source) {
+ if (typeof source !== "string") {
+ return { source: String(source ?? ""), error: "rule must be a string" };
+ }
+ const raw = source.trim();
+ if (!raw) return { source: raw, error: "rule is empty" };
+
+ let scope = "";
+ let rest = raw;
+ for (const candidate of SCOPES) {
+ if (rest.startsWith(`${candidate}:`)) {
+ scope = candidate;
+ rest = rest.slice(candidate.length + 1);
+ break;
+ }
+ }
+
+ const op = rest[0] || "";
+ if (!OPS.has(op)) {
+ return {
+ source: raw,
+ error: `unknown operation "${op}" — expected s (substitute), d (drop) or k (keep)`,
+ };
+ }
+ const delim = rest[1];
+ if (delim === undefined) {
+ return { source: raw, error: `missing delimiter after "${op}"` };
+ }
+ if (/[A-Za-z0-9\s\\]/.test(delim)) {
+ return {
+ source: raw,
+ error: `invalid delimiter ${JSON.stringify(delim)} — use punctuation such as / | # or :`,
+ };
+ }
+
+ const first = scanField(rest, 2, delim);
+ const pattern = first.value;
+ let replacement = null;
+ let flagsRaw = "";
+ if (op === "s") {
+ if (first.end < 0) {
+ return { source: raw, error: `missing ${JSON.stringify(delim)} after the pattern` };
+ }
+ const second = scanField(rest, first.end + 1, delim);
+ if (second.end < 0) {
+ return {
+ source: raw,
+ error: `missing ${JSON.stringify(delim)} after the replacement — write s${delim}pattern${delim}replacement${delim}`,
+ };
+ }
+ replacement = second.value;
+ flagsRaw = rest.slice(second.end + 1);
+ } else {
+ flagsRaw = first.end < 0 ? "" : rest.slice(first.end + 1);
+ }
+
+ if (!pattern) return { source: raw, error: "pattern is empty" };
+
+ const flags = new Set();
+ for (const flag of flagsRaw) {
+ if (!ALLOWED_FLAGS.includes(flag)) {
+ return { source: raw, error: `unknown flag "${flag}" — allowed flags are i, m, s, u, g` };
+ }
+ if (flags.has(flag)) return { source: raw, error: `duplicate flag "${flag}"` };
+ flags.add(flag);
+ }
+ // d/k decide with .test(), which advances lastIndex on a global regex and
+ // would then alternate between calls. `g` means nothing there, so drop it.
+ if (op !== "s") flags.delete("g");
+
+ let re;
+ try {
+ re = new RegExp(pattern, [...flags].join(""));
+ } catch (err) {
+ // V8 already says "Invalid regular expression: <pattern>: <why>"; keep that
+ // detail without stuttering the prefix the doctors key off.
+ const message = err?.message || String(err);
+ return {
+ source: raw,
+ error: /^invalid regular expression/i.test(message)
+ ? `invalid${message.slice("Invalid".length)}`
+ : `invalid regular expression: ${message}`,
+ };
+ }
+
+ return { op, scope, re, replacement, source: raw };
+}
+
+export function compileInputFilters(rules) {
+ const compiled = [];
+ const errors = [];
+ const list = Array.isArray(rules) ? rules : [];
+ list.forEach((entry, index) => {
+ if (typeof entry !== "string") {
+ errors.push({ index, source: String(entry ?? ""), message: "rule must be a string" });
+ return;
+ }
+ if (!entry.trim()) return;
+ const parsed = parseInputFilterRule(entry);
+ if (parsed.error) errors.push({ index, source: parsed.source, message: parsed.error });
+ else compiled.push({ ...parsed, index });
+ });
+ return { rules: compiled, errors };
+}
+
+/**
+ * Run compiled rules over `text`.
+ *
+ * `substituteOnly` skips the d/k operations, so a caller that already took one
+ * drop decision for a turn can rewrite its individual pieces without taking a
+ * second, possibly contradictory one.
+ */
+export function applyInputFilters(text, compiled, { role = "", substituteOnly = false } = {}) {
+ const original = typeof text === "string" ? text : String(text ?? "");
+ let current = original;
+ for (const rule of Array.isArray(compiled) ? compiled : []) {
+ if (rule.scope && rule.scope !== role) continue;
+ if (rule.op === "s") {
+ current = current.replace(rule.re, rule.replacement);
+ continue;
+ }
+ if (substituteOnly) continue;
+ const matched = rule.re.test(current);
+ if ((rule.op === "d" && matched) || (rule.op === "k" && !matched)) {
+ return { text: "", changed: false, dropped: true, ruleIndex: rule.index, op: rule.op };
+ }
+ }
+ const finalText = current.trim();
+ return {
+ text: finalText,
+ changed: finalText !== original,
+ dropped: false,
+ ruleIndex: -1,
+ op: "",
+ };
+}
+
+/** Doctor-facing summary of both filter knobs on a resolved plugin config. */
+export function describeInputFilters(cfg = {}) {
+ return INPUT_FILTER_KNOBS.map(({ key, env, label }) => {
+ const configured = Array.isArray(cfg?.[key]) ? cfg[key] : [];
+ const compiled = compileInputFilters(configured);
+ const ops = { s: 0, d: 0, k: 0 };
+ for (const rule of compiled.rules) ops[rule.op] += 1;
+ const bits = [];
+ if (ops.s) bits.push(`${ops.s} substitute`);
+ if (ops.d) bits.push(`${ops.d} drop`);
+ if (ops.k) bits.push(`${ops.k} keep-only`);
+ const active = compiled.rules.length;
+ return {
+ key,
+ env,
+ label,
+ total: configured.length,
+ active,
+ summary: `${active} rule${active === 1 ? "" : "s"}${bits.length ? ` (${bits.join(", ")})` : ""}`,
+ errors: compiled.errors,
+ };
+ });
+}
diff --git a/examples/zcode-memory-plugin/scripts/shared/capture-utils.mjs b/examples/zcode-memory-plugin/scripts/shared/capture-utils.mjs
index 38e0baecc..22bc1b63c 100644
--- a/examples/zcode-memory-plugin/scripts/shared/capture-utils.mjs
+++ b/examples/zcode-memory-plugin/scripts/shared/capture-utils.mjs
@@ -1,4 +1,6 @@
// GENERATED FROM examples/memory-plugin-shared/lib. DO NOT EDIT.
+import { applyInputFilters, compileInputFilters } from "./input-filters.mjs";
+
const TEXT_BLOCK_TYPES = new Set(["text", "input_text", "output_text"]);
const TOOL_CALL_TYPES = new Set([
"tool_call",
@@ -354,6 +356,53 @@ export function extractPartsFromPayload(payload, options = {}) {
return parts;
}
+/**
+ * Prune blank text parts and run `cfg.captureFilters` over the ones that remain.
+ *
+ * One drop verdict is taken per turn, on the aggregate of its text parts; the
+ * individual parts are then rewritten with the substitutions only, so a rule can
+ * never both keep and drop the same turn. Tool call/result parts are carried
+ * through untouched — filters match conversation text, not tool payloads — but a
+ * turn dropped on its text takes its tool parts with it.
+ */
+export function filterCaptureParts(parts, role, cfg = {}) {
+ const kept = [];
+ for (const part of parts || []) {
+ if (!part) continue;
+ if (part.type !== "text") {
+ kept.push(part);
+ continue;
+ }
+ const text = typeof part.text === "string" ? part.text.trim() : "";
+ if (!text) continue;
+ kept.push(text === part.text ? part : { ...part, text });
+ }
+
+ const blank = { parts: kept, dropped: false };
+ const compiled = compileInputFilters(cfg?.captureFilters);
+ if (!compiled.rules.length || kept.length === 0) return blank;
+
+ const textParts = kept.filter((part) => part.type === "text");
+ if (textParts.length === 0) return blank;
+
+ const verdict = applyInputFilters(textParts.map((part) => part.text).join("\n\n"), compiled.rules, {
+ role,
+ });
+ if (verdict.dropped) return { parts: [], dropped: true };
+
+ const out = [];
+ for (const part of kept) {
+ if (part.type !== "text") {
+ out.push(part);
+ continue;
+ }
+ const shaped = applyInputFilters(part.text, compiled.rules, { role, substituteOnly: true });
+ if (!shaped.text) continue;
+ out.push(shaped.text === part.text ? part : { ...part, text: shaped.text });
+ }
+ return { parts: out, dropped: out.length === 0 };
+}
+
export function extractCaptureTurns(rolloutEntries, cfg = {}) {
const toolNameById = collectToolNamesByIdFromEntries(rolloutEntries);
const turns = [];
@@ -371,10 +420,15 @@ export function extractCaptureTurns(rolloutEntries, cfg = {}) {
toolMaxChars: cfg.captureToolMaxChars,
toolNameById,
});
- const decision = shouldCaptureText(rawText, role, cfg);
- if (!decision.shouldCapture && parts.length === 0) continue;
+ const shaped = filterCaptureParts(parts, role, cfg);
+ if (shaped.dropped) continue;
+ // With parts on the wire the drop decision was already taken above, so the
+ // text path only runs the filters for the `content` fallback. That also
+ // keeps `turn.text` faithful for callers that scan it for trigger words.
+ const decision = shouldCaptureText(rawText, role, cfg, { filters: shaped.parts.length === 0 });
+ if (!decision.shouldCapture && shaped.parts.length === 0) continue;
const text = decision.shouldCapture ? decision.text : "";
- turns.push({ role, text, parts });
+ turns.push({ role, text, parts: shaped.parts });
}
return turns;
}
@@ -486,12 +540,21 @@ function isPunctuationOnly(text) {
return !/[a-z0-9\u3400-\u9fff]/i.test(text);
}
-export function shouldCaptureText(text, role, cfg = {}) {
+export function shouldCaptureText(text, role, cfg = {}, { filters = true } = {}) {
const maxLength = cfg.captureMaxLength || 24000;
const sanitized = sanitizeCapturedText(text);
if (!sanitized) return { shouldCapture: false, reason: "empty", text: "" };
- const capped = truncateCaptureText(sanitized, maxLength);
+ let capped = truncateCaptureText(sanitized, maxLength);
+ if (filters) {
+ const compiled = compileInputFilters(cfg?.captureFilters);
+ if (compiled.rules.length) {
+ const verdict = applyInputFilters(capped, compiled.rules, { role });
+ if (verdict.dropped) return { shouldCapture: false, reason: "filtered", text: "" };
+ capped = verdict.text;
+ if (!capped) return { shouldCapture: false, reason: "empty", text: "" };
+ }
+ }
const compact = oneLine(capped);
const isToolSummary = /^\[tool-(?:call|result)\b/i.test(compact);
diff --git a/examples/zcode-memory-plugin/scripts/shared/input-filters.mjs b/examples/zcode-memory-plugin/scripts/shared/input-filters.mjs
new file mode 100644
index 000000000..e20bb03a7
--- /dev/null
+++ b/examples/zcode-memory-plugin/scripts/shared/input-filters.mjs
@@ -0,0 +1,219 @@
+// GENERATED FROM examples/memory-plugin-shared/lib. DO NOT EDIT.
+/**
+ * Ordered, operator-configured regex filters for memory plugin input.
+ *
+ * Rules are sed-style strings, applied in order to a single piece of text:
+ * s/^\s*ultrathink\s+//i substitute (add `g` to replace every match)
+ * d|^\s*[/!]| drop the text when the pattern matches
+ * k/^\?ov\b/ keep the text only when the pattern matches
+ * user:d/^\s*\/clear\b/ apply to one role only
+ *
+ * A malformed rule becomes an entry in `errors` and is skipped rather than
+ * throwing, so a typo in a config file cannot take a hook down.
+ */
+
+const OPS = new Set(["s", "d", "k"]);
+const SCOPES = ["user", "assistant"];
+const ALLOWED_FLAGS = "imsug";
+
+export const INPUT_FILTER_KNOBS = [
+ {
+ key: "recallQueryFilters",
+ env: "OPENVIKING_RECALL_QUERY_FILTERS",
+ label: "recall query filters",
+ },
+ {
+ key: "captureFilters",
+ env: "OPENVIKING_CAPTURE_FILTERS",
+ label: "capture filters",
+ },
+];
+
+/**
+ * Read one delimited field. `\` always consumes the next character, and only
+ * `\<delim>` is un-escaped, so `\d` / `\b` / `\x2c` reach RegExp verbatim.
+ * Returns the index of the closing delimiter, or -1 when it is missing.
+ */
+function scanField(source, start, delim) {
+ let out = "";
+ let i = start;
+ while (i < source.length) {
+ const c = source[i];
+ if (c === "\\") {
+ const next = source[i + 1];
+ if (next === undefined) {
+ out += "\\";
+ i += 1;
+ continue;
+ }
+ out += next === delim ? next : "\\" + next;
+ i += 2;
+ continue;
+ }
+ if (c === delim) return { value: out, end: i };
+ out += c;
+ i += 1;
+ }
+ return { value: out, end: -1 };
+}
+
+export function parseInputFilterRule(source) {
+ if (typeof source !== "string") {
+ return { source: String(source ?? ""), error: "rule must be a string" };
+ }
+ const raw = source.trim();
+ if (!raw) return { source: raw, error: "rule is empty" };
+
+ let scope = "";
+ let rest = raw;
+ for (const candidate of SCOPES) {
+ if (rest.startsWith(`${candidate}:`)) {
+ scope = candidate;
+ rest = rest.slice(candidate.length + 1);
+ break;
+ }
+ }
+
+ const op = rest[0] || "";
+ if (!OPS.has(op)) {
+ return {
+ source: raw,
+ error: `unknown operation "${op}" — expected s (substitute), d (drop) or k (keep)`,
+ };
+ }
+ const delim = rest[1];
+ if (delim === undefined) {
+ return { source: raw, error: `missing delimiter after "${op}"` };
+ }
+ if (/[A-Za-z0-9\s\\]/.test(delim)) {
+ return {
+ source: raw,
+ error: `invalid delimiter ${JSON.stringify(delim)} — use punctuation such as / | # or :`,
+ };
+ }
+
+ const first = scanField(rest, 2, delim);
+ const pattern = first.value;
+ let replacement = null;
+ let flagsRaw = "";
+ if (op === "s") {
+ if (first.end < 0) {
+ return { source: raw, error: `missing ${JSON.stringify(delim)} after the pattern` };
+ }
+ const second = scanField(rest, first.end + 1, delim);
+ if (second.end < 0) {
+ return {
+ source: raw,
+ error: `missing ${JSON.stringify(delim)} after the replacement — write s${delim}pattern${delim}replacement${delim}`,
+ };
+ }
+ replacement = second.value;
+ flagsRaw = rest.slice(second.end + 1);
+ } else {
+ flagsRaw = first.end < 0 ? "" : rest.slice(first.end + 1);
+ }
+
+ if (!pattern) return { source: raw, error: "pattern is empty" };
+
+ const flags = new Set();
+ for (const flag of flagsRaw) {
+ if (!ALLOWED_FLAGS.includes(flag)) {
+ return { source: raw, error: `unknown flag "${flag}" — allowed flags are i, m, s, u, g` };
+ }
+ if (flags.has(flag)) return { source: raw, error: `duplicate flag "${flag}"` };
+ flags.add(flag);
+ }
+ // d/k decide with .test(), which advances lastIndex on a global regex and
+ // would then alternate between calls. `g` means nothing there, so drop it.
+ if (op !== "s") flags.delete("g");
+
+ let re;
+ try {
+ re = new RegExp(pattern, [...flags].join(""));
+ } catch (err) {
+ // V8 already says "Invalid regular expression: <pattern>: <why>"; keep that
+ // detail without stuttering the prefix the doctors key off.
+ const message = err?.message || String(err);
+ return {
+ source: raw,
+ error: /^invalid regular expression/i.test(message)
+ ? `invalid${message.slice("Invalid".length)}`
+ : `invalid regular expression: ${message}`,
+ };
+ }
+
+ return { op, scope, re, replacement, source: raw };
+}
+
+export function compileInputFilters(rules) {
+ const compiled = [];
+ const errors = [];
+ const list = Array.isArray(rules) ? rules : [];
+ list.forEach((entry, index) => {
+ if (typeof entry !== "string") {
+ errors.push({ index, source: String(entry ?? ""), message: "rule must be a string" });
+ return;
+ }
+ if (!entry.trim()) return;
+ const parsed = parseInputFilterRule(entry);
+ if (parsed.error) errors.push({ index, source: parsed.source, message: parsed.error });
+ else compiled.push({ ...parsed, index });
+ });
+ return { rules: compiled, errors };
+}
+
+/**
+ * Run compiled rules over `text`.
+ *
+ * `substituteOnly` skips the d/k operations, so a caller that already took one
+ * drop decision for a turn can rewrite its individual pieces without taking a
+ * second, possibly contradictory one.
+ */
+export function applyInputFilters(text, compiled, { role = "", substituteOnly = false } = {}) {
+ const original = typeof text === "string" ? text : String(text ?? "");
+ let current = original;
+ for (const rule of Array.isArray(compiled) ? compiled : []) {
+ if (rule.scope && rule.scope !== role) continue;
+ if (rule.op === "s") {
+ current = current.replace(rule.re, rule.replacement);
+ continue;
+ }
+ if (substituteOnly) continue;
+ const matched = rule.re.test(current);
+ if ((rule.op === "d" && matched) || (rule.op === "k" && !matched)) {
+ return { text: "", changed: false, dropped: true, ruleIndex: rule.index, op: rule.op };
+ }
+ }
+ const finalText = current.trim();
+ return {
+ text: finalText,
+ changed: finalText !== original,
+ dropped: false,
+ ruleIndex: -1,
+ op: "",
+ };
+}
+
+/** Doctor-facing summary of both filter knobs on a resolved plugin config. */
+export function describeInputFilters(cfg = {}) {
+ return INPUT_FILTER_KNOBS.map(({ key, env, label }) => {
+ const configured = Array.isArray(cfg?.[key]) ? cfg[key] : [];
+ const compiled = compileInputFilters(configured);
+ const ops = { s: 0, d: 0, k: 0 };
+ for (const rule of compiled.rules) ops[rule.op] += 1;
+ const bits = [];
+ if (ops.s) bits.push(`${ops.s} substitute`);
+ if (ops.d) bits.push(`${ops.d} drop`);
+ if (ops.k) bits.push(`${ops.k} keep-only`);
+ const active = compiled.rules.length;
+ return {
+ key,
+ env,
+ label,
+ total: configured.length,
+ active,
+ summary: `${active} rule${active === 1 ? "" : "s"}${bits.length ? ` (${bits.join(", ")})` : ""}`,
+ errors: compiled.errors,
+ };
+ });
+}