Skip to content

armgit / mso

Public Imported from GitHub
0

Browser-based graphical shell and control plane for a Linux server you own — real terminal, file manager, live metrics and BYOK AI in one mobile-first workspace. Next.js 16 · React 19 · node-pty · Tailwind v4 · shadcn/ui. Public alpha, self-hosted.

feat(mcp): add read-only result pipeline

Rahman835ec245fd41
diff --git a/.gitignore b/.gitignore
index b42b9e57..99bc9f93 100644
--- a/.gitignore
+++ b/.gitignore
@@ -22,6 +22,9 @@ os-browser/*
!os-browser/package-lock.json
convex-backend/
.mso/
+# RASMIC repo-local orchestration memory/evidence is runtime state. It must survive
+# locally but must not make the MSO source checkout dirty and block `mso update`.
+.agent/
# local session progress log (not tracked)
progress.md
diff --git a/CLAUDE.md b/CLAUDE.md
index 32773cb9..0e291cc5 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -349,8 +349,8 @@ the real runtime and working analogue before changing a tool or skill contract.
- **Tool names are a cross-repo contract.** `rahmanef63/connectors-gateway` registered
mso as a connector on 2026-08-17 and historically pinned a subset of the MCP names as
literal `x-upstream` strings.
- <!-- mcp-toolset: server=1.6.0 version=2026.09.02.9 tools=69 read=33 write=24 exec=12 -->
- MSO currently exposes **70 transport tools**: **69 model/operator tools** (33 read, 24 write,
+ <!-- mcp-toolset: server=1.6.0 version=2026.09.03.1 tools=70 read=34 write=24 exec=12 -->
+ MSO currently exposes **71 transport tools**: **70 model/operator tools** (34 read, 24 write,
12 exec) plus the app-only `workflow_status` bridge; `GET /mcp` plus `_meta.toolset` expose
the current version/hash/name manifest.
Treat the gateway mapping as a separate cross-repo contract and verify it before renaming. Renaming or removing
diff --git a/README.md b/README.md
index 61c0b2e4..ac075904 100644
--- a/README.md
+++ b/README.md
@@ -79,7 +79,7 @@ For a real deployment, put MSO behind **Tailscale, a VPN, or a TLS reverse proxy
- **Message live local sessions by a short name** — every durable same-principal MSO session gets a unique familiar handle such as `@milo`, `@luna`, or `@nara`, independent from its longer auto-generated session title. `/rename <name>` changes that handle, while `/title <text>` changes the session description. `@milo …` resolves only agents with a current lease **and** a live receiver subscription, then creates an explicitly correlated request; replies relay deterministically into the originating conversation while notify-only events remain passive. Presence lease (`idle`/`busy`/`offline`) and receiver subscription (`consumerConnected`) are reported separately so an idle-but-unsubscribed target is diagnosable. See [Local Agents](./docs/LOCAL-AGENTS.md).
- **Spawn focused same-session subagents** — `/spawn` and `agent_subagent_run` create bounded foreground workers with isolated context, explicit delegated scope, turn/timeout caps, no recursive spawn, and final-result-only return. They are not Local Agent peers and do not run autonomously after the parent call returns. See [Subagents](./docs/SUBAGENTS.md).
- **Delegate between remote agents with standard A2A v1** — register public HTTPS peers, attach private API-key/Bearer/OAuth access-token profiles, stream remote tasks over SSE, or expose MSO's authenticated inbound Agent Card when a public HTTPS origin is configured. Inbound credentials are owner-minted `read | write | exec` capabilities and run in a memory-isolated task context. See [A2A](./docs/A2A.md).
-- **Keep agents efficient independently of the model provider** — the MCP-first [Cognitive Runtime](./docs/COGNITIVE-RUNTIME.md) isolates conversations from OAuth identity, budgets model context/tool output, selects a compact per-turn capability set without weakening scopes, compacts durable sessions with sanitized 30-day backup archives, and learns only verified workflow recipes. `bun run bench:cognitive` guards routing recall and context footprint.
+- **Keep agents efficient independently of the model provider** — the MCP-first [Cognitive Runtime](./docs/COGNITIVE-RUNTIME.md) isolates conversations from OAuth identity, budgets model context/tool output, uses the RASMIC catalog-first router to select a tiny per-turn capability pack without weakening scopes, compacts durable sessions with sanitized 30-day backup archives, learns only verified workflow recipes, promotes repeated procedures only through the eval-gated Tool Forge, and batches/filter/aggregates independent reads through `read_pipeline` before raw data reaches the model. `bun run bench:cognitive` and `bun run bench:pipeline` guard routing, correctness and context savings.
- **Make orchestration risk-aware and self-improving** — [RASMIC](./docs/RASMIC.md) classifies task risk/contention, retrieves compact repo-local memory, records manual user tests and Evidence Receipts, detects path/shared-resource collisions before merge conflicts, and promotes repeated safe routes from observed traces → recipes → tested bounded scripts.
- **Add app slices** — features are modular under `frontend/slices/<slug>/`.
- **Personalize the interface** — macOS, Windows, iOS, and Android shell layouts are UI preferences, not the core product.
diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md
index a83e3d2f..b222ce03 100644
--- a/docs/ARCHITECTURE.md
+++ b/docs/ARCHITECTURE.md
@@ -160,8 +160,8 @@ park a visible Approve/Deny card. The complete semantic contract is
**MCP** is for external clients such as ChatGPT, Claude.ai and Cursor. Its catalog lives in
`lib/mcp/`; access is controlled by the OAuth token scope rather than Alfa approval cards.
-<!-- mcp-toolset: server=1.6.0 version=2026.09.02.9 tools=69 read=33 write=24 exec=12 -->
-At the current toolset it exposes **70 transport tools**: **69 model/operator tools** (33 read, 24 write, 12 exec) plus the app-only `workflow_status` progress bridge. Project-specific
+<!-- mcp-toolset: server=1.6.0 version=2026.09.03.1 tools=70 read=34 write=24 exec=12 -->
+At the current toolset it exposes **71 transport tools**: **70 model/operator tools** (34 read, 24 write, 12 exec) plus the app-only `workflow_status` progress bridge. Project-specific
function names remain data behind `project_capabilities` / `project_function_call`, so one
project cannot dynamically rewrite the global MCP tool prefix. `GET /mcp` remains the live count/hash authority.
diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md
index 2f8f58e4..3550d7d6 100644
--- a/docs/CHANGELOG.md
+++ b/docs/CHANGELOG.md
@@ -8,6 +8,7 @@ this is the *what*, and it is what Settings → About shows as “What's new”.
**Added**
+- `mcp` add read-only result pipeline
- add RASMIC orchestration memory automation
- `agent` section terminal and recover API errors
- `agent` name sessions and wrap composer
diff --git a/docs/CHATGPT-PLUGIN.md b/docs/CHATGPT-PLUGIN.md
index 1c21b41e..cb30ea68 100644
--- a/docs/CHATGPT-PLUGIN.md
+++ b/docs/CHATGPT-PLUGIN.md
@@ -12,10 +12,10 @@
> <https://help.openai.com/en/articles/12584461-developer-mode-and-mcp-apps-in-chatgpt-beta>
> and <https://help.openai.com/en/articles/20001256-plugins-in-chatgpt-and-codex>.
-<!-- mcp-toolset: server=1.6.0 version=2026.09.02.9 tools=69 read=33 write=24 exec=12 -->
+<!-- mcp-toolset: server=1.6.0 version=2026.09.03.1 tools=70 read=34 write=24 exec=12 -->
-MSO currently exposes MCP server **1.6.0**, toolset **2026.09.02.9**: **70 transport tools** total; **69 model/operator tools**
-(33 read, 24 write, 12 exec) plus app-only `workflow_status` for the progress widget. Use `GET /mcp` or Settings → MCP as the live authority if
+MSO currently exposes MCP server **1.6.0**, toolset **2026.09.03.1**: **71 transport tools** total; **70 model/operator tools**
+(34 read, 24 write, 12 exec) plus app-only `workflow_status` for the progress widget. Use `GET /mcp` or Settings → MCP as the live authority if
this document and a deployed instance ever disagree.
### Local-agent and subagent tools
@@ -145,7 +145,7 @@ A token sees a scope prefix; there is no per-project or per-agent hidden tool fi
`tools/list` filters the catalog and `tools/call` independently re-checks the required
scope.
-### `read` — 33 model/operator tools
+### `read` — 34 model/operator tools
- `a2a_agent_discover`
- `a2a_agents_list`
@@ -173,6 +173,7 @@ scope.
- `project_capabilities`
- `project_memory_search`
- `projects_list`
+- `read_pipeline`
- `screen_capture`
- `skills_list`
- `skills_read`
@@ -231,6 +232,12 @@ with the same care as a remote shell.
+### Read pipeline for lower latency and context
+
+Use `read_pipeline` when several independent reads can be batched or when a large read result should be filtered/projected/aggregated before ChatGPT sees it. The pipeline is **read scope only**: child calls reuse the original tool's host guards and rate limits, inherit the current session/workflow, and are forced to read scope even if the OAuth bearer is `exec`.
+
+The transform language is deliberately data-only: simple dotted paths, scalar comparisons, field projection, sorting, de-duplication, limiting, and numeric/count aggregates. There is no shell, JavaScript, arbitrary expression, nested pipeline, child `workflow_id`, or write/exec capability. A total 15-second response deadline and bounded intermediate/final results keep the orchestration predictable. P3's deterministic fixture measured 4 → 1 model round-trip and 99.2% lower model-visible bytes with exact answers.
+
### Tool Forge approval model
`tool_forge_propose` creates only inert candidates from repeated verified workflow recipes. Skill candidates are guidance only; a project-function candidate must point to an existing project-owned Node script and must come from an exec-scope verified recipe. Generated shell/code, credential-like fixture data, and implicit scope escalation are rejected.
@@ -272,7 +279,7 @@ Settings → MCP:
server: 1.6.0
toolset: 2026.09.02.7
hash: <live schema hash>
-count: 69 model/operator tools (+ 1 app-only bridge)
+count: 70 model/operator tools (+ 1 app-only bridge)
```
Use this sequence after an MSO MCP change:
diff --git a/docs/COGNITIVE-RUNTIME.md b/docs/COGNITIVE-RUNTIME.md
index 01313bf8..56c457e8 100644
--- a/docs/COGNITIVE-RUNTIME.md
+++ b/docs/COGNITIVE-RUNTIME.md
@@ -131,7 +131,7 @@ probe. Overall product superiority also cannot be inferred from a prompt-size re
must add equivalent task-success, tool-error, token-per-success, latency-per-success, and policy-compliance
scenarios before making that claim.
-## Current P2 + A2A baseline
+## Historical P2 + A2A baseline
On the development host after P2 + outbound A2A integration:
@@ -153,20 +153,40 @@ A non-interactive full MSO tool loop is now available as `mso agent --oneshot <p
These numbers are reproducible harness baselines, not an overall quality leaderboard. Cross-agent ranking is permitted only when at least two runners complete the same task with matching model-family evidence; one-task latency order is explicitly not an overall-agent claim.
-## P2 status and next direction
+## Current P3 + RASMIC baseline
-P1 supplies typed/provenance-aware temporal memory, deterministic retrieval evaluation, workflow/tool quality telemetry, the non-interactive MSO Agent runner, and a fail-closed cross-agent scratch harness. P2 adds the first self-improvement primitive without granting opaque autonomy: an **eval-gated Tool Forge**.
+P3 runs on the catalog-first RASMIC router rather than the older lexical active-tool selector. The merged deterministic harness now reports:
-Forge only creates inert candidates from repeated high-quality recipes. Skill candidates are deterministic redacted guidance; executable project-function candidates require an exec-scope verified recipe and may reference only existing project-owned Node code. Fixtures run in the dedicated cached Docker sandbox with no network and a read-only project mount. Evaluation records current toolset, candidate/target state, source SHA-256 and exact sandbox image evidence; promotion requires literal confirmation and immediately repeats the checks. A candidate cannot silently raise scope, overwrite an existing capability, or turn generated text into host-executable code.
+| Metric | MSO P3 + RASMIC | Hermes local baseline |
+|---|---:|---:|
+| Full MSO transport catalog | 71 tools | — |
+| Full MSO schema bytes | 55,560 | — |
+| Average MSO active tools / turn | 2.6 | — |
+| Average active schema bytes / turn | 2,261 | 44,758 tool-schema bytes |
+| MSO schema reduction vs full catalog | 95.9% | — |
+| Required-tool routing recall / catalog hit | 100% / 100% | not the same benchmark |
+| Average routing text / requested history budget | 77 bytes / 10,545 tokens | — |
+| Typed-memory fixture accuracy | 100% | not implemented in this harness |
+| P3 read fixture model round-trips | 1 vs 4 raw calls (75% lower) | not implemented in this harness |
+| P3 read fixture model-visible bytes | 1,891 vs 232,205 raw bytes (99.2% lower) | not implemented in this harness |
+| P3 read fixture correctness / determinism | exact / yes | not the same benchmark |
+
+The Hermes comparison above is **tool-schema footprint only**. It does not establish overall product superiority, and OpenClaw still exposes no directly equivalent offline prompt-size probe in this harness. Overall ranking remains blocked until the same model/provider completes the same task-quality corpus on each agent.
+
+## P3 status and next direction
+
+P1 supplies typed/provenance-aware temporal memory and workflow quality telemetry. P2 adds the eval-gated Tool Forge. RASMIC makes intent routing catalog-first and risk/evidence aware. P3 adds `read_pipeline`: one read-scope MCP primitive that executes up to six independent eligible read calls server-side and reduces raw results through an intentionally non-programmable transform language.
+
+Child reads retain their original MSO host/path guards and rate-limit buckets, inherit the parent session/workflow, are internally forced to read scope, and cannot inject a child workflow id. Nested pipelines, screenshots/direct-file results, wait/poll tools, prototype traversal, arbitrary code and every write/exec tool are rejected. Raw results, transformed results and total wall time are bounded. Parallel results are emitted in declaration order.
-Outbound A2A v1 remains orthogonal: it adds remote-agent delegation, while Forge turns repeated **local verified procedures** into reviewed project capabilities. The combined routing benchmark covers these surfaces with 100% required-tool recall while selecting only 2.7 of 70 tools on average; it also requires the catalog-first routing and history-budget gates above.
+The first P3 fixture proves the intended economics without weakening correctness: four independent model read calls become one model round-trip, while 232,205 raw model-visible bytes become 1,891 bytes after filtering/aggregation. Combined with RASMIC, the current router selects only 2.6 of 71 transport tools on average while keeping 100% required-tool recall in the deterministic corpus.
-The next phase should continue from measured primitives:
+The next phase should continue from measured primitives rather than add broad autonomy:
-1. **Benchmark corpus expansion** — keep adding read/write/recovery/security task classes and configure Hermes/OpenClaw with an actually equivalent model/provider path before any overall comparison.
-2. **Real token-per-success telemetry** — use provider-reported usage plus the non-prompt routing telemetry; missing provider usage remains unknown, never zero.
-3. **Memory calibration** — grow post-P1 quality telemetry and validate deterministic relation/conflict policy against real corrections.
-4. **Optional external memory transport** — keep repo-local `.agent/` canonical; any sync service consumes/produces the portable bundle and never becomes a runtime dependency.
+1. **Comparable task-quality corpus** — add read/write/recovery/security scenarios and configure Hermes/OpenClaw with an equivalent model/provider path before any overall ranking.
+2. **Real token/cost-per-success telemetry** — use provider-reported usage plus non-prompt routing/pipeline telemetry; missing usage remains unknown, never zero.
+3. **Memory calibration** — grow post-P1 quality telemetry and validate confidence/authority/temporal policies against real user corrections.
+4. **Pipeline composition only if earned by evals** — references/fan-out between child reads should not be added until a real corpus proves material gain over the intentionally simple independent-call contract.
## Security notes
@@ -175,6 +195,7 @@ The next phase should continue from measured primitives:
- A discovered Skill or recipe cannot grant a tool the caller's token does not already permit.
- Untrusted Skills keep their instructions withheld until promoted through the existing trust process.
- Tool Forge candidates are inert. Executable fixtures require the labelled local Docker sandbox; evaluation never auto-pulls an image or executes generated shell/code.
+- `read_pipeline` is an efficiency surface, not authority: only eligible read-only tools may run, child scope is forced to read, child rate limits/host guards stay active, and arbitrary program execution is impossible by contract.
- `exec_run` remains full service-user shell power at `exec` scope; cognitive routing is not a sandbox.
- Never expose raw ChatGPT conversation ids, tokens, credentials, hidden chain-of-thought, or unrestricted
tool output in memory/recipes/archives.
diff --git a/docs/CONNECTORS-GATEWAY-INTEGRATION.md b/docs/CONNECTORS-GATEWAY-INTEGRATION.md
index 6aace2a0..a8b5f459 100644
--- a/docs/CONNECTORS-GATEWAY-INTEGRATION.md
+++ b/docs/CONNECTORS-GATEWAY-INTEGRATION.md
@@ -4,10 +4,10 @@
> can evolve independently, so do not claim its mapping count is current without checking
> that repository at the time of the change.
-<!-- mcp-toolset: server=1.6.0 version=2026.09.02.9 tools=69 read=33 write=24 exec=12 -->
+<!-- mcp-toolset: server=1.6.0 version=2026.09.03.1 tools=70 read=34 write=24 exec=12 -->
-MSO currently exposes MCP server **1.6.0**, toolset **2026.09.02.9**: **70 transport tools** total; **69 model/operator tools**
-(33 read, 24 write, 12 exec) plus app-only `workflow_status`. `GET /mcp` exposes the live names, version/hash and scoped
+MSO currently exposes MCP server **1.6.0**, toolset **2026.09.03.1**: **71 transport tools** total; **70 model/operator tools**
+(34 read, 24 write, 12 exec) plus app-only `workflow_status`. `GET /mcp` exposes the live names, version/hash and scoped
manifest and is the machine-readable parity source.
## Local collaboration additions
@@ -39,9 +39,9 @@ Before changing an MCP tool name:
## Current MSO catalog
-### Read (33 model/operator)
+### Read (34 model/operator)
-`a2a_agent_discover`, `a2a_agents_list`, `a2a_task_get`, `agent_memory_read`, `agent_memory_search`, `agent_session_current`, `agent_session_resume`, `agent_sessions_list`, `apps_list`, `apps_logs`, `browser_status`, `cloudflare_zones_list`, `dokploy_projects_list`, `exec_job_status`, `fs_list`, `fs_read`, `fs_search`, `fs_usage`, `infra_provider_doctor`, `infra_providers_list`, `local_agent_inbox`, `local_agent_request_wait`, `local_agents_list`, `project_capabilities`, `project_memory_search`, `projects_list`, `screen_capture`, `skills_list`, `skills_read`, `skills_search`, `sys_processes`, `sys_stats`, `tool_forge_candidates`.
+`a2a_agent_discover`, `a2a_agents_list`, `a2a_task_get`, `agent_memory_read`, `agent_memory_search`, `agent_session_current`, `agent_session_resume`, `agent_sessions_list`, `apps_list`, `apps_logs`, `browser_status`, `cloudflare_zones_list`, `dokploy_projects_list`, `exec_job_status`, `fs_list`, `fs_read`, `fs_search`, `fs_usage`, `infra_provider_doctor`, `infra_providers_list`, `local_agent_inbox`, `local_agent_request_wait`, `local_agents_list`, `project_capabilities`, `project_memory_search`, `projects_list`, `read_pipeline`, `screen_capture`, `skills_list`, `skills_read`, `skills_search`, `sys_processes`, `sys_stats`, `tool_forge_candidates`.
### Write (24 beyond read)
@@ -73,6 +73,8 @@ MSO learned workflow. For multi-step orchestration, expose the workflow trio and
the exact id returned by `workflow_start`.
+`read_pipeline` is one stable global read action rather than a gateway-generated action per child call. Gateways should forward its declarative JSON contract unchanged: the server permits only eligible read-only tools, inherits the caller session/workflow, keeps each child's normal rate limit and host guard, and returns bounded transformed evidence. It must never be translated into arbitrary script execution or used as a proxy for write/exec actions.
+
Tool Forge remains a stable global action family as well: gateways should expose the four generic `tool_forge_*` names, never synthesize one action per generated candidate. Candidate ids are data. Executable evaluation is server-side in the dedicated local Docker sandbox, and promotion remains exec-scope with exact confirmation.
## Project functions are data, not dynamic MCP names
diff --git a/docs/MCP.md b/docs/MCP.md
index cec04502..c458736d 100644
--- a/docs/MCP.md
+++ b/docs/MCP.md
@@ -75,7 +75,7 @@ Picked per token, on the consent screen, capped by `OS_MCP_MAX_SCOPE`. The highe
| Scope | Tools |
|---|---|
-| `read` | `a2a_agent_discover` `a2a_agents_list` `a2a_task_get` `agent_memory_read` `agent_memory_search` `agent_session_current` `agent_session_resume` `agent_sessions_list` `apps_list` `apps_logs` `browser_status` `cloudflare_zones_list` `dokploy_projects_list` `exec_job_status` `fs_list` `fs_read` `fs_search` `fs_usage` `infra_provider_doctor` `infra_providers_list` `local_agent_inbox` `local_agent_request_wait` `local_agents_list` `project_capabilities` `project_memory_search` `projects_list` `screen_capture` `skills_list` `skills_read` `skills_search` `sys_processes` `sys_stats` `tool_forge_candidates` |
+| `read` | `a2a_agent_discover` `a2a_agents_list` `a2a_task_get` `agent_memory_read` `agent_memory_search` `agent_session_current` `agent_session_resume` `agent_sessions_list` `apps_list` `apps_logs` `browser_status` `cloudflare_zones_list` `dokploy_projects_list` `exec_job_status` `fs_list` `fs_read` `fs_search` `fs_usage` `infra_provider_doctor` `infra_providers_list` `local_agent_inbox` `local_agent_request_wait` `local_agents_list` `project_capabilities` `project_memory_search` `projects_list` `read_pipeline` `screen_capture` `skills_list` `skills_read` `skills_search` `sys_processes` `sys_stats` `tool_forge_candidates` |
| `write` | + `a2a_agent_register` `a2a_agent_remove` `agent_memory_forget` `agent_memory_remember` `agent_session_note` `agent_session_rename` `apps_power` `cloudflare_dns_upsert` `dokploy_project_ensure` `fs_copy` `fs_delete` `fs_mkdir` `fs_move` `fs_upload_file` `fs_write` `hostinger_dns_upsert` `local_agent_message_send` `local_agent_reply` `project_memory_upsert` `project_script_run` `tool_forge_propose` `workflow_cancel` `workflow_finish` `workflow_start` |
| `exec` | + `a2a_handoff` `a2a_message_send` `a2a_task_cancel` `agent_subagent_run` `browser_power` `exec_job_cancel` `exec_job_start` `exec_run` `local_agent_request` `project_function_call` `tool_forge_evaluate` `tool_forge_promote` |
@@ -126,9 +126,9 @@ The catalog has a stable server version plus a schema-derived toolset signature.
Settings → MCP shows the current version/hash/count and stores a browser-local acknowledgement when the operator marks ChatGPT refreshed. A later signature change becomes an explicit stale-snapshot warning. This does not mutate ChatGPT remotely; it makes the required refresh visible instead of relying on memory.
-<!-- mcp-toolset: server=1.6.0 version=2026.09.02.9 tools=69 read=33 write=24 exec=12 -->
+<!-- mcp-toolset: server=1.6.0 version=2026.09.03.1 tools=70 read=34 write=24 exec=12 -->
-Current transport catalog: **70 tools**, server `1.6.0` / toolset `2026.09.02.9`.
+Current transport catalog: **71 tools**, server `1.6.0` / toolset `2026.09.03.1`.
The model/operator catalog is **69 tools** (33 read, 24 write, 12 exec); `workflow_status` is the
one app-only MCP Apps bridge used by the progress widget and is documented separately.
`agent_memory_search` is the typed-memory retrieval surface. It resolves semantic/episodic/procedural claims at an optional point in time, returns confidence/provenance and competing effective claims, and can expose superseded/retracted history when explicitly requested. `agent_memory_remember` remains the write surface and now accepts typed metadata; raw ChatGPT conversation ids are never stored as provenance.
@@ -158,6 +158,14 @@ message/context/task arguments; a handoff never auto-attaches the caller's hidde
memory or raw session/workflow ids. See [`A2A.md`](./A2A.md) for the transport and trust contract.
+### Programmatic read-only orchestration
+
+`read_pipeline` is P3's provider-neutral batching seam. One read-scope call can execute **1–6 eligible read-only MSO tools** in parallel or sequentially, then apply bounded declarative transforms (`path`, scalar filters, `select`, `sort`, `uniqueBy`, `limit`, and `count`/`sum`/`avg`/`min`/`max`) before returning data to the model. It accepts no JavaScript, shell, arbitrary expressions, nested pipeline, child `workflow_id`, screenshot/direct-file result, wait/poll primitive, or write/exec tool.
+
+Each child call still runs the original MSO tool handler and host/path guards, keeps that child's own rate-limit bucket, inherits the parent durable session/workflow, and is internally forced to `read` scope even when the parent bearer holds `exec`. Raw child results cap at 1 MiB, transformed output at 12 KiB per child and 40 KiB total, with a 15-second total response deadline. Parallel completion is reassembled in declaration order for deterministic caching and evaluation.
+
+On the deterministic P3 fixture, four independent raw reads become **one model tool round-trip** (75% fewer), while model-visible bytes fall from **232,205 to 1,891 (99.2%)** with exact deterministic answers. This is a context/latency optimization, not a new permission or sandbox boundary.
+
### Eval-gated Tool Forge
Tool Forge is a **candidate pipeline**, not a self-authorizing code generator. `tool_forge_propose` accepts only learned recipes with P1 quality telemetry, at least two verified successes, at least 90% workflow success, and a fully-completed best trace. `tool_forge_candidates` exposes reviewable metadata while redacting fixed argv, fixture payloads, and internal workflow targets. Candidates are inert until explicit evaluation and promotion.
diff --git a/docs/PROGRESS.md b/docs/PROGRESS.md
index 809f2c6a..2079a065 100644
--- a/docs/PROGRESS.md
+++ b/docs/PROGRESS.md
@@ -2,11 +2,19 @@
Running log of what shipped each phase. Newest at top.
+## 2026-09-03 — Cognitive Runtime P3 read-only orchestration
+
+- **Self-update invariant fixed:** RASMIC `.agent/` evidence/memory is runtime-local state and is now ignored by the MSO repo. This prevents successful workflow memory writes from making the canonical checkout dirty and causing `mso update` to refuse its own upgrade.
+- **Programmatic reads without program execution:** added read-scope `read_pipeline`, batching 1–6 independent eligible read tools in parallel/sequential mode and reducing their results with bounded declarative path/filter/select/sort/unique/limit/count/sum/avg/min/max operations. No shell, JavaScript, nested pipeline, foreign child `workflow_id`, screenshot/direct-file result, wait/poll primitive, write tool or exec tool can enter the pipeline.
+- **Guard reuse instead of a second security stack:** direct MCP dispatch and pipeline children now share one per-tool rate-limit helper. Child calls still execute the original tool handler and host/path checks and are internally forced to `read` scope even when the parent bearer is `exec`. Raw results cap at 1 MiB/child; transformed output caps at 12 KiB/child and 40 KiB total; the complete response has a 15-second wall-time deadline. Parallel results are emitted in declaration order for deterministic caching/evals.
+- **Measured context reduction:** `bench:pipeline` preserves exact deterministic fixture answers while reducing four independent model round-trips to one (**75%**) and model-visible bytes from **232,205 to 1,891 (99.2%)**. Integrated Cognitive Runtime remains **100% required-tool routing recall**, **100% typed-memory fixture accuracy**, and averages **2.6/71 active tools** with **95.9% schema reduction** under the merged RASMIC catalog-first router.
+- **Docs/tool catalog SSOT:** `check-docs.mjs` now derives tool modules from the canonical `lib/mcp/tools.ts` composition and fails on orphan `tools-*.ts` modules, so adding a new tool module cannot silently escape MCP/ChatGPT/connectors documentation. Toolset advances to `2026.09.03.1`: 71 transport tools = 70 model/operator tools (34 read / 24 write / 12 exec) + app-only `workflow_status`. The docs checker follows the recursive `tools-*` import closure from `lib/mcp/tools.ts`, so split RASMIC workflow modules remain in the same SSOT.
+
## 2026-09-03 — MSO 1.12 documentation and Agent contract cleanup
- **Full Markdown audit:** reviewed all 71 authored Markdown files in the repository, preserving dated release/audit/design evidence while correcting current-reference drift. Structural validation reports zero malformed fence/heading issues and all 125 relative Markdown links resolve.
- **Current Agent docs:** synchronized `TESTING-HANDOFF`, Local Agents, install/provider guidance, troubleshooting, architecture and the documentation map with the shipped 1.12 identity-first composer, sectioned transcript, compact exact approval and recoverable `not_started | completed | uncertain` mutation semantics. Historical 1.8–1.11 release notes remain intact rather than being rewritten as if they were current instructions.
-- **MCP/A2A drift fixed:** current references now agree on server `1.6.0`, toolset `2026.09.02.7`, 67 transport tools = 66 model/operator tools (32 read / 22 write / 12 exec) + app-only `workflow_status`. Stale 31/29/59/56-tool claims and obsolete anonymous-only A2A wording were removed from current docs; historical benchmark snapshots were relabeled as snapshots instead of live metrics.
+- **MCP/A2A drift fixed:** at that 1.12 documentation checkpoint, references agreed on server `1.6.0`, toolset `2026.09.02.7`, 67 transport tools = 66 model/operator tools (32 read / 22 write / 12 exec) + app-only `workflow_status`. Stale 31/29/59/56-tool claims and obsolete anonymous-only A2A wording were removed from current docs; historical benchmark snapshots were relabeled as snapshots instead of live metrics.
- **Docs gate strengthened:** `check-docs.mjs` now verifies the MCP marker in architecture/operator docs and exact scope membership in `docs/MCP.md`, `docs/CHATGPT-PLUGIN.md`, and `docs/CONNECTORS-GATEWAY-INTEGRATION.md`, so a future tool addition cannot leave one human catalog silently stale.
- **Unused surface cleanup:** removed the dead `coreToolNames()` helper and stopped exporting Agent constants/helpers that are private implementation details while retaining every internally used behavior. No compatibility placeholder or public CLI/MCP capability was removed.
diff --git a/docs/RASMIC.md b/docs/RASMIC.md
index be44f046..16fb2ef4 100644
--- a/docs/RASMIC.md
+++ b/docs/RASMIC.md
@@ -62,6 +62,10 @@ A successful tool call is not proof that the user-facing result works. New HIGH-
└── evidence/
```
+The `.agent/` tree is **runtime-local orchestration state**, not source code. MSO's own repo ignores it so evidence/memory can survive locally without making the canonical checkout dirty or blocking `mso update`. Projects that adopt RASMIC should likewise keep `.agent/` out of normal source commits; portability/sync is an explicit memory/export concern, not an implicit Git side effect.
+
+
+
Memory is compact structured JSON, not raw chat history. It distills what happened, what was learned, what failed, what worked, and what should be reused.
Lifecycle: `active → confirmed → superseded → archived`.
diff --git a/docs/TESTING-HANDOFF.md b/docs/TESTING-HANDOFF.md
index 70bedf0e..ce5986b7 100644
--- a/docs/TESTING-HANDOFF.md
+++ b/docs/TESTING-HANDOFF.md
@@ -4,7 +4,7 @@ Update this file at the end of every manual or automated test session. Keep it f
## Current status
-The current deployed baseline is **MSO CLI 1.12.0** with toolset `2026.09.02.7`; use `mso --version` plus `/api/health` for the exact Git build because documentation-only release commits also advance that identity. The v1.10/v1.11 findings and the v1.12 sectioned-terminal/recoverable-error work below are resolved and retained as historical test evidence. Do not re-open a resolved finding from prose alone: reproduce it against the current build first. There is no known 1.12 release blocker in this handoff.
+The current deployed baseline is **MSO CLI 1.12.0** with toolset `2026.09.03.1`; use `mso --version` plus `/api/health` for the exact Git build because documentation-only release commits also advance that identity. The v1.10/v1.11 findings and the v1.12 sectioned-terminal/recoverable-error work below are resolved and retained as historical test evidence. Do not re-open a resolved finding from prose alone: reproduce it against the current build first. There is no known 1.12 release blocker in this handoff.
Current verified behavior:
- every durable session has a unique short public `name` (`milo`, `luna`, `nara`, …) independent from its longer `title`; `/rename` changes the handle and `/title` changes the description;
@@ -14,6 +14,7 @@ Current verified behavior:
- `local_agent_request_wait` provides bounded foreground outcomes without resend/background polling;
- `local_agent_request` has MCP-dispatch coverage and remains a fresh bounded worker, never a claim to wake/control another terminal/ChatGPT process;
- standalone Local Agent MCP tests isolate the A2A worker import boundary instead of failing on Vitest's unresolved `server-only`;
+- `read_pipeline` batches up to six eligible read-only MCP calls and applies bounded declarative transforms server-side; child calls retain their normal guards/rate limits, are forced to read scope, and cannot carry a foreign `workflow_id`;
- exact write/exec approval defaults to one compact safe line; Enter opens redacted-safe details plus the unchanged canonical digest, and a separate explicit `allow` / `deny` decision is required.
- the terminal separates `Assistant`, `Agent work`, `Local agent`, `Error`, and `Input · @name` with full-width dividers; the composer identity is `@name ›` and permission lives in the bottom footer as `mode ask|auto|yolo`;
- recoverable HTTP/API failures preserve the interaction and correlation state, classify mutation outcome as `not_started`, `completed`, or `uncertain`, redact safe error summaries, and never auto-retry an uncertain write/exec mutation.
@@ -23,7 +24,7 @@ Current verified behavior:
```text
You are taking over MSO testing work in /home/rahman/projects/mso.
-Read docs/TESTING-HANDOFF.md first. Current verified baseline: CLI 1.12.0, toolset 2026.09.02.7; resolve the exact live Git build from `mso --version`/`/api/health`. Preserve existing work and do not re-open resolved v1.10-v1.12 findings unless you can reproduce them against the current build.
+Read docs/TESTING-HANDOFF.md first. Current verified baseline: CLI 1.12.0, toolset 2026.09.03.1; resolve the exact live Git build from `mso --version`/`/api/health`. Preserve existing work and do not re-open resolved v1.10-v1.12 findings unless you can reproduce them against the current build.
For a new issue: record the exact reproduction, distinguish durable state from live receiver/process state, avoid duplicate write/exec retries when outcome is uncertain, add the smallest focused regression, run typecheck plus the relevant contract tests, then update this handoff with factual results.
```
@@ -68,7 +69,7 @@ Validation minimum: focused unit tests for success, offline target, absent subsc
**Changed**
- Audited all 71 repository Markdown files; current references were synchronized while `PROGRESS`, generated changelog, dated audits/plans and older release evidence were preserved as historical records.
-- Corrected current MCP scope/count catalogs to 66 model/operator tools (32 read / 22 write / 12 exec) plus app-only `workflow_status`, and corrected current A2A docs to include authenticated outbound credentials, streaming and optional authenticated inbound serving.
+- At the 1.12 documentation-cleanup checkpoint, corrected MCP scope/count catalogs to 66 model/operator tools (32 read / 22 write / 12 exec) plus app-only `workflow_status`, and corrected current A2A docs to include authenticated outbound credentials, streaming and optional authenticated inbound serving.
- Updated 1.12 terminal/provider/install/Local Agent/architecture/troubleshooting guidance and replaced the stale takeover prompt with a current-baseline prompt while retaining the resolved old prompt as historical acceptance evidence.
- Strengthened `check-docs.mjs` so architecture/operator markers plus MCP, ChatGPT and connectors scope partitions must match source exactly.
- Removed dead `coreToolNames()` and made implementation-only Agent helpers private instead of exporting unused internal API.
diff --git a/lib/mcp/dispatch-tools.ts b/lib/mcp/dispatch-tools.ts
index 33be53e6..73d10bc5 100644
--- a/lib/mcp/dispatch-tools.ts
+++ b/lib/mcp/dispatch-tools.ts
@@ -1,4 +1,4 @@
-import { audit, rateLimited } from "@/lib/host";
+import { audit } from "@/lib/host";
import { maybeAutoTitleAgentSession } from "@/lib/agent/session-store";
import { touchLocalAgentPresence } from "@/lib/agent/local-agent-presence";
import { activeWorkflowForActor, recordWorkflowStep } from "@/lib/skills/memory";
@@ -8,6 +8,7 @@ import { TOOLS_BY_NAME } from "./tools";
import { rpcFail, rpcOk, type McpAgentContext, type RpcRequest } from "./dispatch-types";
import { recipeActor, workflowActor } from "./dispatch-actors";
import { flowFields, recordAgentEvent, sessionDetail, structuredResult, workflowFromResult } from "./dispatch-tool-support";
+import { toolRateLimited } from "./tool-rate-limit";
export async function dispatchToolCall(req: RpcRequest, scope: Scope, actor?: string, context?: McpAgentContext) {
const id = req.id ?? null;
@@ -55,18 +56,15 @@ export async function dispatchToolCall(req: RpcRequest, scope: Scope, actor?: st
return rpcOk(id, { content: [{ type: "text", text: `error: ${message}. Use this conversation's exact workflow_id.` }], isError: true });
}
- if (tool.limit) {
- const suffix = tool.limit.keyArg ? String(args[tool.limit.keyArg] ?? "") : (actor ?? "mcp");
- if (rateLimited(`${tool.limit.key}:${suffix}`, tool.limit.max, tool.limit.windowMs)) {
- void audit({ action: "mcp.denied", actor, target: name, ok: false, detail: "rate limited" });
- const activityId = newActivityId();
- if (!workflowProbe) {
- void recordMcpActivity({ id: activityId, actor, tool: name, state: "rate_limited", scope, ...initialFlow, target, detail: "rate limited" });
- await recordWorkflowStep(flowActor, activeWorkflow?.id, { id: activityId, tool: name, state: "rate_limited", target, args, ts: new Date().toISOString() });
- await recordAgentEvent(context, name, "rate_limited", args, activeWorkflow?.id, target);
- }
- return rpcOk(id, { content: [{ type: "text", text: `error: ${name} is rate limited.` }], isError: true });
+ if (toolRateLimited(tool, args, actor)) {
+ void audit({ action: "mcp.denied", actor, target: name, ok: false, detail: "rate limited" });
+ const activityId = newActivityId();
+ if (!workflowProbe) {
+ void recordMcpActivity({ id: activityId, actor, tool: name, state: "rate_limited", scope, ...initialFlow, target, detail: "rate limited" });
+ await recordWorkflowStep(flowActor, activeWorkflow?.id, { id: activityId, tool: name, state: "rate_limited", target, args, ts: new Date().toISOString() });
+ await recordAgentEvent(context, name, "rate_limited", args, activeWorkflow?.id, target);
}
+ return rpcOk(id, { content: [{ type: "text", text: `error: ${name} is rate limited.` }], isError: true });
}
const trail = tool.audit, auditTarget = trail?.targetArg != null ? String(args[trail.targetArg] ?? "") : undefined;
diff --git a/lib/mcp/parity.test.ts b/lib/mcp/parity.test.ts
index 077578c6..0f1cec63 100644
--- a/lib/mcp/parity.test.ts
+++ b/lib/mcp/parity.test.ts
@@ -83,6 +83,7 @@ const MCP_ONLY: Record<string, string> = {
"tool.forge.propose": "same Forge boundary; creating an inert candidate depends on learned external workflow recipes rather than Alfa owner recall",
"tool.forge.evaluate": "Forge evaluation may execute project-owned fixtures only inside the dedicated sandbox and therefore stays on the scoped external/terminal automation surface",
"tool.forge.promote": "explicit Forge promotion mutates project capability metadata and remains an exec-scoped audited external/terminal action",
+ "read.pipeline": "external/terminal agents can batch already-authorized read tools and compact their data server-side; Alfa already owns an in-process tool loop and does not need a second orchestration surface",
};
describe("Alfa ↔ MCP capability parity", () => {
@@ -178,6 +179,7 @@ describe("MCP rate limits mirror the routes", () => {
"tool.forge.propose": 20,
"tool.forge.evaluate": 10,
"tool.forge.promote": 6,
+ "read.pipeline": 30,
};
for (const t of TOOLS) {
if (!t.limit) continue;
diff --git a/lib/mcp/read-pipeline-engine.test.ts b/lib/mcp/read-pipeline-engine.test.ts
new file mode 100644
index 00000000..d7a3b38b
--- /dev/null
+++ b/lib/mcp/read-pipeline-engine.test.ts
@@ -0,0 +1,88 @@
+import { describe, expect, it } from "vitest";
+import { mcpDirect, type McpTool } from "./tool-kit";
+import { runReadPipeline } from "./read-pipeline-engine";
+
+const readTool = (name: string, run: McpTool["run"], limit?: McpTool["limit"]): McpTool => ({
+ name, description: name, scope: "read", annotations: { readOnlyHint: true, idempotentHint: true },
+ inputSchema: { type: "object", properties: {} }, ...(limit ? { limit } : {}), run,
+});
+const mutateTool = (name: string): McpTool => ({
+ name, description: name, scope: "write", inputSchema: { type: "object", properties: {} }, run: async () => ({ changed: true }),
+});
+const resolver = (tools: McpTool[]) => { const map = new Map(tools.map((tool) => [tool.name, tool])); return (name: string) => map.get(name); };
+
+const inventory = readTool("inventory", async () => ({ items: [
+ { name: "api", status: "up", cpu: 25, team: "a" },
+ { name: "web", status: "up", cpu: 8, team: "b" },
+ { name: "jobs", status: "down", cpu: 1, team: "a" },
+ { name: "worker", status: "up", cpu: 17, team: "a" },
+] }));
+
+describe("read_pipeline engine", () => {
+ it("filters, sorts, limits and projects raw results before returning them", async () => {
+ const out = await runReadPipeline({ calls: [{ id: "healthy", tool: "inventory", transform: {
+ path: "items", where: [{ field: "status", op: "eq", value: "up" }], sort: { field: "cpu", direction: "desc" },
+ limit: 2, select: ["name", "cpu"],
+ } }] }, { scope: "read", actor: "pipeline:test-1" }, resolver([inventory]));
+ expect(out.results.healthy).toEqual([{ name: "api", cpu: 25 }, { name: "worker", cpu: 17 }]);
+ expect(out.metrics.callCount).toBe(1);
+ expect(out.evidence[0]).toMatchObject({ id: "healthy", tool: "inventory", ok: true });
+ });
+
+ it("supports deterministic declarative aggregation", async () => {
+ const out = await runReadPipeline({ calls: [{ id: "cpu", tool: "inventory", transform: {
+ path: "items", where: [{ field: "team", op: "eq", value: "a" }], aggregate: { op: "avg", field: "cpu" },
+ } }] }, { scope: "read", actor: "pipeline:test-2" }, resolver([inventory]));
+ expect(out.results.cpu).toEqual({ op: "avg", field: "cpu", value: 43 / 3, count: 3 });
+ });
+
+ it("keeps declaration order even when parallel calls finish out of order", async () => {
+ const slow = readTool("slow", async () => { await new Promise((r) => setTimeout(r, 25)); return { value: "slow" }; });
+ const fast = readTool("fast", async () => ({ value: "fast" }));
+ const out = await runReadPipeline({ calls: [{ id: "first", tool: "slow" }, { id: "second", tool: "fast" }] },
+ { scope: "read", actor: "pipeline:test-3" }, resolver([slow, fast]));
+ expect(Object.keys(out.results)).toEqual(["first", "second"]);
+ expect(out.evidence.map((row) => row.id)).toEqual(["first", "second"]);
+ });
+
+ it("refuses write tools, nested pipelines, direct image results and child workflow ids", async () => {
+ const nested = readTool("read_pipeline", async () => ({ nope: true }));
+ const image = readTool("visual", async () => mcpDirect([{ type: "image", data: "AA==", mimeType: "image/png" }]));
+ await expect(runReadPipeline({ calls: [{ id: "bad", tool: "write", arguments: {} }] }, { scope: "exec", actor: "pipeline:test-4" }, resolver([mutateTool("write")]))).rejects.toThrow(/read-only/);
+ await expect(runReadPipeline({ calls: [{ id: "bad", tool: "read_pipeline" }] }, { scope: "read", actor: "pipeline:test-5" }, resolver([nested]))).rejects.toThrow(/read-only/);
+ await expect(runReadPipeline({ calls: [{ id: "bad", tool: "visual" }] }, { scope: "read", actor: "pipeline:test-6" }, resolver([image]))).rejects.toThrow(/direct image\/file/);
+ await expect(runReadPipeline({ calls: [{ id: "bad", tool: "inventory", arguments: { workflow_id: "other" } }] }, { scope: "read", actor: "pipeline:test-7" }, resolver([inventory]))).rejects.toThrow(/child workflow_id is forbidden/);
+ });
+
+ it("preserves the child tool's own rate limit", async () => {
+ const limited = readTool("limited", async () => ({ ok: true }), { key: "pipeline-test-limit", max: 1, windowMs: 60_000 });
+ const out = await runReadPipeline({ mode: "sequential", continueOnError: true, calls: [
+ { id: "one", tool: "limited" }, { id: "two", tool: "limited" },
+ ] }, { scope: "read", actor: "pipeline:unique-rate-limit" }, resolver([limited]));
+ expect(out.evidence.map((row) => row.ok)).toEqual([true, false]);
+ expect((out.results.two as { error: string }).error).toMatch(/rate limited/);
+ });
+
+ it("fails closed by default but can explicitly collect bounded read errors", async () => {
+ const broken = readTool("broken", async () => { throw new Error("read failed"); });
+ await expect(runReadPipeline({ calls: [{ id: "x", tool: "broken" }] }, { scope: "read", actor: "pipeline:test-8" }, resolver([broken]))).rejects.toThrow("read failed");
+ const out = await runReadPipeline({ continueOnError: true, calls: [{ id: "x", tool: "broken" }, { id: "y", tool: "inventory" }] },
+ { scope: "read", actor: "pipeline:test-9" }, resolver([broken, inventory]));
+ expect(out.ok).toBe(false); expect(out.evidence.map((row) => row.ok)).toEqual([false, true]);
+ });
+ it("rejects prototype traversal and never executes child reads above read scope", async () => {
+ let seenScope = "";
+ const scoped = readTool("scoped", async (_a, context) => { seenScope = context.scope; return { safe: { value: 1 } }; });
+ await runReadPipeline({ calls: [{ id: "safe", tool: "scoped" }] }, { scope: "exec", actor: "pipeline:test-scope" }, resolver([scoped]));
+ expect(seenScope).toBe("read");
+ await expect(runReadPipeline({ calls: [{ id: "bad", tool: "scoped", transform: { path: "constructor.name" } }] },
+ { scope: "read", actor: "pipeline:test-proto" }, resolver([scoped]))).rejects.toThrow(/forbidden field segment/);
+ });
+
+ it("bounds total response wall time even when an eligible read does not finish promptly", async () => {
+ const slow = readTool("very_slow", async () => { await new Promise((r) => setTimeout(r, 40)); return { late: true }; });
+ await expect(runReadPipeline({ mode: "sequential", calls: [{ id: "late", tool: "very_slow" }] },
+ { scope: "read", actor: "pipeline:test-deadline" }, resolver([slow]), 8)).rejects.toThrow(/wall-time budget/);
+ });
+
+});
diff --git a/lib/mcp/read-pipeline-engine.ts b/lib/mcp/read-pipeline-engine.ts
new file mode 100644
index 00000000..f2ec96f4
--- /dev/null
+++ b/lib/mcp/read-pipeline-engine.ts
@@ -0,0 +1,133 @@
+import type { McpRunContext, McpTool } from "./tool-kit";
+import { isMcpDirectResult } from "./tool-kit";
+import { toolRateLimited } from "./tool-rate-limit";
+import { transformReadResult, validPipelineId } from "./read-pipeline-transform";
+import {
+ READ_PIPELINE_MAX_CALLS,
+ READ_PIPELINE_OUTPUT_STEP_BYTES,
+ READ_PIPELINE_OUTPUT_TOTAL_BYTES,
+ READ_PIPELINE_RAW_STEP_BYTES,
+ READ_PIPELINE_WALL_MS,
+ type ReadPipelineCall,
+ type ReadPipelineEvidence,
+ type ReadPipelineInput,
+} from "./read-pipeline-types";
+
+const EXCLUDED = new Set(["read_pipeline", "workflow_status", "screen_capture", "local_agent_request_wait"]);
+
+function jsonBytes(value: unknown): number {
+ try { return Buffer.byteLength(JSON.stringify(value), "utf8"); }
+ catch { throw new Error("read tool returned a non-serializable result"); }
+}
+
+function boundedValue(value: unknown, maxBytes = READ_PIPELINE_OUTPUT_STEP_BYTES): unknown {
+ const bytes = jsonBytes(value); if (bytes <= maxBytes) return value;
+ if (Array.isArray(value)) {
+ let low = 0, high = value.length;
+ while (low < high) {
+ const mid = Math.ceil((low + high) / 2);
+ const candidate = { msoPipelineTruncated: true, totalItems: value.length, returnedItems: mid, items: value.slice(0, mid) };
+ if (jsonBytes(candidate) <= maxBytes) low = mid; else high = mid - 1;
+ }
+ return { msoPipelineTruncated: true, totalItems: value.length, returnedItems: low, items: value.slice(0, low) };
+ }
+ let raw: string; try { raw = JSON.stringify(value); } catch { raw = String(value); }
+ const preview = raw.slice(0, Math.max(512, Math.floor(maxBytes / 2)));
+ return { msoPipelineTruncated: true, originalBytes: bytes, preview };
+}
+
+function validateCalls(calls: ReadPipelineCall[]): void {
+ if (!Array.isArray(calls) || calls.length < 1 || calls.length > READ_PIPELINE_MAX_CALLS) throw new Error(`calls must contain 1-${READ_PIPELINE_MAX_CALLS} entries`);
+ const ids = new Set<string>();
+ for (const call of calls) {
+ if (!call || typeof call !== "object" || !validPipelineId(call.id)) throw new Error("each call.id must be a simple unique identifier");
+ if (ids.has(call.id)) throw new Error(`duplicate call.id: ${call.id}`); ids.add(call.id);
+ if (typeof call.tool !== "string" || !call.tool) throw new Error(`${call.id}: tool is required`);
+ if (call.arguments !== undefined && (!call.arguments || typeof call.arguments !== "object" || Array.isArray(call.arguments))) throw new Error(`${call.id}: arguments must be an object`);
+ if (Object.prototype.hasOwnProperty.call(call.arguments ?? {}, "workflow_id")) throw new Error(`${call.id}: child workflow_id is forbidden; the pipeline inherits the parent workflow`);
+ }
+}
+
+function readTool(tool: McpTool | undefined, name: string): McpTool {
+ if (!tool || tool.scope !== "read" || tool.annotations?.readOnlyHint !== true || EXCLUDED.has(name)) {
+ throw new Error(`${name}: read_pipeline can invoke only eligible read-only tools`);
+ }
+ return tool;
+}
+
+
+async function beforeDeadline<T>(promise: Promise<T>, deadlineAt: number): Promise<T> {
+ const remaining = deadlineAt - Date.now();
+ if (remaining <= 0) throw new Error(`read_pipeline exceeded its ${READ_PIPELINE_WALL_MS}ms wall-time budget`);
+ return new Promise<T>((resolve, reject) => {
+ const timer = setTimeout(() => reject(new Error(`read_pipeline exceeded its ${READ_PIPELINE_WALL_MS}ms wall-time budget`)), remaining);
+ promise.then(
+ (value) => { clearTimeout(timer); resolve(value); },
+ (error) => { clearTimeout(timer); reject(error); },
+ );
+ });
+}
+
+function validateRequired(tool: McpTool, args: Record<string, unknown>): void {
+ for (const key of tool.inputSchema.required ?? []) if (args[key] == null) throw new Error(`${tool.name} needs { ${(tool.inputSchema.required ?? []).join(", ")} }`);
+}
+
+export async function runReadPipeline(
+ input: ReadPipelineInput,
+ context: McpRunContext,
+ resolveTool: (name: string) => McpTool | undefined,
+ wallMs = READ_PIPELINE_WALL_MS,
+): Promise<{ ok: boolean; mode: string; results: Record<string, unknown>; evidence: ReadPipelineEvidence[]; metrics: Record<string, number> }> {
+ validateCalls(input.calls);
+ const started = Date.now(), deadlineAt = started + Math.max(1, Math.min(READ_PIPELINE_WALL_MS, wallMs));
+
+ const one = async (call: ReadPipelineCall): Promise<{
+ id: string; result: unknown; evidence: ReadPipelineEvidence; rawBytes: number; outputBytes: number;
+ }> => {
+ const stepStarted = Date.now(), args = { ...(call.arguments ?? {}) };
+ try {
+ const tool = readTool(resolveTool(call.tool), call.tool);
+ validateRequired(tool, args);
+ if (toolRateLimited(tool, args, context.actor)) throw new Error(`${tool.name} is rate limited`);
+ const raw = await beforeDeadline(tool.run(args, { ...context, scope: "read" }), deadlineAt);
+ if (isMcpDirectResult(raw)) throw new Error(`${tool.name}: direct image/file results are not pipeline-compatible`);
+ const rawBytes = jsonBytes(raw);
+ if (rawBytes > READ_PIPELINE_RAW_STEP_BYTES) throw new Error(`${tool.name}: raw result exceeds ${READ_PIPELINE_RAW_STEP_BYTES} bytes; narrow the read first`);
+ const transformed = transformReadResult(raw, call.transform), bounded = boundedValue(transformed);
+ const outputBytes = jsonBytes(bounded);
+ return {
+ id: call.id, result: bounded, rawBytes, outputBytes,
+ evidence: { id: call.id, tool: tool.name, ok: true, durationMs: Date.now() - stepStarted, rawBytes, outputBytes,
+ reductionPct: rawBytes ? Math.round((1 - outputBytes / rawBytes) * 1000) / 10 : 0 },
+ };
+ } catch (error) {
+ const message = error instanceof Error ? error.message : String(error);
+ if (!input.continueOnError) throw error;
+ return {
+ id: call.id, result: { error: message.slice(0, 300) }, rawBytes: 0, outputBytes: 0,
+ evidence: { id: call.id, tool: call.tool, ok: false, durationMs: Date.now() - stepStarted, error: message.slice(0, 300) },
+ };
+ }
+ };
+
+ const rows: Awaited<ReturnType<typeof one>>[] = [];
+ if (input.mode === "sequential") for (const call of input.calls) rows.push(await one(call));
+ else rows.push(...await Promise.all(input.calls.map(one)));
+
+ const results: Record<string, unknown> = {};
+ let rawBytesTotal = 0, outputBytesTotal = 0;
+ for (const row of rows) {
+ results[row.id] = row.result; rawBytesTotal += row.rawBytes; outputBytesTotal += row.outputBytes;
+ }
+ const compactResults = boundedValue(results, READ_PIPELINE_OUTPUT_TOTAL_BYTES) as Record<string, unknown>;
+ const finalOutputBytes = jsonBytes(compactResults), evidence = rows.map((row) => row.evidence);
+ return {
+ ok: evidence.every((row) => row.ok), mode: input.mode === "sequential" ? "sequential" : "parallel",
+ results: compactResults, evidence,
+ metrics: {
+ callCount: input.calls.length, rawBytes: rawBytesTotal, transformedBytes: outputBytesTotal, returnedBytes: finalOutputBytes,
+ reductionPct: rawBytesTotal ? Math.round((1 - finalOutputBytes / rawBytesTotal) * 1000) / 10 : 0,
+ elapsedMs: Date.now() - started,
+ },
+ };
+}
diff --git a/lib/mcp/read-pipeline-transform.ts b/lib/mcp/read-pipeline-transform.ts
new file mode 100644
index 00000000..123bcb23
--- /dev/null
+++ b/lib/mcp/read-pipeline-transform.ts
@@ -0,0 +1,128 @@
+import {
+ READ_PIPELINE_MAX_FILTERS,
+ READ_PIPELINE_MAX_LIMIT,
+ READ_PIPELINE_MAX_SELECT,
+ type ReadPipelineFilter,
+ type ReadPipelineTransform,
+} from "./read-pipeline-types";
+
+const PIPELINE_ID = /^[A-Za-z0-9_-]{1,64}$/;
+const FIELD_PATH = /^(?:[A-Za-z0-9_-]{1,64})(?:\.[A-Za-z0-9_-]{1,64}){0,7}$/;
+const DANGEROUS = new Set(["__proto__", "prototype", "constructor"]);
+
+function fieldPath(value: unknown, label: string): string {
+ if (typeof value !== "string" || !FIELD_PATH.test(value)) throw new Error(`${label} must be a simple dotted field path`);
+ if (value.split(".").some((segment) => DANGEROUS.has(segment))) throw new Error(`${label} contains a forbidden field segment`);
+ return value;
+}
+
+export function readPath(value: unknown, path?: string): unknown {
+ if (!path) return value;
+ fieldPath(path, "path");
+ let current = value;
+ for (const segment of path.split(".")) {
+ if (Array.isArray(current) && /^\d+$/.test(segment)) current = current[Number(segment)];
+ else if (current && typeof current === "object" && Object.prototype.hasOwnProperty.call(current, segment)) current = (current as Record<string, unknown>)[segment];
+ else return undefined;
+ }
+ return current;
+}
+
+function scalar(value: unknown): string | number | boolean | null | undefined {
+ return value === null || ["string", "number", "boolean"].includes(typeof value)
+ ? value as string | number | boolean | null
+ : undefined;
+}
+
+function compare(actual: unknown, filter: ReadPipelineFilter): boolean {
+ const found = readPath(actual, filter.field);
+ if (filter.op === "exists") return found !== undefined && found !== null;
+ const a = scalar(found), b = filter.value;
+ if (filter.op === "eq") return a === b;
+ if (filter.op === "ne") return a !== b;
+ if (filter.op === "contains") return typeof a === "string" && typeof b === "string" && a.toLowerCase().includes(b.toLowerCase());
+ if (typeof a !== "number" || typeof b !== "number") return false;
+ if (filter.op === "gt") return a > b;
+ if (filter.op === "gte") return a >= b;
+ if (filter.op === "lt") return a < b;
+ if (filter.op === "lte") return a <= b;
+ return false;
+}
+
+function validateTransform(transform: ReadPipelineTransform): void {
+ if (transform.path) fieldPath(transform.path, "transform.path");
+ if ((transform.where?.length ?? 0) > READ_PIPELINE_MAX_FILTERS) throw new Error(`where supports at most ${READ_PIPELINE_MAX_FILTERS} filters`);
+ for (const row of transform.where ?? []) {
+ fieldPath(row.field, "where.field");
+ if (!["eq", "ne", "contains", "gt", "gte", "lt", "lte", "exists"].includes(row.op)) throw new Error(`unsupported filter operator: ${row.op}`);
+ if (row.op !== "exists" && scalar(row.value) === undefined) throw new Error(`${row.op} requires a scalar value`);
+ if (typeof row.value === "string" && row.value.length > 2048) throw new Error("filter string values must be 2048 characters or fewer");
+ }
+ if ((transform.select?.length ?? 0) > READ_PIPELINE_MAX_SELECT) throw new Error(`select supports at most ${READ_PIPELINE_MAX_SELECT} fields`);
+ for (const field of transform.select ?? []) fieldPath(field, "select field");
+ if (transform.sort) fieldPath(transform.sort.field, "sort.field");
+ if (transform.uniqueBy) fieldPath(transform.uniqueBy, "uniqueBy");
+ if (transform.limit !== undefined && (!Number.isInteger(transform.limit) || transform.limit < 1 || transform.limit > READ_PIPELINE_MAX_LIMIT)) {
+ throw new Error(`limit must be 1-${READ_PIPELINE_MAX_LIMIT}`);
+ }
+ if (transform.aggregate) {
+ if (transform.select?.length) throw new Error("select and aggregate cannot be combined in one transform");
+ if (!["count", "sum", "avg", "min", "max"].includes(transform.aggregate.op)) throw new Error(`unsupported aggregate: ${transform.aggregate.op}`);
+ if (transform.aggregate.op !== "count" && !transform.aggregate.field) throw new Error(`${transform.aggregate.op} requires aggregate.field`);
+ if (transform.aggregate.field) fieldPath(transform.aggregate.field, "aggregate.field");
+ }
+}
+
+function selected(row: unknown, fields: string[]): Record<string, unknown> {
+ const out: Record<string, unknown> = {};
+ for (const field of fields) out[field] = readPath(row, field);
+ return out;
+}
+
+function aggregate(value: unknown, op: NonNullable<ReadPipelineTransform["aggregate"]>): Record<string, unknown> {
+ const rows = value == null ? [] : Array.isArray(value) ? value : [value];
+ if (op.op === "count") return { op: "count", value: rows.length };
+ const numbers = rows.map((row) => readPath(row, op.field!)).filter((v): v is number => typeof v === "number" && Number.isFinite(v));
+ if (!numbers.length) return { op: op.op, field: op.field, value: null, count: 0 };
+ const valueOut = op.op === "sum" ? numbers.reduce((a, b) => a + b, 0)
+ : op.op === "avg" ? numbers.reduce((a, b) => a + b, 0) / numbers.length
+ : op.op === "min" ? Math.min(...numbers) : Math.max(...numbers);
+ return { op: op.op, field: op.field, value: valueOut, count: numbers.length };
+}
+
+export function transformReadResult(input: unknown, transform?: ReadPipelineTransform): unknown {
+ if (!transform) return input;
+ validateTransform(transform);
+ let value = readPath(input, transform.path);
+ if (transform.where?.length) {
+ if (!Array.isArray(value)) throw new Error("where requires an array after path selection");
+ value = value.filter((row) => transform.where!.every((filter) => compare(row, filter)));
+ }
+ if (transform.sort) {
+ if (!Array.isArray(value)) throw new Error("sort requires an array");
+ const direction = transform.sort.direction === "desc" ? -1 : 1, field = transform.sort.field;
+ value = value.map((row, index) => ({ row, index })).sort((a, b) => {
+ const av = scalar(readPath(a.row, field)), bv = scalar(readPath(b.row, field));
+ if (av === bv) return a.index - b.index;
+ if (av == null) return 1; if (bv == null) return -1;
+ return (av < bv ? -1 : 1) * direction;
+ }).map(({ row }) => row);
+ }
+ if (transform.uniqueBy) {
+ if (!Array.isArray(value)) throw new Error("uniqueBy requires an array");
+ const seen = new Set<string>(), field = transform.uniqueBy;
+ value = value.filter((row) => { const key = JSON.stringify(readPath(row, field)); if (seen.has(key)) return false; seen.add(key); return true; });
+ }
+ if (transform.limit !== undefined) {
+ if (!Array.isArray(value)) throw new Error("limit requires an array");
+ value = value.slice(0, transform.limit);
+ }
+ if (transform.select?.length) {
+ if (Array.isArray(value)) value = value.map((row) => selected(row, transform.select!));
+ else value = selected(value, transform.select);
+ }
+ if (transform.aggregate) value = aggregate(value, transform.aggregate);
+ return value;
+}
+
+export function validPipelineId(value: string): boolean { return PIPELINE_ID.test(value); }
diff --git a/lib/mcp/read-pipeline-types.ts b/lib/mcp/read-pipeline-types.ts
new file mode 100644
index 00000000..b0984af7
--- /dev/null
+++ b/lib/mcp/read-pipeline-types.ts
@@ -0,0 +1,51 @@
+export const READ_PIPELINE_MAX_CALLS = 6;
+export const READ_PIPELINE_MAX_FILTERS = 4;
+export const READ_PIPELINE_MAX_SELECT = 12;
+export const READ_PIPELINE_MAX_LIMIT = 100;
+export const READ_PIPELINE_RAW_STEP_BYTES = 1024 * 1024;
+export const READ_PIPELINE_OUTPUT_STEP_BYTES = 12 * 1024;
+export const READ_PIPELINE_OUTPUT_TOTAL_BYTES = 40 * 1024;
+export const READ_PIPELINE_WALL_MS = 15_000;
+
+export type ReadPipelineOperator = "eq" | "ne" | "contains" | "gt" | "gte" | "lt" | "lte" | "exists";
+export type ReadPipelineAggregate = "count" | "sum" | "avg" | "min" | "max";
+
+export type ReadPipelineFilter = {
+ field: string;
+ op: ReadPipelineOperator;
+ value?: string | number | boolean | null;
+};
+
+export type ReadPipelineTransform = {
+ path?: string;
+ where?: ReadPipelineFilter[];
+ select?: string[];
+ sort?: { field: string; direction?: "asc" | "desc" };
+ uniqueBy?: string;
+ limit?: number;
+ aggregate?: { op: ReadPipelineAggregate; field?: string };
+};
+
+export type ReadPipelineCall = {
+ id: string;
+ tool: string;
+ arguments?: Record<string, unknown>;
+ transform?: ReadPipelineTransform;
+};
+
+export type ReadPipelineInput = {
+ calls: ReadPipelineCall[];
+ mode?: "parallel" | "sequential";
+ continueOnError?: boolean;
+};
+
+export type ReadPipelineEvidence = {
+ id: string;
+ tool: string;
+ ok: boolean;
+ durationMs: number;
+ rawBytes?: number;
+ outputBytes?: number;
+ reductionPct?: number;
+ error?: string;
+};
diff --git a/lib/mcp/tool-rate-limit.ts b/lib/mcp/tool-rate-limit.ts
new file mode 100644
index 00000000..a8fa80c1
--- /dev/null
+++ b/lib/mcp/tool-rate-limit.ts
@@ -0,0 +1,13 @@
+import { rateLimited } from "@/lib/host/rate-limit";
+import type { McpTool } from "./tool-kit";
+
+/** Shared per-tool bucket semantics for direct MCP calls and server-side read pipelines. */
+export function toolRateLimited(
+ tool: McpTool,
+ args: Record<string, unknown>,
+ actor?: string,
+): boolean {
+ if (!tool.limit) return false;
+ const suffix = tool.limit.keyArg ? String(args[tool.limit.keyArg] ?? "") : (actor ?? "mcp");
+ return rateLimited(`${tool.limit.key}:${suffix}`, tool.limit.max, tool.limit.windowMs);
+}
diff --git a/lib/mcp/tools-read-pipeline.test.ts b/lib/mcp/tools-read-pipeline.test.ts
new file mode 100644
index 00000000..8c7b7e33
--- /dev/null
+++ b/lib/mcp/tools-read-pipeline.test.ts
@@ -0,0 +1,13 @@
+import { describe, expect, it } from "vitest";
+import { READ_PIPELINE_TOOLS } from "./tools-read-pipeline";
+
+describe("read_pipeline MCP contract", () => {
+ it("is a single read-only, idempotent, bounded orchestration surface", () => {
+ expect(READ_PIPELINE_TOOLS).toHaveLength(1);
+ const tool = READ_PIPELINE_TOOLS[0]!;
+ expect(tool.name).toBe("read_pipeline"); expect(tool.scope).toBe("read");
+ expect(tool.annotations).toMatchObject({ readOnlyHint: true, idempotentHint: true });
+ expect(tool.limit).toMatchObject({ key: "read.pipeline", max: 30 });
+ expect(tool.inputSchema.required).toEqual(["calls"]);
+ });
+});
diff --git a/lib/mcp/tools-read-pipeline.ts b/lib/mcp/tools-read-pipeline.ts
new file mode 100644
index 00000000..6476ce20
--- /dev/null
+++ b/lib/mcp/tools-read-pipeline.ts
@@ -0,0 +1,48 @@
+import { runReadPipeline } from "./read-pipeline-engine";
+import type { ReadPipelineCall } from "./read-pipeline-types";
+import { type McpTool, S } from "./tool-kit";
+
+const TRANSFORM = {
+ type: "object",
+ properties: {
+ path: { type: "string", description: "Optional simple dotted path selected from the raw result before other operations." },
+ where: { type: "array", maxItems: 4, items: { type: "object", properties: {
+ field: { type: "string" }, op: { type: "string", enum: ["eq", "ne", "contains", "gt", "gte", "lt", "lte", "exists"] }, value: {},
+ }, required: ["field", "op"], additionalProperties: false } },
+ select: { type: "array", maxItems: 12, items: { type: "string" } },
+ sort: { type: "object", properties: { field: { type: "string" }, direction: { type: "string", enum: ["asc", "desc"] } }, required: ["field"], additionalProperties: false },
+ uniqueBy: { type: "string" }, limit: { type: "integer", minimum: 1, maximum: 100 },
+ aggregate: { type: "object", properties: { op: { type: "string", enum: ["count", "sum", "avg", "min", "max"] }, field: { type: "string" } }, required: ["op"], additionalProperties: false },
+ },
+ additionalProperties: false,
+} as const;
+
+export const READ_PIPELINE_TOOLS: McpTool[] = [{
+ name: "read_pipeline",
+ description:
+ "Execute 1-6 independent READ-ONLY MSO tool calls server-side, then deterministically filter/project/sort/unique/limit/aggregate their results before returning them to the model. " +
+ "Use this when several reads can be batched or raw lists/log metadata would waste context. It cannot call write/exec tools, workflow_status, screenshots, wait/poll tools, arbitrary code, shell, or nested pipelines. Child calls inherit this session/workflow and retain their own rate limits.",
+ scope: "read",
+ annotations: { readOnlyHint: true, idempotentHint: true },
+ limit: { key: "read.pipeline", max: 30, windowMs: 60_000 },
+ result: { maxTextBytes: 48 * 1024, overflowHint: "Pipeline output was compacted; add path/select/filter/aggregate or lower per-call limit." },
+ inputSchema: S({
+ calls: { type: "array", minItems: 1, maxItems: 6, description: "Independent read calls. Child workflow_id is forbidden; the parent workflow is inherited.", items: {
+ type: "object", properties: {
+ id: { type: "string", description: "Unique simple id used as the result key." },
+ tool: { type: "string", description: "Exact eligible read-only MSO tool name." },
+ arguments: { type: "object", additionalProperties: true }, transform: TRANSFORM,
+ }, required: ["id", "tool"], additionalProperties: false,
+ } },
+ mode: { type: "string", enum: ["parallel", "sequential"], description: "parallel by default; output order remains declaration order." },
+ continueOnError: { type: "boolean", description: "Default false. When true, failed child reads become bounded error rows while other calls continue." },
+ }, ["calls"]),
+ run: async (a, context) => {
+ const { TOOLS_BY_NAME } = await import("./tools");
+ return runReadPipeline({
+ calls: a.calls as ReadPipelineCall[],
+ mode: a.mode === "sequential" ? "sequential" : "parallel",
+ continueOnError: a.continueOnError === true,
+ }, context, (name) => TOOLS_BY_NAME.get(name));
+ },
+}];
diff --git a/lib/mcp/tools.ts b/lib/mcp/tools.ts
index e178071f..91ec987a 100644
--- a/lib/mcp/tools.ts
+++ b/lib/mcp/tools.ts
@@ -11,6 +11,7 @@ import { A2A_TOOLS } from "./tools-a2a";
import { LOCAL_AGENT_TOOLS } from "./tools-local-agents";
import { SUBAGENT_TOOLS } from "./tools-subagents";
import { FORGE_TOOLS } from "./tools-forge";
+import { READ_PIPELINE_TOOLS } from "./tools-read-pipeline";
import { withWorkflowContext } from "./tool-context";
// The write and exec tiers. Each carries an `audit` descriptor — the dispatcher,
@@ -220,4 +221,4 @@ const MUTATE_TOOLS: McpTool[] = [
},
];
-export const TOOLS: McpTool[] = [...READ_TOOLS, ...DISCOVERY_TOOLS, ...LEARNING_TOOLS, ...AGENT_TOOLS, ...LOCAL_AGENT_TOOLS, ...SUBAGENT_TOOLS, ...A2A_TOOLS, ...FORGE_TOOLS, ...INFRA_TOOLS, ...MUTATE_TOOLS, ...POWER_TOOLS].map(withWorkflowContext); export const TOOLS_BY_NAME = new Map(TOOLS.map((t) => [t.name, t]));
+export const TOOLS: McpTool[] = [...READ_TOOLS, ...DISCOVERY_TOOLS, ...LEARNING_TOOLS, ...AGENT_TOOLS, ...LOCAL_AGENT_TOOLS, ...SUBAGENT_TOOLS, ...A2A_TOOLS, ...FORGE_TOOLS, ...READ_PIPELINE_TOOLS, ...INFRA_TOOLS, ...MUTATE_TOOLS, ...POWER_TOOLS].map(withWorkflowContext); export const TOOLS_BY_NAME = new Map(TOOLS.map((t) => [t.name, t]));
diff --git a/lib/mcp/toolset.ts b/lib/mcp/toolset.ts
index f7d55447..66e07d09 100644
--- a/lib/mcp/toolset.ts
+++ b/lib/mcp/toolset.ts
@@ -3,8 +3,8 @@ import type { Scope } from "./scope";
import type { McpTool } from "./tool-kit";
export const MCP_SERVER_VERSION = "1.6.0";
-export const MCP_TOOLSET_VERSION = "2026.09.02.9";
-export const MCP_TOOLSET_CHANGED_AT = "2026-09-02T18:30:00Z";
+export const MCP_TOOLSET_VERSION = "2026.09.03.1";
+export const MCP_TOOLSET_CHANGED_AT = "2026-09-02T20:10:00Z";
export type McpToolsetInfo = {
serverVersion: string;
diff --git a/lib/orchestration/capability-catalog.mjs b/lib/orchestration/capability-catalog.mjs
index 71b71a3f..d329539c 100644
--- a/lib/orchestration/capability-catalog.mjs
+++ b/lib/orchestration/capability-catalog.mjs
@@ -21,6 +21,7 @@ const ROUTES = [
{ id: "subagent", priority: 100, patterns: [/\b(subagent|spawn .*reviewer|spawn .*worker|independent reviewer|isolated worker)\b/i], tools: ["agent_subagent_run"], budget: 12_000 },
{ id: "a2a", priority: 100, patterns: [/\b(a2a|agent card|remote agent|peer agent|handoff)\b/i], tools: ["a2a_agents_list", "a2a_agent_discover", "a2a_message_send", "a2a_handoff", "a2a_task_get"], budget: 12_000 },
{ id: "forge", priority: 100, patterns: [/\b(tool forge|forge candidate|promote .*tool|self[- ]improv|generate tool)\b/i], tools: ["tool_forge_candidates", "tool_forge_propose", "tool_forge_evaluate", "tool_forge_promote"], budget: 12_000 },
+ { id: "read-pipeline", priority: 98, patterns: [/\b(read pipeline|read_pipeline|batch|aggregate|aggregation|filter|projection|bulk read|multiple reads|orchestrat(?:e|ion))\b/i], tools: ["read_pipeline"], budget: 10_000 },
{ id: "session-resume", priority: 90, patterns: [/\b(resume|continue|list|previous)\b.*\b(session|conversation|thread)\b/i], tools: ["agent_sessions_list", "agent_session_resume"], budget: 12_000 },
{ id: "repo-change", priority: 55, patterns: [/\b(implement|refactor|fix|update|change|migrate|add|remove|cleanup|audit|terapkan|implementasi|perbaiki|ubah|tambahkan|hapus|rapikan)\b.*\b(repo|repository|project|codebase|runtime|architecture|feature|system|proyek|kode|arsitektur|fitur|sistem)\b/i], tools: ["workflow_start"], budget: 16_000 },
];
diff --git a/lib/orchestration/repo-memory-checkout.test.ts b/lib/orchestration/repo-memory-checkout.test.ts
new file mode 100644
index 00000000..6ba264b6
--- /dev/null
+++ b/lib/orchestration/repo-memory-checkout.test.ts
@@ -0,0 +1,10 @@
+import { describe, expect, it } from "vitest";
+import { readFile } from "node:fs/promises";
+import path from "node:path";
+
+describe("RASMIC checkout hygiene", () => {
+ it("keeps generated .agent runtime memory from dirtying the MSO source checkout", async () => {
+ const ignore = await readFile(path.join(process.cwd(), ".gitignore"), "utf8");
+ expect(ignore.split(/\r?\n/).map((row) => row.trim())).toContain(".agent/");
+ });
+});
diff --git a/package.json b/package.json
index f55a58ed..b63bf817 100644
--- a/package.json
+++ b/package.json
@@ -30,7 +30,8 @@
"bench:memory": "bun scripts/bench-memory-retrieval.mjs",
"bench:quality": "bun scripts/bench-task-quality.mjs",
"bench:agents": "bun scripts/bench-agent-comparison.mjs",
- "forge:sandbox": "bash scripts/provision-forge-sandbox.sh"
+ "forge:sandbox": "bash scripts/provision-forge-sandbox.sh",
+ "bench:pipeline": "bun scripts/bench-read-pipeline.mjs"
},
"dependencies": {
"@anthropic-ai/sdk": "^0.120.0",
diff --git a/scripts/bench-cognitive-runtime.mjs b/scripts/bench-cognitive-runtime.mjs
index 312c0aad..ef55e33f 100755
--- a/scripts/bench-cognitive-runtime.mjs
+++ b/scripts/bench-cognitive-runtime.mjs
@@ -3,6 +3,7 @@ import { execFileSync } from "node:child_process";
import { mock } from "bun:test";
import { selectToolsForTurn } from "./mso-agent-tool-router.mjs";
import { runMemoryRetrievalBenchmark } from "./bench-memory-retrieval.mjs";
+import { runReadPipelineBenchmark } from "./bench-read-pipeline.mjs";
mock.module("server-only", () => ({}));
const { TOOLS } = await import("../lib/mcp/tools.ts");
@@ -23,6 +24,7 @@ const scenarios = [
{ id: "memory-retrieval", prompt: "Search my persistent memory for a prior deployment rule and inspect conflicting or temporal claims.", required: ["agent_memory_search"] },
{ id: "resume", prompt: "List previous agent sessions and resume the deployment session.", required: ["agent_sessions_list", "agent_session_resume"] },
{ id: "skills", prompt: "Find the best trusted skill for a repository security review and read its instructions.", required: ["skills_search", "skills_read"] },
+ { id: "read-pipeline", prompt: "Batch several independent read-only checks, filter the large lists, and aggregate the results before returning them.", required: ["read_pipeline"] },
{ id: "tool-forge", prompt: "Turn this repeated verified workflow into a Tool Forge candidate, evaluate it in the sandbox, review the candidate, then explicitly promote it.", required: ["tool_forge_candidates", "tool_forge_propose", "tool_forge_evaluate", "tool_forge_promote"] },
{ id: "manual-regression-id", prompt: "Saya sudah tes manual dan setelah reconnect masih freeze; simpan hasil test gagal ini untuk debugging berikutnya.", required: ["project_memory_search", "project_memory_upsert"] },
{ id: "repo-change-id", prompt: "Tolong terapkan perubahan arsitektur ini di codebase dengan aman dan verifikasi semuanya.", required: ["workflow_start"] },
@@ -76,6 +78,7 @@ const catalogHitPct = scenarios.length ? (catalogHits / scenarios.length) * 100
const avgRoutingTextBytes = Math.round(routingTextBytes / scenarios.length);
const avgHistoryBudgetTokens = Math.round(historyBudgetTokens / scenarios.length);
const memoryBenchmark = runMemoryRetrievalBenchmark();
+const pipelineBenchmark = await runReadPipelineBenchmark();
const result = {
generatedAt: new Date().toISOString(),
providerNeutral: true,
@@ -94,6 +97,7 @@ const result = {
scenarios: rows,
phaseAwareRepoChange: { activeTools: repoPhase.activeCount, selectedNames: repoPhase.selectedNames, missing: repoPhaseMissing },
memory: memoryBenchmark,
+ readPipeline: pipelineBenchmark,
},
hermes: hermes ? {
version: version("hermes"), toolCount: hermes.tools?.count ?? null,
@@ -122,6 +126,9 @@ result.gates = {
memoryTemporal100: memoryBenchmark.temporal.accuracyPct === 100,
memoryConflict100: memoryBenchmark.conflict.accuracyPct === 100,
memoryDeterministic: memoryBenchmark.deterministic,
+ readPipelineCorrect: pipelineBenchmark.correctness,
+ readPipelineByteReduction80: pipelineBenchmark.savings.modelVisibleByteReductionPct >= 80,
+ readPipelineDeterministic: pipelineBenchmark.deterministic,
};
result.passed = Object.values(result.gates).every(Boolean);
@@ -133,6 +140,7 @@ else {
console.log(` routing ${result.mso.routingRecallPct}% required-tool recall · ${result.mso.catalogHitPct}% catalog hit · deterministic=${deterministic}`);
console.log(` routing context ${avgRoutingTextBytes.toLocaleString()} bytes avg · history budget ${avgHistoryBudgetTokens.toLocaleString()} tokens avg`);
console.log(` typed memory ${memoryBenchmark.overallAccuracyPct}% retrieval/temporal/conflict · deterministic=${memoryBenchmark.deterministic}`);
+ console.log(` read_pipeline ${pipelineBenchmark.savings.roundTripReductionPct}% round-trip reduction · ${pipelineBenchmark.savings.modelVisibleByteReductionPct}% byte reduction · exact=${pipelineBenchmark.correctness}`);
if (hermes?.tools) console.log(` Hermes baseline ${hermes.tools.count} tools · ${Number(hermes.tools.json_bytes).toLocaleString()} schema bytes`);
else console.log(" Hermes baseline unavailable");
console.log(` OpenClaw ${result.openclaw.version || "unavailable"} · no comparable offline prompt-size probe`);
diff --git a/scripts/bench-read-pipeline.mjs b/scripts/bench-read-pipeline.mjs
new file mode 100644
index 00000000..98b768a0
--- /dev/null
+++ b/scripts/bench-read-pipeline.mjs
@@ -0,0 +1,71 @@
+#!/usr/bin/env bun
+import { runReadPipeline } from "../lib/mcp/read-pipeline-engine.ts";
+
+const services = { rows: Array.from({ length: 500 }, (_, i) => ({
+ name: `service-${String(i).padStart(3, "0")}`, status: i % 9 === 0 ? "degraded" : "healthy",
+ cpu: (i * 17) % 100, memoryMb: 128 + (i % 24) * 64, detail: `bounded-service-metadata-${i}-`.repeat(4),
+})) };
+const projects = { rows: Array.from({ length: 400 }, (_, i) => ({
+ id: `project-${i}`, state: i % 5 === 0 ? "paused" : "active", owner: `team-${i % 12}`, branch: `feature/${i}`,
+ detail: `repository-metadata-${i}-`.repeat(3),
+})) };
+const sessions = { rows: Array.from({ length: 300 }, (_, i) => ({
+ title: `Session ${i}`, updated: 1_800_000_000_000 - i * 1000, source: i % 2 ? "mcp" : "cli", note: `session-summary-${i}-`.repeat(3),
+})) };
+const apps = { apps: Array.from({ length: 200 }, (_, i) => ({
+ id: `app-${i}`, running: i % 3 !== 0, version: `1.${i % 20}.${i % 7}`, statusDetail: `managed-app-status-${i}-`.repeat(4),
+})) };
+
+const fixtures = new Map([
+ ["fixture_services", services], ["fixture_projects", projects], ["fixture_sessions", sessions], ["fixture_apps", apps],
+]);
+const tools = new Map([...fixtures].map(([name, value]) => [name, {
+ name, description: name, scope: "read", annotations: { readOnlyHint: true, idempotentHint: true },
+ inputSchema: { type: "object", properties: {} }, run: async () => value,
+}]));
+const input = { calls: [
+ { id: "degraded", tool: "fixture_services", transform: { path: "rows", where: [{ field: "status", op: "eq", value: "degraded" }], sort: { field: "cpu", direction: "desc" }, limit: 10, select: ["name", "cpu", "memoryMb"] } },
+ { id: "activeProjects", tool: "fixture_projects", transform: { path: "rows", where: [{ field: "state", op: "eq", value: "active" }], aggregate: { op: "count" } } },
+ { id: "recentSessions", tool: "fixture_sessions", transform: { path: "rows", sort: { field: "updated", direction: "desc" }, limit: 5, select: ["title", "source", "updated"] } },
+ { id: "runningApps", tool: "fixture_apps", transform: { path: "apps", where: [{ field: "running", op: "eq", value: true }], limit: 10, select: ["id", "version"] } },
+] };
+
+const bytes = (value) => Buffer.byteLength(JSON.stringify(value), "utf8");
+const stable = (out) => ({
+ ok: out.ok, mode: out.mode, results: out.results,
+ evidence: out.evidence.map(({ id, tool, ok, rawBytes, outputBytes, reductionPct }) => ({ id, tool, ok, rawBytes, outputBytes, reductionPct })),
+});
+
+export async function runReadPipelineBenchmark() {
+ const baselineBytes = [...fixtures.values()].reduce((sum, value) => sum + bytes(value), 0);
+ const resolve = (name) => tools.get(name);
+ const a = await runReadPipeline(input, { scope: "read", actor: "bench:pipeline-a" }, resolve);
+ const b = await runReadPipeline(input, { scope: "read", actor: "bench:pipeline-b" }, resolve);
+ const returnedBytes = bytes(a), reductionPct = Math.round((1 - returnedBytes / baselineBytes) * 1000) / 10;
+ const expectedActiveProjects = projects.rows.filter((row) => row.state === "active").length;
+ const correctness = a.results.activeProjects?.value === expectedActiveProjects
+ && Array.isArray(a.results.degraded) && a.results.degraded.length === 10
+ && Array.isArray(a.results.recentSessions) && a.results.recentSessions.length === 5
+ && Array.isArray(a.results.runningApps) && a.results.runningApps.length === 10;
+ const deterministic = JSON.stringify(stable(a)) === JSON.stringify(stable(b));
+ return {
+ baseline: { toolRoundTrips: input.calls.length, modelVisibleBytes: baselineBytes },
+ pipeline: { toolRoundTrips: 1, modelVisibleBytes: returnedBytes, callsInsidePipeline: input.calls.length },
+ savings: { roundTripReductionPct: Math.round((1 - 1 / input.calls.length) * 1000) / 10, modelVisibleByteReductionPct: reductionPct },
+ correctness, deterministic,
+ passed: correctness && deterministic && reductionPct >= 80,
+ };
+}
+
+if (import.meta.main) {
+ const result = await runReadPipelineBenchmark();
+ if (process.argv.includes("--json")) console.log(JSON.stringify(result, null, 2));
+ else {
+ console.log("MSO read_pipeline benchmark");
+ console.log(` raw baseline ${result.baseline.toolRoundTrips} model round-trips · ${result.baseline.modelVisibleBytes.toLocaleString()} bytes`);
+ console.log(` read_pipeline ${result.pipeline.toolRoundTrips} model round-trip · ${result.pipeline.modelVisibleBytes.toLocaleString()} bytes`);
+ console.log(` savings ${result.savings.roundTripReductionPct}% round-trips · ${result.savings.modelVisibleByteReductionPct}% model-visible bytes`);
+ console.log(` correctness ${result.correctness ? "PASS" : "FAIL"} · deterministic=${result.deterministic}`);
+ }
+ if (!result.passed) process.exitCode = 1;
+}
diff --git a/scripts/bench-read-pipeline.test.ts b/scripts/bench-read-pipeline.test.ts
new file mode 100644
index 00000000..5e4d138c
--- /dev/null
+++ b/scripts/bench-read-pipeline.test.ts
@@ -0,0 +1,12 @@
+import { describe, expect, it } from "vitest";
+import { runReadPipelineBenchmark } from "./bench-read-pipeline.mjs";
+
+describe("read_pipeline benchmark", () => {
+ it("preserves exact fixture answers while reducing round-trips and model-visible bytes", async () => {
+ const result = await runReadPipelineBenchmark();
+ expect(result.correctness).toBe(true); expect(result.deterministic).toBe(true);
+ expect(result.savings.roundTripReductionPct).toBeGreaterThanOrEqual(75);
+ expect(result.savings.modelVisibleByteReductionPct).toBeGreaterThanOrEqual(80);
+ expect(result.passed).toBe(true);
+ });
+});
diff --git a/scripts/check-docs.mjs b/scripts/check-docs.mjs
index 92422b68..57e76358 100755
--- a/scripts/check-docs.mjs
+++ b/scripts/check-docs.mjs
@@ -14,22 +14,33 @@ const server = /MCP_SERVER_VERSION\s*=\s*"([^"]+)"/.exec(toolset)?.[1];
const version = /MCP_TOOLSET_VERSION\s*=\s*"([^"]+)"/.exec(toolset)?.[1];
if (!server || !version) fail.push("could not parse MCP version constants");
-const toolFiles = [
- "lib/mcp/tools.ts",
- "lib/mcp/tools-read.ts",
- "lib/mcp/tools-discovery.ts",
- "lib/mcp/tools-learning.ts",
- "lib/mcp/tools-workflow-start.ts",
- "lib/mcp/tools-workflow-lifecycle.ts",
- "lib/mcp/tools-project-memory.ts",
- "lib/mcp/tools-infra.ts",
- "lib/mcp/tools-power.ts",
- "lib/mcp/tools-agent.ts",
- "lib/mcp/tools-a2a.ts",
- "lib/mcp/tools-local-agents.ts",
- "lib/mcp/tools-subagents.ts",
- "lib/mcp/tools-forge.ts",
-];
+// Tool catalog modules may be split recursively (for example RASMIC workflow
+// modules). Start at tools.ts and follow every local tools-* import transitively.
+// Any top-level tools-*.ts module that is not reachable is dead/unregistered code
+// and fails CI instead of silently escaping the documentation contract.
+const toolModuleName = /^tools-[A-Za-z0-9-]+\.ts$/;
+const discoveredToolModules = readdirSync(join(ROOT, "lib/mcp"))
+ .filter((name) => toolModuleName.test(name))
+ .sort();
+const reachable = new Set();
+const pending = ["tools.ts"];
+while (pending.length) {
+ const name = pending.pop();
+ if (!name || reachable.has(name)) continue;
+ reachable.add(name);
+ const text = read(`lib/mcp/${name}`);
+ for (const match of text.matchAll(/from\s+["']\.\/(tools-[A-Za-z0-9-]+)["']/g)) {
+ const child = `${match[1]}.ts`;
+ if (!reachable.has(child)) pending.push(child);
+ }
+}
+for (const name of discoveredToolModules) {
+ if (!reachable.has(name)) fail.push(`lib/mcp/${name}: tool module exists but is not reachable from lib/mcp/tools.ts`);
+}
+for (const name of reachable) {
+ if (name !== "tools.ts" && !discoveredToolModules.includes(name)) fail.push(`lib/mcp/${name}: imported tool module is missing`);
+}
+const toolFiles = [...reachable].sort().map((name) => `lib/mcp/${name}`);
const tools = new Map();
for (const file of toolFiles) {
const text = read(file);
diff --git a/scripts/mso-agent-tool-router.test.ts b/scripts/mso-agent-tool-router.test.ts
index e7cbc35c..ce3db565 100644
--- a/scripts/mso-agent-tool-router.test.ts
+++ b/scripts/mso-agent-tool-router.test.ts
@@ -23,6 +23,7 @@ const catalog = [
tool("a2a_message_send", "Send a message to an A2A peer"), tool("a2a_handoff", "Delegate explicit objective to A2A peer"), tool("a2a_task_get", "Read A2A task status"),
tool("tool_forge_candidates", "List Tool Forge candidates"), tool("tool_forge_propose", "Propose a Tool Forge candidate from a repeated recipe"),
tool("tool_forge_evaluate", "Evaluate a Tool Forge candidate in the sandbox"), tool("tool_forge_promote", "Promote an evaluated Tool Forge candidate"),
+ tool("read_pipeline", "Batch multiple read-only tools and aggregate or filter results"),
];
describe("MSO per-turn tool router", () => {
@@ -126,4 +127,14 @@ describe("MSO per-turn tool router", () => {
expect(out.selectedNames).toContain("project_memory_search");
});
+ it("keeps read_pipeline out of ordinary reads and selects it through the RASMIC catalog for batch aggregation", () => {
+ const ordinary = selectToolsForTurn(catalog, [{ role: "user", text: "read the README file" }]);
+ expect(ordinary.selectedNames).not.toContain("read_pipeline");
+ const batched = selectToolsForTurn(catalog, [{ role: "user", text: "batch multiple reads and aggregate the results before returning them" }]);
+ expect(batched.routeIds).toContain("read-pipeline");
+ expect(batched.selectedNames).toContain("read_pipeline");
+ expect(batched.catalogMatched).toBe(true);
+ expect(batched.fallbackUsed).toBe(false);
+ });
+
});