Browser-based graphical shell and control plane for a Linux server you own — real terminal, file manager, live metrics and BYOK AI in one mobile-first workspace. Next.js 16 · React 19 · node-pty · Tailwind v4 · shadcn/ui. Public alpha, self-hosted.
TypeScript browser-shellcontrol-planefile-managerhomelabmobile-firstnextjsptyreactself-hostedserver-managementshadcn-uisysadmintailwindcssterminaltypescriptvpsweb-desktopweb-terminal
feat(mcp): add read-only result pipeline
Rahman835ec245fd41
diff --git a/.gitignore b/.gitignoreindex b42b9e57..99bc9f93 100644--- a/.gitignore+++ b/.gitignore@@ -22,6 +22,9 @@ os-browser/*!os-browser/package-lock.jsonconvex-backend/.mso/+# RASMIC repo-local orchestration memory/evidence is runtime state. It must survive+# locally but must not make the MSO source checkout dirty and block `mso update`.+.agent/# local session progress log (not tracked)progress.mddiff --git a/CLAUDE.md b/CLAUDE.mdindex 32773cb9..0e291cc5 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -349,8 +349,8 @@ the real runtime and working analogue before changing a tool or skill contract.- **Tool names are a cross-repo contract.** `rahmanef63/connectors-gateway` registeredmso as a connector on 2026-08-17 and historically pinned a subset of the MCP names asliteral `x-upstream` strings.- <!-- mcp-toolset: server=1.6.0 version=2026.09.02.9 tools=69 read=33 write=24 exec=12 -->- MSO currently exposes **70 transport tools**: **69 model/operator tools** (33 read, 24 write,+ <!-- mcp-toolset: server=1.6.0 version=2026.09.03.1 tools=70 read=34 write=24 exec=12 -->+ MSO currently exposes **71 transport tools**: **70 model/operator tools** (34 read, 24 write,12 exec) plus the app-only `workflow_status` bridge; `GET /mcp` plus `_meta.toolset` exposethe current version/hash/name manifest.Treat the gateway mapping as a separate cross-repo contract and verify it before renaming. Renaming or removingdiff --git a/README.md b/README.mdindex 61c0b2e4..ac075904 100644--- a/README.md+++ b/README.md@@ -79,7 +79,7 @@ For a real deployment, put MSO behind **Tailscale, a VPN, or a TLS reverse proxy- **Message live local sessions by a short name** — every durable same-principal MSO session gets a unique familiar handle such as `@milo`, `@luna`, or `@nara`, independent from its longer auto-generated session title. `/rename <name>` changes that handle, while `/title <text>` changes the session description. `@milo …` resolves only agents with a current lease **and** a live receiver subscription, then creates an explicitly correlated request; replies relay deterministically into the originating conversation while notify-only events remain passive. Presence lease (`idle`/`busy`/`offline`) and receiver subscription (`consumerConnected`) are reported separately so an idle-but-unsubscribed target is diagnosable. See [Local Agents](./docs/LOCAL-AGENTS.md).- **Spawn focused same-session subagents** — `/spawn` and `agent_subagent_run` create bounded foreground workers with isolated context, explicit delegated scope, turn/timeout caps, no recursive spawn, and final-result-only return. They are not Local Agent peers and do not run autonomously after the parent call returns. See [Subagents](./docs/SUBAGENTS.md).- **Delegate between remote agents with standard A2A v1** — register public HTTPS peers, attach private API-key/Bearer/OAuth access-token profiles, stream remote tasks over SSE, or expose MSO's authenticated inbound Agent Card when a public HTTPS origin is configured. Inbound credentials are owner-minted `read | write | exec` capabilities and run in a memory-isolated task context. See [A2A](./docs/A2A.md).-- **Keep agents efficient independently of the model provider** — the MCP-first [Cognitive Runtime](./docs/COGNITIVE-RUNTIME.md) isolates conversations from OAuth identity, budgets model context/tool output, selects a compact per-turn capability set without weakening scopes, compacts durable sessions with sanitized 30-day backup archives, and learns only verified workflow recipes. `bun run bench:cognitive` guards routing recall and context footprint.+- **Keep agents efficient independently of the model provider** — the MCP-first [Cognitive Runtime](./docs/COGNITIVE-RUNTIME.md) isolates conversations from OAuth identity, budgets model context/tool output, uses the RASMIC catalog-first router to select a tiny per-turn capability pack without weakening scopes, compacts durable sessions with sanitized 30-day backup archives, learns only verified workflow recipes, promotes repeated procedures only through the eval-gated Tool Forge, and batches/filter/aggregates independent reads through `read_pipeline` before raw data reaches the model. `bun run bench:cognitive` and `bun run bench:pipeline` guard routing, correctness and context savings.- **Make orchestration risk-aware and self-improving** — [RASMIC](./docs/RASMIC.md) classifies task risk/contention, retrieves compact repo-local memory, records manual user tests and Evidence Receipts, detects path/shared-resource collisions before merge conflicts, and promotes repeated safe routes from observed traces → recipes → tested bounded scripts.- **Add app slices** — features are modular under `frontend/slices/<slug>/`.- **Personalize the interface** — macOS, Windows, iOS, and Android shell layouts are UI preferences, not the core product.diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.mdindex a83e3d2f..b222ce03 100644--- a/docs/ARCHITECTURE.md+++ b/docs/ARCHITECTURE.md@@ -160,8 +160,8 @@ park a visible Approve/Deny card. The complete semantic contract is**MCP** is for external clients such as ChatGPT, Claude.ai and Cursor. Its catalog lives in`lib/mcp/`; access is controlled by the OAuth token scope rather than Alfa approval cards.-<!-- mcp-toolset: server=1.6.0 version=2026.09.02.9 tools=69 read=33 write=24 exec=12 -->-At the current toolset it exposes **70 transport tools**: **69 model/operator tools** (33 read, 24 write, 12 exec) plus the app-only `workflow_status` progress bridge. Project-specific+<!-- mcp-toolset: server=1.6.0 version=2026.09.03.1 tools=70 read=34 write=24 exec=12 -->+At the current toolset it exposes **71 transport tools**: **70 model/operator tools** (34 read, 24 write, 12 exec) plus the app-only `workflow_status` progress bridge. Project-specificfunction names remain data behind `project_capabilities` / `project_function_call`, so oneproject cannot dynamically rewrite the global MCP tool prefix. `GET /mcp` remains the live count/hash authority.diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.mdindex 2f8f58e4..3550d7d6 100644--- a/docs/CHANGELOG.md+++ b/docs/CHANGELOG.md@@ -8,6 +8,7 @@ this is the *what*, and it is what Settings → About shows as “What's new”.**Added**+- `mcp` add read-only result pipeline- add RASMIC orchestration memory automation- `agent` section terminal and recover API errors- `agent` name sessions and wrap composerdiff --git a/docs/CHATGPT-PLUGIN.md b/docs/CHATGPT-PLUGIN.mdindex 1c21b41e..cb30ea68 100644--- a/docs/CHATGPT-PLUGIN.md+++ b/docs/CHATGPT-PLUGIN.md@@ -12,10 +12,10 @@> <https://help.openai.com/en/articles/12584461-developer-mode-and-mcp-apps-in-chatgpt-beta>> and <https://help.openai.com/en/articles/20001256-plugins-in-chatgpt-and-codex>.-<!-- mcp-toolset: server=1.6.0 version=2026.09.02.9 tools=69 read=33 write=24 exec=12 -->+<!-- mcp-toolset: server=1.6.0 version=2026.09.03.1 tools=70 read=34 write=24 exec=12 -->-MSO currently exposes MCP server **1.6.0**, toolset **2026.09.02.9**: **70 transport tools** total; **69 model/operator tools**-(33 read, 24 write, 12 exec) plus app-only `workflow_status` for the progress widget. Use `GET /mcp` or Settings → MCP as the live authority if+MSO currently exposes MCP server **1.6.0**, toolset **2026.09.03.1**: **71 transport tools** total; **70 model/operator tools**+(34 read, 24 write, 12 exec) plus app-only `workflow_status` for the progress widget. Use `GET /mcp` or Settings → MCP as the live authority ifthis document and a deployed instance ever disagree.### Local-agent and subagent tools@@ -145,7 +145,7 @@ A token sees a scope prefix; there is no per-project or per-agent hidden tool fi`tools/list` filters the catalog and `tools/call` independently re-checks the requiredscope.-### `read` — 33 model/operator tools+### `read` — 34 model/operator tools- `a2a_agent_discover`- `a2a_agents_list`@@ -173,6 +173,7 @@ scope.- `project_capabilities`- `project_memory_search`- `projects_list`+- `read_pipeline`- `screen_capture`- `skills_list`- `skills_read`@@ -231,6 +232,12 @@ with the same care as a remote shell.+### Read pipeline for lower latency and context++Use `read_pipeline` when several independent reads can be batched or when a large read result should be filtered/projected/aggregated before ChatGPT sees it. The pipeline is **read scope only**: child calls reuse the original tool's host guards and rate limits, inherit the current session/workflow, and are forced to read scope even if the OAuth bearer is `exec`.++The transform language is deliberately data-only: simple dotted paths, scalar comparisons, field projection, sorting, de-duplication, limiting, and numeric/count aggregates. There is no shell, JavaScript, arbitrary expression, nested pipeline, child `workflow_id`, or write/exec capability. A total 15-second response deadline and bounded intermediate/final results keep the orchestration predictable. P3's deterministic fixture measured 4 → 1 model round-trip and 99.2% lower model-visible bytes with exact answers.+### Tool Forge approval model`tool_forge_propose` creates only inert candidates from repeated verified workflow recipes. Skill candidates are guidance only; a project-function candidate must point to an existing project-owned Node script and must come from an exec-scope verified recipe. Generated shell/code, credential-like fixture data, and implicit scope escalation are rejected.@@ -272,7 +279,7 @@ Settings → MCP:server: 1.6.0toolset: 2026.09.02.7hash: <live schema hash>-count: 69 model/operator tools (+ 1 app-only bridge)+count: 70 model/operator tools (+ 1 app-only bridge)```Use this sequence after an MSO MCP change:diff --git a/docs/COGNITIVE-RUNTIME.md b/docs/COGNITIVE-RUNTIME.mdindex 01313bf8..56c457e8 100644--- a/docs/COGNITIVE-RUNTIME.md+++ b/docs/COGNITIVE-RUNTIME.md@@ -131,7 +131,7 @@ probe. Overall product superiority also cannot be inferred from a prompt-size remust add equivalent task-success, tool-error, token-per-success, latency-per-success, and policy-compliancescenarios before making that claim.-## Current P2 + A2A baseline+## Historical P2 + A2A baselineOn the development host after P2 + outbound A2A integration:@@ -153,20 +153,40 @@ A non-interactive full MSO tool loop is now available as `mso agent --oneshot <pThese numbers are reproducible harness baselines, not an overall quality leaderboard. Cross-agent ranking is permitted only when at least two runners complete the same task with matching model-family evidence; one-task latency order is explicitly not an overall-agent claim.-## P2 status and next direction+## Current P3 + RASMIC baseline-P1 supplies typed/provenance-aware temporal memory, deterministic retrieval evaluation, workflow/tool quality telemetry, the non-interactive MSO Agent runner, and a fail-closed cross-agent scratch harness. P2 adds the first self-improvement primitive without granting opaque autonomy: an **eval-gated Tool Forge**.+P3 runs on the catalog-first RASMIC router rather than the older lexical active-tool selector. The merged deterministic harness now reports:-Forge only creates inert candidates from repeated high-quality recipes. Skill candidates are deterministic redacted guidance; executable project-function candidates require an exec-scope verified recipe and may reference only existing project-owned Node code. Fixtures run in the dedicated cached Docker sandbox with no network and a read-only project mount. Evaluation records current toolset, candidate/target state, source SHA-256 and exact sandbox image evidence; promotion requires literal confirmation and immediately repeats the checks. A candidate cannot silently raise scope, overwrite an existing capability, or turn generated text into host-executable code.+| Metric | MSO P3 + RASMIC | Hermes local baseline |+|---|---:|---:|+| Full MSO transport catalog | 71 tools | — |+| Full MSO schema bytes | 55,560 | — |+| Average MSO active tools / turn | 2.6 | — |+| Average active schema bytes / turn | 2,261 | 44,758 tool-schema bytes |+| MSO schema reduction vs full catalog | 95.9% | — |+| Required-tool routing recall / catalog hit | 100% / 100% | not the same benchmark |+| Average routing text / requested history budget | 77 bytes / 10,545 tokens | — |+| Typed-memory fixture accuracy | 100% | not implemented in this harness |+| P3 read fixture model round-trips | 1 vs 4 raw calls (75% lower) | not implemented in this harness |+| P3 read fixture model-visible bytes | 1,891 vs 232,205 raw bytes (99.2% lower) | not implemented in this harness |+| P3 read fixture correctness / determinism | exact / yes | not the same benchmark |++The Hermes comparison above is **tool-schema footprint only**. It does not establish overall product superiority, and OpenClaw still exposes no directly equivalent offline prompt-size probe in this harness. Overall ranking remains blocked until the same model/provider completes the same task-quality corpus on each agent.++## P3 status and next direction++P1 supplies typed/provenance-aware temporal memory and workflow quality telemetry. P2 adds the eval-gated Tool Forge. RASMIC makes intent routing catalog-first and risk/evidence aware. P3 adds `read_pipeline`: one read-scope MCP primitive that executes up to six independent eligible read calls server-side and reduces raw results through an intentionally non-programmable transform language.++Child reads retain their original MSO host/path guards and rate-limit buckets, inherit the parent session/workflow, are internally forced to read scope, and cannot inject a child workflow id. Nested pipelines, screenshots/direct-file results, wait/poll tools, prototype traversal, arbitrary code and every write/exec tool are rejected. Raw results, transformed results and total wall time are bounded. Parallel results are emitted in declaration order.-Outbound A2A v1 remains orthogonal: it adds remote-agent delegation, while Forge turns repeated **local verified procedures** into reviewed project capabilities. The combined routing benchmark covers these surfaces with 100% required-tool recall while selecting only 2.7 of 70 tools on average; it also requires the catalog-first routing and history-budget gates above.+The first P3 fixture proves the intended economics without weakening correctness: four independent model read calls become one model round-trip, while 232,205 raw model-visible bytes become 1,891 bytes after filtering/aggregation. Combined with RASMIC, the current router selects only 2.6 of 71 transport tools on average while keeping 100% required-tool recall in the deterministic corpus.-The next phase should continue from measured primitives:+The next phase should continue from measured primitives rather than add broad autonomy:-1. **Benchmark corpus expansion** — keep adding read/write/recovery/security task classes and configure Hermes/OpenClaw with an actually equivalent model/provider path before any overall comparison.-2. **Real token-per-success telemetry** — use provider-reported usage plus the non-prompt routing telemetry; missing provider usage remains unknown, never zero.-3. **Memory calibration** — grow post-P1 quality telemetry and validate deterministic relation/conflict policy against real corrections.-4. **Optional external memory transport** — keep repo-local `.agent/` canonical; any sync service consumes/produces the portable bundle and never becomes a runtime dependency.+1. **Comparable task-quality corpus** — add read/write/recovery/security scenarios and configure Hermes/OpenClaw with an equivalent model/provider path before any overall ranking.+2. **Real token/cost-per-success telemetry** — use provider-reported usage plus non-prompt routing/pipeline telemetry; missing usage remains unknown, never zero.+3. **Memory calibration** — grow post-P1 quality telemetry and validate confidence/authority/temporal policies against real user corrections.+4. **Pipeline composition only if earned by evals** — references/fan-out between child reads should not be added until a real corpus proves material gain over the intentionally simple independent-call contract.## Security notes@@ -175,6 +195,7 @@ The next phase should continue from measured primitives:- A discovered Skill or recipe cannot grant a tool the caller's token does not already permit.- Untrusted Skills keep their instructions withheld until promoted through the existing trust process.- Tool Forge candidates are inert. Executable fixtures require the labelled local Docker sandbox; evaluation never auto-pulls an image or executes generated shell/code.+- `read_pipeline` is an efficiency surface, not authority: only eligible read-only tools may run, child scope is forced to read, child rate limits/host guards stay active, and arbitrary program execution is impossible by contract.- `exec_run` remains full service-user shell power at `exec` scope; cognitive routing is not a sandbox.- Never expose raw ChatGPT conversation ids, tokens, credentials, hidden chain-of-thought, or unrestrictedtool output in memory/recipes/archives.diff --git a/docs/CONNECTORS-GATEWAY-INTEGRATION.md b/docs/CONNECTORS-GATEWAY-INTEGRATION.mdindex 6aace2a0..a8b5f459 100644--- a/docs/CONNECTORS-GATEWAY-INTEGRATION.md+++ b/docs/CONNECTORS-GATEWAY-INTEGRATION.md@@ -4,10 +4,10 @@> can evolve independently, so do not claim its mapping count is current without checking> that repository at the time of the change.-<!-- mcp-toolset: server=1.6.0 version=2026.09.02.9 tools=69 read=33 write=24 exec=12 -->+<!-- mcp-toolset: server=1.6.0 version=2026.09.03.1 tools=70 read=34 write=24 exec=12 -->-MSO currently exposes MCP server **1.6.0**, toolset **2026.09.02.9**: **70 transport tools** total; **69 model/operator tools**-(33 read, 24 write, 12 exec) plus app-only `workflow_status`. `GET /mcp` exposes the live names, version/hash and scoped+MSO currently exposes MCP server **1.6.0**, toolset **2026.09.03.1**: **71 transport tools** total; **70 model/operator tools**+(34 read, 24 write, 12 exec) plus app-only `workflow_status`. `GET /mcp` exposes the live names, version/hash and scopedmanifest and is the machine-readable parity source.## Local collaboration additions@@ -39,9 +39,9 @@ Before changing an MCP tool name:## Current MSO catalog-### Read (33 model/operator)+### Read (34 model/operator)-`a2a_agent_discover`, `a2a_agents_list`, `a2a_task_get`, `agent_memory_read`, `agent_memory_search`, `agent_session_current`, `agent_session_resume`, `agent_sessions_list`, `apps_list`, `apps_logs`, `browser_status`, `cloudflare_zones_list`, `dokploy_projects_list`, `exec_job_status`, `fs_list`, `fs_read`, `fs_search`, `fs_usage`, `infra_provider_doctor`, `infra_providers_list`, `local_agent_inbox`, `local_agent_request_wait`, `local_agents_list`, `project_capabilities`, `project_memory_search`, `projects_list`, `screen_capture`, `skills_list`, `skills_read`, `skills_search`, `sys_processes`, `sys_stats`, `tool_forge_candidates`.+`a2a_agent_discover`, `a2a_agents_list`, `a2a_task_get`, `agent_memory_read`, `agent_memory_search`, `agent_session_current`, `agent_session_resume`, `agent_sessions_list`, `apps_list`, `apps_logs`, `browser_status`, `cloudflare_zones_list`, `dokploy_projects_list`, `exec_job_status`, `fs_list`, `fs_read`, `fs_search`, `fs_usage`, `infra_provider_doctor`, `infra_providers_list`, `local_agent_inbox`, `local_agent_request_wait`, `local_agents_list`, `project_capabilities`, `project_memory_search`, `projects_list`, `read_pipeline`, `screen_capture`, `skills_list`, `skills_read`, `skills_search`, `sys_processes`, `sys_stats`, `tool_forge_candidates`.### Write (24 beyond read)@@ -73,6 +73,8 @@ MSO learned workflow. For multi-step orchestration, expose the workflow trio andthe exact id returned by `workflow_start`.+`read_pipeline` is one stable global read action rather than a gateway-generated action per child call. Gateways should forward its declarative JSON contract unchanged: the server permits only eligible read-only tools, inherits the caller session/workflow, keeps each child's normal rate limit and host guard, and returns bounded transformed evidence. It must never be translated into arbitrary script execution or used as a proxy for write/exec actions.+Tool Forge remains a stable global action family as well: gateways should expose the four generic `tool_forge_*` names, never synthesize one action per generated candidate. Candidate ids are data. Executable evaluation is server-side in the dedicated local Docker sandbox, and promotion remains exec-scope with exact confirmation.## Project functions are data, not dynamic MCP namesdiff --git a/docs/MCP.md b/docs/MCP.mdindex cec04502..c458736d 100644--- a/docs/MCP.md+++ b/docs/MCP.md@@ -75,7 +75,7 @@ Picked per token, on the consent screen, capped by `OS_MCP_MAX_SCOPE`. The highe| Scope | Tools ||---|---|-| `read` | `a2a_agent_discover` `a2a_agents_list` `a2a_task_get` `agent_memory_read` `agent_memory_search` `agent_session_current` `agent_session_resume` `agent_sessions_list` `apps_list` `apps_logs` `browser_status` `cloudflare_zones_list` `dokploy_projects_list` `exec_job_status` `fs_list` `fs_read` `fs_search` `fs_usage` `infra_provider_doctor` `infra_providers_list` `local_agent_inbox` `local_agent_request_wait` `local_agents_list` `project_capabilities` `project_memory_search` `projects_list` `screen_capture` `skills_list` `skills_read` `skills_search` `sys_processes` `sys_stats` `tool_forge_candidates` |+| `read` | `a2a_agent_discover` `a2a_agents_list` `a2a_task_get` `agent_memory_read` `agent_memory_search` `agent_session_current` `agent_session_resume` `agent_sessions_list` `apps_list` `apps_logs` `browser_status` `cloudflare_zones_list` `dokploy_projects_list` `exec_job_status` `fs_list` `fs_read` `fs_search` `fs_usage` `infra_provider_doctor` `infra_providers_list` `local_agent_inbox` `local_agent_request_wait` `local_agents_list` `project_capabilities` `project_memory_search` `projects_list` `read_pipeline` `screen_capture` `skills_list` `skills_read` `skills_search` `sys_processes` `sys_stats` `tool_forge_candidates` || `write` | + `a2a_agent_register` `a2a_agent_remove` `agent_memory_forget` `agent_memory_remember` `agent_session_note` `agent_session_rename` `apps_power` `cloudflare_dns_upsert` `dokploy_project_ensure` `fs_copy` `fs_delete` `fs_mkdir` `fs_move` `fs_upload_file` `fs_write` `hostinger_dns_upsert` `local_agent_message_send` `local_agent_reply` `project_memory_upsert` `project_script_run` `tool_forge_propose` `workflow_cancel` `workflow_finish` `workflow_start` || `exec` | + `a2a_handoff` `a2a_message_send` `a2a_task_cancel` `agent_subagent_run` `browser_power` `exec_job_cancel` `exec_job_start` `exec_run` `local_agent_request` `project_function_call` `tool_forge_evaluate` `tool_forge_promote` |@@ -126,9 +126,9 @@ The catalog has a stable server version plus a schema-derived toolset signature.Settings → MCP shows the current version/hash/count and stores a browser-local acknowledgement when the operator marks ChatGPT refreshed. A later signature change becomes an explicit stale-snapshot warning. This does not mutate ChatGPT remotely; it makes the required refresh visible instead of relying on memory.-<!-- mcp-toolset: server=1.6.0 version=2026.09.02.9 tools=69 read=33 write=24 exec=12 -->+<!-- mcp-toolset: server=1.6.0 version=2026.09.03.1 tools=70 read=34 write=24 exec=12 -->-Current transport catalog: **70 tools**, server `1.6.0` / toolset `2026.09.02.9`.+Current transport catalog: **71 tools**, server `1.6.0` / toolset `2026.09.03.1`.The model/operator catalog is **69 tools** (33 read, 24 write, 12 exec); `workflow_status` is theone app-only MCP Apps bridge used by the progress widget and is documented separately.`agent_memory_search` is the typed-memory retrieval surface. It resolves semantic/episodic/procedural claims at an optional point in time, returns confidence/provenance and competing effective claims, and can expose superseded/retracted history when explicitly requested. `agent_memory_remember` remains the write surface and now accepts typed metadata; raw ChatGPT conversation ids are never stored as provenance.@@ -158,6 +158,14 @@ message/context/task arguments; a handoff never auto-attaches the caller's hiddememory or raw session/workflow ids. See [`A2A.md`](./A2A.md) for the transport and trust contract.+### Programmatic read-only orchestration++`read_pipeline` is P3's provider-neutral batching seam. One read-scope call can execute **1–6 eligible read-only MSO tools** in parallel or sequentially, then apply bounded declarative transforms (`path`, scalar filters, `select`, `sort`, `uniqueBy`, `limit`, and `count`/`sum`/`avg`/`min`/`max`) before returning data to the model. It accepts no JavaScript, shell, arbitrary expressions, nested pipeline, child `workflow_id`, screenshot/direct-file result, wait/poll primitive, or write/exec tool.++Each child call still runs the original MSO tool handler and host/path guards, keeps that child's own rate-limit bucket, inherits the parent durable session/workflow, and is internally forced to `read` scope even when the parent bearer holds `exec`. Raw child results cap at 1 MiB, transformed output at 12 KiB per child and 40 KiB total, with a 15-second total response deadline. Parallel completion is reassembled in declaration order for deterministic caching and evaluation.++On the deterministic P3 fixture, four independent raw reads become **one model tool round-trip** (75% fewer), while model-visible bytes fall from **232,205 to 1,891 (99.2%)** with exact deterministic answers. This is a context/latency optimization, not a new permission or sandbox boundary.+### Eval-gated Tool ForgeTool Forge is a **candidate pipeline**, not a self-authorizing code generator. `tool_forge_propose` accepts only learned recipes with P1 quality telemetry, at least two verified successes, at least 90% workflow success, and a fully-completed best trace. `tool_forge_candidates` exposes reviewable metadata while redacting fixed argv, fixture payloads, and internal workflow targets. Candidates are inert until explicit evaluation and promotion.diff --git a/docs/PROGRESS.md b/docs/PROGRESS.mdindex 809f2c6a..2079a065 100644--- a/docs/PROGRESS.md+++ b/docs/PROGRESS.md@@ -2,11 +2,19 @@Running log of what shipped each phase. Newest at top.+## 2026-09-03 — Cognitive Runtime P3 read-only orchestration++- **Self-update invariant fixed:** RASMIC `.agent/` evidence/memory is runtime-local state and is now ignored by the MSO repo. This prevents successful workflow memory writes from making the canonical checkout dirty and causing `mso update` to refuse its own upgrade.+- **Programmatic reads without program execution:** added read-scope `read_pipeline`, batching 1–6 independent eligible read tools in parallel/sequential mode and reducing their results with bounded declarative path/filter/select/sort/unique/limit/count/sum/avg/min/max operations. No shell, JavaScript, nested pipeline, foreign child `workflow_id`, screenshot/direct-file result, wait/poll primitive, write tool or exec tool can enter the pipeline.+- **Guard reuse instead of a second security stack:** direct MCP dispatch and pipeline children now share one per-tool rate-limit helper. Child calls still execute the original tool handler and host/path checks and are internally forced to `read` scope even when the parent bearer is `exec`. Raw results cap at 1 MiB/child; transformed output caps at 12 KiB/child and 40 KiB total; the complete response has a 15-second wall-time deadline. Parallel results are emitted in declaration order for deterministic caching/evals.+- **Measured context reduction:** `bench:pipeline` preserves exact deterministic fixture answers while reducing four independent model round-trips to one (**75%**) and model-visible bytes from **232,205 to 1,891 (99.2%)**. Integrated Cognitive Runtime remains **100% required-tool routing recall**, **100% typed-memory fixture accuracy**, and averages **2.6/71 active tools** with **95.9% schema reduction** under the merged RASMIC catalog-first router.+- **Docs/tool catalog SSOT:** `check-docs.mjs` now derives tool modules from the canonical `lib/mcp/tools.ts` composition and fails on orphan `tools-*.ts` modules, so adding a new tool module cannot silently escape MCP/ChatGPT/connectors documentation. Toolset advances to `2026.09.03.1`: 71 transport tools = 70 model/operator tools (34 read / 24 write / 12 exec) + app-only `workflow_status`. The docs checker follows the recursive `tools-*` import closure from `lib/mcp/tools.ts`, so split RASMIC workflow modules remain in the same SSOT.+## 2026-09-03 — MSO 1.12 documentation and Agent contract cleanup- **Full Markdown audit:** reviewed all 71 authored Markdown files in the repository, preserving dated release/audit/design evidence while correcting current-reference drift. Structural validation reports zero malformed fence/heading issues and all 125 relative Markdown links resolve.- **Current Agent docs:** synchronized `TESTING-HANDOFF`, Local Agents, install/provider guidance, troubleshooting, architecture and the documentation map with the shipped 1.12 identity-first composer, sectioned transcript, compact exact approval and recoverable `not_started | completed | uncertain` mutation semantics. Historical 1.8–1.11 release notes remain intact rather than being rewritten as if they were current instructions.-- **MCP/A2A drift fixed:** current references now agree on server `1.6.0`, toolset `2026.09.02.7`, 67 transport tools = 66 model/operator tools (32 read / 22 write / 12 exec) + app-only `workflow_status`. Stale 31/29/59/56-tool claims and obsolete anonymous-only A2A wording were removed from current docs; historical benchmark snapshots were relabeled as snapshots instead of live metrics.+- **MCP/A2A drift fixed:** at that 1.12 documentation checkpoint, references agreed on server `1.6.0`, toolset `2026.09.02.7`, 67 transport tools = 66 model/operator tools (32 read / 22 write / 12 exec) + app-only `workflow_status`. Stale 31/29/59/56-tool claims and obsolete anonymous-only A2A wording were removed from current docs; historical benchmark snapshots were relabeled as snapshots instead of live metrics.- **Docs gate strengthened:** `check-docs.mjs` now verifies the MCP marker in architecture/operator docs and exact scope membership in `docs/MCP.md`, `docs/CHATGPT-PLUGIN.md`, and `docs/CONNECTORS-GATEWAY-INTEGRATION.md`, so a future tool addition cannot leave one human catalog silently stale.- **Unused surface cleanup:** removed the dead `coreToolNames()` helper and stopped exporting Agent constants/helpers that are private implementation details while retaining every internally used behavior. No compatibility placeholder or public CLI/MCP capability was removed.diff --git a/docs/RASMIC.md b/docs/RASMIC.mdindex be44f046..16fb2ef4 100644--- a/docs/RASMIC.md+++ b/docs/RASMIC.md@@ -62,6 +62,10 @@ A successful tool call is not proof that the user-facing result works. New HIGH-└── evidence/```+The `.agent/` tree is **runtime-local orchestration state**, not source code. MSO's own repo ignores it so evidence/memory can survive locally without making the canonical checkout dirty or blocking `mso update`. Projects that adopt RASMIC should likewise keep `.agent/` out of normal source commits; portability/sync is an explicit memory/export concern, not an implicit Git side effect.+++Memory is compact structured JSON, not raw chat history. It distills what happened, what was learned, what failed, what worked, and what should be reused.Lifecycle: `active → confirmed → superseded → archived`.diff --git a/docs/TESTING-HANDOFF.md b/docs/TESTING-HANDOFF.mdindex 70bedf0e..ce5986b7 100644--- a/docs/TESTING-HANDOFF.md+++ b/docs/TESTING-HANDOFF.md@@ -4,7 +4,7 @@ Update this file at the end of every manual or automated test session. Keep it f## Current status-The current deployed baseline is **MSO CLI 1.12.0** with toolset `2026.09.02.7`; use `mso --version` plus `/api/health` for the exact Git build because documentation-only release commits also advance that identity. The v1.10/v1.11 findings and the v1.12 sectioned-terminal/recoverable-error work below are resolved and retained as historical test evidence. Do not re-open a resolved finding from prose alone: reproduce it against the current build first. There is no known 1.12 release blocker in this handoff.+The current deployed baseline is **MSO CLI 1.12.0** with toolset `2026.09.03.1`; use `mso --version` plus `/api/health` for the exact Git build because documentation-only release commits also advance that identity. The v1.10/v1.11 findings and the v1.12 sectioned-terminal/recoverable-error work below are resolved and retained as historical test evidence. Do not re-open a resolved finding from prose alone: reproduce it against the current build first. There is no known 1.12 release blocker in this handoff.Current verified behavior:- every durable session has a unique short public `name` (`milo`, `luna`, `nara`, …) independent from its longer `title`; `/rename` changes the handle and `/title` changes the description;@@ -14,6 +14,7 @@ Current verified behavior:- `local_agent_request_wait` provides bounded foreground outcomes without resend/background polling;- `local_agent_request` has MCP-dispatch coverage and remains a fresh bounded worker, never a claim to wake/control another terminal/ChatGPT process;- standalone Local Agent MCP tests isolate the A2A worker import boundary instead of failing on Vitest's unresolved `server-only`;+- `read_pipeline` batches up to six eligible read-only MCP calls and applies bounded declarative transforms server-side; child calls retain their normal guards/rate limits, are forced to read scope, and cannot carry a foreign `workflow_id`;- exact write/exec approval defaults to one compact safe line; Enter opens redacted-safe details plus the unchanged canonical digest, and a separate explicit `allow` / `deny` decision is required.- the terminal separates `Assistant`, `Agent work`, `Local agent`, `Error`, and `Input · @name` with full-width dividers; the composer identity is `@name ›` and permission lives in the bottom footer as `mode ask|auto|yolo`;- recoverable HTTP/API failures preserve the interaction and correlation state, classify mutation outcome as `not_started`, `completed`, or `uncertain`, redact safe error summaries, and never auto-retry an uncertain write/exec mutation.@@ -23,7 +24,7 @@ Current verified behavior:```textYou are taking over MSO testing work in /home/rahman/projects/mso.-Read docs/TESTING-HANDOFF.md first. Current verified baseline: CLI 1.12.0, toolset 2026.09.02.7; resolve the exact live Git build from `mso --version`/`/api/health`. Preserve existing work and do not re-open resolved v1.10-v1.12 findings unless you can reproduce them against the current build.+Read docs/TESTING-HANDOFF.md first. Current verified baseline: CLI 1.12.0, toolset 2026.09.03.1; resolve the exact live Git build from `mso --version`/`/api/health`. Preserve existing work and do not re-open resolved v1.10-v1.12 findings unless you can reproduce them against the current build.For a new issue: record the exact reproduction, distinguish durable state from live receiver/process state, avoid duplicate write/exec retries when outcome is uncertain, add the smallest focused regression, run typecheck plus the relevant contract tests, then update this handoff with factual results.```@@ -68,7 +69,7 @@ Validation minimum: focused unit tests for success, offline target, absent subsc**Changed**- Audited all 71 repository Markdown files; current references were synchronized while `PROGRESS`, generated changelog, dated audits/plans and older release evidence were preserved as historical records.-- Corrected current MCP scope/count catalogs to 66 model/operator tools (32 read / 22 write / 12 exec) plus app-only `workflow_status`, and corrected current A2A docs to include authenticated outbound credentials, streaming and optional authenticated inbound serving.+- At the 1.12 documentation-cleanup checkpoint, corrected MCP scope/count catalogs to 66 model/operator tools (32 read / 22 write / 12 exec) plus app-only `workflow_status`, and corrected current A2A docs to include authenticated outbound credentials, streaming and optional authenticated inbound serving.- Updated 1.12 terminal/provider/install/Local Agent/architecture/troubleshooting guidance and replaced the stale takeover prompt with a current-baseline prompt while retaining the resolved old prompt as historical acceptance evidence.- Strengthened `check-docs.mjs` so architecture/operator markers plus MCP, ChatGPT and connectors scope partitions must match source exactly.- Removed dead `coreToolNames()` and made implementation-only Agent helpers private instead of exporting unused internal API.diff --git a/lib/mcp/dispatch-tools.ts b/lib/mcp/dispatch-tools.tsindex 33be53e6..73d10bc5 100644--- a/lib/mcp/dispatch-tools.ts+++ b/lib/mcp/dispatch-tools.ts@@ -1,4 +1,4 @@-import { audit, rateLimited } from "@/lib/host";+import { audit } from "@/lib/host";import { maybeAutoTitleAgentSession } from "@/lib/agent/session-store";import { touchLocalAgentPresence } from "@/lib/agent/local-agent-presence";import { activeWorkflowForActor, recordWorkflowStep } from "@/lib/skills/memory";@@ -8,6 +8,7 @@ import { TOOLS_BY_NAME } from "./tools";import { rpcFail, rpcOk, type McpAgentContext, type RpcRequest } from "./dispatch-types";import { recipeActor, workflowActor } from "./dispatch-actors";import { flowFields, recordAgentEvent, sessionDetail, structuredResult, workflowFromResult } from "./dispatch-tool-support";+import { toolRateLimited } from "./tool-rate-limit";export async function dispatchToolCall(req: RpcRequest, scope: Scope, actor?: string, context?: McpAgentContext) {const id = req.id ?? null;@@ -55,18 +56,15 @@ export async function dispatchToolCall(req: RpcRequest, scope: Scope, actor?: streturn rpcOk(id, { content: [{ type: "text", text: `error: ${message}. Use this conversation's exact workflow_id.` }], isError: true });}- if (tool.limit) {- const suffix = tool.limit.keyArg ? String(args[tool.limit.keyArg] ?? "") : (actor ?? "mcp");- if (rateLimited(`${tool.limit.key}:${suffix}`, tool.limit.max, tool.limit.windowMs)) {- void audit({ action: "mcp.denied", actor, target: name, ok: false, detail: "rate limited" });- const activityId = newActivityId();- if (!workflowProbe) {- void recordMcpActivity({ id: activityId, actor, tool: name, state: "rate_limited", scope, ...initialFlow, target, detail: "rate limited" });- await recordWorkflowStep(flowActor, activeWorkflow?.id, { id: activityId, tool: name, state: "rate_limited", target, args, ts: new Date().toISOString() });- await recordAgentEvent(context, name, "rate_limited", args, activeWorkflow?.id, target);- }- return rpcOk(id, { content: [{ type: "text", text: `error: ${name} is rate limited.` }], isError: true });+ if (toolRateLimited(tool, args, actor)) {+ void audit({ action: "mcp.denied", actor, target: name, ok: false, detail: "rate limited" });+ const activityId = newActivityId();+ if (!workflowProbe) {+ void recordMcpActivity({ id: activityId, actor, tool: name, state: "rate_limited", scope, ...initialFlow, target, detail: "rate limited" });+ await recordWorkflowStep(flowActor, activeWorkflow?.id, { id: activityId, tool: name, state: "rate_limited", target, args, ts: new Date().toISOString() });+ await recordAgentEvent(context, name, "rate_limited", args, activeWorkflow?.id, target);}+ return rpcOk(id, { content: [{ type: "text", text: `error: ${name} is rate limited.` }], isError: true });}const trail = tool.audit, auditTarget = trail?.targetArg != null ? String(args[trail.targetArg] ?? "") : undefined;diff --git a/lib/mcp/parity.test.ts b/lib/mcp/parity.test.tsindex 077578c6..0f1cec63 100644--- a/lib/mcp/parity.test.ts+++ b/lib/mcp/parity.test.ts@@ -83,6 +83,7 @@ const MCP_ONLY: Record<string, string> = {"tool.forge.propose": "same Forge boundary; creating an inert candidate depends on learned external workflow recipes rather than Alfa owner recall","tool.forge.evaluate": "Forge evaluation may execute project-owned fixtures only inside the dedicated sandbox and therefore stays on the scoped external/terminal automation surface","tool.forge.promote": "explicit Forge promotion mutates project capability metadata and remains an exec-scoped audited external/terminal action",+ "read.pipeline": "external/terminal agents can batch already-authorized read tools and compact their data server-side; Alfa already owns an in-process tool loop and does not need a second orchestration surface",};describe("Alfa ↔ MCP capability parity", () => {@@ -178,6 +179,7 @@ describe("MCP rate limits mirror the routes", () => {"tool.forge.propose": 20,"tool.forge.evaluate": 10,"tool.forge.promote": 6,+ "read.pipeline": 30,};for (const t of TOOLS) {if (!t.limit) continue;diff --git a/lib/mcp/read-pipeline-engine.test.ts b/lib/mcp/read-pipeline-engine.test.tsnew file mode 100644index 00000000..d7a3b38b--- /dev/null+++ b/lib/mcp/read-pipeline-engine.test.ts@@ -0,0 +1,88 @@+import { describe, expect, it } from "vitest";+import { mcpDirect, type McpTool } from "./tool-kit";+import { runReadPipeline } from "./read-pipeline-engine";++const readTool = (name: string, run: McpTool["run"], limit?: McpTool["limit"]): McpTool => ({+ name, description: name, scope: "read", annotations: { readOnlyHint: true, idempotentHint: true },+ inputSchema: { type: "object", properties: {} }, ...(limit ? { limit } : {}), run,+});+const mutateTool = (name: string): McpTool => ({+ name, description: name, scope: "write", inputSchema: { type: "object", properties: {} }, run: async () => ({ changed: true }),+});+const resolver = (tools: McpTool[]) => { const map = new Map(tools.map((tool) => [tool.name, tool])); return (name: string) => map.get(name); };++const inventory = readTool("inventory", async () => ({ items: [+ { name: "api", status: "up", cpu: 25, team: "a" },+ { name: "web", status: "up", cpu: 8, team: "b" },+ { name: "jobs", status: "down", cpu: 1, team: "a" },+ { name: "worker", status: "up", cpu: 17, team: "a" },+] }));++describe("read_pipeline engine", () => {+ it("filters, sorts, limits and projects raw results before returning them", async () => {+ const out = await runReadPipeline({ calls: [{ id: "healthy", tool: "inventory", transform: {+ path: "items", where: [{ field: "status", op: "eq", value: "up" }], sort: { field: "cpu", direction: "desc" },+ limit: 2, select: ["name", "cpu"],+ } }] }, { scope: "read", actor: "pipeline:test-1" }, resolver([inventory]));+ expect(out.results.healthy).toEqual([{ name: "api", cpu: 25 }, { name: "worker", cpu: 17 }]);+ expect(out.metrics.callCount).toBe(1);+ expect(out.evidence[0]).toMatchObject({ id: "healthy", tool: "inventory", ok: true });+ });++ it("supports deterministic declarative aggregation", async () => {+ const out = await runReadPipeline({ calls: [{ id: "cpu", tool: "inventory", transform: {+ path: "items", where: [{ field: "team", op: "eq", value: "a" }], aggregate: { op: "avg", field: "cpu" },+ } }] }, { scope: "read", actor: "pipeline:test-2" }, resolver([inventory]));+ expect(out.results.cpu).toEqual({ op: "avg", field: "cpu", value: 43 / 3, count: 3 });+ });++ it("keeps declaration order even when parallel calls finish out of order", async () => {+ const slow = readTool("slow", async () => { await new Promise((r) => setTimeout(r, 25)); return { value: "slow" }; });+ const fast = readTool("fast", async () => ({ value: "fast" }));+ const out = await runReadPipeline({ calls: [{ id: "first", tool: "slow" }, { id: "second", tool: "fast" }] },+ { scope: "read", actor: "pipeline:test-3" }, resolver([slow, fast]));+ expect(Object.keys(out.results)).toEqual(["first", "second"]);+ expect(out.evidence.map((row) => row.id)).toEqual(["first", "second"]);+ });++ it("refuses write tools, nested pipelines, direct image results and child workflow ids", async () => {+ const nested = readTool("read_pipeline", async () => ({ nope: true }));+ const image = readTool("visual", async () => mcpDirect([{ type: "image", data: "AA==", mimeType: "image/png" }]));+ await expect(runReadPipeline({ calls: [{ id: "bad", tool: "write", arguments: {} }] }, { scope: "exec", actor: "pipeline:test-4" }, resolver([mutateTool("write")]))).rejects.toThrow(/read-only/);+ await expect(runReadPipeline({ calls: [{ id: "bad", tool: "read_pipeline" }] }, { scope: "read", actor: "pipeline:test-5" }, resolver([nested]))).rejects.toThrow(/read-only/);+ await expect(runReadPipeline({ calls: [{ id: "bad", tool: "visual" }] }, { scope: "read", actor: "pipeline:test-6" }, resolver([image]))).rejects.toThrow(/direct image\/file/);+ await expect(runReadPipeline({ calls: [{ id: "bad", tool: "inventory", arguments: { workflow_id: "other" } }] }, { scope: "read", actor: "pipeline:test-7" }, resolver([inventory]))).rejects.toThrow(/child workflow_id is forbidden/);+ });++ it("preserves the child tool's own rate limit", async () => {+ const limited = readTool("limited", async () => ({ ok: true }), { key: "pipeline-test-limit", max: 1, windowMs: 60_000 });+ const out = await runReadPipeline({ mode: "sequential", continueOnError: true, calls: [+ { id: "one", tool: "limited" }, { id: "two", tool: "limited" },+ ] }, { scope: "read", actor: "pipeline:unique-rate-limit" }, resolver([limited]));+ expect(out.evidence.map((row) => row.ok)).toEqual([true, false]);+ expect((out.results.two as { error: string }).error).toMatch(/rate limited/);+ });++ it("fails closed by default but can explicitly collect bounded read errors", async () => {+ const broken = readTool("broken", async () => { throw new Error("read failed"); });+ await expect(runReadPipeline({ calls: [{ id: "x", tool: "broken" }] }, { scope: "read", actor: "pipeline:test-8" }, resolver([broken]))).rejects.toThrow("read failed");+ const out = await runReadPipeline({ continueOnError: true, calls: [{ id: "x", tool: "broken" }, { id: "y", tool: "inventory" }] },+ { scope: "read", actor: "pipeline:test-9" }, resolver([broken, inventory]));+ expect(out.ok).toBe(false); expect(out.evidence.map((row) => row.ok)).toEqual([false, true]);+ });+ it("rejects prototype traversal and never executes child reads above read scope", async () => {+ let seenScope = "";+ const scoped = readTool("scoped", async (_a, context) => { seenScope = context.scope; return { safe: { value: 1 } }; });+ await runReadPipeline({ calls: [{ id: "safe", tool: "scoped" }] }, { scope: "exec", actor: "pipeline:test-scope" }, resolver([scoped]));+ expect(seenScope).toBe("read");+ await expect(runReadPipeline({ calls: [{ id: "bad", tool: "scoped", transform: { path: "constructor.name" } }] },+ { scope: "read", actor: "pipeline:test-proto" }, resolver([scoped]))).rejects.toThrow(/forbidden field segment/);+ });++ it("bounds total response wall time even when an eligible read does not finish promptly", async () => {+ const slow = readTool("very_slow", async () => { await new Promise((r) => setTimeout(r, 40)); return { late: true }; });+ await expect(runReadPipeline({ mode: "sequential", calls: [{ id: "late", tool: "very_slow" }] },+ { scope: "read", actor: "pipeline:test-deadline" }, resolver([slow]), 8)).rejects.toThrow(/wall-time budget/);+ });++});diff --git a/lib/mcp/read-pipeline-engine.ts b/lib/mcp/read-pipeline-engine.tsnew file mode 100644index 00000000..f2ec96f4--- /dev/null+++ b/lib/mcp/read-pipeline-engine.ts@@ -0,0 +1,133 @@+import type { McpRunContext, McpTool } from "./tool-kit";+import { isMcpDirectResult } from "./tool-kit";+import { toolRateLimited } from "./tool-rate-limit";+import { transformReadResult, validPipelineId } from "./read-pipeline-transform";+import {+ READ_PIPELINE_MAX_CALLS,+ READ_PIPELINE_OUTPUT_STEP_BYTES,+ READ_PIPELINE_OUTPUT_TOTAL_BYTES,+ READ_PIPELINE_RAW_STEP_BYTES,+ READ_PIPELINE_WALL_MS,+ type ReadPipelineCall,+ type ReadPipelineEvidence,+ type ReadPipelineInput,+} from "./read-pipeline-types";++const EXCLUDED = new Set(["read_pipeline", "workflow_status", "screen_capture", "local_agent_request_wait"]);++function jsonBytes(value: unknown): number {+ try { return Buffer.byteLength(JSON.stringify(value), "utf8"); }+ catch { throw new Error("read tool returned a non-serializable result"); }+}++function boundedValue(value: unknown, maxBytes = READ_PIPELINE_OUTPUT_STEP_BYTES): unknown {+ const bytes = jsonBytes(value); if (bytes <= maxBytes) return value;+ if (Array.isArray(value)) {+ let low = 0, high = value.length;+ while (low < high) {+ const mid = Math.ceil((low + high) / 2);+ const candidate = { msoPipelineTruncated: true, totalItems: value.length, returnedItems: mid, items: value.slice(0, mid) };+ if (jsonBytes(candidate) <= maxBytes) low = mid; else high = mid - 1;+ }+ return { msoPipelineTruncated: true, totalItems: value.length, returnedItems: low, items: value.slice(0, low) };+ }+ let raw: string; try { raw = JSON.stringify(value); } catch { raw = String(value); }+ const preview = raw.slice(0, Math.max(512, Math.floor(maxBytes / 2)));+ return { msoPipelineTruncated: true, originalBytes: bytes, preview };+}++function validateCalls(calls: ReadPipelineCall[]): void {+ if (!Array.isArray(calls) || calls.length < 1 || calls.length > READ_PIPELINE_MAX_CALLS) throw new Error(`calls must contain 1-${READ_PIPELINE_MAX_CALLS} entries`);+ const ids = new Set<string>();+ for (const call of calls) {+ if (!call || typeof call !== "object" || !validPipelineId(call.id)) throw new Error("each call.id must be a simple unique identifier");+ if (ids.has(call.id)) throw new Error(`duplicate call.id: ${call.id}`); ids.add(call.id);+ if (typeof call.tool !== "string" || !call.tool) throw new Error(`${call.id}: tool is required`);+ if (call.arguments !== undefined && (!call.arguments || typeof call.arguments !== "object" || Array.isArray(call.arguments))) throw new Error(`${call.id}: arguments must be an object`);+ if (Object.prototype.hasOwnProperty.call(call.arguments ?? {}, "workflow_id")) throw new Error(`${call.id}: child workflow_id is forbidden; the pipeline inherits the parent workflow`);+ }+}++function readTool(tool: McpTool | undefined, name: string): McpTool {+ if (!tool || tool.scope !== "read" || tool.annotations?.readOnlyHint !== true || EXCLUDED.has(name)) {+ throw new Error(`${name}: read_pipeline can invoke only eligible read-only tools`);+ }+ return tool;+}+++async function beforeDeadline<T>(promise: Promise<T>, deadlineAt: number): Promise<T> {+ const remaining = deadlineAt - Date.now();+ if (remaining <= 0) throw new Error(`read_pipeline exceeded its ${READ_PIPELINE_WALL_MS}ms wall-time budget`);+ return new Promise<T>((resolve, reject) => {+ const timer = setTimeout(() => reject(new Error(`read_pipeline exceeded its ${READ_PIPELINE_WALL_MS}ms wall-time budget`)), remaining);+ promise.then(+ (value) => { clearTimeout(timer); resolve(value); },+ (error) => { clearTimeout(timer); reject(error); },+ );+ });+}++function validateRequired(tool: McpTool, args: Record<string, unknown>): void {+ for (const key of tool.inputSchema.required ?? []) if (args[key] == null) throw new Error(`${tool.name} needs { ${(tool.inputSchema.required ?? []).join(", ")} }`);+}++export async function runReadPipeline(+ input: ReadPipelineInput,+ context: McpRunContext,+ resolveTool: (name: string) => McpTool | undefined,+ wallMs = READ_PIPELINE_WALL_MS,+): Promise<{ ok: boolean; mode: string; results: Record<string, unknown>; evidence: ReadPipelineEvidence[]; metrics: Record<string, number> }> {+ validateCalls(input.calls);+ const started = Date.now(), deadlineAt = started + Math.max(1, Math.min(READ_PIPELINE_WALL_MS, wallMs));++ const one = async (call: ReadPipelineCall): Promise<{+ id: string; result: unknown; evidence: ReadPipelineEvidence; rawBytes: number; outputBytes: number;+ }> => {+ const stepStarted = Date.now(), args = { ...(call.arguments ?? {}) };+ try {+ const tool = readTool(resolveTool(call.tool), call.tool);+ validateRequired(tool, args);+ if (toolRateLimited(tool, args, context.actor)) throw new Error(`${tool.name} is rate limited`);+ const raw = await beforeDeadline(tool.run(args, { ...context, scope: "read" }), deadlineAt);+ if (isMcpDirectResult(raw)) throw new Error(`${tool.name}: direct image/file results are not pipeline-compatible`);+ const rawBytes = jsonBytes(raw);+ if (rawBytes > READ_PIPELINE_RAW_STEP_BYTES) throw new Error(`${tool.name}: raw result exceeds ${READ_PIPELINE_RAW_STEP_BYTES} bytes; narrow the read first`);+ const transformed = transformReadResult(raw, call.transform), bounded = boundedValue(transformed);+ const outputBytes = jsonBytes(bounded);+ return {+ id: call.id, result: bounded, rawBytes, outputBytes,+ evidence: { id: call.id, tool: tool.name, ok: true, durationMs: Date.now() - stepStarted, rawBytes, outputBytes,+ reductionPct: rawBytes ? Math.round((1 - outputBytes / rawBytes) * 1000) / 10 : 0 },+ };+ } catch (error) {+ const message = error instanceof Error ? error.message : String(error);+ if (!input.continueOnError) throw error;+ return {+ id: call.id, result: { error: message.slice(0, 300) }, rawBytes: 0, outputBytes: 0,+ evidence: { id: call.id, tool: call.tool, ok: false, durationMs: Date.now() - stepStarted, error: message.slice(0, 300) },+ };+ }+ };++ const rows: Awaited<ReturnType<typeof one>>[] = [];+ if (input.mode === "sequential") for (const call of input.calls) rows.push(await one(call));+ else rows.push(...await Promise.all(input.calls.map(one)));++ const results: Record<string, unknown> = {};+ let rawBytesTotal = 0, outputBytesTotal = 0;+ for (const row of rows) {+ results[row.id] = row.result; rawBytesTotal += row.rawBytes; outputBytesTotal += row.outputBytes;+ }+ const compactResults = boundedValue(results, READ_PIPELINE_OUTPUT_TOTAL_BYTES) as Record<string, unknown>;+ const finalOutputBytes = jsonBytes(compactResults), evidence = rows.map((row) => row.evidence);+ return {+ ok: evidence.every((row) => row.ok), mode: input.mode === "sequential" ? "sequential" : "parallel",+ results: compactResults, evidence,+ metrics: {+ callCount: input.calls.length, rawBytes: rawBytesTotal, transformedBytes: outputBytesTotal, returnedBytes: finalOutputBytes,+ reductionPct: rawBytesTotal ? Math.round((1 - finalOutputBytes / rawBytesTotal) * 1000) / 10 : 0,+ elapsedMs: Date.now() - started,+ },+ };+}diff --git a/lib/mcp/read-pipeline-transform.ts b/lib/mcp/read-pipeline-transform.tsnew file mode 100644index 00000000..123bcb23--- /dev/null+++ b/lib/mcp/read-pipeline-transform.ts@@ -0,0 +1,128 @@+import {+ READ_PIPELINE_MAX_FILTERS,+ READ_PIPELINE_MAX_LIMIT,+ READ_PIPELINE_MAX_SELECT,+ type ReadPipelineFilter,+ type ReadPipelineTransform,+} from "./read-pipeline-types";++const PIPELINE_ID = /^[A-Za-z0-9_-]{1,64}$/;+const FIELD_PATH = /^(?:[A-Za-z0-9_-]{1,64})(?:\.[A-Za-z0-9_-]{1,64}){0,7}$/;+const DANGEROUS = new Set(["__proto__", "prototype", "constructor"]);++function fieldPath(value: unknown, label: string): string {+ if (typeof value !== "string" || !FIELD_PATH.test(value)) throw new Error(`${label} must be a simple dotted field path`);+ if (value.split(".").some((segment) => DANGEROUS.has(segment))) throw new Error(`${label} contains a forbidden field segment`);+ return value;+}++export function readPath(value: unknown, path?: string): unknown {+ if (!path) return value;+ fieldPath(path, "path");+ let current = value;+ for (const segment of path.split(".")) {+ if (Array.isArray(current) && /^\d+$/.test(segment)) current = current[Number(segment)];+ else if (current && typeof current === "object" && Object.prototype.hasOwnProperty.call(current, segment)) current = (current as Record<string, unknown>)[segment];+ else return undefined;+ }+ return current;+}++function scalar(value: unknown): string | number | boolean | null | undefined {+ return value === null || ["string", "number", "boolean"].includes(typeof value)+ ? value as string | number | boolean | null+ : undefined;+}++function compare(actual: unknown, filter: ReadPipelineFilter): boolean {+ const found = readPath(actual, filter.field);+ if (filter.op === "exists") return found !== undefined && found !== null;+ const a = scalar(found), b = filter.value;+ if (filter.op === "eq") return a === b;+ if (filter.op === "ne") return a !== b;+ if (filter.op === "contains") return typeof a === "string" && typeof b === "string" && a.toLowerCase().includes(b.toLowerCase());+ if (typeof a !== "number" || typeof b !== "number") return false;+ if (filter.op === "gt") return a > b;+ if (filter.op === "gte") return a >= b;+ if (filter.op === "lt") return a < b;+ if (filter.op === "lte") return a <= b;+ return false;+}++function validateTransform(transform: ReadPipelineTransform): void {+ if (transform.path) fieldPath(transform.path, "transform.path");+ if ((transform.where?.length ?? 0) > READ_PIPELINE_MAX_FILTERS) throw new Error(`where supports at most ${READ_PIPELINE_MAX_FILTERS} filters`);+ for (const row of transform.where ?? []) {+ fieldPath(row.field, "where.field");+ if (!["eq", "ne", "contains", "gt", "gte", "lt", "lte", "exists"].includes(row.op)) throw new Error(`unsupported filter operator: ${row.op}`);+ if (row.op !== "exists" && scalar(row.value) === undefined) throw new Error(`${row.op} requires a scalar value`);+ if (typeof row.value === "string" && row.value.length > 2048) throw new Error("filter string values must be 2048 characters or fewer");+ }+ if ((transform.select?.length ?? 0) > READ_PIPELINE_MAX_SELECT) throw new Error(`select supports at most ${READ_PIPELINE_MAX_SELECT} fields`);+ for (const field of transform.select ?? []) fieldPath(field, "select field");+ if (transform.sort) fieldPath(transform.sort.field, "sort.field");+ if (transform.uniqueBy) fieldPath(transform.uniqueBy, "uniqueBy");+ if (transform.limit !== undefined && (!Number.isInteger(transform.limit) || transform.limit < 1 || transform.limit > READ_PIPELINE_MAX_LIMIT)) {+ throw new Error(`limit must be 1-${READ_PIPELINE_MAX_LIMIT}`);+ }+ if (transform.aggregate) {+ if (transform.select?.length) throw new Error("select and aggregate cannot be combined in one transform");+ if (!["count", "sum", "avg", "min", "max"].includes(transform.aggregate.op)) throw new Error(`unsupported aggregate: ${transform.aggregate.op}`);+ if (transform.aggregate.op !== "count" && !transform.aggregate.field) throw new Error(`${transform.aggregate.op} requires aggregate.field`);+ if (transform.aggregate.field) fieldPath(transform.aggregate.field, "aggregate.field");+ }+}++function selected(row: unknown, fields: string[]): Record<string, unknown> {+ const out: Record<string, unknown> = {};+ for (const field of fields) out[field] = readPath(row, field);+ return out;+}++function aggregate(value: unknown, op: NonNullable<ReadPipelineTransform["aggregate"]>): Record<string, unknown> {+ const rows = value == null ? [] : Array.isArray(value) ? value : [value];+ if (op.op === "count") return { op: "count", value: rows.length };+ const numbers = rows.map((row) => readPath(row, op.field!)).filter((v): v is number => typeof v === "number" && Number.isFinite(v));+ if (!numbers.length) return { op: op.op, field: op.field, value: null, count: 0 };+ const valueOut = op.op === "sum" ? numbers.reduce((a, b) => a + b, 0)+ : op.op === "avg" ? numbers.reduce((a, b) => a + b, 0) / numbers.length+ : op.op === "min" ? Math.min(...numbers) : Math.max(...numbers);+ return { op: op.op, field: op.field, value: valueOut, count: numbers.length };+}++export function transformReadResult(input: unknown, transform?: ReadPipelineTransform): unknown {+ if (!transform) return input;+ validateTransform(transform);+ let value = readPath(input, transform.path);+ if (transform.where?.length) {+ if (!Array.isArray(value)) throw new Error("where requires an array after path selection");+ value = value.filter((row) => transform.where!.every((filter) => compare(row, filter)));+ }+ if (transform.sort) {+ if (!Array.isArray(value)) throw new Error("sort requires an array");+ const direction = transform.sort.direction === "desc" ? -1 : 1, field = transform.sort.field;+ value = value.map((row, index) => ({ row, index })).sort((a, b) => {+ const av = scalar(readPath(a.row, field)), bv = scalar(readPath(b.row, field));+ if (av === bv) return a.index - b.index;+ if (av == null) return 1; if (bv == null) return -1;+ return (av < bv ? -1 : 1) * direction;+ }).map(({ row }) => row);+ }+ if (transform.uniqueBy) {+ if (!Array.isArray(value)) throw new Error("uniqueBy requires an array");+ const seen = new Set<string>(), field = transform.uniqueBy;+ value = value.filter((row) => { const key = JSON.stringify(readPath(row, field)); if (seen.has(key)) return false; seen.add(key); return true; });+ }+ if (transform.limit !== undefined) {+ if (!Array.isArray(value)) throw new Error("limit requires an array");+ value = value.slice(0, transform.limit);+ }+ if (transform.select?.length) {+ if (Array.isArray(value)) value = value.map((row) => selected(row, transform.select!));+ else value = selected(value, transform.select);+ }+ if (transform.aggregate) value = aggregate(value, transform.aggregate);+ return value;+}++export function validPipelineId(value: string): boolean { return PIPELINE_ID.test(value); }diff --git a/lib/mcp/read-pipeline-types.ts b/lib/mcp/read-pipeline-types.tsnew file mode 100644index 00000000..b0984af7--- /dev/null+++ b/lib/mcp/read-pipeline-types.ts@@ -0,0 +1,51 @@+export const READ_PIPELINE_MAX_CALLS = 6;+export const READ_PIPELINE_MAX_FILTERS = 4;+export const READ_PIPELINE_MAX_SELECT = 12;+export const READ_PIPELINE_MAX_LIMIT = 100;+export const READ_PIPELINE_RAW_STEP_BYTES = 1024 * 1024;+export const READ_PIPELINE_OUTPUT_STEP_BYTES = 12 * 1024;+export const READ_PIPELINE_OUTPUT_TOTAL_BYTES = 40 * 1024;+export const READ_PIPELINE_WALL_MS = 15_000;++export type ReadPipelineOperator = "eq" | "ne" | "contains" | "gt" | "gte" | "lt" | "lte" | "exists";+export type ReadPipelineAggregate = "count" | "sum" | "avg" | "min" | "max";++export type ReadPipelineFilter = {+ field: string;+ op: ReadPipelineOperator;+ value?: string | number | boolean | null;+};++export type ReadPipelineTransform = {+ path?: string;+ where?: ReadPipelineFilter[];+ select?: string[];+ sort?: { field: string; direction?: "asc" | "desc" };+ uniqueBy?: string;+ limit?: number;+ aggregate?: { op: ReadPipelineAggregate; field?: string };+};++export type ReadPipelineCall = {+ id: string;+ tool: string;+ arguments?: Record<string, unknown>;+ transform?: ReadPipelineTransform;+};++export type ReadPipelineInput = {+ calls: ReadPipelineCall[];+ mode?: "parallel" | "sequential";+ continueOnError?: boolean;+};++export type ReadPipelineEvidence = {+ id: string;+ tool: string;+ ok: boolean;+ durationMs: number;+ rawBytes?: number;+ outputBytes?: number;+ reductionPct?: number;+ error?: string;+};diff --git a/lib/mcp/tool-rate-limit.ts b/lib/mcp/tool-rate-limit.tsnew file mode 100644index 00000000..a8fa80c1--- /dev/null+++ b/lib/mcp/tool-rate-limit.ts@@ -0,0 +1,13 @@+import { rateLimited } from "@/lib/host/rate-limit";+import type { McpTool } from "./tool-kit";++/** Shared per-tool bucket semantics for direct MCP calls and server-side read pipelines. */+export function toolRateLimited(+ tool: McpTool,+ args: Record<string, unknown>,+ actor?: string,+): boolean {+ if (!tool.limit) return false;+ const suffix = tool.limit.keyArg ? String(args[tool.limit.keyArg] ?? "") : (actor ?? "mcp");+ return rateLimited(`${tool.limit.key}:${suffix}`, tool.limit.max, tool.limit.windowMs);+}diff --git a/lib/mcp/tools-read-pipeline.test.ts b/lib/mcp/tools-read-pipeline.test.tsnew file mode 100644index 00000000..8c7b7e33--- /dev/null+++ b/lib/mcp/tools-read-pipeline.test.ts@@ -0,0 +1,13 @@+import { describe, expect, it } from "vitest";+import { READ_PIPELINE_TOOLS } from "./tools-read-pipeline";++describe("read_pipeline MCP contract", () => {+ it("is a single read-only, idempotent, bounded orchestration surface", () => {+ expect(READ_PIPELINE_TOOLS).toHaveLength(1);+ const tool = READ_PIPELINE_TOOLS[0]!;+ expect(tool.name).toBe("read_pipeline"); expect(tool.scope).toBe("read");+ expect(tool.annotations).toMatchObject({ readOnlyHint: true, idempotentHint: true });+ expect(tool.limit).toMatchObject({ key: "read.pipeline", max: 30 });+ expect(tool.inputSchema.required).toEqual(["calls"]);+ });+});diff --git a/lib/mcp/tools-read-pipeline.ts b/lib/mcp/tools-read-pipeline.tsnew file mode 100644index 00000000..6476ce20--- /dev/null+++ b/lib/mcp/tools-read-pipeline.ts@@ -0,0 +1,48 @@+import { runReadPipeline } from "./read-pipeline-engine";+import type { ReadPipelineCall } from "./read-pipeline-types";+import { type McpTool, S } from "./tool-kit";++const TRANSFORM = {+ type: "object",+ properties: {+ path: { type: "string", description: "Optional simple dotted path selected from the raw result before other operations." },+ where: { type: "array", maxItems: 4, items: { type: "object", properties: {+ field: { type: "string" }, op: { type: "string", enum: ["eq", "ne", "contains", "gt", "gte", "lt", "lte", "exists"] }, value: {},+ }, required: ["field", "op"], additionalProperties: false } },+ select: { type: "array", maxItems: 12, items: { type: "string" } },+ sort: { type: "object", properties: { field: { type: "string" }, direction: { type: "string", enum: ["asc", "desc"] } }, required: ["field"], additionalProperties: false },+ uniqueBy: { type: "string" }, limit: { type: "integer", minimum: 1, maximum: 100 },+ aggregate: { type: "object", properties: { op: { type: "string", enum: ["count", "sum", "avg", "min", "max"] }, field: { type: "string" } }, required: ["op"], additionalProperties: false },+ },+ additionalProperties: false,+} as const;++export const READ_PIPELINE_TOOLS: McpTool[] = [{+ name: "read_pipeline",+ description:+ "Execute 1-6 independent READ-ONLY MSO tool calls server-side, then deterministically filter/project/sort/unique/limit/aggregate their results before returning them to the model. " ++ "Use this when several reads can be batched or raw lists/log metadata would waste context. It cannot call write/exec tools, workflow_status, screenshots, wait/poll tools, arbitrary code, shell, or nested pipelines. Child calls inherit this session/workflow and retain their own rate limits.",+ scope: "read",+ annotations: { readOnlyHint: true, idempotentHint: true },+ limit: { key: "read.pipeline", max: 30, windowMs: 60_000 },+ result: { maxTextBytes: 48 * 1024, overflowHint: "Pipeline output was compacted; add path/select/filter/aggregate or lower per-call limit." },+ inputSchema: S({+ calls: { type: "array", minItems: 1, maxItems: 6, description: "Independent read calls. Child workflow_id is forbidden; the parent workflow is inherited.", items: {+ type: "object", properties: {+ id: { type: "string", description: "Unique simple id used as the result key." },+ tool: { type: "string", description: "Exact eligible read-only MSO tool name." },+ arguments: { type: "object", additionalProperties: true }, transform: TRANSFORM,+ }, required: ["id", "tool"], additionalProperties: false,+ } },+ mode: { type: "string", enum: ["parallel", "sequential"], description: "parallel by default; output order remains declaration order." },+ continueOnError: { type: "boolean", description: "Default false. When true, failed child reads become bounded error rows while other calls continue." },+ }, ["calls"]),+ run: async (a, context) => {+ const { TOOLS_BY_NAME } = await import("./tools");+ return runReadPipeline({+ calls: a.calls as ReadPipelineCall[],+ mode: a.mode === "sequential" ? "sequential" : "parallel",+ continueOnError: a.continueOnError === true,+ }, context, (name) => TOOLS_BY_NAME.get(name));+ },+}];diff --git a/lib/mcp/tools.ts b/lib/mcp/tools.tsindex e178071f..91ec987a 100644--- a/lib/mcp/tools.ts+++ b/lib/mcp/tools.ts@@ -11,6 +11,7 @@ import { A2A_TOOLS } from "./tools-a2a";import { LOCAL_AGENT_TOOLS } from "./tools-local-agents";import { SUBAGENT_TOOLS } from "./tools-subagents";import { FORGE_TOOLS } from "./tools-forge";+import { READ_PIPELINE_TOOLS } from "./tools-read-pipeline";import { withWorkflowContext } from "./tool-context";// The write and exec tiers. Each carries an `audit` descriptor — the dispatcher,@@ -220,4 +221,4 @@ const MUTATE_TOOLS: McpTool[] = [},];-export const TOOLS: McpTool[] = [...READ_TOOLS, ...DISCOVERY_TOOLS, ...LEARNING_TOOLS, ...AGENT_TOOLS, ...LOCAL_AGENT_TOOLS, ...SUBAGENT_TOOLS, ...A2A_TOOLS, ...FORGE_TOOLS, ...INFRA_TOOLS, ...MUTATE_TOOLS, ...POWER_TOOLS].map(withWorkflowContext); export const TOOLS_BY_NAME = new Map(TOOLS.map((t) => [t.name, t]));+export const TOOLS: McpTool[] = [...READ_TOOLS, ...DISCOVERY_TOOLS, ...LEARNING_TOOLS, ...AGENT_TOOLS, ...LOCAL_AGENT_TOOLS, ...SUBAGENT_TOOLS, ...A2A_TOOLS, ...FORGE_TOOLS, ...READ_PIPELINE_TOOLS, ...INFRA_TOOLS, ...MUTATE_TOOLS, ...POWER_TOOLS].map(withWorkflowContext); export const TOOLS_BY_NAME = new Map(TOOLS.map((t) => [t.name, t]));diff --git a/lib/mcp/toolset.ts b/lib/mcp/toolset.tsindex f7d55447..66e07d09 100644--- a/lib/mcp/toolset.ts+++ b/lib/mcp/toolset.ts@@ -3,8 +3,8 @@ import type { Scope } from "./scope";import type { McpTool } from "./tool-kit";export const MCP_SERVER_VERSION = "1.6.0";-export const MCP_TOOLSET_VERSION = "2026.09.02.9";-export const MCP_TOOLSET_CHANGED_AT = "2026-09-02T18:30:00Z";+export const MCP_TOOLSET_VERSION = "2026.09.03.1";+export const MCP_TOOLSET_CHANGED_AT = "2026-09-02T20:10:00Z";export type McpToolsetInfo = {serverVersion: string;diff --git a/lib/orchestration/capability-catalog.mjs b/lib/orchestration/capability-catalog.mjsindex 71b71a3f..d329539c 100644--- a/lib/orchestration/capability-catalog.mjs+++ b/lib/orchestration/capability-catalog.mjs@@ -21,6 +21,7 @@ const ROUTES = [{ id: "subagent", priority: 100, patterns: [/\b(subagent|spawn .*reviewer|spawn .*worker|independent reviewer|isolated worker)\b/i], tools: ["agent_subagent_run"], budget: 12_000 },{ id: "a2a", priority: 100, patterns: [/\b(a2a|agent card|remote agent|peer agent|handoff)\b/i], tools: ["a2a_agents_list", "a2a_agent_discover", "a2a_message_send", "a2a_handoff", "a2a_task_get"], budget: 12_000 },{ id: "forge", priority: 100, patterns: [/\b(tool forge|forge candidate|promote .*tool|self[- ]improv|generate tool)\b/i], tools: ["tool_forge_candidates", "tool_forge_propose", "tool_forge_evaluate", "tool_forge_promote"], budget: 12_000 },+ { id: "read-pipeline", priority: 98, patterns: [/\b(read pipeline|read_pipeline|batch|aggregate|aggregation|filter|projection|bulk read|multiple reads|orchestrat(?:e|ion))\b/i], tools: ["read_pipeline"], budget: 10_000 },{ id: "session-resume", priority: 90, patterns: [/\b(resume|continue|list|previous)\b.*\b(session|conversation|thread)\b/i], tools: ["agent_sessions_list", "agent_session_resume"], budget: 12_000 },{ id: "repo-change", priority: 55, patterns: [/\b(implement|refactor|fix|update|change|migrate|add|remove|cleanup|audit|terapkan|implementasi|perbaiki|ubah|tambahkan|hapus|rapikan)\b.*\b(repo|repository|project|codebase|runtime|architecture|feature|system|proyek|kode|arsitektur|fitur|sistem)\b/i], tools: ["workflow_start"], budget: 16_000 },];diff --git a/lib/orchestration/repo-memory-checkout.test.ts b/lib/orchestration/repo-memory-checkout.test.tsnew file mode 100644index 00000000..6ba264b6--- /dev/null+++ b/lib/orchestration/repo-memory-checkout.test.ts@@ -0,0 +1,10 @@+import { describe, expect, it } from "vitest";+import { readFile } from "node:fs/promises";+import path from "node:path";++describe("RASMIC checkout hygiene", () => {+ it("keeps generated .agent runtime memory from dirtying the MSO source checkout", async () => {+ const ignore = await readFile(path.join(process.cwd(), ".gitignore"), "utf8");+ expect(ignore.split(/\r?\n/).map((row) => row.trim())).toContain(".agent/");+ });+});diff --git a/package.json b/package.jsonindex f55a58ed..b63bf817 100644--- a/package.json+++ b/package.json@@ -30,7 +30,8 @@"bench:memory": "bun scripts/bench-memory-retrieval.mjs","bench:quality": "bun scripts/bench-task-quality.mjs","bench:agents": "bun scripts/bench-agent-comparison.mjs",- "forge:sandbox": "bash scripts/provision-forge-sandbox.sh"+ "forge:sandbox": "bash scripts/provision-forge-sandbox.sh",+ "bench:pipeline": "bun scripts/bench-read-pipeline.mjs"},"dependencies": {"@anthropic-ai/sdk": "^0.120.0",diff --git a/scripts/bench-cognitive-runtime.mjs b/scripts/bench-cognitive-runtime.mjsindex 312c0aad..ef55e33f 100755--- a/scripts/bench-cognitive-runtime.mjs+++ b/scripts/bench-cognitive-runtime.mjs@@ -3,6 +3,7 @@ import { execFileSync } from "node:child_process";import { mock } from "bun:test";import { selectToolsForTurn } from "./mso-agent-tool-router.mjs";import { runMemoryRetrievalBenchmark } from "./bench-memory-retrieval.mjs";+import { runReadPipelineBenchmark } from "./bench-read-pipeline.mjs";mock.module("server-only", () => ({}));const { TOOLS } = await import("../lib/mcp/tools.ts");@@ -23,6 +24,7 @@ const scenarios = [{ id: "memory-retrieval", prompt: "Search my persistent memory for a prior deployment rule and inspect conflicting or temporal claims.", required: ["agent_memory_search"] },{ id: "resume", prompt: "List previous agent sessions and resume the deployment session.", required: ["agent_sessions_list", "agent_session_resume"] },{ id: "skills", prompt: "Find the best trusted skill for a repository security review and read its instructions.", required: ["skills_search", "skills_read"] },+ { id: "read-pipeline", prompt: "Batch several independent read-only checks, filter the large lists, and aggregate the results before returning them.", required: ["read_pipeline"] },{ id: "tool-forge", prompt: "Turn this repeated verified workflow into a Tool Forge candidate, evaluate it in the sandbox, review the candidate, then explicitly promote it.", required: ["tool_forge_candidates", "tool_forge_propose", "tool_forge_evaluate", "tool_forge_promote"] },{ id: "manual-regression-id", prompt: "Saya sudah tes manual dan setelah reconnect masih freeze; simpan hasil test gagal ini untuk debugging berikutnya.", required: ["project_memory_search", "project_memory_upsert"] },{ id: "repo-change-id", prompt: "Tolong terapkan perubahan arsitektur ini di codebase dengan aman dan verifikasi semuanya.", required: ["workflow_start"] },@@ -76,6 +78,7 @@ const catalogHitPct = scenarios.length ? (catalogHits / scenarios.length) * 100const avgRoutingTextBytes = Math.round(routingTextBytes / scenarios.length);const avgHistoryBudgetTokens = Math.round(historyBudgetTokens / scenarios.length);const memoryBenchmark = runMemoryRetrievalBenchmark();+const pipelineBenchmark = await runReadPipelineBenchmark();const result = {generatedAt: new Date().toISOString(),providerNeutral: true,@@ -94,6 +97,7 @@ const result = {scenarios: rows,phaseAwareRepoChange: { activeTools: repoPhase.activeCount, selectedNames: repoPhase.selectedNames, missing: repoPhaseMissing },memory: memoryBenchmark,+ readPipeline: pipelineBenchmark,},hermes: hermes ? {version: version("hermes"), toolCount: hermes.tools?.count ?? null,@@ -122,6 +126,9 @@ result.gates = {memoryTemporal100: memoryBenchmark.temporal.accuracyPct === 100,memoryConflict100: memoryBenchmark.conflict.accuracyPct === 100,memoryDeterministic: memoryBenchmark.deterministic,+ readPipelineCorrect: pipelineBenchmark.correctness,+ readPipelineByteReduction80: pipelineBenchmark.savings.modelVisibleByteReductionPct >= 80,+ readPipelineDeterministic: pipelineBenchmark.deterministic,};result.passed = Object.values(result.gates).every(Boolean);@@ -133,6 +140,7 @@ else {console.log(` routing ${result.mso.routingRecallPct}% required-tool recall · ${result.mso.catalogHitPct}% catalog hit · deterministic=${deterministic}`);console.log(` routing context ${avgRoutingTextBytes.toLocaleString()} bytes avg · history budget ${avgHistoryBudgetTokens.toLocaleString()} tokens avg`);console.log(` typed memory ${memoryBenchmark.overallAccuracyPct}% retrieval/temporal/conflict · deterministic=${memoryBenchmark.deterministic}`);+ console.log(` read_pipeline ${pipelineBenchmark.savings.roundTripReductionPct}% round-trip reduction · ${pipelineBenchmark.savings.modelVisibleByteReductionPct}% byte reduction · exact=${pipelineBenchmark.correctness}`);if (hermes?.tools) console.log(` Hermes baseline ${hermes.tools.count} tools · ${Number(hermes.tools.json_bytes).toLocaleString()} schema bytes`);else console.log(" Hermes baseline unavailable");console.log(` OpenClaw ${result.openclaw.version || "unavailable"} · no comparable offline prompt-size probe`);diff --git a/scripts/bench-read-pipeline.mjs b/scripts/bench-read-pipeline.mjsnew file mode 100644index 00000000..98b768a0--- /dev/null+++ b/scripts/bench-read-pipeline.mjs@@ -0,0 +1,71 @@+#!/usr/bin/env bun+import { runReadPipeline } from "../lib/mcp/read-pipeline-engine.ts";++const services = { rows: Array.from({ length: 500 }, (_, i) => ({+ name: `service-${String(i).padStart(3, "0")}`, status: i % 9 === 0 ? "degraded" : "healthy",+ cpu: (i * 17) % 100, memoryMb: 128 + (i % 24) * 64, detail: `bounded-service-metadata-${i}-`.repeat(4),+})) };+const projects = { rows: Array.from({ length: 400 }, (_, i) => ({+ id: `project-${i}`, state: i % 5 === 0 ? "paused" : "active", owner: `team-${i % 12}`, branch: `feature/${i}`,+ detail: `repository-metadata-${i}-`.repeat(3),+})) };+const sessions = { rows: Array.from({ length: 300 }, (_, i) => ({+ title: `Session ${i}`, updated: 1_800_000_000_000 - i * 1000, source: i % 2 ? "mcp" : "cli", note: `session-summary-${i}-`.repeat(3),+})) };+const apps = { apps: Array.from({ length: 200 }, (_, i) => ({+ id: `app-${i}`, running: i % 3 !== 0, version: `1.${i % 20}.${i % 7}`, statusDetail: `managed-app-status-${i}-`.repeat(4),+})) };++const fixtures = new Map([+ ["fixture_services", services], ["fixture_projects", projects], ["fixture_sessions", sessions], ["fixture_apps", apps],+]);+const tools = new Map([...fixtures].map(([name, value]) => [name, {+ name, description: name, scope: "read", annotations: { readOnlyHint: true, idempotentHint: true },+ inputSchema: { type: "object", properties: {} }, run: async () => value,+}]));+const input = { calls: [+ { id: "degraded", tool: "fixture_services", transform: { path: "rows", where: [{ field: "status", op: "eq", value: "degraded" }], sort: { field: "cpu", direction: "desc" }, limit: 10, select: ["name", "cpu", "memoryMb"] } },+ { id: "activeProjects", tool: "fixture_projects", transform: { path: "rows", where: [{ field: "state", op: "eq", value: "active" }], aggregate: { op: "count" } } },+ { id: "recentSessions", tool: "fixture_sessions", transform: { path: "rows", sort: { field: "updated", direction: "desc" }, limit: 5, select: ["title", "source", "updated"] } },+ { id: "runningApps", tool: "fixture_apps", transform: { path: "apps", where: [{ field: "running", op: "eq", value: true }], limit: 10, select: ["id", "version"] } },+] };++const bytes = (value) => Buffer.byteLength(JSON.stringify(value), "utf8");+const stable = (out) => ({+ ok: out.ok, mode: out.mode, results: out.results,+ evidence: out.evidence.map(({ id, tool, ok, rawBytes, outputBytes, reductionPct }) => ({ id, tool, ok, rawBytes, outputBytes, reductionPct })),+});++export async function runReadPipelineBenchmark() {+ const baselineBytes = [...fixtures.values()].reduce((sum, value) => sum + bytes(value), 0);+ const resolve = (name) => tools.get(name);+ const a = await runReadPipeline(input, { scope: "read", actor: "bench:pipeline-a" }, resolve);+ const b = await runReadPipeline(input, { scope: "read", actor: "bench:pipeline-b" }, resolve);+ const returnedBytes = bytes(a), reductionPct = Math.round((1 - returnedBytes / baselineBytes) * 1000) / 10;+ const expectedActiveProjects = projects.rows.filter((row) => row.state === "active").length;+ const correctness = a.results.activeProjects?.value === expectedActiveProjects+ && Array.isArray(a.results.degraded) && a.results.degraded.length === 10+ && Array.isArray(a.results.recentSessions) && a.results.recentSessions.length === 5+ && Array.isArray(a.results.runningApps) && a.results.runningApps.length === 10;+ const deterministic = JSON.stringify(stable(a)) === JSON.stringify(stable(b));+ return {+ baseline: { toolRoundTrips: input.calls.length, modelVisibleBytes: baselineBytes },+ pipeline: { toolRoundTrips: 1, modelVisibleBytes: returnedBytes, callsInsidePipeline: input.calls.length },+ savings: { roundTripReductionPct: Math.round((1 - 1 / input.calls.length) * 1000) / 10, modelVisibleByteReductionPct: reductionPct },+ correctness, deterministic,+ passed: correctness && deterministic && reductionPct >= 80,+ };+}++if (import.meta.main) {+ const result = await runReadPipelineBenchmark();+ if (process.argv.includes("--json")) console.log(JSON.stringify(result, null, 2));+ else {+ console.log("MSO read_pipeline benchmark");+ console.log(` raw baseline ${result.baseline.toolRoundTrips} model round-trips · ${result.baseline.modelVisibleBytes.toLocaleString()} bytes`);+ console.log(` read_pipeline ${result.pipeline.toolRoundTrips} model round-trip · ${result.pipeline.modelVisibleBytes.toLocaleString()} bytes`);+ console.log(` savings ${result.savings.roundTripReductionPct}% round-trips · ${result.savings.modelVisibleByteReductionPct}% model-visible bytes`);+ console.log(` correctness ${result.correctness ? "PASS" : "FAIL"} · deterministic=${result.deterministic}`);+ }+ if (!result.passed) process.exitCode = 1;+}diff --git a/scripts/bench-read-pipeline.test.ts b/scripts/bench-read-pipeline.test.tsnew file mode 100644index 00000000..5e4d138c--- /dev/null+++ b/scripts/bench-read-pipeline.test.ts@@ -0,0 +1,12 @@+import { describe, expect, it } from "vitest";+import { runReadPipelineBenchmark } from "./bench-read-pipeline.mjs";++describe("read_pipeline benchmark", () => {+ it("preserves exact fixture answers while reducing round-trips and model-visible bytes", async () => {+ const result = await runReadPipelineBenchmark();+ expect(result.correctness).toBe(true); expect(result.deterministic).toBe(true);+ expect(result.savings.roundTripReductionPct).toBeGreaterThanOrEqual(75);+ expect(result.savings.modelVisibleByteReductionPct).toBeGreaterThanOrEqual(80);+ expect(result.passed).toBe(true);+ });+});diff --git a/scripts/check-docs.mjs b/scripts/check-docs.mjsindex 92422b68..57e76358 100755--- a/scripts/check-docs.mjs+++ b/scripts/check-docs.mjs@@ -14,22 +14,33 @@ const server = /MCP_SERVER_VERSION\s*=\s*"([^"]+)"/.exec(toolset)?.[1];const version = /MCP_TOOLSET_VERSION\s*=\s*"([^"]+)"/.exec(toolset)?.[1];if (!server || !version) fail.push("could not parse MCP version constants");-const toolFiles = [- "lib/mcp/tools.ts",- "lib/mcp/tools-read.ts",- "lib/mcp/tools-discovery.ts",- "lib/mcp/tools-learning.ts",- "lib/mcp/tools-workflow-start.ts",- "lib/mcp/tools-workflow-lifecycle.ts",- "lib/mcp/tools-project-memory.ts",- "lib/mcp/tools-infra.ts",- "lib/mcp/tools-power.ts",- "lib/mcp/tools-agent.ts",- "lib/mcp/tools-a2a.ts",- "lib/mcp/tools-local-agents.ts",- "lib/mcp/tools-subagents.ts",- "lib/mcp/tools-forge.ts",-];+// Tool catalog modules may be split recursively (for example RASMIC workflow+// modules). Start at tools.ts and follow every local tools-* import transitively.+// Any top-level tools-*.ts module that is not reachable is dead/unregistered code+// and fails CI instead of silently escaping the documentation contract.+const toolModuleName = /^tools-[A-Za-z0-9-]+\.ts$/;+const discoveredToolModules = readdirSync(join(ROOT, "lib/mcp"))+ .filter((name) => toolModuleName.test(name))+ .sort();+const reachable = new Set();+const pending = ["tools.ts"];+while (pending.length) {+ const name = pending.pop();+ if (!name || reachable.has(name)) continue;+ reachable.add(name);+ const text = read(`lib/mcp/${name}`);+ for (const match of text.matchAll(/from\s+["']\.\/(tools-[A-Za-z0-9-]+)["']/g)) {+ const child = `${match[1]}.ts`;+ if (!reachable.has(child)) pending.push(child);+ }+}+for (const name of discoveredToolModules) {+ if (!reachable.has(name)) fail.push(`lib/mcp/${name}: tool module exists but is not reachable from lib/mcp/tools.ts`);+}+for (const name of reachable) {+ if (name !== "tools.ts" && !discoveredToolModules.includes(name)) fail.push(`lib/mcp/${name}: imported tool module is missing`);+}+const toolFiles = [...reachable].sort().map((name) => `lib/mcp/${name}`);const tools = new Map();for (const file of toolFiles) {const text = read(file);diff --git a/scripts/mso-agent-tool-router.test.ts b/scripts/mso-agent-tool-router.test.tsindex e7cbc35c..ce3db565 100644--- a/scripts/mso-agent-tool-router.test.ts+++ b/scripts/mso-agent-tool-router.test.ts@@ -23,6 +23,7 @@ const catalog = [tool("a2a_message_send", "Send a message to an A2A peer"), tool("a2a_handoff", "Delegate explicit objective to A2A peer"), tool("a2a_task_get", "Read A2A task status"),tool("tool_forge_candidates", "List Tool Forge candidates"), tool("tool_forge_propose", "Propose a Tool Forge candidate from a repeated recipe"),tool("tool_forge_evaluate", "Evaluate a Tool Forge candidate in the sandbox"), tool("tool_forge_promote", "Promote an evaluated Tool Forge candidate"),+ tool("read_pipeline", "Batch multiple read-only tools and aggregate or filter results"),];describe("MSO per-turn tool router", () => {@@ -126,4 +127,14 @@ describe("MSO per-turn tool router", () => {expect(out.selectedNames).toContain("project_memory_search");});+ it("keeps read_pipeline out of ordinary reads and selects it through the RASMIC catalog for batch aggregation", () => {+ const ordinary = selectToolsForTurn(catalog, [{ role: "user", text: "read the README file" }]);+ expect(ordinary.selectedNames).not.toContain("read_pipeline");+ const batched = selectToolsForTurn(catalog, [{ role: "user", text: "batch multiple reads and aggregate the results before returning them" }]);+ expect(batched.routeIds).toContain("read-pipeline");+ expect(batched.selectedNames).toContain("read_pipeline");+ expect(batched.catalogMatched).toBe(true);+ expect(batched.fallbackUsed).toBe(false);+ });+});