Browser-based graphical shell and control plane for a Linux server you own — real terminal, file manager, live metrics and BYOK AI in one mobile-first workspace. Next.js 16 · React 19 · node-pty · Tailwind v4 · shadcn/ui. Public alpha, self-hosted.
TypeScript browser-shellcontrol-planefile-managerhomelabmobile-firstnextjsptyreactself-hostedserver-managementshadcn-uisysadmintailwindcssterminaltypescriptvpsweb-desktopweb-terminal
docs(agent): record P8 calibration baseline
Rahman6cd997a533ee
diff --git a/README.md b/README.mdindex 95ff76bb..8377dd6e 100644--- a/README.md+++ b/README.md@@ -79,7 +79,7 @@ For a real deployment, put MSO behind **Tailscale, a VPN, or a TLS reverse proxy- **Message live local sessions by a short name** — every durable same-principal MSO session gets a unique familiar handle such as `@milo`, `@luna`, or `@nara`, independent from its longer auto-generated session title. `/rename <name>` changes that handle, while `/title <text>` changes the session description. `@milo …` resolves only agents with a current lease **and** a live receiver subscription, then creates an explicitly correlated request; replies relay deterministically into the originating conversation while notify-only events remain passive. Presence lease (`idle`/`busy`/`offline`) and receiver subscription (`consumerConnected`) are reported separately so an idle-but-unsubscribed target is diagnosable. For ChatGPT/MCP sessions, `local_agent_inbox(wait_ms=...)` temporarily turns the foreground MCP call into that live receiver for up to 20 seconds, enabling direct two-way session messaging without a worker, DB, webhook, or broker. See [Local Agents](./docs/LOCAL-AGENTS.md).- **Spawn focused same-session subagents** — `/spawn` and `agent_subagent_run` create bounded foreground workers with isolated context, explicit delegated scope, turn/timeout caps, no recursive spawn, and final-result-only return. They are not Local Agent peers and do not run autonomously after the parent call returns. See [Subagents](./docs/SUBAGENTS.md).- **Delegate between remote agents with standard A2A v1** — register public HTTPS peers, attach private API-key/Bearer/OAuth access-token profiles, stream remote tasks over SSE, or expose MSO's authenticated inbound Agent Card when a public HTTPS origin is configured. Inbound credentials are owner-minted `read | write | exec` capabilities and run in a memory-isolated task context. See [A2A](./docs/A2A.md).-- **Keep agents efficient independently of the model provider** — the MCP-first [Cognitive Runtime](./docs/COGNITIVE-RUNTIME.md) isolates conversations from OAuth identity, budgets model context/tool output, uses the RASMIC catalog-first router to select a tiny per-turn capability pack without weakening scopes, compacts durable sessions with sanitized 30-day backup archives, learns only verified workflow recipes, promotes repeated procedures only through the eval-gated Tool Forge, and batches/filter/aggregates independent reads through `read_pipeline` before raw data reaches the model. `bun run bench:cognitive`, `bun run bench:pipeline`, isolated scratch-fixture `bun run bench:corpus`, and deterministic repeated `bun run bench:corpus:repeat` guard routing, correctness, policy compliance, repeat coverage, latency, context savings, and proof-gated normalized provider usage so cache/reasoning details are never double-counted or guessed.+- **Keep agents efficient independently of the model provider** — the MCP-first [Cognitive Runtime](./docs/COGNITIVE-RUNTIME.md) isolates conversations from OAuth identity, budgets model context/tool output, uses the RASMIC catalog-first router to select a tiny per-turn capability pack without weakening scopes, compacts durable sessions with sanitized 30-day backup archives, learns only verified workflow recipes, promotes repeated procedures only through the eval-gated Tool Forge, and batches/filter/aggregates independent reads through `read_pipeline` before raw data reaches the model. `bun run bench:cognitive`, `bun run bench:pipeline`, isolated scratch-fixture `bun run bench:corpus`, deterministic repeated `bun run bench:corpus:repeat`, and observation-only `bun run bench:cache-calibration` guard routing, correctness, policy compliance, repeat coverage/spread, latency, context savings, cache telemetry, and proof-gated normalized provider usage so cache/reasoning details are never double-counted or guessed.- **Make orchestration risk-aware and self-improving** — [RASMIC](./docs/RASMIC.md) classifies task risk/contention, retrieves compact repo-local memory, records manual user tests and Evidence Receipts, detects path/shared-resource collisions before merge conflicts, and promotes repeated safe routes from observed traces → recipes → tested bounded scripts.- **Add app slices** — features are modular under `frontend/slices/<slug>/`.- **Personalize the interface** — macOS, Windows, iOS, and Android shell layouts are UI preferences, not the core product.diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.mdindex bd362190..c8715b1a 100644--- a/docs/CHANGELOG.md+++ b/docs/CHANGELOG.md@@ -13,6 +13,7 @@ this is the *what*, and it is what Settings → About shows as “What's new”.**Fixed**+- `agent` compare normalized token proofs per row- `mcp` advertise implemented protocol era- `mcp` support trusted browser probes- `oauth` issue distinct dynamic client ids@@ -20,11 +21,13 @@ this is the *what*, and it is what Settings → About shows as “What's new”.**Faster**+- `agent` add P8 calibration harness- `agent` add repeatable corpus runner- `agent` expand objective repo corpus**Docs**+- `agent` record P8 calibration baseline- `agent` record P7 repeated corpus baseline## 2026-09-02diff --git a/docs/COGNITIVE-RUNTIME.md b/docs/COGNITIVE-RUNTIME.mdindex 4ac237b5..6581f200 100644--- a/docs/COGNITIVE-RUNTIME.md+++ b/docs/COGNITIVE-RUNTIME.md@@ -112,6 +112,7 @@ bun run bench:corpus -- --provider openai-codex --model gpt-5.6-terra # nbun run bench:corpus -- --provider openai-codex --model gpt-5.6-terra --run # isolated scratch-fixture quality corpusbun run bench:corpus:repeat -- --runs 2 --seed baseline-1 --provider openai-codex --model gpt-5.6-terra # deterministic repeated planbun run bench:corpus:repeat -- --runs 2 --seed baseline-1 --provider openai-codex --model gpt-5.6-terra --run # repeated full corpus+bun run bench:cache-calibration -- --rounds 4 --prefix-chars 12000 --seed cache-1 --provider openai-codex --model gpt-5.6-terra --run # observation-only repeated-prefix cache calibration```The core benchmark is deterministic and provider-neutral. Its checked scenarios cover application logs, server health, safe file editing, long/short execution, project functions, Cloudflare/Hostinger/Dokploy, browser/screenshots, memory, session resume, Skills, manual regression evidence, Indonesian repository changes, local-agent requests, subagents, A2A and Tool Forge. It also verifies the second phase after `workflow_start`.@@ -309,6 +310,44 @@ P7 also observed real provider-reported cache-read activity without forcing a ca5. **Keep benchmark → engineering feedback closed-loop** — fix deterministic routing/lifecycle/tool waste exposed by the corpus, then rerun the same objective fixture rather than prompting harder.6. **Memory calibration** — grow post-P1 telemetry and test authority/confidence/temporal policies against real corrections before introducing graph-memory complexity.+## P8 calibration: repeat spread, cache observation, and cost gate++P8 turns the P7 follow-up list into explicit calibration gates without changing the nine-scenario corpus. Repeated aggregation now reports **descriptive per-run distributions** for task success, average latency, p50 latency, and normalized tokens/attempt: count, min/max, median/mean, range, sample standard deviation, and coefficient of variation. These are observed sample descriptors only; the harness still does not manufacture confidence intervals, p-values, or universal reliability claims.++P8 also adds `bench:cache-calibration`. It creates one deterministic inert shared prefix, changes only a small exact reply marker per round, alternates the starting agent, and records provider-reported cache fields separately as positive, zero, or absent. Cache-hit frequency is intentionally **not ranked** because MSO and Hermes still have different request envelopes/system/tool context. The benchmark never forces a cache hit and never infers one from latency.++The dedicated cache fixture exposed a benchmark-accounting defect: one runner can legitimately move between multiple independently proven normalized shapes across calls (for example, `exact-inclusive-total-identity` when cache is zero and `exact-exclusive-cache-sum` when cache is positive). The old aggregate gate treated the mixed proof *names* as non-comparable even though every row normalized exactly to the same inclusive total semantics. P8 fixes this by proof-gating **each row**, while opaque/unknown rows still fail closed.++The final P8 cache calibration was run from source `51ae31f` on `openai-codex/gpt-5.6-terra` with four rounds and a 12,000-character shared prefix. All **8/8 calls succeeded** and provider/model evidence covered every row. MSO reported `cacheReadTokens=0` on **4/4** rows. Hermes reported zero on 2/4 and positive cache reads on **2/4** rows (**9,728** and **19,968** tokens). Token semantics remain comparable after per-row proof normalization; cache frequency remains observation-only. Cost remains non-comparable: MSO has no attributed cost field, while Hermes reports zero with source `none`, which is not evidence of a shared USD billing contract.++P8 then added three more complete independently seeded nine-scenario runs, also from `51ae31f`. Independent validation checked all **54 rows**, exact per-run/per-agent scenario identity, provider/model evidence, exit status, task/policy outcome, and normalized token arithmetic. Both agents achieved **27/27 task success + 27/27 scenario-policy and 3/3 perfect runs**.++| P8 3-run metric | MSO | Hermes |+|---|---:|---:|+| Task success | **27/27 (100%)** | **27/27 (100%)** |+| Scenario-observable policy compliance | **27/27 (100%)** | **27/27 (100%)** |+| Perfect full runs | **3/3** | **3/3** |+| Average latency | **19,028.2 ms** | 22,248.0 ms |+| Aggregate p50 latency | **14,259 ms** | 20,472 ms |+| Normalized tokens / attempt | **16,665.0** | 67,936.2 |+| Run-level avg-latency CV | 6.8% | 4.0% |+| Run-level p50 CV | 10.6% | 12.6% |+| Run-level token/attempt CV | 2.3% | 1.9% |+| Cost semantics comparable | no | no |++On these three P8 runs, correctness/reliability is a tie. MSO used about **75.5% fewer normalized tokens per attempt (~4.08× smaller)**, with about **14.5% lower average latency** and **30.3% lower aggregate p50**. Those are bounded observed-run measurements, not provider/model-wide claims.++Combining the raw P7 two-run artifact and raw P8 three-run artifact gives a five-run descriptive history on the same corpus/provider/model. Both agents are **5/5 perfect**. Mean normalized tokens/attempt are **16,907.6 MSO vs 66,791.1 Hermes** (~**74.7% lower**, Hermes ~3.95× higher); mean per-run average latency is **18,985.9 vs 21,779.8 ms** (~**12.8% lower** for MSO); mean per-run p50 is **14,130.8 vs 21,457.2 ms** (~**34.1% lower**). Run-level token CV is 5.5% vs 2.7%. Five runs are enough to expose observed spread and remove the earlier one-run reliability anomaly, but still too small for an inferential reliability claim.++## P8 next direction++1. **Reliability stays descriptive until the sample is materially deeper** — add more full independent seeds only when a decision actually needs confidence bounds; do not turn `n=5` into pseudo-statistics.+2. **Equalize cache request envelopes before cache comparison** — the dedicated fixture now proves telemetry handling, but cache-hit frequency should remain unranked until the runners expose equivalent system/tool/request prefix behavior.+3. **Cost remains blocked on attribution, not arithmetic** — enable cost comparison only when both runners expose the same attributable USD source/contract; zero with source `none` remains unknown for comparative billing.+4. **Add OpenClaw only with a legitimate matched provider/model path** — do not mutate credentials merely to populate a benchmark table.+5. **Move next to memory calibration** — test P1 authority/confidence/temporal/correction policies against real repeated corrections before considering graph-memory complexity.+6. **Keep benchmark → engineering feedback closed-loop** — when a fixture exposes deterministic tool/routing/accounting waste, fix the system and rerun the same objective gate rather than prompting harder.+## Security notes- Session archives are private backups, not secret vaults; recursive redaction happens before compression.diff --git a/docs/PROGRESS.md b/docs/PROGRESS.mdindex 0ce962c7..a33bbae4 100644--- a/docs/PROGRESS.md+++ b/docs/PROGRESS.md@@ -4,6 +4,14 @@- **MCP protocol-era correctness (2026-09-03):** server `1.7.3` stops claiming the stateless `2026-07-28` protocol while serving the legacy initialize/session wire shape. The supported ceiling is deliberately `2025-06-18` (matching the known-good CareerPack ChatGPT connector); a modern HTTP probe receives a 400 and can fall back to initialize. `server/discover` is no longer aliased to the legacy initialize payload. Browser CORS also permits `Mcp-Method`/`Mcp-Name` so a standards-aware probe can reach the server and receive the fallback signal. Toolset stays `2026.09.03.4`.Running log of what shipped each phase. Newest at top.+## 2026-09-03 — Cognitive Runtime P8 calibration++- **Observed run spread, not pseudo-statistics:** repeated corpus aggregation now reports per-run min/max/mean/median/range/sample-SD/CV for success, average latency, p50 and normalized tokens/attempt. Three new complete `openai-codex/gpt-5.6-terra` runs from source `51ae31f` produced **27/27 task + 27/27 scenario-policy** and **3/3 perfect runs** for both agents; all 54 rows and token arithmetic proofs passed independent validation.+- **Five-run reliability history is a tie:** combining the prior two P7 runs with the three P8 runs yields **5/5 perfect full runs for both MSO and Hermes**. The observed five-run means are 16,907.6 vs 66,791.1 normalized tokens/attempt (~74.7% lower for MSO), 18,985.9 vs 21,779.8 ms average latency (~12.8% lower), and 14,130.8 vs 21,457.2 ms mean per-run p50 (~34.1% lower). `n=5` remains descriptive; no CI/p-value or universal reliability claim is made.+- **Dedicated repeated-prefix cache calibration:** new `bench:cache-calibration` keeps one inert shared prefix, varies only exact reply markers, rotates starting agent, and distinguishes positive/zero/absent provider cache fields. The final 4-round 12K-prefix run was 8/8 success; MSO reported zero cache reads on 4/4, Hermes naturally reported positive cache reads on 2/4 (9,728 and 19,968 tokens). Cache frequency remains unranked because runner request envelopes differ.+- **Benchmark accounting bug fixed:** valid normalized proof shapes may legitimately vary by row when cache behavior changes. Comparability now proof-gates every row instead of requiring one aggregate proof label; opaque/unknown rows still fail closed. This preserves token comparability in the real cache-hit run without weakening arithmetic requirements.+- **Cost gate remains closed:** MSO still has no attributable cost field while Hermes reports zero with source `none`. Cost comparison stays withheld until both runners expose the same attributable USD contract.+## 2026-09-03 — Cognitive Runtime P7 repeatable corpus baseline- **Deterministic repeat contract:** added `bench:corpus:repeat`, explicit single-run `--seed`, deterministic per-run seed derivation, and alternating starting-agent order. Repeated aggregation is fail-closed on missing runs, seed/order drift, duplicate or missing per-agent scenario identities, and cross-run provider/model drift.diff --git a/docs/TESTING-HANDOFF.md b/docs/TESTING-HANDOFF.mdindex 2e4ff539..2488651c 100644--- a/docs/TESTING-HANDOFF.md+++ b/docs/TESTING-HANDOFF.md@@ -7,6 +7,7 @@ Update this file at the end of every manual or automated test session. Keep it fThe current release target is **MSO CLI 1.12.0**, MCP server `1.7.3`, toolset `2026.09.03.4`; use `mso --version` plus `/api/health` for the exact Git build because documentation-only release commits also advance that identity. The v1.10/v1.11 findings and the v1.12 sectioned-terminal/recoverable-error work below are resolved and retained as historical test evidence. Do not re-open a resolved finding from prose alone: reproduce it against the current build first. There is no known 1.12 release blocker in this handoff.Current verified behavior:+- P8 calibration: repeat aggregation now reports descriptive per-run spread without inferential claims; three new complete nine-scenario runs from source `51ae31f` were **27/27 task + 27/27 scenario-policy and 3/3 perfect runs for both agents**, with 54/54 row identity/provider/model/token-arithmetic checks passing. Combined with P7, both agents are **5/5 perfect**; five-run means are MSO vs Hermes 16,907.6 vs 66,791.1 normalized tokens/attempt (~74.7% lower for MSO), 18,985.9 vs 21,779.8 ms average latency, and 14,130.8 vs 21,457.2 ms per-run p50. New `bench:cache-calibration` final 4-round/12K-prefix run was 8/8 success; MSO cache-read was zero on 4/4 while Hermes naturally reported positive cache reads on 2/4 (9,728 + 19,968), but cache frequency is unranked because request envelopes differ. P8 also fixes false token non-comparability when independently valid normalized proof shapes vary across cache/no-cache rows. Cost remains non-comparable because attribution is asymmetric.- P7 repeatability gate: `bench:corpus:repeat` adds explicit/deterministic seeds, alternating starting-agent order, and strict per-run/per-agent scenario + provider/model identity checks before any cross-run ranking. The first two complete 9-scenario runs from source `5bd5788` on `openai-codex/gpt-5.6-terra` were 18/18 task + 18/18 scenario-policy for both agents and 2/2 perfect runs each. Aggregate MSO vs Hermes: 17,271.3 vs 65,073.3 normalized tokens/attempt (~73.5% lower for MSO), 18,922.6 vs 21,077.4 ms average latency, and 13,761 vs 20,513.5 ms p50. All 36 token-proof arithmetic checks passed. This is a bounded two-run baseline, not statistical reliability proof; cost remains non-comparable.- MCP 1.7 release candidate: ChatGPT is fail-closed to a 29-tool compact MSO-only profile (~31.8 KiB descriptor JSON / ~8k rough tokens), full MSO remains 73 transport tools, and project-specific MCP names stay dynamic behind `project_mcp_tools` / `project_mcp_call`. OAuth resource binding + rotating refresh credentials, Streamable HTTP protocol handling, and legacy registered ChatGPT-client profile detection have focused regression coverage. Deterministic security gates (Trivy/OSV/Gitleaks/Semgrep/ShellCheck) pass; Codex Security AI scanning is explicitly incomplete because executor/cost guards stopped it, with 0 findings in the targeted partial reports.- P6 `mso-agent-quality-v2` expands the matched corpus to nine scenarios with repo-debug, schema migration, and transactional rollback. Final `openai-codex/gpt-5.6-terra` full run: MSO 9/9 task + 9/9 scenario-policy, Hermes 8/9 + 9/9; normalized tokens/attempt 16,683.4 vs 67,433.1 (~75.3% lower for MSO), p50 15,285 vs 23,294 ms, while MSO average latency was ~2.6% higher. The Hermes `multi-read` miss did not reproduce in a 3/3-vs-3/3 alternating-order follow-up, so it is not treated as a universal reliability claim. P6 also fixes over-broad manual-test routing, restores explicit exec validation for repo tasks, and prevents duplicate `workflow_start` inside long single-user turns. Cost remains non-comparable.