TypeScript
Acceptance — 20 verification metrics (PRD §19)
Each maps to a regression test under tests/. Run pnpm test.
BR — behavior (tests/br_behavior.test.ts)
| id | check |
|---|---|
| BR1 | combo validity & exclude reasons (no silent drop) |
| BR2 | subset determinism (same bench/tasks/seed → same subset; no Math.random) |
| BR3 | runner never throws (returns success:false + artifacts) |
| BR4 | harness per-run application (args/env/status) |
| BR5 | bench setup + recompute-verify (independent solver writes correct answer → verify true; wrong → false) |
| BR6 | multi-level subagent parsing (layout B, depth-2 via meta.toolUseId; token totals summed) |
| BR7 | fail-soft (synthetic failure recorded, not dropped) |
CO — consistency (tests/co_consistency.test.ts)
| id | check |
|---|---|
| CO1 | totalTokens == sum of token fields |
| CO2 | subagentCount ≤ toolCalls; codex/gpt = 0 subagents |
| CO3 | sort base key subagentCount → totalTokens → wallTimeMs |
| CO4 | cost precedence actual > estimated > none |
| CO5 | null ≠ 0 (uncollected stays null) |
| CO6 | codex failure fail-soft (missing binary → success:false, artifacts written) |
| CO7 | golden correctness via independent solvers (Bellman-Ford vs dijkstra; independent ledger sum) |
UX — presentation (tests/ux_presentation.test.ts)
| id | check |
|---|---|
| UX1 | scatter present |
| UX2 | task matrix present |
| UX3 | flow graph present (inline SVG, MAIN LOOP) |
| UX4 | node toggle present (skills/tools/log via native details) |
| UX5 | i18n EN↔KR switch — no Korean leak in EN report; KR headers present |
| UX6 | self-contained (no <script>, no CDN, no external src/link; only allowed http ref is the SVG namespace) |
Integration (tests/integration.test.ts): full mock pipeline, recorder layout, determinism, accuracy bounds.
Definition of Done
pnpm typecheck && pnpm lint && pnpm test && pnpm build all exit 0; runtime changes verified by real
(mock) execution end-to-end. Real model (cc/codex) call paths are implemented and type-checked but not
exercised with billable calls in CI.