Claude Code. Any Model. The most powerful AI coding agent now speaks every language.
docs(reports): record why deepseek-v4-pro answers wrong on Zen Go
Jack Rudenkoa7e17707c815
diff --git a/ai-docs/reports/recommended-models-bench-run-20261020.md b/ai-docs/reports/recommended-models-bench-run-20261020.mdindex 978386d9..d87016da 100644--- a/ai-docs/reports/recommended-models-bench-run-20261020.md+++ b/ai-docs/reports/recommended-models-bench-run-20261020.md@@ -107,3 +107,40 @@ With the earlier 16 passing rows, every one of the 24 recommended rows passes thand every model passes at least two of the three MCP Scenarios on one run. The `qwen3.8-max`pass came from its first hop serving; it does not exercise the #310 routing change, whoseevidence is `fallback-handler.subscription-only.test.ts`.++## Why the three MCP answers were wrong (measured 2026-10-21)++**Repeat with `--trials 3`** (the three models, all three MCP Scenarios; 27 Sessions): every+Session completed and relayed the model's answer. Wrong answers, all digit slips:++| Model | `run_prompt` | `create-session` | `team` |+|---|---|---|---|+| `deepseek-v4-pro` | 3/3 | 1/3 (`7713`, `7713`) | 2/3 (`7183`) |+| `deepseek-v4.1-flash` | 3/3 | 3/3 | 2/3 (`7713`) |+| `mimo-v2.6-pro` (both tiers) | 6/6 | 6/6 | 6/6 |++**What claudish sends.** A forwarding recorder on `OPENCODE_GO_BASE_URL` captured the real+request of an interactive claudish 12.2.0 session to `https://opencode.ai/inference/go/openai/v1/chat/completions`:+`{"model":"deepseek-v4-pro","temperature":1,"stream":true,"stream_options":{…},"max_tokens":32000}`+and no reasoning field, although Claude Code showed `xhigh effort`. That is the wire rule+(`wire.md`, decisions table): the catalog's `go-subscription` connection declares+`wire.reasoning.status: "unknown"`, and the owner's direct connection is on the `responses` API,+not the same API, so nothing is borrowed. MiMo's request also has no `max_tokens`+(`tokenParam` unknown).++**Replaying that exact body to Zen Go, 10 times per setting** (fresh `x-opencode-session` each):++| Body | `deepseek-v4-pro` correct | `deepseek-v4.1-flash` correct | `mimo-v2.6-pro` correct |+|---|---|---|---|+| as claudish sends it | 2/10 (8 replies had 0 reasoning tokens; all 8 wrong) | 9/10 | 9/10 |+| `+ reasoning_effort: "high"` | 5/10 | not run | 10/10 |+| `+ reasoning_effort: "max"` | 10/10 (reasoning 15-78 tokens) | 9/10 (0 reasoning tokens every time) | not run |+| `+ thinking: {"type":"enabled"}` | 6/10 | 10/10 (0 reasoning tokens) | 10/10 |++Conclusion: the wrong numbers come from the models, not from claudish. `deepseek-v4-pro` on Zen Go+guesses whenever it skips reasoning, and Zen Go honours `reasoning_effort` for it on Chat+Completions. With that field published in the catalog wire, claudish would clamp Claude Code's+`xhigh` to `max` and send it. That is catalog data for the `opencode`/`go-subscription` connection,+owned by models-index; claudish needs no change. `deepseek-v4.1-flash` never reasons on Zen Go+under any setting tried, and slips about 1 in 20. MiMo slips about 1 in 10 with no setting and+0 in 20 with reasoning requested.