Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
revert(docs): restore DeepSWE benchmark results (#1056)
nachiketb-nvidia5415db205c1f
diff --git a/README.md b/README.mdindex 519c27ca5..d4e94b441 100644--- a/README.md+++ b/README.md@@ -101,7 +101,7 @@ The [routing overview](docs/routing_algorithms/overview.md) retains the full cat## Evaluation and reference-[Compare benchmark results and configs](docs/benchmarks.md) for Terminal-Bench 2.1.+[Compare benchmark results and configs](docs/benchmarks.md) for Terminal-Bench 2.1 and DeepSWE.For latency and routing overhead testing, see [Soak Testing](docs/operations/soak_test.md).### Further readingdiff --git a/docs/benchmarks.md b/docs/benchmarks.mdindex 50c772bc6..4a63d0e9d 100644--- a/docs/benchmarks.md+++ b/docs/benchmarks.md@@ -2,7 +2,7 @@Task completion, cost, and runnable configs. All configs use `OPENROUTER_API_KEY`.-[Terminal-Bench 2.1](#terminal-bench-21) · [Run a benchmark](#run-a-benchmark)+[Terminal-Bench 2.1](#terminal-bench-21) · [DeepSWE](#deepswe) · [Run a benchmark](#run-a-benchmark)## Terminal-Bench 2.1@@ -19,8 +19,6 @@ Task completion, cost, and runnable configs. All configs use `OPENROUTER_API_KEY| Opus 4.8 only | 77.15% | $148.83 | — || GLM 5.2 only | 74.16% | $71.31 | — |-<!-- DeepSWE results are hidden while run validity is being reviewed.-## DeepSWE**v1.1 · 113 tasks · Codex CLI · GPT-5.6 Luna + Sol**@@ -37,8 +35,6 @@ Task completion, cost, and runnable configs. All configs use `OPENROUTER_API_KEYLuna: one run; others: three-run means. [Evaluation settings](../benchmark/DEEPSWE_V11_QUALIFICATION.md).--->-## Run a benchmark- **Terminal-Bench 2.1:** [Setup](../benchmark/README.md#one-time-setup) · [Run](../benchmark/README.md#run-with-switchyard-routing)diff --git a/docs/index.md b/docs/index.mdindex a71c97193..67c220ea0 100644--- a/docs/index.md+++ b/docs/index.md@@ -18,7 +18,7 @@ The Server Path builds and runs the standalone `switchyard-server` binary.## Explore-- [Benchmarks](benchmarks.md): compare Terminal-Bench 2.1 results and open their routing configs+- [Benchmarks](benchmarks.md): compare Terminal-Bench 2.1 and DeepSWE results and open their routing configs- [Core Concepts](core_concepts.md): learn the LLM client, target, and route layers- [Routing Algorithms](routing_algorithms/overview.md): choose how requests select a model- [Architecture](architecture.md): understand the proxy and library componentsdiff --git a/docs/routing_algorithms/model_selection.md b/docs/routing_algorithms/model_selection.mdindex b625e232f..519942617 100644--- a/docs/routing_algorithms/model_selection.md+++ b/docs/routing_algorithms/model_selection.md@@ -129,8 +129,8 @@ behave differently on different endpoints. See## Measured pairs-See [Benchmarks](../benchmarks.md) for consolidated Terminal-Bench 2.1 results-and config links.+See [Benchmarks](../benchmarks.md) for consolidated Terminal-Bench 2.1 and DeepSWE results,+config links, and evaluation settings.These pairs have results from checked-in profiles or a public report. Each rowdescribes one pair on one workload. The thresholds and results do not transfer@@ -146,7 +146,7 @@ serving stack.What the rows show about model choice:- The gap between the two single models sets the room for routing. Compare the- fixed-model baselines with the routing results, as in the [Terminal-Bench 2.1 results](../benchmarks.md#terminal-bench-21).+ fixed-model baselines with the routing results, as in the [DeepSWE studies](../benchmarks.md#deepswe).- When the efficient model is close to the capable model, escalation keeps mostof the accuracy at a fraction of the cost. On the LangChain suite the twosingle models were 8 points apart. Routing kept 80 of the 86 points at about