Skip to content

Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.

Rust

revert(docs): restore DeepSWE benchmark results (#1056)

nachiketb-nvidia5415db205c1f
diff --git a/README.md b/README.md
index 519c27ca5..d4e94b441 100644
--- a/README.md
+++ b/README.md
@@ -101,7 +101,7 @@ The [routing overview](docs/routing_algorithms/overview.md) retains the full cat
## Evaluation and reference
-[Compare benchmark results and configs](docs/benchmarks.md) for Terminal-Bench 2.1.
+[Compare benchmark results and configs](docs/benchmarks.md) for Terminal-Bench 2.1 and DeepSWE.
For latency and routing overhead testing, see [Soak Testing](docs/operations/soak_test.md).
### Further reading
diff --git a/docs/benchmarks.md b/docs/benchmarks.md
index 50c772bc6..4a63d0e9d 100644
--- a/docs/benchmarks.md
+++ b/docs/benchmarks.md
@@ -2,7 +2,7 @@
Task completion, cost, and runnable configs. All configs use `OPENROUTER_API_KEY`.
-[Terminal-Bench 2.1](#terminal-bench-21) · [Run a benchmark](#run-a-benchmark)
+[Terminal-Bench 2.1](#terminal-bench-21) · [DeepSWE](#deepswe) · [Run a benchmark](#run-a-benchmark)
## Terminal-Bench 2.1
@@ -19,8 +19,6 @@ Task completion, cost, and runnable configs. All configs use `OPENROUTER_API_KEY
| Opus 4.8 only | 77.15% | $148.83 | — |
| GLM 5.2 only | 74.16% | $71.31 | — |
-<!-- DeepSWE results are hidden while run validity is being reviewed.
-
## DeepSWE
**v1.1 · 113 tasks · Codex CLI · GPT-5.6 Luna + Sol**
@@ -37,8 +35,6 @@ Task completion, cost, and runnable configs. All configs use `OPENROUTER_API_KEY
Luna: one run; others: three-run means. [Evaluation settings](../benchmark/DEEPSWE_V11_QUALIFICATION.md).
--->
-
## Run a benchmark
- **Terminal-Bench 2.1:** [Setup](../benchmark/README.md#one-time-setup) · [Run](../benchmark/README.md#run-with-switchyard-routing)
diff --git a/docs/index.md b/docs/index.md
index a71c97193..67c220ea0 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -18,7 +18,7 @@ The Server Path builds and runs the standalone `switchyard-server` binary.
## Explore
-- [Benchmarks](benchmarks.md): compare Terminal-Bench 2.1 results and open their routing configs
+- [Benchmarks](benchmarks.md): compare Terminal-Bench 2.1 and DeepSWE results and open their routing configs
- [Core Concepts](core_concepts.md): learn the LLM client, target, and route layers
- [Routing Algorithms](routing_algorithms/overview.md): choose how requests select a model
- [Architecture](architecture.md): understand the proxy and library components
diff --git a/docs/routing_algorithms/model_selection.md b/docs/routing_algorithms/model_selection.md
index b625e232f..519942617 100644
--- a/docs/routing_algorithms/model_selection.md
+++ b/docs/routing_algorithms/model_selection.md
@@ -129,8 +129,8 @@ behave differently on different endpoints. See
## Measured pairs
-See [Benchmarks](../benchmarks.md) for consolidated Terminal-Bench 2.1 results
-and config links.
+See [Benchmarks](../benchmarks.md) for consolidated Terminal-Bench 2.1 and DeepSWE results,
+config links, and evaluation settings.
These pairs have results from checked-in profiles or a public report. Each row
describes one pair on one workload. The thresholds and results do not transfer
@@ -146,7 +146,7 @@ serving stack.
What the rows show about model choice:
- The gap between the two single models sets the room for routing. Compare the
- fixed-model baselines with the routing results, as in the [Terminal-Bench 2.1 results](../benchmarks.md#terminal-bench-21).
+ fixed-model baselines with the routing results, as in the [DeepSWE studies](../benchmarks.md#deepswe).
- When the efficient model is close to the capable model, escalation keeps most
of the accuracy at a fraction of the cost. On the LangChain suite the two
single models were 8 points apart. Routing kept 80 of the 86 points at about