Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
feat: add pi and Oh My Pi support for switchyard-server (#827)
Elyas Mehtabuddincff376ffb32c
diff --git a/README.md b/README.mdindex 767f98ad0..e2b0f9ece 100644--- a/README.md+++ b/README.md@@ -65,8 +65,8 @@ flowchart LR### Run Switchyard as a standalone proxyA server in front of an agent, when you have no gateway to put Switchyard in.-Point Claude Code, Codex CLI, or any OpenAI/Anthropic SDK client at it;-Switchyard decides per turn which model serves it.+Point Claude Code, Codex CLI, pi, Oh My Pi, or any OpenAI/Anthropic SDK client+at it; Switchyard decides per turn which model serves it.- Install: `cargo install --locked switchyard-server`- Then follow [Path 3 — Run the Standalone Proxy](#path-3--run-the-standalone-proxy):@@ -260,6 +260,13 @@ codex --model switchyard -c 'model_provider="switchyard"'No Codex API key is needed for this local setup. Switchyard uses the server's`OPENROUTER_API_KEY` for upstream requests.+For pi, add a `switchyard` provider to `~/.pi/agent/models.json`. For Oh My Pi, add+the same provider to `~/.omp/agent/models.yml`. Both entries set `baseUrl` to the+server address and list the route id `switchyard` as a model.+[Use Switchyard with pi](docs/integrations/pi.md) and+[Use Switchyard with Oh My Pi](docs/integrations/oh_my_pi.md) give the exact entries,+say which request API to pick, and show how to confirm the routing.+## Routing AlgorithmsStart with **Auto**. Choose Task or Execution when you want more control overdiff --git a/benchmark/DATASETS.md b/benchmark/DATASETS.mdindex 3e8596dfa..178b5170f 100644--- a/benchmark/DATASETS.md+++ b/benchmark/DATASETS.md@@ -130,10 +130,11 @@ For each task, the rewriter prepares an image with pinned agent tooling:- If the task already has `environment/Dockerfile`, it appends the pinned agent install layer.- If neither exists, it creates a minimal `environment/Dockerfile` from `ubuntu:22.04`.-The injected layer installs the pinned Node, Claude Code, Codex, and OpenCode versions from-`benchmark/agent-versions.env`, then runs `claude --version`, `codex --version`, and-`opencode --version` during image build. The base image must support `apt-get`, `apk`, or `yum`, or-already provide `curl`, `tar`, and `gzip`. It must be `x86_64/amd64` or `aarch64/arm64`.+The injected layer installs the pinned Node, Claude Code, Codex, OpenCode, and pi versions from+`benchmark/agent-versions.env`, then runs `claude --version`, `codex --version`,+`opencode --version`, and `pi --version` during image build. The base image must support `apt-get`,+`apk`, or `yum`, or already provide `curl`, `tar`, and `gzip`. It must be `x86_64/amd64` or+`aarch64/arm64`.For non-TB datasets, inspect generated Dockerfiles before a full run:diff --git a/benchmark/README.md b/benchmark/README.mdindex 0876f73d0..2b8f9e696 100644--- a/benchmark/README.md+++ b/benchmark/README.md@@ -267,6 +267,30 @@ bash benchmark/run-baseline.sh \Tune `--n-concurrent` for your machine and provider quota. Use `--task-id`, `--task-list-file`, or`--n-tasks` for subsets.+### Run with the pi coding agent++```bash+bash benchmark/run-baseline.sh \+ --harbor-path benchmark/datasets/openthoughts-tblite-closed-book \+ --server-config benchmark/server-configs/tb-lite-llm-classifier-opus-kimi-gemini.toml \+ --agent pi \+ --model switchyard \+ --reasoning-effort high \+ --harbor-extra --ae --harbor-extra PI_CONTEXT_WINDOW=200000 \+ --harbor-extra --ae --harbor-extra PI_MAX_OUTPUT_TOKENS=32000 \+ --n-concurrent 8 \+ --max-retries 2+```++With `--server-config`, the script passes the model label `switchyard/<route>` to Harbor's pi+agent. The patched agent then writes `~/.pi/agent/models.json` inside the task container. That file+defines a `switchyard` provider that points at `OPENAI_BASE_URL` and uses the `openai-completions`+API. The agent environment variables `PI_CONTEXT_WINDOW` and `PI_MAX_OUTPUT_TOKENS` set+`contextWindow` and `maxTokens` on that model entry. When they are unset, pi uses its defaults of+128000 and 16384. `--reasoning-effort` sets pi's `--thinking` level, so pass one of `off`,+`minimal`, `low`, `medium`, `high`, or `xhigh`. Without `--server-config`, pass pi's own provider+label as `--model`, for example `openrouter/openai/gpt-5.5`.+## Inspect A RunRun directories are created under `benchmark/tb_runs/`. The most useful artifacts are:diff --git a/benchmark/agent-versions.env b/benchmark/agent-versions.envindex a064b2b37..efe46c35d 100644--- a/benchmark/agent-versions.env+++ b/benchmark/agent-versions.env@@ -4,6 +4,8 @@CLAUDE_CODE_VERSION=2.1.211CODEX_VERSION=0.144.5OPENCODE_VERSION=1.18.3+# pi coding agent (npm package @earendil-works/pi-coding-agent).+PI_VERSION=0.84.3NODE_VERSION=20.11.1# Hermes (NousResearch hermes-agent), installed from GitHub at dataset-bake time.# Must be a full 40-character commit SHA; anything else is rejected at build time.diff --git a/benchmark/patches/harbor-agent-patches.diff b/benchmark/patches/harbor-agent-patches.diffindex 54454cd68..b2c66d9b1 100644--- a/benchmark/patches/harbor-agent-patches.diff+++ b/benchmark/patches/harbor-agent-patches.diff@@ -413,6 +413,154 @@f"opencode --model={self.model_name} run --format=json {cli_flags_arg}--thinking --dangerously-skip-permissions -- {escaped_instruction} "f"2>&1 </dev/null | stdbuf -oL tee /logs/agent/opencode.txt"),+--- a/harbor/agents/installed/pi.py++++ b/harbor/agents/installed/pi.py+@@ -14,6 +14,8 @@++ class Pi(BaseInstalledAgent):+ _OUTPUT_FILENAME = "pi.txt"++ # Source nvm only if it exists: the prebaked dataset image installs pi on PATH without nvm.++ _NVM_PRELUDE = '[ -s "$HOME/.nvm/nvm.sh" ] && . "$HOME/.nvm/nvm.sh"; '++ CLI_FLAGS = [+ CliFlag(+@@ -29,7 +31,7 @@+ return AgentName.PI.value++ def get_version_command(self) -> str | None:+- return ". ~/.nvm/nvm.sh; pi --version"++ return f"{self._NVM_PRELUDE}pi --version"++ def parse_version(self, stdout: str) -> str:+ return stdout.strip().splitlines()[-1].strip()+@@ -37,7 +39,14 @@+ async def install(self, environment: BaseEnvironment) -> None:+ await self.exec_as_root(+ environment,+- command="apt-get update && apt-get install -y curl",++ command=(++ # Skip if pi is already installed (e.g. from agent-bake layer).++ "if command -v pi >/dev/null 2>&1; then"++ f" echo 'pi {self._version or 'unknown'} already installed, skipping system pkg install';"++ " exit 0;"++ "fi;"++ "apt-get update && apt-get install -y curl"++ ),+ env={"DEBIAN_FRONTEND": "noninteractive"},+ )+ version_spec = f"@{self._version}" if self._version else "@latest"+@@ -45,12 +54,17 @@+ environment,+ command=(+ "set -euo pipefail; "++ # Skip if pi is already installed (e.g. from agent-bake layer).++ "if command -v pi >/dev/null 2>&1; then"++ f" echo 'pi {self._version or 'unknown'} already installed, skipping install';"++ " exit 0;"++ "fi;"+ "curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.2/install.sh | bash && "+ 'export NVM_DIR="$HOME/.nvm" && '+ '\\. "$NVM_DIR/nvm.sh" || true && '+ "command -v nvm &>/dev/null || { echo 'Error: NVM failed to load' >&2; exit 1; } && "+ "nvm install 22 && npm -v && "+- f"npm install -g @mariozechner/pi-coding-agent{version_spec} && "++ f"npm install -g @earendil-works/pi-coding-agent{version_spec} && "+ "pi --version"+ ),+ )+@@ -65,6 +79,46 @@+ f"$HOME/.agents/skills/ 2>/dev/null || true"+ )+++ def _build_models_json_command(self, provider: str, model_id: str) -> str | None:++ """Return a shell command that writes ~/.pi/agent/models.json for Switchyard.++++ pi has no built-in "switchyard" provider and does not read OPENAI_BASE_URL, so this++ command adds the gateway as a custom provider before pi starts.++ """++ if provider != "switchyard":++ return None++ base_url = self._get_env("OPENAI_BASE_URL")++ if not base_url:++ raise ValueError("--model switchyard/<route> requires OPENAI_BASE_URL")++ model_entry: dict[str, object] = {++ "id": model_id,++ "name": f"{model_id} (Switchyard)",++ "reasoning": True,++ }++ context_window = self._get_env("PI_CONTEXT_WINDOW")++ if context_window:++ model_entry["contextWindow"] = int(context_window)++ max_output_tokens = self._get_env("PI_MAX_OUTPUT_TOKENS")++ if max_output_tokens:++ model_entry["maxTokens"] = int(max_output_tokens)++ models_config = {++ "providers": {++ "switchyard": {++ "baseUrl": base_url,++ "api": "openai-completions",++ # Keep the literal string "$OPENAI_API_KEY"; pi expands the variable itself.++ "apiKey": "$OPENAI_API_KEY",++ "compat": {"supportsDeveloperRole": False},++ "models": [model_entry],++ }++ }++ }++ # shlex.quote wraps the JSON in single quotes so the shell does not expand "$OPENAI_API_KEY".++ return (++ 'mkdir -p "$HOME/.pi/agent" && '++ f'echo {shlex.quote(json.dumps(models_config))} > "$HOME/.pi/agent/models.json"'++ )+++ @with_prompt_template+ async def run(+ self,+@@ -77,7 +131,7 @@+ if not self.model_name or "/" not in self.model_name:+ raise ValueError("Model name must be in the format provider/model_name")++- provider, _ = self.model_name.split("/", 1)++ provider, model_id = self.model_name.split("/", 1)++ env: dict[str, str] = {}+ keys: list[str] = []+@@ -106,7 +160,7 @@+ keys.append("HF_TOKEN")+ elif provider == "mistral":+ keys.append("MISTRAL_API_KEY")+- elif provider == "openai":++ elif provider in ("openai", "switchyard"):+ keys.append("OPENAI_API_KEY")+ elif provider == "openrouter":+ keys.append("OPENROUTER_API_KEY")+@@ -123,9 +177,7 @@+ if val:+ env[key] = val++- model_args = (+- f"--provider {provider} --model {self.model_name.split('/', 1)[1]} "+- )++ model_args = f"--provider {provider} --model {model_id} "++ cli_flags = self.build_cli_flags()+ if cli_flags:+@@ -134,11 +186,14 @@+ skills_command = self._build_register_skills_command()+ if skills_command:+ await self.exec_as_agent(environment, command=skills_command)++ models_json_command = self._build_models_json_command(provider, model_id)++ if models_json_command:++ await self.exec_as_agent(environment, command=models_json_command)++ await self.exec_as_agent(+ environment,+ command=(+- f". ~/.nvm/nvm.sh; "++ f"{self._NVM_PRELUDE}"+ f"pi --print --mode json --no-session "+ f"{model_args}"+ f"{cli_flags}"+--- a/harbor/models/trial/config.py+++ b/harbor/models/trial/config.py@@ -219,4 +219,5 @@@@ -425,7 +573,7 @@--- /dev/null+++ b/harbor/switchyard_patch_id.txt@@ -0,0 +1 @@-+switchyard-harbor-patches-2026-05-22-v2++switchyard-harbor-patches-2026-09-22-v3--- a/harbor/agents/installed/hermes.py+++ b/harbor/agents/installed/hermes.pydiff --git a/benchmark/prepare_harbor_dataset.py b/benchmark/prepare_harbor_dataset.pyindex 79d5c1993..d555fa789 100644--- a/benchmark/prepare_harbor_dataset.py+++ b/benchmark/prepare_harbor_dataset.py@@ -3,8 +3,6 @@"""Prepare a local closed-book Harbor dataset with prebaked coding agents."""-from __future__ import annotations-import argparseimport hashlibimport json@@ -200,6 +198,7 @@ def _install_layer(pins: dict[str, str]) -> str:claude_version = pins["CLAUDE_CODE_VERSION"]codex_version = pins["CODEX_VERSION"]opencode_version = pins["OPENCODE_VERSION"]+ pi_version = pins["PI_VERSION"]# Hermes (NousResearch hermes-agent) is a per-user uv app installed from# GitHub, not an npm package. Baking it here (build-time, with host network)# means the runtime install() skip-guard short-circuits, so tasks need no@@ -228,7 +227,7 @@ def _install_layer(pins: dict[str, str]) -> str:return f"""# Switchyard benchmark prebaked coding agents.-ENV SWITCHYARD_PREBAKED_AGENT_VERSIONS="claude-code={claude_version},codex={codex_version},opencode={opencode_version},node={node_version},hermes={hermes_version}"+ENV SWITCHYARD_PREBAKED_AGENT_VERSIONS="claude-code={claude_version},codex={codex_version},opencode={opencode_version},pi={pi_version},node={node_version},hermes={hermes_version}"RUN set -eux; \\if command -v apt-get >/dev/null 2>&1; then \\apt-get update; \\@@ -265,10 +264,12 @@ RUN set -eux; \\npm install -g \\"@anthropic-ai/claude-code@{claude_version}" \\"@openai/codex@{codex_version}" \\- "opencode-ai@{opencode_version}"; \\+ "opencode-ai@{opencode_version}" \\+ "@earendil-works/pi-coding-agent@{pi_version}"; \\claude --version; \\codex --version; \\- opencode --version+ opencode --version; \\+ pi --versionRUN set -eux; \\export HOME=/root; \\export PATH="/root/.local/bin:$PATH"; \\@@ -506,13 +507,17 @@ def _merge_compose(task_dir: Path, proxy_allowlist_hosts: tuple[str, ...]) -> didef prepare_dataset(- *,source_dataset: str,source_dir: Path | None,output_dir: Path,harbor_command: str,overwrite: bool,) -> Path:+ """Copy a Harbor dataset to `output_dir` and bake the pinned agents into its task images.++ Every agent pin in `benchmark/agent-versions.env` must be present; a missing pin raises+ `ValueError` before any download or copy starts.+ """pins = _read_env_file(AGENT_VERSIONS_FILE)required = {"CLAUDE_CODE_VERSION",@@ -520,6 +525,7 @@ def prepare_dataset("HERMES_VERSION","NODE_VERSION","OPENCODE_VERSION",+ "PI_VERSION",}missing = sorted(required - pins.keys())if missing:diff --git a/benchmark/run-baseline.sh b/benchmark/run-baseline.shindex 37a195e9a..115a31103 100755--- a/benchmark/run-baseline.sh+++ b/benchmark/run-baseline.sh@@ -24,6 +24,7 @@ fiCLAUDE_CODE_VERSION="${CLAUDE_CODE_VERSION:-2.1.211}"CODEX_VERSION="${CODEX_VERSION:-0.144.5}"OPENCODE_VERSION="${OPENCODE_VERSION:-1.18.3}"+PI_VERSION="${PI_VERSION:-0.84.3}"NODE_VERSION="${NODE_VERSION:-20.11.1}"DEFAULT_HARBOR_MODEL="openai/gpt-5.2"@@ -97,8 +98,9 @@ Main options:--server-config this is a Switchyard routekey; without it, this is the upstream model.Defaults --harbor-model to nvidia/MODEL for- opencode, openai/MODEL for other OpenAI-style- agents, and MODEL for claude-code/codex.+ opencode, switchyard/MODEL for pi, openai/MODEL+ for other OpenAI-style agents, and MODEL for+ claude-code/codex.--route-model MODEL Deprecated alias for --model in Switchyard mode.--agent NAME Harbor agent (default: terminus-2)--harbor-model MODEL Explicit Harbor model label override.@@ -111,7 +113,8 @@ Main options:--book-mode MODE closed or open (default: closed). Both modes usethe generated dataset proxy topology; open modeallows broad egress through the proxy.- --reasoning-effort VALUE Forwarded as --ak reasoning_effort=VALUE.+ --reasoning-effort VALUE Forwarded as --ak reasoning_effort=VALUE+ (--ak thinking=VALUE for pi).Defaults by agent/model; pass an empty value to omit.--harbor-bin PATH Optional Harbor executable override(default: uv run --no-sync harbor)@@ -288,6 +291,7 @@ add_agent_version_kwarg() {claude-code) HARBOR_CMD+=(--ak "version=${CLAUDE_CODE_VERSION}") ;;codex) HARBOR_CMD+=(--ak "version=${CODEX_VERSION}") ;;opencode) HARBOR_CMD+=(--ak "version=${OPENCODE_VERSION}") ;;+ pi) HARBOR_CMD+=(--ak "version=${PI_VERSION}") ;;esac}@@ -401,6 +405,7 @@ if [[ "${HARBOR_MODEL_SET}" -eq 0 ]]; thencase "${AGENT}" inclaude-code|codex) HARBOR_MODEL="${MODEL}" ;;opencode) HARBOR_MODEL="nvidia/${MODEL}" ;;+ pi) HARBOR_MODEL="switchyard/${MODEL}" ;;*) HARBOR_MODEL="openai/${MODEL}" ;;esacelse@@ -573,7 +578,12 @@ fiadd_agent_version_kwargif [[ -n "${REASONING_EFFORT}" ]]; then- HARBOR_CMD+=(--ak "reasoning_effort=${REASONING_EFFORT}")+ if [[ "${AGENT}" == "pi" ]]; then+ # Harbor's pi agent takes --thinking (off|minimal|low|medium|high|xhigh), not reasoning_effort.+ HARBOR_CMD+=(--ak "thinking=${REASONING_EFFORT}")+ else+ HARBOR_CMD+=(--ak "reasoning_effort=${REASONING_EFFORT}")+ fi# Claude code does not support thinking=true for now, so we use adaptive insteadif [[ "${AGENT}" == "claude-code" ]] && ! harbor_extra_has_agent_kwarg thinking; thenHARBOR_CMD+=(--ak "thinking=adaptive")@@ -747,7 +757,7 @@ fiAGENT_VERSIONS_JSON="$(json_object_from_pairs \claude_code "${CLAUDE_CODE_VERSION}" codex "${CODEX_VERSION}" \- opencode "${OPENCODE_VERSION}" node "${NODE_VERSION}")"+ opencode "${OPENCODE_VERSION}" pi "${PI_VERSION}" node "${NODE_VERSION}")"HARBOR_EXTRA_JSON="$(json_array)"if [[ "${#HARBOR_EXTRA[@]}" -gt 0 ]]; thenHARBOR_EXTRA_JSON="$(json_array "${HARBOR_EXTRA[@]}")"diff --git a/crates/switchyard-server/README.md b/crates/switchyard-server/README.mdindex 7cf6930e8..152dbfe02 100644--- a/crates/switchyard-server/README.md+++ b/crates/switchyard-server/README.md@@ -147,9 +147,15 @@ are required. All configured semantic names use exact ASCII case-insensitive mathandoff notes, per-tier system prompts, and a capability-judge fallback are documented in[Stage-Router Routing](../../docs/routing_algorithms/stage_router_routing.md).-## Codex model discovery+## Model discovery`GET /v1/models` returns the standard `data` list and an empty Codex `models` list.+Each entry reports the route's declared `tool_calling` and `vision` under `capabilities`.+It reports the route's declared `context_window` as the top-level `context_length` field.+OpenAI-compatible clients such as Oh My Pi read `context_length` when they build their+model list from this endpoint. See+[Use Switchyard with Oh My Pi](../../docs/integrations/oh_my_pi.md).+Codex keeps its own model catalog and instructions. Select a Switchyard route explicitlywith `codex --model route-id`; route aliases do not appear automatically in Codex's modelpicker. Unknown aliases use Codex's generic defaults and do not receive Switchyard'sdiff --git a/crates/switchyard-server/src/lib.rs b/crates/switchyard-server/src/lib.rsindex 4faf09821..d2d81fe4f 100644--- a/crates/switchyard-server/src/lib.rs+++ b/crates/switchyard-server/src/lib.rs@@ -1556,11 +1556,12 @@ fn model_entry_json(model: &str, capabilities: ModelCapabilities) -> Value {"created": 0,"owned_by": "switchyard","display_name": model,+ // OpenAI-compatible clients read the context window from this field.+ "context_length": capabilities.context_window,"capabilities": {"streaming": true,"tool_calling": capabilities.tool_calling,"vision": capabilities.vision,- "context_window": capabilities.context_window,"supported_inbound_formats": ["openai-chat-completions","openai-responses",diff --git a/crates/switchyard-server/tests/server.rs b/crates/switchyard-server/tests/server.rsindex 29bb561e7..dae5c1ad0 100644--- a/crates/switchyard-server/tests/server.rs+++ b/crates/switchyard-server/tests/server.rs@@ -3432,21 +3432,22 @@ target = "shared"assert_eq!(models.status, StatusCode::OK);let body = models.json()?;let data = body["data"].as_array().cloned().unwrap_or_default();- let capabilities = data+ let entries = data.iter()- .filter_map(|entry| entry["id"].as_str().map(|id| (id, &entry["capabilities"])))+ .filter_map(|entry| entry["id"].as_str().map(|id| (id, entry))).collect::<BTreeMap<_, _>>();-- assert_eq!(capabilities["declared"]["context_window"], json!(1_000_000));- assert_eq!(capabilities["declared"]["tool_calling"], json!(true));- assert_eq!(capabilities["restricted"]["context_window"], json!(262_000));- assert_eq!(capabilities["restricted"]["tool_calling"], json!(false));- assert_eq!(capabilities["undeclared"]["context_window"], json!(null));- assert_eq!(capabilities["undeclared"]["tool_calling"], json!(null));-- assert_eq!(capabilities["declared"]["vision"], json!(true));- assert_eq!(capabilities["restricted"]["vision"], json!(false));- assert_eq!(capabilities["undeclared"]["vision"], json!(null));+ let capabilities = |id: &str| &entries[id]["capabilities"];++ assert_eq!(entries["declared"]["context_length"], json!(1_000_000));+ assert_eq!(capabilities("declared")["tool_calling"], json!(true));+ assert_eq!(entries["restricted"]["context_length"], json!(262_000));+ assert_eq!(capabilities("restricted")["tool_calling"], json!(false));+ assert_eq!(entries["undeclared"]["context_length"], json!(null));+ assert_eq!(capabilities("undeclared")["tool_calling"], json!(null));++ assert_eq!(capabilities("declared")["vision"], json!(true));+ assert_eq!(capabilities("restricted")["vision"], json!(false));+ assert_eq!(capabilities("undeclared")["vision"], json!(null));assert_eq!(body["models"], json!([]));Ok(())diff --git a/docs/core_concepts.md b/docs/core_concepts.mdindex 9688a501d..e91bcb80b 100644--- a/docs/core_concepts.md+++ b/docs/core_concepts.md@@ -62,8 +62,10 @@ inside the TOML file. Their `id` fields have different external meanings:The server lists route IDs on `GET /v1/models`. A request selects a route byputting that ID in its `model` field. The native Rust server does not discover-or register additional provider models automatically. The response includes an empty-Codex `models` array so Codex keeps its own catalog and instructions. Select route+or register additional provider models automatically. Each entry reports the route's+declared `context_window` as its top-level `context_length` field (see+[model discovery](../crates/switchyard-server/README.md#model-discovery)). The+response includes an empty Codex `models` array so Codex keeps its own catalog and instructions. Select routealiases explicitly; they do not appear automatically in Codex's model picker.## Routing Algorithmsdiff --git a/docs/index.md b/docs/index.mdindex a3c480ece..b477c1eeb 100644--- a/docs/index.md+++ b/docs/index.md@@ -10,6 +10,8 @@ It supports OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages.| Run Switchyard as a standalone proxy for API clients | Server Path | [Build and run the Rust server](getting_started.md#server-path) || Add Switchyard routing to a Rust application | Library Path | [`switchyard-libsy`](../crates/libsy/README.md) || Add Switchyard routing to NeMo Relay | Native Plugin Path | [Use Switchyard with NeMo Relay](integrations/nemo_relay.md) |+| Point the pi coding agent at the standalone proxy | Server Path | [Use Switchyard with pi](integrations/pi.md) |+| Point Oh My Pi at the standalone proxy | Server Path | [Use Switchyard with Oh My Pi](integrations/oh_my_pi.md) |The Server Path builds and runs the standalone `switchyard-server` binary.diff --git a/docs/integrations/oh_my_pi.md b/docs/integrations/oh_my_pi.mdnew file mode 100644index 000000000..0e0d9f89d--- /dev/null+++ b/docs/integrations/oh_my_pi.md@@ -0,0 +1,96 @@+# Use Switchyard with Oh My Pi++[Oh My Pi](https://github.com/can1357/oh-my-pi) (`omp`) is a coding agent based on pi.+It reads its model providers from `~/.omp/agent/models.yml`. Add Switchyard there and+`omp` sends every model call to `switchyard-server`. `omp` has no `--base-url` flag, and+setting `OPENAI_BASE_URL` does not change the address of its built-in `openai` provider.+This page was tested with Oh My Pi 18.1.21 against the+[Getting Started](../getting_started.md#server-path) server on `http://localhost:4000`+with route id `switchyard`.++## Configure++`~/.omp/agent/models.yml`:++```yaml+providers:+ switchyard:+ baseUrl: http://localhost:4000/v1+ api: openai-completions+ auth: none+ models:+ - id: switchyard+ name: Switchyard stage router+ reasoning: true+ input: [text, image]+ contextWindow: 200000+ maxTokens: 32000+```++- `auth: none` marks the provider as keyless. Switchyard ignores client keys unless an+ LLM client sets `forward_auth = true`. Without `auth: none`, `omp` refuses to send a+ request.+- `models[].id` must equal a route `id` from your TOML file. `contextWindow` and+ `maxTokens` set `omp`'s compaction limit and output cap. `reasoning: true` turns on the+ `--thinking` flag.+- `omp models find switchyard` shows the loaded entry.++### Or let omp list the routes++```yaml+providers:+ switchyard:+ baseUrl: http://localhost:4000/v1+ api: openai-completions+ auth: none+ discovery:+ type: openai-models-list+```++`omp` reads `GET /v1/models` and adds one model per route. For routes that declare+`context_window`, `omp` takes the context window from the `context_length` field that+the server reports. Other routes get `omp`'s default of 128K tokens. Discovered models+have no thinking controls and accept text only. List the models by hand when you need+`--thinking` or image input. `omp` caches the list for a day, so run `omp models refresh`+after you change the routes.++## Run++```bash+omp --model switchyard/switchyard+omp -p --model switchyard/switchyard "List the files in this directory."+```++`--thinking off|minimal|low|medium|high|xhigh` sets the reasoning level. Setting+`modelRoles.default: switchyard/switchyard` in `~/.omp/agent/config.yml` makes the route+the default model.++## Check the routing++The checks in [Use Switchyard with pi](pi.md#check-the-routing) work the same way for+`omp`. On the Chat Completions API, `omp` sends no session header for a custom provider,+so the routing log records `"session_id": null`. Routes with+`classify_trigger = "user_turn"` or `"new_session"`, advisor budgets, and the stage+router's `capable_hold_turns` then treat each request as its own session. If you need+per-session routing, use `anthropic-messages`. On that API `omp` sends the+`X-Claude-Code-Session-Id` header, which Switchyard reads as the session id.++## Which request API++| `api` | Endpoint | Use it when |+|---|---|---|+| `openai-completions` | `/v1/chat/completions` | Default. The targets use `format = "openai_chat"`, for example OpenRouter. |+| `openai-responses` | `/v1/responses` | The targets use `format = "openai_responses"`. `omp` sends `store: false` and the full history every turn. |+| `anthropic-messages` | `/v1/messages` | You need per-session routing. Set `baseUrl: http://localhost:4000` and keep `auth: none`. |++When the route's LLM client uses the same format as the request, Switchyard forwards the+body unchanged except for `model`. When the client uses another format, Switchyard+translates the request. Unlike pi, `omp` keeps the local model id `switchyard` on stored+messages, so the served target's name in the response does not affect its thinking+replay or compaction.++Port 4000 is also the default port for `omp`'s `litellm` provider and for+`omp auth-gateway`. If `LITELLM_API_KEY` is set, `omp` probes `http://localhost:4000/v1`+as a LiteLLM proxy. In that case, run Switchyard on another port or set+`LITELLM_BASE_URL`. Set `cost` on the model entry if you want `omp` to show a non-zero+cost.diff --git a/docs/integrations/pi.md b/docs/integrations/pi.mdnew file mode 100644index 000000000..1560d14a2--- /dev/null+++ b/docs/integrations/pi.md@@ -0,0 +1,98 @@+# Use Switchyard with pi++[pi](https://github.com/earendil-works/pi) reads its model providers from+`~/.pi/agent/models.json`. Add Switchyard there and pi sends every model call to+`switchyard-server`, which picks the target model for each call. pi does not read+`OPENAI_BASE_URL`, so a provider entry is how you point pi at a proxy. This+page was tested with pi 0.84.3 against the+[Getting Started](../getting_started.md#server-path) server on `http://localhost:4000`+with route id `switchyard`.++## Configure++`~/.pi/agent/models.json`:++```json+{+ "providers": {+ "switchyard": {+ "baseUrl": "http://localhost:4000/v1",+ "api": "openai-completions",+ "apiKey": "switchyard",+ "compat": {+ "supportsDeveloperRole": false,+ "sendSessionAffinityHeaders": true,+ "sessionAffinityFormat": "openrouter"+ },+ "models": [+ {+ "id": "switchyard",+ "name": "Switchyard stage router",+ "reasoning": true,+ "input": ["text", "image"],+ "contextWindow": 200000,+ "maxTokens": 32000+ }+ ]+ }+ }+}+```++- `models[].id` must equal a route `id` from your TOML file. Add one entry per route.+- `apiKey` is a placeholder. Switchyard ignores client keys unless an LLM client sets+ `forward_auth = true`. pi still needs some value here before it lists the model.+- `contextWindow` and `maxTokens` set pi's compaction limit and output cap. pi does not+ read these values from the server. Use the smallest context window among the route's+ targets.+- `reasoning: true` turns on the `--thinking` flag. pi then sends `reasoning_effort`.+- `supportsDeveloperRole: false` keeps the system prompt in the `system` role, which+ every upstream provider accepts.+- `sendSessionAffinityHeaders: true` with `sessionAffinityFormat: "openrouter"` makes pi+ send the `x-session-id` header. Switchyard reads that header as the session id. Routes+ with `classify_trigger = "user_turn"` or `"new_session"`, advisor budgets, the stage+ router's `capable_hold_turns`, and `GET /v1/routing/session-stats` all depend on the+ session id.++## Run++```bash+pi --provider switchyard --model switchyard+pi -p --provider switchyard --model switchyard "List the files in this directory."+```++`--thinking off|minimal|low|medium|high|xhigh` sets the reasoning level that pi sends.++## Check the routing++The command `curl -s localhost:4000/v1/stats | jq '.models | map_values({calls, prompt_tokens})'`+lists each target model with its call count. Every response carries the header+`x-model-router-selected-model`. With `--routing-log-file PATH`, the server writes one+record per call. In the record below, `session_id` is the value of pi's `x-session-id`+header:++```json+{"route_id":"switchyard","algorithm":"stage_router","model":"azure/anthropic/claude-haiku-4-5","session_id":"01a0caac-e421-7328-adaf-79d3440c0406","prompt_tokens":2659,"completion_tokens":97}+```++## Which request API++When the route's LLM client uses the same format as the request, Switchyard forwards the+body unchanged except for `model`. When the client uses another format, Switchyard+translates the request.++| `api` | Endpoint | Use it when |+|---|---|---|+| `openai-completions` | `/v1/chat/completions` | Default. The targets use `format = "openai_chat"`, for example OpenRouter. |+| `openai-responses` | `/v1/responses` | The targets use `format = "openai_responses"`. pi sends `store: false` and the full history every turn. Keep `sessionAffinityFormat: "openrouter"`. |+| `anthropic-messages` | `/v1/messages` | Do not use it with pi. See below. |++Do not use `anthropic-messages` with pi. Switchyard puts the served target's id in the+response `model` field, and pi's Anthropic client stores that id on the assistant+message. When routing picks another target on the next turn, pi treats the change as a+model switch: it drops thinking signatures and turns off overflow compaction. pi's OpenAI+clients keep the local id `switchyard` instead.++Set `cost` on the model entry if you want pi to show a non-zero cost.+[`benchmark/run-baseline.sh`](../../benchmark/README.md) runs Terminal-Bench tasks with+pi through Switchyard when you pass `--agent pi`.diff --git a/docs/reference/toml_schema.md b/docs/reference/toml_schema.mdindex f0c7d5cc5..7fc96b709 100644--- a/docs/reference/toml_schema.md+++ b/docs/reference/toml_schema.md@@ -131,7 +131,7 @@ Every route takes the common keys below, plus the keys for its type.|---|:---:|---|---|| `id` | Yes | — | Public model ID that callers send in requests. || `type` | Yes | — | Routing algorithm for this route. |-| `context_window` | No | unset | Positive token count advertised for this route by `GET /v1/models`. Unset values appear as `null`. This does not enforce a request limit. |+| `context_window` | No | unset | Positive token count that `GET /v1/models` advertises for this route as `data[].context_length`. Unset values appear as `null`. This does not enforce a request limit. || `tool_calling` | No | unset | Tool-calling support advertised by `GET /v1/models`. When `false`, the server rejects tool definitions, tool controls, and tool history with HTTP 400 before dispatch. Unset values appear as `null`. Explicit `true` and unset values do not restrict requests. || `reasoning` | No | unset | When `false`, the server rejects reasoning controls with HTTP 400 before dispatch. Explicit `true` and unset values do not restrict requests. The server does not include this declaration in `GET /v1/models`. || `vision` | No | unset | Image-input support advertised in `GET /v1/models` under `data[].capabilities.vision`. When `false`, the server rejects images with HTTP 400 before dispatch, including images in tool results. Unset values appear as `null`. Explicit `true` and unset values do not restrict requests. Declare `true` only when every target the route can select accepts images. |diff --git a/mkdocs.yml b/mkdocs.ymlindex 797ad68dd..5cd2a986c 100644--- a/mkdocs.yml+++ b/mkdocs.yml@@ -25,6 +25,8 @@ nav:- Architecture: architecture.md- Integrations:- NeMo Relay: integrations/nemo_relay.md+ - pi: integrations/pi.md+ - Oh My Pi: integrations/oh_my_pi.md- Routing:- Overview: routing_algorithms/overview.md- Task (LLM Classifier): routing_algorithms/llm_classifier_routing.mddiff --git a/tests/test_prepare_harbor_dataset.py b/tests/test_prepare_harbor_dataset.pyindex 74dd620bf..9bdafb39d 100644--- a/tests/test_prepare_harbor_dataset.py+++ b/tests/test_prepare_harbor_dataset.py@@ -1,8 +1,6 @@# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.# SPDX-License-Identifier: Apache-2.0-from __future__ import annotations-import importlib.utilimport jsonfrom pathlib import Path@@ -37,7 +35,6 @@ def _write_task(root: Path, name: str, task_toml: str, dockerfile: str | None =def _prepare(tmp_path: Path,source: Path,- *,source_dataset: str = "[email protected]",) -> Path:module = _load_generator_module()@@ -191,6 +188,7 @@ def test_prebuilt_docker_image_task_becomes_derived_dockerfile(tmp_path: Path) -assert "@anthropic-ai/[email protected]" in dockerfileassert "@openai/[email protected]" in dockerfileassert "[email protected]" in dockerfile+ assert "@earendil-works/[email protected]" in dockerfiledef test_dockerfile_only_task_gets_prebake_layer(tmp_path: Path) -> None:@@ -288,6 +286,7 @@ def test_generated_dataset_manifest_records_pins_tasks_and_digests(tmp_path: Pat"HERMES_VERSION": "3c27eb6234bf91b8ceee9e9071591b31e9b148cb","NODE_VERSION": "20.11.1","OPENCODE_VERSION": "1.18.3",+ "PI_VERSION": "0.84.3",}assert manifest["closed_book"]["proxy_asset_digest"].startswith("sha256:")assert manifest["closed_book"]["verifier_egress"] == "open-via-authenticated-proxy"@@ -322,6 +321,7 @@ def test_a_hermes_ref_that_is_not_a_commit_sha_is_rejected() -> None:"CLAUDE_CODE_VERSION": "1","CODEX_VERSION": "2","OPENCODE_VERSION": "3",+ "PI_VERSION": "5","NODE_VERSION": "4",}rejected = (@@ -346,6 +346,7 @@ def test_the_hermes_installer_is_fetched_at_the_pinned_commit() -> None:"CLAUDE_CODE_VERSION": "1","CODEX_VERSION": "2","OPENCODE_VERSION": "3",+ "PI_VERSION": "5","NODE_VERSION": "4","HERMES_VERSION": sha,}@@ -369,6 +370,7 @@ def test_the_hermes_pin_is_applied_by_commit_and_forced() -> None:"CLAUDE_CODE_VERSION": "1","CODEX_VERSION": "2","OPENCODE_VERSION": "3",+ "PI_VERSION": "5","NODE_VERSION": "4","HERMES_VERSION": sha,}@@ -386,6 +388,7 @@ def test_the_alpine_branch_installs_the_shell_the_installer_needs() -> None:"CLAUDE_CODE_VERSION": "1","CODEX_VERSION": "2","OPENCODE_VERSION": "3",+ "PI_VERSION": "5","NODE_VERSION": "4","HERMES_VERSION": "3c27eb6234bf91b8ceee9e9071591b31e9b148cb",}@@ -400,7 +403,7 @@ def test_a_missing_hermes_pin_is_reported_with_the_other_pins(tmp_path: Path) ->module = _load_generator_module()versions = tmp_path / "agent-versions.env"versions.write_text(- "CLAUDE_CODE_VERSION=1\nCODEX_VERSION=2\nOPENCODE_VERSION=3\nNODE_VERSION=4\n"+ "CLAUDE_CODE_VERSION=1\nCODEX_VERSION=2\nOPENCODE_VERSION=3\nPI_VERSION=5\nNODE_VERSION=4\n")module.AGENT_VERSIONS_FILE = versionssource = tmp_path / "source"diff --git a/tests/test_run_baseline_script.py b/tests/test_run_baseline_script.pyindex 650fa6bdc..03f796761 100644--- a/tests/test_run_baseline_script.py+++ b/tests/test_run_baseline_script.py@@ -1,8 +1,6 @@# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.# SPDX-License-Identifier: Apache-2.0-from __future__ import annotations-import jsonimport osimport shlex@@ -124,7 +122,7 @@ def _write_fake_harbor_patch(tmp_path: Path) -> Path:return patch_file-def _write_fake_harbor_python(tmp_path: Path, *, patched: bool = True) -> Path:+def _write_fake_harbor_python(tmp_path: Path, patched: bool = True) -> Path:tmp_path.mkdir(parents=True, exist_ok=True)harbor_site = tmp_path / "fake-site-packages" / "harbor"base = harbor_site / "agents" / "installed" / "base.py"@@ -758,6 +756,32 @@ def test_dry_run_opencode_closed_book_disables_webfetch(tmp_path: Path) -> None:assert "OPENCODE_DISABLE_WEBFETCH=1" in _option_values(harbor, "--ae")+def test_dry_run_pi_uses_switchyard_label_pin_and_thinking(tmp_path: Path) -> None:+ profile = _write_server_config(tmp_path / "routes.toml")++ result = _run_baseline(+ tmp_path,+ "--server-config",+ str(profile),+ "--model",+ "tb-lite-random-routing",+ "--agent",+ "pi",+ "--reasoning-effort",+ "high",+ "--dry-run",+ )++ assert result.returncode == 0, result.stderr+ harbor = _line_argv(result.stdout, "HARBOR_CMD: ")+ assert _option_value(harbor, "--model") == "switchyard/tb-lite-random-routing"+ kwargs = _option_values(harbor, "--ak")+ assert "version=0.84.3" in kwargs+ # Harbor's pi agent has a --thinking flag, not reasoning_effort.+ assert "thinking=high" in kwargs+ assert not any(kwarg.startswith("reasoning_effort=") for kwarg in kwargs)++def test_task_list_file_expands_to_include_task_name(tmp_path: Path) -> None:task_list = tmp_path / "tasks.txt"task_list.write_text("# comment\nalpha\n\nbeta # inline\n")