Skip to content

Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.

Rust

chore: getting started cleanup (#218)

Ayush Agarwal74324f194235
diff --git a/.github/workflows/getting-started.yml b/.github/workflows/getting-started.yml
index 748929ec1..6b7511b02 100644
--- a/.github/workflows/getting-started.yml
+++ b/.github/workflows/getting-started.yml
@@ -2,18 +2,20 @@ name: Getting Started
# Executable coverage for docs/getting_started.md.
#
-# Fails when the guide drifts from the product — renamed CLI subcommand,
-# YAML schema change, broken Python snippet, /health regression, etc. The
-# exercise lives in tests/getting_started/ so it reproduces locally with
+# Fails when the documented Rust server flow drifts from the product, including
+# CLI changes, TOML schema changes, and operational endpoint regressions. It lives in
+# tests/getting_started/ so it reproduces locally with
# `uv run pytest tests/getting_started`.
on:
pull_request:
paths:
- "docs/getting_started.md"
- - "switchyard/**"
- - "switchyard_rust/**"
+ - "crates/**"
- "tests/getting_started/**"
+ - "Cargo.toml"
+ - "Cargo.lock"
+ - "rust-toolchain.toml"
- "pyproject.toml"
- "uv.lock"
- ".github/workflows/getting-started.yml"
@@ -21,9 +23,11 @@ on:
branches: [main]
paths:
- "docs/getting_started.md"
- - "switchyard/**"
- - "switchyard_rust/**"
+ - "crates/**"
- "tests/getting_started/**"
+ - "Cargo.toml"
+ - "Cargo.lock"
+ - "rust-toolchain.toml"
- "pyproject.toml"
- "uv.lock"
- ".github/workflows/getting-started.yml"
@@ -49,17 +53,4 @@ jobs:
- name: Install
run: uv sync --locked
- name: Exercise the guide
- # Single invocation: pytest picks up tests/getting_started/ (CLI, YAML,
- # serve lifecycle) AND --markdown-docs execs the Python snippet from
- # the guide. The local mock-upstream fixture lives in
- # tests/getting_started/conftest.py so it only loads when collecting
- # from that subtree.
- run: uv run pytest tests/getting_started --markdown-docs docs/getting_started.md -v
- env:
- # The server-lifecycle test serves a type: model route pointed at a
- # local in-process mock OpenAI upstream; these keys are never sent
- # upstream. Stub so the bundle loader's env-var resolution doesn't
- # fail on a clean runner.
- OPENAI_API_KEY: sk-test
- NVIDIA_API_KEY: nvapi-test
- ANTHROPIC_API_KEY: sk-ant-test
+ run: uv run pytest tests/getting_started -v
diff --git a/docs/getting_started.md b/docs/getting_started.md
index d0b645f3c..2d7a71c9a 100644
--- a/docs/getting_started.md
+++ b/docs/getting_started.md
@@ -2,215 +2,155 @@
## Prerequisites
-- Python 3.12 or later
-- macOS, Linux, or Windows
+- Git, a native build toolchain, and Rust with Cargo
- An API key for OpenRouter, OpenAI, Anthropic, or another OpenAI-compatible endpoint.
To use OpenRouter, create an account at [openrouter.ai](https://openrouter.ai/)
and generate a key from the [OpenRouter keys page](https://openrouter.ai/keys).
-## Install
+On Ubuntu or WSL, install the build prerequisites and Rust with `rustup`:
```bash
-pip install "nemo-switchyard[cli,server]"
+sudo apt-get update
+sudo apt-get install -y build-essential curl git
+curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
+source "$HOME/.cargo/env"
```
-## Configure
+On macOS or native Windows, follow the
+[official Rust installation instructions](https://rust-lang.org/tools/install/).
+The Rust installer includes `rustc`, Cargo, and `rustup`.
-Interactive setup saves your provider credentials and routing bundle to
-`~/.config/switchyard/`. All paths below pick them up automatically at runtime.
+Install `uv` for the repository's Python-based tooling and CI checks. It is not
+required to build or run the Rust server:
```bash
-switchyard configure
+curl -LsSf https://astral.sh/uv/install.sh | sh
```
-Or non-interactively with a routing-profile YAML:
+If either installer updates your shell configuration, restart the shell before
+continuing. Verify the tools:
```bash
-export OPENROUTER_API_KEY="your-openrouter-key" # pragma: allowlist secret
-
-cat > routes.yaml <<'EOF'
-defaults:
- api_key: ${OPENROUTER_API_KEY}
- base_url: https://openrouter.ai/api/v1
- format: openai
-
-routes:
- smart:
- type: random_routing
- strong:
- model: openai/gpt-4o
- weak:
- model: openai/gpt-4o-mini
- strong_probability: 0.3
- fallback_target_on_evict: strong
-EOF
-
-switchyard --routing-profiles routes.yaml -- configure --target provider \
- --provider openrouter --api-key "$OPENROUTER_API_KEY" \
- --base-url https://openrouter.ai/api/v1 --no-tui --no-model-discovery
+git --version
+rustc --version
+cargo --version
+uv --version
```
-`api_key` inside the route bundle is used when Switchyard serves or launches the
-bundle. `configure --no-tui` still requires provider credentials as explicit
-flags, so CI setup should pass `--api-key` even when the YAML contains
-`${OPENROUTER_API_KEY}`.
-
-> **Format default and caching.** Omitting `format:` from a tier silently defaults to `OPENAI` (Chat Completions) — not `AUTO`. For Claude/Anthropic/Bedrock tiers this is wrong: set `format: anthropic` explicitly. The native `/v1/messages` path preserves `cache_control`, which is what enables prompt caching. `format: openai` routes Claude through OpenAI-format translation that strips `cache_control`: the request still succeeds, but caching silently never engages and you pay full input price. Always use `format: openai` for NIM/non-Claude models and `format: anthropic` for Claude and Bedrock models. Use `format: auto` only when the upstream is genuinely unknown.
+## Install
-Inspect what was saved:
+Build the Rust server from source:
```bash
-switchyard configure --show # redacted snapshot
-switchyard configure --show --check # also probes GET /models
+git clone https://github.com/NVIDIA-NeMo/Switchyard.git
+cd Switchyard
+cargo build --locked --release -p switchyard-server
+./target/release/switchyard-server --help
```
----
+The repository pins Rust `1.96.1` in `rust-toolchain.toml`; `rustup` selects and
+installs it automatically when Cargo runs from the repository. Prebuilt Rust
+binaries are not published yet.
-## Path A: Server mode
+## Configure
-Serves the saved routing bundle as a long-running proxy. Any client that speaks
-OpenAI Chat Completions, Anthropic Messages, or OpenAI Responses API can connect.
+The Rust server reads an explicit TOML file. It does not use the legacy Python
+CLI's saved configuration or YAML routing profiles.
-```bash
-switchyard serve
-```
+Create `routes.toml` with an LLM-classifier route:
-Test with curl:
+```toml
+schema_version = 1
-```bash
-curl http://localhost:4000/v1/chat/completions \
- -H "Content-Type: application/json" \
- -d '{"model": "smart", "messages": [{"role": "user", "content": "hello"}]}'
-```
+[llm_clients.openrouter]
+format = "openai_chat"
+base_url = "https://openrouter.ai/api/v1"
+api_key_env = "OPENROUTER_API_KEY"
-**CI pattern:** run configure in your environment setup with explicit provider
-credentials:
-
-```bash
-switchyard --routing-profiles routes.yaml -- configure --target provider \
- --provider openrouter --api-key "$OPENROUTER_API_KEY" \
- --base-url https://openrouter.ai/api/v1 --no-tui --no-model-discovery
-```
+[targets.weak]
+id = "openai/gpt-4o-mini"
+llm_client = "openrouter"
-Then run `switchyard serve` in your service start step. No flags needed at serve
-time.
+[targets.strong]
+id = "openai/gpt-4o"
+llm_client = "openrouter"
-> **Override (dev / one-off work):** pass `--routing-profiles` to use a different
-> bundle for a session without overwriting your saved config:
-> ```bash
-> switchyard --routing-profiles dev.yaml -- serve --port 4001
-> ```
+[routes.smart]
+id = "switchyard"
+type = "llm_classifier"
+classifier_target = "weak"
+strong_target = "strong"
+weak_target = "weak"
+base_threshold = 0.5
+```
----
+`format` selects the upstream protocol and must be `openai_chat`,
+`openai_responses`, or `anthropic_messages`. `api_key_env` names the environment
+variable the server reads; the secret does not belong in the TOML file.
-## Path B: Agent launcher
+## Server mode
-Starts a proxy and spawns a coding agent against it in one command. The proxy
-shuts down when the agent exits. The live stats footer shows per-tier token usage.
+Export the provider credential, validate the configuration without binding a
+socket, then start the release binary:
```bash
-switchyard launch claude # Claude Code
-switchyard launch codex # Codex CLI
-switchyard launch openclaw # OpenClaw
+export OPENROUTER_API_KEY="your-openrouter-key" # pragma: allowlist secret
+./target/release/switchyard-server --config routes.toml --dry-run
+./target/release/switchyard-server --config routes.toml \
+ --host 127.0.0.1 --port 4000
```
-Each launcher reads the routing bundle and provider credentials saved by
-`switchyard configure`. See [Agent Launchers](guides/agent_launchers.md) for
-supported harness versions, model requirements, and Claude Code `/model` picker
-aliasing.
+Any client that speaks OpenAI Chat Completions, Anthropic Messages, or OpenAI
+Responses API can connect. The route `id` is the model name clients use.
-> **Override (dev / one-off work):** pass `--routing-profiles` (global switchyard
-> flag) or `--model` (launcher flag) to use a different bundle or single model for
-> a session without changing your saved config (the two are mutually exclusive):
-> ```bash
-> switchyard --routing-profiles dev.yaml -- launch claude
-> switchyard launch claude --model openai/gpt-4o
-> ```
+In another terminal:
----
+```bash
+curl http://localhost:4000/health
+curl http://localhost:4000/v1/models
+curl http://localhost:4000/v1/chat/completions \
+ -H "Content-Type: application/json" \
+ -d '{"model":"switchyard","messages":[{"role":"user","content":"hello"}]}'
+```
-## Routing profiles
+<!-- TODO: Add agent launchers back when they are wired to the Rust server. -->
-All route types work with both [Path A](#path-a-server-mode) and
-[Path B](#path-b-agent-launcher). Declare a type in your YAML, run the
-non-interactive configure command above, then `serve` or `launch` as above.
+## Routing algorithms
### Choose a route type
-This guide used `random_routing` so you can get a working proxy quickly. Choose
-another route type when the routing decision needs different inputs:
+This guide uses `llm_classifier`, which asks a classifier target whether each
+request should use the weak or strong target. The Rust server also supports:
| Algorithm | Use it when | Config |
|---|---|---|
-| [Random Routing](routing_algorithms/random_routing.md) | You need a fixed strong/weak split for A/B tests or baselines. | `random_routing` |
-| [LLM Classifier Routing](routing_algorithms/llm_classifier_routing.md) | Request content should decide whether to use `weak` or `strong`. | `deterministic` |
-| [Stage-Router Routing](routing_algorithms/stage_router_routing.md) | Tool-result and progress signals should route most turns without an extra classifier call. | `stage_router` |
-
-LLM classifier routes can also enable
-[Session Affinity (Sticky Routing)](routing_algorithms/sticky_routing.md) to pin
-multi-turn conversations to one tier.
-
-A single YAML file can declare multiple routes. Each route becomes a model id on
-`GET /v1/models`; the first declared route is the launcher's initial model. See
-[Routing Overview](routing_algorithms/overview.md) for route selection and the
-strategy-specific pages for full examples and tuning notes.
-
----
-
-## Path C: Python library
-
-Embed Switchyard directly in your application without a separate proxy process:
-
-```python
-import asyncio
-from switchyard import ChatRequest, PassthroughProfileConfig, ProfileSwitchyard
-
-switchyard = ProfileSwitchyard(PassthroughProfileConfig(
- api_key="sk-or-...", # pragma: allowlist secret
- base_url="https://openrouter.ai/api/v1",
-).build())
-
-async def chat(user_message: str) -> str:
- request = ChatRequest.openai_chat({
- "model": "openai/gpt-4o",
- "messages": [{"role": "user", "content": user_message}],
- })
- response = await switchyard.call(request)
- return response["choices"][0]["message"]["content"]
-
-print(asyncio.run(chat("What is 2+2?")))
-```
+| Random | You need a weighted split for A/B tests or baselines. | `random` |
+| LLM classifier | Request content should decide whether to use the weak or strong target. | `llm_classifier` |
+| Stage router | Tool-result and progress signals should select an efficient or capable target. | `stage_router` |
-To host the chain as an HTTP server:
+A single TOML file can declare multiple routes. The table key, such as
+`routes.smart`, is a local configuration name; each route's `id` is exposed as a
+model on `GET /v1/models`.
-```python
-import uvicorn
-from switchyard import PassthroughProfileConfig, ProfileSwitchyard, build_switchyard_app
+See the [`switchyard-server` guide](../crates/switchyard-server/README.md) for
+the complete TOML schema, route options, TLS, and metrics.
-switchyard = ProfileSwitchyard(PassthroughProfileConfig(
- api_key="sk-or-...", # pragma: allowlist secret
- base_url="https://openrouter.ai/api/v1",
-).build())
-uvicorn.run(build_switchyard_app(switchyard), port=4000)
-```
+<!-- TODO: Restore routing-guide links after those pages use the Rust TOML schema. -->
----
+<!-- TODO: Add Python library usage back when the supported Rust-backed API is finalized. -->
## Troubleshooting
**No API key / auth error**
```bash
-switchyard configure # re-run interactive setup to update credentials
-switchyard configure --show # confirm what key source is in use
+test -n "$OPENROUTER_API_KEY" && echo "key is set" || echo "key is missing"
+./target/release/switchyard-server --config routes.toml --dry-run
```
-For launchers and verification, you can pass `--api-key` directly. For `serve`, put credentials in the routing-profile YAML or saved config.
-
-```bash
-switchyard launch claude --api-key sk-...
-switchyard verify --api-key sk-...
-```
+Confirm that `api_key_env` in `routes.toml` names the environment variable you
+exported. The dry run validates the schema, environment lookup, target
+references, and route construction without starting the server.
**Connection refused**
@@ -225,27 +165,19 @@ release attribution. No request or response content is included. To disable:
export SWITCHYARD_TELEMETRY_OPT_OUT=1
```
-**Development setup**
-
-```bash
-git clone https://github.com/NVIDIA-NeMo/Switchyard.git
-cd Switchyard
-uv sync
-source .venv/bin/activate
-uv run pytest tests/ -v
-uv run ruff check .
-uv run mypy switchyard
-```
+<!-- TODO: Add contributor setup back when the documented workflow is Rust-first. -->
---
## Next steps
-- [CLI Reference](cli_reference.md): full flag reference for every verb
-- [Agent Launchers](guides/agent_launchers.md): Claude Code, Codex, and OpenClaw launcher details
-- [Architecture](architecture.md): system context and end-to-end request flow
-- [Routing Overview](routing_algorithms/overview.md): choose the right routing strategy
-- [Random Routing](routing_algorithms/random_routing.md): fixed strong/weak split routing
-- [LLM Classifier Routing](routing_algorithms/llm_classifier_routing.md): classifier-driven strong/weak routing
-- [Stage-Router Routing](routing_algorithms/stage_router_routing.md): picker layers, signal dimensions, calibration
-- [Sticky Routing](routing_algorithms/sticky_routing.md): conversation-level route affinity
+- [`switchyard-server`](../crates/switchyard-server/README.md): server configuration,
+ routing algorithms, TLS, and metrics
+- [`switchyard-libsy`](../crates/libsy/README.md): embed routing algorithms in a
+ Rust application
+- [`switchyard-protocol`](../crates/protocol/README.md): provider-neutral
+ request, response, and streaming types
+- [`switchyard-translation`](../crates/switchyard-translation/README.md):
+ request, response, and stream translation
+
+<!-- TODO: Restore architecture, CLI, launcher, and routing links when those pages are Rust-first. -->
diff --git a/tests/getting_started/conftest.py b/tests/getting_started/conftest.py
index 7a46aaedc..730818b36 100644
--- a/tests/getting_started/conftest.py
+++ b/tests/getting_started/conftest.py
@@ -1,41 +1,7 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
-"""Markdown-docs fixtures for executing the guide's Python snippets safely.
+"""Getting Started test configuration."""
-Points the guide's passthrough profile at a local in-process mock OpenAI
-server (loopback, a fixed ``chat.completion``) instead of a live backend, and
-stubs ``uvicorn.run`` to a no-op — the "host as HTTP server" snippet would
-otherwise block the test session forever. Both are gated on the
-``--markdown-docs`` flag so regular runs are untouched.
-"""
-
-from __future__ import annotations
-
-from collections.abc import Iterator
-
-import pytest
-
-from tests._mock_openai_server import (
- _MockOpenAIServer,
- markdown_docs_active,
- redirect_passthrough_to_local_mock,
-)
-
-
-def local_mock_openai_server() -> Iterator[_MockOpenAIServer]:
- """A running local mock OpenAI upstream for hermetic serve tests."""
- with _MockOpenAIServer() as server:
- yield server
-
-
[email protected](autouse=True, scope="session")
-def _markdown_docs_hermetic_upstream(
- request: pytest.FixtureRequest,
-) -> Iterator[None]:
- if not markdown_docs_active(request.config):
- yield
- return
- with redirect_passthrough_to_local_mock(stub_uvicorn=True):
- yield
+# TODO: Add a local classifier/upstream fixture when the documented Rust request
+# path can be exercised without provider credentials.
diff --git a/tests/getting_started/test_getting_started.py b/tests/getting_started/test_getting_started.py
index 2d5608463..c7d54552b 100644
--- a/tests/getting_started/test_getting_started.py
+++ b/tests/getting_started/test_getting_started.py
@@ -5,54 +5,43 @@
from __future__ import annotations
-import argparse
-import subprocess
-import sys
-import textwrap
-from collections.abc import Iterator
-from contextlib import contextmanager
+import asyncio
+import os
+from collections.abc import AsyncIterator
+from contextlib import asynccontextmanager
from pathlib import Path
-from typing import TYPE_CHECKING
import httpx
import pytest
-import yaml as pyyaml
from markdown_it import MarkdownIt
-from switchyard.cli.launchers.launcher_runtime import (
- find_free_port,
- wait_for_proxy_ready,
-)
-from switchyard.cli.route_bundle import RouteBundleConfigError, build_route_bundle_table
-from switchyard.cli.switchyard_cli import _build_parser
-
-if TYPE_CHECKING:
- from tests._mock_openai_server import _MockOpenAIServer
-
REPO_ROOT = Path(__file__).resolve().parents[2]
GUIDE_PATH = REPO_ROOT / "docs" / "getting_started.md"
-
-#: How long to wait for ``switchyard serve`` to become ready on /health.
-STARTUP_TIMEOUT_S: float = 30.0
-
-#: HTTP read timeout for in-test requests to the local proxy.
-REQUEST_TIMEOUT_S: float = 10.0
-
-#: Grace period given to ``switchyard serve`` after SIGTERM before SIGKILL.
-TEARDOWN_GRACE_S: float = 10.0
-
-#: Final wait after SIGKILL so zombies are reaped before the test ends.
-KILL_REAP_S: float = 5.0
+STARTUP_TIMEOUT_S = 30.0
+REQUEST_TIMEOUT_S = 1.0
+TEARDOWN_GRACE_S = 10.0
@pytest.fixture(scope="module")
def guide_text() -> str:
+ """Return the published Getting Started source."""
return GUIDE_PATH.read_text()
[email protected](scope="session")
+async def rust_server_binary() -> Path:
+ """Build and return the Rust server binary exercised by the guide."""
+ returncode, stdout, stderr = await _run_command(
+ ["cargo", "build", "--quiet", "--locked", "-p", "switchyard-server"],
+ )
+ assert returncode == 0, f"failed to build switchyard-server:\n{stderr}{stdout}"
+ binary = REPO_ROOT / "target" / "debug" / "switchyard-server"
+ assert binary.is_file(), f"cargo did not create {binary}"
+ return binary
+
+
def _code_blocks(text: str, lang: str) -> list[str]:
- # markdown-it-py handles indented fences + trailing whitespace correctly,
- # which a naive ```lang ... ``` regex does not.
+ """Extract fenced Markdown blocks for one language."""
md = MarkdownIt()
return [
token.content
@@ -61,209 +50,211 @@ def _code_blocks(text: str, lang: str) -> list[str]:
]
-def _route_bundle_blocks(text: str) -> list[str]:
- blocks = _code_blocks(text, "yaml")
- for block in _code_blocks(text, "bash"):
- lines = block.splitlines()
- for start, line in enumerate(lines):
- if line.startswith("cat > ") and "<<'EOF'" in line:
- for end in range(start + 1, len(lines)):
- if lines[end] == "EOF":
- blocks.append("\n".join(lines[start + 1:end]))
- break
- return blocks
-
-
- "needle",
- [
- # `switchyard launch claude` is not exercised live — needs the Claude
- # Code binary, which CI doesn't install — so the only check that the
- # guide still names these invocations is this string-match.
- "switchyard launch claude",
- "switchyard --routing-profiles dev.yaml -- launch claude",
- "switchyard launch claude --model openai/gpt-4o",
- ],
-)
-def test_guide_still_references_launch_claude_invocations(needle: str, guide_text: str) -> None:
- assert needle in guide_text, (
- f"Guide no longer mentions {needle!r}; if intentional, update this test."
- )
+async def test_rust_server_toml_blocks_validate_with_rust_schema(
+ guide_text: str,
+ rust_server_binary: Path,
+ tmp_path: Path,
+) -> None:
+ """Validate every documented server config with the production Rust loader."""
+ configs = [
+ block
+ for block in _code_blocks(guide_text, "toml")
+ if "schema_version" in block
+ and "[llm_clients." in block
+ and "[targets." in block
+ and "[routes." in block
+ ]
+ assert configs, "no Rust server TOML config found in the guide"
+
+ env = os.environ.copy()
+ env["OPENROUTER_API_KEY"] = "getting-started-test-key"
+ for index, config in enumerate(configs):
+ config_path = tmp_path / f"getting-started-{index}.toml"
+ config_path.write_text(config)
+ returncode, stdout, stderr = await _run_command(
+ [rust_server_binary, "--config", config_path, "--dry-run"],
+ env=env,
+ )
+ assert returncode == 0, (
+ f"TOML block {index} failed Rust schema validation:\n{stderr}{stdout}"
+ )
+ assert "server OK: switchyard" in stdout
-def _subparsers(parser: argparse.ArgumentParser) -> dict[str, argparse.ArgumentParser]:
- action = next(a for a in parser._actions if isinstance(a, argparse._SubParsersAction))
- return action.choices # type: ignore[return-value]
+async def test_rust_server_help_advertises_documented_flags(
+ rust_server_binary: Path,
+) -> None:
+ """Keep the guide's server flags aligned with the Rust CLI."""
+ returncode, stdout, stderr = await _run_command(
+ [rust_server_binary, "--help"],
+ )
+ assert returncode == 0, stderr
+ for flag in ("--config", "--dry-run", "--host", "--port"):
+ assert flag in stdout, f"documented flag {flag} is missing from --help"
-def test_cli_parser_exposes_every_subcommand_the_guide_names() -> None:
- parser = _build_parser()
- subs = _subparsers(parser)
- # Only check subcommands the guide actually invokes — `configure` is not in
- # the guide, so excluding it keeps the test honest about what it covers.
- for cmd in ("serve", "launch"):
- assert cmd in subs, f"top-level `switchyard {cmd}` is documented but missing"
- assert subs[cmd].format_help().strip()
+def test_guide_uses_the_rust_server_flow(guide_text: str) -> None:
+ """Prevent legacy Python install, configure, and serve commands from returning."""
+ assert "cargo build --locked --release -p switchyard-server" in guide_text
+ assert "./target/release/switchyard-server --config routes.toml" in guide_text
+ for legacy_command in (
+ 'pip install "nemo-switchyard',
+ "switchyard configure",
+ "switchyard serve",
+ "--routing-profiles",
+ "routes.yaml",
+ ):
+ assert legacy_command not in guide_text
- launch_subs = _subparsers(subs["launch"])
- assert "claude" in launch_subs, "`switchyard launch claude` is documented but missing"
- assert launch_subs["claude"].format_help().strip()
+def noop_config(tmp_path: Path) -> Path:
+ """Create a network-free Rust route for exercising the documented endpoints."""
+ config_path = tmp_path / "routes.toml"
+ config_path.write_text(
+ """\
+schema_version = 1
-def test_cli_parser_advertises_documented_flags() -> None:
- parser = _build_parser()
- subs = _subparsers(parser)
- # --routing-profiles is a global switchyard flag, not a subcommand flag
- assert "--routing-profiles" in parser.format_help()
- claude_help = _subparsers(subs["launch"])["claude"].format_help()
- assert "--base-url" in claude_help, "`launch claude --base-url` documented but missing"
+[llm_clients]
+[targets]
-def test_switchyard_cli_entrypoint_script_is_wired_up() -> None:
- # The in-process parser tests miss broken console-scripts wiring; one
- # subprocess via `python -m` is hermetic (no PATH dependency) and catches
- # that.
- completed = subprocess.run(
- [sys.executable, "-m", "switchyard.cli.switchyard_cli", "--help"],
- capture_output=True, text=True, timeout=STARTUP_TIMEOUT_S, check=False,
- )
- assert completed.returncode == 0, (
- f"switchyard CLI failed to run:\nstdout:\n{completed.stdout}\nstderr:\n{completed.stderr}"
+[routes.health_check]
+id = "switchyard/test"
+type = "noop"
+"""
)
- assert "Switchyard" in completed.stdout
+ return config_path
-def test_all_yaml_blocks_in_guide_validate_as_route_bundles(
- guide_text: str, monkeypatch: pytest.MonkeyPatch,
+async def test_rust_server_health_and_models(
+ rust_server_binary: Path,
+ noop_config: Path,
) -> None:
- # Stub the env vars the YAML examples reference so ``${VAR}`` interpolation
- # works on a clean CI runner.
- for key, value in {
- "OPENROUTER_API_KEY": "sk-or-test", # pragma: allowlist secret
- "OPENAI_API_KEY": "sk-test",
- "ANTHROPIC_API_KEY": "sk-ant-test",
- }.items():
- monkeypatch.setenv(key, value)
-
- blocks = _route_bundle_blocks(guide_text)
- assert blocks, "Guide unexpectedly has no route-bundle examples"
-
- for idx, block in enumerate(blocks):
- payload = pyyaml.safe_load(block)
- if not isinstance(payload, dict) or "routes" not in payload:
- continue
- try:
- build_route_bundle_table(payload)
- except RouteBundleConfigError as exc:
- raise AssertionError(
- f"YAML block {idx} in getting_started.md failed to parse "
- f"as a route bundle: {exc}\n\nBlock:\n{block}"
- ) from exc
-
-
-def model_routes_yaml(tmp_path: Path, local_mock_openai_server: _MockOpenAIServer) -> Path:
- # Same shape as the guide's Step 3 YAML (defaults + a single named route),
- # but a `type: model` route pointed at a local in-process mock OpenAI
- # server so the lifecycle test serves a real chain fully offline.
- base_url = local_mock_openai_server.base_url
- path = tmp_path / "routes.yaml"
- path.write_text(
- textwrap.dedent(f"""\
- defaults:
- api_key: dummy
- base_url: {base_url}
- format: openai
-
- routes:
- gpt-4o:
- type: model
- model: gpt-4o
- """)
+ """Start the Rust server and exercise the guide's operational endpoints."""
+ async with _serve_in_background(rust_server_binary, noop_config) as client:
+ health = await client.get("/health")
+ assert health.status_code == 200
+
+ models = await client.get("/v1/models")
+ assert models.status_code == 200
+ assert any(model["id"] == "switchyard/test" for model in models.json()["data"])
+
+
+async def _run_command(
+ command: list[str | os.PathLike[str]],
+ *,
+ env: dict[str, str] | None = None,
+) -> tuple[int, str, str]:
+ """Run a command without blocking the active test event loop."""
+ process = await asyncio.create_subprocess_exec(
+ *command,
+ cwd=REPO_ROOT,
+ env=env,
+ stdout=asyncio.subprocess.PIPE,
+ stderr=asyncio.subprocess.PIPE,
+ )
+ stdout, stderr = await process.communicate()
+ if process.returncode is None:
+ raise RuntimeError(f"command did not exit: {command}")
+ return (
+ process.returncode,
+ stdout.decode(errors="replace"),
+ stderr.decode(errors="replace"),
)
- return path
-
-def test_step3_and_step4_serve_lifecycle_with_local_mock(model_routes_yaml: Path) -> None:
- port = find_free_port()
- with _serve_in_background(model_routes_yaml, port):
- health_status, health_body = _http_get(f"http://127.0.0.1:{port}/health")
- assert health_status == 200, f"GET /health → {health_status}: {health_body!r}"
- status, body = _http_post_json(
- f"http://127.0.0.1:{port}/v1/chat/completions",
- {
- "model": "gpt-4o",
- "messages": [{"role": "user", "content": "What is 2+2?"}],
- },
- )
- assert status == 200, (
- f"POST /v1/chat/completions → {status}: {body!r}"
- )
- assert b'"choices"' in body, f"Response missing 'choices': {body!r}"
+async def _read_server_base_url(process: asyncio.subprocess.Process) -> str:
+ """Read the ephemeral address from the Rust server startup banner."""
+ if process.stdout is None:
+ raise RuntimeError("switchyard-server stdout was not captured")
+ deadline = asyncio.get_running_loop().time() + STARTUP_TIMEOUT_S
+ output: list[str] = []
+ while True:
+ remaining = deadline - asyncio.get_running_loop().time()
+ if remaining <= 0:
+ raise TimeoutError(
+ "switchyard-server did not report its bound address:\n" + "".join(output)
+ )
+ try:
+ raw_line = await asyncio.wait_for(process.stdout.readline(), timeout=remaining)
+ except TimeoutError as error:
+ raise TimeoutError(
+ "switchyard-server did not report its bound address:\n" + "".join(output)
+ ) from error
+ if not raw_line:
+ await process.wait()
+ raise RuntimeError(
+ f"switchyard-server exited early with {process.returncode}:\n"
+ + "".join(output)
+ )
-# Snippet execution lives in pytest-markdown-docs; this tripwire guards the
-# shape the conftest's fixture depends on.
+ line = raw_line.decode(errors="replace")
+ output.append(line)
+ if line.startswith(" listening: "):
+ return line.removeprefix(" listening: ").strip()
-def test_python_snippet_tripwire(guide_text: str) -> None:
- assert (
- "from switchyard import ChatRequest, PassthroughProfileConfig, ProfileSwitchyard"
- in guide_text
- ), (
- "Python snippet's imports moved — update conftest.py."
- )
- assert "ProfileSwitchyard(PassthroughProfileConfig(" in guide_text, (
- "Python snippet no longer builds a passthrough profile — update the "
- "markdown-docs fixture in conftest.py."
+async def _wait_for_proxy_ready(
+ process: asyncio.subprocess.Process,
+ client: httpx.AsyncClient,
+) -> None:
+ """Wait until the server accepts requests or exits."""
+ deadline = asyncio.get_running_loop().time() + STARTUP_TIMEOUT_S
+ while asyncio.get_running_loop().time() < deadline:
+ if process.returncode is not None:
+ raise RuntimeError(
+ f"switchyard-server exited early with {process.returncode}"
+ )
+ try:
+ response = await client.get("/health")
+ if response.status_code == 200:
+ return
+ except httpx.HTTPError:
+ pass
+ await asyncio.sleep(0.1)
+ raise TimeoutError(
+ f"switchyard-server did not become ready within {STARTUP_TIMEOUT_S}s"
)
-@contextmanager
-def _serve_in_background(routes_yaml: Path, port: int) -> Iterator[subprocess.Popen[bytes]]:
- proc = subprocess.Popen(
- [
- sys.executable, "-m", "switchyard.cli.switchyard_cli",
- "--routing-profiles", str(routes_yaml), "--", "serve",
- "--port", str(port),
- ],
- stdout=subprocess.PIPE,
- stderr=subprocess.STDOUT,
+@asynccontextmanager
+async def _serve_in_background(
+ binary: Path,
+ config_path: Path,
+) -> AsyncIterator[httpx.AsyncClient]:
+ """Run the Rust server until the lifecycle assertion completes."""
+ process = await asyncio.create_subprocess_exec(
+ binary,
+ "--config",
+ config_path,
+ "--host",
+ "127.0.0.1",
+ "--port",
+ "0",
+ cwd=REPO_ROOT,
+ stdout=asyncio.subprocess.PIPE,
+ stderr=asyncio.subprocess.STDOUT,
)
try:
- if not wait_for_proxy_ready(port, timeout_s=STARTUP_TIMEOUT_S):
- # Distinguish "exited early" from "still starting" so failures
- # point at the right thing.
- if proc.poll() is not None:
- stdout = proc.stdout.read().decode(errors="replace") if proc.stdout else ""
- raise RuntimeError(
- f"`switchyard serve` exited early (code={proc.returncode}):\n{stdout}"
- )
- raise TimeoutError(
- f"`switchyard serve` not ready on port {port} within {STARTUP_TIMEOUT_S}s"
- )
- yield proc
+ base_url = await _read_server_base_url(process)
+ async with httpx.AsyncClient(
+ base_url=base_url,
+ timeout=REQUEST_TIMEOUT_S,
+ ) as client:
+ await _wait_for_proxy_ready(process, client)
+ yield client
finally:
- proc.terminate()
+ if process.returncode is None:
+ process.terminate()
try:
- proc.wait(timeout=TEARDOWN_GRACE_S)
- except subprocess.TimeoutExpired:
- proc.kill()
- proc.wait(timeout=KILL_REAP_S)
-
+ await asyncio.wait_for(process.wait(), timeout=TEARDOWN_GRACE_S)
+ except TimeoutError:
+ process.kill()
+ await asyncio.wait_for(process.wait(), timeout=TEARDOWN_GRACE_S)
-def _http_get(url: str) -> tuple[int, bytes]:
- try:
- response = httpx.get(url, timeout=REQUEST_TIMEOUT_S)
- except httpx.HTTPError as exc:
- return 0, str(exc).encode()
- return response.status_code, response.content
-
-def _http_post_json(url: str, payload: dict[str, object]) -> tuple[int, bytes]:
- try:
- response = httpx.post(url, json=payload, timeout=STARTUP_TIMEOUT_S)
- except httpx.HTTPError as exc:
- return 0, str(exc).encode()
- return response.status_code, response.content
+# TODO: Add launcher coverage back when launchers are wired to the Rust server.
+# TODO: Add Python snippet coverage back with the supported Rust-backed API.
+# TODO: Add YAML route-bundle coverage back only if YAML becomes a supported Rust format.