Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
refactor(python): remove deprecated server stack (#343)
nachiketb-nvidiab10476593c56
diff --git a/.agents/skills/switchyard-stage-router-scorer/SKILL.md b/.agents/skills/switchyard-stage-router-scorer/SKILL.mddeleted file mode 100644index b11cea0e8..000000000--- a/.agents/skills/switchyard-stage-router-scorer/SKILL.md+++ /dev/null@@ -1,97 +0,0 @@-----name: switchyard-stage-router-scorer-description: Score benchmark run trajectories through the stage-router Rust scorer and picker, then visualise score distributions. Use when you want to replay trajectories through the picker, analyse routing splits, or compare score distributions across configs.------# Skill: switchyard-stage-router-scorer--## Scripts--| Script | Purpose | Input | Output |-|--------|---------|-------|--------|-| `benchmark/score_staged_run.py` | Score a live run dir via real picker | run dir path | per-turn JSONL + per-task CSV |--## Quick Reference--```bash-# Score a run-uv run python benchmark/score_staged_run.py --run benchmark/tb_runs/<run-name>-# → /tmp/<run-name>-scores.jsonl (per turn)-# → /tmp/<run-name>-per-task.csv (per task)--# Custom threshold or window-uv run python benchmark/score_staged_run.py \- --run benchmark/tb_runs/<run-name> \- --threshold 0.15 --window 3-```--## Scoring pipeline (what score_staged_run.py does per turn)--```-trajectory step (tool_use + tool_result)- → append to cumulative Anthropic messages list- → ChatRequest.anthropic({"model": ..., "messages": messages}) # Rust binding- → dc.process(ctx, request) # DimensionCollector — one per task, accumulates state- → get_tool_result_signal(ctx) # read signal from ctx- → stage_score_signal(signal) # raw (score, confidence) from the Rust scorer (for analysis)- → stage_pick_tier(signal, "capable_first", threshold) # actual cf decision- → stage_pick_tier(signal, "efficient_first", threshold) # actual ef decision-```--**Key:** `dc.process()` is called **once per turn** on a single context. Both picker modes use the-same extracted signal, with no duplicate processing.--**What the picker does beyond raw score:**-- **escalate** (`should_escalate`): `compacted` OR `severity >= 1.0` → force CAPABLE-- **de-escalate** (`should_deescalate`): `tests_passed AND recent_write+edit >= 1 AND severity <= 0` → force EFFICIENT-- `confidence < threshold` → fall_open to default tier (CAPABLE for cf, EFFICIENT for ef)-- Only when `confidence >= threshold`: route by score direction--## Per-turn JSONL schema--```json-{- "task_name": "terminal-bench/...", "trial_name": "...", "run_id": "...",- "reward": 1.0, "turn_depth": 5,- "score": -0.83, "confidence": 0.95,- "tool_name": "Bash", "is_error": false,- "write_count": 2, "edit_count": 1, "read_count": 3,- "no_error_streak": 4, "pure_bash_streak": 2, "tests_passed": false,- "pick_cf": 1,- "pick_ef": 0-}-```--`pick_cf` / `pick_ef`: `1` = CAPABLE (Opus), `0` = EFFICIENT (Nemotron)--## Per-task CSV columns--`run_id, task_name, reward, n_turns, mean_score, mean_confidence,-pct_strong_clear, pct_strong_uncertain, pct_weak_uncertain, pct_weak_clear,-opus_pct_cf, nemotron_pct_cf, opus_pct_ef, nemotron_pct_ef`--`opus_pct_cf` / `opus_pct_ef` are derived from actual `pick_cf` / `pick_ef` decisions, not score bands.--## Band definitions (for histogram colouring, threshold T)--| Band | Score range | cf default | ef default |-|------|-------------|------------|------------|-| strong_clear | ≥ T | Opus | Opus |-| strong_uncertain | (0, T) | Opus (fall_open) | Nemotron (fall_open) |-| weak_uncertain | (-T, 0) | Opus (fall_open) | Nemotron (fall_open) |-| weak_clear | ≤ -T | Nemotron | Nemotron |--Overrides can change any band. Use `pick_cf`/`pick_ef` for the true decision.--## Key findings (v0.2.0 baseline tarballs, T=0.20)--- Distribution is **highly bimodal** — most turns land in strong_clear or weak_clear-- `cf, t=0.20`: ~34% Opus on partial run; ~42% on full baseline-- `ef, t=0.20`: ~20% Opus on partial run; ~41% on full baseline-- Live routing split differs from shadow score — Nemotron trajectories are shorter and reshape the distribution--## Anti-patterns--- Don't call `dc.process()` multiple times per turn for different picker modes. Extract one signal per turn and reuse it.-- Don't infer routing split from score bands alone — overrides and fall_open change the actual decision. Always use `pick_cf` / `pick_ef`.-- Don't compare cost directly across configs: Nemotron has ~39% cache hit rate vs ~92% for Opus.diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.mdindex bcbb7eed7..26283f7d7 100644--- a/.github/ISSUE_TEMPLATE/bug_report.md+++ b/.github/ISSUE_TEMPLATE/bug_report.md@@ -16,7 +16,7 @@ Minimal steps to reproduce. Include the command line, the inbound request shape,```bash# example-switchyard serve --routes examples/route.yaml --port 4000+switchyard-server --config routes.toml --port 4000curl -s http://localhost:4000/v1/chat/completions -d '{"model":"...","messages":[...]}'```diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex 67e3ae3b9..9d46b6ab3 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -168,7 +168,7 @@ jobs:runs-on: ubuntu-latestenv:SWITCHYARD_DEFAULT_PACKAGE: "nemo-switchyard @ file://${{ github.workspace }}"- SWITCHYARD_EXTRAS_PACKAGE: "nemo-switchyard[cli,server] @ file://${{ github.workspace }}"+ SWITCHYARD_EXTRAS_PACKAGE: "nemo-switchyard[cli] @ file://${{ github.workspace }}"steps:- uses: actions/checkout@v4- uses: astral-sh/setup-uv@v6@@ -213,7 +213,7 @@ jobs:sys.exit(f"FAIL: heavy packages pulled into slim install: {extras}")print(f"OK: all {len(forbidden)} heavy packages absent from slim install.")PY- - name: Verify CLI and server imports work with extras+ - name: Verify CLI works with its extraworking-directory: /tmprun: |uv run --isolated --no-project --python 3.12 \@@ -225,9 +225,6 @@ jobs:uv run --isolated --no-project --python 3.12 \--with "${SWITCHYARD_EXTRAS_PACKAGE}" \switchyard launch claude --help 2>&1 | grep -q -- '--config' || { echo 'FAIL: --config flag missing from help'; exit 1; }- uv run --isolated --no-project --python 3.12 \- --with "${SWITCHYARD_EXTRAS_PACKAGE}" \- switchyard serve --help > /dev/null# Single required check for branch protection. Gates on every job above so# the ruleset never needs editing when jobs are added or renamed.diff --git a/.github/workflows/perf.yml b/.github/workflows/perf.ymlindex cd797d567..32f1209da 100644--- a/.github/workflows/perf.yml+++ b/.github/workflows/perf.yml@@ -55,8 +55,8 @@ jobs:run: uv pip install aiperf# ------------------------------------------------------------------- # Start a local zero-latency mock OpenAI upstream. This replaces the- # The proxy serves a real `type: passthrough` chain that+ # Start a local zero-latency mock OpenAI upstream. The proxy serves a+ # real `type: passthrough` route that# forwards to this loopback stub, which returns a fixed completion# instantly. The extra loopback hop adds a small constant overhead.# ------------------------------------------------------------------@@ -127,21 +127,30 @@ jobs:echo "STUB_PID=$!" >> "$GITHUB_ENV"# ------------------------------------------------------------------- # Start the proxy: a `type: passthrough` route pointed at the local stub.+ # Build and start the native proxy against the local stub.# ------------------------------------------------------------------+ - name: Build switchyard server+ run: cargo build --locked --release -p switchyard-server+- name: Start switchyard proxyrun: |- cat > bench.yaml <<YAML- defaults:- api_key: dummy- base_url: http://localhost:$STUB_PORT/v1- format: openai- routes:- mock-model:- type: passthrough- target: mock-model- YAML- uv run switchyard serve --routes bench.yaml --port $PROXY_PORT &+ cat > bench.toml <<TOML+ schema_version = 1++ [llm_clients.stub]+ format = "openai_chat"+ base_url = "http://localhost:$STUB_PORT/v1"++ [targets.mock]+ id = "mock-model"+ llm_client = "stub"++ [routes.mock]+ id = "mock-model"+ type = "passthrough"+ target = "mock"+ TOML+ target/release/switchyard-server --config bench.toml --port $PROXY_PORT &echo "PROXY_PID=$!" >> "$GITHUB_ENV"- name: Wait for proxy to be readydiff --git a/AGENTS.md b/AGENTS.mdindex cc046923e..463ba90c9 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -130,97 +130,59 @@ testing, and review do not require loading a skill.| `switchyard-coding-agent-launchers` | Claude Code, Codex, or OpenClaw launcher behavior || `switchyard-docs` | Published MkDocs pages, strict builds, previews, and docs CI || `switchyard-rust-review` | Focused review of Rust, PyO3, async, streaming, and crate boundaries |-| `switchyard-stage-router-scorer` | Replaying benchmark trajectories through the stage-router scorer and picker || `switchyard-testing-ci` | Selecting non-obvious validation or diagnosing CI failures |Skills should contain stable operational constraints, not mutable architecture inventories. Read thecurrent source and CI workflows for implementation details.-## Architecture: staged chain+## Architecture-Everything flows through a fixed-shape chain enforced at construction time:+The supported serving path is native Rust:```-request-side component* → LLMBackend → response-side component* → TranslationEngine+HTTP request → switchyard-server → libsy Algorithm → LlmTarget/RoutedLlmClient+ → switchyard-translation → upstream model```-The chain executor is `Switchyard` (`switchyard/lib/switchyard.py`). `LLMBackend` is the shared-Python role class re-exported from `switchyard/lib/roles.py`; native implementations register with-it, and request-side and response-side processors are plain async components with `process(...)`-methods.-Direct Rust bindings for migrated concrete processors/backends are exposed from-`switchyard_rust.components` and implemented under `crates/switchyard-py/src/component_bindings/`.+`switchyard-server` loads explicit TOML deployments and exposes the OpenAI Chat,+OpenAI Responses, and Anthropic Messages APIs. `switchyard-libsy` owns routing+algorithms, `switchyard-protocol` owns provider-neutral request and response+types, and `switchyard-llm-client` performs translated HTTP calls.-| Stage | Binding | Method | Purpose |-|------|-----|--------|---------|-| Request component | Plain Python/Rust object | `async process(ctx, request) -> ChatRequest` | Pre-process (routing, buffering, auth) |-| `LLMBackend` | `switchyard.lib.roles` | `async call(ctx, request) -> ChatResponse` | Make the LLM call. Exactly one per chain. |-| Response component | Plain Python/Rust object | `async process(ctx, response) -> ChatResponse` | Post-process (logging, stats) |-| `TranslationEngine` | `switchyard_rust.translation` | `async translate(ctx, request, response) -> Any` | Convert to client's wire format |+Python is an integration layer. `switchyard launch` hosts the native server for+coding agents, `switchyard.libsy` exposes selected algorithms, and+`switchyard_rust.server` exposes the native server lifecycle through PyO3.## Project Structure```switchyard/-├── __init__.py # Public API — all exports live here-├── lib/ # Core library-│ ├── roles.py # Python LLMBackend re-export and translation aliases-│ ├── switchyard.py # Switchyard — chain executor-│ ├── proxy_context.py # ProxyContext — per-request state carrier-│ ├── route_table.py # RouteTable — model-id dispatch to runnable chains-│ ├── llm_client.py # OpenAILLMClient-│ ├── cost_estimator.py # Token-cost bookkeeping-│ ├── stats_accumulator.py # Stats accumulation helpers-│ ├── request_metadata.py # RequestMetadata-│ ├── chat_response/ # Rust-backed response re-exports + stream adapters-│ │ ├── base.py # ChatResponse, ChatResponseType compatibility re-export-│ │ ├── openai_chat.py # ResponseStream-│ │ ├── openai_responses.py # ResponsesApiStream-│ │ └── anthropic.py # AnthropicResponseStream-│ ├── backends/ # LLMBackend implementations-│ │ ├── openai_llm_backend.py # OpenAiPassthroughBackend-│ │ ├── openai_native_backend.py # OpenAiNativeBackend-│ │ ├── anthropic_native_llm_backend.py # AnthropicNativeBackend-│ │ ├── llm_target.py # LlmTarget, BackendFormat-│ │ ├── multi_llm_backend.py # MultiLlmBackend helpers-│ │ ├── stats_llm_backend.py # StatsLlmBackend-│ │ └── backend_format_resolver.py # BackendFormatResolver-│ ├── processors/ # Request-side / response-side component implementations-│ │ ├── format_translate.py-│ │ ├── stats_request_processor.py-│ │ └── stats_response_processor_accumulator.py-│ ├── endpoints/ # FastAPI endpoint wrappers (require `nemo-switchyard[server]`)-│ │ ├── openai_chat_endpoint.py # OpenAIChatEndpoint-│ │ ├── anthropic_messages_endpoint.py # AnthropicMessagesEndpoint-│ │ ├── responses_endpoint.py # ResponsesEndpoint-│ │ ├── stats_endpoint.py # StatsEndpoint-│ │ ├── sse_helpers.py-│ │ └── base.py+├── __init__.py # Package version├── cli/ # CLI (requires `nemo-switchyard[cli]`)│ ├── switchyard_cli.py # `switchyard` entry point│ ├── launch_command.py # `switchyard launch`-│ ├── command_utils.py│ ├── defaults/ # packaged OpenRouter TOML deployment-│ ├── launchers/ # Claude, Codex, and OpenClaw launchers-│ ├── model_catalog/ # model_discovery-│ └── route_bundle.py # YAML route parsing for `serve`-└── server/ # FastAPI app factory + server utilities- ├── switchyard_app.py # build_switchyard_app()- ├── server_util.py # Shared CLI / server plumbing- └── shell_tui.py # Shell TUI session+│ └── launchers/ # Claude, Codex, and OpenClaw launchers+└── libsy/ # typed Python wrappers for libsy algorithms++switchyard_rust/ # Python facades over the PyO3 extension+crates/libsy/ # routing algorithms and driver+crates/libsy-llm-client/ # translated HTTP LLM client+crates/protocol/ # provider-neutral protocol types+crates/switchyard-server/ # native HTTP server and TOML config+crates/switchyard-translation/ # wire-format codecs+crates/switchyard-py/ # libsy and server PyO3 bindingstests/ # Unit tests (pytest)```## Tech Stack-- **Python 3.12+**, async-first (asyncio)-- **FastAPI + Uvicorn** for HTTP (`nemo-switchyard[server]`), **sse-starlette** for SSE streaming-- **OpenAI SDK** (`openai>=2.30`) — primary client; the translation engine converts all inbound formats to Chat Completions-- **Anthropic SDK** (`anthropic>=0.94`)-- **httpx** for direct HTTP (health polling, Anthropic Messages)+- **Rust 1.96.1**, edition 2024, Tokio, Axum, and PyO3+- **Python 3.12+** for launchers and native bindings+- **prompt-toolkit** for interactive launcher sessions- **uv** as the package manager (preferred over pip)-- **pytest + pytest-asyncio** for testing, **respx** for HTTP mocking+- **Cargo test + pytest** for testing- **ruff** for linting, **mypy** (strict) for type checking## Setup@@ -237,20 +199,20 @@ and their transitives never appear in downstream vulnerability scans.## Commands-### Running the server+### Running Switchyard```bashexport OPENROUTER_API_KEY="sk-or-..."-# Serve the minimal Python YAML bundle (noop and passthrough only).-switchyard serve --routes examples/route.yaml --port 4000-# Launch against the packaged OpenRouter deployment.switchyard launch claude --model switchyardswitchyard launch codex --model switchyard# Or select a route from a custom native TOML deployment.switchyard launch claude --model my-route --config routes.toml++# Run a standalone native server.+switchyard-server --config routes.toml --port 4000```### Testing@@ -260,8 +222,7 @@ switchyard launch claude --model my-route --config routes.tomluv run pytest tests/ -v# Single test file / function-uv run pytest tests/test_switchyard.py -v-uv run pytest tests/test_route_bundle.py::test_noop_route_returns_ok_without_an_upstream -v+uv run pytest tests/test_launchers.py -v# Live end-to-end tests are not part of the public test suite; if you write# one, set the provider key explicitly and run it directly, e.g.:@@ -270,49 +231,7 @@ uv run pytest tests/test_route_bundle.py::test_noop_route_returns_ok_without_an_# Lint / type check (run before every commit)uv run ruff check .uv run mypy switchyard-```--### Adding a new component--1. Pick the right stage: request component (pre-call), response component (post-call), `LLMBackend` (rare), or Rust translation codec work.-2. Create a file with the explicit name (`snake_case` of the class name), one class per file.-3. Implement the async method for that stage (`process` for components, `call` for backends).-4. Wire it into the owning explicit chain or route-bundle builder.-5. Add tests under `tests/`.-6. Export from the relevant `__init__.py` and from `switchyard/__init__.py`'s `__all__`.--### Example: custom request component--```python-from switchyard.lib.proxy_context import ProxyContext-from switchyard import ChatRequest---class MyRequestComponent:- async def process(self, ctx: ProxyContext, request: ChatRequest) -> ChatRequest:- ctx.metadata["my_key"] = "my_value"- return request-```--### App factory--```python-from switchyard import BackendFormat, LlmTarget, Switchyard, TranslationEngine-from switchyard.lib.backends import OpenAiNativeBackend-from switchyard import build_switchyard_app-import uvicorn--target = LlmTarget(- model="gpt-4o",- format=BackendFormat.OPENAI,- api_key="sk-...",- base_url="https://api.openai.com/v1",-)-switchyard = Switchyard(- backend=OpenAiNativeBackend(target),- translator=TranslationEngine(),-)-uvicorn.run(build_switchyard_app(switchyard), port=4000)+cargo test --workspace```## Environment Variables@@ -349,16 +268,15 @@ uvicorn.run(build_switchyard_app(switchyard), port=4000)### Always do- File name = snake_case of the primary class exported. Rename on touch.- Run `uv run ruff check .` (zero errors) and `uv run pytest tests/` before pushing.-- Export new public classes from `switchyard/__init__.py` with `__all__`.-- Write unit tests for new roles and bug fixes.-- Use `ProxyContext.metadata` for cross-component state within a request.-- In a new `LLMBackend`, map upstream context-window 4xx to `SwitchyardError::ContextWindowExceeded` (Rust) — the chain executor uses it for evict-and-retry. See [Context-Window Handling](docs/operations/context_window.md).+- Run `cargo test --workspace` for Rust behavior changes.+- Write focused unit tests for new behavior and bug fixes.+- Keep provider-neutral request and response types in `switchyard-protocol`.+- Map upstream context-window errors to `SwitchyardError::ContextWindowExceeded`.### Ask first- Modifying `pyproject.toml` dependencies.-- Changes to the chain shape or public role classes in `switchyard/lib/roles.py`.- Adding new HTTP endpoints.-- Removing or renaming any public API currently in `switchyard/__init__.__all__`.+- Removing or renaming public Rust, PyO3, or Python APIs.### Never do- Commit API keys or secrets (`secrets/` is gitignored).diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 444c44956..de81bbc1d 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -4,6 +4,15 @@ All notable changes to Switchyard are documented here. The format is based on[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this projectadheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).+## [Unreleased]++### Removed++- **Deprecated Python server stack** — `switchyard serve`, YAML route bundles,+ the FastAPI endpoints and legacy chain, the `switchyard-components` crate,+ and their compatibility PyO3 bindings are removed. Use `switchyard-server`+ with native TOML deployments, or `switchyard launch` for coding agents.+## [0.2.0]Switchyard 0.2.0 introduces the native Rust server and libsy library path,diff --git a/Cargo.lock b/Cargo.lockindex e5e151e0a..39871f0e9 100644--- a/Cargo.lock+++ b/Cargo.lock@@ -2263,27 +2263,6 @@ version = "2.6.1"source = "registry+https://github.com/rust-lang/crates.io-index"checksum = "13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292"-[[package]]-name = "switchyard-components"-version = "0.2.0"-dependencies = [- "async-stream",- "async-trait",- "futures-core",- "futures-util",- "parking_lot",- "rand 0.10.2",- "reqwest",- "serde",- "serde_json",- "switchyard-libsy",- "switchyard-protocol",- "switchyard-translation",- "thiserror 2.0.18",- "tokio",- "tracing",-]-[[package]]name = "switchyard-libsy"version = "0.2.0"@@ -2349,22 +2328,17 @@ name = "switchyard-py"version = "0.2.0"dependencies = ["async-trait",- "futures-util","http",- "parking_lot","pyo3","pyo3-async-runtimes","pythonize","serde","serde_json",- "switchyard-components","switchyard-libsy","switchyard-llm-client","switchyard-protocol","switchyard-server",- "switchyard-translation","tokio",- "tracing",][[package]]diff --git a/Cargo.toml b/Cargo.tomlindex 0c05c4767..07d133bb3 100644--- a/Cargo.toml+++ b/Cargo.toml@@ -6,7 +6,6 @@ resolver = "3"members = ["crates/libsy","crates/libsy-llm-client",- "crates/switchyard-components","crates/switchyard-py","crates/protocol","crates/switchyard-server",@@ -36,7 +35,6 @@ rand = "0.10"reqwest = { version = "0.13.4", default-features = false, features = ["json", "rustls", "stream"] }serde = { version = "1", features = ["derive"] }serde_json = "1"-switchyard-components = { path = "crates/switchyard-components", version = "0.2.0" }switchyard-libsy = { path = "crates/libsy", version = "0.2.0" }switchyard-llm-client = { path = "crates/libsy-llm-client", version = "0.2.0" }switchyard-protocol = { path = "crates/protocol", version = "0.2.0" }diff --git a/INSTALLATION.md b/INSTALLATION.mdindex b63ae8d58..29c74ae63 100644--- a/INSTALLATION.md+++ b/INSTALLATION.md@@ -1,186 +1,81 @@# Installation Guide-Switchyard supports modular installation based on your use case. Install only the dependencies you need.+Switchyard has separate packages for Python integrations and standalone Rust+serving.-## System Requirements+## Requirements-- Python 3.12 or newer. If the active interpreter is older, run- `uv pip install --python 3.12 nemo-switchyard`.-- Linux x86_64 wheels require an x86-64-v3 / AVX2-class CPU (post 2013).-- Linux aarch64 wheels require a Neoverse N1-class CPU (post 2020).+- Python 3.12 or newer for `nemo-switchyard`+- Rust 1.96.1 or newer for `switchyard-server` and the Rust libraries+- Linux x86_64 wheels require an x86-64-v3 / AVX2-class CPU+- Linux aarch64 wheels require a Neoverse N1-class CPU-## Core Installation (Library Only)+## Python Bindings-For applications that use Switchyard as a Python library for routing and recipe composition:+Install the Python package to embed libsy algorithms or host the native server+through PyO3:```bashpip install nemo-switchyard```-**Includes:**-- Core routing logic-- All recipe factories (Passthrough, RandomRouting, etc.)-- Format translation engine (Anthropic ↔ OpenAI)-- Request/response processors+The base package has no Python runtime dependencies. Its native extension owns+the libsy and server implementations.-**Does NOT include:**-- FastAPI / Uvicorn (server)-- prompt-toolkit (interactive command-line UI)+## Coding-Agent Launchers-**Use case:** Library users, middleware plugins, embedded integrations.--## Optional Extras--### `[server]` — Run as a Proxy Server--Add FastAPI and Uvicorn to run Switchyard as a standalone HTTP proxy:+Install the CLI extra to launch Claude Code, Codex CLI, or OpenClaw through the+packaged native server:```bash-pip install nemo-switchyard[server]+uv tool install --python 3.12 "nemo-switchyard[cli]"+export OPENROUTER_API_KEY="your-openrouter-key" # pragma: allowlist secret+switchyard launch claude --model switchyard```-**Adds:**-- FastAPI-- Uvicorn with standard extras-- sse-starlette (for SSE streaming)+The selected coding agent must already be installed and available on `PATH`.+Use `--config routes.toml` to select a custom native TOML deployment.-**Use case:** Deploying Switchyard as a service, e2e proxy operations.+## Standalone Server-### `[cli]` — Command-Line Tools--Add prompt-toolkit support for interactive command-line workflows:+Install the native Rust proxy from crates.io:```bash-pip install nemo-switchyard[cli]+cargo install --locked switchyard-server+switchyard-server --config routes.toml --dry-run+switchyard-server --config routes.toml --port 4000```-## Combined Extras--### Full Installation--Install all optional dependencies:+See [Getting Started](docs/getting_started.md#server-path) for a complete TOML+deployment and [`switchyard-server`](crates/switchyard-server/README.md) for the+configuration reference.-```bash-pip install nemo-switchyard[all]-```--Equivalent to: `nemo-switchyard[server,cli,tracing,affinity-redis]`+## Rust Libraries-### Common Combinations+Add the crates needed by an embedded application:-**Middleware plugin (no server/CLI):**-```bash-pip install nemo-switchyard # Core only+```toml+[dependencies]+switchyard-libsy = "0.2.0"+switchyard-protocol = "0.2.0"+switchyard-llm-client = "0.2.0"+switchyard-translation = "0.2.0"```-**Proxy server:**-```bash-pip install nemo-switchyard[server]-```--**Command-line tools:**-```bash-pip install nemo-switchyard[cli] # Includes core-```--**Production deployment with all features:**-```bash-pip install nemo-switchyard[all]-```--## Dependency Structure--### Core (Always Installed)-```-- openai>=2.34.0,<3.0-- anthropic>=0.99.0,<1.0-- httpx>=0.28.1,<1.0-- pydantic>=2.13.3,<3.0-```--### Optional Dependencies-| Extra | Size | Purpose |-|-------|------|---------|-| `[server]` | ~50 MB | HTTP proxy (FastAPI + Uvicorn) |-| `[cli]` | ~5 MB | Interactive terminal UI (prompt-toolkit) |--## Embedding in Your Own Application--### For Custom Applications--Embed Switchyard with minimal overhead:--```python-from switchyard import LlmTarget, OpenAiNativeBackend, Switchyard, TranslationEngine--# Core library only — no server/CLI dependencies-target = LlmTarget(- id="direct",- model="gpt-4o-mini",- format="openai",- api_key="sk-...",- base_url="https://api.openai.com/v1",-)-switchyard = Switchyard(- backend=OpenAiNativeBackend(target),- translator=TranslationEngine(),-)-```--## Troubleshooting--### Import Error: "No module named 'fastapi'"--You're trying to run the HTTP server without the `[server]` extra:--```bash-# Install with server support-pip install nemo-switchyard[server]-```--```python-from switchyard import Switchyard # OK (core)-from switchyard.server.switchyard_app import build_switchyard_app # needs [server]-```--### Import Error: "No module named 'prompt_toolkit'"--You're using an interactive command-line workflow without the `[cli]` extra:--```bash-# Install CLI support-pip install nemo-switchyard[cli]-```+`switchyard-libsy` owns algorithms, `switchyard-protocol` owns provider-neutral+request and response types, `switchyard-translation` owns wire conversion, and+`switchyard-llm-client` performs translated HTTP calls.## Development-For development with all testing tools, use `uv` (recommended):+From a checkout:```bash-uv sync # core + dev tooling (dev is uv's default group)-uv sync --all-extras # add every user-facing extra as well+uv sync+uv run maturin develop+cargo test --workspace+uv run pytest tests/ -v```-Or with pip ≥ 25.1 from a checkout:--```bash-pip install -e ".[all]"-pip install --group dev .-```--This includes:-- Core + all optional extras-- pytest, ruff, mypy, respx for testing--> **Note:** dev tooling lives in a PEP 735 dependency group, not an extra,-> so `pip install nemo-switchyard[dev]` is **not supported** and dev tooling-> never appears in the published wheel's METADATA (it's invisible to-> downstream vulnerability scans).--## Version Compatibility--Switchyard requires Python 3.12 or later.--Supported versions:-- Python 3.12-- Python 3.13+The `dev` dependency group contains testing and linting tools and is not exposed+in the published wheel metadata.diff --git a/README.md b/README.mdindex 7ead4efd3..cdc2b51de 100644--- a/README.md+++ b/README.md@@ -43,7 +43,7 @@ not already available, then install the published Switchyard tool:```bashcurl -LsSf https://astral.sh/uv/install.sh | shsource "$HOME/.local/bin/env"-uv tool install --python 3.12 "nemo-switchyard[cli,server]"+uv tool install --python 3.12 "nemo-switchyard[cli]"```The coding agent you launch must also be installed and on your `PATH`. This doesdiff --git a/benchmark/README.md b/benchmark/README.mdindex 3a78eeaf0..86feef084 100644--- a/benchmark/README.md+++ b/benchmark/README.md@@ -288,23 +288,6 @@ Neither artifact provides task or trial attribution. The runner writes them onlyand while the Rust server is still reachable; otherwise the manifest records them as missing.`routing_requests.jsonl` and `routing_stats_by_task.json` are not produced by the Rust server.-### Replay Stage-Router Scores--Replay completed trajectories through the stage-router scorer and both picker policies:--```bash-uv run python benchmark/score_staged_run.py \- --run benchmark/tb_runs/<run-name>-```--Use `--threshold` and `--window` to override the scorer defaults. The command writes per-turn JSONL-to `/tmp/<run-name>-scores.jsonl` and a per-task summary to-`/tmp/<run-name>-per-task.csv` unless `--output` or `--csv` is supplied.--The script processes each turn once, then applies both pickers to the same signal. Use `pick_cf` and-`pick_ef` for actual routing decisions; score bands alone do not include picker overrides or-fall-open behavior.-## Docker Image NotesBaseline runs build `switchyard-baseline:local` fromdiff --git a/benchmark/score_staged_run.py b/benchmark/score_staged_run.pydeleted file mode 100644index 6e4c9a2f6..000000000--- a/benchmark/score_staged_run.py+++ /dev/null@@ -1,283 +0,0 @@-#!/usr/bin/env python3-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0-"""Score a benchmark run directory via the stage-router Rust scorer.--Reads trajectory.json from each completed task, feeds each tool-use turn through:- DimensionCollector.process(ctx, request) → stage_pick_tier--The picker functions replicate live routing exactly: overrides, scorer, and fall_open.--Usage:- uv run python benchmark/score_staged_run.py --run benchmark/tb_runs/<run-name>- uv run python benchmark/score_staged_run.py --run benchmark/tb_runs/<run-name> \\- --output /tmp/scores.jsonl --threshold 0.20 --window 3-"""-import argparse-import asyncio-import csv-import json-import sys-from collections import defaultdict-from pathlib import Path-from statistics import mean--from switchyard_rust.components import (- DimensionCollector,- get_tool_result_signal,- stage_pick_tier,- stage_score_signal,-)-from switchyard_rust.core import ChatRequest, ProxyContext--RECENT_WINDOW = 3-CAPABLE = 1---def _tool_use_id(message: str) -> str:- parts = message.split()- return parts[-1] if len(parts) >= 2 else f"tu_{id(message)}"---async def score_trajectory(- traj: dict,- reward: float | None,- task_name: str,- trial_name: str,- run_id: str,- recent_window: int,- confidence_threshold: float,-) -> list[dict]:- steps = traj.get("steps", [])-- # One DimensionCollector per task — accumulates state across turns- dc = DimensionCollector(recent_window=recent_window)- await dc.startup()-- rows: list[dict] = []- messages: list[dict] = []-- for s in steps:- if s["source"] == "user" and not (s.get("extra") or {}).get("is_sidechain"):- messages.append({"role": "user", "content": [{"type": "text", "text": s["message"]}]})- break-- if not messages:- return rows-- for s in steps:- if s["source"] != "agent":- continue- extra = s.get("extra") or {}- if extra.get("is_sidechain"):- continue-- tool_name = extra.get("tool_use_name")- if not tool_name:- text = s.get("message", "")- if text:- messages.append({"role": "assistant", "content": [{"type": "text", "text": text}]})- continue-- tool_use_id = _tool_use_id(s.get("message", ""))- raw_args = extra.get("raw_arguments") or {}- is_error = bool(extra.get("tool_result_is_error", False))-- metadata = extra.get("metadata") or {}- raw_result = metadata.get("raw_tool_result") or {}- content = raw_result.get("content", "")- if not isinstance(content, str):- content = json.dumps(content)-- messages.append({- "role": "assistant",- "content": [{"type": "tool_use", "id": tool_use_id, "name": tool_name, "input": raw_args}],- })- messages.append({- "role": "user",- "content": [{"type": "tool_result", "tool_use_id": tool_use_id, "content": content, "is_error": is_error}],- })-- # Build Anthropic ChatRequest and process once through DimensionCollector- ctx = ProxyContext()- request = ChatRequest.anthropic({- "model": "claude-opus-4-8",- "max_tokens": 8096,- "messages": messages,- })- await dc.process(ctx, request)-- signal = get_tool_result_signal(ctx)- if signal is None:- continue-- # Raw score for analysis (Rust scorer): wrong signals score positive- # (→CAPABLE), progress negative (→EFFICIENT). Picker-independent, so the- # histogram shows the capable/efficient separation directly.- score, confidence = stage_score_signal(signal)-- # Apply both picker modes to the same signal without reprocessing the turn.- outcome_cf = stage_pick_tier(signal, "capable_first", confidence_threshold)- outcome_ef = stage_pick_tier(signal, "efficient_first", confidence_threshold)- tier_cf_name = outcome_cf.tier if outcome_cf.resolved else outcome_cf.default_tier- tier_ef_name = outcome_ef.tier if outcome_ef.resolved else outcome_ef.default_tier- tier_cf = int(tier_cf_name == "capable")- tier_ef = int(tier_ef_name == "capable")-- rows.append({- "task_name": task_name,- "trial_name": trial_name,- "run_id": run_id,- "reward": reward,- "turn_depth": signal.turn_depth,- "score": score,- "confidence": confidence,- "tool_name": tool_name,- "is_error": is_error,- "write_count": signal.write_count,- "edit_count": signal.edit_count,- "read_count": signal.read_count,- "no_error_streak": signal.no_error_streak,- "pure_bash_streak": signal.pure_bash_streak,- "tests_passed": signal.tests_passed,- "pick_cf": tier_cf, # CAPABLE=1, EFFICIENT=0- "pick_ef": tier_ef,- })-- return rows---def band(score: float, threshold: float) -> str:- if score >= threshold:- return "strong_clear"- if score > 0:- return "strong_uncertain"- if score <= -threshold:- return "weak_clear"- if score < 0:- return "weak_uncertain"- return "zero"---def write_per_task_csv(all_rows: list[dict], csv_path: Path, threshold: float) -> None:- groups: dict[tuple, list] = defaultdict(list)- for r in all_rows:- groups[(r["run_id"], r["task_name"])].append(r)-- fieldnames = [- "run_id", "task_name", "reward", "n_turns",- "mean_score", "mean_confidence",- "pct_strong_clear", "pct_strong_uncertain", "pct_weak_uncertain", "pct_weak_clear",- "opus_pct_cf", "nemotron_pct_cf", "opus_pct_ef", "nemotron_pct_ef",- ]- rows_out = []- for (run_id, task_name), turns in sorted(groups.items()):- n = len(turns)- scores = [t["score"] for t in turns]- confs = [t["confidence"] for t in turns]- reward = turns[0]["reward"]- bands = [band(s, threshold) for s in scores]- n_sc = bands.count("strong_clear")- n_su = bands.count("strong_uncertain")- n_wu = bands.count("weak_uncertain")- n_wc = bands.count("weak_clear")- n_cf_opus = sum(1 for t in turns if t["pick_cf"] == CAPABLE)- n_ef_opus = sum(1 for t in turns if t["pick_ef"] == CAPABLE)- rows_out.append({- "run_id": run_id,- "task_name": task_name,- "reward": reward,- "n_turns": n,- "mean_score": round(mean(scores), 4),- "mean_confidence": round(mean(confs), 4),- "pct_strong_clear": round(n_sc / n, 4),- "pct_strong_uncertain": round(n_su / n, 4),- "pct_weak_uncertain": round(n_wu / n, 4),- "pct_weak_clear": round(n_wc / n, 4),- "opus_pct_cf": round(n_cf_opus / n, 4),- "nemotron_pct_cf": round(1 - n_cf_opus / n, 4),- "opus_pct_ef": round(n_ef_opus / n, 4),- "nemotron_pct_ef": round(1 - n_ef_opus / n, 4),- })-- csv_path.parent.mkdir(parents=True, exist_ok=True)- with open(csv_path, "w", newline="") as f:- w = csv.DictWriter(f, fieldnames=fieldnames)- w.writeheader()- w.writerows(rows_out)- print(f"Per-task CSV → {csv_path} ({len(rows_out)} rows)")---async def score_run(run_dir: Path, recent_window: int, threshold: float) -> list[dict]:- jobs_dirs = list(run_dir.glob("jobs/*/")) or [run_dir]- run_id = run_dir.name-- all_rows: list[dict] = []- for jobs_dir in jobs_dirs:- task_dirs = [d for d in jobs_dir.iterdir() if d.is_dir() and d.name != "verifier"]- print(f"{jobs_dir.name}: {len(task_dirs)} task dirs")-- for task_dir in sorted(task_dirs):- traj_path = task_dir / "agent" / "trajectory.json"- result_path = task_dir / "result.json"- if not traj_path.exists():- continue-- traj = json.loads(traj_path.read_text())- reward: float | None = None- task_name = task_dir.name- if result_path.exists():- res = json.loads(result_path.read_text())- task_name = res.get("task_name") or task_name- r = (res.get("verifier_result") or {}).get("rewards", {}).get("reward")- reward = float(r) if r is not None else None-- rows = await score_trajectory(- traj, reward, task_name, task_dir.name, run_id, recent_window, threshold- )- all_rows.extend(rows)- print(f" {task_dir.name}: {len(rows)} turns, reward={reward}")-- return all_rows---async def main() -> None:- parser = argparse.ArgumentParser(description=__doc__)- parser.add_argument("--run", required=True, help="Path to run directory under benchmark/tb_runs/")- parser.add_argument("--output", help="Per-turn JSONL (default: /tmp/<run-name>-scores.jsonl)")- parser.add_argument("--csv", help="Per-task CSV (default: /tmp/<run-name>-per-task.csv)")- parser.add_argument("--threshold", type=float, default=0.20)- parser.add_argument("--window", type=int, default=RECENT_WINDOW)- args = parser.parse_args()-- run_dir = Path(args.run)- if not run_dir.exists():- sys.exit(f"Run directory not found: {run_dir}")-- jsonl_out = Path(args.output or f"/tmp/{run_dir.name}-scores.jsonl")- csv_out = Path(args.csv or f"/tmp/{run_dir.name}-per-task.csv")-- all_rows = await score_run(run_dir, args.window, args.threshold)-- jsonl_out.parent.mkdir(parents=True, exist_ok=True)- with open(jsonl_out, "w") as fh:- for row in all_rows:- fh.write(json.dumps(row) + "\n")- print(f"Per-turn JSONL → {jsonl_out} ({len(all_rows)} rows)")-- if all_rows:- write_per_task_csv(all_rows, csv_out, args.threshold)- total = len(all_rows)- scores = [r["score"] for r in all_rows]- cf_opus = sum(1 for r in all_rows if r["pick_cf"] == CAPABLE)- ef_opus = sum(1 for r in all_rows if r["pick_ef"] == CAPABLE)- print(f"\nGlobal summary ({total} turns, threshold={args.threshold}):")- for bn in ["strong_clear", "strong_uncertain", "weak_uncertain", "weak_clear"]:- n = sum(1 for s in scores if band(s, args.threshold) == bn)- print(f" {bn:<22} {n:>5} ({100*n/total:.1f}%)")- print(f" cf → Opus: {cf_opus}/{total} ({100*cf_opus/total:.1f}%)")- print(f" ef → Opus: {ef_opus}/{total} ({100*ef_opus/total:.1f}%)")---if __name__ == "__main__":- asyncio.run(main())diff --git a/crates/libsy-llm-client/src/client.rs b/crates/libsy-llm-client/src/client.rsindex 08ff14497..d6ca1191d 100644--- a/crates/libsy-llm-client/src/client.rs+++ b/crates/libsy-llm-client/src/client.rs@@ -664,8 +664,7 @@ fn set_json_model(body: &mut Value, model: &str) {// later turns from an Anthropic one. Clients such as Claude Code send// `context_management` on every turn, so the Anthropic leg must strip it or the// upstream rejects the request (for example `clear_thinking_20251015` strategy-// requires `thinking` to be enabled or adaptive). Mirrors-// `switchyard-components`' `strip_anthropic_incompatible_fields`.+// requires `thinking` to be enabled or adaptive).fn strip_anthropic_incompatible_fields(body: &mut Value) {if let Value::Object(object) = body {object.remove("reasoning_effort");@@ -679,8 +678,7 @@ fn strip_anthropic_incompatible_fields(body: &mut Value) {// turns of a session from an OpenAI-format target whose thinking blocks are// unsigned, so the Anthropic leg must drop them or the upstream rejects the// request. Bedrock enforces this (surfacing as a SigV4 signature mismatch) where-// Azure-hosted Anthropic currently does not. Mirrors `switchyard-components`'-// `strip_unsigned_thinking_blocks`.+// Azure-hosted Anthropic currently does not.fn strip_unsigned_thinking_blocks(body: &mut Value) {let Value::Object(object) = body else {return;diff --git a/crates/libsy-llm-client/src/error.rs b/crates/libsy-llm-client/src/error.rsindex b99850a45..0a1219da5 100644--- a/crates/libsy-llm-client/src/error.rs+++ b/crates/libsy-llm-client/src/error.rs@@ -3,10 +3,8 @@//! Canonical client error re-export and shared context-window-overflow detection.//!-//! The overflow detection is ported from-//! `switchyard-components/src/backends/context_overflow.rs`; that helper is-//! crate-private there and this crate cannot depend on `switchyard-components`,-//! so the small, self-contained logic is vendored here.+//! The client owns overflow detection so callers receive one stable error+//! classification across providers.use serde_json::Value;diff --git a/crates/switchyard-components/Cargo.toml b/crates/switchyard-components/Cargo.tomldeleted file mode 100644index 8775026ef..000000000--- a/crates/switchyard-components/Cargo.toml+++ /dev/null@@ -1,32 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--[package]-name = "switchyard-components"-version.workspace = true-description = "Concrete Switchyard backends and processors"-authors.workspace = true-edition.workspace = true-license.workspace = true-repository.workspace = true-rust-version.workspace = true--[dependencies]-async-stream.workspace = true-async-trait.workspace = true-futures-core = "0.3"-futures-util.workspace = true-parking_lot.workspace = true-rand.workspace = true-reqwest.workspace = true-serde.workspace = true-serde_json.workspace = true-switchyard-libsy.workspace = true-switchyard-protocol.workspace = true-switchyard-translation.workspace = true-tokio.workspace = true-thiserror.workspace = true-tracing.workspace = true--[dev-dependencies]-tokio.workspace = truediff --git a/crates/switchyard-components/src/backends/anthropic.rs b/crates/switchyard-components/src/backends/anthropic.rsdeleted file mode 100644index 8fb611373..000000000--- a/crates/switchyard-components/src/backends/anthropic.rs+++ /dev/null@@ -1,684 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Anthropic-compatible Messages backend.--use std::collections::BTreeMap;-use std::env;-use std::fmt;-use std::sync::Arc;--use crate::{- BackendFormat, BoxResponseStream, ChatRequest, ChatRequestType, ChatResponse, LlmBackend,- LlmTarget, LlmTargetId, ProxyContext, Result, StreamEvent, SwitchyardError,- merge_target_extra_body,-};-use async_stream::try_stream;-use async_trait::async_trait;-use futures_util::StreamExt;-use serde_json::{Map, Value, json};-use switchyard_translation::{- TranslationEngine, TranslationPolicy, WireFormat, normalize_anthropic_tool_use_ids,-};--use super::BackendSelection;-use super::common::{- ParsedSseFrame, build_reqwest_client, decode_sse_frame, drain_next_sse_frame,- has_non_whitespace_bytes, parse_json_sse_frame, request_wire_format, set_json_model,- shared_translation_engine,-};-use crate::telemetry::{SWITCHYARD_VERSION_HEADER, telemetry_header_value};--const DEFAULT_ANTHROPIC_BASE_URL: &str = "https://api.anthropic.com";-const ANTHROPIC_API_KEY_ENV: &str = "ANTHROPIC_API_KEY";-const ANTHROPIC_VERSION: &str = "2023-06-01";-static ANTHROPIC_ONLY: [ChatRequestType; 1] = [ChatRequestType::Anthropic];--/// Backend that calls an Anthropic-compatible Messages API.-pub struct AnthropicNativeBackend {- /// Resolved target used for endpoint credentials and model rewriting.- target: LlmTarget,- /// HTTP transport, injectable for deterministic tests.- transport: Arc<dyn AnthropicTransport>,- /// Shared request translator for non-Anthropic inbound payloads.- translation: Arc<TranslationEngine>,- /// Translation policy kept explicit so backend behavior is inspectable.- translation_policy: TranslationPolicy,-}--impl fmt::Debug for AnthropicNativeBackend {- fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {- formatter- .debug_struct("AnthropicNativeBackend")- .field("target", &self.target)- .finish_non_exhaustive()- }-}--impl AnthropicNativeBackend {- /// Creates an Anthropic-compatible backend for one target.- pub fn new(target: LlmTarget) -> Result<Self> {- let transport = Arc::new(ReqwestAnthropicTransport::new(- target.endpoint.timeout_secs,- )?);- Self::with_transport(target, transport)- }-- /// Returns the configured upstream target.- pub fn target(&self) -> &LlmTarget {- &self.target- }-- fn with_transport(target: LlmTarget, transport: Arc<dyn AnthropicTransport>) -> Result<Self> {- validate_target_format(&target)?;- Ok(Self {- target,- transport,- translation: shared_translation_engine(),- translation_policy: TranslationPolicy::default(),- })- }-- fn outbound_body(&self, request: &ChatRequest) -> Result<Value> {- let mut body = match request.request_type() {- ChatRequestType::Anthropic => request.body().clone(),- source => {- self.translation- .translate_request(- request_wire_format(source),- WireFormat::AnthropicMessages,- request.body(),- &self.translation_policy,- )- .map_err(|error| {- SwitchyardError::Backend(format!(- "failed to translate {source:?} request to Anthropic Messages: {error}"- ))- })?- .body- }- };- set_json_model(&mut body, self.target.model.as_str());- strip_anthropic_incompatible_fields(&mut body);- normalize_anthropic_body(&mut body);- // Per-target ``extra_body`` merged last; caller wins on key- // conflicts (see :func:`merge_target_extra_body`).- merge_target_extra_body(&mut body, self.target.extra_body.as_ref());- Ok(body)- }-- /// Calls this target without requiring chain-local `ProxyContext` state.- pub async fn call_without_context(&self, request: &ChatRequest) -> Result<ChatResponse> {- let http_request = self.http_request(request)?;- self.send_http_request(http_request).await- }-- // Builds the upstream HTTP request before any context observations are recorded.- fn http_request(&self, request: &ChatRequest) -> Result<AnthropicHttpRequest> {- let body = self.outbound_body(request)?;- let stream = body.get("stream").and_then(Value::as_bool).unwrap_or(false);- Ok(AnthropicHttpRequest {- target_id: self.target.id.clone(),- url: messages_url(self.target.endpoint.base_url.as_deref()),- api_key: anthropic_api_key(self.target.endpoint.api_key.as_deref()),- body,- stream,- extra_headers: self.target.extra_headers.clone(),- })- }-- // Sends an already-normalized upstream request.- async fn send_http_request(&self, request: AnthropicHttpRequest) -> Result<ChatResponse> {- match self.transport.send(request).await? {- AnthropicHttpResponse::Buffered(body) => Ok(ChatResponse::anthropic_completion(body)),- AnthropicHttpResponse::Stream(stream) => Ok(ChatResponse::AnthropicStream(stream)),- }- }-}--#[async_trait]-impl LlmBackend for AnthropicNativeBackend {- fn supported_request_types(&self) -> &[ChatRequestType] {- &ANTHROPIC_ONLY- }-- async fn call(&self, ctx: &mut ProxyContext, request: &ChatRequest) -> Result<ChatResponse> {- let http_request = self.http_request(request)?;-- ctx.inbound_format = ctx.inbound_format.or(Some(request.request_type()));- let previous_selection = ctx.get::<BackendSelection>().cloned();- ctx.insert(BackendSelection::native_target_observation(- previous_selection.as_ref(),- self.target.id.clone(),- self.target.model.clone(),- request.model().map(str::to_string),- ));-- self.send_http_request(http_request).await- }-}--#[derive(Clone, Debug, PartialEq)]-struct AnthropicHttpRequest {- /// Target ID used only for logging and diagnostics.- target_id: LlmTargetId,- /// Fully resolved Messages API URL.- url: String,- /// Per-target API key or process environment fallback.- api_key: Option<String>,- /// Already-normalized Anthropic Messages request body.- body: Value,- /// Whether the upstream call should be treated as SSE.- stream: bool,- /// Per-target headers merged onto the outbound request.- extra_headers: BTreeMap<String, String>,-}--enum AnthropicHttpResponse {- /// Complete JSON response from a non-streaming upstream call.- Buffered(Value),- /// Streamed SSE response converted into Switchyard stream events.- Stream(BoxResponseStream),-}--#[async_trait]-trait AnthropicTransport: Send + Sync {- /// Sends one already-normalized Anthropic Messages request.- async fn send(&self, request: AnthropicHttpRequest) -> Result<AnthropicHttpResponse>;-}--struct ReqwestAnthropicTransport {- /// Reused async HTTP client with configured timeout behavior.- client: reqwest::Client,-}--impl ReqwestAnthropicTransport {- fn new(timeout_secs: Option<f64>) -> Result<Self> {- let client = build_reqwest_client("Anthropic", timeout_secs)?;- Ok(Self { client })- }-}--#[async_trait]-impl AnthropicTransport for ReqwestAnthropicTransport {- async fn send(&self, request: AnthropicHttpRequest) -> Result<AnthropicHttpResponse> {- let target_id = request.target_id.clone();- let mut builder = self- .client- .post(&request.url)- .header("anthropic-version", ANTHROPIC_VERSION)- .json(&request.body);- if let Some(api_key) = request.api_key {- builder = builder.header("x-api-key", api_key);- }- if let Some(version) = telemetry_header_value() {- builder = builder.header(SWITCHYARD_VERSION_HEADER, version);- }- for (name, value) in &request.extra_headers {- builder = builder.header(name, value);- }-- let response = builder.send().await.map_err(|error| {- tracing::warn!(- target_id = %target_id,- error = %error,- "Anthropic messages request failed"- );- SwitchyardError::Upstream(format!("Anthropic messages request failed: {error}"))- })?;- let status = response.status();- if !status.is_success() {- let body = response- .text()- .await- .unwrap_or_else(|error| format!("<failed to read error body: {error}>"));- tracing::warn!(- target_id = %target_id,- status = %status,- "Anthropic messages returned error status"- );- if status == reqwest::StatusCode::BAD_REQUEST && is_context_overflow(&body) {- let model = request- .body- .get("model")- .and_then(Value::as_str)- .unwrap_or("")- .to_string();- return Err(SwitchyardError::ContextWindowExceeded {- target_id: target_id.to_string(),- model,- message: body,- });- }- return Err(SwitchyardError::UpstreamHttp {- provider: "Anthropic messages".to_string(),- status_code: status.as_u16(),- body,- });- }-- if request.stream {- return Ok(AnthropicHttpResponse::Stream(anthropic_sse_stream(- response,- )));- }-- let body = response.json::<Value>().await.map_err(|error| {- SwitchyardError::Upstream(format!("Anthropic messages returned invalid JSON: {error}"))- })?;- Ok(AnthropicHttpResponse::Buffered(body))- }-}--fn validate_target_format(target: &LlmTarget) -> Result<()> {- match target.format {- BackendFormat::Anthropic => Ok(()),- BackendFormat::Auto | BackendFormat::OpenAi | BackendFormat::Responses => {- Err(SwitchyardError::InvalidConfig(format!(- "AnthropicNativeBackend requires a target with resolved Anthropic format, got {:?} for {}",- target.format, target.id- )))- }- }-}--// Drop fields accepted by OpenAI-like APIs but rejected by Anthropic Messages.-fn strip_anthropic_incompatible_fields(body: &mut Value) {- if let Value::Object(object) = body {- object.remove("reasoning_effort");- object.remove("context_management");- }-}--// Normalize translated Anthropic payloads before applying target overrides.-fn normalize_anthropic_body(body: &mut Value) {- let Value::Object(object) = body else {- return;- };- if let Some(messages) = object.remove("messages") {- // AWS documents message-level `role: "system"` as an Opus 4.8-only- // Anthropic dialect:- // https://docs.aws.amazon.com/bedrock/latest/userguide/claude-messages-mid-conversation-system.html- //- // Keep the default conservative for older Bedrock/LiteLLM targets by- // lifting those turns into top-level `system`.- // TODO: add target-level Anthropic dialect support so- // Opus 4.8 can opt into mid-conversation system messages explicitly.- let (messages, system_text) = lift_message_level_system(messages);- append_lifted_system_text(object, system_text);- let messages = normalize_anthropic_tool_use_ids(messages);- object.insert(- "messages".to_string(),- strip_unsigned_thinking_blocks(messages),- );- }-}--// Moves Anthropic Opus-4.8-style message-level system turns out of the-// conversation so legacy Anthropic-compatible backends do not reject them.-fn lift_message_level_system(messages: Value) -> (Value, Vec<String>) {- let Value::Array(messages) = messages else {- return (messages, Vec::new());- };-- let mut kept_messages = Vec::with_capacity(messages.len());- let mut system_text = Vec::new();- for message in messages {- if is_message_level_system(&message) {- if let Some(text) = system_text_from_message(&message) {- system_text.push(text);- }- } else {- kept_messages.push(message);- }- }-- (Value::Array(kept_messages), system_text)-}--// Treats `system` and OpenAI/Codex `developer` roles as instruction-like-// turns. `developer` is not an Anthropic Opus 4.8 role; lifting it matches the-// existing OpenAI-to-Anthropic translator behavior and prevents malformed-// Anthropic-bound traffic from leaking an invalid role upstream.-fn is_message_level_system(message: &Value) -> bool {- matches!(- message.get("role").and_then(Value::as_str),- Some("system") | Some("developer")- )-}--// Extracts text from an invalid message-level system/developer turn.-fn system_text_from_message(message: &Value) -> Option<String> {- message.get("content").and_then(system_text_from_content)-}--// Converts Anthropic text-ish content into top-level system text.-fn system_text_from_content(content: &Value) -> Option<String> {- match content {- Value::String(text) if !text.is_empty() => Some(text.clone()),- Value::String(_) | Value::Null => None,- Value::Array(blocks) => {- let parts = blocks- .iter()- .filter_map(system_text_from_content_block)- .collect::<Vec<_>>();- (!parts.is_empty()).then(|| parts.join("\n\n"))- }- other => Some(other.to_string()),- }-}--// Extracts the supported text shape from one structured content block.-fn system_text_from_content_block(block: &Value) -> Option<String> {- match block {- Value::String(text) if !text.is_empty() => Some(text.clone()),- Value::Object(object) => match object.get("type").and_then(Value::as_str) {- Some("text") | Some("input_text") => object- .get("text")- .and_then(Value::as_str)- .filter(|text| !text.is_empty())- .map(ToOwned::to_owned),- // Message-level system/developer turns are downgraded for legacy Anthropic- // compatibility. Only text-like instruction content is replayed; images and- // other non-text blocks are intentionally not promoted into top-level system.- _ => None,- },- _ => None,- }-}--// Appends lifted message-level system text onto any existing Anthropic system field.-fn append_lifted_system_text(object: &mut Map<String, Value>, system_text: Vec<String>) {- if system_text.is_empty() {- return;- }-- let joined = system_text.join("\n\n");- match object.remove("system") {- None | Some(Value::Null) => {- object.insert("system".to_string(), Value::String(joined));- }- Some(Value::String(existing)) if existing.is_empty() => {- object.insert("system".to_string(), Value::String(joined));- }- Some(Value::String(existing)) => {- object.insert(- "system".to_string(),- Value::String(format!("{existing}\n\n{joined}")),- );- }- Some(Value::Array(mut blocks)) => {- blocks.extend(- system_text- .into_iter()- .map(|text| json!({"type": "text", "text": text})),- );- object.insert("system".to_string(), Value::Array(blocks));- }- Some(other) => {- object.insert(- "system".to_string(),- Value::String(format!("{other}\n\n{joined}")),- );- }- }-}--// Anthropic requires signed thinking blocks on replay; remove unsigned blocks-// so passthrough and translated requests remain accepted by the API.-fn strip_unsigned_thinking_blocks(messages: Value) -> Value {- match messages {- Value::Array(messages) => Value::Array(- messages- .into_iter()- .map(strip_unsigned_thinking_from_message)- .collect(),- ),- other => other,- }-}--fn strip_unsigned_thinking_from_message(message: Value) -> Value {- match message {- Value::Object(mut message) => {- let Some(content) = message.remove("content") else {- return Value::Object(message);- };- let Value::Array(blocks) = content else {- message.insert("content".to_string(), content);- return Value::Object(message);- };-- let kept = blocks- .into_iter()- .filter(|block| !is_unsigned_thinking_block(block))- .collect::<Vec<_>>();- let content = if kept.is_empty() {- Value::String(String::new())- } else {- Value::Array(kept)- };- message.insert("content".to_string(), content);- Value::Object(message)- }- other => other,- }-}--fn is_unsigned_thinking_block(block: &Value) -> bool {- if block.get("type").and_then(Value::as_str) != Some("thinking") {- return false;- }- !matches!(- block.get("signature").and_then(Value::as_str),- Some(signature) if !signature.is_empty()- )-}--fn messages_url(base_url: Option<&str>) -> String {- let base_url = base_url- .unwrap_or(DEFAULT_ANTHROPIC_BASE_URL)- .trim_end_matches('/');- if base_url.ends_with("/v1/messages") {- base_url.to_string()- } else if base_url.ends_with("/v1") {- format!("{base_url}/messages")- } else {- format!("{base_url}/v1/messages")- }-}--fn anthropic_api_key(configured: Option<&str>) -> Option<String> {- // Resolve per call so long-lived backends can pick up rotated environment credentials.- configured- .map(str::to_string)- .or_else(|| env::var(ANTHROPIC_API_KEY_ENV).ok())- .filter(|value| !value.trim().is_empty())-}--// Canonical Anthropic 4xx looks like-// `{"type":"error","error":{"type":"invalid_request_error","message":"prompt is too long: ..."}}`.-// Anthropic has no structured `error.code` field, so detection is phrase-based only.-const ANTHROPIC_OVERFLOW_PHRASES: &[&str] = &[- "prompt is too long",- "maximum number of tokens",- "context window",- "context length",-];--fn is_context_overflow(body: &str) -> bool {- super::context_overflow::is_overflow_body(body, |_| false, ANTHROPIC_OVERFLOW_PHRASES)-}--fn anthropic_sse_stream(response: reqwest::Response) -> BoxResponseStream {- Box::pin(try_stream! {- let mut chunks = response.bytes_stream();- let mut buffer = Vec::new();-- while let Some(chunk) = chunks.next().await {- let chunk = chunk.map_err(|error| {- SwitchyardError::Upstream(format!("Anthropic stream read failed: {error}"))- })?;- buffer.extend_from_slice(&chunk);-- // Anthropic SSE has named events, but the payload we care about is- // still the JSON `data:` line.- while let Some(frame) = drain_next_sse_frame(&mut buffer, "Anthropic")? {- match parse_json_sse_frame(&frame, "Anthropic", Some("[DONE]"))? {- ParsedSseFrame::Json(value) => yield StreamEvent::Json(value),- ParsedSseFrame::Done => return,- ParsedSseFrame::Empty => {}- }- }- }-- // Preserve the last frame when an upstream closes without a final SSE- // separator.- if has_non_whitespace_bytes(&buffer) {- let frame = decode_sse_frame(&buffer, "Anthropic")?;- match parse_json_sse_frame(&frame, "Anthropic", Some("[DONE]"))? {- ParsedSseFrame::Json(value) => yield StreamEvent::Json(value),- ParsedSseFrame::Done | ParsedSseFrame::Empty => {}- }- }- })-}--#[cfg(test)]-mod tests {- use crate::{EndpointConfig, LlmTargetId, ModelId};- use parking_lot::Mutex;- use serde_json::json;-- use super::*;-- struct FakeAnthropicTransport {- requests: Mutex<Vec<AnthropicHttpRequest>>,- response: Mutex<Option<Result<AnthropicHttpResponse>>>,- }-- impl FakeAnthropicTransport {- fn with_error(message: &str) -> Self {- Self {- requests: Mutex::new(Vec::new()),- response: Mutex::new(Some(Err(SwitchyardError::Upstream(message.to_string())))),- }- }- }-- #[async_trait]- impl AnthropicTransport for FakeAnthropicTransport {- async fn send(&self, request: AnthropicHttpRequest) -> Result<AnthropicHttpResponse> {- self.requests.lock().push(request);- self.response.lock().take().ok_or_else(|| {- SwitchyardError::Other("fake transport response already consumed".to_string())- })?- }- }-- fn anthropic_target() -> LlmTarget {- LlmTarget {- id: LlmTargetId::from_static("primary"),- model: ModelId::from_static("target-claude"),- format: BackendFormat::Anthropic,- endpoint: EndpointConfig {- base_url: Some("https://example.test/v1".to_string()),- api_key: Some("secret".to_string()),- timeout_secs: None,- },- extra_body: None,- extra_headers: BTreeMap::new(),- }- }-- #[tokio::test]- async fn transport_errors_are_backend_errors() -> Result<()> {- let transport = Arc::new(FakeAnthropicTransport::with_error("upstream exploded"));- let backend = AnthropicNativeBackend::with_transport(anthropic_target(), transport)?;- let request = ChatRequest::anthropic(json!({- "model": "client-model",- "max_tokens": 128,- "messages": [],- }));- let mut ctx = ProxyContext::new();-- let Err(error) = backend.call(&mut ctx, &request).await else {- return Err(SwitchyardError::Other(- "backend call should fail".to_string(),- ));- };-- assert!(matches!(error, SwitchyardError::Upstream(_)));- assert!(error.to_string().contains("upstream exploded"));- Ok(())- }-- #[test]- fn rejects_openai_targets() -> Result<()> {- let mut target = anthropic_target();- target.format = BackendFormat::OpenAi;-- let Err(error) = AnthropicNativeBackend::new(target) else {- return Err(SwitchyardError::Other(- "OpenAI target should be rejected".to_string(),- ));- };-- assert!(matches!(error, SwitchyardError::InvalidConfig(_)));- Ok(())- }-- #[test]- fn formats_messages_urls_for_root_v1_and_explicit_paths() {- assert_eq!(- messages_url(Some("https://example.test")),- "https://example.test/v1/messages"- );- assert_eq!(- messages_url(Some("https://example.test/v1")),- "https://example.test/v1/messages"- );- assert_eq!(- messages_url(Some("https://example.test/v1/messages")),- "https://example.test/v1/messages"- );- }-- #[test]- fn parses_anthropic_sse_json_frames() -> Result<()> {- let ParsedSseFrame::Json(value) = parse_json_sse_frame(- "event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"id\":\"m\"}}\n",- "Anthropic",- None,- )?- else {- return Err(SwitchyardError::Other(- "JSON frame should produce a JSON value".to_string(),- ));- };- assert_eq!(value["type"], "message_start");- assert!(matches!(- parse_json_sse_frame("event: ping\n\n", "Anthropic", None)?,- ParsedSseFrame::Empty- ));- Ok(())- }-- #[test]- fn anthropic_context_overflow_canonical_shape_matches() {- let body = r#"{"type":"error","error":{"type":"invalid_request_error","message":"prompt is too long: 200001 tokens > 200000 maximum"}}"#;- assert!(is_context_overflow(body));- }-- #[test]- fn anthropic_context_overflow_max_tokens_phrase_matches() {- let body = r#"{"type":"error","error":{"message":"exceeds the maximum number of tokens for this model"}}"#;- assert!(is_context_overflow(body));- }-- #[test]- fn anthropic_context_overflow_unrelated_400_does_not_match() {- let body = r#"{"type":"error","error":{"type":"invalid_request_error","message":"missing system prompt"}}"#;- assert!(!is_context_overflow(body));- }-}diff --git a/crates/switchyard-components/src/backends/common.rs b/crates/switchyard-components/src/backends/common.rsdeleted file mode 100644index fd4d1c4a5..000000000--- a/crates/switchyard-components/src/backends/common.rs+++ /dev/null@@ -1,192 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Shared helpers for native backend implementations.--use std::sync::{Arc, OnceLock};-use std::time::Duration;--use crate::{ChatRequestType, Result, SwitchyardError};-use serde_json::{Map, Value};-use switchyard_translation::{TranslationEngine, WireFormat};--pub(crate) enum ParsedSseFrame {- /// Frame contained a JSON payload.- Json(Value),- /// Frame contained the provider's terminal marker.- Done,- /// Frame had no data payload.- Empty,-}--/// Returns the shared translation engine used by native backends.-pub(crate) fn shared_translation_engine() -> Arc<TranslationEngine> {- static ENGINE: OnceLock<Arc<TranslationEngine>> = OnceLock::new();- Arc::clone(ENGINE.get_or_init(|| Arc::new(TranslationEngine::default())))-}--/// Builds a reqwest client with validated optional timeout.-pub(crate) fn build_reqwest_client(- backend_name: &str,- timeout_secs: Option<f64>,-) -> Result<reqwest::Client> {- validate_timeout_secs(backend_name, timeout_secs)?;- let mut builder = reqwest::Client::builder();- if let Some(timeout_secs) = timeout_secs {- builder = builder.timeout(Duration::from_secs_f64(timeout_secs));- }- builder.build().map_err(|error| {- SwitchyardError::InvalidConfig(format!(- "failed to build {backend_name} HTTP client: {error}"- ))- })-}--/// Validates timeout values before they reach reqwest.-pub(crate) fn validate_timeout_secs(backend_name: &str, timeout_secs: Option<f64>) -> Result<()> {- if let Some(timeout_secs) = timeout_secs- && (!timeout_secs.is_finite() || timeout_secs <= 0.0)- {- return Err(SwitchyardError::InvalidConfig(format!(- "{backend_name} target timeout_secs must be finite and positive, got {timeout_secs:?}"- )));- }- Ok(())-}--/// Maps a Switchyard request type to its wire format.-pub(crate) fn request_wire_format(request_type: ChatRequestType) -> WireFormat {- match request_type {- ChatRequestType::OpenAiChat => WireFormat::OpenAiChat,- ChatRequestType::OpenAiResponses => WireFormat::OpenAiResponses,- ChatRequestType::Anthropic => WireFormat::AnthropicMessages,- }-}--/// Sets or creates the JSON `model` field for an outbound provider request.-pub(crate) fn set_json_model(body: &mut Value, model: &str) {- match body {- Value::Object(object) => {- object.insert("model".to_string(), Value::String(model.to_string()));- }- other => {- let mut object = Map::new();- object.insert("model".to_string(), Value::String(model.to_string()));- *other = Value::Object(object);- }- }-}--/// Drains one complete SSE frame from the buffer when a boundary is present.-pub(crate) fn drain_next_sse_frame(- buffer: &mut Vec<u8>,- backend_name: &str,-) -> Result<Option<String>> {- let Some((index, separator_len)) = next_sse_boundary(buffer) else {- return Ok(None);- };- let frame = decode_sse_frame(&buffer[..index], backend_name)?;- buffer.drain(..index + separator_len);- Ok(Some(frame))-}--/// Decodes one raw SSE frame as UTF-8.-pub(crate) fn decode_sse_frame(frame: &[u8], backend_name: &str) -> Result<String> {- std::str::from_utf8(frame)- .map(str::to_string)- .map_err(|error| {- SwitchyardError::Upstream(format!(- "{backend_name} stream emitted invalid UTF-8 frame: {error}"- ))- })-}--/// Returns whether the buffer has any non-whitespace bytes.-pub(crate) fn has_non_whitespace_bytes(buffer: &[u8]) -> bool {- buffer.iter().any(|byte| !byte.is_ascii_whitespace())-}--/// Parses data lines from one SSE frame into JSON, terminal, or empty states.-pub(crate) fn parse_json_sse_frame(- frame: &str,- backend_name: &str,- done_marker: Option<&str>,-) -> Result<ParsedSseFrame> {- let mut data_lines = Vec::new();- for line in frame.lines() {- // SSE comments and blank lines do not contribute data.- if line.is_empty() || line.starts_with(':') {- continue;- }- if let Some(data) = line.strip_prefix("data:") {- data_lines.push(data.trim_start().to_string());- }- }-- if data_lines.is_empty() {- return Ok(ParsedSseFrame::Empty);- }-- let data = data_lines.join("\n");- if done_marker.is_some_and(|marker| data.trim() == marker) {- return Ok(ParsedSseFrame::Done);- }-- let value = serde_json::from_str::<Value>(&data).map_err(|error| {- SwitchyardError::Upstream(format!(- "{backend_name} stream emitted invalid JSON frame: {error}"- ))- })?;- Ok(ParsedSseFrame::Json(value))-}--/// Finds the next CRLF or LF SSE frame boundary.-fn next_sse_boundary(buffer: &[u8]) -> Option<(usize, usize)> {- match (find_bytes(buffer, b"\r\n\r\n"), find_bytes(buffer, b"\n\n")) {- (Some(crlf), Some(lf)) if crlf < lf => Some((crlf, 4)),- (Some(_), Some(lf)) => Some((lf, 2)),- (Some(crlf), None) => Some((crlf, 4)),- (None, Some(lf)) => Some((lf, 2)),- (None, None) => None,- }-}--/// Finds a byte needle inside a byte haystack.-fn find_bytes(haystack: &[u8], needle: &[u8]) -> Option<usize> {- haystack- .windows(needle.len())- .position(|window| window == needle)-}--#[cfg(test)]-mod tests {- use serde_json::json;-- use super::*;-- // Multi-byte UTF-8 split across network chunks should wait for a full frame.- #[test]- fn buffers_incomplete_utf8_until_a_complete_sse_frame_arrives() -> Result<()> {- let mut buffer = b"data: {\"text\":\"".to_vec();- let multibyte = "é".as_bytes();- buffer.extend_from_slice(&multibyte[..1]);- assert!(drain_next_sse_frame(&mut buffer, "test")?.is_none());-- buffer.extend_from_slice(&multibyte[1..]);- buffer.extend_from_slice(b"\"}\n\n");-- let Some(frame) = drain_next_sse_frame(&mut buffer, "test")? else {- return Err(SwitchyardError::Other(- "complete SSE frame should be drained".to_string(),- ));- };- let ParsedSseFrame::Json(value) = parse_json_sse_frame(&frame, "test", None)? else {- return Err(SwitchyardError::Other(- "SSE frame should parse as JSON".to_string(),- ));- };- assert_eq!(value, json!({"text": "é"}));- assert!(buffer.is_empty());- Ok(())- }-}diff --git a/crates/switchyard-components/src/backends/context_overflow.rs b/crates/switchyard-components/src/backends/context_overflow.rsdeleted file mode 100644index 110a307a4..000000000--- a/crates/switchyard-components/src/backends/context_overflow.rs+++ /dev/null@@ -1,92 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Shared detection for upstream 4xx context-window-overflow bodies.-//!-//! Provider-specific detectors (`openai`, `anthropic`) supply a structured-//! check (against the parsed error envelope) and a list of substring phrases.-//! The common shell parses the body once, runs the structured check, then-//! falls back to matching phrases against `error.message` or — if the body-//! isn't JSON — the raw body. Centralising the shape here means each new-//! provider-wrap (e.g. NVIDIA/LiteLLM's wrapping of the canonical OpenAI-//! error) is one-line phrase entry per provider, not a duplicated rewrite.--use serde_json::Value;--/// Detect a context-overflow body using a provider-supplied structured check-/// and substring phrase list. See module docs for the matching strategy.-pub(super) fn is_overflow_body<F>(body: &str, structured_check: F, phrases: &[&str]) -> bool-where- F: Fn(&Value) -> bool,-{- if let Ok(value) = serde_json::from_str::<Value>(body) {- if structured_check(&value) {- return true;- }- if let Some(message) = value- .get("error")- .and_then(|err| err.get("message"))- .and_then(Value::as_str)- && contains_any(message, phrases)- {- return true;- }- }- // Some upstream proxies return plain-text bodies; fall through to a- // string match on the raw body.- contains_any(body, phrases)-}--fn contains_any(message: &str, phrases: &[&str]) -> bool {- let lower = message.to_ascii_lowercase();- phrases.iter().any(|phrase| lower.contains(phrase))-}--#[cfg(test)]-mod tests {- use super::*;-- const PHRASES: &[&str] = &["context window", "too long"];-- fn never(_value: &Value) -> bool {- false- }-- #[test]- fn structured_check_short_circuits() {- let body = r#"{"error":{"code":"context_length_exceeded","message":"unrelated"}}"#;- let matched = is_overflow_body(- body,- |value| {- value- .get("error")- .and_then(|err| err.get("code"))- .and_then(Value::as_str)- == Some("context_length_exceeded")- },- &[], // empty phrases — structured check is the only path- );- assert!(matched);- }-- #[test]- fn falls_back_to_message_phrase_match() {- let body = r#"{"error":{"message":"prompt too long"}}"#;- assert!(is_overflow_body(body, never, PHRASES));- }-- #[test]- fn matches_plain_text_body() {- assert!(is_overflow_body(- "plain text mentioning context window",- never,- PHRASES- ));- }-- #[test]- fn non_match_returns_false() {- let body = r#"{"error":{"message":"rate limit exceeded"}}"#;- assert!(!is_overflow_body(body, never, PHRASES));- }-}diff --git a/crates/switchyard-components/src/backends/mod.rs b/crates/switchyard-components/src/backends/mod.rsdeleted file mode 100644index 7d20b1011..000000000--- a/crates/switchyard-components/src/backends/mod.rs+++ /dev/null@@ -1,18 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Built-in backend implementations.--pub mod anthropic;-mod common;-mod context_overflow;-pub mod multi;-pub mod openai;-mod selection;-pub mod stats;--pub use anthropic::AnthropicNativeBackend;-pub use multi::{LlmTargetBackend, MultiLlmBackend};-pub use openai::{OpenAiNativeBackend, OpenAiPassthroughBackend};-pub use selection::{BackendSelection, BackendSelectionReason};-pub use stats::StatsLlmBackend;diff --git a/crates/switchyard-components/src/backends/multi.rs b/crates/switchyard-components/src/backends/multi.rsdeleted file mode 100644index 0180adea4..000000000--- a/crates/switchyard-components/src/backends/multi.rs+++ /dev/null@@ -1,330 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Multi-target LLM backend dispatch.-//!-//! This backend owns the mechanical part of routing once a processor or caller-//! has selected a target: rewrite the request model, stamp typed context, and-//! delegate to the configured backend for that target. Selection policy stays-//! outside this type; processors and algorithms decide which target should run.--use std::collections::HashSet;-use std::fmt;-use std::sync::Arc;--use crate::{- ChatRequest, ChatRequestType, ChatResponse, LlmBackend, LlmTarget, LlmTargetId, ProxyContext,- Result, SwitchyardError,-};-use async_trait::async_trait;--use super::{BackendSelection, BackendSelectionReason};--const DEFAULT_SUPPORTED_REQUEST_TYPES: [ChatRequestType; 3] = [- ChatRequestType::OpenAiChat,- ChatRequestType::OpenAiResponses,- ChatRequestType::Anthropic,-];--/// One configured upstream target and the backend that can call it.-#[derive(Clone)]-pub struct LlmTargetBackend {- // Target metadata used for model rewriting and public stats.- target: LlmTarget,- // Backend implementation that executes calls for the target.- backend: Arc<dyn LlmBackend>,-}--impl LlmTargetBackend {- /// Creates a target/backend pair.- pub fn new(target: LlmTarget, backend: Arc<dyn LlmBackend>) -> Self {- Self { target, backend }- }-- /// Returns the target metadata.- pub fn target(&self) -> &LlmTarget {- &self.target- }-- /// Returns the backend configured for the target.- pub fn backend(&self) -> &dyn LlmBackend {- self.backend.as_ref()- }-}--impl fmt::Debug for LlmTargetBackend {- fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {- formatter- .debug_struct("LlmTargetBackend")- .field("target", &self.target)- .finish_non_exhaustive()- }-}--/// Backend that delegates each request to one of several configured targets.-#[derive(Clone)]-pub struct MultiLlmBackend {- // Target/backends are stored in lifecycle order.- targets: Vec<LlmTargetBackend>,- // Request formats advertised by this backend.- supported_request_types: Vec<ChatRequestType>,- // Deterministic fallback when no router selected a target.- default_target_id: Option<LlmTargetId>,-}--impl MultiLlmBackend {- /// Creates a multi-target backend with support for all Switchyard request formats.- pub fn new(targets: impl IntoIterator<Item = LlmTargetBackend>) -> Result<Self> {- let targets = targets.into_iter().collect::<Vec<_>>();- validate_targets(&targets)?;- Ok(Self {- targets,- supported_request_types: DEFAULT_SUPPORTED_REQUEST_TYPES.to_vec(),- default_target_id: None,- })- }-- /// Replaces the advertised request formats.- pub fn with_supported_request_types(- mut self,- request_types: impl IntoIterator<Item = ChatRequestType>,- ) -> Result<Self> {- self.supported_request_types = normalize_request_types(request_types)?;- Ok(self)- }-- /// Sets the target used when no router selected a target.- pub fn with_default_target(mut self, target_id: LlmTargetId) -> Result<Self> {- if self.target(&target_id).is_none() {- return Err(SwitchyardError::InvalidConfig(format!(- "default target {target_id} is not configured; known targets: {}",- self.known_target_ids()- )));- }- self.default_target_id = Some(target_id);- Ok(self)- }-- /// Returns the deterministic default target, when configured.- pub fn default_target_id(&self) -> Option<&LlmTargetId> {- self.default_target_id.as_ref()- }-- /// Returns configured target/backend pairs in lifecycle order.- pub fn targets(&self) -> &[LlmTargetBackend] {- &self.targets- }-- /// Looks up a configured target by ID.- pub fn target(&self, target_id: &LlmTargetId) -> Option<&LlmTargetBackend> {- self.targets- .iter()- .find(|entry| &entry.target.id == target_id)- }-- fn selected_target<'a>(- &'a self,- ctx: &ProxyContext,- request: &ChatRequest,- ) -> Result<(&'a LlmTargetBackend, BackendSelectionReason)> {- // Explicit context selection wins because request processors are the- // routing policy layer.- if let Some(target_id) = ctx.selected_target() {- let Some(target) = self.target(target_id) else {- return Err(SwitchyardError::InvalidConfig(format!(- "selected target {target_id} is not configured; known targets: {}",- self.known_target_ids()- )));- };- return Ok((target, BackendSelectionReason::ContextTarget));- }-- // A configured default only applies when no processor selected a target.- if let Some(target_id) = &self.default_target_id {- let Some(target) = self.target(target_id) else {- return Err(SwitchyardError::InvalidConfig(format!(- "default target {target_id} is not configured; known targets: {}",- self.known_target_ids()- )));- };- return Ok((target, BackendSelectionReason::DefaultTarget));- }-- // A single-target backend is unambiguous even without a router.- if self.targets.len() == 1 {- let Some(target) = self.targets.first() else {- return Err(SwitchyardError::InvalidConfig(- "MultiLlmBackend requires at least one target".to_string(),- ));- };- return Ok((target, BackendSelectionReason::SingleTarget));- }-- // As a final convenience, match the request model to a unique target- // model. Duplicate matches remain ambiguous.- self.target_for_request_model(request)- }-- /// Selects a target by matching the request model to configured target models.- fn target_for_request_model<'a>(- &'a self,- request: &ChatRequest,- ) -> Result<(&'a LlmTargetBackend, BackendSelectionReason)> {- let Some(model) = request.model() else {- return Err(self.missing_selection_error(None));- };-- let matches = self- .targets- .iter()- .filter(|entry| entry.target.model.as_str() == model)- .collect::<Vec<_>>();-- match matches.as_slice() {- [target] => Ok((*target, BackendSelectionReason::RequestModel)),- [] => Err(self.missing_selection_error(Some(model))),- _ => Err(SwitchyardError::InvalidConfig(format!(- "request model {model:?} matches multiple targets; set selected_target explicitly"- ))),- }- }-- /// Builds the error used when no routing signal identifies one target.- fn missing_selection_error(&self, request_model: Option<&str>) -> SwitchyardError {- let request_model = request_model- .map(|model| format!(" and request model {model:?} did not match a configured target"))- .unwrap_or_default();- SwitchyardError::InvalidConfig(format!(- "MultiLlmBackend has multiple targets but no selected target{request_model}; known targets: {}",- self.known_target_ids()- ))- }-- /// Formats configured target IDs for diagnostics.- fn known_target_ids(&self) -> String {- self.targets- .iter()- .map(|entry| entry.target.id.to_string())- .collect::<Vec<_>>()- .join(", ")- }-}--impl fmt::Debug for MultiLlmBackend {- fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {- formatter- .debug_struct("MultiLlmBackend")- .field(- "targets",- &self- .targets- .iter()- .map(|entry| &entry.target)- .collect::<Vec<_>>(),- )- .field("supported_request_types", &self.supported_request_types)- .field("default_target_id", &self.default_target_id)- .finish()- }-}--#[async_trait]-impl LlmBackend for MultiLlmBackend {- // Returns request formats this backend accepts before delegation.- fn supported_request_types(&self) -> &[ChatRequestType] {- &self.supported_request_types- }-- // Rewrites the request model to the selected target and delegates once.- async fn call(&self, ctx: &mut ProxyContext, request: &ChatRequest) -> Result<ChatResponse> {- let request_type = request.request_type();- if !self.supported_request_types.contains(&request_type) {- return Err(SwitchyardError::UnsupportedRequestType {- component: "MultiLlmBackend".to_string(),- request_type,- });- }-- let (target, reason) = self.selected_target(ctx, request)?;- let mut routed_request = request.clone();- routed_request.set_model(target.target.model.as_str());-- // Stamping BackendSelection lets stats processors attribute the final- // provider call without coupling to this backend's internals.- let _ = ctx.insert(BackendSelection::for_target(- target.target.id.clone(),- target.target.model.clone(),- request.model().map(str::to_string),- reason,- ));-- target.backend.call(ctx, &routed_request).await- }-- // Starts child backends in configured order and rolls back on failure.- async fn startup(&self) -> Result<()> {- let mut started: Vec<&LlmTargetBackend> = Vec::new();- for target in &self.targets {- if let Err(error) = target.backend.startup().await {- for started_target in started.into_iter().rev() {- let _ = started_target.backend.shutdown().await;- }- return Err(error);- }- started.push(target);- }- Ok(())- }-- // Shuts child backends down in reverse order while preserving first error.- async fn shutdown(&self) -> Result<()> {- let mut first_error = None;- for target in self.targets.iter().rev() {- if let Err(error) = target.backend.shutdown().await {- first_error.get_or_insert(error);- }- }- match first_error {- Some(error) => Err(error),- None => Ok(()),- }- }-}--/// Validates constructor invariants for target lists.-fn validate_targets(targets: &[LlmTargetBackend]) -> Result<()> {- if targets.is_empty() {- return Err(SwitchyardError::InvalidConfig(- "MultiLlmBackend requires at least one target".to_string(),- ));- }-- let mut seen = HashSet::new();- for target in targets {- if !seen.insert(target.target.id.clone()) {- return Err(SwitchyardError::InvalidConfig(format!(- "duplicate LLM target id: {}",- target.target.id- )));- }- }- Ok(())-}--/// Deduplicates request types while preserving caller order.-fn normalize_request_types(- request_types: impl IntoIterator<Item = ChatRequestType>,-) -> Result<Vec<ChatRequestType>> {- let mut normalized = Vec::new();- for request_type in request_types {- if !normalized.contains(&request_type) {- normalized.push(request_type);- }- }- if normalized.is_empty() {- return Err(SwitchyardError::InvalidConfig(- "MultiLlmBackend must support at least one request type".to_string(),- ));- }- Ok(normalized)-}diff --git a/crates/switchyard-components/src/backends/openai.rs b/crates/switchyard-components/src/backends/openai.rsdeleted file mode 100644index 7cdaf5550..000000000--- a/crates/switchyard-components/src/backends/openai.rs+++ /dev/null@@ -1,778 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! OpenAI-compatible backend for Chat Completions or Responses targets.--use std::collections::BTreeMap;-use std::env;-use std::fmt;-use std::sync::Arc;--use crate::{- BackendFormat, BoxResponseStream, ChatRequest, ChatRequestType, ChatResponse, EndpointConfig,- LlmBackend, LlmTarget, LlmTargetId, ModelId, ProxyContext, Result, StreamEvent,- SwitchyardError, merge_target_extra_body,-};-use async_stream::try_stream;-use async_trait::async_trait;-use futures_util::StreamExt;-use serde_json::{Map, Value};-use switchyard_translation::{TranslationEngine, TranslationPolicy, WireFormat};--use super::common::{- ParsedSseFrame, build_reqwest_client, decode_sse_frame, drain_next_sse_frame,- has_non_whitespace_bytes, parse_json_sse_frame, request_wire_format, set_json_model,- shared_translation_engine,-};-use super::{BackendSelection, BackendSelectionReason};-use crate::telemetry::{SWITCHYARD_VERSION_HEADER, telemetry_header_value};--const DEFAULT_OPENAI_BASE_URL: &str = "https://api.openai.com/v1";-const OPENAI_API_KEY_ENV: &str = "OPENAI_API_KEY";-static OPENAI_CHAT_ONLY: [ChatRequestType; 1] = [ChatRequestType::OpenAiChat];-static OPENAI_RESPONSES_ONLY: [ChatRequestType; 1] = [ChatRequestType::OpenAiResponses];-static OPENAI_PASSTHROUGH_TARGET_ID: &str = "passthrough";--/// Backend that calls an OpenAI-compatible Chat Completions or Responses API.-pub struct OpenAiNativeBackend {- /// Resolved target used for endpoint credentials and model rewriting.- target: LlmTarget,- /// HTTP transport, injectable for deterministic tests.- transport: Arc<dyn OpenAiTransport>,- /// Shared request translator for non-OpenAI inbound payloads.- translation: Arc<TranslationEngine>,- /// Translation policy kept explicit so future server policy remains visible.- translation_policy: TranslationPolicy,-}--impl fmt::Debug for OpenAiNativeBackend {- fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {- formatter- .debug_struct("OpenAiNativeBackend")- .field("target", &self.target)- .finish_non_exhaustive()- }-}--impl OpenAiNativeBackend {- /// Creates an OpenAI-compatible backend for one target.- pub fn new(target: LlmTarget) -> Result<Self> {- let transport = Arc::new(ReqwestOpenAiTransport::new(target.endpoint.timeout_secs)?);- Self::with_transport(target, transport)- }-- /// Returns the configured upstream target.- pub fn target(&self) -> &LlmTarget {- &self.target- }-- fn with_transport(target: LlmTarget, transport: Arc<dyn OpenAiTransport>) -> Result<Self> {- validate_target_format(&target)?;- Ok(Self {- target,- transport,- translation: shared_translation_engine(),- translation_policy: TranslationPolicy::default(),- })- }-- fn target_request_type(&self) -> ChatRequestType {- match self.target.format {- BackendFormat::OpenAi => ChatRequestType::OpenAiChat,- BackendFormat::Responses => ChatRequestType::OpenAiResponses,- BackendFormat::Auto | BackendFormat::Anthropic => {- unreachable!("OpenAiNativeBackend target format is validated at construction")- }- }- }-- fn target_wire_format(&self) -> WireFormat {- match self.target_request_type() {- ChatRequestType::OpenAiChat => WireFormat::OpenAiChat,- ChatRequestType::OpenAiResponses => WireFormat::OpenAiResponses,- ChatRequestType::Anthropic => {- unreachable!("OpenAiNativeBackend only targets OpenAI wire formats")- }- }- }-- fn outbound_body(&self, request: &ChatRequest) -> Result<Value> {- let target_request_type = self.target_request_type();- let mut body = match request.request_type() {- source if source == target_request_type => request.body().clone(),- source => {- self.translation- .translate_request(- request_wire_format(source),- self.target_wire_format(),- request.body(),- &self.translation_policy,- )- .map_err(|error| {- SwitchyardError::Backend(format!(- "failed to translate {source:?} request to {:?}: {error}",- self.target.format- ))- })?- .body- }- };- set_json_model(&mut body, self.target.model.as_str());- if self.target.format == BackendFormat::OpenAi {- ensure_stream_usage(&mut body);- }- // Merge per-target ``extra_body`` last so e.g. DeepSeek V4's- // ``chat_template_kwargs.enable_thinking=False`` reaches the- // upstream. Caller wins on key conflicts (see- // :func:`merge_target_extra_body`).- merge_target_extra_body(&mut body, self.target.extra_body.as_ref());- Ok(body)- }-- /// Calls this target without requiring chain-local `ProxyContext` state.- pub async fn call_without_context(&self, request: &ChatRequest) -> Result<ChatResponse> {- let http_request = self.http_request(request)?;- self.send_http_request(http_request).await- }-- // Builds the upstream HTTP request before any context observations are recorded.- fn http_request(&self, request: &ChatRequest) -> Result<OpenAiHttpRequest> {- let body = self.outbound_body(request)?;- let stream = body.get("stream").and_then(Value::as_bool).unwrap_or(false);- let endpoint = endpoint_for_backend_format(self.target.format)?;- Ok(OpenAiHttpRequest {- target_id: self.target.id.clone(),- url: openai_url(self.target.endpoint.base_url.as_deref(), endpoint),- api_key: openai_api_key(self.target.endpoint.api_key.as_deref()),- body,- stream,- extra_headers: self.target.extra_headers.clone(),- endpoint,- })- }-- // Sends an already-normalized upstream request.- async fn send_http_request(&self, request: OpenAiHttpRequest) -> Result<ChatResponse> {- let endpoint = request.endpoint;- match self.transport.send(request).await? {- OpenAiHttpResponse::Buffered(body) => match endpoint {- OpenAiEndpoint::ChatCompletions => Ok(ChatResponse::openai_completion(body)),- OpenAiEndpoint::Responses => Ok(ChatResponse::openai_responses_completion(body)),- },- OpenAiHttpResponse::Stream(stream) => match endpoint {- OpenAiEndpoint::ChatCompletions => Ok(ChatResponse::OpenAiStream(stream)),- OpenAiEndpoint::Responses => Ok(ChatResponse::OpenAiResponsesStream(stream)),- },- }- }-}--/// Backend that calls an OpenAI-compatible Chat Completions API without rewriting `model`.-pub struct OpenAiPassthroughBackend {- /// Endpoint used without an owning LLM target or model rewrite.- endpoint: EndpointConfig,- /// HTTP transport, injectable for tests.- transport: Arc<dyn OpenAiTransport>,- /// Shared request translator for supported non-OpenAI inbound payloads.- translation: Arc<TranslationEngine>,- /// Translation policy kept local to the backend.- translation_policy: TranslationPolicy,-}--impl fmt::Debug for OpenAiPassthroughBackend {- fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {- formatter- .debug_struct("OpenAiPassthroughBackend")- .field("base_url", &self.endpoint.base_url)- .field("timeout_secs", &self.endpoint.timeout_secs)- .finish_non_exhaustive()- }-}--impl OpenAiPassthroughBackend {- /// Creates a passthrough OpenAI-compatible backend.- pub fn new(endpoint: EndpointConfig) -> Result<Self> {- let transport = Arc::new(ReqwestOpenAiTransport::new(endpoint.timeout_secs)?);- Self::with_transport(endpoint, transport)- }-- /// Returns the configured upstream endpoint.- pub fn endpoint(&self) -> &EndpointConfig {- &self.endpoint- }-- fn with_transport(- endpoint: EndpointConfig,- transport: Arc<dyn OpenAiTransport>,- ) -> Result<Self> {- Ok(Self {- endpoint,- transport,- translation: shared_translation_engine(),- translation_policy: TranslationPolicy::default(),- })- }-- fn outbound_body(&self, request: &ChatRequest) -> Result<Value> {- let mut body = match request.request_type() {- ChatRequestType::OpenAiChat => request.body().clone(),- source => {- self.translation- .translate_request(- request_wire_format(source),- WireFormat::OpenAiChat,- request.body(),- &self.translation_policy,- )- .map_err(|error| {- SwitchyardError::Backend(format!(- "failed to translate {source:?} request to OpenAI Chat: {error}"- ))- })?- .body- }- };- ensure_stream_usage(&mut body);- Ok(body)- }-}--#[async_trait]-impl LlmBackend for OpenAiNativeBackend {- fn supported_request_types(&self) -> &[ChatRequestType] {- match self.target.format {- BackendFormat::OpenAi => &OPENAI_CHAT_ONLY,- BackendFormat::Responses => &OPENAI_RESPONSES_ONLY,- BackendFormat::Auto | BackendFormat::Anthropic => {- unreachable!("OpenAiNativeBackend target format is validated at construction")- }- }- }-- async fn call(&self, ctx: &mut ProxyContext, request: &ChatRequest) -> Result<ChatResponse> {- let http_request = self.http_request(request)?;-- ctx.inbound_format = ctx.inbound_format.or(Some(request.request_type()));- let previous_selection = ctx.get::<BackendSelection>().cloned();- ctx.insert(BackendSelection::native_target_observation(- previous_selection.as_ref(),- self.target.id.clone(),- self.target.model.clone(),- request.model().map(str::to_string),- ));-- self.send_http_request(http_request).await- }-}--#[async_trait]-impl LlmBackend for OpenAiPassthroughBackend {- fn supported_request_types(&self) -> &[ChatRequestType] {- &OPENAI_CHAT_ONLY- }-- async fn call(&self, ctx: &mut ProxyContext, request: &ChatRequest) -> Result<ChatResponse> {- let body = self.outbound_body(request)?;- let stream = body.get("stream").and_then(Value::as_bool).unwrap_or(false);- let http_request = OpenAiHttpRequest {- target_id: LlmTargetId::from_static(OPENAI_PASSTHROUGH_TARGET_ID),- url: openai_url(- self.endpoint.base_url.as_deref(),- OpenAiEndpoint::ChatCompletions,- ),- api_key: openai_api_key(self.endpoint.api_key.as_deref()),- body,- stream,- extra_headers: BTreeMap::new(),- endpoint: OpenAiEndpoint::ChatCompletions,- };-- ctx.inbound_format = ctx.inbound_format.or(Some(request.request_type()));- if let Some(model) = http_request- .body- .get("model")- .and_then(Value::as_str)- .and_then(|model| ModelId::new(model.to_string()).ok())- {- ctx.insert(BackendSelection::for_model(- model,- request.model().map(str::to_string),- BackendSelectionReason::PassthroughModel,- ));- }-- match self.transport.send(http_request).await? {- OpenAiHttpResponse::Buffered(body) => Ok(ChatResponse::openai_completion(body)),- OpenAiHttpResponse::Stream(stream) => Ok(ChatResponse::OpenAiStream(stream)),- }- }-}--#[derive(Clone, Debug, PartialEq)]-struct OpenAiHttpRequest {- /// Target ID used only for logging and diagnostics.- target_id: LlmTargetId,- /// Fully resolved OpenAI-compatible endpoint URL.- url: String,- /// Per-target API key or process environment fallback.- api_key: Option<String>,- /// Already-normalized OpenAI-compatible request body.- body: Value,- /// Whether the upstream call should be treated as SSE.- stream: bool,- /// Per-target extra headers (e.g. ``X-Inference-Priority: batch``- /// for NIH evals gateway routing on DeepSeek V4). Empty for the- /// passthrough backend, which has no LlmTarget.- extra_headers: BTreeMap<String, String>,- /// Upstream endpoint family used for response wrapping and diagnostics.- endpoint: OpenAiEndpoint,-}--#[derive(Clone, Copy, Debug, Eq, PartialEq)]-enum OpenAiEndpoint {- /// OpenAI Chat Completions API.- ChatCompletions,- /// OpenAI Responses API.- Responses,-}--impl OpenAiEndpoint {- fn label(self) -> &'static str {- match self {- Self::ChatCompletions => "OpenAI chat completions",- Self::Responses => "OpenAI responses",- }- }-}--enum OpenAiHttpResponse {- /// Complete JSON response from a non-streaming upstream call.- Buffered(Value),- /// Streamed SSE response converted into Switchyard stream events.- Stream(BoxResponseStream),-}--#[async_trait]-trait OpenAiTransport: Send + Sync {- /// Sends one already-normalized OpenAI-compatible request.- async fn send(&self, request: OpenAiHttpRequest) -> Result<OpenAiHttpResponse>;-}--struct ReqwestOpenAiTransport {- /// Reused async HTTP client with configured timeout behavior.- client: reqwest::Client,-}--impl ReqwestOpenAiTransport {- fn new(timeout_secs: Option<f64>) -> Result<Self> {- let client = build_reqwest_client("OpenAI", timeout_secs)?;- Ok(Self { client })- }-}--#[async_trait]-impl OpenAiTransport for ReqwestOpenAiTransport {- async fn send(&self, request: OpenAiHttpRequest) -> Result<OpenAiHttpResponse> {- let target_id = request.target_id.clone();- let endpoint = request.endpoint;- let mut builder = self.client.post(&request.url).json(&request.body);- if let Some(api_key) = request.api_key {- builder = builder.bearer_auth(api_key);- }- if let Some(version) = telemetry_header_value() {- builder = builder.header(SWITCHYARD_VERSION_HEADER, version);- }- for (name, value) in &request.extra_headers {- builder = builder.header(name, value);- }-- let response = builder.send().await.map_err(|error| {- tracing::warn!(- target_id = %target_id,- error = %error,- endpoint = endpoint.label(),- "OpenAI request failed"- );- SwitchyardError::Upstream(format!("{} request failed: {error}", endpoint.label()))- })?;- let status = response.status();- if !status.is_success() {- let body = response- .text()- .await- .unwrap_or_else(|error| format!("<failed to read error body: {error}>"));- tracing::warn!(- target_id = %target_id,- status = %status,- endpoint = endpoint.label(),- "OpenAI request returned error status"- );- if status == reqwest::StatusCode::BAD_REQUEST && is_context_overflow(&body) {- let model = request- .body- .get("model")- .and_then(Value::as_str)- .unwrap_or("")- .to_string();- return Err(SwitchyardError::ContextWindowExceeded {- target_id: target_id.to_string(),- model,- message: body,- });- }- return Err(SwitchyardError::UpstreamHttp {- provider: endpoint.label().to_string(),- status_code: status.as_u16(),- body,- });- }-- if request.stream {- return Ok(OpenAiHttpResponse::Stream(openai_sse_stream(response)));- }-- let body = response.json::<Value>().await.map_err(|error| {- SwitchyardError::Upstream(format!(- "{} returned invalid JSON: {error}",- endpoint.label()- ))- })?;- Ok(OpenAiHttpResponse::Buffered(body))- }-}--fn validate_target_format(target: &LlmTarget) -> Result<()> {- match target.format {- BackendFormat::OpenAi | BackendFormat::Responses => Ok(()),- BackendFormat::Auto | BackendFormat::Anthropic => {- Err(SwitchyardError::InvalidConfig(format!(- "OpenAiNativeBackend requires a target with resolved OpenAI format, got {:?} for {}",- target.format, target.id- )))- }- }-}--fn endpoint_for_backend_format(format: BackendFormat) -> Result<OpenAiEndpoint> {- match format {- BackendFormat::OpenAi => Ok(OpenAiEndpoint::ChatCompletions),- BackendFormat::Responses => Ok(OpenAiEndpoint::Responses),- BackendFormat::Auto | BackendFormat::Anthropic => Err(SwitchyardError::InvalidConfig(- format!("OpenAiNativeBackend cannot dispatch target format {format:?}"),- )),- }-}--// OpenAI-compatible streaming users need usage events for stats accounting.-fn ensure_stream_usage(body: &mut Value) {- let Value::Object(object) = body else {- return;- };- if !object- .get("stream")- .and_then(Value::as_bool)- .unwrap_or(false)- {- return;- }-- match object.get_mut("stream_options") {- Some(Value::Object(options)) => {- options- .entry("include_usage".to_string())- .or_insert(Value::Bool(true));- }- _ => {- let mut options = Map::new();- options.insert("include_usage".to_string(), Value::Bool(true));- object.insert("stream_options".to_string(), Value::Object(options));- }- }-}--// Accept either a root `/v1` URL or an already-specific OpenAI endpoint URL.-fn openai_url(base_url: Option<&str>, endpoint: OpenAiEndpoint) -> String {- let base_url = base_url- .unwrap_or(DEFAULT_OPENAI_BASE_URL)- .trim_end_matches('/');- let base_root = base_url- .strip_suffix("/chat/completions")- .or_else(|| base_url.strip_suffix("/responses"))- .unwrap_or(base_url);- let suffix = match endpoint {- OpenAiEndpoint::ChatCompletions => "/chat/completions",- OpenAiEndpoint::Responses => "/responses",- };- format!("{base_root}{suffix}")-}--fn openai_api_key(configured: Option<&str>) -> Option<String> {- // Resolve per call so long-lived backends can pick up rotated environment credentials.- configured- .map(str::to_string)- .or_else(|| env::var(OPENAI_API_KEY_ENV).ok())- .filter(|value| !value.trim().is_empty())-}--// Best-effort match for OpenAI-shape context-window-overflow error bodies.-// Canonical signal is `error.code == "context_length_exceeded"`; NVIDIA and-// other proxies sometimes only set the human-readable message, so we fall back-// to substring matching. Only reached for upstream 400s, and a false positive-// triggers a single bounded evict-and-retry (never an infinite loop), so erring-// toward matching the message is safe.-// OpenAI canonical phrase + NVIDIA/LiteLLM wrap variants. Adding a new-// provider-wrap is a one-line entry here, not a fork of the parsing logic.-const OPENAI_OVERFLOW_PHRASES: &[&str] = &[- "maximum context length",- "context length exceeded",- "context window",- "context length is only",- "please reduce the length of the input",- "exceeds the maximum allowed input length",-];--fn is_context_overflow(body: &str) -> bool {- super::context_overflow::is_overflow_body(- body,- |value| {- value- .get("error")- .and_then(|err| err.get("code"))- .and_then(Value::as_str)- == Some("context_length_exceeded")- },- OPENAI_OVERFLOW_PHRASES,- )-}--fn openai_sse_stream(response: reqwest::Response) -> BoxResponseStream {- Box::pin(try_stream! {- let mut chunks = response.bytes_stream();- let mut buffer = Vec::new();-- while let Some(chunk) = chunks.next().await {- let chunk = chunk.map_err(|error| {- SwitchyardError::Upstream(format!("OpenAI stream read failed: {error}"))- })?;- buffer.extend_from_slice(&chunk);-- // Drain complete frames immediately while preserving partial frames- // across TCP chunks.- while let Some(frame) = drain_next_sse_frame(&mut buffer, "OpenAI")? {- match parse_json_sse_frame(&frame, "OpenAI", Some("[DONE]"))? {- ParsedSseFrame::Json(value) => yield StreamEvent::Json(value),- ParsedSseFrame::Done => return,- ParsedSseFrame::Empty => {}- }- }- }-- // A non-standard upstream might omit the final double newline; parse a- // trailing complete frame instead of losing its usage chunk.- if has_non_whitespace_bytes(&buffer) {- let frame = decode_sse_frame(&buffer, "OpenAI")?;- match parse_json_sse_frame(&frame, "OpenAI", Some("[DONE]"))? {- ParsedSseFrame::Json(value) => yield StreamEvent::Json(value),- ParsedSseFrame::Done | ParsedSseFrame::Empty => {}- }- }- })-}--#[cfg(test)]-mod tests {- use crate::{EndpointConfig, LlmTargetId, ModelId};- use parking_lot::Mutex;- use serde_json::json;-- use super::*;-- struct FakeOpenAiTransport {- requests: Mutex<Vec<OpenAiHttpRequest>>,- response: Mutex<Option<Result<OpenAiHttpResponse>>>,- }-- impl FakeOpenAiTransport {- fn with_error(message: &str) -> Self {- Self {- requests: Mutex::new(Vec::new()),- response: Mutex::new(Some(Err(SwitchyardError::Upstream(message.to_string())))),- }- }- }-- #[async_trait]- impl OpenAiTransport for FakeOpenAiTransport {- async fn send(&self, request: OpenAiHttpRequest) -> Result<OpenAiHttpResponse> {- self.requests.lock().push(request);- self.response.lock().take().ok_or_else(|| {- SwitchyardError::Other("fake transport response already consumed".to_string())- })?- }- }-- fn openai_target() -> LlmTarget {- LlmTarget {- id: LlmTargetId::from_static("primary"),- model: ModelId::from_static("target-model"),- format: BackendFormat::OpenAi,- endpoint: EndpointConfig {- base_url: Some("https://example.test/v1".to_string()),- api_key: Some("secret".to_string()),- timeout_secs: None,- },- extra_body: None,- extra_headers: BTreeMap::new(),- }- }-- #[test]- fn outbound_body_merges_target_extra_body() -> Result<()> {- // Use-case: DeepSeek V4 on NVIDIA Inference Hub. The target sets- // ``chat_template_kwargs.enable_thinking=False`` so V4 skips its- // chain-of-thought pass. Without this the model 504s at -n 8- // concurrency from the Hub gateway timeout.- let mut target = openai_target();- target.extra_body = Some(json!({- "chat_template_kwargs": {"enable_thinking": false}- }));- let transport = Arc::new(FakeOpenAiTransport::with_error("ignored"));- let backend = OpenAiNativeBackend::with_transport(target, transport)?;- let request = ChatRequest::openai_chat(json!({- "model": "client-model",- "messages": [{"role": "user", "content": "hi"}],- }));- let body = backend.outbound_body(&request)?;- assert_eq!(- body.get("chat_template_kwargs"),- Some(&json!({"enable_thinking": false})),- "target.extra_body should land at the top level of the outbound body",- );- // Caller fields preserved.- assert_eq!(- body.get("model").and_then(|v| v.as_str()),- Some("target-model"),- "outbound_body should rewrite model to the target's",- );- Ok(())- }-- #[test]- fn outbound_body_caller_wins_on_extra_body_key_conflict() -> Result<()> {- let mut target = openai_target();- target.extra_body = Some(json!({- "chat_template_kwargs": {"enable_thinking": false},- "logit_bias": {"50256": -100},- }));- let transport = Arc::new(FakeOpenAiTransport::with_error("ignored"));- let backend = OpenAiNativeBackend::with_transport(target, transport)?;- // Caller sets chat_template_kwargs explicitly with thinking=true.- let request = ChatRequest::openai_chat(json!({- "model": "client-model",- "messages": [],- "chat_template_kwargs": {"enable_thinking": true},- }));- let body = backend.outbound_body(&request)?;- // Caller-supplied chat_template_kwargs wins on the top-level key- // (the merge is shallow / caller-wins). Target's logit_bias- // still lands because caller didn't set it.- assert_eq!(- body.get("chat_template_kwargs"),- Some(&json!({"enable_thinking": true})),- );- assert_eq!(body.get("logit_bias"), Some(&json!({"50256": -100})),);- Ok(())- }-- #[tokio::test]- async fn transport_errors_are_backend_errors() -> Result<()> {- let transport = Arc::new(FakeOpenAiTransport::with_error("upstream exploded"));- let backend = OpenAiNativeBackend::with_transport(openai_target(), transport)?;- let request = ChatRequest::openai_chat(json!({- "model": "client-model",- "messages": [],- }));- let mut ctx = ProxyContext::new();-- let Err(error) = backend.call(&mut ctx, &request).await else {- return Err(SwitchyardError::Other(- "backend call should fail".to_string(),- ));- };-- assert!(matches!(error, SwitchyardError::Upstream(_)));- assert!(error.to_string().contains("upstream exploded"));- Ok(())- }-- #[test]- fn rejects_anthropic_targets() -> Result<()> {- let mut target = openai_target();- target.format = BackendFormat::Anthropic;-- let Err(error) = OpenAiNativeBackend::new(target) else {- return Err(SwitchyardError::Other(- "Anthropic target should be rejected".to_string(),- ));- };-- assert!(matches!(error, SwitchyardError::InvalidConfig(_)));- Ok(())- }-- #[test]- fn parses_openai_sse_json_frames_and_done() -> Result<()> {- let ParsedSseFrame::Json(value) = parse_json_sse_frame(- "event: message\ndata: {\"choices\":[]}\n",- "OpenAI",- Some("[DONE]"),- )?- else {- return Err(SwitchyardError::Other(- "JSON frame should produce a JSON value".to_string(),- ));- };- assert_eq!(value, json!({"choices": []}));-- let ParsedSseFrame::Done =- parse_json_sse_frame("data: [DONE]\n", "OpenAI", Some("[DONE]"))?- else {- return Err(SwitchyardError::Other("DONE frame should stop".to_string()));- };- Ok(())- }-- #[test]- fn context_overflow_canonical_code_matches() {- let body = r#"{"error":{"code":"context_length_exceeded","message":"x","type":"invalid_request_error"}}"#;- assert!(is_context_overflow(body));- }-- #[test]- fn context_overflow_nvidia_message_matches() {- let body = r#"{"error":{"message":"This model's maximum context length is 131072 tokens, however you requested ..."}}"#;- assert!(is_context_overflow(body));- }-- #[test]- fn context_overflow_unrelated_400_does_not_match() {- let body = r#"{"error":{"code":"invalid_api_key","message":"bad key"}}"#;- assert!(!is_context_overflow(body));- }-- #[test]- fn context_overflow_hub_glm_matches() {- // Hub GLM error via LiteLLM: code is "400" (not context_length_exceeded),- // so detection relies on phrase matching.- let body = r#"{"error":{"message":"Input length 877338 exceeds the maximum allowed input length of 639968 tokens","code":"400"}}"#;- assert!(is_context_overflow(body));- }-- #[test]- fn context_overflow_nvidia_litellm_wrap_matches() {- // Body shape observed from inference-api.nvidia.com's LiteLLM proxy- // wrapping a Nemotron context-window overflow.- let body = r#"{"error":{"message":"litellm.BadRequestError: OpenAIException - {\"error\":{\"message\":\"You passed 131041 input tokens and requested 32 output tokens. However, the model's context length is only 131072 tokens, resulting in a maximum input length of 131040 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=131041)\",\"type\":\"BadRequestError\",\"param\":\"input_tokens\",\"code\":400}}","type":null,"param":null,"code":"400"}}"#;- assert!(is_context_overflow(body));- }-}diff --git a/crates/switchyard-components/src/backends/selection.rs b/crates/switchyard-components/src/backends/selection.rsdeleted file mode 100644index 1416b5871..000000000--- a/crates/switchyard-components/src/backends/selection.rs+++ /dev/null@@ -1,92 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Backend execution metadata for observability components.--use crate::{LlmTargetId, ModelId};-use serde::{Deserialize, Serialize};--/// How a backend resolved the final upstream target/model for a request.-#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq, Serialize, Deserialize)]-#[serde(rename_all = "snake_case")]-pub enum BackendSelectionReason {- /// A router or caller set a selected target on `ProxyContext`.- ContextTarget,- /// The backend was configured with a deterministic default target.- DefaultTarget,- /// Only one target is configured, so there is no routing ambiguity.- SingleTarget,- /// The inbound request model uniquely matched a configured target model.- RequestModel,- /// A native backend has exactly one configured target.- NativeTarget,- /// A passthrough backend used the caller-provided model.- PassthroughModel,-}--/// Final upstream backend selection for a request.-#[derive(Clone, Debug, Eq, PartialEq, Serialize, Deserialize)]-pub struct BackendSelection {- /// Selected target ID when the backend resolved a concrete configured target.- pub target_id: Option<LlmTargetId>,- /// Final upstream model name used for the provider call.- pub model: ModelId,- /// Client-provided model name before backend routing or rewriting.- pub original_model: Option<String>,- /// Reason the backend selected this target/model.- pub reason: BackendSelectionReason,-}--impl BackendSelection {- /// Creates a selection for a concrete target-backed backend call.- pub fn for_target(- target_id: LlmTargetId,- model: ModelId,- original_model: Option<String>,- reason: BackendSelectionReason,- ) -> Self {- Self {- target_id: Some(target_id),- model,- original_model,- reason,- }- }-- /// Creates a selection for a backend call that only resolved a model.- pub fn for_model(- model: ModelId,- original_model: Option<String>,- reason: BackendSelectionReason,- ) -> Self {- Self {- target_id: None,- model,- original_model,- reason,- }- }-- /// Records a native backend call while preserving an upstream routing reason- /// when a parent backend already selected the same target/model.- pub fn native_target_observation(- previous: Option<&Self>,- target_id: LlmTargetId,- model: ModelId,- original_model: Option<String>,- ) -> Self {- let matching_previous = previous.filter(|selection| {- selection.target_id.as_ref() == Some(&target_id) && selection.model == model- });- Self {- target_id: Some(target_id),- model,- original_model: matching_previous- .and_then(|selection| selection.original_model.clone())- .or(original_model),- reason: matching_previous- .map(|selection| selection.reason)- .unwrap_or(BackendSelectionReason::NativeTarget),- }- }-}diff --git a/crates/switchyard-components/src/backends/stats.rs b/crates/switchyard-components/src/backends/stats.rsdeleted file mode 100644index 777f7026d..000000000--- a/crates/switchyard-components/src/backends/stats.rs+++ /dev/null@@ -1,88 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Backend wrapper that records backend-call stats.--use std::fmt;-use std::sync::Arc;-use std::time::Instant;--use crate::{ChatRequest, ChatRequestType, ChatResponse, LlmBackend, ProxyContext, Result};-use async_trait::async_trait;--use crate::stats::{- StatsAccumulator, StatsBackendLatency, selected_stats_model, selected_stats_tier,-};--/// Transparent backend wrapper that records call success/error and backend latency.-#[derive(Clone)]-pub struct StatsLlmBackend {- inner: Arc<dyn LlmBackend>,- accumulator: StatsAccumulator,-}--impl StatsLlmBackend {- /// Creates a stats wrapper around an existing backend.- pub fn new(inner: Arc<dyn LlmBackend>, accumulator: StatsAccumulator) -> Self {- Self { inner, accumulator }- }-- /// Returns the wrapped backend.- pub fn inner(&self) -> &dyn LlmBackend {- self.inner.as_ref()- }-- /// Returns the shared accumulator.- pub fn accumulator(&self) -> &StatsAccumulator {- &self.accumulator- }-}--impl fmt::Debug for StatsLlmBackend {- fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {- formatter- .debug_struct("StatsLlmBackend")- .field("accumulator", &self.accumulator)- .finish_non_exhaustive()- }-}--#[async_trait]-impl LlmBackend for StatsLlmBackend {- fn supported_request_types(&self) -> &[ChatRequestType] {- self.inner.supported_request_types()- }-- async fn call(&self, ctx: &mut ProxyContext, request: &ChatRequest) -> Result<ChatResponse> {- let request_model = request.model().map(str::to_string);- let started_at = Instant::now();- match self.inner.call(ctx, request).await {- Ok(response) => {- let latency = started_at.elapsed();- ctx.insert(StatsBackendLatency(latency));- let model = selected_stats_model(ctx, request_model.as_deref());- let tier = selected_stats_tier(ctx);- self.accumulator.record_success(- model,- Some(latency.as_secs_f64() * 1000.0),- tier.as_deref(),- )?;- Ok(response)- }- Err(error) => {- let model = selected_stats_model(ctx, request_model.as_deref());- let tier = selected_stats_tier(ctx);- self.accumulator.record_error(model, tier.as_deref())?;- Err(error)- }- }- }-- async fn startup(&self) -> Result<()> {- self.inner.startup().await- }-- async fn shutdown(&self) -> Result<()> {- self.inner.shutdown().await- }-}diff --git a/crates/switchyard-components/src/contracts/backend.rs b/crates/switchyard-components/src/contracts/backend.rsdeleted file mode 100644index 9beef8a1a..000000000--- a/crates/switchyard-components/src/contracts/backend.rs+++ /dev/null@@ -1,145 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! LLM target configuration shared by compatibility routing and factory code.--use std::collections::BTreeMap;--use serde::{Deserialize, Serialize};-use serde_json::Value;--use super::ids::{LlmTargetId, ModelId};--/// Wire format expected by an LLM target.-#[derive(Clone, Copy, Debug, Default, Eq, Hash, PartialEq, Serialize, Deserialize)]-#[serde(rename_all = "snake_case")]-pub enum BackendFormat {- /// Format has not been resolved yet and must not reach native backend build.- #[default]- Auto,- /// OpenAI-compatible Chat Completions fallback wire format.- #[serde(rename = "openai")]- OpenAi,- /// OpenAI Responses API wire format for native `/v1/responses` targets.- Responses,- /// Anthropic Messages wire format.- Anthropic,-}--/// Optional endpoint overrides for an LLM target.-#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]-#[serde(deny_unknown_fields)]-pub struct EndpointConfig {- /// Optional upstream base URL, usually ending in `/v1`.- pub base_url: Option<String>,- /// Optional upstream API key; environment fallback remains backend-specific.- pub api_key: Option<String>,- /// Optional upstream request timeout in seconds.- pub timeout_secs: Option<f64>,-}--impl EndpointConfig {- /// Merges a shared endpoint with target-local overrides.- ///- /// Fields set directly on the target win over fields inherited from the- /// shared endpoint definition.- pub fn with_overrides(&self, overrides: &Self) -> Self {- Self {- base_url: overrides.base_url.clone().or_else(|| self.base_url.clone()),- api_key: overrides.api_key.clone().or_else(|| self.api_key.clone()),- timeout_secs: overrides.timeout_secs.or(self.timeout_secs),- }- }-}--/// A concrete upstream model target that routing processors can select.-#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]-#[serde(deny_unknown_fields)]-pub struct LlmTarget {- /// Stable target ID used by routers and config references.- pub id: LlmTargetId,- /// Upstream model name sent to the provider.- pub model: ModelId,- /// Native wire format expected by the upstream target.- pub format: BackendFormat,- /// Connection settings for the upstream target.- #[serde(default)]- pub endpoint: EndpointConfig,- /// Per-target outbound request extensions, merged into the request- /// body by the wire-specific backend before the upstream call.- ///- /// Use cases:- ///- /// * **`chat_template_kwargs.enable_thinking=False`** for DeepSeek- /// V4 (Flash / Pro) on NVIDIA Inference Hub. V4 is a chain-of-- /// thought model whose default reasoning blows past Hub's proxy- /// gateway timeout at ``-n 8`` concurrency (504s on ~5% of- /// requests); the flag disables thinking and pegs response- /// times at ~5 s flat. Cannot be set client-side because Hub- /// ignores the request-level ``reasoning_effort`` field for- /// these models.- /// * Provider-specific options (vLLM ``guided_json``,- /// ``logit_bias``, ``response_format`` shimming) that should- /// apply to *every* request to this target.- ///- /// Merge semantics are shallow and **caller-wins**: keys already- /// present in the inbound request body are not overridden. Set on- /// the target only what the caller is not expected to set itself.- #[serde(default, skip_serializing_if = "Option::is_none")]- pub extra_body: Option<Value>,- /// Per-target outbound HTTP headers attached to every upstream- /// request issued for this target.- ///- /// Distinct from :attr:`extra_body` because some gateway-level- /// routing keys live in headers, not the request body. Example:- /// NVIDIA Inference Hub exposes an evals/benchmarking gateway- /// behind ``X-Inference-Priority: batch`` — required for DeepSeek- /// V4 calls during long benchmark sweeps because the regular- /// gateway enforces a ~6-min timeout that under ``-n 8``- /// concurrency manifests as cascading 504s.- ///- /// Headers added here are appended to whatever the backend would- /// already send (``Authorization``, ``anthropic-version``,- /// telemetry). Reserved header names supplied by the backend- /// (``Authorization`` / ``x-api-key`` / ``anthropic-version``)- /// are still authoritative; ``extra_headers`` cannot override- /// them — the underlying ``reqwest`` builder appends each entry- /// rather than replacing existing ones, so a duplicate name- /// would create a multi-valued header rather than a- /// silent-override security hazard.- #[serde(default, skip_serializing_if = "BTreeMap::is_empty")]- pub extra_headers: BTreeMap<String, String>,-}--impl LlmTarget {- /// Creates a target with automatic format detection and no endpoint overrides.- pub fn new(id: LlmTargetId, model: ModelId) -> Self {- Self {- id,- model,- format: BackendFormat::Auto,- endpoint: EndpointConfig::default(),- extra_body: None,- extra_headers: BTreeMap::new(),- }- }-}--/// Shallow-merges ``target_extra`` into ``body``: keys already present-/// in ``body`` are preserved (caller wins); new keys from-/// ``target_extra`` are added. No-op if either side is not an object.-///-/// Used by wire-specific backends to inject per-target-/// :attr:`LlmTarget.extra_body` into outbound request bodies without-/// stomping caller-supplied fields.-pub fn merge_target_extra_body(body: &mut Value, target_extra: Option<&Value>) {- let Some(Value::Object(extra)) = target_extra else {- return;- };- let Value::Object(body_map) = body else {- return;- };- for (key, value) in extra {- body_map.entry(key.clone()).or_insert_with(|| value.clone());- }-}diff --git a/crates/switchyard-components/src/contracts/context.rs b/crates/switchyard-components/src/contracts/context.rsdeleted file mode 100644index f864508b6..000000000--- a/crates/switchyard-components/src/contracts/context.rs+++ /dev/null@@ -1,216 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Per-request context and typed extension storage for compatibility components.--use std::any::{Any, TypeId};-use std::collections::{HashMap, HashSet};-use std::fmt;--use super::ids::{LlmTargetId, RequestId};-use super::types::ChatRequestType;--/// Mutable state shared across processors and the backend for one request.-#[derive(Default)]-pub struct ProxyContext {- /// Optional request ID propagated across processors and backends.- pub request_id: Option<RequestId>,- /// Optional inbound wire format recorded by endpoint or translation code.- pub inbound_format: Option<ChatRequestType>,- /// Optional target selected by a request processor for backend dispatch.- pub selected_target: Option<LlmTargetId>,- extensions: Extensions,-}--impl ProxyContext {- /// Creates an empty request context.- pub fn new() -> Self {- Self::default()- }-- /// Creates a context with a known request identifier.- pub fn with_request_id(request_id: RequestId) -> Self {- Self {- request_id: Some(request_id),- ..Self::default()- }- }-- /// Returns read-only access to typed extension values.- pub fn extensions(&self) -> &Extensions {- &self.extensions- }-- /// Returns mutable access to typed extension values.- pub fn extensions_mut(&mut self) -> &mut Extensions {- &mut self.extensions- }-- /// Returns the selected target for backend dispatch.- pub fn selected_target(&self) -> Option<&LlmTargetId> {- self.selected_target.as_ref()- }-- /// Replaces the selected target, returning the previous target.- pub fn set_selected_target(&mut self, target_id: LlmTargetId) -> Option<LlmTargetId> {- self.selected_target.replace(target_id)- }-- /// Clears the selected target.- pub fn clear_selected_target(&mut self) -> Option<LlmTargetId> {- self.selected_target.take()- }-- /// Inserts a typed extension, returning the previous value of the same type.- pub fn insert<T>(&mut self, value: T) -> Option<T>- where- T: Send + Sync + 'static,- {- self.extensions.insert(value)- }-- /// Gets an immutable typed extension by Rust type.- pub fn get<T>(&self) -> Option<&T>- where- T: Send + Sync + 'static,- {- self.extensions.get()- }-- /// Gets a mutable typed extension by Rust type.- pub fn get_mut<T>(&mut self) -> Option<&mut T>- where- T: Send + Sync + 'static,- {- self.extensions.get_mut()- }-- /// Removes a typed extension and returns it when present.- pub fn remove<T>(&mut self) -> Option<T>- where- T: Send + Sync + 'static,- {- self.extensions.remove()- }-}--impl fmt::Debug for ProxyContext {- fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {- f.debug_struct("ProxyContext")- .field("request_id", &self.request_id)- .field("inbound_format", &self.inbound_format)- .field("selected_target", &self.selected_target)- .field("extensions_len", &self.extensions.len())- .finish()- }-}--/// Type-indexed storage used for cross-component request metadata.-#[derive(Default)]-pub struct Extensions {- values: HashMap<TypeId, Box<dyn Any + Send + Sync>>,-}--impl Extensions {- /// Creates an empty extension map.- pub fn new() -> Self {- Self::default()- }-- /// Stores a value under its concrete Rust type.- pub fn insert<T>(&mut self, value: T) -> Option<T>- where- T: Send + Sync + 'static,- {- self.values- .insert(TypeId::of::<T>(), Box::new(value))- .and_then(|previous| previous.downcast::<T>().ok())- .map(|boxed| *boxed)- }-- /// Gets a value by its concrete Rust type.- pub fn get<T>(&self) -> Option<&T>- where- T: Send + Sync + 'static,- {- self.values- .get(&TypeId::of::<T>())- .and_then(|value| value.downcast_ref())- }-- /// Gets a mutable value by its concrete Rust type.- pub fn get_mut<T>(&mut self) -> Option<&mut T>- where- T: Send + Sync + 'static,- {- self.values- .get_mut(&TypeId::of::<T>())- .and_then(|value| value.downcast_mut())- }-- /// Removes a value by its concrete Rust type.- pub fn remove<T>(&mut self) -> Option<T>- where- T: Send + Sync + 'static,- {- self.values- .remove(&TypeId::of::<T>())- .and_then(|value| value.downcast::<T>().ok())- .map(|boxed| *boxed)- }-- /// Returns whether a value of type `T` is present.- pub fn contains<T>(&self) -> bool- where- T: Send + Sync + 'static,- {- self.values.contains_key(&TypeId::of::<T>())- }-- /// Returns the number of stored extension values.- pub fn len(&self) -> usize {- self.values.len()- }-- /// Returns whether the extension map is empty.- pub fn is_empty(&self) -> bool {- self.values.is_empty()- }-}--impl fmt::Debug for Extensions {- fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {- f.debug_struct("Extensions")- .field("len", &self.values.len())- .finish()- }-}--/// Targets evicted from a routing pool after a `ContextWindowExceeded` failure.-///-/// Stored in `ProxyContext` for compatibility routers that retry the same-/// logical request on a fallback target after an upstream context-window error.-#[derive(Clone, Debug, Default, PartialEq, Eq)]-pub struct EvictedTargets(HashSet<LlmTargetId>);--impl EvictedTargets {- /// Returns whether a target has already overflowed on this request.- pub fn contains(&self, target_id: &LlmTargetId) -> bool {- self.0.contains(target_id)- }-- /// Records a target as evicted, returning whether it was newly inserted.- pub fn insert(&mut self, target_id: LlmTargetId) -> bool {- self.0.insert(target_id)- }-- /// Returns whether no targets have been evicted yet.- pub fn is_empty(&self) -> bool {- self.0.is_empty()- }-- /// Iterate the evicted target ids in arbitrary order.- pub fn iter(&self) -> impl Iterator<Item = &LlmTargetId> {- self.0.iter()- }-}diff --git a/crates/switchyard-components/src/contracts/error.rs b/crates/switchyard-components/src/contracts/error.rsdeleted file mode 100644index 115e3b92e..000000000--- a/crates/switchyard-components/src/contracts/error.rs+++ /dev/null@@ -1,103 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Errors shared by compatibility backends and Python bindings.--use thiserror::Error;--use super::ids::{InvalidId, ModelId};-use super::types::ChatRequestType;--/// Result alias for core Switchyard operations.-pub type Result<T> = std::result::Result<T, SwitchyardError>;--/// Shared error enum for configuration, processor, and backend failures.-#[derive(Debug, Error)]-pub enum SwitchyardError {- #[error("invalid configuration: {0}")]- InvalidConfig(String),-- #[error(transparent)]- InvalidId(#[from] InvalidId),-- #[error("{kind} {id:?} is already registered")]- DuplicateRegistration { kind: &'static str, id: String },-- #[error("no model registered for {model}")]- ModelNotFound { model: ModelId },-- #[error("{component} does not support request type {request_type:?}")]- UnsupportedRequestType {- component: String,- request_type: ChatRequestType,- },-- // Client sent a structurally valid but semantically invalid request- // (e.g. an empty `messages` array). Surfaced as a 4xx, never a 5xx,- // so agents can distinguish a client bug from a transient server failure.- #[error("{0}")]- InvalidRequest(String),-- #[error("processor failed: {0}")]- Processor(String),-- #[error("backend failed: {0}")]- Backend(String),-- #[error("upstream failed: {0}")]- Upstream(String),-- #[error("upstream failed: {provider} returned HTTP {status_code}: {body}")]- UpstreamHttp {- provider: String,- status_code: u16,- body: String,- },-- // Target hit its context window; routing runtime may evict + retry once.- #[error("context window exceeded on target {target_id} ({model}): {message}")]- ContextWindowExceeded {- target_id: String,- model: String,- message: String,- },-- // Every target was evicted; no fallback left to satisfy the request.- #[error("context pool exhausted (last target {last_target_id}): {reason}")]- ContextPoolExhausted {- last_target_id: String,- reason: String,- },-- #[error("{0}")]- Other(String),-}--#[cfg(test)]-mod tests {- use super::*;-- #[test]- fn context_window_exceeded_renders_target_and_model() {- let err = SwitchyardError::ContextWindowExceeded {- target_id: "weak".into(),- model: "nvidia/deepseek-ai/evals-deepseek-v4-pro".into(),- message: "prompt is too long".into(),- };- let rendered = err.to_string();- assert!(rendered.contains("weak"));- assert!(rendered.contains("evals-deepseek-v4-pro"));- assert!(rendered.contains("prompt is too long"));- }-- #[test]- fn context_pool_exhausted_renders_last_target() {- let err = SwitchyardError::ContextPoolExhausted {- last_target_id: "strong".into(),- reason: "all targets evicted".into(),- };- let rendered = err.to_string();- assert!(rendered.contains("strong"));- assert!(rendered.contains("all targets evicted"));- }-}diff --git a/crates/switchyard-components/src/contracts/ids.rs b/crates/switchyard-components/src/contracts/ids.rsdeleted file mode 100644index 90104c60f..000000000--- a/crates/switchyard-components/src/contracts/ids.rs+++ /dev/null@@ -1,111 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Non-empty string identifiers used by compatibility component wiring.--use std::fmt;-use std::str::FromStr;--use serde::{Deserialize, Serialize};-use thiserror::Error;--/// Error returned when an identifier is empty or whitespace.-#[derive(Clone, Debug, Eq, Error, PartialEq)]-#[error("{kind} must not be empty")]-pub struct InvalidId {- kind: &'static str,-}--impl InvalidId {- // Keeps the constructor private so only validated ID types can create this error.- fn empty(kind: &'static str) -> Self {- Self { kind }- }-}--// Defines the repeated non-empty string ID behavior without runtime inheritance.-macro_rules! string_id {- ($name:ident) => {- #[doc = concat!("Validated non-empty identifier for `", stringify!($name), "`.")]- #[derive(Clone, Debug, Eq, Hash, Ord, PartialEq, PartialOrd, Serialize, Deserialize)]- #[serde(try_from = "String", into = "String")]- pub struct $name(String);-- impl $name {- const KIND: &'static str = stringify!($name);-- /// Creates a new identifier after rejecting empty or whitespace input.- pub fn new(value: impl Into<String>) -> Result<Self, InvalidId> {- let value = value.into();- if value.trim().is_empty() {- return Err(InvalidId::empty(Self::KIND));- }- Ok(Self(value))- }-- /// Creates an identifier from a compile-time string.- pub fn from_static(value: &'static str) -> Self {- match Self::new(value) {- Ok(id) => id,- Err(_) => panic!("static Switchyard IDs must not be empty"),- }- }-- /// Returns the identifier as a borrowed string.- pub fn as_str(&self) -> &str {- &self.0- }-- /// Consumes the identifier and returns its owned string.- pub fn into_inner(self) -> String {- self.0- }- }-- impl AsRef<str> for $name {- fn as_ref(&self) -> &str {- self.as_str()- }- }-- impl TryFrom<String> for $name {- type Error = InvalidId;-- fn try_from(value: String) -> Result<Self, Self::Error> {- Self::new(value)- }- }-- impl TryFrom<&str> for $name {- type Error = InvalidId;-- fn try_from(value: &str) -> Result<Self, Self::Error> {- Self::new(value)- }- }-- impl From<$name> for String {- fn from(value: $name) -> Self {- value.into_inner()- }- }-- impl FromStr for $name {- type Err = InvalidId;-- fn from_str(value: &str) -> Result<Self, Self::Err> {- Self::new(value)- }- }-- impl fmt::Display for $name {- fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {- f.write_str(self.as_str())- }- }- };-}--string_id!(LlmTargetId);-string_id!(ModelId);-string_id!(RequestId);diff --git a/crates/switchyard-components/src/contracts/mod.rs b/crates/switchyard-components/src/contracts/mod.rsdeleted file mode 100644index 4ef3dde00..000000000--- a/crates/switchyard-components/src/contracts/mod.rs+++ /dev/null@@ -1,18 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Contracts retained by the compatibility component and Python layers.--mod backend;-mod context;-mod error;-mod ids;-mod roles;-mod types;--pub use backend::*;-pub use context::*;-pub use error::*;-pub use ids::*;-pub use roles::*;-pub use types::*;diff --git a/crates/switchyard-components/src/contracts/roles.rs b/crates/switchyard-components/src/contracts/roles.rsdeleted file mode 100644index 8ad074eca..000000000--- a/crates/switchyard-components/src/contracts/roles.rs+++ /dev/null@@ -1,30 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Backend trait shared by compatibility LLM callers.--use async_trait::async_trait;--use super::context::ProxyContext;-use super::error::Result;-use super::types::{ChatRequest, ChatRequestType, ChatResponse};--/// Backend abstraction responsible for making the LLM call.-#[async_trait]-pub trait LlmBackend: Send + Sync {- /// Returns the request formats this backend can accept directly.- fn supported_request_types(&self) -> &[ChatRequestType];-- /// Calls the backend with the processed request.- async fn call(&self, ctx: &mut ProxyContext, request: &ChatRequest) -> Result<ChatResponse>;-- /// Starts any resources owned by the backend.- async fn startup(&self) -> Result<()> {- Ok(())- }-- /// Stops any resources owned by the backend.- async fn shutdown(&self) -> Result<()> {- Ok(())- }-}diff --git a/crates/switchyard-components/src/contracts/types.rs b/crates/switchyard-components/src/contracts/types.rsdeleted file mode 100644index c0c7e0e52..000000000--- a/crates/switchyard-components/src/contracts/types.rs+++ /dev/null@@ -1,326 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Provider-agnostic request and response wrappers used by compatibility chains.--use std::fmt;-use std::pin::Pin;--use futures_core::Stream;-use serde::{Deserialize, Serialize};-use serde_json::{Map, Value};--use super::error::{Result, SwitchyardError};--/// Supported inbound request wire formats.-#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq, Serialize, Deserialize)]-pub enum ChatRequestType {- #[serde(rename = "openai_chat")]- OpenAiChat,- #[serde(rename = "openai_responses")]- OpenAiResponses,- #[serde(rename = "anthropic")]- Anthropic,-}--/// JSON request body wrapper shared by all request variants.-#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]-pub struct WireRequest {- body: Value,-}--impl WireRequest {- /// Wraps an arbitrary JSON request body.- pub fn new(body: Value) -> Self {- Self { body }- }-- /// Returns the wrapped JSON request body.- pub fn body(&self) -> &Value {- &self.body- }-- /// Returns a mutable reference to the wrapped JSON request body.- pub fn body_mut(&mut self) -> &mut Value {- &mut self.body- }-- /// Consumes the wrapper and returns the JSON request body.- pub fn into_body(self) -> Value {- self.body- }-}--/// Request body tagged by source API format.-#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]-#[serde(tag = "request_type", content = "request", rename_all = "snake_case")]-pub enum ChatRequest {- #[serde(rename = "openai_chat")]- OpenAiChat(WireRequest),- #[serde(rename = "openai_responses")]- OpenAiResponses(WireRequest),- #[serde(rename = "anthropic")]- Anthropic(WireRequest),-}--impl ChatRequest {- /// Creates an OpenAI Chat Completions request.- pub fn openai_chat(body: Value) -> Self {- Self::OpenAiChat(WireRequest::new(body))- }-- /// Creates an OpenAI Responses API request.- pub fn openai_responses(body: Value) -> Self {- Self::OpenAiResponses(WireRequest::new(body))- }-- /// Creates an Anthropic Messages request.- pub fn anthropic(body: Value) -> Self {- Self::Anthropic(WireRequest::new(body))- }-- /// Validates the request body before it enters the chain.- ///- /// Catches structurally valid but semantically invalid input so it- /// fails fast with a 4xx instead of reaching the backend and surfacing- /// as an opaque upstream 5xx. Currently rejects a present-but-empty- /// `messages` array on the message-based formats (OpenAI Chat,- /// Anthropic). Absent or non-array `messages` are left for the backend- /// to interpret; the Responses format carries `input`, not `messages`,- /// and is exempt.- pub fn validate(&self) -> Result<()> {- let checks_messages = matches!(self, Self::OpenAiChat(_) | Self::Anthropic(_));- if checks_messages- && let Some(messages) = self.body().get("messages").and_then(Value::as_array)- && messages.is_empty()- {- return Err(SwitchyardError::InvalidRequest(- "messages must be a non-empty array".to_string(),- ));- }- Ok(())- }-- /// Returns the request's tagged wire format.- pub fn request_type(&self) -> ChatRequestType {- match self {- Self::OpenAiChat(_) => ChatRequestType::OpenAiChat,- Self::OpenAiResponses(_) => ChatRequestType::OpenAiResponses,- Self::Anthropic(_) => ChatRequestType::Anthropic,- }- }-- /// Returns the request body regardless of source format.- pub fn body(&self) -> &Value {- match self {- Self::OpenAiChat(request)- | Self::OpenAiResponses(request)- | Self::Anthropic(request) => request.body(),- }- }-- /// Returns the mutable request body regardless of source format.- pub fn body_mut(&mut self) -> &mut Value {- match self {- Self::OpenAiChat(request)- | Self::OpenAiResponses(request)- | Self::Anthropic(request) => request.body_mut(),- }- }-- /// Reads the request's `model` field when present and string-valued.- pub fn model(&self) -> Option<&str> {- self.body().get("model").and_then(Value::as_str)- }-- /// Writes the request's `model` field, creating an object body if necessary.- pub fn set_model(&mut self, model: impl Into<String>) {- match self.body_mut() {- Value::Object(body) => {- body.insert("model".to_string(), Value::String(model.into()));- }- body => {- let mut object = Map::new();- object.insert("model".to_string(), Value::String(model.into()));- *body = Value::Object(object);- }- }- }-}--/// Supported backend response wire shapes.-#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq, Serialize, Deserialize)]-pub enum ChatResponseType {- #[serde(rename = "openai_completion")]- OpenAiCompletion,- #[serde(rename = "openai_stream")]- OpenAiStream,- #[serde(rename = "openai_responses_completion")]- OpenAiResponsesCompletion,- #[serde(rename = "openai_responses_stream")]- OpenAiResponsesStream,- #[serde(rename = "anthropic_completion")]- AnthropicCompletion,- #[serde(rename = "anthropic_stream")]- AnthropicStream,-}--/// JSON response body wrapper shared by buffered response variants.-#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]-pub struct WireResponse {- body: Value,-}--impl WireResponse {- /// Wraps an arbitrary JSON response body.- pub fn new(body: Value) -> Self {- Self { body }- }-- /// Returns the wrapped JSON response body.- pub fn body(&self) -> &Value {- &self.body- }-- /// Consumes the wrapper and returns the JSON response body.- pub fn into_body(self) -> Value {- self.body- }-}--/// Streaming event payload carried by streaming response variants.-#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]-#[serde(tag = "kind", content = "data", rename_all = "snake_case")]-pub enum StreamEvent {- Json(Value),- Text(String),-}--/// Boxed async stream for backend response events.-pub type BoxResponseStream = Pin<Box<dyn Stream<Item = Result<StreamEvent>> + Send>>;--/// Backend response tagged by provider format and buffered/streaming shape.-pub enum ChatResponse {- OpenAiCompletion(WireResponse),- OpenAiStream(BoxResponseStream),- OpenAiResponsesCompletion(WireResponse),- OpenAiResponsesStream(BoxResponseStream),- AnthropicCompletion(WireResponse),- AnthropicStream(BoxResponseStream),-}--impl ChatResponse {- /// Creates a buffered OpenAI Chat Completions response.- pub fn openai_completion(body: Value) -> Self {- Self::OpenAiCompletion(WireResponse::new(body))- }-- /// Creates a buffered OpenAI Responses API response.- pub fn openai_responses_completion(body: Value) -> Self {- Self::OpenAiResponsesCompletion(WireResponse::new(body))- }-- /// Creates a buffered Anthropic Messages response.- pub fn anthropic_completion(body: Value) -> Self {- Self::AnthropicCompletion(WireResponse::new(body))- }-- /// Returns the response's tagged wire shape.- pub fn response_type(&self) -> ChatResponseType {- match self {- Self::OpenAiCompletion(_) => ChatResponseType::OpenAiCompletion,- Self::OpenAiStream(_) => ChatResponseType::OpenAiStream,- Self::OpenAiResponsesCompletion(_) => ChatResponseType::OpenAiResponsesCompletion,- Self::OpenAiResponsesStream(_) => ChatResponseType::OpenAiResponsesStream,- Self::AnthropicCompletion(_) => ChatResponseType::AnthropicCompletion,- Self::AnthropicStream(_) => ChatResponseType::AnthropicStream,- }- }-- /// Returns the JSON body for buffered responses and `None` for streams.- pub fn body(&self) -> Option<&Value> {- match self {- Self::OpenAiCompletion(response)- | Self::OpenAiResponsesCompletion(response)- | Self::AnthropicCompletion(response) => Some(response.body()),- Self::OpenAiStream(_) | Self::OpenAiResponsesStream(_) | Self::AnthropicStream(_) => {- None- }- }- }-}--impl fmt::Debug for ChatResponse {- fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {- match self {- Self::OpenAiCompletion(response) => f- .debug_tuple("OpenAiCompletion")- .field(response.body())- .finish(),- Self::OpenAiStream(_) => f.debug_tuple("OpenAiStream").field(&"<stream>").finish(),- Self::OpenAiResponsesCompletion(response) => f- .debug_tuple("OpenAiResponsesCompletion")- .field(response.body())- .finish(),- Self::OpenAiResponsesStream(_) => f- .debug_tuple("OpenAiResponsesStream")- .field(&"<stream>")- .finish(),- Self::AnthropicCompletion(response) => f- .debug_tuple("AnthropicCompletion")- .field(response.body())- .finish(),- Self::AnthropicStream(_) => {- f.debug_tuple("AnthropicStream").field(&"<stream>").finish()- }- }- }-}--#[cfg(test)]-mod tests {- use super::*;- use serde_json::json;-- #[test]- fn validate_rejects_empty_messages_for_openai_chat() {- let request = ChatRequest::openai_chat(json!({"model": "m", "messages": []}));- match request.validate() {- Err(SwitchyardError::InvalidRequest(message)) => {- assert!(message.contains("messages must be a non-empty array"));- }- other => panic!("expected InvalidRequest, got {other:?}"),- }- }-- #[test]- fn validate_rejects_empty_messages_for_anthropic() {- let request = ChatRequest::anthropic(json!({"model": "m", "messages": []}));- match request.validate() {- Err(SwitchyardError::InvalidRequest(_)) => {}- other => panic!("expected InvalidRequest, got {other:?}"),- }- }-- #[test]- fn validate_accepts_non_empty_messages() {- let request = ChatRequest::openai_chat(- json!({"model": "m", "messages": [{"role": "user", "content": "hi"}]}),- );- assert!(request.validate().is_ok());- }-- #[test]- fn validate_is_lenient_when_messages_absent() {- // Absent or non-array `messages` is left for the backend to interpret.- let request = ChatRequest::openai_chat(json!({"model": "m"}));- assert!(request.validate().is_ok());- }-- #[test]- fn validate_exempts_responses_format() {- // The Responses format carries `input`, not `messages`.- let request = ChatRequest::openai_responses(json!({"model": "m", "input": "hi"}));- assert!(request.validate().is_ok());- }-}diff --git a/crates/switchyard-components/src/dimension_collector/mod.rs b/crates/switchyard-components/src/dimension_collector/mod.rsdeleted file mode 100644index 95bdecaf0..000000000--- a/crates/switchyard-components/src/dimension_collector/mod.rs+++ /dev/null@@ -1,18 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Signal extraction for the stage_router.-//!-//! Owns the pure logic for reading a coding-agent request's tool-call history-//! into a [`ToolResultSignal`] (write/edit/read counts, error severity, streaks),-//! plus response-side signals. The-//! [`crate::request_processors::dimension_collector`] module wraps it as a-//! request-side Switchyard component.--pub mod response;-pub mod tool_signals;--pub use response::{ResponseFlag, ResponseSignals, extract_response_signals};-pub use tool_signals::{- DEFAULT_RECENT_WINDOW, ToolResultSignal, extract_tool_signals, extract_tool_signals_with_window,-};diff --git a/crates/switchyard-components/src/dimension_collector/response/checks.rs b/crates/switchyard-components/src/dimension_collector/response/checks.rsdeleted file mode 100644index fac3b0c1d..000000000--- a/crates/switchyard-components/src/dimension_collector/response/checks.rs+++ /dev/null@@ -1,361 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Pure response-side checks emitted by [`super::extract_response_signals`].-//!-//! Each check is a `fn(&ChatResponse) -> bool` operating on buffered JSON-//! bodies only — streams are treated as "no signal" upstream because they-//! can't be introspected without consuming the stream. Adding a streaming-//! variant is a separate effort (collect-as-you-go, emit at end-of-stream)-//! tracked outside the first-cut scope.-//!-//! Each fn is one OpenAI / Anthropic shape match. Keep them shape-aware-//! rather than running a uniform JSONPath over every wire — the wire-//! formats are different enough that uniform JSONPath would either be-//! lossy or 4x more code than direct shape access.--use crate::ChatResponse;-use serde_json::Value;--/// Returns `true` if any `tool_calls[].function.arguments` string is not-/// valid JSON. Operates on OpenAI-Chat and Anthropic responses.-///-/// Mirrors the failure mode where a weak model emits a tool call shape-/// that looks well-formed at the top level but has malformed JSON in the-/// arguments field. Common with smaller models under tight token budgets.-pub fn is_malformed_tool_call(response: &ChatResponse) -> bool {- let Some(body) = response.body() else {- return false;- };- if let Some(tool_calls) = openai_chat_tool_calls(body) {- return tool_calls_have_invalid_args(tool_calls);- }- if let Some(content_blocks) = anthropic_content_blocks(body) {- return anthropic_tool_use_blocks_have_invalid_args(content_blocks);- }- false-}--/// Returns `true` if the response carries no content **and** no tool calls-/// **and** `finish_reason` is not the legitimate `"tool_calls"` /-/// `"tool_use"` sentinel. Distinguishes "model said nothing" from "model-/// finished with a tool call" — the second is normal, the first is a-/// quality failure.-pub fn is_empty_response(response: &ChatResponse) -> bool {- let Some(body) = response.body() else {- return false;- };- if let Some(choice) = openai_first_choice(body) {- let message = choice.get("message").and_then(Value::as_object);- let content_empty = message- .and_then(|m| m.get("content"))- .map(content_is_empty)- .unwrap_or(true);- let tool_calls_empty = message- .and_then(|m| m.get("tool_calls"))- .and_then(Value::as_array)- .is_none_or(|calls| calls.is_empty());- let finish_reason = choice- .get("finish_reason")- .and_then(Value::as_str)- .unwrap_or("");- return content_empty- && tool_calls_empty- && finish_reason != "tool_calls"- && finish_reason != "tool_use";- }- if let Some(blocks) = anthropic_content_blocks(body) {- let any_content = blocks.iter().any(|block| {- let kind = block.get("type").and_then(Value::as_str).unwrap_or("");- kind == "text" || kind == "tool_use"- });- let stop_reason = body- .get("stop_reason")- .and_then(Value::as_str)- .unwrap_or("");- return !any_content && stop_reason != "tool_use";- }- false-}--/// Returns `true` if the response was truncated by the model-side-/// `max_tokens` budget — `finish_reason == "length"` on OpenAI Chat,-/// `stop_reason == "max_tokens"` on Anthropic. A common weak-model-/// failure mode under cost-pressured `max_tokens` caps.-pub fn is_truncated_completion(response: &ChatResponse) -> bool {- let Some(body) = response.body() else {- return false;- };- if let Some(choice) = openai_first_choice(body) {- return choice- .get("finish_reason")- .and_then(Value::as_str)- .map(|reason| reason == "length")- .unwrap_or(false);- }- body.get("stop_reason")- .and_then(Value::as_str)- .map(|reason| reason == "max_tokens")- .unwrap_or(false)-}--/// Returns `true` if any emitted tool call is missing a top-level required-/// field per the OpenAI Chat / Anthropic tool-call shape itself.-///-/// Note: this checks *shape* requirements (must have `name`, must have-/// `arguments`/`input`), **not** the per-tool argument schemas. The-/// per-tool schema check requires the request's tool declarations and is-/// future work — the shape check catches the most common malformed-/// tool-call cases.-pub fn is_missing_required_args(response: &ChatResponse) -> bool {- let Some(body) = response.body() else {- return false;- };- if let Some(tool_calls) = openai_chat_tool_calls(body) {- return tool_calls.iter().any(|call| {- let function = call.get("function").and_then(Value::as_object);- let has_name = function- .and_then(|f| f.get("name"))- .and_then(Value::as_str)- .is_some_and(|name| !name.is_empty());- let has_args = function- .and_then(|f| f.get("arguments"))- .and_then(Value::as_str)- .is_some();- !(has_name && has_args)- });- }- if let Some(blocks) = anthropic_content_blocks(body) {- return blocks.iter().any(|block| {- let kind = block.get("type").and_then(Value::as_str).unwrap_or("");- if kind != "tool_use" {- return false;- }- let has_name = block- .get("name")- .and_then(Value::as_str)- .is_some_and(|name| !name.is_empty());- let has_input = block.get("input").is_some();- !(has_name && has_input)- });- }- false-}--// ─── Shape-access helpers ────────────────────────────────────────────────--fn openai_first_choice(body: &Value) -> Option<&serde_json::Map<String, Value>> {- body.get("choices")- .and_then(Value::as_array)- .and_then(|choices| choices.first())- .and_then(Value::as_object)-}--fn openai_chat_tool_calls(body: &Value) -> Option<&Vec<Value>> {- openai_first_choice(body)- .and_then(|choice| choice.get("message"))- .and_then(Value::as_object)- .and_then(|message| message.get("tool_calls"))- .and_then(Value::as_array)-}--fn anthropic_content_blocks(body: &Value) -> Option<&Vec<Value>> {- body.get("content").and_then(Value::as_array)-}--fn tool_calls_have_invalid_args(tool_calls: &[Value]) -> bool {- tool_calls.iter().any(|call| {- let Some(args) = call- .get("function")- .and_then(|function| function.get("arguments"))- .and_then(Value::as_str)- else {- // Missing arguments is `is_missing_required_args`'s concern;- // here we only flag *malformed JSON* in the provided string.- return false;- };- // Empty-string arguments are convention for zero-arg tool calls;- // treat as well-formed (`{}` equivalent).- if args.is_empty() {- return false;- }- serde_json::from_str::<Value>(args).is_err()- })-}--fn anthropic_tool_use_blocks_have_invalid_args(blocks: &[Value]) -> bool {- // Anthropic emits `input` as already-parsed JSON, not a string, so- // there's no malformed-string failure mode equivalent to OpenAI's.- // Flagged only if `input` is a string that fails to parse as JSON,- // which is non-standard but defensive.- blocks.iter().any(|block| {- let kind = block.get("type").and_then(Value::as_str).unwrap_or("");- if kind != "tool_use" {- return false;- }- let Some(input) = block.get("input") else {- return false;- };- match input {- Value::String(s) if !s.is_empty() => serde_json::from_str::<Value>(s).is_err(),- _ => false,- }- })-}--fn content_is_empty(content: &Value) -> bool {- match content {- Value::Null => true,- Value::String(s) => s.trim().is_empty(),- Value::Array(parts) => parts.iter().all(|part| {- part.as_object()- .and_then(|object| object.get("text"))- .and_then(Value::as_str)- .is_none_or(|text| text.trim().is_empty())- }),- _ => false,- }-}--#[cfg(test)]-mod tests {- use super::*;- use serde_json::json;-- fn openai_chat(body: Value) -> ChatResponse {- ChatResponse::openai_completion(body)- }-- fn anthropic(body: Value) -> ChatResponse {- ChatResponse::anthropic_completion(body)- }-- #[test]- fn malformed_tool_call_openai_chat() {- let bad = openai_chat(json!({- "choices": [{- "message": {- "tool_calls": [{- "function": {- "name": "get_weather",- "arguments": "{\"city\": \"new york\"" // missing closing brace- }- }]- }- }]- }));- assert!(is_malformed_tool_call(&bad));-- let good = openai_chat(json!({- "choices": [{- "message": {- "tool_calls": [{- "function": {- "name": "get_weather",- "arguments": "{\"city\": \"new york\"}"- }- }]- }- }]- }));- assert!(!is_malformed_tool_call(&good));- }-- #[test]- fn empty_arguments_string_is_well_formed() {- let resp = openai_chat(json!({- "choices": [{- "message": {- "tool_calls": [{- "function": { "name": "ping", "arguments": "" }- }]- }- }]- }));- assert!(!is_malformed_tool_call(&resp));- }-- #[test]- fn empty_response_distinguishes_from_tool_call_finish() {- let empty = openai_chat(json!({- "choices": [{- "message": { "content": null },- "finish_reason": "stop"- }]- }));- assert!(is_empty_response(&empty));-- let tool_call_finish = openai_chat(json!({- "choices": [{- "message": {- "content": null,- "tool_calls": [{- "function": { "name": "x", "arguments": "{}" }- }]- },- "finish_reason": "tool_calls"- }]- }));- assert!(!is_empty_response(&tool_call_finish));- }-- #[test]- fn truncated_completion_finish_reason_length() {- let truncated = openai_chat(json!({- "choices": [{- "message": { "content": "this got cut off..." },- "finish_reason": "length"- }]- }));- assert!(is_truncated_completion(&truncated));-- let ok = openai_chat(json!({- "choices": [{- "message": { "content": "all done" },- "finish_reason": "stop"- }]- }));- assert!(!is_truncated_completion(&ok));- }-- #[test]- fn truncated_anthropic_stop_reason_max_tokens() {- let truncated = anthropic(json!({- "content": [{ "type": "text", "text": "got cut off" }],- "stop_reason": "max_tokens"- }));- assert!(is_truncated_completion(&truncated));- }-- #[test]- fn missing_required_args_flags_nameless_or_argless_tool_calls() {- let no_name = openai_chat(json!({- "choices": [{- "message": {- "tool_calls": [{ "function": { "arguments": "{}" } }]- }- }]- }));- assert!(is_missing_required_args(&no_name));-- let no_args = openai_chat(json!({- "choices": [{- "message": {- "tool_calls": [{ "function": { "name": "ping" } }]- }- }]- }));- assert!(is_missing_required_args(&no_args));-- let well_formed = openai_chat(json!({- "choices": [{- "message": {- "tool_calls": [{- "function": { "name": "ping", "arguments": "{}" }- }]- }- }]- }));- assert!(!is_missing_required_args(&well_formed));- }-}diff --git a/crates/switchyard-components/src/dimension_collector/response/mod.rs b/crates/switchyard-components/src/dimension_collector/response/mod.rsdeleted file mode 100644index 685b185ef..000000000--- a/crates/switchyard-components/src/dimension_collector/response/mod.rs+++ /dev/null@@ -1,128 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Response-side context signals.-//!-//! Sibling module to the request-side dimension collector. Same character:-//! deterministic, no estimator needed, calibration-friendly. Each-//! [`ResponseFlag`] is the binary outcome of one pure check against a-//! `ChatResponse` body.-//!-//! [`ResponseSignalCollector`] (in `request_processors/`) wraps this as a-//! response-side adapter; the pure logic lives here so it's-//! independently unit-testable and reusable by future routers that want-//! to inspect response quality without going through the full chain.--pub mod checks;--use crate::ChatResponse;-use serde::{Deserialize, Serialize};--/// Aggregate of response-side quality flags emitted by [`extract_response_signals`].-///-/// Stamped into `ProxyContext.extensions` by the-/// `ResponseSignalCollector` adapter. Empty `flags` means all checks-/// passed — the response is considered acceptable from the stage-router-/// router's reactive-escalation viewpoint.-#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]-pub struct ResponseSignals {- pub flags: Vec<ResponseFlag>,-}--impl ResponseSignals {- /// Returns `true` when at least one check flagged the response.- ///- /// StageRouter routers consume this as the per-attempt acceptability- /// gate; a `true` here triggers escalation to the next tier.- pub fn has_failures(&self) -> bool {- !self.flags.is_empty()- }-- /// Returns `true` if the given flag is present.- pub fn contains(&self, flag: ResponseFlag) -> bool {- self.flags.contains(&flag)- }-}--/// Closed set of response-side quality failures emitted by the-/// dimension-collector response layer.-///-/// Adding a new flag is a deliberate API change: downstream estimators-/// match on this enum, and growth here is observable in their behavior.-/// Keep it small.-#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]-#[serde(rename_all = "snake_case")]-pub enum ResponseFlag {- /// `tool_calls[].function.arguments` doesn't parse as JSON.- MalformedToolCallJson,- /// No `content` and no `tool_calls`; not a legitimate- /// `tool_calls`-finish.- EmptyResponse,- /// `finish_reason == "length"` (OpenAI) or- /// `stop_reason == "max_tokens"` (Anthropic).- TruncatedCompletion,- /// A tool call is missing `name` or `arguments` at the shape level- /// (not per-tool-schema; that's future work).- MissingRequiredArgs,-}--/// Runs all four checks against a buffered `ChatResponse` and returns-/// the set of failing flags.-///-/// Streaming responses (where [`ChatResponse::body`] returns `None`)-/// short-circuit to an empty `ResponseSignals` — streams can't be-/// introspected without consuming them. A streaming-aware checker is-/// future work outside the first cut.-pub fn extract_response_signals(response: &ChatResponse) -> ResponseSignals {- if response.body().is_none() {- return ResponseSignals::default();- }- let mut flags = Vec::new();- if checks::is_malformed_tool_call(response) {- flags.push(ResponseFlag::MalformedToolCallJson);- }- if checks::is_empty_response(response) {- flags.push(ResponseFlag::EmptyResponse);- }- if checks::is_truncated_completion(response) {- flags.push(ResponseFlag::TruncatedCompletion);- }- if checks::is_missing_required_args(response) {- flags.push(ResponseFlag::MissingRequiredArgs);- }- ResponseSignals { flags }-}--#[cfg(test)]-mod tests {- use super::*;- use serde_json::json;-- #[test]- fn extract_returns_empty_for_well_formed_response() {- let resp = ChatResponse::openai_completion(json!({- "choices": [{- "message": { "content": "ok" },- "finish_reason": "stop"- }]- }));- let signals = extract_response_signals(&resp);- assert!(signals.flags.is_empty());- assert!(!signals.has_failures());- }-- #[test]- fn extract_aggregates_multiple_failures() {- // Empty content + truncated finish at the same time.- let resp = ChatResponse::openai_completion(json!({- "choices": [{- "message": { "content": "" },- "finish_reason": "length"- }]- }));- let signals = extract_response_signals(&resp);- assert!(signals.has_failures());- assert!(signals.contains(ResponseFlag::EmptyResponse));- assert!(signals.contains(ResponseFlag::TruncatedCompletion));- }-}diff --git a/crates/switchyard-components/src/dimension_collector/tool_signals.rs b/crates/switchyard-components/src/dimension_collector/tool_signals.rsdeleted file mode 100644index 895af3f7b..000000000--- a/crates/switchyard-components/src/dimension_collector/tool_signals.rs+++ /dev/null@@ -1,52 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Tool-result context signals — thin adapter over libsy's extractor.-//!-//! The extraction logic lives in [`switchyard_libsy::ToolSignals`]. This module-//! bridges the crate's [`ChatRequest`] (a format-tagged JSON body) to libsy's-//! [`switchyard_protocol::Request`] (raw body + wire-format metadata) so the two-//! request models share one implementation.--use crate::{ChatRequest, ChatRequestType};-use switchyard_protocol::{Metadata, Request, WireFormat};--/// The tool-signal output type. Re-exported from libsy so downstream consumers-/// (the `ToolResultSignal` stamped on `ProxyContext`) see a single type.-pub use switchyard_libsy::{DEFAULT_RECENT_WINDOW, ToolSignals as ToolResultSignal};--/// Adapt a format-tagged [`ChatRequest`] to a [`switchyard_protocol::Request`]-/// carrying the raw body and its wire format, which is all libsy's extractor reads.-fn to_protocol_request(request: &ChatRequest) -> Request {- let wire_format = match request.request_type() {- ChatRequestType::OpenAiChat => WireFormat::OpenAiChat,- ChatRequestType::Anthropic => WireFormat::AnthropicMessages,- ChatRequestType::OpenAiResponses => WireFormat::OpenAiResponses,- };- // The signals are read off the decoded conversation, so decode here rather- // than handing over a raw body the extractor cannot interpret. A body that- // fails to decode yields no signals, which is what an absent body did before.- let llm_request =- switchyard_translation::decode_request(wire_format, request.body()).unwrap_or_default();- Request {- llm_request,- raw_request: Some(request.body().clone()),- metadata: Some(Metadata {- wire_format: Some(wire_format),- ..Default::default()- }),- }-}--/// Extract tool-execution signals using the default `recent_*` window.-pub fn extract_tool_signals(request: &ChatRequest) -> ToolResultSignal {- ToolResultSignal::from_request(&to_protocol_request(request), None)-}--/// Extract tool-execution signals with a caller-supplied `recent_*` window.-pub fn extract_tool_signals_with_window(- request: &ChatRequest,- recent_window: usize,-) -> ToolResultSignal {- ToolResultSignal::from_request(&to_protocol_request(request), Some(recent_window))-}diff --git a/crates/switchyard-components/src/lib.rs b/crates/switchyard-components/src/lib.rsdeleted file mode 100644index e9c6983be..000000000--- a/crates/switchyard-components/src/lib.rs+++ /dev/null@@ -1,36 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Compatibility backends and processors for the Python Switchyard runtime.-//!-//! New Rust orchestration belongs in libsy algorithms and clients. The contracts-//! here remain only while the Python compatibility runtime uses these components.--pub mod backends;-mod contracts;-pub mod dimension_collector;-pub mod request_processors;-pub mod response_processors;-pub mod stage_router;-pub mod stats;-mod telemetry;--pub use backends::{- AnthropicNativeBackend, BackendSelection, BackendSelectionReason, LlmTargetBackend,- MultiLlmBackend, OpenAiNativeBackend, OpenAiPassthroughBackend, StatsLlmBackend,-};-pub use contracts::*;-pub use dimension_collector::{- ResponseFlag, ResponseSignals, ToolResultSignal, extract_tool_signals,-};-pub use request_processors::{- DimensionCollector, RandomRoutingDecision, RandomRoutingEngine, RandomRoutingProcessorConfig,- RandomRoutingTier, StatsRequestProcessor,-};-pub use response_processors::{ResponseSignalCollector, StatsResponseProcessor};-pub use stats::{- ClassifierStatsSnapshot, CostBreakdown, CostEstimate, LatencyHistogramSnapshot,- ModelStatsSnapshot, PrefixProbe, StatsAccumulator, StatsBackendLatency, StatsRequestStart,- StatsRouteLabel, StatsSnapshot, TierStatsSnapshot, TokenTotals, TokenUsage, prefix_probe,- tracking_enabled_from_env,-};diff --git a/crates/switchyard-components/src/request_processors/dimension_collector.rs b/crates/switchyard-components/src/request_processors/dimension_collector.rsdeleted file mode 100644index 41b5c4d62..000000000--- a/crates/switchyard-components/src/request_processors/dimension_collector.rs+++ /dev/null@@ -1,78 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Request-processor adapter for the tool-signal extractor.-//!-//! Thin wrapper around [`crate::dimension_collector::extract_tool_signals_with_window`].-//! It walks the request's tool-call history and stamps the resulting-//! [`ToolResultSignal`] into `ProxyContext` for the stage_router picker to read.--use crate::{ChatRequest, ProxyContext, Result};--use crate::dimension_collector::{- DEFAULT_RECENT_WINDOW, ToolResultSignal, extract_tool_signals_with_window,-};--/// Populates `ProxyContext` with a [`ToolResultSignal`] read from the request's-/// tool-call history. The stage_router picker reads it via-/// `ctx.get::<ToolResultSignal>()`.-#[derive(Clone, Debug)]-pub struct DimensionCollector {- recent_window: usize,-}--impl Default for DimensionCollector {- fn default() -> Self {- Self {- recent_window: DEFAULT_RECENT_WINDOW,- }- }-}--impl DimensionCollector {- /// Construct a collector with a caller-supplied sliding-window size for the- /// `recent_*` signal counts. Smaller windows make the picker more reactive- /// to the latest tool call; larger windows smooth over turn-by-turn noise.- pub fn with_recent_window(recent_window: usize) -> Self {- Self { recent_window }- }-- /// Returns the configured `recent_*` sliding-window size.- pub fn recent_window(&self) -> usize {- self.recent_window- }-- /// Extracts the tool-result signal and stores it on the request context.- pub async fn process(- &self,- ctx: &mut ProxyContext,- request: ChatRequest,- ) -> Result<ChatRequest> {- let tool_signal = extract_tool_signals_with_window(&request, self.recent_window);- ctx.insert::<ToolResultSignal>(tool_signal);- Ok(request)- }-}--#[cfg(test)]-mod tests {- use super::*;- use serde_json::json;-- #[tokio::test]- async fn stamps_tool_result_signal_into_proxy_context() {- let collector = DimensionCollector::default();- let request = ChatRequest::openai_chat(json!({- "model": "test-model",- "messages": [{"role": "user", "content": "hi"}],- }));-- let mut ctx = ProxyContext::new();- collector- .process(&mut ctx, request)- .await- .expect("process ok");-- assert!(ctx.get::<ToolResultSignal>().is_some());- }-}diff --git a/crates/switchyard-components/src/request_processors/mod.rs b/crates/switchyard-components/src/request_processors/mod.rsdeleted file mode 100644index 3855c81c3..000000000--- a/crates/switchyard-components/src/request_processors/mod.rs+++ /dev/null@@ -1,12 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Built-in request processor implementations.--pub mod dimension_collector;-pub mod random_routing;-pub mod stats;--pub use dimension_collector::DimensionCollector;-pub use random_routing::*;-pub use stats::*;diff --git a/crates/switchyard-components/src/request_processors/random_routing.rs b/crates/switchyard-components/src/request_processors/random_routing.rsdeleted file mode 100644index dc53d0828..000000000--- a/crates/switchyard-components/src/request_processors/random_routing.rs+++ /dev/null@@ -1,172 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Runtime support for random routing.--use std::fmt;--use crate::{LlmTarget, LlmTargetId, ModelId, Result, SwitchyardError};-use parking_lot::Mutex;-use rand::rngs::StdRng;-use rand::{RngExt, SeedableRng};-use serde::{Deserialize, Serialize};--const DEFAULT_STRONG_PROBABILITY: f64 = 0.5;--/// Named side of a strong/weak random-routing decision.-#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq, Serialize, Deserialize)]-#[serde(rename_all = "snake_case")]-pub enum RandomRoutingTier {- Strong,- Weak,-}--impl RandomRoutingTier {- /// Returns the stable lowercase tier label.- pub fn as_str(self) -> &'static str {- match self {- Self::Strong => "strong",- Self::Weak => "weak",- }- }-}--/// Runtime config for weighted random routing between two LLM targets.-#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]-pub struct RandomRoutingProcessorConfig {- pub strong: LlmTarget,- pub weak: LlmTarget,- pub strong_probability: f64,- pub rng_seed: Option<u64>,-}--impl RandomRoutingProcessorConfig {- /// Creates a config with a 50/50 split and entropy-backed randomness.- pub fn new(strong: LlmTarget, weak: LlmTarget) -> Self {- Self {- strong,- weak,- strong_probability: DEFAULT_STRONG_PROBABILITY,- rng_seed: None,- }- }-- /// Creates a config from just strong and weak model names.- pub fn from_models(- strong_model: impl Into<String>,- weak_model: impl Into<String>,- ) -> Result<Self> {- Ok(Self::new(- LlmTarget::new(- LlmTargetId::from_static("strong"),- ModelId::new(strong_model)?,- ),- LlmTarget::new(LlmTargetId::from_static("weak"), ModelId::new(weak_model)?),- ))- }-- /// Sets the probability of selecting the strong tier.- pub fn with_strong_probability(mut self, strong_probability: f64) -> Result<Self> {- validate_probability(strong_probability)?;- self.strong_probability = strong_probability;- Ok(self)- }-- /// Sets an optional seed for deterministic routing sequences.- pub fn with_rng_seed(mut self, rng_seed: impl Into<Option<u64>>) -> Self {- self.rng_seed = rng_seed.into();- self- }-- /// Validates the random-routing configuration.- pub fn validate(&self) -> Result<()> {- validate_probability(self.strong_probability)- }-}--/// Captures the chosen strong/weak side and target for downstream components.-#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]-pub struct RandomRoutingDecision {- pub tier: RandomRoutingTier,- pub selected_target: LlmTargetId,- pub selected_model: ModelId,- pub original_model: Option<String>,- pub strong_probability: f64,- pub draw: f64,-}--/// Pure random-routing engine decoupled from request mutation.-pub struct RandomRoutingEngine {- config: RandomRoutingProcessorConfig,- rng: Mutex<StdRng>,-}--impl fmt::Debug for RandomRoutingEngine {- fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {- f.debug_struct("RandomRoutingEngine")- .field("config", &self.config)- .finish_non_exhaustive()- }-}--impl RandomRoutingEngine {- /// Creates a routing engine and initializes its random number generator.- pub fn new(config: RandomRoutingProcessorConfig) -> Result<Self> {- config.validate()?;- let rng = match config.rng_seed {- Some(seed) => StdRng::seed_from_u64(seed),- None => rand::make_rng(),- };- Ok(Self {- config,- rng: Mutex::new(rng),- })- }-- /// Returns the immutable routing configuration.- pub fn config(&self) -> &RandomRoutingProcessorConfig {- &self.config- }-- /// Selects a target without mutating a request.- pub fn select(&self, original_model: Option<String>) -> Result<RandomRoutingDecision> {- let draw = self.next_draw();- let tier = if draw < self.config.strong_probability {- RandomRoutingTier::Strong- } else {- RandomRoutingTier::Weak- };- let selected = self.tier_config(tier);- Ok(RandomRoutingDecision {- tier,- selected_target: selected.id.clone(),- selected_model: selected.model.clone(),- original_model,- strong_probability: self.config.strong_probability,- draw,- })- }-- // Returns the configured target for the selected tier.- fn tier_config(&self, tier: RandomRoutingTier) -> &LlmTarget {- match tier {- RandomRoutingTier::Strong => &self.config.strong,- RandomRoutingTier::Weak => &self.config.weak,- }- }-- // Draws the next probability sample.- fn next_draw(&self) -> f64 {- (*self.rng.lock()).random()- }-}--// Validates the weighted random-routing probability.-fn validate_probability(strong_probability: f64) -> Result<()> {- if strong_probability.is_finite() && (0.0..=1.0).contains(&strong_probability) {- return Ok(());- }- Err(SwitchyardError::InvalidConfig(format!(- "strong_probability must be finite and in [0.0, 1.0], got {strong_probability:?}"- )))-}diff --git a/crates/switchyard-components/src/request_processors/stats.rs b/crates/switchyard-components/src/request_processors/stats.rsdeleted file mode 100644index 5bdda7142..000000000--- a/crates/switchyard-components/src/request_processors/stats.rs+++ /dev/null@@ -1,38 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Request-side stats processor.--use crate::{ChatRequest, ProxyContext, Result};--use crate::stats::{StatsRequestStart, prefix_probe};--/// Records request start time, and optionally the prefix fingerprints used for-/// switch-aware theoretical cache eligibility (gated, since fingerprinting hashes-/// the full prompt each turn and the per-model seen-sets grow over a run).-#[derive(Clone, Copy, Debug, Default)]-pub struct StatsRequestProcessor {- track_cache_eligibility: bool,-}--impl StatsRequestProcessor {- /// Creates a processor; `track_cache_eligibility` gates prefix fingerprinting.- pub fn new(track_cache_eligibility: bool) -> Self {- Self {- track_cache_eligibility,- }- }-- /// Records request-start metadata and returns the request unchanged.- pub async fn process(- &self,- ctx: &mut ProxyContext,- request: ChatRequest,- ) -> Result<ChatRequest> {- ctx.insert(StatsRequestStart::now());- if self.track_cache_eligibility {- ctx.insert(prefix_probe(request.body()));- }- Ok(request)- }-}diff --git a/crates/switchyard-components/src/response_processors/mod.rs b/crates/switchyard-components/src/response_processors/mod.rsdeleted file mode 100644index 453e0978f..000000000--- a/crates/switchyard-components/src/response_processors/mod.rs+++ /dev/null@@ -1,10 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Built-in response processor implementations.--pub mod response_signals;-pub mod stats;--pub use response_signals::ResponseSignalCollector;-pub use stats::*;diff --git a/crates/switchyard-components/src/response_processors/response_signals.rs b/crates/switchyard-components/src/response_processors/response_signals.rsdeleted file mode 100644index f546f93c2..000000000--- a/crates/switchyard-components/src/response_processors/response_signals.rs+++ /dev/null@@ -1,98 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Response-side context-signal collector.-//!-//! Thin adapter around-//! [`crate::dimension_collector::response::extract_response_signals`]. Stamps-//! the resulting [`ResponseSignals`] into `ProxyContext` so downstream-//! stage_router / escalation logic can read structured response-quality-//! flags without re-parsing the wire body.--use crate::{ChatResponse, ProxyContext, Result};--use crate::dimension_collector::response::{ResponseSignals, extract_response_signals};--/// Populates `ProxyContext` with [`ResponseSignals`] derived from the-/// buffered response body.-///-/// Stamps nothing for streaming responses — those can't be introspected-/// without consuming the stream. Consumers that read-/// `ctx.get::<ResponseSignals>()` and find `None` after this processor-/// ran should treat that as "stream / not-yet-checked," not as-/// "response was acceptable."-#[derive(Clone, Copy, Debug, Default)]-pub struct ResponseSignalCollector;--impl ResponseSignalCollector {- /// Extracts response-side signals from buffered responses and leaves streams untouched.- pub async fn process(- &self,- ctx: &mut ProxyContext,- response: ChatResponse,- ) -> Result<ChatResponse> {- // Only buffered responses carry an inspectable body. Streams pass- // through unchanged with no signals stamped.- if response.body().is_some() {- let signals = extract_response_signals(&response);- ctx.insert::<ResponseSignals>(signals);- }- Ok(response)- }-}--#[cfg(test)]-mod tests {- use super::*;- use crate::dimension_collector::response::ResponseFlag;- use serde_json::json;-- use crate::Result;-- #[tokio::test]- async fn stamps_response_signals_for_buffered_response() -> Result<()> {- let collector = ResponseSignalCollector;- let response = ChatResponse::openai_completion(json!({- "choices": [{- "message": {- "tool_calls": [{- "function": {- "name": "get_weather",- "arguments": "{\"city\":" // malformed- }- }]- },- "finish_reason": "tool_calls"- }]- }));-- let mut ctx = ProxyContext::new();- let _ = collector.process(&mut ctx, response).await?;-- let Some(signals) = ctx.get::<ResponseSignals>() else {- panic!("ResponseSignalCollector did not stamp ResponseSignals onto ctx");- };- assert!(signals.contains(ResponseFlag::MalformedToolCallJson));- Ok(())- }-- #[tokio::test]- async fn stamps_empty_signals_for_clean_response() -> Result<()> {- let collector = ResponseSignalCollector;- let response = ChatResponse::openai_completion(json!({- "choices": [{- "message": { "content": "looks good" },- "finish_reason": "stop"- }]- }));-- let mut ctx = ProxyContext::new();- let _ = collector.process(&mut ctx, response).await?;-- let Some(signals) = ctx.get::<ResponseSignals>() else {- panic!("ResponseSignalCollector did not stamp ResponseSignals onto ctx");- };- assert!(!signals.has_failures());- Ok(())- }-}diff --git a/crates/switchyard-components/src/response_processors/stats.rs b/crates/switchyard-components/src/response_processors/stats.rsdeleted file mode 100644index e5e4a657c..000000000--- a/crates/switchyard-components/src/response_processors/stats.rs+++ /dev/null@@ -1,249 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Response-side stats processor.--use crate::{BoxResponseStream, ChatResponse, ProxyContext, Result};-use async_stream::try_stream;-use futures_util::StreamExt;--use crate::stats::{- AnthropicStreamUsage, PrefixProbe, StatsAccumulator, StatsBackendLatency, StatsRequestStart,- TokenUsage, openai_chat_usage_from_stream_event, openai_responses_usage_from_stream_event,- selected_stats_model, selected_stats_tier, usage_from_body,-};--/// Records token usage, total latency, and routing overhead.-#[derive(Clone, Debug)]-pub struct StatsResponseProcessor {- accumulator: StatsAccumulator,-}--impl StatsResponseProcessor {- /// Creates a response processor sharing the supplied accumulator.- pub fn new(accumulator: StatsAccumulator) -> Self {- Self { accumulator }- }-- /// Returns the shared accumulator.- pub fn accumulator(&self) -> &StatsAccumulator {- &self.accumulator- }-- /// Records response usage and wraps streams so usage is captured on completion.- pub async fn process(- &self,- ctx: &mut ProxyContext,- response: ChatResponse,- ) -> Result<ChatResponse> {- let model = selected_stats_model(ctx, None);- let tier = selected_stats_tier(ctx);- let started_at = ctx.get::<StatsRequestStart>().copied();- let backend_latency = ctx.get::<StatsBackendLatency>().copied();- // Switch-aware: eligible only against the prefixes this model has already seen.- let cache_eligible = ctx- .get::<PrefixProbe>()- .map(|probe| self.accumulator.prefix_eligibility(&model, probe))- .unwrap_or(0.0);-- match response {- ChatResponse::OpenAiCompletion(response) => {- record_usage(- &self.accumulator,- &model,- usage_from_body(response.body()),- started_at,- backend_latency,- tier.as_deref(),- cache_eligible,- )?;- Ok(ChatResponse::OpenAiCompletion(response))- }- ChatResponse::OpenAiResponsesCompletion(response) => {- record_usage(- &self.accumulator,- &model,- usage_from_body(response.body()),- started_at,- backend_latency,- tier.as_deref(),- cache_eligible,- )?;- Ok(ChatResponse::OpenAiResponsesCompletion(response))- }- ChatResponse::AnthropicCompletion(response) => {- record_usage(- &self.accumulator,- &model,- usage_from_body(response.body()),- started_at,- backend_latency,- tier.as_deref(),- cache_eligible,- )?;- Ok(ChatResponse::AnthropicCompletion(response))- }- ChatResponse::OpenAiStream(stream) => {- Ok(ChatResponse::OpenAiStream(wrap_openai_chat_stream(- stream,- self.accumulator.clone(),- model,- started_at,- backend_latency,- tier,- cache_eligible,- )))- }- ChatResponse::OpenAiResponsesStream(stream) => Ok(ChatResponse::OpenAiResponsesStream(- wrap_openai_responses_stream(- stream,- self.accumulator.clone(),- model,- started_at,- backend_latency,- tier,- cache_eligible,- ),- )),- ChatResponse::AnthropicStream(stream) => {- Ok(ChatResponse::AnthropicStream(wrap_anthropic_stream(- stream,- self.accumulator.clone(),- model,- started_at,- backend_latency,- tier,- cache_eligible,- )))- }- }- }-}--fn wrap_openai_chat_stream(- mut stream: BoxResponseStream,- accumulator: StatsAccumulator,- model: String,- started_at: Option<StatsRequestStart>,- backend_latency: Option<StatsBackendLatency>,- tier: Option<String>,- cache_eligible: f64,-) -> BoxResponseStream {- Box::pin(try_stream! {- let mut committed = false;- while let Some(event) = stream.next().await {- let event = event?;- if !committed- && let Some(usage) = openai_chat_usage_from_stream_event(&event) {- log_stream_record_result(record_usage(- &accumulator,- &model,- usage,- started_at,- backend_latency,- tier.as_deref(),- cache_eligible,- ), &model);- committed = true;- }- yield event;- }- })-}--fn wrap_openai_responses_stream(- mut stream: BoxResponseStream,- accumulator: StatsAccumulator,- model: String,- started_at: Option<StatsRequestStart>,- backend_latency: Option<StatsBackendLatency>,- tier: Option<String>,- cache_eligible: f64,-) -> BoxResponseStream {- Box::pin(try_stream! {- let mut committed = false;- while let Some(event) = stream.next().await {- let event = event?;- if !committed- && let Some(usage) = openai_responses_usage_from_stream_event(&event) {- log_stream_record_result(record_usage(- &accumulator,- &model,- usage,- started_at,- backend_latency,- tier.as_deref(),- cache_eligible,- ), &model);- committed = true;- }- yield event;- }- })-}--fn wrap_anthropic_stream(- mut stream: BoxResponseStream,- accumulator: StatsAccumulator,- model: String,- started_at: Option<StatsRequestStart>,- backend_latency: Option<StatsBackendLatency>,- tier: Option<String>,- cache_eligible: f64,-) -> BoxResponseStream {- Box::pin(try_stream! {- let mut stream_usage = AnthropicStreamUsage::default();- while let Some(event) = stream.next().await {- let event = event?;- if let Some(usage) = stream_usage.observe(&event) {- log_stream_record_result(record_usage(- &accumulator,- &model,- usage,- started_at,- backend_latency,- tier.as_deref(),- cache_eligible,- ), &model);- }- yield event;- }- })-}--fn log_stream_record_result(result: Result<()>, model: &str) {- if let Err(error) = result {- tracing::warn!(- error = %error,- model = %model,- "failed to record stream usage"- );- }-}--fn record_usage(- accumulator: &StatsAccumulator,- model: &str,- mut usage: TokenUsage,- started_at: Option<StatsRequestStart>,- backend_latency: Option<StatsBackendLatency>,- tier: Option<&str>,- cache_eligible: f64,-) -> Result<()> {- usage.cacheable_prompt_tokens = (usage.prompt_tokens as f64 * cache_eligible).round() as u64;- let total_latency_ms = started_at.map(StatsRequestStart::elapsed_ms);- let backend_latency_ms = backend_latency.map(StatsBackendLatency::as_millis_f64);- let routing_overhead_ms =- total_latency_ms- .zip(backend_latency_ms)- .map(|(total_latency_ms, backend_latency_ms)| {- (total_latency_ms - backend_latency_ms).max(0.0)- });- accumulator.record_usage_after_success_attribution(- model.to_string(),- usage,- total_latency_ms,- routing_overhead_ms,- tier,- )-}diff --git a/crates/switchyard-components/src/stage_router.rs b/crates/switchyard-components/src/stage_router.rsdeleted file mode 100644index 04a845030..000000000--- a/crates/switchyard-components/src/stage_router.rs+++ /dev/null@@ -1,15 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Stage-router tier selection — thin re-export of libsy's decision core.-//!-//! The scoring and tier-selection logic lives in [`switchyard_libsy::stage_router`].-//! Its input, a [`ToolResultSignal`][crate::dimension_collector::ToolResultSignal],-//! is already libsy's `ToolSignals`, so no request adaptation is needed here — this-//! module simply re-exports the API so the crate's processors decide a turn's tier-//! through the same implementation as the libsy algorithm.--pub use switchyard_libsy::{- CodingAgentDimensions, DecisionSource, PickOutcome, PickerMode, ScoreResult, StageClassifier,- Tier, dimensions_from_signal, pick_tier, score_signal,-};diff --git a/crates/switchyard-components/src/stats/accumulator.rs b/crates/switchyard-components/src/stats/accumulator.rsdeleted file mode 100644index 455c8cafe..000000000--- a/crates/switchyard-components/src/stats/accumulator.rs+++ /dev/null@@ -1,792 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Thread-safe stats accumulator and serializable snapshot schema.--use std::cmp::{Ordering, Reverse};-use std::collections::{BTreeMap, BinaryHeap, HashSet, btree_map::Entry};-use std::sync::Arc;--use crate::Result;-use parking_lot::{Mutex, MutexGuard};-use serde::{Deserialize, Serialize};--use super::cost::{CostEstimate, estimate_cost};-use super::{PrefixProbe, TokenUsage};--const MAX_LATENCY_SAMPLES: usize = 10_000;--/// Thread-safe stats store shared by stats processors and backend wrappers.-#[derive(Clone, Debug)]-pub struct StatsAccumulator {- inner: Arc<Mutex<StatsAccumulatorInner>>,-}--impl Default for StatsAccumulator {- fn default() -> Self {- Self::new()- }-}--impl StatsAccumulator {- /// Creates an empty stats accumulator.- pub fn new() -> Self {- Self {- inner: Arc::new(Mutex::new(StatsAccumulatorInner::default())),- }- }-- /// Records a successful backend call.- pub fn record_success(- &self,- model: impl Into<String>,- backend_latency_ms: Option<f64>,- tier: Option<&str>,- ) -> Result<()> {- let mut inner = self.lock();- inner.total_requests = inner.total_requests.saturating_add(1);- let model = model.into();- let tier = tier.map(str::trim).filter(|tier| !tier.is_empty());- {- let stats = inner.model_stats_mut(model.clone());- stats.calls = stats.calls.saturating_add(1);- if let Some(tier) = tier {- stats.tier = Some(tier.to_string());- }- if let Some(latency) = backend_latency_ms {- stats.model_call_latency.record(latency);- }- }- if let Some(tier) = tier {- if let Some(tier_stats) = inner.tier_stats_mut(tier, &model) {- tier_stats.calls = tier_stats.calls.saturating_add(1);- }- } else {- inner.record_untiered_success(&model);- }- Ok(())- }-- /// Records a backend error.- pub fn record_error(&self, model: impl Into<String>, tier: Option<&str>) -> Result<()> {- let mut inner = self.lock();- inner.total_requests = inner.total_requests.saturating_add(1);- inner.total_errors = inner.total_errors.saturating_add(1);- let stats = inner.model_stats_mut(model.into());- stats.errors = stats.errors.saturating_add(1);- if let Some(tier) = tier {- stats.tier = Some(tier.to_string());- }- Ok(())- }-- /// Returns the cache-eligible fraction for `model` and records the prefix as seen.- ///- /// Switch-aware: a prefix counts only if this model was previously sent it, so a- /// switch to a cold model yields 0 and a return credits only the shared prefix.- pub fn prefix_eligibility(&self, model: &str, probe: &PrefixProbe) -> f64 {- let mut inner = self.lock();- let stats = inner.model_stats_mut(model.to_string());- let fraction = probe.eligible_fraction(&stats.seen_prefixes);- if let Some(hash) = probe.full_hash() {- stats.seen_prefixes.insert(hash);- }- fraction- }-- /// Records token usage and end-to-end latency for a completed response.- pub fn record_usage(- &self,- model: impl Into<String>,- usage: TokenUsage,- total_latency_ms: Option<f64>,- routing_overhead_ms: Option<f64>,- tier: Option<&str>,- ) -> Result<()> {- self.record_usage_inner(- model,- usage,- total_latency_ms,- routing_overhead_ms,- tier,- TierCallAttribution::LegacyPendingUntiered,- )- }-- /// Records usage and attaches one previously untiered success to `tier`.- ///- /// This is only for compatibility paths that recorded success before the- /// route label was available. Normal stats backends record success with the- /// tier already present, so their usage events must leave this flag false.- pub fn record_usage_with_success_was_untiered(- &self,- model: impl Into<String>,- usage: TokenUsage,- total_latency_ms: Option<f64>,- routing_overhead_ms: Option<f64>,- tier: Option<&str>,- ) -> Result<()> {- self.record_usage_inner(- model,- usage,- total_latency_ms,- routing_overhead_ms,- tier,- TierCallAttribution::ExplicitUntiered,- )- }-- /// Records usage after the corresponding success call was already attributed.- ///- /// `StatsLlmBackend` and routing runtimes record success before the response- /// processor records tokens. Those internal paths must not consume a legacy- /// pending untiered success that belongs to some other direct accumulator caller.- pub fn record_usage_after_success_attribution(- &self,- model: impl Into<String>,- usage: TokenUsage,- total_latency_ms: Option<f64>,- routing_overhead_ms: Option<f64>,- tier: Option<&str>,- ) -> Result<()> {- self.record_usage_inner(- model,- usage,- total_latency_ms,- routing_overhead_ms,- tier,- TierCallAttribution::AlreadyRecorded,- )- }-- fn record_usage_inner(- &self,- model: impl Into<String>,- usage: TokenUsage,- total_latency_ms: Option<f64>,- routing_overhead_ms: Option<f64>,- tier: Option<&str>,- tier_call_attribution: TierCallAttribution,- ) -> Result<()> {- let mut inner = self.lock();- let model = model.into();- {- let stats = inner.model_stats_mut(model.clone());- stats.prompt_tokens = stats.prompt_tokens.saturating_add(usage.prompt_tokens);- stats.max_observed_context_tokens = stats- .max_observed_context_tokens- .max(usage.prompt_tokens.saturating_add(usage.completion_tokens));- stats.completion_tokens = stats- .completion_tokens- .saturating_add(usage.completion_tokens);- stats.cached_tokens = stats.cached_tokens.saturating_add(usage.cached_tokens);- stats.cache_creation_tokens = stats- .cache_creation_tokens- .saturating_add(usage.cache_creation_tokens);- stats.cacheable_prompt_tokens = stats- .cacheable_prompt_tokens- .saturating_add(usage.cacheable_prompt_tokens);- stats.reasoning_tokens = stats- .reasoning_tokens- .saturating_add(usage.reasoning_tokens);- if let Some(tier) = tier {- stats.tier = Some(tier.to_string());- }- if let Some(latency) = total_latency_ms {- stats.total_latency.record(latency);- }- }- if let Some(tier) = tier {- let tier = tier.trim();- if !tier.is_empty() {- let should_attribute_call = match tier_call_attribution {- TierCallAttribution::LegacyPendingUntiered => {- inner.consume_untiered_success(&model)- }- TierCallAttribution::ExplicitUntiered => {- inner.consume_untiered_success(&model);- true- }- TierCallAttribution::AlreadyRecorded => false,- };- if let Some(tier_stats) = inner.tier_stats_mut(tier, &model) {- if should_attribute_call {- tier_stats.calls = tier_stats.calls.saturating_add(1);- }- tier_stats.prompt_tokens =- tier_stats.prompt_tokens.saturating_add(usage.prompt_tokens);- tier_stats.completion_tokens = tier_stats- .completion_tokens- .saturating_add(usage.completion_tokens);- }- }- }- if let Some(overhead) = routing_overhead_ms {- inner.routing_overhead.record(overhead);- }- Ok(())- }-- /// Records one LLM-classifier overhead call.- ///- /// The classifier's per-request call is not part of the routed-backend- /// chain and must stay out of `by_model` — otherwise the default TB-lite- /// config (classifier model == efficient-tier model) double-counts the spend.- /// `record_classifier_usage` writes to a dedicated bucket; the snapshot- /// exposes it under `classifier.models` with its own `cost_estimate`.- pub fn record_classifier_usage(- &self,- model: impl Into<String>,- usage: TokenUsage,- latency_ms: Option<f64>,- ) -> Result<()> {- let mut inner = self.lock();- inner.classifier_requests = inner.classifier_requests.saturating_add(1);- let stats = inner.classifier_stats_mut(model.into());- stats.calls = stats.calls.saturating_add(1);- stats.prompt_tokens = stats.prompt_tokens.saturating_add(usage.prompt_tokens);- stats.max_observed_context_tokens = stats- .max_observed_context_tokens- .max(usage.prompt_tokens.saturating_add(usage.completion_tokens));- stats.completion_tokens = stats- .completion_tokens- .saturating_add(usage.completion_tokens);- stats.cached_tokens = stats.cached_tokens.saturating_add(usage.cached_tokens);- stats.cache_creation_tokens = stats- .cache_creation_tokens- .saturating_add(usage.cache_creation_tokens);- stats.reasoning_tokens = stats- .reasoning_tokens- .saturating_add(usage.reasoning_tokens);- if let Some(latency) = latency_ms {- stats.model_call_latency.record(latency);- stats.total_latency.record(latency);- }- Ok(())- }-- /// Records a classifier-call failure.- ///- /// Bumps the classifier `total_requests` (so the failure shows up in- /// the per-request distribution) and `total_errors`, plus the- /// per-model `errors` counter. Does **not** bump `calls` — that- /// field counts completed (token-bearing) calls only, so the- /// `errors / (calls + errors)` ratio is the failure rate.- pub fn record_classifier_error(&self, model: impl Into<String>) -> Result<()> {- let mut inner = self.lock();- inner.classifier_requests = inner.classifier_requests.saturating_add(1);- inner.classifier_errors = inner.classifier_errors.saturating_add(1);- let stats = inner.classifier_stats_mut(model.into());- stats.errors = stats.errors.saturating_add(1);- Ok(())- }-- /// Records one routing decision source for a routing algorithm.- ///- /// This is intentionally separate from model/tier accounting: a stage-router can- /// choose `efficient` because of an override, a dimensions score, an LLM-classifier- /// verdict, or a fail-open default, and those explanations are useful even- /// when they all land on the same backend model.- pub fn record_routing_decision(- &self,- profile_type: impl Into<String>,- source: impl Into<String>,- ) -> Result<()> {- let mut inner = self.lock();- let sources = inner- .routing_decisions- .entry(profile_type.into())- .or_default();- let count = sources.entry(source.into()).or_insert(0);- *count = count.saturating_add(1);- Ok(())- }-- /// Returns a computed snapshot suitable for JSON serialization.- pub fn snapshot(&self) -> Result<StatsSnapshot> {- let inner = self.lock().clone();- Ok(inner.snapshot())- }-- /// Clears all counters.- pub fn reset(&self) -> Result<()> {- let mut inner = self.lock();- *inner = StatsAccumulatorInner::default();- Ok(())- }-- fn lock(&self) -> MutexGuard<'_, StatsAccumulatorInner> {- self.inner.lock()- }-}--#[derive(Clone, Copy, Debug, Eq, PartialEq)]-enum TierCallAttribution {- LegacyPendingUntiered,- ExplicitUntiered,- AlreadyRecorded,-}--#[derive(Clone, Debug, Default)]-struct StatsAccumulatorInner {- by_model: BTreeMap<String, ModelStats>,- model_order: Vec<String>,- untiered_successes_by_model: BTreeMap<String, u64>,- by_tier: BTreeMap<String, TierStats>,- total_requests: u64,- total_errors: u64,- routing_overhead: LatencyHistogram,- by_classifier: BTreeMap<String, ModelStats>,- classifier_model_order: Vec<String>,- classifier_requests: u64,- classifier_errors: u64,- routing_decisions: BTreeMap<String, BTreeMap<String, u64>>,-}--impl StatsAccumulatorInner {- fn model_stats_mut(&mut self, model: String) -> &mut ModelStats {- match self.by_model.entry(model) {- Entry::Occupied(entry) => entry.into_mut(),- Entry::Vacant(entry) => {- self.model_order.push(entry.key().clone());- entry.insert(ModelStats::default())- }- }- }-- fn classifier_stats_mut(&mut self, model: String) -> &mut ModelStats {- match self.by_classifier.entry(model) {- Entry::Occupied(entry) => entry.into_mut(),- Entry::Vacant(entry) => {- self.classifier_model_order.push(entry.key().clone());- entry.insert(ModelStats::default())- }- }- }-- fn tier_stats_mut(&mut self, tier: &str, model: &str) -> Option<&mut TierStats> {- let tier = tier.trim();- if tier.is_empty() {- return None;- }- Some(match self.by_tier.entry(tier.to_string()) {- Entry::Occupied(entry) => entry.into_mut(),- Entry::Vacant(entry) => entry.insert(TierStats {- model: model.to_string(),- ..TierStats::default()- }),- })- }-- fn record_untiered_success(&mut self, model: &str) {- let count = self- .untiered_successes_by_model- .entry(model.to_string())- .or_insert(0);- *count = count.saturating_add(1);- }-- fn consume_untiered_success(&mut self, model: &str) -> bool {- match self.untiered_successes_by_model.entry(model.to_string()) {- Entry::Occupied(mut entry) => {- let count = entry.get_mut();- if *count > 1 {- *count -= 1;- } else {- entry.remove();- }- true- }- Entry::Vacant(_) => false,- }- }-- fn snapshot(&self) -> StatsSnapshot {- let (models, totals) = build_model_snapshots(&self.by_model, self.total_requests);- let total_tokens = totals.total;- let mut cost_estimate = estimate_cost(&models);-- let classifier = build_classifier_snapshot(- &self.by_classifier,- self.classifier_requests,- self.classifier_errors,- );- cost_estimate.classifier_cost = classifier.cost_estimate.total_cost;-- cost_estimate.total_cost =- round6(cost_estimate.backend_cost + cost_estimate.classifier_cost);-- StatsSnapshot {- total_requests: self.total_requests,- total_errors: self.total_errors,- total_tokens: totals,- tiers: tier_snapshots(&self.by_tier, total_tokens, self.total_requests),- cost_estimate,- models,- routing_overhead: self.routing_overhead.snapshot(),- classifier,- routing_decisions: self.routing_decisions.clone(),- }- }-}--fn build_model_snapshots(- by_model: &BTreeMap<String, ModelStats>,- total_requests: u64,-) -> (BTreeMap<String, ModelStatsSnapshot>, TokenTotals) {- let mut totals = TokenTotals::default();- for stats in by_model.values() {- totals.prompt = totals.prompt.saturating_add(stats.prompt_tokens);- totals.completion = totals.completion.saturating_add(stats.completion_tokens);- totals.cached = totals.cached.saturating_add(stats.cached_tokens);- totals.cache_creation = totals- .cache_creation- .saturating_add(stats.cache_creation_tokens);- totals.reasoning = totals.reasoning.saturating_add(stats.reasoning_tokens);- }- totals.total = totals.prompt.saturating_add(totals.completion);-- let mut models = BTreeMap::new();- for (model, stats) in by_model {- let token_total = stats.prompt_tokens.saturating_add(stats.completion_tokens);- let request_pct = if total_requests == 0 {- 0.0- } else {- round2(stats.calls as f64 / total_requests as f64 * 100.0)- };- let token_pct = if totals.total == 0 {- 0.0- } else {- round2(token_total as f64 / totals.total as f64 * 100.0)- };- let avg_prompt_tokens = if stats.calls == 0 {- 0.0- } else {- round2(stats.prompt_tokens as f64 / stats.calls as f64)- };- let avg_completion_tokens = if stats.calls == 0 {- 0.0- } else {- round2(stats.completion_tokens as f64 / stats.calls as f64)- };- let cache_hit_rate = if stats.prompt_tokens == 0 {- 0.0- } else {- round4(stats.cached_tokens as f64 / stats.prompt_tokens as f64)- };- // Switch-aware ceiling: prefix this model had already seen; gap to actual is the backend.- let theoretical_cache_hit_rate = if stats.prompt_tokens == 0 {- 0.0- } else {- round4(stats.cacheable_prompt_tokens as f64 / stats.prompt_tokens as f64)- };-- models.insert(- model.clone(),- ModelStatsSnapshot {- calls: stats.calls,- errors: stats.errors,- request_pct,- prompt_tokens: stats.prompt_tokens,- max_observed_context_tokens: stats.max_observed_context_tokens,- completion_tokens: stats.completion_tokens,- total_tokens: token_total,- token_pct,- cached_tokens: stats.cached_tokens,- cache_creation_tokens: stats.cache_creation_tokens,- reasoning_tokens: stats.reasoning_tokens,- avg_prompt_tokens,- avg_completion_tokens,- cache_hit_rate,- theoretical_cache_hit_rate,- model_call_latency: stats.model_call_latency.snapshot(),- total_latency: stats.total_latency.snapshot(),- tier: stats.tier.clone(),- },- );- }- (models, totals)-}--fn build_classifier_snapshot(- by_classifier: &BTreeMap<String, ModelStats>,- total_requests: u64,- total_errors: u64,-) -> ClassifierStatsSnapshot {- let (models, totals) = build_model_snapshots(by_classifier, total_requests);- let cost_estimate = estimate_cost(&models);- ClassifierStatsSnapshot {- total_requests,- total_errors,- total_tokens: totals,- models,- cost_estimate,- }-}--#[derive(Clone, Debug, Default)]-struct ModelStats {- calls: u64,- errors: u64,- prompt_tokens: u64,- max_observed_context_tokens: u64,- completion_tokens: u64,- cached_tokens: u64,- cache_creation_tokens: u64,- cacheable_prompt_tokens: u64,- reasoning_tokens: u64,- /// Prefix fingerprints this model has been sent; basis for switch-aware theoretical.- seen_prefixes: HashSet<u64>,- model_call_latency: LatencyHistogram,- total_latency: LatencyHistogram,- tier: Option<String>,-}--#[derive(Clone, Debug, Default)]-struct TierStats {- model: String,- calls: u64,- prompt_tokens: u64,- completion_tokens: u64,-}--#[derive(Clone, Debug)]-struct LatencyHistogram {- count: u64,- total_ms: f64,- min_ms: f64,- max_ms: f64,- samples: BinaryHeap<Reverse<LatencySample>>,-}--impl Default for LatencyHistogram {- fn default() -> Self {- Self {- count: 0,- total_ms: 0.0,- min_ms: f64::INFINITY,- max_ms: 0.0,- samples: BinaryHeap::new(),- }- }-}--impl LatencyHistogram {- fn record(&mut self, latency_ms: f64) {- if !latency_ms.is_finite() {- tracing::debug!(latency_ms, "dropping non-finite latency sample");- return;- }- let latency_ms = latency_ms.max(0.0);- self.count = self.count.saturating_add(1);- self.total_ms += latency_ms;- self.min_ms = self.min_ms.min(latency_ms);- self.max_ms = self.max_ms.max(latency_ms);- let sample = Reverse(LatencySample(latency_ms));- if self.samples.len() < MAX_LATENCY_SAMPLES {- self.samples.push(sample);- } else if let Some(smallest_sample) = self.samples.peek()- && latency_ms > smallest_sample.0.value()- && let Some(mut smallest_sample) = self.samples.peek_mut()- {- *smallest_sample = sample;- }- }-- fn snapshot(&self) -> LatencyHistogramSnapshot {- if self.count == 0 {- return LatencyHistogramSnapshot::default();- }- let mut samples = self- .samples- .iter()- .map(|sample| sample.0.value())- .collect::<Vec<_>>();- samples.sort_by(f64::total_cmp);- let sample_count = samples.len();- let p50_ms = samples- .get(sample_count / 2)- .copied()- .map(round2)- .unwrap_or(0.0);- let p99_index = sample_count- .saturating_sub(1)- .min((sample_count as f64 * 0.99) as usize);- let p99_ms = samples.get(p99_index).copied().map(round2).unwrap_or(0.0);-- LatencyHistogramSnapshot {- count: self.count,- total_ms: round2(self.total_ms),- min_ms: round2(self.min_ms),- max_ms: round2(self.max_ms),- avg_ms: round2(self.total_ms / self.count as f64),- p50_ms,- p99_ms,- }- }-}--#[derive(Clone, Copy, Debug)]-struct LatencySample(f64);--impl LatencySample {- fn value(self) -> f64 {- self.0- }-}--impl PartialEq for LatencySample {- fn eq(&self, other: &Self) -> bool {- self.0.total_cmp(&other.0) == Ordering::Equal- }-}--impl Eq for LatencySample {}--impl PartialOrd for LatencySample {- fn partial_cmp(&self, other: &Self) -> Option<Ordering> {- Some(self.cmp(other))- }-}--impl Ord for LatencySample {- fn cmp(&self, other: &Self) -> Ordering {- self.0.total_cmp(&other.0)- }-}--/// Full stats snapshot.-#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]-pub struct StatsSnapshot {- pub total_requests: u64,- pub total_errors: u64,- pub total_tokens: TokenTotals,- pub models: BTreeMap<String, ModelStatsSnapshot>,- pub tiers: BTreeMap<String, TierStatsSnapshot>,- pub routing_overhead: LatencyHistogramSnapshot,- pub cost_estimate: CostEstimate,- pub classifier: ClassifierStatsSnapshot,- pub routing_decisions: BTreeMap<String, BTreeMap<String, u64>>,-}--/// LLM-classifier overhead stats, recorded out-of-band from routed-backend-/// traffic so the two never alias on a shared model id.-#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]-pub struct ClassifierStatsSnapshot {- pub total_requests: u64,- pub total_errors: u64,- pub total_tokens: TokenTotals,- pub models: BTreeMap<String, ModelStatsSnapshot>,- pub cost_estimate: CostEstimate,-}--/// Aggregate token totals.-#[derive(Clone, Debug, Default, Eq, PartialEq, Serialize, Deserialize)]-pub struct TokenTotals {- pub prompt: u64,- pub completion: u64,- pub cached: u64,- pub cache_creation: u64,- pub reasoning: u64,- pub total: u64,-}--/// Per-model stats in a snapshot.-#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]-pub struct ModelStatsSnapshot {- pub calls: u64,- pub errors: u64,- pub request_pct: f64,- pub prompt_tokens: u64,- /// Largest prompt-plus-completion token count observed in one completed response.- pub max_observed_context_tokens: u64,- pub completion_tokens: u64,- pub total_tokens: u64,- pub token_pct: f64,- pub cached_tokens: u64,- pub cache_creation_tokens: u64,- pub reasoning_tokens: u64,- pub avg_prompt_tokens: f64,- pub avg_completion_tokens: f64,- pub cache_hit_rate: f64,- /// Switch-aware ceiling: fraction of the prompt this model had already been sent.- pub theoretical_cache_hit_rate: f64,- pub model_call_latency: LatencyHistogramSnapshot,- pub total_latency: LatencyHistogramSnapshot,- pub tier: Option<String>,-}--/// Per-tier stats in a snapshot.-#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]-pub struct TierStatsSnapshot {- pub model: String,- pub calls: u64,- pub request_pct: f64,- pub prompt_tokens: u64,- pub completion_tokens: u64,- pub total_tokens: u64,- pub token_pct: f64,-}--/// Latency histogram summary in milliseconds.-///-/// Non-finite latency samples are ignored and logged at debug level.-#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]-pub struct LatencyHistogramSnapshot {- pub count: u64,- pub total_ms: f64,- pub min_ms: f64,- pub max_ms: f64,- pub avg_ms: f64,- pub p50_ms: f64,- pub p99_ms: f64,-}--fn tier_snapshots(- by_tier: &BTreeMap<String, TierStats>,- total_tokens: u64,- total_requests: u64,-) -> BTreeMap<String, TierStatsSnapshot> {- let mut tiers = BTreeMap::new();- for (tier, stats) in by_tier {- tiers.insert(- tier.clone(),- TierStatsSnapshot {- model: stats.model.clone(),- calls: stats.calls,- request_pct: 0.0,- prompt_tokens: stats.prompt_tokens,- completion_tokens: stats.completion_tokens,- total_tokens: 0,- token_pct: 0.0,- },- );- }-- for tier in tiers.values_mut() {- tier.total_tokens = tier.prompt_tokens.saturating_add(tier.completion_tokens);- tier.request_pct = if total_requests == 0 {- 0.0- } else {- round2(tier.calls as f64 / total_requests as f64 * 100.0)- };- tier.token_pct = if total_tokens == 0 {- 0.0- } else {- round2(tier.total_tokens as f64 / total_tokens as f64 * 100.0)- };- }- tiers-}--fn round2(value: f64) -> f64 {- (value * 100.0).round() / 100.0-}--fn round4(value: f64) -> f64 {- (value * 10_000.0).round() / 10_000.0-}--fn round6(value: f64) -> f64 {- (value * 1_000_000.0).round() / 1_000_000.0-}diff --git a/crates/switchyard-components/src/stats/cache_eligibility.rs b/crates/switchyard-components/src/stats/cache_eligibility.rsdeleted file mode 100644index 9724e04ca..000000000--- a/crates/switchyard-components/src/stats/cache_eligibility.rs+++ /dev/null@@ -1,164 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Switch-aware cache eligibility: how much of a prompt a model has already been sent.--use std::collections::HashSet;-use std::collections::hash_map::DefaultHasher;-use std::hash::{Hash, Hasher};--use serde_json::Value;--/// Env var that opts in to per-model theoretical cache-hit tracking.-const TRACK_ENV: &str = "SWITCHYARD_THEORETICAL_CACHE";--/// Whether theoretical cache-hit tracking is enabled via the environment.-///-/// Off by default: prefix fingerprinting and the per-model seen-sets are skipped-/// unless opted in, so the hot path adds nothing and memory stays flat.-pub fn tracking_enabled_from_env() -> bool {- env_opts_in(std::env::var(TRACK_ENV).ok().as_deref())-}--fn env_opts_in(value: Option<&str>) -> bool {- matches!(- value.map(|v| v.trim().to_ascii_lowercase()).as_deref(),- Some("1" | "true" | "yes" | "on")- )-}--/// Cumulative prefix fingerprints of a request, in message order.-#[derive(Clone, Debug, Default)]-pub struct PrefixProbe {- /// `(cumulative_text_len, rolling_hash)` after the prefix fields and each turn.- boundaries: Vec<(u64, u64)>,- /// Total prompt text length, including the newest turn.- total_len: u64,-}--impl PrefixProbe {- /// Eligible fraction given the prefix fingerprints a model has already seen.- /// Longest matching boundary over total; the newest turn is unseen, so excluded.- pub fn eligible_fraction(&self, seen: &HashSet<u64>) -> f64 {- if self.total_len == 0 {- return 0.0;- }- let eligible = self- .boundaries- .iter()- .filter(|(_, hash)| seen.contains(hash))- .map(|(len, _)| *len)- .max()- .unwrap_or(0);- eligible as f64 / self.total_len as f64- }-- /// Fingerprint of the full prompt, recorded once a model has processed it.- pub fn full_hash(&self) -> Option<u64> {- self.boundaries.last().map(|(_, hash)| *hash)- }-}--/// Builds prefix fingerprints from a request body.-/// Format-agnostic: `system`/`instructions`, then each `messages`/`input` turn.-pub fn prefix_probe(body: &Value) -> PrefixProbe {- let mut boundaries = Vec::new();- let mut acc_len = 0u64;- let mut hasher = DefaultHasher::new();-- let system = body.get("system");- let instructions = body.get("instructions");- let prefix_len = system.map(text_len).unwrap_or(0) + instructions.map(text_len).unwrap_or(0);- if prefix_len > 0 {- acc_len += prefix_len;- for value in [system, instructions].into_iter().flatten() {- hash_text_into(value, &mut hasher);- }- boundaries.push((acc_len, hasher.finish()));- }-- let turns = body- .get("messages")- .or_else(|| body.get("input"))- .and_then(Value::as_array);- if let Some(turns) = turns {- for turn in turns {- acc_len += text_len(turn);- hash_text_into(turn, &mut hasher);- boundaries.push((acc_len, hasher.finish()));- }- }- PrefixProbe {- boundaries,- total_len: acc_len,- }-}--/// Recursively sums the byte length of every JSON string value.-fn text_len(value: &Value) -> u64 {- match value {- Value::String(s) => s.len() as u64,- Value::Array(items) => items.iter().map(text_len).sum(),- Value::Object(map) => map.values().map(text_len).sum(),- _ => 0,- }-}--/// Recursively feeds every JSON scalar value into the hasher, in order.-/// Includes numbers and bools so prompts differing only in a scalar don't collide.-fn hash_text_into(value: &Value, hasher: &mut DefaultHasher) {- match value {- Value::String(s) => s.hash(hasher),- Value::Number(n) => n.to_string().hash(hasher),- Value::Bool(b) => b.hash(hasher),- Value::Array(items) => items.iter().for_each(|item| hash_text_into(item, hasher)),- Value::Object(map) => map.values().for_each(|val| hash_text_into(val, hasher)),- Value::Null => {}- }-}--#[cfg(test)]-mod tests {- use super::*;- use serde_json::json;-- #[test]- fn env_opt_in_parsing() {- assert!(env_opts_in(Some("1")));- assert!(env_opts_in(Some(" TRUE ")));- assert!(env_opts_in(Some("on")));- assert!(!env_opts_in(Some("0")));- assert!(!env_opts_in(Some("false")));- assert!(!env_opts_in(None));- }-- #[test]- fn empty_body_is_zero() {- assert_eq!(- prefix_probe(&json!({})).eligible_fraction(&HashSet::new()),- 0.0- );- }-- #[test]- fn unseen_prefix_is_not_eligible() {- let probe = prefix_probe(&json!({"messages": [{"role": "user", "content": "aaaa"}]}));- assert_eq!(probe.eligible_fraction(&HashSet::new()), 0.0);- }-- #[test]- fn previously_seen_prefix_is_eligible_newest_turn_is_not() {- let turn1 = prefix_probe(&json!({"messages": [{"role": "user", "content": "aaaa"}]}));- let mut seen = HashSet::new();- seen.insert(turn1.full_hash().unwrap());-- // Same first turn plus an equal-length newest turn -> half is re-presentable.- let turn2 = prefix_probe(&json!({- "messages": [- {"role": "user", "content": "aaaa"},- {"role": "user", "content": "bbbb"},- ],- }));- assert_eq!(turn2.eligible_fraction(&seen), 0.5);- }-}diff --git a/crates/switchyard-components/src/stats/context.rs b/crates/switchyard-components/src/stats/context.rsdeleted file mode 100644index 6d397eb97..000000000--- a/crates/switchyard-components/src/stats/context.rs+++ /dev/null@@ -1,72 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Typed context markers shared by stats components.--use std::time::{Duration, Instant};--use crate::ProxyContext;-use serde::{Deserialize, Serialize};--use crate::backends::BackendSelection;-use crate::request_processors::RandomRoutingDecision;--/// Request start time captured by `StatsRequestProcessor`.-#[derive(Clone, Copy, Debug)]-pub struct StatsRequestStart(Instant);--impl StatsRequestStart {- /// Captures the current monotonic clock instant.- pub fn now() -> Self {- Self(Instant::now())- }-- /// Returns elapsed milliseconds since the captured start.- pub fn elapsed_ms(self) -> f64 {- self.0.elapsed().as_secs_f64() * 1000.0- }-}--/// Backend-call duration captured by `StatsLlmBackend`.-#[derive(Clone, Copy, Debug)]-pub struct StatsBackendLatency(pub Duration);--impl StatsBackendLatency {- /// Converts the duration to milliseconds.- pub fn as_millis_f64(self) -> f64 {- self.0.as_secs_f64() * 1000.0- }-}--/// Optional generic tier label for non-random routing stats.-#[derive(Clone, Debug, Eq, PartialEq, Serialize, Deserialize)]-pub struct StatsRouteLabel(pub String);--impl StatsRouteLabel {- /// Creates a route label, preserving caller-provided naming.- pub fn new(label: impl Into<String>) -> Self {- Self(label.into())- }-}--/// Returns the selected model label for stats attribution.-pub fn selected_stats_model(ctx: &ProxyContext, fallback: Option<&str>) -> String {- ctx.get::<BackendSelection>()- .map(|selection| selection.model.as_str().to_string())- .or_else(|| fallback.map(str::to_string))- .unwrap_or_else(|| "<unknown>".to_string())-}--/// Returns a tier label when a routing component supplied one.-pub fn selected_stats_tier(ctx: &ProxyContext) -> Option<String> {- if let Some(label) = ctx.get::<StatsRouteLabel>() {- return Some(label.0.clone());- }- if let Some(decision) = ctx.get::<RandomRoutingDecision>() {- return Some(decision.tier.as_str().to_string());- }- // Fallback: Python-based pickers (e.g. stage_router) stamp selected_target but- // don't write a typed marker — use it as the tier label so /v1/routing/stats- // populates the `tiers` field for those routes too.- ctx.selected_target().map(|t| t.as_str().to_string())-}diff --git a/crates/switchyard-components/src/stats/cost.rs b/crates/switchyard-components/src/stats/cost.rsdeleted file mode 100644index 8d1ba43ce..000000000--- a/crates/switchyard-components/src/stats/cost.rs+++ /dev/null@@ -1,269 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Best-effort token cost estimation for stats snapshots.--use std::collections::BTreeMap;--use serde::{Deserialize, Serialize};--use super::accumulator::ModelStatsSnapshot;--/// Cost estimate for all recorded models.-///-/// `total_cost` is the grand total spend including overhead calls;-/// `backend_cost` is the portion attributed to routed-backend traffic;-/// `classifier_cost` is the LLM-classifier-overhead portion (zero unless-/// `record_classifier_usage` has been called). The split exists because the-/// default TB-lite configs can use the same model id for the classifier and-/// backend buckets, and a single-row aggregation cannot distinguish them.-#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]-pub struct CostEstimate {- pub models: BTreeMap<String, CostBreakdown>,- pub total_cost: f64,- pub backend_cost: f64,- pub classifier_cost: f64,-}--/// Per-model cost breakdown.-#[derive(Clone, Copy, Debug, Default, PartialEq, Serialize, Deserialize)]-pub struct CostBreakdown {- pub base_input_cost: f64,- pub cached_input_cost: f64,- pub cache_write_cost: f64,- pub input_cost: f64,- pub output_cost: f64,- pub total_cost: f64,-}--#[derive(Clone, Copy, Debug)]-struct ModelPrice {- input: f64,- output: f64,- cached: f64,- cache_write: f64,-}--/// Estimates token cost from per-model stats.-pub fn estimate_cost(models: &BTreeMap<String, ModelStatsSnapshot>) -> CostEstimate {- let mut estimated = BTreeMap::new();- let mut total_cost = 0.0;-- for (model, stats) in models {- let breakdown = estimate_model_cost(- model,- stats.prompt_tokens,- stats.completion_tokens,- stats.cached_tokens,- stats.cache_creation_tokens,- );- total_cost += breakdown.total_cost;- estimated.insert(model.clone(), breakdown);- }-- let total = round6(total_cost);- CostEstimate {- models: estimated,- total_cost: total,- backend_cost: total,- classifier_cost: 0.0,- }-}--pub fn estimate_model_cost(- model: &str,- prompt_tokens: u64,- completion_tokens: u64,- cached_tokens: u64,- cache_creation_tokens: u64,-) -> CostBreakdown {- let prices = raw_model_price(model).unwrap_or(ModelPrice {- input: 0.0,- output: 0.0,- cached: 0.0,- cache_write: 0.0,- });- let base_input = prompt_tokens- .saturating_sub(cached_tokens)- .saturating_sub(cache_creation_tokens);- let base_input_cost = base_input as f64 / 1e6 * prices.input;- let cached_input_cost = cached_tokens as f64 / 1e6 * prices.cached;- let cache_write_cost = cache_creation_tokens as f64 / 1e6 * prices.cache_write;- let input_cost = base_input_cost + cached_input_cost + cache_write_cost;- let output_cost = completion_tokens as f64 / 1e6 * prices.output;- CostBreakdown {- base_input_cost: round6(base_input_cost),- cached_input_cost: round6(cached_input_cost),- cache_write_cost: round6(cache_write_cost),- input_cost: round6(input_cost),- output_cost: round6(output_cost),- total_cost: round6(input_cost + output_cost),- }-}--pub fn has_model_price(model: &str) -> bool {- raw_model_price(model).is_some()-}--fn raw_model_price(model: &str) -> Option<ModelPrice> {- let price = match model {- "openai/openai/gpt-5.2" | "openai/openai/openai/gpt-5.2" => ModelPrice {- input: 1.75,- output: 14.00,- cached: 0.175,- cache_write: 1.75,- },- "nvidia/nvidia/nemotron-3-super-v3" | "openai/nvidia/nvidia/nemotron-3-super-v3" => {- ModelPrice {- input: 0.10,- output: 0.50,- cached: 0.01,- cache_write: 0.10,- }- }- // Moonshot Kimi K2.6 — official platform.kimi.ai pricing (May 2026).- // OpenAI wire format on NVIDIA Inference Hub; no cache_write- // premium (cache_write equals input).- "nvidia/moonshotai/kimi-k2.6" | "openai/nvidia/moonshotai/kimi-k2.6" => ModelPrice {- input: 0.95,- output: 4.00,- cached: 0.16,- cache_write: 0.95,- },- // Moonshot Kimi K2 / K2.5 — platform.kimi.ai (k2-thinking variant- // pricing, the closest standard tier; NVIDIA hub serves this as- // ``kimi-k2.5``). Same no-cache-write-premium posture.- "nvidia/moonshotai/kimi-k2.5" | "openai/nvidia/moonshotai/kimi-k2.5" => ModelPrice {- input: 0.60,- output: 2.50,- cached: 0.15,- cache_write: 0.60,- },- // DeepSeek V4 Flash — official api-docs.deepseek.com standard list- // price (post-promo). 284B total / 13B active, 1M-token context- // window. Aggressive cache discount (98% off on hits). OpenAI- // wire format on NVIDIA hub; no cache_write premium.- "nvidia/deepseek-ai/deepseek-v4-flash"- | "openai/nvidia/deepseek-ai/deepseek-v4-flash"- | "deepseek-v4-flash" => ModelPrice {- input: 0.14,- output: 0.28,- cached: 0.0028,- cache_write: 0.14,- },- // DeepSeek V4 Pro — official api-docs.deepseek.com standard list- // price (post-promo). 1.6T total / 49B active, 1M-token context- // window. Pro tier is currently under a 75% promotional discount- // through 2026-05-31 UTC ($0.435 / $0.87 effective); we price at- // the standard rate for stable cost-model comparisons that- // outlive the promo window. ``evals-`` prefix variant is the- // same model exposed on NVIDIA Inference Hub's benchmarking- // gateway (paired with the ``X-Inference-Priority: batch``- // header) — it bypasses the regular gateway's 6-min timeout- // that under high concurrency manifests as cascading 504s on- // V4-class models. Same per-token pricing.- "nvidia/deepseek-ai/deepseek-v4-pro"- | "openai/nvidia/deepseek-ai/deepseek-v4-pro"- | "nvidia/deepseek-ai/evals-deepseek-v4-pro"- | "openai/nvidia/deepseek-ai/evals-deepseek-v4-pro"- | "deepseek-v4-pro" => ModelPrice {- input: 1.74,- output: 3.48,- cached: 0.0145,- cache_write: 1.74,- },- // Gemini 3.5 Flash — Google Vertex global list price (ai.google.dev):- // input $1.50, output $9.00, cache-read $0.15 (90% off input). The- // default LLM-classifier model (gcp wire). No per-token cache-write- // premium (cache storage is billed per-hour, not per-token), so- // cache_write = input.- "gcp/google/gemini-3.5-flash"- | "openai/gcp/google/gemini-3.5-flash"- | "gemini-3.5-flash" => ModelPrice {- input: 1.50,- output: 9.00,- cached: 0.15,- cache_write: 1.50,- },- // Nemotron 3 Ultra (550B/55B MoE) — OpenRouter reference list- // price, June 2026. evals-N ids are parallel benchmarking-gateway- // deployments of the same model.- "nvidia/nvidia/nemotron-3-ultra"- | "openai/nvidia/nvidia/nemotron-3-ultra"- | "nvidia/nvidia/evals-nemotron-ultra"- | "nvidia/nvidia/evals-nemotron-ultra-2"- | "nvidia/nvidia/evals-nemotron-ultra-3"- | "nvidia/nvidia/evals-nemotron-ultra-4" => ModelPrice {- input: 0.50,- output: 2.20,- cached: 0.05,- cache_write: 0.50,- },- "aws/anthropic/bedrock-claude-opus-4-8"- | "aws/anthropic/bedrock-claude-opus-4-7"- | "aws/anthropic/bedrock-claude-opus-4-6"- | "aws/anthropic/bedrock-claude-opus-4-5"- | "azure/anthropic/claude-opus-4-8"- | "azure/anthropic/claude-opus-4-7"- | "azure/anthropic/claude-opus-4-6"- | "claude-opus-4-8"- | "claude-opus-4-7"- | "claude-opus-4-6"- | "claude-opus-4-5" => ModelPrice {- input: 5.00,- output: 25.00,- cached: 0.50,- cache_write: 6.25,- },- "aws/anthropic/bedrock-claude-sonnet-4-6"- | "aws/anthropic/bedrock-claude-sonnet-4-5"- | "azure/anthropic/claude-sonnet-4-5"- | "claude-sonnet-4-6"- | "claude-sonnet-4-5" => ModelPrice {- input: 3.00,- output: 15.00,- cached: 0.30,- cache_write: 3.75,- },- "aws/anthropic/bedrock-claude-haiku-4-5" | "claude-haiku-4-5" => ModelPrice {- input: 1.00,- output: 5.00,- cached: 0.10,- cache_write: 1.25,- },- _ => return None,- };- Some(price)-}--fn round6(value: f64) -> f64 {- (value * 1_000_000.0).round() / 1_000_000.0-}--#[cfg(test)]-mod tests {- use super::*;-- #[test]- fn gemini_3_5_flash_is_priced() {- // The default LLM-classifier model must not silently cost $0.- for model in [- "gcp/google/gemini-3.5-flash",- "openai/gcp/google/gemini-3.5-flash",- "gemini-3.5-flash",- ] {- assert!(has_model_price(model), "{model} should be priced");- let cost = estimate_model_cost(model, 1_000_000, 1_000_000, 0, 0);- assert_eq!(cost.base_input_cost, 1.50);- assert_eq!(cost.output_cost, 9.00);- assert_eq!(cost.total_cost, 10.50);- }- }-- #[test]- fn unknown_model_defaults_to_zero() {- assert!(!has_model_price("nvidia/qwen/qwen3.6-35b-a3b"));- let cost = estimate_model_cost("nvidia/qwen/qwen3.6-35b-a3b", 1_000_000, 1_000_000, 0, 0);- assert_eq!(cost.total_cost, 0.0);- }-}diff --git a/crates/switchyard-components/src/stats/mod.rs b/crates/switchyard-components/src/stats/mod.rsdeleted file mode 100644index e88efac23..000000000--- a/crates/switchyard-components/src/stats/mod.rs+++ /dev/null@@ -1,25 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Shared stats accounting used by stats processors and backend wrappers.--mod accumulator;-mod cache_eligibility;-mod context;-mod cost;-mod usage;--pub use accumulator::{- ClassifierStatsSnapshot, LatencyHistogramSnapshot, ModelStatsSnapshot, StatsAccumulator,- StatsSnapshot, TierStatsSnapshot, TokenTotals,-};-pub use cache_eligibility::{PrefixProbe, prefix_probe, tracking_enabled_from_env};-pub use context::{- StatsBackendLatency, StatsRequestStart, StatsRouteLabel, selected_stats_model,- selected_stats_tier,-};-pub use cost::{CostBreakdown, CostEstimate, estimate_model_cost, has_model_price};-pub use usage::{- AnthropicStreamUsage, TokenUsage, openai_chat_usage_from_stream_event,- openai_responses_usage_from_stream_event, usage_from_body,-};diff --git a/crates/switchyard-components/src/stats/usage.rs b/crates/switchyard-components/src/stats/usage.rsdeleted file mode 100644index 10b939a30..000000000--- a/crates/switchyard-components/src/stats/usage.rs+++ /dev/null@@ -1,221 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Provider usage extraction for buffered and streaming responses.--use crate::StreamEvent;-use serde::{Deserialize, Serialize};-use serde_json::Value;--/// Normalized token usage counters.-#[derive(Clone, Copy, Debug, Default, Eq, PartialEq, Serialize, Deserialize)]-pub struct TokenUsage {- pub prompt_tokens: u64,- pub completion_tokens: u64,- pub cached_tokens: u64,- pub cache_creation_tokens: u64,- pub reasoning_tokens: u64,- /// Set by the stats response processor, not by provider usage extraction.- #[serde(default)]- pub cacheable_prompt_tokens: u64,-}--impl TokenUsage {- /// Returns whether all counters are zero.- pub fn is_zero(self) -> bool {- self.prompt_tokens == 0- && self.completion_tokens == 0- && self.cached_tokens == 0- && self.cache_creation_tokens == 0- && self.reasoning_tokens == 0- }-}--/// Extracts usage from a buffered response body.-pub fn usage_from_body(body: &Value) -> TokenUsage {- body.get("usage")- .and_then(usage_from_candidate)- .unwrap_or_default()-}--/// Extracts OpenAI Chat streaming usage from an event.-pub fn openai_chat_usage_from_stream_event(event: &StreamEvent) -> Option<TokenUsage> {- let StreamEvent::Json(value) = event else {- return None;- };- value.get("usage").and_then(usage_from_candidate)-}--/// Extracts OpenAI Responses streaming usage from an event.-///-/// Fidelity-preserving backends yield raw SSE frame *strings*-/// (`StreamEvent::Text`) instead of decoded JSON events; those frames are-/// parsed here so usage accounting survives verbatim passthrough.-pub fn openai_responses_usage_from_stream_event(event: &StreamEvent) -> Option<TokenUsage> {- match event {- StreamEvent::Json(value) => usage_from_responses_value(value),- StreamEvent::Text(text) => sse_data_payloads(text)- .iter()- .find_map(usage_from_responses_value),- }-}--/// Reads `response.usage` from one decoded Responses stream event.-fn usage_from_responses_value(value: &Value) -> Option<TokenUsage> {- value- .get("response")- .and_then(|response| response.get("usage"))- .and_then(usage_from_candidate)-}--/// Parses the JSON `data:` payload(s) out of a raw SSE frame string.-///-/// Per the SSE contract, a frame's `data:` lines join with newlines to form-/// one payload; a single leading space after the colon is stripped. Comment-/// frames, `[DONE]` sentinels, and non-JSON payloads yield nothing.-fn sse_data_payloads(text: &str) -> Vec<Value> {- let mut payloads = Vec::new();- for block in text.split("\n\n") {- let data_lines: Vec<&str> = block- .split('\n')- .filter_map(|line| line.strip_prefix("data:"))- .map(|value| value.strip_prefix(' ').unwrap_or(value))- .collect();- if data_lines.is_empty() {- continue;- }- let data = data_lines.join("\n");- if data.trim() == "[DONE]" {- continue;- }- if let Ok(parsed) = serde_json::from_str::<Value>(&data) {- payloads.push(parsed);- }- }- payloads-}--/// Accumulates Anthropic streaming usage and commits once at `message_stop`.-#[derive(Clone, Copy, Debug, Default)]-pub struct AnthropicStreamUsage {- input_tokens: u64,- output_tokens: u64,- cache_read_input_tokens: u64,- cache_creation_input_tokens: u64,- saw_usage: bool,- committed: bool,-}--impl AnthropicStreamUsage {- /// Observes one stream event and returns usage exactly once at `message_stop`.- /// A stop event before any usage frame is a known no-op, matching Python stream taps.- pub fn observe(&mut self, event: &StreamEvent) -> Option<TokenUsage> {- let StreamEvent::Json(value) = event else {- return None;- };- match value.get("type").and_then(Value::as_str) {- Some("message_start") => {- if let Some(usage) = value- .get("message")- .and_then(|message| message.get("usage"))- {- self.merge(usage);- }- None- }- Some("message_delta") => {- if let Some(usage) = value- .get("usage")- .or_else(|| value.get("delta").and_then(|delta| delta.get("usage")))- {- self.merge(usage);- }- None- }- Some("message_stop") if self.saw_usage && !self.committed => {- self.committed = true;- Some(TokenUsage {- prompt_tokens: self- .input_tokens- .saturating_add(self.cache_read_input_tokens)- .saturating_add(self.cache_creation_input_tokens),- completion_tokens: self.output_tokens,- cached_tokens: self.cache_read_input_tokens,- cache_creation_tokens: self.cache_creation_input_tokens,- reasoning_tokens: 0,- cacheable_prompt_tokens: 0,- })- }- _ => None,- }- }-- fn merge(&mut self, usage: &Value) {- if !usage.is_object() {- return;- }- self.saw_usage = true;- if let Some(value) = int_field(usage, "input_tokens") {- self.input_tokens = value;- }- if let Some(value) = int_field(usage, "output_tokens") {- self.output_tokens = value;- }- if let Some(value) = int_field(usage, "cache_read_input_tokens") {- self.cache_read_input_tokens = value;- }- if let Some(value) = int_field(usage, "cache_creation_input_tokens") {- self.cache_creation_input_tokens = value;- }- }-}--fn usage_from_candidate(usage: &Value) -> Option<TokenUsage> {- usage.is_object().then(|| usage_from_value(usage))-}--fn usage_from_value(usage: &Value) -> TokenUsage {- let completion_tokens = int_field(usage, "completion_tokens")- .unwrap_or_else(|| int_field(usage, "output_tokens").unwrap_or(0));- let mut output = TokenUsage {- completion_tokens,- ..TokenUsage::default()- };-- if let Some(prompt_tokens) = int_field(usage, "prompt_tokens") {- output.prompt_tokens = prompt_tokens;- if let Some(details) = usage.get("prompt_tokens_details") {- output.cached_tokens = int_field(details, "cached_tokens").unwrap_or(0);- output.cache_creation_tokens = int_field(details, "cache_creation_tokens").unwrap_or(0);- }- } else {- let base = int_field(usage, "input_tokens").unwrap_or(0);- if let Some(details) = usage.get("input_tokens_details") {- output.cached_tokens = int_field(details, "cached_tokens").unwrap_or(0);- }- let cache_read = int_field(usage, "cache_read_input_tokens").unwrap_or(0);- let cache_creation = int_field(usage, "cache_creation_input_tokens").unwrap_or(0);- if output.cached_tokens == 0 {- output.cached_tokens = cache_read;- }- output.cache_creation_tokens = cache_creation;- output.prompt_tokens = base- .saturating_add(cache_read)- .saturating_add(cache_creation);- }-- output.reasoning_tokens = usage- .get("completion_tokens_details")- .and_then(|details| int_field(details, "reasoning_tokens"))- .or_else(|| {- usage- .get("output_tokens_details")- .and_then(|details| int_field(details, "reasoning_tokens"))- })- .unwrap_or(0);- output-}--fn int_field(value: &Value, name: &str) -> Option<u64> {- value.get(name).and_then(Value::as_u64)-}diff --git a/crates/switchyard-components/src/telemetry.rs b/crates/switchyard-components/src/telemetry.rsdeleted file mode 100644index 10b7dfd66..000000000--- a/crates/switchyard-components/src/telemetry.rs+++ /dev/null@@ -1,97 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Shared version and telemetry helpers for Rust-owned components.--use std::env;--pub(crate) const SWITCHYARD_VERSION_HEADER: &str = "X-Switchyard-Version";--const SWITCHYARD_VERSION_ENV: &str = "SWITCHYARD_VERSION";-const SWITCHYARD_TELEMETRY_OPT_OUT_ENV: &str = "SWITCHYARD_TELEMETRY_OPT_OUT";-const NEMO_SWITCHYARD_TELEMETRY_OPT_OUT_ENV: &str = "NEMO_SWITCHYARD_TELEMETRY_OPT_OUT";--pub(crate) fn telemetry_header_value() -> Option<String> {- telemetry_header_value_from_values(- env::var(SWITCHYARD_TELEMETRY_OPT_OUT_ENV).ok().as_deref(),- env::var(NEMO_SWITCHYARD_TELEMETRY_OPT_OUT_ENV)- .ok()- .as_deref(),- env::var(SWITCHYARD_VERSION_ENV).ok().as_deref(),- )-}--fn telemetry_header_value_from_values(- opt_out: Option<&str>,- legacy_opt_out: Option<&str>,- version: Option<&str>,-) -> Option<String> {- if env_value_opts_out(opt_out) || env_value_opts_out(legacy_opt_out) {- return None;- }- Some(configured_version(version).unwrap_or_else(|| env!("CARGO_PKG_VERSION").to_string()))-}--fn configured_version(value: Option<&str>) -> Option<String> {- value- .map(str::trim)- .filter(|value| !value.is_empty())- .map(str::to_string)-}--fn env_value_opts_out(value: Option<&str>) -> bool {- let Some(value) = value.map(str::trim) else {- return false;- };- !matches!(- value.to_ascii_lowercase().as_str(),- "" | "0" | "false" | "no"- )-}--#[cfg(test)]-mod tests {- use super::*;-- #[test]- fn telemetry_header_uses_configured_version_when_not_opted_out() {- assert_eq!(- telemetry_header_value_from_values(None, None, Some(" 1.2.3 ")),- Some("1.2.3".to_string())- );- }-- #[test]- fn telemetry_header_falls_back_to_crate_version() {- assert_eq!(- telemetry_header_value_from_values(None, None, None),- Some(env!("CARGO_PKG_VERSION").to_string())- );- assert_eq!(- telemetry_header_value_from_values(None, None, Some(" ")),- Some(env!("CARGO_PKG_VERSION").to_string())- );- }-- #[test]- fn telemetry_header_respects_current_and_legacy_opt_out_env_values() {- assert_eq!(- telemetry_header_value_from_values(Some("true"), None, Some("1.2.3")),- None- );- assert_eq!(- telemetry_header_value_from_values(None, Some("yes"), Some("1.2.3")),- None- );- }-- #[test]- fn telemetry_header_ignores_falsey_opt_out_values() {- for value in ["", "0", "false", "FALSE", "no", " No "] {- assert_eq!(- telemetry_header_value_from_values(Some(value), None, Some("1.2.3")),- Some("1.2.3".to_string())- );- }- }-}diff --git a/crates/switchyard-components/tests/adversarial_multi_llm_backend.rs b/crates/switchyard-components/tests/adversarial_multi_llm_backend.rsdeleted file mode 100644index d17772575..000000000--- a/crates/switchyard-components/tests/adversarial_multi_llm_backend.rs+++ /dev/null@@ -1,1268 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Adversarial tests for the Rust multi-target LLM backend.--use std::sync::Arc;--use async_trait::async_trait;-use parking_lot::Mutex;-use serde_json::{Value, json};-use switchyard_components::{- BackendFormat, ChatRequest, ChatRequestType, ChatResponse, LlmBackend, LlmTarget, LlmTargetId,- ModelId, ProxyContext, Result, SwitchyardError,-};-use switchyard_components::{- BackendSelection, BackendSelectionReason, LlmTargetBackend, MultiLlmBackend,-};--/// One backend call observed by the recording backend.-#[derive(Clone, Debug, PartialEq)]-struct ObservedCall {- /// Name of the backend that handled the request.- backend_name: &'static str,- /// Request wire type received by the child backend.- request_type: ChatRequestType,- /// Request model visible to the child backend.- model: Option<String>,- /// Full request body visible to the child backend.- body: Value,- /// Context-selected target visible at delegation time.- ctx_selected_target: Option<LlmTargetId>,- /// Context-selected model visible at delegation time.- ctx_selected_model: Option<ModelId>,-}--/// Mutex-backed shared vector used for test observations.-#[derive(Clone)]-struct Shared<T>(Arc<Mutex<Vec<T>>>);--impl<T> Default for Shared<T> {- fn default() -> Self {- Self(Arc::new(Mutex::new(Vec::new())))- }-}--impl<T: Clone> Shared<T> {- /// Appends one observation.- fn push(&self, value: T) -> Result<()> {- self.0.lock().push(value);- Ok(())- }-- /// Returns a cloned copy of all observations.- fn values(&self) -> Result<Vec<T>> {- Ok(self.0.lock().clone())- }-}--/// Backend fixture that records calls and optional lifecycle failures.-struct RecordingBackend {- /// Stable backend name for assertions.- name: &'static str,- /// Captured backend calls.- calls: Shared<ObservedCall>,- /// Captured startup/shutdown events.- events: Shared<String>,- /// Request types this fixture advertises.- supported_request_types: &'static [ChatRequestType],- /// Optional call error.- call_error: Option<&'static str>,- /// Optional startup error.- startup_error: Option<&'static str>,- /// Optional shutdown error.- shutdown_error: Option<&'static str>,-}--impl RecordingBackend {- /// Creates a recording backend with all request types enabled.- fn new(name: &'static str, calls: Shared<ObservedCall>, events: Shared<String>) -> Self {- Self {- name,- calls,- events,- supported_request_types: &ALL_REQUEST_TYPES,- call_error: None,- startup_error: None,- shutdown_error: None,- }- }-- /// Overrides the advertised request types.- fn with_supported_request_types(- mut self,- supported_request_types: &'static [ChatRequestType],- ) -> Self {- self.supported_request_types = supported_request_types;- self- }-- /// Configures a call-time backend error.- fn with_call_error(mut self, message: &'static str) -> Self {- self.call_error = Some(message);- self- }-- /// Configures a startup failure.- fn with_startup_error(mut self, message: &'static str) -> Self {- self.startup_error = Some(message);- self- }-- /// Configures a shutdown failure.- fn with_shutdown_error(mut self, message: &'static str) -> Self {- self.shutdown_error = Some(message);- self- }-}--static ALL_REQUEST_TYPES: [ChatRequestType; 3] = [- ChatRequestType::OpenAiChat,- ChatRequestType::OpenAiResponses,- ChatRequestType::Anthropic,-];-static OPENAI_CHAT_ONLY: [ChatRequestType; 1] = [ChatRequestType::OpenAiChat];--#[async_trait]-impl LlmBackend for RecordingBackend {- fn supported_request_types(&self) -> &[ChatRequestType] {- self.supported_request_types- }-- async fn call(&self, ctx: &mut ProxyContext, request: &ChatRequest) -> Result<ChatResponse> {- self.calls.push(ObservedCall {- backend_name: self.name,- request_type: request.request_type(),- model: request.model().map(str::to_string),- body: request.body().clone(),- ctx_selected_target: selected_target(ctx).cloned(),- ctx_selected_model: selected_model(ctx).cloned(),- })?;- if let Some(message) = self.call_error {- return Err(SwitchyardError::Backend(message.to_string()));- }- Ok(ChatResponse::openai_completion(json!({- "backend": self.name,- "model": request.model(),- })))- }-- async fn startup(&self) -> Result<()> {- self.events.push(format!("{}:startup", self.name))?;- if let Some(message) = self.startup_error {- return Err(SwitchyardError::Backend(message.to_string()));- }- Ok(())- }-- async fn shutdown(&self) -> Result<()> {- self.events.push(format!("{}:shutdown", self.name))?;- if let Some(message) = self.shutdown_error {- return Err(SwitchyardError::Backend(message.to_string()));- }- Ok(())- }-}--/// Builds an OpenAI-format test target.-fn target(id: &'static str, model: &'static str) -> LlmTarget {- let mut target = LlmTarget::new(LlmTargetId::from_static(id), ModelId::from_static(model));- target.format = BackendFormat::OpenAi;- target-}--/// Builds a target/backend pair for `MultiLlmBackend`.-fn target_backend(- id: &'static str,- model: &'static str,- backend: RecordingBackend,-) -> LlmTargetBackend {- LlmTargetBackend::new(target(id, model), Arc::new(backend))-}--/// Builds an OpenAI Chat request fixture.-fn request(model: &str) -> ChatRequest {- ChatRequest::openai_chat(json!({- "model": model,- "messages": [{"role": "user", "content": "preserve me"}],- "temperature": 0.2- }))-}--/// Builds an Anthropic request fixture.-fn anthropic_request(model: &str) -> ChatRequest {- ChatRequest::anthropic(json!({- "model": model,- "max_tokens": 128,- "messages": [{"role": "user", "content": "preserve anthropic"}],- "metadata": {"kept": true}- }))-}--/// Builds a Responses API request fixture.-fn responses_request(model: &str) -> ChatRequest {- ChatRequest::openai_responses(json!({- "model": model,- "input": "preserve responses",- "metadata": {"kept": true}- }))-}--/// Returns the backend selection stamped by `MultiLlmBackend`.-fn selection(ctx: &ProxyContext) -> Result<&BackendSelection> {- ctx.get::<BackendSelection>()- .ok_or_else(|| SwitchyardError::Other("multi-LLM selection should be recorded".to_string()))-}--/// Returns the selected model stamped in context.-fn selected_model(ctx: &ProxyContext) -> Option<&ModelId> {- ctx.get::<BackendSelection>()- .map(|selection| &selection.model)-}--/// Returns the selected target stamped in context.-fn selected_target(ctx: &ProxyContext) -> Option<&LlmTargetId> {- ctx.get::<BackendSelection>()- .and_then(|selection| selection.target_id.as_ref())-}--// Default support should cover all inbound wire formats in stable order.-#[test]-fn default_supported_request_types_are_all_wire_formats_in_stable_order() -> Result<()> {- let calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([target_backend(- "only-target",- "served-model",- RecordingBackend::new("only", calls, events),- )])?;-- assert_eq!(- backend.supported_request_types(),- &[- ChatRequestType::OpenAiChat,- ChatRequestType::OpenAiResponses,- ChatRequestType::Anthropic,- ]- );- Ok(())-}--// Custom request type support should de-dupe without reordering caller input.-#[test]-fn custom_supported_request_types_are_deduped_without_reordering() -> Result<()> {- let calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([target_backend(- "only-target",- "served-model",- RecordingBackend::new("only", calls, events),- )])?- .with_supported_request_types([- ChatRequestType::Anthropic,- ChatRequestType::OpenAiChat,- ChatRequestType::Anthropic,- ChatRequestType::OpenAiChat,- ChatRequestType::OpenAiResponses,- ])?;-- assert_eq!(- backend.supported_request_types(),- &[- ChatRequestType::Anthropic,- ChatRequestType::OpenAiChat,- ChatRequestType::OpenAiResponses,- ]- );- Ok(())-}--// Context-selected targets should win and cloned requests should keep caller state intact.-#[tokio::test]-async fn context_selected_target_wins_and_request_is_cloned() -> Result<()> {- let strong_calls = Shared::default();- let weak_calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([- target_backend(- "strong-target",- "strong-model",- RecordingBackend::new("strong", strong_calls.clone(), events.clone()),- ),- target_backend(- "weak-target",- "weak-model",- RecordingBackend::new("weak", weak_calls.clone(), events.clone()),- ),- ])?;- let original = request("client-model");- let mut ctx = ProxyContext::new();- ctx.set_selected_target(LlmTargetId::from_static("weak-target"));-- let response = backend.call(&mut ctx, &original).await?;-- assert_eq!(- response.body(),- Some(&json!({"backend": "weak", "model": "weak-model"}))- );- assert_eq!(original.model(), Some("client-model"));- assert_eq!(- selected_target(&ctx),- Some(&LlmTargetId::from_static("weak-target"))- );- assert_eq!(- selected_model(&ctx),- Some(&ModelId::from_static("weak-model"))- );- assert_eq!(- selection(&ctx)?.reason,- BackendSelectionReason::ContextTarget- );- assert_eq!(- selection(&ctx)?.original_model.as_deref(),- Some("client-model")- );- assert!(strong_calls.values()?.is_empty());-- let weak = weak_calls.values()?;- assert_eq!(weak.len(), 1);- assert_eq!(weak[0].backend_name, "weak");- assert_eq!(weak[0].request_type, ChatRequestType::OpenAiChat);- assert_eq!(weak[0].model.as_deref(), Some("weak-model"));- assert_eq!(weak[0].body["messages"][0]["content"], "preserve me");- assert_eq!(- weak[0].ctx_selected_target,- Some(LlmTargetId::from_static("weak-target"))- );- assert_eq!(- weak[0].ctx_selected_model,- Some(ModelId::from_static("weak-model"))- );- Ok(())-}--// Explicit context target selection should override request model matches.-#[tokio::test]-async fn context_selected_target_wins_even_when_request_model_matches_another_target() -> Result<()>-{- let strong_calls = Shared::default();- let weak_calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([- target_backend(- "strong-target",- "strong-model",- RecordingBackend::new("strong", strong_calls.clone(), events.clone()),- ),- target_backend(- "weak-target",- "weak-model",- RecordingBackend::new("weak", weak_calls.clone(), events),- ),- ])?;- let mut ctx = ProxyContext::new();- ctx.set_selected_target(LlmTargetId::from_static("weak-target"));-- backend.call(&mut ctx, &request("strong-model")).await?;-- assert!(strong_calls.values()?.is_empty());- let weak = weak_calls.values()?;- assert_eq!(weak.len(), 1);- assert_eq!(weak[0].model.as_deref(), Some("weak-model"));- assert_eq!(- selection(&ctx)?.reason,- BackendSelectionReason::ContextTarget- );- assert_eq!(- selection(&ctx)?.original_model.as_deref(),- Some("strong-model")- );- Ok(())-}--// A single configured target should route without any selector processor.-#[tokio::test]-async fn single_target_fallback_routes_without_selector() -> Result<()> {- let calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([target_backend(- "only-target",- "served-model",- RecordingBackend::new("only", calls.clone(), events),- )])?;- let mut ctx = ProxyContext::new();-- backend.call(&mut ctx, &request("client-model")).await?;-- assert_eq!(- selected_target(&ctx),- Some(&LlmTargetId::from_static("only-target"))- );- assert_eq!(- selected_model(&ctx),- Some(&ModelId::from_static("served-model"))- );- assert_eq!(- selection(&ctx)?.reason,- BackendSelectionReason::SingleTarget- );- let calls = calls.values()?;- assert_eq!(calls.len(), 1);- assert_eq!(calls[0].model.as_deref(), Some("served-model"));- Ok(())-}--// Non-object request bodies should be repaired only in the delegated clone.-#[tokio::test]-async fn single_target_fallback_recovers_non_object_request_body_without_mutating_original()--> Result<()> {- let calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([target_backend(- "only-target",- "served-model",- RecordingBackend::new("only", calls.clone(), events),- )])?;- let original = ChatRequest::openai_chat(json!("not an object"));- let mut ctx = ProxyContext::new();-- backend.call(&mut ctx, &original).await?;-- assert_eq!(original.body(), &json!("not an object"));- let calls = calls.values()?;- assert_eq!(calls.len(), 1);- assert_eq!(calls[0].body, json!({"model": "served-model"}));- assert_eq!(selection(&ctx)?.original_model, None);- Ok(())-}--// Anthropic request bodies should keep their wire shape while the model is rewritten.-#[tokio::test]-async fn routing_preserves_anthropic_payload_shape_while_rewriting_model() -> Result<()> {- let calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([target_backend(- "anthropic-target",- "served-claude",- RecordingBackend::new("anthropic", calls.clone(), events),- )])?;- let original = anthropic_request("client-claude");- let mut ctx = ProxyContext::new();-- backend.call(&mut ctx, &original).await?;-- assert_eq!(original.model(), Some("client-claude"));- let calls = calls.values()?;- assert_eq!(calls.len(), 1);- assert_eq!(calls[0].request_type, ChatRequestType::Anthropic);- assert_eq!(calls[0].model.as_deref(), Some("served-claude"));- assert_eq!(calls[0].body["max_tokens"], 128);- assert_eq!(- calls[0].body["messages"][0]["content"],- "preserve anthropic"- );- assert_eq!(calls[0].body["metadata"], json!({"kept": true}));- Ok(())-}--// Responses request bodies should keep their wire shape while the model is rewritten.-#[tokio::test]-async fn routing_preserves_responses_payload_shape_while_rewriting_model() -> Result<()> {- let calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([target_backend(- "responses-target",- "served-responses",- RecordingBackend::new("responses", calls.clone(), events),- )])?;- let original = responses_request("client-responses");- let mut ctx = ProxyContext::new();-- backend.call(&mut ctx, &original).await?;-- assert_eq!(original.model(), Some("client-responses"));- let calls = calls.values()?;- assert_eq!(calls.len(), 1);- assert_eq!(calls[0].request_type, ChatRequestType::OpenAiResponses);- assert_eq!(calls[0].model.as_deref(), Some("served-responses"));- assert_eq!(calls[0].body["input"], "preserve responses");- assert_eq!(calls[0].body["metadata"], json!({"kept": true}));- Ok(())-}--// Multi-LLM dispatch should not reject a selected child based on its native format list.-#[tokio::test]-async fn selected_child_backend_receives_request_even_when_its_direct_formats_do_not_match()--> Result<()> {- let calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([target_backend(- "translated-target",- "served-model",- RecordingBackend::new("translated", calls.clone(), events)- .with_supported_request_types(&OPENAI_CHAT_ONLY),- )])?;- let mut ctx = ProxyContext::new();-- backend- .call(&mut ctx, &anthropic_request("client-claude"))- .await?;-- let calls = calls.values()?;- assert_eq!(calls.len(), 1);- assert_eq!(calls[0].request_type, ChatRequestType::Anthropic);- assert_eq!(calls[0].model.as_deref(), Some("served-model"));- assert_eq!(- calls[0].ctx_selected_target,- Some(LlmTargetId::from_static("translated-target"))- );- Ok(())-}--// Request model should select a unique matching target when context is empty.-#[tokio::test]-async fn request_model_selects_unique_target_when_context_is_empty() -> Result<()> {- let strong_calls = Shared::default();- let weak_calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([- target_backend(- "strong-target",- "strong-model",- RecordingBackend::new("strong", strong_calls.clone(), events.clone()),- ),- target_backend(- "weak-target",- "weak-model",- RecordingBackend::new("weak", weak_calls.clone(), events),- ),- ])?;- let mut ctx = ProxyContext::new();-- backend.call(&mut ctx, &request("strong-model")).await?;-- assert_eq!(- selected_target(&ctx),- Some(&LlmTargetId::from_static("strong-target"))- );- assert_eq!(- selection(&ctx)?.reason,- BackendSelectionReason::RequestModel- );- assert_eq!(strong_calls.values()?.len(), 1);- assert!(weak_calls.values()?.is_empty());- Ok(())-}--// Configured default targets should handle model-less requests.-#[tokio::test]-async fn configured_default_target_routes_when_no_selector_ran() -> Result<()> {- let strong_calls = Shared::default();- let weak_calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([- target_backend(- "strong-target",- "strong-model",- RecordingBackend::new("strong", strong_calls.clone(), events.clone()),- ),- target_backend(- "weak-target",- "weak-model",- RecordingBackend::new("weak", weak_calls.clone(), events),- ),- ])?- .with_default_target(LlmTargetId::from_static("strong-target"))?;- let mut ctx = ProxyContext::new();- let request = ChatRequest::openai_chat(json!({- "messages": [{"role": "user", "content": "no model from client"}]- }));-- backend.call(&mut ctx, &request).await?;-- assert_eq!(- selected_target(&ctx),- Some(&LlmTargetId::from_static("strong-target"))- );- assert_eq!(- selected_model(&ctx),- Some(&ModelId::from_static("strong-model"))- );- assert_eq!(- selection(&ctx)?.reason,- BackendSelectionReason::DefaultTarget- );- assert_eq!(strong_calls.values()?.len(), 1);- assert!(weak_calls.values()?.is_empty());- Ok(())-}--// Configured default targets intentionally override request-model selection.-#[tokio::test]-async fn configured_default_target_overrides_request_model_selection() -> Result<()> {- let strong_calls = Shared::default();- let weak_calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([- target_backend(- "strong-target",- "strong-model",- RecordingBackend::new("strong", strong_calls.clone(), events.clone()),- ),- target_backend(- "weak-target",- "weak-model",- RecordingBackend::new("weak", weak_calls.clone(), events),- ),- ])?- .with_default_target(LlmTargetId::from_static("strong-target"))?;- let mut ctx = ProxyContext::new();-- backend.call(&mut ctx, &request("weak-model")).await?;-- assert_eq!(- selected_target(&ctx),- Some(&LlmTargetId::from_static("strong-target"))- );- assert_eq!(- selection(&ctx)?.reason,- BackendSelectionReason::DefaultTarget- );- assert_eq!(- selection(&ctx)?.original_model.as_deref(),- Some("weak-model")- );- assert_eq!(strong_calls.values()?.len(), 1);- assert!(weak_calls.values()?.is_empty());- Ok(())-}--// Context-selected targets still have the highest routing priority.-#[tokio::test]-async fn explicit_context_target_still_wins_over_configured_default_target() -> Result<()> {- let strong_calls = Shared::default();- let weak_calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([- target_backend(- "strong-target",- "strong-model",- RecordingBackend::new("strong", strong_calls.clone(), events.clone()),- ),- target_backend(- "weak-target",- "weak-model",- RecordingBackend::new("weak", weak_calls.clone(), events),- ),- ])?- .with_default_target(LlmTargetId::from_static("strong-target"))?;- let mut ctx = ProxyContext::new();- ctx.set_selected_target(LlmTargetId::from_static("weak-target"));-- backend.call(&mut ctx, &request("client-model")).await?;-- assert_eq!(- selection(&ctx)?.reason,- BackendSelectionReason::ContextTarget- );- assert!(strong_calls.values()?.is_empty());- assert_eq!(weak_calls.values()?.len(), 1);- Ok(())-}--// Ambiguous model-less requests should fail before any backend sees them.-#[tokio::test]-async fn model_less_request_with_multiple_targets_fails_without_mutating_context() -> Result<()> {- let calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([- target_backend(- "left-target",- "left-model",- RecordingBackend::new("left", calls.clone(), events.clone()),- ),- target_backend(- "right-target",- "right-model",- RecordingBackend::new("right", calls.clone(), events),- ),- ])?;- let mut ctx = ProxyContext::new();- let request = ChatRequest::openai_chat(json!({- "messages": [{"role": "user", "content": "no model"}]- }));-- let Err(error) = backend.call(&mut ctx, &request).await else {- return Err(SwitchyardError::Other(- "model-less multi-target request should fail".to_string(),- ));- };-- assert!(matches!(error, SwitchyardError::InvalidConfig(_)));- assert_eq!(selected_target(&ctx), None);- assert_eq!(selected_model(&ctx), None);- assert!(ctx.get::<BackendSelection>().is_none());- assert!(calls.values()?.is_empty());- Ok(())-}--// Duplicate model names are legal when an explicit target disambiguates them.-#[tokio::test]-async fn duplicate_models_route_successfully_when_target_is_explicit() -> Result<()> {- let left_calls = Shared::default();- let right_calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([- target_backend(- "left-target",- "shared-model",- RecordingBackend::new("left", left_calls.clone(), events.clone()),- ),- target_backend(- "right-target",- "shared-model",- RecordingBackend::new("right", right_calls.clone(), events),- ),- ])?;- let mut ctx = ProxyContext::new();- ctx.set_selected_target(LlmTargetId::from_static("right-target"));-- backend.call(&mut ctx, &request("shared-model")).await?;-- assert!(left_calls.values()?.is_empty());- let right = right_calls.values()?;- assert_eq!(right.len(), 1);- assert_eq!(right[0].backend_name, "right");- assert_eq!(- selection(&ctx)?.reason,- BackendSelectionReason::ContextTarget- );- assert_eq!(- selection(&ctx)?.target_id,- Some(LlmTargetId::from_static("right-target"))- );- Ok(())-}--// Successful explicit routing should replace stale context selection before delegation.-#[tokio::test]-async fn explicit_target_overwrites_stale_selected_model_and_selection_before_delegation()--> Result<()> {- let calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([target_backend(- "fresh-target",- "fresh-model",- RecordingBackend::new("fresh", calls.clone(), events),- )])?;- let mut ctx = ProxyContext::new();- ctx.set_selected_target(LlmTargetId::from_static("fresh-target"));- let _ = ctx.insert(BackendSelection {- target_id: Some(LlmTargetId::from_static("stale-target")),- model: ModelId::from_static("stale-model"),- original_model: Some("stale-client".to_string()),- reason: BackendSelectionReason::RequestModel,- });-- backend.call(&mut ctx, &request("client-model")).await?;-- assert_eq!(- selected_model(&ctx),- Some(&ModelId::from_static("fresh-model"))- );- let calls = calls.values()?;- assert_eq!(calls.len(), 1);- assert_eq!(- calls[0].ctx_selected_model,- Some(ModelId::from_static("fresh-model"))- );- let selection = selection(&ctx)?;- assert_eq!(- selection.target_id,- Some(LlmTargetId::from_static("fresh-target"))- );- assert_eq!(selection.model, ModelId::from_static("fresh-model"));- assert_eq!(selection.original_model.as_deref(), Some("client-model"));- assert_eq!(selection.reason, BackendSelectionReason::ContextTarget);- Ok(())-}--// Unknown selected targets should fail before delegation.-#[tokio::test]-async fn unknown_selected_target_fails_before_any_backend_call() -> Result<()> {- let calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([target_backend(- "known-target",- "known-model",- RecordingBackend::new("known", calls.clone(), events),- )])?;- let mut ctx = ProxyContext::new();- ctx.set_selected_target(LlmTargetId::from_static("missing-target"));-- let Err(error) = backend.call(&mut ctx, &request("known-model")).await else {- return Err(SwitchyardError::Other(- "unknown selected target should fail".to_string(),- ));- };-- assert!(matches!(error, SwitchyardError::InvalidConfig(_)));- assert!(calls.values()?.is_empty());- Ok(())-}--// Failed routing should leave any existing context selection untouched.-#[tokio::test]-async fn failed_target_selection_does_not_replace_existing_context_selection() -> Result<()> {- let calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([target_backend(- "known-target",- "known-model",- RecordingBackend::new("known", calls.clone(), events),- )])?;- let mut ctx = ProxyContext::new();- ctx.set_selected_target(LlmTargetId::from_static("missing-target"));- let stale = BackendSelection {- target_id: Some(LlmTargetId::from_static("stale-target")),- model: ModelId::from_static("stale-model"),- original_model: Some("stale-client-model".to_string()),- reason: BackendSelectionReason::RequestModel,- };- let _ = ctx.insert(stale.clone());-- let Err(error) = backend.call(&mut ctx, &request("known-model")).await else {- return Err(SwitchyardError::Other(- "unknown selected target should fail".to_string(),- ));- };-- assert!(matches!(error, SwitchyardError::InvalidConfig(_)));- assert_eq!(ctx.get::<BackendSelection>(), Some(&stale));- assert!(calls.values()?.is_empty());- Ok(())-}--// Multiple possible targets without a unique selector should be rejected.-#[tokio::test]-async fn multiple_targets_without_a_unique_selection_are_rejected() -> Result<()> {- let calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([- target_backend(- "left-target",- "left-model",- RecordingBackend::new("left", calls.clone(), events.clone()),- ),- target_backend(- "right-target",- "right-model",- RecordingBackend::new("right", calls.clone(), events),- ),- ])?;- let mut ctx = ProxyContext::new();-- let Err(error) = backend.call(&mut ctx, &request("client-model")).await else {- return Err(SwitchyardError::Other(- "ambiguous request should fail".to_string(),- ));- };-- assert!(matches!(error, SwitchyardError::InvalidConfig(_)));- assert!(calls.values()?.is_empty());- Ok(())-}--// Duplicate model names require target-level disambiguation.-#[tokio::test]-async fn duplicate_models_require_explicit_target_selection() -> Result<()> {- let calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([- target_backend(- "left-target",- "shared-model",- RecordingBackend::new("left", calls.clone(), events.clone()),- ),- target_backend(- "right-target",- "shared-model",- RecordingBackend::new("right", calls.clone(), events),- ),- ])?;- let mut ctx = ProxyContext::new();-- let Err(error) = backend.call(&mut ctx, &request("shared-model")).await else {- return Err(SwitchyardError::Other(- "duplicate model selection should fail".to_string(),- ));- };-- assert!(matches!(error, SwitchyardError::InvalidConfig(_)));- assert!(calls.values()?.is_empty());- Ok(())-}--// Successful calls should replace stale typed backend-selection metadata.-#[tokio::test]-async fn successful_call_replaces_stale_typed_selection_extension() -> Result<()> {- let calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([target_backend(- "only-target",- "fresh-model",- RecordingBackend::new("only", calls, events),- )])?;- let mut ctx = ProxyContext::new();- let _ = ctx.insert(BackendSelection {- target_id: Some(LlmTargetId::from_static("stale-target")),- model: ModelId::from_static("stale-model"),- original_model: None,- reason: BackendSelectionReason::RequestModel,- });-- backend.call(&mut ctx, &request("client-model")).await?;-- let selection = selection(&ctx)?;- assert_eq!(- selection.target_id,- Some(LlmTargetId::from_static("only-target"))- );- assert_eq!(selection.model, ModelId::from_static("fresh-model"));- assert_eq!(selection.original_model.as_deref(), Some("client-model"));- assert_eq!(selection.reason, BackendSelectionReason::SingleTarget);- Ok(())-}--// Unsupported request types should fail before child backend delegation.-#[tokio::test]-async fn unsupported_request_type_is_rejected_before_delegation() -> Result<()> {- let calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([target_backend(- "only-target",- "served-model",- RecordingBackend::new("only", calls.clone(), events),- )])?- .with_supported_request_types([ChatRequestType::OpenAiChat])?;- let mut ctx = ProxyContext::new();- let request = ChatRequest::anthropic(json!({- "model": "served-model",- "max_tokens": 128,- "messages": [{"role": "user", "content": "hello"}]- }));-- let Err(error) = backend.call(&mut ctx, &request).await else {- return Err(SwitchyardError::Other(- "unsupported request type should fail".to_string(),- ));- };-- assert!(matches!(- error,- SwitchyardError::UnsupportedRequestType { .. }- ));- assert!(calls.values()?.is_empty());- Ok(())-}--// Child backend errors should propagate without trying other targets.-#[tokio::test]-async fn selected_backend_error_propagates_and_does_not_try_fallback_target() -> Result<()> {- let first_calls = Shared::default();- let second_calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([- target_backend(- "first-target",- "first-model",- RecordingBackend::new("first", first_calls.clone(), events.clone())- .with_call_error("upstream exploded"),- ),- target_backend(- "second-target",- "second-model",- RecordingBackend::new("second", second_calls.clone(), events),- ),- ])?;- let mut ctx = ProxyContext::new();- ctx.set_selected_target(LlmTargetId::from_static("first-target"));-- let Err(error) = backend.call(&mut ctx, &request("client-model")).await else {- return Err(SwitchyardError::Other(- "selected backend error should propagate".to_string(),- ));- };-- assert!(matches!(error, SwitchyardError::Backend(message) if message == "upstream exploded"));- let first_calls = first_calls.values()?;- assert_eq!(first_calls.len(), 1);- assert_eq!(first_calls[0].model.as_deref(), Some("first-model"));- assert!(second_calls.values()?.is_empty());- assert_eq!(- selected_target(&ctx),- Some(&LlmTargetId::from_static("first-target"))- );- assert_eq!(- selected_model(&ctx),- Some(&ModelId::from_static("first-model"))- );- assert_eq!(- selection(&ctx)?.reason,- BackendSelectionReason::ContextTarget- );- Ok(())-}--// Startup rollback should keep shutting down started backends even if rollback fails.-#[tokio::test]-async fn startup_failure_rolls_back_all_started_backends_even_when_shutdowns_fail() -> Result<()> {- let calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([- target_backend(- "first-target",- "first-model",- RecordingBackend::new("first", calls.clone(), events.clone())- .with_shutdown_error("first rollback failed"),- ),- target_backend(- "second-target",- "second-model",- RecordingBackend::new("second", calls.clone(), events.clone())- .with_shutdown_error("second rollback failed"),- ),- target_backend(- "third-target",- "third-model",- RecordingBackend::new("third", calls, events.clone())- .with_startup_error("third startup failed"),- ),- ])?;-- let Err(error) = backend.startup().await else {- return Err(SwitchyardError::Other("startup should fail".to_string()));- };-- assert!(- matches!(error, SwitchyardError::Backend(message) if message == "third startup failed")- );- assert_eq!(- events.values()?,- vec![- "first:startup".to_string(),- "second:startup".to_string(),- "third:startup".to_string(),- "second:shutdown".to_string(),- "first:shutdown".to_string(),- ]- );- Ok(())-}--// Normal lifecycle should start forward and shut down in reverse order.-#[tokio::test]-async fn successful_startup_and_shutdown_use_forward_then_reverse_order() -> Result<()> {- let calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([- target_backend(- "first-target",- "first-model",- RecordingBackend::new("first", calls.clone(), events.clone()),- ),- target_backend(- "second-target",- "second-model",- RecordingBackend::new("second", calls, events.clone()),- ),- ])?;-- backend.startup().await?;- backend.shutdown().await?;-- assert_eq!(- events.values()?,- vec![- "first:startup".to_string(),- "second:startup".to_string(),- "second:shutdown".to_string(),- "first:shutdown".to_string(),- ]- );- Ok(())-}--// Startup failure should roll back only the backends that actually started.-#[tokio::test]-async fn startup_rolls_back_already_started_backends() -> Result<()> {- let calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([- target_backend(- "first-target",- "first-model",- RecordingBackend::new("first", calls.clone(), events.clone()),- ),- target_backend(- "second-target",- "second-model",- RecordingBackend::new("second", calls.clone(), events.clone())- .with_startup_error("startup failed"),- ),- target_backend(- "third-target",- "third-model",- RecordingBackend::new("third", calls, events.clone()),- ),- ])?;-- let Err(error) = backend.startup().await else {- return Err(SwitchyardError::Other("startup should fail".to_string()));- };-- assert!(matches!(error, SwitchyardError::Backend(_)));- assert_eq!(- events.values()?,- vec![- "first:startup".to_string(),- "second:startup".to_string(),- "first:shutdown".to_string(),- ]- );- Ok(())-}--// Shutdown should run every backend in reverse and return the first observed failure.-#[tokio::test]-async fn shutdown_runs_all_backends_in_reverse_and_returns_first_error() -> Result<()> {- let calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([- target_backend(- "first-target",- "first-model",- RecordingBackend::new("first", calls.clone(), events.clone())- .with_shutdown_error("first shutdown failed"),- ),- target_backend(- "second-target",- "second-model",- RecordingBackend::new("second", calls, events.clone())- .with_shutdown_error("second shutdown failed"),- ),- ])?;-- let Err(error) = backend.shutdown().await else {- return Err(SwitchyardError::Other("shutdown should fail".to_string()));- };-- assert!(- matches!(error, SwitchyardError::Backend(message) if message == "second shutdown failed")- );- assert_eq!(- events.values()?,- vec!["second:shutdown".to_string(), "first:shutdown".to_string()]- );- Ok(())-}--// Public accessors should expose targets without coupling tests to storage internals.-#[test]-fn public_accessors_return_targets_and_backends_without_leaking_storage_shape() -> Result<()> {- let calls = Shared::default();- let events = Shared::default();- let backend = MultiLlmBackend::new([- target_backend(- "first-target",- "first-model",- RecordingBackend::new("first", calls.clone(), events.clone()),- ),- target_backend(- "second-target",- "second-model",- RecordingBackend::new("second", calls, events),- ),- ])?;-- assert_eq!(backend.targets().len(), 2);- assert_eq!(- backend.targets()[0].target().id,- LlmTargetId::from_static("first-target")- );- assert_eq!(- backend- .target(&LlmTargetId::from_static("second-target"))- .ok_or_else(|| SwitchyardError::Other("second target should exist".to_string()))?- .target()- .model,- ModelId::from_static("second-model")- );- assert!(- backend- .target(&LlmTargetId::from_static("missing-target"))- .is_none()- );- assert_eq!(- backend.targets()[0].backend().supported_request_types(),- &ALL_REQUEST_TYPES- );- Ok(())-}--// Constructor validation should reject empty target lists and duplicate IDs.-#[test]-fn configuration_rejects_empty_targets_duplicate_ids_and_empty_supported_types() -> Result<()> {- let calls = Shared::default();- let events = Shared::default();-- let Err(empty_targets) = MultiLlmBackend::new([]) else {- return Err(SwitchyardError::Other(- "empty target list should fail".to_string(),- ));- };- assert!(matches!(empty_targets, SwitchyardError::InvalidConfig(_)));-- let Err(duplicate_ids) = MultiLlmBackend::new([- target_backend(- "same-target",- "left-model",- RecordingBackend::new("left", calls.clone(), events.clone()),- ),- target_backend(- "same-target",- "right-model",- RecordingBackend::new("right", calls.clone(), events.clone()),- ),- ]) else {- return Err(SwitchyardError::Other(- "duplicate target IDs should fail".to_string(),- ));- };- assert!(matches!(duplicate_ids, SwitchyardError::InvalidConfig(_)));-- let backend = MultiLlmBackend::new([target_backend(- "only-target",- "only-model",- RecordingBackend::new("only", calls, events),- )])?;- let Err(invalid_default) = backend- .clone()- .with_default_target(LlmTargetId::from_static("missing-target"))- else {- return Err(SwitchyardError::Other(- "unknown default target should fail".to_string(),- ));- };- assert!(matches!(invalid_default, SwitchyardError::InvalidConfig(_)));- let Err(empty_supported_types) = backend.with_supported_request_types([]) else {- return Err(SwitchyardError::Other(- "empty supported request types should fail".to_string(),- ));- };- assert!(matches!(- empty_supported_types,- SwitchyardError::InvalidConfig(_)- ));- Ok(())-}diff --git a/crates/switchyard-components/tests/adversarial_native_backends.rs b/crates/switchyard-components/tests/adversarial_native_backends.rsdeleted file mode 100644index 3cc581389..000000000--- a/crates/switchyard-components/tests/adversarial_native_backends.rs+++ /dev/null@@ -1,1160 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Adversarial integration tests for native LLM backends.--mod support;--use futures_util::StreamExt;-use serde_json::{Value, json};-use switchyard_components::{- AnthropicNativeBackend, BackendSelection, OpenAiNativeBackend, OpenAiPassthroughBackend,-};-use switchyard_components::{- BackendFormat, ChatRequest, ChatRequestType, ChatResponse, ChatResponseType, EndpointConfig,- LlmBackend, LlmTarget, LlmTargetId, ModelId, ProxyContext, Result, StreamEvent,- SwitchyardError,-};--use support::{CapturedRequest, OneShotServer};--// Reads the selected model stamped by the backend into context.-fn selected_model(ctx: &ProxyContext) -> Option<&ModelId> {- ctx.get::<BackendSelection>()- .map(|selection| &selection.model)-}--// Reads the selected target stamped by native backends into context.-fn selected_target(ctx: &ProxyContext) -> Option<&LlmTargetId> {- ctx.get::<BackendSelection>()- .and_then(|selection| selection.target_id.as_ref())-}--// Mirrors runtime telemetry header resolution for assertions.-fn expected_switchyard_version_header() -> Option<String> {- if env_value_opts_out(- std::env::var("SWITCHYARD_TELEMETRY_OPT_OUT")- .ok()- .as_deref(),- ) || env_value_opts_out(- std::env::var("NEMO_SWITCHYARD_TELEMETRY_OPT_OUT")- .ok()- .as_deref(),- ) {- return None;- }- Some(- std::env::var("SWITCHYARD_VERSION")- .ok()- .map(|value| value.trim().to_string())- .filter(|value| !value.is_empty())- .unwrap_or_else(|| env!("CARGO_PKG_VERSION").to_string()),- )-}--// Parses telemetry opt-out environment values like the production helper.-fn env_value_opts_out(value: Option<&str>) -> bool {- let Some(value) = value.map(str::trim) else {- return false;- };- !matches!(- value.to_ascii_lowercase().as_str(),- "" | "0" | "false" | "no"- )-}--// Builds a target with explicit endpoint credentials for native backend tests.-fn target(- format: BackendFormat,- base_url: String,- api_key: &str,- model: &str,-) -> Result<LlmTarget> {- let mut target = LlmTarget::new(- LlmTargetId::from_static("primary"),- ModelId::new(model).map_err(|error| SwitchyardError::Other(error.to_string()))?,- );- target.format = format;- target.endpoint = EndpointConfig {- base_url: Some(base_url),- api_key: Some(api_key.to_string()),- timeout_secs: None,- };- Ok(target)-}--// Builds an OpenAI target fixture.-fn openai_target(base_url: String) -> Result<LlmTarget> {- target(- BackendFormat::OpenAi,- base_url,- "openai-secret",- "target-gpt",- )-}--// Builds an OpenAI Responses target fixture.-fn responses_target(base_url: String) -> Result<LlmTarget> {- target(- BackendFormat::Responses,- base_url,- "openai-secret",- "target-responses",- )-}--// Builds an Anthropic target fixture.-fn anthropic_target(base_url: String) -> Result<LlmTarget> {- target(- BackendFormat::Anthropic,- base_url,- "anthropic-secret",- "target-claude",- )-}--// Builds a passthrough OpenAI endpoint fixture.-fn openai_endpoint(base_url: String) -> EndpointConfig {- EndpointConfig {- base_url: Some(base_url),- api_key: Some("openai-secret".to_string()),- timeout_secs: None,- }-}--// Calls the OpenAI backend against a one-shot mock server.-async fn openai_call(body: Value) -> Result<(ChatResponse, CapturedRequest, ProxyContext)> {- let server = OneShotServer::json(- 200,- json!({- "id": "chatcmpl-test",- "object": "chat.completion",- "choices": []- }),- )?;- let backend = OpenAiNativeBackend::new(openai_target(format!("{}/v1", server.base_url()))?)?;- let mut ctx = ProxyContext::new();- let response = backend- .call(&mut ctx, &ChatRequest::openai_chat(body))- .await?;- Ok((response, server.captured()?, ctx))-}--// Drains JSON stream events and rejects unexpected text frames.-async fn collect_json_events(response: ChatResponse) -> Result<Vec<Value>> {- let mut stream = match response {- ChatResponse::OpenAiStream(stream)- | ChatResponse::OpenAiResponsesStream(stream)- | ChatResponse::AnthropicStream(stream) => stream,- other => {- return Err(SwitchyardError::Other(format!(- "expected streaming response, got {:?}",- other.response_type()- )));- }- };- let mut events = Vec::new();- while let Some(event) = stream.next().await {- match event? {- StreamEvent::Json(value) => events.push(value),- StreamEvent::Text(text) => {- return Err(SwitchyardError::Other(format!(- "unexpected text stream event: {text}"- )));- }- }- }- Ok(events)-}--// OpenAI native backends should only advertise OpenAI Chat input.-#[test]-fn openai_backend_is_openai_chat_only() -> Result<()> {- let backend = OpenAiNativeBackend::new(openai_target("http://127.0.0.1:1/v1".to_string())?)?;-- assert_eq!(- backend.supported_request_types(),- &[ChatRequestType::OpenAiChat]- );- Ok(())-}--// Responses-format OpenAI targets should advertise OpenAI Responses input.-#[test]-fn openai_backend_can_be_responses_only() -> Result<()> {- let backend = OpenAiNativeBackend::new(responses_target("http://127.0.0.1:1/v1".to_string())?)?;-- assert_eq!(- backend.supported_request_types(),- &[ChatRequestType::OpenAiResponses]- );- Ok(())-}--// OpenAI passthrough keeps the same OpenAI Chat-only role contract.-#[test]-fn openai_passthrough_backend_is_openai_chat_only() -> Result<()> {- let backend =- OpenAiPassthroughBackend::new(openai_endpoint("http://127.0.0.1:1/v1".to_string()))?;-- assert_eq!(- backend.supported_request_types(),- &[ChatRequestType::OpenAiChat]- );- Ok(())-}--// Non-streaming OpenAI calls should preserve body fields and stamp context.-#[tokio::test]-async fn openai_non_streaming_posts_configured_body_and_records_context() -> Result<()> {- let (response, request, ctx) = openai_call(json!({- "model": "client-gpt",- "messages": [{"role": "user", "content": "hello"}],- "temperature": 0.2,- "made_up_beta_field": {"kept": true},- "stream": false- }))- .await?;-- assert_eq!(response.response_type(), ChatResponseType::OpenAiCompletion);- assert_eq!(- response- .body()- .ok_or_else(|| SwitchyardError::Other("buffered response".to_string()))?["id"],- "chatcmpl-test"- );- assert_eq!(ctx.inbound_format, Some(ChatRequestType::OpenAiChat));- assert_eq!(- selected_model(&ctx),- Some(&ModelId::from_static("target-gpt"))- );- assert_eq!(- selected_target(&ctx),- Some(&LlmTargetId::from_static("primary"))- );-- assert_eq!(request.method, "POST");- assert_eq!(request.path, "/v1/chat/completions");- assert_eq!(- request.header("authorization"),- Some("Bearer openai-secret")- );- let expected_version = expected_switchyard_version_header();- assert_eq!(- request.header("x-switchyard-version"),- expected_version.as_deref()- );- assert_eq!(request.body["model"], "target-gpt");- assert_eq!(request.body["messages"][0]["content"], "hello");- assert_eq!(request.body["temperature"], 0.2);- assert_eq!(request.body["made_up_beta_field"], json!({"kept": true}));- assert!(request.body.get("stream_options").is_none());- Ok(())-}--// Responses-format OpenAI targets should call /v1/responses without translating through Chat.-#[tokio::test]-async fn openai_responses_target_posts_responses_body_and_records_context() -> Result<()> {- let server = OneShotServer::json(- 200,- json!({- "id": "resp-test",- "object": "response",- "output": []- }),- )?;- let backend = OpenAiNativeBackend::new(responses_target(format!("{}/v1", server.base_url()))?)?;- let mut ctx = ProxyContext::new();-- let response = backend- .call(- &mut ctx,- &ChatRequest::openai_responses(json!({- "model": "client-gpt",- "input": "hello",- "stream": false- })),- )- .await?;- let request = server.captured()?;-- assert_eq!(- response.response_type(),- ChatResponseType::OpenAiResponsesCompletion- );- assert_eq!(ctx.inbound_format, Some(ChatRequestType::OpenAiResponses));- assert_eq!(- selected_model(&ctx),- Some(&ModelId::from_static("target-responses"))- );- assert_eq!(request.method, "POST");- assert_eq!(request.path, "/v1/responses");- assert_eq!(request.body["model"], "target-responses");- assert_eq!(request.body["input"], "hello");- assert!(request.body.get("messages").is_none());- assert!(request.body.get("stream_options").is_none());- Ok(())-}--// Endpoint-specific base URLs should normalize to the endpoint selected by target format.-#[tokio::test]-async fn openai_specific_base_url_uses_selected_endpoint() -> Result<()> {- let chat_server = OneShotServer::json(200, json!({"id": "chatcmpl-test", "choices": []}))?;- let chat_backend = OpenAiNativeBackend::new(openai_target(format!(- "{}/v1/responses",- chat_server.base_url()- ))?)?;- let mut chat_ctx = ProxyContext::new();-- chat_backend- .call(- &mut chat_ctx,- &ChatRequest::openai_chat(json!({- "model": "client-gpt",- "messages": [{"role": "user", "content": "hello"}]- })),- )- .await?;- assert_eq!(chat_server.captured()?.path, "/v1/chat/completions");-- let responses_server = OneShotServer::json(200, json!({"id": "resp-test", "output": []}))?;- let responses_backend = OpenAiNativeBackend::new(responses_target(format!(- "{}/v1/chat/completions",- responses_server.base_url()- ))?)?;- let mut responses_ctx = ProxyContext::new();-- responses_backend- .call(- &mut responses_ctx,- &ChatRequest::openai_responses(json!({- "model": "client-gpt",- "input": "hello"- })),- )- .await?;- assert_eq!(responses_server.captured()?.path, "/v1/responses");- Ok(())-}--// Chat-format OpenAI targets should keep translating Responses requests to Chat fallback.-#[tokio::test]-async fn openai_chat_target_translates_responses_to_chat_fallback() -> Result<()> {- let server = OneShotServer::json(200, json!({"id": "chatcmpl-test", "choices": []}))?;- let backend = OpenAiNativeBackend::new(openai_target(format!("{}/v1", server.base_url()))?)?;- let mut ctx = ProxyContext::new();-- let response = backend- .call(- &mut ctx,- &ChatRequest::openai_responses(json!({- "model": "client-gpt",- "input": "translate me"- })),- )- .await?;- let request = server.captured()?;-- assert_eq!(response.response_type(), ChatResponseType::OpenAiCompletion);- assert_eq!(ctx.inbound_format, Some(ChatRequestType::OpenAiResponses));- assert_eq!(request.path, "/v1/chat/completions");- assert_eq!(request.body["model"], "target-gpt");- assert_eq!(request.body["messages"][0]["content"], "translate me");- Ok(())-}--// Native OpenAI targets should apply per-target body and header overrides.-#[tokio::test]-async fn openai_native_applies_target_extra_body_and_headers() -> Result<()> {- let server = OneShotServer::json(- 200,- json!({- "id": "chatcmpl-test",- "object": "chat.completion",- "choices": []- }),- )?;- let mut target = openai_target(format!("{}/v1", server.base_url()))?;- target.extra_body = Some(json!({- "chat_template_kwargs": {"enable_thinking": false}- }));- target- .extra_headers- .insert("X-Inference-Priority".to_string(), "batch".to_string());- let backend = OpenAiNativeBackend::new(target)?;- let mut ctx = ProxyContext::new();-- backend- .call(- &mut ctx,- &ChatRequest::openai_chat(json!({- "model": "client-gpt",- "messages": [{"role": "user", "content": "hello"}],- "stream": false- })),- )- .await?;- let request = server.captured()?;-- assert_eq!(- request.body["chat_template_kwargs"],- json!({"enable_thinking": false})- );- assert_eq!(request.header("x-inference-priority"), Some("batch"));- Ok(())-}--// Passthrough OpenAI calls should not rewrite caller model names.-#[tokio::test]-async fn openai_passthrough_preserves_client_model_and_records_context() -> Result<()> {- let server = OneShotServer::json(- 200,- json!({- "id": "chatcmpl-test",- "object": "chat.completion",- "choices": []- }),- )?;- let backend =- OpenAiPassthroughBackend::new(openai_endpoint(format!("{}/v1", server.base_url())))?;- let mut ctx = ProxyContext::new();-- let response = backend- .call(- &mut ctx,- &ChatRequest::openai_chat(json!({- "model": "client-gpt",- "messages": [{"role": "user", "content": "hello"}],- "stream": false- })),- )- .await?;- let request = server.captured()?;-- assert_eq!(response.response_type(), ChatResponseType::OpenAiCompletion);- assert_eq!(ctx.inbound_format, Some(ChatRequestType::OpenAiChat));- assert_eq!(- selected_model(&ctx),- Some(&ModelId::from_static("client-gpt"))- );- assert!(selected_target(&ctx).is_none());- assert_eq!(request.path, "/v1/chat/completions");- assert_eq!(- request.header("authorization"),- Some("Bearer openai-secret")- );- assert_eq!(request.body["model"], "client-gpt");- Ok(())-}--// OpenAI streams should request usage and stop cleanly on `[DONE]`.-#[tokio::test]-async fn openai_streaming_injects_usage_opt_in_and_parses_done() -> Result<()> {- let server = OneShotServer::sse(- "data: {\"choices\":[{\"delta\":{\"content\":\"hi\"}}]}\n\n\- data: {\"usage\":{\"prompt_tokens\":1,\"completion_tokens\":2,\"total_tokens\":3}}\n\n\- data: [DONE]\n\n",- )?;- let backend = OpenAiNativeBackend::new(openai_target(format!("{}/v1", server.base_url()))?)?;- let mut ctx = ProxyContext::new();-- let response = backend- .call(- &mut ctx,- &ChatRequest::openai_chat(json!({- "model": "client-gpt",- "messages": [{"role": "user", "content": "stream"}],- "stream": true- })),- )- .await?;- assert_eq!(response.response_type(), ChatResponseType::OpenAiStream);- let events = collect_json_events(response).await?;- let request = server.captured()?;-- assert_eq!(events.len(), 2);- assert_eq!(events[0]["choices"][0]["delta"]["content"], "hi");- assert_eq!(events[1]["usage"]["total_tokens"], 3);- assert_eq!(- request.body["stream_options"],- json!({"include_usage": true})- );- Ok(())-}--// Existing stream options should win over backend usage defaults.-#[tokio::test]-async fn openai_streaming_respects_usage_opt_out_and_preserves_other_options() -> Result<()> {- let server = OneShotServer::sse("data: [DONE]\n\n")?;- let backend = OpenAiNativeBackend::new(openai_target(format!("{}/v1", server.base_url()))?)?;- let mut ctx = ProxyContext::new();-- let response = backend- .call(- &mut ctx,- &ChatRequest::openai_chat(json!({- "model": "client-gpt",- "messages": [{"role": "user", "content": "stream"}],- "stream": true,- "stream_options": {- "include_usage": false,- "continuous_usage_stats": true- }- })),- )- .await?;- let events = collect_json_events(response).await?;- let request = server.captured()?;-- assert!(events.is_empty());- assert_eq!(- request.body["stream_options"],- json!({- "include_usage": false,- "continuous_usage_stats": true- })- );- Ok(())-}--// OpenAI native should translate Anthropic requests before upstream dispatch.-#[tokio::test]-async fn openai_translates_anthropic_requests_before_native_call() -> Result<()> {- let server = OneShotServer::json(200, json!({"id": "chatcmpl-test", "choices": []}))?;- let backend = OpenAiNativeBackend::new(openai_target(format!("{}/v1", server.base_url()))?)?;- let mut ctx = ProxyContext::new();-- backend- .call(- &mut ctx,- &ChatRequest::anthropic(json!({- "model": "client-claude",- "max_tokens": 128,- "messages": [{"role": "user", "content": "translate me"}]- })),- )- .await?;- let request = server.captured()?;-- assert_eq!(ctx.inbound_format, Some(ChatRequestType::Anthropic));- assert_eq!(request.body["model"], "target-gpt");- assert_eq!(request.body["messages"][0]["role"], "user");- assert_eq!(request.body["messages"][0]["content"], "translate me");- Ok(())-}--// OpenAI passthrough should translate Anthropic shape without model rewriting.-#[tokio::test]-async fn openai_passthrough_translates_anthropic_without_rewriting_model() -> Result<()> {- let server = OneShotServer::json(200, json!({"id": "chatcmpl-test", "choices": []}))?;- let backend =- OpenAiPassthroughBackend::new(openai_endpoint(format!("{}/v1", server.base_url())))?;- let mut ctx = ProxyContext::new();-- backend- .call(- &mut ctx,- &ChatRequest::anthropic(json!({- "model": "client-claude",- "max_tokens": 128,- "messages": [{"role": "user", "content": "translate me"}]- })),- )- .await?;- let request = server.captured()?;-- assert_eq!(ctx.inbound_format, Some(ChatRequestType::Anthropic));- assert_eq!(- selected_model(&ctx),- Some(&ModelId::from_static("client-claude"))- );- assert_eq!(request.body["model"], "client-claude");- assert_eq!(request.body["messages"][0]["content"], "translate me");- Ok(())-}--// Responses streams should call /responses and return Responses stream variants.-#[tokio::test]-async fn openai_responses_streaming_uses_responses_stream_variant() -> Result<()> {- let server = OneShotServer::sse(- "data: {\"type\":\"response.created\",\"response\":{\"id\":\"resp-stream\"}}\n\n",- )?;- let backend = OpenAiNativeBackend::new(responses_target(format!("{}/v1", server.base_url()))?)?;- let mut ctx = ProxyContext::new();-- let response = backend- .call(- &mut ctx,- &ChatRequest::openai_responses(json!({- "model": "client-gpt",- "input": "stream",- "stream": true- })),- )- .await?;- assert_eq!(- response.response_type(),- ChatResponseType::OpenAiResponsesStream- );- let events = collect_json_events(response).await?;- let request = server.captured()?;-- assert_eq!(events.len(), 1);- assert_eq!(events[0]["type"], "response.created");- assert_eq!(request.path, "/v1/responses");- assert!(request.body.get("stream_options").is_none());- Ok(())-}--// OpenAI native must reject targets configured for Anthropic format.-#[test]-fn openai_backend_rejects_anthropic_targets() -> Result<()> {- let Err(error) = OpenAiNativeBackend::new(target(- BackendFormat::Anthropic,- "http://127.0.0.1:1".to_string(),- "secret",- "claude",- )?) else {- return Err(SwitchyardError::Other(- "OpenAI backend should reject Anthropic targets".to_string(),- ));- };-- assert!(matches!(error, SwitchyardError::InvalidConfig(_)));- Ok(())-}--// OpenAI upstream error responses should preserve status and body in the error.-#[tokio::test]-async fn openai_error_status_includes_status_and_body() -> Result<()> {- let server = OneShotServer::json(- 429,- json!({"error": {"message": "rate limited", "type": "rate_limit"}}),- )?;- let backend = OpenAiNativeBackend::new(openai_target(format!("{}/v1", server.base_url()))?)?;- let mut ctx = ProxyContext::new();-- let Err(error) = backend- .call(- &mut ctx,- &ChatRequest::openai_chat(json!({- "model": "client-gpt",- "messages": [{"role": "user", "content": "hello"}]- })),- )- .await- else {- return Err(SwitchyardError::Other(- "OpenAI backend should propagate HTTP errors".to_string(),- ));- };- let captured = server.captured()?;-- assert!(matches!(error, SwitchyardError::UpstreamHttp { .. }));- assert!(error.to_string().contains("HTTP 429"));- assert!(error.to_string().contains("rate limited"));- assert_eq!(captured.path, "/v1/chat/completions");- Ok(())-}--// Anthropic native backends should only advertise Anthropic input.-#[test]-fn anthropic_backend_is_anthropic_only() -> Result<()> {- let backend = AnthropicNativeBackend::new(anthropic_target("http://127.0.0.1:1".to_string())?)?;-- assert_eq!(- backend.supported_request_types(),- &[ChatRequestType::Anthropic]- );- Ok(())-}--// Non-streaming Anthropic calls should strip incompatible fields and stamp context.-#[tokio::test]-async fn anthropic_non_streaming_strips_incompatible_fields_and_records_context() -> Result<()> {- let server = OneShotServer::json(- 200,- json!({- "id": "msg-test",- "type": "message",- "content": []- }),- )?;- let backend = AnthropicNativeBackend::new(anthropic_target(server.base_url().to_string())?)?;- let mut ctx = ProxyContext::new();-- let response = backend- .call(- &mut ctx,- &ChatRequest::anthropic(json!({- "model": "client-claude",- "max_tokens": 128,- "messages": [{"role": "user", "content": "hello"}],- "reasoning_effort": "high",- "context_management": {"strategy": "auto"},- "made_up_beta_field": {"kept": true},- "extra_body": {"caller": "value"},- "stream": false- })),- )- .await?;- let request = server.captured()?;-- assert_eq!(- response.response_type(),- ChatResponseType::AnthropicCompletion- );- assert_eq!(- response- .body()- .ok_or_else(|| SwitchyardError::Other("buffered response".to_string()))?["id"],- "msg-test"- );- assert_eq!(ctx.inbound_format, Some(ChatRequestType::Anthropic));- assert_eq!(- selected_model(&ctx),- Some(&ModelId::from_static("target-claude"))- );- assert_eq!(- selected_target(&ctx),- Some(&LlmTargetId::from_static("primary"))- );-- assert_eq!(request.method, "POST");- assert_eq!(request.path, "/v1/messages");- assert_eq!(request.header("x-api-key"), Some("anthropic-secret"));- assert_eq!(request.header("anthropic-version"), Some("2023-06-01"));- let expected_version = expected_switchyard_version_header();- assert_eq!(- request.header("x-switchyard-version"),- expected_version.as_deref()- );- assert_eq!(request.body["model"], "target-claude");- assert_eq!(request.body["messages"][0]["content"], "hello");- assert!(request.body.get("reasoning_effort").is_none());- assert!(request.body.get("context_management").is_none());- assert_eq!(request.body["made_up_beta_field"], json!({"kept": true}));- assert_eq!(request.body["extra_body"], json!({"caller": "value"}));- Ok(())-}--// Anthropic-native calls should downgrade Opus-4.8-style system turns for legacy targets.-#[tokio::test]-async fn anthropic_lifts_message_level_system_roles_before_native_call() -> Result<()> {- let server = OneShotServer::json(200, json!({"id": "msg-test", "content": []}))?;- let backend = AnthropicNativeBackend::new(anthropic_target(server.base_url().to_string())?)?;- let mut ctx = ProxyContext::new();-- backend- .call(- &mut ctx,- &ChatRequest::anthropic(json!({- "model": "client-claude",- "max_tokens": 128,- "messages": [- {"role": "system", "content": "System rules."},- {"role": "user", "content": "hello"},- {- "role": "developer",- "content": [- {"type": "text", "text": "Developer rules."}- ]- },- {"role": "assistant", "content": "ready"}- ]- })),- )- .await?;- let request = server.captured()?;-- assert_eq!(request.body["system"], "System rules.\n\nDeveloper rules.");- let messages = request.body["messages"]- .as_array()- .ok_or_else(|| SwitchyardError::Other("messages should be an array".to_string()))?;- let roles = messages- .iter()- .map(|message| {- message- .get("role")- .and_then(Value::as_str)- .unwrap_or("<missing>")- })- .collect::<Vec<_>>();- assert_eq!(roles, vec!["user", "assistant"]);- assert_eq!(messages[0]["content"], "hello");- assert_eq!(messages[1]["content"], "ready");- Ok(())-}--// Interleaved system turns should preserve encounter order after lifting.-#[tokio::test]-async fn anthropic_lifts_multiple_interleaved_system_messages_in_order() -> Result<()> {- let server = OneShotServer::json(200, json!({"id": "msg-test", "content": []}))?;- let backend = AnthropicNativeBackend::new(anthropic_target(server.base_url().to_string())?)?;- let mut ctx = ProxyContext::new();-- backend- .call(- &mut ctx,- &ChatRequest::anthropic(json!({- "model": "client-claude",- "max_tokens": 128,- "system": "Top-level rules.",- "messages": [- {"role": "system", "content": "First lifted system."},- {"role": "user", "content": "first user"},- {"role": "system", "content": "Second lifted system."},- {"role": "assistant", "content": "assistant reply"},- {"role": "developer", "content": "Developer lifted system."},- {"role": "user", "content": "second user"}- ]- })),- )- .await?;- let request = server.captured()?;-- assert_eq!(- request.body["system"],- "Top-level rules.\n\nFirst lifted system.\n\nSecond lifted system.\n\nDeveloper lifted system."- );- assert_eq!(- request.body["messages"],- json!([- {"role": "user", "content": "first user"},- {"role": "assistant", "content": "assistant reply"},- {"role": "user", "content": "second user"}- ])- );- Ok(())-}--// Existing structured Anthropic system prompts should keep their shape when lifted text is added.-#[tokio::test]-async fn anthropic_lifts_message_level_system_into_existing_system_blocks() -> Result<()> {- let server = OneShotServer::json(200, json!({"id": "msg-test", "content": []}))?;- let backend = AnthropicNativeBackend::new(anthropic_target(server.base_url().to_string())?)?;- let mut ctx = ProxyContext::new();-- backend- .call(- &mut ctx,- &ChatRequest::anthropic(json!({- "model": "client-claude",- "max_tokens": 128,- "system": [{"type": "text", "text": "Existing system."}],- "messages": [- {- "role": "system",- "content": [- {"type": "text", "text": "Lifted system."},- {"type": "image", "source": {"type": "url", "url": "https://example.test/a.png"}},- {"type": "input_text", "text": "Lifted input text."}- ]- },- {"role": "user", "content": "hello"}- ]- })),- )- .await?;- let request = server.captured()?;-- assert_eq!(- request.body["system"],- json!([- {"type": "text", "text": "Existing system."},- {"type": "text", "text": "Lifted system.\n\nLifted input text."}- ])- );- assert_eq!(- request.body["messages"],- json!([{"role": "user", "content": "hello"}])- );- Ok(())-}--// Responses requests should translate into Anthropic Messages with default max_tokens.-#[tokio::test]-async fn anthropic_translates_responses_requests_with_default_max_tokens() -> Result<()> {- let server = OneShotServer::json(200, json!({"id": "msg-test", "content": []}))?;- let backend = AnthropicNativeBackend::new(anthropic_target(server.base_url().to_string())?)?;- let mut ctx = ProxyContext::new();-- backend- .call(- &mut ctx,- &ChatRequest::openai_responses(json!({- "model": "client-gpt",- "input": "translate me"- })),- )- .await?;- let request = server.captured()?;-- assert_eq!(ctx.inbound_format, Some(ChatRequestType::OpenAiResponses));- assert_eq!(request.body["model"], "target-claude");- assert_eq!(request.body["max_tokens"], 64000);- assert_eq!(request.body["messages"][0]["role"], "user");- assert_eq!(request.body["messages"][0]["content"], "translate me");- Ok(())-}--// Invalid Anthropic tool-use IDs should be sanitized consistently with results.-#[tokio::test]-async fn anthropic_sanitizes_invalid_tool_use_ids_and_matching_results() -> Result<()> {- let server = OneShotServer::json(200, json!({"id": "msg-test", "content": []}))?;- let backend = AnthropicNativeBackend::new(anthropic_target(server.base_url().to_string())?)?;- let mut ctx = ProxyContext::new();-- backend- .call(- &mut ctx,- &ChatRequest::anthropic(json!({- "model": "client-claude",- "max_tokens": 128,- "messages": [- {"role": "user", "content": "use the tool"},- {- "role": "assistant",- "content": [{- "type": "tool_use",- "id": "toolu_01*bad:id",- "name": "lookup",- "input": {}- }]- },- {- "role": "user",- "content": [{- "type": "tool_result",- "tool_use_id": "toolu_01*bad:id",- "content": "done"- }]- }- ]- })),- )- .await?;- let request = server.captured()?;-- let tool_use_id = &request.body["messages"][1]["content"][0]["id"];- assert_eq!(tool_use_id, "toolu_01_bad_id");- assert_eq!(- &request.body["messages"][2]["content"][0]["tool_use_id"],- tool_use_id- );- Ok(())-}--// Unsigned synthetic thinking blocks should be removed before Anthropic replay.-#[tokio::test]-async fn anthropic_strips_unsigned_thinking_blocks_before_native_call() -> Result<()> {- let server = OneShotServer::json(200, json!({"id": "msg-test", "content": []}))?;- let backend = AnthropicNativeBackend::new(anthropic_target(server.base_url().to_string())?)?;- let mut ctx = ProxyContext::new();-- backend- .call(- &mut ctx,- &ChatRequest::anthropic(json!({- "model": "client-claude",- "max_tokens": 128,- "messages": [- {- "role": "assistant",- "content": [- {"type": "thinking", "thinking": "synthetic", "signature": ""},- {"type": "tool_use", "id": "toolu_ok", "name": "lookup", "input": {}}- ]- },- {- "role": "assistant",- "content": [- {"type": "thinking", "thinking": "real", "signature": "signed"},- {"type": "text", "text": "visible"}- ]- },- {- "role": "assistant",- "content": [- {"type": "thinking", "thinking": "only synthetic"}- ]- }- ]- })),- )- .await?;- let request = server.captured()?;-- assert_eq!(- request.body["messages"][0]["content"]- .as_array()- .ok_or_else(|| SwitchyardError::Other("content should be an array".to_string()))?- .len(),- 1- );- assert_eq!(- request.body["messages"][0]["content"][0]["type"],- "tool_use"- );- assert_eq!(- request.body["messages"][1]["content"][0]["type"],- "thinking"- );- assert_eq!(- request.body["messages"][1]["content"][0]["thinking"],- "real"- );- assert_eq!(request.body["messages"][2]["content"], "");- Ok(())-}--// Anthropic SSE should produce JSON stream events.-#[tokio::test]-async fn anthropic_streaming_returns_stream_events() -> Result<()> {- let server = OneShotServer::sse(- "event: message_start\n\- data: {\"type\":\"message_start\",\"message\":{\"id\":\"msg-stream\"}}\n\n",- )?;- let backend = AnthropicNativeBackend::new(anthropic_target(server.base_url().to_string())?)?;- let mut ctx = ProxyContext::new();-- let response = backend- .call(- &mut ctx,- &ChatRequest::anthropic(json!({- "model": "client-claude",- "max_tokens": 128,- "messages": [{"role": "user", "content": "stream"}],- "stream": true- })),- )- .await?;- assert_eq!(response.response_type(), ChatResponseType::AnthropicStream);- let events = collect_json_events(response).await?;- let request = server.captured()?;-- assert_eq!(events.len(), 1);- assert_eq!(events[0]["type"], "message_start");- assert_eq!(events[0]["message"]["id"], "msg-stream");- assert_eq!(request.path, "/v1/messages");- Ok(())-}--#[tokio::test]-async fn anthropic_streaming_stops_at_done_marker() -> Result<()> {- let server = OneShotServer::sse(- "event: message_start\n\- data: {\"type\":\"message_start\",\"message\":{\"id\":\"msg-stream\"}}\n\n\- data: [DONE]\n\n\- event: message_stop\n\- data: {\"type\":\"message_stop\"}\n\n",- )?;- let backend = AnthropicNativeBackend::new(anthropic_target(server.base_url().to_string())?)?;- let mut ctx = ProxyContext::new();-- let response = backend- .call(- &mut ctx,- &ChatRequest::anthropic(json!({- "model": "client-claude",- "max_tokens": 128,- "messages": [{"role": "user", "content": "stream"}],- "stream": true- })),- )- .await?;- let events = collect_json_events(response).await?;- server.captured()?;-- assert_eq!(- events,- vec![json!({- "type": "message_start",- "message": {"id": "msg-stream"}- })]- );- Ok(())-}--// Anthropic native must reject targets configured for OpenAI format.-#[test]-fn anthropic_backend_rejects_openai_targets() -> Result<()> {- let Err(error) = AnthropicNativeBackend::new(target(- BackendFormat::OpenAi,- "http://127.0.0.1:1".to_string(),- "secret",- "gpt",- )?) else {- return Err(SwitchyardError::Other(- "Anthropic backend should reject OpenAI targets".to_string(),- ));- };-- assert!(matches!(error, SwitchyardError::InvalidConfig(_)));- Ok(())-}--// Anthropic upstream error responses should preserve status and body in the error.-#[tokio::test]-async fn anthropic_error_status_includes_status_and_body() -> Result<()> {- let server = OneShotServer::json(- 400,- json!({"error": {"message": "invalid request", "type": "bad_request"}}),- )?;- let backend = AnthropicNativeBackend::new(anthropic_target(server.base_url().to_string())?)?;- let mut ctx = ProxyContext::new();-- let Err(error) = backend- .call(- &mut ctx,- &ChatRequest::anthropic(json!({- "model": "client-claude",- "max_tokens": 128,- "messages": [{"role": "user", "content": "hello"}]- })),- )- .await- else {- return Err(SwitchyardError::Other(- "Anthropic backend should propagate HTTP errors".to_string(),- ));- };- let captured = server.captured()?;-- assert!(matches!(error, SwitchyardError::UpstreamHttp { .. }));- assert!(error.to_string().contains("HTTP 400"));- assert!(error.to_string().contains("invalid request"));- assert_eq!(captured.path, "/v1/messages");- Ok(())-}--// Native backends should reject unresolved Auto formats until config resolves them.-#[test]-fn native_backends_reject_unresolved_auto_targets() -> Result<()> {- let auto_openai = target(- BackendFormat::Auto,- "http://127.0.0.1:1".to_string(),- "secret",- "gpt",- )?;- let Err(openai_error) = OpenAiNativeBackend::new(auto_openai) else {- return Err(SwitchyardError::Other(- "OpenAI backend should reject Auto targets".to_string(),- ));- };- assert!(matches!(openai_error, SwitchyardError::InvalidConfig(_)));-- let auto_anthropic = target(- BackendFormat::Auto,- "http://127.0.0.1:1".to_string(),- "secret",- "claude",- )?;- let Err(anthropic_error) = AnthropicNativeBackend::new(auto_anthropic) else {- return Err(SwitchyardError::Other(- "Anthropic backend should reject Auto targets".to_string(),- ));- };- assert!(matches!(anthropic_error, SwitchyardError::InvalidConfig(_)));- Ok(())-}diff --git a/crates/switchyard-components/tests/adversarial_random_routing.rs b/crates/switchyard-components/tests/adversarial_random_routing.rsdeleted file mode 100644index 3cd0fcaed..000000000--- a/crates/switchyard-components/tests/adversarial_random_routing.rs+++ /dev/null@@ -1,209 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Adversarial tests for the random routing engine.--use serde_json::json;-use switchyard_components::{- BackendFormat, ChatRequest, EndpointConfig, LlmTarget, LlmTargetId, ModelId, Result,- SwitchyardError,-};-use switchyard_components::{RandomRoutingEngine, RandomRoutingProcessorConfig, RandomRoutingTier};--// Builds a deterministic strong/weak config for routing tests.-fn config(strong_probability: f64, rng_seed: u64) -> Result<RandomRoutingProcessorConfig> {- Ok(RandomRoutingProcessorConfig::new(- LlmTarget::new(- LlmTargetId::from_static("strong-target"),- ModelId::from_static("strong-model"),- ),- LlmTarget::new(- LlmTargetId::from_static("weak-target"),- ModelId::from_static("weak-model"),- ),- )- .with_strong_probability(strong_probability)?- .with_rng_seed(Some(rng_seed)))-}--// Builds an OpenAI Chat request whose non-model fields must be preserved.-fn openai_request(model: &str) -> ChatRequest {- ChatRequest::openai_chat(json!({- "model": model,- "messages": [{"role": "user", "content": "keep me"}],- "temperature": 0.3,- }))-}--// Runs one request through the engine and applies the route-local model rewrite.-fn route_once(engine: &RandomRoutingEngine, mut request: ChatRequest) -> Result<ChatRequest> {- let decision = engine.select(request.model().map(str::to_owned))?;- request.set_model(decision.selected_model.as_str());- Ok(request)-}--// Probability zero is a hard weak route and should preserve request payload fields.-#[test]-fn probability_zero_always_routes_to_weak_and_preserves_body_fields() -> Result<()> {- let engine = RandomRoutingEngine::new(config(0.0, 7)?)?;- let request = route_once(&engine, openai_request("client-model"))?;- let decision = engine.select(Some("client-model".to_string()))?;-- assert_eq!(request.model(), Some("weak-model"));- assert_eq!(request.body()["messages"][0]["content"], "keep me");- assert_eq!(request.body()["temperature"], 0.3);- assert_eq!(decision.tier, RandomRoutingTier::Weak);- assert_eq!(decision.selected_model, ModelId::from_static("weak-model"));- assert_eq!(decision.original_model.as_deref(), Some("client-model"));- assert_eq!(decision.strong_probability, 0.0);- assert!((0.0..1.0).contains(&decision.draw));- Ok(())-}--// Probability one is a hard strong route.-#[test]-fn probability_one_always_routes_to_strong() -> Result<()> {- let engine = RandomRoutingEngine::new(config(1.0, 7)?)?;- let request = route_once(&engine, openai_request("client-model"))?;- let decision = engine.select(Some("client-model".to_string()))?;-- assert_eq!(request.model(), Some("strong-model"));- assert_eq!(- decision.selected_target,- LlmTargetId::from_static("strong-target")- );- assert_eq!(decision.tier, RandomRoutingTier::Strong);- Ok(())-}--// Equal seeds should produce identical routing sequences across processors.-#[test]-fn seeded_engines_produce_the_same_routing_sequence() -> Result<()> {- let left = RandomRoutingEngine::new(config(0.5, 42)?)?;- let right = RandomRoutingEngine::new(config(0.5, 42)?)?;-- let mut left_sequence = Vec::new();- let mut right_sequence = Vec::new();- for _ in 0..32 {- left_sequence.push(left.select(Some("client-model".to_string()))?.tier);- right_sequence.push(right.select(Some("client-model".to_string()))?.tier);- }-- assert_eq!(left_sequence, right_sequence);- assert!(left_sequence.contains(&RandomRoutingTier::Strong));- assert!(left_sequence.contains(&RandomRoutingTier::Weak));- Ok(())-}--// Malformed request bodies should be repaired into an object with the selected model.-#[test]-fn malformed_non_object_request_body_is_recovered_with_selected_model() -> Result<()> {- let engine = RandomRoutingEngine::new(config(1.0, 11)?)?;- let request = route_once(&engine, ChatRequest::openai_chat(json!("bad")))?;-- assert_eq!(request.body(), &json!({"model": "strong-model"}));- Ok(())-}--// Invalid probabilities must fail during engine construction, not at call time.-#[test]-fn invalid_probability_is_rejected_before_routing_requests() -> Result<()> {- for value in [-0.1, 1.1, f64::NAN, f64::INFINITY] {- let Err(error) = RandomRoutingEngine::new(RandomRoutingProcessorConfig {- strong: LlmTarget::new(- LlmTargetId::from_static("strong"),- ModelId::from_static("strong-model"),- ),- weak: LlmTarget::new(- LlmTargetId::from_static("weak"),- ModelId::from_static("weak-model"),- ),- strong_probability: value,- rng_seed: Some(1),- }) else {- return Err(SwitchyardError::Other(- "invalid probability should be rejected".to_string(),- ));- };-- assert!(matches!(error, SwitchyardError::InvalidConfig(_)));- }- Ok(())-}--// LLM target serde remains compatible with Python-authored configs.-#[test]-fn llm_target_format_wire_values_match_python_config_contract() -> Result<()> {- assert_eq!(- serde_json::to_value(BackendFormat::Auto)- .map_err(|error| SwitchyardError::Other(error.to_string()))?,- json!("auto")- );- assert_eq!(- serde_json::to_value(BackendFormat::OpenAi)- .map_err(|error| SwitchyardError::Other(error.to_string()))?,- json!("openai")- );- assert_eq!(- serde_json::to_value(BackendFormat::Responses)- .map_err(|error| SwitchyardError::Other(error.to_string()))?,- json!("responses")- );- assert_eq!(- serde_json::to_value(BackendFormat::Anthropic)- .map_err(|error| SwitchyardError::Other(error.to_string()))?,- json!("anthropic")- );- assert!(serde_json::from_value::<BackendFormat>(json!("unknown")).is_err());-- let minimal = LlmTarget::new(- LlmTargetId::from_static("minimal"),- ModelId::from_static("model"),- );- assert_eq!(minimal.format, BackendFormat::Auto);- assert_eq!(minimal.endpoint, EndpointConfig::default());-- let explicit: LlmTarget = serde_json::from_value(json!({- "id": "explicit",- "model": "model",- "format": "openai",- "endpoint": {- "base_url": "https://example.test/v1",- "api_key": "secret",- "timeout_secs": 2.5- }- }))- .map_err(|error| SwitchyardError::Other(error.to_string()))?;- assert_eq!(explicit.format, BackendFormat::OpenAi);- assert_eq!(- explicit.endpoint.base_url.as_deref(),- Some("https://example.test/v1")- );- assert_eq!(explicit.endpoint.api_key.as_deref(), Some("secret"));- assert_eq!(explicit.endpoint.timeout_secs, Some(2.5));- Ok(())-}--// Random routing config defaults remain stable and accept inclusive boundaries.-#[test]-fn random_routing_config_defaults_and_accepts_boundary_probabilities() -> Result<()> {- let base = RandomRoutingProcessorConfig::new(- LlmTarget::new(- LlmTargetId::from_static("strong"),- ModelId::from_static("strong-model"),- ),- LlmTarget::new(- LlmTargetId::from_static("weak"),- ModelId::from_static("weak-model"),- ),- );-- assert_eq!(base.strong_probability, 0.5);- assert_eq!(base.rng_seed, None);-- for probability in [0.0, 0.25, 0.5, 0.75, 1.0] {- let configured = base.clone().with_strong_probability(probability)?;- assert_eq!(configured.strong_probability, probability);- }- Ok(())-}diff --git a/crates/switchyard-components/tests/contracts.rs b/crates/switchyard-components/tests/contracts.rsdeleted file mode 100644index b9955c874..000000000--- a/crates/switchyard-components/tests/contracts.rs+++ /dev/null@@ -1,186 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Adversarial tests for compatibility identifiers, context storage, and wire wrappers.--use std::pin::Pin;-use std::task::{Context, Poll};--use futures_core::Stream;-use serde_json::json;-use switchyard_components::{- BackendFormat, ChatRequest, ChatRequestType, ChatResponse, ChatResponseType, LlmTarget,- LlmTargetId, ModelId, ProxyContext, StreamEvent,-};--type TestResult = std::result::Result<(), Box<dyn std::error::Error + Send + Sync>>;--#[derive(Debug, Eq, PartialEq)]-struct ContextMarker(&'static str);--// Empty stream fixture used to prove streams do not expose buffered JSON bodies.-struct EmptyStream;--impl Stream for EmptyStream {- type Item = switchyard_components::Result<StreamEvent>;-- fn poll_next(self: Pin<&mut Self>, _ctx: &mut Context<'_>) -> Poll<Option<Self::Item>> {- Poll::Ready(None)- }-}--// Verifies every ID constructor and serde path rejects empty identifiers.-#[test]-fn ids_reject_empty_and_whitespace_values_from_constructors_and_serde() -> TestResult {- assert!(ModelId::new("").is_err());- assert!(ModelId::new(" ").is_err());- assert!(serde_json::from_value::<ModelId>(json!("")).is_err());- assert!(serde_json::from_value::<ModelId>(json!(" ")).is_err());-- let parsed = serde_json::from_value::<ModelId>(json!("real-model"))?;- assert_eq!(parsed.as_str(), "real-model");- Ok(())-}--// Verifies typed context extensions are keyed by type, not string names.-#[test]-fn typed_context_extensions_do_not_collide_or_require_string_keys() -> TestResult {- let mut ctx = ProxyContext::new();-- assert!(ctx.insert(ContextMarker("first")).is_none());- assert_eq!(ctx.insert(String::from("unrelated")), None);- assert_eq!(- ctx.insert(ContextMarker("second")),- Some(ContextMarker("first"))- );-- match ctx.get::<ContextMarker>() {- Some(marker) => assert_eq!(marker, &ContextMarker("second")),- None => panic!("marker should be present"),- }- assert_eq!(ctx.get::<String>().map(String::as_str), Some("unrelated"));- assert_eq!(ctx.remove::<ContextMarker>(), Some(ContextMarker("second")));- assert!(ctx.get::<ContextMarker>().is_none());- assert_eq!(ctx.get::<String>().map(String::as_str), Some("unrelated"));- Ok(())-}--// Verifies model rewriting can recover malformed request bodies.-#[test]-fn set_model_recovers_from_malformed_non_object_request_bodies() {- let mut request = ChatRequest::openai_chat(json!("not-an-object"));-- assert_eq!(request.model(), None);- request.set_model("recovered-model");-- assert_eq!(request.request_type(), ChatRequestType::OpenAiChat);- assert_eq!(request.model(), Some("recovered-model"));- assert_eq!(request.body(), &json!({"model": "recovered-model"}));-}--// Verifies serialized wire values stay compatible with the Python side.-#[test]-fn request_and_response_wire_enum_serialization_stays_stable() -> TestResult {- assert_eq!(- serde_json::to_value(ChatRequestType::OpenAiResponses)?,- json!("openai_responses")- );- assert_eq!(- serde_json::to_value(ChatResponseType::AnthropicStream)?,- json!("anthropic_stream")- );-- let request = ChatRequest::anthropic(json!({- "model": "claude",- "messages": [],- }));- assert_eq!(- serde_json::to_value(request)?,- json!({- "request_type": "anthropic",- "request": {- "body": {- "model": "claude",- "messages": [],- },- },- })- );- Ok(())-}--// Verifies OpenAI backend format remains `openai`, not Rust enum snake_case.-#[test]-fn backend_format_stays_wire_compatible_with_python_configs() -> TestResult {- assert_eq!(- serde_json::to_value(BackendFormat::OpenAi)?,- json!("openai")- );- assert_eq!(- serde_json::to_value(BackendFormat::Responses)?,- json!("responses")- );- assert_eq!(- serde_json::from_value::<BackendFormat>(json!("openai"))?,- BackendFormat::OpenAi- );- assert_eq!(- serde_json::from_value::<BackendFormat>(json!("responses"))?,- BackendFormat::Responses- );- assert!(serde_json::from_value::<BackendFormat>(json!("open_ai")).is_err());- Ok(())-}--// Verifies Rust LLM targets reject stale provider tuning fields instead of dropping typos.-#[test]-fn llm_target_rejects_provider_tuning_fields() -> TestResult {- assert!(- serde_json::from_value::<LlmTarget>(json!({- "id": "primary",- "model": "gpt-5",- "format": "openai",- "endpoint": {- "base_url": "https://example.test/v1",- "api_key": null,- "timeout_secs": 30- },- "tuning": {- "max_output_tokens": 4096,- "reasoning_effort": "xhigh"- }- }))- .is_err()- );-- let target = serde_json::from_value::<LlmTarget>(json!({- "id": "primary",- "model": "gpt-5",- "format": "openai",- "endpoint": {- "base_url": "https://example.test/v1",- "api_key": null,- "timeout_secs": 30- }- }))?;- assert_eq!(target.id, LlmTargetId::from_static("primary"));- assert_eq!(target.model, ModelId::from_static("gpt-5"));- assert_eq!(target.format, BackendFormat::OpenAi);- let serialized = serde_json::to_value(target)?;- assert_eq!(- serialized["endpoint"]["base_url"],- "https://example.test/v1"- );- assert!(serialized.get("tuning").is_none());- Ok(())-}--// Verifies streaming responses are distinguishable from buffered JSON responses.-#[test]-fn streaming_responses_are_not_mistaken_for_buffered_json_bodies() {- let response = ChatResponse::OpenAiStream(Box::pin(EmptyStream));-- assert_eq!(response.response_type(), ChatResponseType::OpenAiStream);- assert!(response.body().is_none());- assert!(format!("{response:?}").contains("<stream>"));-}diff --git a/crates/switchyard-components/tests/stats_accumulator.rs b/crates/switchyard-components/tests/stats_accumulator.rsdeleted file mode 100644index e018c4c52..000000000--- a/crates/switchyard-components/tests/stats_accumulator.rs+++ /dev/null@@ -1,789 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--use std::thread;--use serde_json::json;-use switchyard_components::{- ModelStatsSnapshot, StatsAccumulator, StatsSnapshot, TokenUsage, prefix_probe,-};-use switchyard_components::{Result, SwitchyardError};--#[test]-fn accumulator_snapshot_starts_zero_and_reset_clears_all_state() -> Result<()> {- let accumulator = StatsAccumulator::new();- let initial = accumulator.snapshot()?;- assert_eq!(initial.total_requests, 0);- assert_eq!(initial.total_tokens.total, 0);- assert!(initial.models.is_empty());-- accumulator.record_success("model-a", Some(12.5), Some("strong"))?;- accumulator.record_usage(- "model-a",- TokenUsage {- prompt_tokens: 5,- completion_tokens: 7,- cached_tokens: 2,- cache_creation_tokens: 1,- reasoning_tokens: 3,- cacheable_prompt_tokens: 0,- },- Some(20.0),- Some(7.5),- Some("strong"),- )?;- assert_eq!(accumulator.snapshot()?.total_requests, 1);-- accumulator.reset()?;- let reset = accumulator.snapshot()?;- assert_eq!(reset.total_requests, 0);- assert_eq!(reset.total_tokens.total, 0);- assert!(reset.models.is_empty());- Ok(())-}--#[test]-fn accumulator_matches_python_two_tier_snapshot_contract() -> Result<()> {- let accumulator = StatsAccumulator::new();- accumulator.record_success("strong/model", Some(12.0), Some("strong"))?;- accumulator.record_usage(- "strong/model",- TokenUsage {- prompt_tokens: 100,- completion_tokens: 25,- cached_tokens: 10,- cache_creation_tokens: 5,- reasoning_tokens: 3,- cacheable_prompt_tokens: 90,- },- Some(20.0),- Some(8.0),- Some("strong"),- )?;- accumulator.record_success("weak/model", None, Some("weak"))?;- accumulator.record_usage(- "weak/model",- TokenUsage {- prompt_tokens: 40,- completion_tokens: 5,- ..TokenUsage::default()- },- None,- None,- Some("weak"),- )?;-- let snapshot = accumulator.snapshot()?;-- assert_eq!(snapshot.total_requests, 2);- assert_eq!(snapshot.total_tokens.prompt, 140);- assert_eq!(snapshot.total_tokens.completion, 30);- assert_eq!(snapshot.total_tokens.cached, 10);- assert_eq!(snapshot.total_tokens.cache_creation, 5);- assert_eq!(snapshot.total_tokens.reasoning, 3);- assert_eq!(snapshot.total_tokens.total, 170);-- let strong = model_stats(&snapshot, "strong/model")?;- assert_eq!(strong.tier.as_deref(), Some("strong"));- assert_eq!(strong.request_pct, 50.0);- assert_eq!(strong.token_pct, 73.53);- assert_eq!(strong.max_observed_context_tokens, 125);- assert_eq!(strong.avg_prompt_tokens, 100.0);- assert_eq!(strong.avg_completion_tokens, 25.0);- assert_eq!(strong.cache_hit_rate, 0.1);- assert_eq!(strong.theoretical_cache_hit_rate, 0.9);-- let strong_tier = snapshot- .tiers- .get("strong")- .ok_or_else(|| SwitchyardError::Other("strong tier should exist".to_string()))?;- assert_eq!(strong_tier.model, "strong/model");- assert_eq!(strong_tier.prompt_tokens, 100);- assert_eq!(strong_tier.completion_tokens, 25);-- let weak_tier = snapshot- .tiers- .get("weak")- .ok_or_else(|| SwitchyardError::Other("weak tier should exist".to_string()))?;- assert_eq!(weak_tier.model, "weak/model");- assert_eq!(weak_tier.prompt_tokens, 40);- assert_eq!(weak_tier.completion_tokens, 5);- Ok(())-}--#[test]-fn accumulator_tracks_max_observed_context_tokens_per_model() -> Result<()> {- let accumulator = StatsAccumulator::new();- for (prompt_tokens, completion_tokens) in [(100, 10), (90, 50), (120, 5)] {- accumulator.record_success("model-a", None, None)?;- accumulator.record_usage(- "model-a",- TokenUsage {- prompt_tokens,- completion_tokens,- ..TokenUsage::default()- },- None,- None,- None,- )?;- }-- let snapshot = accumulator.snapshot()?;- let model = model_stats(&snapshot, "model-a")?;- assert_eq!(model.prompt_tokens, 310);- assert_eq!(model.completion_tokens, 65);- assert_eq!(model.max_observed_context_tokens, 140);- Ok(())-}--// Switch-aware theoretical: cold on switch-in, partial credit on switching back.-#[test]-fn theoretical_is_switch_aware_across_a_model_switch_and_back() -> Result<()> {- let acc = StatsAccumulator::new();- // One nested conversation; equal-length turns so fractions are exact.- let turn = |content: &[&str]| {- let messages: Vec<_> = content- .iter()- .map(|c| json!({"role": "user", "content": c}))- .collect();- prefix_probe(&json!({ "messages": messages }))- };- let p1 = turn(&["aaaa"]);- let p2 = turn(&["aaaa", "bbbb"]);- let p3 = turn(&["aaaa", "bbbb", "cccc"]);- let p4 = turn(&["aaaa", "bbbb", "cccc", "dddd"]);-- // strong serves turns 1-2: cold first sight, then the turn-1 prefix is eligible.- assert_eq!(acc.prefix_eligibility("strong", &p1), 0.0);- assert_eq!(acc.prefix_eligibility("strong", &p2), 0.5);-- // switch to weak at turn 3: weak's cache is empty.- assert_eq!(acc.prefix_eligibility("weak", &p3), 0.0);-- // back to strong at turn 4: it saw turns 1-2 but not turn 3 (weak's) -> 2/4.- assert_eq!(acc.prefix_eligibility("strong", &p4), 0.5);- Ok(())-}--// Cache stats are attributed per model across an N-model rotation.-#[test]-fn cache_stats_are_per_model_across_multi_model_rotation() -> Result<()> {- let accumulator = StatsAccumulator::new();-- // Three models each entered cold (full re-warm).- for (model, tier) in [- ("model/a", "strong"),- ("model/b", "weak"),- ("model/c", "third"),- ] {- accumulator.record_usage(- model,- TokenUsage {- prompt_tokens: 100,- cache_creation_tokens: 100,- ..TokenUsage::default()- },- None,- None,- Some(tier),- )?;- }- // a and b take a second, warm turn served from cache; c stays cold.- for model in ["model/a", "model/b"] {- accumulator.record_usage(- model,- TokenUsage {- prompt_tokens: 100,- cached_tokens: 100,- ..TokenUsage::default()- },- None,- None,- None,- )?;- }-- let snapshot = accumulator.snapshot()?;- let a = model_stats(&snapshot, "model/a")?;- assert_eq!(a.cache_hit_rate, 0.5);- assert_eq!(a.cache_creation_tokens, 100);- let c = model_stats(&snapshot, "model/c")?;- assert_eq!(c.cache_hit_rate, 0.0);- assert_eq!(c.cache_creation_tokens, 100);- Ok(())-}--#[test]-fn accumulator_is_thread_safe_under_concurrent_recording() -> Result<()> {- let accumulator = StatsAccumulator::new();- let n_threads = 16_u64;- let calls_per_thread = 250_u64;- let mut handles = Vec::new();-- for _ in 0..n_threads {- let accumulator = accumulator.clone();- handles.push(thread::spawn(move || -> Result<()> {- for _ in 0..calls_per_thread {- accumulator.record_success("threaded/model", None, Some("strong"))?;- accumulator.record_usage(- "threaded/model",- TokenUsage {- prompt_tokens: 1,- completion_tokens: 1,- ..TokenUsage::default()- },- None,- None,- Some("strong"),- )?;- }- Ok(())- }));- }-- for handle in handles {- match handle.join() {- Ok(result) => result?,- Err(_) => {- return Err(SwitchyardError::Other(- "stats worker thread panicked".to_string(),- ));- }- }- }-- let expected = n_threads * calls_per_thread;- let snapshot = accumulator.snapshot()?;- let model = model_stats(&snapshot, "threaded/model")?;- assert_eq!(snapshot.total_requests, expected);- assert_eq!(model.calls, expected);- assert_eq!(model.prompt_tokens, expected);- assert_eq!(model.completion_tokens, expected);- Ok(())-}--#[test]-fn latency_reservoir_uses_replacement_path_after_saturation() -> Result<()> {- let accumulator = StatsAccumulator::new();- for latency_ms in 0..10_050 {- accumulator.record_success("reservoir/model", Some(latency_ms as f64), Some("strong"))?;- }-- let snapshot = accumulator.snapshot()?;- let model = model_stats(&snapshot, "reservoir/model")?;-- assert_eq!(model.model_call_latency.count, 10_050);- assert_eq!(model.model_call_latency.min_ms, 0.0);- assert_eq!(model.model_call_latency.max_ms, 10_049.0);- assert_eq!(model.model_call_latency.p50_ms, 5_050.0);- assert_eq!(model.model_call_latency.p99_ms, 9_950.0);- Ok(())-}--#[test]-fn cost_estimate_matches_python_known_model_and_unknown_model_behavior() -> Result<()> {- let accumulator = StatsAccumulator::new();- accumulator.record_usage(- "claude-sonnet-4-6",- TokenUsage {- prompt_tokens: 1_000_000,- ..TokenUsage::default()- },- None,- None,- None,- )?;- accumulator.record_usage(- "claude-sonnet-4-6-20251022",- TokenUsage {- prompt_tokens: 1_000_000,- ..TokenUsage::default()- },- None,- None,- None,- )?;- accumulator.record_usage(- "unknown/model",- TokenUsage {- prompt_tokens: 100,- completion_tokens: 50,- ..TokenUsage::default()- },- None,- None,- None,- )?;-- let snapshot = accumulator.snapshot()?;- let sonnet = snapshot- .cost_estimate- .models- .get("claude-sonnet-4-6")- .ok_or_else(|| SwitchyardError::Other("sonnet cost should exist".to_string()))?;- let dated_sonnet = snapshot- .cost_estimate- .models- .get("claude-sonnet-4-6-20251022")- .ok_or_else(|| SwitchyardError::Other("dated sonnet cost should exist".to_string()))?;- let unknown = snapshot- .cost_estimate- .models- .get("unknown/model")- .ok_or_else(|| SwitchyardError::Other("unknown cost should exist".to_string()))?;-- assert_eq!(sonnet.total_cost, 3.0);- assert_eq!(dated_sonnet.total_cost, 0.0);- assert_eq!(unknown.total_cost, 0.0);- assert_eq!(snapshot.cost_estimate.total_cost, 3.0);- Ok(())-}--#[test]-fn tier_rollup_aggregates_shared_tier_under_first_model() -> Result<()> {- let accumulator = StatsAccumulator::new();- accumulator.record_success("strong/first", None, Some("strong"))?;- accumulator.record_usage(- "strong/first",- TokenUsage {- prompt_tokens: 2,- completion_tokens: 3,- ..TokenUsage::default()- },- None,- None,- Some("strong"),- )?;- accumulator.record_success("strong/second", None, Some("strong"))?;- accumulator.record_usage(- "strong/second",- TokenUsage {- prompt_tokens: 5,- completion_tokens: 7,- ..TokenUsage::default()- },- None,- None,- Some("strong"),- )?;-- let snapshot = accumulator.snapshot()?;- let tier = snapshot- .tiers- .get("strong")- .ok_or_else(|| SwitchyardError::Other("strong tier should exist".to_string()))?;-- assert_eq!(tier.model, "strong/first");- assert_eq!(tier.calls, 2);- assert_eq!(tier.prompt_tokens, 7);- assert_eq!(tier.completion_tokens, 10);- assert_eq!(tier.total_tokens, 17);- Ok(())-}--#[test]-fn tier_rollup_keeps_distinct_tiers_for_shared_model() -> Result<()> {- let accumulator = StatsAccumulator::new();- accumulator.record_success("shared/model", None, Some("weak"))?;- accumulator.record_usage(- "shared/model",- TokenUsage {- prompt_tokens: 2,- completion_tokens: 3,- ..TokenUsage::default()- },- None,- None,- Some("weak"),- )?;- accumulator.record_success("shared/model", None, Some("primary"))?;- accumulator.record_usage(- "shared/model",- TokenUsage {- prompt_tokens: 5,- completion_tokens: 7,- ..TokenUsage::default()- },- None,- None,- Some("primary"),- )?;-- let snapshot = accumulator.snapshot()?;- let shared = model_stats(&snapshot, "shared/model")?;- assert_eq!(shared.calls, 2);- assert_eq!(shared.prompt_tokens, 7);- assert_eq!(shared.completion_tokens, 10);-- let weak = snapshot- .tiers- .get("weak")- .ok_or_else(|| SwitchyardError::Other("weak tier should exist".to_string()))?;- assert_eq!(weak.model, "shared/model");- assert_eq!(weak.calls, 1);- assert_eq!(weak.prompt_tokens, 2);- assert_eq!(weak.completion_tokens, 3);-- let primary = snapshot- .tiers- .get("primary")- .ok_or_else(|| SwitchyardError::Other("primary tier should exist".to_string()))?;- assert_eq!(primary.model, "shared/model");- assert_eq!(primary.calls, 1);- assert_eq!(primary.prompt_tokens, 5);- assert_eq!(primary.completion_tokens, 7);- Ok(())-}--#[test]-fn tier_usage_can_attach_explicit_untiered_success() -> Result<()> {- let accumulator = StatsAccumulator::new();- accumulator.record_success("shared/model", None, None)?;- accumulator.record_success("shared/model", None, None)?;- accumulator.record_usage_with_success_was_untiered(- "shared/model",- TokenUsage {- prompt_tokens: 2,- completion_tokens: 3,- ..TokenUsage::default()- },- None,- None,- Some("weak"),- )?;- accumulator.record_usage_with_success_was_untiered(- "shared/model",- TokenUsage {- prompt_tokens: 5,- completion_tokens: 7,- ..TokenUsage::default()- },- None,- None,- Some("primary"),- )?;-- let snapshot = accumulator.snapshot()?;- let shared = model_stats(&snapshot, "shared/model")?;- assert_eq!(shared.calls, 2);- assert_eq!(shared.prompt_tokens, 7);- assert_eq!(shared.completion_tokens, 10);-- let weak = snapshot- .tiers- .get("weak")- .ok_or_else(|| SwitchyardError::Other("weak tier should exist".to_string()))?;- assert_eq!(weak.calls, 1);- assert_eq!(weak.prompt_tokens, 2);- assert_eq!(weak.completion_tokens, 3);-- let primary = snapshot- .tiers- .get("primary")- .ok_or_else(|| SwitchyardError::Other("primary tier should exist".to_string()))?;- assert_eq!(primary.calls, 1);- assert_eq!(primary.prompt_tokens, 5);- assert_eq!(primary.completion_tokens, 7);- Ok(())-}--#[test]-fn tier_usage_preserves_legacy_untiered_success_sequence() -> Result<()> {- let accumulator = StatsAccumulator::new();- accumulator.record_success("shared/model", None, None)?;- accumulator.record_usage(- "shared/model",- TokenUsage {- prompt_tokens: 2,- completion_tokens: 3,- ..TokenUsage::default()- },- None,- None,- Some("weak"),- )?;-- let snapshot = accumulator.snapshot()?;- let shared = model_stats(&snapshot, "shared/model")?;- assert_eq!(shared.calls, 1);- assert_eq!(shared.prompt_tokens, 2);- assert_eq!(shared.completion_tokens, 3);-- let weak = snapshot- .tiers- .get("weak")- .ok_or_else(|| SwitchyardError::Other("weak tier should exist".to_string()))?;- assert_eq!(weak.calls, 1);- assert_eq!(weak.prompt_tokens, 2);- assert_eq!(weak.completion_tokens, 3);- Ok(())-}--#[test]-fn already_attributed_usage_does_not_consume_legacy_pending_success() -> Result<()> {- let accumulator = StatsAccumulator::new();- accumulator.record_success("shared/model", None, None)?;- accumulator.record_success("shared/model", None, Some("weak"))?;- accumulator.record_usage_after_success_attribution(- "shared/model",- TokenUsage {- prompt_tokens: 2,- completion_tokens: 3,- ..TokenUsage::default()- },- None,- None,- Some("weak"),- )?;- accumulator.record_usage(- "shared/model",- TokenUsage {- prompt_tokens: 5,- completion_tokens: 7,- ..TokenUsage::default()- },- None,- None,- Some("primary"),- )?;-- let snapshot = accumulator.snapshot()?;- let shared = model_stats(&snapshot, "shared/model")?;- assert_eq!(shared.calls, 2);- assert_eq!(shared.prompt_tokens, 7);- assert_eq!(shared.completion_tokens, 10);-- let weak = snapshot- .tiers- .get("weak")- .ok_or_else(|| SwitchyardError::Other("weak tier should exist".to_string()))?;- assert_eq!(weak.calls, 1);- assert_eq!(weak.prompt_tokens, 2);- assert_eq!(weak.completion_tokens, 3);-- let primary = snapshot- .tiers- .get("primary")- .ok_or_else(|| SwitchyardError::Other("primary tier should exist".to_string()))?;- assert_eq!(primary.calls, 1);- assert_eq!(primary.prompt_tokens, 5);- assert_eq!(primary.completion_tokens, 7);- Ok(())-}--#[test]-fn generic_tier_labels_are_included_in_tier_rollups() -> Result<()> {- let accumulator = StatsAccumulator::new();- accumulator.record_success("plugin-a", None, Some("plugin"))?;- accumulator.record_usage(- "plugin-a",- TokenUsage {- prompt_tokens: 2,- completion_tokens: 3,- ..TokenUsage::default()- },- None,- None,- Some("plugin"),- )?;- accumulator.record_success("plugin-b", None, Some("plugin"))?;- accumulator.record_usage(- "plugin-b",- TokenUsage {- prompt_tokens: 5,- completion_tokens: 7,- ..TokenUsage::default()- },- None,- None,- Some("plugin"),- )?;-- let snapshot = accumulator.snapshot()?;- let plugin_a = model_stats(&snapshot, "plugin-a")?;- let plugin_b = model_stats(&snapshot, "plugin-b")?;-- assert_eq!(plugin_a.tier.as_deref(), Some("plugin"));- assert_eq!(plugin_b.tier.as_deref(), Some("plugin"));- let plugin_tier = snapshot- .tiers- .get("plugin")- .ok_or_else(|| SwitchyardError::Other("plugin tier should exist".to_string()))?;- assert_eq!(plugin_tier.model, "plugin-a");- assert_eq!(plugin_tier.calls, 2);- assert_eq!(plugin_tier.prompt_tokens, 7);- assert_eq!(plugin_tier.completion_tokens, 10);- assert_eq!(plugin_tier.total_tokens, 17);- assert_eq!(snapshot.total_tokens.prompt, 7);- assert_eq!(snapshot.total_tokens.completion, 10);- Ok(())-}--#[test]-fn classifier_bucket_is_empty_by_default() -> Result<()> {- let accumulator = StatsAccumulator::new();- accumulator.record_usage(- "claude-sonnet-4-6",- TokenUsage {- prompt_tokens: 1_000_000,- ..TokenUsage::default()- },- None,- None,- Some("strong"),- )?;- let snapshot = accumulator.snapshot()?;- assert!(snapshot.classifier.models.is_empty());- assert_eq!(snapshot.classifier.total_requests, 0);- assert_eq!(snapshot.cost_estimate.classifier_cost, 0.0);- // backend_cost == total_cost when classifier is empty.- assert_eq!(- snapshot.cost_estimate.backend_cost,- snapshot.cost_estimate.total_cost- );- Ok(())-}--#[test]-fn classifier_bucket_keeps_same_model_separate_from_routed_traffic() -> Result<()> {- // Default TB-lite config: classifier_model == weak_model. Both record- // against the same model id but must land in distinct buckets so spend- // is plainly attributable.- let accumulator = StatsAccumulator::new();- let model = "claude-sonnet-4-6";- accumulator.record_success(model, None, Some("weak"))?;- accumulator.record_usage(- model,- TokenUsage {- prompt_tokens: 1_000_000,- ..TokenUsage::default()- },- None,- None,- Some("weak"),- )?;- accumulator.record_classifier_usage(- model,- TokenUsage {- prompt_tokens: 500_000,- ..TokenUsage::default()- },- Some(42.0),- )?;-- let snapshot = accumulator.snapshot()?;-- // Backend bucket: one row, original 1M prompt tokens.- let backend = snapshot- .models- .get(model)- .ok_or_else(|| SwitchyardError::Other("backend row missing".to_string()))?;- assert_eq!(backend.prompt_tokens, 1_000_000);- assert_eq!(backend.max_observed_context_tokens, 1_000_000);- assert_eq!(backend.calls, 1);-- // Classifier bucket: same model id, separate row, separate counts.- let classifier = snapshot- .classifier- .models- .get(model)- .ok_or_else(|| SwitchyardError::Other("classifier row missing".to_string()))?;- assert_eq!(classifier.prompt_tokens, 500_000);- assert_eq!(classifier.max_observed_context_tokens, 500_000);- assert_eq!(classifier.calls, 1);- assert_eq!(snapshot.classifier.total_requests, 1);-- // Cost split: backend = 3.0 (1M @ $3/M); classifier = 1.5 (500k @ $3/M).- // total_cost rolls both in.- assert_eq!(snapshot.cost_estimate.backend_cost, 3.0);- assert_eq!(snapshot.cost_estimate.classifier_cost, 1.5);- assert_eq!(snapshot.cost_estimate.total_cost, 4.5);- Ok(())-}--#[test]-fn classifier_latency_lands_on_classifier_model_call_latency() -> Result<()> {- let accumulator = StatsAccumulator::new();- accumulator.record_classifier_usage(- "claude-sonnet-4-6",- TokenUsage {- prompt_tokens: 10,- completion_tokens: 5,- ..TokenUsage::default()- },- Some(15.0),- )?;- let snapshot = accumulator.snapshot()?;- let row = snapshot- .classifier- .models- .get("claude-sonnet-4-6")- .ok_or_else(|| SwitchyardError::Other("classifier row missing".to_string()))?;- assert_eq!(row.max_observed_context_tokens, 15);- assert_eq!(row.model_call_latency.count, 1);- assert_eq!(row.model_call_latency.total_ms, 15.0);- Ok(())-}--#[test]-fn reset_clears_classifier_bucket() -> Result<()> {- let accumulator = StatsAccumulator::new();- accumulator.record_classifier_usage(- "claude-sonnet-4-6",- TokenUsage {- prompt_tokens: 100,- ..TokenUsage::default()- },- None,- )?;- accumulator.reset()?;- let snapshot = accumulator.snapshot()?;- assert!(snapshot.classifier.models.is_empty());- assert_eq!(snapshot.classifier.total_requests, 0);- assert_eq!(snapshot.cost_estimate.classifier_cost, 0.0);- Ok(())-}--#[test]-fn routing_decision_counts_are_grouped_by_profile_type() -> Result<()> {- let accumulator = StatsAccumulator::new();- accumulator.record_routing_decision("stage_router", "dimensions")?;- accumulator.record_routing_decision("stage_router", "dimensions")?;- accumulator.record_routing_decision("stage_router", "llm-classifier")?;- accumulator.record_routing_decision("escalation_router", "judge")?;-- let snapshot = accumulator.snapshot()?;- assert_eq!(- snapshot- .routing_decisions- .get("stage_router")- .and_then(|sources| sources.get("dimensions")),- Some(&2)- );- assert_eq!(- snapshot- .routing_decisions- .get("stage_router")- .and_then(|sources| sources.get("llm-classifier")),- Some(&1)- );- assert_eq!(- snapshot- .routing_decisions- .get("escalation_router")- .and_then(|sources| sources.get("judge")),- Some(&1)- );-- accumulator.reset()?;- assert!(accumulator.snapshot()?.routing_decisions.is_empty());- Ok(())-}--fn model_stats<'a>(snapshot: &'a StatsSnapshot, model: &str) -> Result<&'a ModelStatsSnapshot> {- snapshot- .models- .get(model)- .ok_or_else(|| SwitchyardError::Other(format!("model stats missing for {model}")))-}diff --git a/crates/switchyard-components/tests/stats_processors.rs b/crates/switchyard-components/tests/stats_processors.rsdeleted file mode 100644index 434b7f0e9..000000000--- a/crates/switchyard-components/tests/stats_processors.rs+++ /dev/null@@ -1,838 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Stats processor tests covering request stamps, backend wrappers, and stream usage.--use std::sync::Arc;-use std::sync::atomic::{AtomicUsize, Ordering};-use std::time::Duration;--use async_trait::async_trait;-use futures_util::StreamExt;-use parking_lot::Mutex;-use serde_json::json;-use switchyard_components::{- BackendSelection, BackendSelectionReason, RandomRoutingDecision, RandomRoutingTier,- StatsAccumulator, StatsBackendLatency, StatsLlmBackend, StatsRequestProcessor,- StatsRequestStart, StatsResponseProcessor, StatsRouteLabel,-};-use switchyard_components::{- ChatRequest, ChatRequestType, ChatResponse, LlmBackend, LlmTargetId, ModelId, ProxyContext,- Result, StreamEvent, SwitchyardError,-};--static SUPPORTED_OPENAI_CHAT: [ChatRequestType; 1] = [ChatRequestType::OpenAiChat];--// Stamps a served model into context the same way native backends do.-fn record_backend_selection(ctx: &mut ProxyContext, model: ModelId) {- let _ = ctx.insert(BackendSelection::for_model(- model,- None,- BackendSelectionReason::PassthroughModel,- ));-}--// Fake backend records lifecycle and request observations for stats tests.-struct FakeBackend {- response: Mutex<Option<Result<ChatResponse>>>,- calls: Mutex<Vec<Option<String>>>,- selected_model: Option<ModelId>,- tier: Option<String>,- startup_count: AtomicUsize,- shutdown_count: AtomicUsize,-}--impl FakeBackend {- // Creates a fake backend that returns one successful response.- fn success(response: ChatResponse) -> Self {- Self {- response: Mutex::new(Some(Ok(response))),- calls: Mutex::new(Vec::new()),- selected_model: None,- tier: None,- startup_count: AtomicUsize::new(0),- shutdown_count: AtomicUsize::new(0),- }- }-- // Creates a fake backend that returns one error.- fn error(error: SwitchyardError) -> Self {- Self {- response: Mutex::new(Some(Err(error))),- calls: Mutex::new(Vec::new()),- selected_model: None,- tier: None,- startup_count: AtomicUsize::new(0),- shutdown_count: AtomicUsize::new(0),- }- }-- // Configures the model that the fake backend records in context.- fn with_selected_model(mut self, model: ModelId) -> Self {- self.selected_model = Some(model);- self- }-- // Configures the route label that the fake backend records in context.- fn with_tier(mut self, tier: impl Into<String>) -> Self {- self.tier = Some(tier.into());- self- }-- // Returns every request model observed by the fake backend.- fn calls(&self) -> Result<Vec<Option<String>>> {- Ok(self.calls.lock().clone())- }-}--#[async_trait]-impl LlmBackend for FakeBackend {- fn supported_request_types(&self) -> &[ChatRequestType] {- &SUPPORTED_OPENAI_CHAT- }-- async fn call(&self, ctx: &mut ProxyContext, request: &ChatRequest) -> Result<ChatResponse> {- self.calls.lock().push(request.model().map(str::to_string));- if let Some(model) = &self.selected_model {- record_backend_selection(ctx, model.clone());- }- if let Some(tier) = &self.tier {- ctx.insert(StatsRouteLabel::new(tier.clone()));- }- self.response- .lock()- .take()- .ok_or_else(|| SwitchyardError::Other("fake response already consumed".to_string()))?- }-- async fn startup(&self) -> Result<()> {- self.startup_count.fetch_add(1, Ordering::SeqCst);- Ok(())- }-- async fn shutdown(&self) -> Result<()> {- self.shutdown_count.fetch_add(1, Ordering::SeqCst);- Ok(())- }-}--// Request stats should add timing state without changing the request.-#[tokio::test]-async fn request_processor_stamps_start_without_mutating_request() -> Result<()> {- let processor = StatsRequestProcessor::default();- let request = ChatRequest::openai_chat(json!({"model": "client", "messages": []}));- let mut ctx = ProxyContext::new();-- let processed = processor.process(&mut ctx, request.clone()).await?;-- assert_eq!(processed, request);- if ctx.get::<StatsRequestStart>().is_none() {- return Err(SwitchyardError::Other(- "stats request start should be stamped".to_string(),- ));- }- Ok(())-}--// End-to-end stats components should share one accumulator across the chain.-#[tokio::test]-async fn full_stats_chain_shares_one_accumulator_across_all_components() -> Result<()> {- let accumulator = StatsAccumulator::new();- let request_processor = StatsRequestProcessor::default();- let backend = StatsLlmBackend::new(- Arc::new(- FakeBackend::success(ChatResponse::openai_completion(json!({- "usage": {"prompt_tokens": 12, "completion_tokens": 8}- })))- .with_selected_model(ModelId::new("served-chain-model")?)- .with_tier("strong"),- ),- accumulator.clone(),- );- let response_processor = StatsResponseProcessor::new(accumulator.clone());- let mut ctx = ProxyContext::new();-- let request = request_processor- .process(- &mut ctx,- ChatRequest::openai_chat(json!({"model": "client-model", "messages": []})),- )- .await?;- let response = backend.call(&mut ctx, &request).await?;- response_processor.process(&mut ctx, response).await?;-- let snapshot = accumulator.snapshot()?;- assert_eq!(snapshot.total_requests, 1);- assert_eq!(snapshot.total_tokens.prompt, 12);- assert_eq!(snapshot.total_tokens.completion, 8);- assert_eq!(snapshot.routing_overhead.count, 1);- let model = model_stats(&snapshot, "served-chain-model")?;- assert_eq!(model.calls, 1);- assert_eq!(model.model_call_latency.count, 1);- assert_eq!(model.total_latency.count, 1);- Ok(())-}--// Theoretical flows request -> ctx -> response -> snapshot, and is switch-aware:-// a model only credits a prefix it has already been sent.-#[tokio::test]-async fn theoretical_cache_hit_rate_flows_through_the_chain() -> Result<()> {- let accumulator = StatsAccumulator::new();- let request_processor = StatsRequestProcessor::new(true);- let response_processor = StatsResponseProcessor::new(accumulator.clone());- // FakeBackend serves one response, so build a fresh one per turn.- let make_backend = || -> Result<StatsLlmBackend> {- Ok(StatsLlmBackend::new(- Arc::new(- FakeBackend::success(ChatResponse::openai_completion(json!({- "usage": {"prompt_tokens": 100, "completion_tokens": 4}- })))- .with_selected_model(ModelId::new("served-model")?)- .with_tier("strong"),- ),- accumulator.clone(),- ))- };-- // Turn 1: first sight of the conversation, nothing cached yet.- let mut ctx = ProxyContext::new();- let request = request_processor- .process(- &mut ctx,- ChatRequest::openai_chat(json!({- "model": "m",- "messages": [{"role": "user", "content": "aaaa"}],- })),- )- .await?;- let response = make_backend()?.call(&mut ctx, &request).await?;- response_processor.process(&mut ctx, response).await?;-- // Turn 2: same model, the prior turn is re-presented, so half is eligible.- let mut ctx = ProxyContext::new();- let request = request_processor- .process(- &mut ctx,- ChatRequest::openai_chat(json!({- "model": "m",- "messages": [- {"role": "user", "content": "aaaa"},- {"role": "user", "content": "bbbb"},- ],- })),- )- .await?;- let response = make_backend()?.call(&mut ctx, &request).await?;- response_processor.process(&mut ctx, response).await?;-- let snapshot = accumulator.snapshot()?;- let model = model_stats(&snapshot, "served-model")?;- assert_eq!(model.cache_hit_rate, 0.0);- // Turn 1 cold (0/100) + turn 2 half (50/100) = 50 over 200 prompt tokens.- assert_eq!(model.theoretical_cache_hit_rate, 0.25);- Ok(())-}--// Backend wrapper should record served model latency and delegate lifecycle.-#[tokio::test]-async fn backend_wrapper_records_success_using_served_model_and_delegates_lifecycle() -> Result<()>-{- let accumulator = StatsAccumulator::new();- let inner = Arc::new(- FakeBackend::success(ChatResponse::openai_completion(json!({"id": "ok"})))- .with_selected_model(ModelId::new("served-model")?)- .with_tier("strong"),- );- let backend = StatsLlmBackend::new(inner.clone(), accumulator.clone());- let request = ChatRequest::openai_chat(json!({"model": "client-model", "messages": []}));- let mut ctx = ProxyContext::new();-- assert_eq!(- backend.supported_request_types(),- &[ChatRequestType::OpenAiChat]- );- backend.startup().await?;- let response = backend.call(&mut ctx, &request).await?;- backend.shutdown().await?;-- assert!(matches!(response, ChatResponse::OpenAiCompletion(_)));- assert_eq!(inner.startup_count.load(Ordering::SeqCst), 1);- assert_eq!(inner.shutdown_count.load(Ordering::SeqCst), 1);- assert_eq!(inner.calls()?, vec![Some("client-model".to_string())]);- if ctx.get::<StatsBackendLatency>().is_none() {- return Err(SwitchyardError::Other(- "backend latency should be stamped".to_string(),- ));- }-- let snapshot = accumulator.snapshot()?;- assert_eq!(snapshot.total_requests, 1);- let model = model_stats(&snapshot, "served-model")?;- assert_eq!(model.calls, 1);- assert_eq!(model.errors, 0);- assert_eq!(model.tier.as_deref(), Some("strong"));- assert_eq!(model.model_call_latency.count, 1);- Ok(())-}--// If a backend does not stamp a served model, request model is the fallback.-#[tokio::test]-async fn backend_wrapper_falls_back_to_request_model_when_backend_does_not_stamp_model()--> Result<()> {- let accumulator = StatsAccumulator::new();- let backend = StatsLlmBackend::new(- Arc::new(FakeBackend::success(ChatResponse::openai_completion(- json!({"id": "ok"}),- ))),- accumulator.clone(),- );- let request = ChatRequest::openai_chat(json!({"model": "client-fallback", "messages": []}));- let mut ctx = ProxyContext::new();-- backend.call(&mut ctx, &request).await?;-- let snapshot = accumulator.snapshot()?;- let model = model_stats(&snapshot, "client-fallback")?;- assert_eq!(model.calls, 1);- Ok(())-}--// Backend wrapper should preserve the original error while counting it.-#[tokio::test]-async fn backend_wrapper_records_errors_and_preserves_original_error() -> Result<()> {- let accumulator = StatsAccumulator::new();- let inner = Arc::new(- FakeBackend::error(SwitchyardError::Upstream("boom".to_string()))- .with_selected_model(ModelId::new("served-error-model")?)- .with_tier("weak"),- );- let backend = StatsLlmBackend::new(inner, accumulator.clone());- let request = ChatRequest::openai_chat(json!({"model": "client-model", "messages": []}));- let mut ctx = ProxyContext::new();-- let Err(error) = backend.call(&mut ctx, &request).await else {- return Err(SwitchyardError::Other(- "backend wrapper should return inner error".to_string(),- ));- };-- assert!(matches!(error, SwitchyardError::Upstream(_)));- assert!(error.to_string().contains("boom"));- let snapshot = accumulator.snapshot()?;- assert_eq!(snapshot.total_requests, 1);- assert_eq!(snapshot.total_errors, 1);- let model = model_stats(&snapshot, "served-error-model")?;- assert_eq!(model.calls, 0);- assert_eq!(model.errors, 1);- assert_eq!(model.tier.as_deref(), Some("weak"));- Ok(())-}--// Response stats should clamp impossible negative overhead to zero.-#[tokio::test]-async fn response_processor_records_openai_usage_latency_and_clamps_negative_overhead() -> Result<()>-{- let accumulator = StatsAccumulator::new();- let processor = StatsResponseProcessor::new(accumulator.clone());- let mut ctx = ProxyContext::new();- record_backend_selection(&mut ctx, ModelId::new("openai-model")?);- ctx.insert(StatsRequestStart::now());- ctx.insert(StatsBackendLatency(Duration::from_secs(60)));-- let response = ChatResponse::openai_completion(json!({- "usage": {- "prompt_tokens": 11,- "completion_tokens": 5,- "prompt_tokens_details": {- "cached_tokens": 3,- "cache_creation_tokens": 2- },- "completion_tokens_details": {- "reasoning_tokens": 4- }- }- }));- let processed = processor.process(&mut ctx, response).await?;-- assert!(matches!(processed, ChatResponse::OpenAiCompletion(_)));- let snapshot = accumulator.snapshot()?;- assert_eq!(snapshot.total_requests, 0);- assert_eq!(snapshot.total_tokens.prompt, 11);- assert_eq!(snapshot.total_tokens.completion, 5);- assert_eq!(snapshot.total_tokens.cached, 3);- assert_eq!(snapshot.total_tokens.cache_creation, 2);- assert_eq!(snapshot.total_tokens.reasoning, 4);- assert_eq!(snapshot.routing_overhead.count, 1);- assert_eq!(snapshot.routing_overhead.max_ms, 0.0);- let model = model_stats(&snapshot, "openai-model")?;- assert_eq!(model.total_latency.count, 1);- Ok(())-}--// Anthropic cache counters should contribute to prompt token accounting.-#[tokio::test]-async fn response_processor_sums_anthropic_cache_buckets_into_prompt_tokens() -> Result<()> {- let accumulator = StatsAccumulator::new();- let processor = StatsResponseProcessor::new(accumulator.clone());- let mut ctx = ProxyContext::new();- record_backend_selection(&mut ctx, ModelId::new("claude-model")?);-- let response = ChatResponse::anthropic_completion(json!({- "usage": {- "input_tokens": 10,- "output_tokens": 6,- "cache_read_input_tokens": 3,- "cache_creation_input_tokens": 2,- "output_tokens_details": {"reasoning_tokens": 1}- }- }));- processor.process(&mut ctx, response).await?;-- let snapshot = accumulator.snapshot()?;- let model = model_stats(&snapshot, "claude-model")?;- assert_eq!(model.prompt_tokens, 15);- assert_eq!(model.completion_tokens, 6);- assert_eq!(model.cached_tokens, 3);- assert_eq!(model.cache_creation_tokens, 2);- assert_eq!(model.reasoning_tokens, 1);- Ok(())-}--// Responses streams should pass through unchanged while capturing final usage.-#[tokio::test]-async fn streaming_response_is_forwarded_and_records_nested_responses_usage() -> Result<()> {- let accumulator = StatsAccumulator::new();- let processor = StatsResponseProcessor::new(accumulator.clone());- let mut ctx = ProxyContext::new();- record_backend_selection(&mut ctx, ModelId::new("responses-model")?);- ctx.insert(StatsRequestStart::now());-- let events = vec![- StreamEvent::Json(json!({"type": "response.output_text.delta", "delta": "hi"})),- StreamEvent::Json(json!({- "type": "response.in_progress",- "response": {"usage": null}- })),- StreamEvent::Json(json!({- "type": "response.completed",- "response": {- "usage": {- "input_tokens": 8,- "output_tokens": 13,- "input_tokens_details": {"cached_tokens": 5}- }- }- })),- ];- let stream_events = events.clone();- let stream = futures_util::stream::iter(stream_events.into_iter().map(Ok));- let response = ChatResponse::OpenAiResponsesStream(Box::pin(stream));-- let processed = processor.process(&mut ctx, response).await?;- let drained = drain_responses_stream(processed).await?;-- assert_eq!(drained, events);- let snapshot = accumulator.snapshot()?;- let model = model_stats(&snapshot, "responses-model")?;- assert_eq!(model.prompt_tokens, 8);- assert_eq!(model.completion_tokens, 13);- assert_eq!(model.cached_tokens, 5);- assert_eq!(model.total_latency.count, 1);- Ok(())-}--// Raw SSE frame strings (verbatim Responses passthrough) still record usage.-#[tokio::test]-async fn raw_sse_frame_responses_stream_passes_through_and_records_usage() -> Result<()> {- let accumulator = StatsAccumulator::new();- let processor = StatsResponseProcessor::new(accumulator.clone());- let mut ctx = ProxyContext::new();- record_backend_selection(&mut ctx, ModelId::new("raw-frame-model")?);- ctx.insert(StatsRequestStart::now());-- let events = vec![- StreamEvent::Text(- "event: response.output_text.delta\n\- data: {\"type\":\"response.output_text.delta\",\"delta\":\"hi\"}\n\n"- .to_string(),- ),- StreamEvent::Text(": keep-alive\n\n".to_string()),- StreamEvent::Text(- "event: response.completed\n\- data: {\"type\":\"response.completed\",\"response\":{\"usage\":\- {\"input_tokens\":8,\"output_tokens\":13,\- \"input_tokens_details\":{\"cached_tokens\":5}}}}\n\n"- .to_string(),- ),- ];- let stream_events = events.clone();- let stream = futures_util::stream::iter(stream_events.into_iter().map(Ok));- let response = ChatResponse::OpenAiResponsesStream(Box::pin(stream));-- let processed = processor.process(&mut ctx, response).await?;- let drained = drain_responses_stream(processed).await?;-- // Frames pass through byte-identical; usage still lands in the snapshot.- assert_eq!(drained, events);- let snapshot = accumulator.snapshot()?;- let model = model_stats(&snapshot, "raw-frame-model")?;- assert_eq!(model.prompt_tokens, 8);- assert_eq!(model.completion_tokens, 13);- assert_eq!(model.cached_tokens, 5);- Ok(())-}--// OpenAI streams should record the first real usage block only.-#[tokio::test]-async fn openai_chat_stream_records_first_usage_chunk_only_with_details() -> Result<()> {- let accumulator = StatsAccumulator::new();- let processor = StatsResponseProcessor::new(accumulator.clone());- let mut ctx = ProxyContext::new();- record_backend_selection(&mut ctx, ModelId::new("chat-stream-model")?);-- let events = vec![- StreamEvent::Json(json!({"usage": null})),- StreamEvent::Json(json!({"choices": [{"delta": {"content": "hi"}}]})),- StreamEvent::Json(json!({- "usage": {- "prompt_tokens": 20,- "completion_tokens": 7,- "prompt_tokens_details": {"cached_tokens": 4},- "completion_tokens_details": {"reasoning_tokens": 2}- }- })),- StreamEvent::Json(json!({- "usage": {"prompt_tokens": 200, "completion_tokens": 70}- })),- ];- let stream = futures_util::stream::iter(events.clone().into_iter().map(Ok));- let processed = processor- .process(&mut ctx, ChatResponse::OpenAiStream(Box::pin(stream)))- .await?;- let drained = drain_openai_stream(processed).await?;-- assert_eq!(drained, events);- let snapshot = accumulator.snapshot()?;- let model = model_stats(&snapshot, "chat-stream-model")?;- assert_eq!(model.prompt_tokens, 20);- assert_eq!(model.completion_tokens, 7);- assert_eq!(model.cached_tokens, 4);- assert_eq!(model.reasoning_tokens, 2);- assert_eq!(model.total_latency.count, 0);- Ok(())-}--// Anthropic streaming usage can arrive in start and delta events.-#[tokio::test]-async fn anthropic_stream_merges_start_and_delta_usage_and_commits_once_at_stop() -> Result<()> {- let accumulator = StatsAccumulator::new();- let processor = StatsResponseProcessor::new(accumulator.clone());- let mut ctx = ProxyContext::new();- record_backend_selection(&mut ctx, ModelId::new("claude-stream-model")?);- ctx.insert(StatsRequestStart::now());-- let events = vec![- StreamEvent::Json(json!({- "type": "message_start",- "message": {- "usage": {- "input_tokens": 50,- "cache_read_input_tokens": 10,- "cache_creation_input_tokens": 5- }- }- })),- StreamEvent::Json(json!({- "type": "message_delta",- "usage": {"output_tokens": 20}- })),- StreamEvent::Json(json!({"type": "message_stop"})),- StreamEvent::Json(json!({"type": "message_stop"})),- ];- let stream = futures_util::stream::iter(events.clone().into_iter().map(Ok));- let processed = processor- .process(&mut ctx, ChatResponse::AnthropicStream(Box::pin(stream)))- .await?;- let drained = drain_anthropic_stream(processed).await?;-- assert_eq!(drained, events);- let snapshot = accumulator.snapshot()?;- let model = model_stats(&snapshot, "claude-stream-model")?;- assert_eq!(model.prompt_tokens, 65);- assert_eq!(model.completion_tokens, 20);- assert_eq!(model.cached_tokens, 10);- assert_eq!(model.cache_creation_tokens, 5);- assert_eq!(model.total_latency.count, 1);- Ok(())-}--// Later Anthropic input-token deltas should override a zero start value.-#[tokio::test]-async fn anthropic_stream_delta_input_tokens_override_zero_start_tokens() -> Result<()> {- let accumulator = StatsAccumulator::new();- let processor = StatsResponseProcessor::new(accumulator.clone());- let mut ctx = ProxyContext::new();- record_backend_selection(&mut ctx, ModelId::new("claude-delta-input")?);-- let events = vec![- StreamEvent::Json(json!({- "type": "message_start",- "message": {"usage": {"input_tokens": 0}}- })),- StreamEvent::Json(json!({- "type": "message_delta",- "usage": {"input_tokens": 75, "output_tokens": 30}- })),- StreamEvent::Json(json!({"type": "message_stop"})),- ];- let stream = futures_util::stream::iter(events.into_iter().map(Ok));- let processed = processor- .process(&mut ctx, ChatResponse::AnthropicStream(Box::pin(stream)))- .await?;- let _ = drain_anthropic_stream(processed).await?;-- let snapshot = accumulator.snapshot()?;- let model = model_stats(&snapshot, "claude-delta-input")?;- assert_eq!(model.prompt_tokens, 75);- assert_eq!(model.completion_tokens, 30);- Ok(())-}--// Streams with no usage should remain transparent and avoid fake stats entries.-#[tokio::test]-async fn stream_without_usage_passes_through_without_creating_stats_entry() -> Result<()> {- let accumulator = StatsAccumulator::new();- let processor = StatsResponseProcessor::new(accumulator.clone());- let mut ctx = ProxyContext::new();- record_backend_selection(&mut ctx, ModelId::new("no-usage-stream")?);-- let events = vec![- StreamEvent::Json(json!({"choices": [{"delta": {"content": "a"}}]})),- StreamEvent::Text("plain text ignored".to_string()),- ];- let stream = futures_util::stream::iter(events.clone().into_iter().map(Ok));- let processed = processor- .process(&mut ctx, ChatResponse::OpenAiStream(Box::pin(stream)))- .await?;-- assert_eq!(drain_openai_stream(processed).await?, events);- assert!(accumulator.snapshot()?.models.is_empty());- Ok(())-}--// Buffered responses without usage still create a zero-token model entry.-#[tokio::test]-async fn buffered_response_without_usage_records_zero_token_model_entry() -> Result<()> {- let accumulator = StatsAccumulator::new();- let processor = StatsResponseProcessor::new(accumulator.clone());- let mut ctx = ProxyContext::new();- record_backend_selection(&mut ctx, ModelId::new("missing-usage-model")?);-- processor- .process(- &mut ctx,- ChatResponse::openai_completion(json!({"id": "no-usage"})),- )- .await?;-- let snapshot = accumulator.snapshot()?;- let model = model_stats(&snapshot, "missing-usage-model")?;- assert_eq!(model.prompt_tokens, 0);- assert_eq!(model.completion_tokens, 0);- assert_eq!(snapshot.total_tokens.total, 0);- Ok(())-}--// Malformed usage fields should be ignored without losing valid alternatives.-#[tokio::test]-async fn malformed_usage_values_are_ignored_without_wrapping_or_panicking() -> Result<()> {- let accumulator = StatsAccumulator::new();- let processor = StatsResponseProcessor::new(accumulator.clone());- let mut ctx = ProxyContext::new();- record_backend_selection(&mut ctx, ModelId::new("malformed-usage-model")?);-- processor- .process(- &mut ctx,- ChatResponse::openai_completion(json!({- "usage": {- "prompt_tokens": -1,- "completion_tokens": "bad",- "input_tokens": 9,- "output_tokens": true,- "prompt_tokens_details": {"cached_tokens": null},- "completion_tokens_details": {"reasoning_tokens": 3.5}- }- })),- )- .await?;-- let snapshot = accumulator.snapshot()?;- let model = model_stats(&snapshot, "malformed-usage-model")?;- assert_eq!(model.prompt_tokens, 9);- assert_eq!(model.completion_tokens, 0);- assert_eq!(model.cached_tokens, 0);- assert_eq!(model.reasoning_tokens, 0);- Ok(())-}--// Random-routing context should populate tier rollups in response stats.-#[tokio::test]-async fn response_processor_uses_random_routing_decision_for_tier_rollups() -> Result<()> {- let accumulator = StatsAccumulator::new();- let processor = StatsResponseProcessor::new(accumulator.clone());- let mut ctx = ProxyContext::new();- record_backend_selection(&mut ctx, ModelId::new("weak-model")?);- ctx.insert(RandomRoutingDecision {- tier: RandomRoutingTier::Weak,- selected_target: LlmTargetId::from_static("weak"),- selected_model: ModelId::new("weak-model")?,- original_model: Some("client-model".to_string()),- strong_probability: 0.25,- draw: 0.75,- });-- processor- .process(- &mut ctx,- ChatResponse::openai_completion(json!({- "usage": {"prompt_tokens": 4, "completion_tokens": 9}- })),- )- .await?;-- let snapshot = accumulator.snapshot()?;- let tier = snapshot- .tiers- .get("weak")- .ok_or_else(|| SwitchyardError::Other("weak tier should be present".to_string()))?;- assert_eq!(tier.model, "weak-model");- assert_eq!(tier.prompt_tokens, 4);- assert_eq!(tier.completion_tokens, 9);- assert_eq!(tier.total_tokens, 13);- Ok(())-}--// Context-selected targets should be a generic tier fallback when no richer-// router-specific label exists.-#[tokio::test]-async fn response_processor_uses_selected_target_as_tier_fallback() -> Result<()> {- let accumulator = StatsAccumulator::new();- let processor = StatsResponseProcessor::new(accumulator.clone());- let mut ctx = ProxyContext::new();- record_backend_selection(&mut ctx, ModelId::new("primary-model")?);- ctx.set_selected_target(LlmTargetId::from_static("primary"));-- processor- .process(- &mut ctx,- ChatResponse::openai_completion(json!({- "usage": {"prompt_tokens": 3, "completion_tokens": 5}- })),- )- .await?;-- let snapshot = accumulator.snapshot()?;- let tier = snapshot- .tiers- .get("primary")- .ok_or_else(|| SwitchyardError::Other("primary tier should be present".to_string()))?;- assert_eq!(tier.model, "primary-model");- assert_eq!(tier.prompt_tokens, 3);- assert_eq!(tier.completion_tokens, 5);- Ok(())-}--// Explicit stats route labels intentionally override random-routing labels.-#[tokio::test]-async fn explicit_stats_route_label_takes_precedence_over_random_routing_decision() -> Result<()> {- let accumulator = StatsAccumulator::new();- let processor = StatsResponseProcessor::new(accumulator.clone());- let mut ctx = ProxyContext::new();- record_backend_selection(&mut ctx, ModelId::new("labeled-model")?);- ctx.insert(StatsRouteLabel::new("plugin"));- ctx.insert(RandomRoutingDecision {- tier: RandomRoutingTier::Weak,- selected_target: LlmTargetId::from_static("weak"),- selected_model: ModelId::new("labeled-model")?,- original_model: None,- strong_probability: 0.25,- draw: 0.75,- });-- processor- .process(- &mut ctx,- ChatResponse::openai_completion(json!({- "usage": {"prompt_tokens": 1, "completion_tokens": 1}- })),- )- .await?;-- let snapshot = accumulator.snapshot()?;- let model = model_stats(&snapshot, "labeled-model")?;- assert_eq!(model.tier.as_deref(), Some("plugin"));- let plugin = snapshot- .tiers- .get("plugin")- .ok_or_else(|| SwitchyardError::Other("plugin tier should be present".to_string()))?;- assert_eq!(plugin.model, "labeled-model");- assert_eq!(plugin.calls, 0);- assert_eq!(plugin.prompt_tokens, 1);- assert_eq!(plugin.completion_tokens, 1);- assert!(!snapshot.tiers.contains_key("weak"));- Ok(())-}--// Drains an OpenAI Responses stream after the stats wrapper has observed it.-async fn drain_responses_stream(response: ChatResponse) -> Result<Vec<StreamEvent>> {- let ChatResponse::OpenAiResponsesStream(mut stream) = response else {- return Err(SwitchyardError::Other(- "response should remain an OpenAI Responses stream".to_string(),- ));- };- let mut events = Vec::new();- while let Some(event) = stream.next().await {- events.push(event?);- }- Ok(events)-}--// Drains an OpenAI Chat stream after the stats wrapper has observed it.-async fn drain_openai_stream(response: ChatResponse) -> Result<Vec<StreamEvent>> {- let ChatResponse::OpenAiStream(mut stream) = response else {- return Err(SwitchyardError::Other(- "response should remain an OpenAI stream".to_string(),- ));- };- let mut events = Vec::new();- while let Some(event) = stream.next().await {- events.push(event?);- }- Ok(events)-}--// Drains an Anthropic stream after the stats wrapper has observed it.-async fn drain_anthropic_stream(response: ChatResponse) -> Result<Vec<StreamEvent>> {- let ChatResponse::AnthropicStream(mut stream) = response else {- return Err(SwitchyardError::Other(- "response should remain an Anthropic stream".to_string(),- ));- };- let mut events = Vec::new();- while let Some(event) = stream.next().await {- events.push(event?);- }- Ok(events)-}--// Fetches one model stats block with an explicit test error on absence.-fn model_stats<'a>(- snapshot: &'a switchyard_components::StatsSnapshot,- model: &str,-) -> Result<&'a switchyard_components::ModelStatsSnapshot> {- snapshot- .models- .get(model)- .ok_or_else(|| SwitchyardError::Other(format!("model stats missing for {model}")))-}diff --git a/crates/switchyard-components/tests/stats_usage_shapes.rs b/crates/switchyard-components/tests/stats_usage_shapes.rsdeleted file mode 100644index d25f983b1..000000000--- a/crates/switchyard-components/tests/stats_usage_shapes.rs+++ /dev/null@@ -1,241 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Parity tests for the OpenAI vs Anthropic usage shapes.-//!-//! OpenAI exposes `prompt_tokens` as the *inclusive* total with-//! `prompt_tokens_details.{cached_tokens, cache_creation_tokens}` as nested-//! subsets. Anthropic exposes `input_tokens` (non-cached base),-//! `cache_read_input_tokens`, and `cache_creation_input_tokens` as-//! *sibling* fields — each disjoint from the others. The extractor in-//! `switchyard_components::stats::usage_from_body` must normalise both-//! into the OpenAI-style inclusive convention so the downstream cost-//! estimator (`base = prompt - cached - cache_creation`) never goes-//! negative and the cache buckets sum into the prompt total. The tests-//! below pin that contract.-//!-//! Replaces the Python `tests/test_cost_estimator_cache_wiring.py` Layer 1-//! cases, which targeted a `_record()` helper that the Rust migration-//! eliminated.--use serde_json::json;-use switchyard_components::TokenUsage;-use switchyard_components::stats::usage_from_body;--#[test]-fn openai_inclusive_shape_extracts_prompt_as_inclusive_total() {- // OpenAI: `prompt_tokens` already counts the cached + cache_creation tokens.- let body = json!({- "usage": {- "prompt_tokens": 550,- "completion_tokens": 100,- "prompt_tokens_details": {- "cached_tokens": 100,- "cache_creation_tokens": 50,- },- }- });- assert_eq!(- usage_from_body(&body),- TokenUsage {- prompt_tokens: 550,- completion_tokens: 100,- cached_tokens: 100,- cache_creation_tokens: 50,- reasoning_tokens: 0,- cacheable_prompt_tokens: 0,- }- );-}--#[test]-fn anthropic_sibling_shape_sums_into_inclusive_prompt_total() {- // Anthropic: `input_tokens` is the BASE (non-cached, non-creation) —- // `cache_read_input_tokens` and `cache_creation_input_tokens` are siblings.- // The extractor must sum the three to produce the inclusive total.- let body = json!({- "usage": {- "input_tokens": 400,- "output_tokens": 100,- "cache_read_input_tokens": 100,- "cache_creation_input_tokens": 50,- }- });- assert_eq!(- usage_from_body(&body),- TokenUsage {- prompt_tokens: 550,- completion_tokens: 100,- cached_tokens: 100,- cache_creation_tokens: 50,- reasoning_tokens: 0,- cacheable_prompt_tokens: 0,- }- );-}--#[test]-fn openai_and_anthropic_shapes_produce_identical_canonical_output() {- // Same logical request expressed in both shapes — extractor output- // must be byte-for-byte identical so cost math doesn't drift between- // providers.- let openai_body = json!({- "usage": {- "prompt_tokens": 550,- "completion_tokens": 100,- "prompt_tokens_details": {- "cached_tokens": 100,- "cache_creation_tokens": 50,- },- }- });- let anthropic_body = json!({- "usage": {- "input_tokens": 400,- "output_tokens": 100,- "cache_read_input_tokens": 100,- "cache_creation_input_tokens": 50,- }- });- assert_eq!(- usage_from_body(&openai_body),- usage_from_body(&anthropic_body)- );-}--#[test]-fn anthropic_without_cache_fields_keeps_prompt_equal_to_input() {- // No cache fields: prompt_tokens degenerates to input_tokens.- let body = json!({- "usage": {- "input_tokens": 200,- "output_tokens": 50,- }- });- assert_eq!(- usage_from_body(&body),- TokenUsage {- prompt_tokens: 200,- completion_tokens: 50,- cached_tokens: 0,- cache_creation_tokens: 0,- reasoning_tokens: 0,- cacheable_prompt_tokens: 0,- }- );-}--#[test]-fn openai_without_prompt_tokens_details_keeps_cache_counts_zero() {- let body = json!({- "usage": {- "prompt_tokens": 200,- "completion_tokens": 50,- }- });- assert_eq!(- usage_from_body(&body),- TokenUsage {- prompt_tokens: 200,- completion_tokens: 50,- cached_tokens: 0,- cache_creation_tokens: 0,- reasoning_tokens: 0,- cacheable_prompt_tokens: 0,- }- );-}--#[test]-fn openai_reasoning_tokens_extracted_from_completion_tokens_details() {- let body = json!({- "usage": {- "prompt_tokens": 100,- "completion_tokens": 200,- "completion_tokens_details": {- "reasoning_tokens": 150,- },- }- });- let usage = usage_from_body(&body);- assert_eq!(usage.reasoning_tokens, 150);- assert_eq!(usage.completion_tokens, 200);-}--#[test]-fn anthropic_reasoning_tokens_extracted_from_output_tokens_details() {- let body = json!({- "usage": {- "input_tokens": 100,- "output_tokens": 200,- "output_tokens_details": {- "reasoning_tokens": 150,- },- }- });- let usage = usage_from_body(&body);- assert_eq!(usage.reasoning_tokens, 150);- assert_eq!(usage.completion_tokens, 200);-}--#[test]-fn missing_usage_block_yields_zero_usage() {- let body = json!({"id": "msg-1", "content": "hi"});- assert!(usage_from_body(&body).is_zero());-}--#[test]-fn non_object_usage_block_is_ignored() {- // Defensive: a malformed `usage: "garbage"` must not panic; extractor- // returns default rather than half-populating fields.- let body = json!({"usage": "garbage"});- assert!(usage_from_body(&body).is_zero());-}--#[test]-fn anthropic_with_input_tokens_details_cached_takes_precedence_over_cache_read() {- // Defensive corner: when both `input_tokens_details.cached_tokens` and- // `cache_read_input_tokens` are present, the details object wins so- // providers that surface both naming conventions don't double-count.- let body = json!({- "usage": {- "input_tokens": 100,- "output_tokens": 50,- "cache_read_input_tokens": 30,- "input_tokens_details": {"cached_tokens": 25},- }- });- let usage = usage_from_body(&body);- assert_eq!(usage.cached_tokens, 25);-}--#[test]-fn cost_estimator_invariant_no_negative_base_input() {- // The cost estimator computes- // base = prompt_tokens - cached_tokens - cache_creation_tokens- // and relies on this being non-negative. Both shapes must respect that- // — otherwise the base-input cost line silently clamps to 0 and we- // under-bill the cache-write cost.- let cases = [- json!({"usage": {- "prompt_tokens": 550,- "prompt_tokens_details": {"cached_tokens": 100, "cache_creation_tokens": 50},- }}),- json!({"usage": {- "input_tokens": 400,- "cache_read_input_tokens": 100,- "cache_creation_input_tokens": 50,- }}),- ];- for body in cases {- let usage = usage_from_body(&body);- let base = usage- .prompt_tokens- .saturating_sub(usage.cached_tokens)- .saturating_sub(usage.cache_creation_tokens);- assert_eq!(- base, 400,- "{body:?} should leave 400 non-cached prompt tokens"- );- }-}diff --git a/crates/switchyard-components/tests/support/config.rs b/crates/switchyard-components/tests/support/config.rsdeleted file mode 100644index 404873d3b..000000000--- a/crates/switchyard-components/tests/support/config.rs+++ /dev/null@@ -1,67 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Shared test helpers that do not depend on the removed config graph.--use std::collections::BTreeMap;--use switchyard_components::{- BackendFormat, EndpointConfig, LlmTarget, LlmTargetId, ModelId, Result, SwitchyardError,-};--/// Test result alias that works across async and blocking integration tests.-pub type TestResult = std::result::Result<(), Box<dyn std::error::Error + Send + Sync>>;--/// Asserts Switchyard validation fails with a stable, useful error fragment.-pub fn assert_invalid<T>(result: Result<T>, message: &'static str, expected: &str) -> TestResult {- let error = error_from(result, message);- assert!(- error.to_string().contains(expected),- "expected {error:?} to contain {expected:?}"- );- Ok(())-}--/// Builds an OpenAI-compatible runtime target for component tests.-pub fn openai_target(id: &'static str, model: &'static str, base_url: &str) -> Result<LlmTarget> {- Ok(LlmTarget {- id: LlmTargetId::from_static(id),- model: ModelId::from_static(model),- format: BackendFormat::OpenAi,- endpoint: EndpointConfig {- base_url: Some(base_url.to_string()),- api_key: Some("test-key".to_string()),- timeout_secs: Some(5.0),- },- extra_body: None,- extra_headers: BTreeMap::new(),- })-}--/// Builds an Anthropic-compatible runtime target for component tests.-pub fn anthropic_target(- id: &'static str,- model: &'static str,- base_url: &str,-) -> Result<LlmTarget> {- Ok(LlmTarget {- id: LlmTargetId::from_static(id),- model: ModelId::from_static(model),- format: BackendFormat::Anthropic,- endpoint: EndpointConfig {- base_url: Some(base_url.to_string()),- api_key: Some("test-key".to_string()),- timeout_secs: Some(5.0),- },- extra_body: None,- extra_headers: BTreeMap::new(),- })-}--// Extracts the error from a result that is expected to fail.-fn error_from<T>(result: Result<T>, message: &'static str) -> SwitchyardError {- match result {- Ok(_) => panic!("{message}"),- Err(error) => error,- }-}diff --git a/crates/switchyard-components/tests/support/mod.rs b/crates/switchyard-components/tests/support/mod.rsdeleted file mode 100644index 6a0744c89..000000000--- a/crates/switchyard-components/tests/support/mod.rs+++ /dev/null@@ -1,289 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0-#![allow(dead_code)]--//! Lightweight HTTP test servers shared by backend and config tests.--use std::collections::BTreeMap;-use std::io::{Read, Write};-use std::net::TcpListener;-use std::sync::mpsc::{self, Receiver, RecvTimeoutError};-use std::thread::{self, JoinHandle};-use std::time::Duration;--use serde_json::Value;-use switchyard_components::{Result, SwitchyardError};--pub mod config;--/// One HTTP request captured by a local mock server.-#[derive(Debug)]-pub struct CapturedRequest {- /// Request method.- pub method: String,- /// Request path including query.- pub path: String,- /// Lowercased request headers.- pub headers: BTreeMap<String, String>,- /// JSON request body.- pub body: Value,-}--impl CapturedRequest {- /// Returns a header value using case-insensitive matching.- pub fn header(&self, name: &str) -> Option<&str> {- self.headers- .get(&name.to_ascii_lowercase())- .map(String::as_str)- }-}--/// Mock server that accepts exactly one request.-pub struct OneShotServer {- /// Base URL clients can call.- base_url: String,- /// Captured request result from the server thread.- receiver: Receiver<Result<CapturedRequest>>,- /// Server thread handle joined when captured output is read.- handle: Option<JoinHandle<()>>,-}--/// Mock server that accepts a fixed sequence of requests.-pub struct SequenceServer {- /// Base URL clients can call.- base_url: String,- /// Captured request sequence from the server thread.- receiver: Receiver<Result<Vec<CapturedRequest>>>,- /// Server thread handle joined when captured output is read.- handle: Option<JoinHandle<()>>,-}--impl OneShotServer {- /// Creates a one-shot server returning JSON.- pub fn json(status: u16, body: Value) -> Result<Self> {- Self::raw(status, "application/json", body.to_string())- }-- /// Creates a one-shot server returning raw SSE.- #[allow(dead_code)]- pub fn sse(body: impl Into<String>) -> Result<Self> {- Self::raw(200, "text/event-stream", body.into())- }-- /// Returns the base URL for this server.- pub fn base_url(&self) -> &str {- &self.base_url- }-- /// Waits for and returns the captured request.- pub fn captured(mut self) -> Result<CapturedRequest> {- let request = self.receiver.recv_timeout(Duration::from_secs(5));- match &request {- Ok(_) | Err(RecvTimeoutError::Disconnected) => {- if let Some(handle) = self.handle.take()- && handle.join().is_err()- {- return Err(SwitchyardError::Other("server thread panicked".to_string()));- }- }- Err(RecvTimeoutError::Timeout) => {}- }- request.map_err(|error| {- SwitchyardError::Other(format!("server should capture one request: {error}"))- })?- }-- /// Creates a one-shot server returning an arbitrary content type and body.- fn raw(status: u16, content_type: &'static str, body: String) -> Result<Self> {- let listener = TcpListener::bind("127.0.0.1:0")- .map_err(|error| SwitchyardError::Other(format!("bind test server: {error}")))?;- let address = listener.local_addr().map_err(|error| {- SwitchyardError::Other(format!("read test server address: {error}"))- })?;- let base_url = format!("http://{address}");- let (sender, receiver) = mpsc::channel();- let handle = thread::spawn(move || {- let result = handle_one_request(listener, status, content_type, body);- let _ignored = sender.send(result);- });-- Ok(Self {- base_url,- receiver,- handle: Some(handle),- })- }-}--impl SequenceServer {- /// Creates a sequence server returning one JSON response per request.- pub fn json(responses: Vec<(u16, Value)>) -> Result<Self> {- let listener = TcpListener::bind("127.0.0.1:0")- .map_err(|error| SwitchyardError::Other(format!("bind test server: {error}")))?;- let address = listener.local_addr().map_err(|error| {- SwitchyardError::Other(format!("read test server address: {error}"))- })?;- let base_url = format!("http://{address}");- let (sender, receiver) = mpsc::channel();- let handle = thread::spawn(move || {- let result = handle_request_sequence(listener, responses);- let _ignored = sender.send(result);- });-- Ok(Self {- base_url,- receiver,- handle: Some(handle),- })- }-- /// Returns the base URL for this server.- pub fn base_url(&self) -> &str {- &self.base_url- }-- /// Waits for and returns every captured request.- pub fn captured(mut self) -> Result<Vec<CapturedRequest>> {- let requests = self.receiver.recv_timeout(Duration::from_secs(5));- match &requests {- Ok(_) | Err(RecvTimeoutError::Disconnected) => {- if let Some(handle) = self.handle.take()- && handle.join().is_err()- {- return Err(SwitchyardError::Other("server thread panicked".to_string()));- }- }- Err(RecvTimeoutError::Timeout) => {}- }- requests.map_err(|error| {- SwitchyardError::Other(format!("server should capture requests: {error}"))- })?- }-}--/// Handles one HTTP request and writes a fixed response.-fn handle_one_request(- listener: TcpListener,- status: u16,- content_type: &'static str,- body: String,-) -> Result<CapturedRequest> {- let (mut stream, _) = listener- .accept()- .map_err(|error| SwitchyardError::Other(format!("accept one request: {error}")))?;- let request = read_request(&mut stream)?;-- let response = format!(- "HTTP/1.1 {status} OK\r\ncontent-type: {content_type}\r\ncontent-length: {}\r\nconnection: close\r\n\r\n{body}",- body.len()- );- stream- .write_all(response.as_bytes())- .map_err(|error| SwitchyardError::Other(format!("write test response: {error}")))?;- Ok(request)-}--/// Handles a fixed sequence of HTTP requests with matching JSON responses.-fn handle_request_sequence(- listener: TcpListener,- responses: Vec<(u16, Value)>,-) -> Result<Vec<CapturedRequest>> {- let mut requests = Vec::with_capacity(responses.len());- for (status, body) in responses {- requests.push(handle_one_request(- listener- .try_clone()- .map_err(|error| SwitchyardError::Other(format!("clone test listener: {error}")))?,- status,- "application/json",- body.to_string(),- )?);- }- Ok(requests)-}--/// Reads one HTTP/1.1 request from a blocking test socket.-fn read_request(stream: &mut std::net::TcpStream) -> Result<CapturedRequest> {- let mut bytes = Vec::new();- let mut chunk = [0_u8; 4096];- loop {- let read = stream- .read(&mut chunk)- .map_err(|error| SwitchyardError::Other(format!("read request: {error}")))?;- if read == 0 {- break;- }- bytes.extend_from_slice(&chunk[..read]);- if let Some((header_end, content_length)) = request_shape(&bytes) {- let body_start = header_end + 4;- if bytes.len() >= body_start + content_length {- break;- }- }- }-- let (header_end, content_length) = request_shape(&bytes)- .ok_or_else(|| SwitchyardError::Other("request should contain HTTP headers".to_string()))?;- let header_text = std::str::from_utf8(&bytes[..header_end]).map_err(|error| {- SwitchyardError::Other(format!("headers should be valid UTF-8: {error}"))- })?;- let body_start = header_end + 4;- let body_end = body_start + content_length;- let raw_body = std::str::from_utf8(&bytes[body_start..body_end])- .map_err(|error| SwitchyardError::Other(format!("body should be valid UTF-8: {error}")))?- .to_string();-- let mut lines = header_text.lines();- let start_line = lines- .next()- .ok_or_else(|| SwitchyardError::Other("request line".to_string()))?;- let mut start_parts = start_line.split_whitespace();- let method = start_parts.next().unwrap_or_default().to_string();- let path = start_parts.next().unwrap_or_default().to_string();-- let mut headers = BTreeMap::new();- for line in lines {- let Some((name, value)) = line.split_once(':') else {- continue;- };- headers.insert(name.to_ascii_lowercase(), value.trim().to_string());- }-- let body = if raw_body.is_empty() {- Value::Null- } else {- serde_json::from_str(&raw_body).map_err(|error| {- SwitchyardError::Other(format!("request body should be JSON: {error}"))- })?- };-- Ok(CapturedRequest {- method,- path,- headers,- body,- })-}--/// Returns the header/body split and content length once headers are complete.-fn request_shape(bytes: &[u8]) -> Option<(usize, usize)> {- let header_end = find_bytes(bytes, b"\r\n\r\n")?;- let headers = std::str::from_utf8(&bytes[..header_end]).ok()?;- let content_length = headers- .lines()- .filter_map(|line| line.split_once(':'))- .find_map(|(name, value)| {- name.eq_ignore_ascii_case("content-length")- .then(|| value.trim().parse::<usize>().ok())- .flatten()- })- .unwrap_or(0);- Some((header_end, content_length))-}--/// Finds a byte sequence without pulling in memchr for tests.-fn find_bytes(haystack: &[u8], needle: &[u8]) -> Option<usize> {- haystack- .windows(needle.len())- .position(|window| window == needle)-}diff --git a/crates/switchyard-py/Cargo.toml b/crates/switchyard-py/Cargo.tomlindex ceaa7cf7f..b2b0b0c22 100644--- a/crates/switchyard-py/Cargo.toml+++ b/crates/switchyard-py/Cargo.toml@@ -17,9 +17,7 @@ crate-type = ["cdylib", "rlib"][dependencies]async-trait.workspace = true-futures-util.workspace = truehttp.workspace = true-parking_lot.workspace = trueswitchyard-libsy.workspace = trueswitchyard-llm-client.workspace = truepyo3 = { version = "0.28.3", features = ["abi3-py312", "extension-module"] }@@ -29,7 +27,4 @@ serde.workspace = trueserde_json.workspace = trueswitchyard-protocol.workspace = trueswitchyard-server.workspace = true-switchyard-components.workspace = true-switchyard-translation.workspace = truetokio.workspace = true-tracing.workspace = truediff --git a/crates/switchyard-py/src/component_bindings.rs b/crates/switchyard-py/src/component_bindings.rsdeleted file mode 100644index 43a672805..000000000--- a/crates/switchyard-py/src/component_bindings.rs+++ /dev/null@@ -1,25 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! PyO3 bindings for concrete `switchyard-components` implementations.--use pyo3::prelude::*;--mod backends;-pub(crate) mod config;-mod dimension_collector;-mod request_processors;-mod response_processors;-mod stage_router;-pub(crate) mod stats;--pub(crate) fn register(module: &Bound<'_, PyModule>) -> PyResult<()> {- config::register(module)?;- stats::register(module)?;- request_processors::register(module)?;- response_processors::register(module)?;- backends::register(module)?;- dimension_collector::register(module)?;- stage_router::register(module)?;- Ok(())-}diff --git a/crates/switchyard-py/src/component_bindings/backends.rs b/crates/switchyard-py/src/component_bindings/backends.rsdeleted file mode 100644index bf3285ed9..000000000--- a/crates/switchyard-py/src/component_bindings/backends.rs+++ /dev/null@@ -1,350 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Python bindings for concrete LLM backends.--use std::sync::Arc;--use pyo3::exceptions::{PyTypeError, PyValueError};-use pyo3::prelude::*;-use pyo3::types::{PyIterator, PyTuple};-use switchyard_components::{- AnthropicNativeBackend, LlmTargetBackend, MultiLlmBackend, OpenAiNativeBackend,- OpenAiPassthroughBackend, StatsLlmBackend,-};-use switchyard_components::{ChatRequestType, EndpointConfig, LlmBackend, LlmTargetId};--use super::config::{PyEndpointConfig, PyLlmTarget, endpoint_config_from_python};-use super::stats::PyStatsAccumulator;-use crate::errors::py_core_error;-use crate::interop::request::request_type_from_python;-use crate::interop::roles::PyLlmBackend;--#[pyclass(name = "LlmTargetBackend", frozen, skip_from_py_object)]-#[derive(Clone, Debug)]-pub(crate) struct PyLlmTargetBackend {- inner: LlmTargetBackend,-}--impl PyLlmTargetBackend {- fn clone_core(&self) -> LlmTargetBackend {- self.inner.clone()- }-}--#[pymethods]-impl PyLlmTargetBackend {- #[new]- fn py_new(target: PyRef<'_, PyLlmTarget>, backend: PyRef<'_, PyLlmBackend>) -> PyResult<Self> {- Ok(Self {- inner: LlmTargetBackend::new(- target.clone_core(),- native_backend(&backend, "LlmTargetBackend")?,- ),- })- }-- #[getter]- fn target(&self) -> PyLlmTarget {- PyLlmTarget::from_core(self.inner.target().clone())- }-- fn __repr__(&self) -> String {- format!(- "LlmTargetBackend(target_id={:?}, model={:?})",- self.inner.target().id.as_str(),- self.inner.target().model.as_str(),- )- }-}--#[pyclass(- name = "OpenAiNativeBackend",- extends = PyLlmBackend,- skip_from_py_object-)]-#[derive(Clone, Debug)]-pub(crate) struct PyOpenAiNativeBackend {- inner: Arc<OpenAiNativeBackend>,-}--#[pymethods]-impl PyOpenAiNativeBackend {- #[new]- fn py_new(target: PyRef<'_, PyLlmTarget>) -> PyResult<PyClassInitializer<Self>> {- let backend =- Arc::new(OpenAiNativeBackend::new(target.clone_core()).map_err(py_core_error)?);- let base: Arc<dyn LlmBackend> = backend.clone();- Ok(PyClassInitializer::from(PyLlmBackend::from_native(base))- .add_subclass(Self { inner: backend }))- }-- #[getter]- fn target(&self) -> PyLlmTarget {- PyLlmTarget::from_core(self.inner.target().clone())- }-- fn __repr__(&self) -> String {- format!(- "OpenAiNativeBackend(target_id={:?}, model={:?})",- self.inner.target().id.as_str(),- self.inner.target().model.as_str(),- )- }-}--#[pyclass(- name = "OpenAiPassthroughBackend",- extends = PyLlmBackend,- skip_from_py_object-)]-#[derive(Clone, Debug)]-pub(crate) struct PyOpenAiPassthroughBackend {- inner: Arc<OpenAiPassthroughBackend>,-}--#[pymethods]-impl PyOpenAiPassthroughBackend {- #[new]- #[pyo3(signature = (endpoint=None, api_key=None, base_url=None, timeout_secs=None, timeout=None))]- fn py_new(- endpoint: Option<&Bound<'_, PyAny>>,- api_key: Option<String>,- base_url: Option<String>,- timeout_secs: Option<f64>,- timeout: Option<f64>,- ) -> PyResult<PyClassInitializer<Self>> {- let mut endpoint_config = endpoint_config_from_python(endpoint)?;- if api_key.is_some() {- endpoint_config.api_key = api_key;- }- if base_url.is_some() {- endpoint_config.base_url = base_url;- }- if timeout_secs.is_some() {- endpoint_config.timeout_secs = timeout_secs;- }- if timeout.is_some() {- endpoint_config.timeout_secs = timeout;- }-- let backend =- Arc::new(OpenAiPassthroughBackend::new(endpoint_config).map_err(py_core_error)?);- let base: Arc<dyn LlmBackend> = backend.clone();- Ok(PyClassInitializer::from(PyLlmBackend::from_native(base))- .add_subclass(Self { inner: backend }))- }-- #[getter]- fn endpoint(&self) -> PyEndpointConfig {- PyEndpointConfig::from_core(self.inner.endpoint().clone())- }-- fn __repr__(&self) -> String {- let endpoint: &EndpointConfig = self.inner.endpoint();- format!(- "OpenAiPassthroughBackend(base_url={:?}, timeout_secs={:?})",- endpoint.base_url, endpoint.timeout_secs,- )- }-}--#[pyclass(- name = "AnthropicNativeBackend",- extends = PyLlmBackend,- skip_from_py_object-)]-#[derive(Clone, Debug)]-pub(crate) struct PyAnthropicNativeBackend {- inner: Arc<AnthropicNativeBackend>,-}--#[pymethods]-impl PyAnthropicNativeBackend {- #[new]- fn py_new(target: PyRef<'_, PyLlmTarget>) -> PyResult<PyClassInitializer<Self>> {- let backend = AnthropicNativeBackend::new(target.clone_core()).map_err(py_core_error)?;- let backend = Arc::new(backend);- let base: Arc<dyn LlmBackend> = backend.clone();- Ok(PyClassInitializer::from(PyLlmBackend::from_native(base))- .add_subclass(Self { inner: backend }))- }-- #[getter]- fn target(&self) -> PyLlmTarget {- PyLlmTarget::from_core(self.inner.target().clone())- }-- fn __repr__(&self) -> String {- format!(- "AnthropicNativeBackend(target_id={:?}, model={:?})",- self.inner.target().id.as_str(),- self.inner.target().model.as_str(),- )- }-}--#[pyclass(- name = "MultiLlmBackend",- extends = PyLlmBackend,- skip_from_py_object-)]-#[derive(Clone, Debug)]-pub(crate) struct PyMultiLlmBackend {- inner: Arc<MultiLlmBackend>,-}--#[pymethods]-impl PyMultiLlmBackend {- #[new]- #[pyo3(signature = (targets, supported_request_types=None, default_target_id=None))]- fn py_new(- targets: &Bound<'_, PyAny>,- supported_request_types: Option<&Bound<'_, PyAny>>,- default_target_id: Option<String>,- ) -> PyResult<PyClassInitializer<Self>> {- let targets = target_backends_from_python(targets)?;- let mut backend = MultiLlmBackend::new(targets).map_err(py_core_error)?;- if let Some(supported_request_types) = request_types_from_python(supported_request_types)? {- backend = backend- .with_supported_request_types(supported_request_types)- .map_err(py_core_error)?;- }- if let Some(default_target_id) = default_target_id {- let default_target_id = LlmTargetId::new(default_target_id).map_err(|error| {- PyValueError::new_err(format!("invalid default target id: {error}"))- })?;- backend = backend- .with_default_target(default_target_id)- .map_err(py_core_error)?;- }- let backend = Arc::new(backend);- let base: Arc<dyn LlmBackend> = backend.clone();- Ok(PyClassInitializer::from(PyLlmBackend::from_native(base))- .add_subclass(Self { inner: backend }))- }-- fn target_ids(&self) -> Vec<String> {- self.inner- .targets()- .iter()- .map(|target| target.target().id.as_str().to_string())- .collect()- }-- fn default_target_id(&self) -> Option<String> {- self.inner- .default_target_id()- .map(|target_id| target_id.as_str().to_string())- }-- fn __repr__(&self) -> String {- format!(- "MultiLlmBackend(target_ids={:?}, default_target_id={:?})",- self.target_ids(),- self.default_target_id(),- )- }-}--#[pyclass(- name = "StatsLlmBackend",- extends = PyLlmBackend,- skip_from_py_object-)]-#[derive(Clone, Debug)]-pub(crate) struct PyStatsLlmBackend {- accumulator: PyStatsAccumulator,-}--#[pymethods]-impl PyStatsLlmBackend {- #[new]- fn py_new(- inner: PyRef<'_, PyLlmBackend>,- accumulator: PyRef<'_, PyStatsAccumulator>,- ) -> PyResult<PyClassInitializer<Self>> {- let accumulator = PyStatsAccumulator::from_core(accumulator.clone_core());- let backend: Arc<dyn LlmBackend> = Arc::new(StatsLlmBackend::new(- native_backend(&inner, "StatsLlmBackend")?,- accumulator.clone_core(),- ));- Ok(PyClassInitializer::from(PyLlmBackend::from_native(backend))- .add_subclass(Self { accumulator }))- }-- #[getter]- fn accumulator(&self) -> PyStatsAccumulator {- self.accumulator.clone()- }-- fn __repr__(&self) -> &'static str {- "StatsLlmBackend()"- }-}--fn native_backend(backend: &PyRef<'_, PyLlmBackend>, owner: &str) -> PyResult<Arc<dyn LlmBackend>> {- backend.native().ok_or_else(|| {- PyTypeError::new_err(format!(- "{owner} requires a Rust-native LLMBackend binding, not a Python-only subclass"- ))- })-}--fn target_backends_from_python(value: &Bound<'_, PyAny>) -> PyResult<Vec<LlmTargetBackend>> {- let iterator = PyIterator::from_object(value)?;- let mut targets = Vec::new();- for item in iterator {- let item = item?;- targets.push(target_backend_from_python(&item)?);- }- Ok(targets)-}--fn target_backend_from_python(value: &Bound<'_, PyAny>) -> PyResult<LlmTargetBackend> {- if let Ok(target_backend) = value.extract::<PyRef<'_, PyLlmTargetBackend>>() {- return Ok(target_backend.clone_core());- }-- let tuple = value.cast::<PyTuple>().map_err(|_| {- PyTypeError::new_err(- "MultiLlmBackend targets must be LlmTargetBackend objects or (target, backend) tuples",- )- })?;- if tuple.len() != 2 {- return Err(PyValueError::new_err(- "MultiLlmBackend target tuples must contain exactly (target, backend)",- ));- }-- let target = tuple.get_item(0)?.extract::<PyRef<'_, PyLlmTarget>>()?;- let backend = tuple.get_item(1)?.extract::<PyRef<'_, PyLlmBackend>>()?;- Ok(LlmTargetBackend::new(- target.clone_core(),- native_backend(&backend, "MultiLlmBackend")?,- ))-}--fn request_types_from_python(- value: Option<&Bound<'_, PyAny>>,-) -> PyResult<Option<Vec<ChatRequestType>>> {- let Some(value) = value.filter(|value| !value.is_none()) else {- return Ok(None);- };- let iterator = PyIterator::from_object(value)?;- let mut request_types = Vec::new();- for item in iterator {- request_types.push(request_type_from_python(&item?)?);- }- Ok(Some(request_types))-}--pub(crate) fn register(module: &Bound<'_, PyModule>) -> PyResult<()> {- module.add_class::<PyLlmTargetBackend>()?;- module.add_class::<PyOpenAiNativeBackend>()?;- module.add_class::<PyOpenAiPassthroughBackend>()?;- module.add_class::<PyAnthropicNativeBackend>()?;- module.add_class::<PyMultiLlmBackend>()?;- module.add_class::<PyStatsLlmBackend>()?;- Ok(())-}diff --git a/crates/switchyard-py/src/component_bindings/config.rs b/crates/switchyard-py/src/component_bindings/config.rsdeleted file mode 100644index 7ce3644e5..000000000--- a/crates/switchyard-py/src/component_bindings/config.rs+++ /dev/null@@ -1,533 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Python bindings for component configuration values.--use pyo3::class::basic::CompareOp;-use pyo3::exceptions::PyValueError;-use pyo3::prelude::*;-use pyo3::types::PyBool;-use serde::Serialize;-use switchyard_components::{- BackendFormat, EndpointConfig, LlmTarget, LlmTargetId, ModelId, RandomRoutingProcessorConfig,-};--use crate::errors::py_core_error;-use crate::py_serde::{value_from_python, value_to_python};--#[pyclass(name = "BackendFormat", frozen, skip_from_py_object)]-#[derive(Clone, Copy, Debug, Eq, PartialEq)]-pub(crate) struct PyBackendFormat {- inner: BackendFormat,-}--impl PyBackendFormat {- const fn new_inner(inner: BackendFormat) -> Self {- Self { inner }- }-}--#[pymethods]-impl PyBackendFormat {- #[new]- #[pyo3(signature = (value="auto"))]- fn py_new(value: &str) -> PyResult<Self> {- Ok(Self {- inner: backend_format_from_str(value)?,- })- }-- #[classattr]- const AUTO: Self = Self::new_inner(BackendFormat::Auto);-- #[classattr]- const OPENAI: Self = Self::new_inner(BackendFormat::OpenAi);-- #[classattr]- const RESPONSES: Self = Self::new_inner(BackendFormat::Responses);-- #[classattr]- const ANTHROPIC: Self = Self::new_inner(BackendFormat::Anthropic);-- #[getter]- fn value(&self) -> &'static str {- backend_format_name(self.inner)- }-- fn __repr__(&self) -> String {- format!("BackendFormat.{}", backend_format_variant_name(self.inner))- }-- fn __str__(&self) -> &'static str {- backend_format_name(self.inner)- }-- fn __hash__(&self) -> isize {- match self.inner {- BackendFormat::Auto => 1,- BackendFormat::OpenAi => 2,- BackendFormat::Responses => 3,- BackendFormat::Anthropic => 4,- }- }-- fn __richcmp__(- &self,- py: Python<'_>,- other: &Bound<'_, PyAny>,- op: CompareOp,- ) -> PyResult<Py<PyAny>> {- match op {- CompareOp::Eq | CompareOp::Ne => {- let equals = match backend_format_from_python(Some(other)) {- Ok(other) => self.inner == other,- Err(_) => false,- };- let result = if matches!(op, CompareOp::Eq) {- equals- } else {- !equals- };- Ok(PyBool::new(py, result).to_owned().unbind().into_any())- }- _ => Ok(py.NotImplemented()),- }- }-}--#[pyclass(name = "EndpointConfig", frozen, skip_from_py_object)]-#[derive(Clone, Debug)]-pub(crate) struct PyEndpointConfig {- inner: EndpointConfig,-}--impl PyEndpointConfig {- pub(crate) fn from_core(inner: EndpointConfig) -> Self {- Self { inner }- }-- pub(crate) fn clone_core(&self) -> EndpointConfig {- self.inner.clone()- }-}--#[pymethods]-impl PyEndpointConfig {- #[new]- #[pyo3(signature = (base_url=None, api_key=None, timeout_secs=None))]- fn py_new(- base_url: Option<String>,- api_key: Option<String>,- timeout_secs: Option<f64>,- ) -> Self {- Self {- inner: EndpointConfig {- base_url,- api_key,- timeout_secs,- },- }- }-- #[getter]- fn base_url(&self) -> Option<String> {- self.inner.base_url.clone()- }-- #[getter]- fn api_key(&self) -> Option<String> {- self.inner.api_key.clone()- }-- #[getter]- fn timeout_secs(&self) -> Option<f64> {- self.inner.timeout_secs- }-- fn to_dict(&self, py: Python<'_>) -> PyResult<Py<PyAny>> {- to_python(py, &self.inner)- }-- fn __repr__(&self) -> String {- format!(- "EndpointConfig(base_url={:?}, api_key={}, timeout_secs={:?})",- self.inner.base_url,- if self.inner.api_key.is_some() {- "'<redacted>'"- } else {- "None"- },- self.inner.timeout_secs,- )- }-}--#[pyclass(name = "LlmTarget", frozen, skip_from_py_object)]-#[derive(Clone, Debug)]-pub(crate) struct PyLlmTarget {- inner: LlmTarget,-}--impl PyLlmTarget {- pub(crate) fn from_core(inner: LlmTarget) -> Self {- Self { inner }- }-- pub(crate) fn clone_core(&self) -> LlmTarget {- self.inner.clone()- }-}--#[pymethods]-impl PyLlmTarget {- #[new]- #[allow(clippy::too_many_arguments)]- #[pyo3(signature = (- id=None,- model=None,- format=None,- backend_format=None,- endpoint=None,- base_url=None,- api_key=None,- timeout_secs=None,- timeout=None,- extra_body=None,- extra_headers=None,- ))]- fn py_new(- id: Option<String>,- model: Option<String>,- format: Option<&Bound<'_, PyAny>>,- backend_format: Option<&Bound<'_, PyAny>>,- endpoint: Option<&Bound<'_, PyAny>>,- base_url: Option<String>,- api_key: Option<String>,- timeout_secs: Option<f64>,- timeout: Option<f64>,- extra_body: Option<&Bound<'_, PyAny>>,- extra_headers: Option<&Bound<'_, PyAny>>,- ) -> PyResult<Self> {- let mut endpoint = endpoint_config_from_python(endpoint)?;- if base_url.is_some() {- endpoint.base_url = base_url;- }- if api_key.is_some() {- endpoint.api_key = api_key;- }- if timeout_secs.is_some() {- endpoint.timeout_secs = timeout_secs;- }- if timeout.is_some() {- endpoint.timeout_secs = timeout;- }-- let (id, model) = match (id, model) {- (Some(id), Some(model)) => (id, model),- (None, Some(model)) => ("default".to_string(), model),- (Some(_), None) => {- return Err(PyValueError::new_err(- "LlmTarget requires a model string when id is provided",- ));- }- (None, None) => return Err(PyValueError::new_err("LlmTarget requires a model string")),- };-- let extra_body = match extra_body {- None => None,- Some(value) if value.is_none() => None,- Some(value) => Some(value_from_python(value).map_err(|error| {- PyValueError::new_err(format!(- "LlmTarget.extra_body must be JSON-serialisable: {error}"- ))- })?),- };-- let extra_headers = match extra_headers {- None => std::collections::BTreeMap::new(),- Some(value) if value.is_none() => std::collections::BTreeMap::new(),- Some(value) => {- let raw = value_from_python(value).map_err(|error| {- PyValueError::new_err(format!(- "LlmTarget.extra_headers must be a JSON-serialisable mapping: {error}"- ))- })?;- let serde_json::Value::Object(map) = raw else {- return Err(PyValueError::new_err(- "LlmTarget.extra_headers must be a dict of str -> str",- ));- };- let mut out = std::collections::BTreeMap::new();- for (k, v) in map {- let serde_json::Value::String(s) = v else {- return Err(PyValueError::new_err(format!(- "LlmTarget.extra_headers[{:?}] must be a string, got {}",- k, v- )));- };- out.insert(k, s);- }- out- }- };-- Ok(Self {- inner: LlmTarget {- id: LlmTargetId::new(id).map_err(|error| {- PyValueError::new_err(format!("invalid LLM target id: {error}"))- })?,- model: ModelId::new(model)- .map_err(|error| PyValueError::new_err(format!("invalid model id: {error}")))?,- format: backend_format_from_python(format.or(backend_format))?,- endpoint,- extra_body,- extra_headers,- },- })- }-- #[getter]- fn id(&self) -> String {- self.inner.id.as_str().to_string()- }-- #[getter]- fn model(&self) -> String {- self.inner.model.as_str().to_string()- }-- #[getter]- fn format(&self, py: Python<'_>) -> PyResult<Py<PyAny>> {- backend_format_object(py, self.inner.format)- }-- #[getter]- fn backend_format(&self, py: Python<'_>) -> PyResult<Py<PyAny>> {- backend_format_object(py, self.inner.format)- }-- #[getter]- fn endpoint(&self) -> PyEndpointConfig {- PyEndpointConfig {- inner: self.inner.endpoint.clone(),- }- }-- #[getter]- fn base_url(&self) -> Option<String> {- self.inner.endpoint.base_url.clone()- }-- #[getter]- fn api_key(&self) -> Option<String> {- self.inner.endpoint.api_key.clone()- }-- #[getter]- fn timeout(&self) -> Option<f64> {- self.inner.endpoint.timeout_secs- }-- #[getter]- fn extra_body(&self, py: Python<'_>) -> PyResult<Py<PyAny>> {- match &self.inner.extra_body {- None => Ok(py.None()),- Some(value) => value_to_python(py, value),- }- }-- #[getter]- fn extra_headers(&self, py: Python<'_>) -> PyResult<Py<PyAny>> {- let map: serde_json::Map<String, serde_json::Value> = self- .inner- .extra_headers- .iter()- .map(|(k, v)| (k.clone(), serde_json::Value::String(v.clone())))- .collect();- value_to_python(py, &serde_json::Value::Object(map))- }-- fn to_dict(&self, py: Python<'_>) -> PyResult<Py<PyAny>> {- to_python(py, &self.inner)- }-- fn model_dump(&self, py: Python<'_>) -> PyResult<Py<PyAny>> {- self.to_dict(py)- }-- fn __richcmp__(- &self,- py: Python<'_>,- other: &Bound<'_, PyAny>,- op: CompareOp,- ) -> PyResult<Py<PyAny>> {- match op {- CompareOp::Eq | CompareOp::Ne => {- let equals = if let Ok(other) = other.extract::<PyRef<'_, PyLlmTarget>>() {- self.inner == other.inner- } else if let Ok(other) = value_from_python(other).and_then(|value| {- serde_json::from_value::<LlmTarget>(value)- .map_err(|error| PyValueError::new_err(error.to_string()))- }) {- self.inner == other- } else {- false- };- let result = if matches!(op, CompareOp::Eq) {- equals- } else {- !equals- };- Ok(PyBool::new(py, result).to_owned().unbind().into_any())- }- _ => Ok(py.NotImplemented()),- }- }-- fn __repr__(&self) -> String {- format!(- "LlmTarget(id={:?}, model={:?}, format='{}')",- self.inner.id.as_str(),- self.inner.model.as_str(),- backend_format_name(self.inner.format),- )- }-}--#[pyclass(name = "RandomRoutingProcessorConfig", frozen, skip_from_py_object)]-#[derive(Clone, Debug)]-pub(crate) struct PyRandomRoutingProcessorConfig {- inner: RandomRoutingProcessorConfig,-}--#[pymethods]-impl PyRandomRoutingProcessorConfig {- #[new]- #[pyo3(signature = (strong, weak, strong_probability=0.5, rng_seed=None))]- fn py_new(- strong: PyRef<'_, PyLlmTarget>,- weak: PyRef<'_, PyLlmTarget>,- strong_probability: f64,- rng_seed: Option<u64>,- ) -> PyResult<Self> {- let config = RandomRoutingProcessorConfig::new(strong.clone_core(), weak.clone_core())- .with_strong_probability(strong_probability)- .map_err(py_core_error)?- .with_rng_seed(rng_seed);- Ok(Self { inner: config })- }-- #[getter]- fn strong(&self) -> PyLlmTarget {- PyLlmTarget::from_core(self.inner.strong.clone())- }-- #[getter]- fn weak(&self) -> PyLlmTarget {- PyLlmTarget::from_core(self.inner.weak.clone())- }-- #[getter]- fn strong_probability(&self) -> f64 {- self.inner.strong_probability- }-- #[getter]- fn rng_seed(&self) -> Option<u64> {- self.inner.rng_seed- }-- fn to_dict(&self, py: Python<'_>) -> PyResult<Py<PyAny>> {- to_python(py, &self.inner)- }-- fn __repr__(&self) -> String {- format!(- "RandomRoutingProcessorConfig(strong={}, weak={}, strong_probability={}, rng_seed={:?})",- self.inner.strong.model,- self.inner.weak.model,- self.inner.strong_probability,- self.inner.rng_seed,- )- }-}--pub(crate) fn backend_format_from_python(- value: Option<&Bound<'_, PyAny>>,-) -> PyResult<BackendFormat> {- let Some(value) = value.filter(|value| !value.is_none()) else {- return Ok(BackendFormat::Auto);- };- if let Ok(format) = value.extract::<PyRef<'_, PyBackendFormat>>() {- return Ok(format.inner);- }- let raw = if let Ok(value_attr) = value.getattr("value") {- value_attr.extract::<String>()?- } else {- value.extract::<String>()?- };- backend_format_from_str(&raw)-}--fn backend_format_from_str(value: &str) -> PyResult<BackendFormat> {- match value {- "auto" => Ok(BackendFormat::Auto),- "openai" => Ok(BackendFormat::OpenAi),- "responses" => Ok(BackendFormat::Responses),- "anthropic" => Ok(BackendFormat::Anthropic),- _ => Err(PyValueError::new_err(format!(- "Unknown backend format: {value:?}"- ))),- }-}--fn backend_format_name(format: BackendFormat) -> &'static str {- match format {- BackendFormat::Auto => "auto",- BackendFormat::OpenAi => "openai",- BackendFormat::Responses => "responses",- BackendFormat::Anthropic => "anthropic",- }-}--fn backend_format_variant_name(format: BackendFormat) -> &'static str {- match format {- BackendFormat::Auto => "AUTO",- BackendFormat::OpenAi => "OPENAI",- BackendFormat::Responses => "RESPONSES",- BackendFormat::Anthropic => "ANTHROPIC",- }-}--fn backend_format_object(py: Python<'_>, format: BackendFormat) -> PyResult<Py<PyAny>> {- py.get_type::<PyBackendFormat>()- .getattr(backend_format_variant_name(format))- .map(Bound::unbind)-}--pub(crate) fn endpoint_config_from_python(- value: Option<&Bound<'_, PyAny>>,-) -> PyResult<EndpointConfig> {- let Some(value) = value.filter(|value| !value.is_none()) else {- return Ok(EndpointConfig::default());- };- if let Ok(endpoint) = value.extract::<PyRef<'_, PyEndpointConfig>>() {- return Ok(endpoint.clone_core());- }- serde_json::from_value(value_from_python(value)?)- .map_err(|error| PyValueError::new_err(error.to_string()))-}--fn to_python(py: Python<'_>, value: &impl Serialize) -> PyResult<Py<PyAny>> {- let value =- serde_json::to_value(value).map_err(|error| PyValueError::new_err(error.to_string()))?;- value_to_python(py, &value)-}--pub(crate) fn register(module: &Bound<'_, PyModule>) -> PyResult<()> {- module.add_class::<PyBackendFormat>()?;- module.add_class::<PyEndpointConfig>()?;- module.add_class::<PyLlmTarget>()?;- module.add_class::<PyRandomRoutingProcessorConfig>()?;- Ok(())-}diff --git a/crates/switchyard-py/src/component_bindings/dimension_collector.rs b/crates/switchyard-py/src/component_bindings/dimension_collector.rsdeleted file mode 100644index 1cc6c22b0..000000000--- a/crates/switchyard-py/src/component_bindings/dimension_collector.rs+++ /dev/null@@ -1,398 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Python bindings for the dimension-collector context-signal layer.-//!-//! Exposes:-//!-//! * `DimensionScore` — one scorer's output (name + score + signal).-//! * `ContextSignals` — the aggregate stamped into `ProxyContext`-//! (dimensions tuple + token-count estimate + agentic scalar).-//! * `ScoringConfig` — keyword lists + token-count thresholds.-//! * `DimensionCollector` — request-side component that runs the 15 scorers-//! and stamps `ContextSignals` into the context.-//! * `get_context_signals(ctx)` — Python-facing reader so estimators-//! built on top (LLM classifier, future rules estimator) can pick up-//! the stamped signals.--use pyo3::prelude::*;-use pyo3::types::PyList;-use switchyard_components::ChatResponse;-use switchyard_components::{- DimensionCollector, ResponseSignalCollector,- dimension_collector::{- DEFAULT_RECENT_WINDOW, ResponseFlag, ResponseSignals, ToolResultSignal,- extract_response_signals as core_extract_response_signals,- },-};--use crate::py_serde::value_from_python;--use crate::interop::context::{get_cloned_from_python, lease_from_python};-use crate::interop::request::{request_from_python, request_to_python};-use crate::interop::response::{response_from_python, response_to_python};--/// Request-side component that runs the dimension collector.-#[pyclass(name = "DimensionCollector", skip_from_py_object)]-#[derive(Clone, Debug)]-pub(crate) struct PyDimensionCollector {- inner: DimensionCollector,-}--#[pymethods]-impl PyDimensionCollector {- #[new]- #[pyo3(signature = (*, recent_window = None))]- fn py_new(recent_window: Option<usize>) -> Self {- let window = recent_window.unwrap_or(DEFAULT_RECENT_WINDOW);- Self {- inner: DimensionCollector::with_recent_window(window),- }- }-- fn startup<'py>(&self, py: Python<'py>) -> PyResult<Bound<'py, PyAny>> {- pyo3_async_runtimes::tokio::future_into_py(py, async { Ok(()) })- }-- fn shutdown<'py>(&self, py: Python<'py>) -> PyResult<Bound<'py, PyAny>> {- pyo3_async_runtimes::tokio::future_into_py(py, async { Ok(()) })- }-- fn process<'py>(- &self,- py: Python<'py>,- ctx: &Bound<'_, PyAny>,- request: &Bound<'_, PyAny>,- ) -> PyResult<Bound<'py, PyAny>> {- let processor = self.inner.clone();- let mut lease = lease_from_python(ctx)?;- let request = request_from_python(request)?;- pyo3_async_runtimes::tokio::future_into_py(py, async move {- let result = processor.process(lease.context_mut()?, request).await;- let restore_result = lease.restore();- let request = result.map_err(crate::errors::py_core_error)?;- restore_result?;- Python::attach(|py| request_to_python(py, request))- })- }-- fn __repr__(&self) -> &'static str {- "DimensionCollector()"- }-}--/// Closed set of response-side quality flags emitted by the response-/// signal collector. Mirrors Rust's-/// [`switchyard_components::dimension_collector::ResponseFlag`] enum.-///-/// Python sees these as a class with attribute-style variants-/// (`ResponseFlag.MALFORMED_TOOL_CALL_JSON`, etc.) plus an `__eq__`-/// implementation so set / list membership checks work naturally.-#[pyclass(name = "ResponseFlag", eq, frozen, from_py_object)]-#[derive(Clone, Copy, Debug, PartialEq, Eq)]-pub(crate) enum PyResponseFlag {- MalformedToolCallJson,- EmptyResponse,- TruncatedCompletion,- MissingRequiredArgs,-}--impl PyResponseFlag {- fn from_core(flag: ResponseFlag) -> Self {- match flag {- ResponseFlag::MalformedToolCallJson => Self::MalformedToolCallJson,- ResponseFlag::EmptyResponse => Self::EmptyResponse,- ResponseFlag::TruncatedCompletion => Self::TruncatedCompletion,- ResponseFlag::MissingRequiredArgs => Self::MissingRequiredArgs,- }- }-}--#[pymethods]-impl PyResponseFlag {- fn __repr__(&self) -> &'static str {- match self {- Self::MalformedToolCallJson => "ResponseFlag.MALFORMED_TOOL_CALL_JSON",- Self::EmptyResponse => "ResponseFlag.EMPTY_RESPONSE",- Self::TruncatedCompletion => "ResponseFlag.TRUNCATED_COMPLETION",- Self::MissingRequiredArgs => "ResponseFlag.MISSING_REQUIRED_ARGS",- }- }-- fn __hash__(&self) -> u64 {- *self as u64- }-}--/// Aggregate response-side signals stamped by [`PyResponseSignalCollector`].-#[pyclass(name = "ResponseSignals", frozen, skip_from_py_object)]-#[derive(Clone, Debug)]-pub(crate) struct PyResponseSignals {- inner: ResponseSignals,-}--#[pymethods]-impl PyResponseSignals {- /// Failing flags, in the order the checker ran them.- #[getter]- fn flags(&self, py: Python<'_>) -> PyResult<Py<PyList>> {- let items: Vec<Py<PyResponseFlag>> = self- .inner- .flags- .iter()- .map(|flag| Py::new(py, PyResponseFlag::from_core(*flag)))- .collect::<PyResult<Vec<_>>>()?;- Ok(PyList::new(py, items)?.unbind())- }-- /// True when at least one check failed; stage_router routers use this as- /// the per-attempt acceptability gate.- fn has_failures(&self) -> bool {- self.inner.has_failures()- }-- /// Python-friendly membership check; equivalent to- /// `flag in signals.flags`.- fn contains(&self, flag: PyResponseFlag) -> bool {- self.inner- .flags- .iter()- .any(|inner| PyResponseFlag::from_core(*inner) == flag)- }-- fn __repr__(&self) -> String {- format!("ResponseSignals(flags=<{} items>)", self.inner.flags.len())- }-}--impl PyResponseSignals {- fn from_core(inner: ResponseSignals) -> Self {- Self { inner }- }-}--/// Response-side component that runs response-side signal extraction.-#[pyclass(name = "ResponseSignalCollector", skip_from_py_object)]-#[derive(Clone, Debug)]-pub(crate) struct PyResponseSignalCollector;--#[pymethods]-impl PyResponseSignalCollector {- #[new]- fn py_new() -> Self {- Self- }-- fn startup<'py>(&self, py: Python<'py>) -> PyResult<Bound<'py, PyAny>> {- pyo3_async_runtimes::tokio::future_into_py(py, async { Ok(()) })- }-- fn shutdown<'py>(&self, py: Python<'py>) -> PyResult<Bound<'py, PyAny>> {- pyo3_async_runtimes::tokio::future_into_py(py, async { Ok(()) })- }-- fn process<'py>(- &self,- py: Python<'py>,- ctx: &Bound<'_, PyAny>,- response: &Bound<'_, PyAny>,- ) -> PyResult<Bound<'py, PyAny>> {- let mut lease = lease_from_python(ctx)?;- let response = response_from_python(response)?;- pyo3_async_runtimes::tokio::future_into_py(py, async move {- let result = ResponseSignalCollector- .process(lease.context_mut()?, response)- .await;- let restore_result = lease.restore();- let response = result.map_err(crate::errors::py_core_error)?;- restore_result?;- Python::attach(|py| response_to_python(py, response))- })- }-- fn __repr__(&self) -> &'static str {- "ResponseSignalCollector()"- }-}--/// Returns the `ResponseSignals` stamped by a `ResponseSignalCollector` run.-///-/// `None` either means the collector hasn't run yet on this `ctx` or the-/// response was a streaming response (which the buffered-body checks-/// can't introspect).-#[pyfunction]-fn get_response_signals(ctx: &Bound<'_, PyAny>) -> PyResult<Option<PyResponseSignals>> {- Ok(get_cloned_from_python::<ResponseSignals>(ctx)?.map(PyResponseSignals::from_core))-}--/// Runs the response-side checks against an inline response body dict.-///-/// Intended for the stage-router, which needs to evaluate-/// `ResponseSignals` between attempts without going through a full-/// response-side pass. Accepts the response's `.body` Python-/// dict directly; works for any wire shape because the checks are-/// structure-based, not variant-based (`choices[...]` vs `content[...]`-/// dispatch happens inside the Rust checks themselves).-///-/// Returns an empty `ResponseSignals` (no failures) if the body is-/// `None` or not a dict — same fail-safe posture as the-/// `ResponseSignalCollector` adapter.-#[pyfunction]-fn extract_response_signals(body: Option<&Bound<'_, PyAny>>) -> PyResult<PyResponseSignals> {- let Some(body) = body else {- return Ok(PyResponseSignals::from_core(ResponseSignals::default()));- };- if body.is_none() {- return Ok(PyResponseSignals::from_core(ResponseSignals::default()));- }- let value = value_from_python(body)?;- // The four checks dispatch on body structure, not on the- // `ChatResponse` variant. Wrap in OpenAI-completion just to satisfy- // the type; Anthropic-shaped bodies still get their dedicated- // checks via the `content[]` walk.- let response = ChatResponse::openai_completion(value);- Ok(PyResponseSignals::from_core(core_extract_response_signals(- &response,- )))-}--/// Tool-result context signals stamped by [`PyDimensionCollector`].-///-/// Read via :func:`get_tool_result_signal`. All fields default to neutral-/// values (0.0 / False) when no tool results are present in the request.-#[pyclass(name = "ToolResultSignal", frozen, skip_from_py_object)]-#[derive(Clone, Debug)]-pub(crate) struct PyToolResultSignal {- inner: ToolResultSignal,-}--#[pymethods]-impl PyToolResultSignal {- /// Max error severity across all matched patterns in the last tool result.- /// ``0.0`` = clean; ``0.3`` = soft; ``0.7`` = hard; ``1.0`` = critical.- #[getter]- fn severity(&self) -> f32 {- self.inner.severity- }-- /// Consecutive clean tool results at the end of history (``0`` if last failed).- #[getter]- fn no_error_streak(&self) -> u32 {- self.inner.no_error_streak- }-- /// Edit-type tool calls in the conversation (refinement work).- #[getter]- fn edit_count(&self) -> u32 {- self.inner.edit_count- }-- /// Write/create-type tool calls in the conversation (scaffolding work).- #[getter]- fn write_count(&self) -> u32 {- self.inner.write_count- }-- /// Read-type tool calls (Read tool + read-like Bash inspections).- #[getter]- fn read_count(&self) -> u32 {- self.inner.read_count- }-- /// TodoWrite calls — Opus struggle signal used by the strong-default drop gate.- #[getter]- fn todowrite_count(&self) -> u32 {- self.inner.todowrite_count- }-- /// Edit-type tool calls in the most recent 3 tool calls (sliding window).- #[getter]- fn recent_edit_count(&self) -> u32 {- self.inner.recent_edit_count- }-- /// Write/create-type tool calls in the most recent 3 tool calls (sliding window).- #[getter]- fn recent_write_count(&self) -> u32 {- self.inner.recent_write_count- }-- /// Read-type tool calls in the most recent 3 tool calls (sliding window).- #[getter]- fn recent_read_count(&self) -> u32 {- self.inner.recent_read_count- }-- /// TodoWrite calls in the most recent 3 tool calls (sliding window).- #[getter]- fn recent_todowrite_count(&self) -> u32 {- self.inner.recent_todowrite_count- }-- /// Consecutive trailing `Other`-category tool calls (build-pit proxy).- #[getter]- fn pure_bash_streak(&self) -> u32 {- self.inner.pure_bash_streak- }-- /// ``True`` when a recent tool result contained passing test output.- #[getter]- fn tests_passed(&self) -> bool {- self.inner.tests_passed- }-- /// Total messages in the conversation (turn-depth proxy).- #[getter]- fn turn_depth(&self) -> u32 {- self.inner.turn_depth- }-- /// The request carries a context-compaction summary — the picker forces + holds- /// the strong tier, since compaction otherwise de-escalates the router to weak.- #[getter]- fn compacted(&self) -> bool {- self.inner.compacted- }-- fn __repr__(&self) -> String {- format!(- "ToolResultSignal(severity={:.2}, streak={}, edit={}, write={}, tests_passed={})",- self.inner.severity,- self.inner.no_error_streak,- self.inner.edit_count,- self.inner.write_count,- self.inner.tests_passed,- )- }-}--impl PyToolResultSignal {- fn from_core(inner: ToolResultSignal) -> Self {- Self { inner }- }-- /// The underlying Rust signal — used by the stage_router picker binding.- pub(crate) fn core(&self) -> &ToolResultSignal {- &self.inner- }-}--/// Returns the :class:`ToolResultSignal` stamped by a :class:`DimensionCollector` run.-///-/// Returns ``None`` when the collector has not run on this context yet.-#[pyfunction]-fn get_tool_result_signal(ctx: &Bound<'_, PyAny>) -> PyResult<Option<PyToolResultSignal>> {- Ok(get_cloned_from_python::<ToolResultSignal>(ctx)?.map(PyToolResultSignal::from_core))-}--pub(crate) fn register(module: &Bound<'_, PyModule>) -> PyResult<()> {- module.add_class::<PyDimensionCollector>()?;- module.add_class::<PyResponseFlag>()?;- module.add_class::<PyResponseSignals>()?;- module.add_class::<PyResponseSignalCollector>()?;- module.add_class::<PyToolResultSignal>()?;- module.add_function(wrap_pyfunction!(get_response_signals, module)?)?;- module.add_function(wrap_pyfunction!(extract_response_signals, module)?)?;- module.add_function(wrap_pyfunction!(get_tool_result_signal, module)?)?;- Ok(())-}diff --git a/crates/switchyard-py/src/component_bindings/request_processors.rs b/crates/switchyard-py/src/component_bindings/request_processors.rsdeleted file mode 100644index 19a133c3d..000000000--- a/crates/switchyard-py/src/component_bindings/request_processors.rs+++ /dev/null@@ -1,62 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Python bindings for concrete request-side compatibility components.--use pyo3::prelude::*;-use switchyard_components::{StatsRequestProcessor, tracking_enabled_from_env};--use crate::errors::py_core_error;-use crate::interop::context::lease_from_python;-use crate::interop::request::{request_from_python, request_to_python};--#[pyclass(name = "StatsRequestProcessor", skip_from_py_object)]-#[derive(Clone, Copy, Debug)]-pub(crate) struct PyStatsRequestProcessor {- inner: StatsRequestProcessor,-}--#[pymethods]-impl PyStatsRequestProcessor {- #[new]- fn py_new() -> Self {- Self {- inner: StatsRequestProcessor::new(tracking_enabled_from_env()),- }- }-- fn startup<'py>(&self, py: Python<'py>) -> PyResult<Bound<'py, PyAny>> {- pyo3_async_runtimes::tokio::future_into_py(py, async { Ok(()) })- }-- fn shutdown<'py>(&self, py: Python<'py>) -> PyResult<Bound<'py, PyAny>> {- pyo3_async_runtimes::tokio::future_into_py(py, async { Ok(()) })- }-- fn process<'py>(- &self,- py: Python<'py>,- ctx: &Bound<'_, PyAny>,- request: &Bound<'_, PyAny>,- ) -> PyResult<Bound<'py, PyAny>> {- let processor = self.inner;- let mut lease = lease_from_python(ctx)?;- let request = request_from_python(request)?;- pyo3_async_runtimes::tokio::future_into_py(py, async move {- let result = processor.process(lease.context_mut()?, request).await;- let restore_result = lease.restore();- let request = result.map_err(py_core_error)?;- restore_result?;- Python::attach(|py| request_to_python(py, request))- })- }-- fn __repr__(&self) -> &'static str {- "StatsRequestProcessor()"- }-}--pub(crate) fn register(module: &Bound<'_, PyModule>) -> PyResult<()> {- module.add_class::<PyStatsRequestProcessor>()?;- Ok(())-}diff --git a/crates/switchyard-py/src/component_bindings/response_processors.rs b/crates/switchyard-py/src/component_bindings/response_processors.rsdeleted file mode 100644index eccaf5444..000000000--- a/crates/switchyard-py/src/component_bindings/response_processors.rs+++ /dev/null@@ -1,79 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Python bindings for concrete response-side components.--use pyo3::prelude::*;-use switchyard_components::StatsResponseProcessor;--use super::stats::PyStatsAccumulator;-use crate::errors::py_core_error;-use crate::interop::context::lease_from_python;-use crate::interop::response::{response_from_python, response_to_python};--#[pyclass(name = "StatsResponseProcessor", skip_from_py_object)]-#[derive(Clone, Debug)]-pub(crate) struct PyStatsResponseProcessor {- inner: StatsResponseProcessor,- accumulator: PyStatsAccumulator,-}--#[pymethods]-impl PyStatsResponseProcessor {- #[new]- fn py_new(accumulator: PyRef<'_, PyStatsAccumulator>) -> Self {- let accumulator = PyStatsAccumulator::from_core(accumulator.clone_core());- Self {- inner: StatsResponseProcessor::new(accumulator.clone_core()),- accumulator,- }- }-- #[getter]- fn accumulator(&self) -> PyStatsAccumulator {- self.accumulator.clone()- }-- fn startup<'py>(&self, py: Python<'py>) -> PyResult<Bound<'py, PyAny>> {- pyo3_async_runtimes::tokio::future_into_py(py, async { Ok(()) })- }-- fn shutdown<'py>(&self, py: Python<'py>) -> PyResult<Bound<'py, PyAny>> {- pyo3_async_runtimes::tokio::future_into_py(py, async { Ok(()) })- }-- fn process<'py>(- &self,- py: Python<'py>,- ctx: &Bound<'_, PyAny>,- response: &Bound<'_, PyAny>,- ) -> PyResult<Bound<'py, PyAny>> {- let processor = self.inner.clone();- let mut lease = lease_from_python(ctx)?;- let response = response_from_python(response)?;- pyo3_async_runtimes::tokio::future_into_py(py, async move {- let result = processor.process(lease.context_mut()?, response).await;- let restore_result = lease.restore();- let response = result.map_err(py_core_error)?;- restore_result?;- Python::attach(|py| response_to_python(py, response))- })- }-- fn get_endpoint(&self, py: Python<'_>) -> PyResult<Py<PyAny>> {- let endpoint = py- .import("switchyard.lib.endpoints.stats_endpoint")?- .getattr("StatsEndpoint")?- .call1((self.accumulator.clone(),))?;- Ok(endpoint.unbind())- }-- fn __repr__(&self) -> &'static str {- "StatsResponseProcessor()"- }-}--pub(crate) fn register(module: &Bound<'_, PyModule>) -> PyResult<()> {- module.add_class::<PyStatsResponseProcessor>()?;- Ok(())-}diff --git a/crates/switchyard-py/src/component_bindings/stage_router.rs b/crates/switchyard-py/src/component_bindings/stage_router.rsdeleted file mode 100644index 33a33d3a0..000000000--- a/crates/switchyard-py/src/component_bindings/stage_router.rs+++ /dev/null@@ -1,154 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Python binding for the shared stage_router picker.-//!-//! Exposes [`switchyard_components::stage_router::pick_tier`] as-//! `stage_pick_tier(signal, picker_mode, confidence_threshold) -> PickOutcome`-//! for Python analysis tools. This returns a resolved decision or a request to-//! consult a classifier.--use pyo3::exceptions::PyValueError;-use pyo3::prelude::*;--use switchyard_components::stage_router::{PickOutcome, PickerMode, Tier, pick_tier, score_signal};--use super::dimension_collector::PyToolResultSignal;--fn parse_picker_mode(mode: &str) -> PyResult<PickerMode> {- match mode {- "capable_first" => Ok(PickerMode::CapableFirst),- "efficient_first" => Ok(PickerMode::EfficientFirst),- other => Err(PyValueError::new_err(format!(- "unknown picker mode {other:?} (expected capable_first or efficient_first)"- ))),- }-}--fn tier_str(tier: Tier) -> &'static str {- match tier {- Tier::Capable => "capable",- Tier::Efficient => "efficient",- }-}--/// Result of [`stage_pick_tier`].-///-/// `resolved` is `True` when the picker decided without the classifier — then-/// `tier` and `source` are set. `resolved` is `False` when the scorer was not-/// confident: the caller runs its classifier, and falls back to `default_tier`-/// if it has none or it fails. `score` / `confidence` are always the scorer's.-#[pyclass(name = "PickOutcome", frozen)]-pub(crate) struct PyPickOutcome {- resolved: bool,- tier: Option<&'static str>,- source: Option<&'static str>,- default_tier: &'static str,- score: f64,- confidence: Option<f64>,-}--#[pymethods]-impl PyPickOutcome {- #[getter]- fn resolved(&self) -> bool {- self.resolved- }-- /// Chosen tier (`"capable"` / `"efficient"`) — only when `resolved`.- #[getter]- fn tier(&self) -> Option<&'static str> {- self.tier- }-- /// Decision source (`"override"` / `"tests_passed"` / `"dimensions"`) — only- /// when `resolved`.- #[getter]- fn source(&self) -> Option<&'static str> {- self.source- }-- /// Tier to fall open to when the classifier does not resolve the turn.- #[getter]- fn default_tier(&self) -> &'static str {- self.default_tier- }-- #[getter]- fn score(&self) -> f64 {- self.score- }-- #[getter]- fn confidence(&self) -> Option<f64> {- self.confidence- }-- fn __repr__(&self) -> String {- if self.resolved {- format!(- "PickOutcome(resolved, tier={:?}, source={:?}, score={:.3})",- self.tier, self.source, self.score- )- } else {- format!(- "PickOutcome(consult_classifier, default_tier={:?}, score={:.3})",- self.default_tier, self.score- )- }- }-}--/// Decide a turn's tier from its [`ToolResultSignal`], up to (but not including)-/// the classifier. `picker_mode` is `"capable_first"` or `"efficient_first"`.-#[pyfunction]-fn stage_pick_tier(- signal: PyRef<'_, PyToolResultSignal>,- picker_mode: &str,- confidence_threshold: f64,-) -> PyResult<PyPickOutcome> {- let mode = parse_picker_mode(picker_mode)?;- let outcome = match pick_tier(signal.core(), mode, confidence_threshold) {- PickOutcome::Resolved {- tier,- source,- score,- confidence,- } => PyPickOutcome {- resolved: true,- tier: Some(tier_str(tier)),- source: Some(source.as_str()),- default_tier: tier_str(tier),- score,- confidence,- },- PickOutcome::ConsultClassifier {- score,- confidence,- default_tier,- } => PyPickOutcome {- resolved: false,- tier: None,- source: None,- default_tier: tier_str(default_tier),- score,- confidence: Some(confidence),- },- };- Ok(outcome)-}--/// The pure two-axis scorer for a signal, as `(score, confidence)`. Used by-/// offline analysis to replay the raw score independent of the picker mode.-#[pyfunction]-fn stage_score_signal(signal: PyRef<'_, PyToolResultSignal>) -> (f64, f64) {- let result = score_signal(signal.core());- (result.score, result.confidence)-}--pub(crate) fn register(module: &Bound<'_, PyModule>) -> PyResult<()> {- module.add_class::<PyPickOutcome>()?;- module.add_function(wrap_pyfunction!(stage_pick_tier, module)?)?;- module.add_function(wrap_pyfunction!(stage_score_signal, module)?)?;- Ok(())-}diff --git a/crates/switchyard-py/src/component_bindings/stats.rs b/crates/switchyard-py/src/component_bindings/stats.rsdeleted file mode 100644index 08a7023f8..000000000--- a/crates/switchyard-py/src/component_bindings/stats.rs+++ /dev/null@@ -1,241 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Python bindings for shared stats state.--use pyo3::exceptions::PyValueError;-use pyo3::prelude::*;-use serde::Serialize;-use switchyard_components::{StatsAccumulator, StatsRouteLabel, TokenUsage};--use crate::errors::py_core_error;-use crate::interop::context::insert_into_python;-use crate::py_serde::value_to_python;--#[pyclass(name = "StatsAccumulator", skip_from_py_object)]-#[derive(Clone, Debug)]-pub(crate) struct PyStatsAccumulator {- inner: StatsAccumulator,-}--impl PyStatsAccumulator {- pub(crate) fn from_core(inner: StatsAccumulator) -> Self {- Self { inner }- }-- pub(crate) fn clone_core(&self) -> StatsAccumulator {- self.inner.clone()- }-}--#[pymethods]-impl PyStatsAccumulator {- #[new]- fn py_new() -> Self {- Self {- inner: StatsAccumulator::new(),- }- }-- #[pyo3(signature = (model, backend_latency_ms=None, tier=None))]- fn record_success<'py>(- &self,- py: Python<'py>,- model: String,- backend_latency_ms: Option<f64>,- tier: Option<String>,- ) -> PyResult<Bound<'py, PyAny>> {- let accumulator = self.inner.clone();- pyo3_async_runtimes::tokio::future_into_py(py, async move {- accumulator- .record_success(model, backend_latency_ms, tier.as_deref())- .map_err(py_core_error)- })- }-- #[pyo3(signature = (model, tier=None))]- fn record_error<'py>(- &self,- py: Python<'py>,- model: String,- tier: Option<String>,- ) -> PyResult<Bound<'py, PyAny>> {- let accumulator = self.inner.clone();- pyo3_async_runtimes::tokio::future_into_py(py, async move {- accumulator- .record_error(model, tier.as_deref())- .map_err(py_core_error)- })- }-- #[pyo3(signature = (- model,- prompt_tokens=0,- completion_tokens=0,- cached_tokens=0,- cache_creation_tokens=0,- reasoning_tokens=0,- total_latency_ms=None,- routing_overhead_ms=None,- tier=None,- success_was_untiered=false- ))]- #[allow(clippy::too_many_arguments)]- fn record_usage<'py>(- &self,- py: Python<'py>,- model: String,- prompt_tokens: u64,- completion_tokens: u64,- cached_tokens: u64,- cache_creation_tokens: u64,- reasoning_tokens: u64,- total_latency_ms: Option<f64>,- routing_overhead_ms: Option<f64>,- tier: Option<String>,- success_was_untiered: bool,- ) -> PyResult<Bound<'py, PyAny>> {- let accumulator = self.inner.clone();- pyo3_async_runtimes::tokio::future_into_py(py, async move {- let usage = TokenUsage {- prompt_tokens,- completion_tokens,- cached_tokens,- cache_creation_tokens,- reasoning_tokens,- cacheable_prompt_tokens: 0,- };- let result = if success_was_untiered {- accumulator.record_usage_with_success_was_untiered(- model,- usage,- total_latency_ms,- routing_overhead_ms,- tier.as_deref(),- )- } else {- accumulator.record_usage(- model,- usage,- total_latency_ms,- routing_overhead_ms,- tier.as_deref(),- )- };- result.map_err(py_core_error)- })- }-- #[pyo3(signature = (- model,- prompt_tokens=0,- completion_tokens=0,- cached_tokens=0,- cache_creation_tokens=0,- reasoning_tokens=0,- latency_ms=None,- ))]- #[allow(clippy::too_many_arguments)]- fn record_classifier_usage<'py>(- &self,- py: Python<'py>,- model: String,- prompt_tokens: u64,- completion_tokens: u64,- cached_tokens: u64,- cache_creation_tokens: u64,- reasoning_tokens: u64,- latency_ms: Option<f64>,- ) -> PyResult<Bound<'py, PyAny>> {- let accumulator = self.inner.clone();- pyo3_async_runtimes::tokio::future_into_py(py, async move {- let usage = TokenUsage {- prompt_tokens,- completion_tokens,- cached_tokens,- cache_creation_tokens,- reasoning_tokens,- cacheable_prompt_tokens: 0,- };- accumulator- .record_classifier_usage(model, usage, latency_ms)- .map_err(py_core_error)- })- }-- fn record_classifier_error<'py>(- &self,- py: Python<'py>,- model: String,- ) -> PyResult<Bound<'py, PyAny>> {- let accumulator = self.inner.clone();- pyo3_async_runtimes::tokio::future_into_py(py, async move {- accumulator- .record_classifier_error(model)- .map_err(py_core_error)- })- }-- fn record_routing_decision<'py>(- &self,- py: Python<'py>,- profile_type: String,- source: String,- ) -> PyResult<Bound<'py, PyAny>> {- let accumulator = self.inner.clone();- pyo3_async_runtimes::tokio::future_into_py(py, async move {- accumulator- .record_routing_decision(profile_type, source)- .map_err(py_core_error)- })- }-- fn snapshot<'py>(&self, py: Python<'py>) -> PyResult<Bound<'py, PyAny>> {- let accumulator = self.inner.clone();- pyo3_async_runtimes::tokio::future_into_py(py, async move {- let snapshot = accumulator.snapshot().map_err(py_core_error)?;- Python::attach(|py| to_python(py, &snapshot))- })- }-- fn snapshot_sync(&self, py: Python<'_>) -> PyResult<Py<PyAny>> {- let snapshot = self.inner.snapshot().map_err(py_core_error)?;- to_python(py, &snapshot)- }-- fn reset<'py>(&self, py: Python<'py>) -> PyResult<Bound<'py, PyAny>> {- let accumulator = self.inner.clone();- pyo3_async_runtimes::tokio::future_into_py(py, async move {- accumulator.reset().map_err(py_core_error)- })- }-- fn reset_sync(&self) -> PyResult<()> {- self.inner.reset().map_err(py_core_error)- }-- fn __repr__(&self) -> &'static str {- "StatsAccumulator()"- }-}--fn to_python(py: Python<'_>, value: &impl Serialize) -> PyResult<Py<PyAny>> {- let value =- serde_json::to_value(value).map_err(|error| PyValueError::new_err(error.to_string()))?;- value_to_python(py, &value)-}--#[pyfunction]-fn set_stats_route_label(ctx: &Bound<'_, PyAny>, label: &str) -> PyResult<()> {- let label = label.trim();- if label.is_empty() {- return Err(PyValueError::new_err("stats route label must not be empty"));- }- insert_into_python(ctx, StatsRouteLabel::new(label))-}--pub(crate) fn register(module: &Bound<'_, PyModule>) -> PyResult<()> {- module.add_class::<PyStatsAccumulator>()?;- module.add_function(wrap_pyfunction!(set_stats_route_label, module)?)?;- Ok(())-}diff --git a/crates/switchyard-py/src/errors.rs b/crates/switchyard-py/src/errors.rsindex 99ab5ed88..9e0beb775 100644--- a/crates/switchyard-py/src/errors.rs+++ b/crates/switchyard-py/src/errors.rs@@ -1,216 +1,19 @@// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.// SPDX-License-Identifier: Apache-2.0-//! Error mapping helpers for PyO3 bindings.+//! Error mapping for the libsy Python binding.use pyo3::create_exception;-use pyo3::exceptions::{PyRuntimeError, PyValueError};+use pyo3::exceptions::PyRuntimeError;use pyo3::prelude::*;-use switchyard_components::SwitchyardError;-use switchyard_translation::TranslationError;-create_exception!(_switchyard_rust, SwitchyardRuntimeError, PyRuntimeError);-create_exception!(_switchyard_rust, LibsyError, SwitchyardRuntimeError);-create_exception!(- _switchyard_rust,- SwitchyardConfigError,- SwitchyardRuntimeError-);-create_exception!(- _switchyard_rust,- SwitchyardInvalidIdError,- SwitchyardRuntimeError-);-create_exception!(- _switchyard_rust,- SwitchyardDuplicateRegistrationError,- SwitchyardRuntimeError-);-create_exception!(- _switchyard_rust,- SwitchyardModelNotFoundError,- SwitchyardRuntimeError-);-create_exception!(- _switchyard_rust,- SwitchyardUnsupportedRequestTypeError,- SwitchyardRuntimeError-);-// Raised by `ChatRequest.validate()` when a structurally valid body fails-// semantic validation (e.g. an empty `messages` array). Endpoints map it to-// a 4xx so agents can distinguish a client bug from a server failure.-create_exception!(- _switchyard_rust,- SwitchyardInvalidRequestError,- SwitchyardRuntimeError-);-create_exception!(- _switchyard_rust,- SwitchyardProcessorError,- SwitchyardRuntimeError-);-create_exception!(- _switchyard_rust,- SwitchyardBackendError,- SwitchyardRuntimeError-);-create_exception!(- _switchyard_rust,- SwitchyardUpstreamError,- SwitchyardRuntimeError-);-// Raised by an LLMBackend when the upstream rejects a request because the-// prompt exceeds the model's context window. Subclasses SwitchyardBackendError-// so existing broad catches still match.-create_exception!(- _switchyard_rust,- SwitchyardContextWindowExceededError,- SwitchyardBackendError-);-// Raised by compatibility/runtime code when every attempted target returned a-// context-window overflow and no fallback remains.-create_exception!(- _switchyard_rust,- SwitchyardContextPoolExhaustedError,- SwitchyardBackendError-);--/// Converts translation crate errors into Python `ValueError`s with stable context.-pub(crate) fn py_translation_error(error: TranslationError) -> PyErr {- PyValueError::new_err(format!("{}: {}", error.kind(), error))-}+create_exception!(_switchyard_rust, LibsyError, PyRuntimeError);/// Converts libsy execution failures into one stable Python exception.pub(crate) fn py_libsy_error(error: impl std::fmt::Display) -> PyErr {LibsyError::new_err(error.to_string())}-/// Converts core Switchyard errors into typed Python runtime errors.-///-/// `ContextWindowExceeded` and `ContextPoolExhausted` carry typed fields-/// (`target_id`, `model`, `last_target_id`, `reason`) — we attach those as-/// Python attributes on the raised exception so callers can inspect them-/// programmatically and `backend_error_with_ctx` can recover the typed-/// variant when a Python `LLMBackend` re-raises through the Rust chain.-/// Without these attrs the variant collapses to the string message and the-/// compatibility retry code stamps `"unknown"` for `target_id`/`model`.-pub(crate) fn py_core_error(error: SwitchyardError) -> PyErr {- let message = error.to_string();- match error {- SwitchyardError::InvalidConfig(_) => SwitchyardConfigError::new_err(message),- SwitchyardError::InvalidId(_) => SwitchyardInvalidIdError::new_err(message),- SwitchyardError::DuplicateRegistration { .. } => {- SwitchyardDuplicateRegistrationError::new_err(message)- }- SwitchyardError::ModelNotFound { .. } => SwitchyardModelNotFoundError::new_err(message),- SwitchyardError::UnsupportedRequestType { .. } => {- SwitchyardUnsupportedRequestTypeError::new_err(message)- }- SwitchyardError::InvalidRequest(_) => SwitchyardInvalidRequestError::new_err(message),- SwitchyardError::Processor(_) => SwitchyardProcessorError::new_err(message),- SwitchyardError::Backend(_) => SwitchyardBackendError::new_err(message),- SwitchyardError::Upstream(_) => SwitchyardUpstreamError::new_err(message),- SwitchyardError::UpstreamHttp {- status_code, body, ..- } => {- let err = SwitchyardUpstreamError::new_err(message);- attach_upstream_http_attrs(&err, status_code, &body);- err- }- SwitchyardError::ContextWindowExceeded {- target_id, model, ..- } => {- let err = SwitchyardContextWindowExceededError::new_err(message);- attach_attrs(&err, &[("target_id", &target_id), ("model", &model)]);- err- }- SwitchyardError::ContextPoolExhausted {- last_target_id,- reason,- } => {- let err = SwitchyardContextPoolExhaustedError::new_err(message);- attach_attrs(- &err,- &[("last_target_id", &last_target_id), ("reason", &reason)],- );- err- }- SwitchyardError::Other(_) => SwitchyardRuntimeError::new_err(message),- }-}--/// Set string attributes on a freshly-constructed `PyErr`'s exception value.-/// Errors are intentionally ignored — attribute attachment is best-effort-/// diagnostic metadata, never the path-of-correctness for raising the error.-fn attach_attrs(err: &PyErr, attrs: &[(&str, &str)]) {- Python::attach(|py| {- let bound = err.value(py);- for (name, value) in attrs {- let _ = bound.setattr(*name, *value);- }- });-}--/// Attach typed upstream HTTP details for endpoint error handling.-fn attach_upstream_http_attrs(err: &PyErr, status_code: u16, body: &str) {- Python::attach(|py| {- let bound = err.value(py);- let _ = bound.setattr("status_code", status_code);- let _ = bound.setattr("body", body);- });-}--/// Registers typed exception classes in the native extension module.pub(crate) fn register(module: &Bound<'_, PyModule>) -> PyResult<()> {- let py = module.py();- module.add(- "SwitchyardRuntimeError",- py.get_type::<SwitchyardRuntimeError>(),- )?;- module.add("LibsyError", py.get_type::<LibsyError>())?;- module.add(- "SwitchyardConfigError",- py.get_type::<SwitchyardConfigError>(),- )?;- module.add(- "SwitchyardInvalidIdError",- py.get_type::<SwitchyardInvalidIdError>(),- )?;- module.add(- "SwitchyardDuplicateRegistrationError",- py.get_type::<SwitchyardDuplicateRegistrationError>(),- )?;- module.add(- "SwitchyardModelNotFoundError",- py.get_type::<SwitchyardModelNotFoundError>(),- )?;- module.add(- "SwitchyardUnsupportedRequestTypeError",- py.get_type::<SwitchyardUnsupportedRequestTypeError>(),- )?;- module.add(- "SwitchyardInvalidRequestError",- py.get_type::<SwitchyardInvalidRequestError>(),- )?;- module.add(- "SwitchyardProcessorError",- py.get_type::<SwitchyardProcessorError>(),- )?;- module.add(- "SwitchyardBackendError",- py.get_type::<SwitchyardBackendError>(),- )?;- module.add(- "SwitchyardUpstreamError",- py.get_type::<SwitchyardUpstreamError>(),- )?;- module.add(- "SwitchyardContextWindowExceededError",- py.get_type::<SwitchyardContextWindowExceededError>(),- )?;- module.add(- "SwitchyardContextPoolExhaustedError",- py.get_type::<SwitchyardContextPoolExhaustedError>(),- )?;- Ok(())+ module.add("LibsyError", module.py().get_type::<LibsyError>())}diff --git a/crates/switchyard-py/src/interop.rs b/crates/switchyard-py/src/interop.rsdeleted file mode 100644index 2088ed48a..000000000--- a/crates/switchyard-py/src/interop.rs+++ /dev/null@@ -1,20 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Private adapters between Python compatibility objects and native components.--use pyo3::prelude::*;--pub(crate) mod context;-pub(crate) mod request;-pub(crate) mod response;-pub(crate) mod roles;-pub(crate) mod subagent;--pub(crate) fn register(module: &Bound<'_, PyModule>) -> PyResult<()> {- context::register(module)?;- response::register(module)?;- roles::register(module)?;- subagent::register(module)?;- Ok(())-}diff --git a/crates/switchyard-py/src/interop/context.rs b/crates/switchyard-py/src/interop/context.rsdeleted file mode 100644index cd84bde3e..000000000--- a/crates/switchyard-py/src/interop/context.rs+++ /dev/null@@ -1,361 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Private adapter for Rust-owned request context values.--use std::sync::Arc;-use std::sync::atomic::{AtomicBool, Ordering};--use parking_lot::{Mutex, MutexGuard};-use pyo3::exceptions::{PyRuntimeError, PyValueError};-use pyo3::prelude::*;-use std::time::Duration;-use switchyard_components::{BackendSelection, BackendSelectionReason, StatsBackendLatency};-use switchyard_components::{EvictedTargets, LlmTargetId, ModelId, ProxyContext, RequestId};--use super::request::{request_type_from_python, request_type_object};--/// Python-facing proxy context backed by the Rust `ProxyContext`.-#[pyclass(name = "_NativeProxyContext", skip_from_py_object)]-pub(crate) struct PyProxyContext {- /// Shared Rust context guarded across Python and async Rust calls.- inner: Arc<Mutex<ProxyContext>>,- /// Borrow flag that prevents Python mutation while Rust owns the context.- in_use: Arc<AtomicBool>,-}--impl PyProxyContext {- /// Locks the context unless an async Rust component currently owns it.- fn lock(&self) -> PyResult<MutexGuard<'_, ProxyContext>> {- if self.in_use.load(Ordering::Acquire) {- return Err(PyRuntimeError::new_err(- "ProxyContext is already borrowed by an async Rust component",- ));- }- Ok(self.inner.lock())- }-- /// Temporarily moves the Rust context out for async Rust component execution.- pub(crate) fn lease(&self) -> PyResult<PyProxyContextLease> {- if self.in_use.swap(true, Ordering::AcqRel) {- return Err(PyRuntimeError::new_err(- "ProxyContext is already borrowed by an async Rust component",- ));- }- let context = {- let mut guard = self.inner.lock();- std::mem::take(&mut *guard)- };- Ok(PyProxyContextLease {- inner: Arc::clone(&self.inner),- in_use: Arc::clone(&self.in_use),- context: Some(context),- })- }-- /// Inserts a typed Rust extension into the wrapped context.- pub(crate) fn insert_value<T>(&self, value: T) -> PyResult<()>- where- T: Send + Sync + 'static,- {- self.lock()?.insert(value);- Ok(())- }-- /// Returns a clone of a typed Rust extension if one is present.- ///- /// Companion to [`insert_value`] for Python-facing readers of typed- /// extensions stamped by Rust processors (e.g. `ContextSignals`).- pub(crate) fn get_cloned<T>(&self) -> PyResult<Option<T>>- where- T: Clone + Send + Sync + 'static,- {- Ok(self.lock()?.get::<T>().cloned())- }-}--/// Leases the native context carried by a Python `ProxyContext`.-pub(crate) fn lease_from_python(value: &Bound<'_, PyAny>) -> PyResult<PyProxyContextLease> {- value- .getattr("_native")?- .extract::<PyRef<'_, PyProxyContext>>()?- .lease()-}--/// Inserts a typed value into a Python context's native state.-pub(crate) fn insert_into_python<T>(value: &Bound<'_, PyAny>, item: T) -> PyResult<()>-where- T: Send + Sync + 'static,-{- value- .getattr("_native")?- .extract::<PyRef<'_, PyProxyContext>>()?- .insert_value(item)-}--/// Reads a typed value from a Python context's native state.-pub(crate) fn get_cloned_from_python<T>(value: &Bound<'_, PyAny>) -> PyResult<Option<T>>-where- T: Clone + Send + Sync + 'static,-{- value- .getattr("_native")?- .extract::<PyRef<'_, PyProxyContext>>()?- .get_cloned::<T>()-}--/// Temporary ownership lease for passing `ProxyContext` into async Rust roles.-pub(crate) struct PyProxyContextLease {- /// Shared storage that receives the context when the lease is restored.- inner: Arc<Mutex<ProxyContext>>,- /// Borrow flag cleared on restore or drop.- in_use: Arc<AtomicBool>,- /// Moved-out Rust context.- context: Option<ProxyContext>,-}--impl PyProxyContextLease {- /// Returns mutable access to the leased Rust context.- pub(crate) fn context_mut(&mut self) -> PyResult<&mut ProxyContext> {- self.context- .as_mut()- .ok_or_else(|| PyRuntimeError::new_err("ProxyContext lease has already been restored"))- }-- /// Restores the leased context into the Python wrapper.- pub(crate) fn restore(mut self) -> PyResult<()> {- if let Some(context) = self.context.take() {- let mut guard = self.inner.lock();- *guard = context;- }- self.in_use.store(false, Ordering::Release);- Ok(())- }-}--impl Drop for PyProxyContextLease {- /// Restores the context during unwinding and always clears the borrow flag.- fn drop(&mut self) {- if let Some(context) = self.context.take() {- *self.inner.lock() = context;- }- self.in_use.store(false, Ordering::Release);- }-}--#[pymethods]-impl PyProxyContext {- /// Creates a context with a validated native request ID.- ///- /// Metadata is accepted for Python compatibility but remains in the- /// Python-owned `ProxyMetadata` rather than the native context.- #[new]- #[pyo3(signature = (metadata=None, request_id=None))]- fn new(metadata: Option<&Bound<'_, PyAny>>, request_id: Option<String>) -> PyResult<Self> {- let _ = metadata;- let request_id = request_id- .map(RequestId::new)- .transpose()- .map_err(|error| {- PyValueError::new_err(format!("invalid request_id for ProxyContext: {error}"))- })?;-- let mut inner = ProxyContext::default();- inner.request_id = request_id;-- Ok(Self {- inner: Arc::new(Mutex::new(inner)),- in_use: Arc::new(AtomicBool::new(false)),- })- }-- /// Returns the optional request ID.- #[getter]- fn request_id(&self) -> PyResult<Option<String>> {- Ok(self- .lock()?- .request_id- .as_ref()- .map(|request_id| request_id.as_str().to_string()))- }-- /// Updates the optional request ID.- #[setter]- fn set_request_id(&self, value: Option<String>) -> PyResult<()> {- self.lock()?.request_id = value.map(RequestId::new).transpose().map_err(|error| {- PyValueError::new_err(format!("invalid request_id for ProxyContext: {error}"))- })?;- Ok(())- }-- /// Returns the inbound request format as a Python enum object.- #[getter]- fn inbound_format(&self, py: Python<'_>) -> PyResult<Option<Py<PyAny>>> {- self.lock()?- .inbound_format- .map(|request_type| request_type_object(py, request_type))- .transpose()- }-- /// Updates the inbound request format from a Python enum object.- #[setter]- fn set_inbound_format(&self, value: Option<&Bound<'_, PyAny>>) -> PyResult<()> {- self.lock()?.inbound_format = value- .filter(|value| !value.is_none())- .map(request_type_from_python)- .transpose()?;- Ok(())- }-- /// Returns the selected served model, if backend selection exists.- #[getter]- fn selected_model(&self) -> PyResult<Option<String>> {- Ok(self- .lock()?- .get::<BackendSelection>()- .map(|selection| selection.model.as_str().to_string()))- }-- /// Updates the selected served model using backend-selection metadata.- #[setter]- fn set_selected_model(&self, value: Option<String>) -> PyResult<()> {- let mut inner = self.lock()?;- match value {- Some(value) => {- let model = ModelId::new(value).map_err(|error| {- PyValueError::new_err(format!(- "invalid selected_model for ProxyContext: {error}"- ))- })?;- inner.insert(BackendSelection::for_model(- model,- None,- BackendSelectionReason::PassthroughModel,- ));- }- None => {- inner.remove::<BackendSelection>();- }- }- Ok(())- }-- /// Returns the selected target ID, if any.- #[getter]- fn selected_target(&self) -> PyResult<Option<String>> {- Ok(self- .lock()?- .selected_target()- .map(|target| target.as_str().to_string()))- }-- /// Returns the set of target IDs evicted from the routing pool after a- /// context-window overflow on the current request.- /// `None` when no evictions have happened yet.- #[getter]- fn evicted_targets(&self) -> PyResult<Option<Vec<String>>> {- Ok(self.lock()?.get::<EvictedTargets>().map(|evicted| {- let mut ids: Vec<String> = evicted.iter().map(|id| id.as_str().to_string()).collect();- ids.sort();- ids- }))- }-- /// Replaces the evicted target set used by the Python compatibility chain.- #[setter]- fn set_evicted_targets(&self, value: Option<Vec<String>>) -> PyResult<()> {- let mut inner = self.lock()?;- match value {- Some(values) => {- let mut evicted = EvictedTargets::default();- for target in values {- evicted.insert(LlmTargetId::new(target).map_err(|error| {- PyValueError::new_err(format!(- "invalid evicted target for ProxyContext: {error}"- ))- })?);- }- inner.insert(evicted);- }- None => {- inner.remove::<EvictedTargets>();- }- }- Ok(())- }-- /// Returns the measured backend-call latency in ms, if any backend recorded one.- #[getter]- fn backend_call_latency_ms(&self) -> PyResult<Option<f64>> {- Ok(self- .lock()?- .get::<StatsBackendLatency>()- .map(|latency| latency.as_millis_f64()))- }-- /// Records the measured backend-call latency in ms.- ///- /// The Rust ``StatsLlmBackend`` wraps native backends and writes this- /// slot automatically. Python-only backends that can't be wrapped record- /// their measurement here so the downstream ``StatsResponseProcessor`` can- /// compute ``routing_overhead_ms = total_latency - backend_latency`` and- /// emit it on ``/metrics``. Setting ``None`` clears the slot.- #[setter]- fn set_backend_call_latency_ms(&self, value: Option<f64>) -> PyResult<()> {- let mut inner = self.lock()?;- match value {- Some(ms) => {- if !ms.is_finite() || ms < 0.0 {- return Err(PyValueError::new_err(- "backend_call_latency_ms must be a finite, non-negative number",- ));- }- inner.insert(StatsBackendLatency(Duration::from_secs_f64(ms / 1000.0)));- }- None => {- inner.remove::<StatsBackendLatency>();- }- }- Ok(())- }-- /// Updates the selected target ID.- #[setter]- fn set_selected_target(&self, value: Option<String>) -> PyResult<()> {- let mut inner = self.lock()?;- match value {- Some(value) => {- inner.set_selected_target(LlmTargetId::new(value).map_err(|error| {- PyValueError::new_err(format!(- "invalid selected_target for ProxyContext: {error}"- ))- })?);- }- None => {- inner.clear_selected_target();- }- }- Ok(())- }-- /// Returns a compact debug representation for Python users.- fn __repr__(&self) -> PyResult<String> {- let inner = self.lock()?;- let selected_model = inner- .get::<BackendSelection>()- .map(|selection| selection.model.as_str().to_string());- let selected_target = inner- .selected_target()- .map(|target| target.as_str().to_string());- Ok(format!(- "ProxyContext(request_id={:?}, inbound_format={:?}, selected_model={:?}, selected_target={:?})",- inner.request_id, inner.inbound_format, selected_model, selected_target,- ))- }-}--/// Registers proxy context bindings into the Python module.-pub(crate) fn register(module: &Bound<'_, PyModule>) -> PyResult<()> {- module.add_class::<PyProxyContext>()?;- Ok(())-}diff --git a/crates/switchyard-py/src/interop/request.rs b/crates/switchyard-py/src/interop/request.rsdeleted file mode 100644index 376d32722..000000000--- a/crates/switchyard-py/src/interop/request.rs+++ /dev/null@@ -1,70 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Private adapter for Rust-owned chat request values.--use pyo3::exceptions::PyValueError;-use pyo3::prelude::*;-use switchyard_components::{ChatRequest, ChatRequestType};--use crate::py_serde::{value_from_python, value_to_python};--/// Converts a public Python request into the native request value.-pub(crate) fn request_from_python(value: &Bound<'_, PyAny>) -> PyResult<ChatRequest> {- let request_type = request_type_from_python(&value.getattr("request_type")?)?;- let body = value_from_python(&value.getattr("_body")?)?;- Ok(match request_type {- ChatRequestType::OpenAiChat => ChatRequest::openai_chat(body),- ChatRequestType::OpenAiResponses => ChatRequest::openai_responses(body),- ChatRequestType::Anthropic => ChatRequest::anthropic(body),- })-}--/// Converts a native request into the public Python request value.-pub(crate) fn request_to_python(py: Python<'_>, request: ChatRequest) -> PyResult<Py<PyAny>> {- let request_type = request.request_type();- let body = value_to_python(py, request.body())?;- let factory = match request_type {- ChatRequestType::OpenAiChat => "openai_chat",- ChatRequestType::OpenAiResponses => "openai_responses",- ChatRequestType::Anthropic => "anthropic",- };- py.import("switchyard_rust.core")?- .getattr("ChatRequest")?- .call_method1(factory, (body,))- .map(Bound::unbind)-}--pub(crate) fn request_type_from_python(value: &Bound<'_, PyAny>) -> PyResult<ChatRequestType> {- let raw = if let Ok(value_attr) = value.getattr("value") {- value_attr.extract::<String>()?- } else {- value.extract::<String>()?- };- match raw.as_str() {- "openai_chat" => Ok(ChatRequestType::OpenAiChat),- "openai_responses" => Ok(ChatRequestType::OpenAiResponses),- "anthropic" | "anthropic_messages" => Ok(ChatRequestType::Anthropic),- _ => Err(PyValueError::new_err(format!(- "Unknown request type: {raw:?}"- ))),- }-}--pub(crate) fn request_type_variant_name(request_type: ChatRequestType) -> &'static str {- match request_type {- ChatRequestType::OpenAiChat => "OPENAI_CHAT",- ChatRequestType::OpenAiResponses => "OPENAI_RESPONSES",- ChatRequestType::Anthropic => "ANTHROPIC",- }-}--pub(crate) fn request_type_object(- py: Python<'_>,- request_type: ChatRequestType,-) -> PyResult<Py<PyAny>> {- py.import("switchyard_rust.core")?- .getattr("ChatRequestType")?- .getattr(request_type_variant_name(request_type))- .map(Bound::unbind)-}diff --git a/crates/switchyard-py/src/interop/response.rs b/crates/switchyard-py/src/interop/response.rsdeleted file mode 100644index 1e07c0706..000000000--- a/crates/switchyard-py/src/interop/response.rs+++ /dev/null@@ -1,613 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Private adapter for Rust-owned chat response values.--use std::pin::Pin;-use std::sync::Arc;-use std::sync::atomic::{AtomicBool, Ordering};-use std::task::{Context, Poll};--use futures_util::{Stream, StreamExt, stream};-use parking_lot::Mutex;-use pyo3::exceptions::{PyRuntimeError, PyStopAsyncIteration, PyValueError};-use pyo3::prelude::*;-use switchyard_components::{BoxResponseStream, ChatResponse, ChatResponseType, StreamEvent};--use crate::errors::py_core_error;-use crate::py_serde::{value_from_python, value_to_python};--/// Converts a public Python response into the native response value.-pub(crate) fn response_from_python(value: &Bound<'_, PyAny>) -> PyResult<ChatResponse> {- let response_type = response_type_from_python(&value.getattr("response_type")?)?;- Ok(match response_type {- ChatResponseType::OpenAiCompletion => {- ChatResponse::openai_completion(value_from_python(&value.getattr("_body")?)?)- }- ChatResponseType::OpenAiResponsesCompletion => {- ChatResponse::openai_responses_completion(value_from_python(&value.getattr("_body")?)?)- }- ChatResponseType::AnthropicCompletion => {- ChatResponse::anthropic_completion(value_from_python(&value.getattr("_body")?)?)- }- ChatResponseType::OpenAiStream => ChatResponse::OpenAiStream(stream_from_python(value)?),- ChatResponseType::OpenAiResponsesStream => {- ChatResponse::OpenAiResponsesStream(stream_from_python(value)?)- }- ChatResponseType::AnthropicStream => {- ChatResponse::AnthropicStream(stream_from_python(value)?)- }- })-}--/// Converts a native response into the public Python response value.-pub(crate) fn response_to_python(py: Python<'_>, response: ChatResponse) -> PyResult<Py<PyAny>> {- let core = py.import("switchyard_rust.core")?;- let (factory, payload) = match response {- ChatResponse::OpenAiCompletion(response) => (- "openai_completion",- value_to_python(py, &response.into_body())?,- ),- ChatResponse::OpenAiResponsesCompletion(response) => (- "openai_responses_completion",- value_to_python(py, &response.into_body())?,- ),- ChatResponse::AnthropicCompletion(response) => (- "anthropic_completion",- value_to_python(py, &response.into_body())?,- ),- ChatResponse::OpenAiStream(stream) => {- ("openai_stream", stream_to_python(py, &core, stream)?)- }- ChatResponse::OpenAiResponsesStream(stream) => (- "openai_responses_stream",- stream_to_python(py, &core, stream)?,- ),- ChatResponse::AnthropicStream(stream) => {- ("anthropic_stream", stream_to_python(py, &core, stream)?)- }- };- core.getattr("ChatResponse")?- .call_method1(factory, (payload,))- .map(Bound::unbind)-}--fn response_type_from_python(value: &Bound<'_, PyAny>) -> PyResult<ChatResponseType> {- let raw = if let Ok(value_attr) = value.getattr("value") {- value_attr.extract::<String>()?- } else {- value.extract::<String>()?- };- match raw.as_str() {- "openai_completion" => Ok(ChatResponseType::OpenAiCompletion),- "openai_stream" => Ok(ChatResponseType::OpenAiStream),- "openai_responses_completion" => Ok(ChatResponseType::OpenAiResponsesCompletion),- "openai_responses_stream" => Ok(ChatResponseType::OpenAiResponsesStream),- "anthropic_completion" => Ok(ChatResponseType::AnthropicCompletion),- "anthropic_stream" => Ok(ChatResponseType::AnthropicStream),- _ => Err(PyValueError::new_err(format!(- "Unknown response type: {raw:?}"- ))),- }-}--fn stream_from_python(value: &Bound<'_, PyAny>) -> PyResult<BoxResponseStream> {- let native = value.getattr("stream")?.getattr("_native")?;- native- .extract::<PyRef<'_, PyResponseStream>>()?- .take_core_stream()-}--fn stream_to_python(- py: Python<'_>,- core: &Bound<'_, PyModule>,- stream: BoxResponseStream,-) -> PyResult<Py<PyAny>> {- let native = Py::new(py, PyResponseStream::from_core_stream(stream))?;- core.getattr("ChatResponseStream")?- .call_method1("_from_native", (native,))- .map(Bound::unbind)-}--struct PyResponseStreamSource {- source: Py<PyAny>,- iterator: Option<Py<PyAny>>,- done: bool,-}--type BoxPyResponseStream = std::pin::Pin<Box<dyn Stream<Item = PyResult<Py<PyAny>>> + Send>>;--#[pyclass(name = "_NativeChatResponseStream")]-pub(crate) struct PyResponseStream {- stream: Arc<tokio::sync::Mutex<Option<BoxPyResponseStream>>>,- taps: Arc<Mutex<Vec<Py<PyAny>>>>,- maps: Arc<Mutex<Vec<Py<PyAny>>>>,- on_complete: Arc<Mutex<Vec<Py<PyAny>>>>,- consumed: Arc<AtomicBool>,- completed: Arc<AtomicBool>,- // The original Python stream object (e.g. the OpenAI SDK ``AsyncStream``)- // when this stream was built from a Python source. Retained so ``aclose``- // can release the upstream response — and the pooled connection it holds —- // on early termination. ``None`` for Rust-native streams, which own no- // closable Python resource.- source: Option<Py<PyAny>>,-}--impl PyResponseStream {- fn new(source: Py<PyAny>) -> Self {- let retained = Python::attach(|py| source.clone_ref(py));- let mut stream = Self::from_stream(stream_from_python_source(source));- stream.source = Some(retained);- stream- }-- fn from_stream(stream: BoxPyResponseStream) -> Self {- Self {- stream: Arc::new(tokio::sync::Mutex::new(Some(stream))),- taps: Arc::new(Mutex::new(Vec::new())),- maps: Arc::new(Mutex::new(Vec::new())),- on_complete: Arc::new(Mutex::new(Vec::new())),- consumed: Arc::new(AtomicBool::new(false)),- completed: Arc::new(AtomicBool::new(false)),- source: None,- }- }-- fn from_core_stream(stream: BoxResponseStream) -> Self {- Self::from_stream(Box::pin(stream.map(|event| {- Python::attach(|py| match event {- Ok(StreamEvent::Json(value)) => value_to_python(py, &value),- Ok(StreamEvent::Text(value)) => Ok(value.into_pyobject(py)?.unbind().into_any()),- Err(error) => Err(py_core_error(error)),- })- })))- }-- fn take_core_stream(&self) -> PyResult<BoxResponseStream> {- if self.consumed.swap(true, Ordering::AcqRel) {- return Err(PyRuntimeError::new_err(- "ChatResponseStream has already been consumed",- ));- }- let stream = Arc::clone(&self.stream);- let taps = Arc::clone(&self.taps);- let maps = Arc::clone(&self.maps);- let on_complete = Arc::clone(&self.on_complete);- let completed = Arc::clone(&self.completed);- let core: BoxResponseStream = Box::pin(stream::unfold(- (stream, taps, maps, on_complete, completed),- |(stream, taps, maps, on_complete, completed)| async move {- let event = next_stream_item_with_callbacks(- Arc::clone(&stream),- Arc::clone(&taps),- Arc::clone(&maps),- Arc::clone(&on_complete),- Arc::clone(&completed),- )- .await;- event.map(|event| {- let event = match event {- Ok(event) => Python::attach(|py| stream_event_from_python(py, event)),- Err(error) => Err(error),- };- (- event.map_err(|error| {- switchyard_components::SwitchyardError::Processor(error.to_string())- }),- (stream, taps, maps, on_complete, completed),- )- })- },- ));- // Preserve close ownership across the Python -> core -> Python round- // trip: ``from_core_stream`` rebuilds a ``PyResponseStream`` with no- // ``source``, so its ``aclose`` could not release the upstream SDK- // stream. Carrying the source on the core stream — and closing it when- // that stream is dropped — keeps the connection releasable even after- // response processors re-wrap the stream.- match self.source.as_ref() {- Some(source) => {- let source = Python::attach(|py| source.clone_ref(py));- Ok(Box::pin(SourceClosingStream::new(core, source)))- }- None => Ok(core),- }- }-}--#[pymethods]-impl PyResponseStream {- #[new]- fn py_new(source: &Bound<'_, PyAny>) -> Self {- Self::new(source.clone().unbind())- }-- fn tap(slf: PyRef<'_, Self>, callback: &Bound<'_, PyAny>) -> PyResult<Py<PyAny>> {- push_callback(&slf.taps, callback)?;- Ok(slf.into_pyobject(callback.py())?.unbind().into_any())- }-- fn map(slf: PyRef<'_, Self>, callback: &Bound<'_, PyAny>) -> PyResult<Py<PyAny>> {- push_callback(&slf.maps, callback)?;- Ok(slf.into_pyobject(callback.py())?.unbind().into_any())- }-- fn on_complete(slf: PyRef<'_, Self>, callback: &Bound<'_, PyAny>) -> PyResult<Py<PyAny>> {- push_callback(&slf.on_complete, callback)?;- Ok(slf.into_pyobject(callback.py())?.unbind().into_any())- }-- fn __aiter__(slf: PyRef<'_, Self>) -> PyResult<Py<PyAny>> {- if slf.consumed.swap(true, Ordering::AcqRel) {- return Err(PyRuntimeError::new_err(- "ChatResponseStream has already been consumed",- ));- }- let py = slf.py();- Ok(slf.into_pyobject(py)?.unbind().into_any())- }-- fn __anext__<'py>(&self, py: Python<'py>) -> PyResult<Bound<'py, PyAny>> {- let stream = Arc::clone(&self.stream);- let taps = Arc::clone(&self.taps);- let maps = Arc::clone(&self.maps);- let on_complete = Arc::clone(&self.on_complete);- let completed = Arc::clone(&self.completed);-- pyo3_async_runtimes::tokio::future_into_py(py, async move {- match next_stream_item_with_callbacks(stream, taps, maps, on_complete, completed).await- {- Some(event) => event,- None => Err(PyStopAsyncIteration::new_err(())),- }- })- }-- fn __repr__(&self) -> &'static str {- "ChatResponseStream()"- }-- /// Release the upstream stream and its underlying connection.- ///- /// Streaming proxies must close the upstream response when iteration ends- /// early (client disconnect, mid-stream error) — otherwise the SDK- /// ``AsyncStream``'s httpx response is never closed and its pooled- /// connection leaks, exhausting the pool and pinning buffers. Marks the- /// stream consumed and drops the inner adapter; idempotent and safe to- /// call after completion.- ///- /// The upstream source is released by **two** complementary paths, because- /// the source is reachable in only one of two stream shapes:- /// * Built directly from a Python source (``ChatResponseStream(sdk_stream)``,- /// ``source`` is set): closed here, best-effort, via ``close``/``aclose``.- /// * Rebuilt from a core stream by ``from_core_stream`` after the- /// ``Switchyard.call`` round trip (``source`` is ``None``): the source was- /// moved onto the core stream by ``take_core_stream`` as a- /// ``SourceClosingStream``; dropping the inner adapter here drops that- /// wrapper, which closes the source on drop. This is the production path- /// where response processors re-wrap the core stream and ``source``- /// cannot be recovered on this object.- fn aclose<'py>(&self, py: Python<'py>) -> PyResult<Bound<'py, PyAny>> {- self.consumed.store(true, Ordering::Release);- let stream = Arc::clone(&self.stream);- let completed = Arc::clone(&self.completed);- let source = self.source.as_ref().map(|source| source.clone_ref(py));- pyo3_async_runtimes::tokio::future_into_py(py, async move {- // Drop the inner adapter so it releases its borrow of the upstream- // iterator; subsequent ``__anext__`` then yields StopAsyncIteration.- {- let mut guard = stream.lock().await;- *guard = None;- }- completed.store(true, Ordering::Release);- if let Some(source) = source {- close_python_source(source).await;- }- Ok(())- })- }-}--fn push_callback(- callbacks: &Arc<Mutex<Vec<Py<PyAny>>>>,- callback: &Bound<'_, PyAny>,-) -> PyResult<()> {- callbacks.lock().push(callback.clone().unbind());- Ok(())-}--async fn next_stream_item(- stream: Arc<tokio::sync::Mutex<Option<BoxPyResponseStream>>>,-) -> Option<PyResult<Py<PyAny>>> {- let mut guard = stream.lock().await;- match guard.as_mut() {- Some(response_stream) => {- let event = response_stream.next().await;- if event.is_none() {- *guard = None;- }- event- }- None => None,- }-}--async fn next_stream_item_with_callbacks(- stream: Arc<tokio::sync::Mutex<Option<BoxPyResponseStream>>>,- taps: Arc<Mutex<Vec<Py<PyAny>>>>,- maps: Arc<Mutex<Vec<Py<PyAny>>>>,- on_complete: Arc<Mutex<Vec<Py<PyAny>>>>,- completed: Arc<AtomicBool>,-) -> Option<PyResult<Py<PyAny>>> {- match next_stream_item(stream).await {- Some(Ok(event)) => {- run_taps(taps, &event).await;- Some(run_maps(maps, event).await)- }- Some(Err(error)) => Some(Err(error)),- None => {- run_completion_once(on_complete, completed).await;- None- }- }-}--fn stream_from_python_source(source: Py<PyAny>) -> BoxPyResponseStream {- Box::pin(stream::unfold(- PyResponseStreamSource {- source,- iterator: None,- done: false,- },- |mut state| async move {- if state.done {- return None;- }- let future = Python::attach(|py| {- let iterator = match &state.iterator {- Some(iterator) => iterator.clone_ref(py),- None => {- let iterator = state.source.bind(py).call_method0("__aiter__")?.unbind();- state.iterator = Some(iterator.clone_ref(py));- iterator- }- };- let awaitable = iterator.bind(py).call_method0("__anext__")?;- pyo3_async_runtimes::tokio::into_future(awaitable)- });- match future {- Ok(future) => match future.await {- Ok(event) => Some((Ok(event), state)),- Err(error) if is_stop_async_iteration(&error) => None,- Err(error) => {- state.done = true;- Some((Err(error), state))- }- },- Err(error) => {- state.done = true;- Some((Err(error), state))- }- }- },- ))-}--/// Core stream that closes its originating Python source when dropped.-///-/// ``take_core_stream`` converts a Python-backed ``PyResponseStream`` into a-/// Rust-core ``BoxResponseStream`` for runtime processing; ``from_core_stream``-/// later rebuilds a Python wrapper that no longer references the source, so the-/// rebuilt wrapper's ``aclose`` can no longer release the upstream response.-/// Wrapping the core stream here preserves close ownership across that-/// conversion: dropping the stream — directly, or via a response processor that-/// re-wrapped it — schedules a best-effort close of the source so the SDK-/// stream and its pooled connection are released on early termination.-struct SourceClosingStream {- inner: BoxResponseStream,- source: Option<Py<PyAny>>,- // The asyncio event loop + contextvars captured at construction. The- // source's close is a Python coroutine that must be driven on *that* loop,- // not on a bare tokio worker thread; the loop also lets the nested- // ``aclose`` re-enter ``future_into_py`` without a "no running event loop"- // error.- locals: Option<pyo3_async_runtimes::TaskLocals>,- // Runtime captured at construction. ``Drop`` is synchronous and cannot await, so the- // close future is scheduled onto this handle as fire-and-forget.- handle: Option<tokio::runtime::Handle>,-}--impl SourceClosingStream {- fn new(inner: BoxResponseStream, source: Py<PyAny>) -> Self {- Self {- inner,- source: Some(source),- locals: Python::attach(|py| pyo3_async_runtimes::tokio::get_current_locals(py).ok()),- handle: tokio::runtime::Handle::try_current().ok(),- }- }-}--impl Stream for SourceClosingStream {- type Item = switchyard_components::Result<StreamEvent>;-- fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {- // All fields are ``Unpin``, so projecting through ``get_mut`` is sound.- self.get_mut().inner.as_mut().poll_next(cx)- }-}--impl Drop for SourceClosingStream {- fn drop(&mut self) {- let Some(source) = self.source.take() else {- return;- };- let (Some(locals), Some(handle)) = (self.locals.take(), self.handle.take()) else {- // No event loop / runtime to drive the async close (e.g. interpreter- // teardown); dropping the source ref is the best we can do.- tracing::warn!("ChatResponseStream: no event loop to close stream source on drop");- return;- };- // Fire-and-forget on the captured runtime, scoped to the captured event- // loop so the source's async ``aclose`` runs on asyncio. Releasing the- // connection here is what stops the pool leak on client disconnect /- // mid-stream error.- handle.spawn(pyo3_async_runtimes::tokio::scope(locals, async move {- close_python_source(source).await;- }));- }-}--/// Best-effort close of a Python stream source on teardown.-///-/// Async generators expose ``aclose``; SDK ``AsyncStream`` objects expose-/// ``close``. Either may return a coroutine that must be awaited. Closing must-/// never mask the teardown that triggered it, so failures are logged and-/// swallowed rather than propagated.-async fn close_python_source(source: Py<PyAny>) {- let awaitable = match Python::attach(|py| detect_close_awaitable(source.bind(py))) {- Ok(awaitable) => awaitable,- Err(error) => {- tracing::warn!(error = %error, "ChatResponseStream: failed to close stream source");- return;- }- };- let Some(awaitable) = awaitable else {- return;- };- let future =- Python::attach(|py| pyo3_async_runtimes::tokio::into_future(awaitable.bind(py).clone()));- match future {- Ok(future) => {- if let Err(error) = future.await {- tracing::warn!(error = %error, "ChatResponseStream: error closing stream source");- }- }- Err(error) => {- tracing::warn!(- error = %error,- "ChatResponseStream: failed to schedule stream-source close"- );- }- }-}--/// Call ``aclose`` (preferred) or ``close`` on a stream source, returning the-/// coroutine to await when the method is asynchronous. Returns ``None`` when-/// the source exposes no closer or the closer is synchronous (already done).-fn detect_close_awaitable(source: &Bound<'_, PyAny>) -> PyResult<Option<Py<PyAny>>> {- let method = if source.hasattr("aclose")? {- "aclose"- } else if source.hasattr("close")? {- "close"- } else {- return Ok(None);- };- let result = source.call_method0(method)?;- if result.hasattr("__await__")? {- Ok(Some(result.unbind()))- } else {- Ok(None)- }-}--fn stream_event_from_python(py: Python<'_>, event: Py<PyAny>) -> PyResult<StreamEvent> {- let event = event.bind(py);- if let Ok(value) = event.extract::<String>() {- return Ok(StreamEvent::Text(value));- }- Ok(StreamEvent::Json(value_from_python(event)?))-}--async fn run_taps(callbacks: Arc<Mutex<Vec<Py<PyAny>>>>, event: &Py<PyAny>) {- let snapshot = match clone_callbacks(&callbacks) {- Ok(snapshot) => snapshot,- Err(_) => return,- };- let mut failed = Vec::new();- for (index, callback) in snapshot {- if let Err(error) = call_python_callback(callback, event).await {- tracing::warn!(- error = %error,- callback_index = index,- "ChatResponseStream tap failed, quarantining"- );- failed.push(index);- }- }- if failed.is_empty() {- return;- }- let mut callbacks = callbacks.lock();- for index in failed.into_iter().rev() {- if index < callbacks.len() {- callbacks.remove(index);- }- }-}--async fn run_maps(- callbacks: Arc<Mutex<Vec<Py<PyAny>>>>,- mut event: Py<PyAny>,-) -> PyResult<Py<PyAny>> {- let callbacks = clone_callbacks(&callbacks)?;- for (_, callback) in callbacks {- event = call_python_callback(callback, &event).await?;- }- Ok(event)-}--async fn run_completion_once(callbacks: Arc<Mutex<Vec<Py<PyAny>>>>, completed: Arc<AtomicBool>) {- if completed.swap(true, Ordering::AcqRel) {- return;- }- let snapshot = match clone_callbacks(&callbacks) {- Ok(snapshot) => snapshot,- Err(_) => return,- };- for (index, callback) in snapshot {- if let Err(error) = call_python_callback_no_args(callback).await {- tracing::warn!(- error = %error,- callback_index = index,- "ChatResponseStream completion callback failed"- );- }- }-}--fn clone_callbacks(callbacks: &Arc<Mutex<Vec<Py<PyAny>>>>) -> PyResult<Vec<(usize, Py<PyAny>)>> {- Python::attach(|py| {- let callbacks = callbacks.lock();- Ok(callbacks- .iter()- .enumerate()- .map(|(index, callback)| (index, callback.clone_ref(py)))- .collect::<Vec<_>>())- })-}--async fn call_python_callback(callback: Py<PyAny>, arg: &Py<PyAny>) -> PyResult<Py<PyAny>> {- let future = Python::attach(|py| {- let result = callback.bind(py).call1((arg.clone_ref(py),))?;- pyo3_async_runtimes::tokio::into_future(result)- })?;- future.await-}--async fn call_python_callback_no_args(callback: Py<PyAny>) -> PyResult<Py<PyAny>> {- let future = Python::attach(|py| {- let result = callback.bind(py).call0()?;- pyo3_async_runtimes::tokio::into_future(result)- })?;- future.await-}--fn is_stop_async_iteration(error: &PyErr) -> bool {- Python::attach(|py| error.is_instance_of::<PyStopAsyncIteration>(py))-}--pub(crate) fn register(module: &Bound<'_, PyModule>) -> PyResult<()> {- module.add_class::<PyResponseStream>()?;- Ok(())-}diff --git a/crates/switchyard-py/src/interop/roles.rs b/crates/switchyard-py/src/interop/roles.rsdeleted file mode 100644index 732ec2acc..000000000--- a/crates/switchyard-py/src/interop/roles.rs+++ /dev/null@@ -1,119 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Private adapter for Rust-owned backend role abstractions.--use std::sync::Arc;--use pyo3::PyTypeInfo;-use pyo3::exceptions::{PyNotImplementedError, PyTypeError};-use pyo3::prelude::*;-use pyo3::types::{PyDict, PyTuple, PyType};-use switchyard_components::LlmBackend;--use super::context::lease_from_python;-use super::request::{request_from_python, request_type_object};-use super::response::response_to_python;-use crate::errors::py_core_error;--#[pyclass(name = "_NativeLlmBackend", subclass)]-pub(crate) struct PyLlmBackend {- inner: Option<Arc<dyn LlmBackend>>,-}--impl PyLlmBackend {- pub(crate) fn from_native(inner: Arc<dyn LlmBackend>) -> Self {- Self { inner: Some(inner) }- }-- pub(crate) fn native(&self) -> Option<Arc<dyn LlmBackend>> {- self.inner.clone()- }-}--#[pymethods]-impl PyLlmBackend {- #[new]- #[classmethod]- #[pyo3(signature = (*_args, **_kwargs))]- fn py_new(- cls: &Bound<'_, PyType>,- _args: &Bound<'_, PyTuple>,- _kwargs: Option<&Bound<'_, PyDict>>,- ) -> PyResult<Self> {- reject_base_instantiation::<Self>(cls, "LLMBackend")?;- Ok(Self { inner: None })- }-- #[getter]- fn supported_request_types(&self, py: Python<'_>) -> PyResult<Vec<Py<PyAny>>> {- let backend = self.inner.as_ref().ok_or_else(|| {- PyNotImplementedError::new_err("LLMBackend.supported_request_types must be implemented")- })?;- backend- .supported_request_types()- .iter()- .map(|request_type| request_type_object(py, *request_type))- .collect()- }-- fn startup<'py>(&self, py: Python<'py>) -> PyResult<Bound<'py, PyAny>> {- match self.inner.clone() {- Some(backend) => pyo3_async_runtimes::tokio::future_into_py(py, async move {- backend.startup().await.map_err(py_core_error)- }),- None => pyo3_async_runtimes::tokio::future_into_py(py, async { Ok(()) }),- }- }-- fn shutdown<'py>(&self, py: Python<'py>) -> PyResult<Bound<'py, PyAny>> {- match self.inner.clone() {- Some(backend) => pyo3_async_runtimes::tokio::future_into_py(py, async move {- backend.shutdown().await.map_err(py_core_error)- }),- None => pyo3_async_runtimes::tokio::future_into_py(py, async { Ok(()) }),- }- }-- fn call<'py>(- &self,- py: Python<'py>,- ctx: &Bound<'_, PyAny>,- request: &Bound<'_, PyAny>,- ) -> PyResult<Bound<'py, PyAny>> {- let backend = self- .inner- .clone()- .ok_or_else(|| PyNotImplementedError::new_err("LLMBackend.call must be implemented"))?;- let mut lease = lease_from_python(ctx)?;- let request = request_from_python(request)?;- pyo3_async_runtimes::tokio::future_into_py(py, async move {- let result = backend.call(lease.context_mut()?, &request).await;- let restore_result = lease.restore();- let response = result.map_err(py_core_error)?;- restore_result?;- Python::attach(|py| response_to_python(py, response))- })- }-- fn __repr__(&self) -> &'static str {- "LLMBackend()"- }-}--fn reject_base_instantiation<T: PyTypeInfo>(- cls: &Bound<'_, PyType>,- name: &'static str,-) -> PyResult<()> {- if cls.is(cls.py().get_type::<T>()) {- return Err(PyTypeError::new_err(format!(- "can't instantiate abstract role {name}"- )));- }- Ok(())-}--pub(crate) fn register(module: &Bound<'_, PyModule>) -> PyResult<()> {- module.add_class::<PyLlmBackend>()?;- Ok(())-}diff --git a/crates/switchyard-py/src/interop/subagent.rs b/crates/switchyard-py/src/interop/subagent.rsdeleted file mode 100644index c74885cda..000000000--- a/crates/switchyard-py/src/interop/subagent.rs+++ /dev/null@@ -1,51 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! Python binding for the canonical sub-agent detection policy.-//!-//! Routing implementations must not sniff lineage headers themselves: the fact (explicit-//! `x-switchyard-is-subagent`, Claude Code agent lineage, Codex/relay markers) and the-//! work-vs-maintenance policy both live in the protocol crate, so every engine — the libsy-//! classifier and the server alike — answers "is this delegated work?"-//! identically.--use std::collections::HashMap;--use http::header::{HeaderName, HeaderValue};-use pyo3::exceptions::PyValueError;-use pyo3::prelude::*;--/// Convert Python-owned headers without allowing invalid or excessive entries.-pub(crate) fn header_map_from_python(- headers: &HashMap<String, String>,-) -> PyResult<http::HeaderMap> {- let mut result = http::HeaderMap::new();-- for (name, value) in headers {- let name = HeaderName::from_bytes(name.as_bytes())- .map_err(|error| PyValueError::new_err(error.to_string()))?;- let value = HeaderValue::from_str(value)- .map_err(|error| PyValueError::new_err(error.to_string()))?;- result- .try_append(name, value)- .map_err(|error| PyValueError::new_err(error.to_string()))?;- }-- Ok(result)-}--/// Whether `headers` mark this request as delegated sub-agent *work*.-///-/// Wraps [`switchyard_protocol::Metadata::from_headers`] for the lineage fact and-/// `is_subagent_work` for the kind policy, so harness maintenance turns (Codex `compact`,-/// `memory_consolidation`) stay on normal routing rather than being sent to a worker target.-#[pyfunction]-fn is_subagent_request(headers: HashMap<String, String>) -> PyResult<bool> {- let headers = header_map_from_python(&headers)?;- Ok(switchyard_protocol::Metadata::from_headers(&headers).is_subagent_work())-}--pub(crate) fn register(module: &Bound<'_, PyModule>) -> PyResult<()> {- module.add_function(wrap_pyfunction!(is_subagent_request, module)?)?;- Ok(())-}diff --git a/crates/switchyard-py/src/lib.rs b/crates/switchyard-py/src/lib.rsindex 3082697a1..1749c9ce4 100644--- a/crates/switchyard-py/src/lib.rs+++ b/crates/switchyard-py/src/lib.rs@@ -3,20 +3,14 @@use pyo3::prelude::*;-mod component_bindings;mod errors;-mod interop;mod libsy_bindings;mod py_serde;mod server_bindings;-mod translation;#[pymodule]fn _switchyard_rust(module: &Bound<'_, PyModule>) -> PyResult<()> {errors::register(module)?;- translation::register(module)?;- interop::register(module)?;- component_bindings::register(module)?;libsy_bindings::register(module)?;server_bindings::register(module)?;Ok(())diff --git a/crates/switchyard-py/src/libsy_bindings.rs b/crates/switchyard-py/src/libsy_bindings.rsindex afc647fd7..8be1bede0 100644--- a/crates/switchyard-py/src/libsy_bindings.rs+++ b/crates/switchyard-py/src/libsy_bindings.rs@@ -3,9 +3,11 @@//! Minimal Python API for running Rust-owned libsy algorithms.+use std::collections::HashMap;use std::sync::Arc;use async_trait::async_trait;+use http::header::{HeaderName, HeaderValue};use pyo3::exceptions::{PyTypeError, PyValueError};use pyo3::prelude::*;use serde_json::{Value, json};@@ -21,9 +23,23 @@ use switchyard_protocol::{};use crate::errors::py_libsy_error;-use crate::interop::subagent::header_map_from_python;use crate::py_serde::{from_python, to_python};+/// Convert Python-owned headers into the request metadata expected by libsy.+fn header_map_from_python(headers: &HashMap<String, String>) -> PyResult<http::HeaderMap> {+ let mut result = http::HeaderMap::new();+ for (name, value) in headers {+ let name = HeaderName::from_bytes(name.as_bytes())+ .map_err(|error| PyValueError::new_err(error.to_string()))?;+ let value = HeaderValue::from_str(value)+ .map_err(|error| PyValueError::new_err(error.to_string()))?;+ result+ .try_append(name, value)+ .map_err(|error| PyValueError::new_err(error.to_string()))?;+ }+ Ok(result)+}+/// Adapts a Python object with `async call(request)` to libsy.struct PythonLlmClient {inner: Py<PyAny>,diff --git a/crates/switchyard-py/src/py_serde.rs b/crates/switchyard-py/src/py_serde.rsindex f02f962d3..8a79e16bd 100644--- a/crates/switchyard-py/src/py_serde.rs+++ b/crates/switchyard-py/src/py_serde.rs@@ -8,7 +8,6 @@ use pyo3::prelude::*;use pyo3::types::PyDict;use pythonize::{depythonize, pythonize};use serde::{Serialize, de::DeserializeOwned};-use serde_json::Value;/// Converts a Python mapping-like object into a Serde-owned Rust value.pub(crate) fn from_python<T: DeserializeOwned>(value: &Bound<'_, PyAny>) -> PyResult<T> {@@ -24,16 +23,6 @@ pub(crate) fn to_python<T: Serialize>(py: Python<'_>, value: &T) -> PyResult<Py<.map_err(|error| PyValueError::new_err(error.to_string()))}-/// Converts an arbitrary Python object into a JSON value.-pub(crate) fn value_from_python(value: &Bound<'_, PyAny>) -> PyResult<Value> {- from_python(value)-}--/// Converts a JSON value into a Python object.-pub(crate) fn value_to_python(py: Python<'_>, value: &Value) -> PyResult<Py<PyAny>> {- to_python(py, value)-}-fn jsonable_python(value: &Bound<'_, PyAny>) -> PyResult<Py<PyAny>> {if let Ok(model_dump) = value.getattr("model_dump")&& model_dump.is_callable()diff --git a/crates/switchyard-py/src/translation.rs b/crates/switchyard-py/src/translation.rsdeleted file mode 100644index 5f86d7bf5..000000000--- a/crates/switchyard-py/src/translation.rs+++ /dev/null@@ -1,123 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--//! PyO3 bindings for `switchyard-translation`.--use pyo3::prelude::*;-use serde_json::Value;-use switchyard_translation::{- StreamTranslationState, TranslationEngine, TranslationPolicy, normalize_anthropic_tool_use_ids,-};--use crate::errors::py_translation_error;-use crate::py_serde::{value_from_python, value_to_python};--#[pyclass(name = "TranslationEngine")]-struct PyTranslationEngine {- inner: TranslationEngine,- policy: TranslationPolicy,-}--#[pymethods]-impl PyTranslationEngine {- #[new]- fn new() -> Self {- Self {- inner: TranslationEngine::default(),- policy: TranslationPolicy::default(),- }- }-- fn translate_request(- &self,- py: Python<'_>,- source: &str,- target: &str,- body: &Bound<'_, PyAny>,- ) -> PyResult<Py<PyAny>> {- let body = value_from_python(body)?;- let output = self- .inner- .translate_request(source, target, &body, &self.policy)- .map_err(py_translation_error)?;- value_to_python(py, &output.body)- }-- fn translate_response(- &self,- py: Python<'_>,- source: &str,- target: &str,- body: &Bound<'_, PyAny>,- ) -> PyResult<Py<PyAny>> {- let body = value_from_python(body)?;- let output = self- .inner- .translate_response(source, target, &body, &self.policy)- .map_err(py_translation_error)?;- value_to_python(py, &output.body)- }-- #[pyo3(signature = (source, target, model=None, message_id=None))]- fn stream(- &self,- source: &str,- target: &str,- model: Option<String>,- message_id: Option<String>,- ) -> PyStreamTranslation {- let mut state = StreamTranslationState::new(source, target);- state.target_model = model;- state.target_message_id = message_id;- PyStreamTranslation {- source: source.to_string(),- target: target.to_string(),- state,- inner: TranslationEngine::default(),- }- }-- fn normalize_anthropic_tool_use_ids(- &self,- py: Python<'_>,- messages: &Bound<'_, PyAny>,- ) -> PyResult<Py<PyAny>> {- let messages = value_from_python(messages)?;- value_to_python(py, &normalize_anthropic_tool_use_ids(messages))- }-}--#[pyclass(name = "StreamTranslation")]-struct PyStreamTranslation {- source: String,- target: String,- state: StreamTranslationState,- inner: TranslationEngine,-}--#[pymethods]-impl PyStreamTranslation {- fn translate_event(&mut self, py: Python<'_>, event: &Bound<'_, PyAny>) -> PyResult<Py<PyAny>> {- let event = value_from_python(event)?;- let output = self- .inner- .translate_event(&mut self.state, &self.source, &self.target, &event)- .map_err(py_translation_error)?;- value_to_python(py, &Value::Array(output))- }-- fn finish(&mut self, py: Python<'_>) -> PyResult<Py<PyAny>> {- let output = self- .inner- .finish_stream(&mut self.state, &self.target)- .map_err(py_translation_error)?;- value_to_python(py, &Value::Array(output))- }-}--/// Registers translation bindings with the native Python module.-pub(crate) fn register(module: &Bound<'_, PyModule>) -> PyResult<()> {- module.add_class::<PyTranslationEngine>()?;- module.add_class::<PyStreamTranslation>()?;- Ok(())-}diff --git a/crates/switchyard-server/README.md b/crates/switchyard-server/README.mdindex dd57c57d7..5e99a4d9b 100644--- a/crates/switchyard-server/README.md+++ b/crates/switchyard-server/README.md@@ -169,11 +169,8 @@ fixed error categories.`switchyard_total_latency_ms` observes an aggregate when it becomes available or a stream when itends cleanly. Its clock starts in a router-wide middleware, before the request body is read and-decoded, so it covers the same span as the Python server's request-ingress-to-completion-measurement. It still excludes connection accept and TLS handshake, which hyper completes before-the server sees the request. The Rust server exports this metric as a histogram, while the Python-server exports its counterpart as a summary; this matches the existing histogram/summary difference-for model-call latency.+decoded, so it measures request ingress through response completion. It still excludes connection+accept and TLS handshake, which hyper completes before the server sees the request.`switchyard_routing_overhead_ms` is what routing cost on top of the model call: the algorithm's runtime minus the call that served the request. Classifier calls are not subtracted, so andiff --git a/docs/cli_reference.md b/docs/cli_reference.mdindex 9f50c693e..b2374ecd7 100644--- a/docs/cli_reference.md+++ b/docs/cli_reference.md@@ -9,7 +9,7 @@ Switchyard has two command-line paths:## Launcher Path: `switchyard launch`-Install the launcher with `uv tool install --python 3.12 "nemo-switchyard[cli,server]"`. The+Install the launcher with `uv tool install --python 3.12 "nemo-switchyard[cli]"`. Theselected coding agent must also be installed and available on `PATH`.### Usage@@ -83,7 +83,8 @@ switchyard-server --config routes.toml \## Removed Setup Commands-`switchyard configure` and `switchyard verify` are not available. Export the+`switchyard configure`, `switchyard serve`, and `switchyard verify` are not+available. Export theenvironment variable named by `api_key_env` in the native TOML deployment,then pass that deployment on each run. For example:@@ -97,7 +98,8 @@ export PROVIDER_API_KEY="your-provider-key" # pragma: allowlist secretswitchyard launch claude --model my-route --config routes.toml```-The CLI does not save provider credentials or deployment paths.+The CLI does not save provider credentials, deployment paths, or host a+standalone server.Use `switchyard-server --config routes.toml --dry-run` tovalidate a native deployment before starting the standalone server.diff --git a/docs/getting_started.md b/docs/getting_started.mdindex fc03dad73..74207065f 100644--- a/docs/getting_started.md+++ b/docs/getting_started.md@@ -27,11 +27,11 @@ source "$HOME/.local/bin/env"Then install the published Switchyard tool:```bash-uv tool install --python 3.12 "nemo-switchyard[cli,server]"+uv tool install --python 3.12 "nemo-switchyard[cli]"```This creates an isolated Python tool environment containing the `switchyard`-CLI, its CLI and server dependencies, and the packaged PyO3 Rust extension.+CLI, its launcher dependency, and the packaged PyO3 Rust extension.`switchyard launch` starts the native Rust server through that extension; thispath does not install or run the standalone `switchyard-server` binary.@@ -122,8 +122,7 @@ Cargo builds the release binary and installs it into `~/.cargo/bin` by default.### Configure-The Rust server reads an explicit TOML file. It does not use the Python-server's minimal YAML route bundle.+The Rust server reads an explicit TOML file.Create `routes.toml` with an LLM-classifier route:diff --git a/docs/internal/metrics_reference.md b/docs/internal/metrics_reference.mdindex aea807d9e..0d29bb59f 100644--- a/docs/internal/metrics_reference.md+++ b/docs/internal/metrics_reference.md@@ -14,8 +14,7 @@ a drop-in scrape config and starter alert rules.| Auth | None || Default scrape interval | 15s |-`GET /metrics` is served by the Python route-bundle server started with-`switchyard serve --routes PATH` and by the native Rust server.+`GET /metrics` is served by the native Rust server.A JSON summary of the same traffic lives at `GET /v1/stats`.diff --git a/docs/routing_algorithms/stage_router_routing.md b/docs/routing_algorithms/stage_router_routing.mdindex bba42fbd2..26bee1189 100644--- a/docs/routing_algorithms/stage_router_routing.md+++ b/docs/routing_algorithms/stage_router_routing.md@@ -151,31 +151,10 @@ From the overlap tasks (those with both capable and efficient results):- `SAFE` = both pass- `HARD` = both fail-**Running the sweep**--Replay your runs through the real Rust scorer and picker with-`benchmark/score_staged_run.py` (the `switchyard-stage-router-scorer` skill). It emits-per-turn scores and per-task routing splits at a given threshold and window —-the actual `capable_first` / `efficient_first` picker decisions, not a-counterfactual:--```bash-# Score a probe run at a candidate threshold-uv run python benchmark/score_staged_run.py --run benchmark/tb_runs/<your_run> \- --threshold 0.5 --window 3-# → /tmp/<run>-scores.jsonl (per turn: score, confidence, pick_cf, pick_ef)-# → /tmp/<run>-per-task.csv (per task: routing split, mean score/confidence)-```--Sweep a few candidate thresholds and read the routing split and pass rate off-the per-task CSV; the lowest threshold that rescues the RESCUE quadrant without-over-escalating the LOSS quadrant is your calibrated value. Because the scorer-is corroborative, a `0.5` threshold takes ~1.5 signals of agreement — a policy-that escalates ~20% of tasks maps roughly to `confidence_threshold: 0.5` with-`capable_first`.--Signals come from the actual picker replay, so even 15–20 probe tasks give a-stable result.+Sweep a few candidate thresholds in representative benchmark runs. Choose the+lowest threshold that rescues the RESCUE quadrant without over-escalating the+LOSS quadrant. Because the scorer is corroborative, a `0.5` threshold takes+roughly 1.5 signals of agreement.**Caveat on efficient outcomes in stage-router vs. pure-efficient**diff --git a/examples/minimal.py b/examples/minimal.pydeleted file mode 100644index 95b5a610b..000000000--- a/examples/minimal.py+++ /dev/null@@ -1,62 +0,0 @@-#!/usr/bin/env python3-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""-Minimal Switchyard Example--This example demonstrates how to use Switchyard as a Python library-to route LLM requests through a backend.--Usage:- export OPENROUTER_API_KEY="sk-or-..."- python examples/minimal.py-"""--import asyncio-import sys-from pathlib import Path--# Add package to path for development (not needed when installed via pip)-sys.path.insert(0, str(Path(__file__).parent.parent))--from switchyard import ChatRequest-from switchyard.cli.route_bundle import load_route_bundle_table---async def main() -> None:- """Run a minimal Switchyard example."""-- routes = load_route_bundle_table(Path(__file__).with_name("route.yaml"))- switchyard = routes.lookup_switchyard("fast-kimi")-- print("=" * 60)- print("Switchyard Minimal Example")- print("=" * 60)-- # Create a chat request- request = ChatRequest.openai_chat({- "model": "fast-kimi",- "messages": [- {"role": "system", "content": "You are a helpful assistant."},- {"role": "user", "content": "What is 2+2?"},- ],- "max_tokens": 100,- })-- print(f"Sending request to {request.body['model']}...")-- # Call the LLM through the switchyard- response = await switchyard.call(request)-- print("\nResponse:")- print(f" Content: {response['choices'][0]['message']['content']}")- print(f" Tokens: {response['usage']['total_tokens']}")-- print("\n" + "=" * 60)- print("Example completed!")- print("=" * 60)---if __name__ == "__main__":- asyncio.run(main())diff --git a/examples/route.yaml b/examples/route.yamldeleted file mode 100644index 5b3ef858e..000000000--- a/examples/route.yaml+++ /dev/null@@ -1,12 +0,0 @@-defaults:- api_key: ${OPENROUTER_API_KEY}- base_url: https://openrouter.ai/api/v1- format: openai--routes:- fast-kimi:- type: passthrough- target: moonshotai/kimi-k2.6-- smoke-test:- type: noopdiff --git a/examples/utils.py b/examples/utils.pydeleted file mode 100644index 5d9527b1b..000000000--- a/examples/utils.py+++ /dev/null@@ -1,97 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""-Utility functions for Switchyard examples.--This module provides common configuration loading for all examples.-"""--import json-import os-from pathlib import Path--# Repository root (parent of examples folder)-REPO_ROOT = Path(__file__).parent.parent--# Path to secrets file-SECRETS_FILE = REPO_ROOT / "secrets" / "secrets.json"--# Defaults for OpenAI API-DEFAULT_BASE_URL = "https://api.openai.com/v1"-DEFAULT_MODEL = "gpt-4o-mini"---def load_secrets() -> dict:- """- Load secrets from secrets/secrets.json if it exists.-- Returns:- Dictionary with secrets, or empty dict if file doesn't exist.- """- if SECRETS_FILE.exists():- with open(SECRETS_FILE) as f:- return json.load(f)- return {}---def get_config() -> dict:- """- Get API configuration from environment variables or secrets file.-- Priority:- 1. Environment variables (OPENAI_API_KEY, OPENAI_BASE_URL)- 2. Secrets file (secrets/secrets.json)- 3. Defaults for base_url and model-- Returns:- Dictionary with api_key, base_url, and model.- """- secrets = load_secrets()- openai_secrets = secrets.get("openai", {})-- return {- "api_key": os.environ.get("OPENAI_API_KEY") or openai_secrets.get("api_key"),- "base_url": (- os.environ.get("OPENAI_BASE_URL")- or openai_secrets.get("base_url")- or DEFAULT_BASE_URL- ),- "model": (- os.environ.get("OPENAI_MODEL")- or openai_secrets.get("model")- or DEFAULT_MODEL- ),- }---def print_config(config: dict) -> None:- """Print configuration in a consistent format."""- api_key = config["api_key"]- print("Configuration:")- if api_key:- print(f" API Key: {'*' * 8}...{api_key[-4:]}")- else:- print(" API Key: Not set")- print(f" Base URL: {config['base_url']}")- print(f" Model: {config['model']}")- print()---def check_api_key(config: dict) -> bool:- """- Check if API key is available and print error message if not.-- Returns:- True if API key is available, False otherwise.- """- if config["api_key"]:- return True-- print("Error: API key not found.")- print()- print("Set it via environment variable:")- print(" export OPENAI_API_KEY='sk-...'")- print()- print("Or create secrets/secrets.json (copy from secrets/secrets.template.json)")- return Falsediff --git a/pyproject.toml b/pyproject.tomlindex 30203d106..857dbb893 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -23,51 +23,15 @@ classifiers = ["Topic :: Software Development :: Libraries :: Python Modules",]-# Core dependencies — minimal set for the foundation library (core/ + recipes/).-# No server, CLI, ML, GPU, or orchestrator packages.-# Users install extras based on their use case:-# pip install nemo-switchyard # Core + recipes only-# pip install nemo-switchyard[server] # Add FastAPI/Uvicorn for e2e-# pip install nemo-switchyard[cli] # Add prompt-toolkit for CLI-# pip install nemo-switchyard[all] # Everything-dependencies = [- # openai: request/response schema types, plus the async client in lib/llm_client.py.- # Keep this floor low enough for downstream consumers to co-install us; NeMo Gym pins- # openai<=2.7.2. The suite passes unchanged from 2.7.0 through 2.48.0.- "openai>=2.7,<3.0",- "anthropic>=0.99.0,<1.0",- "httpx>=0.28.1,<1.0",- "pydantic>=2.13.3,<3.0",-]+dependencies = [][project.optional-dependencies]-# Server dependencies — FastAPI + Uvicorn for e2e users who want to run-# switchyard as a proxy. Not needed for library-only usage.-server = [- "fastapi>=0.136.1,<1.0",- "uvicorn[standard]>=0.46.0,<1.0",- "sse-starlette>=3.4.1,<4.0",-]-# CLI dependencies — prompt-toolkit for Claude Code launcher and ShellTUI.# Only needed for users running the switchyard CLI.cli = ["prompt-toolkit>=3.0.52,<4.0",]-# Tracing — ddtrace for Datadog APM spans on the proxy routing path-# Optional: ``switchyard.lib.tracing`` no-ops when ddtrace is not-# importable, so the default install and non-Datadog deployments are unaffected-# and pay no overhead.-tracing = [- "ddtrace>=2.9,<4",-]--# Everything — all optional dependencies for full-featured deployment.-all = [- "nemo-switchyard[server,cli,tracing]",-]-# Dev tooling lives in a PEP 735 dependency group rather than an optional# extra so it is not advertised in the published wheel's METADATA. This# keeps pytest / ruff / mypy / pygments-via-pytest out of downstream@@ -82,19 +46,13 @@ dev = ["maturin>=1.9,<2.0","pytest>=9.0.3,<10.0","pytest-asyncio>=1.3.0,<2.0",- "pytest-cov>=7.1.0,<8.0",- "pytest-mock>=3.15.1,<4.0","pytest-timeout>=2.4.0,<3.0",- "respx>=0.23.1,<1.0","ruff>=0.15.12,<1.0","mypy>=1.20.2,<2.0",- "httpx>=0.28.1,<1.0",- "socksio>=1.0.0,<2.0",- "prometheus-client>=0.21.0,<1.0",# Harbor is needed for local evaluation runs and supports the package floor.# Package users still do not see it in published metadata."harbor @ git+https://github.com/harbor-framework/[email protected] ; python_version >= '3.12'",- "nemo-switchyard[server]",+ "nemo-switchyard[cli]","pytest-markdown-docs>=0.9.2",]docs = [@@ -122,8 +80,6 @@ include = ["switchyard*"]switchyard = ["py.typed","cli/defaults/*.toml",- "lib/processors/prompts/*.md",- "lib/processors/stage_router/prompts/*.md",]switchyard_rust = ["py.typed", "*.pyi"]@@ -158,23 +114,11 @@ warn_unused_configs = truewarn_unreachable = trueignore_missing_imports = true-# Problem 2 scoreboard: every `data: Any` annotation is a tracked error.-# 303 violations today — each one fixed = one step toward typed chain.-# Disabled for open-source release; can be re-enabled as part of gradual typing effort.-# disallow_any_explicit = true-# Readabilitypretty = trueshow_error_codes = trueshow_column_numbers = true-# TODO: enable after Phase 6 migration (3,271 violations today)-# disallow_any_expr = true--[[tool.mypy.overrides]]-module = "tests.*"-ignore_errors = true-[tool.pyright]include = ["switchyard"]exclude = [diff --git a/switchyard/__init__.py b/switchyard/__init__.pyindex ae1574d2f..d978b6355 100644--- a/switchyard/__init__.py+++ b/switchyard/__init__.py@@ -1,146 +1,14 @@# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.# SPDX-License-Identifier: Apache-2.0-"""-Switchyard - Typed LLM routing and orchestration.--This library provides a composable, type-safe foundation for routing-requests across multiple LLM backends with intelligent tier selection,-format translation, and extensible middleware.-"""+"""Switchyard's Python launcher and libsy bindings."""from importlib import metadata as _metadata-from typing import TYPE_CHECKING, Any--from switchyard.lib.backends import (- AnthropicNativeBackend,- OpenAiNativeBackend,-)-from switchyard.lib.backends.llm_target import (- BackendFormat,- LlmTarget,-)-from switchyard.lib.chat_request import (- AnthropicChatRequest,- OpenAIChatRequest,- ResponsesChatRequest,-)-from switchyard.lib.chat_response import (- AnthropicChatResponse,- AnthropicResponseStream,- AnthropicStreamingChatResponse,- AnyResponseStream,- CompletionChatResponse,- ResponsesApiChatResponse,- ResponsesApiStream,- ResponsesApiStreamingChatResponse,- ResponseStream,- StreamingChatResponse,-)-from switchyard.lib.processors.rl_logging_request_processor import RlLoggingRequestProcessor-from switchyard.lib.processors.rl_logging_response_processor import RlLoggingResponseProcessor-from switchyard.lib.request_metadata import RequestMetadata-from switchyard.lib.roles import (- LLMBackend,-)-from switchyard.lib.route_table import RouteTable-from switchyard.lib.switchyard import Switchyard-from switchyard_rust.components import RandomRoutingProcessorConfig-from switchyard_rust.core import (- ChatRequest,- ChatRequestType,- ChatResponse,- ChatResponseType,-)-from switchyard_rust.translation import TranslationEngine--if TYPE_CHECKING:- from switchyard.lib.endpoints.anthropic_messages_endpoint import (- AnthropicMessagesEndpoint,- )- from switchyard.lib.endpoints.models_endpoint import ModelsEndpoint- from switchyard.lib.endpoints.openai_chat_endpoint import (- OpenAIChatEndpoint,- )- from switchyard.lib.endpoints.responses_endpoint import ResponsesEndpoint- from switchyard.server.switchyard_app import build_switchyard_app---def __getattr__(name: str) -> Any:- """Lazy-load optional server exports that require the ``server`` extra."""- if name == "OpenAIChatEndpoint":- from switchyard.lib.endpoints.openai_chat_endpoint import (- OpenAIChatEndpoint,- )-- return OpenAIChatEndpoint- if name == "AnthropicMessagesEndpoint":- from switchyard.lib.endpoints.anthropic_messages_endpoint import (- AnthropicMessagesEndpoint,- )-- return AnthropicMessagesEndpoint- if name == "ResponsesEndpoint":- from switchyard.lib.endpoints.responses_endpoint import ResponsesEndpoint-- return ResponsesEndpoint- if name == "ModelsEndpoint":- from switchyard.lib.endpoints.models_endpoint import ModelsEndpoint-- return ModelsEndpoint- if name == "build_switchyard_app":- from switchyard.server.switchyard_app import build_switchyard_app-- return build_switchyard_app- raise AttributeError(f"module {__name__!r} has no attribute {name!r}")---__all__ = [- # ChatRequest types- "AnthropicChatRequest",- "ChatRequest",- "ChatRequestType",- "OpenAIChatRequest",- "ResponsesChatRequest",- # Chain infrastructure- "Switchyard",- "LLMBackend",- "AnthropicNativeBackend",- "OpenAiNativeBackend",- "OpenAIChatEndpoint",- "AnthropicMessagesEndpoint",- "ResponsesEndpoint",- "ModelsEndpoint",- "build_switchyard_app",- # Route dispatch table- "RouteTable",- "RequestMetadata",- "RlLoggingRequestProcessor",- "RlLoggingResponseProcessor",- # Random Routing usage case- "BackendFormat",- "RandomRoutingProcessorConfig",- "LlmTarget",- # Deterministic (LLM-classifier) routing usage case- # Translation engine- "TranslationEngine",- # ChatResponse types- "AnthropicChatResponse",- "ChatResponse",- "ChatResponseType",- "CompletionChatResponse",- "StreamingChatResponse",- "ResponsesApiChatResponse",- "ResponsesApiStreamingChatResponse",- "AnthropicStreamingChatResponse",- "ResponseStream",- "ResponsesApiStream",- "AnthropicResponseStream",- "AnyResponseStream",-]try:__version__ = _metadata.version("nemo-switchyard")except _metadata.PackageNotFoundError:# A source checkout may not have installed distribution metadata.__version__ = "0.0.0+unknown"++__all__ = ["__version__"]diff --git a/switchyard/cli/command_utils.py b/switchyard/cli/command_utils.pyindex 1539a2124..1b826383c 100644--- a/switchyard/cli/command_utils.py+++ b/switchyard/cli/command_utils.py@@ -3,15 +3,6 @@"""Small helpers shared by the remaining CLI commands."""-import logging---def quiet_dependency_loggers() -> None:- """Keep third-party INFO logs out of launcher terminal UIs."""-- for noisy in ("httpx", "httpcore", "openai", "anthropic", "uvicorn.access"):- logging.getLogger(noisy).setLevel(logging.WARNING)-def strip_forwarded_args(args: list[str] | None) -> list[str]:"""Strip argparse's leading ``--`` sentinel from forwarded arguments."""@@ -22,4 +13,4 @@ def strip_forwarded_args(args: list[str] | None) -> list[str]:return forwarded-__all__ = ["quiet_dependency_loggers", "strip_forwarded_args"]+__all__ = ["strip_forwarded_args"]diff --git a/switchyard/cli/launchers/claude_code_launcher.py b/switchyard/cli/launchers/claude_code_launcher.pyindex 4b69ab298..42279731d 100644--- a/switchyard/cli/launchers/claude_code_launcher.py+++ b/switchyard/cli/launchers/claude_code_launcher.py@@ -22,7 +22,7 @@ from switchyard.cli.launchers.live_stats_footer import LiveStatsFooterfrom switchyard.cli.launchers.native_server import NativeServerfrom switchyard.cli.launchers.proxy_health_monitor import ProxyHealthMonitorfrom switchyard.cli.launchers.session_summary import print_session_summary-from switchyard.server.shell_tui import ShellTUI+from switchyard.cli.launchers.shell_tui import ShellTUIlogger = logging.getLogger(__name__)diff --git a/switchyard/cli/launchers/codex_cli_launcher.py b/switchyard/cli/launchers/codex_cli_launcher.pyindex 973835d7d..78c85f659 100644--- a/switchyard/cli/launchers/codex_cli_launcher.py+++ b/switchyard/cli/launchers/codex_cli_launcher.py@@ -30,7 +30,7 @@ from switchyard.cli.launchers.live_stats_footer import LiveStatsFooterfrom switchyard.cli.launchers.native_server import NativeServerfrom switchyard.cli.launchers.proxy_health_monitor import ProxyHealthMonitorfrom switchyard.cli.launchers.session_summary import print_session_summary-from switchyard.server.shell_tui import ShellTUI+from switchyard.cli.launchers.shell_tui import ShellTUIlogger = logging.getLogger(__name__)diff --git a/switchyard/lib/cost_estimator.py b/switchyard/cli/launchers/cost_estimator.pysimilarity index 99%rename from switchyard/lib/cost_estimator.pyrename to switchyard/cli/launchers/cost_estimator.pyindex ed86c00f2..642da5e25 100644--- a/switchyard/lib/cost_estimator.py+++ b/switchyard/cli/launchers/cost_estimator.py@@ -9,10 +9,9 @@ field names instead of stringly-typed dict lookups.Usage::- from switchyard.lib.cost_estimator import estimate_cost, MODEL_PRICING+ from switchyard.cli.launchers.cost_estimator import estimate_cost- stats = requests.get("http://localhost:4000/v1/routing/stats").json()- breakdown = estimate_cost(stats["models"])+ breakdown = estimate_cost({"model": {"prompt_tokens": 10}})print(f"Total: ${breakdown['total_cost']:.4f}")See ``docs.anthropic.com/en/docs/about-claude/pricing`` for thediff --git a/switchyard/cli/launchers/launcher_runtime.py b/switchyard/cli/launchers/launcher_runtime.pyindex 4584d1235..8a5149aee 100644--- a/switchyard/cli/launchers/launcher_runtime.py+++ b/switchyard/cli/launchers/launcher_runtime.py@@ -69,20 +69,13 @@ def configure_debug_file_logging(*, display_model: str) -> Path:handler.close()root.setLevel(logging.WARNING)- for name in (- "switchyard",- "httpx",- "httpcore",- "openai",- "anthropic",- ):- logger = logging.getLogger(name)- for handler in logger.handlers[:]:- logger.removeHandler(handler)- handler.close()- logger.addHandler(file_handler)- logger.setLevel(logging.DEBUG)- logger.propagate = False+ switchyard_logger = logging.getLogger("switchyard")+ for handler in switchyard_logger.handlers[:]:+ switchyard_logger.removeHandler(handler)+ handler.close()+ switchyard_logger.addHandler(file_handler)+ switchyard_logger.setLevel(logging.DEBUG)+ switchyard_logger.propagate = Falselogging.getLogger("switchyard").info("=== switchyard debug log: model=%s pid=%d ===",@@ -94,14 +87,7 @@ def configure_debug_file_logging(*, display_model: str) -> Path:def silence_launch_loggers(*, local_logger: logging.Logger) -> None:"""Keep dependency chatter out of a child process terminal UI."""- for noisy in (- "switchyard",- "httpx",- "httpcore",- "openai",- "anthropic",- ):- logging.getLogger(noisy).setLevel(logging.WARNING)+ logging.getLogger("switchyard").setLevel(logging.WARNING)local_logger.setLevel(logging.INFO)@@ -113,30 +99,6 @@ def stdin_is_tty() -> bool:return False-def route_bundle_strategy_summary(route_bundle: str, default_model: str) -> str:- """Describe the default route in a Python server bundle."""- try:- from collections.abc import Mapping as _Mapping- from importlib import import_module- yaml = import_module("yaml")- raw = yaml.safe_load(Path(route_bundle).read_text())- routes = raw.get("routes") if isinstance(raw, dict) else None- if isinstance(routes, _Mapping) and routes:- first_key = next(iter(routes))- route = routes[first_key]- route_type = route.get("type") if isinstance(route, _Mapping) else None- if isinstance(route_type, str):- if route_type == "noop":- return "noop"- if route_type == "passthrough":- target = route.get("target")- model = target.get("model") if isinstance(target, _Mapping) else target- return f"passthrough: model={model or first_key}"- except Exception:- pass- return f"route: {default_model}"--# Keys that are abbreviated in the banner display._KEY_ABBREV: dict[str, str] = {"confidence_threshold": "conf",diff --git a/switchyard/cli/launchers/live_stats_footer.py b/switchyard/cli/launchers/live_stats_footer.pyindex ade41d6c6..89855e7ca 100644--- a/switchyard/cli/launchers/live_stats_footer.py+++ b/switchyard/cli/launchers/live_stats_footer.py@@ -3,8 +3,8 @@"""Shared live token-usage footer for launcher TUI sessions.-One layout for every routing strategy: an aggregate row across all chains plus-one indented row per active outbound model tier.+One layout for every routing strategy: an aggregate row across all requests+plus one indented row per active outbound model."""from __future__ import annotations@@ -14,13 +14,12 @@ from typing import castfrom switchyard.cli.launchers.proxy_health_monitor import ProxyHealthMonitorfrom switchyard.cli.launchers.stats_source import StatsSource-from switchyard.lib.route_table import RouteTableFOOTER_ROWS = 2class LiveStatsFooter:- """Live stats footer: aggregate row + one row per active model tier."""+ """Live stats footer: aggregate row plus one row per active model."""def __init__(self,@@ -28,13 +27,11 @@ class LiveStatsFooter:model: str,health: ProxyHealthMonitor,*,- table: RouteTable | None = None,strategy_label: str | None = None,) -> None:self._stats = statsself._default_model_short = model.rsplit("/", 1)[-1]self._health = health- self._table = tableself._strategy_label = strategy_label# Ordered list of models seen in traffic so far. Grows as new tiers# receive their first request; order is first-seen, which keeps the@@ -44,7 +41,7 @@ class LiveStatsFooter:@propertydef height(self) -> int:- """Current footer height: 1 aggregate row + 1 row per seen tier (min 2)."""+ """Current footer height: one aggregate plus each seen model (minimum two)."""return FOOTER_ROWS - 1 + max(1, len(self._seen_models))def as_footer_fn(self) -> Callable[[int], list[tuple[str, int]]]:@@ -87,20 +84,21 @@ class LiveStatsFooter:def _tier_rows(self, snapshot: Mapping[str, object],) -> list[tuple[str, int]]:- """Return one row per model tier that has received traffic.+ """Return one row per model that has received traffic.- Before any traffic lands, returns a single placeholder row using the- table's last-looked-up id or the launch default. Once traffic- arrives, the list grows to match the number of distinct models seen,- in first-seen order, and never shrinks.+ Before traffic lands, a placeholder uses the launch model. Once traffic+ arrives, rows grow in first-seen order and never shrink."""models = _mapping(snapshot, "models")if not models:- fallback = (- (self._table.last_looked_up if self._table else None)- or self._default_model_short- )- return [_model_row(fallback, calls=0, errors=0, prompt=0, completion=0, cached=0)]+ return [_model_row(+ self._default_model_short,+ calls=0,+ errors=0,+ prompt=0,+ completion=0,+ cached=0,+ )]for m in models:if m not in self._seen_set:diff --git a/switchyard/cli/launchers/openclaw_launcher.py b/switchyard/cli/launchers/openclaw_launcher.pyindex da5bcbc25..a0dbcc87a 100644--- a/switchyard/cli/launchers/openclaw_launcher.py+++ b/switchyard/cli/launchers/openclaw_launcher.py@@ -26,7 +26,7 @@ from switchyard.cli.launchers.live_stats_footer import LiveStatsFooterfrom switchyard.cli.launchers.native_server import NativeServerfrom switchyard.cli.launchers.proxy_health_monitor import ProxyHealthMonitorfrom switchyard.cli.launchers.session_summary import print_session_summary-from switchyard.server.shell_tui import ShellTUI+from switchyard.cli.launchers.shell_tui import ShellTUIlogger = logging.getLogger(__name__)diff --git a/switchyard/cli/launchers/session_summary.py b/switchyard/cli/launchers/session_summary.pyindex e676782a1..681ae466b 100644--- a/switchyard/cli/launchers/session_summary.py+++ b/switchyard/cli/launchers/session_summary.py@@ -10,8 +10,8 @@ import loggingfrom collections.abc import Mappingfrom typing import cast+from switchyard.cli.launchers.cost_estimator import estimate_model_costfrom switchyard.cli.launchers.stats_source import StatsSource-from switchyard.lib.cost_estimator import estimate_model_cost_RULE = "─" * 51_LOG = logging.getLogger(__name__)diff --git a/switchyard/server/shell_tui.py b/switchyard/cli/launchers/shell_tui.pysimilarity index 100%rename from switchyard/server/shell_tui.pyrename to switchyard/cli/launchers/shell_tui.pydiff --git a/switchyard/cli/model_catalog/__init__.py b/switchyard/cli/model_catalog/__init__.pydeleted file mode 100644index a6b4c862b..000000000--- a/switchyard/cli/model_catalog/__init__.py+++ /dev/null@@ -1,5 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Model catalog discovery and selection helpers."""-diff --git a/switchyard/cli/model_catalog/model_discovery.py b/switchyard/cli/model_catalog/model_discovery.pydeleted file mode 100644index 24f29593d..000000000--- a/switchyard/cli/model_catalog/model_discovery.py+++ /dev/null@@ -1,50 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""OpenAI-compatible model discovery for YAML route bundles."""--import httpx---class ModelDiscoveryError(RuntimeError):- """Raised when an upstream model catalog cannot be fetched."""---def fetch_model_ids(- base_url: str,- api_key: str,- timeout_s: float = 10.0,-) -> list[str]:- """Return sorted model IDs from an OpenAI-compatible ``GET /models``."""-- try:- with httpx.Client(timeout=timeout_s) as client:- response = client.get(- f"{base_url.rstrip('/')}/models",- headers={"Authorization": f"Bearer {api_key}"},- )- response.raise_for_status()- body = response.json()- except (httpx.HTTPError, ValueError) as exc:- raise ModelDiscoveryError(str(exc)) from exc-- raw_items = body.get("data", []) if isinstance(body, dict) else body- if not isinstance(raw_items, list):- raise ModelDiscoveryError("GET /models response did not contain a model list")-- model_ids: list[str] = []- for item in raw_items:- if isinstance(item, str):- model_ids.append(item)- continue- if not isinstance(item, dict):- continue- for key in ("id", "model", "name"):- value = item.get(key)- if isinstance(value, str) and value:- model_ids.append(value)- break- return sorted(set(model_ids))---__all__ = ["ModelDiscoveryError", "fetch_model_ids"]diff --git a/switchyard/cli/route_bundle.py b/switchyard/cli/route_bundle.pydeleted file mode 100644index 61f210fa8..000000000--- a/switchyard/cli/route_bundle.py+++ /dev/null@@ -1,231 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Build a Python server route table from a minimal YAML bundle."""--from __future__ import annotations--import os-import re-import time-from collections.abc import Mapping, Sequence-from importlib import import_module-from pathlib import Path-from typing import Any, Protocol, cast--from switchyard.lib.backends.llm_target import LlmTarget, coerce_llm_target-from switchyard.lib.backends.multi_llm_backend import build_native_backend-from switchyard.lib.backends.stats_llm_backend import StatsLlmBackend-from switchyard.lib.processors.stats_request_processor import StatsRequestProcessor-from switchyard.lib.processors.stats_response_processor_accumulator import (- StatsResponseProcessor,-)-from switchyard.lib.proxy_context import ProxyContext-from switchyard.lib.roles import LLMBackend-from switchyard.lib.route_table import ChainRuntime, RouteTable-from switchyard.lib.stats_accumulator import StatsAccumulator-from switchyard.lib.switchyard import Switchyard-from switchyard_rust.core import ChatRequest, ChatRequestType, ChatResponse-from switchyard_rust.translation import TranslationEngine--_ENV_REF_RE = re.compile(r"\$\{([A-Za-z_][A-Za-z0-9_]*)\}")-_TOP_LEVEL_KEYS = frozenset({"defaults", "routes"})-_ROUTE_METADATA_KEYS = frozenset({"display_name", "description"})-_ROUTE_KEYS = {- "noop": frozenset({"type"}) | _ROUTE_METADATA_KEYS,- "passthrough": frozenset({"type", "target"}) | _ROUTE_METADATA_KEYS,-}---class RouteBundleConfigError(ValueError):- """Raised when a Python server route bundle is invalid."""---class _YamlModule(Protocol):- def safe_load(self, stream: str) -> object: ...---class _NoopBackend(LLMBackend):- """Return a fixed response without making an upstream request."""-- @property- def supported_request_types(self) -> list[ChatRequestType]:- return list(ChatRequestType)-- async def call(self, ctx: ProxyContext, request: ChatRequest) -> ChatResponse:- model = request.model or "switchyard/noop"- ctx.selected_model = model- ctx.selected_target = model- return ChatResponse.openai_completion({- "id": "switchyard-noop",- "object": "chat.completion",- "created": int(time.time()),- "model": model,- "choices": [{- "index": 0,- "message": {"role": "assistant", "content": "OK"},- "finish_reason": "stop",- }],- "usage": {- "prompt_tokens": 0,- "completion_tokens": 0,- "total_tokens": 0,- },- })---def parse_route_bundle_file(path: str | Path) -> dict[str, object]:- """Read a YAML route bundle and return its top-level mapping."""- resolved = Path(path)- try:- contents = resolved.read_text()- except FileNotFoundError as error:- raise RouteBundleConfigError(f"{resolved}: file not found") from error- except (OSError, UnicodeError) as error:- raise RouteBundleConfigError(f"{resolved}: cannot read: {error}") from error-- try:- yaml = cast(_YamlModule, import_module("yaml"))- raw = yaml.safe_load(contents)- except Exception as error:- message = " ".join(str(error).splitlines())- raise RouteBundleConfigError(f"{resolved}: invalid YAML: {message}") from error- return _mapping(raw, "route bundle")---def load_route_bundle_table(- path: str | Path,- *,- stats_accumulator: StatsAccumulator | None = None,- pre_routing_request_processors: Sequence[Any] = (),- extra_response_processors: Sequence[Any] = (),-) -> RouteTable:- """Load a YAML route bundle into a server dispatch table."""- return build_route_bundle_table(- parse_route_bundle_file(path),- stats_accumulator=stats_accumulator,- pre_routing_request_processors=pre_routing_request_processors,- extra_response_processors=extra_response_processors,- )---def build_route_bundle_table(- raw: object,- *,- stats_accumulator: StatsAccumulator | None = None,- pre_routing_request_processors: Sequence[Any] = (),- extra_response_processors: Sequence[Any] = (),-) -> RouteTable:- """Build a table containing only noop and passthrough routes."""- bundle = _mapping(_expand_env(raw), "route bundle")- _reject_unknown_keys(bundle, _TOP_LEVEL_KEYS, "route bundle")- defaults = _mapping(bundle.get("defaults", {}), "defaults")- routes = _mapping(bundle.get("routes"), "routes")- if not routes:- raise RouteBundleConfigError("routes must contain at least one route")-- stats = stats_accumulator or StatsAccumulator()- table = RouteTable()- for route_id, raw_route in routes.items():- if not route_id:- raise RouteBundleConfigError("route ids must be non-empty strings")- route = _mapping(raw_route, f"route {route_id!r}")- route_type = route.get("type")- if not isinstance(route_type, str):- raise RouteBundleConfigError(f"route {route_id!r}: missing string 'type'")- if route_type not in _ROUTE_KEYS:- raise RouteBundleConfigError(- f"route {route_id!r}: unsupported route type {route_type!r}; "- "expected 'noop' or 'passthrough'"- )- _reject_unknown_keys(route, _ROUTE_KEYS[route_type], f"route {route_id!r}")-- if route_type == "noop":- runtime = _build_runtime(- _NoopBackend(),- stats,- pre_routing_request_processors,- extra_response_processors,- )- else:- target = _target(route_id, route.get("target"), defaults)- runtime = _build_runtime(- StatsLlmBackend(build_native_backend(target), stats),- stats,- pre_routing_request_processors,- extra_response_processors,- )-- metadata = {- key: value- for key in _ROUTE_METADATA_KEYS- if (value := route.get(key)) is not None- }- table.register(route_id, runtime, metadata=metadata, default=table.default_model() is None)- return table---def _build_runtime(- backend: LLMBackend,- stats: StatsAccumulator,- request_processors: Sequence[Any],- response_processors: Sequence[Any],-) -> ChainRuntime:- return Switchyard(- request_processors=[StatsRequestProcessor(), *request_processors],- backend=backend,- response_processors=[StatsResponseProcessor(stats), *response_processors],- translator=TranslationEngine(),- )---def _target(route_id: str, value: object, defaults: Mapping[str, object]) -> LlmTarget:- if isinstance(value, str):- target: dict[str, object] = {"model": value}- else:- target = _mapping(value, f"route {route_id!r} target")- try:- return coerce_llm_target({**defaults, **target}, default_id=route_id)- except (TypeError, ValueError) as error:- raise RouteBundleConfigError(f"route {route_id!r}: invalid target: {error}") from error---def _expand_env(value: object) -> object:- if isinstance(value, dict):- return {key: _expand_env(item) for key, item in value.items()}- if isinstance(value, list):- return [_expand_env(item) for item in value]- if not isinstance(value, str):- return value-- def replace(match: re.Match[str]) -> str:- name = match.group(1)- if name not in os.environ:- raise RouteBundleConfigError(f"environment variable {name} is not set")- return os.environ[name]-- return _ENV_REF_RE.sub(replace, value)---def _mapping(value: object, where: str) -> dict[str, object]:- if not isinstance(value, Mapping):- raise RouteBundleConfigError(f"{where} must be a mapping")- if not all(isinstance(key, str) for key in value):- raise RouteBundleConfigError(f"{where} keys must be strings")- return {str(key): item for key, item in value.items()}---def _reject_unknown_keys(- value: Mapping[str, object], allowed: frozenset[str], where: str-) -> None:- unknown = sorted(set(value) - allowed)- if unknown:- raise RouteBundleConfigError(f"unknown key(s) for {where}: {', '.join(unknown)}")---__all__ = [- "RouteBundleConfigError",- "build_route_bundle_table",- "load_route_bundle_table",- "parse_route_bundle_file",-]diff --git a/switchyard/cli/switchyard_cli.py b/switchyard/cli/switchyard_cli.pyindex 3698e4aa9..49ed84bdd 100644--- a/switchyard/cli/switchyard_cli.py+++ b/switchyard/cli/switchyard_cli.py@@ -5,68 +5,13 @@"""Switchyard command-line entry point."""import argparse-import logging-import osfrom switchyard import __version__-from switchyard.cli.command_utils import (- quiet_dependency_loggers as _quiet_dependency_loggers,-)from switchyard.cli.launch_command import (cmd_launch_claude,cmd_launch_codex,cmd_launch_openclaw,)-from switchyard.cli.route_bundle import RouteBundleConfigError, load_route_bundle_table-from switchyard.lib.processors.rl_logging_response_processor import build_rl_logging_processors-from switchyard.server.server_util import (- add_transport_args,- build_and_serve,- resolve_rl_log_dir,-)--logger = logging.getLogger(__name__)--def _cmd_serve(args: argparse.Namespace) -> None:- """Serve an explicit Python route bundle."""-- request_processors, response_processors = build_rl_logging_processors(- resolve_rl_log_dir(args)- )- if args.routing_log_file:- from switchyard.lib.processors.routing_log_response_processor import (- RoutingLogResponseProcessor,- )-- response_processors.append(RoutingLogResponseProcessor(args.routing_log_file))-- table = load_route_bundle_table(- args.routes,- pre_routing_request_processors=request_processors,- extra_response_processors=response_processors,- )- logger.info(- "Switchyard route bundle loaded %d route(s) from %s",- len(table.registered_models()),- args.routes,- )- strategy_summary: str | None = None- default_model = table.default_model()- if default_model:- from switchyard.cli.launchers.launcher_runtime import (- route_bundle_strategy_summary,- )-- strategy_summary = route_bundle_strategy_summary(- args.routes,- default_model,- )- build_and_serve(- args,- table,- inbound_default="both",- strategy_summary=strategy_summary,- )def _add_launch_parser(@@ -116,26 +61,6 @@ def _build_parser() -> argparse.ArgumentParser:)subparsers = parser.add_subparsers(dest="command")- serve = subparsers.add_parser("serve", help="Serve a Python route bundle")- serve.add_argument(- "--routes",- "-c",- required=True,- metavar="PATH",- help="YAML bundle containing noop and passthrough routes.",- )- serve.add_argument("--enable-rl-logging", action="store_true")- serve.add_argument("--rl-log-dir", default="./rl_data", metavar="DIR")- add_transport_args(serve)- serve.add_argument("--routing-log-file", default=None, metavar="PATH")- serve.add_argument(- "--workers",- "-w",- type=int,- default=int(os.environ.get("SWITCHYARD_WORKERS", "1")),- )- serve.set_defaults(func=_cmd_serve)-_add_launch_parser(subparsers)return parser@@ -143,20 +68,12 @@ def _build_parser() -> argparse.ArgumentParser:def main() -> None:"""Run the Switchyard CLI."""- logging.basicConfig(- level=logging.INFO,- format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",- )- _quiet_dependency_loggers()parser = _build_parser()args = parser.parse_args()if not hasattr(args, "func"):parser.print_help()raise SystemExit(1)- try:- args.func(args)- except RouteBundleConfigError as exc:- raise SystemExit(f"error: invalid route bundle: {exc}") from exc+ args.func(args)if __name__ == "__main__":diff --git a/switchyard/lib/__init__.py b/switchyard/lib/__init__.pydeleted file mode 100644index b7032e225..000000000--- a/switchyard/lib/__init__.py+++ /dev/null@@ -1,15 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Core library — Rust-backed chat values, translation, and Python serving.--This subpackage holds the protocol-agnostic building blocks used across the rest-of the library:--- ``ChatRequest`` — Rust-backed request values (OpenAI, Responses, Anthropic)-- ``chat_response`` — Rust-backed response values plus Python stream adapters-- ``translation`` — pure format-conversion functions and typed translation engines-- ``processors`` — reusable request/response components-- ``backends`` — LLM backend implementations (OpenAI, Anthropic, multi-tier routing)-- ``roles`` — backend role definitions and translation response aliases-"""diff --git a/switchyard/lib/backends/__init__.py b/switchyard/lib/backends/__init__.pydeleted file mode 100644index 53c20ae6f..000000000--- a/switchyard/lib/backends/__init__.py+++ /dev/null@@ -1,30 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Concrete :class:`LLMBackend` implementations + colocated backend config.--Each file defines one ``LLMBackend``. Re-exports live here for ergonomic imports like-``from switchyard.lib.backends import OpenAiNativeBackend``.-"""--from switchyard.lib.backends.backend_format_resolver import (- BackendFormatResolution,- BackendFormatResolver,-)-from switchyard.lib.backends.stats_llm_backend import (- StatsLlmBackend,-)-from switchyard_rust.components import (- AnthropicNativeBackend,- OpenAiNativeBackend,- OpenAiPassthroughBackend,-)--__all__ = [- "AnthropicNativeBackend",- "BackendFormatResolution",- "BackendFormatResolver",- "OpenAiPassthroughBackend",- "OpenAiNativeBackend",- "StatsLlmBackend",-]diff --git a/switchyard/lib/backends/anthropic_native_llm_backend.py b/switchyard/lib/backends/anthropic_native_llm_backend.pydeleted file mode 100644index 2d824641d..000000000--- a/switchyard/lib/backends/anthropic_native_llm_backend.py+++ /dev/null@@ -1,8 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Rust-owned Anthropic-native backend export."""--from switchyard_rust.components import AnthropicNativeBackend--__all__ = ["AnthropicNativeBackend"]diff --git a/switchyard/lib/backends/backend_format_resolver.py b/switchyard/lib/backends/backend_format_resolver.pydeleted file mode 100644index e74ffa10c..000000000--- a/switchyard/lib/backends/backend_format_resolver.py+++ /dev/null@@ -1,425 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Resolve generic backend tier formats into concrete backend wire formats.--``BackendFormat.AUTO`` probes the upstream in priority order:- 1. OpenAI Chat Completions (``/v1/chat/completions``) → ``OPENAI``- 2. Anthropic Messages (``/v1/messages``) → ``ANTHROPIC``- 3. OpenAI Responses (``/v1/responses``) → ``RESPONSES``- 4. Fallback → ``OPENAI`` (Chat Completions, assumed universal)--Chat Completions is probed first because it is the most widely supported format.-Endpoints that bridge multiple API surfaces (e.g. NVIDIA Inference Hub via-LiteLLM) will satisfy all three probes; preferring Chat Completions avoids-silently upgrading NIM models to Anthropic Messages format. Anthropic-native-endpoints (api.anthropic.com) return 404 for Chat Completions, so they correctly-fall through to the Anthropic probe.--The TranslationEngine converts any inbound format to any backend format through-a neutral IR, so all (inbound, backend) combinations are valid regardless of-which format the client uses.-"""--import json-import logging-from dataclasses import dataclass-from typing import Any--import httpx--from switchyard.lib import startup_timing-from switchyard.lib.backends.llm_target import (- BackendFormat,- LlmTarget,-)--log = logging.getLogger(__name__)-_DEFAULT_TIMEOUT_S = 3.0---@dataclass(frozen=True)-class BackendFormatResolution:- """Concrete backend format selected for a generic tier."""-- format: BackendFormat- reason: str---class BackendFormatResolver:- """Resolve ``BackendFormat.AUTO`` through reusable capability probes."""-- @staticmethod- def resolve(tier: LlmTarget) -> BackendFormatResolution:- """Return the concrete backend format for ``tier``.-- Explicit formats are already resolved. ``AUTO`` needs a real endpoint- probe, so missing probe inputs fail fast instead of silently picking- a backend that may only work by accident.- """- if tier.format != BackendFormat.AUTO:- return BackendFormatResolution(- format=tier.format,- reason="backend format is explicitly configured",- )-- return BackendFormatResolver._resolve_auto(tier)-- @staticmethod- def _resolve_auto(tier: LlmTarget) -> BackendFormatResolution:- if not tier.endpoint.base_url:- raise ValueError(- "format='auto' requires base_url so Switchyard can probe upstream capabilities.",- )- if not tier.endpoint.api_key:- raise ValueError(- "format='auto' requires api_key so Switchyard can probe upstream capabilities.",- )-- if _model_is_anthropic(tier.model):- return BackendFormatResolution(- format=BackendFormat.ANTHROPIC,- reason="model prefix indicates native Anthropic; skipping probes",- )-- timeout_s = tier.endpoint.timeout_secs or _DEFAULT_TIMEOUT_S-- # startup_timing marks let `switchyard launch --startup-timing` show each- # probe as its own line, so a slow AUTO detection names which route stalled.- startup_timing.mark("chain init")-- # Chat Completions is probed first: it is the most widely supported- # format and the universal fallback. A transport timeout (the endpoint- # is reachable but slow, e.g. a cold NVCF/Azure deployment) is a- # different signal from a fast "not wired" 404. On a timeout, assume- # Chat Completions rather than serially probing the rarer /v1/messages- # and /v1/responses routes — each of those would stack another timeout- # and turn one slow startup into several seconds. A 404 (even a slow- # one) still falls through to the probes below, so Anthropic-native and- # Responses-only endpoints are detected as before.- try:- chat_supported = probe_openai_chat_completions_support_sync(- base_url=tier.endpoint.base_url,- api_key=tier.endpoint.api_key,- model=tier.model,- timeout_s=timeout_s,- )- except httpx.TimeoutException:- startup_timing.mark("probe: /v1/chat/completions (timed out)")- return BackendFormatResolution(- format=BackendFormat.OPENAI,- reason="chat-completions probe timed out; assuming Chat Completions",- )- startup_timing.mark("probe: /v1/chat/completions")- if chat_supported:- return BackendFormatResolution(- format=BackendFormat.OPENAI,- reason="upstream /v1/chat/completions probe succeeded",- )-- anthropic_supported = probe_anthropic_messages_support_sync(- base_url=tier.endpoint.base_url,- api_key=tier.endpoint.api_key,- model=tier.model,- timeout_s=timeout_s,- )- startup_timing.mark("probe: /v1/messages")- if anthropic_supported:- return BackendFormatResolution(- format=BackendFormat.ANTHROPIC,- reason="upstream /v1/messages probe succeeded; Chat Completions not available",- )-- responses_supported = probe_openai_responses_support_sync(- base_url=tier.endpoint.base_url,- api_key=tier.endpoint.api_key,- model=tier.model,- timeout_s=timeout_s,- )- startup_timing.mark("probe: /v1/responses")- if responses_supported:- return BackendFormatResolution(- format=BackendFormat.RESPONSES,- reason="upstream /v1/responses probe succeeded; Chat Completions not available",- )-- return BackendFormatResolution(- format=BackendFormat.OPENAI,- reason="all probes failed; assuming Chat Completions",- )---def _model_is_anthropic(model: str | None) -> bool:- """Return True if the model ID prefix unambiguously identifies a native Anthropic model.-- Matches ``anthropic/<name>`` and ``claude<…>`` prefixes only — these map- exclusively to the Anthropic API or OpenRouter's direct Anthropic passthrough,- both of which require ``/v1/messages``.-- Gateway-namespaced paths like ``aws/anthropic/bedrock-…`` and- ``openrouter/anthropic/…`` are intentionally NOT matched: those gateways- also expose Chat Completions, so probing is preferred over assuming.- """- if not model:- return False- m = model.lower()- return m.startswith("anthropic/") or m.startswith("claude")---def probe_openai_chat_completions_support_sync(- *,- base_url: str,- api_key: str,- model: str | None = None,- timeout_s: float = _DEFAULT_TIMEOUT_S,-) -> bool:- """Return True iff ``{base_url}/chat/completions`` is a functional route.-- Sends a minimal-body probe POST with Bearer auth. A 404 means the- Chat Completions endpoint is not wired — the caller should probe- Anthropic Messages or Responses next.-- Raises ``httpx.TimeoutException`` on a transport timeout. A timeout means- the endpoint is reachable but slow (e.g. a cold NVCF/Azure deployment),- which is a different signal from a fast "not wired" 404: the resolver- stops probing and assumes Chat Completions instead of stacking the slower- ``/v1/messages`` and ``/v1/responses`` probes.- """- url = f"{base_url.rstrip('/')}/chat/completions"- headers = {"Authorization": f"Bearer {api_key}", "content-type": "application/json"}- req_body: dict[str, Any] = {- "messages": [{"role": "user", "content": " "}],- "max_tokens": 1,- **({"model": model} if model else {}),- }- try:- with httpx.Client(timeout=timeout_s) as client:- resp = client.post(url, headers=headers, json=req_body)- except httpx.TimeoutException:- raise- except httpx.RequestError as e:- log.warning(- "OpenAI /v1/chat/completions probe failed (%s); "- "falling back to Anthropic/Responses probes.",- type(e).__name__,- )- return False- if resp.status_code == 404:- return False- if resp.status_code == 401:- log.warning(- "OpenAI /v1/chat/completions probe got HTTP 401 — check --api-key. "- "Falling back to Anthropic/Responses probes.",- )- return False- if 200 <= resp.status_code < 500:- return True- log.warning(- "OpenAI /v1/chat/completions probe got HTTP %d; "- "falling back to Anthropic/Responses probes.",- resp.status_code,- )- return False---def _interpret_status(status: int, body: bytes = b"") -> bool:- """Return True iff the status code indicates the route is wired.-- When ``body`` is provided, a 400/422 that names the model as not found or- unsupported is treated as a probe failure — the route exists but this model- is not valid for it. Without a body (legacy call-sites) the old behaviour- is preserved.- """- if status == 404:- return False- if status == 401:- log.warning(- "Anthropic /v1/messages probe got HTTP 401 — check --api-key. "- "Falling back to translation mode.",- )- return False- if 200 <= status < 300:- return True- if status in (400, 422) and body:- if _body_signals_model_error(body):- log.debug(- "Anthropic /v1/messages probe got HTTP %d with model error; "- "model is unsupported on this endpoint.",- status,- )- return False- return True # validation error about other fields — route and model exist- if 200 <= status < 500:- return True- log.warning(- "Anthropic /v1/messages probe got HTTP %d; falling back to translation mode.",- status,- )- return False---def _body_signals_model_error(body: bytes) -> bool:- """Return True if the JSON error body indicates the model is not available."""- try:- data = json.loads(body)- except Exception:- return False- if not isinstance(data, dict):- return False- error = data.get("error")- if not isinstance(error, dict):- return False- if error.get("type") in ("not_found_error",):- return True- msg = (error.get("message") or "").lower()- return "model" in msg and any(- kw in msg for kw in ("not found", "not supported", "unsupported", "unknown", "invalid")- )---def _probe_headers(api_key: str) -> dict[str, str]:- # Include both auth styles: native Anthropic uses x-api-key, but- # OpenAI-compatible gateways (e.g. NVIDIA Inference Hub) that also expose- # /v1/messages expect Authorization: Bearer.- return {- "x-api-key": api_key,- "authorization": f"Bearer {api_key}",- "anthropic-version": "2023-06-01",- "content-type": "application/json",- }---def strip_v1_suffix(base_url: str) -> str:- """Return *base_url* with a trailing ``/v1`` path component removed.-- Switchyard's ``--base-url`` convention follows OpenAI's (e.g.- ``https://openrouter.ai/api/v1``), but the Anthropic SDK and- raw ``/v1/messages`` probing both treat the base URL as the API- root and append ``/v1/messages`` themselves. Without this trim the- two conventions collide — ``https://host/v1`` + ``/v1/messages``- becomes ``https://host/v1/v1/messages``.- """- stripped = base_url.rstrip("/")- if stripped.endswith("/v1"):- return stripped[:-3]- return stripped---def probe_openai_responses_support_sync(- *,- base_url: str,- api_key: str,- model: str | None = None,- timeout_s: float = _DEFAULT_TIMEOUT_S,-) -> bool:- """Return True iff ``{base_url}/responses`` is a functional route.-- Sends a minimal-body probe POST with Bearer auth. A 404 means the- Responses endpoint is not wired upstream; callers should fall back to- ``BackendFormat.OPENAI`` (Chat Completions). Non-OpenAI upstreams- (e.g. NVIDIA NIM) commonly 404 here even when they support Chat- Completions.- """- url = f"{base_url.rstrip('/')}/responses"- headers = {"Authorization": f"Bearer {api_key}", "content-type": "application/json"}- req_body: dict[str, Any] = {- "input": "",- "stream": False,- **({"model": model} if model else {}),- }- try:- with httpx.Client(timeout=timeout_s) as client:- resp = client.post(url, headers=headers, json=req_body)- except httpx.RequestError as e:- log.warning(- "OpenAI /v1/responses probe failed (%s); "- "falling back to Chat Completions format.",- type(e).__name__,- )- return False- if resp.status_code == 404:- return False- if resp.status_code == 401:- log.warning(- "OpenAI /v1/responses probe got HTTP 401; "- "falling back to Chat Completions format.",- )- return False- if 200 <= resp.status_code < 500:- return True- log.warning(- "OpenAI /v1/responses probe got HTTP %d; falling back to Chat Completions format.",- resp.status_code,- )- return False---def probe_anthropic_messages_support_sync(- *,- base_url: str,- api_key: str,- model: str | None = None,- timeout_s: float = _DEFAULT_TIMEOUT_S,-) -> bool:- """Synchronous version of :func:`probe_anthropic_messages_support`.-- Preferred at startup (no running event loop required). Uses- ``httpx.Client`` so no asyncio event loop is created; async clients- built afterward bind their connection pools to uvicorn's event loop- on first use rather than to a now-closed startup loop.- """- url = f"{strip_v1_suffix(base_url)}/v1/messages"- req_body: dict[str, Any] = {- "messages": [{"role": "user", "content": " "}],- "max_tokens": 1,- **({"model": model} if model else {}),- }- try:- with httpx.Client(timeout=timeout_s) as client:- resp = client.post(url, headers=_probe_headers(api_key), json=req_body)- except httpx.RequestError as e:- log.warning(- "Anthropic /v1/messages probe failed (%s); "- "falling back to translation mode.",- type(e).__name__,- )- return False- return _interpret_status(resp.status_code, resp.content)---async def probe_anthropic_messages_support(- *,- base_url: str,- api_key: str,- model: str | None = None,- timeout_s: float = _DEFAULT_TIMEOUT_S,-) -> bool:- """Return True iff ``{base_url}/v1/messages`` is a functional route.-- Sends a model-scoped probe POST with real auth. Response interpretation:-- * 404 → route not wired → return False (use translation)- * 401 → return False (credential validation happens on the real request)- * 400 / 422 with model error body → model unsupported → return False- * 400 / 422 with field error body → route exists → return True- * 200 → route exists → return True- * 5xx / timeout / network error → return False- """- url = f"{strip_v1_suffix(base_url)}/v1/messages"- req_body: dict[str, Any] = {- "messages": [{"role": "user", "content": " "}],- "max_tokens": 1,- **({"model": model} if model else {}),- }- try:- async with httpx.AsyncClient(timeout=timeout_s) as client:- resp = await client.post(url, headers=_probe_headers(api_key), json=req_body)- except httpx.RequestError as e:- log.debug(- "Anthropic /v1/messages probe unavailable (%s); "- "using OpenAI translation mode.",- type(e).__name__,- )- return False-- return _interpret_status(resp.status_code, resp.content)diff --git a/switchyard/lib/backends/llm_target.py b/switchyard/lib/backends/llm_target.pydeleted file mode 100644index ad22d0738..000000000--- a/switchyard/lib/backends/llm_target.py+++ /dev/null@@ -1,106 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Rust-owned LLM target configuration helpers."""--from __future__ import annotations--from typing import Any--from pydantic import BaseModel--from switchyard_rust.components import BackendFormat, EndpointConfig, LlmTarget--_DISABLE_THINKING_MODEL_FRAGMENTS = ("nemotron-3-super",)---def coerce_llm_target(value: object, *, default_id: str) -> LlmTarget:- """Build a Rust ``LlmTarget`` from a target object or legacy mapping."""- if isinstance(value, LlmTarget):- if value.id == "default" and default_id != "default":- return LlmTarget(- id=default_id,- model=value.model,- format=value.format,- endpoint=value.endpoint,- extra_body=value.extra_body,- extra_headers=value.extra_headers,- )- return value- if isinstance(value, BaseModel):- value = value.model_dump()- if not isinstance(value, dict):- raise TypeError(f"expected LlmTarget or dict, got {type(value).__name__}")-- data: dict[str, Any] = dict(value)- target_id = str(data.pop("id", default_id))- model = data.pop("model", None)- if not isinstance(model, str):- raise TypeError("LlmTarget.model must be a string")-- target_format = data.pop("format", data.pop("backend_format", BackendFormat.OPENAI))- endpoint = data.pop("endpoint", None)- base_url = data.pop("base_url", None)- api_key = data.pop("api_key", None)- timeout_secs = data.pop("timeout_secs", data.pop("timeout", None))- extra_body = data.pop("extra_body", None)- extra_headers = data.pop("extra_headers", None)- data.pop("tuning", None)- if data:- unknown = ", ".join(sorted(data))- raise ValueError(f"unknown LlmTarget fields: {unknown}")-- return LlmTarget(- id=target_id,- model=model,- format=target_format,- endpoint=endpoint,- base_url=base_url,- api_key=api_key,- timeout_secs=timeout_secs,- extra_body=extra_body,- extra_headers=extra_headers,- )---def llm_target_with_format(target: LlmTarget, target_format: BackendFormat) -> LlmTarget:- """Return ``target`` with a resolved backend format."""- return LlmTarget(- id=target.id,- model=target.model,- format=target_format,- endpoint=target.endpoint,- extra_body=target.extra_body,- extra_headers=target.extra_headers,- )---def llm_target_with_runtime_defaults(target: LlmTarget) -> LlmTarget:- """Return ``target`` with Switchyard's safe per-model runtime defaults."""- if target.extra_body:- return target- if not _should_disable_thinking(target.model):- return target- return LlmTarget(- id=target.id,- model=target.model,- format=target.format,- endpoint=target.endpoint,- extra_body={"chat_template_kwargs": {"enable_thinking": False}},- extra_headers=target.extra_headers,- )---def _should_disable_thinking(model: str) -> bool:- normalized = model.lower()- return any(fragment in normalized for fragment in _DISABLE_THINKING_MODEL_FRAGMENTS)---__all__ = [- "BackendFormat",- "EndpointConfig",- "LlmTarget",- "coerce_llm_target",- "llm_target_with_format",- "llm_target_with_runtime_defaults",-]diff --git a/switchyard/lib/backends/multi_llm_backend.py b/switchyard/lib/backends/multi_llm_backend.pydeleted file mode 100644index 48afdcb4f..000000000--- a/switchyard/lib/backends/multi_llm_backend.py+++ /dev/null@@ -1,54 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Build a native backend for a single LLM target."""--from __future__ import annotations--import logging--from switchyard.lib.backends.backend_format_resolver import BackendFormatResolver-from switchyard.lib.backends.llm_target import (- BackendFormat,- LlmTarget,- llm_target_with_format,- llm_target_with_runtime_defaults,-)-from switchyard.lib.roles import LLMBackend-from switchyard_rust.components import (- AnthropicNativeBackend,- OpenAiNativeBackend,-)--log = logging.getLogger(__name__)---def resolve_llm_target(target: LlmTarget) -> LlmTarget:- """Resolve ``BackendFormat.AUTO`` into the concrete native backend format."""- if target.format != BackendFormat.AUTO:- return target- resolution = BackendFormatResolver.resolve(target)- log.debug(- "resolved LLM target id=%s model=%s format=%s: %s",- target.id,- target.model,- resolution.format.value,- resolution.reason,- )- return llm_target_with_format(target, resolution.format)---def build_native_backend(target: LlmTarget) -> LLMBackend:- """Build the native Rust backend for one resolved or auto ``LlmTarget``."""- target = llm_target_with_runtime_defaults(resolve_llm_target(target))- if target.format in (BackendFormat.OPENAI, BackendFormat.RESPONSES):- return OpenAiNativeBackend(target)- if target.format == BackendFormat.ANTHROPIC:- return AnthropicNativeBackend(target)- raise ValueError(f"Unsupported backend format: {target.format!r}")---__all__ = [- "build_native_backend",- "resolve_llm_target",-]diff --git a/switchyard/lib/backends/openai_llm_backend.py b/switchyard/lib/backends/openai_llm_backend.pydeleted file mode 100644index 8fcab95d6..000000000--- a/switchyard/lib/backends/openai_llm_backend.py+++ /dev/null@@ -1,8 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Compatibility import path for the Rust-owned OpenAI passthrough backend."""--from switchyard_rust.components import OpenAiPassthroughBackend--__all__ = ["OpenAiPassthroughBackend"]diff --git a/switchyard/lib/backends/openai_native_backend.py b/switchyard/lib/backends/openai_native_backend.pydeleted file mode 100644index 646445880..000000000--- a/switchyard/lib/backends/openai_native_backend.py+++ /dev/null@@ -1,8 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Rust-owned OpenAI-native backend export."""--from switchyard_rust.components import OpenAiNativeBackend--__all__ = ["OpenAiNativeBackend"]diff --git a/switchyard/lib/backends/stats_llm_backend.py b/switchyard/lib/backends/stats_llm_backend.pydeleted file mode 100644index 9ed76d617..000000000--- a/switchyard/lib/backends/stats_llm_backend.py+++ /dev/null@@ -1,8 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Rust-owned stats backend wrapper export."""--from switchyard_rust.components import StatsLlmBackend--__all__ = ["StatsLlmBackend"]diff --git a/switchyard/lib/chat_request/__init__.py b/switchyard/lib/chat_request/__init__.pydeleted file mode 100644index 90113e115..000000000--- a/switchyard/lib/chat_request/__init__.py+++ /dev/null@@ -1,17 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Compatibility aliases for Rust-backed request values."""--from switchyard.lib.chat_request.anthropic import AnthropicChatRequest-from switchyard.lib.chat_request.base import ChatRequest, ChatRequestType-from switchyard.lib.chat_request.openai_chat import OpenAIChatRequest-from switchyard.lib.chat_request.openai_responses import ResponsesChatRequest--__all__ = [- "AnthropicChatRequest",- "ChatRequest",- "ChatRequestType",- "OpenAIChatRequest",- "ResponsesChatRequest",-]diff --git a/switchyard/lib/chat_request/anthropic.py b/switchyard/lib/chat_request/anthropic.pydeleted file mode 100644index 3fbdf162f..000000000--- a/switchyard/lib/chat_request/anthropic.py+++ /dev/null@@ -1,12 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Anthropic Messages request compatibility alias."""--from typing import TypeAlias--from switchyard_rust.core import ChatRequest as _ChatRequest--AnthropicChatRequest: TypeAlias = _ChatRequest--__all__ = ["AnthropicChatRequest"]diff --git a/switchyard/lib/chat_request/base.py b/switchyard/lib/chat_request/base.pydeleted file mode 100644index 5aff06bb2..000000000--- a/switchyard/lib/chat_request/base.py+++ /dev/null@@ -1,14 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Compatibility re-export for Rust-owned request values."""--from typing import TypeAlias--from switchyard_rust.core import ChatRequest as _ChatRequest-from switchyard_rust.core import ChatRequestType as _ChatRequestType--ChatRequest: TypeAlias = _ChatRequest-ChatRequestType: TypeAlias = _ChatRequestType--__all__ = ["ChatRequest", "ChatRequestType"]diff --git a/switchyard/lib/chat_request/openai_chat.py b/switchyard/lib/chat_request/openai_chat.pydeleted file mode 100644index 3051a683e..000000000--- a/switchyard/lib/chat_request/openai_chat.py+++ /dev/null@@ -1,12 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""OpenAI Chat Completions request compatibility alias."""--from typing import TypeAlias--from switchyard_rust.core import ChatRequest as _ChatRequest--OpenAIChatRequest: TypeAlias = _ChatRequest--__all__ = ["OpenAIChatRequest"]diff --git a/switchyard/lib/chat_request/openai_responses.py b/switchyard/lib/chat_request/openai_responses.pydeleted file mode 100644index 7a35a5e57..000000000--- a/switchyard/lib/chat_request/openai_responses.py+++ /dev/null@@ -1,12 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""OpenAI Responses API request compatibility alias."""--from typing import TypeAlias--from switchyard_rust.core import ChatRequest as _ChatRequest--ResponsesChatRequest: TypeAlias = _ChatRequest--__all__ = ["ResponsesChatRequest"]diff --git a/switchyard/lib/chat_response/__init__.py b/switchyard/lib/chat_response/__init__.pydeleted file mode 100644index c453843ba..000000000--- a/switchyard/lib/chat_response/__init__.py+++ /dev/null@@ -1,39 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Rust-backed response values and stream adapters."""--from switchyard.lib.chat_response.anthropic import (- AnthropicChatResponse,- AnthropicResponseStream,- AnthropicStreamingChatResponse,-)-from switchyard.lib.chat_response.openai_chat import (- CompletionChatResponse,- ResponseStream,- StreamingChatResponse,-)-from switchyard.lib.chat_response.openai_responses import (- ResponsesApiChatResponse,- ResponsesApiStream,- ResponsesApiStreamingChatResponse,-)-from switchyard_rust.core import ChatResponse, ChatResponseStream, ChatResponseType--AnyResponseStream = ChatResponseStream--__all__ = [- "AnyResponseStream",- "AnthropicChatResponse",- "AnthropicResponseStream",- "AnthropicStreamingChatResponse",- "ChatResponse",- "ChatResponseStream",- "ChatResponseType",- "CompletionChatResponse",- "ResponsesApiChatResponse",- "ResponsesApiStream",- "ResponsesApiStreamingChatResponse",- "ResponseStream",- "StreamingChatResponse",-]diff --git a/switchyard/lib/chat_response/anthropic.py b/switchyard/lib/chat_response/anthropic.pydeleted file mode 100644index fc8da3024..000000000--- a/switchyard/lib/chat_response/anthropic.py+++ /dev/null@@ -1,19 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Anthropic Messages API stream adapter re-export."""--from typing import TypeAlias--from switchyard_rust.core import ChatResponse as _ChatResponse-from switchyard_rust.core import ChatResponseStream as _ChatResponseStream--AnthropicChatResponse: TypeAlias = _ChatResponse-AnthropicStreamingChatResponse: TypeAlias = _ChatResponse-AnthropicResponseStream: TypeAlias = _ChatResponseStream--__all__ = [- "AnthropicChatResponse",- "AnthropicResponseStream",- "AnthropicStreamingChatResponse",-]diff --git a/switchyard/lib/chat_response/base.py b/switchyard/lib/chat_response/base.pydeleted file mode 100644index 8fcc14f72..000000000--- a/switchyard/lib/chat_response/base.py+++ /dev/null@@ -1,14 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Compatibility re-export for Rust-owned response values."""--from typing import TypeAlias--from switchyard_rust.core import ChatResponse as _ChatResponse-from switchyard_rust.core import ChatResponseType as _ChatResponseType--ChatResponse: TypeAlias = _ChatResponse-ChatResponseType: TypeAlias = _ChatResponseType--__all__ = ["ChatResponse", "ChatResponseType"]diff --git a/switchyard/lib/chat_response/openai_chat.py b/switchyard/lib/chat_response/openai_chat.pydeleted file mode 100644index a181c3a36..000000000--- a/switchyard/lib/chat_response/openai_chat.py+++ /dev/null@@ -1,15 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""OpenAI Chat Completions stream adapter re-export."""--from typing import TypeAlias--from switchyard_rust.core import ChatResponse as _ChatResponse-from switchyard_rust.core import ChatResponseStream as _ChatResponseStream--CompletionChatResponse: TypeAlias = _ChatResponse-StreamingChatResponse: TypeAlias = _ChatResponse-ResponseStream: TypeAlias = _ChatResponseStream--__all__ = ["CompletionChatResponse", "ResponseStream", "StreamingChatResponse"]diff --git a/switchyard/lib/chat_response/openai_responses.py b/switchyard/lib/chat_response/openai_responses.pydeleted file mode 100644index c18cca56a..000000000--- a/switchyard/lib/chat_response/openai_responses.py+++ /dev/null@@ -1,19 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""OpenAI Responses API stream adapter re-export."""--from typing import TypeAlias--from switchyard_rust.core import ChatResponse as _ChatResponse-from switchyard_rust.core import ChatResponseStream as _ChatResponseStream--ResponsesApiChatResponse: TypeAlias = _ChatResponse-ResponsesApiStreamingChatResponse: TypeAlias = _ChatResponse-ResponsesApiStream: TypeAlias = _ChatResponseStream--__all__ = [- "ResponsesApiChatResponse",- "ResponsesApiStream",- "ResponsesApiStreamingChatResponse",-]diff --git a/switchyard/lib/chat_response/streaming_response_accumulator.py b/switchyard/lib/chat_response/streaming_response_accumulator.pydeleted file mode 100644index 968321457..000000000--- a/switchyard/lib/chat_response/streaming_response_accumulator.py+++ /dev/null@@ -1,752 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Accumulate streaming responses into completed native responses."""--from __future__ import annotations--import json-import time-from collections.abc import Awaitable, Callable, Mapping-from dataclasses import dataclass-from typing import Literal, Protocol, cast--from anthropic.types import (- ContentBlock,- InputJSONDelta,- RawContentBlockDeltaEvent,- RawContentBlockStartEvent,- RawMessageDeltaEvent,- RawMessageStartEvent,- RawMessageStreamEvent,- StopReason,- TextBlock,- TextDelta,- ThinkingDelta,- ToolUseBlock,-)-from anthropic.types import (- Message as AnthropicMessage,-)-from anthropic.types import (- Usage as AnthropicUsage,-)-from openai.types.chat import ChatCompletion, ChatCompletionChunk-from openai.types.chat.chat_completion import Choice as ChatCompletionChoice-from openai.types.chat.chat_completion_message import ChatCompletionMessage-from openai.types.chat.chat_completion_message_function_tool_call import (- ChatCompletionMessageFunctionToolCall,-)-from openai.types.chat.chat_completion_message_function_tool_call import (- Function as ChatCompletionToolCallFunction,-)-from openai.types.completion_usage import CompletionUsage-from openai.types.responses import (- Response as OpenAIResponse,-)-from openai.types.responses import (- ResponseCompletedEvent,- ResponseFunctionCallArgumentsDeltaEvent,- ResponseFunctionToolCall,- ResponseOutputItemAddedEvent,- ResponseOutputItemDoneEvent,- ResponseOutputMessage,- ResponseOutputText,- ResponseTextDeltaEvent,-)-from openai.types.responses.response_usage import (- InputTokensDetails,- OutputTokensDetails,- ResponseUsage,-)-from pydantic import TypeAdapter, ValidationError--from switchyard_rust.core import (- ChatResponse,- ChatResponseType,- response_type_matches,-)--_ANTHROPIC_EVENT_ADAPTER: TypeAdapter[RawMessageStreamEvent] = TypeAdapter(RawMessageStreamEvent)-_OpenAIFinishReason = Literal[- "stop",- "length",- "tool_calls",- "content_filter",- "function_call",-]---class StreamingResponseAccumulator(Protocol):- """Accumulates provider-native stream events into a completed response."""-- def consume(self, event: object) -> None: ...-- def as_response(self) -> ChatResponse: ...---CompletedResponseCallback = Callable[[ChatResponse], Awaitable[None]]---@dataclass-class _OpenAIToolCallState:- id: str | None = None- name: str | None = None- arguments: str = ""-- def to_openai(self, *, fallback_id: str) -> ChatCompletionMessageFunctionToolCall:- return ChatCompletionMessageFunctionToolCall(- id=self.id or fallback_id,- type="function",- function=ChatCompletionToolCallFunction(- name=self.name or "",- arguments=self.arguments,- ),- )---@dataclass-class _AnthropicContentBlockState:- kind: Literal["text", "tool_use"]- text: str = ""- id: str | None = None- name: str | None = None- arguments: str = ""-- def to_anthropic(self, *, fallback_id: str) -> TextBlock | ToolUseBlock:- if self.kind == "text":- return TextBlock(type="text", text=self.text)- return ToolUseBlock(- type="tool_use",- id=self.id or fallback_id,- name=self.name or "",- input=_json_object_from_argument_string(self.arguments),- )---@dataclass-class _ResponsesToolCallState:- id: str | None = None- call_id: str | None = None- name: str = ""- arguments: str = ""-- def to_response_output_item(self, *, fallback_id: str) -> ResponseFunctionToolCall | None:- if not self.name and not self.arguments:- return None- return ResponseFunctionToolCall(- type="function_call",- id=self.id,- call_id=self.call_id or self.id or fallback_id,- name=self.name,- arguments=self.arguments,- status="completed",- )---def attach_final_response_callback(- response: ChatResponse,- *,- served_model: str,- callback: CompletedResponseCallback,-) -> bool:- """Attach a callback that receives the completed native response.-- Returns ``False`` when *response* is not a supported streaming response.- The callback only runs when the stream drains normally; stream wrappers own- that completion contract.- """- accumulator = create_streaming_response_accumulator(- response,- served_model=served_model,- )- if accumulator is None:- return False-- async def _tap(event: object) -> None:- accumulator.consume(event)-- async def _on_complete() -> None:- await callback(accumulator.as_response())-- if response_type_matches(response, ChatResponseType.OPENAI_STREAM):- response.stream.tap(_tap).on_complete(_on_complete)- elif response_type_matches(response, ChatResponseType.ANTHROPIC_STREAM):- response.stream.tap(_tap).on_complete(_on_complete)- elif response_type_matches(response, ChatResponseType.OPENAI_RESPONSES_STREAM):- response.stream.tap(_tap).on_complete(_on_complete)- else:- return False- return True---def create_streaming_response_accumulator(- response: ChatResponse,- *,- served_model: str,-) -> StreamingResponseAccumulator | None:- """Create the native accumulator for a streaming response."""- if response_type_matches(response, ChatResponseType.OPENAI_STREAM):- return _OpenAIChatStreamAccumulator(served_model=served_model)- if response_type_matches(response, ChatResponseType.ANTHROPIC_STREAM):- return _AnthropicStreamAccumulator(served_model=served_model)- if response_type_matches(response, ChatResponseType.OPENAI_RESPONSES_STREAM):- return _ResponsesStreamAccumulator(served_model=served_model)- return None---class _OpenAIChatStreamAccumulator:- """Accumulate OpenAI Chat Completion chunks into one ChatCompletion."""-- def __init__(self, *, served_model: str) -> None:- self._served_model = served_model- self._content = ""- self._reasoning_content = ""- self._tool_calls: list[_OpenAIToolCallState] = []- self._usage: CompletionUsage | None = None- self._finish_reason: str | None = None- self._response_id: str | None = None- self._created: int | None = None-- def consume(self, event: object) -> None:- chunk = event- if isinstance(chunk, Mapping):- chunk = ChatCompletionChunk.model_validate(chunk)- if not isinstance(chunk, ChatCompletionChunk):- return-- self._response_id = chunk.id or self._response_id- self._created = chunk.created or self._created- self._served_model = chunk.model or self._served_model- if chunk.usage is not None:- self._usage = chunk.usage- if not chunk.choices:- return-- choice = chunk.choices[0]- delta = choice.delta- if delta is not None:- if isinstance(delta.content, str):- self._content += delta.content- reasoning_content = getattr(delta, "reasoning_content", None)- if isinstance(reasoning_content, str):- self._reasoning_content += reasoning_content- for tool_call in delta.tool_calls or []:- self._merge_tool_call(tool_call)-- if choice.finish_reason is not None:- self._finish_reason = choice.finish_reason-- def as_response(self) -> ChatResponse:- message = ChatCompletionMessage(- role="assistant",- content=self._content or None,- tool_calls=[- tool_call.to_openai(fallback_id=f"call_switchyard_{index}")- for index, tool_call in enumerate(self._tool_calls)- ] or None,- )- response = ChatCompletion(- id=self._response_id or "chatcmpl-switchyard-stream",- object="chat.completion",- created=self._created or int(time.time()),- model=self._served_model,- choices=[- ChatCompletionChoice(- index=0,- message=message,- finish_reason=_openai_finish_reason(- self._finish_reason,- has_tools=bool(self._tool_calls),- ),- ),- ],- usage=self._usage,- )- if self._reasoning_content:- # The OpenAI SDK does not type vendor-specific reasoning fields- # on final messages. Preserve them at the serialization boundary.- response_dict = response.model_dump(mode="json", exclude_none=True)- response_dict["choices"][0]["message"]["reasoning_content"] = (- self._reasoning_content- )- response = ChatCompletion.model_validate(response_dict)- return ChatResponse.openai_completion(response)-- def _merge_tool_call(self, tool_call: object) -> None:- index = getattr(tool_call, "index", None)- if not isinstance(index, int):- index = len(self._tool_calls)- while len(self._tool_calls) <= index:- self._tool_calls.append(_OpenAIToolCallState())- existing = self._tool_calls[index]- tool_call_id = getattr(tool_call, "id", None)- if isinstance(tool_call_id, str) and tool_call_id:- existing.id = tool_call_id- func = getattr(tool_call, "function", None)- if func is None:- return- name = getattr(func, "name", None)- if isinstance(name, str) and name:- existing.name = name- arguments = getattr(func, "arguments", None)- if isinstance(arguments, str) and arguments:- existing.arguments += arguments---class _AnthropicStreamAccumulator:- """Accumulate Anthropic Messages events into one Anthropic message."""-- def __init__(self, *, served_model: str) -> None:- self._served_model = served_model- self._response_id: str | None = None- self._content_blocks: dict[int, _AnthropicContentBlockState] = {}- self._usage: dict[str, int] = {}- self._stop_reason: str | None = None-- def consume(self, event: object) -> None:- typed_event = _coerce_anthropic_event(event)- if isinstance(typed_event, Mapping):- self._consume_mapping_event(typed_event)- return-- if isinstance(typed_event, RawMessageStartEvent):- self._response_id = typed_event.message.id or self._response_id- self._served_model = typed_event.message.model or self._served_model- self._merge_usage(typed_event.message.usage)- return-- if isinstance(typed_event, RawContentBlockStartEvent):- block = typed_event.content_block- if isinstance(block, ToolUseBlock):- self._content_blocks[typed_event.index] = _AnthropicContentBlockState(- kind="tool_use",- id=block.id,- name=block.name,- arguments=_json_argument_string(block.input),- )- return- if isinstance(block, TextBlock):- self._content_blocks[typed_event.index] = _AnthropicContentBlockState(- kind="text",- text=block.text,- )- return- self._content_blocks[typed_event.index] = _AnthropicContentBlockState(- kind="text",- )- return-- if isinstance(typed_event, RawContentBlockDeltaEvent):- state = self._content_blocks.setdefault(- typed_event.index,- _AnthropicContentBlockState(kind="text"),- )- delta = typed_event.delta- if isinstance(delta, InputJSONDelta):- if state.kind != "tool_use":- state.kind = "tool_use"- state.text = ""- state.id = None- state.name = None- state.arguments = ""- state.arguments += delta.partial_json- return- if isinstance(delta, TextDelta) and state.kind == "text":- state.text += delta.text- return- if isinstance(delta, ThinkingDelta) and state.kind == "text":- state.text += delta.thinking- return-- if isinstance(typed_event, RawMessageDeltaEvent):- self._stop_reason = typed_event.delta.stop_reason or self._stop_reason- self._merge_usage(typed_event.usage)-- def _consume_mapping_event(self, event: Mapping[str, object]) -> None:- event_type = event.get("type")- if event_type == "message_start":- message = event.get("message")- if not isinstance(message, Mapping):- return- response_id = message.get("id")- if isinstance(response_id, str):- self._response_id = response_id- model = message.get("model")- if isinstance(model, str):- self._served_model = model- self._merge_usage(message.get("usage"))- return-- if event_type == "content_block_start":- index = _int_value(event.get("index"), 0)- block = event.get("content_block")- if not isinstance(block, Mapping):- return- if block.get("type") == "tool_use":- self._content_blocks[index] = _AnthropicContentBlockState(- kind="tool_use",- id=_str_value(block.get("id")),- name=_str_value(block.get("name")),- arguments=_json_argument_string(block.get("input")),- )- return- text = block.get("text")- self._content_blocks[index] = _AnthropicContentBlockState(- kind="text",- text=text if isinstance(text, str) else "",- )- return-- if event_type == "content_block_delta":- index = _int_value(event.get("index"), 0)- delta = event.get("delta")- if not isinstance(delta, Mapping):- return- block = self._content_blocks.setdefault(- index,- _AnthropicContentBlockState(kind="text"),- )- if delta.get("type") == "input_json_delta":- if block.kind != "tool_use":- block.kind = "tool_use"- block.text = ""- block.id = None- block.name = None- block.arguments = ""- partial_json = delta.get("partial_json")- if isinstance(partial_json, str):- block.arguments += partial_json- return- if block.kind != "text":- return- text = delta.get("text")- if isinstance(text, str):- block.text += text- return- thinking = delta.get("thinking")- if isinstance(thinking, str):- block.text += thinking- return-- if event_type == "message_delta":- delta = event.get("delta")- if isinstance(delta, Mapping):- stop_reason = delta.get("stop_reason")- if isinstance(stop_reason, str):- self._stop_reason = stop_reason- self._merge_usage(event.get("usage"))-- def as_response(self) -> ChatResponse:- content: list[ContentBlock] = []- has_tools = False- for _, block in sorted(self._content_blocks.items()):- if block.kind == "tool_use":- has_tools = True- content.append(- block.to_anthropic(fallback_id=f"toolu_switchyard_{len(content)}"),- )-- response = AnthropicMessage(- id=self._response_id or "msg_switchyard_stream",- type="message",- role="assistant",- content=content,- model=self._served_model,- stop_reason=cast(StopReason, self._stop_reason or ("tool_use" if has_tools else "end_turn")),- stop_sequence=None,- usage=AnthropicUsage(- input_tokens=self._usage.get("input_tokens", 0),- output_tokens=self._usage.get("output_tokens", 0),- cache_creation_input_tokens=self._usage.get(- "cache_creation_input_tokens",- 0,- ),- cache_read_input_tokens=self._usage.get("cache_read_input_tokens", 0),- ),- )- return ChatResponse.anthropic_completion(response)-- def _merge_usage(self, usage: object) -> None:- if usage is None:- return- for key in (- "input_tokens",- "output_tokens",- "cache_creation_input_tokens",- "cache_read_input_tokens",- ):- value = usage.get(key) if isinstance(usage, Mapping) else getattr(usage, key, None)- if isinstance(value, int):- self._usage[key] = value---class _ResponsesStreamAccumulator:- """Accumulate Responses API stream events into one Response."""-- def __init__(self, *, served_model: str) -> None:- self._served_model = served_model- self._response_id: str | None = None- self._created_at: float | None = None- self._content = ""- self._tool_calls: dict[int, _ResponsesToolCallState] = {}- self._usage: ResponseUsage | None = None- self._final_response: OpenAIResponse | Mapping[str, object] | None = None-- def consume(self, event: object) -> None:- if isinstance(event, ResponseTextDeltaEvent):- self._content += event.delta- return-- if isinstance(event, (ResponseOutputItemAddedEvent, ResponseOutputItemDoneEvent)):- self._merge_output_item(event.item, event.output_index)- return-- if isinstance(event, ResponseFunctionCallArgumentsDeltaEvent):- tool_call = self._tool_calls.setdefault(- event.output_index,- _ResponsesToolCallState(),- )- tool_call.arguments += event.delta- return-- if isinstance(event, ResponseCompletedEvent):- self._capture_response_metadata(event.response)- self._final_response = event.response- return-- if isinstance(event, Mapping):- self._consume_mapping_event(event)- return-- response = getattr(event, "response", None)- if isinstance(response, OpenAIResponse):- self._capture_response_metadata(response)-- def as_response(self) -> ChatResponse:- if isinstance(self._final_response, OpenAIResponse):- return ChatResponse.openai_responses_completion(self._final_response)- if isinstance(self._final_response, Mapping):- return ChatResponse.openai_responses_completion(- OpenAIResponse.model_validate(dict(self._final_response)),- )-- response = OpenAIResponse.model_validate({- "id": self._response_id or "resp_switchyard_stream",- "object": "response",- "created_at": self._created_at or time.time(),- "status": "completed",- "model": self._served_model,- "output": self._output_items(),- "parallel_tool_calls": False,- "tool_choice": "auto",- "tools": [],- "usage": self._usage.model_dump(mode="json") if self._usage else None,- })- return ChatResponse.openai_responses_completion(response)-- def _consume_mapping_event(self, event: Mapping[str, object]) -> None:- event_type = event.get("type")- response = event.get("response")- if isinstance(response, Mapping):- self._capture_response_mapping(response)-- if event_type == "response.output_text.delta":- delta = event.get("delta")- if isinstance(delta, str):- self._content += delta- return- if event_type in {"response.output_item.added", "response.output_item.done"}:- self._merge_output_item_mapping(- event.get("item"),- _int_value(event.get("output_index"), 0),- )- return- if event_type == "response.function_call_arguments.delta":- tool_call = self._tool_calls.setdefault(- _int_value(event.get("output_index"), 0),- _ResponsesToolCallState(),- )- delta = event.get("delta")- if isinstance(delta, str):- tool_call.arguments += delta- return- if event_type == "response.completed" and isinstance(response, Mapping):- self._final_response = response-- def _capture_response_metadata(self, response: OpenAIResponse) -> None:- self._response_id = response.id or self._response_id- self._served_model = response.model or self._served_model- self._created_at = response.created_at or self._created_at- if response.usage is not None:- self._usage = response.usage-- def _capture_response_mapping(self, response: Mapping[str, object]) -> None:- response_id = response.get("id")- if isinstance(response_id, str):- self._response_id = response_id- model = response.get("model")- if isinstance(model, str):- self._served_model = model- created = response.get("created_at") or response.get("created")- if isinstance(created, (int, float)):- self._created_at = float(created)- usage = response.get("usage")- if isinstance(usage, Mapping):- self._usage = _responses_usage_from_mapping(usage)-- def _merge_output_item(self, item: object, output_index: int) -> None:- if isinstance(item, ResponseOutputMessage):- content = _extract_responses_message_text(item)- if content and not self._content:- self._content = content- return- if not isinstance(item, ResponseFunctionToolCall):- return- tool_call = self._tool_calls.setdefault(output_index, _ResponsesToolCallState())- tool_call.id = item.id- tool_call.call_id = item.call_id- tool_call.name = item.name- tool_call.arguments = item.arguments-- def _merge_output_item_mapping(self, item: object, output_index: int) -> None:- if not isinstance(item, Mapping):- return- item_type = item.get("type")- if item_type == "message":- content = _extract_responses_message_text_mapping(item)- if content and not self._content:- self._content = content- return- if item_type != "function_call":- return- tool_call = self._tool_calls.setdefault(output_index, _ResponsesToolCallState())- call_id = item.get("call_id") or item.get("id")- if isinstance(call_id, str):- tool_call.call_id = call_id- name = item.get("name")- if isinstance(name, str):- tool_call.name = name- arguments = item.get("arguments")- if isinstance(arguments, str):- tool_call.arguments = arguments-- def _output_items(self) -> list[dict[str, object]]:- output: list[dict[str, object]] = []- if self._content:- output.append({- "type": "message",- "role": "assistant",- "status": "completed",- "content": [{"type": "output_text", "text": self._content}],- })- for _, call in sorted(self._tool_calls.items()):- item = call.to_response_output_item(- fallback_id=f"fc_switchyard_{len(output)}",- )- if item is not None:- output.append(item.model_dump(mode="json", exclude_none=True))- return output---def _coerce_anthropic_event(- event: object,-) -> RawMessageStreamEvent | Mapping[str, object]:- if isinstance(event, Mapping):- try:- return _ANTHROPIC_EVENT_ADAPTER.validate_python(event)- except ValidationError:- return event- return cast(RawMessageStreamEvent, event)---def _json_argument_string(value: object) -> str:- if isinstance(value, str):- return value- if isinstance(value, Mapping):- return json.dumps(dict(value))- return ""---def _json_object_from_argument_string(value: object) -> dict[str, object]:- if isinstance(value, Mapping):- return dict(value)- if not isinstance(value, str) or not value:- return {}- try:- parsed = json.loads(value)- except json.JSONDecodeError:- return {}- return dict(parsed) if isinstance(parsed, Mapping) else {}---def _openai_finish_reason(- value: str | None,- *,- has_tools: bool,-) -> _OpenAIFinishReason:- if value in {"stop", "length", "tool_calls", "content_filter", "function_call"}:- return cast(_OpenAIFinishReason, value)- return "tool_calls" if has_tools else "stop"---def _responses_usage_from_mapping(usage: Mapping[str, object]) -> ResponseUsage:- input_tokens = _int_value(usage.get("input_tokens"), 0)- output_tokens = _int_value(usage.get("output_tokens"), 0)- total_tokens = _int_value(- usage.get("total_tokens"),- input_tokens + output_tokens,- )- input_details = usage.get("input_tokens_details")- output_details = usage.get("output_tokens_details")- return ResponseUsage(- input_tokens=input_tokens,- output_tokens=output_tokens,- total_tokens=total_tokens,- input_tokens_details=InputTokensDetails(- cached_tokens=(- _int_value(input_details.get("cached_tokens"), 0)- if isinstance(input_details, Mapping)- else 0- ),- ),- output_tokens_details=OutputTokensDetails(- reasoning_tokens=(- _int_value(output_details.get("reasoning_tokens"), 0)- if isinstance(output_details, Mapping)- else 0- ),- ),- )---def _extract_responses_message_text(item: ResponseOutputMessage) -> str:- parts: list[str] = []- for part in item.content:- if isinstance(part, ResponseOutputText):- parts.append(part.text)- return "".join(parts)---def _extract_responses_message_text_mapping(item: Mapping[str, object]) -> str:- content = item.get("content")- if not isinstance(content, list):- return ""- parts: list[str] = []- for part in content:- if not isinstance(part, Mapping):- continue- if part.get("type") not in {"output_text", "text"}:- continue- text = part.get("text")- if isinstance(text, str):- parts.append(text)- return "".join(parts)---def _int_value(value: object, default: int) -> int:- return value if isinstance(value, int) else default---def _str_value(value: object) -> str | None:- return value if isinstance(value, str) and value else Nonediff --git a/switchyard/lib/conversation_turn.py b/switchyard/lib/conversation_turn.pydeleted file mode 100644index b075dc4cd..000000000--- a/switchyard/lib/conversation_turn.py+++ /dev/null@@ -1,65 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Turn counting helpers for conversation-scoped routing decisions."""--from __future__ import annotations--from typing import TYPE_CHECKING, Any--if TYPE_CHECKING:- from switchyard_rust.core import ChatRequest---def conversation_turn_number(request: ChatRequest) -> int:- """Return the 1-indexed LLM invocation number for ``request``.-- OpenAI Chat and Anthropic Messages count prior assistant messages. OpenAI- Responses uses a coarse acknowledgement count because responses can emit- multiple model-side items per turn. Unknown or malformed request bodies are- treated as turn 1 so routing falls back to first-turn behavior.- """- from switchyard_rust.core import ChatRequestType-- body = getattr(request, "body", None)- if not isinstance(body, dict):- return 1-- match request.request_type:- case ChatRequestType.OPENAI_CHAT | ChatRequestType.ANTHROPIC:- return _count_assistant_messages(body.get("messages")) + 1- case ChatRequestType.OPENAI_RESPONSES:- return _count_responses_turn(body)---def _count_assistant_messages(messages: Any) -> int:- """Count prior assistant messages in chat-style request bodies."""- if not isinstance(messages, list):- return 0- return sum(- 1- for msg in messages- if isinstance(msg, dict) and msg.get("role") == "assistant"- )---def _count_responses_turn(body: dict[str, Any]) -> int:- """Approximate the turn number for an OpenAI Responses request body."""- input_val = body.get("input")- if isinstance(input_val, str):- return 1- if not isinstance(input_val, list):- return 1-- acks = 0- for item in input_val:- if not isinstance(item, dict):- continue- if item.get("role") == "user":- acks += 1- elif item.get("type") == "function_call_output":- acks += 1- return max(acks, 1)---__all__ = ["conversation_turn_number"]diff --git a/switchyard/lib/endpoints/__init__.py b/switchyard/lib/endpoints/__init__.pydeleted file mode 100644index fa6c61bd1..000000000--- a/switchyard/lib/endpoints/__init__.py+++ /dev/null@@ -1,64 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""HTTP endpoint modules (and their SSE helpers).--Note: HTTP endpoint classes require fastapi (install with [server] extra).-They are lazily loaded to avoid hard dependency on fastapi for library-only users.-"""--from typing import TYPE_CHECKING, Any--if TYPE_CHECKING:- from switchyard.lib.endpoints.anthropic_messages_endpoint import (- AnthropicMessagesEndpoint,- )- from switchyard.lib.endpoints.models_endpoint import ModelsEndpoint- from switchyard.lib.endpoints.openai_chat_endpoint import (- OpenAIChatEndpoint,- )- from switchyard.lib.endpoints.responses_endpoint import ResponsesEndpoint- from switchyard.lib.endpoints.routing_log_stats_endpoint import (- RoutingLogStatsEndpoint,- )- from switchyard.lib.endpoints.stats_endpoint import StatsEndpoint--__all__ = [- "StatsEndpoint",- "AnthropicMessagesEndpoint",- "ModelsEndpoint",- "OpenAIChatEndpoint",- "ResponsesEndpoint",- "RoutingLogStatsEndpoint",-]---def __getattr__(name: str) -> Any:- """Lazy load HTTP endpoint classes that require fastapi."""- if name == "StatsEndpoint":- from switchyard.lib.endpoints.stats_endpoint import StatsEndpoint- return StatsEndpoint- elif name == "AnthropicMessagesEndpoint":- from switchyard.lib.endpoints.anthropic_messages_endpoint import (- AnthropicMessagesEndpoint,- )- return AnthropicMessagesEndpoint- elif name == "OpenAIChatEndpoint":- from switchyard.lib.endpoints.openai_chat_endpoint import (- OpenAIChatEndpoint,- )- return OpenAIChatEndpoint- elif name == "ModelsEndpoint":- from switchyard.lib.endpoints.models_endpoint import ModelsEndpoint- return ModelsEndpoint- elif name == "ResponsesEndpoint":- from switchyard.lib.endpoints.responses_endpoint import (- ResponsesEndpoint,- )- return ResponsesEndpoint- elif name == "RoutingLogStatsEndpoint":- from switchyard.lib.endpoints.routing_log_stats_endpoint import (- RoutingLogStatsEndpoint,- )- return RoutingLogStatsEndpoint- raise AttributeError(f"module {__name__!r} has no attribute {name!r}")diff --git a/switchyard/lib/endpoints/anthropic_messages_endpoint.py b/switchyard/lib/endpoints/anthropic_messages_endpoint.pydeleted file mode 100644index a497a4075..000000000--- a/switchyard/lib/endpoints/anthropic_messages_endpoint.py+++ /dev/null@@ -1,123 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""HTTP endpoint serving a ``Switchyard`` as ``POST /v1/messages`` (Anthropic Messages API).--Paper-thin by design: wrap the raw JSON body in a Rust-backed Anthropic request,-run the chain, serialize the result. All Anthropic ↔ OpenAI format-conversion lives inside the chain (``TranslationEngine`` and-``TranslationEngine``), so the endpoint itself contains zero-translation logic.--Streaming contract:--- When the request body carries ``"stream": true``, the chain's- translation engine surfaces an async iterator of Anthropic event dicts; the- endpoint frames them into Anthropic-style named-event SSE- (``event: message_start\\ndata: {...}\\n\\n``, …) via- :func:`iter_anthropic_sse`.-- Non-streaming requests return the Anthropic ``Message`` body as JSON.-"""--import logging-from typing import Annotated, Any--from fastapi import APIRouter, Body, FastAPI, Request-from fastapi.responses import Response--from switchyard.lib.endpoints.base import Endpoint as NemoSwitchyardEndpoint-from switchyard.lib.endpoints.dispatch import dispatch_chat_request, serialize_chain_result-from switchyard.lib.endpoints.sse_helpers import iter_anthropic_sse-from switchyard.lib.endpoints.upstream_error import (- context_exhausted_response,- handle_chain_exception,-)-from switchyard.lib.proxy_context import ProxyContext-from switchyard.lib.request_metadata import (- RequestMetadata,- attach_caller_api_key,- attach_request_metadata,-)-from switchyard_rust.core import (- ChatRequest,- SwitchyardContextPoolExhaustedError,- SwitchyardContextWindowExceededError,-)--log = logging.getLogger(__name__)---def _strip_unsupported_output_config(body: dict[str, Any]) -> None:- """Drop ``output_config.format`` from an inbound Anthropic body in place.-- Claude Code 2.1.1x sends ``output_config.format`` (a structured-output- schema) that upstream Anthropic model groups reject with HTTP 400.- ``output_config.effort`` is accepted, so only the ``format`` key is- removed; if that leaves ``output_config`` empty it is dropped entirely.- """- oc = body.get("output_config")- if isinstance(oc, dict) and "format" in oc:- oc.pop("format", None)- if not oc:- body.pop("output_config", None)---class AnthropicMessagesEndpoint(NemoSwitchyardEndpoint):- """Composable endpoint that exposes ``POST /v1/messages``."""-- def register(self, app: FastAPI) -> None:- """Attach ``POST /v1/messages`` onto *app*."""- router = APIRouter()-- @router.post("/v1/messages", response_model=None)- async def anthropic_messages(- request: Request,- body: Annotated[dict[str, Any], Body(...)],- ) -> Response:- """Anthropic-compatible Messages endpoint."""- obj = request.app.state.switchyard- _strip_unsupported_output_config(body)- model = str(body.get("model", "<none>"))- stream = bool(body.get("stream"))- log.debug(- "POST /v1/messages model=%s stream=%s keys=%s",- model,- stream,- list(body.keys()),- )- ctx = ProxyContext()- attach_request_metadata(- ctx,- RequestMetadata.from_headers(request.headers),- request.headers,- )- attach_caller_api_key(ctx, request.headers)-- chat_request = ChatRequest.anthropic(body)- # Reject semantically invalid input (e.g. empty messages) at the- # inbound boundary; raises SwitchyardInvalidRequestError -> 400.- chat_request.validate()-- try:- result: Any = await dispatch_chat_request(obj, chat_request, ctx)- if not isinstance(result, Response):- log.debug(- "POST /v1/messages chain returned model=%s stream=%s result=%s",- model,- stream,- type(result).__name__,- )- return serialize_chain_result(- result, stream=stream, sse_iter=iter_anthropic_sse, ctx=ctx- )- except (SwitchyardContextPoolExhaustedError, SwitchyardContextWindowExceededError) as exc:- return context_exhausted_response(exc, inbound="anthropic")- except Exception as exc:- return handle_chain_exception(- exc,- ctx,- inbound="anthropic",- log_msg=f"POST /v1/messages chain raised model={model}",- )-- app.include_router(router, tags=["Anthropic Compatible"])diff --git a/switchyard/lib/endpoints/base.py b/switchyard/lib/endpoints/base.pydeleted file mode 100644index 8e4ac59db..000000000--- a/switchyard/lib/endpoints/base.py+++ /dev/null@@ -1,39 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Abstract base class for HTTP endpoints."""--from abc import ABC, abstractmethod-from typing import TYPE_CHECKING, ClassVar--if TYPE_CHECKING:- from fastapi import FastAPI---class Endpoint(ABC):- """- Abstract base class for composable HTTP endpoint modules.-- Each endpoint module encapsulates a set of HTTP routes (e.g., OpenAI chat,- Anthropic messages, health checks) and registers them onto a FastAPI app.-- Endpoint modules are composed into a list and registered via- :func:`build_switchyard_app`.- """-- register_once: ClassVar[bool] = False- """Whether only the first instance of this concrete endpoint type is mounted."""-- @abstractmethod- def register(self, app: "FastAPI") -> None:- """Register this endpoint's routes onto the FastAPI application.-- Args:- app: The FastAPI application instance to register routes on.- """- pass-- @property- def name(self) -> str:- """Human-readable name for logging and introspection."""- return type(self).__name__diff --git a/switchyard/lib/endpoints/dispatch.py b/switchyard/lib/endpoints/dispatch.pydeleted file mode 100644index 12f55e1a9..000000000--- a/switchyard/lib/endpoints/dispatch.py+++ /dev/null@@ -1,126 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Shared endpoint dispatch for Python chains and model registries."""--from collections.abc import AsyncIterator, Callable-from typing import Any, cast--from fastapi.responses import JSONResponse, Response, StreamingResponse--from switchyard.lib.endpoints.error_envelope import error_response-from switchyard.lib.endpoints.route_selection import route_selection_headers-from switchyard.lib.endpoints.upstream_error import record_upstream_attempt_success-from switchyard.lib.proxy_context import ProxyContext-from switchyard.lib.roles import TranslatedResponse-from switchyard.lib.route_table import RouteTable-from switchyard_rust.core import ChatRequest--_MISSING_MODEL_LABEL = "<missing>"---def model_not_found_response(model: str) -> JSONResponse:- """Build the OpenAI-compatible error payload for unknown model IDs."""- return error_response(- 404,- f"No route registered for model {model}",- error_type="model_not_found",- code="model_not_found",- )---def _model_label(model: object | None) -> str:- """Return a stable human label for model-not-found errors."""- return str(model) if model else _MISSING_MODEL_LABEL---def invalid_request_response(message: str, *, code: str = "invalid_request_error") -> JSONResponse:- """Build the OpenAI-compatible error payload for invalid requests."""- return error_response(- 400,- message,- error_type="invalid_request_error",- code=code,- )---async def dispatch_chat_request(- app_state: object,- chat_request: ChatRequest,- ctx: ProxyContext,-) -> TranslatedResponse | Response:- """Dispatch one request through the configured app state.-- Single-chain apps return already-translated Python payloads because- ``Switchyard`` still owns its terminal translator.- """- if isinstance(app_state, RouteTable):- model = _model_label(chat_request.model)- try:- table_chain = app_state.lookup_switchyard(model)- except KeyError:- # No upstream call happened — a 404 is not an upstream attempt.- return model_not_found_response(model)- result = await table_chain.call(chat_request, ctx=ctx)- record_upstream_attempt_success(ctx)- return result-- chain: Any = app_state- result = cast(TranslatedResponse, await chain.call(chat_request, ctx=ctx))- record_upstream_attempt_success(ctx)- return result---def model_entries(app_state: object) -> list[dict[str, Any]]:- """Return OpenAI-compatible model entries for table app state."""- if isinstance(app_state, RouteTable):- return app_state.registered_model_entries()- return []---def model_listing_warnings(app_state: object) -> list[str]:- """Return non-fatal model listing warnings for table-backed apps."""- if isinstance(app_state, RouteTable):- return app_state.model_listing_warnings()- return []---def model_listing_default(app_state: object) -> str | None:- """Return the default model id advertised by ``GET /v1/models``."""- if isinstance(app_state, RouteTable):- return app_state.default_model()- return None---def serialize_chain_result(- result: Any,- *,- stream: bool,- sse_iter: Callable[[Any], AsyncIterator[str]],- ctx: ProxyContext,-) -> Response:- """Serialize a chain result to the appropriate HTTP response.-- Returns the result itself if it is already a ``Response``, wraps it in a- ``StreamingResponse`` when streaming is requested, or JSON-serializes it.- Any route selection recorded on *ctx* is stamped as ``x-switchyard-*``- response headers on every branch, pre-built responses included (streaming- too — the backend call completed before the response object is built, so- the selection is final). ``ctx`` is required so a new endpoint cannot- silently opt out of spend attribution.- """- headers = route_selection_headers(ctx)- if isinstance(result, Response):- # Merge rather than pass through untouched: no current chain path- # yields a pre-built Response after a billed upstream success, but if- # one ever does, dropping the recorded selection here would silently- # break spend attribution.- result.headers.update(headers)- return result- if stream and hasattr(result, "__aiter__"):- return StreamingResponse(- sse_iter(result), media_type="text/event-stream", headers=headers- )- if hasattr(result, "model_dump"):- return JSONResponse(content=result.model_dump(), headers=headers)- return JSONResponse(content=result, headers=headers)diff --git a/switchyard/lib/endpoints/error_envelope.py b/switchyard/lib/endpoints/error_envelope.pydeleted file mode 100644index d4e995336..000000000--- a/switchyard/lib/endpoints/error_envelope.py+++ /dev/null@@ -1,171 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Shared HTTP error envelopes for Switchyard LLM-serving endpoints."""--import json-from collections.abc import Mapping-from dataclasses import dataclass-from typing import Any--from fastapi.responses import JSONResponse--from switchyard.lib.proxy_context import ERROR_SOURCE_PROVIDER, ERROR_SOURCE_SWITCHYARD--_DEFAULT_UPSTREAM_MESSAGE = "upstream returned HTTP {status}"--#: Response header naming the layer that originated the error: ``switchyard``-#: (this proxy rejected or failed the request itself) or ``provider`` (an-#: upstream LLM failure passed through). Layers above Switchyard (e.g. a-#: LiteLLM front proxy) are expected to tag their own failures the same way-#: and propagate this header from below — Switchyard cannot see them. The-#: values live in :mod:`switchyard.lib.proxy_context` so FastAPI-free backend-#: code can stamp them.-ERROR_SOURCE_HEADER = "x-switchyard-error-source"--#: Response header carrying the upstream model actually attempted when the-#: surfaced failure happened, when a routing selection took place.-UPSTREAM_MODEL_HEADER = "x-switchyard-upstream-model"---def error_payload(- message: str,- *,- error_type: str,- code: str,- extra: Mapping[str, object] | None = None,-) -> dict[str, dict[str, object]]:- """Return the normalized JSON body used by all LLM HTTP endpoints."""- error: dict[str, object] = {- "message": message,- "type": error_type,- "code": code,- }- if extra:- error.update(extra)- return {"error": error}---def error_response(- status_code: int,- message: str,- *,- error_type: str,- code: str,- extra: Mapping[str, object] | None = None,- error_source: str | None = ERROR_SOURCE_SWITCHYARD,- upstream_model: str | None = None,-) -> JSONResponse:- """Build a JSONResponse with Switchyard's normalized error envelope.-- Stamps the failure-source headers: every direct caller synthesizes a- Switchyard-originated envelope, so ``error_source`` defaults to- ``switchyard``; the upstream passthrough path overrides it with- ``provider``. Headers rather than body fields keep the passthrough- contract intact — provider error bodies flow through unmodified.- """- headers: dict[str, str] = {}- if error_source:- headers[ERROR_SOURCE_HEADER] = error_source- if upstream_model:- headers[UPSTREAM_MODEL_HEADER] = upstream_model- return JSONResponse(- status_code=status_code,- content=error_payload(message, error_type=error_type, code=code, extra=extra),- headers=headers or None,- )---def upstream_error_response(- status_code: int,- body: object,- *,- error_source: str = ERROR_SOURCE_PROVIDER,- upstream_model: str | None = None,-) -> JSONResponse:- """Normalize an upstream provider error body into Switchyard's envelope.-- ``error_source`` defaults to ``provider`` — this path renders upstream- failures — but a backend that deliberately routes its own rejection- through the upstream-status stash (e.g. the ``caller_required`` 401)- overrides it back to ``switchyard`` via ``ctx``.- """- parsed = _upstream_error_fields(status_code, body)- return error_response(- status_code,- parsed.message,- error_type=parsed.error_type,- code=parsed.code,- extra=parsed.extra,- error_source=error_source,- upstream_model=upstream_model,- )---@dataclass(frozen=True)-class _UpstreamErrorFields:- """Internal value object for provider error fields after normalization."""-- message: str- error_type: str = "upstream_error"- code: str = "upstream_error"- extra: Mapping[str, object] | None = None---def _upstream_error_fields(status_code: int, body: object) -> _UpstreamErrorFields:- """Extract stable error fields from common provider error shapes."""- default_message = _DEFAULT_UPSTREAM_MESSAGE.format(status=status_code)- if isinstance(body, str):- return _UpstreamErrorFields(message=body or default_message)- if isinstance(body, Mapping):- return _fields_from_mapping(status_code, body)- if isinstance(body, list):- return _UpstreamErrorFields(message=_compact_json(body))- return _UpstreamErrorFields(message=default_message)---def _fields_from_mapping(status_code: int, body: Mapping[str, object]) -> _UpstreamErrorFields:- """Handle OpenAI-style ``{"error": {...}}`` and flat error dictionaries."""- error = body.get("error")- source = error if isinstance(error, Mapping) else body- default_message = _DEFAULT_UPSTREAM_MESSAGE.format(status=status_code)-- message = _string_field(source, "message") or _compact_json(body) or default_message- error_type = _string_field(source, "type") or "upstream_error"- code = _string_or_number_field(source, "code") or (- error_type if error_type != "upstream_error" else "upstream_error"- )- extra = {- key: value- for key, value in {- "param": _string_or_number_field(source, "param"),- }.items()- if value is not None- }- return _UpstreamErrorFields(- message=message,- error_type=error_type,- code=code,- extra=extra,- )---def _string_field(source: Mapping[str, object], key: str) -> str | None:- value = source.get(key)- return value if isinstance(value, str) and value else None---def _string_or_number_field(source: Mapping[str, object], key: str) -> str | None:- value = source.get(key)- if isinstance(value, str) and value:- return value- if isinstance(value, int | float):- return str(value)- return None---def _compact_json(value: Any) -> str:- try:- return json.dumps(value, separators=(",", ":"), sort_keys=True)- except TypeError:- return str(value)diff --git a/switchyard/lib/endpoints/models_endpoint.py b/switchyard/lib/endpoints/models_endpoint.pydeleted file mode 100644index 9c735fdf8..000000000--- a/switchyard/lib/endpoints/models_endpoint.py+++ /dev/null@@ -1,42 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""HTTP endpoint serving ``GET /v1/models`` for local model discovery."""--import logging--from fastapi import APIRouter, FastAPI, Request-from fastapi.responses import JSONResponse--from switchyard.lib.endpoints.base import Endpoint as NemoSwitchyardEndpoint-from switchyard.lib.endpoints.dispatch import (- model_entries,- model_listing_default,- model_listing_warnings,-)-from switchyard.lib.model_listing import model_list_payload--log = logging.getLogger(__name__)---class ModelsEndpoint(NemoSwitchyardEndpoint):- """Expose registered model ids for clients with model discovery."""-- def register(self, app: FastAPI) -> None:- """Attach ``GET /v1/models`` onto *app*."""- router = APIRouter()-- @router.get("/v1/models", response_model=None)- async def models(request: Request) -> JSONResponse:- obj = request.app.state.switchyard- entries = model_entries(obj)- log.debug("GET /v1/models returned %d model(s)", len(entries))- return JSONResponse(- content=model_list_payload(- entries,- default_model=model_listing_default(obj),- warnings=model_listing_warnings(obj),- )- )-- app.include_router(router, tags=["Model Discovery"])diff --git a/switchyard/lib/endpoints/openai_chat_endpoint.py b/switchyard/lib/endpoints/openai_chat_endpoint.pydeleted file mode 100644index fb3880df5..000000000--- a/switchyard/lib/endpoints/openai_chat_endpoint.py+++ /dev/null@@ -1,102 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""HTTP endpoint serving a ``Switchyard`` as ``POST /v1/chat/completions``.--The class is stateless — at request time it reads the switchyard from-``request.app.state.switchyard``. Wire-up is performed by the-``build_switchyard_app()`` convenience factory.--Streaming contract:--- When the request body carries ``"stream": true``, the chain's- translation engine surfaces an async iterator of ``ChatCompletionChunk``; the- endpoint wraps it in a ``StreamingResponse`` emitting OpenAI-style- SSE frames (``data: {...}\\n\\n`` + ``data: [DONE]\\n\\n``).-- Upstream failures (auth, rate-limit, connection) surface before the- ``StreamingResponse`` is constructed — they propagate as exceptions- to the global handler and map to proper HTTP error responses. Only- mid-stream iteration errors land in the SSE error branch of- :func:`iter_chat_completion_sse`.-"""--import logging-from typing import Annotated, Any--from fastapi import APIRouter, Body, FastAPI, Request-from fastapi.responses import Response--from switchyard.lib.endpoints.base import Endpoint as NemoSwitchyardEndpoint-from switchyard.lib.endpoints.dispatch import dispatch_chat_request, serialize_chain_result-from switchyard.lib.endpoints.sse_helpers import iter_chat_completion_sse-from switchyard.lib.endpoints.upstream_error import (- context_exhausted_response,- handle_chain_exception,-)-from switchyard.lib.proxy_context import ProxyContext-from switchyard.lib.request_metadata import (- RequestMetadata,- attach_caller_api_key,- attach_request_metadata,-)-from switchyard_rust.core import (- ChatRequest,- SwitchyardContextPoolExhaustedError,- SwitchyardContextWindowExceededError,-)--log = logging.getLogger(__name__)---class OpenAIChatEndpoint(NemoSwitchyardEndpoint):- """Composable endpoint that exposes ``POST /v1/chat/completions``.-- Reads the raw JSON body, wraps it in a Rust-backed OpenAI chat request (no- validation or field-stripping, so provider-specific fields pass- through transparently), runs the chain, and either JSON-serializes- the result (non-streaming) or wraps the async chunk iterator in an- SSE ``StreamingResponse`` (streaming).-- Streaming support is limited to same-format passthrough today —- i.e. OpenAI Chat Completions inbound against an OpenAI-native- backend. Cross-format streaming (Anthropic / Responses inbound)- raises ``NotImplementedError`` from ``TranslationEngine``- until streaming translation lands for those formats.- """-- def register(self, app: FastAPI) -> None:- """Attach ``POST /v1/chat/completions`` onto *app*."""- router = APIRouter()-- @router.post("/v1/chat/completions", response_model=None)- async def chat_completions(- request: Request,- body: Annotated[dict[str, Any], Body(...)],- ) -> Response:- """OpenAI-compatible Chat Completions endpoint."""- obj = request.app.state.switchyard- chat_request = ChatRequest.openai_chat(body)- # Reject semantically invalid input (e.g. empty messages) at the- # inbound boundary; raises SwitchyardInvalidRequestError -> 400.- chat_request.validate()- ctx = ProxyContext()- attach_request_metadata(- ctx,- RequestMetadata.from_headers(request.headers),- request.headers,- )- attach_caller_api_key(ctx, request.headers)- stream = bool(body.get("stream"))- try:- result: Any = await dispatch_chat_request(obj, chat_request, ctx)- return serialize_chain_result(- result, stream=stream, sse_iter=iter_chat_completion_sse, ctx=ctx- )- except (SwitchyardContextPoolExhaustedError, SwitchyardContextWindowExceededError) as exc:- return context_exhausted_response(exc, inbound="openai")- except Exception as exc:- return handle_chain_exception(- exc, ctx, inbound="openai", log_msg="POST /v1/chat/completions chain raised"- )-- app.include_router(router, tags=["OpenAI Compatible"])diff --git a/switchyard/lib/endpoints/outcome_metrics.py b/switchyard/lib/endpoints/outcome_metrics.pydeleted file mode 100644index dacf87fbf..000000000--- a/switchyard/lib/endpoints/outcome_metrics.py+++ /dev/null@@ -1,249 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Outcome counters used to compute router-vs-direct error-rate ratios.--Three process-wide counters published on ``/metrics``:--* ``switchyard_client_responses_total{outcome}`` — every HTTP response- returned to a client on an LLM-serving route. The denominator for the- router-served error rate.-* ``switchyard_upstream_attempts_total{outcome, code}`` — every individual- upstream call attempt (including ones absorbed by retry). The- denominator for the direct-to-endpoint baseline error rate. The ``code``- label carries the raw upstream HTTP status (``"429"``, ``"500"`` …) so a- dashboard can plot the error-code distribution over time;- ``code="none"`` marks a non-HTTP failure (network error, pre-status- timeout) that has no status code. Unknown codes are clamped to their- class (``"4xx"`` / ``"5xx"`` / …) to keep label cardinality bounded.-* ``switchyard_router_retry_recovered_total`` — global counter- incremented whenever a request's first upstream attempt failed and a- subsequent attempt succeeded — direct evidence the steering logic- rescued the request.--Bucket semantics:--* ``success`` — HTTP 2xx.-* ``retryable_error`` — HTTP 429 / 500 / 504 — the categories the- success criterion measures (router should be absorbing these).-* ``other_error`` — everything else (400 / 401 / 403 / 422 / …),- i.e. bad payload, bad credentials, high-reasoning timeout. Excluded- from the success criterion per the spec.--Computing the ratios from these::-- router_error_rate = client_responses{outcome="retryable_error"}- / sum(client_responses)-- direct_error_rate = sum(upstream_attempts{outcome="retryable_error"})- / sum(upstream_attempts)-- error_rate_reduction = direct_error_rate - router_error_rate--Because ``upstream_attempts`` now carries the ``code`` label, a bare-selector returns one series per code — always aggregate it with ``sum()``-when you want the layer total. The error-code distribution itself is just-``sum by (code) (rate(upstream_attempts{code!="200"}[5m]))``.--The two layers have different denominators by design: one client request-can produce N upstream attempts (retry fan-out), and that asymmetry is-exactly the reason a health-aware router reduces the rate seen by the-client.-"""--from __future__ import annotations--from threading import Lock-from typing import Literal--OutcomeBucket = Literal["success", "retryable_error", "other_error"]--#: HTTP status codes the success criterion counts as router-rescuable-#: errors. 429 (rate limit), 500 (server error), 504 (gateway timeout).-RETRYABLE_STATUSES: frozenset[int] = frozenset({429, 500, 504})--#: Status codes emitted verbatim as the ``code`` label. Anything else seen-#: on the wire is clamped to its class (``"4xx"`` / ``"5xx"`` / …) so a-#: misbehaving upstream returning oddball codes cannot inflate label-#: cardinality. Covers the common success / client-error / server-error-#: codes an LLM endpoint actually returns.-KNOWN_STATUS_CODES: frozenset[int] = frozenset(- {200, 400, 401, 403, 404, 408, 409, 422, 429, 500, 502, 503, 504}-)--#: ``code`` label value for a non-HTTP failure (network error, pre-status-#: timeout) — the request never received a status line, so there is no code.-NO_STATUS_CODE: str = "none"--_lock = Lock()-_client_responses: dict[str, int] = {- "success": 0,- "retryable_error": 0,- "other_error": 0,-}---def _seed_upstream() -> dict[tuple[str, str], int]:- """Fresh upstream-attempt counter with canonical ``(outcome, code)`` series at 0.-- Seeding the codes a dashboard plots means their time series exist from- process start, so a Grafana ``rate()`` renders a flat zero line rather- than "no data" before the first matching attempt. Non-seeded codes (a- one-off 403, say) are created lazily on first occurrence.- """- return {- ("success", "200"): 0,- ("retryable_error", "429"): 0,- ("retryable_error", "500"): 0,- ("retryable_error", "504"): 0,- ("retryable_error", NO_STATUS_CODE): 0,- }---#: Keyed by ``(outcome, code)``: ``code`` is the upstream HTTP status as a-#: string, ``"none"`` for a non-HTTP failure, or a clamped ``"Nxx"`` class.-_upstream_attempts: dict[tuple[str, str], int] = _seed_upstream()-_retry_recovered: int = 0---def classify(status_code: int) -> OutcomeBucket:- """Map an HTTP status code to its outcome bucket.-- 2xx → ``success``. The codes listed in :data:`RETRYABLE_STATUSES`- (429 / 500 / 504) → ``retryable_error``. Everything else (1xx, 3xx,- most 4xx, other 5xx) → ``other_error``.- """- if 200 <= status_code < 300:- return "success"- if status_code in RETRYABLE_STATUSES:- return "retryable_error"- return "other_error"---def code_label(status_code: int | None) -> str:- """Render the ``code`` label for one upstream attempt.-- ``None`` (non-HTTP failure) → :data:`NO_STATUS_CODE`. A code in- :data:`KNOWN_STATUS_CODES` is emitted verbatim (``"429"``). Any other- HTTP code is clamped to its class (``"4xx"``, ``"5xx"``, …), and an- out-of-range value to ``"other"``, so label cardinality stays bounded- no matter what an upstream returns.- """- if status_code is None:- return NO_STATUS_CODE- if status_code in KNOWN_STATUS_CODES:- return str(status_code)- if 100 <= status_code < 600:- return f"{status_code // 100}xx"- return "other"---def record_client_response(status_code: int) -> None:- """Record one HTTP response sent to a client on an LLM-serving route."""- bucket = classify(status_code)- with _lock:- _client_responses[bucket] += 1---def record_upstream_attempt(status_code: int | None) -> None:- """Record one individual upstream attempt outcome.-- ``status_code=None`` is used for non-HTTP failures (network errors,- pre-status timeouts) and is bucketed as ``retryable_error`` — those- are exactly the kind of fault a health-aware router should be able- to absorb by retrying on a different endpoint.- """- bucket: OutcomeBucket- if status_code is None:- bucket = "retryable_error"- else:- bucket = classify(status_code)- key = (bucket, code_label(status_code))- with _lock:- _upstream_attempts[key] = _upstream_attempts.get(key, 0) + 1---def record_retry_recovered() -> None:- """Record that a retry succeeded after at least one prior attempt failed.-- Direct evidence the router's steering logic kicked in: without- retry, this request would have surfaced as a client-side error.- """- global _retry_recovered- with _lock:- _retry_recovered += 1---def render_lines() -> list[str]:- """Render the current counter state as Prometheus exposition lines.-- Returns an ordered list of lines (no trailing newline). The- ``/metrics`` endpoint concatenates this with the accumulator output.- """- with _lock:- client = dict(_client_responses)- upstream = dict(_upstream_attempts)- recovered = _retry_recovered-- lines: list[str] = []- lines.append(- "# HELP switchyard_client_responses_total "- "HTTP responses returned to clients on LLM-serving routes, "- "bucketed by outcome (success / retryable_error / other_error)."- )- lines.append("# TYPE switchyard_client_responses_total counter")- for outcome in ("success", "retryable_error", "other_error"):- lines.append(- f'switchyard_client_responses_total{{outcome="{outcome}"}} '- f"{client[outcome]}"- )-- lines.append(- "# HELP switchyard_upstream_attempts_total "- "Individual upstream call attempts, bucketed by outcome and labelled "- "with the upstream HTTP status code (code=\"none\" for non-HTTP "- "failures). One client request can produce multiple attempts via retry."- )- lines.append("# TYPE switchyard_upstream_attempts_total counter")- # Sorted for deterministic exposition order across scrapes.- for outcome, code in sorted(upstream):- lines.append(- f'switchyard_upstream_attempts_total{{outcome="{outcome}",code="{code}"}} '- f"{upstream[(outcome, code)]}"- )-- lines.append(- "# HELP switchyard_router_retry_recovered_total "- "Requests whose first upstream attempt failed but a subsequent "- "attempt succeeded — direct evidence steering logic rescued the request."- )- lines.append("# TYPE switchyard_router_retry_recovered_total counter")- lines.append(f"switchyard_router_retry_recovered_total {recovered}")-- return lines---def _reset_for_tests() -> None:- """Zero every counter — tests only."""- global _retry_recovered, _upstream_attempts- with _lock:- for key in _client_responses:- _client_responses[key] = 0- # Rebuild rather than zero in place: drops any lazily-added codes so- # each test starts from the same canonical seed.- _upstream_attempts = _seed_upstream()- _retry_recovered = 0---__all__ = [- "KNOWN_STATUS_CODES",- "NO_STATUS_CODE",- "RETRYABLE_STATUSES",- "OutcomeBucket",- "classify",- "code_label",- "record_client_response",- "record_retry_recovered",- "record_upstream_attempt",- "render_lines",-]diff --git a/switchyard/lib/endpoints/prometheus_emitter.py b/switchyard/lib/endpoints/prometheus_emitter.pydeleted file mode 100644index aafe0274c..000000000--- a/switchyard/lib/endpoints/prometheus_emitter.py+++ /dev/null@@ -1,70 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Process-wide table for objects that contribute extra ``/metrics`` lines.--The accumulator-derived exposition rendered by-:func:`switchyard.lib.prometheus_exposition.render_prometheus` covers-request flow (counts, tokens, latency). Components that own state which-is not request-derived register an emitter here so their lines appear on-the same ``/metrics`` scrape rather than a sidecar URL.--Single-process table by design: a Switchyard server is one process,-emitters are owned by component lifetimes, and the table is-write-once-read-many across startup.-"""--from __future__ import annotations--from collections.abc import Callable--PrometheusEmitter = Callable[[], list[str]]-"""A no-arg callable returning Prometheus exposition lines (no trailing newline).--Each call snapshots the emitter's current state. The table composes-output in registration order; emitters must not assume any ordering-relative to other emitters or to the accumulator-derived block.-"""--_EMITTERS: list[PrometheusEmitter] = []---def register(emitter: PrometheusEmitter) -> None:- """Register an emitter. Idempotent — re-registering is a no-op."""- if emitter not in _EMITTERS:- _EMITTERS.append(emitter)---def unregister(emitter: PrometheusEmitter) -> None:- """Remove a previously-registered emitter. No-op if not present.-- Backends call this from their ``shutdown()`` hook so a re-built chain- does not leave a stale closure pointing at a torn-down backend.- """- try:- _EMITTERS.remove(emitter)- except ValueError:- pass---def render() -> str:- """Compose registered emitter output as Prometheus exposition text.-- Returns an empty string when no emitter is registered, so callers can- unconditionally concatenate the result to the accumulator-derived- exposition without producing trailing whitespace artefacts.- """- lines: list[str] = []- for emitter in _EMITTERS:- lines.extend(emitter())- if not lines:- return ""- return "\n".join(lines) + "\n"---def _clear_for_tests() -> None:- """Drop every registered emitter — test fixtures only."""- _EMITTERS.clear()---__all__ = ["PrometheusEmitter", "register", "unregister", "render"]diff --git a/switchyard/lib/endpoints/responses_endpoint.py b/switchyard/lib/endpoints/responses_endpoint.pydeleted file mode 100644index d5ae97e2e..000000000--- a/switchyard/lib/endpoints/responses_endpoint.py+++ /dev/null@@ -1,101 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""HTTP endpoint serving a ``Switchyard`` as ``POST /v1/responses`` (OpenAI Responses API).--Paper-thin by design: wrap the raw JSON body in a Rust-backed Responses request,-run the chain, serialize the result. All Responses ↔ Chat Completions-format conversion lives inside the chain's ``TranslationEngine``, so the-endpoint itself contains zero translation logic.--Streaming contract:--- When the request body carries ``"stream": true``, the chain's- translation engine surfaces an async iterator of pre-formatted Responses API- SSE frames; :func:`iter_preframed_sse` forwards them verbatim through- a ``StreamingResponse`` with mid-stream error quarantine.-- Non-streaming requests return the Responses ``Response`` body as JSON.-"""--import logging-from typing import Annotated, Any--from fastapi import APIRouter, Body, FastAPI, Request-from fastapi.responses import Response--from switchyard.lib.endpoints.base import Endpoint as NemoSwitchyardEndpoint-from switchyard.lib.endpoints.dispatch import dispatch_chat_request, serialize_chain_result-from switchyard.lib.endpoints.sse_helpers import iter_preframed_sse-from switchyard.lib.endpoints.upstream_error import (- context_exhausted_response,- handle_chain_exception,-)-from switchyard.lib.proxy_context import ProxyContext-from switchyard.lib.request_metadata import (- RequestMetadata,- attach_caller_api_key,- attach_request_metadata,-)-from switchyard_rust.core import (- ChatRequest,- SwitchyardContextPoolExhaustedError,- SwitchyardContextWindowExceededError,-)--log = logging.getLogger(__name__)---class ResponsesEndpoint(NemoSwitchyardEndpoint):- """Composable endpoint that exposes ``POST /v1/responses``."""-- def register(self, app: FastAPI) -> None:- """Attach ``POST /v1/responses`` onto *app*."""- router = APIRouter()-- @router.post("/v1/responses", response_model=None)- async def responses(- request: Request,- body: Annotated[dict[str, Any], Body(...)],- ) -> Response:- """OpenAI-compatible Responses endpoint."""- obj = request.app.state.switchyard- model = str(body.get("model", "<none>"))- stream = bool(body.get("stream"))- log.debug(- "POST /v1/responses model=%s stream=%s keys=%s",- model,- stream,- list(body.keys()),- )-- chat_request = ChatRequest.openai_responses(body)- ctx = ProxyContext()- attach_request_metadata(- ctx,- RequestMetadata.from_headers(request.headers),- request.headers,- )- attach_caller_api_key(ctx, request.headers)- try:- result: Any = await dispatch_chat_request(obj, chat_request, ctx)- if not isinstance(result, Response):- log.debug(- "POST /v1/responses chain returned model=%s stream=%s result=%s",- model,- stream,- type(result).__name__,- )- return serialize_chain_result(- result, stream=stream, sse_iter=iter_preframed_sse, ctx=ctx- )- except (SwitchyardContextPoolExhaustedError, SwitchyardContextWindowExceededError) as exc:- return context_exhausted_response(exc, inbound="openai-responses")- except Exception as exc:- return handle_chain_exception(- exc,- ctx,- inbound="openai-responses",- log_msg=f"POST /v1/responses chain raised model={model}",- )-- app.include_router(router, tags=["OpenAI Responses"])diff --git a/switchyard/lib/endpoints/route_selection.py b/switchyard/lib/endpoints/route_selection.pydeleted file mode 100644index 209110535..000000000--- a/switchyard/lib/endpoints/route_selection.py+++ /dev/null@@ -1,68 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Route-selection response headers for spend/tokenomics attribution.--Maps the :data:`CTX_ROUTE_SELECTION` record a routing backend stored on-``ctx`` (see :mod:`switchyard.lib.proxy_context`) to the ``x-switchyard-*``-response headers a front proxy such as LiteLLM copies into its parent-spend-log row. Shared by the success serializer (``dispatch``) and the-error path (``upstream_error``) — a failure that happens *after* a billed-upstream success must still expose the selection, or the provider spend-log-row's correlation id becomes unjoinable.-"""--from collections.abc import Mapping--from switchyard.lib.proxy_context import CTX_ROUTE_SELECTION, ProxyContext--#: Response headers exposing the route selection behind an upstream call,-#: carrying the same correlation id the provider row received via the-#: outbound ``x-litellm-spend-logs-metadata`` header.-ROUTER_MODEL_HEADER = "x-switchyard-router-model"-SELECTED_MODEL_HEADER = "x-switchyard-selected-model"-SELECTED_PROVIDER_HEADER = "x-switchyard-selected-provider"-ROUTER_CORRELATION_ID_HEADER = "x-switchyard-router-correlation-id"--_ROUTE_SELECTION_RESPONSE_HEADERS = (- (ROUTER_MODEL_HEADER, "router_model"),- (SELECTED_MODEL_HEADER, "router_selected_model"),- (SELECTED_PROVIDER_HEADER, "router_selected_provider"),- (ROUTER_CORRELATION_ID_HEADER, "router_correlation_id"),-)---def _is_header_value_safe(value: str) -> bool:- """Whether *value* can be emitted as an HTTP/1.1 response-header value.-- ``router_model`` echoes the client-supplied model string, so it must be- re-validated as header material: Starlette encodes response-header values- as latin-1 (a non-encodable value would fail response construction after- the upstream call already succeeded and was billed), and CTL characters —- CR/LF above all — would be a response-splitting vector on permissive- ASGI stacks.- """- try:- value.encode("latin-1")- except UnicodeEncodeError:- return False- return not any(ord(ch) < 0x20 or ord(ch) == 0x7F for ch in value)---def route_selection_headers(ctx: ProxyContext) -> dict[str, str]:- """Response headers for the route selection recorded on *ctx*, if any.-- Empty when no routing backend recorded a selection (passthrough chains,- failures before any upstream success). A recorded field that is absent or- not emittable as a header value is skipped — headers never carry- placeholder or unsafe values.- """- selection = ctx.metadata.get(CTX_ROUTE_SELECTION)- if not isinstance(selection, Mapping):- return {}- headers: dict[str, str] = {}- for header_name, selection_key in _ROUTE_SELECTION_RESPONSE_HEADERS:- value = selection.get(selection_key)- if isinstance(value, str) and value and _is_header_value_safe(value):- headers[header_name] = value- return headersdiff --git a/switchyard/lib/endpoints/routing_log_stats_endpoint.py b/switchyard/lib/endpoints/routing_log_stats_endpoint.pydeleted file mode 100644index 4491f877b..000000000--- a/switchyard/lib/endpoints/routing_log_stats_endpoint.py+++ /dev/null@@ -1,45 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""HTTP access to session-scoped aggregates from the durable routing log."""--from __future__ import annotations--import asyncio-from typing import TYPE_CHECKING--from fastapi import APIRouter, HTTPException--from switchyard.lib.endpoints.base import Endpoint--if TYPE_CHECKING:- from fastapi import FastAPI-- from switchyard.lib.processors.routing_log_response_processor import (- RoutingLogResponseProcessor,- )---class RoutingLogStatsEndpoint(Endpoint):- """Expose one trial session's model and token aggregates."""-- register_once = True-- def __init__(self, processor: RoutingLogResponseProcessor) -> None:- self._processor = processor-- def register(self, app: FastAPI) -> None:- routes = APIRouter()- processor = self._processor-- async def get_session_stats(session_id: str) -> dict[str, object]:- snapshot = await asyncio.to_thread(processor.snapshot_session, session_id)- if snapshot is None:- raise HTTPException(status_code=404, detail="routing session not found")- return snapshot-- routes.get("/v1/routing/session-stats")(get_session_stats)- app.include_router(routes, tags=["Routing log"])---__all__ = ["RoutingLogStatsEndpoint"]diff --git a/switchyard/lib/endpoints/sse_helpers.py b/switchyard/lib/endpoints/sse_helpers.pydeleted file mode 100644index d40224bb8..000000000--- a/switchyard/lib/endpoints/sse_helpers.py+++ /dev/null@@ -1,207 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Shared SSE serialization helpers for HTTP endpoints.--Pure async generators that turn typed response streams into the raw-SSE frames FastAPI's ``StreamingResponse`` expects. Each wire format-(OpenAI Chat Completions, Anthropic Messages, Responses API) has its-own helper with the format-specific framing contract.--Error contract (shared across all helpers):-(:mod:`switchyard.server.endpoints.openai_endpoint`):--- The endpoint awaits ``switchyard.call()`` *before* handing the stream- to ``StreamingResponse``, so upstream auth / connection / rate-limit- failures raise exceptions that the global handler turns into proper- HTTP error responses.-- Only failures during chunk iteration land in ``except`` here. At- that point HTTP 200 has already been committed, so the best we can- do is emit a final SSE error frame (format-specific shape).-"""--from __future__ import annotations--import inspect-import json-import logging-from collections.abc import AsyncGenerator, AsyncIterator, Mapping--from anthropic.types import RawMessageStreamEvent-from openai.types.chat import ChatCompletionChunk--log = logging.getLogger(__name__)---async def _aclose_stream(stream: object) -> None:- """Best-effort close of an upstream response stream.-- On every exit path — normal completion, mid-stream error, or the client- disconnecting (which makes the ASGI server ``aclose()`` the SSE generator,- raising ``GeneratorExit`` at the suspended ``yield``) — the upstream stream- must be closed so its underlying httpx connection is returned to the pool- instead of leaking. ``ChatResponseStream`` and async generators expose- ``aclose``; SDK ``AsyncStream`` objects expose ``close``; either may be a- coroutine. Closing must never mask the original control flow, so failures- are logged and swallowed.- """- closer = getattr(stream, "aclose", None) or getattr(stream, "close", None)- if closer is None:- return- try:- result = closer()- if inspect.isawaitable(result):- await result- except Exception as exc:- log.debug("Failed to close upstream stream: %s: %s", type(exc).__name__, exc)---async def iter_chat_completion_sse(- stream: AsyncIterator[ChatCompletionChunk],-) -> AsyncGenerator[str, None]:- """Serialize a Chat Completions chunk stream to OpenAI-style SSE frames.-- Accepts any async iterator of objects with ``model_dump()`` (OpenAI- SDK ``ChatCompletionChunk``) or ``to_dict()``; falls back to- ``dict(chunk)`` for dict-likes. This is deliberately duck-typed —- the same helper serves the backend's raw ``ResponseStream`` and- any future transformed stream that still yields chunk-like objects.-- Emits ``data: [DONE]\\n\\n`` after successful completion, matching- the OpenAI streaming contract.-- Args:- stream: Async iterator of ``ChatCompletionChunk`` (or compatible).-- Yields:- SSE-framed strings suitable for ``StreamingResponse``.- """- try:- async for chunk in stream:- if hasattr(chunk, "model_dump"):- chunk_dict = chunk.model_dump(exclude_none=True)- elif hasattr(chunk, "to_dict"):- chunk_dict = chunk.to_dict()- else:- chunk_dict = (- dict(chunk) if hasattr(chunk, "__iter__") else {"data": str(chunk)}- )- yield f"data: {json.dumps(chunk_dict)}\n\n"-- yield "data: [DONE]\n\n"-- except Exception as e:- log.error("Error during chat completions streaming: %s: %s", type(e).__name__, e)- error_data = {- "error": {- "message": repr(e)[:200],- "type": "internal_error",- "code": "internal_chain_error",- }- }- yield f"data: {json.dumps(error_data)}\n\n"- finally:- await _aclose_stream(stream)---async def iter_anthropic_sse(- events: AsyncIterator[RawMessageStreamEvent | Mapping[str, object]],-) -> AsyncGenerator[str, None]:- """Frame Anthropic events into ``event: <type>\\ndata: <json>\\n\\n``.-- Anthropic's SSE contract carries a named event per frame (unlike- OpenAI Chat Completions, which only uses ``data:`` lines). The- event name comes from each event's ``"type"`` field —- ``message_start``, ``content_block_delta``, ``message_stop``, etc.-- Accepts two producer shapes:-- * Plain ``dict`` events (from ``stream_openai_to_anthropic`` when- the chain translates OpenAI → Anthropic on the fly).- * Pydantic ``RawMessageStreamEvent`` models (from the Anthropic- SDK's ``AsyncStream`` when the backend speaks Anthropic natively- — see :class:`AnthropicNativeBackend`). Serialized via- ``model_dump(exclude_none=True)``.-- No ``[DONE]`` terminator: Anthropic signals end-of-stream with a- ``message_stop`` event, not a sentinel frame.-- Mid-stream iteration failures emit a final ``event: error`` frame- with an ``{"error": {...}}`` payload and terminate — same error- quarantine pattern as :func:`iter_chat_completion_sse`.-- Args:- events: Async iterator of Anthropic events — dicts or pydantic- models with ``model_dump``.-- Yields:- SSE-framed strings suitable for ``StreamingResponse``.- """- try:- async for event in events:- if isinstance(event, Mapping):- event_dict = dict(event)- else:- event_dict = event.model_dump(exclude_none=True)- event_type = event_dict.get("type", "message")- yield f"event: {event_type}\ndata: {json.dumps(event_dict)}\n\n"- except Exception as e:- log.error("Error during anthropic streaming: %s: %s", type(e).__name__, e)- error_data = {- "type": "error",- "error": {- "message": repr(e)[:200],- "type": "internal_error",- },- }- yield f"event: error\ndata: {json.dumps(error_data)}\n\n"- finally:- await _aclose_stream(events)---async def iter_preframed_sse(- frames: AsyncIterator[object],-) -> AsyncGenerator[str, None]:- """Forward Responses SSE strings or frame native Responses events.-- The Responses API translator (:func:`stream_chat_to_responses_sse`)- already yields fully-formatted SSE frames (``"event: ...\\ndata: ...\\n\\n"``)- so the endpoint just needs a thin wrapper that preserves the same- mid-stream error contract as the other helpers. Native Responses upstreams- yield JSON events instead, so this helper frames those mappings using each- event's ``type`` field.-- Args:- frames: Async iterator of pre-formatted SSE strings or Responses events.-- Yields:- SSE frames; on exception, a final ``error`` frame.- """- try:- async for frame in frames:- if isinstance(frame, str):- yield frame- continue- if isinstance(frame, Mapping):- event_dict = dict(frame)- elif hasattr(frame, "model_dump"):- event_dict = frame.model_dump(exclude_none=True)- elif hasattr(frame, "to_dict"):- event_dict = frame.to_dict()- else:- event_dict = {"type": "message", "data": str(frame)}- event_type = event_dict.get("type", "message")- yield f"event: {event_type}\ndata: {json.dumps(event_dict)}\n\n"- except Exception as e:- log.error("Error during responses streaming: %s: %s", type(e).__name__, e)- error_data = {- "type": "error",- "error": {- "message": repr(e)[:200],- "type": "internal_error",- "code": "internal_chain_error",- },- }- yield f"event: error\ndata: {json.dumps(error_data)}\n\n"- finally:- await _aclose_stream(frames)diff --git a/switchyard/lib/endpoints/stats_endpoint.py b/switchyard/lib/endpoints/stats_endpoint.pydeleted file mode 100644index 3ffdc2674..000000000--- a/switchyard/lib/endpoints/stats_endpoint.py+++ /dev/null@@ -1,100 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""FastAPI endpoint module exposing stats over HTTP.--Serves three paths off the same shared stats source:--- ``GET /v1/stats`` — native JSON snapshot.-- ``GET /v1/routing/stats`` — alias of ``/v1/stats`` for backwards compat.-- ``GET /metrics`` — Prometheus text-format exposition rendered from the- same snapshot via :func:`switchyard.lib.prometheus_exposition.render_prometheus`.-"""--from __future__ import annotations--from typing import TYPE_CHECKING, Any--from fastapi import APIRouter-from fastapi.responses import Response--from switchyard.lib.endpoints import outcome_metrics-from switchyard.lib.endpoints.base import Endpoint as NemoSwitchyardEndpoint-from switchyard.lib.endpoints.prometheus_emitter import render as render_extra_metrics-from switchyard.lib.prometheus_exposition import render_prometheus-from switchyard.lib.stats_accumulator import StatsAccumulator--if TYPE_CHECKING:- from fastapi import FastAPI---# Prometheus text exposition format 0.0.4 content-type.-PROMETHEUS_CONTENT_TYPE = "text/plain; version=0.0.4; charset=utf-8"---class StatsEndpoint(NemoSwitchyardEndpoint):- """Exposes stats via ``GET /v1/stats``, ``/v1/routing/stats`` alias,- and ``GET /metrics`` (Prometheus exposition).-- Contributed automatically by :class:`StatsResponseProcessor.get_endpoint`- — no manual wiring required.-- The ``/v1/routing/stats`` alias exists so existing consumers of the- historical endpoint path (``benchmark/run_terminal_bench_harbor.sh``, external- dashboards) work against passthrough without any config change.-- ``/metrics`` renders the same underlying snapshot for Prometheus scrapers;- JSON behavior on ``/v1/stats`` is untouched.- """-- register_once = True-- def __init__(self, stats: StatsAccumulator) -> None:- self._stats = stats-- def register(self, app: FastAPI) -> None:- routes = APIRouter()- stats = self._stats-- async def get_stats() -> dict[str, Any]:- """Snapshot of per-model request / token / latency / cost stats."""- return await stats.snapshot()-- async def reset_stats() -> dict[str, str]:- """Zero all stats counters."""- await stats.reset()- return {"status": "reset"}-- async def get_metrics() -> Response:- """Prometheus text-format exposition of the shared stats snapshot.-- Components that own non-request-derived state contribute extra- lines via :mod:`switchyard.lib.endpoints.prometheus_emitter` so a- single ``/metrics`` scrape carries both surfaces.- """- snapshot = await stats.snapshot()- outcome_block = "\n".join(outcome_metrics.render_lines()) + "\n"- return Response(- content=(- render_prometheus(snapshot)- + outcome_block- + render_extra_metrics()- ),- media_type=PROMETHEUS_CONTENT_TYPE,- )-- # native path.- routes.get("/v1/stats")(get_stats)- routes.post("/v1/stats/reset")(reset_stats)- # Compatibility alias.- routes.get("/v1/routing/stats")(get_stats)- routes.post("/v1/routing/stats/reset")(reset_stats)-- app.include_router(routes, tags=["Stats"])-- # Prometheus exposition lives at the conventional ``/metrics`` path,- # untagged from the JSON Stats routes so scraper discovery / OpenAPI- # consumers see them separately.- metrics_router = APIRouter()- metrics_router.get("/metrics")(get_metrics)- app.include_router(metrics_router, tags=["Metrics"])diff --git a/switchyard/lib/endpoints/upstream_error.py b/switchyard/lib/endpoints/upstream_error.pydeleted file mode 100644index 66de361d2..000000000--- a/switchyard/lib/endpoints/upstream_error.py+++ /dev/null@@ -1,233 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Endpoint-side helper: convert chain exceptions into upstream-status responses.--Python LLM backends stash the upstream HTTP status / body into-``ctx.metadata`` before raising the upstream provider's exception. Rust-backends attach typed-``status_code`` and ``body`` attributes to ``SwitchyardUpstreamError`` so-endpoints can preserve provider failures without parsing exception text.-"""--from __future__ import annotations--import json-import logging-from typing import TYPE_CHECKING, Literal--from fastapi.responses import JSONResponse--from switchyard.lib.endpoints import outcome_metrics-from switchyard.lib.endpoints.error_envelope import (- error_response,- upstream_error_response,-)-from switchyard.lib.endpoints.route_selection import route_selection_headers-from switchyard.lib.proxy_context import (- CTX_ERROR_SOURCE,- CTX_UPSTREAM_ATTEMPTS_RECORDED,- CTX_UPSTREAM_HTTP_BODY,- CTX_UPSTREAM_HTTP_STATUS,- CTX_UPSTREAM_MODEL,- ERROR_SOURCE_PROVIDER,- ERROR_SOURCE_SWITCHYARD,-)-from switchyard_rust.core import SwitchyardUpstreamError--if TYPE_CHECKING:- from switchyard.lib.proxy_context import ProxyContext--Inbound = Literal["anthropic", "openai", "openai-responses"]--_log = logging.getLogger(__name__)---def record_upstream_attempt_success(ctx: ProxyContext) -> None:- """Record one successful (HTTP 200) ``switchyard_upstream_attempts_total``.-- The endpoint-layer fallback that wires the upstream-attempt counter for- every chain whose backend does not record per-attempt itself: the Rust- native / passthrough / multi backends issue exactly one upstream attempt- per call and have no Python retry loop, so one served client request maps- to one successful attempt observable here.-- No-op when a backend already counted its own attempts- (:data:`CTX_UPSTREAM_ATTEMPTS_RECORDED` set) — its retry fan-out must not- be double-counted here.- """- if ctx.metadata.get(CTX_UPSTREAM_ATTEMPTS_RECORDED):- return- outcome_metrics.record_upstream_attempt(200)---def record_upstream_attempt_failure(ctx: ProxyContext, exc: BaseException) -> None:- """Record one failed ``switchyard_upstream_attempts_total`` when attributable.-- Counts an attempt only for failures attributable to the upstream call — a- Python backend that stashed :data:`CTX_UPSTREAM_HTTP_STATUS`, or a Rust- backend's :class:`SwitchyardUpstreamError` (HTTP status carried verbatim;- a status-less upstream error is a network / pre-status failure recorded as- ``None`` → ``retryable_error``). Internal chain failures (translation,- processor, validation) are not upstream attempts and are skipped.-- No-op when a backend already counted its own attempts- (:data:`CTX_UPSTREAM_ATTEMPTS_RECORDED` set).- """- if ctx.metadata.get(CTX_UPSTREAM_ATTEMPTS_RECORDED):- return- status = ctx.metadata.get(CTX_UPSTREAM_HTTP_STATUS)- if isinstance(status, int):- outcome_metrics.record_upstream_attempt(status)- return- if isinstance(exc, SwitchyardUpstreamError):- rust_status = getattr(exc, "status_code", None)- outcome_metrics.record_upstream_attempt(- rust_status if isinstance(rust_status, int) else None- )---def upstream_response_from_ctx(- ctx: ProxyContext,- *,- inbound: Inbound = "openai",- exc: BaseException | None = None,-) -> JSONResponse | None:- """Return a structured upstream-status response when one can be recovered.-- Python backends store status/body in ``ctx.metadata``. Rust backends attach- typed upstream status/body attributes to ``SwitchyardUpstreamError``.- ``inbound`` is accepted for endpoint compatibility; HTTP errors use one- Switchyard envelope across all LLM routes so clients see stable fields.- Returns ``None`` when neither source carries an upstream status and the- endpoint should re-raise.- """- status = ctx.metadata.get(CTX_UPSTREAM_HTTP_STATUS)- if isinstance(status, int):- return upstream_error_response(- status,- ctx.metadata.get(CTX_UPSTREAM_HTTP_BODY),- # A stashed status without an explicit source is an upstream- # passthrough; backends that reuse this channel for their own- # rejections (caller_required 401, translation 400) mark the- # stash ``switchyard`` so the header stays truthful.- error_source=_ctx_error_source(ctx, default=ERROR_SOURCE_PROVIDER),- upstream_model=_ctx_upstream_model(ctx),- )- return upstream_response_from_error(exc, inbound=inbound)---def _ctx_error_source(ctx: ProxyContext, *, default: str) -> str:- """Failure origin stamped by the backend, or ``default`` for the path."""- source = ctx.metadata.get(CTX_ERROR_SOURCE)- return source if isinstance(source, str) and source else default---def _ctx_upstream_model(ctx: ProxyContext) -> str | None:- """Upstream model recorded at failure time, when a selection happened."""- model = ctx.metadata.get(CTX_UPSTREAM_MODEL)- return model if isinstance(model, str) and model else None---def upstream_response_from_error(- exc: BaseException | None,- *,- inbound: Inbound = "openai",-) -> JSONResponse | None:- """Return a normalized response for typed Rust upstream HTTP failures."""- if not isinstance(exc, SwitchyardUpstreamError):- return None- status = getattr(exc, "status_code", None)- raw_body = getattr(exc, "body", None)- if not isinstance(status, int) or not isinstance(raw_body, str):- return None- body = _parse_body(raw_body)- return upstream_error_response(status, body)---def _parse_body(raw: str) -> object:- text = raw.strip()- try:- return json.loads(text)- except json.JSONDecodeError:- return text---def internal_chain_error_response(- exc: BaseException,- inbound: Inbound,- *,- error_source: str = ERROR_SOURCE_SWITCHYARD,- upstream_model: str | None = None,-) -> JSONResponse:- """Translate an unexpected chain failure into the client error envelope.-- Used when an exception escapes dispatch or response-processing that is not- a known upstream HTTP error (no status stashed in ctx) and not a- ``SwitchyardUpstreamError``. LLM clients expect a JSON error object rather- than FastAPI's plain-text 500; callers must log the traceback before calling- this helper so the full context is preserved server-side. ``inbound`` is- retained in the signature because endpoint callers already pass it, but the- HTTP envelope is intentionally shared across inbound formats.-- ``error_source`` defaults to ``switchyard`` (an unexpected internal- failure) but a backend that failed on a status-less upstream fault (e.g.- a network error after retries) marks ``ctx`` so this 500 is labeled- ``provider`` instead.- """- message = repr(exc)[:200]- return error_response(- 500,- message,- error_type="internal_error",- code="internal_chain_error",- error_source=error_source,- upstream_model=upstream_model,- )---def handle_chain_exception(- exc: BaseException,- ctx: ProxyContext,- *,- inbound: Inbound,- log_msg: str,-) -> JSONResponse:- """Handle an unexpected chain exception: check for upstream status, log, and return envelope."""- record_upstream_attempt_failure(ctx, exc)- upstream = upstream_response_from_ctx(ctx, inbound=inbound, exc=exc)- if upstream is not None:- response = upstream- else:- _log.error(log_msg, exc_info=exc)- response = internal_chain_error_response(- exc,- inbound=inbound,- error_source=_ctx_error_source(ctx, default=ERROR_SOURCE_SWITCHYARD),- upstream_model=_ctx_upstream_model(ctx),- )- # A selection on ctx means an upstream call already SUCCEEDED (and was- # billed, with the spend-logs header stamped) before this failure — e.g.- # response-side translation rejected the 200 payload. Surface the- # selection headers on the error response too, so the front proxy can- # still join its (failed) parent spend-log row to the billed provider row.- for header_name, value in route_selection_headers(ctx).items():- response.headers[header_name] = value- return response---def context_exhausted_response(exc: BaseException, inbound: Inbound) -> JSONResponse:- """Translate :class:`SwitchyardContextPoolExhaustedError` into a 400.-- Raised by the chain executor when every routing target has been evicted- after consecutive context-window overflows; FastAPI endpoints catch it- and call this helper to produce the shared Switchyard HTTP error envelope.- """- # Chain-executor rejection, not an upstream failure — ``error_response``- # stamps the ``switchyard`` source header by default.- return error_response(- 400,- str(exc),- error_type="invalid_request_error",- code="context_length_exceeded",- )diff --git a/switchyard/lib/endpoints/upstream_error_log.py b/switchyard/lib/endpoints/upstream_error_log.pydeleted file mode 100644index 70977506f..000000000--- a/switchyard/lib/endpoints/upstream_error_log.py+++ /dev/null@@ -1,90 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Structured per-attempt upstream-failure log for Loki ingestion.--The aggregate ``switchyard_upstream_attempts_total`` counter on ``/metrics``-answers "how many of each error code" but, by Prometheus' data model, holds-no per-event timestamps. This module is its per-event complement: it emits-one JSON line per *failed* upstream attempt on a dedicated logger, carrying-the exact event timestamp so a Loki/Grafana pipeline can audit, replay, or-plot individual failures.--The line *is* a JSON object (not a human sentence with structured ``extra``)-on purpose: Switchyard configures plain-text logging via ``logging.basicConfig``-with no JSON formatter, so embedding the document in the message is what makes-``| json`` work in a Loki query with zero deployment-side formatter config.-The dedicated ``switchyard.upstream_errors`` logger still propagates to the-root handler, so the line also shows on the console.--The ``code`` and ``outcome`` fields are computed with the same helpers as the-metric labels (:func:`~switchyard.lib.endpoints.outcome_metrics.code_label`,-:func:`~switchyard.lib.endpoints.outcome_metrics.classify`) so the event log-joins cleanly to ``switchyard_upstream_attempts_total``.-"""--from __future__ import annotations--import json-import logging-from datetime import UTC, datetime--from switchyard.lib.endpoints.outcome_metrics import classify, code_label--#: Dedicated logger so operators can route or level upstream-failure events-#: independently of the (noisier) backend logger. Propagates to root.-log = logging.getLogger("switchyard.upstream_errors")--#: Structured-log event name — the value a Loki query filters on-#: (``| json | event="upstream_attempt_failed"``).-EVENT_NAME = "upstream_attempt_failed"--#: Upstream error bodies can be large; cap the logged message so a single-#: pathological error cannot blow up a log line / Loki entry.-_MAX_ERROR_CHARS = 500---def log_upstream_attempt_failure(- *,- model: str,- attempt: int,- status_code: int | None,- error: BaseException,- upstream_model: str | None = None,-) -> None:- """Emit one structured JSON record for a single failed upstream attempt.-- ``status_code`` is the raw upstream HTTP status, or ``None`` for a- non-HTTP failure (network error, pre-status timeout) — recorded as- ``status_code: null`` with ``code="none"``. ``attempt`` is 1-based.- ``upstream_model`` is the model actually sent upstream (``body["model"]``)- when the caller knows it; ``model`` remains the internal route/endpoint id.-- ``code`` and ``outcome`` mirror the labels on- ``switchyard_upstream_attempts_total`` so the event log is joinable to- the aggregate counter. ``error_source`` is always ``provider`` — this- event exists only for failures of actual upstream attempts. The record- is logged at WARNING.- """- record = {- "event": EVENT_NAME,- "timestamp": datetime.now(UTC).isoformat(),- "model": model,- "upstream_model": upstream_model,- "attempt": attempt,- "status_code": status_code,- "code": code_label(status_code),- # None (non-HTTP failure) is a retryable_error, matching how- # record_upstream_attempt buckets it.- "outcome": "retryable_error" if status_code is None else classify(status_code),- # Attempt failures are upstream-side by definition; the constant field- # keeps the event joinable to the response-header/span vocabulary.- "error_source": "provider",- "error_type": type(error).__name__,- "error": str(error)[:_MAX_ERROR_CHARS],- }- # Compact separators keep the line small; the message is valid JSON.- log.warning(json.dumps(record, separators=(",", ":")))---__all__ = ["EVENT_NAME", "log_upstream_attempt_failure"]diff --git a/switchyard/lib/llm_client.py b/switchyard/lib/llm_client.pydeleted file mode 100644index c01a4e72f..000000000--- a/switchyard/lib/llm_client.py+++ /dev/null@@ -1,190 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Minimal LLM client wrapper for backends.--This provides a thin wrapper around the OpenAI SDK to support-OpenAI-compatible backends in the chain.-"""--from __future__ import annotations--from typing import Any--from openai import AsyncOpenAI--from switchyard.telemetry import get_telemetry_headers---class OpenAILLMClient:- """Client that wraps the official OpenAI Python SDK.-- Works with any OpenAI-compatible API (OpenAI, NVIDIA NIM, Azure,- vLLM, etc.) by accepting a custom ``base_url``.-- Used by :class:`~switchyard.lib.backends.openai_llm_backend.OpenAiNativeBackend`- and other OpenAI-compatible backends.- """-- async_client: AsyncOpenAI-- def __init__(- self,- *,- api_key: str | None = None,- base_url: str | None = None,- timeout: float | None = None,- max_retries: int | None = None,- ) -> None:- """Initialize the async OpenAI client.-- Args:- api_key: API key for authentication. When omitted or empty,- an inert placeholder is used so the SDK can construct;- callers must then supply a real key per-request via- ``acompletion(api_key=...)`` (BYO-key mode). The- placeholder never reaches a real upstream — without a- caller key the SDK call fails with the upstream's 401.- base_url: Custom base URL for OpenAI-compatible APIs (e.g.,- Azure, vLLM, NVIDIA NIM). Defaults to OpenAI's standard URL.- timeout: Request timeout in seconds. None means no timeout.- max_retries: Override the OpenAI SDK's default 2-retry budget.- ``None`` (default) keeps the SDK default. Set ``0`` for the- classifier path so a slow-upstream ``ReadTimeout`` falls- through to our own ``fail_open`` fallback at the configured- timeout rather than compounding via SDK exponential backoff.- """- client_kwargs: dict[str, Any] = {}- if api_key:- client_kwargs["api_key"] = api_key- else:- # BYO-key deployments construct the client with no- # server-side credential and supply the caller's key per- # request via ``acompletion(api_key=...)``. The SDK refuses- # to construct without *some* key, so we inject an inert- # placeholder that is overridden on every real call. If a- # caller forgets to send a key, the upstream sees this- # placeholder and returns 401 — no real secret can leak.- client_kwargs["api_key"] = "switchyard-byo-key-required"- if base_url:- client_kwargs["base_url"] = base_url- if timeout is not None:- client_kwargs["timeout"] = timeout- if max_retries is not None:- client_kwargs["max_retries"] = max_retries- client_kwargs["default_headers"] = get_telemetry_headers()-- # Only the async client is ever used (backends call ``acompletion``).- # A sync ``OpenAI`` client would allocate a second, never-used httpx- # connection pool (1000 connections by default) per instance, so it is- # intentionally not constructed.- self.async_client = AsyncOpenAI(**client_kwargs)-- def _client_for_api_key(self, api_key: str | None) -> AsyncOpenAI:- if api_key and api_key.strip():- return self.async_client.with_options(api_key=api_key)- return self.async_client-- async def acompletion(- self,- *,- api_key: str | None = None,- **kwargs: Any,- ) -> Any:- """Async wrapper for chat completions.-- When a non-blank ``api_key`` is supplied, the call uses that credential- via the SDK's ``with_options`` override instead of the client's- construction-time key. Used by backends that forward a per-request- caller credential (BYO-key multi-tenant deployments).-- When ``api_key`` is ``None`` or blank (no caller key, or a- whitespace-only header), the override is skipped so the call falls- back to the construction-time key — the per-endpoint ``api_key`` an- operator configured. A blank value must not override a real configured- key with nothing, which would unauthenticate the upstream call (a 401- even though a valid key was configured).- """- return await self._client_for_api_key(api_key).chat.completions.create(**kwargs)-- async def aresponses(- self,- *,- api_key: str | None = None,- **kwargs: Any,- ) -> Any:- """Async wrapper for the OpenAI Responses API.-- When a non-blank ``api_key`` is supplied, the call uses that- per-request credential via the SDK's ``with_options`` override. When- ``api_key`` is ``None`` or blank, no override is applied and the- construction-time key configured on the client is used. SDK validation- and upstream errors are intentionally propagated unchanged.-- Non-streaming calls return the upstream's **exact JSON body** (a- ``dict``) rather than the SDK's typed ``Response`` model: round-tripping- through the typed model re-normalizes the payload and its- ``exclude_none`` serialization drops explicit-null fields, eroding- schema fidelity for Responses passthrough.-- Streaming calls return a :class:`RawSSEFrameStream` yielding the- upstream's SSE frames as **verbatim strings** (modulo CRLF → LF line- normalization) for the same reason — the SDK's typed event stream- drops provider extras and explicit nulls per event. The HTTP request- is sent (and error statuses raise) *before* this method returns, so- the caller's retry/failover contract is unchanged.- """- client = self._client_for_api_key(api_key)- if kwargs.get("stream"):- cm = client.responses.with_streaming_response.create(**kwargs)- # Enter eagerly: the request goes out and non-2xx statuses raise- # ``APIStatusError`` here, not at first iteration — after first- # iteration the endpoint has already committed an HTTP 200.- response = await cm.__aenter__()- return RawSSEFrameStream(cm, response.http_response)- raw = await client.responses.with_raw_response.create(**kwargs)- return raw.http_response.json()---class RawSSEFrameStream:- """Async iterator over an SSE response's frames as verbatim strings.-- Each item is one complete frame (all lines up to and including the blank- separator, e.g. ``"event: x\\ndata: {...}\\n\\n"``), byte-equivalent to the- upstream modulo CRLF → LF normalization. Comment/keep-alive frames pass- through unchanged. ``aclose`` releases the underlying HTTP response and is- safe to call at any point, including before the first ``__anext__``.- """-- def __init__(self, cm: Any, http_response: Any) -> None:- self._cm = cm- self._lines = http_response.aiter_lines()- self._closed = False-- def __aiter__(self) -> RawSSEFrameStream:- return self-- async def __anext__(self) -> str:- buffer: list[str] = []- try:- async for line in self._lines:- if line == "":- if buffer:- return "\n".join(buffer) + "\n\n"- continue- buffer.append(line)- except BaseException:- await self.aclose()- raise- await self.aclose()- if buffer:- # Upstream closed without a trailing blank line; emit the tail as- # a well-formed frame rather than dropping it.- return "\n".join(buffer) + "\n\n"- raise StopAsyncIteration-- async def aclose(self) -> None:- if self._closed:- return- self._closed = True- await self._cm.__aexit__(None, None, None)diff --git a/switchyard/lib/model_listing.py b/switchyard/lib/model_listing.pydeleted file mode 100644index 83a4b8070..000000000--- a/switchyard/lib/model_listing.py+++ /dev/null@@ -1,131 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""OpenAI-compatible model listing payload helpers."""--from collections.abc import Mapping, Sequence-from typing import Any--SUPPORTED_INBOUND_FORMATS: tuple[str, ...] = (- "openai-chat-completions",- "openai-responses",- "anthropic-messages",-)--DEFAULT_CONTEXT_WINDOW = 128_000--_CONTEXT_WINDOW_BY_MODEL_FRAGMENT: tuple[tuple[str, int], ...] = (- ("nemotron-3-super", 1_000_000),- ("nemotron-3-nano", 262_000),- ("deepseek-v4", 1_000_000),- ("claude", 200_000),- ("kimi-k2", 256_000),-)---def _default_capabilities() -> dict[str, Any]:- return {- "streaming": True,- "tool_calling": True,- "context_window": DEFAULT_CONTEXT_WINDOW,- "supported_inbound_formats": list(SUPPORTED_INBOUND_FORMATS),- }---def model_capabilities(- model_id: str,- *,- context_window: int | None = None,- tool_calling: bool = True,-) -> dict[str, Any]:- """Infer capability metadata for a Switchyard-advertised model id."""- return {- "tool_calling": tool_calling,- "context_window": (- context_window if context_window is not None else inferred_context_window(model_id)- ),- }---def combined_model_capabilities(model_ids: Sequence[str]) -> dict[str, Any]:- """Return conservative capabilities for a route spanning multiple models."""- windows = [inferred_context_window(model_id) for model_id in model_ids]- return {- "tool_calling": True,- "context_window": min(windows) if windows else DEFAULT_CONTEXT_WINDOW,- }---def inferred_context_window(model_id: str) -> int:- """Infer a context window from known model-family fragments."""- normalized = model_id.lower()- for fragment, context_window in _CONTEXT_WINDOW_BY_MODEL_FRAGMENT:- if fragment in normalized:- return context_window- return DEFAULT_CONTEXT_WINDOW---def model_entry(- model_id: str,- display_name: str | None = None,- metadata: Mapping[str, Any] | None = None,-) -> dict[str, Any]:- """Build one OpenAI-compatible model object with Switchyard metadata."""- extra = dict(metadata or {})- display_name = display_name or str(extra.pop("display_name", model_id))- extra.pop("id", None)- extra.pop("object", None)- extra.pop("type", None)-- capabilities = _default_capabilities()- raw_capabilities = extra.pop("capabilities", None)- if isinstance(raw_capabilities, Mapping):- capabilities.update(raw_capabilities)-- entry: dict[str, Any] = {- "id": model_id,- "object": "model",- "type": "model",- "created": extra.pop("created", 0),- "owned_by": extra.pop("owned_by", "switchyard"),- "display_name": display_name,- "capabilities": capabilities,- }- entry.update(extra)- return entry---def model_list_payload(- entries: Sequence[Mapping[str, Any]],- default_model: str | None = None,- warnings: Sequence[str] = (),-) -> dict[str, Any]:- """Build the ``GET /v1/models`` response envelope."""- data = [dict(entry) for entry in entries]- model_ids = [str(entry["id"]) for entry in data if "id" in entry]- advertised_default = model_ids[0] if model_ids else None- if default_model in model_ids:- advertised_default = default_model- payload: dict[str, Any] = {- "object": "list",- "data": data,- "first_id": model_ids[0] if model_ids else None,- "last_id": model_ids[-1] if model_ids else None,- "has_more": False,- "default_model": advertised_default,- "model_pool": model_ids,- }- if warnings:- payload["warnings"] = list(warnings)- return payload---__all__ = [- "DEFAULT_CONTEXT_WINDOW",- "SUPPORTED_INBOUND_FORMATS",- "combined_model_capabilities",- "inferred_context_window",- "model_capabilities",- "model_entry",- "model_list_payload",-]diff --git a/switchyard/lib/processors/__init__.py b/switchyard/lib/processors/__init__.pydeleted file mode 100644index c640aa02a..000000000--- a/switchyard/lib/processors/__init__.py+++ /dev/null@@ -1,20 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Concrete request-side and response-side processor implementations."""--from switchyard.lib.processors.model_rewrite_request_processor import (- ModelRewriteRequestProcessor,-)-from switchyard.lib.processors.stats_request_processor import (- StatsRequestProcessor,-)-from switchyard.lib.processors.stats_response_processor_accumulator import (- StatsResponseProcessor,-)--__all__ = [- "ModelRewriteRequestProcessor",- "StatsRequestProcessor",- "StatsResponseProcessor",-]diff --git a/switchyard/lib/processors/format_translate.py b/switchyard/lib/processors/format_translate.pydeleted file mode 100644index c4c2d5b23..000000000--- a/switchyard/lib/processors/format_translate.py+++ /dev/null@@ -1,311 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Router-agnostic format-translation processors for the IGW path.--When Switchyard owns the LLM call (standalone server), inbound→target-format translation happens *inside* the LLM backend (e.g.-:class:`OpenAiNativeBackend.call` runs-:meth:`TranslationEngine.request_to_any_of` at the top), and-outbound translation happens in :class:`TranslationEngine`.-The host's chain has dedicated slots for both.--When the host (NMP IGW) owns the LLM call, neither slot is available-— IGW supplies its own backend and there's no-:class:`TranslationEngine` slot. Format translation has to live in-processors instead. The three processors here are the IGW equivalent-of the standalone path, and reusable by *any* router (random-routing today, others later) that needs cross-format dispatch:--* :class:`StampOriginalFormatProcessor` — runs at the head of the- chain and captures the inbound :class:`ChatRequestType` into- ``ctx.metadata[CTX_ORIGINAL_FORMAT]`` so the response processor can- translate back later.-* :class:`FormatTranslateRequestProcessor` — runs after a router- picks a tier and reads ``ctx.metadata[CTX_TARGET_FORMAT]``.- No-ops when the target is unset or matches- ``request.request_type``; otherwise delegates to- :meth:`TranslationEngine.request_to_any_of`.-* :class:`FormatTranslateResponseProcessor` — runs after the host- backend returns. Reads ``ctx.metadata[CTX_ORIGINAL_FORMAT]`` and- converts the response back to the client's inbound format via- :class:`TranslationEngine`.--Composability: any router that wants cross-format support stamps a-:class:`ChatRequestType` into ``CTX_TARGET_FORMAT``; these processors-handle the actual conversion. Router and converter are fully-decoupled — neither imports the other.-"""--from __future__ import annotations--import logging-from collections.abc import Mapping-from copy import deepcopy-from typing import TYPE_CHECKING, cast--from pydantic import BaseModel, ConfigDict--from switchyard.lib.backends.llm_target import BackendFormat-from switchyard.lib.proxy_context import (- CTX_ORIGINAL_FORMAT,- CTX_ORIGINAL_REQUEST,- CTX_PROXY_ACTUAL_MODEL,- CTX_TARGET_FORMAT,-)-from switchyard_rust.core import (- ChatRequest,- ChatRequestType,- ChatResponse,- request_type_matches,- request_type_value,- request_with_type,- response_is_streaming,- response_matches_request_type,-)-from switchyard_rust.translation import TranslationEngine--if TYPE_CHECKING:- from anthropic.types.message_create_params import MessageCreateParamsBase- from openai.types.chat.completion_create_params import CompletionCreateParamsBase- from openai.types.responses.response_create_params import ResponseCreateParamsBase-- from switchyard.lib.proxy_context import ProxyContext--log = logging.getLogger(__name__)---class TranslateConfig(BaseModel):- """Model-to-format map used by IGW-owned translation processors."""-- model_config = ConfigDict(frozen=True)-- models: list[dict[str, str]] = []---class ModelFormatLookupProcessor:- """Look up the selected model's target format from the translate config.-- After a router picks a tier and stamps its model name into- ``ctx.selected_model``, this processor looks up the model in the- translate config's models list and stamps the corresponding backend- format into ``ctx.metadata[CTX_TARGET_FORMAT]``. The legacy- ``ctx.metadata[CTX_PROXY_ACTUAL_MODEL]`` key remains a fallback for- Python-only routers that have not migrated yet.-- ``BackendFormat.AUTO`` is resolved locally because the IGW path has- no Switchyard-owned backend slot where backend construction can probe- endpoint capabilities. The resolver keeps Anthropic-native inbound- traffic native, and otherwise normalizes to OpenAI Chat as the broad- fallback wire format.-- No-op when the model is not found in the config (passthrough).- """-- def __init__(self, config: TranslateConfig) -> None:- self._config = config- # Build a lookup dict: model name → backend_format- self._model_to_format: dict[str, BackendFormat] = {}- for entry in config.models:- model_name = entry.get("model")- format_str = entry.get("backend_format")- if model_name and format_str:- try:- self._model_to_format[model_name] = BackendFormat(format_str)- except ValueError:- log.warning(- "Unknown backend_format in translate config: %s", format_str,- )-- async def process(self, ctx: ProxyContext, request: ChatRequest) -> ChatRequest:- selected_model = ctx.selected_model or ctx.metadata.get(CTX_PROXY_ACTUAL_MODEL)- if not isinstance(selected_model, str) or not selected_model:- return request- target_format = self._model_to_format.get(selected_model)- if target_format is not None:- target_request_type = _target_request_type_for_backend_format(- target_format, request,- )- ctx.metadata[CTX_TARGET_FORMAT] = target_request_type- log.info(- "ModelFormatLookupProcessor: model=%s format=%s "- "target_request_type=%s",- selected_model,- target_format.value,- target_request_type.value,- )- return request---class StampOriginalFormatProcessor:- """Capture the inbound request type into ``ctx.metadata``.-- Place at the head of any IGW chain that may rewrite the request- format before the host backend sees it. The matching response- processor reads ``CTX_ORIGINAL_FORMAT`` to know what format to- translate the response back to.- """-- async def process(self, ctx: ProxyContext, request: ChatRequest) -> ChatRequest:- ctx.metadata[CTX_ORIGINAL_FORMAT] = request.request_type- if CTX_ORIGINAL_REQUEST not in ctx.metadata:- body = getattr(request, "body", None)- if isinstance(body, Mapping):- ctx.metadata[CTX_ORIGINAL_REQUEST] = deepcopy(dict(body))- return request---class FormatTranslateRequestProcessor:- """Translate the request to the target wire format chosen by a router.-- Reads ``ctx.metadata[CTX_TARGET_FORMAT]`` (a- :class:`ChatRequestType`) and routes through- :meth:`TranslationEngine.request_to_any_of` when the target- differs from ``request.request_type``. Pure passthrough when:-- * No router stamped a target (``CTX_TARGET_FORMAT`` absent).- * Target matches the inbound type (already in the right format).-- Router-agnostic — no notion of *why* a target was chosen, only- what to do with it. Reusable by every routing factory that needs- cross-format dispatch.- """-- async def process(self, ctx: ProxyContext, request: ChatRequest) -> ChatRequest:- target = ctx.metadata.get(CTX_TARGET_FORMAT)- if target is None:- return request- if not _is_request_type(target):- raise TypeError(- f"ctx.metadata[{CTX_TARGET_FORMAT!r}] must be a ChatRequestType, "- f"got {type(target).__name__}",- )- if request_type_matches(request, target):- return request- return TranslationEngine().request_to_any_of(request, [target])---class FormatTranslateResponseProcessor:- """Translate the response back to the inbound format the client expects.-- Reads ``ctx.metadata[CTX_ORIGINAL_FORMAT]`` (stamped by- :class:`StampOriginalFormatProcessor` at the head of the chain)- and converts the response via- :class:`TranslationEngine`. No-ops when the response- is already in the original format or when ``CTX_ORIGINAL_FORMAT``- is absent.-- Streaming responses take the response model from the routing context so- the translated stream names the model that actually served the call,- while still translating lazily.- """-- async def process(self, ctx: ProxyContext, response: ChatResponse) -> ChatResponse:- original = ctx.metadata.get(CTX_ORIGINAL_FORMAT)- if original is None:- return response- if not _is_request_type(original):- raise TypeError(- f"ctx.metadata[{CTX_ORIGINAL_FORMAT!r}] must be a ChatRequestType, "- f"got {type(original).__name__}",- )- if response_is_streaming(response):- return _translate_streaming_response(ctx, response, original)- if _matches_format(response, original):- return response- if request_type_value(original) == request_type_value(ChatRequestType.OPENAI_CHAT):- return TranslationEngine().response_to(ChatRequestType.OPENAI_CHAT, response)- if request_type_value(original) == request_type_value(ChatRequestType.ANTHROPIC):- return TranslationEngine().response_to(ChatRequestType.ANTHROPIC, response)- if request_type_value(original) == request_type_value(ChatRequestType.OPENAI_RESPONSES):- return TranslationEngine().response_to(- ChatRequestType.OPENAI_RESPONSES,- response,- served_model=_served_model(ctx),- )- raise NotImplementedError(- f"Unsupported original format for response translation: {original!r}",- )---def _translate_streaming_response(- ctx: ProxyContext,- response: ChatResponse,- original: ChatRequestType | str,-) -> ChatResponse:- if _matches_streaming_format(response, original):- return response- return TranslationEngine().response_to(- original,- response,- served_model=_served_model(ctx),- )---def _request_for_original_format(- original: ChatRequestType | str,- body: dict[str, object],-) -> ChatRequest:- if request_type_value(original) == request_type_value(ChatRequestType.OPENAI_CHAT):- return request_with_type(original, cast("CompletionCreateParamsBase", body))- if request_type_value(original) == request_type_value(ChatRequestType.ANTHROPIC):- return request_with_type(original, cast("MessageCreateParamsBase", body))- if request_type_value(original) == request_type_value(ChatRequestType.OPENAI_RESPONSES):- return request_with_type(original, cast("ResponseCreateParamsBase", body))- raise NotImplementedError(f"Unsupported original format: {original!r}")---def _served_model(ctx: ProxyContext) -> str | None:- """Return the model the backend actually called, or ``None`` when unknown."""- model = ctx.selected_model or ctx.metadata.get(CTX_PROXY_ACTUAL_MODEL)- return model if isinstance(model, str) and model else None---def _matches_format(response: ChatResponse, target: ChatRequestType | str) -> bool:- """Return ``True`` when *response* is already in *target*'s wire format."""- return bool(response_matches_request_type(response, target)) and not bool(- response_is_streaming(response)- )---def _matches_streaming_format(response: ChatResponse, target: ChatRequestType | str) -> bool:- return bool(response_matches_request_type(response, target)) and bool(- response_is_streaming(response)- )---def _is_request_type(value: object) -> bool:- try:- request_type_value(cast("ChatRequestType | str", value))- except (AttributeError, TypeError, ValueError):- return False- return True---def _target_request_type_for_backend_format(- backend_format: BackendFormat,- request: ChatRequest,-) -> ChatRequestType:- """Resolve a configured backend wire format to a request format.-- ``BackendFormat.AUTO`` on the processor-only IGW path cannot do the- backend-owned capability probe used by backend factories. It instead- chooses the least surprising concrete request shape:-- * Anthropic inbound remains Anthropic, preserving native Claude- fields instead of translating them away.- * OpenAI Chat inbound remains OpenAI Chat.- * Responses-capable targets preserve OpenAI Responses instead of- normalizing through Chat.- """- if backend_format == BackendFormat.OPENAI:- return ChatRequestType.OPENAI_CHAT- if backend_format == BackendFormat.RESPONSES:- return ChatRequestType.OPENAI_RESPONSES- if backend_format == BackendFormat.ANTHROPIC:- return ChatRequestType.ANTHROPIC- if backend_format == BackendFormat.AUTO:- if request_type_matches(request, ChatRequestType.ANTHROPIC):- return ChatRequestType.ANTHROPIC- return ChatRequestType.OPENAI_CHAT- raise ValueError(f"Unsupported backend_format: {backend_format!r}")diff --git a/switchyard/lib/processors/model_rewrite_request_processor.py b/switchyard/lib/processors/model_rewrite_request_processor.pydeleted file mode 100644index f218fc3cb..000000000--- a/switchyard/lib/processors/model_rewrite_request_processor.py+++ /dev/null@@ -1,26 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Request processor that pins every inbound request to one backend model."""--from __future__ import annotations--from switchyard.lib.proxy_context import ProxyContext-from switchyard_rust.core import ChatRequest---class ModelRewriteRequestProcessor:- """Force-rewrite ``request.body["model"]`` to a fixed value.-- All request subclasses expose a top-level ``model`` key in their provider- body. Launchers use this as a safety net so the child process can display a- model while Switchyard remains authoritative about the upstream route.- """-- def __init__(self, model: str) -> None:- self._model = model-- async def process(self, ctx: ProxyContext, request: ChatRequest) -> ChatRequest:- del ctx- request.set_model(self._model)- return requestdiff --git a/switchyard/lib/processors/rl_logging_request_processor.py b/switchyard/lib/processors/rl_logging_request_processor.pydeleted file mode 100644index c02af3739..000000000--- a/switchyard/lib/processors/rl_logging_request_processor.py+++ /dev/null@@ -1,43 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Request-side processor that snapshots the inbound request for RL trace logging."""--from __future__ import annotations--from copy import deepcopy--from switchyard.lib.proxy_context import ProxyContext-from switchyard_rust.core import ChatRequest, ChatRequestType-from switchyard_rust.translation import TranslationEngine--#: Context metadata key holding the OpenAI-Chat-shaped snapshot of the inbound-#: request body. Written by :class:`RlLoggingRequestProcessor`; read by-#: :class:`~switchyard.lib.processors.rl_logging_response_processor.RlLoggingResponseProcessor`-#: to reconstruct the logged conversation.-CTX_RL_LOGGING_REQUEST = "_rl_logging_request"---class RlLoggingRequestProcessor:- """Snapshot the inbound request as an OpenAI-Chat body for RL trace logging.-- The response-side logger only receives ``(ctx, response)``, so the request- has to be captured on the request side. We store the *translated* body (a- plain dict) rather than the request wrapper so the snapshot is both- format-normalized (every inbound format becomes OpenAI Chat) and immune to- any in-place mutation later processors might perform.- """-- def __init__(self) -> None:- self._translation = TranslationEngine()-- async def process(self, ctx: ProxyContext, request: ChatRequest) -> ChatRequest:- """Snapshot the OpenAI-Chat-translated request body onto ``ctx``.-- Writes a deep copy under :data:`CTX_RL_LOGGING_REQUEST` (so the snapshot- stays stable even if later processors mutate the request) and returns- the request unchanged.- """- openai_request = self._translation.request_to(ChatRequestType.OPENAI_CHAT, request)- ctx.metadata[CTX_RL_LOGGING_REQUEST] = deepcopy(dict(openai_request.body))- return requestdiff --git a/switchyard/lib/processors/rl_logging_response_processor.py b/switchyard/lib/processors/rl_logging_response_processor.pydeleted file mode 100644index 529a8c1c2..000000000--- a/switchyard/lib/processors/rl_logging_response_processor.py+++ /dev/null@@ -1,187 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Response-side processor that writes per-turn RL training traces to local JSON files."""--from __future__ import annotations--import json-import logging-import uuid as uuid_lib-from datetime import UTC, datetime-from pathlib import Path-from typing import Any--from switchyard.lib.chat_response.streaming_response_accumulator import (- attach_final_response_callback,-)-from switchyard.lib.processors.rl_logging_request_processor import (- CTX_RL_LOGGING_REQUEST,- RlLoggingRequestProcessor,-)-from switchyard.lib.proxy_context import CTX_PROXY_ACTUAL_MODEL, ProxyContext-from switchyard_rust.core import (- ChatRequestType,- ChatResponse,- ChatResponseType,- response_type_matches,-)-from switchyard_rust.translation import TranslationEngine--logger = logging.getLogger(__name__)--JsonObject = dict[str, Any]---class RlLoggingResponseProcessor:- """Write one ``message_history`` JSON trace per completed turn to ``log_dir``.-- Restores the pre-1.0 ``--enable-rl-logging`` local trace format: each- request/response pair is written to its own file as- ``{uuid, messages, tools, tool_choice, token_count, is_valid}``.-- Streaming responses are captured via- :func:`attach_final_response_callback`, which accumulates the native stream- and fires once it drains; non-streaming responses are logged inline. The- response is always returned unchanged — this processor only observes.- """-- def __init__(self, log_dir: Path | str) -> None:- self._log_dir = Path(log_dir)- self._log_dir.mkdir(parents=True, exist_ok=True)- self._translation = TranslationEngine()-- async def process(self, ctx: ProxyContext, response: ChatResponse) -> ChatResponse:- """Log one ``message_history`` trace for the completed turn; return the response.-- Streaming responses are captured when the stream drains; non-streaming- responses log inline. Write failures are swallowed (logged, never- raised) so trace logging can never break the proxied response.- """- served_model: str = ctx.selected_model or ctx.metadata.get(- CTX_PROXY_ACTUAL_MODEL, "unknown",- )-- async def _emit(final: ChatResponse) -> None:- self._write_trace(ctx, final)-- # Streaming responses log on stream completion; everything else is- # already complete and logs inline.- attached = attach_final_response_callback(- response, served_model=served_model, callback=_emit,- )- if not attached:- await _emit(response)- return response-- def _write_trace(self, ctx: ProxyContext, response: ChatResponse) -> None:- request = ctx.metadata.get(CTX_RL_LOGGING_REQUEST)- if not isinstance(request, dict):- return- entry = self._build_entry(request, response)- if entry is None:- return- try:- self._write_entry(entry)- except OSError as exc:- logger.warning("RL logging: failed to write trace to %s: %s", self._log_dir, exc)-- def _build_entry(self, request: JsonObject, response: ChatResponse) -> JsonObject | None:- translated = self._translation.response_to(ChatRequestType.OPENAI_CHAT, response)- if not response_type_matches(translated, ChatResponseType.OPENAI_COMPLETION):- return None- body = dict(translated.body)- choices = body.get("choices")- if not isinstance(choices, list) or not choices or not isinstance(choices[0], dict):- return None- message = choices[0].get("message")- if not isinstance(message, dict):- return None-- messages = [dict(m) for m in request.get("messages", []) if isinstance(m, dict)]- assistant: JsonObject = {"role": "assistant"}- content = message.get("content")- if content is not None:- assistant["content"] = content- tool_calls = message.get("tool_calls")- if tool_calls:- assistant["tool_calls"] = tool_calls- messages.append(assistant)-- usage = body.get("usage")- usage = usage if isinstance(usage, dict) else {}- return {- "uuid": str(uuid_lib.uuid4()),- "messages": messages,- "tools": _format_tools(request.get("tools", [])),- "tool_choice": _format_tool_choice(request.get("tool_choice")),- "token_count": {- "prompt_tokens": usage.get("prompt_tokens", 0),- "completion_tokens": usage.get("completion_tokens", 0),- "total_tokens": usage.get("total_tokens", 0),- },- "is_valid": True,- }-- def _write_entry(self, entry: JsonObject) -> None:- path = self._log_dir / _trace_filename()- with open(path, "w") as handle:- json.dump(entry, handle, indent=2)---def build_rl_logging_processors(- rl_log_dir: Path | None,-) -> tuple[list[Any], list[Any]]:- """Request/response processor lists for local RL trace logging.-- Returns ``([], [])`` when ``rl_log_dir`` is ``None`` (logging disabled), or- the paired snapshot + writer processors otherwise. Shared by the ``launch``- and ``serve`` wiring.- """- if rl_log_dir is None:- return [], []- return [RlLoggingRequestProcessor()], [RlLoggingResponseProcessor(rl_log_dir)]---def _trace_filename() -> str:- """File-safe ``{timestamp}_trace_{trace_id}_{suffix}.json`` (one file per turn)."""- timestamp = datetime.now(UTC).strftime("%Y-%m-%dT%H-%M-%S-%f")[:-3] + "Z"- trace_id = uuid_lib.uuid4().hex[:8]- suffix_id = uuid_lib.uuid4().hex[:8]- return f"{timestamp}_trace_{trace_id}_{suffix_id}.json"---def _format_tools(raw_tools: object) -> list[JsonObject]:- """Port the V1 message_history tool shape: ``{id, description, inputSchema}``."""- if not isinstance(raw_tools, list):- return []- tools: list[JsonObject] = []- for tool in raw_tools:- if not isinstance(tool, dict):- continue- entry: JsonObject = {}- function = tool.get("function")- if isinstance(function, dict):- entry["id"] = function.get("name", "")- entry["description"] = function.get("description", "")- if "parameters" in function:- entry["inputSchema"] = {"jsonSchema": function["parameters"]}- else:- entry["id"] = tool.get("name", tool.get("id", ""))- entry["description"] = tool.get("description", "")- if "input_schema" in tool:- entry["inputSchema"] = {"jsonSchema": tool["input_schema"]}- elif "parameters" in tool:- entry["inputSchema"] = {"jsonSchema": tool["parameters"]}- tools.append(entry)- return tools---def _format_tool_choice(tool_choice: object) -> str:- if isinstance(tool_choice, str):- return tool_choice- if isinstance(tool_choice, dict):- choice_type = tool_choice.get("type")- if isinstance(choice_type, str):- return choice_type- return "auto"diff --git a/switchyard/lib/processors/routing_log_response_processor.py b/switchyard/lib/processors/routing_log_response_processor.pydeleted file mode 100644index cc87c0183..000000000--- a/switchyard/lib/processors/routing_log_response_processor.py+++ /dev/null@@ -1,205 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Response-side processor that appends one JSONL routing record per request."""--from __future__ import annotations--import asyncio-import json-import logging-import threading-from collections.abc import Mapping-from datetime import UTC, datetime-from pathlib import Path-from typing import TYPE_CHECKING, Any--from switchyard.lib.chat_response.streaming_response_accumulator import (- attach_final_response_callback,-)-from switchyard.lib.proxy_context import CTX_PROXY_ACTUAL_MODEL, ProxyContext-from switchyard.lib.request_metadata import CTX_REQUEST_HEADERS, CTX_REQUEST_METADATA-from switchyard_rust.core import ChatResponse--if TYPE_CHECKING:- from switchyard.lib.endpoints.base import Endpoint--logger = logging.getLogger(__name__)---class RoutingLogResponseProcessor:- """Append one JSON line per completed request to ``log_file``.-- Each record carries the routing decision (selected model and tier), the- caller-supplied task and session identity headers, and token usage, so a- benchmark harness can attribute router traffic to individual tasks.- Streaming responses log once the stream drains; write failures are logged- and never break the proxied response.- """-- def __init__(self, log_file: Path | str) -> None:- self._log_file = Path(log_file)- self._log_file.parent.mkdir(parents=True, exist_ok=True)- self._lock = threading.Lock()-- async def process(self, ctx: ProxyContext, response: ChatResponse) -> ChatResponse:- """Log one routing record for the completed request; return the response unchanged."""- served_model: str = (- ctx.metadata.get(CTX_PROXY_ACTUAL_MODEL) or ctx.selected_model or "unknown"- )-- async def _emit(final: ChatResponse) -> None:- await asyncio.to_thread(self._write_record, ctx, served_model, final)-- attached = attach_final_response_callback(- response, served_model=served_model, callback=_emit,- )- if not attached:- await _emit(response)- return response-- def _write_record(self, ctx: ProxyContext, served_model: str, response: ChatResponse) -> None:- metadata = ctx.metadata.get(CTX_REQUEST_METADATA)- headers = ctx.metadata.get(CTX_REQUEST_HEADERS) or {}- # Prefer the model id the backend actually served so buckets reconcile- # with the global routing_stats schema, which keys on the served id.- actual_model = _field(response.body, "model")- record = {- "ts": datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S.%f")[:-3] + "Z",- "task": getattr(metadata, "task", None),- "trial_id": headers.get("x-switchyard-trial-id"),- "session_id": getattr(metadata, "session_id", None),- "model": actual_model if isinstance(actual_model, str) and actual_model else served_model,- "tier": ctx.selected_target or "",- **_usage_tokens(response.body),- }- try:- line = json.dumps(record, separators=(",", ":"))- with self._lock, self._log_file.open("a", encoding="utf-8") as handle:- handle.write(line + "\n")- except OSError as exc:- logger.warning("Routing log: failed to append to %s: %s", self._log_file, exc)-- def snapshot_session(self, session_id: str) -> dict[str, object] | None:- """Aggregate the durable request log for one exact trial session.-- This re-reads and re-parses the whole log on every call. It is sized for- benchmark runs (thousands of records), not a long-lived production server- with millions of sessions, where it would be O(records) per query.- """- models: dict[str, dict[str, int]] = {}- totals = {- "total_calls": 0,- "total_prompt_tokens": 0,- "total_cached_tokens": 0,- "total_cache_creation_tokens": 0,- "total_completion_tokens": 0,- }- try:- if not self._log_file.is_file():- return None- lines = self._log_file.read_text(encoding="utf-8").splitlines()- except OSError as exc:- logger.warning("Routing log: failed to read %s: %s", self._log_file, exc)- return None-- for line in lines:- try:- record = json.loads(line)- except json.JSONDecodeError:- continue- if not isinstance(record, Mapping) or record.get("session_id") != session_id:- continue- model_value = record.get("model")- model = model_value if isinstance(model_value, str) and model_value else "unknown"- bucket = models.setdefault(- model,- {- "calls": 0,- "prompt_tokens": 0,- "cached_tokens": 0,- "cache_creation_tokens": 0,- "completion_tokens": 0,- },- )- bucket["calls"] += 1- totals["total_calls"] += 1- for record_key, bucket_key, total_key in (- ("prompt_tokens", "prompt_tokens", "total_prompt_tokens"),- ("cached_tokens", "cached_tokens", "total_cached_tokens"),- (- "cache_creation_tokens",- "cache_creation_tokens",- "total_cache_creation_tokens",- ),- ("completion_tokens", "completion_tokens", "total_completion_tokens"),- ):- value = record.get(record_key)- if isinstance(value, int) and not isinstance(value, bool) and value >= 0:- bucket[bucket_key] += value- totals[total_key] += value-- if not totals["total_calls"]:- return None- return {"session_id": session_id, **totals, "models": models}-- def get_endpoint(self) -> Endpoint:- """Contribute the session-scoped routing-stat snapshot endpoint."""- from switchyard.lib.endpoints.routing_log_stats_endpoint import (- RoutingLogStatsEndpoint,- )-- return RoutingLogStatsEndpoint(self)---def _usage_tokens(body: object) -> dict[str, int]:- """Six-field token breakdown matching the global routing-stats schema.-- cached/cache_creation are subsets of prompt_tokens, reasoning a subset of- completion_tokens, total = prompt + completion.- """- usage = _field(body, "usage")- prompt = _int_field(usage, "prompt_tokens")- completion = _int_field(usage, "completion_tokens")- cached = 0- cache_creation = 0- reasoning = 0- if prompt or completion:- # OpenAI Chat Completions.- prompt_details = _field(usage, "prompt_tokens_details")- cached = _int_field(prompt_details, "cached_tokens")- cache_creation = _int_field(prompt_details, "cache_creation_tokens")- reasoning = _int_field(_field(usage, "completion_tokens_details"), "reasoning_tokens")- else:- completion = _int_field(usage, "output_tokens")- reasoning = _int_field(_field(usage, "output_tokens_details"), "reasoning_tokens")- input_details = _field(usage, "input_tokens_details")- if input_details is not None:- # OpenAI Responses API: prompt_tokens already includes cached.- prompt = _int_field(usage, "input_tokens")- cached = _int_field(input_details, "cached_tokens")- cache_creation = _int_field(input_details, "cache_creation_tokens")- else:- # Anthropic Messages: cache tokens are prompt siblings, so fold in.- cached = _int_field(usage, "cache_read_input_tokens")- cache_creation = _int_field(usage, "cache_creation_input_tokens")- prompt = _int_field(usage, "input_tokens") + cached + cache_creation- return {- "prompt_tokens": prompt,- "cached_tokens": cached,- "cache_creation_tokens": cache_creation,- "completion_tokens": completion,- "reasoning_tokens": reasoning,- "total_tokens": prompt + completion,- }---def _field(value: object, name: str) -> Any:- if isinstance(value, Mapping):- return value.get(name)- return getattr(value, name, None)---def _int_field(value: object, name: str) -> int:- field = _field(value, name)- return field if isinstance(field, int) else 0diff --git a/switchyard/lib/processors/stats_request_processor.py b/switchyard/lib/processors/stats_request_processor.pydeleted file mode 100644index cf6c3c8a9..000000000--- a/switchyard/lib/processors/stats_request_processor.py+++ /dev/null@@ -1,8 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Rust-owned stats request processor export."""--from switchyard_rust.components import StatsRequestProcessor--__all__ = ["StatsRequestProcessor"]diff --git a/switchyard/lib/processors/stats_response_processor_accumulator.py b/switchyard/lib/processors/stats_response_processor_accumulator.pydeleted file mode 100644index 48526a927..000000000--- a/switchyard/lib/processors/stats_response_processor_accumulator.py+++ /dev/null@@ -1,8 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Rust-owned stats response processor export."""--from switchyard_rust.components import StatsResponseProcessor--__all__ = ["StatsResponseProcessor"]diff --git a/switchyard/lib/prometheus_exposition.py b/switchyard/lib/prometheus_exposition.pydeleted file mode 100644index 146ad0a26..000000000--- a/switchyard/lib/prometheus_exposition.py+++ /dev/null@@ -1,216 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Render a :class:`StatsAccumulator` snapshot as Prometheus exposition text.--Hand-rolled so the metrics surface ships without adding the ``prometheus-client``-dependency. The exposition follows Prometheus text format 0.0.4: ``# HELP`` /-``# TYPE`` headers, one sample per line, label values escaped per the spec.--Metric design (all names prefixed ``switchyard_``):--- ``requests_total{model,tier}`` — counter; selected model / tier traffic.-- ``errors_total{model,tier}`` — counter; backend-call errors.-- ``prompt_tokens_total`` / ``completion_tokens_total`` /- ``cached_tokens_total`` / ``cache_creation_tokens_total`` /- ``reasoning_tokens_total`` — counters per model/tier.-- ``model_call_latency_ms`` — summary; backend-only call latency,- per model/tier, with ``quantile="0.5"`` and ``quantile="0.99"``.-- ``total_latency_ms`` — summary; end-to-end request latency, per model/tier.-- ``routing_overhead_ms`` — summary; global router decision overhead- (``total_latency - backend_latency``).-- ``total_requests`` / ``total_errors`` — gauges; running totals across all- models, mirroring the ``/v1/stats`` JSON top-level fields.--Label cardinality is intentionally bounded: model names and tier ids are-both small fixed sets in any real deployment; nothing per-request (no-request ids, no prompts) is ever emitted.-"""--from __future__ import annotations--from importlib.metadata import version as _pkg_version-from typing import Any--try:- _SWITCHYARD_VERSION = _pkg_version("nemo-switchyard")-except Exception:- _SWITCHYARD_VERSION = "unknown"---def render_prometheus(snapshot: dict[str, Any]) -> str:- """Render a stats snapshot as Prometheus text-format exposition.-- Args:- snapshot: The dict returned by :meth:`StatsAccumulator.snapshot_sync`.-- Returns:- A string containing the full exposition payload (UTF-8, LF-terminated- lines, trailing newline).- """- lines: list[str] = []-- _emit_gauge(- lines,- "switchyard_build_info",- "Switchyard build information.",- [({"version": _SWITCHYARD_VERSION}, 1)],- )-- total_requests = snapshot.get("total_requests", 0)- total_errors = snapshot.get("total_errors", 0)- _emit_gauge(- lines,- "switchyard_total_requests",- "Total chain-level requests recorded.",- [({}, total_requests)],- )- _emit_gauge(- lines,- "switchyard_total_errors",- "Total chain-level backend errors recorded.",- [({}, total_errors)],- )-- models: dict[str, dict[str, Any]] = snapshot.get("models", {}) or {}-- # Counter families keyed by (description, attribute on per-model dict).- counters: list[tuple[str, str, str]] = [- ("switchyard_requests_total", "Calls per selected model/tier.", "calls"),- ("switchyard_errors_total", "Backend errors per selected model/tier.", "errors"),- ("switchyard_prompt_tokens_total", "Prompt tokens billed per model/tier.", "prompt_tokens"),- (- "switchyard_completion_tokens_total",- "Completion tokens generated per model/tier.",- "completion_tokens",- ),- ("switchyard_cached_tokens_total", "Cached prompt tokens per model/tier.", "cached_tokens"),- (- "switchyard_cache_creation_tokens_total",- "Cache-creation tokens per model/tier.",- "cache_creation_tokens",- ),- (- "switchyard_reasoning_tokens_total",- "Reasoning tokens per model/tier.",- "reasoning_tokens",- ),- ]- for metric, help_text, attr in counters:- samples = [- (_labels_for(model, m), m.get(attr, 0)) for model, m in sorted(models.items())- ]- _emit_counter(lines, metric, help_text, samples)-- # Summary families: backend-call latency and total-latency per model/tier.- summaries: list[tuple[str, str, str]] = [- (- "switchyard_model_call_latency_ms",- "Backend-call latency per model/tier (ms).",- "model_call_latency",- ),- (- "switchyard_total_latency_ms",- "End-to-end request latency per model/tier (ms).",- "total_latency",- ),- ]- for metric, help_text, attr in summaries:- per_model = [- (_labels_for(model, m), m.get(attr) or _empty_histogram())- for model, m in sorted(models.items())- ]- _emit_summary(lines, metric, help_text, per_model)-- # Global routing-overhead summary (router decision time minus backend time).- overhead = snapshot.get("routing_overhead") or _empty_histogram()- _emit_summary(- lines,- "switchyard_routing_overhead_ms",- "Router decision latency overhead across all requests (ms).",- [({}, overhead)],- )-- return "\n".join(lines) + "\n"---def _labels_for(model: str, m: dict[str, Any]) -> dict[str, str]:- labels: dict[str, str] = {"model": model}- tier = m.get("tier")- if tier:- labels["tier"] = str(tier)- return labels---def _empty_histogram() -> dict[str, float | int]:- return {"count": 0, "total_ms": 0.0, "p50_ms": 0.0, "p99_ms": 0.0}---def _emit_gauge(- lines: list[str],- name: str,- help_text: str,- samples: list[tuple[dict[str, str], float | int]],-) -> None:- lines.append(f"# HELP {name} {help_text}")- lines.append(f"# TYPE {name} gauge")- for labels, value in samples:- lines.append(f"{name}{render_labels(labels)} {format_number(value)}")---def _emit_counter(- lines: list[str],- name: str,- help_text: str,- samples: list[tuple[dict[str, str], float | int]],-) -> None:- lines.append(f"# HELP {name} {help_text}")- lines.append(f"# TYPE {name} counter")- for labels, value in samples:- lines.append(f"{name}{render_labels(labels)} {format_number(value)}")---def _emit_summary(- lines: list[str],- name: str,- help_text: str,- samples: list[tuple[dict[str, str], dict[str, float | int]]],-) -> None:- lines.append(f"# HELP {name} {help_text}")- lines.append(f"# TYPE {name} summary")- for labels, hist in samples:- for quantile in ("0.5", "0.99"):- q_labels = dict(labels)- q_labels["quantile"] = quantile- field = "p50_ms" if quantile == "0.5" else "p99_ms"- lines.append(- f"{name}{render_labels(q_labels)} {format_number(hist.get(field, 0.0))}"- )- lines.append(- f"{name}_sum{render_labels(labels)} {format_number(hist.get('total_ms', 0.0))}"- )- lines.append(- f"{name}_count{render_labels(labels)} {format_number(hist.get('count', 0))}"- )---def render_labels(labels: dict[str, str]) -> str:- if not labels:- return ""- parts = [f'{k}="{_escape_label_value(v)}"' for k, v in labels.items()]- return "{" + ",".join(parts) + "}"---def _escape_label_value(value: str) -> str:- # Per Prometheus exposition spec: escape backslash, double-quote, newline.- return value.replace("\\", "\\\\").replace('"', '\\"').replace("\n", "\\n")---def format_number(value: float | int) -> str:- if isinstance(value, int):- return str(value)- if value != value or value in (float("inf"), float("-inf")): # NaN / inf- return "0"- if value.is_integer():- return str(int(value))- return f"{value:g}"diff --git a/switchyard/lib/proxy_context.py b/switchyard/lib/proxy_context.pydeleted file mode 100644index 9b715a7f7..000000000--- a/switchyard/lib/proxy_context.py+++ /dev/null@@ -1,123 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""ProxyContext re-export and shared metadata keys."""--from switchyard_rust.core import ProxyContext--# ----------------------------------------------------------------------------# Metadata key constants-# ----------------------------------------------------------------------------# Use these instead of bare strings to avoid silent typos and to make-# cross-component key contracts discoverable at import time.--#: Stores a deep-copy of the incoming request dict.-#: Written by RequestBufferProcessor.-CTX_ORIGINAL_REQUEST = "original_request"--#: Model that was actually used for the LLM call, after any routing/override.-#: Written by routing backends and processors.-CTX_PROXY_ACTUAL_MODEL = "_proxy_actual_model"--#: Routing metadata dict produced by tier-routing request processors.-CTX_ROUTING = "_routing"--#: Original inbound format stored by translation layers.-CTX_ORIGINAL_FORMAT = "_original_format"--#: Original model name stored by translation layers.-CTX_ORIGINAL_MODEL = "_original_model"--#: Target wire format chosen by a router (e.g. random routing).-#: Written by router request-side components; read by-#: FormatTranslateRequestProcessor.-CTX_TARGET_FORMAT = "_target_format"--#: Caller-supplied API key extracted from the inbound request's-#: ``x-switchyard-api-key`` header (preferred — survives proxies that strip-#: ``Authorization``), or ``Authorization: Bearer <key>`` / ``x-api-key``. Set by-#: the HTTP endpoint after header parsing; consumed by backends that support-#: opt-in per-caller credential forwarding.-#: Absent when the caller did not supply a credential or supplied a known-#: launcher-sentinel placeholder.-CTX_CALLER_API_KEY = "_caller_api_key"--#: Upstream HTTP status code recorded by a Python backend when an LLM-#: provider returns a non-2xx response. Endpoints inspect this on the-#: error path to passthrough the upstream status (e.g. 401 from a bad-#: API key) instead of masking it as a 500. Rust backend errors don't-#: round-trip structurally — they are reported as opaque strings via-#: ``SwitchyardError::Upstream`` — so this is currently the Python-side-#: channel only.-CTX_UPSTREAM_HTTP_STATUS = "_upstream_http_status"--#: Upstream HTTP response body recorded alongside-#: :data:`CTX_UPSTREAM_HTTP_STATUS`. May be a string or a JSON-decodable-#: dict — endpoints pass it through to the caller verbatim.-CTX_UPSTREAM_HTTP_BODY = "_upstream_http_body"--#: Set truthy by a backend that records its own per-attempt-#: ``switchyard.lib.endpoints.outcome_metrics`` upstream-attempt counters-#: (e.g. a backend that retries across endpoints and must count each-#: attempt). When present, the endpoint layer skips its-#: single-attempt fallback recording for this request so retry fan-out is not-#: double-counted. Absent for the Rust native / passthrough / multi backends,-#: which issue exactly one upstream attempt per call and have no Python retry-#: loop — those rely on the endpoint fallback.-CTX_UPSTREAM_ATTEMPTS_RECORDED = "_upstream_attempts_recorded"--#: Which layer originated the failure being surfaced to the client:-#: ``"provider"`` for an upstream LLM failure passed through, ``"switchyard"``-#: for an error this proxy synthesized (credential rejection, translation-#: rejection, routing failure). Written by backends on the error path; read by-#: the endpoint layer to stamp the ``x-switchyard-error-source`` response-#: header. Unset means the endpoint's per-path default applies (``provider``-#: for a stashed upstream status, ``switchyard`` for synthesized envelopes).-CTX_ERROR_SOURCE = "_error_source"--#: :data:`CTX_ERROR_SOURCE` value for errors Switchyard itself originated.-#: Defined here (not in the endpoints layer) so backends can stamp it without-#: importing FastAPI-dependent modules.-ERROR_SOURCE_SWITCHYARD = "switchyard"--#: :data:`CTX_ERROR_SOURCE` value for upstream provider failures passed through.-ERROR_SOURCE_PROVIDER = "provider"--#: Upstream model actually attempted when the surfaced failure happened, when-#: a routing selection took place. Written alongside-#: :data:`CTX_ERROR_SOURCE`; read by the endpoint layer to stamp the-#: ``x-switchyard-upstream-model`` response header.-CTX_UPSTREAM_MODEL = "_upstream_model"--#: Route-selection record for spend/tokenomics attribution, written by a-#: routing backend after a successful upstream call. A dict with the keys-#: ``router_model`` (client-facing route id), ``router_strategy``,-#: ``router_selected_endpoint``, ``router_selected_model``,-#: ``router_selected_provider``, and ``router_correlation_id``. The same-#: payload is stamped on the outbound upstream call as the-#: ``x-litellm-spend-logs-metadata`` header (one per attempt, so provider-#: spend-log rows record the endpoint they actually hit); the endpoint layer-#: reads this key to return the ``x-switchyard-*`` route-selection response-#: headers, letting a front proxy enrich its own spend-log row with the same-#: correlation id.-CTX_ROUTE_SELECTION = "_route_selection"---__all__ = [- "CTX_CALLER_API_KEY",- "CTX_ERROR_SOURCE",- "CTX_ORIGINAL_FORMAT",- "CTX_ORIGINAL_MODEL",- "CTX_ORIGINAL_REQUEST",- "CTX_PROXY_ACTUAL_MODEL",- "CTX_ROUTE_SELECTION",- "CTX_ROUTING",- "CTX_TARGET_FORMAT",- "CTX_UPSTREAM_ATTEMPTS_RECORDED",- "CTX_UPSTREAM_HTTP_BODY",- "CTX_UPSTREAM_HTTP_STATUS",- "CTX_UPSTREAM_MODEL",- "ERROR_SOURCE_PROVIDER",- "ERROR_SOURCE_SWITCHYARD",- "ProxyContext",-]diff --git a/switchyard/lib/request_metadata.py b/switchyard/lib/request_metadata.pydeleted file mode 100644index de4eed7be..000000000--- a/switchyard/lib/request_metadata.py+++ /dev/null@@ -1,142 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""request metadata helpers for HTTP endpoint context."""--from collections.abc import Mapping-from dataclasses import dataclass-from typing import Any--from switchyard.lib.proxy_context import CTX_CALLER_API_KEY--CTX_REQUEST_METADATA = "_request_metadata"-CTX_REQUEST_HEADERS = "_request_headers"--# Existing Switchyard session header. Do not add aliases here unless a-# concrete client requires one; keeping one spelling avoids ambiguity.-PROXY_SESSION_ID_HEADER = "proxy_x_session_id"-INTAKE_TASK_HEADER = "x-switchyard-intake-task"--# Sentinel values our own launchers send as the ``Authorization`` /-# ``OPENAI_API_KEY`` value so coding agents satisfy their "no key set"-# preconditions. Treat as if no key was supplied. The codex launcher-# sets ``OPENAI_API_KEY="switchyard"`` (see codex_cli_launcher.py).-_CALLER_KEY_SENTINELS = frozenset({"switchyard", ""})--# Dedicated forwarded credential header. Preferred over ``Authorization``-# because a proxy in front of Switchyard (e.g. LiteLLM) consumes the-# ``Authorization`` header for its own auth and strips it before the upstream-# call, while a custom header passes through untouched — so a BYO-key caller-# behind such a proxy stays correctly attributed for upstream inference spend.-CALLER_API_KEY_HEADER = "x-switchyard-api-key" # pragma: allowlist secret--# Request headers whose values carry a caller credential. Values are redacted-# in the map retained as ``CTX_REQUEST_HEADERS``. The raw key remains separately-# in ``CTX_CALLER_API_KEY`` for upstream forwarding and is not serialized by-# logging or tracing.-_SENSITIVE_HEADERS = frozenset({"authorization", "x-api-key", CALLER_API_KEY_HEADER})-_REDACTED = "[REDACTED]"---@dataclass(frozen=True, slots=True)-class RequestMetadata:- """Session and task identifiers retained for compatibility routing logs."""-- session_id: str | None = None- task: str | None = None-- @classmethod- def from_headers(cls, headers: Mapping[str, str]) -> "RequestMetadata":- """Extract request metadata from a case-insensitive header mapping."""- normalized = {name.lower(): value for name, value in headers.items()}- return cls(- session_id=_nonempty_header(normalized, PROXY_SESSION_ID_HEADER),- task=_nonempty_header(normalized, INTAKE_TASK_HEADER),- )---def attach_request_metadata(- ctx: Any,- metadata: RequestMetadata,- headers: Mapping[str, str] | None = None,-) -> None:- """Attach request metadata and redacted headers to the Python context."""- ctx.metadata[CTX_REQUEST_METADATA] = metadata- if headers is not None:- # Redact credential headers before retaining the map: the caller key is- # already extracted into ``CTX_CALLER_API_KEY`` for upstream forwarding,- # so nothing downstream needs the raw value, and a retained/logged header- # map must not expose it.- ctx.metadata[CTX_REQUEST_HEADERS] = redact_sensitive_headers(headers)---def _nonempty_header(headers: Mapping[str, str], name: str) -> str | None:- value = headers.get(name)- return value if value else None---def redact_sensitive_headers(headers: Mapping[str, str]) -> dict[str, str]:- """Return a copy of *headers* with credential-bearing values redacted.-- Headers named in :data:`_SENSITIVE_HEADERS` (``Authorization``, ``x-api-key``,- ``x-switchyard-api-key``) have their values replaced with ``"[REDACTED]"`` so a- retained or logged header map can never expose the caller's API key.- Header-name matching is case-insensitive.- """- return {- name: (_REDACTED if name.lower() in _SENSITIVE_HEADERS else value)- for name, value in headers.items()- }---def attach_caller_api_key(ctx: Any, headers: Mapping[str, str]) -> None:- """Attach the caller-supplied API key to *ctx* when the request carries one."""- caller_key = extract_caller_api_key(headers)- if caller_key is not None:- ctx.metadata[CTX_CALLER_API_KEY] = caller_key---def extract_caller_api_key(headers: Mapping[str, str]) -> str | None:- """Pull the caller-supplied API key out of an HTTP request's headers.-- Precedence: the dedicated ``x-switchyard-api-key`` header first, then- ``Authorization: Bearer <key>``, then ``x-api-key``. The dedicated header is- preferred because a proxy in front of Switchyard (e.g. LiteLLM) strips- ``Authorization`` before the upstream call; the custom header survives, so the- caller's key — not a service key — is the credential billed for upstream- inference. Returns ``None`` when no usable header is present, the bearer- scheme is missing, or the value is a known launcher sentinel (so coding-agent- placeholder keys do not get forwarded upstream as real credentials).- """- forwarded = headers.get(CALLER_API_KEY_HEADER) or headers.get("X-Switchyard-Api-Key")- if forwarded:- candidate = forwarded.strip()- if candidate.lower() not in _CALLER_KEY_SENTINELS:- return candidate- auth = headers.get("authorization") or headers.get("Authorization")- if auth:- scheme, _, value = auth.partition(" ")- if scheme.lower() == "bearer" and value:- candidate = value.strip()- if candidate.lower() not in _CALLER_KEY_SENTINELS:- return candidate- api_key = headers.get("x-api-key") or headers.get("X-Api-Key")- if api_key:- candidate = api_key.strip()- if candidate.lower() not in _CALLER_KEY_SENTINELS:- return candidate- return None---__all__ = [- "CALLER_API_KEY_HEADER",- "CTX_REQUEST_METADATA",- "CTX_REQUEST_HEADERS",- "INTAKE_TASK_HEADER",- "PROXY_SESSION_ID_HEADER",- "RequestMetadata",- "attach_caller_api_key",- "attach_request_metadata",- "extract_caller_api_key",- "redact_sensitive_headers",-]diff --git a/switchyard/lib/roles.py b/switchyard/lib/roles.pydeleted file mode 100644index 397b9f2e9..000000000--- a/switchyard/lib/roles.py+++ /dev/null@@ -1,48 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Python backend role class and translated-response aliases.--Request-side and response-side components are plain objects with async-``process(...)`` methods. The backend remains nominal because it owns upstream-transport behavior and request-format support; native implementations register-with the same Python role class.-"""--from __future__ import annotations--from collections.abc import AsyncIterable, Mapping-from typing import TypeAlias--from anthropic.types import Message as AnthropicMessage-from anthropic.types import RawMessageStreamEvent-from openai.types.chat import ChatCompletion, ChatCompletionChunk-from openai.types.responses import Response as OpenAIResponse-from openai.types.responses import ResponseStreamEvent--from switchyard_rust.core import LLMBackend--# The final translated response surfaced by TranslationEngine.translate().-# Union covers all three formats x (non-streaming | streaming) plus the-# dict-returning converters.-TranslatedStream: TypeAlias = (- AsyncIterable[ChatCompletionChunk]- | AsyncIterable[RawMessageStreamEvent]- | AsyncIterable[ResponseStreamEvent]- | AsyncIterable[Mapping[str, object]]- | AsyncIterable[str]-)--TranslatedResponse: TypeAlias = (- ChatCompletion- | OpenAIResponse- | AnthropicMessage- | Mapping[str, object]- | TranslatedStream-)--__all__ = [- "LLMBackend",- "TranslatedResponse",- "TranslatedStream",-]diff --git a/switchyard/lib/route_table.py b/switchyard/lib/route_table.pydeleted file mode 100644index 1ea7b6251..000000000--- a/switchyard/lib/route_table.py+++ /dev/null@@ -1,170 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""RouteTable — route inbound requests to per-model runtimes.--Stores a static mapping of model name → callable runtime. The launcher-populates the table at startup after building all configured runtimes. The-three HTTP endpoints read ``body["model"]``, call-:meth:`lookup_switchyard`, and dispatch to the returned chain before making any-backend calls.-"""--import logging-from collections.abc import Iterator, Mapping-from typing import Any, ClassVar, Protocol, TypeAlias--from switchyard.lib.model_listing import model_entry-from switchyard.lib.proxy_context import ProxyContext-from switchyard.lib.roles import TranslatedResponse-from switchyard_rust.core import ChatRequest--log = logging.getLogger(__name__)---#: Type accepted by app factories and launcher runtimes — either a single-#: :class:`Switchyard` chain or a :class:`RouteTable` that dispatches-#: by inbound model id.-class ChainRuntime(Protocol):- """Runtime accepted by the table and FastAPI dispatcher."""-- async def call(- self,- request: ChatRequest,- *,- ctx: ProxyContext | None = None,- ) -> TranslatedResponse:- """Execute one request through the runtime."""- ...-- def iter_components(self) -> list[Any]:- """Return lifecycle components in startup order."""- ...---SwitchyardApp: TypeAlias = "ChainRuntime | RouteTable"---class RouteTable:- """Table that maps model names to pre-built runtimes.-- Register one chain per model the proxy should handle explicitly. Unknown- models raise ``KeyError`` so endpoint handlers can return ``model_not_found``- instead of silently forwarding a request to the wrong backend.- """-- #: Same key as :class:`Switchyard` so app factories store this under the- #: attribute the V2 endpoint handlers already read.- state_key: ClassVar[str] = "switchyard"-- def __init__(self) -> None:- self._by_model: dict[str, ChainRuntime] = {}- self._metadata_by_model: dict[str, dict[str, Any]] = {}- self._model_listing_warnings: list[str] = []- self._default_model: str | None = None- # Last model id that `lookup_switchyard` successfully resolved.- # Updated on every request ingress (each endpoint calls- # `lookup_switchyard` once per request before any backend work).- # Read by the launcher's live stats footer so the displayed model- # tracks what the user actually picked via /model, with no delay- # for streaming responses.- self._last_looked_up: str | None = None-- def register(- self,- model: str,- switchyard: ChainRuntime,- metadata: Mapping[str, Any] | None = None,- default: bool = False,- ) -> None:- """Register *switchyard* as the exact-match chain for *model*."""- self._by_model[model] = switchyard- self._metadata_by_model[model] = dict(metadata or {})- if default:- self._default_model = model- log.debug("RouteTable: registered chain for model=%r", model)-- def registered_models(self) -> list[str]:- """Return registered model ids in registration order."""- return list(self._by_model)-- def set_default_model(self, model: str) -> None:- """Mark *model* as the default entry advertised by ``/v1/models``."""- if model not in self._by_model:- raise KeyError(model)- self._default_model = model-- def default_model(self) -> str | None:- """Return the advertised default model id, falling back to first entry."""- if self._default_model in self._by_model:- return self._default_model- return next(iter(self._by_model), None)-- def items(self) -> Iterator[tuple[str, ChainRuntime, dict[str, Any]]]:- """Iterate ``(model_id, chain, metadata)`` triples in registration order.-- Used when a caller needs to merge one table into another — e.g. a- caller that composes multiple route tables.- """- for model in self._by_model:- yield model, self._by_model[model], dict(self._metadata_by_model.get(model, {}))-- def lookup_switchyard(self, model: str) -> ChainRuntime:- """Return the chain for *model*.-- Records *model* as the last successfully resolved id (see- :attr:`last_looked_up`).-- Raises:- KeyError: *model* is unregistered.- """- chain = self._by_model.get(model)- if chain is not None:- log.debug("RouteTable: model=%r → registered chain", model)- self._last_looked_up = model- return chain- raise KeyError(model)-- @property- def last_looked_up(self) -> str | None:- """Model id from the most recent successful :meth:`lookup_switchyard`.-- ``None`` until the first request arrives. Set at request ingress, so- the value reflects the model the user is *currently* sending traffic- for — useful for live launcher TUIs that want to display which- table entry the client most recently picked.- """- return self._last_looked_up-- def registered_model_entries(self) -> list[dict[str, Any]]:- """Return OpenAI-compatible model entries with optional metadata."""- entries: list[dict[str, Any]] = []- for model in self._by_model:- metadata = dict(self._metadata_by_model.get(model, {}))- entries.append(model_entry(model, metadata=metadata))- return entries-- def add_model_listing_warning(self, warning: str) -> None:- """Record non-fatal model catalog discovery warnings for ``/v1/models``."""- if warning not in self._model_listing_warnings:- self._model_listing_warnings.append(warning)-- def model_listing_warnings(self) -> list[str]:- """Return non-fatal model catalog warnings in discovery order."""- return list(self._model_listing_warnings)-- def iter_components(self) -> list[Any]:- """Return all chain components across registered chains, deduplicated.-- Components shared by object identity are returned once so endpoint and- shutdown hooks are not double-registered.- """- seen: set[int] = set()- result: list[Any] = []- for switchyard in self._by_model.values():- for component in switchyard.iter_components():- if id(component) in seen:- continue- seen.add(id(component))- result.append(component)- return resultdiff --git a/switchyard/lib/startup_timing.py b/switchyard/lib/startup_timing.pydeleted file mode 100644index bff550e42..000000000--- a/switchyard/lib/startup_timing.py+++ /dev/null@@ -1,56 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Per-stage startup timing for ``switchyard launch``.--Turn it on with ``--startup-timing`` or ``SWITCHYARD_STARTUP_TIMING=1``; it does-nothing when off. ``mark()`` records a checkpoint during startup, and ``dump()``-prints the time between checkpoints to stderr just before the agent starts. It-times only switchyard's own startup work, not Python's one-time import cost.-"""--import os-import sys-import time-from typing import TextIO--# On unless SWITCHYARD_STARTUP_TIMING is unset or "0", or once --startup-timing calls-# enable(). Read once at import — the env var does not change during a launch.-enabled: bool = os.environ.get("SWITCHYARD_STARTUP_TIMING", "0") != "0"--# (label, perf_counter timestamp) for each point reached during startup.-_marks: list[tuple[str, float]] = []---def enable() -> None:- """Turn timing on for this process (called by the ``--startup-timing`` flag)."""- global enabled- enabled = True---def mark(label: str) -> None:- """Record that startup reached *label*. No-op unless timing is enabled."""- if enabled:- _marks.append((label, time.perf_counter()))---def dump(stream: TextIO | None = None) -> None:- """Print the per-stage breakdown to stderr, then reset. No-op when disabled.-- Each line is the time between consecutive marks; the last line is the total- from the first mark to the last.- """- if not enabled or len(_marks) < 2:- _marks.clear()- return- out = stream if stream is not None else sys.stderr- start = _marks[0][1]- prev = start- lines = ["switchyard startup timing:"]- for label, stamp in _marks[1:]:- lines.append(f" {(stamp - prev) * 1000:8.1f} ms {label}")- prev = stamp- lines.append(f" {'-' * 8}")- lines.append(f" {(prev - start) * 1000:8.1f} ms total (launch invoked -> child spawn)")- print("\n".join(lines), file=out)- _marks.clear()diff --git a/switchyard/lib/stats_accumulator.py b/switchyard/lib/stats_accumulator.pydeleted file mode 100644index 0b062681c..000000000--- a/switchyard/lib/stats_accumulator.py+++ /dev/null@@ -1,8 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Rust-owned stats accumulator export."""--from switchyard_rust.components import StatsAccumulator--__all__ = ["StatsAccumulator"]diff --git a/switchyard/lib/switchyard.py b/switchyard/lib/switchyard.pydeleted file mode 100644index e4515ef0e..000000000--- a/switchyard/lib/switchyard.py+++ /dev/null@@ -1,10 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Python compatibility Switchyard chain export."""--from __future__ import annotations--from switchyard_rust.core import Switchyard--__all__ = ["Switchyard"]diff --git a/switchyard/lib/tracing.py b/switchyard/lib/tracing.pydeleted file mode 100644index 5f80a7f15..000000000--- a/switchyard/lib/tracing.py+++ /dev/null@@ -1,72 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Optional Datadog (ddtrace) spans for the proxy routing path.--Switchyard runs inside a proxy that owns the Datadog APM trace. These helpers-add child spans and tags around the route-decision and upstream-attempt blocks-so routing behaviour is visible in APM.--``ddtrace`` is an **optional** dependency (the ``tracing`` extra). When it is-not installed the helpers are complete no-ops with no overhead, so the default-install and non-Datadog deployments are unaffected. When it is installed, the-spans created here nest under whatever span the surrounding proxy already has-active, so they appear inline in the proxy's trace.-"""--from __future__ import annotations--from collections.abc import Iterator, Mapping-from contextlib import contextmanager-from typing import Any, Protocol, runtime_checkable--try: # ddtrace >= 2.0 exposes the global tracer here- from ddtrace.trace import tracer as _dd_tracer-except Exception: # pragma: no cover - import shape varies / absent in dev- try: # ddtrace < 2.0 fallback- from ddtrace import tracer as _dd_tracer- except Exception:- _dd_tracer = None---@runtime_checkable-class Span(Protocol):- """Minimal span surface the routing instrumentation relies on."""-- def set_tag(self, key: str, value: Any) -> None: ...---class _NoopSpan:- """Span stand-in used when no tracer is available; drops every tag."""-- def set_tag(self, key: str, value: Any) -> None:- return None---_NOOP_SPAN = _NoopSpan()---@contextmanager-def routing_span(name: str) -> Iterator[Span]:- """Open a child span named *name*, or yield a no-op span when ddtrace is absent.-- The span nests under whatever span the surrounding proxy has active, so the- routing instrumentation shows up inline in the proxy's Datadog trace and is- finished automatically when the ``with`` block exits.- """- if _dd_tracer is None:- yield _NOOP_SPAN- return- with _dd_tracer.trace(name) as span:- yield span---def set_tags(span: Span, tags: Mapping[str, Any]) -> None:- """Set each non-``None`` tag on *span*.-- ``None`` values are skipped so an unavailable signal (e.g. a signal that- has not been recorded yet) leaves no empty/misleading tag on the span.- """- for key, value in tags.items():- if value is not None:- span.set_tag(key, value)diff --git a/switchyard/server/__init__.py b/switchyard/server/__init__.pydeleted file mode 100644index 1482e4502..000000000--- a/switchyard/server/__init__.py+++ /dev/null@@ -1,34 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""HTTP serving primitives.--Exposes:--* ``server_util`` — CLI helpers.-* ``switchyard_app`` — FastAPI app factory for a ``Switchyard`` chain.-"""--from switchyard.server.server_util import (- DEFAULT_SECRETS_FILE,- REPO_ROOT,- add_common_args,- add_transport_args,- build_and_serve,- ensure_openai_api_key_env,- load_secrets,- resolve_config_with_secrets,- resolve_credentials_from_env,-)--__all__ = [- "DEFAULT_SECRETS_FILE",- "REPO_ROOT",- "add_common_args",- "add_transport_args",- "build_and_serve",- "ensure_openai_api_key_env",- "load_secrets",- "resolve_config_with_secrets",- "resolve_credentials_from_env",-]diff --git a/switchyard/server/server_util.py b/switchyard/server/server_util.pydeleted file mode 100644index b62275cc6..000000000--- a/switchyard/server/server_util.py+++ /dev/null@@ -1,389 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Shared CLI helpers for ``switchyard.cli.switchyard_cli``.--Provides:--- secrets-file loading-- common argparse argument surface-- credential resolution from CLI / env-- :func:`build_and_serve`, which wraps a :class:`~switchyard.lib.switchyard.Switchyard`- in a FastAPI app via :func:`~switchyard.server.switchyard_app.build_switchyard_app`- and starts uvicorn-"""--from __future__ import annotations--import argparse-import json-import logging-import os-from collections.abc import Mapping-from enum import Enum-from pathlib import Path-from typing import TYPE_CHECKING--from switchyard.lib.endpoints.base import Endpoint as NemoSwitchyardEndpoint-from switchyard.lib.switchyard import Switchyard--if TYPE_CHECKING:- from switchyard.lib.route_table import RouteTable---class InboundFormat(Enum):- """Inbound wire format accepted by the proxy."""- OPENAI = "openai"- ANTHROPIC = "anthropic"- BOTH = "both"---logger = logging.getLogger(__name__)--# Four parents up: foundation/server/server_util.py → foundation/server →-# foundation → switchyard → repo root.-REPO_ROOT = Path(__file__).resolve().parent.parent.parent.parent-DEFAULT_SECRETS_FILE = REPO_ROOT / "secrets" / "secrets.json"---# ----------------------------------------------------------------------------# Secrets-# ------------------------------------------------------------------------------# secrets.json is a flat map of section name → section body. Each section-# body is itself a dict (provider creds or server config). Values are-# typed ``object`` so the narrow ``isinstance(section, dict)`` checks below-# stay honest — the alternative ``Any`` would silently absorb type errors.-SecretsFile = Mapping[str, Mapping[str, object]]---def load_secrets(secrets_file: Path | None = None) -> SecretsFile:- """Load secrets from ``secrets.json``, returning ``{}`` if not found."""- path = secrets_file or DEFAULT_SECRETS_FILE- if path.exists():- with open(path) as f:- loaded: SecretsFile = json.load(f)- return loaded- return {}---# ----------------------------------------------------------------------------# Argparse surfaces-# ------------------------------------------------------------------------------def add_transport_args(parser: argparse.ArgumentParser) -> None:- """Register transport-layer arguments: ``--host``, ``--port``, ``--inbound``, ``--reload``.-- Subcommands that define their own credential args with non-standard- names (e.g. ``--api-base`` instead of ``--base-url``) use this- helper to avoid the full :func:`add_common_args` surface.- """- parser.add_argument("--host", type=str, default="0.0.0.0", help="Host to bind to")- parser.add_argument(- "--port", "-p", type=int, default=None,- help="Port to bind to (default: 4000, or server.port from secrets.json)",- )- parser.add_argument(- "--inbound",- type=str,- default=None,- choices=[f.value for f in InboundFormat],- help="Inbound API format: openai, anthropic, or both",- )- parser.add_argument("--reload", action="store_true", help="Enable auto-reload")---def add_common_args(- parser: argparse.ArgumentParser,- *,- include_model: bool = True,-) -> None:- """Register arguments shared by every subcommand.-- Adds the transport args (:func:`add_transport_args`) plus backend- credentials (``--api-key`` / ``--base-url``), the optional model- override (``--model``), and uvicorn worker count (``--workers``).-- Args:- parser: Target parser (subcommand parser, typically).- include_model: Whether to expose ``--model`` for backend model- override. Subcommands that don't yet support a model-override- request processor should pass ``False``.- """- add_transport_args(parser)- parser.add_argument(- "--api-key", type=str, default=None,- help="API key for the backend LLM (falls back to env vars / secrets.json)",- )- parser.add_argument(- "--base-url", type=str, default=None,- help="Base URL for the backend LLM API",- )- if include_model:- parser.add_argument(- "--model", type=str, default=None,- help="Model name override (replaces model from incoming requests)",- )- parser.add_argument(- "--workers", "-w", type=int,- default=int(os.environ.get("SWITCHYARD_WORKERS", "1")),- help="Number of uvicorn worker processes (default: 1, or SWITCHYARD_WORKERS env var)",- )---def resolve_rl_log_dir(args: argparse.Namespace) -> Path | None:- """Resolve the RL trace-log directory from the global rl-logging flags.-- Returns ``None`` unless the global ``--enable-rl-logging`` flag is set, in- which case the directory is ``--rl-log-dir`` (default ``./rl_data``). Shared- by the ``launch`` and ``serve`` entry points.- """- if not getattr(args, "enable_rl_logging", False):- return None- return Path(getattr(args, "rl_log_dir", None) or "./rl_data").expanduser()---# ----------------------------------------------------------------------------# Credential resolution-# ------------------------------------------------------------------------------def resolve_credentials_from_env(- args: argparse.Namespace,- *,- check_anthropic: bool = False,-) -> tuple[str | None, str | None]:- """Return ``(api_key, base_url)`` resolved from ``args`` + env vars.-- Resolution order for *api_key*:- 1. ``args.api_key``- 2. ``ANTHROPIC_API_KEY`` (only when ``check_anthropic=True``)- 3. ``OPENAI_API_KEY``-- Resolution order for *base_url*:- 1. ``args.base_url``- 2. ``OPENAI_BASE_URL``- 3. ``OPENAI_API_BASE``-- Does not read ``secrets.json`` — use :func:`resolve_config_with_secrets`- when secrets-file fallback is also needed.- """- if check_anthropic:- api_key = (- args.api_key- or os.environ.get("ANTHROPIC_API_KEY")- or os.environ.get("OPENAI_API_KEY")- )- else:- api_key = args.api_key or os.environ.get("OPENAI_API_KEY")- base_url = (- args.base_url- or os.environ.get("OPENAI_BASE_URL")- or os.environ.get("OPENAI_API_BASE")- )- return api_key, base_url---def resolve_config_with_secrets(- args: argparse.Namespace,- *,- api_key_env_vars: tuple[str, ...] = ("OPENAI_API_KEY",),- base_url_env_vars: tuple[str, ...] = (),- base_url_arg: str = "api_base",- secrets_section_priority: tuple[str, ...] = (),-) -> tuple[str | None, str | None]:- """Resolve ``(api_key, base_url)`` from CLI → env → ``secrets.json``.-- Also mutates ``args.port`` in place when the CLI didn't set it and- ``secrets.json`` has a ``server.port`` entry.-- Resolution order:- * api_key: ``args.api_key`` → *api_key_env_vars* (in order) →- secrets sections (``secrets_section_priority`` in order, then- first provider as a final fallback)- * base_url: ``getattr(args, base_url_arg)`` → *base_url_env_vars*- (in order) → same secrets-section traversal as api_key- * port: ``args.port`` (if set) else ``secrets["server"]["port"]``-- The secrets-section traversal mirrors the pattern used by the- legacy subcommands: a priority-list of sections (e.g.- ``("nvidia",)``) is checked first, and if none of them yielded an- api_key we fall all the way back to the first provider entry —- whichever section happens to come first in ``secrets.json``.-- Args:- args: Parsed CLI namespace. Must have ``api_key`` and- ``port`` attributes; ``base_url_arg`` must also resolve.- api_key_env_vars: Env var names checked in order for the- api_key fallback.- base_url_env_vars: Env var names checked in order for the- base_url fallback. Empty means "CLI + secrets only".- base_url_arg: Attribute name on ``args`` that holds the CLI- base URL — typically ``"api_base"`` (for ``--api-base``)- or ``"base_url"`` (for ``--base-url``).- secrets_section_priority: Top-level section names in- ``secrets.json`` to consult first (e.g. ``("nvidia",)``).- Empty defaults to "first provider only".-- Returns:- ``(api_key, base_url)`` — either element may be ``None``.- """- secrets = load_secrets()- secrets_api_key: str | None = None- secrets_base_url: str | None = None- secrets_port: int | None = None-- if secrets:- for section_name in secrets_section_priority:- section = secrets.get(section_name, {})- if isinstance(section, dict):- api_key_value = section.get("api_key")- if secrets_api_key is None and isinstance(api_key_value, str):- secrets_api_key = api_key_value- base_url_value = section.get("base_url")- if secrets_base_url is None and isinstance(base_url_value, str):- secrets_base_url = base_url_value-- # Mirror the legacy behavior: fall back to the first provider- # section only when the priority sections didn't yield an api_key.- if not secrets_api_key:- first_provider: Mapping[str, object] = next(iter(secrets.values()), {})- if isinstance(first_provider, dict):- api_key_value = first_provider.get("api_key")- if secrets_api_key is None and isinstance(api_key_value, str):- secrets_api_key = api_key_value- base_url_value = first_provider.get("base_url")- if secrets_base_url is None and isinstance(base_url_value, str):- secrets_base_url = base_url_value-- server_section = secrets.get("server", {})- if isinstance(server_section, dict):- port_value = server_section.get("port")- if isinstance(port_value, (int, str)):- secrets_port = int(port_value)-- # api_key: CLI > env vars (in order) > secrets- api_key: str | None = args.api_key- for env_var in api_key_env_vars:- if api_key:- break- api_key = os.environ.get(env_var)- api_key = api_key or secrets_api_key-- # base_url: CLI > env vars (in order) > secrets- base_url: str | None = getattr(args, base_url_arg, None)- for env_var in base_url_env_vars:- if base_url:- break- base_url = os.environ.get(env_var)- base_url = base_url or secrets_base_url-- if args.port is None and secrets_port is not None:- args.port = secrets_port-- return api_key, base_url---def ensure_openai_api_key_env(api_key: str | None) -> None:- """Set ``OPENAI_API_KEY`` env var iff unset and *api_key* is provided.-- Some downstream libraries read- ``OPENAI_API_KEY`` directly from the environment, so subcommands- that resolve credentials from CLI / secrets.json pin them into the- environment for those libraries to see.- """- if api_key and not os.environ.get("OPENAI_API_KEY"):- os.environ["OPENAI_API_KEY"] = api_key---def resolve_port(default: int = 4000) -> int:- """Resolve the server port from ``secrets.json`` with a fixed fallback."""- server_section = load_secrets().get("server", {})- if isinstance(server_section, Mapping):- value = server_section.get("port")- if isinstance(value, int):- return value- if isinstance(value, str):- try:- return int(value)- except ValueError:- logger.warning("Ignoring non-integer secrets.json server.port=%r", value)- return default---# ----------------------------------------------------------------------------# Build + serve-# ------------------------------------------------------------------------------def build_and_serve(- args: argparse.Namespace,- switchyard: Switchyard | RouteTable,- *,- inbound_default: str = "openai",- disable_backend_streaming: bool = False,- extra_endpoints: list[NemoSwitchyardEndpoint] | None = None,- strategy_summary: str | None = None,-) -> None:- """Wire a Switchyard runtime object into a FastAPI app and serve it.-- Builds the app via :func:`~switchyard.server.switchyard_app.build_switchyard_app`- (which registers all three inbound formats — OpenAI Chat, Anthropic Messages, and- OpenAI Responses API), optionally appends *extra_endpoints*, then starts uvicorn.-- Expected attributes on *args*:- * ``host`` (str), ``port`` (int | None), ``inbound`` (str | None),- ``reload`` (bool), ``workers`` (int, optional; defaults to 1).-- Args:- args: Parsed CLI namespace.- switchyard: Already-built chain or table to serve.- inbound_default: Unused — always registers all inbound formats.- disable_backend_streaming: Unused — kept for signature compatibility.- extra_endpoints: Additional endpoint modules to register after the defaults.- """- del inbound_default, disable_backend_streaming-- import threading-- import uvicorn-- from switchyard.server.switchyard_app import build_switchyard_app-- app = build_switchyard_app(switchyard)- if extra_endpoints:- for endpoint in extra_endpoints:- endpoint.register(app)-- port = args.port if isinstance(args.port, int) else resolve_port()-- def _print_banner() -> None:- from switchyard.cli.launchers.launcher_runtime import (- print_ready_banner,- wait_for_proxy_ready,- )- from switchyard.lib.route_table import RouteTable as _RT- if not wait_for_proxy_ready(port, timeout_s=15.0):- return- table = switchyard if isinstance(switchyard, _RT) else None- default_model = table.default_model() if table else None- print_ready_banner(- port=port,- display_model=default_model or "switchyard",- strategy_summary=strategy_summary,- routes=table.registered_models() if table else None,- default_route=default_model,- )-- threading.Thread(target=_print_banner, daemon=True).start()-- workers = getattr(args, "workers", 1)- uvicorn.run(- app,- host=args.host,- port=port,- reload=args.reload,- workers=workers,- )diff --git a/switchyard/server/switchyard_app.py b/switchyard/server/switchyard_app.pydeleted file mode 100644index 404b77b95..000000000--- a/switchyard/server/switchyard_app.py+++ /dev/null@@ -1,214 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Convenience factory for serving a ``Switchyard`` or model table.--The default setup registers all three inbound endpoints so the app-can serve OpenAI Chat Completions, Anthropic Messages, and OpenAI-Responses API clients simultaneously — the chain handles format-translation internally.-"""--from __future__ import annotations--import inspect-from collections.abc import AsyncIterator, Callable, Iterable-from contextlib import asynccontextmanager-from typing import TYPE_CHECKING, cast--from fastapi import FastAPI, Request-from fastapi.exception_handlers import request_validation_exception_handler-from fastapi.exceptions import RequestValidationError-from fastapi.responses import Response--from switchyard.lib.endpoints import outcome_metrics-from switchyard.lib.endpoints.anthropic_messages_endpoint import (- AnthropicMessagesEndpoint,-)-from switchyard.lib.endpoints.base import Endpoint-from switchyard.lib.endpoints.dispatch import invalid_request_response-from switchyard.lib.endpoints.models_endpoint import ModelsEndpoint-from switchyard.lib.endpoints.openai_chat_endpoint import (- OpenAIChatEndpoint,-)-from switchyard.lib.endpoints.responses_endpoint import (- ResponsesEndpoint,-)-from switchyard_rust.core import SwitchyardInvalidRequestError--#: Inbound LLM-serving paths whose response status codes feed the-#: client-side outcome counter. Other routes (/v1/models, /v1/stats,-#: /metrics, /health) are excluded — they don't represent router-served-#: LLM traffic and would distort the error-rate ratio.-_LLM_ROUTES: frozenset[str] = frozenset({- "/v1/chat/completions",- "/v1/messages",- "/v1/responses",-})--if TYPE_CHECKING:- from switchyard.lib.route_table import SwitchyardApp---async def _run_lifecycle_method(component: object, method_name: str) -> None:- method = getattr(component, method_name, None)- if not callable(method):- return- result = method()- if inspect.isawaitable(result):- await result---async def _shutdown_components(components: Iterable[object]) -> None:- for component in components:- await _run_lifecycle_method(component, "shutdown")---def build_switchyard_app(switchyard: SwitchyardApp) -> FastAPI:- """Create a FastAPI app serving *switchyard* over all inbound formats.-- Registers three LLM endpoints plus a liveness probe:-- - ``POST /v1/chat/completions`` (OpenAI Chat Completions)- - ``POST /v1/messages`` (Anthropic Messages)- - ``POST /v1/responses`` (OpenAI Responses API)- - ``GET /v1/models`` (local model discovery)- - ``GET /health`` (liveness — always 200 when the process is up)-- All three LLM routes go through the same chain — translation- between wire formats is handled by ``TranslationEngine``- inside the backend and ``TranslationEngine`` inside the- translator.-- Example::-- from switchyard.lib.backends.llm_target import BackendFormat, LlmTarget- from switchyard.lib.backends import OpenAiNativeBackend- from switchyard_rust.translation import TranslationEngine- from switchyard.lib.switchyard import Switchyard- from switchyard import build_switchyard_app- import uvicorn-- switchyard = Switchyard(- backend=OpenAiNativeBackend(LlmTarget(model="gpt-4o", format=BackendFormat.OPENAI)),- translator=TranslationEngine(),- )- uvicorn.run(build_switchyard_app(switchyard), port=4000)- """- components = _switchyard_components(switchyard)-- @asynccontextmanager- async def _lifespan(_app: FastAPI) -> AsyncIterator[None]:- started: list[object] = []- startup_complete = False- try:- for component in components:- await _run_lifecycle_method(component, "startup")- started.append(component)- startup_complete = True- yield- finally:- shutdown_targets = components if startup_complete else started- await _shutdown_components(reversed(shutdown_targets))-- app = FastAPI(title="Switchyard", lifespan=_lifespan)-- @app.exception_handler(RequestValidationError)- async def _request_validation_error_handler(- request: Request, exc: RequestValidationError- ) -> Response:- """Map body validation failures on LLM routes to the Switchyard 400 envelope.-- FastAPI raises RequestValidationError for both malformed JSON and wrong- body types (e.g. array instead of object) when a route declares a typed- body parameter. Non-LLM routes and non-body errors fall through to- FastAPI's default 422 handler.- """- if request.url.path in _LLM_ROUTES:- body_errors = [e for e in exc.errors() if e.get("loc", (None,))[0] == "body"]- if body_errors:- is_json_parse = any(e["type"] == "json_invalid" for e in body_errors)- message = (- "Request body is not valid JSON"- if is_json_parse- else "Request body must be a JSON object"- )- return invalid_request_response(message, code="invalid_body")- return await request_validation_exception_handler(request, exc)-- @app.exception_handler(SwitchyardInvalidRequestError)- async def _invalid_request_handler(- _request: Request, exc: SwitchyardInvalidRequestError- ) -> Response:- """Map request validation failures to the 400 envelope.-- ``ChatRequest.validate()`` (called by the inbound endpoints) raises- this when a body is structurally valid but semantically invalid. The- only such check today is a present-but-empty- ``messages`` array, so the envelope uses ``code="empty_messages"``;- revisit if more validations start sharing this error.- """- return invalid_request_response(str(exc), code="empty_messages")-- app.state.switchyard = switchyard-- @app.middleware("http")- async def _record_client_outcome(request, call_next): # type: ignore[no-untyped-def]- """Tally every LLM-route response into the outcome counters.-- Runs after the endpoint produces its response, so it sees the- final status code regardless of how it was generated (success,- upstream-error passthrough, internal exception, model-not-found).- """- response = await call_next(request)- if request.url.path in _LLM_ROUTES:- outcome_metrics.record_client_response(response.status_code)- return response-- # Route tables can contain hundreds of per-model components that contribute- # the same fixed-path endpoint. Registering each copy creates unreachable- # duplicate routes and recursively nests FastAPI lifespan contexts.- registered_once_endpoint_types: set[type[Endpoint]] = set()- for endpoint in [- OpenAIChatEndpoint(),- AnthropicMessagesEndpoint(),- ResponsesEndpoint(),- ModelsEndpoint(),- ]:- endpoint.register(app)- if endpoint.register_once:- registered_once_endpoint_types.add(type(endpoint))-- for component in components:- get_endpoint = getattr(component, "get_endpoint", None)- if not callable(get_endpoint):- continue- contributed = cast(Endpoint | None, get_endpoint())- if contributed is None:- continue- endpoint_type = type(contributed)- if contributed.register_once and endpoint_type in registered_once_endpoint_types:- continue- contributed.register(app)- if contributed.register_once:- registered_once_endpoint_types.add(endpoint_type)-- @app.get("/health", include_in_schema=False)- async def _health() -> dict[str, str]:- return {"status": "ok"}-- return app---def _switchyard_components(- switchyard: SwitchyardApp,-) -> list[object]:- iter_components = getattr(switchyard, "iter_components", None)- if not callable(iter_components):- if callable(getattr(switchyard, "startup", None)) or callable(- getattr(switchyard, "shutdown", None)- ):- return [switchyard]- return []- component_iter = cast(Callable[[], Iterable[object]], iter_components)- return list(component_iter())diff --git a/switchyard/telemetry.py b/switchyard/telemetry.pydeleted file mode 100644index fd33635cc..000000000--- a/switchyard/telemetry.py+++ /dev/null@@ -1,64 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Lightweight telemetry headers for outbound LLM SDK calls.--Switchyard sends its package version as a single HTTP header on upstream LLM-requests. This lets downstream request logs attribute traffic to a Switchyard-version without a side channel or additional reporting infrastructure.--Set ``SWITCHYARD_TELEMETRY_OPT_OUT=1`` to suppress the header. The legacy-``NEMO_SWITCHYARD_TELEMETRY_OPT_OUT`` name is also honored for compatibility-with pre-rename environments.-"""--from __future__ import annotations--import importlib.metadata-import logging-import os-from functools import lru_cache--log = logging.getLogger(__name__)--HEADER_NAME = "X-Switchyard-Version"-OPT_OUT_ENVVAR = "SWITCHYARD_TELEMETRY_OPT_OUT"-LEGACY_OPT_OUT_ENVVAR = "NEMO_SWITCHYARD_TELEMETRY_OPT_OUT"--_FALSEY_VALUES = {"", "0", "false", "no"}---def _is_truthy_opt_out_value(value: str | None) -> bool:- """Return whether *value* should opt out of telemetry headers."""- if value is None:- return False- return value.strip().lower() not in _FALSEY_VALUES---def _is_opted_out() -> bool:- """Return whether telemetry headers are disabled by environment."""- return any(- _is_truthy_opt_out_value(os.environ.get(name))- for name in (OPT_OUT_ENVVAR, LEGACY_OPT_OUT_ENVVAR)- )---@lru_cache(maxsize=1)-def _get_version() -> str:- """Read the installed ``nemo-switchyard`` package version once."""- try:- return importlib.metadata.version("nemo-switchyard")- except Exception:- log.debug("telemetry: could not read switchyard package version", exc_info=True)- return "unknown"---def get_telemetry_headers() -> dict[str, str]:- """Return headers to attach to outbound LLM SDK clients.-- Returns an empty dict when telemetry is opted out, so callers can pass or- merge the result unconditionally.- """- if _is_opted_out():- return {}- return {HEADER_NAME: _get_version()}diff --git a/switchyard_rust/__init__.py b/switchyard_rust/__init__.pyindex 2603ef237..a18dd8c97 100644--- a/switchyard_rust/__init__.py+++ b/switchyard_rust/__init__.py@@ -1,155 +1,4 @@# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.# SPDX-License-Identifier: Apache-2.0-"""Python companion wrappers for the ``crates/switchyard-py`` bindings."""--from __future__ import annotations--from typing import TYPE_CHECKING--from switchyard_rust.translation import (- TranslationEngine,- is_native_translation_available,-)--if TYPE_CHECKING:- from switchyard_rust.components import AnthropicNativeBackend as AnthropicNativeBackend- from switchyard_rust.components import BackendFormat as BackendFormat- from switchyard_rust.components import EndpointConfig as EndpointConfig- from switchyard_rust.components import LlmTarget as LlmTarget- from switchyard_rust.components import LlmTargetBackend as LlmTargetBackend- from switchyard_rust.components import MultiLlmBackend as MultiLlmBackend- from switchyard_rust.components import OpenAiNativeBackend as OpenAiNativeBackend- from switchyard_rust.components import OpenAiPassthroughBackend as OpenAiPassthroughBackend- from switchyard_rust.components import (- RandomRoutingProcessorConfig as RandomRoutingProcessorConfig,- )- from switchyard_rust.components import StatsAccumulator as StatsAccumulator- from switchyard_rust.components import StatsLlmBackend as StatsLlmBackend- from switchyard_rust.components import StatsRequestProcessor as StatsRequestProcessor- from switchyard_rust.components import StatsResponseProcessor as StatsResponseProcessor- from switchyard_rust.core import ChatRequest as ChatRequest- from switchyard_rust.core import ChatRequestType as ChatRequestType- from switchyard_rust.core import ChatResponse as ChatResponse- from switchyard_rust.core import ChatResponseStream as ChatResponseStream- from switchyard_rust.core import ChatResponseType as ChatResponseType- from switchyard_rust.core import LLMBackend as LLMBackend- from switchyard_rust.core import ProxyContext as ProxyContext- from switchyard_rust.core import ProxyMetadata as ProxyMetadata- from switchyard_rust.core import SwitchyardBackendError as SwitchyardBackendError- from switchyard_rust.core import SwitchyardConfigError as SwitchyardConfigError- from switchyard_rust.core import (- SwitchyardDuplicateRegistrationError as SwitchyardDuplicateRegistrationError,- )- from switchyard_rust.core import SwitchyardInvalidIdError as SwitchyardInvalidIdError- from switchyard_rust.core import SwitchyardModelNotFoundError as SwitchyardModelNotFoundError- from switchyard_rust.core import SwitchyardProcessorError as SwitchyardProcessorError- from switchyard_rust.core import SwitchyardRuntimeError as SwitchyardRuntimeError- from switchyard_rust.core import (- SwitchyardUnsupportedRequestTypeError as SwitchyardUnsupportedRequestTypeError,- )- from switchyard_rust.core import SwitchyardUpstreamError as SwitchyardUpstreamError---def __getattr__(name: str) -> object:- if name in {- "AnthropicNativeBackend",- "BackendFormat",- "EndpointConfig",- "LlmTarget",- "LlmTargetBackend",- "MultiLlmBackend",- "OpenAiNativeBackend",- "OpenAiPassthroughBackend",- "RandomRoutingProcessorConfig",- "StatsAccumulator",- "StatsLlmBackend",- "StatsRequestProcessor",- "StatsResponseProcessor",- }:- from switchyard_rust import components-- return getattr(components, name)- if name == "ChatRequest":- from switchyard_rust.core import ChatRequest-- return ChatRequest- if name == "ChatRequestType":- from switchyard_rust.core import ChatRequestType-- return ChatRequestType- if name == "ChatResponse":- from switchyard_rust.core import ChatResponse-- return ChatResponse- if name == "ChatResponseStream":- from switchyard_rust.core import ChatResponseStream-- return ChatResponseStream- if name == "ChatResponseType":- from switchyard_rust.core import ChatResponseType-- return ChatResponseType- if name == "LLMBackend":- from switchyard_rust.core import LLMBackend-- return LLMBackend- if name == "ProxyMetadata":- from switchyard_rust.core import ProxyMetadata-- return ProxyMetadata- if name == "ProxyContext":- from switchyard_rust.core import ProxyContext-- return ProxyContext- if name in {- "SwitchyardRuntimeError",- "SwitchyardConfigError",- "SwitchyardInvalidIdError",- "SwitchyardDuplicateRegistrationError",- "SwitchyardModelNotFoundError",- "SwitchyardUnsupportedRequestTypeError",- "SwitchyardProcessorError",- "SwitchyardBackendError",- "SwitchyardUpstreamError",- }:- from switchyard_rust import core-- return getattr(core, name)- raise AttributeError(f"module {__name__!r} has no attribute {name!r}")---__all__ = [- "AnthropicNativeBackend",- "BackendFormat",- "ChatRequest",- "ChatRequestType",- "ChatResponse",- "ChatResponseStream",- "ChatResponseType",- "EndpointConfig",- "LLMBackend",- "LlmTarget",- "LlmTargetBackend",- "MultiLlmBackend",- "OpenAiNativeBackend",- "OpenAiPassthroughBackend",- "ProxyMetadata",- "ProxyContext",- "RandomRoutingProcessorConfig",- "StatsAccumulator",- "StatsLlmBackend",- "StatsRequestProcessor",- "StatsResponseProcessor",- "SwitchyardBackendError",- "SwitchyardConfigError",- "SwitchyardDuplicateRegistrationError",- "SwitchyardInvalidIdError",- "SwitchyardModelNotFoundError",- "SwitchyardProcessorError",- "SwitchyardRuntimeError",- "SwitchyardUnsupportedRequestTypeError",- "SwitchyardUpstreamError",- "TranslationEngine",- "is_native_translation_available",-]+"""Python wrappers for Switchyard's native libsy and server bindings."""diff --git a/switchyard_rust/_native.py b/switchyard_rust/_native.pynew file mode 100644index 000000000..3ddd376e3--- /dev/null+++ b/switchyard_rust/_native.py@@ -0,0 +1,34 @@+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.+# SPDX-License-Identifier: Apache-2.0++"""Load the private native extension shared by the public wrappers."""++from __future__ import annotations++import importlib+import importlib.metadata+import os+from typing import Any+++def _ensure_switchyard_version_env() -> None:+ if os.environ.get("SWITCHYARD_VERSION", "").strip():+ return+ try:+ version = importlib.metadata.version("nemo-switchyard")+ except importlib.metadata.PackageNotFoundError:+ return+ if version.strip():+ os.environ["SWITCHYARD_VERSION"] = version+++def load_native() -> Any:+ """Load the extension or report how to repair a source checkout."""+ _ensure_switchyard_version_env()+ try:+ return importlib.import_module("switchyard_rust._switchyard_rust")+ except ImportError as exc: # pragma: no cover - broken install guard+ raise RuntimeError(+ "The Switchyard native extension is required. Run `uv run maturin develop` "+ "or install a built switchyard wheel."+ ) from excdiff --git a/switchyard_rust/components.py b/switchyard_rust/components.pydeleted file mode 100644index e34cadea6..000000000--- a/switchyard_rust/components.py+++ /dev/null@@ -1,90 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Direct bindings for Rust-owned Switchyard components."""--from __future__ import annotations--from typing import TYPE_CHECKING, Any--from switchyard_rust.core import _load_native--_COMPONENT_EXPORTS = frozenset(- {- "AnthropicNativeBackend",- "BackendFormat",- "DimensionCollector",- "EndpointConfig",- "LlmTarget",- "LlmTargetBackend",- "MultiLlmBackend",- "OpenAiNativeBackend",- "OpenAiPassthroughBackend",- "PickOutcome",- "RandomRoutingProcessorConfig",- "ResponseFlag",- "ResponseSignalCollector",- "ResponseSignals",- "StatsAccumulator",- "StatsLlmBackend",- "StatsRequestProcessor",- "StatsResponseProcessor",- "ToolResultSignal",- "extract_response_signals",- "get_response_signals",- "get_tool_result_signal",- "set_stats_route_label",- "stage_pick_tier",- "stage_score_signal",- }-)-_BACKEND_EXPORTS = frozenset(- {- "AnthropicNativeBackend",- "MultiLlmBackend",- "OpenAiNativeBackend",- "OpenAiPassthroughBackend",- "StatsLlmBackend",- }-)--if TYPE_CHECKING:- AnthropicNativeBackend: type[Any]- BackendFormat: type[Any]- DimensionCollector: type[Any]- EndpointConfig: type[Any]- LlmTarget: type[Any]- LlmTargetBackend: type[Any]- MultiLlmBackend: type[Any]- OpenAiNativeBackend: type[Any]- OpenAiPassthroughBackend: type[Any]- PickOutcome: type[Any]- RandomRoutingProcessorConfig: type[Any]- ResponseFlag: type[Any]- ResponseSignalCollector: type[Any]- ResponseSignals: type[Any]- StatsAccumulator: type[Any]- StatsLlmBackend: type[Any]- StatsRequestProcessor: type[Any]- StatsResponseProcessor: type[Any]- ToolResultSignal: type[Any]- extract_response_signals: Any- get_response_signals: Any- get_tool_result_signal: Any- set_stats_route_label: Any- stage_pick_tier: Any- stage_score_signal: Any---def __getattr__(name: str) -> object:- if name in _COMPONENT_EXPORTS:- value = getattr(_load_native(), name)- if name in _BACKEND_EXPORTS:- from switchyard_rust.core import LLMBackend-- LLMBackend.register(value)- return value- raise AttributeError(f"module {__name__!r} has no attribute {name!r}")---__all__ = sorted(_COMPONENT_EXPORTS)diff --git a/switchyard_rust/components.pyi b/switchyard_rust/components.pyideleted file mode 100644index 36714b8fc..000000000--- a/switchyard_rust/components.pyi+++ /dev/null@@ -1,270 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--from __future__ import annotations--from collections.abc import Iterable-from typing import Any, ClassVar--from switchyard_rust.core import (- ChatRequestType,- LLMBackend,- ProxyContext,-)--class BackendFormat:- AUTO: ClassVar[BackendFormat]- OPENAI: ClassVar[BackendFormat]- RESPONSES: ClassVar[BackendFormat]- ANTHROPIC: ClassVar[BackendFormat]-- value: str-- def __init__(self, value: str = "auto") -> None: ...---class EndpointConfig:- base_url: str | None- api_key: str | None- timeout_secs: float | None-- def __init__(- self,- base_url: str | None = None,- api_key: str | None = None,- timeout_secs: float | None = None,- ) -> None: ...- def to_dict(self) -> dict[str, Any]: ...---class LlmTarget:- id: str- model: str- format: BackendFormat- backend_format: BackendFormat- endpoint: EndpointConfig- base_url: str | None- api_key: str | None- timeout: float | None- extra_body: dict[str, Any] | None- extra_headers: dict[str, str]-- def __init__(- self,- id: str | None = None,- model: str | None = None,- format: BackendFormat | str | None = None,- backend_format: BackendFormat | str | None = None,- endpoint: EndpointConfig | dict[str, Any] | None = None,- base_url: str | None = None,- api_key: str | None = None,- timeout_secs: float | None = None,- timeout: float | None = None,- extra_body: dict[str, Any] | None = None,- extra_headers: dict[str, str] | None = None,- ) -> None: ...- def to_dict(self) -> dict[str, Any]: ...- def model_dump(self) -> dict[str, Any]: ...---class RandomRoutingProcessorConfig:- strong: LlmTarget- weak: LlmTarget- strong_probability: float- rng_seed: int | None-- def __init__(- self,- strong: LlmTarget,- weak: LlmTarget,- strong_probability: float = 0.5,- rng_seed: int | None = None,- ) -> None: ...- def to_dict(self) -> dict[str, Any]: ...---class StatsAccumulator:- def __init__(self) -> None: ...- async def record_success(- self,- model: str,- backend_latency_ms: float | None = None,- tier: str | None = None,- ) -> None: ...- async def record_error(self, model: str, tier: str | None = None) -> None: ...- async def record_usage(- self,- model: str,- prompt_tokens: int = 0,- completion_tokens: int = 0,- cached_tokens: int = 0,- cache_creation_tokens: int = 0,- reasoning_tokens: int = 0,- total_latency_ms: float | None = None,- routing_overhead_ms: float | None = None,- tier: str | None = None,- ) -> None: ...- async def record_classifier_usage(- self,- model: str,- prompt_tokens: int = 0,- completion_tokens: int = 0,- cached_tokens: int = 0,- cache_creation_tokens: int = 0,- reasoning_tokens: int = 0,- latency_ms: float | None = None,- ) -> None: ...- async def record_classifier_error(self, model: str) -> None: ...- async def snapshot(self) -> dict[str, Any]: ...- def snapshot_sync(self) -> dict[str, Any]: ...- async def reset(self) -> None: ...- def reset_sync(self) -> None: ...---def set_stats_route_label(ctx: Any, label: str) -> None: ...---class LlmTargetBackend:- target: LlmTarget-- def __init__(self, target: LlmTarget, backend: LLMBackend) -> None: ...---class OpenAiNativeBackend(LLMBackend):- target: LlmTarget-- def __init__(self, target: LlmTarget) -> None: ...---class OpenAiPassthroughBackend(LLMBackend):- endpoint: EndpointConfig-- def __init__(- self,- endpoint: EndpointConfig | dict[str, Any] | None = None,- api_key: str | None = None,- base_url: str | None = None,- timeout_secs: float | None = None,- timeout: float | None = None,- ) -> None: ...---class AnthropicNativeBackend(LLMBackend):- target: LlmTarget-- def __init__(self, target: LlmTarget) -> None: ...---class MultiLlmBackend(LLMBackend):- def __init__(- self,- targets: Iterable[LlmTargetBackend | tuple[LlmTarget, LLMBackend]],- supported_request_types: Iterable[ChatRequestType | str] | None = None,- default_target_id: str | None = None,- ) -> None: ...- def target_ids(self) -> list[str]: ...- def default_target_id(self) -> str | None: ...---class StatsLlmBackend(LLMBackend):- accumulator: StatsAccumulator-- def __init__(self, inner: LLMBackend, accumulator: StatsAccumulator) -> None: ...---class StatsRequestProcessor:- def __init__(self) -> None: ...- async def process(self, ctx: ProxyContext, request: Any) -> Any: ...- async def startup(self) -> None: ...- async def shutdown(self) -> None: ...---class StatsResponseProcessor:- accumulator: StatsAccumulator-- def __init__(self, accumulator: StatsAccumulator) -> None: ...- async def process(self, ctx: ProxyContext, response: Any) -> Any: ...- async def startup(self) -> None: ...- async def shutdown(self) -> None: ...- def get_endpoint(self) -> object: ...---class DimensionCollector:- def __init__(- self,- *,- recent_window: int | None = None,- ) -> None: ...- async def process(self, ctx: ProxyContext, request: Any) -> Any: ...- async def startup(self) -> None: ...- async def shutdown(self) -> None: ...---class ToolResultSignal:- severity: float- turn_depth: int- write_count: int- edit_count: int- read_count: int- todowrite_count: int- recent_write_count: int- recent_edit_count: int- recent_read_count: int- recent_todowrite_count: int- pure_bash_streak: int- no_error_streak: int- tests_passed: bool- compacted: bool---def get_tool_result_signal(ctx: ProxyContext) -> ToolResultSignal | None: ...---class PickOutcome:- resolved: bool- tier: str | None- source: str | None- default_tier: str- score: float- confidence: float | None---def stage_pick_tier(- signal: ToolResultSignal, picker_mode: str, confidence_threshold: float-) -> PickOutcome: ...---def stage_score_signal(signal: ToolResultSignal) -> tuple[float, float]: ...---class ResponseFlag:- MALFORMED_TOOL_CALL_JSON: ClassVar[ResponseFlag]- EMPTY_RESPONSE: ClassVar[ResponseFlag]- TRUNCATED_COMPLETION: ClassVar[ResponseFlag]- MISSING_REQUIRED_ARGS: ClassVar[ResponseFlag]-- def __eq__(self, other: object) -> bool: ...- def __hash__(self) -> int: ...---class ResponseSignals:- flags: list[ResponseFlag]-- def has_failures(self) -> bool: ...- def contains(self, flag: ResponseFlag) -> bool: ...---class ResponseSignalCollector:- def __init__(self) -> None: ...- async def process(self, ctx: ProxyContext, response: Any) -> Any: ...- async def startup(self) -> None: ...- async def shutdown(self) -> None: ...---def get_response_signals(ctx: ProxyContext) -> ResponseSignals | None: ...---def extract_response_signals(body: dict[str, Any] | None) -> ResponseSignals: ...---__all__: list[str]diff --git a/switchyard_rust/core.py b/switchyard_rust/core.pydeleted file mode 100644index f9ed33bfd..000000000--- a/switchyard_rust/core.py+++ /dev/null@@ -1,793 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Python core values and compatibility chain wrappers."""--from __future__ import annotations--import importlib-import importlib.metadata-import json-import os-from abc import ABCMeta-from collections.abc import Iterable, Mapping-from enum import Enum-from typing import TYPE_CHECKING, Any, ClassVar, Protocol, TypeAlias, cast--JsonScalar: TypeAlias = bool | int | float | str | None-JsonValue: TypeAlias = JsonScalar | list["JsonValue"] | dict[str, "JsonValue"]---class ChatRequestType(Enum):- """Wire format carried by a :class:`ChatRequest`."""-- OPENAI_CHAT = "openai_chat"- OPENAI_RESPONSES = "openai_responses"- ANTHROPIC = "anthropic"-- def __str__(self) -> str:- return self.value---class ChatResponseType(Enum):- """Wire format and delivery mode carried by a :class:`ChatResponse`."""-- OPENAI_COMPLETION = "openai_completion"- OPENAI_STREAM = "openai_stream"- OPENAI_RESPONSES_COMPLETION = "openai_responses_completion"- OPENAI_RESPONSES_STREAM = "openai_responses_stream"- ANTHROPIC_COMPLETION = "anthropic_completion"- ANTHROPIC_STREAM = "anthropic_stream"-- def __str__(self) -> str:- return self.value---def _json_copy(value: object) -> JsonValue:- """Normalize an object to an owned JSON value."""- model_dump = getattr(value, "model_dump", None)- if callable(model_dump):- value = model_dump(mode="json", exclude_none=True)- else:- to_dict = getattr(value, "to_dict", None)- if callable(to_dict):- value = to_dict()- try:- return cast(JsonValue, json.loads(json.dumps(value, allow_nan=False)))- except (TypeError, ValueError) as error:- raise ValueError(str(error)) from error---class ChatRequest:- """Owned provider request body with an explicit wire format."""-- def __init__(self, request_type: ChatRequestType, body: object) -> None:- self.request_type = request_type- self._body = _json_copy(body)-- @classmethod- def openai_chat(cls, body: object) -> ChatRequest:- """Build an OpenAI Chat Completions request."""- return cls(ChatRequestType.OPENAI_CHAT, body)-- @classmethod- def openai_responses(cls, body: object) -> ChatRequest:- """Build an OpenAI Responses request."""- return cls(ChatRequestType.OPENAI_RESPONSES, body)-- @classmethod- def anthropic(cls, body: object) -> ChatRequest:- """Build an Anthropic Messages request."""- return cls(ChatRequestType.ANTHROPIC, body)-- @property- def body(self) -> Any:- """Return an owned copy of the request body."""- return _json_copy(self._body)-- @property- def model(self) -> str | None:- """Return the request model when it is a string."""- if isinstance(self._body, dict):- model = self._body.get("model")- return model if isinstance(model, str) else None- return None-- def validate(self) -> None:- """Reject semantically invalid message-based requests."""- if self.request_type in {ChatRequestType.OPENAI_CHAT, ChatRequestType.ANTHROPIC}:- if isinstance(self._body, dict) and self._body.get("messages") == []:- raise _load_native().SwitchyardInvalidRequestError(- "messages must be a non-empty array"- )-- def set_model(self, model: str) -> None:- """Set the request model, replacing a malformed non-object body."""- if not isinstance(self._body, dict):- self._body = {}- self._body["model"] = model-- def replace_body(self, body: object) -> None:- """Replace the body without changing its wire format."""- self._body = _json_copy(body)-- def to_body(self) -> JsonValue:- """Return an owned copy of the request body."""- return _json_copy(self._body)-- def __repr__(self) -> str:- model = f", model={self.model!r}" if self.model is not None else ""- return f"ChatRequest(request_type={self.request_type.value!r}{model})"---class ChatResponseStream:- """Single-consumer async response stream with callback transforms."""-- def __init__(self, source: object) -> None:- self._native = _load_native()._NativeChatResponseStream(source)-- @classmethod- def _from_native(cls, stream: Any) -> ChatResponseStream:- instance = cls.__new__(cls)- instance._native = stream- return instance-- def tap(self, callback: object) -> ChatResponseStream:- """Observe each event before mapping."""- self._native.tap(callback)- return self-- def map(self, callback: object) -> ChatResponseStream:- """Transform each event in registration order."""- self._native.map(callback)- return self-- def on_complete(self, callback: object) -> ChatResponseStream:- """Run a callback once after normal stream exhaustion."""- self._native.on_complete(callback)- return self-- def __aiter__(self) -> ChatResponseStream:- self._native.__aiter__()- return self-- async def __anext__(self) -> Any:- return await self._native.__anext__()-- async def aclose(self) -> None:- """Close the upstream source and release its connection."""- await self._native.aclose()-- def __repr__(self) -> str:- return "ChatResponseStream()"---class ChatResponse:- """Owned buffered response or live response stream."""-- def __init__(- self,- response_type: ChatResponseType,- *,- body: object | None = None,- stream: object | None = None,- ) -> None:- self.response_type = response_type- self._body: JsonValue | None- self._stream: ChatResponseStream | None- if response_type.value.endswith("_stream"):- if stream is None:- raise TypeError("streaming ChatResponse requires a stream")- self._stream = (- stream if isinstance(stream, ChatResponseStream) else ChatResponseStream(stream)- )- self._body = None- else:- self._body = _json_copy(body)- self._stream = None-- @classmethod- def openai_completion(cls, body: object) -> ChatResponse:- """Build a buffered OpenAI Chat Completions response."""- return cls(ChatResponseType.OPENAI_COMPLETION, body=body)-- @classmethod- def openai_stream(cls, stream: object) -> ChatResponse:- """Build a streaming OpenAI Chat Completions response."""- return cls(ChatResponseType.OPENAI_STREAM, stream=stream)-- @classmethod- def openai_responses_completion(cls, body: object) -> ChatResponse:- """Build a buffered OpenAI Responses response."""- return cls(ChatResponseType.OPENAI_RESPONSES_COMPLETION, body=body)-- @classmethod- def openai_responses_stream(cls, stream: object) -> ChatResponse:- """Build a streaming OpenAI Responses response."""- return cls(ChatResponseType.OPENAI_RESPONSES_STREAM, stream=stream)-- @classmethod- def anthropic_completion(cls, body: object) -> ChatResponse:- """Build a buffered Anthropic response."""- return cls(ChatResponseType.ANTHROPIC_COMPLETION, body=body)-- @classmethod- def anthropic_stream(cls, stream: object) -> ChatResponse:- """Build a streaming Anthropic response."""- return cls(ChatResponseType.ANTHROPIC_STREAM, stream=stream)-- @property- def body(self) -> Any:- """Return an owned buffered body."""- if self._stream is not None:- raise AttributeError("streaming ChatResponse values do not have a buffered body")- return _json_copy(self._body)-- @property- def stream(self) -> ChatResponseStream:- """Return the live stream."""- if self._stream is None:- raise AttributeError("buffered ChatResponse values do not have a stream")- return self._stream-- def replace_body(self, body: object) -> None:- """Replace a buffered body without changing its wire shape."""- if self._stream is not None:- raise ValueError("streaming ChatResponse values do not have a replaceable body")- self._body = _json_copy(body)-- def to_body(self) -> JsonValue:- """Return an owned buffered body."""- if self._stream is not None:- raise AttributeError("streaming ChatResponse values do not have a buffered body")- return _json_copy(self._body)-- def __repr__(self) -> str:- return f"ChatResponse(response_type={self.response_type.value!r})"---class ProxyMetadata(dict[str, Any]):- """Mutable per-request metadata owned by Python."""---class ProxyContext:- """Per-request state shared by Python and native components."""-- def __init__(- self,- metadata: Mapping[str, Any] | None = None,- request_id: str | None = None,- ) -> None:- self._native = _load_native()._NativeProxyContext(metadata, request_id)- self.metadata = ProxyMetadata(metadata or {})-- @property- def request_id(self) -> str | None:- return cast(str | None, self._native.request_id)-- @request_id.setter- def request_id(self, value: str | None) -> None:- self._native.request_id = value-- @property- def inbound_format(self) -> ChatRequestType | None:- value = self._native.inbound_format- return request_type_enum(value) if value is not None else None-- @inbound_format.setter- def inbound_format(self, value: object | None) -> None:- self._native.inbound_format = value-- @property- def selected_model(self) -> str | None:- return cast(str | None, self._native.selected_model)-- @selected_model.setter- def selected_model(self, value: str | None) -> None:- self._native.selected_model = value-- @property- def selected_target(self) -> str | None:- return cast(str | None, self._native.selected_target)-- @selected_target.setter- def selected_target(self, value: str | None) -> None:- self._native.selected_target = value-- @property- def evicted_targets(self) -> list[str] | None:- return cast(list[str] | None, self._native.evicted_targets)-- @evicted_targets.setter- def evicted_targets(self, value: list[str] | None) -> None:- self._native.evicted_targets = value-- @property- def backend_call_latency_ms(self) -> float | None:- return cast(float | None, self._native.backend_call_latency_ms)-- @backend_call_latency_ms.setter- def backend_call_latency_ms(self, value: float | None) -> None:- self._native.backend_call_latency_ms = value-- def __repr__(self) -> str:- return repr(self._native)---class LLMBackend(metaclass=ABCMeta): # noqa: B024- """Base class for Python and registered native LLM backends."""-- def __new__(cls, *args: object, **kwargs: object) -> LLMBackend:- if cls is LLMBackend:- raise TypeError("can't instantiate abstract role LLMBackend")- return super().__new__(cls)-- @property- def supported_request_types(self) -> list[ChatRequestType]:- """Return request formats accepted by this backend."""- raise NotImplementedError("LLMBackend.supported_request_types must be implemented")-- async def startup(self) -> None:- """Start backend resources."""- return None-- async def shutdown(self) -> None:- """Stop backend resources."""- return None-- async def call(self, ctx: ProxyContext, request: ChatRequest) -> ChatResponse:- """Call the upstream model."""- raise NotImplementedError("LLMBackend.call must be implemented")---class _NativeModule(Protocol):- _NativeChatResponseStream: Any- _NativeProxyContext: Any- is_subagent_request: Any- SwitchyardRuntimeError: type[RuntimeError]- SwitchyardConfigError: type[RuntimeError]- SwitchyardInvalidIdError: type[RuntimeError]- SwitchyardDuplicateRegistrationError: type[RuntimeError]- SwitchyardModelNotFoundError: type[RuntimeError]- SwitchyardUnsupportedRequestTypeError: type[RuntimeError]- SwitchyardInvalidRequestError: type[RuntimeError]- SwitchyardProcessorError: type[RuntimeError]- SwitchyardBackendError: type[RuntimeError]- SwitchyardUpstreamError: type[RuntimeError]- SwitchyardContextWindowExceededError: type[RuntimeError]- SwitchyardContextPoolExhaustedError: type[RuntimeError]---def _ensure_switchyard_version_env() -> None:- if os.environ.get("SWITCHYARD_VERSION", "").strip():- return- for distribution in ("switchyard", "nemo-switchyard"):- try:- version = importlib.metadata.version(distribution)- except importlib.metadata.PackageNotFoundError:- continue- if version.strip():- os.environ["SWITCHYARD_VERSION"] = version- return---def _load_native() -> _NativeModule:- _ensure_switchyard_version_env()- try:- return cast(_NativeModule, importlib.import_module("switchyard_rust._switchyard_rust"))- except ImportError as exc: # pragma: no cover - broken install guard- raise RuntimeError(- "The Switchyard native extension is required. Run `uv run maturin develop` "- "or install a built switchyard wheel."- ) from exc---if TYPE_CHECKING:- class SwitchyardRuntimeError(RuntimeError):- """Base class for native Switchyard runtime errors."""-- class SwitchyardConfigError(SwitchyardRuntimeError):- """Raised for invalid Switchyard configuration."""-- class SwitchyardInvalidIdError(SwitchyardRuntimeError):- """Raised when a Switchyard identifier is invalid."""-- class SwitchyardDuplicateRegistrationError(SwitchyardRuntimeError):- """Raised when a registry receives a duplicate ID."""-- class SwitchyardModelNotFoundError(SwitchyardRuntimeError):- """Raised when a route table cannot find a model."""-- class SwitchyardUnsupportedRequestTypeError(SwitchyardRuntimeError):- """Raised when a component rejects a request format."""-- class SwitchyardInvalidRequestError(SwitchyardRuntimeError):- """Raised when a request body fails semantic validation."""-- class SwitchyardProcessorError(SwitchyardRuntimeError):- """Raised when a processor fails."""-- class SwitchyardBackendError(SwitchyardRuntimeError):- """Raised when a backend fails."""-- class SwitchyardUpstreamError(SwitchyardRuntimeError):- """Raised when an upstream call fails."""-- status_code: int- body: str-- class SwitchyardContextWindowExceededError(SwitchyardBackendError):- """Raised when an upstream request exceeds the context window."""-- class SwitchyardContextPoolExhaustedError(SwitchyardBackendError):- """Raised when every attempted routing target was evicted."""-else:- SwitchyardRuntimeError = _load_native().SwitchyardRuntimeError- SwitchyardConfigError = _load_native().SwitchyardConfigError- SwitchyardInvalidIdError = _load_native().SwitchyardInvalidIdError- SwitchyardDuplicateRegistrationError = _load_native().SwitchyardDuplicateRegistrationError- SwitchyardModelNotFoundError = _load_native().SwitchyardModelNotFoundError- SwitchyardUnsupportedRequestTypeError = _load_native().SwitchyardUnsupportedRequestTypeError- SwitchyardInvalidRequestError = _load_native().SwitchyardInvalidRequestError- SwitchyardProcessorError = _load_native().SwitchyardProcessorError- SwitchyardBackendError = _load_native().SwitchyardBackendError- SwitchyardUpstreamError = _load_native().SwitchyardUpstreamError- SwitchyardContextWindowExceededError = _load_native().SwitchyardContextWindowExceededError- SwitchyardContextPoolExhaustedError = _load_native().SwitchyardContextPoolExhaustedError---def is_subagent_request(headers: Mapping[str, str]) -> bool:- """Return whether canonical protocol metadata marks delegated agent work."""- return bool(_load_native().is_subagent_request(headers))---def _role_type_name(value: object) -> str:- """Return a stable display name for role validation errors."""- return type(value).__name__---def _processor_error(error: BaseException) -> RuntimeError:- """Wrap processor failures in the public Switchyard processor error."""- native = _load_native()- if isinstance(error, native.SwitchyardRuntimeError):- return error- return native.SwitchyardProcessorError(str(error))---def _backend_error(error: BaseException) -> RuntimeError:- """Wrap backend failures in the public Switchyard backend error."""- native = _load_native()- if isinstance(error, native.SwitchyardRuntimeError):- return error- return native.SwitchyardBackendError(str(error))---class Switchyard:- """Python-only compatibility chain for the current FastAPI/recipe surface."""-- state_key: ClassVar[str] = "switchyard"-- def __init__(- self,- *,- request_processors: Iterable[Any] | None = None,- backend: Any,- response_processors: Iterable[Any] | None = None,- translator: Any,- fallback_target_on_evict: str | None = None,- ) -> None:- if not isinstance(backend, LLMBackend):- actual = _role_type_name(backend)- raise TypeError(f"Switchyard backend must be LLMBackend, got {actual}")- self._request_components = tuple(request_processors or ())- self._backend = backend- self._response_components = tuple(response_processors or ())- self._translator = translator- self._fallback_target_on_evict = fallback_target_on_evict-- def iter_components(self) -> list[Any]:- """Return lifecycle components in startup order."""- return [- *self._request_components,- self._backend,- *self._response_components,- self._translator,- ]-- async def call(- self,- request: Any,- *,- ctx: Any | None = None,- ) -> Any:- """Run the compatibility chain and translate the final response."""- context = ctx if ctx is not None else ProxyContext()- processed_request = await self._process_request_components(context, request)- try:- response = await self._call_backend_stage(context, processed_request)- except _load_native().SwitchyardContextWindowExceededError as error:- if self._fallback_target_on_evict is None:- raise- response = await self._retry_after_context_overflow(- context,- processed_request,- error,- )- return await self._translator.translate(context, processed_request, response)-- async def _call_backend_stage(- self,- ctx: Any,- request: Any,- ) -> Any:- """Call the backend and then response processors."""- native = _load_native()- try:- response: object = await cast(Any, self._backend).call(ctx, request)- except native.SwitchyardContextWindowExceededError:- raise- except Exception as error:- raise _backend_error(error) from error- if not isinstance(response, ChatResponse):- actual = _role_type_name(response)- raise native.SwitchyardBackendError(- f"Switchyard backend returned {actual}, expected ChatResponse",- )- return await self._process_response_components(ctx, response)-- async def _process_request_components(self, ctx: Any, request: Any) -> Any:- """Run request-side compatibility components in order."""- native = _load_native()- current = request- for component in self._request_components:- process = getattr(component, "process", None)- if not callable(process):- actual = _role_type_name(component)- raise native.SwitchyardProcessorError(- f"Request component {actual} must define process(ctx, request)",- )- try:- current = await process(ctx, current)- except Exception as error:- raise _processor_error(error) from error- if not isinstance(current, ChatRequest):- actual = _role_type_name(current)- raise native.SwitchyardProcessorError(- f"Request component returned {actual}, expected ChatRequest",- )- return current-- async def _process_response_components(self, ctx: Any, response: Any) -> Any:- """Run response-side compatibility components in order."""- native = _load_native()- current = response- for component in self._response_components:- process = getattr(component, "process", None)- if not callable(process):- actual = _role_type_name(component)- raise native.SwitchyardProcessorError(- f"Response component {actual} must define process(ctx, response)",- )- try:- current = await process(ctx, current)- except Exception as error:- raise _processor_error(error) from error- if not isinstance(current, ChatResponse):- actual = _role_type_name(current)- raise native.SwitchyardProcessorError(- f"Response component returned {actual}, expected ChatResponse",- )- return current-- async def _retry_after_context_overflow(- self,- ctx: Any,- request: Any,- error: BaseException,- ) -> Any:- """Record the evicted target, rewrite the selection, and retry once."""- native = _load_native()- target_id = self._overflow_target_id(ctx, error)- if target_id is not None:- evicted = set(ctx.evicted_targets or [])- evicted.add(target_id)- ctx.evicted_targets = sorted(evicted)- self._rewrite_evicted_pick(ctx)- try:- return await self._call_backend_stage(ctx, request)- except native.SwitchyardContextWindowExceededError as second:- last_target = self._overflow_target_id(ctx, second) or "unknown"- reason = "all attempted targets returned context-window overflow"- pool_error = native.SwitchyardContextPoolExhaustedError(- f"context pool exhausted after target {last_target}: {reason}",- )- cast(Any, pool_error).last_target_id = last_target- cast(Any, pool_error).reason = reason- raise pool_error from second-- def _overflow_target_id(- self,- ctx: Any,- error: BaseException,- ) -> str | None:- """Return the target id carried by an overflow error or current context."""- target_id = getattr(error, "target_id", None)- if isinstance(target_id, str) and target_id:- return target_id- selected = ctx.selected_target- return selected if isinstance(selected, str) and selected else None-- def _rewrite_evicted_pick(self, ctx: Any) -> None:- """Rewrite an evicted or exception-only target to the configured fallback."""- selected = ctx.selected_target- evicted = set(ctx.evicted_targets or [])- if (selected is not None and selected in evicted) or (not selected and evicted):- ctx.selected_target = self._fallback_target_on_evict---def request_type_value(value: object) -> str:- """Normalize request-format tags and wire strings."""- raw = value.value if hasattr(value, "value") else value- if not isinstance(raw, str):- raise TypeError(f"Request type must be a string-like value, got {type(raw).__name__}")- if raw == "anthropic_messages":- return "anthropic"- if raw in {"openai_chat", "openai_responses", "anthropic"}:- return raw- raise ValueError(f"Unknown request type: {value!r}")---def request_type_enum(value: object) -> ChatRequestType:- """Normalize a request-format tag to the Python enum."""- return ChatRequestType(request_type_value(value))---def request_type_matches(request: object, request_type: object) -> bool:- """Return whether a request has the requested wire format."""- return request_type_value(cast(Any, request).request_type) == request_type_value(request_type)---def request_with_type(request_type: object, body: Mapping[str, Any] | JsonValue) -> ChatRequest:- """Build a request with the given wire format."""- normalized = request_type_value(request_type)- if normalized == "openai_chat":- return ChatRequest.openai_chat(body)- if normalized == "openai_responses":- return ChatRequest.openai_responses(body)- if normalized == "anthropic":- return ChatRequest.anthropic(body)- raise ValueError(f"Unknown request type: {request_type!r}")---def response_type_value(value: object) -> str:- """Normalize response-format tags and wire strings."""- raw = value.value if hasattr(value, "value") else value- if not isinstance(raw, str):- raise TypeError(f"Response type must be a string-like value, got {type(raw).__name__}")- legacy_aliases = {- "completion": "openai_completion",- "stream": "openai_stream",- "responses_api_completion": "openai_responses_completion",- "responses_api_stream": "openai_responses_stream",- }- raw = legacy_aliases.get(raw, raw)- if raw in {- "openai_completion",- "openai_stream",- "openai_responses_completion",- "openai_responses_stream",- "anthropic_completion",- "anthropic_stream",- }:- return raw- raise ValueError(f"Unknown response type: {value!r}")---def response_type_enum(value: object) -> ChatResponseType:- """Normalize a response-format tag to the Python enum."""- return ChatResponseType(response_type_value(value))---def response_type_matches(response: object, response_type: object) -> bool:- """Return whether a Rust-backed response has the requested wire shape."""- return response_type_value(cast(Any, response).response_type) == response_type_value(response_type)---def response_with_type(- response_type: object,- body_or_stream: Mapping[str, Any] | JsonValue | object,-) -> ChatResponse:- """Build a response with the given wire shape."""- normalized = response_type_value(response_type)- if normalized == "openai_completion":- return ChatResponse.openai_completion(body_or_stream)- if normalized == "openai_stream":- return ChatResponse.openai_stream(body_or_stream)- if normalized == "openai_responses_completion":- return ChatResponse.openai_responses_completion(body_or_stream)- if normalized == "openai_responses_stream":- return ChatResponse.openai_responses_stream(body_or_stream)- if normalized == "anthropic_completion":- return ChatResponse.anthropic_completion(body_or_stream)- if normalized == "anthropic_stream":- return ChatResponse.anthropic_stream(body_or_stream)- raise ValueError(f"Unknown response type: {response_type!r}")---def response_type_for_request_type(- request_type: object,- *,- stream: bool,-) -> ChatResponseType:- """Return the response shape that corresponds to a request format."""- normalized = request_type_value(request_type)- if normalized == "openai_chat":- return response_type_enum("openai_stream" if stream else "openai_completion")- if normalized == "openai_responses":- return response_type_enum(- "openai_responses_stream" if stream else "openai_responses_completion"- )- if normalized == "anthropic":- return response_type_enum("anthropic_stream" if stream else "anthropic_completion")- raise ValueError(f"Unknown request type: {request_type!r}")---def response_matches_request_type(- response: object,- request_type: object,-) -> bool:- """Return whether a response's provider format matches a request format."""- response_type = response_type_value(cast(Any, response).response_type)- request_type_normalized = request_type_value(request_type)- if request_type_normalized == "openai_chat":- return response_type in {"openai_completion", "openai_stream"}- if request_type_normalized == "openai_responses":- return response_type in {- "openai_responses_completion",- "openai_responses_stream",- }- if request_type_normalized == "anthropic":- return response_type in {"anthropic_completion", "anthropic_stream"}- raise ValueError(f"Unknown request type: {request_type!r}")---def response_is_streaming(response: object) -> bool:- """Return whether a response carries a live stream."""- return response_type_value(cast(Any, response).response_type).endswith("_stream")---__all__ = [- "ChatRequest",- "ChatRequestType",- "ChatResponse",- "ChatResponseStream",- "ChatResponseType",- "LLMBackend",- "ProxyMetadata",- "ProxyContext",- "Switchyard",- "SwitchyardBackendError",- "SwitchyardConfigError",- "SwitchyardContextPoolExhaustedError",- "SwitchyardContextWindowExceededError",- "SwitchyardDuplicateRegistrationError",- "SwitchyardInvalidIdError",- "SwitchyardInvalidRequestError",- "SwitchyardModelNotFoundError",- "SwitchyardProcessorError",- "SwitchyardRuntimeError",- "SwitchyardUnsupportedRequestTypeError",- "SwitchyardUpstreamError",- "is_subagent_request",- "request_type_enum",- "request_type_matches",- "request_type_value",- "request_with_type",- "response_is_streaming",- "response_matches_request_type",- "response_type_enum",- "response_type_for_request_type",- "response_type_matches",- "response_type_value",- "response_with_type",-]diff --git a/switchyard_rust/libsy.py b/switchyard_rust/libsy.pyindex dd01ed5d4..a7eb5c85a 100644--- a/switchyard_rust/libsy.py+++ b/switchyard_rust/libsy.py@@ -8,7 +8,7 @@ from __future__ import annotationsfrom collections.abc import Mappingfrom typing import TYPE_CHECKING, Any, Protocol-from switchyard_rust.core import _load_native+from switchyard_rust._native import load_native_EXPORTS = frozenset({@@ -40,9 +40,7 @@ if TYPE_CHECKING:from collections.abc import Sequencefrom typing import final- from switchyard_rust.core import SwitchyardRuntimeError-- class LibsyError(SwitchyardRuntimeError): ...+ class LibsyError(RuntimeError): ...@finalclass LlmTarget:@@ -112,7 +110,7 @@ if TYPE_CHECKING:def __getattr__(name: str) -> object:if name in _EXPORTS:- native: Any = _load_native()+ native: Any = load_native()return getattr(native.libsy, name)raise AttributeError(f"module {__name__!r} has no attribute {name!r}")diff --git a/switchyard_rust/server.py b/switchyard_rust/server.pyindex 46b5d6036..193aef353 100644--- a/switchyard_rust/server.py+++ b/switchyard_rust/server.py@@ -8,7 +8,7 @@ from __future__ import annotationsfrom os import PathLikefrom typing import TYPE_CHECKING, Any, final-from switchyard_rust.core import _load_native+from switchyard_rust._native import load_nativeif TYPE_CHECKING:@@ -38,7 +38,7 @@ if TYPE_CHECKING:def __getattr__(name: str) -> object:if name == "Server":- native: Any = _load_native()+ native: Any = load_native()return native.server.Serverraise AttributeError(f"module {__name__!r} has no attribute {name!r}")diff --git a/switchyard_rust/translation.py b/switchyard_rust/translation.pydeleted file mode 100644index b7b5c9843..000000000--- a/switchyard_rust/translation.py+++ /dev/null@@ -1,527 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Python-side Switchyard wrappers over the Rust translation engine."""--from __future__ import annotations--import dataclasses-import importlib-import inspect-import json-import logging-from collections.abc import AsyncGenerator, AsyncIterable, Iterable, Mapping, Sequence-from typing import TYPE_CHECKING, Any, Literal, Protocol, TypeVar, cast--from openai.types.chat import ChatCompletionChunk--if TYPE_CHECKING:- from switchyard.lib.proxy_context import ProxyContext- from switchyard.lib.roles import TranslatedResponse, TranslatedStream- from switchyard_rust.core import ChatRequest, ChatRequestType, ChatResponse---class _NativeStreamTranslation(Protocol):- def translate_event(self, event: Mapping[str, Any]) -> list[dict[str, Any]]: ...- def finish(self) -> list[dict[str, Any]]: ...---class _NativeTranslationEngine(Protocol):- def translate_request(- self,- source: str,- target: str,- body: Mapping[str, Any],- ) -> dict[str, Any]: ...-- def translate_response(- self,- source: str,- target: str,- body: Mapping[str, Any],- ) -> dict[str, Any]: ...-- def stream(- self,- source: str,- target: str,- model: str | None = None,- message_id: str | None = None,- ) -> _NativeStreamTranslation: ...-- def normalize_anthropic_tool_use_ids(self, messages: object) -> object: ...---class _NativeModule(Protocol):- TranslationEngine: type[_NativeTranslationEngine]---_T = TypeVar("_T")-_NativeFormat = Literal["openai_chat", "openai_responses", "anthropic_messages"]-_StreamOutput = Literal["objects", "chat_chunks", "responses_sse"]-_native: _NativeModule | None = None-_log = logging.getLogger(__name__)---def _load_native() -> _NativeModule:- global _native- if _native is None:- try:- _native = cast(- _NativeModule,- importlib.import_module("switchyard_rust._switchyard_rust"),- )- except ImportError as exc: # pragma: no cover - broken install guard- raise RuntimeError(- "Rust translation extension is required. Run `uv run maturin develop` "- "or install a built switchyard wheel."- ) from exc- return _native---class TranslationEngine:- """Single Python-facing engine for request, response, and stream translation."""-- def __init__(self) -> None:- self._inner = _load_native().TranslationEngine()-- def translate_request(- self,- source: str | ChatRequestType,- target: str | ChatRequestType,- body: Mapping[str, Any] | object,- ) -> dict[str, Any]:- """Translate a JSON request body between provider wire formats."""- return self._inner.translate_request(- _format_name(source),- _format_name(target),- _jsonable_mapping(body),- )-- def translate_response(- self,- source: str | ChatRequestType,- target: str | ChatRequestType,- body: Mapping[str, Any] | object,- ) -> dict[str, Any]:- """Translate a JSON response body between provider wire formats."""- return self._inner.translate_response(- _format_name(source),- _format_name(target),- _jsonable_mapping(body),- )-- def request_to(- self,- target: str | ChatRequestType,- request: ChatRequest,- ) -> ChatRequest:- """Return *request* in the target wire format."""- source = _request_format(request)- target_format = _format_name(target)- if source == target_format:- return request- return _wrap_request(- target_format,- self.translate_request(source, target_format, request.body),- )-- def request_to_any_of(- self,- request: ChatRequest,- supported: Sequence[ChatRequestType],- ) -> ChatRequest:- """Passthrough when possible, otherwise translate to the first supported type."""- if not supported:- raise ValueError("supported must be non-empty")- source = _request_format(request)- supported_formats = [_format_name(item) for item in supported]- if source in supported_formats:- return request- return self.request_to(supported_formats[0], request)-- def response_to(- self,- target: str | ChatRequestType,- response: ChatResponse,- *,- served_model: str | None = None,- ) -> ChatResponse:- """Return a ChatResponse wrapper in the target wire format."""- source = _response_format(response)- target_format = _format_name(target)- if source == target_format:- return response- if _is_streaming_response(response):- return _wrap_streaming_response(- target_format,- self._translate_response_stream(source, target_format, response, served_model),- )- return _wrap_response(- target_format,- self.translate_response(source, target_format, _response_body(response)),- )-- def response_for_request(- self,- request: ChatRequest,- response: ChatResponse,- *,- served_model: str | None = None,- ) -> TranslatedResponse:- """Translate a backend response to the original client's wire format."""- if _is_streaming_response(response):- return self.stream_for_request(request, response, served_model=served_model)- source = _response_format(response)- target = _request_format(request)- if source == target:- return cast("TranslatedResponse", _response_body(response))- return cast(- "TranslatedResponse",- self.translate_response(source, target, _response_body(response)),- )-- def stream_for_request(- self,- request: ChatRequest,- response: ChatResponse,- *,- served_model: str | None = None,- ) -> TranslatedStream:- """Translate a backend stream to the original client's stream contract."""- source = _response_format(response)- target = _request_format(request)- if source == target:- return cast("TranslatedStream", _response_stream(response))- return cast(- "TranslatedStream",- self._translate_response_stream(source, target, response, served_model),- )-- async def translate(- self,- ctx: ProxyContext,- request: ChatRequest,- response: ChatResponse,- ) -> TranslatedResponse:- """Implement Switchyard's terminal TranslationEngine role.-- The served model comes from the routing context, never from a request- body: routers rewrite the model on their own copy of the request, so the- request reaching this role can still name the route the client addressed.- """- return self.response_for_request(request, response, served_model=ctx.selected_model)-- async def translate_stream(- self,- source: str | ChatRequestType,- target: str | ChatRequestType,- stream: AsyncIterable[Any],- *,- model: str | None = None,- message_id: str | None = None,- output: _StreamOutput = "objects",- ) -> AsyncGenerator[Any, None]:- """Translate an async stream between provider event formats.-- Closes *stream* on every exit path. When the client disconnects, the- ASGI server ``aclose()``-es the SSE generator consuming this one, which- raises ``GeneratorExit`` at the suspended ``yield``; without the- ``finally`` the upstream input stream (and the pooled connection it- holds) would never be released.- """- target_format = _format_name(target)- translator = self._inner.stream(- _format_name(source),- target_format,- model,- message_id,- )- try:- async for event in stream:- for payload in _stream_event_payloads(event):- for translated in translator.translate_event(payload):- yield _coerce_stream_output(target_format, translated, output)- for translated in translator.finish():- yield _coerce_stream_output(target_format, translated, output)- finally:- await _aclose_input_stream(stream)-- def _translate_response_stream(- self,- source: _NativeFormat,- target: _NativeFormat,- response: ChatResponse,- served_model: str | None,- ) -> AsyncGenerator[Any, None]:- return self.translate_stream(- source,- target,- _response_stream(response),- model=served_model,- output=_stream_wire_output_for_target(target),- )-- def normalize_anthropic_tool_use_ids(self, messages: object) -> object:- """Normalize Anthropic tool IDs without breaking result references."""- return self._inner.normalize_anthropic_tool_use_ids(_jsonable(messages, set()))---async def _aclose_input_stream(stream: object) -> None:- """Best-effort close of a translated stream's upstream input.-- ``ChatResponseStream`` and async generators expose ``aclose``; SDK- ``AsyncStream`` objects expose ``close``; either may be a coroutine.- Closing must never mask the control flow that triggered it, so failures- are logged and swallowed.- """- closer = getattr(stream, "aclose", None) or getattr(stream, "close", None)- if closer is None:- return- try:- result = closer()- if inspect.isawaitable(result):- await result- except Exception as exc:- _log.debug("Failed to close translated input stream: %s: %s", type(exc).__name__, exc)---def is_native_translation_available() -> bool:- """Return whether the required native translation extension loaded."""- _load_native()- return True---def _format_name(value: str | ChatRequestType) -> _NativeFormat:- raw = value.value if hasattr(value, "value") else str(value)- if raw == "anthropic":- raw = "anthropic_messages"- if raw in {"openai_chat", "openai_responses", "anthropic_messages"}:- return cast(_NativeFormat, raw)- raise ValueError(f"Unknown translation format: {value!r}")---def _request_format(request: ChatRequest) -> _NativeFormat:- from switchyard_rust.core import request_type_value-- try:- return _format_name(request_type_value(request.request_type))- except (AttributeError, ValueError) as exc:- raise NotImplementedError(- f"Request translation not implemented for {type(request).__name__}"- ) from exc---def _response_format(response: ChatResponse) -> _NativeFormat:- from switchyard_rust.core import response_type_value-- response_type = response_type_value(response.response_type)- if response_type in {"openai_completion", "openai_stream"}:- return "openai_chat"- if response_type in {"openai_responses_completion", "openai_responses_stream"}:- return "openai_responses"- if response_type in {"anthropic_completion", "anthropic_stream"}:- return "anthropic_messages"- raise NotImplementedError(- f"Response translation not implemented for {type(response).__name__}"- )---def _wrap_request(format_name: _NativeFormat, body: dict[str, Any]) -> ChatRequest:- from switchyard_rust.core import request_with_type-- if format_name == "openai_chat":- return request_with_type("openai_chat", body)- if format_name == "openai_responses":- return request_with_type("openai_responses", body)- if format_name == "anthropic_messages":- return request_with_type("anthropic", body)- raise ValueError(f"Unknown request format: {format_name!r}")---def _wrap_response(format_name: _NativeFormat, body: dict[str, Any]) -> ChatResponse:- from switchyard_rust.core import ChatResponse-- if format_name == "openai_chat":- return ChatResponse.openai_completion(body)- if format_name == "openai_responses":- return ChatResponse.openai_responses_completion(body)- if format_name == "anthropic_messages":- return ChatResponse.anthropic_completion(body)- raise ValueError(f"Unknown response format: {format_name!r}")---def _wrap_streaming_response(- format_name: _NativeFormat,- stream: AsyncIterable[Any],-) -> ChatResponse:- from switchyard.lib.chat_response.anthropic import AnthropicResponseStream- from switchyard.lib.chat_response.openai_chat import ResponseStream- from switchyard.lib.chat_response.openai_responses import ResponsesApiStream- from switchyard_rust.core import ChatResponse-- if format_name == "openai_chat":- return ChatResponse.openai_stream(ResponseStream(cast(Any, stream)))- if format_name == "openai_responses":- return ChatResponse.openai_responses_stream(ResponsesApiStream(cast(Any, stream)))- if format_name == "anthropic_messages":- return ChatResponse.anthropic_stream(AnthropicResponseStream(cast(Any, stream)))- raise ValueError(f"Unknown streaming response format: {format_name!r}")---def _stream_wire_output_for_target(target: _NativeFormat) -> _StreamOutput:- if target == "openai_chat":- return "chat_chunks"- if target == "openai_responses":- return "responses_sse"- return "objects"---def _response_body(response: ChatResponse) -> Any:- return response.body---def _response_stream(response: ChatResponse) -> AsyncIterable[Any]:- return cast(AsyncIterable[Any], response.stream)---def _is_streaming_response(response: ChatResponse) -> bool:- from switchyard_rust.core import response_is_streaming-- return response_is_streaming(response)---def _coerce_stream_output(- target: _NativeFormat,- event: Mapping[str, Any],- output: _StreamOutput,-) -> Any:- if output == "responses_sse":- return _sse_frame(event)- if output == "chat_chunks":- return _validate_or_construct(ChatCompletionChunk, dict(event))- _ = target- return dict(event)---def _validate_or_construct(model_cls: type[_T], body: dict[str, Any]) -> _T:- validator = getattr(model_cls, "model_validate", None)- if callable(validator):- try:- return cast(_T, validator(body))- except Exception:- pass- constructor = getattr(model_cls, "model_construct", None)- if callable(constructor):- return cast(_T, constructor(**body))- return model_cls(**body)---def _sse_frame(event: Mapping[str, Any]) -> str:- event_type = event.get("type", "message")- return f"event: {event_type}\ndata: {json.dumps(dict(event))}\n\n"---def _jsonable_mapping(value: Any) -> dict[str, Any]:- value = _jsonable(value, set())- return dict(value) if isinstance(value, Mapping) else {}---def _stream_event_payloads(event: Any) -> list[dict[str, Any]]:- """JSON payload(s) of one stream item, for cross-format translation.-- Backends that preserve wire fidelity yield raw SSE frame *strings*- (see ``RawSSEFrameStream``); those are parsed here so the translator- still sees event dicts. Every other item shape keeps the existing- jsonable-mapping coercion.- """- if isinstance(event, str):- return sse_frame_payloads(event)- return [_jsonable_mapping(event)]---def sse_frame_payloads(frame: str) -> list[dict[str, Any]]:- """Parse an SSE frame string into its JSON ``data:`` payload(s).-- Follows the SSE contract: a frame's ``data:`` lines are joined with- newlines to form one payload; an optional single leading space after the- colon is stripped. Comment/keep-alive frames, the ``[DONE]`` sentinel,- and non-JSON or non-object payloads yield nothing. Accepts a string- containing multiple ``\\n\\n``-separated frames and returns payloads in- order.- """- payloads: list[dict[str, Any]] = []- for block in frame.split("\n\n"):- data_lines: list[str] = []- for line in block.split("\n"):- if line.startswith("data:"):- value = line[5:]- data_lines.append(value[1:] if value.startswith(" ") else value)- if not data_lines:- continue- data = "\n".join(data_lines)- if data.strip() == "[DONE]":- continue- try:- parsed = json.loads(data)- except ValueError:- continue- if isinstance(parsed, dict):- payloads.append(parsed)- return payloads---def _jsonable(value: Any, seen: set[int]) -> Any:- if hasattr(value, "model_dump"):- try:- return _jsonable(value.model_dump(exclude_none=True), seen)- except TypeError:- return _jsonable(value.model_dump(), seen)- if hasattr(value, "to_dict"):- return _jsonable(value.to_dict(), seen)- if dataclasses.is_dataclass(value) and not isinstance(value, type):- return _jsonable(dataclasses.asdict(value), seen)- if isinstance(value, Mapping):- obj_id = id(value)- if obj_id in seen:- return str(value)- seen.add(obj_id)- try:- if any(id(item) in seen for item in value.values()):- return str(value)- return {str(key): _jsonable(item, seen) for key, item in value.items()}- finally:- seen.remove(obj_id)- if isinstance(value, (list, tuple)):- obj_id = id(value)- if obj_id in seen:- return str(value)- seen.add(obj_id)- try:- if any(id(item) in seen for item in value):- return str(value)- return [_jsonable(item, seen) for item in value]- finally:- seen.remove(obj_id)- if isinstance(value, Iterable) and not isinstance(value, (str, bytes, bytearray)):- return [_jsonable(item, seen) for item in value]- if hasattr(value, "__dict__") and not isinstance(value, type):- obj_id = id(value)- if obj_id in seen:- return str(value)- seen.add(obj_id)- try:- return {- key: _jsonable(item, seen)- for key, item in vars(value).items()- if not key.startswith("_")- }- finally:- seen.remove(obj_id)- return value---__all__ = [- "TranslationEngine",- "is_native_translation_available",-]diff --git a/tests/_chain_test_helpers.py b/tests/_chain_test_helpers.pydeleted file mode 100644index 29f290999..000000000--- a/tests/_chain_test_helpers.py+++ /dev/null@@ -1,234 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Shared test infrastructure for classifier chain tests.--Private (underscore-prefixed) module so pytest doesn't try to collect-it as a test file. Symbols inside also keep the underscore-private-convention used by the test files that import them.--The owner is :mod:`tests.test_llm_classifier_e2e`, which exercises the-classifier-only routing chain.-"""--from __future__ import annotations--import json-import threading-from dataclasses import dataclass-from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer-from typing import cast--import httpx--# ----------------------------------------------------------------------------# URL + model constants shared between classifier and chain harnesses.-# -----------------------------------------------------------------------------#: respx-intercepted classifier endpoint. Distinct from the backend URL so an-#: unexpected hit is caught by missing-mock errors.-_CLASSIFIER_BASE = "https://classifier.test/v1"-_CLASSIFIER_URL = f"{_CLASSIFIER_BASE}/chat/completions"--_CLASSIFIER_MODEL = "router-classifier-llm"--#: Tier model IDs the deterministic backend exposes to its OpenAI-compatible-#: stubs. The two-tier (simple / complex) setup is enough to validate-#: classifier-driven dispatch without proliferating tiers.-_SIMPLE_MODEL = "tier/simple-model"-_COMPLEX_MODEL = "tier/complex-model"---# ----------------------------------------------------------------------------# Classifier payloads (shared between classifier-only and chain tests).-# ------------------------------------------------------------------------------def _classifier_payload(content: str) -> dict[str, object]:- """An OpenAI Chat Completion JSON body whose content is the classifier output."""- return {- "id": "chatcmpl-classifier",- "object": "chat.completion",- "created": 1700000000,- "model": _CLASSIFIER_MODEL,- "choices": [- {- "index": 0,- "message": {"role": "assistant", "content": content},- "finish_reason": "stop",- }- ],- "usage": {"prompt_tokens": 5, "completion_tokens": 10, "total_tokens": 15},- }---def _signals_json(**overrides: object) -> str:- payload: dict[str, object] = {- "task_type": "debugging",- "complexity": "complex",- "reasoning_depth": "multi_step",- "tool_planning_required": False,- "precision_requirement": "high",- "context_dependency": "conversation",- "structured_output_risk": "low",- "recommended_tier": "complex",- "confidence": 0.88,- "reason_code": "debugging",- "abstain": False,- }- payload.update(overrides)- return json.dumps(payload)---# ----------------------------------------------------------------------------# Generic backend stub bodies (tier-agnostic).-# ------------------------------------------------------------------------------def _backend_payload(*, content: str, model: str) -> dict[str, object]:- return {- "id": "chatcmpl-backend",- "object": "chat.completion",- "created": 1700000001,- "model": model,- "choices": [- {- "index": 0,- "message": {"role": "assistant", "content": content},- "finish_reason": "stop",- }- ],- "usage": {"prompt_tokens": 8, "completion_tokens": 4, "total_tokens": 12},- }---def _stream_chunk(*, content: str = "", finish: str | None = None) -> dict[str, object]:- delta: dict[str, object] = {}- if content:- delta["content"] = content- return {- "id": "chatcmpl-backend-stream",- "object": "chat.completion.chunk",- "created": 1700000002,- "model": _COMPLEX_MODEL,- "choices": [{"index": 0, "delta": delta, "finish_reason": finish}],- }---def _sse_body(chunks: list[dict[str, object]]) -> bytes:- return (- "".join(f"data: {json.dumps(c)}\n\n" for c in chunks) + "data: [DONE]\n\n"- ).encode("utf-8")---def _last_body(stub: _OpenAICompatStub) -> dict[str, object]:- return cast(dict[str, object], stub.requests[-1]["body"])---# ----------------------------------------------------------------------------# OpenAI-compatible loopback stub. Used for tier backends because Rust-# reqwest bypasses respx; a real local HTTP server lets us assert on the-# exact body the backend emitted.-# ------------------------------------------------------------------------------class _OpenAICompatStub:- def __init__(self) -> None:- self._server: ThreadingHTTPServer | None = None- self._thread: threading.Thread | None = None- self._lock = threading.Lock()- self._requests: list[dict[str, object]] = []- self._responses: list[tuple[int, bytes, str]] = []-- def __enter__(self) -> _OpenAICompatStub:- owner = self-- class Handler(BaseHTTPRequestHandler):- protocol_version = "HTTP/1.1"-- def do_POST(self) -> None:- """Record the request (path, body, headers) and pop the queued reply."""- length = int(self.headers.get("content-length", "0"))- raw = self.rfile.read(length)- body = json.loads(raw.decode("utf-8"))- with owner._lock:- # Header names lower-cased so tests can look them up- # without caring how the client cased them on the wire.- owner._requests.append({- "path": self.path,- "body": body,- "headers": {k.lower(): v for k, v in self.headers.items()},- })- if owner._responses:- status, content, content_type = owner._responses.pop(0)- else:- status = 500- content = b'{"error":{"message":"no stub response queued"}}'- content_type = "application/json"-- self.send_response(status)- self.send_header("content-type", content_type)- self.send_header("content-length", str(len(content)))- self.send_header("connection", "close")- self.end_headers()- self.wfile.write(content)-- def log_message(self, _format: str, *args: object) -> None:- return None-- self._server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)- # ``shutdown()`` blocks up to serve_forever's poll interval; the 0.5s- # default adds half a second of idle teardown to every test using the- # stub, so poll frequently.- server = self._server- self._thread = threading.Thread(- target=lambda: server.serve_forever(poll_interval=0.05), daemon=True- )- self._thread.start()- return self-- def __exit__(self, *args: object) -> None:- if self._server is not None:- self._server.shutdown()- self._server.server_close()- if self._thread is not None:- self._thread.join(timeout=2)-- @property- def base_url(self) -> str:- if self._server is None:- raise RuntimeError("stub server is not running")- host, port = self._server.server_address- return f"http://{host}:{port}/v1"-- @property- def requests(self) -> list[dict[str, object]]:- with self._lock:- return list(self._requests)-- @property- def called(self) -> bool:- return bool(self.requests)-- def respond_json(self, body: dict[str, object], *, status: int = 200) -> None:- content = json.dumps(body).encode("utf-8")- with self._lock:- self._responses.append((status, content, "application/json"))-- def respond_sse(self, body: bytes) -> None:- with self._lock:- self._responses.append((200, body, "text/event-stream"))---# ----------------------------------------------------------------------------# Harness dataclass returned by chain-test fixtures.-# ------------------------------------------------------------------------------@dataclass(frozen=True)-class _ClassifierHarness:- """ASGI-driven client + per-tier loopback stubs for chain composition tests."""-- client: httpx.AsyncClient- simple: _OpenAICompatStub- complex: _OpenAICompatStubdiff --git a/tests/conftest.py b/tests/conftest.pydeleted file mode 100644index 324b7cca9..000000000--- a/tests/conftest.py+++ /dev/null@@ -1,23 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Shared pytest fixtures."""--from __future__ import annotations--import pytest--[email protected](autouse=True)-def _stub_anthropic_messages_probe(monkeypatch: pytest.MonkeyPatch) -> None:- """Keep unit tests hermetic.-- ``format=auto`` resolution probes ``/v1/messages`` over the network at- backend-build time, so presets that default a Claude tier to ``auto`` would- otherwise make live calls from unit tests. Stub it to a no-network default;- tests that exercise the probe set their own value, which overrides this.- """- monkeypatch.setattr(- "switchyard.lib.backends.backend_format_resolver.probe_anthropic_messages_support_sync",- lambda **_kwargs: False,- )diff --git a/tests/contract/__init__.py b/tests/contract/__init__.pydeleted file mode 100644index 0260e26e2..000000000--- a/tests/contract/__init__.py+++ /dev/null@@ -1,14 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Contract tests for the API surface that downstream consumers (e.g. the-NeMo Platform `nemo-switchyard` middleware plugin) depend on.--These tests intentionally restate import paths and class shapes verbatim so a-PR that renames, deletes, or refactors a symbol fails *here* with a clear-message instead of breaking downstream at integration time.--If a test in this suite fails because you intentionally changed the contract,-that's the signal to coordinate the migration with downstream consumers-*before* merging — don't silently update the test to match the new shape.-"""diff --git a/tests/contract/test_platform_imports.py b/tests/contract/test_platform_imports.pydeleted file mode 100644index d44081ee9..000000000--- a/tests/contract/test_platform_imports.py+++ /dev/null@@ -1,68 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Import-surface contract: every symbol the NeMo Platform `nemo-switchyard`-plugin imports from switchyard must resolve.--A failure here means a downstream IGW middleware plugin will hit-``ModuleNotFoundError`` or ``ImportError`` at process startup. The single-source of truth for this list is the platform plugin source tree at-``plugins/nemo-switchyard/src/nemo_switchyard/`` in the NVIDIA-NeMo/Platform-repo — keep these two in lockstep when adding or removing imports there.-"""--from __future__ import annotations--import importlib--import pytest--# (module path, [attribute names that must exist on the module])-#-# Listed by source file in the downstream plugin so reviewers can grep the-# Platform side easily:-# _format.py — chat_request.{anthropic,base,openai_chat,openai_responses}-# _bridge.py — chat_request.base, chat_response.*, proxy_context.ProxyContext-# _processors.py — proxy_context.CTX_TARGET_FORMAT-# middleware.py — proxy_context-PLATFORM_IMPORT_SURFACE: list[tuple[str, list[str]]] = [- ("switchyard.lib.chat_request.anthropic", ["AnthropicChatRequest"]),- ("switchyard.lib.chat_request.base", ["ChatRequest"]),- ("switchyard.lib.chat_request.openai_chat", ["OpenAIChatRequest"]),- ("switchyard.lib.chat_request.openai_responses", ["ResponsesChatRequest"]),- (- "switchyard.lib.chat_response.anthropic",- ["AnthropicChatResponse", "AnthropicStreamingChatResponse", "AnthropicResponseStream"],- ),- ("switchyard.lib.chat_response.base", ["ChatResponse"]),- (- "switchyard.lib.chat_response.openai_chat",- ["CompletionChatResponse", "StreamingChatResponse", "ResponseStream"],- ),- (- "switchyard.lib.chat_response.openai_responses",- ["ResponsesApiChatResponse", "ResponsesApiStreamingChatResponse", "ResponsesApiStream"],- ),- ("switchyard.lib.proxy_context", ["CTX_TARGET_FORMAT", "ProxyContext"]),- ("switchyard.lib.processors.format_translate", ["TranslateConfig"]),-]--- ("module_path", "attr"),- [(mod, attr) for mod, attrs in PLATFORM_IMPORT_SURFACE for attr in attrs],-)-def test_platform_import_resolves(module_path: str, attr: str) -> None:- """Each symbol the Platform plugin imports must resolve from upstream switchyard.-- Failure mode: a switchyard PR that deletes ``module_path`` or renames- ``attr`` ships a broken contract. Downstream IGW startup will fail with- ``ModuleNotFoundError`` or ``ImportError`` at the plugin entry-point load- step.- """- module = importlib.import_module(module_path)- assert hasattr(module, attr), (- f"Platform's nemo-switchyard plugin imports `{attr}` from `{module_path}`, "- f"but it is no longer exported. Coordinate the migration with the Platform team "- f"before merging."- )diff --git a/tests/contract/test_proxy_context.py b/tests/contract/test_proxy_context.pydeleted file mode 100644index 57db68400..000000000--- a/tests/contract/test_proxy_context.py+++ /dev/null@@ -1,67 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""ProxyContext + metadata-key constant contract.--Platform reads ``CTX_TARGET_FORMAT`` and writes ``CTX_ORIGINAL_FORMAT`` into-``ProxyContext.metadata`` to bridge state across IGW request/response hooks-(see ``plugins/nemo-switchyard/src/nemo_switchyard/_processors.py`` and-``middleware.py`` in the Platform repo).--If the constant is renamed or its sentinel value changes, request- and-response-side translate stops sharing state — the user sees broken responses-at runtime with no obvious error.-"""--from __future__ import annotations--import pytest--from switchyard.lib.proxy_context import CTX_TARGET_FORMAT, ProxyContext---def test_ctx_target_format_is_a_stable_key() -> None:- """``CTX_TARGET_FORMAT`` must be a hashable value usable as a dict key.-- Platform stores it verbatim under ``ctx.metadata[CTX_TARGET_FORMAT]`` and- reads it back later. Any change here (string rename, enum migration) breaks- cross-phase metadata bridging silently.- """- # Must be hashable (used as dict key)- probe = {CTX_TARGET_FORMAT: "sentinel"}- assert probe[CTX_TARGET_FORMAT] == "sentinel"- # Must not be None (Platform branches on key-present-vs-missing)- assert CTX_TARGET_FORMAT is not None---def test_proxy_context_metadata_round_trip() -> None:- """ProxyContext.metadata must accept arbitrary keys + survive round-trip.-- Platform's _bridge.py copies metadata between an IGW context and a- switchyard ProxyContext on each phase. A mapping that drops unknown keys or- enforces a typed schema breaks Platform.- """- ctx = ProxyContext()- ctx.metadata[CTX_TARGET_FORMAT] = "openai_chat"- ctx.metadata["arbitrary_string_key"] = {"nested": "value"}- assert ctx.metadata[CTX_TARGET_FORMAT] == "openai_chat"- assert ctx.metadata["arbitrary_string_key"] == {"nested": "value"}---def test_proxy_context_is_constructible_with_no_args() -> None:- """Platform's bridge constructs ``ProxyContext()`` with no positional args- when building a side-pipeline. Required positional/kwargs is a breaking- change."""- ctx = ProxyContext()- assert ctx is not None- assert hasattr(ctx, "metadata"), "ProxyContext must expose `.metadata` mapping"--- "attr",- ["metadata"],-)-def test_proxy_context_exposes_required_attrs(attr: str) -> None:- """Attribute presence — exercised independently of constructor args."""- ctx = ProxyContext()- assert hasattr(ctx, attr), f"ProxyContext.{attr} is required by Platform but is missing"diff --git a/tests/contract/test_request_response_types.py b/tests/contract/test_request_response_types.pydeleted file mode 100644index 3d6b7bb40..000000000--- a/tests/contract/test_request_response_types.py+++ /dev/null@@ -1,121 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Construction + dispatch contract for Rust-backed chat request aliases.--The Python concrete request subclasses were collapsed into one Rust-backed-``ChatRequest`` type. The legacy provider-specific names remain importable as-aliases only; provider dispatch is keyed by ``request_type``.-"""--from __future__ import annotations--from collections.abc import Callable-from typing import TypeAlias--import pytest--from switchyard.lib.chat_request.anthropic import AnthropicChatRequest-from switchyard.lib.chat_request.base import ChatRequest-from switchyard.lib.chat_request.openai_chat import OpenAIChatRequest-from switchyard.lib.chat_request.openai_responses import ResponsesChatRequest-from switchyard_rust.core import ChatRequestType, request_type_matches--RequestAlias: TypeAlias = type[ChatRequest]-RequestFactory: TypeAlias = Callable[[dict], ChatRequest]--OPENAI_CHAT_BODY = {- "model": "gpt-4",- "messages": [{"role": "user", "content": "ping"}],-}--OPENAI_RESPONSES_BODY = {- "model": "gpt-4",- "input": "ping",-}--ANTHROPIC_BODY = {- "model": "claude-3-haiku",- "messages": [{"role": "user", "content": "ping"}],- "max_tokens": 16,-}---REQUEST_CASES: list[tuple[str, RequestAlias, RequestFactory, object, dict]] = [- (- "OpenAIChatRequest",- OpenAIChatRequest,- ChatRequest.openai_chat,- ChatRequestType.OPENAI_CHAT,- OPENAI_CHAT_BODY,- ),- (- "ResponsesChatRequest",- ResponsesChatRequest,- ChatRequest.openai_responses,- ChatRequestType.OPENAI_RESPONSES,- OPENAI_RESPONSES_BODY,- ),- (- "AnthropicChatRequest",- AnthropicChatRequest,- ChatRequest.anthropic,- ChatRequestType.ANTHROPIC,- ANTHROPIC_BODY,- ),-]--- ("alias_name", "alias", "_factory", "_request_type", "_body"),- REQUEST_CASES,-)-def test_request_legacy_names_are_aliases(- alias_name: str,- alias: RequestAlias,- _factory: RequestFactory,- _request_type: object,- _body: dict,-) -> None:- """Provider-specific request names remain aliases for the Rust type."""- assert alias is ChatRequest, f"{alias_name} must alias ChatRequest"--- ("_alias_name", "_alias", "factory", "_request_type", "body"),- REQUEST_CASES,-)-def test_request_constructible_with_factory(- _alias_name: str,- _alias: RequestAlias,- factory: RequestFactory,- _request_type: object,- body: dict,-) -> None:- """Each request factory must round-trip the raw body.-- Platform's bridge layer constructs requests from raw dict payloads coming- off the IGW wire. The Rust-backed type exposes provider factories instead- of the deleted ``cls(body=...)`` Python subclass constructors.- """- req = factory(body)- assert req.body == body, "ChatRequest did not round-trip body"--- ("alias_name", "alias", "factory", "request_type", "body"),- REQUEST_CASES,-)-def test_request_type_dispatch_works(- alias_name: str,- alias: RequestAlias,- factory: RequestFactory,- request_type: object,- body: dict,-) -> None:- """The alias-backed request must expose the Rust request type for dispatch."""- req = factory(body)-- assert isinstance(req, alias), f"isinstance({alias_name}, ChatRequest alias) is false"- assert isinstance(req, ChatRequest), f"{alias_name} is not a ChatRequest alias"- assert request_type_matches(req, request_type), f"{alias_name} request_type is wrong"diff --git a/tests/e2e/_helpers.py b/tests/e2e/_helpers.pydeleted file mode 100644index 3495d1305..000000000--- a/tests/e2e/_helpers.py+++ /dev/null@@ -1,103 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Shared helpers for passthrough e2e tests.--Hosts the three vendor-specific ``get_weather`` tool definitions and-the tool-capable-model resolver used by the Chat Completions,-Responses, and Anthropic Messages e2e suites. Pure Python — no-fixtures, so we keep this as a regular module rather than a-``conftest.py``.-"""--from __future__ import annotations--import os---def resolve_tool_capable_model(default_model: str) -> str:- """Pick a backend model that supports OpenAI-style tool calling.-- Some NVIDIA-hosted vLLM deployments (notably the- ``nvidia/qwen/qwen3.5-*`` family) aren't launched with- ``--enable-auto-tool-choice`` / ``--tool-call-parser`` and reject- ``tool_choice`` with HTTP 400 — the passthrough chain itself is- fine, the backend just won't parse tool calls. Tool-call tests- should therefore run against a known tool-capable model so we're- validating the passthrough's tool wiring rather than the backend's- vLLM flags.-- Resolution order:-- 1. ``OPENROUTER_TOOL_MODEL`` / ``NVIDIA_TOOL_MODEL`` env var —- explicit override for any backend or any model- 2. ``openai/openai/gpt-5.2`` when ``default_model`` starts with- ``nvidia/`` — that's the vLLM-hosted family described above,- and ``openai/openai/gpt-5.2`` is known to support tool calling- on ``inference-api.nvidia.com``- 3. ``default_model`` otherwise — so non-NVIDIA backends keep- whatever the test suite was configured with- """- override = os.environ.get("OPENROUTER_TOOL_MODEL") or os.environ.get("NVIDIA_TOOL_MODEL")- if override:- return override- if default_model.startswith("nvidia/"):- return "openai/openai/gpt-5.2"- return default_model---# ----------------------------------------------------------------------------# Tool definitions — same semantic tool (``get_weather(city)``) rendered-# in each vendor's wire format, because each API has subtly different-# expectations:-#-# * Chat Completions nests the function metadata under ``function``.-# * Responses API flattens it out alongside ``type=function``.-# * Anthropic uses ``input_schema`` instead of ``parameters`` and has-# no ``type`` wrapper at all.-# ------------------------------------------------------------------------------_WEATHER_SCHEMA: dict = {- "type": "object",- "properties": {- "city": {- "type": "string",- "description": "The city name, e.g. 'Tokyo'.",- },- },- "required": ["city"],-}--_WEATHER_NAME = "get_weather"-_WEATHER_DESC = "Get the current weather for a city."---# Chat Completions format: ``{"type": "function", "function": {...}}``.-CHAT_COMPLETIONS_WEATHER_TOOL: dict = {- "type": "function",- "function": {- "name": _WEATHER_NAME,- "description": _WEATHER_DESC,- "parameters": _WEATHER_SCHEMA,- },-}---# Responses API format: flat — ``type``, ``name``, ``description``,-# ``parameters`` all at the top level.-RESPONSES_WEATHER_TOOL: dict = {- "type": "function",- "name": _WEATHER_NAME,- "description": _WEATHER_DESC,- "parameters": _WEATHER_SCHEMA,-}---# Anthropic Messages format: no ``type``, ``input_schema`` instead of-# ``parameters``.-ANTHROPIC_WEATHER_TOOL: dict = {- "name": _WEATHER_NAME,- "description": _WEATHER_DESC,- "input_schema": _WEATHER_SCHEMA,-}diff --git a/tests/e2e/conftest.py b/tests/e2e/conftest.pydeleted file mode 100644index 5ae9b4883..000000000--- a/tests/e2e/conftest.py+++ /dev/null@@ -1,249 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Pytest fixtures for switchyard end-to-end production tests.--These tests hit a real OpenAI-compatible backend (OpenRouter by default,-but any compatible URL works via env vars) through a subprocess-launched-``switchyard`` CLI. They're gated on an API key env var being set; without-one every test in this directory skips.--Configuration (env vars, in resolution order):--* ``OPENROUTER_API_KEY`` / ``NVIDIA_API_KEY`` — required; the test suite- skips without one-* ``OPENROUTER_BASE_URL`` / ``NVIDIA_BASE_URL`` — defaults to the selected- provider's OpenAI-compatible base URL-* ``OPENROUTER_MODEL`` / ``NVIDIA_MODEL`` — defaults to the selected provider's- GPT-5.2 model id--Run with::-- OPENROUTER_API_KEY=sk-or-... uv run pytest tests/e2e/ -v-"""--from __future__ import annotations--import logging-import os-import socket-import subprocess-import sys-import time-from collections.abc import Generator-from pathlib import Path--import pytest--logging.basicConfig(- level=logging.INFO,- format="%(asctime)s [%(levelname)s] %(message)s",- datefmt="%H:%M:%S",- stream=sys.stdout,-)-logger = logging.getLogger("e2e")--REPO_ROOT = Path(__file__).parent.parent.parent--SERVER_STARTUP_TIMEOUT = 60.0---def get_nvidia_config() -> dict:- """Resolve backend configuration from env vars.-- Returns a dict with ``api_key``, ``base_url``, and ``model``.- ``api_key`` is ``None`` when nothing's set — fixtures that depend- on it call ``pytest.skip``.- """- openrouter_key = os.environ.get("OPENROUTER_API_KEY")- if openrouter_key:- return {- "provider": "openrouter",- "api_key": openrouter_key,- "base_url": (- os.environ.get("OPENROUTER_BASE_URL")- or "https://openrouter.ai/api/v1"- ),- "model": os.environ.get("OPENROUTER_MODEL") or "openai/gpt-5.2",- }-- nvidia_key = os.environ.get("NVIDIA_API_KEY")- if nvidia_key:- return {- "provider": "nvidia",- "api_key": nvidia_key,- "base_url": (- os.environ.get("NVIDIA_BASE_URL")- or "https://inference-api.nvidia.com/v1"- ),- "model": os.environ.get("NVIDIA_MODEL") or "openai/openai/gpt-5.2",- }-- return {- "provider": "openrouter",- "api_key": None,- "base_url": (- os.environ.get("OPENROUTER_BASE_URL")- or "https://openrouter.ai/api/v1"- ),- "model": os.environ.get("OPENROUTER_MODEL") or "openai/gpt-5.2",- }---def find_free_port() -> int:- """Bind a socket to port 0 to claim a free port, then close it."""- with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:- s.bind(("127.0.0.1", 0))- s.listen(1)- return s.getsockname()[1]---def wait_for_server(port: int, timeout: float = 30.0, server_name: str = "server") -> bool:- """Poll ``127.0.0.1:port`` until it accepts TCP connections, or timeout."""- start_time = time.time()- last_log_time = start_time- attempt = 0-- while time.time() - start_time < timeout:- attempt += 1- elapsed = time.time() - start_time- try:- with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:- s.settimeout(1.0)- s.connect(("127.0.0.1", port))- logger.info(f" {server_name} ready after {elapsed:.1f}s (attempt {attempt})")- return True- except (TimeoutError, ConnectionRefusedError, OSError):- if time.time() - last_log_time >= 5.0:- logger.info(- f" Still waiting for {server_name}... "- f"({elapsed:.1f}s elapsed, attempt {attempt})"- )- last_log_time = time.time()- time.sleep(0.5)-- logger.warning(f" {server_name} failed to start after {timeout:.1f}s ({attempt} attempts)")- return False---def stop_server_subprocess(proc: subprocess.Popen, kill_timeout: float = 5.0) -> None:- """Gracefully terminate a subprocess, force-killing if it hangs."""- if proc.poll() is not None:- return- proc.terminate()- try:- proc.wait(timeout=kill_timeout)- except subprocess.TimeoutExpired:- logger.warning("[Subprocess] Did not stop gracefully, killing...")- proc.kill()- proc.wait()--[email protected](scope="session")-def nvidia_config() -> dict:- """Backend configuration shared across the e2e suite.-- Skips the entire dependent test if no backend API key is set —- we don't want silent successes from a no-op backend.- """- config = get_nvidia_config()- if not config["api_key"]:- pytest.skip("OPENROUTER_API_KEY or NVIDIA_API_KEY not set — required for e2e tests")- return config---def _start_passthrough_server(- port: int,- api_key: str,- base_url: str,-) -> subprocess.Popen:- """Launch ``switchyard passthrough`` as a subprocess.-- Invoked via ``python -m switchyard.cli.switchyard_cli`` (not- ``.venv/bin/switchyard``) so the tests are independent of whether- the editable-install script shim has been regenerated since the- last package install.-- Uses ``--inbound both`` so the one server exposes all three- inbound formats (``/v1/chat/completions``, ``/v1/responses``,- ``/v1/messages``) simultaneously — the Chat Completions, Responses,- and Anthropic Messages tests share this one process.- """- cmd = [- sys.executable,- "-m", "switchyard.cli.switchyard_cli",- "passthrough",- "--host", "127.0.0.1",- "--port", str(port),- "--inbound", "both",- "--api-key", api_key,- "--base-url", base_url,- ]-- env = os.environ.copy()- env.setdefault("OPENAI_API_KEY", api_key)-- return subprocess.Popen(- cmd,- stdout=subprocess.PIPE,- stderr=subprocess.PIPE,- cwd=str(REPO_ROOT),- env=env,- )--[email protected](scope="session")-def passthrough_server(nvidia_config: dict) -> Generator[dict, None, None]:- """Start a real switchyard passthrough server on a free port.-- Session-scoped: every e2e file (Chat Completions, Responses,- Anthropic Messages) shares one running server, saving subprocess- startup overhead.-- Yields a dict with ``process``, ``port``, ``base_url``, ``model``.- """- port = find_free_port()-- logger.info("")- logger.info(f"[Passthrough] {'=' * 60}")- logger.info("[Passthrough] Starting switchyard passthrough server")- logger.info(f"[Passthrough] Port: {port}")- logger.info(f"[Passthrough] Backend: {nvidia_config['base_url']}")- logger.info(f"[Passthrough] Model: {nvidia_config['model']}")- logger.info(f"[Passthrough] {'=' * 60}")-- proc = _start_passthrough_server(- port=port,- api_key=nvidia_config["api_key"],- base_url=nvidia_config["base_url"],- )-- server_ready = wait_for_server(- port,- timeout=SERVER_STARTUP_TIMEOUT,- server_name="Passthrough",- )- if not server_ready:- stop_server_subprocess(proc)- stdout = proc.stdout.read().decode() if proc.stdout else ""- stderr = proc.stderr.read().decode() if proc.stderr else ""- pytest.fail(- f"Passthrough server failed to start within "- f"{SERVER_STARTUP_TIMEOUT}s.\n"- f"stdout: {stdout[:2000]}\n"- f"stderr: {stderr[:2000]}"- )-- base_url = f"http://127.0.0.1:{port}"- logger.info(f"[Passthrough] Server ready at {base_url}")-- yield {- "process": proc,- "port": port,- "base_url": base_url,- "model": nvidia_config["model"],- }-- logger.info("[Passthrough] Shutting down server...")- stop_server_subprocess(proc)- logger.info("[Passthrough] Server stopped")diff --git a/tests/e2e/test_passthrough_e2e.py b/tests/e2e/test_passthrough_e2e.pydeleted file mode 100644index 39cbf9c89..000000000--- a/tests/e2e/test_passthrough_e2e.py+++ /dev/null@@ -1,552 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Production end-to-end smoke test for ``switchyard`` passthrough.--Exercises the real production path end-to-end::-- openai SDK → HTTP (127.0.0.1) → switchyard passthrough- → real backend → response--Uses the official ``openai`` Python SDK as the client so we're-validating the exact request shape a real customer integration would-produce, and we get SDK-side validation of the response wire shape-for free.--Unlike the in-process integration tests under ``tests/`` — which use-FastAPI's ``TestClient`` with a mocked backend — this test hits a real-OpenAI-compatible backend through the subprocess-launched-``switchyard`` CLI entry point.--The subprocess / server fixture lives in this package's-``conftest.py`` so the Responses and Anthropic sibling suites can share-the same running server.--Prerequisites:- - ``OPENROUTER_API_KEY`` or ``NVIDIA_API_KEY`` env var (skips otherwise)--Run with::-- OPENROUTER_API_KEY=sk-or-... pytest tests/e2e/test_passthrough_e2e.py -v-"""--from __future__ import annotations--import json-import logging-from collections.abc import Iterable-from typing import Any--import openai-import pytest--from ._helpers import (- CHAT_COMPLETIONS_WEATHER_TOOL,- resolve_tool_capable_model,-)--pytestmark = pytest.mark.integration--logger = logging.getLogger("e2e.passthrough")---# ----------------------------------------------------------------------------# Shared helpers-# ------------------------------------------------------------------------------def _make_openai_client(passthrough_server: dict) -> openai.OpenAI:- """Build an OpenAI SDK client pointed at the passthrough server.-- The passthrough server injects its own upstream ``api_key`` into- calls it makes to the real backend, so the ``api_key`` we give the- SDK here is just a non-empty placeholder the SDK needs for its- ``Authorization`` header (the passthrough inbound layer doesn't- validate it).- """- return openai.OpenAI(- base_url=f"{passthrough_server['base_url']}/v1",- api_key="not-used-passthrough-forwards-its-own",- timeout=60.0,- max_retries=0,- )---def _reasoning_text(delta_or_message: Any) -> str | None:- """Return ``reasoning`` / ``reasoning_content`` from a pydantic field.-- Both names are vendor extensions produced by reasoning models- (different providers use different spellings), so they live in the- pydantic ``model_extra`` bag rather than as typed attributes on- ``ChatCompletionMessage`` / ``ChoiceDelta``.- """- extra = getattr(delta_or_message, "model_extra", None) or {}- return extra.get("reasoning") or extra.get("reasoning_content")---def _collect_sdk_stream_deltas(- chunks: Iterable[Any],-) -> tuple[int, str, str, list[dict]]:- """Aggregate deltas from an OpenAI SDK ``ChatCompletionChunk`` stream.-- Reassembles fragmented fields across chunks using the OpenAI- streaming contract:-- * ``delta.content`` fragments → concatenated string- * ``delta.reasoning`` / ``delta.reasoning_content`` fragments (vendor- extensions for reasoning models) → concatenated string- * ``delta.tool_calls[*]`` fragments → rebuilt per-``index`` tool-call- dicts matching the non-streaming ``message.tool_calls`` shape- (``id``, ``type``, ``function.name``, ``function.arguments``)-- Returns ``(chunk_count, content, reasoning, tool_calls)``.- """- chunk_count = 0- content_parts: list[str] = []- reasoning_parts: list[str] = []- tool_calls_by_index: dict[int, dict] = {}-- for chunk in chunks:- chunk_count += 1- if not chunk.choices:- continue- delta = chunk.choices[0].delta-- if delta.content:- content_parts.append(delta.content)-- reasoning = _reasoning_text(delta)- if reasoning:- reasoning_parts.append(str(reasoning))-- for tc_delta in delta.tool_calls or []:- idx = tc_delta.index if tc_delta.index is not None else 0- entry = tool_calls_by_index.setdefault(- idx,- {- "id": "",- "type": "function",- "function": {"name": "", "arguments": ""},- },- )- if tc_delta.id:- entry["id"] = tc_delta.id- if tc_delta.function:- if tc_delta.function.name:- entry["function"]["name"] += tc_delta.function.name- if tc_delta.function.arguments:- entry["function"]["arguments"] += tc_delta.function.arguments-- tool_calls = [tool_calls_by_index[i] for i in sorted(tool_calls_by_index)]- return (- chunk_count,- "".join(content_parts),- "".join(reasoning_parts),- tool_calls,- )---# ----------------------------------------------------------------------------# Tests-# ------------------------------------------------------------------------------class TestPassthroughE2E:- """OpenAI SDK → Switchyard chain → real backend round-trip."""-- # ------------------------------------------------------------------- # Basic say-hi- # -------------------------------------------------------------------- def test_say_hi_get_response_back(self, passthrough_server: dict) -> None:- """``client.chat.completions.create("hi")`` round-trips successfully.-- Validates the minimum contract of the passthrough chain:-- * the SDK can reach the server and the request deserializes- cleanly into a ``ChatCompletion`` object- * the assistant role is returned- * non-empty assistant text comes back — in ``content`` for- non-reasoning models, or ``reasoning_content`` for reasoning- models that still had tokens left in their thinking phase-- ``max_tokens`` is deliberately generous to accommodate- *reasoning* models (e.g. Qwen 3.5 / GPT-5 series) that spend- the first several hundred tokens on internal reasoning before- emitting user-visible ``content``.- """- client = _make_openai_client(passthrough_server)- response = client.chat.completions.create(- model=passthrough_server["model"],- messages=[{"role": "user", "content": "hi"}],- max_tokens=2048,- )-- assert response.object == "chat.completion"- assert response.choices and len(response.choices) >= 1-- choice = response.choices[0]- message = choice.message- assert message.role == "assistant"-- content = message.content- reasoning = _reasoning_text(message)- assert content or reasoning, (- f"No assistant text in either 'content' or 'reasoning_content' "- f"(finish_reason={choice.finish_reason!r}): "- f"{response.model_dump()}"- )-- logger.info(f" [passthrough] finish_reason: {choice.finish_reason!r}")- if content:- logger.info(f" [passthrough] content[:100]: {content[:100]!r}")- if reasoning:- logger.info(f" [passthrough] reasoning[:100]: {reasoning[:100]!r}")-- def test_say_hi_streaming(self, passthrough_server: dict) -> None:- """``stream=True`` yields ``ChatCompletionChunk`` objects.-- Validates the streaming contract end-to-end: the SDK can- iterate the SSE stream (so the passthrough's envelope —- ``Content-Type: text/event-stream``, ``data: ...`` frames,- ``[DONE]`` terminator — must be well-formed), chunks- deserialize as ``chat.completion.chunk``, and at least one- delta carries visible text (``delta.content`` or vendor- ``delta.reasoning``).- """- client = _make_openai_client(passthrough_server)- stream = client.chat.completions.create(- model=passthrough_server["model"],- messages=[{"role": "user", "content": "hi"}],- max_tokens=2048,- stream=True,- )-- chunk_count, content, reasoning, _ = _collect_sdk_stream_deltas(stream)-- assert chunk_count >= 1, "expected at least one streamed chunk"- assert content or reasoning, (- f"no delta carried visible content or reasoning tokens "- f"across {chunk_count} chunks"- )-- logger.info(f" [passthrough] streamed {chunk_count} chunk frames + [DONE]")-- # ------------------------------------------------------------------- # System prompt — verifies a ``system`` role is forwarded through- # the chain to the backend and the response is well-formed.- # -------------------------------------------------------------------- def test_system_prompt_non_streaming(self, passthrough_server: dict) -> None:- """``system`` + ``user`` messages flow through and the backend honors them.-- Planted keyword ``pineapple`` is instructed via the system- prompt; when the model produces user-visible ``content`` we- assert the keyword is present, proving the system turn reached- the backend. For reasoning-only truncations (content empty,- all budget spent in ``reasoning_content``) we fall back to- asserting well-formedness — the chain still faithfully carried- the system turn even if the model didn't get to emit it.- """- client = _make_openai_client(passthrough_server)- response = client.chat.completions.create(- model=passthrough_server["model"],- messages=[- {- "role": "system",- "content": (- "You are a helpful assistant. Always include "- "the exact word 'pineapple' in your response."- ),- },- {"role": "user", "content": "Say hello."},- ],- max_tokens=2048,- )-- assert response.object == "chat.completion"- choice = response.choices[0]- message = choice.message- assert message.role == "assistant"-- content = message.content or ""- reasoning = _reasoning_text(message) or ""- assert content or reasoning, (- f"no assistant text: {response.model_dump()}"- )-- if content:- assert "pineapple" in content.lower(), (- f"system prompt keyword missing from content — "- f"system turn may not have been forwarded. "- f"content={content[:200]!r}"- )-- logger.info(f" [passthrough][system] finish_reason={choice.finish_reason!r}")- if content:- logger.info(f" [passthrough][system] content[:150]: {content[:150]!r}")-- def test_system_prompt_streaming(self, passthrough_server: dict) -> None:- """Same system-prompt round-trip, but over SSE via SDK streaming."""- client = _make_openai_client(passthrough_server)- stream = client.chat.completions.create(- model=passthrough_server["model"],- messages=[- {- "role": "system",- "content": (- "You are a helpful assistant. Always include "- "the exact word 'pineapple' in your response."- ),- },- {"role": "user", "content": "Say hello."},- ],- max_tokens=2048,- stream=True,- )-- chunk_count, content, reasoning, _ = _collect_sdk_stream_deltas(stream)- assert content or reasoning, (- f"no streamed assistant text across {chunk_count} chunks"- )-- if content:- assert "pineapple" in content.lower(), (- f"system prompt keyword missing from streamed content — "- f"system turn may not have been forwarded. "- f"content={content[:200]!r}"- )-- logger.info(- f" [passthrough][system-stream] {chunk_count} chunks, "- f"content[:100]={content[:100]!r}"- )-- # ------------------------------------------------------------------- # Multi-turn — verifies prior ``user`` + ``assistant`` turns are- # forwarded so the backend sees full conversation history.- # -------------------------------------------------------------------- def test_multi_turn_non_streaming(self, passthrough_server: dict) -> None:- """Prior conversation history flows through the chain.-- Plants the name ``Alice`` two turns back, then asks the model- to recall it. If the history reached the backend, the name- appears in the reply; if only the last user turn reached it,- the model has nothing to recall. Reasoning-only cut-offs are- forgiven (same convention as the system-prompt test above).- """- client = _make_openai_client(passthrough_server)- response = client.chat.completions.create(- model=passthrough_server["model"],- messages=[- {- "role": "user",- "content": "My name is Alice. Please remember this.",- },- {- "role": "assistant",- "content": "Got it, Alice. I will remember your name.",- },- {- "role": "user",- "content": (- "What is my name? Reply with only the name "- "itself, nothing else."- ),- },- ],- max_tokens=2048,- )-- assert response.object == "chat.completion"- choice = response.choices[0]- message = choice.message- assert message.role == "assistant"-- content = message.content or ""- reasoning = _reasoning_text(message) or ""- assert content or reasoning, (- f"no assistant text: {response.model_dump()}"- )-- if content:- assert "alice" in content.lower(), (- f"prior-turn context not recalled — multi-turn history "- f"may not have been forwarded. content={content[:200]!r}"- )-- logger.info(f" [passthrough][multi-turn] finish_reason={choice.finish_reason!r}")- if content:- logger.info(f" [passthrough][multi-turn] content[:150]: {content[:150]!r}")-- def test_multi_turn_streaming(self, passthrough_server: dict) -> None:- """Same multi-turn round-trip, but over SSE via SDK streaming."""- client = _make_openai_client(passthrough_server)- stream = client.chat.completions.create(- model=passthrough_server["model"],- messages=[- {- "role": "user",- "content": "My name is Alice. Please remember this.",- },- {- "role": "assistant",- "content": "Got it, Alice. I will remember your name.",- },- {- "role": "user",- "content": (- "What is my name? Reply with only the name "- "itself, nothing else."- ),- },- ],- max_tokens=2048,- stream=True,- )-- chunk_count, content, reasoning, _ = _collect_sdk_stream_deltas(stream)- assert content or reasoning, (- f"no streamed assistant text across {chunk_count} chunks"- )-- if content:- assert "alice" in content.lower(), (- f"prior-turn context not recalled over SSE — multi-turn "- f"history may not have been forwarded. "- f"content={content[:200]!r}"- )-- logger.info(- f" [passthrough][multi-turn-stream] {chunk_count} chunks, "- f"content[:100]={content[:100]!r}"- )-- # ------------------------------------------------------------------- # Tool calls — verifies ``tools`` definitions flow to the backend- # and an OpenAI-format tool call comes back through the chain.- # -------------------------------------------------------------------- def test_tool_call_non_streaming(self, passthrough_server: dict) -> None:- """``tools`` + ``tool_choice`` round-trip end-to-end.-- Validates that:-- * the ``tools`` definition is forwarded to the backend,- * the backend's ``tool_calls`` response survives the chain,- * the first tool call has the expected OpenAI shape- (``id``, ``type=function``, ``function.name``,- ``function.arguments``) and the arguments parse as JSON- containing the requested parameter.-- Uses :func:`resolve_tool_capable_model` to avoid vLLM-hosted- models that lack ``--enable-auto-tool-choice`` and would reject- ``tool_choice`` with HTTP 400 — we want to test the- passthrough's tool wiring, not the backend's flags.- """- tool_model = resolve_tool_capable_model(passthrough_server["model"])- logger.info(f" [passthrough][tools] using tool-capable model: {tool_model!r}")-- client = _make_openai_client(passthrough_server)- response = client.chat.completions.create(- model=tool_model,- messages=[{- "role": "user",- "content": (- "What is the weather in Tokyo? "- "You MUST use the get_weather tool."- ),- }],- tools=[CHAT_COMPLETIONS_WEATHER_TOOL],- tool_choice="auto",- max_tokens=2048,- )-- assert response.object == "chat.completion"- message = response.choices[0].message- assert message.role == "assistant"-- tool_calls = message.tool_calls or []- assert tool_calls, (- f"expected at least one tool call from the backend, "- f"got message={message.model_dump()}"- )-- first = tool_calls[0]- assert first.type == "function"- assert first.function.name == "get_weather", (- f"unexpected function name: {first.function.name!r}"- )-- raw_args = first.function.arguments or "{}"- args = json.loads(raw_args) if isinstance(raw_args, str) else raw_args- assert "city" in args, (- f"expected 'city' in arguments, got {args!r}"- )-- logger.info(- f" [passthrough][tools] tool_calls={len(tool_calls)}, "- f"fn={first.function.name!r}, args={args!r}"- )-- def test_tool_call_streaming(self, passthrough_server: dict) -> None:- """Streaming tool-call deltas reassemble into a valid call.-- Each chunk's ``delta.tool_calls`` carries incremental fragments- (``id`` on the first chunk, ``function.name`` on one chunk,- ``function.arguments`` split across many). The helper- :func:`_collect_sdk_stream_deltas` rebuilds them using the- OpenAI streaming contract; this test then runs the same- structural assertions as the non-streaming variant.-- Uses :func:`resolve_tool_capable_model` for the same reason as- the non-streaming tool test.- """- tool_model = resolve_tool_capable_model(passthrough_server["model"])- logger.info(f" [passthrough][tools-stream] using tool-capable model: {tool_model!r}")-- client = _make_openai_client(passthrough_server)- stream = client.chat.completions.create(- model=tool_model,- messages=[{- "role": "user",- "content": (- "What is the weather in Tokyo? "- "You MUST use the get_weather tool."- ),- }],- tools=[CHAT_COMPLETIONS_WEATHER_TOOL],- tool_choice="auto",- max_tokens=2048,- stream=True,- )-- chunk_count, content, reasoning, tool_calls = _collect_sdk_stream_deltas(- stream,- )- assert tool_calls, (- f"expected at least one streamed tool call, got "- f"{chunk_count} chunks, "- f"content[:100]={content[:100]!r}, "- f"reasoning[:100]={reasoning[:100]!r}"- )-- first = tool_calls[0]- assert first.get("type") == "function"- fn = first.get("function") or {}- assert fn.get("name") == "get_weather", (- f"unexpected function name: {fn.get('name')!r}"- )-- raw_args = fn.get("arguments") or "{}"- args = json.loads(raw_args) if isinstance(raw_args, str) else raw_args- assert "city" in args, (- f"expected 'city' in reassembled arguments, got {args!r}"- )-- logger.info(- f" [passthrough][tools-stream] {chunk_count} chunks, "- f"tool_calls={len(tool_calls)}, fn={fn.get('name')!r}, "- f"args={args!r}"- )diff --git a/tests/e2e/test_passthrough_responses_e2e.py b/tests/e2e/test_passthrough_responses_e2e.pydeleted file mode 100644index 481331db5..000000000--- a/tests/e2e/test_passthrough_responses_e2e.py+++ /dev/null@@ -1,557 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Production end-to-end smoke test for ``switchyard`` passthrough on ``/v1/responses``.--Exercises the cross-format production path end-to-end::-- openai SDK (responses.create) → HTTP (127.0.0.1)- → switchyard passthrough- → TranslationEngine.request_to (Responses → Chat)- → real backend (Chat Completions)- → TranslationEngine.response_to (Chat → Responses)- → Responses-shaped response--Unlike the sibling Chat Completions suite (which tests same-format-passthrough), this suite specifically validates the inbound and-outbound translators — a Responses-shaped request must survive-translation to Chat Completions, execute against the backend, and be-translated back to a Responses-shaped response.--Covers both non-streaming (``responses.create``) and streaming-(``responses.create(stream=True)``) paths — the latter validates that-``TranslationEngine.stream_for_request`` drives the-OpenAI-chunk → Responses-SSE conversion end-to-end through-``iter_preframed_sse`` and the ``/v1/responses`` endpoint.--Uses the official ``openai`` Python SDK as the client-(``client.responses.create(...)``) so we validate the exact request-shape a real customer integration would produce, and we get typed-response objects for free.--Shares the ``passthrough_server`` fixture with the other e2e files-via ``conftest.py`` — one subprocess server per session.--Run with::-- OPENROUTER_API_KEY=sk-or-... pytest tests/e2e/test_passthrough_responses_e2e.py -v-"""--from __future__ import annotations--import json-import logging-from typing import Any--import openai-import pytest--from ._helpers import (- RESPONSES_WEATHER_TOOL,- resolve_tool_capable_model,-)--pytestmark = pytest.mark.integration--logger = logging.getLogger("e2e.responses")---# ----------------------------------------------------------------------------# Shared helpers-# ------------------------------------------------------------------------------def _make_openai_client(passthrough_server: dict) -> openai.OpenAI:- """Build an OpenAI SDK client pointed at the passthrough server.-- Same convention as the Chat Completions sibling — the passthrough- injects its own upstream ``api_key`` when calling the real backend,- so the SDK-side ``api_key`` is just a placeholder for the- ``Authorization`` header.- """- return openai.OpenAI(- base_url=f"{passthrough_server['base_url']}/v1",- api_key="not-used-passthrough-forwards-its-own",- timeout=60.0,- max_retries=0,- )---def _extract_assistant_text(response: Any) -> str:- """Concatenate all ``output_text`` blocks across ``output`` items.-- The Responses API ``output`` array is heterogeneous — it can- contain ``message``, ``reasoning``, ``function_call``, and other- item types. This helper walks only ``type=message`` items and- pulls out their ``output_text`` blocks, returning the combined- assistant-visible text (empty string if none).- """- parts: list[str] = []- for item in response.output or []:- if getattr(item, "type", None) != "message":- continue- for block in getattr(item, "content", None) or []:- if getattr(block, "type", None) == "output_text":- text = getattr(block, "text", None)- if text:- parts.append(text)- return "".join(parts)---def _extract_function_calls(response: Any) -> list[Any]:- """Return every ``type=function_call`` item in ``response.output``.-- In the Responses API tool calls are top-level items in ``output``- rather than nested inside a ``message`` (unlike Chat Completions- where they live under ``message.tool_calls``). This helper keeps- the extraction logic in one place for the tool test below.- """- return [- item- for item in (response.output or [])- if getattr(item, "type", None) == "function_call"- ]---def _collect_responses_stream_events(- stream: Any,-) -> tuple[list[str], str, dict[int, dict[str, Any]]]:- """Drain a Responses API stream and summarize events.-- Returns ``(event_types, text, function_calls)``:-- * ``event_types`` — the ``type`` field of every ``ResponseStreamEvent``- received, in order. Used for lifecycle structural assertions- (``response.created`` first, ``response.completed`` last).- * ``text`` — all ``response.output_text.delta`` strings concatenated.- * ``function_calls`` — keyed by ``output_index``, each entry carries- ``{"name", "call_id", "arguments"}`` with ``arguments`` being the- concatenated ``response.function_call_arguments.delta`` fragments.- """- event_types: list[str] = []- text_parts: list[str] = []- function_calls: dict[int, dict[str, Any]] = {}-- for event in stream:- ev_type = getattr(event, "type", None)- if ev_type:- event_types.append(ev_type)-- if ev_type == "response.output_item.added":- item = getattr(event, "item", None)- if item is not None and getattr(item, "type", None) == "function_call":- idx = getattr(event, "output_index", 0)- function_calls[idx] = {- "name": getattr(item, "name", ""),- "call_id": getattr(item, "call_id", ""),- "arguments": "",- }- elif ev_type == "response.output_text.delta":- text_parts.append(getattr(event, "delta", "") or "")- elif ev_type == "response.function_call_arguments.delta":- idx = getattr(event, "output_index", 0)- if idx in function_calls:- function_calls[idx]["arguments"] += (- getattr(event, "delta", "") or ""- )-- return event_types, "".join(text_parts), function_calls---# ----------------------------------------------------------------------------# Tests-# ------------------------------------------------------------------------------class TestResponsesPassthroughE2E:- """OpenAI Responses SDK → cross-format passthrough → backend → Responses out."""-- # ------------------------------------------------------------------- # Basic say-hi- # -------------------------------------------------------------------- def test_say_hi_get_response_back(self, passthrough_server: dict) -> None:- """``client.responses.create(input="hi")`` round-trips successfully.-- Validates the minimum contract of the Responses passthrough:-- * the Responses SDK can reach the server and the request- deserializes cleanly into a ``Response`` object- * ``output`` is non-empty and contains an ``assistant`` message- with at least one ``output_text`` block that has visible text- """- client = _make_openai_client(passthrough_server)- response = client.responses.create(- model=passthrough_server["model"],- input="hi",- max_output_tokens=2048,- )-- assert response.output, (- f"expected non-empty output, got {response.model_dump()}"- )-- text = _extract_assistant_text(response)- assert text, (- f"no assistant output_text across {len(response.output)} items: "- f"{response.model_dump()}"- )-- logger.info(- f" [Responses] output_items={len(response.output)}, "- f"text[:100]={text[:100]!r}"- )-- # ------------------------------------------------------------------- # System prompt — Responses API uses ``instructions`` as the- # top-level system-equivalent parameter (not a ``system`` role).- # -------------------------------------------------------------------- def test_system_prompt_non_streaming(self, passthrough_server: dict) -> None:- """``instructions`` parameter flows through the cross-format translator.-- Responses API models the system prompt as a top-level- ``instructions`` string, distinct from the message list. The- ``TranslationEngine.request_to`` must lift- ``instructions`` into a ``role=system`` message before calling- the backend. If that translation drops the field, the model- has no way to honor the planted ``pineapple`` keyword, and- this test catches it.- """- client = _make_openai_client(passthrough_server)- response = client.responses.create(- model=passthrough_server["model"],- instructions=(- "You are a helpful assistant. Always include the exact "- "word 'pineapple' in your response."- ),- input="Say hello.",- max_output_tokens=2048,- )-- assert response.output, (- f"expected non-empty output: {response.model_dump()}"- )-- # Keyword check — only when the model produced visible- # output_text. Reasoning models can burn their entire- # ``max_output_tokens`` budget on internal thought before the- # Responses translator surfaces any ``output_text`` block;- # that's a backend-scheduling artifact, not a passthrough- # failure, so we forgive it. The shape checks above still- # prove the ``instructions`` translation didn't blow up.- text = _extract_assistant_text(response)- if text:- assert "pineapple" in text.lower(), (- f"instructions not honored — the ``instructions`` → "- f"system translation may have dropped the field. "- f"text={text[:200]!r}"- )-- logger.info(- f" [Responses][system] output_items={len(response.output)}, "- f"text[:150]={text[:150]!r}"- )-- # ------------------------------------------------------------------- # Multi-turn — Responses API accepts a list of message items as- # ``input``, exercising the translator's handling of assistant-role- # history items.- # -------------------------------------------------------------------- def test_multi_turn_non_streaming(self, passthrough_server: dict) -> None:- """Multi-turn history via message-list ``input`` flows through.-- Sends a 3-item ``input`` (``user`` / ``assistant`` / ``user``)- and asks the model to recall a name planted two turns back.- If the translator drops the assistant turn, the model has no- context and can't recall the name.- """- # Each input item is explicitly tagged ``type="message"``:- # ``TranslationEngine._convert_input_items_to_messages``- # dispatches on ``item["type"]`` and silently drops items that- # don't carry one, which would leave ``messages=[]`` and 400- # the backend with "list index out of range".- client = _make_openai_client(passthrough_server)- response = client.responses.create(- model=passthrough_server["model"],- input=[- {- "type": "message",- "role": "user",- "content": "My name is Alice. Please remember this.",- },- {- "type": "message",- "role": "assistant",- "content": "Got it, Alice. I will remember your name.",- },- {- "type": "message",- "role": "user",- "content": (- "What is my name? Reply with only the name "- "itself, nothing else."- ),- },- ],- max_output_tokens=2048,- )-- assert response.output, (- f"expected non-empty output: {response.model_dump()}"- )-- # Same reasoning-truncation forgiveness as the ``instructions``- # test above.- text = _extract_assistant_text(response)- if text:- assert "alice" in text.lower(), (- f"prior-turn context not recalled — the ``input`` list "- f"→ messages translation may have dropped history items. "- f"text={text[:200]!r}"- )-- logger.info(- f" [Responses][multi-turn] output_items={len(response.output)}, "- f"text[:150]={text[:150]!r}"- )-- # ------------------------------------------------------------------- # Tool call — Responses API puts ``function_call`` items directly- # into ``output`` rather than nesting them under a message's- # ``tool_calls`` field. This test confirms that shape survives the- # round-trip through Chat Completions format.- # -------------------------------------------------------------------- def test_tool_call_non_streaming(self, passthrough_server: dict) -> None:- """``tools`` definitions flow through and a ``function_call`` item returns.-- Validates that:-- * the flat Responses-shape tool definition is forwarded through- the translator (Responses → Chat Completions nested shape)- * the backend's ``tool_calls`` on the Chat response are- translated back to top-level ``function_call`` items in- ``response.output``- * the first function call has the expected name and its- ``arguments`` string parses as JSON containing ``city``-- Uses :func:`resolve_tool_capable_model` to avoid vLLM-hosted- models that lack ``--enable-auto-tool-choice``.- """- tool_model = resolve_tool_capable_model(passthrough_server["model"])- logger.info(- f" [Responses][tools] using tool-capable model: {tool_model!r}"- )-- client = _make_openai_client(passthrough_server)- response = client.responses.create(- model=tool_model,- input=(- "What is the weather in Tokyo? "- "You MUST use the get_weather tool."- ),- tools=[RESPONSES_WEATHER_TOOL],- tool_choice="auto",- max_output_tokens=2048,- )-- function_calls = _extract_function_calls(response)- assert function_calls, (- f"expected at least one function_call item in response.output, "- f"got {[getattr(i, 'type', None) for i in response.output or []]}"- )-- first = function_calls[0]- assert first.name == "get_weather", (- f"unexpected function name: {first.name!r}"- )-- raw_args = first.arguments or "{}"- args = json.loads(raw_args) if isinstance(raw_args, str) else raw_args- assert "city" in args, (- f"expected 'city' in arguments, got {args!r}"- )-- logger.info(- f" [Responses][tools] function_calls={len(function_calls)}, "- f"name={first.name!r}, args={args!r}"- )-- # ------------------------------------------------------------------- # Streaming — validates the Chat Completions → Responses SSE- # translation drives through the chain end-to-end.- #- # Wire path:- # OpenAI SDK responses.create(stream=True) → /v1/responses- # → chain → OpenAiPassthroughBackend returns an OpenAI stream response- # → translate_stream dispatches on the Responses request format- # → stream_chat_to_responses_sse yields pre-framed SSE strings- # → iter_preframed_sse forwards them → client SDK parses them- # -------------------------------------------------------------------- def test_say_hi_streaming(self, passthrough_server: dict) -> None:- """``responses.create(stream=True)`` yields a valid Responses SSE stream.-- Validates the minimum streaming contract:-- * the SDK can iterate the SSE stream (so the passthrough's- envelope — ``Content-Type: text/event-stream``, ``event:- response.*\\ndata: {...}\\n\\n`` frames — must be well-formed)- * the stream emits ``response.created`` before any content and- ``response.completed`` at the end- * at least one ``response.output_text.delta`` carries visible- text (so the translator text-path works chunk-by-chunk)- """- client = _make_openai_client(passthrough_server)- stream = client.responses.create(- model=passthrough_server["model"],- input="hi",- max_output_tokens=2048,- stream=True,- )- event_types, text, _ = _collect_responses_stream_events(stream)-- assert event_types and event_types[0] == "response.created", (- f"expected first event to be response.created, got {event_types[:3]!r}"- )- assert "response.completed" in event_types, (- f"missing response.completed, got types={event_types!r}"- )- assert text, (- f"no output_text.delta across {len(event_types)} events: "- f"types={event_types!r}"- )-- logger.info(- f" [Responses-stream] events={len(event_types)}, "- f"text[:100]={text[:100]!r}"- )-- def test_system_prompt_streaming(self, passthrough_server: dict) -> None:- """``instructions`` flows through the streaming cross-format translator."""- client = _make_openai_client(passthrough_server)- stream = client.responses.create(- model=passthrough_server["model"],- instructions=(- "You are a helpful assistant. Always include the exact "- "word 'pineapple' in your response."- ),- input="Say hello.",- max_output_tokens=2048,- stream=True,- )- event_types, text, _ = _collect_responses_stream_events(stream)-- assert "response.created" in event_types- assert "response.completed" in event_types-- if text:- assert "pineapple" in text.lower(), (- f"instructions not honored over the stream path. "- f"text={text[:200]!r}"- )-- logger.info(- f" [Responses-stream][system] events={len(event_types)}, "- f"text[:150]={text[:150]!r}"- )-- def test_multi_turn_streaming(self, passthrough_server: dict) -> None:- """Multi-turn history via message-list ``input`` streams back."""- client = _make_openai_client(passthrough_server)- stream = client.responses.create(- model=passthrough_server["model"],- input=[- {- "type": "message",- "role": "user",- "content": "My name is Alice. Please remember this.",- },- {- "type": "message",- "role": "assistant",- "content": "Got it, Alice. I will remember your name.",- },- {- "type": "message",- "role": "user",- "content": (- "What is my name? Reply with only the name "- "itself, nothing else."- ),- },- ],- max_output_tokens=2048,- stream=True,- )- event_types, text, _ = _collect_responses_stream_events(stream)-- assert "response.created" in event_types- assert "response.completed" in event_types-- if text:- assert "alice" in text.lower(), (- f"prior-turn context not recalled over the stream path. "- f"text={text[:200]!r}"- )-- logger.info(- f" [Responses-stream][multi-turn] events={len(event_types)}, "- f"text[:150]={text[:150]!r}"- )-- def test_tool_call_streaming(self, passthrough_server: dict) -> None:- """Streaming tool-call deltas reassemble into a valid ``function_call``.-- Validates:-- * ``response.output_item.added`` carries a ``function_call``- item with the expected ``name``- * ``response.function_call_arguments.delta`` fragments- concatenate into a parseable JSON object containing ``city``- * the stream terminates with ``response.completed``- """- tool_model = resolve_tool_capable_model(passthrough_server["model"])- logger.info(- f" [Responses-stream][tools] using tool-capable model: {tool_model!r}"- )-- client = _make_openai_client(passthrough_server)- stream = client.responses.create(- model=tool_model,- input=(- "What is the weather in Tokyo? "- "You MUST use the get_weather tool."- ),- tools=[RESPONSES_WEATHER_TOOL],- tool_choice="auto",- max_output_tokens=2048,- stream=True,- )- event_types, _, function_calls = _collect_responses_stream_events(stream)-- assert "response.completed" in event_types- assert function_calls, (- f"no streamed function_call item across {len(event_types)} events: "- f"types={event_types!r}"- )-- first_idx = sorted(function_calls)[0]- first = function_calls[first_idx]- assert first["name"] == "get_weather", (- f"unexpected function name: {first['name']!r}"- )-- raw_args = first["arguments"] or "{}"- args = json.loads(raw_args)- assert "city" in args, (- f"expected 'city' in reassembled arguments, got {args!r}"- )-- logger.info(- f" [Responses-stream][tools] events={len(event_types)}, "- f"function_calls={len(function_calls)}, "- f"name={first['name']!r}, args={args!r}"- )diff --git a/tests/e2e_multiturn_responses.py b/tests/e2e_multiturn_responses.pydeleted file mode 100644index 08cd79992..000000000--- a/tests/e2e_multiturn_responses.py+++ /dev/null@@ -1,369 +0,0 @@-#!/usr/bin/env python3-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""E2E test: multi-turn Responses API with GPT-OSS-120B via NVIDIA API.--Simulates Codex-like multi-turn tool-call conversations through the proxy's-/v1/responses endpoint. After all turns complete, loads the saved traces-from --rl-log-dir and verifies that each turn produced a separate assistant-message (i.e. the turn-merge bug is fixed).--Usage:- 1. Start the proxy in a separate terminal:- source .venv/bin/activate- switchyard passthrough \- --port 4000 \- --api-key "$OPENAI_API_KEY" \- --base-url https://inference-api.nvidia.com/v1 \- --enable-rl-logging --rl-log-dir ./e2e_traces-- 2. Run this script:- python tests/e2e_multiturn_responses.py --proxy-url http://localhost:4000-- The script exits 0 on success, 1 on failure.-"""--import argparse-import json-import sys-import time-import uuid-from pathlib import Path--import httpx--# ----------------------------------------------------------------------------# Config-# -----------------------------------------------------------------------------DEFAULT_PROXY_URL = "http://localhost:4000"-DEFAULT_MODEL = "nvidia/openai/gpt-oss-120b"-DEFAULT_TRACE_DIR = Path(__file__).parent.parent / "e2e_traces"-SESSION_ID = str(uuid.uuid4())--TOOLS = [- {- "type": "function",- "name": "exec_command",- "description": "Runs a shell command and returns stdout.",- "parameters": {- "type": "object",- "properties": {- "cmd": {"type": "string", "description": "Shell command to execute."},- },- "required": ["cmd"],- "additionalProperties": False,- },- },-]---# ----------------------------------------------------------------------------# Helpers-# -----------------------------------------------------------------------------def send_responses_request(- proxy_url: str,- model: str,- input_items: list,- session_id: str,- stream: bool = True,- timeout: float = 120.0,-) -> dict:- """Send a Responses API request and return the parsed response.-- For streaming, collects SSE events and returns the response.completed payload.- For non-streaming, returns the JSON response directly.- """- body = {- "model": model,- "input": input_items,- "tools": TOOLS,- "tool_choice": "auto",- "stream": stream,- "temperature": 0.2,- }-- headers = {- "Content-Type": "application/json",- "proxy_x_session_id": session_id,- }-- url = f"{proxy_url}/v1/responses"-- if not stream:- with httpx.Client(timeout=timeout) as client:- resp = client.post(url, json=body, headers=headers)- resp.raise_for_status()- return resp.json()-- # Streaming: collect SSE events- completed_response = None- with httpx.Client(timeout=timeout) as client:- with client.stream("POST", url, json=body, headers=headers) as resp:- resp.raise_for_status()- buffer = ""- for chunk in resp.iter_text():- buffer += chunk- while "\n\n" in buffer:- event_str, buffer = buffer.split("\n\n", 1)- lines = event_str.strip().split("\n")- event_type = None- event_data = None- for line in lines:- if line.startswith("event: "):- event_type = line[7:]- elif line.startswith("data: "):- event_data = line[6:]-- if event_type == "response.completed" and event_data:- completed_response = json.loads(event_data)- elif event_type == "error" and event_data:- err = json.loads(event_data)- print(f" SSE error: {err}", file=sys.stderr)-- if completed_response is None:- raise RuntimeError("Never received response.completed event")-- return completed_response---def extract_tool_calls(response: dict) -> list[dict]:- """Extract function_call items from a Responses API response."""- output = response.get("response", response).get("output", [])- return [item for item in output if item.get("type") == "function_call"]---def extract_text(response: dict) -> str:- """Extract text content from a Responses API response."""- output = response.get("response", response).get("output", [])- parts = []- for item in output:- if item.get("type") == "message":- for content in item.get("content", []):- if content.get("type") == "output_text":- parts.append(content.get("text", ""))- return "\n".join(parts)---def simulate_tool_results(tool_calls: list[dict]) -> list[dict]:- """Generate fake tool results for the given tool calls."""- results = []- for tc in tool_calls:- args = json.loads(tc.get("arguments", "{}"))- cmd = args.get("cmd", "unknown")- # Fake output based on command- if "ls" in cmd:- output = "file1.py\nfile2.py\nREADME.md\n"- elif "echo" in cmd:- output = "hello\n"- elif "cat" in cmd:- output = "# File content\nprint('hello world')\n"- elif "pwd" in cmd:- output = "/workspace\n"- else:- output = f"Executed: {cmd}\n"-- results.append({- "type": "function_call_output",- "call_id": tc.get("call_id", ""),- "output": output,- })- return results---def build_history(turns: list[tuple[list[dict], list[dict]]]) -> list[dict]:- """Build Responses API input items from a list of (tool_calls, tool_results) turns.-- Each turn's function_calls come first (grouped), then function_call_outputs.- This matches how Codex sends multi-turn history.- """- items = []- for tool_calls, tool_results in turns:- for tc in tool_calls:- items.append({- "type": "function_call",- "name": tc.get("name", ""),- "call_id": tc.get("call_id", ""),- "arguments": tc.get("arguments", "{}"),- })- for tr in tool_results:- items.append(tr)- return items---# ----------------------------------------------------------------------------# Load and verify traces-# -----------------------------------------------------------------------------def load_session_traces(trace_dir: Path, session_id: str) -> list[dict]:- """Load trace files for a specific session, sorted by timestamp."""- if not trace_dir.exists():- return []- traces = []- for f in sorted(trace_dir.glob("*.json")):- with open(f) as fh:- t = json.load(fh)- if t.get("sessionID") == session_id:- traces.append(t)- # Also check subdirectories (e.g. openai/)- for f in sorted(trace_dir.rglob("*.json")):- with open(f) as fh:- t = json.load(fh)- if t.get("sessionID") == session_id and t not in traces:- traces.append(t)- # Sort by timestamp- traces.sort(key=lambda t: t.get("timestamp", ""))- return traces---def verify_traces(traces: list[dict], expected_turns: int) -> bool:- """Verify that traces have correct multi-turn structure.-- Returns True if all traces show separate assistant messages per turn.- """- ok = True-- for i, trace in enumerate(traces):- msgs = trace["request"]["messages"]- asst_msgs = [m for m in msgs if m["role"] == "assistant"]- tool_msgs = [m for m in msgs if m["role"] == "tool"]-- # Trace i should have exactly i assistant messages in its history- # (one per previous turn)- expected_asst = i # trace 0 has 0, trace 1 has 1, etc.-- if len(asst_msgs) != expected_asst:- print(- f" FAIL trace {i+1}: expected {expected_asst} assistant msg(s) "- f"in history, got {len(asst_msgs)}"- )- if len(asst_msgs) > 0:- for j, am in enumerate(asst_msgs):- n_calls = len(am.get("tool_calls", []))- print(f" assistant msg {j}: {n_calls} tool_calls")- ok = False- else:- # Verify each assistant message has the right tool_calls count- total_tool_calls = sum(- len(am.get("tool_calls", [])) for am in asst_msgs- )- print(- f" OK trace {i+1}: {len(asst_msgs)} assistant msg(s), "- f"{total_tool_calls} total tool_calls, {len(tool_msgs)} tool results"- )-- return ok---# ----------------------------------------------------------------------------# Main-# -----------------------------------------------------------------------------def main():- parser = argparse.ArgumentParser(description="E2E multi-turn Responses API test")- parser.add_argument("--proxy-url", default=DEFAULT_PROXY_URL)- parser.add_argument("--model", default=DEFAULT_MODEL)- parser.add_argument("--trace-dir", type=Path, default=DEFAULT_TRACE_DIR)- parser.add_argument("--no-stream", action="store_true", help="Use non-streaming mode")- parser.add_argument("--max-turns", type=int, default=3, help="Max conversation turns")- args = parser.parse_args()-- stream = not args.no_stream- print(f"Session ID: {SESSION_ID}")- print(f"Proxy: {args.proxy_url}")- print(f"Model: {args.model}")- print(f"Stream: {stream}")- print(f"Trace dir: {args.trace_dir}")- print()-- # Initial user message- user_prompt = (- "You are a helpful coding assistant with access to exec_command tool. "- "Please: 1) List files in the current directory, 2) Show the current "- "working directory, 3) Echo 'hello world'. Do all three using exec_command."- )-- # Build the conversation input, starting with just the user message- input_items: list[dict] = [- {"type": "message", "role": "user", "content": user_prompt},- ]-- turns: list[tuple[list[dict], list[dict]]] = []- turn = 0-- while turn < args.max_turns:- turn += 1- print(f"--- Turn {turn} ---")- print(f" Sending {len(input_items)} input items...")-- try:- response = send_responses_request(- args.proxy_url, args.model, input_items, SESSION_ID,- stream=stream, timeout=120.0,- )- except Exception as e:- print(f" ERROR: {e}", file=sys.stderr)- sys.exit(1)-- tool_calls = extract_tool_calls(response)- text = extract_text(response)-- if text:- print(f" Text: {text[:100]}{'...' if len(text) > 100 else ''}")-- if not tool_calls:- print(f" No tool calls — conversation complete after {turn} turn(s)")- break-- print(f" Got {len(tool_calls)} tool call(s):")- for tc in tool_calls:- args_str = tc.get("arguments", "{}")- cmd = json.loads(args_str).get("cmd", "?")- print(f" - {tc['name']}({cmd})")-- # Simulate tool execution- tool_results = simulate_tool_results(tool_calls)- turns.append((tool_calls, tool_results))-- # Rebuild input with full history for next turn- history_items = build_history(turns)- input_items = [- {"type": "message", "role": "user", "content": user_prompt},- *history_items,- ]-- # Small delay to let trace writes flush- time.sleep(1)-- print()-- # Wait for background trace writer to flush- print("Waiting 3s for trace writer to flush...")- time.sleep(3)-- # Load and verify traces- print(f"\n--- Verifying traces in {args.trace_dir} ---")- traces = load_session_traces(args.trace_dir, SESSION_ID)-- if not traces:- print(f" WARNING: No traces found for session {SESSION_ID}")- print(f" Make sure --enable-rl-logging and --rl-log-dir={args.trace_dir} are set")- sys.exit(1)-- print(f" Found {len(traces)} trace(s) for session {SESSION_ID}")-- ok = verify_traces(traces, turn)-- if ok:- print("\nPASS: All traces show correct multi-turn structure!")- print("The turn-merge bug is fixed.")- sys.exit(0)- else:- print("\nFAIL: Traces show merged turns — the bug is NOT fixed.")- sys.exit(1)---if __name__ == "__main__":- main()diff --git a/tests/getting_started/test_getting_started.py b/tests/getting_started/test_getting_started.pyindex d862ac9ad..d41a648c0 100644--- a/tests/getting_started/test_getting_started.py+++ b/tests/getting_started/test_getting_started.py@@ -12,7 +12,7 @@ def test_getting_started_documents_current_paths() -> None:).read_text()assert guide.index("## Launcher Path") < guide.index("## Server Path")- assert 'uv tool install --python 3.12 "nemo-switchyard[cli,server]"' in guide+ assert 'uv tool install --python 3.12 "nemo-switchyard[cli]"' in guideassert "switchyard launch claude --model switchyard" in guideassert "cargo install --locked switchyard-server" in guideassert "switchyard-server --config routes.toml --dry-run" in guidediff --git a/tests/readme/test_readme.py b/tests/readme/test_readme.pyindex fc7cbcd96..0cdfbb528 100644--- a/tests/readme/test_readme.py+++ b/tests/readme/test_readme.py@@ -10,7 +10,7 @@ def test_readme_documents_current_paths() -> None:readme = (Path(__file__).resolve().parents[2] / "README.md").read_text()assert readme.index("### Launcher Path") < readme.index("### Server Path")- assert 'uv tool install --python 3.12 "nemo-switchyard[cli,server]"' in readme+ assert 'uv tool install --python 3.12 "nemo-switchyard[cli]"' in readmeassert "switchyard launch claude --model switchyard" in readmeassert "cargo install --locked switchyard-server" in readmeassert "switchyard-server --config routes.toml --dry-run" in readmediff --git a/tests/test_anthropic_native_llm_backend.py b/tests/test_anthropic_native_llm_backend.pydeleted file mode 100644index 54ed14a9d..000000000--- a/tests/test_anthropic_native_llm_backend.py+++ /dev/null@@ -1,74 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Python binding tests for the Rust-owned Anthropic native backend."""--from __future__ import annotations--import pytest--from switchyard.lib.backends.multi_llm_backend import build_native_backend-from switchyard_rust.components import AnthropicNativeBackend, BackendFormat, LlmTarget-from switchyard_rust.core import ChatRequestType---def _anthropic_target(**overrides: object) -> LlmTarget:- data: dict[str, object] = {- "id": "anthropic",- "model": "claude-sonnet-test",- "format": BackendFormat.ANTHROPIC,- "base_url": "https://api.anthropic.com",- "api_key": "sk-ant-test",- "timeout_secs": 12.5,- }- data.update(overrides)- return LlmTarget(**data)---def test_constructs_from_resolved_anthropic_target() -> None:- target = _anthropic_target()-- backend = AnthropicNativeBackend(target)-- assert backend.target == target- assert backend.target.endpoint.base_url == "https://api.anthropic.com"- assert backend.target.endpoint.api_key == "sk-ant-test"- assert backend.target.endpoint.timeout_secs == 12.5---def test_supported_request_types_is_anthropic_only() -> None:- backend = AnthropicNativeBackend(_anthropic_target())-- assert backend.supported_request_types == [ChatRequestType.ANTHROPIC]--- "target_format",- [BackendFormat.OPENAI, BackendFormat.AUTO, "openai", "auto"],-)-def test_rejects_unresolved_or_non_anthropic_target_format(target_format: object) -> None:- with pytest.raises(RuntimeError, match="resolved Anthropic format"):- AnthropicNativeBackend(_anthropic_target(format=target_format))---def test_build_native_backend_selects_anthropic_binding() -> None:- backend = build_native_backend(_anthropic_target())-- assert isinstance(backend, AnthropicNativeBackend)---def test_target_is_immutable_after_binding_construction() -> None:- target = _anthropic_target()- backend = AnthropicNativeBackend(target)-- with pytest.raises(AttributeError):- backend.target.model = "mutated"- assert backend.target.model == "claude-sonnet-test"---def test_request_types_are_value_objects_not_strings() -> None:- request_type = AnthropicNativeBackend(_anthropic_target()).supported_request_types[0]-- assert request_type is ChatRequestType.ANTHROPIC- assert request_type.value == "anthropic"- assert not isinstance(request_type, str)diff --git a/tests/test_anthropic_openai_translation.py b/tests/test_anthropic_openai_translation.pydeleted file mode 100644index 471b544a3..000000000--- a/tests/test_anthropic_openai_translation.py+++ /dev/null@@ -1,101 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Tests for Anthropic Messages to OpenAI Chat request translation."""--from __future__ import annotations--from collections.abc import Iterable, Mapping-from typing import Any--from switchyard_rust.translation import TranslationEngine--ENGINE = TranslationEngine()---def _translate_anthropic_request_to_openai(- *,- messages: Iterable[Mapping[str, Any]],- **kwargs: Any,-) -> dict[str, Any]:- return ENGINE.translate_request(- "anthropic_messages",- "openai_chat",- {"messages": list(messages), **kwargs},- )---def _system_messages(result: dict) -> list[dict]:- return [m for m in result["messages"] if m.get("role") == "system"]---def test_system_as_string():- result = _translate_anthropic_request_to_openai(- messages=[{"role": "user", "content": "Hello"}],- system="You are helpful.",- model="claude-3-5-sonnet-20241022",- max_tokens=100,- )- sys_msgs = _system_messages(result)- assert len(sys_msgs) == 1- assert sys_msgs[0]["content"] == "You are helpful."---def test_system_as_list_of_blocks():- result = _translate_anthropic_request_to_openai(- messages=[{"role": "user", "content": "Hello"}],- system=[{"type": "text", "text": "You are helpful."}],- model="claude-3-5-sonnet-20241022",- max_tokens=100,- )- sys_msgs = _system_messages(result)- assert len(sys_msgs) == 1- assert "You are helpful." in sys_msgs[0]["content"]---def test_system_as_tuple_of_blocks_not_silently_dropped():- """Tuples are valid Iterable[TextBlockParam] but isinstance(..., list) is False.-- Before the fix this test fails — the system prompt is silently dropped.- """- system = ({"type": "text", "text": "You are helpful."},)- result = _translate_anthropic_request_to_openai(- messages=[{"role": "user", "content": "Hello"}],- system=system,- model="claude-3-5-sonnet-20241022",- max_tokens=100,- )- sys_msgs = _system_messages(result)- assert len(sys_msgs) == 1, (- f"System prompt from tuple was silently dropped. messages={result['messages']}"- )- assert "You are helpful." in sys_msgs[0]["content"]---def test_system_as_generator_of_blocks_not_silently_dropped():- """Generators are valid Iterable[TextBlockParam] but isinstance(..., list) is False."""-- def block_gen():- yield {"type": "text", "text": "You are a generator."}-- result = _translate_anthropic_request_to_openai(- messages=[{"role": "user", "content": "Hello"}],- system=block_gen(),- model="claude-3-5-sonnet-20241022",- max_tokens=100,- )- sys_msgs = _system_messages(result)- assert len(sys_msgs) == 1, (- f"System prompt from generator was silently dropped. messages={result['messages']}"- )- assert "You are a generator." in sys_msgs[0]["content"]---def test_system_none_produces_no_system_message():- result = _translate_anthropic_request_to_openai(- messages=[{"role": "user", "content": "Hello"}],- system=None,- model="claude-3-5-sonnet-20241022",- max_tokens=100,- )- assert _system_messages(result) == []diff --git a/tests/test_anthropic_output_config_strip.py b/tests/test_anthropic_output_config_strip.pydeleted file mode 100644index e426926c9..000000000--- a/tests/test_anthropic_output_config_strip.py+++ /dev/null@@ -1,34 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Inbound Anthropic bodies must drop output_config.format (a Claude Code-structured-output schema upstream Anthropic model groups reject) while keeping-output_config.effort."""--from switchyard.lib.endpoints.anthropic_messages_endpoint import (- _strip_unsupported_output_config,-)---def test_strips_format_keeps_effort():- body = {"model": "x", "output_config": {"effort": "high", "format": {"schema": {}}}}- _strip_unsupported_output_config(body)- assert body["output_config"] == {"effort": "high"}---def test_drops_output_config_when_only_format():- body = {"model": "x", "output_config": {"format": {"schema": {}}}}- _strip_unsupported_output_config(body)- assert "output_config" not in body---def test_noop_without_output_config():- body = {"model": "x", "messages": []}- _strip_unsupported_output_config(body)- assert body == {"model": "x", "messages": []}---def test_noop_when_no_format_key():- body = {"model": "x", "output_config": {"effort": "high"}}- _strip_unsupported_output_config(body)- assert body["output_config"] == {"effort": "high"}diff --git a/tests/test_anthropic_probe.py b/tests/test_anthropic_probe.pydeleted file mode 100644index fc323d511..000000000--- a/tests/test_anthropic_probe.py+++ /dev/null@@ -1,156 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Tests for :func:`probe_anthropic_messages_support`.--The probe is a single HTTP POST with empty body + real auth; responses-partition cleanly into "endpoint wired" vs "not wired / broken". Tests-mock httpx at the ``AsyncClient`` level.-"""--from __future__ import annotations--import logging-from unittest.mock import AsyncMock, MagicMock, patch--import httpx-import pytest--from switchyard.lib.backends.backend_format_resolver import (- probe_anthropic_messages_support,- strip_v1_suffix,-)---def _fake_response(status_code: int) -> MagicMock:- resp = MagicMock()- resp.status_code = status_code- return resp---def _patch_httpx(response: MagicMock | Exception):- """Return a context manager patching AsyncClient.post.-- If ``response`` is an Exception, it's raised from the POST; otherwise- it's returned.- """- async_client = MagicMock()- if isinstance(response, Exception):- async_client.post = AsyncMock(side_effect=response)- else:- async_client.post = AsyncMock(return_value=response)-- # AsyncClient is used as an async context manager- async_cm = MagicMock()- async_cm.__aenter__ = AsyncMock(return_value=async_client)- async_cm.__aexit__ = AsyncMock(return_value=False)-- return patch.object(httpx, "AsyncClient", return_value=async_cm)---class TestProbeAnthropicMessagesSupport:- async def test_404_returns_false(self):- with _patch_httpx(_fake_response(404)):- assert await probe_anthropic_messages_support(- base_url="https://x.example/v1", api_key="sk-test",- ) is False-- async def test_400_returns_true(self):- # Endpoint exists, validator ran and rejected our empty body.- with _patch_httpx(_fake_response(400)):- assert await probe_anthropic_messages_support(- base_url="https://x.example/v1", api_key="sk-test",- ) is True-- async def test_422_returns_true(self):- with _patch_httpx(_fake_response(422)):- assert await probe_anthropic_messages_support(- base_url="https://x.example/v1", api_key="sk-test",- ) is True-- async def test_200_returns_true(self):- with _patch_httpx(_fake_response(200)):- assert await probe_anthropic_messages_support(- base_url="https://x.example/v1", api_key="sk-test",- ) is True-- async def test_401_returns_false(self):- with _patch_httpx(_fake_response(401)):- assert await probe_anthropic_messages_support(- base_url="https://x.example/v1", api_key="sk-test",- ) is False-- async def test_5xx_returns_false(self):- with _patch_httpx(_fake_response(500)):- assert await probe_anthropic_messages_support(- base_url="https://x.example/v1", api_key="sk-test",- ) is False-- async def test_timeout_returns_false(self):- with _patch_httpx(httpx.TimeoutException("slow")):- assert await probe_anthropic_messages_support(- base_url="https://x.example/v1", api_key="sk-test",- ) is False-- async def test_timeout_does_not_log_warning(self, caplog):- caplog.set_level(logging.WARNING)- with _patch_httpx(httpx.TimeoutException("slow")):- assert await probe_anthropic_messages_support(- base_url="https://x.example/v1", api_key="sk-test",- ) is False-- assert "Anthropic /v1/messages probe" not in caplog.text-- async def test_connection_error_returns_false(self):- with _patch_httpx(httpx.ConnectError("dns fail")):- assert await probe_anthropic_messages_support(- base_url="https://x.example/v1", api_key="sk-test",- ) is False-- async def test_probe_url_is_root_plus_v1_messages(self):- """``--base-url`` follows OpenAI convention (ends in ``/v1``) but- the probe's target is ``{root}/v1/messages``. Make sure we don't- produce ``/v1/v1/messages`` or strip essential path components.- """- captured: dict = {}-- async def fake_post(url, headers, json):- captured["url"] = url- return _fake_response(400)-- async_client = MagicMock()- async_client.post = AsyncMock(side_effect=fake_post)- async_cm = MagicMock()- async_cm.__aenter__ = AsyncMock(return_value=async_client)- async_cm.__aexit__ = AsyncMock(return_value=False)-- with patch.object(httpx, "AsyncClient", return_value=async_cm):- await probe_anthropic_messages_support(- base_url="https://inference-api.nvidia.com/v1", api_key="sk-test",- )- assert captured["url"] == "https://inference-api.nvidia.com/v1/messages"-- @pytest.mark.parametrize("status", [402, 403, 405, 429])- async def test_other_4xx_treated_as_endpoint_exists(self, status):- with _patch_httpx(_fake_response(status)):- assert await probe_anthropic_messages_support(- base_url="https://x.example/v1", api_key="sk-test",- ) is True---class TestStripV1Suffix:- def test_strips_trailing_v1(self):- assert strip_v1_suffix("https://host/v1") == "https://host"-- def test_strips_trailing_v1_with_slash(self):- assert strip_v1_suffix("https://host/v1/") == "https://host"-- def test_no_v1_suffix_untouched(self):- assert strip_v1_suffix("https://host") == "https://host"-- def test_trailing_slash_stripped_even_when_no_v1(self):- assert strip_v1_suffix("https://host/") == "https://host"-- def test_v1_in_middle_of_path_untouched(self):- # Defensive: /v1 embedded in a longer path must not be stripped.- assert strip_v1_suffix("https://host/v1/sub") == "https://host/v1/sub"diff --git a/tests/test_backend_format_resolver.py b/tests/test_backend_format_resolver.pydeleted file mode 100644index 4d1b1f714..000000000--- a/tests/test_backend_format_resolver.py+++ /dev/null@@ -1,399 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--import httpx-import pytest--from switchyard.lib import startup_timing-from switchyard.lib.backends import (- backend_format_resolver as resolver_mod,-)-from switchyard.lib.backends.llm_target import (- BackendFormat,- LlmTarget,-)---class _RecordingProbe:- def __init__(self, result: bool) -> None:- self.result = result- self.calls: list[dict[str, object]] = []-- def __call__(self, **kwargs: object) -> bool:- self.calls.append(dict(kwargs))- return self.result---def _no_probe(name: str):- def fail(**_: object) -> bool:- pytest.fail(f"explicit backend formats must not probe ({name})")- return fail---# ----------------------------------------------------------------------------# Explicit format — no probing at all-# ------------------------------------------------------------------------------def test_explicit_format_does_not_probe(monkeypatch: pytest.MonkeyPatch) -> None:- monkeypatch.setattr(resolver_mod, "probe_openai_chat_completions_support_sync",- _no_probe("chat-completions"))- monkeypatch.setattr(resolver_mod, "probe_anthropic_messages_support_sync",- _no_probe("anthropic"))- monkeypatch.setattr(resolver_mod, "probe_openai_responses_support_sync",- _no_probe("responses"))-- resolution = resolver_mod.BackendFormatResolver.resolve(- LlmTarget(model="m", format=BackendFormat.ANTHROPIC),- )-- assert resolution.format is BackendFormat.ANTHROPIC- assert resolution.reason == "backend format is explicitly configured"---# ----------------------------------------------------------------------------# Model-prefix fast-path — skips all probes-# -----------------------------------------------------------------------------[email protected]("model", [- "anthropic/claude-3-5-sonnet",- "anthropic/claude-haiku",- "claude-3-opus-20240229",- "claude-sonnet-4-6",-])-def test_auto_model_prefix_anthropic_skips_probes(- monkeypatch: pytest.MonkeyPatch, model: str-) -> None:- """anthropic/ and claude prefixes → ANTHROPIC without probing."""- monkeypatch.setattr(resolver_mod, "probe_openai_chat_completions_support_sync",- _no_probe("chat-completions"))- monkeypatch.setattr(resolver_mod, "probe_anthropic_messages_support_sync",- _no_probe("anthropic"))- monkeypatch.setattr(resolver_mod, "probe_openai_responses_support_sync",- _no_probe("responses"))-- resolution = resolver_mod.BackendFormatResolver.resolve(- LlmTarget(- model=model,- format=BackendFormat.AUTO,- base_url="https://provider.test/v1",- api_key="sk-test", # pragma: allowlist secret- ),- )-- assert resolution.format is BackendFormat.ANTHROPIC- assert "prefix" in resolution.reason--[email protected]("model", [- "openrouter/anthropic/claude-3-5-sonnet",- "aws/anthropic/bedrock-claude-opus-4-7",-])-def test_auto_gateway_anthropic_model_probes_chat_completions_first(- monkeypatch: pytest.MonkeyPatch, model: str-) -> None:- """Gateway-namespaced models are NOT fast-pathed; Chat Completions probe runs first."""- chat_probe = _RecordingProbe(result=True)- monkeypatch.setattr(resolver_mod, "probe_openai_chat_completions_support_sync", chat_probe)- monkeypatch.setattr(resolver_mod, "probe_anthropic_messages_support_sync",- _no_probe("anthropic"))- monkeypatch.setattr(resolver_mod, "probe_openai_responses_support_sync",- _no_probe("responses"))-- resolution = resolver_mod.BackendFormatResolver.resolve(- LlmTarget(- model=model,- format=BackendFormat.AUTO,- base_url="https://openrouter.ai/api/v1",- api_key="sk-test", # pragma: allowlist secret- ),- )-- assert resolution.format is BackendFormat.OPENAI- assert len(chat_probe.calls) == 1---# ----------------------------------------------------------------------------# AUTO probe order: Chat Completions → Anthropic → Responses-# ------------------------------------------------------------------------------def test_auto_chat_completions_wins_when_supported(- monkeypatch: pytest.MonkeyPatch,-) -> None:- """Chat Completions probe succeeds → OPENAI; remaining probes are never called."""- monkeypatch.setattr(resolver_mod, "probe_openai_chat_completions_support_sync",- _RecordingProbe(result=True))- monkeypatch.setattr(resolver_mod, "probe_anthropic_messages_support_sync",- _no_probe("anthropic"))- monkeypatch.setattr(resolver_mod, "probe_openai_responses_support_sync",- _no_probe("responses"))-- resolution = resolver_mod.BackendFormatResolver.resolve(- LlmTarget(- model="nvidia/nvidia/nemotron-nano-9b-v2",- format=BackendFormat.AUTO,- base_url="https://inference-api.nvidia.com/v1",- api_key="sk-test", # pragma: allowlist secret- ),- )-- assert resolution.format is BackendFormat.OPENAI---def test_auto_resolves_to_anthropic_when_chat_completions_unavailable(- monkeypatch: pytest.MonkeyPatch,-) -> None:- """Chat Completions probe fails, Anthropic probe succeeds → ANTHROPIC."""- probe = _RecordingProbe(result=True)- monkeypatch.setattr(resolver_mod, "probe_openai_chat_completions_support_sync",- _RecordingProbe(result=False))- monkeypatch.setattr(resolver_mod, "probe_anthropic_messages_support_sync", probe)- monkeypatch.setattr(resolver_mod, "probe_openai_responses_support_sync",- _no_probe("responses"))-- resolution = resolver_mod.BackendFormatResolver.resolve(- LlmTarget(- model="some-model",- format=BackendFormat.AUTO,- base_url="https://api.anthropic.com/v1",- api_key="sk-test", # pragma: allowlist secret- ),- )-- assert resolution.format is BackendFormat.ANTHROPIC- assert probe.calls == [{- "base_url": "https://api.anthropic.com/v1",- "api_key": "sk-test",- "model": "some-model",- "timeout_s": 3.0,- }]---def test_auto_resolves_to_responses_when_only_responses_probe_succeeds(- monkeypatch: pytest.MonkeyPatch,-) -> None:- """Chat Completions and Anthropic probes fail, Responses probe succeeds → RESPONSES."""- monkeypatch.setattr(resolver_mod, "probe_openai_chat_completions_support_sync",- _RecordingProbe(result=False))- monkeypatch.setattr(resolver_mod, "probe_anthropic_messages_support_sync",- _RecordingProbe(result=False))- monkeypatch.setattr(resolver_mod, "probe_openai_responses_support_sync",- lambda **_: True)-- resolution = resolver_mod.BackendFormatResolver.resolve(- LlmTarget(- model="some-model",- format=BackendFormat.AUTO,- base_url="https://provider.test/v1",- api_key="sk-test", # pragma: allowlist secret- ),- )-- assert resolution.format is BackendFormat.RESPONSES---def test_auto_falls_back_to_openai_when_all_probes_fail(- monkeypatch: pytest.MonkeyPatch,-) -> None:- """All three probes fail → OPENAI (Chat Completions assumed universal)."""- monkeypatch.setattr(resolver_mod, "probe_openai_chat_completions_support_sync",- _RecordingProbe(result=False))- monkeypatch.setattr(resolver_mod, "probe_anthropic_messages_support_sync",- _RecordingProbe(result=False))- monkeypatch.setattr(resolver_mod, "probe_openai_responses_support_sync",- lambda **_: False)-- resolution = resolver_mod.BackendFormatResolver.resolve(- LlmTarget(- model="some-model",- format=BackendFormat.AUTO,- base_url="https://provider.test/v1",- api_key="sk-test", # pragma: allowlist secret- ),- )-- assert resolution.format is BackendFormat.OPENAI---def test_auto_chat_completions_timeout_assumes_openai(- monkeypatch: pytest.MonkeyPatch,-) -> None:- """A Chat Completions probe that times out (raises, not a fast 404) resolves- to OPENAI without stacking the slower /v1/messages and /v1/responses probes.- A 404 returns False and still falls through — that path is covered by- test_auto_resolves_to_anthropic_when_chat_completions_unavailable."""- def timed_out(**_: object) -> bool:- raise httpx.ReadTimeout("probe timed out")-- monkeypatch.setattr(resolver_mod, "probe_openai_chat_completions_support_sync",- timed_out)- monkeypatch.setattr(resolver_mod, "probe_anthropic_messages_support_sync",- _no_probe("anthropic"))- monkeypatch.setattr(resolver_mod, "probe_openai_responses_support_sync",- _no_probe("responses"))-- resolution = resolver_mod.BackendFormatResolver.resolve(- LlmTarget(- model="slow-endpoint-model",- format=BackendFormat.AUTO,- base_url="https://slow.test/v1",- api_key="sk-test", # pragma: allowlist secret- timeout_secs=0.05,- ),- )-- assert resolution.format is BackendFormat.OPENAI- assert "timed out" in resolution.reason---def test_auto_records_each_probe_in_startup_timing(- monkeypatch: pytest.MonkeyPatch,-) -> None:- """With timing on, each probe reached leaves its own mark so- `launch --startup-timing` can show a per-route breakdown."""- monkeypatch.setattr(startup_timing, "enabled", True)- startup_timing._marks.clear()-- monkeypatch.setattr(resolver_mod, "probe_openai_chat_completions_support_sync",- _RecordingProbe(result=False))- monkeypatch.setattr(resolver_mod, "probe_anthropic_messages_support_sync",- _RecordingProbe(result=True))- monkeypatch.setattr(resolver_mod, "probe_openai_responses_support_sync",- _no_probe("responses"))-- resolution = resolver_mod.BackendFormatResolver.resolve(- LlmTarget(- model="some-model",- format=BackendFormat.AUTO,- base_url="https://api.anthropic.com/v1",- api_key="sk-test", # pragma: allowlist secret- ),- )-- assert resolution.format is BackendFormat.ANTHROPIC- labels = [label for label, _ in startup_timing._marks]- assert labels == ["chain init", "probe: /v1/chat/completions", "probe: /v1/messages"]- startup_timing._marks.clear()---# ----------------------------------------------------------------------------# Missing inputs-# ------------------------------------------------------------------------------def test_auto_format_requires_base_url() -> None:- with pytest.raises(ValueError, match="requires base_url"):- resolver_mod.BackendFormatResolver.resolve(- LlmTarget(- model="m",- format=BackendFormat.AUTO,- api_key="sk-test", # pragma: allowlist secret- ),- )---def test_auto_format_requires_api_key() -> None:- with pytest.raises(ValueError, match="requires api_key"):- resolver_mod.BackendFormatResolver.resolve(- LlmTarget(- model="m",- format=BackendFormat.AUTO,- base_url="https://provider.test/v1",- ),- )---# ----------------------------------------------------------------------------# Model + timeout forwarding-# ------------------------------------------------------------------------------def test_auto_forwards_endpoint_timeout_to_all_probes(- monkeypatch: pytest.MonkeyPatch,-) -> None:- """tier.endpoint.timeout_secs is forwarded as timeout_s to all three probes."""- chat_probe = _RecordingProbe(result=False)- anthropic_probe = _RecordingProbe(result=False)- responses_probe = _RecordingProbe(result=False)- monkeypatch.setattr(resolver_mod, "probe_openai_chat_completions_support_sync", chat_probe)- monkeypatch.setattr(resolver_mod, "probe_anthropic_messages_support_sync", anthropic_probe)- monkeypatch.setattr(resolver_mod, "probe_openai_responses_support_sync", responses_probe)-- resolver_mod.BackendFormatResolver.resolve(- LlmTarget(- model="nvidia/nvidia/nemotron-nano",- format=BackendFormat.AUTO,- base_url="https://integrate.api.nvidia.com/v1",- api_key="sk-test", # pragma: allowlist secret- timeout_secs=30.0,- ),- )-- assert chat_probe.calls[0]["timeout_s"] == 30.0- assert anthropic_probe.calls[0]["timeout_s"] == 30.0- assert responses_probe.calls[0]["timeout_s"] == 30.0---def test_auto_passes_model_to_all_probes(- monkeypatch: pytest.MonkeyPatch,-) -> None:- """model is forwarded to all three probes so mixed providers are correctly scoped."""- chat_probe = _RecordingProbe(result=False)- anthropic_probe = _RecordingProbe(result=False)- responses_probe = _RecordingProbe(result=False)- monkeypatch.setattr(resolver_mod, "probe_openai_chat_completions_support_sync", chat_probe)- monkeypatch.setattr(resolver_mod, "probe_anthropic_messages_support_sync", anthropic_probe)- monkeypatch.setattr(resolver_mod, "probe_openai_responses_support_sync", responses_probe)-- resolver_mod.BackendFormatResolver.resolve(- LlmTarget(- model="openrouter/non-claude-model",- format=BackendFormat.AUTO,- base_url="https://openrouter.ai/api/v1",- api_key="sk-test", # pragma: allowlist secret- ),- )-- assert chat_probe.calls[0]["model"] == "openrouter/non-claude-model"- assert anthropic_probe.calls[0]["model"] == "openrouter/non-claude-model"- assert responses_probe.calls[0]["model"] == "openrouter/non-claude-model"---# ----------------------------------------------------------------------------# _interpret_status — model-error body detection-# ------------------------------------------------------------------------------def test_interpret_status_400_with_model_not_found_body_returns_false() -> None:- """400 whose body names the model as not found is a probe failure."""- import json as _json-- body = _json.dumps({- "error": {"type": "invalid_request_error", "message": "model: gpt-4o not found"},- }).encode()- assert resolver_mod._interpret_status(400, body) is False---def test_interpret_status_400_with_not_found_error_type_returns_false() -> None:- """400/404-style 'not_found_error' type is treated as a probe failure."""- import json as _json-- body = _json.dumps({"error": {"type": "not_found_error", "message": "model not found"}}).encode()- assert resolver_mod._interpret_status(400, body) is False---def test_interpret_status_400_with_field_error_returns_true() -> None:- """400 about missing fields (not the model) means route exists — True."""- import json as _json-- body = _json.dumps({- "error": {"type": "invalid_request_error", "message": "messages: field required"},- }).encode()- assert resolver_mod._interpret_status(400, body) is True---def test_interpret_status_400_without_body_returns_true() -> None:- """400 with no body (legacy call-site) preserves old behaviour — True."""- assert resolver_mod._interpret_status(400) is Truediff --git a/tests/test_build_and_serve.py b/tests/test_build_and_serve.pydeleted file mode 100644index 547f8ddd7..000000000--- a/tests/test_build_and_serve.py+++ /dev/null@@ -1,384 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Tests for ``switchyard.server.server_util.build_and_serve``.--``build_and_serve`` is the second app-construction code path (the first is-``build_switchyard_app`` directly). PR #28 fixed a regression where this-function lazy-imported three modules that no longer exist after the-open-source cleanup, raising ``ModuleNotFoundError`` for every CLI subcommand-that called it (e.g., ``random-routing``).--These tests exercise the function offline by patching out ``uvicorn.run``,-capturing the FastAPI app it would have served, and driving real HTTP-requests through that app via ``httpx.ASGITransport``. A regression in any-of build_and_serve's imports, app construction, or extra-endpoint wiring-fails one of these tests.-"""--from __future__ import annotations--import argparse-from collections.abc import AsyncIterator-from typing import Any--import httpx-import pytest-from fastapi import APIRouter, FastAPI-from openai.types.chat import ChatCompletion-from openai.types.chat.chat_completion import Choice-from openai.types.chat.chat_completion_message import ChatCompletionMessage-from openai.types.completion_usage import CompletionUsage--from switchyard.lib.endpoints.base import Endpoint-from switchyard.lib.proxy_context import ProxyContext-from switchyard.lib.roles import LLMBackend-from switchyard.lib.switchyard import Switchyard-from switchyard.server import server_util-from switchyard_rust.core import ChatRequest, ChatRequestType, ChatResponse-from switchyard_rust.translation import TranslationEngine--_ALL_REQUEST_TYPES = [- ChatRequestType.OPENAI_CHAT,- ChatRequestType.OPENAI_RESPONSES,- ChatRequestType.ANTHROPIC,-]--# ----------------------------------------------------------------------------# Fakes-# ------------------------------------------------------------------------------class _StubBackend(LLMBackend):- def supported_request_types(self) -> list[ChatRequestType]:- return list(_ALL_REQUEST_TYPES)-- async def call(self, ctx: ProxyContext, request: ChatRequest) -> ChatResponse:- completion = ChatCompletion(- id="chatcmpl-stub",- object="chat.completion",- created=1700000000,- model="stub",- choices=[- Choice(- index=0,- message=ChatCompletionMessage(role="assistant", content="ok"),- finish_reason="stop",- )- ],- usage=CompletionUsage(prompt_tokens=1, completion_tokens=1, total_tokens=2),- )- return ChatResponse.openai_completion(completion)---class _CountingStubBackend(_StubBackend):- """Variant of _StubBackend that records each call so tests can assert short-circuit."""-- def __init__(self) -> None:- self.call_count = 0-- async def call(self, ctx: ProxyContext, request: ChatRequest) -> ChatResponse:- self.call_count += 1- return await super().call(ctx, request)---class _SentinelEndpoint(Endpoint):- """Adds ``GET /sentinel`` so tests can verify ``extra_endpoints`` are wired."""-- def register(self, app: FastAPI) -> None:- router = APIRouter()-- @router.get("/sentinel")- async def _sentinel() -> dict[str, str]:- return {"sentinel": "ok"}-- app.include_router(router)---# ----------------------------------------------------------------------------# Helpers-# ------------------------------------------------------------------------------def _ns(**overrides: Any) -> argparse.Namespace:- """Build the argparse namespace ``build_and_serve`` expects."""- defaults = {"host": "127.0.0.1", "port": 4000, "reload": False, "workers": 1}- defaults.update(overrides)- return argparse.Namespace(**defaults)---def _capture_uvicorn(monkeypatch: pytest.MonkeyPatch) -> dict[str, Any]:- """Patch ``uvicorn.run`` to capture its kwargs without starting a server."""- captured: dict[str, Any] = {}-- def _fake_run(app: FastAPI, **kwargs: Any) -> None:- captured["app"] = app- captured["kwargs"] = kwargs-- import uvicorn-- monkeypatch.setattr(uvicorn, "run", _fake_run)- return captured---def _switchyard() -> Switchyard:- return Switchyard(backend=_StubBackend(), translator=TranslationEngine())---# ----------------------------------------------------------------------------# Tests-# ------------------------------------------------------------------------------def test_build_and_serve_imports_succeed(monkeypatch: pytest.MonkeyPatch) -> None:- """Direct guard for PR #28: ``build_and_serve`` must not raise on import.-- The previous regression was a lazy import of deleted modules- (``endpoint_sets``, ``server``, ``server_config``); the function ran- cleanly until called, then crashed at the lazy-import line. Calling- it with a stub uvicorn proves all of its imports resolve.- """- captured = _capture_uvicorn(monkeypatch)- server_util.build_and_serve(_ns(), _switchyard())- assert "app" in captured, "build_and_serve never reached uvicorn.run"---def test_build_and_serve_passes_host_and_port_through(monkeypatch: pytest.MonkeyPatch) -> None:- captured = _capture_uvicorn(monkeypatch)- server_util.build_and_serve(_ns(host="0.0.0.0", port=5555), _switchyard())- assert captured["kwargs"]["host"] == "0.0.0.0"- assert captured["kwargs"]["port"] == 5555---def test_build_and_serve_defaults_port_to_4000_when_none(- monkeypatch: pytest.MonkeyPatch,-) -> None:- """Documented default in --port help; regression-guard the fallback path."""- captured = _capture_uvicorn(monkeypatch)- server_util.build_and_serve(_ns(port=None), _switchyard())- assert captured["kwargs"]["port"] == 4000---def test_build_and_serve_workers_default_to_one(monkeypatch: pytest.MonkeyPatch) -> None:- """Namespace without ``workers`` must not raise — ``getattr`` fallback applies."""- captured = _capture_uvicorn(monkeypatch)- args = argparse.Namespace(host="127.0.0.1", port=4000, reload=False)- server_util.build_and_serve(args, _switchyard())- assert captured["kwargs"]["workers"] == 1---def test_build_and_serve_registers_extra_endpoints(monkeypatch: pytest.MonkeyPatch) -> None:- """``extra_endpoints`` must be registered onto the app before serving."""- captured = _capture_uvicorn(monkeypatch)- server_util.build_and_serve(- _ns(),- _switchyard(),- extra_endpoints=[_SentinelEndpoint()],- )- routes = {getattr(r, "path", None) for r in captured["app"].routes}- assert "/sentinel" in routes---async def served_client(- monkeypatch: pytest.MonkeyPatch,-) -> AsyncIterator[httpx.AsyncClient]:- """Drive the app ``build_and_serve`` would have served through ASGI.-- The full app boot path runs (build_switchyard_app → endpoint- registration → app.state wiring) but uvicorn never starts.- """- captured = _capture_uvicorn(monkeypatch)- server_util.build_and_serve(_ns(), _switchyard(), extra_endpoints=[_SentinelEndpoint()])- async with httpx.AsyncClient(- transport=httpx.ASGITransport(app=captured["app"]),- base_url="http://test",- ) as client:- yield client---class TestBuiltAppRoundTrips:- """Round-trip through the app build_and_serve assembled.-- Catches the kind of failure PR #28 fixed: server starts cleanly, but- every request hits ``AttributeError`` because of an internal wiring- mismatch. A unit test on ``build_and_serve`` alone wouldn't have- caught the ``app.state.switchyard`` vs ``app.state.switchyard``- bug — only a real request through the assembled app does.- """-- async def test_health(self, served_client: httpx.AsyncClient) -> None:- resp = await served_client.get("/health")- assert resp.status_code == 200- assert resp.json() == {"status": "ok"}-- async def test_openai_chat_completions(self, served_client: httpx.AsyncClient) -> None:- resp = await served_client.post(- "/v1/chat/completions",- json={- "model": "any",- "messages": [{"role": "user", "content": "hi"}],- },- )- assert resp.status_code == 200, resp.text- assert resp.json()["choices"][0]["message"]["content"] == "ok"-- async def test_anthropic_messages(self, served_client: httpx.AsyncClient) -> None:- resp = await served_client.post(- "/v1/messages",- json={- "model": "any",- "max_tokens": 16,- "messages": [{"role": "user", "content": "hi"}],- },- )- assert resp.status_code == 200, resp.text- assert resp.json()["content"][0]["text"] == "ok"-- async def test_extra_endpoint_reachable(self, served_client: httpx.AsyncClient) -> None:- resp = await served_client.get("/sentinel")- assert resp.status_code == 200- assert resp.json() == {"sentinel": "ok"}---class TestInvalidRequestBody:- """Malformed or non-object JSON bodies must return 400 with a structured error envelope.-- REQ-AG3: agents must receive structured errors with retry semantics so they can- distinguish client-side parse bugs (no retry) from transient server failures (retry).- These tests confirm the fix for the bare-500 regression reported in bug 6267258.- """-- _ENDPOINTS = [- "/v1/chat/completions",- "/v1/messages",- "/v1/responses",- ]-- @pytest.mark.parametrize("path", _ENDPOINTS)- async def test_malformed_json_returns_400(- self, served_client: httpx.AsyncClient, path: str- ) -> None:- resp = await served_client.post(- path,- content="{invalid json,,",- headers={"Content-Type": "application/json"},- )- assert resp.status_code == 400- assert resp.headers["content-type"].startswith("application/json")- body = resp.json()- assert body["error"]["type"] == "invalid_request_error"- assert body["error"]["code"] == "invalid_body"-- @pytest.mark.parametrize("path", _ENDPOINTS)- async def test_json_array_body_returns_400(- self, served_client: httpx.AsyncClient, path: str- ) -> None:- resp = await served_client.post(- path,- content='["not", "an", "object"]',- headers={"Content-Type": "application/json"},- )- assert resp.status_code == 400- assert resp.headers["content-type"].startswith("application/json")- body = resp.json()- assert body["error"]["type"] == "invalid_request_error"- assert body["error"]["code"] == "invalid_body"-- async def test_server_stays_healthy_after_bad_request(- self, served_client: httpx.AsyncClient- ) -> None:- await served_client.post(- "/v1/chat/completions",- content="{bad json",- headers={"Content-Type": "application/json"},- )- resp = await served_client.get("/health")- assert resp.status_code == 200---async def counting_client(- monkeypatch: pytest.MonkeyPatch,-) -> AsyncIterator[tuple[httpx.AsyncClient, _CountingStubBackend]]:- """Like served_client but exposes a call-counting backend for short-circuit checks."""- captured: dict[str, Any] = {}-- def _fake_run(app: FastAPI, **kwargs: Any) -> None:- captured["app"] = app- captured["kwargs"] = kwargs-- import uvicorn-- monkeypatch.setattr(uvicorn, "run", _fake_run)- backend = _CountingStubBackend()- sw = Switchyard(backend=backend, translator=TranslationEngine())- server_util.build_and_serve(_ns(), sw)- async with httpx.AsyncClient(- transport=httpx.ASGITransport(app=captured["app"]),- base_url="http://test",- ) as client:- yield client, backend---class TestEmptyMessages:- """Empty messages array must short-circuit with a structured 400 before reaching the backend.-- REQ-AG3: agents must distinguish client-side errors (no retry) from server failures- (retry). An empty messages array is a client bug — reporting it as 500 breaks- agent retry logic. This class pins the fix and confirms dispatch is skipped.- """-- async def test_openai_chat_empty_messages_returns_400(- self,- counting_client: tuple[httpx.AsyncClient, _CountingStubBackend],- ) -> None:- client, backend = counting_client- resp = await client.post(- "/v1/chat/completions",- json={"model": "any", "messages": []},- )- assert resp.status_code == 400- assert resp.headers["content-type"].startswith("application/json")- body = resp.json()- assert body["error"]["type"] == "invalid_request_error"- assert body["error"]["code"] == "empty_messages"- assert "messages" in body["error"]["message"]- assert backend.call_count == 0, "backend must not be invoked for empty messages"-- async def test_anthropic_messages_empty_messages_returns_400(- self,- counting_client: tuple[httpx.AsyncClient, _CountingStubBackend],- ) -> None:- client, backend = counting_client- resp = await client.post(- "/v1/messages",- json={"model": "any", "max_tokens": 16, "messages": []},- )- assert resp.status_code == 400- assert resp.headers["content-type"].startswith("application/json")- body = resp.json()- assert body["error"]["type"] == "invalid_request_error"- assert body["error"]["code"] == "empty_messages"- assert backend.call_count == 0, "backend must not be invoked for empty messages"-- async def test_non_empty_messages_still_succeed(- self,- counting_client: tuple[httpx.AsyncClient, _CountingStubBackend],- ) -> None:- client, backend = counting_client- resp = await client.post(- "/v1/chat/completions",- json={"model": "any", "messages": [{"role": "user", "content": "hi"}]},- )- assert resp.status_code == 200, resp.text- assert backend.call_count == 1-- async def test_server_stays_healthy_after_empty_messages(- self,- counting_client: tuple[httpx.AsyncClient, _CountingStubBackend],- ) -> None:- client, _ = counting_client- await client.post("/v1/chat/completions", json={"model": "any", "messages": []})- resp = await client.get("/health")- assert resp.status_code == 200diff --git a/tests/test_chat_request.py b/tests/test_chat_request.pydeleted file mode 100644index b315e94f9..000000000--- a/tests/test_chat_request.py+++ /dev/null@@ -1,213 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Tests for Rust-backed ChatRequest values."""--import pytest--from switchyard_rust.core import (- ChatRequest,- ChatRequestType,- request_type_enum,- request_type_matches,-)---class TestChatRequestType:- def test_enum_values(self):- assert ChatRequestType.OPENAI_CHAT.value == "openai_chat"- assert ChatRequestType.OPENAI_RESPONSES.value == "openai_responses"- assert ChatRequestType.ANTHROPIC.value == "anthropic"-- def test_enum_members(self):- assert [- request_type_enum("openai_chat").value,- request_type_enum("openai_responses").value,- request_type_enum("anthropic").value,- ] == ["openai_chat", "openai_responses", "anthropic"]-- def test_unknown_request_type_is_rejected(self):- with pytest.raises(ValueError):- request_type_enum("not_a_real_format")---class TestChatRequestConstructor:- def test_cannot_instantiate(self):- with pytest.raises(TypeError):- ChatRequest() # type: ignore[abstract]---class TestOpenAIChatBinding:- @pytest.fixture()- def body(self):- return {- "model": "gpt-4o",- "messages": [{"role": "user", "content": "Hello"}],- }-- def test_request_type(self, body):- req = ChatRequest.openai_chat(body)- assert request_type_matches(req, ChatRequestType.OPENAI_CHAT)-- def test_body_access(self, body):- req = ChatRequest.openai_chat(body)- assert req.body == body- assert req.body is not body-- def test_body_field_access(self, body):- req = ChatRequest.openai_chat(body)- assert req.body["model"] == "gpt-4o"- assert req.body["messages"][0]["role"] == "user"-- def test_isinstance(self, body):- req = ChatRequest.openai_chat(body)- assert isinstance(req, ChatRequest)- assert request_type_matches(req, ChatRequestType.OPENAI_CHAT)-- def test_body_unpack(self, body):- """Verify **request.body works for SDK create() calls."""- req = ChatRequest.openai_chat(body)- unpacked = {**req.body}- assert unpacked == body-- def test_body_mutation(self, body):- req = ChatRequest.openai_chat(body)- exported = req.body- exported["model"] = "mutating-exported-copy-does-not-write-through"- assert req.body["model"] == "gpt-4o"- req.set_model("gpt-4o-mini")- assert req.body["model"] == "gpt-4o-mini"---class TestResponsesChatBinding:- @pytest.fixture()- def body(self):- return {- "model": "gpt-4o",- "input": "Hello, world!",- "instructions": "Be helpful.",- }-- def test_request_type(self, body):- req = ChatRequest.openai_responses(body)- assert request_type_matches(req, ChatRequestType.OPENAI_RESPONSES)-- def test_body_access(self, body):- req = ChatRequest.openai_responses(body)- assert req.body == body- assert req.body is not body-- def test_body_field_access(self, body):- req = ChatRequest.openai_responses(body)- assert req.body["model"] == "gpt-4o"- assert req.body["input"] == "Hello, world!"- assert req.body["instructions"] == "Be helpful."-- def test_isinstance(self, body):- req = ChatRequest.openai_responses(body)- assert isinstance(req, ChatRequest)- assert request_type_matches(req, ChatRequestType.OPENAI_RESPONSES)-- def test_responses_specific_fields(self):- body = {- "model": "gpt-4o",- "input": "Continue",- "previous_response_id": "resp_abc123",- "truncation": "auto",- }- req = ChatRequest.openai_responses(body)- assert req.body["previous_response_id"] == "resp_abc123"- assert req.body["truncation"] == "auto"---class TestAnthropicChatBinding:- @pytest.fixture()- def body(self):- return {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "Hello"}],- "max_tokens": 1024,- }-- def test_request_type(self, body):- req = ChatRequest.anthropic(body)- assert request_type_matches(req, ChatRequestType.ANTHROPIC)-- def test_body_access(self, body):- req = ChatRequest.anthropic(body)- assert req.body == body- assert req.body is not body-- def test_body_field_access(self, body):- req = ChatRequest.anthropic(body)- assert req.body["model"] == "claude-sonnet-4-20250514"- assert req.body["max_tokens"] == 1024-- def test_isinstance(self, body):- req = ChatRequest.anthropic(body)- assert isinstance(req, ChatRequest)- assert request_type_matches(req, ChatRequestType.ANTHROPIC)-- def test_anthropic_specific_fields(self):- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "Hi"}],- "max_tokens": 2048,- "system": "You are a helpful assistant.",- "stream": True,- }- req = ChatRequest.anthropic(body)- assert req.body["system"] == "You are a helpful assistant."- assert req.body["stream"] is True---class TestRequestTypeDispatch:- """Verify request-format dispatch uses the Rust request tag."""-- def _dispatch(self, request: ChatRequest) -> str:- if request_type_matches(request, ChatRequestType.OPENAI_CHAT):- return "openai_chat"- if request_type_matches(request, ChatRequestType.OPENAI_RESPONSES):- return "responses"- if request_type_matches(request, ChatRequestType.ANTHROPIC):- return "anthropic"- return "unknown"-- def test_dispatch_openai(self):- req = ChatRequest.openai_chat({"model": "gpt-4o", "messages": []})- assert self._dispatch(req) == "openai_chat"-- def test_dispatch_responses(self):- req = ChatRequest.openai_responses({"model": "gpt-4o", "input": "hi"})- assert self._dispatch(req) == "responses"-- def test_dispatch_anthropic(self):- req = ChatRequest.anthropic({"model": "claude-sonnet-4-20250514", "messages": [], "max_tokens": 1024})- assert self._dispatch(req) == "anthropic"---class TestMatchDispatch:- """Verify match dispatch uses the request-format enum."""-- def _dispatch(self, request: ChatRequest) -> str:- match request_type_enum(request.request_type):- case ChatRequestType.OPENAI_CHAT:- return "openai_chat"- case ChatRequestType.OPENAI_RESPONSES:- return "responses"- case ChatRequestType.ANTHROPIC:- return "anthropic"- case _:- return "unknown"-- def test_match_openai(self):- req = ChatRequest.openai_chat({"model": "gpt-4o", "messages": []})- assert self._dispatch(req) == "openai_chat"-- def test_match_responses(self):- req = ChatRequest.openai_responses({"model": "gpt-4o", "input": "hi"})- assert self._dispatch(req) == "responses"-- def test_match_anthropic(self):- req = ChatRequest.anthropic({"model": "claude-sonnet-4-20250514", "messages": [], "max_tokens": 1024})- assert self._dispatch(req) == "anthropic"diff --git a/tests/test_chat_response.py b/tests/test_chat_response.pydeleted file mode 100644index 496f3d1c9..000000000--- a/tests/test_chat_response.py+++ /dev/null@@ -1,770 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Tests for the ChatResponse type hierarchy and all stream wrappers."""--from collections.abc import AsyncIterator--import pytest-from anthropic.types import Message as AnthropicMessage-from anthropic.types import RawContentBlockDeltaEvent-from anthropic.types import Usage as AnthropicUsage-from anthropic.types.text_delta import TextDelta-from openai.types.chat import ChatCompletion, ChatCompletionChunk-from openai.types.chat.chat_completion import Choice-from openai.types.chat.chat_completion_chunk import Choice as ChunkChoice-from openai.types.chat.chat_completion_chunk import ChoiceDelta-from openai.types.chat.chat_completion_message import ChatCompletionMessage-from openai.types.completion_usage import CompletionUsage-from openai.types.responses import Response as OpenAIResponse-from openai.types.responses import ResponseTextDeltaEvent--from switchyard.lib.chat_response import AnthropicResponseStream, ResponsesApiStream, ResponseStream-from switchyard_rust.core import (- ChatResponse,- ChatResponseType,- response_type_matches,-)--# ----------------------------------------------------------------------------# Fixtures-# ------------------------------------------------------------------------------def make_completion(*, model: str = "gpt-4o", content: str = "hello") -> ChatCompletion:- return ChatCompletion(- id="chatcmpl-test",- object="chat.completion",- created=1700000000,- model=model,- choices=[- Choice(- index=0,- message=ChatCompletionMessage(role="assistant", content=content),- finish_reason="stop",- )- ],- usage=CompletionUsage(prompt_tokens=10, completion_tokens=5, total_tokens=15),- )---def make_chunk(*, index: int = 0, content: str = "hi") -> ChatCompletionChunk:- return ChatCompletionChunk(- id="chatcmpl-test",- object="chat.completion.chunk",- created=1700000000,- model="gpt-4o",- choices=[- ChunkChoice(- index=index,- delta=ChoiceDelta(content=content),- finish_reason=None,- )- ],- )---async def fake_stream(chunks: list[ChatCompletionChunk]) -> AsyncIterator[ChatCompletionChunk]:- for chunk in chunks:- yield chunk---# --- Anthropic fixtures ------def make_anthropic_message(- *, model: str = "claude-sonnet-4-20250514", text: str = "hello"-) -> AnthropicMessage:- return AnthropicMessage(- id="msg_test",- type="message",- role="assistant",- model=model,- content=[{"type": "text", "text": text}],- stop_reason="end_turn",- usage=AnthropicUsage(input_tokens=10, output_tokens=5),- )---def make_anthropic_content_delta(*, text: str = "hi") -> RawContentBlockDeltaEvent:- return RawContentBlockDeltaEvent(- type="content_block_delta",- index=0,- delta=TextDelta(type="text_delta", text=text),- )---async def fake_anthropic_stream(- events: list[RawContentBlockDeltaEvent],-) -> AsyncIterator[RawContentBlockDeltaEvent]:- for event in events:- yield event---# --- OpenAI Responses API fixtures ------def make_responses_api_response(- *, model: str = "gpt-4o", text: str = "hello"-) -> OpenAIResponse:- return OpenAIResponse(- id="resp_test",- created_at=1700000000,- model=model,- object="response",- output=[- {- "type": "message",- "id": "msg_test",- "status": "completed",- "role": "assistant",- "content": [{"type": "output_text", "text": text, "annotations": []}],- }- ],- tool_choice="auto",- tools=[],- status="completed",- parallel_tool_calls=True,- text={"format": {"type": "text"}},- )---def make_responses_text_delta(*, delta: str = "hi") -> ResponseTextDeltaEvent:- return ResponseTextDeltaEvent(- type="response.output_text.delta",- item_id="item_test",- output_index=0,- content_index=0,- delta=delta,- logprobs=[],- sequence_number=0,- )---async def fake_responses_stream(- events: list[ResponseTextDeltaEvent],-) -> AsyncIterator[ResponseTextDeltaEvent]:- for event in events:- yield event---# ----------------------------------------------------------------------------# ChatResponseType-# ------------------------------------------------------------------------------class TestChatResponseType:- def test_enum_values(self):- assert ChatResponseType.OPENAI_COMPLETION.value == "openai_completion"- assert ChatResponseType.OPENAI_STREAM.value == "openai_stream"- assert (- ChatResponseType.OPENAI_RESPONSES_COMPLETION.value- == "openai_responses_completion"- )- assert ChatResponseType.OPENAI_RESPONSES_STREAM.value == "openai_responses_stream"- assert ChatResponseType.ANTHROPIC_COMPLETION.value == "anthropic_completion"- assert ChatResponseType.ANTHROPIC_STREAM.value == "anthropic_stream"-- def test_enum_members(self):- assert [- item.value- for item in (- ChatResponseType.OPENAI_COMPLETION,- ChatResponseType.OPENAI_STREAM,- ChatResponseType.OPENAI_RESPONSES_COMPLETION,- ChatResponseType.OPENAI_RESPONSES_STREAM,- ChatResponseType.ANTHROPIC_COMPLETION,- ChatResponseType.ANTHROPIC_STREAM,- )- ] == [- "openai_completion",- "openai_stream",- "openai_responses_completion",- "openai_responses_stream",- "anthropic_completion",- "anthropic_stream",- ]---# ----------------------------------------------------------------------------# ChatResponse ABC-# ------------------------------------------------------------------------------class TestChatResponseABC:- def test_cannot_instantiate(self):- with pytest.raises(TypeError):- ChatResponse() # type: ignore[abstract]---# ----------------------------------------------------------------------------# OpenAI completion ChatResponse-# ------------------------------------------------------------------------------class TestOpenAICompletionResponse:- def test_response_type(self):- resp = ChatResponse.openai_completion(make_completion())- assert resp.response_type == ChatResponseType.OPENAI_COMPLETION-- def test_body_access(self):- completion = make_completion()- resp = ChatResponse.openai_completion(completion)- assert resp.body == completion.model_dump(mode="json", exclude_none=True)- assert resp.body is not completion-- def test_body_sdk_fields(self):- resp = ChatResponse.openai_completion(make_completion(model="gpt-4o", content="world"))- assert resp.body["model"] == "gpt-4o"- assert resp.body["choices"][0]["message"]["content"] == "world"- assert resp.body["usage"]["total_tokens"] == 15-- def test_response_type_match(self):- resp = ChatResponse.openai_completion(make_completion())- assert isinstance(resp, ChatResponse)- assert response_type_matches(resp, ChatResponseType.OPENAI_COMPLETION)- assert not response_type_matches(resp, ChatResponseType.OPENAI_STREAM)---# ----------------------------------------------------------------------------# OpenAI stream ChatResponse-# ------------------------------------------------------------------------------class TestOpenAIStreamChatResponse:- def test_response_type(self):- stream = ResponseStream(fake_stream([]))- resp = ChatResponse.openai_stream(stream)- assert resp.response_type == ChatResponseType.OPENAI_STREAM-- async def test_stream_access(self):- chunks = [make_chunk(content="streamed")]- stream = ResponseStream(fake_stream(chunks))- resp = ChatResponse.openai_stream(stream)- assert [chunk async for chunk in resp.stream] == chunks-- def test_response_type_match(self):- stream = ResponseStream(fake_stream([]))- resp = ChatResponse.openai_stream(stream)- assert isinstance(resp, ChatResponse)- assert response_type_matches(resp, ChatResponseType.OPENAI_STREAM)- assert not response_type_matches(resp, ChatResponseType.OPENAI_COMPLETION)---# ----------------------------------------------------------------------------# response type dispatch-# ------------------------------------------------------------------------------class TestIsinstanceDispatch:- def _dispatch(self, response: ChatResponse) -> str:- if response_type_matches(response, ChatResponseType.OPENAI_COMPLETION):- return "openai_completion"- if response_type_matches(response, ChatResponseType.OPENAI_STREAM):- return "openai_stream"- if response_type_matches(response, ChatResponseType.OPENAI_RESPONSES_COMPLETION):- return "responses_completion"- if response_type_matches(response, ChatResponseType.OPENAI_RESPONSES_STREAM):- return "responses_stream"- if response_type_matches(response, ChatResponseType.ANTHROPIC_COMPLETION):- return "anthropic_completion"- if response_type_matches(response, ChatResponseType.ANTHROPIC_STREAM):- return "anthropic_stream"- return "unknown"-- def test_dispatch_completion(self):- assert self._dispatch(ChatResponse.openai_completion(make_completion())) == "openai_completion"-- def test_dispatch_stream(self):- resp = ChatResponse.openai_stream(ResponseStream(fake_stream([])))- assert self._dispatch(resp) == "openai_stream"-- def test_dispatch_responses_completion(self):- resp = ChatResponse.openai_responses_completion(make_responses_api_response())- assert self._dispatch(resp) == "responses_completion"-- def test_dispatch_responses_stream(self):- stream = ResponsesApiStream(fake_responses_stream([]))- resp = ChatResponse.openai_responses_stream(stream)- assert self._dispatch(resp) == "responses_stream"-- def test_dispatch_anthropic_completion(self):- assert self._dispatch(ChatResponse.anthropic_completion(make_anthropic_message())) == "anthropic_completion"-- def test_dispatch_anthropic_stream(self):- stream = AnthropicResponseStream(fake_anthropic_stream([]))- resp = ChatResponse.anthropic_stream(stream)- assert self._dispatch(resp) == "anthropic_stream"---# ----------------------------------------------------------------------------# ResponseStream-# ------------------------------------------------------------------------------class TestResponseStream:- async def test_basic_iteration(self):- chunks = [make_chunk(content="a"), make_chunk(content="b")]- stream = ResponseStream(fake_stream(chunks))- result = [chunk async for chunk in stream]- assert len(result) == 2- assert result[0].choices[0].delta.content == "a"- assert result[1].choices[0].delta.content == "b"-- async def test_empty_stream(self):- stream = ResponseStream(fake_stream([]))- result = [chunk async for chunk in stream]- assert result == []-- async def test_single_consumption(self):- stream = ResponseStream(fake_stream([make_chunk()]))- _ = [chunk async for chunk in stream]- with pytest.raises(RuntimeError, match="already been consumed"):- _ = [chunk async for chunk in stream]-- async def test_tap_observes_all_chunks(self):- observed: list[str] = []-- async def log_tap(chunk: ChatCompletionChunk) -> None:- observed.append(chunk.choices[0].delta.content or "")-- chunks = [make_chunk(content="x"), make_chunk(content="y")]- stream = ResponseStream(fake_stream(chunks))- stream.tap(log_tap)-- _ = [chunk async for chunk in stream]- assert observed == ["x", "y"]-- async def test_map_transforms_chunks(self):- async def upper_map(chunk: ChatCompletionChunk) -> ChatCompletionChunk:- chunk.choices[0].delta.content = (chunk.choices[0].delta.content or "").upper()- return chunk-- stream = ResponseStream(fake_stream([make_chunk(content="hello")]))- stream.map(upper_map)-- result = [chunk async for chunk in stream]- assert result[0].choices[0].delta.content == "HELLO"-- async def test_tap_sees_original_before_map(self):- tap_saw: list[str] = []-- async def observe(chunk: ChatCompletionChunk) -> None:- tap_saw.append(chunk.choices[0].delta.content or "")-- async def transform(chunk: ChatCompletionChunk) -> ChatCompletionChunk:- chunk.choices[0].delta.content = "TRANSFORMED"- return chunk-- stream = ResponseStream(fake_stream([make_chunk(content="original")]))- stream.tap(observe)- stream.map(transform)-- result = [chunk async for chunk in stream]- assert tap_saw == ["original"]- assert result[0].choices[0].delta.content == "TRANSFORMED"-- async def test_multiple_maps_compose(self):- async def add_exclaim(chunk: ChatCompletionChunk) -> ChatCompletionChunk:- chunk.choices[0].delta.content = (chunk.choices[0].delta.content or "") + "!"- return chunk-- async def add_question(chunk: ChatCompletionChunk) -> ChatCompletionChunk:- chunk.choices[0].delta.content = (chunk.choices[0].delta.content or "") + "?"- return chunk-- stream = ResponseStream(fake_stream([make_chunk(content="hi")]))- stream.map(add_exclaim).map(add_question)-- result = [chunk async for chunk in stream]- assert result[0].choices[0].delta.content == "hi!?"-- async def test_tap_exception_does_not_break_stream(self):- async def bad_tap(chunk: ChatCompletionChunk) -> None:- raise ValueError("tap exploded")-- stream = ResponseStream(fake_stream([make_chunk(content="safe")]))- stream.tap(bad_tap)-- result = [chunk async for chunk in stream]- assert len(result) == 1- assert result[0].choices[0].delta.content == "safe"-- async def test_failing_tap_is_quarantined(self):- call_count = 0-- async def flaky_tap(chunk: ChatCompletionChunk) -> None:- nonlocal call_count- call_count += 1- raise ValueError("always fails")-- chunks = [make_chunk(content="a"), make_chunk(content="b"), make_chunk(content="c")]- stream = ResponseStream(fake_stream(chunks))- stream.tap(flaky_tap)-- result = [chunk async for chunk in stream]- assert len(result) == 3- assert call_count == 1 # called once, then quarantined-- async def test_tap_returns_self_for_chaining(self):- async def noop(chunk: ChatCompletionChunk) -> None:- pass-- stream = ResponseStream(fake_stream([]))- assert stream.tap(noop) is stream-- async def test_map_returns_self_for_chaining(self):- async def identity(chunk: ChatCompletionChunk) -> ChatCompletionChunk:- return chunk-- stream = ResponseStream(fake_stream([]))- assert stream.map(identity) is stream-- async def test_multiple_taps(self):- seen_a: list[str] = []- seen_b: list[str] = []-- async def tap_a(chunk: ChatCompletionChunk) -> None:- seen_a.append(chunk.choices[0].delta.content or "")-- async def tap_b(chunk: ChatCompletionChunk) -> None:- seen_b.append(chunk.choices[0].delta.content or "")-- stream = ResponseStream(fake_stream([make_chunk(content="x")]))- stream.tap(tap_a).tap(tap_b)-- _ = [chunk async for chunk in stream]- assert seen_a == ["x"]- assert seen_b == ["x"]---# ----------------------------------------------------------------------------# OpenAI Responses completion ChatResponse-# ------------------------------------------------------------------------------class TestOpenAIResponsesCompletionResponse:- def test_response_type(self):- resp = ChatResponse.openai_responses_completion(make_responses_api_response())- assert resp.response_type == ChatResponseType.OPENAI_RESPONSES_COMPLETION-- def test_body_access(self):- body = make_responses_api_response()- resp = ChatResponse.openai_responses_completion(body)- assert resp.body == body.model_dump(mode="json", exclude_none=True)- assert resp.body is not body-- def test_body_sdk_fields(self):- resp = ChatResponse.openai_responses_completion(- make_responses_api_response(model="gpt-4o", text="world")- )- assert resp.body["model"] == "gpt-4o"- assert resp.body["output"][0]["content"][0]["text"] == "world"- assert resp.body["status"] == "completed"-- def test_response_type_match(self):- resp = ChatResponse.openai_responses_completion(make_responses_api_response())- assert isinstance(resp, ChatResponse)- assert response_type_matches(resp, ChatResponseType.OPENAI_RESPONSES_COMPLETION)- assert not response_type_matches(resp, ChatResponseType.OPENAI_COMPLETION)- assert not response_type_matches(resp, ChatResponseType.OPENAI_STREAM)- assert not response_type_matches(resp, ChatResponseType.ANTHROPIC_COMPLETION)---# ----------------------------------------------------------------------------# OpenAI Responses stream ChatResponse-# ------------------------------------------------------------------------------class TestOpenAIResponsesStreamChatResponse:- def test_response_type(self):- stream = ResponsesApiStream(fake_responses_stream([]))- resp = ChatResponse.openai_responses_stream(stream)- assert resp.response_type == ChatResponseType.OPENAI_RESPONSES_STREAM-- async def test_stream_access(self):- events = [make_responses_text_delta(delta="streamed")]- stream = ResponsesApiStream(fake_responses_stream(events))- resp = ChatResponse.openai_responses_stream(stream)- assert [event async for event in resp.stream] == events-- def test_response_type_match(self):- stream = ResponsesApiStream(fake_responses_stream([]))- resp = ChatResponse.openai_responses_stream(stream)- assert isinstance(resp, ChatResponse)- assert response_type_matches(resp, ChatResponseType.OPENAI_RESPONSES_STREAM)- assert not response_type_matches(resp, ChatResponseType.OPENAI_COMPLETION)- assert not response_type_matches(resp, ChatResponseType.OPENAI_RESPONSES_COMPLETION)---# ----------------------------------------------------------------------------# ResponsesApiStream-# ------------------------------------------------------------------------------class TestResponsesApiStream:- async def test_basic_iteration(self):- events = [make_responses_text_delta(delta="a"), make_responses_text_delta(delta="b")]- stream = ResponsesApiStream(fake_responses_stream(events))- result = [event async for event in stream]- assert len(result) == 2- assert result[0].delta == "a"- assert result[1].delta == "b"-- async def test_empty_stream(self):- stream = ResponsesApiStream(fake_responses_stream([]))- result = [event async for event in stream]- assert result == []-- async def test_single_consumption(self):- stream = ResponsesApiStream(fake_responses_stream([make_responses_text_delta()]))- _ = [event async for event in stream]- with pytest.raises(RuntimeError, match="already been consumed"):- _ = [event async for event in stream]-- async def test_tap_observes_all_events(self):- observed: list[str] = []-- async def log_tap(event: ResponseTextDeltaEvent) -> None:- observed.append(event.delta)-- events = [make_responses_text_delta(delta="x"), make_responses_text_delta(delta="y")]- stream = ResponsesApiStream(fake_responses_stream(events))- stream.tap(log_tap)-- _ = [event async for event in stream]- assert observed == ["x", "y"]-- async def test_tap_exception_does_not_break_stream(self):- async def bad_tap(event: ResponseTextDeltaEvent) -> None:- raise ValueError("tap exploded")-- stream = ResponsesApiStream(- fake_responses_stream([make_responses_text_delta(delta="safe")])- )- stream.tap(bad_tap)-- result = [event async for event in stream]- assert len(result) == 1- assert result[0].delta == "safe"-- async def test_failing_tap_is_quarantined(self):- call_count = 0-- async def flaky_tap(event: ResponseTextDeltaEvent) -> None:- nonlocal call_count- call_count += 1- raise ValueError("always fails")-- events = [- make_responses_text_delta(delta="a"),- make_responses_text_delta(delta="b"),- make_responses_text_delta(delta="c"),- ]- stream = ResponsesApiStream(fake_responses_stream(events))- stream.tap(flaky_tap)-- result = [event async for event in stream]- assert len(result) == 3- assert call_count == 1-- async def test_tap_returns_self_for_chaining(self):- async def noop(event: ResponseTextDeltaEvent) -> None:- pass-- stream = ResponsesApiStream(fake_responses_stream([]))- assert stream.tap(noop) is stream-- async def test_map_returns_self_for_chaining(self):- async def identity(event: ResponseTextDeltaEvent) -> ResponseTextDeltaEvent:- return event-- stream = ResponsesApiStream(fake_responses_stream([]))- assert stream.map(identity) is stream---# ----------------------------------------------------------------------------# Anthropic completion ChatResponse-# ------------------------------------------------------------------------------class TestAnthropicCompletionResponse:- def test_response_type(self):- resp = ChatResponse.anthropic_completion(make_anthropic_message())- assert resp.response_type == ChatResponseType.ANTHROPIC_COMPLETION-- def test_body_access(self):- msg = make_anthropic_message()- resp = ChatResponse.anthropic_completion(msg)- assert resp.body == msg.model_dump(mode="json", exclude_none=True)- assert resp.body is not msg-- def test_body_sdk_fields(self):- resp = ChatResponse.anthropic_completion(- make_anthropic_message(model="claude-sonnet-4-20250514", text="world")- )- assert resp.body["model"] == "claude-sonnet-4-20250514"- assert resp.body["content"][0]["text"] == "world"- assert resp.body["usage"]["input_tokens"] == 10- assert resp.body["usage"]["output_tokens"] == 5- assert resp.body["stop_reason"] == "end_turn"-- def test_response_type_match(self):- resp = ChatResponse.anthropic_completion(make_anthropic_message())- assert isinstance(resp, ChatResponse)- assert response_type_matches(resp, ChatResponseType.ANTHROPIC_COMPLETION)- assert not response_type_matches(resp, ChatResponseType.OPENAI_COMPLETION)- assert not response_type_matches(resp, ChatResponseType.OPENAI_STREAM)- assert not response_type_matches(resp, ChatResponseType.ANTHROPIC_STREAM)---# ----------------------------------------------------------------------------# Anthropic stream ChatResponse-# ------------------------------------------------------------------------------class TestAnthropicStreamChatResponse:- def test_response_type(self):- stream = AnthropicResponseStream(fake_anthropic_stream([]))- resp = ChatResponse.anthropic_stream(stream)- assert resp.response_type == ChatResponseType.ANTHROPIC_STREAM-- async def test_stream_access(self):- events = [make_anthropic_content_delta(text="streamed")]- stream = AnthropicResponseStream(fake_anthropic_stream(events))- resp = ChatResponse.anthropic_stream(stream)- assert [event async for event in resp.stream] == events-- def test_response_type_match(self):- stream = AnthropicResponseStream(fake_anthropic_stream([]))- resp = ChatResponse.anthropic_stream(stream)- assert isinstance(resp, ChatResponse)- assert response_type_matches(resp, ChatResponseType.ANTHROPIC_STREAM)- assert not response_type_matches(resp, ChatResponseType.OPENAI_COMPLETION)- assert not response_type_matches(resp, ChatResponseType.OPENAI_STREAM)- assert not response_type_matches(resp, ChatResponseType.ANTHROPIC_COMPLETION)---# ----------------------------------------------------------------------------# AnthropicResponseStream-# ------------------------------------------------------------------------------class TestAnthropicResponseStream:- async def test_basic_iteration(self):- events = [make_anthropic_content_delta(text="a"), make_anthropic_content_delta(text="b")]- stream = AnthropicResponseStream(fake_anthropic_stream(events))- result = [event async for event in stream]- assert len(result) == 2- assert result[0].delta.text == "a"- assert result[1].delta.text == "b"-- async def test_empty_stream(self):- stream = AnthropicResponseStream(fake_anthropic_stream([]))- result = [event async for event in stream]- assert result == []-- async def test_single_consumption(self):- stream = AnthropicResponseStream(fake_anthropic_stream([make_anthropic_content_delta()]))- _ = [event async for event in stream]- with pytest.raises(RuntimeError, match="already been consumed"):- _ = [event async for event in stream]-- async def test_tap_observes_all_events(self):- observed: list[str] = []-- async def log_tap(event: RawContentBlockDeltaEvent) -> None:- observed.append(event.delta.text)-- events = [make_anthropic_content_delta(text="x"), make_anthropic_content_delta(text="y")]- stream = AnthropicResponseStream(fake_anthropic_stream(events))- stream.tap(log_tap)-- _ = [event async for event in stream]- assert observed == ["x", "y"]-- async def test_map_transforms_events(self):- async def upper_map(event: RawContentBlockDeltaEvent) -> RawContentBlockDeltaEvent:- return RawContentBlockDeltaEvent(- type="content_block_delta",- index=event.index,- delta=TextDelta(type="text_delta", text=event.delta.text.upper()),- )-- stream = AnthropicResponseStream(- fake_anthropic_stream([make_anthropic_content_delta(text="hello")])- )- stream.map(upper_map)-- result = [event async for event in stream]- assert result[0].delta.text == "HELLO"-- async def test_tap_sees_original_before_map(self):- tap_saw: list[str] = []-- async def observe(event: RawContentBlockDeltaEvent) -> None:- tap_saw.append(event.delta.text)-- async def transform(event: RawContentBlockDeltaEvent) -> RawContentBlockDeltaEvent:- return RawContentBlockDeltaEvent(- type="content_block_delta",- index=event.index,- delta=TextDelta(type="text_delta", text="TRANSFORMED"),- )-- stream = AnthropicResponseStream(- fake_anthropic_stream([make_anthropic_content_delta(text="original")])- )- stream.tap(observe)- stream.map(transform)-- result = [event async for event in stream]- assert tap_saw == ["original"]- assert result[0].delta.text == "TRANSFORMED"-- async def test_tap_exception_does_not_break_stream(self):- async def bad_tap(event: RawContentBlockDeltaEvent) -> None:- raise ValueError("tap exploded")-- stream = AnthropicResponseStream(- fake_anthropic_stream([make_anthropic_content_delta(text="safe")])- )- stream.tap(bad_tap)-- result = [event async for event in stream]- assert len(result) == 1- assert result[0].delta.text == "safe"-- async def test_failing_tap_is_quarantined(self):- call_count = 0-- async def flaky_tap(event: RawContentBlockDeltaEvent) -> None:- nonlocal call_count- call_count += 1- raise ValueError("always fails")-- events = [- make_anthropic_content_delta(text="a"),- make_anthropic_content_delta(text="b"),- make_anthropic_content_delta(text="c"),- ]- stream = AnthropicResponseStream(fake_anthropic_stream(events))- stream.tap(flaky_tap)-- result = [event async for event in stream]- assert len(result) == 3- assert call_count == 1-- async def test_tap_returns_self_for_chaining(self):- async def noop(event: RawContentBlockDeltaEvent) -> None:- pass-- stream = AnthropicResponseStream(fake_anthropic_stream([]))- assert stream.tap(noop) is stream-- async def test_map_returns_self_for_chaining(self):- async def identity(event: RawContentBlockDeltaEvent) -> RawContentBlockDeltaEvent:- return event-- stream = AnthropicResponseStream(fake_anthropic_stream([]))- assert stream.map(identity) is streamdiff --git a/tests/test_cli_reference_docs.py b/tests/test_cli_reference_docs.pyindex 682ade769..857e3eb99 100644--- a/tests/test_cli_reference_docs.py+++ b/tests/test_cli_reference_docs.py@@ -38,7 +38,6 @@ def test_reference_documents_launcher_before_server() -> None:related = text.index("## Related Documentation")assert launcher < server < removed < related- assert "switchyard serve" not in textdef test_reference_marks_removed_setup_commands() -> None:@@ -48,10 +47,13 @@ def test_reference_marks_removed_setup_commands() -> None:removed = text[removed_start:related_start]commands = _subparsers(_build_parser())- assert "`switchyard configure` and `switchyard verify` are not available" in removed+ assert (+ "`switchyard configure`, `switchyard serve`, and `switchyard verify` are not"+ in removed+ )assert "api_key_env" in removed- assert "The CLI does not save provider credentials or deployment paths" in removed- assert {"configure", "verify"}.isdisjoint(commands)+ assert "The CLI does not save provider credentials, deployment paths" in removed+ assert {"configure", "serve", "verify"}.isdisjoint(commands)def test_reference_lists_launcher_contract() -> None:diff --git a/tests/test_codex_multiturn_traces.py b/tests/test_codex_multiturn_traces.pydeleted file mode 100644index d238e33b6..000000000--- a/tests/test_codex_multiturn_traces.py+++ /dev/null@@ -1,510 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Diagnostic tests for Codex multi-turn tool call history merging.--Uses actual trace data from tmp/output/logs to verify whether the observed-"missing intermediate assistant-tool turns" is:- (a) Codex CLI behavior (client sends merged history)- (b) A bug in our Responses API → Chat Completions translation layer- (c) User misinterpretation of the logs--CONCLUSION: It's (b) — a bug in Responses input-item translation. The-translator didn't flush the pending tool block at turn boundaries-(function_call_output → function_call transition), so tool calls across turns-were merged into one assistant message.-"""--import json-from pathlib import Path--import pytest--from switchyard_rust.translation import TranslationEngine--TRACE_DIR = Path(__file__).parent.parent / "tmp" / "output" / "logs" / "openai"-ENGINE = TranslationEngine()---def _responses_items_to_messages(items: list) -> list:- body = ENGINE.translate_request("openai_responses", "openai_chat", {"input": items})- return list(body.get("messages", []))---# ----------------------------------------------------------------------------# Helper: load trace files sorted by timestamp-# -----------------------------------------------------------------------------def _load_traces():- """Load all trace JSON files from the output directory, sorted by timestamp."""- if not TRACE_DIR.exists():- pytest.skip(f"Trace directory not found: {TRACE_DIR}")- files = sorted(TRACE_DIR.glob("*.json"))- if not files:- pytest.skip("No trace files found")- traces = []- for f in files:- with open(f) as fh:- traces.append(json.load(fh))- return traces---# ----------------------------------------------------------------------------# Test 1: Verify what the traces actually show (structural analysis)-# -----------------------------------------------------------------------------class TestTraceStructuralAnalysis:- """Analyze the actual trace files to understand the conversation structure."""-- def test_trace_count(self):- """There should be 4 traces for this Codex session."""- traces = _load_traces()- assert len(traces) == 4, f"Expected 4 traces, got {len(traces)}"-- def test_trace1_is_initial_request(self):- """Trace 1: Initial request with only user messages, no history."""- traces = _load_traces()- t1 = traces[0]- msgs = t1["request"]["messages"]-- # Should have: system, developer, user(AGENTS.md), user(env), user(task)- roles = [m["role"] for m in msgs]- assert roles == ["system", "developer", "user", "user", "user"]-- # Response should have 2 tool_calls (update_plan + apply_patch)- response_tool_calls = t1["response"]["choices"][0]["message"]["tool_calls"]- assert len(response_tool_calls) == 2- names = [tc["function"]["name"] for tc in response_tool_calls]- assert names == ["update_plan", "apply_patch"]-- def test_trace2_has_turn1_history(self):- """Trace 2: Should include turn 1's assistant + tool results as history."""- traces = _load_traces()- t2 = traces[1]- msgs = t2["request"]["messages"]-- roles = [m["role"] for m in msgs]- # Expected: system, developer, user, user, user, assistant, tool, tool- assert roles == ["system", "developer", "user", "user", "user",- "assistant", "tool", "tool"]-- # The assistant message should have exactly 2 tool_calls (from turn 1)- assistant_msg = msgs[5]- assert len(assistant_msg["tool_calls"]) == 2- names = [tc["function"]["name"] for tc in assistant_msg["tool_calls"]]- assert names == ["update_plan", "apply_patch"]-- def test_trace3_shows_merged_turns_pre_fix(self):- """Trace 3: Captured before fix — turns 1+2 merged into single assistant msg.-- These traces were recorded with the old buggy translation layer.- After the fix in _responses_items_to_messages(), new traces- would show separate assistant messages per turn.- """- traces = _load_traces()- t3 = traces[2]- msgs = t3["request"]["messages"]-- [m["role"] for m in msgs]-- # Count assistant messages in the history- assistant_msgs = [m for m in msgs if m["role"] == "assistant"]- tool_msgs = [m for m in msgs if m["role"] == "tool"]-- # BUG: There's only 1 assistant message when there should be 2- assert len(assistant_msgs) == 1, (- f"BUG CONFIRMED: Only {len(assistant_msgs)} assistant message(s) — "- f"turns 1+2 are merged into a single assistant message"- )-- # The single assistant message has 9 tool_calls (2 from turn 1 + 7 from turn 2)- merged_tool_calls = assistant_msgs[0]["tool_calls"]- assert len(merged_tool_calls) == 9, (- f"Expected 9 merged tool_calls (2+7), got {len(merged_tool_calls)}"- )-- # Verify the first 2 are from turn 1- assert merged_tool_calls[0]["function"]["name"] == "update_plan"- assert merged_tool_calls[1]["function"]["name"] == "apply_patch"- # The remaining 7 are from turn 2- turn2_names = [tc["function"]["name"] for tc in merged_tool_calls[2:]]- assert all(n == "exec_command" for n in turn2_names)-- # There should be 9 tool messages too- assert len(tool_msgs) == 9-- def test_trace4_shows_all_turns_merged_pre_fix(self):- """Trace 4: Captured before fix — all turns merged into single assistant msg."""- traces = _load_traces()- t4 = traces[3]- msgs = t4["request"]["messages"]-- assistant_msgs = [m for m in msgs if m["role"] == "assistant"]- [m for m in msgs if m["role"] == "tool"]-- # BUG: Still only 1 assistant message- assert len(assistant_msgs) == 1, (- f"BUG CONFIRMED: Only {len(assistant_msgs)} assistant message(s) — "- f"all turns merged"- )-- # Should have even more tool_calls (2 + 7 + 3 = 12 from turn 3 additions)- total_tool_calls = len(assistant_msgs[0]["tool_calls"])- assert total_tool_calls > 9, (- f"Expected >9 tool_calls in merged message, got {total_tool_calls}"- )---# ----------------------------------------------------------------------------# Test 2: Reproduce the bug with reconstructed Responses API input-# -----------------------------------------------------------------------------class TestTranslationLayerMultiTurnBug:- """Reproduce the turn-merging bug using _responses_items_to_messages()- with input items reconstructed from the trace data."""-- @staticmethod- def _make_two_turn_input():- """Reconstruct the Responses API input Codex would send for trace 3.-- Turn 1: 2 parallel tool calls (update_plan + apply_patch) + results- Turn 2: 7 parallel tool calls (5× ls + 2× echo) + results-- This is the standard Responses API format: all calls from a turn- grouped together, then all outputs, then next turn's calls, etc.- """- return [- # --- User message ---- {"type": "message", "role": "user", "content": "Create a Python script"},-- # --- Turn 1: 2 tool calls ---- {- "type": "function_call",- "name": "update_plan",- "call_id": "call_turn1_a",- "arguments": '{"plan": [{"step": "Create script", "status": "completed"}]}',- },- {- "type": "function_call",- "name": "apply_patch",- "call_id": "call_turn1_b",- "arguments": '{"command": "*** Begin Patch..."}',- },- # --- Turn 1: results ---- {- "type": "function_call_output",- "call_id": "call_turn1_a",- "output": "Plan updated",- },- {- "type": "function_call_output",- "call_id": "call_turn1_b",- "output": "unsupported call: apply_patch",- },-- # --- Turn 2: 7 tool calls ---- {- "type": "function_call",- "name": "exec_command",- "call_id": "call_turn2_a",- "arguments": '{"cmd": "ls -R ."}',- },- {- "type": "function_call",- "name": "exec_command",- "call_id": "call_turn2_b",- "arguments": '{"cmd": "ls -R ."}',- },- {- "type": "function_call",- "name": "exec_command",- "call_id": "call_turn2_c",- "arguments": '{"cmd": "ls -R ."}',- },- {- "type": "function_call",- "name": "exec_command",- "call_id": "call_turn2_d",- "arguments": '{"cmd": "ls -R ."}',- },- {- "type": "function_call",- "name": "exec_command",- "call_id": "call_turn2_e",- "arguments": '{"cmd": "ls -R .", "max_output_tokens": 2000}',- },- {- "type": "function_call",- "name": "exec_command",- "call_id": "call_turn2_f",- "arguments": '{"cmd": "echo hello", "max_output_tokens": 20}',- },- {- "type": "function_call",- "name": "exec_command",- "call_id": "call_turn2_g",- "arguments": '{"cmd": "echo hello", "shell": "/bin/bash"}',- },- # --- Turn 2: results ---- {- "type": "function_call_output",- "call_id": "call_turn2_a",- "output": ".:\ninput.json\nlogs\n",- },- {- "type": "function_call_output",- "call_id": "call_turn2_b",- "output": ".:\ninput.json\nlogs\n",- },- {- "type": "function_call_output",- "call_id": "call_turn2_c",- "output": ".:\ninput.json\nlogs\n",- },- {- "type": "function_call_output",- "call_id": "call_turn2_d",- "output": ".:\ninput.json\nlogs\n",- },- {- "type": "function_call_output",- "call_id": "call_turn2_e",- "output": ".:\ninput.json\nlogs\n",- },- {- "type": "function_call_output",- "call_id": "call_turn2_f",- "output": "hello\n",- },- {- "type": "function_call_output",- "call_id": "call_turn2_g",- "output": "hello\n",- },- ]-- def test_two_turns_produce_separate_assistant_messages(self):- """Two turns of tool calls produce separate assistant messages.-- The proxy detects the function_call_output → function_call turn- boundary and flushes, producing:- user → assistant(2 calls) → 2 tools → assistant(7 calls) → 7 tools- """- items = self._make_two_turn_input()- messages = _responses_items_to_messages(items)-- user_msgs = [m for m in messages if m["role"] == "user"]- asst_msgs = [m for m in messages if m["role"] == "assistant"]- tool_msgs = [m for m in messages if m["role"] == "tool"]-- assert len(user_msgs) == 1- assert len(asst_msgs) == 2, "Should have 2 assistant messages (one per turn)"- assert len(asst_msgs[0]["tool_calls"]) == 2, "Turn 1: 2 tool calls"- assert len(asst_msgs[1]["tool_calls"]) == 7, "Turn 2: 7 tool calls"- assert len(tool_msgs) == 9, "Total 9 tool results"-- # Verify message ordering- expected_roles = [- "user", # original request- "assistant", # turn 1: 2 tool calls- "tool", # turn 1 result 1- "tool", # turn 1 result 2- "assistant", # turn 2: 7 tool calls- "tool", # turn 2 results...- "tool",- "tool",- "tool",- "tool",- "tool",- "tool",- ]- actual_roles = [m["role"] for m in messages]- assert actual_roles == expected_roles-- def test_single_turn_still_works(self):- """Single-turn tool calls should still be merged into one assistant message."""- items = [- {"type": "message", "role": "user", "content": "Do two things"},- {- "type": "function_call",- "name": "tool_a",- "call_id": "call_a",- "arguments": "{}",- },- {- "type": "function_call",- "name": "tool_b",- "call_id": "call_b",- "arguments": "{}",- },- {- "type": "function_call_output",- "call_id": "call_a",- "output": "result_a",- },- {- "type": "function_call_output",- "call_id": "call_b",- "output": "result_b",- },- ]- messages = _responses_items_to_messages(items)-- asst_msgs = [m for m in messages if m["role"] == "assistant"]- assert len(asst_msgs) == 1- assert len(asst_msgs[0]["tool_calls"]) == 2-- def test_three_turns_produce_three_assistant_messages(self):- """Three turns of tool calls produce 3 separate assistant messages.-- Reconstructs what Codex sends for trace 4 (turn 1 + turn 2 + turn 3).- """- items = [- {"type": "message", "role": "user", "content": "Create a script"},-- # Turn 1: 2 calls- {"type": "function_call", "name": "update_plan", "call_id": "t1_a", "arguments": "{}"},- {"type": "function_call", "name": "apply_patch", "call_id": "t1_b", "arguments": "{}"},- {"type": "function_call_output", "call_id": "t1_a", "output": "Plan updated"},- {"type": "function_call_output", "call_id": "t1_b", "output": "unsupported"},-- # Turn 2: 3 calls- {"type": "function_call", "name": "exec_command", "call_id": "t2_a", "arguments": '{"cmd":"ls"}'},- {"type": "function_call", "name": "exec_command", "call_id": "t2_b", "arguments": '{"cmd":"echo hi"}'},- {"type": "function_call", "name": "exec_command", "call_id": "t2_c", "arguments": '{"cmd":"pwd"}'},- {"type": "function_call_output", "call_id": "t2_a", "output": "file1 file2"},- {"type": "function_call_output", "call_id": "t2_b", "output": "hi"},- {"type": "function_call_output", "call_id": "t2_c", "output": "/workspace"},-- # Turn 3: 2 calls- {"type": "function_call", "name": "exec_command", "call_id": "t3_a", "arguments": '{"cmd":"cat file1"}'},- {"type": "function_call", "name": "exec_command", "call_id": "t3_b", "arguments": '{"cmd":"cat file2"}'},- {"type": "function_call_output", "call_id": "t3_a", "output": "content1"},- {"type": "function_call_output", "call_id": "t3_b", "output": "content2"},- ]- messages = _responses_items_to_messages(items)-- asst_msgs = [m for m in messages if m["role"] == "assistant"]- tool_msgs = [m for m in messages if m["role"] == "tool"]-- assert len(asst_msgs) == 3, "Should have 3 assistant messages"- assert len(asst_msgs[0]["tool_calls"]) == 2, "Turn 1: 2 calls"- assert len(asst_msgs[1]["tool_calls"]) == 3, "Turn 2: 3 calls"- assert len(asst_msgs[2]["tool_calls"]) == 2, "Turn 3: 2 calls"- assert len(tool_msgs) == 7, "Total 7 tool results"---# ----------------------------------------------------------------------------# Test 3: Verify the specific bug location-# -----------------------------------------------------------------------------class TestBugLocation:- """Pinpoint the exact code path causing the merge."""-- def test_turn_boundary_detected_at_output_to_call_transition(self):- """The converter flushes at function_call_output → function_call transitions.-- This transition marks a turn boundary: the previous turn's outputs are- done and a new LLM response is starting.- """- # Minimal case: two turns with no message item between them- items = [- # Turn 1- {"type": "function_call", "name": "tool_a", "call_id": "a", "arguments": "{}"},- {"type": "function_call_output", "call_id": "a", "output": "done_a"},- # Turn 2 (no message item separating from turn 1)- {"type": "function_call", "name": "tool_b", "call_id": "b", "arguments": "{}"},- {"type": "function_call_output", "call_id": "b", "output": "done_b"},- ]- messages = _responses_items_to_messages(items)-- asst_msgs = [m for m in messages if m["role"] == "assistant"]-- # Correctly produces 2 separate assistant messages- assert len(asst_msgs) == 2- assert len(asst_msgs[0]["tool_calls"]) == 1- assert asst_msgs[0]["tool_calls"][0]["id"] == "a"- assert len(asst_msgs[1]["tool_calls"]) == 1- assert asst_msgs[1]["tool_calls"][0]["id"] == "b"-- def test_message_item_correctly_triggers_flush(self):- """When a message item separates tool blocks, flush works correctly."""- items = [- # Turn 1- {"type": "function_call", "name": "tool_a", "call_id": "a", "arguments": "{}"},- {"type": "function_call_output", "call_id": "a", "output": "done_a"},- # Explicit message item triggers flush- {"type": "message", "role": "assistant", "content": "Intermediate text"},- # Turn 2- {"type": "function_call", "name": "tool_b", "call_id": "b", "arguments": "{}"},- {"type": "function_call_output", "call_id": "b", "output": "done_b"},- ]- messages = _responses_items_to_messages(items)-- asst_msgs = [m for m in messages if m["role"] == "assistant"]- # This works correctly: 3 assistant messages (tool_a, text, tool_b)- assert len(asst_msgs) == 3- assert len(asst_msgs[0]["tool_calls"]) == 1 # tool_a- assert asst_msgs[1]["content"] == "Intermediate text"- assert len(asst_msgs[2]["tool_calls"]) == 1 # tool_b---# ----------------------------------------------------------------------------# Test 4: Verify this matches the trace data from tmp/output-# -----------------------------------------------------------------------------class TestTraceDataConsistency:- """Cross-reference trace data with the translation function output."""-- def test_trace2_tool_call_ids_match(self):- """The tool_call IDs in trace 2's messages should match trace 1's response."""- traces = _load_traces()- t1_response_calls = traces[0]["response"]["choices"][0]["message"]["tool_calls"]- t2_history_assistant = next(- m for m in traces[1]["request"]["messages"] if m["role"] == "assistant"- )-- t1_ids = [tc["id"] for tc in t1_response_calls]- t2_ids = [tc["id"] for tc in t2_history_assistant["tool_calls"]]-- assert t1_ids == t2_ids, (- "Turn 1 response tool_call IDs should appear in trace 2's history"- )-- def test_trace3_contains_turn1_and_turn2_calls_merged(self):- """Trace 3's single assistant message has turn 1 + turn 2 tool_calls merged."""- traces = _load_traces()- t3_assistant = next(- m for m in traces[2]["request"]["messages"] if m["role"] == "assistant"- )-- t1_ids = [tc["id"] for tc in traces[0]["response"]["choices"][0]["message"]["tool_calls"]]- t2_ids = [tc["id"] for tc in traces[1]["response"]["choices"][0]["message"]["tool_calls"]]-- merged_ids = [tc["id"] for tc in t3_assistant["tool_calls"]]-- # The merged assistant message contains ALL IDs from both turns- assert merged_ids[:len(t1_ids)] == t1_ids, "First tool_calls should be from turn 1"- assert merged_ids[len(t1_ids):] == t2_ids, "Remaining tool_calls should be from turn 2"-- def test_all_traces_same_session(self):- """All traces belong to the same Codex session."""- traces = _load_traces()- session_ids = {t["sessionID"] for t in traces}- assert len(session_ids) == 1, f"Expected 1 session, got {session_ids}"-- def test_tool_results_in_trace2_match_tool_call_ids(self):- """The tool_results attached to trace 1 should be consistent with trace 2's history."""- traces = _load_traces()- t1 = traces[0]-- # tool_results from the deferred logging- if "tool_results" not in t1:- pytest.skip("Trace 1 doesn't have tool_results (may be first-turn pattern)")-- t1_result_ids = {tr["tool_call_id"] for tr in t1["tool_results"]}- t1_call_ids = {tc["id"] for tc in t1["response"]["choices"][0]["message"]["tool_calls"]}-- assert t1_result_ids == t1_call_ids, (- "tool_results should correspond to the response's tool_calls"- )diff --git a/tests/test_context_error_translation.py b/tests/test_context_error_translation.pydeleted file mode 100644index b2e0e8038..000000000--- a/tests/test_context_error_translation.py+++ /dev/null@@ -1,50 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Inbound-format JSON shapes for :func:`context_exhausted_response`."""--from __future__ import annotations--import json--from switchyard.lib.endpoints.upstream_error import context_exhausted_response---def _body(exc_message: str, inbound: str) -> dict:- exc = RuntimeError(exc_message)- response = context_exhausted_response(exc, inbound=inbound) # type: ignore[arg-type]- assert response.status_code == 400- return json.loads(response.body)---def test_anthropic_inbound_shape() -> None:- body = _body("context pool exhausted", "anthropic")- assert body == {- "error": {- "message": "context pool exhausted",- "type": "invalid_request_error",- "code": "context_length_exceeded",- },- }---def test_openai_inbound_shape() -> None:- body = _body("context pool exhausted", "openai")- assert body == {- "error": {- "message": "context pool exhausted",- "type": "invalid_request_error",- "code": "context_length_exceeded",- },- }---def test_openai_responses_inbound_shape() -> None:- body = _body("context pool exhausted", "openai-responses")- assert body == {- "error": {- "message": "context pool exhausted",- "type": "invalid_request_error",- "code": "context_length_exceeded",- },- }diff --git a/tests/test_context_window_exceeded_endpoint.py b/tests/test_context_window_exceeded_endpoint.pydeleted file mode 100644index 5cbe71bbd..000000000--- a/tests/test_context_window_exceeded_endpoint.py+++ /dev/null@@ -1,73 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Endpoint-level tests: SwitchyardContextWindowExceededError on a single-target-route must return HTTP 400 context_length_exceeded, not HTTP 500."""--from __future__ import annotations--from collections.abc import AsyncIterator-from unittest.mock import AsyncMock--import httpx-import pytest-from fastapi import FastAPI--from switchyard.lib.endpoints.anthropic_messages_endpoint import AnthropicMessagesEndpoint-from switchyard.lib.endpoints.openai_chat_endpoint import OpenAIChatEndpoint-from switchyard.lib.endpoints.responses_endpoint import ResponsesEndpoint-from switchyard_rust.core import SwitchyardContextWindowExceededError--_CHAT_BODY = {"model": "m", "messages": [{"role": "user", "content": "hi"}]}-_ANTHROPIC_BODY = {"model": "m", "messages": [{"role": "user", "content": "hi"}], "max_tokens": 100}-_RESPONSES_BODY = {"model": "m", "input": "hi"}---def _app_raising(exc: Exception) -> FastAPI:- """Build a minimal FastAPI app whose chain always raises *exc*."""- app = FastAPI()- mock_sw = AsyncMock()- mock_sw.call = AsyncMock(side_effect=exc)- app.state.switchyard = mock_sw- OpenAIChatEndpoint().register(app)- AnthropicMessagesEndpoint().register(app)- ResponsesEndpoint().register(app)- return app---async def window_client() -> AsyncIterator[httpx.AsyncClient]:- """Async client wired to an app whose chain raises SwitchyardContextWindowExceededError."""- exc = SwitchyardContextWindowExceededError("context window exceeded on single target")- app = _app_raising(exc)- async with httpx.AsyncClient(- transport=httpx.ASGITransport(app=app), base_url="http://test"- ) as c:- yield c---async def test_openai_chat_window_exceeded_returns_400(window_client: httpx.AsyncClient) -> None:- """POST /v1/chat/completions returns 400 context_length_exceeded, not 500."""- resp = await window_client.post("/v1/chat/completions", json=_CHAT_BODY)- assert resp.status_code == 400- body = resp.json()- assert body["error"]["code"] == "context_length_exceeded"- assert body["error"]["type"] == "invalid_request_error"---async def test_anthropic_messages_window_exceeded_returns_400(window_client: httpx.AsyncClient) -> None:- """POST /v1/messages returns 400 invalid_request_error, not 500."""- resp = await window_client.post("/v1/messages", json=_ANTHROPIC_BODY)- assert resp.status_code == 400- body = resp.json()- assert body["error"]["code"] == "context_length_exceeded"- assert body["error"]["type"] == "invalid_request_error"---async def test_responses_window_exceeded_returns_400(window_client: httpx.AsyncClient) -> None:- """POST /v1/responses returns 400 context_length_exceeded, not 500."""- resp = await window_client.post("/v1/responses", json=_RESPONSES_BODY)- assert resp.status_code == 400- body = resp.json()- assert body["error"]["code"] == "context_length_exceeded"- assert body["error"]["type"] == "invalid_request_error"diff --git a/tests/test_cost_estimator_gemini.py b/tests/test_cost_estimator_gemini.pyindex 3977102e7..4407b8535 100644--- a/tests/test_cost_estimator_gemini.py+++ b/tests/test_cost_estimator_gemini.py@@ -10,7 +10,7 @@ fails loudly instead.from __future__ import annotations-from switchyard.lib.cost_estimator import MODEL_PRICING, estimate_cost+from switchyard.cli.launchers.cost_estimator import MODEL_PRICING, estimate_cost_GEMINI_KEYS = ("gcp/google/gemini-3.5-flash",diff --git a/tests/test_endpoint_state_contract.py b/tests/test_endpoint_state_contract.pydeleted file mode 100644index 7df4cbe28..000000000--- a/tests/test_endpoint_state_contract.py+++ /dev/null@@ -1,119 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Contract tests for the ``app.state`` wiring between-``build_switchyard_app`` and the three inbound endpoint handlers.--The bug fixed in PR #28 was that the factory wrote ``app.state.switchyard``-while every endpoint read ``app.state.switchyard`` — the server started-cleanly and unit tests stayed green, but every request hit-``AttributeError`` at runtime.--These tests pin both sides of the contract against ``Switchyard.state_key``-so a future rename either updates everything together or fails CI.-"""--from __future__ import annotations--import inspect--import pytest-from fastapi import FastAPI-from openai.types.chat import ChatCompletion-from openai.types.chat.chat_completion import Choice-from openai.types.chat.chat_completion_message import ChatCompletionMessage-from openai.types.completion_usage import CompletionUsage--from switchyard.lib.endpoints.anthropic_messages_endpoint import (- AnthropicMessagesEndpoint,-)-from switchyard.lib.endpoints.openai_chat_endpoint import (- OpenAIChatEndpoint,-)-from switchyard.lib.endpoints.responses_endpoint import (- ResponsesEndpoint,-)-from switchyard.lib.proxy_context import ProxyContext-from switchyard.lib.roles import LLMBackend-from switchyard.lib.switchyard import Switchyard-from switchyard.server.switchyard_app import build_switchyard_app-from switchyard_rust.core import ChatRequest, ChatRequestType, ChatResponse-from switchyard_rust.translation import TranslationEngine--_ENDPOINT_CLASSES = (- OpenAIChatEndpoint,- AnthropicMessagesEndpoint,- ResponsesEndpoint,-)-_ALL_REQUEST_TYPES = [- ChatRequestType.OPENAI_CHAT,- ChatRequestType.OPENAI_RESPONSES,- ChatRequestType.ANTHROPIC,-]---class _StubBackend(LLMBackend):- def supported_request_types(self) -> list[ChatRequestType]:- return list(_ALL_REQUEST_TYPES)-- async def call(self, ctx: ProxyContext, request: ChatRequest) -> ChatResponse:- completion = ChatCompletion(- id="chatcmpl-stub",- object="chat.completion",- created=1700000000,- model="stub",- choices=[- Choice(- index=0,- message=ChatCompletionMessage(role="assistant", content="ok"),- finish_reason="stop",- )- ],- usage=CompletionUsage(prompt_tokens=1, completion_tokens=1, total_tokens=2),- )- return ChatResponse.openai_completion(completion)---def _build_app() -> tuple[FastAPI, Switchyard]:- switchyard = Switchyard(backend=_StubBackend(), translator=TranslationEngine())- app = build_switchyard_app(switchyard)- return app, switchyard---def test_state_key_constant_is_switchyard() -> None:- """Pin the expected key. If this changes, every reader must change too."""- assert Switchyard.state_key == "switchyard"---def test_factory_writes_switchyard_to_state_key_constant() -> None:- """``build_switchyard_app`` must store the instance under ``Switchyard.state_key``.-- Direct regression guard for PR #28: the factory previously wrote to- ``app.state.switchyard`` (wrong) instead of ``app.state.switchyard``.- """- app, switchyard = _build_app()- stored = getattr(app.state, Switchyard.state_key)- assert stored is switchyard--- "endpoint_cls",- _ENDPOINT_CLASSES,- ids=lambda c: c.__name__,-)-def test_endpoint_module_reads_state_key_constant(endpoint_cls: type) -> None:- """Each endpoint module must read from the same key the factory writes.-- Source-level check rather than a runtime assertion because the contract- is "the literal attribute name agrees" — a runtime test still passes if- both sides are wrong-and-matching, but the source check fails the moment- either side drifts from ``Switchyard.state_key``.- """- module = inspect.getmodule(endpoint_cls)- assert module is not None- expected = f"request.app.state.{Switchyard.state_key}"- source = inspect.getsource(module)- assert expected in source, (- f"{endpoint_cls.__name__} does not read from {expected!r}; "- f"the factory and the endpoint disagree on app.state."- )diff --git a/tests/test_error_source_annotation.py b/tests/test_error_source_annotation.pydeleted file mode 100644index 57710da9f..000000000--- a/tests/test_error_source_annotation.py+++ /dev/null@@ -1,156 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""failure-source annotation on error responses and the event log.--Every client-facing error carries ``x-switchyard-error-source`` naming the-layer that originated it (``switchyard`` | ``provider``), plus-``x-switchyard-upstream-model`` when a routing selection had happened. These-tests cover the endpoint layer that renders those annotations from the ctx-stamps a backend sets.-"""--from __future__ import annotations--import json-import logging--import pytest--from switchyard.lib.endpoints.dispatch import model_not_found_response-from switchyard.lib.endpoints.error_envelope import (- ERROR_SOURCE_HEADER,- UPSTREAM_MODEL_HEADER,- error_response,- upstream_error_response,-)-from switchyard.lib.endpoints.upstream_error import (- context_exhausted_response,- handle_chain_exception,- upstream_response_from_ctx,-)-from switchyard.lib.endpoints.upstream_error_log import log_upstream_attempt_failure-from switchyard.lib.proxy_context import (- CTX_ERROR_SOURCE,- CTX_UPSTREAM_HTTP_BODY,- CTX_UPSTREAM_HTTP_STATUS,- CTX_UPSTREAM_MODEL,- ProxyContext,-)--_UPSTREAM_401 = {- "error": {"message": "bad key", "type": "auth_error", "code": "invalid_api_key"}-}---# --- envelope builders ----------------------------------------------------------def test_synthesized_envelope_defaults_to_switchyard_source() -> None:- resp = error_response(400, "bad", error_type="invalid_request_error", code="invalid_body")- assert resp.headers[ERROR_SOURCE_HEADER] == "switchyard"- assert UPSTREAM_MODEL_HEADER not in resp.headers---def test_upstream_envelope_labels_provider_and_upstream_model() -> None:- resp = upstream_error_response(429, _UPSTREAM_401, upstream_model="gpt-5")- assert resp.headers[ERROR_SOURCE_HEADER] == "provider"- assert resp.headers[UPSTREAM_MODEL_HEADER] == "gpt-5"---def test_model_not_found_labels_switchyard() -> None:- resp = model_not_found_response("nope")- assert resp.status_code == 404- assert resp.headers[ERROR_SOURCE_HEADER] == "switchyard"---def test_context_exhausted_labels_switchyard() -> None:- resp = context_exhausted_response(RuntimeError("pool exhausted"), "openai")- assert resp.status_code == 400- assert resp.headers[ERROR_SOURCE_HEADER] == "switchyard"---# --- ctx-driven recovery paths -------------------------------------------------def _ctx_with_upstream_stash(**extra: object) -> ProxyContext:- ctx = ProxyContext()- ctx.metadata[CTX_UPSTREAM_HTTP_STATUS] = 401- ctx.metadata[CTX_UPSTREAM_HTTP_BODY] = _UPSTREAM_401- for key, value in extra.items():- ctx.metadata[key] = value- return ctx---def test_stashed_status_defaults_to_provider() -> None:- """A backend that stashes an upstream status without a source label gets- the passthrough default — the stash channel exists for provider errors."""- resp = upstream_response_from_ctx(_ctx_with_upstream_stash())- assert resp is not None- assert resp.status_code == 401- assert resp.headers[ERROR_SOURCE_HEADER] == "provider"- assert UPSTREAM_MODEL_HEADER not in resp.headers---def test_stashed_switchyard_source_overrides_provider_default() -> None:- """caller_required-style rejections ride the upstream channel but must- surface as switchyard-originated."""- ctx = _ctx_with_upstream_stash(**{CTX_ERROR_SOURCE: "switchyard"})- resp = upstream_response_from_ctx(ctx)- assert resp is not None- assert resp.headers[ERROR_SOURCE_HEADER] == "switchyard"---def test_stashed_upstream_model_reaches_header() -> None:- ctx = _ctx_with_upstream_stash(**{CTX_UPSTREAM_MODEL: "gpt-5"})- resp = upstream_response_from_ctx(ctx)- assert resp is not None- assert resp.headers[UPSTREAM_MODEL_HEADER] == "gpt-5"---def test_unexpected_internal_500_labels_switchyard() -> None:- resp = handle_chain_exception(- RuntimeError("boom"), ProxyContext(), inbound="openai", log_msg="test failure"- )- assert resp.status_code == 500- assert resp.headers[ERROR_SOURCE_HEADER] == "switchyard"- assert UPSTREAM_MODEL_HEADER not in resp.headers---def test_network_failure_500_labels_provider() -> None:- """A status-less upstream fault (network error after retries) renders as- the internal 500 envelope but is labeled provider via the ctx stamp."""- ctx = ProxyContext()- ctx.metadata[CTX_ERROR_SOURCE] = "provider"- ctx.metadata[CTX_UPSTREAM_MODEL] = "gpt-5"-- resp = handle_chain_exception(- RuntimeError("connection reset"), ctx, inbound="openai", log_msg="test failure"- )-- assert resp.status_code == 500- assert json.loads(bytes(resp.body))["error"]["code"] == "internal_chain_error"- assert resp.headers[ERROR_SOURCE_HEADER] == "provider"- assert resp.headers[UPSTREAM_MODEL_HEADER] == "gpt-5"---# --- structured event log -------------------------------------------------------def test_attempt_failure_log_carries_upstream_model_and_source(- caplog: pytest.LogCaptureFixture,-) -> None:- with caplog.at_level(logging.WARNING, logger="switchyard.upstream_errors"):- log_upstream_attempt_failure(- model="route-id",- attempt=1,- status_code=429,- error=RuntimeError("rate limited"),- upstream_model="gpt-5",- )-- record = json.loads(caplog.records[-1].message)- assert record["model"] == "route-id"- assert record["upstream_model"] == "gpt-5"- assert record["error_source"] == "provider"- assert record["code"] == "429"diff --git a/tests/test_format_translate_processor.py b/tests/test_format_translate_processor.pydeleted file mode 100644index 1686723e1..000000000--- a/tests/test_format_translate_processor.py+++ /dev/null@@ -1,593 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--from __future__ import annotations--import json-from collections.abc import AsyncIterator-from typing import Any--import pytest-from openai.types.chat import ChatCompletionChunk--from switchyard.lib.chat_response.anthropic import AnthropicResponseStream-from switchyard.lib.chat_response.openai_chat import ResponseStream-from switchyard.lib.chat_response.openai_responses import ResponsesApiStream-from switchyard.lib.processors.format_translate import (- FormatTranslateResponseProcessor,- ModelFormatLookupProcessor,- StampOriginalFormatProcessor,- TranslateConfig,-)-from switchyard.lib.proxy_context import (- CTX_ORIGINAL_FORMAT,- CTX_ORIGINAL_REQUEST,- CTX_PROXY_ACTUAL_MODEL,- CTX_TARGET_FORMAT,- ProxyContext,-)-from switchyard_rust.core import (- ChatRequest,- ChatRequestType,- ChatResponse,- ChatResponseType,- request_type_value,- response_type_matches,-)---async def _aiter(items: list[Any]) -> AsyncIterator[Any]:- for item in items:- yield item---def _chat_chunk(- *,- content: str | None = None,- finish_reason: str | None = None,-) -> ChatCompletionChunk:- delta: dict[str, Any] = {}- if content is not None:- delta["content"] = content- return ChatCompletionChunk.model_validate(- {- "id": "chatcmpl-test",- "object": "chat.completion.chunk",- "created": 1700000000,- "model": "backend-model",- "choices": [- {- "index": 0,- "delta": delta,- "finish_reason": finish_reason,- }- ],- }- )---def _chat_tool_chunk(- *,- name: str | None = None,- arguments: str | None = None,- finish_reason: str | None = None,-) -> ChatCompletionChunk:- function: dict[str, str] = {}- if name is not None:- function["name"] = name- if arguments is not None:- function["arguments"] = arguments- return ChatCompletionChunk.model_validate(- {- "id": "chatcmpl-test",- "object": "chat.completion.chunk",- "created": 1700000000,- "model": "backend-model",- "choices": [- {- "index": 0,- "delta": {- "tool_calls": [- {- "index": 0,- "id": "call_search",- "type": "function",- "function": function,- }- ]- },- "finish_reason": finish_reason,- }- ],- }- )---def _anthropic_events() -> list[dict[str, Any]]:- return [- {- "type": "message_start",- "message": {- "id": "msg_test",- "type": "message",- "role": "assistant",- "content": [],- "model": "claude-upstream",- "stop_reason": None,- "stop_sequence": None,- "usage": {"input_tokens": 3, "output_tokens": 0},- },- },- {- "type": "content_block_start",- "index": 0,- "content_block": {"type": "text", "text": ""},- },- {- "type": "content_block_delta",- "index": 0,- "delta": {"type": "text_delta", "text": "hello"},- },- {"type": "content_block_stop", "index": 0},- {- "type": "message_delta",- "delta": {"stop_reason": "end_turn", "stop_sequence": None},- "usage": {"output_tokens": 1},- },- {"type": "message_stop"},- ]---def _anthropic_tool_events() -> list[dict[str, Any]]:- return [- {- "type": "message_start",- "message": {- "id": "msg_tool",- "type": "message",- "role": "assistant",- "content": [],- "model": "claude-upstream",- "stop_reason": None,- "stop_sequence": None,- "usage": {"input_tokens": 9, "output_tokens": 0},- },- },- {- "type": "content_block_start",- "index": 0,- "content_block": {- "type": "tool_use",- "id": "toolu_weather",- "name": "get_weather",- "input": {},- },- },- {- "type": "content_block_delta",- "index": 0,- "delta": {"type": "input_json_delta", "partial_json": '{"city":"SF"}'},- },- {"type": "content_block_stop", "index": 0},- {- "type": "message_delta",- "delta": {"stop_reason": "tool_use", "stop_sequence": None},- "usage": {"output_tokens": 4},- },- {"type": "message_stop"},- ]---def _responses_events() -> list[dict[str, Any]]:- return [- {- "type": "response.created",- "response": {- "id": "resp_test",- "object": "response",- "created_at": 1700000000,- "status": "in_progress",- "model": "responses-upstream",- "output": [],- },- },- {- "type": "response.output_item.added",- "output_index": 0,- "item": {- "type": "message",- "id": "msg_test",- "role": "assistant",- "status": "in_progress",- "content": [],- },- },- {- "type": "response.output_text.delta",- "output_index": 0,- "content_index": 0,- "delta": "hello",- },- {- "type": "response.completed",- "response": {- "id": "resp_test",- "object": "response",- "created_at": 1700000000,- "status": "completed",- "model": "responses-upstream",- "output": [- {- "type": "message",- "id": "msg_test",- "role": "assistant",- "status": "completed",- "content": [{"type": "output_text", "text": "hello"}],- }- ],- "usage": {"input_tokens": 5, "output_tokens": 2, "total_tokens": 7},- },- },- ]---async def _collect(stream: AsyncIterator[Any]) -> list[Any]:- return [item async for item in stream]---def _frame_data(frame: str) -> dict[str, Any]:- data_line = next(line for line in frame.splitlines() if line.startswith("data: "))- return json.loads(data_line.removeprefix("data: "))---async def test_model_format_lookup_prefers_rust_selected_model() -> None:- processor = ModelFormatLookupProcessor(- TranslateConfig(models=[- {"model": "rust-selected", "backend_format": "anthropic"},- {"model": "legacy-selected", "backend_format": "openai"},- {"model": "responses-selected", "backend_format": "responses"},- ])- )- ctx = ProxyContext(metadata={CTX_PROXY_ACTUAL_MODEL: "legacy-selected"})- ctx.selected_model = "rust-selected"-- await processor.process(- ctx,- ChatRequest.openai_chat({"model": "client-model", "messages": []}),- )-- assert ctx.metadata[CTX_TARGET_FORMAT] == ChatRequestType.ANTHROPIC---async def test_model_format_lookup_preserves_responses_target() -> None:- processor = ModelFormatLookupProcessor(- TranslateConfig(models=[{"model": "responses-selected", "backend_format": "responses"}])- )- ctx = ProxyContext()- ctx.selected_model = "responses-selected"-- await processor.process(- ctx,- ChatRequest.openai_responses({"model": "client-model", "input": "hi"}),- )-- assert ctx.metadata[CTX_TARGET_FORMAT] == ChatRequestType.OPENAI_RESPONSES---async def test_model_format_lookup_falls_back_to_legacy_metadata() -> None:- processor = ModelFormatLookupProcessor(- TranslateConfig(models=[{"model": "legacy-selected", "backend_format": "openai"}])- )- ctx = ProxyContext(metadata={CTX_PROXY_ACTUAL_MODEL: "legacy-selected"})-- await processor.process(- ctx,- ChatRequest.anthropic({"model": "client-model", "messages": []}),- )-- assert ctx.metadata[CTX_TARGET_FORMAT] == ChatRequestType.OPENAI_CHAT---async def test_stamp_original_format_preserves_original_body_snapshot() -> None:- body = {"model": "gpt-client", "messages": [{"role": "user", "content": "hi"}]}- ctx = ProxyContext()-- await StampOriginalFormatProcessor().process(ctx, ChatRequest.openai_chat(body))- body["messages"][0]["content"] = "mutated"-- assert request_type_value(ctx.metadata[CTX_ORIGINAL_FORMAT]) == "openai_chat"- assert ctx.metadata[CTX_ORIGINAL_REQUEST] == {- "model": "gpt-client",- "messages": [{"role": "user", "content": "hi"}],- }---async def test_streaming_openai_response_translates_back_to_anthropic() -> None:- ctx = ProxyContext(- metadata={- CTX_ORIGINAL_FORMAT: ChatRequestType.ANTHROPIC,- CTX_ORIGINAL_REQUEST: {"model": "claude-client"},- }- )- ctx.selected_model = "served/model"- response = ChatResponse.openai_stream(- ResponseStream(- _aiter(- [- _chat_chunk(content="hel"),- _chat_chunk(content="lo", finish_reason="stop"),- ]- )- )- )-- result = await FormatTranslateResponseProcessor().process(ctx, response)-- assert response_type_matches(result, ChatResponseType.ANTHROPIC_STREAM)- events = await _collect(result.stream)- text = "".join(- event["delta"]["text"]- for event in events- if event["type"] == "content_block_delta" and event["delta"]["type"] == "text_delta"- )- assert events[0]["message"]["model"] == "served/model"- assert text == "hello"- assert [event["type"] for event in events].count("content_block_start") == 1---async def test_streaming_openai_length_finish_translates_to_anthropic_max_tokens() -> None:- ctx = ProxyContext(- metadata={- CTX_ORIGINAL_FORMAT: ChatRequestType.ANTHROPIC,- CTX_ORIGINAL_REQUEST: {"model": "claude-client"},- }- )- response = ChatResponse.openai_stream(- ResponseStream(_aiter([_chat_chunk(content="truncated", finish_reason="length")]))- )-- result = await FormatTranslateResponseProcessor().process(ctx, response)-- assert response_type_matches(result, ChatResponseType.ANTHROPIC_STREAM)- events = await _collect(result.stream)- message_delta = next(event for event in events if event["type"] == "message_delta")- assert message_delta["delta"]["stop_reason"] == "max_tokens"---async def test_streaming_openai_response_translates_back_to_responses() -> None:- ctx = ProxyContext(- metadata={- CTX_ORIGINAL_FORMAT: ChatRequestType.OPENAI_RESPONSES,- CTX_ORIGINAL_REQUEST: {"model": "responses-client", "input": "hi"},- }- )- ctx.selected_model = "served/model"- response = ChatResponse.openai_stream(- ResponseStream(- _aiter(- [- _chat_chunk(content="hel"),- _chat_chunk(content="lo", finish_reason="stop"),- ]- )- )- )-- result = await FormatTranslateResponseProcessor().process(ctx, response)-- assert response_type_matches(result, ChatResponseType.OPENAI_RESPONSES_STREAM)- frames = await _collect(result.stream)- payloads = [_frame_data(frame) for frame in frames]- assert payloads[0]["response"]["model"] == "served/model"- assert (- "".join(- payload["delta"]- for payload in payloads- if payload["type"] == "response.output_text.delta"- )- == "hello"- )- assert payloads[-1]["type"] == "response.completed"---async def test_streaming_openai_to_responses_keeps_unique_output_indexes() -> None:- ctx = ProxyContext(- metadata={- CTX_ORIGINAL_FORMAT: ChatRequestType.OPENAI_RESPONSES,- CTX_ORIGINAL_REQUEST: {"model": "responses-client", "input": "search"},- }- )- response = ChatResponse.openai_stream(- ResponseStream(- _aiter(- [- _chat_tool_chunk(name="search", arguments='{"q":"x"}'),- _chat_chunk(content="Checking"),- _chat_chunk(finish_reason="tool_calls"),- ]- )- )- )-- result = await FormatTranslateResponseProcessor().process(ctx, response)-- assert response_type_matches(result, ChatResponseType.OPENAI_RESPONSES_STREAM)- payloads = [_frame_data(frame) for frame in await _collect(result.stream)]- added = [- (payload["output_index"], payload["item"]["type"])- for payload in payloads- if payload["type"] == "response.output_item.added"- ]- assert added == [(0, "function_call"), (1, "message")]-- function_index = added[0][0]- args_done = next(- payload- for payload in payloads- if payload["type"] == "response.function_call_arguments.done"- )- assert args_done["output_index"] == function_index-- completed = payloads[-1]["response"]- assert [item["type"] for item in completed["output"]] == [- "function_call",- "message",- ]---async def test_streaming_anthropic_response_translates_back_to_responses() -> None:- ctx = ProxyContext(- metadata={- CTX_ORIGINAL_FORMAT: ChatRequestType.OPENAI_RESPONSES,- CTX_ORIGINAL_REQUEST: {"model": "responses-client", "input": "hi"},- }- )- response = ChatResponse.anthropic_stream(- AnthropicResponseStream(_aiter(_anthropic_events())),- )-- result = await FormatTranslateResponseProcessor().process(ctx, response)-- assert response_type_matches(result, ChatResponseType.OPENAI_RESPONSES_STREAM)- payloads = [_frame_data(frame) for frame in await _collect(result.stream)]- assert (- "".join(- payload["delta"]- for payload in payloads- if payload["type"] == "response.output_text.delta"- )- == "hello"- )- assert payloads[-1]["type"] == "response.completed"---async def test_streaming_anthropic_response_translates_back_to_openai() -> None:- ctx = ProxyContext(- metadata={- CTX_ORIGINAL_FORMAT: ChatRequestType.OPENAI_CHAT,- CTX_ORIGINAL_REQUEST: {"model": "gpt-client"},- }- )- response = ChatResponse.anthropic_stream(- AnthropicResponseStream(_aiter(_anthropic_events())),- )-- result = await FormatTranslateResponseProcessor().process(ctx, response)-- assert response_type_matches(result, ChatResponseType.OPENAI_STREAM)- chunks = [chunk.model_dump(exclude_none=True) for chunk in await _collect(result.stream)]- assert chunks[0]["choices"][0]["delta"] == {"role": "assistant"}- assert chunks[1]["model"] == "claude-upstream"- assert chunks[1]["choices"][0]["delta"]["content"] == "hello"- assert chunks[-1]["choices"][0]["finish_reason"] == "stop"- assert chunks[-1]["usage"]["prompt_tokens"] == 3- assert chunks[-1]["usage"]["completion_tokens"] == 1---async def test_streaming_anthropic_cache_usage_translates_back_to_openai() -> None:- ctx = ProxyContext(- metadata={- CTX_ORIGINAL_FORMAT: ChatRequestType.OPENAI_CHAT,- CTX_ORIGINAL_REQUEST: {"model": "gpt-client"},- }- )- events = _anthropic_events()- events[0]["message"]["usage"] = {- "input_tokens": 3,- "cache_creation_input_tokens": 4,- "cache_read_input_tokens": 2,- "output_tokens": 0,- }- response = ChatResponse.anthropic_stream(AnthropicResponseStream(_aiter(events)))-- result = await FormatTranslateResponseProcessor().process(ctx, response)-- assert response_type_matches(result, ChatResponseType.OPENAI_STREAM)- chunks = [chunk.model_dump(exclude_none=True) for chunk in await _collect(result.stream)]- usage = chunks[-1]["usage"]- assert usage["prompt_tokens"] == 9- assert usage["completion_tokens"] == 1- assert usage["total_tokens"] == 10- assert usage["prompt_tokens_details"]["cached_tokens"] == 2- assert usage["prompt_tokens_details"]["cache_creation_tokens"] == 4---async def test_streaming_responses_response_translates_back_to_openai() -> None:- ctx = ProxyContext(- metadata={- CTX_ORIGINAL_FORMAT: ChatRequestType.OPENAI_CHAT,- CTX_ORIGINAL_REQUEST: {"model": "gpt-client"},- }- )- response = ChatResponse.openai_responses_stream(- ResponsesApiStream(_aiter(_responses_events())),- )-- result = await FormatTranslateResponseProcessor().process(ctx, response)-- assert response_type_matches(result, ChatResponseType.OPENAI_STREAM)- chunks = [chunk.model_dump(exclude_none=True) for chunk in await _collect(result.stream)]- assert chunks[0]["choices"][0]["delta"] == {"role": "assistant"}- assert chunks[1]["choices"][0]["delta"]["content"] == "hello"- assert chunks[-1]["choices"][0]["finish_reason"] == "stop"- assert chunks[-1]["usage"]["prompt_tokens"] == 5- assert chunks[-1]["usage"]["completion_tokens"] == 2---async def test_streaming_responses_response_translates_back_to_anthropic() -> None:- ctx = ProxyContext(- metadata={- CTX_ORIGINAL_FORMAT: ChatRequestType.ANTHROPIC,- CTX_ORIGINAL_REQUEST: {"model": "claude-client"},- }- )- ctx.selected_model = "served/model"- response = ChatResponse.openai_responses_stream(- ResponsesApiStream(_aiter(_responses_events())),- )-- result = await FormatTranslateResponseProcessor().process(ctx, response)-- assert response_type_matches(result, ChatResponseType.ANTHROPIC_STREAM)- events = await _collect(result.stream)- text = "".join(- event["delta"]["text"]- for event in events- if event["type"] == "content_block_delta" and event["delta"]["type"] == "text_delta"- )- assert events[0]["message"]["model"] == "served/model"- assert text == "hello"---async def test_streaming_anthropic_tool_use_translates_back_to_openai_tool_call() -> None:- ctx = ProxyContext(- metadata={- CTX_ORIGINAL_FORMAT: ChatRequestType.OPENAI_CHAT,- CTX_ORIGINAL_REQUEST: {"model": "gpt-client"},- }- )- response = ChatResponse.anthropic_stream(- AnthropicResponseStream(_aiter(_anthropic_tool_events())),- )-- result = await FormatTranslateResponseProcessor().process(ctx, response)-- assert response_type_matches(result, ChatResponseType.OPENAI_STREAM)- chunks = [chunk.model_dump(exclude_none=True) for chunk in await _collect(result.stream)]- first_tool_delta = chunks[1]["choices"][0]["delta"]["tool_calls"][0]- args_delta = chunks[2]["choices"][0]["delta"]["tool_calls"][0]- assert first_tool_delta["id"] == "toolu_weather"- assert first_tool_delta["function"]["name"] == "get_weather"- assert args_delta["function"]["arguments"] == '{"city":"SF"}'- assert chunks[-1]["choices"][0]["finish_reason"] == "tool_calls"diff --git a/tests/test_inference_e2e.py b/tests/test_inference_e2e.pydeleted file mode 100644index f8420b46d..000000000--- a/tests/test_inference_e2e.py+++ /dev/null@@ -1,599 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""End-to-end inference tests using a mock LLM backend.--Exercises the full HTTP stack — FastAPI endpoints → Switchyard chain →-mock backend → response translation — without touching any live LLM provider.--All tests run offline: no API keys, no network access, no running model.-"""--from __future__ import annotations--import json-from collections.abc import AsyncIterator-from unittest.mock import AsyncMock, patch--import httpx-import pytest-from fastapi import FastAPI-from fastapi.testclient import TestClient-from openai.types.chat import ChatCompletion, ChatCompletionChunk-from openai.types.chat.chat_completion import Choice-from openai.types.chat.chat_completion_chunk import Choice as ChunkChoice-from openai.types.chat.chat_completion_chunk import ChoiceDelta-from openai.types.chat.chat_completion_message import ChatCompletionMessage-from openai.types.chat.chat_completion_message_function_tool_call import (- ChatCompletionMessageFunctionToolCall,- Function,-)-from openai.types.completion_usage import CompletionUsage--from switchyard.lib.chat_response.openai_chat import ResponseStream-from switchyard.lib.proxy_context import ProxyContext-from switchyard.lib.roles import LLMBackend-from switchyard.lib.switchyard import Switchyard-from switchyard.server.switchyard_app import build_switchyard_app-from switchyard_rust.core import ChatRequest, ChatRequestType, ChatResponse-from switchyard_rust.translation import TranslationEngine--# ----------------------------------------------------------------------------# Mock LLM backends-# -----------------------------------------------------------------------------_REPLY = "hello back"-_ALL_REQUEST_TYPES = [- ChatRequestType.OPENAI_CHAT,- ChatRequestType.OPENAI_RESPONSES,- ChatRequestType.ANTHROPIC,-]---class _MockLLMBackend(LLMBackend):- """Returns a canned OpenAI completion response for every call."""-- def __init__(self, completion: ChatCompletion) -> None:- self._completion = completion-- def supported_request_types(self) -> list[ChatRequestType]:- return list(_ALL_REQUEST_TYPES)-- async def call(self, ctx: ProxyContext, request: ChatRequest) -> ChatResponse:- return ChatResponse.openai_completion(self._completion)---class _StreamingMockLLMBackend(LLMBackend):- """Returns a fixed OpenAI stream response for every call."""-- def __init__(self, chunks: list[ChatCompletionChunk]) -> None:- self._chunks = list(chunks)-- def supported_request_types(self) -> list[ChatRequestType]:- return list(_ALL_REQUEST_TYPES)-- async def call(self, ctx: ProxyContext, request: ChatRequest) -> ChatResponse:- chunks = self._chunks-- async def _iter() -> AsyncIterator[ChatCompletionChunk]:- for chunk in chunks:- yield chunk-- return ChatResponse.openai_stream(ResponseStream(_iter()))---class _RaisingMockLLMBackend(LLMBackend):- """Raises ``exc`` for every call — exercises the error path through the chain."""-- def __init__(self, exc: BaseException) -> None:- self._exc = exc-- def supported_request_types(self) -> list[ChatRequestType]:- return list(_ALL_REQUEST_TYPES)-- async def call(self, ctx: ProxyContext, request: ChatRequest) -> ChatResponse:- raise self._exc---class _TranslatingMockLLMBackend(LLMBackend):- """Chat-only backend that translates the inbound request to Chat at the top of- ``call`` — mirroring any real chat-completions backend (native / passthrough).-- An unsupported inbound field (e.g. a bad message role) is rejected by the- translation engine here, before any upstream call, exactly as it would be for- a production chat backend.- """-- def supported_request_types(self) -> list[ChatRequestType]:- return [ChatRequestType.OPENAI_CHAT]-- async def call(self, ctx: ProxyContext, request: ChatRequest) -> ChatResponse:- request = TranslationEngine().request_to_any_of(request, [ChatRequestType.OPENAI_CHAT])- return ChatResponse.openai_completion(_make_completion())---# ----------------------------------------------------------------------------# Helpers-# ------------------------------------------------------------------------------def _make_completion(*, model: str = "mock-model", content: str = _REPLY) -> ChatCompletion:- return ChatCompletion(- id="chatcmpl-test",- object="chat.completion",- created=1700000000,- model=model,- choices=[- Choice(- index=0,- message=ChatCompletionMessage(role="assistant", content=content),- finish_reason="stop",- )- ],- usage=CompletionUsage(prompt_tokens=5, completion_tokens=3, total_tokens=8),- )---def _make_tool_call_completion(- *,- model: str = "mock-model",- tool_name: str = "get_weather",- tool_args: str = '{"city": "Paris"}',- tool_call_id: str = "call_test_123",-) -> ChatCompletion:- """Completion whose assistant message contains a single tool call."""- return ChatCompletion(- id="chatcmpl-test-tool",- object="chat.completion",- created=1700000000,- model=model,- choices=[- Choice(- index=0,- message=ChatCompletionMessage(- role="assistant",- content=None,- tool_calls=[- ChatCompletionMessageFunctionToolCall(- id=tool_call_id,- type="function",- function=Function(name=tool_name, arguments=tool_args),- )- ],- ),- finish_reason="tool_calls",- )- ],- usage=CompletionUsage(prompt_tokens=20, completion_tokens=10, total_tokens=30),- )---def _make_chunks(*, model: str = "mock-model", content: str = _REPLY) -> list[ChatCompletionChunk]:- return [- ChatCompletionChunk(- id="chatcmpl-stream",- object="chat.completion.chunk",- created=1700000000,- model=model,- choices=[- ChunkChoice(- index=0,- delta=ChoiceDelta(role="assistant", content=content),- finish_reason=None,- )- ],- ),- ChatCompletionChunk(- id="chatcmpl-stream",- object="chat.completion.chunk",- created=1700000000,- model=model,- choices=[ChunkChoice(index=0, delta=ChoiceDelta(), finish_reason="stop")],- usage=CompletionUsage(prompt_tokens=5, completion_tokens=3, total_tokens=8),- ),- ]---# ----------------------------------------------------------------------------# Fixtures-# ------------------------------------------------------------------------------def app() -> FastAPI:- switchyard = Switchyard(- backend=_MockLLMBackend(_make_completion()),- translator=TranslationEngine(),- )- return build_switchyard_app(switchyard)---def streaming_app() -> FastAPI:- switchyard = Switchyard(- backend=_StreamingMockLLMBackend(_make_chunks()),- translator=TranslationEngine(),- )- return build_switchyard_app(switchyard)---async def client(app: FastAPI) -> AsyncIterator[httpx.AsyncClient]:- async with httpx.AsyncClient(- transport=httpx.ASGITransport(app=app),- base_url="http://test",- ) as c:- yield c---async def streaming_client(streaming_app: FastAPI) -> AsyncIterator[httpx.AsyncClient]:- async with httpx.AsyncClient(- transport=httpx.ASGITransport(app=streaming_app),- base_url="http://test",- ) as c:- yield c---def tool_call_app() -> FastAPI:- """App backed by a mock that always returns a tool-calling completion."""- switchyard = Switchyard(- backend=_MockLLMBackend(_make_tool_call_completion()),- translator=TranslationEngine(),- )- return build_switchyard_app(switchyard)---async def tool_call_client(tool_call_app: FastAPI) -> AsyncIterator[httpx.AsyncClient]:- async with httpx.AsyncClient(- transport=httpx.ASGITransport(app=tool_call_app),- base_url="http://test",- ) as c:- yield c---def raising_app() -> FastAPI:- """App backed by a mock that always raises a generic backend error."""- switchyard = Switchyard(- backend=_RaisingMockLLMBackend(RuntimeError("backend boom")),- translator=TranslationEngine(),- )- return build_switchyard_app(switchyard)---async def raising_client(raising_app: FastAPI) -> AsyncIterator[httpx.AsyncClient]:- # ``raise_app_exceptions=False`` mirrors uvicorn's behavior in production:- # an unhandled exception inside a route is mapped to a 500 response, not- # propagated up the call stack. Required for the error-path tests below.- async with httpx.AsyncClient(- transport=httpx.ASGITransport(app=raising_app, raise_app_exceptions=False),- base_url="http://test",- ) as c:- yield c---def translating_app() -> FastAPI:- """App whose chat-only backend translates the inbound request to Chat, so an- invalid inbound role is rejected during translation before any upstream call."""- switchyard = Switchyard(- backend=_TranslatingMockLLMBackend(),- translator=TranslationEngine(),- )- return build_switchyard_app(switchyard)---# ----------------------------------------------------------------------------# Tests-# ------------------------------------------------------------------------------class TestInferenceE2E:-- async def test_health_liveness(self, client: httpx.AsyncClient) -> None:- resp = await client.get("/health")- assert resp.status_code == 200- assert resp.json() == {"status": "ok"}-- async def test_openai_chat_completions(self, client: httpx.AsyncClient) -> None:- resp = await client.post(- "/v1/chat/completions",- json={- "model": "any-model",- "messages": [{"role": "user", "content": "hello"}],- },- )- assert resp.status_code == 200- data = resp.json()- choice = data["choices"][0]- assert choice["message"]["role"] == "assistant"- assert choice["message"]["content"] == _REPLY- assert choice["finish_reason"] == "stop"-- async def test_openai_chat_completions_streaming(- self, streaming_client: httpx.AsyncClient- ) -> None:- resp = await streaming_client.post(- "/v1/chat/completions",- json={- "model": "any-model",- "messages": [{"role": "user", "content": "hello"}],- "stream": True,- },- )- assert resp.status_code == 200- assert "text/event-stream" in resp.headers["content-type"]-- data_lines = [- line[6:] # strip leading "data: "- for line in resp.text.split("\n")- if line.startswith("data: ") and line != "data: [DONE]"- ]- assert data_lines, "expected at least one data frame before [DONE]"-- chunks = [json.loads(line) for line in data_lines]- content = "".join(- c["choices"][0]["delta"].get("content", "")- for c in chunks- if c.get("choices")- )- assert content == _REPLY-- async def test_anthropic_messages(self, client: httpx.AsyncClient) -> None:- resp = await client.post(- "/v1/messages",- json={- "model": "any-model",- "max_tokens": 100,- "messages": [{"role": "user", "content": "hello"}],- },- )- assert resp.status_code == 200- data = resp.json()- assert data["type"] == "message"- assert data["role"] == "assistant"- assert data["content"][0]["type"] == "text"- assert data["content"][0]["text"] == _REPLY- assert data["stop_reason"] == "end_turn"-- async def test_openai_responses_api(self, client: httpx.AsyncClient) -> None:- resp = await client.post(- "/v1/responses",- json={- "model": "any-model",- "input": "hello",- },- )- assert resp.status_code == 200- data = resp.json()- assert "output" in data- msg = data["output"][0]- assert msg["type"] == "message"- assert msg["role"] == "assistant"- assert msg["content"][0]["type"] == "output_text"- assert msg["content"][0]["text"] == _REPLY-- async def test_anthropic_messages_streaming(- self, streaming_client: httpx.AsyncClient- ) -> None:- """Streaming through the Anthropic inbound — chunks must be Anthropic SSE shape."""- resp = await streaming_client.post(- "/v1/messages",- json={- "model": "any-model",- "max_tokens": 100,- "messages": [{"role": "user", "content": "hello"}],- "stream": True,- },- )- assert resp.status_code == 200, resp.text- assert "text/event-stream" in resp.headers["content-type"]-- events = [- json.loads(line[6:])- for line in resp.text.split("\n")- if line.startswith("data: ") and line != "data: [DONE]"- ]- assert events, "expected at least one Anthropic SSE event"-- # Anthropic streaming uses an event-typed envelope (message_start /- # content_block_delta / message_stop). Pin the contract: at least- # one event carries the canonical Anthropic event 'type' field.- anthropic_event_types = {"message_start", "content_block_delta", "message_stop"}- seen_types = {e.get("type") for e in events}- assert seen_types & anthropic_event_types, (- f"no Anthropic-shaped event types in stream: {seen_types}"- )-- # The reply text should be reconstructible from text deltas.- text_deltas = [- e["delta"]["text"]- for e in events- if e.get("type") == "content_block_delta"- and e.get("delta", {}).get("type") == "text_delta"- ]- if text_deltas:- assert "".join(text_deltas) == _REPLY-- async def test_openai_responses_streaming(- self, streaming_client: httpx.AsyncClient- ) -> None:- """Streaming through the Responses API inbound — chunks must be Responses SSE shape."""- resp = await streaming_client.post(- "/v1/responses",- json={- "model": "any-model",- "input": "hello",- "stream": True,- },- )- assert resp.status_code == 200, resp.text- assert "text/event-stream" in resp.headers["content-type"]-- events = [- json.loads(line[6:])- for line in resp.text.split("\n")- if line.startswith("data: ") and line != "data: [DONE]"- ]- assert events, "expected at least one Responses SSE event"-- # Responses streaming events all carry a 'type' starting with- # 'response.' — a regression that emits Chat Completions chunks- # on this endpoint would fail this assertion.- types = {e.get("type", "") for e in events}- assert any(t.startswith("response.") for t in types), (- f"no Responses-shaped event types in stream: {types}"- )-- async def test_backend_exception_returns_500(- self, raising_client: httpx.AsyncClient- ) -> None:- """A backend that raises must surface as an HTTP error, not 200 with garbage."""- resp = await raising_client.post(- "/v1/chat/completions",- json={- "model": "any-model",- "messages": [{"role": "user", "content": "hello"}],- },- )- assert resp.status_code == 500- assert resp.headers["content-type"].startswith("application/json")- body = resp.json()- assert body["error"]["type"] == "internal_error"- assert body["error"]["code"] == "internal_chain_error"- assert "backend boom" in body["error"]["message"]---class TestToolCallRoundTrip:- """Backend returns OpenAI tool_calls; verify each inbound format renders correctly.-- Tool-call translation is a frequent break point in refactors because- the format-specific shapes diverge sharply (OpenAI: ``tool_calls`` on- the assistant message; Anthropic: a ``tool_use`` block in ``content``;- Responses: a top-level ``function_call`` output item).- """-- async def test_openai_tool_calls_passthrough(- self, tool_call_client: httpx.AsyncClient- ) -> None:- resp = await tool_call_client.post(- "/v1/chat/completions",- json={- "model": "any-model",- "messages": [{"role": "user", "content": "what's the weather?"}],- "tools": [- {- "type": "function",- "function": {- "name": "get_weather",- "parameters": {"type": "object"},- },- }- ],- },- )- assert resp.status_code == 200, resp.text- choice = resp.json()["choices"][0]- assert choice["finish_reason"] == "tool_calls"- tool_calls = choice["message"]["tool_calls"]- assert len(tool_calls) == 1- assert tool_calls[0]["function"]["name"] == "get_weather"- assert tool_calls[0]["function"]["arguments"] == '{"city": "Paris"}'-- async def test_anthropic_tool_use_translation(- self, tool_call_client: httpx.AsyncClient- ) -> None:- """OpenAI ``tool_calls`` from the backend must be translated to Anthropic ``tool_use``."""- resp = await tool_call_client.post(- "/v1/messages",- json={- "model": "any-model",- "max_tokens": 100,- "messages": [{"role": "user", "content": "what's the weather?"}],- "tools": [- {- "name": "get_weather",- "input_schema": {"type": "object"},- }- ],- },- )- assert resp.status_code == 200, resp.text- body = resp.json()- assert body["type"] == "message"- assert body["stop_reason"] == "tool_use"-- tool_use_blocks = [b for b in body["content"] if b.get("type") == "tool_use"]- assert len(tool_use_blocks) == 1, f"expected one tool_use block, got: {body['content']}"- block = tool_use_blocks[0]- assert block["name"] == "get_weather"- assert block["input"] == {"city": "Paris"}---class TestEndpointErrorContract:- """Endpoint error-mapping contracts previously covered only through the- removed latency-service backend, re-vehicled onto in-process backends."""-- def test_post_dispatch_exception_returns_json_500(self, app: FastAPI) -> None:- """An exception raised AFTER dispatch (e.g. during result serialization)- must surface as a JSON 500 envelope, not FastAPI's plain-text 500."""-- class _Unserializable:- def model_dump(self) -> None:- raise RuntimeError("serialization exploded")-- with patch(- "switchyard.lib.endpoints.openai_chat_endpoint.dispatch_chat_request",- new_callable=AsyncMock,- return_value=_Unserializable(),- ):- with TestClient(app, raise_server_exceptions=False) as client:- response = client.post(- "/v1/chat/completions",- json={"model": "any-model", "messages": [{"role": "user", "content": "hi"}]},- )-- assert response.status_code == 500- assert response.headers["content-type"].startswith("application/json")- body = response.json()- assert body["error"]["type"] == "internal_error"- assert body["error"]["code"] == "internal_chain_error"- assert "serialization exploded" in body["error"]["message"]-- @pytest.mark.parametrize(- ("path", "payload"),- [- (- "/v1/responses",- {"model": "m", "input": [{"type": "message", "role": "api", "content": "ping"}]},- ),- (- "/v1/messages",- {"model": "m", "max_tokens": 16, "messages": [{"role": "api", "content": "ping"}]},- ),- ],- )- def test_invalid_inbound_role_surfaces_as_internal_error(- self, translating_app: FastAPI, path: str, payload: dict[str, object]- ) -> None:- """An unsupported inbound message role is rejected by the translation engine- when a chat-only backend translates the request, before any upstream call.-- NOTE: this currently surfaces as a 500 ``internal_error``. The removed- latency-service backend special-cased translation ``invalid_value`` errors- into a provider-compatible 400; no surviving backend reproduces that, so- the standalone-server contract is now a 500. Update this test (and the- endpoint/translation layer) if a 400 is restored.- """- with TestClient(translating_app, raise_server_exceptions=False) as client:- response = client.post(path, json=payload)-- assert response.status_code == 500- body = response.json()- assert body["error"]["type"] == "internal_error"- assert '"api"' in body["error"]["message"]- assert "role" in body["error"]["message"]diff --git a/tests/test_infra.py b/tests/test_infra.pydeleted file mode 100644index d4b61ebad..000000000--- a/tests/test_infra.py+++ /dev/null@@ -1,146 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Tests for core role helpers and plain processor components."""--import pytest-from openai.types.chat import ChatCompletion-from openai.types.chat.chat_completion import Choice-from openai.types.chat.chat_completion_message import ChatCompletionMessage--from switchyard.lib.proxy_context import ProxyContext-from switchyard.lib.roles import LLMBackend-from switchyard_rust.core import (- ChatRequest,- ChatRequestType,- ChatResponse,- ChatResponseType,- response_type_matches,-)--# ----------------------------------------------------------------------------# Fixtures-# ------------------------------------------------------------------------------def make_ctx() -> ProxyContext:- return ProxyContext()---def make_request() -> ChatRequest:- return ChatRequest.openai_chat({"model": "gpt-4o", "messages": []})---# ----------------------------------------------------------------------------# Backend instantiation guard-# ------------------------------------------------------------------------------class TestABCsCannotBeInstantiated:- def test_llm_backend(self):- with pytest.raises(TypeError):- LLMBackend() # type: ignore[abstract]---# ----------------------------------------------------------------------------# Concrete implementations for testing-# ------------------------------------------------------------------------------class PassthroughProcessor:- async def process(self, ctx, request):- return request---class TaggedRequestProcessor:- def __init__(self, tag: str) -> None:- self._tag = tag-- async def process(self, ctx, request):- ctx.metadata.setdefault("request_order", []).append(self._tag)- return request---class EchoBackend(LLMBackend):- @property- def supported_request_types(self):- return [ChatRequestType.OPENAI_CHAT]-- async def call(self, ctx, request):- from openai.types.completion_usage import CompletionUsage-- return ChatResponse.openai_completion(- ChatCompletion(- id="chatcmpl-test",- object="chat.completion",- created=1700000000,- model="gpt-4o",- choices=[- Choice(- index=0,- message=ChatCompletionMessage(role="assistant", content="echo"),- finish_reason="stop",- )- ],- usage=CompletionUsage(prompt_tokens=1, completion_tokens=1, total_tokens=2),- )- )---class NoopResponseProcessor:- async def process(self, ctx, response):- return response---class TaggedResponseProcessor:- def __init__(self, tag: str) -> None:- self._tag = tag-- async def process(self, ctx, response):- ctx.metadata.setdefault("response_order", []).append(self._tag)- return response---# ----------------------------------------------------------------------------# Request-side component-# ------------------------------------------------------------------------------class TestRequestProcessor:- async def test_process(self):- proc = PassthroughProcessor()- req = make_request()- result = await proc.process(make_ctx(), req)- assert result is req---# ----------------------------------------------------------------------------# LLMBackend-# ------------------------------------------------------------------------------class TestLLMBackend:- async def test_call(self):- backend = EchoBackend()- resp = await backend.call(make_ctx(), make_request())- assert response_type_matches(resp, ChatResponseType.OPENAI_COMPLETION)- assert resp.body["choices"][0]["message"]["content"] == "echo"---# ----------------------------------------------------------------------------# Response-side component-# ------------------------------------------------------------------------------class TestResponseProcessor:- async def test_process(self):-- resp = ChatResponse.openai_completion(- ChatCompletion(- id="test", object="chat.completion", created=0, model="m",- choices=[Choice(index=0, message=ChatCompletionMessage(role="assistant", content="x"), finish_reason="stop")],- )- )- proc = NoopResponseProcessor()- result = await proc.process(make_ctx(), resp)- assert result is respdiff --git a/tests/test_init_all_exports.py b/tests/test_init_all_exports.pydeleted file mode 100644index 3f5c5c26c..000000000--- a/tests/test_init_all_exports.py+++ /dev/null@@ -1,23 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Every name in switchyard.__all__ must be accessible from the top-level package.--These tests fail when symbols are listed in __all__ but are not handled by the-module-level __getattr__ lazy-loader, causing AttributeError / ImportError.-"""--from __future__ import annotations--import switchyard---def test_all_symbols_accessible() -> None:- """All __all__ exports must be gettable without AttributeError."""- missing = []- for name in switchyard.__all__:- try:- getattr(switchyard, name)- except AttributeError:- missing.append(name)- assert missing == [], f"Symbols in __all__ not importable from switchyard: {missing}"diff --git a/tests/test_launchers.py b/tests/test_launchers.pyindex fa4ec9e2d..b0fc60bb7 100644--- a/tests/test_launchers.py+++ b/tests/test_launchers.py@@ -22,8 +22,8 @@ def _subparsers(parser: argparse.ArgumentParser) -> dict[str, argparse.ArgumentPreturn action.choices # type: ignore[return-value]-def test_cli_exposes_only_serve_and_launch() -> None:- assert set(_subparsers(_build_parser())) == {"serve", "launch"}+def test_cli_exposes_only_launch() -> None:+ assert set(_subparsers(_build_parser())) == {"launch"}@pytest.mark.parametrize("agent", ["claude", "codex", "openclaw"])diff --git a/tests/test_live_stats_footer.py b/tests/test_live_stats_footer.pyindex 4a31b057a..d28e74377 100644--- a/tests/test_live_stats_footer.py+++ b/tests/test_live_stats_footer.py@@ -9,8 +9,9 @@ per active outbound model tier.from __future__ import annotations+from collections.abc import Mapping+from switchyard.cli.launchers.live_stats_footer import FOOTER_ROWS, LiveStatsFooter-from switchyard.lib.stats_accumulator import StatsAccumulatordef _strip_ansi(text: str) -> str:@@ -37,38 +38,60 @@ class _StubHealth:return ("●", 1)-async def _stats_with_model_call(+class _Stats:+ def __init__(self, snapshot: Mapping[str, object] | None = None) -> None:+ self.snapshot = snapshot or {+ "total_requests": 0,+ "total_errors": 0,+ "total_tokens": {},+ "models": {},+ }++ def snapshot_sync(self) -> Mapping[str, object]:+ return self.snapshot+++def _stats_with_model_call(model: str,*,prompt: int = 1234,completion: int = 567,cached: int = 200,-) -> StatsAccumulator:- acc = StatsAccumulator()- await acc.record_success(model=model)- await acc.record_usage(- model=model,- prompt_tokens=prompt,- completion_tokens=completion,- cached_tokens=cached,- )- return acc---def _footer(stats: StatsAccumulator, *, model: str = "nvidia/some/default") -> LiveStatsFooter:+) -> _Stats:+ return _Stats({+ "total_requests": 1,+ "total_errors": 0,+ "total_tokens": {+ "prompt": prompt,+ "completion": completion,+ "cached": cached,+ },+ "models": {+ model: {+ "calls": 1,+ "errors": 0,+ "prompt_tokens": prompt,+ "completion_tokens": completion,+ "cached_tokens": cached,+ }+ },+ })+++def _footer(stats: _Stats, *, model: str = "nvidia/some/default") -> LiveStatsFooter:return LiveStatsFooter(stats, model=model, health=_StubHealth()) # type: ignore[arg-type]-async def test_footer_height_at_zero_traffic() -> None:+def test_footer_height_at_zero_traffic() -> None:"""Before any traffic: aggregate + 1 fallback row = 2 rows."""- footer = _footer(StatsAccumulator())+ footer = _footer(_Stats())assert footer.height == FOOTER_ROWS == 2rows = footer.render(cols=80)assert len(rows) == 2-async def test_aggregate_row_shows_totals_without_model_name() -> None:- stats = await _stats_with_model_call("vendor/some-model")+def test_aggregate_row_shows_totals_without_model_name() -> None:+ stats = _stats_with_model_call("vendor/some-model")rows = _footer(stats).render(cols=80)agg = _strip_ansi(rows[0][0])assert "switchyard" in agg@@ -79,8 +102,8 @@ async def test_aggregate_row_shows_totals_without_model_name() -> None:assert "some-model" not in agg-async def test_active_row_shows_model_with_recent_traffic() -> None:- stats = await _stats_with_model_call("vendor/winner-model")+def test_active_row_shows_model_with_recent_traffic() -> None:+ stats = _stats_with_model_call("vendor/winner-model")rows = _footer(stats).render(cols=80)active = _strip_ansi(rows[1][0])assert "winner-model" in active@@ -90,31 +113,40 @@ async def test_active_row_shows_model_with_recent_traffic() -> None:assert "200 cached" in active-async def test_active_row_falls_back_to_default_when_no_traffic() -> None:+def test_active_row_falls_back_to_default_when_no_traffic() -> None:"""Before any backend call lands, the row labels with the launch default."""- rows = _footer(StatsAccumulator(), model="vendor/launch-default").render(cols=80)+ rows = _footer(_Stats(), model="vendor/launch-default").render(cols=80)active = _strip_ansi(rows[1][0])assert "launch-default" in activeassert "0 req" in active-async def test_new_tier_adds_a_row_on_next_render() -> None:+def test_new_tier_adds_a_row_on_next_render() -> None:"""A new model in traffic adds a row; height grows from 2 to 3."""- stats = StatsAccumulator()- await stats.record_success(model="vendor/first")- await stats.record_usage(- model="vendor/first", prompt_tokens=10, completion_tokens=20,- )+ stats = _stats_with_model_call("vendor/first", prompt=10, completion=20, cached=0)footer = _footer(stats)rows = footer.render(cols=80)assert len(rows) == 2assert footer.height == 2assert "first" in _strip_ansi(rows[1][0])- await stats.record_success(model="vendor/second")- await stats.record_usage(- model="vendor/second", prompt_tokens=30, completion_tokens=40,- )+ stats.snapshot = {+ "total_requests": 2,+ "total_errors": 0,+ "total_tokens": {"prompt": 40, "completion": 60, "cached": 0},+ "models": {+ "vendor/first": {+ "calls": 1,+ "prompt_tokens": 10,+ "completion_tokens": 20,+ },+ "vendor/second": {+ "calls": 1,+ "prompt_tokens": 30,+ "completion_tokens": 40,+ },+ },+ }rows = footer.render(cols=80)assert len(rows) == 3assert footer.height == 3diff --git a/tests/test_llm_client.py b/tests/test_llm_client.pydeleted file mode 100644index 8369b1243..000000000--- a/tests/test_llm_client.py+++ /dev/null@@ -1,205 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Unit tests for :class:`OpenAILLMClient`."""--from unittest.mock import AsyncMock, MagicMock--import httpx-import openai-import pytest-import respx-from openai import AsyncOpenAI--from switchyard.lib.llm_client import OpenAILLMClient, RawSSEFrameStream---def test_constructs_only_the_async_client() -> None:- """Only the async client is built.-- Backends call ``acompletion`` exclusively, so a sync ``OpenAI`` client- would only allocate a second, never-used httpx connection pool (1000- connections by default) per instance.- """- client = OpenAILLMClient(api_key="test-key")- assert isinstance(client.async_client, AsyncOpenAI)- assert not hasattr(client, "client")---def test_max_retries_reaches_the_sdk_client() -> None:- """``max_retries`` is forwarded to the underlying SDK client."""- client = OpenAILLMClient(api_key="test-key", max_retries=0)- assert client.async_client.max_retries == 0---class TestAcompletionApiKeyOverride:- """``acompletion`` overrides the construction-time key only for a real key.-- A blank or absent per-call ``api_key`` must fall back to the- construction-time key (the configured endpoint key) instead of overriding- it with nothing, which would unauthenticate the upstream call.- """-- @staticmethod- def _client_with_spied_options() -> tuple[OpenAILLMClient, MagicMock]:- client = OpenAILLMClient(api_key="endpoint-key")- client.async_client = MagicMock()- client.async_client.chat.completions.create = AsyncMock(return_value="base")- client.async_client.responses.create = AsyncMock(return_value="responses-base")- overridden = MagicMock()- overridden.chat.completions.create = AsyncMock(return_value="overridden")- overridden.responses.create = AsyncMock(return_value="responses-overridden")- client.async_client.with_options.return_value = overridden- # Non-streaming ``aresponses`` fetches the raw HTTP response and- # returns its exact JSON body; wire raw spies per client.- for target, label in ((client.async_client, "base"), (overridden, "overridden")):- raw = MagicMock()- raw.http_response.json.return_value = {"src": f"responses-{label}"}- target.responses.with_raw_response.create = AsyncMock(return_value=raw)- return client, client.async_client-- async def test_real_caller_key_uses_with_options(self) -> None:- client, async_client = self._client_with_spied_options()- result = await client.acompletion(api_key="caller-key", model="m")- async_client.with_options.assert_called_once_with(api_key="caller-key")- assert result == "overridden"-- async def test_none_key_falls_back_to_construction_key(self) -> None:- client, async_client = self._client_with_spied_options()- result = await client.acompletion(api_key=None, model="m")- async_client.with_options.assert_not_called()- assert result == "base"-- async def test_blank_key_falls_back_to_construction_key(self) -> None:- client, async_client = self._client_with_spied_options()- result = await client.acompletion(api_key=" ", model="m")- async_client.with_options.assert_not_called()- assert result == "base"-- async def test_responses_real_caller_key_uses_with_options(self) -> None:- client, async_client = self._client_with_spied_options()- result = await client.aresponses(api_key="caller-key", model="m", input="hi")- async_client.with_options.assert_called_once_with(api_key="caller-key")- assert result == {"src": "responses-overridden"}-- async def test_responses_missing_key_falls_back_to_construction_key(self) -> None:- client, async_client = self._client_with_spied_options()- result = await client.aresponses(api_key=None, model="m", input="hi")- async_client.with_options.assert_not_called()- assert result == {"src": "responses-base"}-- async def test_responses_blank_key_falls_back_to_construction_key(self) -> None:- client, async_client = self._client_with_spied_options()- result = await client.aresponses(api_key=" ", model="m", input="hi")- async_client.with_options.assert_not_called()- assert result == {"src": "responses-base"}-- async def test_responses_streaming_uses_raw_streaming_path(self) -> None:- """Streaming fetches the raw SSE response (verbatim frames) and enters- the SDK context manager eagerly so error statuses raise at call time."""- client, async_client = self._client_with_spied_options()-- async def _lines():- yield "data: {}"- yield ""-- api_response = MagicMock()- api_response.http_response.aiter_lines = _lines- cm = MagicMock()- cm.__aenter__ = AsyncMock(return_value=api_response)- cm.__aexit__ = AsyncMock(return_value=False)- async_client.responses.with_streaming_response.create = MagicMock(return_value=cm)-- result = await client.aresponses(api_key=None, model="m", input="hi", stream=True)-- assert isinstance(result, RawSSEFrameStream)- cm.__aenter__.assert_awaited_once()- async_client.responses.create.assert_not_called()- async_client.responses.with_raw_response.create.assert_not_called()- await result.aclose()- cm.__aexit__.assert_awaited_once()---async def test_aresponses_returns_exact_upstream_json() -> None:- """Non-streaming Responses calls return the upstream body as-is.-- Provider-specific extras and explicit-null fields must survive — the SDK- typed-model round-trip would normalize the former and ``exclude_none``- serialization would drop the latter.- """- upstream = {- "id": "resp_1",- "object": "response",- "created_at": 1719890000,- "model": "gpt-5",- "status": "completed",- "output": [],- "store": False,- "temperature": 1.0,- "top_p": 0.9,- "previous_response_id": None,- "provider_meta": {"azure_region": "eastus"},- }- respx.post("http://upstream.test/v1/responses").mock(- return_value=httpx.Response(200, json=upstream)- )-- client = OpenAILLMClient(api_key="k", base_url="http://upstream.test/v1")- result = await client.aresponses(model="gpt-5", input="hi")-- assert result == upstream---_SSE_FRAMES = [- (- 'event: response.created\n'- 'data: {"type":"response.created","response":{"id":"resp_1","store":false,'- '"temperature":1.0,"reasoning":{"effort":null},"provider_meta":{"az":"eastus"}}}\n\n'- ),- ": keep-alive\n\n",- (- 'event: response.output_text.delta\n'- 'data: {"type":"response.output_text.delta","delta":"hi","vendor_extra":123}\n\n'- ),- (- 'event: response.completed\n'- 'data: {"type":"response.completed","response":{"id":"resp_1","usage":'- '{"input_tokens":3,"output_tokens":2}}}\n\n'- ),-]---async def test_aresponses_streaming_yields_verbatim_sse_frames() -> None:- """Streaming Responses calls yield the upstream SSE frames byte-for-byte- : unknown provider fields, explicit nulls, comment keep-alives,- and event names all survive because no typed-model parse happens.- """- respx.post("http://upstream.test/v1/responses").mock(- return_value=httpx.Response(- 200,- headers={"content-type": "text/event-stream"},- content="".join(_SSE_FRAMES).encode(),- )- )-- client = OpenAILLMClient(api_key="k", base_url="http://upstream.test/v1")- stream = await client.aresponses(model="gpt-5", input="hi", stream=True)-- frames = [frame async for frame in stream]- assert frames == _SSE_FRAMES---async def test_aresponses_streaming_error_status_raises_at_call_time() -> None:- """A non-2xx on a streaming Responses call raises ``APIStatusError`` from- ``aresponses`` itself — before any frame is consumed — so the backend's- retry/failover contract is unchanged by the raw-frame path."""- respx.post("http://upstream.test/v1/responses").mock(- return_value=httpx.Response(500, json={"error": {"message": "boom"}})- )-- client = OpenAILLMClient(api_key="k", base_url="http://upstream.test/v1")- with pytest.raises(openai.APIStatusError):- await client.aresponses(model="gpt-5", input="hi", stream=True)diff --git a/tests/test_metrics_endpoint.py b/tests/test_metrics_endpoint.pydeleted file mode 100644index e3dbd7678..000000000--- a/tests/test_metrics_endpoint.py+++ /dev/null@@ -1,155 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""End-to-end FastAPI test that ``StatsEndpoint`` serves both-``/v1/stats`` (existing JSON) and ``/metrics`` (Prometheus exposition)-off the same shared :class:`StatsAccumulator`.--Pins the contract the ticket calls out: existing ``/v1/stats`` behavior-stays intact, ``/metrics`` returns Prometheus text-format with the core-metric names.-"""--from __future__ import annotations--from collections.abc import AsyncIterator--import httpx-import pytest-from fastapi import FastAPI-from prometheus_client.parser import text_string_to_metric_families--from switchyard.lib.endpoints.stats_endpoint import PROMETHEUS_CONTENT_TYPE, StatsEndpoint-from switchyard.lib.stats_accumulator import StatsAccumulator---def stats() -> StatsAccumulator:- return StatsAccumulator()---async def client(stats: StatsAccumulator) -> AsyncIterator[httpx.AsyncClient]:- app = FastAPI()- StatsEndpoint(stats).register(app)- async with httpx.AsyncClient(- transport=httpx.ASGITransport(app=app), base_url="http://test"- ) as c:- yield c---async def test_v1_stats_returns_existing_json_shape(- client: httpx.AsyncClient, stats: StatsAccumulator-) -> None:- await stats.record_success(model="m", backend_latency_ms=10.0)- await stats.record_usage(- model="m", prompt_tokens=5, completion_tokens=3, total_latency_ms=20.0- )-- resp = await client.get("/v1/stats")- assert resp.status_code == 200- body = resp.json()- # Existing schema is unchanged — keep the contract surface visible.- assert body["total_requests"] == 1- assert body["models"]["m"]["calls"] == 1- assert body["models"]["m"]["prompt_tokens"] == 5- assert body["models"]["m"]["max_observed_context_tokens"] == 8- assert "cost_estimate" in body---async def test_metrics_returns_prometheus_exposition(- client: httpx.AsyncClient, stats: StatsAccumulator-) -> None:- await stats.record_success(model="strong/m", backend_latency_ms=42.5, tier="strong")- await stats.record_usage(- model="strong/m",- prompt_tokens=120,- completion_tokens=30,- total_latency_ms=88.0,- routing_overhead_ms=8.0,- tier="strong",- )-- resp = await client.get("/metrics")- assert resp.status_code == 200- assert resp.headers["content-type"] == PROMETHEUS_CONTENT_TYPE-- body = resp.text- # Core metric headers the ticket calls out.- for line in (- "# TYPE switchyard_requests_total counter",- "# TYPE switchyard_errors_total counter",- "# TYPE switchyard_model_call_latency_ms summary",- "# TYPE switchyard_total_latency_ms summary",- "# TYPE switchyard_routing_overhead_ms summary",- ):- assert line in body, f"missing exposition line: {line}"-- # Selected model/tier counter sample lands with the expected label set.- assert 'switchyard_requests_total{model="strong/m",tier="strong"} 1' in body- assert (- 'switchyard_model_call_latency_ms_count{model="strong/m",tier="strong"} 1'- in body- )---async def test_metrics_output_round_trips_through_official_prometheus_parser(- client: httpx.AsyncClient, stats: StatsAccumulator-) -> None:- """Spec compliance gate: ``prometheus_client.parser`` is the reference- parser used by every real scraper. Anything it accepts, Prometheus will."""- await stats.record_success(model="openai/gpt-5.2", backend_latency_ms=42.5, tier="strong")- await stats.record_error(model="anth/claude", tier="weak")- await stats.record_success(model="anth/claude", backend_latency_ms=5.0, tier="weak")- await stats.record_usage(- model="openai/gpt-5.2",- prompt_tokens=120,- completion_tokens=30,- total_latency_ms=88.0,- routing_overhead_ms=8.0,- tier="strong",- )- await stats.record_usage(- model="anth/claude",- prompt_tokens=40,- completion_tokens=5,- total_latency_ms=15.0,- routing_overhead_ms=3.0,- tier="weak",- )-- resp = await client.get("/metrics")- assert resp.status_code == 200-- families = {f.name: f for f in text_string_to_metric_families(resp.text)}-- # prometheus-client strips the ``_total`` suffix from counter family names- # but preserves it on individual samples — assert against the family form.- expected = {- "switchyard_total_requests": "gauge",- "switchyard_total_errors": "gauge",- "switchyard_requests": "counter",- "switchyard_errors": "counter",- "switchyard_prompt_tokens": "counter",- "switchyard_completion_tokens": "counter",- "switchyard_cached_tokens": "counter",- "switchyard_model_call_latency_ms": "summary",- "switchyard_total_latency_ms": "summary",- "switchyard_routing_overhead_ms": "summary",- }- for name, kind in expected.items():- assert name in families, f"family {name} missing from parsed output"- assert families[name].type == kind, f"family {name} parsed as {families[name].type}"-- # Counter values survive the parse round-trip with the right labels.- req_samples = {- (s.labels["model"], s.labels["tier"]): s.value- for s in families["switchyard_requests"].samples- }- assert req_samples[("openai/gpt-5.2", "strong")] == 1- assert req_samples[("anth/claude", "weak")] == 1-- # Summary sum aggregates across all observations (8 + 3 = 11 ms overhead).- overhead = families["switchyard_routing_overhead_ms"]- overhead_sum = next(s.value for s in overhead.samples if s.name.endswith("_sum"))- assert overhead_sum == 11.0diff --git a/tests/test_no_stale_module_paths.py b/tests/test_no_stale_module_paths.pyindex f67cd9256..a102acd79 100644--- a/tests/test_no_stale_module_paths.py+++ b/tests/test_no_stale_module_paths.py@@ -1,31 +1,7 @@# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.# SPDX-License-Identifier: Apache-2.0-"""Regression guard for stale Python module-path references in source code.--Runtime tests don't catch this class of bug — broken docstring paths-don't fail at import or at request time, only when someone follows the-reference (Sphinx build, IDE go-to-definition, copying the path into an-import). Hence this dedicated source-scan test.--Patterns checked:--* ``switchyard.core.`` — intermediate path that was flattened into- ``switchyard.lib.`` / ``switchyard.cli.`` during the open-source- cleanup. No file in the current tree should reference it.-* ``switchyard.foundation`` — pre-rename sub-package. Renamed to- ``switchyard.lib`` in the same cleanup.-* ``nemo_switchyard.`` — original Python package name. Renamed to- ``switchyard``. ``test_cli_stale_names.py`` covers user-facing CLI- strings (``nemo-switchyard`` with a dash); this guards Python-import- paths (``nemo_switchyard`` with an underscore).-* ``SwitchyardV2`` / ``switchyard_v2`` — V2 vs V1 was a transient- migration concept that has since been collapsed to a single,- unversioned ``Switchyard`` API. Make sure the V2 suffix does not- creep back into module paths or symbols.-* ``switchyard_v2_cli`` / ``switchyard_v2_app`` / ``build_switchyard_v2_app``- — old module/function names from the V2 era.-"""+"""Prevent deleted Python server and compatibility paths from returning."""from __future__ import annotations@@ -36,6 +12,11 @@ import pytest_STALE_PATH_PATTERNS = ("switchyard.core.","switchyard.foundation",+ "switchyard.lib.",+ "switchyard.server.",+ "switchyard_rust.components",+ "switchyard_rust.core",+ "switchyard_rust.translation","nemo_switchyard.","SwitchyardV2","switchyard_v2",diff --git a/tests/test_outcome_metrics.py b/tests/test_outcome_metrics.pydeleted file mode 100644index 3d8397785..000000000--- a/tests/test_outcome_metrics.py+++ /dev/null@@ -1,304 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Unit + integration tests for the outcome counters."""--from __future__ import annotations--import pytest-from fastapi.testclient import TestClient-from openai.types.chat import ChatCompletion-from openai.types.chat.chat_completion import Choice-from openai.types.chat.chat_completion_message import ChatCompletionMessage-from openai.types.completion_usage import CompletionUsage--from switchyard.lib.endpoints import outcome_metrics-from switchyard.lib.endpoints.upstream_error import (- record_upstream_attempt_failure,- record_upstream_attempt_success,-)-from switchyard.lib.proxy_context import (- CTX_UPSTREAM_ATTEMPTS_RECORDED,- CTX_UPSTREAM_HTTP_STATUS,- ProxyContext,-)-from switchyard.lib.roles import LLMBackend-from switchyard.lib.switchyard import Switchyard-from switchyard.server.switchyard_app import build_switchyard_app-from switchyard_rust.core import (- ChatRequest,- ChatRequestType,- ChatResponse,- SwitchyardUpstreamError,-)-from switchyard_rust.translation import TranslationEngine--[email protected](autouse=True)-def _reset_counters():- outcome_metrics._reset_for_tests()- yield- outcome_metrics._reset_for_tests()---# ----------------------------------------------------------------------------# Classification-# ------------------------------------------------------------------------------class TestClassify:- @pytest.mark.parametrize("code", [200, 201, 204, 299])- def test_2xx_is_success(self, code: int) -> None:- assert outcome_metrics.classify(code) == "success"-- @pytest.mark.parametrize("code", [429, 500, 504])- def test_spec_codes_are_retryable_error(self, code: int) -> None:- """Exactly the codes the success criterion lists count as retryable."""- assert outcome_metrics.classify(code) == "retryable_error"-- @pytest.mark.parametrize("code", [400, 401, 403, 404, 422, 502, 503])- def test_other_codes_are_other_error(self, code: int) -> None:- """Bad-payload / bad-key / non-spec 5xx fall outside the criterion."""- assert outcome_metrics.classify(code) == "other_error"---# ----------------------------------------------------------------------------# Code label (the per-status dimension for the distribution dashboard)-# ------------------------------------------------------------------------------class TestCodeLabel:- def test_none_is_the_no_status_sentinel(self) -> None:- """Non-HTTP failures have no status line → the ``none`` sentinel."""- assert outcome_metrics.code_label(None) == outcome_metrics.NO_STATUS_CODE- assert outcome_metrics.code_label(None) == "none"-- @pytest.mark.parametrize("code", sorted(outcome_metrics.KNOWN_STATUS_CODES))- def test_known_codes_emitted_verbatim(self, code: int) -> None:- assert outcome_metrics.code_label(code) == str(code)-- @pytest.mark.parametrize(- ("code", "expected"),- [(418, "4xx"), (451, "4xx"), (599, "5xx"), (100, "1xx"), (302, "3xx")],- )- def test_unknown_codes_clamp_to_class(self, code: int, expected: str) -> None:- """An oddball upstream code collapses to its class, bounding cardinality."""- assert outcome_metrics.code_label(code) == expected-- @pytest.mark.parametrize("code", [0, 99, 600, 700])- def test_out_of_range_codes_clamp_to_other(self, code: int) -> None:- assert outcome_metrics.code_label(code) == "other"---# ----------------------------------------------------------------------------# Render shape-# ------------------------------------------------------------------------------class TestRender:- def test_render_initial_state_is_all_zero(self) -> None:- out = "\n".join(outcome_metrics.render_lines())- assert 'switchyard_client_responses_total{outcome="success"} 0' in out- assert 'switchyard_client_responses_total{outcome="retryable_error"} 0' in out- assert 'switchyard_client_responses_total{outcome="other_error"} 0' in out- # Upstream attempts carry a code label; the canonical codes are- # seeded at 0 so their series exist before the first matching attempt.- assert 'switchyard_upstream_attempts_total{outcome="success",code="200"} 0' in out- assert (- 'switchyard_upstream_attempts_total{outcome="retryable_error",code="429"} 0'- in out- )- assert (- 'switchyard_upstream_attempts_total{outcome="retryable_error",code="none"} 0'- in out- )- assert "switchyard_router_retry_recovered_total 0" in out-- def test_render_includes_help_and_type_lines(self) -> None:- """Prometheus exposition needs HELP+TYPE before each metric family."""- out = "\n".join(outcome_metrics.render_lines())- for metric in (- "switchyard_client_responses_total",- "switchyard_upstream_attempts_total",- "switchyard_router_retry_recovered_total",- ):- assert f"# HELP {metric}" in out- assert f"# TYPE {metric}" in out-- def test_render_reflects_recorded_state(self) -> None:- outcome_metrics.record_client_response(200)- outcome_metrics.record_client_response(429)- outcome_metrics.record_client_response(401)- outcome_metrics.record_upstream_attempt(500)- outcome_metrics.record_upstream_attempt(None)- outcome_metrics.record_retry_recovered()-- out = "\n".join(outcome_metrics.render_lines())- assert 'switchyard_client_responses_total{outcome="success"} 1' in out- assert 'switchyard_client_responses_total{outcome="retryable_error"} 1' in out- assert 'switchyard_client_responses_total{outcome="other_error"} 1' in out- # The two retryable attempts split across their codes — the whole- # point of the new label — rather than collapsing into one bucket.- assert (- 'switchyard_upstream_attempts_total{outcome="retryable_error",code="500"} 1'- in out- )- assert (- 'switchyard_upstream_attempts_total{outcome="retryable_error",code="none"} 1'- in out- )- assert "switchyard_router_retry_recovered_total 1" in out-- def test_distinct_codes_get_distinct_series(self) -> None:- """429 / 500 / 504 must be separately countable, not merged."""- for _ in range(3):- outcome_metrics.record_upstream_attempt(429)- outcome_metrics.record_upstream_attempt(500)- outcome_metrics.record_upstream_attempt(504)- # An unknown 4xx clamps to its class rather than spawning a new series.- outcome_metrics.record_upstream_attempt(418)-- out = "\n".join(outcome_metrics.render_lines())- assert (- 'switchyard_upstream_attempts_total{outcome="retryable_error",code="429"} 3'- in out- )- assert (- 'switchyard_upstream_attempts_total{outcome="retryable_error",code="500"} 1'- in out- )- assert (- 'switchyard_upstream_attempts_total{outcome="retryable_error",code="504"} 1'- in out- )- assert (- 'switchyard_upstream_attempts_total{outcome="other_error",code="4xx"} 1'- in out- )---# ----------------------------------------------------------------------------# Endpoint-layer fallback — wires the upstream-attempt counter for backends-# (Rust native / passthrough / multi) that issue one attempt and can't reach-# the Python-only outcome_metrics themselves.-# ------------------------------------------------------------------------------def _upstream_count(out: str, outcome: str, code: str) -> str:- return f'switchyard_upstream_attempts_total{{outcome="{outcome}",code="{code}"}}'---class TestEndpointUpstreamAttemptFallback:- def test_success_records_one_200(self) -> None:- record_upstream_attempt_success(ProxyContext())- out = "\n".join(outcome_metrics.render_lines())- assert f"{_upstream_count(out, 'success', '200')} 1" in out-- def test_rust_upstream_http_error_records_its_status(self) -> None:- """A Rust backend's typed ``SwitchyardUpstreamError.status_code`` is used."""- exc = SwitchyardUpstreamError("boom")- exc.status_code = 500- record_upstream_attempt_failure(ProxyContext(), exc)- out = "\n".join(outcome_metrics.render_lines())- assert f"{_upstream_count(out, 'retryable_error', '500')} 1" in out-- def test_python_backend_ctx_status_takes_priority(self) -> None:- """A Python backend's stashed ctx status is recorded even without a typed exc."""- ctx = ProxyContext()- ctx.metadata[CTX_UPSTREAM_HTTP_STATUS] = 401- record_upstream_attempt_failure(ctx, RuntimeError("opaque"))- out = "\n".join(outcome_metrics.render_lines())- assert f"{_upstream_count(out, 'other_error', '401')} 1" in out-- def test_status_less_upstream_error_is_retryable_none(self) -> None:- """An upstream failure with no HTTP status (network) maps to code=none."""- record_upstream_attempt_failure(ProxyContext(), SwitchyardUpstreamError("conn reset"))- out = "\n".join(outcome_metrics.render_lines())- assert f"{_upstream_count(out, 'retryable_error', 'none')} 1" in out-- def test_internal_error_is_not_an_upstream_attempt(self) -> None:- """A non-upstream chain failure (e.g. translation/processor) records nothing."""- record_upstream_attempt_failure(ProxyContext(), ValueError("internal bug"))- out = "\n".join(outcome_metrics.render_lines())- # Every seeded series stays at 0 — no attempt was attributed.- assert f"{_upstream_count(out, 'success', '200')} 0" in out- assert f"{_upstream_count(out, 'retryable_error', 'none')} 0" in out-- def test_dedup_flag_suppresses_fallback(self) -> None:- """A backend that records its own attempts opts the endpoint out."""- ctx = ProxyContext()- ctx.metadata[CTX_UPSTREAM_ATTEMPTS_RECORDED] = True- record_upstream_attempt_success(ctx)- record_upstream_attempt_failure(ctx, SwitchyardUpstreamError("boom"))- out = "\n".join(outcome_metrics.render_lines())- assert f"{_upstream_count(out, 'success', '200')} 0" in out- assert f"{_upstream_count(out, 'retryable_error', 'none')} 0" in out---# ----------------------------------------------------------------------------# Client-response middleware — only the /v1/* LLM routes feed the client-# outcome counter; the operational routes (/metrics, /health, /v1/models) do-# not, so a scraper polling them cannot inflate the success count.-# ------------------------------------------------------------------------------class _CannedBackend(LLMBackend):- """Passthrough-style backend: returns one canned OpenAI completion per call."""-- def supported_request_types(self) -> list[ChatRequestType]:- return [- ChatRequestType.OPENAI_CHAT,- ChatRequestType.OPENAI_RESPONSES,- ChatRequestType.ANTHROPIC,- ]-- async def call(self, ctx: ProxyContext, request: ChatRequest) -> ChatResponse:- return ChatResponse.openai_completion(- ChatCompletion(- id="chatcmpl-test",- object="chat.completion",- created=1700000000,- model="mock-model",- choices=[- Choice(- index=0,- message=ChatCompletionMessage(role="assistant", content="hi"),- finish_reason="stop",- )- ],- usage=CompletionUsage(prompt_tokens=5, completion_tokens=3, total_tokens=8),- )- )---def _canned_app() -> object:- return build_switchyard_app(- Switchyard(backend=_CannedBackend(), translator=TranslationEngine())- )---class TestClientResponseMiddleware:- def test_successful_chat_completion_counts_as_success(self) -> None:- """A served /v1/chat/completions success increments the client outcome counter."""- with TestClient(_canned_app(), raise_server_exceptions=False) as client:- response = client.post(- "/v1/chat/completions",- json={"model": "mock-model", "messages": [{"role": "user", "content": "hi"}]},- )- assert response.status_code == 200-- metrics = "\n".join(outcome_metrics.render_lines())- assert 'switchyard_client_responses_total{outcome="success"} 1' in metrics- assert 'switchyard_client_responses_total{outcome="retryable_error"} 0' in metrics-- def test_operational_routes_are_not_counted_as_client_responses(self) -> None:- """Polling /metrics, /health, /v1/models must not feed the client counter —- otherwise a scraper on a fixed cadence would inflate the success count."""- with TestClient(_canned_app(), raise_server_exceptions=False) as client:- for _ in range(5):- client.get("/metrics")- client.get("/health")- client.get("/v1/models")-- metrics = "\n".join(outcome_metrics.render_lines())- assert 'switchyard_client_responses_total{outcome="success"} 0' in metricsdiff --git a/tests/test_prometheus_emitter.py b/tests/test_prometheus_emitter.pydeleted file mode 100644index 698b5aca8..000000000--- a/tests/test_prometheus_emitter.py+++ /dev/null@@ -1,56 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Unit tests for the module-level Prometheus emitter table."""--from __future__ import annotations--from collections.abc import Iterator--import pytest--from switchyard.lib.endpoints import prometheus_emitter--[email protected](autouse=True)-def _clean_table() -> Iterator[None]:- """Isolate every test: emitters registered here must not leak."""- prometheus_emitter._clear_for_tests()- yield- prometheus_emitter._clear_for_tests()---class TestTableLifecycle:- def test_render_empty_returns_empty_string(self) -> None:- """No emitters → empty string so callers can concat unconditionally."""- assert prometheus_emitter.render() == ""-- def test_register_then_render_composes_lines(self) -> None:- prometheus_emitter.register(lambda: ["foo 1", "bar 2"])- rendered = prometheus_emitter.render()- assert "foo 1" in rendered- assert "bar 2" in rendered- assert rendered.endswith("\n")-- def test_register_is_idempotent(self) -> None:- """Re-registering the same callable must not double-count."""- emitter = lambda: ["x 1"] # noqa: E731- prometheus_emitter.register(emitter)- prometheus_emitter.register(emitter)- assert prometheus_emitter.render().count("x 1") == 1-- def test_unregister_removes_emitter(self) -> None:- emitter = lambda: ["x 1"] # noqa: E731- prometheus_emitter.register(emitter)- prometheus_emitter.unregister(emitter)- assert prometheus_emitter.render() == ""-- def test_unregister_unknown_is_noop(self) -> None:- """Shutdown paths should not throw if registration was skipped."""- prometheus_emitter.unregister(lambda: ["x"])-- def test_multiple_emitters_compose_in_registration_order(self) -> None:- prometheus_emitter.register(lambda: ["a 1"])- prometheus_emitter.register(lambda: ["b 2"])- rendered = prometheus_emitter.render()- assert rendered.index("a 1") < rendered.index("b 2")diff --git a/tests/test_prometheus_exposition.py b/tests/test_prometheus_exposition.pydeleted file mode 100644index 4431a41ef..000000000--- a/tests/test_prometheus_exposition.py+++ /dev/null@@ -1,176 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Unit tests for :mod:`switchyard.lib.prometheus_exposition`.--Covers metric names, label sets, summary quantile rendering, and the-empty-accumulator edge case. Exposition is parsed line-by-line back into-samples for assertion rather than string-matched, so reorder-safe.-"""--from __future__ import annotations--import re--from switchyard.lib.prometheus_exposition import render_prometheus-from switchyard.lib.stats_accumulator import StatsAccumulator--_SAMPLE_RE = re.compile(r"^(?P<name>[a-zA-Z_:][a-zA-Z0-9_:]*)(?P<labels>\{[^}]*\})? (?P<value>.+)$")---def _parse(exposition: str) -> tuple[dict[str, str], dict[str, str], dict[tuple[str, frozenset[tuple[str, str]]], str]]:- """Return ``(help_map, type_map, samples)`` from exposition text.-- samples keys are ``(metric_name, frozenset(labels.items()))``.- """- help_map: dict[str, str] = {}- type_map: dict[str, str] = {}- samples: dict[tuple[str, frozenset[tuple[str, str]]], str] = {}- for raw in exposition.splitlines():- line = raw.strip()- if not line:- continue- if line.startswith("# HELP "):- name, _, text = line[len("# HELP ") :].partition(" ")- help_map[name] = text- continue- if line.startswith("# TYPE "):- name, _, kind = line[len("# TYPE ") :].partition(" ")- type_map[name] = kind- continue- match = _SAMPLE_RE.match(line)- assert match, f"unparseable exposition line: {line!r}"- name = match.group("name")- labels_blob = match.group("labels") or ""- labels: dict[str, str] = {}- if labels_blob:- inner = labels_blob[1:-1]- # Cheap parser — values never contain "," or "\"" in these tests.- for part in inner.split(","):- k, _, v = part.partition("=")- labels[k] = v.strip('"')- samples[(name, frozenset(labels.items()))] = match.group("value")- return help_map, type_map, samples---async def test_empty_accumulator_renders_zero_totals():- snapshot = await StatsAccumulator().snapshot()- text = render_prometheus(snapshot)-- help_map, type_map, samples = _parse(text)-- assert type_map["switchyard_total_requests"] == "gauge"- assert type_map["switchyard_total_errors"] == "gauge"- assert samples[("switchyard_total_requests", frozenset())] == "0"- assert samples[("switchyard_total_errors", frozenset())] == "0"- # Routing overhead summary should still emit a header even with no data.- assert type_map["switchyard_routing_overhead_ms"] == "summary"- assert samples[("switchyard_routing_overhead_ms_count", frozenset())] == "0"- # Exposition must end in a single trailing newline (scraper requirement).- assert text.endswith("\n")---async def test_two_tier_snapshot_emits_expected_metric_names_and_labels():- stats = StatsAccumulator()- await stats.record_success(model="strong/m", backend_latency_ms=42.5, tier="strong")- await stats.record_usage(- model="strong/m",- prompt_tokens=120,- completion_tokens=30,- cached_tokens=10,- total_latency_ms=88.0,- routing_overhead_ms=8.0,- tier="strong",- )- await stats.record_error(model="weak/m", tier="weak")- await stats.record_success(model="weak/m", backend_latency_ms=5.0, tier="weak")- await stats.record_usage(- model="weak/m",- prompt_tokens=40,- completion_tokens=5,- total_latency_ms=15.0,- routing_overhead_ms=3.0,- tier="weak",- )-- text = render_prometheus(await stats.snapshot())- _, type_map, samples = _parse(text)-- # Every metric the ticket lists must be present with the expected type.- expected_types = {- "switchyard_total_requests": "gauge",- "switchyard_total_errors": "gauge",- "switchyard_requests_total": "counter",- "switchyard_errors_total": "counter",- "switchyard_prompt_tokens_total": "counter",- "switchyard_completion_tokens_total": "counter",- "switchyard_cached_tokens_total": "counter",- "switchyard_model_call_latency_ms": "summary",- "switchyard_total_latency_ms": "summary",- "switchyard_routing_overhead_ms": "summary",- }- for metric, kind in expected_types.items():- assert type_map.get(metric) == kind, f"{metric} missing or wrong type"-- strong_labels = frozenset({("model", "strong/m"), ("tier", "strong")})- weak_labels = frozenset({("model", "weak/m"), ("tier", "weak")})-- # weak/m had one record_error then one record_success → calls=1, errors=1.- assert samples[("switchyard_requests_total", strong_labels)] == "1"- assert samples[("switchyard_requests_total", weak_labels)] == "1"- assert samples[("switchyard_errors_total", weak_labels)] == "1"- assert samples[("switchyard_prompt_tokens_total", strong_labels)] == "120"- assert samples[("switchyard_completion_tokens_total", weak_labels)] == "5"- assert samples[("switchyard_cached_tokens_total", strong_labels)] == "10"-- # Summaries must emit quantile=0.5 and quantile=0.99 plus _sum / _count.- strong_p50_key = ("switchyard_model_call_latency_ms", strong_labels | {("quantile", "0.5")})- strong_p99_key = ("switchyard_model_call_latency_ms", strong_labels | {("quantile", "0.99")})- assert strong_p50_key in samples and strong_p99_key in samples- assert samples[("switchyard_model_call_latency_ms_count", strong_labels)] == "1"- assert samples[("switchyard_total_latency_ms_count", weak_labels)] == "1"-- # Global routing-overhead summary has no labels.- assert samples[("switchyard_routing_overhead_ms_count", frozenset())] == "2"---def test_label_value_escapes_backslash_quote_and_newline():- snapshot = {- "total_requests": 0,- "total_errors": 0,- "models": {- 'weird"name\\with\nnewline': {- "calls": 1,- "errors": 0,- "tier": None,- "prompt_tokens": 0,- "completion_tokens": 0,- "cached_tokens": 0,- "cache_creation_tokens": 0,- "reasoning_tokens": 0,- "model_call_latency": {"count": 0, "total_ms": 0, "p50_ms": 0, "p99_ms": 0},- "total_latency": {"count": 0, "total_ms": 0, "p50_ms": 0, "p99_ms": 0},- },- },- "routing_overhead": {"count": 0, "total_ms": 0, "p50_ms": 0, "p99_ms": 0},- }- text = render_prometheus(snapshot)- # Escaped form per Prometheus exposition spec.- assert 'model="weird\\"name\\\\with\\nnewline"' in text---def test_build_info_gauge_present() -> None:- from importlib.metadata import version as pkg_version- ver = pkg_version("nemo-switchyard")- text = render_prometheus({"total_requests": 0, "total_errors": 0, "models": {}})- _, type_map, samples = _parse(text)- assert type_map.get("switchyard_build_info") == "gauge"- key = ("switchyard_build_info", frozenset({("version", ver)}))- assert samples[key] == "1"---def test_build_info_gauge_carries_version_label() -> None:- from importlib.metadata import version as pkg_version- expected = pkg_version("nemo-switchyard")- text = render_prometheus({"total_requests": 0, "total_errors": 0, "models": {}})- assert f'version="{expected}"' in textdiff --git a/tests/test_python_server_passthrough.py b/tests/test_python_server_passthrough.pydeleted file mode 100644index 4dc5eabaa..000000000--- a/tests/test_python_server_passthrough.py+++ /dev/null@@ -1,482 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""End-to-end Python server passthrough tests with a mocked upstream.--This file is the highest-leverage entry in the mocked-test suite added after-the post-PR-#12 wiring regressions. Where ``test_inference_e2e.py`` swaps in-a fake ``LLMBackend`` subclass — bypassing the entire ``OpenAiPassthroughBackend`` +-openai-SDK + httpx code path — these tests exercise that path end-to-end:-- inbound HTTP request- → Switchyard chain (real RequestProcessors / ResponseProcessors)- → OpenAiPassthroughBackend (Rust HTTP call)- → local OpenAI-compatible upstream stub- → response back through the chain- → outbound HTTP response--A regression anywhere in that pipeline (renamed module, missing kwarg, wrong-base_url forwarding, stale processor wiring, broken stats accumulator)-fails one of these tests instead of slipping to top-of-tree.--All tests run offline: ASGITransport intercepts inbound HTTP, the upstream-is a local loopback HTTP server, and no external network is touched.-"""--from __future__ import annotations--import json-import threading-from collections.abc import AsyncIterator, Iterator-from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer-from typing import Any--import httpx-import pytest--from switchyard.cli.route_bundle import build_route_bundle_table-from switchyard.lib.endpoints import outcome_metrics-from switchyard.server.switchyard_app import build_switchyard_app--# ----------------------------------------------------------------------------# Upstream payloads-# ------------------------------------------------------------------------------def _completion_payload(*, content: str = "hello back") -> dict[str, object]:- """An OpenAI Chat Completion JSON body (non-streaming)."""- return {- "id": "chatcmpl-upstream",- "object": "chat.completion",- "created": 1700000000,- "model": "upstream-model",- "choices": [- {- "index": 0,- "message": {"role": "assistant", "content": content},- "finish_reason": "stop",- }- ],- "usage": {"prompt_tokens": 5, "completion_tokens": 3, "total_tokens": 8},- }---def _completion_chunk(*, content: str = "", finish: str | None = None) -> dict[str, object]:- delta: dict[str, object] = {}- if content:- delta["content"] = content- return {- "id": "chatcmpl-upstream-stream",- "object": "chat.completion.chunk",- "created": 1700000000,- "model": "upstream-model",- "choices": [{"index": 0, "delta": delta, "finish_reason": finish}],- }---def _sse_stream_body(chunks: list[dict[str, object]]) -> bytes:- """Encode chunks as the upstream SSE wire format the OpenAI SDK reads."""- out = []- for c in chunks:- out.append(f"data: {json.dumps(c)}\n\n")- out.append("data: [DONE]\n\n")- return "".join(out).encode("utf-8")---class _OpenAICompatStub:- def __init__(self) -> None:- self._server: ThreadingHTTPServer | None = None- self._thread: threading.Thread | None = None- self._lock = threading.Lock()- self._requests: list[dict[str, Any]] = []- self._responses: list[tuple[int, bytes, str]] = []-- def __enter__(self) -> _OpenAICompatStub:- owner = self-- class Handler(BaseHTTPRequestHandler):- protocol_version = "HTTP/1.1"-- def do_POST(self) -> None:- length = int(self.headers.get("content-length", "0"))- raw = self.rfile.read(length)- body = json.loads(raw.decode("utf-8"))- with owner._lock:- owner._requests.append({- "path": self.path,- "authorization": self.headers.get("authorization"),- "body": body,- })- if owner._responses:- status, content, content_type = owner._responses.pop(0)- else:- status = 500- content = b'{"error":{"message":"no stub response queued"}}'- content_type = "application/json"-- self.send_response(status)- self.send_header("content-type", content_type)- self.send_header("content-length", str(len(content)))- self.send_header("connection", "close")- self.end_headers()- self.wfile.write(content)-- def log_message(self, _format: str, *args: object) -> None:- return None-- self._server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)- self._thread = threading.Thread(target=self._server.serve_forever, daemon=True)- self._thread.start()- return self-- def __exit__(self, *args: object) -> None:- if self._server is not None:- self._server.shutdown()- self._server.server_close()- if self._thread is not None:- self._thread.join(timeout=2)-- @property- def base_url(self) -> str:- if self._server is None:- raise RuntimeError("stub server is not running")- host, port = self._server.server_address- return f"http://{host}:{port}/v1"-- @property- def requests(self) -> list[dict[str, Any]]:- with self._lock:- return list(self._requests)-- def respond_json(self, status: int, body: dict[str, object]) -> None:- content = json.dumps(body).encode("utf-8")- with self._lock:- self._responses.append((status, content, "application/json"))-- def respond_sse(self, body: bytes) -> None:- with self._lock:- self._responses.append((200, body, "text/event-stream"))---# ----------------------------------------------------------------------------# Fixtures-# ------------------------------------------------------------------------------def passthrough_upstream() -> Iterator[_OpenAICompatStub]:- with _OpenAICompatStub() as upstream:- yield upstream---async def passthrough_client(- passthrough_upstream: _OpenAICompatStub,-) -> AsyncIterator[httpx.AsyncClient]:- """Return a client using the retained passthrough route."""- table = build_route_bundle_table({- "defaults": {- "api_key": "test-key-not-used",- "base_url": passthrough_upstream.base_url,- "format": "openai",- },- "routes": {- "any-model": {"type": "passthrough", "target": "any-model"},- },- })- app = build_switchyard_app(table)- # ``raise_app_exceptions=False`` mirrors what uvicorn does in- # production: an unhandled exception inside a route is mapped to a- # 500 response, not propagated up the call stack. Without this the- # error-path tests would see the openai SDK exception escape the- # transport entirely.- async with httpx.AsyncClient(- transport=httpx.ASGITransport(app=app, raise_app_exceptions=False),- base_url="http://test",- ) as client:- yield client---# ----------------------------------------------------------------------------# Passthrough route — OpenAI inbound, OpenAI upstream-# ------------------------------------------------------------------------------class TestPassthroughOpenAI:- """Inbound OpenAI Chat Completions through the passthrough route."""-- async def test_non_streaming_round_trip(- self,- passthrough_client: httpx.AsyncClient,- passthrough_upstream: _OpenAICompatStub,- ) -> None:- passthrough_upstream.respond_json(200, _completion_payload(content="pong"))-- resp = await passthrough_client.post(- "/v1/chat/completions",- json={- "model": "any-model",- "messages": [{"role": "user", "content": "ping"}],- },- )-- assert resp.status_code == 200, resp.text- body = resp.json()- assert body["choices"][0]["message"]["content"] == "pong"- assert passthrough_upstream.requests, "passthrough never invoked the upstream"-- async def test_streaming_round_trip(- self,- passthrough_client: httpx.AsyncClient,- passthrough_upstream: _OpenAICompatStub,- ) -> None:- chunks = [- _completion_chunk(content="foo"),- _completion_chunk(content="bar"),- _completion_chunk(finish="stop"),- ]- passthrough_upstream.respond_sse(_sse_stream_body(chunks))-- resp = await passthrough_client.post(- "/v1/chat/completions",- json={- "model": "any-model",- "messages": [{"role": "user", "content": "ping"}],- "stream": True,- },- )-- assert resp.status_code == 200, resp.text- assert "text/event-stream" in resp.headers["content-type"]- assert passthrough_upstream.requests-- data_lines = [- line[6:]- for line in resp.text.split("\n")- if line.startswith("data: ") and line != "data: [DONE]"- ]- assert data_lines, "expected at least one outbound data frame"- decoded = [json.loads(line) for line in data_lines]- joined = "".join(- c["choices"][0]["delta"].get("content", "")- for c in decoded- if c.get("choices")- )- assert joined == "foobar"---# ----------------------------------------------------------------------------# Passthrough route — Anthropic inbound, OpenAI upstream-# ------------------------------------------------------------------------------class TestPassthroughAnthropic:- """Inbound Anthropic Messages, translated to OpenAI for the upstream call."""-- async def test_non_streaming_translates_both_directions(- self,- passthrough_client: httpx.AsyncClient,- passthrough_upstream: _OpenAICompatStub,- ) -> None:- passthrough_upstream.respond_json(200, _completion_payload(content="hi"))-- resp = await passthrough_client.post(- "/v1/messages",- json={- "model": "any-model",- "max_tokens": 16,- "messages": [{"role": "user", "content": "ping"}],- },- )-- assert resp.status_code == 200, resp.text- body = resp.json()- assert body["type"] == "message"- assert body["role"] == "assistant"- assert body["content"][0]["type"] == "text"- assert body["content"][0]["text"] == "hi"- assert body["stop_reason"] == "end_turn"- assert passthrough_upstream.requests, "Anthropic inbound did not reach the OpenAI upstream"-- # Verify the request the upstream SAW was already in OpenAI Chat- # Completions shape — i.e. the request translator ran.- upstream_body = passthrough_upstream.requests[-1]["body"]- assert "messages" in upstream_body- assert upstream_body["messages"][-1]["content"] == "ping"---# ----------------------------------------------------------------------------# Passthrough route — Responses API inbound, OpenAI Chat upstream-# ------------------------------------------------------------------------------class TestPassthroughResponsesApi:- """Inbound OpenAI Responses API, translated to Chat Completions upstream."""-- async def test_non_streaming_translates_both_directions(- self,- passthrough_client: httpx.AsyncClient,- passthrough_upstream: _OpenAICompatStub,- ) -> None:- passthrough_upstream.respond_json(200, _completion_payload(content="resp-ok"))-- resp = await passthrough_client.post(- "/v1/responses",- json={"model": "any-model", "input": "ping"},- )-- assert resp.status_code == 200, resp.text- body = resp.json()- msg = body["output"][0]- assert msg["type"] == "message"- assert msg["role"] == "assistant"- assert msg["content"][0]["type"] == "output_text"- assert msg["content"][0]["text"] == "resp-ok"- assert passthrough_upstream.requests---# ----------------------------------------------------------------------------# Backend errors propagate as HTTP errors, not 200 with garbage-# ------------------------------------------------------------------------------class TestPassthroughBackendErrors:- """Upstream HTTP errors must surface as HTTP errors to the client.-- A regression where the chain swallows an upstream 4xx/5xx and still- returns 200 with malformed JSON is exactly the kind of silent failure- we want CI to catch.- """-- async def test_upstream_500_does_not_return_200(- self,- passthrough_client: httpx.AsyncClient,- passthrough_upstream: _OpenAICompatStub,- ) -> None:- passthrough_upstream.respond_json(500, {"error": {"message": "boom"}})-- resp = await passthrough_client.post(- "/v1/chat/completions",- json={- "model": "any-model",- "messages": [{"role": "user", "content": "ping"}],- },- )-- # Whatever the exact mapping is, returning 200 on an upstream 500 is wrong.- assert resp.status_code != 200, (- f"Upstream 500 leaked through as a successful response: body={resp.text!r}"- )- assert resp.status_code >= 400-- async def test_upstream_401_does_not_return_200(- self,- passthrough_client: httpx.AsyncClient,- passthrough_upstream: _OpenAICompatStub,- ) -> None:- passthrough_upstream.respond_json(- 401,- {"error": {"message": "bad key", "type": "invalid_api_key"}},- )-- resp = await passthrough_client.post(- "/v1/chat/completions",- json={- "model": "any-model",- "messages": [{"role": "user", "content": "ping"}],- },- )-- assert resp.status_code != 200- assert resp.status_code >= 400---# ----------------------------------------------------------------------------# Upstream-attempt outcome counters are wired for the Rust passthrough backend-# ------------------------------------------------------------------------------class TestPassthroughUpstreamAttemptCounters:- """`switchyard_upstream_attempts_total` must populate for passthrough chains.-- The endpoint-layer fallback records one upstream attempt per request for- backends (here the Rust ``OpenAiPassthroughBackend``) that issue exactly- one upstream call and have no Python retry loop — they cannot, by- themselves, reach the Python-only ``outcome_metrics`` counters.- """-- @pytest.fixture(autouse=True)- def _reset_counters(self) -> Iterator[None]:- outcome_metrics._reset_for_tests()- yield- outcome_metrics._reset_for_tests()-- async def test_success_records_one_200_attempt(- self,- passthrough_client: httpx.AsyncClient,- passthrough_upstream: _OpenAICompatStub,- ) -> None:- passthrough_upstream.respond_json(200, _completion_payload(content="ok"))-- resp = await passthrough_client.post(- "/v1/chat/completions",- json={- "model": "any-model",- "messages": [{"role": "user", "content": "ping"}],- },- )- assert resp.status_code == 200, resp.text-- out = "\n".join(outcome_metrics.render_lines())- assert 'switchyard_upstream_attempts_total{outcome="success",code="200"} 1' in out- assert "switchyard_router_retry_recovered_total 0" in out-- async def test_upstream_500_records_one_retryable_attempt(- self,- passthrough_client: httpx.AsyncClient,- passthrough_upstream: _OpenAICompatStub,- ) -> None:- passthrough_upstream.respond_json(500, {"error": {"message": "boom"}})-- resp = await passthrough_client.post(- "/v1/chat/completions",- json={- "model": "any-model",- "messages": [{"role": "user", "content": "ping"}],- },- )- assert resp.status_code >= 400-- out = "\n".join(outcome_metrics.render_lines())- assert (- 'switchyard_upstream_attempts_total{outcome="retryable_error",code="500"} 1'- in out- )- assert 'switchyard_upstream_attempts_total{outcome="success",code="200"} 0' in out-- async def test_upstream_401_records_one_other_error_attempt(- self,- passthrough_client: httpx.AsyncClient,- passthrough_upstream: _OpenAICompatStub,- ) -> None:- passthrough_upstream.respond_json(- 401,- {"error": {"message": "bad key", "type": "invalid_api_key"}},- )-- resp = await passthrough_client.post(- "/v1/chat/completions",- json={- "model": "any-model",- "messages": [{"role": "user", "content": "ping"}],- },- )- assert resp.status_code >= 400-- out = "\n".join(outcome_metrics.render_lines())- assert (- 'switchyard_upstream_attempts_total{outcome="other_error",code="401"} 1'- in out- )- # A 4xx client error is not retryable and never recovers.- assert "switchyard_router_retry_recovered_total 0" in outdiff --git a/tests/test_request_metadata.py b/tests/test_request_metadata.pydeleted file mode 100644index 62e3187aa..000000000--- a/tests/test_request_metadata.py+++ /dev/null@@ -1,129 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Unit tests for header helpers in :mod:`switchyard.lib.request_metadata`."""--from __future__ import annotations--import pytest--from switchyard.lib.proxy_context import CTX_CALLER_API_KEY, ProxyContext-from switchyard.lib.request_metadata import (- CTX_REQUEST_HEADERS,- RequestMetadata,- attach_caller_api_key,- attach_request_metadata,- extract_caller_api_key,- redact_sensitive_headers,-)---class TestExtractCallerApiKey:- """``extract_caller_api_key`` parses caller credentials from HTTP headers.-- Multi-tenant deploys forward each caller's key per request. The dedicated- ``x-switchyard-api-key`` header is preferred (it survives proxies such as- LiteLLM that strip ``Authorization``); ``Authorization: Bearer <key>`` and- ``x-api-key`` remain supported for direct callers. The codex launcher sends- ``"switchyard"`` as a sentinel placeholder, which must not be forwarded.- """-- @pytest.mark.parametrize(- "headers, expected",- [- ({"Authorization": "Bearer nvapi-real"}, "nvapi-real"),- ({"authorization": "bearer nvapi-lowercase"}, "nvapi-lowercase"),- ({"x-api-key": "nvapi-via-x-api-key"}, "nvapi-via-x-api-key"),- ({"X-Api-Key": "nvapi-titlecase-header"}, "nvapi-titlecase-header"),- # The dedicated forwarded header is honored...- ({"x-switchyard-api-key": "nvapi-forwarded"}, "nvapi-forwarded"),- ({"X-Switchyard-Api-Key": "nvapi-fwd-titlecase"}, "nvapi-fwd-titlecase"),- # ...and wins over Authorization and x-api-key when several are set- # (the case behind a proxy that strips Authorization upstream).- (- {- "x-switchyard-api-key": "forwarded-wins",- "Authorization": "Bearer lose",- "x-api-key": "lose-too",- },- "forwarded-wins",- ),- # Authorization wins over x-api-key when both are present.- (- {"Authorization": "Bearer first", "x-api-key": "second"},- "first",- ),- # Surrounding whitespace is stripped.- ({"Authorization": "Bearer nvapi-spaces "}, "nvapi-spaces"),- ({"x-switchyard-api-key": " nvapi-fwd-spaces "}, "nvapi-fwd-spaces"),- ],- )- def test_extraction(self, headers: dict[str, str], expected: str) -> None:- assert extract_caller_api_key(headers) == expected-- @pytest.mark.parametrize(- "headers",- [- {},- {"Authorization": ""},- {"Authorization": "Bearer "},- # Non-bearer schemes are not forwarded.- {"Authorization": "Basic dXNlcjpwYXNz"},- # Codex launcher sentinel — in any supported header.- {"Authorization": "Bearer switchyard"},- {"x-api-key": "switchyard"},- {"x-switchyard-api-key": "switchyard"},- {"x-switchyard-api-key": ""},- # Case-insensitive sentinel match — both halves of the case- # space should be treated as the same placeholder.- {"Authorization": "Bearer Switchyard"},- ],- )- def test_no_key_returned(self, headers: dict[str, str]) -> None:- assert extract_caller_api_key(headers) is None---class TestRedactSensitiveHeaders:- """Credential headers are scrubbed before the header map is retained."""-- def test_redacts_credential_headers(self) -> None:- headers = {- "Authorization": "Bearer nvapi-real",- "x-api-key": "nvapi-x",- "x-switchyard-api-key": "nvapi-forwarded",- "x-switchyard-intake-task": "demo",- "content-type": "application/json",- }- redacted = redact_sensitive_headers(headers)- assert redacted["Authorization"] == "[REDACTED]"- assert redacted["x-api-key"] == "[REDACTED]"- assert redacted["x-switchyard-api-key"] == "[REDACTED]"- # Non-credential headers pass through untouched.- assert redacted["x-switchyard-intake-task"] == "demo"- assert redacted["content-type"] == "application/json"-- def test_matching_is_case_insensitive(self) -> None:- redacted = redact_sensitive_headers({"X-Switchyard-Api-Key": "nvapi-real"})- assert redacted["X-Switchyard-Api-Key"] == "[REDACTED]"---class TestCallerKeyForwardedButNotRetained:- """The endpoint extracts the caller key for upstream use, then retains a- redacted header map so the key cannot leak into logs or traces."""-- def test_key_extracted_but_redacted_in_stored_headers(self) -> None:- headers = {- "x-switchyard-api-key": "nvapi-secret",- "x-switchyard-intake-task": "demo",- }- ctx = ProxyContext()- # Mirror the endpoint: both helpers receive the raw headers.- attach_request_metadata(ctx, RequestMetadata.from_headers(headers), headers)- attach_caller_api_key(ctx, headers)-- # Extracted for upstream forwarding...- assert ctx.metadata[CTX_CALLER_API_KEY] == "nvapi-secret" # pragma: allowlist secret- # ...but the retained header map carries no raw credential.- stored = ctx.metadata[CTX_REQUEST_HEADERS]- assert stored["x-switchyard-api-key"] == "[REDACTED]"- assert stored["x-switchyard-intake-task"] == "demo"diff --git a/tests/test_request_translation_engine.py b/tests/test_request_translation_engine.pydeleted file mode 100644index 1e01c831b..000000000--- a/tests/test_request_translation_engine.py+++ /dev/null@@ -1,495 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Tests for the unified TranslationEngine request wrapper."""--import copy-import json--from switchyard_rust.core import ChatRequest, ChatRequestType, request_type_matches-from switchyard_rust.translation import TranslationEngine--E = TranslationEngine()---# =========================================================================-# to_openai_chat-# =========================================================================---class TestToOpenAIChat:- def test_openai_passthrough(self):- """OpenAI chat requests pass through unchanged (same object)."""- body = {"model": "gpt-4o", "messages": [{"role": "user", "content": "Hi"}]}- req = ChatRequest.openai_chat(body)- result = E.request_to(ChatRequestType.OPENAI_CHAT, req)- assert result is req-- def test_anthropic_to_openai_basic(self):- """Anthropic requests are converted to OpenAI chat requests."""- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "Hello"}],- "max_tokens": 1024,- "system": "Be helpful.",- }- req = ChatRequest.anthropic(body)- result = E.request_to(ChatRequestType.OPENAI_CHAT, req)-- assert request_type_matches(result, ChatRequestType.OPENAI_CHAT)- msgs = result.body["messages"]- # System prompt becomes first message- assert msgs[0]["role"] == "system"- assert msgs[0]["content"] == "Be helpful."- # User message follows- assert msgs[1]["role"] == "user"- assert msgs[1]["content"] == "Hello"- assert result.body["model"] == "claude-sonnet-4-20250514"-- def test_anthropic_to_openai_with_tools(self):- """Anthropic tools are converted to OpenAI tool format."""- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "What's the weather?"}],- "max_tokens": 1024,- "tools": [- {- "name": "get_weather",- "description": "Get weather for a location",- "input_schema": {- "type": "object",- "properties": {"location": {"type": "string"}},- },- }- ],- }- req = ChatRequest.anthropic(body)- result = E.request_to(ChatRequestType.OPENAI_CHAT, req)-- assert request_type_matches(result, ChatRequestType.OPENAI_CHAT)- tools = result.body.get("tools", [])- assert len(tools) == 1- assert tools[0]["function"]["name"] == "get_weather"-- def test_anthropic_to_openai_does_not_mutate_original(self):- """The original Anthropic request body is not modified."""- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "Hi"}],- "max_tokens": 1024,- }- original = copy.deepcopy(body)- req = ChatRequest.anthropic(body)- E.request_to(ChatRequestType.OPENAI_CHAT, req)- assert req.body == original-- def test_responses_to_openai_basic(self):- """Responses requests are converted to OpenAI chat requests."""- body = {- "model": "gpt-4o",- "input": "Hello world",- "instructions": "Be brief.",- }- req = ChatRequest.openai_responses(body)- result = E.request_to(ChatRequestType.OPENAI_CHAT, req)-- assert request_type_matches(result, ChatRequestType.OPENAI_CHAT)- msgs = result.body["messages"]- assert msgs[0]["role"] == "system"- assert msgs[0]["content"] == "Be brief."- assert msgs[1]["role"] == "user"- assert msgs[1]["content"] == "Hello world"-- def test_responses_to_openai_with_tools(self):- """Responses API tools are converted to OpenAI tool format."""- body = {- "model": "gpt-4o",- "input": "Get weather",- "tools": [- {- "name": "get_weather",- "description": "Get weather",- "parameters": {- "type": "object",- "properties": {"loc": {"type": "string"}},- },- }- ],- }- req = ChatRequest.openai_responses(body)- result = E.request_to(ChatRequestType.OPENAI_CHAT, req)-- tools = result.body.get("tools", [])- assert len(tools) == 1- assert tools[0]["function"]["name"] == "get_weather"-- def test_responses_to_openai_does_not_mutate_original(self):- """The original Responses request body is not modified."""- body = {"model": "gpt-4o", "input": "Hi"}- original = copy.deepcopy(body)- req = ChatRequest.openai_responses(body)- E.request_to(ChatRequestType.OPENAI_CHAT, req)- assert req.body == original-- def test_anthropic_only_top_level_fields_dropped(self):- """Anthropic-only fields must not leak into the OpenAI request — the- OpenAI SDK rejects them with TypeError at call time.-- Claude Code sends several of these on every request (``thinking`` for- extended thinking, ``cache_control`` for prompt caching,- ``context_management`` for long-context management, ``container`` for- Claude containers). Including a synthetic ``made_up_beta_field`` to- assert the whitelist strategy also handles future Anthropic-only- fields we don't know about yet.- """- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "Hi"}],- "max_tokens": 1024,- "thinking": {"type": "enabled", "budget_tokens": 8000},- "cache_control": {"type": "ephemeral"},- "container": "my-container",- "inference_geo": "us-east-1",- "output_config": {"some": "config"},- "context_management": {"strategy": "auto"},- "made_up_beta_field": "future-proofing",- }- req = ChatRequest.anthropic(body)- result = E.request_to(ChatRequestType.OPENAI_CHAT, req)- for field in (- "thinking", "cache_control", "container", "inference_geo",- "output_config", "context_management", "made_up_beta_field",- ):- assert field not in result.body, (- f"{field!r} leaked into OpenAI request — "- "OpenAI SDK would reject with TypeError"- )-- def test_anthropic_thinking_content_blocks_do_not_leak_to_openai(self):- """Anthropic thinking blocks are preserved internally, not sent as Chat content."""- body = {- "model": "claude-opus-4-20250514",- "messages": [- {"role": "user", "content": "Use the tool."},- {- "role": "assistant",- "content": [- {- "type": "thinking",- "thinking": "I should call the tool.",- "signature": "sig-abc",- },- {"type": "redacted_thinking", "data": "encrypted"},- {- "type": "tool_use",- "id": "toolu_1",- "name": "lookup",- "input": {"query": "status"},- },- ],- },- ],- "tools": [{"name": "lookup", "input_schema": {"type": "object"}}],- "max_tokens": 2048,- }- req = ChatRequest.anthropic(body)- result = E.request_to(ChatRequestType.OPENAI_CHAT, req)-- assistant = result.body["messages"][1]- assert assistant["role"] == "assistant"- assert assistant["content"] is None- assert "reasoning_content" not in assistant- assert assistant["tool_calls"][0]["function"]["name"] == "lookup"- assert "thinking" not in str(result.body)- assert "redacted_thinking" not in str(result.body)---# =========================================================================-# to_anthropic-# =========================================================================---class TestToAnthropic:- def test_anthropic_passthrough(self):- """Anthropic requests pass through unchanged."""- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "Hi"}],- "max_tokens": 1024,- }- req = ChatRequest.anthropic(body)- result = E.request_to(ChatRequestType.ANTHROPIC, req)- assert result is req-- def test_openai_to_anthropic(self):- """OpenAI chat requests are converted to Anthropic requests."""- body = {- "model": "gpt-4o",- "messages": [- {"role": "system", "content": "Be helpful."},- {"role": "user", "content": "Hello"},- ],- "max_tokens": 1024,- }- req = ChatRequest.openai_chat(body)- result = E.request_to(ChatRequestType.ANTHROPIC, req)-- assert request_type_matches(result, ChatRequestType.ANTHROPIC)- assert result.body["system"] == "Be helpful."- assert result.body["model"] == "gpt-4o"-- def test_openai_to_anthropic_developer_and_system_concat(self):- """OpenAI developer/system messages must not leak as invalid Anthropic roles."""- body = {- "model": "gpt-4o",- "messages": [- {"role": "system", "content": "System rules."},- {"role": "developer", "content": "Developer rules."},- {"role": "user", "content": "Hello"},- ],- }- req = ChatRequest.openai_chat(body)- result = E.request_to(ChatRequestType.ANTHROPIC, req)-- assert result.body["system"] == "System rules.\n\nDeveloper rules."- assert [m["role"] for m in result.body["messages"]] == ["user"]-- def test_openai_to_anthropic_uses_max_completion_tokens(self):- body = {- "model": "gpt-4o",- "messages": [{"role": "user", "content": "Hello"}],- "max_completion_tokens": 512,- }- req = ChatRequest.openai_chat(body)- result = E.request_to(ChatRequestType.ANTHROPIC, req)- assert result.body["max_tokens"] == 512-- def test_openai_to_anthropic_maps_reasoning_effort(self):- body = {- "model": "gpt-4o",- "messages": [{"role": "user", "content": "Hello"}],- "reasoning_effort": "high",- }- req = ChatRequest.openai_chat(body)- result = E.request_to(ChatRequestType.ANTHROPIC, req)- assert result.body["thinking"] == {"type": "adaptive"}- assert result.body["output_config"] == {"effort": "high"}-- def test_openai_to_anthropic_maps_image_url_content(self):- body = {- "model": "gpt-4o",- "messages": [- {- "role": "user",- "content": [- {"type": "text", "text": "Describe"},- {- "type": "image_url",- "image_url": {- "url": "https://example.test/image.png",- },- },- ],- }- ],- }- req = ChatRequest.openai_chat(body)- result = E.request_to(ChatRequestType.ANTHROPIC, req)- content = result.body["messages"][0]["content"]- assert content == [- {"type": "text", "text": "Describe"},- {- "type": "image",- "source": {- "type": "url",- "url": "https://example.test/image.png",- },- },- ]-- def test_openai_to_anthropic_merges_consecutive_tool_results(self):- body = {- "model": "gpt-4o",- "messages": [- {"role": "user", "content": "call tools"},- {- "role": "assistant",- "content": None,- "tool_calls": [- {- "id": "call_1",- "type": "function",- "function": {"name": "a", "arguments": "{}"},- },- {- "id": "call_2",- "type": "function",- "function": {"name": "b", "arguments": "{}"},- },- ],- },- {"role": "tool", "tool_call_id": "call_1", "content": "one"},- {"role": "tool", "tool_call_id": "call_2", "content": "two"},- ],- }- req = ChatRequest.openai_chat(body)- result = E.request_to(ChatRequestType.ANTHROPIC, req)-- tool_result_msg = result.body["messages"][2]- assert tool_result_msg["role"] == "user"- assert tool_result_msg["content"] == [- {"type": "tool_result", "tool_use_id": "call_1", "content": "one"},- {"type": "tool_result", "tool_use_id": "call_2", "content": "two"},- ]-- def test_anthropic_tool_result_followup_text_is_preserved(self):- body = {- "model": "claude-sonnet-4-20250514",- "messages": [- {- "role": "user",- "content": [- {- "type": "tool_result",- "tool_use_id": "toolu_1",- "content": "72F",- },- {"type": "text", "text": "Now summarize it."},- ],- }- ],- "max_tokens": 1024,- }- req = ChatRequest.anthropic(body)- result = E.request_to(ChatRequestType.OPENAI_CHAT, req)- assert result.body["messages"] == [- {"role": "tool", "tool_call_id": "toolu_1", "content": "72F"},- {"role": "user", "content": "Now summarize it."},- ]-- def test_openai_to_anthropic_sanitizes_tool_call_ids(self):- """Tool call IDs and matching tool results must stay valid together."""- invalid_id = "call.bad:id/with space"- body = {- "model": "gpt-4o",- "messages": [- {"role": "user", "content": "search"},- {- "role": "assistant",- "content": None,- "tool_calls": [- {- "id": invalid_id,- "type": "function",- "function": {- "name": "search",- "arguments": '{"q":"test"}',- },- },- ],- },- {- "role": "tool",- "tool_call_id": invalid_id,- "content": "ok",- },- ],- "max_tokens": 1024,- }- req = ChatRequest.openai_chat(body)-- result = E.request_to(ChatRequestType.ANTHROPIC, req)-- messages = result.body["messages"]- sanitized_id = "call_bad_id_with_space"- assert messages[1]["content"][0]["id"] == sanitized_id- assert messages[2]["content"][0]["tool_use_id"] == sanitized_id-- def test_openai_unknown_content_does_not_leak_to_anthropic(self):- """Unknown OpenAI content is kept as text instead of raw Anthropic blocks."""- unknown_part = {"type": "future_openai_part", "payload": {"keep": True}}- body = {- "model": "gpt-4o",- "messages": [- {- "role": "user",- "content": [- {"type": "text", "text": "hi"},- unknown_part,- ],- }- ],- }- req = ChatRequest.openai_chat(body)-- result = E.request_to(ChatRequestType.ANTHROPIC, req)-- content = result.body["messages"][0]["content"]- assert content[1]["type"] == "text"- assert json.loads(content[1]["text"]) == unknown_part- assert "future_openai_part" not in [- block.get("type") for block in content if isinstance(block, dict)- ]-- def test_responses_to_anthropic(self):- """Responses requests can be converted to Anthropic requests."""- body = {"model": "gpt-4o", "input": "Hi"}- req = ChatRequest.openai_responses(body)- result = E.request_to(ChatRequestType.ANTHROPIC, req)- assert request_type_matches(result, ChatRequestType.ANTHROPIC)- assert result.body["messages"] == [{"role": "user", "content": "Hi"}]-- def test_responses_tool_arguments_to_anthropic_are_object_shaped(self):- """Responses tool-call argument strings become valid Anthropic tool input objects."""- body = {- "model": "gpt-4o",- "input": [- {"type": "message", "role": "user", "content": "List files"},- {- "type": "function_call",- "name": "exec_command",- "call_id": "call_1",- "arguments": '{"cmd":"ls","limit":2}',- },- ],- }- req = ChatRequest.openai_responses(body)- result = E.request_to(ChatRequestType.ANTHROPIC, req)-- tool_use = result.body["messages"][1]["content"][0]- assert tool_use["type"] == "tool_use"- assert tool_use["input"] == {"cmd": "ls", "limit": 2}---# =========================================================================-# to_responses-# =========================================================================---class TestToResponses:- def test_responses_passthrough(self):- """Responses requests pass through unchanged."""- body = {"model": "gpt-4o", "input": "Hi"}- req = ChatRequest.openai_responses(body)- result = E.request_to(ChatRequestType.OPENAI_RESPONSES, req)- assert result is req-- def test_openai_to_responses(self):- """OpenAI chat requests can be converted to Responses requests."""- body = {"model": "gpt-4o", "messages": [{"role": "user", "content": "Hi"}]}- req = ChatRequest.openai_chat(body)- result = E.request_to(ChatRequestType.OPENAI_RESPONSES, req)- assert request_type_matches(result, ChatRequestType.OPENAI_RESPONSES)- assert result.body["input"] == "Hi"-- def test_anthropic_to_responses(self):- """Anthropic requests can be converted to Responses requests."""- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "Hi"}],- "max_tokens": 1024,- }- req = ChatRequest.anthropic(body)- result = E.request_to(ChatRequestType.OPENAI_RESPONSES, req)- assert request_type_matches(result, ChatRequestType.OPENAI_RESPONSES)- assert result.body["input"] == "Hi"diff --git a/tests/test_request_translation_engine_to_any_of.py b/tests/test_request_translation_engine_to_any_of.pydeleted file mode 100644index f145f843c..000000000--- a/tests/test_request_translation_engine_to_any_of.py+++ /dev/null@@ -1,162 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Unit tests for :meth:`TranslationEngine.request_to_any_of`.--Covers the four behaviors:--- Passthrough when the inbound type is in ``supported``.-- Translation to ``supported[0]`` when the inbound type isn't supported.-- ``ValueError`` on empty ``supported``.-- Any built-in request format can translate to any other built-in format.-"""--import pytest--from switchyard_rust.core import ChatRequest, ChatRequestType, request_type_matches-from switchyard_rust.translation import TranslationEngine--ENGINE = TranslationEngine()--# ----------------------------------------------------------------------------# Fixtures-# ------------------------------------------------------------------------------def _openai_req() -> ChatRequest:- return ChatRequest.openai_chat({ # type: ignore[arg-type]- "model": "gpt-4o",- "messages": [{"role": "user", "content": "hi"}],- })---def _anthropic_req() -> ChatRequest:- return ChatRequest.anthropic({ # type: ignore[arg-type]- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 1024,- })---def _responses_req() -> ChatRequest:- return ChatRequest.openai_responses({ # type: ignore[arg-type]- "model": "gpt-4o",- "input": "hi",- })---# ----------------------------------------------------------------------------# Passthrough (inbound type is in `supported`)-# ------------------------------------------------------------------------------class TestPassthrough:- def test_openai_in_singleton_supported(self):- req = _openai_req()- out = ENGINE.request_to_any_of(- req, [ChatRequestType.OPENAI_CHAT],- )- assert out is req-- def test_anthropic_in_singleton_supported(self):- req = _anthropic_req()- out = ENGINE.request_to_any_of(- req, [ChatRequestType.ANTHROPIC],- )- assert out is req-- def test_responses_in_singleton_supported(self):- req = _responses_req()- out = ENGINE.request_to_any_of(- req, [ChatRequestType.OPENAI_RESPONSES],- )- assert out is req-- def test_openai_in_multi_element_supported(self):- """Passthrough still wins even when supported lists more formats."""- req = _openai_req()- out = ENGINE.request_to_any_of(- req,- [ChatRequestType.OPENAI_RESPONSES, ChatRequestType.OPENAI_CHAT],- )- assert out is req-- def test_anthropic_in_multi_element_supported(self):- req = _anthropic_req()- out = ENGINE.request_to_any_of(- req,- [ChatRequestType.OPENAI_CHAT, ChatRequestType.ANTHROPIC],- )- assert out is req---# ----------------------------------------------------------------------------# Translation (inbound type not in `supported`, translate to supported[0])-# ------------------------------------------------------------------------------class TestTranslation:- def test_openai_chat_to_anthropic(self):- req = _openai_req()- out = ENGINE.request_to_any_of(- req, [ChatRequestType.ANTHROPIC],- )- assert request_type_matches(out, ChatRequestType.ANTHROPIC)- assert out is not req-- def test_anthropic_to_openai_chat(self):- req = _anthropic_req()- out = ENGINE.request_to_any_of(- req, [ChatRequestType.OPENAI_CHAT],- )- assert request_type_matches(out, ChatRequestType.OPENAI_CHAT)- assert out is not req-- def test_responses_to_openai_chat(self):- req = _responses_req()- out = ENGINE.request_to_any_of(- req, [ChatRequestType.OPENAI_CHAT],- )- assert request_type_matches(out, ChatRequestType.OPENAI_CHAT)- assert out is not req-- def test_translation_target_is_supported_zero(self):- """When multiple unsupported formats translate, supported[0] is chosen."""- req = _anthropic_req()- # Anthropic is not in supported; supported[0] is OPENAI_CHAT, so we- # should translate to OpenAI Chat (not to Responses, the second entry).- out = ENGINE.request_to_any_of(- req,- [ChatRequestType.OPENAI_CHAT, ChatRequestType.OPENAI_RESPONSES],- )- assert request_type_matches(out, ChatRequestType.OPENAI_CHAT)---# ----------------------------------------------------------------------------# Error paths-# ------------------------------------------------------------------------------class TestErrors:- def test_empty_supported_raises_value_error(self):- req = _openai_req()- with pytest.raises(ValueError, match="must be non-empty"):- ENGINE.request_to_any_of(req, [])-- def test_openai_chat_to_responses(self):- """OpenAI Chat can translate to Responses."""- req = _openai_req()- out = ENGINE.request_to_any_of(req, [ChatRequestType.OPENAI_RESPONSES])- assert request_type_matches(out, ChatRequestType.OPENAI_RESPONSES)-- def test_anthropic_to_responses(self):- """Anthropic can translate to Responses."""- req = _anthropic_req()- out = ENGINE.request_to_any_of(req, [ChatRequestType.OPENAI_RESPONSES])- assert request_type_matches(out, ChatRequestType.OPENAI_RESPONSES)-- def test_responses_to_anthropic(self):- """Responses can translate to Anthropic."""- req = _responses_req()- out = ENGINE.request_to_any_of(req, [ChatRequestType.ANTHROPIC])- assert request_type_matches(out, ChatRequestType.ANTHROPIC)diff --git a/tests/test_response_translation_engine.py b/tests/test_response_translation_engine.pydeleted file mode 100644index 6ef605757..000000000--- a/tests/test_response_translation_engine.py+++ /dev/null@@ -1,540 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Tests for the unified TranslationEngine response wrapper."""--from __future__ import annotations--from collections.abc import AsyncIterator-from types import SimpleNamespace-from unittest.mock import MagicMock--import pytest-from openai.types.chat import ChatCompletion-from openai.types.chat.chat_completion import Choice-from openai.types.chat.chat_completion_message import ChatCompletionMessage-from openai.types.completion_usage import CompletionUsage-from openai.types.responses import Response as OpenAIResponse--from switchyard.lib.chat_response.anthropic import AnthropicResponseStream-from switchyard.lib.chat_response.openai_chat import ResponseStream-from switchyard_rust.core import (- ChatRequest,- ChatRequestType,- ChatResponse,- ChatResponseType,- ProxyContext,- response_type_matches,-)-from switchyard_rust.translation import TranslationEngine, sse_frame_payloads--E = TranslationEngine()---def _make_completion(content: str = "Hello!", model: str = "gpt-4o") -> ChatCompletion:- return ChatCompletion(- id="chatcmpl-test",- object="chat.completion",- created=1700000000,- model=model,- choices=[- Choice(- index=0,- message=ChatCompletionMessage(role="assistant", content=content),- finish_reason="stop",- )- ],- usage=CompletionUsage(prompt_tokens=10, completion_tokens=5, total_tokens=15),- )---def _make_anthropic_message(content: str = "Hi there", model: str = "claude-sonnet-4-20250514"):- from anthropic.types import Message, TextBlock, Usage-- return Message(- id="msg_test123",- type="message",- role="assistant",- content=[TextBlock(type="text", text=content)],- model=model,- stop_reason="end_turn",- stop_sequence=None,- usage=Usage(input_tokens=10, output_tokens=5),- )---def _make_responses_api_response(- *, content: str = "Hello from Responses", model: str = "gpt-4o",-) -> OpenAIResponse:- return OpenAIResponse(- id="resp_test",- created_at=1700000000,- model=model,- object="response",- output=[- {- "type": "message",- "id": "msg_test",- "status": "completed",- "role": "assistant",- "content": [{"type": "output_text", "text": content, "annotations": []}],- }- ],- tool_choice="auto",- tools=[],- status="completed",- parallel_tool_calls=True,- text={"format": {"type": "text"}},- )---# =========================================================================-# to_openai_chat-# =========================================================================---class TestToOpenAIChat:- def test_completion_passthrough(self):- """OpenAI completion responses pass through unchanged."""- resp = ChatResponse.openai_completion(_make_completion())- result = E.response_to(ChatRequestType.OPENAI_CHAT, resp)- assert result is resp-- def test_streaming_passthrough(self):- """OpenAI streaming responses pass through unchanged."""- stream = MagicMock(spec=ResponseStream)- resp = ChatResponse.openai_stream(stream)- result = E.response_to(ChatRequestType.OPENAI_CHAT, resp)- assert result is resp-- def test_anthropic_to_openai(self):- """Anthropic responses are converted to OpenAI completion responses."""- resp = ChatResponse.anthropic_completion(_make_anthropic_message(content="Hello world"))- result = E.response_to(ChatRequestType.OPENAI_CHAT, resp)-- assert response_type_matches(result, ChatResponseType.OPENAI_COMPLETION)- assert result.body["choices"][0]["message"]["content"] == "Hello world"- assert result.body["choices"][0]["finish_reason"] == "stop"-- def test_responses_api_to_openai(self):- """Responses API responses are converted to OpenAI completion responses."""- resp = ChatResponse.openai_responses_completion(- _make_responses_api_response(content="Hello from Responses"),- )- result = E.response_to(ChatRequestType.OPENAI_CHAT, resp)-- assert response_type_matches(result, ChatResponseType.OPENAI_COMPLETION)- assert result.body["model"] == "gpt-4o"- assert result.body["choices"][0]["message"]["content"] == "Hello from Responses"- assert result.body["choices"][0]["finish_reason"] == "stop"---# =========================================================================-# to_anthropic-# =========================================================================---class TestToAnthropic:- def test_anthropic_passthrough(self):- """Anthropic completion responses pass through unchanged."""- resp = ChatResponse.anthropic_completion(_make_anthropic_message())- result = E.response_to(ChatRequestType.ANTHROPIC, resp)- assert result is resp-- def test_anthropic_streaming_passthrough(self):- """Anthropic streaming responses pass through unchanged."""- from switchyard.lib.chat_response.anthropic import AnthropicResponseStream-- stream = MagicMock(spec=AnthropicResponseStream)- resp = ChatResponse.anthropic_stream(stream)- result = E.response_to(ChatRequestType.ANTHROPIC, resp)- assert result is resp-- def test_openai_to_anthropic(self):- """OpenAI completion responses are converted to Anthropic responses."""- resp = ChatResponse.openai_completion(_make_completion(content="Hello world"))- result = E.response_to(ChatRequestType.ANTHROPIC, resp)-- assert response_type_matches(result, ChatResponseType.ANTHROPIC_COMPLETION)- body = result.body- assert body["role"] == "assistant"- assert body["stop_reason"] == "end_turn"- assert len(body["content"]) == 1- assert body["content"][0]["text"] == "Hello world"-- def test_responses_api_to_anthropic(self):- """Responses API responses are converted to Anthropic responses."""- resp = ChatResponse.openai_responses_completion(_make_responses_api_response(content="Hello world"))- result = E.response_to(ChatRequestType.ANTHROPIC, resp)-- assert response_type_matches(result, ChatResponseType.ANTHROPIC_COMPLETION)- assert result.body["content"][0]["text"] == "Hello world"---# =========================================================================-# to_responses-# =========================================================================---class TestToResponses:- def test_responses_passthrough(self):- """Responses API completion responses pass through unchanged."""- resp = ChatResponse.openai_responses_completion(_make_responses_api_response())- result = E.response_to(ChatRequestType.OPENAI_RESPONSES, resp)- assert result is resp-- def test_responses_streaming_passthrough(self):- """Responses API streaming responses pass through unchanged."""- from switchyard.lib.chat_response.openai_responses import ResponsesApiStream-- resp = ChatResponse.openai_responses_stream(ResponsesApiStream(_async_iter([])))- result = E.response_to(ChatRequestType.OPENAI_RESPONSES, resp)- assert result is resp-- def test_openai_to_responses(self):- """OpenAI completion responses are converted to Responses API responses."""- resp = ChatResponse.openai_completion(_make_completion(content="Hello!"))- result = E.response_to(ChatRequestType.OPENAI_RESPONSES, resp)-- assert response_type_matches(result, ChatResponseType.OPENAI_RESPONSES_COMPLETION)- body = result.body- assert body["object"] == "response"- assert body["status"] == "completed"- assert len(body["output"]) == 1- assert body["output"][0]["type"] == "message"-- def test_anthropic_to_responses(self):- """Anthropic responses are converted to Responses API responses."""- resp = ChatResponse.anthropic_completion(_make_anthropic_message())- result = E.response_to(ChatRequestType.OPENAI_RESPONSES, resp)-- assert response_type_matches(result, ChatResponseType.OPENAI_RESPONSES_COMPLETION)- assert result.body["output"][0]["content"][0]["text"] == "Hi there"---# =========================================================================-# stream_openai_to_anthropic — usage propagation-# =========================================================================---def _chunk(- content: str | None = None,- reasoning_content: str | None = None,- finish_reason: str | None = None,- usage: SimpleNamespace | None = None,-) -> SimpleNamespace:- """Build a minimal OpenAI-shaped streaming chunk."""- delta = SimpleNamespace(- content=content,- reasoning=None,- reasoning_content=reasoning_content,- tool_calls=None,- )- choice = SimpleNamespace(delta=delta, finish_reason=finish_reason)- chunk = SimpleNamespace(choices=[choice])- if usage is not None:- chunk.usage = usage- return chunk---async def _async_iter(items: list) -> AsyncIterator:- for item in items:- yield item---async def _collect_events(chunks: list) -> list[dict]:- events: list[dict] = []- async for ev in E.translate_stream(- ChatRequestType.OPENAI_CHAT,- ChatRequestType.ANTHROPIC,- _async_iter(chunks),- model="test-model",- ):- events.append(ev)- return events---def _find_event(events: list[dict], event_type: str) -> dict:- return next(ev for ev in events if ev["type"] == event_type)---# =========================================================================-# terminal translate() role — served-model stamping (SWITCH-922)-# =========================================================================---_ROUTE_ID = "switchyard"-_SERVED_MODEL = "served/model"---def _routed_ctx(served_model: str | None = _SERVED_MODEL) -> ProxyContext:- """A context shaped like one a router leaves behind for the terminal role."""- ctx = ProxyContext()- ctx.selected_model = served_model- return ctx---class TestTranslateStampsTheServedModel:- """The client-visible ``model`` must name the model that answered.-- Routers rewrite the model on their own copy of the request, so the request- reaching the terminal role still names the route the client addressed- (``switchyard`` for the benchmark bundles). Reading it from there labelled- every routed turn with the route id.- """-- @pytest.mark.asyncio- async def test_anthropic_stream_reports_the_served_model(self):- request = ChatRequest.anthropic(- {"model": _ROUTE_ID, "max_tokens": 16, "messages": [{"role": "user", "content": "hi"}]}- )- response = ChatResponse.openai_stream(ResponseStream(_async_iter([_chunk(content="hi")])))-- events = [ev async for ev in await E.translate(_routed_ctx(), request, response)]-- assert events[0]["message"]["model"] == _SERVED_MODEL-- @pytest.mark.asyncio- async def test_responses_stream_reports_the_served_model(self):- request = ChatRequest.openai_responses({"model": _ROUTE_ID, "input": "hi"})- response = ChatResponse.openai_stream(ResponseStream(_async_iter([_chunk(content="hi")])))-- frames = [frame async for frame in await E.translate(_routed_ctx(), request, response)]- payloads = [- payload for frame in frames for payload in sse_frame_payloads(frame)- ]-- assert payloads[0]["response"]["model"] == _SERVED_MODEL-- @pytest.mark.asyncio- async def test_openai_chat_stream_reports_the_served_model(self):- request = ChatRequest.openai_chat(- {"model": _ROUTE_ID, "messages": [{"role": "user", "content": "hi"}]}- )- response = ChatResponse.anthropic_stream(- AnthropicResponseStream(- _async_iter(- [- {- "type": "message_start",- "message": {- "id": "msg_1",- "model": "upstream/model",- "role": "assistant",- "content": [],- },- },- {- "type": "content_block_delta",- "index": 0,- "delta": {"type": "text_delta", "text": "hi"},- },- ]- )- )- )-- chunks = [chunk async for chunk in await E.translate(_routed_ctx(), request, response)]-- # The OpenAI-chat target yields SDK chunk objects, not dicts.- assert chunks[0].model == _SERVED_MODEL-- @pytest.mark.asyncio- async def test_buffered_response_reports_the_served_model(self):- request = ChatRequest.anthropic(- {"model": _ROUTE_ID, "max_tokens": 16, "messages": [{"role": "user", "content": "hi"}]}- )- response = ChatResponse.openai_completion(_make_completion(model=_SERVED_MODEL))-- result = await E.translate(_routed_ctx(), request, response)-- assert result["model"] == _SERVED_MODEL-- @pytest.mark.asyncio- async def test_stream_falls_back_to_the_upstream_model_without_a_selection(self):- """Passthrough routes keep reporting whatever the provider announced."""- request = ChatRequest.openai_chat(- {"model": _ROUTE_ID, "messages": [{"role": "user", "content": "hi"}]}- )- response = ChatResponse.anthropic_stream(- AnthropicResponseStream(- _async_iter(- [- {- "type": "message_start",- "message": {- "id": "msg_1",- "model": "upstream/model",- "role": "assistant",- "content": [],- },- },- {- "type": "content_block_delta",- "index": 0,- "delta": {"type": "text_delta", "text": "hi"},- },- ]- )- )- )-- chunks = [- chunk async for chunk in await E.translate(_routed_ctx(None), request, response)- ]-- assert chunks[0].model == "upstream/model"---class TestStreamOpenAIToAnthropicUsage:- @pytest.mark.asyncio- async def test_backend_usage_preferred_over_heuristic(self):- """Real usage from the backend replaces the per-delta +1 counter."""- usage = SimpleNamespace(- prompt_tokens=500,- completion_tokens=42,- total_tokens=542,- prompt_tokens_details=None,- )- chunks = [- _chunk(content="Hello"),- _chunk(content=" world", usage=usage),- _chunk(finish_reason="stop"),- ]- events = await _collect_events(chunks)- msg_delta = _find_event(events, "message_delta")-- assert msg_delta["usage"]["input_tokens"] == 500- assert msg_delta["usage"]["output_tokens"] == 42-- @pytest.mark.asyncio- async def test_cached_tokens_propagated(self):- """Anthropic usage splits uncached input from cache-read tokens."""- usage = SimpleNamespace(- prompt_tokens=1000,- completion_tokens=10,- total_tokens=1010,- prompt_tokens_details=SimpleNamespace(cached_tokens=800),- )- chunks = [_chunk(content="Hi", usage=usage)]- events = await _collect_events(chunks)- msg_delta = _find_event(events, "message_delta")-- assert msg_delta["usage"]["cache_read_input_tokens"] == 800- assert msg_delta["usage"]["input_tokens"] == 200-- @pytest.mark.asyncio- async def test_heuristic_fallback_when_no_backend_usage(self):- """Without backend usage, output_tokens falls back to delta counting."""- chunks = [- _chunk(content="a"),- _chunk(content="b"),- _chunk(content="c"),- _chunk(finish_reason="stop"),- ]- events = await _collect_events(chunks)- msg_delta = _find_event(events, "message_delta")-- assert msg_delta["usage"]["output_tokens"] == 3- assert "input_tokens" not in msg_delta["usage"]- assert "cache_read_input_tokens" not in msg_delta["usage"]-- @pytest.mark.asyncio- async def test_reasoning_stream_events_validate_against_anthropic_sdk(self):- from anthropic.types import ( # noqa: PLC0415- RawContentBlockDeltaEvent,- RawContentBlockStartEvent,- )-- events = await _collect_events([- _chunk(reasoning_content="private"),- _chunk(content="visible"),- _chunk(finish_reason="stop"),- ])-- thinking_start = next(- event for event in events- if event["type"] == "content_block_start"- and event["content_block"]["type"] == "thinking"- )- signature_delta = next(- event for event in events- if event["type"] == "content_block_delta"- and event["delta"]["type"] == "signature_delta"- )- assert thinking_start["content_block"]["signature"] == ""- assert signature_delta["delta"]["signature"] == ""- RawContentBlockStartEvent.model_validate(thinking_start)- RawContentBlockDeltaEvent.model_validate(signature_delta)---# ----------------------------------------------------------------------------# Raw SSE frame parsing for cross-format stream translation-# ------------------------------------------------------------------------------class TestSseFramePayloads:- """``sse_frame_payloads`` feeds verbatim-passthrough frame strings back- into the translator; the SSE grammar corners must parse correctly."""-- def test_single_frame_with_event_name(self):- from switchyard_rust.translation import sse_frame_payloads-- payloads = sse_frame_payloads(- 'event: response.created\ndata: {"type":"response.created"}\n\n'- )- assert payloads == [{"type": "response.created"}]-- def test_multi_data_lines_join_with_newline(self):- from switchyard_rust.translation import sse_frame_payloads-- payloads = sse_frame_payloads('data: {"a":\ndata: 1}\n\n')- assert payloads == [{"a": 1}]-- def test_comment_done_and_non_json_frames_yield_nothing(self):- from switchyard_rust.translation import sse_frame_payloads-- assert sse_frame_payloads(": keep-alive\n\n") == []- assert sse_frame_payloads("data: [DONE]\n\n") == []- assert sse_frame_payloads("data: not-json\n\n") == []-- def test_multiple_frames_in_one_string_parse_in_order(self):- from switchyard_rust.translation import sse_frame_payloads-- payloads = sse_frame_payloads('data: {"a":1}\n\ndata: {"b":2}\n\n')- assert payloads == [{"a": 1}, {"b": 2}]---async def test_translate_stream_accepts_raw_sse_frame_strings():- """Cross-format translation parses raw Responses frames into chat chunks."""-- async def _frames() -> AsyncIterator[str]:- yield (- 'event: response.created\n'- 'data: {"type":"response.created","response":{"id":"r1","model":"m"}}\n\n'- )- yield (- 'event: response.output_text.delta\n'- 'data: {"type":"response.output_text.delta","delta":"hello"}\n\n'- )- yield ": keep-alive\n\n"- yield (- 'event: response.completed\n'- 'data: {"type":"response.completed","response":{"id":"r1","usage":'- '{"input_tokens":1,"output_tokens":1}}}\n\n'- )-- chunks = [- chunk- async for chunk in E.translate_stream(- "openai_responses", "openai_chat", _frames(), model="m",- )- ]-- deltas = [- choice.get("delta", {}).get("content")- for chunk in chunks- if isinstance(chunk, dict)- for choice in chunk.get("choices", [])- ]- assert "hello" in deltasdiff --git a/tests/test_responses_openai_translation.py b/tests/test_responses_openai_translation.pydeleted file mode 100644index dff117c0e..000000000--- a/tests/test_responses_openai_translation.py+++ /dev/null@@ -1,1137 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Tests for OpenAI Responses <-> OpenAI Chat translation."""--from unittest.mock import MagicMock--import pytest--from switchyard_rust.translation import TranslationEngine--ENGINE = TranslationEngine()---def _responses_request_to_chat(body: object) -> dict:- return ENGINE.translate_request("openai_responses", "openai_chat", body)---def _chat_response_to_responses(response: object) -> dict:- return ENGINE.translate_response("openai_chat", "openai_responses", response)--# ----------------------------------------------------------------------------# Request conversion tests-# ------------------------------------------------------------------------------class TestConvertResponsesRequestToChatCompletions:- """Tests for convert_responses_request_to_chat_completions."""-- def test_simple_string_input(self):- body = {- "model": "gpt-4",- "input": "Hello, world!",- }- result = _responses_request_to_chat(body)-- assert result["model"] == "gpt-4"- assert len(result["messages"]) == 1- assert result["messages"][0] == {"role": "user", "content": "Hello, world!"}-- def test_instructions_becomes_system_message(self):- body = {- "model": "gpt-4",- "input": "What is 2+2?",- "instructions": "You are a math tutor.",- }- result = _responses_request_to_chat(body)-- assert len(result["messages"]) == 2- assert result["messages"][0] == {"role": "system", "content": "You are a math tutor."}- assert result["messages"][1] == {"role": "user", "content": "What is 2+2?"}-- def test_max_output_tokens_becomes_max_completion_tokens(self):- body = {- "model": "gpt-4",- "input": "Hi",- "max_output_tokens": 1024,- }- result = _responses_request_to_chat(body)-- assert result["max_completion_tokens"] == 1024- assert "max_tokens" not in result- assert "max_output_tokens" not in result-- def test_tools_conversion(self):- body = {- "model": "gpt-4",- "input": "Get weather",- "tools": [- {- "type": "function",- "name": "get_weather",- "description": "Get the weather",- "parameters": {- "type": "object",- "properties": {- "location": {"type": "string"},- },- "required": ["location"],- },- }- ],- }- result = _responses_request_to_chat(body)-- assert len(result["tools"]) == 1- tool = result["tools"][0]- assert tool["type"] == "function"- assert tool["function"]["name"] == "get_weather"- assert tool["function"]["description"] == "Get the weather"- assert tool["function"]["parameters"]["type"] == "object"-- def test_tools_preserve_strict_and_nested_chat_shape(self):- body = {- "model": "gpt-4",- "input": "Get weather",- "tools": [- {- "type": "function",- "function": {- "name": "lookup",- "description": "Lookup data",- "parameters": {"type": "object"},- "strict": True,- },- }- ],- }- result = _responses_request_to_chat(body)-- assert result["tools"] == [- {- "type": "function",- "function": {- "name": "lookup",- "description": "Lookup data",- "parameters": {"type": "object"},- "strict": True,- },- }- ]-- def test_codex_tools_format(self):- """Codex CLI sends tools with id/inputSchema instead of name/parameters."""- body = {- "model": "gpt-4",- "input": "List files",- "tools": [- {- "id": "exec_command",- "description": "Runs a command in a PTY.",- "inputSchema": {- "jsonSchema": {- "type": "object",- "properties": {- "cmd": {"type": "string", "description": "Shell command"},- },- "required": ["cmd"],- "additionalProperties": False,- }- },- },- {- "id": "view_image",- "description": "View a local image.",- "inputSchema": {- "jsonSchema": {- "type": "object",- "properties": {- "path": {"type": "string"},- },- "required": ["path"],- }- },- },- ],- }- result = _responses_request_to_chat(body)-- assert len(result["tools"]) == 2- tool0 = result["tools"][0]- assert tool0["type"] == "function"- assert tool0["function"]["name"] == "exec_command"- assert tool0["function"]["description"] == "Runs a command in a PTY."- assert tool0["function"]["parameters"]["type"] == "object"- assert "cmd" in tool0["function"]["parameters"]["properties"]-- tool1 = result["tools"][1]- assert tool1["function"]["name"] == "view_image"- assert tool1["function"]["parameters"]["required"] == ["path"]-- def test_empty_tools_filtered(self):- """Ghost tools with empty id/name should be filtered out."""- body = {- "model": "gpt-4",- "input": "Hi",- "tools": [- {- "id": "exec_command",- "description": "Run a command",- "inputSchema": {"jsonSchema": {"type": "object", "properties": {}}},- },- {- "id": "",- "description": "",- "inputSchema": {"jsonSchema": {}},- },- ],- }- result = _responses_request_to_chat(body)-- assert len(result["tools"]) == 1- assert result["tools"][0]["function"]["name"] == "exec_command"-- def test_tool_choice_dropped_when_no_tools(self):- # tool_choice without tools is invalid for OpenAI — must not be forwarded- body = {- "model": "gpt-4",- "input": "Hi",- "tool_choice": "required",- }- result = _responses_request_to_chat(body)- assert "tool_choice" not in result-- def test_function_tool_choice_maps_to_chat_shape(self):- body = {- "model": "gpt-4",- "input": "Hi",- "tools": [{"type": "function", "name": "lookup", "parameters": {}}],- "tool_choice": {"type": "function", "name": "lookup"},- }- result = _responses_request_to_chat(body)- assert result["tool_choice"] == {- "type": "function",- "function": {"name": "lookup"},- }-- def test_unsupported_hosted_tools_and_tool_choice_are_dropped(self):- body = {- "model": "gpt-4",- "input": "Search the web",- "tools": [{"type": "web_search_preview"}],- "tool_choice": {"type": "web_search_preview"},- }- result = _responses_request_to_chat(body)- assert "tools" not in result- assert "tool_choice" not in result-- def test_passthrough_params(self):- body = {- "model": "gpt-4",- "input": "Hi",- "temperature": 0.7,- "top_p": 0.9,- "stream": True,- "parallel_tool_calls": False,- "metadata": {"trace": "abc"},- "store": False,- "stream_options": {"include_usage": True},- "prompt_cache_key": "session-1",- "service_tier": "flex",- "user": "u-123",- }- result = _responses_request_to_chat(body)-- assert result["temperature"] == 0.7- assert result["top_p"] == 0.9- assert result["stream"] is True- assert result["parallel_tool_calls"] is False- assert result["metadata"] == {"trace": "abc"}- assert result["store"] is False- assert result["stream_options"] == {"include_usage": True}- assert result["prompt_cache_key"] == "session-1"- assert result["service_tier"] == "flex"- assert result["user"] == "u-123"-- def test_reasoning_and_text_format_map_to_chat_fields(self):- body = {- "model": "gpt-5",- "input": "Return JSON",- "reasoning": {"effort": "high"},- "text": {- "format": {- "type": "json_schema",- "name": "answer",- "schema": {"type": "object"},- "strict": True,- }- },- }- result = _responses_request_to_chat(body)- assert result["reasoning_effort"] == "high"- assert result["response_format"] == {- "type": "json_schema",- "json_schema": {- "name": "answer",- "schema": {"type": "object"},- "strict": True,- },- }-- @pytest.mark.parametrize(- ("field", "value"),- [- ("previous_response_id", "resp_123"),- ("conversation", "conv_123"),- ("conversation", {"id": "conv_123"}),- ],- )- def test_stateful_responses_fields_are_forgotten(self, field, value):- body = {- "model": "gpt-4",- "input": "Continue",- field: value,- }- result = _responses_request_to_chat(body)-- assert result["messages"] == [{"role": "user", "content": "Continue"}]- assert result["model"] == "gpt-4"- assert field not in result-- def test_multi_turn_input_with_messages(self):- body = {- "model": "gpt-4",- "input": [- {"type": "message", "role": "user", "content": "Hello"},- {- "type": "message",- "role": "assistant",- "content": [{"type": "output_text", "text": "Hi there!"}],- },- {"type": "message", "role": "user", "content": "How are you?"},- ],- }- result = _responses_request_to_chat(body)-- assert len(result["messages"]) == 3- assert result["messages"][0] == {"role": "user", "content": "Hello"}- assert result["messages"][1] == {"role": "assistant", "content": "Hi there!"}- assert result["messages"][2] == {"role": "user", "content": "How are you?"}-- def test_multi_turn_with_function_calls_and_outputs(self):- body = {- "model": "gpt-4",- "input": [- {"type": "message", "role": "user", "content": "Get weather in SF and NY"},- {- "type": "function_call",- "name": "get_weather",- "call_id": "call_abc",- "arguments": '{"location": "SF"}',- },- {- "type": "function_call",- "name": "get_weather",- "call_id": "call_def",- "arguments": '{"location": "NY"}',- },- {- "type": "function_call_output",- "call_id": "call_abc",- "output": "72F sunny",- },- {- "type": "function_call_output",- "call_id": "call_def",- "output": "65F cloudy",- },- ],- }- result = _responses_request_to_chat(body)-- msgs = result["messages"]- assert msgs[0] == {"role": "user", "content": "Get weather in SF and NY"}-- # Consecutive function_calls should be merged into one assistant message- assert msgs[1]["role"] == "assistant"- assert msgs[1]["content"] is None- assert len(msgs[1]["tool_calls"]) == 2- assert msgs[1]["tool_calls"][0]["id"] == "call_abc"- assert msgs[1]["tool_calls"][0]["function"]["name"] == "get_weather"- assert msgs[1]["tool_calls"][1]["id"] == "call_def"-- # function_call_output -> tool messages- assert msgs[2]["role"] == "tool"- assert msgs[2]["tool_call_id"] == "call_abc"- assert msgs[2]["content"] == "72F sunny"-- assert msgs[3]["role"] == "tool"- assert msgs[3]["tool_call_id"] == "call_def"- assert msgs[3]["content"] == "65F cloudy"-- def test_interleaved_function_calls_produce_separate_turns(self):- """Sequential function_call/output pairs represent separate turns.-- Codex sends interleaved pairs (call A, output A, call B, output B)- when each call was a separate LLM turn. The transition from- function_call_output -> function_call marks a turn boundary.- Each turn should produce its own assistant message.- """- body = {- "model": "gpt-4",- "input": [- {"type": "message", "role": "user", "content": "Explore the repo"},- {- "type": "function_call",- "name": "exec_command",- "call_id": "call_1",- "arguments": '{"cmd": "ls -la"}',- },- {- "type": "function_call_output",- "call_id": "call_1",- "output": "file1.py file2.py",- },- {- "type": "function_call",- "name": "str_replace_editor",- "call_id": "call_2",- "arguments": '{"filename": "AGENTS.md", "command": "view"}',- },- {- "type": "function_call_output",- "call_id": "call_2",- "output": "# Agents\n...",- },- {- "type": "function_call",- "name": "exec_command",- "call_id": "call_3",- "arguments": '{"cmd": "cat README.md"}',- },- {- "type": "function_call_output",- "call_id": "call_3",- "output": "# README",- },- ],- }- result = _responses_request_to_chat(body)- msgs = result["messages"]-- # user message- assert msgs[0] == {"role": "user", "content": "Explore the repo"}-- # Turn 1: call_1- assert msgs[1]["role"] == "assistant"- assert msgs[1]["content"] is None- assert len(msgs[1]["tool_calls"]) == 1- assert msgs[1]["tool_calls"][0]["id"] == "call_1"- assert msgs[1]["tool_calls"][0]["function"]["name"] == "exec_command"- assert msgs[2] == {"role": "tool", "tool_call_id": "call_1", "content": "file1.py file2.py"}-- # Turn 2: call_2- assert msgs[3]["role"] == "assistant"- assert len(msgs[3]["tool_calls"]) == 1- assert msgs[3]["tool_calls"][0]["id"] == "call_2"- assert msgs[3]["tool_calls"][0]["function"]["name"] == "str_replace_editor"- assert msgs[4] == {"role": "tool", "tool_call_id": "call_2", "content": "# Agents\n..."}-- # Turn 3: call_3- assert msgs[5]["role"] == "assistant"- assert len(msgs[5]["tool_calls"]) == 1- assert msgs[5]["tool_calls"][0]["id"] == "call_3"- assert msgs[5]["tool_calls"][0]["function"]["name"] == "exec_command"- assert msgs[6] == {"role": "tool", "tool_call_id": "call_3", "content": "# README"}-- # Total: 1 user + 3 * (assistant + tool) = 7 messages- assert len(msgs) == 7-- def test_tool_blocks_separated_by_message(self):- """Tool blocks separated by a message should produce separate assistant messages."""- body = {- "model": "gpt-4",- "input": [- {"type": "message", "role": "user", "content": "Do task"},- {- "type": "function_call",- "name": "tool_a",- "call_id": "call_a",- "arguments": "{}",- },- {- "type": "function_call_output",- "call_id": "call_a",- "output": "result_a",- },- # Assistant text response separates the two blocks- {- "type": "message",- "role": "assistant",- "content": [{"type": "output_text", "text": "I found something."}],- },- {- "type": "function_call",- "name": "tool_b",- "call_id": "call_b",- "arguments": "{}",- },- {- "type": "function_call_output",- "call_id": "call_b",- "output": "result_b",- },- ],- }- result = _responses_request_to_chat(body)- msgs = result["messages"]-- # user, assistant(tool_a), tool_a, assistant_text, assistant(tool_b), tool_b- assert msgs[0]["role"] == "user"- assert msgs[1]["role"] == "assistant"- assert len(msgs[1]["tool_calls"]) == 1- assert msgs[1]["tool_calls"][0]["id"] == "call_a"- assert msgs[2]["role"] == "tool"- assert msgs[3] == {"role": "assistant", "content": "I found something."}- assert msgs[4]["role"] == "assistant"- assert len(msgs[4]["tool_calls"]) == 1- assert msgs[4]["tool_calls"][0]["id"] == "call_b"- assert msgs[5]["role"] == "tool"- assert len(msgs) == 6-- def test_message_between_function_call_and_output_is_deferred(self):- """Codex can inject warnings between a function_call and its output.-- Chat/Anthropic-compatible histories need the tool result immediately- after the assistant tool call, so preserve the warning as later context.- """- body = {- "model": "gpt-4",- "input": [- {"type": "message", "role": "user", "content": "Do task"},- {- "type": "function_call",- "name": "shell",- "call_id": "tooluse_1",- "arguments": '{"command":["apply_patch","..."]}',- },- {- "type": "message",- "role": "user",- "content": [- {- "type": "input_text",- "text": (- "Warning: apply_patch was requested via shell. "- "Use the apply_patch tool instead."- ),- }- ],- },- {- "type": "function_call_output",- "call_id": "tooluse_1",- "output": "Success",- },- ],- }- result = _responses_request_to_chat(body)-- assert result["messages"] == [- {"role": "user", "content": "Do task"},- {- "role": "assistant",- "content": None,- "tool_calls": [- {- "id": "tooluse_1",- "type": "function",- "function": {- "name": "shell",- "arguments": '{"command":["apply_patch","..."]}',- },- }- ],- },- {"role": "tool", "tool_call_id": "tooluse_1", "content": "Success"},- {- "role": "user",- "content": (- "Warning: apply_patch was requested via shell. "- "Use the apply_patch tool instead."- ),- },- ]-- def test_content_blocks_flattened(self):- """Content blocks with input_text/output_text types should be flattened."""- body = {- "model": "gpt-4",- "input": [- {- "type": "message",- "role": "user",- "content": [- {"type": "input_text", "text": "Hello "},- {"type": "input_text", "text": "world"},- ],- },- ],- }- result = _responses_request_to_chat(body)- assert result["messages"][0]["content"] == "Hello \nworld"-- def test_multimodal_content_blocks_preserved_for_chat(self):- body = {- "model": "gpt-4o",- "input": [- {- "type": "message",- "role": "user",- "content": [- {"type": "input_text", "text": "Describe this"},- {- "type": "input_image",- "image_url": "https://example.test/cat.png",- "detail": "low",- },- ],- },- ],- }- result = _responses_request_to_chat(body)-- assert result["messages"][0]["content"] == [- {"type": "text", "text": "Describe this"},- {- "type": "image_url",- "image_url": {- "url": "https://example.test/cat.png",- "detail": "low",- },- },- ]-- def test_orphan_function_call_output_becomes_user_context(self):- body = {- "model": "gpt-4",- "input": [- {- "type": "function_call_output",- "call_id": "call_orphan",- "output": "result",- },- ],- }- result = _responses_request_to_chat(body)-- assert result["messages"] == [- {"role": "user", "content": "Tool result call_orphan: result"}- ]-- def test_non_json_tool_payloads_fall_back_to_text(self):- recursive: list[object] = []- recursive.append(recursive)- body = {- "model": "gpt-4",- "input": [- {- "type": "function_call",- "name": "lookup",- "call_id": "call_recursive",- "arguments": recursive,- },- {- "type": "function_call_output",- "call_id": "call_recursive",- "output": recursive,- },- ],- }- result = _responses_request_to_chat(body)-- messages = result["messages"]- assert messages[0]["tool_calls"][0]["function"]["arguments"] == "[[...]]"- assert messages[1]["content"] == "[[...]]"-- def test_codex_reasoning_items_do_not_become_empty_assistant_messages(self):- """Codex replays each model turn as message + reasoning + function_call items.-- The reasoning item must ride with the turn's tool-call message instead of- surfacing as a fabricated ``{"role": "assistant", "content": ""}`` chat- message; those empty turns accumulate every round-trip and degrade the- upstream model until it stops emitting tool calls.- """- body = {- "model": "big-reasoner",- "instructions": "You are a coding agent.",- "input": [- {- "type": "message",- "role": "user",- "content": [{"type": "input_text", "text": "Fix the broken pip install."}],- },- {- "type": "message",- "role": "assistant",- "content": [{"type": "output_text", "text": "\n\n"}],- },- {- "type": "reasoning",- "summary": [],- "content": [- {"type": "reasoning_text", "text": "Let me check the python setup."}- ],- "encrypted_content": None,- },- {- "type": "function_call",- "name": "shell_command",- "arguments": '{"command":"pip3 --version","workdir":"/app"}',- "call_id": "call-1",- },- {- "type": "function_call_output",- "call_id": "call-1",- "output": "ModuleNotFoundError: No module named 'pip'",- },- {- "type": "message",- "role": "assistant",- "content": [{"type": "output_text", "text": "\n\n"}],- },- {- "type": "reasoning",- "summary": [],- "content": [{"type": "reasoning_text", "text": "pip is missing; ensurepip."}],- "encrypted_content": None,- },- {- "type": "function_call",- "name": "shell_command",- "arguments": '{"command":"python3 -m ensurepip","workdir":"/app"}',- "call_id": "call-2",- },- {- "type": "function_call_output",- "call_id": "call-2",- "output": "Successfully installed pip",- },- ],- }- result = _responses_request_to_chat(body)-- assert result["messages"] == [- {"role": "system", "content": "You are a coding agent."},- {"role": "user", "content": "Fix the broken pip install."},- {"role": "assistant", "content": "\n\n"},- {- "role": "assistant",- "content": None,- "tool_calls": [- {- "id": "call-1",- "type": "function",- "function": {- "name": "shell_command",- "arguments": '{"command":"pip3 --version","workdir":"/app"}',- },- }- ],- },- {- "role": "tool",- "tool_call_id": "call-1",- "content": "ModuleNotFoundError: No module named 'pip'",- },- {"role": "assistant", "content": "\n\n"},- {- "role": "assistant",- "content": None,- "tool_calls": [- {- "id": "call-2",- "type": "function",- "function": {- "name": "shell_command",- "arguments": '{"command":"python3 -m ensurepip","workdir":"/app"}',- },- }- ],- },- {- "role": "tool",- "tool_call_id": "call-2",- "content": "Successfully installed pip",- },- ]-- def test_reasoning_item_merges_into_following_assistant_message(self):- """Native item ordering puts reasoning before the assistant message it belongs to."""- body = {- "model": "gpt-5",- "input": [- {"type": "message", "role": "user", "content": "Check the file"},- {- "type": "reasoning",- "summary": [{"type": "summary_text", "text": "Reading it now."}],- },- {- "type": "message",- "role": "assistant",- "content": [{"type": "output_text", "text": "Let me check."}],- },- {- "type": "function_call",- "name": "read_file",- "call_id": "call-r",- "arguments": '{"path":"a.py"}',- },- {"type": "function_call_output", "call_id": "call-r", "output": "print(1)"},- ],- }- result = _responses_request_to_chat(body)-- assert result["messages"] == [- {"role": "user", "content": "Check the file"},- {"role": "assistant", "content": "Let me check."},- {- "role": "assistant",- "content": None,- "tool_calls": [- {- "id": "call-r",- "type": "function",- "function": {"name": "read_file", "arguments": '{"path":"a.py"}'},- }- ],- },- {"role": "tool", "tool_call_id": "call-r", "content": "print(1)"},- ]-- def test_reasoning_only_turn_does_not_add_empty_assistant_message(self):- """A reasoning item directly before a tool call must not split the turn."""- body = {- "model": "gpt-5",- "input": [- {"type": "message", "role": "user", "content": "List files"},- {- "type": "reasoning",- "summary": [],- "content": [{"type": "reasoning_text", "text": "Simple ls."}],- },- {- "type": "function_call",- "name": "shell",- "call_id": "call-ls",- "arguments": '{"command":"ls"}',- },- {"type": "function_call_output", "call_id": "call-ls", "output": "a.py"},- ],- }- result = _responses_request_to_chat(body)-- empty_assistants = [- message- for message in result["messages"]- if message.get("role") == "assistant" and message.get("content") == ""- ]- assert empty_assistants == []- assert result["messages"][1]["tool_calls"][0]["id"] == "call-ls"---# ----------------------------------------------------------------------------# Response conversion tests-# ------------------------------------------------------------------------------class TestConvertChatResponseToResponses:- """Tests for convert_chat_response_to_responses."""-- def test_text_response(self):- response = {- "id": "chatcmpl-123",- "model": "gpt-4",- "choices": [- {- "index": 0,- "message": {"role": "assistant", "content": "Hello!"},- "finish_reason": "stop",- }- ],- "usage": {- "prompt_tokens": 10,- "completion_tokens": 5,- "total_tokens": 15,- },- }- result = _chat_response_to_responses(response)-- assert result["id"] == "chatcmpl-123"- assert result["object"] == "response"- assert result["status"] == "completed"- assert result["model"] == "gpt-4"-- assert len(result["output"]) == 1- out = result["output"][0]- assert out["type"] == "message"- assert out["role"] == "assistant"- assert out["content"][0]["type"] == "output_text"- assert out["content"][0]["text"] == "Hello!"-- def test_tool_calls_response(self):- response = {- "id": "chatcmpl-456",- "model": "gpt-4",- "choices": [- {- "index": 0,- "message": {- "role": "assistant",- "content": None,- "tool_calls": [- {- "id": "call_abc",- "type": "function",- "function": {- "name": "get_weather",- "arguments": '{"location": "SF"}',- },- },- {- "id": "call_def",- "type": "function",- "function": {- "name": "get_weather",- "arguments": '{"location": "NY"}',- },- },- ],- },- "finish_reason": "tool_calls",- }- ],- "usage": {- "prompt_tokens": 20,- "completion_tokens": 30,- "total_tokens": 50,- },- }- result = _chat_response_to_responses(response)-- assert result["status"] == "completed"- # No text message, just function calls- assert len(result["output"]) == 2- assert result["output"][0]["type"] == "function_call"- assert result["output"][0]["call_id"] == "call_abc"- assert result["output"][0]["name"] == "get_weather"- assert result["output"][0]["arguments"] == '{"location": "SF"}'-- assert result["output"][1]["type"] == "function_call"- assert result["output"][1]["call_id"] == "call_def"-- def test_usage_mapping(self):- response = {- "id": "chatcmpl-789",- "model": "gpt-4",- "choices": [- {- "message": {"role": "assistant", "content": "Hi"},- "finish_reason": "stop",- }- ],- "usage": {- "prompt_tokens": 100,- "completion_tokens": 50,- "total_tokens": 150,- "completion_tokens_details": {"reasoning_tokens": 12},- },- }- result = _chat_response_to_responses(response)-- assert result["usage"]["input_tokens"] == 100- assert result["usage"]["output_tokens"] == 50- assert result["usage"]["total_tokens"] == 150- assert result["usage"]["output_tokens_details"] == {"reasoning_tokens": 12}-- def test_response_object_with_model_dump(self):- """Test with a response object that has model_dump (like litellm responses)."""- mock_response = MagicMock()- mock_response.model_dump.return_value = {- "id": "chatcmpl-obj",- "model": "gpt-4",- "choices": [- {- "message": {"role": "assistant", "content": "From object"},- "finish_reason": "stop",- }- ],- "usage": {"prompt_tokens": 5, "completion_tokens": 3, "total_tokens": 8},- }-- result = _chat_response_to_responses(mock_response)- assert result["output"][0]["content"][0]["text"] == "From object"-- def test_mixed_text_and_tool_calls(self):- response = {- "id": "chatcmpl-mixed",- "model": "gpt-4",- "choices": [- {- "message": {- "role": "assistant",- "content": "Let me check that.",- "tool_calls": [- {- "id": "call_123",- "type": "function",- "function": {- "name": "lookup",- "arguments": '{"query": "test"}',- },- }- ],- },- "finish_reason": "tool_calls",- }- ],- "usage": {"prompt_tokens": 10, "completion_tokens": 20, "total_tokens": 30},- }- result = _chat_response_to_responses(response)-- assert len(result["output"]) == 2- assert result["output"][0]["type"] == "message"- assert result["output"][0]["content"][0]["text"] == "Let me check that."- assert result["output"][1]["type"] == "function_call"- assert result["output"][1]["name"] == "lookup"-- def test_reasoning_content_with_tool_calls_has_no_empty_message_item(self):- """Reasoning + tool calls with empty text must not fabricate an empty message item."""- response = {- "id": "chatcmpl-reason",- "model": "big-reasoner",- "choices": [- {- "message": {- "role": "assistant",- "content": None,- "reasoning_content": "Inspect the repository first.",- "tool_calls": [- {- "id": "call-9",- "type": "function",- "function": {- "name": "shell",- "arguments": '{"command":"ls"}',- },- }- ],- },- "finish_reason": "tool_calls",- }- ],- "usage": {"prompt_tokens": 10, "completion_tokens": 20, "total_tokens": 30},- }- result = _chat_response_to_responses(response)-- assert [item["type"] for item in result["output"]] == ["reasoning", "function_call"]- reasoning = result["output"][0]- assert reasoning["content"] == [- {"type": "reasoning_text", "text": "Inspect the repository first."}- ]- function_call = result["output"][1]- assert function_call["call_id"] == "call-9"- assert function_call["name"] == "shell"- assert function_call["arguments"] == '{"command": "ls"}'---# ----------------------------------------------------------------------------# Streaming conversion tests (chat chunks -> Responses SSE events)-# ------------------------------------------------------------------------------def _chat_chunk(delta: dict, finish: str | None = None) -> dict:- return {- "id": "chatcmpl-stream",- "object": "chat.completion.chunk",- "model": "big-reasoner",- "choices": [{"index": 0, "delta": delta, "finish_reason": finish}],- }---async def _translate_chat_stream_to_responses(chunks: list[dict]) -> list[dict]:- async def _source():- for chunk in chunks:- yield chunk-- return [- event- async for event in ENGINE.translate_stream("openai_chat", "openai_responses", _source())- ]---class TestChatStreamToResponsesSse:- """Pins the SSE contract codex consumes for streamed tool calls."""-- async def test_tool_call_stream_produces_well_formed_function_call_events(self):- events = await _translate_chat_stream_to_responses(- [- _chat_chunk({"role": "assistant", "content": ""}),- _chat_chunk({"reasoning_content": "Need to inspect the repo."}),- _chat_chunk({"content": "\n\n"}),- _chat_chunk(- {- "tool_calls": [- {- "index": 0,- "id": "call-9",- "type": "function",- "function": {"name": "shell", "arguments": '{"command":'},- }- ]- }- ),- _chat_chunk(- {"tool_calls": [{"index": 0, "function": {"arguments": '"ls"}'}}]}- ),- _chat_chunk({}, finish="tool_calls"),- ]- )-- added = [- event["item"]- for event in events- if event["type"] == "response.output_item.added"- and event["item"]["type"] == "function_call"- ]- assert len(added) == 1- assert added[0]["call_id"] == "call-9"- assert added[0]["name"] == "shell"-- argument_deltas = [- event["delta"]- for event in events- if event["type"] == "response.function_call_arguments.delta"- ]- assert "".join(argument_deltas) == '{"command":"ls"}'-- done = [- event["item"]- for event in events- if event["type"] == "response.output_item.done"- and event["item"]["type"] == "function_call"- ]- assert len(done) == 1- assert done[0]["call_id"] == "call-9"- assert done[0]["name"] == "shell"- assert done[0]["arguments"] == '{"command":"ls"}'- assert done[0]["status"] == "completed"-- completed = [event for event in events if event["type"] == "response.completed"]- assert len(completed) == 1- output_types = [item["type"] for item in completed[0]["response"]["output"]]- assert output_types == ["reasoning", "message", "function_call"]- function_item = completed[0]["response"]["output"][2]- assert function_item["call_id"] == "call-9"- assert function_item["arguments"] == '{"command":"ls"}'diff --git a/tests/test_rl_logging.py b/tests/test_rl_logging.pydeleted file mode 100644index dffe157e2..000000000--- a/tests/test_rl_logging.py+++ /dev/null@@ -1,336 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Tests for local RL trace logging (`--enable-rl-logging`)."""--from __future__ import annotations--import argparse-import json-from collections.abc import AsyncIterator-from pathlib import Path--import pytest-from openai.types.chat import ChatCompletionChunk-from openai.types.chat.chat_completion_chunk import Choice as ChunkChoice-from openai.types.chat.chat_completion_chunk import ChoiceDelta-from openai.types.completion_usage import CompletionUsage--from switchyard.cli.switchyard_cli import _build_parser-from switchyard.lib.chat_response import ResponseStream-from switchyard.lib.processors.rl_logging_request_processor import (- CTX_RL_LOGGING_REQUEST,- RlLoggingRequestProcessor,-)-from switchyard.lib.processors.rl_logging_response_processor import (- RlLoggingResponseProcessor,- build_rl_logging_processors,-)-from switchyard.server.server_util import resolve_rl_log_dir-from switchyard_rust.core import ChatRequest, ChatResponse, ProxyContext---def _request() -> ChatRequest:- return ChatRequest.openai_chat({- "model": "gpt-test",- "messages": [- {"role": "system", "content": "be brief"},- {"role": "user", "content": "hello"},- ],- "tools": [- {- "type": "function",- "function": {- "name": "get_weather",- "description": "Look up weather",- "parameters": {"type": "object", "properties": {}},- },- },- ],- "tool_choice": "auto",- })---def _completion(*, content: str | None = "hi there", tool_calls: list | None = None,- choices: list | None = None) -> ChatResponse:- message: dict = {"role": "assistant"}- if content is not None:- message["content"] = content- if tool_calls is not None:- message["tool_calls"] = tool_calls- if choices is None:- choices = [{"index": 0, "message": message, "finish_reason": "stop"}]- return ChatResponse.openai_completion({- "id": "chatcmpl-test",- "object": "chat.completion",- "created": 1700000000,- "model": "gpt-test",- "choices": choices,- "usage": {"prompt_tokens": 5, "completion_tokens": 3, "total_tokens": 8},- })---def _read_only_trace(log_dir: Path) -> dict:- files = list(log_dir.glob("*.json"))- assert len(files) == 1, f"expected exactly one trace file, got {files}"- return json.loads(files[0].read_text())---async def _run(log_dir: Path, response: ChatResponse, *, snapshot: bool = True) -> ProxyContext:- ctx = ProxyContext()- if snapshot:- await RlLoggingRequestProcessor().process(ctx, _request())- await RlLoggingResponseProcessor(log_dir).process(ctx, response)- return ctx---async def test_request_processor_snapshots_openai_body() -> None:- ctx = ProxyContext()- await RlLoggingRequestProcessor().process(ctx, _request())- snapshot = ctx.metadata[CTX_RL_LOGGING_REQUEST]- assert isinstance(snapshot, dict)- assert [m["role"] for m in snapshot["messages"]] == ["system", "user"]---async def test_non_streaming_writes_message_history_trace(tmp_path: Path) -> None:- await _run(tmp_path, _completion(content="hi there"))- entry = _read_only_trace(tmp_path)-- assert entry["is_valid"] is True- assert entry["tool_choice"] == "auto"- assert entry["token_count"] == {- "prompt_tokens": 5, "completion_tokens": 3, "total_tokens": 8,- }- # Request history + appended assistant turn.- assert [m["role"] for m in entry["messages"]] == ["system", "user", "assistant"]- assert entry["messages"][-1]["content"] == "hi there"- # Tools rewritten to the message_history shape.- assert entry["tools"] == [{- "id": "get_weather",- "description": "Look up weather",- "inputSchema": {"jsonSchema": {"type": "object", "properties": {}}},- }]---async def test_assistant_tool_calls_are_logged(tmp_path: Path) -> None:- tool_calls = [{- "id": "call_1",- "type": "function",- "function": {"name": "get_weather", "arguments": "{}"},- }]- await _run(tmp_path, _completion(content=None, tool_calls=tool_calls))- entry = _read_only_trace(tmp_path)- assistant = entry["messages"][-1]- assert assistant["role"] == "assistant"- assert assistant["tool_calls"] == tool_calls- assert "content" not in assistant---async def test_empty_string_assistant_content_is_preserved(tmp_path: Path) -> None:- """An empty-string completion is valid content and must not be dropped."""- await _run(tmp_path, _completion(content=""))- entry = _read_only_trace(tmp_path)- assert entry["messages"][-1] == {"role": "assistant", "content": ""}---async def test_empty_choices_writes_nothing(tmp_path: Path) -> None:- await _run(tmp_path, _completion(choices=[]))- assert list(tmp_path.glob("*.json")) == []---async def test_missing_request_snapshot_writes_nothing(tmp_path: Path) -> None:- await _run(tmp_path, _completion(), snapshot=False)- assert list(tmp_path.glob("*.json")) == []---def _anthropic_completion(*, content: str, input_tokens: int, output_tokens: int) -> ChatResponse:- return ChatResponse.anthropic_completion({- "id": "msg_test",- "type": "message",- "role": "assistant",- "content": [{"type": "text", "text": content}],- "model": "claude-test",- "stop_reason": "end_turn",- "stop_sequence": None,- "usage": {"input_tokens": input_tokens, "output_tokens": output_tokens},- })---def _responses_completion(*, content: str, input_tokens: int, output_tokens: int) -> ChatResponse:- return ChatResponse.openai_responses_completion({- "id": "resp_test",- "object": "response",- "created_at": 1700000000,- "status": "completed",- "model": "codex-test",- "output": [{- "type": "message",- "role": "assistant",- "status": "completed",- "content": [{"type": "output_text", "text": content}],- }],- "parallel_tool_calls": False,- "tool_choice": "auto",- "tools": [],- "usage": {- "input_tokens": input_tokens,- "output_tokens": output_tokens,- "total_tokens": input_tokens + output_tokens,- },- })---async def test_anthropic_response_is_translated_and_logged(tmp_path: Path) -> None:- """claude's backend can answer in Anthropic format; it must translate to message_history."""- await _run(tmp_path, _anthropic_completion(content="hello world", input_tokens=7, output_tokens=3))- entry = _read_only_trace(tmp_path)- assert [m["role"] for m in entry["messages"]] == ["system", "user", "assistant"]- assert entry["messages"][-1]["content"] == "hello world"- assert entry["token_count"]["prompt_tokens"] == 7- assert entry["token_count"]["completion_tokens"] == 3- assert entry["is_valid"] is True---async def test_openai_responses_response_is_translated_and_logged(tmp_path: Path) -> None:- """codex talks the Responses API; a Responses completion must translate too."""- await _run(tmp_path, _responses_completion(content="hello world", input_tokens=6, output_tokens=2))- entry = _read_only_trace(tmp_path)- assert entry["messages"][-1] == {"role": "assistant", "content": "hello world"}- assert entry["token_count"]["prompt_tokens"] == 6- assert entry["token_count"]["completion_tokens"] == 2- assert entry["is_valid"] is True---async def test_writes_one_file_per_turn(tmp_path: Path) -> None:- """A reused processor writes one independent file per completed turn (no overwrite)."""- request_processor = RlLoggingRequestProcessor()- response_processor = RlLoggingResponseProcessor(tmp_path)- for i in range(3):- ctx = ProxyContext()- await request_processor.process(ctx, _request())- await response_processor.process(ctx, _completion(content=f"turn {i}"))-- files = list(tmp_path.glob("*.json"))- assert len(files) == 3 # distinct filename per turn, nothing overwritten- last_contents = sorted(- json.loads(f.read_text())["messages"][-1]["content"] for f in files- )- assert last_contents == ["turn 0", "turn 1", "turn 2"]---async def test_write_failure_does_not_break_the_response(- tmp_path: Path, monkeypatch: pytest.MonkeyPatch,-) -> None:- """A trace-write failure is swallowed — the proxy response must flow untouched."""- processor = RlLoggingResponseProcessor(tmp_path)- ctx = ProxyContext()- await RlLoggingRequestProcessor().process(ctx, _request())-- def _boom(_entry: dict) -> None:- raise OSError("disk full")-- monkeypatch.setattr(processor, "_write_entry", _boom)- response = _completion()- assert await processor.process(ctx, response) is response- assert list(tmp_path.glob("*.json")) == []---def test_resolve_rl_log_dir() -> None:- off = argparse.Namespace(enable_rl_logging=False, rl_log_dir="./rl_data")- assert resolve_rl_log_dir(off) is None-- on = argparse.Namespace(enable_rl_logging=True, rl_log_dir="/tmp/traces")- assert resolve_rl_log_dir(on) == Path("/tmp/traces")---def test_build_rl_logging_processors() -> None:- assert build_rl_logging_processors(None) == ([], [])-- req, resp = build_rl_logging_processors(Path("/tmp/x"))- assert [type(p).__name__ for p in req] == ["RlLoggingRequestProcessor"]- assert [type(p).__name__ for p in resp] == ["RlLoggingResponseProcessor"]---async def test_streaming_logs_after_drain(tmp_path: Path) -> None:- """Streaming turns are captured on stream completion, not before."""- ctx = ProxyContext()- await RlLoggingRequestProcessor().process(ctx, _request())-- content_chunk = ChatCompletionChunk(- id="chatcmpl-test", object="chat.completion.chunk", created=1700000000,- model="gpt-test",- choices=[ChunkChoice(index=0, delta=ChoiceDelta(content="hi there"), finish_reason="stop")],- )- usage_chunk = ChatCompletionChunk(- id="chatcmpl-test", object="chat.completion.chunk", created=1700000000,- model="gpt-test", choices=[],- usage=CompletionUsage(prompt_tokens=5, completion_tokens=3, total_tokens=8),- )-- async def _iter() -> AsyncIterator[ChatCompletionChunk]:- yield content_chunk- yield usage_chunk-- response = ChatResponse.openai_stream(ResponseStream(_iter()))- out = await RlLoggingResponseProcessor(tmp_path).process(ctx, response)- # Nothing is written until the stream actually drains.- assert list(tmp_path.glob("*.json")) == []-- forwarded = [chunk async for chunk in out.stream]- assert len(forwarded) == 2 # stream still forwards every chunk to the client-- entry = _read_only_trace(tmp_path)- assert entry["messages"][-1] == {"role": "assistant", "content": "hi there"}- assert entry["token_count"] == {- "prompt_tokens": 5, "completion_tokens": 3, "total_tokens": 8,- }---def test_rl_flags_are_scoped_to_serve() -> None:- parser = _build_parser()- args = parser.parse_args(- [- "serve",- "--routes",- "routes.yaml",- "--enable-rl-logging",- "--rl-log-dir",- "/tmp/x",- ],- )- assert args.command == "serve"- assert args.enable_rl_logging is True- assert resolve_rl_log_dir(args) == Path("/tmp/x")---def test_serve_attaches_rl_logging_processors(monkeypatch, tmp_path: Path) -> None:- """`serve --routes --enable-rl-logging` wires the trace logger into the chain."""- import switchyard.cli.switchyard_cli as cli-- captured: dict[str, list] = {}-- class _FakeTable:- def registered_models(self) -> list[str]:- return ["m"]-- def default_model(self) -> str | None:- return None-- def _fake_load(routes, *, pre_routing_request_processors=(),- extra_response_processors=(), **_kwargs):- captured["request"] = list(pre_routing_request_processors)- captured["response"] = list(extra_response_processors)- return _FakeTable()-- monkeypatch.setattr(cli, "load_route_bundle_table", _fake_load)- monkeypatch.setattr(cli, "build_and_serve", lambda *a, **k: None)-- args = argparse.Namespace(- routes="routes.yaml",- enable_rl_logging=True, rl_log_dir=str(tmp_path),- routing_log_file=None,- )- cli._cmd_serve(args)-- assert [type(p).__name__ for p in captured["request"]] == ["RlLoggingRequestProcessor"]- assert [type(p).__name__ for p in captured["response"]] == ["RlLoggingResponseProcessor"]diff --git a/tests/test_rl_logging_e2e.py b/tests/test_rl_logging_e2e.pydeleted file mode 100644index 4a44b988b..000000000--- a/tests/test_rl_logging_e2e.py+++ /dev/null@@ -1,121 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""In-process end-to-end test for ``--enable-rl-logging`` on the serve chain.--Drives a real HTTP request through the production serve path — route-bundle-table → ``build_switchyard_app`` → endpoint → chain → backend — with the-RL-logging processors wired in exactly as ``switchyard serve---enable-rl-logging`` wires them, and asserts a ``message_history`` trace file-is written. The upstream is a real loopback HTTP server (``_OpenAICompatStub``)-because the Rust backend uses ``reqwest``, which ``respx`` cannot intercept.--No API key, no ``claude`` binary, no outbound network — safe for CI.-"""--from __future__ import annotations--import json-from pathlib import Path--import httpx--from switchyard.cli.route_bundle import build_route_bundle_table-from switchyard.lib.processors.rl_logging_response_processor import build_rl_logging_processors-from switchyard.server.switchyard_app import build_switchyard_app-from tests._chain_test_helpers import _backend_payload, _OpenAICompatStub, _sse_body, _stream_chunk---def _build_app(stub: _OpenAICompatStub, log_dir: Path):- """Build the serve chain pointed at ``stub`` with RL-logging attached.-- Mirrors ``_cmd_serve``: ``build_rl_logging_processors`` produces the paired- snapshot + writer processors, which feed the route-bundle table builder.- """- rl_request, rl_response = build_rl_logging_processors(log_dir)- table = build_route_bundle_table(- {- "defaults": {- "api_key": "dummy",- "base_url": stub.base_url,- "format": "openai",- },- "routes": {- "mock-model": {"type": "passthrough", "target": "mock-model"}- },- },- pre_routing_request_processors=rl_request,- extra_response_processors=rl_response,- )- return build_switchyard_app(table)---def _client(app) -> httpx.AsyncClient:- return httpx.AsyncClient(- transport=httpx.ASGITransport(app=app, raise_app_exceptions=False),- base_url="http://test",- )---def _only_trace(log_dir: Path) -> dict:- files = list(log_dir.glob("*.json"))- assert len(files) == 1, f"expected one trace file, got {files}"- return json.loads(files[0].read_text())---async def test_serve_chain_writes_trace_non_streaming(tmp_path: Path) -> None:- with _OpenAICompatStub() as stub:- stub.respond_json(_backend_payload(content="hello world", model="mock-model"))- app = _build_app(stub, tmp_path)- async with _client(app) as client:- resp = await client.post(- "/v1/chat/completions",- json={"model": "mock-model", "messages": [{"role": "user", "content": "hi"}]},- headers={"authorization": "Bearer test"},- )- assert resp.status_code == 200- assert stub.called-- entry = _only_trace(tmp_path)- assert entry["is_valid"] is True- assert [m["role"] for m in entry["messages"]] == ["user", "assistant"]- assert entry["messages"][-1]["content"] == "hello world"- assert entry["token_count"] == {- "prompt_tokens": 8, "completion_tokens": 4, "total_tokens": 12,- }---async def test_serve_chain_writes_trace_streaming(tmp_path: Path) -> None:- usage_chunk = {- "id": "chatcmpl-backend-stream",- "object": "chat.completion.chunk",- "created": 1700000002,- "model": "mock-model",- "choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}],- "usage": {"prompt_tokens": 8, "completion_tokens": 4, "total_tokens": 12},- }- with _OpenAICompatStub() as stub:- stub.respond_sse(_sse_body([_stream_chunk(content="hello world"), usage_chunk]))- app = _build_app(stub, tmp_path)- async with _client(app) as client:- # The trace is written when the stream drains, so the body must be- # fully consumed before asserting.- async with client.stream(- "POST",- "/v1/chat/completions",- json={- "model": "mock-model",- "stream": True,- "messages": [{"role": "user", "content": "hi"}],- },- headers={"authorization": "Bearer test"},- ) as resp:- assert resp.status_code == 200- async for _ in resp.aiter_bytes():- pass-- entry = _only_trace(tmp_path)- assert entry["is_valid"] is True- assert [m["role"] for m in entry["messages"]] == ["user", "assistant"]- assert entry["messages"][-1]["content"] == "hello world"- assert entry["token_count"]["total_tokens"] == 12diff --git a/tests/test_route_bundle.py b/tests/test_route_bundle.pydeleted file mode 100644index d8773c273..000000000--- a/tests/test_route_bundle.py+++ /dev/null@@ -1,176 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Tests for the minimal Python server route bundle."""--from __future__ import annotations--import sys-from pathlib import Path--import httpx-import pytest--import switchyard.cli.switchyard_cli as cli-from switchyard.cli.launchers.launcher_runtime import route_bundle_strategy_summary-from switchyard.cli.route_bundle import RouteBundleConfigError, build_route_bundle_table-from switchyard.lib.route_table import RouteTable-from switchyard.server.switchyard_app import build_switchyard_app-from switchyard_rust.components import StatsLlmBackend---async def test_noop_route_returns_ok_without_an_upstream() -> None:- table = build_route_bundle_table({- "routes": {"test/noop": {"type": "noop"}},- })-- async with httpx.AsyncClient(- transport=httpx.ASGITransport(app=build_switchyard_app(table)),- base_url="http://test",- ) as client:- response = await client.post(- "/v1/chat/completions",- json={- "model": "test/noop",- "messages": [{"role": "user", "content": "hello"}],- },- )-- assert response.status_code == 200- assert response.json()["choices"][0]["message"]["content"] == "OK"--- ("path", "body", "expected"),- [- (- "/v1/messages",- {- "model": "test/noop",- "max_tokens": 16,- "messages": [{"role": "user", "content": "hello"}],- },- ("content", 0, "text"),- ),- (- "/v1/responses",- {"model": "test/noop", "input": "hello"},- ("output", 0, "content"),- ),- ],-)-async def test_noop_route_translates_to_inbound_format(- path: str,- body: dict[str, object],- expected: tuple[str, int, str],-) -> None:- table = build_route_bundle_table({"routes": {"test/noop": {"type": "noop"}}})- async with httpx.AsyncClient(- transport=httpx.ASGITransport(app=build_switchyard_app(table)),- base_url="http://test",- ) as client:- response = await client.post(path, json=body)-- assert response.status_code == 200- value: object = response.json()- for part in expected:- value = value[part] # type: ignore[index]- assert value---def test_passthrough_route_builds_one_native_backend(monkeypatch: pytest.MonkeyPatch) -> None:- monkeypatch.setenv("UPSTREAM_KEY", "secret")- table = build_route_bundle_table({- "defaults": {- "api_key": "${UPSTREAM_KEY}",- "base_url": "https://example.invalid/v1",- "format": "openai",- },- "routes": {- "direct": {- "type": "passthrough",- "target": {"model": "upstream/model"},- "display_name": "Direct model",- }- },- })-- assert table.registered_models() == ["direct"]- assert table.default_model() == "direct"- assert table.registered_model_entries()[0]["display_name"] == "Direct model"- components = table.lookup_switchyard("direct").iter_components()- stats_backend = next(component for component in components if isinstance(component, StatsLlmBackend))- assert isinstance(stats_backend, StatsLlmBackend)---def test_passthrough_summary_labels_the_model(tmp_path: Path) -> None:- path = tmp_path / "routes.yaml"- path.write_text("routes:\n direct:\n type: passthrough\n target: upstream/model\n")-- assert route_bundle_strategy_summary(str(path), "direct") == (- "passthrough: model=upstream/model"- )--- "bundle, match",- [- ({}, "routes must be a mapping"),- ({"routes": {}}, "at least one route"),- ({"routes": {"r": {}}}, "missing string 'type'"),- (- {"routes": {"r": {"type": "random"}}},- "expected 'noop' or 'passthrough'",- ),- (- {"routes": {"r": {"type": "noop", "target": "unused"}}},- "unknown key",- ),- (- {"routes": {"r": {"type": "passthrough"}}},- "target must be a mapping",- ),- ],-)-def test_invalid_bundles_fail_closed(bundle: object, match: str) -> None:- with pytest.raises(RouteBundleConfigError, match=match):- build_route_bundle_table(bundle)---def test_missing_environment_variable_is_rejected() -> None:- with pytest.raises(RouteBundleConfigError, match="MISSING_ROUTE_KEY"):- build_route_bundle_table({- "defaults": {"api_key": "${MISSING_ROUTE_KEY}"},- "routes": {"direct": {"type": "passthrough", "target": "model"}},- })---def test_main_reports_missing_bundle_without_traceback(- monkeypatch: pytest.MonkeyPatch, tmp_path: Path-) -> None:- missing = tmp_path / "missing.yaml"- monkeypatch.setattr(sys, "argv", ["switchyard", "serve", "--routes", str(missing)])-- with pytest.raises(SystemExit) as error:- cli.main()-- assert error.value.code == f"error: invalid route bundle: {missing}: file not found"---def test_serve_loads_noop_bundle(- monkeypatch: pytest.MonkeyPatch, tmp_path: Path-) -> None:- path = tmp_path / "routes.yaml"- path.write_text("routes:\n test/noop:\n type: noop\n")- captured: dict[str, object] = {}-- def fake_serve(args: object, switchyard: object, **kwargs: object) -> None:- captured.update(args=args, switchyard=switchyard, **kwargs)-- monkeypatch.setattr(cli, "build_and_serve", fake_serve)- parser = cli._build_parser()- args = parser.parse_args(["serve", "--routes", str(path)])- args.func(args)-- assert isinstance(captured["switchyard"], RouteTable)- assert captured["switchyard"].registered_models() == ["test/noop"]diff --git a/tests/test_route_selection_headers.py b/tests/test_route_selection_headers.pydeleted file mode 100644index 4b1dabacd..000000000--- a/tests/test_route_selection_headers.py+++ /dev/null@@ -1,233 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Route-selection spend-attribution response headers.--For tokenomics reporting, a front proxy (e.g. LiteLLM) must be able to tie-its provider spend-log rows back to the Switchyard logical route that-selected them: the Switchyard response returns ``x-switchyard-*`` headers with-the successful attempt's selection plus a per-request correlation id, so the-parent spend-log row can be enriched to match the provider row.-"""--from __future__ import annotations--import json-from typing import Any--from fastapi.responses import JSONResponse--from switchyard.lib.endpoints.dispatch import serialize_chain_result-from switchyard.lib.endpoints.route_selection import (- ROUTER_CORRELATION_ID_HEADER,- ROUTER_MODEL_HEADER,- SELECTED_MODEL_HEADER,- SELECTED_PROVIDER_HEADER,- route_selection_headers,-)-from switchyard.lib.endpoints.upstream_error import handle_chain_exception-from switchyard.lib.proxy_context import CTX_ROUTE_SELECTION, ProxyContext--ROUTE_MODEL = "nvidia/switchyard/test-route"-ENDPOINT_ID = "openai/test-model"-UPSTREAM_MODEL = "openai/openai/test-model"---def _selection(**overrides: object) -> dict[str, object]:- """Recorded route selection, with *overrides* applied."""- payload: dict[str, object] = {- "router_model": ROUTE_MODEL,- "router_selected_endpoint": ENDPOINT_ID,- "router_selected_model": UPSTREAM_MODEL,- "router_selected_provider": "openai",- "router_correlation_id": "11111111-2222-3333-4444-555555555555",- }- payload.update(overrides)- return payload---# ----------------------------------------------------------------------------# Unit: ctx → response-header mapping-# ------------------------------------------------------------------------------class TestRouteSelectionHeaders:- """Mapping of the ctx selection record to ``x-switchyard-*`` headers."""-- def test_empty_without_selection(self) -> None:- """A ctx without a recorded selection yields no headers."""- assert route_selection_headers(ProxyContext()) == {}-- def test_maps_selection_to_response_headers(self) -> None:- """Every exposed selection field maps to its response header."""- ctx = ProxyContext()- ctx.metadata[CTX_ROUTE_SELECTION] = _selection()-- assert route_selection_headers(ctx) == {- ROUTER_MODEL_HEADER: ROUTE_MODEL,- SELECTED_MODEL_HEADER: UPSTREAM_MODEL,- SELECTED_PROVIDER_HEADER: "openai",- ROUTER_CORRELATION_ID_HEADER: "11111111-2222-3333-4444-555555555555",- }-- def test_skips_absent_fields_instead_of_stamping_placeholders(self) -> None:- """An absent field is skipped — never stamped as a placeholder value."""- ctx = ProxyContext()- ctx.metadata[CTX_ROUTE_SELECTION] = _selection(router_model=None)-- headers = route_selection_headers(ctx)-- assert ROUTER_MODEL_HEADER not in headers- assert headers[SELECTED_MODEL_HEADER] == UPSTREAM_MODEL-- def test_ignores_non_mapping_value(self) -> None:- """A malformed (non-mapping) ctx record yields no headers."""- ctx = ProxyContext()- ctx.metadata[CTX_ROUTE_SELECTION] = "bogus"-- assert route_selection_headers(ctx) == {}-- def test_skips_values_unsafe_as_header_material(self) -> None:- """The client-controlled router_model must be re-validated as a header.-- A CRLF-bearing value would be a response-splitting vector, and a- non-latin-1 value would crash Starlette response construction after- the upstream call already succeeded and was billed.- """- ctx = ProxyContext()- ctx.metadata[CTX_ROUTE_SELECTION] = _selection(router_model="gpt\r\nx-evil: 1")- headers = route_selection_headers(ctx)- assert ROUTER_MODEL_HEADER not in headers- assert headers[SELECTED_MODEL_HEADER] == UPSTREAM_MODEL-- ctx = ProxyContext()- ctx.metadata[CTX_ROUTE_SELECTION] = _selection(router_model="gpt-4中文")- headers = route_selection_headers(ctx)- assert ROUTER_MODEL_HEADER not in headers- assert headers[ROUTER_CORRELATION_ID_HEADER] == (- "11111111-2222-3333-4444-555555555555"- )---class TestSerializeChainResultHeaders:- """``serialize_chain_result`` stamps the selection on every branch."""-- @staticmethod- async def _sse_iter(_result: Any) -> Any:- """Minimal SSE iterator satisfying the serializer signature."""- yield "data: {}\n\n"-- def test_json_response_carries_selection_headers(self) -> None:- """A JSON-serialized result carries the selection headers."""- ctx = ProxyContext()- ctx.metadata[CTX_ROUTE_SELECTION] = _selection()-- response = serialize_chain_result(- {"ok": True}, stream=False, sse_iter=self._sse_iter, ctx=ctx- )-- assert response.headers[SELECTED_MODEL_HEADER] == UPSTREAM_MODEL- assert (- response.headers[ROUTER_CORRELATION_ID_HEADER]- == "11111111-2222-3333-4444-555555555555"- )-- def test_streaming_response_carries_selection_headers(self) -> None:- """The SSE branch stamps the headers too.-- The backend call completes before the StreamingResponse is built, so- the selection is final by the time SSE headers are committed.- """- class _EmptyStream:- """Async iterator yielding nothing, standing in for a backend stream."""-- def __aiter__(self) -> _EmptyStream:- return self-- async def __anext__(self) -> str:- raise StopAsyncIteration-- ctx = ProxyContext()- ctx.metadata[CTX_ROUTE_SELECTION] = _selection()-- response = serialize_chain_result(- _EmptyStream(), stream=True, sse_iter=self._sse_iter, ctx=ctx- )-- assert response.media_type == "text/event-stream"- assert response.headers[ROUTER_MODEL_HEADER] == ROUTE_MODEL-- def test_selection_free_ctx_no_selection_headers(self) -> None:- """A selection-free ctx adds no ``x-switchyard-*`` headers."""- response = serialize_chain_result(- {"ok": True}, stream=False, sse_iter=self._sse_iter, ctx=ProxyContext()- )-- assert ROUTER_CORRELATION_ID_HEADER not in response.headers-- def test_prebuilt_response_passes_through_with_selection_merged(self) -> None:- """A result that is already a ``Response`` gains the selection headers.-- No current chain path yields a pre-built Response after a billed- upstream success, but the serializer contract — a recorded selection- is never dropped — must hold on that branch too, with the response's- own status, body, and headers preserved.- """- ctx = ProxyContext()- ctx.metadata[CTX_ROUTE_SELECTION] = _selection()- prebuilt = JSONResponse(- content={"ok": True}, status_code=404, headers={"x-existing": "1"}- )-- response = serialize_chain_result(- prebuilt, stream=False, sse_iter=self._sse_iter, ctx=ctx- )-- assert response is prebuilt- assert response.status_code == 404- assert response.headers["x-existing"] == "1"- assert json.loads(response.body) == {"ok": True}- assert response.headers[SELECTED_MODEL_HEADER] == UPSTREAM_MODEL- assert response.headers[ROUTER_CORRELATION_ID_HEADER] == (- "11111111-2222-3333-4444-555555555555"- )---class TestErrorPathSelectionHeaders:- """``handle_chain_exception`` keeps a billed selection on error responses."""-- def test_failure_after_billed_success_keeps_selection_headers(self) -> None:- """A post-backend failure must still expose the billed selection.-- The upstream call succeeded (provider spend-log row written with the- stamped correlation id) before e.g. response translation raised; the- error response must carry the selection headers or that provider row- becomes unjoinable.- """- ctx = ProxyContext()- ctx.metadata[CTX_ROUTE_SELECTION] = _selection()-- response = handle_chain_exception(- RuntimeError("response translation failed"),- ctx,- inbound="openai",- log_msg="test",- )-- assert response.status_code == 500- assert response.headers[ROUTER_CORRELATION_ID_HEADER] == (- "11111111-2222-3333-4444-555555555555"- )- assert response.headers[SELECTED_MODEL_HEADER] == UPSTREAM_MODEL-- def test_failure_without_selection_has_no_selection_headers(self) -> None:- """No billed success → the error response claims no selection."""- response = handle_chain_exception(- RuntimeError("backend never succeeded"),- ProxyContext(),- inbound="openai",- log_msg="test",- )-- assert response.status_code == 500- assert ROUTER_CORRELATION_ID_HEADER not in response.headersdiff --git a/tests/test_route_table.py b/tests/test_route_table.pydeleted file mode 100644index e11b98f08..000000000--- a/tests/test_route_table.py+++ /dev/null@@ -1,282 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Tests for RouteTable model-based HTTP dispatch."""--from unittest.mock import AsyncMock, MagicMock--import pytest-from fastapi import FastAPI-from fastapi.testclient import TestClient--from switchyard.lib.endpoints.anthropic_messages_endpoint import (- AnthropicMessagesEndpoint,-)-from switchyard.lib.endpoints.openai_chat_endpoint import OpenAIChatEndpoint-from switchyard.lib.endpoints.responses_endpoint import ResponsesEndpoint-from switchyard.lib.proxy_context import CTX_CALLER_API_KEY-from switchyard.lib.route_table import RouteTable-from switchyard.lib.switchyard import Switchyard-from switchyard.server.switchyard_app import build_switchyard_app---def _make_chain(name: str = "chain") -> MagicMock:- chain = MagicMock(spec=Switchyard)- chain.call = AsyncMock(return_value={"chain": name})- chain.iter_components.return_value = [MagicMock(name=f"{name}-component")]- return chain---def _make_app(table: RouteTable) -> TestClient:- app = FastAPI()- app.state.switchyard = table- OpenAIChatEndpoint().register(app)- AnthropicMessagesEndpoint().register(app)- ResponsesEndpoint().register(app)- return TestClient(app, raise_server_exceptions=False)---class TestRouteTable:- def test_lookup_returns_registered_chain(self) -> None:- chain = _make_chain("specific")- table = RouteTable()- table.register("gpt-4o", chain)-- assert table.lookup_switchyard("gpt-4o") is chain-- def test_lookup_raises_key_error_for_unknown_model(self) -> None:- table = RouteTable()- table.register("gpt-4o", _make_chain())-- with pytest.raises(KeyError):- table.lookup_switchyard("not-registered")-- def test_register_overwrites_existing_key(self) -> None:- first = _make_chain("first")- second = _make_chain("second")- table = RouteTable()- table.register("m", first)- table.register("m", second)-- assert table.lookup_switchyard("m") is second-- def test_registered_models_preserves_registration_order(self) -> None:- table = RouteTable()- table.register("first", _make_chain("first"))- table.register("second", _make_chain("second"))-- assert table.registered_models() == ["first", "second"]-- def test_explicit_default_can_differ_from_registration_order(self) -> None:- table = RouteTable()- table.register("first", _make_chain("first"))- table.register("second", _make_chain("second"), default=True)-- assert table.default_model() == "second"-- table.set_default_model("first")- assert table.default_model() == "first"-- with pytest.raises(KeyError):- table.set_default_model("missing")-- def test_state_key_matches_switchyard(self) -> None:- assert RouteTable.state_key == Switchyard.state_key == "switchyard"-- def test_iter_components_deduplicates_shared_instances(self) -> None:- shared = MagicMock(name="shared")- unique_a = MagicMock(name="unique-a")- unique_b = MagicMock(name="unique-b")- first = MagicMock(spec=Switchyard)- first.iter_components.return_value = [shared, unique_a]- second = MagicMock(spec=Switchyard)- second.iter_components.return_value = [shared, unique_b]- table = RouteTable()- table.register("first", first)- table.register("second", second)-- components = table.iter_components()-- assert components.count(shared) == 1- assert unique_a in components- assert unique_b in components- assert len(components) == 3---def test_build_switchyard_app_accepts_table() -> None:- table = RouteTable()- app = build_switchyard_app(table)-- assert app.state.switchyard is table---def test_models_endpoint_lists_registered_models() -> None:- table = RouteTable()- table.register(- "switchyard-default-random-12345678",- _make_chain("random"),- metadata={- "display_name": "Switchyard random routing",- "description": "Random routes strong and weak.",- "switchyard": {- "algorithm": "random",- "strong_model": "strong/model",- "weak_model": "weak/model",- "strong_probability": 0.5,- },- },- )- table.register("strong/model", _make_chain("strong"))-- with TestClient(build_switchyard_app(table), raise_server_exceptions=False) as client:- response = client.get("/v1/models?limit=1000")-- assert response.status_code == 200- body = response.json()- assert body["object"] == "list"- assert body["has_more"] is False- assert body["default_model"] == "switchyard-default-random-12345678"- assert body["model_pool"] == [- "switchyard-default-random-12345678",- "strong/model",- ]- assert [item["id"] for item in body["data"]] == [- "switchyard-default-random-12345678",- "strong/model",- ]- assert body["data"][0]["display_name"] == "Switchyard random routing"- assert body["data"][0]["description"] == "Random routes strong and weak."- assert body["data"][0]["switchyard"]["algorithm"] == "random"- assert body["data"][0]["capabilities"]["streaming"] is True- assert body["data"][0]["capabilities"]["supported_inbound_formats"] == [- "openai-chat-completions",- "openai-responses",- "anthropic-messages",- ]---def test_models_endpoint_uses_table_default_model() -> None:- table = RouteTable()- table.register("strong/model", _make_chain("strong"))- table.register("weak/model", _make_chain("weak"))- table.register("switchyard-route", _make_chain("random"), default=True)-- with TestClient(build_switchyard_app(table), raise_server_exceptions=False) as client:- response = client.get("/v1/models?limit=1000")-- assert response.status_code == 200- body = response.json()- assert body["first_id"] == "strong/model"- assert body["default_model"] == "switchyard-route"- assert body["model_pool"] == [- "strong/model",- "weak/model",- "switchyard-route",- ]- assert [item["id"] for item in body["data"]] == [- "strong/model",- "weak/model",- "switchyard-route",- ]--- ("path", "body"),- [- ("/v1/chat/completions", {"model": "registered", "messages": [{"role": "user", "content": "hi"}]}),- ("/v1/messages", {"model": "registered", "max_tokens": 1, "messages": [{"role": "user", "content": "hi"}]}),- ("/v1/responses", {"model": "registered", "input": "hi"}),- ],-)-def test_http_dispatch_uses_registered_chain(path: str, body: dict[str, object]) -> None:- registered = _make_chain("registered")- table = RouteTable()- table.register("registered", registered)-- with _make_app(table) as client:- response = client.post(path, json=body)-- assert response.status_code == 200- assert response.json() == {"chain": "registered"}- registered.call.assert_awaited_once()--- ("path", "body"),- [- ("/v1/chat/completions", {"model": "registered", "messages": [{"role": "user", "content": "hi"}]}),- ("/v1/messages", {"model": "registered", "max_tokens": 1, "messages": [{"role": "user", "content": "hi"}]}),- ("/v1/responses", {"model": "registered", "input": "hi"}),- ],-)-def test_http_dispatch_attaches_caller_api_key_to_context(- path: str,- body: dict[str, object],-) -> None:- registered = _make_chain("registered")- table = RouteTable()- table.register("registered", registered)-- with _make_app(table) as client:- response = client.post(- path,- json=body,- headers={"Authorization": "Bearer caller-key"},- )-- assert response.status_code == 200- ctx = registered.call.await_args.kwargs["ctx"]- assert ctx.metadata[CTX_CALLER_API_KEY] == "caller-key" # pragma: allowlist secret--- ("path", "body"),- [- ("/v1/chat/completions", {"model": "unknown", "messages": [{"role": "user", "content": "hi"}]}),- ("/v1/messages", {"model": "unknown", "max_tokens": 1, "messages": [{"role": "user", "content": "hi"}]}),- ("/v1/responses", {"model": "unknown", "input": "hi"}),- ],-)-def test_http_dispatch_returns_404_for_unknown_model(- path: str, body: dict[str, object],-) -> None:- registered = _make_chain("registered")- table = RouteTable()- table.register("registered", registered)-- with _make_app(table) as client:- response = client.post(path, json=body)-- assert response.status_code == 404- assert response.json() == {- "error": {- "message": "No route registered for model unknown",- "type": "model_not_found",- "code": "model_not_found",- }- }- registered.call.assert_not_awaited()--- ("path", "body"),- [- ("/v1/chat/completions", {"model": "missing", "messages": [{"role": "user", "content": "hi"}]}),- ("/v1/messages", {"model": "missing", "max_tokens": 1, "messages": [{"role": "user", "content": "hi"}]}),- ("/v1/responses", {"model": "missing", "input": "hi"}),- ],-)-def test_http_dispatch_returns_404_without_default(path: str, body: dict[str, object]) -> None:- table = RouteTable()-- with _make_app(table) as client:- response = client.post(path, json=body)-- assert response.status_code == 404- assert response.json() == {- "error": {- "message": "No route registered for model missing",- "type": "model_not_found",- "code": "model_not_found",- }- }diff --git a/tests/test_routing_log_response_processor.py b/tests/test_routing_log_response_processor.pydeleted file mode 100644index c63d321e3..000000000--- a/tests/test_routing_log_response_processor.py+++ /dev/null@@ -1,297 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Tests for the per-request JSONL routing log (`serve --routing-log-file`)."""--from __future__ import annotations--import json-from collections.abc import AsyncIterator-from pathlib import Path--from fastapi import FastAPI-from fastapi.testclient import TestClient-from openai.types.chat import ChatCompletionChunk-from openai.types.chat.chat_completion_chunk import Choice as ChunkChoice-from openai.types.chat.chat_completion_chunk import ChoiceDelta-from openai.types.completion_usage import CompletionUsage--from switchyard.cli.switchyard_cli import _build_parser-from switchyard.lib.chat_response import ResponseStream-from switchyard.lib.endpoints import RoutingLogStatsEndpoint-from switchyard.lib.processors.routing_log_response_processor import (- RoutingLogResponseProcessor,-)-from switchyard.lib.proxy_context import CTX_PROXY_ACTUAL_MODEL-from switchyard.lib.request_metadata import RequestMetadata, attach_request_metadata-from switchyard_rust.core import ChatResponse, ProxyContext--TASK_HEADERS = {- "x-switchyard-intake-task": "hello-world-abc1",- "x-switchyard-trial-id": "hello-world-abc1-Xy7",- "proxy_x_session_id": "trial-session-1",-}---def _ctx(*, headers: dict[str, str] | None = None, model: str = "gpt-test") -> ProxyContext:- ctx = ProxyContext()- if headers is not None:- attach_request_metadata(ctx, RequestMetadata.from_headers(headers), headers)- ctx.metadata[CTX_PROXY_ACTUAL_MODEL] = model- return ctx---def _openai_completion() -> ChatResponse:- return ChatResponse.openai_completion({- "id": "chatcmpl-test",- "object": "chat.completion",- "created": 1700000000,- "model": "gpt-test",- "choices": [{- "index": 0,- "message": {"role": "assistant", "content": "hi"},- "finish_reason": "stop",- }],- "usage": {- "prompt_tokens": 8,- "completion_tokens": 3,- "total_tokens": 11,- "prompt_tokens_details": {- "cached_tokens": 6,- "cache_creation_tokens": 1,- },- "completion_tokens_details": {"reasoning_tokens": 2},- },- })---def _anthropic_completion() -> ChatResponse:- return ChatResponse.anthropic_completion({- "id": "msg_test",- "type": "message",- "role": "assistant",- "content": [{"type": "text", "text": "hi"}],- "model": "claude-test",- "stop_reason": "end_turn",- "stop_sequence": None,- "usage": {- "input_tokens": 7,- "output_tokens": 3,- "cache_creation_input_tokens": 2,- "cache_read_input_tokens": 4,- },- })---def _responses_completion() -> ChatResponse:- return ChatResponse.openai_responses_completion({- "id": "resp_test",- "object": "response",- "created_at": 1700000000,- "status": "completed",- "model": "codex-test",- "output": [{- "type": "message",- "role": "assistant",- "status": "completed",- "content": [{"type": "output_text", "text": "hi"}],- }],- "parallel_tool_calls": False,- "tool_choice": "auto",- "tools": [],- "usage": {- "input_tokens": 6,- "output_tokens": 2,- "total_tokens": 8,- "input_tokens_details": {"cached_tokens": 3},- "output_tokens_details": {"reasoning_tokens": 1},- },- })---def _records(log_file: Path) -> list[dict]:- return [json.loads(line) for line in log_file.read_text().splitlines()]---async def test_openai_completion_record_carries_task_and_usage(tmp_path: Path) -> None:- log_file = tmp_path / "routing_requests.jsonl"- ctx = _ctx(headers=TASK_HEADERS)- ctx.selected_model = "configured-route"- ctx.selected_target = "weak"- await RoutingLogResponseProcessor(log_file).process(ctx, _openai_completion())-- (record,) = _records(log_file)- assert record["task"] == "hello-world-abc1"- assert record["trial_id"] == "hello-world-abc1-Xy7"- assert record["session_id"] == "trial-session-1"- assert record["model"] == "gpt-test"- assert record["tier"] == "weak"- assert record["prompt_tokens"] == 8- assert record["cached_tokens"] == 6- assert record["cache_creation_tokens"] == 1- assert record["completion_tokens"] == 3- assert record["reasoning_tokens"] == 2- assert record["total_tokens"] == 11---async def test_anthropic_usage_sums_cache_siblings(tmp_path: Path) -> None:- log_file = tmp_path / "routing_requests.jsonl"- await RoutingLogResponseProcessor(log_file).process(- _ctx(headers=TASK_HEADERS, model="claude-test"), _anthropic_completion(),- )-- (record,) = _records(log_file)- assert record["prompt_tokens"] == 13 # input + cache_creation + cache_read- assert record["cached_tokens"] == 4- assert record["cache_creation_tokens"] == 2- assert record["completion_tokens"] == 3- assert record["reasoning_tokens"] == 0---async def test_responses_completion_uses_input_output_tokens(tmp_path: Path) -> None:- log_file = tmp_path / "routing_requests.jsonl"- await RoutingLogResponseProcessor(log_file).process(- _ctx(headers=TASK_HEADERS, model="codex-test"), _responses_completion(),- )-- (record,) = _records(log_file)- assert record["prompt_tokens"] == 6- assert record["cached_tokens"] == 3- assert record["cache_creation_tokens"] == 0- assert record["completion_tokens"] == 2- assert record["reasoning_tokens"] == 1---async def test_missing_headers_log_null_task_and_session(tmp_path: Path) -> None:- log_file = tmp_path / "routing_requests.jsonl"- await RoutingLogResponseProcessor(log_file).process(_ctx(), _openai_completion())-- (record,) = _records(log_file)- assert record["task"] is None- assert record["trial_id"] is None- assert record["session_id"] is None---async def test_streaming_appends_after_drain(tmp_path: Path) -> None:- log_file = tmp_path / "routing_requests.jsonl"- content_chunk = ChatCompletionChunk(- id="chatcmpl-test", object="chat.completion.chunk", created=1700000000,- model="gpt-test",- choices=[ChunkChoice(index=0, delta=ChoiceDelta(content="hi"), finish_reason="stop")],- )- usage_chunk = ChatCompletionChunk(- id="chatcmpl-test", object="chat.completion.chunk", created=1700000000,- model="gpt-test", choices=[],- usage=CompletionUsage(prompt_tokens=5, completion_tokens=3, total_tokens=8),- )-- async def _iter() -> AsyncIterator[ChatCompletionChunk]:- yield content_chunk- yield usage_chunk-- response = ChatResponse.openai_stream(ResponseStream(_iter()))- out = await RoutingLogResponseProcessor(log_file).process(- _ctx(headers=TASK_HEADERS), response,- )- assert not log_file.exists() # nothing until the stream drains-- forwarded = [chunk async for chunk in out.stream]- assert len(forwarded) == 2-- (record,) = _records(log_file)- assert record["task"] == "hello-world-abc1"- assert record["total_tokens"] == 8---async def test_appends_one_line_per_request(tmp_path: Path) -> None:- log_file = tmp_path / "routing_requests.jsonl"- processor = RoutingLogResponseProcessor(log_file)- for _ in range(3):- await processor.process(_ctx(headers=TASK_HEADERS), _openai_completion())- assert len(_records(log_file)) == 3---def test_snapshot_session_aggregates_models_and_ignores_bad_records(tmp_path: Path) -> None:- log_file = tmp_path / "routing_requests.jsonl"- log_file.write_text(- "\n".join([- json.dumps({- "session_id": "trial-session-1", "model": "model-a",- "prompt_tokens": 8, "cached_tokens": 2,- "cache_creation_tokens": 1, "completion_tokens": 3,- }),- json.dumps({- "session_id": "trial-session-1", "model": "model-b",- "prompt_tokens": 13, "cached_tokens": 4,- "cache_creation_tokens": 2, "completion_tokens": 5,- }),- json.dumps({"session_id": "other", "model": "model-a"}),- json.dumps(["not", "an", "object"]),- "not json",- ]) + "\n",- encoding="utf-8",- )-- snapshot = RoutingLogResponseProcessor(log_file).snapshot_session("trial-session-1")-- assert snapshot == {- "session_id": "trial-session-1",- "total_calls": 2,- "total_prompt_tokens": 21,- "total_cached_tokens": 6,- "total_cache_creation_tokens": 3,- "total_completion_tokens": 8,- "models": {- "model-a": {- "calls": 1, "prompt_tokens": 8, "cached_tokens": 2,- "cache_creation_tokens": 1, "completion_tokens": 3,- },- "model-b": {- "calls": 1, "prompt_tokens": 13, "cached_tokens": 4,- "cache_creation_tokens": 2, "completion_tokens": 5,- },- },- }---def test_session_stats_endpoint_returns_snapshot_and_404(tmp_path: Path) -> None:- log_file = tmp_path / "routing_requests.jsonl"- processor = RoutingLogResponseProcessor(log_file)- app = FastAPI()- endpoint = processor.get_endpoint()- assert isinstance(endpoint, RoutingLogStatsEndpoint)- endpoint.register(app)-- with TestClient(app) as client:- assert client.get(- "/v1/routing/session-stats", params={"session_id": "missing"}- ).status_code == 404-- log_file.write_text(- json.dumps({- "session_id": "trial-session-1", "model": "model-a",- "prompt_tokens": 8, "cached_tokens": 2,- "cache_creation_tokens": 1, "completion_tokens": 3,- }) + "\n",- encoding="utf-8",- )- with TestClient(app) as client:- response = client.get(- "/v1/routing/session-stats", params={"session_id": "trial-session-1"}- )- assert response.status_code == 200- assert response.json()["models"]["model-a"]["cached_tokens"] == 2---def test_serve_parser_accepts_routing_log_file() -> None:- parser = _build_parser()- args = parser.parse_args(- [- "serve",- "--routes",- "routes.yaml",- "--routing-log-file",- "tmp/routing.jsonl",- ]- )- assert args.routing_log_file == "tmp/routing.jsonl"diff --git a/tests/test_shell_tui.py b/tests/test_shell_tui.pyindex d898c9ac6..6904e5f9e 100644--- a/tests/test_shell_tui.py+++ b/tests/test_shell_tui.py@@ -38,7 +38,7 @@ import ttyimport pytest-from switchyard.server.shell_tui import CSI, ShellTUI+from switchyard.cli.launchers.shell_tui import CSI, ShellTUIdef _make_tui(@@ -56,7 +56,7 @@ def _make_tui(@pytest.fixturedef fixed_winsize(monkeypatch: pytest.MonkeyPatch) -> None:monkeypatch.setattr(- "switchyard.server.shell_tui._get_winsize",+ "switchyard.cli.launchers.shell_tui._get_winsize",lambda _fd: (24, 80),)@@ -370,7 +370,7 @@ def test_handle_winch_propagates_size_and_repaints(fcntl.fcntl(master_fd, fcntl.F_SETFL, flags | os.O_NONBLOCK)monkeypatch.setattr(- "switchyard.server.shell_tui._get_winsize",+ "switchyard.cli.launchers.shell_tui._get_winsize",lambda _fd: (40, 100),)@@ -437,7 +437,7 @@ def test_handle_winch_bumps_activity_ts_to_block_footer_thread_barge(fcntl.fcntl(master_fd, fcntl.F_SETFL, flags | os.O_NONBLOCK)monkeypatch.setattr(- "switchyard.server.shell_tui._get_winsize",+ "switchyard.cli.launchers.shell_tui._get_winsize",lambda _fd: (24, 80),)diff --git a/tests/test_sse_stream_close.py b/tests/test_sse_stream_close.pydeleted file mode 100644index 0376c60df..000000000--- a/tests/test_sse_stream_close.py+++ /dev/null@@ -1,159 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Streaming teardown: the upstream stream is closed on every exit path.--Regression coverage for the connection-pool leak where an interrupted SSE-response (client disconnect) never closed the upstream SDK ``AsyncStream``, so-its httpx connection was never returned to the pool. The fix closes the stream-from the SSE helpers' ``finally`` and gives ``ChatResponseStream`` an ``aclose``-that releases the original Python source.-"""--from __future__ import annotations--from typing import Any--from switchyard.lib.chat_response.openai_chat import ResponseStream-from switchyard.lib.endpoints.sse_helpers import (- _aclose_stream,- iter_anthropic_sse,- iter_chat_completion_sse,- iter_preframed_sse,-)---class _FakeAsyncStream:- """Mimics the OpenAI/Anthropic SDK ``AsyncStream``.-- Async-iterable and closable via an async ``close()`` (the SDK shape — not- the async-generator ``aclose``). Records whether ``close()`` ran so tests- can assert the upstream connection would have been released.- """-- def __init__(self, chunks: list[Any]) -> None:- self._chunks = list(chunks)- self._index = 0- self.closed = False-- def __aiter__(self) -> _FakeAsyncStream:- return self-- async def __anext__(self) -> Any:- if self._index >= len(self._chunks):- raise StopAsyncIteration- chunk = self._chunks[self._index]- self._index += 1- return chunk--- async def close(self) -> None:- self.closed = True---# ----------------------------------------------------------------------------# ChatResponseStream.aclose (Rust binding)-# ------------------------------------------------------------------------------async def test_chat_response_stream_aclose_closes_source() -> None:- """``aclose`` releases the original Python source even before iteration."""- fake = _FakeAsyncStream([{"id": "1"}])- stream = ResponseStream(fake)- await stream.aclose()- assert fake.closed---async def test_chat_response_stream_aclose_is_idempotent() -> None:- fake = _FakeAsyncStream([{"id": "1"}])- stream = ResponseStream(fake)- await stream.aclose()- await stream.aclose()- assert fake.closed---# ----------------------------------------------------------------------------# SSE helpers close their stream on early termination and normal completion-# ------------------------------------------------------------------------------async def test_chat_sse_closes_stream_on_client_disconnect() -> None:- """A client disconnect makes the server ``aclose()`` the SSE generator;- the helper's ``finally`` must then close the upstream stream."""- fake = _FakeAsyncStream([{"id": "1"}, {"id": "2"}, {"id": "3"}])- generator = iter_chat_completion_sse(fake)- first = await generator.__anext__()- assert first.startswith("data:")- await generator.aclose() # simulate disconnect mid-stream- assert fake.closed---async def test_chat_sse_closes_stream_on_normal_completion() -> None:- fake = _FakeAsyncStream([{"id": "1"}])- frames = [frame async for frame in iter_chat_completion_sse(fake)]- assert frames[-1] == "data: [DONE]\n\n"- assert fake.closed---async def test_chat_sse_closes_rust_stream_source_on_disconnect() -> None:- """End-to-end: SSE over a ``ChatResponseStream`` → disconnect → the wrapped- SDK ``AsyncStream`` (and its connection) is released. This is the exact leak- scenario from the OOM incident."""- fake = _FakeAsyncStream([{"id": "1"}, {"id": "2"}])- generator = iter_chat_completion_sse(ResponseStream(fake))- await generator.__anext__()- await generator.aclose()- assert fake.closed---async def test_anthropic_sse_closes_stream_on_disconnect() -> None:- fake = _FakeAsyncStream([{"type": "message_start"}, {"type": "content_block_delta"}])- generator = iter_anthropic_sse(fake)- await generator.__anext__()- await generator.aclose()- assert fake.closed---async def test_preframed_sse_closes_stream_on_disconnect() -> None:- fake = _FakeAsyncStream(["event: ping\ndata: {}\n\n", "event: ping\ndata: {}\n\n"])- generator = iter_preframed_sse(fake)- await generator.__anext__()- await generator.aclose()- assert fake.closed---async def test_preframed_sse_frames_responses_mapping_events() -> None:- fake = _FakeAsyncStream([- {- "type": "response.created",- "response": {"id": "resp-test"},- }- ])- frames = [frame async for frame in iter_preframed_sse(fake)]- assert frames == [- 'event: response.created\ndata: {"type": "response.created", "response": {"id": "resp-test"}}\n\n'- ]- assert fake.closed---# ----------------------------------------------------------------------------# _aclose_stream contract-# ------------------------------------------------------------------------------async def test_aclose_stream_prefers_aclose() -> None:- class WithAclose:- def __init__(self) -> None:- self.aclosed = False-- async def aclose(self) -> None:- self.aclosed = True-- obj = WithAclose()- await _aclose_stream(obj)- assert obj.aclosed---async def test_aclose_stream_tolerates_missing_closer() -> None:- # An object with neither ``aclose`` nor ``close`` must not raise.- await _aclose_stream(object())diff --git a/tests/test_stats_accumulator.py b/tests/test_stats_accumulator.pydeleted file mode 100644index c8b881a31..000000000--- a/tests/test_stats_accumulator.py+++ /dev/null@@ -1,247 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--from __future__ import annotations--import pytest--from switchyard.lib.stats_accumulator import StatsAccumulator---async def test_snapshot_sync_matches_async_snapshot_and_includes_tier_tokens():- stats = StatsAccumulator()- await stats.record_success(- model="strong/model",- backend_latency_ms=12.0,- tier="strong",- )- await stats.record_usage(- model="strong/model",- prompt_tokens=100,- completion_tokens=25,- cached_tokens=10,- total_latency_ms=20.0,- routing_overhead_ms=8.0,- tier="strong",- )- await stats.record_success(model="weak/model", tier="weak")- await stats.record_usage(- model="weak/model",- prompt_tokens=40,- completion_tokens=5,- tier="weak",- )-- sync_snapshot = stats.snapshot_sync()-- assert sync_snapshot == await stats.snapshot()- assert sync_snapshot["total_requests"] == 2- assert sync_snapshot["total_tokens"]["prompt"] == 140- assert sync_snapshot["total_tokens"]["completion"] == 30- assert sync_snapshot["models"]["strong/model"]["tier"] == "strong"- assert sync_snapshot["tiers"]["strong"]["prompt_tokens"] == 100- assert sync_snapshot["tiers"]["strong"]["completion_tokens"] == 25- assert sync_snapshot["tiers"]["weak"]["prompt_tokens"] == 40- assert sync_snapshot["tiers"]["weak"]["completion_tokens"] == 5---async def test_snapshot_includes_generic_tier_rollup():- stats = StatsAccumulator()- await stats.record_success(model="plugin/model-a", tier="plugin")- await stats.record_usage(- model="plugin/model-a",- prompt_tokens=2,- completion_tokens=3,- tier="plugin",- )- await stats.record_success(model="plugin/model-b", tier="plugin")- await stats.record_usage(- model="plugin/model-b",- prompt_tokens=5,- completion_tokens=7,- tier="plugin",- )-- snapshot = await stats.snapshot()-- assert snapshot["models"]["plugin/model-a"]["tier"] == "plugin"- assert snapshot["models"]["plugin/model-b"]["tier"] == "plugin"- assert snapshot["tiers"]["plugin"]["model"] == "plugin/model-a"- assert snapshot["tiers"]["plugin"]["calls"] == 2- assert snapshot["tiers"]["plugin"]["prompt_tokens"] == 7- assert snapshot["tiers"]["plugin"]["completion_tokens"] == 10---async def test_same_model_can_contribute_to_distinct_tier_rollups():- stats = StatsAccumulator()- await stats.record_success(model="shared/model", tier="weak")- await stats.record_usage(- model="shared/model",- prompt_tokens=2,- completion_tokens=3,- tier="weak",- )- await stats.record_success(model="shared/model", tier="primary")- await stats.record_usage(- model="shared/model",- prompt_tokens=5,- completion_tokens=7,- tier="primary",- )-- snapshot = await stats.snapshot()-- assert snapshot["models"]["shared/model"]["calls"] == 2- assert snapshot["models"]["shared/model"]["prompt_tokens"] == 7- assert snapshot["models"]["shared/model"]["completion_tokens"] == 10- assert snapshot["tiers"]["weak"]["calls"] == 1- assert snapshot["tiers"]["weak"]["prompt_tokens"] == 2- assert snapshot["tiers"]["weak"]["completion_tokens"] == 3- assert snapshot["tiers"]["primary"]["calls"] == 1- assert snapshot["tiers"]["primary"]["prompt_tokens"] == 5- assert snapshot["tiers"]["primary"]["completion_tokens"] == 7---async def test_usage_can_attach_explicit_untiered_success_to_tier():- stats = StatsAccumulator()- await stats.record_success(model="shared/model")- await stats.record_usage(- model="shared/model",- prompt_tokens=2,- completion_tokens=3,- tier="weak",- success_was_untiered=True,- )-- snapshot = await stats.snapshot()-- assert snapshot["models"]["shared/model"]["calls"] == 1- assert snapshot["tiers"]["weak"]["calls"] == 1- assert snapshot["tiers"]["weak"]["prompt_tokens"] == 2- assert snapshot["tiers"]["weak"]["completion_tokens"] == 3---async def test_legacy_untiered_success_then_tiered_usage_counts_tier_call():- stats = StatsAccumulator()- await stats.record_success(model="shared/model")- await stats.record_usage(- model="shared/model",- prompt_tokens=2,- completion_tokens=3,- tier="weak",- )-- snapshot = await stats.snapshot()-- assert snapshot["models"]["shared/model"]["calls"] == 1- assert snapshot["tiers"]["weak"]["calls"] == 1- assert snapshot["tiers"]["weak"]["prompt_tokens"] == 2- assert snapshot["tiers"]["weak"]["completion_tokens"] == 3---async def test_reset_sync_clears_async_recorded_stats():- stats = StatsAccumulator()- await stats.record_success(model="model")- await stats.record_usage(model="model", prompt_tokens=10, completion_tokens=5)-- stats.reset_sync()-- snapshot = await stats.snapshot()- assert snapshot["total_requests"] == 0- assert snapshot["total_tokens"]["total"] == 0- assert snapshot["models"] == {}---async def test_classifier_usage_recorded_into_separate_bucket():- """Classifier calls don't leak into the routed-backend ``models`` block.-- Default TB-lite config has classifier-model == weak-model- (Nemotron-3-Super-v3). Without the separate bucket the two would- accumulate into the same entry and the per-classifier breakdown- would be lost.- """- stats = StatsAccumulator()- # Same model name on both sides — exactly the collision case.- await stats.record_success(model="nvidia/nemotron-3-super-v3", tier="weak")- await stats.record_usage(- model="nvidia/nemotron-3-super-v3",- prompt_tokens=1_000,- completion_tokens=200,- tier="weak",- )- await stats.record_classifier_usage(- model="nvidia/nemotron-3-super-v3",- prompt_tokens=300,- completion_tokens=50,- latency_ms=42.0,- )-- snapshot = await stats.snapshot()-- # Backend bucket counts the routed call only.- backend = snapshot["models"]["nvidia/nemotron-3-super-v3"]- assert backend["prompt_tokens"] == 1_000- assert backend["completion_tokens"] == 200- assert backend["calls"] == 1- # Classifier bucket counts the classifier call only.- classifier = snapshot["classifier"]["models"]["nvidia/nemotron-3-super-v3"]- assert classifier["prompt_tokens"] == 300- assert classifier["completion_tokens"] == 50- assert classifier["calls"] == 1- assert classifier["model_call_latency"]["count"] == 1---async def test_cost_estimate_total_includes_classifier_overhead():- """Headline ``cost_estimate.total_cost`` is backend + classifier.-- Existing consumers (baseline manifests, dashboards) read- ``total_cost`` and don't know about the new classifier bucket; the- accumulator must roll the two together so those readers reflect- true spend.- """- stats = StatsAccumulator()- # Use a model with known pricing so we get non-zero numbers.- await stats.record_success(model="nvidia/nvidia/nemotron-3-super-v3", tier="weak")- await stats.record_usage(- model="nvidia/nvidia/nemotron-3-super-v3",- prompt_tokens=1_000_000,- completion_tokens=0,- tier="weak",- )- await stats.record_classifier_usage(- model="nvidia/nvidia/nemotron-3-super-v3",- prompt_tokens=1_000_000,- completion_tokens=0,- )-- snapshot = await stats.snapshot()- cost = snapshot["cost_estimate"]- # Nemotron-3 Super input is $0.10/Mtok — backend 1M tokens = $0.10,- # classifier 1M tokens = $0.10, total = $0.20.- assert cost["backend_cost"] == pytest.approx(0.10, rel=0.01)- assert cost["classifier_cost"] == pytest.approx(0.10, rel=0.01)- assert cost["total_cost"] == pytest.approx(0.20, rel=0.01)---async def test_reset_clears_classifier_bucket():- stats = StatsAccumulator()- await stats.record_classifier_usage(- model="router/clf",- prompt_tokens=100,- completion_tokens=20,- )-- stats.reset_sync()-- snapshot = await stats.snapshot()- assert snapshot["classifier"]["total_requests"] == 0- assert snapshot["classifier"]["models"] == {}- assert snapshot["cost_estimate"]["classifier_cost"] == 0.0---# Per-target attribution after evict-and-reroute is covered by-# `evict_and_reroute_attributes_error_to_weak_and_success_to_strong` in-# crates/switchyard-components/tests/stats_processors.rs — that integration-# test drives the chain executor end-to-end. Re-asserting the same shape here-# by hand-recording into the accumulator would only verify the accumulator's-# incrementers (already covered above), not the eviction code path.diff --git a/tests/test_stream_close_chain.py b/tests/test_stream_close_chain.pydeleted file mode 100644index 81b6e085e..000000000--- a/tests/test_stream_close_chain.py+++ /dev/null@@ -1,212 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Close ownership survives the Python -> Rust-core -> Python round trip.--The production path is: a Python backend returns-``ChatResponse.openai_stream(ResponseStream(sdk_stream))``; the backend adapter-converts it to a Rust-core stream via ``take_core``; response processors run in-core; the terminal translator rebuilds a Python ``ChatResponseStream`` via-``from_core``; the endpoint feeds that into an SSE helper. The rebuilt stream no-longer references the original SDK ``AsyncStream``, so unless close ownership is-preserved across the conversion, closing it on client disconnect never releases-the upstream connection — the OOM connection-pool leak.--These tests exercise the *real* chain (``Switchyard`` + ``TranslationEngine``),-not just the SSE helper in isolation, and assert the fake SDK stream's close-hook runs after the response stream is closed mid-flight — for both same-format-(OpenAI in / OpenAI backend) and translated (Anthropic in / OpenAI backend)-streaming.-"""--from __future__ import annotations--import asyncio-from typing import Any--from switchyard.lib.chat_response.openai_chat import ResponseStream-from switchyard.lib.endpoints.sse_helpers import (- iter_anthropic_sse,- iter_chat_completion_sse,-)-from switchyard.lib.proxy_context import ProxyContext-from switchyard.lib.roles import LLMBackend-from switchyard.lib.switchyard import Switchyard-from switchyard_rust.core import ChatRequest, ChatRequestType, ChatResponse-from switchyard_rust.translation import TranslationEngine---class _FakeSdkStream:- """Mimics the OpenAI SDK ``AsyncStream``: async-iterable with async ``close``.-- Records whether ``close()`` ran so a test can assert the upstream response- (and its pooled connection) would have been released.- """-- def __init__(self, chunks: list[dict[str, Any]]) -> None:- self._chunks = list(chunks)- self._index = 0- self.closed = False-- def __aiter__(self) -> _FakeSdkStream:- return self-- async def __anext__(self) -> dict[str, Any]:- if self._index >= len(self._chunks):- raise StopAsyncIteration- chunk = self._chunks[self._index]- self._index += 1- return chunk-- async def close(self) -> None:- self.closed = True---def _chunk(content: str | None, *, role: str | None = None, finish: str | None = None) -> dict:- delta: dict[str, Any] = {}- if role is not None:- delta["role"] = role- if content is not None:- delta["content"] = content- return {- "id": "chatcmpl-close",- "object": "chat.completion.chunk",- "created": 1700000000,- "model": "close-model",- "choices": [{"index": 0, "delta": delta, "finish_reason": finish}],- }---def _chunks() -> list[dict]:- return [- _chunk("", role="assistant"),- _chunk("hello"),- _chunk(" world"),- _chunk(None, finish="stop"),- ]---class _StreamingSdkBackend(LLMBackend):- """Backend returning a streaming response wrapping a fake SDK stream.-- Holds the fake so a test can assert it was closed. Supports both OpenAI Chat- and Anthropic inbound so the same backend serves the same-format and the- translated paths (the response is always an OpenAI chat stream).- """-- def __init__(self, fake: _FakeSdkStream) -> None:- self._fake = fake-- @property- def supported_request_types(self) -> list[ChatRequestType]:- return [ChatRequestType.OPENAI_CHAT, ChatRequestType.ANTHROPIC]-- async def call(self, ctx: ProxyContext, request: ChatRequest) -> ChatResponse:- _ = ctx, request- return ChatResponse.openai_stream(ResponseStream(self._fake))---async def _wait_closed(fake: _FakeSdkStream, timeout: float = 2.0) -> bool:- """Wait for the best-effort, runtime-scheduled source close to land.-- ``ChatResponseStream``'s drop-time close runs as a fire-and-forget task on- the Rust tokio runtime (a separate thread), so it may not have completed the- instant ``aclose`` returns to the asyncio loop. Poll briefly for it.- """- loop = asyncio.get_event_loop()- deadline = loop.time() + timeout- while not fake.closed and loop.time() < deadline:- await asyncio.sleep(0.005)- return fake.closed---def _openai_request() -> ChatRequest:- return ChatRequest.openai_chat({- "model": "close-model",- "messages": [{"role": "user", "content": "hi"}],- "stream": True,- })---def _anthropic_request() -> ChatRequest:- return ChatRequest.anthropic({- "model": "close-model",- "max_tokens": 16,- "messages": [{"role": "user", "content": "hi"}],- "stream": True,- })---async def test_same_format_stream_close_releases_sdk_source_through_chain() -> None:- """OpenAI in / OpenAI backend: closing the SSE stream after one frame closes- the SDK stream the backend wrapped — across take_core/from_core."""- fake = _FakeSdkStream(_chunks())- chain = Switchyard(backend=_StreamingSdkBackend(fake), translator=TranslationEngine())-- result = await chain.call(_openai_request())- sse = iter_chat_completion_sse(result)- first = await sse.__anext__()- assert first.startswith("data:")-- await sse.aclose() # client disconnect mid-stream-- assert await _wait_closed(fake), "backend SDK stream was not closed after teardown"---async def test_translated_stream_close_releases_sdk_source_through_chain() -> None:- """Anthropic in / OpenAI backend: the response is translated through- ``translate_stream``; closing the SSE stream still closes the SDK source."""- fake = _FakeSdkStream(_chunks())- chain = Switchyard(backend=_StreamingSdkBackend(fake), translator=TranslationEngine())-- result = await chain.call(_anthropic_request())- sse = iter_anthropic_sse(result)- first = await sse.__anext__()- assert first.startswith("event:")-- await sse.aclose() # client disconnect mid-stream-- assert await _wait_closed(fake), "backend SDK stream was not closed after translated teardown"---class _FakeTranslatableInput:- """Async-iterable openai chunk source with an async ``aclose`` it records."""-- def __init__(self, chunks: list[dict[str, Any]]) -> None:- self._chunks = list(chunks)- self._index = 0- self.aclosed = False-- def __aiter__(self) -> _FakeTranslatableInput:- return self-- async def __anext__(self) -> dict[str, Any]:- if self._index >= len(self._chunks):- raise StopAsyncIteration- chunk = self._chunks[self._index]- self._index += 1- return chunk-- async def aclose(self) -> None:- self.aclosed = True---async def test_translate_stream_closes_input_on_disconnect() -> None:- """``translate_stream``'s ``finally`` closes its input when the consuming- SSE generator is ``aclose``-ed mid-stream (the client-disconnect path)."""- fake = _FakeTranslatableInput(_chunks())- gen = TranslationEngine().translate_stream("openai_chat", "anthropic_messages", fake)-- await gen.__anext__()- await gen.aclose()-- assert fake.aclosed, "translate_stream did not close its input stream"---async def test_translate_stream_closes_input_on_normal_completion() -> None:- fake = _FakeTranslatableInput(_chunks())- gen = TranslationEngine().translate_stream("openai_chat", "anthropic_messages", fake)-- _ = [frame async for frame in gen]-- assert fake.aclosed, "translate_stream did not close its input on completion"diff --git a/tests/test_stream_leak_repro.py b/tests/test_stream_leak_repro.pydeleted file mode 100644index 22c1f571c..000000000--- a/tests/test_stream_leak_repro.py+++ /dev/null@@ -1,261 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Empirical repro: unclosed streaming responses exhaust the httpx pool, and our-SSE-helper close path releases the connection.--Uses the REAL OpenAI ``AsyncStream`` against a loopback SSE server, with the-httpx pool capped at a single connection so exhaustion is deterministic. The-stream is driven through the real ``iter_chat_completion_sse`` helper, so the-``finally: await _aclose_stream(...)`` we added is the thing under test.--Marked ``integration`` (real sockets + a pool timeout) so the default unit gate-can deselect it with ``-m "not integration"``; it is the canonical regression-for the OOM connection-pool leak.-"""--from __future__ import annotations--import json-import threading-import time-from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer--import httpx-import openai-import pytest-from openai import AsyncOpenAI--from switchyard.lib.chat_response.openai_chat import ResponseStream-from switchyard.lib.endpoints.sse_helpers import iter_chat_completion_sse-from switchyard.lib.proxy_context import ProxyContext-from switchyard.lib.roles import LLMBackend-from switchyard.lib.switchyard import Switchyard-from switchyard_rust.core import ChatRequest, ChatRequestType, ChatResponse-from switchyard_rust.translation import TranslationEngine--pytestmark = pytest.mark.integration---def _chunk(i: int) -> dict:- return {- "id": "chatcmpl-leak",- "object": "chat.completion.chunk",- "created": 1700000000,- "model": "leak-model",- "choices": [{"index": 0, "delta": {"content": str(i)}, "finish_reason": None}],- }---class _SlowSSEStub:- """Loopback OpenAI-compatible server that streams SSE chunks slowly.-- Streams many chunks with a delay so a client that reads one frame and- abandons the response leaves the connection checked out — the production- client-disconnect scenario.- """-- def __init__(self, n_chunks: int = 500, delay_s: float = 0.02) -> None:- self._n = n_chunks- self._delay = delay_s- self._server: ThreadingHTTPServer | None = None- self._thread: threading.Thread | None = None-- def __enter__(self) -> _SlowSSEStub:- n, delay = self._n, self._delay-- class Handler(BaseHTTPRequestHandler):- protocol_version = "HTTP/1.1"-- def do_POST(self) -> None:- length = int(self.headers.get("content-length", "0"))- if length:- self.rfile.read(length)- self.send_response(200)- self.send_header("Content-Type", "text/event-stream")- self.send_header("Transfer-Encoding", "chunked")- self.end_headers()-- def send(data: bytes) -> None:- self.wfile.write(f"{len(data):X}\r\n".encode())- self.wfile.write(data)- self.wfile.write(b"\r\n")- self.wfile.flush()-- try:- for i in range(n):- send(f"data: {json.dumps(_chunk(i))}\n\n".encode())- time.sleep(delay)- send(b"data: [DONE]\n\n")- self.wfile.write(b"0\r\n\r\n")- self.wfile.flush()- except (BrokenPipeError, ConnectionResetError, OSError):- return # client disconnected — expected in these tests-- def log_message(self, *_args: object) -> None:- return-- self._server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)- self._thread = threading.Thread(target=self._server.serve_forever, daemon=True)- self._thread.start()- return self-- def __exit__(self, *_args: object) -> None:- if self._server is not None:- self._server.shutdown()- self._server.server_close()- if self._thread is not None:- self._thread.join(timeout=2)-- @property- def base_url(self) -> str:- assert self._server is not None- host, port = self._server.server_address- return f"http://{host}:{port}/v1"---def _client(base_url: str) -> AsyncOpenAI:- # max_connections=1 makes exhaustion deterministic: one leaked stream takes- # the only slot. pool=1.5s fails the next acquire fast; max_retries=0 keeps- # the SDK from retrying the pool timeout (which would only slow the test).- return AsyncOpenAI(- base_url=base_url,- api_key="test",- max_retries=0,- http_client=httpx.AsyncClient(- limits=httpx.Limits(max_connections=1, max_keepalive_connections=1),- timeout=httpx.Timeout(5.0, pool=1.5),- ),- )---def _pool_conns(client: AsyncOpenAI) -> int:- """Best-effort live connection count for evidence logging (version-dependent)."""- try:- return len(client._client._transport._pool.connections) # type: ignore[attr-defined]- except Exception:- return -1---async def _open_and_read_one(client: AsyncOpenAI, *, close: bool):- """Open a streaming completion, drive it through the real SSE helper for one- frame, then either close it (our fix) or abandon it (the leak)."""- stream = await client.chat.completions.create(- model="leak-model",- messages=[{"role": "user", "content": "hi"}],- stream=True,- )- sse = iter_chat_completion_sse(stream)- first = await sse.__anext__()- assert first.startswith("data:"), first- assert "error" not in first, f"stream errored on first frame: {first}"- if close:- await sse.aclose() # our fix: finally -> _aclose_stream -> AsyncStream.close()- return None- return sse, stream # hold refs so the connection stays checked out---async def test_unclosed_stream_exhausts_pool() -> None:- """An abandoned stream pins its pooled connection; the next request starves."""- with _SlowSSEStub() as stub:- client = _client(stub.base_url)- try:- held = await _open_and_read_one(client, close=False) # leak: never closed- assert held is not None- assert _pool_conns(client) == 1- # The only pool slot is occupied by the abandoned stream; a new- # streaming request cannot acquire a connection and must fail.- with pytest.raises(- (- openai.APITimeoutError,- openai.APIConnectionError,- httpx.PoolTimeout,- httpx.TimeoutException,- )- ):- await _open_and_read_one(client, close=False)- finally:- await client.close()---async def test_our_close_releases_pool_connection() -> None:- """Closing via the SSE helper frees the slot, so a sequence of streams runs."""- with _SlowSSEStub() as stub:- client = _client(stub.base_url)- try:- # Our fix closes each stream, freeing the single pool slot, so a- # whole sequence of streaming requests succeeds — no exhaustion.- for _ in range(5):- await _open_and_read_one(client, close=True)- assert _pool_conns(client) <= 1- finally:- await client.close()---class _RealSdkStreamingBackend(LLMBackend):- """Backend that issues a real streaming SDK call and wraps the AsyncStream.-- Mirrors the production shape: ``call`` returns- ``ChatResponse.openai_stream(ResponseStream(sdk_stream))`` where- ``sdk_stream`` is a genuine OpenAI ``AsyncStream`` holding a pooled httpx- connection. Running this through ``Switchyard`` exercises the real- ``take_core`` -> Rust-core -> ``from_core`` round trip that drops the- source — the exact path the in-process fakes cannot reach.- """-- def __init__(self, client: AsyncOpenAI) -> None:- self._client = client-- @property- def supported_request_types(self) -> list[ChatRequestType]:- return [ChatRequestType.OPENAI_CHAT]-- async def call(self, ctx: ProxyContext, request: ChatRequest) -> ChatResponse:- _ = ctx, request- sdk_stream = await self._client.chat.completions.create(- model="leak-model",- messages=[{"role": "user", "content": "hi"}],- stream=True,- )- return ChatResponse.openai_stream(ResponseStream(sdk_stream))---def _chain_request() -> ChatRequest:- return ChatRequest.openai_chat({- "model": "leak-model",- "messages": [{"role": "user", "content": "hi"}],- "stream": True,- })---async def test_chain_round_trip_releases_pool_connection() -> None:- """End-to-end through the production chain: a real SDK stream survives- ``take_core``/``from_core`` and is released on client disconnect.-- With a single pool slot, each iteration acquires the only connection, reads- one frame, and abandons the stream. The next iteration's upstream call must- re-acquire that slot, which only succeeds if the prior stream's connection- was actually returned to the pool. If close ownership were lost across the- Rust-core conversion, the second iteration would starve and raise- ``PoolTimeout``; the loop completing proves the fix holds through the real- chain (backend -> take_core -> core -> from_core -> translator -> SSE).- """- with _SlowSSEStub() as stub:- client = _client(stub.base_url)- chain = Switchyard(- backend=_RealSdkStreamingBackend(client),- translator=TranslationEngine(),- )- try:- for _ in range(5):- result = await chain.call(_chain_request())- sse = iter_chat_completion_sse(result)- first = await sse.__anext__()- assert first.startswith("data:"), first- assert "error" not in first, f"stream errored on first frame: {first}"- # Disconnect mid-stream. The next iteration's acquire blocks on- # the pool until the runtime-scheduled close returns this slot.- await sse.aclose()- assert _pool_conns(client) <= 1- finally:- await client.close()diff --git a/tests/test_switchyard.py b/tests/test_switchyard.pydeleted file mode 100644index 34e794688..000000000--- a/tests/test_switchyard.py+++ /dev/null@@ -1,517 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Tests for Switchyard executor."""--import asyncio--import pytest-from openai.types.chat import ChatCompletion-from openai.types.chat.chat_completion import Choice-from openai.types.chat.chat_completion_message import ChatCompletionMessage-from openai.types.completion_usage import CompletionUsage--from switchyard.lib.proxy_context import ProxyContext-from switchyard.lib.roles import LLMBackend-from switchyard.lib.switchyard import Switchyard-from switchyard_rust.components import (- StatsRequestProcessor,-)-from switchyard_rust.core import (- ChatRequest,- ChatRequestType,- ChatResponse,- ChatResponseType,- SwitchyardContextWindowExceededError,- response_type_matches,-)-from switchyard_rust.translation import TranslationEngine--# ----------------------------------------------------------------------------# Fixtures-# ------------------------------------------------------------------------------def make_completion(*, model: str = "gpt-4o", content: str = "hello") -> ChatCompletion:- return ChatCompletion(- id="chatcmpl-test",- object="chat.completion",- created=1700000000,- model=model,- choices=[- Choice(- index=0,- message=ChatCompletionMessage(role="assistant", content=content),- finish_reason="stop",- )- ],- usage=CompletionUsage(prompt_tokens=10, completion_tokens=5, total_tokens=15),- )---def make_request(*, model: str = "gpt-4o") -> ChatRequest:- return ChatRequest.openai_chat({- "model": model,- "messages": [{"role": "user", "content": "hi"}],- })---# ----------------------------------------------------------------------------# Test helpers — concrete processors for testing-# ------------------------------------------------------------------------------class ModelOverrideProcessor:- """Mutates the model field on the request body."""-- def __init__(self, new_model: str) -> None:- self._new_model = new_model-- async def process(self, ctx, request):- request.set_model(self._new_model)- return request---class MetadataTagProcessor:- """Writes a tag into ctx.metadata to prove it ran."""-- def __init__(self, tag: str) -> None:- self._tag = tag-- async def process(self, ctx, request):- ctx.metadata[self._tag] = True- return request---class MetadataAssertResponseProcessor:- """Asserts request metadata remains visible after Rust-native components run."""-- def __init__(self, expected: dict[str, object]) -> None:- self._expected = expected-- async def process(self, ctx, response):- for key, expected in self._expected.items():- assert ctx.metadata[key] == expected- return response---class ModelMetadataProcessor:- async def process(self, ctx, request):- ctx.metadata["seen_model"] = request.model- return request---class ModelMetadataAssertProcessor:- async def process(self, ctx, response):- assert ctx.metadata["seen_model"] == response.body["model"]- return response---class FailingRequestProcessor:- async def process(self, ctx, request):- ctx.metadata["request_started"] = request.model- raise RuntimeError("request processor exploded")---class InvalidResponseProcessor:- async def process(self, ctx, response):- ctx.metadata["response_started"] = True- return {"not": "a ChatResponse"}---class ContextReadingStreamTapProcessor:- def __init__(self) -> None:- self.seen_models: list[str | None] = []-- async def process(self, ctx, response):- ctx.selected_model = "stream-selected"-- async def tap(_event):- self.seen_models.append(ctx.selected_model)-- response.stream.tap(tap)- return response---class ContentUpperProcessor:- """Uppercases the first choice's content (non-streaming only)."""-- async def process(self, ctx, response):- if response_type_matches(response, ChatResponseType.OPENAI_COMPLETION):- body = response.body- body["choices"][0]["message"]["content"] = (- body["choices"][0]["message"].get("content") or ""- ).upper()- response.replace_body(body)- return response---class RecordingBackend(LLMBackend):- """Returns a canned OpenAI response and records processed requests."""-- def __init__(self, *, content: str = "base") -> None:- self._content = content- self.requests: list[ChatRequest] = []-- @property- def supported_request_types(self) -> list[ChatRequestType]:- return [ChatRequestType.OPENAI_CHAT]-- async def call(self, ctx: ProxyContext, request: ChatRequest) -> ChatResponse:- self.requests.append(request)- if request.model:- ctx.selected_model = request.model- return ChatResponse.openai_completion(- make_completion(model=request.model or "test-model", content=self._content)- )---class LegacyBackendWithoutSupportedTypes(LLMBackend):- """Compatibility backend that relies on the legacy Python executor surface."""-- def __init__(self) -> None:- self.requests: list[ChatRequest] = []-- async def call(self, ctx: ProxyContext, request: ChatRequest) -> ChatResponse:- self.requests.append(request)- if request.model:- ctx.selected_model = request.model- return ChatResponse.openai_completion(- make_completion(model=request.model or "test-model", content="legacy")- )---class FailingBackend(LLMBackend):- """Raises after mutating context so restoration-on-error is observable."""-- @property- def supported_request_types(self) -> list[ChatRequestType]:- return [ChatRequestType.OPENAI_CHAT]-- async def call(self, ctx: ProxyContext, request: ChatRequest) -> ChatResponse:- ctx.metadata["backend_started"] = request.model- raise RuntimeError("backend exploded")---class RecordingTranslator:- """Records the request the executor hands to the terminal translator."""-- def __init__(self) -> None:- self.request_model: str | None = None-- async def translate(- self,- ctx: ProxyContext,- request: ChatRequest,- response: ChatResponse,- ):- _ = ctx- self.request_model = request.model- return response.body---async def _single_chunk_stream():- yield {- "id": "chatcmpl-test",- "object": "chat.completion.chunk",- "model": "gpt-4o",- "choices": [- {- "index": 0,- "delta": {"content": "hello"},- "finish_reason": None,- }- ],- }---class StreamingBackend(LLMBackend):- @property- def supported_request_types(self) -> list[ChatRequestType]:- return [ChatRequestType.OPENAI_CHAT]-- async def call(self, ctx: ProxyContext, request: ChatRequest) -> ChatResponse:- _ = ctx, request- return ChatResponse.openai_stream(_single_chunk_stream())---class PickWeakProcessor:- """Routes the first attempt to the weak target."""-- async def process(self, ctx: ProxyContext, request: ChatRequest) -> ChatRequest:- ctx.selected_target = "weak"- return request---class OverflowWeakBackend(LLMBackend):- """Overflows on weak and succeeds once the compatibility chain rewrites to strong."""-- def __init__(self) -> None:- self.calls: list[str | None] = []-- @property- def supported_request_types(self) -> list[ChatRequestType]:- return [ChatRequestType.OPENAI_CHAT]-- async def call(self, ctx: ProxyContext, request: ChatRequest) -> ChatResponse:- self.calls.append(ctx.selected_target)- if ctx.selected_target == "weak":- error = SwitchyardContextWindowExceededError("weak target overflowed")- error.target_id = "weak"- error.model = "weak-model"- raise error- ctx.selected_model = "strong-model"- return ChatResponse.openai_completion(- make_completion(model=request.model or "strong-model", content="fallback")- )---class ExceptionOnlyOverflowBackend(LLMBackend):- """Overflows with only an exception target id, then succeeds on fallback."""-- def __init__(self) -> None:- self.calls: list[str | None] = []-- @property- def supported_request_types(self) -> list[ChatRequestType]:- return [ChatRequestType.OPENAI_CHAT]-- async def call(self, ctx: ProxyContext, request: ChatRequest) -> ChatResponse:- self.calls.append(ctx.selected_target)- if len(self.calls) == 1:- error = SwitchyardContextWindowExceededError("weak target overflowed")- error.target_id = "weak"- error.model = "weak-model"- raise error- ctx.selected_model = "strong-model"- return ChatResponse.openai_completion(- make_completion(model=request.model or "strong-model", content="fallback")- )---# ----------------------------------------------------------------------------# Switchyard executor-# ------------------------------------------------------------------------------class TestSwitchyard:- def _make_chain(self, **overrides):- defaults = {- "backend": RecordingBackend(content="base"),- "translator": TranslationEngine(),- }- defaults.update(overrides)- return Switchyard(**defaults)-- def test_public_chain_classes_are_switchyard_rust_compatibility_exports(self):- from switchyard_rust import core as rust_core-- assert Switchyard is rust_core.Switchyard-- async def test_minimal_chain(self):- """Backend + translator only, no processors."""- chain = self._make_chain()- result = await chain.call(make_request())- assert result["choices"][0]["message"]["content"] == "base"-- async def test_python_backend_without_supported_request_types_keeps_old_switchyard_behavior(- self,- ):- backend = LegacyBackendWithoutSupportedTypes()- chain = self._make_chain(backend=backend)-- result = await chain.call(make_request(model="gpt-4o"))-- assert result["choices"][0]["message"]["content"] == "legacy"- assert backend.requests[-1].request_type == ChatRequestType.OPENAI_CHAT-- async def test_python_backend_without_supported_request_types_accepts_legacy_anthropic_request(- self,- ):- """The compatibility fallback is intentionally all formats, matching old Python."""- backend = LegacyBackendWithoutSupportedTypes()- chain = self._make_chain(- backend=backend,- translator=RecordingTranslator(),- )- request = ChatRequest.anthropic({- "model": "claude-test",- "messages": [{"role": "user", "content": "hi"}],- })-- result = await chain.call(request)-- assert result["choices"][0]["message"]["content"] == "legacy"- assert backend.requests[-1].request_type == ChatRequestType.ANTHROPIC-- async def test_request_processor_runs(self):- chain = self._make_chain(- request_processors=[ModelOverrideProcessor("gpt-4o-mini")],- )- await chain.call(make_request(model="gpt-4o"))- backend = chain._backend- assert isinstance(backend, RecordingBackend)- assert backend.requests[-1].model == "gpt-4o-mini"-- async def test_multiple_request_processors_chain(self):- chain = self._make_chain(- request_processors=[- MetadataTagProcessor("first"),- MetadataTagProcessor("second"),- ],- )- await chain.call(make_request())-- async def test_response_processor_runs(self):- chain = self._make_chain(- response_processors=[ContentUpperProcessor()],- )- result = await chain.call(make_request())- assert result["choices"][0]["message"]["content"] == "BASE"-- async def test_full_chain(self):- """Request processors → backend → response processors → translator."""- chain = self._make_chain(- request_processors=[- MetadataTagProcessor("tagged"),- ModelOverrideProcessor("gpt-4o-mini"),- ],- response_processors=[ContentUpperProcessor()],- )- result = await chain.call(make_request())- assert result["choices"][0]["message"]["content"] == "BASE"-- async def test_translator_receives_processed_request_from_compatibility_executor(self):- """Terminal translation uses the post-request-pipeline request."""- translator = RecordingTranslator()- chain = self._make_chain(- request_processors=[ModelOverrideProcessor("gpt-4o-mini")],- translator=translator,- )-- await chain.call(make_request(model="gpt-4o"))-- assert translator.request_model == "gpt-4o-mini"-- async def test_python_metadata_survives_across_mixed_rust_and_python_components(self):- """Python metadata is carried through Rust context across mixed components."""- chain = self._make_chain(- request_processors=[- MetadataTagProcessor("before_native"),- StatsRequestProcessor(),- MetadataTagProcessor("after_native"),- ],- response_processors=[- MetadataAssertResponseProcessor(- {- "before_native": True,- "after_native": True,- }- )- ],- )-- result = await chain.call(make_request())-- assert result["choices"][0]["message"]["content"] == "base"-- async def test_backend_adapter_restores_context_after_error(self):- """Context mutations made before Python backend failure are not lost."""- chain = self._make_chain(backend=FailingBackend())- ctx = ProxyContext()-- with pytest.raises(RuntimeError, match="backend exploded"):- await chain.call(make_request(model="gpt-4o"), ctx=ctx)-- assert ctx.metadata["backend_started"] == "gpt-4o"-- async def test_request_processor_adapter_restores_context_after_error(self):- chain = self._make_chain(request_processors=[FailingRequestProcessor()])- ctx = ProxyContext()-- with pytest.raises(RuntimeError, match="request processor exploded"):- await chain.call(make_request(model="gpt-4o"), ctx=ctx)-- assert ctx.metadata["request_started"] == "gpt-4o"-- async def test_response_processor_invalid_return_restores_context(self):- chain = self._make_chain(response_processors=[InvalidResponseProcessor()])- ctx = ProxyContext()-- with pytest.raises(RuntimeError, match="ChatResponse"):- await chain.call(make_request(model="gpt-4o"), ctx=ctx)-- assert ctx.metadata["response_started"] is True-- async def test_concurrent_calls_do_not_share_context_metadata(self):- chain = self._make_chain(- request_processors=[ModelMetadataProcessor()],- response_processors=[ModelMetadataAssertProcessor()],- )-- results = await asyncio.gather(- *(chain.call(make_request(model=f"model-{index}")) for index in range(20))- )-- assert {result["model"] for result in results} == {- f"model-{index}" for index in range(20)- }-- async def test_streaming_response_survives_compatibility_executor_boundary(self):- chain = self._make_chain(backend=StreamingBackend())-- stream = await chain.call(make_request())- events = [event async for event in stream]-- assert events[0]["choices"][0]["delta"]["content"] == "hello"-- async def test_streaming_callbacks_can_read_context_after_executor_returns(self):- processor = ContextReadingStreamTapProcessor()- chain = self._make_chain(- backend=StreamingBackend(),- response_processors=[processor],- )-- stream = await chain.call(make_request())- _ = [event async for event in stream]-- assert processor.seen_models == ["stream-selected"]-- async def test_context_overflow_evicts_target_and_retries_fallback(self):- backend = OverflowWeakBackend()- chain = self._make_chain(- request_processors=[PickWeakProcessor()],- backend=backend,- fallback_target_on_evict="strong",- )- ctx = ProxyContext()-- result = await chain.call(make_request(model="client-model"), ctx=ctx)-- assert result["choices"][0]["message"]["content"] == "fallback"- assert backend.calls == ["weak", "strong"]- assert ctx.selected_target == "strong"- assert ctx.evicted_targets == ["weak"]-- async def test_context_overflow_exception_target_retries_fallback_when_context_unset(self):- backend = ExceptionOnlyOverflowBackend()- chain = self._make_chain(- backend=backend,- fallback_target_on_evict="strong",- )- ctx = ProxyContext()-- result = await chain.call(make_request(model="client-model"), ctx=ctx)-- assert result["choices"][0]["message"]["content"] == "fallback"- assert backend.calls == [None, "strong"]- assert ctx.selected_target == "strong"- assert ctx.evicted_targets == ["weak"]-- async def test_empty_processors_ok(self):- chain = self._make_chain(- request_processors=[],- response_processors=[],- )- result = await chain.call(make_request())- assert result["choices"][0]["message"]["content"] == "base"diff --git a/tests/test_switchyard_app_factory.py b/tests/test_switchyard_app_factory.pydeleted file mode 100644index e77d1e244..000000000--- a/tests/test_switchyard_app_factory.py+++ /dev/null@@ -1,88 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Tests for the FastAPI app factory wiring."""--from __future__ import annotations--from typing import Protocol--from fastapi import FastAPI-from fastapi.testclient import TestClient--from switchyard.lib.endpoints.base import Endpoint-from switchyard.server.switchyard_app import build_switchyard_app---class _RequestWithBody(Protocol):- body: dict[str, object]---class _RecordingSwitchyard:- def __init__(self) -> None:- self.requests: list[_RequestWithBody] = []-- async def call(- self,- request: _RequestWithBody,- *,- ctx: object | None = None,- ) -> dict[str, object]:- self.requests.append(request)- return {- "id": "resp-test",- "object": "response",- "model": request.body["model"],- "output": [],- }---class _MarkerEndpoint(Endpoint):- def register(self, app: FastAPI) -> None:- @app.get("/marker")- async def marker() -> dict[str, str]:- return {"status": "ok"}---class _EndpointContributor:- def get_endpoint(self) -> Endpoint:- return _MarkerEndpoint()---class _SwitchyardWithComponents(_RecordingSwitchyard):- def iter_components(self) -> list[_EndpointContributor]:- return [_EndpointContributor()]---def test_app_exposes_switchyard_under_endpoint_state_key() -> None:- switchyard = _RecordingSwitchyard()-- app = build_switchyard_app(switchyard) # type: ignore[arg-type]-- assert app.state.switchyard is switchyard- assert app.state.switchyard is switchyard---def test_responses_endpoint_uses_app_factory_switchyard() -> None:- switchyard = _RecordingSwitchyard()- app = build_switchyard_app(switchyard) # type: ignore[arg-type]-- with TestClient(app, raise_server_exceptions=False) as client:- response = client.post(- "/v1/responses",- json={"model": "test-model", "input": "ping"},- )-- assert response.status_code == 200- assert response.json()["model"] == "test-model"- assert len(switchyard.requests) == 1---def test_app_registers_component_contributed_endpoints() -> None:- app = build_switchyard_app(_SwitchyardWithComponents()) # type: ignore[arg-type]-- with TestClient(app, raise_server_exceptions=False) as client:- response = client.get("/marker")-- assert response.status_code == 200- assert response.json() == {"status": "ok"}diff --git a/tests/test_switchyard_app_lifecycle.py b/tests/test_switchyard_app_lifecycle.pydeleted file mode 100644index df73ad5f5..000000000--- a/tests/test_switchyard_app_lifecycle.py+++ /dev/null@@ -1,49 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Tests for app-level component lifecycle wiring."""--from __future__ import annotations--from fastapi.testclient import TestClient--from switchyard.server.switchyard_app import build_switchyard_app---class _LifecycleSwitchyard:- state_key = "switchyard"-- def __init__(self) -> None:- self.events: list[str] = []-- def iter_components(self) -> list[object]:- return [_AsyncLifecycleComponent(self.events), _SyncLifecycleComponent(self.events)]---class _AsyncLifecycleComponent:- def __init__(self, events: list[str]) -> None:- self._events = events-- async def startup(self) -> None:- self._events.append("async-startup")-- async def shutdown(self) -> None:- self._events.append("async-shutdown")---class _SyncLifecycleComponent:- def __init__(self, events: list[str]) -> None:- self._events = events-- def shutdown(self) -> None:- self._events.append("sync-shutdown")---def test_build_switchyard_app_runs_component_lifecycle() -> None:- switchyard = _LifecycleSwitchyard()- app = build_switchyard_app(switchyard) # type: ignore[arg-type]-- with TestClient(app):- assert switchyard.events == ["async-startup"]-- assert switchyard.events == ["async-startup", "sync-shutdown", "async-shutdown"]diff --git a/tests/test_switchyard_rust_component_bindings.py b/tests/test_switchyard_rust_component_bindings.pydeleted file mode 100644index b85dcec4d..000000000--- a/tests/test_switchyard_rust_component_bindings.py+++ /dev/null@@ -1,233 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Tests for direct Rust component bindings."""--from __future__ import annotations--import math--import pytest--from switchyard_rust import (- AnthropicNativeBackend,- BackendFormat,- ChatRequest,- ChatRequestType,- ChatResponse,- EndpointConfig,- LLMBackend,- LlmTarget,- LlmTargetBackend,- MultiLlmBackend,- OpenAiNativeBackend,- ProxyContext,- RandomRoutingProcessorConfig,- StatsAccumulator,- StatsLlmBackend,- StatsRequestProcessor,- StatsResponseProcessor,-)---def _target(- target_id: str,- model: str,- *,- format: object = BackendFormat.OPENAI,-) -> LlmTarget:- return LlmTarget(- target_id,- model,- format=format,- endpoint=EndpointConfig(base_url="https://example.test/v1", api_key="test-key"),- )---def test_component_exports_are_callable_processors_and_native_backends() -> None:- openai_target = _target("openai", "gpt-test", format=BackendFormat.OPENAI)- anthropic_target = _target("anthropic", "claude-test", format=BackendFormat.ANTHROPIC)-- assert callable(StatsRequestProcessor().process)- assert callable(StatsResponseProcessor(StatsAccumulator()).process)- assert isinstance(OpenAiNativeBackend(openai_target), LLMBackend)- assert isinstance(AnthropicNativeBackend(anthropic_target), LLMBackend)---def test_config_bindings_validate_and_own_values() -> None:- endpoint = EndpointConfig(base_url="https://example.test/v1", api_key="secret", timeout_secs=3.5)- target = LlmTarget(- "target-a",- "model-a",- format="openai",- endpoint=endpoint,- extra_body={"chat_template_kwargs": {"enable_thinking": False}},- extra_headers={"X-Inference-Priority": "batch"},- )-- assert BackendFormat("openai") == BackendFormat.OPENAI- assert BackendFormat.OPENAI == "openai"- assert BackendFormat("responses") == BackendFormat.RESPONSES- assert BackendFormat.RESPONSES == "responses"- assert target.id == "target-a"- assert target.model == "model-a"- assert target.format == BackendFormat.OPENAI- assert target.endpoint.to_dict() == {- "api_key": "secret",- "base_url": "https://example.test/v1",- "timeout_secs": 3.5,- }- assert target.extra_body == {"chat_template_kwargs": {"enable_thinking": False}}- assert target.extra_headers == {"X-Inference-Priority": "batch"}- with pytest.raises(ValueError, match="Unknown backend format"):- BackendFormat("bedrock")- with pytest.raises(ValueError, match="must not be empty"):- LlmTarget(" ", "model-a")- with pytest.raises(ValueError, match="requires a model string"):- LlmTarget(id="target-a")- with pytest.raises(RuntimeError, match="strong_probability"):- RandomRoutingProcessorConfig(- _target("strong", "strong-model"),- _target("weak", "weak-model"),- strong_probability=math.nan,- )---async def test_stats_processors_share_rust_accumulator() -> None:- stats = StatsAccumulator()- request_processor = StatsRequestProcessor()- response_processor = StatsResponseProcessor(stats)- ctx = ProxyContext()- request = ChatRequest.openai_chat({"model": "client-model", "messages": []})-- processed = await request_processor.process(ctx, request)- ctx.selected_model = "served-model"- response = await response_processor.process(- ctx,- ChatResponse.openai_completion({- "model": "served-model",- "usage": {- "prompt_tokens": 11,- "completion_tokens": 7,- "prompt_tokens_details": {- "cached_tokens": 3,- "cache_creation_tokens": 2,- },- "completion_tokens_details": {"reasoning_tokens": 5},- },- }),- )-- assert processed.model == "client-model"- assert response.body["model"] == "served-model"- snapshot = stats.snapshot_sync()- model_stats = snapshot["models"]["served-model"]- assert model_stats["prompt_tokens"] == 11- assert model_stats["completion_tokens"] == 7- assert model_stats["cached_tokens"] == 3- assert model_stats["cache_creation_tokens"] == 2- assert model_stats["reasoning_tokens"] == 5- assert model_stats["total_latency"]["count"] == 1---async def test_stream_callbacks_survive_handoff_to_rust_response_processor() -> None:- async def source():- yield {"choices": [{"delta": {"content": "hi"}}]}- yield {- "choices": [{"delta": {}, "finish_reason": "stop"}],- "usage": {"prompt_tokens": 1, "completion_tokens": 2},- }-- tapped: list[dict[str, object]] = []- mapped: list[dict[str, object]] = []- completed = False-- async def tap(event: dict[str, object]) -> None:- tapped.append(dict(event))-- async def map_event(event: dict[str, object]) -> dict[str, object]:- mapped.append(dict(event))- return {**event, "mapped": True}-- async def on_complete() -> None:- nonlocal completed- completed = True-- stats = StatsAccumulator()- ctx = ProxyContext()- ctx.selected_model = "served-model"- response = ChatResponse.openai_stream(source())- response.stream.tap(tap).map(map_event).on_complete(on_complete)-- processed = await StatsResponseProcessor(stats).process(ctx, response)- events = [event async for event in processed.stream]-- assert [event["mapped"] for event in events] == [True, True]- assert len(tapped) == 2- assert len(mapped) == 2- assert completed is True- assert stats.snapshot_sync()["models"]["served-model"]["prompt_tokens"] == 1---def test_backend_bindings_construct_without_provider_sdks_or_network() -> None:- openai_target = _target("openai", "gpt-test", format=BackendFormat.OPENAI)- responses_target = _target("responses", "gpt-responses", format=BackendFormat.RESPONSES)- anthropic_target = _target("anthropic", "claude-test", format=BackendFormat.ANTHROPIC)- openai = OpenAiNativeBackend(openai_target)- responses = OpenAiNativeBackend(responses_target)- anthropic = AnthropicNativeBackend(anthropic_target)- stats = StatsAccumulator()-- multi = MultiLlmBackend([- LlmTargetBackend(openai_target, openai),- (anthropic_target, anthropic),- ], default_target_id="openai")- stats_backend = StatsLlmBackend(openai, stats)-- assert [request_type.value for request_type in openai.supported_request_types] == [- "openai_chat"- ]- assert [request_type.value for request_type in responses.supported_request_types] == [- "openai_responses"- ]- assert [request_type.value for request_type in anthropic.supported_request_types] == [- "anthropic"- ]- assert set(multi.target_ids()) == {"openai", "anthropic"}- assert multi.default_target_id() == "openai"- assert stats_backend.supported_request_types == openai.supported_request_types---def test_backend_bindings_reject_invalid_native_composition() -> None:- openai_target = _target("openai", "gpt-test", format=BackendFormat.OPENAI)- openai = OpenAiNativeBackend(openai_target)-- with pytest.raises(RuntimeError, match="requires a target with resolved OpenAI format"):- OpenAiNativeBackend(_target("bad", "claude-test", format=BackendFormat.ANTHROPIC))- with pytest.raises(RuntimeError, match="requires a target with resolved Anthropic format"):- AnthropicNativeBackend(_target("bad", "gpt-test", format=BackendFormat.OPENAI))- with pytest.raises(RuntimeError, match="duplicate LLM target id"):- MultiLlmBackend([- LlmTargetBackend(openai_target, openai),- LlmTargetBackend(openai_target, openai),- ])- with pytest.raises(RuntimeError, match="default target missing is not configured"):- MultiLlmBackend(- [LlmTargetBackend(openai_target, openai)],- default_target_id="missing",- )- with pytest.raises(RuntimeError, match="at least one request type"):- MultiLlmBackend([LlmTargetBackend(openai_target, openai)], supported_request_types=[])---def test_wrappers_require_rust_native_backend_instances() -> None:- class PythonOnlyBackend(LLMBackend):- @property- def supported_request_types(self) -> list[ChatRequestType]:- return [ChatRequestType.OPENAI_CHAT]-- async def call(self, ctx: ProxyContext, request: ChatRequest) -> ChatResponse:- return ChatResponse.openai_completion({"model": request.model})-- with pytest.raises(TypeError):- StatsLlmBackend(PythonOnlyBackend(), StatsAccumulator())diff --git a/tests/test_switchyard_rust_core_bindings.py b/tests/test_switchyard_rust_core_bindings.pydeleted file mode 100644index 3ad6f1a72..000000000--- a/tests/test_switchyard_rust_core_bindings.py+++ /dev/null@@ -1,273 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Tests for Python core values and their native component interop."""--from __future__ import annotations--import pytest--from switchyard.lib.chat_response import (- AnthropicResponseStream,- ResponsesApiStream,- ResponseStream,-)-from switchyard_rust import (- ChatRequest,- ChatRequestType,- ChatResponse,- ChatResponseStream,- ChatResponseType,- LLMBackend,- ProxyContext,- ProxyMetadata,-)---def test_openai_chat_request_owns_body_and_exposes_model() -> None:- body = {- "model": "gpt-4o",- "messages": [{"role": "user", "content": "hello"}],- }-- request = ChatRequest.openai_chat(body)-- assert request.request_type == ChatRequestType.OPENAI_CHAT- assert request.request_type is ChatRequestType.OPENAI_CHAT- assert hash(request.request_type) == hash(ChatRequestType.OPENAI_CHAT)- assert request.request_type.value == "openai_chat"- assert request.model == "gpt-4o"- assert request.body == body- assert request.body is not body-- body["model"] = "mutated-after-construction"- assert request.model == "gpt-4o"---def test_request_constructors_preserve_wire_format() -> None:- responses = ChatRequest.openai_responses({"model": "gpt-4o", "input": "hello"})- anthropic = ChatRequest.anthropic({"model": "claude-sonnet-4.5", "messages": []})-- assert responses.request_type == ChatRequestType.OPENAI_RESPONSES- assert responses.request_type.value == "openai_responses"- assert responses.model == "gpt-4o"- assert anthropic.request_type == ChatRequestType.ANTHROPIC- assert anthropic.request_type.value == "anthropic"- assert anthropic.model == "claude-sonnet-4.5"---def test_set_model_mutates_owned_body() -> None:- request = ChatRequest.openai_chat({"model": "old", "messages": []})-- request.set_model("new")-- assert request.model == "new"- assert request.to_body() == {"model": "new", "messages": []}---def test_set_model_recovers_malformed_non_object_body() -> None:- request = ChatRequest.anthropic(["not", "an", "object"])-- request.set_model("claude-sonnet-4.5")-- assert request.model == "claude-sonnet-4.5"- assert request.body == {"model": "claude-sonnet-4.5"}---def test_replace_body_preserves_request_type() -> None:- request = ChatRequest.openai_responses({"model": "old", "input": "hello"})-- request.replace_body({"model": "new", "input": "replacement"})-- assert request.request_type == ChatRequestType.OPENAI_RESPONSES- assert request.request_type.value == "openai_responses"- assert request.model == "new"- assert request.body == {"model": "new", "input": "replacement"}---def test_non_json_body_is_rejected() -> None:- class NonJsonable:- pass-- with pytest.raises(ValueError):- ChatRequest.openai_chat(NonJsonable())---def test_openai_completion_response_owns_body() -> None:- body = {- "id": "chatcmpl-test",- "model": "gpt-4o",- "choices": [{"message": {"role": "assistant", "content": "hello"}}],- }-- response = ChatResponse.openai_completion(body)-- assert response.response_type == ChatResponseType.OPENAI_COMPLETION- assert response.response_type is ChatResponseType.OPENAI_COMPLETION- assert hash(response.response_type) == hash(ChatResponseType.OPENAI_COMPLETION)- assert response.response_type.value == "openai_completion"- assert response.body == body- assert response.body is not body-- body["model"] = "mutated-after-construction"- assert response.body["model"] == "gpt-4o"---def test_buffered_response_preserves_json_null_body() -> None:- response = ChatResponse.openai_completion(None)-- assert response.body is None- assert response.to_body() is None---def test_response_constructors_preserve_wire_shape() -> None:- responses = ChatResponse.openai_responses_completion({- "id": "resp-test",- "model": "gpt-4o",- "output": [],- })- anthropic = ChatResponse.anthropic_completion({- "id": "msg-test",- "model": "claude-sonnet-4.5",- "content": [],- })-- assert responses.response_type == ChatResponseType.OPENAI_RESPONSES_COMPLETION- assert responses.response_type.value == "openai_responses_completion"- assert anthropic.response_type == ChatResponseType.ANTHROPIC_COMPLETION- assert anthropic.response_type.value == "anthropic_completion"---async def test_stream_response_uses_owned_async_stream_and_rejects_body_access() -> None:- async def source():- yield {"delta": "hello"}-- response = ChatResponse.openai_stream(source())-- assert response.response_type == ChatResponseType.OPENAI_STREAM- assert [event async for event in response.stream] == [{"delta": "hello"}]- with pytest.raises(AttributeError):- _ = response.body- with pytest.raises(ValueError):- response.replace_body({"not": "allowed"})---def test_stream_replace_body_rejects_before_serializing_body() -> None:- class ExplodingBody:- def model_dump(self, **kwargs: object) -> object:- raise RuntimeError("should not serialize streaming replacement")-- response = ChatResponse.openai_stream(object())-- with pytest.raises(ValueError, match="streaming ChatResponse"):- response.replace_body(ExplodingBody())---async def test_stream_response_supports_taps_maps_and_completion_callbacks() -> None:- async def source():- yield {"index": 0}- yield {"index": 1}-- tapped: list[dict[str, int]] = []- completed = False-- async def tap(event: dict[str, int]) -> None:- tapped.append(dict(event))-- async def map_event(event: dict[str, int]) -> dict[str, int]:- return {"index": event["index"] + 10}-- async def on_complete() -> None:- nonlocal completed- completed = True-- response = ChatResponse.openai_stream(source())- stream = response.stream.tap(tap).map(map_event).on_complete(on_complete)-- assert [event async for event in stream] == [{"index": 10}, {"index": 11}]- assert tapped == [{"index": 0}, {"index": 1}]- assert completed is True-- with pytest.raises(RuntimeError, match="already been consumed"):- _ = [event async for event in stream]---def test_proxy_context_uses_python_wrappers_with_shared_native_state() -> None:- metadata = {"request_id": "client-visible", "nested": {"value": 1}}-- ctx = ProxyContext(metadata=metadata, request_id="rust-request")-- assert ctx.request_id == "rust-request"- assert isinstance(ctx.metadata, ProxyMetadata)- assert ctx.metadata is ctx.metadata- assert ctx.metadata == metadata- assert ctx.metadata is not metadata-- ctx.metadata["new"] = "value"- ctx.metadata.setdefault("order", []).append("first")- ctx.metadata.update({"updated": True})- ctx.selected_model = "model-a"- ctx.selected_target = "target-a"- ctx.inbound_format = ChatRequestType.OPENAI_CHAT- ctx.backend_call_latency_ms = 42.5-- assert ctx.metadata["new"] == "value"- assert ctx.metadata["order"] == ["first"]- assert ctx.metadata.get("missing", "fallback") == "fallback"- assert ctx.metadata.copy()["updated"] is True- assert sorted(ctx.metadata.keys()) == ["nested", "new", "order", "request_id", "updated"]- del ctx.metadata["updated"]- assert "updated" not in ctx.metadata- assert ctx.selected_model == "model-a"- assert ctx.selected_target == "target-a"- assert ctx.inbound_format == ChatRequestType.OPENAI_CHAT- assert ctx.backend_call_latency_ms == 42.5-- ctx.backend_call_latency_ms = None- assert ctx.backend_call_latency_ms is None---def test_proxy_context_evicted_targets_are_mutable_from_python() -> None:- ctx = ProxyContext()-- assert ctx.evicted_targets is None- ctx.evicted_targets = ["weak", "strong"]-- assert ctx.evicted_targets == ["strong", "weak"]-- ctx.evicted_targets = None- assert ctx.evicted_targets is None- with pytest.raises(ValueError, match="invalid evicted target"):- ctx.evicted_targets = [" "]---def test_provider_stream_adapters_are_rust_chat_response_stream_aliases() -> None:- assert ResponseStream is ChatResponseStream- assert ResponsesApiStream is ChatResponseStream- assert AnthropicResponseStream is ChatResponseStream---def test_backend_role_class_is_the_public_python_export() -> None:- from switchyard.lib.roles import LLMBackend as PublicLLMBackend-- assert PublicLLMBackend is LLMBackend---async def test_request_response_components_are_method_based() -> None:- class Passthrough:- async def process(self, ctx: ProxyContext, request: ChatRequest) -> ChatRequest:- return request-- processor = Passthrough()- request = ChatRequest.openai_chat({"model": "gpt-4o", "messages": []})-- assert await processor.process(ProxyContext(), request) is request---def test_replace_body_preserves_response_type() -> None:- response = ChatResponse.anthropic_completion({"model": "old"})-- response.replace_body({"model": "new", "content": []})-- assert response.response_type == ChatResponseType.ANTHROPIC_COMPLETION- assert response.body == {"model": "new", "content": []}diff --git a/tests/test_telemetry.py b/tests/test_telemetry.pydeleted file mode 100644index d2238af79..000000000--- a/tests/test_telemetry.py+++ /dev/null@@ -1,123 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Tests for outbound telemetry version headers."""--from __future__ import annotations--import importlib.metadata-from typing import Any-from unittest.mock import patch--import pytest--from switchyard.telemetry import (- HEADER_NAME,- LEGACY_OPT_OUT_ENVVAR,- OPT_OUT_ENVVAR,- _get_version,- get_telemetry_headers,-)--[email protected](autouse=True)-def _reset_telemetry_env(monkeypatch: pytest.MonkeyPatch) -> None:- monkeypatch.delenv(OPT_OUT_ENVVAR, raising=False)- monkeypatch.delenv(LEGACY_OPT_OUT_ENVVAR, raising=False)- _get_version.cache_clear()- yield- _get_version.cache_clear()---def test_get_telemetry_headers_uses_distribution_package_version(- monkeypatch: pytest.MonkeyPatch,-) -> None:- package_names: list[str] = []-- def fake_version(package_name: str) -> str:- package_names.append(package_name)- return "1.2.3"-- monkeypatch.setattr(importlib.metadata, "version", fake_version)-- assert get_telemetry_headers() == {HEADER_NAME: "1.2.3"}- assert package_names == ["nemo-switchyard"]--[email protected]("envvar", [OPT_OUT_ENVVAR, LEGACY_OPT_OUT_ENVVAR])[email protected]("value", ["1", "true", "yes", "TRUE", "on"])-def test_get_telemetry_headers_respects_opt_out_envvars(- monkeypatch: pytest.MonkeyPatch,- envvar: str,- value: str,-) -> None:- monkeypatch.setenv(envvar, value)-- assert get_telemetry_headers() == {}--[email protected]("value", ["", "0", "false", "no", " FALSE "])-def test_get_telemetry_headers_ignores_falsey_opt_out_values(- monkeypatch: pytest.MonkeyPatch,- value: str,-) -> None:- monkeypatch.setattr(importlib.metadata, "version", lambda _name: "9.8.7")- monkeypatch.setenv(OPT_OUT_ENVVAR, value)-- assert get_telemetry_headers() == {HEADER_NAME: "9.8.7"}---def test_get_telemetry_headers_falls_back_to_unknown(- monkeypatch: pytest.MonkeyPatch,-) -> None:- def raise_package_not_found(_package_name: str) -> str:- raise importlib.metadata.PackageNotFoundError-- monkeypatch.setattr(importlib.metadata, "version", raise_package_not_found)-- assert get_telemetry_headers() == {HEADER_NAME: "unknown"}---def test_openai_llm_client_passes_default_headers(- monkeypatch: pytest.MonkeyPatch,-) -> None:- monkeypatch.setattr(importlib.metadata, "version", lambda _name: "1.2.3")- captured: list[dict[str, Any]] = []-- class FakeAsyncOpenAI:- def __init__(self, **kwargs: Any) -> None:- captured.append(kwargs)-- with patch("switchyard.lib.llm_client.AsyncOpenAI", FakeAsyncOpenAI):- from switchyard.lib.llm_client import OpenAILLMClient-- OpenAILLMClient(api_key="sk-test", base_url="https://llm.test/v1", timeout=3.0)-- assert captured == [- {- "api_key": "sk-test",- "base_url": "https://llm.test/v1",- "timeout": 3.0,- "default_headers": {HEADER_NAME: "1.2.3"},- },- ]---def test_openai_llm_client_passes_empty_default_headers_when_opted_out(- monkeypatch: pytest.MonkeyPatch,-) -> None:- monkeypatch.setenv(OPT_OUT_ENVVAR, "1")- captured: list[dict[str, Any]] = []-- class FakeAsyncOpenAI:- def __init__(self, **kwargs: Any) -> None:- captured.append(kwargs)-- with patch("switchyard.lib.llm_client.AsyncOpenAI", FakeAsyncOpenAI):- from switchyard.lib.llm_client import OpenAILLMClient-- OpenAILLMClient(api_key="sk-test")-- assert captured == [- {"api_key": "sk-test", "default_headers": {}},- ]diff --git a/tests/test_tool_result_signal_collector.py b/tests/test_tool_result_signal_collector.pydeleted file mode 100644index 5c5846478..000000000--- a/tests/test_tool_result_signal_collector.py+++ /dev/null@@ -1,192 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Tests for tool-result signal extraction now built into DimensionCollector."""--from __future__ import annotations--import pytest--from switchyard_rust.components import DimensionCollector, get_tool_result_signal-from switchyard_rust.core import ChatRequest, ProxyContext--# ─── unit: classify_text (via Rust) ──────────────────────────────────────────-# DimensionCollector is Rust; test the logic through the full processor path.---async def _run_collector(body: dict, fmt: str = "openai_chat") -> ProxyContext:- """Run DimensionCollector.process() and return the populated context."""- collector = DimensionCollector()- if fmt == "anthropic":- request = ChatRequest.anthropic(body)- elif fmt == "openai_responses":- request = ChatRequest.openai_responses(body)- else:- request = ChatRequest.openai_chat(body)- ctx = ProxyContext()- await collector.process(ctx, request)- return ctx---# ─── severity tests via DimensionCollector ────────────────────────────────────---async def test_traceback_stamps_hard_severity():- ctx = await _run_collector({- "messages": [- {"role": "user", "content": "do something"},- {"role": "tool", "tool_call_id": "1",- "content": "Traceback (most recent call last):\n ValueError"},- ]- })- signal = get_tool_result_signal(ctx)- assert signal is not None- assert signal.severity == pytest.approx(0.7)---async def test_oom_stamps_critical_severity():- ctx = await _run_collector({- "messages": [- {"role": "tool", "tool_call_id": "1", "content": "Out of memory: kill process"},- ]- })- signal = get_tool_result_signal(ctx)- assert signal is not None- assert signal.severity == pytest.approx(1.0)---async def test_clean_result_has_zero_severity():- ctx = await _run_collector({- "messages": [- {"role": "tool", "tool_call_id": "1", "content": "file written successfully"},- ]- })- signal = get_tool_result_signal(ctx)- assert signal is not None- assert signal.severity == pytest.approx(0.0)---async def test_no_tool_results_has_zero_severity():- ctx = await _run_collector({- "messages": [{"role": "user", "content": "hello"}]- })- signal = get_tool_result_signal(ctx)- assert signal is not None- assert signal.severity == pytest.approx(0.0)---# ─── conversation metrics ─────────────────────────────────────────────────────---async def test_edit_and_write_counts():- ctx = await _run_collector({- "messages": [- {"role": "assistant", "tool_calls": [- {"function": {"name": "Edit", "arguments": "{}"}},- {"function": {"name": "Edit", "arguments": "{}"}},- {"function": {"name": "Write", "arguments": "{}"}},- ]},- {"role": "tool", "tool_call_id": "1", "content": "ok"},- ]- })- signal = get_tool_result_signal(ctx)- assert signal is not None- assert signal.edit_count == 2- assert signal.write_count == 1---async def test_no_error_streak_all_clean():- ctx = await _run_collector({- "messages": [- {"role": "tool", "tool_call_id": "1", "content": "ok"},- {"role": "tool", "tool_call_id": "2", "content": "also ok"},- ]- })- signal = get_tool_result_signal(ctx)- assert signal is not None- assert signal.no_error_streak == 2---async def test_no_error_streak_stops_at_error():- ctx = await _run_collector({- "messages": [- {"role": "tool", "tool_call_id": "1",- "content": "Traceback (most recent call last):\n ValueError"},- {"role": "tool", "tool_call_id": "2", "content": "ok"},- {"role": "tool", "tool_call_id": "3", "content": "ok"},- ]- })- signal = get_tool_result_signal(ctx)- assert signal is not None- assert signal.no_error_streak == 2---async def test_tests_passed_detection():- ctx = await _run_collector({- "messages": [- {"role": "tool", "tool_call_id": "1",- "content": "====== 5 passed in 0.3s ======"},- ]- })- signal = get_tool_result_signal(ctx)- assert signal is not None- assert signal.tests_passed is True---async def test_tests_passed_false_when_failures_present():- ctx = await _run_collector({- "messages": [- {"role": "tool", "tool_call_id": "1",- "content": "2 failed, 3 passed in 0.5s"},- ]- })- signal = get_tool_result_signal(ctx)- assert signal is not None- assert signal.tests_passed is False---async def test_turn_depth_matches_message_count():- ctx = await _run_collector({- "messages": [- {"role": "user", "content": "step 1"},- {"role": "assistant", "content": "ok"},- {"role": "tool", "tool_call_id": "1", "content": "done"},- ]- })- signal = get_tool_result_signal(ctx)- assert signal is not None- assert signal.turn_depth == 3---# ─── Anthropic format ─────────────────────────────────────────────────────────---async def test_anthropic_tool_result_extracted():- ctx = await _run_collector({- "messages": [- {"role": "user", "content": [- {"type": "tool_result", "tool_use_id": "1",- "content": "Traceback (most recent call last):\n ImportError:"}- ]}- ]- }, fmt="anthropic")- signal = get_tool_result_signal(ctx)- assert signal is not None- assert signal.severity == pytest.approx(0.7)---# ─── OpenAI Responses format ──────────────────────────────────────────────────---async def test_responses_api_tool_output_extracted():- ctx = await _run_collector({- "input": [- {"type": "function_call", "name": "Write"},- {"type": "function_call_output", "call_id": "1",- "output": "file created"},- ]- }, fmt="openai_responses")- signal = get_tool_result_signal(ctx)- assert signal is not None- assert signal.severity == pytest.approx(0.0)- assert signal.write_count == 1diff --git a/tests/test_tracing.py b/tests/test_tracing.pydeleted file mode 100644index bb7f8a72d..000000000--- a/tests/test_tracing.py+++ /dev/null@@ -1,59 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Unit tests for the optional ddtrace wrapper in :mod:`switchyard.lib.tracing`."""--from __future__ import annotations--from typing import Any--import pytest--from switchyard.lib import tracing---class _FakeSpan:- def __init__(self) -> None:- self.tags: dict[str, Any] = {}-- def set_tag(self, key: str, value: Any) -> None:- self.tags[key] = value-- def __enter__(self) -> _FakeSpan:- return self-- def __exit__(self, *_args: object) -> bool:- return False---class _FakeTracer:- def __init__(self) -> None:- self.started: list[tuple[str, _FakeSpan]] = []-- def trace(self, name: str) -> _FakeSpan:- span = _FakeSpan()- self.started.append((name, span))- return span---def test_routing_span_noops_without_tracer(monkeypatch: pytest.MonkeyPatch) -> None:- """With no tracer, the context manager still yields a usable no-op span."""- monkeypatch.setattr(tracing, "_dd_tracer", None)- with tracing.routing_span("switchyard.route_decision") as span:- span.set_tag("switchyard.model", "m") # must not raise---def test_routing_span_uses_tracer_when_present(monkeypatch: pytest.MonkeyPatch) -> None:- tracer = _FakeTracer()- monkeypatch.setattr(tracing, "_dd_tracer", tracer)- with tracing.routing_span("switchyard.upstream_attempt") as span:- span.set_tag("switchyard.selected_endpoint", "model-A")- assert tracer.started[0][0] == "switchyard.upstream_attempt"- assert tracer.started[0][1].tags == {"switchyard.selected_endpoint": "model-A"}---def test_set_tags_skips_none() -> None:- span = _FakeSpan()- tracing.set_tags(span, {"a": 1, "b": None, "c": "x", "d": False})- # ``None`` is dropped; falsey-but-meaningful values (0, False, "") are kept.- assert span.tags == {"a": 1, "c": "x", "d": False}diff --git a/tests/test_translation_engine_chaos.py b/tests/test_translation_engine_chaos.pydeleted file mode 100644index dfd133a36..000000000--- a/tests/test_translation_engine_chaos.py+++ /dev/null@@ -1,2064 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Adversarial / edge-case tests for ChatRequest and ChatResponse translation engines.--Every test targets a specific boundary condition, failure mode, or surprising-interaction that the happy-path tests in test_request_translation_engine.py and-test_response_translation_engine.py do not cover. Nothing here is hypothetical--- each case was verified to exercise real code paths.-"""--from __future__ import annotations--import asyncio-import copy-import json-from dataclasses import dataclass-from typing import Any-from unittest.mock import MagicMock--import pytest--from switchyard.lib.chat_response.openai_chat import ResponseStream-from switchyard_rust.core import ChatRequest, ChatRequestType, ChatResponse, request_type_matches-from switchyard_rust.translation import TranslationEngine--# ----------------------------------------------------------------------------# Helpers-# -----------------------------------------------------------------------------def req_engine():- return TranslationEngine()---def resp_engine():- return TranslationEngine()---class _AlienChatRequest:- """A request-shaped object unknown to either engine.-- Used to verify that NotImplementedError is raised with a useful message- instead of silently passing through or crashing.- """-- def __init__(self, body: dict[str, Any] | None = None):- self._body = body or {}-- @property- def request_type(self) -> str:- return "alien"-- @property- def body(self) -> dict[str, Any]:- return self._body---def _make_completion_dict(- content: str | None = "Hello!",- tool_calls: list[dict[str, Any]] | None = None,- finish_reason: str = "stop",- model: str = "gpt-4o",- prompt_tokens: int = 10,- completion_tokens: int = 5,-) -> dict[str, Any]:- """Build a plain-dict ChatCompletion-shaped payload."""- message: dict[str, Any] = {"role": "assistant", "content": content}- if tool_calls is not None:- message["tool_calls"] = tool_calls- return {- "id": "chatcmpl-test",- "model": model,- "choices": [- {"message": message, "finish_reason": finish_reason},- ],- "usage": {- "prompt_tokens": prompt_tokens,- "completion_tokens": completion_tokens,- "total_tokens": prompt_tokens + completion_tokens,- },- }---def _mock_completion(- content: str | None = "Hello!",- tool_calls_mock: list[Any] | None = None,- finish_reason: str = "stop",- model: str = "gpt-4o",- prompt_tokens: int = 10,- completion_tokens: int = 5,-) -> MagicMock:- """Build a MagicMock that quacks like ``ChatCompletion``."""- completion = MagicMock()- completion.choices = [- MagicMock(- message=MagicMock(- content=content,- tool_calls=tool_calls_mock,- refusal=None,- ),- finish_reason=finish_reason,- )- ]- completion.usage = MagicMock(- prompt_tokens=prompt_tokens,- completion_tokens=completion_tokens,- total_tokens=prompt_tokens + completion_tokens,- )- completion.model = model- completion.id = "chatcmpl-test"-- tc_dicts = None- if tool_calls_mock:- tc_dicts = []- for tc in tool_calls_mock:- tc_dicts.append({- "id": tc.id,- "function": {- "name": tc.function.name,- "arguments": tc.function.arguments,- },- })-- completion.model_dump = lambda **kwargs: _make_completion_dict(- content=content,- tool_calls=tc_dicts,- finish_reason=finish_reason,- model=model,- prompt_tokens=prompt_tokens,- completion_tokens=completion_tokens,- )- return completion---def _mock_tool_call(- call_id: str = "call_abc",- name: str = "get_weather",- arguments: str = '{"location":"SF"}',-) -> MagicMock:- tc = MagicMock()- tc.id = call_id- tc.function = MagicMock(name=name, arguments=arguments)- # MagicMock special-cases .name, so set it explicitly- tc.function.name = name- tc.function.arguments = arguments- return tc---# Streaming helpers -----------------------------------------------------------@dataclass-class MockDelta:- content: str | None = None- tool_calls: Any = None- reasoning: str | None = None- reasoning_content: str | None = None---@dataclass-class MockToolCallDelta:- index: int = 0- id: str | None = None- function: Any = None---@dataclass-class MockFunctionDelta:- name: str | None = None- arguments: str | None = None---@dataclass-class MockChoice:- delta: MockDelta | None = None- finish_reason: str | None = None---@dataclass-class MockChunk:- choices: list[MockChoice] | None = None- usage: Any = None---@dataclass-class MockUsage:- prompt_tokens: int = 0- completion_tokens: int = 0- total_tokens: int = 0---async def _async_chunks(*chunks: Any):- for c in chunks:- yield c---# =========================================================================-# REQUEST ENGINE EDGE CASES-# =========================================================================---class TestRequestEngineEdgeCases:- """Edge cases that exercise the TranslationEngine."""-- # -- Empty / minimal bodies -------------------------------------------- def test_anthropic_empty_messages(self, req_engine):- """An Anthropic request with an empty messages list should produce- a valid OpenAI request with no messages (except possibly system).- """- body = {"model": "claude-sonnet-4-20250514", "messages": [], "max_tokens": 100}- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- assert request_type_matches(result, ChatRequestType.OPENAI_CHAT)- assert result.body["messages"] == []-- def test_anthropic_empty_model_string(self, req_engine):- """model="" is falsy -- verify it does NOT get forwarded."""- body = {"model": "", "messages": [{"role": "user", "content": "hi"}], "max_tokens": 100}- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- assert "model" not in result.body-- def test_anthropic_content_none(self, req_engine):- """Content=None in a message should not crash; it maps to empty string."""- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": None}],- "max_tokens": 100,- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- msgs = result.body["messages"]- assert len(msgs) == 1- assert msgs[0]["content"] == ""-- def test_anthropic_content_integer(self, req_engine):- """Content as a non-string, non-list value (int) should be coerced to str."""- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": 42}],- "max_tokens": 100,- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- msgs = result.body["messages"]- assert msgs[0]["content"] == "42"-- def test_anthropic_content_boolean(self, req_engine):- """Content as a boolean -- a type confusion that can happen with- malformed requests. Should not crash.- """- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": True}],- "max_tokens": 100,- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- msgs = result.body["messages"]- # bool is truthy, not a list, not a str => str(True) = "True"- assert msgs[0]["content"] == "True"-- # -- System prompt variants -------------------------------------------- def test_anthropic_structured_system_blocks(self, req_engine):- """Anthropic system as a list of {type:text} blocks should be- concatenated into a single system message while preserving block- boundaries.- """- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- "system": [- {"type": "text", "text": "You are helpful."},- {"type": "text", "text": "Be concise."},- ],- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- system_msgs = [m for m in result.body["messages"] if m["role"] == "system"]- assert len(system_msgs) == 1- assert system_msgs[0]["content"] == "You are helpful.\n\nBe concise."-- def test_anthropic_empty_system_string(self, req_engine):- """system="" is falsy -- should NOT produce a system message."""- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- "system": "",- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- system_msgs = [m for m in result.body["messages"] if m["role"] == "system"]- assert len(system_msgs) == 0-- def test_anthropic_whitespace_only_system(self, req_engine):- """system=" " is truthy -- it WILL produce a system message- (arguably a bug, but this test documents the current behavior).- """- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- "system": " ",- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- system_msgs = [m for m in result.body["messages"] if m["role"] == "system"]- assert len(system_msgs) == 1- assert system_msgs[0]["content"] == " "-- def test_anthropic_system_blocks_with_non_text(self, req_engine):- """Structured system with a non-text block type should silently skip it."""- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- "system": [- {"type": "text", "text": "Be helpful."},- {"type": "image", "source": {"data": "base64..."}},- ],- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- system_msgs = [m for m in result.body["messages"] if m["role"] == "system"]- assert len(system_msgs) == 1- assert system_msgs[0]["content"] == "Be helpful."-- # -- Deeply nested content blocks -------------------------------------- def test_anthropic_mixed_text_tool_use_in_one_message(self, req_engine):- """A single assistant message with both text and tool_use content- blocks should produce ONE assistant message with content + tool_calls.- """- body = {- "model": "claude-sonnet-4-20250514",- "messages": [- {- "role": "assistant",- "content": [- {"type": "text", "text": "Let me look that up."},- {- "type": "tool_use",- "id": "toolu_1",- "name": "search",- "input": {"query": "weather"},- },- ],- },- ],- "max_tokens": 1024,- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- msgs = result.body["messages"]- assert len(msgs) == 1- assert msgs[0]["role"] == "assistant"- assert msgs[0]["content"] == "Let me look that up."- assert len(msgs[0]["tool_calls"]) == 1- assert msgs[0]["tool_calls"][0]["function"]["name"] == "search"-- def test_anthropic_tool_result_with_structured_content(self, req_engine):- """tool_result where content is a list of blocks (text + image)- should flatten text parts and JSON-serialize non-text blocks.- """- body = {- "model": "claude-sonnet-4-20250514",- "messages": [- {- "role": "user",- "content": [- {- "type": "tool_result",- "tool_use_id": "toolu_1",- "content": [- {"type": "text", "text": "Temperature: 72F"},- {- "type": "image",- "source": {- "type": "base64",- "data": "iVBORw...",- },- },- ],- },- ],- },- ],- "max_tokens": 1024,- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- tool_msgs = [m for m in result.body["messages"] if m["role"] == "tool"]- assert len(tool_msgs) == 1- # Text is preserved- assert "Temperature: 72F" in tool_msgs[0]["content"]- # Non-text block is JSON-serialized, not dropped- assert "image" in tool_msgs[0]["content"]-- def test_anthropic_content_list_with_non_dict_items(self, req_engine):- """Content blocks that are not dicts (e.g. raw strings in the list)- should be silently skipped, not crash.- """- body = {- "model": "claude-sonnet-4-20250514",- "messages": [- {- "role": "user",- "content": ["just a string", 123, {"type": "text", "text": "real block"}],- },- ],- "max_tokens": 1024,- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- msgs = result.body["messages"]- assert len(msgs) == 1- assert msgs[0]["content"] == "real block"-- def test_anthropic_empty_content_list(self, req_engine):- """Content as an empty list should produce a message with empty content."""- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": []}],- "max_tokens": 100,- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- msgs = result.body["messages"]- assert len(msgs) == 1- assert msgs[0]["content"] == ""-- # -- Multi-turn conversations ------------------------------------------ def test_twenty_message_conversation(self, req_engine):- """A realistic 20-message conversation with alternating roles,- including system, tool_use, and tool_result, should translate- without loss.- """- messages = []- for i in range(10):- messages.append({"role": "user", "content": f"Question {i}"})- messages.append({"role": "assistant", "content": f"Answer {i}"})- body = {- "model": "claude-sonnet-4-20250514",- "messages": messages,- "max_tokens": 1024,- "system": "You are a test assistant.",- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- # 1 system + 20 conversation messages- assert len(result.body["messages"]) == 21- assert result.body["messages"][0]["role"] == "system"- # Verify message ordering preserved- for i in range(10):- assert result.body["messages"][1 + 2 * i]["content"] == f"Question {i}"- assert result.body["messages"][2 + 2 * i]["content"] == f"Answer {i}"-- def test_multi_turn_with_tool_roundtrip(self, req_engine):- """A conversation with tool_use -> tool_result -> assistant follow-up- should map cleanly to OpenAI assistant(tool_calls) -> tool -> assistant.- """- messages = [- {"role": "user", "content": "What's the weather?"},- {- "role": "assistant",- "content": [- {"type": "text", "text": "Let me check."},- {- "type": "tool_use",- "id": "toolu_1",- "name": "get_weather",- "input": {"location": "SF"},- },- ],- },- {- "role": "user",- "content": [- {- "type": "tool_result",- "tool_use_id": "toolu_1",- "content": "72F and sunny",- },- ],- },- {"role": "assistant", "content": "It's 72F and sunny in SF!"},- ]- body = {- "model": "claude-sonnet-4-20250514",- "messages": messages,- "max_tokens": 1024,- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- roles = [m["role"] for m in result.body["messages"]]- assert roles == ["user", "assistant", "tool", "assistant"]- assert result.body["messages"][1].get("tool_calls") is not None-- # -- Tool edge cases --------------------------------------------------- def test_anthropic_tool_with_empty_input_schema(self, req_engine):- """A tool with input_schema={} should produce parameters={}."""- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- "tools": [- {"name": "noop", "description": "Does nothing", "input_schema": {}},- ],- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- assert result.body["tools"][0]["function"]["parameters"] == {}-- def test_anthropic_tool_with_no_name(self, req_engine):- """A tool definition missing the 'name' key should be dropped."""- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- "tools": [{"description": "mystery tool", "input_schema": {}}],- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- assert "tools" not in result.body-- def test_anthropic_tool_with_unicode_name(self, req_engine):- """Tool names with unicode should pass through without mangling."""- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- "tools": [- {- "name": "recherche_meteo",- "description": "Recherche meteo",- "input_schema": {"type": "object", "properties": {}},- },- ],- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- assert result.body["tools"][0]["function"]["name"] == "recherche_meteo"-- def test_anthropic_tool_with_deeply_nested_schema(self, req_engine):- """A deeply nested JSON schema in input_schema should pass through."""- deep_schema = {- "type": "object",- "properties": {- "config": {- "type": "object",- "properties": {- "nested": {- "type": "object",- "properties": {- "deep": {"type": "string"},- },- },- },- },- },- }- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- "tools": [- {"name": "deep_tool", "description": "d", "input_schema": deep_schema},- ],- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- params = result.body["tools"][0]["function"]["parameters"]- assert params["properties"]["config"]["properties"]["nested"]["properties"]["deep"]["type"] == "string"-- def test_anthropic_tool_use_with_string_input(self, req_engine):- """tool_use block where input is a pre-serialized JSON string- should be used directly without double-encoding.- """- body = {- "model": "claude-sonnet-4-20250514",- "messages": [- {- "role": "assistant",- "content": [- {- "type": "tool_use",- "id": "toolu_1",- "name": "run_cmd",- "input": '{"cmd": "ls -la"}',- },- ],- },- ],- "max_tokens": 1024,- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- args = result.body["messages"][0]["tool_calls"][0]["function"]["arguments"]- # Should be the string itself, NOT json.dumps(string)- assert args == '{"cmd": "ls -la"}'- # Verify it's valid JSON- parsed = json.loads(args)- assert parsed["cmd"] == "ls -la"-- # -- Extra kwargs / pass-through fields -------------------------------- def test_anthropic_extra_kwargs_filtered_by_openai_whitelist(self, req_engine):- """Only fields that exist on OpenAI Chat Completions survive the- conversion — Anthropic-only fields (``thinking``, ``cache_control``,- etc.) must be dropped because the OpenAI SDK raises TypeError on- unknown kwargs. Fields that happen to exist on both APIs- (``metadata``) pass through.- """- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- "metadata": {"user_id": "u123"}, # exists on OpenAI → survives- "thinking": {"type": "enabled", "budget_tokens": 5000}, # Anthropic-only → dropped- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- assert result.body["metadata"] == {"user_id": "u123"}- assert "thinking" not in result.body-- def test_anthropic_stream_flag_preserved(self, req_engine):- """stream=True in the Anthropic body should appear in the OpenAI body."""- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- "stream": True,- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- assert result.body["stream"] is True-- def test_anthropic_stop_sequences_to_stop(self, req_engine):- """stop_sequences should become stop in the OpenAI body."""- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- "stop_sequences": ["\n\nHuman:", "END"],- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- assert result.body["stop"] == ["\n\nHuman:", "END"]-- # -- Mutation safety --------------------------------------------------- def test_anthropic_passthrough_metadata_does_not_share_nested_objects(self, req_engine):- """Native translation serializes passthrough metadata instead of- exposing source object identity in the translated request.- """- inner_tags = ["tag1", "tag2"]- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- "metadata": {"tags": inner_tags},- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)-- result_meta = result.body.get("metadata")- if result_meta is not None:- assert result_meta["tags"] == inner_tags- assert result_meta["tags"] is not inner_tags-- def test_responses_shallow_copy_does_not_mutate_body(self, req_engine):- """Verify the Responses path also shallow-copies."""- body = {"model": "gpt-4o", "input": "hello", "instructions": "Be nice."}- original = copy.deepcopy(body)- req = ChatRequest.openai_responses(body)- req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- assert req.body == original-- # -- Responses API input variants -------------------------------------- def test_responses_empty_input_string(self, req_engine):- """input="" should produce a user message with empty content."""- body = {"model": "gpt-4o", "input": ""}- req = ChatRequest.openai_responses(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- user_msgs = [m for m in result.body["messages"] if m["role"] == "user"]- assert len(user_msgs) == 1- assert user_msgs[0]["content"] == ""-- def test_responses_empty_input_list(self, req_engine):- """input=[] should produce an empty messages list."""- body = {"model": "gpt-4o", "input": []}- req = ChatRequest.openai_responses(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- assert result.body["messages"] == []-- def test_responses_unknown_input_item_type_is_preserved(self, req_engine):- """Unknown input items should survive as valid Chat text content."""- body = {- "model": "gpt-4o",- "input": [- {"type": "message", "role": "user", "content": "hi"},- {"type": "audio_clip", "data": "base64..."}, # unknown- {"type": "message", "role": "assistant", "content": "hello"},- ],- }- req = ChatRequest.openai_responses(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- msgs = result.body["messages"]- assert len(msgs) == 3- assert msgs[0]["content"] == "hi"- assert msgs[1]["content"][0]["type"] == "text"- assert json.loads(msgs[1]["content"][0]["text"]) == {- "type": "audio_clip",- "data": "base64...",- }- assert msgs[2]["content"] == "hello"-- def test_responses_orphan_function_call_output(self, req_engine):- """A function_call_output without a preceding function_call should- not produce an invalid Chat ``tool`` message.- """- body = {- "model": "gpt-4o",- "input": [- {- "type": "function_call_output",- "call_id": "call_orphan",- "output": "result",- },- ],- }- req = ChatRequest.openai_responses(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- assert result.body["messages"] == [- {"role": "user", "content": "Tool result call_orphan: result"}- ]-- def test_responses_max_output_tokens_to_max_completion_tokens(self, req_engine):- """max_output_tokens should map to Chat's current max token cap."""- body = {"model": "gpt-4o", "input": "hi", "max_output_tokens": 4096}- req = ChatRequest.openai_responses(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- assert result.body["max_completion_tokens"] == 4096- assert "max_tokens" not in result.body- assert "max_output_tokens" not in result.body-- def test_responses_tool_with_codex_format(self, req_engine):- """Tools in Codex CLI format (id + inputSchema.jsonSchema) should- be converted correctly.- """- body = {- "model": "gpt-4o",- "input": "hi",- "tools": [- {- "type": "function",- "id": "codex_tool",- "inputSchema": {- "jsonSchema": {- "type": "object",- "properties": {"path": {"type": "string"}},- }- },- }- ],- }- req = ChatRequest.openai_responses(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- tools = result.body["tools"]- assert len(tools) == 1- assert tools[0]["function"]["name"] == "codex_tool"- assert tools[0]["function"]["parameters"]["properties"]["path"]["type"] == "string"-- def test_responses_tool_with_empty_name_skipped(self, req_engine):- """Tools with no name AND no id should be skipped."""- body = {- "model": "gpt-4o",- "input": "hi",- "tools": [- {"type": "function", "description": "ghost", "parameters": {}},- {"type": "function", "name": "real_tool", "parameters": {}},- ],- }- req = ChatRequest.openai_responses(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- tools = result.body["tools"]- assert len(tools) == 1- assert tools[0]["function"]["name"] == "real_tool"-- def test_responses_multi_turn_tool_roundtrip(self, req_engine):- """function_call + function_call_output sequences should be merged- into assistant(tool_calls) + tool messages.- """- body = {- "model": "gpt-4o",- "input": [- {"type": "message", "role": "user", "content": "Search for X"},- {- "type": "function_call",- "name": "search",- "call_id": "call_1",- "arguments": '{"q":"X"}',- },- {- "type": "function_call_output",- "call_id": "call_1",- "output": "Found X",- },- {"type": "message", "role": "assistant", "content": "I found X!"},- ],- }- req = ChatRequest.openai_responses(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- msgs = result.body["messages"]- roles = [m["role"] for m in msgs]- assert roles == ["user", "assistant", "tool", "assistant"]- assert msgs[1]["tool_calls"][0]["function"]["name"] == "search"-- def test_responses_parallel_tool_calls_merged(self, req_engine):- """Multiple consecutive function_call items (before any output)- should be merged into a single assistant message.- """- body = {- "model": "gpt-4o",- "input": [- {"type": "message", "role": "user", "content": "Do two things"},- {- "type": "function_call",- "name": "tool_a",- "call_id": "call_a",- "arguments": "{}",- },- {- "type": "function_call",- "name": "tool_b",- "call_id": "call_b",- "arguments": "{}",- },- {- "type": "function_call_output",- "call_id": "call_a",- "output": "A done",- },- {- "type": "function_call_output",- "call_id": "call_b",- "output": "B done",- },- ],- }- req = ChatRequest.openai_responses(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- msgs = result.body["messages"]- # user, assistant(2 tool_calls), tool(a), tool(b)- assert msgs[0]["role"] == "user"- assert msgs[1]["role"] == "assistant"- assert len(msgs[1]["tool_calls"]) == 2- assert msgs[2]["role"] == "tool"- assert msgs[3]["role"] == "tool"-- def test_responses_intervening_message_deferred_until_after_tool_output(self, req_engine):- """Messages between a function_call and matching output must not- separate Chat tool_calls from their tool result.- """- body = {- "model": "gpt-4o",- "input": [- {"type": "message", "role": "user", "content": "Search for X"},- {- "type": "function_call",- "name": "search",- "call_id": "call_1",- "arguments": "{}",- },- {- "type": "message",- "role": "assistant",- "content": "I will summarize after the tool.",- },- {- "type": "function_call_output",- "call_id": "call_1",- "output": "Found X",- },- ],- }- req = ChatRequest.openai_responses(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- msgs = result.body["messages"]- assert [m["role"] for m in msgs] == ["user", "assistant", "tool", "assistant"]- assert msgs[1]["tool_calls"][0]["id"] == "call_1"- assert msgs[2]["tool_call_id"] == "call_1"- assert msgs[3]["content"] == "I will summarize after the tool."-- def test_responses_chat_compatible_fields_survive_native_translation(self, req_engine):- body = {- "model": "gpt-4o",- "input": "hi",- "metadata": {"trace": "abc"},- "parallel_tool_calls": False,- "prompt_cache_key": "session-1",- "prompt_cache_retention": "24h",- "safety_identifier": "safe-1",- "service_tier": "flex",- "store": False,- "stream_options": {"include_usage": True},- "top_logprobs": 2,- "user": "u-123",- }- req = ChatRequest.openai_responses(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- assert result.body["metadata"] == {"trace": "abc"}- assert result.body["parallel_tool_calls"] is False- assert result.body["prompt_cache_key"] == "session-1"- assert result.body["prompt_cache_retention"] == "24h"- assert result.body["safety_identifier"] == "safe-1"- assert result.body["service_tier"] == "flex"- assert result.body["store"] is False- assert result.body["stream_options"] == {"include_usage": True}- assert result.body["top_logprobs"] == 2- assert result.body["user"] == "u-123"-- def test_responses_json_schema_text_format_maps_to_chat_response_format(self, req_engine):- body = {- "model": "gpt-4o",- "input": "Return JSON",- "text": {- "format": {- "type": "json_schema",- "name": "answer",- "schema": {"type": "object"},- "strict": True,- }- },- }- req = ChatRequest.openai_responses(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- assert result.body["response_format"] == {- "type": "json_schema",- "json_schema": {- "name": "answer",- "schema": {"type": "object"},- "strict": True,- },- }-- # -- NotImplementedError paths ----------------------------------------- def test_unknown_request_type_to_openai(self, req_engine):- """An unknown ChatRequest subclass should raise NotImplementedError- with a message that includes the class name.- """- req = _AlienChatRequest({"model": "x"})- with pytest.raises(NotImplementedError, match="_AlienChatRequest"):- req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)-- def test_unknown_request_type_to_anthropic(self, req_engine):- """An unknown subclass going to_anthropic should mention the class name."""- req = _AlienChatRequest()- with pytest.raises(NotImplementedError, match="_AlienChatRequest"):- req_engine.request_to(ChatRequestType.ANTHROPIC, req)-- def test_unknown_request_type_to_responses(self, req_engine):- """An unknown subclass going to_responses should raise."""- req = _AlienChatRequest()- with pytest.raises(NotImplementedError, match="_AlienChatRequest"):- req_engine.request_to(ChatRequestType.OPENAI_RESPONSES, req)-- # -- OpenAI -> Anthropic edge cases ------------------------------------ def test_openai_to_anthropic_default_max_tokens(self, req_engine):- """When ``max_tokens`` is omitted, Anthropic requires a default.-- The translation layer injects ``64_000`` to provide room for long- coding outputs without defaulting every request to the model's- absolute output ceiling.- """- body = {- "model": "gpt-4o",- "messages": [{"role": "user", "content": "hi"}],- }- req = ChatRequest.openai_chat(body)- result = req_engine.request_to(ChatRequestType.ANTHROPIC, req)- assert result.body["max_tokens"] == 64_000-- def test_openai_to_anthropic_system_extraction(self, req_engine):- """System messages should be extracted into the Anthropic system param."""- body = {- "model": "gpt-4o",- "messages": [- {"role": "system", "content": "Be helpful."},- {"role": "user", "content": "hi"},- ],- "max_tokens": 100,- }- req = ChatRequest.openai_chat(body)- result = req_engine.request_to(ChatRequestType.ANTHROPIC, req)- assert result.body["system"] == "Be helpful."- # System should NOT appear in messages- for m in result.body["messages"]:- assert m["role"] != "system"-- def test_openai_to_anthropic_tool_calls_in_messages(self, req_engine):- """OpenAI assistant messages with tool_calls should become Anthropic- assistant messages with tool_use content blocks.- """- body = {- "model": "gpt-4o",- "messages": [- {"role": "user", "content": "weather?"},- {- "role": "assistant",- "content": "Checking...",- "tool_calls": [- {- "id": "call_1",- "function": {- "name": "get_weather",- "arguments": '{"loc": "SF"}',- },- },- ],- },- {- "role": "tool",- "tool_call_id": "call_1",- "content": "72F",- },- ],- "max_tokens": 100,- }- req = ChatRequest.openai_chat(body)- result = req_engine.request_to(ChatRequestType.ANTHROPIC, req)- msgs = result.body["messages"]- # assistant with tool_use blocks- asst = msgs[1]- assert asst["role"] == "assistant"- assert any(b["type"] == "tool_use" for b in asst["content"])- # tool result- tool_result = msgs[2]- assert tool_result["role"] == "user"- assert tool_result["content"][0]["type"] == "tool_result"-- def test_openai_to_anthropic_stop_string_to_list(self, req_engine):- """stop as a single string should become stop_sequences=[string]."""- body = {- "model": "gpt-4o",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- "stop": "END",- }- req = ChatRequest.openai_chat(body)- result = req_engine.request_to(ChatRequestType.ANTHROPIC, req)- assert result.body["stop_sequences"] == ["END"]-- def test_openai_to_anthropic_multimodal_content(self, req_engine):- """OpenAI messages with list content (text + image_url) should be- converted to Anthropic content blocks.- """- body = {- "model": "gpt-4o",- "messages": [- {- "role": "user",- "content": [- {"type": "text", "text": "What's in this image?"},- {"type": "image_url", "image_url": {"url": "http://example.com/img.png"}},- ],- },- ],- "max_tokens": 100,- }- req = ChatRequest.openai_chat(body)- result = req_engine.request_to(ChatRequestType.ANTHROPIC, req)- msg = result.body["messages"][0]- assert isinstance(msg["content"], list)- text_blocks = [b for b in msg["content"] if b.get("type") == "text"]- assert len(text_blocks) == 1---# =========================================================================-# RESPONSE ENGINE EDGE CASES-# =========================================================================---class TestResponseEngineEdgeCases:- """Edge cases for TranslationEngine.translate()."""-- # -- Empty / degenerate responses -------------------------------------- def test_anthropic_empty_choices(self, resp_engine):- """ChatCompletion with choices=[] should produce a valid Anthropic- response with empty text content, not crash.- """- completion = MagicMock()- completion.model_dump.return_value = {- "id": "chatcmpl-test",- "model": "gpt-4o",- "choices": [],- "usage": {"prompt_tokens": 0, "completion_tokens": 0},- }- resp = ChatResponse.openai_completion(completion)- req = ChatRequest.anthropic({- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 1024,- })- result = resp_engine.response_for_request(req, resp)- assert result["type"] == "message"- assert result["content"][0]["type"] == "text"- assert result["content"][0]["text"] == ""-- def test_responses_empty_choices(self, resp_engine):- """ChatCompletion with choices=[] translated for Responses API- should produce a response with empty output.- """- completion = MagicMock()- completion.model_dump.return_value = {- "id": "chatcmpl-test",- "model": "gpt-4o",- "choices": [],- "usage": {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0},- }- resp = ChatResponse.openai_completion(completion)- req = ChatRequest.openai_responses({"model": "gpt-4o", "input": "hi"})- result = resp_engine.response_for_request(req, resp)- assert result["status"] == "completed"- assert result["output"] == []-- def test_anthropic_multiple_choices_uses_first(self, resp_engine):- """When the completion has multiple choices, only the first one- should be used for the Anthropic response.- """- completion = MagicMock()- completion.model_dump.return_value = {- "id": "chatcmpl-test",- "model": "gpt-4o",- "choices": [- {- "message": {"role": "assistant", "content": "first"},- "finish_reason": "stop",- },- {- "message": {"role": "assistant", "content": "second"},- "finish_reason": "stop",- },- ],- "usage": {"prompt_tokens": 10, "completion_tokens": 5},- }- resp = ChatResponse.openai_completion(completion)- req = ChatRequest.anthropic({- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 1024,- })- result = resp_engine.response_for_request(req, resp)- text = [b for b in result["content"] if b["type"] == "text"]- assert text[0]["text"] == "first"-- def test_anthropic_null_content_no_tool_calls(self, resp_engine):- """content=None and no tool_calls should produce an empty text block."""- completion = _mock_completion(content=None, tool_calls_mock=None)- resp = ChatResponse.openai_completion(completion)- req = ChatRequest.anthropic({- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 1024,- })- result = resp_engine.response_for_request(req, resp)- assert result["type"] == "message"- # Should have at least one content block- assert len(result["content"]) >= 1-- def test_anthropic_content_and_tool_calls_together(self, resp_engine):- """A response with both text content AND tool_calls should produce- both text and tool_use content blocks.- """- tc = _mock_tool_call(call_id="call_1", name="search", arguments='{"q":"test"}')- completion = _mock_completion(- content="Let me search for that.",- tool_calls_mock=[tc],- finish_reason="tool_calls",- )- resp = ChatResponse.openai_completion(completion)- req = ChatRequest.anthropic({- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "search"}],- "max_tokens": 1024,- })- result = resp_engine.response_for_request(req, resp)- text_blocks = [b for b in result["content"] if b["type"] == "text"]- tool_blocks = [b for b in result["content"] if b["type"] == "tool_use"]- assert len(text_blocks) == 1- assert text_blocks[0]["text"] == "Let me search for that."- assert len(tool_blocks) == 1- assert tool_blocks[0]["name"] == "search"-- def test_anthropic_tool_call_ids_are_sanitized(self, resp_engine):- """OpenAI tool call IDs must become valid Anthropic ``tool_use.id``s."""- tc = _mock_tool_call(- call_id="call.bad:id/with space",- name="search",- arguments='{"q":"test"}',- )- completion = _mock_completion(- content=None,- tool_calls_mock=[tc],- finish_reason="tool_calls",- )- resp = ChatResponse.openai_completion(completion)- req = ChatRequest.anthropic({- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "search"}],- "max_tokens": 1024,- })-- result = resp_engine.response_for_request(req, resp)-- tool_blocks = [b for b in result["content"] if b["type"] == "tool_use"]- assert len(tool_blocks) == 1- assert tool_blocks[0]["id"] == "call_bad_id_with_space"-- def test_anthropic_model_fallback_chain(self, resp_engine):- """When the Anthropic request body has no 'model' key, the response- should use the model from the ChatCompletion response or 'unknown'.- """- completion = MagicMock()- completion.model_dump.return_value = {- "id": "chatcmpl-test",- "model": "llama-3.1-70b",- "choices": [- {- "message": {"role": "assistant", "content": "hi"},- "finish_reason": "stop",- },- ],- "usage": {"prompt_tokens": 5, "completion_tokens": 1},- }- resp = ChatResponse.openai_completion(completion)- req = ChatRequest.anthropic({- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- })- result = resp_engine.response_for_request(req, resp)- # model=None from request => falls through to response model- assert result["model"] == "llama-3.1-70b"-- def test_anthropic_finish_reason_mapping(self, resp_engine):- """All finish_reason values should map correctly to Anthropic stop_reason."""- mappings = {- "stop": "end_turn",- "length": "max_tokens",- "tool_calls": "tool_use",- "content_filter": "end_turn",- }- for oai_reason, expected_reason in mappings.items():- completion = MagicMock()- completion.model_dump.return_value = {- "id": "chatcmpl-test",- "model": "gpt-4o",- "choices": [- {- "message": {"role": "assistant", "content": "x"},- "finish_reason": oai_reason,- },- ],- "usage": {"prompt_tokens": 1, "completion_tokens": 1},- }- resp = ChatResponse.openai_completion(completion)- req = ChatRequest.anthropic({- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- })- result = resp_engine.response_for_request(req, resp)- assert result["stop_reason"] == expected_reason, (- f"finish_reason={oai_reason!r} should map to {expected_reason!r}"- )-- def test_responses_content_and_tool_calls(self, resp_engine):- """Responses translation with both content and tool_calls should- produce both message and function_call output items.- """- completion = MagicMock()- completion.model_dump.return_value = {- "id": "chatcmpl-test",- "model": "gpt-4o",- "choices": [- {- "message": {- "role": "assistant",- "content": "Let me check.",- "tool_calls": [- {- "id": "call_1",- "function": {- "name": "search",- "arguments": '{"q":"test"}',- },- },- ],- },- "finish_reason": "tool_calls",- },- ],- "usage": {"prompt_tokens": 10, "completion_tokens": 15, "total_tokens": 25},- }- resp = ChatResponse.openai_completion(completion)- req = ChatRequest.openai_responses({"model": "gpt-4o", "input": "search"})- result = resp_engine.response_for_request(req, resp)- msg_items = [o for o in result["output"] if o["type"] == "message"]- fc_items = [o for o in result["output"] if o["type"] == "function_call"]- assert len(msg_items) == 1- assert len(fc_items) == 1-- def test_model_dump_failure_propagates(self, resp_engine):- """If the response's model_dump() raises, the error should propagate- (not be silently swallowed).- """- class BadCompletion:- def model_dump(self, **kwargs):- del kwargs- raise RuntimeError("serialization failed")-- with pytest.raises(RuntimeError, match="serialization failed"):- ChatResponse.openai_completion(BadCompletion())-- def test_unknown_request_type_for_response(self, resp_engine):- """An unknown ChatRequest subclass should raise NotImplementedError."""- completion = _mock_completion()- resp = ChatResponse.openai_completion(completion)- req = _AlienChatRequest()- with pytest.raises(NotImplementedError, match="_AlienChatRequest"):- resp_engine.response_for_request(req, resp)-- def test_unknown_request_type_for_stream(self, resp_engine):- """An unknown ChatRequest subclass in translate_stream should raise."""- stream = ResponseStream(_async_chunks())- resp = ChatResponse.openai_stream(stream)- req = _AlienChatRequest()- with pytest.raises(NotImplementedError, match="_AlienChatRequest"):- resp_engine.stream_for_request(req, resp)-- def test_anthropic_tool_call_with_invalid_json_arguments(self, resp_engine):- """Tool call arguments that are not valid JSON should be wrapped- in {"raw": ...} rather than crashing.- """- tc = _mock_tool_call(- call_id="call_bad",- name="broken_tool",- arguments="this is not json{{{",- )- completion = _mock_completion(- content=None,- tool_calls_mock=[tc],- finish_reason="tool_calls",- )- resp = ChatResponse.openai_completion(completion)- req = ChatRequest.anthropic({- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- })- result = resp_engine.response_for_request(req, resp)- tool_blocks = [b for b in result["content"] if b["type"] == "tool_use"]- assert len(tool_blocks) == 1- assert tool_blocks[0]["input"] == {"raw": "this is not json{{{"}-- def test_anthropic_usage_as_dict(self, resp_engine):- """When the response dict has usage as a plain dict (no attributes),- it should still be extracted correctly.- """- completion = MagicMock()- completion.model_dump.return_value = {- "id": "chatcmpl-test",- "model": "gpt-4o",- "choices": [- {- "message": {"role": "assistant", "content": "hi"},- "finish_reason": "stop",- },- ],- "usage": {"prompt_tokens": 42, "completion_tokens": 7},- }- resp = ChatResponse.openai_completion(completion)- req = ChatRequest.anthropic({- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- })- result = resp_engine.response_for_request(req, resp)- assert result["usage"]["input_tokens"] == 42- assert result["usage"]["output_tokens"] == 7---# =========================================================================-# STREAMING EDGE CASES-# =========================================================================---class TestStreamingEdgeCases:- """Edge cases in translate_stream for both Anthropic and Responses paths."""-- async def test_anthropic_stream_empty_chunks(self, resp_engine):- """Chunks with choices=[] or choices=None should be skipped- without crashing, and the stream should still emit valid- lifecycle events.- """- chunks = [- MockChunk(choices=[]),- MockChunk(choices=None),- MockChunk(choices=[MockChoice(delta=MockDelta(content="Hi"))]),- MockChunk(choices=[MockChoice(delta=MockDelta(), finish_reason="stop")]),- ]- stream = ResponseStream(_async_chunks(*chunks))- resp = ChatResponse.openai_stream(stream)- req = ChatRequest.anthropic({- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- })- result = resp_engine.stream_for_request(req, resp)- events = [e async for e in result]- types = [e["type"] for e in events]- assert "message_start" in types- assert "message_stop" in types- # The text "Hi" should be in a content_block_delta- deltas = [e for e in events if e["type"] == "content_block_delta"]- assert any(d["delta"]["text"] == "Hi" for d in deltas)-- async def test_openai_reasoning_stream_deltas_do_not_become_anthropic_text(- self,- resp_engine,- ):- """OpenAI-compatible reasoning deltas are not visible assistant text."""- chunks = [- MockChunk(choices=[MockChoice(delta=MockDelta(- reasoning="private reasoning",- reasoning_content="private reasoning content",- ))]),- MockChunk(choices=[MockChoice(delta=MockDelta(content="Visible"))]),- MockChunk(choices=[MockChoice(delta=MockDelta(), finish_reason="stop")]),- ]- stream = ResponseStream(_async_chunks(*chunks))- resp = ChatResponse.openai_stream(stream)- req = ChatRequest.anthropic({- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- })- result = resp_engine.stream_for_request(req, resp)- events = [e async for e in result]- serialized = json.dumps(events)-- assert "Visible" in serialized- assert "private reasoning" in serialized- assert "private reasoning content" in serialized- assert any(- event.get("type") == "content_block_delta"- and event.get("delta", {}).get("type") == "thinking_delta"- for event in events- )- assert not any(- event.get("type") == "content_block_delta"- and event.get("delta", {}).get("type") == "text_delta"- and "private reasoning" in event.get("delta", {}).get("text", "")- for event in events- )-- from anthropic.types import ( # noqa: PLC0415- RawContentBlockDeltaEvent,- RawContentBlockStartEvent,- )-- for event in events:- if event.get("type") == "content_block_start":- RawContentBlockStartEvent.model_validate(event)- if event.get("type") == "content_block_delta":- RawContentBlockDeltaEvent.model_validate(event)-- async def test_anthropic_stream_only_usage_chunk(self, resp_engine):- """A stream where the only chunk has no choices but has usage- should produce valid message_start, empty content block, message_stop.- """- chunks = [- MockChunk(choices=None, usage=MockUsage(prompt_tokens=5, completion_tokens=0)),- ]- stream = ResponseStream(_async_chunks(*chunks))- resp = ChatResponse.openai_stream(stream)- req = ChatRequest.anthropic({- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- })- result = resp_engine.stream_for_request(req, resp)- events = [e async for e in result]- types = [e["type"] for e in events]- assert types[0] == "message_start"- assert types[-1] == "message_stop"- # Should have emitted a minimal empty text block- assert "content_block_start" in types- assert "content_block_stop" in types-- async def test_anthropic_stream_completely_empty(self, resp_engine):- """An empty async iterator (no chunks at all) should still- emit the full Anthropic lifecycle: message_start, empty text block,- message_delta, message_stop.- """- stream = ResponseStream(_async_chunks())- resp = ChatResponse.openai_stream(stream)- req = ChatRequest.anthropic({- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- })- result = resp_engine.stream_for_request(req, resp)- events = [e async for e in result]- types = [e["type"] for e in events]- assert types[0] == "message_start"- assert types[-1] == "message_stop"- # Empty stream produces a minimal text block- assert "content_block_start" in types-- async def test_anthropic_stream_interleaved_tool_calls(self, resp_engine):- """Tool call chunks that arrive with name in one chunk and arguments- split across multiple chunks should be correctly assembled.- """- chunks = [- # First chunk: tool call with name- MockChunk(choices=[MockChoice(delta=MockDelta(- tool_calls=[MockToolCallDelta(- index=0,- id="call_1",- function=MockFunctionDelta(name="get_weather", arguments='{"loc'),- )],- ))]),- # Second chunk: more arguments- MockChunk(choices=[MockChoice(delta=MockDelta(- tool_calls=[MockToolCallDelta(- index=0,- function=MockFunctionDelta(arguments='ation":"SF"}'),- )],- ))]),- # Finish- MockChunk(choices=[MockChoice(- delta=MockDelta(),- finish_reason="tool_calls",- )]),- ]- stream = ResponseStream(_async_chunks(*chunks))- resp = ChatResponse.openai_stream(stream)- req = ChatRequest.anthropic({- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "weather?"}],- "max_tokens": 100,- })- result = resp_engine.stream_for_request(req, resp)- events = [e async for e in result]-- # Should have content_block_start for tool_use- starts = [e for e in events if e["type"] == "content_block_start"]- tool_starts = [s for s in starts if s["content_block"]["type"] == "tool_use"]- assert len(tool_starts) == 1- assert tool_starts[0]["content_block"]["name"] == "get_weather"-- # Should have input_json_delta events- json_deltas = [- e for e in events- if e["type"] == "content_block_delta"- and e["delta"].get("type") == "input_json_delta"- ]- assert len(json_deltas) >= 1-- # Message delta should have stop_reason = tool_use- msg_delta = [e for e in events if e["type"] == "message_delta"]- assert msg_delta[0]["delta"]["stop_reason"] == "tool_use"-- async def test_anthropic_stream_text_then_tool(self, resp_engine):- """A stream with text content followed by a tool call should- close the text block before starting the tool_use block.- """- chunks = [- MockChunk(choices=[MockChoice(delta=MockDelta(content="Checking..."))]),- MockChunk(choices=[MockChoice(delta=MockDelta(- tool_calls=[MockToolCallDelta(- index=0,- id="call_1",- function=MockFunctionDelta(name="search", arguments='{"q":"x"}'),- )],- ))]),- MockChunk(choices=[MockChoice(- delta=MockDelta(), finish_reason="tool_calls",- )]),- ]- stream = ResponseStream(_async_chunks(*chunks))- resp = ChatResponse.openai_stream(stream)- req = ChatRequest.anthropic({- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "search"}],- "max_tokens": 100,- })- result = resp_engine.stream_for_request(req, resp)- events = [e async for e in result]- types = [e["type"] for e in events]-- # Text block should be started and stopped before tool block starts- text_stop_idx = types.index("content_block_stop")- # Find tool_use content_block_start- tool_start_idx = None- for i, e in enumerate(events):- if (- e["type"] == "content_block_start"- and e.get("content_block", {}).get("type") == "tool_use"- ):- tool_start_idx = i- break- assert tool_start_idx is not None- assert text_stop_idx < tool_start_idx-- async def test_responses_stream_empty_chunks(self, resp_engine):- """Responses stream with empty/null choice chunks should still- produce valid lifecycle SSE events.- """- chunks = [- MockChunk(choices=[]),- MockChunk(choices=[MockChoice(delta=MockDelta(content="Hi"))]),- MockChunk(choices=[MockChoice(delta=MockDelta(), finish_reason="stop")]),- ]- stream = ResponseStream(_async_chunks(*chunks))- resp = ChatResponse.openai_stream(stream)- req = ChatRequest.openai_responses({"model": "gpt-4o", "input": "hi"})- result = resp_engine.stream_for_request(req, resp)- events = [e async for e in result]- all_text = "".join(events)- assert "response.created" in all_text- assert "response.completed" in all_text- assert "Hi" in all_text-- async def test_responses_stream_captures_usage_from_final_chunk(self, resp_engine):- """The usage-only final chunk should be captured in the- response.completed event.- """- chunks = [- MockChunk(choices=[MockChoice(delta=MockDelta(content="Hello"))]),- MockChunk(- choices=[MockChoice(delta=MockDelta(), finish_reason="stop")],- ),- # Usage-only final chunk (no choices)- MockChunk(- choices=None,- usage=MockUsage(prompt_tokens=10, completion_tokens=5, total_tokens=15),- ),- ]- stream = ResponseStream(_async_chunks(*chunks))- resp = ChatResponse.openai_stream(stream)- req = ChatRequest.openai_responses({"model": "gpt-4o", "input": "hi"})- result = resp_engine.stream_for_request(req, resp)- events = [e async for e in result]- # Find the response.completed event and parse it- for event_str in events:- if "response.completed" in event_str:- # Extract JSON data- for line in event_str.split("\n"):- if line.startswith("data: "):- data = json.loads(line[6:])- assert data["response"]["usage"]["input_tokens"] == 10- assert data["response"]["usage"]["output_tokens"] == 5- break-- async def test_responses_stream_tool_calls(self, resp_engine):- """Responses streaming with tool calls should emit function_call- lifecycle events.- """- chunks = [- MockChunk(choices=[MockChoice(delta=MockDelta(- tool_calls=[MockToolCallDelta(- index=0,- id="call_1",- function=MockFunctionDelta(name="search", arguments='{"q":"x"}'),- )],- ))]),- MockChunk(choices=[MockChoice(- delta=MockDelta(), finish_reason="tool_calls",- )]),- ]- stream = ResponseStream(_async_chunks(*chunks))- resp = ChatResponse.openai_stream(stream)- req = ChatRequest.openai_responses({"model": "gpt-4o", "input": "search"})- result = resp_engine.stream_for_request(req, resp)- events = [e async for e in result]- all_text = "".join(events)- assert "response.output_item.added" in all_text- assert "function_call" in all_text- assert "response.function_call_arguments.delta" in all_text-- async def test_response_stream_double_consume_raises(self, resp_engine):- """ResponseStream should raise RuntimeError on second iteration."""- chunks = [- MockChunk(choices=[MockChoice(delta=MockDelta(content="Hi"))]),- ]- stream = ResponseStream(_async_chunks(*chunks))- resp = ChatResponse.openai_stream(stream)- req = ChatRequest.anthropic({- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- })- result = resp_engine.stream_for_request(req, resp)- _ = [e async for e in result]- # Second consume of the same stream- with pytest.raises(RuntimeError, match="already been consumed"):- _ = [e async for e in resp.stream]-- async def test_anthropic_stream_multiple_tool_calls(self, resp_engine):- """Multiple parallel tool calls should each get their own- content_block_start/stop events with correct block indices.- """- chunks = [- # Tool 0 name- MockChunk(choices=[MockChoice(delta=MockDelta(- tool_calls=[MockToolCallDelta(- index=0,- id="call_1",- function=MockFunctionDelta(name="search", arguments=None),- )],- ))]),- # Tool 1 name- MockChunk(choices=[MockChoice(delta=MockDelta(- tool_calls=[MockToolCallDelta(- index=1,- id="call_2",- function=MockFunctionDelta(name="fetch", arguments=None),- )],- ))]),- # Tool 0 args- MockChunk(choices=[MockChoice(delta=MockDelta(- tool_calls=[MockToolCallDelta(- index=0,- function=MockFunctionDelta(arguments='{"q":"test"}'),- )],- ))]),- # Tool 1 args- MockChunk(choices=[MockChoice(delta=MockDelta(- tool_calls=[MockToolCallDelta(- index=1,- function=MockFunctionDelta(arguments='{"url":"http://x"}'),- )],- ))]),- # Finish- MockChunk(choices=[MockChoice(- delta=MockDelta(), finish_reason="tool_calls",- )]),- ]- stream = ResponseStream(_async_chunks(*chunks))- resp = ChatResponse.openai_stream(stream)- req = ChatRequest.anthropic({- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "parallel"}],- "max_tokens": 100,- })- result = resp_engine.stream_for_request(req, resp)- events = [e async for e in result]-- # Two tool_use block starts- tool_starts = [- e for e in events- if e["type"] == "content_block_start"- and e.get("content_block", {}).get("type") == "tool_use"- ]- assert len(tool_starts) == 2- names = {s["content_block"]["name"] for s in tool_starts}- assert names == {"search", "fetch"}- # Block indices should be distinct- indices = {s["index"] for s in tool_starts}- assert len(indices) == 2---# =========================================================================-# CROSS-CUTTING CONCERNS-# =========================================================================---class TestCrossCutting:- """Tests that verify properties across the entire translation pipeline."""-- def test_idempotency_request_engine(self, req_engine):- """Translating the same request twice should produce equivalent results."""- body = {- "model": "claude-sonnet-4-20250514",- "messages": [- {"role": "user", "content": "Hello"},- {"role": "assistant", "content": "Hi there!"},- {"role": "user", "content": "How are you?"},- ],- "max_tokens": 1024,- "system": "Be helpful.",- }- req = ChatRequest.anthropic(body)- r1 = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- r2 = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- assert r1.body["messages"] == r2.body["messages"]- assert r1.body.get("model") == r2.body.get("model")- assert r1.body.get("max_tokens") == r2.body.get("max_tokens")-- def test_idempotency_response_engine(self, resp_engine):- """Translating the same response twice should produce equivalent results- (modulo non-deterministic IDs).- """- completion = _mock_completion(content="Hello!")- resp = ChatResponse.openai_completion(completion)- req = ChatRequest.anthropic({- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- })- r1 = resp_engine.response_for_request(req, resp)- r2 = resp_engine.response_for_request(req, resp)- # IDs are randomly generated, so compare everything else- assert r1["type"] == r2["type"]- assert r1["role"] == r2["role"]- assert r1["content"] == r2["content"]- assert r1["model"] == r2["model"]- assert r1["stop_reason"] == r2["stop_reason"]- assert r1["usage"] == r2["usage"]-- def test_round_trip_openai_to_anthropic_to_openai(self, req_engine):- """OpenAI -> Anthropic -> OpenAI should preserve core semantics:- model, messages content, max_tokens.- """- original_body = {- "model": "gpt-4o",- "messages": [- {"role": "system", "content": "Be helpful."},- {"role": "user", "content": "Hello!"},- {"role": "assistant", "content": "Hi there!"},- {"role": "user", "content": "How are you?"},- ],- "max_tokens": 1024,- "temperature": 0.7,- }- openai_req = ChatRequest.openai_chat(original_body)-- # OpenAI -> Anthropic- anthropic_req = req_engine.request_to(ChatRequestType.ANTHROPIC, openai_req)- assert request_type_matches(anthropic_req, ChatRequestType.ANTHROPIC)- assert anthropic_req.body["model"] == "gpt-4o"- assert anthropic_req.body["system"] == "Be helpful."-- # Anthropic -> OpenAI- round_tripped = req_engine.request_to(ChatRequestType.OPENAI_CHAT, anthropic_req)- assert request_type_matches(round_tripped, ChatRequestType.OPENAI_CHAT)-- # Core semantics preserved- assert round_tripped.body["model"] == "gpt-4o"- rt_msgs = round_tripped.body["messages"]- # System should be first- assert rt_msgs[0]["role"] == "system"- assert rt_msgs[0]["content"] == "Be helpful."- # User messages preserved- user_msgs = [m for m in rt_msgs if m["role"] == "user"]- assert len(user_msgs) == 2- assert user_msgs[0]["content"] == "Hello!"- assert user_msgs[1]["content"] == "How are you?"-- def test_round_trip_preserves_tools(self, req_engine):- """OpenAI -> Anthropic -> OpenAI should preserve tool definitions."""- original_body = {- "model": "gpt-4o",- "messages": [{"role": "user", "content": "weather?"}],- "max_tokens": 100,- "tools": [- {- "type": "function",- "function": {- "name": "get_weather",- "description": "Get the weather",- "parameters": {- "type": "object",- "properties": {- "location": {"type": "string"},- },- "required": ["location"],- },- },- },- ],- }- openai_req = ChatRequest.openai_chat(original_body)- anthropic_req = req_engine.request_to(ChatRequestType.ANTHROPIC, openai_req)- round_tripped = req_engine.request_to(ChatRequestType.OPENAI_CHAT, anthropic_req)-- rt_tools = round_tripped.body.get("tools", [])- assert len(rt_tools) == 1- assert rt_tools[0]["function"]["name"] == "get_weather"- assert (- rt_tools[0]["function"]["parameters"]["properties"]["location"]["type"]- == "string"- )-- async def test_concurrent_translation_no_state_corruption(self, req_engine):- """The engine is documented as stateless. Verify that calling it- from multiple concurrent coroutines doesn't corrupt state.- """- async def translate_one(i: int) -> ChatRequest:- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": f"msg-{i}"}],- "max_tokens": 100,- "system": f"system-{i}",- }- req = ChatRequest.anthropic(body)- return req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)-- results = await asyncio.gather(*(translate_one(i) for i in range(50)))- # Each result should have its own unique content- for i, r in enumerate(results):- msgs = r.body["messages"]- system_msg = [m for m in msgs if m["role"] == "system"][0]- user_msg = [m for m in msgs if m["role"] == "user"][0]- assert system_msg["content"] == f"system-{i}"- assert user_msg["content"] == f"msg-{i}"-- async def test_concurrent_response_translation(self, resp_engine):- """Verify concurrent response translation doesn't corrupt state."""- async def translate_one(i: int) -> dict:- completion = MagicMock()- completion.model_dump.return_value = {- "id": f"chatcmpl-{i}",- "model": f"model-{i}",- "choices": [- {- "message": {"role": "assistant", "content": f"response-{i}"},- "finish_reason": "stop",- },- ],- "usage": {"prompt_tokens": i, "completion_tokens": i * 2},- }- resp = ChatResponse.openai_completion(completion)- req = ChatRequest.anthropic({- "model": f"model-{i}",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- })- return resp_engine.response_for_request(req, resp)-- results = await asyncio.gather(*(translate_one(i) for i in range(50)))- for i, r in enumerate(results):- text = [b for b in r["content"] if b["type"] == "text"]- assert text[0]["text"] == f"response-{i}"- assert r["usage"]["input_tokens"] == i- assert r["usage"]["output_tokens"] == i * 2-- def test_engine_instances_are_truly_stateless(self):- """Creating multiple engine instances and using them interleaved- should not cause any interference.- """- e1 = TranslationEngine()- e2 = TranslationEngine()- body1 = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "from e1"}],- "max_tokens": 100,- }- body2 = {- "model": "gpt-4o-mini",- "messages": [{"role": "user", "content": "from e2"}],- "max_tokens": 200,- }- r1 = e1.request_to(ChatRequestType.OPENAI_CHAT, ChatRequest.anthropic(body1))- r2 = e2.request_to(ChatRequestType.OPENAI_CHAT, ChatRequest.anthropic(body2))- assert r1.body["model"] == "claude-sonnet-4-20250514"- assert r2.body["model"] == "gpt-4o-mini"-- def test_unicode_content_survives_translation(self, req_engine):- """Unicode characters (emoji, CJK, RTL) should survive- Anthropic -> OpenAI translation without mangling.- """- body = {- "model": "claude-sonnet-4-20250514",- "messages": [- {"role": "user", "content": "Hello world"},- {"role": "assistant", "content": "Hola mundo"},- ],- "max_tokens": 100,- "system": "multilingual assistant",- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- msgs = result.body["messages"]- assert msgs[1]["content"] == "Hello world"- assert msgs[2]["content"] == "Hola mundo"-- def test_very_long_content_no_truncation(self, req_engine):- """A message with very long content (100K chars) should not be- truncated by the translation layer.- """- long_text = "x" * 100_000- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": long_text}],- "max_tokens": 100,- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- assert len(result.body["messages"][0]["content"]) == 100_000-- def test_responses_to_openai_passthrough_params(self, req_engine):- """temperature, top_p, and stream should pass through from- Responses API to Chat Completions.- """- body = {- "model": "gpt-4o",- "input": "hi",- "temperature": 0.5,- "top_p": 0.9,- "stream": True,- }- req = ChatRequest.openai_responses(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- assert result.body["temperature"] == 0.5- assert result.body["top_p"] == 0.9- assert result.body["stream"] is True-- def test_anthropic_tool_choice_variants(self, req_engine):- """All Anthropic tool_choice variants should map correctly."""- # String variants- for ant_choice, expected in [("auto", "auto"), ("any", "required"), ("none", "none")]:- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- "tools": [{"name": "t", "description": "d", "input_schema": {}}],- "tool_choice": ant_choice,- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- assert result.body["tool_choice"] == expected, (- f"tool_choice={ant_choice!r} should map to {expected!r}"- )-- def test_anthropic_tool_choice_specific_tool(self, req_engine):- """Anthropic tool_choice of type 'tool' with a name should map- to OpenAI's function-specific tool_choice.- """- body = {- "model": "claude-sonnet-4-20250514",- "messages": [{"role": "user", "content": "hi"}],- "max_tokens": 100,- "tools": [{"name": "search", "description": "d", "input_schema": {}}],- "tool_choice": {"type": "tool", "name": "search"},- }- req = ChatRequest.anthropic(body)- result = req_engine.request_to(ChatRequestType.OPENAI_CHAT, req)- tc = result.body["tool_choice"]- assert tc["type"] == "function"- assert tc["function"]["name"] == "search"diff --git a/tests/test_upstream_error_log.py b/tests/test_upstream_error_log.pydeleted file mode 100644index 7929fe75a..000000000--- a/tests/test_upstream_error_log.py+++ /dev/null@@ -1,132 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Unit tests for the structured per-attempt upstream-failure log."""--from __future__ import annotations--import json-import logging-from datetime import datetime--import pytest--from switchyard.lib.endpoints import upstream_error_log-from switchyard.lib.endpoints.upstream_error_log import (- EVENT_NAME,- log_upstream_attempt_failure,-)--_LOGGER_NAME = "switchyard.upstream_errors"---def _emit_and_parse(- caplog: pytest.LogCaptureFixture,- *,- model: str,- attempt: int,- status_code: int | None,- error: BaseException,-) -> dict[str, object]:- """Call the logger and return the single emitted record parsed as JSON."""- caplog.clear()- with caplog.at_level(logging.WARNING, logger=_LOGGER_NAME):- log_upstream_attempt_failure(- model=model, attempt=attempt, status_code=status_code, error=error- )- records = [r for r in caplog.records if r.name == _LOGGER_NAME]- assert len(records) == 1, "expected exactly one structured record"- assert records[0].levelno == logging.WARNING- return json.loads(records[0].getMessage())---class TestRecordShape:- def test_message_is_valid_json_with_expected_keys(- self, caplog: pytest.LogCaptureFixture- ) -> None:- rec = _emit_and_parse(- caplog, model="m", attempt=1, status_code=500, error=RuntimeError("boom")- )- assert set(rec) == {- "event",- "timestamp",- "model",- "upstream_model",- "attempt",- "status_code",- "code",- "outcome",- "error_source",- "error_type",- "error",- }- assert rec["event"] == EVENT_NAME == "upstream_attempt_failed"- assert rec["model"] == "m"- # No upstream_model supplied → logged as null; source is constant.- assert rec["upstream_model"] is None- assert rec["error_source"] == "provider"- assert rec["attempt"] == 1- assert rec["error_type"] == "RuntimeError"- assert rec["error"] == "boom"-- def test_timestamp_is_iso8601_and_utc(- self, caplog: pytest.LogCaptureFixture- ) -> None:- rec = _emit_and_parse(- caplog, model="m", attempt=1, status_code=429, error=ValueError("x")- )- # Round-trips through fromisoformat and carries a UTC offset.- parsed = datetime.fromisoformat(str(rec["timestamp"]))- assert parsed.utcoffset() is not None- assert parsed.utcoffset().total_seconds() == 0 # type: ignore[union-attr]---class TestCodeAndOutcomeMirrorTheMetric:- def test_none_is_network_failure(self, caplog: pytest.LogCaptureFixture) -> None:- rec = _emit_and_parse(- caplog, model="m", attempt=2, status_code=None,- error=ConnectionError("reset"),- )- assert rec["status_code"] is None- assert rec["code"] == "none"- assert rec["outcome"] == "retryable_error"-- @pytest.mark.parametrize("code", [429, 500, 504])- def test_retryable_codes(- self, caplog: pytest.LogCaptureFixture, code: int- ) -> None:- rec = _emit_and_parse(- caplog, model="m", attempt=1, status_code=code, error=RuntimeError("e")- )- assert rec["status_code"] == code- assert rec["code"] == str(code)- assert rec["outcome"] == "retryable_error"-- def test_other_error_code(self, caplog: pytest.LogCaptureFixture) -> None:- rec = _emit_and_parse(- caplog, model="m", attempt=1, status_code=401, error=RuntimeError("e")- )- assert rec["status_code"] == 401- assert rec["code"] == "401"- assert rec["outcome"] == "other_error"-- def test_unknown_code_keeps_raw_status_but_clamps_label(- self, caplog: pytest.LogCaptureFixture- ) -> None:- """The raw code is preserved for audit; the joinable label is clamped."""- rec = _emit_and_parse(- caplog, model="m", attempt=1, status_code=418, error=RuntimeError("e")- )- assert rec["status_code"] == 418- assert rec["code"] == "4xx"- assert rec["outcome"] == "other_error"---class TestErrorTruncation:- def test_long_error_is_capped(self, caplog: pytest.LogCaptureFixture) -> None:- rec = _emit_and_parse(- caplog, model="m", attempt=1, status_code=500,- error=RuntimeError("x" * 5000),- )- assert isinstance(rec["error"], str)- assert len(rec["error"]) == upstream_error_log._MAX_ERROR_CHARSdiff --git a/tests/test_upstream_error_passthrough.py b/tests/test_upstream_error_passthrough.pydeleted file mode 100644index ac61dfe48..000000000--- a/tests/test_upstream_error_passthrough.py+++ /dev/null@@ -1,317 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""End-to-end gate that upstream HTTP errors keep stable provider details.--Before this fix, a 401 from the upstream LLM (typically a bad API key-or expired credential) became a generic 500 at the client because the-compatibility executor wrapped the Python ``openai.APIStatusError`` in a-``SwitchyardError::Backend(error.to_string())``, which surfaced as a-plain Python ``RuntimeError`` and FastAPI defaulted it to 500.--Python backends stash upstream status/body on ``ProxyContext.metadata``.-Rust backends surface a typed upstream exception with ``status_code`` and-``body`` attributes. The endpoints recover either signal, preserve the-HTTP status plus stable provider fields, and return the normalized-Switchyard error envelope instead of FastAPI's default plain-text 500.-"""--from __future__ import annotations--import json--from fastapi.testclient import TestClient--from switchyard.cli.route_bundle import build_route_bundle_table-from switchyard.lib.endpoints.error_envelope import ERROR_SOURCE_HEADER-from switchyard.lib.endpoints.upstream_error import (- internal_chain_error_response,- upstream_response_from_ctx,-)-from switchyard.lib.proxy_context import (- CTX_UPSTREAM_HTTP_BODY,- CTX_UPSTREAM_HTTP_STATUS,- ProxyContext,-)-from switchyard.server.switchyard_app import build_switchyard_app-from tests._chain_test_helpers import _OpenAICompatStub--# ----------------------------------------------------------------------------# Helper unit tests-# ------------------------------------------------------------------------------class TestUpstreamResponseFromCtx:- def test_returns_none_when_no_status_recorded(self) -> None:- ctx = ProxyContext()- assert upstream_response_from_ctx(ctx) is None-- def test_wraps_string_body_in_error_envelope(self) -> None:- ctx = ProxyContext()- ctx.metadata[CTX_UPSTREAM_HTTP_STATUS] = 401- ctx.metadata[CTX_UPSTREAM_HTTP_BODY] = "Unauthorized"-- response = upstream_response_from_ctx(ctx)-- assert response is not None- assert response.status_code == 401-- def test_normalizes_dict_body_into_switchyard_envelope(self) -> None:- ctx = ProxyContext()- ctx.metadata[CTX_UPSTREAM_HTTP_STATUS] = 429- ctx.metadata[CTX_UPSTREAM_HTTP_BODY] = {- "error": {"message": "rate limited", "type": "rate_limit"},- }-- response = upstream_response_from_ctx(ctx)-- assert response is not None- assert response.status_code == 429- body = json.loads(response.body)- assert body == {- "error": {- "message": "rate limited",- "type": "rate_limit",- "code": "rate_limit",- }- }-- def test_preserves_provider_error_param_when_present(self) -> None:- ctx = ProxyContext()- ctx.metadata[CTX_UPSTREAM_HTTP_STATUS] = 400- ctx.metadata[CTX_UPSTREAM_HTTP_BODY] = {- "error": {- "message": "bad input",- "type": "invalid_request_error",- "code": "invalid_value",- "param": "messages.0.role",- },- }-- response = upstream_response_from_ctx(ctx)-- assert response is not None- assert response.status_code == 400- assert json.loads(response.body) == {- "error": {- "message": "bad input",- "type": "invalid_request_error",- "code": "invalid_value",- "param": "messages.0.role",- }- }-- def test_synthesizes_envelope_when_body_missing(self) -> None:- ctx = ProxyContext()- ctx.metadata[CTX_UPSTREAM_HTTP_STATUS] = 503-- response = upstream_response_from_ctx(ctx)-- assert response is not None- assert response.status_code == 503- body = json.loads(response.body)- assert body == {- "error": {- "message": "upstream returned HTTP 503",- "type": "upstream_error",- "code": "upstream_error",- }- }-- def test_ignores_non_int_status(self) -> None:- """Defensive: a stray non-int value must not crash the error path."""- ctx = ProxyContext()- ctx.metadata[CTX_UPSTREAM_HTTP_STATUS] = "401" # wrong type — ignore-- assert upstream_response_from_ctx(ctx) is None-- def test_internal_chain_error_uses_openai_error_envelope(self) -> None:- response = internal_chain_error_response(RuntimeError("connection refused"), "openai")-- assert response.status_code == 500- assert response.headers["content-type"].startswith("application/json")- body = json.loads(response.body)- assert "connection refused" in body["error"]["message"]- assert body["error"]["type"] == "internal_error"- assert body["error"]["code"] == "internal_chain_error"-- def test_internal_chain_error_uses_same_envelope_for_anthropic_inbound(self) -> None:- response = internal_chain_error_response(RuntimeError("connection refused"), "anthropic")-- assert response.status_code == 500- assert response.headers["content-type"].startswith("application/json")- body = json.loads(response.body)- assert body["error"]["type"] == "internal_error"- assert body["error"]["code"] == "internal_chain_error"- assert "connection refused" in body["error"]["message"]-- def test_internal_chain_error_truncates_long_repr(self) -> None:- long_msg = "x" * 500- response = internal_chain_error_response(RuntimeError(long_msg), "openai")- body = json.loads(response.body)- assert len(body["error"]["message"]) <= 200---# ----------------------------------------------------------------------------# Anthropic and Responses endpoints share the same helper-# ------------------------------------------------------------------------------def test_rust_openai_route_upstream_401_returns_structured_openai_error() -> None:- """Rust OpenAI-native backend errors must not fall through to FastAPI's 500."""- with _OpenAICompatStub() as upstream:- upstream.respond_json(- {"error": {"message": "bad key", "type": "invalid_api_key"}},- status=401,- )- table = build_route_bundle_table({- "defaults": {- "api_key": "bad-key",- "base_url": upstream.base_url,- "format": "openai",- },- "routes": {- "bad-key": {- "type": "passthrough",- "target": "nvidia/nvidia/nemotron-nano-9b-v2",- }- },- })-- with TestClient(build_switchyard_app(table), raise_server_exceptions=False) as client:- response = client.post(- "/v1/chat/completions",- json={- "model": "bad-key",- "messages": [{"role": "user", "content": "ping"}],- },- )-- assert response.status_code == 401- assert response.json() == {- "error": {- "message": "bad key",- "type": "invalid_api_key",- "code": "invalid_api_key",- },- }---def test_rust_openai_route_upstream_401_returns_same_error_shape_for_anthropic_inbound() -> None:- """Anthropic inbound clients should receive the same HTTP error envelope."""- with _OpenAICompatStub() as upstream:- upstream.respond_json(- {"error": {"message": "bad key", "type": "invalid_api_key"}},- status=401,- )- table = build_route_bundle_table({- "defaults": {- "api_key": "bad-key",- "base_url": upstream.base_url,- "format": "openai",- },- "routes": {- "bad-key": {- "type": "passthrough",- "target": "nvidia/nvidia/nemotron-nano-9b-v2",- }- },- })-- with TestClient(build_switchyard_app(table), raise_server_exceptions=False) as client:- response = client.post(- "/v1/messages",- json={- "model": "bad-key",- "max_tokens": 16,- "messages": [{"role": "user", "content": "ping"}],- },- )-- assert response.status_code == 401- assert response.json() == {- "error": {- "message": "bad key",- "type": "invalid_api_key",- "code": "invalid_api_key",- },- }---def test_rust_openai_route_upstream_401_returns_same_error_shape_for_responses_inbound() -> None:- """Responses inbound clients should receive the same HTTP error envelope."""- with _OpenAICompatStub() as upstream:- upstream.respond_json(- {"error": {"message": "bad key", "type": "invalid_api_key"}},- status=401,- )- table = build_route_bundle_table({- "defaults": {- "api_key": "bad-key",- "base_url": upstream.base_url,- "format": "openai",- },- "routes": {- "bad-key": {- "type": "passthrough",- "target": "nvidia/nvidia/nemotron-nano-9b-v2",- }- },- })-- with TestClient(build_switchyard_app(table), raise_server_exceptions=False) as client:- response = client.post(- "/v1/responses",- json={- "model": "bad-key",- "input": "ping",- },- )-- assert response.status_code == 401- assert response.json() == {- "error": {- "message": "bad key",- "type": "invalid_api_key",- "code": "invalid_api_key",- },- }---# ----------------------------------------------------------------------------# Failure-source headers on the wire-# ----------------------------------------------------------------------------# The header tests above the endpoint layer call helpers directly; these prove-# the annotation actually survives the real FastAPI handlers + middleware to-# the HTTP response a client receives.---def test_model_not_found_404_header_labels_switchyard_on_the_wire() -> None:- """RouteTable dispatch 404s carry the switchyard source header."""- with _OpenAICompatStub() as upstream:- table = build_route_bundle_table({- "defaults": {- "api_key": "k",- "base_url": upstream.base_url,- "format": "openai",- },- "routes": {- "registered": {- "type": "passthrough",- "target": "nvidia/nvidia/nemotron-nano-9b-v2",- }- },- })-- with TestClient(build_switchyard_app(table), raise_server_exceptions=False) as client:- response = client.post(- "/v1/chat/completions",- json={- "model": "no-such-model",- "messages": [{"role": "user", "content": "hi"}],- },- )-- assert response.status_code == 404- assert response.json()["error"]["code"] == "model_not_found"- assert response.headers[ERROR_SOURCE_HEADER] == "switchyard"diff --git a/tests/translation/__init__.py b/tests/translation/__init__.pydeleted file mode 100644index 52a7a9daf..000000000--- a/tests/translation/__init__.py+++ /dev/null@@ -1,2 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0diff --git a/tests/translation/test_format_fidelity_contract.py b/tests/translation/test_format_fidelity_contract.pydeleted file mode 100644index 15c403ebd..000000000--- a/tests/translation/test_format_fidelity_contract.py+++ /dev/null@@ -1,148 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--"""Format-fidelity contract: AUTO probes upstream in priority order.--BackendFormat.AUTO probes:- 1. /v1/chat/completions → OPENAI- 2. /v1/messages → ANTHROPIC- 3. /v1/responses → RESPONSES- 4. fallback → OPENAI (Chat Completions assumed universal)--The TranslationEngine converts any inbound format to any backend format-through a neutral IR, so all combinations are valid regardless of client.-"""--import pytest--from switchyard.lib.backends import backend_format_resolver as resolver_mod-from switchyard.lib.backends.llm_target import BackendFormat, LlmTarget-from switchyard.lib.backends.multi_llm_backend import resolve_llm_target---class TestAutoFormatResolution:- """BackendFormat.AUTO probes upstream capabilities and selects the best format."""-- def test_auto_resolves_to_responses_for_openai_upstream(- self, monkeypatch: pytest.MonkeyPatch- ) -> None:- """Chat Completions fails, /v1/messages fails, /v1/responses succeeds → RESPONSES."""- monkeypatch.setattr(- resolver_mod, "probe_openai_chat_completions_support_sync",- lambda *, base_url, api_key, **_kw: False,- )- monkeypatch.setattr(- resolver_mod, "probe_anthropic_messages_support_sync",- lambda *, base_url, api_key, **_kw: False,- )- monkeypatch.setattr(- resolver_mod, "probe_openai_responses_support_sync",- lambda *, base_url, api_key, **_kw: True,- )-- target = LlmTarget(- model="openai/gpt-5.2",- format=BackendFormat.AUTO,- base_url="https://api.openai.com/v1",- api_key="sk-test", # pragma: allowlist secret- )- resolved = resolve_llm_target(target)-- assert resolved.format is BackendFormat.RESPONSES-- def test_auto_falls_back_to_openai_for_nim_upstream(- self, monkeypatch: pytest.MonkeyPatch- ) -> None:- """NIM upstream: Chat Completions succeeds → OPENAI (first probe wins)."""- monkeypatch.setattr(- resolver_mod, "probe_openai_chat_completions_support_sync",- lambda *, base_url, api_key, **_kw: True,- )- monkeypatch.setattr(- resolver_mod, "probe_anthropic_messages_support_sync",- lambda *, base_url, api_key, **_kw: pytest.fail("should not probe Anthropic"),- )- monkeypatch.setattr(- resolver_mod, "probe_openai_responses_support_sync",- lambda *, base_url, api_key, **_kw: pytest.fail("should not probe Responses"),- )-- target = LlmTarget(- model="nvidia/nvidia/nemotron-nano-9b-v2",- format=BackendFormat.AUTO,- base_url="https://integrate.api.nvidia.com/v1",- api_key="sk-test", # pragma: allowlist secret- )- resolved = resolve_llm_target(target)-- assert resolved.format is BackendFormat.OPENAI-- def test_auto_resolves_to_anthropic_for_messages_upstream(- self, monkeypatch: pytest.MonkeyPatch- ) -> None:- """Chat Completions fails, /v1/messages succeeds → ANTHROPIC."""- monkeypatch.setattr(- resolver_mod, "probe_openai_chat_completions_support_sync",- lambda *, base_url, api_key, **_kw: False,- )- monkeypatch.setattr(- resolver_mod, "probe_anthropic_messages_support_sync",- lambda *, base_url, api_key, **_kw: True,- )-- target = LlmTarget(- model="some-non-prefixed-model",- format=BackendFormat.AUTO,- base_url="https://api.anthropic.com/v1",- api_key="sk-test", # pragma: allowlist secret- )- resolved = resolve_llm_target(target)-- assert resolved.format is BackendFormat.ANTHROPIC-- def test_auto_prefix_fast_path_skips_all_probes(- self, monkeypatch: pytest.MonkeyPatch- ) -> None:- """anthropic/ and claude prefixes → ANTHROPIC immediately, no probes fired."""- monkeypatch.setattr(- resolver_mod, "probe_openai_chat_completions_support_sync",- lambda **_kw: pytest.fail("prefix fast-path must not probe"),- )- monkeypatch.setattr(- resolver_mod, "probe_anthropic_messages_support_sync",- lambda **_kw: pytest.fail("prefix fast-path must not probe"),- )- monkeypatch.setattr(- resolver_mod, "probe_openai_responses_support_sync",- lambda **_kw: pytest.fail("prefix fast-path must not probe"),- )-- target = LlmTarget(- model="anthropic/claude-sonnet-4-5",- format=BackendFormat.AUTO,- base_url="https://api.anthropic.com/v1",- api_key="sk-test", # pragma: allowlist secret- )- resolved = resolve_llm_target(target)-- assert resolved.format is BackendFormat.ANTHROPIC-- def test_explicit_formats_bypass_probe(- self, monkeypatch: pytest.MonkeyPatch- ) -> None:- """Explicit RESPONSES/OPENAI/ANTHROPIC are honored as-is without probing."""- def fail_probe(**_kw: object) -> bool:- pytest.fail("explicit formats must not trigger a probe")-- monkeypatch.setattr(resolver_mod, "probe_openai_chat_completions_support_sync", fail_probe)- monkeypatch.setattr(resolver_mod, "probe_anthropic_messages_support_sync", fail_probe)- monkeypatch.setattr(resolver_mod, "probe_openai_responses_support_sync", fail_probe)-- for fmt in (BackendFormat.RESPONSES, BackendFormat.OPENAI, BackendFormat.ANTHROPIC):- target = LlmTarget(- model="some/model",- format=fmt,- base_url="https://api.openai.com/v1",- api_key="sk-test", # pragma: allowlist secret- )- assert resolve_llm_target(target).format is fmtdiff --git a/uv.lock b/uv.lockindex 0c7d61866..8a1234ca6 100644--- a/uv.lock+++ b/uv.lock@@ -141,25 +141,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/78/b6/6307fbef88d9b5ee7421e68d78a9f162e0da4900bc5f5793f6d3d0e34fb8/annotated_types-0.7.0-py3-none-any.whl", hash = "sha256:1f02e8b43a8fbbc3f3e0d4f0f4bfc8131bcb4eebe8849b8e5c773f3a1c582a53", size = 13643, upload-time = "2024-05-20T21:33:24.1Z" },]-[[package]]-name = "anthropic"-version = "0.99.0"-source = { registry = "https://pypi.org/simple" }-dependencies = [- { name = "anyio" },- { name = "distro" },- { name = "docstring-parser" },- { name = "httpx" },- { name = "jiter" },- { name = "pydantic" },- { name = "sniffio" },- { name = "typing-extensions" },-]-sdist = { url = "https://files.pythonhosted.org/packages/0d/c9/e8a3a1caeab575e80551b30b084096b5a430abc52739a526a1daaadd038c/anthropic-0.99.0.tar.gz", hash = "sha256:16f41e00f215ed2d193b146be3dd567c4319c32ed3af6c8725d68ba875257c1c", size = 727239, upload-time = "2026-05-05T16:03:07.986Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/d1/84/d0917506744e1707cf55659a57f1e3ff952eda5636df0ffffe3e884b7c61/anthropic-0.99.0-py3-none-any.whl", hash = "sha256:c44469b746ab2ef19a4c52dcbdb98e17bc95c60bebdd18ec40d76d2d23592b49", size = 700564, upload-time = "2026-05-05T16:03:06.059Z" },-]-[[package]]name = "anyio"version = "4.13.0"@@ -204,15 +185,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/3e/5c/fb93d3092640a24dfb7bd7727a24016d7c01774ca013e60efd3f683c8002/backrefs-7.0-py314-none-any.whl", hash = "sha256:a6448b28180e3ca01134c9cf09dcebafad8531072e09903c5451748a05f24bc9", size = 412349, upload-time = "2026-04-28T16:28:02.412Z" },]-[[package]]-name = "bytecode"-version = "0.18.1"-source = { registry = "https://pypi.org/simple" }-sdist = { url = "https://files.pythonhosted.org/packages/27/8f/7d12c539869a5cbd801d550b86cc0f030ecaeb12f57f8b3ff19f2d2a184c/bytecode-0.18.1.tar.gz", hash = "sha256:d9564f1565fe1ae6a1173e544ef43a85f093e83997ef45af65d0d250eb48d7a1", size = 104631, upload-time = "2026-06-03T14:17:59.115Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/d9/ef/6a629424ec08adc3819ddfd7ec0a710361eaa29d1e5fffb4e02f074be5c9/bytecode-0.18.1-py3-none-any.whl", hash = "sha256:9535bfdd665260b2888ec4121569e3ca5106965a7fedbb6de6ba1bafebc5c7d7", size = 42868, upload-time = "2026-06-03T14:17:57.654Z" },-]-[[package]]name = "cachetools"version = "6.2.6"@@ -400,90 +372,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },]-[[package]]-name = "coverage"-version = "7.13.5"-source = { registry = "https://pypi.org/simple" }-sdist = { url = "https://files.pythonhosted.org/packages/9d/e0/70553e3000e345daff267cec284ce4cbf3fc141b6da229ac52775b5428f1/coverage-7.13.5.tar.gz", hash = "sha256:c81f6515c4c40141f83f502b07bbfa5c240ba25bbe73da7b33f1e5b6120ff179", size = 915967, upload-time = "2026-03-17T10:33:18.341Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/a0/c3/a396306ba7db865bf96fc1fb3b7fd29bcbf3d829df642e77b13555163cd6/coverage-7.13.5-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:460cf0114c5016fa841214ff5564aa4864f11948da9440bc97e21ad1f4ba1e01", size = 219554, upload-time = "2026-03-17T10:30:42.208Z" },- { url = "https://files.pythonhosted.org/packages/a6/16/a68a19e5384e93f811dccc51034b1fd0b865841c390e3c931dcc4699e035/coverage-7.13.5-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:0e223ce4b4ed47f065bfb123687686512e37629be25cc63728557ae7db261422", size = 219908, upload-time = "2026-03-17T10:30:43.906Z" },- { url = "https://files.pythonhosted.org/packages/29/72/20b917c6793af3a5ceb7fb9c50033f3ec7865f2911a1416b34a7cfa0813b/coverage-7.13.5-cp312-cp312-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:6e3370441f4513c6252bf042b9c36d22491142385049243253c7e48398a15a9f", size = 251419, upload-time = "2026-03-17T10:30:45.545Z" },- { url = "https://files.pythonhosted.org/packages/8c/49/cd14b789536ac6a4778c453c6a2338bc0a2fb60c5a5a41b4008328b9acc1/coverage-7.13.5-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:03ccc709a17a1de074fb1d11f217342fb0d2b1582ed544f554fc9fc3f07e95f5", size = 254159, upload-time = "2026-03-17T10:30:47.204Z" },- { url = "https://files.pythonhosted.org/packages/9d/00/7b0edcfe64e2ed4c0340dac14a52ad0f4c9bd0b8b5e531af7d55b703db7c/coverage-7.13.5-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3f4818d065964db3c1c66dc0fbdac5ac692ecbc875555e13374fdbe7eedb4376", size = 255270, upload-time = "2026-03-17T10:30:48.812Z" },- { url = "https://files.pythonhosted.org/packages/93/89/7ffc4ba0f5d0a55c1e84ea7cee39c9fc06af7b170513d83fbf3bbefce280/coverage-7.13.5-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:012d5319e66e9d5a218834642d6c35d265515a62f01157a45bcc036ecf947256", size = 257538, upload-time = "2026-03-17T10:30:50.77Z" },- { url = "https://files.pythonhosted.org/packages/81/bd/73ddf85f93f7e6fa83e77ccecb6162d9415c79007b4bc124008a4995e4a7/coverage-7.13.5-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:8dd02af98971bdb956363e4827d34425cb3df19ee550ef92855b0acb9c7ce51c", size = 251821, upload-time = "2026-03-17T10:30:52.5Z" },- { url = "https://files.pythonhosted.org/packages/a0/81/278aff4e8dec4926a0bcb9486320752811f543a3ce5b602cc7a29978d073/coverage-7.13.5-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:f08fd75c50a760c7eb068ae823777268daaf16a80b918fa58eea888f8e3919f5", size = 253191, upload-time = "2026-03-17T10:30:54.543Z" },- { url = "https://files.pythonhosted.org/packages/70/ee/fe1621488e2e0a58d7e94c4800f0d96f79671553488d401a612bebae324b/coverage-7.13.5-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:843ea8643cf967d1ac7e8ecd4bb00c99135adf4816c0c0593fdcc47b597fcf09", size = 251337, upload-time = "2026-03-17T10:30:56.663Z" },- { url = "https://files.pythonhosted.org/packages/37/a6/f79fb37aa104b562207cc23cb5711ab6793608e246cae1e93f26b2236ed9/coverage-7.13.5-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:9d44d7aa963820b1b971dbecd90bfe5fe8f81cff79787eb6cca15750bd2f79b9", size = 255404, upload-time = "2026-03-17T10:30:58.427Z" },- { url = "https://files.pythonhosted.org/packages/75/f0/ed15262a58ec81ce457ceb717b7f78752a1713556b19081b76e90896e8d4/coverage-7.13.5-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:7132bed4bd7b836200c591410ae7d97bf7ae8be6fc87d160b2bd881df929e7bf", size = 250903, upload-time = "2026-03-17T10:31:00.093Z" },- { url = "https://files.pythonhosted.org/packages/0f/e9/9129958f20e7e9d4d56d51d42ccf708d15cac355ff4ac6e736e97a9393d2/coverage-7.13.5-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:a698e363641b98843c517817db75373c83254781426e94ada3197cabbc2c919c", size = 252780, upload-time = "2026-03-17T10:31:01.916Z" },- { url = "https://files.pythonhosted.org/packages/a4/d7/0ad9b15812d81272db94379fe4c6df8fd17781cc7671fdfa30c76ba5ff7b/coverage-7.13.5-cp312-cp312-win32.whl", hash = "sha256:bdba0a6b8812e8c7df002d908a9a2ea3c36e92611b5708633c50869e6d922fdf", size = 222093, upload-time = "2026-03-17T10:31:03.642Z" },- { url = "https://files.pythonhosted.org/packages/29/3d/821a9a5799fac2556bcf0bd37a70d1d11fa9e49784b6d22e92e8b2f85f18/coverage-7.13.5-cp312-cp312-win_amd64.whl", hash = "sha256:d2c87e0c473a10bffe991502eac389220533024c8082ec1ce849f4218dded810", size = 222900, upload-time = "2026-03-17T10:31:05.651Z" },- { url = "https://files.pythonhosted.org/packages/d4/fa/2238c2ad08e35cf4f020ea721f717e09ec3152aea75d191a7faf3ef009a8/coverage-7.13.5-cp312-cp312-win_arm64.whl", hash = "sha256:bf69236a9a81bdca3bff53796237aab096cdbf8d78a66ad61e992d9dac7eb2de", size = 221515, upload-time = "2026-03-17T10:31:07.293Z" },- { url = "https://files.pythonhosted.org/packages/74/8c/74fedc9663dcf168b0a059d4ea756ecae4da77a489048f94b5f512a8d0b3/coverage-7.13.5-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:5ec4af212df513e399cf11610cc27063f1586419e814755ab362e50a85ea69c1", size = 219576, upload-time = "2026-03-17T10:31:09.045Z" },- { url = "https://files.pythonhosted.org/packages/0c/c9/44fb661c55062f0818a6ffd2685c67aa30816200d5f2817543717d4b92eb/coverage-7.13.5-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:941617e518602e2d64942c88ec8499f7fbd49d3f6c4327d3a71d43a1973032f3", size = 219942, upload-time = "2026-03-17T10:31:10.708Z" },- { url = "https://files.pythonhosted.org/packages/5f/13/93419671cee82b780bab7ea96b67c8ef448f5f295f36bf5031154ec9a790/coverage-7.13.5-cp313-cp313-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:da305e9937617ee95c2e39d8ff9f040e0487cbf1ac174f777ed5eddd7a7c1f26", size = 250935, upload-time = "2026-03-17T10:31:12.392Z" },- { url = "https://files.pythonhosted.org/packages/ac/68/1666e3a4462f8202d836920114fa7a5ee9275d1fa45366d336c551a162dd/coverage-7.13.5-cp313-cp313-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:78e696e1cc714e57e8b25760b33a8b1026b7048d270140d25dafe1b0a1ee05a3", size = 253541, upload-time = "2026-03-17T10:31:14.247Z" },- { url = "https://files.pythonhosted.org/packages/4e/5e/3ee3b835647be646dcf3c65a7c6c18f87c27326a858f72ab22c12730773d/coverage-7.13.5-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:02ca0eed225b2ff301c474aeeeae27d26e2537942aa0f87491d3e147e784a82b", size = 254780, upload-time = "2026-03-17T10:31:16.193Z" },- { url = "https://files.pythonhosted.org/packages/44/b3/cb5bd1a04cfcc49ede6cd8409d80bee17661167686741e041abc7ee1b9a9/coverage-7.13.5-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:04690832cbea4e4663d9149e05dba142546ca05cb1848816760e7f58285c970a", size = 256912, upload-time = "2026-03-17T10:31:17.89Z" },- { url = "https://files.pythonhosted.org/packages/1b/66/c1dceb7b9714473800b075f5c8a84f4588f887a90eb8645282031676e242/coverage-7.13.5-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:0590e44dd2745c696a778f7bab6aa95256de2cbc8b8cff4f7db8ff09813d6969", size = 251165, upload-time = "2026-03-17T10:31:19.605Z" },- { url = "https://files.pythonhosted.org/packages/b7/62/5502b73b97aa2e53ea22a39cf8649ff44827bef76d90bf638777daa27a9d/coverage-7.13.5-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:d7cfad2d6d81dd298ab6b89fe72c3b7b05ec7544bdda3b707ddaecff8d25c161", size = 252908, upload-time = "2026-03-17T10:31:21.312Z" },- { url = "https://files.pythonhosted.org/packages/7d/37/7792c2d69854397ca77a55c4646e5897c467928b0e27f2d235d83b5d08c6/coverage-7.13.5-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:e092b9499de38ae0fbfbc603a74660eb6ff3e869e507b50d85a13b6db9863e15", size = 250873, upload-time = "2026-03-17T10:31:23.565Z" },- { url = "https://files.pythonhosted.org/packages/a3/23/bc866fb6163be52a8a9e5d708ba0d3b1283c12158cefca0a8bbb6e247a43/coverage-7.13.5-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:48c39bc4a04d983a54a705a6389512883d4a3b9862991b3617d547940e9f52b1", size = 255030, upload-time = "2026-03-17T10:31:25.58Z" },- { url = "https://files.pythonhosted.org/packages/7d/8b/ef67e1c222ef49860701d346b8bbb70881bef283bd5f6cbba68a39a086c7/coverage-7.13.5-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:2d3807015f138ffea1ed9afeeb8624fd781703f2858b62a8dd8da5a0994c57b6", size = 250694, upload-time = "2026-03-17T10:31:27.316Z" },- { url = "https://files.pythonhosted.org/packages/46/0d/866d1f74f0acddbb906db212e096dee77a8e2158ca5e6bb44729f9d93298/coverage-7.13.5-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:ee2aa19e03161671ec964004fb74b2257805d9710bf14a5c704558b9d8dbaf17", size = 252469, upload-time = "2026-03-17T10:31:29.472Z" },- { url = "https://files.pythonhosted.org/packages/7a/f5/be742fec31118f02ce42b21c6af187ad6a344fed546b56ca60caacc6a9a0/coverage-7.13.5-cp313-cp313-win32.whl", hash = "sha256:ce1998c0483007608c8382f4ff50164bfc5bd07a2246dd272aa4043b75e61e85", size = 222112, upload-time = "2026-03-17T10:31:31.526Z" },- { url = "https://files.pythonhosted.org/packages/66/40/7732d648ab9d069a46e686043241f01206348e2bbf128daea85be4d6414b/coverage-7.13.5-cp313-cp313-win_amd64.whl", hash = "sha256:631efb83f01569670a5e866ceb80fe483e7c159fac6f167e6571522636104a0b", size = 222923, upload-time = "2026-03-17T10:31:33.633Z" },- { url = "https://files.pythonhosted.org/packages/48/af/fea819c12a095781f6ccd504890aaddaf88b8fab263c4940e82c7b770124/coverage-7.13.5-cp313-cp313-win_arm64.whl", hash = "sha256:f4cd16206ad171cbc2470dbea9103cf9a7607d5fe8c242fdf1edf36174020664", size = 221540, upload-time = "2026-03-17T10:31:35.445Z" },- { url = "https://files.pythonhosted.org/packages/23/d2/17879af479df7fbbd44bd528a31692a48f6b25055d16482fdf5cdb633805/coverage-7.13.5-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:0428cbef5783ad91fe240f673cc1f76b25e74bbfe1a13115e4aa30d3f538162d", size = 220262, upload-time = "2026-03-17T10:31:37.184Z" },- { url = "https://files.pythonhosted.org/packages/5b/4c/d20e554f988c8f91d6a02c5118f9abbbf73a8768a3048cb4962230d5743f/coverage-7.13.5-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:e0b216a19534b2427cc201a26c25da4a48633f29a487c61258643e89d28200c0", size = 220617, upload-time = "2026-03-17T10:31:39.245Z" },- { url = "https://files.pythonhosted.org/packages/29/9c/f9f5277b95184f764b24e7231e166dfdb5780a46d408a2ac665969416d61/coverage-7.13.5-cp313-cp313t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:972a9cd27894afe4bc2b1480107054e062df08e671df7c2f18c205e805ccd806", size = 261912, upload-time = "2026-03-17T10:31:41.324Z" },- { url = "https://files.pythonhosted.org/packages/d5/f6/7f1ab39393eeb50cfe4747ae8ef0e4fc564b989225aa1152e13a180d74f8/coverage-7.13.5-cp313-cp313t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:4b59148601efcd2bac8c4dbf1f0ad6391693ccf7a74b8205781751637076aee3", size = 263987, upload-time = "2026-03-17T10:31:43.724Z" },- { url = "https://files.pythonhosted.org/packages/a0/d7/62c084fb489ed9c6fbdf57e006752e7c516ea46fd690e5ed8b8617c7d52e/coverage-7.13.5-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:505d7083c8b0c87a8fa8c07370c285847c1f77739b22e299ad75a6af6c32c5c9", size = 266416, upload-time = "2026-03-17T10:31:45.769Z" },- { url = "https://files.pythonhosted.org/packages/a9/f6/df63d8660e1a0bff6125947afda112a0502736f470d62ca68b288ea762d8/coverage-7.13.5-cp313-cp313t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:60365289c3741e4db327e7baff2a4aaacf22f788e80fa4683393891b70a89fbd", size = 267558, upload-time = "2026-03-17T10:31:48.293Z" },- { url = "https://files.pythonhosted.org/packages/5b/02/353ca81d36779bd108f6d384425f7139ac3c58c750dcfaafe5d0bee6436b/coverage-7.13.5-cp313-cp313t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:1b88c69c8ef5d4b6fe7dea66d6636056a0f6a7527c440e890cf9259011f5e606", size = 261163, upload-time = "2026-03-17T10:31:50.125Z" },- { url = "https://files.pythonhosted.org/packages/2c/16/2e79106d5749bcaf3aee6d309123548e3276517cd7851faa8da213bc61bf/coverage-7.13.5-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:5b13955d31d1633cf9376908089b7cebe7d15ddad7aeaabcbe969a595a97e95e", size = 263981, upload-time = "2026-03-17T10:31:51.961Z" },- { url = "https://files.pythonhosted.org/packages/29/c7/c29e0c59ffa6942030ae6f50b88ae49988e7e8da06de7ecdbf49c6d4feae/coverage-7.13.5-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:f70c9ab2595c56f81a89620e22899eea8b212a4041bd728ac6f4a28bf5d3ddd0", size = 261604, upload-time = "2026-03-17T10:31:53.872Z" },- { url = "https://files.pythonhosted.org/packages/40/48/097cdc3db342f34006a308ab41c3a7c11c3f0d84750d340f45d88a782e00/coverage-7.13.5-cp313-cp313t-musllinux_1_2_ppc64le.whl", hash = "sha256:084b84a8c63e8d6fc7e3931b316a9bcafca1458d753c539db82d31ed20091a87", size = 265321, upload-time = "2026-03-17T10:31:55.997Z" },- { url = "https://files.pythonhosted.org/packages/bb/1f/4994af354689e14fd03a75f8ec85a9a68d94e0188bbdab3fc1516b55e512/coverage-7.13.5-cp313-cp313t-musllinux_1_2_riscv64.whl", hash = "sha256:ad14385487393e386e2ea988b09d62dd42c397662ac2dabc3832d71253eee479", size = 260502, upload-time = "2026-03-17T10:31:58.308Z" },- { url = "https://files.pythonhosted.org/packages/22/c6/9bb9ef55903e628033560885f5c31aa227e46878118b63ab15dc7ba87797/coverage-7.13.5-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:7f2c47b36fe7709a6e83bfadf4eefb90bd25fbe4014d715224c4316f808e59a2", size = 262688, upload-time = "2026-03-17T10:32:00.141Z" },- { url = "https://files.pythonhosted.org/packages/14/4f/f5df9007e50b15e53e01edea486814783a7f019893733d9e4d6caad75557/coverage-7.13.5-cp313-cp313t-win32.whl", hash = "sha256:67e9bc5449801fad0e5dff329499fb090ba4c5800b86805c80617b4e29809b2a", size = 222788, upload-time = "2026-03-17T10:32:02.246Z" },- { url = "https://files.pythonhosted.org/packages/e1/98/aa7fccaa97d0f3192bec013c4e6fd6d294a6ed44b640e6bb61f479e00ed5/coverage-7.13.5-cp313-cp313t-win_amd64.whl", hash = "sha256:da86cdcf10d2519e10cabb8ac2de03da1bcb6e4853790b7fbd48523332e3a819", size = 223851, upload-time = "2026-03-17T10:32:04.416Z" },- { url = "https://files.pythonhosted.org/packages/3d/8b/e5c469f7352651e5f013198e9e21f97510b23de957dd06a84071683b4b60/coverage-7.13.5-cp313-cp313t-win_arm64.whl", hash = "sha256:0ecf12ecb326fe2c339d93fc131816f3a7367d223db37817208905c89bded911", size = 222104, upload-time = "2026-03-17T10:32:06.65Z" },- { url = "https://files.pythonhosted.org/packages/8e/77/39703f0d1d4b478bfd30191d3c14f53caf596fac00efb3f8f6ee23646439/coverage-7.13.5-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:fbabfaceaeb587e16f7008f7795cd80d20ec548dc7f94fbb0d4ec2e038ce563f", size = 219621, upload-time = "2026-03-17T10:32:08.589Z" },- { url = "https://files.pythonhosted.org/packages/e2/3e/51dff36d99ae14639a133d9b164d63e628532e2974d8b1edb99dd1ebc733/coverage-7.13.5-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:9bb2a28101a443669a423b665939381084412b81c3f8c0fcfbac57f4e30b5b8e", size = 219953, upload-time = "2026-03-17T10:32:10.507Z" },- { url = "https://files.pythonhosted.org/packages/6a/6c/1f1917b01eb647c2f2adc9962bd66c79eb978951cab61bdc1acab3290c07/coverage-7.13.5-cp314-cp314-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:bd3a2fbc1c6cccb3c5106140d87cc6a8715110373ef42b63cf5aea29df8c217a", size = 250992, upload-time = "2026-03-17T10:32:12.41Z" },- { url = "https://files.pythonhosted.org/packages/22/e5/06b1f88f42a5a99df42ce61208bdec3bddb3d261412874280a19796fc09c/coverage-7.13.5-cp314-cp314-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:6c36ddb64ed9d7e496028d1d00dfec3e428e0aabf4006583bb1839958d280510", size = 253503, upload-time = "2026-03-17T10:32:14.449Z" },- { url = "https://files.pythonhosted.org/packages/80/28/2a148a51e5907e504fa7b85490277734e6771d8844ebcc48764a15e28155/coverage-7.13.5-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:380e8e9084d8eb38db3a9176a1a4f3c0082c3806fa0dc882d1d87abc3c789247", size = 254852, upload-time = "2026-03-17T10:32:16.56Z" },- { url = "https://files.pythonhosted.org/packages/61/77/50e8d3d85cc0b7ebe09f30f151d670e302c7ff4a1bf6243f71dd8b0981fa/coverage-7.13.5-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:e808af52a0513762df4d945ea164a24b37f2f518cbe97e03deaa0ee66139b4d6", size = 257161, upload-time = "2026-03-17T10:32:19.004Z" },- { url = "https://files.pythonhosted.org/packages/3b/c4/b5fd1d4b7bf8d0e75d997afd3925c59ba629fc8616f1b3aae7605132e256/coverage-7.13.5-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e301d30dd7e95ae068671d746ba8c34e945a82682e62918e41b2679acd2051a0", size = 251021, upload-time = "2026-03-17T10:32:21.344Z" },- { url = "https://files.pythonhosted.org/packages/f8/66/6ea21f910e92d69ef0b1c3346ea5922a51bad4446c9126db2ae96ee24c4c/coverage-7.13.5-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:800bc829053c80d240a687ceeb927a94fd108bbdc68dfbe505d0d75ab578a882", size = 252858, upload-time = "2026-03-17T10:32:23.506Z" },- { url = "https://files.pythonhosted.org/packages/9e/ea/879c83cb5d61aa2a35fb80e72715e92672daef8191b84911a643f533840c/coverage-7.13.5-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:0b67af5492adb31940ee418a5a655c28e48165da5afab8c7fa6fd72a142f8740", size = 250823, upload-time = "2026-03-17T10:32:25.516Z" },- { url = "https://files.pythonhosted.org/packages/8a/fb/616d95d3adb88b9803b275580bdeee8bd1b69a886d057652521f83d7322f/coverage-7.13.5-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:c9136ff29c3a91e25b1d1552b5308e53a1e0653a23e53b6366d7c2dcbbaf8a16", size = 255099, upload-time = "2026-03-17T10:32:27.944Z" },- { url = "https://files.pythonhosted.org/packages/1c/93/25e6917c90ec1c9a56b0b26f6cad6408e5f13bb6b35d484a0d75c9cf000d/coverage-7.13.5-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:cff784eef7f0b8f6cb28804fbddcfa99f89efe4cc35fb5627e3ac58f91ed3ac0", size = 250638, upload-time = "2026-03-17T10:32:29.914Z" },- { url = "https://files.pythonhosted.org/packages/fc/7b/dc1776b0464145a929deed214aef9fb1493f159b59ff3c7eeeedf91eddd0/coverage-7.13.5-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:68a4953be99b17ac3c23b6efbc8a38330d99680c9458927491d18700ef23ded0", size = 252295, upload-time = "2026-03-17T10:32:31.981Z" },- { url = "https://files.pythonhosted.org/packages/ea/fb/99cbbc56a26e07762a2740713f3c8f9f3f3106e3a3dd8cc4474954bccd34/coverage-7.13.5-cp314-cp314-win32.whl", hash = "sha256:35a31f2b1578185fbe6aa2e74cea1b1d0bbf4c552774247d9160d29b80ed56cc", size = 222360, upload-time = "2026-03-17T10:32:34.233Z" },- { url = "https://files.pythonhosted.org/packages/8d/b7/4758d4f73fb536347cc5e4ad63662f9d60ba9118cb6785e9616b2ce5d7fa/coverage-7.13.5-cp314-cp314-win_amd64.whl", hash = "sha256:2aa055ae1857258f9e0045be26a6d62bdb47a72448b62d7b55f4820f361a2633", size = 223174, upload-time = "2026-03-17T10:32:36.369Z" },- { url = "https://files.pythonhosted.org/packages/2c/f2/24d84e1dfe70f8ac9fdf30d338239860d0d1d5da0bda528959d0ebc9da28/coverage-7.13.5-cp314-cp314-win_arm64.whl", hash = "sha256:1b11eef33edeae9d142f9b4358edb76273b3bfd30bc3df9a4f95d0e49caf94e8", size = 221739, upload-time = "2026-03-17T10:32:38.736Z" },- { url = "https://files.pythonhosted.org/packages/60/5b/4a168591057b3668c2428bff25dd3ebc21b629d666d90bcdfa0217940e84/coverage-7.13.5-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:10a0c37f0b646eaff7cce1874c31d1f1ccb297688d4c747291f4f4c70741cc8b", size = 220351, upload-time = "2026-03-17T10:32:41.196Z" },- { url = "https://files.pythonhosted.org/packages/f5/21/1fd5c4dbfe4a58b6b99649125635df46decdfd4a784c3cd6d410d303e370/coverage-7.13.5-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:b5db73ba3c41c7008037fa731ad5459fc3944cb7452fc0aa9f822ad3533c583c", size = 220612, upload-time = "2026-03-17T10:32:43.204Z" },- { url = "https://files.pythonhosted.org/packages/d6/fe/2a924b3055a5e7e4512655a9d4609781b0d62334fa0140c3e742926834e2/coverage-7.13.5-cp314-cp314t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:750db93a81e3e5a9831b534be7b1229df848b2e125a604fe6651e48aa070e5f9", size = 261985, upload-time = "2026-03-17T10:32:45.514Z" },- { url = "https://files.pythonhosted.org/packages/d7/0d/c8928f2bd518c45990fe1a2ab8db42e914ef9b726c975facc4282578c3eb/coverage-7.13.5-cp314-cp314t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:9ddb4f4a5479f2539644be484da179b653273bca1a323947d48ab107b3ed1f29", size = 264107, upload-time = "2026-03-17T10:32:47.971Z" },- { url = "https://files.pythonhosted.org/packages/ef/ae/4ae35bbd9a0af9d820362751f0766582833c211224b38665c0f8de3d487f/coverage-7.13.5-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d8a7a2049c14f413163e2bdabd37e41179b1d1ccb10ffc6ccc4b7a718429c607", size = 266513, upload-time = "2026-03-17T10:32:50.1Z" },- { url = "https://files.pythonhosted.org/packages/9c/20/d326174c55af36f74eac6ae781612d9492f060ce8244b570bb9d50d9d609/coverage-7.13.5-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:e1c85e0b6c05c592ea6d8768a66a254bfb3874b53774b12d4c89c481eb78cb90", size = 267650, upload-time = "2026-03-17T10:32:52.391Z" },- { url = "https://files.pythonhosted.org/packages/7a/5e/31484d62cbd0eabd3412e30d74386ece4a0837d4f6c3040a653878bfc019/coverage-7.13.5-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:777c4d1eff1b67876139d24288aaf1817f6c03d6bae9c5cc8d27b83bcfe38fe3", size = 261089, upload-time = "2026-03-17T10:32:54.544Z" },- { url = "https://files.pythonhosted.org/packages/e9/d8/49a72d6de146eebb0b7e48cc0f4bc2c0dd858e3d4790ab2b39a2872b62bd/coverage-7.13.5-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:6697e29b93707167687543480a40f0db8f356e86d9f67ddf2e37e2dfd91a9dab", size = 263982, upload-time = "2026-03-17T10:32:56.803Z" },- { url = "https://files.pythonhosted.org/packages/06/3b/0351f1bd566e6e4dd39e978efe7958bde1d32f879e85589de147654f57bb/coverage-7.13.5-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:8fdf453a942c3e4d99bd80088141c4c6960bb232c409d9c3558e2dbaa3998562", size = 261579, upload-time = "2026-03-17T10:32:59.466Z" },- { url = "https://files.pythonhosted.org/packages/5d/ce/796a2a2f4017f554d7810f5c573449b35b1e46788424a548d4d19201b222/coverage-7.13.5-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:32ca0c0114c9834a43f045a87dcebd69d108d8ffb666957ea65aa132f50332e2", size = 265316, upload-time = "2026-03-17T10:33:01.847Z" },- { url = "https://files.pythonhosted.org/packages/3d/16/d5ae91455541d1a78bc90abf495be600588aff8f6db5c8b0dae739fa39c9/coverage-7.13.5-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:8769751c10f339021e2638cd354e13adeac54004d1941119b2c96fe5276d45ea", size = 260427, upload-time = "2026-03-17T10:33:03.945Z" },- { url = "https://files.pythonhosted.org/packages/48/11/07f413dba62db21fb3fad5d0de013a50e073cc4e2dc4306e770360f6dfc8/coverage-7.13.5-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:cec2d83125531bd153175354055cdb7a09987af08a9430bd173c937c6d0fba2a", size = 262745, upload-time = "2026-03-17T10:33:06.285Z" },- { url = "https://files.pythonhosted.org/packages/91/15/d792371332eb4663115becf4bad47e047d16234b1aff687b1b18c58d60ae/coverage-7.13.5-cp314-cp314t-win32.whl", hash = "sha256:0cd9ed7a8b181775459296e402ca4fb27db1279740a24e93b3b41942ebe4b215", size = 223146, upload-time = "2026-03-17T10:33:08.756Z" },- { url = "https://files.pythonhosted.org/packages/db/51/37221f59a111dca5e85be7dbf09696323b5b9f13ff65e0641d535ed06ea8/coverage-7.13.5-cp314-cp314t-win_amd64.whl", hash = "sha256:301e3b7dfefecaca37c9f1aa6f0049b7d4ab8dd933742b607765d757aca77d43", size = 224254, upload-time = "2026-03-17T10:33:11.174Z" },- { url = "https://files.pythonhosted.org/packages/54/83/6acacc889de8987441aa7d5adfbdbf33d288dad28704a67e574f1df9bcbb/coverage-7.13.5-cp314-cp314t-win_arm64.whl", hash = "sha256:9dacc2ad679b292709e0f5fc1ac74a6d4d5562e424058962c7bb0c658ad25e45", size = 222276, upload-time = "2026-03-17T10:33:13.466Z" },- { url = "https://files.pythonhosted.org/packages/9e/ee/a4cf96b8ce1e566ed238f0659ac2d3f007ed1d14b181bcb684e19561a69a/coverage-7.13.5-py3-none-any.whl", hash = "sha256:34b02417cf070e173989b3db962f7ed56d2f644307b2cf9d5a0f258e13084a61", size = 211346, upload-time = "2026-03-17T10:33:15.691Z" },-]-[[package]]name = "cryptography"version = "48.0.0"@@ -562,53 +450,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/65/99/00f3196036501b53032c4b1ab8337a0b978dee832ed276dae3815df4e8b5/datasets-4.8.5-py3-none-any.whl", hash = "sha256:5079900781719c0e063a8efdd2cd95a31ad0c63209178669cd23cf1b926149ff", size = 528973, upload-time = "2026-04-27T15:43:53.702Z" },]-[[package]]-name = "ddtrace"-version = "3.19.7"-source = { registry = "https://pypi.org/simple" }-dependencies = [- { name = "bytecode" },- { name = "envier" },- { name = "opentelemetry-api" },- { name = "wrapt" },-]-sdist = { url = "https://files.pythonhosted.org/packages/19/0c/05ab6cf1db00b89fc4f86d7d4b262bc4a17e547ee4f09086c76916976211/ddtrace-3.19.7.tar.gz", hash = "sha256:e531a7c6370458d0e61b498714969471514033f6851a4603ac62f2cac1adc925", size = 7653384, upload-time = "2026-03-13T19:03:53.67Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/3b/00/beadc842ac96cc872c744c9f55d940a1f9dc0cbbfe8663415f2b273db34c/ddtrace-3.19.7-cp312-cp312-macosx_12_0_arm64.whl", hash = "sha256:2a1b4ccf33d82cc03334f6f54b8e37711f0e46fddaade56769ac42e6c6683d98", size = 6538107, upload-time = "2026-03-13T19:01:08.903Z" },- { url = "https://files.pythonhosted.org/packages/92/0d/4dde240e6a70b132728399ff196aab33da07dd5be3191e53d7d9e7d8b2d0/ddtrace-3.19.7-cp312-cp312-macosx_12_0_x86_64.whl", hash = "sha256:b6037dba7f0b7fa59b8e1ab44a2750fccc3e619917d451a94cc1bc108b16d3cb", size = 6944628, upload-time = "2026-03-13T19:01:11.159Z" },- { url = "https://files.pythonhosted.org/packages/76/78/770178a9f826b115e9c1ce8d053873d30e231db82856b307c1e87879b044/ddtrace-3.19.7-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:38b515c8b5100e6b3c513e58160eb268bddbd9783fa2cb6f7f65ee7c387936ee", size = 7531273, upload-time = "2026-03-13T19:01:13.771Z" },- { url = "https://files.pythonhosted.org/packages/10/43/9601599f7988bd12b59f083dbaec321f1ce66f27eae06b8afb90841917d8/ddtrace-3.19.7-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:300807941a2f005aacf4ef3d81e94dd4034d9acf1bf3786e73c382659bd8b617", size = 7836056, upload-time = "2026-03-13T19:01:15.991Z" },- { url = "https://files.pythonhosted.org/packages/b7/56/8e60e9a0f46f53595936576fac97c1689873498518e68382fa54fcd7d158/ddtrace-3.19.7-cp312-cp312-manylinux_2_28_i686.whl", hash = "sha256:c74524559e7d21beb29b8e13c5090644525b5df03a6a7450e9286997e31ac9c9", size = 5622525, upload-time = "2026-03-13T19:01:18.284Z" },- { url = "https://files.pythonhosted.org/packages/6a/4a/3cf7dccd4011e5030f507fe765495b375a6e4d23851fe7811d72395f6e5a/ddtrace-3.19.7-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:817abdd18b24ec0f881843af4c6bae55c4ecd997dc0c3688c4a45b0a18a1368c", size = 8588806, upload-time = "2026-03-13T19:01:20.554Z" },- { url = "https://files.pythonhosted.org/packages/f6/92/3d1c0bcb7412586c39c3ac5226cd87ae6819ebcb3a8edce2f6eed29b19f6/ddtrace-3.19.7-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:c59faed5fa5330ffd5fe60284b1ec1a0525d7fc063787de227a418810851c2f3", size = 6677414, upload-time = "2026-03-13T19:01:23.07Z" },- { url = "https://files.pythonhosted.org/packages/4f/0e/8a4d7211f1d35082956cab1437c2c81245158547677445017a2130782869/ddtrace-3.19.7-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:af3f1200c3e4c850a9c408047a6f83f5b87cbb0c641022a4107e566e5064c434", size = 8897744, upload-time = "2026-03-13T19:01:25.702Z" },- { url = "https://files.pythonhosted.org/packages/d0/65/4733cf6cd78c00f34cbce8fdba86d26c5adbd6613e049d5f3430449300ae/ddtrace-3.19.7-cp312-cp312-win32.whl", hash = "sha256:6fd3409777c2549b5bf743abc3c1ab17ba7a1a399c3a580e2a4bac383a576e35", size = 5116495, upload-time = "2026-03-13T19:01:28.243Z" },- { url = "https://files.pythonhosted.org/packages/dc/49/fa81f8084d43011ce4e80a3a52dcf144b5069ee7f8f191d41f0d1dcc7d6f/ddtrace-3.19.7-cp312-cp312-win_amd64.whl", hash = "sha256:69448c80f440aee4d1a3e19295e66c44a84ae014da6c05a52d846b4b7ac821c9", size = 5611097, upload-time = "2026-03-13T19:01:30.848Z" },- { url = "https://files.pythonhosted.org/packages/48/d8/3c3a1e67df4c54b5d2585ad180d9eaf5a1d05a98e81cedf8b2b69930a0d6/ddtrace-3.19.7-cp312-cp312-win_arm64.whl", hash = "sha256:30827b6bbbbb75f883717c59339e43feef31a7a31da623f2f291ac6e9a044c6f", size = 5334126, upload-time = "2026-03-13T19:01:33.526Z" },- { url = "https://files.pythonhosted.org/packages/de/30/fe91d89021ceeb7a41bcbc8a05f3c54fc619ea482fd14257f423e1e1aa0e/ddtrace-3.19.7-cp313-cp313-macosx_12_0_arm64.whl", hash = "sha256:425bacf644a772aa3faca075234f3bc5d37972fa1f32932343b00653190cdab5", size = 6532341, upload-time = "2026-03-13T19:01:35.891Z" },- { url = "https://files.pythonhosted.org/packages/4a/b1/99664152487bd8612169506cc33f9f3fabe988884bf2fdcb3877f169df8d/ddtrace-3.19.7-cp313-cp313-macosx_12_0_x86_64.whl", hash = "sha256:dec566649d660bfe179d2d97d8217304a466c0294ac6cec84ab9ee5bd2c20056", size = 6940282, upload-time = "2026-03-13T19:01:38.642Z" },- { url = "https://files.pythonhosted.org/packages/8d/42/191d84881dcd99ba32577efa3cc4e4d3bd519629798421fa8a6165e679af/ddtrace-3.19.7-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:91e62d9d206150ddc360af2628615572f3bcc314ae420ab59041e6e493477993", size = 7520474, upload-time = "2026-03-13T19:01:41.157Z" },- { url = "https://files.pythonhosted.org/packages/cb/4d/4413bacc2448629223effaa9cc14c8830872853bcd7e2673e186179c32cf/ddtrace-3.19.7-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:251607dec183356cfe1ec8f102f01ac556a38136216f74e31b5748b6fa994d97", size = 7825317, upload-time = "2026-03-13T19:01:43.87Z" },- { url = "https://files.pythonhosted.org/packages/bc/d2/a7ed2c22b9069fe7cb0797f04b983f0dd17ce06ab581cd5e5433dd69ea3a/ddtrace-3.19.7-cp313-cp313-manylinux_2_28_i686.whl", hash = "sha256:6952e2e78f126dce55d4d988490bc76c24e439d52d2686ff618f8d7640534e61", size = 5611950, upload-time = "2026-03-13T19:01:46.47Z" },- { url = "https://files.pythonhosted.org/packages/5d/bf/6caa894b8ef814bba6f5514d6249f0ab33e61173d795dd34a2114de39aa1/ddtrace-3.19.7-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:82b7fbfedac7986274d2c947d247affdb9d918edcb6d65587ab9c1d8d81af9bf", size = 8583032, upload-time = "2026-03-13T19:01:50.098Z" },- { url = "https://files.pythonhosted.org/packages/5c/5b/761df8107c1689ab09feeb46c1791a7ebed5c248b4df620dcda2c6f6e722/ddtrace-3.19.7-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:b96f689ab0225d2d8934c4aa42a7ce1001b91fbf1ff8e8158a6428e0f0faece6", size = 6668576, upload-time = "2026-03-13T19:01:53.456Z" },- { url = "https://files.pythonhosted.org/packages/88/90/cc40e6ce5888782c972d2aaa81828648b0c24db2e84c34134274a8d50ce9/ddtrace-3.19.7-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:9a0995ac17c3e43fae6f6c2ac0d9a90d035ea13d629342f84ba7596ac002ce87", size = 8889877, upload-time = "2026-03-13T19:01:56.557Z" },- { url = "https://files.pythonhosted.org/packages/14/d3/0b396f24703521b102f6220869b4931d6388c55402c16e8ace8c155d6871/ddtrace-3.19.7-cp313-cp313-win32.whl", hash = "sha256:846b695964580cb4d91c4600907119f38b57b792d7ff1f21793d2140157b820b", size = 5113764, upload-time = "2026-03-13T19:01:59.688Z" },- { url = "https://files.pythonhosted.org/packages/6e/67/12806c2523c22d394e072f0d696dafe794b1ac3f441d3cb0b3e37ed4b770/ddtrace-3.19.7-cp313-cp313-win_amd64.whl", hash = "sha256:de1aabc50c663f27c1ae341d16b31bdc3b8fb1c28c4b8a683e9000cbfd11fa44", size = 5607876, upload-time = "2026-03-13T19:02:02.588Z" },- { url = "https://files.pythonhosted.org/packages/ba/c3/27bff2e1ba9bc8323516d6c6bf379f5de8c0e3dddd672f2fb82bc58fc99d/ddtrace-3.19.7-cp313-cp313-win_arm64.whl", hash = "sha256:2d41fd500ec915be8b3b95cd7a5535f87323acbdc7e9b2c02797d83b6985b451", size = 5331124, upload-time = "2026-03-13T19:02:05.267Z" },- { url = "https://files.pythonhosted.org/packages/6a/01/34d3af11ee01870366af5a71aa5b1d1ce342fdad6f030a37538aa5d21961/ddtrace-3.19.7-cp314-cp314-macosx_12_0_arm64.whl", hash = "sha256:528fb2565048e669cfe5ad0e24f4f39f73747977982c78339f9ece7e39430180", size = 6090587, upload-time = "2026-03-13T19:02:08.355Z" },- { url = "https://files.pythonhosted.org/packages/81/3b/1dcf930c60e7e1393e86ce2fb9202b6e3b3eeb01aeccb92b2170768fdcd8/ddtrace-3.19.7-cp314-cp314-macosx_12_0_x86_64.whl", hash = "sha256:29202bdd4496098b27907dfdfd85000549901461ebea6f0f90d19eb4ebac2b76", size = 6433258, upload-time = "2026-03-13T19:02:11.192Z" },- { url = "https://files.pythonhosted.org/packages/90/56/0c45a0aac2c728acb6500a87155bad3dc18bd9990909fb0388eac7d81f47/ddtrace-3.19.7-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:abb3b5b8cac3247b01d381fe71aeb56b20f8759275619701418e53f0306eb6a0", size = 7125123, upload-time = "2026-03-13T19:02:14.001Z" },- { url = "https://files.pythonhosted.org/packages/b9/bd/5b6c42d8c23390e24210988a01d317732e72cbb063850220aa99827f86b4/ddtrace-3.19.7-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:7d27d310e272aa7772fdf7087bd5056966448d8051586de75577076ad8c8e323", size = 7357163, upload-time = "2026-03-13T19:02:16.849Z" },- { url = "https://files.pythonhosted.org/packages/f3/d2/832cfa6b2cd1d16038997853128fbf0989b197695b36b61974ac2b1c720b/ddtrace-3.19.7-cp314-cp314-manylinux_2_28_i686.whl", hash = "sha256:aec1ec1fbcdafa76362c7f468aab240224d14060c4cda73e9a0676293be471c5", size = 5614026, upload-time = "2026-03-13T19:02:19.648Z" },- { url = "https://files.pythonhosted.org/packages/dc/ca/edfa36f942d043e6b5cbf9312bcae3e5b22c785b9db894d37aff608e1895/ddtrace-3.19.7-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:26c300dbb6d8c29b6fcd4847da29010a988150a28426c4cc46e7e1f3816ea195", size = 8095787, upload-time = "2026-03-13T19:02:22.838Z" },- { url = "https://files.pythonhosted.org/packages/fc/91/173203d14cdd5e6084290266f05c550fef96e38006bca486584f870f7fc4/ddtrace-3.19.7-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:8c3a74eda7907e5d4a5a36c62e02ee41fca10de83ae2f592e0a63530d8eda7aa", size = 6669581, upload-time = "2026-03-13T19:02:25.684Z" },- { url = "https://files.pythonhosted.org/packages/a3/61/b33224286d3ee39686e4e1be85a9da2ecd58b4e8661875fb9fd113e73f34/ddtrace-3.19.7-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:79cdb4246da2a6e32b768a0c7f952890ddeb9c8bea9228adf2d82bcf66849177", size = 8438684, upload-time = "2026-03-13T19:02:28.648Z" },- { url = "https://files.pythonhosted.org/packages/7d/d9/88b83c185fe26849aff109d80f9c93eca170a6365a2a1885756d90b3213d/ddtrace-3.19.7-cp314-cp314-win32.whl", hash = "sha256:759d49298575079e31da354780ca37220775c80a007bbe2ae7fb756a11210740", size = 5208198, upload-time = "2026-03-13T19:02:31.979Z" },- { url = "https://files.pythonhosted.org/packages/45/a1/57d6266163c180272f2eb0191c93ec8d4bf6b67476817dab5f5749411aab/ddtrace-3.19.7-cp314-cp314-win_amd64.whl", hash = "sha256:054ebbdd1e02e90874cda803b31721f4e4ab6d0d3833bc8e82de8b78f1fc9d57", size = 5740240, upload-time = "2026-03-13T19:02:34.79Z" },- { url = "https://files.pythonhosted.org/packages/a9/16/6c269f658c8b2e79083ecc8af1dd8cfb64ea031492eaf040a13a4ac86178/ddtrace-3.19.7-cp314-cp314-win_arm64.whl", hash = "sha256:ff953b2d7dc8a7d8e8d452aa0f9ac1e094e9deba371f3715bc73613bc1fa0069", size = 5475688, upload-time = "2026-03-13T19:02:37.758Z" },-]-[[package]]name = "deprecation"version = "2.1.0"@@ -651,24 +492,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/12/b3/231ffd4ab1fc9d679809f356cebee130ac7daa00d6d6f3206dd4fd137e9e/distro-1.9.0-py3-none-any.whl", hash = "sha256:7bffd925d65168f85027d8da9af6bddab658135b840670a223589bc0c8ef02b2", size = 20277, upload-time = "2023-12-24T09:54:30.421Z" },]-[[package]]-name = "docstring-parser"-version = "0.18.0"-source = { registry = "https://pypi.org/simple" }-sdist = { url = "https://files.pythonhosted.org/packages/e0/4d/f332313098c1de1b2d2ff91cf2674415cc7cddab2ca1b01ae29774bd5fdf/docstring_parser-0.18.0.tar.gz", hash = "sha256:292510982205c12b1248696f44959db3cdd1740237a968ea1e2e7a900eeb2015", size = 29341, upload-time = "2026-04-14T04:09:19.867Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/a7/5f/ed01f9a3cdffbd5a008556fc7b2a08ddb1cc6ace7effa7340604b1d16699/docstring_parser-0.18.0-py3-none-any.whl", hash = "sha256:b3fcbed555c47d8479be0796ef7e19c2670d428d72e96da63f3a40122860374b", size = 22484, upload-time = "2026-04-14T04:09:18.638Z" },-]--[[package]]-name = "envier"-version = "0.6.1"-source = { registry = "https://pypi.org/simple" }-sdist = { url = "https://files.pythonhosted.org/packages/19/e7/4fe4d3f6e21213cea9bcddc36ba60e6ae4003035f9ce8055e6a9f0322ddb/envier-0.6.1.tar.gz", hash = "sha256:3309a01bb3d8850c9e7a31a5166d5a836846db2faecb79b9cb32654dd50ca9f9", size = 10063, upload-time = "2024-10-22T09:56:47.226Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/56/e9/30493b1cc967f7c07869de4b2ab3929151a58e6bb04495015554d24b61db/envier-0.6.1-py3-none-any.whl", hash = "sha256:73609040a76be48bbcb97074d9969666484aa0de706183a6e9ef773156a8a6a9", size = 10638, upload-time = "2024-10-22T09:56:45.968Z" },-]-[[package]]name = "fastapi"version = "0.136.1"@@ -954,35 +777,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/7e/f5/f66802a942d491edb555dd61e3a9961140fd64c90bce1eafd741609d334d/httpcore-1.0.9-py3-none-any.whl", hash = "sha256:2d400746a40668fc9dec9810239072b40b4484b640a8c38fd654a024c7a1bf55", size = 78784, upload-time = "2025-04-24T22:06:20.566Z" },]-[[package]]-name = "httptools"-version = "0.7.1"-source = { registry = "https://pypi.org/simple" }-sdist = { url = "https://files.pythonhosted.org/packages/b5/46/120a669232c7bdedb9d52d4aeae7e6c7dfe151e99dc70802e2fc7a5e1993/httptools-0.7.1.tar.gz", hash = "sha256:abd72556974f8e7c74a259655924a717a2365b236c882c3f6f8a45fe94703ac9", size = 258961, upload-time = "2025-10-10T03:55:08.559Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/53/7f/403e5d787dc4942316e515e949b0c8a013d84078a915910e9f391ba9b3ed/httptools-0.7.1-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:38e0c83a2ea9746ebbd643bdfb521b9aa4a91703e2cd705c20443405d2fd16a5", size = 206280, upload-time = "2025-10-10T03:54:39.274Z" },- { url = "https://files.pythonhosted.org/packages/2a/0d/7f3fd28e2ce311ccc998c388dd1c53b18120fda3b70ebb022b135dc9839b/httptools-0.7.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:f25bbaf1235e27704f1a7b86cd3304eabc04f569c828101d94a0e605ef7205a5", size = 110004, upload-time = "2025-10-10T03:54:40.403Z" },- { url = "https://files.pythonhosted.org/packages/84/a6/b3965e1e146ef5762870bbe76117876ceba51a201e18cc31f5703e454596/httptools-0.7.1-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:2c15f37ef679ab9ecc06bfc4e6e8628c32a8e4b305459de7cf6785acd57e4d03", size = 517655, upload-time = "2025-10-10T03:54:41.347Z" },- { url = "https://files.pythonhosted.org/packages/11/7d/71fee6f1844e6fa378f2eddde6c3e41ce3a1fb4b2d81118dd544e3441ec0/httptools-0.7.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7fe6e96090df46b36ccfaf746f03034e5ab723162bc51b0a4cf58305324036f2", size = 511440, upload-time = "2025-10-10T03:54:42.452Z" },- { url = "https://files.pythonhosted.org/packages/22/a5/079d216712a4f3ffa24af4a0381b108aa9c45b7a5cc6eb141f81726b1823/httptools-0.7.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:f72fdbae2dbc6e68b8239defb48e6a5937b12218e6ffc2c7846cc37befa84362", size = 495186, upload-time = "2025-10-10T03:54:43.937Z" },- { url = "https://files.pythonhosted.org/packages/e9/9e/025ad7b65278745dee3bd0ebf9314934c4592560878308a6121f7f812084/httptools-0.7.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:e99c7b90a29fd82fea9ef57943d501a16f3404d7b9ee81799d41639bdaae412c", size = 499192, upload-time = "2025-10-10T03:54:45.003Z" },- { url = "https://files.pythonhosted.org/packages/6d/de/40a8f202b987d43afc4d54689600ff03ce65680ede2f31df348d7f368b8f/httptools-0.7.1-cp312-cp312-win_amd64.whl", hash = "sha256:3e14f530fefa7499334a79b0cf7e7cd2992870eb893526fb097d51b4f2d0f321", size = 86694, upload-time = "2025-10-10T03:54:45.923Z" },- { url = "https://files.pythonhosted.org/packages/09/8f/c77b1fcbfd262d422f12da02feb0d218fa228d52485b77b953832105bb90/httptools-0.7.1-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:6babce6cfa2a99545c60bfef8bee0cc0545413cb0018f617c8059a30ad985de3", size = 202889, upload-time = "2025-10-10T03:54:47.089Z" },- { url = "https://files.pythonhosted.org/packages/0a/1a/22887f53602feaa066354867bc49a68fc295c2293433177ee90870a7d517/httptools-0.7.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:601b7628de7504077dd3dcb3791c6b8694bbd967148a6d1f01806509254fb1ca", size = 108180, upload-time = "2025-10-10T03:54:48.052Z" },- { url = "https://files.pythonhosted.org/packages/32/6a/6aaa91937f0010d288d3d124ca2946d48d60c3a5ee7ca62afe870e3ea011/httptools-0.7.1-cp313-cp313-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:04c6c0e6c5fb0739c5b8a9eb046d298650a0ff38cf42537fc372b28dc7e4472c", size = 478596, upload-time = "2025-10-10T03:54:48.919Z" },- { url = "https://files.pythonhosted.org/packages/6d/70/023d7ce117993107be88d2cbca566a7c1323ccbaf0af7eabf2064fe356f6/httptools-0.7.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:69d4f9705c405ae3ee83d6a12283dc9feba8cc6aaec671b412917e644ab4fa66", size = 473268, upload-time = "2025-10-10T03:54:49.993Z" },- { url = "https://files.pythonhosted.org/packages/32/4d/9dd616c38da088e3f436e9a616e1d0cc66544b8cdac405cc4e81c8679fc7/httptools-0.7.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:44c8f4347d4b31269c8a9205d8a5ee2df5322b09bbbd30f8f862185bb6b05346", size = 455517, upload-time = "2025-10-10T03:54:51.066Z" },- { url = "https://files.pythonhosted.org/packages/1d/3a/a6c595c310b7df958e739aae88724e24f9246a514d909547778d776799be/httptools-0.7.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:465275d76db4d554918aba40bf1cbebe324670f3dfc979eaffaa5d108e2ed650", size = 458337, upload-time = "2025-10-10T03:54:52.196Z" },- { url = "https://files.pythonhosted.org/packages/fd/82/88e8d6d2c51edc1cc391b6e044c6c435b6aebe97b1abc33db1b0b24cd582/httptools-0.7.1-cp313-cp313-win_amd64.whl", hash = "sha256:322d00c2068d125bd570f7bf78b2d367dad02b919d8581d7476d8b75b294e3e6", size = 85743, upload-time = "2025-10-10T03:54:53.448Z" },- { url = "https://files.pythonhosted.org/packages/34/50/9d095fcbb6de2d523e027a2f304d4551855c2f46e0b82befd718b8b20056/httptools-0.7.1-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:c08fe65728b8d70b6923ce31e3956f859d5e1e8548e6f22ec520a962c6757270", size = 203619, upload-time = "2025-10-10T03:54:54.321Z" },- { url = "https://files.pythonhosted.org/packages/07/f0/89720dc5139ae54b03f861b5e2c55a37dba9a5da7d51e1e824a1f343627f/httptools-0.7.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:7aea2e3c3953521c3c51106ee11487a910d45586e351202474d45472db7d72d3", size = 108714, upload-time = "2025-10-10T03:54:55.163Z" },- { url = "https://files.pythonhosted.org/packages/b3/cb/eea88506f191fb552c11787c23f9a405f4c7b0c5799bf73f2249cd4f5228/httptools-0.7.1-cp314-cp314-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:0e68b8582f4ea9166be62926077a3334064d422cf08ab87d8b74664f8e9058e1", size = 472909, upload-time = "2025-10-10T03:54:56.056Z" },- { url = "https://files.pythonhosted.org/packages/e0/4a/a548bdfae6369c0d078bab5769f7b66f17f1bfaa6fa28f81d6be6959066b/httptools-0.7.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:df091cf961a3be783d6aebae963cc9b71e00d57fa6f149025075217bc6a55a7b", size = 470831, upload-time = "2025-10-10T03:54:57.219Z" },- { url = "https://files.pythonhosted.org/packages/4d/31/14df99e1c43bd132eec921c2e7e11cda7852f65619bc0fc5bdc2d0cb126c/httptools-0.7.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:f084813239e1eb403ddacd06a30de3d3e09a9b76e7894dcda2b22f8a726e9c60", size = 452631, upload-time = "2025-10-10T03:54:58.219Z" },- { url = "https://files.pythonhosted.org/packages/22/d2/b7e131f7be8d854d48cb6d048113c30f9a46dca0c9a8b08fcb3fcd588cdc/httptools-0.7.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:7347714368fb2b335e9063bc2b96f2f87a9ceffcd9758ac295f8bbcd3ffbc0ca", size = 452910, upload-time = "2025-10-10T03:54:59.366Z" },- { url = "https://files.pythonhosted.org/packages/53/cf/878f3b91e4e6e011eff6d1fa9ca39f7eb17d19c9d7971b04873734112f30/httptools-0.7.1-cp314-cp314-win_amd64.whl", hash = "sha256:cfabda2a5bb85aa2a904ce06d974a3f30fb36cc63d7feaddec05d2050acede96", size = 88205, upload-time = "2025-10-10T03:55:00.389Z" },-]-[[package]]name = "httpx"version = "0.28.1"@@ -1736,50 +1530,23 @@ wheels = [name = "nemo-switchyard"version = "0.2.0"source = { editable = "." }-dependencies = [- { name = "anthropic" },- { name = "httpx" },- { name = "openai" },- { name = "pydantic" },-][package.optional-dependencies]-all = [- { name = "ddtrace" },- { name = "fastapi" },- { name = "prompt-toolkit" },- { name = "sse-starlette" },- { name = "uvicorn", extra = ["standard"] },-]cli = [{ name = "prompt-toolkit" },]-server = [- { name = "fastapi" },- { name = "sse-starlette" },- { name = "uvicorn", extra = ["standard"] },-]-tracing = [- { name = "ddtrace" },-][package.dev-dependencies]dev = [{ name = "harbor" },- { name = "httpx" },{ name = "maturin" },{ name = "mypy" },- { name = "nemo-switchyard", extra = ["server"] },- { name = "prometheus-client" },+ { name = "nemo-switchyard", extra = ["cli"] },{ name = "pytest" },{ name = "pytest-asyncio" },- { name = "pytest-cov" },{ name = "pytest-markdown-docs" },- { name = "pytest-mock" },{ name = "pytest-timeout" },- { name = "respx" },{ name = "ruff" },- { name = "socksio" },]docs = [{ name = "mkdocs" },@@ -1788,37 +1555,20 @@ docs = [][package.metadata]-requires-dist = [- { name = "anthropic", specifier = ">=0.99.0,<1.0" },- { name = "ddtrace", marker = "extra == 'tracing'", specifier = ">=2.9,<4" },- { name = "fastapi", marker = "extra == 'server'", specifier = ">=0.136.1,<1.0" },- { name = "httpx", specifier = ">=0.28.1,<1.0" },- { name = "nemo-switchyard", extras = ["server", "cli", "tracing"], marker = "extra == 'all'" },- { name = "openai", specifier = ">=2.7,<3.0" },- { name = "prompt-toolkit", marker = "extra == 'cli'", specifier = ">=3.0.52,<4.0" },- { name = "pydantic", specifier = ">=2.13.3,<3.0" },- { name = "sse-starlette", marker = "extra == 'server'", specifier = ">=3.4.1,<4.0" },- { name = "uvicorn", extras = ["standard"], marker = "extra == 'server'", specifier = ">=0.46.0,<1.0" },-]-provides-extras = ["server", "cli", "tracing", "all"]+requires-dist = [{ name = "prompt-toolkit", marker = "extra == 'cli'", specifier = ">=3.0.52,<4.0" }]+provides-extras = ["cli"][package.metadata.requires-dev]dev = [{ name = "harbor", marker = "python_full_version >= '3.12'", git = "https://github.com/harbor-framework/harbor.git?rev=v0.6.4" },- { name = "httpx", specifier = ">=0.28.1,<1.0" },{ name = "maturin", specifier = ">=1.9,<2.0" },{ name = "mypy", specifier = ">=1.20.2,<2.0" },- { name = "nemo-switchyard", extras = ["server"] },- { name = "prometheus-client", specifier = ">=0.21.0,<1.0" },+ { name = "nemo-switchyard", extras = ["cli"] },{ name = "pytest", specifier = ">=9.0.3,<10.0" },{ name = "pytest-asyncio", specifier = ">=1.3.0,<2.0" },- { name = "pytest-cov", specifier = ">=7.1.0,<8.0" },{ name = "pytest-markdown-docs", specifier = ">=0.9.2" },- { name = "pytest-mock", specifier = ">=3.15.1,<4.0" },{ name = "pytest-timeout", specifier = ">=2.4.0,<3.0" },- { name = "respx", specifier = ">=0.23.1,<1.0" },{ name = "ruff", specifier = ">=0.15.12,<1.0" },- { name = "socksio", specifier = ">=1.0.0,<2.0" },]docs = [{ name = "mkdocs", specifier = ">=1.6.0,<2.0" },@@ -1906,18 +1656,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/f2/40/f090499f10514515081d09cb9da09f25b821eb20497e9423afe4f07b4ecf/openai-2.34.0-py3-none-any.whl", hash = "sha256:c996a71b1a210f3569844572ad4c609307e978515fb76877cf449b72596e549e", size = 1316535, upload-time = "2026-05-04T17:34:06.773Z" },]-[[package]]-name = "opentelemetry-api"-version = "1.42.1"-source = { registry = "https://pypi.org/simple" }-dependencies = [- { name = "typing-extensions" },-]-sdist = { url = "https://files.pythonhosted.org/packages/b4/1c/125e1c936c0873796771b7f04f6c93b9f1bf5d424cea90fda94a99f61da8/opentelemetry_api-1.42.1.tar.gz", hash = "sha256:56c63bea9f77b62856be8c47600474acad853b2924b99b1687c4cb6297166716", size = 72296, upload-time = "2026-05-21T16:32:49.335Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/a3/ca/9520cc1f3dfbbd03ac5903bbf55833e257bc64b1cf30fa8b0d6df374d821/opentelemetry_api-1.42.1-py3-none-any.whl", hash = "sha256:51a69edacadbc03a8950ace1c4c21099cacc538820ac2c9e36277e78cebba714", size = 61311, upload-time = "2026-05-21T16:32:28.822Z" },-]-[[package]]name = "packaging"version = "26.2"@@ -2030,15 +1768,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/22/aa/ff2e09f99f95ea96fddeb373646bf907dd89a24fc00b5d38e5674ca7c9ca/postgrest-2.30.0-py3-none-any.whl", hash = "sha256:30631e7993da542419f4217cf3b60aa641084731ea15e66a18526a3a52e40a7d", size = 23108, upload-time = "2026-05-06T17:35:20.531Z" },]-[[package]]-name = "prometheus-client"-version = "0.25.0"-source = { registry = "https://pypi.org/simple" }-sdist = { url = "https://files.pythonhosted.org/packages/1b/fb/d9aa83ffe43ce1f19e557c0971d04b90561b0cfd50762aafb01968285553/prometheus_client-0.25.0.tar.gz", hash = "sha256:5e373b75c31afb3c86f1a52fa1ad470c9aace18082d39ec0d2f918d11cc9ba28", size = 86035, upload-time = "2026-04-09T19:53:42.359Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/8d/9b/d4b1e644385499c8346fa9b622a3f030dce14cd6ef8a1871c221a17a67e7/prometheus_client-0.25.0-py3-none-any.whl", hash = "sha256:d5aec89e349a6ec230805d0df882f3807f74fd6c1a2fa86864e3c2279059fed1", size = 64154, upload-time = "2026-04-09T19:53:41.324Z" },-]-[[package]]name = "prompt-toolkit"version = "3.0.52"@@ -2457,20 +2186,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl", hash = "sha256:611e26147c7f77640e6d0a92a38ed17c3e9848063698d5c93d5aa7aa11cebff5", size = 15075, upload-time = "2025-11-10T16:07:45.537Z" },]-[[package]]-name = "pytest-cov"-version = "7.1.0"-source = { registry = "https://pypi.org/simple" }-dependencies = [- { name = "coverage" },- { name = "pluggy" },- { name = "pytest" },-]-sdist = { url = "https://files.pythonhosted.org/packages/b1/51/a849f96e117386044471c8ec2bd6cfebacda285da9525c9106aeb28da671/pytest_cov-7.1.0.tar.gz", hash = "sha256:30674f2b5f6351aa09702a9c8c364f6a01c27aae0c1366ae8016160d1efc56b2", size = 55592, upload-time = "2026-03-21T20:11:16.284Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/9d/7a/d968e294073affff457b041c2be9868a40c1c71f4a35fcc1e45e5493067b/pytest_cov-7.1.0-py3-none-any.whl", hash = "sha256:a0461110b7865f9a271aa1b51e516c9a95de9d696734a2f71e3e78f46e1d4678", size = 22876, upload-time = "2026-03-21T20:11:14.438Z" },-]-[[package]]name = "pytest-markdown-docs"version = "0.9.2"@@ -2484,18 +2199,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/0c/b9/c3df11997d29e69b3f8edae1e903bf44eaf4774ccf4c5b6ddcebde88931c/pytest_markdown_docs-0.9.2-py3-none-any.whl", hash = "sha256:9c05a5bee48214cb36583d4a5131d3a23b327a8d82c92ae860106053fd7d1f9e", size = 13536, upload-time = "2026-03-23T12:35:03.862Z" },]-[[package]]-name = "pytest-mock"-version = "3.15.1"-source = { registry = "https://pypi.org/simple" }-dependencies = [- { name = "pytest" },-]-sdist = { url = "https://files.pythonhosted.org/packages/68/14/eb014d26be205d38ad5ad20d9a80f7d201472e08167f0bb4361e251084a9/pytest_mock-3.15.1.tar.gz", hash = "sha256:1849a238f6f396da19762269de72cb1814ab44416fa73a8686deac10b0d87a0f", size = 34036, upload-time = "2025-09-16T16:37:27.081Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/5a/cc/06253936f4a7fa2e0f48dfe6d851d9c56df896a9ab09ac019d70b760619c/pytest_mock-3.15.1-py3-none-any.whl", hash = "sha256:0a25e2eb88fe5168d535041d09a4529a188176ae608a6d249ee65abc0949630d", size = 10095, upload-time = "2025-09-16T16:37:25.734Z" },-]-[[package]]name = "pytest-timeout"version = "2.4.0"@@ -2743,18 +2446,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/d7/8e/7540e8a2036f79a125c1d2ebadf69ed7901608859186c856fa0388ef4197/requests-2.33.1-py3-none-any.whl", hash = "sha256:4e6d1ef462f3626a1f0a0a9c42dd93c63bad33f9f1c1937509b8c5c8718ab56a", size = 64947, upload-time = "2026-03-30T16:09:13.83Z" },]-[[package]]-name = "respx"-version = "0.23.1"-source = { registry = "https://pypi.org/simple" }-dependencies = [- { name = "httpx" },-]-sdist = { url = "https://files.pythonhosted.org/packages/43/98/4e55c9c486404ec12373708d015ebce157966965a5ebe7f28ff2c784d41b/respx-0.23.1.tar.gz", hash = "sha256:242dcc6ce6b5b9bf621f5870c82a63997e8e82bc7c947f9ffe272b8f3dd5a780", size = 29243, upload-time = "2026-04-08T14:37:16.008Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/1d/4a/221da6ca167db45693d8d26c7dc79ccfc978a440251bf6721c9aaf251ac0/respx-0.23.1-py2.py3-none-any.whl", hash = "sha256:b18004b029935384bccfa6d7d9d74b4ec9af73a081cc28600fffc0447f4b8c1a", size = 25557, upload-time = "2026-04-08T14:37:14.613Z" },-]-[[package]]name = "rich"version = "14.3.4"@@ -2923,15 +2614,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/e9/44/75a9c9421471a6c4805dbf2356f7c181a29c1879239abab1ea2cc8f38b40/sniffio-1.3.1-py3-none-any.whl", hash = "sha256:2f6da418d1f1e0fddd844478f41680e794e6051915791a034ff65e5f100525a2", size = 10235, upload-time = "2024-02-25T23:20:01.196Z" },]-[[package]]-name = "socksio"-version = "1.0.0"-source = { registry = "https://pypi.org/simple" }-sdist = { url = "https://files.pythonhosted.org/packages/f8/5c/48a7d9495be3d1c651198fd99dbb6ce190e2274d0f28b9051307bdec6b85/socksio-1.0.0.tar.gz", hash = "sha256:f88beb3da5b5c38b9890469de67d0cb0f9d494b78b106ca1845f96c10b91c4ac", size = 19055, upload-time = "2020-04-17T15:50:34.664Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/37/c3/6eeb6034408dac0fa653d126c9204ade96b819c936e136c5e8a6897eee9c/socksio-1.0.0-py3-none-any.whl", hash = "sha256:95dc1f15f9b34e8d7b16f06d74b8ccf48f609af32ab33c608d08761c5dcbb1f3", size = 12763, upload-time = "2020-04-17T15:50:31.878Z" },-]-[[package]]name = "sse-starlette"version = "3.4.1"@@ -3211,49 +2893,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/31/a3/5b1562db76a5a488274b2332a97199b32d0442aca0ed193697fd47786316/uvicorn-0.46.0-py3-none-any.whl", hash = "sha256:bbebbcbed972d162afca128605223022bedd345b7bc7855ce66deb31487a9048", size = 70926, upload-time = "2026-04-23T07:15:58.355Z" },]-[package.optional-dependencies]-standard = [- { name = "colorama", marker = "sys_platform == 'win32'" },- { name = "httptools" },- { name = "python-dotenv" },- { name = "pyyaml" },- { name = "uvloop", marker = "platform_python_implementation != 'PyPy' and sys_platform != 'cygwin' and sys_platform != 'win32'" },- { name = "watchfiles" },- { name = "websockets" },-]--[[package]]-name = "uvloop"-version = "0.22.1"-source = { registry = "https://pypi.org/simple" }-sdist = { url = "https://files.pythonhosted.org/packages/06/f0/18d39dbd1971d6d62c4629cc7fa67f74821b0dc1f5a77af43719de7936a7/uvloop-0.22.1.tar.gz", hash = "sha256:6c84bae345b9147082b17371e3dd5d42775bddce91f885499017f4607fdaf39f", size = 2443250, upload-time = "2025-10-16T22:17:19.342Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/3d/ff/7f72e8170be527b4977b033239a83a68d5c881cc4775fca255c677f7ac5d/uvloop-0.22.1-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:fe94b4564e865d968414598eea1a6de60adba0c040ba4ed05ac1300de402cd42", size = 1359936, upload-time = "2025-10-16T22:16:29.436Z" },- { url = "https://files.pythonhosted.org/packages/c3/c6/e5d433f88fd54d81ef4be58b2b7b0cea13c442454a1db703a1eea0db1a59/uvloop-0.22.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:51eb9bd88391483410daad430813d982010f9c9c89512321f5b60e2cddbdddd6", size = 752769, upload-time = "2025-10-16T22:16:30.493Z" },- { url = "https://files.pythonhosted.org/packages/24/68/a6ac446820273e71aa762fa21cdcc09861edd3536ff47c5cd3b7afb10eeb/uvloop-0.22.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:700e674a166ca5778255e0e1dc4e9d79ab2acc57b9171b79e65feba7184b3370", size = 4317413, upload-time = "2025-10-16T22:16:31.644Z" },- { url = "https://files.pythonhosted.org/packages/5f/6f/e62b4dfc7ad6518e7eff2516f680d02a0f6eb62c0c212e152ca708a0085e/uvloop-0.22.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7b5b1ac819a3f946d3b2ee07f09149578ae76066d70b44df3fa990add49a82e4", size = 4426307, upload-time = "2025-10-16T22:16:32.917Z" },- { url = "https://files.pythonhosted.org/packages/90/60/97362554ac21e20e81bcef1150cb2a7e4ffdaf8ea1e5b2e8bf7a053caa18/uvloop-0.22.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:e047cc068570bac9866237739607d1313b9253c3051ad84738cbb095be0537b2", size = 4131970, upload-time = "2025-10-16T22:16:34.015Z" },- { url = "https://files.pythonhosted.org/packages/99/39/6b3f7d234ba3964c428a6e40006340f53ba37993f46ed6e111c6e9141d18/uvloop-0.22.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:512fec6815e2dd45161054592441ef76c830eddaad55c8aa30952e6fe1ed07c0", size = 4296343, upload-time = "2025-10-16T22:16:35.149Z" },- { url = "https://files.pythonhosted.org/packages/89/8c/182a2a593195bfd39842ea68ebc084e20c850806117213f5a299dfc513d9/uvloop-0.22.1-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:561577354eb94200d75aca23fbde86ee11be36b00e52a4eaf8f50fb0c86b7705", size = 1358611, upload-time = "2025-10-16T22:16:36.833Z" },- { url = "https://files.pythonhosted.org/packages/d2/14/e301ee96a6dc95224b6f1162cd3312f6d1217be3907b79173b06785f2fe7/uvloop-0.22.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:1cdf5192ab3e674ca26da2eada35b288d2fa49fdd0f357a19f0e7c4e7d5077c8", size = 751811, upload-time = "2025-10-16T22:16:38.275Z" },- { url = "https://files.pythonhosted.org/packages/b7/02/654426ce265ac19e2980bfd9ea6590ca96a56f10c76e63801a2df01c0486/uvloop-0.22.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6e2ea3d6190a2968f4a14a23019d3b16870dd2190cd69c8180f7c632d21de68d", size = 4288562, upload-time = "2025-10-16T22:16:39.375Z" },- { url = "https://files.pythonhosted.org/packages/15/c0/0be24758891ef825f2065cd5db8741aaddabe3e248ee6acc5e8a80f04005/uvloop-0.22.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0530a5fbad9c9e4ee3f2b33b148c6a64d47bbad8000ea63704fa8260f4cf728e", size = 4366890, upload-time = "2025-10-16T22:16:40.547Z" },- { url = "https://files.pythonhosted.org/packages/d2/53/8369e5219a5855869bcee5f4d317f6da0e2c669aecf0ef7d371e3d084449/uvloop-0.22.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:bc5ef13bbc10b5335792360623cc378d52d7e62c2de64660616478c32cd0598e", size = 4119472, upload-time = "2025-10-16T22:16:41.694Z" },- { url = "https://files.pythonhosted.org/packages/f8/ba/d69adbe699b768f6b29a5eec7b47dd610bd17a69de51b251126a801369ea/uvloop-0.22.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:1f38ec5e3f18c8a10ded09742f7fb8de0108796eb673f30ce7762ce1b8550cad", size = 4239051, upload-time = "2025-10-16T22:16:43.224Z" },- { url = "https://files.pythonhosted.org/packages/90/cd/b62bdeaa429758aee8de8b00ac0dd26593a9de93d302bff3d21439e9791d/uvloop-0.22.1-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:3879b88423ec7e97cd4eba2a443aa26ed4e59b45e6b76aabf13fe2f27023a142", size = 1362067, upload-time = "2025-10-16T22:16:44.503Z" },- { url = "https://files.pythonhosted.org/packages/0d/f8/a132124dfda0777e489ca86732e85e69afcd1ff7686647000050ba670689/uvloop-0.22.1-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:4baa86acedf1d62115c1dc6ad1e17134476688f08c6efd8a2ab076e815665c74", size = 752423, upload-time = "2025-10-16T22:16:45.968Z" },- { url = "https://files.pythonhosted.org/packages/a3/94/94af78c156f88da4b3a733773ad5ba0b164393e357cc4bd0ab2e2677a7d6/uvloop-0.22.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:297c27d8003520596236bdb2335e6b3f649480bd09e00d1e3a99144b691d2a35", size = 4272437, upload-time = "2025-10-16T22:16:47.451Z" },- { url = "https://files.pythonhosted.org/packages/b5/35/60249e9fd07b32c665192cec7af29e06c7cd96fa1d08b84f012a56a0b38e/uvloop-0.22.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c1955d5a1dd43198244d47664a5858082a3239766a839b2102a269aaff7a4e25", size = 4292101, upload-time = "2025-10-16T22:16:49.318Z" },- { url = "https://files.pythonhosted.org/packages/02/62/67d382dfcb25d0a98ce73c11ed1a6fba5037a1a1d533dcbb7cab033a2636/uvloop-0.22.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:b31dc2fccbd42adc73bc4e7cdbae4fc5086cf378979e53ca5d0301838c5682c6", size = 4114158, upload-time = "2025-10-16T22:16:50.517Z" },- { url = "https://files.pythonhosted.org/packages/f0/7a/f1171b4a882a5d13c8b7576f348acfe6074d72eaf52cccef752f748d4a9f/uvloop-0.22.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:93f617675b2d03af4e72a5333ef89450dfaa5321303ede6e67ba9c9d26878079", size = 4177360, upload-time = "2025-10-16T22:16:52.646Z" },- { url = "https://files.pythonhosted.org/packages/79/7b/b01414f31546caf0919da80ad57cbfe24c56b151d12af68cee1b04922ca8/uvloop-0.22.1-cp314-cp314t-macosx_10_13_universal2.whl", hash = "sha256:37554f70528f60cad66945b885eb01f1bb514f132d92b6eeed1c90fd54ed6289", size = 1454790, upload-time = "2025-10-16T22:16:54.355Z" },- { url = "https://files.pythonhosted.org/packages/d4/31/0bb232318dd838cad3fa8fb0c68c8b40e1145b32025581975e18b11fab40/uvloop-0.22.1-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:b76324e2dc033a0b2f435f33eb88ff9913c156ef78e153fb210e03c13da746b3", size = 796783, upload-time = "2025-10-16T22:16:55.906Z" },- { url = "https://files.pythonhosted.org/packages/42/38/c9b09f3271a7a723a5de69f8e237ab8e7803183131bc57c890db0b6bb872/uvloop-0.22.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:badb4d8e58ee08dad957002027830d5c3b06aea446a6a3744483c2b3b745345c", size = 4647548, upload-time = "2025-10-16T22:16:57.008Z" },- { url = "https://files.pythonhosted.org/packages/c1/37/945b4ca0ac27e3dc4952642d4c900edd030b3da6c9634875af6e13ae80e5/uvloop-0.22.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b91328c72635f6f9e0282e4a57da7470c7350ab1c9f48546c0f2866205349d21", size = 4467065, upload-time = "2025-10-16T22:16:58.206Z" },- { url = "https://files.pythonhosted.org/packages/97/cc/48d232f33d60e2e2e0b42f4e73455b146b76ebe216487e862700457fbf3c/uvloop-0.22.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:daf620c2995d193449393d6c62131b3fbd40a63bf7b307a1527856ace637fe88", size = 4328384, upload-time = "2025-10-16T22:16:59.36Z" },- { url = "https://files.pythonhosted.org/packages/e4/16/c1fd27e9549f3c4baf1dc9c20c456cd2f822dbf8de9f463824b0c0357e06/uvloop-0.22.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:6cde23eeda1a25c75b2e07d39970f3374105d5eafbaab2a4482be82f272d5a5e", size = 4296730, upload-time = "2025-10-16T22:17:00.744Z" },-]-[[package]]name = "watchdog"version = "4.0.2"@@ -3278,76 +2917,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/b0/0b/43b96a9ecdd65ff5545b1b13b687ca486da5c6249475b1a45f24d63a1858/watchdog-4.0.2-py3-none-win_ia64.whl", hash = "sha256:baececaa8edff42cd16558a639a9b0ddf425f93d892e8392a56bf904f5eff22c", size = 82933, upload-time = "2024-08-11T07:37:59.573Z" },]-[[package]]-name = "watchfiles"-version = "1.1.1"-source = { registry = "https://pypi.org/simple" }-dependencies = [- { name = "anyio" },-]-sdist = { url = "https://files.pythonhosted.org/packages/c2/c9/8869df9b2a2d6c59d79220a4db37679e74f807c559ffe5265e08b227a210/watchfiles-1.1.1.tar.gz", hash = "sha256:a173cb5c16c4f40ab19cecf48a534c409f7ea983ab8fed0741304a1c0a31b3f2", size = 94440, upload-time = "2025-10-14T15:06:21.08Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/74/d5/f039e7e3c639d9b1d09b07ea412a6806d38123f0508e5f9b48a87b0a76cc/watchfiles-1.1.1-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:8c89f9f2f740a6b7dcc753140dd5e1ab9215966f7a3530d0c0705c83b401bd7d", size = 404745, upload-time = "2025-10-14T15:04:46.731Z" },- { url = "https://files.pythonhosted.org/packages/a5/96/a881a13aa1349827490dab2d363c8039527060cfcc2c92cc6d13d1b1049e/watchfiles-1.1.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:bd404be08018c37350f0d6e34676bd1e2889990117a2b90070b3007f172d0610", size = 391769, upload-time = "2025-10-14T15:04:48.003Z" },- { url = "https://files.pythonhosted.org/packages/4b/5b/d3b460364aeb8da471c1989238ea0e56bec24b6042a68046adf3d9ddb01c/watchfiles-1.1.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8526e8f916bb5b9a0a777c8317c23ce65de259422bba5b31325a6fa6029d33af", size = 449374, upload-time = "2025-10-14T15:04:49.179Z" },- { url = "https://files.pythonhosted.org/packages/b9/44/5769cb62d4ed055cb17417c0a109a92f007114a4e07f30812a73a4efdb11/watchfiles-1.1.1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:2edc3553362b1c38d9f06242416a5d8e9fe235c204a4072e988ce2e5bb1f69f6", size = 459485, upload-time = "2025-10-14T15:04:50.155Z" },- { url = "https://files.pythonhosted.org/packages/19/0c/286b6301ded2eccd4ffd0041a1b726afda999926cf720aab63adb68a1e36/watchfiles-1.1.1-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:30f7da3fb3f2844259cba4720c3fc7138eb0f7b659c38f3bfa65084c7fc7abce", size = 488813, upload-time = "2025-10-14T15:04:51.059Z" },- { url = "https://files.pythonhosted.org/packages/c7/2b/8530ed41112dd4a22f4dcfdb5ccf6a1baad1ff6eed8dc5a5f09e7e8c41c7/watchfiles-1.1.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:f8979280bdafff686ba5e4d8f97840f929a87ed9cdf133cbbd42f7766774d2aa", size = 594816, upload-time = "2025-10-14T15:04:52.031Z" },- { url = "https://files.pythonhosted.org/packages/ce/d2/f5f9fb49489f184f18470d4f99f4e862a4b3e9ac2865688eb2099e3d837a/watchfiles-1.1.1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:dcc5c24523771db3a294c77d94771abcfcb82a0e0ee8efd910c37c59ec1b31bb", size = 475186, upload-time = "2025-10-14T15:04:53.064Z" },- { url = "https://files.pythonhosted.org/packages/cf/68/5707da262a119fb06fbe214d82dd1fe4a6f4af32d2d14de368d0349eb52a/watchfiles-1.1.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1db5d7ae38ff20153d542460752ff397fcf5c96090c1230803713cf3147a6803", size = 456812, upload-time = "2025-10-14T15:04:55.174Z" },- { url = "https://files.pythonhosted.org/packages/66/ab/3cbb8756323e8f9b6f9acb9ef4ec26d42b2109bce830cc1f3468df20511d/watchfiles-1.1.1-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:28475ddbde92df1874b6c5c8aaeb24ad5be47a11f87cde5a28ef3835932e3e94", size = 630196, upload-time = "2025-10-14T15:04:56.22Z" },- { url = "https://files.pythonhosted.org/packages/78/46/7152ec29b8335f80167928944a94955015a345440f524d2dfe63fc2f437b/watchfiles-1.1.1-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:36193ed342f5b9842edd3532729a2ad55c4160ffcfa3700e0d54be496b70dd43", size = 622657, upload-time = "2025-10-14T15:04:57.521Z" },- { url = "https://files.pythonhosted.org/packages/0a/bf/95895e78dd75efe9a7f31733607f384b42eb5feb54bd2eb6ed57cc2e94f4/watchfiles-1.1.1-cp312-cp312-win32.whl", hash = "sha256:859e43a1951717cc8de7f4c77674a6d389b106361585951d9e69572823f311d9", size = 272042, upload-time = "2025-10-14T15:04:59.046Z" },- { url = "https://files.pythonhosted.org/packages/87/0a/90eb755f568de2688cb220171c4191df932232c20946966c27a59c400850/watchfiles-1.1.1-cp312-cp312-win_amd64.whl", hash = "sha256:91d4c9a823a8c987cce8fa2690923b069966dabb196dd8d137ea2cede885fde9", size = 288410, upload-time = "2025-10-14T15:05:00.081Z" },- { url = "https://files.pythonhosted.org/packages/36/76/f322701530586922fbd6723c4f91ace21364924822a8772c549483abed13/watchfiles-1.1.1-cp312-cp312-win_arm64.whl", hash = "sha256:a625815d4a2bdca61953dbba5a39d60164451ef34c88d751f6c368c3ea73d404", size = 278209, upload-time = "2025-10-14T15:05:01.168Z" },- { url = "https://files.pythonhosted.org/packages/bb/f4/f750b29225fe77139f7ae5de89d4949f5a99f934c65a1f1c0b248f26f747/watchfiles-1.1.1-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:130e4876309e8686a5e37dba7d5e9bc77e6ed908266996ca26572437a5271e18", size = 404321, upload-time = "2025-10-14T15:05:02.063Z" },- { url = "https://files.pythonhosted.org/packages/2b/f9/f07a295cde762644aa4c4bb0f88921d2d141af45e735b965fb2e87858328/watchfiles-1.1.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:5f3bde70f157f84ece3765b42b4a52c6ac1a50334903c6eaf765362f6ccca88a", size = 391783, upload-time = "2025-10-14T15:05:03.052Z" },- { url = "https://files.pythonhosted.org/packages/bc/11/fc2502457e0bea39a5c958d86d2cb69e407a4d00b85735ca724bfa6e0d1a/watchfiles-1.1.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:14e0b1fe858430fc0251737ef3824c54027bedb8c37c38114488b8e131cf8219", size = 449279, upload-time = "2025-10-14T15:05:04.004Z" },- { url = "https://files.pythonhosted.org/packages/e3/1f/d66bc15ea0b728df3ed96a539c777acfcad0eb78555ad9efcaa1274688f0/watchfiles-1.1.1-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:f27db948078f3823a6bb3b465180db8ebecf26dd5dae6f6180bd87383b6b4428", size = 459405, upload-time = "2025-10-14T15:05:04.942Z" },- { url = "https://files.pythonhosted.org/packages/be/90/9f4a65c0aec3ccf032703e6db02d89a157462fbb2cf20dd415128251cac0/watchfiles-1.1.1-cp313-cp313-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:059098c3a429f62fc98e8ec62b982230ef2c8df68c79e826e37b895bc359a9c0", size = 488976, upload-time = "2025-10-14T15:05:05.905Z" },- { url = "https://files.pythonhosted.org/packages/37/57/ee347af605d867f712be7029bb94c8c071732a4b44792e3176fa3c612d39/watchfiles-1.1.1-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:bfb5862016acc9b869bb57284e6cb35fdf8e22fe59f7548858e2f971d045f150", size = 595506, upload-time = "2025-10-14T15:05:06.906Z" },- { url = "https://files.pythonhosted.org/packages/a8/78/cc5ab0b86c122047f75e8fc471c67a04dee395daf847d3e59381996c8707/watchfiles-1.1.1-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:319b27255aacd9923b8a276bb14d21a5f7ff82564c744235fc5eae58d95422ae", size = 474936, upload-time = "2025-10-14T15:05:07.906Z" },- { url = "https://files.pythonhosted.org/packages/62/da/def65b170a3815af7bd40a3e7010bf6ab53089ef1b75d05dd5385b87cf08/watchfiles-1.1.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c755367e51db90e75b19454b680903631d41f9e3607fbd941d296a020c2d752d", size = 456147, upload-time = "2025-10-14T15:05:09.138Z" },- { url = "https://files.pythonhosted.org/packages/57/99/da6573ba71166e82d288d4df0839128004c67d2778d3b566c138695f5c0b/watchfiles-1.1.1-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:c22c776292a23bfc7237a98f791b9ad3144b02116ff10d820829ce62dff46d0b", size = 630007, upload-time = "2025-10-14T15:05:10.117Z" },- { url = "https://files.pythonhosted.org/packages/a8/51/7439c4dd39511368849eb1e53279cd3454b4a4dbace80bab88feeb83c6b5/watchfiles-1.1.1-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:3a476189be23c3686bc2f4321dd501cb329c0a0469e77b7b534ee10129ae6374", size = 622280, upload-time = "2025-10-14T15:05:11.146Z" },- { url = "https://files.pythonhosted.org/packages/95/9c/8ed97d4bba5db6fdcdb2b298d3898f2dd5c20f6b73aee04eabe56c59677e/watchfiles-1.1.1-cp313-cp313-win32.whl", hash = "sha256:bf0a91bfb5574a2f7fc223cf95eeea79abfefa404bf1ea5e339c0c1560ae99a0", size = 272056, upload-time = "2025-10-14T15:05:12.156Z" },- { url = "https://files.pythonhosted.org/packages/1f/f3/c14e28429f744a260d8ceae18bf58c1d5fa56b50d006a7a9f80e1882cb0d/watchfiles-1.1.1-cp313-cp313-win_amd64.whl", hash = "sha256:52e06553899e11e8074503c8e716d574adeeb7e68913115c4b3653c53f9bae42", size = 288162, upload-time = "2025-10-14T15:05:13.208Z" },- { url = "https://files.pythonhosted.org/packages/dc/61/fe0e56c40d5cd29523e398d31153218718c5786b5e636d9ae8ae79453d27/watchfiles-1.1.1-cp313-cp313-win_arm64.whl", hash = "sha256:ac3cc5759570cd02662b15fbcd9d917f7ecd47efe0d6b40474eafd246f91ea18", size = 277909, upload-time = "2025-10-14T15:05:14.49Z" },- { url = "https://files.pythonhosted.org/packages/79/42/e0a7d749626f1e28c7108a99fb9bf524b501bbbeb9b261ceecde644d5a07/watchfiles-1.1.1-cp313-cp313t-macosx_10_12_x86_64.whl", hash = "sha256:563b116874a9a7ce6f96f87cd0b94f7faf92d08d0021e837796f0a14318ef8da", size = 403389, upload-time = "2025-10-14T15:05:15.777Z" },- { url = "https://files.pythonhosted.org/packages/15/49/08732f90ce0fbbc13913f9f215c689cfc9ced345fb1bcd8829a50007cc8d/watchfiles-1.1.1-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:3ad9fe1dae4ab4212d8c91e80b832425e24f421703b5a42ef2e4a1e215aff051", size = 389964, upload-time = "2025-10-14T15:05:16.85Z" },- { url = "https://files.pythonhosted.org/packages/27/0d/7c315d4bd5f2538910491a0393c56bf70d333d51bc5b34bee8e68e8cea19/watchfiles-1.1.1-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ce70f96a46b894b36eba678f153f052967a0d06d5b5a19b336ab0dbbd029f73e", size = 448114, upload-time = "2025-10-14T15:05:17.876Z" },- { url = "https://files.pythonhosted.org/packages/c3/24/9e096de47a4d11bc4df41e9d1e61776393eac4cb6eb11b3e23315b78b2cc/watchfiles-1.1.1-cp313-cp313t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:cb467c999c2eff23a6417e58d75e5828716f42ed8289fe6b77a7e5a91036ca70", size = 460264, upload-time = "2025-10-14T15:05:18.962Z" },- { url = "https://files.pythonhosted.org/packages/cc/0f/e8dea6375f1d3ba5fcb0b3583e2b493e77379834c74fd5a22d66d85d6540/watchfiles-1.1.1-cp313-cp313t-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:836398932192dae4146c8f6f737d74baeac8b70ce14831a239bdb1ca882fc261", size = 487877, upload-time = "2025-10-14T15:05:20.094Z" },- { url = "https://files.pythonhosted.org/packages/ac/5b/df24cfc6424a12deb41503b64d42fbea6b8cb357ec62ca84a5a3476f654a/watchfiles-1.1.1-cp313-cp313t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:743185e7372b7bc7c389e1badcc606931a827112fbbd37f14c537320fca08620", size = 595176, upload-time = "2025-10-14T15:05:21.134Z" },- { url = "https://files.pythonhosted.org/packages/8f/b5/853b6757f7347de4e9b37e8cc3289283fb983cba1ab4d2d7144694871d9c/watchfiles-1.1.1-cp313-cp313t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:afaeff7696e0ad9f02cbb8f56365ff4686ab205fcf9c4c5b6fdfaaa16549dd04", size = 473577, upload-time = "2025-10-14T15:05:22.306Z" },- { url = "https://files.pythonhosted.org/packages/e1/f7/0a4467be0a56e80447c8529c9fce5b38eab4f513cb3d9bf82e7392a5696b/watchfiles-1.1.1-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:3f7eb7da0eb23aa2ba036d4f616d46906013a68caf61b7fdbe42fc8b25132e77", size = 455425, upload-time = "2025-10-14T15:05:23.348Z" },- { url = "https://files.pythonhosted.org/packages/8e/e0/82583485ea00137ddf69bc84a2db88bd92ab4a6e3c405e5fb878ead8d0e7/watchfiles-1.1.1-cp313-cp313t-musllinux_1_1_aarch64.whl", hash = "sha256:831a62658609f0e5c64178211c942ace999517f5770fe9436be4c2faeba0c0ef", size = 628826, upload-time = "2025-10-14T15:05:24.398Z" },- { url = "https://files.pythonhosted.org/packages/28/9a/a785356fccf9fae84c0cc90570f11702ae9571036fb25932f1242c82191c/watchfiles-1.1.1-cp313-cp313t-musllinux_1_1_x86_64.whl", hash = "sha256:f9a2ae5c91cecc9edd47e041a930490c31c3afb1f5e6d71de3dc671bfaca02bf", size = 622208, upload-time = "2025-10-14T15:05:25.45Z" },- { url = "https://files.pythonhosted.org/packages/c3/f4/0872229324ef69b2c3edec35e84bd57a1289e7d3fe74588048ed8947a323/watchfiles-1.1.1-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:d1715143123baeeaeadec0528bb7441103979a1d5f6fd0e1f915383fea7ea6d5", size = 404315, upload-time = "2025-10-14T15:05:26.501Z" },- { url = "https://files.pythonhosted.org/packages/7b/22/16d5331eaed1cb107b873f6ae1b69e9ced582fcf0c59a50cd84f403b1c32/watchfiles-1.1.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:39574d6370c4579d7f5d0ad940ce5b20db0e4117444e39b6d8f99db5676c52fd", size = 390869, upload-time = "2025-10-14T15:05:27.649Z" },- { url = "https://files.pythonhosted.org/packages/b2/7e/5643bfff5acb6539b18483128fdc0ef2cccc94a5b8fbda130c823e8ed636/watchfiles-1.1.1-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:7365b92c2e69ee952902e8f70f3ba6360d0d596d9299d55d7d386df84b6941fb", size = 449919, upload-time = "2025-10-14T15:05:28.701Z" },- { url = "https://files.pythonhosted.org/packages/51/2e/c410993ba5025a9f9357c376f48976ef0e1b1aefb73b97a5ae01a5972755/watchfiles-1.1.1-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:bfff9740c69c0e4ed32416f013f3c45e2ae42ccedd1167ef2d805c000b6c71a5", size = 460845, upload-time = "2025-10-14T15:05:30.064Z" },- { url = "https://files.pythonhosted.org/packages/8e/a4/2df3b404469122e8680f0fcd06079317e48db58a2da2950fb45020947734/watchfiles-1.1.1-cp314-cp314-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:b27cf2eb1dda37b2089e3907d8ea92922b673c0c427886d4edc6b94d8dfe5db3", size = 489027, upload-time = "2025-10-14T15:05:31.064Z" },- { url = "https://files.pythonhosted.org/packages/ea/84/4587ba5b1f267167ee715b7f66e6382cca6938e0a4b870adad93e44747e6/watchfiles-1.1.1-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:526e86aced14a65a5b0ec50827c745597c782ff46b571dbfe46192ab9e0b3c33", size = 595615, upload-time = "2025-10-14T15:05:32.074Z" },- { url = "https://files.pythonhosted.org/packages/6a/0f/c6988c91d06e93cd0bb3d4a808bcf32375ca1904609835c3031799e3ecae/watchfiles-1.1.1-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:04e78dd0b6352db95507fd8cb46f39d185cf8c74e4cf1e4fbad1d3df96faf510", size = 474836, upload-time = "2025-10-14T15:05:33.209Z" },- { url = "https://files.pythonhosted.org/packages/b4/36/ded8aebea91919485b7bbabbd14f5f359326cb5ec218cd67074d1e426d74/watchfiles-1.1.1-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5c85794a4cfa094714fb9c08d4a218375b2b95b8ed1666e8677c349906246c05", size = 455099, upload-time = "2025-10-14T15:05:34.189Z" },- { url = "https://files.pythonhosted.org/packages/98/e0/8c9bdba88af756a2fce230dd365fab2baf927ba42cd47521ee7498fd5211/watchfiles-1.1.1-cp314-cp314-musllinux_1_1_aarch64.whl", hash = "sha256:74d5012b7630714b66be7b7b7a78855ef7ad58e8650c73afc4c076a1f480a8d6", size = 630626, upload-time = "2025-10-14T15:05:35.216Z" },- { url = "https://files.pythonhosted.org/packages/2a/84/a95db05354bf2d19e438520d92a8ca475e578c647f78f53197f5a2f17aaf/watchfiles-1.1.1-cp314-cp314-musllinux_1_1_x86_64.whl", hash = "sha256:8fbe85cb3201c7d380d3d0b90e63d520f15d6afe217165d7f98c9c649654db81", size = 622519, upload-time = "2025-10-14T15:05:36.259Z" },- { url = "https://files.pythonhosted.org/packages/1d/ce/d8acdc8de545de995c339be67711e474c77d643555a9bb74a9334252bd55/watchfiles-1.1.1-cp314-cp314-win32.whl", hash = "sha256:3fa0b59c92278b5a7800d3ee7733da9d096d4aabcfabb9a928918bd276ef9b9b", size = 272078, upload-time = "2025-10-14T15:05:37.63Z" },- { url = "https://files.pythonhosted.org/packages/c4/c9/a74487f72d0451524be827e8edec251da0cc1fcf111646a511ae752e1a3d/watchfiles-1.1.1-cp314-cp314-win_amd64.whl", hash = "sha256:c2047d0b6cea13b3316bdbafbfa0c4228ae593d995030fda39089d36e64fc03a", size = 287664, upload-time = "2025-10-14T15:05:38.95Z" },- { url = "https://files.pythonhosted.org/packages/df/b8/8ac000702cdd496cdce998c6f4ee0ca1f15977bba51bdf07d872ebdfc34c/watchfiles-1.1.1-cp314-cp314-win_arm64.whl", hash = "sha256:842178b126593addc05acf6fce960d28bc5fae7afbaa2c6c1b3a7b9460e5be02", size = 277154, upload-time = "2025-10-14T15:05:39.954Z" },- { url = "https://files.pythonhosted.org/packages/47/a8/e3af2184707c29f0f14b1963c0aace6529f9d1b8582d5b99f31bbf42f59e/watchfiles-1.1.1-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:88863fbbc1a7312972f1c511f202eb30866370ebb8493aef2812b9ff28156a21", size = 403820, upload-time = "2025-10-14T15:05:40.932Z" },- { url = "https://files.pythonhosted.org/packages/c0/ec/e47e307c2f4bd75f9f9e8afbe3876679b18e1bcec449beca132a1c5ffb2d/watchfiles-1.1.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:55c7475190662e202c08c6c0f4d9e345a29367438cf8e8037f3155e10a88d5a5", size = 390510, upload-time = "2025-10-14T15:05:41.945Z" },- { url = "https://files.pythonhosted.org/packages/d5/a0/ad235642118090f66e7b2f18fd5c42082418404a79205cdfca50b6309c13/watchfiles-1.1.1-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3f53fa183d53a1d7a8852277c92b967ae99c2d4dcee2bfacff8868e6e30b15f7", size = 448408, upload-time = "2025-10-14T15:05:43.385Z" },- { url = "https://files.pythonhosted.org/packages/df/85/97fa10fd5ff3332ae17e7e40e20784e419e28521549780869f1413742e9d/watchfiles-1.1.1-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:6aae418a8b323732fa89721d86f39ec8f092fc2af67f4217a2b07fd3e93c6101", size = 458968, upload-time = "2025-10-14T15:05:44.404Z" },- { url = "https://files.pythonhosted.org/packages/47/c2/9059c2e8966ea5ce678166617a7f75ecba6164375f3b288e50a40dc6d489/watchfiles-1.1.1-cp314-cp314t-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f096076119da54a6080e8920cbdaac3dbee667eb91dcc5e5b78840b87415bd44", size = 488096, upload-time = "2025-10-14T15:05:45.398Z" },- { url = "https://files.pythonhosted.org/packages/94/44/d90a9ec8ac309bc26db808a13e7bfc0e4e78b6fc051078a554e132e80160/watchfiles-1.1.1-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:00485f441d183717038ed2e887a7c868154f216877653121068107b227a2f64c", size = 596040, upload-time = "2025-10-14T15:05:46.502Z" },- { url = "https://files.pythonhosted.org/packages/95/68/4e3479b20ca305cfc561db3ed207a8a1c745ee32bf24f2026a129d0ddb6e/watchfiles-1.1.1-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:a55f3e9e493158d7bfdb60a1165035f1cf7d320914e7b7ea83fe22c6023b58fc", size = 473847, upload-time = "2025-10-14T15:05:47.484Z" },- { url = "https://files.pythonhosted.org/packages/4f/55/2af26693fd15165c4ff7857e38330e1b61ab8c37d15dc79118cdba115b7a/watchfiles-1.1.1-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8c91ed27800188c2ae96d16e3149f199d62f86c7af5f5f4d2c61a3ed8cd3666c", size = 455072, upload-time = "2025-10-14T15:05:48.928Z" },- { url = "https://files.pythonhosted.org/packages/66/1d/d0d200b10c9311ec25d2273f8aad8c3ef7cc7ea11808022501811208a750/watchfiles-1.1.1-cp314-cp314t-musllinux_1_1_aarch64.whl", hash = "sha256:311ff15a0bae3714ffb603e6ba6dbfba4065ab60865d15a6ec544133bdb21099", size = 629104, upload-time = "2025-10-14T15:05:49.908Z" },- { url = "https://files.pythonhosted.org/packages/e3/bd/fa9bb053192491b3867ba07d2343d9f2252e00811567d30ae8d0f78136fe/watchfiles-1.1.1-cp314-cp314t-musllinux_1_1_x86_64.whl", hash = "sha256:a916a2932da8f8ab582f242c065f5c81bed3462849ca79ee357dd9551b0e9b01", size = 622112, upload-time = "2025-10-14T15:05:50.941Z" },-]-[[package]]name = "wcwidth"version = "0.7.0"@@ -3388,70 +2957,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/fa/a8/5b41e0da817d64113292ab1f8247140aac61cbf6cfd085d6a0fa77f4984f/websockets-15.0.1-py3-none-any.whl", hash = "sha256:f7a866fbc1e97b5c617ee4116daaa09b722101d4a3c170c787450ba409f9736f", size = 169743, upload-time = "2025-03-05T20:03:39.41Z" },]-[[package]]-name = "wrapt"-version = "2.2.1"-source = { registry = "https://pypi.org/simple" }-sdist = { url = "https://files.pythonhosted.org/packages/2d/9f/06263fcd8ad6c405f05a3905fd7a84dd3176eb5ad46e44bccc0cd16348bb/wrapt-2.2.1.tar.gz", hash = "sha256:6744f504375775d7609c82c8d3d94af1c9a6f05586984536905908ba905277b9", size = 127620, upload-time = "2026-05-22T14:49:43.056Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/89/0c/bfae7b9401583b6d05938cd16dedc43857d96da2f8a3d50d78cc515bf6ff/wrapt-2.2.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:3ffad790d9d11d8ecf9f17c4bb671a5b4089e4d8b575c46c5129597f41f836b0", size = 81021, upload-time = "2026-05-22T14:48:00.313Z" },- { url = "https://files.pythonhosted.org/packages/26/58/80f6a6599f933f4caecc1cb3ee88a04faf81e8b9bddbd6109c688dd63e0f/wrapt-2.2.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:628f5220c7a904d5fc78f7075c8d7871433eb6d035c94728a22fdf85f193d2a8", size = 81692, upload-time = "2026-05-22T14:48:01.49Z" },- { url = "https://files.pythonhosted.org/packages/17/93/fb357cc7847c58a8ae790be718903afa81a28d23e642c843dc4129e8a0b2/wrapt-2.2.1-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:61acce4257a9883669703c525447c5b4c392edf0f987ae77ec32668440158f0e", size = 169364, upload-time = "2026-05-22T14:48:02.791Z" },- { url = "https://files.pythonhosted.org/packages/aa/0b/76b601ee309a8bd556af0eecb184394c20b3c49aa9c8e085aa1ffacc2568/wrapt-2.2.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:727ab4244622cd6ad2390f322642090c877d2e83a608d2653a7643ae5368d926", size = 171079, upload-time = "2026-05-22T14:48:04.22Z" },- { url = "https://files.pythonhosted.org/packages/cd/87/ee3f32d5658e3e26d3e0e457922b47a36dd3bfbdfee7f97bb3e802344a66/wrapt-2.2.1-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:03df9ebed4c73ab93fa8c07e3d41d818dfca1852b15731a3de59457b27814624", size = 160205, upload-time = "2026-05-22T14:48:05.553Z" },- { url = "https://files.pythonhosted.org/packages/b1/d0/ae2fd64277a67f5d7bffcf2d05eea1e476263fb2a072baf0b0129ab85984/wrapt-2.2.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:0d9ff006f420b2ec8296aa56ade43ea7da3e997e85769f0aafc5e0661aacb710", size = 168922, upload-time = "2026-05-22T14:48:07.132Z" },- { url = "https://files.pythonhosted.org/packages/b1/f3/2d541a060c5bbafb9400bca4917e4d78bfd1f239f404782c86831a8f6b29/wrapt-2.2.1-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:844c858fc3bb7eacc0ba8efa904935d16aac6a4470948ad1e7e55c9f5a2a665f", size = 158388, upload-time = "2026-05-22T14:48:08.629Z" },- { url = "https://files.pythonhosted.org/packages/1d/68/8d92c8800c57e93cb116ae9e9d6cbafc34fade5ee9f9107b6f203fb4dc35/wrapt-2.2.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:87bacdaf225117a342a20d9c03438d701c02112f6e3f351ce9b7f32354f14797", size = 167682, upload-time = "2026-05-22T14:48:10.042Z" },- { url = "https://files.pythonhosted.org/packages/30/72/83ea3790ea352439442349388e29ff07b76e0686265f9088bbb505d1608d/wrapt-2.2.1-cp312-cp312-win32.whl", hash = "sha256:2f8c90c8afde51969487be4e1343ae049b268854877d415c2510baf833775052", size = 77857, upload-time = "2026-05-22T14:48:11.782Z" },- { url = "https://files.pythonhosted.org/packages/ef/cb/99450668dd3502d62a54a1c8aa56e44f34cb8c1261b381cfe2e7926c3b75/wrapt-2.2.1-cp312-cp312-win_amd64.whl", hash = "sha256:6ce32763ac31ce94fe9aada947e479b1975012bff166da409b4b9e4e376cf7e5", size = 80825, upload-time = "2026-05-22T14:48:13.046Z" },- { url = "https://files.pythonhosted.org/packages/e6/3a/87512881be64e743f9ee4c66f4cbe8e884974bef2a5989af71f999653ac7/wrapt-2.2.1-cp312-cp312-win_arm64.whl", hash = "sha256:8d1b4d0e0c2119587a31f5c029abd547e0c81d93b89d394566fe1588659eb579", size = 79087, upload-time = "2026-05-22T14:48:14.323Z" },- { url = "https://files.pythonhosted.org/packages/88/d1/a1b08f8f4fac8cbb156fa51cf64ee2c7f7f74f9875ba3cf70b3c58368694/wrapt-2.2.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:d2beb1c7cab10603aecdc42f8edd6ff013f9a32e4543474e38e6b77ce9975aeb", size = 80831, upload-time = "2026-05-22T14:48:15.598Z" },- { url = "https://files.pythonhosted.org/packages/54/ce/57890814991446a845e09b3445ce8b694f27eb0577004f2c2a36a9772ed4/wrapt-2.2.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:e0cb7e4dd71f4c32e5e84843cd3c4cd65dda034314004bbe1d7f99af2426ab80", size = 81375, upload-time = "2026-05-22T14:48:17.071Z" },- { url = "https://files.pythonhosted.org/packages/38/65/08d7a6c76ac4493bdb668205ee9c1de1bd5daca61717c3e9aa49b4c01499/wrapt-2.2.1-cp313-cp313-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:95821352042722cd9f1108874579a47989d0a7e12a37d87d2fc4af20fd99ab8a", size = 167417, upload-time = "2026-05-22T14:48:18.303Z" },- { url = "https://files.pythonhosted.org/packages/62/ce/f1ccbee7a1bfe5cdc6b3da6bab4b45713d628b9294da32a39f563d648140/wrapt-2.2.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:abd621552ede77c4c69be7fac44ba911225b0c812b6ba604e5964cf98085b474", size = 166948, upload-time = "2026-05-22T14:48:19.768Z" },- { url = "https://files.pythonhosted.org/packages/86/2a/f85d48d1cd4869aee6704028d257d740a47c1c467b457ce396b4b5b55d07/wrapt-2.2.1-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e3677c7146ce694874941ba82b57092cc4875445aadf29d72807351023105143", size = 158148, upload-time = "2026-05-22T14:48:21.96Z" },- { url = "https://files.pythonhosted.org/packages/fe/5c/93939ad11d4a12358ab1aab219a2ef5efa5612e0db6b9fc65af8af1a891b/wrapt-2.2.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:9a5934eaea872e17936b5f45501eba5ab0bce9a74122e172b663d7c28c459c4a", size = 165905, upload-time = "2026-05-22T14:48:23.373Z" },- { url = "https://files.pythonhosted.org/packages/e0/22/b8c2aa89862ff58605934d7abf4b70e6a5a1c33df96656f49035ccdf1c8a/wrapt-2.2.1-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:f5b9daf6b629fce418e0cc3dd0436eac045188fa35deadb7a7f3941d5b8203f9", size = 156712, upload-time = "2026-05-22T14:48:24.767Z" },- { url = "https://files.pythonhosted.org/packages/5d/78/bf00a7b02239c12bb02ddcc3c0b971bfcc36e578c5a44f1ccfef5b458545/wrapt-2.2.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:f53ac9f3ef573326d009ed809beff4efcac6451931c2b8132586da4b9e53ff31", size = 166560, upload-time = "2026-05-22T14:48:26.83Z" },- { url = "https://files.pythonhosted.org/packages/fe/93/6390ca9c5b787683cef588d04f57c8d41b9a2323b5597a65f18638c90ef2/wrapt-2.2.1-cp313-cp313-win32.whl", hash = "sha256:1ffa9cfd4bdb581539951b14ae661ff20ed0c3599b3e911a131ee0ec5ac11337", size = 77817, upload-time = "2026-05-22T14:48:28.221Z" },- { url = "https://files.pythonhosted.org/packages/97/73/ce10f0e71c0cfaa1a65faadb8efd4852028b3bb9ba28932b8889df769d38/wrapt-2.2.1-cp313-cp313-win_amd64.whl", hash = "sha256:368eac1e20fd0bb03dd3cc42bf9887154c3861b60989389ccb5fac032617d215", size = 80736, upload-time = "2026-05-22T14:48:30.139Z" },- { url = "https://files.pythonhosted.org/packages/c7/4c/89f4a6818fafbbd840330e4fa3873073e1bfc166133a64cac7f8fde7a5e3/wrapt-2.2.1-cp313-cp313-win_arm64.whl", hash = "sha256:c754dafdf5aaf0b401b644a90a30046929a0dd1a536e0ff0ec959a59155d9c7f", size = 79099, upload-time = "2026-05-22T14:48:31.405Z" },- { url = "https://files.pythonhosted.org/packages/bf/f2/9a8741c46f8c208ac0a45b25ba170bcb4fb72a2781d5fb97dbd7b6be73cb/wrapt-2.2.1-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:ed928d0fda15fc0adc8d13305c8b3c0f2fba5b0669950c9e6d019d9162a3b3e8", size = 82802, upload-time = "2026-05-22T14:48:33.307Z" },- { url = "https://files.pythonhosted.org/packages/9c/0d/e9c855716a3705eef1416456bdf062b60620726fdc59428ff670fc3c60dc/wrapt-2.2.1-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:fafb4e739e43544d12cb4abd1605fd4683b6ca6a9ad682b7fd8f4d21973eafa8", size = 83329, upload-time = "2026-05-22T14:48:34.593Z" },- { url = "https://files.pythonhosted.org/packages/3b/d6/a88f1c13112b7831adac75cea65d8310e0d696d570c8961844c90a57b865/wrapt-2.2.1-cp313-cp313t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:74d6a0c31472fe5d814917266b9f46495d7c61ed890af08b468acea92fb89a8d", size = 202937, upload-time = "2026-05-22T14:48:35.859Z" },- { url = "https://files.pythonhosted.org/packages/42/65/e29d54aef06a4d898a5b8a25589a0b3769bde454f922fad8f6f89fbfb650/wrapt-2.2.1-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ab5be648d5a0b86b7438864f8df3c705a65cef35a2fd3e5561e3e203167e0f27", size = 209997, upload-time = "2026-05-22T14:48:38.153Z" },- { url = "https://files.pythonhosted.org/packages/2a/91/e4454263516cf0e12640912fbca9a83654e424f0a6ddb79f5cd7ce14bf33/wrapt-2.2.1-cp313-cp313t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:9d8f204c8e3a8bf9ece17e0a83d137fd807440977f8a5e762d59306795011440", size = 194856, upload-time = "2026-05-22T14:48:39.69Z" },- { url = "https://files.pythonhosted.org/packages/de/d0/fe0ee202286afdf4a7f77dd29f195703145764d572aec209c5086e57d924/wrapt-2.2.1-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:d047f6498c973874ba08ac3f97c69a2c4b2211c8de6f4c205f75cb1c9522596e", size = 205654, upload-time = "2026-05-22T14:48:43.456Z" },- { url = "https://files.pythonhosted.org/packages/23/b6/87d860dfc6460c246af70b1fd5c8b76df77571b42a493459423ded94fd7d/wrapt-2.2.1-cp313-cp313t-musllinux_1_2_riscv64.whl", hash = "sha256:7a4fdb9326aab4a5a477a1640e5ad786a8495901009d7e7b038371edd23a9d2b", size = 192206, upload-time = "2026-05-22T14:48:44.858Z" },- { url = "https://files.pythonhosted.org/packages/df/46/3eea8cde077d985f239a38c0257087b8064fd9ee9b1a99e282d2c86da4ef/wrapt-2.2.1-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:c8cc5094b08abeae52da9c73c8a32003623be691a5193df2f4e3eac3d557c394", size = 198428, upload-time = "2026-05-22T14:48:46.319Z" },- { url = "https://files.pythonhosted.org/packages/18/dc/b927ee9c7fc67adc3a5658f246a0d275425eb840ba36e7b702e70f18bde8/wrapt-2.2.1-cp313-cp313t-win32.whl", hash = "sha256:9907a4402ab6db12b7077a0ea5d7a4d028ecb22c8eee2b53527080d347cd1562", size = 79448, upload-time = "2026-05-22T14:48:47.901Z" },- { url = "https://files.pythonhosted.org/packages/ec/b3/fd30b473fe498c70e6b9a5f328b8d3fbaf1b8c3c481465f59724bba8eb70/wrapt-2.2.1-cp313-cp313t-win_amd64.whl", hash = "sha256:5590d63f5243251641cf543009b4c9314a79d0598fdb8a8e4cfc918494536c53", size = 83021, upload-time = "2026-05-22T14:48:49.201Z" },- { url = "https://files.pythonhosted.org/packages/ee/f3/96c39153a8737a6e9aa85adef254ac4195bea3f2d24efc60472ccc3c9e2e/wrapt-2.2.1-cp313-cp313t-win_arm64.whl", hash = "sha256:c318a64b53d97b841d7b5e637517e50a27be64bc695128422953d4b21710954e", size = 80295, upload-time = "2026-05-22T14:48:50.479Z" },- { url = "https://files.pythonhosted.org/packages/0a/a3/11d7f34ebbf3231bc907a3e6d5ee051b14d034c1bc7b65a97d5cc00516df/wrapt-2.2.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:6f56a647e4eaf5f0ca40330fb070f566bdf9f7b0db89a1af20d71c28dcd7a0ab", size = 80879, upload-time = "2026-05-22T14:48:51.802Z" },- { url = "https://files.pythonhosted.org/packages/13/3c/b74cfd984cef560b900fb1a727af20352d89e1f06bf2e1114dd3f00f5f5a/wrapt-2.2.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:64b7deeda4b70408e382328d8bbe52a256fe9bc63ae3db86d804608367e5422c", size = 81462, upload-time = "2026-05-22T14:48:53.18Z" },- { url = "https://files.pythonhosted.org/packages/15/a3/7c8f704b8dc07dfe0a5d01c2edbfd88317aa8e5e3fa7c743eb7a085ae767/wrapt-2.2.1-cp314-cp314-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:b9cf53ba90717db2e292401de290776c498d4bbfb0d4a559ca2895db8b9dcb5c", size = 167251, upload-time = "2026-05-22T14:48:54.562Z" },- { url = "https://files.pythonhosted.org/packages/80/85/a34d1888d97247da6c2ff6118c3a721c73ed8cc4dd198c00208bb73b6f80/wrapt-2.2.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:cf3638274ab9d9b724c9baa0b4c04e132cd6faefb78b4dd3dd1a02a4bdaad41e", size = 166316, upload-time = "2026-05-22T14:48:56.065Z" },- { url = "https://files.pythonhosted.org/packages/e9/d7/72ffaeb01eebc704afe3fb99e840480f4bda45f0fa66e3381b6a39251c8f/wrapt-2.2.1-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:aed9658797d0b45d6c49adcfc6b41f66e6f2d0c6de3ec79e16cf4b1855df240f", size = 157952, upload-time = "2026-05-22T14:48:57.924Z" },- { url = "https://files.pythonhosted.org/packages/24/5b/36f5d6b024e4edfdd90b140742d11ebcf7836daf5c9daf326c55c24db412/wrapt-2.2.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:1d676ee388bc42a04d56dd7deb5605244dac2e35cc2fadbb43c9fa25bbd93508", size = 166130, upload-time = "2026-05-22T14:48:59.384Z" },- { url = "https://files.pythonhosted.org/packages/81/06/9296d9e97bfdef5483dfcc859d57b095b257144b2bc5300ab521e06f4bc7/wrapt-2.2.1-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:e395f7bc31851ef9b612050368cb446e9bc14cd7454b025018980349caf25ae5", size = 156604, upload-time = "2026-05-22T14:49:00.921Z" },- { url = "https://files.pythonhosted.org/packages/53/37/16953929ed6776175720e58fc966e779926d8d71e2c7b2273230590ca71f/wrapt-2.2.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:5f1845c2a8cc1180ccccfa45785dd06f562730d19ef75be180334254012b6283", size = 166007, upload-time = "2026-05-22T14:49:02.332Z" },- { url = "https://files.pythonhosted.org/packages/b9/73/20ee58c0612dae7c31131a7095345812ed2c7b389019e175f68cde34e5b4/wrapt-2.2.1-cp314-cp314-win32.whl", hash = "sha256:436addbc4bb4fc0a88c702577f51195d7d73683a7f3e0e5b253d8404d7847243", size = 78327, upload-time = "2026-05-22T14:49:03.722Z" },- { url = "https://files.pythonhosted.org/packages/22/b3/ef7c3295d02e0448a71c639a36a057f46d524d057c9486291a7a3039e65c/wrapt-2.2.1-cp314-cp314-win_amd64.whl", hash = "sha256:50972a1d974ea07725a7f6b1cec5f8759008afd030a0024843ebe7d52de47f2b", size = 81144, upload-time = "2026-05-22T14:49:05.093Z" },- { url = "https://files.pythonhosted.org/packages/ac/dc/7bdf336953f99f4ceb0a584bb8870e42c8f26f93ea10c87834dad62f1668/wrapt-2.2.1-cp314-cp314-win_arm64.whl", hash = "sha256:1c9934ea5d92957e3cd0adbc0845539dccfd62710ebe16195a8c66c53954db36", size = 79569, upload-time = "2026-05-22T14:49:06.413Z" },- { url = "https://files.pythonhosted.org/packages/6a/6d/6dfae80150ff1919c356d1dd528f049bcdfaae29b4d284bc957e022caef4/wrapt-2.2.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:17de18fc12cea55b8a9587314cb830573e37fb33b247a7515696350863714188", size = 82892, upload-time = "2026-05-22T14:49:07.925Z" },- { url = "https://files.pythonhosted.org/packages/82/7b/4e34766a7d7804ffce9e71befe47e9b3225dc350c49c94493c4ab39fd3a5/wrapt-2.2.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:a9dec1aca52dddde7df94818310fa2fe79739c8f385b2014c4cb1035f5508199", size = 83333, upload-time = "2026-05-22T14:49:09.257Z" },- { url = "https://files.pythonhosted.org/packages/9d/57/0b34db3e8de44ccfece62d7b337abd1631dd810f5adc5f3db571727836b5/wrapt-2.2.1-cp314-cp314t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:69f2e9244542cb34dd59c7f073445b9e54ad9f3fce8d93606c368a1b499fc413", size = 202899, upload-time = "2026-05-22T14:49:10.572Z" },- { url = "https://files.pythonhosted.org/packages/e5/45/ac0c459f154b99d92789a6cba7ca727185b83513b986f8ec7fe2aacddcbf/wrapt-2.2.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:2d83966dc7f4f45e8b97b5933685ac2e6e67fc0e19246ea314bceb9a8970c956", size = 209986, upload-time = "2026-05-22T14:49:12.229Z" },- { url = "https://files.pythonhosted.org/packages/b7/e4/77e37ff33ad018fa81ade52c25fa327b80b56f81d734279a63614fcb4cbc/wrapt-2.2.1-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:78b0aa6bfb7be8deed0ab23e7aa028cc5210c29bc2d32a04d52b50e517a7307e", size = 194893, upload-time = "2026-05-22T14:49:14.139Z" },- { url = "https://files.pythonhosted.org/packages/dd/9d/7ea651d1ab032fc5fa222fbec91d0f8a1397f6ae04ebb93fa7219aa921d7/wrapt-2.2.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:05d5cb74d1b232ec8cfa130a8f900708699ff2491d97b8f85a4cdc5996294b85", size = 205636, upload-time = "2026-05-22T14:49:15.714Z" },- { url = "https://files.pythonhosted.org/packages/09/af/8e88031a701275b9085c54e64bc88c0b1cd55c77eadd400691c371cd76c4/wrapt-2.2.1-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:f6518b94edb9150452e9aba08027d4cc293433753ec1fbefb4629a21cbc74181", size = 192267, upload-time = "2026-05-22T14:49:17.283Z" },- { url = "https://files.pythonhosted.org/packages/bf/a8/e657ca876b06710194f243d81c4b0896ade646e244bdbec2d87c8c56a8bd/wrapt-2.2.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:ed55af48b3eb28f43228ca2306788892bcb629eb2b5c4876e2a3659872c2f17a", size = 198378, upload-time = "2026-05-22T14:49:18.785Z" },- { url = "https://files.pythonhosted.org/packages/c8/59/822efe4ea722a3961331bfa35b7d90937790d2c20f0616de1997ccc3aebd/wrapt-2.2.1-cp314-cp314t-win32.whl", hash = "sha256:2e08688ab16525897da6589d56d0aebaf417bbe91c2d8e3b96203b1efa596e85", size = 80226, upload-time = "2026-05-22T14:49:20.264Z" },- { url = "https://files.pythonhosted.org/packages/ab/31/2a7dc5f6abb2fca0b6e1610e120419f603650aceb4f1d3ac4cae0354e162/wrapt-2.2.1-cp314-cp314t-win_amd64.whl", hash = "sha256:fd0135d34387f5fd087d9be368ea77ea89cf2451dc1cd1c622d35021bcb3ab50", size = 83835, upload-time = "2026-05-22T14:49:21.634Z" },- { url = "https://files.pythonhosted.org/packages/9f/c0/782b86e28d1ceebeb74cccea12d2cd3d2ba0bd68e3dec20b1bc5873f6127/wrapt-2.2.1-cp314-cp314t-win_arm64.whl", hash = "sha256:f70db64e8266d7c45d3b735f2e08eeb434b5e03da9a479ae42b2e2e486a21a00", size = 80722, upload-time = "2026-05-22T14:49:23.59Z" },- { url = "https://files.pythonhosted.org/packages/53/46/29ac9daf11a86c22a8c38cd9236c62928ccae83f7ceb06bd3b0467cf9d05/wrapt-2.2.1-py3-none-any.whl", hash = "sha256:3aafea2975caef8ca49400640dde02cc7426e798f24870ed01f490bc3cffd32f", size = 61000, upload-time = "2026-05-22T14:49:41.593Z" },-]-[[package]]name = "xxhash"version = "3.7.0"