Skip to content

Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.

Rust

build: Update AGENTS.md (#652)

Graham Kingab4a59cda032
diff --git a/AGENTS.md b/AGENTS.md
index 65d67d4ea..b9a329cfd 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -1,76 +1,37 @@
-# AGENTS.md — Switchyard
+Switchyard is an LLM router library. It sits between an agent's request (e.g. Claude Code, Codex CLI) and an inference server, selecting the best model for that request.
-Switchyard is a Python library for LLM traffic orchestration. It sits between client applications (Claude Code, OpenAI / Anthropic SDK clients, Codex CLI) and LLM backends, handling routing, format translation, logging, A/B testing, and health-aware multi-endpoint serving.
+It is written in Rust with Python bindings.
-> **Note:** The public distribution is `nemo-switchyard`. All imports use `switchyard.*`,
-> and the CLI command is `switchyard` (registered via `pyproject.toml` scripts).
+Core components in `crates/`. These are layered:
+- `libsy`: The core library and routing algorithms. This is the heart of Switchyard. Main entry point is `Algorithm::run_stream` method.
+- `libsy-llm-client`: HTTP client that makes requests for `libsy` algorithms, and drives `run_stream`. Main entry point is `run` function in `run.rs`.
+- `switchyard-runner`: Parsing TOML configuration, uses `libsy-llm-client` to run until the algorithm resolves to the selected model. The entry point is `Runner` struct.
+- `switchyard-server`: A thin HTTP demo server wrapped around `switchyard-runner`. Has a TOML config file.
+- `switchyard-py`: Python bindings for `libsy` and `libsy-llm-client`.
-## Engineering Guidelines
+Support components (also in `crates/`):
+- `protocol`: Types shared between many components.
+- `switchyard-translation`: Convert between various JSON inference formats: OpenAI Chat Completions, OpenAI Responses and Anthropic Messages. We convert to/from a vendor neutral independent representation (IR). All the core components with with this IR.
-Working principles for any agent (or human) writing code in this repo. These are
-about *how* to work; project-specific conventions and validation commands live
-elsewhere in this `AGENTS.md`.
+Integrations:
+- `crates/switchyard-nemo-relay-plugin/`: Integrate with NeMo Relay.
+- `examples/litellm/`: Integrate with LiteLLM.
-### 1. Think Before Coding
+Write for a high-school level in short, simple sentences. Avoid jargon, analogies and metaphors. Be direct.
-**Don't assume. Don't hide confusion. Surface tradeoffs.**
+## Engineering guidance
-Before implementing:
-- State your assumptions explicitly. If uncertain, ask.
-- If multiple interpretations exist, present them — don't pick silently.
-- If a simpler approach exists, say so. Push back when warranted.
-- If something is unclear, stop. Name what's confusing. Ask.
+- Prefer the smallest direct solution.
+- Avoid abstractions, configurability, and defensive code for hypothetical needs.
+- If the implementation grows unexpectedly large, reconsider and simplify it.
+- For bugs, reproduce the failure before fixing it; for refactors, establish a behavioral baseline first.
+- Make reasonable, reversible assumptions when consequences are small. Ask only when ambiguity would materially change the result, expand scope, or risk an irreversible acti
-### 2. Simplicity First
+## Git guidance
-**Minimum code that solves the problem. Nothing speculative.**
+1. Comments Explain Code, Not Project Management
-- No features beyond what was asked.
-- No abstractions for single-use code.
-- No "flexibility" or "configurability" that wasn't requested.
-- No error handling for impossible scenarios.
-- If you write 200 lines and it could be 50, rewrite it.
-
-Ask yourself: "Would a senior engineer say this is overcomplicated?" If yes, simplify.
-
-### 3. Surgical Changes
-
-**Touch only what you must. Clean up only your own mess.**
-
-When editing existing code:
-- Don't "improve" adjacent code, comments, or formatting.
-- Don't refactor things that aren't broken.
-- Match existing style, even if you'd do it differently.
-- If you notice unrelated dead code, mention it — don't delete it.
-
-When your changes create orphans:
-- Remove imports/variables/functions that YOUR changes made unused.
-- Don't remove pre-existing dead code unless asked.
-
-The test: Every changed line should trace directly to the user's request.
-
-### 4. Goal-Driven Execution
-
-**Define success criteria. Loop until verified.**
-
-Transform tasks into verifiable goals:
-- "Add validation" → "Write tests for invalid inputs, then make them pass"
-- "Fix the bug" → "Write a test that reproduces it, then make it pass"
-- "Refactor X" → "Ensure tests pass before and after"
-
-For multi-step tasks, state a brief plan:
-
-```text
-1. [Step] → verify: [check]
-2. [Step] → verify: [check]
-3. [Step] → verify: [check]
-```
-
-Strong success criteria let you loop independently. Weak criteria ("make it work") require constant clarification.
-
-### 5. Comments Explain Code, Not Project Management
-
-**Source comments are about the code. Tracking lives in the tracker.**
+Source comments are about the code. Tracking lives in the tracker.
- No issue/PLAN/step references in code (`TODO(step-6)`, "lands in step 4",
"tracked as ISSUE-001", links to `docs/issues/`). These rot the moment the
@@ -82,9 +43,9 @@ Strong success criteria let you loop independently. Weak criteria ("make it work
- Module doc comments should state what the module is *for*, not its build
schedule or its "empty for now" status.
-### 6. Commit Discipline
+2. Commit Discipline
-**One step, one reviewed, one-line commit.**
+One step, one reviewed, one-line commit.
- One focused commit per step; every changed line traces to that step.
- Single-line commit message in Conventional Commits form
@@ -93,176 +54,3 @@ Strong success criteria let you loop independently. Weak criteria ("make it work
- Use `git commit -s` so every commit carries the required DCO sign-off.
- Never commit unprompted. Show the diff, get approval, then commit.
-Before repairing DCO, inspect every affected commit:
-
-```bash
-git log origin/main..HEAD --format='%h %an <%ae> %s'
-```
-
-If every affected commit is yours, add the trailers and update the remote safely:
-
-```bash
-git rebase origin/main --signoff
-git push --force-with-lease origin HEAD
-```
-
-For a mixed-author branch, use an interactive rebase and mark only your unsigned commits for
-editing. At each stop, run `git commit --amend --no-edit --signoff`, then
-`git rebase --continue`. Never add your sign-off to another contributor's commit.
-
-**These guidelines are working if:** fewer unnecessary changes in diffs, fewer rewrites due to overcomplication, and clarifying questions come before implementation rather than after mistakes.
-
-### 7. Review Discipline
-
-- Verify every finding against the current code path before reporting it.
-- Draft findings before posting review comments unless the user explicitly asks you to post them.
-- Resolve only your own review threads, and only after verifying the fix in code.
-
-## Task-Specific Skills
-
-The repository keeps a small set of optional runbooks under `.agents/skills/`. Read a skill only
-when its description directly matches the task. Ordinary code exploration, implementation,
-testing, and review do not require loading a skill.
-
-| Skill | Use it for |
-|---|---|
-| `publish-python-release` | Python wheel artifacts, PyPI releases, and release workflow changes |
-| `switchyard-docs` | Published MkDocs pages, strict builds, previews, and docs CI |
-| `switchyard-rust-review` | Focused review of Rust, PyO3, async, streaming, and crate boundaries |
-| `switchyard-testing-ci` | Selecting non-obvious validation or diagnosing CI failures |
-
-Skills should contain stable operational constraints, not mutable architecture inventories. Read the
-current source and CI workflows for implementation details.
-
-## Architecture
-
-The supported serving path is native Rust:
-
-```
-HTTP request → switchyard-server → libsy Algorithm → RoutedLlmClient
- → switchyard-translation → upstream model
-```
-
-`switchyard-server` loads explicit TOML deployments and exposes the OpenAI Chat,
-OpenAI Responses, and Anthropic Messages APIs. `switchyard-libsy` owns routing
-algorithms, `switchyard-protocol` owns provider-neutral request and response
-types, and `switchyard-llm-client` performs translated HTTP calls.
-
-Python is an integration layer. `switchyard.libsy` exposes selected algorithms, and
-`switchyard_rust.server` exposes the native server lifecycle through PyO3.
-
-## Project Structure
-
-```
-switchyard/
-├── __init__.py # Package version
-└── libsy/ # typed Python wrappers for libsy algorithms
-
-switchyard_rust/ # Python facades over the PyO3 extension
-crates/libsy/ # routing algorithms and driver
-crates/libsy-llm-client/ # translated HTTP LLM client
-crates/protocol/ # provider-neutral protocol types
-crates/switchyard-server/ # native HTTP server and TOML config
-crates/switchyard-translation/ # wire-format codecs
-crates/switchyard-py/ # libsy and server PyO3 bindings
-crates/switchyard-skill-distillation/ # skill-distillation records and port traits
-crates/switchyard-soak/ # release-candidate soak tester
-
-tests/ # Unit tests (pytest)
-```
-
-## Tech Stack
-
-- **Rust 1.96.1**, edition 2024, Tokio, Axum, and PyO3
-- **Python 3.10+** for native bindings
-- **uv** as the package manager (preferred over pip)
-- **Cargo test + pytest** for testing
-- **ruff** for linting, **mypy** (strict) for type checking
-
-## Setup
-
-```bash
-uv sync # Core + dev tooling (dev is uv's default group)
-uv sync --group dev # Explicit form, equivalent to the above
-source .venv/bin/activate
-```
-
-`dev` lives in `[dependency-groups]` (PEP 735), not in `[project.optional-dependencies]`,
-so it is **not** advertised in the published wheel's METADATA — pytest, ruff, mypy,
-and their transitives never appear in downstream vulnerability scans.
-
-## Commands
-
-### Running Switchyard
-
-```bash
-# Run the standalone native server.
-switchyard-server --config routes.toml --port 4000
-```
-
-### Testing
-
-```bash
-# Unit tests — no API keys needed
-uv run pytest tests/ -v
-
-# Single test file / function
-uv run pytest tests/test_libsy_minimal_bindings.py -v
-
-# Live end-to-end tests are not part of the public test suite; if you write
-# one, set the provider key explicitly and run it directly, e.g.:
-# OPENAI_API_KEY=sk-... uv run pytest tests/your_e2e_test.py -v -x
-
-# Lint / type check (run before every commit)
-uv run ruff check .
-uv run mypy switchyard
-cargo test --workspace
-```
-
-## Environment Variables
-
-| Variable | Purpose |
-|----------|---------|
-| `OPENAI_API_KEY` | API key for OpenAI-compatible backends |
-| `OPENAI_BASE_URL` | Base URL for OpenAI-compatible API |
-| `ANTHROPIC_API_KEY` | API key for Anthropic Claude |
-| `NVIDIA_API_KEY` | API key for NVIDIA NIM / Inference Hub |
-| `OPENROUTER_API_KEY` | API key for OpenRouter-backed routes |
-
-## Code Style
-
-- **Line length**: 100 chars (ruff, E501 ignored)
-- **Target version**: Python 3.10 — use `X | Y` union syntax
-- **Imports**: sorted by ruff (`I` rules). Use `TYPE_CHECKING` guards for circular imports.
-- **File naming**: file name = `snake_case` of its primary class. One class per file when practical.
-- **Type hints**: throughout. `py.typed` marker present; mypy runs strict.
-- **Async**: async-only. If you need sync, use `asyncio.run()`.
-- **Testing**: `respx` for HTTP mocking, `pytest-mock` for general mocking. `asyncio_mode = "auto"` — no `@pytest.mark.asyncio` needed.
-- **Rust**: Panicking calls such as `panic!()`, `unwrap()`, and `.expect()` are not allowed in production source code (they are allowed in tests).
- Propagate errors with `?` — return typed errors, or match explicitly in tests so failures stay intentional and visible.
-- **Comments**: For Rust changes, add concise comments for module/file intent, public structs/enums,
- public methods, private helpers with non-obvious behavior, and tests that encode important
- behavior. Prefer one-line comments when enough. Add block comments before complex validation,
- routing, config-building, async, lifecycle, or concurrency logic.
-- **Docstrings**: Add docstrings for public functions, classes, methods, and API entry points. In
- Rust, use `///` doc comments for public items; in Python, use concise triple-quoted docstrings.
- Public docs should state what the API does, important invariants, and error behavior when relevant.
-
-## Boundaries
-
-### Always do
-- File name = snake_case of the primary class exported. Rename on touch.
-- Run `uv run ruff check .` (zero errors) and `uv run pytest tests/` before pushing.
-- Run `cargo test --workspace` for Rust behavior changes.
-- Write focused unit tests for new behavior and bug fixes.
-- Keep provider-neutral request and response types in `switchyard-protocol`.
-- Map upstream context-window errors to `SwitchyardError::ContextWindowExceeded`.
-
-### Ask first
-- Modifying `pyproject.toml` dependencies.
-- Adding new HTTP endpoints.
-- Removing or renaming public Rust, PyO3, or Python APIs.
-
-### Never do
-- Commit API keys or secrets (`secrets/` is gitignored).
-- Remove or rename public API exports without an explicit deprecation plan.