Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
build: Update AGENTS.md (#652)
Graham Kingab4a59cda032
diff --git a/AGENTS.md b/AGENTS.mdindex 65d67d4ea..b9a329cfd 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -1,76 +1,37 @@-# AGENTS.md — Switchyard+Switchyard is an LLM router library. It sits between an agent's request (e.g. Claude Code, Codex CLI) and an inference server, selecting the best model for that request.-Switchyard is a Python library for LLM traffic orchestration. It sits between client applications (Claude Code, OpenAI / Anthropic SDK clients, Codex CLI) and LLM backends, handling routing, format translation, logging, A/B testing, and health-aware multi-endpoint serving.+It is written in Rust with Python bindings.-> **Note:** The public distribution is `nemo-switchyard`. All imports use `switchyard.*`,-> and the CLI command is `switchyard` (registered via `pyproject.toml` scripts).+Core components in `crates/`. These are layered:+- `libsy`: The core library and routing algorithms. This is the heart of Switchyard. Main entry point is `Algorithm::run_stream` method.+- `libsy-llm-client`: HTTP client that makes requests for `libsy` algorithms, and drives `run_stream`. Main entry point is `run` function in `run.rs`.+- `switchyard-runner`: Parsing TOML configuration, uses `libsy-llm-client` to run until the algorithm resolves to the selected model. The entry point is `Runner` struct.+- `switchyard-server`: A thin HTTP demo server wrapped around `switchyard-runner`. Has a TOML config file.+- `switchyard-py`: Python bindings for `libsy` and `libsy-llm-client`.-## Engineering Guidelines+Support components (also in `crates/`):+- `protocol`: Types shared between many components.+- `switchyard-translation`: Convert between various JSON inference formats: OpenAI Chat Completions, OpenAI Responses and Anthropic Messages. We convert to/from a vendor neutral independent representation (IR). All the core components with with this IR.-Working principles for any agent (or human) writing code in this repo. These are-about *how* to work; project-specific conventions and validation commands live-elsewhere in this `AGENTS.md`.+Integrations:+- `crates/switchyard-nemo-relay-plugin/`: Integrate with NeMo Relay.+- `examples/litellm/`: Integrate with LiteLLM.-### 1. Think Before Coding+Write for a high-school level in short, simple sentences. Avoid jargon, analogies and metaphors. Be direct.-**Don't assume. Don't hide confusion. Surface tradeoffs.**+## Engineering guidance-Before implementing:-- State your assumptions explicitly. If uncertain, ask.-- If multiple interpretations exist, present them — don't pick silently.-- If a simpler approach exists, say so. Push back when warranted.-- If something is unclear, stop. Name what's confusing. Ask.+- Prefer the smallest direct solution.+- Avoid abstractions, configurability, and defensive code for hypothetical needs.+- If the implementation grows unexpectedly large, reconsider and simplify it.+- For bugs, reproduce the failure before fixing it; for refactors, establish a behavioral baseline first.+- Make reasonable, reversible assumptions when consequences are small. Ask only when ambiguity would materially change the result, expand scope, or risk an irreversible acti-### 2. Simplicity First+## Git guidance-**Minimum code that solves the problem. Nothing speculative.**+1. Comments Explain Code, Not Project Management-- No features beyond what was asked.-- No abstractions for single-use code.-- No "flexibility" or "configurability" that wasn't requested.-- No error handling for impossible scenarios.-- If you write 200 lines and it could be 50, rewrite it.--Ask yourself: "Would a senior engineer say this is overcomplicated?" If yes, simplify.--### 3. Surgical Changes--**Touch only what you must. Clean up only your own mess.**--When editing existing code:-- Don't "improve" adjacent code, comments, or formatting.-- Don't refactor things that aren't broken.-- Match existing style, even if you'd do it differently.-- If you notice unrelated dead code, mention it — don't delete it.--When your changes create orphans:-- Remove imports/variables/functions that YOUR changes made unused.-- Don't remove pre-existing dead code unless asked.--The test: Every changed line should trace directly to the user's request.--### 4. Goal-Driven Execution--**Define success criteria. Loop until verified.**--Transform tasks into verifiable goals:-- "Add validation" → "Write tests for invalid inputs, then make them pass"-- "Fix the bug" → "Write a test that reproduces it, then make it pass"-- "Refactor X" → "Ensure tests pass before and after"--For multi-step tasks, state a brief plan:--```text-1. [Step] → verify: [check]-2. [Step] → verify: [check]-3. [Step] → verify: [check]-```--Strong success criteria let you loop independently. Weak criteria ("make it work") require constant clarification.--### 5. Comments Explain Code, Not Project Management--**Source comments are about the code. Tracking lives in the tracker.**+Source comments are about the code. Tracking lives in the tracker.- No issue/PLAN/step references in code (`TODO(step-6)`, "lands in step 4","tracked as ISSUE-001", links to `docs/issues/`). These rot the moment the@@ -82,9 +43,9 @@ Strong success criteria let you loop independently. Weak criteria ("make it work- Module doc comments should state what the module is *for*, not its buildschedule or its "empty for now" status.-### 6. Commit Discipline+2. Commit Discipline-**One step, one reviewed, one-line commit.**+One step, one reviewed, one-line commit.- One focused commit per step; every changed line traces to that step.- Single-line commit message in Conventional Commits form@@ -93,176 +54,3 @@ Strong success criteria let you loop independently. Weak criteria ("make it work- Use `git commit -s` so every commit carries the required DCO sign-off.- Never commit unprompted. Show the diff, get approval, then commit.-Before repairing DCO, inspect every affected commit:--```bash-git log origin/main..HEAD --format='%h %an <%ae> %s'-```--If every affected commit is yours, add the trailers and update the remote safely:--```bash-git rebase origin/main --signoff-git push --force-with-lease origin HEAD-```--For a mixed-author branch, use an interactive rebase and mark only your unsigned commits for-editing. At each stop, run `git commit --amend --no-edit --signoff`, then-`git rebase --continue`. Never add your sign-off to another contributor's commit.--**These guidelines are working if:** fewer unnecessary changes in diffs, fewer rewrites due to overcomplication, and clarifying questions come before implementation rather than after mistakes.--### 7. Review Discipline--- Verify every finding against the current code path before reporting it.-- Draft findings before posting review comments unless the user explicitly asks you to post them.-- Resolve only your own review threads, and only after verifying the fix in code.--## Task-Specific Skills--The repository keeps a small set of optional runbooks under `.agents/skills/`. Read a skill only-when its description directly matches the task. Ordinary code exploration, implementation,-testing, and review do not require loading a skill.--| Skill | Use it for |-|---|---|-| `publish-python-release` | Python wheel artifacts, PyPI releases, and release workflow changes |-| `switchyard-docs` | Published MkDocs pages, strict builds, previews, and docs CI |-| `switchyard-rust-review` | Focused review of Rust, PyO3, async, streaming, and crate boundaries |-| `switchyard-testing-ci` | Selecting non-obvious validation or diagnosing CI failures |--Skills should contain stable operational constraints, not mutable architecture inventories. Read the-current source and CI workflows for implementation details.--## Architecture--The supported serving path is native Rust:--```-HTTP request → switchyard-server → libsy Algorithm → RoutedLlmClient- → switchyard-translation → upstream model-```--`switchyard-server` loads explicit TOML deployments and exposes the OpenAI Chat,-OpenAI Responses, and Anthropic Messages APIs. `switchyard-libsy` owns routing-algorithms, `switchyard-protocol` owns provider-neutral request and response-types, and `switchyard-llm-client` performs translated HTTP calls.--Python is an integration layer. `switchyard.libsy` exposes selected algorithms, and-`switchyard_rust.server` exposes the native server lifecycle through PyO3.--## Project Structure--```-switchyard/-├── __init__.py # Package version-└── libsy/ # typed Python wrappers for libsy algorithms--switchyard_rust/ # Python facades over the PyO3 extension-crates/libsy/ # routing algorithms and driver-crates/libsy-llm-client/ # translated HTTP LLM client-crates/protocol/ # provider-neutral protocol types-crates/switchyard-server/ # native HTTP server and TOML config-crates/switchyard-translation/ # wire-format codecs-crates/switchyard-py/ # libsy and server PyO3 bindings-crates/switchyard-skill-distillation/ # skill-distillation records and port traits-crates/switchyard-soak/ # release-candidate soak tester--tests/ # Unit tests (pytest)-```--## Tech Stack--- **Rust 1.96.1**, edition 2024, Tokio, Axum, and PyO3-- **Python 3.10+** for native bindings-- **uv** as the package manager (preferred over pip)-- **Cargo test + pytest** for testing-- **ruff** for linting, **mypy** (strict) for type checking--## Setup--```bash-uv sync # Core + dev tooling (dev is uv's default group)-uv sync --group dev # Explicit form, equivalent to the above-source .venv/bin/activate-```--`dev` lives in `[dependency-groups]` (PEP 735), not in `[project.optional-dependencies]`,-so it is **not** advertised in the published wheel's METADATA — pytest, ruff, mypy,-and their transitives never appear in downstream vulnerability scans.--## Commands--### Running Switchyard--```bash-# Run the standalone native server.-switchyard-server --config routes.toml --port 4000-```--### Testing--```bash-# Unit tests — no API keys needed-uv run pytest tests/ -v--# Single test file / function-uv run pytest tests/test_libsy_minimal_bindings.py -v--# Live end-to-end tests are not part of the public test suite; if you write-# one, set the provider key explicitly and run it directly, e.g.:-# OPENAI_API_KEY=sk-... uv run pytest tests/your_e2e_test.py -v -x--# Lint / type check (run before every commit)-uv run ruff check .-uv run mypy switchyard-cargo test --workspace-```--## Environment Variables--| Variable | Purpose |-|----------|---------|-| `OPENAI_API_KEY` | API key for OpenAI-compatible backends |-| `OPENAI_BASE_URL` | Base URL for OpenAI-compatible API |-| `ANTHROPIC_API_KEY` | API key for Anthropic Claude |-| `NVIDIA_API_KEY` | API key for NVIDIA NIM / Inference Hub |-| `OPENROUTER_API_KEY` | API key for OpenRouter-backed routes |--## Code Style--- **Line length**: 100 chars (ruff, E501 ignored)-- **Target version**: Python 3.10 — use `X | Y` union syntax-- **Imports**: sorted by ruff (`I` rules). Use `TYPE_CHECKING` guards for circular imports.-- **File naming**: file name = `snake_case` of its primary class. One class per file when practical.-- **Type hints**: throughout. `py.typed` marker present; mypy runs strict.-- **Async**: async-only. If you need sync, use `asyncio.run()`.-- **Testing**: `respx` for HTTP mocking, `pytest-mock` for general mocking. `asyncio_mode = "auto"` — no `@pytest.mark.asyncio` needed.-- **Rust**: Panicking calls such as `panic!()`, `unwrap()`, and `.expect()` are not allowed in production source code (they are allowed in tests).- Propagate errors with `?` — return typed errors, or match explicitly in tests so failures stay intentional and visible.-- **Comments**: For Rust changes, add concise comments for module/file intent, public structs/enums,- public methods, private helpers with non-obvious behavior, and tests that encode important- behavior. Prefer one-line comments when enough. Add block comments before complex validation,- routing, config-building, async, lifecycle, or concurrency logic.-- **Docstrings**: Add docstrings for public functions, classes, methods, and API entry points. In- Rust, use `///` doc comments for public items; in Python, use concise triple-quoted docstrings.- Public docs should state what the API does, important invariants, and error behavior when relevant.--## Boundaries--### Always do-- File name = snake_case of the primary class exported. Rename on touch.-- Run `uv run ruff check .` (zero errors) and `uv run pytest tests/` before pushing.-- Run `cargo test --workspace` for Rust behavior changes.-- Write focused unit tests for new behavior and bug fixes.-- Keep provider-neutral request and response types in `switchyard-protocol`.-- Map upstream context-window errors to `SwitchyardError::ContextWindowExceeded`.--### Ask first-- Modifying `pyproject.toml` dependencies.-- Adding new HTTP endpoints.-- Removing or renaming public Rust, PyO3, or Python APIs.--### Never do-- Commit API keys or secrets (`secrets/` is gitignored).-- Remove or rename public API exports without an explicit deprecation plan.