Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
Commits on main
fix(llm-client): filter inbound Anthropic beta headers (#219)
7a40d51 nachiketb-nvidia committed 3 months ago
chore(switchyard-server): A single version of reqwest (#215)
d495386 Graham King committed 3 months ago
refactor(libsy): classifier::score returns the response alongside the score (#203)
6c3f265 Lin Jia committed 3 months ago
feat(libsy): expose task classifier and stage router in Python (#199)
ace80c8 nachiketb-nvidia committed 3 months ago
feat(server): add native Python host with OpenTelemetry (#210)
50e8d74 nachiketb-nvidia committed 3 months ago
feat(switchyard-server): Support /v1/stats endpoint (#206)
88a9fd3 Graham King committed 3 months ago
feat(libsy): remember context-window evictions for the session (#198)
7a4bb2a ryan-lempka committed 3 months ago
docs(server): document stage_router as a supported route type (#208)
400b368 ryan-lempka committed 3 months ago
refactor(libsy): extract load_judge_config and drop reasoning fallback in judge verdict parsing (#204)
6b9a168 Lin Jia committed 3 months ago
fix(translation): report the served model on every response path (#169)
a3a3d31 eric-liu-nvidia committed 3 months ago
fix(libsy): update escalation judge param defaults to benchmarked values (#201)
54ae66c Lin Jia committed 3 months ago
fix(libsy): preserve classifier system instruction (#195)
f3ff7ff Bryan Bednarski committed 3 months ago
feat(libsy): stage router with handoff notes, per-tier prompts, and LLM fallback (#170)
cb30e92 sabhatinas committed 3 months ago
feat(libsy): evict and reroute on context-window overflow (#187)
27a9af0 ryan-lempka committed 3 months ago
feat(llm-client): add bounded upstream retries (#173)
ef646cd nachiketb-nvidia committed 3 months ago
feat(litellm): add experimental Switchyard integration (#182)
c8ca731 Dhruv Nandakumar committed 3 months ago
feat(switchyard-server): Record routing overhead metric (#188)
b0c9ac5 Graham King committed 3 months ago
feat(llm-client): add request defaults and streaming usage (#184)
1f5d173 nachiketb-nvidia committed 3 months ago
feat(switchyard-server): Metrics for tokens and latency (#179)
36b9636 Graham King committed 3 months ago
refactor(core): replace Rust compatibility APIs with Python (#175)
e4632e3 nachiketb-nvidia committed 3 months ago
feat: task affinity based llm classifier with message hash fallback (#168)
f3e29af Ayush Agarwal committed 3 months ago
refactor(libsy): generalize fall-through state (#158)
61f41ff nachiketb-nvidia committed 3 months ago
feat(server): /v1/messages/count_tokens forwarding on the Rust server (#138)
82d5fbb sabhatinas committed 3 months ago