Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
# Prometheus alert rules for a Switchyard deployment.
#
# Implements the operational and success-criterion alerts documented in
# docs/internal/metrics_reference.md. Drop into Prometheus's rule_files: list and
# validate with `promtool check rules switchyard.rules.yaml` before
# reloading.
#
# Thresholds are conservative defaults. Review against your SLOs in
# examples/prometheus/README.md before wiring to a pager.
groups:
- name: switchyard.routing.evidence
rules:
- alert: RouterOverheadHigh
expr: |
histogram_quantile(
0.99,
sum by (le) (rate(switchyard_routing_overhead_ms_bucket[5m]))
) > 1.0
for: 5m
labels:
severity: warning
component: switchyard
annotations:
summary: "Switchyard router overhead p99 above 1 ms"
description: |
switchyard_routing_overhead_ms p99 has exceeded 1 ms for 5
minutes. Routing overhead includes classifier calls, target
resolution, and decision publication. Cross-check the algorithm
label and classifier traffic in /v1/stats.
- alert: RouterErrorRateNotReducing
# Headline success criterion. The router should serve a lower
# rate of retryable errors than what raw upstream attempts
# would produce. If router_error_rate >= direct_error_rate the
# router is not adding the value it claims to.
expr: |
(
sum(rate(switchyard_client_responses_total{outcome="retryable_error"}[5m]))
/ clamp_min(sum(rate(switchyard_client_responses_total[5m])), 1)
)
>=
(
sum(rate(switchyard_upstream_attempts_total{outcome="retryable_error"}[5m]))
/ clamp_min(sum(rate(switchyard_upstream_attempts_total[5m])), 1)
)
and
sum(rate(switchyard_upstream_attempts_total{outcome="retryable_error"}[5m])) > 0
for: 10m
labels:
severity: warning
component: switchyard
annotations:
summary: "Switchyard router is not reducing client-visible error rate"
description: |
Over the last 5 m, the rate of retryable_error responses to
clients is at least as high as the per-attempt upstream
retryable_error rate. The router is not absorbing failures
via retry — either retries are exhausting or every
endpoint is producing the same failure mode. Look at the
per-model breakdown of switchyard_errors_total{model}.