Skip to content

Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.

Rust
2.5 KB
Raw Blame
# Prometheus alert rules for a Switchyard deployment.
#
# Implements the operational and success-criterion alerts documented in
# docs/internal/metrics_reference.md. Drop into Prometheus's rule_files: list and
# validate with `promtool check rules switchyard.rules.yaml` before
# reloading.
#
# Thresholds are conservative defaults. Review against your SLOs in
# examples/prometheus/README.md before wiring to a pager.

groups:
  - name: switchyard.routing.evidence
    rules:
      - alert: RouterOverheadHigh
        expr: |
          histogram_quantile(
            0.99,
            sum by (le) (rate(switchyard_routing_overhead_ms_bucket[5m]))
          ) > 1.0
        for: 5m
        labels:
          severity: warning
          component: switchyard
        annotations:
          summary: "Switchyard router overhead p99 above 1 ms"
          description: |
            switchyard_routing_overhead_ms p99 has exceeded 1 ms for 5
            minutes. Routing overhead includes classifier calls, target
            resolution, and decision publication. Cross-check the algorithm
            label and classifier traffic in /v1/stats.

      - alert: RouterErrorRateNotReducing
        # Headline success criterion. The router should serve a lower
        # rate of retryable errors than what raw upstream attempts
        # would produce. If router_error_rate >= direct_error_rate the
        # router is not adding the value it claims to.
        expr: |
          (
            sum(rate(switchyard_client_responses_total{outcome="retryable_error"}[5m]))
            / clamp_min(sum(rate(switchyard_client_responses_total[5m])), 1)
          )
          >=
          (
            sum(rate(switchyard_upstream_attempts_total{outcome="retryable_error"}[5m]))
            / clamp_min(sum(rate(switchyard_upstream_attempts_total[5m])), 1)
          )
          and
          sum(rate(switchyard_upstream_attempts_total{outcome="retryable_error"}[5m])) > 0
        for: 10m
        labels:
          severity: warning
          component: switchyard
        annotations:
          summary: "Switchyard router is not reducing client-visible error rate"
          description: |
            Over the last 5 m, the rate of retryable_error responses to
            clients is at least as high as the per-attempt upstream
            retryable_error rate. The router is not absorbing failures
            via retry — either retries are exhausting or every
            endpoint is producing the same failure mode. Look at the
            per-model breakdown of switchyard_errors_total{model}.