Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
fix(metrics): unify outcome label vocabulary to ok (#485) (#609)
Talha Demir6cfbca2f49b3
diff --git a/crates/libsy-llm-client/src/client.rs b/crates/libsy-llm-client/src/client.rsindex 9f47828aa..371fc586f 100644--- a/crates/libsy-llm-client/src/client.rs+++ b/crates/libsy-llm-client/src/client.rs@@ -289,7 +289,7 @@ impl TranslatingLlmClient {// The retained handle updates this same attempt span with its outcome.match result {Ok(response) => {- span.record("outcome", "success");+ span.record("outcome", "ok");span.record("status_code", response.status());span.record("will_retry", false);if attempt > 0 {diff --git a/crates/libsy-llm-client/src/metrics.rs b/crates/libsy-llm-client/src/metrics.rsindex bf22b421e..76730ebfe 100644--- a/crates/libsy-llm-client/src/metrics.rs+++ b/crates/libsy-llm-client/src/metrics.rs@@ -42,9 +42,13 @@ pub(crate) const fn is_retryable_http_status(status: u16) -> bool {status == 408 || status == 429 || (status >= 500 && status <= 599)}+/// Maps an HTTP status code (or lack thereof) to a normalized outcome metric label:+/// - `"ok"` for HTTP 2xx success responses.+/// - `"retryable_error"` for retryable HTTP statuses (408, 429, 5xx) or non-HTTP failures (`None`).+/// - `"other_error"` for all other client or non-retryable statuses.pub const fn http_outcome_label(status: Option<u16>) -> &'static str {match status {- Some(200..=299) => "success",+ Some(200..=299) => "ok",Some(status) if is_retryable_http_status(status) => "retryable_error",None => "retryable_error",Some(_) => "other_error",@@ -170,6 +174,7 @@ mod tests {#[test]fn outcome_labels_match_the_retry_policy() {+ assert_eq!(http_outcome_label(Some(200)), "ok");for status in [408, 429, 500, 502, 599] {assert!(is_retryable_http_status(status));assert_eq!(http_outcome_label(Some(status)), "retryable_error");diff --git a/crates/switchyard-server/src/metrics.rs b/crates/switchyard-server/src/metrics.rsindex 1f05c6f75..d1d75f327 100644--- a/crates/switchyard-server/src/metrics.rs+++ b/crates/switchyard-server/src/metrics.rs@@ -129,7 +129,7 @@ fn seed_outcome_metrics() {let client_responses = meter.u64_counter("switchyard.client_responses").build();for outcome in [- "success",+ "ok","retryable_error","other_error",CLIENT_DISCONNECTED_OUTCOME,diff --git a/crates/switchyard-server/tests/server.rs b/crates/switchyard-server/tests/server.rsindex 4fca26f02..a45fa7151 100644--- a/crates/switchyard-server/tests/server.rs+++ b/crates/switchyard-server/tests/server.rs@@ -694,11 +694,11 @@ async fn metrics_exposes_switchyard_otel_instruments() -> TestResult {let seeded = before.text()?;for expected in ["# TYPE switchyard_client_responses_total counter",- "switchyard_client_responses_total{outcome=\"success\",",+ "switchyard_client_responses_total{outcome=\"ok\",","switchyard_client_responses_total{outcome=\"retryable_error\",","switchyard_client_responses_total{outcome=\"other_error\",","# TYPE switchyard_upstream_attempts_total counter",- "switchyard_upstream_attempts_total{code=\"200\",outcome=\"success\",",+ "switchyard_upstream_attempts_total{code=\"200\",outcome=\"ok\",","switchyard_upstream_attempts_total{code=\"429\",outcome=\"retryable_error\",","switchyard_upstream_attempts_total{code=\"500\",outcome=\"retryable_error\",","switchyard_upstream_attempts_total{code=\"504\",outcome=\"retryable_error\",",@@ -742,8 +742,8 @@ async fn metrics_exposes_switchyard_otel_instruments() -> TestResult {"# TYPE switchyard_total_errors gauge","# TYPE switchyard_requests_total counter","# TYPE switchyard_model_call_latency_ms histogram",- "switchyard_client_responses_total{outcome=\"success\",",- "switchyard_upstream_attempts_total{code=\"200\",outcome=\"success\",",+ "switchyard_client_responses_total{outcome=\"ok\",",+ "switchyard_upstream_attempts_total{code=\"200\",outcome=\"ok\",","# TYPE switchyard_runs_total counter","# TYPE switchyard_run_duration_ms histogram","# TYPE switchyard_prompt_tokens_total counter",diff --git a/docs/internal/metrics_reference.md b/docs/internal/metrics_reference.mdindex 464c5d737..782f99f90 100644--- a/docs/internal/metrics_reference.md+++ b/docs/internal/metrics_reference.md@@ -77,12 +77,14 @@ Each histogram emits `_bucket`, `_sum`, and `_count` series. Use## Outcome counters for error-rate ratios-The `outcome` label takes exactly three values:+For HTTP-derived responses and attempts, the `outcome` label takes three values:-* `success` = HTTP 2xx+* `ok` = HTTP 2xx* `retryable_error` = HTTP 408, 429, any 5xx, or a failure before an HTTP status* `other_error` = everything else (400, 401, 403, 422, …)+In addition, `switchyard_client_responses_total` records `outcome="client_disconnected"` when a client drops the connection before a response is returned.+| Metric | Type | Meaning ||---|---|---|| `switchyard_client_responses_total{outcome}` | counter | HTTP responses returned to clients on the LLM-serving routes (`/v1/chat/completions`, `/v1/messages`, `/v1/responses`). The denominator for the **router-served** error rate. |@@ -152,7 +154,7 @@ into label space.| Label | Values | Where ||---|---|---|| `model` | One per configured endpoint, typically 2–6 per deployment. | All per-endpoint metrics. |-| `outcome` | Exactly 3: `success`, `retryable_error`, `other_error`. | Outcome counters |+| `outcome` | `ok`, `retryable_error`, `other_error`; plus `client_disconnected` on `switchyard_client_responses_total`. | Outcome counters || `code` | Bounded: the known-code allowlist (`200`, `400`, `401`, `403`, `404`, `408`, `409`, `422`, `429`, `500`, `502`, `503`, `504`), plus `none` and the per-class buckets `1xx`/`2xx`/`3xx`/`4xx`/`5xx`/`other`. About 20 values max. | `switchyard_upstream_attempts_total` || `le` | The configured histogram bucket boundaries. | Histogram buckets || `algorithm` | One stable value per configured algorithm. | Routing-overhead histogram, in-flight gauge |