Skip to content

Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.

Rust

fix(metrics): unify outcome label vocabulary to ok (#485) (#609)

Talha Demir6cfbca2f49b3
diff --git a/crates/libsy-llm-client/src/client.rs b/crates/libsy-llm-client/src/client.rs
index 9f47828aa..371fc586f 100644
--- a/crates/libsy-llm-client/src/client.rs
+++ b/crates/libsy-llm-client/src/client.rs
@@ -289,7 +289,7 @@ impl TranslatingLlmClient {
// The retained handle updates this same attempt span with its outcome.
match result {
Ok(response) => {
- span.record("outcome", "success");
+ span.record("outcome", "ok");
span.record("status_code", response.status());
span.record("will_retry", false);
if attempt > 0 {
diff --git a/crates/libsy-llm-client/src/metrics.rs b/crates/libsy-llm-client/src/metrics.rs
index bf22b421e..76730ebfe 100644
--- a/crates/libsy-llm-client/src/metrics.rs
+++ b/crates/libsy-llm-client/src/metrics.rs
@@ -42,9 +42,13 @@ pub(crate) const fn is_retryable_http_status(status: u16) -> bool {
status == 408 || status == 429 || (status >= 500 && status <= 599)
}
+/// Maps an HTTP status code (or lack thereof) to a normalized outcome metric label:
+/// - `"ok"` for HTTP 2xx success responses.
+/// - `"retryable_error"` for retryable HTTP statuses (408, 429, 5xx) or non-HTTP failures (`None`).
+/// - `"other_error"` for all other client or non-retryable statuses.
pub const fn http_outcome_label(status: Option<u16>) -> &'static str {
match status {
- Some(200..=299) => "success",
+ Some(200..=299) => "ok",
Some(status) if is_retryable_http_status(status) => "retryable_error",
None => "retryable_error",
Some(_) => "other_error",
@@ -170,6 +174,7 @@ mod tests {
#[test]
fn outcome_labels_match_the_retry_policy() {
+ assert_eq!(http_outcome_label(Some(200)), "ok");
for status in [408, 429, 500, 502, 599] {
assert!(is_retryable_http_status(status));
assert_eq!(http_outcome_label(Some(status)), "retryable_error");
diff --git a/crates/switchyard-server/src/metrics.rs b/crates/switchyard-server/src/metrics.rs
index 1f05c6f75..d1d75f327 100644
--- a/crates/switchyard-server/src/metrics.rs
+++ b/crates/switchyard-server/src/metrics.rs
@@ -129,7 +129,7 @@ fn seed_outcome_metrics() {
let client_responses = meter.u64_counter("switchyard.client_responses").build();
for outcome in [
- "success",
+ "ok",
"retryable_error",
"other_error",
CLIENT_DISCONNECTED_OUTCOME,
diff --git a/crates/switchyard-server/tests/server.rs b/crates/switchyard-server/tests/server.rs
index 4fca26f02..a45fa7151 100644
--- a/crates/switchyard-server/tests/server.rs
+++ b/crates/switchyard-server/tests/server.rs
@@ -694,11 +694,11 @@ async fn metrics_exposes_switchyard_otel_instruments() -> TestResult {
let seeded = before.text()?;
for expected in [
"# TYPE switchyard_client_responses_total counter",
- "switchyard_client_responses_total{outcome=\"success\",",
+ "switchyard_client_responses_total{outcome=\"ok\",",
"switchyard_client_responses_total{outcome=\"retryable_error\",",
"switchyard_client_responses_total{outcome=\"other_error\",",
"# TYPE switchyard_upstream_attempts_total counter",
- "switchyard_upstream_attempts_total{code=\"200\",outcome=\"success\",",
+ "switchyard_upstream_attempts_total{code=\"200\",outcome=\"ok\",",
"switchyard_upstream_attempts_total{code=\"429\",outcome=\"retryable_error\",",
"switchyard_upstream_attempts_total{code=\"500\",outcome=\"retryable_error\",",
"switchyard_upstream_attempts_total{code=\"504\",outcome=\"retryable_error\",",
@@ -742,8 +742,8 @@ async fn metrics_exposes_switchyard_otel_instruments() -> TestResult {
"# TYPE switchyard_total_errors gauge",
"# TYPE switchyard_requests_total counter",
"# TYPE switchyard_model_call_latency_ms histogram",
- "switchyard_client_responses_total{outcome=\"success\",",
- "switchyard_upstream_attempts_total{code=\"200\",outcome=\"success\",",
+ "switchyard_client_responses_total{outcome=\"ok\",",
+ "switchyard_upstream_attempts_total{code=\"200\",outcome=\"ok\",",
"# TYPE switchyard_runs_total counter",
"# TYPE switchyard_run_duration_ms histogram",
"# TYPE switchyard_prompt_tokens_total counter",
diff --git a/docs/internal/metrics_reference.md b/docs/internal/metrics_reference.md
index 464c5d737..782f99f90 100644
--- a/docs/internal/metrics_reference.md
+++ b/docs/internal/metrics_reference.md
@@ -77,12 +77,14 @@ Each histogram emits `_bucket`, `_sum`, and `_count` series. Use
## Outcome counters for error-rate ratios
-The `outcome` label takes exactly three values:
+For HTTP-derived responses and attempts, the `outcome` label takes three values:
-* `success` = HTTP 2xx
+* `ok` = HTTP 2xx
* `retryable_error` = HTTP 408, 429, any 5xx, or a failure before an HTTP status
* `other_error` = everything else (400, 401, 403, 422, …)
+In addition, `switchyard_client_responses_total` records `outcome="client_disconnected"` when a client drops the connection before a response is returned.
+
| Metric | Type | Meaning |
|---|---|---|
| `switchyard_client_responses_total{outcome}` | counter | HTTP responses returned to clients on the LLM-serving routes (`/v1/chat/completions`, `/v1/messages`, `/v1/responses`). The denominator for the **router-served** error rate. |
@@ -152,7 +154,7 @@ into label space.
| Label | Values | Where |
|---|---|---|
| `model` | One per configured endpoint, typically 2–6 per deployment. | All per-endpoint metrics. |
-| `outcome` | Exactly 3: `success`, `retryable_error`, `other_error`. | Outcome counters |
+| `outcome` | `ok`, `retryable_error`, `other_error`; plus `client_disconnected` on `switchyard_client_responses_total`. | Outcome counters |
| `code` | Bounded: the known-code allowlist (`200`, `400`, `401`, `403`, `404`, `408`, `409`, `422`, `429`, `500`, `502`, `503`, `504`), plus `none` and the per-class buckets `1xx`/`2xx`/`3xx`/`4xx`/`5xx`/`other`. About 20 values max. | `switchyard_upstream_attempts_total` |
| `le` | The configured histogram bucket boundaries. | Histogram buckets |
| `algorithm` | One stable value per configured algorithm. | Routing-overhead histogram, in-flight gauge |