Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
docs(telemetry): document emitted LLM client metrics (#787)
Graham King1dfbf3475961
diff --git a/docs/reference/opentelemetry.md b/docs/reference/opentelemetry.mdindex 35a604af8..dae067c06 100644--- a/docs/reference/opentelemetry.md+++ b/docs/reference/opentelemetry.md@@ -85,6 +85,11 @@ Metrics use the `switchyard` meter scope. The tables use OTel instrument names.| `switchyard.decisions` | Counter | `algorithm`, `selected_model` | Published routing decisions. || `switchyard.llm_calls` | Counter | `algorithm`, `selected_model`, `outcome` | Logical offloaded and terminal model calls. || `switchyard.llm_call_duration_ms` | Histogram | `algorithm`, `selected_model`, `outcome` | Logical call duration in milliseconds; see streaming limits below. |+| `switchyard.total_requests` | ObservableGauge | none | Process-wide total of successful and failed answer candidates after routing, including reused routing responses. |+| `switchyard.total_errors` | ObservableGauge | none | Process-wide total of failures after routing, including failed answer candidates and reused routing responses. |+| `switchyard.requests` | Counter | `model` | Successful answer candidates or reused routing responses, by model ID. |+| `switchyard.errors` | Counter | `model` | Failed answer candidates or reused routing responses, by model ID. |+| `switchyard.model_call_latency_ms` | Histogram | `model` | Successful answer-candidate duration in milliseconds, including that candidate's retries and stream consumption. Excludes routing time and reused routing responses. || `switchyard.routing_overhead_ms` | Histogram | `algorithm` | LLM client driver's time to obtain a successful routing outcome, including judge calls but excluding any subsequent answer call. || `switchyard.classifier_fail_open` | Counter | `judge_model`, `reason` | Judge failures that caused classification to proceed without a verdict. || `switchyard.upstream_attempts` | Counter | `outcome`, `code` | HTTP attempts, including retries, made by the translating client. |@@ -100,6 +105,22 @@ Logical calls, candidate calls, and HTTP attempts are different counts. Candidatfallbacks and HTTP retries remain within one logical call. A response producedduring routing is not counted again as a new terminal model call.+The five request/error/latency instruments above record each answer candidate after+routing, not each HTTP attempt. A failed candidate followed by a successful fallback+adds two to `total_requests`, one to `total_errors`, one to `errors` for the failed+model, and one to `requests` for the successful model. Routing-time calls and failures+before a routing outcome do not contribute. A response reused from routing contributes+one to `total_requests` and one to either `requests` or `errors` (also `total_errors`+on failure), but no `model_call_latency_ms` sample. The `model` label is the+candidate's configured model ID (the serving model for a reused response), not the+`selected_model` label used by logical-call metrics. The two total gauges are cumulative+process-wide compatibility values, not counts of requests currently in flight.++For streams, these five instruments record when the response stream ends or is dropped.+Stream errors count as failures. Dropping an unfinished stream also counts as a failure;+a terminal message permits a successful drop. Failed calls have no+`model_call_latency_ms` sample.+### Algorithm-specific metricsStage Router instruments use the prefix `switchyard.stage_router.`: