Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
docs: explain Switchyard LLM and decision protocols (#1143)
nachiketb-nvidia29d113adff04
diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.ymlindex a6debccc3..2aa07f961 100644--- a/.github/workflows/docs.yml+++ b/.github/workflows/docs.yml@@ -12,6 +12,7 @@ on:- "Cargo.lock"- "rust-toolchain.toml"- "crates/libsy/**"+ - "crates/libsy-llm-client/**"- "crates/protocol/**"- "crates/switchyard-translation/**"- ".github/workflows/docs.yml"@@ -27,6 +28,7 @@ on:- "Cargo.lock"- "rust-toolchain.toml"- "crates/libsy/**"+ - "crates/libsy-llm-client/**"- "crates/protocol/**"- "crates/switchyard-translation/**"- ".github/workflows/docs.yml"@@ -64,6 +66,7 @@ jobs:--package switchyard-libsy--package switchyard-protocol--package switchyard-translation+ --package switchyard-llm-client- name: Add crate documentation to siterun: |mkdir -p site/rustdiff --git a/docs/mechanics_of_libsy.md b/docs/mechanics_of_libsy.mdindex 2e05a9d44..3752724fb 100644--- a/docs/mechanics_of_libsy.md+++ b/docs/mechanics_of_libsy.md@@ -404,5 +404,7 @@ or undo side effects that have already occurred. See the [host integrationguide](reference/host_integration.md#concurrent-calls-and-cancellation) forPython task cleanup.-The [Rust API reference](reference/rust_api.md) links the complete crate-documentation for these types.+Next, [Switchyard Protocols](protocols/index.md) explains the request and response+data inside these calls and how it maps to provider APIs. The+[Rust API reference](reference/rust_api.md) links the complete crate documentation+for these types.diff --git a/docs/protocols/decisions.md b/docs/protocols/decisions.mdnew file mode 100644index 000000000..daf397551--- /dev/null+++ b/docs/protocols/decisions.md@@ -0,0 +1,132 @@+# Decision Model-Related Protocols++A decision model evaluates a context against named questions and returns typed+answers. An algorithm can use those answers to choose a model or decide what+work to request next.++Switchyard represents this exchange with+<a href="../../rust/switchyard_protocol/decision/struct.DecisionRequest.html"><code>DecisionRequest</code></a>+and+<a href="../../rust/switchyard_protocol/decision/struct.DecisionResponse.html"><code>DecisionResponse</code></a>.+These are the types exchanged through <a href="../../rust/switchyard_libsy/enum.Step.html#variant.CallDecision"><code>Step::CallDecision</code></a>.++System One is a common API for open decision models, including Jev.+Switchyard supports it alongside OpenAI Decisions through the same protocol.++!!! info "Question fields, answer types, and API mappings"++ - **[Question reference](../reference/protocols/decision_questions.md):** shared context, question IDs, Boolean/choice/score questions, and provider request mappings.+ - **[Answer reference](../reference/protocols/decision_answers.md):** probabilities, rubric scores, confidence, usage, provider response mappings, and errors.++## Describe the questions++One request carries shared context and independent questions keyed by ID.+Each <a href="../../rust/switchyard_protocol/decision/struct.DecisionQuestion.html"><code>DecisionQuestion</code></a>+defines what to evaluate and which answer kind to return. The response uses the+same question IDs, so the algorithm can match each answer to its question.++Here is a request asking whether a task contains private information. This is+request construction only; it does not call a provider.++```rust+use std::collections::BTreeMap;+use serde_json::json;+use switchyard_protocol::{DecisionKind, DecisionQuestion, DecisionRequest};++let request = DecisionRequest {+ model: Some("privacy-judge".into()),+ context: json!({"task": "Summarize this customer's account history."}),+ questions: BTreeMap::from([(+ "contains_private_data".into(),+ DecisionQuestion {+ instructions: json!("Does the task contain private customer data?"),+ kind: DecisionKind::Boolean {+ true_description: Some(json!("Contains private customer data")),+ false_description: Some(json!("Contains only public information")),+ },+ },+ )]),+};+```++For required option IDs, rubric levels, and validation responsibilities, see+**[question construction and validation](../reference/protocols/decision_questions.md#construction-and-validation)**.++## Interpret the answers++<a href="../../rust/switchyard_protocol/decision/struct.DecisionResponse.html#structfield.answers"><code>DecisionResponse::answers</code></a> maps the same question IDs to <a href="../../rust/switchyard_protocol/decision/struct.DecisionAnswer.html"><code>DecisionAnswer</code></a> values.+Each answer separates its typed <a href="../../rust/switchyard_protocol/decision/struct.DecisionAnswer.html#structfield.value"><code>value</code></a> from optional <a href="../../rust/switchyard_protocol/decision/struct.DecisionAnswer.html#structfield.provider_confidence"><code>provider_confidence</code></a>.+For the privacy question, <a href="../../rust/switchyard_protocol/decision/enum.BooleanEstimate.html#variant.ProbabilityTrue"><code>ProbabilityTrue(Probability(0.8))</code></a> means an estimated+0.8 probability that the answer is true. It is not a routing choice until the+algorithm applies its policy. Choice questions can name any supported set of+candidates; their shape is not limited to two models.++The **[answer value reference](../reference/protocols/decision_answers.md#answer-values)**+explains rubric scores, distributions, and the difference between a Boolean+judgment and an answer probability.++## Understand conversion++**For supported questions and answers, conversion is a direct, lossless mapping+of decision data.** IDs, answer probabilities, rubric positions, and confidence+retain their meanings. Structured values become JSON text where OpenAI requires+strings. This retains their content, rather than making the two wire formats+identical or providing an arbitrary JSON round-trip guarantee.++Both adapters return probabilities for Boolean answers. A custom backend that+returns only true or false can use <a href="../../rust/switchyard_protocol/decision/enum.BooleanEstimate.html#variant.Value"><code>BooleanEstimate::Value</code></a> without inventing+certainty. Optional distributions remain `None` when absent. The OpenAI adapter+reports refusals as errors because <a href="../../rust/switchyard_protocol/decision/enum.DecisionValue.html"><code>DecisionValue</code></a> has no refusal variant.++## Own the conversion++When handling <a href="../../rust/switchyard_libsy/enum.Step.html#variant.CallDecision"><code>Step::CallDecision</code></a>, use your existing client or a function to+convert <a href="../../rust/switchyard_libsy/struct.CallDecision.html#structfield.request"><code>call.request</code></a> into the backend's format, send it, and convert the answer+into <a href="../../rust/switchyard_protocol/decision/struct.DecisionResponse.html"><code>DecisionResponse</code></a>. Pass that work to <a href="../../rust/switchyard_libsy/struct.CallDecision.html#method.respond"><code>call.respond</code></a> as a future returning+<a href="../../rust/switchyard_libsy/type.Result.html"><code>libsy::Result<DecisionResponse></code></a>.++Match question IDs and kinds, and retain available distributions and confidence+separately. The+[question reference](../reference/protocols/decision_questions.md#provider-adapters)+links the System One and OpenAI adapter implementations. Decision conversions live in+<a href="../../rust/switchyard_llm_client/"><code>switchyard-llm-client</code></a>; <a href="../../rust/switchyard_translation/"><code>switchyard-translation</code></a> handles LLM formats.++<a href="../../rust/switchyard_protocol/client/trait.RoutedDecisionClient.html"><code>RoutedDecisionClient</code></a>+is an optional interface that lets Switchyard's clients and execution helpers+work together. Direct <a href="../../rust/switchyard_libsy/trait.Algorithm.html#method.run_stream"><code>run_stream</code></a> handling does not require implementing it.+See [Mechanics of Libsy](../mechanics_of_libsy.md#stepcalldecision) for the+call-handling loop and cancellation behavior.++## Use the supplied clients++You can instead use the public <a href="../../rust/switchyard_llm_client/struct.SystemOneClient.html"><code>SystemOneClient</code></a> or <a href="../../rust/switchyard_llm_client/struct.OpenAiDecisionClient.html"><code>OpenAiDecisionClient</code></a> from+[`switchyard-llm-client`](../../crates/libsy-llm-client/README.md). Use dependencies at+the same API revision as libsy and the protocol crate.++This async excerpt sends the earlier `request` to System One. Set its `model` to+an ID served by your endpoint. `SYSTEM_ONE_ENDPOINT` must include `/v1/systemone`.+Running it makes a provider call and may incur a charge.+The trait import makes the supplied client's <a href="../../rust/switchyard_protocol/client/trait.RoutedDecisionClient.html#tymethod.call"><code>call</code></a> method available; you do not+implement a client in this example.++```rust+use std::time::Duration;+use switchyard_llm_client::SystemOneClient;+use switchyard_protocol::RoutedDecisionClient;++let client = SystemOneClient::new(+ std::env::var("SYSTEM_ONE_ENDPOINT")?.parse()?,+ std::env::var("SYSTEM_ONE_API_KEY")?,+ Duration::from_secs(30),+)?;+let response = client.call(request).await?;+```++<a href="../../rust/switchyard_llm_client/struct.OpenAiDecisionClient.html#method.new"><code>OpenAiDecisionClient::new</code></a> takes the same arguments, with an endpoint ending in+`/v1/decisions` and that provider's key. Each adapter makes one attempt per call;+its timeout covers the request and reading the response body.++For the built-in execution path, register decision clients by model ID with+<a href="../../rust/switchyard_llm_client/run/struct.ClientRouter.html#method.single_with_decision_clients"><code>ClientRouter::single_with_decision_clients</code></a> or <a href="../../rust/switchyard_llm_client/run/struct.ClientRouter.html#method.new_with_decision_clients"><code>new_with_decision_clients</code></a>, then+use <a href="../../rust/switchyard_llm_client/run/fn.run.html"><code>switchyard_llm_client::run</code></a>. The algorithm still receives the same typed+<a href="../../rust/switchyard_protocol/decision/struct.DecisionResponse.html"><code>DecisionResponse</code></a> whichever client performs the call.diff --git a/docs/protocols/index.md b/docs/protocols/index.mdnew file mode 100644index 000000000..d685f48bc--- /dev/null+++ b/docs/protocols/index.md@@ -0,0 +1,67 @@+# Switchyard Protocols++[Mechanics of Libsy](../mechanics_of_libsy.md) follows the work exchanged through+<a href="../rust/switchyard_libsy/trait.Algorithm.html#method.run_stream"><code>run_stream</code></a>. This section explains the data inside that work: an LLM conversation+or a set of typed questions for a decision model.++Both use **provider-neutral Rust types** from the+<a href="../rust/switchyard_protocol/"><code>switchyard-protocol</code> crate</a>. These types+define Switchyard's protocols. Your application converts its inputs into these+types, passes them to libsy, and converts model calls and responses at its+provider boundary.++## Why use a neutral protocol?++Provider APIs describe the same interactions in different ways. An Anthropic+tool call is a `tool_use` content block. OpenAI Responses uses a `function_call`+item. Chat Completions puts it in an assistant message's `tool_calls` array.++An algorithm needs to know that the model called a tool, its name, and its+arguments. A shared <a href="../rust/switchyard_protocol/llm/struct.ToolCall.html"><code>ToolCall</code></a> gives it those facts without tying the algorithm+to one provider's JSON. Your gateway or agent harness keeps control of the client,+credentials, and execution.++```text+Application or provider data+ |+ convert+ v+Switchyard protocol types <--> libsy algorithm+ |+ convert+ v+Your model client and provider API+```++The protocol crate defines the data exchanged with libsy. Conversion and network+calls happen in your application or in optional Switchyard crates. When handling+<a href="../rust/switchyard_libsy/trait.Algorithm.html#method.run_stream"><code>run_stream</code></a>, your application can use any client and return the protocol response+through <a href="../rust/switchyard_libsy/struct.CallModel.html#method.respond"><code>CallModel::respond</code></a> or+<a href="../rust/switchyard_libsy/struct.CallDecision.html#method.respond"><code>CallDecision::respond</code></a>. The crate also provides optional client traits that help+Switchyard's supplied clients and execution helpers work together.++## Two kinds of model interaction++| Interaction | Protocol fields and types | Used when | Read next |+|---|---|---|---|+| LLM conversation | <a href="../rust/switchyard_protocol/envelope/struct.Request.html#structfield.llm_request"><code>Request.llm_request: LlmRequest</code></a>; <a href="../rust/switchyard_protocol/envelope/struct.Response.html#structfield.llm_response"><code>Response.llm_response: LlmResponse</code></a> | Your application starts a run or serves <a href="../rust/switchyard_libsy/enum.Step.html#variant.CallModel"><code>Step::CallModel</code></a>. Content can include text, tools, reasoning, and media. | [LLM-Related Protocols](llm.md) |+| Typed decision | <a href="../rust/switchyard_protocol/decision/struct.DecisionRequest.html"><code>CallDecision.request: DecisionRequest</code></a>; the work future returns <a href="../rust/switchyard_protocol/decision/struct.DecisionResponse.html"><code>libsy::Result<DecisionResponse></code></a> | Your application serves <a href="../rust/switchyard_libsy/enum.Step.html#variant.CallDecision"><code>Step::CallDecision</code></a>. Named questions ask for a Boolean estimate, a choice, or a rubric score. | [Decision Model-Related Protocols](decisions.md) |++The LLM envelopes also carry <a href="../rust/switchyard_protocol/metadata/struct.Metadata.html"><code>metadata: Option<Metadata></code></a>. That application+metadata is separate from the provider's conversation content. Decision calls+use their own request and response types directly.++## Choose who owns conversion++| Integration choice | Use it for |+|---|---|+| Your own adapters | Convert an existing SDK's types or an in-process model's data to and from <a href="../rust/switchyard_protocol/"><code>switchyard-protocol</code></a>. Keep your existing client and transport. |+| <a href="../rust/switchyard_translation/"><code>switchyard-translation</code></a> | Decode and encode supported LLM JSON and streaming events while keeping your own transport. |+| <a href="../rust/switchyard_llm_client/"><code>switchyard-llm-client</code></a> | Use Switchyard's HTTP clients, including LLM translation and typed decision adapters. |++The two guide pages explain these boundaries and the integration flow. For+field-level details, use the **[LLM request reference](../reference/protocols/llm_requests.md)**+or **[decision question reference](../reference/protocols/decision_questions.md)**.+Each reference page links back to its guide and to the next part of the exchange.+For complete type and method definitions, open the+<a href="../rust/switchyard_protocol/">protocol crate reference</a>.diff --git a/docs/protocols/llm.md b/docs/protocols/llm.mdnew file mode 100644index 000000000..1008ff8dc--- /dev/null+++ b/docs/protocols/llm.md@@ -0,0 +1,152 @@+# LLM-Related Protocols++An LLM request describes a conversation, the tools available to the model, and+generation settings. Switchyard represents these in+<a href="../../rust/switchyard_protocol/llm/struct.LlmRequest.html"><code>LlmRequest</code></a>.+The same type works whether the next call goes to Anthropic Messages, OpenAI+Responses, or Chat Completions.++!!! info "Fields, API mappings, and conversion rules"++ - **[Request reference](../reference/protocols/llm_requests.md):** messages, tools, generation settings, and their provider API fields.+ - **[Response and stream reference](../reference/protocols/llm_responses.md):** output content, token usage, event types, and stream lifetime.+ - **[Translation reference](../reference/protocols/translation.md):** engine methods, preservation policies, diagnostics, and edits before encoding.++## Read a request++<a href="../../rust/switchyard_protocol/llm/struct.LlmRequest.html"><code>LlmRequest</code></a>+separates instructions from conversation turns. A+<a href="../../rust/switchyard_protocol/llm/struct.Message.html"><code>Message</code></a>+contains a role and ordered+<a href="../../rust/switchyard_protocol/llm/enum.ContentBlock.html"><code>ContentBlock</code></a>+values. Content can include text, media, reasoning, tool calls, and tool results.++This example constructs the neutral request passed to an algorithm. It+uses the [quickstart's dependencies](../getting_started.md#install).++```rust+use switchyard_protocol::{ContentBlock, InstructionBlock, LlmRequest, Message, Request, Role};++let request = Request {+ llm_request: LlmRequest {+ instructions: vec![InstructionBlock {+ role: Role::System,+ content: vec![ContentBlock::Text { text: "Answer briefly.".into() }],+ }],+ messages: vec![Message::text(Role::User, "Explain tail latency.")],+ ..LlmRequest::default()+ },+ ..Request::default()+};+```++<a href="../../rust/switchyard_protocol/envelope/struct.Request.html"><code>Request</code></a> is the host-facing envelope. It adds optional correlation metadata+and <a href="../../rust/switchyard_protocol/envelope/struct.Request.html#structfield.raw_request"><code>raw_request</code></a>, a host-owned JSON body. **Codecs use+<a href="../../rust/switchyard_protocol/llm/struct.LlmRequest.html#structfield.preservation"><code>LlmRequest::preservation</code></a> for replay**, not <a href="../../rust/switchyard_protocol/envelope/struct.Request.html#structfield.raw_request"><code>Request::raw_request</code></a>.++## Read a response++The host returns a <a href="../../rust/switchyard_protocol/envelope/struct.Response.html"><code>Response</code></a> envelope containing+<a href="../../rust/switchyard_protocol/stream/enum.LlmResponse.html"><code>LlmResponse</code></a>.+<a href="../../rust/switchyard_protocol/stream/enum.LlmResponse.html#variant.Agg"><code>LlmResponse::Agg</code></a> holds a completed <a href="../../rust/switchyard_protocol/llm/struct.AggLlmResponse.html"><code>AggLlmResponse</code></a>. <a href="../../rust/switchyard_protocol/stream/enum.LlmResponse.html#variant.Stream"><code>LlmResponse::Stream</code></a>+holds a single-consumption stream of+<a href="../../rust/switchyard_protocol/stream/struct.LlmResponseStreamEvent.html"><code>LlmResponseStreamEvent</code></a>+values.+Each stream event exposes neutral chunks through <a href="../../rust/switchyard_protocol/stream/struct.LlmResponseStreamEvent.html#method.normalized"><code>normalized()</code></a>. Chunks describe+message starts, text deltas, tool arguments, usage, and completion. An event may+also retain parsed provider JSON for same-format replay.++<a href="../../rust/switchyard_protocol/llm/struct.Usage.html#structfield.input_tokens"><code>Usage::input_tokens</code></a> counts **non-cached input**. The codecs separate cache-read+and cache-write tokens from OpenAI's aggregate input counts. Missing counts stay+`None`. Call <a href="../../rust/switchyard_protocol/stream/enum.LlmResponse.html#method.into_agg"><code>LlmResponse::into_agg().await</code></a> when you need a buffered result; this+consumes the stream and aggregates its normalized chunks.++## Understand lossless conversion++**The protocol retains source data for lossless same-format round trips.** With+the default preservation policy, decoding a supported API body and encoding it+back to that API reuses its preserved JSON. The+[preservation reference](../reference/protocols/translation.md#preservation-and-edits)+explains exactly what is retained and when replay takes precedence.++**Conversion between different provider APIs can be lossy.** For example,+Anthropic has no separate developer role, and Chat Completions has no native+equivalent for every Responses built-in tool. Keeping the original data in the+protocol does not give the destination API those capabilities.++Translation reports known losses in <a href="../../rust/switchyard_translation/engine/struct.TranslationOutput.html#structfield.diagnostics"><code>diagnostics</code></a>.+The [translation policy reference](../reference/protocols/translation.md#policies)+explains how to allow or reject these conversions.++!!! warning "Encoding an edited request"++ A preserved same-format body takes precedence over normalized fields. If you+ change the protocol fields and want those edits encoded, remove that format's+ entry from <a href="../../rust/switchyard_protocol/llm/struct.PreservationMetadata.html#structfield.requests"><code>preservation.requests</code></a>, or encode with <a href="../../rust/switchyard_translation/policy/enum.PreservationPolicy.html#variant.Disabled"><code>PreservationPolicy::Disabled</code></a>.+ The same rule applies to <a href="../../rust/switchyard_protocol/llm/struct.PreservationMetadata.html#structfield.responses"><code>preservation.responses</code></a> for buffered responses.++See [stream preservation and aggregation](../reference/protocols/llm_responses.md#stream-lifetime-and-preservation)+for how replay and edits work on streamed responses.++## Use the translation crate++[`switchyard-translation`](../../crates/switchyard-translation/README.md) supplies+the LLM codecs without making network calls. Add it at the same API revision as+libsy and <a href="../../rust/switchyard_protocol/"><code>switchyard-protocol</code></a>, following the+[quickstart's translation setup](../getting_started.md#use-libsy-with-your-client).++This excerpt decodes an inbound Chat Completions body and encodes it as an+OpenAI Responses request. It can run inside your adapter; <a href="../../rust/switchyard_translation/engine/struct.TranslationOutput.html#structfield.body"><code>outbound.body</code></a> is the+JSON your client sends.++```rust+use serde_json::json;+use switchyard_translation::{TranslationEngine, TranslationPolicy, WireFormat};++let engine = TranslationEngine::default();+let policy = TranslationPolicy::default();+let inbound = json!({+ "model": "your-model",+ "messages": [{"role": "user", "content": "Explain tail latency."}]+});++let decoded = engine.decode_request(WireFormat::OpenAiChat, &inbound, &policy)?;+let outbound = engine.encode_request(+ WireFormat::OpenAiResponses,+ &decoded.request,+ &policy,+)?;+// Check diagnostics from both operations before sending the encoded body.+```++During a libsy run, place <a href="../../rust/switchyard_translation/engine/struct.RequestIrOutput.html#structfield.request"><code>decoded.request</code></a> in <a href="../../rust/switchyard_protocol/envelope/struct.Request.html#structfield.llm_request"><code>Request::llm_request</code></a>. Encode+the request from <a href="../../rust/switchyard_libsy/enum.Step.html#variant.CallModel"><code>Step::CallModel</code></a> for the selected backend. Decode its reply+with <a href="../../rust/switchyard_translation/engine/struct.TranslationEngine.html#method.decode_response"><code>decode_response</code></a>, or use <a href="../../rust/switchyard_translation/engine/struct.TranslationEngine.html#method.decode_stream_event"><code>decode_stream_event</code></a> and <a href="../../rust/switchyard_translation/engine/struct.TranslationEngine.html#method.encode_stream_event"><code>encode_stream_event</code></a>+with a separate <a href="../../rust/switchyard_translation/codecs/stream/struct.StreamTranslationState.html"><code>StreamTranslationState</code></a> per response stream. Return the neutral+<a href="../../rust/switchyard_protocol/envelope/struct.Response.html"><code>Response</code></a> through the call object, as shown in+[Mechanics of Libsy](../mechanics_of_libsy.md#stepcallmodel).++The engine also exposes <a href="../../rust/switchyard_translation/engine/struct.TranslationEngine.html#method.translate_request"><code>translate_request</code></a> and <a href="../../rust/switchyard_translation/engine/struct.TranslationEngine.html#method.translate_response"><code>translate_response</code></a> for direct+API-to-API conversion. See the+<a href="../../rust/switchyard_translation/engine/struct.TranslationEngine.html">translation engine reference</a>+for signatures and results.++## Keep your own client or conversions++You can construct these protocol types from your existing SDK and convert them+back yourself. When handling <a href="../../rust/switchyard_libsy/enum.Step.html#variant.CallModel"><code>Step::CallModel</code></a>, pass <a href="../../rust/switchyard_libsy/struct.CallModel.html#method.respond"><code>call.respond</code></a> a future that+uses your client and returns <a href="../../rust/switchyard_libsy/type.Result.html"><code>libsy::Result<Response></code></a>. Keep content order, tool+IDs, provider-only data, and streaming lifetime intact where your integration+needs them.++<a href="../../rust/switchyard_protocol/client/trait.RoutedLlmClient.html"><code>RoutedLlmClient</code></a>+is an optional shared interface used by Switchyard's clients and execution+helpers. Direct <a href="../../rust/switchyard_libsy/trait.Algorithm.html#method.run_stream"><code>run_stream</code></a> handling works with your own functions or clients;+it does not require implementing that trait.++If you want Switchyard to handle HTTP and translation, use+<a href="../../rust/switchyard_llm_client/client/struct.TranslatingLlmClient.html"><code>TranslatingLlmClient</code></a>+from <a href="../../rust/switchyard_llm_client/">the HTTP client crate</a>.+The [quickstart](../getting_started.md#use-libsy-with-your-client) shows how to+keep your own HTTP client. Typed decisions use the separate contract described+in [Decision Model-Related Protocols](decisions.md).diff --git a/docs/reference/protocols/decision_answers.md b/docs/reference/protocols/decision_answers.mdnew file mode 100644index 000000000..e04b7b3cf--- /dev/null+++ b/docs/reference/protocols/decision_answers.md@@ -0,0 +1,67 @@+# Decision Answers++For the integration flow, read **[Decision Model-Related Protocols](../../protocols/decisions.md)**.+For question IDs, options, and rubric definitions, use **[Decision Questions](decision_questions.md)**.++## Response fields++<a href="../../../rust/switchyard_protocol/decision/struct.DecisionResponse.html#structfield.answers"><code>DecisionResponse.answers</code></a> maps the request's question IDs to <a href="../../../rust/switchyard_protocol/decision/struct.DecisionAnswer.html"><code>DecisionAnswer</code></a>+values. Each answer separates its value from optional provider confidence.++| Protocol field and type | Meaning | System One | OpenAI Decisions |+|---|---|---|---|+| <a href="../../../rust/switchyard_protocol/decision/struct.DecisionResponse.html#structfield.id"><code>DecisionResponse.id: Option<String></code></a> | Provider response identifier, when available. | Optional `id` | The current adapter reads no response ID and returns none |+| <a href="../../../rust/switchyard_protocol/decision/struct.DecisionResponse.html#structfield.model"><code>DecisionResponse.model: Option<ModelId></code></a> | Provider-reported model identifier. | Optional `model` | `model` |+| <a href="../../../rust/switchyard_protocol/decision/struct.DecisionResponse.html#structfield.answers"><code>DecisionResponse.answers: BTreeMap<String, DecisionAnswer></code></a> | Answers matched by question ID. | `answers` object | `answers[]`, identified by `name` |+| <a href="../../../rust/switchyard_protocol/decision/struct.DecisionAnswer.html#structfield.value"><code>DecisionAnswer.value: DecisionValue</code></a> | Typed estimate matching the question's kind. | `noul`, `choice`, or `score` | `probability`, `choice`, or `score` |+| <a href="../../../rust/switchyard_protocol/decision/struct.DecisionAnswer.html#structfield.provider_confidence"><code>DecisionAnswer.provider_confidence: Option<ProviderConfidence></code></a> | Provider-specific confidence with its own scale and meaning. | Optional `confidence` | Optional `confidence` |+| <a href="../../../rust/switchyard_protocol/decision/struct.DecisionResponse.html#structfield.usage"><code>DecisionResponse.usage: Usage</code></a> | Available token counts; absent counts remain unknown. | Deserialized into <a href="../../../rust/switchyard_protocol/llm/struct.Usage.html"><code>Usage</code></a> | Counts normalized into <a href="../../../rust/switchyard_protocol/llm/struct.Usage.html"><code>Usage</code></a>, separating cache reads and writes from aggregate input |++## Answer values++| Variant and fields | Contract | System One | OpenAI Decisions |+|---|---|---|---|+| <a href="../../../rust/switchyard_protocol/decision/enum.BooleanEstimate.html#variant.Value"><code>BooleanEstimate::Value(bool)</code></a> | A Boolean-only judgment with no implied certainty. | The supplied adapter returns a probability instead | The supplied adapter returns a probability instead |+| <a href="../../../rust/switchyard_protocol/decision/enum.BooleanEstimate.html#variant.ProbabilityTrue"><code>BooleanEstimate::ProbabilityTrue(Probability)</code></a> | Probability of true on a `[0, 1]` scale. The algorithm chooses any threshold. | `type: "noul"`, field `noul` | `type: "predicate"`, field `probability` |+| <a href="../../../rust/switchyard_protocol/decision/enum.DecisionValue.html#variant.Choice"><code>DecisionValue::Choice</code></a> with `selected: String` and <a href="../../../rust/switchyard_protocol/decision/struct.Probability.html"><code>probabilities: Option<BTreeMap<String, Probability>></code></a> | Select a declared ID. When supplied, the distribution covers every declared option. | `choice`; optional ID-keyed `probabilities` | `choice`; `probabilities[]` with option `value` and `probability`, converted to a map |+| <a href="../../../rust/switchyard_protocol/decision/enum.DecisionValue.html#variant.Score"><code>DecisionValue::Score</code></a> with <a href="../../../rust/switchyard_protocol/decision/struct.ScoreValue.html"><code>value: ScoreValue</code></a> and <a href="../../../rust/switchyard_protocol/decision/struct.Probability.html"><code>probabilities: Option<Vec<Probability>></code></a> | Position in `0..=N-1`; the optional distribution follows the question's level order. | `score`; optional `probabilities` keyed by numeric strings, reordered by level | `score`; numeric `value` / `probability` entries, reordered by level |++<a href="../../../rust/switchyard_protocol/decision/enum.DecisionValue.html#variant.Boolean"><code>DecisionValue::Boolean</code></a> wraps a <a href="../../../rust/switchyard_protocol/decision/enum.BooleanEstimate.html"><code>BooleanEstimate</code></a>. An absent distribution stays+unknown; it does not become a one-hot distribution. Likewise, a Boolean-only+answer does not imply probability zero or one.++<a href="../../../rust/switchyard_protocol/decision/struct.Probability.html"><code>Probability</code></a>, <a href="../../../rust/switchyard_protocol/decision/struct.ScoreValue.html"><code>ScoreValue</code></a>, and <a href="../../../rust/switchyard_protocol/decision/struct.ProviderConfidence.html"><code>ProviderConfidence</code></a> are public numeric wrappers.+Their stated ranges and meanings are contracts, not validation performed by+their constructors. Keep answer probability separate from provider confidence.++## Matching and errors++Retain the request when converting a response. Question IDs identify the expected+kind; choice IDs identify valid selections; rubric order gives scores meaning.+Both clients match responses against the request and perform format-specific+checks. An unexpected ID, kind, or malformed supported response can produce+<a href="../../../rust/switchyard_protocol/client/enum.LlmClientError.html#variant.ResponseTranslation"><code>LlmClientError::ResponseTranslation</code></a>.++The OpenAI adapter treats a refusal as an error because <a href="../../../rust/switchyard_protocol/decision/enum.DecisionValue.html"><code>DecisionValue</code></a> has no+refusal variant. Transport failures and upstream HTTP errors also surface through+<a href="../../../rust/switchyard_protocol/client/enum.LlmClientError.html"><code>LlmClientError</code></a>. A host converts its work result to <a href="../../../rust/switchyard_libsy/type.Result.html"><code>libsy::Result<DecisionResponse></code></a>+when replying through <a href="../../../rust/switchyard_libsy/struct.CallDecision.html#method.respond"><code>CallDecision::respond</code></a>.++These clients do not retry automatically. Their constructors accept a timeout+covering the HTTP request and reading the body. See the+**[question reference's adapter links](decision_questions.md#provider-adapters)**+for the exact conversion and error paths.++## Conversion guarantees++Supported decision fields map directly while retaining question IDs, answer+probabilities, rubric positions, and confidence. OpenAI text-only fields carry+structured protocol values as JSON text. This preserves content, rather than+making the wire formats identical or guaranteeing arbitrary JSON round trips.++The provider must actually report a field for it to be available. Keep missing+IDs, confidence, distributions, and usage unknown. For cache and reasoning token+semantics, see **[LLM usage](llm_responses.md#usage)**.++Return to **[the decision protocol guide](../../protocols/decisions.md#own-the-conversion)**+to connect the response to your application's execution loop.diff --git a/docs/reference/protocols/decision_questions.md b/docs/reference/protocols/decision_questions.mdnew file mode 100644index 000000000..145726bc5--- /dev/null+++ b/docs/reference/protocols/decision_questions.md@@ -0,0 +1,64 @@+# Decision Questions++For the integration flow and a request example, read **[Decision Model-Related Protocols](../../protocols/decisions.md)**.+For result interpretation and provider response mappings, continue to+**[Decision Answers](decision_answers.md)**.++System One is a common API for open decision models, including Jev.+<a href="../../../rust/switchyard_llm_client/struct.SystemOneClient.html"><code>SystemOneClient</code></a> uses `/v1/systemone`; <a href="../../../rust/switchyard_llm_client/struct.OpenAiDecisionClient.html"><code>OpenAiDecisionClient</code></a> uses+[`/v1/decisions`](https://developers.openai.com/api/reference/resources/decisions/methods/create).+Both accept <a href="../../../rust/switchyard_protocol/decision/struct.DecisionRequest.html"><code>DecisionRequest</code></a>.++## Request fields++| Protocol field and type | Meaning | System One | OpenAI Decisions |+|---|---|---|---|+| <a href="../../../rust/switchyard_protocol/decision/struct.DecisionRequest.html#structfield.model"><code>DecisionRequest.model: Option<ModelId></code></a> | Selected decision model. The HTTP adapters require it before sending. | `model` | `model` |+| <a href="../../../rust/switchyard_protocol/decision/struct.DecisionRequest.html#structfield.context"><code>DecisionRequest.context: Value</code></a> | Shared conversation, application state, or other material to evaluate. | JSON-valued `state` | Text-valued `input`; structured values are serialized as JSON text |+| <a href="../../../rust/switchyard_protocol/decision/struct.DecisionRequest.html#structfield.questions"><code>DecisionRequest.questions: BTreeMap<String, DecisionQuestion></code></a> | Independent questions keyed by stable IDs. | `questions` object, keyed by ID | `questions[]`, with each ID in `name` |+| <a href="../../../rust/switchyard_protocol/decision/struct.DecisionQuestion.html#structfield.instructions"><code>DecisionQuestion.instructions: Value</code></a> | Instructions for one question. | JSON-valued `instructions` | Text-valued `instructions`; structured values become JSON text |+| <a href="../../../rust/switchyard_protocol/decision/struct.DecisionQuestion.html#structfield.kind"><code>DecisionQuestion.kind: DecisionKind</code></a> | Expected Boolean, choice, or rubric-score answer. | Question `type` and `criteria` | Question `type`, with `choices` or `levels` where applicable |++Question IDs identify the answers. Preserve them through conversion; an array+position alone is not a replacement for an ID. Context is shared across the+request, while each question has its own instructions.++## Question kinds++| Kind and fields | Contract | System One | OpenAI Decisions |+|---|---|---|---|+| <a href="../../../rust/switchyard_protocol/decision/enum.DecisionKind.html#variant.Boolean"><code>DecisionKind::Boolean</code></a> with `true_description: Option<Value>` and `false_description: Option<Value>` | Ask for a Boolean judgment or probability of true. Descriptions define the two answers when needed. | `type: "noul"`; descriptions in `criteria.true` and `criteria.false` | `type: "predicate"`; descriptions are included in instruction text because there are no separate fields |+| <a href="../../../rust/switchyard_protocol/decision/enum.DecisionKind.html#variant.Choice"><code>DecisionKind::Choice</code></a> with <a href="../../../rust/switchyard_protocol/decision/struct.ChoiceOption.html"><code>options: Vec<ChoiceOption></code></a> | Declare a nonempty list of options with unique IDs, preserving their order in the protocol. | `type: "choice"`; `criteria` maps option IDs to descriptions | `type: "choice"`; `choices[]` carries `value` and optional `description` |+| <a href="../../../rust/switchyard_protocol/decision/enum.DecisionKind.html#variant.Score"><code>DecisionKind::Score</code></a> with `levels: Vec<Value>` | Declare at least two levels, ordered low to high and indexed from zero. | `type: "score"`; `criteria` is the level array | `type: "score"`; `levels[].label` carries each level's text |+| <a href="../../../rust/switchyard_protocol/decision/struct.ChoiceOption.html#structfield.id"><code>ChoiceOption.id: String</code></a> | Stable ID used by the selected answer and any distribution. | Key in `criteria` | `choices[].value` |+| <a href="../../../rust/switchyard_protocol/decision/struct.ChoiceOption.html#structfield.description"><code>ChoiceOption.description: Option<Value></code></a> | Optional meaning when the ID alone is insufficient. | JSON-valued description; absent description becomes null | Text description when supplied; structured values become JSON text |++Choice questions can contain more than two options. A score is a position on+the declared rubric, including fractional positions; it is not an arbitrary+numeric measurement. Keep the question's levels when interpreting the answer.++## Construction and validation++The fields are public. Constructing or deserializing a <a href="../../../rust/switchyard_protocol/decision/struct.DecisionRequest.html"><code>DecisionRequest</code></a> does+not enforce all question contracts. Supply a selected model for HTTP calls,+valid option IDs, and a correctly ordered rubric. The adapters apply their+format-specific checks; they are not a general validation layer for every+possible backend.++Enums in the protocol's own serialized form use snake-case `type` tags with a+`data` payload. Provider payloads use the mappings above. Serializing the protocol+value directly is therefore not a substitute for a provider adapter.++## Provider adapters++The [System One implementation](../../../crates/libsy-llm-client/src/system_one.rs)+and [OpenAI Decisions implementation](../../../crates/libsy-llm-client/src/openai_decisions.rs)+are reference examples of request encoding and response conversion. Both supplied+clients make one attempt per call, with a timeout covering the request and reading+the body.++You can use either supplied client, or convert an existing client's data yourself.+<a href="../../../rust/switchyard_protocol/client/trait.RoutedDecisionClient.html"><code>RoutedDecisionClient</code></a> is an optional shared client interface. The+**[guide's direct call handling](../../protocols/decisions.md#own-the-conversion)**+shows the application boundary; **[Decision Answers](decision_answers.md)** covers+the response contract.diff --git a/docs/reference/protocols/llm_requests.md b/docs/reference/protocols/llm_requests.mdnew file mode 100644index 000000000..9c552e755--- /dev/null+++ b/docs/reference/protocols/llm_requests.md@@ -0,0 +1,89 @@+# LLM Requests++This reference covers request fields and the built-in provider mappings. For the+integration flow and a construction example, read **[LLM-Related Protocols](../../protocols/llm.md)**.+For the reply, continue to **[Responses and Streams](llm_responses.md)**.++## Request envelope++An algorithm receives <a href="../../../rust/switchyard_protocol/envelope/struct.Request.html"><code>Request</code></a>. Its `llm_request` contains the conversation;+the remaining envelope data belongs to the application.++| Field and type | Meaning |+|---|---|+| <a href="../../../rust/switchyard_protocol/envelope/struct.Request.html#structfield.llm_request"><code>Request.llm_request: LlmRequest</code></a> | The conversation and settings used by the algorithm and model client. |+| <a href="../../../rust/switchyard_protocol/envelope/struct.Request.html#structfield.metadata"><code>Request.metadata: Option<Metadata></code></a> | Optional session, correlation, and routing metadata. |+| <a href="../../../rust/switchyard_protocol/envelope/struct.Request.html#structfield.raw_request"><code>Request.raw_request: Option<Value></code></a> | An optional original body retained by the host. Translation codecs do not read this field. |++Codec replay uses <a href="../../../rust/switchyard_protocol/llm/struct.LlmRequest.html#structfield.preservation"><code>LlmRequest.preservation</code></a>, separately from the envelope's raw+body. See **[preservation and edits](translation.md#preservation-and-edits)**+before changing a decoded request.++## Conversation fields++These mappings describe the built-in codecs. Supported features also depend on+the destination model. Select a Rust field or type for its full crate definition.++| Protocol field and type | Meaning | Anthropic Messages | OpenAI Responses | Chat Completions |+|---|---|---|---|---|+| <a href="../../../rust/switchyard_protocol/llm/struct.LlmRequest.html#structfield.model"><code>LlmRequest.model: Option<String></code></a> | Addressed model; the host can replace it with the selected target. | `model` | `model` | `model` |+| <a href="../../../rust/switchyard_protocol/llm/struct.LlmRequest.html#structfield.instructions"><code>LlmRequest.instructions: Vec<InstructionBlock></code></a> | System and developer instructions. | Top-level `system`; no distinct developer authority | `instructions` and system/developer messages in `input` | System/developer messages in `messages` |+| <a href="../../../rust/switchyard_protocol/llm/struct.LlmRequest.html#structfield.messages"><code>LlmRequest.messages: Vec<Message></code></a> | Ordered turns, each with a role and content blocks. | `messages[]` | Messages and tool items in `input` | `messages[]` |+| <a href="../../../rust/switchyard_protocol/llm/struct.LlmRequest.html#structfield.tools"><code>LlmRequest.tools: Vec<Tool></code></a> | Available functions, MCP toolsets, or retained provider definitions. | `tools[]` | `tools[]` | `tools[]`; function definitions are nested in `function` |+| <a href="../../../rust/switchyard_protocol/llm/struct.LlmRequest.html#structfield.tool_choice"><code>LlmRequest.tool_choice: Option<ToolChoice></code></a> | Let the model choose, require a call, forbid calls, or select a named tool. | `auto`, `any`, `none`, or `tool` | `auto`, `required`, `none`, or `function` | `auto`, `required`, `none`, or `function` |+| <a href="../../../rust/switchyard_protocol/llm/struct.LlmRequest.html#structfield.stream"><code>LlmRequest.stream: bool</code></a> | Request incremental output. | `stream` | `stream` | `stream` |+| <a href="../../../rust/switchyard_protocol/llm/struct.LlmRequest.html#structfield.extensions"><code>LlmRequest.extensions: ProviderExtensions</code></a> | Provider fields without a first-class neutral field. | Source fields retained by the codec | Source fields retained by the codec | Source fields retained by the codec |+| <a href="../../../rust/switchyard_protocol/llm/struct.LlmRequest.html#structfield.preservation"><code>LlmRequest.preservation: PreservationMetadata</code></a> | Original JSON used for same-format replay. | Original parsed body | Original parsed body | Original parsed body |++### Messages and tool content++<a href="../../../rust/switchyard_protocol/llm/struct.Message.html#structfield.role"><code>Message.role</code></a> is a <a href="../../../rust/switchyard_protocol/llm/enum.Role.html"><code>Role</code></a>; <a href="../../../rust/switchyard_protocol/llm/struct.Message.html#structfield.content"><code>Message.content</code></a> is an ordered <a href="../../../rust/switchyard_protocol/llm/enum.ContentBlock.html"><code>Vec<ContentBlock></code></a>.+The content variants cover text, reasoning, media, tools, MCP activity, refusals,+and unknown provider blocks. An unknown block retains its source format and raw+JSON. Its retention does not make it usable in a different API.++| Protocol field and type | Meaning | Anthropic Messages | OpenAI Responses | Chat Completions |+|---|---|---|---|---|+| <a href="../../../rust/switchyard_protocol/llm/struct.ToolDefinition.html#structfield.parameters"><code>ToolDefinition.parameters: Value</code></a> | JSON Schema for function arguments. | `input_schema` | Function `parameters` | `tools[].function.parameters` |+| <a href="../../../rust/switchyard_protocol/llm/struct.ToolCall.html#structfield.id"><code>ToolCall.id: String</code></a> | Identifier that pairs a call with its result. | `tool_use.id` | `function_call.call_id` | `tool_calls[].id` |+| <a href="../../../rust/switchyard_protocol/llm/struct.ToolCall.html#structfield.name"><code>ToolCall.name: String</code></a> | Function name. | `tool_use.name` | `function_call.name` | `tool_calls[].function.name` |+| <a href="../../../rust/switchyard_protocol/llm/struct.ToolCall.html#structfield.arguments"><code>ToolCall.arguments: Value</code></a> | Parsed argument value. | JSON-valued `input` | `arguments`, commonly a JSON-encoded string | `function.arguments`, commonly a JSON-encoded string |+| <a href="../../../rust/switchyard_protocol/llm/struct.ToolResult.html#structfield.tool_call_id"><code>ToolResult.tool_call_id: String</code></a> | Identifier of the call being answered. | `tool_result.tool_use_id` | `function_call_output.call_id` | Tool message `tool_call_id` |+| <a href="../../../rust/switchyard_protocol/llm/struct.ToolResult.html#structfield.content"><code>ToolResult.content: Vec<ContentBlock></code></a> | Ordered tool output. | `tool_result.content` | `function_call_output.output` | Tool message `content` |+| <a href="../../../rust/switchyard_protocol/llm/struct.ToolResult.html#structfield.is_error"><code>ToolResult.is_error: Option<bool></code></a> | Whether tool execution failed, if reported. | `tool_result.is_error` | No dedicated error flag | No standard error flag |++Tool definitions describe available work. A <a href="../../../rust/switchyard_protocol/llm/struct.ToolCall.html"><code>ToolCall</code></a> records a model's request+to execute that work; it does not execute the tool. Keep call IDs and content+order intact when writing an adapter.++<a href="../../../rust/switchyard_protocol/llm/enum.Tool.html#variant.Provider"><code>Tool::Provider</code></a> retains an unsupported tool definition for its source format.+Built-in provider tools and reasoning signatures may have no valid counterpart+in another API. Consult the [translation limits](translation.md#conversion-limits)+instead of assuming a successful parse implies cross-provider support.++## Generation settings++The request groups these fields in <a href="../../../rust/switchyard_protocol/llm/struct.SamplingParams.html"><code>SamplingParams</code></a>, <a href="../../../rust/switchyard_protocol/llm/struct.OutputParams.html"><code>OutputParams</code></a>, and+<a href="../../../rust/switchyard_protocol/llm/struct.ReasoningParams.html"><code>ReasoningParams</code></a>. Optional values remain unspecified when absent; individual+providers apply their own defaults and accepted ranges.++| Protocol field and type | Meaning | Anthropic Messages | OpenAI Responses | Chat Completions |+|---|---|---|---|---|+| <a href="../../../rust/switchyard_protocol/llm/struct.SamplingParams.html#structfield.temperature"><code>SamplingParams.temperature: Option<f64></code></a> | Sampling temperature. | `temperature` | `temperature` | `temperature` |+| <a href="../../../rust/switchyard_protocol/llm/struct.SamplingParams.html#structfield.top_p"><code>SamplingParams.top_p: Option<f64></code></a> | Nucleus-sampling probability mass. | `top_p` | `top_p` | `top_p` |+| <a href="../../../rust/switchyard_protocol/llm/struct.SamplingParams.html#structfield.top_k"><code>SamplingParams.top_k: Option<i64></code></a> | Maximum candidate-token count. | `top_k` | No standard field | No standard field |+| <a href="../../../rust/switchyard_protocol/llm/struct.OutputParams.html#structfield.max_output_tokens"><code>OutputParams.max_output_tokens: Option<u64></code></a> | Output token budget. | `max_tokens` | `max_output_tokens` | `max_completion_tokens`, or legacy `max_tokens` |+| <a href="../../../rust/switchyard_protocol/llm/struct.OutputParams.html#structfield.response_format"><code>OutputParams.response_format: Option<Value></code></a> | Requested structured-output shape. | `output_config.format` | `text.format` | `response_format` |+| <a href="../../../rust/switchyard_protocol/llm/struct.OutputParams.html#structfield.is_schema_enforced"><code>OutputParams.is_schema_enforced: Option<bool></code></a> | Override whether the provider must enforce the schema; absence retains native behavior. | Schema-backed output configuration | Schema strictness in `text.format` | Schema strictness in `response_format` |+| <a href="../../../rust/switchyard_protocol/llm/struct.ReasoningParams.html#structfield.effort"><code>ReasoningParams.effort: Option<String></code></a> | Requested reasoning effort. | `output_config.effort` | `reasoning.effort` | `reasoning_effort` |+| <a href="../../../rust/switchyard_protocol/llm/struct.ReasoningParams.html#structfield.raw"><code>ReasoningParams.raw: Option<Value></code></a> | Provider-specific reasoning configuration. | `thinking` controls | `reasoning` object | No standard equivalent for the whole object |++Preserved same-format bodies take precedence over edits to these settings,+including schema enforcement. See **[encoding an edited request](translation.md#preservation-and-edits)**.++## Provider contracts++For complete provider schemas, see [Anthropic Messages](https://platform.claude.com/docs/en/api/messages/create),+[OpenAI Responses](https://developers.openai.com/api/reference/resources/responses/methods/create),+and [Chat Completions](https://developers.openai.com/api/reference/resources/chat/subresources/completions/methods/create).+For response fields and event ownership, continue to **[Responses and Streams](llm_responses.md)**.diff --git a/docs/reference/protocols/llm_responses.md b/docs/reference/protocols/llm_responses.mdnew file mode 100644index 000000000..10c13530d--- /dev/null+++ b/docs/reference/protocols/llm_responses.md@@ -0,0 +1,84 @@+# LLM Responses and Streams++For the integration flow, read **[LLM-Related Protocols](../../protocols/llm.md)**.+For the input side, use **[LLM Requests](llm_requests.md)**.++## Response forms++<a href="../../../rust/switchyard_protocol/envelope/struct.Response.html"><code>Response</code></a> is the host-facing envelope. Its `llm_response` is either+<a href="../../../rust/switchyard_protocol/stream/enum.LlmResponse.html#variant.Agg"><code>LlmResponse::Agg</code></a>, containing an <a href="../../../rust/switchyard_protocol/llm/struct.AggLlmResponse.html"><code>AggLlmResponse</code></a>, or <a href="../../../rust/switchyard_protocol/stream/enum.LlmResponse.html#variant.Stream"><code>LlmResponse::Stream</code></a>,+containing a live <a href="../../../rust/switchyard_protocol/stream/type.LlmResponseStream.html"><code>LlmResponseStream</code></a>. The stream has one consumer.++| Protocol field and type | Meaning | Anthropic Messages | OpenAI Responses | Chat Completions |+|---|---|---|---|---|+| <a href="../../../rust/switchyard_protocol/llm/struct.AggLlmResponse.html#structfield.id"><code>AggLlmResponse.id: Option<String></code></a> | Provider response identifier, if available. | `id` | `id` | `id` |+| <a href="../../../rust/switchyard_protocol/llm/struct.AggLlmResponse.html#structfield.model"><code>AggLlmResponse.model: Option<String></code></a> | Provider-reported model name. | `model` | `model` | `model` |+| <a href="../../../rust/switchyard_protocol/llm/struct.AggLlmResponse.html#structfield.outputs"><code>AggLlmResponse.outputs: Vec<ResponseOutput></code></a> | Completed output items. | Assistant `content` blocks | Items in `output` | `choices[].message` |+| <a href="../../../rust/switchyard_protocol/llm/struct.ResponseOutput.html#structfield.content"><code>ResponseOutput.content: Vec<ContentBlock></code></a> | Ordered text, reasoning, tool, and other output content. | `text`, `thinking`, `tool_use`, and other blocks | Message content, reasoning, function calls, and other items | Message `content`, `tool_calls`, and provider-specific reasoning fields |+| <a href="../../../rust/switchyard_protocol/llm/struct.ResponseOutput.html#structfield.stop_reason"><code>ResponseOutput.stop_reason: Option<StopReason></code></a> | Why this output stopped. | `stop_reason` | Response `status` and `incomplete_details` | `choices[].finish_reason` |+| <a href="../../../rust/switchyard_protocol/llm/struct.ResponseOutput.html#structfield.url_citations"><code>ResponseOutput.url_citations: Vec<UrlCitation></code></a> | Cited URLs with text offsets. | Not normalized by the current codec | `output_text.annotations` | Message `annotations` when supplied |+| <a href="../../../rust/switchyard_protocol/llm/struct.AggLlmResponse.html#structfield.usage"><code>AggLlmResponse.usage: Usage</code></a> | Normalized usage counts and any provider-reported cost. | `usage` | `usage` | `usage` |+| <a href="../../../rust/switchyard_protocol/llm/struct.AggLlmResponse.html#structfield.extensions"><code>AggLlmResponse.extensions: ProviderExtensions</code></a> | Provider-only response fields. | Source fields retained by the codec | Source fields retained by the codec | Source fields retained by the codec |+| <a href="../../../rust/switchyard_protocol/llm/struct.AggLlmResponse.html#structfield.preservation"><code>AggLlmResponse.preservation: PreservationMetadata</code></a> | Original JSON for same-format buffered replay. | Original parsed body | Original parsed body | Original parsed body |++The envelope also carries <a href="../../../rust/switchyard_protocol/envelope/struct.Response.html#structfield.metadata"><code>Response.metadata</code></a> and <a href="../../../rust/switchyard_protocol/envelope/struct.Response.html#structfield.upstream_headers"><code>Response.upstream_headers</code></a>.+<a href="../../../rust/switchyard_protocol/envelope/struct.Response.html#method.served_model"><code>Response.served_model()</code></a> reads the Switchyard target recorded in metadata.+<a href="../../../rust/switchyard_protocol/envelope/struct.Response.html#method.selected_model"><code>Response.selected_model()</code></a> reads the provider model from a buffered result and+returns no model for a live stream, since inspecting its events would consume it.++## Usage++<a href="../../../rust/switchyard_protocol/llm/struct.Usage.html#structfield.input_tokens"><code>Usage.input_tokens</code></a> counts **non-cached input**. When an OpenAI-style provider+reports aggregate input counts, the codecs subtract reported cache-read and+cache-write counts. The cache details remain separate.++| Protocol field and type | Meaning | Anthropic Messages | OpenAI Responses | Chat Completions |+|---|---|---|---|---|+| <a href="../../../rust/switchyard_protocol/llm/struct.Usage.html#structfield.input_tokens"><code>Usage.input_tokens: Option<u64></code></a> | Non-cached input tokens. | `input_tokens` | `input_tokens` minus reported cache detail | `prompt_tokens` minus reported cache detail |+| <a href="../../../rust/switchyard_protocol/llm/struct.InputCacheUsage.html#structfield.cached_input_tokens"><code>InputCacheUsage.cached_input_tokens: Option<u64></code></a> | Input read from cache. | `cache_read_input_tokens` | Input token details, when reported | Prompt token details, when reported |+| <a href="../../../rust/switchyard_protocol/llm/struct.InputCacheUsage.html#structfield.cache_creation_input_tokens"><code>InputCacheUsage.cache_creation_input_tokens: Option<u64></code></a> | Input written to cache. | `cache_creation_input_tokens` | Input token details, when reported | Prompt token details, when reported |+| <a href="../../../rust/switchyard_protocol/llm/struct.Usage.html#structfield.output_tokens"><code>Usage.output_tokens: Option<u64></code></a> | Output tokens, including reasoning when counted by the provider. | `output_tokens` | `output_tokens` | `completion_tokens` |+| <a href="../../../rust/switchyard_protocol/llm/struct.Usage.html#structfield.reasoning_tokens"><code>Usage.reasoning_tokens: Option<u64></code></a> | Reasoning output reported separately. | Provider token details, when reported | `output_tokens_details.reasoning_tokens` | Completion token details, when reported |+| <a href="../../../rust/switchyard_protocol/llm/struct.Usage.html#structfield.total_tokens"><code>Usage.total_tokens: Option<u64></code></a> | Provider-reported or codec-computed total. | Computed when needed | `total_tokens` or computed total | `total_tokens` or computed total |+| <a href="../../../rust/switchyard_protocol/llm/struct.Usage.html#structfield.provider_cost"><code>Usage.provider_cost: Option<f64></code></a> | Provider-reported monetary charge in USD. | Only when reported | Only when reported | Only when reported |++Missing counts remain unknown. Do not turn an absent count into a measured zero+when adapting a response. <a href="../../../rust/switchyard_protocol/llm/struct.Usage.html#structfield.cache"><code>Usage.cache</code></a> holds optional <a href="../../../rust/switchyard_protocol/llm/struct.InputCacheUsage.html"><code>InputCacheUsage</code></a> details.++## Stream events++Each item of <a href="../../../rust/switchyard_protocol/stream/type.LlmResponseStream.html"><code>LlmResponseStream</code></a> is a result containing <a href="../../../rust/switchyard_protocol/stream/struct.LlmResponseStreamEvent.html"><code>LlmResponseStreamEvent</code></a>.+An event contains zero or more neutral <a href="../../../rust/switchyard_protocol/stream/enum.LlmResponseChunk.html"><code>LlmResponseChunk</code></a> values and may retain+one parsed provider event for same-format replay.++| Protocol variant | Meaning | Anthropic Messages | OpenAI Responses | Chat Completions |+|---|---|---|---|---|+| <a href="../../../rust/switchyard_protocol/stream/enum.LlmResponseChunk.html#variant.MessageStart"><code>LlmResponseChunk::MessageStart</code></a> | Establish or update response identity. | `message_start` | Response creation/start events | Initial model and response fields |+| <a href="../../../rust/switchyard_protocol/stream/enum.LlmResponseChunk.html#variant.TextDelta"><code>LlmResponseChunk::TextDelta</code></a> | Append text at an output index. | Text `content_block_delta` | Output text delta events | `choices[].delta.content` |+| <a href="../../../rust/switchyard_protocol/stream/enum.LlmResponseChunk.html#variant.ReasoningDelta"><code>LlmResponseChunk::ReasoningDelta</code></a> | Append reasoning text. | Thinking deltas | Reasoning summary deltas | Provider-specific reasoning deltas |+| <a href="../../../rust/switchyard_protocol/stream/enum.LlmResponseChunk.html#variant.ReasoningDetailsDelta"><code>LlmResponseChunk::ReasoningDetailsDelta</code></a> | Retain structured or encrypted reasoning detail. | Reasoning signatures and related detail | Encrypted reasoning items | Provider-specific structured reasoning fields |+| <a href="../../../rust/switchyard_protocol/stream/enum.LlmResponseChunk.html#variant.ToolCallDelta"><code>LlmResponseChunk::ToolCallDelta</code></a> | Build a tool call's ID, name, and argument string. | Tool start and input JSON deltas | Function-call item and argument events | Tool-call deltas |+| <a href="../../../rust/switchyard_protocol/stream/enum.LlmResponseChunk.html#variant.Usage"><code>LlmResponseChunk::Usage</code></a> | Update available token counts. | Message usage and deltas | Usage on response events | Usage chunks |+| <a href="../../../rust/switchyard_protocol/stream/enum.LlmResponseChunk.html#variant.MessageStop"><code>LlmResponseChunk::MessageStop</code></a> | End the message. | Message stop events | Response completion events | Finish reasons and terminal framing |+| <a href="../../../rust/switchyard_protocol/stream/enum.LlmResponseChunk.html#variant.DecodeError"><code>LlmResponseChunk::DecodeError</code></a> / <a href="../../../rust/switchyard_protocol/stream/enum.LlmResponseChunk.html#variant.StreamError"><code>LlmResponseChunk::StreamError</code></a> | Report decoding or in-band upstream failure. | Codec/upstream error events | Codec/upstream error events | Codec/upstream error events |++These are semantic mappings. One provider event can produce several chunks,+and some events have no normalized chunks. Raw replay can still retain those+events. Algorithms read neutral chunks through <a href="../../../rust/switchyard_protocol/stream/struct.LlmResponseStreamEvent.html#method.normalized"><code>normalized()</code></a>.++## Stream lifetime and preservation++Pass original <a href="../../../rust/switchyard_protocol/stream/struct.LlmResponseStreamEvent.html"><code>LlmResponseStreamEvent</code></a> values through when exact same-format+parsed-JSON replay matters. <a href="../../../rust/switchyard_protocol/stream/struct.LlmResponseStreamEvent.html#method.replace_normalized"><code>replace_normalized()</code></a> returns an event with the new+chunks and clears its stale raw replay data.++<a href="../../../rust/switchyard_protocol/stream/enum.LlmResponse.html#method.into_agg"><code>LlmResponse::into_agg().await</code></a> consumes the stream and folds its normalized+chunks into an <a href="../../../rust/switchyard_protocol/llm/struct.AggLlmResponse.html"><code>AggLlmResponse</code></a>. Stream item failures, decode failures, and+in-band upstream errors become errors. Aggregation does not preserve the original+provider event sequence, MCP activity, unknown blocks, or all response extensions.+The current accumulator also folds output indices into one assistant output.++Dropping a stream stops local consumption. It cannot guarantee that the provider+stops generation or billing. See [concurrency and cancellation](../host_integration.md#concurrent-calls-and-cancellation)+for host-owned work, and **[Translation](translation.md#stream-translation)** for+decoding, encoding, and finishing provider streams.diff --git a/docs/reference/protocols/translation.md b/docs/reference/protocols/translation.mdnew file mode 100644index 000000000..995e74eee--- /dev/null+++ b/docs/reference/protocols/translation.md@@ -0,0 +1,95 @@+# LLM Translation++For a small integration example, read **[Use the translation crate](../../protocols/llm.md#use-the-translation-crate)**.+For field mappings, use **[LLM Requests](llm_requests.md)** and+**[Responses and Streams](llm_responses.md)**.++<a href="../../../rust/switchyard_translation/engine/struct.TranslationEngine.html"><code>TranslationEngine</code></a> converts between provider JSON and Switchyard protocol+types. It makes no network calls. <a href="../../../rust/switchyard_protocol/format/enum.WireFormat.html"><code>WireFormat</code></a> identifies the built-in Chat+Completions, Responses, and Anthropic Messages formats. <a href="../../../rust/switchyard_protocol/format/struct.FormatId.html"><code>FormatId</code></a> can also+identify custom formats registered through the codec interfaces.++## Engine operations++All operations below are methods of <a href="../../../rust/switchyard_translation/engine/struct.TranslationEngine.html"><code>TranslationEngine</code></a>. The request and+buffered-response methods take a <a href="../../../rust/switchyard_translation/policy/struct.TranslationPolicy.html"><code>TranslationPolicy</code></a>.++| Method | Input and result | Use it for |+|---|---|---|+| <a href="../../../rust/switchyard_translation/engine/struct.TranslationEngine.html#method.decode_request"><code>decode_request</code></a> | Provider JSON to <a href="../../../rust/switchyard_protocol/llm/struct.LlmRequest.html"><code>LlmRequest</code></a>, plus diagnostics. | Accepting a request from a provider-facing gateway or harness. |+| <a href="../../../rust/switchyard_translation/engine/struct.TranslationEngine.html#method.encode_request"><code>encode_request</code></a> | <a href="../../../rust/switchyard_protocol/llm/struct.LlmRequest.html"><code>LlmRequest</code></a> to provider JSON, plus diagnostics. | Sending a call through an existing client. |+| <a href="../../../rust/switchyard_translation/engine/struct.TranslationEngine.html#method.translate_request"><code>translate_request</code></a> | Source provider JSON to target provider JSON. | Direct request conversion without handling the protocol value yourself. |+| <a href="../../../rust/switchyard_translation/engine/struct.TranslationEngine.html#method.decode_response"><code>decode_response</code></a> | Provider JSON to <a href="../../../rust/switchyard_protocol/llm/struct.AggLlmResponse.html"><code>AggLlmResponse</code></a>, plus diagnostics. | Returning a buffered response to libsy. |+| <a href="../../../rust/switchyard_translation/engine/struct.TranslationEngine.html#method.encode_response"><code>encode_response</code></a> | <a href="../../../rust/switchyard_protocol/llm/struct.AggLlmResponse.html"><code>AggLlmResponse</code></a> to provider JSON, plus diagnostics. | Returning a buffered answer in the caller's format. |+| <a href="../../../rust/switchyard_translation/engine/struct.TranslationEngine.html#method.translate_response"><code>translate_response</code></a> | Source provider JSON to target provider JSON. | Direct buffered-response conversion. |+| <a href="../../../rust/switchyard_translation/engine/struct.TranslationEngine.html#method.decode_stream_event"><code>decode_stream_event</code></a> | One parsed provider event to <a href="../../../rust/switchyard_protocol/stream/struct.LlmResponseStreamEvent.html"><code>LlmResponseStreamEvent</code></a>. | Retaining the source event alongside neutral chunks. |+| <a href="../../../rust/switchyard_translation/engine/struct.TranslationEngine.html#method.encode_stream_event"><code>encode_stream_event</code></a> | One <a href="../../../rust/switchyard_protocol/stream/struct.LlmResponseStreamEvent.html"><code>LlmResponseStreamEvent</code></a> to zero or more provider events. | Replaying or converting incremental output. |+| <a href="../../../rust/switchyard_translation/engine/struct.TranslationEngine.html#method.finish_stream"><code>finish_stream</code></a> | Stream state to any remaining target events. | Closing the target format after the source stream ends. |++Buffered encoders return <a href="../../../rust/switchyard_translation/engine/struct.TranslationOutput.html"><code>TranslationOutput</code></a> with the encoded body and+<a href="../../../rust/switchyard_translation/diagnostic/struct.TranslationDiagnostic.html"><code>TranslationDiagnostic</code></a> values. Inspect diagnostics from decoding and encoding;+the direct translation helpers combine them and annotate source/target formats.++## Policies++These defaults come from <a href="../../../rust/switchyard_translation/policy/struct.TranslationPolicy.html#method.default"><code>TranslationPolicy::default()</code></a>.++| Field and type | Default | Meaning |+|---|---|---|+| <a href="../../../rust/switchyard_translation/policy/struct.TranslationPolicy.html#structfield.unknown_field_policy"><code>TranslationPolicy.unknown_field_policy: UnknownFieldPolicy</code></a> | <a href="../../../rust/switchyard_translation/policy/enum.UnknownFieldPolicy.html#variant.Preserve"><code>UnknownFieldPolicy::Preserve</code></a> | Keep unknown provider fields where the codec can retain them. Alternatives warn and drop, or reject. |+| <a href="../../../rust/switchyard_translation/policy/struct.TranslationPolicy.html#structfield.lossy_conversion_policy"><code>TranslationPolicy.lossy_conversion_policy: LossyConversionPolicy</code></a> | <a href="../../../rust/switchyard_translation/policy/enum.LossyConversionPolicy.html#variant.AllowWithDiagnostics"><code>LossyConversionPolicy::AllowWithDiagnostics</code></a> | Allow known losses and record warnings. Use <a href="../../../rust/switchyard_translation/policy/enum.LossyConversionPolicy.html#variant.Reject"><code>LossyConversionPolicy::Reject</code></a> to fail on known lossy conversions. |+| <a href="../../../rust/switchyard_translation/policy/struct.TranslationPolicy.html#structfield.deterministic_ids"><code>TranslationPolicy.deterministic_ids: DeterministicIdPolicy</code></a> | <a href="../../../rust/switchyard_translation/policy/enum.DeterministicIdPolicy.html#variant.GenerateStable"><code>DeterministicIdPolicy::GenerateStable</code></a>, prefix `sw` | Generate stable IDs when needed. The alternative preserves IDs without generation. |+| <a href="../../../rust/switchyard_translation/policy/struct.TranslationPolicy.html#structfield.preservation"><code>TranslationPolicy.preservation: PreservationPolicy</code></a> | <a href="../../../rust/switchyard_translation/policy/enum.PreservationPolicy.html#variant.InMemory"><code>PreservationPolicy::InMemory</code></a> | Retain original parsed bodies in protocol values for replay. |+| <a href="../../../rust/switchyard_translation/policy/struct.TranslationPolicy.html#structfield.target_capabilities"><code>TranslationPolicy.target_capabilities: TargetCapabilities</code></a> | All capability fields unspecified | Optionally constrain supported tools, media, reasoning, schema output, and other target features. |++Unspecified capabilities are not proof that a model supports a feature. A+provider can still reject the encoded request. Known-loss rejection covers the+codec's implemented checks, rather than proving semantic equivalence for every+possible provider extension.++## Preservation and edits++**An original body takes precedence over reconstructed fields for its own format.**+The codec looks up a request in <a href="../../../rust/switchyard_protocol/llm/struct.PreservationMetadata.html#structfield.requests"><code>PreservationMetadata.requests</code></a>, or a buffered+response in <a href="../../../rust/switchyard_protocol/llm/struct.PreservationMetadata.html#structfield.responses"><code>PreservationMetadata.responses</code></a>.++| Policy | Behavior | Boundary |+|---|---|---|+| <a href="../../../rust/switchyard_translation/policy/enum.PreservationPolicy.html#variant.InMemory"><code>PreservationPolicy::InMemory</code></a> | Keep original JSON in protocol values. | Same-format replay works while that preservation data remains available. |+| <a href="../../../rust/switchyard_translation/policy/enum.PreservationPolicy.html#variant.Embed"><code>PreservationPolicy::Embed</code></a> | Also embed preservation metadata in translated JSON. | Supports multi-hop round trips when intermediaries retain it; provider metadata limits can prevent use. |+| <a href="../../../rust/switchyard_translation/policy/enum.PreservationPolicy.html#variant.Disabled"><code>PreservationPolicy::Disabled</code></a> | Do not use preservation for replay. | Encoders reconstruct the body from current protocol fields. |++If you edit a decoded request, remove that format's entry from+<a href="../../../rust/switchyard_protocol/llm/struct.PreservationMetadata.html#structfield.requests"><code>PreservationMetadata.requests</code></a> or encode with <a href="../../../rust/switchyard_translation/policy/enum.PreservationPolicy.html#variant.Disabled"><code>PreservationPolicy::Disabled</code></a>.+This applies to model changes, messages, generation settings, and schema+enforcement. Apply the same rule to buffered response edits.++Replay preserves parsed JSON values, not original HTTP bytes, whitespace, or+headers. Input cleanup can happen before capture, including removal of Codex+internal compaction markers. <a href="../../../rust/switchyard_protocol/envelope/struct.Request.html#structfield.raw_request"><code>Request.raw_request</code></a> is host-owned and is not the+codec's replay source. Preserved data can contain the full conversation and+other sensitive provider fields.++## Stream translation++Use a separate <a href="../../../rust/switchyard_translation/codecs/stream/struct.StreamTranslationState.html"><code>StreamTranslationState</code></a> for each response stream. Feed parsed+provider events to <a href="../../../rust/switchyard_translation/engine/struct.TranslationEngine.html#method.decode_stream_event"><code>decode_stream_event</code></a>, pass the resulting event to+<a href="../../../rust/switchyard_translation/engine/struct.TranslationEngine.html#method.encode_stream_event"><code>encode_stream_event</code></a>, and call <a href="../../../rust/switchyard_translation/engine/struct.TranslationEngine.html#method.finish_stream"><code>finish_stream</code></a> when the source closes.+The encoder may emit several target events for one input event.++When the source and target formats match, a preserved event replays its retained+JSON. Cross-format encoding uses normalized chunks. After changing a stream+event, use <a href="../../../rust/switchyard_protocol/stream/struct.LlmResponseStreamEvent.html#method.replace_normalized"><code>replace_normalized()</code></a> to clear raw data that no longer describes it.+The [response reference](llm_responses.md#stream-lifetime-and-preservation)+explains what aggregation retains and drops.++## Conversion limits++Retention and destination support are separate. Anthropic has no distinct+developer authority, the OpenAI formats have no standard tool-result error flag,+and provider built-in tools or encrypted reasoning may lack equivalents.++Use <a href="../../../rust/switchyard_protocol/llm/struct.ProviderExtensions.html"><code>ProviderExtensions</code></a> and preservation to retain source data, then inspect+diagnostics and the destination model's capabilities. For a custom format,+<a href="../../../rust/switchyard_translation/codecs/trait.FormatCodec.html"><code>FormatCodec</code></a> and <a href="../../../rust/switchyard_translation/codecs/stream/trait.StreamCodec.html"><code>StreamCodec</code></a> define the codec contracts;+<a href="../../../rust/switchyard_translation/engine/struct.TranslationEngine.html#method.with_registries"><code>TranslationEngine::with_registries</code></a> accepts explicit registries.diff --git a/docs/reference/rust_api.md b/docs/reference/rust_api.mdindex 2cce7f567..e0fa41e82 100644--- a/docs/reference/rust_api.md+++ b/docs/reference/rust_api.md@@ -24,6 +24,16 @@ Completions, OpenAI Responses, and Anthropic Messages formats.<a href="../../rust/switchyard_translation/">Open the generated translation API documentation</a>+## `switchyard-llm-client`++Use the optional HTTP clients for LLM and decision calls, or the supplied+execution helpers when your application does not drive the step stream itself.++<a href="../../rust/switchyard_llm_client/">Open the generated client API documentation</a>++For field meanings and provider API mappings, use the [LLM protocol reference](protocols/llm_requests.md)+or [decision protocol reference](protocols/decision_questions.md).+## Crate boundaryUse the [Rust quickstart](../getting_started.md#install) for compatible dependencydiff --git a/mkdocs.yml b/mkdocs.ymlindex 49371fd59..48bf8dfb3 100644--- a/mkdocs.yml+++ b/mkdocs.yml@@ -20,6 +20,10 @@ nav:- Get Started:- Getting Started: getting_started.md- Mechanics of Libsy: mechanics_of_libsy.md+ - Switchyard Protocols:+ - Overview: protocols/index.md+ - LLM-Related Protocols: protocols/llm.md+ - Decision Model-Related Protocols: protocols/decisions.md- Evaluate Locally: operations/local_evaluation.md- Known Issues: known_issues.md- Benchmarks: benchmarks.md@@ -52,6 +56,14 @@ nav:- Context-Window Handling: operations/context_window.md- Soak Testing: operations/soak_test.md- Reference:+ - Protocols:+ - LLM:+ - Requests: reference/protocols/llm_requests.md+ - Responses and Streams: reference/protocols/llm_responses.md+ - Translation: reference/protocols/translation.md+ - Decision Models:+ - Questions: reference/protocols/decision_questions.md+ - Answers: reference/protocols/decision_answers.md- Host Integration: reference/host_integration.md- Server CLI Reference: cli_reference.md- TOML Schema: reference/toml_schema.md