Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
docs(relay): clarify caller and target format translation (#736)
Bryan Bednarski0205775ca7a6
diff --git a/crates/switchyard-nemo-relay-plugin/README.md b/crates/switchyard-nemo-relay-plugin/README.mdindex ff4d43b8c..8a7f485f4 100644--- a/crates/switchyard-nemo-relay-plugin/README.md+++ b/crates/switchyard-nemo-relay-plugin/README.md@@ -191,11 +191,23 @@ deployment's route IDs.- Streaming responses are returned as unpolled translated streams; Relay ownscancellation and the outer serving-call lifecycle.-Each route's target client must use the caller's wire format: `openai_chat`,-`openai_responses`, or `anthropic_messages`. The runner selects the upstream-backend from that format rather than translating a route to a different-provider API. When one upstream model must serve multiple caller formats,-declare a target and route for each corresponding client format.+The caller and selected target may use different supported API formats:+`openai_chat`, `openai_responses`, or `anthropic_messages`. Switchyard translates+the request into the selected target's configured format and returns buffered+or streaming responses in the caller's original format. With server-owned+credentials, one route targeting an `openai_chat` client can serve all three+caller formats. Separate targets and routes are not required solely for format+translation.++Where `forward_auth` is supported, the caller and target must use the same+credential family: OpenAI-compatible (Chat Completions and Responses) or+Anthropic (Messages). Credential forwarding through Relay is also subject to+the limitation tracked in [NVBug 6777302](https://nvbugs/6777302) (NVIDIA internal).++Support for provider-specific fields depends on the source and target formats.+Test any fields that your application relies on before deploying a translated+route. See the [integration guide](../../docs/integrations/nemo_relay.md#request-handling)+for request handling, header forwarding, and streaming details.The plugin emits routing request, model-call, measured-overhead, and decisionmarks. Call marks distinguish routing from answer calls; decisions distinguishdiff --git a/docs/integrations/nemo_relay.md b/docs/integrations/nemo_relay.mdindex aec8465c7..66f38118b 100644--- a/docs/integrations/nemo_relay.md+++ b/docs/integrations/nemo_relay.md@@ -182,14 +182,23 @@ Only requests whose `model` is a string matching a Switchyard route ID arerouted. Other call types, missing or non-string model values, and unconfiguredmodel names are left unchanged by Switchyard and passed to Relay's next handler.-The caller and selected target may use different supported API formats.-Switchyard normalizes the request, routes it, and returns the response in the-caller's original format. If Switchyard forwards the caller's credential, both-formats must use the same credential family: OpenAI-compatible or Anthropic.+The caller and selected target may use different supported API formats:+`openai_chat`, `openai_responses`, or `anthropic_messages`. Switchyard translates+the request into the selected target's configured format and returns buffered+or streaming responses in the caller's original format. With server-owned+credentials, one route targeting an `openai_chat` client can serve all three+caller formats. Separate targets and routes are not required solely for format+translation.++Where `forward_auth` is supported, the caller and target must use the same+credential family: OpenAI-compatible (Chat Completions and Responses) or+Anthropic (Messages). Credential forwarding through Relay is also subject to+the limitation tracked in [NVBug 6777302](https://nvbugs/6777302) (NVIDIA internal).Support for provider-specific fields depends on the source and target formats.Test any fields that your application relies on before deploying a translated-route.+route. The [native-plugin README](../../crates/switchyard-nemo-relay-plugin/README.md#request-handling)+describes the same format and credential rules.### Header Forwarding