Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
feat(relay-plugin): support nemo relay 0.8 through 0.x (#688)
Bryan Bednarski35f43af6addb
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex c6adadd03..3860fa8a6 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -19,7 +19,7 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).- **NeMo Relay native plugin** — a dynamically loaded integration that loadsSwitchyard's standard TOML deployment and executes its `switchyard-runner`-supported configured routes in process. Managed calls require NeMo Relay- `>=0.8.1,<0.9.0`; unknown models use Relay's continuation unchanged.+ `>=0.8.0, <1.0.0`; unknown models use Relay's continuation unchanged.- **NeMo Relay routing marks** — routing-model usage, measured routingoverhead, and selected-model decisions are emitted as ATOF marks. The finaldiff --git a/Cargo.lock b/Cargo.lockindex 44193fb0e..b353d0657 100644--- a/Cargo.lock+++ b/Cargo.lock@@ -512,7 +512,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"dependencies = ["libc",- "windows-sys 0.61.2",+ "windows-sys 0.52.0",][[package]]@@ -1241,9 +1241,9 @@ dependencies = [[[package]]name = "nemo-relay-plugin"-version = "0.8.1"+version = "0.8.4"source = "registry+https://github.com/rust-lang/crates.io-index"-checksum = "6ab4eaa56f021794a8b858c1ec6783679ad6798653e00a179166182bc4868bec"+checksum = "b2186727d62ca3c1be2e56ae302e9f7799db00c2dfc2b5ab58ebe2ec820c08be"dependencies = ["futures","nemo-relay-types",@@ -1255,9 +1255,9 @@ dependencies = [[[package]]name = "nemo-relay-types"-version = "0.8.1"+version = "0.8.4"source = "registry+https://github.com/rust-lang/crates.io-index"-checksum = "1bd5df5a9286dd167f668939decc7c719c6b7e4253a2f758b5fd3b8af22cc3b0"+checksum = "e31bccc10dd27dda5035f9ba9dcda9b9a5ad17abe6474e43d7147b006163e95b"dependencies = ["bitflags","chrono",@@ -1764,7 +1764,7 @@ dependencies = ["once_cell","socket2","tracing",- "windows-sys 0.61.2",+ "windows-sys 0.52.0",][[package]]@@ -1998,7 +1998,7 @@ dependencies = ["errno","libc","linux-raw-sys",- "windows-sys 0.61.2",+ "windows-sys 0.52.0",][[package]]@@ -2055,7 +2055,7 @@ dependencies = ["security-framework","security-framework-sys","webpki-root-certs",- "windows-sys 0.61.2",+ "windows-sys 0.52.0",][[package]]@@ -2575,7 +2575,7 @@ dependencies = ["getrandom 0.4.3","once_cell","rustix",- "windows-sys 0.61.2",+ "windows-sys 0.52.0",][[package]]@@ -3123,7 +3123,7 @@ version = "0.1.11"source = "registry+https://github.com/rust-lang/crates.io-index"checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22"dependencies = [- "windows-sys 0.61.2",+ "windows-sys 0.52.0",][[package]]diff --git a/Cargo.toml b/Cargo.tomlindex e3e26e114..c8a3a6449 100644--- a/Cargo.toml+++ b/Cargo.toml@@ -35,7 +35,7 @@ http = "1"httpdate = "1"jsonschema = { version = "0.49.4", default-features = false }jsonptr = { version = "0.8.1", default-features = false, features = ["std", "json", "resolve"] }-nemo-relay-plugin = ">=0.8.1, <0.9.0"+nemo-relay-plugin = ">=0.8.0, <1.0.0"parking_lot = "0.12"rand = "0.10"regex = "1"diff --git a/README.md b/README.mdindex 90c482e7a..7d2b6e067 100644--- a/README.md+++ b/README.md@@ -104,7 +104,7 @@ step 1, stop when you reach the result named under the heading.### Path 1 — Load the NeMo Relay PluginYou finish with an existing NeMo Relay deployment routing through Switchyard.-Requires NeMo Relay `>=0.8.1,<0.9.0` and a Rust toolchain to build the plugin.+Requires NeMo Relay `>=0.8.0, <1.0.0` and a Rust toolchain to build the plugin.**1. Build, package, and register the plugin.** Follow steps 1–3 of the[install guide in the plugin README](crates/switchyard-nemo-relay-plugin/README.md#install).diff --git a/crates/switchyard-nemo-relay-plugin/README.md b/crates/switchyard-nemo-relay-plugin/README.mdindex ce74bb02f..b1a4b48ef 100644--- a/crates/switchyard-nemo-relay-plugin/README.md+++ b/crates/switchyard-nemo-relay-plugin/README.md@@ -10,8 +10,29 @@ algorithm construction, retry policy, and route validation.## Install-The plugin requires NeMo Relay `>=0.8.1,<0.9.0`, a Rust toolchain, and Python 3-for the packaging script. Run every command from the repository root.+The plugin requires NeMo Relay `>=0.8.0, <1.0.0`.++### Install a released bundle++Released bundles are published in the+[NeMo Relay Plugins repository](https://github.com/NVIDIA/NeMo-Relay-Plugins/releases).+Find the `switchyard-plugin` release that corresponds to the Switchyard version+or source commit you need. The release's `.json` sidecar records the exact+Switchyard source commit and Relay version used to validate each platform+artifact.++Download the archive and matching `.sha256` file for your platform, verify the+checksum, extract the archive into a directory you plan to keep, then register+the extracted `relay-plugin.toml` with Relay using step 3 below. The release+version is independent of the Switchyard workspace version, so use the release+metadata rather than assuming that version numbers match.++### Build from source++Build from source when you need a Switchyard commit that does not have a+released bundle or when you need to customize the plugin. This path requires a+Rust toolchain and Python 3 for the packaging script. Run every command from+the repository root.**1. Build the shared library.**diff --git a/crates/switchyard-nemo-relay-plugin/relay-plugin.toml b/crates/switchyard-nemo-relay-plugin/relay-plugin.tomlindex e2f23aa17..865ad10fc 100644--- a/crates/switchyard-nemo-relay-plugin/relay-plugin.toml+++ b/crates/switchyard-nemo-relay-plugin/relay-plugin.toml@@ -8,7 +8,7 @@ id = "nvidia.switchyard"kind = "rust_dynamic"[compat]-relay = ">=0.8.1,<0.9.0"+relay = ">=0.8.0, <1.0.0"native_api = "1"[defaults]diff --git a/docs/integrations/nemo_relay.md b/docs/integrations/nemo_relay.mdindex 8cf072c36..83e17828e 100644--- a/docs/integrations/nemo_relay.md+++ b/docs/integrations/nemo_relay.md@@ -126,15 +126,17 @@ and## Set Up the PluginFollow the [plugin README](../../crates/switchyard-nemo-relay-plugin/README.md)-to build and package the native library, register and enable it in Relay, and-configure its deployment. Relay documents how to+to install a released bundle from the+[NeMo Relay Plugins repository](https://github.com/NVIDIA/NeMo-Relay-Plugins/releases)+or build and package the native library from source, then register and enable+it in Relay and configure its deployment. Relay documents how to[add and enable a discoverable plugin](https://docs.nvidia.com/nemo/relay/configure-plugins/discoverable-plugins#add-and-enable-a-plugin)and how it[validates the package before loading code](https://docs.nvidia.com/nemo/relay/configure-plugins/discoverable-plugins#validate-before-loading-code).!!! note "Relay compatibility"- The plugin requires `relay = ">=0.8.1,<0.9.0"` and native plugin API `1`.+ The plugin requires `relay = ">=0.8.0, <1.0.0"` and native plugin API `1`.The packaged[`relay-plugin.toml`](../../crates/switchyard-nemo-relay-plugin/relay-plugin.toml)is the source of truth.diff --git a/tests/relay_plugin/test_package_bundle.py b/tests/relay_plugin/test_package_bundle.pyindex 05da044d8..35b7ec303 100644--- a/tests/relay_plugin/test_package_bundle.py+++ b/tests/relay_plugin/test_package_bundle.py@@ -58,6 +58,7 @@ class PackageBundleTest(unittest.TestCase):self.assertEqual({path.name for path in output.iterdir()}, expected)manifest = (output / "relay-plugin.toml").read_text(encoding="utf-8")self.assertIn(f'artifact = "{library.name}"', manifest)+ self.assertIn('relay = ">=0.8.0, <1.0.0"', manifest)self.assertIn(hashlib.sha256(library.read_bytes()).hexdigest(), manifest)self.assertNotIn("<platform-library-file>", manifest)self.assertNotIn("<artifact-sha256>", manifest)