Skip to content

Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.

Rust

feat(relay-plugin): support nemo relay 0.8 through 0.x (#688)

Bryan Bednarski35f43af6addb
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c6adadd03..3860fa8a6 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -19,7 +19,7 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
- **NeMo Relay native plugin** — a dynamically loaded integration that loads
Switchyard's standard TOML deployment and executes its `switchyard-runner`-
supported configured routes in process. Managed calls require NeMo Relay
- `>=0.8.1,<0.9.0`; unknown models use Relay's continuation unchanged.
+ `>=0.8.0, <1.0.0`; unknown models use Relay's continuation unchanged.
- **NeMo Relay routing marks** — routing-model usage, measured routing
overhead, and selected-model decisions are emitted as ATOF marks. The final
diff --git a/Cargo.lock b/Cargo.lock
index 44193fb0e..b353d0657 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -512,7 +512,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
dependencies = [
"libc",
- "windows-sys 0.61.2",
+ "windows-sys 0.52.0",
]
[[package]]
@@ -1241,9 +1241,9 @@ dependencies = [
[[package]]
name = "nemo-relay-plugin"
-version = "0.8.1"
+version = "0.8.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "6ab4eaa56f021794a8b858c1ec6783679ad6798653e00a179166182bc4868bec"
+checksum = "b2186727d62ca3c1be2e56ae302e9f7799db00c2dfc2b5ab58ebe2ec820c08be"
dependencies = [
"futures",
"nemo-relay-types",
@@ -1255,9 +1255,9 @@ dependencies = [
[[package]]
name = "nemo-relay-types"
-version = "0.8.1"
+version = "0.8.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "1bd5df5a9286dd167f668939decc7c719c6b7e4253a2f758b5fd3b8af22cc3b0"
+checksum = "e31bccc10dd27dda5035f9ba9dcda9b9a5ad17abe6474e43d7147b006163e95b"
dependencies = [
"bitflags",
"chrono",
@@ -1764,7 +1764,7 @@ dependencies = [
"once_cell",
"socket2",
"tracing",
- "windows-sys 0.61.2",
+ "windows-sys 0.52.0",
]
[[package]]
@@ -1998,7 +1998,7 @@ dependencies = [
"errno",
"libc",
"linux-raw-sys",
- "windows-sys 0.61.2",
+ "windows-sys 0.52.0",
]
[[package]]
@@ -2055,7 +2055,7 @@ dependencies = [
"security-framework",
"security-framework-sys",
"webpki-root-certs",
- "windows-sys 0.61.2",
+ "windows-sys 0.52.0",
]
[[package]]
@@ -2575,7 +2575,7 @@ dependencies = [
"getrandom 0.4.3",
"once_cell",
"rustix",
- "windows-sys 0.61.2",
+ "windows-sys 0.52.0",
]
[[package]]
@@ -3123,7 +3123,7 @@ version = "0.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22"
dependencies = [
- "windows-sys 0.61.2",
+ "windows-sys 0.52.0",
]
[[package]]
diff --git a/Cargo.toml b/Cargo.toml
index e3e26e114..c8a3a6449 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -35,7 +35,7 @@ http = "1"
httpdate = "1"
jsonschema = { version = "0.49.4", default-features = false }
jsonptr = { version = "0.8.1", default-features = false, features = ["std", "json", "resolve"] }
-nemo-relay-plugin = ">=0.8.1, <0.9.0"
+nemo-relay-plugin = ">=0.8.0, <1.0.0"
parking_lot = "0.12"
rand = "0.10"
regex = "1"
diff --git a/README.md b/README.md
index 90c482e7a..7d2b6e067 100644
--- a/README.md
+++ b/README.md
@@ -104,7 +104,7 @@ step 1, stop when you reach the result named under the heading.
### Path 1 — Load the NeMo Relay Plugin
You finish with an existing NeMo Relay deployment routing through Switchyard.
-Requires NeMo Relay `>=0.8.1,<0.9.0` and a Rust toolchain to build the plugin.
+Requires NeMo Relay `>=0.8.0, <1.0.0` and a Rust toolchain to build the plugin.
**1. Build, package, and register the plugin.** Follow steps 1–3 of the
[install guide in the plugin README](crates/switchyard-nemo-relay-plugin/README.md#install).
diff --git a/crates/switchyard-nemo-relay-plugin/README.md b/crates/switchyard-nemo-relay-plugin/README.md
index ce74bb02f..b1a4b48ef 100644
--- a/crates/switchyard-nemo-relay-plugin/README.md
+++ b/crates/switchyard-nemo-relay-plugin/README.md
@@ -10,8 +10,29 @@ algorithm construction, retry policy, and route validation.
## Install
-The plugin requires NeMo Relay `>=0.8.1,<0.9.0`, a Rust toolchain, and Python 3
-for the packaging script. Run every command from the repository root.
+The plugin requires NeMo Relay `>=0.8.0, <1.0.0`.
+
+### Install a released bundle
+
+Released bundles are published in the
+[NeMo Relay Plugins repository](https://github.com/NVIDIA/NeMo-Relay-Plugins/releases).
+Find the `switchyard-plugin` release that corresponds to the Switchyard version
+or source commit you need. The release's `.json` sidecar records the exact
+Switchyard source commit and Relay version used to validate each platform
+artifact.
+
+Download the archive and matching `.sha256` file for your platform, verify the
+checksum, extract the archive into a directory you plan to keep, then register
+the extracted `relay-plugin.toml` with Relay using step 3 below. The release
+version is independent of the Switchyard workspace version, so use the release
+metadata rather than assuming that version numbers match.
+
+### Build from source
+
+Build from source when you need a Switchyard commit that does not have a
+released bundle or when you need to customize the plugin. This path requires a
+Rust toolchain and Python 3 for the packaging script. Run every command from
+the repository root.
**1. Build the shared library.**
diff --git a/crates/switchyard-nemo-relay-plugin/relay-plugin.toml b/crates/switchyard-nemo-relay-plugin/relay-plugin.toml
index e2f23aa17..865ad10fc 100644
--- a/crates/switchyard-nemo-relay-plugin/relay-plugin.toml
+++ b/crates/switchyard-nemo-relay-plugin/relay-plugin.toml
@@ -8,7 +8,7 @@ id = "nvidia.switchyard"
kind = "rust_dynamic"
[compat]
-relay = ">=0.8.1,<0.9.0"
+relay = ">=0.8.0, <1.0.0"
native_api = "1"
[defaults]
diff --git a/docs/integrations/nemo_relay.md b/docs/integrations/nemo_relay.md
index 8cf072c36..83e17828e 100644
--- a/docs/integrations/nemo_relay.md
+++ b/docs/integrations/nemo_relay.md
@@ -126,15 +126,17 @@ and
## Set Up the Plugin
Follow the [plugin README](../../crates/switchyard-nemo-relay-plugin/README.md)
-to build and package the native library, register and enable it in Relay, and
-configure its deployment. Relay documents how to
+to install a released bundle from the
+[NeMo Relay Plugins repository](https://github.com/NVIDIA/NeMo-Relay-Plugins/releases)
+or build and package the native library from source, then register and enable
+it in Relay and configure its deployment. Relay documents how to
[add and enable a discoverable plugin](https://docs.nvidia.com/nemo/relay/configure-plugins/discoverable-plugins#add-and-enable-a-plugin)
and how it
[validates the package before loading code](https://docs.nvidia.com/nemo/relay/configure-plugins/discoverable-plugins#validate-before-loading-code).
!!! note "Relay compatibility"
- The plugin requires `relay = ">=0.8.1,<0.9.0"` and native plugin API `1`.
+ The plugin requires `relay = ">=0.8.0, <1.0.0"` and native plugin API `1`.
The packaged
[`relay-plugin.toml`](../../crates/switchyard-nemo-relay-plugin/relay-plugin.toml)
is the source of truth.
diff --git a/tests/relay_plugin/test_package_bundle.py b/tests/relay_plugin/test_package_bundle.py
index 05da044d8..35b7ec303 100644
--- a/tests/relay_plugin/test_package_bundle.py
+++ b/tests/relay_plugin/test_package_bundle.py
@@ -58,6 +58,7 @@ class PackageBundleTest(unittest.TestCase):
self.assertEqual({path.name for path in output.iterdir()}, expected)
manifest = (output / "relay-plugin.toml").read_text(encoding="utf-8")
self.assertIn(f'artifact = "{library.name}"', manifest)
+ self.assertIn('relay = ">=0.8.0, <1.0.0"', manifest)
self.assertIn(hashlib.sha256(library.read_bytes()).hexdigest(), manifest)
self.assertNotIn("<platform-library-file>", manifest)
self.assertNotIn("<artifact-sha256>", manifest)