Skip to content

Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.

Rust

feat(go): add native embedded routing bindings

Alex Fournier0240746473c5
diff --git a/bindings/go/README.md b/bindings/go/README.md
new file mode 100644
index 000000000..267ffe5db
--- /dev/null
+++ b/bindings/go/README.md
@@ -0,0 +1,116 @@
+# NVIDIA NeMo Switchyard
+
+NVIDIA NeMo Switchyard is a library for routing LLM requests between models.
+These Go bindings run Switchyard algorithms inside your application. Your
+application owns model eligibility, credentials, provider calls, and retries.
+
+## Installation
+
+Requires Go 1.22+, cgo, a C compiler, and the repository's Rust toolchain. This is
+a source-only module. From a checkout of this repository, build the native
+library for the same platform as your Go application:
+
+```sh
+cargo build -p switchyard-ffi --release --locked
+```
+
+From your application's Go module, point Go at that checkout. Replace the path
+below with its absolute location:
+
+```sh
+go mod edit -replace=github.com/NVIDIA-NeMo/Switchyard/bindings/go=/absolute/path/to/Switchyard/bindings/go
+go get github.com/NVIDIA-NeMo/Switchyard/bindings/go
+```
+
+Native builds link `target/release/libswitchyard_ffi.a` into the Go host artifact.
+The native path still requires cgo and platform system libraries.
+Native artifacts are not downloaded or published by this package.
+
+## Use
+
+```go
+router, err := switchyard.New(json.RawMessage(`{
+ "algorithm": {"type": "random", "targets": ["fast", "capable"]},
+ "targets": {"fast": "fast-model", "capable": "capable-model"}
+}`))
+if err != nil {
+ return err
+}
+defer router.Close()
+
+outcome, err := router.Route(ctx, switchyard.Input{
+ Request: json.RawMessage(`{"messages":[{
+ "role":"user", "content":[{"type":"text", "text":"Fix this bug"}]
+ }]}`),
+ Models: map[string][]string{"any": {"fast-model", "capable-model"}},
+}, nil)
+```
+
+Import `encoding/json` and
+`github.com/NVIDIA-NeMo/Switchyard/bindings/go`.
+
+Supply the model groups needed by the configured algorithm. Composite needs
+`any` for eligible answer targets, plus `capable`, `efficient`, and `judge`.
+Random and signal-only Stage need no host callback. For Composite, supply a
+`HostCall` that handles `Work.Type`:
+
+- `call_model`: call the ordered `Work.Models` candidates and return `Reply` with
+ a normalized response and the target ID that answered in `ServedModel`.
+- `call_decision`: send `Work.Request` to `Work.Model` and return its normalized
+ decision response. `ServedModel` does not apply to decision calls.
+
+Requests and replies use Switchyard's protocol JSON. `DecodeRequest`,
+`EncodeRequest`, and `DecodeResponse` use the existing translation codecs for
+`openai_chat`, `openai_responses`, and `anthropic_messages`. Typed decision
+requests use the decision protocol directly.
+
+`Outcome` contains ordered targets, the prepared request, optional routing
+evidence, and an optional answer. When it has no answer, the host calls the
+selected target. Routing-produced responses are buffered; the host may stream
+the final selected provider response separately.
+
+## Cancellation and ownership
+
+`Route` waits in its calling goroutine. A host that needs immediate return can
+call it in a goroutine; waiting for routing steps does not occupy a blocked
+native-reader thread. Concurrent routes share one router.
+
+Host callbacks may run concurrently and must honor their context. Cancellation
+stops the native run and cancels the host context, but cannot forcibly stop Go
+code that ignores it. Late host results cannot access the freed native run.
+Host errors cross the boundary as messages, not concrete Go error types.
+
+`Close` prevents new runs; existing runs retain their algorithm and finish under
+their own contexts. Cancel those contexts to stop them. Router value copies
+share closed state, and repeated `Close` calls are safe. Native callback handles
+are released only after native cleanup has waited for callbacks to return.
+Call `Close` when finished; Go garbage collection does not release the router.
+
+`New` checks ABI generation 1 against the linked library. Scope session and agent
+IDs to the host's tenant and conversation. `Input.Metadata` and
+`Input.SubagentModels` carry routing context separately from provider requests.
+
+## Test
+
+From the repository root, after building the native library:
+
+```sh
+cd bindings/go
+go test -race ./...
+GOEXPERIMENT=cgocheck2 go test -race ./...
+go vet ./...
+```
+
+Rebuild Go with `go test -a` after changing the C header or native library; Go's
+cache does not track headers outside the package directory.
+
+### Go versions
+
+`go 1.22` in `go.mod` is the compatibility minimum, not a compiler pin. Use the
+latest patch of a [supported Go release](https://go.dev/doc/devel/release#policy)
+for development and production; compatibility with an older release does not
+make it a recommended production toolchain.
+
+Before merging changes, run the checks above with the latest patch of the
+declared minimum and the latest stable Go. Raise the minimum only when the
+binding or a dependency needs newer Go, and call out that change.
diff --git a/bindings/go/callbacks.go b/bindings/go/callbacks.go
new file mode 100644
index 000000000..f299fd5c9
--- /dev/null
+++ b/bindings/go/callbacks.go
@@ -0,0 +1,23 @@
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+// SPDX-License-Identifier: Apache-2.0
+
+package switchyard
+
+/*
+#include "switchyard.h"
+*/
+import "C"
+
+import "runtime/cgo"
+
+// The handle stays registered until native free has joined every accepted callback.
+//
+//export syGoNext
+func syGoNext(token C.uintptr_t, failure *C.char, output *C.char) {
+ result := nextResult{err: nativeError(failure)}
+ if output != nil {
+ result.data = []byte(C.GoString(output))
+ C.sy_string_free(output)
+ }
+ cgo.Handle(token).Value().(chan nextResult) <- result
+}
diff --git a/bindings/go/go.mod b/bindings/go/go.mod
new file mode 100644
index 000000000..deb11ad65
--- /dev/null
+++ b/bindings/go/go.mod
@@ -0,0 +1,3 @@
+module github.com/NVIDIA-NeMo/Switchyard/bindings/go
+
+go 1.22
diff --git a/bindings/go/switchyard.go b/bindings/go/switchyard.go
new file mode 100644
index 000000000..7c2e47d4c
--- /dev/null
+++ b/bindings/go/switchyard.go
@@ -0,0 +1,249 @@
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+// SPDX-License-Identifier: Apache-2.0
+
+// Package switchyard runs embedded routing algorithms with host-owned model calls.
+package switchyard
+
+/*
+#cgo CFLAGS: -I${SRCDIR}/../../crates/switchyard-ffi/include
+#cgo LDFLAGS: ${SRCDIR}/../../target/release/libswitchyard_ffi.a
+#cgo linux LDFLAGS: -ldl -lpthread -lm
+#cgo darwin LDFLAGS: -framework Security -framework CoreFoundation -liconv
+#include <stdlib.h>
+#include "switchyard.h"
+extern void syGoNext(uintptr_t user_data, char *error, char *step_json);
+*/
+import "C"
+
+import (
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "runtime/cgo"
+ "sync"
+ "unsafe"
+)
+
+// Router shares an algorithm across concurrent requests and value copies. Its zero value is closed.
+type Router struct {
+ state *routerState
+}
+
+type routerState struct {
+ mu sync.RWMutex
+ ptr *C.SyRouter
+}
+
+// Input supplies a normalized request and the host-approved model groups for this run.
+type Input struct {
+ Request json.RawMessage `json:"request"`
+ Models map[string][]string `json:"models"`
+ SubagentModels map[string][]string `json:"subagent_models"`
+ Metadata *RequestMetadata `json:"metadata,omitempty"`
+}
+
+// RequestMetadata carries routing context separately from the model request.
+type RequestMetadata struct {
+ SessionID string `json:"session_id,omitempty"`
+ AgentID string `json:"agent_id,omitempty"`
+ IsSubagent bool `json:"is_subagent,omitempty"`
+ IsDelegatedWork bool `json:"is_delegated_work,omitempty"`
+ SessionFinal *bool `json:"session_final,omitempty"`
+}
+
+// Work requests either a model call (Models) or a typed decision call (Model).
+type Work struct {
+ Type string `json:"type"`
+ ID uint64 `json:"id"`
+ Models []string `json:"models"`
+ Model string `json:"model"`
+ Request json.RawMessage `json:"request"`
+}
+
+// Reply contains a normalized response and, for model calls, the target that actually answered.
+type Reply struct {
+ Response json.RawMessage `json:"response"`
+ ServedModel string `json:"served_model,omitempty"`
+}
+
+// HostCall executes model or decision work. It must honor ctx and may run concurrently.
+// A call that ignores cancellation may outlive Route, but cannot access the native run.
+type HostCall func(context.Context, Work) (Reply, error)
+
+// Outcome retains the ordered candidates, prepared requests, and any routing-produced answer.
+type Outcome struct {
+ SelectedModelIDs []string `json:"selected_model_ids"`
+ Request json.RawMessage `json:"request"`
+ FallbackRequest *json.RawMessage `json:"fallback_request"`
+ Response *json.RawMessage `json:"response"`
+ Metadata json.RawMessage `json:"metadata"`
+}
+
+// New constructs an algorithm from {"algorithm": AlgorithmSpec, "targets": {alias: modelID}}.
+func New(config json.RawMessage) (*Router, error) {
+ if version := C.sy_abi_version(); version != C.SY_ABI_VERSION {
+ return nil, fmt.Errorf("Switchyard ABI version mismatch: library=%d, binding=%d", version, C.SY_ABI_VERSION)
+ }
+ value := C.CString(string(config))
+ defer C.free(unsafe.Pointer(value))
+ var ptr *C.SyRouter
+ if err := nativeError(C.sy_router_new(value, &ptr)); err != nil {
+ return nil, err
+ }
+ return &Router{state: &routerState{ptr: ptr}}, nil
+}
+
+// Close prevents new runs and releases the router. Active runs retain their algorithm.
+// Repeated calls, including through Router value copies, are safe.
+func (router *Router) Close() {
+ state := router.state
+ if state == nil {
+ return
+ }
+ state.mu.Lock()
+ defer state.mu.Unlock()
+ if state.ptr != nil {
+ C.sy_router_free(state.ptr)
+ state.ptr = nil
+ }
+}
+
+// Limit the router borrow to construction; each native run retains its own engine reference.
+func (router *Router) newRun(input Input) (*C.SyRun, error) {
+ data, err := json.Marshal(input)
+ if err != nil {
+ return nil, err
+ }
+ value := C.CString(string(data))
+ defer C.free(unsafe.Pointer(value))
+ state := router.state
+ if state == nil {
+ return nil, errors.New("Switchyard router is closed")
+ }
+ state.mu.RLock()
+ defer state.mu.RUnlock()
+ if state.ptr == nil {
+ return nil, errors.New("Switchyard router is closed")
+ }
+ var run *C.SyRun
+ err = nativeError(C.sy_run_new(state.ptr, value, &run))
+ return run, err
+}
+
+type nextResult struct {
+ data json.RawMessage
+ err error
+}
+
+type hostReply struct {
+ id uint64
+ reply Reply
+ err error
+}
+
+// Route waits for an outcome while the host services calls with its own clients.
+// Cancellation stops routing without waiting for host calls that ignore their context.
+func (router *Router) Route(ctx context.Context, input Input, call HostCall) (Outcome, error) {
+ if err := ctx.Err(); err != nil {
+ return Outcome{}, err
+ }
+ run, err := router.newRun(input)
+ if err != nil {
+ return Outcome{}, err
+ }
+ ctx, cancel := context.WithCancel(ctx)
+ // At most one read is outstanding; consume its result before requesting another.
+ next := make(chan nextResult, 1)
+ handle := cgo.NewHandle(next)
+ defer func() {
+ cancel()
+ C.sy_run_cancel(run)
+ C.sy_run_free(run)
+ handle.Delete()
+ }()
+ readNext := func() error {
+ return nativeError(C.sy_run_next_async(run, C.SyNextCallback(C.syGoNext), C.uintptr_t(handle)))
+ }
+ if err := readNext(); err != nil {
+ return Outcome{}, err
+ }
+ replies := make(chan hostReply)
+ for {
+ select {
+ case <-ctx.Done():
+ return Outcome{}, ctx.Err()
+ case result := <-next:
+ if err := ctx.Err(); err != nil {
+ return Outcome{}, err
+ }
+ if result.err != nil {
+ return Outcome{}, result.err
+ }
+ var work Work
+ if err := json.Unmarshal(result.data, &work); err != nil {
+ return Outcome{}, err
+ }
+ if work.Type == "done" {
+ var outcome Outcome
+ err := json.Unmarshal(result.data, &outcome)
+ if cancelErr := ctx.Err(); cancelErr != nil {
+ return Outcome{}, cancelErr
+ }
+ return outcome, err
+ }
+ if work.Type != "call_model" && work.Type != "call_decision" {
+ return Outcome{}, errors.New("unknown Switchyard step: " + work.Type)
+ }
+ if call == nil {
+ return Outcome{}, errors.New("Switchyard algorithm requires a host model client")
+ }
+ // Workers own Go data only; late results cannot call into a freed native run.
+ go func(work Work) {
+ if ctx.Err() != nil {
+ return
+ }
+ reply, err := call(ctx, work)
+ select {
+ case replies <- hostReply{work.ID, reply, err}:
+ case <-ctx.Done():
+ }
+ }(work)
+ if err := readNext(); err != nil {
+ return Outcome{}, err
+ }
+ case reply := <-replies:
+ envelope := struct {
+ Response json.RawMessage `json:"response,omitempty"`
+ ServedModel string `json:"served_model,omitempty"`
+ Error *string `json:"error,omitempty"`
+ }{Response: reply.reply.Response, ServedModel: reply.reply.ServedModel}
+ if reply.err != nil {
+ envelope.Response = nil
+ message := reply.err.Error()
+ envelope.Error = &message
+ }
+ data, err := json.Marshal(envelope)
+ if err != nil {
+ return Outcome{}, err
+ }
+ if err := ctx.Err(); err != nil {
+ return Outcome{}, err
+ }
+ value := C.CString(string(data))
+ err = nativeError(C.sy_run_respond(run, C.uint64_t(reply.id), value))
+ C.free(unsafe.Pointer(value))
+ if err != nil {
+ return Outcome{}, err
+ }
+ }
+ }
+}
+
+func nativeError(value *C.char) error {
+ if value == nil {
+ return nil
+ }
+ defer C.sy_string_free(value)
+ return errors.New(C.GoString(value))
+}
diff --git a/bindings/go/switchyard_test.go b/bindings/go/switchyard_test.go
new file mode 100644
index 000000000..36125245c
--- /dev/null
+++ b/bindings/go/switchyard_test.go
@@ -0,0 +1,382 @@
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+// SPDX-License-Identifier: Apache-2.0
+
+package switchyard
+
+import (
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "runtime"
+ "slices"
+ "strings"
+ "sync"
+ "testing"
+ "time"
+)
+
+const stage = `"capable_target":"strong","efficient_target":"weak","confidence_threshold":0.5`
+const composite = `{"type":"composite","classifier":{"target":"judge","base_threshold":0.5,"classify_trigger":"user_turn","fail_open":false},"stage":{` + stage + `}}`
+const judgeAnswer = `{"model":"provider-model","outputs":[{"role":"assistant","content":[{"type":"text","text":"{\"crux\":\"bounded task\",\"primary_rule\":\"SUP-1\",\"capability_boundary\":\"supported\",\"p_solve\":0.9}"}],"stop_reason":"end_turn"}]}`
+
+func newRouter(t *testing.T, algorithm string) *Router {
+ t.Helper()
+ router, err := New(json.RawMessage(`{"algorithm":` + algorithm + `,"targets":{"weak":"weak-model","strong":"strong-model","judge":"judge-model"}}`))
+ if err != nil {
+ t.Fatal(err)
+ }
+ t.Cleanup(router.Close)
+ return router
+}
+
+func input() Input {
+ return Input{
+ Request: json.RawMessage(`{"messages":[{"role":"user","content":[{"type":"text","text":"Fix this small bug"}]}]}`),
+ Models: map[string][]string{
+ "any": {"strong-model", "weak-model"}, "capable": {"strong-model"},
+ "efficient": {"weak-model"}, "judge": {"judge-model"},
+ },
+ }
+}
+
+// Completed outcomes retain fallback requests while absent requests remain optional.
+func TestOutcomeFallbackRequest(t *testing.T) {
+ request := input().Request
+ for _, tc := range []struct {
+ name, field string
+ present bool
+ }{
+ {"present", `"fallback_request":` + string(request), true},
+ {"missing", "", false},
+ {"null", `"fallback_request":null`, false},
+ } {
+ t.Run(tc.name, func(t *testing.T) {
+ var outcome Outcome
+ data := []byte(`{` + tc.field + `}`)
+ if err := json.Unmarshal(data, &outcome); err != nil {
+ t.Fatal(err)
+ }
+ if present := outcome.FallbackRequest != nil; present != tc.present {
+ t.Fatalf("fallback request present=%v, want %v", present, tc.present)
+ }
+ if tc.present && string(*outcome.FallbackRequest) != string(request) {
+ t.Fatalf("fallback request=%s, want %s", *outcome.FallbackRequest, request)
+ }
+ })
+ }
+}
+
+// Selection retains eligible targets and Stage's prepared request across the codecs.
+func TestSelectionAndTranslation(t *testing.T) {
+ type chatRequest struct {
+ Model string
+ Store *bool
+ Messages []struct{ Role, Content string }
+ }
+ for _, tc := range []struct {
+ name, algorithm, expected string
+ failed bool
+ }{
+ {"random", `{"type":"random","targets":["strong","weak"]}`, "weak-model", false},
+ {"stage", `{"type":"stage_router","picker":"efficient_first",` + stage + `}`, "weak-model", false},
+ {"escalation", `{"type":"stage_router","picker":"efficient_first",` + stage + `,"handoff_notes":{"escalation_note":"Review the failure"}}`, "strong-model", true},
+ } {
+ t.Run(tc.name, func(t *testing.T) {
+ router := newRouter(t, tc.algorithm)
+ request := input()
+ if tc.name == "random" {
+ request.Models["any"] = []string{"weak-model"}
+ }
+ if tc.failed {
+ request.Request = json.RawMessage(`{"messages":[{"role":"assistant","content":[{"type":"tool_call","id":"c","name":"bash","arguments":{}}]},{"role":"tool","content":[{"type":"tool_result","tool_call_id":"c","content":[{"type":"text","text":"Out of memory"}],"is_error":true}]}]}`)
+ }
+ outcome, err := router.Route(context.Background(), request, nil)
+ if err != nil || len(outcome.SelectedModelIDs) == 0 || outcome.SelectedModelIDs[0] != tc.expected {
+ t.Fatalf("outcome=%+v err=%v", outcome, err)
+ }
+ wire, err := EncodeRequest("openai_chat", outcome.Request, tc.expected)
+ if err != nil {
+ t.Fatal(err)
+ }
+ var prepared chatRequest
+ if err := json.Unmarshal(wire, &prepared); err != nil || prepared.Model != tc.expected {
+ t.Fatalf("prepared request=%s err=%v", wire, err)
+ }
+ if tc.failed {
+ if len(prepared.Messages) == 0 {
+ t.Fatal("missing handoff message")
+ }
+ last := prepared.Messages[len(prepared.Messages)-1]
+ if last.Role != "user" || last.Content != "Review the failure" {
+ t.Fatalf("handoff message=%+v", last)
+ }
+ }
+ })
+ }
+ request, err := DecodeRequest("openai_chat", json.RawMessage(`{"model":"route","messages":[{"role":"user","content":"hello"}],"store":false}`))
+ if err != nil {
+ t.Fatal(err)
+ }
+ wire, err := EncodeRequest("openai_chat", request, "actual-target")
+ if err != nil {
+ t.Fatal(err)
+ }
+ var encoded chatRequest
+ if err := json.Unmarshal(wire, &encoded); err != nil || encoded.Model != "actual-target" || encoded.Store == nil || *encoded.Store {
+ t.Fatalf("wire=%s err=%v", wire, err)
+ }
+ response, err := DecodeResponse("openai_chat", json.RawMessage(`{"choices":[{"index":0,"message":{"role":"assistant","content":"hello"},"finish_reason":"stop"}]}`))
+ if err != nil {
+ t.Fatal(err)
+ }
+ var decoded struct {
+ Outputs []struct {
+ Content []struct{ Text string }
+ }
+ }
+ if err := json.Unmarshal(response, &decoded); err != nil || len(decoded.Outputs) != 1 || len(decoded.Outputs[0].Content) != 1 || decoded.Outputs[0].Content[0].Text != "hello" {
+ t.Fatalf("response=%s err=%v", response, err)
+ }
+}
+
+// Both judge protocols use Go's client; missing clients and host failures remain errors.
+func TestCompositeHostCalls(t *testing.T) {
+ decision := `{"type":"composite","classifier":{"target":"judge","classify_trigger":"user_turn","fail_open":false,"decision":{"cutoff":0.5,"candidates":{"a":"strong","b":"weak"},"evidence":{}}},"stage":{` + stage + `}}`
+ for _, tc := range []struct {
+ name, algorithm, kind, selected, response, invalidResponse string
+ }{
+ {"model", composite, "call_model", "weak-model", judgeAnswer, `{"outputs":"wrong"}`},
+ {"decision", decision, "call_decision", "strong-model", `{"answers":{"route":{"value":{"type":"choice","data":{"selected":"advantage","probabilities":{"advantage":0.9,"no_advantage":0.1}}}}}}`, `{"answers":"wrong"}`},
+ } {
+ t.Run(tc.name, func(t *testing.T) {
+ router := newRouter(t, tc.algorithm)
+ calls := 0
+ outcome, err := router.Route(context.Background(), input(), func(_ context.Context, work Work) (Reply, error) {
+ calls++
+ isJudge := work.Model == "judge-model"
+ if tc.kind == "call_model" {
+ isJudge = slices.Equal(work.Models, []string{"judge-model"})
+ }
+ if work.Type != tc.kind || !isJudge || !json.Valid(work.Request) {
+ return Reply{}, fmt.Errorf("unexpected work: %+v", work)
+ }
+ return Reply{Response: json.RawMessage(tc.response)}, nil
+ })
+ if err != nil || calls != 1 || len(outcome.SelectedModelIDs) == 0 || outcome.SelectedModelIDs[0] != tc.selected {
+ t.Fatalf("calls=%d outcome=%+v err=%v", calls, outcome, err)
+ }
+ if _, err := router.Route(context.Background(), input(), nil); err == nil {
+ t.Fatal("missing host client succeeded")
+ }
+ _, err = router.Route(context.Background(), input(), func(context.Context, Work) (Reply, error) {
+ return Reply{}, errors.New("host unavailable")
+ })
+ if err == nil || !strings.Contains(err.Error(), "host unavailable") {
+ t.Fatalf("host failure returned %v", err)
+ }
+ _, err = router.Route(context.Background(), input(), func(context.Context, Work) (Reply, error) {
+ return Reply{Response: json.RawMessage(tc.invalidResponse)}, nil
+ })
+ if err == nil {
+ t.Fatal("invalid host response succeeded")
+ }
+ })
+ }
+}
+
+// Host failures remain subject to the algorithm's configured recovery policy.
+func TestHostFailureUsesAlgorithmPolicy(t *testing.T) {
+ algorithm := strings.Replace(composite, `"fail_open":false`, `"fail_open":true`, 1)
+ router := newRouter(t, algorithm)
+ calls := 0
+ outcome, err := router.Route(context.Background(), input(), func(context.Context, Work) (Reply, error) {
+ calls++
+ return Reply{}, errors.New("judge unavailable")
+ })
+ if err != nil || calls != 1 || !slices.Equal(outcome.SelectedModelIDs, []string{"weak-model", "strong-model"}) {
+ t.Fatalf("calls=%d outcome=%+v err=%v", calls, outcome, err)
+ }
+}
+
+// Cancellation releases native work even if a host ignores its context and replies later.
+func TestCancellationAndLateHostReply(t *testing.T) {
+ router := newRouter(t, composite)
+ ctx, cancel := context.WithCancel(context.Background())
+ defer cancel()
+ started, release, returned := make(chan struct{}), make(chan struct{}), make(chan struct{})
+ releaseHost := sync.OnceFunc(func() { close(release) })
+ defer releaseHost()
+ finished := make(chan error, 1)
+ go func() {
+ _, err := router.Route(ctx, input(), func(context.Context, Work) (Reply, error) {
+ defer close(returned)
+ router.Close()
+ close(started)
+ <-release
+ return Reply{Response: json.RawMessage(judgeAnswer)}, nil
+ })
+ finished <- err
+ }()
+ select {
+ case <-started:
+ case <-time.After(5 * time.Second):
+ t.Fatal("host callback could not close its router")
+ }
+ runtime.GC()
+ cancel()
+ select {
+ case err := <-finished:
+ if !errors.Is(err, context.Canceled) {
+ t.Fatalf("cancellation returned %v", err)
+ }
+ case <-time.After(5 * time.Second):
+ t.Fatal("cancellation waited for host code")
+ }
+ runtime.GC()
+ // Release after Route has freed its native run, then let the late Go reply drain.
+ releaseHost()
+ select {
+ case <-returned:
+ case <-time.After(5 * time.Second):
+ t.Fatal("late host callback did not finish")
+ }
+}
+
+// Concurrent requests keep eligible pools separate; router copies share closed state.
+func TestConcurrentRoutesAndClose(t *testing.T) {
+ router := newRouter(t, `{"type":"random","targets":["weak","strong"],"seed":7}`)
+ ctx, cancel := context.WithCancel(context.Background())
+ cancel()
+ if _, err := router.Route(ctx, input(), nil); !errors.Is(err, context.Canceled) {
+ t.Fatalf("cancelled context returned %v", err)
+ }
+ results := make(chan error, 24)
+ for i := 0; i < cap(results); i++ {
+ go func(i int) {
+ request := input()
+ model := []string{"weak-model", "strong-model"}[i%2]
+ request.Models = map[string][]string{"any": {model}}
+ outcome, err := router.Route(context.Background(), request, nil)
+ if err == nil && (len(outcome.SelectedModelIDs) != 1 || outcome.SelectedModelIDs[0] != model) {
+ err = fmt.Errorf("expected %s, got %v", model, outcome.SelectedModelIDs)
+ }
+ results <- err
+ }(i)
+ }
+ for i := 0; i < cap(results); i++ {
+ select {
+ case err := <-results:
+ if err != nil {
+ t.Fatal(err)
+ }
+ case <-time.After(5 * time.Second):
+ t.Fatal("concurrent route did not finish")
+ }
+ }
+ copy := *router
+ for _, closed := range []*Router{&copy, router, new(Router)} {
+ closed.Close()
+ closed.Close()
+ if _, err := closed.Route(context.Background(), input(), nil); err == nil {
+ t.Fatal("closed router accepted a request")
+ }
+ }
+ racing := newRouter(t, `{"type":"random","targets":["weak"]}`)
+ racingCopy, start := *racing, make(chan struct{})
+ go func() {
+ <-start
+ request := input()
+ request.Models["any"] = []string{"weak-model"}
+ outcome, err := racing.Route(context.Background(), request, nil)
+ if err != nil && err.Error() == "Switchyard router is closed" {
+ err = nil
+ } else if err == nil && (len(outcome.SelectedModelIDs) != 1 || outcome.SelectedModelIDs[0] != "weak-model") {
+ err = fmt.Errorf("close race selected %v", outcome.SelectedModelIDs)
+ }
+ results <- err
+ }()
+ go func() { <-start; racingCopy.Close(); results <- nil }()
+ close(start)
+ for i := 0; i < 2; i++ {
+ select {
+ case err := <-results:
+ if err != nil {
+ t.Fatal(err)
+ }
+ case <-time.After(5 * time.Second):
+ t.Fatal("close race did not finish")
+ }
+ }
+ runtime.GC()
+}
+
+// Served-target attribution and agent/session metadata survive the bridge.
+func TestReplyAttributionAndMetadata(t *testing.T) {
+ router := newRouter(t, `{"type":"advisor","executor_target":"strong","advisor_target":"judge","gate_trigger":"pattern","gate_trigger_pattern":"NEVER"}`)
+ request := input()
+ request.Models["efficient"] = []string{"strong-model", "weak-model"}
+ outcome, err := router.Route(context.Background(), request, func(_ context.Context, work Work) (Reply, error) {
+ if len(work.Models) != 2 {
+ return Reply{}, fmt.Errorf("missing fallback: %+v", work)
+ }
+ return Reply{Response: json.RawMessage(judgeAnswer), ServedModel: "weak-model"}, nil
+ })
+ if err != nil || len(outcome.SelectedModelIDs) != 1 || outcome.SelectedModelIDs[0] != "weak-model" || outcome.Response == nil {
+ t.Fatalf("outcome=%+v err=%v", outcome, err)
+ }
+ delegated := newRouter(t, `{"type":"passthrough","target":"strong","subagents":{"type":"passthrough","target":"weak"}}`)
+ for _, child := range []bool{false, true} {
+ request := input()
+ request.Models = map[string][]string{"any": {"strong-model"}}
+ request.SubagentModels = map[string][]string{"any": {"weak-model"}}
+ request.Metadata = &RequestMetadata{SessionID: "session", AgentID: "child", IsSubagent: true, IsDelegatedWork: child}
+ expected := "strong-model"
+ if child {
+ expected = "weak-model"
+ }
+ outcome, err := delegated.Route(context.Background(), request, nil)
+ if err != nil || len(outcome.SelectedModelIDs) != 1 || outcome.SelectedModelIDs[0] != expected {
+ t.Fatalf("delegated=%v outcome=%+v err=%v", child, outcome, err)
+ }
+ }
+ request = input()
+ request.SubagentModels = map[string][]string{}
+ request.Metadata = &RequestMetadata{IsSubagent: true, IsDelegatedWork: true}
+ encoded, err := json.Marshal(request)
+ if err != nil {
+ t.Fatal(err)
+ }
+ var envelope struct {
+ SubagentModels map[string][]string `json:"subagent_models"`
+ }
+ if err := json.Unmarshal(encoded, &envelope); err != nil || envelope.SubagentModels == nil || len(envelope.SubagentModels) != 0 {
+ t.Fatalf("explicit empty child pool was lost: %s err=%v", encoded, err)
+ }
+ if _, err := delegated.Route(context.Background(), request, nil); err == nil {
+ t.Fatal("empty child pool selected a parent model")
+ }
+ plan := newRouter(t, `{"type":"plan_execute","capable_target":"strong","efficient_target":"weak"}`)
+ for _, tc := range []struct {
+ agent string
+ child, final, edited bool
+ expected string
+ }{
+ {"a", true, false, true, "weak-model"},
+ {"b", true, false, false, "strong-model"},
+ {"root", false, false, false, "strong-model"},
+ {"a", true, true, false, "weak-model"},
+ {"a", true, false, false, "strong-model"},
+ } {
+ request := input()
+ request.Request = json.RawMessage(`{}`)
+ if tc.edited {
+ request.Request = json.RawMessage(`{"messages":[{"role":"assistant","content":[{"type":"tool_call","id":"edit","name":"apply_patch","arguments":{}}]}]}`)
+ }
+ request.Metadata = &RequestMetadata{SessionID: "session", AgentID: tc.agent, IsSubagent: tc.child, SessionFinal: &tc.final}
+ outcome, err := plan.Route(context.Background(), request, nil)
+ if err != nil || len(outcome.SelectedModelIDs) != 1 || outcome.SelectedModelIDs[0] != tc.expected {
+ t.Fatalf("agent=%s final=%v outcome=%+v err=%v", tc.agent, tc.final, outcome, err)
+ }
+ }
+}
diff --git a/bindings/go/translation.go b/bindings/go/translation.go
new file mode 100644
index 000000000..a322a06a2
--- /dev/null
+++ b/bindings/go/translation.go
@@ -0,0 +1,61 @@
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+// SPDX-License-Identifier: Apache-2.0
+
+package switchyard
+
+/*
+#include <stdlib.h>
+#include "switchyard.h"
+*/
+import "C"
+
+import (
+ "encoding/json"
+ "unsafe"
+)
+
+// DecodeRequest converts a provider wire body to Switchyard's normalized request.
+func DecodeRequest(format string, body json.RawMessage) (json.RawMessage, error) {
+ return translated(struct {
+ Format string `json:"format"`
+ Body json.RawMessage `json:"body"`
+ }{format, body}, func(input *C.char, output **C.char) *C.char {
+ return C.sy_decode_request(input, output)
+ })
+}
+
+// EncodeRequest stamps the target model and encodes the prepared normalized request.
+func EncodeRequest(format string, request json.RawMessage, model string) (json.RawMessage, error) {
+ return translated(struct {
+ Format string `json:"format"`
+ Request json.RawMessage `json:"request"`
+ Model string `json:"model"`
+ }{format, request, model}, func(input *C.char, output **C.char) *C.char {
+ return C.sy_encode_request(input, output)
+ })
+}
+
+// DecodeResponse converts a buffered provider response to Switchyard's normalized aggregate.
+func DecodeResponse(format string, body json.RawMessage) (json.RawMessage, error) {
+ return translated(struct {
+ Format string `json:"format"`
+ Body json.RawMessage `json:"body"`
+ }{format, body}, func(input *C.char, output **C.char) *C.char {
+ return C.sy_decode_response(input, output)
+ })
+}
+
+func translated(input any, call func(*C.char, **C.char) *C.char) (json.RawMessage, error) {
+ data, err := json.Marshal(input)
+ if err != nil {
+ return nil, err
+ }
+ value := C.CString(string(data))
+ defer C.free(unsafe.Pointer(value))
+ var output *C.char
+ if err := nativeError(call(value, &output)); err != nil {
+ return nil, err
+ }
+ defer C.sy_string_free(output)
+ return json.RawMessage(C.GoString(output)), nil
+}