Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
feat: Typescript bindings for libsy, for Pi extension (#1039)
Graham King6180e15c0d6c
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex f5dd62e40..63717b1a5 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -147,9 +147,11 @@ jobs:- run: npm --prefix bindings/typescript run build- run: npm --prefix bindings/typescript run typecheck- run: npm --prefix bindings/typescript test- - run: npm --prefix examples/pi ci- - run: npm --prefix examples/pi run typecheck- - run: npm --prefix examples/pi test+ # Re-enable when the Pi extension and its tests use @switchyard/libsy's+ # CompositeRouter, JSON config, and Pi-owned classifier calls.+ # - run: npm --prefix examples/pi ci+ # - run: npm --prefix examples/pi run typecheck+ # - run: npm --prefix examples/pi testtypecheck:name: Typecheck (mypy)diff --git a/Cargo.lock b/Cargo.lockindex 1be38b113..9cad0127e 100644--- a/Cargo.lock+++ b/Cargo.lock@@ -3383,14 +3383,15 @@ dependencies = [name = "switchyard-node"version = "0.3.0"dependencies = [+ "futures","napi","napi-build","napi-derive",+ "parking_lot",+ "serde","serde_json",+ "switchyard-libsy","switchyard-protocol",- "switchyard-runner",- "tokio",- "tokio-util",][[package]]diff --git a/bindings/typescript/README.md b/bindings/typescript/README.mdindex 1f9dfebdd..9c3306c0a 100644--- a/bindings/typescript/README.md+++ b/bindings/typescript/README.md@@ -1,7 +1,8 @@-# Switchyard runner for Node.js+# Switchyard libsy for Node.js-Local TypeScript bindings for choosing a model with Switchyard. The caller sends-the selected model's completion request and owns session state.+TypeScript bindings for composite routing. Rust builds a Decision request and uses+the caller's answer with local tool signals to select a model. The caller handles+classifier calls, completion requests, and branch state.Requires Node.js 22.19 or newer and the repository's Rust toolchain. The packageis private. Its native addon is built for the local host. Linux is covered in CI.@@ -17,10 +18,9 @@ npm --prefix bindings/typescript run typechecknpm --prefix bindings/typescript test```-The build uses Cargo's development profile. It copies the addon and license files-into the package. Rebuild after changing Rust code. Three loopback tests cover-model selection, safe errors, and cancellation. The Pi tests also exercise auto-routing with tool history.+The build uses Cargo's development profile and copies the addon and license files+into the package. Rebuild after changing Rust code. Tests use local decision+callbacks.Install the built package in a local Node project:@@ -31,80 +31,77 @@ npm install /absolute/path/to/Switchyard/bindings/typescript## API```ts-import { readFileSync } from "node:fs";-import { Runner } from "@switchyard/runner";+import { CompositeRouter, type DecisionRequest, type DecisionResponse } from "@switchyard/libsy";++const router = CompositeRouter.create({+ targets: {+ capable: "provider/large-model",+ efficient: "provider/small-model",+ classifier: "provider/decision-model",+ },+ classifier: {+ classify_trigger: "user_turn",+ fail_open: false,+ decision: {+ cutoff: 0.4,+ candidates: { a: "capable", b: "efficient" },+ evidence: { description: "a handles complex work; b handles routine work" },+ },+ },+ stage: { efficient_first_threshold: 0.5, capable_first_threshold: 0.2 },+});-const runner = Runner.fromToml(readFileSync("deployment.toml", "utf8"));+// Supply an adapter for your classifier provider.+declare function decide(request: DecisionRequest): Promise<DecisionResponse>;const controller = new AbortController();-const decision = await runner.decide("switchyard/auto", "Explain this bug", {- signal: controller.signal,-});-// decision: { target: "luna", model: "gpt-5.6-luna" }+const selection = await router.route([+ { role: "user", content: [{ type: "text", text: "Fix the parser" }] },+], { decide, signal: controller.signal });+// selection.model is a configured target model ID.```-`fromToml` synchronously loads and validates a version-1 runner deployment. Reuse-that runner for later decisions. `decide` takes a configured route **ID** and either-one user text prompt or an array of `RoutingMessage` values. It returns the selected-target name and configured model ID. Concurrent decisions can share a runner.+`create` synchronously validates the options. Reuse the router for concurrent+requests. `route` takes `RoutingMessage` values with text, tool calls, and tool+results. Pass the current branch's conversation on each call.-Use conversation messages for algorithms such as `auto` that inspect tool history.-The exported `RoutingMessage` and `RoutingContent` types cover text, tool calls,-and tool results in Switchyard's format:+`decide` receives protocol JSON with `context` and named `questions`. Questions+and answer values use `{ type, data }` tags. The router supplies packaged+instructions unless `classifier.decision.instructions` is set. A response looks+like this:```ts-const decision = await runner.decide("switchyard/auto", [- { role: "user", content: [{ type: "text", text: "Fix the tests" }] },- { role: "assistant", content: [{- type: "tool_call", id: "call-1", name: "bash", arguments: { command: "pytest" },- }] },- { role: "tool", content: [{- type: "tool_result", tool_call_id: "call-1", is_error: true,- content: [{ type: "text", text: "out of memory" }],- }] },-]);+const response: DecisionResponse = {+ answers: { route: {+ value: { type: "choice", data: {+ selected: "advantage",+ probabilities: { advantage: 0.6, no_advantage: 0.4 },+ } },+ } },+};```-Routing state is fresh for each call. Supply the current branch's conversation on-each decision. The API leaves session identity unset, so recovery signals come-from the transcript rather than cross-request capable holds.+The judge selects capable when `probabilities.advantage` exceeds `cutoff`.+Stage tool signals then select the final model. `user_turn` judges new user+messages; `new_session` judges when carried state has no tier.-`AbortSignal` cancels the Rust routing future, including pending classifier HTTP-work and retry waits. Cancellation rejects with `name: "AbortError"` and-`code: "ABORT_ERR"`. Other errors have one of these codes:+Store `selection.state` when present and pass it as `options.state` on the next+request on the same branch. Keep the previous state when the property is absent.+Treat `CompositeState` as opaque. It carries the judge's tier and stage holds.++## Errors and cancellation| Code | Meaning || --- | --- |-| `ERR_CONFIG` | Invalid or unsupported configuration, including missing classifier credentials |-| `ERR_UNKNOWN_ROUTE` | The requested route ID is absent |-| `ERR_INVALID_REQUEST` | Routing messages have an invalid shape |+| `ERR_CONFIG` | Invalid composite options |+| `ERR_INVALID_REQUEST` | Invalid messages, state, or decision response JSON || `ERR_ROUTING` | The algorithm or classifier call failed |-| `ERR_UNSUPPORTED_OUTCOME` | The outcome cannot be represented by a model choice |--Configuration errors preserve the runner's diagnostic message. TOML parse errors-include a line and column when available, with document excerpts kept in Rust.-Provider failures use safe summaries.--## Supported routes--Use `auto`, `passthrough` with its subagent policy unset, or `llm_classifier` in-capability mode. Auto uses local tool signals with an efficient-first default.-Capability mode supports both LLM judges and System One decision judges.-Use `classify_trigger = "every_request"` and `message_hash_fallback = false`-(their defaults). The host persists any chosen model or phase.--The host owns completion prompts, reasoning, and request options. Completion-targets must leave `system_prompt`, `reasoning_effort`, `extra_body`, and-`omit_body_fields` unset. Classifier targets can use their own settings. Clients-must leave `forward_auth` disabled.--Classifier API keys come from the existing `api_key_env` settings. The binding-requires no completion-provider credentials when those clients omit-`api_key_env`. The runner still constructs their clients during loading, so they-need valid HTTP(S) base URLs. For host-owned dispatch, use a credential-free-placeholder such as `http://127.0.0.1:1/v1`.+| `ABORT_ERR` | Routing was cancelled; the error name is `AbortError` |-Use the returned **target name** to look up the host's provider and model. Model-IDs alone can overlap across providers. The binding returns the selected target;-the host owns completion retries and fallback policy.+A rejected `decide` call follows `classifier.fail_open`. With `false`, routing+fails with the original error as `cause`. With `true`, routing keeps the previous+tier or uses the stage default. An answer without a usable advantage score also+keeps that fallback tier.-See [`examples/pi`](../../examples/pi/README.md) for a virtual-model extension.+An aborted signal closes the Rust run and rejects promptly, including while+`decide` is pending. Late callback results are ignored. Pass the same signal to+your classifier client to cancel its work too.diff --git a/bindings/typescript/package-lock.json b/bindings/typescript/package-lock.jsonindex d010a986d..bd8512aab 100644--- a/bindings/typescript/package-lock.json+++ b/bindings/typescript/package-lock.json@@ -1,11 +1,11 @@{- "name": "@switchyard/runner",+ "name": "@switchyard/libsy","version": "0.3.0","lockfileVersion": 3,"requires": true,"packages": {"": {- "name": "@switchyard/runner",+ "name": "@switchyard/libsy","version": "0.3.0","license": "Apache-2.0","devDependencies": {diff --git a/bindings/typescript/package.json b/bindings/typescript/package.jsonindex 640d99460..f631fea7d 100644--- a/bindings/typescript/package.json+++ b/bindings/typescript/package.json@@ -1,8 +1,8 @@{- "name": "@switchyard/runner",+ "name": "@switchyard/libsy","version": "0.3.0","private": true,- "description": "Model-selection bindings for Switchyard",+ "description": "Composite routing bindings for Switchyard","license": "Apache-2.0","type": "module","engines": { "node": ">=22.19.0" },diff --git a/bindings/typescript/src/index.ts b/bindings/typescript/src/index.tsindex 42a59284f..e79bd4740 100644--- a/bindings/typescript/src/index.ts+++ b/bindings/typescript/src/index.ts@@ -3,9 +3,28 @@import { createRequire } from "node:module";-export interface Decision {- target: string;+/** Opaque routing state. Store it unchanged and pass it back on the same branch. */+export type CompositeState = { readonly [key: string]: unknown };++export interface CompositeOptions {+ targets: { capable: string; efficient: string; classifier: string };+ classifier: {+ classify_trigger: "user_turn" | "new_session";+ fail_open: boolean;+ decision: {+ cutoff: number;+ candidates: Record<string, "capable" | "efficient">;+ evidence: unknown;+ instructions?: unknown;+ };+ };+ stage: { efficient_first_threshold: number; capable_first_threshold: number };+}++export interface Selection {model: string;+ /** Present only when the state changed. */+ state?: CompositeState;}/** Text and tool activity used for routing, in Switchyard's message format. */@@ -19,69 +38,148 @@ export interface RoutingMessage {content: readonly RoutingContent[];}-interface Cancellation {- cancel(): void;+export interface DecisionRequest {+ model?: string | null;+ context: unknown;+ questions: Record<string, DecisionQuestion>;+}++export interface DecisionQuestion {+ instructions: unknown;+ kind:+ | { type: "boolean"; data: { true_description?: unknown; false_description?: unknown } }+ | { type: "choice"; data: { options: { id: string; description?: unknown }[] } }+ | { type: "score"; data: { levels: unknown[] } };+}++export interface DecisionResponse {+ id?: string | null;+ model?: string | null;+ answers: Record<string, DecisionAnswer>;+ usage?: {+ input_tokens?: number | null;+ cached_input_tokens?: number | null;+ cache_creation_input_tokens?: number | null;+ output_tokens?: number | null;+ total_tokens?: number | null;+ reasoning_tokens?: number | null;+ };+}++export interface DecisionAnswer {+ value:+ | { type: "boolean"; data: { type: "value"; data: boolean } | { type: "probability_true"; data: number } }+ | { type: "choice"; data: { selected: string; probabilities?: Record<string, number> | null } }+ | { type: "score"; data: { value: number; probabilities?: number[] | null } };+ provider_confidence?: number | null;+}++type NativeStep =+ | { kind: "decision"; request: string }+ | { kind: "done"; model: string; state: string | null };++interface NativeRun {+ next(response: string | null, error: string | null): Promise<NativeStep>;+ close(): void;}-interface NativeRunner {- decide(routeId: string, messages: string, cancellation: Cancellation): Promise<Decision>;+interface NativeComposite {+ start(messages: string, state: string | null): NativeRun;}-const native: {- NativeRunner: new (source: string) => NativeRunner;- Cancellation: new () => Cancellation;-} = createRequire(import.meta.url)("./switchyard.node");+const native: { NativeComposite: new (options: string) => NativeComposite } =+ createRequire(import.meta.url)("./switchyard.node");function abortError(): Error {return Object.assign(new Error("Routing cancelled"), { name: "AbortError", code: "ABORT_ERR" });}-function publicError(cause: unknown): Error {- const message = cause instanceof Error ? cause.message : String(cause);- const match = /^(ERR_CONFIG|ERR_UNKNOWN_ROUTE|ERR_INVALID_REQUEST|ERR_ROUTING|ERR_UNSUPPORTED_OUTCOME|ABORT_ERR): (.*)$/s.exec(message);+function errorMessage(error: unknown): string {+ return error instanceof Error ? error.message : String(error);+}++function publicError(error: unknown, cause?: unknown): Error {+ const match = /^(ERR_CONFIG|ERR_INVALID_REQUEST|ERR_ROUTING|ABORT_ERR): (.*)$/s.exec(errorMessage(error));if (match?.[1] === "ABORT_ERR") return abortError();- return Object.assign(new Error(match?.[2] ?? "Routing failed"), {- code: match?.[1] ?? "ERR_ROUTING",- });+ const code = match?.[1] ?? "ERR_ROUTING";+ return Object.assign(new Error(match?.[2] ?? errorMessage(error),code === "ERR_ROUTING" ? { cause } : undefined), { code });+}++function json(value: unknown, code = "ERR_INVALID_REQUEST"): string {+ try {+ const result = JSON.stringify(value);+ if (result === undefined) throw new Error("Expected a JSON value");+ return result;+ } catch (error) {+ throw new Error(`${code}: ${errorMessage(error)}`);+ }}-/** A reusable model selector. The caller owns completion dispatch and session state. */-export class Runner {- readonly #native: NativeRunner;+/** Reusable composite router. The caller performs decision calls and stores branch state. */+export class CompositeRouter {+ readonly #native: NativeComposite;- private constructor(source: string) {- this.#native = new native.NativeRunner(source);+ private constructor(options: CompositeOptions) {+ this.#native = new native.NativeComposite(json(options, "ERR_CONFIG"));}- /** Loads a model-selection deployment. Classifier credentials come from its environment settings. */- static fromToml(source: string): Runner {+ static create(options: CompositeOptions): CompositeRouter {try {- return new Runner(source);+ return new CompositeRouter(options);} catch (error) {throw publicError(error);}}- /** Selects a target from a conversation or a single user prompt. */- async decide(routeId: string, input: string | readonly RoutingMessage[], options: { signal?: AbortSignal } = {}): Promise<Decision> {- const signal = options.signal;+ async route(messages: readonly RoutingMessage[], options: {+ state?: CompositeState;+ decide(request: DecisionRequest): Promise<DecisionResponse>;+ signal?: AbortSignal;+ }): Promise<Selection> {+ const { signal } = options;if (signal?.aborted) throw abortError();- const cancellation = new native.Cancellation();- const onAbort = () => cancellation.cancel();- signal?.addEventListener("abort", onAbort, { once: true });+ let run: NativeRun | undefined;+ let decisionError: unknown;+ let onAbort!: () => void;+ const aborted = new Promise<never>((_, reject) => {+ onAbort = () => {+ run?.close();+ reject(abortError());+ };+ });try {- if (signal?.aborted) throw abortError();- const messages: readonly RoutingMessage[] = typeof input === "string"- ? [{ role: "user", content: [{ type: "text", text: input }] }]- : input;- const result = await this.#native.decide(routeId, JSON.stringify(messages), cancellation);- if (signal?.aborted) throw abortError();- return result;+ run = this.#native.start(json(messages), options.state === undefined ? null : json(options.state));+ signal?.addEventListener("abort", onAbort, { once: true });+ let response: string | null = null;+ let failure: string | null = null;+ for (;;) {+ const step = await Promise.race([run.next(response, failure), aborted]);+ if (signal?.aborted) throw abortError();+ if (step.kind === "done") {+ return step.state === null+ ? { model: step.model }+ : { model: step.model, state: JSON.parse(step.state) };+ }+ response = null;+ failure = null;+ let answer: DecisionResponse;+ try {+ answer = await Promise.race([options.decide(JSON.parse(step.request)), aborted]);+ } catch (error) {+ if (signal?.aborted) throw abortError();+ decisionError = error;+ failure = errorMessage(error);+ continue;+ }+ if (signal?.aborted) throw abortError();+ response = json(answer);+ }} catch (error) {if (signal?.aborted) throw abortError();- throw publicError(error);+ throw publicError(error, decisionError);} finally {signal?.removeEventListener("abort", onAbort);+ run?.close();}}}diff --git a/bindings/typescript/test/classifier.toml b/bindings/typescript/test/classifier.tomldeleted file mode 100644index 97cb97f05..000000000--- a/bindings/typescript/test/classifier.toml+++ /dev/null@@ -1,42 +0,0 @@-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-# SPDX-License-Identifier: Apache-2.0--schema_version = 1--[decision_clients.classifier]-format = "system_one"-endpoint = "http://127.0.0.1:1/classify"-api_key_env = "SWITCHYARD_TEST_KEY"-timeout_ms = 10000--[decision_targets.judge]-id = "jev-latest"-decision_client = "classifier"--# The caller owns completion dispatch.-[llm_clients.pi]-format = "openai_chat"-base_url = "http://127.0.0.1:1/v1"--[targets.complex]-id = "gpt-5.6-sol"-llm_client = "pi"--[targets.standard]-id = "gpt-5.6-terra"-llm_client = "pi"--[routes.planner]-id = "switchyard/planner"-type = "llm_classifier"-mode = "capability"-classifier_target = "judge"-strong_target = "complex"-weak_target = "standard"-classify_trigger = "every_request"-fail_open = false--[routes.planner.decision]-cutoff = 0.5-candidates = { a = "complex", b = "standard" }-evidence = { candidates = { a = "More capable software engineering model for difficult design and debugging", b = "Lower-cost planning model for ordinary software engineering tasks" } }diff --git a/bindings/typescript/test/composite.test.mjs b/bindings/typescript/test/composite.test.mjsnew file mode 100644index 000000000..7d6a68252--- /dev/null+++ b/bindings/typescript/test/composite.test.mjs@@ -0,0 +1,135 @@+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.+// SPDX-License-Identifier: Apache-2.0++import assert from "node:assert/strict";+import { getEventListeners } from "node:events";+import { readFileSync } from "node:fs";+import { test } from "node:test";+import { CompositeRouter } from "../dist/index.js";++const options = {+ targets: { capable: "provider/strong", efficient: "provider/weak", classifier: "provider/judge" },+ classifier: {+ classify_trigger: "user_turn",+ fail_open: false,+ decision: {+ cutoff: 0.4,+ candidates: { a: "capable", b: "efficient" },+ evidence: { description: "a handles complex work; b handles routine work" },+ },+ },+ stage: { efficient_first_threshold: 0.5, capable_first_threshold: 0.2 },+};+const messages = [{ role: "user", content: [{ type: "text", text: "Fix the parser" }] }];+const continuation = [+ ...messages,+ { role: "assistant", content: [{ type: "tool_call", id: "read-1", name: "read", arguments: { path: "parser.rs" } }] },+ { role: "tool", content: [{ type: "tool_result", tool_call_id: "read-1", is_error: false, content: [{ type: "text", text: "fn parse() {}" }] }] },+];+const instructions = JSON.parse(readFileSync(new URL(+ "../../../crates/libsy/src/prompts/capability-classifier/relative_advantage.json", import.meta.url,+), "utf8"));+const unexpectedDecision = async () => assert.fail("unexpected decision call");++function response(advantage) {+ return { answers: { route: { value: {+ type: "choice", data: { selected: "advantage", probabilities: { advantage, no_advantage: 1 - advantage } },+ } } } };+}++test("decision request, cutoff, and branch-owned state", async () => {+ const router = CompositeRouter.create(options);+ const states = await Promise.all([0.41, 0.4].map(async (advantage) => {+ let calls = 0;+ const selected = await router.route(messages, { decide: async (request) => {+ calls++;+ assert.equal(request.model, options.targets.classifier);+ assert.deepEqual(request.context, {+ task: messages, candidates: ["a", "b"], comparison: { capable: "a", efficient: "b" },+ evidence: options.classifier.decision.evidence,+ });+ assert.deepEqual(Object.keys(request.questions), ["route"]);+ assert.deepEqual(request.questions.route.instructions, instructions);+ assert.equal(request.questions.route.kind.type, "choice");+ assert.deepEqual(request.questions.route.kind.data.options.map((option) => option.id), ["advantage", "no_advantage"]);+ return response(advantage);+ } });+ assert.equal(calls, 1);+ assert.equal(selected.model, advantage > 0.4 ? options.targets.capable : options.targets.efficient);+ assert.ok(selected.state);+ return selected;+ }));+ for (const { model, state } of states) {+ const saved = structuredClone(state);+ assert.deepEqual(await router.route(continuation, { state, decide: unexpectedDecision }), { model });+ assert.deepEqual(state, saved);+ }+ assert.deepEqual(await router.route(continuation, { decide: unexpectedDecision }), { model: options.targets.efficient });++ const once = structuredClone(options);+ once.classifier.classify_trigger = "new_session";+ const sessionRouter = CompositeRouter.create(once);+ const { state } = await sessionRouter.route(messages, { decide: async () => response(0.41) });+ assert.deepEqual(await sessionRouter.route(messages, { state, decide: unexpectedDecision }), { model: options.targets.capable });+});++test("validation and decision failure policy", async () => {+ for (const change of [+ (config) => { config.targets.efficient = config.targets.capable; },+ (config) => { config.classifier.decision.cutoff = 2; },+ (config) => { config.classifier.decision.extra = true; },+ (config) => { config.classifier.classify_trigger = "every_request"; },+ ]) {+ const config = structuredClone(options);+ change(config);+ assert.throws(() => CompositeRouter.create(config), { code: "ERR_CONFIG" });+ }+ const router = CompositeRouter.create(options);+ await assert.rejects(router.route([{ role: "invalid", content: [] }], { decide: unexpectedDecision }), { code: "ERR_INVALID_REQUEST" });+ await assert.rejects(router.route(messages, { state: { fall_open: "strong" }, decide: unexpectedDecision }), { code: "ERR_INVALID_REQUEST" });+ for (const answer of [{}, undefined, { answers: 1n }]) {+ await assert.rejects(router.route(messages, { decide: async () => answer }), { code: "ERR_INVALID_REQUEST" });+ }++ const cause = new Error("classifier unavailable");+ const decide = async () => { throw cause; };+ await assert.rejects(router.route(messages, { decide }), (error) => {+ assert.equal(error.code, "ERR_ROUTING");+ assert.match(error.message, /classifier unavailable/);+ assert.equal(error.cause, cause);+ return true;+ });+ const config = structuredClone(options);+ config.classifier.fail_open = true;+ const fallback = CompositeRouter.create(config);+ const { state } = await fallback.route(messages, { decide: async () => response(0.41) });+ assert.deepEqual(await fallback.route(messages, { state, decide }), { model: options.targets.capable });+ assert.deepEqual(await fallback.route(messages, { decide }), { model: options.targets.efficient });+ assert.deepEqual(await router.route(messages, { state, decide: async () => ({ answers: {} }) }), { model: options.targets.capable });+});++test("abort settles promptly, ignores late answers, and removes listeners", { timeout: 5000 }, async () => {+ const router = CompositeRouter.create(options);+ await assert.rejects(router.route(messages, {+ signal: AbortSignal.abort(), decide: unexpectedDecision,+ }), { name: "AbortError", code: "ABORT_ERR" });++ for (const timing of ["native", "resolve", "reject"]) {+ const entered = Promise.withResolvers();+ const pending = Promise.withResolvers();+ const controller = new AbortController();+ const rejected = assert.rejects(router.route(messages, {+ signal: controller.signal,+ decide: () => { entered.resolve(); return pending.promise; },+ }), { name: "AbortError", code: "ABORT_ERR" });+ if (timing !== "native") await entered.promise;+ controller.abort();+ await rejected;+ assert.equal(getEventListeners(controller.signal, "abort").length, 0);+ if (timing === "reject") pending.reject(new Error("late rejection"));+ else pending.resolve(response(0.41));+ }+ const controller = new AbortController();+ await router.route(messages, { signal: controller.signal, decide: async () => response(0.4) });+ assert.equal(getEventListeners(controller.signal, "abort").length, 0);+});diff --git a/bindings/typescript/test/runner.test.mjs b/bindings/typescript/test/runner.test.mjsdeleted file mode 100644index 9c00f2390..000000000--- a/bindings/typescript/test/runner.test.mjs+++ /dev/null@@ -1,91 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--import assert from "node:assert/strict";-import { getEventListeners, once } from "node:events";-import { readFileSync } from "node:fs";-import { createServer } from "node:http";-import { test } from "node:test";-import { Runner } from "../dist/index.js";--const config = readFileSync(new URL("./classifier.toml", import.meta.url), "utf8");--async function setup(t, handler) {- const calls = [];- const http = createServer(async (req, res) => {- let body = "";- for await (const chunk of req) body += chunk;- calls.push({ url: req.url, body: JSON.parse(body) });- handler(res);- });- http.listen(0, "127.0.0.1");- await once(http, "listening");- t.after(() => {- http.closeAllConnections();- return new Promise((resolve) => http.close(resolve));- });- const previous = process.env.SWITCHYARD_TEST_KEY;- process.env.SWITCHYARD_TEST_KEY = "secret-sentinel";- t.after(() => {- if (previous === undefined) delete process.env.SWITCHYARD_TEST_KEY;- else process.env.SWITCHYARD_TEST_KEY = previous;- });- const source = config.replace("http://127.0.0.1:1/classify", `http://127.0.0.1:${http.address().port}/classify`);- return { runner: Runner.fromToml(source), calls };-}--test("selects the configured target through the classifier", async (t) => {- const { runner, calls } = await setup(t, (res) => {- res.setHeader("content-type", "application/json");- res.end(JSON.stringify({ answers: { route: {- type: "choice", choice: "advantage", probabilities: { advantage: 0.9, no_advantage: 0.1 },- } } }));- });- assert.equal(calls.length, 0);- assert.deepEqual(await runner.decide("switchyard/planner", "hard task"), {- target: "complex", model: "gpt-5.6-sol",- });- assert.equal(calls.length, 1);- assert.equal(calls[0].url, "/classify");- assert.deepEqual(calls[0].body.state.task[0].content, [{ type: "text", text: "hard task" }]);-});--test("reports safe configuration, route, and classifier errors", async (t) => {- assert.throws(() => Runner.fromToml("secret-sentinel = ["), (error) => {- assert.equal(error.code, "ERR_CONFIG");- assert.match(error.message, /failed to parse TOML at line 1, column \d+/);- assert.doesNotMatch(error.message, /secret-sentinel/);- return true;- });- assert.throws(- () => Runner.fromToml(config.replace('classify_trigger = "every_request"', 'classify_trigger = "new_session"')),- { code: "ERR_CONFIG", message: /route planner:.*every_request/ },- );- const { runner } = await setup(t, (res) => { res.writeHead(401); res.end("secret-sentinel"); });- await assert.rejects(runner.decide("unknown", "hello"), { code: "ERR_UNKNOWN_ROUTE" });- await assert.rejects(runner.decide("switchyard/planner", [{ role: "invalid", content: [] }]), { code: "ERR_INVALID_REQUEST" });- await assert.rejects(runner.decide("switchyard/planner", "hello"), (error) => {- assert.equal(error.code, "ERR_ROUTING");- assert.doesNotMatch(error.message, /secret-sentinel/);- return true;- });-});--test("abort cancels the native HTTP request and releases its listener", { timeout: 5000 }, async (t) => {- const received = Promise.withResolvers();- const closed = Promise.withResolvers();- const { runner } = await setup(t, (res) => {- res.on("close", closed.resolve);- received.resolve();- });- const controller = new AbortController();- const rejected = assert.rejects(- runner.decide("switchyard/planner", "hello", { signal: controller.signal }),- { name: "AbortError", code: "ABORT_ERR" },- );- await received.promise;- controller.abort();- await rejected;- await closed.promise;- assert.equal(getEventListeners(controller.signal, "abort").length, 0);-});diff --git a/crates/switchyard-node/Cargo.toml b/crates/switchyard-node/Cargo.tomlindex d8990a6e2..e008c132f 100644--- a/crates/switchyard-node/Cargo.toml+++ b/crates/switchyard-node/Cargo.toml@@ -9,7 +9,7 @@ rust-version.workspace = truelicense.workspace = trueauthors.workspace = truerepository.workspace = true-description = "Node.js model-selection bindings for Switchyard"+description = "Node.js composite routing bindings for Switchyard"publish = false[lib]@@ -18,11 +18,12 @@ crate-type = ["cdylib"][dependencies]napi = { version = "3", default-features = false, features = ["napi8", "tokio_rt", "dyn-symbols"] }napi-derive = "3"+futures.workspace = true+parking_lot.workspace = true+serde.workspace = trueserde_json.workspace = true-switchyard-runner.workspace = true+switchyard-libsy.workspace = trueswitchyard-protocol.workspace = true-tokio.workspace = true-tokio-util = { version = "0.7", features = ["rt"] }[build-dependencies]napi-build = "2"diff --git a/crates/switchyard-node/src/lib.rs b/crates/switchyard-node/src/lib.rsindex a49a15a33..77d8408fa 100644--- a/crates/switchyard-node/src/lib.rs+++ b/crates/switchyard-node/src/lib.rs@@ -1,114 +1,320 @@// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.// SPDX-License-Identifier: Apache-2.0-//! Node.js bindings for caller-owned completion dispatch and routing state.+//! Composite routing with caller-owned decision calls and state.+use std::collections::BTreeMap;+use std::future::ready;+use std::sync::Arc;++use futures::StreamExt;+use futures::future::{AbortHandle, Abortable};use napi_derive::napi;-use switchyard_protocol::{LlmRequest, ModelId, Request};-use switchyard_runner::{Runner, RunnerError};-use tokio_util::sync::CancellationToken;+use parking_lot::Mutex;+use serde::Deserialize;+use serde_json::Value;+use switchyard_libsy::{+ Algorithm, CallDecision, CapabilityJudgeConfig, ClassifyTrigger, CompositeJudgeConfig,+ CompositeRouter, CompositeRouterConfig, DecisionJudgeConfig, LibsyError, PickerMode,+ RuntimeModels, StageRouterConfig, State, Step, StepStream, TaskClassifierConfig,+};+use switchyard_protocol::{+ Category, DecisionResponse, LlmClientError, LlmRequest, ModelId, Request,+};-// The JS wrapper turns these fixed prefixes into public Error codes.-fn error(code: &str, message: &str) -> napi::Error {+// The JS wrapper turns these prefixes into public Error codes.+fn error(code: &str, message: impl std::fmt::Display) -> napi::Error {napi::Error::from_reason(format!("{code}: {message}"))}+fn routing_error(message: impl std::fmt::Display) -> napi::Error {+ error("ERR_ROUTING", message)+}++fn parse<T: serde::de::DeserializeOwned>(json: &str) -> napi::Result<T> {+ serde_json::from_str(json).map_err(|err| error("ERR_INVALID_REQUEST", err))+}++#[derive(Deserialize)]+#[serde(deny_unknown_fields)]+struct Options {+ targets: Targets,+ classifier: ClassifierOptions,+ stage: StageOptions,+}++#[derive(Deserialize)]+#[serde(deny_unknown_fields)]+struct Targets {+ capable: ModelId,+ efficient: ModelId,+ classifier: ModelId,+}++#[derive(Deserialize)]+#[serde(deny_unknown_fields)]+struct ClassifierOptions {+ classify_trigger: ClassifyTrigger,+ fail_open: bool,+ decision: DecisionOptions,+}++#[derive(Deserialize)]+#[serde(deny_unknown_fields)]+struct DecisionOptions {+ cutoff: f64,+ candidates: BTreeMap<String, Candidate>,+ evidence: Value,+ instructions: Option<Value>,+}++#[derive(Deserialize)]+#[serde(rename_all = "snake_case")]+enum Candidate {+ Capable,+ Efficient,+}++#[derive(Deserialize)]+#[serde(deny_unknown_fields)]+struct StageOptions {+ efficient_first_threshold: f64,+ capable_first_threshold: f64,+}+#[napi]-#[derive(Default)]-pub struct Cancellation {- token: CancellationToken,+pub struct NativeComposite {+ router: Arc<CompositeRouter>,+ models: Arc<RuntimeModels>,}#[napi]-impl Cancellation {+impl NativeComposite {#[napi(constructor)]- pub fn new() -> Self {- Self::default()+ pub fn new(options_json: String) -> napi::Result<Self> {+ let Options {+ targets,+ classifier,+ stage,+ } = serde_json::from_str(&options_json).map_err(|err| error("ERR_CONFIG", err))?;+ if targets.capable == targets.efficient {+ return Err(error(+ "ERR_CONFIG",+ "capable and efficient targets must differ",+ ));+ }+ let decision = classifier.decision;+ let candidates = decision+ .candidates+ .into_iter()+ .map(|(id, tier)| {+ let model = match tier {+ Candidate::Capable => &targets.capable,+ Candidate::Efficient => &targets.efficient,+ };+ (id, model.clone())+ })+ .collect();+ let mut stage_config =+ StageRouterConfig::new(PickerMode::EfficientFirst, stage.efficient_first_threshold);+ stage_config.capable_first_threshold = stage.capable_first_threshold;+ let router = CompositeRouter::new(CompositeRouterConfig {+ judge: CompositeJudgeConfig::Capability(TaskClassifierConfig {+ judge: CapabilityJudgeConfig::Decision(DecisionJudgeConfig {+ cutoff: decision.cutoff,+ candidates,+ evidence: decision.evidence,+ instructions: decision.instructions,+ }),+ classify_trigger: classifier.classify_trigger,+ fail_open: classifier.fail_open,+ ..Default::default()+ }),+ stage: stage_config,+ })+ .map_err(|err| error("ERR_CONFIG", err))?;+ let models = RuntimeModels::new(+ [+ (Category::Judge, vec![targets.classifier]),+ (Category::Capable, vec![targets.capable.clone()]),+ (Category::Efficient, vec![targets.efficient.clone()]),+ (Category::Any, vec![targets.capable, targets.efficient]),+ ]+ .into(),+ );+ Ok(Self {+ router: Arc::new(router),+ models: Arc::new(models),+ })}#[napi]- pub fn cancel(&self) {- self.token.cancel();+ pub fn start(+ &self,+ messages_json: String,+ state_json: Option<String>,+ ) -> napi::Result<NativeRun> {+ let request = Request {+ llm_request: LlmRequest {+ model: Some("switchyard/auto".into()),+ messages: parse(&messages_json)?,+ ..Default::default()+ },+ ..Default::default()+ };+ let initial_state: State = state_json+ .as_deref()+ .map(parse)+ .transpose()?+ .unwrap_or_default();+ let state = initial_state.clone();+ let router = self.router.clone();+ let models = self.models.clone();+ // Defer run_stream until next() polls on napi's Tokio runtime.+ let stream =+ futures::stream::once(async move { router.run_stream(request, models, state) })+ .flatten();+ Ok(NativeRun {+ slot: Mutex::new(RunSlot {+ run: Some(Run {+ initial_state,+ stream: Box::pin(stream),+ pending: None,+ }),+ abort: None,+ }),+ })}}-#[napi(object)]-pub struct Decision {- pub target: String,- pub model: String,+#[napi(object, use_nullable = true)]+#[derive(Default)]+pub struct NativeStep {+ pub kind: String,+ pub request: Option<String>,+ pub model: Option<String>,+ pub state: Option<String>,}-#[napi]-pub struct NativeRunner {- runner: Runner,+struct Run {+ initial_state: State,+ stream: StepStream,+ pending: Option<Box<CallDecision>>,}-#[napi]-impl NativeRunner {- #[napi(constructor)]- pub fn new(source: String) -> napi::Result<Self> {- let runner = Runner::from_toml_for_model_selection(&source).map_err(|err| match err {- RunnerError::Configuration { message, .. } => error("ERR_CONFIG", &message),- _ => error("ERR_CONFIG", "Invalid model-selection configuration"),- })?;- Ok(Self { runner })+impl Run {+ async fn next(+ &mut self,+ response: Option<String>,+ failure: Option<String>,+ ) -> napi::Result<NativeStep> {+ if let Some(call) = self.pending.take() {+ let result = match (response, failure) {+ (Some(json), None) => Ok(parse::<DecisionResponse>(&json)?),+ (None, Some(message)) => Err(LibsyError::client_call(+ call.model.clone(),+ LlmClientError::General(message),+ )),+ _ => {+ return Err(error(+ "ERR_INVALID_REQUEST",+ "Expected a decision response or error",+ ));+ }+ };+ call.respond(ready(result)).await.map_err(routing_error)?;+ } else if response.is_some() || failure.is_some() {+ return Err(error("ERR_INVALID_REQUEST", "No decision is pending"));+ }+ let step = self+ .stream+ .next()+ .await+ .ok_or_else(|| error("ERR_ROUTING", "Routing ended without an outcome"))?+ .map_err(routing_error)?;+ let mut result = NativeStep::default();+ match step {+ Step::CallDecision(call) => {+ result.kind = "decision".into();+ result.request = Some(serde_json::to_string(&call.request).map_err(routing_error)?);+ self.pending = Some(call);+ }+ Step::Done(outcome) => {+ result.kind = "done".into();+ result.model = Some(+ outcome+ .selected_model_id()+ .map_err(routing_error)?+ .to_string(),+ );+ if outcome.state != self.initial_state {+ result.state =+ Some(serde_json::to_string(&outcome.state).map_err(routing_error)?);+ }+ }+ Step::CallModel(_) => return Err(error("ERR_ROUTING", "Unexpected model call")),+ }+ Ok(result)}+}++struct RunSlot {+ run: Option<Run>,+ abort: Option<AbortHandle>,+}++#[napi]+pub struct NativeRun {+ slot: Mutex<RunSlot>,+}+#[napi]+impl NativeRun {#[napi]- pub async fn decide(+ pub async fn next(&self,- route_id: String,- messages: String,- cancellation: &Cancellation,- ) -> napi::Result<Decision> {- // napi-rs retains both JS objects while this async method borrows them.- tokio::select! {- biased;- _ = cancellation.token.cancelled() => Err(error("ABORT_ERR", "Routing cancelled")),- result = select_model(&self.runner, route_id, messages) => result,+ response_json: Option<String>,+ error_message: Option<String>,+ ) -> napi::Result<NativeStep> {+ let (abort, registration) = AbortHandle::new_pair();+ let mut run = {+ let mut slot = self.slot.lock();+ let run = slot.run.take().ok_or_else(|| {+ error(+ "ERR_INVALID_REQUEST",+ "Run is closed or next is already pending",+ )+ })?;+ slot.abort = Some(abort.clone());+ run+ };+ // Move ownership into the future so close() also drops an in-flight stream.+ let (run, result) = Abortable::new(+ async move {+ let result = run.next(response_json, error_message).await;+ (run, result)+ },+ registration,+ )+ .await+ .map_err(|_| error("ABORT_ERR", "Routing cancelled"))?;+ let mut slot = self.slot.lock();+ if abort.is_aborted() {+ return Err(error("ABORT_ERR", "Routing cancelled"));+ }+ slot.abort = None;+ if result.as_ref().is_ok_and(|step| step.kind == "decision") {+ slot.run = Some(run);}+ result}-}-async fn select_model(- runner: &Runner,- route_id: String,- messages: String,-) -> napi::Result<Decision> {- let route = runner- .route(&route_id)- .ok_or_else(|| error("ERR_UNKNOWN_ROUTE", "Unknown route ID"))?;- let request = Request {- llm_request: LlmRequest {- model: Some(route_id.clone()),- messages: serde_json::from_str(&messages)- .map_err(|_| error("ERR_INVALID_REQUEST", "Invalid routing messages"))?,- ..Default::default()- },- ..Default::default()- };- let mut expected = request.llm_request.clone();- let outcome = route- .decide(request)- .await- .map_err(|err| error("ERR_ROUTING", err.execution_error_summary().kind.as_str()))?;- expected.model = outcome.request.llm_request.model.clone();- if outcome.response.is_some() || outcome.request.llm_request != expected {- return Err(error(- "ERR_UNSUPPORTED_OUTCOME",- "Routing changed the completion request or produced an answer",- ));+ #[napi]+ pub fn close(&self) {+ let mut slot = self.slot.lock();+ if let Some(abort) = slot.abort.take() {+ abort.abort();+ }+ slot.run = None;}- let decision = runner- .describe_decision(&ModelId::from(route_id), &outcome)- .ok_or_else(|| {- error(- "ERR_UNSUPPORTED_OUTCOME",- "Routing selected an unresolved target",- )- })?;- Ok(Decision {- target: decision.selected.target,- model: decision.selected.model.to_string(),- })}diff --git a/crates/switchyard-runner/src/algorithm.rs b/crates/switchyard-runner/src/algorithm.rsindex a59eb2e3c..67736cd29 100644--- a/crates/switchyard-runner/src/algorithm.rs+++ b/crates/switchyard-runner/src/algorithm.rs@@ -733,21 +733,6 @@ impl StageClassifierConfig {}impl AlgorithmSpec {- /// Whether this configuration only selects a model, with state owned by the caller.- pub(crate) fn supports_model_selection(&self) -> bool {- match self {- Self::Passthrough {- subagents: None, ..- }- | Self::Auto { .. } => true,- Self::LlmClassifier { config } => {- matches!(config.classifier_mode(), ClassifierMode::Capability)- && config.classify_trigger == ClassifyTrigger::EveryRequest- }- _ => false,- }- }-pub(crate) fn decision_judge(&self) -> Option<(&str, &DecisionJudgeRouteConfig)> {match self {Self::LlmClassifier { config, .. } => configdiff --git a/crates/switchyard-runner/src/config.rs b/crates/switchyard-runner/src/config.rsindex 1b4b2fc8a..ddd3812ab 100644--- a/crates/switchyard-runner/src/config.rs+++ b/crates/switchyard-runner/src/config.rs@@ -85,27 +85,6 @@ pub(crate) fn runner_from_toml(source: &str) -> RunnerResult<Runner> {config.build()}-pub(crate) fn model_selection_from_toml(source: &str) -> RunnerResult<Runner> {- let config: DeploymentConfig = toml::from_str(source).map_err(|error: toml::de::Error| {- // Report the location while keeping document excerpts in the source error.- let mut message = "failed to parse TOML".to_string();- if let Some(prefix) = error.span().and_then(|span| source.get(..span.start)) {- let line = prefix.bytes().filter(|byte| *byte == b'\n').count() + 1;- let column = prefix- .rsplit('\n')- .next()- .unwrap_or_default()- .chars()- .count()- + 1;- message = format!("{message} at line {line}, column {column}");- }- RunnerError::configuration_source(message, error)- })?;- config.validate_model_selection()?;- config.build()-}-#[derive(Debug, Deserialize)]#[serde(deny_unknown_fields)]pub(crate) struct DeploymentConfig {@@ -200,36 +179,6 @@ impl RouteConfig {}impl DeploymentConfig {- fn validate_model_selection(&self) -> RunnerResult<()> {- for (name, client) in &self.llm_clients {- if client.forward_auth {- return Err(RunnerError::configuration(format!(- "llm client {name}: model selection requires configured classifier credentials"- )));- }- }- for (name, route) in &self.routes {- if !route.algorithm.supports_model_selection() {- return Err(RunnerError::configuration(format!(- "route {name}: model selection requires auto, passthrough, or capability llm_classifier with every_request"- )));- }- for target_name in route.routing_target_names() {- if let Some(target) = self.targets.get(target_name)- && (target.system_prompt.is_some()- || target.reasoning_effort.is_some()- || !target.extra_body.is_empty()- || !target.omit_body_fields.is_empty())- {- return Err(RunnerError::configuration(format!(- "target {target_name}: model selection requires completion settings to be owned by the caller"- )));- }- }- }- Ok(())- }-fn decision_target(&self, name: &str) -> Option<DecisionTarget> {let target = self.targets.get(name)?;let client = self.llm_clients.get(&target.llm_client)?;diff --git a/crates/switchyard-runner/src/runner.rs b/crates/switchyard-runner/src/runner.rsindex e71fe204c..f69712c29 100644--- a/crates/switchyard-runner/src/runner.rs+++ b/crates/switchyard-runner/src/runner.rs@@ -66,15 +66,6 @@ impl Runner {config::runner_from_toml(source)}- /// Loads routes whose only completion-side effect is model selection.- ///- /// Accepts auto, passthrough, and capability classifiers with per-request decisions.- /// Completion request overrides and forwarded authentication are rejected;- /// the caller owns completion dispatch and session state.- pub fn from_toml_for_model_selection(source: &str) -> Result<Self, RunnerError> {- config::model_selection_from_toml(source)- }-/// Builds a runner from named routes in caller-provided order./// Pre-condition: There must be at least one route.pub fn new(routes: Vec<(ModelId, Route)>) -> Self {diff --git a/crates/switchyard-runner/tests/model_selection.rs b/crates/switchyard-runner/tests/model_selection.rsdeleted file mode 100644index 02a23a80b..000000000--- a/crates/switchyard-runner/tests/model_selection.rs+++ /dev/null@@ -1,114 +0,0 @@-// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.-// SPDX-License-Identifier: Apache-2.0--use async_trait::async_trait;-use std::sync::Arc;-use switchyard_protocol::{- LlmClientError, LlmResponse, Request, Response, RoutedLlmClient, text_request, text_response,-};-use switchyard_runner::Runner;--const CONFIG: &str = r#"-schema_version = 1-[llm_clients.local]-format = "openai_chat"-base_url = "http://127.0.0.1:1/v1"-[targets.fast]-id = "fast-model"-llm_client = "local"-[targets.strong]-id = "strong-model"-llm_client = "local"-[routes.test]-id = "route-id"-type = "llm_classifier"-classifier_target = "strong"-strong_target = "strong"-weak_target = "fast"-base_threshold = 0.5-"#;--#[test]-fn accepts_per_request_classification_and_rejects_session_state() {- assert!(Runner::from_toml_for_model_selection(CONFIG).is_ok());- let stateful = format!("{CONFIG}\nclassify_trigger = 'new_session'");- assert!(Runner::from_toml(&stateful).is_ok());- assert!(Runner::from_toml_for_model_selection(&stateful).is_err());-- let invalid = format!("{CONFIG}\nmessage_hash_fallback = true");- assert!(Runner::from_toml(&invalid).is_err());- assert!(Runner::from_toml_for_model_selection(&invalid).is_err());-}--#[test]-fn rejects_answer_producing_algorithms() {- let config = CONFIG.replace(- "type = \"llm_classifier\"\nclassifier_target = \"strong\"\nstrong_target = \"strong\"\nweak_target = \"fast\"\nbase_threshold = 0.5",- "type = 'advisor'\nexecutor_target = 'fast'\nadvisor_target = 'strong'",- );- assert!(Runner::from_toml(&config).is_ok());- assert!(Runner::from_toml_for_model_selection(&config).is_err());-}--#[test]-fn rejects_completion_overrides_and_forwarded_auth() {- for (table, setting) in [- ("targets.fast", "system_prompt = 'override'"),- ("llm_clients.local", "forward_auth = true"),- ] {- let config = CONFIG.replace(&format!("[{table}]"), &format!("[{table}]\n{setting}"));- assert!(Runner::from_toml(&config).is_ok(), "{setting}");- assert!(- Runner::from_toml_for_model_selection(&config).is_err(),- "{setting}"- );- }-}--#[tokio::test]-async fn composite_uses_host_client_and_only_changes_the_model() {- struct Judge;- #[async_trait]- impl RoutedLlmClient for Judge {- async fn call(&self, request: Request) -> Result<Response, LlmClientError> {- assert_eq!(request.llm_request.model.as_deref(), Some("fast-model"));- Ok(Response {- llm_response: LlmResponse::Agg(text_response(- None,- r#"{"crux":"bounded test","primary_rule":"SUP-1","capability_boundary":"supported","p_solve":0.1}"#,- )),- metadata: None,- upstream_headers: Default::default(),- })- }- }- let config = format!(- r#"{CONFIG}-[routes.composed]-id = "composite-id"-type = "composite"-[routes.composed.classifier]-target = "fast"-base_threshold = 0.5-classify_trigger = "user_turn"-[routes.composed.stage]-efficient_target = "fast"-capable_target = "strong"-"#- );- let runner = Runner::from_toml(&config).unwrap();- let request = Request {- llm_request: text_request(Some("composite-id".into()), "Solve this task"),- ..Default::default()- };- let outcome = runner- .route("composite-id")- .unwrap()- .decide_with_client(request.clone(), Arc::new(Judge))- .await- .unwrap();- let mut expected = request.llm_request;- expected.model = Some("strong-model".into());- assert_eq!(outcome.request.llm_request, expected);- assert!(outcome.response.is_none());-}