Skip to content

Build and run agents you can see, understand and trust.

feat(classifier): add Jev support and chat model routing middleware (#2758)

qbc6d32be1dfcf1
diff --git a/README.md b/README.md
index 30ab3ec6f..71acd4252 100644
--- a/README.md
+++ b/README.md
@@ -73,6 +73,8 @@ rather than constraining them with strict prompts and opinionated orchestrations
## News
<!-- BEGIN NEWS -->
+- **[2026-09] `FEAT`:** Model routing supported — use `ModelRouterMiddleware` to select an appropriate chat model for each reply.
+- **[2026-09] `INTE`:** Jev classifier model supported through the TypeSafe SDK via `JevClassifierModel`.
- **[2026-09] `FEAT`:** [agentscope-skill](https://github.com/agentscope-ai/skills/tree/main/skills/agentscope-skill) now supports AgentScope v2.
- **[2026-09] `INTE`:** Support DashScope, OpenAI, Gemini and xAI realtime APIs in `RealtimeAgent`. [Example](https://github.com/agentscope-ai/agentscope/tree/main/examples/realtime) | [Docs](https://docs.agentscope.io/latest/en/building-blocks/realtime/speech-to-speech)
- **[2026-09] `FEAT` `Experimental`:** Realtime voice agent supported. [Example](https://github.com/agentscope-ai/agentscope/tree/main/examples/realtime) | [Docs](https://docs.agentscope.io/latest/en/building-blocks/realtime/overview)
@@ -81,8 +83,6 @@ rather than constraining them with strict prompts and opinionated orchestrations
- **[2026-08] `INTE`:** DingTalk channel supported. [Docs](https://docs.agentscope.io/latest/en/deploy/channel/dingtalk)
- **[2026-08] `FEAT`:** Console supported — test and debug agents in the terminal. [Example](https://github.com/agentscope-ai/agentscope/tree/main/examples/console) | [Docs](https://docs.agentscope.io/latest/en/building-blocks/console)
- **[2026-08] `INTE`:** Feishu (Lark) and Discord channels supported. [Feishu](https://docs.agentscope.io/latest/en/deploy/channel/feishu) | [Discord](https://docs.agentscope.io/latest/en/deploy/channel/discord)
-- **[2026-08] `FEAT`:** Channels supported — connect agents to IM platforms in agent service. [Example](https://github.com/agentscope-ai/agentscope/tree/main/examples/agent_service) | [Docs](https://docs.agentscope.io/latest/en/deploy/channel/overview)
-- **[2026-08] `INTE`:** GitHub MCP Registry and ClawHub supported as built-in hubs. [Example](https://github.com/agentscope-ai/agentscope/tree/main/examples/agent_service) | [Docs](https://docs.agentscope.io/latest/en/deploy/hub)
<!-- END NEWS -->
[More news →](./docs/NEWS.md)
diff --git a/README_zh.md b/README_zh.md
index ddb8a9149..d3874b9cf 100644
--- a/README_zh.md
+++ b/README_zh.md
@@ -72,6 +72,8 @@ AgentScope 的目标是充分发挥大模型的推理与工具调用能力,
## 新闻
<!-- BEGIN NEWS -->
+- **[2026-09] `功能`:** 支持模型路由 —— 使用 `ModelRouterMiddleware` 为每次回复选择合适的聊天模型。
+- **[2026-09] `集成`:** 通过 TypeSafe SDK 集成 Jev 分类模型 `JevClassifierModel`。
- **[2026-09] `功能`:** [agentscope-skill](https://github.com/agentscope-ai/skills/tree/main/skills/agentscope-skill) 已适配 AgentScope v2。
- **[2026-09] `集成`:** `RealtimeAgent` 支持 DashScope、OpenAI、Gemini 与 xAI 的实时语音 API。[样例](https://github.com/agentscope-ai/agentscope/tree/main/examples/realtime) | [文档](https://docs.agentscope.io/latest/zh/building-blocks/realtime/speech-to-speech)
- **[2026-09] `功能` `实验性`:** 支持实时语音智能体。[样例](https://github.com/agentscope-ai/agentscope/tree/main/examples/realtime) | [文档](https://docs.agentscope.io/latest/zh/building-blocks/realtime/overview)
@@ -80,8 +82,6 @@ AgentScope 的目标是充分发挥大模型的推理与工具调用能力,
- **[2026-08] `集成`:** 支持钉钉(DingTalk)频道。[文档](https://docs.agentscope.io/latest/zh/deploy/channel/dingtalk)
- **[2026-08] `功能`:** 支持 Console —— 在终端中测试与调试智能体。[样例](https://github.com/agentscope-ai/agentscope/tree/main/examples/console) | [文档](https://docs.agentscope.io/latest/zh/building-blocks/console)
- **[2026-08] `集成`:** 支持飞书(Lark)与 Discord 频道。[飞书](https://docs.agentscope.io/latest/zh/deploy/channel/feishu) | [Discord](https://docs.agentscope.io/latest/zh/deploy/channel/discord)
-- **[2026-08] `功能`:** 支持消息频道 —— 将智能体接入即时通讯平台。[样例](https://github.com/agentscope-ai/agentscope/tree/main/examples/agent_service) | [文档](https://docs.agentscope.io/latest/zh/deploy/channel/overview)
-- **[2026-08] `集成`:** 内置集成 GitHub MCP Registry 与 ClawHub。[样例](https://github.com/agentscope-ai/agentscope/tree/main/examples/agent_service) | [文档](https://docs.agentscope.io/latest/zh/deploy/hub)
<!-- END NEWS -->
[更多新闻 →](./docs/NEWS_zh.md)
diff --git a/docs/NEWS.md b/docs/NEWS.md
index 597cc57c3..af3166115 100644
--- a/docs/NEWS.md
+++ b/docs/NEWS.md
@@ -2,6 +2,8 @@
<!-- The first 10 items are automatically synced to README.md and README_zh.md via GitHub Actions. -->
<!-- To update news in READMEs, modify this file and push to trigger the workflow. -->
+- **[2026-09] `FEAT`:** Model routing supported — use `ModelRouterMiddleware` to select an appropriate chat model for each reply.
+- **[2026-09] `INTE`:** Jev classifier model supported through the TypeSafe SDK via `JevClassifierModel`.
- **[2026-09] `FEAT`:** [agentscope-skill](https://github.com/agentscope-ai/skills/tree/main/skills/agentscope-skill) now supports AgentScope v2.
- **[2026-09] `INTE`:** Support DashScope, OpenAI, Gemini and xAI realtime APIs in `RealtimeAgent`. [Example](https://github.com/agentscope-ai/agentscope/tree/main/examples/realtime) | [Docs](https://docs.agentscope.io/latest/en/building-blocks/realtime/speech-to-speech)
- **[2026-09] `FEAT` `Experimental`:** Realtime voice agent supported. [Example](https://github.com/agentscope-ai/agentscope/tree/main/examples/realtime) | [Docs](https://docs.agentscope.io/latest/en/building-blocks/realtime/overview)
diff --git a/docs/NEWS_zh.md b/docs/NEWS_zh.md
index 90b797afc..3070d4e2a 100644
--- a/docs/NEWS_zh.md
+++ b/docs/NEWS_zh.md
@@ -2,6 +2,8 @@
<!-- The first 10 items are automatically synced to README.md and README_zh.md via GitHub Actions. -->
<!-- To update news in READMEs, modify this file and push to trigger the workflow. -->
+- **[2026-09] `功能`:** 支持模型路由 —— 使用 `ModelRouterMiddleware` 为每次回复选择合适的聊天模型。
+- **[2026-09] `集成`:** 通过 TypeSafe SDK 集成 Jev 分类模型 `JevClassifierModel`。
- **[2026-09] `功能`:** [agentscope-skill](https://github.com/agentscope-ai/skills/tree/main/skills/agentscope-skill) 已适配 AgentScope v2。
- **[2026-09] `集成`:** `RealtimeAgent` 支持 DashScope、OpenAI、Gemini 与 xAI 的实时语音 API。[样例](https://github.com/agentscope-ai/agentscope/tree/main/examples/realtime) | [文档](https://docs.agentscope.io/latest/zh/building-blocks/realtime/speech-to-speech)
- **[2026-09] `功能` `实验性`:** 支持实时语音智能体。[样例](https://github.com/agentscope-ai/agentscope/tree/main/examples/realtime) | [文档](https://docs.agentscope.io/latest/zh/building-blocks/realtime/overview)
diff --git a/pyproject.toml b/pyproject.toml
index 580a4de30..d815e2d73 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -54,11 +54,13 @@ dependencies = [
model-gemini = ["google-genai"]
model-ollama = ["ollama>=0.5.4"]
model-xai = ["xai-sdk"]
+classifier-jev = ["typesafe-sdk>=0.7"]
models = [
"agentscope[model-gemini]",
"agentscope[model-ollama]",
"agentscope[model-xai]",
]
+classifiers = ["agentscope[classifier-jev]"]
# ------------ Service ------------
service = [
"fastapi",
@@ -170,6 +172,7 @@ reme = ["agentscope[memory-reme]"]
# ------------ Full ------------
full = [
"agentscope[models]",
+ "agentscope[classifiers]",
"agentscope[service]",
"agentscope[storage-redis]",
"agentscope[storage-sql]",
diff --git a/src/agentscope/classifier/__init__.py b/src/agentscope/classifier/__init__.py
new file mode 100644
index 000000000..2811148f5
--- /dev/null
+++ b/src/agentscope/classifier/__init__.py
@@ -0,0 +1,36 @@
+# -*- coding: utf-8 -*-
+"""Typed probabilistic classifier models."""
+
+from ._base import ClassifierModelBase
+from ._question import (
+ BinaryCriteria,
+ BinaryQuestion,
+ ChoiceQuestion,
+ ClassifierQuestion,
+ ScoreQuestion,
+)
+from ._response import (
+ BinaryAnswer,
+ ChoiceAnswer,
+ ClassifierAnswer,
+ ClassifierResponse,
+ ScoreAnswer,
+)
+from ._usage import ClassifierUsage
+from ._jev import JevClassifierModel
+
+__all__ = [
+ "BinaryAnswer",
+ "BinaryCriteria",
+ "BinaryQuestion",
+ "ChoiceAnswer",
+ "ChoiceQuestion",
+ "ClassifierAnswer",
+ "ClassifierModelBase",
+ "ClassifierQuestion",
+ "ClassifierResponse",
+ "ClassifierUsage",
+ "JevClassifierModel",
+ "ScoreAnswer",
+ "ScoreQuestion",
+]
diff --git a/src/agentscope/classifier/_base.py b/src/agentscope/classifier/_base.py
new file mode 100644
index 000000000..790dbfdd8
--- /dev/null
+++ b/src/agentscope/classifier/_base.py
@@ -0,0 +1,59 @@
+# -*- coding: utf-8 -*-
+"""The base class for classifier models."""
+from abc import ABC, abstractmethod
+from typing import Any, Mapping
+
+from pydantic import BaseModel
+
+from ._question import ClassifierQuestion
+from ._response import ClassifierResponse
+from ..credential import CredentialBase
+
+
+class ClassifierModelBase(ABC):
+ """Base class for models that return typed probabilistic decisions."""
+
+ class Parameters(BaseModel):
+ """Provider-specific classifier parameters."""
+
+ def __init__(
+ self,
+ credential: CredentialBase,
+ model: str,
+ parameters: BaseModel | None = None,
+ ) -> None:
+ """Initialize a classifier model.
+
+ Args:
+ credential (`CredentialBase`):
+ The credential used to authenticate with the provider.
+ model (`str`):
+ The classifier model name or alias.
+ parameters (`BaseModel | None`, defaults to `None`):
+ Provider-specific model parameters.
+ """
+ self.credential = credential
+ self.model = model
+ self.parameters = parameters or self.Parameters()
+
+ @abstractmethod
+ async def __call__(
+ self,
+ state: str | dict,
+ questions: Mapping[str, ClassifierQuestion],
+ **kwargs: Any,
+ ) -> ClassifierResponse:
+ """Evaluate the named questions against the shared input state.
+
+ Args:
+ state (`str | dict`):
+ The text or JSON content shared by all questions.
+ questions (`Mapping[str, ClassifierQuestion]`):
+ The typed questions keyed by name.
+ **kwargs (`Any`):
+ Provider-specific per-call options.
+
+ Returns:
+ `ClassifierResponse`:
+ The typed probabilistic answers keyed by question name.
+ """
diff --git a/src/agentscope/classifier/_jev/__init__.py b/src/agentscope/classifier/_jev/__init__.py
new file mode 100644
index 000000000..69a0695f2
--- /dev/null
+++ b/src/agentscope/classifier/_jev/__init__.py
@@ -0,0 +1,6 @@
+# -*- coding: utf-8 -*-
+"""The TypeSafe Jev classifier model."""
+
+from ._model import JevClassifierModel
+
+__all__ = ["JevClassifierModel"]
diff --git a/src/agentscope/classifier/_jev/_model.py b/src/agentscope/classifier/_jev/_model.py
new file mode 100644
index 000000000..03efc323a
--- /dev/null
+++ b/src/agentscope/classifier/_jev/_model.py
@@ -0,0 +1,166 @@
+# -*- coding: utf-8 -*-
+"""The TypeSafe Jev classifier model implementation."""
+from time import perf_counter
+from typing import Any, Mapping
+
+from pydantic import ConfigDict
+
+from .._base import ClassifierModelBase
+from .._question import (
+ BinaryQuestion,
+ ChoiceQuestion,
+ ClassifierQuestion,
+ ScoreQuestion,
+)
+from .._response import (
+ BinaryAnswer,
+ ChoiceAnswer,
+ ClassifierAnswer,
+ ClassifierResponse,
+ ScoreAnswer,
+)
+from .._usage import ClassifierUsage
+from ...credential import TypeSafeCredential
+
+
+def _to_sdk_question(question: ClassifierQuestion) -> Any:
+ """Translate an AgentScope question to a TypeSafe question."""
+ from typesafe_sdk import Choice, Noul, Score
+
+ match question:
+ case BinaryQuestion():
+ # The binary question maps to TypeSafe's Noul primitive
+ criteria = question.criteria and question.criteria.model_dump(
+ exclude_unset=True,
+ )
+ return Noul(instructions=question.instructions, criteria=criteria)
+ case ChoiceQuestion():
+ return Choice(
+ instructions=question.instructions,
+ criteria=question.criteria,
+ )
+ case ScoreQuestion():
+ return Score(
+ instructions=question.instructions,
+ criteria=question.criteria,
+ )
+
+
+def _from_sdk_answer(answer: Any) -> ClassifierAnswer:
+ """Translate a TypeSafe answer to an AgentScope answer."""
+ match answer.type:
+ case "noul":
+ return BinaryAnswer(probability=answer.noul)
+ case "choice":
+ return ChoiceAnswer(
+ choice=answer.choice,
+ confidence=answer.confidence,
+ probabilities=dict(answer.probabilities),
+ )
+ case "score":
+ return ScoreAnswer(
+ score=answer.score,
+ confidence=answer.confidence,
+ legend=dict(answer.legend),
+ probabilities=dict(answer.probabilities),
+ )
+ raise ValueError(f"Unsupported TypeSafe answer type: {answer.type!r}.")
+
+
+class JevClassifierModel(ClassifierModelBase):
+ """A classifier backed by TypeSafe's Jev System One API."""
+
+ class Parameters(ClassifierModelBase.Parameters):
+ """Provider-specific Jev model parameters."""
+
+ model_config = ConfigDict(extra="forbid")
+
+ def __init__(
+ self,
+ credential: TypeSafeCredential,
+ model: str = "jev-latest",
+ parameters: "JevClassifierModel.Parameters | None" = None,
+ timeout: float = 30.0,
+ max_retries: int = 2,
+ retry_delay: float = 0.5,
+ ) -> None:
+ """Initialize the Jev classifier.
+
+ Args:
+ credential (`TypeSafeCredential`):
+ The TypeSafe API credential.
+ model (`str`, defaults to ``"jev-latest"``):
+ The Jev model name or alias.
+ parameters (`JevClassifierModel.Parameters | None`):
+ Provider-specific model parameters.
+ timeout (`float`, defaults to `30.0`):
+ Per-request timeout in seconds.
+ max_retries (`int`, defaults to `2`):
+ Maximum retries after the initial request, done by the SDK.
+ retry_delay (`float`, defaults to `0.5`):
+ Initial exponential backoff delay in seconds.
+ """
+ try:
+ from typesafe_sdk import AsyncTypeSafeClient, RetryPolicy
+ except ImportError as error:
+ raise ImportError(
+ "JevClassifierModel requires a compatible optional "
+ "'typesafe-sdk' dependency. Install it with "
+ "`pip install 'agentscope[classifier-jev]'`.",
+ ) from error
+
+ super().__init__(credential, model, parameters)
+ self.client = AsyncTypeSafeClient(
+ api_key=credential.api_key.get_secret_value(),
+ base_url=credential.base_url,
+ model=model,
+ timeout=timeout,
+ retry=RetryPolicy(
+ max_retries=max_retries,
+ backoff_initial=retry_delay,
+ ),
+ )
+
+ async def __call__(
+ self,
+ state: str | dict,
+ questions: Mapping[str, ClassifierQuestion],
+ **kwargs: Any,
+ ) -> ClassifierResponse:
+ """Call the TypeSafe System One API and normalize its response.
+
+ Args:
+ state (`str | dict`):
+ The text or JSON content to classify.
+ questions (`Mapping[str, ClassifierQuestion]`):
+ The typed questions keyed by name.
+ **kwargs (`Any`):
+ Per-call TypeSafe options, e.g. ``extra_headers`` and
+ ``extra_body``.
+
+ Returns:
+ `ClassifierResponse`:
+ The typed probabilistic answers keyed by question name.
+ """
+ start_time = perf_counter()
+ response = await self.client.system_one(
+ state=state,
+ questions={
+ name: _to_sdk_question(question)
+ for name, question in questions.items()
+ },
+ model=self.model,
+ **kwargs,
+ )
+ return ClassifierResponse(
+ model=response.model,
+ content={
+ name: _from_sdk_answer(answer)
+ for name, answer in response.answers.items()
+ },
+ usage=ClassifierUsage(
+ input_tokens=response.usage.input_tokens,
+ output_tokens=response.usage.output_tokens,
+ time=perf_counter() - start_time,
+ ),
+ )
diff --git a/src/agentscope/classifier/_question.py b/src/agentscope/classifier/_question.py
new file mode 100644
index 000000000..2d3370283
--- /dev/null
+++ b/src/agentscope/classifier/_question.py
@@ -0,0 +1,69 @@
+# -*- coding: utf-8 -*-
+"""Provider-independent question types for classifier models."""
+from typing import Annotated, Literal, TypeAlias
+
+from pydantic import BaseModel, ConfigDict, Field
+
+
+class BinaryCriteria(BaseModel):
+ """Optional descriptions for the two outcomes of a binary question."""
+
+ model_config = ConfigDict(extra="forbid")
+
+ true: str | None = None
+ """What counts as a positive outcome."""
+
+ false: str | None = None
+ """What counts as a negative outcome."""
+
+
+class BinaryQuestion(BaseModel):
+ """A question whose answer is a probability between zero and one."""
+
+ model_config = ConfigDict(extra="forbid")
+
+ type: Literal["binary"] = "binary"
+ """The question type discriminator."""
+
+ instructions: str | None = None
+ """The question or statement to evaluate."""
+
+ criteria: BinaryCriteria | None = None
+ """Optional descriptions of the positive and negative outcomes."""
+
+
+class ChoiceQuestion(BaseModel):
+ """A question that selects one named alternative."""
+
+ model_config = ConfigDict(extra="forbid")
+
+ type: Literal["choice"] = "choice"
+ """The question type discriminator."""
+
+ criteria: dict[str, str | None] = Field(min_length=1)
+ """Named alternatives and optional descriptions."""
+
+ instructions: str | None = None
+ """The decision the classifier should make."""
+
+
+class ScoreQuestion(BaseModel):
+ """A question that assigns an expected score using an ordered rubric."""
+
+ model_config = ConfigDict(extra="forbid")
+
+ type: Literal["score"] = "score"
+ """The question type discriminator."""
+
+ criteria: list[str] = Field(min_length=1)
+ """Ordered score descriptions, starting at score zero."""
+
+ instructions: str | None = None
+ """The value the classifier should score."""
+
+
+#: A provider-independent classifier question.
+ClassifierQuestion: TypeAlias = Annotated[
+ BinaryQuestion | ChoiceQuestion | ScoreQuestion,
+ Field(discriminator="type"),
+]
diff --git a/src/agentscope/classifier/_response.py b/src/agentscope/classifier/_response.py
new file mode 100644
index 000000000..9043025eb
--- /dev/null
+++ b/src/agentscope/classifier/_response.py
@@ -0,0 +1,103 @@
+# -*- coding: utf-8 -*-
+"""Provider-independent response types for classifier models."""
+from dataclasses import dataclass, field
+from typing import Literal, TypeAlias
+
+from ._usage import ClassifierUsage
+from .._utils._common import _generate_id, _generate_timestamp
+from .._utils._mixin import DictMixin
+from ..types import JSONSerializableObject
+
+
+@dataclass
+class BinaryAnswer(DictMixin):
+ """The probability of a positive answer to a binary question."""
+
+ probability: float
+ """The positive-outcome probability, between zero and one."""
+
+ type: Literal["binary_answer"] = field(
+ default_factory=lambda: "binary_answer",
+ )
+ """The answer type discriminator."""
+
+
+@dataclass
+class ChoiceAnswer(DictMixin):
+ """The selected alternative and its probability distribution."""
+
+ choice: str
+ """The alternative with the highest probability."""
+
+ confidence: float
+ """The provider-reported confidence in the selected alternative."""
+
+ probabilities: dict[str, float]
+ """The probability of each requested alternative."""
+
+ type: Literal["choice_answer"] = field(
+ default_factory=lambda: "choice_answer",
+ )
+ """The answer type discriminator."""
+
+
+@dataclass
+class ScoreAnswer(DictMixin):
+ """An expected score and its probability distribution."""
+
+ score: float
+ """The probability-weighted expected score."""
+
+ confidence: float
+ """The provider-reported confidence in the score."""
+
+ legend: dict[int, JSONSerializableObject]
+ """The requested descriptions keyed by integer score level.
+
+ JSON serialization converts these integer keys to strings.
+ """
+
+ probabilities: dict[int, float]
+ """The probability of each integer score level.
+
+ JSON serialization converts these integer keys to strings.
+ """
+
+ type: Literal["score_answer"] = field(
+ default_factory=lambda: "score_answer",
+ )
+ """The answer type discriminator."""
+
+
+#: A provider-independent classifier answer.
+ClassifierAnswer: TypeAlias = BinaryAnswer | ChoiceAnswer | ScoreAnswer
+
+
+@dataclass
+class ClassifierResponse(DictMixin):
+ """The normalized response of a classifier model."""
+
+ model: str
+ """The concrete model that produced the answers."""
+
+ content: dict[str, ClassifierAnswer]
+ """Answers keyed by the corresponding question names."""
+
+ usage: ClassifierUsage | None = field(default_factory=lambda: None)
+ """Usage information reported for this invocation."""
+
+ id: str = field(default_factory=_generate_id)
+ """The unique response identifier."""
+
+ created_at: str = field(default_factory=_generate_timestamp)
+ """When the response was created."""
+
+ type: Literal["classifier_response"] = field(
+ default_factory=lambda: "classifier_response",
+ )
+ """The response type discriminator."""
+
+ metadata: dict[str, JSONSerializableObject] = field(
+ default_factory=dict,
+ )
+ """Additional provider-independent response metadata."""
diff --git a/src/agentscope/classifier/_usage.py b/src/agentscope/classifier/_usage.py
new file mode 100644
index 000000000..69016e6d4
--- /dev/null
+++ b/src/agentscope/classifier/_usage.py
@@ -0,0 +1,25 @@
+# -*- coding: utf-8 -*-
+"""Usage information for classifier model calls."""
+from dataclasses import dataclass, field
+from typing import Literal
+
+from .._utils._mixin import DictMixin
+
+
+@dataclass
+class ClassifierUsage(DictMixin):
+ """The usage of a classifier model API invocation."""
+
+ time: float
+ """The wall-clock time used in seconds."""
+
+ input_tokens: int | None = field(default_factory=lambda: None)
+ """The number of input tokens, if reported by the provider."""
+
+ output_tokens: int | None = field(default_factory=lambda: None)
+ """The number of output tokens, if reported by the provider."""
+
+ type: Literal["classifier"] = field(
+ default_factory=lambda: "classifier",
+ )
+ """The usage type discriminator."""
diff --git a/src/agentscope/credential/__init__.py b/src/agentscope/credential/__init__.py
index 71d1df981..83386b747 100644
--- a/src/agentscope/credential/__init__.py
+++ b/src/agentscope/credential/__init__.py
@@ -9,6 +9,7 @@ from ._gemini import GeminiCredential
from ._moonshot import MoonshotCredential
from ._ollama import OllamaCredential
from ._openai import OpenAICredential
+from ._typesafe import TypeSafeCredential
from ._xai import XAICredential
from ._volcengine import VolcengineCredential
from ._factory import CredentialFactory
@@ -23,6 +24,7 @@ __all__ = [
"MoonshotCredential",
"OllamaCredential",
"OpenAICredential",
+ "TypeSafeCredential",
"XAICredential",
"VolcengineCredential",
"CredentialFactory",
diff --git a/src/agentscope/credential/_typesafe.py b/src/agentscope/credential/_typesafe.py
new file mode 100644
index 000000000..bf23073b4
--- /dev/null
+++ b/src/agentscope/credential/_typesafe.py
@@ -0,0 +1,33 @@
+# -*- coding: utf-8 -*-
+"""The TypeSafe AI credential."""
+from typing import Literal, TYPE_CHECKING, Type
+
+from pydantic import ConfigDict, Field, SecretStr
+
+from ._base import CredentialBase
+
+if TYPE_CHECKING:
+ from ..model import ChatModelBase
+
+
+class TypeSafeCredential(CredentialBase):
+ """The credential used by TypeSafe System One models."""
+
+ model_config = ConfigDict(title="TypeSafe AI API")
+
+ type: Literal["typesafe_credential"] = "typesafe_credential"
+ """The credential type discriminator."""
+
+ api_key: SecretStr = Field(description="The TypeSafe API key.")
+ """The TypeSafe API key."""
+
+ base_url: str | None = Field(
+ default=None,
+ description="An optional custom TypeSafe API base URL.",
+ )
+ """An optional custom TypeSafe API base URL."""
+
+ @classmethod
+ def get_chat_model_class(cls) -> Type["ChatModelBase"]:
+ """TypeSafe credentials serve classifier models only."""
+ raise NotImplementedError(f"{cls.__name__} has no chat model.")
diff --git a/src/agentscope/middleware/__init__.py b/src/agentscope/middleware/__init__.py
index 822b56c43..e2b7f11b6 100644
--- a/src/agentscope/middleware/__init__.py
+++ b/src/agentscope/middleware/__init__.py
@@ -11,6 +11,7 @@ from ._longterm_memory import (
)
from ._tracing import TracingMiddleware
from ._tts_middleware import TTSMiddleware
+from ._model_router import ChatModelCandidate, ModelRouterMiddleware
__all__ = [
"MiddlewareBase",
@@ -21,4 +22,6 @@ __all__ = [
"TracingMiddleware",
"ReplyBudgetControlMiddleware",
"TTSMiddleware",
+ "ChatModelCandidate",
+ "ModelRouterMiddleware",
]
diff --git a/src/agentscope/middleware/_model_router.py b/src/agentscope/middleware/_model_router.py
new file mode 100644
index 000000000..3d42c554a
--- /dev/null
+++ b/src/agentscope/middleware/_model_router.py
@@ -0,0 +1,157 @@
+# -*- coding: utf-8 -*-
+"""Route each reply to a chat model chosen by a classifier."""
+import json
+from dataclasses import dataclass
+from typing import AsyncGenerator, Callable, Sequence, TYPE_CHECKING
+
+from ._base import MiddlewareBase
+from .._logging import logger
+from ..classifier import ChoiceAnswer, ChoiceQuestion, ClassifierModelBase
+from ..event import ReplyStartEvent
+from ..message import Msg, SystemMsg, UserMsg
+from ..model import ChatModelBase
+
+if TYPE_CHECKING:
+ from ..agent import Agent
+
+
+_DEFAULT_INSTRUCTIONS = (
+ "Select the most suitable chat model for responding to the user input."
+)
+
+
+@dataclass
+class ChatModelCandidate:
+ """A named chat model and the criteria for selecting it."""
+
+ name: str
+ """The stable name returned by the classifier."""
+
+ model: ChatModelBase
+ """The chat model used when this candidate is selected."""
+
+ description: str
+ """When this candidate should be selected."""
+
+
+class ModelRouterMiddleware(MiddlewareBase):
+ """Select the chat model of each reply by classifying the user input.
+
+ The decision is made once per reply and kept when the reply resumes,
+ e.g. after a user confirmation. When the routing model fails or picks
+ an unknown candidate, the agent keeps its own model.
+ """
+
+ def __init__(
+ self,
+ classifier_model: ClassifierModelBase | ChatModelBase,
+ candidates: Sequence[ChatModelCandidate],
+ instructions: str = _DEFAULT_INSTRUCTIONS,
+ ) -> None:
+ """Initialize the model router.
+
+ Args:
+ classifier_model (`ClassifierModelBase | ChatModelBase`):
+ The classifier or chat model used to select a candidate.
+ candidates (`Sequence[ChatModelCandidate]`):
+ The uniquely named chat model candidates.
+ instructions (`str`):
+ The instructions of the routing question.
+ """
+ names = [candidate.name for candidate in candidates]
+ if len(set(names)) != len(names):
+ raise ValueError(f"Duplicate chat model candidates: {names}")
+
+ self.classifier_model = classifier_model
+ self._models = {_.name: _.model for _ in candidates}
+ criteria = {_.name: _.description for _ in candidates}
+ self._question = ChoiceQuestion(
+ instructions=instructions,
+ criteria=criteria,
+ )
+ self._prompt = (
+ f"{instructions}\n\n"
+ "Select exactly one candidate using these criteria:\n"
+ f"{json.dumps(criteria, ensure_ascii=False, indent=2)}"
+ )
+ self._schema = {
+ "type": "object",
+ "properties": {"choice": {"type": "string", "enum": names}},
+ "required": ["choice"],
+ "additionalProperties": False,
+ }
+
+ async def on_reply(
+ self,
+ agent: "Agent",
+ input_kwargs: dict,
+ next_handler: Callable[..., AsyncGenerator],
+ ) -> AsyncGenerator:
+ """Swap ``agent.model`` for the selected candidate during the reply."""
+ key = await self.get_middleware_key()
+ original_model = agent.model
+ # A resumed reply keeps its route, a new one is routed on its start
+ routed = agent.state.middle_context.get(key, {})
+ agent.model = self._models.get(
+ routed.get(agent.state.reply_id),
+ original_model,
+ )
+ try:
+ async for event in next_handler(**input_kwargs):
+ if isinstance(event, ReplyStartEvent):
+ name = await self._route(
+ self._latest_user_text(input_kwargs["inputs"]),
+ )
+ agent.state.middle_context[key] = {event.reply_id: name}
+ agent.model = self._models.get(name, original_model)
+ yield event
+ finally:
+ agent.model = original_model
+
+ async def _route(self, text: str | None) -> str | None:
+ """Return the selected candidate name, `None` to keep the agent's."""
+ if not text:
+ return None
+ try:
+ if isinstance(self.classifier_model, ClassifierModelBase):
+ res = await self.classifier_model(
+ state=text,
+ questions={"chat_model": self._question},
+ )
+ answer = res.content["chat_model"]
+ name = (
+ answer.choice if isinstance(answer, ChoiceAnswer) else None
+ )
+ else:
+ res = await self.classifier_model.generate_structured_output(
+ messages=[
+ SystemMsg(name="system", content=self._prompt),
+ UserMsg(name="user", content=text),
+ ],
+ structured_model=self._schema,
+ )
+ name = res.content["choice"]
+ except Exception as error:
+ logger.warning(
+ "Chat model routing failed, keeping the agent's model: %s",
+ error,
+ )
+ return None
+ if name not in self._models:
+ logger.warning(
+ "Routing model selected unknown candidate %r, keeping the "
+ "agent's model",
+ name,
+ )
+ return None
+ logger.debug("Routed the reply to chat model candidate %r", name)
+ return name
+
+ @staticmethod
+ def _latest_user_text(inputs: object) -> str | None:
+ """The text of the latest user message, if any."""
+ msgs = inputs if isinstance(inputs, list) else [inputs]
+ for msg in reversed(msgs):
+ if isinstance(msg, Msg) and msg.role == "user":
+ return msg.get_text_content()
+ return None
diff --git a/tests/classifier_jev_test.py b/tests/classifier_jev_test.py
new file mode 100644
index 000000000..8c78cd65f
--- /dev/null
+++ b/tests/classifier_jev_test.py
@@ -0,0 +1,205 @@
+# -*- coding: utf-8 -*-
+"""Tests for the TypeSafe Jev classifier adapter."""
+from dataclasses import asdict
+from types import SimpleNamespace
+from typing import Any
+from unittest import IsolatedAsyncioTestCase
+from unittest.mock import AsyncMock, MagicMock, patch
+
+import pytest
+
+from utils import AnyValue
+from agentscope.classifier import (
+ BinaryCriteria,
+ BinaryQuestion,
+ ChoiceQuestion,
+ JevClassifierModel,
+ ScoreQuestion,
+)
+from agentscope.credential import TypeSafeCredential
+
+typesafe_sdk = pytest.importorskip("typesafe_sdk")
+
+A = AnyValue()
+
+
+class JevClassifierModelTest(IsolatedAsyncioTestCase):
+ """Test request and response translation against official SDK types."""
+
+ @patch("typesafe_sdk.AsyncTypeSafeClient")
+ async def test_call_translates_all_question_types(
+ self,
+ client_cls: Any,
+ ) -> None:
+ """All supported questions and answers should round-trip."""
+ client = MagicMock()
+ client.system_one = AsyncMock(
+ return_value=SimpleNamespace(
+ model="jev-1.13.0",
+ answers={
+ "urgent": typesafe_sdk.NoulAnswer(type="noul", noul=0.8),
+ "route": typesafe_sdk.ChoiceAnswer(
+ type="choice",
+ choice="billing",
+ confidence=0.9,
+ probabilities={"billing": 0.9, "support": 0.1},
+ ),
+ "priority": typesafe_sdk.ScoreAnswer(
+ type="score",
+ score=1.7,
+ confidence=0.85,
+ legend={0: "low", 1: "medium", 2: "high"},
+ probabilities={0: 0.05, 1: 0.2, 2: 0.75},
+ ),
+ },
+ usage=typesafe_sdk.Usage(input_tokens=120, output_tokens=3),
+ ),
+ )
+ client_cls.return_value = client
+
+ model = JevClassifierModel(
+ credential=TypeSafeCredential(
+ api_key="secret",
+ base_url="https://typesafe.example",
+ ),
+ max_retries=4,
+ retry_delay=0.25,
+ )
+ response = await model(
+ state="I was charged twice.",
+ questions={
+ "urgent": BinaryQuestion(
+ instructions="Is this urgent?",
+ criteria=BinaryCriteria(true="Urgent."),
+ ),
+ "route": ChoiceQuestion(
+ instructions="Select a route.",
+ criteria={"billing": None, "support": None},
+ ),
+ "priority": ScoreQuestion(
+ instructions="Rate priority.",
+ criteria=["low", "medium", "high"],
+ ),
+ },
+ extra_body={"trace": True},
+ )
+
+ self.assertDictEqual(
+ client_cls.call_args.kwargs,
+ {
+ "api_key": "secret",
+ "base_url": "https://typesafe.example",
+ "model": "jev-latest",
+ "timeout": 30.0,
+ "retry": typesafe_sdk.RetryPolicy(
+ max_retries=4,
+ backoff_initial=0.25,
+ ),
+ },
+ )
+ call_kwargs = client.system_one.await_args.kwargs
+ self.assertDictEqual(
+ {
+ **call_kwargs,
+ "questions": {
+ name: question.model_dump()
+ for name, question in call_kwargs["questions"].items()
+ },
+ },
+ {
+ "state": "I was charged twice.",
+ "questions": {
+ "urgent": {
+ "type": "noul",
+ "instructions": "Is this urgent?",
+ "criteria": {"true": "Urgent."},
+ },
+ "route": {
+ "type": "choice",
+ "instructions": "Select a route.",
+ "criteria": {"billing": None, "support": None},
+ },
+ "priority": {
+ "type": "score",
+ "instructions": "Rate priority.",
+ "criteria": ["low", "medium", "high"],
+ },
+ },
+ "model": "jev-latest",
+ "extra_body": {"trace": True},
+ },
+ )
+ self.assertDictEqual(
+ asdict(response),
+ {
+ "model": "jev-1.13.0",
+ "content": {
+ "urgent": {
+ "probability": 0.8,
+ "type": "binary_answer",
+ },
+ "route": {
+ "choice": "billing",
+ "confidence": 0.9,
+ "probabilities": {
+ "billing": 0.9,
+ "support": 0.1,
+ },
+ "type": "choice_answer",
+ },
+ "priority": {
+ "score": 1.7,
+ "confidence": 0.85,
+ "legend": {0: "low", 1: "medium", 2: "high"},
+ "probabilities": {0: 0.05, 1: 0.2, 2: 0.75},
+ "type": "score_answer",
+ },
+ },
+ "usage": {
+ "time": A,
+ "input_tokens": 120,
+ "output_tokens": 3,
+ "type": "classifier",
+ },
+ "id": A,
+ "created_at": A,
+ "type": "classifier_response",
+ "metadata": {},
+ },
+ )
+
+ @patch("typesafe_sdk.AsyncTypeSafeClient")
+ async def test_provider_error_is_raised(self, client_cls: Any) -> None:
+ """Retries belong to the SDK, so its errors are raised as they are."""
+ client = MagicMock()
+ client.system_one = AsyncMock(side_effect=RuntimeError("failed"))
+ client_cls.return_value = client
+ model = JevClassifierModel(TypeSafeCredential(api_key="secret"))
+
+ with self.assertRaisesRegex(RuntimeError, "failed"):
+ await model(
+ state="hello",
+ questions={"route": ChoiceQuestion(criteria={"a": None})},
+ )
+
+ self.assertDictEqual(
+ client_cls.call_args.kwargs,
+ {
+ "api_key": "secret",
+ "base_url": None,
+ "model": "jev-latest",
+ "timeout": 30.0,
+ "retry": typesafe_sdk.RetryPolicy(),
+ },
+ )
+
+ async def test_incompatible_sdk_has_clear_error(self) -> None:
+ """Missing SDK exports should produce an actionable error."""
+ incompatible_sdk = SimpleNamespace(AsyncTypeSafeClient=MagicMock())
+
+ with patch.dict("sys.modules", {"typesafe_sdk": incompatible_sdk}):
+ with self.assertRaisesRegex(
+ ImportError,
+ "requires a compatible optional",
+ ):
+ JevClassifierModel(TypeSafeCredential(api_key="secret"))
diff --git a/tests/classifier_test.py b/tests/classifier_test.py
new file mode 100644
index 000000000..987f98f86
--- /dev/null
+++ b/tests/classifier_test.py
@@ -0,0 +1,116 @@
+# -*- coding: utf-8 -*-
+"""Tests for the provider-independent classifier model contract."""
+from dataclasses import asdict
+from typing import Any, Mapping
+from unittest import IsolatedAsyncioTestCase, TestCase
+
+from pydantic import ValidationError
+
+from utils import AnyString
+from agentscope.classifier import (
+ BinaryAnswer,
+ BinaryQuestion,
+ ChoiceQuestion,
+ ClassifierModelBase,
+ ClassifierQuestion,
+ ClassifierResponse,
+ ScoreQuestion,
+)
+from agentscope.credential import CredentialBase
+from agentscope.message import TextBlock
+
+
+class _MockClassifier(ClassifierModelBase):
+ """A minimal classifier used to exercise the base class."""
+
+ def __init__(self) -> None:
+ """Initialize the mock classifier."""
+ super().__init__(CredentialBase(), "mock-classifier")
+
+ async def __call__(
+ self,
+ state: str | dict,
+ questions: Mapping[str, ClassifierQuestion],
+ **kwargs: Any,
+ ) -> ClassifierResponse:
+ """Return a deterministic response."""
+ return ClassifierResponse(
+ model=self.model,
+ content={"safe": BinaryAnswer(probability=0.75)},
+ )
+
+
+class ClassifierQuestionTest(TestCase):
+ """Validate the framework-owned question types."""
+
+ def test_question_models(self) -> None:
+ """Question models should preserve their complete typed structure."""
+ choice = ChoiceQuestion(
+ instructions="Select a route.",
+ criteria={"billing": None, "support": "Technical support."},
+ )
+ score = ScoreQuestion(
+ instructions="Rate urgency.",
+ criteria=["Can wait.", "Handle today."],
+ )
+
+ self.assertDictEqual(
+ choice.model_dump(),
+ {
+ "type": "choice",
+ "criteria": {
+ "billing": None,
+ "support": "Technical support.",
+ },
+ "instructions": "Select a route.",
+ },
+ )
+ self.assertDictEqual(
+ score.model_dump(),
+ {
+ "type": "score",
+ "criteria": ["Can wait.", "Handle today."],
+ "instructions": "Rate urgency.",
+ },
+ )
+
+ def test_empty_criteria_are_rejected(self) -> None:
+ """Choice and score questions require at least one criterion."""
+ with self.assertRaises(ValidationError):
+ ChoiceQuestion(criteria={})
+ with self.assertRaises(ValidationError):
+ ScoreQuestion(criteria=[])
+
+ def test_non_string_instructions_are_rejected(self) -> None:
+ """Question instructions must be plain text."""
+ with self.assertRaises(ValidationError):
+ BinaryQuestion(instructions=TextBlock(text="hello"))
+
+
+class ClassifierModelBaseTest(IsolatedAsyncioTestCase):
+ """Test the classifier model contract."""
+
+ async def test_call(self) -> None:
+ """A call returns the typed answers keyed by question name."""
+ response = await _MockClassifier()(
+ state="hello",
+ questions={"safe": ChoiceQuestion(criteria={"yes": None})},
+ )
+
+ self.assertDictEqual(
+ asdict(response),
+ {
+ "model": "mock-classifier",
+ "content": {
+ "safe": {
+ "probability": 0.75,
+ "type": "binary_answer",
+ },
+ },
+ "usage": None,
+ "id": AnyString(),
+ "created_at": AnyString(),
+ "type": "classifier_response",
+ "metadata": {},
+ },
+ )
diff --git a/tests/middleware_model_router_test.py b/tests/middleware_model_router_test.py
new file mode 100644
index 000000000..542364624
--- /dev/null
+++ b/tests/middleware_model_router_test.py
@@ -0,0 +1,300 @@
+# -*- coding: utf-8 -*-
+"""Tests for the model router middleware."""
+from typing import Any, Mapping
+from unittest import IsolatedAsyncioTestCase
+
+from pydantic import BaseModel
+
+from utils import MockModel
+from agentscope.agent import Agent, InjectionConfig
+from agentscope.classifier import (
+ ChoiceAnswer,
+ ClassifierModelBase,
+ ClassifierQuestion,
+ ClassifierResponse,
+)
+from agentscope.credential import CredentialBase
+from agentscope.event import ModelCallStartEvent, ReplyStartEvent
+from agentscope.message import (
+ Base64Source,
+ DataBlock,
+ Msg,
+ TextBlock,
+ UserMsg,
+)
+from agentscope.middleware import ChatModelCandidate, ModelRouterMiddleware
+from agentscope.model import ChatResponse, StructuredResponse
+
+
+class _MockClassifier(ClassifierModelBase):
+ """A classifier that returns the configured choices in turn."""
+
+ def __init__(self, outcomes: list[str | Exception]) -> None:
+ """Initialize the classifier with choices or exceptions."""
+ super().__init__(CredentialBase(), "mock-classifier")
+ self.outcomes = outcomes
+ self.calls: list[dict] = []
+
+ async def __call__(
+ self,
+ state: str | dict,
+ questions: Mapping[str, ClassifierQuestion],
+ **kwargs: Any,
+ ) -> ClassifierResponse:
+ """Return the next configured routing choice."""
+ self.calls.append(
+ {
+ "state": state,
+ "questions": {k: v.model_dump() for k, v in questions.items()},
+ },
+ )
+ outcome = self.outcomes[len(self.calls) - 1]
+ if isinstance(outcome, Exception):
+ raise outcome
+ return ClassifierResponse(
+ model=self.model,
+ content={
+ "chat_model": ChoiceAnswer(
+ choice=outcome,
+ confidence=0.9,
+ probabilities={outcome: 0.9},
+ ),
+ },
+ )
+
+
+class _MockRoutingChatModel(MockModel):
+ """A chat model that returns the configured choice as structured output."""
+
+ def __init__(self, outcome: str) -> None:
+ """Initialize the chat model with its choice."""
+ super().__init__(model="routing-chat-model")
+ self.outcome = outcome
+ self.calls: list[dict] = []
+
+ async def generate_structured_output(
+ self,
+ messages: list[Msg],
+ structured_model: type[BaseModel] | dict,
+ **kwargs: Any,
+ ) -> StructuredResponse:
+ """Return the configured routing choice."""
+ self.calls.append(
+ {
+ "messages": [(m.role, m.get_text_content()) for m in messages],
+ "structured_model": structured_model,
+ },
+ )
+ return StructuredResponse(content={"choice": self.outcome})
+
+
+class ModelRouterMiddlewareTest(IsolatedAsyncioTestCase):
+ """Test routing through a full agent reply."""
+
+ def setUp(self) -> None:
+ """Create the primary and candidate chat models."""
+ self.primary = MockModel(model="primary")
+ self.fast = MockModel(model="fast-model")
+ self.reasoning = MockModel(model="reasoning-model")
+ for model in (self.primary, self.fast, self.reasoning):
+ model.set_responses(
+ [[ChatResponse(content=[TextBlock(text="ok")], is_last=True)]]
+ * 3,
+ )
+ self.candidates = [
+ ChatModelCandidate(
+ name="fast",
+ model=self.fast,
+ description="Short and simple requests.",
+ ),
+ ChatModelCandidate(
+ name="reasoning",
+ model=self.reasoning,
+ description="Complex reasoning is required.",
+ ),
+ ]
+
+ async def _reply(
+ self,
+ middleware: ModelRouterMiddleware,
+ *contents: list[TextBlock | DataBlock],
+ ) -> tuple[Agent, list[str]]:
+ """Run one reply per content through a routed agent and return the
+ agent and the model called in each reply."""
+ agent = Agent(
+ name="Friday",
+ system_prompt="Help the user.",
+ model=self.primary,
+ middlewares=[middleware],
+ injection_config=InjectionConfig(inject_runtime_state=False),
+ )
+ called = []
+ for content in contents:
+ async for event in agent.reply_stream(
+ UserMsg(name="user", content=content),
+ ):
+ if isinstance(event, ModelCallStartEvent):
+ called.append(event.model_name)
+ return agent, called
+
+ async def test_classifier_routes_each_reply(self) -> None:
+ """Each reply is routed by the classifier and the agent's own model
+ is restored afterwards."""
+ classifier = _MockClassifier(["reasoning", "fast"])
+ middleware = ModelRouterMiddleware(classifier, self.candidates)
+
+ agent, called = await self._reply(
+ middleware,
+ [TextBlock(text="Prove this theorem.")],
+ [TextBlock(text="Say hello.")],
+ )
+
+ self.assertListEqual(called, ["reasoning-model", "fast-model"])
+ self.assertIs(agent.model, self.primary)
+ self.assertListEqual(
+ classifier.calls,
+ [
+ {
+ "state": "Prove this theorem.",
+ "questions": {
+ "chat_model": {
+ "type": "choice",
+ "criteria": {
+ "fast": "Short and simple requests.",
+ "reasoning": "Complex reasoning is required.",
+ },
+ "instructions": (
+ "Select the most suitable chat model for "
+ "responding to the user input."
+ ),
+ },
+ },
+ },
+ {
+ "state": "Say hello.",
+ "questions": {
+ "chat_model": {
+ "type": "choice",
+ "criteria": {
+ "fast": "Short and simple requests.",
+ "reasoning": "Complex reasoning is required.",
+ },
+ "instructions": (
+ "Select the most suitable chat model for "
+ "responding to the user input."
+ ),
+ },
+ },
+ },
+ ],
+ )
+ self.assertDictEqual(
+ agent.state.middle_context,
+ {"ModelRouterMiddleware": {agent.state.reply_id: "fast"}},
+ )
+
+ async def test_resumed_reply_keeps_its_route(self) -> None:
+ """A reply resumed without a ReplyStartEvent reuses its route."""
+ middleware = ModelRouterMiddleware(
+ _MockClassifier(["reasoning"]),
+ self.candidates,
+ )
+ agent, _ = await self._reply(
+ middleware,
+ [TextBlock(text="Prove this theorem.")],
+ )
+ active = []
+
+ async def resume(**_: Any) -> Any:
+ active.append(agent.model)
+ yield ReplyStartEvent(
+ session_id=agent.state.session_id,
+ reply_id="another-reply",
+ name=agent.name,
+ )
+ active.append(agent.model)
+
+ async for _ in middleware.on_reply(agent, {"inputs": None}, resume):
+ pass
+
+ # The resumed reply keeps the route, a new one is routed again
+ self.assertListEqual(active, [self.reasoning, self.primary])
+ self.assertIs(agent.model, self.primary)
+
+ async def test_chat_model_routes_with_structured_output(self) -> None:
+ """A chat model routes through a structured choice."""
+ routing_model = _MockRoutingChatModel("fast")
+ middleware = ModelRouterMiddleware(routing_model, self.candidates)
+
+ _, called = await self._reply(middleware, [TextBlock(text="Hi.")])
+
+ self.assertListEqual(called, ["fast-model"])
+ self.assertListEqual(
+ routing_model.calls,
+ [
+ {
+ "messages": [
+ (
+ "system",
+ "Select the most suitable chat model for "
+ "responding to the user input.\n\n"
+ "Select exactly one candidate using these "
+ "criteria:\n"
+ "{\n"
+ ' "fast": "Short and simple requests.",\n'
+ ' "reasoning": "Complex reasoning is required."'
+ "\n}",
+ ),
+ ("user", "Hi."),
+ ],
+ "structured_model": {
+ "type": "object",
+ "properties": {
+ "choice": {
+ "type": "string",
+ "enum": ["fast", "reasoning"],
+ },
+ },
+ "required": ["choice"],
+ "additionalProperties": False,
+ },
+ },
+ ],
+ )
+
+ async def test_failures_keep_the_agents_model(self) -> None:
+ """A routing error, an unknown candidate and an input without text
+ all keep the agent's own model."""
+ classifier = _MockClassifier(
+ [RuntimeError("boom"), "unknown", "reasoning"],
+ )
+ middleware = ModelRouterMiddleware(classifier, self.candidates)
+
+ agent, called = await self._reply(
+ middleware,
+ [TextBlock(text="Error.")],
+ [TextBlock(text="Unknown.")],
+ [
+ DataBlock(
+ source=Base64Source(data="AA==", media_type="image/png"),
+ ),
+ ],
+ )
+
+ self.assertListEqual(called, ["primary", "primary", "primary"])
+ self.assertListEqual(
+ [_["state"] for _ in classifier.calls],
+ ["Error.", "Unknown."],
+ )
+ self.assertDictEqual(
+ agent.state.middle_context,
+ {"ModelRouterMiddleware": {agent.state.reply_id: None}},
+ )
+
+ def test_duplicate_candidates_are_rejected(self) -> None:
+ """Candidate names must be unique."""
+ with self.assertRaisesRegex(ValueError, "Duplicate"):
+ ModelRouterMiddleware(
+ _MockClassifier([]),
+ self.candidates + self.candidates[:1],
+ )