Build and run agents you can see, understand and trust.
Python agentchatbotlarge-language-modelsllmllm-agentmcpmulti-agentmulti-modalreact-agentrealtime-agentvoice-agentvoice-assistant
feat(classifier): add Jev support and chat model routing middleware (#2758)
qbc6d32be1dfcf1
diff --git a/README.md b/README.mdindex 30ab3ec6f..71acd4252 100644--- a/README.md+++ b/README.md@@ -73,6 +73,8 @@ rather than constraining them with strict prompts and opinionated orchestrations## News<!-- BEGIN NEWS -->+- **[2026-09] `FEAT`:** Model routing supported — use `ModelRouterMiddleware` to select an appropriate chat model for each reply.+- **[2026-09] `INTE`:** Jev classifier model supported through the TypeSafe SDK via `JevClassifierModel`.- **[2026-09] `FEAT`:** [agentscope-skill](https://github.com/agentscope-ai/skills/tree/main/skills/agentscope-skill) now supports AgentScope v2.- **[2026-09] `INTE`:** Support DashScope, OpenAI, Gemini and xAI realtime APIs in `RealtimeAgent`. [Example](https://github.com/agentscope-ai/agentscope/tree/main/examples/realtime) | [Docs](https://docs.agentscope.io/latest/en/building-blocks/realtime/speech-to-speech)- **[2026-09] `FEAT` `Experimental`:** Realtime voice agent supported. [Example](https://github.com/agentscope-ai/agentscope/tree/main/examples/realtime) | [Docs](https://docs.agentscope.io/latest/en/building-blocks/realtime/overview)@@ -81,8 +83,6 @@ rather than constraining them with strict prompts and opinionated orchestrations- **[2026-08] `INTE`:** DingTalk channel supported. [Docs](https://docs.agentscope.io/latest/en/deploy/channel/dingtalk)- **[2026-08] `FEAT`:** Console supported — test and debug agents in the terminal. [Example](https://github.com/agentscope-ai/agentscope/tree/main/examples/console) | [Docs](https://docs.agentscope.io/latest/en/building-blocks/console)- **[2026-08] `INTE`:** Feishu (Lark) and Discord channels supported. [Feishu](https://docs.agentscope.io/latest/en/deploy/channel/feishu) | [Discord](https://docs.agentscope.io/latest/en/deploy/channel/discord)-- **[2026-08] `FEAT`:** Channels supported — connect agents to IM platforms in agent service. [Example](https://github.com/agentscope-ai/agentscope/tree/main/examples/agent_service) | [Docs](https://docs.agentscope.io/latest/en/deploy/channel/overview)-- **[2026-08] `INTE`:** GitHub MCP Registry and ClawHub supported as built-in hubs. [Example](https://github.com/agentscope-ai/agentscope/tree/main/examples/agent_service) | [Docs](https://docs.agentscope.io/latest/en/deploy/hub)<!-- END NEWS -->[More news →](./docs/NEWS.md)diff --git a/README_zh.md b/README_zh.mdindex ddb8a9149..d3874b9cf 100644--- a/README_zh.md+++ b/README_zh.md@@ -72,6 +72,8 @@ AgentScope 的目标是充分发挥大模型的推理与工具调用能力,## 新闻<!-- BEGIN NEWS -->+- **[2026-09] `功能`:** 支持模型路由 —— 使用 `ModelRouterMiddleware` 为每次回复选择合适的聊天模型。+- **[2026-09] `集成`:** 通过 TypeSafe SDK 集成 Jev 分类模型 `JevClassifierModel`。- **[2026-09] `功能`:** [agentscope-skill](https://github.com/agentscope-ai/skills/tree/main/skills/agentscope-skill) 已适配 AgentScope v2。- **[2026-09] `集成`:** `RealtimeAgent` 支持 DashScope、OpenAI、Gemini 与 xAI 的实时语音 API。[样例](https://github.com/agentscope-ai/agentscope/tree/main/examples/realtime) | [文档](https://docs.agentscope.io/latest/zh/building-blocks/realtime/speech-to-speech)- **[2026-09] `功能` `实验性`:** 支持实时语音智能体。[样例](https://github.com/agentscope-ai/agentscope/tree/main/examples/realtime) | [文档](https://docs.agentscope.io/latest/zh/building-blocks/realtime/overview)@@ -80,8 +82,6 @@ AgentScope 的目标是充分发挥大模型的推理与工具调用能力,- **[2026-08] `集成`:** 支持钉钉(DingTalk)频道。[文档](https://docs.agentscope.io/latest/zh/deploy/channel/dingtalk)- **[2026-08] `功能`:** 支持 Console —— 在终端中测试与调试智能体。[样例](https://github.com/agentscope-ai/agentscope/tree/main/examples/console) | [文档](https://docs.agentscope.io/latest/zh/building-blocks/console)- **[2026-08] `集成`:** 支持飞书(Lark)与 Discord 频道。[飞书](https://docs.agentscope.io/latest/zh/deploy/channel/feishu) | [Discord](https://docs.agentscope.io/latest/zh/deploy/channel/discord)-- **[2026-08] `功能`:** 支持消息频道 —— 将智能体接入即时通讯平台。[样例](https://github.com/agentscope-ai/agentscope/tree/main/examples/agent_service) | [文档](https://docs.agentscope.io/latest/zh/deploy/channel/overview)-- **[2026-08] `集成`:** 内置集成 GitHub MCP Registry 与 ClawHub。[样例](https://github.com/agentscope-ai/agentscope/tree/main/examples/agent_service) | [文档](https://docs.agentscope.io/latest/zh/deploy/hub)<!-- END NEWS -->[更多新闻 →](./docs/NEWS_zh.md)diff --git a/docs/NEWS.md b/docs/NEWS.mdindex 597cc57c3..af3166115 100644--- a/docs/NEWS.md+++ b/docs/NEWS.md@@ -2,6 +2,8 @@<!-- The first 10 items are automatically synced to README.md and README_zh.md via GitHub Actions. --><!-- To update news in READMEs, modify this file and push to trigger the workflow. -->+- **[2026-09] `FEAT`:** Model routing supported — use `ModelRouterMiddleware` to select an appropriate chat model for each reply.+- **[2026-09] `INTE`:** Jev classifier model supported through the TypeSafe SDK via `JevClassifierModel`.- **[2026-09] `FEAT`:** [agentscope-skill](https://github.com/agentscope-ai/skills/tree/main/skills/agentscope-skill) now supports AgentScope v2.- **[2026-09] `INTE`:** Support DashScope, OpenAI, Gemini and xAI realtime APIs in `RealtimeAgent`. [Example](https://github.com/agentscope-ai/agentscope/tree/main/examples/realtime) | [Docs](https://docs.agentscope.io/latest/en/building-blocks/realtime/speech-to-speech)- **[2026-09] `FEAT` `Experimental`:** Realtime voice agent supported. [Example](https://github.com/agentscope-ai/agentscope/tree/main/examples/realtime) | [Docs](https://docs.agentscope.io/latest/en/building-blocks/realtime/overview)diff --git a/docs/NEWS_zh.md b/docs/NEWS_zh.mdindex 90b797afc..3070d4e2a 100644--- a/docs/NEWS_zh.md+++ b/docs/NEWS_zh.md@@ -2,6 +2,8 @@<!-- The first 10 items are automatically synced to README.md and README_zh.md via GitHub Actions. --><!-- To update news in READMEs, modify this file and push to trigger the workflow. -->+- **[2026-09] `功能`:** 支持模型路由 —— 使用 `ModelRouterMiddleware` 为每次回复选择合适的聊天模型。+- **[2026-09] `集成`:** 通过 TypeSafe SDK 集成 Jev 分类模型 `JevClassifierModel`。- **[2026-09] `功能`:** [agentscope-skill](https://github.com/agentscope-ai/skills/tree/main/skills/agentscope-skill) 已适配 AgentScope v2。- **[2026-09] `集成`:** `RealtimeAgent` 支持 DashScope、OpenAI、Gemini 与 xAI 的实时语音 API。[样例](https://github.com/agentscope-ai/agentscope/tree/main/examples/realtime) | [文档](https://docs.agentscope.io/latest/zh/building-blocks/realtime/speech-to-speech)- **[2026-09] `功能` `实验性`:** 支持实时语音智能体。[样例](https://github.com/agentscope-ai/agentscope/tree/main/examples/realtime) | [文档](https://docs.agentscope.io/latest/zh/building-blocks/realtime/overview)diff --git a/pyproject.toml b/pyproject.tomlindex 580a4de30..d815e2d73 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -54,11 +54,13 @@ dependencies = [model-gemini = ["google-genai"]model-ollama = ["ollama>=0.5.4"]model-xai = ["xai-sdk"]+classifier-jev = ["typesafe-sdk>=0.7"]models = ["agentscope[model-gemini]","agentscope[model-ollama]","agentscope[model-xai]",]+classifiers = ["agentscope[classifier-jev]"]# ------------ Service ------------service = ["fastapi",@@ -170,6 +172,7 @@ reme = ["agentscope[memory-reme]"]# ------------ Full ------------full = ["agentscope[models]",+ "agentscope[classifiers]","agentscope[service]","agentscope[storage-redis]","agentscope[storage-sql]",diff --git a/src/agentscope/classifier/__init__.py b/src/agentscope/classifier/__init__.pynew file mode 100644index 000000000..2811148f5--- /dev/null+++ b/src/agentscope/classifier/__init__.py@@ -0,0 +1,36 @@+# -*- coding: utf-8 -*-+"""Typed probabilistic classifier models."""++from ._base import ClassifierModelBase+from ._question import (+ BinaryCriteria,+ BinaryQuestion,+ ChoiceQuestion,+ ClassifierQuestion,+ ScoreQuestion,+)+from ._response import (+ BinaryAnswer,+ ChoiceAnswer,+ ClassifierAnswer,+ ClassifierResponse,+ ScoreAnswer,+)+from ._usage import ClassifierUsage+from ._jev import JevClassifierModel++__all__ = [+ "BinaryAnswer",+ "BinaryCriteria",+ "BinaryQuestion",+ "ChoiceAnswer",+ "ChoiceQuestion",+ "ClassifierAnswer",+ "ClassifierModelBase",+ "ClassifierQuestion",+ "ClassifierResponse",+ "ClassifierUsage",+ "JevClassifierModel",+ "ScoreAnswer",+ "ScoreQuestion",+]diff --git a/src/agentscope/classifier/_base.py b/src/agentscope/classifier/_base.pynew file mode 100644index 000000000..790dbfdd8--- /dev/null+++ b/src/agentscope/classifier/_base.py@@ -0,0 +1,59 @@+# -*- coding: utf-8 -*-+"""The base class for classifier models."""+from abc import ABC, abstractmethod+from typing import Any, Mapping++from pydantic import BaseModel++from ._question import ClassifierQuestion+from ._response import ClassifierResponse+from ..credential import CredentialBase+++class ClassifierModelBase(ABC):+ """Base class for models that return typed probabilistic decisions."""++ class Parameters(BaseModel):+ """Provider-specific classifier parameters."""++ def __init__(+ self,+ credential: CredentialBase,+ model: str,+ parameters: BaseModel | None = None,+ ) -> None:+ """Initialize a classifier model.++ Args:+ credential (`CredentialBase`):+ The credential used to authenticate with the provider.+ model (`str`):+ The classifier model name or alias.+ parameters (`BaseModel | None`, defaults to `None`):+ Provider-specific model parameters.+ """+ self.credential = credential+ self.model = model+ self.parameters = parameters or self.Parameters()++ @abstractmethod+ async def __call__(+ self,+ state: str | dict,+ questions: Mapping[str, ClassifierQuestion],+ **kwargs: Any,+ ) -> ClassifierResponse:+ """Evaluate the named questions against the shared input state.++ Args:+ state (`str | dict`):+ The text or JSON content shared by all questions.+ questions (`Mapping[str, ClassifierQuestion]`):+ The typed questions keyed by name.+ **kwargs (`Any`):+ Provider-specific per-call options.++ Returns:+ `ClassifierResponse`:+ The typed probabilistic answers keyed by question name.+ """diff --git a/src/agentscope/classifier/_jev/__init__.py b/src/agentscope/classifier/_jev/__init__.pynew file mode 100644index 000000000..69a0695f2--- /dev/null+++ b/src/agentscope/classifier/_jev/__init__.py@@ -0,0 +1,6 @@+# -*- coding: utf-8 -*-+"""The TypeSafe Jev classifier model."""++from ._model import JevClassifierModel++__all__ = ["JevClassifierModel"]diff --git a/src/agentscope/classifier/_jev/_model.py b/src/agentscope/classifier/_jev/_model.pynew file mode 100644index 000000000..03efc323a--- /dev/null+++ b/src/agentscope/classifier/_jev/_model.py@@ -0,0 +1,166 @@+# -*- coding: utf-8 -*-+"""The TypeSafe Jev classifier model implementation."""+from time import perf_counter+from typing import Any, Mapping++from pydantic import ConfigDict++from .._base import ClassifierModelBase+from .._question import (+ BinaryQuestion,+ ChoiceQuestion,+ ClassifierQuestion,+ ScoreQuestion,+)+from .._response import (+ BinaryAnswer,+ ChoiceAnswer,+ ClassifierAnswer,+ ClassifierResponse,+ ScoreAnswer,+)+from .._usage import ClassifierUsage+from ...credential import TypeSafeCredential+++def _to_sdk_question(question: ClassifierQuestion) -> Any:+ """Translate an AgentScope question to a TypeSafe question."""+ from typesafe_sdk import Choice, Noul, Score++ match question:+ case BinaryQuestion():+ # The binary question maps to TypeSafe's Noul primitive+ criteria = question.criteria and question.criteria.model_dump(+ exclude_unset=True,+ )+ return Noul(instructions=question.instructions, criteria=criteria)+ case ChoiceQuestion():+ return Choice(+ instructions=question.instructions,+ criteria=question.criteria,+ )+ case ScoreQuestion():+ return Score(+ instructions=question.instructions,+ criteria=question.criteria,+ )+++def _from_sdk_answer(answer: Any) -> ClassifierAnswer:+ """Translate a TypeSafe answer to an AgentScope answer."""+ match answer.type:+ case "noul":+ return BinaryAnswer(probability=answer.noul)+ case "choice":+ return ChoiceAnswer(+ choice=answer.choice,+ confidence=answer.confidence,+ probabilities=dict(answer.probabilities),+ )+ case "score":+ return ScoreAnswer(+ score=answer.score,+ confidence=answer.confidence,+ legend=dict(answer.legend),+ probabilities=dict(answer.probabilities),+ )+ raise ValueError(f"Unsupported TypeSafe answer type: {answer.type!r}.")+++class JevClassifierModel(ClassifierModelBase):+ """A classifier backed by TypeSafe's Jev System One API."""++ class Parameters(ClassifierModelBase.Parameters):+ """Provider-specific Jev model parameters."""++ model_config = ConfigDict(extra="forbid")++ def __init__(+ self,+ credential: TypeSafeCredential,+ model: str = "jev-latest",+ parameters: "JevClassifierModel.Parameters | None" = None,+ timeout: float = 30.0,+ max_retries: int = 2,+ retry_delay: float = 0.5,+ ) -> None:+ """Initialize the Jev classifier.++ Args:+ credential (`TypeSafeCredential`):+ The TypeSafe API credential.+ model (`str`, defaults to ``"jev-latest"``):+ The Jev model name or alias.+ parameters (`JevClassifierModel.Parameters | None`):+ Provider-specific model parameters.+ timeout (`float`, defaults to `30.0`):+ Per-request timeout in seconds.+ max_retries (`int`, defaults to `2`):+ Maximum retries after the initial request, done by the SDK.+ retry_delay (`float`, defaults to `0.5`):+ Initial exponential backoff delay in seconds.+ """+ try:+ from typesafe_sdk import AsyncTypeSafeClient, RetryPolicy+ except ImportError as error:+ raise ImportError(+ "JevClassifierModel requires a compatible optional "+ "'typesafe-sdk' dependency. Install it with "+ "`pip install 'agentscope[classifier-jev]'`.",+ ) from error++ super().__init__(credential, model, parameters)+ self.client = AsyncTypeSafeClient(+ api_key=credential.api_key.get_secret_value(),+ base_url=credential.base_url,+ model=model,+ timeout=timeout,+ retry=RetryPolicy(+ max_retries=max_retries,+ backoff_initial=retry_delay,+ ),+ )++ async def __call__(+ self,+ state: str | dict,+ questions: Mapping[str, ClassifierQuestion],+ **kwargs: Any,+ ) -> ClassifierResponse:+ """Call the TypeSafe System One API and normalize its response.++ Args:+ state (`str | dict`):+ The text or JSON content to classify.+ questions (`Mapping[str, ClassifierQuestion]`):+ The typed questions keyed by name.+ **kwargs (`Any`):+ Per-call TypeSafe options, e.g. ``extra_headers`` and+ ``extra_body``.++ Returns:+ `ClassifierResponse`:+ The typed probabilistic answers keyed by question name.+ """+ start_time = perf_counter()+ response = await self.client.system_one(+ state=state,+ questions={+ name: _to_sdk_question(question)+ for name, question in questions.items()+ },+ model=self.model,+ **kwargs,+ )+ return ClassifierResponse(+ model=response.model,+ content={+ name: _from_sdk_answer(answer)+ for name, answer in response.answers.items()+ },+ usage=ClassifierUsage(+ input_tokens=response.usage.input_tokens,+ output_tokens=response.usage.output_tokens,+ time=perf_counter() - start_time,+ ),+ )diff --git a/src/agentscope/classifier/_question.py b/src/agentscope/classifier/_question.pynew file mode 100644index 000000000..2d3370283--- /dev/null+++ b/src/agentscope/classifier/_question.py@@ -0,0 +1,69 @@+# -*- coding: utf-8 -*-+"""Provider-independent question types for classifier models."""+from typing import Annotated, Literal, TypeAlias++from pydantic import BaseModel, ConfigDict, Field+++class BinaryCriteria(BaseModel):+ """Optional descriptions for the two outcomes of a binary question."""++ model_config = ConfigDict(extra="forbid")++ true: str | None = None+ """What counts as a positive outcome."""++ false: str | None = None+ """What counts as a negative outcome."""+++class BinaryQuestion(BaseModel):+ """A question whose answer is a probability between zero and one."""++ model_config = ConfigDict(extra="forbid")++ type: Literal["binary"] = "binary"+ """The question type discriminator."""++ instructions: str | None = None+ """The question or statement to evaluate."""++ criteria: BinaryCriteria | None = None+ """Optional descriptions of the positive and negative outcomes."""+++class ChoiceQuestion(BaseModel):+ """A question that selects one named alternative."""++ model_config = ConfigDict(extra="forbid")++ type: Literal["choice"] = "choice"+ """The question type discriminator."""++ criteria: dict[str, str | None] = Field(min_length=1)+ """Named alternatives and optional descriptions."""++ instructions: str | None = None+ """The decision the classifier should make."""+++class ScoreQuestion(BaseModel):+ """A question that assigns an expected score using an ordered rubric."""++ model_config = ConfigDict(extra="forbid")++ type: Literal["score"] = "score"+ """The question type discriminator."""++ criteria: list[str] = Field(min_length=1)+ """Ordered score descriptions, starting at score zero."""++ instructions: str | None = None+ """The value the classifier should score."""+++#: A provider-independent classifier question.+ClassifierQuestion: TypeAlias = Annotated[+ BinaryQuestion | ChoiceQuestion | ScoreQuestion,+ Field(discriminator="type"),+]diff --git a/src/agentscope/classifier/_response.py b/src/agentscope/classifier/_response.pynew file mode 100644index 000000000..9043025eb--- /dev/null+++ b/src/agentscope/classifier/_response.py@@ -0,0 +1,103 @@+# -*- coding: utf-8 -*-+"""Provider-independent response types for classifier models."""+from dataclasses import dataclass, field+from typing import Literal, TypeAlias++from ._usage import ClassifierUsage+from .._utils._common import _generate_id, _generate_timestamp+from .._utils._mixin import DictMixin+from ..types import JSONSerializableObject+++@dataclass+class BinaryAnswer(DictMixin):+ """The probability of a positive answer to a binary question."""++ probability: float+ """The positive-outcome probability, between zero and one."""++ type: Literal["binary_answer"] = field(+ default_factory=lambda: "binary_answer",+ )+ """The answer type discriminator."""+++@dataclass+class ChoiceAnswer(DictMixin):+ """The selected alternative and its probability distribution."""++ choice: str+ """The alternative with the highest probability."""++ confidence: float+ """The provider-reported confidence in the selected alternative."""++ probabilities: dict[str, float]+ """The probability of each requested alternative."""++ type: Literal["choice_answer"] = field(+ default_factory=lambda: "choice_answer",+ )+ """The answer type discriminator."""+++@dataclass+class ScoreAnswer(DictMixin):+ """An expected score and its probability distribution."""++ score: float+ """The probability-weighted expected score."""++ confidence: float+ """The provider-reported confidence in the score."""++ legend: dict[int, JSONSerializableObject]+ """The requested descriptions keyed by integer score level.++ JSON serialization converts these integer keys to strings.+ """++ probabilities: dict[int, float]+ """The probability of each integer score level.++ JSON serialization converts these integer keys to strings.+ """++ type: Literal["score_answer"] = field(+ default_factory=lambda: "score_answer",+ )+ """The answer type discriminator."""+++#: A provider-independent classifier answer.+ClassifierAnswer: TypeAlias = BinaryAnswer | ChoiceAnswer | ScoreAnswer+++@dataclass+class ClassifierResponse(DictMixin):+ """The normalized response of a classifier model."""++ model: str+ """The concrete model that produced the answers."""++ content: dict[str, ClassifierAnswer]+ """Answers keyed by the corresponding question names."""++ usage: ClassifierUsage | None = field(default_factory=lambda: None)+ """Usage information reported for this invocation."""++ id: str = field(default_factory=_generate_id)+ """The unique response identifier."""++ created_at: str = field(default_factory=_generate_timestamp)+ """When the response was created."""++ type: Literal["classifier_response"] = field(+ default_factory=lambda: "classifier_response",+ )+ """The response type discriminator."""++ metadata: dict[str, JSONSerializableObject] = field(+ default_factory=dict,+ )+ """Additional provider-independent response metadata."""diff --git a/src/agentscope/classifier/_usage.py b/src/agentscope/classifier/_usage.pynew file mode 100644index 000000000..69016e6d4--- /dev/null+++ b/src/agentscope/classifier/_usage.py@@ -0,0 +1,25 @@+# -*- coding: utf-8 -*-+"""Usage information for classifier model calls."""+from dataclasses import dataclass, field+from typing import Literal++from .._utils._mixin import DictMixin+++@dataclass+class ClassifierUsage(DictMixin):+ """The usage of a classifier model API invocation."""++ time: float+ """The wall-clock time used in seconds."""++ input_tokens: int | None = field(default_factory=lambda: None)+ """The number of input tokens, if reported by the provider."""++ output_tokens: int | None = field(default_factory=lambda: None)+ """The number of output tokens, if reported by the provider."""++ type: Literal["classifier"] = field(+ default_factory=lambda: "classifier",+ )+ """The usage type discriminator."""diff --git a/src/agentscope/credential/__init__.py b/src/agentscope/credential/__init__.pyindex 71d1df981..83386b747 100644--- a/src/agentscope/credential/__init__.py+++ b/src/agentscope/credential/__init__.py@@ -9,6 +9,7 @@ from ._gemini import GeminiCredentialfrom ._moonshot import MoonshotCredentialfrom ._ollama import OllamaCredentialfrom ._openai import OpenAICredential+from ._typesafe import TypeSafeCredentialfrom ._xai import XAICredentialfrom ._volcengine import VolcengineCredentialfrom ._factory import CredentialFactory@@ -23,6 +24,7 @@ __all__ = ["MoonshotCredential","OllamaCredential","OpenAICredential",+ "TypeSafeCredential","XAICredential","VolcengineCredential","CredentialFactory",diff --git a/src/agentscope/credential/_typesafe.py b/src/agentscope/credential/_typesafe.pynew file mode 100644index 000000000..bf23073b4--- /dev/null+++ b/src/agentscope/credential/_typesafe.py@@ -0,0 +1,33 @@+# -*- coding: utf-8 -*-+"""The TypeSafe AI credential."""+from typing import Literal, TYPE_CHECKING, Type++from pydantic import ConfigDict, Field, SecretStr++from ._base import CredentialBase++if TYPE_CHECKING:+ from ..model import ChatModelBase+++class TypeSafeCredential(CredentialBase):+ """The credential used by TypeSafe System One models."""++ model_config = ConfigDict(title="TypeSafe AI API")++ type: Literal["typesafe_credential"] = "typesafe_credential"+ """The credential type discriminator."""++ api_key: SecretStr = Field(description="The TypeSafe API key.")+ """The TypeSafe API key."""++ base_url: str | None = Field(+ default=None,+ description="An optional custom TypeSafe API base URL.",+ )+ """An optional custom TypeSafe API base URL."""++ @classmethod+ def get_chat_model_class(cls) -> Type["ChatModelBase"]:+ """TypeSafe credentials serve classifier models only."""+ raise NotImplementedError(f"{cls.__name__} has no chat model.")diff --git a/src/agentscope/middleware/__init__.py b/src/agentscope/middleware/__init__.pyindex 822b56c43..e2b7f11b6 100644--- a/src/agentscope/middleware/__init__.py+++ b/src/agentscope/middleware/__init__.py@@ -11,6 +11,7 @@ from ._longterm_memory import ()from ._tracing import TracingMiddlewarefrom ._tts_middleware import TTSMiddleware+from ._model_router import ChatModelCandidate, ModelRouterMiddleware__all__ = ["MiddlewareBase",@@ -21,4 +22,6 @@ __all__ = ["TracingMiddleware","ReplyBudgetControlMiddleware","TTSMiddleware",+ "ChatModelCandidate",+ "ModelRouterMiddleware",]diff --git a/src/agentscope/middleware/_model_router.py b/src/agentscope/middleware/_model_router.pynew file mode 100644index 000000000..3d42c554a--- /dev/null+++ b/src/agentscope/middleware/_model_router.py@@ -0,0 +1,157 @@+# -*- coding: utf-8 -*-+"""Route each reply to a chat model chosen by a classifier."""+import json+from dataclasses import dataclass+from typing import AsyncGenerator, Callable, Sequence, TYPE_CHECKING++from ._base import MiddlewareBase+from .._logging import logger+from ..classifier import ChoiceAnswer, ChoiceQuestion, ClassifierModelBase+from ..event import ReplyStartEvent+from ..message import Msg, SystemMsg, UserMsg+from ..model import ChatModelBase++if TYPE_CHECKING:+ from ..agent import Agent+++_DEFAULT_INSTRUCTIONS = (+ "Select the most suitable chat model for responding to the user input."+)+++@dataclass+class ChatModelCandidate:+ """A named chat model and the criteria for selecting it."""++ name: str+ """The stable name returned by the classifier."""++ model: ChatModelBase+ """The chat model used when this candidate is selected."""++ description: str+ """When this candidate should be selected."""+++class ModelRouterMiddleware(MiddlewareBase):+ """Select the chat model of each reply by classifying the user input.++ The decision is made once per reply and kept when the reply resumes,+ e.g. after a user confirmation. When the routing model fails or picks+ an unknown candidate, the agent keeps its own model.+ """++ def __init__(+ self,+ classifier_model: ClassifierModelBase | ChatModelBase,+ candidates: Sequence[ChatModelCandidate],+ instructions: str = _DEFAULT_INSTRUCTIONS,+ ) -> None:+ """Initialize the model router.++ Args:+ classifier_model (`ClassifierModelBase | ChatModelBase`):+ The classifier or chat model used to select a candidate.+ candidates (`Sequence[ChatModelCandidate]`):+ The uniquely named chat model candidates.+ instructions (`str`):+ The instructions of the routing question.+ """+ names = [candidate.name for candidate in candidates]+ if len(set(names)) != len(names):+ raise ValueError(f"Duplicate chat model candidates: {names}")++ self.classifier_model = classifier_model+ self._models = {_.name: _.model for _ in candidates}+ criteria = {_.name: _.description for _ in candidates}+ self._question = ChoiceQuestion(+ instructions=instructions,+ criteria=criteria,+ )+ self._prompt = (+ f"{instructions}\n\n"+ "Select exactly one candidate using these criteria:\n"+ f"{json.dumps(criteria, ensure_ascii=False, indent=2)}"+ )+ self._schema = {+ "type": "object",+ "properties": {"choice": {"type": "string", "enum": names}},+ "required": ["choice"],+ "additionalProperties": False,+ }++ async def on_reply(+ self,+ agent: "Agent",+ input_kwargs: dict,+ next_handler: Callable[..., AsyncGenerator],+ ) -> AsyncGenerator:+ """Swap ``agent.model`` for the selected candidate during the reply."""+ key = await self.get_middleware_key()+ original_model = agent.model+ # A resumed reply keeps its route, a new one is routed on its start+ routed = agent.state.middle_context.get(key, {})+ agent.model = self._models.get(+ routed.get(agent.state.reply_id),+ original_model,+ )+ try:+ async for event in next_handler(**input_kwargs):+ if isinstance(event, ReplyStartEvent):+ name = await self._route(+ self._latest_user_text(input_kwargs["inputs"]),+ )+ agent.state.middle_context[key] = {event.reply_id: name}+ agent.model = self._models.get(name, original_model)+ yield event+ finally:+ agent.model = original_model++ async def _route(self, text: str | None) -> str | None:+ """Return the selected candidate name, `None` to keep the agent's."""+ if not text:+ return None+ try:+ if isinstance(self.classifier_model, ClassifierModelBase):+ res = await self.classifier_model(+ state=text,+ questions={"chat_model": self._question},+ )+ answer = res.content["chat_model"]+ name = (+ answer.choice if isinstance(answer, ChoiceAnswer) else None+ )+ else:+ res = await self.classifier_model.generate_structured_output(+ messages=[+ SystemMsg(name="system", content=self._prompt),+ UserMsg(name="user", content=text),+ ],+ structured_model=self._schema,+ )+ name = res.content["choice"]+ except Exception as error:+ logger.warning(+ "Chat model routing failed, keeping the agent's model: %s",+ error,+ )+ return None+ if name not in self._models:+ logger.warning(+ "Routing model selected unknown candidate %r, keeping the "+ "agent's model",+ name,+ )+ return None+ logger.debug("Routed the reply to chat model candidate %r", name)+ return name++ @staticmethod+ def _latest_user_text(inputs: object) -> str | None:+ """The text of the latest user message, if any."""+ msgs = inputs if isinstance(inputs, list) else [inputs]+ for msg in reversed(msgs):+ if isinstance(msg, Msg) and msg.role == "user":+ return msg.get_text_content()+ return Nonediff --git a/tests/classifier_jev_test.py b/tests/classifier_jev_test.pynew file mode 100644index 000000000..8c78cd65f--- /dev/null+++ b/tests/classifier_jev_test.py@@ -0,0 +1,205 @@+# -*- coding: utf-8 -*-+"""Tests for the TypeSafe Jev classifier adapter."""+from dataclasses import asdict+from types import SimpleNamespace+from typing import Any+from unittest import IsolatedAsyncioTestCase+from unittest.mock import AsyncMock, MagicMock, patch++import pytest++from utils import AnyValue+from agentscope.classifier import (+ BinaryCriteria,+ BinaryQuestion,+ ChoiceQuestion,+ JevClassifierModel,+ ScoreQuestion,+)+from agentscope.credential import TypeSafeCredential++typesafe_sdk = pytest.importorskip("typesafe_sdk")++A = AnyValue()+++class JevClassifierModelTest(IsolatedAsyncioTestCase):+ """Test request and response translation against official SDK types."""++ @patch("typesafe_sdk.AsyncTypeSafeClient")+ async def test_call_translates_all_question_types(+ self,+ client_cls: Any,+ ) -> None:+ """All supported questions and answers should round-trip."""+ client = MagicMock()+ client.system_one = AsyncMock(+ return_value=SimpleNamespace(+ model="jev-1.13.0",+ answers={+ "urgent": typesafe_sdk.NoulAnswer(type="noul", noul=0.8),+ "route": typesafe_sdk.ChoiceAnswer(+ type="choice",+ choice="billing",+ confidence=0.9,+ probabilities={"billing": 0.9, "support": 0.1},+ ),+ "priority": typesafe_sdk.ScoreAnswer(+ type="score",+ score=1.7,+ confidence=0.85,+ legend={0: "low", 1: "medium", 2: "high"},+ probabilities={0: 0.05, 1: 0.2, 2: 0.75},+ ),+ },+ usage=typesafe_sdk.Usage(input_tokens=120, output_tokens=3),+ ),+ )+ client_cls.return_value = client++ model = JevClassifierModel(+ credential=TypeSafeCredential(+ api_key="secret",+ base_url="https://typesafe.example",+ ),+ max_retries=4,+ retry_delay=0.25,+ )+ response = await model(+ state="I was charged twice.",+ questions={+ "urgent": BinaryQuestion(+ instructions="Is this urgent?",+ criteria=BinaryCriteria(true="Urgent."),+ ),+ "route": ChoiceQuestion(+ instructions="Select a route.",+ criteria={"billing": None, "support": None},+ ),+ "priority": ScoreQuestion(+ instructions="Rate priority.",+ criteria=["low", "medium", "high"],+ ),+ },+ extra_body={"trace": True},+ )++ self.assertDictEqual(+ client_cls.call_args.kwargs,+ {+ "api_key": "secret",+ "base_url": "https://typesafe.example",+ "model": "jev-latest",+ "timeout": 30.0,+ "retry": typesafe_sdk.RetryPolicy(+ max_retries=4,+ backoff_initial=0.25,+ ),+ },+ )+ call_kwargs = client.system_one.await_args.kwargs+ self.assertDictEqual(+ {+ **call_kwargs,+ "questions": {+ name: question.model_dump()+ for name, question in call_kwargs["questions"].items()+ },+ },+ {+ "state": "I was charged twice.",+ "questions": {+ "urgent": {+ "type": "noul",+ "instructions": "Is this urgent?",+ "criteria": {"true": "Urgent."},+ },+ "route": {+ "type": "choice",+ "instructions": "Select a route.",+ "criteria": {"billing": None, "support": None},+ },+ "priority": {+ "type": "score",+ "instructions": "Rate priority.",+ "criteria": ["low", "medium", "high"],+ },+ },+ "model": "jev-latest",+ "extra_body": {"trace": True},+ },+ )+ self.assertDictEqual(+ asdict(response),+ {+ "model": "jev-1.13.0",+ "content": {+ "urgent": {+ "probability": 0.8,+ "type": "binary_answer",+ },+ "route": {+ "choice": "billing",+ "confidence": 0.9,+ "probabilities": {+ "billing": 0.9,+ "support": 0.1,+ },+ "type": "choice_answer",+ },+ "priority": {+ "score": 1.7,+ "confidence": 0.85,+ "legend": {0: "low", 1: "medium", 2: "high"},+ "probabilities": {0: 0.05, 1: 0.2, 2: 0.75},+ "type": "score_answer",+ },+ },+ "usage": {+ "time": A,+ "input_tokens": 120,+ "output_tokens": 3,+ "type": "classifier",+ },+ "id": A,+ "created_at": A,+ "type": "classifier_response",+ "metadata": {},+ },+ )++ @patch("typesafe_sdk.AsyncTypeSafeClient")+ async def test_provider_error_is_raised(self, client_cls: Any) -> None:+ """Retries belong to the SDK, so its errors are raised as they are."""+ client = MagicMock()+ client.system_one = AsyncMock(side_effect=RuntimeError("failed"))+ client_cls.return_value = client+ model = JevClassifierModel(TypeSafeCredential(api_key="secret"))++ with self.assertRaisesRegex(RuntimeError, "failed"):+ await model(+ state="hello",+ questions={"route": ChoiceQuestion(criteria={"a": None})},+ )++ self.assertDictEqual(+ client_cls.call_args.kwargs,+ {+ "api_key": "secret",+ "base_url": None,+ "model": "jev-latest",+ "timeout": 30.0,+ "retry": typesafe_sdk.RetryPolicy(),+ },+ )++ async def test_incompatible_sdk_has_clear_error(self) -> None:+ """Missing SDK exports should produce an actionable error."""+ incompatible_sdk = SimpleNamespace(AsyncTypeSafeClient=MagicMock())++ with patch.dict("sys.modules", {"typesafe_sdk": incompatible_sdk}):+ with self.assertRaisesRegex(+ ImportError,+ "requires a compatible optional",+ ):+ JevClassifierModel(TypeSafeCredential(api_key="secret"))diff --git a/tests/classifier_test.py b/tests/classifier_test.pynew file mode 100644index 000000000..987f98f86--- /dev/null+++ b/tests/classifier_test.py@@ -0,0 +1,116 @@+# -*- coding: utf-8 -*-+"""Tests for the provider-independent classifier model contract."""+from dataclasses import asdict+from typing import Any, Mapping+from unittest import IsolatedAsyncioTestCase, TestCase++from pydantic import ValidationError++from utils import AnyString+from agentscope.classifier import (+ BinaryAnswer,+ BinaryQuestion,+ ChoiceQuestion,+ ClassifierModelBase,+ ClassifierQuestion,+ ClassifierResponse,+ ScoreQuestion,+)+from agentscope.credential import CredentialBase+from agentscope.message import TextBlock+++class _MockClassifier(ClassifierModelBase):+ """A minimal classifier used to exercise the base class."""++ def __init__(self) -> None:+ """Initialize the mock classifier."""+ super().__init__(CredentialBase(), "mock-classifier")++ async def __call__(+ self,+ state: str | dict,+ questions: Mapping[str, ClassifierQuestion],+ **kwargs: Any,+ ) -> ClassifierResponse:+ """Return a deterministic response."""+ return ClassifierResponse(+ model=self.model,+ content={"safe": BinaryAnswer(probability=0.75)},+ )+++class ClassifierQuestionTest(TestCase):+ """Validate the framework-owned question types."""++ def test_question_models(self) -> None:+ """Question models should preserve their complete typed structure."""+ choice = ChoiceQuestion(+ instructions="Select a route.",+ criteria={"billing": None, "support": "Technical support."},+ )+ score = ScoreQuestion(+ instructions="Rate urgency.",+ criteria=["Can wait.", "Handle today."],+ )++ self.assertDictEqual(+ choice.model_dump(),+ {+ "type": "choice",+ "criteria": {+ "billing": None,+ "support": "Technical support.",+ },+ "instructions": "Select a route.",+ },+ )+ self.assertDictEqual(+ score.model_dump(),+ {+ "type": "score",+ "criteria": ["Can wait.", "Handle today."],+ "instructions": "Rate urgency.",+ },+ )++ def test_empty_criteria_are_rejected(self) -> None:+ """Choice and score questions require at least one criterion."""+ with self.assertRaises(ValidationError):+ ChoiceQuestion(criteria={})+ with self.assertRaises(ValidationError):+ ScoreQuestion(criteria=[])++ def test_non_string_instructions_are_rejected(self) -> None:+ """Question instructions must be plain text."""+ with self.assertRaises(ValidationError):+ BinaryQuestion(instructions=TextBlock(text="hello"))+++class ClassifierModelBaseTest(IsolatedAsyncioTestCase):+ """Test the classifier model contract."""++ async def test_call(self) -> None:+ """A call returns the typed answers keyed by question name."""+ response = await _MockClassifier()(+ state="hello",+ questions={"safe": ChoiceQuestion(criteria={"yes": None})},+ )++ self.assertDictEqual(+ asdict(response),+ {+ "model": "mock-classifier",+ "content": {+ "safe": {+ "probability": 0.75,+ "type": "binary_answer",+ },+ },+ "usage": None,+ "id": AnyString(),+ "created_at": AnyString(),+ "type": "classifier_response",+ "metadata": {},+ },+ )diff --git a/tests/middleware_model_router_test.py b/tests/middleware_model_router_test.pynew file mode 100644index 000000000..542364624--- /dev/null+++ b/tests/middleware_model_router_test.py@@ -0,0 +1,300 @@+# -*- coding: utf-8 -*-+"""Tests for the model router middleware."""+from typing import Any, Mapping+from unittest import IsolatedAsyncioTestCase++from pydantic import BaseModel++from utils import MockModel+from agentscope.agent import Agent, InjectionConfig+from agentscope.classifier import (+ ChoiceAnswer,+ ClassifierModelBase,+ ClassifierQuestion,+ ClassifierResponse,+)+from agentscope.credential import CredentialBase+from agentscope.event import ModelCallStartEvent, ReplyStartEvent+from agentscope.message import (+ Base64Source,+ DataBlock,+ Msg,+ TextBlock,+ UserMsg,+)+from agentscope.middleware import ChatModelCandidate, ModelRouterMiddleware+from agentscope.model import ChatResponse, StructuredResponse+++class _MockClassifier(ClassifierModelBase):+ """A classifier that returns the configured choices in turn."""++ def __init__(self, outcomes: list[str | Exception]) -> None:+ """Initialize the classifier with choices or exceptions."""+ super().__init__(CredentialBase(), "mock-classifier")+ self.outcomes = outcomes+ self.calls: list[dict] = []++ async def __call__(+ self,+ state: str | dict,+ questions: Mapping[str, ClassifierQuestion],+ **kwargs: Any,+ ) -> ClassifierResponse:+ """Return the next configured routing choice."""+ self.calls.append(+ {+ "state": state,+ "questions": {k: v.model_dump() for k, v in questions.items()},+ },+ )+ outcome = self.outcomes[len(self.calls) - 1]+ if isinstance(outcome, Exception):+ raise outcome+ return ClassifierResponse(+ model=self.model,+ content={+ "chat_model": ChoiceAnswer(+ choice=outcome,+ confidence=0.9,+ probabilities={outcome: 0.9},+ ),+ },+ )+++class _MockRoutingChatModel(MockModel):+ """A chat model that returns the configured choice as structured output."""++ def __init__(self, outcome: str) -> None:+ """Initialize the chat model with its choice."""+ super().__init__(model="routing-chat-model")+ self.outcome = outcome+ self.calls: list[dict] = []++ async def generate_structured_output(+ self,+ messages: list[Msg],+ structured_model: type[BaseModel] | dict,+ **kwargs: Any,+ ) -> StructuredResponse:+ """Return the configured routing choice."""+ self.calls.append(+ {+ "messages": [(m.role, m.get_text_content()) for m in messages],+ "structured_model": structured_model,+ },+ )+ return StructuredResponse(content={"choice": self.outcome})+++class ModelRouterMiddlewareTest(IsolatedAsyncioTestCase):+ """Test routing through a full agent reply."""++ def setUp(self) -> None:+ """Create the primary and candidate chat models."""+ self.primary = MockModel(model="primary")+ self.fast = MockModel(model="fast-model")+ self.reasoning = MockModel(model="reasoning-model")+ for model in (self.primary, self.fast, self.reasoning):+ model.set_responses(+ [[ChatResponse(content=[TextBlock(text="ok")], is_last=True)]]+ * 3,+ )+ self.candidates = [+ ChatModelCandidate(+ name="fast",+ model=self.fast,+ description="Short and simple requests.",+ ),+ ChatModelCandidate(+ name="reasoning",+ model=self.reasoning,+ description="Complex reasoning is required.",+ ),+ ]++ async def _reply(+ self,+ middleware: ModelRouterMiddleware,+ *contents: list[TextBlock | DataBlock],+ ) -> tuple[Agent, list[str]]:+ """Run one reply per content through a routed agent and return the+ agent and the model called in each reply."""+ agent = Agent(+ name="Friday",+ system_prompt="Help the user.",+ model=self.primary,+ middlewares=[middleware],+ injection_config=InjectionConfig(inject_runtime_state=False),+ )+ called = []+ for content in contents:+ async for event in agent.reply_stream(+ UserMsg(name="user", content=content),+ ):+ if isinstance(event, ModelCallStartEvent):+ called.append(event.model_name)+ return agent, called++ async def test_classifier_routes_each_reply(self) -> None:+ """Each reply is routed by the classifier and the agent's own model+ is restored afterwards."""+ classifier = _MockClassifier(["reasoning", "fast"])+ middleware = ModelRouterMiddleware(classifier, self.candidates)++ agent, called = await self._reply(+ middleware,+ [TextBlock(text="Prove this theorem.")],+ [TextBlock(text="Say hello.")],+ )++ self.assertListEqual(called, ["reasoning-model", "fast-model"])+ self.assertIs(agent.model, self.primary)+ self.assertListEqual(+ classifier.calls,+ [+ {+ "state": "Prove this theorem.",+ "questions": {+ "chat_model": {+ "type": "choice",+ "criteria": {+ "fast": "Short and simple requests.",+ "reasoning": "Complex reasoning is required.",+ },+ "instructions": (+ "Select the most suitable chat model for "+ "responding to the user input."+ ),+ },+ },+ },+ {+ "state": "Say hello.",+ "questions": {+ "chat_model": {+ "type": "choice",+ "criteria": {+ "fast": "Short and simple requests.",+ "reasoning": "Complex reasoning is required.",+ },+ "instructions": (+ "Select the most suitable chat model for "+ "responding to the user input."+ ),+ },+ },+ },+ ],+ )+ self.assertDictEqual(+ agent.state.middle_context,+ {"ModelRouterMiddleware": {agent.state.reply_id: "fast"}},+ )++ async def test_resumed_reply_keeps_its_route(self) -> None:+ """A reply resumed without a ReplyStartEvent reuses its route."""+ middleware = ModelRouterMiddleware(+ _MockClassifier(["reasoning"]),+ self.candidates,+ )+ agent, _ = await self._reply(+ middleware,+ [TextBlock(text="Prove this theorem.")],+ )+ active = []++ async def resume(**_: Any) -> Any:+ active.append(agent.model)+ yield ReplyStartEvent(+ session_id=agent.state.session_id,+ reply_id="another-reply",+ name=agent.name,+ )+ active.append(agent.model)++ async for _ in middleware.on_reply(agent, {"inputs": None}, resume):+ pass++ # The resumed reply keeps the route, a new one is routed again+ self.assertListEqual(active, [self.reasoning, self.primary])+ self.assertIs(agent.model, self.primary)++ async def test_chat_model_routes_with_structured_output(self) -> None:+ """A chat model routes through a structured choice."""+ routing_model = _MockRoutingChatModel("fast")+ middleware = ModelRouterMiddleware(routing_model, self.candidates)++ _, called = await self._reply(middleware, [TextBlock(text="Hi.")])++ self.assertListEqual(called, ["fast-model"])+ self.assertListEqual(+ routing_model.calls,+ [+ {+ "messages": [+ (+ "system",+ "Select the most suitable chat model for "+ "responding to the user input.\n\n"+ "Select exactly one candidate using these "+ "criteria:\n"+ "{\n"+ ' "fast": "Short and simple requests.",\n'+ ' "reasoning": "Complex reasoning is required."'+ "\n}",+ ),+ ("user", "Hi."),+ ],+ "structured_model": {+ "type": "object",+ "properties": {+ "choice": {+ "type": "string",+ "enum": ["fast", "reasoning"],+ },+ },+ "required": ["choice"],+ "additionalProperties": False,+ },+ },+ ],+ )++ async def test_failures_keep_the_agents_model(self) -> None:+ """A routing error, an unknown candidate and an input without text+ all keep the agent's own model."""+ classifier = _MockClassifier(+ [RuntimeError("boom"), "unknown", "reasoning"],+ )+ middleware = ModelRouterMiddleware(classifier, self.candidates)++ agent, called = await self._reply(+ middleware,+ [TextBlock(text="Error.")],+ [TextBlock(text="Unknown.")],+ [+ DataBlock(+ source=Base64Source(data="AA==", media_type="image/png"),+ ),+ ],+ )++ self.assertListEqual(called, ["primary", "primary", "primary"])+ self.assertListEqual(+ [_["state"] for _ in classifier.calls],+ ["Error.", "Unknown."],+ )+ self.assertDictEqual(+ agent.state.middle_context,+ {"ModelRouterMiddleware": {agent.state.reply_id: None}},+ )++ def test_duplicate_candidates_are_rejected(self) -> None:+ """Candidate names must be unique."""+ with self.assertRaisesRegex(ValueError, "Duplicate"):+ ModelRouterMiddleware(+ _MockClassifier([]),+ self.candidates + self.candidates[:1],+ )