A programming framework for agentic AI
add router to message handler Routing Predicates for message_handler in RoutedAgent #397 (#446)
Eric Zhu2345557c62cd
diff --git a/python/packages/autogen-core/docs/src/getting-started/message-and-communication.ipynb b/python/packages/autogen-core/docs/src/getting-started/message-and-communication.ipynbindex d90835319..bdbdafa98 100644--- a/python/packages/autogen-core/docs/src/getting-started/message-and-communication.ipynb+++ b/python/packages/autogen-core/docs/src/getting-started/message-and-communication.ipynb@@ -59,16 +59,25 @@"cell_type": "markdown","metadata": {},"source": [- "### Message Handlers\n",+ "## Message Handlers\n","\n","When an agent receives a message the runtime will invoke the agent's message handler\n","({py:meth}`~autogen_core.base.Agent.on_message`) which should implement the agents message handling logic.\n","If this message cannot be handled by the agent, the agent should raise a\n","{py:class}`~autogen_core.base.exceptions.CantHandleException`.\n","\n",- "For convenience, the {py:class}`~autogen_core.components.RoutedAgent` base class\n",- "provides the {py:meth}`~autogen_core.components.message_handler` decorator\n",- "for associating message types with message handlers,\n",+ "The base class {py:class}`~autogen_core.base.BaseAgent` provides no message handling logic\n",+ "and implementing the {py:meth}`~autogen_core.base.Agent.on_message` method directly is not recommended\n",+ "unless for the advanced use cases.\n",+ "\n",+ "Developers should start with implementing the {py:class}`~autogen_core.components.RoutedAgent` base class\n",+ "which provides built-in message routing capability.\n",+ "\n",+ "### Routing Messages by Type\n",+ "\n",+ "The {py:class}`~autogen_core.components.RoutedAgent` base class provides a mechanism\n",+ "for associating message types with message handlers \n",+ "with the {py:meth}`~autogen_core.components.message_handler` decorator,\n","so developers do not need to implement the {py:meth}`~autogen_core.base.Agent.on_message` method.\n","\n","For example, the following type-routed agent responds to `TextMessage` and `ImageMessage`\n",@@ -153,6 +162,93 @@"await runtime.stop_when_idle()"]},+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "### Routing Messages of the Same Type\n",+ "\n",+ "In some scenarios, it is useful to route messages of the same type to different handlers.\n",+ "For examples, messages from different sender agents should be handled differently.\n",+ "You can use the `match` parameter of the {py:meth}`~autogen_core.components.message_handler` decorator.\n",+ "\n",+ "The `match` parameter associates handlers for the same message type\n",+ "to a specific message -- it is secondary to the message type routing. \n",+ "It accepts a callable that takes the message and \n",+ "{py:class}`~autogen_core.base.MessageContext` as arguments, and\n",+ "returns a boolean indicating whether the message should be handled by the decorated handler.\n",+ "The callable is checked in the alphabetical order of the handlers.\n",+ "\n",+ "Here is an example of an agent that routes messages based on the sender agent\n",+ "using the `match` parameter:"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 5,+ "metadata": {},+ "outputs": [],+ "source": [+ "class RoutedBySenderAgent(RoutedAgent):\n",+ " @message_handler(match=lambda msg, ctx: msg.source.startswith(\"user1\")) # type: ignore\n",+ " async def on_user1_message(self, message: TextMessage, ctx: MessageContext) -> None:\n",+ " print(f\"Hello from user 1 handler, {message.source}, you said {message.content}!\")\n",+ "\n",+ " @message_handler(match=lambda msg, ctx: msg.source.startswith(\"user2\")) # type: ignore\n",+ " async def on_user2_message(self, message: TextMessage, ctx: MessageContext) -> None:\n",+ " print(f\"Hello from user 2 handler, {message.source}, you said {message.content}!\")\n",+ "\n",+ " @message_handler(match=lambda msg, ctx: msg.source.startswith(\"user2\")) # type: ignore\n",+ " async def on_image_message(self, message: ImageMessage, ctx: MessageContext) -> None:\n",+ " print(f\"Hello, {message.source}, you sent me {message.url}!\")"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "The above agent uses the `source` field of the message to determine the sender agent.\n",+ "You can also use the `sender` field of {py:class}`~autogen_core.base.MessageContext` to determine the sender agent\n",+ "using the agent ID if available.\n",+ "\n",+ "Let's test this agent with messages with different `source` values:"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 6,+ "metadata": {},+ "outputs": [+ {+ "name": "stdout",+ "output_type": "stream",+ "text": [+ "Hello from user 1 handler, user1-test, you said Hello, World!!\n",+ "Hello from user 2 handler, user2-test, you said Hello, World!!\n",+ "Hello, user2-test, you sent me https://example.com/image.jpg!\n"+ ]+ }+ ],+ "source": [+ "runtime = SingleThreadedAgentRuntime()\n",+ "await runtime.register(\"my_agent\", lambda: RoutedBySenderAgent(\"Routed by sender agent\"))\n",+ "runtime.start()\n",+ "agent_id = AgentId(\"my_agent\", \"default\")\n",+ "await runtime.send_message(TextMessage(content=\"Hello, World!\", source=\"user1-test\"), agent_id)\n",+ "await runtime.send_message(TextMessage(content=\"Hello, World!\", source=\"user2-test\"), agent_id)\n",+ "await runtime.send_message(ImageMessage(url=\"https://example.com/image.jpg\", source=\"user1-test\"), agent_id)\n",+ "await runtime.send_message(ImageMessage(url=\"https://example.com/image.jpg\", source=\"user2-test\"), agent_id)\n",+ "await runtime.stop_when_idle()"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "In the above example, the first `ImageMessage` is not handled because the `source` field\n",+ "of the message does not match the handler's `match` condition."+ ]+ },{"cell_type": "markdown","metadata": {},@@ -189,7 +285,7 @@},{"cell_type": "code",- "execution_count": 5,+ "execution_count": 7,"metadata": {},"outputs": [],"source": [@@ -236,7 +332,7 @@},{"cell_type": "code",- "execution_count": 6,+ "execution_count": 8,"metadata": {},"outputs": [{@@ -314,7 +410,7 @@},{"cell_type": "code",- "execution_count": 7,+ "execution_count": 9,"metadata": {},"outputs": [],"source": [@@ -365,7 +461,7 @@},{"cell_type": "code",- "execution_count": 8,+ "execution_count": 10,"metadata": {},"outputs": [{@@ -434,7 +530,7 @@},{"cell_type": "code",- "execution_count": 9,+ "execution_count": 11,"metadata": {},"outputs": [],"source": [@@ -468,7 +564,7 @@},{"cell_type": "code",- "execution_count": 10,+ "execution_count": 12,"metadata": {},"outputs": [{diff --git a/python/packages/autogen-core/src/autogen_core/application/_single_threaded_agent_runtime.py b/python/packages/autogen-core/src/autogen_core/application/_single_threaded_agent_runtime.pyindex 8fba5894e..efaecf1ec 100644--- a/python/packages/autogen-core/src/autogen_core/application/_single_threaded_agent_runtime.py+++ b/python/packages/autogen-core/src/autogen_core/application/_single_threaded_agent_runtime.py@@ -32,6 +32,10 @@ from ._helpers import SubscriptionManager, get_impllogger = logging.getLogger("autogen_core")event_logger = logging.getLogger("autogen_core.events")+# We use a type parameter in some functions which shadows the built-in `type` function.+# This is a workaround to avoid shadowing the built-in `type` function.+type_func_alias = type+@dataclass(kw_only=True)class PublishMessageEnvelope:@@ -546,7 +550,9 @@ class SingleThreadedAgentRuntime(AgentRuntime):agent_instance = await self._get_agent(id)if not isinstance(agent_instance, type):- raise TypeError(f"Agent with name {id.type} is not of type {type.__name__}")+ raise TypeError(+ f"Agent with name {id.type} is not of type {type.__name__}. It is of type {type_func_alias(agent_instance).__name__}"+ )return agent_instancediff --git a/python/packages/autogen-core/src/autogen_core/components/_routed_agent.py b/python/packages/autogen-core/src/autogen_core/components/_routed_agent.pyindex c7b57adcd..77362091a 100644--- a/python/packages/autogen-core/src/autogen_core/components/_routed_agent.py+++ b/python/packages/autogen-core/src/autogen_core/components/_routed_agent.py@@ -6,6 +6,7 @@ from typing import (Callable,Coroutine,Dict,+ List,Literal,Protocol,Sequence,@@ -25,18 +26,21 @@ from ._type_helpers import AnyType, get_typeslogger = logging.getLogger("autogen_core")-ReceivesT = TypeVar("ReceivesT", contravariant=True)+ReceivesT = TypeVar("ReceivesT")ProducesT = TypeVar("ProducesT", covariant=True)# TODO: Generic typevar bound binding U to agent type# Can't do because python doesnt support it+# Pyright and mypy disagree on the variance of ReceivesT. Mypy thinks it should be contravariant here.+# Revisit this later to see if we can remove the ignore.@runtime_checkable-class MessageHandler(Protocol[ReceivesT, ProducesT]):+class MessageHandler(Protocol[ReceivesT, ProducesT]): # type: ignoretarget_types: Sequence[type]produces_types: Sequence[type]is_message_handler: Literal[True]+ router: Callable[[ReceivesT, MessageContext], bool]async def __call__(self, message: ReceivesT, ctx: MessageContext) -> ProducesT: ...@@ -55,6 +59,19 @@ def message_handler(def message_handler(func: None = None,*,+ match: None = ...,+ strict: bool = ...,+) -> Callable[+ [Callable[[Any, ReceivesT, MessageContext], Coroutine[Any, Any, ProducesT]]],+ MessageHandler[ReceivesT, ProducesT],+]: ...+++@overload+def message_handler(+ func: None = None,+ *,+ match: Callable[[ReceivesT, MessageContext], bool],strict: bool = ...,) -> Callable[[Callable[[Any, ReceivesT, MessageContext], Coroutine[Any, Any, ProducesT]]],@@ -66,6 +83,7 @@ def message_handler(func: None | Callable[[Any, ReceivesT, MessageContext], Coroutine[Any, Any, ProducesT]] = None,*,strict: bool = True,+ match: None | Callable[[ReceivesT, MessageContext], bool] = None,) -> (Callable[[Callable[[Any, ReceivesT, MessageContext], Coroutine[Any, Any, ProducesT]]],@@ -118,6 +136,7 @@ def message_handler(wrapper_handler.target_types = list(target_types)wrapper_handler.produces_types = list(return_types)wrapper_handler.is_message_handler = True+ wrapper_handler.router = match or (lambda _message, _ctx: True)return wrapper_handler@@ -134,16 +153,17 @@ class RoutedAgent(BaseAgent):# Self is already bound to the handlersself._handlers: Dict[Type[Any],- Callable[[Any, MessageContext], Coroutine[Any, Any, Any | None]],+ List[MessageHandler[Any, Any]],] = {}+ # Iterate over all attributes in alphabetical order and find message handlers.for attr in dir(self):if callable(getattr(self, attr, None)):handler = getattr(self, attr)if hasattr(handler, "is_message_handler"):message_handler = cast(MessageHandler[Any, Any], handler)for target_type in message_handler.target_types:- self._handlers[target_type] = message_handler+ self._handlers.setdefault(target_type, []).append(message_handler)for message_type in self._handlers.keys():for serializer in try_get_known_serializers_for_type(message_type):@@ -153,11 +173,14 @@ class RoutedAgent(BaseAgent):async def on_message(self, message: Any, ctx: MessageContext) -> Any | None:key_type: Type[Any] = type(message) # type: ignore- handler = self._handlers.get(key_type) # type: ignore- if handler is not None:- return await handler(message, ctx)- else:- return await self.on_unhandled_message(message, ctx) # type: ignore+ handlers = self._handlers.get(key_type) # type: ignore+ if handlers is not None:+ # Iterate over all handlers for this matching message type.+ # Call the first handler whose router returns True and then return the result.+ for h in handlers:+ if h.router(message, ctx):+ return await h(message, ctx)+ return await self.on_unhandled_message(message, ctx) # type: ignoreasync def on_unhandled_message(self, message: Any, ctx: MessageContext) -> None:logger.info(f"Unhandled message: {message}")diff --git a/python/packages/autogen-core/tests/test_routed_agent.py b/python/packages/autogen-core/tests/test_routed_agent.pyindex f70cb3841..3d0c79a38 100644--- a/python/packages/autogen-core/tests/test_routed_agent.py+++ b/python/packages/autogen-core/tests/test_routed_agent.py@@ -1,11 +1,11 @@-from dataclasses import dataclass-import pytestimport logging+from dataclasses import dataclass+from typing import Callable, cast+import pytestfrom autogen_core.application import SingleThreadedAgentRuntime-from autogen_core.components import TypeSubscription-from autogen_core.base import TopicId-+from autogen_core.base import AgentId, MessageContext, TopicId+from autogen_core.components import RoutedAgent, TypeSubscription, message_handlerfrom test_utils import LoopbackAgent@@ -13,12 +13,104 @@ from test_utils import LoopbackAgentclass UnhandledMessageType: ...+@dataclass+class MessageType: ...+++class CounterAgent(RoutedAgent):+ def __init__(self) -> None:+ super().__init__("A loop back agent.")+ self.num_calls_rpc = 0+ self.num_calls_broadcast = 0++ @message_handler(match=lambda _, ctx: ctx.is_rpc)+ async def on_rpc_message(self, message: MessageType, ctx: MessageContext) -> MessageType:+ self.num_calls_rpc += 1+ return message++ @message_handler(match=lambda _, ctx: not ctx.is_rpc)+ async def on_broadcast_message(self, message: MessageType, ctx: MessageContext) -> None:+ self.num_calls_broadcast += 1++@pytest.mark.asyncioasync def test_routed_agent(caplog: pytest.LogCaptureFixture) -> None:runtime = SingleThreadedAgentRuntime()with caplog.at_level(logging.INFO):- await runtime.register("loopback", lambda: LoopbackAgent(), lambda: [TypeSubscription("default", "loopback")])+ await runtime.register("loopback", LoopbackAgent, lambda: [TypeSubscription("default", "loopback")])runtime.start()await runtime.publish_message(UnhandledMessageType(), topic_id=TopicId("default", "default"))await runtime.stop_when_idle()assert any("Unhandled message: " in e.message for e in caplog.records)+++async def test_message_handler_router() -> None:+ runtime = SingleThreadedAgentRuntime()+ await runtime.register("counter", CounterAgent, lambda: [TypeSubscription("default", "counter")])+ agent_id = AgentId(type="counter", key="default")++ # Send a broadcast message.+ runtime.start()+ await runtime.publish_message(MessageType(), topic_id=TopicId("default", "default"))+ await runtime.stop_when_idle()+ agent = await runtime.try_get_underlying_agent_instance(agent_id, type=CounterAgent)+ assert agent.num_calls_broadcast == 1+ assert agent.num_calls_rpc == 0++ # Send an RPC message.+ runtime.start()+ await runtime.send_message(MessageType(), recipient=agent_id)+ await runtime.stop_when_idle()+ agent = await runtime.try_get_underlying_agent_instance(agent_id, type=CounterAgent)+ assert agent.num_calls_broadcast == 1+ assert agent.num_calls_rpc == 1+++@dataclass+class TestMessage:+ value: str+++class RoutedAgentMessageCustomMatch(RoutedAgent):+ def __init__(self) -> None:+ super().__init__("")+ self.handler_one_called = False+ self.handler_two_called = False++ @staticmethod+ def match_one(message: TestMessage, ctx: MessageContext) -> bool:+ return message.value == "one"++ @message_handler(match=match_one)+ async def handler_one(self, message: TestMessage, ctx: MessageContext) -> None:+ self.handler_one_called = True++ @message_handler(match=cast(Callable[[TestMessage, MessageContext], bool], lambda msg, ctx: msg.value == "two"))+ async def handler_two(self, message: TestMessage, ctx: MessageContext) -> None:+ self.handler_two_called = True++async def test_routed_agent_message_matching() -> None:+ runtime = SingleThreadedAgentRuntime()+ await runtime.register("message_match", RoutedAgentMessageCustomMatch)+ agent_id = AgentId(type="message_match", key="default")++ agent = await runtime.try_get_underlying_agent_instance(agent_id, type=RoutedAgentMessageCustomMatch)+ assert agent is not None+ assert agent.handler_one_called == False+ assert agent.handler_two_called == False++ runtime.start()+ await runtime.send_message(TestMessage("one"), recipient=agent_id)+ await runtime.stop_when_idle()+ agent = await runtime.try_get_underlying_agent_instance(agent_id, type=RoutedAgentMessageCustomMatch)+ assert agent.handler_one_called == True+ assert agent.handler_two_called == False++ runtime.start()+ await runtime.send_message(TestMessage("two"), recipient=agent_id)+ await runtime.stop_when_idle()+ agent = await runtime.try_get_underlying_agent_instance(agent_id, type=RoutedAgentMessageCustomMatch)+ assert agent.handler_one_called == True+ assert agent.handler_two_called == True