Skip to content

A programming framework for agentic AI

add router to message handler Routing Predicates for message_handler in RoutedAgent #397 (#446)

Eric Zhu2345557c62cd
diff --git a/python/packages/autogen-core/docs/src/getting-started/message-and-communication.ipynb b/python/packages/autogen-core/docs/src/getting-started/message-and-communication.ipynb
index d90835319..bdbdafa98 100644
--- a/python/packages/autogen-core/docs/src/getting-started/message-and-communication.ipynb
+++ b/python/packages/autogen-core/docs/src/getting-started/message-and-communication.ipynb
@@ -59,16 +59,25 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "### Message Handlers\n",
+ "## Message Handlers\n",
"\n",
"When an agent receives a message the runtime will invoke the agent's message handler\n",
"({py:meth}`~autogen_core.base.Agent.on_message`) which should implement the agents message handling logic.\n",
"If this message cannot be handled by the agent, the agent should raise a\n",
"{py:class}`~autogen_core.base.exceptions.CantHandleException`.\n",
"\n",
- "For convenience, the {py:class}`~autogen_core.components.RoutedAgent` base class\n",
- "provides the {py:meth}`~autogen_core.components.message_handler` decorator\n",
- "for associating message types with message handlers,\n",
+ "The base class {py:class}`~autogen_core.base.BaseAgent` provides no message handling logic\n",
+ "and implementing the {py:meth}`~autogen_core.base.Agent.on_message` method directly is not recommended\n",
+ "unless for the advanced use cases.\n",
+ "\n",
+ "Developers should start with implementing the {py:class}`~autogen_core.components.RoutedAgent` base class\n",
+ "which provides built-in message routing capability.\n",
+ "\n",
+ "### Routing Messages by Type\n",
+ "\n",
+ "The {py:class}`~autogen_core.components.RoutedAgent` base class provides a mechanism\n",
+ "for associating message types with message handlers \n",
+ "with the {py:meth}`~autogen_core.components.message_handler` decorator,\n",
"so developers do not need to implement the {py:meth}`~autogen_core.base.Agent.on_message` method.\n",
"\n",
"For example, the following type-routed agent responds to `TextMessage` and `ImageMessage`\n",
@@ -153,6 +162,93 @@
"await runtime.stop_when_idle()"
]
},
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Routing Messages of the Same Type\n",
+ "\n",
+ "In some scenarios, it is useful to route messages of the same type to different handlers.\n",
+ "For examples, messages from different sender agents should be handled differently.\n",
+ "You can use the `match` parameter of the {py:meth}`~autogen_core.components.message_handler` decorator.\n",
+ "\n",
+ "The `match` parameter associates handlers for the same message type\n",
+ "to a specific message -- it is secondary to the message type routing. \n",
+ "It accepts a callable that takes the message and \n",
+ "{py:class}`~autogen_core.base.MessageContext` as arguments, and\n",
+ "returns a boolean indicating whether the message should be handled by the decorated handler.\n",
+ "The callable is checked in the alphabetical order of the handlers.\n",
+ "\n",
+ "Here is an example of an agent that routes messages based on the sender agent\n",
+ "using the `match` parameter:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "class RoutedBySenderAgent(RoutedAgent):\n",
+ " @message_handler(match=lambda msg, ctx: msg.source.startswith(\"user1\")) # type: ignore\n",
+ " async def on_user1_message(self, message: TextMessage, ctx: MessageContext) -> None:\n",
+ " print(f\"Hello from user 1 handler, {message.source}, you said {message.content}!\")\n",
+ "\n",
+ " @message_handler(match=lambda msg, ctx: msg.source.startswith(\"user2\")) # type: ignore\n",
+ " async def on_user2_message(self, message: TextMessage, ctx: MessageContext) -> None:\n",
+ " print(f\"Hello from user 2 handler, {message.source}, you said {message.content}!\")\n",
+ "\n",
+ " @message_handler(match=lambda msg, ctx: msg.source.startswith(\"user2\")) # type: ignore\n",
+ " async def on_image_message(self, message: ImageMessage, ctx: MessageContext) -> None:\n",
+ " print(f\"Hello, {message.source}, you sent me {message.url}!\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The above agent uses the `source` field of the message to determine the sender agent.\n",
+ "You can also use the `sender` field of {py:class}`~autogen_core.base.MessageContext` to determine the sender agent\n",
+ "using the agent ID if available.\n",
+ "\n",
+ "Let's test this agent with messages with different `source` values:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Hello from user 1 handler, user1-test, you said Hello, World!!\n",
+ "Hello from user 2 handler, user2-test, you said Hello, World!!\n",
+ "Hello, user2-test, you sent me https://example.com/image.jpg!\n"
+ ]
+ }
+ ],
+ "source": [
+ "runtime = SingleThreadedAgentRuntime()\n",
+ "await runtime.register(\"my_agent\", lambda: RoutedBySenderAgent(\"Routed by sender agent\"))\n",
+ "runtime.start()\n",
+ "agent_id = AgentId(\"my_agent\", \"default\")\n",
+ "await runtime.send_message(TextMessage(content=\"Hello, World!\", source=\"user1-test\"), agent_id)\n",
+ "await runtime.send_message(TextMessage(content=\"Hello, World!\", source=\"user2-test\"), agent_id)\n",
+ "await runtime.send_message(ImageMessage(url=\"https://example.com/image.jpg\", source=\"user1-test\"), agent_id)\n",
+ "await runtime.send_message(ImageMessage(url=\"https://example.com/image.jpg\", source=\"user2-test\"), agent_id)\n",
+ "await runtime.stop_when_idle()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "In the above example, the first `ImageMessage` is not handled because the `source` field\n",
+ "of the message does not match the handler's `match` condition."
+ ]
+ },
{
"cell_type": "markdown",
"metadata": {},
@@ -189,7 +285,7 @@
},
{
"cell_type": "code",
- "execution_count": 5,
+ "execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
@@ -236,7 +332,7 @@
},
{
"cell_type": "code",
- "execution_count": 6,
+ "execution_count": 8,
"metadata": {},
"outputs": [
{
@@ -314,7 +410,7 @@
},
{
"cell_type": "code",
- "execution_count": 7,
+ "execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
@@ -365,7 +461,7 @@
},
{
"cell_type": "code",
- "execution_count": 8,
+ "execution_count": 10,
"metadata": {},
"outputs": [
{
@@ -434,7 +530,7 @@
},
{
"cell_type": "code",
- "execution_count": 9,
+ "execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
@@ -468,7 +564,7 @@
},
{
"cell_type": "code",
- "execution_count": 10,
+ "execution_count": 12,
"metadata": {},
"outputs": [
{
diff --git a/python/packages/autogen-core/src/autogen_core/application/_single_threaded_agent_runtime.py b/python/packages/autogen-core/src/autogen_core/application/_single_threaded_agent_runtime.py
index 8fba5894e..efaecf1ec 100644
--- a/python/packages/autogen-core/src/autogen_core/application/_single_threaded_agent_runtime.py
+++ b/python/packages/autogen-core/src/autogen_core/application/_single_threaded_agent_runtime.py
@@ -32,6 +32,10 @@ from ._helpers import SubscriptionManager, get_impl
logger = logging.getLogger("autogen_core")
event_logger = logging.getLogger("autogen_core.events")
+# We use a type parameter in some functions which shadows the built-in `type` function.
+# This is a workaround to avoid shadowing the built-in `type` function.
+type_func_alias = type
+
@dataclass(kw_only=True)
class PublishMessageEnvelope:
@@ -546,7 +550,9 @@ class SingleThreadedAgentRuntime(AgentRuntime):
agent_instance = await self._get_agent(id)
if not isinstance(agent_instance, type):
- raise TypeError(f"Agent with name {id.type} is not of type {type.__name__}")
+ raise TypeError(
+ f"Agent with name {id.type} is not of type {type.__name__}. It is of type {type_func_alias(agent_instance).__name__}"
+ )
return agent_instance
diff --git a/python/packages/autogen-core/src/autogen_core/components/_routed_agent.py b/python/packages/autogen-core/src/autogen_core/components/_routed_agent.py
index c7b57adcd..77362091a 100644
--- a/python/packages/autogen-core/src/autogen_core/components/_routed_agent.py
+++ b/python/packages/autogen-core/src/autogen_core/components/_routed_agent.py
@@ -6,6 +6,7 @@ from typing import (
Callable,
Coroutine,
Dict,
+ List,
Literal,
Protocol,
Sequence,
@@ -25,18 +26,21 @@ from ._type_helpers import AnyType, get_types
logger = logging.getLogger("autogen_core")
-ReceivesT = TypeVar("ReceivesT", contravariant=True)
+ReceivesT = TypeVar("ReceivesT")
ProducesT = TypeVar("ProducesT", covariant=True)
# TODO: Generic typevar bound binding U to agent type
# Can't do because python doesnt support it
+# Pyright and mypy disagree on the variance of ReceivesT. Mypy thinks it should be contravariant here.
+# Revisit this later to see if we can remove the ignore.
@runtime_checkable
-class MessageHandler(Protocol[ReceivesT, ProducesT]):
+class MessageHandler(Protocol[ReceivesT, ProducesT]): # type: ignore
target_types: Sequence[type]
produces_types: Sequence[type]
is_message_handler: Literal[True]
+ router: Callable[[ReceivesT, MessageContext], bool]
async def __call__(self, message: ReceivesT, ctx: MessageContext) -> ProducesT: ...
@@ -55,6 +59,19 @@ def message_handler(
def message_handler(
func: None = None,
*,
+ match: None = ...,
+ strict: bool = ...,
+) -> Callable[
+ [Callable[[Any, ReceivesT, MessageContext], Coroutine[Any, Any, ProducesT]]],
+ MessageHandler[ReceivesT, ProducesT],
+]: ...
+
+
+@overload
+def message_handler(
+ func: None = None,
+ *,
+ match: Callable[[ReceivesT, MessageContext], bool],
strict: bool = ...,
) -> Callable[
[Callable[[Any, ReceivesT, MessageContext], Coroutine[Any, Any, ProducesT]]],
@@ -66,6 +83,7 @@ def message_handler(
func: None | Callable[[Any, ReceivesT, MessageContext], Coroutine[Any, Any, ProducesT]] = None,
*,
strict: bool = True,
+ match: None | Callable[[ReceivesT, MessageContext], bool] = None,
) -> (
Callable[
[Callable[[Any, ReceivesT, MessageContext], Coroutine[Any, Any, ProducesT]]],
@@ -118,6 +136,7 @@ def message_handler(
wrapper_handler.target_types = list(target_types)
wrapper_handler.produces_types = list(return_types)
wrapper_handler.is_message_handler = True
+ wrapper_handler.router = match or (lambda _message, _ctx: True)
return wrapper_handler
@@ -134,16 +153,17 @@ class RoutedAgent(BaseAgent):
# Self is already bound to the handlers
self._handlers: Dict[
Type[Any],
- Callable[[Any, MessageContext], Coroutine[Any, Any, Any | None]],
+ List[MessageHandler[Any, Any]],
] = {}
+ # Iterate over all attributes in alphabetical order and find message handlers.
for attr in dir(self):
if callable(getattr(self, attr, None)):
handler = getattr(self, attr)
if hasattr(handler, "is_message_handler"):
message_handler = cast(MessageHandler[Any, Any], handler)
for target_type in message_handler.target_types:
- self._handlers[target_type] = message_handler
+ self._handlers.setdefault(target_type, []).append(message_handler)
for message_type in self._handlers.keys():
for serializer in try_get_known_serializers_for_type(message_type):
@@ -153,11 +173,14 @@ class RoutedAgent(BaseAgent):
async def on_message(self, message: Any, ctx: MessageContext) -> Any | None:
key_type: Type[Any] = type(message) # type: ignore
- handler = self._handlers.get(key_type) # type: ignore
- if handler is not None:
- return await handler(message, ctx)
- else:
- return await self.on_unhandled_message(message, ctx) # type: ignore
+ handlers = self._handlers.get(key_type) # type: ignore
+ if handlers is not None:
+ # Iterate over all handlers for this matching message type.
+ # Call the first handler whose router returns True and then return the result.
+ for h in handlers:
+ if h.router(message, ctx):
+ return await h(message, ctx)
+ return await self.on_unhandled_message(message, ctx) # type: ignore
async def on_unhandled_message(self, message: Any, ctx: MessageContext) -> None:
logger.info(f"Unhandled message: {message}")
diff --git a/python/packages/autogen-core/tests/test_routed_agent.py b/python/packages/autogen-core/tests/test_routed_agent.py
index f70cb3841..3d0c79a38 100644
--- a/python/packages/autogen-core/tests/test_routed_agent.py
+++ b/python/packages/autogen-core/tests/test_routed_agent.py
@@ -1,11 +1,11 @@
-from dataclasses import dataclass
-import pytest
import logging
+from dataclasses import dataclass
+from typing import Callable, cast
+import pytest
from autogen_core.application import SingleThreadedAgentRuntime
-from autogen_core.components import TypeSubscription
-from autogen_core.base import TopicId
-
+from autogen_core.base import AgentId, MessageContext, TopicId
+from autogen_core.components import RoutedAgent, TypeSubscription, message_handler
from test_utils import LoopbackAgent
@@ -13,12 +13,104 @@ from test_utils import LoopbackAgent
class UnhandledMessageType: ...
+@dataclass
+class MessageType: ...
+
+
+class CounterAgent(RoutedAgent):
+ def __init__(self) -> None:
+ super().__init__("A loop back agent.")
+ self.num_calls_rpc = 0
+ self.num_calls_broadcast = 0
+
+ @message_handler(match=lambda _, ctx: ctx.is_rpc)
+ async def on_rpc_message(self, message: MessageType, ctx: MessageContext) -> MessageType:
+ self.num_calls_rpc += 1
+ return message
+
+ @message_handler(match=lambda _, ctx: not ctx.is_rpc)
+ async def on_broadcast_message(self, message: MessageType, ctx: MessageContext) -> None:
+ self.num_calls_broadcast += 1
+
+
@pytest.mark.asyncio
async def test_routed_agent(caplog: pytest.LogCaptureFixture) -> None:
runtime = SingleThreadedAgentRuntime()
with caplog.at_level(logging.INFO):
- await runtime.register("loopback", lambda: LoopbackAgent(), lambda: [TypeSubscription("default", "loopback")])
+ await runtime.register("loopback", LoopbackAgent, lambda: [TypeSubscription("default", "loopback")])
runtime.start()
await runtime.publish_message(UnhandledMessageType(), topic_id=TopicId("default", "default"))
await runtime.stop_when_idle()
assert any("Unhandled message: " in e.message for e in caplog.records)
+
+
+async def test_message_handler_router() -> None:
+ runtime = SingleThreadedAgentRuntime()
+ await runtime.register("counter", CounterAgent, lambda: [TypeSubscription("default", "counter")])
+ agent_id = AgentId(type="counter", key="default")
+
+ # Send a broadcast message.
+ runtime.start()
+ await runtime.publish_message(MessageType(), topic_id=TopicId("default", "default"))
+ await runtime.stop_when_idle()
+ agent = await runtime.try_get_underlying_agent_instance(agent_id, type=CounterAgent)
+ assert agent.num_calls_broadcast == 1
+ assert agent.num_calls_rpc == 0
+
+ # Send an RPC message.
+ runtime.start()
+ await runtime.send_message(MessageType(), recipient=agent_id)
+ await runtime.stop_when_idle()
+ agent = await runtime.try_get_underlying_agent_instance(agent_id, type=CounterAgent)
+ assert agent.num_calls_broadcast == 1
+ assert agent.num_calls_rpc == 1
+
+
+@dataclass
+class TestMessage:
+ value: str
+
+
+class RoutedAgentMessageCustomMatch(RoutedAgent):
+ def __init__(self) -> None:
+ super().__init__("")
+ self.handler_one_called = False
+ self.handler_two_called = False
+
+ @staticmethod
+ def match_one(message: TestMessage, ctx: MessageContext) -> bool:
+ return message.value == "one"
+
+ @message_handler(match=match_one)
+ async def handler_one(self, message: TestMessage, ctx: MessageContext) -> None:
+ self.handler_one_called = True
+
+ @message_handler(match=cast(Callable[[TestMessage, MessageContext], bool], lambda msg, ctx: msg.value == "two"))
+ async def handler_two(self, message: TestMessage, ctx: MessageContext) -> None:
+ self.handler_two_called = True
+
+async def test_routed_agent_message_matching() -> None:
+ runtime = SingleThreadedAgentRuntime()
+ await runtime.register("message_match", RoutedAgentMessageCustomMatch)
+ agent_id = AgentId(type="message_match", key="default")
+
+ agent = await runtime.try_get_underlying_agent_instance(agent_id, type=RoutedAgentMessageCustomMatch)
+ assert agent is not None
+ assert agent.handler_one_called == False
+ assert agent.handler_two_called == False
+
+ runtime.start()
+ await runtime.send_message(TestMessage("one"), recipient=agent_id)
+ await runtime.stop_when_idle()
+ agent = await runtime.try_get_underlying_agent_instance(agent_id, type=RoutedAgentMessageCustomMatch)
+ assert agent.handler_one_called == True
+ assert agent.handler_two_called == False
+
+ runtime.start()
+ await runtime.send_message(TestMessage("two"), recipient=agent_id)
+ await runtime.stop_when_idle()
+ agent = await runtime.try_get_underlying_agent_instance(agent_id, type=RoutedAgentMessageCustomMatch)
+ assert agent.handler_one_called == True
+ assert agent.handler_two_called == True