A programming framework for agentic AI
use myst_nb for notebook docs; create getting started tutorial (#326)
Eric Zhu5c7bb29fb3a1
diff --git a/python/.gitignore b/python/.gitignoreindex acadaf0f7..1e5623e46 100644--- a/python/.gitignore+++ b/python/.gitignore@@ -169,3 +169,9 @@ src/agnext/worker/protos/agent*# Generated log fileslog.jsonl++# Jupyter notebooks executions in docs.+docs/**/jupyter_execute++# Temporary files+tmp_code_*.py\ No newline at end of filediff --git a/python/docs/src/conf.py b/python/docs/src/conf.pyindex 862219965..80b059e7a 100644--- a/python/docs/src/conf.py+++ b/python/docs/src/conf.py@@ -18,9 +18,8 @@ extensions = ["sphinx.ext.autosummary","sphinx.ext.napoleon","sphinxcontrib.apidoc",- "myst_parser",+ "myst_nb","sphinx.ext.intersphinx",- "nbsphinx","IPython.sphinxext.ipython_console_highlighting",]@@ -37,6 +36,9 @@ exclude_patterns = ["reference/agnext.rst"]autoclass_content = "init"+# TODO: incldue all notebooks excluding those requiring remote API access.+nb_execution_mode = "off"+# Guides and tutorials must succeed.nb_execution_raise_on_error = Truenb_execution_timeout = 60diff --git a/python/docs/src/core-concepts/ai-agents.md b/python/docs/src/core-concepts/ai-agents.mddeleted file mode 100644index 906d947f5..000000000--- a/python/docs/src/core-concepts/ai-agents.md+++ /dev/null@@ -1,332 +0,0 @@-# AI Agents--This section is still under construction.-A good place to start is the [samples](https://github.com/microsoft/agnext/tree/main/python/samples).--## What is an AI Agent?--In AGNext, AI agents makes use of language models and tools to perform actions-in response to messages. Actions can be anything from generating a response-to a message, to executing code and making API calls.-AGNext does not prescribe a specific architecture for AI agents, but provides-a set of components to make it easier to build them.-You can also bring your own agent and integrate it with AGNext.--## Model Client--AGNext provides the {py:mod}`agnext.components.models` module with a suite of built-in-model clients for using ChatCompletion API.-All model clients implement the {py:class}`agnext.components.models.ChatCompletionClient` protocol class.--Let's create a simple AI agent that can respond to messages using the ChatCompletion API.--```python-import asyncio-from dataclasses import dataclass-from agnext.application import SingleThreadedAgentRuntime-from agnext.components import TypeRoutedAgent, message_handler-from agnext.components.models import ChatCompletionClient, SystemMessage, UserMessage, OpenAIChatCompletionClient-from agnext.core import CancellationToken--@dataclass-class Message:- content: str--class SimpleAgent(TypeRoutedAgent):-- def __init__(self, model_client: ChatCompletionClient) -> None:- super().__init__("A simple agent")- self._system_messages = [SystemMessage("You are a helpful AI assistant.")]- self._model_client = model_client-- @message_handler- async def handle_user_message(self, message: Message, cancellation_token: CancellationToken) -> MyMessage:- # Prepare input to the chat completion model.- user_message = UserMessage(content=message.content, source="user")- response = await self._model_client.create(self._system_messages + [user_message], cancellation_token=cancellation_token)- # Return with the model's response.- return Message(content=response.content)-```--The `SimpleAgent` class is a subclass of the-{py:class}`agnext.components.TypeRoutedAgent` class for the convenience of automatically routing messages to the appropriate handlers.-It has a single handler, `handle_user_message`, which handles message from the user. It uses the `ChatCompletionClient` to generate a response to the message.-It then returns the response to the user, following the direct communication model.--To run the agent, we need to create a runtime and start it.--```python-async def main():- # Create the runtime and register the agent.- runtime = SingleThreadedAgentRuntime()- agent = await runtime.register_and_get(- "simple-agent",- lambda: SimpleAgent(OpenAIChatCompletionClient(model="gpt-4o-mini", api_key="YOUR_API_KEY")),- # Leave out the api_key field if you have the API key in the environment variable OPENAI_API_KEY- )- # Start the runtime processing messages.- run_context = runtime.start()- # Send a message to the agent and get the response.- message = Message("Hello, what are some fun things to do in Seattle?")- response = await runtime.send_message(message, agent)- print(response)- # Stop the runtime processing messages.- await run_context.stop()--asyncio.run(main())-```--In the example above, we use the direct communication to send a string message-to the simple agent and get a response. A response might look like this:--```text-Hello! There are plenty of fun things to do in Seattle. Here are a few suggestions:--1. Visit the Space Needle for panoramic views of the city and surrounding areas.-2. Explore Pike Place Market, one of the oldest continually operated public farmers' markets in the United States.-3. Take a stroll through the famous Chihuly Garden and Glass exhibit showcasing the work of glass artist Dale Chihuly.-4. Enjoy a ferry ride to Bainbridge Island or explore the waterfront areas like Alki Beach.-5. Visit the Museum of Pop Culture (MoPOP) to explore exhibits on music, science fiction, and popular culture.-6. Walk around the historic neighborhoods of Pioneer Square and Capitol Hill.-7. Check out the Seattle Art Museum or the Seattle Aquarium.-8. Explore the scenic beauty of Discovery Park or the Washington Park Arboretum.--There's so much to see and do in Seattle, and these are just a few options to get you started. Let me know if you would like more specific recommendations or information on any of these suggestions!-```--## Tools--Tools are code that can be executed by an agent to perform actions. A tool-can be a simple function such as a calculator, or an API call to a third-party service-such as stock price lookup and weather forecast.-In the context of AI agents, tools are designed to be executed by agents in-response to model-generated function calls.--AGNext provides the {py:mod}`agnext.components.tools` module with a suite of built-in-tools and utilities for creating and running custom tools.--### Built-in Tools--One of the built-in tools is the {py:class}`agnext.components.tools.PythonCodeExecutionTool`,-which allows agents to execute Python code snippets.--Here is how you create the tool and use it.--```python-import asyncio--from agnext.components.code_executor import LocalCommandLineCodeExecutor-from agnext.components.tools import PythonCodeExecutionTool-from agnext.core import CancellationToken---async def main() -> None:- # Create the tool.- code_executor = LocalCommandLineCodeExecutor()- code_execution_tool = PythonCodeExecutionTool(code_executor)- cancellation_token = CancellationToken()-- # Use the tool directly without an agent.- code = "print('Hello, world!')"- result = await code_execution_tool.run_json({"code": code}, cancellation_token)- print(code_execution_tool.return_value_as_string(result))--asyncio.run(main())-```--The {py:class}`~agnext.components.code_executor.LocalCommandLineCodeExecutor`-class is a built-in code executor that runs Python code snippets in a subprocess-in the local command line environment.-The {py:class}`~agnext.components.tools.PythonCodeExecutionTool` class wraps the code executor-and provides a simple interface to execute Python code snippets.--The code above should print the following output:--```text-Hello, world!--```--Other built-in tools will be added in the future.--### Custom Function Tools--A tool can also be a simple Python function that performs a specific action.-To create a custom function tool, you just need to create a Python function-and use the {py:class}`agnext.components.tools.FunctionTool` class to wrap it.--For example, a simple tool to obtain the stock price of a company might look like this:--```python-import random-from typing_extensions import Annotated-from agnext.components.tools import FunctionTool--async def get_stock_price(ticker: str, date: Annotated[str, "Date in YYYY/MM/DD"]) -> float:- # Returns a random stock price for demonstration purposes.- return random.uniform(10, 200)--# Create a function tool.-stock_price_tool = FunctionTool(get_stock_price, description="Get the stock price.")-```--### Tool-Equipped Agent--To use tools with an agent, you can use {py:class}`agnext.components.tool_agent.ToolAgent`,-by using it in a composition pattern.-Here is an example tool-use agent that uses {py:class}`~agnext.components.tool_agent.ToolAgent`-as an inner agent for executing tools.--```python-import asyncio-from dataclasses import dataclass-from typing import List--from agnext.application import SingleThreadedAgentRuntime-from agnext.components import FunctionCall, TypeRoutedAgent, message_handler-from agnext.components.models import (- AssistantMessage,- ChatCompletionClient,- FunctionExecutionResult,- FunctionExecutionResultMessage,- OpenAIChatCompletionClient,- SystemMessage,- UserMessage,-)-from agnext.components.tool_agent import ToolAgent, ToolException-from agnext.components.tools import FunctionTool, Tool, ToolSchema-from agnext.core import AgentId, CancellationToken---@dataclass-class Message:- content: str---class ToolUseAgent(TypeRoutedAgent):- def __init__(self, model_client: ChatCompletionClient, tool_schema: List[ToolSchema], tool_agent: AgentId) -> None:- super().__init__("An agent with tools")- self._system_messages = [SystemMessage("You are a helpful AI assistant.")]- self._model_client = model_client- self._tool_schema = tool_schema- self._tool_agent = tool_agent-- @message_handler- async def handle_user_message(self, message: Message, cancellation_token: CancellationToken) -> Message:- # Create a session of messages.- session = [UserMessage(content=message.content, source="user")]- # Get a response from the model.- response = await self._model_client.create(- self._system_messages + session, tools=self._tool_schema, cancellation_token=cancellation_token- )- # Add the response to the session.- session.append(AssistantMessage(content=response.content, source="assistant"))-- # Keep iterating until the model stops generating tool calls.- while isinstance(response.content, list) and all(isinstance(item, FunctionCall) for item in response.content):- # Execute functions called by the model by sending messages to itself.- results: List[FunctionExecutionResult | BaseException] = await asyncio.gather(- *[self.send_message(call, self._tool_agent) for call in response.content],- return_exceptions=True,- )- # Combine the results into a single response and handle exceptions.- function_results: List[FunctionExecutionResult] = []- for result in results:- if isinstance(result, FunctionExecutionResult):- function_results.append(result)- elif isinstance(result, ToolException):- function_results.append(FunctionExecutionResult(content=f"Error: {result}", call_id=result.call_id))- elif isinstance(result, BaseException):- raise result # Unexpected exception.- session.append(FunctionExecutionResultMessage(content=function_results))- # Query the model again with the new response.- response = await self._model_client.create(- self._system_messages + session, tools=self._tool_schema, cancellation_token=cancellation_token- )- session.append(AssistantMessage(content=response.content, source=self.metadata["name"]))-- # Return the final response.- return Message(content=response.content)-```--The `ToolUseAgent` class is much more involved than the `SimpleAgent` class, however,-the core idea can be described using a simple control flow graph:----The `ToolUseAgent`'s `handle_user_message` handler handles messages from the user,-and determines whether the model has generated a tool call.-If the model has generated tool calls, then the handler sends a function call-message to the {py:class}`~agnext.components.tool_agent.ToolAgent` agent-to execute the tools,-and then queries the model again with the results of the tool calls.-This process continues until the model stops generating tool calls,-at which point the final response is returned to the user.--By having the tool execution logic in a separate agent,-we expose the model-tool interactions to the agent runtime as messages, so the tool executions-can be observed externally and intercepted if necessary.--To run the agent, we need to create a runtime and register the agent.--```python-async def main() -> None:- # Create a runtime.- runtime = SingleThreadedAgentRuntime()- # Create the tools.- tools: List[Tool] = [FunctionTool(get_stock_price, description="Get the stock price.")]- # Register the agents.- tool_executor_agent = await runtime.register_and_get(- "tool-executor-agent",- lambda: ToolAgent(- description="Tool Executor Agent",- tools=tools,- ),- )- tool_use_agent = await runtime.register_and_get(- "tool-use-agent",- lambda: ToolUseAgent(- # OPENAI_API_KEY environment variable must be set.- OpenAIChatCompletionClient(model="gpt-4o-mini"),- tool_schema=[tool.schema for tool in tools],- tool_agent=tool_executor_agent,- ),- )- # Start processing messages.- run_context = runtime.start()- # Send a direct message to the tool agent.- response = await runtime.send_message(Message("What is the stock price of NVDA on 2024/06/01?"), tool_use_agent)- print(response.content)- # Stop processing messages.- await run_context.stop()-```--In the example above, we use the direct communication to send a message to-the tool use agent. The agent's response might look like this:--```text-The stock price of NVDA on June 1, 2024, was $26.49.-```--See [samples](https://github.com/microsoft/agnext/tree/main/python/samples#tool-use-examples)-for more examples of using tools with agents, including how to use-broadcast communication model for tool execution, and how to intercept tool-execution for human-in-the-loop approval.--## Memory--Memory is a collection of data corresponding to the conversation history-of an agent.-Data in meory can be just a simple list of all messages,-or one which provides a view of the last N messages.-Memory can be used in agents to provide context to the model, and make sure-the context is within the limits of the model's context window.-AGNext provides the {py:mod}`agnext.components.memory` module with a suite of built-in-memory implementations.--## Bring Your Own Agent--AGNext is designed to be extensible, and you can bring your own agent implemented-by any framework or directly using the ChatCompletion API.-See [samples](https://github.com/microsoft/agnext/tree/main/python/samples#bring-your-own-agent)-for bring your own agent examples.diff --git a/python/docs/src/core-concepts/foundation.md b/python/docs/src/core-concepts/foundation.mddeleted file mode 100644index 85c9e04ee..000000000--- a/python/docs/src/core-concepts/foundation.md+++ /dev/null@@ -1,380 +0,0 @@-# Foundation--In this section, we focus on the core concepts of AGNext:-agents, agent runtime, messages, and communication.-You will not find any AI models or tools here, just the foundational-building blocks for building multi-agent applications.--## Agent and Agent Runtime--An agent in AGNext can react to, send, and publish messages.-Messages are the only means through which agents can communicate-with each other.--An agent runtime is the execution environment for agents in AGNext.-Similar to the runtime environment of a programming language,-an agent runtime provides the necessary infrastructure to facilitate communication-between agents, manage agent lifecycles, enforce security boundaries, and support monitoring and-debugging.-For local development, developers can use {py:class}`~agnext.application.SingleThreadedAgentRuntime`,-which can be embedded in a Python application.--```{note}-Agents are not directly instantiated and managed by application code.-Instead, they are created by the runtime when needed and managed by the runtime.-```--### Implementing an Agent--To implement an agent, developer must subclass the {py:class}`~agnext.core.BaseAgent` class,-declare the message types it can handle in the {py:attr}`~agnext.core.AgentMetadata.subscriptions` metadata,-and implement the {py:meth}`~agnext.core.BaseAgent.on_message` method.-This method is invoked when the agent receives a message. For example,-the following agent handles a simple message type and simply prints message it receives:--```python-from dataclasses import dataclass-from agnext.core import BaseAgent, CancellationToken--@dataclass-class MyMessage:- content: str--class MyAgent(BaseAgent):- def __init__(self):- super().__init__("MyAgent", subscriptions=[MyMessage])-- async def on_message(self, message: MyMessage, cancellation_token: CancellationToken) -> None:- print(f"Received message: {message.content}")-```--For convenience, developers can subclass the {py:class}`~agnext.components.TypeRoutedAgent` class-which provides an easy-to use API to implement different message handlers for different message types.-See the section on message handlers below.--### Registering Agents--To make an agent available to the runtime, developers can use the-{py:meth}`~agnext.core.AgentRuntime.register` method.-The process of registration associates a name and a factory function-that creates an instance of the agent in a given namespace.-The factory function is used to allow automatic creation of agents when they are needed.--For example, to register an agent with the {py:class}`~agnext.application.SingleThreadedAgentRuntime`,-the following code can be used:--```python-from agnext.application import SingleThreadedAgentRuntime--runtime = SingleThreadedAgentRuntime()-runtime.register("my_agent", lambda: MyAgent())-```--Once an agent is registered, a reference to the agent can be retrieved by-calling {py:meth}`~agnext.core.AgentRuntime.get` or-{py:meth}`~agnext.core.AgentRuntime.get_proxy`. For example, to-send a message to the agent we just registered:--```python-agent = runtime.get("my_agent")-run_context = runtime.start() # Start processing messages in the background.-await runtime.send_message(MyMessage(content="Hello, World!"), agent)-```--There is a convenience method-{py:meth}`~agnext.core.AgentRuntime.register_and_get` that both registers an agent-and gets a reference.--```{note}-Because the runtime manages the lifecycle of agents, a reference to an agent,-whether it is {py:class}`~agnext.core.AgentId` or {py:class}`~agnext.core.AgentProxy`,-is only used to communicate with the agent or retrieve its metadata (e.g., description).-```--### Running the Agent Runtime--The above code snippet uses `runtime.start()` to start a background task-to process and deliver messages to recepients' message handlers.-This is a feature of the-local embedded runtime {py:class}`~agnext.application.SingleThreadedAgentRuntime`.--To stop the background task immediately, use the `stop()` method:--```python-run_context = runtime.start()-# ... Send messages, publish messages, etc.-await run_context.stop() # This will return immediately but will not cancel-# any in-progress message handling.-```--You can resume the background task by calling `start()` again.--For batch scenarios such as running benchmarks for evaluating agents,-you may want to wait for the background task to stop automatically when-there are no unprocessed messages and no agent is handling messages ---the batch may considered complete.-You can achieve this by using the `stop_when_idle()` method:--```python-run_context = runtime.start()-# ... Send messages, publish messages, etc.-await run_context.stop_when_idle() # This will block until the runtime is idle.-```--You can also directly process messages one-by-one without a background task using:--```python-await runtime.process_next()-```--Other runtime implementations will have their own ways of running the runtime.--## Messages--Agents communicate with each other via messages.-Messages are serializable objects, they can be defined using:--- A subclass of Pydantic's {py:class}`pydantic.BaseModel`, or-- A dataclass--For example:--```python-from dataclasses import dataclass--@dataclass-class TextMessage:- content: str- source: str--@dataclass-class ImageMessage:- url: str- source: str-```--```{note}-Messages are purely data, and should not contain any logic.-```--### Message Handlers--When an agent receives a message the runtime will invoke the agent's message handler-({py:meth}`~agnext.core.Agent.on_message`) which should implement the agents message handling logic.-If this message cannot be handled by the agent, the agent should raise a-{py:class}`~agnext.core.exceptions.CantHandleException`.--For convenience, the {py:class}`~agnext.components.TypeRoutedAgent` base class-provides the {py:meth}`~agnext.components.message_handler` decorator-for associating message types with message handlers,-so developers do not need to implement the {py:meth}`~agnext.core.Agent.on_message` method.--For example, the following type-routed agent responds to `TextMessage` and `ImageMessage`-using different message handlers:--```python-from agnext.application import SingleThreadedAgentRuntime-from agnext.components import TypeRoutedAgent, message_handler-from agnext.core import CancellationToken--class MyAgent(TypeRoutedAgent):- @message_handler- async def on_text_message(self, message: TextMessage, cancellation_token: CancellationToken) -> None:- print(f"Hello, {message.source}, you said {message.content}!")-- @message_handler- async def on_image_message(self, message: ImageMessage, cancellation_token: CancellationToken) -> None:- print(f"Hello, {message.source}, you sent me {message.url}!")--async def main() -> None:- runtime = SingleThreadedAgentRuntime()- agent = await runtime.register_and_get("my_agent", lambda: MyAgent("My Agent"))- run_context = runtime.start()- await runtime.send_message(TextMessage(content="Hello, World!", source="User"), agent)- await runtime.send_message(ImageMessage(url="https://example.com/image.jpg", source="User"), agent)- await run_context.stop_when_idle()--import asyncio-asyncio.run(main())-```--## Communication--There are two types of communication in AGNext:--- **Direct communication**: An agent sends a direct message to another agent.-- **Broadcast communication**: An agent publishes a message to all agents in the same namespace.--### Direct Communication--To send a direct message to another agent, within a message handler use-the {py:meth}`agnext.core.BaseAgent.send_message` method,-from the runtime use the {py:meth}`agnext.core.AgentRuntime.send_message` method.-Awaiting calls to these methods will return the return value of the-receiving agent's message handler.--```{note}-If the invoked agent raises an exception while the sender is awaiting,-the exception will be propagated back to the sender.-```--#### Request/Response--Direct communication can be used for request/response scenarios,-where the sender expects a response from the receiver.-The receiver can respond to the message by returning a value from its message handler.-You can think of this as a function call between agents.--For example, consider the following type-routed agents:--```python-from dataclasses import dataclass-from agnext.application import SingleThreadedAgentRuntime-from agnext.components import TypeRoutedAgent, message_handler-from agnext.core import CancellationToken, AgentId--@dataclass-class MyMessage:- content: str--class InnerAgent(TypeRoutedAgent):- @message_handler- async def on_my_message(self, message: MyMessage, cancellation_token: CancellationToken) -> str:- return MyMessage(content=f"Hello from inner, {message}")--class OuterAgent(TypeRoutedAgent):- def __init__(self, description: str, inner_agent_id: AgentId):- super().__init__(description)- self.inner_agent_id = inner_agent_id-- @message_handler- async def on_my_message(self, message: MyMessage, cancellation_token: CancellationToken) -> None:- print(f"Received message: {message.content}")- # Send a direct message to the inner agent and receves a response.- response = await self.send_message(MyMessage(f"Hello from outer, {message.content}", self.inner_agent_id))- print(f"Received inner response: {response.content}")--async def main() -> None:- runtime = SingleThreadedAgentRuntime()- inner = await runtime.register_and_get("inner_agent", lambda: InnerAgent("InnerAgent"))- outer = await runtime.register_and_get("outer_agent", lambda: OuterAgent("OuterAgent", inner))- run_context = runtime.start()- await runtime.send_message(MyMessage("Hello, World!"), outer)- await run_context.stop_when_idle()--import asyncio-asyncio.run(main())-```--In the above example, upone receving a message,-the `OuterAgent` sends a direct message to the `InnerAgent` and receives-a message in response. The following output will be produced:--```text-Received message: Hello, World!-Received inner response: Hello from inner, Hello from outer, Hello, World!-```--#### Command/Notification--In many scenarios, an agent can commanded another agent to perform an action,-or notify another agent of an event. In this case,-the sender does not need a response from the receiver -- it is a command or notification,-and the receiver does not need to return a value from the message handler.-For example, the `InnerAgent` can be modified to just print the message it receives:--```python-class InnerAgent(TypeRoutedAgent):- @message_handler- async def on_my_message(self, message: MyMessage, cancellation_token: CancellationToken) -> None:- print(f"Hello from inner, {message.content}")-```--### Broadcast Communication--Broadcast communication is effectively the publish/subscribe model.-As part of the base agent ({py:class}`~agnext.core.BaseAgent`) implementation,-it must advertise the message types that-it would like to receive when published ({py:attr}`~agnext.core.AgentMetadata.subscriptions`).-If one of these messages is published, the agent's message handler will be invoked.--The key difference between direct and broadcast communication is that broadcast-communication cannot be used for request/response scenarios.-When an agent publishes a message it is one way only, it cannot receive a response-from any other agent, even if a receiving agent sends a response.--```{note}-An agent receiving a message does not know if it is handling a published or direct message.-So, if a response is given to a published message, it will be thrown away.-```--To publish a message to all agents in the same namespace,-use the {py:meth}`agnext.core.BaseAgent.publish_message` method.-This call must still be awaited to allow the runtime to deliver the message to all agents,-but it will always return `None`.-If an agent raises an exception while handling a published message,-this will be logged but will not be propagated back to the publishing agent.--The following example shows a `BroadcastingAgent` that publishes a message-upong receiving a message. A `ReceivingAgent` that prints the message-it receives.--```python-from agnext.application import SingleThreadedAgentRuntime-from agnext.components import TypeRoutedAgent, message_handler-from agnext.core import CancellationToken--class BroadcastingAgent(TypeRoutedAgent):- @message_handler- async def on_my_message(self, message: MyMessage, cancellation_token: CancellationToken) -> None:- # Publish a message to all agents in the same namespace.- await self.publish_message(MyMessage(f"Publishing a message: {message.content}!"))--class ReceivingAgent(TypeRoutedAgent):- @message_handler- async def on_my_message(self, message: MyMessage, cancellation_token: CancellationToken) -> None:- print(f"Received a message: {message.content}")--async def main() -> None:- runtime = SingleThreadedAgentRuntime()- broadcaster = await runtime.register_and_get("broadcasting_agent", lambda: BroadcastingAgent("Broadcasting Agent"))- await runtime.register("receiving_agent", lambda: ReceivingAgent("Receiving Agent"))- run_context = runtime.start()- await runtime.send_message(MyMessage("Hello, World!"), broadcaster)- await run_context.stop_when_idle()--import asyncio-asyncio.run(main())-```--Running the above code will produce the following output produced by the `ReceivingAgent`:--```text-Received a message: Publishing a message: Hello, World!-```--To publish a message to all agents outside of an agent handling a message,-the message should be published via the runtime with the-{py:meth}`agnext.core.AgentRuntime.publish_message` method.--```python-# ... Replace send_message with publish_message in the above example.-await runtime.publish_message(MyMessage("Hello, World! From the runtime!"), namespace="default")-# ...-```--Running the above code will produce the following output:--```text-Received a message: Hello, World! From the runtime!-Received a message: Publishing a message: Hello, World! From the runtime!-```--The first output is from the `ReceivingAgent` that received a message published-by the runtime. The second output is from the `ReceivingAgent` that received-a message published by the `BroadcastingAgent`.--```{note}-If an agent publishes a message type for which it is subscribed it will not-receive the message it published. This is to prevent infinite loops.-```diff --git a/python/docs/src/core-concepts/overview.md b/python/docs/src/core-concepts/overview.mdindex 6457fe22f..d61612ec2 100644--- a/python/docs/src/core-concepts/overview.md+++ b/python/docs/src/core-concepts/overview.md@@ -2,10 +2,17 @@This section provides the background and overview of AGNext.-## Multi-Agent Application+## Agent and Multi-Agent Application++An agent is a software entity that+communicates via messages, maintains a state,+and performs actions in response to messages or a change in its state.+Actions can result in changes to the agent's state and external effects,+for example, updating message history, sending a message, executing code,+or making external API calls.A wide variety of software applications can be modeled as a collection of independent-agents that communicate with each other through messages:+agents that communicate with each other:sensors on a factory floor,distributed services powering web applications,business workflows involving multiple stakeholders,@@ -14,6 +21,11 @@ and more recently, artificial intelligence (AI) agents powered by language modelother software systems.We refer to them as multi-agent applications.+```{note}+AI agents make use of language models as part of+their software stacks to perform actions.+```+In a multi-agent application, agents can live in the same process, on the same machine,or on different machines and across organizational boundaries.They can be implemented using different AI models, instructions, and programming languages.@@ -95,7 +107,7 @@ The following diagram shows the AGNext application stack.At the bottom of the stack is the base messaging and routing facilities thatenable agents to communicate with each other. These are managed by theagent runtime, and for most applications, developers only need to interact-with the high-level APIs provided by the runtime (see [Foundation](foundation.md)).+with the high-level APIs provided by the runtime (see [Agent and Agent Runtime](../getting-started/agent-and-agent-runtime.ipynb)).On top of the communication stack, developers need to define thetypes of the messages that agents exchange. A set of message types@@ -103,10 +115,8 @@ forms a behavior contract that agents must adhere to, and theimplementation of the contracts determines how agents handle messages.The behavior contract is sometimes referred to as the message protocol.It is the developer's responsibility to implement the behavior contract.-See [AI Agents](ai-agents.md) for how to build AI agents-that implement behavior contract.Multi-agent patterns are design patterns that emerge from behavior contracts-(see [Multi-Agent Patterns](patterns.md)).+(see [Multi-Agent Design Patterns](../getting-started/multi-agent-design-patterns.ipynb)).### An Example Applicationdiff --git a/python/docs/src/core-concepts/patterns.md b/python/docs/src/core-concepts/patterns.mddeleted file mode 100644index 1b9f09bcc..000000000--- a/python/docs/src/core-concepts/patterns.md+++ /dev/null@@ -1,13 +0,0 @@-# Multi-Agent Patterns--Agents can work together in a variety of ways to solve problems.-Research works like [AutoGen](https://aka.ms/autogen-paper),-[MetaGPT](https://arxiv.org/abs/2308.00352)-and [ChatDev](https://arxiv.org/abs/2307.07924) have shown-multi-agent systems out-performing single agent systems at complex tasks-like software development.--You can implement any multi-agent pattern using AGNext agents, which-communicate with each other using messages through the agent runtime.-See [samples](https://github.com/microsoft/agnext/tree/main/python/samples#pattern-examples)-for how to implement patterns like reflection and group chat.diff --git a/python/docs/src/getting-started/agent-and-agent-runtime.ipynb b/python/docs/src/getting-started/agent-and-agent-runtime.ipynbnew file mode 100644index 000000000..2edfdf642--- /dev/null+++ b/python/docs/src/getting-started/agent-and-agent-runtime.ipynb@@ -0,0 +1,249 @@+{+ "cells": [+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "# Agent and Agent Runtime\n",+ "\n",+ "In this and the following section, we focus on the \n",+ "[core concepts](../core-concepts/overview.md) of AGNext:\n",+ "agents, agent runtime, messages, and communication.\n",+ "You will not find any AI models or tools here, just the foundational\n",+ "building blocks for building multi-agent applications."+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "An agent in AGNext is an entity defined by the base class {py:class}`agnext.core.BaseAgent`.\n",+ "It has a unique identifier of the type {py:class}`agnext.core.AgentId`,\n",+ "a metadata dictionary of the type {py:class}`agnext.core.AgentMetadata`,\n",+ "and method for handling messages {py:meth}`agnext.core.BaseAgent.on_message`.\n",+ "\n",+ "An agent runtime is the execution environment for agents in AGNext.\n",+ "Similar to the runtime environment of a programming language,\n",+ "an agent runtime provides the necessary infrastructure to facilitate communication\n",+ "between agents, manage agent lifecycles, enforce security boundaries, and support monitoring and\n",+ "debugging.\n",+ "For local development, developers can use {py:class}`~agnext.application.SingleThreadedAgentRuntime`,\n",+ "which can be embedded in a Python application.\n",+ "\n",+ "```{note}\n",+ "Agents are not directly instantiated and managed by application code.\n",+ "Instead, they are created by the runtime when needed and managed by the runtime.\n",+ "```"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "## Implementing an Agent\n",+ "\n",+ "To implement an agent, developer must subclass the {py:class}`~agnext.core.BaseAgent` class,\n",+ "declare the message types it can handle in the {py:attr}`~agnext.core.AgentMetadata.subscriptions` metadata,\n",+ "and implement the {py:meth}`~agnext.core.BaseAgent.on_message` method.\n",+ "This method is invoked when the agent receives a message. For example,\n",+ "the following agent handles a simple message type and simply prints message it receives:"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 2,+ "metadata": {},+ "outputs": [],+ "source": [+ "from dataclasses import dataclass\n",+ "from agnext.core import BaseAgent, CancellationToken\n",+ "\n",+ "@dataclass\n",+ "class MyMessage:\n",+ " content: str\n",+ "\n",+ "class MyAgent(BaseAgent):\n",+ " def __init__(self):\n",+ " super().__init__(\"MyAgent\", subscriptions=[MyMessage])\n",+ "\n",+ " async def on_message(self, message: MyMessage, cancellation_token: CancellationToken) -> None:\n",+ " print(f\"Received message: {message.content}\")"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "For convenience, developers can subclass the {py:class}`~agnext.components.TypeRoutedAgent` class\n",+ "which provides an easy-to use API to implement different message handlers for different message types.\n",+ "See the section on message handlers below."+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "## Registering Agents\n",+ "\n",+ "To make an agent available to the runtime, developers can use the\n",+ "{py:meth}`~agnext.core.AgentRuntime.register` method.\n",+ "The process of registration associates a name and a factory function\n",+ "that creates an instance of the agent in a given namespace.\n",+ "The factory function is used to allow automatic creation of agents when they are needed.\n",+ "\n",+ "For example, to register an agent with the {py:class}`~agnext.application.SingleThreadedAgentRuntime`,\n",+ "the following code can be used:"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 4,+ "metadata": {},+ "outputs": [],+ "source": [+ "from agnext.application import SingleThreadedAgentRuntime\n",+ "\n",+ "runtime = SingleThreadedAgentRuntime()\n",+ "await runtime.register(\"my_agent\", lambda: MyAgent())"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "Once an agent is registered, the agent's unique ID of the type {py:class}`~agnext.core.AgentId` \n",+ "can be retrieved by calling {py:meth}`~agnext.core.AgentRuntime.get`.\n",+ "Using the agent ID, we can send messages to the agent:"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": null,+ "metadata": {},+ "outputs": [+ {+ "name": "stdout",+ "output_type": "stream",+ "text": [+ "Received message: Hello, World!\n"+ ]+ }+ ],+ "source": [+ "agent_id = await runtime.get(\"my_agent\")\n",+ "run_context = runtime.start() # Start processing messages in the background.\n",+ "await runtime.send_message(MyMessage(content=\"Hello, World!\"), agent_id)\n",+ "await run_context.stop() # Stop processing messages in the background."+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "There is a convenience method\n",+ "{py:meth}`~agnext.core.AgentRuntime.register_and_get` that both registers an agent\n",+ "and gets its ID.\n",+ "\n",+ "```{note}\n",+ "Because the runtime manages the lifecycle of agents, a reference to an agent,\n",+ "whether it is {py:class}`~agnext.core.AgentId` or {py:class}`~agnext.core.AgentProxy`,\n",+ "is only used to communicate with the agent or retrieve its metadata (e.g., description).\n",+ "```"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "## Running the Single-Threaded Agent Runtime\n",+ "\n",+ "The above code snippet uses `runtime.start()` to start a background task\n",+ "to process and deliver messages to recepients' message handlers.\n",+ "This is a feature of the\n",+ "local embedded runtime {py:class}`~agnext.application.SingleThreadedAgentRuntime`.\n",+ "\n",+ "To stop the background task immediately, use the `stop()` method:"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 6,+ "metadata": {},+ "outputs": [],+ "source": [+ "run_context = runtime.start()\n",+ "# ... Send messages, publish messages, etc.\n",+ "await run_context.stop() # This will return immediately but will not cancel\n",+ "# any in-progress message handling."+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "You can resume the background task by calling `start()` again.\n",+ "\n",+ "For batch scenarios such as running benchmarks for evaluating agents,\n",+ "you may want to wait for the background task to stop automatically when\n",+ "there are no unprocessed messages and no agent is handling messages --\n",+ "the batch may considered complete.\n",+ "You can achieve this by using the `stop_when_idle()` method:"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 7,+ "metadata": {},+ "outputs": [],+ "source": [+ "run_context = runtime.start()\n",+ "# ... Send messages, publish messages, etc.\n",+ "await run_context.stop_when_idle() # This will block until the runtime is idle."+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "You can also directly process messages one-by-one without a background task using:"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 8,+ "metadata": {},+ "outputs": [],+ "source": [+ "await runtime.process_next()"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "Other runtime implementations will have their own ways of running the runtime."+ ]+ }+ ],+ "metadata": {+ "kernelspec": {+ "display_name": "agnext",+ "language": "python",+ "name": "python3"+ },+ "language_info": {+ "codemirror_mode": {+ "name": "ipython",+ "version": 3+ },+ "file_extension": ".py",+ "mimetype": "text/x-python",+ "name": "python",+ "nbconvert_exporter": "python",+ "pygments_lexer": "ipython3",+ "version": "3.11.9"+ }+ },+ "nbformat": 4,+ "nbformat_minor": 2+}diff --git a/python/docs/src/getting-started/message-and-communication.ipynb b/python/docs/src/getting-started/message-and-communication.ipynbnew file mode 100644index 000000000..84f7e84f0--- /dev/null+++ b/python/docs/src/getting-started/message-and-communication.ipynb@@ -0,0 +1,404 @@+{+ "cells": [+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "# Message and Communication\n",+ "\n",+ "An agent in AGNext can react to, send, and publish messages,\n",+ "and messages are the only means through which agents can communicate\n",+ "with each other."+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "## Messages\n",+ "\n",+ "Messages are serializable objects, they can be defined using:\n",+ "\n",+ "- A subclass of Pydantic's {py:class}`pydantic.BaseModel`, or\n",+ "- A dataclass\n",+ "\n",+ "For example:"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 2,+ "metadata": {},+ "outputs": [],+ "source": [+ "from dataclasses import dataclass\n",+ "\n",+ "@dataclass\n",+ "class TextMessage:\n",+ " content: str\n",+ " source: str\n",+ "\n",+ "@dataclass\n",+ "class ImageMessage:\n",+ " url: str\n",+ " source: str"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "```{note}\n",+ "Messages are purely data, and should not contain any logic.\n",+ "```"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "### Message Handlers\n",+ "\n",+ "When an agent receives a message the runtime will invoke the agent's message handler\n",+ "({py:meth}`~agnext.core.Agent.on_message`) which should implement the agents message handling logic.\n",+ "If this message cannot be handled by the agent, the agent should raise a\n",+ "{py:class}`~agnext.core.exceptions.CantHandleException`.\n",+ "\n",+ "For convenience, the {py:class}`~agnext.components.TypeRoutedAgent` base class\n",+ "provides the {py:meth}`~agnext.components.message_handler` decorator\n",+ "for associating message types with message handlers,\n",+ "so developers do not need to implement the {py:meth}`~agnext.core.Agent.on_message` method.\n",+ "\n",+ "For example, the following type-routed agent responds to `TextMessage` and `ImageMessage`\n",+ "using different message handlers:"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 4,+ "metadata": {},+ "outputs": [],+ "source": [+ "from agnext.application import SingleThreadedAgentRuntime\n",+ "from agnext.components import TypeRoutedAgent, message_handler\n",+ "from agnext.core import CancellationToken\n",+ "\n",+ "class MyAgent(TypeRoutedAgent):\n",+ " @message_handler\n",+ " async def on_text_message(self, message: TextMessage, cancellation_token: CancellationToken) -> None:\n",+ " print(f\"Hello, {message.source}, you said {message.content}!\")\n",+ "\n",+ " @message_handler\n",+ " async def on_image_message(self, message: ImageMessage, cancellation_token: CancellationToken) -> None:\n",+ " print(f\"Hello, {message.source}, you sent me {message.url}!\")"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "Create the agent runtime and register the agent (see [Agent and Agent Runtime](agent-and-agent-runtime.ipynb)):"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 6,+ "metadata": {},+ "outputs": [],+ "source": [+ "runtime = SingleThreadedAgentRuntime()\n",+ "agent = await runtime.register_and_get(\"my_agent\", lambda: MyAgent(\"My Agent\"))"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "Test this agent with `TextMessage` and `ImageMessage`."+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 7,+ "metadata": {},+ "outputs": [+ {+ "name": "stdout",+ "output_type": "stream",+ "text": [+ "Hello, User, you said Hello, World!!\n",+ "Hello, User, you sent me https://example.com/image.jpg!\n"+ ]+ }+ ],+ "source": [+ "run_context = runtime.start()\n",+ "await runtime.send_message(TextMessage(content=\"Hello, World!\", source=\"User\"), agent)\n",+ "await runtime.send_message(ImageMessage(url=\"https://example.com/image.jpg\", source=\"User\"), agent)\n",+ "await run_context.stop_when_idle()"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "## Communication\n",+ "\n",+ "There are two types of communication in AGNext:\n",+ "\n",+ "- **Direct communication**: An agent sends a direct message to another agent.\n",+ "- **Broadcast communication**: An agent publishes a message to all agents in the same namespace.\n",+ "\n",+ "### Direct Communication\n",+ "\n",+ "To send a direct message to another agent, within a message handler use\n",+ "the {py:meth}`agnext.core.BaseAgent.send_message` method,\n",+ "from the runtime use the {py:meth}`agnext.core.AgentRuntime.send_message` method.\n",+ "Awaiting calls to these methods will return the return value of the\n",+ "receiving agent's message handler.\n",+ "\n",+ "```{note}\n",+ "If the invoked agent raises an exception while the sender is awaiting,\n",+ "the exception will be propagated back to the sender.\n",+ "```\n",+ "\n",+ "#### Request/Response\n",+ "\n",+ "Direct communication can be used for request/response scenarios,\n",+ "where the sender expects a response from the receiver.\n",+ "The receiver can respond to the message by returning a value from its message handler.\n",+ "You can think of this as a function call between agents.\n",+ "\n",+ "For example, consider the following type-routed agent:"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 17,+ "metadata": {},+ "outputs": [],+ "source": [+ "from dataclasses import dataclass\n",+ "from agnext.application import SingleThreadedAgentRuntime\n",+ "from agnext.components import TypeRoutedAgent, message_handler\n",+ "from agnext.core import CancellationToken, AgentId\n",+ "\n",+ "@dataclass\n",+ "class Message:\n",+ " content: str\n",+ "\n",+ "class InnerAgent(TypeRoutedAgent):\n",+ " @message_handler\n",+ " async def on_my_message(self, message: Message, cancellation_token: CancellationToken) -> Message:\n",+ " return Message(content=f\"Hello from inner, {message.content}\")\n",+ "\n",+ "class OuterAgent(TypeRoutedAgent):\n",+ " def __init__(self, description: str, inner_agent_id: AgentId):\n",+ " super().__init__(description)\n",+ " self.inner_agent_id = inner_agent_id\n",+ "\n",+ " @message_handler\n",+ " async def on_my_message(self, message: Message, cancellation_token: CancellationToken) -> None:\n",+ " print(f\"Received message: {message.content}\")\n",+ " # Send a direct message to the inner agent and receves a response.\n",+ " response = await self.send_message(Message(f\"Hello from outer, {message.content}\"), self.inner_agent_id)\n",+ " print(f\"Received inner response: {response.content}\")"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "Upone receving a message, the `OuterAgent` sends a direct message to the `InnerAgent` and receives\n",+ "a message in response.\n",+ "\n",+ "We can test these agents by sending a `Message` to the `OuterAgent`."+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 18,+ "metadata": {},+ "outputs": [+ {+ "name": "stdout",+ "output_type": "stream",+ "text": [+ "Received message: Hello, World!\n",+ "Received inner response: Hello from inner, Hello from outer, Hello, World!\n"+ ]+ }+ ],+ "source": [+ "runtime = SingleThreadedAgentRuntime()\n",+ "inner = await runtime.register_and_get(\"inner_agent\", lambda: InnerAgent(\"InnerAgent\"))\n",+ "outer = await runtime.register_and_get(\"outer_agent\", lambda: OuterAgent(\"OuterAgent\", inner))\n",+ "run_context = runtime.start()\n",+ "await runtime.send_message(Message(content=\"Hello, World!\"), outer)\n",+ "await run_context.stop_when_idle()"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "Both outputs are produced by the `OuterAgent`'s message handler, however the second output is based on the response from the `InnerAgent`."+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "### Broadcast Communication\n",+ "\n",+ "Broadcast communication is effectively the publish/subscribe model.\n",+ "As part of the base agent ({py:class}`~agnext.core.BaseAgent`) implementation,\n",+ "it must advertise the message types that\n",+ "it would like to receive when published ({py:attr}`~agnext.core.AgentMetadata.subscriptions`).\n",+ "If one of these messages is published, the agent's message handler will be invoked.\n",+ "\n",+ "The key difference between direct and broadcast communication is that broadcast\n",+ "communication cannot be used for request/response scenarios.\n",+ "When an agent publishes a message it is one way only, it cannot receive a response\n",+ "from any other agent, even if a receiving agent sends a response.\n",+ "\n",+ "```{note}\n",+ "An agent receiving a message does not know if it is handling a published or direct message.\n",+ "So, if a response is given to a published message, it will be thrown away.\n",+ "```\n",+ "\n",+ "To publish a message to all agents in the same namespace,\n",+ "use the {py:meth}`agnext.core.BaseAgent.publish_message` method.\n",+ "This call must still be awaited to allow the runtime to deliver the message to all agents,\n",+ "but it will always return `None`.\n",+ "If an agent raises an exception while handling a published message,\n",+ "this will be logged but will not be propagated back to the publishing agent.\n",+ "\n",+ "The following example shows a `BroadcastingAgent` that publishes a message\n",+ "upong receiving a message. A `ReceivingAgent` that prints the message\n",+ "it receives."+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 20,+ "metadata": {},+ "outputs": [],+ "source": [+ "from agnext.application import SingleThreadedAgentRuntime\n",+ "from agnext.components import TypeRoutedAgent, message_handler\n",+ "from agnext.core import CancellationToken\n",+ "\n",+ "class BroadcastingAgent(TypeRoutedAgent):\n",+ " @message_handler\n",+ " async def on_my_message(self, message: Message, cancellation_token: CancellationToken) -> None:\n",+ " # Publish a message to all agents in the same namespace.\n",+ " await self.publish_message(Message(f\"Publishing a message: {message.content}!\"))\n",+ "\n",+ "class ReceivingAgent(TypeRoutedAgent):\n",+ " @message_handler\n",+ " async def on_my_message(self, message: Message, cancellation_token: CancellationToken) -> None:\n",+ " print(f\"Received a message: {message.content}\")"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "Sending a direct message to the `BroadcastingAgent` will result in a message being published by\n",+ "the `BroadcastingAgent` and received by the `ReceivingAgent`."+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 21,+ "metadata": {},+ "outputs": [+ {+ "name": "stdout",+ "output_type": "stream",+ "text": [+ "Received a message: Publishing a message: Hello, World!!\n"+ ]+ }+ ],+ "source": [+ "runtime = SingleThreadedAgentRuntime()\n",+ "broadcaster = await runtime.register_and_get(\"broadcasting_agent\", lambda: BroadcastingAgent(\"Broadcasting Agent\"))\n",+ "await runtime.register(\"receiving_agent\", lambda: ReceivingAgent(\"Receiving Agent\"))\n",+ "run_context = runtime.start()\n",+ "await runtime.send_message(Message(\"Hello, World!\"), broadcaster)\n",+ "await run_context.stop_when_idle()"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "To publish a message to all agents outside of an agent handling a message,\n",+ "the message should be published via the runtime with the\n",+ "{py:meth}`agnext.core.AgentRuntime.publish_message` method."+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 22,+ "metadata": {},+ "outputs": [+ {+ "name": "stdout",+ "output_type": "stream",+ "text": [+ "Received a message: Hello, World! From the runtime!\n",+ "Received a message: Publishing a message: Hello, World! From the runtime!!\n"+ ]+ }+ ],+ "source": [+ "# Replace send_message with publish_message in the above example.\n",+ "\n",+ "runtime = SingleThreadedAgentRuntime()\n",+ "broadcaster = await runtime.register_and_get(\"broadcasting_agent\", lambda: BroadcastingAgent(\"Broadcasting Agent\"))\n",+ "await runtime.register(\"receiving_agent\", lambda: ReceivingAgent(\"Receiving Agent\"))\n",+ "run_context = runtime.start()\n",+ "await runtime.publish_message(Message(\"Hello, World! From the runtime!\"), namespace=\"default\")\n",+ "await run_context.stop_when_idle()"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "The first output is from the `ReceivingAgent` that received a message published\n",+ "by the runtime. The second output is from the `ReceivingAgent` that received\n",+ "a message published by the `BroadcastingAgent`.\n",+ "\n",+ "```{note}\n",+ "If an agent publishes a message type for which it is subscribed it will not\n",+ "receive the message it published. This is to prevent infinite loops.\n",+ "```"+ ]+ }+ ],+ "metadata": {+ "kernelspec": {+ "display_name": "agnext",+ "language": "python",+ "name": "python3"+ },+ "language_info": {+ "codemirror_mode": {+ "name": "ipython",+ "version": 3+ },+ "file_extension": ".py",+ "mimetype": "text/x-python",+ "name": "python",+ "nbconvert_exporter": "python",+ "pygments_lexer": "ipython3",+ "version": "3.11.9"+ }+ },+ "nbformat": 4,+ "nbformat_minor": 2+}diff --git a/python/docs/src/getting-started/model-clients.ipynb b/python/docs/src/getting-started/model-clients.ipynbnew file mode 100644index 000000000..13ab4e090--- /dev/null+++ b/python/docs/src/getting-started/model-clients.ipynb@@ -0,0 +1,352 @@+{+ "cells": [+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "# Model Clients\n",+ "\n",+ "AGNext provides the {py:mod}`agnext.components.models` module with a suite of built-in\n",+ "model clients for using ChatCompletion API.\n",+ "All model clients implement the {py:class}`agnext.components.models.ChatCompletionClient` protocol class."+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "## Built-in Model Clients\n",+ "\n",+ "Currently there are two built-in model clients:\n",+ "{py:class}`agnext.components.models.OpenAIChatCompletionClient` and\n",+ "{py:class}`agnext.components.models.AzureOpenAIChatCompletionClient`.\n",+ "Both clients are asynchronous.\n",+ "\n",+ "To use the {py:class}`~agnext.components.models.OpenAIChatCompletionClient`, you need to provide the API key\n",+ "either through the environment variable `OPENAI_API_KEY` or through the `api_key` argument."+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 16,+ "metadata": {},+ "outputs": [],+ "source": [+ "from agnext.components.models import OpenAIChatCompletionClient, UserMessage\n",+ "\n",+ "# Create an OpenAI model client.\n",+ "model_client = OpenAIChatCompletionClient(\n",+ " model=\"gpt-4o\",\n",+ " # api_key=\"sk-...\", # Optional if you have an API key set in the environment.\n",+ ")"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "You can call the {py:meth}`~agnext.components.models.OpenAIChatCompletionClient.create` method to create a\n",+ "chat completion request, and await for an {py:class}`~agnext.components.models.CreateResult` object in return."+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 17,+ "metadata": {},+ "outputs": [+ {+ "name": "stdout",+ "output_type": "stream",+ "text": [+ "The capital of France is Paris.\n"+ ]+ }+ ],+ "source": [+ "# Send a message list to the model and await the response.\n",+ "messages = [\n",+ " UserMessage(content=\"What is the capital of France?\", source=\"user\"),\n",+ "]\n",+ "response = await model_client.create(messages=messages)\n",+ "\n",+ "# Print the response\n",+ "print(response.content)"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "### Streaming Response\n",+ "\n",+ "You can use the {py:meth}`~agnext.components.models.OpenAIChatCompletionClient.create_streaming` method to create a\n",+ "chat completion request with streaming response."+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 24,+ "metadata": {},+ "outputs": [+ {+ "name": "stdout",+ "output_type": "stream",+ "text": [+ "Streamed responses:\n",+ "In the heart of the Whispering Woods lived Ember, a small dragon with scales of shimmering gold. Unlike other dragons, Ember breathed not fire but music, each note a whisper of ancient songs. The villagers, initially fearful, soon realized her gift brought harmony and joy.\n",+ "\n",+ "One night, as darkness threatened the land, Ember's melodies summoned the stars, casting a protective glow. The villagers danced beneath the celestial orchestra, their worries dissolving like morning mist.\n",+ "\n",+ "From that day on, Ember's song became a nightly ritual, a promise that light and harmony would always prevail. The dragon of the Whispering Woods was a symbol of peace, her golden scales a testament to the magic of gentleness.\n",+ "\n",+ "------------\n",+ "\n",+ "The complete response:\n",+ "In the heart of the Whispering Woods lived Ember, a small dragon with scales of shimmering gold. Unlike other dragons, Ember breathed not fire but music, each note a whisper of ancient songs. The villagers, initially fearful, soon realized her gift brought harmony and joy.\n",+ "\n",+ "One night, as darkness threatened the land, Ember's melodies summoned the stars, casting a protective glow. The villagers danced beneath the celestial orchestra, their worries dissolving like morning mist.\n",+ "\n",+ "From that day on, Ember's song became a nightly ritual, a promise that light and harmony would always prevail. The dragon of the Whispering Woods was a symbol of peace, her golden scales a testament to the magic of gentleness.\n"+ ]+ }+ ],+ "source": [+ "messages = [\n",+ " UserMessage(content=\"Write a very short story about a dragon.\", source=\"user\"),\n",+ "]\n",+ "\n",+ "# Create a stream.\n",+ "stream = model_client.create_stream(messages=messages)\n",+ "\n",+ "# Iterate over the stream and print the responses.\n",+ "print(\"Streamed responses:\")\n",+ "async for response in stream:\n",+ " if isinstance(response, str):\n",+ " # A partial response is a string.\n",+ " print(response, flush=True, end=\"\")\n",+ " else:\n",+ " # The last response is a CreateResult object with the complete message.\n",+ " print(\"\\n\\n------------\\n\")\n",+ " print(\"The complete response:\", flush=True)\n",+ " print(response.content, flush=True)"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "```{note}\n",+ "The last response in the streaming response is always the final response\n",+ "of the type {py:class}`~agnext.components.models.CreateResult`.\n",+ "```"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "### Azure OpenAI\n",+ "\n",+ "To use the {py:class}`~agnext.components.models.AzureOpenAIChatCompletionClient`, you need to provide\n",+ "the deployment id, Azure Cognitive Services endpoint, api version, and model capabilities.\n",+ "For authentication, you can either provide an API key or an Azure Active Directory (AAD) token credential.\n",+ "To use AAD authentication, you need to first install the `azure-identity` package."+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 11,+ "metadata": {+ "vscode": {+ "languageId": "shellscript"+ }+ },+ "outputs": [],+ "source": [+ "# pip install azure-identity"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "The following code snippet shows how to use AAD authentication.\n",+ "The identity used must be assigned the [**Cognitive Services OpenAI User**](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/role-based-access-control#cognitive-services-openai-user) role."+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 15,+ "metadata": {},+ "outputs": [],+ "source": [+ "from agnext.components.models import AzureOpenAIChatCompletionClient\n",+ "from azure.identity import DefaultAzureCredential, get_bearer_token_provider\n",+ "\n",+ "# Create the token provider\n",+ "token_provider = get_bearer_token_provider(\n",+ " DefaultAzureCredential(), \"https://cognitiveservices.azure.com/.default\"\n",+ ")\n",+ "\n",+ "model_client = AzureOpenAIChatCompletionClient(\n",+ " model=\"{your-azure-deployment}\",\n",+ " api_version=\"2024-06-01\",\n",+ " azure_endpoint=\"https://{your-custom-endpoint}.openai.azure.com/\",\n",+ " azure_ad_token_provider=token_provider, # Optional if you choose key-based authentication.\n",+ " # api_key=\"sk-...\", # For key-based authentication.\n",+ " model_capabilities={\n",+ " \"vision\":True,\n",+ " \"function_calling\":True,\n",+ " \"json_output\":True,\n",+ " }\n",+ ")"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "```{note}\n",+ "See [here](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/managed-identity#chat-completions) for how to use the Azure client directly or for more info.\n",+ "```"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "## Build Agent using Model Client\n",+ "\n",+ "Let's create a simple AI agent that can respond to messages using the ChatCompletion API."+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 4,+ "metadata": {},+ "outputs": [],+ "source": [+ "from dataclasses import dataclass\n",+ "from agnext.application import SingleThreadedAgentRuntime\n",+ "from agnext.components import TypeRoutedAgent, message_handler\n",+ "from agnext.components.models import ChatCompletionClient, SystemMessage, UserMessage, OpenAIChatCompletionClient\n",+ "from agnext.core import CancellationToken\n",+ "\n",+ "@dataclass\n",+ "class Message:\n",+ " content: str\n",+ "\n",+ "class SimpleAgent(TypeRoutedAgent):\n",+ "\n",+ " def __init__(self, model_client: ChatCompletionClient) -> None:\n",+ " super().__init__(\"A simple agent\")\n",+ " self._system_messages = [SystemMessage(\"You are a helpful AI assistant.\")]\n",+ " self._model_client = model_client\n",+ " \n",+ " @message_handler\n",+ " async def handle_user_message(self, message: Message, cancellation_token: CancellationToken) -> Message:\n",+ " # Prepare input to the chat completion model.\n",+ " user_message = UserMessage(content=message.content, source=\"user\")\n",+ " response = await self._model_client.create(self._system_messages + [user_message], cancellation_token=cancellation_token)\n",+ " # Return with the model's response.\n",+ " return Message(content=response.content)"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "The `SimpleAgent` class is a subclass of the\n",+ "{py:class}`agnext.components.TypeRoutedAgent` class for the convenience of automatically routing messages to the appropriate handlers.\n",+ "It has a single handler, `handle_user_message`, which handles message from the user. It uses the `ChatCompletionClient` to generate a response to the message.\n",+ "It then returns the response to the user, following the direct communication model."+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 25,+ "metadata": {},+ "outputs": [+ {+ "name": "stdout",+ "output_type": "stream",+ "text": [+ "Seattle offers a wide range of activities and attractions for visitors. Here are some fun things to do in the city:\n",+ "\n",+ "1. **Space Needle**: Visit this iconic landmark for stunning panoramic views of the city and surrounding mountains.\n",+ "\n",+ "2. **Pike Place Market**: Explore this historic market where you can shop for fresh produce, local crafts, and enjoy delicious street food. Don't miss the famous fish-throwing!\n",+ "\n",+ "3. **Chihuly Garden and Glass**: Admire the breathtaking glass art installations by artist Dale Chihuly, both indoors and in the beautiful outdoor garden.\n",+ "\n",+ "4. **Museum of Pop Culture (MoPOP)**: Discover exhibits focused on music, science fiction, and pop culture, including artifacts from famous films and music legends.\n",+ "\n",+ "5. **Seattle Aquarium**: Learn about marine life native to the Pacific Northwest and see fascinating exhibits, including sea otters and jellyfish.\n",+ "\n",+ "6. **Fremont Troll**: Take a photo with this quirky public art installation, a large troll sculpture located under the Aurora Bridge.\n",+ "\n",+ "7. **Kerry Park**: Enjoy one of the best viewpoints of Seattle's skyline, especially at sunset or during the evening when the city lights up.\n",+ "\n",+ "8. **Discovery Park**: Explore this large urban park with trails, beaches, and beautiful views of Puget Sound and the Olympic Mountains.\n",+ "\n",+ "9. **Seattle Art Museum**: Browse a diverse collection of art from around the world, including contemporary and Native American art.\n",+ "\n",+ "10. **Take a Ferry Ride**: Enjoy a scenic boat ride to nearby islands like Bainbridge Island or Vashon Island. The views of the Seattle skyline from the water are stunning.\n",+ "\n",+ "11. **Underground Tour**: Learn about Seattle's history on a guided tour of the underground passageways that played a significant role in the city’s development.\n",+ "\n",+ "12. **Ballard Locks**: Visit the Hiram M. Chittenden Locks to see boats pass between Lake Washington and Puget Sound and watch salmon swim upstream in the fish ladder (seasonal).\n",+ "\n",+ "13. **Local Breweries**: Seattle is known for its craft beer scene; take a brewery tour or visit a taproom to sample local brews.\n",+ "\n",+ "14. **Attend a Sports Game**: Catch a Seattle Seahawks (NFL), Seattle Mariners (MLB), or Seattle Sounders (MLS) game, depending on the season.\n",+ "\n",+ "15. **Seattle Great Wheel**: Ride this Ferris wheel on the waterfront for beautiful views, especially at night when it’s illuminated.\n",+ "\n",+ "These activities showcase Seattle’s vibrant culture, unique attractions, and stunning natural beauty. Enjoy your visit!\n"+ ]+ }+ ],+ "source": [+ " # Create the runtime and register the agent.\n",+ "runtime = SingleThreadedAgentRuntime()\n",+ "agent = await runtime.register_and_get(\n",+ " \"simple-agent\",\n",+ " lambda: SimpleAgent(OpenAIChatCompletionClient(\n",+ " model=\"gpt-4o-mini\",\n",+ " # api_key=\"sk-...\", # Optional if you have an OPENAI_API_KEY set in the environment.\n",+ " )), \n",+ ")\n",+ "# Start the runtime processing messages.\n",+ "run_context = runtime.start()\n",+ "# Send a message to the agent and get the response.\n",+ "message = Message(\"Hello, what are some fun things to do in Seattle?\")\n",+ "response = await runtime.send_message(message, agent)\n",+ "print(response.content)\n",+ "# Stop the runtime processing messages.\n",+ "await run_context.stop()"+ ]+ }+ ],+ "metadata": {+ "kernelspec": {+ "display_name": "agnext",+ "language": "python",+ "name": "python3"+ },+ "language_info": {+ "codemirror_mode": {+ "name": "ipython",+ "version": 3+ },+ "file_extension": ".py",+ "mimetype": "text/x-python",+ "name": "python",+ "nbconvert_exporter": "python",+ "pygments_lexer": "ipython3",+ "version": "3.11.9"+ }+ },+ "nbformat": 4,+ "nbformat_minor": 2+}diff --git a/python/docs/src/getting-started/multi-agent-design-patterns.ipynb b/python/docs/src/getting-started/multi-agent-design-patterns.ipynbnew file mode 100644index 000000000..da242a851--- /dev/null+++ b/python/docs/src/getting-started/multi-agent-design-patterns.ipynb@@ -0,0 +1,30 @@+{+ "cells": [+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "# Multi-Agent Design Patterns\n",+ "\n",+ "Agents can work together in a variety of ways to solve problems.\n",+ "Research works like [AutoGen](https://aka.ms/autogen-paper),\n",+ "[MetaGPT](https://arxiv.org/abs/2308.00352)\n",+ "and [ChatDev](https://arxiv.org/abs/2307.07924) have shown\n",+ "multi-agent systems out-performing single agent systems at complex tasks\n",+ "like software development.\n",+ "\n",+ "You can implement any multi-agent design pattern using AGNext agents, which\n",+ "communicate with each other using messages through the agent runtime.\n",+ "See [samples](https://github.com/microsoft/agnext/tree/main/python/samples#pattern-examples)\n",+ "for how to implement patterns like reflection and group chat.\n"+ ]+ }+ ],+ "metadata": {+ "language_info": {+ "name": "python"+ }+ },+ "nbformat": 4,+ "nbformat_minor": 2+}diff --git a/python/docs/src/core-concepts/tool-use-agent-cfg.svg b/python/docs/src/getting-started/tool-use-agent-cfg.svgsimilarity index 100%rename from python/docs/src/core-concepts/tool-use-agent-cfg.svgrename to python/docs/src/getting-started/tool-use-agent-cfg.svgdiff --git a/python/docs/src/getting-started/tools.ipynb b/python/docs/src/getting-started/tools.ipynbnew file mode 100644index 000000000..b3f82d489--- /dev/null+++ b/python/docs/src/getting-started/tools.ipynb@@ -0,0 +1,318 @@+{+ "cells": [+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "# Tools\n",+ "\n",+ "Tools are code that can be executed by an agent to perform actions. A tool\n",+ "can be a simple function such as a calculator, or an API call to a third-party service\n",+ "such as stock price lookup and weather forecast.\n",+ "In the context of AI agents, tools are designed to be executed by agents in\n",+ "response to model-generated function calls.\n",+ "\n",+ "AGNext provides the {py:mod}`agnext.components.tools` module with a suite of built-in\n",+ "tools and utilities for creating and running custom tools."+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "## Built-in Tools\n",+ "\n",+ "One of the built-in tools is the {py:class}`agnext.components.tools.PythonCodeExecutionTool`,\n",+ "which allows agents to execute Python code snippets.\n",+ "\n",+ "Here is how you create the tool and use it."+ ]+ },+ {+ "cell_type": "code",+ "execution_count": null,+ "metadata": {},+ "outputs": [],+ "source": [+ "from agnext.components.code_executor import LocalCommandLineCodeExecutor\n",+ "from agnext.components.tools import PythonCodeExecutionTool\n",+ "from agnext.core import CancellationToken\n",+ "\n",+ "# Create the tool.\n",+ "code_executor = LocalCommandLineCodeExecutor()\n",+ "code_execution_tool = PythonCodeExecutionTool(code_executor)\n",+ "cancellation_token = CancellationToken()\n",+ "\n",+ "# Use the tool directly without an agent.\n",+ "code = \"print('Hello, world!')\"\n",+ "result = await code_execution_tool.run_json({\"code\": code}, cancellation_token)\n",+ "print(code_execution_tool.return_value_as_string(result))"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "The {py:class}`~agnext.components.code_executor.LocalCommandLineCodeExecutor`\n",+ "class is a built-in code executor that runs Python code snippets in a subprocess\n",+ "in the local command line environment.\n",+ "The {py:class}`~agnext.components.tools.PythonCodeExecutionTool` class wraps the code executor\n",+ "and provides a simple interface to execute Python code snippets.\n",+ "\n",+ "Other built-in tools will be added in the future."+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "## Custom Function Tools\n",+ "\n",+ "A tool can also be a simple Python function that performs a specific action.\n",+ "To create a custom function tool, you just need to create a Python function\n",+ "and use the {py:class}`agnext.components.tools.FunctionTool` class to wrap it.\n",+ "\n",+ "For example, a simple tool to obtain the stock price of a company might look like this:"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 5,+ "metadata": {},+ "outputs": [+ {+ "name": "stdout",+ "output_type": "stream",+ "text": [+ "138.75280591295171\n"+ ]+ }+ ],+ "source": [+ "import random\n",+ "from typing_extensions import Annotated\n",+ "from agnext.components.tools import FunctionTool\n",+ "from agnext.core import CancellationToken\n",+ "\n",+ "async def get_stock_price(ticker: str, date: Annotated[str, \"Date in YYYY/MM/DD\"]) -> float:\n",+ " # Returns a random stock price for demonstration purposes.\n",+ " return random.uniform(10, 200)\n",+ "\n",+ "# Create a function tool.\n",+ "stock_price_tool = FunctionTool(get_stock_price, description=\"Get the stock price.\")\n",+ "\n",+ "# Run the tool.\n",+ "cancellation_token = CancellationToken()\n",+ "result = await stock_price_tool.run_json({\"ticker\": \"AAPL\", \"date\": \"2021/01/01\"}, cancellation_token)\n",+ "\n",+ "# Print the result.\n",+ "print(stock_price_tool.return_value_as_string(result))"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "## Tool-Equipped Agent\n",+ "\n",+ "To use tools with an agent, you can use {py:class}`agnext.components.tool_agent.ToolAgent`,\n",+ "by using it in a composition pattern.\n",+ "Here is an example tool-use agent that uses {py:class}`~agnext.components.tool_agent.ToolAgent`\n",+ "as an inner agent for executing tools."+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 6,+ "metadata": {},+ "outputs": [],+ "source": [+ "import asyncio\n",+ "from dataclasses import dataclass\n",+ "from typing import List\n",+ "\n",+ "from agnext.application import SingleThreadedAgentRuntime\n",+ "from agnext.components import FunctionCall, TypeRoutedAgent, message_handler\n",+ "from agnext.components.models import (\n",+ " AssistantMessage,\n",+ " ChatCompletionClient,\n",+ " FunctionExecutionResult,\n",+ " FunctionExecutionResultMessage,\n",+ " OpenAIChatCompletionClient,\n",+ " SystemMessage,\n",+ " UserMessage,\n",+ ")\n",+ "from agnext.components.tool_agent import ToolAgent, ToolException\n",+ "from agnext.components.tools import FunctionTool, Tool, ToolSchema\n",+ "from agnext.core import AgentId, CancellationToken\n",+ "\n",+ "\n",+ "@dataclass\n",+ "class Message:\n",+ " content: str\n",+ "\n",+ "\n",+ "class ToolUseAgent(TypeRoutedAgent):\n",+ " def __init__(self, model_client: ChatCompletionClient, tool_schema: List[ToolSchema], tool_agent: AgentId) -> None:\n",+ " super().__init__(\"An agent with tools\")\n",+ " self._system_messages = [SystemMessage(\"You are a helpful AI assistant.\")]\n",+ " self._model_client = model_client\n",+ " self._tool_schema = tool_schema\n",+ " self._tool_agent = tool_agent\n",+ "\n",+ " @message_handler\n",+ " async def handle_user_message(self, message: Message, cancellation_token: CancellationToken) -> Message:\n",+ " # Create a session of messages.\n",+ " session = [UserMessage(content=message.content, source=\"user\")]\n",+ " # Get a response from the model.\n",+ " response = await self._model_client.create(\n",+ " self._system_messages + session, tools=self._tool_schema, cancellation_token=cancellation_token\n",+ " )\n",+ " # Add the response to the session.\n",+ " session.append(AssistantMessage(content=response.content, source=\"assistant\"))\n",+ "\n",+ " # Keep iterating until the model stops generating tool calls.\n",+ " while isinstance(response.content, list) and all(isinstance(item, FunctionCall) for item in response.content):\n",+ " # Execute functions called by the model by sending messages to itself.\n",+ " results: List[FunctionExecutionResult | BaseException] = await asyncio.gather(\n",+ " *[self.send_message(call, self._tool_agent) for call in response.content],\n",+ " return_exceptions=True,\n",+ " )\n",+ " # Combine the results into a single response and handle exceptions.\n",+ " function_results: List[FunctionExecutionResult] = []\n",+ " for result in results:\n",+ " if isinstance(result, FunctionExecutionResult):\n",+ " function_results.append(result)\n",+ " elif isinstance(result, ToolException):\n",+ " function_results.append(FunctionExecutionResult(content=f\"Error: {result}\", call_id=result.call_id))\n",+ " elif isinstance(result, BaseException):\n",+ " raise result # Unexpected exception.\n",+ " session.append(FunctionExecutionResultMessage(content=function_results))\n",+ " # Query the model again with the new response.\n",+ " response = await self._model_client.create(\n",+ " self._system_messages + session, tools=self._tool_schema, cancellation_token=cancellation_token\n",+ " )\n",+ " session.append(AssistantMessage(content=response.content, source=self.metadata[\"name\"]))\n",+ "\n",+ " # Return the final response.\n",+ " return Message(content=response.content)"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "The `ToolUseAgent` class is a bit involved, however,\n",+ "the core idea can be described using a simple control flow graph:\n",+ "\n",+ "\n",+ "\n",+ "The `ToolUseAgent`'s `handle_user_message` handler handles messages from the user,\n",+ "and determines whether the model has generated a tool call.\n",+ "If the model has generated tool calls, then the handler sends a function call\n",+ "message to the {py:class}`~agnext.components.tool_agent.ToolAgent` agent\n",+ "to execute the tools,\n",+ "and then queries the model again with the results of the tool calls.\n",+ "This process continues until the model stops generating tool calls,\n",+ "at which point the final response is returned to the user.\n",+ "\n",+ "By having the tool execution logic in a separate agent,\n",+ "we expose the model-tool interactions to the agent runtime as messages, so the tool executions\n",+ "can be observed externally and intercepted if necessary.\n",+ "\n",+ "To run the agent, we need to create a runtime and register the agent."+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 7,+ "metadata": {},+ "outputs": [],+ "source": [+ "# Create a runtime.\n",+ "runtime = SingleThreadedAgentRuntime()\n",+ "# Create the tools.\n",+ "tools: List[Tool] = [FunctionTool(get_stock_price, description=\"Get the stock price.\")]\n",+ "# Register the agents.\n",+ "tool_executor_agent = await runtime.register_and_get(\n",+ " \"tool-executor-agent\",\n",+ " lambda: ToolAgent(\n",+ " description=\"Tool Executor Agent\",\n",+ " tools=tools,\n",+ " ),\n",+ ")\n",+ "tool_use_agent = await runtime.register_and_get(\n",+ " \"tool-use-agent\",\n",+ " lambda: ToolUseAgent(\n",+ " OpenAIChatCompletionClient(model=\"gpt-4o-mini\"),\n",+ " tool_schema=[tool.schema for tool in tools],\n",+ " tool_agent=tool_executor_agent,\n",+ " ),\n",+ ")"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "This example uses the {py:class}`agnext.components.models.OpenAIChatCompletionClient`,\n",+ "for Azure OpenAI and other clients, see [Model Clients](./model-clients.ipynb).\n",+ "Let's test the agent with a question about stock price."+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 8,+ "metadata": {},+ "outputs": [+ {+ "name": "stdout",+ "output_type": "stream",+ "text": [+ "The stock price of NVDA on June 1, 2024, is approximately $49.28.\n"+ ]+ }+ ],+ "source": [+ "# Start processing messages.\n",+ "run_context = runtime.start()\n",+ "# Send a direct message to the tool agent.\n",+ "response = await runtime.send_message(Message(\"What is the stock price of NVDA on 2024/06/01?\"), tool_use_agent)\n",+ "print(response.content)\n",+ "# Stop processing messages.\n",+ "await run_context.stop()"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "See [samples](https://github.com/microsoft/agnext/tree/main/python/samples#tool-use-examples)\n",+ "for more examples of using tools with agents, including how to use\n",+ "broadcast communication model for tool execution, and how to intercept tool\n",+ "execution for human-in-the-loop approval."+ ]+ }+ ],+ "metadata": {+ "kernelspec": {+ "display_name": "agnext",+ "language": "python",+ "name": "python3"+ },+ "language_info": {+ "codemirror_mode": {+ "name": "ipython",+ "version": 3+ },+ "file_extension": ".py",+ "mimetype": "text/x-python",+ "name": "python",+ "nbconvert_exporter": "python",+ "pygments_lexer": "ipython3",+ "version": "3.11.9"+ }+ },+ "nbformat": 4,+ "nbformat_minor": 2+}diff --git a/python/docs/src/index.rst b/python/docs/src/index.rstindex 9179eb02d..0765d44c7 100644--- a/python/docs/src/index.rst+++ b/python/docs/src/index.rst@@ -12,27 +12,24 @@ You can also implement agents using other agent frameworks and run them in AGNexTo get you started quickly, we offers`a suite of samples <https://github.com/microsoft/agnext/tree/main/python/samples>`_.-To learn about the core concepts of AGNext, read the following sections:--- `Overview <core-concepts/overview>`_ on the architecture design.-- `Foundation <core-concepts/foundation>`_ on agent runtime and inter-agent communication.-- `AI Agents <core-concepts/ai-agents>`_ on how to build AI agents.-- `Multi-Agent Patterns <core-concepts/patterns>`_ on multi-agent collaboration patterns.+To learn about the core concepts of AGNext, read the `overview <core-concepts/overview.html>`_... toctree::- :caption: Getting started+ :caption: Core Concepts:hidden:- getting-started/installation+ core-concepts/overview.. toctree::- :caption: Core Concepts+ :caption: Getting started:hidden:- core-concepts/overview- core-concepts/foundation- core-concepts/ai-agents- core-concepts/patterns+ getting-started/installation+ getting-started/agent-and-agent-runtime+ getting-started/message-and-communication+ getting-started/model-clients+ getting-started/tools+ getting-started/multi-agent-design-patterns.. toctree:::caption: Guidesdiff --git a/python/pyproject.toml b/python/pyproject.tomlindex 8d2ce269c..488c3d4f3 100644--- a/python/pyproject.toml+++ b/python/pyproject.toml@@ -89,9 +89,8 @@ dependencies = ["sphinx","furo","sphinxcontrib-apidoc",- "myst-parser",+ "myst-nb","sphinx-autobuild",- "nbsphinx","Ipython",]diff --git a/python/samples/common/agents/_oai_assistant.py b/python/samples/common/agents/_oai_assistant.pyindex a545322b6..ee4484c83 100644--- a/python/samples/common/agents/_oai_assistant.py+++ b/python/samples/common/agents/_oai_assistant.py@@ -4,7 +4,7 @@ import openaifrom agnext.components import TypeRoutedAgent, message_handlerfrom agnext.core import CancellationTokenfrom openai import AsyncAssistantEventHandler-from openai.types.beta import AssistantResponseFormatParam+from openai.types import ResponseFormatJSONObject, ResponseFormatText # type: ignorefrom ..types import PublishNow, Reset, RespondNow, ResponseFormat, TextMessage@@ -30,7 +30,7 @@ class OpenAIAssistantAgent(TypeRoutedAgent):client: openai.AsyncClient,assistant_id: str,thread_id: str,- assistant_event_handler_factory: Callable[[], AsyncAssistantEventHandler] | None = None,+ assistant_event_handler_factory: (Callable[[], AsyncAssistantEventHandler] | None) = None,) -> None:super().__init__(description)self._client = client@@ -80,13 +80,15 @@ class OpenAIAssistantAgent(TypeRoutedAgent):await self.publish_message(response)async def _generate_response(- self, requested_response_format: ResponseFormat, cancellation_token: CancellationToken+ self,+ requested_response_format: ResponseFormat,+ cancellation_token: CancellationToken,) -> TextMessage:# Handle response format.if requested_response_format == ResponseFormat.json_object:- response_format = AssistantResponseFormatParam(type="json_object")+ response_format = ResponseFormatJSONObject(type="json_object") # type: ignoreelse:- response_format = AssistantResponseFormatParam(type="text")+ response_format = ResponseFormatText(type="text") # type: ignoreif self._assistant_event_handler_factory is not None:# Use event handler and streaming mode if available.@@ -94,7 +96,7 @@ class OpenAIAssistantAgent(TypeRoutedAgent):thread_id=self._thread_id,assistant_id=self._assistant_id,event_handler=self._assistant_event_handler_factory(),- response_format=response_format,+ response_format=response_format, # type: ignore) as stream:run = await stream.get_final_run()else:@@ -102,7 +104,7 @@ class OpenAIAssistantAgent(TypeRoutedAgent):run = await self._client.beta.threads.runs.create(thread_id=self._thread_id,assistant_id=self._assistant_id,- response_format=response_format,+ response_format=response_format, # type: ignore)if run.status != "completed":