A programming framework for agentic AI
doc: Improve documentation around model client and tool and how it works under the hood (#6050)
Eric Zhu26364e3dfb62
diff --git a/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/agents.ipynb b/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/agents.ipynbindex 505e93f20..35c9052de 100644--- a/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/agents.ipynb+++ b/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/agents.ipynb@@ -294,6 +294,11 @@"The `web_search` tool is one such tool that allows the assistant agent to search the web for information.\n","A custom tool can be a Python function or a subclass of the {py:class}`~autogen_core.tools.BaseTool`.\n","\n",+ "```{note}\n",+ "For how to use model clients directly with tools, refer to the [Tools](../../core-user-guide/components/tools.ipynb) section\n",+ "in the Core User Guide.\n",+ "```\n",+ "\n","By default, when {py:class}`~autogen_agentchat.agents.AssistantAgent` executes a tool,\n","it will return the tool's output as a string in {py:class}`~autogen_agentchat.messages.ToolCallSummaryMessage` in its response.\n","If your tool does not return a well-formed string in natural language, you\n",@@ -834,7 +839,7 @@"name": "python","nbconvert_exporter": "python","pygments_lexer": "ipython3",- "version": "3.12.7"+ "version": "3.12.3"}},"nbformat": 4,diff --git a/python/packages/autogen-core/docs/src/user-guide/core-user-guide/components/tools.ipynb b/python/packages/autogen-core/docs/src/user-guide/core-user-guide/components/tools.ipynbindex 5757ae7cc..2599f4ceb 100644--- a/python/packages/autogen-core/docs/src/user-guide/core-user-guide/components/tools.ipynb+++ b/python/packages/autogen-core/docs/src/user-guide/core-user-guide/components/tools.ipynb@@ -103,7 +103,7 @@"name": "stdout","output_type": "stream","text": [- "36.63801673457121\n"+ "143.83831971965762\n"]}],@@ -137,7 +137,48 @@"source": ["## Calling Tools with Model Clients\n","\n",- "Model clients can generate tool calls when they are provided with a list of tools.\n",+ "In AutoGen, every tool is a subclass of {py:class}`~autogen_core.tools.BaseTool`,\n",+ "which automatically generates the JSON schema for the tool.\n",+ "For example, to get the JSON schema for the `stock_price_tool`, we can use the\n",+ "{py:attr}`~autogen_core.tools.BaseTool.schema` property."+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 2,+ "metadata": {},+ "outputs": [+ {+ "data": {+ "text/plain": [+ "{'name': 'get_stock_price',\n",+ " 'description': 'Get the stock price.',\n",+ " 'parameters': {'type': 'object',\n",+ " 'properties': {'ticker': {'description': 'ticker',\n",+ " 'title': 'Ticker',\n",+ " 'type': 'string'},\n",+ " 'date': {'description': 'Date in YYYY/MM/DD',\n",+ " 'title': 'Date',\n",+ " 'type': 'string'}},\n",+ " 'required': ['ticker', 'date'],\n",+ " 'additionalProperties': False},\n",+ " 'strict': False}"+ ]+ },+ "execution_count": 2,+ "metadata": {},+ "output_type": "execute_result"+ }+ ],+ "source": [+ "stock_price_tool.schema"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "Model clients use the JSON schema of the tools to generate tool calls.\n","\n","Here is an example of how to use the {py:class}`~autogen_core.tools.FunctionTool` class\n","with a {py:class}`~autogen_ext.models.openai.OpenAIChatCompletionClient`.\n",@@ -185,13 +226,28 @@"cell_type": "markdown","metadata": {},"source": [+ "What is actually going on under the hood of the call to the\n",+ "{py:class}`~autogen_ext.models.openai.BaseOpenAIChatCompletionClient.create`\n",+ "method? The model client takes the list of tools and generates a JSON schema\n",+ "for the parameters of each tool. Then, it generates a request to the model\n",+ "API with the tool's JSON schema and the other messages to obtain a result.\n",+ "\n",+ "Many models, such as OpenAI's GPT-4o and Llama-3.2, are trained to produce\n",+ "tool calls in the form of structured JSON strings that conform to the\n",+ "JSON schema of the tool. AutoGen's model clients then parse the model's response\n",+ "and extract the tool call from the JSON string.\n",+ "\n","The result is a list of {py:class}`~autogen_core.FunctionCall` objects, which can be\n",- "used to run the corresponding tools."+ "used to run the corresponding tools.\n",+ "\n",+ "We use `json.loads` to parse the JSON string in the {py:class}`~autogen_core.FunctionCall.arguments`\n",+ "field into a Python dictionary. The {py:meth}`~autogen_core.tools.BaseTool.run_json`\n",+ "method takes the dictionary and runs the tool with the provided arguments."]},{"cell_type": "code",- "execution_count": 12,+ "execution_count": null,"metadata": {},"outputs": [{@@ -206,6 +262,7 @@}],"source": [+ "assert isinstance(create_result.content, list)\n","arguments = json.loads(create_result.content[0].arguments) # type: ignore\n","tool_result = await stock_price_tool.run_json(arguments, cancellation_token)\n","tool_result_str = stock_price_tool.return_value_as_string(tool_result)\n",@@ -264,7 +321,12 @@"## Tool-Equipped Agent\n","\n","Putting the model client and the tools together, you can create a tool-equipped agent\n",- "that can use tools to perform actions, and reflect on the results of those actions."+ "that can use tools to perform actions, and reflect on the results of those actions.\n",+ "\n",+ "```{note}\n",+ "The Core API is designed to be minimal and you need to build your own agent logic around model clients and tools.\n",+ "For \"pre-built\" agents that can use tools, please refer to the [AgentChat API](../../agentchat-user-guide/index.md).\n",+ "```"]},{@@ -463,7 +525,7 @@"name": "python","nbconvert_exporter": "python","pygments_lexer": "ipython3",- "version": "3.12.7"+ "version": "3.12.3"}},"nbformat": 4,