Skip to content

A programming framework for agentic AI

doc: Improve documentation around model client and tool and how it works under the hood (#6050)

Eric Zhu26364e3dfb62
diff --git a/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/agents.ipynb b/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/agents.ipynb
index 505e93f20..35c9052de 100644
--- a/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/agents.ipynb
+++ b/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/agents.ipynb
@@ -294,6 +294,11 @@
"The `web_search` tool is one such tool that allows the assistant agent to search the web for information.\n",
"A custom tool can be a Python function or a subclass of the {py:class}`~autogen_core.tools.BaseTool`.\n",
"\n",
+ "```{note}\n",
+ "For how to use model clients directly with tools, refer to the [Tools](../../core-user-guide/components/tools.ipynb) section\n",
+ "in the Core User Guide.\n",
+ "```\n",
+ "\n",
"By default, when {py:class}`~autogen_agentchat.agents.AssistantAgent` executes a tool,\n",
"it will return the tool's output as a string in {py:class}`~autogen_agentchat.messages.ToolCallSummaryMessage` in its response.\n",
"If your tool does not return a well-formed string in natural language, you\n",
@@ -834,7 +839,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.12.7"
+ "version": "3.12.3"
}
},
"nbformat": 4,
diff --git a/python/packages/autogen-core/docs/src/user-guide/core-user-guide/components/tools.ipynb b/python/packages/autogen-core/docs/src/user-guide/core-user-guide/components/tools.ipynb
index 5757ae7cc..2599f4ceb 100644
--- a/python/packages/autogen-core/docs/src/user-guide/core-user-guide/components/tools.ipynb
+++ b/python/packages/autogen-core/docs/src/user-guide/core-user-guide/components/tools.ipynb
@@ -103,7 +103,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
- "36.63801673457121\n"
+ "143.83831971965762\n"
]
}
],
@@ -137,7 +137,48 @@
"source": [
"## Calling Tools with Model Clients\n",
"\n",
- "Model clients can generate tool calls when they are provided with a list of tools.\n",
+ "In AutoGen, every tool is a subclass of {py:class}`~autogen_core.tools.BaseTool`,\n",
+ "which automatically generates the JSON schema for the tool.\n",
+ "For example, to get the JSON schema for the `stock_price_tool`, we can use the\n",
+ "{py:attr}`~autogen_core.tools.BaseTool.schema` property."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{'name': 'get_stock_price',\n",
+ " 'description': 'Get the stock price.',\n",
+ " 'parameters': {'type': 'object',\n",
+ " 'properties': {'ticker': {'description': 'ticker',\n",
+ " 'title': 'Ticker',\n",
+ " 'type': 'string'},\n",
+ " 'date': {'description': 'Date in YYYY/MM/DD',\n",
+ " 'title': 'Date',\n",
+ " 'type': 'string'}},\n",
+ " 'required': ['ticker', 'date'],\n",
+ " 'additionalProperties': False},\n",
+ " 'strict': False}"
+ ]
+ },
+ "execution_count": 2,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "stock_price_tool.schema"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Model clients use the JSON schema of the tools to generate tool calls.\n",
"\n",
"Here is an example of how to use the {py:class}`~autogen_core.tools.FunctionTool` class\n",
"with a {py:class}`~autogen_ext.models.openai.OpenAIChatCompletionClient`.\n",
@@ -185,13 +226,28 @@
"cell_type": "markdown",
"metadata": {},
"source": [
+ "What is actually going on under the hood of the call to the\n",
+ "{py:class}`~autogen_ext.models.openai.BaseOpenAIChatCompletionClient.create`\n",
+ "method? The model client takes the list of tools and generates a JSON schema\n",
+ "for the parameters of each tool. Then, it generates a request to the model\n",
+ "API with the tool's JSON schema and the other messages to obtain a result.\n",
+ "\n",
+ "Many models, such as OpenAI's GPT-4o and Llama-3.2, are trained to produce\n",
+ "tool calls in the form of structured JSON strings that conform to the\n",
+ "JSON schema of the tool. AutoGen's model clients then parse the model's response\n",
+ "and extract the tool call from the JSON string.\n",
+ "\n",
"The result is a list of {py:class}`~autogen_core.FunctionCall` objects, which can be\n",
- "used to run the corresponding tools."
+ "used to run the corresponding tools.\n",
+ "\n",
+ "We use `json.loads` to parse the JSON string in the {py:class}`~autogen_core.FunctionCall.arguments`\n",
+ "field into a Python dictionary. The {py:meth}`~autogen_core.tools.BaseTool.run_json`\n",
+ "method takes the dictionary and runs the tool with the provided arguments."
]
},
{
"cell_type": "code",
- "execution_count": 12,
+ "execution_count": null,
"metadata": {},
"outputs": [
{
@@ -206,6 +262,7 @@
}
],
"source": [
+ "assert isinstance(create_result.content, list)\n",
"arguments = json.loads(create_result.content[0].arguments) # type: ignore\n",
"tool_result = await stock_price_tool.run_json(arguments, cancellation_token)\n",
"tool_result_str = stock_price_tool.return_value_as_string(tool_result)\n",
@@ -264,7 +321,12 @@
"## Tool-Equipped Agent\n",
"\n",
"Putting the model client and the tools together, you can create a tool-equipped agent\n",
- "that can use tools to perform actions, and reflect on the results of those actions."
+ "that can use tools to perform actions, and reflect on the results of those actions.\n",
+ "\n",
+ "```{note}\n",
+ "The Core API is designed to be minimal and you need to build your own agent logic around model clients and tools.\n",
+ "For \"pre-built\" agents that can use tools, please refer to the [AgentChat API](../../agentchat-user-guide/index.md).\n",
+ "```"
]
},
{
@@ -463,7 +525,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.12.7"
+ "version": "3.12.3"
}
},
"nbformat": 4,