Skip to content

A programming framework for agentic AI

Add tool use chapter to tutorial (#2079)

Eric Zhuecc459f8c5e0
diff --git a/website/docs/topics/task_decomposition.ipynb b/website/docs/topics/task_decomposition.ipynb
index 23c5dee91..d6278c9c7 100644
--- a/website/docs/topics/task_decomposition.ipynb
+++ b/website/docs/topics/task_decomposition.ipynb
@@ -110,6 +110,8 @@
")\n",
"\n",
"# register the function task_decomposition for execution\n",
+ "\n",
+ "\n",
"@user_proxy.register_for_execution()\n",
"@assistant.register_for_llm(\n",
" name=\"task_decomposition\",\n",
diff --git a/website/docs/tutorial/code-executors.ipynb b/website/docs/tutorial/code-executors.ipynb
index 774ed20a3..6a8d43987 100644
--- a/website/docs/tutorial/code-executors.ipynb
+++ b/website/docs/tutorial/code-executors.ipynb
@@ -806,8 +806,8 @@
"\n",
"In this chapter, we introduced code executors, how to set up Docker and local\n",
"execution, and how to use code execution in a conversation to solve tasks.\n",
- "In the next chapter, we will introduce conversation patterns, and show\n",
- "how to use the result of a conversation."
+ "In the next chapter, we will introduce tool use, which is similar to code\n",
+ "executors but restricts what code agent can execute."
]
}
],
diff --git a/website/docs/tutorial/tool-use.ipynb b/website/docs/tutorial/tool-use.ipynb
new file mode 100644
index 000000000..bcc161d5a
--- /dev/null
+++ b/website/docs/tutorial/tool-use.ipynb
@@ -0,0 +1,756 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Tool Use\n",
+ "\n",
+ "In the previous chapter, we explored code executors which give\n",
+ "agents the super power of programming.\n",
+ "Agents writing arbitrary code is useful, however,\n",
+ "controlling what code an agent writes can be challenging. \n",
+ "This is where tools come in.\n",
+ "\n",
+ "Tools are pre-defined functions that agents can use. Instead of writing arbitrary\n",
+ "code, agents can call tools to perform actions, such as searching the web,\n",
+ "performing calculations, reading files, or calling remote APIs.\n",
+ "Because you can control what tools are available to an agent, you can control what\n",
+ "actions an agent can perform.\n",
+ "\n",
+ "````mdx-code-block\n",
+ ":::note\n",
+ "Tool use is currently only available for LLMs\n",
+ "that support OpenAI-comaptible tool call API.\n",
+ ":::\n",
+ "````"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Creating Tools\n",
+ "\n",
+ "Tools can be created as regular Python functions.\n",
+ "For example, let's create a calculator tool which\n",
+ "can only perform a single operation at a time."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from typing import Annotated, Literal\n",
+ "\n",
+ "Operator = Literal[\"+\", \"-\", \"*\", \"/\"]\n",
+ "\n",
+ "\n",
+ "def calculator(a: int, b: int, operator: Annotated[Operator, \"operator\"]) -> int:\n",
+ " if operator == \"+\":\n",
+ " return a + b\n",
+ " elif operator == \"-\":\n",
+ " return a - b\n",
+ " elif operator == \"*\":\n",
+ " return a * b\n",
+ " elif operator == \"/\":\n",
+ " return int(a / b)\n",
+ " else:\n",
+ " raise ValueError(\"Invalid operator\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The above function takes three arguments:\n",
+ "`a` and `b` are the integer numbers to be operated on;\n",
+ "`operator` is the operation to be performed.\n",
+ "We used type hints to define the types of the arguments and the return value.\n",
+ "\n",
+ "````mdx-code-block\n",
+ ":::tip\n",
+ "Always use type hints to define the types of the arguments and the return value\n",
+ "as they provide helpful hints to the agent about the tool's usage.\n",
+ ":::\n",
+ "````"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Registering Tools\n",
+ "\n",
+ "Once you have created a tool, you can register it with the agents that\n",
+ "are involved in conversation."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import os\n",
+ "from autogen import ConversableAgent\n",
+ "\n",
+ "# Let's first define the assistant agent that suggests tool calls.\n",
+ "assistant = ConversableAgent(\n",
+ " name=\"Assistant\",\n",
+ " system_message=\"You are a helpful AI assistant. \"\n",
+ " \"You can help with simple calculations. \"\n",
+ " \"Return 'TERMINATE' when the task is done.\",\n",
+ " llm_config={\"config_list\": [{\"model\": \"gpt-4\", \"api_key\": os.environ[\"OPENAI_API_KEY\"]}]},\n",
+ ")\n",
+ "\n",
+ "# The user proxy agent is used for interacting with the assistant agent\n",
+ "# and executes tool calls.\n",
+ "user_proxy = ConversableAgent(\n",
+ " name=\"User\",\n",
+ " llm_config=False,\n",
+ " is_termination_msg=lambda msg: msg.get(\"content\") is not None and \"TERMINATE\" in msg[\"content\"],\n",
+ " human_input_mode=\"NEVER\",\n",
+ ")\n",
+ "\n",
+ "# Register the tool signature with the assistant agent.\n",
+ "assistant.register_for_llm(name=\"calculator\", description=\"A simple calculator\")(calculator)\n",
+ "\n",
+ "# Register the tool function with the user proxy agent.\n",
+ "user_proxy.register_for_execution(name=\"calculator\")(calculator)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "In the above code, we registered the `calculator` function as a tool with\n",
+ "the assistant and user proxy agents. We also provide a name and a description\n",
+ "for the tool for the assistant agent to understand its usage.\n",
+ "\n",
+ "````mdx-code-block\n",
+ ":::tip\n",
+ "Always provide a clear and concise description for the tool as it helps the\n",
+ "agent's underlying LLM to understand the tool's usage.\n",
+ ":::\n",
+ "````\n",
+ "\n",
+ "Similar to code executors, a tool must be registered with at least two agents\n",
+ "for it to be useful in conversation. \n",
+ "The agent registered with the tool's signature\n",
+ "through \n",
+ "[`register_for_llm`](/docs/reference/agentchat/conversable_agent#register_for_llm)\n",
+ "can call the tool;\n",
+ "the agent registered with the tool's function object through \n",
+ "[`register_for_execution`](/docs/reference/agentchat/conversable_agent#register_for_execution)\n",
+ "can execute the tool's function."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Alternatively, you can use \n",
+ "[`autogen.register_function`](/docs/reference/agentchat/conversable_agent#register_function-1)\n",
+ "function to register a tool with both agents at once."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from autogen import register_function\n",
+ "\n",
+ "# Register the calculator function to the two agents.\n",
+ "register_function(\n",
+ " calculator,\n",
+ " caller=assistant, # The assistant agent can suggest calls to the calculator.\n",
+ " executor=user_proxy, # The user proxy agent can execute the calculator calls.\n",
+ " name=\"calculator\", # By default, the function name is used as the tool name.\n",
+ " description=\"A simple calculator\", # A description of the tool.\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Using Tool\n",
+ "\n",
+ "Once the tool is registered, we can use it in conversation.\n",
+ "In the code below, we ask the assistant to perform some arithmetic\n",
+ "calculation using the `calculator` tool."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
+ "\n",
+ "What is (44232 + 13312 / (232 - 32)) * 5?\n",
+ "\n",
+ "--------------------------------------------------------------------------------\n",
+ "\u001b[31m\n",
+ ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
+ "\u001b[33mAssistant\u001b[0m (to User):\n",
+ "\n",
+ "\u001b[32m***** Suggested tool Call (call_bACquf0OreI0VHh7rWiP6ZE7): calculator *****\u001b[0m\n",
+ "Arguments: \n",
+ "{\n",
+ " \"a\": 13312,\n",
+ " \"b\": 232 - 32,\n",
+ " \"operator\": \"/\"\n",
+ "}\n",
+ "\u001b[32m***************************************************************************\u001b[0m\n",
+ "\n",
+ "--------------------------------------------------------------------------------\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
+ "\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
+ "\n",
+ "\u001b[32m***** Response from calling tool \"call_bACquf0OreI0VHh7rWiP6ZE7\" *****\u001b[0m\n",
+ "Error: Expecting ',' delimiter: line 1 column 26 (char 25)\n",
+ " You argument should follow json format.\n",
+ "\u001b[32m**********************************************************************\u001b[0m\n",
+ "\n",
+ "--------------------------------------------------------------------------------\n",
+ "\u001b[31m\n",
+ ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
+ "\u001b[33mAssistant\u001b[0m (to User):\n",
+ "\n",
+ "\u001b[32m***** Suggested tool Call (call_2c0H5gzX9SWsJ05x7nEOVbav): calculator *****\u001b[0m\n",
+ "Arguments: \n",
+ "{\n",
+ " \"a\": 13312,\n",
+ " \"b\": 200,\n",
+ " \"operator\": \"/\"\n",
+ "}\n",
+ "\u001b[32m***************************************************************************\u001b[0m\n",
+ "\n",
+ "--------------------------------------------------------------------------------\n",
+ "\u001b[35m\n",
+ ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
+ "\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
+ "\n",
+ "\u001b[32m***** Response from calling tool \"call_2c0H5gzX9SWsJ05x7nEOVbav\" *****\u001b[0m\n",
+ "66\n",
+ "\u001b[32m**********************************************************************\u001b[0m\n",
+ "\n",
+ "--------------------------------------------------------------------------------\n",
+ "\u001b[31m\n",
+ ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
+ "\u001b[33mAssistant\u001b[0m (to User):\n",
+ "\n",
+ "\u001b[32m***** Suggested tool Call (call_ioceLhuKMpfU131E7TSQ8wCD): calculator *****\u001b[0m\n",
+ "Arguments: \n",
+ "{\n",
+ " \"a\": 44232,\n",
+ " \"b\": 66,\n",
+ " \"operator\": \"+\"\n",
+ "}\n",
+ "\u001b[32m***************************************************************************\u001b[0m\n",
+ "\n",
+ "--------------------------------------------------------------------------------\n",
+ "\u001b[35m\n",
+ ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
+ "\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
+ "\n",
+ "\u001b[32m***** Response from calling tool \"call_ioceLhuKMpfU131E7TSQ8wCD\" *****\u001b[0m\n",
+ "44298\n",
+ "\u001b[32m**********************************************************************\u001b[0m\n",
+ "\n",
+ "--------------------------------------------------------------------------------\n",
+ "\u001b[31m\n",
+ ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
+ "\u001b[33mAssistant\u001b[0m (to User):\n",
+ "\n",
+ "\u001b[32m***** Suggested tool Call (call_0rhx9vrbigcbqLssKLh4sS7j): calculator *****\u001b[0m\n",
+ "Arguments: \n",
+ "{\n",
+ " \"a\": 44298,\n",
+ " \"b\": 5,\n",
+ " \"operator\": \"*\"\n",
+ "}\n",
+ "\u001b[32m***************************************************************************\u001b[0m\n",
+ "\n",
+ "--------------------------------------------------------------------------------\n",
+ "\u001b[35m\n",
+ ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
+ "\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
+ "\n",
+ "\u001b[32m***** Response from calling tool \"call_0rhx9vrbigcbqLssKLh4sS7j\" *****\u001b[0m\n",
+ "221490\n",
+ "\u001b[32m**********************************************************************\u001b[0m\n",
+ "\n",
+ "--------------------------------------------------------------------------------\n",
+ "\u001b[31m\n",
+ ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
+ "\u001b[33mAssistant\u001b[0m (to User):\n",
+ "\n",
+ "The result of the calculation (44232 + 13312 / (232 - 32)) * 5 is 221490. \n",
+ "\n",
+ "TERMINATE\n",
+ "\n",
+ "--------------------------------------------------------------------------------\n"
+ ]
+ }
+ ],
+ "source": [
+ "chat_result = user_proxy.initiate_chat(assistant, message=\"What is (44232 + 13312 / (232 - 32)) * 5?\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let's verify the answer:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "221490"
+ ]
+ },
+ "execution_count": 5,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "(44232 + int(13312 / (232 - 32))) * 5"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The answer is correct.\n",
+ "You can see that the assistant is able to understand the tool's usage\n",
+ "and perform calculation correctly."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Tool Schema\n",
+ "\n",
+ "If you are familiar with [OpenAI's tool use API](https://platform.openai.com/docs/guides/function-calling), \n",
+ "you might be wondering\n",
+ "why we didn't create a tool schema.\n",
+ "In fact, the tool schema is automatically generated from the function signature\n",
+ "and the type hints.\n",
+ "You can see the tool schema by inspecting the `llm_config` attribute of the\n",
+ "agent."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "[{'type': 'function',\n",
+ " 'function': {'description': 'A simple calculator',\n",
+ " 'name': 'calculator',\n",
+ " 'parameters': {'type': 'object',\n",
+ " 'properties': {'a': {'type': 'integer', 'description': 'a'},\n",
+ " 'b': {'type': 'integer', 'description': 'b'},\n",
+ " 'operator': {'enum': ['+', '-', '*', '/'],\n",
+ " 'type': 'string',\n",
+ " 'description': 'operator'}},\n",
+ " 'required': ['a', 'b', 'operator']}}}]"
+ ]
+ },
+ "execution_count": 6,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "assistant.llm_config[\"tools\"]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "You can see the tool schema has been automatically generated from the function\n",
+ "signature and the type hints, as well as the description.\n",
+ "This is why it is important to use type hints and provide a clear description\n",
+ "for the tool as the LLM uses them to understand the tool's usage."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "You can also use Pydantic model for the type hints to provide more complex\n",
+ "type schema. In the example below, we use a Pydantic model to define the\n",
+ "calculator input."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 33,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from pydantic import BaseModel, Field\n",
+ "\n",
+ "\n",
+ "class CalculatorInput(BaseModel):\n",
+ " a: Annotated[int, Field(description=\"The first number.\")]\n",
+ " b: Annotated[int, Field(description=\"The second number.\")]\n",
+ " operator: Annotated[Operator, Field(description=\"The operator.\")]\n",
+ "\n",
+ "\n",
+ "def calculator(input: Annotated[CalculatorInput, \"Input to the calculator.\"]) -> int:\n",
+ " if input.operator == \"+\":\n",
+ " return input.a + input.b\n",
+ " elif input.operator == \"-\":\n",
+ " return input.a - input.b\n",
+ " elif input.operator == \"*\":\n",
+ " return input.a * input.b\n",
+ " elif input.operator == \"/\":\n",
+ " return int(input.a / input.b)\n",
+ " else:\n",
+ " raise ValueError(\"Invalid operator\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Same as before, we register the tool with the agents using the name `\"calculator\"`.\n",
+ "\n",
+ "````mdx-code-block\n",
+ ":::tip\n",
+ "Registering tool to the same name will override the previous tool.\n",
+ ":::\n",
+ "````"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "assistant.register_for_llm(name=\"calculator\", description=\"A calculator tool that accepts nested expression as input\")(\n",
+ " calculator\n",
+ ")\n",
+ "user_proxy.register_for_execution(name=\"calculator\")(calculator)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "You can see the tool schema has been updated to reflect the new type schema."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 28,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "[{'type': 'function',\n",
+ " 'function': {'description': 'A calculator tool that accepts nested expression as input',\n",
+ " 'name': 'calculator',\n",
+ " 'parameters': {'type': 'object',\n",
+ " 'properties': {'input': {'properties': {'a': {'description': 'The first number.',\n",
+ " 'title': 'A',\n",
+ " 'type': 'integer'},\n",
+ " 'b': {'description': 'The second number.',\n",
+ " 'title': 'B',\n",
+ " 'type': 'integer'},\n",
+ " 'operator': {'description': 'The operator.',\n",
+ " 'enum': ['+', '-', '*', '/'],\n",
+ " 'title': 'Operator',\n",
+ " 'type': 'string'}},\n",
+ " 'required': ['a', 'b', 'operator'],\n",
+ " 'title': 'CalculatorInput',\n",
+ " 'type': 'object',\n",
+ " 'description': 'Input to the calculator.'}},\n",
+ " 'required': ['input']}}}]"
+ ]
+ },
+ "execution_count": 28,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "assistant.llm_config[\"tools\"]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let's use the tool in conversation."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 32,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
+ "\n",
+ "What is (1423 - 123) / 3 + (32 + 23) * 5?\n",
+ "\n",
+ "--------------------------------------------------------------------------------\n",
+ "\u001b[31m\n",
+ ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
+ "\u001b[33mAssistant\u001b[0m (to User):\n",
+ "\n",
+ "\u001b[32m***** Suggested tool Call (call_t9By3vewGRoSLWsvdTR7p8Zo): calculator *****\u001b[0m\n",
+ "Arguments: \n",
+ "{\n",
+ " \"input\": {\n",
+ " \"a\": 1423,\n",
+ " \"b\": 123,\n",
+ " \"operator\": \"-\"\n",
+ " }\n",
+ "}\n",
+ "\u001b[32m***************************************************************************\u001b[0m\n",
+ "\n",
+ "--------------------------------------------------------------------------------\n",
+ "\u001b[35m\n",
+ ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
+ "\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
+ "\n",
+ "\u001b[32m***** Response from calling tool \"call_t9By3vewGRoSLWsvdTR7p8Zo\" *****\u001b[0m\n",
+ "1300\n",
+ "\u001b[32m**********************************************************************\u001b[0m\n",
+ "\n",
+ "--------------------------------------------------------------------------------\n",
+ "\u001b[31m\n",
+ ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
+ "\u001b[33mAssistant\u001b[0m (to User):\n",
+ "\n",
+ "\u001b[32m***** Suggested tool Call (call_rhecyhVCo0Y8HPL193xOUPE6): calculator *****\u001b[0m\n",
+ "Arguments: \n",
+ "{\n",
+ " \"input\": {\n",
+ " \"a\": 1300,\n",
+ " \"b\": 3,\n",
+ " \"operator\": \"/\"\n",
+ " }\n",
+ "}\n",
+ "\u001b[32m***************************************************************************\u001b[0m\n",
+ "\n",
+ "--------------------------------------------------------------------------------\n",
+ "\u001b[35m\n",
+ ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
+ "\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
+ "\n",
+ "\u001b[32m***** Response from calling tool \"call_rhecyhVCo0Y8HPL193xOUPE6\" *****\u001b[0m\n",
+ "433\n",
+ "\u001b[32m**********************************************************************\u001b[0m\n",
+ "\n",
+ "--------------------------------------------------------------------------------\n",
+ "\u001b[31m\n",
+ ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
+ "\u001b[33mAssistant\u001b[0m (to User):\n",
+ "\n",
+ "\u001b[32m***** Suggested tool Call (call_zDpq9J5MYAsL7uS8cobOwa7S): calculator *****\u001b[0m\n",
+ "Arguments: \n",
+ "{\n",
+ " \"input\": {\n",
+ " \"a\": 32,\n",
+ " \"b\": 23,\n",
+ " \"operator\": \"+\"\n",
+ " }\n",
+ "}\n",
+ "\u001b[32m***************************************************************************\u001b[0m\n",
+ "\n",
+ "--------------------------------------------------------------------------------\n",
+ "\u001b[35m\n",
+ ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
+ "\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
+ "\n",
+ "\u001b[32m***** Response from calling tool \"call_zDpq9J5MYAsL7uS8cobOwa7S\" *****\u001b[0m\n",
+ "55\n",
+ "\u001b[32m**********************************************************************\u001b[0m\n",
+ "\n",
+ "--------------------------------------------------------------------------------\n",
+ "\u001b[31m\n",
+ ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
+ "\u001b[33mAssistant\u001b[0m (to User):\n",
+ "\n",
+ "\u001b[32m***** Suggested tool Call (call_mjDuVMojOIdaxmvDUIF4QtVi): calculator *****\u001b[0m\n",
+ "Arguments: \n",
+ "{\n",
+ " \"input\": {\n",
+ " \"a\": 55,\n",
+ " \"b\": 5,\n",
+ " \"operator\": \"*\"\n",
+ " }\n",
+ "}\n",
+ "\u001b[32m***************************************************************************\u001b[0m\n",
+ "\n",
+ "--------------------------------------------------------------------------------\n",
+ "\u001b[35m\n",
+ ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
+ "\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
+ "\n",
+ "\u001b[32m***** Response from calling tool \"call_mjDuVMojOIdaxmvDUIF4QtVi\" *****\u001b[0m\n",
+ "275\n",
+ "\u001b[32m**********************************************************************\u001b[0m\n",
+ "\n",
+ "--------------------------------------------------------------------------------\n",
+ "\u001b[31m\n",
+ ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
+ "\u001b[33mAssistant\u001b[0m (to User):\n",
+ "\n",
+ "\u001b[32m***** Suggested tool Call (call_hpirkAGKOewZstsDOxL2sYNW): calculator *****\u001b[0m\n",
+ "Arguments: \n",
+ "{\n",
+ " \"input\": {\n",
+ " \"a\": 433,\n",
+ " \"b\": 275,\n",
+ " \"operator\": \"+\"\n",
+ " }\n",
+ "}\n",
+ "\u001b[32m***************************************************************************\u001b[0m\n",
+ "\n",
+ "--------------------------------------------------------------------------------\n",
+ "\u001b[35m\n",
+ ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
+ "\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
+ "\n",
+ "\u001b[32m***** Response from calling tool \"call_hpirkAGKOewZstsDOxL2sYNW\" *****\u001b[0m\n",
+ "708\n",
+ "\u001b[32m**********************************************************************\u001b[0m\n",
+ "\n",
+ "--------------------------------------------------------------------------------\n",
+ "\u001b[31m\n",
+ ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
+ "\u001b[33mAssistant\u001b[0m (to User):\n",
+ "\n",
+ "The result of the calculation is 708. \n",
+ "\n",
+ "TERMINATE\n",
+ "\n",
+ "--------------------------------------------------------------------------------\n"
+ ]
+ }
+ ],
+ "source": [
+ "chat_result = user_proxy.initiate_chat(assistant, message=\"What is (1423 - 123) / 3 + (32 + 23) * 5?\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let's verify the answer:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 31,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "708"
+ ]
+ },
+ "execution_count": 31,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "int((1423 - 123) / 3) + (32 + 23) * 5"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Again, the answer is correct. You can see that the assistant is able to understand\n",
+ "the new tool schema and perform calculation correctly."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Summary\n",
+ "\n",
+ "In this chapter, we showed you how to create, register and use tools.\n",
+ "Tools allows agents to perform actions without writing arbitrary code.\n",
+ "In the next chapter, we will introduce conversation patterns, and show\n",
+ "how to use the result of a conversation."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "autogen",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.11.5"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/website/docs/tutorial/what-next.md b/website/docs/tutorial/what-next.md
index 7e3820cad..74a28364a 100644
--- a/website/docs/tutorial/what-next.md
+++ b/website/docs/tutorial/what-next.md
@@ -6,7 +6,9 @@ topics:
1. **Chat with LLMs**: In [Human in the Loop](./human-in-the-loop) we covered
the basic human-in-the-loop usage. You can try to hook up different LLMs
- using proxy servers like [Ollama](https://github.com/ollama/ollama), and
+ using local model servers like
+ [Ollama](https://github.com/ollama/ollama)
+ and [LM Studio](https://lmstudio.ai/), and
chat with them using the human-in-the-loop component of your human proxy
agent.
2. **Prompt Engineering**: In [Code Executors](./code-executors) we
@@ -18,6 +20,7 @@ topics:
3. **Complex Tasks**: In [ConversationPatterns](./conversation-patterns)
we covered the basic conversation patterns. You can try to find other tasks
that can be decomposed into these patterns, and leverage the code executors
+ and tools
to make the agents more powerful.
## Dig Deeper
diff --git a/website/sidebars.js b/website/sidebars.js
index 1abe16d68..44829e607 100644
--- a/website/sidebars.js
+++ b/website/sidebars.js
@@ -46,6 +46,11 @@
id: 'tutorial/code-executors',
label: 'Code Executors',
},
+ {
+ type: 'doc',
+ id: 'tutorial/tool-use',
+ label: 'Tool Use',
+ },
{
type: 'doc',
id: 'tutorial/conversation-patterns',