Skip to content

armgit / dspy

Public Imported from GitHub
0

DSPy: The framework for programming—not prompting—language models

Python

Add tutorial on building agents with DSPy (#8288)

Chen Qian99d0e70606a6
diff --git a/docs/docs/tutorials/build_ai_program/index.md b/docs/docs/tutorials/build_ai_program/index.md
index 11fd4dc1c..e4943dbb4 100644
--- a/docs/docs/tutorials/build_ai_program/index.md
+++ b/docs/docs/tutorials/build_ai_program/index.md
@@ -1,5 +1,6 @@
- [Retrieval-Augmented Generation (RAG)](/tutorials/rag/)
- [Building RAG as Agent](/tutorials/agents/)
+- [Build AI Agents with DSPy](/tutorials/customer_service_agent/)
- [Entity Extraction](/tutorials/entity_extraction/)
- [Classification](/tutorials/classification/)
- [Multi-Hop RAG](/tutorials/multihop_search/)
diff --git a/docs/docs/tutorials/customer_service_agent/index.ipynb b/docs/docs/tutorials/customer_service_agent/index.ipynb
new file mode 100644
index 000000000..bc5ad9068
--- /dev/null
+++ b/docs/docs/tutorials/customer_service_agent/index.ipynb
@@ -0,0 +1,1194 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "XicO2XPbEBVD"
+ },
+ "source": [
+ "# Build AI Agents with DSPy\n",
+ "\n",
+ "In this tutorial, we will walk you through how to build an AI agents with DSPy. AI agents refer to the system that can autonomously perceive its environment, make decisions, and take actions to achieve specific goals. Unlike a single model prompt, an agent typically follows a loop of reasoning, planning, and acting, often integrating tools like search engines, APIs, or memory to complete complex tasks.\n",
+ "\n",
+ "This tutorial focuses on a popular architecture of AI agents called ReAct, standing for **Re**asoning and **Act**ing, which provides a task description along with a list of tools to LM, then lets LM decide whether to call tools for more obseravations, or generate the final output.\n",
+ "\n",
+ "As the demo, let's build a simple airline customer service agent that can do the following:\n",
+ "\n",
+ "- Book new trips on behalf of the user.\n",
+ "- Modify existing trips, including flight change and cancellation.\n",
+ "- On tasks it cannot handle, raise a customer support ticket.\n",
+ "\n",
+ "We will build it from `dspy.ReAct` module.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "8dYvGkoWG3C7"
+ },
+ "source": [
+ "## Install Dependencies\n",
+ "\n",
+ "Before starting, let's install the required packages:\n",
+ "\n",
+ "```\n",
+ "!pip install -qU dspy pydantic\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "W_rhpcE2HAMO"
+ },
+ "source": [
+ "## Define Tools\n",
+ "\n",
+ "We need to prepare a list of tools so that the agent can behave like a human airline service agent:\n",
+ "\n",
+ "- `fetch_flight_info`: get flight information for certain dates.\n",
+ "- `pick_flight`: pick the best flight based on some criteria.\n",
+ "- `book_flight`: book a flight on behalf of the user.\n",
+ "- `fetch_itinerary`: get the information of a booked itinerary.\n",
+ "- `cancel_itinerary`: cancel a booked itinerary.\n",
+ "- `get_user_info`: get users' information.\n",
+ "- `file_ticket`: file a backlog ticket to have human assist.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "Zr0r8FVgI8vQ"
+ },
+ "source": [
+ "### Define Data Structure\n",
+ "\n",
+ "Before defining the tools, we need to define the data structure. In real production, this will be the database schema. As a demo, we just define the data structure as [pydantic models](https://docs.pydantic.dev/latest/concepts/models/) for simplicity."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {
+ "id": "nKjBVt3MJQR-"
+ },
+ "outputs": [],
+ "source": [
+ "from pydantic import BaseModel\n",
+ "\n",
+ "class Date(BaseModel):\n",
+ " # Somehow LLM is bad at specifying `datetime.datetime`, so\n",
+ " # we define a custom class to represent the date.\n",
+ " year: int\n",
+ " month: int\n",
+ " day: int\n",
+ " hour: int\n",
+ "\n",
+ "class UserProfile(BaseModel):\n",
+ " user_id: str\n",
+ " name: str\n",
+ " email: str\n",
+ "\n",
+ "class Flight(BaseModel):\n",
+ " flight_id: str\n",
+ " date_time: Date\n",
+ " origin: str\n",
+ " destination: str\n",
+ " duration: float\n",
+ " price: float\n",
+ "\n",
+ "class Itinerary(BaseModel):\n",
+ " confirmation_number: str\n",
+ " user_profile: UserProfile\n",
+ " flight: Flight\n",
+ "\n",
+ "class Ticket(BaseModel):\n",
+ " user_request: str\n",
+ " user_profile: UserProfile"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "JrZkTeI4JbcW"
+ },
+ "source": [
+ "### Create Dummy Data\n",
+ "\n",
+ "Let's also create some dummy data so that the airline agent can do the work. We need to create a few flights and a few users, and initilize empty dictionaries for the itineraries and custom support tickets."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {
+ "id": "FJDzsHjKJwWb"
+ },
+ "outputs": [],
+ "source": [
+ "user_database = {\n",
+ " \"Adam\": UserProfile(user_id=\"1\", name=\"Adam\", email=\"[email protected]\"),\n",
+ " \"Bob\": UserProfile(user_id=\"2\", name=\"Bob\", email=\"[email protected]\"),\n",
+ " \"Chelsie\": UserProfile(user_id=\"3\", name=\"Chelsie\", email=\"[email protected]\"),\n",
+ " \"David\": UserProfile(user_id=\"4\", name=\"David\", email=\"[email protected]\"),\n",
+ "}\n",
+ "\n",
+ "flight_database = {\n",
+ " \"DA123\": Flight(\n",
+ " flight_id=\"DA123\", # DSPy Airline 123\n",
+ " origin=\"SFO\",\n",
+ " destination=\"JFK\",\n",
+ " date_time=Date(year=2025, month=9, day=1, hour=1),\n",
+ " duration=3,\n",
+ " price=200,\n",
+ " ),\n",
+ " \"DA125\": Flight(\n",
+ " flight_id=\"DA125\",\n",
+ " origin=\"SFO\",\n",
+ " destination=\"JFK\",\n",
+ " date_time=Date(year=2025, month=9, day=1, hour=7),\n",
+ " duration=9,\n",
+ " price=500,\n",
+ " ),\n",
+ " \"DA456\": Flight(\n",
+ " flight_id=\"DA456\",\n",
+ " origin=\"SFO\",\n",
+ " destination=\"SNA\",\n",
+ " date_time=Date(year=2025, month=10, day=1, hour=1),\n",
+ " duration=2,\n",
+ " price=100,\n",
+ " ),\n",
+ " \"DA460\": Flight(\n",
+ " flight_id=\"DA460\",\n",
+ " origin=\"SFO\",\n",
+ " destination=\"SNA\",\n",
+ " date_time=Date(year=2025, month=10, day=1, hour=9),\n",
+ " duration=2,\n",
+ " price=120,\n",
+ " ),\n",
+ "}\n",
+ "\n",
+ "itinery_database = {}\n",
+ "ticket_database = {}"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "-iGVa5WiKFp_"
+ },
+ "source": [
+ "### Define the Tools\n",
+ "\n",
+ "Now we can define the tools. In order to have `dspy.ReAct` function properly, every function should:\n",
+ "\n",
+ "- Have a docstring which defines what the tool does. If the function name is self-explanable, then you can leave the docstring empty.\n",
+ "- Have type hint for the arguments, which is necessary for LM to generate the arguments in the right format."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {
+ "id": "-wQGzf6qK7P5"
+ },
+ "outputs": [],
+ "source": [
+ "import random\n",
+ "import string\n",
+ "\n",
+ "\n",
+ "def fetch_flight_info(date: Date, origin: str, destination: str):\n",
+ " \"\"\"Fetch flight information from origin to destination on the given date\"\"\"\n",
+ " flights = []\n",
+ "\n",
+ " for flight_id, flight in flight_database.items():\n",
+ " if (\n",
+ " flight.date_time.year == date.year\n",
+ " and flight.date_time.month == date.month\n",
+ " and flight.date_time.day == date.day\n",
+ " and flight.origin == origin\n",
+ " and flight.destination == destination\n",
+ " ):\n",
+ " flights.append(flight)\n",
+ " if len(flights) == 0:\n",
+ " raise ValueError(\"No matching flight found!\")\n",
+ " return flights\n",
+ "\n",
+ "\n",
+ "def fetch_itinerary(confirmation_number: str):\n",
+ " \"\"\"Fetch a booked itinerary information from database\"\"\"\n",
+ " return itinery_database.get(confirmation_number)\n",
+ "\n",
+ "\n",
+ "def pick_flight(flights: list[Flight]):\n",
+ " \"\"\"Pick up the best flight that matches users' request. we pick the shortest, and cheaper one on ties.\"\"\"\n",
+ " sorted_flights = sorted(\n",
+ " flights,\n",
+ " key=lambda x: (\n",
+ " x.get(\"duration\") if isinstance(x, dict) else x.duration,\n",
+ " x.get(\"price\") if isinstance(x, dict) else x.price,\n",
+ " ),\n",
+ " )\n",
+ " return sorted_flights[0]\n",
+ "\n",
+ "\n",
+ "def _generate_id(length=8):\n",
+ " chars = string.ascii_lowercase + string.digits\n",
+ " return \"\".join(random.choices(chars, k=length))\n",
+ "\n",
+ "\n",
+ "def book_flight(flight: Flight, user_profile: UserProfile):\n",
+ " \"\"\"Book a flight on behalf of the user.\"\"\"\n",
+ " confirmation_number = _generate_id()\n",
+ " while confirmation_number in itinery_database:\n",
+ " confirmation_number = _generate_id()\n",
+ " itinery_database[confirmation_number] = Itinerary(\n",
+ " confirmation_number=confirmation_number,\n",
+ " user_profile=user_profile,\n",
+ " flight=flight,\n",
+ " )\n",
+ " return confirmation_number, itinery_database[confirmation_number]\n",
+ "\n",
+ "\n",
+ "def cancel_itinerary(confirmation_number: str, user_profile: UserProfile):\n",
+ " \"\"\"Cancel an itinerary on behalf of the user.\"\"\"\n",
+ " if confirmation_number in itinery_database:\n",
+ " del itinery_database[confirmation_number]\n",
+ " return\n",
+ " raise ValueError(\"Cannot find the itinerary, please check your confirmation number.\")\n",
+ "\n",
+ "\n",
+ "def get_user_info(name: str):\n",
+ " \"\"\"Fetch the user profile from database with given name.\"\"\"\n",
+ " return user_database.get(name)\n",
+ "\n",
+ "\n",
+ "def file_ticket(user_request: str, user_profile: UserProfile):\n",
+ " \"\"\"File a customer support ticket if this is something the agent cannot handle.\"\"\"\n",
+ " ticket_id = _generate_id(length=6)\n",
+ " ticket_database[ticket_id] = Ticket(\n",
+ " user_request=user_request,\n",
+ " user_profile=user_profile,\n",
+ " )\n",
+ " return ticket_id\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "rzibYLS0LqXc"
+ },
+ "source": [
+ "### Create ReAct Agent\n",
+ "\n",
+ "Now we can create the ReAct agent via `dspy.ReAct`. We need to provide a signature to `dspy.ReAct` to define task, and the inputs and outputs of the agent, and tell it about the tools it can access."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {
+ "id": "OZ2qOWLGLk2J"
+ },
+ "outputs": [],
+ "source": [
+ "import dspy\n",
+ "\n",
+ "class DSPyAirlineCustomerSerice(dspy.Signature):\n",
+ " \"\"\"You are an airline customer service agent that helps user book and manage flights.\n",
+ "\n",
+ " You are given a list of tools to handle user request, and you should decide the right tool to use in order to\n",
+ " fullfil users' request.\"\"\"\n",
+ "\n",
+ " user_request: str = dspy.InputField()\n",
+ " process_result: str = dspy.OutputField(\n",
+ " desc=(\n",
+ " \"Message that summarizes the process result, and the information users need, e.g., the \"\n",
+ " \"confirmation_number if a new flight is booked.\"\n",
+ " )\n",
+ " )"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {
+ "id": "ooDb84LIM5Pn"
+ },
+ "outputs": [],
+ "source": [
+ "agent = dspy.ReAct(\n",
+ " DSPyAirlineCustomerSerice,\n",
+ " tools = [\n",
+ " fetch_flight_info,\n",
+ " fetch_itinerary,\n",
+ " pick_flight,\n",
+ " book_flight,\n",
+ " cancel_itinerary,\n",
+ " get_user_info,\n",
+ " file_ticket,\n",
+ " ]\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "4NA2QtQgNyrO"
+ },
+ "source": [
+ "## Use the Agent\n",
+ "\n",
+ "To interact with the agent, simply provide the request through `user_request`, and the agent will start doing its job."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "CXnSE6P1OHbm"
+ },
+ "source": [
+ "Select a language model and set up the API keys. We are using gpt-4o-mini here, but you can change to other models. For how to configure the language model, please refer to [this guide](https://dspy.ai/learn/programming/language_models/)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {
+ "id": "5F3TVB1DOW82"
+ },
+ "outputs": [],
+ "source": [
+ "import os\n",
+ "\n",
+ "os.environ[\"OPENAI_API_KEY\"] = \"{your openai key}\"\n",
+ "\n",
+ "dspy.configure(lm=dspy.LM(\"openai/gpt-4o-mini\"))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "hL3q0iS4Nwn8",
+ "outputId": "460af1b7-bceb-4edc-8d51-30f87eb7d454"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Prediction(\n",
+ " trajectory={'thought_0': 'I need to fetch flight information for the route from SFO to JFK on the specified date of 09/01/2025. Once I have the flight options, I can help Adam pick the best flight and proceed with the booking.', 'tool_name_0': 'fetch_flight_info', 'tool_args_0': {'date': {'year': 2025, 'month': 9, 'day': 1, 'hour': 0}, 'origin': 'SFO', 'destination': 'JFK'}, 'observation_0': [Flight(flight_id='DA123', date_time=Date(year=2025, month=9, day=1, hour=1), origin='SFO', destination='JFK', duration=3.0, price=200.0), Flight(flight_id='DA125', date_time=Date(year=2025, month=9, day=1, hour=7), origin='SFO', destination='JFK', duration=9.0, price=500.0)], 'thought_1': 'I have received the flight options from SFO to JFK on 09/01/2025. There are two flights available: DA123 at 1 AM for $200 and DA125 at 7 AM for $500. I will now pick the best flight, which is the cheaper option, DA123.', 'tool_name_1': 'pick_flight', 'tool_args_1': {'flights': [{'flight_id': 'DA123', 'date_time': {'year': 2025, 'month': 9, 'day': 1, 'hour': 1}, 'origin': 'SFO', 'destination': 'JFK', 'duration': 3.0, 'price': 200.0}, {'flight_id': 'DA125', 'date_time': {'year': 2025, 'month': 9, 'day': 1, 'hour': 7}, 'origin': 'SFO', 'destination': 'JFK', 'duration': 9.0, 'price': 500.0}]}, 'observation_1': Flight(flight_id='DA123', date_time=Date(year=2025, month=9, day=1, hour=1), origin='SFO', destination='JFK', duration=3.0, price=200.0), 'thought_2': \"I have successfully picked the best flight, which is DA123. Now, I need to gather Adam's user profile information to proceed with the booking.\", 'tool_name_2': 'get_user_info', 'tool_args_2': {'name': 'Adam'}, 'observation_2': UserProfile(user_id='1', name='Adam', email='[email protected]'), 'thought_3': \"I have gathered Adam's user profile information. Now I can proceed to book the selected flight DA123 for him.\", 'tool_name_3': 'book_flight', 'tool_args_3': {'flight': {'flight_id': 'DA123', 'date_time': {'year': 2025, 'month': 9, 'day': 1, 'hour': 1}, 'origin': 'SFO', 'destination': 'JFK', 'duration': 3.0, 'price': 200.0}, 'user_profile': {'user_id': '1', 'name': 'Adam', 'email': '[email protected]'}}, 'observation_3': ('94ldyhsx', Itinerary(confirmation_number='94ldyhsx', user_profile=UserProfile(user_id='1', name='Adam', email='[email protected]'), flight=Flight(flight_id='DA123', date_time=Date(year=2025, month=9, day=1, hour=1), origin='SFO', destination='JFK', duration=3.0, price=200.0))), 'thought_4': \"Adam's flight has been successfully booked, and I have received the confirmation number. I will now mark the task as complete.\", 'tool_name_4': 'finish', 'tool_args_4': {}, 'observation_4': 'Completed.'},\n",
+ " reasoning=\"I successfully fetched flight information for the route from SFO to JFK on 09/01/2025. There were two available flights: DA123 at 1 AM for $200 and DA125 at 7 AM for $500. I selected the cheaper option, DA123, and then retrieved Adam's user profile information to proceed with the booking. After booking the flight, I received a confirmation number for the reservation.\",\n",
+ " process_result='Your flight from SFO to JFK on 09/01/2025 has been successfully booked. Your confirmation number is 94ldyhsx.'\n",
+ ")\n"
+ ]
+ }
+ ],
+ "source": [
+ "result = agent(user_request=\"please help me book a flight from SFO to JFK on 09/01/2025, my name is Adam\")\n",
+ "print(result)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "5kwbgMNUWQUX"
+ },
+ "source": [
+ "We can see the booked itinerarie in the database."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "J3fO7G3yVtoa",
+ "outputId": "76f665ca-81a4-45df-bf08-d66b946a45c7"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "{'94ldyhsx': Itinerary(confirmation_number='94ldyhsx', user_profile=UserProfile(user_id='1', name='Adam', email='[email protected]'), flight=Flight(flight_id='DA123', date_time=Date(year=2025, month=9, day=1, hour=1), origin='SFO', destination='JFK', duration=3.0, price=200.0))}\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(itinery_database)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "mxFF1tMrWU_7"
+ },
+ "source": [
+ "### Interpret the Result\n",
+ "\n",
+ "The result contains the the `process_result` as required by the user, and a `reasoning` field that carries the reasoning behind the answer. In addition, it has a `trajectory` field which contains:\n",
+ "\n",
+ "- Reasoning (thought) at each step\n",
+ "- Tools picked by LM at each step\n",
+ "- Arguments for tool calling, determined by LM at each step\n",
+ "- Tool execution results at each step\n",
+ "\n",
+ "Behind scene, the `dspy.ReAct` is executing a loop, which accumulates tool call information along with the task description, and send to the LM until hits `max_iters` or the LM decides to wrap up. To better interpret the process, let's use `dspy.inspect_history()` to see what's happening inside each step.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "Pm7lZhLZV_ko",
+ "outputId": "e4387b2d-7368-4aea-8705-1127c9c8fd31"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\u001b[34m[2025-05-28T01:06:46.819048]\u001b[0m\n",
+ "\n",
+ "\u001b[31mSystem message:\u001b[0m\n",
+ "\n",
+ "Your input fields are:\n",
+ "1. `user_request` (str)\n",
+ "2. `trajectory` (str)\n",
+ "Your output fields are:\n",
+ "1. `next_thought` (str)\n",
+ "2. `next_tool_name` (Literal['fetch_flight_info', 'fetch_itinerary', 'pick_flight', 'book_flight', 'cancel_itinerary', 'get_user_info', 'file_ticket', 'finish'])\n",
+ "3. `next_tool_args` (dict[str, Any])\n",
+ "All interactions will be structured in the following way, with the appropriate values filled in.\n",
+ "\n",
+ "[[ ## user_request ## ]]\n",
+ "{user_request}\n",
+ "\n",
+ "[[ ## trajectory ## ]]\n",
+ "{trajectory}\n",
+ "\n",
+ "[[ ## next_thought ## ]]\n",
+ "{next_thought}\n",
+ "\n",
+ "[[ ## next_tool_name ## ]]\n",
+ "{next_tool_name} # note: the value you produce must exactly match (no extra characters) one of: fetch_flight_info; fetch_itinerary; pick_flight; book_flight; cancel_itinerary; get_user_info; file_ticket; finish\n",
+ "\n",
+ "[[ ## next_tool_args ## ]]\n",
+ "{next_tool_args} # note: the value you produce must adhere to the JSON schema: {\"type\": \"object\", \"additionalProperties\": true}\n",
+ "\n",
+ "[[ ## completed ## ]]\n",
+ "In adhering to this structure, your objective is: \n",
+ " You are an airline customer service agent that helps user book and manage flights. \n",
+ " \n",
+ " You are given a list of tools to handle user request, and you should decide the right tool to use in order to\n",
+ " fullfil users' request.\n",
+ " \n",
+ " You are an Agent. In each episode, you will be given the fields `user_request` as input. And you can see your past trajectory so far.\n",
+ " Your goal is to use one or more of the supplied tools to collect any necessary information for producing `process_result`.\n",
+ " \n",
+ " To do this, you will interleave next_thought, next_tool_name, and next_tool_args in each turn, and also when finishing the task.\n",
+ " After each tool call, you receive a resulting observation, which gets appended to your trajectory.\n",
+ " \n",
+ " When writing next_thought, you may reason about the current situation and plan for future steps.\n",
+ " When selecting the next_tool_name and its next_tool_args, the tool must be one of:\n",
+ " \n",
+ " (1) fetch_flight_info, whose description is <desc>Fetch flight information from origin to destination on the given date</desc>. It takes arguments {'date': {'properties': {'year': {'title': 'Year', 'type': 'integer'}, 'month': {'title': 'Month', 'type': 'integer'}, 'day': {'title': 'Day', 'type': 'integer'}, 'hour': {'title': 'Hour', 'type': 'integer'}}, 'required': ['year', 'month', 'day', 'hour'], 'title': 'Date', 'type': 'object'}, 'origin': {'type': 'string'}, 'destination': {'type': 'string'}} in JSON format.\n",
+ " (2) fetch_itinerary, whose description is <desc>Fetch a booked itinerary information from database</desc>. It takes arguments {'confirmation_number': {'type': 'string'}} in JSON format.\n",
+ " (3) pick_flight, whose description is <desc>Pick up the best flight that matches users' request. we pick the shortest, and cheaper one on ties.</desc>. It takes arguments {'flights': {'$defs': {'Date': {'properties': {'year': {'title': 'Year', 'type': 'integer'}, 'month': {'title': 'Month', 'type': 'integer'}, 'day': {'title': 'Day', 'type': 'integer'}, 'hour': {'title': 'Hour', 'type': 'integer'}}, 'required': ['year', 'month', 'day', 'hour'], 'title': 'Date', 'type': 'object'}, 'Flight': {'properties': {'flight_id': {'title': 'Flight Id', 'type': 'string'}, 'date_time': {'$ref': '#/$defs/Date'}, 'origin': {'title': 'Origin', 'type': 'string'}, 'destination': {'title': 'Destination', 'type': 'string'}, 'duration': {'title': 'Duration', 'type': 'number'}, 'price': {'title': 'Price', 'type': 'number'}}, 'required': ['flight_id', 'date_time', 'origin', 'destination', 'duration', 'price'], 'title': 'Flight', 'type': 'object'}}, 'items': {'$ref': '#/$defs/Flight'}, 'type': 'array'}} in JSON format.\n",
+ " (4) book_flight, whose description is <desc>Book a flight on behalf of the user.</desc>. It takes arguments {'flight': {'properties': {'flight_id': {'title': 'Flight Id', 'type': 'string'}, 'date_time': {'properties': {'year': {'title': 'Year', 'type': 'integer'}, 'month': {'title': 'Month', 'type': 'integer'}, 'day': {'title': 'Day', 'type': 'integer'}, 'hour': {'title': 'Hour', 'type': 'integer'}}, 'required': ['year', 'month', 'day', 'hour'], 'title': 'Date', 'type': 'object'}, 'origin': {'title': 'Origin', 'type': 'string'}, 'destination': {'title': 'Destination', 'type': 'string'}, 'duration': {'title': 'Duration', 'type': 'number'}, 'price': {'title': 'Price', 'type': 'number'}}, 'required': ['flight_id', 'date_time', 'origin', 'destination', 'duration', 'price'], 'title': 'Flight', 'type': 'object'}, 'user_profile': {'properties': {'user_id': {'title': 'User Id', 'type': 'string'}, 'name': {'title': 'Name', 'type': 'string'}, 'email': {'title': 'Email', 'type': 'string'}}, 'required': ['user_id', 'name', 'email'], 'title': 'UserProfile', 'type': 'object'}} in JSON format.\n",
+ " (5) cancel_itinerary, whose description is <desc>Cancel an itinerary on behalf of the user.</desc>. It takes arguments {'confirmation_number': {'type': 'string'}, 'user_profile': {'properties': {'user_id': {'title': 'User Id', 'type': 'string'}, 'name': {'title': 'Name', 'type': 'string'}, 'email': {'title': 'Email', 'type': 'string'}}, 'required': ['user_id', 'name', 'email'], 'title': 'UserProfile', 'type': 'object'}} in JSON format.\n",
+ " (6) get_user_info, whose description is <desc>Fetch the user profile from database with given name.</desc>. It takes arguments {'name': {'type': 'string'}} in JSON format.\n",
+ " (7) file_ticket, whose description is <desc>File a customer support ticket if this is something the agent cannot handle.</desc>. It takes arguments {'user_request': {'type': 'string'}, 'user_profile': {'properties': {'user_id': {'title': 'User Id', 'type': 'string'}, 'name': {'title': 'Name', 'type': 'string'}, 'email': {'title': 'Email', 'type': 'string'}}, 'required': ['user_id', 'name', 'email'], 'title': 'UserProfile', 'type': 'object'}} in JSON format.\n",
+ " (8) finish, whose description is <desc>Marks the task as complete. That is, signals that all information for producing the outputs, i.e. `process_result`, are now available to be extracted.</desc>. It takes arguments {} in JSON format.\n",
+ "\n",
+ "\n",
+ "\u001b[31mUser message:\u001b[0m\n",
+ "\n",
+ "[[ ## user_request ## ]]\n",
+ "please help me book a flight from SFO to JFK on 09/01/2025, my name is Adam\n",
+ "\n",
+ "[[ ## trajectory ## ]]\n",
+ "\n",
+ "\n",
+ "Respond with the corresponding output fields, starting with the field `[[ ## next_thought ## ]]`, then `[[ ## next_tool_name ## ]]` (must be formatted as a valid Python Literal['fetch_flight_info', 'fetch_itinerary', 'pick_flight', 'book_flight', 'cancel_itinerary', 'get_user_info', 'file_ticket', 'finish']), then `[[ ## next_tool_args ## ]]` (must be formatted as a valid Python dict[str, Any]), and then ending with the marker for `[[ ## completed ## ]]`.\n",
+ "\n",
+ "\n",
+ "\u001b[31mResponse:\u001b[0m\n",
+ "\n",
+ "\u001b[32m[[ ## next_thought ## ]]\n",
+ "I need to fetch flight information for the route from SFO to JFK on the specified date of 09/01/2025. Once I have the flight options, I can help Adam pick the best flight and proceed with the booking.\n",
+ "\n",
+ "[[ ## next_tool_name ## ]]\n",
+ "fetch_flight_info\n",
+ "\n",
+ "[[ ## next_tool_args ## ]]\n",
+ "{\"date\":{\"year\":2025,\"month\":9,\"day\":1,\"hour\":0},\"origin\":\"SFO\",\"destination\":\"JFK\"}\n",
+ "\n",
+ "[[ ## completed ## ]]\u001b[0m\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\u001b[34m[2025-05-28T01:06:46.832209]\u001b[0m\n",
+ "\n",
+ "\u001b[31mSystem message:\u001b[0m\n",
+ "\n",
+ "Your input fields are:\n",
+ "1. `user_request` (str)\n",
+ "2. `trajectory` (str)\n",
+ "Your output fields are:\n",
+ "1. `next_thought` (str)\n",
+ "2. `next_tool_name` (Literal['fetch_flight_info', 'fetch_itinerary', 'pick_flight', 'book_flight', 'cancel_itinerary', 'get_user_info', 'file_ticket', 'finish'])\n",
+ "3. `next_tool_args` (dict[str, Any])\n",
+ "All interactions will be structured in the following way, with the appropriate values filled in.\n",
+ "\n",
+ "[[ ## user_request ## ]]\n",
+ "{user_request}\n",
+ "\n",
+ "[[ ## trajectory ## ]]\n",
+ "{trajectory}\n",
+ "\n",
+ "[[ ## next_thought ## ]]\n",
+ "{next_thought}\n",
+ "\n",
+ "[[ ## next_tool_name ## ]]\n",
+ "{next_tool_name} # note: the value you produce must exactly match (no extra characters) one of: fetch_flight_info; fetch_itinerary; pick_flight; book_flight; cancel_itinerary; get_user_info; file_ticket; finish\n",
+ "\n",
+ "[[ ## next_tool_args ## ]]\n",
+ "{next_tool_args} # note: the value you produce must adhere to the JSON schema: {\"type\": \"object\", \"additionalProperties\": true}\n",
+ "\n",
+ "[[ ## completed ## ]]\n",
+ "In adhering to this structure, your objective is: \n",
+ " You are an airline customer service agent that helps user book and manage flights. \n",
+ " \n",
+ " You are given a list of tools to handle user request, and you should decide the right tool to use in order to\n",
+ " fullfil users' request.\n",
+ " \n",
+ " You are an Agent. In each episode, you will be given the fields `user_request` as input. And you can see your past trajectory so far.\n",
+ " Your goal is to use one or more of the supplied tools to collect any necessary information for producing `process_result`.\n",
+ " \n",
+ " To do this, you will interleave next_thought, next_tool_name, and next_tool_args in each turn, and also when finishing the task.\n",
+ " After each tool call, you receive a resulting observation, which gets appended to your trajectory.\n",
+ " \n",
+ " When writing next_thought, you may reason about the current situation and plan for future steps.\n",
+ " When selecting the next_tool_name and its next_tool_args, the tool must be one of:\n",
+ " \n",
+ " (1) fetch_flight_info, whose description is <desc>Fetch flight information from origin to destination on the given date</desc>. It takes arguments {'date': {'properties': {'year': {'title': 'Year', 'type': 'integer'}, 'month': {'title': 'Month', 'type': 'integer'}, 'day': {'title': 'Day', 'type': 'integer'}, 'hour': {'title': 'Hour', 'type': 'integer'}}, 'required': ['year', 'month', 'day', 'hour'], 'title': 'Date', 'type': 'object'}, 'origin': {'type': 'string'}, 'destination': {'type': 'string'}} in JSON format.\n",
+ " (2) fetch_itinerary, whose description is <desc>Fetch a booked itinerary information from database</desc>. It takes arguments {'confirmation_number': {'type': 'string'}} in JSON format.\n",
+ " (3) pick_flight, whose description is <desc>Pick up the best flight that matches users' request. we pick the shortest, and cheaper one on ties.</desc>. It takes arguments {'flights': {'$defs': {'Date': {'properties': {'year': {'title': 'Year', 'type': 'integer'}, 'month': {'title': 'Month', 'type': 'integer'}, 'day': {'title': 'Day', 'type': 'integer'}, 'hour': {'title': 'Hour', 'type': 'integer'}}, 'required': ['year', 'month', 'day', 'hour'], 'title': 'Date', 'type': 'object'}, 'Flight': {'properties': {'flight_id': {'title': 'Flight Id', 'type': 'string'}, 'date_time': {'$ref': '#/$defs/Date'}, 'origin': {'title': 'Origin', 'type': 'string'}, 'destination': {'title': 'Destination', 'type': 'string'}, 'duration': {'title': 'Duration', 'type': 'number'}, 'price': {'title': 'Price', 'type': 'number'}}, 'required': ['flight_id', 'date_time', 'origin', 'destination', 'duration', 'price'], 'title': 'Flight', 'type': 'object'}}, 'items': {'$ref': '#/$defs/Flight'}, 'type': 'array'}} in JSON format.\n",
+ " (4) book_flight, whose description is <desc>Book a flight on behalf of the user.</desc>. It takes arguments {'flight': {'properties': {'flight_id': {'title': 'Flight Id', 'type': 'string'}, 'date_time': {'properties': {'year': {'title': 'Year', 'type': 'integer'}, 'month': {'title': 'Month', 'type': 'integer'}, 'day': {'title': 'Day', 'type': 'integer'}, 'hour': {'title': 'Hour', 'type': 'integer'}}, 'required': ['year', 'month', 'day', 'hour'], 'title': 'Date', 'type': 'object'}, 'origin': {'title': 'Origin', 'type': 'string'}, 'destination': {'title': 'Destination', 'type': 'string'}, 'duration': {'title': 'Duration', 'type': 'number'}, 'price': {'title': 'Price', 'type': 'number'}}, 'required': ['flight_id', 'date_time', 'origin', 'destination', 'duration', 'price'], 'title': 'Flight', 'type': 'object'}, 'user_profile': {'properties': {'user_id': {'title': 'User Id', 'type': 'string'}, 'name': {'title': 'Name', 'type': 'string'}, 'email': {'title': 'Email', 'type': 'string'}}, 'required': ['user_id', 'name', 'email'], 'title': 'UserProfile', 'type': 'object'}} in JSON format.\n",
+ " (5) cancel_itinerary, whose description is <desc>Cancel an itinerary on behalf of the user.</desc>. It takes arguments {'confirmation_number': {'type': 'string'}, 'user_profile': {'properties': {'user_id': {'title': 'User Id', 'type': 'string'}, 'name': {'title': 'Name', 'type': 'string'}, 'email': {'title': 'Email', 'type': 'string'}}, 'required': ['user_id', 'name', 'email'], 'title': 'UserProfile', 'type': 'object'}} in JSON format.\n",
+ " (6) get_user_info, whose description is <desc>Fetch the user profile from database with given name.</desc>. It takes arguments {'name': {'type': 'string'}} in JSON format.\n",
+ " (7) file_ticket, whose description is <desc>File a customer support ticket if this is something the agent cannot handle.</desc>. It takes arguments {'user_request': {'type': 'string'}, 'user_profile': {'properties': {'user_id': {'title': 'User Id', 'type': 'string'}, 'name': {'title': 'Name', 'type': 'string'}, 'email': {'title': 'Email', 'type': 'string'}}, 'required': ['user_id', 'name', 'email'], 'title': 'UserProfile', 'type': 'object'}} in JSON format.\n",
+ " (8) finish, whose description is <desc>Marks the task as complete. That is, signals that all information for producing the outputs, i.e. `process_result`, are now available to be extracted.</desc>. It takes arguments {} in JSON format.\n",
+ "\n",
+ "\n",
+ "\u001b[31mUser message:\u001b[0m\n",
+ "\n",
+ "[[ ## user_request ## ]]\n",
+ "please help me book a flight from SFO to JFK on 09/01/2025, my name is Adam\n",
+ "\n",
+ "[[ ## trajectory ## ]]\n",
+ "[[ ## thought_0 ## ]]\n",
+ "I need to fetch flight information for the route from SFO to JFK on the specified date of 09/01/2025. Once I have the flight options, I can help Adam pick the best flight and proceed with the booking.\n",
+ "\n",
+ "[[ ## tool_name_0 ## ]]\n",
+ "fetch_flight_info\n",
+ "\n",
+ "[[ ## tool_args_0 ## ]]\n",
+ "{\"date\": {\"year\": 2025, \"month\": 9, \"day\": 1, \"hour\": 0}, \"origin\": \"SFO\", \"destination\": \"JFK\"}\n",
+ "\n",
+ "[[ ## observation_0 ## ]]\n",
+ "[1] «flight_id='DA123' date_time=Date(year=2025, month=9, day=1, hour=1) origin='SFO' destination='JFK' duration=3.0 price=200.0»\n",
+ "[2] «flight_id='DA125' date_time=Date(year=2025, month=9, day=1, hour=7) origin='SFO' destination='JFK' duration=9.0 price=500.0»\n",
+ "\n",
+ "Respond with the corresponding output fields, starting with the field `[[ ## next_thought ## ]]`, then `[[ ## next_tool_name ## ]]` (must be formatted as a valid Python Literal['fetch_flight_info', 'fetch_itinerary', 'pick_flight', 'book_flight', 'cancel_itinerary', 'get_user_info', 'file_ticket', 'finish']), then `[[ ## next_tool_args ## ]]` (must be formatted as a valid Python dict[str, Any]), and then ending with the marker for `[[ ## completed ## ]]`.\n",
+ "\n",
+ "\n",
+ "\u001b[31mResponse:\u001b[0m\n",
+ "\n",
+ "\u001b[32m[[ ## next_thought ## ]]\n",
+ "I have received the flight options from SFO to JFK on 09/01/2025. There are two flights available: DA123 at 1 AM for $200 and DA125 at 7 AM for $500. I will now pick the best flight, which is the cheaper option, DA123.\n",
+ "\n",
+ "[[ ## next_tool_name ## ]]\n",
+ "pick_flight\n",
+ "\n",
+ "[[ ## next_tool_args ## ]]\n",
+ "{\"flights\":[{\"flight_id\":\"DA123\",\"date_time\":{\"year\":2025,\"month\":9,\"day\":1,\"hour\":1},\"origin\":\"SFO\",\"destination\":\"JFK\",\"duration\":3.0,\"price\":200.0},{\"flight_id\":\"DA125\",\"date_time\":{\"year\":2025,\"month\":9,\"day\":1,\"hour\":7},\"origin\":\"SFO\",\"destination\":\"JFK\",\"duration\":9.0,\"price\":500.0}]}\n",
+ "\n",
+ "[[ ## completed ## ]]\u001b[0m\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\u001b[34m[2025-05-28T01:06:46.844268]\u001b[0m\n",
+ "\n",
+ "\u001b[31mSystem message:\u001b[0m\n",
+ "\n",
+ "Your input fields are:\n",
+ "1. `user_request` (str)\n",
+ "2. `trajectory` (str)\n",
+ "Your output fields are:\n",
+ "1. `next_thought` (str)\n",
+ "2. `next_tool_name` (Literal['fetch_flight_info', 'fetch_itinerary', 'pick_flight', 'book_flight', 'cancel_itinerary', 'get_user_info', 'file_ticket', 'finish'])\n",
+ "3. `next_tool_args` (dict[str, Any])\n",
+ "All interactions will be structured in the following way, with the appropriate values filled in.\n",
+ "\n",
+ "[[ ## user_request ## ]]\n",
+ "{user_request}\n",
+ "\n",
+ "[[ ## trajectory ## ]]\n",
+ "{trajectory}\n",
+ "\n",
+ "[[ ## next_thought ## ]]\n",
+ "{next_thought}\n",
+ "\n",
+ "[[ ## next_tool_name ## ]]\n",
+ "{next_tool_name} # note: the value you produce must exactly match (no extra characters) one of: fetch_flight_info; fetch_itinerary; pick_flight; book_flight; cancel_itinerary; get_user_info; file_ticket; finish\n",
+ "\n",
+ "[[ ## next_tool_args ## ]]\n",
+ "{next_tool_args} # note: the value you produce must adhere to the JSON schema: {\"type\": \"object\", \"additionalProperties\": true}\n",
+ "\n",
+ "[[ ## completed ## ]]\n",
+ "In adhering to this structure, your objective is: \n",
+ " You are an airline customer service agent that helps user book and manage flights. \n",
+ " \n",
+ " You are given a list of tools to handle user request, and you should decide the right tool to use in order to\n",
+ " fullfil users' request.\n",
+ " \n",
+ " You are an Agent. In each episode, you will be given the fields `user_request` as input. And you can see your past trajectory so far.\n",
+ " Your goal is to use one or more of the supplied tools to collect any necessary information for producing `process_result`.\n",
+ " \n",
+ " To do this, you will interleave next_thought, next_tool_name, and next_tool_args in each turn, and also when finishing the task.\n",
+ " After each tool call, you receive a resulting observation, which gets appended to your trajectory.\n",
+ " \n",
+ " When writing next_thought, you may reason about the current situation and plan for future steps.\n",
+ " When selecting the next_tool_name and its next_tool_args, the tool must be one of:\n",
+ " \n",
+ " (1) fetch_flight_info, whose description is <desc>Fetch flight information from origin to destination on the given date</desc>. It takes arguments {'date': {'properties': {'year': {'title': 'Year', 'type': 'integer'}, 'month': {'title': 'Month', 'type': 'integer'}, 'day': {'title': 'Day', 'type': 'integer'}, 'hour': {'title': 'Hour', 'type': 'integer'}}, 'required': ['year', 'month', 'day', 'hour'], 'title': 'Date', 'type': 'object'}, 'origin': {'type': 'string'}, 'destination': {'type': 'string'}} in JSON format.\n",
+ " (2) fetch_itinerary, whose description is <desc>Fetch a booked itinerary information from database</desc>. It takes arguments {'confirmation_number': {'type': 'string'}} in JSON format.\n",
+ " (3) pick_flight, whose description is <desc>Pick up the best flight that matches users' request. we pick the shortest, and cheaper one on ties.</desc>. It takes arguments {'flights': {'$defs': {'Date': {'properties': {'year': {'title': 'Year', 'type': 'integer'}, 'month': {'title': 'Month', 'type': 'integer'}, 'day': {'title': 'Day', 'type': 'integer'}, 'hour': {'title': 'Hour', 'type': 'integer'}}, 'required': ['year', 'month', 'day', 'hour'], 'title': 'Date', 'type': 'object'}, 'Flight': {'properties': {'flight_id': {'title': 'Flight Id', 'type': 'string'}, 'date_time': {'$ref': '#/$defs/Date'}, 'origin': {'title': 'Origin', 'type': 'string'}, 'destination': {'title': 'Destination', 'type': 'string'}, 'duration': {'title': 'Duration', 'type': 'number'}, 'price': {'title': 'Price', 'type': 'number'}}, 'required': ['flight_id', 'date_time', 'origin', 'destination', 'duration', 'price'], 'title': 'Flight', 'type': 'object'}}, 'items': {'$ref': '#/$defs/Flight'}, 'type': 'array'}} in JSON format.\n",
+ " (4) book_flight, whose description is <desc>Book a flight on behalf of the user.</desc>. It takes arguments {'flight': {'properties': {'flight_id': {'title': 'Flight Id', 'type': 'string'}, 'date_time': {'properties': {'year': {'title': 'Year', 'type': 'integer'}, 'month': {'title': 'Month', 'type': 'integer'}, 'day': {'title': 'Day', 'type': 'integer'}, 'hour': {'title': 'Hour', 'type': 'integer'}}, 'required': ['year', 'month', 'day', 'hour'], 'title': 'Date', 'type': 'object'}, 'origin': {'title': 'Origin', 'type': 'string'}, 'destination': {'title': 'Destination', 'type': 'string'}, 'duration': {'title': 'Duration', 'type': 'number'}, 'price': {'title': 'Price', 'type': 'number'}}, 'required': ['flight_id', 'date_time', 'origin', 'destination', 'duration', 'price'], 'title': 'Flight', 'type': 'object'}, 'user_profile': {'properties': {'user_id': {'title': 'User Id', 'type': 'string'}, 'name': {'title': 'Name', 'type': 'string'}, 'email': {'title': 'Email', 'type': 'string'}}, 'required': ['user_id', 'name', 'email'], 'title': 'UserProfile', 'type': 'object'}} in JSON format.\n",
+ " (5) cancel_itinerary, whose description is <desc>Cancel an itinerary on behalf of the user.</desc>. It takes arguments {'confirmation_number': {'type': 'string'}, 'user_profile': {'properties': {'user_id': {'title': 'User Id', 'type': 'string'}, 'name': {'title': 'Name', 'type': 'string'}, 'email': {'title': 'Email', 'type': 'string'}}, 'required': ['user_id', 'name', 'email'], 'title': 'UserProfile', 'type': 'object'}} in JSON format.\n",
+ " (6) get_user_info, whose description is <desc>Fetch the user profile from database with given name.</desc>. It takes arguments {'name': {'type': 'string'}} in JSON format.\n",
+ " (7) file_ticket, whose description is <desc>File a customer support ticket if this is something the agent cannot handle.</desc>. It takes arguments {'user_request': {'type': 'string'}, 'user_profile': {'properties': {'user_id': {'title': 'User Id', 'type': 'string'}, 'name': {'title': 'Name', 'type': 'string'}, 'email': {'title': 'Email', 'type': 'string'}}, 'required': ['user_id', 'name', 'email'], 'title': 'UserProfile', 'type': 'object'}} in JSON format.\n",
+ " (8) finish, whose description is <desc>Marks the task as complete. That is, signals that all information for producing the outputs, i.e. `process_result`, are now available to be extracted.</desc>. It takes arguments {} in JSON format.\n",
+ "\n",
+ "\n",
+ "\u001b[31mUser message:\u001b[0m\n",
+ "\n",
+ "[[ ## user_request ## ]]\n",
+ "please help me book a flight from SFO to JFK on 09/01/2025, my name is Adam\n",
+ "\n",
+ "[[ ## trajectory ## ]]\n",
+ "[[ ## thought_0 ## ]]\n",
+ "I need to fetch flight information for the route from SFO to JFK on the specified date of 09/01/2025. Once I have the flight options, I can help Adam pick the best flight and proceed with the booking.\n",
+ "\n",
+ "[[ ## tool_name_0 ## ]]\n",
+ "fetch_flight_info\n",
+ "\n",
+ "[[ ## tool_args_0 ## ]]\n",
+ "{\"date\": {\"year\": 2025, \"month\": 9, \"day\": 1, \"hour\": 0}, \"origin\": \"SFO\", \"destination\": \"JFK\"}\n",
+ "\n",
+ "[[ ## observation_0 ## ]]\n",
+ "[1] «flight_id='DA123' date_time=Date(year=2025, month=9, day=1, hour=1) origin='SFO' destination='JFK' duration=3.0 price=200.0»\n",
+ "[2] «flight_id='DA125' date_time=Date(year=2025, month=9, day=1, hour=7) origin='SFO' destination='JFK' duration=9.0 price=500.0»\n",
+ "\n",
+ "[[ ## thought_1 ## ]]\n",
+ "I have received the flight options from SFO to JFK on 09/01/2025. There are two flights available: DA123 at 1 AM for $200 and DA125 at 7 AM for $500. I will now pick the best flight, which is the cheaper option, DA123.\n",
+ "\n",
+ "[[ ## tool_name_1 ## ]]\n",
+ "pick_flight\n",
+ "\n",
+ "[[ ## tool_args_1 ## ]]\n",
+ "{\"flights\": [{\"flight_id\": \"DA123\", \"date_time\": {\"year\": 2025, \"month\": 9, \"day\": 1, \"hour\": 1}, \"origin\": \"SFO\", \"destination\": \"JFK\", \"duration\": 3.0, \"price\": 200.0}, {\"flight_id\": \"DA125\", \"date_time\": {\"year\": 2025, \"month\": 9, \"day\": 1, \"hour\": 7}, \"origin\": \"SFO\", \"destination\": \"JFK\", \"duration\": 9.0, \"price\": 500.0}]}\n",
+ "\n",
+ "[[ ## observation_1 ## ]]\n",
+ "{\"flight_id\": \"DA123\", \"date_time\": {\"year\": 2025, \"month\": 9, \"day\": 1, \"hour\": 1}, \"origin\": \"SFO\", \"destination\": \"JFK\", \"duration\": 3.0, \"price\": 200.0}\n",
+ "\n",
+ "Respond with the corresponding output fields, starting with the field `[[ ## next_thought ## ]]`, then `[[ ## next_tool_name ## ]]` (must be formatted as a valid Python Literal['fetch_flight_info', 'fetch_itinerary', 'pick_flight', 'book_flight', 'cancel_itinerary', 'get_user_info', 'file_ticket', 'finish']), then `[[ ## next_tool_args ## ]]` (must be formatted as a valid Python dict[str, Any]), and then ending with the marker for `[[ ## completed ## ]]`.\n",
+ "\n",
+ "\n",
+ "\u001b[31mResponse:\u001b[0m\n",
+ "\n",
+ "\u001b[32m[[ ## next_thought ## ]]\n",
+ "I have successfully picked the best flight, which is DA123. Now, I need to gather Adam's user profile information to proceed with the booking.\n",
+ "\n",
+ "[[ ## next_tool_name ## ]]\n",
+ "get_user_info\n",
+ "\n",
+ "[[ ## next_tool_args ## ]]\n",
+ "{\"name\": \"Adam\"}\n",
+ "\n",
+ "[[ ## completed ## ]]\u001b[0m\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\u001b[34m[2025-05-28T01:06:46.851917]\u001b[0m\n",
+ "\n",
+ "\u001b[31mSystem message:\u001b[0m\n",
+ "\n",
+ "Your input fields are:\n",
+ "1. `user_request` (str)\n",
+ "2. `trajectory` (str)\n",
+ "Your output fields are:\n",
+ "1. `next_thought` (str)\n",
+ "2. `next_tool_name` (Literal['fetch_flight_info', 'fetch_itinerary', 'pick_flight', 'book_flight', 'cancel_itinerary', 'get_user_info', 'file_ticket', 'finish'])\n",
+ "3. `next_tool_args` (dict[str, Any])\n",
+ "All interactions will be structured in the following way, with the appropriate values filled in.\n",
+ "\n",
+ "[[ ## user_request ## ]]\n",
+ "{user_request}\n",
+ "\n",
+ "[[ ## trajectory ## ]]\n",
+ "{trajectory}\n",
+ "\n",
+ "[[ ## next_thought ## ]]\n",
+ "{next_thought}\n",
+ "\n",
+ "[[ ## next_tool_name ## ]]\n",
+ "{next_tool_name} # note: the value you produce must exactly match (no extra characters) one of: fetch_flight_info; fetch_itinerary; pick_flight; book_flight; cancel_itinerary; get_user_info; file_ticket; finish\n",
+ "\n",
+ "[[ ## next_tool_args ## ]]\n",
+ "{next_tool_args} # note: the value you produce must adhere to the JSON schema: {\"type\": \"object\", \"additionalProperties\": true}\n",
+ "\n",
+ "[[ ## completed ## ]]\n",
+ "In adhering to this structure, your objective is: \n",
+ " You are an airline customer service agent that helps user book and manage flights. \n",
+ " \n",
+ " You are given a list of tools to handle user request, and you should decide the right tool to use in order to\n",
+ " fullfil users' request.\n",
+ " \n",
+ " You are an Agent. In each episode, you will be given the fields `user_request` as input. And you can see your past trajectory so far.\n",
+ " Your goal is to use one or more of the supplied tools to collect any necessary information for producing `process_result`.\n",
+ " \n",
+ " To do this, you will interleave next_thought, next_tool_name, and next_tool_args in each turn, and also when finishing the task.\n",
+ " After each tool call, you receive a resulting observation, which gets appended to your trajectory.\n",
+ " \n",
+ " When writing next_thought, you may reason about the current situation and plan for future steps.\n",
+ " When selecting the next_tool_name and its next_tool_args, the tool must be one of:\n",
+ " \n",
+ " (1) fetch_flight_info, whose description is <desc>Fetch flight information from origin to destination on the given date</desc>. It takes arguments {'date': {'properties': {'year': {'title': 'Year', 'type': 'integer'}, 'month': {'title': 'Month', 'type': 'integer'}, 'day': {'title': 'Day', 'type': 'integer'}, 'hour': {'title': 'Hour', 'type': 'integer'}}, 'required': ['year', 'month', 'day', 'hour'], 'title': 'Date', 'type': 'object'}, 'origin': {'type': 'string'}, 'destination': {'type': 'string'}} in JSON format.\n",
+ " (2) fetch_itinerary, whose description is <desc>Fetch a booked itinerary information from database</desc>. It takes arguments {'confirmation_number': {'type': 'string'}} in JSON format.\n",
+ " (3) pick_flight, whose description is <desc>Pick up the best flight that matches users' request. we pick the shortest, and cheaper one on ties.</desc>. It takes arguments {'flights': {'$defs': {'Date': {'properties': {'year': {'title': 'Year', 'type': 'integer'}, 'month': {'title': 'Month', 'type': 'integer'}, 'day': {'title': 'Day', 'type': 'integer'}, 'hour': {'title': 'Hour', 'type': 'integer'}}, 'required': ['year', 'month', 'day', 'hour'], 'title': 'Date', 'type': 'object'}, 'Flight': {'properties': {'flight_id': {'title': 'Flight Id', 'type': 'string'}, 'date_time': {'$ref': '#/$defs/Date'}, 'origin': {'title': 'Origin', 'type': 'string'}, 'destination': {'title': 'Destination', 'type': 'string'}, 'duration': {'title': 'Duration', 'type': 'number'}, 'price': {'title': 'Price', 'type': 'number'}}, 'required': ['flight_id', 'date_time', 'origin', 'destination', 'duration', 'price'], 'title': 'Flight', 'type': 'object'}}, 'items': {'$ref': '#/$defs/Flight'}, 'type': 'array'}} in JSON format.\n",
+ " (4) book_flight, whose description is <desc>Book a flight on behalf of the user.</desc>. It takes arguments {'flight': {'properties': {'flight_id': {'title': 'Flight Id', 'type': 'string'}, 'date_time': {'properties': {'year': {'title': 'Year', 'type': 'integer'}, 'month': {'title': 'Month', 'type': 'integer'}, 'day': {'title': 'Day', 'type': 'integer'}, 'hour': {'title': 'Hour', 'type': 'integer'}}, 'required': ['year', 'month', 'day', 'hour'], 'title': 'Date', 'type': 'object'}, 'origin': {'title': 'Origin', 'type': 'string'}, 'destination': {'title': 'Destination', 'type': 'string'}, 'duration': {'title': 'Duration', 'type': 'number'}, 'price': {'title': 'Price', 'type': 'number'}}, 'required': ['flight_id', 'date_time', 'origin', 'destination', 'duration', 'price'], 'title': 'Flight', 'type': 'object'}, 'user_profile': {'properties': {'user_id': {'title': 'User Id', 'type': 'string'}, 'name': {'title': 'Name', 'type': 'string'}, 'email': {'title': 'Email', 'type': 'string'}}, 'required': ['user_id', 'name', 'email'], 'title': 'UserProfile', 'type': 'object'}} in JSON format.\n",
+ " (5) cancel_itinerary, whose description is <desc>Cancel an itinerary on behalf of the user.</desc>. It takes arguments {'confirmation_number': {'type': 'string'}, 'user_profile': {'properties': {'user_id': {'title': 'User Id', 'type': 'string'}, 'name': {'title': 'Name', 'type': 'string'}, 'email': {'title': 'Email', 'type': 'string'}}, 'required': ['user_id', 'name', 'email'], 'title': 'UserProfile', 'type': 'object'}} in JSON format.\n",
+ " (6) get_user_info, whose description is <desc>Fetch the user profile from database with given name.</desc>. It takes arguments {'name': {'type': 'string'}} in JSON format.\n",
+ " (7) file_ticket, whose description is <desc>File a customer support ticket if this is something the agent cannot handle.</desc>. It takes arguments {'user_request': {'type': 'string'}, 'user_profile': {'properties': {'user_id': {'title': 'User Id', 'type': 'string'}, 'name': {'title': 'Name', 'type': 'string'}, 'email': {'title': 'Email', 'type': 'string'}}, 'required': ['user_id', 'name', 'email'], 'title': 'UserProfile', 'type': 'object'}} in JSON format.\n",
+ " (8) finish, whose description is <desc>Marks the task as complete. That is, signals that all information for producing the outputs, i.e. `process_result`, are now available to be extracted.</desc>. It takes arguments {} in JSON format.\n",
+ "\n",
+ "\n",
+ "\u001b[31mUser message:\u001b[0m\n",
+ "\n",
+ "[[ ## user_request ## ]]\n",
+ "please help me book a flight from SFO to JFK on 09/01/2025, my name is Adam\n",
+ "\n",
+ "[[ ## trajectory ## ]]\n",
+ "[[ ## thought_0 ## ]]\n",
+ "I need to fetch flight information for the route from SFO to JFK on the specified date of 09/01/2025. Once I have the flight options, I can help Adam pick the best flight and proceed with the booking.\n",
+ "\n",
+ "[[ ## tool_name_0 ## ]]\n",
+ "fetch_flight_info\n",
+ "\n",
+ "[[ ## tool_args_0 ## ]]\n",
+ "{\"date\": {\"year\": 2025, \"month\": 9, \"day\": 1, \"hour\": 0}, \"origin\": \"SFO\", \"destination\": \"JFK\"}\n",
+ "\n",
+ "[[ ## observation_0 ## ]]\n",
+ "[1] «flight_id='DA123' date_time=Date(year=2025, month=9, day=1, hour=1) origin='SFO' destination='JFK' duration=3.0 price=200.0»\n",
+ "[2] «flight_id='DA125' date_time=Date(year=2025, month=9, day=1, hour=7) origin='SFO' destination='JFK' duration=9.0 price=500.0»\n",
+ "\n",
+ "[[ ## thought_1 ## ]]\n",
+ "I have received the flight options from SFO to JFK on 09/01/2025. There are two flights available: DA123 at 1 AM for $200 and DA125 at 7 AM for $500. I will now pick the best flight, which is the cheaper option, DA123.\n",
+ "\n",
+ "[[ ## tool_name_1 ## ]]\n",
+ "pick_flight\n",
+ "\n",
+ "[[ ## tool_args_1 ## ]]\n",
+ "{\"flights\": [{\"flight_id\": \"DA123\", \"date_time\": {\"year\": 2025, \"month\": 9, \"day\": 1, \"hour\": 1}, \"origin\": \"SFO\", \"destination\": \"JFK\", \"duration\": 3.0, \"price\": 200.0}, {\"flight_id\": \"DA125\", \"date_time\": {\"year\": 2025, \"month\": 9, \"day\": 1, \"hour\": 7}, \"origin\": \"SFO\", \"destination\": \"JFK\", \"duration\": 9.0, \"price\": 500.0}]}\n",
+ "\n",
+ "[[ ## observation_1 ## ]]\n",
+ "{\"flight_id\": \"DA123\", \"date_time\": {\"year\": 2025, \"month\": 9, \"day\": 1, \"hour\": 1}, \"origin\": \"SFO\", \"destination\": \"JFK\", \"duration\": 3.0, \"price\": 200.0}\n",
+ "\n",
+ "[[ ## thought_2 ## ]]\n",
+ "I have successfully picked the best flight, which is DA123. Now, I need to gather Adam's user profile information to proceed with the booking.\n",
+ "\n",
+ "[[ ## tool_name_2 ## ]]\n",
+ "get_user_info\n",
+ "\n",
+ "[[ ## tool_args_2 ## ]]\n",
+ "{\"name\": \"Adam\"}\n",
+ "\n",
+ "[[ ## observation_2 ## ]]\n",
+ "{\"user_id\": \"1\", \"name\": \"Adam\", \"email\": \"[email protected]\"}\n",
+ "\n",
+ "Respond with the corresponding output fields, starting with the field `[[ ## next_thought ## ]]`, then `[[ ## next_tool_name ## ]]` (must be formatted as a valid Python Literal['fetch_flight_info', 'fetch_itinerary', 'pick_flight', 'book_flight', 'cancel_itinerary', 'get_user_info', 'file_ticket', 'finish']), then `[[ ## next_tool_args ## ]]` (must be formatted as a valid Python dict[str, Any]), and then ending with the marker for `[[ ## completed ## ]]`.\n",
+ "\n",
+ "\n",
+ "\u001b[31mResponse:\u001b[0m\n",
+ "\n",
+ "\u001b[32m[[ ## next_thought ## ]]\n",
+ "I have gathered Adam's user profile information. Now I can proceed to book the selected flight DA123 for him.\n",
+ "\n",
+ "[[ ## next_tool_name ## ]]\n",
+ "book_flight\n",
+ "\n",
+ "[[ ## next_tool_args ## ]]\n",
+ "{\"flight\":{\"flight_id\":\"DA123\",\"date_time\":{\"year\":2025,\"month\":9,\"day\":1,\"hour\":1},\"origin\":\"SFO\",\"destination\":\"JFK\",\"duration\":3.0,\"price\":200.0},\"user_profile\":{\"user_id\":\"1\",\"name\":\"Adam\",\"email\":\"[email protected]\"}}\n",
+ "\n",
+ "[[ ## completed ## ]]\u001b[0m\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\u001b[34m[2025-05-28T01:06:49.025746]\u001b[0m\n",
+ "\n",
+ "\u001b[31mSystem message:\u001b[0m\n",
+ "\n",
+ "Your input fields are:\n",
+ "1. `user_request` (str)\n",
+ "2. `trajectory` (str)\n",
+ "Your output fields are:\n",
+ "1. `next_thought` (str)\n",
+ "2. `next_tool_name` (Literal['fetch_flight_info', 'fetch_itinerary', 'pick_flight', 'book_flight', 'cancel_itinerary', 'get_user_info', 'file_ticket', 'finish'])\n",
+ "3. `next_tool_args` (dict[str, Any])\n",
+ "All interactions will be structured in the following way, with the appropriate values filled in.\n",
+ "\n",
+ "[[ ## user_request ## ]]\n",
+ "{user_request}\n",
+ "\n",
+ "[[ ## trajectory ## ]]\n",
+ "{trajectory}\n",
+ "\n",
+ "[[ ## next_thought ## ]]\n",
+ "{next_thought}\n",
+ "\n",
+ "[[ ## next_tool_name ## ]]\n",
+ "{next_tool_name} # note: the value you produce must exactly match (no extra characters) one of: fetch_flight_info; fetch_itinerary; pick_flight; book_flight; cancel_itinerary; get_user_info; file_ticket; finish\n",
+ "\n",
+ "[[ ## next_tool_args ## ]]\n",
+ "{next_tool_args} # note: the value you produce must adhere to the JSON schema: {\"type\": \"object\", \"additionalProperties\": true}\n",
+ "\n",
+ "[[ ## completed ## ]]\n",
+ "In adhering to this structure, your objective is: \n",
+ " You are an airline customer service agent that helps user book and manage flights. \n",
+ " \n",
+ " You are given a list of tools to handle user request, and you should decide the right tool to use in order to\n",
+ " fullfil users' request.\n",
+ " \n",
+ " You are an Agent. In each episode, you will be given the fields `user_request` as input. And you can see your past trajectory so far.\n",
+ " Your goal is to use one or more of the supplied tools to collect any necessary information for producing `process_result`.\n",
+ " \n",
+ " To do this, you will interleave next_thought, next_tool_name, and next_tool_args in each turn, and also when finishing the task.\n",
+ " After each tool call, you receive a resulting observation, which gets appended to your trajectory.\n",
+ " \n",
+ " When writing next_thought, you may reason about the current situation and plan for future steps.\n",
+ " When selecting the next_tool_name and its next_tool_args, the tool must be one of:\n",
+ " \n",
+ " (1) fetch_flight_info, whose description is <desc>Fetch flight information from origin to destination on the given date</desc>. It takes arguments {'date': {'properties': {'year': {'title': 'Year', 'type': 'integer'}, 'month': {'title': 'Month', 'type': 'integer'}, 'day': {'title': 'Day', 'type': 'integer'}, 'hour': {'title': 'Hour', 'type': 'integer'}}, 'required': ['year', 'month', 'day', 'hour'], 'title': 'Date', 'type': 'object'}, 'origin': {'type': 'string'}, 'destination': {'type': 'string'}} in JSON format.\n",
+ " (2) fetch_itinerary, whose description is <desc>Fetch a booked itinerary information from database</desc>. It takes arguments {'confirmation_number': {'type': 'string'}} in JSON format.\n",
+ " (3) pick_flight, whose description is <desc>Pick up the best flight that matches users' request. we pick the shortest, and cheaper one on ties.</desc>. It takes arguments {'flights': {'$defs': {'Date': {'properties': {'year': {'title': 'Year', 'type': 'integer'}, 'month': {'title': 'Month', 'type': 'integer'}, 'day': {'title': 'Day', 'type': 'integer'}, 'hour': {'title': 'Hour', 'type': 'integer'}}, 'required': ['year', 'month', 'day', 'hour'], 'title': 'Date', 'type': 'object'}, 'Flight': {'properties': {'flight_id': {'title': 'Flight Id', 'type': 'string'}, 'date_time': {'$ref': '#/$defs/Date'}, 'origin': {'title': 'Origin', 'type': 'string'}, 'destination': {'title': 'Destination', 'type': 'string'}, 'duration': {'title': 'Duration', 'type': 'number'}, 'price': {'title': 'Price', 'type': 'number'}}, 'required': ['flight_id', 'date_time', 'origin', 'destination', 'duration', 'price'], 'title': 'Flight', 'type': 'object'}}, 'items': {'$ref': '#/$defs/Flight'}, 'type': 'array'}} in JSON format.\n",
+ " (4) book_flight, whose description is <desc>Book a flight on behalf of the user.</desc>. It takes arguments {'flight': {'properties': {'flight_id': {'title': 'Flight Id', 'type': 'string'}, 'date_time': {'properties': {'year': {'title': 'Year', 'type': 'integer'}, 'month': {'title': 'Month', 'type': 'integer'}, 'day': {'title': 'Day', 'type': 'integer'}, 'hour': {'title': 'Hour', 'type': 'integer'}}, 'required': ['year', 'month', 'day', 'hour'], 'title': 'Date', 'type': 'object'}, 'origin': {'title': 'Origin', 'type': 'string'}, 'destination': {'title': 'Destination', 'type': 'string'}, 'duration': {'title': 'Duration', 'type': 'number'}, 'price': {'title': 'Price', 'type': 'number'}}, 'required': ['flight_id', 'date_time', 'origin', 'destination', 'duration', 'price'], 'title': 'Flight', 'type': 'object'}, 'user_profile': {'properties': {'user_id': {'title': 'User Id', 'type': 'string'}, 'name': {'title': 'Name', 'type': 'string'}, 'email': {'title': 'Email', 'type': 'string'}}, 'required': ['user_id', 'name', 'email'], 'title': 'UserProfile', 'type': 'object'}} in JSON format.\n",
+ " (5) cancel_itinerary, whose description is <desc>Cancel an itinerary on behalf of the user.</desc>. It takes arguments {'confirmation_number': {'type': 'string'}, 'user_profile': {'properties': {'user_id': {'title': 'User Id', 'type': 'string'}, 'name': {'title': 'Name', 'type': 'string'}, 'email': {'title': 'Email', 'type': 'string'}}, 'required': ['user_id', 'name', 'email'], 'title': 'UserProfile', 'type': 'object'}} in JSON format.\n",
+ " (6) get_user_info, whose description is <desc>Fetch the user profile from database with given name.</desc>. It takes arguments {'name': {'type': 'string'}} in JSON format.\n",
+ " (7) file_ticket, whose description is <desc>File a customer support ticket if this is something the agent cannot handle.</desc>. It takes arguments {'user_request': {'type': 'string'}, 'user_profile': {'properties': {'user_id': {'title': 'User Id', 'type': 'string'}, 'name': {'title': 'Name', 'type': 'string'}, 'email': {'title': 'Email', 'type': 'string'}}, 'required': ['user_id', 'name', 'email'], 'title': 'UserProfile', 'type': 'object'}} in JSON format.\n",
+ " (8) finish, whose description is <desc>Marks the task as complete. That is, signals that all information for producing the outputs, i.e. `process_result`, are now available to be extracted.</desc>. It takes arguments {} in JSON format.\n",
+ "\n",
+ "\n",
+ "\u001b[31mUser message:\u001b[0m\n",
+ "\n",
+ "[[ ## user_request ## ]]\n",
+ "please help me book a flight from SFO to JFK on 09/01/2025, my name is Adam\n",
+ "\n",
+ "[[ ## trajectory ## ]]\n",
+ "[[ ## thought_0 ## ]]\n",
+ "I need to fetch flight information for the route from SFO to JFK on the specified date of 09/01/2025. Once I have the flight options, I can help Adam pick the best flight and proceed with the booking.\n",
+ "\n",
+ "[[ ## tool_name_0 ## ]]\n",
+ "fetch_flight_info\n",
+ "\n",
+ "[[ ## tool_args_0 ## ]]\n",
+ "{\"date\": {\"year\": 2025, \"month\": 9, \"day\": 1, \"hour\": 0}, \"origin\": \"SFO\", \"destination\": \"JFK\"}\n",
+ "\n",
+ "[[ ## observation_0 ## ]]\n",
+ "[1] «flight_id='DA123' date_time=Date(year=2025, month=9, day=1, hour=1) origin='SFO' destination='JFK' duration=3.0 price=200.0»\n",
+ "[2] «flight_id='DA125' date_time=Date(year=2025, month=9, day=1, hour=7) origin='SFO' destination='JFK' duration=9.0 price=500.0»\n",
+ "\n",
+ "[[ ## thought_1 ## ]]\n",
+ "I have received the flight options from SFO to JFK on 09/01/2025. There are two flights available: DA123 at 1 AM for $200 and DA125 at 7 AM for $500. I will now pick the best flight, which is the cheaper option, DA123.\n",
+ "\n",
+ "[[ ## tool_name_1 ## ]]\n",
+ "pick_flight\n",
+ "\n",
+ "[[ ## tool_args_1 ## ]]\n",
+ "{\"flights\": [{\"flight_id\": \"DA123\", \"date_time\": {\"year\": 2025, \"month\": 9, \"day\": 1, \"hour\": 1}, \"origin\": \"SFO\", \"destination\": \"JFK\", \"duration\": 3.0, \"price\": 200.0}, {\"flight_id\": \"DA125\", \"date_time\": {\"year\": 2025, \"month\": 9, \"day\": 1, \"hour\": 7}, \"origin\": \"SFO\", \"destination\": \"JFK\", \"duration\": 9.0, \"price\": 500.0}]}\n",
+ "\n",
+ "[[ ## observation_1 ## ]]\n",
+ "{\"flight_id\": \"DA123\", \"date_time\": {\"year\": 2025, \"month\": 9, \"day\": 1, \"hour\": 1}, \"origin\": \"SFO\", \"destination\": \"JFK\", \"duration\": 3.0, \"price\": 200.0}\n",
+ "\n",
+ "[[ ## thought_2 ## ]]\n",
+ "I have successfully picked the best flight, which is DA123. Now, I need to gather Adam's user profile information to proceed with the booking.\n",
+ "\n",
+ "[[ ## tool_name_2 ## ]]\n",
+ "get_user_info\n",
+ "\n",
+ "[[ ## tool_args_2 ## ]]\n",
+ "{\"name\": \"Adam\"}\n",
+ "\n",
+ "[[ ## observation_2 ## ]]\n",
+ "{\"user_id\": \"1\", \"name\": \"Adam\", \"email\": \"[email protected]\"}\n",
+ "\n",
+ "[[ ## thought_3 ## ]]\n",
+ "I have gathered Adam's user profile information. Now I can proceed to book the selected flight DA123 for him.\n",
+ "\n",
+ "[[ ## tool_name_3 ## ]]\n",
+ "book_flight\n",
+ "\n",
+ "[[ ## tool_args_3 ## ]]\n",
+ "{\"flight\": {\"flight_id\": \"DA123\", \"date_time\": {\"year\": 2025, \"month\": 9, \"day\": 1, \"hour\": 1}, \"origin\": \"SFO\", \"destination\": \"JFK\", \"duration\": 3.0, \"price\": 200.0}, \"user_profile\": {\"user_id\": \"1\", \"name\": \"Adam\", \"email\": \"[email protected]\"}}\n",
+ "\n",
+ "[[ ## observation_3 ## ]]\n",
+ "[\"94ldyhsx\", {\"confirmation_number\": \"94ldyhsx\", \"user_profile\": {\"user_id\": \"1\", \"name\": \"Adam\", \"email\": \"[email protected]\"}, \"flight\": {\"flight_id\": \"DA123\", \"date_time\": {\"year\": 2025, \"month\": 9, \"day\": 1, \"hour\": 1}, \"origin\": \"SFO\", \"destination\": \"JFK\", \"duration\": 3.0, \"price\": 200.0}}]\n",
+ "\n",
+ "Respond with the corresponding output fields, starting with the field `[[ ## next_thought ## ]]`, then `[[ ## next_tool_name ## ]]` (must be formatted as a valid Python Literal['fetch_flight_info', 'fetch_itinerary', 'pick_flight', 'book_flight', 'cancel_itinerary', 'get_user_info', 'file_ticket', 'finish']), then `[[ ## next_tool_args ## ]]` (must be formatted as a valid Python dict[str, Any]), and then ending with the marker for `[[ ## completed ## ]]`.\n",
+ "\n",
+ "\n",
+ "\u001b[31mResponse:\u001b[0m\n",
+ "\n",
+ "\u001b[32m[[ ## next_thought ## ]]\n",
+ "Adam's flight has been successfully booked, and I have received the confirmation number. I will now mark the task as complete.\n",
+ "\n",
+ "[[ ## next_tool_name ## ]]\n",
+ "finish\n",
+ "\n",
+ "[[ ## next_tool_args ## ]]\n",
+ "{}\n",
+ "\n",
+ "[[ ## completed ## ]]\u001b[0m\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\u001b[34m[2025-05-28T01:06:51.599439]\u001b[0m\n",
+ "\n",
+ "\u001b[31mSystem message:\u001b[0m\n",
+ "\n",
+ "Your input fields are:\n",
+ "1. `user_request` (str)\n",
+ "2. `trajectory` (str)\n",
+ "Your output fields are:\n",
+ "1. `reasoning` (str)\n",
+ "2. `process_result` (str): Message that summarizes the process result, and the information users need, e.g., the confirmation_number if a new flight is booked.\n",
+ "All interactions will be structured in the following way, with the appropriate values filled in.\n",
+ "\n",
+ "[[ ## user_request ## ]]\n",
+ "{user_request}\n",
+ "\n",
+ "[[ ## trajectory ## ]]\n",
+ "{trajectory}\n",
+ "\n",
+ "[[ ## reasoning ## ]]\n",
+ "{reasoning}\n",
+ "\n",
+ "[[ ## process_result ## ]]\n",
+ "{process_result}\n",
+ "\n",
+ "[[ ## completed ## ]]\n",
+ "In adhering to this structure, your objective is: \n",
+ " You are an airline customer service agent that helps user book and manage flights. \n",
+ " \n",
+ " You are given a list of tools to handle user request, and you should decide the right tool to use in order to\n",
+ " fullfil users' request.\n",
+ "\n",
+ "\n",
+ "\u001b[31mUser message:\u001b[0m\n",
+ "\n",
+ "[[ ## user_request ## ]]\n",
+ "please help me book a flight from SFO to JFK on 09/01/2025, my name is Adam\n",
+ "\n",
+ "[[ ## trajectory ## ]]\n",
+ "[[ ## thought_0 ## ]]\n",
+ "I need to fetch flight information for the route from SFO to JFK on the specified date of 09/01/2025. Once I have the flight options, I can help Adam pick the best flight and proceed with the booking.\n",
+ "\n",
+ "[[ ## tool_name_0 ## ]]\n",
+ "fetch_flight_info\n",
+ "\n",
+ "[[ ## tool_args_0 ## ]]\n",
+ "{\"date\": {\"year\": 2025, \"month\": 9, \"day\": 1, \"hour\": 0}, \"origin\": \"SFO\", \"destination\": \"JFK\"}\n",
+ "\n",
+ "[[ ## observation_0 ## ]]\n",
+ "[1] «flight_id='DA123' date_time=Date(year=2025, month=9, day=1, hour=1) origin='SFO' destination='JFK' duration=3.0 price=200.0»\n",
+ "[2] «flight_id='DA125' date_time=Date(year=2025, month=9, day=1, hour=7) origin='SFO' destination='JFK' duration=9.0 price=500.0»\n",
+ "\n",
+ "[[ ## thought_1 ## ]]\n",
+ "I have received the flight options from SFO to JFK on 09/01/2025. There are two flights available: DA123 at 1 AM for $200 and DA125 at 7 AM for $500. I will now pick the best flight, which is the cheaper option, DA123.\n",
+ "\n",
+ "[[ ## tool_name_1 ## ]]\n",
+ "pick_flight\n",
+ "\n",
+ "[[ ## tool_args_1 ## ]]\n",
+ "{\"flights\": [{\"flight_id\": \"DA123\", \"date_time\": {\"year\": 2025, \"month\": 9, \"day\": 1, \"hour\": 1}, \"origin\": \"SFO\", \"destination\": \"JFK\", \"duration\": 3.0, \"price\": 200.0}, {\"flight_id\": \"DA125\", \"date_time\": {\"year\": 2025, \"month\": 9, \"day\": 1, \"hour\": 7}, \"origin\": \"SFO\", \"destination\": \"JFK\", \"duration\": 9.0, \"price\": 500.0}]}\n",
+ "\n",
+ "[[ ## observation_1 ## ]]\n",
+ "{\"flight_id\": \"DA123\", \"date_time\": {\"year\": 2025, \"month\": 9, \"day\": 1, \"hour\": 1}, \"origin\": \"SFO\", \"destination\": \"JFK\", \"duration\": 3.0, \"price\": 200.0}\n",
+ "\n",
+ "[[ ## thought_2 ## ]]\n",
+ "I have successfully picked the best flight, which is DA123. Now, I need to gather Adam's user profile information to proceed with the booking.\n",
+ "\n",
+ "[[ ## tool_name_2 ## ]]\n",
+ "get_user_info\n",
+ "\n",
+ "[[ ## tool_args_2 ## ]]\n",
+ "{\"name\": \"Adam\"}\n",
+ "\n",
+ "[[ ## observation_2 ## ]]\n",
+ "{\"user_id\": \"1\", \"name\": \"Adam\", \"email\": \"[email protected]\"}\n",
+ "\n",
+ "[[ ## thought_3 ## ]]\n",
+ "I have gathered Adam's user profile information. Now I can proceed to book the selected flight DA123 for him.\n",
+ "\n",
+ "[[ ## tool_name_3 ## ]]\n",
+ "book_flight\n",
+ "\n",
+ "[[ ## tool_args_3 ## ]]\n",
+ "{\"flight\": {\"flight_id\": \"DA123\", \"date_time\": {\"year\": 2025, \"month\": 9, \"day\": 1, \"hour\": 1}, \"origin\": \"SFO\", \"destination\": \"JFK\", \"duration\": 3.0, \"price\": 200.0}, \"user_profile\": {\"user_id\": \"1\", \"name\": \"Adam\", \"email\": \"[email protected]\"}}\n",
+ "\n",
+ "[[ ## observation_3 ## ]]\n",
+ "[\"94ldyhsx\", {\"confirmation_number\": \"94ldyhsx\", \"user_profile\": {\"user_id\": \"1\", \"name\": \"Adam\", \"email\": \"[email protected]\"}, \"flight\": {\"flight_id\": \"DA123\", \"date_time\": {\"year\": 2025, \"month\": 9, \"day\": 1, \"hour\": 1}, \"origin\": \"SFO\", \"destination\": \"JFK\", \"duration\": 3.0, \"price\": 200.0}}]\n",
+ "\n",
+ "[[ ## thought_4 ## ]]\n",
+ "Adam's flight has been successfully booked, and I have received the confirmation number. I will now mark the task as complete.\n",
+ "\n",
+ "[[ ## tool_name_4 ## ]]\n",
+ "finish\n",
+ "\n",
+ "[[ ## tool_args_4 ## ]]\n",
+ "{}\n",
+ "\n",
+ "[[ ## observation_4 ## ]]\n",
+ "Completed.\n",
+ "\n",
+ "Respond with the corresponding output fields, starting with the field `[[ ## reasoning ## ]]`, then `[[ ## process_result ## ]]`, and then ending with the marker for `[[ ## completed ## ]]`.\n",
+ "\n",
+ "\n",
+ "\u001b[31mResponse:\u001b[0m\n",
+ "\n",
+ "\u001b[32m[[ ## reasoning ## ]]\n",
+ "I successfully fetched flight information for the route from SFO to JFK on 09/01/2025. There were two available flights: DA123 at 1 AM for $200 and DA125 at 7 AM for $500. I selected the cheaper option, DA123, and then retrieved Adam's user profile information to proceed with the booking. After booking the flight, I received a confirmation number for the reservation.\n",
+ "\n",
+ "[[ ## process_result ## ]]\n",
+ "Your flight from SFO to JFK on 09/01/2025 has been successfully booked. Your confirmation number is 94ldyhsx.\n",
+ "\n",
+ "[[ ## completed ## ]]\u001b[0m\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "dspy.inspect_history(n=10)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "TXu6hnIDZLjn"
+ },
+ "source": [
+ "We can see that in each LM call, the user message includes the information of previous tool calls, along with the task description."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "qhMU7V0SZcU1"
+ },
+ "source": [
+ "Let's try a different task."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "iN2XoVmGYs1e",
+ "outputId": "93df2938-3560-4976-d6cc-a64e911d9030"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Prediction(\n",
+ " trajectory={'thought_0': \"I need to fetch the user's current itinerary using the confirmation number provided (7zokt5v5) to understand the details of their existing booking before making any modifications.\", 'tool_name_0': 'fetch_itinerary', 'tool_args_0': {'confirmation_number': '7zokt5v5'}, 'observation_0': None, 'thought_1': \"I need to fetch the user's current itinerary using the confirmation number provided (7zokt5v5) to understand the details of their existing booking before making any modifications.\", 'tool_name_1': 'fetch_itinerary', 'tool_args_1': {'confirmation_number': '7zokt5v5'}, 'observation_1': None, 'thought_2': \"I need to file a customer support ticket since I am unable to fetch the user's itinerary with the provided confirmation number.\", 'tool_name_2': 'file_ticket', 'tool_args_2': {'user_request': 'i want to take DA125 instead on 09/01, please help me modify my itinerary 7zokt5v5', 'user_profile': {'user_id': '', 'name': '', 'email': ''}}, 'observation_2': 'lf3n2t', 'thought_3': \"Since I have filed a customer support ticket for the user's request, I will wait for a response from the support team regarding the modification of the itinerary.\", 'tool_name_3': 'finish', 'tool_args_3': {}, 'observation_3': 'Completed.'},\n",
+ " reasoning=\"I attempted to modify the user's itinerary by fetching the current details using the provided confirmation number (7zokt5v5). However, I was unable to retrieve the itinerary. As a result, I filed a customer support ticket to address the issue and request the modification to the user's itinerary for flight DA125 on 09/01.\",\n",
+ " process_result='A customer support ticket has been filed regarding your request to modify your itinerary. The ticket number is lf3n2t. Please wait for a response from the support team for further assistance.'\n",
+ ")\n"
+ ]
+ }
+ ],
+ "source": [
+ "confirmation_number = \"{copy the confirmation number here}\"\n",
+ "\n",
+ "result = agent(user_request=f\"i want to take DA125 instead on 09/01, please help me modify my itinerary {confirmation_number}\")\n",
+ "print(result)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "vkxpCJ6OZXQK"
+ },
+ "source": [
+ "## Conclusion\n",
+ "\n",
+ "Congrats on finishing the tutorial! In this tutorial we have seen how to build a customer service agent with DSPy. The gists are:\n",
+ "\n",
+ "- Define the tools as python function, and **add docstring and type hints**.\n",
+ "- Provide the tools to `dspy.ReAct` along with a signature to define the task.\n",
+ "- Invoke the `dspy.ReAct` with the inputs field defined in the signature, and it will start the reasoning and acting loop behind the scene."
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3",
+ "name": "python3"
+ },
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
diff --git a/docs/docs/tutorials/index.md b/docs/docs/tutorials/index.md
index 95f90d504..af1d0384d 100644
--- a/docs/docs/tutorials/index.md
+++ b/docs/docs/tutorials/index.md
@@ -15,6 +15,7 @@ Welcome to DSPy tutorials! We've organized our tutorials into three main categor
- Build AI Programs with DSPy
- [Retrieval-Augmented Generation (RAG)](/tutorials/rag/)
- [Building RAG as Agent](/tutorials/agents/)
+ - [Build AI Agents with DSPy](/tutorials/customer_service_agent/)
- [Entity Extraction](/tutorials/entity_extraction/)
- [Classification](/tutorials/classification/)
- [Multi-Hop RAG](/tutorials/multihop_search/)
diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml
index 6e9a7e15f..2db00f2d9 100644
--- a/docs/mkdocs.yml
+++ b/docs/mkdocs.yml
@@ -28,6 +28,7 @@ nav:
- Build AI Programs with DSPy:
- Overview: tutorials/build_ai_program/index.md
- Retrieval-Augmented Generation (RAG): tutorials/rag/index.ipynb
+ - Build AI Agents with DSPy: tutorials/customer_service_agent/index.ipynb
- Building RAG as Agent: tutorials/agents/index.ipynb
- Entity Extraction: tutorials/entity_extraction/index.ipynb
- Classification: tutorials/classification/index.md