DSPy: The framework for programming—not prompting—language models
Python
MCP tutorial (#8149)
Chen Qiana572bfaf95d2
diff --git a/docs/docs/tutorials/mcp/index.md b/docs/docs/tutorials/mcp/index.mdnew file mode 100644index 000000000..c659c7bfc--- /dev/null+++ b/docs/docs/tutorials/mcp/index.md@@ -0,0 +1,453 @@+# Tutorial: Use MCP tools in DSPy++MCP, standing for Model Context Protocol, is an open protocol that standardizes how applications+provide context to LLMs. Despite some development overhead, MCP offers a valuable opportunity to+share tools, resources, and prompts with other developers regardless of the technical stack you are+using. Likewise, you can use the tools built by other developers without rewriting code.++In this guide, we will walk you through how to use MCP tools in DSPy. For demonstration purposes,+we will build an airline service agent that can help users book flights and modify or cancel+existing bookings. This will rely on an MCP server with custom tools, but it should be easy to generalize+to [MCP servers built by the community](https://modelcontextprotocol.io/examples).++??? "How to run this tutorial"+ This tutorial cannot be run in hosted IPython notebooks like Google Colab or Databricks notebooks.+ To run the code, you will need to follow the guide to write code on your local device. The code+ is tested on macOS and should work the same way in Linux environments.++## Install Dependencies++Before starting, let's install the required dependencies:++```shell+pip install -U dspy mcp+```++## MCP Server Setup++Let's first set up the MCP server for the airline agent, which contains:++- A set of databases+ - User database, storing user information.+ - Flight database, storing flight information.+ - Ticket database, storing customer tickets.+- A set of tools+ - fetch_flight_info: get flight information for specific dates.+ - fetch_itinerary: get information about booked itineraries.+ - book_itinerary: book a flight on behalf of the user.+ - modify_itinerary: modify an itinerary, either through flight changes or cancellation.+ - get_user_info: get user information.+ - file_ticket: file a backlog ticket for human assistance.++In your working directory, create a file `mcp_server.py`, and paste the following content into+it:++```python+import random+import string++from mcp.server.fastmcp import FastMCP+from pydantic import BaseModel++# Create an MCP server+mcp = FastMCP("Airline Agent")+++class Date(BaseModel):+ # Somehow LLM is bad at specifying `datetime.datetime`+ year: int+ month: int+ day: int+ hour: int+++class UserProfile(BaseModel):+ user_id: str+ name: str+ email: str+++class Flight(BaseModel):+ flight_id: str+ date_time: Date+ origin: str+ destination: str+ duration: float+ price: float+++class Itinerary(BaseModel):+ confirmation_number: str+ user_profile: UserProfile+ flight: Flight+++class Ticket(BaseModel):+ user_request: str+ user_profile: UserProfile+++user_database = {+ "Adam": UserProfile(user_id="1", name="Adam", email="[email protected]"),+ "Bob": UserProfile(user_id="2", name="Bob", email="[email protected]"),+ "Chelsie": UserProfile(user_id="3", name="Chelsie", email="[email protected]"),+ "David": UserProfile(user_id="4", name="David", email="[email protected]"),+}++flight_database = {+ "DA123": Flight(+ flight_id="DA123",+ origin="SFO",+ destination="JFK",+ date_time=Date(year=2025, month=9, day=1, hour=1),+ duration=3,+ price=200,+ ),+ "DA125": Flight(+ flight_id="DA125",+ origin="SFO",+ destination="JFK",+ date_time=Date(year=2025, month=9, day=1, hour=7),+ duration=9,+ price=500,+ ),+ "DA456": Flight(+ flight_id="DA456",+ origin="SFO",+ destination="SNA",+ date_time=Date(year=2025, month=10, day=1, hour=1),+ duration=2,+ price=100,+ ),+ "DA460": Flight(+ flight_id="DA460",+ origin="SFO",+ destination="SNA",+ date_time=Date(year=2025, month=10, day=1, hour=9),+ duration=2,+ price=120,+ ),+}++itinery_database = {}+ticket_database = {}+++def fetch_flight_info(date: Date, origin: str, destination: str):+ """Fetch flight information from origin to destination on the given date"""+ flights = []++ for flight_id, flight in flight_database.items():+ if (+ flight.date_time.year == date.year+ and flight.date_time.month == date.month+ and flight.date_time.day == date.day+ and flight.origin == origin+ and flight.destination == destination+ ):+ flights.append(flight)+ return flights+++def fetch_itinerary(confirmation_number: str):+ """Fetch a booked itinerary information from database"""+ return itinery_database.get(confirmation_number)+++def pick_flight(flights: list[Flight]):+ """Pick up the best flight that matches users' request."""+ sorted_flights = sorted(+ flights,+ key=lambda x: (+ x.get("duration") if isinstance(x, dict) else x.duration,+ x.get("price") if isinstance(x, dict) else x.price,+ ),+ )+ return sorted_flights[0]+++def generate_id(length=8):+ chars = string.ascii_lowercase + string.digits+ return "".join(random.choices(chars, k=length))+++def book_itinerary(flight: Flight, user_profile: UserProfile):+ """Book a flight on behalf of the user."""+ confirmation_number = generate_id()+ while confirmation_number in itinery_database:+ confirmation_number = generate_id()+ itinery_database[confirmation_number] = Itinerary(+ confirmation_number=confirmation_number,+ user_profile=user_profile,+ flight=flight,+ )+ return confirmation_number, itinery_database[confirmation_number]+++def cancel_itinerary(confirmation_number: str, user_profile: UserProfile):+ """Cancel an itinerary on behalf of the user."""+ if confirmation_number in itinery_database:+ del itinery_database[confirmation_number]+ return+ raise ValueError("Cannot find the itinerary, please check your confirmation number.")+++def get_user_info(name: str):+ """Fetch the user profile from database with given name."""+ return user_database.get(name)+++def file_ticket(user_request: str, user_profile: UserProfile):+ """File a customer support ticket if this is something the agent cannot handle."""+ ticket_id = generate_id(length=6)+ ticket_database[ticket_id] = Ticket(+ user_request=user_request,+ user_profile=user_profile,+ )+ return ticket_id+++if __name__ == "__main__":+ mcp.run()+```++Before we start the server, let's take a look at the code.++We first create a `FastMCP` instance, which is a utility that helps quickly build an MCP server:++```python+mcp = FastMCP("Airline Agent")+```++Then we define our data structures, which in a real-world application would be the database schema, e.g.:++```python+class Flight(BaseModel):+ flight_id: str+ date_time: Date+ origin: str+ destination: str+ duration: float+ price: float+```++Following that, we initialize our database instances. In a real-world application, these would be connectors to+actual databases, but for simplicity, we just use dictionaries:++```python+user_database = {+ "Adam": UserProfile(user_id="1", name="Adam", email="[email protected]"),+ "Bob": UserProfile(user_id="2", name="Bob", email="[email protected]"),+ "Chelsie": UserProfile(user_id="3", name="Chelsie", email="[email protected]"),+ "David": UserProfile(user_id="4", name="David", email="[email protected]"),+}+```++The next step is to define the tools and mark them with `@mcp.tool()` so that they are discoverable by+MCP clients as MCP tools:++```python+def fetch_flight_info(date: Date, origin: str, destination: str):+ """Fetch flight information from origin to destination on the given date"""+ flights = []++ for flight_id, flight in flight_database.items():+ if (+ flight.date_time.year == date.year+ and flight.date_time.month == date.month+ and flight.date_time.day == date.day+ and flight.origin == origin+ and flight.destination == destination+ ):+ flights.append(flight)+ return flights+```++The last step is spinning up the server:++```python+if __name__ == "__main__":+ mcp.run()+```++Now we have finished writing the server! Let's launch it:++```shell+python path_to_your_working_directory/mcp_server.py+```++## Write a DSPy Program That Utilizes Tools in MCP Server++Now that the server is running, let's build the actual airline service agent which+utilizes the MCP tools in our server to assist users. In your working directory,+create a file named `dspy_mcp_agent.py`, and follow the guide to add code to it.++### Gather Tools from MCP Servers++We first need to gather all available tools from the MCP server and make them+usable by DSPy. DSPy provides an API [`dspy.Tool`](https://dspy.ai/api/primitives/Tool/)+as the standard tool interface. Let's convert all the MCP tools to `dspy.Tool`.++We need to create an MCP client instance to communicate with the MCP server, fetch all available+tools, and convert them to `dspy.Tool` using the static method `from_mcp_tool`:++```python+from mcp import ClientSession, StdioServerParameters+from mcp.client.stdio import stdio_client++# Create server parameters for stdio connection+server_params = StdioServerParameters(+ command="python", # Executable+ args=["path_to_your_working_directory/mcp_server.py"],+ env=None,+)++async def run():+ async with stdio_client(server_params) as (read, write):+ async with ClientSession(read, write) as session:+ # Initialize the connection+ await session.initialize()+ # List available tools+ tools = await session.list_tools()++ # Convert MCP tools to DSPy tools+ dspy_tools = []+ for tool in tools.tools:+ dspy_tools.append(dspy.Tool.from_mcp_tool(session, tool))++ print(len(dspy_tools))+ print(dspy_tools[0].args)++if __name__ == "__main__":+ import asyncio++ asyncio.run(run())+```++With the code above, we have successfully collected all available MCP tools and converted+them to DSPy tools.+++### Build a DSPy Agent to Handle Customer Requests++Now we will use `dspy.ReAct` to build the agent for handling customer requests. `ReAct` stands+for "reasoning and acting," which asks the LLM to decide whether to call a tool or wrap up the process.+If a tool is required, the LLM takes responsibility for deciding which tool to call and providing+the appropriate arguments.++As usual, we need to create a `dspy.Signature` to define the input and output of our agent:++```python+import dspy++class DSPyAirlineCustomerService(dspy.Signature):+ """You are an airline customer service agent. You are given a list of tools to handle user requests. You should decide the right tool to use in order to fulfill users' requests."""++ user_request: str = dspy.InputField()+ process_result: str = dspy.OutputField(+ desc=(+ "Message that summarizes the process result, and the information users need, "+ "e.g., the confirmation_number if it's a flight booking request."+ )+ )+```++And choose an LM for our agent:++```python+dspy.configure(lm=dspy.LM("openai/gpt-4o-mini"))+```++Then we create the ReAct agent by passing the tools and signature into the `dspy.ReAct` API. We can now+put together the complete code script:++```python+from mcp import ClientSession, StdioServerParameters+from mcp.client.stdio import stdio_client++import dspy++# Create server parameters for stdio connection+server_params = StdioServerParameters(+ command="python", # Executable+ args=["script_tmp/mcp_server.py"], # Optional command line arguments+ env=None, # Optional environment variables+)+++class DSPyAirlineCustomerService(dspy.Signature):+ """You are an airline customer service agent. You are given a list of tools to handle user requests.+ You should decide the right tool to use in order to fulfill users' requests."""++ user_request: str = dspy.InputField()+ process_result: str = dspy.OutputField(+ desc=(+ "Message that summarizes the process result, and the information users need, "+ "e.g., the confirmation_number if it's a flight booking request."+ )+ )+++dspy.configure(lm=dspy.LM("openai/gpt-4o-mini"))+++async def run(user_request):+ async with stdio_client(server_params) as (read, write):+ async with ClientSession(read, write) as session:+ # Initialize the connection+ await session.initialize()+ # List available tools+ tools = await session.list_tools()++ # Convert MCP tools to DSPy tools+ dspy_tools = []+ for tool in tools.tools:+ dspy_tools.append(dspy.Tool.from_mcp_tool(session, tool))++ # Create the agent+ react = dspy.ReAct(DSPyAirlineCustomerService, tools=dspy_tools)++ result = await react.acall(user_request=user_request)+ print(result)+++if __name__ == "__main__":+ import asyncio++ asyncio.run(run("please help me book a flight from SFO to JFK on 09/01/2025, my name is Adam"))+```++Note that we must call `react.acall` because MCP tools are async by default. Let's execute the script:++```shell+python path_to_your_working_directory/dspy_mcp_agent.py+```++You should see output similar to this:++```+Prediction(+ trajectory={'thought_0': 'I need to fetch flight information for Adam from SFO to JFK on 09/01/2025 to find available flights for booking.', 'tool_name_0': 'fetch_flight_info', 'tool_args_0': {'date': {'year': 2025, 'month': 9, 'day': 1, 'hour': 0}, 'origin': 'SFO', 'destination': 'JFK'}, 'observation_0': ['{"flight_id": "DA123", "date_time": {"year": 2025, "month": 9, "day": 1, "hour": 1}, "origin": "SFO", "destination": "JFK", "duration": 3.0, "price": 200.0}', '{"flight_id": "DA125", "date_time": {"year": 2025, "month": 9, "day": 1, "hour": 7}, "origin": "SFO", "destination": "JFK", "duration": 9.0, "price": 500.0}'], ..., 'tool_name_4': 'finish', 'tool_args_4': {}, 'observation_4': 'Completed.'},+ reasoning="I successfully booked a flight for Adam from SFO to JFK on 09/01/2025. I found two available flights, selected the more economical option (flight DA123 at 1 AM for $200), retrieved Adam's user profile, and completed the booking process. The confirmation number for the flight is 8h7clk3q.",+ process_result='Your flight from SFO to JFK on 09/01/2025 has been successfully booked. Your confirmation number is 8h7clk3q.'+)+```++The `trajectory` field contains the entire thinking and acting process. If you're curious about what's happening+under the hood, check out the [Observability Guide](https://dspy.ai/tutorials/observability/) to set up MLflow,+which visualizes every step happening inside `dspy.ReAct`!+++## Conclusion++In this guide, we built an airline service agent that utilizes a custom MCP server and the `dspy.ReAct` module. In the context+of MCP support, DSPy provides a simple interface for interacting with MCP tools, giving you the flexibility to implement+any functionality you need.diff --git a/docs/mkdocs.yml b/docs/mkdocs.ymlindex 264aa1361..c7e747e5d 100644--- a/docs/mkdocs.yml+++ b/docs/mkdocs.yml@@ -29,6 +29,7 @@ nav:- Agents: tutorials/agents/index.ipynb- Reasoning: tutorials/math/index.ipynb- Entity Extraction: tutorials/entity_extraction/index.ipynb+ - MCP in DSPy: tutorials/mcp/index.md- Classification: tutorials/classification/index.md- Classification Finetuning: tutorials/classification_finetuning/index.ipynb- Multi-Hop Search: tutorials/multihop_search/index.ipynb