The agent engineering platform.
Python agentsaiai-agentsanthropicchatgptdeepagentsenterpriseframeworkgeminigenerative-ailangchainlanggraphllmmultiagentopen-sourceopenaipydanticpythonragtypescript
feat(docs): clarify structured output with tools ordering (#32527)
Mason Daugherty7932e1edd170
diff --git a/docs/docs/concepts/structured_outputs.mdx b/docs/docs/concepts/structured_outputs.mdxindex d6d31a5d21..2c0d31d632 100644--- a/docs/docs/concepts/structured_outputs.mdx+++ b/docs/docs/concepts/structured_outputs.mdx@@ -29,6 +29,22 @@ model_with_structure = model.with_structured_output(schema)structured_output = model_with_structure.invoke(user_input)```+:::warning[Tool Order Matters]++When combining structured output with additional tools, bind tools **first**, then apply structured output:++```python+# Correct+model_with_tools = model.bind_tools([tool1, tool2])+structured_model = model_with_tools.with_structured_output(schema)++# Incorrect - will cause tool resolution errors+structured_model = model.with_structured_output(schema)+broken_model = structured_model.bind_tools([tool1, tool2])+```++:::+## Schema definitionThe central concept is that the output structure of model responses needs to be represented in some way.diff --git a/docs/docs/how_to/structured_output.ipynb b/docs/docs/how_to/structured_output.ipynbindex ae647d7078..fa2fe54b1f 100644--- a/docs/docs/how_to/structured_output.ipynb+++ b/docs/docs/how_to/structured_output.ipynb@@ -998,6 +998,91 @@"\n","chain.invoke({\"query\": query})"]+ },+ {+ "cell_type": "markdown",+ "id": "xfejabhtn2",+ "metadata": {},+ "source": [+ "## Combining with Additional Tools\n",+ "\n",+ "When you need to use both structured output and additional tools (like web search), note the order of operations:\n",+ "\n",+ "**Correct Order**:\n",+ "```python\n",+ "# 1. Bind tools first\n",+ "llm_with_tools = llm.bind_tools([web_search_tool, calculator_tool])\n",+ "\n",+ "# 2. Apply structured output\n",+ "structured_llm = llm_with_tools.with_structured_output(MySchema)\n",+ "```\n",+ "\n",+ "**Incorrect Order**:\n",+ "\n",+ "```python\n",+ "# This will fail with \"Tool 'MySchema' not found\" error\n",+ "structured_llm = llm.with_structured_output(MySchema)\n",+ "broken_llm = structured_llm.bind_tools([web_search_tool])\n",+ "```"+ ]+ },+ {+ "cell_type": "markdown",+ "id": "653798ca",+ "metadata": {},+ "source": [+ "**Why Order Matters:**\n",+ "`with_structured_output()` internally uses tool calling to enforce the schema. When you bind additional tools afterward, it creates a conflict in the tool resolution system."+ ]+ },+ {+ "cell_type": "markdown",+ "id": "1345f4a4",+ "metadata": {},+ "source": [+ "**Complete Example:**"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": null,+ "id": "0835637b",+ "metadata": {},+ "outputs": [],+ "source": [+ "from pydantic import BaseModel, Field\n",+ "from langchain_openai import ChatOpenAI\n",+ "\n",+ "\n",+ "class SearchResult(BaseModel):\n",+ " \"\"\"Structured search result.\"\"\"\n",+ "\n",+ " query: str = Field(description=\"The search query\")\n",+ " findings: str = Field(description=\"Summary of findings\")\n",+ "\n",+ "\n",+ "# Define tools\n",+ "search_tool = {\n",+ " \"type\": \"function\",\n",+ " \"function\": {\n",+ " \"name\": \"web_search\",\n",+ " \"description\": \"Search the web for information\",\n",+ " \"parameters\": {\n",+ " \"type\": \"object\",\n",+ " \"properties\": {\"query\": {\"type\": \"string\", \"description\": \"Search query\"}},\n",+ " \"required\": [\"query\"],\n",+ " },\n",+ " },\n",+ "}\n",+ "\n",+ "# Correct approach\n",+ "llm = ChatOpenAI()\n",+ "llm_with_search = llm.bind_tools([search_tool])\n",+ "structured_search_llm = llm_with_search.with_structured_output(SearchResult)\n",+ "\n",+ "# Now you can use both search and get structured output\n",+ "result = structured_search_llm.invoke(\"Search for latest AI research and summarize\")"+ ]}],"metadata": {diff --git a/docs/docs/integrations/chat/anthropic.ipynb b/docs/docs/integrations/chat/anthropic.ipynbindex 6e2c028386..bcd6637ce5 100644--- a/docs/docs/integrations/chat/anthropic.ipynb+++ b/docs/docs/integrations/chat/anthropic.ipynb@@ -1240,6 +1240,58 @@"response = llm_with_tools.invoke(\"How do I update a web app to TypeScript 5.5?\")"]},+ {+ "cell_type": "markdown",+ "id": "kloc4rvd1w",+ "metadata": {},+ "source": [+ "#### Web search + structured output\n",+ "\n",+ "When combining web search tools with structured output, it's important to **bind the tools first and then apply structured output**:"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": null,+ "id": "rjjergy6ef",+ "metadata": {},+ "outputs": [],+ "source": [+ "from pydantic import BaseModel, Field\n",+ "from langchain_anthropic import ChatAnthropic\n",+ "\n",+ "\n",+ "# Define structured output schema\n",+ "class ResearchResult(BaseModel):\n",+ " \"\"\"Structured research result from web search.\"\"\"\n",+ "\n",+ " topic: str = Field(description=\"The research topic\")\n",+ " summary: str = Field(description=\"Summary of key findings\")\n",+ " key_points: list[str] = Field(description=\"List of important points discovered\")\n",+ "\n",+ "\n",+ "# Configure web search tool\n",+ "websearch_tools = [\n",+ " {\n",+ " \"type\": \"web_search_20250305\",\n",+ " \"name\": \"web_search\",\n",+ " \"max_uses\": 10,\n",+ " }\n",+ "]\n",+ "\n",+ "llm = ChatAnthropic(model=\"claude-3-5-sonnet-20241022\")\n",+ "\n",+ "# Correct order: bind tools first, then structured output\n",+ "llm_with_search = llm.bind_tools(websearch_tools)\n",+ "research_llm = llm_with_search.with_structured_output(ResearchResult)\n",+ "\n",+ "# Now you can use both web search and get structured output\n",+ "result = research_llm.invoke(\"Research the latest developments in quantum computing\")\n",+ "print(f\"Topic: {result.topic}\")\n",+ "print(f\"Summary: {result.summary}\")\n",+ "print(f\"Key Points: {result.key_points}\")"+ ]+ },{"cell_type": "markdown","id": "1478cdc6-2e52-4870-80f9-b4ddf88f2db2",