Skip to content

The agent engineering platform.

feat(docs): clarify structured output with tools ordering (#32527)

Mason Daugherty7932e1edd170
diff --git a/docs/docs/concepts/structured_outputs.mdx b/docs/docs/concepts/structured_outputs.mdx
index d6d31a5d21..2c0d31d632 100644
--- a/docs/docs/concepts/structured_outputs.mdx
+++ b/docs/docs/concepts/structured_outputs.mdx
@@ -29,6 +29,22 @@ model_with_structure = model.with_structured_output(schema)
structured_output = model_with_structure.invoke(user_input)
```
+:::warning[Tool Order Matters]
+
+When combining structured output with additional tools, bind tools **first**, then apply structured output:
+
+```python
+# Correct
+model_with_tools = model.bind_tools([tool1, tool2])
+structured_model = model_with_tools.with_structured_output(schema)
+
+# Incorrect - will cause tool resolution errors
+structured_model = model.with_structured_output(schema)
+broken_model = structured_model.bind_tools([tool1, tool2])
+```
+
+:::
+
## Schema definition
The central concept is that the output structure of model responses needs to be represented in some way.
diff --git a/docs/docs/how_to/structured_output.ipynb b/docs/docs/how_to/structured_output.ipynb
index ae647d7078..fa2fe54b1f 100644
--- a/docs/docs/how_to/structured_output.ipynb
+++ b/docs/docs/how_to/structured_output.ipynb
@@ -998,6 +998,91 @@
"\n",
"chain.invoke({\"query\": query})"
]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "xfejabhtn2",
+ "metadata": {},
+ "source": [
+ "## Combining with Additional Tools\n",
+ "\n",
+ "When you need to use both structured output and additional tools (like web search), note the order of operations:\n",
+ "\n",
+ "**Correct Order**:\n",
+ "```python\n",
+ "# 1. Bind tools first\n",
+ "llm_with_tools = llm.bind_tools([web_search_tool, calculator_tool])\n",
+ "\n",
+ "# 2. Apply structured output\n",
+ "structured_llm = llm_with_tools.with_structured_output(MySchema)\n",
+ "```\n",
+ "\n",
+ "**Incorrect Order**:\n",
+ "\n",
+ "```python\n",
+ "# This will fail with \"Tool 'MySchema' not found\" error\n",
+ "structured_llm = llm.with_structured_output(MySchema)\n",
+ "broken_llm = structured_llm.bind_tools([web_search_tool])\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "653798ca",
+ "metadata": {},
+ "source": [
+ "**Why Order Matters:**\n",
+ "`with_structured_output()` internally uses tool calling to enforce the schema. When you bind additional tools afterward, it creates a conflict in the tool resolution system."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1345f4a4",
+ "metadata": {},
+ "source": [
+ "**Complete Example:**"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "0835637b",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from pydantic import BaseModel, Field\n",
+ "from langchain_openai import ChatOpenAI\n",
+ "\n",
+ "\n",
+ "class SearchResult(BaseModel):\n",
+ " \"\"\"Structured search result.\"\"\"\n",
+ "\n",
+ " query: str = Field(description=\"The search query\")\n",
+ " findings: str = Field(description=\"Summary of findings\")\n",
+ "\n",
+ "\n",
+ "# Define tools\n",
+ "search_tool = {\n",
+ " \"type\": \"function\",\n",
+ " \"function\": {\n",
+ " \"name\": \"web_search\",\n",
+ " \"description\": \"Search the web for information\",\n",
+ " \"parameters\": {\n",
+ " \"type\": \"object\",\n",
+ " \"properties\": {\"query\": {\"type\": \"string\", \"description\": \"Search query\"}},\n",
+ " \"required\": [\"query\"],\n",
+ " },\n",
+ " },\n",
+ "}\n",
+ "\n",
+ "# Correct approach\n",
+ "llm = ChatOpenAI()\n",
+ "llm_with_search = llm.bind_tools([search_tool])\n",
+ "structured_search_llm = llm_with_search.with_structured_output(SearchResult)\n",
+ "\n",
+ "# Now you can use both search and get structured output\n",
+ "result = structured_search_llm.invoke(\"Search for latest AI research and summarize\")"
+ ]
}
],
"metadata": {
diff --git a/docs/docs/integrations/chat/anthropic.ipynb b/docs/docs/integrations/chat/anthropic.ipynb
index 6e2c028386..bcd6637ce5 100644
--- a/docs/docs/integrations/chat/anthropic.ipynb
+++ b/docs/docs/integrations/chat/anthropic.ipynb
@@ -1240,6 +1240,58 @@
"response = llm_with_tools.invoke(\"How do I update a web app to TypeScript 5.5?\")"
]
},
+ {
+ "cell_type": "markdown",
+ "id": "kloc4rvd1w",
+ "metadata": {},
+ "source": [
+ "#### Web search + structured output\n",
+ "\n",
+ "When combining web search tools with structured output, it's important to **bind the tools first and then apply structured output**:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "rjjergy6ef",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from pydantic import BaseModel, Field\n",
+ "from langchain_anthropic import ChatAnthropic\n",
+ "\n",
+ "\n",
+ "# Define structured output schema\n",
+ "class ResearchResult(BaseModel):\n",
+ " \"\"\"Structured research result from web search.\"\"\"\n",
+ "\n",
+ " topic: str = Field(description=\"The research topic\")\n",
+ " summary: str = Field(description=\"Summary of key findings\")\n",
+ " key_points: list[str] = Field(description=\"List of important points discovered\")\n",
+ "\n",
+ "\n",
+ "# Configure web search tool\n",
+ "websearch_tools = [\n",
+ " {\n",
+ " \"type\": \"web_search_20250305\",\n",
+ " \"name\": \"web_search\",\n",
+ " \"max_uses\": 10,\n",
+ " }\n",
+ "]\n",
+ "\n",
+ "llm = ChatAnthropic(model=\"claude-3-5-sonnet-20241022\")\n",
+ "\n",
+ "# Correct order: bind tools first, then structured output\n",
+ "llm_with_search = llm.bind_tools(websearch_tools)\n",
+ "research_llm = llm_with_search.with_structured_output(ResearchResult)\n",
+ "\n",
+ "# Now you can use both web search and get structured output\n",
+ "result = research_llm.invoke(\"Research the latest developments in quantum computing\")\n",
+ "print(f\"Topic: {result.topic}\")\n",
+ "print(f\"Summary: {result.summary}\")\n",
+ "print(f\"Key Points: {result.key_points}\")"
+ ]
+ },
{
"cell_type": "markdown",
"id": "1478cdc6-2e52-4870-80f9-b4ddf88f2db2",