The agent engineering platform.
Python agentsaiai-agentsanthropicchatgptdeepagentsenterpriseframeworkgeminigenerative-ailangchainlanggraphllmmultiagentopen-sourceopenaipydanticpythonragtypescript
docs: ollama doc update (toolcalling, install, notebook examples) (#25549)
Bob Merkus8e3e532e7dd6
diff --git a/docs/docs/integrations/chat/ollama.ipynb b/docs/docs/integrations/chat/ollama.ipynbindex f59474094a..427590d01e 100644--- a/docs/docs/integrations/chat/ollama.ipynb+++ b/docs/docs/integrations/chat/ollama.ipynb@@ -110,7 +110,7 @@},{"cell_type": "code",- "execution_count": 3,+ "execution_count": 9,"id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae","metadata": {},"outputs": [],@@ -118,7 +118,7 @@"from langchain_ollama import ChatOllama\n","\n","llm = ChatOllama(\n",- " model=\"llama3\",\n",+ " model=\"llama3.1\",\n"," temperature=0,\n"," # other params...\n",")"@@ -134,7 +134,7 @@},{"cell_type": "code",- "execution_count": 4,+ "execution_count": 10,"id": "62e0dbc3","metadata": {"tags": []@@ -143,10 +143,10 @@{"data": {"text/plain": [- "AIMessage(content='Je adore le programmation.\\n\\n(Note: \"programmation\" is not commonly used in French, but I translated it as \"le programmation\" to maintain the same grammatical structure and meaning as the original English sentence.)', response_metadata={'model': 'llama3', 'created_at': '2024-07-22T17:43:54.731273Z', 'message': {'role': 'assistant', 'content': ''}, 'done_reason': 'stop', 'done': True, 'total_duration': 11094839375, 'load_duration': 10121854667, 'prompt_eval_count': 36, 'prompt_eval_duration': 146569000, 'eval_count': 46, 'eval_duration': 816593000}, id='run-befccbdc-e1f9-42a9-85cf-e69b926d6b8b-0', usage_metadata={'input_tokens': 36, 'output_tokens': 46, 'total_tokens': 82})"+ "AIMessage(content='The translation of \"I love programming\" from English to French is:\\n\\n\"J\\'adore programmer.\"', response_metadata={'model': 'llama3.1', 'created_at': '2024-08-19T16:05:32.81965Z', 'message': {'role': 'assistant', 'content': ''}, 'done_reason': 'stop', 'done': True, 'total_duration': 2167842917, 'load_duration': 54222584, 'prompt_eval_count': 35, 'prompt_eval_duration': 893007000, 'eval_count': 22, 'eval_duration': 1218962000}, id='run-0863daa2-43bf-4a43-86cc-611b23eae466-0', usage_metadata={'input_tokens': 35, 'output_tokens': 22, 'total_tokens': 57})"]},- "execution_count": 4,+ "execution_count": 10,"metadata": {},"output_type": "execute_result"}@@ -167,7 +167,7 @@},{"cell_type": "code",- "execution_count": 5,+ "execution_count": 11,"id": "d86145b3-bfef-46e8-b227-4dda5c9c2705","metadata": {},"outputs": [@@ -175,9 +175,9 @@"name": "stdout","output_type": "stream","text": [- "Je adore le programmation.\n",+ "The translation of \"I love programming\" from English to French is:\n","\n",- "(Note: \"programmation\" is not commonly used in French, but I translated it as \"le programmation\" to maintain the same grammatical structure and meaning as the original English sentence.)\n"+ "\"J'adore programmer.\"\n"]}],@@ -197,17 +197,17 @@},{"cell_type": "code",- "execution_count": 9,+ "execution_count": 12,"id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b","metadata": {},"outputs": [{"data": {"text/plain": [- "AIMessage(content='Ich liebe Programmieren!\\n\\n(Note: \"Ich liebe\" means \"I love\", \"Programmieren\" is the verb for \"programming\")', response_metadata={'model': 'llama3', 'created_at': '2024-07-04T04:22:33.864132Z', 'message': {'role': 'assistant', 'content': ''}, 'done_reason': 'stop', 'done': True, 'total_duration': 1310800083, 'load_duration': 1782000, 'prompt_eval_count': 16, 'prompt_eval_duration': 250199000, 'eval_count': 29, 'eval_duration': 1057192000}, id='run-cbadbe59-2de2-4ec0-a18a-b3220226c3d2-0')"+ "AIMessage(content='Das Programmieren ist mir ein Leidenschaft! (That\\'s \"Programming is my passion!\" in German.) Would you like me to translate anything else?', response_metadata={'model': 'llama3.1', 'created_at': '2024-08-19T16:05:34.893548Z', 'message': {'role': 'assistant', 'content': ''}, 'done_reason': 'stop', 'done': True, 'total_duration': 2045997333, 'load_duration': 22584792, 'prompt_eval_count': 30, 'prompt_eval_duration': 213210000, 'eval_count': 32, 'eval_duration': 1808541000}, id='run-d18e1c6b-50e0-4b1d-b23a-973fa058edad-0', usage_metadata={'input_tokens': 30, 'output_tokens': 32, 'total_tokens': 62})"]},- "execution_count": 9,+ "execution_count": 12,"metadata": {},"output_type": "execute_result"}@@ -242,33 +242,32 @@"source": ["## Tool calling\n","\n",- "We can use [tool calling](https://blog.langchain.dev/improving-core-tool-interfaces-and-docs-in-langchain/) with an LLM [that has been fine-tuned for tool use](https://ollama.com/library/llama3-groq-tool-use): \n",+ "We can use [tool calling](https://blog.langchain.dev/improving-core-tool-interfaces-and-docs-in-langchain/) with an LLM [that has been fine-tuned for tool use](https://ollama.com/library/llama3.1): \n","\n","```\n",- "ollama pull llama3-groq-tool-use\n",+ "ollama pull llama3.1\n","```\n","\n",- "We can just pass normal Python functions directly as tools."+ "Details on creating custom tools are available in [this guide](/docs/how_to/custom_tools/). Below, we demonstrate how to create a tool using the `@tool` decorator on a normal python function."]},{"cell_type": "code",- "execution_count": 10,- "id": "5250bceb-1029-41ff-b447-983518704d88",+ "execution_count": 13,+ "id": "f767015f","metadata": {},"outputs": [{"data": {"text/plain": ["[{'name': 'validate_user',\n",- " 'args': {'addresses': ['123 Fake St, Boston MA',\n",- " '234 Pretend Boulevard, Houston TX'],\n",- " 'user_id': 123},\n",- " 'id': 'fe2148d3-95fb-48e9-845a-4bfecc1f1f96',\n",+ " 'args': {'addresses': '[\"123 Fake St, Boston, MA\", \"234 Pretend Boulevard, Houston, TX\"]',\n",+ " 'user_id': '123'},\n",+ " 'id': '40fe3de0-500c-4b91-9616-5932a929e640',\n"," 'type': 'tool_call'}]"]},- "execution_count": 10,+ "execution_count": 13,"metadata": {},"output_type": "execute_result"}@@ -276,22 +275,23 @@"source": ["from typing import List\n","\n",+ "from langchain_core.tools import tool\n","from langchain_ollama import ChatOllama\n",- "from typing_extensions import TypedDict\n","\n","\n",- "def validate_user(user_id: int, addresses: List) -> bool:\n",+ "@tool\n",+ "def validate_user(user_id: int, addresses: List[str]) -> bool:\n"," \"\"\"Validate user using historical addresses.\n","\n"," Args:\n",- " user_id: (int) the user ID.\n",- " addresses: Previous addresses.\n",+ " user_id (int): the user ID.\n",+ " addresses (List[str]): Previous addresses as a list of strings.\n"," \"\"\"\n"," return True\n","\n","\n","llm = ChatOllama(\n",- " model=\"llama3-groq-tool-use\",\n",+ " model=\"llama3.1\",\n"," temperature=0,\n",").bind_tools([validate_user])\n","\n",@@ -303,18 +303,6 @@"result.tool_calls"]},- {- "cell_type": "markdown",- "id": "2bb034ff-218f-4865-afea-3f5e57d3bdee",- "metadata": {},- "source": [- "We look at the LangSmith trace to see that the tool call was performed: \n",- "\n",- "https://smith.langchain.com/public/4169348a-d6be-45df-a7cf-032f6baa4697/r\n",- "\n",- "In particular, the trace shows how the tool schema was populated."- ]- },{"cell_type": "markdown","id": "4c5e0197",@@ -331,7 +319,7 @@},{"cell_type": "code",- "execution_count": 11,+ "execution_count": 15,"id": "36c9b1c2","metadata": {},"outputs": [@@ -391,7 +379,7 @@},{"cell_type": "code",- "execution_count": 12,+ "execution_count": 16,"id": "32b3ba7b","metadata": {},"outputs": [@@ -467,7 +455,7 @@"name": "python","nbconvert_exporter": "python","pygments_lexer": "ipython3",- "version": "3.11.8"+ "version": "3.12.4"}},"nbformat": 4,diff --git a/docs/docs/integrations/chat/ollama_functions.ipynb b/docs/docs/integrations/chat/ollama_functions.ipynbdeleted file mode 100644index 96dc9f3f23..0000000000--- a/docs/docs/integrations/chat/ollama_functions.ipynb+++ /dev/null@@ -1,322 +0,0 @@-{- "cells": [- {- "cell_type": "raw",- "metadata": {},- "source": [- "---\n",- "sidebar_label: Ollama Functions\n",- "sidebar_class_name: hidden\n",- "---"- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "# OllamaFunctions\n",- "\n",- ":::warning\n",- "\n",- "This was an experimental wrapper that attempts to bolt-on tool calling support to models that do not natively support it. The [primary Ollama integration](/docs/integrations/chat/ollama/) now supports tool calling, and should be used instead.\n",- "\n",- ":::\n",- "This notebook shows how to use an experimental wrapper around Ollama that gives it [tool calling capabilities](https://python.langchain.com/v0.2/docs/concepts/#functiontool-calling).\n",- "\n",- "Note that more powerful and capable models will perform better with complex schema and/or multiple functions. The examples below use llama3 and phi3 models.\n",- "For a complete list of supported models and model variants, see the [Ollama model library](https://ollama.ai/library).\n",- "\n",- "## Overview\n",- "\n",- "### Integration details\n",- "\n",- "| Class | Package | Local | Serializable | JS support | Package downloads | Package latest |\n",- "|:-----------------------------------------------------------------------------------------------------------------------------------:|:-------:|:-----:|:------------:|:----------:|:-----------------:|:--------------:|\n",- "| [OllamaFunctions](https://api.python.langchain.com/en/latest/llms/langchain_experimental.llms.ollama_function.OllamaFunctions.html) | [langchain-experimental](https://api.python.langchain.com/en/latest/openai_api_reference.html) | ✅ | ❌ | ❌ |  |  |\n",- "\n",- "### Model features\n",- "\n",- "| [Tool calling](/docs/how_to/tool_calling/) | [Structured output](/docs/how_to/structured_output/) | JSON mode | Image input | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n",- "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n",- "| ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ |\n",- "\n",- "## Setup\n",- "\n",- "To access `OllamaFunctions` you will need to install `langchain-experimental` integration package.\n",- "Follow [these instructions](https://github.com/jmorganca/ollama) to set up and run a local Ollama instance as well as download and serve [supported models](https://ollama.com/library).\n",- "\n",- "### Credentials\n",- "\n",- "Credentials support is not present at this time.\n",- "\n",- "### Installation\n",- "\n",- "The `OllamaFunctions` class lives in the `langchain-experimental` package:\n"- ]- },- {- "cell_type": "code",- "execution_count": null,- "metadata": {},- "outputs": [],- "source": [- "%pip install -qU langchain-experimental"- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "## Instantiation\n",- "\n",- "`OllamaFunctions` takes the same init parameters as `ChatOllama`. \n",- "\n",- "In order to use tool calling, you must also specify `format=\"json\"`."- ]- },- {- "cell_type": "code",- "execution_count": 6,- "metadata": {- "ExecuteTime": {- "end_time": "2024-06-23T15:20:21.818089Z",- "start_time": "2024-06-23T15:20:21.815759Z"- }- },- "outputs": [],- "source": [- "from langchain_experimental.llms.ollama_functions import OllamaFunctions\n",- "\n",- "llm = OllamaFunctions(model=\"phi3\")"- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "## Invocation"- ]- },- {- "cell_type": "code",- "execution_count": 3,- "metadata": {- "ExecuteTime": {- "end_time": "2024-06-23T15:20:46.794689Z",- "start_time": "2024-06-23T15:20:44.982632Z"- }- },- "outputs": [- {- "data": {- "text/plain": [- "AIMessage(content=\"J'adore programmer.\", id='run-94815fcf-ae11-438a-ba3f-00819328b5cd-0')"- ]- },- "execution_count": 3,- "metadata": {},- "output_type": "execute_result"- }- ],- "source": [- "messages = [\n",- " (\n",- " \"system\",\n",- " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n",- " ),\n",- " (\"human\", \"I love programming.\"),\n",- "]\n",- "ai_msg = llm.invoke(messages)\n",- "ai_msg"- ]- },- {- "cell_type": "code",- "execution_count": 4,- "metadata": {},- "outputs": [- {- "data": {- "text/plain": [- "\"J'adore programmer.\""- ]- },- "execution_count": 4,- "metadata": {},- "output_type": "execute_result"- }- ],- "source": [- "ai_msg.content"- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "## Chaining\n",- "\n",- "We can [chain](https://python.langchain.com/v0.2/docs/how_to/sequence/) our model with a prompt template like so:"- ]- },- {- "cell_type": "code",- "execution_count": 5,- "metadata": {},- "outputs": [- {- "data": {- "text/plain": [- "AIMessage(content='Programmieren ist sehr verrückt! Es freut mich, dass Sie auf Programmierung so positiv eingestellt sind.', id='run-ee99be5e-4d48-4ab6-b602-35415f0bdbde-0')"- ]- },- "execution_count": 5,- "metadata": {},- "output_type": "execute_result"- }- ],- "source": [- "from langchain_core.prompts import ChatPromptTemplate\n",- "\n",- "prompt = ChatPromptTemplate.from_messages(\n",- " [\n",- " (\n",- " \"system\",\n",- " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n",- " ),\n",- " (\"human\", \"{input}\"),\n",- " ]\n",- ")\n",- "\n",- "chain = prompt | llm\n",- "chain.invoke(\n",- " {\n",- " \"input_language\": \"English\",\n",- " \"output_language\": \"German\",\n",- " \"input\": \"I love programming.\",\n",- " }\n",- ")"- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "## Tool Calling\n",- "\n",- "### OllamaFunctions.bind_tools()\n",- "\n",- "With `OllamaFunctions.bind_tools`, we can easily pass in Pydantic classes, dict schemas, LangChain tools, or even functions as tools to the model. Under the hood these are converted to a tool definition schemas, which looks like:"- ]- },- {- "cell_type": "code",- "execution_count": 7,- "metadata": {},- "outputs": [],- "source": [- "from langchain_core.pydantic_v1 import BaseModel, Field\n",- "\n",- "\n",- "class GetWeather(BaseModel):\n",- " \"\"\"Get the current weather in a given location\"\"\"\n",- "\n",- " location: str = Field(..., description=\"The city and state, e.g. San Francisco, CA\")\n",- "\n",- "\n",- "llm_with_tools = llm.bind_tools([GetWeather])"- ]- },- {- "cell_type": "code",- "execution_count": 8,- "metadata": {},- "outputs": [- {- "data": {- "text/plain": [- "AIMessage(content='', id='run-b9769435-ec6a-4cb8-8545-5a5035fc19bd-0', tool_calls=[{'name': 'GetWeather', 'args': {'location': 'San Francisco, CA'}, 'id': 'call_064c4e1cb27e4adb9e4e7ed60362ecc9'}])"- ]- },- "execution_count": 8,- "metadata": {},- "output_type": "execute_result"- }- ],- "source": [- "ai_msg = llm_with_tools.invoke(\n",- " \"what is the weather like in San Francisco\",\n",- ")\n",- "ai_msg"- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "### AIMessage.tool_calls\n",- "\n",- "Notice that the AIMessage has a `tool_calls` attribute. This contains in a standardized `ToolCall` format that is model-provider agnostic."- ]- },- {- "cell_type": "code",- "execution_count": 10,- "metadata": {},- "outputs": [- {- "data": {- "text/plain": [- "[{'name': 'GetWeather',\n",- " 'args': {'location': 'San Francisco, CA'},\n",- " 'id': 'call_064c4e1cb27e4adb9e4e7ed60362ecc9'}]"- ]- },- "execution_count": 10,- "metadata": {},- "output_type": "execute_result"- }- ],- "source": [- "ai_msg.tool_calls"- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "For more on binding tools and tool call outputs, head to the [tool calling](../../how_to/function_calling.ipynb) docs."- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "## API reference\n",- "\n",- "For detailed documentation of all ToolCallingLLM features and configurations head to the API reference: https://api.python.langchain.com/en/latest/llms/langchain_experimental.llms.ollama_functions.OllamaFunctions.html\n"- ]- }- ],- "metadata": {- "kernelspec": {- "display_name": "Python 3 (ipykernel)",- "language": "python",- "name": "python3"- },- "language_info": {- "codemirror_mode": {- "name": "ipython",- "version": 3- },- "file_extension": ".py",- "mimetype": "text/x-python",- "name": "python",- "nbconvert_exporter": "python",- "pygments_lexer": "ipython3",- "version": "3.10.12"- }- },- "nbformat": 4,- "nbformat_minor": 4-}diff --git a/docs/docs/integrations/llms/ollama.ipynb b/docs/docs/integrations/llms/ollama.ipynbindex 1ab33f6c01..702bd912db 100644--- a/docs/docs/integrations/llms/ollama.ipynb+++ b/docs/docs/integrations/llms/ollama.ipynb@@ -68,7 +68,7 @@},{"cell_type": "code",- "execution_count": 4,+ "execution_count": 2,"id": "035dea0f","metadata": {"tags": []@@ -77,10 +77,10 @@{"data": {"text/plain": [- "'A great start!\\n\\nLangChain is a type of AI model that uses language processing techniques to generate human-like text based on input prompts or chains of reasoning. In other words, it can have a conversation with humans, understanding the context and responding accordingly.\\n\\nHere\\'s a possible breakdown:\\n\\n* \"Lang\" likely refers to its focus on natural language processing (NLP) and linguistic analysis.\\n* \"Chain\" suggests that LangChain is designed to generate text in response to a series of connected ideas or prompts, rather than simply generating random text.\\n\\nSo, what do you think LangChain\\'s capabilities might be?'"+ "\"Sounds like a plan!\\n\\nTo answer what LangChain is, let's break it down step by step.\\n\\n**Step 1: Understand the Context**\\nLangChain seems to be related to language or programming, possibly in an AI context. This makes me wonder if it's a framework, library, or tool for building models or interacting with them.\\n\\n**Step 2: Research Possible Definitions**\\nAfter some quick searching, I found that LangChain is actually a Python library for building and composing conversational AI models. It seems to provide a way to create modular and reusable components for chatbots, voice assistants, and other conversational interfaces.\\n\\n**Step 3: Explore Key Features and Use Cases**\\nLangChain likely offers features such as:\\n\\n* Easy composition of conversational flows\\n* Support for various input/output formats (e.g., text, audio)\\n* Integration with popular AI frameworks and libraries\\n\\nUse cases might include building chatbots for customer service, creating voice assistants for smart homes, or developing interactive stories.\\n\\n**Step 4: Confirm the Definition**\\nAfter this step-by-step analysis, I'm fairly confident that LangChain is a Python library for building conversational AI models. If you'd like to verify or provide more context, feel free to do so!\""]},- "execution_count": 4,+ "execution_count": 2,"metadata": {},"output_type": "execute_result"}@@ -95,7 +95,7 @@"\n","prompt = ChatPromptTemplate.from_template(template)\n","\n",- "model = OllamaLLM(model=\"llama3\")\n",+ "model = OllamaLLM(model=\"llama3.1\")\n","\n","chain = prompt | model\n","\n",@@ -177,7 +177,7 @@},{"cell_type": "code",- "execution_count": 4,+ "execution_count": 3,"id": "79aaf863","metadata": {},"outputs": [@@ -218,7 +218,7 @@"name": "python","nbconvert_exporter": "python","pygments_lexer": "ipython3",- "version": "3.12.3"+ "version": "3.12.4"},"vscode": {"interpreter": {diff --git a/docs/docs/integrations/providers/ollama.mdx b/docs/docs/integrations/providers/ollama.mdxindex 704b02ab15..6a05b5e2be 100644--- a/docs/docs/integrations/providers/ollama.mdx+++ b/docs/docs/integrations/providers/ollama.mdx@@ -1,7 +1,7 @@# Ollama>[Ollama](https://ollama.com/) allows you to run open-source large language models,-> such as LLaMA2, locally.+> such as [Llama3.1](https://ai.meta.com/blog/meta-llama-3-1/), locally.>>`Ollama` bundles model weights, configuration, and data into a single package, defined by a Modelfile.>It optimizes setup and configuration details, including GPU usage.@@ -11,14 +11,36 @@ See [this guide](/docs/how_to/local_llms) for more detailson how to use `Ollama` with LangChain.## Installation and Setup--Follow [these instructions](https://github.com/ollama/ollama?tab=readme-ov-file#ollama)+### Ollama installation+Follow [these instructions](https://github.com/ollama/ollama?tab=readme-ov-file#ollama)to set up and run a local Ollama instance.+Ollama will start as a background service automatically, if this is disabled, run:++```bash+# export OLLAMA_HOST=127.0.0.1 # environment variable to set ollama host+# export OLLAMA_PORT=11434 # environment variable to set the ollama port+ollama serve+```++After starting ollama, run `ollama pull <model_checkpoint>` to download a model+from the [Ollama model library](https://ollama.ai/library).++```bash+ollama pull llama3.1+```++We're now ready to install the `langchain-ollama` partner package and run a model.++### Ollama LangChain partner package install+Install the integration package with:+```bash+pip install langchain-ollama+```## LLM```python-from langchain_community.llms import Ollama+from langchain_ollama.llms import OllamaLLM```See the notebook example [here](/docs/integrations/llms/ollama).@@ -28,18 +50,17 @@ See the notebook example [here](/docs/integrations/llms/ollama).### Chat Ollama```python-from langchain_community.chat_models import ChatOllama+from langchain_ollama.chat_models import ChatOllama```See the notebook example [here](/docs/integrations/chat/ollama).-### Ollama functions--```python-from langchain_experimental.llms.ollama_functions import OllamaFunctions-```--See the notebook example [here](/docs/integrations/chat/ollama_functions).+### Ollama tool calling+[Ollama tool calling](https://ollama.com/blog/tool-support) uses the+OpenAI compatible web server specification, and can be used with+the default `BaseChatModel.bind_tools()` methods+as described [here](/docs/how_to/tool_calling/).+Make sure to select an ollama model that supports [tool calling](https://ollama.com/search?&c=tools).## Embedding modelsdiff --git a/docs/vercel.json b/docs/vercel.jsonindex 0648210037..6afbee9e5e 100644--- a/docs/vercel.json+++ b/docs/vercel.json@@ -101,6 +101,10 @@{"source": "/v0.2/docs/integrations/toolkits/xorbits/","destination": "/v0.2/docs/integrations/tools#search"+ },+ {+ "source": "/v0.2/docs/integrations/chat/ollama_functions/",+ "destination": "https://python.langchain.com/v0.1/docs/integrations/chat/ollama_functions/"}]}