Skip to content

A programming framework for agentic AI

update ollama usage docs (#5854)

Eric Zhu907dad46b2d0
diff --git a/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/models.ipynb b/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/models.ipynb
index cd8427839..5cde8edd6 100644
--- a/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/models.ipynb
+++ b/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/models.ipynb
@@ -232,49 +232,52 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## Ollama\n",
+ "## Ollama (experimental)\n",
"\n",
"[Ollama](https://ollama.com/) is a local model server that can run models locally on your machine.\n",
"\n",
- "Currently, we recommend using the {py:class}`~autogen_ext.models.openai.OpenAIChatCompletionClient`\n",
- "to interact with Ollama server.\n",
- "\n",
"```{note}\n",
"Small local models are typically not as capable as larger models on the cloud.\n",
"For some tasks they may not perform as well and the output may be suprising.\n",
- "```"
+ "```\n",
+ "\n",
+ "To use Ollama, install the `ollama` extension and use the {py:class}`~autogen_ext.models.ollama.OllamaChatCompletionClient`."
]
},
{
"cell_type": "code",
- "execution_count": 2,
+ "execution_count": null,
+ "metadata": {
+ "vscode": {
+ "languageId": "shellscript"
+ }
+ },
+ "outputs": [],
+ "source": [
+ "pip install -U \"autogen-ext[ollama]\""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
- "finish_reason='unknown' content='The capital of France is Paris.' usage=RequestUsage(prompt_tokens=32, completion_tokens=8) cached=False logprobs=None\n"
+ "finish_reason='unknown' content='The capital of France is Paris.' usage=RequestUsage(prompt_tokens=32, completion_tokens=8) cached=False logprobs=None thought=None\n"
]
}
],
"source": [
"from autogen_core.models import UserMessage\n",
- "from autogen_ext.models.openai import OpenAIChatCompletionClient\n",
+ "from autogen_ext.models.ollama import OllamaChatCompletionClient\n",
"\n",
- "model_client = OpenAIChatCompletionClient(\n",
- " model=\"llama3.2:latest\",\n",
- " base_url=\"http://localhost:11434/v1\",\n",
- " api_key=\"placeholder\",\n",
- " model_info={\n",
- " \"vision\": False,\n",
- " \"function_calling\": True,\n",
- " \"json_output\": False,\n",
- " \"family\": \"unknown\",\n",
- " },\n",
- ")\n",
+ "# Assuming your Ollama server is running locally on port 11434.\n",
+ "ollama_model_client = OllamaChatCompletionClient(model=\"llama3.2\")\n",
"\n",
- "response = await model_client.create([UserMessage(content=\"What is the capital of France?\", source=\"user\")])\n",
+ "response = await ollama_model_client.create([UserMessage(content=\"What is the capital of France?\", source=\"user\")])\n",
"print(response)"
]
},
diff --git a/python/packages/autogen-core/docs/src/user-guide/core-user-guide/components/model-clients.ipynb b/python/packages/autogen-core/docs/src/user-guide/core-user-guide/components/model-clients.ipynb
index 0e4c4b625..20a033f76 100644
--- a/python/packages/autogen-core/docs/src/user-guide/core-user-guide/components/model-clients.ipynb
+++ b/python/packages/autogen-core/docs/src/user-guide/core-user-guide/components/model-clients.ipynb
@@ -253,42 +253,52 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## Ollama\n",
+ "## Ollama (experimental)\n",
"\n",
- "You can use the {py:class}`~autogen_ext.models.openai.OpenAIChatCompletionClient` to interact with OpenAI-compatible APIs such as Ollama and Gemini (beta).\n",
- "The below example shows how to use a local model running on [Ollama](https://ollama.com) server."
+ "[Ollama](https://ollama.com/) is a local model server that can run models locally on your machine.\n",
+ "\n",
+ "```{note}\n",
+ "Small local models are typically not as capable as larger models on the cloud.\n",
+ "For some tasks they may not perform as well and the output may be suprising.\n",
+ "```\n",
+ "\n",
+ "To use Ollama, install the `ollama` extension and use the {py:class}`~autogen_ext.models.ollama.OllamaChatCompletionClient`."
]
},
{
"cell_type": "code",
- "execution_count": 1,
+ "execution_count": null,
+ "metadata": {
+ "vscode": {
+ "languageId": "shellscript"
+ }
+ },
+ "outputs": [],
+ "source": [
+ "pip install -U \"autogen-ext[ollama]\""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
- "finish_reason='unknown' content='The capital of France is Paris.' usage=RequestUsage(prompt_tokens=32, completion_tokens=8) cached=False logprobs=None\n"
+ "finish_reason='unknown' content='The capital of France is Paris.' usage=RequestUsage(prompt_tokens=32, completion_tokens=8) cached=False logprobs=None thought=None\n"
]
}
],
"source": [
"from autogen_core.models import UserMessage\n",
- "from autogen_ext.models.openai import OpenAIChatCompletionClient\n",
+ "from autogen_ext.models.ollama import OllamaChatCompletionClient\n",
"\n",
- "model_client = OpenAIChatCompletionClient(\n",
- " model=\"llama3.2:latest\",\n",
- " base_url=\"http://localhost:11434/v1\",\n",
- " api_key=\"placeholder\",\n",
- " model_info={\n",
- " \"vision\": False,\n",
- " \"function_calling\": True,\n",
- " \"json_output\": False,\n",
- " \"family\": \"unknown\",\n",
- " },\n",
- ")\n",
+ "# Assuming your Ollama server is running locally on port 11434.\n",
+ "ollama_model_client = OllamaChatCompletionClient(model=\"llama3.2\")\n",
"\n",
- "response = await model_client.create([UserMessage(content=\"What is the capital of France?\", source=\"user\")])\n",
+ "response = await ollama_model_client.create([UserMessage(content=\"What is the capital of France?\", source=\"user\")])\n",
"print(response)"
]
},