A programming framework for agentic AI
update ollama usage docs (#5854)
Eric Zhu907dad46b2d0
diff --git a/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/models.ipynb b/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/models.ipynbindex cd8427839..5cde8edd6 100644--- a/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/models.ipynb+++ b/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/models.ipynb@@ -232,49 +232,52 @@"cell_type": "markdown","metadata": {},"source": [- "## Ollama\n",+ "## Ollama (experimental)\n","\n","[Ollama](https://ollama.com/) is a local model server that can run models locally on your machine.\n","\n",- "Currently, we recommend using the {py:class}`~autogen_ext.models.openai.OpenAIChatCompletionClient`\n",- "to interact with Ollama server.\n",- "\n","```{note}\n","Small local models are typically not as capable as larger models on the cloud.\n","For some tasks they may not perform as well and the output may be suprising.\n",- "```"+ "```\n",+ "\n",+ "To use Ollama, install the `ollama` extension and use the {py:class}`~autogen_ext.models.ollama.OllamaChatCompletionClient`."]},{"cell_type": "code",- "execution_count": 2,+ "execution_count": null,+ "metadata": {+ "vscode": {+ "languageId": "shellscript"+ }+ },+ "outputs": [],+ "source": [+ "pip install -U \"autogen-ext[ollama]\""+ ]+ },+ {+ "cell_type": "code",+ "execution_count": null,"metadata": {},"outputs": [{"name": "stdout","output_type": "stream","text": [- "finish_reason='unknown' content='The capital of France is Paris.' usage=RequestUsage(prompt_tokens=32, completion_tokens=8) cached=False logprobs=None\n"+ "finish_reason='unknown' content='The capital of France is Paris.' usage=RequestUsage(prompt_tokens=32, completion_tokens=8) cached=False logprobs=None thought=None\n"]}],"source": ["from autogen_core.models import UserMessage\n",- "from autogen_ext.models.openai import OpenAIChatCompletionClient\n",+ "from autogen_ext.models.ollama import OllamaChatCompletionClient\n","\n",- "model_client = OpenAIChatCompletionClient(\n",- " model=\"llama3.2:latest\",\n",- " base_url=\"http://localhost:11434/v1\",\n",- " api_key=\"placeholder\",\n",- " model_info={\n",- " \"vision\": False,\n",- " \"function_calling\": True,\n",- " \"json_output\": False,\n",- " \"family\": \"unknown\",\n",- " },\n",- ")\n",+ "# Assuming your Ollama server is running locally on port 11434.\n",+ "ollama_model_client = OllamaChatCompletionClient(model=\"llama3.2\")\n","\n",- "response = await model_client.create([UserMessage(content=\"What is the capital of France?\", source=\"user\")])\n",+ "response = await ollama_model_client.create([UserMessage(content=\"What is the capital of France?\", source=\"user\")])\n","print(response)"]},diff --git a/python/packages/autogen-core/docs/src/user-guide/core-user-guide/components/model-clients.ipynb b/python/packages/autogen-core/docs/src/user-guide/core-user-guide/components/model-clients.ipynbindex 0e4c4b625..20a033f76 100644--- a/python/packages/autogen-core/docs/src/user-guide/core-user-guide/components/model-clients.ipynb+++ b/python/packages/autogen-core/docs/src/user-guide/core-user-guide/components/model-clients.ipynb@@ -253,42 +253,52 @@"cell_type": "markdown","metadata": {},"source": [- "## Ollama\n",+ "## Ollama (experimental)\n","\n",- "You can use the {py:class}`~autogen_ext.models.openai.OpenAIChatCompletionClient` to interact with OpenAI-compatible APIs such as Ollama and Gemini (beta).\n",- "The below example shows how to use a local model running on [Ollama](https://ollama.com) server."+ "[Ollama](https://ollama.com/) is a local model server that can run models locally on your machine.\n",+ "\n",+ "```{note}\n",+ "Small local models are typically not as capable as larger models on the cloud.\n",+ "For some tasks they may not perform as well and the output may be suprising.\n",+ "```\n",+ "\n",+ "To use Ollama, install the `ollama` extension and use the {py:class}`~autogen_ext.models.ollama.OllamaChatCompletionClient`."]},{"cell_type": "code",- "execution_count": 1,+ "execution_count": null,+ "metadata": {+ "vscode": {+ "languageId": "shellscript"+ }+ },+ "outputs": [],+ "source": [+ "pip install -U \"autogen-ext[ollama]\""+ ]+ },+ {+ "cell_type": "code",+ "execution_count": null,"metadata": {},"outputs": [{"name": "stdout","output_type": "stream","text": [- "finish_reason='unknown' content='The capital of France is Paris.' usage=RequestUsage(prompt_tokens=32, completion_tokens=8) cached=False logprobs=None\n"+ "finish_reason='unknown' content='The capital of France is Paris.' usage=RequestUsage(prompt_tokens=32, completion_tokens=8) cached=False logprobs=None thought=None\n"]}],"source": ["from autogen_core.models import UserMessage\n",- "from autogen_ext.models.openai import OpenAIChatCompletionClient\n",+ "from autogen_ext.models.ollama import OllamaChatCompletionClient\n","\n",- "model_client = OpenAIChatCompletionClient(\n",- " model=\"llama3.2:latest\",\n",- " base_url=\"http://localhost:11434/v1\",\n",- " api_key=\"placeholder\",\n",- " model_info={\n",- " \"vision\": False,\n",- " \"function_calling\": True,\n",- " \"json_output\": False,\n",- " \"family\": \"unknown\",\n",- " },\n",- ")\n",+ "# Assuming your Ollama server is running locally on port 11434.\n",+ "ollama_model_client = OllamaChatCompletionClient(model=\"llama3.2\")\n","\n",- "response = await model_client.create([UserMessage(content=\"What is the capital of France?\", source=\"user\")])\n",+ "response = await ollama_model_client.create([UserMessage(content=\"What is the capital of France?\", source=\"user\")])\n","print(response)"]},