Skip to content

A programming framework for agentic AI

docs: Update user guide notebooks to enhance clarity and add structured output (#5224)

Eric Zhu2ceb9dcffed4
diff --git a/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/agents.ipynb b/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/agents.ipynb
index cb846ee20..3a07f68fe 100644
--- a/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/agents.ipynb
+++ b/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/agents.ipynb
@@ -35,9 +35,15 @@
"from autogen_agentchat.messages import TextMessage\n",
"from autogen_agentchat.ui import Console\n",
"from autogen_core import CancellationToken\n",
- "from autogen_ext.models.openai import OpenAIChatCompletionClient\n",
- "\n",
- "\n",
+ "from autogen_ext.models.openai import OpenAIChatCompletionClient"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [],
+ "source": [
"# Define a tool that searches the web for information.\n",
"async def web_search(query: str) -> str:\n",
" \"\"\"Find information on the web\"\"\"\n",
@@ -321,6 +327,82 @@
")"
]
},
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Structured Output\n",
+ "\n",
+ "Structured output allows models to return structured JSON text with pre-defined schema\n",
+ "provided by the application. Different from JSON-mode, the schema can be provided\n",
+ "as a [Pydantic BaseModel](https://docs.pydantic.dev/latest/concepts/models/)\n",
+ "class, which can also be used to validate the output. \n",
+ "\n",
+ "```{note}\n",
+ "Structured output is only available for models that support it. It also\n",
+ "requires the model client to support structured output as well.\n",
+ "Currently, the {py:class}`~autogen_ext.models.openai.OpenAIChatCompletionClient`\n",
+ "and {py:class}`~autogen_ext.models.openai.AzureOpenAIChatCompletionClient`\n",
+ "support structured output.\n",
+ "```\n",
+ "\n",
+ "Structured output is also useful for incorporating Chain-of-Thought\n",
+ "reasoning in the agent's responses.\n",
+ "See the example below for how to use structured output with the assistant agent."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "---------- user ----------\n",
+ "I am happy.\n",
+ "---------- assistant ----------\n",
+ "{\"thoughts\":\"The user explicitly states that they are happy.\",\"response\":\"happy\"}\n"
+ ]
+ },
+ {
+ "data": {
+ "text/plain": [
+ "TaskResult(messages=[TextMessage(source='user', models_usage=None, content='I am happy.', type='TextMessage'), TextMessage(source='assistant', models_usage=RequestUsage(prompt_tokens=89, completion_tokens=18), content='{\"thoughts\":\"The user explicitly states that they are happy.\",\"response\":\"happy\"}', type='TextMessage')], stop_reason=None)"
+ ]
+ },
+ "execution_count": 2,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "from typing import Literal\n",
+ "\n",
+ "from pydantic import BaseModel\n",
+ "\n",
+ "\n",
+ "# The response format for the agent as a Pydantic base model.\n",
+ "class AgentResponse(BaseModel):\n",
+ " thoughts: str\n",
+ " response: Literal[\"happy\", \"sad\", \"neutral\"]\n",
+ "\n",
+ "\n",
+ "# Create an agent that uses the OpenAI GPT-4o model with the custom response format.\n",
+ "model_client = OpenAIChatCompletionClient(\n",
+ " model=\"gpt-4o\",\n",
+ " response_format=AgentResponse, # type: ignore\n",
+ ")\n",
+ "agent = AssistantAgent(\n",
+ " \"assistant\",\n",
+ " model_client=model_client,\n",
+ " system_message=\"Categorize the input as happy, sad, or neutral following the JSON format.\",\n",
+ ")\n",
+ "\n",
+ "await Console(agent.run_stream(task=\"I am happy.\"))"
+ ]
+ },
{
"cell_type": "markdown",
"metadata": {},
diff --git a/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/models.ipynb b/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/models.ipynb
index 913da7b54..94a31db3c 100644
--- a/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/models.ipynb
+++ b/python/packages/autogen-core/docs/src/user-guide/agentchat-user-guide/tutorial/models.ipynb
@@ -6,7 +6,10 @@
"source": [
"# Models\n",
"\n",
- "In many cases, agents need access to LLM model services such as OpenAI, Azure OpenAI, or local models. Since there are many different providers with different APIs, `autogen-core` implements a protocol for [model clients](../../core-user-guide/components/model-clients.ipynb) and `autogen-ext` implements a set of model clients for popular model services. AgentChat can use these model clients to interact with model services. \n",
+ "In many cases, agents need access to LLM model services such as OpenAI, Azure OpenAI, or local models. Since there are many different providers with different APIs, `autogen-core` implements a protocol for model clients and `autogen-ext` implements a set of model clients for popular model services. AgentChat can use these model clients to interact with model services. \n",
+ "\n",
+ "This section provides a quick overview of available model clients.\n",
+ "For more details on how to use them directly, please refer to [Model Clients](../../core-user-guide/components/model-clients.ipynb) in the Core API documentation.\n",
"\n",
"```{note}\n",
"See {py:class}`~autogen_ext.models.cache.ChatCompletionCache` for a caching wrapper to use with the following clients.\n",
diff --git a/python/packages/autogen-core/docs/src/user-guide/core-user-guide/components/model-clients.ipynb b/python/packages/autogen-core/docs/src/user-guide/core-user-guide/components/model-clients.ipynb
index c05f6ae11..bb275d2b3 100644
--- a/python/packages/autogen-core/docs/src/user-guide/core-user-guide/components/model-clients.ipynb
+++ b/python/packages/autogen-core/docs/src/user-guide/core-user-guide/components/model-clients.ipynb
@@ -14,17 +14,19 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## Built-in Model Clients\n",
- "\n",
"Currently there are three built-in model clients:\n",
- "* {py:class}`~autogen_ext.models.OpenAIChatCompletionClient`\n",
- "* {py:class}`~autogen_ext.models.AzureOpenAIChatCompletionClient`\n",
- "* {py:class}`~autogen_ext.models.AzureAIChatCompletionClient`\n",
- "\n",
- "\n",
- "### OpenAI\n",
+ "* {py:class}`~autogen_ext.models.openai.OpenAIChatCompletionClient`\n",
+ "* {py:class}`~autogen_ext.models.openai.AzureOpenAIChatCompletionClient`\n",
+ "* {py:class}`~autogen_ext.models.azure.AzureAIChatCompletionClient`"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## OpenAI\n",
"\n",
- "To use the {py:class}`~autogen_ext.models.OpenAIChatCompletionClient`, you need to install the `openai` extra."
+ "To use the {py:class}`~autogen_ext.models.openai.OpenAIChatCompletionClient`, you need to install the `openai` extra."
]
},
{
@@ -68,7 +70,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "You can call the {py:meth}`~autogen_ext.models.OpenAIChatCompletionClient.create` method to create a\n",
+ "You can call the {py:meth}`~autogen_ext.models.openai.BaseOpenAIChatCompletionClient.create` method to create a\n",
"chat completion request, and await for an {py:class}`~autogen_core.models.CreateResult` object in return."
]
},
@@ -118,12 +120,142 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "### OpenAI-Compatible API\n",
+ "## Azure OpenAI\n",
"\n",
- "You can use the {py:class}`~autogen_ext.models.OpenAIChatCompletionClient` to interact with OpenAI-compatible APIs such as Ollama and Gemini (beta).\n",
+ "To use the {py:class}`~autogen_ext.models.openai.AzureOpenAIChatCompletionClient`, you need to provide\n",
+ "the deployment id, Azure Cognitive Services endpoint, api version, and model capabilities.\n",
+ "For authentication, you can either provide an API key or an Azure Active Directory (AAD) token credential."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "vscode": {
+ "languageId": "shellscript"
+ }
+ },
+ "outputs": [],
+ "source": [
+ "# pip install \"autogen-ext[openai,azure]\""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The following code snippet shows how to use AAD authentication.\n",
+ "The identity used must be assigned the [**Cognitive Services OpenAI User**](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/role-based-access-control#cognitive-services-openai-user) role."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from autogen_ext.models.openai import AzureOpenAIChatCompletionClient\n",
+ "from azure.identity import DefaultAzureCredential, get_bearer_token_provider\n",
"\n",
- "#### Ollama\n",
+ "# Create the token provider\n",
+ "token_provider = get_bearer_token_provider(DefaultAzureCredential(), \"https://cognitiveservices.azure.com/.default\")\n",
"\n",
+ "az_model_client = AzureOpenAIChatCompletionClient(\n",
+ " azure_deployment=\"{your-azure-deployment}\",\n",
+ " model=\"{model-name, such as gpt-4o}\",\n",
+ " api_version=\"2024-06-01\",\n",
+ " azure_endpoint=\"https://{your-custom-endpoint}.openai.azure.com/\",\n",
+ " azure_ad_token_provider=token_provider, # Optional if you choose key-based authentication.\n",
+ " # api_key=\"sk-...\", # For key-based authentication.\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "```{note}\n",
+ "See [here](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/managed-identity#chat-completions) for how to use the Azure client directly or for more info.\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Azure AI Foundry\n",
+ "\n",
+ "[Azure AI Foundry](https://learn.microsoft.com/en-us/azure/ai-studio/) (previously known as Azure AI Studio) offers models hosted on Azure.\n",
+ "To use those models, you use the {py:class}`~autogen_ext.models.azure.AzureAIChatCompletionClient`.\n",
+ "\n",
+ "You need to install the `azure` extra to use this client."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "vscode": {
+ "languageId": "shellscript"
+ }
+ },
+ "outputs": [],
+ "source": [
+ "# pip install \"autogen-ext[openai,azure]\""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Below is an example of using this client with the Phi-4 model from [GitHub Marketplace](https://github.com/marketplace/models)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "finish_reason='stop' content='The capital of France is Paris.' usage=RequestUsage(prompt_tokens=14, completion_tokens=8) cached=False logprobs=None\n"
+ ]
+ }
+ ],
+ "source": [
+ "import os\n",
+ "\n",
+ "from autogen_core.models import UserMessage\n",
+ "from autogen_ext.models.azure import AzureAIChatCompletionClient\n",
+ "from azure.core.credentials import AzureKeyCredential\n",
+ "\n",
+ "client = AzureAIChatCompletionClient(\n",
+ " model=\"Phi-4\",\n",
+ " endpoint=\"https://models.inference.ai.azure.com\",\n",
+ " # To authenticate with the model you will need to generate a personal access token (PAT) in your GitHub settings.\n",
+ " # Create your PAT token by following instructions here: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens\n",
+ " credential=AzureKeyCredential(os.environ[\"GITHUB_TOKEN\"]),\n",
+ " model_info={\n",
+ " \"json_output\": False,\n",
+ " \"function_calling\": False,\n",
+ " \"vision\": False,\n",
+ " \"family\": \"unknown\",\n",
+ " },\n",
+ ")\n",
+ "\n",
+ "result = await client.create([UserMessage(content=\"What is the capital of France?\", source=\"user\")])\n",
+ "print(result)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Ollama\n",
+ "\n",
+ "You can use the {py:class}`~autogen_ext.models.openai.OpenAIChatCompletionClient` to interact with OpenAI-compatible APIs such as Ollama and Gemini (beta).\n",
"The below example shows how to use a local model running on [Ollama](https://ollama.com) server."
]
},
@@ -164,9 +296,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "#### Gemini (beta)\n",
+ "## Gemini (beta)\n",
"\n",
- "The below example shows how to use the Gemini model."
+ "The below example shows how to use the Gemini model via the {py:class}`~autogen_ext.models.openai.OpenAIChatCompletionClient`."
]
},
{
@@ -208,9 +340,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "### Streaming Response\n",
+ "## Streaming Response\n",
"\n",
- "You can use the {py:meth}`~autogen_ext.models.OpenAIChatCompletionClient.create_streaming` method to create a\n",
+ "You can use the {py:meth}`~autogen_ext.models.openai.BaseOpenAIChatCompletionClient.create_stream` method to create a\n",
"chat completion request with streaming response."
]
},
@@ -366,133 +498,70 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "### Azure OpenAI\n",
+ "## Structured Output\n",
"\n",
- "To use the {py:class}`~autogen_ext.models.AzureOpenAIChatCompletionClient`, you need to provide\n",
- "the deployment id, Azure Cognitive Services endpoint, api version, and model capabilities.\n",
- "For authentication, you can either provide an API key or an Azure Active Directory (AAD) token credential."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 11,
- "metadata": {
- "vscode": {
- "languageId": "shellscript"
- }
- },
- "outputs": [],
- "source": [
- "# pip install \"autogen-ext[openai,azure]\""
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "The following code snippet shows how to use AAD authentication.\n",
- "The identity used must be assigned the [**Cognitive Services OpenAI User**](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/role-based-access-control#cognitive-services-openai-user) role."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "from autogen_ext.models.openai import AzureOpenAIChatCompletionClient\n",
- "from azure.identity import DefaultAzureCredential, get_bearer_token_provider\n",
- "\n",
- "# Create the token provider\n",
- "token_provider = get_bearer_token_provider(DefaultAzureCredential(), \"https://cognitiveservices.azure.com/.default\")\n",
+ "Structured output can be enabled by setting the `response_format` field in\n",
+ "{py:class}`~autogen_ext.models.openai.OpenAIChatCompletionClient` and {py:class}`~autogen_ext.models.openai.AzureOpenAIChatCompletionClient` to\n",
+ "as a [Pydantic BaseModel](https://docs.pydantic.dev/latest/concepts/models/) class.\n",
"\n",
- "az_model_client = AzureOpenAIChatCompletionClient(\n",
- " azure_deployment=\"{your-azure-deployment}\",\n",
- " model=\"{model-name, such as gpt-4o}\",\n",
- " api_version=\"2024-06-01\",\n",
- " azure_endpoint=\"https://{your-custom-endpoint}.openai.azure.com/\",\n",
- " azure_ad_token_provider=token_provider, # Optional if you choose key-based authentication.\n",
- " # api_key=\"sk-...\", # For key-based authentication.\n",
- ")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
"```{note}\n",
- "See [here](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/managed-identity#chat-completions) for how to use the Azure client directly or for more info.\n",
+ "Structured output is only available for models that support it. It also\n",
+ "requires the model client to support structured output as well.\n",
+ "Currently, the {py:class}`~autogen_ext.models.openai.OpenAIChatCompletionClient`\n",
+ "and {py:class}`~autogen_ext.models.openai.AzureOpenAIChatCompletionClient`\n",
+ "support structured output.\n",
"```"
]
},
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Azure AI Foundry\n",
- "\n",
- "[Azure AI Foundry](https://learn.microsoft.com/en-us/azure/ai-studio/) (previously known as Azure AI Studio) offers models hosted on Azure.\n",
- "To use those models, you use the {py:class}`~autogen_ext.models.azure.AzureAIChatCompletionClient`.\n",
- "\n",
- "You need to install the `azure` extra to use this client."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "vscode": {
- "languageId": "shellscript"
- }
- },
- "outputs": [],
- "source": [
- "# pip install \"autogen-ext[openai,azure]\""
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Below is an example of using this client with the Phi-4 model from [GitHub Marketplace](https://github.com/marketplace/models)."
- ]
- },
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
- "finish_reason='stop' content='The capital of France is Paris.' usage=RequestUsage(prompt_tokens=14, completion_tokens=8) cached=False logprobs=None\n"
+ "I'm glad to hear that you're feeling happy! It's such a great emotion that can brighten your whole day. Is there anything in particular that's bringing you joy today? 😊\n",
+ "happy\n"
]
}
],
"source": [
- "import os\n",
+ "from typing import Literal\n",
"\n",
- "from autogen_core.models import UserMessage\n",
- "from autogen_ext.models.azure import AzureAIChatCompletionClient\n",
- "from azure.core.credentials import AzureKeyCredential\n",
+ "from pydantic import BaseModel\n",
"\n",
- "client = AzureAIChatCompletionClient(\n",
- " model=\"Phi-4\",\n",
- " endpoint=\"https://models.inference.ai.azure.com\",\n",
- " # To authenticate with the model you will need to generate a personal access token (PAT) in your GitHub settings.\n",
- " # Create your PAT token by following instructions here: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens\n",
- " credential=AzureKeyCredential(os.environ[\"GITHUB_TOKEN\"]),\n",
- " model_info={\n",
- " \"json_output\": False,\n",
- " \"function_calling\": False,\n",
- " \"vision\": False,\n",
- " \"family\": \"unknown\",\n",
- " },\n",
+ "\n",
+ "# The response format for the agent as a Pydantic base model.\n",
+ "class AgentResponse(BaseModel):\n",
+ " thoughts: str\n",
+ " response: Literal[\"happy\", \"sad\", \"neutral\"]\n",
+ "\n",
+ "\n",
+ "# Create an agent that uses the OpenAI GPT-4o model with the custom response format.\n",
+ "model_client = OpenAIChatCompletionClient(\n",
+ " model=\"gpt-4o\",\n",
+ " response_format=AgentResponse, # type: ignore\n",
")\n",
"\n",
- "result = await client.create([UserMessage(content=\"What is the capital of France?\", source=\"user\")])\n",
- "print(result)"
+ "# Send a message list to the model and await the response.\n",
+ "messages = [\n",
+ " UserMessage(content=\"I am happy.\", source=\"user\"),\n",
+ "]\n",
+ "response = await model_client.create(messages=messages)\n",
+ "assert isinstance(response.content, str)\n",
+ "parsed_response = AgentResponse.model_validate_json(response.content)\n",
+ "print(parsed_response.thoughts)\n",
+ "print(parsed_response.response)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "You also use the `extra_create_args` parameter in the {py:meth}`~autogen_ext.models.openai.BaseOpenAIChatCompletionClient.create` method\n",
+ "to set the `response_format` field so that the structured output can be configured for each request."
]
},
{