Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI
Python agentagentopsagents-sdkaianthropicautogencost-estimationcrewaievalsevaluation-metricsgroqlangchainllmmistralollamaopenaiopenai-agents
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# OpenAI Agents Tools Demonstration\n",
"\n",
"This notebook demonstrates various tools available in the Agents SDK and how AgentOps provides observability for tool usage.\n",
"\n",
"## General Flow\n",
"\n",
"This notebook will walk you through several key tools:\n",
"\n",
"1. **Code Interpreter Tool** - Execute Python code and perform mathematical calculations\n",
"2. **File Search Tool** - Search through vector stores and documents\n",
"3. **Image Generation Tool** - Generate images from text descriptions\n",
"4. **Web Search Tool** - Search the web for current information\n",
"\n",
"Each tool demonstrates how AgentOps automatically tracks tool usage, providing insights into performance, costs, and effectiveness."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Prerequisites\n",
"\n",
"Before running this notebook, you'll need:\n",
"\n",
"1. **AgentOps Account**: Create a free account at [app.agentops.ai](https://app.agentops.ai)\n",
"2. **AgentOps API Key**: Obtain your API key from your AgentOps dashboard\n",
"3. **OpenAI API Key**: Get your API key from [platform.openai.com](https://platform.openai.com)\n",
"4. **Vector Store ID**: Configure it from [platform.openai.com](https://platform.openai.com)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Install required packages\n",
"%pip install -q agentops\n",
"%pip install -q openai-agents\n",
"%pip install -q pydotenv"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Set the API keys for your AgentOps and OpenAI accounts.\n",
"import os\n",
"from dotenv import load_dotenv\n",
"\n",
"load_dotenv()\n",
"\n",
"os.environ[\"AGENTOPS_API_KEY\"] = os.getenv(\"AGENTOPS_API_KEY\", \"your_api_key_here\")\n",
"os.environ[\"OPENAI_API_KEY\"] = os.getenv(\"OPENAI_API_KEY\", \"your_openai_api_key_here\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import agentops\n",
"\n",
"agentops.init(auto_start_session=False, tags=[\"agentops-example\"])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import base64\n",
"import os\n",
"import subprocess\n",
"import sys\n",
"import tempfile\n",
"\n",
"from agents import (\n",
" Agent,\n",
" CodeInterpreterTool,\n",
" FileSearchTool,\n",
" ImageGenerationTool,\n",
" Runner,\n",
" WebSearchTool,\n",
" trace,\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1. Code Interpreter Tool\n",
"\n",
"The Code Interpreter Tool allows agents to execute Python code in a secure environment. This is particularly useful for mathematical calculations, data analysis, and generating visualizations.\n",
"\n",
"**Key Features:**\n",
"- Execute Python code safely\n",
"- Perform complex mathematical calculations\n",
"- Generate plots and visualizations\n",
"- Handle data processing tasks"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Start the AgentOps trace session\n",
"tracer = agentops.start_trace(\n",
" trace_name=\"Code Interpreter Tool Example\", tags=[\"tools-demo\", \"openai-agents\", \"agentops-example\"]\n",
")\n",
"\n",
"\n",
"async def run_code_interpreter_demo():\n",
" agent = Agent(\n",
" name=\"Code interpreter\",\n",
" instructions=\"You love doing math.\",\n",
" tools=[\n",
" CodeInterpreterTool(\n",
" tool_config={\"type\": \"code_interpreter\", \"container\": {\"type\": \"auto\"}},\n",
" )\n",
" ],\n",
" )\n",
"\n",
" with trace(\"Code interpreter example\"):\n",
" print(\"Solving math problem...\")\n",
" result = Runner.run_streamed(agent, \"What is the square root of 273 * 312821 plus 1782?\")\n",
" async for event in result.stream_events():\n",
" if (\n",
" event.type == \"run_item_stream_event\"\n",
" and event.item.type == \"tool_call_item\"\n",
" and event.item.raw_item.type == \"code_interpreter_call\"\n",
" ):\n",
" print(f\"Code interpreter code:\\n```\\n{event.item.raw_item.code}\\n```\\n\")\n",
" elif event.type == \"run_item_stream_event\":\n",
" print(f\"Other event: {event.item.type}\")\n",
"\n",
" print(f\"Final output: {result.final_output}\")\n",
"\n",
"\n",
"# Run the demo\n",
"await run_code_interpreter_demo()\n",
"\n",
"# End the AgentOps trace session\n",
"agentops.end_trace(tracer, end_state=\"Success\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. File Search Tool\n",
"\n",
"The File Search Tool allows agents to search through vector stores and document collections to find relevant information.\n",
"\n",
"**Key Features:**\n",
"- Search through vector stores\n",
"- Retrieve relevant documents\n",
"- Support for semantic search\n",
"- Configurable result limits\n",
"\n",
"**Note:** This example requires a pre-configured vector store ID."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Start the AgentOps trace session\n",
"tracer = agentops.start_trace(\n",
" trace_name=\"File Search Tool Example\", tags=[\"tools-demo\", \"openai-agents\", \"agentops-example\"]\n",
")\n",
"\n",
"\n",
"async def run_file_search_demo():\n",
" # Note: You'll need to replace this with your actual vector store ID\n",
" vector_store_id = \"vs_67bf88953f748191be42b462090e53e7\"\n",
"\n",
" agent = Agent(\n",
" name=\"File searcher\",\n",
" instructions=\"You are a helpful agent.\",\n",
" tools=[\n",
" FileSearchTool(\n",
" max_num_results=3,\n",
" vector_store_ids=[vector_store_id],\n",
" include_search_results=True,\n",
" )\n",
" ],\n",
" )\n",
"\n",
" with trace(\"File search example\"):\n",
" try:\n",
" result = await Runner.run(agent, \"Be concise, and tell me 1 sentence about Arrakis I might not know.\")\n",
" print(result.final_output)\n",
" print(\"\\n\".join([str(out) for out in result.new_items]))\n",
" except Exception as e:\n",
" print(f\"File search demo requires a valid vector store ID. Error: {e}\")\n",
"\n",
"\n",
"# Run the demo\n",
"await run_file_search_demo()\n",
"\n",
"# End the AgentOps trace session\n",
"agentops.end_trace(tracer, end_state=\"Success\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3. Image Generation Tool\n",
"\n",
"The Image Generation Tool enables agents to create images from text descriptions using AI image generation models.\n",
"\n",
"**Key Features:**\n",
"- Generate images from text prompts\n",
"- Configurable quality settings\n",
"- Support for various image styles\n",
"- Automatic image saving and display"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Start the AgentOps trace session\n",
"tracer = agentops.start_trace(\n",
" trace_name=\"Image Generation Tool Example\", tags=[\"tools-demo\", \"openai-agents\", \"agentops-example\"]\n",
")\n",
"\n",
"\n",
"def open_file(path: str) -> None:\n",
" if sys.platform.startswith(\"darwin\"):\n",
" subprocess.run([\"open\", path], check=False) # macOS\n",
" elif os.name == \"nt\": # Windows\n",
" os.startfile(path) # type: ignore\n",
" elif os.name == \"posix\":\n",
" subprocess.run([\"xdg-open\", path], check=False) # Linux/Unix\n",
" else:\n",
" print(f\"Don't know how to open files on this platform: {sys.platform}\")\n",
"\n",
"\n",
"async def run_image_generation_demo():\n",
" agent = Agent(\n",
" name=\"Image generator\",\n",
" instructions=\"You are a helpful agent.\",\n",
" tools=[\n",
" ImageGenerationTool(\n",
" tool_config={\"type\": \"image_generation\", \"quality\": \"low\"},\n",
" )\n",
" ],\n",
" )\n",
"\n",
" with trace(\"Image generation example\"):\n",
" print(\"Generating image, this may take a while...\")\n",
" result = await Runner.run(agent, \"Create an image of a frog eating a pizza, comic book style.\")\n",
" print(result.final_output)\n",
" for item in result.new_items:\n",
" if (\n",
" item.type == \"tool_call_item\"\n",
" and item.raw_item.type == \"image_generation_call\"\n",
" and (img_result := item.raw_item.result)\n",
" ):\n",
" with tempfile.NamedTemporaryFile(suffix=\".png\", delete=False) as tmp:\n",
" tmp.write(base64.b64decode(img_result))\n",
" temp_path = tmp.name\n",
"\n",
" # Open the image\n",
" print(f\"Image saved to: {temp_path}\")\n",
" try:\n",
" open_file(temp_path)\n",
" except Exception as e:\n",
" print(f\"Could not open image automatically: {e}\")\n",
"\n",
"\n",
"# Run the demo\n",
"await run_image_generation_demo()\n",
"\n",
"# End the AgentOps trace session\n",
"agentops.end_trace(tracer, end_state=\"Success\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 4. Web Search Tool\n",
"\n",
"The Web Search Tool allows agents to search the internet for current information and real-time data.\n",
"\n",
"**Key Features:**\n",
"- Search the web for current information\n",
"- Location-aware search results\n",
"- Real-time data access\n",
"- Configurable search parameters"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Start the AgentOps trace session\n",
"tracer = agentops.start_trace(\n",
" trace_name=\"Web Search Tool Example\", tags=[\"tools-demo\", \"openai-agents\", \"agentops-example\"]\n",
")\n",
"\n",
"\n",
"async def run_web_search_demo():\n",
" agent = Agent(\n",
" name=\"Web searcher\",\n",
" instructions=\"You are a helpful agent.\",\n",
" tools=[WebSearchTool(user_location={\"type\": \"approximate\", \"city\": \"New York\"})],\n",
" )\n",
"\n",
" with trace(\"Web search example\"):\n",
" result = await Runner.run(\n",
" agent,\n",
" \"search the web for 'local sports news' and give me 1 interesting update in a sentence.\",\n",
" )\n",
" print(result.final_output)\n",
" # Example output: The New York Giants are reportedly pursuing quarterback Aaron Rodgers after his ...\n",
"\n",
"\n",
"# Run the demo\n",
"await run_web_search_demo()\n",
"\n",
"# End the AgentOps trace session\n",
"agentops.end_trace(tracer, end_state=\"Success\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Conclusion\n",
"\n",
"Each tool extends agent capabilities and enables sophisticated automation. **AgentOps makes tool observability effortless** - simply import the library and all your tool interactions are automatically tracked, visualized, and analyzed. This enables you to:\n",
"\n",
"- Monitor tool performance across different use cases\n",
"- Optimize costs by understanding tool usage patterns\n",
"- Debug tool integration issues quickly\n",
"- Scale your AI applications with confidence in tool reliability\n",
"\n",
"Visit [app.agentops.ai](https://app.agentops.ai) to explore your tool usage sessions and gain deeper insights into your AI application's tool interactions."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "agentops (3.11.11)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.11"
}
},
"nbformat": 4,
"nbformat_minor": 4
}