Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI
Python agentagentopsagents-sdkaianthropicautogencost-estimationcrewaievalsevaluation-metricsgroqlangchainllmmistralollamaopenaiopenai-agents
Agno Tool Integrations notebook fix (#1167)
Pratyush Shuklac65c899abdfd
diff --git a/examples/agno/agno_tool_integrations.ipynb b/examples/agno/agno_tool_integrations.ipynbindex a3f66e4e..4cee33fc 100644--- a/examples/agno/agno_tool_integrations.ipynb+++ b/examples/agno/agno_tool_integrations.ipynb@@ -1,482 +1,244 @@{- "cells": [- {- "cell_type": "markdown",- "metadata": {},- "source": [- "# Tool Integration Example with Agno\n",- "\n",- "This example demonstrates how to integrate and use various tools with Agno agents, showing how AgentOps automatically tracks tool usage and agent interactions.\n",- "\n",- "## Overview\n",- "\n",- "This example demonstrates:\n",- "\n",- "1. **Using built-in Agno tools** like GoogleSearch, DuckDuckGo, and Arxiv\n",- "2. **Creating agents with tools** and seeing how they use them\n",- "3. **Tool execution tracking** with AgentOps\n",- "4. **Combining multiple tools** for comprehensive research\n",- "\n",- "## Key Concepts\n",- "\n",- "### Tool Integration\n",- "Agno provides various built-in tools that agents can use to access external information and services. These tools are automatically invoked by agents based on the task at hand.\n",- "\n",- "### Available Tools\n",- "- **GoogleSearchTools**: Search the web using Google\n",- "- **DuckDuckGoTools**: Alternative web search engine\n",- "- **ArxivTools**: Search academic papers on arXiv\n",- "- **HackerNewsTools**: Access HackerNews content\n",- "- And many more...\n",- "\n",- "### Automatic Tool Selection\n",- "Agents intelligently select which tools to use based on their instructions and the user's query. They can also combine multiple tools to provide comprehensive responses."- ]- },- {- "cell_type": "code",- "execution_count": null,- "metadata": {},- "outputs": [],- "source": [- "# Install the required dependencies\n",- "%pip install agentops\n",- "%pip install agno\n",- "%pip install python-dotenv"- ]- },- {- "cell_type": "code",- "execution_count": null,- "metadata": {},- "outputs": [],- "source": [- "import os\n",- "from dotenv import load_dotenv\n",- "\n",- "import agentops\n",- "from agno.agent import Agent\n",- "from agno.models.openai import OpenAIChat\n",- "from agno.tools.googlesearch import GoogleSearchTools\n",- "from agno.tools.duckduckgo import DuckDuckGoTools\n",- "from agno.tools.arxiv import ArxivTools"- ]- },- {- "cell_type": "code",- "execution_count": null,- "metadata": {},- "outputs": [],- "source": [- "load_dotenv()\n",- "os.environ[\"OPENAI_API_KEY\"] = os.getenv(\"OPENAI_API_KEY\", \"your_openai_api_key_here\")\n",- "os.environ[\"AGENTOPS_API_KEY\"] = os.getenv(\"AGENTOPS_API_KEY\", \"your_agentops_api_key_here\")"- ]- },- {- "cell_type": "code",- "execution_count": null,- "metadata": {},- "outputs": [],- "source": [- "agentops.init(auto_start_session=False, tags=[\"agno-tools\", \"tool-integration\", \"demo\"])"- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "## Example 1: Agent with Single Tool\n",- "\n",- "Let's start with a simple example where we create an agent that uses Google Search to find information. This demonstrates:\n",- "- How to create an agent with a specific tool\n",- "- How the agent automatically uses the tool when needed\n",- "- How AgentOps tracks the tool execution"- ]- },- {- "cell_type": "code",- "execution_count": null,- "metadata": {},- "outputs": [],- "source": [- "# Start an AgentOps trace for tracking\n",- "tracer = agentops.start_trace(trace_name=\"Example 1: Single Tool Agent\")\n",- "\n",- "try:\n",- " # Create an agent with Google Search tool\n",- " search_agent = Agent(\n",- " name=\"Search Agent\",\n",- " role=\"Research information using Google Search\",\n",- " model=OpenAIChat(id=\"gpt-4o-mini\"),\n",- " tools=[GoogleSearchTools()],\n",- " instructions=\"You are a research assistant. Use Google Search to find accurate, up-to-date information.\"\n",- " )\n",- " \n",- " # Run the agent with a query\n",- " response = search_agent.run(\"What are the latest developments in AI agents?\")\n",- " \n",- " print(\"Search Agent Response:\")\n",- " print(\"-\" * 60)\n",- " print(response.content)\n",- " print(\"-\" * 60)\n",- " \n",- " agentops.end_trace(tracer, end_state=\"Success\")\n",- " \n",- "except Exception as e:\n",- " print(f\"An error occurred: {e}\")\n",- " agentops.end_trace(tracer, end_state=\"Error\")"- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "## Example 2: Agent with Multiple Tools\n",- "\n",- "Now let's create an agent that has access to multiple tools. The agent will intelligently decide which tool(s) to use based on the query. This demonstrates:\n",- "- How to provide multiple tools to an agent\n",- "- How agents select appropriate tools\n",- "- How to combine information from different sources"- ]- },- {- "cell_type": "code",- "execution_count": null,- "metadata": {},- "outputs": [],- "source": [- "# Start a new trace for Example 2\n",- "tracer = agentops.start_trace(trace_name=\"Example 2: Multi-Tool Agent\")\n",- "\n",- "try:\n",- " # Create an agent with multiple research tools\n",- " research_agent = Agent(\n",- " name=\"Research Agent\",\n",- " role=\"Comprehensive research using multiple tools\",\n",- " model=OpenAIChat(id=\"gpt-4o-mini\"),\n",- " tools=[\n",- " GoogleSearchTools(),\n",- " ArxivTools(),\n",- " DuckDuckGoTools()\n",- " ],\n",- " instructions=\"\"\"You are a comprehensive research assistant. \n",- " Use Google Search for general information, Arxiv for academic papers, \n",- " and DuckDuckGo as an alternative search engine. \n",- " Provide well-researched, balanced information from multiple sources.\"\"\"\n",- " )\n",- " \n",- " # Run the agent with a complex query\n",- " response = research_agent.run(\n",- " \"Find information about recent advances in tool-use for AI agents. \"\n",- " \"Include both academic research and practical implementations.\"\n",- " )\n",- " \n",- " print(\"Research Agent Response:\")\n",- " print(\"-\" * 60)\n",- " print(response.content)\n",- " print(\"-\" * 60)\n",- " \n",- " agentops.end_trace(tracer, end_state=\"Success\")\n",- " \n",- "except Exception as e:\n",- " print(f\"An error occurred: {e}\")\n",- " agentops.end_trace(tracer, end_state=\"Error\")"- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "## Example 3: Specialized Tool Usage\n",- "\n",- "Let's create an agent specialized in academic research using the Arxiv tool. This demonstrates:\n",- "- How to create domain-specific agents\n",- "- How specialized tools provide focused results\n",- "- The value of tool specialization for specific tasks"- ]- },- {- "cell_type": "code",- "execution_count": null,- "metadata": {},- "outputs": [],- "source": [- "# Start a new trace for Example 3\n",- "tracer = agentops.start_trace(trace_name=\"Example 3: Academic Research Agent\")\n",- "\n",- "try:\n",- " # Create an agent specialized in academic research\n",- " academic_agent = Agent(\n",- " name=\"Academic Agent\",\n",- " role=\"Find and summarize academic papers\",\n",- " model=OpenAIChat(id=\"gpt-4o-mini\"),\n",- " tools=[ArxivTools()],\n",- " instructions=\"You are an academic research assistant. Use Arxiv to find relevant papers and provide concise summaries.\"\n",- " )\n",- " \n",- " # Run the agent with an academic query\n",- " response = academic_agent.run(\n",- " \"Find recent papers about tool augmented language models\"\n",- " )\n",- " \n",- " print(\"Academic Agent Response:\")\n",- " print(\"-\" * 60)\n",- " print(response.content)\n",- " print(\"-\" * 60)\n",- " \n",- " agentops.end_trace(tracer, end_state=\"Success\")\n",- " \n",- "except Exception as e:\n",- " print(f\"An error occurred: {e}\")\n",- " agentops.end_trace(tracer, end_state=\"Error\")"- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "## Example 4: Comparing Different Search Tools\n",- "\n",- "Finally, let's create an agent that can compare results from different search engines. This demonstrates:\n",- "- How agents can use multiple tools strategically\n",- "- How to compare and synthesize information from different sources\n",- "- The benefits of having alternative data sources"- ]- },- {- "cell_type": "code",- "execution_count": null,- "metadata": {},- "outputs": [],- "source": [- "# Start a new trace for Example 4\n",- "tracer = agentops.start_trace(trace_name=\"Example 4: Tool Comparison Agent\")\n",- "\n",- "try:\n",- " # Create an agent that compares different search tools\n",- " comparison_agent = Agent(\n",- " name=\"Comparison Agent\",\n",- " role=\"Compare results from different search engines\",\n",- " model=OpenAIChat(id=\"gpt-4o-mini\"),\n",- " tools=[\n",- " GoogleSearchTools(),\n",- " DuckDuckGoTools()\n",- " ],\n",- " instructions=\"\"\"Compare search results from Google and DuckDuckGo. \n",- " Note any differences in results, ranking, or information quality.\n",- " Be objective in your comparison.\"\"\"\n",- " )\n",- " \n",- " # Run the agent with a comparison task\n",- " response = comparison_agent.run(\n",- " \"Search for 'AgentOps observability platform' on both search engines and compare the results\"\n",- " )\n",- " \n",- " print(\"Comparison Agent Response:\")\n",- " print(\"-\" * 60)\n",- " print(response.content)\n",- " print(\"-\" * 60)\n",- " \n",- " agentops.end_trace(tracer, end_state=\"Success\")\n",- " \n",- "except Exception as e:\n",- " print(f\"An error occurred: {e}\")\n",- " agentops.end_trace(tracer, end_state=\"Error\")"- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "## Key Takeaways\n",- "\n",- "This notebook demonstrated several important concepts about tool integration in Agno:\n",- "\n",- "1. **Automatic Tool Usage**: Agents automatically determine when and how to use their available tools based on the query.\n",- "\n",- "2. **Tool Selection**: When given multiple tools, agents intelligently select the most appropriate ones for the task.\n",- "\n",- "3. **Information Synthesis**: Agents can combine information from multiple tools to provide comprehensive responses.\n",- "\n",- "4. **AgentOps Tracking**: All tool executions are automatically tracked by AgentOps, providing observability into:\n",- " - Which tools were used\n",- " - What parameters were passed\n",- " - Tool execution times\n",- " - Success/failure rates\n",- "\n",- "5. **Specialized vs. General Agents**: You can create both specialized agents (with specific tools) and general-purpose agents (with multiple tools).\n",- "\n",- "## Viewing Results in AgentOps\n",- "\n",- "After running these examples, you can visit [AgentOps Dashboard](https://app.agentops.ai/) to see:\n",- "- Detailed traces of each agent execution\n",- "- Tool usage patterns\n",- "- Performance metrics\n",- "- Error tracking and debugging information\n",- "\n",- "Each trace URL printed during execution provides direct access to that specific session's details."- ]- }- ],- "metadata": {- "kernelspec": {- "display_name": "Python 3",- "language": "python",- "name": "python3"- },- "language_info": {- "codemirror_mode": {- "name": "ipython",- "version": 3- },- "file_extension": ".py",- "mimetype": "text/x-python",- "name": "python",- "nbconvert_exporter": "python",- "pygments_lexer": "ipython3",- "version": "3.9.0"- }+ "cells": [+ {+ "cell_type": "markdown",+ "id": "intro-cell",+ "metadata": {},+ "source": [+ "# Tool Integration Example with Agno\n",+ "\n",+ "This example demonstrates how to integrate and use various tools with Agno agents,\n",+ "showing how AgentOps automatically tracks tool usage and agent interactions.\n",+ "\n",+ "## Overview\n",+ "This example demonstrates:\n",+ "\n",+ "1. **Using built-in Agno tools** like GoogleSearch, DuckDuckGo, and Arxiv\n",+ "2. **Creating agents with tools** and seeing how they use them\n",+ "3. **Tool execution tracking** with AgentOps\n",+ "4. **Combining multiple tools** for comprehensive research\n",+ "\n",+ "This example uses actual Agno components to show real tool integration patterns."+ ]},{"cell_type": "code","execution_count": null,- "id": "dfa7ae6b",+ "id": "setup-cell","metadata": {},"outputs": [],"source": ["# Install the required dependencies:\n","%pip install agentops\n",- "%pip install agno\n",+ "%pip install \"agno[tools]\"\n","%pip install python-dotenv"]},{"cell_type": "code","execution_count": null,- "id": "208656e8",+ "id": "imports-cell","metadata": {},"outputs": [],"source": ["import os\n","from dotenv import load_dotenv\n",- "\n","import agentops\n","from agno.agent import Agent\n","from agno.models.openai import OpenAIChat\n",- "\n",- "# Knowledge & RAG components\n",- "from agno.knowledge.url import UrlKnowledge\n",- "from agno.vectordb.lancedb import LanceDb\n",- "from agno.vectordb.search import SearchType\n",- "from agno.embedder.cohere import CohereEmbedder\n",- "from agno.reranker.cohere import CohereReranker\n",- "from agno.tools.reasoning import ReasoningTools"+ "from agno.tools.googlesearch import GoogleSearchTools\n",+ "from agno.tools.duckduckgo import DuckDuckGoTools\n",+ "from agno.tools.arxiv import ArxivTools"]},{"cell_type": "code","execution_count": null,- "id": "50eedd68",+ "id": "config-cell","metadata": {},"outputs": [],"source": ["# Load environment variables\n","load_dotenv()\n",+ "\n",+ "# Set environment variables if not already set\n","os.environ[\"OPENAI_API_KEY\"] = os.getenv(\"OPENAI_API_KEY\", \"your_openai_api_key_here\")\n","os.environ[\"AGENTOPS_API_KEY\"] = os.getenv(\"AGENTOPS_API_KEY\", \"your_agentops_api_key_here\")\n",- "os.environ[\"COHERE_API_KEY\"] = os.getenv(\"COHERE_API_KEY\")"+ "\n",+ "# Initialize AgentOps\n",+ "agentops.init(\n",+ " auto_start_session=False, trace_name=\"Agno Tool Integrations\", tags=[\"agno-tools\", \"tool-integration\", \"demo\"]\n",+ ")"]},{"cell_type": "code","execution_count": null,- "id": "51235ba1",- "metadata": {},- "outputs": [],- "source": [- "# Initialize AgentOps for monitoring\n",- "agentops.init(auto_start_session=False, tags=[\"agno-example\", \"tool-integrations\"])"- ]- },- {- "cell_type": "code",- "execution_count": 28,- "id": "24da7b0b",+ "id": "demo-function","metadata": {},"outputs": [],"source": ["def demonstrate_tool_integration():\n",- " \"\"\"\n",- " Demonstrate advanced tool integration with RAG and knowledge bases.\n",+ " \"\"\"Demonstrate tool integration with Agno agents.\"\"\"\n",+ " print(\"🚀 Agno Tool Integration Demonstration\")\n",+ " print(\"=\" * 60)\n",+ "\n",+ " # Start AgentOps trace\n",+ " tracer = agentops.start_trace(trace_name=\"Agno Tool Integration Demo\")\n","\n",- " This function shows how to:\n",- " 1. Create a knowledge base from external sources\n",- " 2. Set up a vector database with embeddings\n",- " 3. Configure an agent with RAG capabilities\n",- " 4. Enable reasoning tools for complex problem-solving\n",- " \"\"\"\n",- " tracer = agentops.start_trace(trace_name=\"Agno Tool Integration Demonstration\")\n"," try:\n",- " # Create knowledge base from documentation URLs\n",- " # This loads content from the specified URLs and prepares it for RAG\n",- " knowledge_base = UrlKnowledge(\n",- " urls=[\"https://docs.agno.com/introduction/agents.md\"],\n",- " vector_db=LanceDb(\n",- " uri=\"tmp/lancedb\",\n",- " table_name=\"agno_docs\",\n",- " search_type=SearchType.hybrid,\n",- " embedder=CohereEmbedder(\n",- " id=\"embed-v4.0\",\n",- " ),\n",- " reranker=CohereReranker(\n",- " model=\"rerank-v3.5\",\n",- " ),\n",- " ),\n",+ " # Example 1: Single Tool Agent\n",+ " print(\"\\n📌 Example 1: Agent with Google Search Tool\")\n",+ " print(\"-\" * 40)\n",+ "\n",+ " search_agent = Agent(\n",+ " name=\"Search Agent\",\n",+ " role=\"Research information using Google Search\",\n",+ " model=OpenAIChat(id=\"gpt-4o-mini\"),\n",+ " tools=[GoogleSearchTools()],\n",+ " instructions=\"You are a research assistant. Use Google Search to find accurate, up-to-date information.\",\n",+ " )\n",+ "\n",+ " response = search_agent.run(\"What are the latest developments in AI agents?\")\n",+ " print(f\"Search Agent Response:\\n{response.content}\")\n",+ "\n",+ " # Example 2: Multi-Tool Agent\n",+ " print(\"\\n\\n📌 Example 2: Agent with Multiple Tools\")\n",+ " print(\"-\" * 40)\n",+ "\n",+ " research_agent = Agent(\n",+ " name=\"Research Agent\",\n",+ " role=\"Comprehensive research using multiple tools\",\n",+ " model=OpenAIChat(id=\"gpt-4o-mini\"),\n",+ " tools=[GoogleSearchTools(), ArxivTools(), DuckDuckGoTools()],\n",+ " instructions=\"\"\"You are a comprehensive research assistant. \n",+ " Use Google Search for general information, Arxiv for academic papers, \n",+ " and DuckDuckGo as an alternative search engine. \n",+ " Provide well-researched, balanced information from multiple sources.\"\"\",\n",+ " )\n",+ "\n",+ " response = research_agent.run(\n",+ " \"Find information about recent advances in tool-use for AI agents. \"\n",+ " \"Include both academic research and practical implementations.\"\n"," )\n",+ " print(f\"Research Agent Response:\\n{response.content}\")\n","\n",- " # Create an intelligent agent with RAG capabilities\n",- " agent = Agent(\n",+ " # Example 3: Specialized Tool Usage\n",+ " print(\"\\n\\n📌 Example 3: Academic Research with Arxiv\")\n",+ " print(\"-\" * 40)\n",+ "\n",+ " academic_agent = Agent(\n",+ " name=\"Academic Agent\",\n",+ " role=\"Find and summarize academic papers\",\n"," model=OpenAIChat(id=\"gpt-4o-mini\"),\n",- " knowledge=knowledge_base,\n",- " search_knowledge=True,\n",- " tools=[ReasoningTools(add_instructions=True)],\n",- " instructions=[\n",- " \"Include sources in your response.\",\n",- " \"Always search your knowledge before answering the question.\",\n",- " \"Only include the output in your response. No other text.\",\n",- " ],\n",+ " tools=[ArxivTools()],\n",+ " instructions=\"You are an academic research assistant. Use Arxiv to find relevant papers and provide concise summaries.\",\n"," )\n","\n",- " # Print response with full reasoning process visible\n",- " agent.print_response(\n",- " \"What are Agents?\",\n",- " show_full_reasoning=True,\n",+ " response = academic_agent.run(\"Find recent papers about tool augmented language models\")\n",+ " print(f\"Academic Agent Response:\\n{response.content}\")\n",+ "\n",+ " # Example 4: Comparing Search Tools\n",+ " print(\"\\n\\n📌 Example 4: Comparing Different Search Tools\")\n",+ " print(\"-\" * 40)\n",+ "\n",+ " comparison_agent = Agent(\n",+ " name=\"Comparison Agent\",\n",+ " role=\"Compare results from different search engines\",\n",+ " model=OpenAIChat(id=\"gpt-4o-mini\"),\n",+ " tools=[GoogleSearchTools(), DuckDuckGoTools()],\n",+ " instructions=\"\"\"Compare search results from Google and DuckDuckGo. \n",+ " Note any differences in results, ranking, or information quality.\n",+ " Be objective in your comparison.\"\"\",\n",+ " )\n",+ "\n",+ " response = comparison_agent.run(\n",+ " \"Search for 'AgentOps observability platform' on both search engines and compare the results\"\n"," )\n",+ " print(f\"Comparison Agent Response:\\n{response.content}\")\n",+ "\n",+ " print(\"\\n\\n✨ Demonstration Complete!\")\n",+ " print(\"\\nKey Takeaways:\")\n",+ " print(\"- Agno agents can use multiple tools seamlessly\")\n",+ " print(\"- Tools are automatically invoked based on the agent's task\")\n",+ " print(\"- AgentOps tracks all tool executions automatically\")\n",+ " print(\"- Different tools serve different purposes (web search, academic search, etc.)\")\n",+ " print(\"- Agents can compare and synthesize information from multiple tools\")\n",+ "\n",+ " # End the AgentOps trace successfully\n",+ " print(\"\\n📊 View your tool execution traces in AgentOps:\")\n",+ " print(\" Visit https://app.agentops.ai/ to see detailed analytics\")\n"," agentops.end_trace(tracer, end_state=\"Success\")\n",- " except Exception:\n",- " agentops.end_trace(tracer, end_state=\"Error\")"+ "\n",+ " except Exception as e:\n",+ " print(f\"\\n❌ An error occurred: {e}\")\n",+ " agentops.end_trace(tracer, end_state=\"Error\")\n",+ " raise\n",+ "\n",+ " # Let's check programmatically that spans were recorded in AgentOps\n",+ " print(\"\\n\" + \"=\" * 50)\n",+ " print(\"Now let's verify that our LLM calls were tracked properly...\")\n",+ " try:\n",+ " agentops.validate_trace_spans(trace_context=tracer)\n",+ " print(\"\\n✅ Success! All LLM spans were properly recorded in AgentOps.\")\n",+ " except agentops.ValidationError as e:\n",+ " print(f\"\\n❌ Error validating spans: {e}\")\n",+ " raise"]},{"cell_type": "code","execution_count": null,- "id": "3cd3e2dc",+ "id": "run-demo","metadata": {},"outputs": [],"source": ["demonstrate_tool_integration()"]+ },+ {+ "cell_type": "markdown",+ "id": "summary-cell",+ "metadata": {},+ "source": [+ "## Summary\n",+ "\n",+ "This notebook demonstrated how to:\n",+ "\n",+ "- Set up Agno agents with various tools (Google Search, DuckDuckGo, Arxiv)\n",+ "- Create single-tool and multi-tool agents\n",+ "- Track tool usage with AgentOps\n",+ "- Compare results from different search engines\n",+ "- Validate that all operations are properly traced\n",+ "\n",+ "## Next Steps\n",+ "\n",+ "Visit the AgentOps dashboard to explore:\n",+ "- Detailed tool execution metrics\n",+ "- Agent performance analytics \n",+ "- Error tracking and debugging information\n",+ "\n",+ "Each trace URL printed during execution provides direct access to that specific session's details."+ ]}],"metadata": {- "jupytext": {- "cell_metadata_filter": "-all",- "main_language": "python",- "notebook_metadata_filter": "-all"+ "kernelspec": {+ "display_name": "Python 3",+ "language": "python",+ "name": "python3"},"language_info": {- "name": "python"+ "codemirror_mode": {+ "name": "ipython",+ "version": 3+ },+ "file_extension": ".py",+ "mimetype": "text/x-python",+ "name": "python",+ "nbconvert_exporter": "python",+ "pygments_lexer": "ipython3",+ "version": "3.9.0"}},"nbformat": 4,