Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI
Python agentagentopsagents-sdkaianthropicautogencost-estimationcrewaievalsevaluation-metricsgroqlangchainllmmistralollamaopenaiopenai-agents
fix: Sync with the latest example notebooks (#1089)
Pratyush Shukla3145b2ce6cc1
diff --git a/docs/v2/examples/autogen.mdx b/docs/v2/examples/autogen.mdxindex 9893b265..7e5ba291 100644--- a/docs/v2/examples/autogen.mdx+++ b/docs/v2/examples/autogen.mdx@@ -1,6 +1,6 @@----title: 'Autogen Example'-description: 'Using Autogen with AgentOps for agent chat monitoring'+title: 'AutoGen'+description: 'Microsoft Autogen Chat Example'---{/* SOURCE_FILE: examples/autogen/AgentChat.ipynb */}@@ -10,25 +10,26 @@ _View Notebook on <a href={'https://github.com/AgentOps-AI/agentops/blob/main/exAgentOps automatically configures itself when it's initialized meaning your agent run data will be tracked and logged to your AgentOps dashboard right away.-First let's install the required packages.+First let's install the required packages++## Installation<CodeGroup>```bash pip- pip install -U autogen-agentchat "autogen-ext[openai]" agentops python-dotenv+ pip install "autogen-ext[openai]" -U agentops autogen-agentchat python-dotenv``````bash poetry- poetry add autogen-agentchat autogen-ext agentops python-dotenv- # For autogen-ext[openai] with poetry, you might need to specify openai as an extra:- # poetry add autogen-ext -E openai+ poetry add "autogen-ext[openai]" -U agentops autogen-agentchat python-dotenv``````bash uv- uv add autogen-agentchat "autogen-ext[openai]" agentops python-dotenv+ uv add "autogen-ext[openai]" -U agentops autogen-agentchat python-dotenv```</CodeGroup>-## Setup-Then import them.+Then import them++```pythonimport osfrom dotenv import load_dotenv@@ -51,23 +52,27 @@ Next, we'll set our API keys. There are several ways to do this, the code below[Get an AgentOps API key](https://agentops.ai/settings/projects)-1. Create an environment variable in a `.env` file or other method. By default, the AgentOps `init()` function will look for an environment variable named `AGENTOPS_API_KEY`. Or...+1. Create an environment variable in a .env file or other method. By default, the AgentOps `init()` function will look for an environment variable named `AGENTOPS_API_KEY`. Or...2. Replace `<your_agentops_key>` below and pass in the optional `api_key` parameter to the AgentOps `init(api_key=...)` function. Remember not to commit your API key to a public repo!+```pythonload_dotenv()-os.environ["AGENTOPS_API_KEY"] = os.getenv("AGENTOPS_API_KEY", "your_agentops_api_key_here")+os.environ["AGENTOPS_API_KEY"] = os.getenv("AGENTOPS_API_KEY", "your_api_key_here")os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY", "your_openai_api_key_here")```+```python# When initializing AgentOps, you can pass in optional tags to help filter sessionsagentops.init(auto_start_session=False)-tracer = agentops.start_trace(trace_name="Microsoft Agent Chat Example", tags=["autogen-chat", "microsoft-autogen", "agentops-example"])+tracer = agentops.start_trace(+ trace_name="Microsoft Agent Chat Example", tags=["autogen-chat", "microsoft-autogen", "agentops-example"]+)```-AutoGen will now start automatically tracking:+AutoGen will now start automatically tracking* LLM prompts and completions* Token usage and costs@@ -77,9 +82,11 @@ AutoGen will now start automatically tracking:* Errors# Simple Chat Example++```python# Define model and API key-model_name = "gpt-4-turbo" # Or "gpt-4o" / "gpt-4o-mini" as per migration guide examples+model_name = "gpt-4-turbo" # Or "gpt-4o" / "gpt-4o-mini" as per migration guide examplesapi_key = os.getenv("OPENAI_API_KEY")# Create the model client@@ -88,26 +95,27 @@ model_client = OpenAIChatCompletionClient(model=model_name, api_key=api_key)# Create the agent that uses the LLM.assistant = AssistantAgent(name="assistant",- system_message="You are a helpful assistant.", # Added system message for clarity- model_client=model_client+ system_message="You are a helpful assistant.", # Added system message for clarity+ model_client=model_client,)user_proxy_initiator = UserProxyAgent("user_initiator")+async def main():- termination = MaxMessageTermination(max_messages=2)+ termination = MaxMessageTermination(max_messages=2)group_chat = RoundRobinGroupChat(- [user_proxy_initiator, assistant], # Corrected: agents as positional argument- termination_condition=termination+ [user_proxy_initiator, assistant], # Corrected: agents as positional argument+ termination_condition=termination,)-+chat_task = "How can I help you today?"print(f"User Initiator: {chat_task}")try:stream = group_chat.run_stream(task=chat_task)- await Console().run(stream)+ await Console().run(stream)agentops.end_trace(tracer, end_state="Success")except StdinNotImplementedError:@@ -120,7 +128,8 @@ async def main():finally:await model_client.close()-if __name__ == '__main__':++if __name__ == "__main__":try:loop = asyncio.get_running_loop()except RuntimeError:@@ -128,6 +137,7 @@ if __name__ == '__main__':if loop and loop.is_running():import nest_asyncio+nest_asyncio.apply()asyncio.run(main())else:@@ -138,7 +148,8 @@ You can view data on this run at [app.agentops.ai](https://app.agentops.ai).The dashboard will display LLM events for each message sent by each agent, including those made by the human user.+<script type="module" src="/scripts/github_stars.js"></script><script type="module" src="/scripts/scroll-img-fadein-animation.js"></script><script type="module" src="/scripts/button_heartbeat_animation.js"></script>-<script type="module" src="/scripts/adjust_api_dynamically.js"></script>+<script type="module" src="/scripts/adjust_api_dynamically.js"></script>\ No newline at end of filediff --git a/docs/v2/examples/examples.mdx b/docs/v2/examples/examples.mdxindex 545d1038..f7e818cc 100644--- a/docs/v2/examples/examples.mdx+++ b/docs/v2/examples/examples.mdx@@ -13,7 +13,7 @@ description: 'Examples of AgentOps with various integrations'Claude integration with tool usage and advanced features</Card>- <Card title="Google Generative AI" icon={<img src="https://www.github.com/agentops-ai/agentops/blob/main/docs/images/external/deepmind/gemini-logo.png?raw=true" alt="Gemini" />} iconType="image" href="/v2/examples/google_generative_ai">+ <Card title="Google Generative AI" icon={<img src="https://www.github.com/agentops-ai/agentops/blob/main/docs/images/external/deepmind/gemini-logo.png?raw=true" alt="Gemini" />} iconType="image" href="/v2/examples/google_genai">Google Gemini models and their examples</Card>diff --git a/docs/v2/examples/google_adk.mdx b/docs/v2/examples/google_adk.mdxindex 6a454790..fd5ffb03 100644--- a/docs/v2/examples/google_adk.mdx+++ b/docs/v2/examples/google_adk.mdx@@ -1,5 +1,5 @@----title: 'Google Adk'+title: 'Google ADK'description: 'Google ADK Example: Human Approval Workflow with AgentOps'---{/* SOURCE_FILE: examples/google_adk/human_approval.ipynb */}@@ -74,7 +74,7 @@ APP_NAME = "human_approval_app_notebook"USER_ID = "test_user_notebook_123"SESSION_ID = "approval_session_notebook_456"MODEL_NAME = "gemini-1.5-flash"-agentops.start_trace(trace_name=APP_NAME, tags=["google_adk","notebook"])+agentops.start_trace(trace_name=APP_NAME, tags=["google_adk", "notebook"])```## 3. Define Schemas@@ -86,6 +86,8 @@ Pydantic models are used to define the structure of data for approval requests aclass ApprovalRequest(BaseModel):amount: float = Field(description="The amount requiring approval")reason: str = Field(description="The reason for the request")++class ApprovalDecision(BaseModel):decision: str = Field(description="The approval decision: 'approved' or 'rejected'")comments: str = Field(description="Additional comments from the approver")@@ -98,10 +100,10 @@ This tool now directly prompts the user for an approval decision. In a real-worl```pythonasync def external_approval_tool(amount: float, reason: str) -> str:- """+ """Prompts for human approval and returns the decision as a JSON string."""- print(f"🔔 HUMAN APPROVAL REQUIRED:")+ print("🔔 HUMAN APPROVAL REQUIRED:")print(f" Amount: ${amount:,.2f}")print(f" Reason: {reason}")decision = ""@@ -112,12 +114,8 @@ async def external_approval_tool(amount: float, reason: str) -> str:comments = input(" Enter comments (optional): ").strip()print(f" Decision: {decision.upper()}")print(f" Comments: {comments if comments else 'N/A'}")- return json.dumps({- "decision": decision,- "comments": comments,- "amount": amount,- "reason": reason- })+ return json.dumps({"decision": decision, "comments": comments, "amount": amount, "reason": reason})+# Create the approval tool instanceapproval_tool = FunctionTool(func=external_approval_tool)@@ -145,13 +143,13 @@ prepare_request = LlmAgent(4. Respond with a summary of what will be submitted for approvalIf the user input is missing amount or reason, ask for clarification.""",- output_key="request_prepared"+ output_key="request_prepared",)# Agent 2: Request human approval using the toolrequest_approval = LlmAgent(model=MODEL_NAME,- name="RequestHumanApprovalAgent",+ name="RequestHumanApprovalAgent",description="Calls the external approval system with prepared request details",instruction="""You are a human approval request agent.Your task:@@ -162,7 +160,7 @@ request_approval = LlmAgent(Always use the exact values from the session state for the tool call.""",tools=[approval_tool],- output_key="approval_requested"+ output_key="approval_requested",)# Agent 3: Process the approval decision@@ -180,7 +178,7 @@ process_decision = LlmAgent(Be professional and helpful in your response.""",- output_key="final_decision"+ output_key="final_decision",)```@@ -193,7 +191,7 @@ Combine the agents into a sequential workflow. The `SequentialAgent` ensures thaapproval_workflow = SequentialAgent(name="HumanApprovalWorkflowNotebook",description="Complete workflow for processing approval requests with human oversight",- sub_agents=[prepare_request, request_approval, process_decision]+ sub_agents=[prepare_request, request_approval, process_decision],)```@@ -205,11 +203,7 @@ Set up an in-memory session service and the workflow runner.```pythonsession_service = InMemorySessionService()# Create runner-workflow_runner = Runner(- agent=approval_workflow,- app_name=APP_NAME,- session_service=session_service-)+workflow_runner = Runner(agent=approval_workflow, app_name=APP_NAME, session_service=session_service)```## 8. Helper Function to Run Workflow@@ -220,15 +214,12 @@ This function encapsulates the logic to run the workflow for a given user reques```pythonasync def run_approval_workflow_notebook(user_request: str, session_id: str):"""Run the complete approval workflow with a user request in the notebook environment"""- print(f"{'='*60}")+ print(f"{'=' * 60}")print(f" Starting Approval Workflow for Session: {session_id}")- print(f"{'='*60}")+ print(f"{'=' * 60}")print(f"User Request: {user_request}")# Create user message- user_content = types.Content(- role='user',- parts=[types.Part(text=user_request)]- )+ user_content = types.Content(role="user", parts=[types.Part(text=user_request)])step_count = 0final_response = "No response received"# Run the workflow@@ -247,12 +238,12 @@ async def run_approval_workflow_notebook(user_request: str, session_id: str):final_response = response_textsession = await session_service.get_session(app_name=APP_NAME,- user_id=USER_ID,+ user_id=USER_ID,session_id=session_id,)- print(f"{'='*60}")+ print(f"{'=' * 60}")print(f"📊 Workflow Complete - Session State ({session_id}):")- print(f"{'='*60}")+ print(f"{'=' * 60}")for key, value in session.state.items():print(f" {key}: {value}")print(f"🎯 Final Response: {final_response}")@@ -269,18 +260,16 @@ async def main_notebook():test_requests = ["I need approval for $750 for team lunch and celebrations","Please approve $3,000 for a conference ticket and travel expenses",- "I need $12,000 approved for critical software licenses renewal"+ "I need $12,000 approved for critical software licenses renewal",]for i, request in enumerate(test_requests, 1):- current_session_id = f"approval_session_notebook_{456 + i -1}"+ current_session_id = f"approval_session_notebook_{456 + i - 1}"# Create the session before running the workflow- await session_service.create_session(- app_name=APP_NAME,- user_id=USER_ID,- session_id=current_session_id- )+ await session_service.create_session(app_name=APP_NAME, user_id=USER_ID, session_id=current_session_id)print(f"Created session: {current_session_id}")await run_approval_workflow_notebook(request, current_session_id)++try:asyncio.run(main_notebook())agentops.end_trace(end_state="Success")diff --git a/docs/v2/examples/google_genai.mdx b/docs/v2/examples/google_genai.mdxnew file mode 100644index 00000000..6bd5770c--- /dev/null+++ b/docs/v2/examples/google_genai.mdx@@ -0,0 +1,93 @@+---+title: 'Google GenAI'+description: 'Google Generative AI Example with AgentOps'+---+{/* SOURCE_FILE: examples/google_genai/gemini_example.ipynb */}++_View Notebook on <a href={'https://github.com/AgentOps-AI/agentops/blob/main/examples/google_genai/gemini_example.ipynb'} target={'_blank'}>Github</a>_++# Google Generative AI Example with AgentOps++This notebook demonstrates how to use AgentOps with Google's Generative AI package for observing both synchronous and streaming text generation.+++++## Installation+<CodeGroup>+ ```bash pip+ pip install agentops google-genai+ ```+ ```bash poetry+ poetry add agentops google-genai+ ```+ ```bash uv+ uv add agentops google-genai+ ```+</CodeGroup>++```python+from google import genai+import agentops+from dotenv import load_dotenv+import os+```+++```python+load_dotenv()++os.environ["AGENTOPS_API_KEY"] = os.getenv("AGENTOPS_API_KEY", "your_api_key_here")+os.environ["GEMINI_API_KEY"] = os.getenv("GEMINI_API_KEY", "your_gemini_api_key_here")+```+++```python+# Initialize AgentOps and Gemini client+agentops.init(tags=["gemini-example", "agentops-example"])+client = genai.Client()+```+++```python+# Test synchronous generation+print("Testing synchronous generation:")+response = client.models.generate_content(model="gemini-1.5-flash", contents="What are the three laws of robotics?")+print(response.text)+```+++```python+# Test streaming generation+print("\nTesting streaming generation:")+response_stream = client.models.generate_content_stream(+ model="gemini-1.5-flash", contents="Explain the concept of machine learning in simple terms."+)++for chunk in response_stream:+ print(chunk.text, end="")+print() # Add newline after streaming output++# Test another synchronous generation+print("\nTesting another synchronous generation:")+response = client.models.generate_content(+ model="gemini-1.5-flash", contents="What is the difference between supervised and unsupervised learning?"+)+print(response.text)+```+++```python+# Example of token counting+print("\nTesting token counting:")+token_response = client.models.count_tokens(+ model="gemini-1.5-flash", contents="This is a test sentence to count tokens."+)+print(f"Token count: {token_response.total_tokens}")+```+++<script type="module" src="/scripts/github_stars.js"></script>+<script type="module" src="/scripts/scroll-img-fadein-animation.js"></script>+<script type="module" src="/scripts/button_heartbeat_animation.js"></script>+<script type="module" src="/scripts/adjust_api_dynamically.js"></script>\ No newline at end of filediff --git a/docs/v2/examples/google_generative_ai.mdx b/docs/v2/examples/google_generative_ai.mdxdeleted file mode 100644index 344959d5..00000000--- a/docs/v2/examples/google_generative_ai.mdx+++ /dev/null@@ -1,102 +0,0 @@-----title: 'Google Generative AI Example'-description: 'Using Google Generative AI API with AgentOps for synchronous and streaming text generation'-----{/* SOURCE_FILE: examples/google_genai/gemini_example.ipynb */}--_View Notebook on <a href={'https://github.com/AgentOps-AI/agentops/blob/main/examples/gemini/gemini_example.ipynb'} target={'_blank'}>Github</a>_--# Google Generative AI API Example with AgentOps--This notebook demonstrates how to use AgentOps with Google's Gemini API for observing both synchronous and streaming text generation.--## Installation-Install necessary packages:-<CodeGroup>- ```bash pip- pip install agentops google-generativeai python-dotenv- ```- ```bash poetry- poetry add agentops google-generativeai python-dotenv- ```- ```bash uv- uv add agentops google-generativeai python-dotenv- ```-</CodeGroup>--## Setup-```python-from google import generativeai as genai # Renamed for clarity as per official docs-import agentops-from dotenv import load_dotenv-import os-```--Set API Keys:-```python-load_dotenv()--os.environ["AGENTOPS_API_KEY"] = os.getenv("AGENTOPS_API_KEY", "your_agentops_api_key_here")-os.environ["GEMINI_API_KEY"] = os.getenv("GEMINI_API_KEY", "your_gemini_api_key_here")--# Configure the GenAI client with the API key-genai.configure(api_key=os.environ["GEMINI_API_KEY"])-```-- Initialize AgentOps:-```python-# Initialize AgentOps and Gemini client-agentops.init(tags=["gemini-example", "agentops-example"])-# The client is configured globally via genai.configure()-```--## Synchronous Generation-```python-# Test synchronous generation-print("Testing synchronous generation:")-model = genai.GenerativeModel('gemini-1.5-flash') # Create model instance-response = model.generate_content("What are the three laws of robotics?")-print(response.text)-```--## Streaming Generation-```python-# Test streaming generation-print("\nTesting streaming generation:")-model = genai.GenerativeModel('gemini-1.5-flash') # Create model instance-response_stream = model.generate_content(- "Explain the concept of machine learning in simple terms.",- stream=True-)--for chunk in response_stream:- print(chunk.text, end="")-print() # Add newline after streaming output-```--## Another Synchronous Generation-```python-# Test another synchronous generation-print("\nTesting another synchronous generation:")-model = genai.GenerativeModel('gemini-1.5-flash') # Create model instance-response = model.generate_content(- "What is the difference between supervised and unsupervised learning?"-)-print(response.text)-```--## Token Counting-```python-# Example of token counting-print("\nTesting token counting:")-model = genai.GenerativeModel('gemini-1.5-flash') # Create model instance-token_response = model.count_tokens("This is a test sentence to count tokens.")-print(f"Token count: {token_response.total_tokens}")--agentops.end_session("Success") # End the session-```--<script type="module" src="/scripts/github_stars.js"></script>-<script type="module" src="/scripts/scroll-img-fadein-animation.js"></script>-<script type="module" src="/scripts/button_heartbeat_animation.js"></script>-<script type="module" src="/scripts/adjust_api_dynamically.js"></script>diff --git a/docs/v2/examples/langchain.mdx b/docs/v2/examples/langchain.mdxindex 9dfea865..da212b84 100644--- a/docs/v2/examples/langchain.mdx+++ b/docs/v2/examples/langchain.mdx@@ -1,6 +1,6 @@----title: 'LangChain Example'-description: 'Using the LangChain Callback Handler with AgentOps'+title: 'LangChain'+description: 'AgentOps Langchain Agent Implementation'---{/* SOURCE_FILE: examples/langchain/langchain_examples.ipynb */}@@ -8,100 +8,88 @@ _View Notebook on <a href={'https://github.com/AgentOps-AI/agentops/blob/main/ex# AgentOps Langchain Agent Implementation-Using AgentOps monitoring with Langchain is simple. We've created a `LangchainCallbackHandler` that will do all of the heavy lifting!+Using AgentOps monitoring with Langchain is simple. We've created a LangchainCallbackHandler that will do all of the heavy lifting!++First let's install the required packages-## Installation-First, let's install the required packages:+## Installation<CodeGroup>-```bash pip-pip install agentops langchain langchain_openai python-dotenv-```-```bash poetry-poetry add agentops langchain langchain_openai python-dotenv-```-```bash uv-uv add agentops langchain langchain_openai python-dotenv-```+ ```bash pip+ pip install agentops langchain langchain_openai python-dotenv+ ```+ ```bash poetry+ poetry add agentops langchain langchain_openai python-dotenv+ ```+ ```bash uv+ uv add agentops langchain langchain_openai python-dotenv+ ```</CodeGroup>-## Setup+Then import them+-### Import Dependencies-Next, import the necessary libraries:```pythonimport osfrom langchain_openai import ChatOpenAIfrom langchain.agents import tool, AgentExecutor, create_openai_tools_agentfrom dotenv import load_dotenvfrom langchain_core.prompts import ChatPromptTemplate+```++The only difference with using AgentOps is that we'll also import this special Callback Handler+++```pythonfrom agentops.integration.callbacks.langchain import (LangchainCallbackHandler as AgentOpsLangchainCallbackHandler,)```-### Configure API Keys-Set up your API keys. This example shows loading from environment variables, which is a common practice.-[Get an AgentOps API key](https://agentops.ai/settings/projects) and ensure your OpenAI API key is also accessible.+Next, we'll set our API keys. There are several ways to do this, the code below is just the most foolproof way for the purposes of this notebook. It accounts for both users who use environment variables and those who just want to set the API Key here in this notebook.-1. Create an environment variable in a `.env` file or other method. By default, the AgentOps SDK will look for an environment variable named `AGENTOPS_API_KEY`.-2. Similarly, ensure `OPENAI_API_KEY` is set for LangChain's OpenAI integration.+[Get an AgentOps API key](https://agentops.ai/settings/projects)-```python-from dotenv import load_dotenv # Typically at the start of your script-import os # Typically at the start of your script+1. Create an environment variable in a .env file or other method. By default, the AgentOps `init()` function will look for an environment variable named `AGENTOPS_API_KEY`. Or...++2. Replace `<your_agentops_key>` below and pass in the optional `api_key` parameter to the AgentOps `init(api_key=...)` function. Remember not to commit your API key to a public repo!-load_dotenv()-# These lines ensure the environment variables are set for the script's context.-# Replace "your_agentops_api_key_here" and "your_openai_api_key_here"-# if you are not using a .env file or pre-set environment variables.-os.environ["AGENTOPS_API_KEY"] = os.getenv("AGENTOPS_API_KEY", "your_agentops_api_key_here")+```python+load_dotenv()+os.environ["AGENTOPS_API_KEY"] = os.getenv("AGENTOPS_API_KEY", "your_api_key_here")os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY", "your_openai_api_key_here")```-## Initialize AgentOps and LangChain Components-<Tip>-Note that you don't need to set up a separate `agentops.init()` call when using the `LangchainCallbackHandler`, as it will automatically initialize the AgentOps client for you if an API key is provided to it or found in the environment.-</Tip>+This is where AgentOps comes into play. Before creating our LLM instance via Langchain, first we'll create an instance of the AO LangchainCallbackHandler. After the handler is initialized, a session will be recorded automatically.++Pass in your API key, and optionally any tags to describe this session for easier lookup in the AO dashboard.+-This is where AgentOps comes into play. Before creating our LLM instance via Langchain, first we'll create an instance of the `AgentOpsLangchainCallbackHandler`. After the handler is initialized, a session will be recorded automatically. Pass in any tags to describe this session for easier lookup in the AgentOps dashboard.```python-agentops_handler = AgentOpsLangchainCallbackHandler(- tags=["Langchain Example", "agentops-example"]- # The AGENTOPS_API_KEY from the environment will be used automatically.- # Alternatively, you could pass api_key=os.getenv("AGENTOPS_API_KEY")-)+agentops_handler = AgentOpsLangchainCallbackHandler(tags=["Langchain Example", "agentops-example"])-llm = ChatOpenAI(- callbacks=[agentops_handler],- model="gpt-3.5-turbo"- # The OPENAI_API_KEY from the environment will be used automatically.- # Alternatively, you could pass openai_api_key=os.getenv("OPENAI_API_KEY")-)+llm = ChatOpenAI(callbacks=[agentops_handler], model="gpt-3.5-turbo")-# This explicit assignment is redundant if callbacks are passed in the constructor,-# but we include it here to match the source notebook.+# You must pass in a callback handler to record your agentllm.callbacks = [agentops_handler]-```-### Define Prompt-Set up the prompt template for your agent.-```pythonprompt = ChatPromptTemplate.from_messages([("system", "You are a helpful assistant. Respond only in Spanish."),("human", "{input}"),# Placeholders fill up a **list** of messages("placeholder", "{agent_scratchpad}"),+ # ("tool_names", "find_movie")])```-### Define Tools-Agents generally use tools. Let's define a simple tool here. Tool usage is also recorded by AgentOps.+Agents generally use tools. Let's define a simple tool here. Tool usage is also recorded.++```python@tooldef find_movie(genre: str) -> str:@@ -111,39 +99,43 @@ def find_movie(genre: str) -> str:else:return "Pineapple Express"+tools = [find_movie]```-For each tool, you need to also add the callback handler:++For each tool, you need to also add the callback handler++```pythonfor t in tools:t.callbacks = [agentops_handler]```-Bind the tools to the LLM. Note: Depending on the agent creation method, this explicit binding step might sometimes be handled internally by the agent constructor. In this example, we follow the notebook's pattern.++Add the tools to our LLM++```pythonllm_with_tools = llm.bind_tools([find_movie])```-## Create and Run Agent-Finally, let's create our agent! Pass in the callback handler to the agent, and all the actions will be recorded in the AgentOps Dashboard.+Finally, let's create our agent! Pass in the callback handler to the agent, and all the actions will be recorded in the AO Dashboard++```python-# Note: The notebook uses 'llm' here, not 'llm_with_tools'.-# create_openai_tools_agent is designed to work with the base llm and tools list.-agent = create_openai_tools_agent(llm, tools, prompt)+agent = create_openai_tools_agent(llm, tools, prompt)agent_executor = AgentExecutor(agent=agent, tools=tools)```-Now, invoke the agent. Ensure the callback handler is included in the configuration for the invocation.+```python-agent_executor.invoke(- {"input": "What comedies are playing?"},- config={"callback": [agentops_handler]} # Matches notebook's singular 'callback'-)+agent_executor.invoke({"input": "What comedies are playing?"}, config={"callback": [agentops_handler]})```## Check your sessionFinally, check your run on [AgentOps](https://app.agentops.ai). You will see a session recorded with the LLM calls and tool usage.+<script type="module" src="/scripts/github_stars.js"></script><script type="module" src="/scripts/scroll-img-fadein-animation.js"></script><script type="module" src="/scripts/button_heartbeat_animation.js"></script>-<script type="module" src="/scripts/adjust_api_dynamically.js"></script>+<script type="module" src="/scripts/adjust_api_dynamically.js"></script>\ No newline at end of filediff --git a/docs/v2/examples/litellm.mdx b/docs/v2/examples/litellm.mdxindex f6392100..224d82f1 100644--- a/docs/v2/examples/litellm.mdx+++ b/docs/v2/examples/litellm.mdx@@ -1,6 +1,6 @@----title: 'LiteLLM Example'-description: 'Using LiteLLM with AgentOps for observing LLM calls'+title: 'LiteLLM'+description: 'AgentOps for observing LiteLLM'---{/* SOURCE_FILE: examples/litellm/litellm_example.ipynb */}@@ -11,25 +11,28 @@ _View Notebook on <a href={'https://github.com/AgentOps-AI/agentops/blob/main/exWe can use AgentOps to observe LiteLLM, a lightweight library for working with large language models. This integration allows you to monitor and log the performance of your LiteLLM applications, providing insights into their behavior and efficiency.LiteLLM integration extends observability to the different agent libraries which rely on LiteLLM and hence make it possible to observe the agents built using these libraries.-[See our LiteLLM docs](/v2/integrations/litellm)+[See our LiteLLM docs](https://docs.agentops.ai/v1/integrations/litellm)++First let's install the required packages+-First let's install the required packages.## Installation<CodeGroup>```bash pip- pip install -U litellm agentops python-dotenv+ pip install -U agentops litellm python-dotenv``````bash poetry- poetry add litellm agentops python-dotenv+ poetry add -U agentops litellm python-dotenv``````bash uv- uv add litellm agentops python-dotenv+ uv add -U agentops litellm python-dotenv```</CodeGroup>-## Setup-Then import them.+Then import them++```pythonimport litellmimport agentops@@ -41,50 +44,56 @@ Next, we'll set our API keys. There are several ways to do this, the code below[Get an AgentOps API key](https://agentops.ai/settings/projects)-1. Create an environment variable in a `.env` file or other method. By default, the AgentOps `init()` function will look for an environment variable named `AGENTOPS_API_KEY`. Or...+1. Create an environment variable in a .env file or other method. By default, the AgentOps `init()` function will look for an environment variable named `AGENTOPS_API_KEY`. Or...2. Replace `<your_agentops_key>` below and pass in the optional `api_key` parameter to the AgentOps `init(api_key=...)` function. Remember not to commit your API key to a public repo!-LiteLLM allows you to use several models including from OpenAI, Llama, Mistral, Claude, Gemini, Gemma, Dall-E, Whisper, and more all using the OpenAI format. To use a different model all you need to change are the API KEY for that provider and the model name in `litellm.completion(model="...")`.+LiteLLM allows you to use several models including from OpenAI, Llama, Mistral, Claude, Gemini, Gemma, Dall-E, Whisper, and more all using the OpenAI format. To use a different model all you need to change are the API KEY and model (litellm.completion(model="...")).+```pythonload_dotenv()-os.environ["AGENTOPS_API_KEY"] = os.getenv("AGENTOPS_API_KEY", "your_agentops_api_key_here")-os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY", "your_openai_api_key_here") # or the provider of your choosing+os.environ["AGENTOPS_API_KEY"] = os.getenv("AGENTOPS_API_KEY", "your_api_key_here")+os.environ["OPENAI_API_KEY"] = os.getenv(+ "OPENAI_API_KEY", "your_openai_api_key_here"+) # or the provider of your choosing```-Initialize AgentOps:+```pythonagentops.init(auto_start_session=False)tracer = agentops.start_trace(trace_name="LiteLLM Example", tags=["litellm-example", "agentops-example"])```-## Making an LLM Call+Note: AgentOps requires that you call LiteLLM completions differently than the LiteLLM's docs mention+Instead of doing this --**Note:** AgentOps requires that you call LiteLLM completions differently than the LiteLLM's docs mention.-Instead of doing this:```pythonfrom litellm import completioncompletion()```-You should do this:++You should do this -+```pythonimport litellmlitellm.completion()```+```pythonmessages = [{"role": "user", "content": "Write a 12 word poem about secret agents."}]response = litellm.completion(model="gpt-4", messages=messages) # or the model of your choosingprint(response.choices[0].message.content)```-End the AgentOps trace:+```pythonagentops.end_trace(tracer, end_state="Success")```+<script type="module" src="/scripts/github_stars.js"></script><script type="module" src="/scripts/scroll-img-fadein-animation.js"></script><script type="module" src="/scripts/button_heartbeat_animation.js"></script>-<script type="module" src="/scripts/adjust_api_dynamically.js"></script>+<script type="module" src="/scripts/adjust_api_dynamically.js"></script>\ No newline at end of filediff --git a/docs/v2/examples/mem0.mdx b/docs/v2/examples/mem0.mdxindex 1dc8dc78..0ce76d2d 100644--- a/docs/v2/examples/mem0.mdx+++ b/docs/v2/examples/mem0.mdx@@ -1,5 +1,5 @@----title: 'Mem0 Example'+title: 'Mem0'description: 'Memory Operations with Mem0'---{/* SOURCE_FILE: examples/mem0/mem0_memory_example.ipynb */}diff --git a/docs/v2/examples/openai.mdx b/docs/v2/examples/openai.mdxindex e7b5104e..ca66e9a5 100755--- a/docs/v2/examples/openai.mdx+++ b/docs/v2/examples/openai.mdx@@ -1,63 +1,69 @@----title: 'OpenAI Multi-Tool RAG Example'-description: 'Demonstrates multi-tool orchestration with a RAG approach using OpenAI, Pinecone, and AgentOps'+title: 'OpenAI'+description: 'Load the dataset (ensure you're logged in with huggingface-cli if needed)'---{/* SOURCE_FILE: examples/openai/multi_tool_orchestration.ipynb */}_View Notebook on <a href={'https://github.com/AgentOps-AI/agentops/blob/main/examples/openai/multi_tool_orchestration.ipynb'} target={'_blank'}>Github</a>_-# Multi-Tool Orchestration with RAG approach using OpenAI+### Multi-Tool Orchestration with RAG approach using OpenAI's Responses API+++This cookbook guides you through building dynamic, multi-tool workflows using OpenAI's Responses API. It demonstrates how to implement a Retrieval-Augmented Generation (RAG) approach that intelligently routes user queries to the appropriate in-built or external tools. Whether your query calls for general knowledge or requires accessing specific internal context from a vector database (like Pinecone), this guide shows you how to integrate function calls, web searches in-built tool, and leverage document retrieval to generate accurate, context-aware responses.++-This cookbook guides you through building dynamic, multi-tool workflows using OpenAI's APIs. It demonstrates how to implement a Retrieval-Augmented Generation (RAG) approach that intelligently routes user queries to the appropriate in-built or external tools. Whether your query calls for general knowledge or requires accessing specific internal context from a vector database (like Pinecone), this guide shows you how to integrate function calls, simulate web searches, and leverage document retrieval to generate accurate, context-aware responses.## Installation-Install required dependencies:<CodeGroup>```bash pip- pip install agentops openai datasets tqdm pandas pinecone-client python-dotenv+ pip install agentops datasets openai pandas pinecone python-dotenv tqdm``````bash poetry- poetry add agentops openai datasets tqdm pandas pinecone-client python-dotenv+ poetry add agentops datasets openai pandas pinecone python-dotenv tqdm``````bash uv- uv add agentops openai datasets tqdm pandas pinecone-client python-dotenv+ uv add agentops datasets openai pandas pinecone python-dotenv tqdm```</CodeGroup>-## Setup```pythonimport osfrom dotenv import load_dotenvimport time-from tqdm.auto import tqdm # tqdm.auto for notebook compatibility+from tqdm.auto import tqdmfrom pandas import DataFramefrom datasets import load_datasetimport randomimport stringfrom openai import OpenAIimport agentops-from pinecone import Pinecone # Pinecone class for client+from pinecone import Pineconefrom pinecone import ServerlessSpec-import json # For handling tool arguments```-Set API Keys:+```pythonload_dotenv()-os.environ["AGENTOPS_API_KEY"] = os.getenv("AGENTOPS_API_KEY", "your_agentops_api_key_here")+os.environ["AGENTOPS_API_KEY"] = os.getenv("AGENTOPS_API_KEY", "your_api_key_here")os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY", "your_openai_api_key_here")os.environ["PINECONE_API_KEY"] = os.getenv("PINECONE_API_KEY", "your_pinecone_api_key_here")```-Initialize AgentOps and OpenAI client:+```python-agentops.init(auto_start_session=True)-tracer = agentops.start_trace(trace_name="Multi-Tool Orchestration with RAG", tags=["multi-tool-orchestration-rag-demo", "openai-responses", "agentops-example"])+agentops.init(auto_start_session=True)+tracer = agentops.start_trace(+ trace_name="Multi-Tool Orchestration with RAG",+ tags=["multi-tool-orchestration-rag-demo", "openai-responses", "agentops-example"],+)client = OpenAI()```In this example we use a sample medical reasoning dataset from Hugging Face. We convert the dataset into a Pandas DataFrame and merge the “Question” and “Response” columns into a single string. This merged text is used for embedding and later stored as metadata.++```python# Load the dataset (ensure you're logged in with huggingface-cli if needed)ds = load_dataset("FreedomIntelligence/medical-o1-reasoning-SFT", "en", split="train[:100]", trust_remote_code=True)@@ -69,13 +75,16 @@ ds_dataframe["merged"] = ds_dataframe.apply()print("Example merged text:", ds_dataframe["merged"].iloc[0])```++```python-# To display the dataframe in a notebook-like fashion (optional in MDX)-# ds_dataframe+ds_dataframe```### Create a Pinecone Index Based on the DatasetUse the dataset itself to determine the embedding dimensionality. For example, compute one embedding from the merged column and then create the index accordingly.++```pythonMODEL = "text-embedding-3-small" # Replace with your production embedding model if needed# Compute an embedding for the first document to obtain the embedding dimension.@@ -83,51 +92,51 @@ sample_embedding_resp = client.embeddings.create(input=[ds_dataframe["merged"].iembed_dim = len(sample_embedding_resp.data[0].embedding)print(f"Embedding dimension: {embed_dim}")```++```python# Initialize Pinecone using your API key.pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])# Define the Pinecone serverless specification.-AWS_REGION = "us-east-1" # As in notebook+AWS_REGION = "us-east-1"spec = ServerlessSpec(cloud="aws", region=AWS_REGION)# Create a random index name with lower case alphanumeric characters and '-'index_name = "pinecone-index-" + "".join(random.choices(string.ascii_lowercase + string.digits, k=10))# Create the index if it doesn't already exist.-if index_name not in pc.list_indexes().names:+if index_name not in pc.list_indexes().names():pc.create_index(index_name, dimension=embed_dim, metric="dotproduct", spec=spec)- # Wait for index to be ready (important)- while not pc.describe_index(index_name).status['ready']:- time.sleep(1)# Connect to the index.index = pc.Index(index_name)-# time.sleep(1) # Notebook has this, usually not needed if create_index waits.+time.sleep(1)print("Index stats:", index.describe_index_stats())```#### Upsert the Dataset into Pinecone index-Process the dataset in batches, generate embeddings for each merged text, prepare metadata (including separate Question and Answer fields), and upsert each batch into the index.+Process the dataset in batches, generate embeddings for each merged text, prepare metadata (including separate Question and Answer fields), and upsert each batch into the index. You may also update metadata for specific entries if needed.++```pythonbatch_size = 32for i in tqdm(range(0, len(ds_dataframe["merged"]), batch_size), desc="Upserting to Pinecone"):i_end = min(i + batch_size, len(ds_dataframe["merged"]))- lines_batch = ds_dataframe["merged"][i:i_end].tolist() # Ensure it's a list of strings+ lines_batch = ds_dataframe["merged"][i:i_end]ids_batch = [str(n) for n in range(i, i_end)]# Create embeddings for the current batch.- res = client.embeddings.create(input=lines_batch, model=MODEL) # Pass list directly+ res = client.embeddings.create(input=[line for line in lines_batch], model=MODEL)embeds = [record.embedding for record in res.data]# Prepare metadata by extracting original Question and Answer.meta = []- # Notebook uses: for record in ds_dataframe.iloc[i:i_end].to_dict("records"):- for record_idx in range(i, i_end):- original_record = ds_dataframe.iloc[record_idx]- q_text = original_record["Question"]- a_text = original_record["Response"]+ for record in ds_dataframe.iloc[i:i_end].to_dict("records"):+ q_text = record["Question"]+ a_text = record["Response"]+ # Optionally update metadata for specific entries.meta.append({"Question": q_text, "Answer": a_text})# Upsert the batch into Pinecone.@@ -138,17 +147,15 @@ for i in tqdm(range(0, len(ds_dataframe["merged"]), batch_size), desc="Upserting### Query the Pinecone IndexCreate a natural language query, compute its embedding, and perform a similarity search on the Pinecone index. The returned results include metadata that provides context for generating answers.++```pythondef query_pinecone_index(client, index, model, query_text):# Generate an embedding for the query.- # Notebook: client.embeddings.create(input=query_text, ...)- # Corrected: input should be a list- query_embedding = client.embeddings.create(input=[query_text], model=model).data[0].embedding+ query_embedding = client.embeddings.create(input=query_text, model=model).data[0].embedding# Query the index and return top 5 matches.- # Notebook: index.query(vector=[query_embedding], ...)- # Corrected: vector should be the embedding itself if it's already a list/vector- res = index.query(vector=query_embedding, top_k=5, include_metadata=True)+ res = index.query(vector=[query_embedding], top_k=5, include_metadata=True)print("Query Results:")for match in res["matches"]:print(@@ -156,6 +163,8 @@ def query_pinecone_index(client, index, model, query_text):)return res```++```python# Example usage with a different query from the train/test setquery = (@@ -167,71 +176,66 @@ query_pinecone_index(client, index, MODEL, query)### Generate a Response Using the Retrieved Context-Select the best matching result from your query results and use the OpenAI Chat Completions API to generate a final answer by combining the retrieved context with the original question.+Select the best matching result from your query results and use the OpenAI Responses API to generate a final answer by combining the retrieved context with the original question.++```python# Retrieve and concatenate top 3 match contexts.-# Notebook: client.embeddings.create(input=query, ...)-# Corrected: input should be a list-query_embedding_for_rag = client.embeddings.create(input=[query], model=MODEL).data[0].embeddingmatches = index.query(- vector=query_embedding_for_rag, top_k=3, include_metadata=True # vector is embedding itself+ vector=[client.embeddings.create(input=query, model=MODEL).data[0].embedding], top_k=3, include_metadata=True)["matches"]context = "\n\n".join(f"Question: {m['metadata'].get('Question', '')}\nAnswer: {m['metadata'].get('Answer', '')}" for m in matches)--# Use the context to generate a final answer using Chat Completions.-response_payload = {- "model": "gpt-4o",- "messages": [- {"role": "system", "content": "You are a helpful medical assistant. Answer based on the provided context."},- {"role": "user", "content": f"Provide the answer based on the context: {context} and the question: {query} as per the internal knowledge base"}- ]-}-completion = client.chat.completions.create(**response_payload)-final_answer_text = completion.choices[0].message.content-print("\nFinal Answer (using Chat Completions):")-print(final_answer_text)+# Use the context to generate a final answer.+response = client.responses.create(+ model="gpt-4o",+ input=f"Provide the answer based on the context: {context} and the question: {query} as per the internal knowledge base",+)+print("\nFinal Answer:")+print(response.output_text)```### Orchestrate Multi-Tool Calls-Now, we'll define tools. One simulates web search, and the other queries our Pinecone vector store.+Now, we'll define the built-in function available through the Responses API, including the ability to invoke the external Vector Store - Pinecone as an example.++*Web Search Preview Tool*: Enables the model to perform live web searches and preview the results. This is ideal for retrieving real-time or up-to-date information from the internet.++*Pinecone Search Tool*: Allows the model to query a vector database using semantic search. This is especially useful for retrieving relevant documents—such as medical literature or other domain-specific content—that have been stored in a vectorized format.++```python+# Tools definition: The list of tools includes:+# - A web search preview tool.+# - A Pinecone search tool for retrieving medical documents.+# Define available tools.tools = [{- "type": "function", # Standard type for custom functions- "function": {- "name": "web_search_preview_tool", # Giving it a function-like name- "description": "Performs a web search and returns a preview of results. Use for real-time or up-to-date information.",- "parameters": { # Even simulated tools can define expected parameters- "type": "object",- "properties": {- "search_query": {"type": "string", "description": "The query to search the web for."}- },- "required": ["search_query"]- }- }+ "type": "web_search_preview",+ "user_location": {"type": "approximate", "country": "US", "region": "California", "city": "SF"},+ "search_context_size": "medium",},{"type": "function",- "function": {- "name": "PineconeSearchDocuments",- "description": "Search for relevant documents based on the medical question asked by the user that is stored within the vector database using a semantic query.",- "parameters": {- "type": "object",- "properties": {- "query": {"type": "string", "description": "The natural language query to search the vector database."},- "top_k": {"type": "integer", "description": "Number of top results to return.", "default": 3},- },- "required": ["query"],- }- }+ "name": "PineconeSearchDocuments",+ "description": "Search for relevant documents based on the medical question asked by the user that is stored within the vector database using a semantic query.",+ "parameters": {+ "type": "object",+ "properties": {+ "query": {"type": "string", "description": "The natural language query to search the vector database."},+ "top_k": {"type": "integer", "description": "Number of top results to return.", "default": 3},+ },+ "required": ["query"],+ "additionalProperties": False,+ },},]```++```python# Example queries that the model should route appropriately.queries = [@@ -247,191 +251,186 @@ queries = [},]```++```python-# Process each query dynamically using Chat Completions for tool routing.+# Process each query dynamically.for item in queries:- current_messages = [{"role": "user", "content": item["query"]}]+ input_messages = [{"role": "user", "content": item["query"]}]print("\n🌟--- Processing Query ---🌟")print(f"🔍 **User Query:** {item['query']}")- # First call to LLM to decide on tool usage- response = client.chat.completions.create(+ # Call the Responses API with tools enabled and allow parallel tool calls.+ response = client.responses.create(model="gpt-4o",- messages=[+ input=[{"role": "system",- "content": "When prompted with a question, select the right tool to use based on the question. If a web search is needed, use 'web_search_preview_tool'. If internal knowledge base search is needed, use 'PineconeSearchDocuments'.",+ "content": "When prompted with a question, select the right tool to use based on the question.",},{"role": "user", "content": item["query"]},],tools=tools,- tool_choice="auto",+ parallel_tool_calls=True,)- response_message = response.choices[0].message- tool_calls = response_message.tool_calls-- current_messages.append(response_message) # Add assistant's decision to message history-- print("\n✨ **Initial LLM Response:**")- if response_message.content:- print(response_message.content)-- if tool_calls:- print(f"🔧 **Model triggered tool call(s):**")-- for tool_call in tool_calls:- function_name = tool_call.function.name- function_args = json.loads(tool_call.function.arguments)-- print(f" - Tool: {function_name}, Args: {function_args}")- function_response_content = ""-- if function_name == "PineconeSearchDocuments":+ print("\n✨ **Initial Response Output:**")+ print(response.output)++ # Determine if a tool call is needed and process accordingly.+ if response.output:+ tool_call = response.output[0]+ if tool_call.type in ["web_search_preview", "function_call"]:+ tool_name = tool_call.name if tool_call.type == "function_call" else "web_search_preview"+ print(f"\n🔧 **Model triggered a tool call:** {tool_name}")++ if tool_name == "PineconeSearchDocuments":print("🔍 **Invoking PineconeSearchDocuments tool...**")- pinecone_query = function_args.get("query")- top_k_results = function_args.get("top_k", 3)- # query_pinecone_index returns the full response object- pinecone_response_obj = query_pinecone_index(client, index, MODEL, pinecone_query)-- matches_str_list = []- if pinecone_response_obj["matches"]:- for match in pinecone_response_obj["matches"][:top_k_results]:- matches_str_list.append(f"Score: {match['score']:.2f}, Q: {match['metadata'].get('Question', 'N/A')}, A: {match['metadata'].get('Answer', 'N/A')}")- function_response_content = "\n".join(matches_str_list) if matches_str_list else "No matching documents found."+ res = query_pinecone_index(client, index, MODEL, item["query"])+ if res["matches"]:+ best_match = res["matches"][0]["metadata"]+ result = f"**Question:** {best_match.get('Question', 'N/A')}\n**Answer:** {best_match.get('Answer', 'N/A')}"+ else:+ result = "**No matching documents found in the index.**"print("✅ **PineconeSearchDocuments tool invoked successfully.**")-- elif function_name == "web_search_preview_tool":- print("🔍 **Simulating web_search_preview_tool...**")- search_query_arg = function_args.get("search_query", item['query'])- function_response_content = f"Simulated web search result for: {search_query_arg}"- print("✅ **Simulated web_search_preview_tool invoked successfully.**")else:- print(f"⚠️ Unknown tool: {function_name}")- function_response_content = "Error: Unknown tool."-- current_messages.append(- {- "tool_call_id": tool_call.id,- "role": "tool",- "name": function_name,- "content": function_response_content,- }+ print("🔍 **Invoking simulated web search tool...**")+ result = "**Simulated web search result.**"+ print("✅ **Simulated web search tool invoked successfully.**")++ # Append the tool call and its output back into the conversation.+ input_messages.append(tool_call)+ input_messages.append({"type": "function_call_output", "call_id": tool_call.call_id, "output": str(result)})++ # Get the final answer incorporating the tool's result.+ final_response = client.responses.create(+ model="gpt-4o", input=input_messages, tools=tools, parallel_tool_calls=True)-- # Second call to LLM with tool results- print("\n🔄 **Getting final response from LLM...**")- final_response = client.chat.completions.create(- model="gpt-4o",- messages=current_messages- )- print("💡 **Final Answer:**")- print(final_response.choices[0].message.content)- elif response_message.content:- # If no tool call is triggered, print the response directly.- print("💡 **Final Answer (no tool call):**")- print(response_message.content)- else:- print("⚠️ No content or tool call in response.")+ print("\n💡 **Final Answer:**")+ print(final_response.output_text)+ else:+ # If no tool call is triggered, print the response directly.+ print("💡 **Final Answer:**")+ print(response.output_text)```As shown above, depending on the query, appropriate tool is invoked in order to determine the optimal response.-For instance, looking at the third example, when the model triggers the tool named "PineconeSearchDocuments", the code calls `query_pinecone_index` with the current query and then extracts the best match (or an appropriate context) as the result. For non health related inquiries or queries where explicit internet search is asked, the code calls the (simulated) `web_search_preview_tool` function and for other queries, it may choose to not call any tool and rather provide a response based on the question under consideration.+For instance, looking at the third example, when the model triggers the tool named "PineconeSearchDocuments", the code calls `query_pinecone_index` with the current query and then extracts the best match (or an appropriate context) as the result. For non health related inqueries or queries where explicit internet search is asked, the code calls the web_search_call function and for other queries, it may choose to not call any tool and rather provide a response based on the question under consideration.++Finally, the tool call and its output are appended to the conversation, and the final answer is generated by the Responses API.-Finally, the tool call and its output are appended to the conversation, and the final answer is generated by the Chat Completions API.+### Multi-tool orchestration flow++Now let us try to modify the input query and the system instructions to the responses API in order to follow a tool calling sequence and generate the output.-### Multi-tool orchestration flow (Sequential Example)-Now let us try to modify the input query and the system instructions to the Chat Completions API in order to follow a tool calling sequence and generate the output.```python# Process one query as an example to understand the tool calls and function calls as part of the response output-item_query_seq = "What is the most common cause of death in the United States"+item = "What is the most common cause of death in the United States"# Initialize input messages with the user's query.-current_messages_seq = [{"role": "user", "content": item_query_seq}]-print("\n🌟--- Processing Query (Sequential Tool Call Example) ---🌟")-print(f"🔍 **User Query:** {item_query_seq}")+input_messages = [{"role": "user", "content": item}]+print("\n🌟--- Processing Query ---🌟")+print(f"🔍 **User Query:** {item}")-# Call the Chat Completions API with tools enabled-print("\n🔧 **Calling Chat Completions API with Tools Enabled**")-print("\n🕵️♂️ **Step 1: Web Search Call (Implicit)**")+# Call the Responses API with tools enabled and allow parallel tool calls.+print("\n🔧 **Calling Responses API with Tools Enabled**")+print("\n🕵️♂️ **Step 1: Web Search Call**")print(" - Initiating web search to gather initial information.")-print("\n📚 **Step 2: Pinecone Search Call (Implicit)**")+print("\n📚 **Step 2: Pinecone Search Call**")print(" - Querying Pinecone to find relevant examples from the internal knowledge base.")-# First LLM call to determine initial tool usage-response_seq_1 = client.chat.completions.create(+response = client.responses.create(model="gpt-4o",- messages=[+ input=[{"role": "system",- "content": "First, call the web_search_preview_tool for results. Then, using that information, call `PineconeSearchDocuments` to find real examples in the internal knowledge base. Finally, synthesize an answer.",+ "content": "Every time it's prompted with a question, first call the web search tool for results, then call `PineconeSearchDocuments` to find real examples in the internal knowledge base.",},- {"role": "user", "content": item_query_seq},+ {"role": "user", "content": item},],tools=tools,- tool_choice="auto", # Let model decide, or force a specific tool if needed for strict sequence+ parallel_tool_calls=True,)-print("\n✨ **Initial Response Output (after first LLM call):**")-response_message_seq_1 = response_seq_1.choices[0].message-tool_calls_seq_1 = response_message_seq_1.tool_calls--if response_message_seq_1.content:- print(f"Assistant Content: {response_message_seq_1.content}")-current_messages_seq.append(response_message_seq_1)--if tool_calls_seq_1:- print("🔧 **Tool calls from first LLM response:**")- for tc in tool_calls_seq_1:- print(f" - ID: {tc.id}, Function: {tc.function.name}, Args: {tc.function.arguments}")- tool_function_args = json.loads(tc.function.arguments)- tool_response_content = ""-- if tc.function.name == "web_search_preview_tool":- simulated_web_search_output = f"Simulated web search result for query: {tool_function_args.get('search_query', item_query_seq)}"- tool_response_content = simulated_web_search_output- print(f" ✔️ Simulated output for {tc.function.name}: {tool_response_content}")- elif tc.function.name == "PineconeSearchDocuments":- pinecone_query_text = tool_function_args.get("query", item_query_seq)- pinecone_res_obj = query_pinecone_index(client, index, MODEL, pinecone_query_text)- matches_str_list_seq = []- if pinecone_res_obj["matches"]:- for match in pinecone_res_obj["matches"][:tool_function_args.get("top_k", 3)]:- matches_str_list_seq.append(f"Score: {match['score']:.2f}, Q: {match['metadata'].get('Question', 'N/A')}, A: {match['metadata'].get('Answer', 'N/A')}")- tool_response_content = "\n".join(matches_str_list_seq) if matches_str_list_seq else "No matching documents found in Pinecone."- print(f" ✔️ Output for {tc.function.name}: {tool_response_content[:100]}...")-- current_messages_seq.append({- "tool_call_id": tc.id,- "role": "tool",- "name": tc.function.name,- "content": tool_response_content,- })-- # Second call to LLM with tool results- print("\n🔄 **Calling Chat Completions API again with tool results...**")- response_seq_2 = client.chat.completions.create(- model="gpt-4o",- messages=current_messages_seq,- tools=tools # Make tools available again if model decides to call another one+# Print the initial response output.+print("input_messages", input_messages)++print("\n✨ **Initial Response Output:**")+print(response.output)+```+++```python+# Understand the tool calls and function calls as part of the response output++import pandas as pd++# Create a list to store the tool call and function call details+tool_calls = []++# Iterate through the response output and collect the details+for i in response.output:+ tool_calls.append(+ {+ "Type": i.type,+ "Call ID": i.call_id if hasattr(i, "call_id") else i.id if hasattr(i, "id") else "N/A",+ "Output": str(i.output) if hasattr(i, "output") else "N/A",+ "Name": i.name if hasattr(i, "name") else "N/A",+ })- final_answer_seq = response_seq_2.choices[0].message.content- print("\n💡 **Final Answer (after sequential tool calls):**")- print(final_answer_seq)-else:- print("⚠️ No tool calls in the first response. Final answer is the initial content.")- if response_message_seq_1.content:- print(response_message_seq_1.content)+# Convert the list to a DataFrame for tabular display+df_tool_calls = pd.DataFrame(tool_calls)-agentops.end_trace(tracer, end_state="Success") # End the main trace+# Display the DataFrame+df_tool_calls```-Here, we have seen how to utilize OpenAI's Chat Completions API to implement a Retrieval-Augmented Generation (RAG) approach with multi-tool calling capabilities. It showcases an example where the model selects the appropriate tool based on the input query: general questions may be handled by built-in tools such as web-search (simulated here), while specific medical inquiries related to internal knowledge are addressed by retrieving context from a vector database (such as Pinecone) via function calls. Additionally, we have showcased how multiple tool calls can be conceptually chained to generate a final response based on our instructions. Happy coding!++```python+tool_call_1 = response.output[0]+print(tool_call_1)+print(tool_call_1.id)++tool_call_2 = response.output[2]+print(tool_call_2)+print(tool_call_2.call_id)+```+++```python+# append the tool call and its output back into the conversation.+input_messages.append(response.output[2])+input_messages.append({"type": "function_call_output", "call_id": tool_call_2.call_id, "output": str(result)})+print(input_messages)+```+++```python+# Get the final answer incorporating the tool's result.+print("\n🔧 **Calling Responses API for Final Answer**")++response_2 = client.responses.create(+ model="gpt-4o",+ input=input_messages,+)+print(response_2)+```+++```python+# print the final answer+print(response_2.output_text)+agentops.end_trace(tracer, end_state="Success")+```+++Here, we have seen how to utilize OpenAI's Responses API to implement a Retrieval-Augmented Generation (RAG) approach with multi-tool calling capabilities. It showcases an example where the model selects the appropriate tool based on the input query: general questions may be handled by built-in tools such as web-search, while specific medical inquiries related to internal knowledge are addressed by retrieving context from a vector database (such as Pinecone) via function calls. Additonally, we have showcased how multiple tool calls can be sequentially combined to generate a final response based on our instructions provided to responses API. Happy coding!+<script type="module" src="/scripts/github_stars.js"></script><script type="module" src="/scripts/scroll-img-fadein-animation.js"></script><script type="module" src="/scripts/button_heartbeat_animation.js"></script>-<script type="module" src="/scripts/adjust_api_dynamically.js"></script>+<script type="module" src="/scripts/adjust_api_dynamically.js"></script>\ No newline at end of filediff --git a/docs/v2/examples/openai_agents.mdx b/docs/v2/examples/openai_agents.mdxindex 760671fe..0d955a14 100644--- a/docs/v2/examples/openai_agents.mdx+++ b/docs/v2/examples/openai_agents.mdx@@ -1,5 +1,5 @@----title: 'OpenAI Agents SDK'+title: 'OpenAI Agents'description: 'OpenAI Agents Patterns Demonstration'---{/* SOURCE_FILE: examples/openai_agents/agent_patterns.ipynb */}@@ -813,4 +813,4 @@ Visit [app.agentops.ai](https://app.agentops.ai) to explore your agent sessions<script type="module" src="/scripts/github_stars.js"></script><script type="module" src="/scripts/scroll-img-fadein-animation.js"></script><script type="module" src="/scripts/button_heartbeat_animation.js"></script>-<script type="module" src="/scripts/adjust_api_dynamically.js"></script>+<script type="module" src="/scripts/adjust_api_dynamically.js"></script>\ No newline at end of filediff --git a/docs/v2/examples/smolagents.mdx b/docs/v2/examples/smolagents.mdxindex 3ab4c3a6..f82f885d 100644--- a/docs/v2/examples/smolagents.mdx+++ b/docs/v2/examples/smolagents.mdx@@ -59,6 +59,7 @@ import reimport requestsfrom markdownify import markdownifyfrom requests.exceptions import RequestException+from smolagents import LiteLLMModel, tool, CodeAgent, ToolCallingAgent, DuckDuckGoSearchToolload_dotenv()os.environ["AGENTOPS_API_KEY"] = os.getenv("AGENTOPS_API_KEY", "your_api_key_here")diff --git a/docs/v2/examples/xai.mdx b/docs/v2/examples/xai.mdxindex 481e0b7c..3e690131 100644--- a/docs/v2/examples/xai.mdx+++ b/docs/v2/examples/xai.mdx@@ -1,33 +1,36 @@----title: 'XAI Grok Example'-description: 'Using XAI Grok via OpenAI client with AgentOps for a transliteration chatbot'+title: 'xAI'+description: 'XAI Example'---{/* SOURCE_FILE: examples/xai/grok_examples.ipynb */}_View Notebook on <a href={'https://github.com/AgentOps-AI/agentops/blob/main/examples/xai/grok_examples.ipynb'} target={'_blank'}>Github</a>_-# XAI Grok Transliteration Chatbot Example+# XAI Example+This notebook demonstrates how to use XAI with AgentOps via the OpenAI python client.++We are going to use the latest Grok model from XAI to create a transliteration chatbot that can understand the major languages of the world and translate them to a user's native language! We will use AgentOps to track the chatbot's performance.++First let's install the required packages-This notebook demonstrates how to use XAI with AgentOps via the OpenAI python client.-We are going to use a Grok model from XAI to create a transliteration chatbot that can understand the major languages of the world and translate them to a user's native language! We will use AgentOps to track the chatbot's performance.## Installation-First let's install the required packages.<CodeGroup>```bash pip- pip install -U openai agentops python-dotenv+ pip install -U agentops openai``````bash poetry- poetry add openai agentops python-dotenv+ poetry add -U agentops openai``````bash uv- uv add openai agentops python-dotenv+ uv add -U agentops openai```</CodeGroup>-## Setup-Then import them.+Then import them++```pythonfrom openai import OpenAIimport agentops@@ -35,31 +38,37 @@ import osfrom dotenv import load_dotenv```-Next, we'll grab our API keys. You can use dotenv like below or however else you like to load environment variables.+Next, we'll grab our API keys. You can use dotenv like below or however else you like to load environment variables++```pythonload_dotenv()-os.environ["AGENTOPS_API_KEY"] = os.getenv("AGENTOPS_API_KEY", "your_agentops_api_key_here")-os.environ["XAI_API_KEY"] = os.getenv("XAI_API_KEY", "your_xai_api_key_here") # Ensure XAI_API_KEY is set+os.environ["AGENTOPS_API_KEY"] = os.getenv("AGENTOPS_API_KEY", "your_api_key_here")```Next we initialize the AgentOps client.++```pythonagentops.init(auto_start_session=False)tracer = agentops.start_trace(trace_name="XAI Example", tags=["xai-example", "grok", "agentops-example"])```-And we are all set! Note the session URL that will be printed by AgentOps. We will use it to track the chatbot.+And we are all set! Note the session URL above. We will use it to track the chatbot.++Let's initialize the OpenAI client with the XAI API key and base url.+-Let's initialize the OpenAI client with the XAI API key and base URL.```pythonclient = OpenAI(base_url="https://api.x.ai/v1",- api_key=os.getenv("XAI_API_KEY", "your_xai_api_key_here"), # Added fallback as in notebook+ api_key=os.getenv("XAI_API_KEY", "your_xai_api_key_here"),)```-## Define PromptsNow we will set the system and instruction prompts for the chatbot. We will set the native language to Spanish and the user prompt to transliterate an excerpt from Haruki Murakami's "Kafka On The Shore".++```pythonSYSTEM_PROMPT = """You are a highly intelligent, multilingual assistant designed to understand user prompts in English and respond in the user's specified native language.@@ -73,6 +82,8 @@ When asked to respond in a language, generate the response entirely in that langIf the specified language is unfamiliar or ambiguous, politely ask for clarification in English."""```++```pythonnative_language = "Spanish"@@ -98,28 +109,34 @@ Ensure that the response is in Unicode characters appropriate for {native_langua"""```-## Generate ResponseNow we will use the OpenAI client to generate the response by passing in the system and instruction prompts.++```pythonresponse = client.chat.completions.create(- model="grok-1.5-flash", # Notebook used "grok-3-mini". Standardizing to a common one. "grok-1.5-flash" is often cited.+ model="grok-3-mini",messages=[{"role": "system", "content": SYSTEM_PROMPT}, {"role": "user", "content": INSTRUCTION_PROMPT}],)```++```python-print(f"Original Prompt (for transliteration):\n{USER_PROMPT}")+print(f"Original Prompt:\n{USER_PROMPT}")generated_response = response.choices[0].message.contentprint(f"Response in {native_language}:\n{generated_response}")```-Awesome! We can now transliterate from English to any language! And all of this can be tracked with AgentOps by going to the session url printed when `agentops.init()` was called.+Awesome! We can now transliterate from English to any language! And all of this can be tracked with AgentOps by going to the session url above.+```pythonagentops.end_trace(tracer, end_state="Success")```-We end the session with a success state. This is useful if you want to track the success or failure of the chatbot. By default, the session will have an indeterminate end state if not explicitly ended.++We end the session with a success state and a success reason. This is useful if you want to track the success or failure of the chatbot. In that case you can set the end state to failure and provide a reason. By default the session will have an indeterminate end state.+<script type="module" src="/scripts/github_stars.js"></script><script type="module" src="/scripts/scroll-img-fadein-animation.js"></script><script type="module" src="/scripts/button_heartbeat_animation.js"></script>-<script type="module" src="/scripts/adjust_api_dynamically.js"></script>+<script type="module" src="/scripts/adjust_api_dynamically.js"></script>\ No newline at end of filediff --git a/examples/autogen/AgentChat.ipynb b/examples/autogen/AgentChat.ipynbindex 5036a776..ef81f224 100644--- a/examples/autogen/AgentChat.ipynb+++ b/examples/autogen/AgentChat.ipynb@@ -198,7 +198,7 @@"id": "7b422137-903a-41ef-a4ca-95b50aea4138","metadata": {},"source": [- "You can view data on this run at [app.agentops.ai](app.agentops.ai).\n",+ "You can view data on this run at [app.agentops.ai](https://app.agentops.ai).\n","\n","The dashboard will display LLM events for each message sent by each agent, including those made by the human user."]diff --git a/examples/smolagents/multi_smolagents_system.ipynb b/examples/smolagents/multi_smolagents_system.ipynbindex f8704619..666edb8b 100644--- a/examples/smolagents/multi_smolagents_system.ipynb+++ b/examples/smolagents/multi_smolagents_system.ipynb@@ -74,6 +74,7 @@"import requests\n","from markdownify import markdownify\n","from requests.exceptions import RequestException\n",+ "from smolagents import LiteLLMModel, tool, CodeAgent, ToolCallingAgent, DuckDuckGoSearchTool\n","\n","load_dotenv()\n","os.environ[\"AGENTOPS_API_KEY\"] = os.getenv(\"AGENTOPS_API_KEY\", \"your_api_key_here\")\n",diff --git a/examples/xai/grok_examples.ipynb b/examples/xai/grok_examples.ipynbindex 513a31d5..8dfdaff2 100755--- a/examples/xai/grok_examples.ipynb+++ b/examples/xai/grok_examples.ipynb@@ -84,7 +84,7 @@"cell_type": "markdown","metadata": {},"source": [- "And we are all set! Note the seesion url above. We will use it to track the chatbot.\n",+ "And we are all set! Note the session URL above. We will use it to track the chatbot.\n","\n","Let's initialize the OpenAI client with the XAI API key and base url."]@@ -182,7 +182,7 @@"metadata": {},"outputs": [],"source": [- "print(f\"Original Response:\\n{USER_PROMPT}\")\n",+ "print(f\"Original Prompt:\\n{USER_PROMPT}\")\n","generated_response = response.choices[0].message.content\n","print(f\"Response in {native_language}:\\n{generated_response}\")"]