Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI
Python agentagentopsagents-sdkaianthropicautogencost-estimationcrewaievalsevaluation-metricsgroqlangchainllmmistralollamaopenaiopenai-agents
---
title: LiteLLM
description: "Track and analyze your LiteLLM calls across multiple providers with AgentOps"
---
AgentOps provides seamless integration with [LiteLLM](https://github.com/BerriAI/litellm), allowing you to automatically track all your LLM API calls across different providers through a unified interface.
## Installation
<CodeGroup>
```bash pip
pip install agentops litellm
```
```bash poetry
poetry add agentops litellm
```
```bash uv
uv pip install agentops litellm
```
</CodeGroup>
## Setting Up API Keys
Before using LiteLLM with AgentOps, you need to set up your API keys. You can obtain:
- **Provider API Keys**: From your chosen LLM provider (OpenAI, Anthropic, Google, etc.)
- **AGENTOPS_API_KEY**: From your [AgentOps Dashboard](https://app.agentops.ai/)
Then to set them up, you can either export them as environment variables or set them in a `.env` file.
<CodeGroup>
```bash Export to CLI
export OPENAI_API_KEY="your_openai_api_key_here"
export ANTHROPIC_API_KEY="your_anthropic_api_key_here"
export AGENTOPS_API_KEY="your_agentops_api_key_here"
```
```txt Set in .env file
OPENAI_API_KEY="your_openai_api_key_here"
ANTHROPIC_API_KEY="your_anthropic_api_key_here"
AGENTOPS_API_KEY="your_agentops_api_key_here"
```
</CodeGroup>
Then load the environment variables in your Python code:
```python
from dotenv import load_dotenv
import os
# Load environment variables from .env file
load_dotenv()
# Set up environment variables with fallback values
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY")
os.environ["ANTHROPIC_API_KEY"] = os.getenv("ANTHROPIC_API_KEY")
os.environ["AGENTOPS_API_KEY"] = os.getenv("AGENTOPS_API_KEY")
```
## Usage
The simplest way to integrate AgentOps with LiteLLM is to set up the success_callback.
```python
import litellm
from litellm import completion
# Configure LiteLLM to use AgentOps
litellm.success_callback = ["agentops"]
# Make completion requests with LiteLLM
response = completion(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": "Hello, how are you?"}]
)
print(response.choices[0].message.content)
```
## Examples
<CodeGroup>
```python Streaming
import litellm
from litellm import completion
# Configure LiteLLM to use AgentOps
litellm.success_callback = ["agentops"]
# Make a streaming completion request
response = completion(
model="gpt-4",
messages=[{"role": "user", "content": "Write a short poem about AI."}],
stream=True
)
# Process the streaming response
for chunk in response:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)
print() # Add a newline at the end
```
```python Multi-Provider
import litellm
from litellm import completion
# Configure LiteLLM to use AgentOps
litellm.success_callback = ["agentops"]
# OpenAI request
openai_response = completion(
model="gpt-4",
messages=[{"role": "user", "content": "What are the advantages of GPT-4?"}]
)
print("OpenAI Response:", openai_response.choices[0].message.content)
# Anthropic request using the same interface
anthropic_response = completion(
model="anthropic/claude-3-opus-20240229",
messages=[{"role": "user", "content": "What are the advantages of Claude?"}]
)
print("Anthropic Response:", anthropic_response.choices[0].message.content)
# All requests across different providers are automatically tracked by AgentOps
```
</CodeGroup>
## More Examples
<CardGroup cols={2}>
<Card title="LiteLLM Quickstart Notebook" icon="notebook" href="/v2/examples/litellm" />
</CardGroup>
For more information on integrating AgentOps with LiteLLM, refer to the [LiteLLM documentation on AgentOps integration](https://docs.litellm.ai/docs/observability/agentops_integration).
<script type="module" src="/scripts/github_stars.js"></script>
<script type="module" src="/scripts/scroll-img-fadein-animation.js"></script>
<script type="module" src="/scripts/button_heartbeat_animation.js"></script>
<script type="css" src="/styles/styles.css"></script>