Skip to content

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI

Docs overhaul - Introduction+Quickstart (#221)

Howard Gil27d8311ccabb
diff --git a/docs/images/link-to-session.gif b/docs/images/link-to-session.gif
new file mode 100644
index 00000000..f376bfa2
Binary files /dev/null and b/docs/images/link-to-session.gif differ
diff --git a/docs/images/overview.gif b/docs/images/overview.gif
new file mode 100644
index 00000000..7927cd5a
Binary files /dev/null and b/docs/images/overview.gif differ
diff --git a/docs/images/overview.png b/docs/images/overview.png
new file mode 100644
index 00000000..640ac2fa
Binary files /dev/null and b/docs/images/overview.png differ
diff --git a/docs/images/session-drawer.gif b/docs/images/session-drawer.gif
new file mode 100644
index 00000000..4912e9c7
Binary files /dev/null and b/docs/images/session-drawer.gif differ
diff --git a/docs/images/session-summary.gif b/docs/images/session-summary.gif
new file mode 100644
index 00000000..57dc0a0f
Binary files /dev/null and b/docs/images/session-summary.gif differ
diff --git a/docs/images/session-summary.png b/docs/images/session-summary.png
new file mode 100644
index 00000000..76094ffd
Binary files /dev/null and b/docs/images/session-summary.png differ
diff --git a/docs/images/session-waterfall.gif b/docs/images/session-waterfall.gif
new file mode 100644
index 00000000..bcef05d8
Binary files /dev/null and b/docs/images/session-waterfall.gif differ
diff --git a/docs/images/session-waterfall.png b/docs/images/session-waterfall.png
new file mode 100644
index 00000000..04af07b2
Binary files /dev/null and b/docs/images/session-waterfall.png differ
diff --git a/docs/mint.json b/docs/mint.json
index 5d746cbe..a0c8be26 100644
--- a/docs/mint.json
+++ b/docs/mint.json
@@ -31,12 +31,7 @@
"navigation": [
{
"group": "Getting Started",
- "pages": [
- "v1/introduction",
- "v1/installation",
- "v1/quickstart",
- "v1/examples"
- ]
+ "pages": ["v1/introduction", "v1/quickstart", "v1/examples"]
},
{
"group": "Details",
diff --git a/docs/v1/details/environment-variables.mdx b/docs/v1/details/environment-variables.mdx
index fcba6fe2..34def78c 100644
--- a/docs/v1/details/environment-variables.mdx
+++ b/docs/v1/details/environment-variables.mdx
@@ -1,19 +1,24 @@
---
title: "Environment Variables"
description: "Environment Variables that AgentOps will look for"
+mode: "wide"
---
### Sample .env file
-```.env
+```python .env
# API KEY for using AgentOps. Find this in your settings page
AGENTOPS_API_KEY=<YOUR API KEY>
-# Secondary API Key for simultaneously recording to a second project
+
+##########################################################
+# Only copy the following if you're going to override them
+
+# (OPTIONAL) Secondary API Key for simultaneously recording to a second project
AGENTOPS_PARENT_KEY=<YOUR PARENT KEY>
-# The AgentOps API endpoint. Defaults to https://api.agentops.ai
+# (OPTIONAL) The AgentOps API endpoint. Defaults to https://api.agentops.ai
AGENTOPS_API_ENDPOINT=https://api.agentops.ai
-# Logging level. <DEBUG, INFO, CRITICAL>
+# (OPTIONAL) Logging level. <DEBUG, INFO, CRITICAL>
AGENTOPS_LOGGING_LEVEL=INFO
-# Whether to opt out of recording environment data. <FALSE, TRUE>
+# (OPTIONAL) Whether to opt out of recording environment data. <FALSE, TRUE>
AGENTOPS_ENV_DATA_OPT_OUT=FALSE
```
\ No newline at end of file
diff --git a/docs/v1/details/tracking-agents.mdx b/docs/v1/details/tracking-agents.mdx
index d620a066..88b0b84a 100644
--- a/docs/v1/details/tracking-agents.mdx
+++ b/docs/v1/details/tracking-agents.mdx
@@ -5,11 +5,9 @@ description: "Use the `@track_agent()` decorator to tag your agents"
All Events are automatically associated with the agent that originated them. Agents are given a name which is what you will see in the dashboard.
-<div
- style={{ backgroundColor: "#f0f0f0", padding: "10px", borderRadius: "10px" }}
->
+<Frame type="glass" caption="This event is labeled with the name of the Agent that originated it">
<img height="200" src="/images/agent-name.png" />
-</div>
+</Frame>
The example below tags the agent MyAgent with name `SomeCustomName`.
diff --git a/docs/v1/details/tracking-llm-calls.mdx b/docs/v1/details/tracking-llm-calls.mdx
index 7cb3cdba..b2e30b05 100644
--- a/docs/v1/details/tracking-llm-calls.mdx
+++ b/docs/v1/details/tracking-llm-calls.mdx
@@ -22,7 +22,7 @@ Still not working? Please let us know! You can find us on [Discord](https://disc
To get started, just follow the quick start guide.
<CardGroup cols={2}>
- <Card title="Quick Start" icon="code" href="/v0/quickstart">
+ <Card title="Quick Start" icon="code" href="/v1/quickstart">
Get started with AgentOps in under 5 minutes
</Card>
</CardGroup>
diff --git a/docs/v1/examples.mdx b/docs/v1/examples.mdx
index 6788a406..44432abd 100644
--- a/docs/v1/examples.mdx
+++ b/docs/v1/examples.mdx
@@ -1,19 +1,20 @@
---
title: 'Examples'
description: 'Sample code that you can run!'
+mode: "wide"
---
## Explore our example code to see AgentOps in action!
<CardGroup cols={2}>
<Card title="Basic Example" icon="square-code" href="https://github.com/AgentOps-AI/agentops-py/blob/main/examples/openai-gpt.ipynb">
- Juypter Notebook with sample code that you can run! Start here!
+ Jupyter Notebook with sample code that you can run! Start here!
</Card>
<Card title="Multi-Agent Code Example" icon="object-group" href="https://github.com/AgentOps-AI/agentops-py/blob/main/examples/multi_agent_example.ipynb">
- Juypter Notebook with a simple multi-agent design
+ Jupyter Notebook with a simple multi-agent design
</Card>
<Card title="LangChain Example" icon="link" href="https://github.com/AgentOps-AI/agentops-py/blob/main/examples/langchain_examples.ipynb">
- Juypter Notebook with a sample LangChain integration
+ Jupyter Notebook with a sample LangChain integration
</Card>
<Card title="Crew.ai Example" icon="ship" href="/v1/integrations/crewai">
CrewAI multi-agent framework with AgentOps support
@@ -28,7 +29,7 @@ description: 'Sample code that you can run!'
![thumbnail](https://cdn.loom.com/sessions/thumbnails/0e0d2986f3d644a58d1e186dc81cd8b1-with-play.gif)
</Card>
<Card title="Multi-Agent" icon="link" href="https://www.loom.com/share/2a62fd70efe8423fb495946b09c2d8ac">
- Juypter Notebook with a sample LangChain integration
+ Jupyter Notebook with a sample LangChain integration
![thumbnail](https://cdn.loom.com/sessions/thumbnails/2a62fd70efe8423fb495946b09c2d8ac-with-play.gif)
</Card>
<Card title="Crew.ai" icon="ship" href="https://www.loom.com/share/cfcaaef8d4a14cc7a974843bda1076bf">
diff --git a/docs/v1/installation.mdx b/docs/v1/installation.mdx
deleted file mode 100644
index 69684b0b..00000000
--- a/docs/v1/installation.mdx
+++ /dev/null
@@ -1,29 +0,0 @@
----
-title: "Installing AgentOps"
----
-
-## Install the SDK
-
-<CodeGroup>
-```bash pip
-pip install agentops
-```
-```bash poetry
-poetry add agentops
-```
-</CodeGroup>
-
-## Get an API key
-
-Retrieve an API Key from the Settings > [Projects & API Keys](https://app.agentops.ai/settings/projects) page.
-
-API keys are tied to individual projects.<br></br>
-A Default Project has been created for you, so just click <span style={{ padding: "5px 10px", backgroundColor: "black", color: "white", borderRadius: "5px"}}>Copy API Key</span>
-
-{/* TODO: Install Tailwind for this stuff */}
-
-<div
- style={{ backgroundColor: "#f0f0f0", padding: "20px", borderRadius: "10px" }}
->
- <img height="200" src="/images/api-keys.png" />
-</div>
diff --git a/docs/v1/integrations/cohere.mdx b/docs/v1/integrations/cohere.mdx
index a27ec1e5..7da43477 100644
--- a/docs/v1/integrations/cohere.mdx
+++ b/docs/v1/integrations/cohere.mdx
@@ -1,6 +1,6 @@
---
title: Cohere
-description: "AgentOps's support for Cohere(>=5.4.0) is first class like Jack Harlow. This is a living integration, should you need any added functionality please message us on Discord!"
+description: "AgentOps's support for Cohere(>=5.4.0) is first class. This is a living integration, should you need any added functionality please message us on Discord!"
---
## Install the SDK
@@ -16,7 +16,7 @@ poetry add agentops
## Get an API key
-Retrieve an API Key from your [settings](https://app.agentops.ai/settings/projects) page.
+Retrieve an API Key from the Settings > [Projects & API Keys](https://app.agentops.ai/settings/projects) page.
API keys are tied to individual projects.<br></br>
A Default Project has been created for you, so just click <span style={{ padding: "5px 10px", backgroundColor: "black", color: "white", borderRadius: "5px"}}>Copy API Key</span>
@@ -32,9 +32,7 @@ A Default Project has been created for you, so just click <span style={{ padding
## Using AgentOps with Cohere
-### Requires cohere>=5.4.0
-
-#### Session replays in 3 lines of code
+<div style={{ color: "#880000" }}>Requires cohere>=5.4.0</div>
Initialize the AgentOps client and automatically get analytics on every LLM call.
diff --git a/docs/v1/introduction.mdx b/docs/v1/introduction.mdx
index 420cceef..291d7be1 100644
--- a/docs/v1/introduction.mdx
+++ b/docs/v1/introduction.mdx
@@ -1,23 +1,60 @@
---
-title: Introduction
-description: "Build your next agent with evals, observability, and replay."
+title: "Introduction"
+description: "Build your next agent with evals, observability, and replays"
+mode: "wide"
---
-## Get Started
+<Check>[Give us a star](https://github.com/AgentOps-AI/agentops) on GitHub if you find AgentOps helpful (you may be our 1,000th 😊)</Check>
-<CardGroup cols={2}>
- <Card title="Installing AgentOps" icon="wrench" href="./installation">
- Prepare your system for AgentOps
- </Card>
- <Card title="Quick Start" icon="rocket" href="./quickstart">
- Start using AgentOps with just 3 lines of code
- </Card>
-</CardGroup>
+# AgentOps is your new terminal
-## Integrate with your agent
+<Card title="Terminals suck" icon="trash" iconType="solid" color="#CC0000">
+ - • Lose track of what your agents did in between executions
+ - • Parsing through terminal output searching for LLM completions
+ - • Printing "tool called"
+</Card>
+<Card title="AgentOps doesn't" icon="face-smile-halo" iconType="duotone" color="#00CC00">
+ - • Visual dashboard so you can see what your agents did in human-readable format
+ - • LLM calls are magically recorded - prompt, completion, timestamps for each - with one line of code
+ - • Agents and their events (including tool calls) are recorded with one more line of code
+ - • Errors are magically associated to its causal event
+ - • Record any other events to your session with two more lines of code
+ - • Tons of other useful data if you're developing with supported agent frameworks: SDK version
+</Card>
-<CardGroup cols={2}>
- <Card title="Example Code" icon="square-code" href="./examples">
- Sample code that you can run!
- </Card>
-</CardGroup>
+## The AgentOps Dashboard
+
+With just two lines of code, you can free yourself from the chains of the terminal and instead visualize your agents' behavior
+in your AgentOps Dashboard. After setting up AgentOps, each execution of your program is recorded as a session and the above
+data is automatically recorded for you.
+
+The examples below were captured with two lines of code.
+
+### Session Drilldown
+Here you will find a list of all of your previously recorded sessions and useful data about each such as total execution time.
+You also get helpful debugging info such as any SDK versions you were on if you're building on a supported agent framework like Crew or AutoGen.
+LLM calls are presented as a familiar chat history view, and charts give you a breakdown of the types of events that were called and how long they took.
+
+<Frame type="glass" caption="Session Summary">
+ <img height="200" src="/images/session-summary.gif" />
+</Frame>
+
+Find any past sessions from your Session Drawer.
+<Frame type="glass" caption="Session Drawer">
+ <img height="250" src="/images/session-drawer.gif" />
+</Frame>
+
+Most powerful of all is the Session Waterfall. On the left, a time visualization of all your LLM calls, Action events, Tool calls, and Errors.
+On the right, specific details about the event you've selected on the waterfall. For instance the exact prompt and completion for a given LLM call.
+Most of which has been automatically recorded for you.
+
+<Frame type="glass" caption="Session Waterfall">
+ <img height="200" src="/images/session-waterfall.gif" />
+</Frame>
+
+
+### Session Overview
+View a meta-analysis of all of your sessions in a single view.
+<Frame type="glass" caption="Session Overview">
+ <img height="200" src="/images/overview.gif" />
+</Frame>
diff --git a/docs/v1/quickstart.mdx b/docs/v1/quickstart.mdx
index e186072e..06cb096a 100644
--- a/docs/v1/quickstart.mdx
+++ b/docs/v1/quickstart.mdx
@@ -1,28 +1,71 @@
---
title: "Quickstart"
-description: "Start using AgentOps with just 3 lines of code"
+description: "Start using AgentOps with just 2 lines of code"
---
-<CodeGroup>
-```python python
-import openai
-import agentops
-
-# Beginning of program's code (i.e. main.py, __init__.py)
-
-agentops.init(<INSERT YOUR API KEY HERE>)
-
-# Woohoo! You're on your way 🎉
-
-```
-</CodeGroup>
-
-When the `openai`, `litellm`, or `cohere` packages have been imported, instantiating the AgentOps client will automatically
-instrument them, meaning you will be able to see all of your sessions on the AgentOps
-dashboard along with the full LLM chat histories, cost, token counts, etc.
-
-
-## Decorate Functions
+<Steps>
+ <Step title="Install the SDK">
+ <CodeGroup>
+ ```bash pip
+ pip install agentops
+ ```
+ ```bash poetry
+ poetry add agentops
+ ```
+ </CodeGroup>
+ <Check>[Give us a star](https://github.com/AgentOps-AI/agentops) on GitHub while you're at it (you may be our 1,000th 😊)</Check>
+ </Step>
+ <Step title="Add 2 lines of code">
+ <CodeGroup>
+ ```python python
+ # IMPORTANT: Must be after 'import openai' or 'from crewai import Agent' etc
+ import agentops
+
+ # Beginning of program's code (i.e. main.py, __init__.py)
+ agentops.init(<INSERT YOUR API KEY HERE>)
+ ```
+ </CodeGroup>
+ <Check>Just these two lines and you have magically tracked LLM calls!</Check>
+ <Tip>
+ After the `openai`, `litellm`, or `cohere` packages have been imported, importing + instantiating the AgentOps client
+ will automatically instrument them, meaning you will be able to see all of your sessions on the AgentOps
+ DEBUGashboard along with the full LLM chat histories, cost, token counts, etc.
+ </Tip>
+
+ </Step>
+ <Step title="Get an API key">
+ Retrieve an API Key from the Settings > [Projects & API Keys](https://app.agentops.ai/settings/projects) page.
+ <div/>{/* Intentionally blank div for newline */}
+ Set this API Key in your [environment variables](/v1/details/environment-variables)
+
+ <Frame type="glass" caption="Settings > Projects & API Keys">
+ <img height="200" src="/images/api-keys.png" />
+ </Frame>
+
+ <Info>
+ API keys are tied to individual projects.<br></br>
+ A Default Project has been created for you, so just click Copy API Key
+ </Info>
+ {/* TODO: Install Tailwind for this stuff */}
+ </Step>
+ <Step title="Run your agent">
+ Execute your program and visit [app.agentops.ai/drilldown](https://app.agentops.ai/drilldown) to observe your Agent! 🕵️
+
+ <Tip>After your run, AgentOps prints a clickable url to console linking directly to your session in the Dashboard</Tip>
+ <div/>{/* Intentionally blank div for newline */}
+
+ <Frame type="glass" caption="Clickable link to session">
+ <img height="200" src="/images/link-to-session.gif" />
+ </Frame>
+ </Step>
+</Steps>
+
+
+## More basic functionality
+
+<CardGroup cols={1}>
+
+<Card icon="code" title="Decorate Functions">
You can instrument other functions inside your code with the handy `@record_function`
decorator, which will record an `action_type`, the parameters, and the returns. You
will see these function calls alongside your LLM calls from instantiating the AgentOps client.
@@ -33,7 +76,8 @@ dashboard along with the full LLM chat histories, cost, token counts, etc.
...
```
-## Track Agents
+</Card>
+<Card icon="robot" title="Track Agents">
If you use specific named agents within your system, you can tie all downstream [Events](/v1/details/events) to a
Named Agent with the `@track_agent` decorator.
```python python
@@ -43,8 +87,8 @@ dashboard along with the full LLM chat histories, cost, token counts, etc.
...
```
-## Ending Your Session
-
+</Card>
+<Card icon="stop" title="Ending Your Session">
Finally, you should end your session by calling `.end_session()` with whether your session
was successful or not `(Success|Fail)`. We suggest setting session state depending on how
your agent exits or whether your agent succeeded or not. You can also specify a end state reason,
@@ -52,12 +96,10 @@ dashboard along with the full LLM chat histories, cost, token counts, etc.
```python python
# End of program
agentops.end_session('Success')
- # Woohoo! You're done 🎉
```
+</Card>
+</CardGroup>
-## Check out your dashboard
- Run your agent and then visit [`app.agentops.ai`](https://app.agentops.ai).
- From the "Session Drill-Down" you should be able to see all of your OpenAI calls!
## Example Code
@@ -82,7 +124,6 @@ class sample_agent(...):
# End of program
agentops.end_session('Success')
-# Woohoo! You're done 🎉
```
<Card
@@ -93,7 +134,9 @@ agentops.end_session('Success')
Jupyter Notebook with sample code that you can run!
</Card>
-That's all you need to get started! Check out the documentation below to see how you can record other events. AgentOps is a lot more powerful this way!
+<Check>
+ That's all you need to get started! Check out the documentation below to see how you can record other events. AgentOps is a lot more powerful this way!
+</Check>
## Explore our more advanced functionality!
@@ -108,4 +151,4 @@ That's all you need to get started! Check out the documentation below to see how
<Card title="Tracking Agents" icon="robot" href="/v1/details/tracking-agents">
Associate Events with specific named agents.
</Card>
-</CardGroup>
+</CardGroup>
\ No newline at end of file