Skip to content

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI

Added simple Ollama example (#515)

AkhilAnand135cf8ae7493
diff --git a/docs/images/external/ollama/ollama-icon.png b/docs/images/external/ollama/ollama-icon.png
new file mode 100644
index 00000000..46060de8
Binary files /dev/null and b/docs/images/external/ollama/ollama-icon.png differ
diff --git a/docs/mint.json b/docs/mint.json
index d2043620..45e61b45 100644
--- a/docs/mint.json
+++ b/docs/mint.json
@@ -92,6 +92,7 @@
"v1/integrations/langchain",
"v1/integrations/cohere",
"v1/integrations/anthropic",
+ "v1/integrations/ollama",
"v1/integrations/litellm",
"v1/integrations/multion",
"v1/integrations/rest"
diff --git a/docs/v1/examples/examples.mdx b/docs/v1/examples/examples.mdx
index 57765789..24ac2e33 100644
--- a/docs/v1/examples/examples.mdx
+++ b/docs/v1/examples/examples.mdx
@@ -31,6 +31,9 @@ mode: "wide"
<Card title="MultiOn Example" icon="atom" href="/v1/examples/multion">
Create an autonomous browser agent capable of navigating the web and extracting information
</Card>
+ <Card title="Ollama Example" icon={<img src="/images/external/ollama/ollama-icon.png" alt="Ollama" />} iconType="image" href="/v1/examples/ollama">
+ Simple Ollama integration with AgentOps
+ </Card>
</CardGroup>
## Video Guides
diff --git a/docs/v1/examples/ollama.mdx b/docs/v1/examples/ollama.mdx
new file mode 100644
index 00000000..a297f192
--- /dev/null
+++ b/docs/v1/examples/ollama.mdx
@@ -0,0 +1,123 @@
+---
+title: 'Ollama Example'
+description: 'Using Ollama with AgentOps'
+mode: "wide"
+---
+
+{/* SOURCE_FILE: examples/ollama_examples/ollama_examples.ipynb */}# AgentOps Ollama Integration
+
+This example demonstrates how to use AgentOps to monitor your Ollama LLM calls.
+
+First let's install the required packages
+
+> ⚠️ **Important**: Make sure you have Ollama installed and running locally before running this notebook. You can install it from [ollama.ai](https://ollama.com).
+
+
+```python
+%pip install -U ollama
+%pip install -U agentops
+%pip install -U python-dotenv
+```
+
+Then import them
+
+
+```python
+import ollama
+import agentops
+import os
+from dotenv import load_dotenv
+
+```
+
+Next, we'll set our API keys. For Ollama, we'll need to make sure Ollama is running locally.
+[Get an AgentOps API key](https://agentops.ai/settings/projects)
+
+1. Create an environment variable in a .env file or other method. By default, the AgentOps `init()` function will look for an environment variable named `AGENTOPS_API_KEY`. Or...
+2. Replace `<your_agentops_key>` below and pass in the optional `api_key` parameter to the AgentOps `init(api_key=...)` function. Remember not to commit your API key to a public repo!
+
+
+```python
+# Let's load our environment variables
+load_dotenv()
+
+AGENTOPS_API_KEY = os.getenv("AGENTOPS_API_KEY") or "<your_agentops_key>"
+```
+
+
+```python
+# Initialize AgentOps with some default tags
+agentops.init(AGENTOPS_API_KEY, default_tags=["ollama-example"])
+```
+
+Now let's make some basic calls to Ollama. Make sure you have pulled the model first, use the following or replace with whichever model you want to use.
+
+
+```python
+ollama.pull("mistral")
+```
+
+
+```python
+# Basic completion,
+response = ollama.chat(model='mistral',
+ messages=[{
+ 'role': 'user',
+ 'content': 'What are the benefits of using AgentOps for monitoring LLMs?',
+ }]
+)
+print(response['message']['content'])
+```
+
+Let's try streaming responses as well
+
+
+```python
+# Streaming Example
+stream = ollama.chat(
+ model='mistral',
+ messages=[{
+ 'role': 'user',
+ 'content': 'Write a haiku about monitoring AI agents',
+ }],
+ stream=True
+)
+
+for chunk in stream:
+ print(chunk['message']['content'], end='')
+
+```
+
+
+```python
+# Conversation Example
+messages = [
+ {
+ 'role': 'user',
+ 'content': 'What is AgentOps?'
+ },
+ {
+ 'role': 'assistant',
+ 'content': 'AgentOps is a monitoring and observability platform for LLM applications.'
+ },
+ {
+ 'role': 'user',
+ 'content': 'Can you give me 3 key features?'
+ }
+]
+
+response = ollama.chat(
+ model='mistral',
+ messages=messages
+)
+print(response['message']['content'])
+```
+
+> 💡 **Note**: In production environments, you should add proper error handling around the Ollama calls and use `agentops.end_session("Error")` when exceptions occur.
+
+Finally, let's end our AgentOps session
+
+
+```python
+agentops.end_session("Success")
+```
diff --git a/docs/v1/integrations/ollama.mdx b/docs/v1/integrations/ollama.mdx
new file mode 100644
index 00000000..b88d78c7
--- /dev/null
+++ b/docs/v1/integrations/ollama.mdx
@@ -0,0 +1,150 @@
+---
+title: Ollama
+description: "AgentOps provides first class support for Ollama"
+---
+
+import CodeTooltip from '/snippets/add-code-tooltip.mdx'
+import EnvTooltip from '/snippets/add-env-tooltip.mdx'
+
+<Note>
+This is a living integration. Should you need any added functionality, message us on [Discord](https://discord.gg/UgJyyxx7uc)!
+</Note>
+
+<Card title="Ollama" icon={<img src="/images/external/ollama/ollama-icon.png" alt="Ollama" />} iconType="image" href="https://ollama.com">
+ First class support for Ollama
+</Card>
+
+<Steps>
+ <Step title="Install the AgentOps SDK">
+ <CodeGroup>
+ ```bash pip
+ pip install agentops ollama
+ ```
+ ```bash poetry
+ poetry add agentops ollama
+ ```
+ </CodeGroup>
+ </Step>
+ <Step title="Add 3 lines of code">
+ <CodeTooltip/>
+ <CodeGroup>
+ ```python python
+ import agentops
+ import ollama
+
+ agentops.init(<INSERT YOUR API KEY HERE>)
+ agentops.start_session()
+
+ ollama.pull("<MODEL NAME>")
+
+ response = ollama.chat(model='mistral',
+ messages=[{
+ 'role': 'user',
+ 'content': 'What are the benefits of using AgentOps for monitoring LLMs?',
+ }]
+ )
+ print(response['message']['content'])
+ ...
+ # End of program (e.g. main.py)
+ agentops.end_session("Success") # Success|Fail|Indeterminate
+ ```
+ </CodeGroup>
+ <EnvTooltip />
+ <CodeGroup>
+ ```python .env
+ # Alternatively, you can set the API key as an environment variable
+ AGENTOPS_API_KEY=<YOUR API KEY>
+ ```
+ </CodeGroup>
+ Read more about environment variables in [Advanced Configuration](/v1/usage/advanced-configuration)
+ </Step>
+ <Step title="Run your Agent">
+ Execute your program and visit [app.agentops.ai/drilldown](https://app.agentops.ai/drilldown) to observe your Agent! 🕵️
+ <Tip>
+ After your run, AgentOps prints a clickable url to console linking directly to your session in the Dashboard
+ </Tip>
+ <div/>
+ </Step>
+</Steps>
+
+## Full Examples
+
+<CodeGroup>
+ ```python basic completion
+ import ollama
+ import agentops
+
+ agentops.init(<INSERT YOUR API KEY HERE>)
+
+ ollama.pull("<MODEL NAME>")
+ response = ollama.chat(
+ model="<MODEL NAME>",
+ max_tokens=1024,
+ messages=[{
+ "role": "user",
+ "content": "Write a haiku about AI and humans working together"
+ }]
+ )
+
+ print(response['message']['content'])
+ agentops.end_session('Success')
+ ```
+
+ ```python streaming
+ import agentops
+ import ollama
+
+ async def main():
+ agentops.init(<INSERT YOUR API KEY HERE>)
+ ollama.pull("<MODEL NAME>")
+
+ stream = ollama.chat(
+ model="<MODEL NAME>",
+ messages=[{
+ 'role': 'user',
+ 'content': 'Write a haiku about monitoring AI agents',
+ }],
+ stream=True
+ )
+
+ for chunk in stream:
+ print(chunk['message']['content'], end='')
+
+ agentops.end_session('Success')
+ ```
+
+ ```python conversation
+ import ollama
+ import agentops
+
+ agentops.init(<INSERT YOUR API KEY HERE>)
+ ollama.pull("<MODEL NAME>")
+
+ messages = [
+ {
+ 'role': 'user',
+ 'content': 'What is AgentOps?'
+ },
+ {
+ 'role': 'assistant',
+ 'content': 'AgentOps is a monitoring and observability platform for LLM applications.'
+ },
+ {
+ 'role': 'user',
+ 'content': 'Can you give me 3 key features?'
+ }
+]
+
+ response = ollama.chat(
+ model="<MODEL NAME>",
+ messages=messages
+ )
+ print(response['message']['content'])
+ agentops.end_session('Success')
+ ```
+</CodeGroup>
+
+<script type="module" src="/scripts/github_stars.js"></script>
+<script type="module" src="/scripts/scroll-img-fadein-animation.js"></script>
+<script type="module" src="/scripts/button_heartbeat_animation.js"></script>
+<script type="css" src="/styles/styles.css"></script>
diff --git a/examples/ollama_examples/ollama_examples.ipynb b/examples/ollama_examples/ollama_examples.ipynb
new file mode 100644
index 00000000..c876ef7a
--- /dev/null
+++ b/examples/ollama_examples/ollama_examples.ipynb
@@ -0,0 +1,212 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# AgentOps Ollama Integration\n",
+ "\n",
+ "This example demonstrates how to use AgentOps to monitor your Ollama LLM calls.\n",
+ "\n",
+ "First let's install the required packages\n",
+ "\n",
+ "> ⚠️ **Important**: Make sure you have Ollama installed and running locally before running this notebook. You can install it from [ollama.ai](https://ollama.com)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "%pip install -U ollama\n",
+ "%pip install -U agentops\n",
+ "%pip install -U python-dotenv"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Then import them"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import ollama\n",
+ "import agentops\n",
+ "import os\n",
+ "from dotenv import load_dotenv\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Next, we'll set our API keys. For Ollama, we'll need to make sure Ollama is running locally.\n",
+ "[Get an AgentOps API key](https://agentops.ai/settings/projects)\n",
+ "\n",
+ "1. Create an environment variable in a .env file or other method. By default, the AgentOps `init()` function will look for an environment variable named `AGENTOPS_API_KEY`. Or...\n",
+ "2. Replace `<your_agentops_key>` below and pass in the optional `api_key` parameter to the AgentOps `init(api_key=...)` function. Remember not to commit your API key to a public repo!"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Let's load our environment variables\n",
+ "load_dotenv()\n",
+ "\n",
+ "AGENTOPS_API_KEY = os.getenv(\"AGENTOPS_API_KEY\") or \"<your_agentops_key>\""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Initialize AgentOps with some default tags\n",
+ "agentops.init(AGENTOPS_API_KEY, default_tags=[\"ollama-example\"])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now let's make some basic calls to Ollama. Make sure you have pulled the model first, use the following or replace with whichever model you want to use."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "ollama.pull(\"mistral\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Basic completion,\n",
+ "response = ollama.chat(model='mistral',\n",
+ " messages=[{\n",
+ " 'role': 'user',\n",
+ " 'content': 'What are the benefits of using AgentOps for monitoring LLMs?',\n",
+ " }]\n",
+ ")\n",
+ "print(response['message']['content'])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let's try streaming responses as well"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Streaming Example\n",
+ "stream = ollama.chat(\n",
+ " model='mistral',\n",
+ " messages=[{\n",
+ " 'role': 'user',\n",
+ " 'content': 'Write a haiku about monitoring AI agents',\n",
+ " }],\n",
+ " stream=True\n",
+ ")\n",
+ "\n",
+ "for chunk in stream:\n",
+ " print(chunk['message']['content'], end='')\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Conversation Example\n",
+ "messages = [\n",
+ " {\n",
+ " 'role': 'user',\n",
+ " 'content': 'What is AgentOps?'\n",
+ " },\n",
+ " {\n",
+ " 'role': 'assistant',\n",
+ " 'content': 'AgentOps is a monitoring and observability platform for LLM applications.'\n",
+ " },\n",
+ " {\n",
+ " 'role': 'user',\n",
+ " 'content': 'Can you give me 3 key features?'\n",
+ " }\n",
+ "]\n",
+ "\n",
+ "response = ollama.chat(\n",
+ " model='mistral',\n",
+ " messages=messages\n",
+ ")\n",
+ "print(response['message']['content'])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "> 💡 **Note**: In production environments, you should add proper error handling around the Ollama calls and use `agentops.end_session(\"Error\")` when exceptions occur."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Finally, let's end our AgentOps session"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "agentops.end_session(\"Success\")"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "gpt_desk",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.11.9"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}