Skip to content

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI

3.8 KB
Raw Blame
---
title: Ollama
description: "AgentOps provides first class support for Ollama"
---

import CodeTooltip from '/snippets/add-code-tooltip.mdx'
import EnvTooltip from '/snippets/add-env-tooltip.mdx'

[Ollama](https://ollama.com) is a lightweight, open-source tool for running and managing LLM models. Track your Ollama model calls with AgentOps.

<Steps>
  <Step title="Install the AgentOps SDK">
    <CodeGroup>
      ```bash pip 
      pip install agentops ollama
      ```
      ```bash poetry
      poetry add agentops ollama
      ```
    </CodeGroup>
  </Step>
  <Step title="Install the Ollama SDK">
    <CodeGroup>
      ```bash pip
      pip install ollama
      ```
      ```bash poetry
      poetry add ollama
      ```
    </CodeGroup>
  </Step>
  <Step title="Add 3 lines of code">
    <CodeTooltip/>
    <CodeGroup>
      ```python python
      import agentops
      import ollama
      
      agentops.init(<INSERT YOUR API KEY HERE>)
      agentops.start_session()

      ollama.pull("<MODEL NAME>")

      response = ollama.chat(model='mistral',
        messages=[{
            'role': 'user',
            'content': 'What are the benefits of using AgentOps for monitoring LLMs?',
        }]
      )
      print(response['message']['content'])
      ...
      # End of program (e.g. main.py)
      agentops.end_session("Success")
      ```
    </CodeGroup>
    <EnvTooltip />
      <CodeGroup>
        ```python .env
        # Alternatively, you can set the API key as an environment variable
        AGENTOPS_API_KEY=<YOUR API KEY>
        ```
      </CodeGroup>
      Read more about environment variables in [Advanced Configuration](/v1/usage/advanced-configuration)
  </Step>
  <Step title="Run your Agent">
    Execute your program and visit [app.agentops.ai/drilldown](https://app.agentops.ai/drilldown) to observe your Agent! 🕵️
    <Tip>
      After your run, AgentOps prints a clickable url to console linking directly to your session in the Dashboard
    </Tip>
    <div/>
  </Step>
</Steps>

## Full Examples

<CodeGroup>
  ```python basic completion
  import ollama
  import agentops

  agentops.init(<INSERT YOUR API KEY HERE>)

  ollama.pull("<MODEL NAME>")
  response = ollama.chat(
      model="<MODEL NAME>",
      max_tokens=1024,
      messages=[{
          "role": "user",
          "content": "Write a haiku about AI and humans working together"
      }]
  )

  print(response['message']['content'])
  agentops.end_session('Success')
  ```

  ```python streaming
  import agentops
  import ollama

  async def main():
      agentops.init(<INSERT YOUR API KEY HERE>)
      ollama.pull("<MODEL NAME>")

      stream = ollama.chat(
        model="<MODEL NAME>",
          messages=[{
              'role': 'user',
              'content': 'Write a haiku about monitoring AI agents',
          }], 
          stream=True
      )

      for chunk in stream:
          print(chunk['message']['content'], end='')

      agentops.end_session('Success')
  ```

  ```python conversation
  import ollama
  import agentops

  agentops.init(<INSERT YOUR API KEY HERE>)
  ollama.pull("<MODEL NAME>")

  messages = [
    {
        'role': 'user',
        'content': 'What is AgentOps?'
    },
    {
        'role': 'assistant',
        'content': 'AgentOps is a monitoring and observability platform for LLM applications.'
    },
    {
        'role': 'user',
        'content': 'Can you give me 3 key features?'
    }
]

  response = ollama.chat(
      model="<MODEL NAME>",
      messages=messages
  )
  print(response['message']['content'])   
  agentops.end_session('Success')
  ```
</CodeGroup>

<script type="module" src="/scripts/github_stars.js"></script>
<script type="module" src="/scripts/scroll-img-fadein-animation.js"></script>
<script type="module" src="/scripts/button_heartbeat_animation.js"></script>
<script type="css" src="/styles/styles.css"></script>