Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI
Python agentagentopsagents-sdkaianthropicautogencost-estimationcrewaievalsevaluation-metricsgroqlangchainllmmistralollamaopenaiopenai-agents
Examples (#146)
Braelyn Boynton46e8f49a8657
diff --git a/agentops/__init__.py b/agentops/__init__.pyindex 909cedd2..ae4c1e58 100755--- a/agentops/__init__.py+++ b/agentops/__init__.py@@ -59,7 +59,7 @@ def start_session(tags: Optional[List[str]] = None, config: Optional[ConfiguratiClient().start_session(tags, config)-def record(event: Event):+def record(event: Event | ErrorEvent):Client().record(event)diff --git a/examples/README.md b/examples/README.mdnew file mode 100644index 00000000..62c35dd3--- /dev/null+++ b/examples/README.md@@ -0,0 +1,11 @@+# AgentOps Examples++## Quickstart+- [Single LLM](./openai-gpt.ipynb)+- [Recording Events](./recording-events.ipynb)+- [Multi-Agent](./multi_agent_example.ipynb)++## Integrations+- [Using Langchain](./langchain_examples.ipynb)+- [Crew.ai](https://github.com/joaomdmoura/crewAI-examples/tree/main/markdown_validator)+ - Crew is a framework for developing agents, a number of their example projects use AgentOps\ No newline at end of filediff --git a/examples/openai-gpt.ipynb b/examples/openai-gpt.ipynbindex f2599d93..e97ad32b 100644--- a/examples/openai-gpt.ipynb+++ b/examples/openai-gpt.ipynb@@ -7,7 +7,7 @@"collapsed": false},"source": [- "# Monitoring \n",+ "# AgentOps Basic Monitoring \n","This is an example of how to use the AgentOps library for basic Agent monitoring with OpenAI's GPT"]},@@ -70,7 +70,7 @@"outputs": [],"source": ["openai = OpenAI(api_key=OPENAI_API_KEY)\n",- "agentops.init(AGENTOPS_API_KEY)\n"+ "agentops.init(AGENTOPS_API_KEY)"]},{diff --git a/examples/recording-events.ipynb b/examples/recording-events.ipynbnew file mode 100644index 00000000..0f224f8a--- /dev/null+++ b/examples/recording-events.ipynb@@ -0,0 +1,179 @@+{+ "cells": [+ {+ "cell_type": "markdown",+ "source": [+ "# Recording Events\n",+ "AgentOps has a number of different [Event Types](https://docs.agentops.ai/v1/details/events)"+ ],+ "metadata": {+ "collapsed": false+ },+ "id": "dc8cfd2cfa8a594b"+ },+ {+ "cell_type": "code",+ "outputs": [],+ "source": [+ "# Create new session\n",+ "agentops.start_session()\n",+ "\n",+ "# Optionally, we can add tags to the session\n",+ "# agentops.start_session(['Hello Tracker'])"+ ],+ "metadata": {+ "collapsed": false,+ "is_executing": true+ },+ "id": "168ecd05cc123de0",+ "execution_count": null+ },+ {+ "cell_type": "markdown",+ "source": [+ "The easiest way to record actions is through the use of AgentOp's decorators"+ ],+ "metadata": {+ "collapsed": false+ },+ "id": "c6d06ee8c66dad17"+ },+ {+ "cell_type": "code",+ "outputs": [],+ "source": [+ "from agentops import record_function\n",+ "\n",+ "@record_function(\"add numbers\")\n",+ "def add(x, y):\n",+ " return x + y\n",+ "\n",+ "add(2,4)"+ ],+ "metadata": {+ "collapsed": false+ },+ "id": "b460318317adc624",+ "execution_count": 0+ },+ {+ "cell_type": "markdown",+ "source": [+ "We can also manually craft an event exactly the way we want"+ ],+ "metadata": {+ "collapsed": false+ },+ "id": "9068a4cdd328f652"+ },+ {+ "cell_type": "code",+ "outputs": [],+ "source": [+ "from agentops import ActionEvent\n",+ "\n",+ "message = {\"role\": \"user\", \"content\": \"Hello\"},\n",+ "response = openai.chat.completions.create(\n",+ " model='gpt-3.5-turbo', messages=message, temperature=0.5)\n",+ "\n",+ "if \"hello\" in str(response.choices[0].message.content).lower():\n",+ " agentops.record(ActionEvent(action_type=\"Agent says hello\", params=str(message), returns=str(response.choices[0].message.content) ))"+ ],+ "metadata": {+ "collapsed": false+ },+ "id": "b62ad88921ff26f2",+ "execution_count": 0+ },+ {+ "cell_type": "code",+ "outputs": [],+ "source": [+ "agentops.end_session('Success')"+ ],+ "metadata": {+ "collapsed": false+ },+ "id": "e10a89e06fe6b2be",+ "execution_count": null+ },+ {+ "cell_type": "markdown",+ "source": [+ "## Tool Event\n",+ "Agents use tools. These tools are useful to track with information such as name, end status, runtime, etc. To record tool usage, you can create and record a `ToolEvent` similar to above."+ ],+ "metadata": {+ "collapsed": false+ },+ "id": "f7c947d815f581e7"+ },+ {+ "cell_type": "code",+ "outputs": [],+ "source": [+ "from agentops import ToolEvent, record\n",+ "def scrape_website(url: str):\n",+ " tool_event = ToolEvent(name='scrape_website', params={'url':url}) # the start timestamp is set when the obj is created\n",+ " result = integration.scrape_website(data) # perform tool logic\n",+ " tool_event.returns = result\n",+ " record(tool_event)"+ ],+ "metadata": {+ "collapsed": false+ },+ "id": "5d387a071a1c70cf"+ },+ {+ "cell_type": "markdown",+ "source": [+ "## Error Events\n",+ "Error events can be used alone or in reference to another event. Lets add a catch block to the code above"+ ],+ "metadata": {+ "collapsed": false+ },+ "id": "968d1503dd0aae9a"+ },+ {+ "cell_type": "code",+ "outputs": [],+ "source": [+ "from agentops import ToolEvent, record, ErrorEvent\n",+ "def scrape_website(url: str):\n",+ " tool_event = ToolEvent(name='scrape_website', params={'url':url}) # the start timestamp is set when the obj is created\n",+ " try:\n",+ " result = integration.scrape_website(data) # perform tool logic\n",+ " tool_event.returns = result\n",+ " except Error as e:\n",+ " record(ErrorEvent(message=e, trigger_event=tool_event))\n",+ " record(tool_event)"+ ],+ "metadata": {+ "collapsed": false+ },+ "id": "eb23c1325298e22f"+ }+ ],+ "metadata": {+ "kernelspec": {+ "display_name": "Python 3",+ "language": "python",+ "name": "python3"+ },+ "language_info": {+ "codemirror_mode": {+ "name": "ipython",+ "version": 2+ },+ "file_extension": ".py",+ "mimetype": "text/x-python",+ "name": "python",+ "nbconvert_exporter": "python",+ "pygments_lexer": "ipython2",+ "version": "2.7.6"+ }+ },+ "nbformat": 4,+ "nbformat_minor": 5+}