Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI
Python agentagentopsagents-sdkaianthropicautogencost-estimationcrewaievalsevaluation-metricsgroqlangchainllmmistralollamaopenaiopenai-agents
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# OpenAI Async Example\n",
"\n",
"We are going to create a simple chatbot that creates stories based on a user provided image. The chatbot will use the gpt-4o-mini LLM to generate the story using a user prompt and its vision model to understand the image.\n",
"\n",
"We will track the chatbot with AgentOps and see how it performs!"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"First let's install the required packages"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Install required dependencies\n",
"%pip install agentops\n",
"%pip install openai\n",
"%pip install python-dotenv"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Then import them"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from openai import AsyncOpenAI\n",
"import agentops\n",
"import os\n",
"from dotenv import load_dotenv"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, we'll grab our API keys. You can use dotenv like below or however else you like to load environment variables"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"load_dotenv()\n",
"os.environ[\"OPENAI_API_KEY\"] = os.getenv(\"OPENAI_API_KEY\", \"your_openai_api_key_here\")\n",
"os.environ[\"AGENTOPS_API_KEY\"] = os.getenv(\"AGENTOPS_API_KEY\", \"your_api_key_here\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next we initialize the AgentOps client."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"agentops.init(auto_start_session=True)\n",
"tracer = agentops.start_trace(\n",
" trace_name=\"OpenAI Async Example\", tags=[\"openai-async-example\", \"openai\", \"agentops-example\"]\n",
")\n",
"client = AsyncOpenAI()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"And we are all set! Note the seesion url above. We will use it to track the chatbot.\n",
"\n",
"Let's create a simple chatbot that generates stories given an image and a user prompt."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"system_prompt = \"\"\"\n",
"You are a master storyteller, with the ability to create vivid and engaging stories.\n",
"You have experience in writing for children and adults alike.\n",
"You are given a prompt and you need to generate a story based on the prompt.\n",
"\"\"\"\n",
"\n",
"user_prompt = [\n",
" {\"type\": \"text\", \"text\": \"Write a mystery thriller story based on your understanding of the provided image.\"},\n",
" {\n",
" \"type\": \"image_url\",\n",
" \"image_url\": {\"url\": \"https://www.cosy.sbg.ac.at/~pmeerw/Watermarking/lena_color.gif\"},\n",
" },\n",
"]\n",
"\n",
"messages = [\n",
" {\"role\": \"system\", \"content\": system_prompt},\n",
" {\"role\": \"user\", \"content\": user_prompt},\n",
"]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"async def main():\n",
" response = await client.chat.completions.create(\n",
" model=\"gpt-4o-mini\",\n",
" messages=messages,\n",
" )\n",
"\n",
" print(response.choices[0].message.content)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"await main()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The response is a string that contains the story. We can track this with AgentOps by navigating to the trace url and viewing the run."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Streaming Version\n",
"We will demonstrate the streaming version of the API."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"async def main_stream():\n",
" stream = await client.chat.completions.create(\n",
" model=\"gpt-4o-mini\",\n",
" messages=messages,\n",
" stream=True,\n",
" )\n",
"\n",
" async for chunk in stream:\n",
" print(chunk.choices[0].delta.content or \"\", end=\"\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"await main_stream()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"agentops.end_trace(tracer, end_state=\"Success\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that the response is a generator that yields chunks of the story. We can track this with AgentOps by navigating to the trace url and viewing the run.\n",
"All done!"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "agentops (3.11.11)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.11"
}
},
"nbformat": 4,
"nbformat_minor": 2
}