Skip to content

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI

GitHub Action - Update examples in docs from notebooks (#624)

github-actions[bot]1ec7784a7e2c
diff --git a/docs/v1/examples/openai_assistants.mdx b/docs/v1/examples/openai_assistants.mdx
index 77e16290..e09857ae 100644
--- a/docs/v1/examples/openai_assistants.mdx
+++ b/docs/v1/examples/openai_assistants.mdx
@@ -28,12 +28,15 @@ The primitives of the **Assistants API** are
We'll take a look at how these can be used to create powerful, stateful experiences.
+
## Setup
> **Note**
> The Assistants API is currently in beta so the latest [Python SDK](https://github.com/openai/openai-python) is needed (`1.58.1` at time of writing) for this example.
-```bash
+
+
+```python
%pip install -U openai
%pip install -U agentops
%pip install -U python-dotenv
@@ -41,6 +44,8 @@ We'll take a look at how these can be used to create powerful, stateful experien
### Pretty Printing Helper
+
+
```python
import json
@@ -50,20 +55,28 @@ def show_json(obj):
## Complete Example with Assistants API
+
### Assistants
+
The easiest way to get started with the Assistants API is through the [Assistants Playground](https://platform.openai.com/playground).
+
![Assistants Playground](https://github.com/AgentOps-AI/agentops/blob/main/examples/openai_examples/images/assistants_overview_assistants_playground.png)
+
Let's begin by creating an assistant! We'll create a Math Tutor just like in our [docs](https://platform.openai.com/docs/assistants/overview).
+
![Creating New Assistant](https://github.com/AgentOps-AI/agentops/blob/main/examples/openai_examples/images/assistants_overview_new_assistant.png)
+
You can view Assistants you've created in the [Assistants Dashboard](https://platform.openai.com/assistants).
+
![Assistants Dashboard](https://github.com/AgentOps-AI/agentops/blob/main/examples/openai_examples/images/assistants_overview_assistants_dashboard.png)
+
You can also create Assistants directly through the Assistants API. But we need to have the AgentOps and OpenAI API keys first.
You can get your OpenAI API key from the [OpenAI Dashboard](https://platform.openai.com/api-keys).
@@ -78,6 +91,7 @@ Next, we'll set our API keys. There are several ways to do this, the code below
Now we are all set! Let's import the necessary libraries and initialize the AgentOps and OpenAI clients.
+
```python
from openai import OpenAI
import agentops
@@ -89,6 +103,7 @@ AGENTOPS_API_KEY = os.getenv("AGENTOPS_API_KEY") or "<your_agentops_key>"
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY") or "<your_openai_key>"
```
+
```python
agentops.init(api_key=AGENTOPS_API_KEY, default_tags=["openai", "beta-assistants"])
client = OpenAI(api_key=OPENAI_API_KEY)
@@ -96,6 +111,7 @@ client = OpenAI(api_key=OPENAI_API_KEY)
Next, we'll create an Assistant which will be our Math Tutor.
+
```python
assistant = client.beta.assistants.create(
name="Math Tutor",
@@ -107,12 +123,17 @@ show_json(assistant)
Regardless of whether you create your Assistant through the Dashboard or with the API, you'll want to keep track of the Assistant ID. This is how you'll refer to your Assistant throughout Threads and Runs.
+
Next, we'll create a new Thread and add a Message to it. This will hold the state of our conversation, so we don't have re-send the entire message history each time.
+
### Threads
+
Create a new thread:
+
+
```python
thread = client.beta.threads.create()
show_json(thread)
@@ -120,6 +141,8 @@ show_json(thread)
Then add the Message to the thread:
+
+
```python
message = client.beta.threads.messages.create(
thread_id=thread.id,
@@ -132,6 +155,7 @@ show_json(message)
> **Note**
> Even though you're no longer sending the entire history each time, you will still be charged for the tokens of the entire conversation history with each Run.
+
### Runs
Notice how the Thread we created is **not** associated with the Assistant we created earlier! Threads exist independently from Assistants, which may be different from what you'd expect if you've used ChatGPT (where a thread is tied to a model/GPT).
@@ -143,6 +167,8 @@ To get a completion from an Assistant for a given Thread, we must create a Run.
To get our Assistant to respond to the user, let's create the Run. As mentioned earlier, you must specify _both_ the Assistant and the Thread.
+
+
```python
run = client.beta.threads.runs.create(
thread_id=thread.id,
@@ -155,6 +181,8 @@ Unlike creating a completion in the Chat Completions API, **creating a Run is an
To know when the Assistant has completed processing, we can poll the Run in a loop. (Support for streaming is coming soon!) While here we are only checking for a `queued` or `in_progress` status, in practice a Run may undergo a [variety of status changes](https://platform.openai.com/docs/api-reference/runs/object#runs/object-status) which you can choose to surface to the user. (These are called Steps, and will be covered later.)
+
+
```python
import time
@@ -168,6 +196,7 @@ def wait_on_run(run, thread):
return run
```
+
```python
run = wait_on_run(run, thread)
show_json(run)
@@ -175,8 +204,11 @@ show_json(run)
### Messages
+
Now that the Run has completed, we can list the Messages in the Thread to see what got added by the Assistant.
+
+
```python
messages = client.beta.threads.messages.list(thread_id=thread.id)
show_json(messages)
@@ -184,8 +216,11 @@ show_json(messages)
As you can see, Messages are ordered in reverse-chronological order – this was done so the most recent results are always on the first `page` (since results can be paginated). Do keep a look out for this, since this is the opposite order to messages in the Chat Completions API.
+
Let's ask our Assistant to explain the result a bit further!
+
+
```python
# Create a message to append to our thread
message = client.beta.threads.messages.create(
@@ -210,10 +245,112 @@ show_json(messages)
This may feel like a lot of steps to get a response back, especially for this simple example. However, you'll soon see how we can add very powerful functionality to our Assistant without changing much code at all!
+
+### Example
+
+
+Let's take a look at how we could potentially put all of this together. Below is all the code you need to use an Assistant you've created.
+
+Since we've already created our Math Assistant, I've saved its ID in `MATH_ASSISTANT_ID`. I then defined two functions:
+
+- `submit_message`: create a Message on a Thread, then start (and return) a new Run
+- `get_response`: returns the list of Messages in a Thread
+
+
+
+```python
+from openai import OpenAI
+
+MATH_ASSISTANT_ID = assistant.id # or a hard-coded ID like "asst-..."
+
+client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY", "<your OpenAI API key if not set as env var>"))
+
+def submit_message(assistant_id, thread, user_message):
+ client.beta.threads.messages.create(
+ thread_id=thread.id, role="user", content=user_message
+ )
+ return client.beta.threads.runs.create(
+ thread_id=thread.id,
+ assistant_id=assistant_id,
+ )
+
+
+def get_response(thread):
+ return client.beta.threads.messages.list(thread_id=thread.id, order="asc")
+```
+
+I've also defined a `create_thread_and_run` function that I can re-use (which is actually almost identical to the [`client.beta.threads.create_and_run`](https://platform.openai.com/docs/api-reference/runs/createThreadAndRun) compound function in our API ;) ). Finally, we can submit our mock user requests each to a new Thread.
+
+Notice how all of these API calls are asynchronous operations; this means we actually get async behavior in our code without the use of async libraries! (e.g. `asyncio`)
+
+
+
+```python
+def create_thread_and_run(user_input):
+ thread = client.beta.threads.create()
+ run = submit_message(MATH_ASSISTANT_ID, thread, user_input)
+ return thread, run
+
+
+# Emulating concurrent user requests
+thread1, run1 = create_thread_and_run(
+ "I need to solve the equation `3x + 11 = 14`. Can you help me?"
+)
+thread2, run2 = create_thread_and_run("Could you explain linear algebra to me?")
+thread3, run3 = create_thread_and_run("I don't like math. What can I do?")
+
+# Now all Runs are executing...
+```
+
+Once all Runs are going, we can wait on each and get the responses.
+
+
+
+```python
+import time
+
+# Pretty printing helper
+def pretty_print(messages):
+ print("# Messages")
+ for m in messages:
+ print(f"{m.role}: {m.content[0].text.value}")
+ print()
+
+
+# Waiting in a loop
+def wait_on_run(run, thread):
+ while run.status == "queued" or run.status == "in_progress":
+ run = client.beta.threads.runs.retrieve(
+ thread_id=thread.id,
+ run_id=run.id,
+ )
+ time.sleep(0.5)
+ return run
+
+
+# Wait for Run 1
+run1 = wait_on_run(run1, thread1)
+pretty_print(get_response(thread1))
+
+# Wait for Run 2
+run2 = wait_on_run(run2, thread2)
+pretty_print(get_response(thread2))
+
+# Wait for Run 3
+run3 = wait_on_run(run3, thread3)
+pretty_print(get_response(thread3))
+
+# Thank our assistant on Thread 3 :)
+run4 = submit_message(MATH_ASSISTANT_ID, thread3, "Thank you!")
+run4 = wait_on_run(run4, thread3)
+pretty_print(get_response(thread3))
+```
+
Et voilà!
You may have noticed that this code is not actually specific to our math Assistant at all... this code will work for any new Assistant you create simply by changing the Assistant ID! That is the power of the Assistants API.
+
## Tools
A key feature of the Assistants API is the ability to equip our Assistants with Tools, like Code Interpreter, Retrieval, and custom Functions. Let's take a look at each.
@@ -222,10 +359,14 @@ A key feature of the Assistants API is the ability to equip our Assistants with
Let's equip our Math Tutor with the [Code Interpreter](https://platform.openai.com/docs/assistants/tools/code-interpreter) tool, which we can do from the Dashboard...
+
![Enabling code interpreter](https://github.com/AgentOps-AI/agentops/blob/main/examples/openai_examples/images/assistants_overview_enable_code_interpreter.png)
+
...or the API, using the Assistant ID.
+
+
```python
assistant = client.beta.assistants.update(
MATH_ASSISTANT_ID,
@@ -236,6 +377,8 @@ show_json(assistant)
Now, let's ask the Assistant to use its new tool.
+
+
```python
thread, run = create_thread_and_run(
"Generate the first 20 fibbonaci numbers with code."
@@ -246,12 +389,15 @@ pretty_print(get_response(thread))
And that's it! The Assistant used Code Interpreter in the background, and gave us a final response.
-For some use cases this may be enough – however, if we want more details on what precisely an Assistant is doing we can take a look at a Run's Steps.
+For some use cases this may be enough – however, if we want more details on what precisely an Assistant is doing we can take a look at a Run's Steps.
### Steps
+
A Run is composed of one or more Steps. Like a Run, each Step has a `status` that you can query. This is useful for surfacing the progress of a Step to a user (e.g. a spinner while the Assistant is writing code or performing retrieval).
+
+
```python
run_steps = client.beta.threads.runs.steps.list(
thread_id=thread.id, run_id=run.id, order="asc"
@@ -260,6 +406,8 @@ run_steps = client.beta.threads.runs.steps.list(
Let's take a look at each Step's `step_details`.
+
+
```python
for step in run_steps.data:
step_details = step.step_details
@@ -278,12 +426,16 @@ The first Step is a `tool_calls`, specifically using the `code_interpreter` whic
The second Step is a `message_creation`, which contains the `message` that was added to the Thread to communicate the results to the user.
+
### Retrieval
Another powerful tool in the Assistants API is [Retrieval](https://platform.openai.com/docs/assistants/tools/knowledge-retrieval): the ability to upload files that the Assistant will use as a knowledge base when answering questions. This can also be enabled from the Dashboard or the API, where we can upload files we want to be used.
+
![Enabling retrieval](https://github.com/AgentOps-AI/agentops/blob/main/examples/openai_examples/images/assistants_overview_enable_retrieval.png)
+
+
```python
# Upload the file
file = client.files.create(
@@ -302,6 +454,7 @@ assistant = client.beta.assistants.update(
show_json(assistant)
```
+
```python
thread, run = create_thread_and_run(
"What are some cool math concepts behind this ML paper pdf? Explain in two sentences."
@@ -313,6 +466,7 @@ pretty_print(get_response(thread))
> **Note**
> There are more intricacies in Retrieval, like [Annotations](https://platform.openai.com/docs/assistants/how-it-works/managing-threads-and-messages), which may be covered in another cookbook.
+
### Functions
As a final powerful tool for your Assistant, you can specify custom [Functions](https://platform.openai.com/docs/assistants/tools/function-calling) (much like the [Function Calling](https://platform.openai.com/docs/guides/function-calling) in the Chat Completions API). During a Run, the Assistant can then indicate it wants to call one or more functions you specified. You are then responsible for calling the Function, and providing the output back to the Assistant.
@@ -329,6 +483,8 @@ This function will take a `title` and an array of `question`s, display the quiz,
Unfortunately I don't know how to get user input within a Python Notebook, so I'll be mocking out responses with `get_mock_response...`. This is where you'd get the user's actual input.
+
+
```python
def get_mock_response_from_user_multiple_choice():
return "a"
@@ -365,6 +521,8 @@ def display_quiz(title, questions):
Here's what a sample quiz would look like:
+
+
```python
responses = display_quiz(
"Sample Quiz",
@@ -382,6 +540,8 @@ print("Responses:", responses)
Now, let's define the interface of this function in JSON format, so our Assistant can call it:
+
+
```python
function_json = {
"name": "display_quiz",
@@ -414,11 +574,14 @@ function_json = {
Once again, let's update our Assistant either through the Dashboard or the API.
+
![Enabling custom function](https://github.com/AgentOps-AI/agentops/blob/main/examples/openai_examples/images/assistants_overview_enable_function.png)
> **Note**
> Pasting the function JSON into the Dashboard was a bit finicky due to indentation, etc. I just asked ChatGPT to format my function the same as one of the examples on the Dashboard :).
+
+
```python
assistant = client.beta.assistants.update(
MATH_ASSISTANT_ID,
@@ -432,6 +595,8 @@ show_json(assistant)
And now, we ask for a quiz.
+
+
```python
thread, run = create_thread_and_run(
"Make a quiz with 2 questions: One open ended, one multiple choice. Then, give me feedback for the responses."
@@ -442,6 +607,8 @@ run.status
Now, however, when we check the Run's `status` we see `requires_action`! Let's take a closer look.
+
+
```python
show_json(run)
```
@@ -451,6 +618,8 @@ The `required_action` field indicates a Tool is waiting for us to run it and sub
> **Note**
> While in this case we know there is only one Tool call, in practice the Assistant may choose to call multiple tools.
+
+
```python
# Extract single tool call
tool_call = run.required_action.submit_tool_outputs.tool_calls[0]
@@ -464,6 +633,8 @@ arguments
Now let's actually call our `display_quiz` function with the arguments provided by the Assistant:
+
+
```python
responses = display_quiz(arguments["title"], arguments["questions"])
print("Responses:", responses)
@@ -473,6 +644,8 @@ Great! (Remember these responses are the one's we mocked earlier. In reality, we
Now that we have our responses, let's submit them back to the Assistant. We'll need the `tool_call` ID, found in the `tool_call` we parsed out earlier. We'll also need to encode our `list`of responses into a `str`.
+
+
```python
tool_outputs = []
tool_calls = run.required_action.submit_tool_outputs.tool_calls
@@ -486,6 +659,7 @@ for tool_call in tool_calls:
})
```
+
```python
run = client.beta.threads.runs.submit_tool_outputs(
thread_id=thread.id,
@@ -497,6 +671,8 @@ show_json(run)
We can now wait for the Run to complete once again, and check our Thread!
+
+
```python
run = wait_on_run(run, thread)
pretty_print(get_response(thread))
@@ -506,6 +682,7 @@ Now let's end the AgentOps session. By default, AgentOps will end the session in
We will end the session in the "Success" state.
+
```python
agentops.end_session(end_state="Success")
```