Skip to content

OpenPanel is an open-source web and product analytics platform, an open-source alternative to Mixpanel with optional self-hosting.

feat(sdk): record OpenAI and Anthropic client calls with @openpanel/ai (#638)

Carl-Gerhard Lindesvärdfb98083b2e08
diff --git a/apps/public/content/docs/(tracking)/sdks/ai.mdx b/apps/public/content/docs/(tracking)/sdks/ai.mdx
index 475b8eeb2..c5a45e134 100644
--- a/apps/public/content/docs/(tracking)/sdks/ai.mdx
+++ b/apps/public/content/docs/(tracking)/sdks/ai.mdx
@@ -1,9 +1,11 @@
---
-title: AI (Vercel AI SDK)
-description: Record every LLM call made through the Vercel AI SDK as OpenPanel AI events.
+title: AI (Vercel AI SDK, OpenAI, Anthropic)
+description: Record every LLM call made through the Vercel AI SDK, the OpenAI client or the Anthropic client as OpenPanel AI events.
---
-`@openpanel/ai` is the OpenPanel integration for the Vercel AI SDK. It is part of the [AI observability](/docs/ai-observability) beta.
+`@openpanel/ai` records LLM calls from the Vercel AI SDK and the official OpenAI and Anthropic clients. It is part of the [AI observability](/docs/ai-observability) beta.
- [Set it up with the Vercel AI SDK](/docs/ai-observability/vercel-ai-sdk)
+- [Wrap an OpenAI client](/docs/ai-observability/openai)
+- [Wrap an Anthropic client](/docs/ai-observability/anthropic)
- [Send events yourself](/docs/ai-observability/manual-capture) with `op.ai()` or the raw payload
diff --git a/apps/public/content/docs/ai-observability/anthropic.mdx b/apps/public/content/docs/ai-observability/anthropic.mdx
new file mode 100644
index 000000000..05dbc8e37
--- /dev/null
+++ b/apps/public/content/docs/ai-observability/anthropic.mdx
@@ -0,0 +1,321 @@
+---
+title: Anthropic SDK
+description: Record every Claude call made through the official @anthropic-ai/sdk client, with cost, tokens, latency and errors tied to your users.
+---
+
+import { Callout } from 'fumadocs-ui/components/callout';
+import { Step, Steps } from 'fumadocs-ui/components/steps';
+
+<Callout type="warn" title="Beta">
+AI observability is in beta; the wrapper options and the event payload may still change. See the [overview](/docs/ai-observability) for how to give feedback.
+</Callout>
+
+`@openpanel/ai/anthropic` wraps an Anthropic client. After that, every Claude call it makes shows up in OpenPanel with its tokens, latency, messages and errors, tied to the user who triggered it. Its cost is added when the model is in the built-in price catalog.
+
+It records:
+
+- `messages.create`, streamed or not, and `beta.messages.create`
+- The SDK's helpers built on them: `messages.stream()` and `messages.parse()`
+- Extended thinking, tool use and prompt caching
+- Claude on Amazon Bedrock and Google Vertex through `@anthropic-ai/bedrock-sdk` and `@anthropic-ai/vertex-sdk`
+
+Using the Vercel AI SDK instead? Use the [Vercel AI SDK middleware](/docs/ai-observability/vercel-ai-sdk).
+
+## Quick start
+
+<Steps>
+<Step>
+
+### Install
+
+```bash
+pnpm install @openpanel/sdk @openpanel/ai @anthropic-ai/sdk
+```
+
+Tested with `@anthropic-ai/sdk` 1 and `@openpanel/sdk` 1.3.3 or later.
+
+</Step>
+<Step>
+
+### Wrap the client
+
+Wrap the client once, where you create it, and import it everywhere else as before.
+
+```ts title="lib/anthropic.ts"
+import { OpenPanel } from '@openpanel/sdk';
+import { wrapAnthropic } from '@openpanel/ai/anthropic';
+import Anthropic from '@anthropic-ai/sdk';
+
+export const op = new OpenPanel({
+ clientId: process.env.OPENPANEL_CLIENT_ID!,
+ clientSecret: process.env.OPENPANEL_CLIENT_SECRET!,
+});
+
+export const anthropic = wrapAnthropic(new Anthropic(), { op });
+```
+
+The client secret is required: AI events are sent from your server, never from the browser.
+
+</Step>
+<Step>
+
+### Say who the call is for
+
+Run the call inside `withAiContext` to attach it to a user and a feature:
+
+```ts
+import { withAiContext } from '@openpanel/ai';
+import { anthropic } from './lib/anthropic';
+
+const message = await withAiContext(
+ { profileId: user.id, name: 'ticket-triage' },
+ () =>
+ anthropic.messages.create({
+ model: 'claude-sonnet-5-5',
+ max_tokens: 1024,
+ messages: [{ role: 'user', content: ticket.body }],
+ })
+);
+```
+
+Use the same `profileId` you pass to `identify()` in the web SDK, so the call shows up on that user's profile next to everything else they did.
+
+</Step>
+<Step>
+
+### Check that it works
+
+Make one call, then open your project in OpenPanel and go to **AI**. Events arrive within a few seconds:
+
+- **Overview** shows cost, generations, tokens, latency and error rate.
+- **Calls** lists each call. Open one to read its messages, thinking, tool calls and token breakdown.
+- **Traces** groups the calls made inside one `withAiContext`.
+
+Nothing there? See [troubleshooting](#troubleshooting).
+
+</Step>
+</Steps>
+
+## Attach calls to users and features
+
+The Anthropic client has no field for OpenPanel's context, so it comes from `withAiContext`. Every call inside the callback, however deep, picks it up.
+
+| Field | What it's for |
+|---|---|
+| `profileId` | The user. Cost per user, the profile's AI widget, funnels with an AI step. |
+| `name` | The feature or step, e.g. `ticket-triage`. Breaks cost and latency down by feature. |
+| `conversationId` | Groups the traces of one chat thread. |
+| `groups` | Group ids, such as the customer's organization, for cost per account. |
+| `properties` | Anything else you want to filter or break down by, e.g. `{ plan: 'pro' }`. |
+| `deviceId` | Binds the call to an anonymous visitor's session; see [linking to sessions](#linking-to-sessions). |
+| `traceId`, `parentId` | Set these only to join a trace you started elsewhere. One is created for you otherwise. |
+| `privacyMode` | Don't send this request's messages. |
+
+Contexts nest. An inner `withAiContext` keeps the outer user and trace and overrides only what it sets, so a pipeline can name each step:
+
+```ts
+await withAiContext({ profileId: user.id, conversationId: thread.id }, async () => {
+ const outline = await withAiContext({ name: 'outline' }, () => outlineReport(data));
+ return withAiContext({ name: 'draft' }, () => draftReport(outline));
+});
+```
+
+For calls that always belong to the same thing, such as a nightly job, set `defaults` when you wrap the client instead:
+
+```ts
+const anthropic = wrapAnthropic(new Anthropic(), {
+ op,
+ defaults: { name: 'nightly-digest' },
+});
+```
+
+## Examples
+
+### Stream a response
+
+Both ways of streaming are recorded. The event is written when the stream ends, with time to first token:
+
+```ts
+// The helper
+const stream = anthropic.messages.stream({
+ model: 'claude-sonnet-5-5',
+ max_tokens: 1024,
+ messages,
+});
+stream.on('text', (text) => process.stdout.write(text));
+await stream.finalMessage();
+
+// Or the raw event stream
+const events = await anthropic.messages.create({
+ model: 'claude-sonnet-5-5',
+ max_tokens: 1024,
+ messages,
+ stream: true,
+});
+for await (const event of events) {
+ // ...
+}
+```
+
+The context is captured when the call starts, so you can return a stream from a request handler and read it after the `withAiContext` callback has returned. If the reader stops early, the event is still recorded with what arrived so far and a `stream cancelled` error.
+
+### A tool-use loop as one trace
+
+Each turn in the loop is its own generation. Because they run inside one `withAiContext`, they share a trace, and **Traces** shows the whole run with its total cost and duration. The tool calls and tool results appear in each generation's conversation:
+
+```ts
+const tools: Anthropic.Tool[] = [
+ {
+ name: 'get_weather',
+ description: 'Current weather for a city',
+ input_schema: {
+ type: 'object',
+ properties: { city: { type: 'string' } },
+ required: ['city'],
+ },
+ },
+];
+
+await withAiContext({ profileId: user.id, name: 'weather-agent' }, async () => {
+ const messages: Anthropic.MessageParam[] = [
+ { role: 'user', content: 'Should I bring a jacket in Oslo today?' },
+ ];
+ for (;;) {
+ const message = await anthropic.messages.create({
+ model: 'claude-sonnet-5-5',
+ max_tokens: 1024,
+ tools,
+ messages,
+ });
+ messages.push({ role: 'assistant', content: message.content });
+ if (message.stop_reason !== 'tool_use') {
+ return message;
+ }
+ const results: Anthropic.ToolResultBlockParam[] = [];
+ for (const block of message.content) {
+ if (block.type === 'tool_use') {
+ results.push({
+ type: 'tool_result',
+ tool_use_id: block.id,
+ content: await runTool(block),
+ });
+ }
+ }
+ messages.push({ role: 'user', content: results });
+ }
+});
+```
+
+### Extended thinking
+
+```ts
+await anthropic.messages.create({
+ model: 'claude-sonnet-5-5',
+ max_tokens: 4096,
+ thinking: { type: 'enabled', budget_tokens: 2048 },
+ messages,
+});
+```
+
+The thinking text is stored as a reasoning part ahead of the answer, so you can read why the model answered the way it did. Redacted thinking shows up as an empty reasoning part.
+
+### Prompt caching
+
+Anthropic caches a prompt only when you ask for it with `cache_control` in the request. Once you do, nothing else is needed on the OpenPanel side: Anthropic reports cache writes and cache reads apart from the other input tokens, and OpenPanel prices each at its own rate from the catalog, so a call that hits the cache shows the saving in its cost. `ai_cache_read_tokens` and `ai_cache_write_tokens` tell you how often you write and hit the cache.
+
+### Bedrock and Vertex
+
+The Bedrock and Vertex clients wrap the same way. Install the platform's client (`@anthropic-ai/bedrock-sdk` or `@anthropic-ai/vertex-sdk`), and set `provider` so the event says where the call went and the cost is looked up for that platform:
+
+```ts
+import { AnthropicBedrock } from '@anthropic-ai/bedrock-sdk';
+
+const bedrock = wrapAnthropic(
+ new AnthropicBedrock({ awsRegion: 'us-east-1' }),
+ { op, provider: 'amazon-bedrock' }
+);
+```
+
+```ts
+import { AnthropicVertex } from '@anthropic-ai/vertex-sdk';
+
+const vertex = wrapAnthropic(
+ new AnthropicVertex({ region: 'us-east5', projectId: 'my-project' }),
+ { op, provider: 'vertex' }
+);
+```
+
+## What gets recorded
+
+Each call becomes an `ai_generation` event: an ordinary OpenPanel event, so the report builder, funnels, retention and cohorts can use it. The numbers are stored as event properties:
+
+| Property | Holds |
+|---|---|
+| `ai_model`, `ai_provider` | The model that answered (e.g. `claude-sonnet-5-5-20261001`) and the provider. |
+| `ai_input_tokens`, `ai_output_tokens` | Token counts. Input includes cache reads and writes, so it is everything the model read. |
+| `ai_cache_read_tokens`, `ai_cache_write_tokens` | The cached parts of the input. |
+| `ai_cost` | USD, computed from a built-in price catalog with cache prices applied. |
+| `ai_latency_ms`, `ai_time_to_first_token_ms` | Total duration, and time to the first token on streams. |
+| `ai_stop_reason` | `stop`, `length`, `tool-calls`, `content-filter` or `other`. Claude's `end_turn` becomes `stop`, `max_tokens` becomes `length`, `tool_use` becomes `tool-calls` and `refusal` becomes `content-filter`. |
+| `ai_is_error`, `ai_http_status`, `ai_error_code` | Whether the call failed, its HTTP status, and Anthropic's error type (e.g. `overloaded_error`). |
+| `ai_stream` | Whether it was streamed. |
+| `ai_name`, `ai_conversation_id`, `ai_trace_id` | From `withAiContext`. |
+
+The system prompt, messages, thinking, tool definitions and model parameters (`max_tokens`, `stop_sequences` and so on) are kept for 30 days and shown on the generation in the dashboard. Images and documents are replaced by a `[file]` placeholder with their media type and never uploaded.
+
+A failed call, including an overloaded or rate-limited one, is recorded with its error and then rethrown, so your own error handling and retries are unchanged. Recording never blocks the model call and can never make it fail.
+
+A successful call read with `.asResponse()` is not recorded, because the wrapper would have to read the body you asked to keep unread. A failed one is still recorded with its error.
+
+## Privacy
+
+To keep prompts and completions out of OpenPanel, set `privacyMode`. Tokens, cost, latency and errors are still recorded.
+
+```ts
+// Every call from this client
+const anthropic = wrapAnthropic(new Anthropic(), { op, privacyMode: true });
+
+// Or only one request
+await withAiContext({ profileId: user.id, privacyMode: true }, () =>
+ summarizeContract(file)
+);
+```
+
+You can also turn content storage off for the whole project under **Settings → AI**.
+
+## Options
+
+| Option | What it does |
+|---|---|
+| `op` | Where events go: your `OpenPanel` client, or any object with an `ai(payload)` method. |
+| `privacyMode` | Never send messages, thinking or tool definitions. |
+| `provider` | Reported instead of `anthropic`, for Bedrock or Vertex. |
+| `defaults` | Context for calls made outside any `withAiContext`. |
+
+## Serverless functions
+
+Events are sent in the background. On Vercel, Cloudflare Workers or Lambda the function can be frozen before a send finishes. Pass an `op` that hands each send to the platform:
+
+```ts
+import { waitUntil } from '@vercel/functions';
+
+export const anthropic = wrapAnthropic(new Anthropic(), {
+ op: { ai: (payload) => waitUntil(op.ai(payload)) },
+});
+```
+
+## Linking to sessions
+
+A logged-in user's model call lands in the same session as their page views, because OpenPanel attaches a server event to that profile's current session. For anonymous visitors, send the web SDK's `op.getDeviceId()` to your backend and pass it as `deviceId` in `withAiContext`.
+
+## Troubleshooting
+
+**Nothing shows up.** Make sure the `OpenPanel` client has a `clientSecret`, and that you call the client you wrapped. `client.withOptions()` returns a new client that is not wrapped; wrap the result too. In a short script or a serverless function, see [serverless functions](#serverless-functions).
+
+**Tokens but no cost.** The model isn't in the price catalog under that name, which can happen with a new model or a platform-specific id. Check that `provider` is set; the tokens are recorded either way.
+
+**Calls don't appear on the user's profile.** The call ran outside `withAiContext`, or `profileId` differs from the id you identify the user with.
+
+**An error shows the code `Error`.** The call failed before reaching Anthropic, for example because no API key was found, so there is no HTTP status or Anthropic error type.
+
+**A streamed call shows `stream cancelled`.** Your code, or the HTTP client, stopped reading before the model finished, so the event has what arrived up to that point.
diff --git a/apps/public/content/docs/ai-observability/index.mdx b/apps/public/content/docs/ai-observability/index.mdx
index 6e7adf6cf..f841cd4bf 100644
--- a/apps/public/content/docs/ai-observability/index.mdx
+++ b/apps/public/content/docs/ai-observability/index.mdx
@@ -32,6 +32,8 @@ What sets it apart from a standalone LLM tool is that the event lands next to ev
| Your stack | Use |
|---|---|
| Vercel AI SDK (any provider) | [`@openpanel/ai/vercel` middleware](/docs/ai-observability/vercel-ai-sdk), one line per model |
+| `openai` client | [`wrapOpenAI` from `@openpanel/ai/openai`](/docs/ai-observability/openai) |
+| `@anthropic-ai/sdk` client | [`wrapAnthropic` from `@openpanel/ai/anthropic`](/docs/ai-observability/anthropic) |
| Node with our SDK | [`op.ai(payload)`](/docs/ai-observability/manual-capture) |
| Anything else (Python, Go, curl) | [the raw payload](/docs/ai-observability/manual-capture#raw-payload) on `POST /track` |
diff --git a/apps/public/content/docs/ai-observability/manual-capture.mdx b/apps/public/content/docs/ai-observability/manual-capture.mdx
index 59529f2e7..49c0ce8f4 100644
--- a/apps/public/content/docs/ai-observability/manual-capture.mdx
+++ b/apps/public/content/docs/ai-observability/manual-capture.mdx
@@ -11,7 +11,7 @@ AI observability is in beta; this payload may still change. Fields will be added
## With the Node SDK
-`@openpanel/sdk` 1.4 adds `op.ai(payload)`. The sticky profile id from `identify()` and groups from `setGroups()` are merged in, like `track()`.
+`@openpanel/sdk` 1.3.3 and later have `op.ai(payload)`. The sticky profile id from `identify()` and groups from `setGroups()` are merged in, like `track()`.
```ts
import { OpenPanel } from '@openpanel/sdk';
diff --git a/apps/public/content/docs/ai-observability/meta.json b/apps/public/content/docs/ai-observability/meta.json
index fbe5c5b64..f104f5e3b 100644
--- a/apps/public/content/docs/ai-observability/meta.json
+++ b/apps/public/content/docs/ai-observability/meta.json
@@ -1,4 +1,11 @@
{
"title": "AI observability",
- "pages": ["index", "vercel-ai-sdk", "manual-capture", "..."]
+ "pages": [
+ "index",
+ "vercel-ai-sdk",
+ "openai",
+ "anthropic",
+ "manual-capture",
+ "..."
+ ]
}
diff --git a/apps/public/content/docs/ai-observability/openai.mdx b/apps/public/content/docs/ai-observability/openai.mdx
new file mode 100644
index 000000000..63a76b8d9
--- /dev/null
+++ b/apps/public/content/docs/ai-observability/openai.mdx
@@ -0,0 +1,334 @@
+---
+title: OpenAI SDK
+description: Record every call made through the official openai client, with cost, tokens, latency and errors tied to your users.
+---
+
+import { Callout } from 'fumadocs-ui/components/callout';
+import { Step, Steps } from 'fumadocs-ui/components/steps';
+
+<Callout type="warn" title="Beta">
+AI observability is in beta; the wrapper options and the event payload may still change. See the [overview](/docs/ai-observability) for how to give feedback.
+</Callout>
+
+`@openpanel/ai/openai` wraps an `openai` client. After that, every model call it makes shows up in OpenPanel with its tokens, latency, messages and errors, tied to the user who triggered it. Its cost is added when the model is in the built-in price catalog.
+
+It records:
+
+- Chat Completions and the Responses API, streamed or not
+- Embeddings
+- The SDK's helpers built on them: `chat.completions.parse`, `chat.completions.stream`, `runTools`, `responses.parse` and `responses.stream`
+- `AzureOpenAI`, and any OpenAI-compatible API you reach through `baseURL` (OpenRouter, Groq, Together, a local server)
+
+Using the Vercel AI SDK instead? Use the [Vercel AI SDK middleware](/docs/ai-observability/vercel-ai-sdk).
+
+## Quick start
+
+<Steps>
+<Step>
+
+### Install
+
+```bash
+pnpm install @openpanel/sdk @openpanel/ai openai
+```
+
+Tested with `openai` 7 and `@openpanel/sdk` 1.3.3 or later.
+
+</Step>
+<Step>
+
+### Wrap the client
+
+Wrap the client once, where you create it, and import it everywhere else as before.
+
+```ts title="lib/openai.ts"
+import { OpenPanel } from '@openpanel/sdk';
+import { wrapOpenAI } from '@openpanel/ai/openai';
+import OpenAI from 'openai';
+
+export const op = new OpenPanel({
+ clientId: process.env.OPENPANEL_CLIENT_ID!,
+ clientSecret: process.env.OPENPANEL_CLIENT_SECRET!,
+});
+
+export const openai = wrapOpenAI(new OpenAI(), { op });
+```
+
+The client secret is required: AI events are sent from your server, never from the browser.
+
+</Step>
+<Step>
+
+### Say who the call is for
+
+Run the call inside `withAiContext` to attach it to a user and a feature:
+
+```ts
+import { withAiContext } from '@openpanel/ai';
+import { openai } from './lib/openai';
+
+const completion = await withAiContext(
+ { profileId: user.id, name: 'inbox-summary' },
+ () =>
+ openai.chat.completions.create({
+ model: 'gpt-5-mini',
+ messages: [{ role: 'user', content: 'Summarize my inbox' }],
+ })
+);
+```
+
+Use the same `profileId` you pass to `identify()` in the web SDK, so the call shows up on that user's profile next to everything else they did.
+
+</Step>
+<Step>
+
+### Check that it works
+
+Make one call, then open your project in OpenPanel and go to **AI**. Events arrive within a few seconds:
+
+- **Overview** shows cost, generations, tokens, latency and error rate.
+- **Calls** lists each call. Open one to read its messages, tool calls and token breakdown.
+- **Traces** groups the calls made inside one `withAiContext`.
+
+Nothing there? See [troubleshooting](#troubleshooting).
+
+</Step>
+</Steps>
+
+## Attach calls to users and features
+
+The OpenAI client has no field for OpenPanel's context, so it comes from `withAiContext`. Every call inside the callback, however deep, picks it up.
+
+| Field | What it's for |
+|---|---|
+| `profileId` | The user. Cost per user, the profile's AI widget, funnels with an AI step. |
+| `name` | The feature or step, e.g. `inbox-summary`. Breaks cost and latency down by feature. |
+| `conversationId` | Groups the traces of one chat thread. |
+| `groups` | Group ids, such as the customer's organization, for cost per account. |
+| `properties` | Anything else you want to filter or break down by, e.g. `{ plan: 'pro' }`. |
+| `deviceId` | Binds the call to an anonymous visitor's session; see [linking to sessions](#linking-to-sessions). |
+| `traceId`, `parentId` | Set these only to join a trace you started elsewhere. One is created for you otherwise. |
+| `privacyMode` | Don't send this request's messages. |
+
+Contexts nest. An inner `withAiContext` keeps the outer user and trace and overrides only what it sets, so a pipeline can name each step:
+
+```ts
+await withAiContext({ profileId: user.id, conversationId: thread.id }, async () => {
+ const plan = await withAiContext({ name: 'plan' }, () => planTask(task));
+ return withAiContext({ name: 'write' }, () => writeAnswer(plan));
+});
+```
+
+For calls that always belong to the same thing, such as a nightly job, set `defaults` when you wrap the client instead:
+
+```ts
+const openai = wrapOpenAI(new OpenAI(), {
+ op,
+ defaults: { name: 'nightly-digest' },
+});
+```
+
+## Examples
+
+### Stream a chat response
+
+The event is recorded when the stream ends, with time to first token. Ask for `include_usage`, or a streamed chat completion reports no tokens and no cost:
+
+```ts title="app/api/chat/route.ts"
+import { withAiContext } from '@openpanel/ai';
+import { findThreadForUser, requireUser } from '@/lib/auth';
+import { openai } from '@/lib/openai';
+
+export async function POST(request: Request) {
+ // Take the user from your auth, never from the request body.
+ const user = await requireUser(request);
+ const { messages, threadId: requestedThreadId } = await request.json();
+ // Resolve the thread against the signed-in user before grouping calls under it.
+ const threadId = await findThreadForUser(user.id, requestedThreadId);
+
+ return withAiContext(
+ { profileId: user.id, conversationId: threadId, name: 'chat' },
+ async () => {
+ const stream = await openai.chat.completions.create({
+ model: 'gpt-5-mini',
+ messages,
+ stream: true,
+ stream_options: { include_usage: true },
+ });
+ return new Response(stream.toReadableStream());
+ }
+ );
+}
+```
+
+The context is captured when the call starts, so the event keeps it even though the stream is read after the callback has returned. If the client stops reading early, the event is still recorded with the text that arrived so far and a `stream cancelled` error.
+
+### An agent loop as one trace
+
+Each model call in the loop is its own generation. Because they run inside one `withAiContext`, they share a trace, and **Traces** shows the whole run with its total cost and duration:
+
+```ts
+const tools: OpenAI.ChatCompletionTool[] = [
+ {
+ type: 'function',
+ function: {
+ name: 'get_weather',
+ description: 'Current weather for a city',
+ parameters: {
+ type: 'object',
+ properties: { city: { type: 'string' } },
+ required: ['city'],
+ },
+ },
+ },
+];
+
+await withAiContext({ profileId: user.id, name: 'weather-agent' }, async () => {
+ const messages: OpenAI.ChatCompletionMessageParam[] = [
+ { role: 'user', content: 'Should I bring a jacket in Oslo today?' },
+ ];
+ for (;;) {
+ const completion = await openai.chat.completions.create({
+ model: 'gpt-5-mini',
+ messages,
+ tools,
+ });
+ const message = completion.choices[0].message;
+ messages.push(message);
+ if (!message.tool_calls?.length) {
+ return message.content;
+ }
+ for (const call of message.tool_calls) {
+ messages.push({
+ role: 'tool',
+ tool_call_id: call.id,
+ content: await runTool(call),
+ });
+ }
+ }
+});
+```
+
+The SDK's own `runTools` helper is recorded the same way.
+
+### Responses API
+
+```ts
+const response = await openai.responses.create({
+ model: 'gpt-5',
+ instructions: 'Answer in one sentence.',
+ input: 'What does OpenPanel do?',
+ reasoning: { effort: 'low' },
+});
+```
+
+`instructions` is stored as the system message, and reasoning summaries, tool calls and tool results appear in the conversation view. Streamed Responses calls always report usage.
+
+### Embeddings
+
+```ts
+await openai.embeddings.create({
+ model: 'text-embedding-3-small',
+ input: chunks,
+});
+```
+
+This records an `ai_embedding` event with the input texts, the token count and the cost. The vectors are never sent.
+
+### OpenRouter, Azure and other compatible APIs
+
+Point the client at another API as usual, and set `provider` so the event says where the call went and the cost is looked up for that provider:
+
+```ts
+const openrouter = wrapOpenAI(
+ new OpenAI({
+ baseURL: 'https://openrouter.ai/api/v1',
+ apiKey: process.env.OPENROUTER_API_KEY,
+ }),
+ { op, provider: 'openrouter' }
+);
+```
+
+```ts
+import { AzureOpenAI } from 'openai';
+
+const azure = wrapOpenAI(
+ new AzureOpenAI({ apiVersion: '2025-04-01-preview' }),
+ { op, provider: 'azure' }
+);
+```
+
+## What gets recorded
+
+Each chat or Responses call becomes an `ai_generation` event, and each embeddings call an `ai_embedding` event. Both are ordinary OpenPanel events, so the report builder, funnels, retention and cohorts can use them. The numbers are stored as event properties:
+
+| Property | Holds |
+|---|---|
+| `ai_model`, `ai_provider` | The model that answered (e.g. `gpt-5-mini-2025-08-07`) and the provider. |
+| `ai_input_tokens`, `ai_output_tokens` | Token counts. Input includes cached tokens; output includes reasoning tokens. |
+| `ai_cache_read_tokens`, `ai_reasoning_tokens` | The cached and reasoning parts of those counts. |
+| `ai_cost` | USD, computed from a built-in price catalog with cache discounts applied. |
+| `ai_latency_ms`, `ai_time_to_first_token_ms` | Total duration, and time to the first token on streams. |
+| `ai_stop_reason` | `stop`, `length`, `tool-calls`, `content-filter`, `error` or `other`. |
+| `ai_is_error`, `ai_http_status`, `ai_error_code` | Whether the call failed, its HTTP status, and OpenAI's error code (e.g. `model_not_found`). |
+| `ai_stream` | Whether it was streamed. |
+| `ai_name`, `ai_conversation_id`, `ai_trace_id` | From `withAiContext`. |
+
+The messages, tool definitions and model parameters (`temperature`, `max_completion_tokens`, `reasoning_effort` and so on) are kept for 30 days and shown on the generation in the dashboard. Images, audio and files are replaced by a `[file]` placeholder and never uploaded.
+
+A failed call is recorded with its error and then rethrown, so your own error handling is unchanged. Recording never blocks the model call and can never make it fail.
+
+Two kinds of call are not recorded: a successful call read with `.asResponse()`, because the wrapper would have to read the body you asked to keep unread (a failed one is still recorded with its error), and the polling of a background Responses call (`background: true`). Its `create` is recorded with the queued status, but `responses.retrieve` is not wrapped.
+
+## Privacy
+
+To keep prompts and completions out of OpenPanel, set `privacyMode`. Tokens, cost, latency and errors are still recorded.
+
+```ts
+// Every call from this client
+const openai = wrapOpenAI(new OpenAI(), { op, privacyMode: true });
+
+// Or only one request
+await withAiContext({ profileId: user.id, privacyMode: true }, () =>
+ draftReply(email)
+);
+```
+
+You can also turn content storage off for the whole project under **Settings → AI**.
+
+## Options
+
+| Option | What it does |
+|---|---|
+| `op` | Where events go: your `OpenPanel` client, or any object with an `ai(payload)` method. |
+| `privacyMode` | Never send messages or tool definitions. |
+| `provider` | Reported instead of `openai`, for compatible APIs. |
+| `defaults` | Context for calls made outside any `withAiContext`. |
+
+## Serverless functions
+
+Events are sent in the background. On Vercel, Cloudflare Workers or Lambda the function can be frozen before a send finishes. Pass an `op` that hands each send to the platform:
+
+```ts
+import { waitUntil } from '@vercel/functions';
+
+export const openai = wrapOpenAI(new OpenAI(), {
+ op: { ai: (payload) => waitUntil(op.ai(payload)) },
+});
+```
+
+## Linking to sessions
+
+A logged-in user's model call lands in the same session as their page views, because OpenPanel attaches a server event to that profile's current session. For anonymous visitors, send the web SDK's `op.getDeviceId()` to your backend and pass it as `deviceId` in `withAiContext`.
+
+## Troubleshooting
+
+**Nothing shows up.** Make sure the `OpenPanel` client has a `clientSecret`, and that you call the client you wrapped. `client.withOptions()` returns a new client that is not wrapped; wrap the result too. In a short script or a serverless function, see [serverless functions](#serverless-functions).
+
+**Streamed chat calls have no tokens or cost.** Add `stream_options: { include_usage: true }`.
+
+**Tokens but no cost.** The model isn't in the price catalog under that name. This happens with some open-weight models on Groq or Together, or a provider-specific model name. Check that `provider` is set; the tokens are recorded either way.
+
+**Calls don't appear on the user's profile.** The call ran outside `withAiContext`, or `profileId` differs from the id you identify the user with.
+
+**A streamed call shows `stream cancelled`.** Your code, or the HTTP client, stopped reading before the model finished, so the event has what arrived up to that point.
diff --git a/apps/public/content/docs/ai-observability/vercel-ai-sdk.mdx b/apps/public/content/docs/ai-observability/vercel-ai-sdk.mdx
index 0c1c13f3b..c192f306c 100644
--- a/apps/public/content/docs/ai-observability/vercel-ai-sdk.mdx
+++ b/apps/public/content/docs/ai-observability/vercel-ai-sdk.mdx
@@ -9,7 +9,7 @@ import { Callout } from 'fumadocs-ui/components/callout';
AI observability is in beta; the middleware options and the event payload may still change. See the [overview](/docs/ai-observability) for how to give feedback.
</Callout>
-`@openpanel/ai/vercel` ships a middleware for `wrapLanguageModel`. The root of `@openpanel/ai` holds what every integration shares, such as `withAiContext`; provider wrappers for the OpenAI and Anthropic clients will arrive as their own subpaths. It sees every call to any provider the AI SDK supports, so one wrap covers OpenAI, Anthropic, Google, Bedrock, Mistral, OpenRouter and the rest, with no OpenTelemetry setup.
+`@openpanel/ai/vercel` ships a middleware for `wrapLanguageModel`. The root of `@openpanel/ai` holds what every integration shares, such as `withAiContext`; if you call the OpenAI or Anthropic clients directly, use the [OpenAI](/docs/ai-observability/openai) or [Anthropic](/docs/ai-observability/anthropic) wrapper instead. It sees every call to any provider the AI SDK supports, so one wrap covers OpenAI, Anthropic, Google, Bedrock, Mistral, OpenRouter and the rest, with no OpenTelemetry setup.
## Installation
diff --git a/apps/start/e2e/data-ai.spec.ts b/apps/start/e2e/data-ai.spec.ts
index 091b9a6d9..7225e6f71 100644
--- a/apps/start/e2e/data-ai.spec.ts
+++ b/apps/start/e2e/data-ai.spec.ts
@@ -87,7 +87,7 @@ test.describe('ai observability', () => {
).toBeVisible();
const cards = page.locator('.card').first();
await expect(cards).toContainText('Cost');
- await expect(cards).toContainText('Generations');
+ await expect(cards).toContainText('Calls');
await expect(cards).toContainText('Latency p95');
// The seeded SaaS project has four models; the cost widget lists them.
await expect(page.getByText('Cost by model')).toBeVisible();
@@ -115,10 +115,16 @@ test.describe('ai observability', () => {
await open(page, `${SAAS}/ai/traces?range=30d`);
await expect(tableRows(page).first()).toBeVisible();
- await expect(tableRows(page).first()).toContainText('$');
+ // A model without a catalog price shows — instead of a cost.
+ // A seeded chat trace (it has a conversation); real SDK calls in the same project come first.
+ const seededTrace = tableRows(page)
+ .filter({ hasText: 'assistant-chat' })
+ .first();
+ await expect(seededTrace).toContainText('$');
await page.screenshot({ path: `${SHOTS}/ai-traces.png`, fullPage: true });
- await tableRows(page).first().click();
+ // The profile cell is a link of its own; the start time opens the trace.
+ await seededTrace.locator('td').first().click();
await expect(page).toHaveURL(/\/ai\/traces\//);
await expect(page.getByText('Timeline')).toBeVisible();
await expect(
@@ -150,8 +156,11 @@ test.describe('ai observability', () => {
const problems = watch(page);
await open(page, `${SAAS}/ai/generations?range=30d`);
- await expect(tableRows(page).first()).toBeVisible();
- await tableRows(page).first().click();
+ const seededGeneration = tableRows(page)
+ .filter({ hasText: 'insight-summary' })
+ .first();
+ await expect(seededGeneration).toBeVisible();
+ await seededGeneration.click();
const drawer = page.getByRole('complementary', { name: 'Event details' });
await expect(drawer).toBeVisible();
await expect(drawer.getByText('Input', { exact: true })).toBeVisible();
@@ -164,7 +173,8 @@ test.describe('ai observability', () => {
// A picked model round-trips through the URL and narrows the rows.
await page.getByRole('button', { name: 'Model', exact: true }).click();
const modelItem = page.getByRole('menuitemcheckbox', {
- name: /gpt-5-mini/,
+ // Anchored: real SDK calls add dated snapshots like gpt-5-mini-2025-08-07.
+ name: /^gpt-5-mini\b(?!-)/,
});
await modelItem.click();
await expect(page).toHaveURL(/models=/);
@@ -176,7 +186,7 @@ test.describe('ai observability', () => {
await page.getByRole('button', { name: 'Errors only' }).click();
await expect(page).toHaveURL(/errors=true/);
- await expect(tableRows(page).first()).toContainText('429');
+ await expect(tableRows(page).first()).toContainText(/\b[45]\d\d\b/);
await page.screenshot({
path: `${SHOTS}/ai-generations-errors.png`,
fullPage: true,
@@ -218,7 +228,7 @@ test.describe('ai observability', () => {
await profileLink.click();
await expect(page).toHaveURL(/\/profiles\//);
await expect(page.getByText('AI usage')).toBeVisible();
- await expect(page.getByText('Generations', { exact: true })).toBeVisible();
+ await expect(page.getByText('Calls', { exact: true })).toBeVisible();
await page.screenshot({
path: `${SHOTS}/ai-profile-usage.png`,
fullPage: true,
diff --git a/apps/start/src/components/ai/ai-breakdown-widget.tsx b/apps/start/src/components/ai/ai-breakdown-widget.tsx
index 04c28fa25..6e439c74a 100644
--- a/apps/start/src/components/ai/ai-breakdown-widget.tsx
+++ b/apps/start/src/components/ai/ai-breakdown-widget.tsx
@@ -2,7 +2,8 @@ import type { IAiBreakdownRow } from '@openpanel/core';
import type { AiBreakdownDimension } from '@openpanel/core/modules/ai/ai.constants';
import { useQuery } from '@tanstack/react-query';
import type { LucideIcon } from 'lucide-react';
-import { formatCost, formatLatency, formatTokens } from './ai-format';
+import { AiCost } from './ai-cost';
+import { formatCost, formatLatencyOrNone, formatTokens } from './ai-format';
import { type AiMeasure, breakdownReportDraft } from './ai-report-draft';
import { OpenAsReportLink } from './open-as-report-link';
import { ProjectLink } from '@/components/links';
@@ -24,20 +25,20 @@ type Measure = AiMeasure;
const MEASURE_LABEL: Record<Measure, string> = {
cost: 'Cost',
- generations: 'Generations',
+ generations: 'Calls',
errors: 'Errors',
tokens: 'Tokens',
p95LatencyMs: 'p95',
};
-function formatMeasure(measure: Measure, value: number): string {
+function formatMeasure(measure: Measure, value: number | null): string {
switch (measure) {
case 'cost':
- return formatCost(value);
+ return formatCost(value ?? 0);
case 'p95LatencyMs':
- return formatLatency(value);
+ return formatLatencyOrNone(value);
default:
- return formatTokens(value);
+ return formatTokens(value ?? 0);
}
}
@@ -131,10 +132,11 @@ export function AiBreakdownWidget({
filters,
dimension,
limit,
+ sort: measure === 'errors' ? 'errors' : 'cost',
})
);
const rows = query.data ?? [];
- const max = Math.max(...rows.map((row) => row[measure]), 0) || 1;
+ const max = Math.max(...rows.map((row) => row[measure] ?? 0), 0) || 1;
return (
<Widget className={cn('col-span-6 md:col-span-3', className)}>
@@ -163,7 +165,7 @@ export function AiBreakdownWidget({
<div className="group relative px-3 py-1.5" key={row.key}>
<div
className="absolute top-0 bottom-0 left-0 rounded bg-def-200 transition-colors group-hover:bg-def-300"
- style={{ width: `${(row[measure] / max) * 100}%` }}
+ style={{ width: `${((row[measure] ?? 0) / max) * 100}%` }}
/>
<div className="row relative min-w-0 items-center justify-between gap-2 text-sm">
<RowLabel dimension={dimension} row={row} />
@@ -172,7 +174,14 @@ export function AiBreakdownWidget({
{formatMeasure(secondary, row[secondary])}
</span>
<span className="w-16 text-right font-medium">
- {formatMeasure(measure, row[measure])}
+ {measure === 'cost' ? (
+ <AiCost
+ cost={row.cost}
+ unpricedCalls={row.unpricedCalls}
+ />
+ ) : (
+ formatMeasure(measure, row[measure])
+ )}
</span>
</span>
</div>
diff --git a/apps/start/src/components/ai/ai-cost.test.ts b/apps/start/src/components/ai/ai-cost.test.ts
new file mode 100644
index 000000000..f23b0daa1
--- /dev/null
+++ b/apps/start/src/components/ai/ai-cost.test.ts
@@ -0,0 +1,66 @@
+import { describe, expect, it } from 'vitest';
+import { describeAiCost, isUnpricedCall } from './ai-cost';
+import { formatCost, formatLatencyOrNone } from './ai-format';
+
+describe('describeAiCost', () => {
+ it('shows a priced cost as is', () => {
+ expect(describeAiCost(0.0086, 0)).toEqual({ text: '$0.0086' });
+ });
+
+ it('shows a dash with a hint when nothing could be priced', () => {
+ const { text, hint } = describeAiCost(0, 2);
+ expect(text).toBe('—');
+ expect(hint).toContain('Cost unknown');
+ });
+
+ it('shows a partly priced total as a lower bound that never rounds up', () => {
+ expect(describeAiCost(0.016, 1).text).toBe('$0.01+');
+ expect(describeAiCost(0.002_39, 3).text).toBe('$0.0023+');
+ expect(describeAiCost(0.016, 1).hint).toContain('1 call has');
+ expect(describeAiCost(0.29, 1).text).toBe('$0.29+');
+ });
+
+ it('shows a dash when the priced part is too small to show', () => {
+ expect(describeAiCost(4e-8, 1).text).toBe('—');
+ });
+});
+
+describe('isUnpricedCall', () => {
+ it('counts only successful model calls without a price', () => {
+ expect(
+ isUnpricedCall({ type: 'generation', isError: false, costSource: 'none' })
+ ).toBe(true);
+ expect(
+ isUnpricedCall({ type: 'generation', isError: true, costSource: 'none' })
+ ).toBe(false);
+ expect(
+ isUnpricedCall({ type: 'span', isError: false, costSource: 'none' })
+ ).toBe(false);
+ expect(
+ isUnpricedCall({
+ type: 'embedding',
+ isError: false,
+ costSource: 'catalog',
+ })
+ ).toBe(false);
+ });
+});
+
+describe('formatCost rounding', () => {
+ it('rounds to nearest by default and down on request', () => {
+ expect(formatCost(0.016)).toBe('$0.02');
+ expect(formatCost(0.016, 'down')).toBe('$0.01');
+ expect(formatCost(0.000_529, 'down')).toBe('$0.00052');
+ expect(formatCost(0.29, 'down')).toBe('$0.29');
+ expect(formatCost(0.000_29, 'down')).toBe('$0.00029');
+ expect(formatCost(0.289_999_999_999_99, 'down')).toBe('$0.28');
+ expect(formatCost(1.5, 'down')).toBe('$1.50');
+ });
+});
+
+describe('formatLatencyOrNone', () => {
+ it('shows a dash when latency was not measured', () => {
+ expect(formatLatencyOrNone(null)).toBe('—');
+ expect(formatLatencyOrNone(0)).toBe('0ms');
+ });
+});
diff --git a/apps/start/src/components/ai/ai-cost.tsx b/apps/start/src/components/ai/ai-cost.tsx
new file mode 100644
index 000000000..e00cb8d42
--- /dev/null
+++ b/apps/start/src/components/ai/ai-cost.tsx
@@ -0,0 +1,59 @@
+import type { IServiceAiEvent } from '@openpanel/core';
+import { isAiModelCall } from '@openpanel/core/modules/ai/ai.constants';
+import { formatCost, SMALLEST_SHOWN_COST } from './ai-format';
+import { Tooltiper } from '@/components/ui/tooltip';
+
+const UNKNOWN_COST_HINT =
+ 'Cost unknown: the model has no price in the price catalog, or the provider reported no token usage.';
+
+/** A successful call we could not price: its $0 means unknown, not free. */
+export function isUnpricedCall(
+ event: Pick<IServiceAiEvent, 'type' | 'isError' | 'costSource'>
+): boolean {
+ return (
+ isAiModelCall(event.type) && !event.isError && event.costSource === 'none'
+ );
+}
+
+/** The text and hint for a cost that may include calls without a known price. */
+export function describeAiCost(
+ cost: number,
+ unpricedCalls: number
+): { text: string; hint?: string } {
+ if (unpricedCalls === 0) {
+ return { text: formatCost(cost) };
+ }
+ // Below the smallest shown amount, `<$0.000001+` would claim an upper and a lower bound at once.
+ if (cost < SMALLEST_SHOWN_COST) {
+ return { text: '—', hint: UNKNOWN_COST_HINT };
+ }
+ const calls =
+ unpricedCalls === 1 ? '1 call has' : `${unpricedCalls} calls have`;
+ return {
+ text: `${formatCost(cost, 'down')}+`,
+ hint: `At least this much: ${calls} no known cost.`,
+ };
+}
+
+export function AiCost({
+ cost,
+ unpricedCalls,
+}: {
+ cost: number;
+ unpricedCalls: number;
+}) {
+ const { text, hint } = describeAiCost(cost, unpricedCalls);
+ if (!hint) {
+ return <>{text}</>;
+ }
+ return (
+ <Tooltiper asChild content={hint}>
+ <span
+ className="cursor-help text-muted-foreground underline decoration-dotted underline-offset-2"
+ tabIndex={0}
+ >
+ {text}
+ </span>
+ </Tooltiper>
+ );
+}
diff --git a/apps/start/src/components/ai/ai-event-detail.tsx b/apps/start/src/components/ai/ai-event-detail.tsx
index b08bffb2d..b1ddb07e1 100644
--- a/apps/start/src/components/ai/ai-event-detail.tsx
+++ b/apps/start/src/components/ai/ai-event-detail.tsx
@@ -1,7 +1,9 @@
import type { IServiceAiEvent } from '@openpanel/core';
+import { isAiModelCall } from '@openpanel/core/modules/ai/ai.constants';
import { AlertTriangleIcon } from 'lucide-react';
import { useState } from 'react';
import { AiContent } from './ai-conversation';
+import { AiCost, isUnpricedCall } from './ai-cost';
import {
AI_TYPE_LABELS,
formatCost,
@@ -52,8 +54,16 @@ export function AiEventStats({ event }: { event: IServiceAiEvent }) {
event.cacheReadTokens > 0
? { label: 'Cached', value: formatTokens(event.cacheReadTokens) }
: null,
- event.totalCost > 0 || event.costSource !== 'none'
- ? { label: 'Cost', value: formatCost(event.totalCost) }
+ event.totalCost > 0 || event.costSource !== 'none' || isUnpricedCall(event)
+ ? {
+ label: 'Cost',
+ value: (
+ <AiCost
+ cost={event.totalCost}
+ unpricedCalls={isUnpricedCall(event) ? 1 : 0}
+ />
+ ),
+ }
: null,
{ label: 'Latency', value: formatLatency(event.latencyMs) },
event.timeToFirstTokenMs > 0
@@ -85,7 +95,7 @@ export function AiEventDetail({
showTraceLink?: boolean;
}) {
const [tab, setTab] = useState<TabId>('conversation');
- const isModelCall = event.type === 'generation' || event.type === 'embedding';
+ const isModelCall = isAiModelCall(event.type);
const errorRecord =
typeof event.error === 'object' && event.error !== null
? (event.error as Record<string, unknown>)
diff --git a/apps/start/src/components/ai/ai-format.test.ts b/apps/start/src/components/ai/ai-format.test.ts
new file mode 100644
index 000000000..6bc1d69e1
--- /dev/null
+++ b/apps/start/src/components/ai/ai-format.test.ts
@@ -0,0 +1,18 @@
+import { describe, expect, it } from 'vitest';
+import { formatCost } from './ai-format';
+
+describe('formatCost', () => {
+ it('keeps two significant digits for sub-cent calls', () => {
+ expect(formatCost(0.000_052)).toBe('$0.000052');
+ expect(formatCost(0.0086)).toBe('$0.0086');
+ });
+
+ it('never prints exponent notation for a few-token embedding', () => {
+ expect(formatCost(4e-8)).toBe('<$0.000001');
+ });
+
+ it('rounds whole amounts to cents', () => {
+ expect(formatCost(0)).toBe('$0');
+ expect(formatCost(12.345)).toBe('$12.35');
+ });
+});
diff --git a/apps/start/src/components/ai/ai-format.ts b/apps/start/src/components/ai/ai-format.ts
index 7b02cb9c4..00dc0b36c 100644
--- a/apps/start/src/components/ai/ai-format.ts
+++ b/apps/start/src/components/ai/ai-format.ts
@@ -1,19 +1,51 @@
// The dashboard formats numbers in one locale (see `useNumber`), not the browser's.
const NUMBER_LOCALE = 'en-US';
const SUB_CENT = 0.01;
+const COST_SIGNIFICANT_DIGITS = 2;
+// Below this a few-token embedding prints as $4e-8 or a digit string too wide for a table cell.
+export const SMALLEST_SHOWN_COST = 0.000_001;
const SECOND_MS = 1000;
const MINUTE_MS = 60_000;
const COMPACT_THRESHOLD = 10_000;
-/** USD. LLM calls cost fractions of a cent, so two decimals would read as $0.00. */
-export function formatCost(value: number): string {
+const CENT_DECIMALS = 2;
+
+/**
+ * Drops digits past the shown precision instead of rounding them up. Works on
+ * the decimal string: `0.29 * 100` is 28.999… in binary and would floor to 0.28.
+ */
+function floorToPrecision(value: number): number {
+ const [whole = '0', fraction = ''] = String(value).split('.');
+ const leadingZeros = fraction.match(/^0*/)?.[0].length ?? 0;
+ const decimals =
+ value >= SUB_CENT ? CENT_DECIMALS : leadingZeros + COST_SIGNIFICANT_DIGITS;
+ return Number(`${whole}.${fraction.slice(0, decimals) || '0'}`);
+}
+
+/**
+ * USD. LLM calls cost fractions of a cent, so two decimals would read as $0.00.
+ * `down` keeps a lower bound from claiming more than was priced.
+ */
+export function formatCost(
+ value: number,
+ rounding: 'nearest' | 'down' = 'nearest'
+): string {
if (!value) {
return '$0';
}
+ if (Math.abs(value) < SMALLEST_SHOWN_COST) {
+ return '<$0.000001';
+ }
+ const shown = rounding === 'down' ? floorToPrecision(value) : value;
if (Math.abs(value) < SUB_CENT) {
- return `$${Number(value.toPrecision(2))}`;
+ return `$${Number(shown.toPrecision(COST_SIGNIFICANT_DIGITS))}`;
}
- return `$${value.toFixed(2)}`;
+ return `$${shown.toFixed(2)}`;
+}
+
+/** No chat calls in range means latency was not measured, not that it was 0ms. */
+export function formatLatencyOrNone(ms: number | null): string {
+ return ms === null ? '—' : formatLatency(ms);
}
export function formatLatency(ms: number): string {
diff --git a/apps/start/src/components/ai/ai-generations-table.tsx b/apps/start/src/components/ai/ai-generations-table.tsx
index 6acdd6b7b..890c46aa7 100644
--- a/apps/start/src/components/ai/ai-generations-table.tsx
+++ b/apps/start/src/components/ai/ai-generations-table.tsx
@@ -4,12 +4,8 @@ import type { ColumnDef } from '@tanstack/react-table';
import { getCoreRowModel, useReactTable } from '@tanstack/react-table';
import { AlertTriangleIcon, BotIcon, CheckIcon } from 'lucide-react';
import { useMemo } from 'react';
-import {
- AI_TYPE_LABELS,
- formatCost,
- formatLatency,
- formatTokens,
-} from './ai-format';
+import { AiCost, isUnpricedCall } from './ai-cost';
+import { AI_TYPE_LABELS, formatLatency, formatTokens } from './ai-format';
import { ProfileCell } from './ai-traces-table';
import { ColumnCreatedAt } from '@/components/column-created-at';
import { useEventDrawer } from '@/components/events/use-event-drawer';
@@ -113,7 +109,10 @@ function useColumns(): ColumnDef<IServiceAiEvent>[] {
size: 100,
cell: ({ row }) => (
<span className="font-mono tabular-nums">
- {formatCost(row.original.totalCost)}
+ <AiCost
+ cost={row.original.totalCost}
+ unpricedCalls={isUnpricedCall(row.original) ? 1 : 0}
+ />
</span>
),
},
@@ -185,7 +184,7 @@ export function AiGenerationsTable({ projectId }: { projectId: string }) {
<div className="col gap-3">
<DataTable
empty={{
- title: 'No generations',
+ title: 'No calls',
description: 'Model calls appear here once your app sends AI events.',
}}
error={query.isError}
diff --git a/apps/start/src/components/ai/ai-metrics.tsx b/apps/start/src/components/ai/ai-metrics.tsx
index 010ffb62b..dcd120000 100644
--- a/apps/start/src/components/ai/ai-metrics.tsx
+++ b/apps/start/src/components/ai/ai-metrics.tsx
@@ -46,7 +46,7 @@ const METRICS: MetricDefinition[] = [
},
{
key: 'generations',
- title: 'Generations',
+ title: 'Calls',
unit: '',
inverted: false,
format: formatTokens,
diff --git a/apps/start/src/components/ai/ai-trace-tree.tsx b/apps/start/src/components/ai/ai-trace-tree.tsx
index 918d32d30..9106c7fa1 100644
--- a/apps/start/src/components/ai/ai-trace-tree.tsx
+++ b/apps/start/src/components/ai/ai-trace-tree.tsx
@@ -7,7 +7,8 @@ import {
WorkflowIcon,
} from 'lucide-react';
import { useMemo } from 'react';
-import { formatCost, formatLatency, formatTokens } from './ai-format';
+import { AiCost, isUnpricedCall } from './ai-cost';
+import { formatLatency, formatTokens } from './ai-format';
import { cn } from '@/utils/cn';
interface TraceNode {
@@ -150,8 +151,14 @@ export function AiTraceTree({ events, selectedId, onSelect }: Props) {
· {formatTokens(event.inputTokens + event.outputTokens)}t
</span>
)}
- {event.totalCost > 0 && (
- <span className="ml-1">· {formatCost(event.totalCost)}</span>
+ {(event.totalCost > 0 || isUnpricedCall(event)) && (
+ <span className="ml-1">
+ ·{' '}
+ <AiCost
+ cost={event.totalCost}
+ unpricedCalls={isUnpricedCall(event) ? 1 : 0}
+ />
+ </span>
)}
</span>
</div>
diff --git a/apps/start/src/components/ai/ai-traces-table.tsx b/apps/start/src/components/ai/ai-traces-table.tsx
index 8b89cf8ad..4fbdd7bd4 100644
--- a/apps/start/src/components/ai/ai-traces-table.tsx
+++ b/apps/start/src/components/ai/ai-traces-table.tsx
@@ -5,7 +5,8 @@ import type { ColumnDef } from '@tanstack/react-table';
import { getCoreRowModel, useReactTable } from '@tanstack/react-table';
import { AlertTriangleIcon, BotIcon, CheckIcon } from 'lucide-react';
import { useMemo } from 'react';
-import { formatCost, formatLatency, formatTokens } from './ai-format';
+import { AiCost } from './ai-cost';
+import { formatLatency, formatTokens } from './ai-format';
import { ColumnCreatedAt } from '@/components/column-created-at';
import { ProjectLink } from '@/components/links';
import { useOverviewOptions } from '@/components/overview/useOverviewOptions';
@@ -146,7 +147,10 @@ function useColumns(): ColumnDef<IServiceAiTrace>[] {
size: 100,
cell: ({ row }) => (
<span className="font-mono tabular-nums">
- {formatCost(row.original.cost)}
+ <AiCost
+ cost={row.original.cost}
+ unpricedCalls={row.original.unpricedCalls}
+ />
</span>
),
},
diff --git a/apps/start/src/components/ai/ai-usage-widget.tsx b/apps/start/src/components/ai/ai-usage-widget.tsx
index 5e5527ff9..d3be1a389 100644
--- a/apps/start/src/components/ai/ai-usage-widget.tsx
+++ b/apps/start/src/components/ai/ai-usage-widget.tsx
@@ -1,7 +1,8 @@
import type { IAiEntitySummary } from '@openpanel/core';
import { type UseQueryResult, useQuery } from '@tanstack/react-query';
import { BotIcon } from 'lucide-react';
-import { formatCost, formatLatency, formatTokens } from './ai-format';
+import { AiCost } from './ai-cost';
+import { formatLatencyOrNone, formatTokens } from './ai-format';
import { ProjectLink } from '@/components/links';
import { WidgetHead, WidgetTitle } from '@/components/overview/overview-widget';
import { Skeleton } from '@/components/skeleton';
@@ -121,13 +122,19 @@ function UsageBody({
const stats = data
? [
- { label: 'Generations', value: formatTokens(data.generations) },
- { label: 'Cost', value: formatCost(data.cost) },
+ { label: 'Calls', value: formatTokens(data.generations) },
+ {
+ label: 'Cost',
+ value: <AiCost cost={data.cost} unpricedCalls={data.unpricedCalls} />,
+ },
{
label: 'Tokens',
value: formatTokens(data.inputTokens + data.outputTokens),
},
- { label: 'Avg latency', value: formatLatency(data.avgLatencyMs) },
+ {
+ label: 'Avg latency',
+ value: formatLatencyOrNone(data.avgLatencyMs),
+ },
{ label: 'Errors', value: formatTokens(data.errors) },
{ label: 'Last call', value: data.lastAt ? timeAgo(data.lastAt) : '—' },
]
diff --git a/apps/start/src/routes/_app.$organizationId.$projectId.ai._tabs.tsx b/apps/start/src/routes/_app.$organizationId.$projectId.ai._tabs.tsx
index 7bc79dd62..4114f1bee 100644
--- a/apps/start/src/routes/_app.$organizationId.$projectId.ai._tabs.tsx
+++ b/apps/start/src/routes/_app.$organizationId.$projectId.ai._tabs.tsx
@@ -26,7 +26,7 @@ function Component() {
const { activeTab, tabs } = usePageTabs([
{ id: 'ai', label: 'Overview' },
{ id: 'traces', label: 'Traces' },
- { id: 'generations', label: 'Generations' },
+ { id: 'generations', label: 'Calls' },
]);
const handleTabChange = (tabId: string) => {
diff --git a/apps/start/src/routes/_app.$organizationId.$projectId.ai_.traces.$traceId.tsx b/apps/start/src/routes/_app.$organizationId.$projectId.ai_.traces.$traceId.tsx
index d539c5ec3..e7bb620be 100644
--- a/apps/start/src/routes/_app.$organizationId.$projectId.ai_.traces.$traceId.tsx
+++ b/apps/start/src/routes/_app.$organizationId.$projectId.ai_.traces.$traceId.tsx
@@ -1,13 +1,11 @@
+import { isAiModelCall } from '@openpanel/core/modules/ai/ai.constants';
import { useSuspenseQuery } from '@tanstack/react-query';
import { createFileRoute } from '@tanstack/react-router';
import { useState } from 'react';
import { z } from 'zod';
+import { AiCost, isUnpricedCall } from '@/components/ai/ai-cost';
import { AiEventDetail } from '@/components/ai/ai-event-detail';
-import {
- formatCost,
- formatLatency,
- formatTokens,
-} from '@/components/ai/ai-format';
+import { formatLatency, formatTokens } from '@/components/ai/ai-format';
import { AiTraceTree, buildTraceNodes } from '@/components/ai/ai-trace-tree';
import { ProfileCell } from '@/components/ai/ai-traces-table';
import FullPageLoadingState from '@/components/full-page-loading-state';
@@ -64,10 +62,11 @@ function Component() {
(acc, event) => ({
cost: acc.cost + event.totalCost,
tokens: acc.tokens + event.inputTokens + event.outputTokens,
- generations: acc.generations + (event.type === 'generation' ? 1 : 0),
+ generations: acc.generations + (isAiModelCall(event.type) ? 1 : 0),
errors: acc.errors + (event.isError ? 1 : 0),
+ unpricedCalls: acc.unpricedCalls + (isUnpricedCall(event) ? 1 : 0),
}),
- { cost: 0, tokens: 0, generations: 0, errors: 0 }
+ { cost: 0, tokens: 0, generations: 0, errors: 0, unpricedCalls: 0 }
);
const root = events.find((event) => event.type === 'trace');
const name =
@@ -114,9 +113,13 @@ function Component() {
</div>
<div className="row flex-wrap gap-x-5 gap-y-1 font-mono text-xs tabular-nums">
<span>{formatLatency(totalMs)}</span>
- <span>{totals.generations} generations</span>
+ <span>
+ {totals.generations} {totals.generations === 1 ? 'call' : 'calls'}
+ </span>
<span>{formatTokens(totals.tokens)} tokens</span>
- <span>{formatCost(totals.cost)}</span>
+ <span>
+ <AiCost cost={totals.cost} unpricedCalls={totals.unpricedCalls} />
+ </span>
{totals.errors > 0 && (
<span className="text-destructive">{totals.errors} errors</span>
)}
diff --git a/bun.lock b/bun.lock
index 6ee9a56d7..4c4c0c444 100644
--- a/bun.lock
+++ b/bun.lock
@@ -484,9 +484,11 @@
"@openpanel/sdk": "workspace:1.3.3-local",
},
"devDependencies": {
+ "@anthropic-ai/sdk": "1.3.0",
"@openpanel/tsconfig": "workspace:*",
"@types/node": "catalog:",
"bun-types": "1.4.0",
+ "openai": "7.47.0",
"tsup": "7.3.0",
"typescript": "catalog:",
},
@@ -741,6 +743,8 @@
"@ampproject/remapping": ["@ampproject/[email protected]", "", { "dependencies": { "@jridgewell/gen-mapping": "^0.3.5", "@jridgewell/trace-mapping": "^0.3.24" } }, "sha512-30iZtAPgz+LTIYoeivqYo853f02jBYSd5uGnGpkFV0M3xOt9aN73erkgYAmZU43x4VfqcnLxW9Kpg3R5LC4YYw=="],
+ "@anthropic-ai/sdk": ["@anthropic-ai/[email protected]", "", { "dependencies": { "@opentelemetry/api": "^1.4.1", "json-schema-to-ts": "^3.1.1", "standardwebhooks": "^1.0.0" }, "peerDependencies": { "zod": "^3.25.0 || ^4.0.0" }, "optionalPeers": ["zod"], "bin": { "anthropic-ai-sdk": "bin/cli" } }, "sha512-k7z9BlLLPxfZH+ov7dS6EnaN/wiZeV8YNT1LZfrpAHIy7oY7pHr4UzirF9f2OlHtFEnRBOX+AZQ91Y5fK+TPTw=="],
+
"@asamuzakjp/css-color": ["@asamuzakjp/[email protected]", "", { "dependencies": { "@csstools/css-calc": "^2.1.3", "@csstools/css-color-parser": "^3.0.9", "@csstools/css-parser-algorithms": "^3.0.4", "@csstools/css-tokenizer": "^3.0.3", "lru-cache": "^10.4.3" } }, "sha512-K1A6z8tS3XsmCMM86xoWdn7Fkdn9m6RSVtocUrJYIwZnFVkng/PvkEoWtOWmP+Scc6saYWHWZYbndEEXxl24jw=="],
"@ast-grep/napi": ["@ast-grep/[email protected]", "", { "optionalDependencies": { "@ast-grep/napi-darwin-arm64": "0.40.5", "@ast-grep/napi-darwin-x64": "0.40.5", "@ast-grep/napi-linux-arm64-gnu": "0.40.5", "@ast-grep/napi-linux-arm64-musl": "0.40.5", "@ast-grep/napi-linux-x64-gnu": "0.40.5", "@ast-grep/napi-linux-x64-musl": "0.40.5", "@ast-grep/napi-win32-arm64-msvc": "0.40.5", "@ast-grep/napi-win32-ia32-msvc": "0.40.5", "@ast-grep/napi-win32-x64-msvc": "0.40.5" } }, "sha512-hJA62OeBKUQT68DD2gDyhOqJxZxycqg8wLxbqjgqSzYttCMSDL9tiAQ9abgekBYNHudbJosm9sWOEbmCDfpX2A=="],
@@ -4377,6 +4381,8 @@
"json-schema": ["[email protected]", "", {}, "sha512-es94M3nTIfsEPisRafak+HDLfHXnKBhV3vU5eqPcS3flIWqcxJWgXHXiey3YrpaNsanY5ei1VoYEbOzijuq9BA=="],
+ "json-schema-to-ts": ["[email protected]", "", { "dependencies": { "@babel/runtime": "^7.18.3", "ts-algebra": "^2.0.0" } }, "sha512-+DWg8jCJG2TEnpy7kOm/7/AxaYoaRbjVB4LFZLySZlWn8exGs3A4OLJR966cVvU26N7X9TWxl+Jsw7dzAqKT6g=="],
+
"json-schema-traverse": ["[email protected]", "", {}, "sha512-NM8/P9n3XjXhIZn1lLhkFaACTOURQXjWhV4BA/RnOv8xvgqtqpAX9IO4mRQxSx1Rlo4tqzeqb0sOlruaOy3dug=="],
"json-schema-typed": ["[email protected]", "", {}, "sha512-fQhoXdcvc3V28x7C7BMs4P5+kNlgUURe2jmUT1T//oBRMDrqy1QPelJimwZGo7Hg9VPV3EQV5Bnq4hbFy2vetA=="],
@@ -4883,7 +4889,7 @@
"open": ["[email protected]", "", { "dependencies": { "define-lazy-prop": "^2.0.0", "is-docker": "^2.1.1", "is-wsl": "^2.2.0" } }, "sha512-7x81NCL719oNbsq/3mh+hVrAWmFuEYUqrq/Iw3kUzH8ReypT9QQ0BLoJS7/G9k6N81XjW4qHWtjWwe/9eLy1EQ=="],
- "openai": ["[email protected]", "", { "peerDependencies": { "@aws-sdk/credential-provider-node": ">=3.972.0 <4", "@smithy/hash-node": ">=4.3.0 <5", "@smithy/signature-v4": ">=5.4.0 <6", "undici": ">=5 <9", "ws": "^8.21.0", "zod": "^3.25 || ^4.0" }, "optionalPeers": ["@aws-sdk/credential-provider-node", "@smithy/hash-node", "@smithy/signature-v4", "undici", "ws", "zod"] }, "sha512-cnk0d4QT58Vq0vl2KG+DCa/C2+cXthsK2rUeaqtprq33mKv9dlX8o7fWSS283wa2IE/aVdsRyX1+Dqwz551olg=="],
+ "openai": ["[email protected]", "", { "peerDependencies": { "@aws-sdk/credential-provider-node": ">=3.972.0 <4", "@smithy/hash-node": ">=4.3.0 <5", "@smithy/signature-v4": ">=5.4.0 <6", "google-auth-library": ">=9.15.1 <12", "undici": ">=5 <9", "ws": "^8.21.0", "zod": "^3.25 || ^4.0" }, "optionalPeers": ["@aws-sdk/credential-provider-node", "@smithy/hash-node", "@smithy/signature-v4", "google-auth-library", "undici", "ws", "zod"] }, "sha512-BYa2RYgocAv9WjPUJmNu8fICuA1fFhkTBEpiR3sC5L1K6v8Hnhc8pcnQEhu8l/+W1XKA+JUaBa7HpT+AK1s77w=="],
"openapi-sampler": ["[email protected]", "", { "dependencies": { "@types/json-schema": "^7.0.7", "fast-xml-parser": "^5.5.1", "json-pointer": "0.6.2" } }, "sha512-zpnod1EYNkMP3xg18bhMknvzdaxz5ePUyHakO/7zwEIq/Y8T0SsliS9a+YXCWQCvn+5LfE//UtDVoUULcKdVAQ=="],
@@ -5691,6 +5697,8 @@
"trough": ["[email protected]", "", {}, "sha512-tmMpK00BjZiUyVyvrBK7knerNgmgvcV/KLVyuma/SC+TQN167GrMRciANTz09+k3zW8L8t60jWO1GpfkZdjTaw=="],
+ "ts-algebra": ["[email protected]", "", {}, "sha512-FPAhNPFMrkwz76P7cdjdmiShwMynZYN6SgOujD1urY4oNm80Ou9oMdmbR45LotcKOXoy7wSmHkRFE6Mxbrhefw=="],
+
"ts-interface-checker": ["[email protected]", "", {}, "sha512-Y/arvbn+rrz3JCKl9C4kVNfTfSm2/mEp5FSz5EsZSANGPSlQrpRI5M4PKF+mJnE52jOO90PnPSc3Ur3bTQw0gA=="],
"ts-toolbelt": ["[email protected]", "", {}, "sha512-nsZd8ZeNUzukXPlJmTBwUAuABDe/9qtVDelJeT/qW0ow3ZS3BsQJtNkan1802aM9Uf68/Y8ljw86Hu0h5IUW3w=="],
@@ -6347,6 +6355,8 @@
"@openpanel/release/bun-types": ["[email protected]", "", { "dependencies": { "@types/node": "*", "undici-types": "*" } }, "sha512-yjrJh5oGX7mMPKGd19YXyIjmEuj4G4zNC30bs7PM9Az7NQ3ndF3j6hbeRdvJTRQEhJ/FYQwW0a2moweEi2hMxQ=="],
+ "@openpanel/release/openai": ["[email protected]", "", { "peerDependencies": { "@aws-sdk/credential-provider-node": ">=3.972.0 <4", "@smithy/hash-node": ">=4.3.0 <5", "@smithy/signature-v4": ">=5.4.0 <6", "undici": ">=5 <9", "ws": "^8.21.0", "zod": "^3.25 || ^4.0" }, "optionalPeers": ["@aws-sdk/credential-provider-node", "@smithy/hash-node", "@smithy/signature-v4", "undici", "ws", "zod"] }, "sha512-cnk0d4QT58Vq0vl2KG+DCa/C2+cXthsK2rUeaqtprq33mKv9dlX8o7fWSS283wa2IE/aVdsRyX1+Dqwz551olg=="],
+
"@openpanel/seed/bun-types": ["[email protected]", "", { "dependencies": { "@types/node": "*", "undici-types": "*" } }, "sha512-yjrJh5oGX7mMPKGd19YXyIjmEuj4G4zNC30bs7PM9Az7NQ3ndF3j6hbeRdvJTRQEhJ/FYQwW0a2moweEi2hMxQ=="],
"@opentelemetry/core/@opentelemetry/semantic-conventions": ["@opentelemetry/[email protected]", "", {}, "sha512-lp4qAiMTD4sNWW4DbKLBkfiMZ4jbAboJIGOQr5DvciMRI494OapieI9qiODpOt0XBr1LjIDy1xAGAnVs5supTA=="],
diff --git a/packages/core/src/modules/ai/ai.constants.ts b/packages/core/src/modules/ai/ai.constants.ts
index c88776c12..1510e3e2a 100644
--- a/packages/core/src/modules/ai/ai.constants.ts
+++ b/packages/core/src/modules/ai/ai.constants.ts
@@ -182,6 +182,11 @@ export const zAiModelParameters = z
const MODEL_CALL_KINDS: readonly AiEventKind[] = ['generation', 'embedding'];
+/** A call billed by a provider, as opposed to a span or trace that groups calls. */
+export function isAiModelCall(kind: string): boolean {
+ return (MODEL_CALL_KINDS as readonly string[]).includes(kind);
+}
+
export const zAiPayload = z
.object({
kind: z.enum(AI_EVENT_KINDS),
@@ -218,7 +223,7 @@ export const zAiPayload = z
message: 'parentId must not be the span itself',
});
}
- if (!MODEL_CALL_KINDS.includes(payload.kind)) {
+ if (!isAiModelCall(payload.kind)) {
return;
}
for (const field of ['model', 'provider'] as const) {
@@ -265,6 +270,10 @@ export type AiBreakdownDimension = (typeof AI_BREAKDOWN_DIMENSIONS)[number];
export const AI_LIST_DEFAULT_TAKE = 50;
export const AI_LIST_MAX_TAKE = 500;
+/** `errors` ranks by error count and drops rows without any. */
+export const AI_BREAKDOWN_SORTS = ['cost', 'errors'] as const;
+export type AiBreakdownSort = (typeof AI_BREAKDOWN_SORTS)[number];
+
export const AI_BREAKDOWN_DEFAULT_LIMIT = 10;
export const AI_BREAKDOWN_MAX_LIMIT = 100;
diff --git a/packages/core/src/modules/ai/ai.rpc.ts b/packages/core/src/modules/ai/ai.rpc.ts
index 2ed18e1ed..bf632ba2d 100644
--- a/packages/core/src/modules/ai/ai.rpc.ts
+++ b/packages/core/src/modules/ai/ai.rpc.ts
@@ -12,6 +12,7 @@ import {
AI_BREAKDOWN_DEFAULT_LIMIT,
AI_BREAKDOWN_DIMENSIONS,
AI_BREAKDOWN_MAX_LIMIT,
+ AI_BREAKDOWN_SORTS,
AI_LIST_DEFAULT_TAKE,
AI_LIST_MAX_TAKE,
zAiFilters,
@@ -88,6 +89,7 @@ export const aiRouter = createTRPCRouter({
.min(1)
.max(AI_BREAKDOWN_MAX_LIMIT)
.default(AI_BREAKDOWN_DEFAULT_LIMIT),
+ sort: z.enum(AI_BREAKDOWN_SORTS).default('cost'),
})
)
.query(async ({ input, ctx }) => {
@@ -103,6 +105,7 @@ export const aiRouter = createTRPCRouter({
timezone,
dimension: input.dimension,
limit: input.limit,
+ sort: input.sort,
filters: input.filters,
});
}),
diff --git a/packages/core/src/modules/ai/ai.service.ts b/packages/core/src/modules/ai/ai.service.ts
index acd8b382e..e9ef980a7 100644
--- a/packages/core/src/modules/ai/ai.service.ts
+++ b/packages/core/src/modules/ai/ai.service.ts
@@ -14,6 +14,7 @@ import {
import type { IInterval } from '../report/report.constants';
import type {
AiBreakdownDimension,
+ AiBreakdownSort,
AiEventKind,
IAiFilters,
} from './ai.constants';
@@ -102,6 +103,7 @@ export interface IServiceAiTrace {
inputTokens: number;
outputTokens: number;
cost: number;
+ unpricedCalls: number;
isError: boolean;
models: string[];
groups: string[];
@@ -139,9 +141,11 @@ export interface IAiBreakdownRow {
users: number;
cost: number;
tokens: number;
- avgLatencyMs: number;
- p95LatencyMs: number;
+ /** `null` when the row has no chat calls to time. */
+ avgLatencyMs: number | null;
+ p95LatencyMs: number | null;
errors: number;
+ unpricedCalls: number;
lastAt: Date;
profile?: IServiceProfile;
group?: IServiceGroup;
@@ -154,8 +158,10 @@ export interface IAiEntitySummary {
cost: number;
inputTokens: number;
outputTokens: number;
- avgLatencyMs: number;
+ /** `null` when the entity made no chat calls to time. */
+ avgLatencyMs: number | null;
errors: number;
+ unpricedCalls: number;
firstAt: Date | null;
lastAt: Date | null;
topModels: string[];
@@ -193,6 +199,7 @@ interface TraceRow {
total_output_tokens: string | number;
cost: number;
has_error: boolean | number;
+ unpriced_calls: string | number;
models: string[];
all_groups: string[];
}
@@ -208,6 +215,8 @@ interface BreakdownRow {
avg_latency_ms: number | null;
p95_latency_ms: number | null;
errors: string | number;
+ unpriced_calls: string | number;
+ chat_calls: string | number;
last_at: string;
}
@@ -220,6 +229,8 @@ interface SummaryRow {
output_tokens: string | number;
avg_latency_ms: number | null;
errors: string | number;
+ unpriced_calls: string | number;
+ chat_calls: string | number;
first_at: string;
last_at: string;
top_models: string[];
@@ -315,6 +326,14 @@ function transformMetrics(row: MetricsRow | undefined): IAiMetrics {
};
}
+/** Latency is timed on chat calls only; with none, there is nothing to report. */
+function latencyOf(
+ value: number | null | undefined,
+ chatCalls: string | number | undefined
+): number | null {
+ return num(chatCalls) > 0 ? Math.round(num(value)) : null;
+}
+
function transformTrace(row: TraceRow): IServiceAiTrace {
return {
id: row.trace_id,
@@ -331,6 +350,7 @@ function transformTrace(row: TraceRow): IServiceAiTrace {
inputTokens: num(row.total_input_tokens),
outputTokens: num(row.total_output_tokens),
cost: num(row.cost),
+ unpricedCalls: num(row.unpriced_calls),
isError: Boolean(row.has_error),
models: row.models ?? [],
groups: row.all_groups ?? [],
@@ -345,8 +365,9 @@ function transformSummary(row: SummaryRow | undefined): IAiEntitySummary {
cost: num(row?.cost),
inputTokens: num(row?.input_tokens),
outputTokens: num(row?.output_tokens),
- avgLatencyMs: Math.round(num(row?.avg_latency_ms)),
+ avgLatencyMs: latencyOf(row?.avg_latency_ms, row?.chat_calls),
errors: num(row?.errors),
+ unpricedCalls: num(row?.unpriced_calls),
firstAt: row ? toDateOrNull(row.first_at) : null,
lastAt: row ? toDateOrNull(row.last_at) : null,
topModels: row?.top_models ?? [],
@@ -367,6 +388,7 @@ export interface AiBreakdownInput {
timezone: string;
dimension: AiBreakdownDimension;
limit: number;
+ sort?: AiBreakdownSort;
filters?: IAiFilters;
}
@@ -431,7 +453,7 @@ export function createAiService(deps: ServiceDeps, _services: () => Services) {
input: AiBreakdownInput
): Promise<IAiBreakdownRow[]> {
const rows = await rangedQuery<BreakdownRow>(
- breakdownQuery(input),
+ breakdownQuery({ ...input, sort: input.sort ?? 'cost' }),
input.timezone
);
const items: IAiBreakdownRow[] = rows.map((row) => ({
@@ -443,9 +465,10 @@ export function createAiService(deps: ServiceDeps, _services: () => Services) {
users: num(row.users),
cost: num(row.cost),
tokens: num(row.tokens),
- avgLatencyMs: Math.round(num(row.avg_latency_ms)),
- p95LatencyMs: Math.round(num(row.p95_latency_ms)),
+ avgLatencyMs: latencyOf(row.avg_latency_ms, row.chat_calls),
+ p95LatencyMs: latencyOf(row.p95_latency_ms, row.chat_calls),
errors: num(row.errors),
+ unpricedCalls: num(row.unpriced_calls),
lastAt: convertClickhouseDateToJs(row.last_at),
}));
if (input.dimension === 'profile') {
diff --git a/packages/core/src/modules/ai/src/pricing/catalog.ts b/packages/core/src/modules/ai/src/pricing/catalog.ts
index d0719db49..3e10a0fd5 100644
--- a/packages/core/src/modules/ai/src/pricing/catalog.ts
+++ b/packages/core/src/modules/ai/src/pricing/catalog.ts
@@ -101,7 +101,12 @@ function candidateNames(model: string): string[] {
for (const name of [...names]) {
names.push(...stripVersionSuffixes(name));
}
- return [...new Set(names)];
+ // OpenRouter writes `claude-sonnet-4.5` where the catalog has `claude-sonnet-4-5`.
+ // Tried last, because `gpt-4.1` and `gemini-2.5-pro` keep their dots in the catalog.
+ const dashed = names
+ .filter((name) => name.includes('.'))
+ .map((name) => name.replaceAll('.', '-'));
+ return [...new Set([...names, ...dashed])];
}
function longestPrefixMatch(name: string): ResolvedModelPrice | null {
diff --git a/packages/core/src/modules/ai/src/pricing/cost.test.ts b/packages/core/src/modules/ai/src/pricing/cost.test.ts
index a4e7927a9..dc4c76be8 100644
--- a/packages/core/src/modules/ai/src/pricing/cost.test.ts
+++ b/packages/core/src/modules/ai/src/pricing/cost.test.ts
@@ -19,6 +19,9 @@ describe('resolveModelPrice', () => {
['claude-sonnet-5-5', 'anthropic', 'claude-sonnet-5-5'],
['claude-sonnet-5-5@20260301', 'anthropic', 'claude-sonnet-5-5'],
['anthropic/claude-sonnet-5-5', 'openrouter', 'claude-sonnet-5-5'],
+ ['anthropic/claude-sonnet-4.5', 'openrouter', 'claude-sonnet-4-5'],
+ ['anthropic/claude-haiku-4.5', 'openrouter', 'claude-haiku-4-5'],
+ ['gpt-4.1', 'openai', 'gpt-4.1'],
[
'anthropic.claude-sonnet-5-5-v1:0',
'bedrock',
@@ -59,6 +62,16 @@ describe('calculateAiCost', () => {
});
});
+ test('a call that reported no tokens has an unknown cost, not $0', () => {
+ expect(
+ calculateAiCost({
+ model: 'gpt-5-mini',
+ provider: 'openai',
+ usage: { inputTokens: 0, outputTokens: 0 },
+ }).source
+ ).toBe('none');
+ });
+
test('prices input and output tokens from the catalog', () => {
const cost = calculateAiCost({
model: 'gpt-5-mini',
diff --git a/packages/core/src/modules/ai/src/pricing/cost.ts b/packages/core/src/modules/ai/src/pricing/cost.ts
index 5798962b3..8c7dc2d68 100644
--- a/packages/core/src/modules/ai/src/pricing/cost.ts
+++ b/packages/core/src/modules/ai/src/pricing/cost.ts
@@ -86,6 +86,13 @@ export function calculateAiCost(input: AiCostInput): AiCost {
if (!(input.model && input.usage)) {
return NO_COST;
}
+ // A completed call always uses tokens; zero means the provider reported no
+ // usage (a streamed chat completion without include_usage), so the cost is unknown.
+ const reportedTokens =
+ (input.usage.inputTokens ?? 0) + (input.usage.outputTokens ?? 0);
+ if (reportedTokens === 0) {
+ return NO_COST;
+ }
const resolved = resolveModelPrice(input.model, input.provider);
if (!resolved) {
return NO_COST;
diff --git a/packages/core/src/modules/ai/src/sql.ts b/packages/core/src/modules/ai/src/sql.ts
index 9e8729c2f..c463c4e68 100644
--- a/packages/core/src/modules/ai/src/sql.ts
+++ b/packages/core/src/modules/ai/src/sql.ts
@@ -4,11 +4,20 @@ import { type SqlFragment, sql } from '@openpanel/db/src/clickhouse/sql';
import { toRangeBoundaryLiteral } from '../../../shared/ch-dates';
import { TABLE_NAMES } from '../../../shared/ch-tables';
import type { IInterval } from '../../report/report.constants';
-import type { AiBreakdownDimension, IAiFilters } from '../ai.constants';
+import type {
+ AiBreakdownDimension,
+ AiBreakdownSort,
+ IAiFilters,
+} from '../ai.constants';
export const AI_EVENTS_TABLE = TABLE_NAMES.ai_events;
/** The columns a list or detail row selects. */
+/** Calls billed by a provider. Latency stays generation-only: embeddings answer in a fraction of the time and would drag p95 down. */
+const IS_MODEL_CALL = sql`type IN ('generation', 'embedding')`;
+/** A successful call whose model has no catalog price; its $0 is unknown, not free. */
+const IS_UNPRICED_CALL = sql`${IS_MODEL_CALL} AND NOT is_error AND cost_source = 'none'`;
+
export const AI_EVENT_COLUMNS = sql`
project_id, id, type, trace_id, span_id, parent_id, conversation_id, name,
model, provider, input_tokens, output_tokens, cache_read_tokens,
@@ -136,7 +145,7 @@ function scopeWhere(input: AiScopedQueryInput): SqlFragment {
}
const METRIC_COLUMNS = sql`
- countIf(type = 'generation') AS generations,
+ countIf(${IS_MODEL_CALL}) AS generations,
count() AS events,
uniq(trace_id) AS traces,
uniqIf(profile_id, profile_id != '') AS users,
@@ -148,7 +157,7 @@ const METRIC_COLUMNS = sql`
quantileIf(0.5)(latency_ms, type = 'generation') AS p50_latency_ms,
quantileIf(0.95)(latency_ms, type = 'generation') AS p95_latency_ms,
avgIf(latency_ms, type = 'generation') AS avg_latency_ms,
- countIf(is_error AND type = 'generation') AS errors
+ countIf(is_error AND ${IS_MODEL_CALL}) AS errors
`;
export function overviewTotalsQuery(input: AiScopedQueryInput): SqlFragment {
@@ -186,18 +195,31 @@ const DIMENSION_COLUMN: Record<AiBreakdownDimension, SqlFragment> = {
};
export function breakdownQuery(
- input: AiScopedQueryInput & { dimension: AiBreakdownDimension; limit: number }
+ input: AiScopedQueryInput & {
+ dimension: AiBreakdownDimension;
+ limit: number;
+ sort: AiBreakdownSort;
+ }
): SqlFragment {
const dimension = DIMENSION_COLUMN[input.dimension];
const onlyErrors =
input.dimension === 'errorCode' ? sql`AND is_error = true` : sql.empty;
+ const byErrors = input.sort === 'errors';
// Anonymous AI usage is a real share of the spend, so it stays as its own row.
- const withoutEmptyKey =
- input.dimension === 'profile' ? sql.empty : sql`HAVING key != ''`;
+ const keepEmptyKey = input.dimension === 'profile';
+ const having = [
+ keepEmptyKey ? null : sql`key != ''`,
+ byErrors ? sql`errors > 0` : null,
+ ].filter((condition): condition is SqlFragment => condition !== null);
+ const havingClause =
+ having.length > 0 ? sql`HAVING ${sql.join(having, ' AND ')}` : sql.empty;
+ const orderBy = byErrors
+ ? sql`errors DESC, cost DESC`
+ : sql`cost DESC, events DESC`;
return sql`
SELECT
${dimension} AS key,
- countIf(type = 'generation') AS generations,
+ countIf(${IS_MODEL_CALL}) AS generations,
count() AS events,
uniq(trace_id) AS traces,
uniqIf(profile_id, profile_id != '') AS users,
@@ -205,14 +227,16 @@ export function breakdownQuery(
sum(input_tokens) + sum(output_tokens) AS tokens,
avgIf(latency_ms, type = 'generation') AS avg_latency_ms,
quantileIf(0.95)(latency_ms, type = 'generation') AS p95_latency_ms,
- countIf(is_error AND type = 'generation') AS errors,
+ countIf(is_error AND ${IS_MODEL_CALL}) AS errors,
+ countIf(${IS_UNPRICED_CALL}) AS unpriced_calls,
+ countIf(type = 'generation') AS chat_calls,
max(created_at) AS last_at
FROM ${sql.id(AI_EVENTS_TABLE)}
WHERE ${scopeWhere(input)}
${onlyErrors}
GROUP BY key
- ${withoutEmptyKey}
- ORDER BY cost DESC, events DESC
+ ${havingClause}
+ ORDER BY ${orderBy}
LIMIT ${sql.uint64(input.limit)}
`;
}
@@ -247,12 +271,13 @@ export function traceListQuery(input: TraceListQueryInput): SqlFragment {
argMin(session_id, created_at) AS first_session_id,
argMin(conversation_id, created_at) AS first_conversation_id,
argMin(id, created_at) AS first_id,
- countIf(type = 'generation') AS generations,
+ countIf(${IS_MODEL_CALL}) AS generations,
count() AS events,
sum(input_tokens) AS total_input_tokens,
sum(output_tokens) AS total_output_tokens,
sum(total_cost) AS cost,
max(is_error) AS has_error,
+ countIf(${IS_UNPRICED_CALL}) AS unpriced_calls,
arrayDistinct(groupArrayIf(model, model != '')) AS models,
arrayDistinct(arrayFlatten(groupArray(groups))) AS all_groups
FROM ${sql.id(AI_EVENTS_TABLE)}
@@ -332,17 +357,19 @@ export function entitySummaryQuery(input: {
}): SqlFragment {
return sql`
SELECT
- countIf(type = 'generation') AS generations,
+ countIf(${IS_MODEL_CALL}) AS generations,
uniq(trace_id) AS traces,
uniqIf(conversation_id, conversation_id != '') AS conversations,
sum(total_cost) AS cost,
sum(input_tokens) AS input_tokens,
sum(output_tokens) AS output_tokens,
avgIf(latency_ms, type = 'generation') AS avg_latency_ms,
- countIf(is_error AND type = 'generation') AS errors,
+ countIf(is_error AND ${IS_MODEL_CALL}) AS errors,
+ countIf(${IS_UNPRICED_CALL}) AS unpriced_calls,
+ countIf(type = 'generation') AS chat_calls,
min(created_at) AS first_at,
max(created_at) AS last_at,
- topKIf(3)(model, type = 'generation') AS top_models
+ topKIf(3)(model, ${IS_MODEL_CALL}) AS top_models
FROM ${sql.id(AI_EVENTS_TABLE)}
WHERE project_id = ${sql.string(input.projectId)}
AND ${input.where}
diff --git a/packages/sdks/ai/anthropic.ts b/packages/sdks/ai/anthropic.ts
new file mode 100644
index 000000000..57c49351a
--- /dev/null
+++ b/packages/sdks/ai/anthropic.ts
@@ -0,0 +1,2 @@
+export { wrapAnthropic } from './src/anthropic';
+export type { AiIntegrationOptions } from './src/payload';
diff --git a/packages/sdks/ai/openai.ts b/packages/sdks/ai/openai.ts
new file mode 100644
index 000000000..2156253fd
--- /dev/null
+++ b/packages/sdks/ai/openai.ts
@@ -0,0 +1,2 @@
+export { wrapOpenAI } from './src/openai';
+export type { AiIntegrationOptions } from './src/payload';
diff --git a/packages/sdks/ai/package.json b/packages/sdks/ai/package.json
index 12f7ecf7d..5ea2ca670 100644
--- a/packages/sdks/ai/package.json
+++ b/packages/sdks/ai/package.json
@@ -11,7 +11,9 @@
"config": {
"docPath": "apps/public/content/docs/(tracking)/sdks/ai.mdx",
"subpaths": [
- "vercel"
+ "vercel",
+ "openai",
+ "anthropic"
]
},
"scripts": {
@@ -24,9 +26,11 @@
"@openpanel/sdk": "workspace:1.3.3-local"
},
"devDependencies": {
+ "@anthropic-ai/sdk": "1.3.0",
"@openpanel/tsconfig": "workspace:*",
"@types/node": "catalog:",
"bun-types": "1.4.0",
+ "openai": "7.47.0",
"tsup": "7.3.0",
"typescript": "catalog:"
}
diff --git a/packages/sdks/ai/src/anthropic.test.ts b/packages/sdks/ai/src/anthropic.test.ts
new file mode 100644
index 000000000..c02d9fc7e
--- /dev/null
+++ b/packages/sdks/ai/src/anthropic.test.ts
@@ -0,0 +1,303 @@
+import { describe, expect, test } from 'bun:test';
+import Anthropic from '@anthropic-ai/sdk';
+import { wrapAnthropic } from './anthropic';
+import { withAiContext } from './context';
+import { collect, jsonResponse, mockFetch, sseResponse } from './test-helpers';
+
+function client(responses: (() => Response)[]) {
+ const { op, payloads } = collect();
+ const { fetch, requests } = mockFetch(responses);
+ const anthropic = wrapAnthropic(
+ new Anthropic({ apiKey: 'sk-ant-test', fetch, maxRetries: 0 }),
+ { op }
+ );
+ return { anthropic, payloads, requests };
+}
+
+const toolUseMessage = {
+ id: 'msg_1',
+ type: 'message',
+ role: 'assistant',
+ model: 'claude-sonnet-5-5-20261001',
+ stop_reason: 'tool_use',
+ stop_sequence: null,
+ content: [
+ { type: 'thinking', thinking: 'Check the weather.', signature: 'sig' },
+ {
+ type: 'tool_use',
+ id: 'toolu_1',
+ name: 'get_weather',
+ input: { city: 'Oslo' },
+ },
+ ],
+ usage: {
+ input_tokens: 20,
+ output_tokens: 40,
+ cache_read_input_tokens: 1000,
+ cache_creation_input_tokens: 200,
+ },
+};
+
+const streamEvents = [
+ {
+ type: 'message_start',
+ message: {
+ id: 'msg_2',
+ type: 'message',
+ role: 'assistant',
+ model: 'claude-sonnet-5-5-20261001',
+ content: [],
+ stop_reason: null,
+ stop_sequence: null,
+ usage: {
+ input_tokens: 15,
+ output_tokens: 1,
+ cache_read_input_tokens: 0,
+ cache_creation_input_tokens: 0,
+ },
+ },
+ },
+ {
+ type: 'content_block_start',
+ index: 0,
+ content_block: { type: 'text', text: '' },
+ },
+ {
+ type: 'content_block_delta',
+ index: 0,
+ delta: { type: 'text_delta', text: 'Let me ' },
+ },
+ {
+ type: 'content_block_delta',
+ index: 0,
+ delta: { type: 'text_delta', text: 'check.' },
+ },
+ { type: 'content_block_stop', index: 0 },
+ {
+ type: 'content_block_start',
+ index: 1,
+ content_block: {
+ type: 'tool_use',
+ id: 'toolu_2',
+ name: 'get_weather',
+ input: {},
+ },
+ },
+ {
+ type: 'content_block_delta',
+ index: 1,
+ delta: { type: 'input_json_delta', partial_json: '{"city":' },
+ },
+ {
+ type: 'content_block_delta',
+ index: 1,
+ delta: { type: 'input_json_delta', partial_json: '"Oslo"}' },
+ },
+ { type: 'content_block_stop', index: 1 },
+ {
+ type: 'message_delta',
+ delta: { stop_reason: 'tool_use', stop_sequence: null },
+ usage: { output_tokens: 25 },
+ },
+ { type: 'message_stop' },
+];
+
+describe('wrapAnthropic', () => {
+ test('records a message with system, tool results and cache-inclusive usage', async () => {
+ const { anthropic, payloads } = client([
+ () => jsonResponse(toolUseMessage),
+ ]);
+
+ const message = await anthropic.messages.create({
+ model: 'claude-sonnet-5-5',
+ max_tokens: 1024,
+ stop_sequences: ['END'],
+ system: 'Be brief.',
+ tools: [
+ {
+ name: 'get_weather',
+ description: 'Weather for a city',
+ input_schema: { type: 'object', properties: {} },
+ },
+ ],
+ messages: [
+ { role: 'user', content: 'Weather in Oslo?' },
+ {
+ role: 'assistant',
+ content: [
+ { type: 'tool_use', id: 'toolu_0', name: 'locate', input: {} },
+ ],
+ },
+ {
+ role: 'user',
+ content: [
+ { type: 'tool_result', tool_use_id: 'toolu_0', content: 'Oslo' },
+ {
+ type: 'image',
+ source: { type: 'base64', media_type: 'image/png', data: 'AAAA' },
+ },
+ ],
+ },
+ ],
+ });
+
+ expect(message.id).toBe('msg_1');
+ expect(payloads).toHaveLength(1);
+ expect(payloads[0]).toMatchObject({
+ kind: 'generation',
+ model: 'claude-sonnet-5-5-20261001',
+ provider: 'anthropic',
+ stream: false,
+ stopReason: 'tool-calls',
+ usage: {
+ inputTokens: 1220,
+ outputTokens: 40,
+ cacheReadTokens: 1000,
+ cacheWriteTokens: 200,
+ },
+ modelParameters: { stopSequences: ['END'], maxTokens: 1024 },
+ tools: [
+ {
+ type: 'function',
+ name: 'get_weather',
+ description: 'Weather for a city',
+ inputSchema: { type: 'object', properties: {} },
+ },
+ ],
+ });
+ expect(payloads[0]?.input).toEqual([
+ { role: 'system', content: 'Be brief.' },
+ { role: 'user', content: 'Weather in Oslo?' },
+ {
+ role: 'assistant',
+ content: [
+ {
+ type: 'tool_call',
+ toolCallId: 'toolu_0',
+ toolName: 'locate',
+ input: {},
+ },
+ ],
+ },
+ {
+ role: 'user',
+ content: [
+ { type: 'tool_result', toolCallId: 'toolu_0', output: 'Oslo' },
+ { type: 'file', mediaType: 'image/png', data: '[file]' },
+ ],
+ },
+ ]);
+ expect(payloads[0]?.output).toEqual([
+ {
+ role: 'assistant',
+ content: [
+ { type: 'reasoning', text: 'Check the weather.' },
+ {
+ type: 'tool_call',
+ toolCallId: 'toolu_1',
+ toolName: 'get_weather',
+ input: { city: 'Oslo' },
+ },
+ ],
+ },
+ ]);
+ });
+
+ test('rebuilds a streamed message from its events', async () => {
+ const { anthropic, payloads } = client([
+ () => sseResponse(streamEvents, { named: true }),
+ ]);
+
+ const stream = await anthropic.messages.create({
+ model: 'claude-sonnet-5-5',
+ max_tokens: 1024,
+ messages: [{ role: 'user', content: 'Weather?' }],
+ stream: true,
+ });
+ let events = 0;
+ for await (const _event of stream) {
+ events++;
+ }
+
+ expect(events).toBe(streamEvents.length);
+ expect(payloads).toHaveLength(1);
+ expect(payloads[0]).toMatchObject({
+ stream: true,
+ stopReason: 'tool-calls',
+ usage: { inputTokens: 15, outputTokens: 25 },
+ output: [
+ {
+ role: 'assistant',
+ content: [
+ { type: 'text', text: 'Let me check.' },
+ {
+ type: 'tool_call',
+ toolCallId: 'toolu_2',
+ toolName: 'get_weather',
+ input: { city: 'Oslo' },
+ },
+ ],
+ },
+ ],
+ });
+ expect(payloads[0]?.timeToFirstTokenMs).toBeNumber();
+ });
+
+ test('records the messages.stream helper once', async () => {
+ const { anthropic, payloads } = client([
+ () => sseResponse(streamEvents, { named: true }),
+ ]);
+
+ const final = await anthropic.messages
+ .stream({
+ model: 'claude-sonnet-5-5',
+ max_tokens: 1024,
+ messages: [{ role: 'user', content: 'Weather?' }],
+ })
+ .finalMessage();
+
+ expect(final.content[0]).toMatchObject({ text: 'Let me check.' });
+ expect(payloads).toHaveLength(1);
+ expect(payloads[0]?.stream).toBe(true);
+ });
+
+ test('records beta messages and failures with their status', async () => {
+ const { anthropic, payloads } = client([
+ () => jsonResponse(toolUseMessage),
+ () =>
+ jsonResponse(
+ {
+ type: 'error',
+ error: { type: 'overloaded_error', message: 'Overloaded' },
+ },
+ 529
+ ),
+ ]);
+
+ await withAiContext({ profileId: 'usr_1', name: 'beta' }, () =>
+ anthropic.beta.messages.create({
+ model: 'claude-sonnet-5-5',
+ max_tokens: 10,
+ messages: [{ role: 'user', content: 'Hi' }],
+ })
+ );
+ const error = await anthropic.messages
+ .create({
+ model: 'claude-sonnet-5-5',
+ max_tokens: 10,
+ messages: [{ role: 'user', content: 'Hi' }],
+ })
+ .then(
+ () => undefined,
+ (caught: unknown) => caught
+ );
+
+ expect(error).toBeInstanceOf(Anthropic.APIError);
+ expect(payloads).toHaveLength(2);
+ expect(payloads[0]).toMatchObject({ profileId: 'usr_1', name: 'beta' });
+ expect(payloads[1]?.error).toMatchObject({
+ httpStatus: 529,
+ message: expect.stringContaining('Overloaded'),
+ });
+ });
+});
diff --git a/packages/sdks/ai/src/anthropic.ts b/packages/sdks/ai/src/anthropic.ts
new file mode 100644
index 000000000..b0badfef8
--- /dev/null
+++ b/packages/sdks/ai/src/anthropic.ts
@@ -0,0 +1,311 @@
+// Records calls made through the official `@anthropic-ai/sdk` client (and the
+// Bedrock and Vertex clients built on it): `messages.create`, streamed or not,
+// and `beta.messages.create`. The interfaces below describe only the fields
+// read, so any SDK version with these shapes works.
+
+import type { AiModelParameters, AiUsage } from '@openpanel/sdk';
+import {
+ type CallDescription,
+ type CallResult,
+ type CreateAdapter,
+ definedOnly,
+ instrumentCreate,
+ type StreamAccumulator,
+} from './instrument';
+import {
+ FILE_PLACEHOLDER,
+ type Message,
+ type MessagePart,
+ toolInput,
+} from './messages';
+import type { AiIntegrationOptions } from './payload';
+
+interface ContentBlock {
+ type: string;
+ text?: string;
+ thinking?: string;
+ id?: string;
+ name?: string;
+ input?: unknown;
+ tool_use_id?: string;
+ content?: string | ContentBlock[];
+ source?: { media_type?: string };
+}
+
+interface MessageParam {
+ role: string;
+ content: string | ContentBlock[];
+}
+
+interface MessagesParams {
+ model: string;
+ messages: MessageParam[];
+ system?: string | ContentBlock[];
+ stream?: boolean;
+ tools?: {
+ type?: string;
+ name: string;
+ description?: string;
+ input_schema?: unknown;
+ }[];
+ max_tokens?: number;
+ temperature?: number;
+ top_p?: number;
+ top_k?: number;
+ stop_sequences?: string[];
+}
+
+interface Usage {
+ input_tokens?: number | null;
+ output_tokens?: number | null;
+ cache_read_input_tokens?: number | null;
+ cache_creation_input_tokens?: number | null;
+}
+
+interface MessageResponse {
+ model?: string;
+ content?: ContentBlock[];
+ stop_reason?: string | null;
+ usage?: Usage;
+}
+
+interface StreamEvent {
+ type: string;
+ index?: number;
+ message?: MessageResponse;
+ content_block?: ContentBlock;
+ delta?: {
+ type?: string;
+ text?: string;
+ thinking?: string;
+ partial_json?: string;
+ stop_reason?: string | null;
+ };
+ usage?: Usage;
+}
+
+/** The resources wrapped; each is optional so a partial or older client still works. */
+interface AnthropicClient {
+ messages?: unknown;
+ beta?: { messages?: unknown };
+}
+
+const PROVIDER = 'anthropic';
+
+const STOP_REASONS: Record<string, string> = {
+ end_turn: 'stop',
+ stop_sequence: 'stop',
+ max_tokens: 'length',
+ model_context_window_exceeded: 'length',
+ tool_use: 'tool-calls',
+ refusal: 'content-filter',
+};
+
+const TOOL_CALL_BLOCKS = new Set(['tool_use', 'server_tool_use']);
+
+function blockPart(block: ContentBlock): MessagePart {
+ if (TOOL_CALL_BLOCKS.has(block.type)) {
+ return {
+ type: 'tool_call',
+ toolCallId: block.id,
+ toolName: block.name,
+ input: block.input,
+ };
+ }
+ switch (block.type) {
+ case 'text':
+ return { type: 'text', text: block.text };
+ case 'thinking':
+ return { type: 'reasoning', text: block.thinking };
+ case 'redacted_thinking':
+ return { type: 'reasoning' };
+ case 'tool_result':
+ return {
+ type: 'tool_result',
+ toolCallId: block.tool_use_id,
+ output:
+ typeof block.content === 'string'
+ ? block.content
+ : block.content?.map(blockPart),
+ };
+ case 'image':
+ case 'document':
+ return {
+ type: 'file',
+ mediaType: block.source?.media_type,
+ data: FILE_PLACEHOLDER,
+ };
+ default:
+ return { type: block.type };
+ }
+}
+
+function toMessage(role: string, content: string | ContentBlock[]): Message {
+ return {
+ role,
+ content: typeof content === 'string' ? content : content.map(blockPart),
+ };
+}
+
+/** Anthropic counts cached tokens apart from `input_tokens`; OpenPanel's input total includes them. */
+function usageOf(usage: Usage | undefined): AiUsage | undefined {
+ if (!usage) {
+ return undefined;
+ }
+ const cacheRead = usage.cache_read_input_tokens ?? undefined;
+ const cacheWrite = usage.cache_creation_input_tokens ?? undefined;
+ return {
+ inputTokens:
+ (usage.input_tokens ?? 0) + (cacheRead ?? 0) + (cacheWrite ?? 0),
+ outputTokens: usage.output_tokens ?? 0,
+ cacheReadTokens: cacheRead,
+ cacheWriteTokens: cacheWrite,
+ };
+}
+
+function stopReasonOf(reason: string | null | undefined): string | undefined {
+ return reason ? (STOP_REASONS[reason] ?? 'other') : undefined;
+}
+
+function resultOf(message: MessageResponse): CallResult {
+ const content = message.content ?? [];
+ return {
+ output: content.length > 0 ? [toMessage('assistant', content)] : [],
+ usage: usageOf(message.usage),
+ model: message.model,
+ stopReason: stopReasonOf(message.stop_reason),
+ };
+}
+
+/** Rebuilds the final message from stream events, the way the SDK's `MessageStream` does. */
+class MessagesStreamAccumulator implements StreamAccumulator {
+ private message: MessageResponse = {};
+ private readonly blocks: ContentBlock[] = [];
+ private readonly toolInputs = new Map<number, string>();
+
+ observe(value: unknown): boolean {
+ const event = value as StreamEvent;
+ const index = event.index ?? 0;
+ switch (event.type) {
+ case 'message_start':
+ this.message = { ...event.message };
+ return false;
+ case 'content_block_start':
+ // A block opens empty; the first token is its first delta.
+ if (event.content_block) {
+ this.blocks[index] = { ...event.content_block };
+ }
+ return false;
+ case 'content_block_delta':
+ this.applyDelta(index, event.delta);
+ return true;
+ case 'message_delta':
+ this.message.stop_reason =
+ event.delta?.stop_reason ?? this.message.stop_reason;
+ this.message.usage = this.mergeUsage(event.usage);
+ return false;
+ default:
+ return false;
+ }
+ }
+
+ private applyDelta(index: number, delta: StreamEvent['delta']): void {
+ const block = this.blocks[index];
+ if (!(block && delta)) {
+ return;
+ }
+ if (delta.text !== undefined) {
+ block.text = (block.text ?? '') + delta.text;
+ }
+ if (delta.thinking !== undefined) {
+ block.thinking = (block.thinking ?? '') + delta.thinking;
+ }
+ if (delta.partial_json !== undefined) {
+ this.toolInputs.set(
+ index,
+ (this.toolInputs.get(index) ?? '') + delta.partial_json
+ );
+ }
+ }
+
+ /** `message_delta` usage is cumulative; a field it leaves out keeps its `message_start` value. */
+ private mergeUsage(usage: Usage | undefined): Usage | undefined {
+ if (!usage) {
+ return this.message.usage;
+ }
+ const updates = Object.fromEntries(
+ Object.entries(usage).filter(([, count]) => count !== null)
+ );
+ return { ...this.message.usage, ...updates };
+ }
+
+ result(): CallResult {
+ const content = this.blocks.flatMap((block, index) => {
+ if (!block) {
+ return [];
+ }
+ const json = this.toolInputs.get(index);
+ return [
+ json === undefined ? block : { ...block, input: toolInput(json) },
+ ];
+ });
+ return resultOf({ ...this.message, content });
+ }
+}
+
+const messages: CreateAdapter = {
+ describe(value): CallDescription {
+ const params = value as MessagesParams;
+ const system: Message[] = params.system
+ ? [toMessage('system', params.system)]
+ : [];
+ return {
+ model: params.model,
+ provider: PROVIDER,
+ stream: params.stream === true,
+ input: [
+ ...system,
+ ...params.messages.map((message) =>
+ toMessage(message.role, message.content)
+ ),
+ ],
+ tools: params.tools?.length
+ ? params.tools.map((tool) => ({
+ type: tool.type ?? 'function',
+ name: tool.name,
+ description: tool.description,
+ inputSchema: tool.input_schema,
+ }))
+ : undefined,
+ modelParameters: definedOnly<AiModelParameters>({
+ temperature: params.temperature,
+ maxTokens: params.max_tokens,
+ topP: params.top_p,
+ topK: params.top_k,
+ stopSequences: params.stop_sequences,
+ }),
+ };
+ },
+ result: (value) => resultOf(value as MessageResponse),
+ accumulator: () => new MessagesStreamAccumulator(),
+};
+
+/**
+ * Records every `messages.create` call made through an Anthropic client as an
+ * OpenPanel AI event, streamed or not, including `messages.stream()`,
+ * `messages.parse()` and the beta tool runner:
+ *
+ * const anthropic = wrapAnthropic(new Anthropic(), { op });
+ *
+ * The client is instrumented in place and returned. Per-request context
+ * (user, conversation, name) comes from `withAiContext`.
+ */
+export function wrapAnthropic<Client extends object>(
+ client: Client,
+ options: AiIntegrationOptions
+): Client {
+ const resources = client as AnthropicClient;
+ instrumentCreate(resources.messages, messages, options);
+ instrumentCreate(resources.beta?.messages, messages, options);
+ return client;
+}
diff --git a/packages/sdks/ai/src/context.test.ts b/packages/sdks/ai/src/context.test.ts
new file mode 100644
index 000000000..2ce9fd711
--- /dev/null
+++ b/packages/sdks/ai/src/context.test.ts
@@ -0,0 +1,16 @@
+import { expect, test } from 'bun:test';
+import { getAiContext, withAiContext } from './context';
+
+test('a separately bundled copy of the module sees the same context', async () => {
+ const copy: typeof import('./context') = await import(
+ `./context.ts?copy=${Date.now()}`
+ );
+ expect(copy.withAiContext).not.toBe(withAiContext);
+
+ withAiContext({ name: 'outer' }, () => {
+ expect(copy.getAiContext()?.name).toBe('outer');
+ });
+ copy.withAiContext({ name: 'from-copy' }, () => {
+ expect(getAiContext()?.name).toBe('from-copy');
+ });
+});
diff --git a/packages/sdks/ai/src/context.ts b/packages/sdks/ai/src/context.ts
index 62a9f348b..e7808b30b 100644
--- a/packages/sdks/ai/src/context.ts
+++ b/packages/sdks/ai/src/context.ts
@@ -21,7 +21,15 @@ export interface AiCallContext {
privacyMode?: boolean;
}
-const storage = new AsyncLocalStorage<AiCallContext>();
+// Every entry point (`@openpanel/ai`, `/openai`, `/vercel`…) is bundled on
+// its own, and an app may load both the ESM and CJS builds. A store per copy
+// would make `withAiContext` invisible to the wrappers, so all copies share one.
+const STORAGE_KEY = Symbol.for('openpanel.ai.context');
+const registry = globalThis as {
+ [STORAGE_KEY]?: AsyncLocalStorage<AiCallContext>;
+};
+registry[STORAGE_KEY] ??= new AsyncLocalStorage<AiCallContext>();
+const storage = registry[STORAGE_KEY];
export function createAiTraceId(): string {
return randomUUID();
diff --git a/packages/sdks/ai/src/instrument.ts b/packages/sdks/ai/src/instrument.ts
new file mode 100644
index 000000000..83359006a
--- /dev/null
+++ b/packages/sdks/ai/src/instrument.ts
@@ -0,0 +1,214 @@
+// Hooks a provider SDK's `create` method so every call is recorded, without
+// changing what the caller gets back. The OpenAI and Anthropic SDKs share a
+// generator (Stainless): `create` returns an `APIPromise` whose
+// `_thenUnwrap` keeps `.withResponse()` and `.asResponse()` working, and a
+// stream whose `iterator` field feeds iteration, `tee()` and
+// `toReadableStream()`. Their own helpers (`parse`, `stream`, `runTools`)
+// call `create` on the client's resource, so patching the resource instance
+// covers them too.
+
+import {
+ type AiCall,
+ type AiCallOutcome,
+ type AiIntegrationOptions,
+ recordCall,
+ resolveContext,
+} from './payload';
+
+export type CallDescription = Omit<AiCall, 'context' | 'startedAt'> & {
+ stream: boolean;
+};
+
+export type CallResult = Omit<AiCallOutcome, 'stream' | 'timeToFirstTokenMs'>;
+
+export interface StreamAccumulator {
+ /** Returns whether the chunk carried generated output, which marks the first token. */
+ observe(chunk: unknown): boolean;
+ result(): CallResult;
+}
+
+/** Converts one provider endpoint's request, response and stream chunks. */
+export interface CreateAdapter {
+ describe(params: unknown): CallDescription;
+ result(response: unknown): CallResult;
+ accumulator?(): StreamAccumulator;
+}
+
+type CreateMethod = ((this: unknown, ...args: unknown[]) => unknown) & {
+ [INSTRUMENTED]?: true;
+};
+
+interface ApiPromiseLike extends PromiseLike<unknown> {
+ _thenUnwrap?(transform: (data: unknown) => unknown): unknown;
+ responsePromise?: Promise<unknown>;
+}
+
+type StreamCall = AiCall & { stream: boolean };
+
+const INSTRUMENTED = Symbol.for('openpanel.ai.instrumented');
+
+function attempt<T>(fn: () => T): T | undefined {
+ try {
+ return fn();
+ } catch {
+ return undefined;
+ }
+}
+
+function isPromiseLike(value: unknown): value is ApiPromiseLike {
+ return (
+ typeof value === 'object' &&
+ value !== null &&
+ typeof (value as PromiseLike<unknown>).then === 'function'
+ );
+}
+
+function record(
+ call: StreamCall,
+ outcome: Omit<AiCallOutcome, 'stream'>,
+ options: AiIntegrationOptions
+): void {
+ recordCall(call, { ...outcome, stream: call.stream }, options);
+}
+
+function observeStream(
+ stream: unknown,
+ call: StreamCall,
+ adapter: CreateAdapter,
+ options: AiIntegrationOptions
+): unknown {
+ const target = stream as { iterator?: unknown };
+ const accumulator = adapter.accumulator?.();
+ if (!accumulator || typeof target?.iterator !== 'function') {
+ record(call, {}, options);
+ return stream;
+ }
+ const source = (target.iterator as () => AsyncIterator<unknown>).bind(stream);
+ target.iterator = async function* observedIterator() {
+ const iterator = source();
+ let firstTokenAt: number | undefined;
+ let finished = false;
+ let error: unknown;
+ try {
+ for (;;) {
+ const next = await iterator.next();
+ if (next.done) {
+ finished = true;
+ return;
+ }
+ const carriedOutput = attempt(() => accumulator.observe(next.value));
+ if (carriedOutput && firstTokenAt === undefined) {
+ firstTokenAt = Date.now();
+ }
+ yield next.value;
+ }
+ } catch (caught) {
+ error = caught;
+ throw caught;
+ } finally {
+ const cancelled = !(finished || error);
+ if (cancelled) {
+ await iterator.return?.();
+ }
+ const result = attempt(() => accumulator.result()) ?? {};
+ record(
+ call,
+ {
+ ...result,
+ error:
+ error ??
+ (cancelled ? new Error('stream cancelled') : undefined) ??
+ result.error,
+ timeToFirstTokenMs:
+ firstTokenAt === undefined
+ ? undefined
+ : firstTokenAt - call.startedAt,
+ },
+ options
+ );
+ }
+ };
+ return stream;
+}
+
+function observeResponse(
+ response: unknown,
+ call: StreamCall,
+ adapter: CreateAdapter,
+ options: AiIntegrationOptions
+): unknown {
+ if (!isPromiseLike(response)) {
+ return response;
+ }
+ const onData = (data: unknown) => {
+ if (call.stream) {
+ return attempt(() => observeStream(data, call, adapter, options)) ?? data;
+ }
+ record(call, attempt(() => adapter.result(data)) ?? {}, options);
+ return data;
+ };
+ const onError = (error: unknown) => record(call, { error }, options);
+
+ // Parsing stays lazy: `.asResponse()` must still get an unread body.
+ if (typeof response._thenUnwrap === 'function') {
+ response.responsePromise?.catch(onError);
+ return response._thenUnwrap(onData);
+ }
+ return response.then(onData, (error: unknown) => {
+ onError(error);
+ throw error;
+ });
+}
+
+/** Drops unset entries; `undefined` when none are left. */
+export function definedOnly<T extends object>(value: T): T | undefined {
+ const entries = Object.entries(value).filter(
+ ([, entry]) => entry !== undefined && entry !== null
+ );
+ return entries.length > 0 ? (Object.fromEntries(entries) as T) : undefined;
+}
+
+/**
+ * Replaces `resource.create` with a recording version. Calling it twice on
+ * the same resource is a no-op, so a client can be wrapped more than once.
+ */
+export function instrumentCreate(
+ resource: unknown,
+ adapter: CreateAdapter,
+ options: AiIntegrationOptions
+): void {
+ const target = resource as { create?: CreateMethod } | undefined;
+ const original = target?.create;
+ if (!target || typeof original !== 'function' || original[INSTRUMENTED]) {
+ return;
+ }
+ const instrumented: CreateMethod = function create(this: unknown, ...args) {
+ // Read now: the ambient `withAiContext` may be gone once a stream is read.
+ const call = attempt(
+ (): StreamCall => ({
+ ...adapter.describe(args[0]),
+ context: resolveContext(undefined, options),
+ startedAt: Date.now(),
+ })
+ );
+ let response: unknown;
+ try {
+ response = original.apply(this, args);
+ } catch (error) {
+ // The SDKs validate some params before any request, e.g. a `max_tokens` too large for a non-streamed call.
+ if (call) {
+ record(call, { error }, options);
+ }
+ throw error;
+ }
+ if (!call) {
+ return response;
+ }
+ return (
+ attempt(() => observeResponse(response, call, adapter, options)) ??
+ response
+ );
+ };
+ instrumented[INSTRUMENTED] = true;
+ target.create = instrumented;
+}
diff --git a/packages/sdks/ai/src/messages.ts b/packages/sdks/ai/src/messages.ts
index ef7d28521..dcb5ff86c 100644
--- a/packages/sdks/ai/src/messages.ts
+++ b/packages/sdks/ai/src/messages.ts
@@ -17,7 +17,20 @@ export interface Message {
content: string | MessagePart[];
}
-const FILE_PLACEHOLDER = '[file]';
+/** Stands in for binary content, which is never sent. */
+export const FILE_PLACEHOLDER = '[file]';
+
+/** Tool arguments arrive as JSON text; keep the raw text when it does not parse. */
+export function toolInput(raw: string | undefined): unknown {
+ if (!raw) {
+ return {};
+ }
+ try {
+ return JSON.parse(raw);
+ } catch {
+ return raw;
+ }
+}
function partOf(part: Record<string, unknown>): MessagePart {
switch (part.type) {
diff --git a/packages/sdks/ai/src/openai.test.ts b/packages/sdks/ai/src/openai.test.ts
new file mode 100644
index 000000000..115ad6e10
--- /dev/null
+++ b/packages/sdks/ai/src/openai.test.ts
@@ -0,0 +1,667 @@
+import { describe, expect, test } from 'bun:test';
+import OpenAI from 'openai';
+import { withAiContext } from './context';
+import { wrapOpenAI } from './openai';
+import { collect, jsonResponse, mockFetch, sseResponse } from './test-helpers';
+
+function client(responses: (() => Response)[], wrapTimes = 1) {
+ const { op, payloads } = collect();
+ const { fetch, requests } = mockFetch(responses);
+ let openai = new OpenAI({ apiKey: 'sk-test', fetch, maxRetries: 0 });
+ for (let i = 0; i < wrapTimes; i++) {
+ openai = wrapOpenAI(openai, { op });
+ }
+ return { openai, payloads, requests };
+}
+
+const toolCompletion = {
+ id: 'chatcmpl_1',
+ object: 'chat.completion',
+ created: 1,
+ model: 'gpt-5-mini-2025-08-07',
+ choices: [
+ {
+ index: 0,
+ finish_reason: 'tool_calls',
+ message: {
+ role: 'assistant',
+ content: null,
+ refusal: null,
+ tool_calls: [
+ {
+ id: 'call_1',
+ type: 'function',
+ function: { name: 'get_weather', arguments: '{"city":"Oslo"}' },
+ },
+ ],
+ },
+ },
+ ],
+ usage: {
+ prompt_tokens: 120,
+ completion_tokens: 30,
+ total_tokens: 150,
+ prompt_tokens_details: { cached_tokens: 100 },
+ completion_tokens_details: { reasoning_tokens: 10 },
+ },
+};
+
+function chatChunk(
+ delta: Record<string, unknown>,
+ finishReason: string | null = null
+) {
+ return {
+ id: 'chatcmpl_2',
+ object: 'chat.completion.chunk',
+ created: 1,
+ model: 'gpt-5-mini-2025-08-07',
+ choices: [{ index: 0, delta, finish_reason: finishReason }],
+ };
+}
+
+const textChunks = [
+ chatChunk({ role: 'assistant', content: 'Hello' }),
+ chatChunk({ content: ' there' }),
+ chatChunk({}, 'stop'),
+ {
+ id: 'chatcmpl_2',
+ object: 'chat.completion.chunk',
+ created: 1,
+ model: 'gpt-5-mini-2025-08-07',
+ choices: [],
+ usage: { prompt_tokens: 12, completion_tokens: 2, total_tokens: 14 },
+ },
+];
+
+const functionCallResponse = {
+ id: 'resp_1',
+ object: 'response',
+ created_at: 1,
+ model: 'gpt-5-2025-08-07',
+ status: 'completed',
+ output: [
+ {
+ type: 'reasoning',
+ id: 'rs_1',
+ summary: [{ type: 'summary_text', text: 'Need the weather.' }],
+ },
+ {
+ type: 'function_call',
+ id: 'fc_1',
+ call_id: 'call_9',
+ name: 'get_weather',
+ arguments: '{"city":"Oslo"}',
+ status: 'completed',
+ },
+ ],
+ usage: {
+ input_tokens: 50,
+ output_tokens: 20,
+ total_tokens: 70,
+ input_tokens_details: { cached_tokens: 0 },
+ output_tokens_details: { reasoning_tokens: 12 },
+ },
+};
+
+describe('wrapOpenAI', () => {
+ test('records a chat completion with tool calls, usage and parameters', async () => {
+ const { openai, payloads } = client([() => jsonResponse(toolCompletion)]);
+
+ const { data, request_id } = await openai.chat.completions
+ .create({
+ model: 'gpt-5-mini',
+ temperature: 0.3,
+ max_completion_tokens: 200,
+ stop: 'END',
+ messages: [
+ { role: 'system', content: 'Be brief.' },
+ {
+ role: 'user',
+ content: [
+ { type: 'text', text: 'Weather?' },
+ {
+ type: 'image_url',
+ image_url: { url: 'data:image/png;base64,AAAA' },
+ },
+ ],
+ },
+ {
+ role: 'assistant',
+ content: null,
+ tool_calls: [
+ {
+ id: 'call_0',
+ type: 'function',
+ function: { name: 'locate', arguments: '{}' },
+ },
+ ],
+ },
+ { role: 'tool', tool_call_id: 'call_0', content: 'Oslo' },
+ ],
+ tools: [
+ {
+ type: 'function',
+ function: {
+ name: 'get_weather',
+ description: 'Weather for a city',
+ parameters: { type: 'object' },
+ },
+ },
+ ],
+ })
+ .withResponse();
+
+ expect(request_id).toBe('req_1');
+ expect(data.choices[0]?.message.tool_calls?.[0]?.id).toBe('call_1');
+ expect(payloads).toHaveLength(1);
+ expect(payloads[0]).toMatchObject({
+ kind: 'generation',
+ model: 'gpt-5-mini-2025-08-07',
+ provider: 'openai',
+ stream: false,
+ stopReason: 'tool-calls',
+ usage: {
+ inputTokens: 120,
+ outputTokens: 30,
+ cacheReadTokens: 100,
+ reasoningTokens: 10,
+ },
+ modelParameters: {
+ temperature: 0.3,
+ maxTokens: 200,
+ stopSequences: ['END'],
+ },
+ tools: [
+ {
+ type: 'function',
+ name: 'get_weather',
+ description: 'Weather for a city',
+ inputSchema: { type: 'object' },
+ },
+ ],
+ });
+ expect(payloads[0]?.input).toEqual([
+ { role: 'system', content: 'Be brief.' },
+ {
+ role: 'user',
+ content: [
+ { type: 'text', text: 'Weather?' },
+ { type: 'file', mediaType: 'image', data: '[file]' },
+ ],
+ },
+ {
+ role: 'assistant',
+ content: [
+ {
+ type: 'tool_call',
+ toolCallId: 'call_0',
+ toolName: 'locate',
+ input: {},
+ },
+ ],
+ },
+ {
+ role: 'tool',
+ content: [
+ { type: 'tool_result', toolCallId: 'call_0', output: 'Oslo' },
+ ],
+ },
+ ]);
+ expect(payloads[0]?.output).toEqual([
+ {
+ role: 'assistant',
+ content: [
+ {
+ type: 'tool_call',
+ toolCallId: 'call_1',
+ toolName: 'get_weather',
+ input: { city: 'Oslo' },
+ },
+ ],
+ },
+ ]);
+ });
+
+ test('records a streamed chat completion once the stream is read', async () => {
+ const { openai, payloads } = client([
+ () => sseResponse(textChunks, { done: true }),
+ ]);
+
+ const stream = await openai.chat.completions.create({
+ model: 'gpt-5-mini',
+ messages: [{ role: 'user', content: 'Hi' }],
+ stream: true,
+ stream_options: { include_usage: true },
+ });
+ expect(payloads).toHaveLength(0);
+
+ let text = '';
+ for await (const chunk of stream) {
+ text += chunk.choices[0]?.delta.content ?? '';
+ }
+
+ expect(text).toBe('Hello there');
+ expect(payloads).toHaveLength(1);
+ expect(payloads[0]).toMatchObject({
+ stream: true,
+ stopReason: 'stop',
+ usage: { inputTokens: 12, outputTokens: 2 },
+ output: [{ role: 'assistant', content: 'Hello there' }],
+ });
+ expect(payloads[0]?.timeToFirstTokenMs).toBeNumber();
+ expect(payloads[0]?.error).toBeUndefined();
+ });
+
+ test('assembles streamed tool calls', async () => {
+ const { openai, payloads } = client([
+ () =>
+ sseResponse(
+ [
+ chatChunk({
+ tool_calls: [
+ {
+ index: 0,
+ id: 'call_5',
+ type: 'function',
+ function: { name: 'get_weather', arguments: '{"ci' },
+ },
+ ],
+ }),
+ chatChunk({
+ tool_calls: [
+ { index: 0, function: { arguments: 'ty":"Oslo"}' } },
+ ],
+ }),
+ chatChunk({}, 'tool_calls'),
+ ],
+ { done: true }
+ ),
+ ]);
+
+ const stream = await openai.chat.completions.create({
+ model: 'gpt-5-mini',
+ messages: [{ role: 'user', content: 'Weather?' }],
+ stream: true,
+ });
+ for await (const _chunk of stream) {
+ // drain
+ }
+
+ expect(payloads[0]).toMatchObject({
+ stopReason: 'tool-calls',
+ output: [
+ {
+ role: 'assistant',
+ content: [
+ {
+ type: 'tool_call',
+ toolCallId: 'call_5',
+ toolName: 'get_weather',
+ input: { city: 'Oslo' },
+ },
+ ],
+ },
+ ],
+ });
+ expect(payloads[0]?.usage).toEqual({ inputTokens: 0, outputTokens: 0 });
+ });
+
+ test('records the SDK stream helper through create', async () => {
+ const { openai, payloads } = client([
+ () => sseResponse(textChunks, { done: true }),
+ ]);
+
+ const completion = await openai.chat.completions
+ .stream({
+ model: 'gpt-5-mini',
+ messages: [{ role: 'user', content: 'Hi' }],
+ })
+ .finalChatCompletion();
+
+ expect(completion.choices[0]?.message.content).toBe('Hello there');
+ expect(payloads).toHaveLength(1);
+ expect(payloads[0]).toMatchObject({
+ stream: true,
+ output: [{ role: 'assistant', content: 'Hello there' }],
+ });
+ });
+
+ test('marks a stream the caller stops reading as cancelled', async () => {
+ const { openai, payloads } = client([
+ () => sseResponse(textChunks, { done: true }),
+ ]);
+
+ const stream = await openai.chat.completions.create({
+ model: 'gpt-5-mini',
+ messages: [{ role: 'user', content: 'Hi' }],
+ stream: true,
+ });
+ for await (const _chunk of stream) {
+ break;
+ }
+
+ expect(payloads).toHaveLength(1);
+ expect(payloads[0]?.error?.message).toBe('stream cancelled');
+ expect(payloads[0]?.output).toEqual([
+ { role: 'assistant', content: 'Hello' },
+ ]);
+ });
+
+ test('records a failed call with its status and rethrows', async () => {
+ const { openai, payloads } = client([
+ () =>
+ jsonResponse(
+ {
+ error: { message: 'Invalid model', type: 'invalid_request_error' },
+ },
+ 400
+ ),
+ ]);
+
+ const call = openai.chat.completions.create({
+ model: 'gpt-nope',
+ messages: [{ role: 'user', content: 'Hi' }],
+ });
+
+ // `rejects` reads the native promise state, which APIPromise always resolves.
+ const error = await call.then(
+ () => undefined,
+ (caught: unknown) => caught
+ );
+ expect(error).toBeInstanceOf(OpenAI.BadRequestError);
+ expect(payloads).toHaveLength(1);
+ expect(payloads[0]).toMatchObject({
+ model: 'gpt-nope',
+ error: { httpStatus: 400, code: 'invalid_request_error' },
+ usage: { inputTokens: 0, outputTokens: 0 },
+ });
+ });
+
+ test('records a Responses API call with reasoning and a function call', async () => {
+ const { openai, payloads } = client([
+ () => jsonResponse(functionCallResponse),
+ ]);
+
+ await openai.responses.create({
+ model: 'gpt-5',
+ instructions: 'Be brief.',
+ input: [
+ { role: 'user', content: 'Weather in Oslo?' },
+ {
+ type: 'function_call_output',
+ call_id: 'call_0',
+ output: 'sunny',
+ },
+ ],
+ max_output_tokens: 500,
+ reasoning: { effort: 'low' },
+ });
+
+ expect(payloads[0]).toMatchObject({
+ model: 'gpt-5-2025-08-07',
+ stopReason: 'tool-calls',
+ usage: {
+ inputTokens: 50,
+ outputTokens: 20,
+ cacheReadTokens: 0,
+ reasoningTokens: 12,
+ },
+ modelParameters: { maxTokens: 500, reasoningEffort: 'low' },
+ });
+ expect(payloads[0]?.input).toEqual([
+ { role: 'system', content: 'Be brief.' },
+ { role: 'user', content: 'Weather in Oslo?' },
+ {
+ role: 'tool',
+ content: [
+ { type: 'tool_result', toolCallId: 'call_0', output: 'sunny' },
+ ],
+ },
+ ]);
+ expect(payloads[0]?.output).toEqual([
+ {
+ role: 'assistant',
+ content: [
+ { type: 'reasoning', text: 'Need the weather.' },
+ {
+ type: 'tool_call',
+ toolCallId: 'call_9',
+ toolName: 'get_weather',
+ input: { city: 'Oslo' },
+ },
+ ],
+ },
+ ]);
+ });
+
+ test('records a streamed Responses API call from its final event', async () => {
+ const final = {
+ ...functionCallResponse,
+ output: [
+ {
+ type: 'message',
+ id: 'msg_1',
+ role: 'assistant',
+ status: 'completed',
+ content: [{ type: 'output_text', text: 'Sunny.', annotations: [] }],
+ },
+ ],
+ };
+ const { openai, payloads } = client([
+ () =>
+ sseResponse(
+ [
+ {
+ type: 'response.created',
+ sequence_number: 0,
+ response: { ...final, status: 'in_progress', output: [] },
+ },
+ {
+ type: 'response.output_text.delta',
+ sequence_number: 1,
+ item_id: 'msg_1',
+ output_index: 0,
+ content_index: 0,
+ delta: 'Sunny.',
+ },
+ { type: 'response.completed', sequence_number: 2, response: final },
+ ],
+ { named: true }
+ ),
+ ]);
+
+ const stream = await openai.responses.create({
+ model: 'gpt-5',
+ input: 'Weather?',
+ stream: true,
+ });
+ for await (const _event of stream) {
+ // drain
+ }
+
+ expect(payloads[0]).toMatchObject({
+ stream: true,
+ stopReason: 'stop',
+ model: 'gpt-5-2025-08-07',
+ input: [{ role: 'user', content: 'Weather?' }],
+ output: [
+ { role: 'assistant', content: [{ type: 'text', text: 'Sunny.' }] },
+ ],
+ usage: { inputTokens: 50, outputTokens: 20 },
+ });
+ expect(payloads[0]?.timeToFirstTokenMs).toBeNumber();
+ });
+
+ test('keeps the Responses text that arrived when the reader stops early', async () => {
+ const { openai, payloads } = client([
+ () =>
+ sseResponse(
+ [
+ {
+ type: 'response.created',
+ sequence_number: 0,
+ response: {
+ ...functionCallResponse,
+ status: 'in_progress',
+ output: [],
+ },
+ },
+ {
+ type: 'response.output_text.delta',
+ sequence_number: 1,
+ item_id: 'msg_1',
+ output_index: 0,
+ content_index: 0,
+ delta: 'Sun',
+ },
+ {
+ type: 'response.output_text.delta',
+ sequence_number: 2,
+ item_id: 'msg_1',
+ output_index: 0,
+ content_index: 0,
+ delta: 'ny',
+ },
+ ],
+ { named: true }
+ ),
+ ]);
+
+ const stream = await openai.responses.create({
+ model: 'gpt-5',
+ input: 'Weather?',
+ stream: true,
+ });
+ let seen = 0;
+ for await (const _event of stream) {
+ seen += 1;
+ if (seen === 3) {
+ break;
+ }
+ }
+
+ expect(payloads[0]?.error?.message).toBe('stream cancelled');
+ expect(payloads[0]?.output).toEqual([
+ { role: 'assistant', content: [{ type: 'text', text: 'Sunny' }] },
+ ]);
+ });
+
+ test('keeps partial text from separate Responses output items apart', async () => {
+ const delta = (outputIndex: number, text: string, sequence: number) => ({
+ type: 'response.output_text.delta',
+ sequence_number: sequence,
+ item_id: `msg_${outputIndex}`,
+ output_index: outputIndex,
+ content_index: 0,
+ delta: text,
+ });
+ const { openai, payloads } = client([
+ () =>
+ sseResponse(
+ [
+ {
+ type: 'response.created',
+ sequence_number: 0,
+ response: {
+ ...functionCallResponse,
+ status: 'in_progress',
+ output: [],
+ },
+ },
+ delta(0, 'Let me check.', 1),
+ delta(1, '', 2),
+ delta(1, 'The answer is 5', 3),
+ { ...delta(1, ' (checked)', 4), content_index: 1 },
+ ],
+ { named: true }
+ ),
+ ]);
+
+ const stream = await openai.responses.create({
+ model: 'gpt-5',
+ input: 'Sum?',
+ stream: true,
+ });
+ let seen = 0;
+ for await (const _event of stream) {
+ seen += 1;
+ if (seen === 5) {
+ break;
+ }
+ }
+
+ expect(payloads[0]?.output).toEqual([
+ {
+ role: 'assistant',
+ content: [
+ { type: 'text', text: 'Let me check.' },
+ { type: 'text', text: 'The answer is 5' },
+ { type: 'text', text: ' (checked)' },
+ ],
+ },
+ ]);
+ });
+
+ test('records embeddings with their texts and token count', async () => {
+ const { openai, payloads } = client([
+ () =>
+ jsonResponse({
+ object: 'list',
+ model: 'text-embedding-3-small',
+ data: [{ object: 'embedding', index: 0, embedding: [0.1, 0.2] }],
+ usage: { prompt_tokens: 4, total_tokens: 4 },
+ }),
+ ]);
+
+ await openai.embeddings.create({
+ model: 'text-embedding-3-small',
+ input: ['hello world'],
+ encoding_format: 'float',
+ });
+
+ expect(payloads[0]).toMatchObject({
+ kind: 'embedding',
+ model: 'text-embedding-3-small',
+ input: ['hello world'],
+ usage: { inputTokens: 4, outputTokens: 0 },
+ });
+ expect(payloads[0]?.output).toBeUndefined();
+ });
+
+ test('takes context from withAiContext and wraps a client only once', async () => {
+ const { openai, payloads } = client(
+ [() => jsonResponse(toolCompletion)],
+ 2
+ );
+
+ await withAiContext(
+ { profileId: 'usr_1', conversationId: 'thread_1', name: 'weather' },
+ () =>
+ openai.chat.completions.create({
+ model: 'gpt-5-mini',
+ messages: [{ role: 'user', content: 'Weather?' }],
+ })
+ );
+
+ expect(payloads).toHaveLength(1);
+ expect(payloads[0]).toMatchObject({
+ profileId: 'usr_1',
+ conversationId: 'thread_1',
+ name: 'weather',
+ });
+ });
+
+ test('leaves the body unread for asResponse', async () => {
+ const { openai, payloads } = client([() => jsonResponse(toolCompletion)]);
+
+ const response = await openai.chat.completions
+ .create({
+ model: 'gpt-5-mini',
+ messages: [{ role: 'user', content: 'Hi' }],
+ })
+ .asResponse();
+
+ expect((await response.json()).id).toBe('chatcmpl_1');
+ expect(payloads).toHaveLength(0);
+ });
+});
diff --git a/packages/sdks/ai/src/openai.ts b/packages/sdks/ai/src/openai.ts
new file mode 100644
index 000000000..b6a285d2a
--- /dev/null
+++ b/packages/sdks/ai/src/openai.ts
@@ -0,0 +1,623 @@
+// Records calls made through the official `openai` client: Chat Completions,
+// the Responses API and embeddings, streamed or not. The interfaces below
+// describe only the fields read, so any SDK version with these shapes works.
+
+import type { AiModelParameters, AiUsage } from '@openpanel/sdk';
+import {
+ type CallDescription,
+ type CallResult,
+ type CreateAdapter,
+ definedOnly,
+ instrumentCreate,
+ type StreamAccumulator,
+} from './instrument';
+import {
+ FILE_PLACEHOLDER,
+ type Message,
+ type MessagePart,
+ toolInput,
+} from './messages';
+import type { AiIntegrationOptions } from './payload';
+
+interface ContentPart {
+ type: string;
+ text?: string;
+ refusal?: string;
+ input_audio?: { format?: string };
+}
+
+interface ChatToolCall {
+ id?: string;
+ function?: { name?: string; arguments?: string };
+}
+
+interface ChatMessage {
+ role: string;
+ content?: string | ContentPart[] | null;
+ refusal?: string | null;
+ tool_calls?: ChatToolCall[] | null;
+ tool_call_id?: string;
+}
+
+interface ChatParams {
+ model: string;
+ messages: ChatMessage[];
+ stream?: boolean | null;
+ tools?: {
+ type: string;
+ function?: { name: string; description?: string; parameters?: unknown };
+ custom?: { name: string; description?: string };
+ }[];
+ temperature?: number | null;
+ max_tokens?: number | null;
+ max_completion_tokens?: number | null;
+ top_p?: number | null;
+ frequency_penalty?: number | null;
+ presence_penalty?: number | null;
+ seed?: number | null;
+ stop?: string | string[] | null;
+ reasoning_effort?: string | null;
+}
+
+interface ChatUsage {
+ prompt_tokens?: number;
+ completion_tokens?: number;
+ prompt_tokens_details?: { cached_tokens?: number } | null;
+ completion_tokens_details?: { reasoning_tokens?: number } | null;
+}
+
+interface ChatCompletion {
+ model?: string;
+ choices?: { message?: ChatMessage; finish_reason?: string | null }[];
+ usage?: ChatUsage | null;
+}
+
+interface ChatChunk {
+ model?: string;
+ usage?: ChatUsage | null;
+ choices?: {
+ index?: number;
+ finish_reason?: string | null;
+ delta?: {
+ content?: string | null;
+ refusal?: string | null;
+ tool_calls?: {
+ index: number;
+ id?: string;
+ function?: { name?: string; arguments?: string };
+ }[];
+ };
+ }[];
+}
+
+interface ResponseItem {
+ type?: string;
+ role?: string;
+ content?: string | ContentPart[];
+ call_id?: string;
+ name?: string;
+ arguments?: string;
+ output?: unknown;
+ summary?: { text?: string }[];
+}
+
+interface ResponsesParams {
+ model?: string;
+ input?: string | ResponseItem[];
+ instructions?: string | null;
+ stream?: boolean | null;
+ tools?: {
+ type: string;
+ name?: string;
+ description?: string | null;
+ parameters?: unknown;
+ }[];
+ temperature?: number | null;
+ max_output_tokens?: number | null;
+ top_p?: number | null;
+ reasoning?: { effort?: string | null } | null;
+}
+
+interface ResponseObject {
+ model?: string;
+ status?: string;
+ output?: ResponseItem[];
+ incomplete_details?: { reason?: string } | null;
+ error?: { code?: string; message?: string } | null;
+ usage?: {
+ input_tokens?: number;
+ output_tokens?: number;
+ input_tokens_details?: { cached_tokens?: number } | null;
+ output_tokens_details?: { reasoning_tokens?: number } | null;
+ } | null;
+}
+
+interface ResponseStreamEvent {
+ type?: string;
+ response?: ResponseObject;
+ delta?: string;
+ output_index?: number;
+ content_index?: number;
+}
+
+interface EmbeddingParams {
+ model: string;
+ input: unknown;
+}
+
+interface EmbeddingResponse {
+ model?: string;
+ usage?: { prompt_tokens?: number } | null;
+}
+
+/** The resources wrapped; each is optional so a partial or older client still works. */
+interface OpenAIClient {
+ chat?: { completions?: unknown };
+ responses?: unknown;
+ embeddings?: unknown;
+}
+
+const PROVIDER = 'openai';
+
+const CHAT_STOP_REASONS: Record<string, string> = {
+ stop: 'stop',
+ length: 'length',
+ tool_calls: 'tool-calls',
+ function_call: 'tool-calls',
+ content_filter: 'content-filter',
+};
+
+const INCOMPLETE_STOP_REASONS: Record<string, string> = {
+ max_output_tokens: 'length',
+ content_filter: 'content-filter',
+};
+
+const FINAL_RESPONSE_EVENTS = new Set([
+ 'response.completed',
+ 'response.incomplete',
+ 'response.failed',
+]);
+const RESPONSE_CREATED_EVENT = 'response.created';
+const DELTA_EVENT_SUFFIX = '.delta';
+const OUTPUT_TEXT_DELTA_EVENT = 'response.output_text.delta';
+
+function contentPart(part: ContentPart): MessagePart {
+ switch (part.type) {
+ case 'text':
+ case 'input_text':
+ case 'output_text':
+ return { type: 'text', text: part.text };
+ case 'refusal':
+ return { type: 'text', text: part.refusal };
+ case 'image_url':
+ case 'input_image':
+ return { type: 'file', mediaType: 'image', data: FILE_PLACEHOLDER };
+ case 'input_audio':
+ return {
+ type: 'file',
+ mediaType: `audio/${part.input_audio?.format ?? '*'}`,
+ data: FILE_PLACEHOLDER,
+ };
+ case 'file':
+ case 'input_file':
+ return { type: 'file', data: FILE_PLACEHOLDER };
+ default:
+ return { type: part.type };
+ }
+}
+
+function toolCallPart(call: {
+ id?: string;
+ name?: string;
+ arguments?: string;
+}): MessagePart {
+ return {
+ type: 'tool_call',
+ toolCallId: call.id,
+ toolName: call.name,
+ input: toolInput(call.arguments),
+ };
+}
+
+function textOf(content: ChatMessage['content']): string | undefined {
+ if (typeof content === 'string') {
+ return content;
+ }
+ return content?.map((part) => part.text ?? '').join('');
+}
+
+function chatMessage(message: ChatMessage): Message {
+ if (message.role === 'tool') {
+ return {
+ role: 'tool',
+ content: [
+ {
+ type: 'tool_result',
+ toolCallId: message.tool_call_id,
+ output: textOf(message.content),
+ },
+ ],
+ };
+ }
+ const toolCalls = (message.tool_calls ?? []).map((call) =>
+ toolCallPart({
+ id: call.id,
+ name: call.function?.name,
+ arguments: call.function?.arguments,
+ })
+ );
+ const refusal: MessagePart[] = message.refusal
+ ? [{ type: 'text', text: message.refusal }]
+ : [];
+ const isPlainText =
+ typeof message.content === 'string' &&
+ toolCalls.length === 0 &&
+ refusal.length === 0;
+ if (isPlainText) {
+ return { role: message.role, content: message.content as string };
+ }
+ const parts =
+ typeof message.content === 'string'
+ ? [{ type: 'text', text: message.content }]
+ : (message.content ?? []).map(contentPart);
+ return { role: message.role, content: [...parts, ...refusal, ...toolCalls] };
+}
+
+function chatUsage(usage: ChatUsage | null | undefined): AiUsage | undefined {
+ if (!usage) {
+ return undefined;
+ }
+ return {
+ inputTokens: usage.prompt_tokens ?? 0,
+ outputTokens: usage.completion_tokens ?? 0,
+ cacheReadTokens: usage.prompt_tokens_details?.cached_tokens,
+ reasoningTokens: usage.completion_tokens_details?.reasoning_tokens,
+ };
+}
+
+function chatStopReason(reason: string | null | undefined): string | undefined {
+ return reason ? (CHAT_STOP_REASONS[reason] ?? 'other') : undefined;
+}
+
+function stopSequences(stop: ChatParams['stop']): string[] | undefined {
+ if (!stop) {
+ return undefined;
+ }
+ return typeof stop === 'string' ? [stop] : stop;
+}
+
+function chatTools(tools: ChatParams['tools']): unknown[] | undefined {
+ if (!tools || tools.length === 0) {
+ return undefined;
+ }
+ return tools.map((tool) => {
+ const definition = tool.function ?? tool.custom;
+ return {
+ type: tool.type,
+ name: definition?.name,
+ description: definition?.description,
+ inputSchema: tool.function?.parameters,
+ };
+ });
+}
+
+class ChatStreamAccumulator implements StreamAccumulator {
+ private text = '';
+ private refusal = '';
+ private readonly toolCalls = new Map<number, Required<ChatToolCall>>();
+ private finishReason: string | null | undefined;
+ private model: string | undefined;
+ private usage: ChatUsage | null | undefined;
+
+ observe(value: unknown): boolean {
+ const chunk = value as ChatChunk;
+ this.model = chunk.model ?? this.model;
+ this.usage = chunk.usage ?? this.usage;
+ // Only the first choice is kept, as for a streamed `n: 1` request.
+ const choice = chunk.choices?.find((entry) => (entry.index ?? 0) === 0);
+ if (!choice) {
+ return false;
+ }
+ this.finishReason = choice.finish_reason ?? this.finishReason;
+ const delta = choice.delta;
+ this.text += delta?.content ?? '';
+ this.refusal += delta?.refusal ?? '';
+ for (const call of delta?.tool_calls ?? []) {
+ const entry = this.toolCalls.get(call.index) ?? {
+ id: '',
+ function: { name: '', arguments: '' },
+ };
+ entry.id ||= call.id ?? '';
+ entry.function.name ||= call.function?.name ?? '';
+ entry.function.arguments += call.function?.arguments ?? '';
+ this.toolCalls.set(call.index, entry);
+ }
+ return Boolean(
+ delta?.content || delta?.refusal || delta?.tool_calls?.length
+ );
+ }
+
+ result(): CallResult {
+ const message: ChatMessage = {
+ role: 'assistant',
+ content: this.text || null,
+ refusal: this.refusal || null,
+ tool_calls: [...this.toolCalls.values()],
+ };
+ return {
+ output: [chatMessage(message)],
+ usage: chatUsage(this.usage),
+ model: this.model,
+ stopReason: chatStopReason(this.finishReason),
+ };
+ }
+}
+
+const chatCompletions: CreateAdapter = {
+ describe(value): CallDescription {
+ const params = value as ChatParams;
+ return {
+ model: params.model,
+ provider: PROVIDER,
+ stream: params.stream === true,
+ input: params.messages.map(chatMessage),
+ tools: chatTools(params.tools),
+ modelParameters: definedOnly<AiModelParameters>({
+ temperature: params.temperature ?? undefined,
+ maxTokens:
+ params.max_completion_tokens ?? params.max_tokens ?? undefined,
+ topP: params.top_p ?? undefined,
+ frequencyPenalty: params.frequency_penalty ?? undefined,
+ presencePenalty: params.presence_penalty ?? undefined,
+ seed: params.seed ?? undefined,
+ stopSequences: stopSequences(params.stop),
+ reasoningEffort: params.reasoning_effort ?? undefined,
+ }),
+ };
+ },
+ result(value): CallResult {
+ const completion = value as ChatCompletion;
+ return {
+ output: (completion.choices ?? []).flatMap((choice) =>
+ choice.message ? [chatMessage(choice.message)] : []
+ ),
+ usage: chatUsage(completion.usage),
+ model: completion.model,
+ stopReason: chatStopReason(completion.choices?.[0]?.finish_reason),
+ };
+ },
+ accumulator: () => new ChatStreamAccumulator(),
+};
+
+function itemParts(item: ResponseItem): MessagePart[] {
+ switch (item.type) {
+ case 'function_call':
+ return [
+ toolCallPart({
+ id: item.call_id,
+ name: item.name,
+ arguments: item.arguments,
+ }),
+ ];
+ case 'function_call_output':
+ return [
+ { type: 'tool_result', toolCallId: item.call_id, output: item.output },
+ ];
+ case 'reasoning':
+ return (item.summary ?? []).map((summary) => ({
+ type: 'reasoning',
+ text: summary.text,
+ }));
+ case undefined:
+ case 'message':
+ return typeof item.content === 'string'
+ ? [{ type: 'text', text: item.content }]
+ : (item.content ?? []).map(contentPart);
+ default:
+ return [{ type: item.type }];
+ }
+}
+
+function itemRole(item: ResponseItem): string {
+ if (item.type === 'function_call_output') {
+ return 'tool';
+ }
+ return item.role ?? 'assistant';
+}
+
+function responsesInput(params: ResponsesParams): Message[] {
+ const instructions: Message[] = params.instructions
+ ? [{ role: 'system', content: params.instructions }]
+ : [];
+ if (typeof params.input === 'string') {
+ return [...instructions, { role: 'user', content: params.input }];
+ }
+ const items = (params.input ?? []).map((item) => {
+ const isPlainText =
+ typeof item.content === 'string' &&
+ (item.type === undefined || item.type === 'message');
+ return {
+ role: itemRole(item),
+ content: isPlainText ? (item.content as string) : itemParts(item),
+ };
+ });
+ return [...instructions, ...items];
+}
+
+function responsesStopReason(response: ResponseObject): string | undefined {
+ switch (response.status) {
+ case 'failed':
+ return 'error';
+ case 'incomplete':
+ return (
+ INCOMPLETE_STOP_REASONS[response.incomplete_details?.reason ?? ''] ??
+ 'other'
+ );
+ case 'completed':
+ return response.output?.some((item) => item.type === 'function_call')
+ ? 'tool-calls'
+ : 'stop';
+ default:
+ return response.status;
+ }
+}
+
+function responsesError(response: ResponseObject): Error | undefined {
+ if (!response.error) {
+ return undefined;
+ }
+ const error = new Error(response.error.message ?? 'response failed');
+ error.name = response.error.code ?? error.name;
+ return error;
+}
+
+function responsesResult(response: ResponseObject): CallResult {
+ const output = (response.output ?? []).flatMap(itemParts);
+ const usage = response.usage;
+ return {
+ output: output.length > 0 ? [{ role: 'assistant', content: output }] : [],
+ usage: usage
+ ? {
+ inputTokens: usage.input_tokens ?? 0,
+ outputTokens: usage.output_tokens ?? 0,
+ cacheReadTokens: usage.input_tokens_details?.cached_tokens,
+ reasoningTokens: usage.output_tokens_details?.reasoning_tokens,
+ }
+ : undefined,
+ model: response.model,
+ stopReason: responsesStopReason(response),
+ error: responsesError(response),
+ };
+}
+
+/**
+ * The final `response.*` event carries the whole response. Text deltas are
+ * kept only for a stream the caller stops reading before that event.
+ */
+class ResponsesStreamAccumulator implements StreamAccumulator {
+ private model: string | undefined;
+ private response: ResponseObject | undefined;
+ /** Text per output item and content part, so separate parts don't run together. */
+ private readonly partialText = new Map<
+ string,
+ { item: number; part: number; text: string }
+ >();
+
+ observe(value: unknown): boolean {
+ const event = value as ResponseStreamEvent;
+ const type = event.type ?? '';
+ if (type === RESPONSE_CREATED_EVENT) {
+ this.model = event.response?.model;
+ }
+ if (FINAL_RESPONSE_EVENTS.has(type)) {
+ this.response = event.response;
+ }
+ if (type === OUTPUT_TEXT_DELTA_EVENT) {
+ this.addPartialText(event);
+ }
+ return type.endsWith(DELTA_EVENT_SUFFIX);
+ }
+
+ private addPartialText(event: ResponseStreamEvent): void {
+ if (!event.delta) {
+ return;
+ }
+ const item = event.output_index ?? 0;
+ const part = event.content_index ?? 0;
+ const key = `${item}:${part}`;
+ const previous = this.partialText.get(key)?.text ?? '';
+ this.partialText.set(key, { item, part, text: previous + event.delta });
+ }
+
+ result(): CallResult {
+ if (this.response) {
+ return responsesResult(this.response);
+ }
+ const parts = [...this.partialText.values()]
+ .sort((a, b) => a.item - b.item || a.part - b.part)
+ .map(({ text }): MessagePart => ({ type: 'text', text }));
+ const output: Message[] =
+ parts.length > 0 ? [{ role: 'assistant', content: parts }] : [];
+ return { model: this.model, output };
+ }
+}
+
+const responses: CreateAdapter = {
+ describe(value): CallDescription {
+ const params = value as ResponsesParams;
+ return {
+ model: params.model ?? '',
+ provider: PROVIDER,
+ stream: params.stream === true,
+ input: responsesInput(params),
+ tools: params.tools?.length
+ ? params.tools.map((tool) => ({
+ type: tool.type,
+ name: tool.name,
+ description: tool.description ?? undefined,
+ inputSchema: tool.parameters,
+ }))
+ : undefined,
+ modelParameters: definedOnly<AiModelParameters>({
+ temperature: params.temperature ?? undefined,
+ maxTokens: params.max_output_tokens ?? undefined,
+ topP: params.top_p ?? undefined,
+ reasoningEffort: params.reasoning?.effort ?? undefined,
+ }),
+ };
+ },
+ result: (value) => responsesResult(value as ResponseObject),
+ accumulator: () => new ResponsesStreamAccumulator(),
+};
+
+/** Token-id inputs are not text, so only string inputs are kept. */
+function embeddedTexts(input: unknown): string[] | undefined {
+ const values = Array.isArray(input) ? input : [input];
+ const texts = values.filter(
+ (value): value is string => typeof value === 'string'
+ );
+ return texts.length > 0 ? texts : undefined;
+}
+
+const embeddings: CreateAdapter = {
+ describe(value): CallDescription {
+ const params = value as EmbeddingParams;
+ return {
+ kind: 'embedding',
+ model: params.model,
+ provider: PROVIDER,
+ stream: false,
+ input: embeddedTexts(params.input),
+ };
+ },
+ result(value): CallResult {
+ const response = value as EmbeddingResponse;
+ return {
+ model: response.model,
+ usage: response.usage
+ ? { inputTokens: response.usage.prompt_tokens ?? 0, outputTokens: 0 }
+ : undefined,
+ };
+ },
+};
+
+/**
+ * Records every Chat Completions, Responses and embeddings call made through
+ * an `openai` client as an OpenPanel AI event, streamed or not, including
+ * the SDK's own helpers (`parse`, `stream`, `runTools`):
+ *
+ * const openai = wrapOpenAI(new OpenAI(), { op });
+ *
+ * The client is instrumented in place and returned. Per-request context
+ * (user, conversation, name) comes from `withAiContext`.
+ */
+export function wrapOpenAI<Client extends object>(
+ client: Client,
+ options: AiIntegrationOptions
+): Client {
+ const resources = client as OpenAIClient;
+ instrumentCreate(resources.chat?.completions, chatCompletions, options);
+ instrumentCreate(resources.responses, responses, options);
+ instrumentCreate(resources.embeddings, embeddings, options);
+ return client;
+}
diff --git a/packages/sdks/ai/src/payload.test.ts b/packages/sdks/ai/src/payload.test.ts
index bf822f3f6..57fb3f9eb 100644
--- a/packages/sdks/ai/src/payload.test.ts
+++ b/packages/sdks/ai/src/payload.test.ts
@@ -1,6 +1,7 @@
import { describe, expect, test } from 'bun:test';
import type { AiPayload } from '@openpanel/sdk';
import { withAiContext } from './context';
+import { instrumentCreate } from './instrument';
import {
type AiCall,
buildPayload,
@@ -8,6 +9,7 @@ import {
recordCall,
resolveContext,
} from './payload';
+import { collect } from './test-helpers';
const call: AiCall = {
context: { profileId: 'usr_1', traceId: 'trace_1' },
@@ -127,3 +129,33 @@ describe('recordCall', () => {
expect(sent[0]?.kind).toBe('generation');
});
});
+
+describe('instrumentCreate', () => {
+ test('records a create that throws before returning, and rethrows', () => {
+ const { op, payloads } = collect();
+ const invalid = new Error(
+ 'max_tokens is too large for a non-streaming call'
+ );
+ const resource = {
+ create: () => {
+ throw invalid;
+ },
+ };
+ instrumentCreate(
+ resource,
+ {
+ describe: () => ({
+ model: 'claude-x',
+ provider: 'anthropic',
+ stream: false,
+ }),
+ result: () => ({}),
+ },
+ { op }
+ );
+
+ expect(() => resource.create()).toThrow(invalid);
+ expect(payloads).toHaveLength(1);
+ expect(payloads[0]?.error?.message).toBe(invalid.message);
+ });
+});
diff --git a/packages/sdks/ai/src/payload.ts b/packages/sdks/ai/src/payload.ts
index 9ab747f04..37beae771 100644
--- a/packages/sdks/ai/src/payload.ts
+++ b/packages/sdks/ai/src/payload.ts
@@ -25,11 +25,14 @@ export interface AiIntegrationOptions {
/** What an integration knows when a model call starts. */
export interface AiCall {
+ /** Defaults to `generation`. */
+ kind?: 'generation' | 'embedding';
context: AiCallContext;
model: string;
provider: string;
startedAt: number;
- input?: Message[];
+ /** Messages for a generation, the embedded texts for an embedding. */
+ input?: Message[] | string[];
tools?: unknown;
modelParameters?: AiPayload['modelParameters'];
}
@@ -49,6 +52,9 @@ export interface AiCallOutcome {
const MAX_ERROR_MESSAGE_LENGTH = 2000;
const HTTP_STATUS_MIN = 100;
const HTTP_STATUS_MAX = 599;
+const GENERIC_ERROR_NAME = 'Error';
+// Ingest requires token counts; a failed call or a stream without usage has none.
+const UNKNOWN_USAGE = { inputTokens: 0, outputTokens: 0 } as const;
/** Per-call context wins over the ambient `withAiContext`, which wins over `defaults`. */
export function resolveContext(
@@ -73,9 +79,11 @@ export function errorOf(error: unknown): NonNullable<AiPayload['error']> {
return { message: String(error).slice(0, MAX_ERROR_MESSAGE_LENGTH) };
}
// Provider SDK errors carry the HTTP status as `statusCode` (AI SDK) or `status` (OpenAI, Anthropic).
- const { statusCode, status } = error as {
+ const { statusCode, status, code, type } = error as {
statusCode?: unknown;
status?: unknown;
+ code?: unknown;
+ type?: unknown;
};
const candidate = typeof statusCode === 'number' ? statusCode : status;
const httpStatus =
@@ -84,9 +92,12 @@ export function errorOf(error: unknown): NonNullable<AiPayload['error']> {
candidate <= HTTP_STATUS_MAX
? candidate
: undefined;
+ // Some SDKs (OpenAI 7) leave `name` as plain `Error` and say what went wrong in `code` or `type`.
+ const providerCode = [code, type].find((value) => typeof value === 'string');
+ const isGenericName = error.name === GENERIC_ERROR_NAME;
return {
message: error.message.slice(0, MAX_ERROR_MESSAGE_LENGTH),
- code: error.name,
+ code: isGenericName ? (providerCode ?? error.name) : error.name,
httpStatus,
};
}
@@ -100,7 +111,7 @@ export function buildPayload(
const privacyMode = options.privacyMode || context.privacyMode;
const hasProperties = Object.keys(context.properties ?? {}).length > 0;
return {
- kind: 'generation',
+ kind: call.kind ?? 'generation',
traceId: context.traceId ?? createAiTraceId(),
parentId: context.parentId,
name: context.name,
@@ -110,7 +121,7 @@ export function buildPayload(
input: privacyMode ? undefined : call.input,
output: privacyMode ? undefined : outcome.output,
tools: privacyMode ? undefined : call.tools,
- usage: outcome.usage,
+ usage: outcome.usage ?? UNKNOWN_USAGE,
latencyMs: Date.now() - call.startedAt,
timeToFirstTokenMs: outcome.timeToFirstTokenMs,
error: outcome.error === undefined ? undefined : errorOf(outcome.error),
diff --git a/packages/sdks/ai/src/test-helpers.ts b/packages/sdks/ai/src/test-helpers.ts
new file mode 100644
index 000000000..8c0f84596
--- /dev/null
+++ b/packages/sdks/ai/src/test-helpers.ts
@@ -0,0 +1,51 @@
+import type { AiPayload } from '@openpanel/sdk';
+
+export interface RecordedRequest {
+ url: string;
+ body: Record<string, unknown>;
+}
+
+export function collect() {
+ const payloads: AiPayload[] = [];
+ const op = { ai: (payload: AiPayload) => payloads.push(payload) };
+ return { op, payloads };
+}
+
+export function jsonResponse(body: unknown, status = 200): Response {
+ return new Response(JSON.stringify(body), {
+ status,
+ headers: { 'content-type': 'application/json', 'x-request-id': 'req_1' },
+ });
+}
+
+/** Server-sent events; `event` lines are included when an event names its type. */
+export function sseResponse(
+ events: unknown[],
+ { named = false, done = false } = {}
+): Response {
+ const lines = events.map((event) => {
+ const type = (event as { type?: string }).type;
+ const prefix = named && type ? `event: ${type}\n` : '';
+ return `${prefix}data: ${JSON.stringify(event)}\n\n`;
+ });
+ const text = lines.join('') + (done ? 'data: [DONE]\n\n' : '');
+ return new Response(text, {
+ headers: { 'content-type': 'text/event-stream' },
+ });
+}
+
+/** A `fetch` that answers each request with the next queued response. */
+export function mockFetch(responses: (() => Response)[]) {
+ const requests: RecordedRequest[] = [];
+ const queue = [...responses];
+ const fetch = async (input: string | URL | Request, init?: RequestInit) => {
+ const url = input instanceof Request ? input.url : String(input);
+ requests.push({ url, body: JSON.parse(String(init?.body ?? '{}')) });
+ const next = queue.shift();
+ if (!next) {
+ throw new Error(`unexpected request to ${url}`);
+ }
+ return next();
+ };
+ return { fetch, requests };
+}
diff --git a/packages/sdks/ai/tsup.config.ts b/packages/sdks/ai/tsup.config.ts
index 57ef9542e..b3c5b1aa2 100644
--- a/packages/sdks/ai/tsup.config.ts
+++ b/packages/sdks/ai/tsup.config.ts
@@ -1,7 +1,7 @@
import { defineConfig } from 'tsup';
export default defineConfig({
- entry: ['index.ts', 'vercel.ts'],
+ entry: ['index.ts', 'vercel.ts', 'openai.ts', 'anthropic.ts'],
format: ['cjs', 'esm'],
dts: { resolve: [/^@openpanel\//] },
splitting: false,