Skip to content

armgit / eve

Public Imported from GitHub
0

The Open Framework for Building Agents

feat(eve): split cache-aware token usage on local trace spans (OTel GenAI semconv) (#1391)

Chad Hietaladbc8eae4b814
diff --git a/.changeset/tidy-pandas-greet.md b/.changeset/tidy-pandas-greet.md
new file mode 100644
index 000000000..db3c7474a
--- /dev/null
+++ b/.changeset/tidy-pandas-greet.md
@@ -0,0 +1,5 @@
+---
+"eve": patch
+---
+
+Splits token usage on local trace spans: model and step spans now record `gen_ai.usage.cache_read.input_tokens` and `gen_ai.usage.cache_creation.input_tokens` (OTel GenAI semantic-convention names) alongside the input/output totals when the provider reports detailed usage — cached tokens price differently, so the split makes cost attribution exact. Providers without detailed usage emit only the totals.
diff --git a/docs/guides/instrumentation.md b/docs/guides/instrumentation.md
index 6d8cf4956..c01ece21d 100644
--- a/docs/guides/instrumentation.md
+++ b/docs/guides/instrumentation.md
@@ -17,6 +17,8 @@ Use `eve trace ls` to list captured traces and `eve trace <trace>` to inspect a
When a model call is served by Vercel AI Gateway, its `agent.step` span also carries the cost the gateway reported: `gen_ai.usage.cost` (raw inference, USD), `gen_ai.usage.gateway_cost` (with the gateway surcharge), `gen_ai.usage.input_cost` / `gen_ai.usage.output_cost` (the split), and `gen_ai.generation.id` for reconciliation with the gateway dashboard. These attributes only exist for gateway-served calls — other providers emit nothing. Cost per turn is the sum across the turn's step spans.
+Model and step spans also split token usage when the provider reports details: `gen_ai.usage.cache_read.input_tokens` and `gen_ai.usage.cache_creation.input_tokens` (named for the [OTel GenAI semantic conventions](https://github.com/open-telemetry/semantic-conventions-genai)) alongside the `agent.usage.input_tokens` / `agent.usage.output_tokens` totals — cached tokens price differently, so the split makes cost attribution exact. Providers without detailed usage emit only the totals.
+
The local writer is an internal development default, not a second provider layered over authored instrumentation. When `instrumentation.ts` exists, its setup retains control and the zero-config writer is not installed.
### Local trace retention
diff --git a/packages/eve/src/harness/agent-otel-provider.test.ts b/packages/eve/src/harness/agent-otel-provider.test.ts
index a77dfe1b1..dcc6e545f 100644
--- a/packages/eve/src/harness/agent-otel-provider.test.ts
+++ b/packages/eve/src/harness/agent-otel-provider.test.ts
@@ -86,7 +86,11 @@ async function emitAttempt(input: {
finishReason: "tool-calls",
performance: { responseTimeMs: 10 },
responseId: "response-1",
- usage: { inputTokens: 10, outputTokens: 5 },
+ usage: {
+ inputTokenDetails: { cacheReadTokens: 4, cacheWriteTokens: 2 },
+ inputTokens: 10,
+ outputTokens: 5,
+ },
},
]);
await Reflect.apply(bridge.onToolExecutionStart!, bridge, [
@@ -202,6 +206,8 @@ describe("createAgentOtelInstrumentation", () => {
"agent.root.session.id": "session-1",
"agent.usage.input_tokens": 10,
"agent.usage.output_tokens": 5,
+ "gen_ai.usage.cache_creation.input_tokens": 2,
+ "gen_ai.usage.cache_read.input_tokens": 4,
});
expect(action.attributes).toMatchObject({
"agent.action.kind": "tool",
diff --git a/packages/eve/src/harness/agent-otel-provider.ts b/packages/eve/src/harness/agent-otel-provider.ts
index c3770f7aa..45fffec3e 100644
--- a/packages/eve/src/harness/agent-otel-provider.ts
+++ b/packages/eve/src/harness/agent-otel-provider.ts
@@ -516,7 +516,14 @@ function modelSpanName(operationName: string): string {
function setUsage(
span: Span,
- usage: { readonly inputTokens?: number; readonly outputTokens?: number },
+ usage: {
+ readonly inputTokenDetails?: {
+ readonly cacheReadTokens?: number;
+ readonly cacheWriteTokens?: number;
+ };
+ readonly inputTokens?: number;
+ readonly outputTokens?: number;
+ },
): void {
if (usage.inputTokens !== undefined) {
span.setAttribute("agent.usage.input_tokens", usage.inputTokens);
@@ -524,6 +531,16 @@ function setUsage(
if (usage.outputTokens !== undefined) {
span.setAttribute("agent.usage.output_tokens", usage.outputTokens);
}
+ // Cached tokens price differently from plain input, so keep the split.
+ // Named for the OTel GenAI semantic conventions; present only when the
+ // provider reports details — others emit nothing.
+ const details = usage.inputTokenDetails;
+ if (details?.cacheReadTokens !== undefined) {
+ span.setAttribute("gen_ai.usage.cache_read.input_tokens", details.cacheReadTokens);
+ }
+ if (details?.cacheWriteTokens !== undefined) {
+ span.setAttribute("gen_ai.usage.cache_creation.input_tokens", details.cacheWriteTokens);
+ }
}
function recordError(span: Span, error: unknown): void {