The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu
Go agent-ai-cliagent-harnessesagent-securitybrowser-extensionciclaude-codecloudcodexcursordetection-engineeringjevknowledge-basememorymobile-device-managementobservabilitysecuritysecurity-information-and-event-managementsecurity-toolstelemetrytraces
fix(exporter): store semconv-inclusive input tokens as uncached input
Claude778c2960f091
diff --git a/browser-extension/src/shared/normalize.ts b/browser-extension/src/shared/normalize.tsindex dca9ce15..4066a85f 100644--- a/browser-extension/src/shared/normalize.ts+++ b/browser-extension/src/shared/normalize.ts@@ -144,13 +144,25 @@ export function normalizeTurn(const respAttrs = base(respAction);if (turn.requestModel) respAttrs.push(str('gen_ai.request.model', turn.requestModel));if (turn.responseModel) respAttrs.push(str('gen_ai.response.model', turn.responseModel));+ // The turn's inputTokens is uncached input (Anthropic's usage.input_tokens),+ // but the OTel GenAI semconv input_tokens "SHOULD include all types of input+ // tokens, including cached tokens", and the collector reads it that way:+ // GenAIUsageFromAttrs subtracts the cache counts below to store uncached input.+ // So the semconv count is emitted here, cache reads and writes folded in.if (turn.usage?.inputTokens != null)- respAttrs.push(int('gen_ai.usage.input_tokens', turn.usage.inputTokens));+ respAttrs.push(+ int(+ 'gen_ai.usage.input_tokens',+ turn.usage.inputTokens ++ (turn.usage.cacheReadInputTokens ?? 0) ++ (turn.usage.cacheCreationInputTokens ?? 0),+ ),+ );if (turn.usage?.outputTokens != null)respAttrs.push(int('gen_ai.usage.output_tokens', turn.usage.outputTokens));// Cache counts use the dotted spelling the collector reads first- // (GenAIUsageFromAttrs), and are disjoint from input_tokens, so a total is- // input + output + cache_read + cache_creation with nothing double-counted.+ // (GenAIUsageFromAttrs). They are subsets of input_tokens above, and the+ // collector stores them disjoint from the uncached remainder.if (turn.usage?.cacheCreationInputTokens != null)respAttrs.push(int('gen_ai.usage.cache_creation.input_tokens', turn.usage.cacheCreationInputTokens),diff --git a/browser-extension/test/unit/normalize.test.ts b/browser-extension/test/unit/normalize.test.tsindex ed4aea5f..c34374c8 100644--- a/browser-extension/test/unit/normalize.test.ts+++ b/browser-extension/test/unit/normalize.test.ts@@ -106,6 +106,40 @@ describe('normalizeTurn — full retention, simple completed turn', () => {expect(r['gen_ai.usage.cache_read.input_tokens']).toBe(18500);});+ it('emits the semconv input count, which includes cache reads and writes', () => {+ // Anthropic's usage.input_tokens excludes both cache counts, but the OTel+ // GenAI semconv input_tokens "SHOULD include all types of input tokens,+ // including cached tokens". The collector reduces every semconv input by+ // the cache counts beside it, so emitting Anthropic's disjoint count here+ // would have it subtract the cache a second time.+ const records = normalize(+ baseTurn({+ usage: { inputTokens: 37, outputTokens: 214, cacheCreationInputTokens: 1024, cacheReadInputTokens: 18500 },+ }),+ ).logRecords;+ const r = flat(byAction(records, 'agent.response.completed')!.attributes);+ expect(r['gen_ai.usage.input_tokens']).toBe(37 + 1024 + 18500);+ expect(r['gen_ai.usage.output_tokens']).toBe(214);+ });++ it('adds only the cache counts the stream reported to the input count', () => {+ const records = normalize(+ baseTurn({ usage: { inputTokens: 37, outputTokens: 5, cacheReadInputTokens: 900 } }),+ ).logRecords;+ const r = flat(byAction(records, 'agent.response.completed')!.attributes);+ expect(r['gen_ai.usage.input_tokens']).toBe(937);+ expect(r['gen_ai.usage.cache_creation.input_tokens']).toBeUndefined();+ });++ it('does not invent an input count from cache counts alone', () => {+ const records = normalize(+ baseTurn({ usage: { outputTokens: 5, cacheReadInputTokens: 900 } }),+ ).logRecords;+ const r = flat(byAction(records, 'agent.response.completed')!.attributes);+ expect(r['gen_ai.usage.input_tokens']).toBeUndefined();+ expect(r['gen_ai.usage.cache_read.input_tokens']).toBe(900);+ });+it('omits cache attributes entirely when the stream reported none', () => {const r = flat(respRec.attributes);expect(r['gen_ai.usage.cache_creation.input_tokens']).toBeUndefined();diff --git a/collector-builder/exporter/beaconjsonexporter/README.md b/collector-builder/exporter/beaconjsonexporter/README.mdindex 868a1766..8f1f4c32 100644--- a/collector-builder/exporter/beaconjsonexporter/README.md+++ b/collector-builder/exporter/beaconjsonexporter/README.md@@ -48,6 +48,14 @@ Token and cost usage:not its anonymous histogram. The exporter keeps that one span by default,promotes `thread.id`/`turn.id`/model, and maps cache reads and cache writes todisjoint canonical usage fields. Legacy Codex metric events remain readable.+- Span and log attributes under the OTel GenAI semconv input names+ (`gen_ai.usage.input_tokens`, and the older `gen_ai.usage.prompt_tokens` /+ `llm.usage.prompt_tokens`) count cached prompt tokens as part of input, so the+ exporter subtracts the cache read and cache creation (`cache_write`) counts+ reported beside them to store uncached input. An input smaller than those+ cache counts is already uncached and is stored as reported. Claude Code's bare+ `input_tokens` is Anthropic's uncached count and is never reduced; `raw` keeps+ every value as the runtime sent it.Noise controls:diff --git a/collector-builder/exporter/beaconjsonexporter/exporter_test.go b/collector-builder/exporter/beaconjsonexporter/exporter_test.goindex 1ceb9de2..39d19b46 100644--- a/collector-builder/exporter/beaconjsonexporter/exporter_test.go+++ b/collector-builder/exporter/beaconjsonexporter/exporter_test.go@@ -1067,6 +1067,66 @@ func TestInferActionMapsMeaningfulToolCallArgumentsToTool(t *testing.T) {}}+// An OpenLLMetry-instrumented Anthropic call reports gen_ai.usage.input_tokens+// as usage.input_tokens + cache reads + cache writes, the semconv's inclusive+// count, beside the two cache counts. The JSONL line must carry uncached input+// so input + output + cache_read + cache_creation is the call's real total,+// while raw keeps the value the runtime sent.+func TestConsumeTracesStoresSemconvInclusiveInputAsUncached(t *testing.T) {+ path := filepath.Join(t.TempDir(), "runtime.jsonl")+ exp, err := newExporter(&Config{+ Path: path,+ MaxEventBytes: defaultMaxEventBytes,+ RotateBytes: defaultRotateBytes,+ RedactSecrets: true,+ }, exporter.Settings{})+ if err != nil {+ t.Fatalf("newExporter returned error: %v", err)+ }++ traces := ptrace.NewTraces()+ rs := traces.ResourceSpans().AppendEmpty()+ rs.Resource().Attributes().PutStr("service.name", "support-agent")+ span := testSpan("anthropic.chat")+ span.Attributes().PutStr("gen_ai.system", "Anthropic")+ span.Attributes().PutStr("gen_ai.operation.name", "chat")+ span.Attributes().PutStr("gen_ai.conversation.id", "conv-cached")+ span.Attributes().PutStr("gen_ai.request.model", "claude-sonnet-4-5")+ span.Attributes().PutInt("gen_ai.usage.input_tokens", 19561)+ span.Attributes().PutInt("gen_ai.usage.output_tokens", 214)+ span.Attributes().PutInt("gen_ai.usage.cache_read_input_tokens", 18500)+ span.Attributes().PutInt("gen_ai.usage.cache_creation_input_tokens", 1024)+ span.Attributes().PutInt("llm.usage.total_tokens", 19775)+ span.CopyTo(rs.ScopeSpans().AppendEmpty().Spans().AppendEmpty())++ if err := exp.consumeTraces(context.Background(), traces); err != nil {+ t.Fatalf("consumeTraces returned error: %v", err)+ }+ data, err := os.ReadFile(path)+ if err != nil {+ t.Fatalf("read runtime log: %v", err)+ }+ var event beaconEvent+ if err := json.Unmarshal([]byte(strings.TrimSpace(string(data))), &event); err != nil {+ t.Fatalf("unmarshal event: %v (%s)", err, string(data))+ }+ usage := event.GenAI.Usage+ if usage == nil || usage.InputTokens == nil || usage.OutputTokens == nil || usage.CacheRead == nil || usage.CacheCreation == nil {+ t.Fatalf("usage incomplete: %#v", usage)+ }+ if *usage.InputTokens != 37 || *usage.OutputTokens != 214 || *usage.CacheRead.InputTokens != 18500 || *usage.CacheCreation.InputTokens != 1024 {+ t.Fatalf("usage = input %d output %d cache_read %d cache_creation %d, want 37/214/18500/1024",+ *usage.InputTokens, *usage.OutputTokens, *usage.CacheRead.InputTokens, *usage.CacheCreation.InputTokens)+ }+ if total := *usage.InputTokens + *usage.OutputTokens + *usage.CacheRead.InputTokens + *usage.CacheCreation.InputTokens; total != 19775 {+ t.Fatalf("disjoint total = %d, want the runtime's own total 19775", total)+ }+ rawAttrs, _ := event.Raw["attributes"].(map[string]interface{})+ if got, _ := rawAttrs["gen_ai.usage.input_tokens"].(float64); got != 19561 {+ t.Fatalf("raw gen_ai.usage.input_tokens = %v, want the reported 19561", rawAttrs["gen_ai.usage.input_tokens"])+ }+}+func TestConsumeTracesDropsCodexUserInputSpan(t *testing.T) {path := filepath.Join(t.TempDir(), "runtime.jsonl")exp, err := newExporter(&Config{diff --git a/collector-builder/exporter/beaconjsonexporter/internal/beaconevent/converter.go b/collector-builder/exporter/beaconjsonexporter/internal/beaconevent/converter.goindex 7af807ef..8f50e338 100644--- a/collector-builder/exporter/beaconjsonexporter/internal/beaconevent/converter.go+++ b/collector-builder/exporter/beaconjsonexporter/internal/beaconevent/converter.go@@ -1354,20 +1354,53 @@ func GenAIToolFromAttrs(attrs map[string]interface{}) *GenAIToolInfo {return tool}+// inclusiveInputTokenKeys are the input-count names whose value includes cached+// prompt tokens. The OTel GenAI semconv says of gen_ai.usage.input_tokens "This+// value SHOULD include all types of input tokens, including cached tokens", and+// of each cache count "The value SHOULD be included in gen_ai.usage.input_tokens".+// The deprecated prompt_tokens spellings it replaced carry OpenAI's prompt_tokens,+// which counts prompt_tokens_details.cached_tokens too. Claude Code's bare+// input_tokens is deliberately absent: it is Anthropic's usage.input_tokens,+// which already excludes cache reads and writes.+var inclusiveInputTokenKeys = []string{"gen_ai.usage.input_tokens", "llm.usage.prompt_tokens", "gen_ai.usage.prompt_tokens"}+// GenAIUsageFromAttrs normalizes runtime token usage into the canonical// gen_ai.usage struct. Alongside the OTel GenAI and legacy llm.usage.* semconv// names it accepts Claude Code's bare claude_code.llm_request span attribute// names (input_tokens, output_tokens, cache_read_tokens, cache_creation_tokens)// so span-level usage and the per-step session drilldown carry real counts.+//+// Beacon keeps input, cache read and cache creation disjoint, so an inclusive+// input count (see inclusiveInputTokenKeys) is reduced by the cache counts+// reported beside it, the same reduction the Codex turn span gets. The raw+// attributes keep the value as reported.func GenAIUsageFromAttrs(attrs map[string]interface{}) *GenAIUsageInfo {usage := &GenAIUsageInfo{}- if value, ok := Int64Attr(attrs, "gen_ai.usage.cache_creation.input_tokens", "gen_ai.usage.cache_creation_input_tokens", "cache_creation_tokens"); ok {+ var cached int64+ if value, ok := Int64Attr(attrs, "gen_ai.usage.cache_creation.input_tokens", "gen_ai.usage.cache_write.input_tokens", "gen_ai.usage.cache_creation_input_tokens", "cache_creation_tokens"); ok {usage.CacheCreation = &GenAIUsageCacheCreationInfo{InputTokens: &value}+ if value > 0 {+ cached += value+ }}if value, ok := Int64Attr(attrs, "gen_ai.usage.cache_read.input_tokens", "gen_ai.usage.cache_read_input_tokens", "cache_read_tokens"); ok {usage.CacheRead = &GenAIUsageCacheReadInfo{InputTokens: &value}+ if value > 0 {+ cached += value+ }}- if value, ok := Int64Attr(attrs, "gen_ai.usage.input_tokens", "llm.usage.prompt_tokens", "gen_ai.usage.prompt_tokens", "input_tokens"); ok {+ if value, ok := Int64Attr(attrs, inclusiveInputTokenKeys...); ok {+ // Guard: an input smaller than its own cache counts cannot contain+ // them, so that source already reports uncached input (Anthropic's+ // shape under semconv names) and is stored as reported rather than+ // having real tokens erased. The semconv names both cache reads and+ // cache writes as part of input, so both are removed; an emitter that+ // folds in only one of them undercounts input by the other.+ if cached > 0 && value >= cached {+ value -= cached+ }+ usage.InputTokens = &value+ } else if value, ok := Int64Attr(attrs, "input_tokens"); ok {usage.InputTokens = &value}if value, ok := Int64Attr(attrs, "gen_ai.usage.output_tokens", "llm.usage.completion_tokens", "gen_ai.usage.completion_tokens", "output_tokens"); ok {diff --git a/collector-builder/exporter/beaconjsonexporter/internal/beaconevent/converter_test.go b/collector-builder/exporter/beaconjsonexporter/internal/beaconevent/converter_test.goindex deaa6ddf..3f45ade0 100644--- a/collector-builder/exporter/beaconjsonexporter/internal/beaconevent/converter_test.go+++ b/collector-builder/exporter/beaconjsonexporter/internal/beaconevent/converter_test.go@@ -664,6 +664,183 @@ func TestGenAIUsageFromAttrsNormalizesAliases(t *testing.T) {}}+// The OTel GenAI semconv input count, and the OpenAI-style prompt_tokens names+// it replaced, include cached prompt tokens ("This value SHOULD include all+// types of input tokens, including cached tokens"), while Beacon's gen_ai.usage+// keeps uncached input disjoint from cache reads and writes. Every inclusive+// spelling must therefore lose its cached subset, and Claude Code's bare,+// already-disjoint names must not.+func TestGenAIUsageFromAttrsReducesInclusiveInputToUncached(t *testing.T) {+ ptr := func(v int64) *int64 { return &v }+ tests := []struct {+ name string+ attrs map[string]interface{}+ wantInput *int64+ wantCacheRead *int64+ wantCacheCrt *int64+ }{+ {+ // OpenLLMetry's Anthropic instrumentation: input_tokens is+ // usage.input_tokens + cache_read + cache_creation.+ name: "semconv input with underscore cache aliases",+ attrs: map[string]interface{}{+ "gen_ai.usage.input_tokens": int64(19561),+ "gen_ai.usage.cache_read_input_tokens": int64(18500),+ "gen_ai.usage.cache_creation_input_tokens": int64(1024),+ "gen_ai.usage.output_tokens": int64(214),+ },+ wantInput: ptr(37), wantCacheRead: ptr(18500), wantCacheCrt: ptr(1024),+ },+ {+ name: "semconv input with dotted cache names",+ attrs: map[string]interface{}{+ "gen_ai.usage.input_tokens": int64(1000),+ "gen_ai.usage.cache_read.input_tokens": int64(600),+ "gen_ai.usage.cache_creation.input_tokens": int64(100),+ },+ wantInput: ptr(300), wantCacheRead: ptr(600), wantCacheCrt: ptr(100),+ },+ {+ // The GenAI semconv repository renamed cache creation to cache_write.+ name: "semconv cache_write spelling is cache creation",+ attrs: map[string]interface{}{+ "gen_ai.usage.input_tokens": int64(1000),+ "gen_ai.usage.cache_write.input_tokens": int64(250),+ },+ wantInput: ptr(750), wantCacheCrt: ptr(250),+ },+ {+ // OpenAI's prompt_tokens includes prompt_tokens_details.cached_tokens.+ name: "legacy gen_ai prompt_tokens with cache read only",+ attrs: map[string]interface{}{+ "gen_ai.usage.prompt_tokens": int64(2048),+ "gen_ai.usage.cache_read.input_tokens": int64(1920),+ },+ wantInput: ptr(128), wantCacheRead: ptr(1920),+ },+ {+ name: "legacy llm.usage prompt_tokens with cache read",+ attrs: map[string]interface{}{+ "llm.usage.prompt_tokens": int64(500),+ "gen_ai.usage.cache_read_input_tokens": int64(200),+ },+ wantInput: ptr(300), wantCacheRead: ptr(200),+ },+ {+ name: "input exactly equal to its cached subsets leaves zero uncached",+ attrs: map[string]interface{}{+ "gen_ai.usage.input_tokens": int64(700),+ "gen_ai.usage.cache_read.input_tokens": int64(600),+ "gen_ai.usage.cache_creation.input_tokens": int64(100),+ },+ wantInput: ptr(0), wantCacheRead: ptr(600), wantCacheCrt: ptr(100),+ },+ {+ // A source whose input is smaller than its own cache counts cannot+ // be inclusive; it already reports uncached input, so subtracting+ // would erase real tokens.+ name: "guard: input below cached subsets is already disjoint",+ attrs: map[string]interface{}{+ "gen_ai.usage.input_tokens": int64(37),+ "gen_ai.usage.cache_read.input_tokens": int64(18500),+ "gen_ai.usage.cache_creation.input_tokens": int64(1024),+ },+ wantInput: ptr(37), wantCacheRead: ptr(18500), wantCacheCrt: ptr(1024),+ },+ {+ name: "semconv input without cache counts is unchanged",+ attrs: map[string]interface{}{"gen_ai.usage.input_tokens": int64(12)},+ wantInput: ptr(12),+ },+ {+ name: "zero cache counts leave input unchanged",+ attrs: map[string]interface{}{+ "gen_ai.usage.input_tokens": int64(12),+ "gen_ai.usage.cache_read.input_tokens": int64(0),+ },+ wantInput: ptr(12), wantCacheRead: ptr(0),+ },+ {+ name: "cache counts without an input count stay as reported",+ attrs: map[string]interface{}{+ "gen_ai.usage.cache_read.input_tokens": int64(90),+ },+ wantCacheRead: ptr(90),+ },+ {+ // Claude Code's claude_code.llm_request span and api_request log use+ // Anthropic's disjoint usage under bare names.+ name: "claude code bare names are already disjoint",+ attrs: map[string]interface{}{+ "input_tokens": int64(1200),+ "cache_read_tokens": int64(800),+ "cache_creation_tokens": int64(256),+ },+ wantInput: ptr(1200), wantCacheRead: ptr(800), wantCacheCrt: ptr(256),+ },+ }+ for _, tt := range tests {+ t.Run(tt.name, func(t *testing.T) {+ usage := GenAIUsageFromAttrs(tt.attrs)+ if usage == nil {+ t.Fatalf("usage = nil, want populated usage")+ }+ assertInt64Ptr(t, "input_tokens", usage.InputTokens, tt.wantInput)+ var cacheRead, cacheCreation *int64+ if usage.CacheRead != nil {+ cacheRead = usage.CacheRead.InputTokens+ }+ if usage.CacheCreation != nil {+ cacheCreation = usage.CacheCreation.InputTokens+ }+ assertInt64Ptr(t, "cache_read", cacheRead, tt.wantCacheRead)+ assertInt64Ptr(t, "cache_creation", cacheCreation, tt.wantCacheCrt)+ })+ }+}++func assertInt64Ptr(t *testing.T, field string, got, want *int64) {+ t.Helper()+ switch {+ case got == nil && want == nil:+ case got == nil || want == nil:+ t.Fatalf("%s = %v, want %v", field, got, want)+ case *got != *want:+ t.Fatalf("%s = %d, want %d", field, *got, *want)+ }+}++// The semconv gen_ai.client.token.usage histogram has only input and output+// token types, so an inclusive input datapoint has no cached sibling to+// double-count against; it is stored as reported.+func TestEventsFromMetricsSemconvClientTokenUsageKeepsInputAsReported(t *testing.T) {+ metrics := pmetric.NewMetrics()+ resourceMetrics := metrics.ResourceMetrics().AppendEmpty()+ resourceMetrics.Resource().Attributes().PutStr("service.name", "agent-api")+ metric := resourceMetrics.ScopeMetrics().AppendEmpty().Metrics().AppendEmpty()+ metric.SetName("gen_ai.client.token.usage")+ histogram := metric.SetEmptyHistogram()+ for _, dp := range []struct {+ tokenType string+ sum float64+ }{{"input", 19561}, {"output", 214}} {+ point := histogram.DataPoints().AppendEmpty()+ point.SetCount(1)+ point.SetSum(dp.sum)+ point.Attributes().PutStr("gen_ai.token.type", dp.tokenType)+ point.Attributes().PutStr("gen_ai.request.model", "claude-sonnet-4-5")+ }++ events := NewConverter(Options{}).EventsFromMetrics(metrics)+ if len(events) != 2 {+ t.Fatalf("expected 2 events, got %d", len(events))+ }+ input := events[0].GenAI.Usage+ if input == nil || input.InputTokens == nil || *input.InputTokens != 19561 || input.CacheRead != nil || input.CacheCreation != nil {+ t.Fatalf("input datapoint usage = %#v, want input_tokens 19561 and no cache", input)+ }+}+func TestPopulateCommonMapsBeaconSessionAttributes(t *testing.T) {span, traces := newObserveSDKTraceSpan("agent.step")span.Attributes().PutStr("beacon.session.id", "cloud-session-42")diff --git a/docs/cli/token-usage.mdx b/docs/cli/token-usage.mdxindex bf2e42b2..a12fc4ae 100644--- a/docs/cli/token-usage.mdx+++ b/docs/cli/token-usage.mdx@@ -23,7 +23,7 @@ Beacon normalizes token telemetry from every runtime into one canonical [`gen_aiA few rules shape how the numbers read:-- **Counts are kept disjoint.** `input_tokens`, `output_tokens`, `cache_read`, and `cache_creation` do not overlap, so a total is `input + output + cache_read + cache_creation` without double-counting. Codex reports input inclusive of both cache reads and cache writes, so Beacon subtracts both before storing uncached input.+- **Counts are kept disjoint.** `input_tokens`, `output_tokens`, `cache_read`, and `cache_creation` do not overlap, so a total is `input + output + cache_read + cache_creation` without double-counting. Codex reports input inclusive of both cache reads and cache writes, so Beacon subtracts both before storing uncached input. The same applies to any source using the OpenTelemetry GenAI names (`gen_ai.usage.input_tokens`, or the older `prompt_tokens` spellings), whose input includes cached tokens by definition: Beacon subtracts the cache read and cache creation counts reported alongside it, unless the input is smaller than they are and so is already uncached. Claude Code's own counts are already disjoint and are stored as reported.- **Cost is runtime-reported only.** `cost_usd` reflects what the runtime emits. Beacon never derives cost from local pricing tables, so cost appears only for runtimes that report it.- **Attribution depends on the metadata a runtime attaches.** Model, harness, repository, and run grouping work whenever those fields are present. Per-session grouping (`--session`, the session step tree) requires a session or conversation id on the usage signal. Per-user grouping uses the event user directly or a matching session context event from a local hook.- **Duplicate channels are de-duplicated.** When a runtime emits the same usage on more than one signal (Claude Code reports it on both logs and metrics), Beacon counts it once.diff --git a/docs/telemetry-schema/normalization.mdx b/docs/telemetry-schema/normalization.mdxindex b8396915..02f8db02 100644--- a/docs/telemetry-schema/normalization.mdx+++ b/docs/telemetry-schema/normalization.mdx@@ -178,13 +178,15 @@ Beacon preserves OpenTelemetry GenAI semantic convention fields under the nested| Source signal | Normalized field ||---------------|------------------|-| `gen_ai.usage.input_tokens`, `llm.usage.prompt_tokens`, `gen_ai.usage.prompt_tokens` | `gen_ai.usage.input_tokens` |+| `gen_ai.usage.input_tokens`, `llm.usage.prompt_tokens`, `gen_ai.usage.prompt_tokens` | `gen_ai.usage.input_tokens`, less the cache read and cache creation counts reported beside it (see below) || `gen_ai.usage.output_tokens`, `llm.usage.completion_tokens`, `gen_ai.usage.completion_tokens` | `gen_ai.usage.output_tokens` |-| `gen_ai.usage.cache_creation.input_tokens` | `gen_ai.usage.cache_creation.input_tokens` |+| `gen_ai.usage.cache_creation.input_tokens`, `gen_ai.usage.cache_write.input_tokens` | `gen_ai.usage.cache_creation.input_tokens` || `gen_ai.usage.cache_read.input_tokens` | `gen_ai.usage.cache_read.input_tokens` || `gen_ai.usage.reasoning.output_tokens` | `gen_ai.usage.reasoning.output_tokens` || `gen_ai.usage.cost`, runtime `*.cost.usage` metric datapoints | `gen_ai.usage.cost_usd` |+Beacon's `gen_ai.usage.input_tokens` is uncached input only, disjoint from cache reads and cache creation. The OTel GenAI semantic conventions define the source attribute the other way: `gen_ai.usage.input_tokens` "SHOULD include all types of input tokens, including cached tokens", and each cache count "SHOULD be included in `gen_ai.usage.input_tokens`". OpenAI-style `prompt_tokens` likewise counts cached prompt tokens. So for those three names Beacon stores the input count minus the cache read and cache creation counts on the same span or log record. If the input is smaller than those cache counts it cannot contain them, the source is already reporting uncached input, and it is stored unchanged. Claude Code's bare `input_tokens` attribute is Anthropic's uncached count and is never reduced, and `raw` keeps the value as reported. The semconv `gen_ai.client.token.usage` metric has only `input` and `output` token types, so its input datapoint has no cache count to reduce and is stored as reported.+Usage also arrives as metric datapoints, not just span and log attributes. Claude Code's `claude_code.token.usage` and `claude_code.cost.usage` are expanded per datapoint. Codex's live source is its completed turn span, whose input, cache-read, cache-write, output, and reasoning attributes map into the same `gen_ai.usage` fields with `thread.id` promoted to `session.id`. Legacy `codex.turn.token_usage` metric rows remain supported for older logs. `gen_ai.usage.cost_usd` carries runtime-reported cost only and is never derived from a local pricing table. See [How token attribution works](/cli/token-usage#how-token-attribution-works) for how these normalized fields aggregate into local reports.## Prompt and content handling