Skip to content

The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu

fix(tokens): count a session read both live and by poll backfill once

Claudec28b2a92a381
diff --git a/CLAUDE.md b/CLAUDE.md
index 20cbcfeb..13659ea5 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -536,7 +536,7 @@ unpacked through Chrome's developer mode.
- For macOS-only behavior, gate tests with `runtime.GOOS == "darwin"` or assert the non-Darwin contract explicitly.
- Keep endpoint event schema fields stable: `vendor`, `product`, `schema_version`, required event fields, and Wazuh-compatible JSONL output are release contracts.
- Preserve optional event fields for agent-native metadata (`session`, `trace`, `tool`, `file`, `command`, `mcp`, `approval`, `content`, `model`, `repository`, and `branch`) without changing existing required field semantics.
-- Reading a runtime's own on-disk session store is a `poll` collection path, not a new method: it is a pull that sees committed records after the fact, which is what `poll` means. fx is the case that exercises it (`cli/beacon/internal/fxsession`); a future runtime with the same shape uses `poll` too rather than adding a fifth value to a release-contract field.
+- Reading a runtime's own on-disk session store is a `poll` collection path, not a new method: it is a pull that sees committed records after the fact, which is what `poll` means. fx is the case that exercises it (`cli/beacon/internal/fxsession`); a future runtime with the same shape uses `poll` too rather than adding a fifth value to a release-contract field. The token aggregator relies on that marker: when one session (endpoint, normalized harness, session id) has usage from both a live method and `poll`, `preferLiveOverPoll` in `cli/beacon/internal/tokens/tokens.go` keeps live usage authoritative from the first event of that live channel in the session and drops the backfill's copies from then on, so a poll mapper must keep the runtime's own session id and real record timestamps rather than deduplicating against the live log itself.
- Set the provenance markers on every new collection path. `harness.collection_method` (`hook`, `otlp`, `plugin`, `poll`) records the mechanism; `event.fidelity` (`observed`, `inferred`) records whether the source named the action or Beacon derived it. Both are defined in `pkg/asymptoteobserve/provenance.go` and are the only sanctioned way to express that difference — do not add per-runtime confidence fields, and do not leave a derived action unmarked. A new plugin-shaped runtime must be added to `CollectionMethodForPlatform`, whose default is `hook`. An action synthesized from an adjacent observation, such as an approval built from a pre-tool notification on a runtime with no approval hook, is `inferred`.
- Keep event identity derivable rather than asserted. `gen_ai.tool.call.id` carries the runtime's own name for one tool invocation, promoted on every capture path from the shared alias list in `pkg/asymptoteobserve.ToolCallIDKeys`; a new runtime adds its spelling there rather than in a mapper. `event.id` is a deterministic UUID derived by `EventIDForLine` from the event about to be written, so both capture paths name one action the same way. Its namespace constant and derivation are a contract: changing either renames every event ID Beacon has written, and a test pins the value.
- Keep `gen_ai.usage` as the single canonical token-usage representation: normalize all runtime token telemetry (span attributes, metric datapoints, legacy `llm.usage.*` aliases) into it, mirror OTel GenAI semconv JSON names exactly, and never add parallel or per-harness token fields. `gen_ai.usage.cost_usd` carries runtime-reported cost only; do not derive cost from local pricing tables.
diff --git a/cli/beacon/internal/tokens/coverage.go b/cli/beacon/internal/tokens/coverage.go
index f64475cb..988a168b 100644
--- a/cli/beacon/internal/tokens/coverage.go
+++ b/cli/beacon/internal/tokens/coverage.go
@@ -234,10 +234,7 @@ func Coverage(events []schema.Event, installed []string) CoverageReport {
// covered on exactly the events that would contribute to its totals -- including the
// dedupe and cumulative-delta handling. Counting raw gen_ai.usage blocks here instead
// would report a runtime as covered whose usage the report then discards.
- usageEvents := collectUsageEvents(events, sessionUserContexts(events))
- usageEvents = preferCodexTurnSpans(usageEvents)
- usageEvents = dedupeOverlappingChannels(usageEvents)
- resolveCumulativeSeries(usageEvents)
+ usageEvents, _ := resolveUsageEvents(events, sessionUserContexts(events), liveCaptureStarts(events))
for _, event := range events {
name := asymptoteobserve.NormalizeHarnessName(strings.TrimSpace(event.Harness.Name))
diff --git a/cli/beacon/internal/tokens/tokens.go b/cli/beacon/internal/tokens/tokens.go
index b995d6fd..df5ba072 100644
--- a/cli/beacon/internal/tokens/tokens.go
+++ b/cli/beacon/internal/tokens/tokens.go
@@ -111,6 +111,11 @@ type Report struct {
SessionDetail *SessionDetail `json:"session_detail,omitempty"`
EventsWithUsage int `json:"events_with_usage"`
TotalEvents int `json:"total_events"`
+ // SuppressedPollEvents counts poll-backfill usage events (collection_method=poll) dropped
+ // because a live channel had already reported the same session's usage from that point on.
+ // They are left out of every total and of EventsWithUsage; the count is here so a reader can
+ // see the overlap was found rather than wonder where a backfill's rows went.
+ SuppressedPollEvents int `json:"suppressed_poll_events,omitempty"`
}
// usageEvent is one usage-bearing event with its usage normalized to a delta
@@ -143,6 +148,10 @@ type usageEvent struct {
sourceStart time.Time
seriesField string
seriesValue float64
+
+ // collectionMethod is harness.collection_method, lowercased: whether this usage came from
+ // live capture or from a poll of the runtime's own session store.
+ collectionMethod string
}
// Aggregate builds a token usage report from endpoint events. Events from
@@ -157,18 +166,19 @@ type usageEvent struct {
// slice order to recover the emission sequence. Passing events newest-first
// makes each cumulative step-down look like a counter reset and inflates totals.
func Aggregate(events []schema.Event, opts Options) Report {
- return aggregate(events, opts, sessionUserContexts(events))
+ return aggregate(events, opts, events)
}
-func aggregate(events []schema.Event, opts Options, sessionUsers sessionUserIndex) Report {
+// aggregate reports on events. contextEvents is the unfiltered set the session-level context is
+// read from -- who the session's user is, and when each live channel started capturing it -- so a
+// report narrowed by time or model still resolves both the way the whole log would.
+func aggregate(events []schema.Event, opts Options, contextEvents []schema.Event) Report {
if opts.NearLimitRatio <= 0 {
opts.NearLimitRatio = defaultNearLimitRatio
}
report := Report{TotalEvents: len(events)}
- usageEvents := collectUsageEvents(events, sessionUsers)
- usageEvents = preferCodexTurnSpans(usageEvents)
- usageEvents = dedupeOverlappingChannels(usageEvents)
- resolveCumulativeSeries(usageEvents)
+ usageEvents, suppressed := resolveUsageEvents(events, sessionUserContexts(contextEvents), liveCaptureStarts(contextEvents))
+ report.SuppressedPollEvents = suppressed
byModel := map[string]*Usage{}
bySession := map[string]*Usage{}
@@ -236,9 +246,8 @@ func AggregateScoped(events []schema.Event, runID string, opts Options) Report {
func AggregateScopedWithContexts(events, contextEvents []schema.Event, runID string, opts Options) Report {
session := strings.TrimSpace(opts.SessionID)
runID = strings.TrimSpace(runID)
- sessionUsers := sessionUserContexts(contextEvents)
if session == "" && runID == "" {
- return aggregate(events, opts, sessionUsers)
+ return aggregate(events, opts, contextEvents)
}
filtered := make([]schema.Event, 0, len(events))
for _, event := range events {
@@ -253,7 +262,24 @@ func AggregateScopedWithContexts(events, contextEvents []schema.Event, runID str
}
filtered = append(filtered, event)
}
- return aggregate(filtered, opts, sessionUsers)
+ return aggregate(filtered, opts, contextEvents)
+}
+
+// resolveUsageEvents is the one pipeline from raw events to the usage contributions every report
+// sums, so the token report and the coverage report cannot disagree about what was counted. It
+// returns the contributions and the number of poll events dropped as copies of live capture.
+//
+// Order matters. The Codex span preference and the live/poll preference each drop whole events
+// before the log/metric dedupe, which reads what the surviving non-metric events report; a poll
+// copy still present at that point would claim fields for the metric's whole session. All three
+// run before cumulative resolution so a dropped event never seeds or breaks a counter series.
+func resolveUsageEvents(events []schema.Event, sessionUsers sessionUserIndex, liveStarts liveCaptureIndex) ([]*usageEvent, int) {
+ usageEvents := collectUsageEvents(events, sessionUsers)
+ usageEvents = preferCodexTurnSpans(usageEvents)
+ usageEvents, suppressed := preferLiveOverPoll(usageEvents, liveStarts)
+ usageEvents = dedupeOverlappingChannels(usageEvents)
+ resolveCumulativeSeries(usageEvents)
+ return usageEvents, suppressed
}
type sessionContextKey struct {
@@ -440,6 +466,7 @@ func collectUsageEvents(events []schema.Event, sessionUsers sessionUserIndex) []
repository: event.Repository,
usage: Usage{Events: 1},
}
+ ue.collectionMethod = strings.ToLower(strings.TrimSpace(event.Harness.CollectionMethod))
if ts, err := schema.ParseTimestamp(event.Timestamp); err == nil {
ue.ts = ts
}
@@ -579,6 +606,164 @@ func preferCodexTurnSpans(events []*usageEvent) []*usageEvent {
return out
}
+// liveScope is one session on one endpoint, the unit within which a live report and a poll
+// report of the same usage can be recognized as the same. The harness is normalized so the
+// spellings a runtime's OTLP resource, hook adapter and session store use land on one key.
+type liveScope struct {
+ endpoint string
+ harness string
+ session string
+}
+
+// liveScopeFor returns the scope for an event, or false when the event names no session. Two
+// session-less reports are not evidence of one turn -- they could be any two sessions on the
+// machine -- so they never take part in the live/poll preference.
+func liveScopeFor(endpoint, harness, session string) (liveScope, bool) {
+ session = strings.ToLower(strings.TrimSpace(session))
+ if session == "" {
+ return liveScope{}, false
+ }
+ return liveScope{
+ endpoint: strings.ToLower(strings.TrimSpace(endpoint)),
+ harness: asymptoteobserve.NormalizeHarnessName(harness),
+ session: session,
+ }, true
+}
+
+// isLiveCollectionMethod reports whether a collection method observed the runtime as it worked.
+// An unset method is not live: it is a log written before provenance existed, or an event Beacon
+// emitted about itself, and neither is evidence that a live channel covered the session.
+func isLiveCollectionMethod(method string) bool {
+ switch method {
+ case schema.CollectionMethodHook, schema.CollectionMethodOTLP, schema.CollectionMethodPlugin:
+ return true
+ }
+ return false
+}
+
+// liveCaptureIndex records, per session scope and live collection method, the earliest timestamp
+// any event arrived over that method -- whether or not the event carried usage.
+type liveCaptureIndex map[liveScope]map[string]time.Time
+
+func liveCaptureStarts(events []schema.Event) liveCaptureIndex {
+ index := liveCaptureIndex{}
+ for _, event := range events {
+ method := strings.ToLower(strings.TrimSpace(event.Harness.CollectionMethod))
+ if !isLiveCollectionMethod(method) || event.Session == nil {
+ continue
+ }
+ scope, ok := liveScopeFor(event.Endpoint.Hostname, event.Harness.Name, event.Session.ID)
+ if !ok {
+ continue
+ }
+ ts, err := schema.ParseTimestamp(event.Timestamp)
+ if err != nil || ts.IsZero() {
+ continue
+ }
+ if index[scope] == nil {
+ index[scope] = map[string]time.Time{}
+ }
+ if current, seen := index[scope][method]; !seen || ts.Before(current) {
+ index[scope][method] = ts
+ }
+ }
+ return index
+}
+
+// preferLiveOverPoll makes live capture authoritative over a poll backfill of the same session.
+//
+// Several runtimes are read twice: live, over OTLP or a hook or a Beacon-managed plugin, and
+// after the fact by `beacon endpoint <runtime> sync`, which reads the session store the runtime
+// commits to disk (collection_method=poll). Claude Code's api_request log and its transcript,
+// Codex's turn span and its rollout file, and the Cline, OpenCode, OpenClaw and Pi plugins and
+// their session stores each report one request's usage twice under one session id. Nothing at
+// write time collapses the two -- a usage event carries no tool-call id, so each path's event.id
+// is a digest of its own line -- so without this step every such session counts double.
+//
+// Per session scope where live events report usage, the cutover is the moment the live channel
+// that reported it started capturing the session: the earliest event of any kind over that
+// collection method (the unfiltered liveStarts), or the earliest live usage event or Codex turn
+// start if that is sooner. It is deliberately not the first live usage timestamp. A live usage
+// record is written when its request completes, while the session store stamps the same request
+// when it began or as it streamed, so a cutover on the live record would leave every session's
+// first request on both sides of it. The earliest same-channel event -- Claude's user_prompt log,
+// a plugin's session start -- precedes the first request instead.
+//
+// Poll usage at or after the cutover gives up the fields the live channel reports in that scope
+// and is dropped once nothing is left, the same field-wise rule the log/metric dedupe uses: a
+// live channel that reports only cost (Claude Code's cost.usage metric without its token.usage
+// sibling) leaves the backfill's tokens standing, and one that reports tokens without cost leaves
+// the backfill's cost, rather than either being lost. Poll usage before the
+// cutover is the only record of turns from before Beacon's live capture started and is kept, as
+// is any poll event with no usable timestamp, which cannot be placed against the cutover.
+//
+// Context-only live events do not count: occupancy is not spend, so a live channel that reports
+// only how full the window was is no authority on what the session cost.
+func preferLiveOverPoll(events []*usageEvent, liveStarts liveCaptureIndex) ([]*usageEvent, int) {
+ type liveCoverage struct {
+ fields usageFieldSet
+ methods map[string]bool
+ cutover time.Time
+ }
+ earlier := func(current, candidate time.Time) time.Time {
+ if candidate.IsZero() || (!current.IsZero() && !candidate.Before(current)) {
+ return current
+ }
+ return candidate
+ }
+ coverage := map[liveScope]*liveCoverage{}
+ hasPoll := false
+ for _, ue := range events {
+ if ue.contextOnly {
+ continue
+ }
+ if ue.collectionMethod == schema.CollectionMethodPoll {
+ hasPoll = true
+ continue
+ }
+ if !isLiveCollectionMethod(ue.collectionMethod) {
+ continue
+ }
+ scope, ok := liveScopeFor(ue.endpoint, ue.harness, ue.session)
+ if !ok {
+ continue
+ }
+ c := coverage[scope]
+ if c == nil {
+ c = &liveCoverage{methods: map[string]bool{}}
+ coverage[scope] = c
+ }
+ c.fields.mark(ue.usage)
+ c.methods[ue.collectionMethod] = true
+ c.cutover = earlier(c.cutover, ue.ts)
+ c.cutover = earlier(c.cutover, ue.sourceStart)
+ }
+ if !hasPoll || len(coverage) == 0 {
+ return events, 0
+ }
+ for scope, c := range coverage {
+ for method := range c.methods {
+ c.cutover = earlier(c.cutover, liveStarts[scope][method])
+ }
+ }
+ suppressed := 0
+ out := events[:0]
+ for _, ue := range events {
+ if ue.collectionMethod == schema.CollectionMethodPoll && !ue.contextOnly && !ue.ts.IsZero() {
+ if scope, ok := liveScopeFor(ue.endpoint, ue.harness, ue.session); ok {
+ if c := coverage[scope]; c != nil && c.fields.any() && !c.cutover.IsZero() && !ue.ts.Before(c.cutover) {
+ if !c.fields.clear(&ue.usage) {
+ suppressed++
+ continue
+ }
+ }
+ }
+ }
+ out = append(out, ue)
+ }
+ return out, suppressed
+}
+
// dedupeOverlappingChannels removes double-counted usage when a runtime reports
// the same tokens through two OTel channels. Claude Code emits each request's
// usage on both a claude_code.api_request log record and the
@@ -591,27 +776,24 @@ func preferCodexTurnSpans(events []*usageEvent) []*usageEvent {
// (Claude Code reports cost only on claude_code.cost.usage) survive on the
// metric channel, so cost still lands exactly once. Runtimes that emit only
// metrics have no log/span channel in scope and are left untouched.
+//
+// Poll events are not the log channel. A poll backfill has no metric_name either, but it is a
+// copy of the session's history, not a third OTel channel, and preferLiveOverPoll has already
+// removed the part of it a live channel covers. What survives is from before live capture began,
+// so letting it claim fields here would zero the metric's tokens for the rest of the session.
func dedupeOverlappingChannels(events []*usageEvent) []*usageEvent {
- type fieldSet struct {
- input, output, cacheRead, cacheCreation, reasoning, cost bool
- }
scopeKey := func(ue *usageEvent) string { return ue.harness + "\x00" + ue.session }
- logFields := map[string]*fieldSet{}
+ logFields := map[string]*usageFieldSet{}
for _, ue := range events {
- if ue.metricName != "" {
+ if ue.metricName != "" || ue.collectionMethod == schema.CollectionMethodPoll {
continue
}
fs := logFields[scopeKey(ue)]
if fs == nil {
- fs = &fieldSet{}
+ fs = &usageFieldSet{}
logFields[scopeKey(ue)] = fs
}
- fs.input = fs.input || ue.usage.InputTokens != 0
- fs.output = fs.output || ue.usage.OutputTokens != 0
- fs.cacheRead = fs.cacheRead || ue.usage.CacheReadInputTokens != 0
- fs.cacheCreation = fs.cacheCreation || ue.usage.CacheCreationInputTokens != 0
- fs.reasoning = fs.reasoning || ue.usage.ReasoningOutputTokens != 0
- fs.cost = fs.cost || ue.usage.CostUSD != 0
+ fs.mark(ue.usage)
}
if len(logFields) == 0 {
return events
@@ -620,27 +802,9 @@ func dedupeOverlappingChannels(events []*usageEvent) []*usageEvent {
for _, ue := range events {
if ue.metricName != "" {
if fs := logFields[scopeKey(ue)]; fs != nil {
- if fs.input {
- ue.usage.InputTokens = 0
- }
- if fs.output {
- ue.usage.OutputTokens = 0
- }
- if fs.cacheRead {
- ue.usage.CacheReadInputTokens = 0
- }
- if fs.cacheCreation {
- ue.usage.CacheCreationInputTokens = 0
- }
- if fs.reasoning {
- ue.usage.ReasoningOutputTokens = 0
- }
- if fs.cost {
- ue.usage.CostUSD = 0
- }
// Drop a metric event left with no usage so it neither inflates
// event counts nor seeds an empty cumulative series.
- if ue.usage.TotalTokens() == 0 && ue.usage.ReasoningOutputTokens == 0 && ue.usage.CostUSD == 0 {
+ if !fs.clear(&ue.usage) {
continue
}
}
@@ -650,6 +814,48 @@ func dedupeOverlappingChannels(events []*usageEvent) []*usageEvent {
return out
}
+// usageFieldSet records which usage fields a channel reported anywhere in a scope, so a second
+// channel can give up exactly those fields and keep the ones only it reports.
+type usageFieldSet struct {
+ input, output, cacheRead, cacheCreation, reasoning, cost bool
+}
+
+func (fs *usageFieldSet) mark(u Usage) {
+ fs.input = fs.input || u.InputTokens != 0
+ fs.output = fs.output || u.OutputTokens != 0
+ fs.cacheRead = fs.cacheRead || u.CacheReadInputTokens != 0
+ fs.cacheCreation = fs.cacheCreation || u.CacheCreationInputTokens != 0
+ fs.reasoning = fs.reasoning || u.ReasoningOutputTokens != 0
+ fs.cost = fs.cost || u.CostUSD != 0
+}
+
+func (fs *usageFieldSet) any() bool {
+ return fs.input || fs.output || fs.cacheRead || fs.cacheCreation || fs.reasoning || fs.cost
+}
+
+// clear zeroes the fields in the set on u and reports whether u still carries any usage.
+func (fs *usageFieldSet) clear(u *Usage) bool {
+ if fs.input {
+ u.InputTokens = 0
+ }
+ if fs.output {
+ u.OutputTokens = 0
+ }
+ if fs.cacheRead {
+ u.CacheReadInputTokens = 0
+ }
+ if fs.cacheCreation {
+ u.CacheCreationInputTokens = 0
+ }
+ if fs.reasoning {
+ u.ReasoningOutputTokens = 0
+ }
+ if fs.cost {
+ u.CostUSD = 0
+ }
+ return u.TotalTokens() != 0 || u.ReasoningOutputTokens != 0 || u.CostUSD != 0
+}
+
// resolveCumulativeSeries rewrites cumulative metric contributions into
// per-interval deltas. Each cumulative series is identified by harness,
// session, model, metric name, and the single usage field the datapoint set.
diff --git a/cli/beacon/internal/tokens/tokens_test.go b/cli/beacon/internal/tokens/tokens_test.go
index 5012eaf9..65275330 100644
--- a/cli/beacon/internal/tokens/tokens_test.go
+++ b/cli/beacon/internal/tokens/tokens_test.go
@@ -707,3 +707,394 @@ func TestAggregateDoesNotMergeModelsThatOnlyLookAlike(t *testing.T) {
t.Fatalf("by-model rows = %d, want 2 -- these are different ids and merging them needs a catalog", len(report.ByModel))
}
}
+
+// TestAggregatePrefersLiveCaptureOverPollBackfill covers runtimes Beacon reads twice: once live
+// (an OTLP export, a hook, a Beacon-managed plugin) and once after the fact from the session store
+// the runtime commits to disk (`beacon endpoint <runtime> sync`, collection_method=poll). Both
+// paths report the same requests' usage under the same session id, and nothing at write time
+// collapses them: usage events carry no tool-call id, so each path's event.id is a digest of its
+// own line. Live is authoritative from the moment its channel was capturing the session; poll
+// usage before that is the only record of those turns and stays.
+func TestAggregatePrefersLiveCaptureOverPollBackfill(t *testing.T) {
+ type tokens struct {
+ input, output, cacheRead, cacheCreation, reasoning int64
+ cost float64
+ }
+ event := func(ts, harness, method, session, action string, usage *tokens, mutate func(*schema.Event)) schema.Event {
+ e := schema.Event{
+ Timestamp: ts,
+ Endpoint: schema.EndpointInfo{Hostname: "host-a", OS: "darwin"},
+ Event: schema.EventInfo{Kind: "agent_runtime", Action: action},
+ Harness: schema.HarnessInfo{Name: harness, CollectionMethod: method},
+ }
+ if session != "" {
+ e.Session = &schema.SessionInfo{ID: session}
+ }
+ if usage != nil {
+ info := &schema.GenAIUsageInfo{}
+ if usage.input != 0 {
+ info.InputTokens = int64Ptr(usage.input)
+ }
+ if usage.output != 0 {
+ info.OutputTokens = int64Ptr(usage.output)
+ }
+ if usage.cacheRead != 0 {
+ info.CacheRead = &schema.GenAIUsageCacheReadInfo{InputTokens: int64Ptr(usage.cacheRead)}
+ }
+ if usage.cacheCreation != 0 {
+ info.CacheCreation = &schema.GenAIUsageCacheCreationInfo{InputTokens: int64Ptr(usage.cacheCreation)}
+ }
+ if usage.reasoning != 0 {
+ info.Reasoning = &schema.GenAIUsageReasoningInfo{OutputTokens: int64Ptr(usage.reasoning)}
+ }
+ if usage.cost != 0 {
+ info.CostUSD = float64Ptr(usage.cost)
+ }
+ e.GenAI = &schema.GenAIInfo{Usage: info}
+ }
+ if mutate != nil {
+ mutate(&e)
+ }
+ return e
+ }
+ model := func(name string) func(*schema.Event) { return func(e *schema.Event) { e.Model = name } }
+ metric := func(name string) func(*schema.Event) {
+ return func(e *schema.Event) {
+ e.Model = "claude-opus-4-6"
+ e.Raw = map[string]interface{}{"metric_name": name, "metric_temporality": "Delta"}
+ }
+ }
+ const (
+ otlp = schema.CollectionMethodOTLP
+ plugin = schema.CollectionMethodPlugin
+ hook = schema.CollectionMethodHook
+ poll = schema.CollectionMethodPoll
+ )
+
+ cases := []struct {
+ name string
+ events []schema.Event
+ want tokens
+ wantEvents int
+ suppressed int
+ }{
+ {
+ // Claude Code's OTLP api_request log is written when a request completes; the
+ // transcript line `claude sync` reads for the same request is written as the
+ // response streams, so it lands a second or two earlier. The cutover is the session's
+ // first OTLP record (the user_prompt log), not its first usage record, or the first
+ // request of every session would be counted twice.
+ name: "claude otlp api_request log and claude sync",
+ events: []schema.Event{
+ event("2026-06-11T10:00:00.100Z", "claude_code", otlp, "c1", "prompt.submitted", nil, nil),
+ event("2026-06-11T10:00:03.900Z", "claude_code", poll, "c1", "agent.response", &tokens{input: 6, output: 410, cacheRead: 21000, cacheCreation: 3100}, model("claude-opus-4-6")),
+ event("2026-06-11T10:00:05.200Z", "claude_code", otlp, "c1", "tool.invoked", &tokens{input: 6, output: 410, cacheRead: 21000, cacheCreation: 3100, cost: 0.0421}, model("claude-opus-4-6")),
+ event("2026-06-11T10:00:41.000Z", "claude_code", poll, "c1", "agent.response", &tokens{input: 3, output: 95, cacheRead: 24100}, model("claude-opus-4-6")),
+ event("2026-06-11T10:00:42.500Z", "claude_code", otlp, "c1", "tool.invoked", &tokens{input: 3, output: 95, cacheRead: 24100, cost: 0.0153}, model("claude-opus-4-6")),
+ },
+ want: tokens{input: 9, output: 505, cacheRead: 45100, cacheCreation: 3100, cost: 0.0574},
+ wantEvents: 2,
+ suppressed: 2,
+ },
+ {
+ // Three channels for one session: the api_request log, the token.usage/cost.usage
+ // metrics, and the transcript. The log wins over the metric for tokens (the existing
+ // channel dedupe), cost survives once on the metric, and the poll copy goes.
+ name: "claude log, metric and claude sync counted once",
+ events: []schema.Event{
+ event("2026-06-11T10:00:00Z", "claude_code", otlp, "c2", "prompt.submitted", nil, nil),
+ event("2026-06-11T10:00:04Z", "claude_code", poll, "c2", "agent.response", &tokens{input: 11, output: 826, cacheRead: 119393, cacheCreation: 15751}, model("claude-opus-4-6")),
+ event("2026-06-11T10:00:05Z", "claude_code", otlp, "c2", "tool.invoked", &tokens{input: 11, output: 826, cacheRead: 119393, cacheCreation: 15751}, model("claude-opus-4-6")),
+ event("2026-06-11T10:00:06Z", "claude_code", otlp, "c2", "token.usage", &tokens{input: 11}, metric("claude_code.token.usage")),
+ event("2026-06-11T10:00:06Z", "claude_code", otlp, "c2", "token.usage", &tokens{output: 826}, metric("claude_code.token.usage")),
+ event("2026-06-11T10:00:06Z", "claude_code", otlp, "c2", "token.usage", &tokens{cacheRead: 119393}, metric("claude_code.token.usage")),
+ event("2026-06-11T10:00:06Z", "claude_code", otlp, "c2", "token.usage", &tokens{cacheCreation: 15751}, metric("claude_code.token.usage")),
+ event("2026-06-11T10:00:06Z", "claude_code", otlp, "c2", "cost.usage", &tokens{cost: 0.1788}, metric("claude_code.cost.usage")),
+ },
+ want: tokens{input: 11, output: 826, cacheRead: 119393, cacheCreation: 15751, cost: 0.1788},
+ wantEvents: 2,
+ suppressed: 1,
+ },
+ {
+ // Metrics exporter only, no logs: the metric is the live channel. The transcript's
+ // pre-cutover turn must not zero the metric's tokens for the whole session, which is
+ // what the log/metric dedupe would do if it read a poll event as the log channel.
+ name: "claude metric only and claude sync",
+ events: []schema.Event{
+ event("2026-06-11T09:30:00Z", "claude_code", poll, "c3", "agent.response", &tokens{input: 40, output: 7}, model("claude-opus-4-6")),
+ event("2026-06-11T10:00:04Z", "claude_code", poll, "c3", "agent.response", &tokens{input: 11, output: 826}, model("claude-opus-4-6")),
+ event("2026-06-11T10:00:00Z", "claude_code", otlp, "c3", "token.usage", &tokens{input: 11}, metric("claude_code.token.usage")),
+ event("2026-06-11T10:00:00Z", "claude_code", otlp, "c3", "token.usage", &tokens{output: 826}, metric("claude_code.token.usage")),
+ },
+ want: tokens{input: 51, output: 833},
+ wantEvents: 3,
+ suppressed: 1,
+ },
+ {
+ // Codex's turn span carries its own start, which precedes the span's timestamp by the
+ // whole turn; `codex sync` stamps token_count at each request inside the turn. The
+ // turn from before Beacon's collector was running survives from the backfill alone.
+ name: "codex otlp turn span and codex sync",
+ events: []schema.Event{
+ event("2026-06-11T09:50:00Z", "codex_cli", poll, "019a-thread", "token.usage", &tokens{input: 3000, cacheRead: 1200, output: 80}, model("gpt-5.6-sol")),
+ event("2026-06-11T10:00:00Z", "codex_cli", hook, "019a-thread", "session.started", nil, nil),
+ event("2026-06-11T10:00:20Z", "codex_cli", poll, "019a-thread", "token.usage", &tokens{input: 13347, cacheCreation: 13344, output: 512, reasoning: 64}, model("gpt-5.6-sol")),
+ event("2026-06-11T10:00:30Z", "codex_cli", otlp, "019a-thread", "token.usage", &tokens{input: 13347, cacheCreation: 13344, output: 512, reasoning: 64}, func(e *schema.Event) {
+ e.Model = "gpt-5.6-sol"
+ e.Raw = map[string]interface{}{"source": "codex_turn_span", "turn_id": "turn-2", "turn_start_timestamp": "2026-06-11T10:00:01Z"}
+ }),
+ },
+ want: tokens{input: 16347, cacheRead: 1200, cacheCreation: 13344, output: 592, reasoning: 64},
+ wantEvents: 2,
+ suppressed: 1,
+ },
+ {
+ // Codex's legacy turn metric has no session id, so it cannot claim a session and
+ // leaves the backfill alone; the turn span does claim one, and the span preference
+ // still drops the overlapping metric.
+ name: "codex legacy metric, turn span and codex sync",
+ events: []schema.Event{
+ event("2026-06-11T09:00:00Z", "codex_cli", otlp, "", "token.usage", &tokens{input: 10}, func(e *schema.Event) {
+ e.Model = "gpt-5.6-sol"
+ e.Raw = map[string]interface{}{"metric_name": "codex.turn.token_usage", "metric_temporality": "Delta"}
+ }),
+ event("2026-06-11T09:59:59Z", "codex_cli", otlp, "", "token.usage", &tokens{input: 20}, func(e *schema.Event) {
+ e.Model = "gpt-5.6-sol"
+ e.Raw = map[string]interface{}{"metric_name": "codex.turn.token_usage", "metric_temporality": "Delta"}
+ }),
+ event("2026-06-11T09:59:58Z", "codex_cli", poll, "t2", "token.usage", &tokens{input: 20}, model("gpt-5.6-sol")),
+ event("2026-06-11T10:00:00Z", "codex_cli", otlp, "t2", "token.usage", &tokens{input: 20}, func(e *schema.Event) {
+ e.Model = "gpt-5.6-sol"
+ e.Raw = map[string]interface{}{"source": "codex_turn_span", "turn_start_timestamp": "2026-06-11T09:59:55Z"}
+ }),
+ },
+ want: tokens{input: 30},
+ wantEvents: 2,
+ suppressed: 1,
+ },
+ {
+ // Cline's plugin reports usage once, on the run result at task end; `cline sync`
+ // reports it per API request from the task's ui_messages. The task-end total covers
+ // every request after the plugin's first event, so all of those poll rows go.
+ name: "cline plugin task end and cline sync",
+ events: []schema.Event{
+ event("2026-06-11T10:00:00Z", "cline", plugin, "1749636000000", "session.started", nil, nil),
+ event("2026-06-11T10:00:10Z", "cline", poll, "1749636000000", "agent.response", &tokens{input: 1200, output: 150, cacheRead: 9000, cost: 0.011}, model("claude-sonnet-4-5")),
+ event("2026-06-11T10:00:40Z", "cline", poll, "1749636000000", "agent.response", &tokens{input: 800, output: 90, cacheRead: 10200, cost: 0.009}, model("claude-sonnet-4-5")),
+ event("2026-06-11T10:01:00Z", "cline", plugin, "1749636000000", "session.ended", &tokens{input: 2000, output: 240, cacheRead: 19200, cost: 0.02}, nil),
+ },
+ want: tokens{input: 2000, output: 240, cacheRead: 19200, cost: 0.02},
+ wantEvents: 1,
+ suppressed: 2,
+ },
+ {
+ // OpenCode's plugin writes on message.updated, a few milliseconds after the store
+ // stamps the message's time.completed that `opencode sync` reads.
+ name: "opencode plugin and opencode sync",
+ events: []schema.Event{
+ event("2026-06-11T10:00:00Z", "opencode", plugin, "ses_7a1f", "session.started", nil, nil),
+ event("2026-06-11T10:00:10.000Z", "opencode", poll, "ses_7a1f", "token.usage", &tokens{input: 9800, output: 300, reasoning: 120, cacheRead: 4000, cost: 0.0312}, model("claude-sonnet-4-5")),
+ event("2026-06-11T10:00:10.004Z", "opencode", plugin, "ses_7a1f", "agent.response", &tokens{input: 9800, output: 300, reasoning: 120, cacheRead: 4000, cost: 0.0312}, model("claude-sonnet-4-5")),
+ },
+ want: tokens{input: 9800, output: 300, reasoning: 120, cacheRead: 4000, cost: 0.0312},
+ wantEvents: 1,
+ suppressed: 1,
+ },
+ {
+ name: "openclaw plugin llm_output and session mapper",
+ events: []schema.Event{
+ event("2026-06-11T10:00:00Z", "openclaw_gateway", plugin, "oc-sess-1", "prompt.submitted", nil, nil),
+ event("2026-06-11T10:00:07Z", "openclaw_gateway", poll, "oc-sess-1", "agent.response", &tokens{input: 5200, output: 410, cacheRead: 1100}, model("gpt-5.4")),
+ event("2026-06-11T10:00:08Z", "openclaw_gateway", plugin, "oc-sess-1", "agent.response", &tokens{input: 5200, output: 410, cacheRead: 1100}, model("gpt-5.4")),
+ },
+ want: tokens{input: 5200, output: 410, cacheRead: 1100},
+ wantEvents: 1,
+ suppressed: 1,
+ },
+ {
+ // The Pi extension reports usage on message_end; the session file stamps the assistant
+ // message when it was created, before it streamed.
+ name: "pi extension and pi sync",
+ events: []schema.Event{
+ event("2026-06-11T10:00:00Z", "pi_cli", plugin, "pi-0197", "session.started", nil, nil),
+ event("2026-06-11T10:00:02Z", "pi_cli", poll, "pi-0197", "agent.response", &tokens{input: 2400, output: 610, cacheRead: 300, cost: 0.0102}, model("claude-sonnet-4-5")),
+ event("2026-06-11T10:00:19Z", "pi_cli", plugin, "pi-0197", "agent.response", &tokens{input: 2400, output: 610, cacheRead: 300, cost: 0.0102}, model("claude-sonnet-4-5")),
+ },
+ want: tokens{input: 2400, output: 610, cacheRead: 300, cost: 0.0102},
+ wantEvents: 1,
+ suppressed: 1,
+ },
+ {
+ // A harness or session spelling difference must not split one session into two scopes.
+ name: "harness spellings normalize to one session",
+ events: []schema.Event{
+ event("2026-06-11T10:00:00Z", "Claude Code", otlp, "c4", "prompt.submitted", nil, nil),
+ event("2026-06-11T10:00:04Z", "claude_code", poll, "C4", "agent.response", &tokens{input: 5, output: 50}, nil),
+ event("2026-06-11T10:00:05Z", "claude-code", otlp, "c4", "tool.invoked", &tokens{input: 5, output: 50}, nil),
+ },
+ want: tokens{input: 5, output: 50},
+ wantEvents: 1,
+ suppressed: 1,
+ },
+ {
+ // Beacon installed mid-session: the backfill is the only record of what came before.
+ name: "poll before the live cutover is kept",
+ events: []schema.Event{
+ event("2026-06-11T09:10:00Z", "claude_code", poll, "c5", "agent.response", &tokens{input: 100, output: 10}, nil),
+ event("2026-06-11T09:20:00Z", "claude_code", poll, "c5", "agent.response", &tokens{input: 200, output: 20}, nil),
+ event("2026-06-11T10:00:00Z", "claude_code", otlp, "c5", "prompt.submitted", nil, nil),
+ event("2026-06-11T10:00:04Z", "claude_code", poll, "c5", "agent.response", &tokens{input: 300, output: 30}, nil),
+ event("2026-06-11T10:00:05Z", "claude_code", otlp, "c5", "tool.invoked", &tokens{input: 300, output: 30}, nil),
+ },
+ want: tokens{input: 600, output: 60},
+ wantEvents: 3,
+ suppressed: 1,
+ },
+ {
+ // Without a session id there is no evidence the two reports are the same turn.
+ name: "no session id is never deduplicated",
+ events: []schema.Event{
+ event("2026-06-11T10:00:00Z", "claude_code", otlp, "", "tool.invoked", &tokens{input: 7}, nil),
+ event("2026-06-11T10:00:01Z", "claude_code", poll, "", "agent.response", &tokens{input: 7}, nil),
+ },
+ want: tokens{input: 14},
+ wantEvents: 2,
+ },
+ {
+ name: "poll only is untouched",
+ events: []schema.Event{
+ event("2026-06-11T10:00:00Z", "opencode", poll, "ses_1", "token.usage", &tokens{input: 10, cost: 0.001}, nil),
+ event("2026-06-11T10:00:01Z", "opencode", poll, "ses_1", "token.usage", &tokens{input: 20, cost: 0.002}, nil),
+ },
+ want: tokens{input: 30, cost: 0.003},
+ wantEvents: 2,
+ },
+ {
+ name: "live only is untouched",
+ events: []schema.Event{
+ event("2026-06-11T10:00:00Z", "opencode", plugin, "ses_1", "agent.response", &tokens{input: 10}, nil),
+ event("2026-06-11T10:00:01Z", "opencode", plugin, "ses_1", "agent.response", &tokens{input: 20}, nil),
+ },
+ want: tokens{input: 30},
+ wantEvents: 2,
+ },
+ {
+ // Each session decides for itself: one with live capture does not suppress another's
+ // backfill, and the same session id on another endpoint is another session.
+ name: "sessions and endpoints are independent",
+ events: []schema.Event{
+ event("2026-06-11T10:00:00Z", "pi_cli", plugin, "pi-a", "session.started", nil, nil),
+ event("2026-06-11T10:00:02Z", "pi_cli", poll, "pi-a", "agent.response", &tokens{input: 100}, nil),
+ event("2026-06-11T10:00:03Z", "pi_cli", plugin, "pi-a", "agent.response", &tokens{input: 100}, nil),
+ event("2026-06-11T10:00:04Z", "pi_cli", poll, "pi-b", "agent.response", &tokens{input: 40}, nil),
+ event("2026-06-11T10:00:05Z", "pi_cli", poll, "pi-a", "agent.response", &tokens{input: 9}, func(e *schema.Event) {
+ e.Endpoint.Hostname = "host-b"
+ }),
+ },
+ want: tokens{input: 149},
+ wantEvents: 3,
+ suppressed: 1,
+ },
+ {
+ // A live channel that reports how full the window is, and nothing spent, is no
+ // authority on spend: it must not suppress the backfill.
+ name: "context-only live capture does not suppress poll",
+ events: []schema.Event{
+ event("2026-06-11T10:00:00Z", "pi_cli", plugin, "pi-c", "session.context", nil, func(e *schema.Event) {
+ e.Model = "claude-sonnet-4-5"
+ e.GenAI = &schema.GenAIInfo{Context: &schema.GenAIContextInfo{UsedTokens: int64Ptr(52000), LimitTokens: int64Ptr(200000)}}
+ }),
+ event("2026-06-11T10:00:05Z", "pi_cli", poll, "pi-c", "agent.response", &tokens{input: 52000, output: 100}, model("claude-sonnet-4-5")),
+ },
+ want: tokens{input: 52000, output: 100},
+ wantEvents: 1,
+ },
+ {
+ // A live channel that reports cost only claims cost only: the backfill's tokens
+ // stay, and its cost does not count twice.
+ name: "live fields are suppressed field by field",
+ events: []schema.Event{
+ event("2026-06-11T10:00:00Z", "claude_code", otlp, "c6", "cost.usage", &tokens{cost: 0.05}, metric("claude_code.cost.usage")),
+ event("2026-06-11T10:00:04Z", "claude_code", poll, "c6", "agent.response", &tokens{input: 30, output: 3, cost: 0.05}, nil),
+ },
+ want: tokens{input: 30, output: 3, cost: 0.05},
+ wantEvents: 2,
+ },
+ {
+ // A record with no usable timestamp cannot be placed against the cutover, and a live
+ // channel with no usable timestamp cannot set one; both stay, as in the Codex span
+ // preference.
+ name: "zero timestamps are kept",
+ events: []schema.Event{
+ event("2026-06-11T10:00:00Z", "opencode", plugin, "ses_z", "agent.response", &tokens{input: 10}, nil),
+ event("", "opencode", poll, "ses_z", "token.usage", &tokens{input: 10}, nil),
+ event("", "cline", plugin, "task-z", "session.ended", &tokens{input: 5}, nil),
+ event("2026-06-11T10:00:00Z", "cline", poll, "task-z", "agent.response", &tokens{input: 5}, nil),
+ },
+ want: tokens{input: 30},
+ wantEvents: 4,
+ },
+ {
+ // An event with no collection method (a log written before provenance existed) is
+ // not evidence of live capture.
+ name: "unmarked events are not live",
+ events: []schema.Event{
+ event("2026-06-11T10:00:00Z", "claude_code", "", "c7", "tool.invoked", &tokens{input: 8}, nil),
+ event("2026-06-11T10:00:01Z", "claude_code", poll, "c7", "agent.response", &tokens{input: 8}, nil),
+ },
+ want: tokens{input: 16},
+ wantEvents: 2,
+ },
+ }
+ for _, tc := range cases {
+ t.Run(tc.name, func(t *testing.T) {
+ report := Aggregate(tc.events, Options{})
+ got := report.Totals
+ if got.InputTokens != tc.want.input || got.OutputTokens != tc.want.output ||
+ got.CacheReadInputTokens != tc.want.cacheRead || got.CacheCreationInputTokens != tc.want.cacheCreation ||
+ got.ReasoningOutputTokens != tc.want.reasoning || !closeTo(got.CostUSD, tc.want.cost) {
+ t.Fatalf("totals = %+v, want %+v", got, tc.want)
+ }
+ if report.EventsWithUsage != tc.wantEvents {
+ t.Fatalf("events_with_usage = %d, want %d", report.EventsWithUsage, tc.wantEvents)
+ }
+ if report.SuppressedPollEvents != tc.suppressed {
+ t.Fatalf("suppressed_poll_events = %d, want %d", report.SuppressedPollEvents, tc.suppressed)
+ }
+ })
+ }
+}
+
+// TestAggregateScopedKeepsTheLiveCutoverOutsideTheWindow checks that a time window that starts
+// after a session's first live record does not move the cutover to the first record inside the
+// window, which would let the backfill's copy of the window's first request count twice.
+func TestAggregateScopedKeepsTheLiveCutoverOutsideTheWindow(t *testing.T) {
+ mk := func(ts, method string, input int64) schema.Event {
+ e := usageEventFixture(ts, "claude_code", "c8", "claude-opus-4-6", func(e *schema.Event) {
+ e.Harness.CollectionMethod = method
+ })
+ if input == 0 {
+ e.GenAI = nil
+ e.Event.Action = "prompt.submitted"
+ } else {
+ e.GenAI.Usage.InputTokens = int64Ptr(input)
+ }
+ return e
+ }
+ all := []schema.Event{
+ mk("2026-06-11T09:00:00Z", schema.CollectionMethodOTLP, 0),
+ mk("2026-06-11T09:00:05Z", schema.CollectionMethodOTLP, 10),
+ mk("2026-06-11T10:00:04Z", schema.CollectionMethodPoll, 20),
+ mk("2026-06-11T10:00:05Z", schema.CollectionMethodOTLP, 20),
+ }
+ window := all[2:]
+ report := AggregateScopedWithContexts(window, all, "", Options{})
+ if report.Totals.InputTokens != 20 || report.SuppressedPollEvents != 1 {
+ t.Fatalf("totals = %+v suppressed = %d, want the window's request once", report.Totals, report.SuppressedPollEvents)
+ }
+}
+
+func closeTo(a, b float64) bool {
+ d := a - b
+ return d < 1e-9 && d > -1e-9
+}
diff --git a/docs/cli/token-usage.mdx b/docs/cli/token-usage.mdx
index bf2e42b2..13981783 100644
--- a/docs/cli/token-usage.mdx
+++ b/docs/cli/token-usage.mdx
@@ -26,7 +26,7 @@ A few rules shape how the numbers read:
- **Counts are kept disjoint.** `input_tokens`, `output_tokens`, `cache_read`, and `cache_creation` do not overlap, so a total is `input + output + cache_read + cache_creation` without double-counting. Codex reports input inclusive of both cache reads and cache writes, so Beacon subtracts both before storing uncached input.
- **Cost is runtime-reported only.** `cost_usd` reflects what the runtime emits. Beacon never derives cost from local pricing tables, so cost appears only for runtimes that report it.
- **Attribution depends on the metadata a runtime attaches.** Model, harness, repository, and run grouping work whenever those fields are present. Per-session grouping (`--session`, the session step tree) requires a session or conversation id on the usage signal. Per-user grouping uses the event user directly or a matching session context event from a local hook.
-- **Duplicate channels are de-duplicated.** When a runtime emits the same usage on more than one signal (Claude Code reports it on both logs and metrics), Beacon counts it once.
+- **Duplicate channels are de-duplicated.** When a runtime emits the same usage on more than one signal (Claude Code reports it on both logs and metrics), Beacon counts it once. The same holds when one session is read both live (OTLP, a hook, or a Beacon-managed plugin) and by a `beacon endpoint <runtime> sync` backfill of the runtime's own session store (`harness.collection_method=poll`), as Claude Code, Codex CLI, Cline, OpenCode, OpenClaw Gateway and Pi sessions can be. Within one session on one endpoint, live capture is authoritative from the moment its channel started capturing that session: backfilled usage from then on gives up the fields the live channel reports and is dropped once nothing is left, while backfilled usage from before it is kept, since it is the only record of turns from before Beacon's live capture started. Usage without a session id, or without a usable timestamp, is never matched this way, and a live channel that reports only context occupancy does not suppress anything. `--json` reports how many backfilled events were dropped as `suppressed_poll_events`.
### Per-runtime coverage