Skip to content

The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu

27.0 KB
Raw Blame
---
title: "beacon token-usage"
description: "Report token usage and attribution from Beacon runtime logs"
---

## Command overview

`beacon token-usage` reads Beacon runtime events from the JSONL log and summarizes token usage, cost, users, models, sessions, runs, harnesses, and repositories.

```bash title="Command syntax"
beacon token-usage [flags]
```

Use this command when you need a local report before opening the dashboard, sending diagnostics, or validating token attribution for a specific agent session.

<Note>
  In Beacon `v0.0.70`, token reporting moved from `beacon endpoint tokens` to the top-level `beacon token-usage` command. Update scripts, CI jobs, and runbooks that still invoke the old endpoint subcommand.
</Note>

## How token attribution works

Beacon normalizes token telemetry from every runtime into one canonical [`gen_ai.usage`](/telemetry-schema/normalization#usage-metadata) representation, then aggregates it locally from the runtime JSONL log. There are no per-harness token fields, so a report over mixed runtimes uses the same vocabulary throughout.

A few rules shape how the numbers read:

- **Counts are kept disjoint.** `input_tokens`, `output_tokens`, `cache_read`, and `cache_creation` do not overlap, so a total is `input + output + cache_read + cache_creation` without double-counting. Codex reports input inclusive of both cache reads and cache writes, so Beacon subtracts both before storing uncached input. The same applies to any source using the OpenTelemetry GenAI names (`gen_ai.usage.input_tokens`, or the older `prompt_tokens` spellings), whose input includes cached tokens by definition: Beacon subtracts the cache read and cache creation counts reported alongside it, unless the input is smaller than they are and so is already uncached. Claude Code's own counts are already disjoint and are stored as reported. `cache_creation.ephemeral_1h_input_tokens` is a breakdown of `cache_creation`, never a fifth count.
- **Reported and estimated cost are kept apart.** `cost_usd` is what the runtime reported and nothing else. Beside it, every report carries a list-price estimate (`estimated_cost_usd`) for every runtime, including the ones that report no cost, and an effective cost that picks one of the two. See [Cost: reported, estimated, effective](#cost-reported-estimated-effective). Events never carry an estimate; it is computed when the report is built.
- **Attribution depends on the metadata a runtime attaches.** Model, harness, repository, and run grouping work whenever those fields are present. Per-session grouping (`--session`, the session step tree) requires a session or conversation id on the usage signal. Per-user grouping uses the event user directly or a matching session context event from a local hook.
- **Duplicate channels are de-duplicated.** When a runtime emits the same usage on more than one signal (Claude Code reports it on both logs and metrics), Beacon counts it once. The same holds when one session is read both live (OTLP, a hook, or a Beacon-managed plugin) and by a `beacon endpoint <runtime> sync` backfill of the runtime's own session store (`harness.collection_method=poll`), as Claude Code, Codex CLI, Cline, OpenCode, OpenClaw Gateway and Pi sessions can be. Within one session on one endpoint, live capture is authoritative from the moment its channel started capturing that session: backfilled usage from then on gives up the fields the live channel reports and is dropped once nothing is left, while backfilled usage from before it is kept, since it is the only record of turns from before Beacon's live capture started. Usage without a session id, or without a usable timestamp, is never matched this way, and a live channel that reports only context occupancy does not suppress anything. `--json` reports how many backfilled events were dropped as `suppressed_poll_events`.

### Per-runtime coverage

Token coverage is not uniform, and the reason is almost always the runtime rather
than Beacon: most agent runtimes do not report usage on the surface Beacon collects
from. A turn with no token attribution is a turn whose runtime never reported any.

**Collected and verified against captured payloads:**

| Runtime | Tokens | Cost | Per-session | Per-user |
|---------|--------|------|-------------|----------|
| Claude Code | input, output, cache read, cache creation, reasoning | Reported (`cost_usd`) | Yes | When local user context is present |
| Codex CLI | input, output, cache read, cache creation, reasoning | Not emitted by Codex | Yes, from the turn trace | Yes, from matching SessionStart context |
| Codex Desktop | input, output, cache read, cache creation, reasoning | Not emitted by Codex | Yes, from the turn trace | When local user context is present |
| Claude Cowork | input, output, cache read, cache creation | Reported when Cowork emits it | Yes | When the OTLP stream carries it |
| Cline | input, output, cache read, cache creation | Reported (`cost_usd`) | Yes, per task | Yes |
| OpenCode | input, output, cache read, cache creation, reasoning | Reported (`cost_usd`) | Yes | Yes |
| Pi | input, output, cache read, cache creation, reasoning | Reported (`cost_usd`) | Yes | Yes |
| Oh My Pi | input, output, cache read, cache creation, reasoning | Reported (`cost_usd`) | Yes | Yes |
| Prime Agent | input, output, cache read, cache creation, reasoning | Reported (`cost_usd`) | Yes | Yes |
| Senpi | input, output, cache read, cache creation, reasoning | Reported (`cost_usd`) | Yes | Yes |
| Hermes Agent | input, output, cache read, cache creation, reasoning | Reported (`cost_usd`): Hermes's actual cost, or its own estimate when it recorded no actual | Yes, from `beacon endpoint hermes sync` | Yes |
| Factory Droid | input, output, cache read, cache creation, reasoning | Not reported in USD; Factory credits stay in `raw` | Yes, from `beacon endpoint factory sync` | Yes |
| DeepSeek Harness | input, output, cache read, cache creation | Not reported | Yes, from committed session records | Yes |
| fx (Vercel Labs) | input, output, cache read, cache creation | Reported (`cost_usd`) | Yes | Yes |
| Harness² | input, output, cache read, cache creation, reasoning | Reported (`cost_usd`) when the OpenCode executor reports it; not emitted by Codex. Neither stream names the model, so there is no list-price estimate | Yes, per rollout from `beacon endpoint harness2 sync` | Yes |
| GitHub Copilot CLI | input, output, cache read, cache creation, reasoning | Reported (`cost_usd`) for sessions started on or after 2026-06-01, from AI credits | Yes | Yes |
| OpenClaw Gateway | input, output, cache read, cache creation | Not reported on the hook | Yes, per model response | Yes |
| Gemini CLI | input, output, cache read, reasoning | Not emitted by Gemini CLI | Yes, per model response | When local user context is present |
| Antigravity CLI | input, output, cache read | Not reported by Antigravity | Yes, per model response from `beacon endpoint antigravity sync` | Yes |

OpenClaw reports usage on its `llm_output` hook, which is a *conversation* hook -- OpenClaw withholds those from every non-bundled plugin until `plugins.entries.beacon-endpoint.hooks.allowConversationAccess` is set in its config. An OpenClaw endpoint collecting prompts, tools, commands and files while reporting no tokens at all is almost always that key, and `beacon endpoint hooks status --harness openclaw` says so. OpenClaw does compute a per-turn price, from an operator-configured cost table rather than from a provider report, so Beacon does not record it as a reported cost. The session-file sync, `beacon endpoint integrations openclaw sessions sync`, reads the usage OpenClaw commits on each assistant message and needs no grant. See [OpenClaw Gateway](/runtimes/openclaw-gateway).

Hermes Agent and Factory Droid report usage only through their session-store sync: neither runtime's hooks carry a token count, so their usage arrives when `beacon endpoint hermes sync` or `beacon endpoint factory sync` runs. Both keep running session totals, which the sync differences into deltas. Until the sync runs, the coverage report lists them as `silent`, with a note naming the command.

Antigravity CLI is the same case with per-response counts: its hooks carry no token count, but every planner response it commits to its transcript records `input_tokens`, `cache_read_tokens` and `output_tokens`, and `beacon endpoint antigravity sync` records them once per response. Antigravity is Gemini-backed and Gemini's input count includes cached content, so Beacon stores input minus the cache reads (unless the input is smaller than they are). Until the sync runs, the coverage report lists Antigravity CLI as `silent`, with a note naming the command.

Gemini CLI reports each model response on its `gemini_cli.api_response` log record and again on the `gemini_cli.token.usage` counter, under Gemini's own counts rather than the semconv names. Those counts are not disjoint: Gemini's input (`promptTokenCount`) already includes the cached content, its output (`candidatesTokenCount`) leaves out thinking tokens, and its tool-use prompt tokens are input that the prompt count does not hold. Beacon therefore records input as prompt minus cached plus tool-use prompt, cache read as the cached count, and output as candidates plus thoughts with the thoughts as its reasoning breakdown, so a response's Beacon total equals Gemini's own `total_token_count`. Gemini also restates input and output on a `gen_ai.client.inference.operation.details` record; Beacon reads usage only from `api_response`, so a response is counted once, and the counter is de-duplicated against the log because both carry Gemini's `session.id`. These shapes are pinned from Gemini CLI's telemetry source rather than from a recorded session.

Cline reports usage once per task rather than per model call, because Beacon's rollups sum `gen_ai.usage` across events and reporting at both granularities would count every token twice. fx reports cumulative totals, which Beacon differences into per-turn deltas for the same reason, and so does a Harness² Codex rollout, whose `turn.completed` carries the thread total. GitHub Copilot CLI reports assistant message output tokens and cumulative shutdown model totals; Beacon records the message output directly, then differences shutdown totals for input, cache, reasoning, and any output not already counted. Cost comes from the shutdown's session-wide `totalNanoAiu`, differenced the same way and converted at one AI credit = 10^9 nano-AIU = US$0.01. It is recorded once per shutdown: on the model's usage event when the shutdown names one model, otherwise on one usage event with no model, so a multi-model session is not counted once per model. Sessions started before GitHub moved Copilot to AI credits on 2026-06-01 were billed per premium request, so they record no cost; the raw nano-AIU total stays in `raw.copilot`. Annual Copilot Pro and Pro+ subscribers stay on premium-request billing until their plan renews, and the session record does not say which plan applies, so for them the recorded cost is what the session would cost in AI credits rather than their bill.

**Collected through the generic OpenTelemetry path, not yet verified against captured payloads:**

| Runtime | Status |
|---------|--------|
| VS Code Copilot | Usage is captured when the runtime emits the OTel GenAI semconv names (`gen_ai.usage.*` attributes, or a `gen_ai.client.token.usage` metric split by `token_type`). The generic path is tested; these runtimes' own payloads have not been captured and pinned. |
| Grok Bot | Reaches Beacon only through Cursor's server-side OpenTelemetry export, so usage arrives only if that export carries the semconv names |
| goose | goose's `chat` spans carry the full `gen_ai.usage.*` set, cache reads and writes included, under the exact semconv names -- that much is read out of the runtime rather than hoped for. What is conditional is whether it arrives: usage reaches Beacon only over manually configured OTLP, so a hooks-only goose setup is correctly silent rather than faulty. goose reports no cost. See [goose](/runtimes/goose) |

This table records what Beacon is built to read, not what your machine actually produced. A runtime can be listed here and still contribute nothing, because it was never run, or because its build reports no usage.

## Context occupancy

Some runtimes report how full the model's context window was without reporting what a turn cost. That is a different measurement from spend, and it is recorded in a different place: `gen_ai.context`, holding `used_tokens` and `limit_tokens`.

The distinction is not cosmetic. Everything in `gen_ai.usage` is **additive** -- every total on this page sums it. Occupancy is a level at one moment: a session whose window was 92% full ten times did not spend ten windows. So context is never summed, never added to a total, and never counted as a runtime reporting its spend. It feeds the utilization view, where a reported window also replaces the model table's default, because a reported limit reflects the tier a call actually ran under.

| Runtime | Reported on | Fields read |
|---------|-------------|-------------|
| Cursor | `preCompact` | `context_tokens`, `context_window_size` |
| Qwen Code | `Stop` | `input_tokens`, `context_limit` |

Both runtimes also report the ratio of the two (`context_usage_percent`, `context_usage`). It is derivable from the pair, so it stays in `raw` rather than being stored a second time in different units.

A runtime that reports occupancy and no spend still shows as `not_instrumented` in the coverage report below. That is deliberate: it has not told Beacon what it cost.


**Not collected:**

| Runtime | Why |
|---------|-----|
| Cursor | No hook payload carries a token count. Cursor does report context occupancy on `preCompact`, which is recorded separately -- see [Context occupancy](#context-occupancy) below |
| Grok Build, Devin CLI, Devin Desktop | No token counts on the hook payloads Beacon receives |
| Muse Code | Usage arrives on `PostLLMCall` and in Muse's own session log, neither of which Beacon reads. See [Muse Code](/runtimes/muse-code) |
| Qwen Code | The `Stop` payload carries session-level context counters, not per-call usage. Those counters are recorded as context occupancy rather than spend -- see [Context occupancy](#context-occupancy) below. See [Qwen Code](/runtimes/qwen-code) |
| OpenHands | None of the six hook payloads carries token counts, and OpenHands exposes no other hook that would. See [OpenHands](/runtimes/openhands) |
| Kiro | No Kiro hook payload carries token counts. See [Kiro](/runtimes/kiro) |
| Kimi Code | No hook payload carries a token count on any of the twenty events. Kimi Code does count tokens -- the TUI shows context occupancy and `PreCompact` carries a `token_count` -- but neither is per-call spend. See [Kimi Code](/runtimes/kimi-code) |
| Claude.ai | The adapter parses counts, but no recorded claude.ai stream has carried a usage object. See [Claude.ai](/runtimes/claude-web#limits) |
| ChatGPT | The chat stream reports no token counts |
| Devin Cloud | Devin bills ACUs rather than tokens. The count is preserved in the Devin raw block; there is no token or USD equivalent to normalize it into |

Legacy metric-only Codex rows written before the turn trace existed remain visible in historical totals, but cannot be assigned to a session or user. See the [Codex CLI](/runtimes/codex-cli#token-usage-and-cost) and [Claude Code](/runtimes/claude-code) runtime pages for details.

The **Cost** column says what the runtime itself reports. Every runtime that reports tokens also gets a list-price estimate in the report, so "Not reported" no longer means the report shows no cost; see the next section.

## Cost: reported, estimated, effective

Each usage row in the report (totals, every grouping, time buckets, the session step tree) carries three costs:

| Field | Meaning |
|-------|---------|
| `cost_usd` | What the runtime reported, summed. Unchanged: Beacon never fills it in, and events never carry anything else in `gen_ai.usage.cost_usd` |
| `estimated_cost_usd` | What the same tokens cost at the provider's published API list price, for every token whose model the catalog prices, whether or not the runtime also reported a cost. It is the API-equivalent cost, useful for comparing runtimes and for runtimes on a subscription |
| `effective_cost_usd` | The best single figure. Within one endpoint, runtime and session, if the runtime reported any cost, the session's cost is what it reported and the estimate takes no part. Otherwise the session contributes its estimate. Events without a session are judged one at a time. `cost_source` says what it is made of: `reported`, `estimated`, `mixed`, or empty when there was nothing to price |

A runtime's own figure wins because it knows the plan, the discounts and the routing that a list price cannot. The same rule keeps Claude Code right: its cost arrives on a separate `claude_code.cost.usage` metric beside token events that carry none, so its sessions are reported and its token events add no estimate on top.

`unpriced_events` and `unpriced_tokens` count usage whose model neither the catalog nor your overrides file prices. Those tokens are in every token total and in no estimate: an unknown model stays unknown rather than reading as free. `beacon pricing show <model>` explains why a model is unpriced, and an [overrides file](/cli/pricing#the-overrides-file) can price it.

The text report prints `COST USD` (reported) and `EST COST USD` (estimate) side by side, a footer naming the catalog, and the effective total. A `?` in the estimate column marks a row whose tokens were all unpriced. The dashboard's token view shows both, the estimate marked `~` and labeled "Est.".

### How the estimate is computed

- **Catalog.** Rates come from a catalog embedded in the binary, generated from LiteLLM's published model price list. The report's `pricing.catalog` block names the source, its upstream commit and the generation date. Nothing is fetched at report time.
- **Overrides.** A local overrides file (`~/.beacon/endpoint/pricing/overrides.json`, or `--pricing-file`) prices models the catalog does not know and replaces list prices with your negotiated rates. It wins over the catalog for every model one of its names matches. `pricing.overrides` records its path and SHA-256 digest, and each `pricing.models[]` entry says `source: override` or `source: catalog` (plus `alias` when a gateway name was mapped). An invalid file is an error, not a silent fallback to list prices; the dashboard, which must keep answering, falls back and reports it in `pricing.overrides.error`. See [beacon pricing](/cli/pricing).
- **Standard tier only.** The catalog has no batch, flex or priority tiers, regional surcharges, or subscription and enterprise discounts: nothing in a usage record says a call used them. Put negotiated rates in the overrides file.
- **Model matching.** The model is matched to a catalog entry by exact name, then by canonical spelling (case, provider prefix), dotted versions (`claude-sonnet-4.6`), Bedrock route prefixes, and after stripping a context-window marker (`[1m]`), a snapshot date or an effort suffix. Never by similarity. `pricing.models` records, per model, the catalog key, how it matched, the base rates in USD per million tokens, and any rate fallback used.
- **Missing rates.** A model with no published cache-read or cache-write price has those tokens priced at its input rate, and the fallback is named in `pricing.models[].fallbacks`.
- **1h cache writes.** Anthropic bills a cache write with a one-hour TTL at 2x the input rate, against 1.25x for the default five-minute TTL. Where the source reports the split, `gen_ai.usage.cache_creation.ephemeral_1h_input_tokens` carries the one-hour subset and those writes are priced at the model's one-hour rate (`cache_write_1h_at_write_rate` is named in `fallbacks` when the catalog has none). The split is reported by Claude Code session files (`beacon endpoint claude sync`), Pi-family runtimes whose usage carries `cacheWrite1h` (Pi fills it for Anthropic models), and the browser extension on claude.ai when the stream includes it. Claude Code's OTLP `api_request` log and token metric report only the total, so OTLP-only Claude Code usage prices every write at the five-minute rate and can understate sessions that use one-hour caching. Reports carry the subset as `cache_creation_1h_input_tokens` wherever it is non-zero.
- **Reasoning** is part of output and is not billed again. It is priced at the output rate only when a runtime reported reasoning and no output.
- **Long-context bands.** Several providers bill every token of a request above a prompt size at a higher rate (Anthropic and Gemini above 200k tokens, OpenAI's newer models above 272k). A band can only apply to a single request, so it is applied only to usage Beacon knows is one request: Claude Code `claude_code.api_request` records and session files, Codex session `token_count` records, Pi-family `message_end` and session files, OpenCode assistant messages, OpenClaw session files and DeepSeek Harness assistant messages. Metric datapoints, Codex turn spans, OpenClaw `llm_output` (one per turn), Cline task totals, Copilot shutdown totals and the Hermes, Factory and fx deltas sum several requests and are priced at base rates, which can understate a session that sent long prompts. `pricing.models[].bands_applied` counts the banded requests.
- **After deduplication.** The estimate prices the final per-event counts, after duplicate channels are removed and cumulative counters are differenced, so it covers exactly the tokens the totals count.

## Checking coverage

A usage report answers "what did I spend". It cannot answer "is this all of it", and that is the question to settle before treating a number as a bill. A runtime whose telemetry never arrives looks exactly like a runtime nobody used: both are simply absent from every rollup, so a total can omit an entire agent and still look complete.

```bash title="Check which runtimes are contributing token telemetry"
beacon token-usage --coverage
```

```
Token coverage over 4812 events: 3 runtime(s) reporting usage, 1 silent

RUNTIME      STATUS            INSTALLED  EVENTS  USAGE EVENTS  TOKENS   NOTE
codex_cli    silent            yes        412     0             0        per-turn usage trace; Codex emits no cost
claude_code  covered           yes        3106    288           4271883  OTLP token and cost telemetry
cline        covered           yes        190     12            283740   plugin reports usage once per task
cursor       not_instrumented  yes        1104    0             0        Cursor hook payloads carry no token counts
```

The status column is the report:

| Status | Meaning |
|--------|---------|
| `covered` | The runtime reported usage in the window. Nothing to do |
| `silent` | The runtime produced events but no usage, and Beacon is built to read usage from it. **This is the one to investigate** — a stale runtime build, a hook that stopped firing, or an OTLP export that is no longer configured |
| `inactive` | Beacon configured this runtime, but it produced no events at all in the window. Usually means it was not used; can also mean its hooks or OTLP export stopped firing |
| `not_instrumented` | The runtime ran, and Beacon cannot read token usage from it. Expected, not a fault. The note column says why |

The `not_instrumented` distinction is what makes the report worth reading. Without it Cursor would appear as a problem on every run, and a column that always shows a problem is a column people stop reading.

The `INSTALLED` column means "Beacon configured this runtime", not "this product is on the machine". Only configuration Beacon wrote counts — a hook command, a managed plugin marker, or a local OTLP endpoint it set. That is the population where silence is informative: Beacon wired it up, so it should be reporting. A runtime present but configured by hand would not be reporting through Beacon either way, so it appears only if it actually emits events.

Coverage applies the `--since`, `--until` and `--harness` scopes, and deliberately ignores `--session`, `--model`, `--repository` and `--run-id`. Those filters select usage-bearing events almost by definition — an event carries no model unless it carried a model call — so a coverage report computed over them would find every runtime covered and prove nothing.

Add `--json` for the machine-readable form, which carries the same per-runtime rows plus `silent`, `covered` and `total_events` counts. A `silent` count above zero is the alertable condition.

The local dashboard serves the same report at `GET /api/tokens/coverage`, with the same body as `--coverage --json`. It honors `since`, `until` and `harness` and ignores the rest, for the reason the command does. Both read one join and one expectation table, so the two cannot reach different conclusions about the same log.

## Examples

Show token usage from the default per-user endpoint log:

```bash title="Show local token usage"
beacon token-usage
```

Print a machine-readable report:

```bash title="Print JSON"
beacon token-usage --json
```

Filter to one session and include per-step detail:

```bash title="Filter by session"
beacon token-usage --session claude-session-123
```

Group usage into hourly time buckets:

```bash title="Bucket usage over time"
beacon token-usage --bucket 1h
```

Inspect a CI runtime log and filter to a run:

```bash title="Inspect a CI run"
beacon token-usage --log-path ./beacon-runtime.jsonl --run-id github/123456 --json
```

## Filters

Filters can be combined to narrow the report to a time range, model, agent harness, repository, session, or CI run.

| Flag | Description |
|------|-------------|
| `--since <timestamp>` | Only include events at or after this RFC3339 timestamp |
| `--until <timestamp>` | Only include events at or before this RFC3339 timestamp |
| `--session <id>` | Filter to one session and include per-step detail |
| `--model <name>` | Filter by model name |
| `--harness <name>` | Filter by harness name |
| `--repository <path-or-url>` | Filter by repository |
| `--run-id <id>` | Filter by CI run ID. Bare run IDs and composite provider/run keys are accepted |
| `--bucket <duration>` | Time-series bucket size, such as `1h` or `15m` |
| `--top <count>` | Limit each grouping to the top N entries. `0` keeps all entries |

## Runtime log selection

By default, the command reads the per-user endpoint runtime log. Use `--system` for a system-mode endpoint, or `--log-path` when reading a copied runtime log or a CI session artifact.

| Flag | Description |
|------|-------------|
| `--user` | Use per-user endpoint paths. Enabled by default |
| `--system` | Use system endpoint paths |
| `--log-path <path>` | Runtime JSONL log path. In CI, point this at the session log |
| `--json` | Print the token usage report as JSON |
| `--pricing-file <path>` | Pricing overrides file for the cost estimate. Defaults to `pricing/overrides.json` in the endpoint directory when it exists. See [beacon pricing](/cli/pricing) |

## Related

<Columns cols={2}>
  <Card title="Endpoint dashboard" icon="chart-line" href="/cli/dashboard">
    Explore endpoint runtime logs locally in a browser.
  </Card>
  <Card title="Endpoint inventory" icon="clipboard-list" href="/cli/endpoint-inventory">
    See configured, detected, hook, and observed endpoint inventory.
  </Card>
</Columns>