Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

feat(reporting): TokensUsageRecord — deliberate client wire shape for tokens_usages

Louis Choquel67115cd3ffff
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c74c6eedb..be873e85c 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,7 +4,12 @@
### Added
-- **Durable runs now persist token usage: the delivery executor writes a `tokens_usages.json` result artifact** (`{"tokens_usages": [...], "usage_assembly_error": null}`) alongside `working_memory.json`, the `main_stuff.*` renders, and the graph outputs. The records use the same per-record `model_dump(mode="json")` wire shape the `/execute` response carries on `pipe_output.tokens_usages` (token counts by category, `unit_costs` in $/1M, model id — for LLM, img-gen, extract, and search calls), so a client polling a durable run's result files can compute costs the same way a sync `/execute` caller does. The artifact is written unconditionally: a run with usage assembly off yields explicit nulls, distinguishable from a pre-artifact run (file absent).
+- **Durable runs now persist token usage: the delivery executor writes a `tokens_usages.json` result artifact** (`{"tokens_usages": [...], "usage_assembly_error": null}`) alongside `working_memory.json`, the `main_stuff.*` renders, and the graph outputs. The records use the client-facing `TokensUsageRecord` wire shape — the same shape the `/execute` response carries on `pipe_output.tokens_usages` — so a client polling a durable run's result files reads costs the same way a sync `/execute` caller does. The artifact is written unconditionally: a run with usage assembly off yields explicit nulls, distinguishable from a pre-artifact run (file absent).
+- **`TokensUsageRecord` — a deliberate client wire shape for token usage** (`pipelex/reporting/usage_records.py`). One flat record per inference call: `model_type`, `inference_model_name`, `inference_model_id`, `pipe_code`, `job_category`, `unit_job_id`, `nb_tokens_by_category` (raw provider counts), computed USD `cost` (`null` when the model has no rate table), and ISO `started_at`/`completed_at`. Enum-ish fields are open string sets on the wire, so runtime enum churn is non-breaking for clients. `compute_tokens_usage_cost` reuses the existing cost engine (`model_cost_per_token` + the cached/non-cached split), so the wire `cost` equals the CLI cost table's total for the same call. Documented in [TokensUsage Wire Records](docs/under-the-hood/tokens-usage-wire-records.md).
+
+### Changed
+
+- **Breaking: `tokens_usages` records on client surfaces are now `TokensUsageRecord` wire records, not dumps of the internal usage models.** Both the `/execute` response (`pipe_output.tokens_usages`, via the new `apply_tokens_usage_wire_shape` helper API servers apply to the response dump) and the durable `tokens_usages.json` artifact stop leaking runtime internals: `job_metadata` (with its `user_id`, `session_id`, `request_id`, `pipe_run_id`, `otel_context`, `trace_context`, `content_generation_job_id`, `pipeline_run_id`) and the `unit_costs` rate table no longer cross the wire — the kept `JobMetadata` fields (`pipe_code`, job-kind enums, timing) are flattened onto the record and `cost` is computed server-side. Null semantics unchanged (`null` = assembly off, `[]` = no inference). Internal crossings (runtime-bridge payloads, Temporal transport, the usage-event telemetry stream) keep full-fidelity records, untouched.
### Fixed
diff --git a/docs/features/cost-tracking.md b/docs/features/cost-tracking.md
index ce29aa182..d863c90ec 100644
--- a/docs/features/cost-tracking.md
+++ b/docs/features/cost-tracking.md
@@ -36,3 +36,7 @@ The `--costs` gate is read off the output itself — `tokens_usages` is `None` e
Implement custom reporting backends via the [Reporting Delegate](advanced-customizations.md) injection point. The reporting protocol receives structured cost events during execution, allowing you to push data to your own analytics, dashboards, or billing systems.
For configuration, see [Reporting Configuration](../configuration/config-practical/reporting-config.md).
+
+## Usage on the API Wire
+
+API clients (the `/execute` response and the durable `tokens_usages.json` result artifact) receive token usage in a trimmed, client-facing record shape with a computed per-call `cost` — not the full internal usage models this page describes. See [TokensUsage Wire Records](../under-the-hood/tokens-usage-wire-records.md).
diff --git a/docs/under-the-hood/tokens-usage-wire-records.md b/docs/under-the-hood/tokens-usage-wire-records.md
new file mode 100644
index 000000000..1d2488cef
--- /dev/null
+++ b/docs/under-the-hood/tokens-usage-wire-records.md
@@ -0,0 +1,68 @@
+---
+title: TokensUsage Wire Records
+description: "The client-facing wire shape of tokens_usages: what a Pipelex API consumer receives on the /execute response and the durable tokens_usages.json artifact."
+---
+
+# TokensUsage Wire Records
+
+`TokensUsageRecord` (`pipelex/reporting/usage_records.py`) is the ONLY shape of a run's token usage that crosses the client boundary. Internally, the runtime tracks usage with full-fidelity models (`AnyTokensUsage` carrying a `JobMetadata`); those internals never reach a client — the converter trims them at the two terminal emission points.
+
+## The two surfaces
+
+Both surfaces carry the same record shape:
+
+- **Blocking `/execute` response** — `pipe_output.tokens_usages` on the response body. The API server dumps its full response, then calls `apply_tokens_usage_wire_shape(response_dump, pipe_output=...)` to replace the usage dumps with wire records.
+- **Durable `tokens_usages.json` result artifact** — written by the delivery executor as `{"tokens_usages": [...] | null, "usage_assembly_error": string | null}` and relayed verbatim by the platform's runs-results endpoint.
+
+## Record fields
+
+One record per inference call, in completion order:
+
+| field | type | semantics |
+|---|---|---|
+| `model_type` | string | Kind of inference: known values `llm`, `img_gen`, `extract`, `search`. Open set. |
+| `inference_model_name` | string | Human model name (e.g. `gpt-4o`). |
+| `inference_model_id` | string | Provider/platform model id (e.g. `gpt-4o-2024-11-20`). |
+| `pipe_code` | string \| null | Pipe that made the call — per-pipe cost attribution. |
+| `job_category` | string \| null | Known values: `llm_job`, `img_gen_job`, `extract_job`, `search_job`, `jinja2_job`, `mock_job`. Open set. |
+| `unit_job_id` | string \| null | Known values: `llm_gen_text`, `llm_gen_object`, `img_gen_text_to_image`, `extract_pages`, `search_sourced_answer`, `search_structured`. Open set. |
+| `nb_tokens_by_category` | object<string, int> | Raw provider-reported token counts, keyed by `TokenCategory` values (`input`, `input_cached`, `output`, ...). Open set. `input` is the joined total; `input_cached` is a subset of it, not additive. |
+| `cost` | number \| null | Computed USD cost of this call. `null` when the model has no rate table (own-GPU, mock, dry-run). |
+| `started_at` | string \| null | ISO 8601. |
+| `completed_at` | string \| null | ISO 8601. Duration is client-derivable; not shipped. |
+
+Enum-ish fields are **open sets** on the wire: consumers must treat them as plain strings, so runtime enum churn is non-breaking.
+
+## What never crosses the wire
+
+The converter deliberately drops runtime plumbing and the rate table: `job_metadata` (the object and any nesting of it), `unit_costs`, `user_id`, `session_id`, `request_id`, `pipe_run_id`, `otel_context`, `trace_context`, `content_generation_job_id`, and `pipeline_run_id` (redundant — the envelope already identifies the run). `TokensUsageRecord` is `extra="forbid"` server-side so an accidental new field fails loudly instead of silently widening the contract.
+
+The record shape is also invariant to server-side observability settings: telemetry/tracing on or off, a consumer always gets structurally identical records.
+
+## Cost semantics
+
+`cost` is computed by `compute_tokens_usage_cost` (`pipelex/cogt/usage/cost_registry.py`) from the rates stamped on the internal record at inference time — the same cost engine and the same canonical total as the CLI cost table: input_non_cached + input_cached + output component costs, with the cached-discount fallback (cached input at 50% of the input rate when no explicit cached rate exists). Categories the cost engine excludes from totals (audio, reasoning, prediction) are excluded here too. A run-level total is the sum of per-record `cost` values.
+
+`cost: null` means "unrated" (no rate table: own-GPU models, mock/dry runs) — distinct from `0.0`, which is a rated call that cost nothing.
+
+## Null semantics
+
+- `tokens_usages: null` — usage assembly was off for the run (or, hosted, the run was delivered before the artifact existed, in which case the durable file is absent entirely).
+- `tokens_usages: []` — assembly on, no inference happened.
+- `usage_assembly_error` non-null — assembly *broke* (event-read failure), distinct from "off". It is a bare diagnostic string, mirroring `graph_assembly_error`.
+
+## Where the trim happens (and where it must NOT)
+
+The trim runs only at the terminal, client-facing emitters:
+
+1. `DeliveryExecutor._generate_usage_file` (durable artifact write, worker-side).
+2. `apply_tokens_usage_wire_shape` on the `/execute` response dump (called by the API server).
+
+Every internal crossing keeps full fidelity: the runtime-bridge `tokens_usages_dump` (rehydrated back into typed records by the host), `PipeOutput` over the Temporal kajson wire, and the usage-event telemetry stream (NDJSON / event log), which keeps emitting full records including `job_metadata` — that stream is operator-side observability, never returned to clients.
+
+For this reason the trim must never be implemented as a `field_serializer`/`model_serializer` on `PipeOutput`: it would silently trim internal transport too.
+
+## Related pages
+
+- [Cost Tracking & Reporting](../features/cost-tracking.md) — the in-process cost table and CSV export (full internal records, unaffected by this wire contract).
+- [Execution Graph Tracing](execution-graph-tracing.md) — the sibling `graph_spec` artifact and the trace event streams.
diff --git a/mkdocs.yml b/mkdocs.yml
index 3c075bee2..2e6190376 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -311,6 +311,7 @@ plugins:
- under-the-hood/codegen-projections.md: "Codegen Projections"
- under-the-hood/dry-run-mock-generation.md: "Dry Run Mock Generation"
- under-the-hood/execution-graph-tracing.md: "Execution Graph Tracing"
+ - under-the-hood/tokens-usage-wire-records.md: "TokensUsage Wire Records"
- under-the-hood/image-handling-in-llm-prompts.md: "Image Handling in LLM Prompts"
- under-the-hood/reasoning-controls.md: "Reasoning Controls"
- under-the-hood/stuffartefact-and-image-rendering.md: "StuffArtefact & Image Rendering"
diff --git a/pipelex/cogt/usage/cost_registry.py b/pipelex/cogt/usage/cost_registry.py
index bfcb929ae..d11e511d7 100644
--- a/pipelex/cogt/usage/cost_registry.py
+++ b/pipelex/cogt/usage/cost_registry.py
@@ -24,6 +24,41 @@ TokenCostReport = LLMTokenCostReport | ImgGenTokenCostReport | ExtractTokenCostR
CostRegistryRoot = list[TokenCostReport]
+def compute_tokens_usage_cost(tokens_usage: TokensUsage) -> float | None:
+ """Compute the canonical USD cost of a single inference call, or None when unrated.
+
+ Returns ``None`` when the usage carries no rate table (``unit_costs`` is empty:
+ own-GPU models, dry/mock runs). Otherwise returns the same canonical total the cost
+ table reports for the call — input_non_cached + input_cached + output component
+ costs, with the cached-discount fallback from ``model_cost_per_token``. Categories
+ the cost engine excludes from totals (audio, reasoning, prediction) are excluded
+ here too: one cost engine, one total.
+ """
+ if not tokens_usage.unit_costs:
+ return None
+ nb_tokens_input_joined = tokens_usage.nb_tokens_by_category.get(TokenCategory.INPUT, 0)
+ nb_tokens_input_cached = tokens_usage.nb_tokens_by_category.get(TokenCategory.INPUT_CACHED, 0)
+ nb_tokens_input_non_cached = nb_tokens_input_joined - nb_tokens_input_cached
+ nb_tokens_output = tokens_usage.nb_tokens_by_category.get(TokenCategory.OUTPUT, 0)
+ input_non_cached_cost = nb_tokens_input_non_cached * model_cost_per_token(
+ costs=tokens_usage.unit_costs,
+ cost_category=CostCategory.INPUT_NON_CACHED,
+ )
+ input_cached_cost = nb_tokens_input_cached * model_cost_per_token(
+ costs=tokens_usage.unit_costs,
+ cost_category=CostCategory.INPUT_CACHED,
+ )
+ output_cost = nb_tokens_output * model_cost_per_token(
+ costs=tokens_usage.unit_costs,
+ cost_category=CostCategory.OUTPUT,
+ )
+ return CostRegistry.compute_total_cost(
+ input_non_cached_cost=input_non_cached_cost,
+ input_cached_cost=input_cached_cost,
+ output_cost=output_cost,
+ )
+
+
class AggregatedCosts(NamedTuple):
"""One run's token usage aggregated for reporting: flat records, per-model groups, and run totals.
diff --git a/pipelex/pipe_run/delivery_executor.py b/pipelex/pipe_run/delivery_executor.py
index 74c9f88a8..0350399da 100644
--- a/pipelex/pipe_run/delivery_executor.py
+++ b/pipelex/pipe_run/delivery_executor.py
@@ -20,6 +20,7 @@ from pipelex.core.stuffs.stuff_viewer import render_stuff_viewer
from pipelex.graph.graph_factory import generate_graph_outputs
from pipelex.hub import get_class_registry, get_storage_provider
from pipelex.pipe_run.exceptions import PipeJobError, StorageDeliveryError, WebhookDeliveryError
+from pipelex.reporting.usage_records import dump_tokens_usage_records
from pipelex.runtime_bridge.primitives.hydration import hydrate_content
from pipelex.tools.misc.json_utils import clean_json_dumps
from pipelex.tools.network.ssrf_guard import SsrfGuardedTransport
@@ -147,15 +148,13 @@ class DeliveryExecutor:
Written unconditionally, so a durable client polling the result files can tell
"usage assembly was off for this run" (file present, ``tokens_usages`` null) from
- "run delivered before the artifact existed" (file absent). The records use the same
- per-usage ``model_dump(mode="json")`` wire shape the ``/execute`` response carries
- on ``pipe_output.tokens_usages``.
+ "run delivered before the artifact existed" (file absent). The records use the
+ client wire shape (``TokensUsageRecord``) — the same shape the ``/execute``
+ response carries on ``pipe_output.tokens_usages`` — never the internal
+ full-fidelity usage models.
"""
- tokens_usages_dump: list[dict[str, Any]] | None = None
- if pipe_output.tokens_usages is not None:
- tokens_usages_dump = [tokens_usage.model_dump(mode="json") for tokens_usage in pipe_output.tokens_usages]
usage_doc: dict[str, Any] = {
- "tokens_usages": tokens_usages_dump,
+ "tokens_usages": dump_tokens_usage_records(pipe_output.tokens_usages),
"usage_assembly_error": pipe_output.usage_assembly_error,
}
return ResultFile(data=clean_json_dumps(usage_doc, indent=2).encode("utf-8"), content_type="application/json")
diff --git a/pipelex/reporting/usage_records.py b/pipelex/reporting/usage_records.py
new file mode 100644
index 000000000..9d9691ea1
--- /dev/null
+++ b/pipelex/reporting/usage_records.py
@@ -0,0 +1,100 @@
+"""Client-facing wire records for token usage.
+
+``TokensUsageRecord`` is the wire shape of ``tokens_usages`` on the two client-facing
+surfaces: the blocking ``/execute`` response (``pipe_output.tokens_usages``) and the
+durable ``tokens_usages.json`` result artifact. The internal runtime models
+(``AnyTokensUsage`` and its ``JobMetadata``) never cross the client boundary: the
+converter here flattens the kept ``JobMetadata`` fields onto the record and drops the
+runtime plumbing (``otel_context``, ``trace_context``, ``session_id``, ``request_id``,
+``user_id``, ``pipe_run_id``, ``content_generation_job_id``, ``pipeline_run_id``) and
+the rate table (``unit_costs`` — replaced by the computed ``cost``).
+
+The trim happens ONLY at the terminal emission points (the delivery artifact write and
+the execute-response dump). Every internal crossing — the runtime-bridge payloads, the
+Temporal kajson transport, the usage-event telemetry stream — keeps full-fidelity
+records.
+"""
+
+from typing import TYPE_CHECKING, Any, cast
+
+from pydantic import BaseModel, ConfigDict
+
+from pipelex.cogt.usage.cost_registry import compute_tokens_usage_cost
+from pipelex.reporting.reporting_types import AnyTokensUsage
+
+if TYPE_CHECKING:
+ from pipelex.core.pipes.pipe_output import PipeOutput
+
+
+class TokensUsageRecord(BaseModel):
+ """One client-facing usage record per inference call.
+
+ ``extra="forbid"`` is deliberate: server-side emission is the shape authority, so an
+ accidental new field fails loudly here instead of silently widening the contract.
+ Enum-ish fields (``model_type``, ``job_category``, ``unit_job_id``, and the
+ ``nb_tokens_by_category`` keys) are open sets on the wire, typed as plain strings —
+ runtime enum churn is non-breaking for clients.
+ """
+
+ model_config = ConfigDict(extra="forbid")
+
+ model_type: str
+ inference_model_name: str
+ inference_model_id: str
+ pipe_code: str | None = None
+ job_category: str | None = None
+ unit_job_id: str | None = None
+ # Raw provider-reported counts: `input` is the joined total; `input_cached` is a
+ # subset of it, not additive.
+ nb_tokens_by_category: dict[str, int]
+ # Computed USD cost of this call; None when the model has no rate table.
+ cost: float | None = None
+ started_at: str | None = None
+ completed_at: str | None = None
+
+
+def make_tokens_usage_record(tokens_usage: AnyTokensUsage) -> TokensUsageRecord:
+ """Convert an internal usage record to its client-facing wire record.
+
+ Flattens the kept ``JobMetadata`` fields (``pipe_code``, job-kind enums, timing)
+ onto the record, ISO-dumps the timestamps, and computes ``cost`` from the usage's
+ own ``unit_costs`` (which then stay server-side).
+ """
+ job_metadata = tokens_usage.job_metadata
+ return TokensUsageRecord(
+ model_type=tokens_usage.model_type,
+ inference_model_name=tokens_usage.inference_model_name,
+ inference_model_id=tokens_usage.inference_model_id,
+ pipe_code=job_metadata.pipe_code,
+ job_category=job_metadata.job_category,
+ unit_job_id=job_metadata.unit_job_id,
+ # TokenCategory is a StrEnum, so the keys ARE strings — the cast just widens the
+ # key type for the open-set wire field.
+ nb_tokens_by_category=cast("dict[str, int]", dict(tokens_usage.nb_tokens_by_category)),
+ cost=compute_tokens_usage_cost(tokens_usage),
+ started_at=job_metadata.started_at.isoformat() if job_metadata.started_at else None,
+ completed_at=job_metadata.completed_at.isoformat() if job_metadata.completed_at else None,
+ )
+
+
+def dump_tokens_usage_records(tokens_usages: list[AnyTokensUsage] | None) -> list[dict[str, Any]] | None:
+ """JSON-safe wire dumps of a run's usage records, preserving the null semantics.
+
+ ``None`` passes through (usage assembly was off for the run), ``[]`` stays ``[]``
+ (assembly on, no inference happened).
+ """
+ if tokens_usages is None:
+ return None
+ return [make_tokens_usage_record(tokens_usage).model_dump(mode="json") for tokens_usage in tokens_usages]
+
+
+def apply_tokens_usage_wire_shape(response_dump: dict[str, Any], *, pipe_output: "PipeOutput") -> dict[str, Any]:
+ """Replace ``pipe_output.tokens_usages`` in an execute-response dump with wire records.
+
+ Applied by API servers at the terminal emission point, right before the JSON response
+ body is built. Deliberately NOT a ``field_serializer``/``model_serializer`` on
+ ``PipeOutput``: that model also rides internal transport (kajson over the Temporal
+ wire), which must keep full fidelity. Mutates and returns ``response_dump``.
+ """
+ response_dump["pipe_output"]["tokens_usages"] = dump_tokens_usage_records(pipe_output.tokens_usages)
+ return response_dump
diff --git a/pipelex/runtime_bridge/payloads.py b/pipelex/runtime_bridge/payloads.py
index a77fc13db..dc9055f47 100644
--- a/pipelex/runtime_bridge/payloads.py
+++ b/pipelex/runtime_bridge/payloads.py
@@ -60,6 +60,10 @@ class PipelexPipeRunOutput(BaseModel):
# apart from "assembly was off" (a None graph_spec_dump / tokens_usages_dump). tokens_usages_dump
# is the JSON-safe dump of the AnyTokensUsage discriminated union so a host can render the
# end-of-run cost report: None when cost reporting was off, [] when on but no inference happened.
+ # NOTE: this dump is an INTERNAL full-fidelity SPI — the host rehydrates it back into typed
+ # `PipeOutput.tokens_usages` (full `JobMetadata` included). The client wire shape is
+ # `pipelex.reporting.usage_records.TokensUsageRecord`, applied only at the terminal emission
+ # points (the delivery artifact write and the execute-response dump) — never here.
graph_assembly_error: str | None = None
tokens_usages_dump: list[dict[str, Any]] | None = None
usage_assembly_error: str | None = None
diff --git a/pyproject.toml b/pyproject.toml
index 61feb08a5..bb352f4d9 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,6 +1,6 @@
[project]
name = "pipelex"
-version = "0.39.2"
+version = "0.40.0"
description = "Execute composable AI methods declared in the MTHDS open standard"
authors = [{ name = "Evotis S.A.S.", email = "[email protected]" }]
maintainers = [{ name = "Pipelex staff", email = "[email protected]" }]
diff --git a/subject_grants.toml b/subject_grants.toml
index 15eeeb747..a32498ec5 100644
--- a/subject_grants.toml
+++ b/subject_grants.toml
@@ -2320,6 +2320,10 @@ rationale = "Transforms/renders the aggregated; single operand."
param = "records"
rationale = "Cost report op on the records; single operand."
+["pipelex/cogt/usage/cost_registry.py::compute_tokens_usage_cost"]
+param = "tokens_usage"
+rationale = "Verb-object: compute the cost of one usage record; the single obvious operand is the usage."
+
["pipelex/core/bundles/pipe_sorter.py::sort_pipes_by_dependencies"]
param = "pipes"
rationale = "Verb-object: sorts the pipes; single operand."
@@ -5340,6 +5344,18 @@ rationale = "Verb-object: reports the search job; single operand."
param = "inference_job"
rationale = "Protocol method: reports the inference job; single operand across implementations."
+["pipelex/reporting/usage_records.py::apply_tokens_usage_wire_shape"]
+param = "response_dump"
+rationale = "Verb-object: apply the wire shape to an execute-response dump; the dump being reshaped is the subject."
+
+["pipelex/reporting/usage_records.py::dump_tokens_usage_records"]
+param = "tokens_usages"
+rationale = "Verb-object: dump the run's usage list to wire records; the single obvious operand is the usage list."
+
+["pipelex/reporting/usage_records.py::make_tokens_usage_record"]
+param = "tokens_usage"
+rationale = "Factory verb-object: make the wire record for one usage; the single obvious operand is the usage."
+
["pipelex/runtime_bridge/primitives/hydration.py::_hydrate_composite_component"]
param = "raw_value"
rationale = "Single-operand hydration of the raw value."
diff --git a/tests/unit/pipelex/cogt/usage/test_data.py b/tests/unit/pipelex/cogt/usage/test_data.py
new file mode 100644
index 000000000..4bf44683b
--- /dev/null
+++ b/tests/unit/pipelex/cogt/usage/test_data.py
@@ -0,0 +1,108 @@
+from datetime import datetime
+
+from pipelex.cogt.extract.extract_report import ExtractTokensUsage
+from pipelex.cogt.img_gen.img_gen_report import ImgGenTokensUsage
+from pipelex.cogt.llm.llm_report import LLMTokensUsage
+from pipelex.cogt.search.search_report import SearchTokensUsage
+from pipelex.cogt.usage.cost_category import CostCategory, CostsByCategoryDict
+from pipelex.cogt.usage.token_category import NbTokensByCategoryDict, TokenCategory
+from pipelex.pipeline.job_metadata import JobCategory, JobMetadata, UnitJobId
+from pipelex.reporting.reporting_types import AnyTokensUsage
+
+RATED_NB_TOKENS: NbTokensByCategoryDict = {
+ TokenCategory.INPUT: 1000,
+ TokenCategory.INPUT_CACHED: 400,
+ TokenCategory.OUTPUT: 250,
+}
+RATED_UNIT_COSTS: CostsByCategoryDict = {
+ CostCategory.INPUT: 2.5,
+ CostCategory.INPUT_CACHED: 1.25,
+ CostCategory.OUTPUT: 10.0,
+}
+# (600 non-cached × $2.5/M) + (400 cached × $1.25/M) + (250 output × $10/M), written in the
+# exact per-token arithmetic the cost engine performs so equality asserts hold bit-for-bit.
+RATED_EXPECTED_COST = 600 * (2.5 / 1_000_000) + 400 * (1.25 / 1_000_000) + 250 * (10.0 / 1_000_000)
+
+
+class UsageFixtures:
+ """Shared usage fixtures for the wire-record and cost tests.
+
+ The cost-parity test and the wire-record tests read the SAME records, pinning the wire
+ ``cost`` to the ``CostRegistry`` totals so the two presentations of the one cost engine
+ cannot drift apart.
+ """
+
+ @staticmethod
+ def full_job_metadata(*, unit_job_id: UnitJobId, job_category: JobCategory) -> JobMetadata:
+ """A JobMetadata with EVERY optional field populated, so leak tests prove the trim."""
+ return JobMetadata(
+ user_id="user-42",
+ pipeline_run_id="plr-fixture",
+ pipe_code="analyze_contract",
+ session_id="session-abc",
+ request_id="req-123",
+ pipe_run_id="a1b2c3d4e5f60718",
+ content_generation_job_id="cgj-9",
+ unit_job_id=unit_job_id,
+ job_category=job_category,
+ started_at=datetime(2026, 7, 19, 10, 0, 0),
+ completed_at=datetime(2026, 7, 19, 10, 0, 5),
+ )
+
+ @classmethod
+ def llm_usage(cls) -> LLMTokensUsage:
+ return LLMTokensUsage(
+ job_metadata=cls.full_job_metadata(unit_job_id=UnitJobId.LLM_GEN_TEXT, job_category=JobCategory.LLM_JOB),
+ inference_model_name="gpt-4o",
+ inference_model_id="gpt-4o-2024-11-20",
+ nb_tokens_by_category=dict(RATED_NB_TOKENS),
+ unit_costs=dict(RATED_UNIT_COSTS),
+ )
+
+ @classmethod
+ def img_gen_usage(cls) -> ImgGenTokensUsage:
+ return ImgGenTokensUsage(
+ job_metadata=cls.full_job_metadata(unit_job_id=UnitJobId.IMG_GEN_TEXT_TO_IMAGE, job_category=JobCategory.IMG_GEN_JOB),
+ inference_model_name="gpt-image-1",
+ inference_model_id="gpt-image-1-2025",
+ nb_tokens_by_category=dict(RATED_NB_TOKENS),
+ unit_costs=dict(RATED_UNIT_COSTS),
+ )
+
+ @classmethod
+ def extract_usage(cls) -> ExtractTokensUsage:
+ return ExtractTokensUsage(
+ job_metadata=cls.full_job_metadata(unit_job_id=UnitJobId.EXTRACT_PAGES, job_category=JobCategory.EXTRACT_JOB),
+ inference_model_name="mistral-ocr",
+ inference_model_id="mistral-ocr-2505",
+ nb_tokens_by_category=dict(RATED_NB_TOKENS),
+ unit_costs=dict(RATED_UNIT_COSTS),
+ )
+
+ @classmethod
+ def search_usage(cls) -> SearchTokensUsage:
+ return SearchTokensUsage(
+ job_metadata=cls.full_job_metadata(unit_job_id=UnitJobId.SEARCH_SOURCED_ANSWER, job_category=JobCategory.SEARCH_JOB),
+ inference_model_name="sonar-pro",
+ inference_model_id="sonar-pro-2025",
+ nb_tokens_by_category=dict(RATED_NB_TOKENS),
+ unit_costs=dict(RATED_UNIT_COSTS),
+ )
+
+ @classmethod
+ def all_variants(cls) -> list[AnyTokensUsage]:
+ return [cls.llm_usage(), cls.img_gen_usage(), cls.extract_usage(), cls.search_usage()]
+
+ @classmethod
+ def unrated_usage(cls) -> LLMTokensUsage:
+ """A usage with no rate table (own-GPU model, dry/mock run): wire ``cost`` must be None."""
+ usage = cls.llm_usage()
+ usage.unit_costs = {}
+ return usage
+
+ @classmethod
+ def cached_fallback_usage(cls) -> LLMTokensUsage:
+ """A usage with cached tokens but no explicit cached rate: the 50%-of-input fallback applies."""
+ usage = cls.llm_usage()
+ usage.unit_costs = {CostCategory.INPUT: 2.0, CostCategory.OUTPUT: 10.0}
+ return usage
diff --git a/tests/unit/pipelex/cogt/usage/test_usage_cost.py b/tests/unit/pipelex/cogt/usage/test_usage_cost.py
new file mode 100644
index 000000000..cc5f07fdf
--- /dev/null
+++ b/tests/unit/pipelex/cogt/usage/test_usage_cost.py
@@ -0,0 +1,46 @@
+import pytest
+
+from pipelex.cogt.usage.cost_category import CostCategory
+from pipelex.cogt.usage.cost_registry import CostRegistry, compute_tokens_usage_cost
+from pipelex.reporting.reporting_types import AnyTokensUsage
+from tests.unit.pipelex.cogt.usage.test_data import RATED_EXPECTED_COST, UsageFixtures
+
+
+class TestComputeTokensUsageCost:
+ def test_none_when_unrated(self):
+ """No rate table (own-GPU, dry/mock run) means no cost claim — None, not 0.0."""
+ assert compute_tokens_usage_cost(UsageFixtures.unrated_usage()) is None
+
+ def test_rated_cost_value(self):
+ assert compute_tokens_usage_cost(UsageFixtures.llm_usage()) == RATED_EXPECTED_COST
+
+ def test_cached_discount_fallback(self):
+ """Without an explicit cached rate, cached input tokens cost 50% of the input rate."""
+ expected = 600 * (2.0 / 1_000_000) + 400 * (0.5 * 2.0 / 1_000_000) + 250 * (10.0 / 1_000_000)
+ assert compute_tokens_usage_cost(UsageFixtures.cached_fallback_usage()) == expected
+
+ @pytest.mark.parametrize(
+ "tokens_usage",
+ UsageFixtures.all_variants(),
+ ids=["llm", "img_gen", "extract", "search"],
+ )
+ def test_parity_with_cost_registry_per_record(self, tokens_usage: AnyTokensUsage):
+ """The wire cost equals the canonical CostRegistry total for the same record — one cost engine, no drift."""
+ cost_report = CostRegistry.complete_cost_report(tokens_usage=tokens_usage)
+ registry_total = CostRegistry.compute_total_cost(
+ input_non_cached_cost=cost_report.costs_by_token_category[CostCategory.INPUT_NON_CACHED],
+ input_cached_cost=cost_report.costs_by_token_category[CostCategory.INPUT_CACHED],
+ output_cost=cost_report.costs_by_token_category.get(CostCategory.OUTPUT, 0.0),
+ )
+ assert compute_tokens_usage_cost(tokens_usage) == registry_total
+
+ def test_parity_with_aggregate_run_total(self):
+ """Summing per-record wire costs reproduces the CLI cost table's run total."""
+ tokens_usages = UsageFixtures.all_variants()
+ wire_total = 0.0
+ for tokens_usage in tokens_usages:
+ record_cost = compute_tokens_usage_cost(tokens_usage)
+ assert record_cost is not None
+ wire_total += record_cost
+ aggregated = CostRegistry.aggregate_costs(tokens_usages=tokens_usages)
+ assert wire_total == aggregated.total_cost
diff --git a/tests/unit/pipelex/pipe_run/test_delivery_executor.py b/tests/unit/pipelex/pipe_run/test_delivery_executor.py
index 6201c6a66..62b965b41 100644
--- a/tests/unit/pipelex/pipe_run/test_delivery_executor.py
+++ b/tests/unit/pipelex/pipe_run/test_delivery_executor.py
@@ -276,9 +276,9 @@ class TestDeliveryExecutor:
assert "main_stuff.html" in files
async def test_generate_result_files_writes_usage_artifact_with_records(self, mocker: MockerFixture) -> None:
- """The tokens_usages.json artifact carries the run's usage records in the same
- per-record ``model_dump(mode="json")`` wire shape the /execute response uses, so a
- durable client can compute costs from the polled result files alone.
+ """The tokens_usages.json artifact carries the run's usage in the client wire shape
+ (``TokensUsageRecord``): computed ``cost``, no ``unit_costs``, no ``job_metadata`` —
+ so a durable client reads costs from the polled result files alone.
"""
mock_output = _make_output_mock(mocker)
mock_output.working_memory_raw = None
@@ -307,7 +307,9 @@ class TestDeliveryExecutor:
assert record["inference_model_name"] == "test-model"
assert record["inference_model_id"] == "test-model-id"
assert record["nb_tokens_by_category"] == {"input": 15, "output": 4}
- assert record["unit_costs"] == {"input": 3.0, "output": 15.0}
+ assert record["cost"] == 15 * (3.0 / 1_000_000) + 4 * (15.0 / 1_000_000)
+ assert "unit_costs" not in record
+ assert "job_metadata" not in record
async def test_generate_result_files_usage_artifact_null_when_usage_off(self, mocker: MockerFixture) -> None:
"""A run with usage assembly off still writes the artifact with explicit nulls, so a
diff --git a/tests/unit/pipelex/reporting/test_usage_records.py b/tests/unit/pipelex/reporting/test_usage_records.py
new file mode 100644
index 000000000..e84fb90d8
--- /dev/null
+++ b/tests/unit/pipelex/reporting/test_usage_records.py
@@ -0,0 +1,139 @@
+import json
+
+import pytest
+from pytest_mock import MockerFixture
+
+from pipelex.reporting.reporting_types import AnyTokensUsage
+from pipelex.reporting.usage_records import (
+ TokensUsageRecord,
+ apply_tokens_usage_wire_shape,
+ dump_tokens_usage_records,
+ make_tokens_usage_record,
+)
+from tests.unit.pipelex.cogt.usage.test_data import RATED_EXPECTED_COST, UsageFixtures
+
+WIRE_FIELDS = {
+ "model_type",
+ "inference_model_name",
+ "inference_model_id",
+ "pipe_code",
+ "job_category",
+ "unit_job_id",
+ "nb_tokens_by_category",
+ "cost",
+ "started_at",
+ "completed_at",
+}
+
+# The contract's MUST-NOT list: none of these may appear on a wire record, flat or nested.
+MUST_NOT_FIELDS = [
+ "job_metadata",
+ "unit_costs",
+ "user_id",
+ "session_id",
+ "request_id",
+ "pipe_run_id",
+ "otel_context",
+ "trace_context",
+ "content_generation_job_id",
+ "pipeline_run_id",
+]
+
+
+class TestTokensUsageRecords:
+ @pytest.mark.parametrize(
+ ("tokens_usage", "expected_model_type", "expected_job_category", "expected_unit_job_id"),
+ [
+ (UsageFixtures.llm_usage(), "llm", "llm_job", "llm_gen_text"),
+ (UsageFixtures.img_gen_usage(), "img_gen", "img_gen_job", "img_gen_text_to_image"),
+ (UsageFixtures.extract_usage(), "extract", "extract_job", "extract_pages"),
+ (UsageFixtures.search_usage(), "search", "search_job", "search_sourced_answer"),
+ ],
+ ids=["llm", "img_gen", "extract", "search"],
+ )
+ def test_conversion_per_variant(
+ self,
+ tokens_usage: AnyTokensUsage,
+ expected_model_type: str,
+ expected_job_category: str,
+ expected_unit_job_id: str,
+ ):
+ """Every usage variant converts to the same single record shape, JobMetadata fields flattened."""
+ record = make_tokens_usage_record(tokens_usage)
+ assert record.model_type == expected_model_type
+ assert record.inference_model_name == tokens_usage.inference_model_name
+ assert record.inference_model_id == tokens_usage.inference_model_id
+ assert record.pipe_code == "analyze_contract"
+ assert record.job_category == expected_job_category
+ assert record.unit_job_id == expected_unit_job_id
+ assert record.nb_tokens_by_category == {"input": 1000, "input_cached": 400, "output": 250}
+ assert record.cost == RATED_EXPECTED_COST
+ assert record.started_at == "2026-07-19T10:00:00"
+ assert record.completed_at == "2026-07-19T10:00:05"
+
+ def test_cost_is_null_when_unrated(self):
+ record = make_tokens_usage_record(UsageFixtures.unrated_usage())
+ assert record.cost is None
+
+ @pytest.mark.parametrize(
+ "tokens_usage",
+ [*UsageFixtures.all_variants(), UsageFixtures.unrated_usage()],
+ ids=["llm", "img_gen", "extract", "search", "unrated"],
+ )
+ def test_leak_regression(self, tokens_usage: AnyTokensUsage):
+ """The dumped wire record carries the contract fields exactly — none of the MUST-NOT
+ internals (JobMetadata plumbing, rate table), flat or nested.
+ """
+ dumped = make_tokens_usage_record(tokens_usage).model_dump(mode="json")
+ assert set(dumped.keys()) == WIRE_FIELDS
+ dumped_text = json.dumps(dumped)
+ for forbidden_field in MUST_NOT_FIELDS:
+ assert forbidden_field not in dumped, f"MUST-NOT field '{forbidden_field}' leaked onto the wire record"
+ assert f'"{forbidden_field}"' not in dumped_text, f"MUST-NOT field '{forbidden_field}' nested in the wire record"
+
+ def test_dumped_record_is_json_safe_and_stringified(self):
+ """Enums and timestamps dump as plain JSON strings — a consumer needs no runtime types."""
+ dumped = make_tokens_usage_record(UsageFixtures.llm_usage()).model_dump(mode="json")
+ reparsed = json.loads(json.dumps(dumped))
+ assert reparsed["job_category"] == "llm_job"
+ assert reparsed["unit_job_id"] == "llm_gen_text"
+ assert reparsed["nb_tokens_by_category"] == {"input": 1000, "input_cached": 400, "output": 250}
+ assert reparsed["started_at"] == "2026-07-19T10:00:00"
+ # The record round-trips through its own model: the shape is closed (extra="forbid").
+ assert TokensUsageRecord.model_validate(reparsed).model_dump(mode="json") == dumped
+
+ def test_dump_null_and_empty_semantics(self):
+ """None passes through (usage assembly off) and [] stays [] (on, no inference) — unchanged semantics."""
+ assert dump_tokens_usage_records(None) is None
+ assert dump_tokens_usage_records([]) == []
+
+ def test_apply_wire_shape_replaces_execute_dump_records(self, mocker: MockerFixture):
+ """The execute-response dump gets its full-fidelity usage dumps replaced by wire records."""
+ tokens_usages = [UsageFixtures.llm_usage()]
+ pipe_output = mocker.MagicMock()
+ pipe_output.tokens_usages = tokens_usages
+ response_dump = {
+ "pipeline_run_id": "plr-fixture",
+ "pipe_output": {
+ "tokens_usages": [tokens_usage.model_dump(mode="json") for tokens_usage in tokens_usages],
+ "usage_assembly_error": None,
+ },
+ }
+ result = apply_tokens_usage_wire_shape(response_dump, pipe_output=pipe_output)
+ assert result is response_dump
+ records = result["pipe_output"]["tokens_usages"]
+ assert len(records) == 1
+ assert records[0]["model_type"] == "llm"
+ assert records[0]["cost"] == RATED_EXPECTED_COST
+ for forbidden_field in MUST_NOT_FIELDS:
+ assert forbidden_field not in records[0]
+ # The sibling diagnostic stays untouched (D5: bare string contract).
+ assert result["pipe_output"]["usage_assembly_error"] is None
+
+ def test_apply_wire_shape_preserves_none(self, mocker: MockerFixture):
+ """A run with usage assembly off keeps tokens_usages null in the execute response."""
+ pipe_output = mocker.MagicMock()
+ pipe_output.tokens_usages = None
+ response_dump = {"pipe_output": {"tokens_usages": None, "usage_assembly_error": None}}
+ result = apply_tokens_usage_wire_shape(response_dump, pipe_output=pipe_output)
+ assert result["pipe_output"]["tokens_usages"] is None