Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
feat(reporting): TokensUsageRecord — deliberate client wire shape for tokens_usages
Louis Choquel67115cd3ffff
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex c74c6eedb..be873e85c 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -4,7 +4,12 @@### Added-- **Durable runs now persist token usage: the delivery executor writes a `tokens_usages.json` result artifact** (`{"tokens_usages": [...], "usage_assembly_error": null}`) alongside `working_memory.json`, the `main_stuff.*` renders, and the graph outputs. The records use the same per-record `model_dump(mode="json")` wire shape the `/execute` response carries on `pipe_output.tokens_usages` (token counts by category, `unit_costs` in $/1M, model id — for LLM, img-gen, extract, and search calls), so a client polling a durable run's result files can compute costs the same way a sync `/execute` caller does. The artifact is written unconditionally: a run with usage assembly off yields explicit nulls, distinguishable from a pre-artifact run (file absent).+- **Durable runs now persist token usage: the delivery executor writes a `tokens_usages.json` result artifact** (`{"tokens_usages": [...], "usage_assembly_error": null}`) alongside `working_memory.json`, the `main_stuff.*` renders, and the graph outputs. The records use the client-facing `TokensUsageRecord` wire shape — the same shape the `/execute` response carries on `pipe_output.tokens_usages` — so a client polling a durable run's result files reads costs the same way a sync `/execute` caller does. The artifact is written unconditionally: a run with usage assembly off yields explicit nulls, distinguishable from a pre-artifact run (file absent).+- **`TokensUsageRecord` — a deliberate client wire shape for token usage** (`pipelex/reporting/usage_records.py`). One flat record per inference call: `model_type`, `inference_model_name`, `inference_model_id`, `pipe_code`, `job_category`, `unit_job_id`, `nb_tokens_by_category` (raw provider counts), computed USD `cost` (`null` when the model has no rate table), and ISO `started_at`/`completed_at`. Enum-ish fields are open string sets on the wire, so runtime enum churn is non-breaking for clients. `compute_tokens_usage_cost` reuses the existing cost engine (`model_cost_per_token` + the cached/non-cached split), so the wire `cost` equals the CLI cost table's total for the same call. Documented in [TokensUsage Wire Records](docs/under-the-hood/tokens-usage-wire-records.md).++### Changed++- **Breaking: `tokens_usages` records on client surfaces are now `TokensUsageRecord` wire records, not dumps of the internal usage models.** Both the `/execute` response (`pipe_output.tokens_usages`, via the new `apply_tokens_usage_wire_shape` helper API servers apply to the response dump) and the durable `tokens_usages.json` artifact stop leaking runtime internals: `job_metadata` (with its `user_id`, `session_id`, `request_id`, `pipe_run_id`, `otel_context`, `trace_context`, `content_generation_job_id`, `pipeline_run_id`) and the `unit_costs` rate table no longer cross the wire — the kept `JobMetadata` fields (`pipe_code`, job-kind enums, timing) are flattened onto the record and `cost` is computed server-side. Null semantics unchanged (`null` = assembly off, `[]` = no inference). Internal crossings (runtime-bridge payloads, Temporal transport, the usage-event telemetry stream) keep full-fidelity records, untouched.### Fixeddiff --git a/docs/features/cost-tracking.md b/docs/features/cost-tracking.mdindex ce29aa182..d863c90ec 100644--- a/docs/features/cost-tracking.md+++ b/docs/features/cost-tracking.md@@ -36,3 +36,7 @@ The `--costs` gate is read off the output itself — `tokens_usages` is `None` eImplement custom reporting backends via the [Reporting Delegate](advanced-customizations.md) injection point. The reporting protocol receives structured cost events during execution, allowing you to push data to your own analytics, dashboards, or billing systems.For configuration, see [Reporting Configuration](../configuration/config-practical/reporting-config.md).++## Usage on the API Wire++API clients (the `/execute` response and the durable `tokens_usages.json` result artifact) receive token usage in a trimmed, client-facing record shape with a computed per-call `cost` — not the full internal usage models this page describes. See [TokensUsage Wire Records](../under-the-hood/tokens-usage-wire-records.md).diff --git a/docs/under-the-hood/tokens-usage-wire-records.md b/docs/under-the-hood/tokens-usage-wire-records.mdnew file mode 100644index 000000000..1d2488cef--- /dev/null+++ b/docs/under-the-hood/tokens-usage-wire-records.md@@ -0,0 +1,68 @@+---+title: TokensUsage Wire Records+description: "The client-facing wire shape of tokens_usages: what a Pipelex API consumer receives on the /execute response and the durable tokens_usages.json artifact."+---++# TokensUsage Wire Records++`TokensUsageRecord` (`pipelex/reporting/usage_records.py`) is the ONLY shape of a run's token usage that crosses the client boundary. Internally, the runtime tracks usage with full-fidelity models (`AnyTokensUsage` carrying a `JobMetadata`); those internals never reach a client — the converter trims them at the two terminal emission points.++## The two surfaces++Both surfaces carry the same record shape:++- **Blocking `/execute` response** — `pipe_output.tokens_usages` on the response body. The API server dumps its full response, then calls `apply_tokens_usage_wire_shape(response_dump, pipe_output=...)` to replace the usage dumps with wire records.+- **Durable `tokens_usages.json` result artifact** — written by the delivery executor as `{"tokens_usages": [...] | null, "usage_assembly_error": string | null}` and relayed verbatim by the platform's runs-results endpoint.++## Record fields++One record per inference call, in completion order:++| field | type | semantics |+|---|---|---|+| `model_type` | string | Kind of inference: known values `llm`, `img_gen`, `extract`, `search`. Open set. |+| `inference_model_name` | string | Human model name (e.g. `gpt-4o`). |+| `inference_model_id` | string | Provider/platform model id (e.g. `gpt-4o-2024-11-20`). |+| `pipe_code` | string \| null | Pipe that made the call — per-pipe cost attribution. |+| `job_category` | string \| null | Known values: `llm_job`, `img_gen_job`, `extract_job`, `search_job`, `jinja2_job`, `mock_job`. Open set. |+| `unit_job_id` | string \| null | Known values: `llm_gen_text`, `llm_gen_object`, `img_gen_text_to_image`, `extract_pages`, `search_sourced_answer`, `search_structured`. Open set. |+| `nb_tokens_by_category` | object<string, int> | Raw provider-reported token counts, keyed by `TokenCategory` values (`input`, `input_cached`, `output`, ...). Open set. `input` is the joined total; `input_cached` is a subset of it, not additive. |+| `cost` | number \| null | Computed USD cost of this call. `null` when the model has no rate table (own-GPU, mock, dry-run). |+| `started_at` | string \| null | ISO 8601. |+| `completed_at` | string \| null | ISO 8601. Duration is client-derivable; not shipped. |++Enum-ish fields are **open sets** on the wire: consumers must treat them as plain strings, so runtime enum churn is non-breaking.++## What never crosses the wire++The converter deliberately drops runtime plumbing and the rate table: `job_metadata` (the object and any nesting of it), `unit_costs`, `user_id`, `session_id`, `request_id`, `pipe_run_id`, `otel_context`, `trace_context`, `content_generation_job_id`, and `pipeline_run_id` (redundant — the envelope already identifies the run). `TokensUsageRecord` is `extra="forbid"` server-side so an accidental new field fails loudly instead of silently widening the contract.++The record shape is also invariant to server-side observability settings: telemetry/tracing on or off, a consumer always gets structurally identical records.++## Cost semantics++`cost` is computed by `compute_tokens_usage_cost` (`pipelex/cogt/usage/cost_registry.py`) from the rates stamped on the internal record at inference time — the same cost engine and the same canonical total as the CLI cost table: input_non_cached + input_cached + output component costs, with the cached-discount fallback (cached input at 50% of the input rate when no explicit cached rate exists). Categories the cost engine excludes from totals (audio, reasoning, prediction) are excluded here too. A run-level total is the sum of per-record `cost` values.++`cost: null` means "unrated" (no rate table: own-GPU models, mock/dry runs) — distinct from `0.0`, which is a rated call that cost nothing.++## Null semantics++- `tokens_usages: null` — usage assembly was off for the run (or, hosted, the run was delivered before the artifact existed, in which case the durable file is absent entirely).+- `tokens_usages: []` — assembly on, no inference happened.+- `usage_assembly_error` non-null — assembly *broke* (event-read failure), distinct from "off". It is a bare diagnostic string, mirroring `graph_assembly_error`.++## Where the trim happens (and where it must NOT)++The trim runs only at the terminal, client-facing emitters:++1. `DeliveryExecutor._generate_usage_file` (durable artifact write, worker-side).+2. `apply_tokens_usage_wire_shape` on the `/execute` response dump (called by the API server).++Every internal crossing keeps full fidelity: the runtime-bridge `tokens_usages_dump` (rehydrated back into typed records by the host), `PipeOutput` over the Temporal kajson wire, and the usage-event telemetry stream (NDJSON / event log), which keeps emitting full records including `job_metadata` — that stream is operator-side observability, never returned to clients.++For this reason the trim must never be implemented as a `field_serializer`/`model_serializer` on `PipeOutput`: it would silently trim internal transport too.++## Related pages++- [Cost Tracking & Reporting](../features/cost-tracking.md) — the in-process cost table and CSV export (full internal records, unaffected by this wire contract).+- [Execution Graph Tracing](execution-graph-tracing.md) — the sibling `graph_spec` artifact and the trace event streams.diff --git a/mkdocs.yml b/mkdocs.ymlindex 3c075bee2..2e6190376 100644--- a/mkdocs.yml+++ b/mkdocs.yml@@ -311,6 +311,7 @@ plugins:- under-the-hood/codegen-projections.md: "Codegen Projections"- under-the-hood/dry-run-mock-generation.md: "Dry Run Mock Generation"- under-the-hood/execution-graph-tracing.md: "Execution Graph Tracing"+ - under-the-hood/tokens-usage-wire-records.md: "TokensUsage Wire Records"- under-the-hood/image-handling-in-llm-prompts.md: "Image Handling in LLM Prompts"- under-the-hood/reasoning-controls.md: "Reasoning Controls"- under-the-hood/stuffartefact-and-image-rendering.md: "StuffArtefact & Image Rendering"diff --git a/pipelex/cogt/usage/cost_registry.py b/pipelex/cogt/usage/cost_registry.pyindex bfcb929ae..d11e511d7 100644--- a/pipelex/cogt/usage/cost_registry.py+++ b/pipelex/cogt/usage/cost_registry.py@@ -24,6 +24,41 @@ TokenCostReport = LLMTokenCostReport | ImgGenTokenCostReport | ExtractTokenCostRCostRegistryRoot = list[TokenCostReport]+def compute_tokens_usage_cost(tokens_usage: TokensUsage) -> float | None:+ """Compute the canonical USD cost of a single inference call, or None when unrated.++ Returns ``None`` when the usage carries no rate table (``unit_costs`` is empty:+ own-GPU models, dry/mock runs). Otherwise returns the same canonical total the cost+ table reports for the call — input_non_cached + input_cached + output component+ costs, with the cached-discount fallback from ``model_cost_per_token``. Categories+ the cost engine excludes from totals (audio, reasoning, prediction) are excluded+ here too: one cost engine, one total.+ """+ if not tokens_usage.unit_costs:+ return None+ nb_tokens_input_joined = tokens_usage.nb_tokens_by_category.get(TokenCategory.INPUT, 0)+ nb_tokens_input_cached = tokens_usage.nb_tokens_by_category.get(TokenCategory.INPUT_CACHED, 0)+ nb_tokens_input_non_cached = nb_tokens_input_joined - nb_tokens_input_cached+ nb_tokens_output = tokens_usage.nb_tokens_by_category.get(TokenCategory.OUTPUT, 0)+ input_non_cached_cost = nb_tokens_input_non_cached * model_cost_per_token(+ costs=tokens_usage.unit_costs,+ cost_category=CostCategory.INPUT_NON_CACHED,+ )+ input_cached_cost = nb_tokens_input_cached * model_cost_per_token(+ costs=tokens_usage.unit_costs,+ cost_category=CostCategory.INPUT_CACHED,+ )+ output_cost = nb_tokens_output * model_cost_per_token(+ costs=tokens_usage.unit_costs,+ cost_category=CostCategory.OUTPUT,+ )+ return CostRegistry.compute_total_cost(+ input_non_cached_cost=input_non_cached_cost,+ input_cached_cost=input_cached_cost,+ output_cost=output_cost,+ )++class AggregatedCosts(NamedTuple):"""One run's token usage aggregated for reporting: flat records, per-model groups, and run totals.diff --git a/pipelex/pipe_run/delivery_executor.py b/pipelex/pipe_run/delivery_executor.pyindex 74c9f88a8..0350399da 100644--- a/pipelex/pipe_run/delivery_executor.py+++ b/pipelex/pipe_run/delivery_executor.py@@ -20,6 +20,7 @@ from pipelex.core.stuffs.stuff_viewer import render_stuff_viewerfrom pipelex.graph.graph_factory import generate_graph_outputsfrom pipelex.hub import get_class_registry, get_storage_providerfrom pipelex.pipe_run.exceptions import PipeJobError, StorageDeliveryError, WebhookDeliveryError+from pipelex.reporting.usage_records import dump_tokens_usage_recordsfrom pipelex.runtime_bridge.primitives.hydration import hydrate_contentfrom pipelex.tools.misc.json_utils import clean_json_dumpsfrom pipelex.tools.network.ssrf_guard import SsrfGuardedTransport@@ -147,15 +148,13 @@ class DeliveryExecutor:Written unconditionally, so a durable client polling the result files can tell"usage assembly was off for this run" (file present, ``tokens_usages`` null) from- "run delivered before the artifact existed" (file absent). The records use the same- per-usage ``model_dump(mode="json")`` wire shape the ``/execute`` response carries- on ``pipe_output.tokens_usages``.+ "run delivered before the artifact existed" (file absent). The records use the+ client wire shape (``TokensUsageRecord``) — the same shape the ``/execute``+ response carries on ``pipe_output.tokens_usages`` — never the internal+ full-fidelity usage models."""- tokens_usages_dump: list[dict[str, Any]] | None = None- if pipe_output.tokens_usages is not None:- tokens_usages_dump = [tokens_usage.model_dump(mode="json") for tokens_usage in pipe_output.tokens_usages]usage_doc: dict[str, Any] = {- "tokens_usages": tokens_usages_dump,+ "tokens_usages": dump_tokens_usage_records(pipe_output.tokens_usages),"usage_assembly_error": pipe_output.usage_assembly_error,}return ResultFile(data=clean_json_dumps(usage_doc, indent=2).encode("utf-8"), content_type="application/json")diff --git a/pipelex/reporting/usage_records.py b/pipelex/reporting/usage_records.pynew file mode 100644index 000000000..9d9691ea1--- /dev/null+++ b/pipelex/reporting/usage_records.py@@ -0,0 +1,100 @@+"""Client-facing wire records for token usage.++``TokensUsageRecord`` is the wire shape of ``tokens_usages`` on the two client-facing+surfaces: the blocking ``/execute`` response (``pipe_output.tokens_usages``) and the+durable ``tokens_usages.json`` result artifact. The internal runtime models+(``AnyTokensUsage`` and its ``JobMetadata``) never cross the client boundary: the+converter here flattens the kept ``JobMetadata`` fields onto the record and drops the+runtime plumbing (``otel_context``, ``trace_context``, ``session_id``, ``request_id``,+``user_id``, ``pipe_run_id``, ``content_generation_job_id``, ``pipeline_run_id``) and+the rate table (``unit_costs`` — replaced by the computed ``cost``).++The trim happens ONLY at the terminal emission points (the delivery artifact write and+the execute-response dump). Every internal crossing — the runtime-bridge payloads, the+Temporal kajson transport, the usage-event telemetry stream — keeps full-fidelity+records.+"""++from typing import TYPE_CHECKING, Any, cast++from pydantic import BaseModel, ConfigDict++from pipelex.cogt.usage.cost_registry import compute_tokens_usage_cost+from pipelex.reporting.reporting_types import AnyTokensUsage++if TYPE_CHECKING:+ from pipelex.core.pipes.pipe_output import PipeOutput+++class TokensUsageRecord(BaseModel):+ """One client-facing usage record per inference call.++ ``extra="forbid"`` is deliberate: server-side emission is the shape authority, so an+ accidental new field fails loudly here instead of silently widening the contract.+ Enum-ish fields (``model_type``, ``job_category``, ``unit_job_id``, and the+ ``nb_tokens_by_category`` keys) are open sets on the wire, typed as plain strings —+ runtime enum churn is non-breaking for clients.+ """++ model_config = ConfigDict(extra="forbid")++ model_type: str+ inference_model_name: str+ inference_model_id: str+ pipe_code: str | None = None+ job_category: str | None = None+ unit_job_id: str | None = None+ # Raw provider-reported counts: `input` is the joined total; `input_cached` is a+ # subset of it, not additive.+ nb_tokens_by_category: dict[str, int]+ # Computed USD cost of this call; None when the model has no rate table.+ cost: float | None = None+ started_at: str | None = None+ completed_at: str | None = None+++def make_tokens_usage_record(tokens_usage: AnyTokensUsage) -> TokensUsageRecord:+ """Convert an internal usage record to its client-facing wire record.++ Flattens the kept ``JobMetadata`` fields (``pipe_code``, job-kind enums, timing)+ onto the record, ISO-dumps the timestamps, and computes ``cost`` from the usage's+ own ``unit_costs`` (which then stay server-side).+ """+ job_metadata = tokens_usage.job_metadata+ return TokensUsageRecord(+ model_type=tokens_usage.model_type,+ inference_model_name=tokens_usage.inference_model_name,+ inference_model_id=tokens_usage.inference_model_id,+ pipe_code=job_metadata.pipe_code,+ job_category=job_metadata.job_category,+ unit_job_id=job_metadata.unit_job_id,+ # TokenCategory is a StrEnum, so the keys ARE strings — the cast just widens the+ # key type for the open-set wire field.+ nb_tokens_by_category=cast("dict[str, int]", dict(tokens_usage.nb_tokens_by_category)),+ cost=compute_tokens_usage_cost(tokens_usage),+ started_at=job_metadata.started_at.isoformat() if job_metadata.started_at else None,+ completed_at=job_metadata.completed_at.isoformat() if job_metadata.completed_at else None,+ )+++def dump_tokens_usage_records(tokens_usages: list[AnyTokensUsage] | None) -> list[dict[str, Any]] | None:+ """JSON-safe wire dumps of a run's usage records, preserving the null semantics.++ ``None`` passes through (usage assembly was off for the run), ``[]`` stays ``[]``+ (assembly on, no inference happened).+ """+ if tokens_usages is None:+ return None+ return [make_tokens_usage_record(tokens_usage).model_dump(mode="json") for tokens_usage in tokens_usages]+++def apply_tokens_usage_wire_shape(response_dump: dict[str, Any], *, pipe_output: "PipeOutput") -> dict[str, Any]:+ """Replace ``pipe_output.tokens_usages`` in an execute-response dump with wire records.++ Applied by API servers at the terminal emission point, right before the JSON response+ body is built. Deliberately NOT a ``field_serializer``/``model_serializer`` on+ ``PipeOutput``: that model also rides internal transport (kajson over the Temporal+ wire), which must keep full fidelity. Mutates and returns ``response_dump``.+ """+ response_dump["pipe_output"]["tokens_usages"] = dump_tokens_usage_records(pipe_output.tokens_usages)+ return response_dumpdiff --git a/pipelex/runtime_bridge/payloads.py b/pipelex/runtime_bridge/payloads.pyindex a77fc13db..dc9055f47 100644--- a/pipelex/runtime_bridge/payloads.py+++ b/pipelex/runtime_bridge/payloads.py@@ -60,6 +60,10 @@ class PipelexPipeRunOutput(BaseModel):# apart from "assembly was off" (a None graph_spec_dump / tokens_usages_dump). tokens_usages_dump# is the JSON-safe dump of the AnyTokensUsage discriminated union so a host can render the# end-of-run cost report: None when cost reporting was off, [] when on but no inference happened.+ # NOTE: this dump is an INTERNAL full-fidelity SPI — the host rehydrates it back into typed+ # `PipeOutput.tokens_usages` (full `JobMetadata` included). The client wire shape is+ # `pipelex.reporting.usage_records.TokensUsageRecord`, applied only at the terminal emission+ # points (the delivery artifact write and the execute-response dump) — never here.graph_assembly_error: str | None = Nonetokens_usages_dump: list[dict[str, Any]] | None = Noneusage_assembly_error: str | None = Nonediff --git a/pyproject.toml b/pyproject.tomlindex 61feb08a5..bb352f4d9 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -1,6 +1,6 @@[project]name = "pipelex"-version = "0.39.2"+version = "0.40.0"description = "Execute composable AI methods declared in the MTHDS open standard"authors = [{ name = "Evotis S.A.S.", email = "[email protected]" }]maintainers = [{ name = "Pipelex staff", email = "[email protected]" }]diff --git a/subject_grants.toml b/subject_grants.tomlindex 15eeeb747..a32498ec5 100644--- a/subject_grants.toml+++ b/subject_grants.toml@@ -2320,6 +2320,10 @@ rationale = "Transforms/renders the aggregated; single operand."param = "records"rationale = "Cost report op on the records; single operand."+["pipelex/cogt/usage/cost_registry.py::compute_tokens_usage_cost"]+param = "tokens_usage"+rationale = "Verb-object: compute the cost of one usage record; the single obvious operand is the usage."+["pipelex/core/bundles/pipe_sorter.py::sort_pipes_by_dependencies"]param = "pipes"rationale = "Verb-object: sorts the pipes; single operand."@@ -5340,6 +5344,18 @@ rationale = "Verb-object: reports the search job; single operand."param = "inference_job"rationale = "Protocol method: reports the inference job; single operand across implementations."+["pipelex/reporting/usage_records.py::apply_tokens_usage_wire_shape"]+param = "response_dump"+rationale = "Verb-object: apply the wire shape to an execute-response dump; the dump being reshaped is the subject."++["pipelex/reporting/usage_records.py::dump_tokens_usage_records"]+param = "tokens_usages"+rationale = "Verb-object: dump the run's usage list to wire records; the single obvious operand is the usage list."++["pipelex/reporting/usage_records.py::make_tokens_usage_record"]+param = "tokens_usage"+rationale = "Factory verb-object: make the wire record for one usage; the single obvious operand is the usage."+["pipelex/runtime_bridge/primitives/hydration.py::_hydrate_composite_component"]param = "raw_value"rationale = "Single-operand hydration of the raw value."diff --git a/tests/unit/pipelex/cogt/usage/test_data.py b/tests/unit/pipelex/cogt/usage/test_data.pynew file mode 100644index 000000000..4bf44683b--- /dev/null+++ b/tests/unit/pipelex/cogt/usage/test_data.py@@ -0,0 +1,108 @@+from datetime import datetime++from pipelex.cogt.extract.extract_report import ExtractTokensUsage+from pipelex.cogt.img_gen.img_gen_report import ImgGenTokensUsage+from pipelex.cogt.llm.llm_report import LLMTokensUsage+from pipelex.cogt.search.search_report import SearchTokensUsage+from pipelex.cogt.usage.cost_category import CostCategory, CostsByCategoryDict+from pipelex.cogt.usage.token_category import NbTokensByCategoryDict, TokenCategory+from pipelex.pipeline.job_metadata import JobCategory, JobMetadata, UnitJobId+from pipelex.reporting.reporting_types import AnyTokensUsage++RATED_NB_TOKENS: NbTokensByCategoryDict = {+ TokenCategory.INPUT: 1000,+ TokenCategory.INPUT_CACHED: 400,+ TokenCategory.OUTPUT: 250,+}+RATED_UNIT_COSTS: CostsByCategoryDict = {+ CostCategory.INPUT: 2.5,+ CostCategory.INPUT_CACHED: 1.25,+ CostCategory.OUTPUT: 10.0,+}+# (600 non-cached × $2.5/M) + (400 cached × $1.25/M) + (250 output × $10/M), written in the+# exact per-token arithmetic the cost engine performs so equality asserts hold bit-for-bit.+RATED_EXPECTED_COST = 600 * (2.5 / 1_000_000) + 400 * (1.25 / 1_000_000) + 250 * (10.0 / 1_000_000)+++class UsageFixtures:+ """Shared usage fixtures for the wire-record and cost tests.++ The cost-parity test and the wire-record tests read the SAME records, pinning the wire+ ``cost`` to the ``CostRegistry`` totals so the two presentations of the one cost engine+ cannot drift apart.+ """++ @staticmethod+ def full_job_metadata(*, unit_job_id: UnitJobId, job_category: JobCategory) -> JobMetadata:+ """A JobMetadata with EVERY optional field populated, so leak tests prove the trim."""+ return JobMetadata(+ user_id="user-42",+ pipeline_run_id="plr-fixture",+ pipe_code="analyze_contract",+ session_id="session-abc",+ request_id="req-123",+ pipe_run_id="a1b2c3d4e5f60718",+ content_generation_job_id="cgj-9",+ unit_job_id=unit_job_id,+ job_category=job_category,+ started_at=datetime(2026, 7, 19, 10, 0, 0),+ completed_at=datetime(2026, 7, 19, 10, 0, 5),+ )++ @classmethod+ def llm_usage(cls) -> LLMTokensUsage:+ return LLMTokensUsage(+ job_metadata=cls.full_job_metadata(unit_job_id=UnitJobId.LLM_GEN_TEXT, job_category=JobCategory.LLM_JOB),+ inference_model_name="gpt-4o",+ inference_model_id="gpt-4o-2024-11-20",+ nb_tokens_by_category=dict(RATED_NB_TOKENS),+ unit_costs=dict(RATED_UNIT_COSTS),+ )++ @classmethod+ def img_gen_usage(cls) -> ImgGenTokensUsage:+ return ImgGenTokensUsage(+ job_metadata=cls.full_job_metadata(unit_job_id=UnitJobId.IMG_GEN_TEXT_TO_IMAGE, job_category=JobCategory.IMG_GEN_JOB),+ inference_model_name="gpt-image-1",+ inference_model_id="gpt-image-1-2025",+ nb_tokens_by_category=dict(RATED_NB_TOKENS),+ unit_costs=dict(RATED_UNIT_COSTS),+ )++ @classmethod+ def extract_usage(cls) -> ExtractTokensUsage:+ return ExtractTokensUsage(+ job_metadata=cls.full_job_metadata(unit_job_id=UnitJobId.EXTRACT_PAGES, job_category=JobCategory.EXTRACT_JOB),+ inference_model_name="mistral-ocr",+ inference_model_id="mistral-ocr-2505",+ nb_tokens_by_category=dict(RATED_NB_TOKENS),+ unit_costs=dict(RATED_UNIT_COSTS),+ )++ @classmethod+ def search_usage(cls) -> SearchTokensUsage:+ return SearchTokensUsage(+ job_metadata=cls.full_job_metadata(unit_job_id=UnitJobId.SEARCH_SOURCED_ANSWER, job_category=JobCategory.SEARCH_JOB),+ inference_model_name="sonar-pro",+ inference_model_id="sonar-pro-2025",+ nb_tokens_by_category=dict(RATED_NB_TOKENS),+ unit_costs=dict(RATED_UNIT_COSTS),+ )++ @classmethod+ def all_variants(cls) -> list[AnyTokensUsage]:+ return [cls.llm_usage(), cls.img_gen_usage(), cls.extract_usage(), cls.search_usage()]++ @classmethod+ def unrated_usage(cls) -> LLMTokensUsage:+ """A usage with no rate table (own-GPU model, dry/mock run): wire ``cost`` must be None."""+ usage = cls.llm_usage()+ usage.unit_costs = {}+ return usage++ @classmethod+ def cached_fallback_usage(cls) -> LLMTokensUsage:+ """A usage with cached tokens but no explicit cached rate: the 50%-of-input fallback applies."""+ usage = cls.llm_usage()+ usage.unit_costs = {CostCategory.INPUT: 2.0, CostCategory.OUTPUT: 10.0}+ return usagediff --git a/tests/unit/pipelex/cogt/usage/test_usage_cost.py b/tests/unit/pipelex/cogt/usage/test_usage_cost.pynew file mode 100644index 000000000..cc5f07fdf--- /dev/null+++ b/tests/unit/pipelex/cogt/usage/test_usage_cost.py@@ -0,0 +1,46 @@+import pytest++from pipelex.cogt.usage.cost_category import CostCategory+from pipelex.cogt.usage.cost_registry import CostRegistry, compute_tokens_usage_cost+from pipelex.reporting.reporting_types import AnyTokensUsage+from tests.unit.pipelex.cogt.usage.test_data import RATED_EXPECTED_COST, UsageFixtures+++class TestComputeTokensUsageCost:+ def test_none_when_unrated(self):+ """No rate table (own-GPU, dry/mock run) means no cost claim — None, not 0.0."""+ assert compute_tokens_usage_cost(UsageFixtures.unrated_usage()) is None++ def test_rated_cost_value(self):+ assert compute_tokens_usage_cost(UsageFixtures.llm_usage()) == RATED_EXPECTED_COST++ def test_cached_discount_fallback(self):+ """Without an explicit cached rate, cached input tokens cost 50% of the input rate."""+ expected = 600 * (2.0 / 1_000_000) + 400 * (0.5 * 2.0 / 1_000_000) + 250 * (10.0 / 1_000_000)+ assert compute_tokens_usage_cost(UsageFixtures.cached_fallback_usage()) == expected++ @pytest.mark.parametrize(+ "tokens_usage",+ UsageFixtures.all_variants(),+ ids=["llm", "img_gen", "extract", "search"],+ )+ def test_parity_with_cost_registry_per_record(self, tokens_usage: AnyTokensUsage):+ """The wire cost equals the canonical CostRegistry total for the same record — one cost engine, no drift."""+ cost_report = CostRegistry.complete_cost_report(tokens_usage=tokens_usage)+ registry_total = CostRegistry.compute_total_cost(+ input_non_cached_cost=cost_report.costs_by_token_category[CostCategory.INPUT_NON_CACHED],+ input_cached_cost=cost_report.costs_by_token_category[CostCategory.INPUT_CACHED],+ output_cost=cost_report.costs_by_token_category.get(CostCategory.OUTPUT, 0.0),+ )+ assert compute_tokens_usage_cost(tokens_usage) == registry_total++ def test_parity_with_aggregate_run_total(self):+ """Summing per-record wire costs reproduces the CLI cost table's run total."""+ tokens_usages = UsageFixtures.all_variants()+ wire_total = 0.0+ for tokens_usage in tokens_usages:+ record_cost = compute_tokens_usage_cost(tokens_usage)+ assert record_cost is not None+ wire_total += record_cost+ aggregated = CostRegistry.aggregate_costs(tokens_usages=tokens_usages)+ assert wire_total == aggregated.total_costdiff --git a/tests/unit/pipelex/pipe_run/test_delivery_executor.py b/tests/unit/pipelex/pipe_run/test_delivery_executor.pyindex 6201c6a66..62b965b41 100644--- a/tests/unit/pipelex/pipe_run/test_delivery_executor.py+++ b/tests/unit/pipelex/pipe_run/test_delivery_executor.py@@ -276,9 +276,9 @@ class TestDeliveryExecutor:assert "main_stuff.html" in filesasync def test_generate_result_files_writes_usage_artifact_with_records(self, mocker: MockerFixture) -> None:- """The tokens_usages.json artifact carries the run's usage records in the same- per-record ``model_dump(mode="json")`` wire shape the /execute response uses, so a- durable client can compute costs from the polled result files alone.+ """The tokens_usages.json artifact carries the run's usage in the client wire shape+ (``TokensUsageRecord``): computed ``cost``, no ``unit_costs``, no ``job_metadata`` —+ so a durable client reads costs from the polled result files alone."""mock_output = _make_output_mock(mocker)mock_output.working_memory_raw = None@@ -307,7 +307,9 @@ class TestDeliveryExecutor:assert record["inference_model_name"] == "test-model"assert record["inference_model_id"] == "test-model-id"assert record["nb_tokens_by_category"] == {"input": 15, "output": 4}- assert record["unit_costs"] == {"input": 3.0, "output": 15.0}+ assert record["cost"] == 15 * (3.0 / 1_000_000) + 4 * (15.0 / 1_000_000)+ assert "unit_costs" not in record+ assert "job_metadata" not in recordasync def test_generate_result_files_usage_artifact_null_when_usage_off(self, mocker: MockerFixture) -> None:"""A run with usage assembly off still writes the artifact with explicit nulls, so adiff --git a/tests/unit/pipelex/reporting/test_usage_records.py b/tests/unit/pipelex/reporting/test_usage_records.pynew file mode 100644index 000000000..e84fb90d8--- /dev/null+++ b/tests/unit/pipelex/reporting/test_usage_records.py@@ -0,0 +1,139 @@+import json++import pytest+from pytest_mock import MockerFixture++from pipelex.reporting.reporting_types import AnyTokensUsage+from pipelex.reporting.usage_records import (+ TokensUsageRecord,+ apply_tokens_usage_wire_shape,+ dump_tokens_usage_records,+ make_tokens_usage_record,+)+from tests.unit.pipelex.cogt.usage.test_data import RATED_EXPECTED_COST, UsageFixtures++WIRE_FIELDS = {+ "model_type",+ "inference_model_name",+ "inference_model_id",+ "pipe_code",+ "job_category",+ "unit_job_id",+ "nb_tokens_by_category",+ "cost",+ "started_at",+ "completed_at",+}++# The contract's MUST-NOT list: none of these may appear on a wire record, flat or nested.+MUST_NOT_FIELDS = [+ "job_metadata",+ "unit_costs",+ "user_id",+ "session_id",+ "request_id",+ "pipe_run_id",+ "otel_context",+ "trace_context",+ "content_generation_job_id",+ "pipeline_run_id",+]+++class TestTokensUsageRecords:+ @pytest.mark.parametrize(+ ("tokens_usage", "expected_model_type", "expected_job_category", "expected_unit_job_id"),+ [+ (UsageFixtures.llm_usage(), "llm", "llm_job", "llm_gen_text"),+ (UsageFixtures.img_gen_usage(), "img_gen", "img_gen_job", "img_gen_text_to_image"),+ (UsageFixtures.extract_usage(), "extract", "extract_job", "extract_pages"),+ (UsageFixtures.search_usage(), "search", "search_job", "search_sourced_answer"),+ ],+ ids=["llm", "img_gen", "extract", "search"],+ )+ def test_conversion_per_variant(+ self,+ tokens_usage: AnyTokensUsage,+ expected_model_type: str,+ expected_job_category: str,+ expected_unit_job_id: str,+ ):+ """Every usage variant converts to the same single record shape, JobMetadata fields flattened."""+ record = make_tokens_usage_record(tokens_usage)+ assert record.model_type == expected_model_type+ assert record.inference_model_name == tokens_usage.inference_model_name+ assert record.inference_model_id == tokens_usage.inference_model_id+ assert record.pipe_code == "analyze_contract"+ assert record.job_category == expected_job_category+ assert record.unit_job_id == expected_unit_job_id+ assert record.nb_tokens_by_category == {"input": 1000, "input_cached": 400, "output": 250}+ assert record.cost == RATED_EXPECTED_COST+ assert record.started_at == "2026-07-19T10:00:00"+ assert record.completed_at == "2026-07-19T10:00:05"++ def test_cost_is_null_when_unrated(self):+ record = make_tokens_usage_record(UsageFixtures.unrated_usage())+ assert record.cost is None++ @pytest.mark.parametrize(+ "tokens_usage",+ [*UsageFixtures.all_variants(), UsageFixtures.unrated_usage()],+ ids=["llm", "img_gen", "extract", "search", "unrated"],+ )+ def test_leak_regression(self, tokens_usage: AnyTokensUsage):+ """The dumped wire record carries the contract fields exactly — none of the MUST-NOT+ internals (JobMetadata plumbing, rate table), flat or nested.+ """+ dumped = make_tokens_usage_record(tokens_usage).model_dump(mode="json")+ assert set(dumped.keys()) == WIRE_FIELDS+ dumped_text = json.dumps(dumped)+ for forbidden_field in MUST_NOT_FIELDS:+ assert forbidden_field not in dumped, f"MUST-NOT field '{forbidden_field}' leaked onto the wire record"+ assert f'"{forbidden_field}"' not in dumped_text, f"MUST-NOT field '{forbidden_field}' nested in the wire record"++ def test_dumped_record_is_json_safe_and_stringified(self):+ """Enums and timestamps dump as plain JSON strings — a consumer needs no runtime types."""+ dumped = make_tokens_usage_record(UsageFixtures.llm_usage()).model_dump(mode="json")+ reparsed = json.loads(json.dumps(dumped))+ assert reparsed["job_category"] == "llm_job"+ assert reparsed["unit_job_id"] == "llm_gen_text"+ assert reparsed["nb_tokens_by_category"] == {"input": 1000, "input_cached": 400, "output": 250}+ assert reparsed["started_at"] == "2026-07-19T10:00:00"+ # The record round-trips through its own model: the shape is closed (extra="forbid").+ assert TokensUsageRecord.model_validate(reparsed).model_dump(mode="json") == dumped++ def test_dump_null_and_empty_semantics(self):+ """None passes through (usage assembly off) and [] stays [] (on, no inference) — unchanged semantics."""+ assert dump_tokens_usage_records(None) is None+ assert dump_tokens_usage_records([]) == []++ def test_apply_wire_shape_replaces_execute_dump_records(self, mocker: MockerFixture):+ """The execute-response dump gets its full-fidelity usage dumps replaced by wire records."""+ tokens_usages = [UsageFixtures.llm_usage()]+ pipe_output = mocker.MagicMock()+ pipe_output.tokens_usages = tokens_usages+ response_dump = {+ "pipeline_run_id": "plr-fixture",+ "pipe_output": {+ "tokens_usages": [tokens_usage.model_dump(mode="json") for tokens_usage in tokens_usages],+ "usage_assembly_error": None,+ },+ }+ result = apply_tokens_usage_wire_shape(response_dump, pipe_output=pipe_output)+ assert result is response_dump+ records = result["pipe_output"]["tokens_usages"]+ assert len(records) == 1+ assert records[0]["model_type"] == "llm"+ assert records[0]["cost"] == RATED_EXPECTED_COST+ for forbidden_field in MUST_NOT_FIELDS:+ assert forbidden_field not in records[0]+ # The sibling diagnostic stays untouched (D5: bare string contract).+ assert result["pipe_output"]["usage_assembly_error"] is None++ def test_apply_wire_shape_preserves_none(self, mocker: MockerFixture):+ """A run with usage assembly off keeps tokens_usages null in the execute response."""+ pipe_output = mocker.MagicMock()+ pipe_output.tokens_usages = None+ response_dump = {"pipe_output": {"tokens_usages": None, "usage_assembly_error": None}}+ result = apply_tokens_usage_wire_shape(response_dump, pipe_output=pipe_output)+ assert result["pipe_output"]["tokens_usages"] is None