Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

Per-node token usage, cost and model attribution on the run GraphSpec (#1100)

Thomas Hebrardea0487f29e92
diff --git a/.drift/acks/pipelex-kernel-docs.toml b/.drift/acks/pipelex-kernel-docs.toml
index c6ef31887..0e1328a1b 100644
--- a/.drift/acks/pipelex-kernel-docs.toml
+++ b/.drift/acks/pipelex-kernel-docs.toml
@@ -1,8 +1,8 @@
contract = "pipelex-kernel-docs"
-digest = "sha256:5ec1035b41c34141c61a699d4310d73f2999c3d8cd6c4cc6d95fc930ab2101d9"
-reviewed_by = "Louis Choquel"
-reviewed_at = "2026-08-14T06:58:58Z"
-rationale = "Cherry-pick of the step-identity change: make_step_metadata gains pipe_code and PipelexKernel.make gains an injectable step_id_source. Updated the doc's run-scoped-state section: the façade now holds identity plus the step_id_source seam, the job_metadata bullet documents pipe_code naming with its omit-not-None contract and the façade's deliberate anonymity."
+digest = "sha256:b3bee1e27d8ffdbf02ce9a7323c43cce190e1a68368fd56833201eabc48edfae"
+reviewed_by = "Claude (Opus 5, 1M context)"
+reviewed_at = "2026-08-14T12:33:25Z"
+rationale = "Reopened with ZERO trigger-file changes: every blob in pipelex/kernel and test_kernel_boot_contract.py is byte-identical to the previous ack, so no kernel surface moved since Louis reviewed it; the contract definition digest is what shifted, across the dev merge. Reviewed docs/under-the-hood/pipelex-kernel.md anyway, both halves the contract names. Module table: diffed it against every public function AST-parsed out of pipelex/kernel/*.py and it is complete, concrete_llm_model_handle included; kernel.exceptions is absent from the table correctly, since it defines error classes and no entry point. Gates table: its claim that the boot-contract test covers every entry point except the three resolve_*_setting helpers, which read the deck, still matches that test's own stated exception. Nothing stale, no doc change needed."
[trigger_files]
"pipelex/kernel/__init__.py" = "blob:c4f1b0fe2ccc17db9110733f71b612f4ac1ba4ce"
@@ -16,7 +16,7 @@ rationale = "Cherry-pick of the step-identity change: make_step_metadata gains p
"pipelex/kernel/img_gen_ops.py" = "blob:b22869bee7dfbb624e3c406a0631969fc8695b7c"
"pipelex/kernel/img_gen_prompt.py" = "blob:90c33eae95298aa76d036f256ffc7d6523249166"
"pipelex/kernel/img_gen_results.py" = "blob:46b4ca11c70d906ae6d58ce66461f9b3414f1bfa"
-"pipelex/kernel/llm_ops.py" = "blob:36a2e2f3a002e0c50ef4f645937b66573e46aa51"
+"pipelex/kernel/llm_ops.py" = "blob:02bec74888ae6a586ceabaf313277706ba2a2036"
"pipelex/kernel/llm_prompt_content.py" = "blob:26dfce734de582320c0ea7b9a09b01f2d1ca8627"
"pipelex/kernel/llm_results.py" = "blob:6e3b8a2878b5bb284ece5fac10262fb59ac2029d"
"pipelex/kernel/memory_ops.py" = "blob:f6a5745e804ac08f19c70f1beb1f78baf50d51cd"
diff --git a/CHANGELOG.md b/CHANGELOG.md
index ec0ca2117..13d88013c 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,6 +4,15 @@
### Added
+- **Per-node token usage and cost on the run graph (Breaking)**: `GraphSpec` nodes gained a `usage` object and the graph gained a run-level `usage` rollup, so a consumer can finally answer "which pipe spent the money". The link already existed in the event stream — every `UsageReportEvent` names its graph node — and the assembler was dropping it; it now folds those events into per-node totals (own inference calls, tokens by category, cost) plus a `subtree_*` rollup up the parentage chain, so a controller reports what its whole branch spent. Cost is deliberately three-valued and never overloaded: `null` means _unrated_ (no rate table — own-GPU models, dry and mock runs, so every dry-run graph is unrated), a `rated_inference_calls` count marks a mixed rated/unrated node's cost as a lower bound rather than a total, and `total_tokens` is input_joined + output — never the sum of `nb_tokens_by_category`, where `input_cached` is a subset of `input`. Usage whose node cannot be resolved lands in a typed `unattributed` bucket instead of being dropped, so the graph's total cannot silently disagree with the cost report's. Every dollar comes from `compute_tokens_usage_cost`, the same engine the cost table and the API wire records use. Breaking: `NodeSpec` and `GraphSpec` are `extra="forbid"`, so an older pipelex rejects the new JSON. See [Per-node Usage Attribution](under-the-hood/per-node-usage-attribution.md).
+
+- **The run graph now records which model actually ran, per node.** `NodeUsageSpec` gained `by_model` (and `subtree_by_model`): the `inference_model_name`, `inference_model_id`, call count and cost of every model a node actually used, ordered most-used first. Until now a GraphSpec named a model in two places and neither was the outcome — the pipe blueprint holds the authored choice (`$writing-factual`), and `execution_data.resolved_model` holds the handle `LLMSetting.model` carried, which is frequently still an unresolved alias (`@default-premium`) because aliases resolve later, at inference time. `by_model` reads from the usage records instead, so it survives alias resolution, deck defaults, and any fallback or retry that landed somewhere other than what was asked for. It is a list rather than a single value because a node routinely uses more than one model: a `PipeLLM`'s text pass and its object-structuring pass resolve separately. Per-model `cost` follows the same rule as the node's — `None` iff no call to that model carried a rate table.
+
+### Fixed
+
+- **A dry run of a `PipeCondition` is now reproducible.** `pipe_dependencies()` returns a `set[str]`, and the dry-run arm walks **every** branch into the same working memory under the same `output_name` — so the branch iterated last is the one whose stuff the enclosing pipe reports as the condition's output. Iterating the raw set made "last" a function of string-hash order, which varies per process, so the same bundle dry-run twice produced two different graphs: in a sequence routing to two branches that each name their own tail result, the sequence's output name flipped between them run to run. Nothing about a dry run should depend on `PYTHONHASHSEED`, and anything diffing graph specs across runs — a fixture corpus, a static-vs-dry parity check — saw phantom changes. The loop sorts now, which is what the two other order-sensitive readers of this set (`signature_walk`, the `output_option_N` renderer) already did. Live runs were never affected: they take exactly one branch.
+
+- **A `{concept, content}` envelope carrying an empty list is now a value, not an error**: a plural slot is never _absent_ — when nothing is found it is the empty list — and the top-down shaper has always honoured that for a bare `[]` (D2). The bottom-up factory did not: it infers a list's item type from the first item, so an empty list raised _"Cannot create Stuff from empty list in content"_. The two spellings of one input therefore disagreed — a caller with no pictures could send `[]` and run, or send `{"concept": "native.Image", "content": []}` and fail — which made a plural input with nothing in it unrunnable for every caller that wraps its inputs. The envelope names the concept, so there was never anything to infer; it now builds an empty `ListContent`. Case 1.5 (a bare empty `ListContent`, no concept anywhere) still raises, and must: nothing there says what the items would have been.
- **Kernel step identity**: `PipelexKernel.make_step_metadata` now accepts `pipe_code` to name the pipe a step is running, mirroring what the interpreter stamps on its live and dry paths — so log correlation, usage accounting, and per-step labelling see a named step for kernel-driven runs. The key is omitted from the metadata update when not supplied, so a run-level `pipe_code` is never silently erased; the direct-call façade (`llm_text`/`llm_object`) stays deliberately anonymous. `PipelexKernel.make` also accepts an injectable `step_id_source` for per-step ids, defaulting to `uuid4`, so a kernel hosted inside a replay-based executor can supply a replay-safe source.
## [v0.44.0] - 2026-08-13
@@ -11,11 +20,11 @@
### Added
- **Entry-point resolution affordances**: Added `get_required_entry_pipe`, `get_optional_entry_pipe`, `get_required_entry_concept` and `get_optional_entry_concept` to the `interpreter_hub` and library APIs. These are designed for human-supplied codes (CLI arguments, API payloads) and safely search across domains, unlike strict in-body references. Installed dependency packages are excluded from that search, so adding a dependency cannot make one of your own codes ambiguous.
-- **Build-time crate qualification**: Introduced a standalone `crate_qualification` pass that fully qualifies in-body references to their owner domain at library build time, ensuring the normalizer and the live library strictly agree on reference resolution. This supersedes v0.43.0's *Cross-domain pipe refs in normalized crates*: the two readers still agree, but on owner-domain qualification rather than on a crate-wide search.
+- **Build-time crate qualification**: Introduced a standalone `crate_qualification` pass that fully qualifies in-body references to their owner domain at library build time, ensuring the normalizer and the live library strictly agree on reference resolution. This supersedes v0.43.0's _Cross-domain pipe refs in normalized crates_: the two readers still agree, but on owner-domain qualification rather than on a crate-wide search.
### Changed
-- **Strict in-body pipe reference resolution (Breaking)**: Bare in-body pipe references (sequence steps, parallel branches, condition outcomes, `batch_over` targets) now resolve *only* within their own domain. To reference a pipe in another domain, use the fully qualified form (e.g. `sales.generate_tagline`). This enforces `[exports]` visibility rules that were previously bypassable, and lets multiple domains safely reuse the same pipe code without library load failures.
+- **Strict in-body pipe reference resolution (Breaking)**: Bare in-body pipe references (sequence steps, parallel branches, condition outcomes, `batch_over` targets) now resolve _only_ within their own domain. To reference a pipe in another domain, use the fully qualified form (e.g. `sales.generate_tagline`). This enforces `[exports]` visibility rules that were previously bypassable, and lets multiple domains safely reuse the same pipe code without library load failures.
- **Automatic search scope for concepts (Breaking)**: Replaced the `search_domain_codes` list parameter across the public API (`PipelexMTHDSProtocol`, `pipeline_run_setup`, `prepare_pipe_job`, `InputShaper`, `WorkingMemoryFactory`, etc.) with a single `search_scope` string derived automatically from the entry pipe. Every entry-lookup refusal — invalid string, miss, ambiguity — is now raised as `ConceptLibraryConceptNotFoundError`, a class the input-shaping handlers catch.
- **Fully qualified validation error payloads (Breaking)**: The `missing_pipe_code` field on unresolved-dependency validation items now returns the fully qualified reference the compiler attempted to resolve (e.g. `marketing.render_html`) rather than the bare spelling the author wrote.
- **Ambiguity handling for bare code lookups**: Bare pipe codes typed by users (via `pipelex run`, `show`, `which`, or API requests) still search across all domains, but when two domains declare the same code the system now raises a clean ambiguity error listing the candidates instead of failing with a traceback.
@@ -38,13 +47,13 @@
### Changed
- **`PipeRunParams.batch_max_concurrency` is now required (Breaking)**: The field no longer defaults to `None`, which previously made an omitted value indistinguishable from an explicitly authored unbounded concurrency (`max_concurrency = "unbounded"`). Omitting it now raises a `ValidationError` early—on both direct construction and payload decoding—to prevent accidental unbounded fan-outs.
- - **Migration:** Construct `PipeRunParams` via `PipeRunParamsFactory.make_run_params()`, the single writer for run-scoped invariants (`run_mode`, `pipe_stack_limit`, `batch_max_concurrency`). Integration test fixtures now route through the factory via a new `make_mode_guarded_run_params` helper.
- - **Docs:** Added a warning to `docs/under-the-hood/pipe-routing-and-execution.md` on using the factory method.
+ - **Migration:** Construct `PipeRunParams` via `PipeRunParamsFactory.make_run_params()`, the single writer for run-scoped invariants (`run_mode`, `pipe_stack_limit`, `batch_max_concurrency`). Integration test fixtures now route through the factory via a new `make_mode_guarded_run_params` helper.
+ - **Docs:** Added a warning to `docs/under-the-hood/pipe-routing-and-execution.md` on using the factory method.
### Fixed
- **Empty lists in concept envelopes are now parsed as values, not errors**: The bottom-up factory previously raised _"Cannot create Stuff from empty list in content"_ when given an empty list inside a concept envelope (e.g., `{"concept": "native.Image", "content": []}`). Since the envelope names the concept, item-type inference is unnecessary, so it now correctly yields an empty `ListContent`. This aligns the bottom-up factory with the top-down shaper; callers can pass `[]` or `{"concept": "...", "content": []}` to represent an empty plural slot. A bare empty list with no concept still raises an error, as its type cannot be inferred.
- - **Docs:** Added a note to `docs/building-methods/pipes/provide-inputs.md` clarifying that an empty list represents an empty plural slot, whereas omitting the key fails with a _"missing required inputs"_ error.
+ - **Docs:** Added a note to `docs/building-methods/pipes/provide-inputs.md` clarifying that an empty list represents an empty plural slot, whereas omitting the key fails with a _"missing required inputs"_ error.
### Security
@@ -54,7 +63,7 @@
### Highlights
-**Every operator's execution semantics is now importable without the interpreter.** The new public `pipelex.kernel` subpackage extracts what a step *does* — deck resolution, prompt assembly, generation, memory write-back — into plain functions covering every operator, all servable on a kernel-only boot with zero `.mthds` loaded. The layer below the interpreter takes the kernel's name, and plugins now declare which side they belong to through their entry-point group, so an interpreter-side plugin is never imported into a kernel-only process. Distributed execution sharpens alongside — a `run_batch_branch` router hook lets a backend isolate each batch branch, and the fan-out bound is frozen onto the run to close a replay hazard. Breaking changes are import-path moves, an error-class unification, a new required `build_registrar` parameter, a retired plugin entry-point group, and stricter ISO 8601 and failure classification.
+**Every operator's execution semantics is now importable without the interpreter.** The new public `pipelex.kernel` subpackage extracts what a step _does_ — deck resolution, prompt assembly, generation, memory write-back — into plain functions covering every operator, all servable on a kernel-only boot with zero `.mthds` loaded. The layer below the interpreter takes the kernel's name, and plugins now declare which side they belong to through their entry-point group, so an interpreter-side plugin is never imported into a kernel-only process. Distributed execution sharpens alongside — a `run_batch_branch` router hook lets a backend isolate each batch branch, and the fan-out bound is frozen onto the run to close a replay hazard. Breaking changes are import-path moves, an error-class unification, a new required `build_registrar` parameter, a retired plugin entry-point group, and stricter ISO 8601 and failure classification.
### Added
@@ -80,8 +89,8 @@
### Fixed
- **`Date` and `Time` could not be produced by a live `PipeLLM`**: A `PipeLLM` whose `output` was `Date`, `Date[]` or `Time` failed every real run on `date_type` / `time_type` — the structured-output layer validates the model's response in strict mode, where a `mode="before"` validator forfeits pydantic's strict-JSON acceptance of ISO strings. Both natives now parse ISO 8601 themselves and return a real `date` / `time`, so the strict and lax paths agree and the guards (no epoch-seconds, no datetime on a calendar date) still fire in both. `Time` gained the numeric-string guard `Date` already had.
-- **One ISO 8601 contract for the `Date` and `Time` natives (Breaking)**: `pipelex.core.stuffs.iso_temporal` now owns what those natives accept — the *extended* forms only (`2026-07-07`, `15:40:00+02:00`) — and both the content models and `StuffContentFactory` parse through it, so a model-generated value and an authored one are held to the same contract. It also rejects the end-of-day `24:00` form, which names the *next* day's midnight: Python 3.14 reads it as this day's midnight where 3.11-3.13 raise, so a value could silently land a full day early on part of the supported interpreter range. Breaking for authored inputs using a non-extended spelling: `"15:40:00+0200"`, `"2026-07-07T154000"`, `"24:00:00"` and surrounding whitespace are now refused.
-- **Cross-domain pipe refs in normalized crates (Breaking)**: A bare pipe ref calling a pipe declared in another domain was normalized by prefixing the *caller's* domain, naming a pipe the crate does not hold — nothing raised on the way in, and the crate round-trip broke the method at run time with `PipeNotFoundError`. Normalization now resolves a bare ref against the whole crate exactly as the live library does, and raises `CrateNormalizationError` when the ref is ambiguous or matches no pipe.
+- **One ISO 8601 contract for the `Date` and `Time` natives (Breaking)**: `pipelex.core.stuffs.iso_temporal` now owns what those natives accept — the _extended_ forms only (`2026-07-07`, `15:40:00+02:00`) — and both the content models and `StuffContentFactory` parse through it, so a model-generated value and an authored one are held to the same contract. It also rejects the end-of-day `24:00` form, which names the _next_ day's midnight: Python 3.14 reads it as this day's midnight where 3.11-3.13 raise, so a value could silently land a full day early on part of the supported interpreter range. Breaking for authored inputs using a non-extended spelling: `"15:40:00+0200"`, `"2026-07-07T154000"`, `"24:00:00"` and surrounding whitespace are now refused.
+- **Cross-domain pipe refs in normalized crates (Breaking)**: A bare pipe ref calling a pipe declared in another domain was normalized by prefixing the _caller's_ domain, naming a pipe the crate does not hold — nothing raised on the way in, and the crate round-trip broke the method at run time with `PipeNotFoundError`. Normalization now resolves a bare ref against the whole crate exactly as the live library does, and raises `CrateNormalizationError` when the ref is ambiguous or matches no pipe.
- **Structureless concepts in the `python-structures` projection (Breaking)**: A concept declaring only a description was projected as a `StructuredContent` subclass while the runtime promotes the same declaration to a refinement of native `Text` — so the interpreter's text-vs-object dispatch answered differently depending on which class it was handed, and the mismatch raised nothing. The projection now emits `TextContent` as the base, matching the runtime.
- **Unimportable `python-structures` module for a concept refining `Anything`**: `Anything` is the one native with no content class of its own, so a refining concept fell back to the root `StructuredContent` — emitting the class name without registering its import, and the generated module raised `NameError` on the consumer's first import. The fallback now goes through the renderer that registers the import.
- **Format-stable import blocks in generated Python**: A generated import line whose merged names crossed 88 columns was emitted flat, so the first `ruff format` in a consumer's tree rewrote it — changing the body bytes and making `pipelex codegen check` report the file as hand-edited. Import blocks now pre-explode past the threshold like every other emitted construct.
@@ -99,9 +108,9 @@
### Added
- **New classified error types**: Introduced specific error classes for clearer debugging and handling, each with dedicated documentation pages:
- - `OutputStructureSchemaError`: raised when an output structure class cannot produce the JSON schema required by a structured leaf.
- - `GatewaySearchEmptyResultError` & `LinkupSearchEmptyResultError`: raised when a search returns no object payload to fill the output structure (pointing to the query rather than the model).
- - `LinkupError` (base class) & `LinkupSearchResponseError`: raised when the Linkup API answers a structured search with an invalid payload shape.
+ - `OutputStructureSchemaError`: raised when an output structure class cannot produce the JSON schema required by a structured leaf.
+ - `GatewaySearchEmptyResultError` & `LinkupSearchEmptyResultError`: raised when a search returns no object payload to fill the output structure (pointing to the query rather than the model).
+ - `LinkupError` (base class) & `LinkupSearchResponseError`: raised when the Linkup API answers a structured search with an invalid payload shape.
### Changed
@@ -128,7 +137,7 @@
- **Claude 5 Support:** `claude-5-sonnet` is now served by the Pipelex gateway.
- **Error Identity Snapshotting:** Added a `generate-error-identity` (`gei`) CLI command and Make target that commits a snapshot of every `PipelexError` subclass's `(error_type, title, type_uri)` triple, so renaming an error class — which changes the `error_type` string consumers branch on — surfaces as a reviewable diff instead of silently breaking them. Documented in [Error Model](under-the-hood/error-model.md).
-- **Hub Layering CI Enforcement:** Added `make check-hub-layering` (`chl`), now a required CI status check (`lint-hub-layering`), to enforce the one-way boundary between the runtime and interpreter layers. The rule is transitive — it reports any runtime-layer module that *reaches* the interpreter hub, naming the shortest import chain and the line of its first hop.
+- **Hub Layering CI Enforcement:** Added `make check-hub-layering` (`chl`), now a required CI status check (`lint-hub-layering`), to enforce the one-way boundary between the runtime and interpreter layers. The rule is transitive — it reports any runtime-layer module that _reaches_ the interpreter hub, naming the shortest import chain and the line of its first hop.
- **Concept Registry Boundary:** A golden-set AST test over `pipelex/core/concepts/**` pins which modules may read the process-global class registry — only the materialization write side may.
- **Vendor-Neutrality Pinning:** Added strict tests pinning `pipelex.cogt`'s third-party import roots and its exact `cogt → pipelex.providers` edges, so a new inference-provider SDK import or a new inversion shows up as a diff. Both packages are runtime-layer, so no other gate can see these edges.
- **Registry Sort Enforcement:** The keyword-only guard now enforces the sort order of the `subject_grants.toml` registry (`unsorted-grant` violation) to prevent silent breakage during bulk rewrites. See [Keyword-only arguments](contribute/keyword-only-arguments.md).
@@ -206,10 +215,10 @@
### Added
- **`pipelex resolve` and the `pipelex codegen` family:** `pipelex resolve` assembles a bundle closure and emits the **normalized library crate** — a flat, fully-qualified, natives-expanded, self-contained, fingerprinted snapshot of a resolved library — as JSON or TOML. On top of the crate:
+ - `pipelex codegen types --target ts-zod|python-pydantic|python-structures` projects the crate's concept set into typed artifacts. The `ts-zod` target emits a pure `types.ts` (imports only `zod`, with wire-native snake_case keys so a schema validates a wire payload directly) plus a `binder.ts` exposing a typed `parse`/`serialize` pair per concept; `python-pydantic` emits self-contained `BaseModel`s; `python-structures` emits runtime `StructuredContent` classes.
+ - `pipelex codegen inputs --pipe <ref>` renders a runnable inputs template for a pipe (defaulting to the closure's `main_pipe`).
+ - Generated files are never edited — each carries an `AUTOGENERATED` header pointing at the sibling extension-file mechanism.
- - `pipelex codegen types --target ts-zod|python-pydantic|python-structures` projects the crate's concept set into typed artifacts. The `ts-zod` target emits a pure `types.ts` (imports only `zod`, with wire-native snake_case keys so a schema validates a wire payload directly) plus a `binder.ts` exposing a typed `parse`/`serialize` pair per concept; `python-pydantic` emits self-contained `BaseModel`s; `python-structures` emits runtime `StructuredContent` classes.
- - `pipelex codegen inputs --pipe <ref>` renders a runnable inputs template for a pipe (defaulting to the closure's `main_pipe`).
- - Generated files are never edited — each carries an `AUTOGENERATED` header pointing at the sibling extension-file mechanism.
- **Codegen trust chain (stamps · `codegen.lock` · offline `codegen check`):** Every file `pipelex codegen types` writes now carries a self-describing **stamp** header (source crate fingerprint, engine version, projection, options, and a content hash of the body below it), and the artifact set is recorded in a sibling **`codegen.lock`**. `pipelex codegen check [DIR]` verifies generated artifacts are current **offline** — pure hashing, no engine, network, or API key — reporting drift by category (missing, modified, hand-edited, orphan) with the verdict on the exit code (`0` current · `1` drift · `2` no lock). Emission is idempotent (write-if-changed — unchanged output leaves files untouched) and prunes previously-generated files that drop out of the set, so a deleted concept never lingers as a stale generated file. Input templates (`codegen inputs`) are deliberately not stamped or locked — they are user-editable scaffolds, not tracked generated code.
- **Agent-CLI codegen mirror (`pipelex-agent codegen types|check`):** The codegen family is now available to machine consumers through the agent CLI's two-stream envelopes (`--format markdown|json` success stream, `--error-format` error stream). `codegen types` runs the same engine and write-if-changed emission as the bare CLI; `codegen check` presents the offline drift verdict structurally — drift is a `CodegenDriftError` envelope enumerating `drifts[]` by category (exit `1`), a missing or unreadable `codegen.lock` is a no-verdict error (exit `2`). With the pipelex runner, `mthds-agent codegen …` forwards to these commands.
- **Host-facing resolve/codegen engine cores (what the HTTP routes ride):** `pipelex.pipeline.resolve_bundle.resolve_crate_from_contents` resolves **in-memory** MTHDS contents into the normalized crate — same engine, same `ValidateBundleError` verdict vocabulary, and the same loaded-on-success library contract as `validate_bundle` — so a host (the `pipelex-api` `/v1/resolve` + `/v1/codegen` routes) serves resolution without touching the filesystem. `pipelex.codegen.emission.build_stamped_projection` is the new **pure** stamping core (stamped files + `codegen.lock` content, no disk writes) that `write_stamped_projection` now rides, so artifacts served over HTTP are byte-identical to locally written ones and pass the offline `codegen check` verbatim. The shared bundle-loading error cascade `translate_to_validate_bundle_error` is now public (renamed from its underscore-private form) since it gained a third entry point.
@@ -218,11 +227,11 @@
- **Suggested fixes surfaced in `pipelex validate`:** When a validation error has a deterministic safe fix, the human error output now shows a per-error `💡 Suggested fix:` line and ends with an actionable footer naming the exact `pipelex fix bundle` command (echoing the invocation's `-L` dirs) in place of the generic tip.
- **`suggested_fix` field in `/validate` payloads:** Each validation error that has a deterministic safe fix now carries an additive structured `suggested_fix` object — `fix_code` (the kebab-case rule id), `description` (the author-facing 💡 text), `safety` (`safe`/`unsafe`), optional `source` file, and `ops` (the semantic patch operations) — on every `validation_errors[]` item, both in the `pipelex-agent validate`/`fix` JSON envelopes and in the API `/validate` response body. This is the machine-actionable counterpart of the 💡 line, letting a software consumer apply the fix directly from the structured payload.
- **Smart Inputs (signature-driven input shaping):** Inputs are now interpreted top-down against the pipe's declared input signature rather than bottom-up from their shape alone.
+ - Bare values (strings, numbers, booleans, dicts, lists) are automatically shaped into the declared concept (e.g., a bare string becomes a `legal.Question`, not a generic `native.Text`).
+ - Lists shape element-wise into the declared item concept; single values auto-wrap into lists where required, and empty lists are now legal.
+ - Bare URLs/paths for `Image` or `Document` inputs are resolved relative to the inputs file's directory.
+ - Declared structured lists (e.g., `Person[]`) accept a direct path to a `.csv` file by signature, without requiring an envelope.
- - Bare values (strings, numbers, booleans, dicts, lists) are automatically shaped into the declared concept (e.g., a bare string becomes a `legal.Question`, not a generic `native.Text`).
- - Lists shape element-wise into the declared item concept; single values auto-wrap into lists where required, and empty lists are now legal.
- - Bare URLs/paths for `Image` or `Document` inputs are resolved relative to the inputs file's directory.
- - Declared structured lists (e.g., `Person[]`) accept a direct path to a `.csv` file by signature, without requiring an envelope.
- **Comprehensive input error handling:** A suite of typed errors for input shaping failures (`WrongScalarKindError`, `ListWhereSingularError`, `MultiplicityCountMismatchError`, `StructureValidationError`, `UnknownInputNameError`) that provide clear hints and render the expected JSON shape.
- **CLI `--explicit` flag:** Added `--explicit` to `pipelex build inputs` and `pipelex-agent inputs` to generate the legacy `{"concept", "content"}` envelope templates.
- **Native `YesNo` concept:** Built-in concept (backed by `YesNoContent`) for boolean judgments. Requires a strict boolean, renders as `yes`/`no` in downstream prompts, and can be read via `pipe_output.main_stuff_as_yes_no.yes_no`.
@@ -232,11 +241,10 @@
- **Pinned normative native definitions:** Native concepts now materialize into the normalized crate by **lookup into the standard's pinned per-version definitions** (`pipelex/core/concepts/native/pinned_blueprints.py`, mirroring the mthds spec's new `native-concepts.md`), never by reflection over the runtime content classes — so crate fingerprints over materialized natives byte-agree across implementations. The old reflection is retained solely as a consistency test proving each runtime content class still matches its pinned blueprint.
- **Subject-grants registry — the keyword-only convention hardened:** A positional first parameter ("subject") in `pipelex/` source is now legal only under an explicit grant recorded in `subject_grants.toml` at the repo root, carrying the subject's param name and an honest, def-specific review rationale — every grant in the registry has been individually reviewed against the grant rubric. `pipelex-dev subject-grant` is the sole registry writer; the `check-keyword-only` AST guard enforces the registry symmetrically (an ungranted subject fails, and so does a stale grant whose def was renamed, moved, demoted, or newly carved out) in `make check`, CI, and the edit-time hook. Subjects typed `bool`/`int`/`float` are banned outright — no grant can cover them. The registry schema is strict: exactly `param` + `rationale`, unknown keys fail the check. See `docs/contribute/keyword-only-arguments.md`.
- **Drift contracts (`pipelex-dev drift plan|check|ack`):** Deterministic review obligations between code and docs. A root `drift.toml` declares contracts ("when these trigger files change, these review targets must be re-examined and the examination recorded"); committed ack files under `.drift/acks/` record each fulfilled review with a digest, reviewer, and rationale. A contract is fulfilled iff its stored ack digest equals the digest recomputed from the current tree (staged blob OIDs from the git index — no base ref, no timestamps), so trigger edits, matched-file adds/deletes, and contract-definition edits all reopen it.
-
- - `make drift-check` gates the `make check` aggregate and runs as an advisory CI job; `make drift-plan` prints per-contract Markdown packets with exact per-file changes since the last ack.
- - `make drift-ack CONTRACT=… RATIONALE="…"` runs the contract's verify commands, then records the review and stages the ack file it writes, so the local gate checks the same index the commit is built from. If staging that ack fails, the ack file is removed rather than left unstaged, so a later check reports a missing ack instead of a false green.
- - For a contract with verify commands, a trigger file left unstaged-modified or untracked is a hard ack error (the verify run would certify content the digest does not cover) — checked both before and after the verify commands, so a trigger the commands themselves format or generate is caught too. For other contracts it stays a warning.
- - Seeded with contracts for the config docs, the CLI docs, and the keyword-only convention spec; workflow documented in `docs/contribute/drift-contracts.md`.
+ - `make drift-check` gates the `make check` aggregate and runs as an advisory CI job; `make drift-plan` prints per-contract Markdown packets with exact per-file changes since the last ack.
+ - `make drift-ack CONTRACT=… RATIONALE="…"` runs the contract's verify commands, then records the review and stages the ack file it writes, so the local gate checks the same index the commit is built from. If staging that ack fails, the ack file is removed rather than left unstaged, so a later check reports a missing ack instead of a false green.
+ - For a contract with verify commands, a trigger file left unstaged-modified or untracked is a hard ack error (the verify run would certify content the digest does not cover) — checked both before and after the verify commands, so a trigger the commands themselves format or generate is caught too. For other contracts it stays a warning.
+ - Seeded with contracts for the config docs, the CLI docs, and the keyword-only convention spec; workflow documented in `docs/contribute/drift-contracts.md`.
### Changed
@@ -249,7 +257,7 @@
- **Clean validation summary message on every surface:** The top-level error `message` (agent JSON, the API `/validate` body) is now a clean, author-facing summary derived from the structured errors, instead of the raw pydantic dump (`Validation error(s): … Value errors: '<field>': Value error, …`). The API's opt-in invalid-verdict `rendered_markdown` now carries the same per-error `💡 Suggested fix` prose as the CLI. A still-invalid fix that stops with no progress now names the re-proposed fix in author terms rather than dumping an internal fingerprint string.
- **Input templates default to "light" shape (Breaking):** `pipelex build inputs` and `pipelex-agent inputs` now generate light templates by default, outputting the bare values expected by the signature rather than the verbose `{"concept", "content"}` envelopes. TOML templates include the declared concept as a `# concept: ...` comment.
- **Structure-field `type = "date"` behavior (Breaking):** A `date` field now generates a `datetime.date` (JSON schema `format: date`), preventing LLMs from hallucinating times for calendar dates. A new `type = "datetime"` field handles full timestamps (`format: date-time`), and default-value validation now rejects a `datetime` default on a `date` field.
-- **Strict input validation (Breaking):** Undeclared input names are now a hard error (previously silently ignored), and top-level `null` values are rejected — absence must be expressed by omitting the key. Known limitation: this also rejects the *extra* stuffs carried by a `pipelex-agent run --with-memory` envelope, so piping one run's full working memory into a downstream pipe that declares only a subset of it now fails with `UnknownInputNameError`. Until envelope-derived inputs get their own leniency, narrow the envelope to the declared inputs before piping, or pass the inputs explicitly with `--inputs`.
+- **Strict input validation (Breaking):** Undeclared input names are now a hard error (previously silently ignored), and top-level `null` values are rejected — absence must be expressed by omitting the key. Known limitation: this also rejects the _extra_ stuffs carried by a `pipelex-agent run --with-memory` envelope, so piping one run's full working memory into a downstream pipe that declares only a subset of it now fails with `UnknownInputNameError`. Until envelope-derived inputs get their own leniency, narrow the envelope to the declared inputs before piping, or pass the inputs explicitly with `--inputs`.
- **Explicit envelope validation (Breaking):** The `{"concept", "content"}` escape hatch is now compatibility-checked against the pipe's declared signature; incompatible concepts raise a typed error.
- **TOML date/time handling (Breaking):** TOML date and datetime literals are natively supported and map directly to the `Date` concept; a bare time-of-day literal maps to the new `Time` concept (it never silently becomes a `Date` — shaping a `Time` into a `Date` slot fails the compatibility check).
- **`ImageContent` flattens its pixel dimensions (Breaking):** `size: ImageSize | None` is replaced by paired optional `width` / `height` integer fields (both-or-neither, enforced by a validator) — a named nested shape in the crate is a concept, and the size grouping did not earn that citizenship. `ImageSize` remains as the internal img-gen parameter type. Image wire payloads change shape accordingly.
@@ -257,8 +265,8 @@
### Fixed
-- **Library loaders now bind the target library for the whole load:** `load_from_blueprints` / `load_from_crate` / `load_libraries` took an explicit `library_id` but resolved concepts and the class registry through the ambient *current-library* ContextVar one hop below — so loading into a library that wasn't current read (and registered generated structure classes into) whatever library the ambient pointer happened to name. Production hosts always set the target current before loading, so this only bit multi-library callers — exactly the isolation the loaders exist to provide. The loaders (and blueprint removal) now wrap their body in `scoped_current_library(library_id)`, restoring the caller's binding on exit, so everything beneath a load resolves against the load's target by construction. A regression test loads into a deliberately non-current library.
-- **`PipeExtract` broke crate normalization (500s on every static-core route):** `PipeExtractBlueprint.validate_output` only accepted the authoring spelling `Page[]`, but normalization qualifies concept refs (`Page[]` → `native.Page[]`) and then rebuilds the blueprints — so the validator rejected its own normalized output, and any bundle containing a `PipeExtract` raised a raw pydantic `ValidationError` out of `resolve_crate_from_contents`, 500-ing `/v1/build/inputs`, `/v1/build/output`, `/v1/resolve`, and `/v1/codegen`. The validator now compares the *parsed* concept ref and multiplicity instead of the raw string, so it holds for both spellings. The output contract is unchanged and still exact — a variable-length list of native `Page` — since a refined output would be an unverifiable promise: `PipeExtract` mechanically produces native pages and cannot make them more specific.
+- **Library loaders now bind the target library for the whole load:** `load_from_blueprints` / `load_from_crate` / `load_libraries` took an explicit `library_id` but resolved concepts and the class registry through the ambient _current-library_ ContextVar one hop below — so loading into a library that wasn't current read (and registered generated structure classes into) whatever library the ambient pointer happened to name. Production hosts always set the target current before loading, so this only bit multi-library callers — exactly the isolation the loaders exist to provide. The loaders (and blueprint removal) now wrap their body in `scoped_current_library(library_id)`, restoring the caller's binding on exit, so everything beneath a load resolves against the load's target by construction. A regression test loads into a deliberately non-current library.
+- **`PipeExtract` broke crate normalization (500s on every static-core route):** `PipeExtractBlueprint.validate_output` only accepted the authoring spelling `Page[]`, but normalization qualifies concept refs (`Page[]` → `native.Page[]`) and then rebuilds the blueprints — so the validator rejected its own normalized output, and any bundle containing a `PipeExtract` raised a raw pydantic `ValidationError` out of `resolve_crate_from_contents`, 500-ing `/v1/build/inputs`, `/v1/build/output`, `/v1/resolve`, and `/v1/codegen`. The validator now compares the _parsed_ concept ref and multiplicity instead of the raw string, so it holds for both spellings. The output contract is unchanged and still exact — a variable-length list of native `Page` — since a refined output would be an unverifiable promise: `PipeExtract` mechanically produces native pages and cannot make them more specific.
- **`resolve` / `codegen` rejected bundles that pin a model:** The resolve and codegen commands booted without model specs, so library validation could not check pipe model pins (`model = "gpt-4o-mini"`) against the deck and rejected any bundle carrying one. They now boot like `validate` — offline, but with model specs loaded.
- **Native `Image` no longer materializes structureless:** Every native now materializes from its pinned normative definition, so generated clients for image-producing pipes expose `url`, `public_url`, `caption`, `width`, `height`, etc. (The brief interim dict-with-imprecision mapping for the nested size model is superseded by the `ImageContent` flatten above.)
- **Deterministic input-template placeholders:** Mock URL placeholders in generated inputs templates (`codegen inputs` / `build inputs`) are now deterministic (`https://mock.invalid/<field>`), so committed templates no longer churn on every regeneration.
@@ -283,12 +291,12 @@
### Added
- **Optionals & presence markers (`?`, `!`):** Concept references in pipe `inputs`/`outputs` can now be marked `?` (optional) or `!` (force), e.g. `clause = "PenaltyClause?"`. Absent values are first-class data, tracked in working memory via an `AbsenceRecord` ledger (variable, producing pipe, kind = declared-absent / skipped / not-provided, reason, upstream chain) with provenance captured at the moment absence is produced:
+ - A plain input fed an absence causes the pipe to be **skipped** (implicitly lifted); its own output is recorded absent, chaining back to the origin so the miss short-circuits like Swift's `a?.b.c`.
+ - A `?` input **absorbs** the absence and the pipe runs, handling both arms.
+ - A `!` input fed an absence **fails loudly** with a typed `OptionalValueAbsentError` naming the variable, the consuming pipe, the producing pipe, and the original reason.
+ - Callers may omit `?`-marked method inputs; the slot starts as a recorded `not_provided` absence rather than raising a missing-inputs error, and the missing-required-inputs message now names the optional inputs a caller may omit.
+ - Post-run reads get a tri-state resolved accessor (`WorkingMemory.resolve_stuff` / `resolve_main_stuff`: a value or a recorded absence). A slot with neither a value nor a record is still a hard error — never-produced is a bug, not an absence. Markers round-trip through blueprints, builder specs, IO contracts, and bundle representations; grammar misuse (`X[]?`, `!` on an output, markers on concept definitions) is rejected at parse with `optional_marker_invalid`.
- - A plain input fed an absence causes the pipe to be **skipped** (implicitly lifted); its own output is recorded absent, chaining back to the origin so the miss short-circuits like Swift's `a?.b.c`.
- - A `?` input **absorbs** the absence and the pipe runs, handling both arms.
- - A `!` input fed an absence **fails loudly** with a typed `OptionalValueAbsentError` naming the variable, the consuming pipe, the producing pipe, and the original reason.
- - Callers may omit `?`-marked method inputs; the slot starts as a recorded `not_provided` absence rather than raising a missing-inputs error, and the missing-required-inputs message now names the optional inputs a caller may omit.
- - Post-run reads get a tri-state resolved accessor (`WorkingMemory.resolve_stuff` / `resolve_main_stuff`: a value or a recorded absence). A slot with neither a value nor a record is still a hard error — never-produced is a bug, not an absence. Markers round-trip through blueprints, builder specs, IO contracts, and bundle representations; grammar misuse (`X[]?`, `!` on an output, markers on concept definitions) is rejected at parse with `optional_marker_invalid`.
- **Static optionality validation (the absence-taint pass):** Validation proves the absence-safety theorem at author time — a taint pass over each controller's dataflow computes per-slot presence (`guaranteed` / `maybe-absent`) and rejects any maybe-absent slot that escapes without an explicit sink, with typed errors (`optional_not_handled`, `optional_output_required`, `optional_branch_required_field`) that each name the absence origin, the propagation path, and the fixes. The validation report also lists every liftable (skippable) pipe (`liftable_pipes`) — build-time visibility for "may be skipped when X is absent" — and gains a general `warnings` array for advisory lints that never flip `is_valid` (first occupant: `optional_force_redundant`, a `!` whose slot is guaranteed present in every flow).
- **Template guard-lint (`optional_input_unguarded`):** Every template reference to a declared-optional (`?`) input must be guarded — `@?var`, a `{% if var %}…{% endif %}` block, or an inline presence conditional — or validation fails with the precise fix. Applies to `PipeLLM` prompts and system prompts, `PipeCompose` templates, and `PipeCondition` expressions. In the same motion, `@?` finally means what it says: an optional variable is no longer presence-required by the controller miss-gates, so the pipe runs and its guarded templates take the absent arm.
- **Execution graph updates:** `GraphSpec` nodes gain the `skipped` status with a `skip_reason` — "why did my workflow produce nothing?" is now answerable from the graph — and data edges fed by a declared-optional (`?`) output carry `optional: true`. Both flow through the in-process tracer and the distributed event-replay assembler (new `pipe_end_skipped` trace event).
@@ -301,7 +309,7 @@
- **Dropped Python 3.10 support (Breaking):** `requires-python` is now `>=3.11,<3.15`. The 3.10 compatibility bridges are gone — `StrEnum`, `Self`, and `Traversable` are imported directly from the stdlib at every call site, the `pipelex.types` re-export module that existed only to paper over 3.10 is deleted, and the `backports.strenum` conditional dependency (and its type stub) is dropped. CI test/lint matrices, the `package-check` `requires-python` floor gate, and the contributor docs now start at 3.11.
- **Plugin API version bumped to 3 (Breaking):** `PLUGIN_API_VERSION` is now `3` (was `2`) to add the `add_storage_provider` and `add_secrets_provider` registrar menu methods. Plugin discovery version-checks with strict equality, so every external plugin must re-declare `targets_api = 3`. Our Mistral Workflows plugin registers no storage or secrets provider, so the `targets_api` bump is its only change. Our Temporal plugin needs the bump **and** a code migration: it imports the now-removed `make_storage_provider_from_config` (see Removed) and must switch to resolving the provider via the registry (`get_storage_provider_registry().get_required(method=...)`).
-- **`PipeSignature` is not a pipe type (Breaking):** A signature is now declared by **omitting** `type` — a `[pipe.x]` section with no `type` and nothing but the contract (`description`, `output`, optional `inputs`, optional `signature_for`) *is* a `PipeSignature`. Writing `type = "PipeSignature"` explicitly is rejected with a migration error ("`PipeSignature` is no longer a pipe type — delete the `type` line"), and a typeless section declaring any non-contract field is a hard error naming the field. The `pipelex-agent pipe` authoring command mirrors this. **MTHDS JSON Schema shape change:** the signature arm no longer carries a `type` property (an explicit tag now fails schema validation), so the downstream schema copies (`mthds`, `vscode-pipelex`, `mthds-ui`) must be re-synced on the next release via the `mthds-schema-sync` skill.
+- **`PipeSignature` is not a pipe type (Breaking):** A signature is now declared by **omitting** `type` — a `[pipe.x]` section with no `type` and nothing but the contract (`description`, `output`, optional `inputs`, optional `signature_for`) _is_ a `PipeSignature`. Writing `type = "PipeSignature"` explicitly is rejected with a migration error ("`PipeSignature` is no longer a pipe type — delete the `type` line"), and a typeless section declaring any non-contract field is a hard error naming the field. The `pipelex-agent pipe` authoring command mirrors this. **MTHDS JSON Schema shape change:** the signature arm no longer carries a `type` property (an explicit tag now fails schema validation), so the downstream schema copies (`mthds`, `vscode-pipelex`, `mthds-ui`) must be re-synced on the next release via the `mthds-schema-sync` skill.
- **`PipeCondition` `continue` now resolves the output as absent (Breaking):** The `continue` special outcome no longer passes the current main stuff through (and no longer errors when there is none). It records a declared-absent `AbsenceRecord` for the condition's declared output — with the evaluated expression as the reason — and returns success, memory otherwise unchanged. A `continue`-reachable `PipeCondition` must therefore declare its output optional (e.g. `output = "Constraint?"`), or validation rejects it (`optional_output_required`). Dry-run parity holds. Migration: the previous value stays in working memory under its own name, so downstream pipes consume it explicitly by that name (declared `?` when it may be absent); a coalescing operator is the planned ergonomic replacement for pass-through.
- **Controllers combine under absence:** `PipeParallel` no longer crashes when a branch result resolves absent (lifted branch or `continue`): with a `Composite` output the absent component is omitted (ledger note kept), and with a structured output a non-required field absorbs the absence as its default (`None` unless the author declared another), while a required field fed an absent branch raises a typed error naming the branch, the field, and the fixes. `PipeBatch` compacts: absent branch results are dropped from the aggregated list, so batching a "keep or skip" condition over items yields only the kept results.
- **Absent output delivery is now a first-class success:** A run whose declared `?` output resolves absent succeeds everywhere. `main_stuff_name` on `PipelexPipeRunOutput` and the `/execute` response names the declared output slot even when it resolved absent — consumers branch on the absence record in the serialized working memory (`absences`), never on transport, and the ledger round-trips cross-process (`hydrate_working_memory` reconstructs it). The `main_stuff.json/md/html` artifact files become an explicit absence document (`{"absent": true}` plus the provenance chain) on both the typed and raw delivery paths, `pipelex run` / `--save-main-stuff` and the agent CLI's `run` surface it instead of crashing, and OTel/Langfuse capture serializes it. A working memory with neither a value nor a recorded absence still fails delivery loudly.
@@ -309,7 +317,7 @@
### Fixed
- **Template truth-tests on singular values no longer crash:** `{% if var %}` (and `@?var`, which expands to it) on a present singular value used to raise `TypeError: '…' content does not support len().` — Jinja2's truth test fell through to the artefact's list-only `__len__`. `StuffArtefact` now defines `__bool__`: a present non-list artefact is truthy, a list artefact follows list emptiness. This makes the optionals guard idiom safe on both arms.
-- **`PipeCompose` escaped-sigil literals (`$$`, `@@`) no longer double-rewrite:** `PipeCompose` alone stored its template already-preprocessed and then re-ran the sigil rewriter at guard-lint and render time. Because the escape collapse (`$$name` → `$name`) is not idempotent, the second pass resurrected escaped literals into interpolations — raising a spurious `optional_input_unguarded` on a `$$name` that only *looks* like an optional reference, and silently rendering the value of `name` instead of the literal `$name`. `PipeCompose` now stores authored source and rewrites exactly once, like every other pipe.
+- **`PipeCompose` escaped-sigil literals (`$$`, `@@`) no longer double-rewrite:** `PipeCompose` alone stored its template already-preprocessed and then re-ran the sigil rewriter at guard-lint and render time. Because the escape collapse (`$$name` → `$name`) is not idempotent, the second pass resurrected escaped literals into interpolations — raising a spurious `optional_input_unguarded` on a `$$name` that only _looks_ like an optional reference, and silently rendering the value of `name` instead of the literal `$name`. `PipeCompose` now stores authored source and rewrites exactly once, like every other pipe.
- **`--inputs ~/…` now expands on `run pipe` / `run bundle`:** a quoted or `=`-form tilde path (`--inputs "~/inputs.json"`, which the shell leaves unexpanded) is now `expanduser()`-ed before loading, resolving to the home directory instead of failing on a literal `~` — matching the existing `run method` behavior, on both the human and agent CLIs.
- **Invalid storage/secrets `method` errors stay actionable under STRICT:** `UnknownStorageMethodError` / `UnknownSecretsMethodError` are marked caller-facing, so their "registered methods: … — check `storage_config.method`" guidance survives STRICT error disclosure instead of being redacted to a generic internal-error message.
- **Non-string JSON pipe `type` gives an actionable error:** a `type` that is a list/dict/number in a `pipelex-agent pipe` JSON spec now surfaces as an `ArgumentError` naming the valid pipe types, instead of a cryptic internal `TypeError: unhashable type`.
@@ -487,7 +495,7 @@
- **Mistral chat requests now send the system message before the user message**: `make_simple_messages` previously appended it after the user message, contradicting its own docstring and the OpenAI-typed sibling.
- **String concept values in bundle specs are constructible again**: a bare string in the `ConceptSpec | str` union now validates (no longer crashing the `mode="before"` validator) and passes through to the blueprint as the concept's description instead of into `structure`, where the loader rejected it.
- **LocalObserver JSONL records always carry the true lifecycle event name**: a payload's own `event_type` key can no longer overwrite the event name in the written record, which had broken event-type filtering for JSONL consumers.
-- **Tests under `tests/**/build/` are no longer silently skipped**: pytest's default `norecursedirs` includes `build`, so the whole `tests/unit/pipelex/cli/commands/build/` tree was never collected; the config now overrides `norecursedirs` and sets `testpaths = ["tests"]`.
+- **Tests under `tests/**/build/`are no longer silently skipped**: pytest's default`norecursedirs`includes`build`, so the whole `tests/unit/pipelex/cli/commands/build/`tree was never collected; the config now overrides`norecursedirs`and sets`testpaths = ["tests"]`.
- **`Anything`-output option numbering is now deterministic**: the output renderer sorts a `PipeCondition`'s possible outputs by pipe code instead of iterating a set, so `output_option_N` / `schema_option_N` numbering is stable across runs.
- **Documentation examples**: Updated code snippets in `docs/under-the-hood/` to match the new keyword-only signatures.
- **Test mock assertions**: Migrated mock assertions from `.call_args.args` to `.call_args.kwargs` to reflect keyword-only calls.
@@ -1925,8 +1933,8 @@ This release hardens Pipelex at its edges. The headliners: a full **error-handli
### Highlights
- Library manager now supports multiple libraries. You can now have multiple libraries in your project, each with its own set of concepts, pipes, and stuffs.
-You can run the same pipe at the same times as much as you want, with different inputs.
-Side effets: Unit tests now run in 30s.
+ You can run the same pipe at the same times as much as you want, with different inputs.
+ Side effets: Unit tests now run in 30s.
### Fixed
diff --git a/docs/features/cost-tracking.md b/docs/features/cost-tracking.md
index 42e1d3385..67da91256 100644
--- a/docs/features/cost-tracking.md
+++ b/docs/features/cost-tracking.md
@@ -31,6 +31,12 @@ render_cost_report_for_output(pipe_output)
The `--costs` gate is read off the output itself — `tokens_usages` is `None` exactly when cost reporting was off for the run — so you never re-read global config to reconstruct whether costs were on.
+## Per-node Attribution in the Run Graph
+
+The same usage stream also lands on the run graph: every node of `pipe_output.graph_spec` carries a `usage` object with its own inference calls, tokens and cost, plus a `subtree_*` rollup covering it and everything below it — so a controller shows what its whole branch spent. The graph's `usage.total` is the run-level rollup.
+
+Cost there is deliberately three-valued: `null` means *unrated* (no rate table — own-GPU models, dry and mock runs), never `0.00`. See [Per-node Usage Attribution](../under-the-hood/per-node-usage-attribution.md) for the invariants a consumer must branch on.
+
## Custom Reporting
Implement custom reporting backends via the [Reporting Delegate](advanced-customizations.md) injection point. The reporting protocol receives structured cost events during execution, allowing you to push data to your own analytics, dashboards, or billing systems.
diff --git a/docs/under-the-hood/execution-graph-tracing.md b/docs/under-the-hood/execution-graph-tracing.md
index 74fa52334..06aa76f13 100644
--- a/docs/under-the-hood/execution-graph-tracing.md
+++ b/docs/under-the-hood/execution-graph-tracing.md
@@ -167,12 +167,15 @@ class GraphSpec(BaseModel):
pipeline_ref: PipelineRef
nodes: list[NodeSpec]
edges: list[EdgeSpec]
+ usage: GraphUsageSpec | None
meta: dict[str, Any]
def to_json(self) -> str:
return self.model_dump_json(by_alias=True, indent=2)
```
+`usage` (and its per-node counterpart `NodeSpec.usage`) carries the run's inference usage attributed to graph position — see [Per-node Usage Attribution](per-node-usage-attribution.md) for the three usage states and the invariants a consumer must branch on.
+
`meta.format` is always `"mthds"` on newly emitted Pipelex graphs. `meta.mode` records provenance for renderers and shared tooling: Pipelex execution graphs emit `"dry"` for dry-run/mock execution and `"live"` for real execution. The shared renderer also accepts `"static"` for graphs produced by a static MTHDS graph builder.
### Node Types
@@ -517,6 +520,7 @@ validate_graphspec(graph_spec)
| File | Purpose |
|------|---------|
| `pipelex/graph/graphspec.py` | Canonical GraphSpec model |
+| `pipelex/tracing/usage_attribution.py` | Per-node usage accumulation and subtree rollup |
| `pipelex/graph/graph_tracer.py` | GraphTracer implementation |
| `pipelex/graph/graph_tracer_manager.py` | Singleton manager for tracers |
| `pipelex/graph/graph_tracer_protocol.py` | Protocol + NoOp implementation |
diff --git a/docs/under-the-hood/per-node-usage-attribution.md b/docs/under-the-hood/per-node-usage-attribution.md
new file mode 100644
index 000000000..5cc6af577
--- /dev/null
+++ b/docs/under-the-hood/per-node-usage-attribution.md
@@ -0,0 +1,150 @@
+---
+title: Per-node Usage Attribution
+description: "How Pipelex attributes token usage and cost to individual graph nodes: the two projections of one trace-event stream, the three usage states, and the invariants a GraphSpec consumer must branch on."
+---
+
+# Per-node Usage Attribution
+
+A run produces two artifacts that describe the same thing from different angles: `pipe_output.graph_spec` (the shape of the run) and `pipe_output.tokens_usages` (what it cost). Both are built from **one** trace-event stream, and every usage event in that stream already names the graph node it belongs to. This page is about the projection that keeps that link — `GraphSpec.usage` and `NodeSpec.usage` — and about the three states a usage number can be in, which is the part consumers get wrong.
+
+## Two projections of one event stream
+
+Every inference call emits a `UsageReportEvent` carrying `node_id` (from `trace_context.parent_node_id`) and the full internal `AnyTokensUsage`. That single stream is read twice:
+
+- **`UsageAggregator`** (`pipelex/tracing/usage_aggregator.py`) projects it into a flat list — one record per call, `node_id` dropped. That list becomes `pipe_output.tokens_usages`, trimmed to [`TokensUsageRecord`](tokens-usage-wire-records.md) at the client boundary, and feeds the console cost table and the CSV.
+- **`usage_attribution`** (`pipelex/tracing/usage_attribution.py`) projects it into per-node totals — `node_id` kept, calls folded together. Those totals become `NodeSpec.usage` and `GraphSpec.usage`, assembled by `GraphSpecAssembler`.
+
+The two projections are deliberately separate — `tokens_usages` is a shipped, `extra="forbid"` client contract, and a graph is not the place to re-litigate its record shape. What they are *not* allowed to do is disagree on arithmetic, so both compute every dollar through the same `compute_tokens_usage_cost` (`pipelex/cogt/usage/cost_registry.py`) and define a token total the same way `AggregatedCosts.total_nb_tokens` does.
+
+```mermaid
+flowchart TB
+ CALL["inference call completes"]
+ EVENT["UsageReportEvent<br/>{ node_id, tokens_usage }"]
+
+ subgraph LOG["event log — one stream, ordered by (workflow_id, sequence)"]
+ STREAM["PipeStart · PipeEndSuccess · Edge · UsageReport"]
+ end
+
+ subgraph ATTRIBUTED["attributed projection"]
+ direction TB
+ ASSEMBLER["GraphSpecAssembler<br/>node_id kept, calls folded"]
+ GRAPH["NodeSpec.usage<br/>GraphSpec.usage"]
+ end
+
+ subgraph FLATTENED["flat projection"]
+ direction TB
+ AGGREGATOR["UsageAggregator<br/>node_id dropped, one record per call"]
+ FLAT["PipeOutput.tokens_usages"]
+ SINKS["cost table · CSV · API wire"]
+ end
+
+ CALL --> EVENT
+ EVENT -- "the link exists here" --> STREAM
+ STREAM --> ASSEMBLER
+ STREAM --> AGGREGATOR
+ ASSEMBLER --> GRAPH
+ AGGREGATOR --> FLAT
+ FLAT --> SINKS
+```
+
+## When attribution happens, and why it waits
+
+`GraphSpecAssembler` folds usage events during pass 1, keyed by the `node_id` the event *named* — with no node lookup. Resolution against the real node set happens in pass 2. This is not an optimisation: a `UsageReportEvent` can legitimately be read **before** the `PipeStartEvent` of the node it names, because the two can come from different workers and the stream is ordered by `(workflow_id, sequence)`. "Is this node real?" is only answerable once every event has been seen.
+
+Pass 2 then does three things: resolves each accumulated total onto its node (or into the unattributed bucket), rolls subtrees up the `parent_node_id` chain, and computes the run total.
+
+## The three states of a usage number
+
+This is the part that matters for anything rendering a graph. A missing number means one of three things, and they must not be conflated — `$0.00` on an unrated node is a lie, and a controller that reports nothing because it was read through the wrong half looks like a bug.
+
+| State | How it is encoded | What a consumer may claim |
+|---|---|---|
+| **Not collected** | `usage is None` | nothing — no measurement was taken |
+| **Ran no inference** | `usage` present, `inference_calls == 0` | nothing (a controller may still report its subtree) |
+| **Unrated** | `cost is None` with `inference_calls > 0` | that calls were made; **no price** |
+| **Partial** | `inference_calls > rated_inference_calls > 0` | a price marked as a lower bound: `≥ $0.0043` |
+| **Rated** | `rated_inference_calls == inference_calls > 0` | the price: `$0.0043` |
+
+Unrated is not an edge case: `compute_tokens_usage_cost` returns `None` whenever the model carries no rate table, and dry/mock runs hardcode an empty rate table — so **every dry-run graph is unrated**.
+
+**A note on token counts, for anyone rendering them.** The counts are faithful to what the run reported, which is not the same as being meaningful to a reader. Two cases to handle before putting a number on screen: a dry run reports one synthetic call per would-be inference with zero tokens (or, under `--mock-usage`, with *invented* non-zero tokens — never display those, they measure nothing), and a `PipeSearch` reports its provider's billing unit in the token field, so a single search shows ~2,000,000 "tokens" for a correct price of $0.005. The `cost` is the number that survives both; `mthds-ui` accordingly displays cost only, and only for a real run.
+
+## The invariants
+
+`NodeUsageSpec` (`pipelex/graph/graphspec.py`) states four invariants in its docstring. They are the contract; each has a named test in `tests/unit/pipelex/tracing/test_usage_attribution.py`.
+
+1. **`usage is None` is all-or-nothing across a graph.** It means no usage was reported anywhere in the run — collection was off, or the run made zero inference calls. As soon as one usage event was seen, **every** node carries a spec, zeroed where nothing ran. A controller, a lifted pipe and a `PipeFunc` all get `inference_calls=0`, never `usage=None`. So the field never distinguishes "this node was not measured" from "that node was".
+2. **`cost is None` ⟺ `rated_inference_calls == 0`.** Nothing else. "Made no call" and "made only unrated calls" both land here, and `inference_calls` tells them apart.
+3. **`inference_calls > rated_inference_calls > 0` means `cost` is a lower bound**, not a total — some of this node's calls carried no rate table. A UI that renders it as a complete figure is silently wrong about money.
+4. **`total_tokens` is input_joined + output**, the same definition as `AggregatedCosts.total_nb_tokens`. It is **not** the sum of `nb_tokens_by_category`: `input_cached` is a *subset* of `input`, not additive, so summing the dict double-counts. Never sum the dict; read `total_tokens`.
+
+## The model that ran, vs the model that was asked for
+
+A GraphSpec names a model in three places, and only one of them is the outcome:
+
+| Rung | Where | Example |
+|---|---|---|
+| Authored choice | pipe blueprint `llm_choices.for_text` | `$writing-factual` (a preset) |
+| Requested handle | `execution_data.resolved_model` | `@default-premium` (still an alias) |
+| **What actually ran** | `NodeUsageSpec.by_model[].inference_model_name` | `claude-4.6-sonnet` |
+
+`execution_data.resolved_model` is captured from `LLMSetting.model` (`pipe_llm.py`), which is a *handle*: alias resolution happens later, at inference time. The name overstates what it holds — roughly a third of the nodes in a typical corpus carry an unresolved `@alias` there. Only `by_model` survives alias resolution, deck defaults, and any fallback or retry that landed somewhere other than what was requested.
+
+**`by_model` is a list because one node routinely uses more than one model.** A `PipeLLM`'s text pass and its object-structuring pass resolve separately (that is what `resolved_model_for_object` is about), and a retry can land elsewhere again. Collapsing them into "the model" would be wrong in exactly the way a single `cost` across mixed rated and unrated calls is wrong. Entries are ordered most-used first, ties broken by name, so a consumer can take `by_model[0]` as the dominant model without sorting. Per-model `cost` follows invariant 2: `None` iff no call to that model was priced.
+
+The same breakdown rolls up: `subtree_by_model` tells you every model a controller's whole branch used.
+
+## Own and subtree
+
+Every node carries both halves: its own inference, and its own plus every descendant's (`subtree_*`). A controller (`PipeSequence`, `PipeBatch`, `PipeParallel`) runs no inference itself, so its own numbers are always zero and only the subtree half says anything — read a controller through its own half and it reports nothing at all. Both tokens and cost roll up, so a consumer that shows either one has the branch figure available without walking the graph.
+
+The rollup is computed once, in the assembler, for the same reason `AggregatedCosts` computes its totals once: several consumers read the same GraphSpec, and they must not each re-derive it and disagree.
+
+```text
+ seq_node calls 0 (0 rated) subtree 3 calls (2 rated) subtree_cost 0.0071 subtree_tokens 4210
+ ├── llm_a calls 1 (1 rated) cost 0.0043 tokens 2130
+ ├── llm_b calls 1 (1 rated) cost 0.0028 tokens 2080
+ └── func_c calls 0 (0 rated) cost None tokens 0 ← ran, made no inference
+```
+
+`func_c` has a spec present with `cost=None` and `inference_calls=0` — invariants 1 and 2 together say "collected, ran nothing", never "unknown".
+
+`roll_up` tolerates two malformed-parentage cases rather than blowing up, because a partial cross-worker event read can produce either: a `parent_node_id` naming an absent node makes the child a root (never a `KeyError`), and a cycle in the parentage chain is broken by an in-progress guard and logged (never an infinite walk). The walk is memoized and uses an explicit stack, so it is O(nodes) and independent of the interpreter's recursion limit.
+
+## The run total and the unattributed bucket
+
+`GraphSpec.usage` is a `GraphUsageSpec` with two `NodeUsageSpec` fields:
+
+- **`total`** — every usage the run reported, attributed or not. This is the graph's comparand for the cost report's own total.
+- **`unattributed`** — the part that named no live node: the `UNATTRIBUTED_NODE_ID` (`"unknown"`) fallback stamped when an inference runs outside any pipe context, or a `node_id` whose `PipeStartEvent` never made it into the stream. Surfaced as its own bucket rather than dropped, so the graph's total can never silently disagree with the cost report's.
+
+On a healthy run `unattributed` is zero. A non-zero one is a real signal worth looking at.
+
+### Cross-checking against the cost report
+
+The two are computed from the same events but encode "unrated" differently: `CostRegistry.aggregate_costs` treats an unrated usage as zero-cost, while `NodeUsageSpec.cost` is `None`. Comparing them naively manufactures a disagreement. The correct check is:
+
+- `graph.usage.total.cost` equals the cost report's total **when `total.rated_inference_calls == total.inference_calls`**;
+- when they differ, the graph total is a lower bound and the report's total is the wrong comparand — compare `rated_inference_calls` against the report's row count instead.
+
+## The in-process tracer does not carry usage
+
+There are two GraphSpec builders. `GraphSpecAssembler` produces every consumed spec. The in-process `GraphTracer.teardown()` output is discarded at all three of its call sites (`runner.py`, `pipe_run.py`, `dry_run_in_process.py` — each reads `pipe_output.graph_spec`, i.e. the assembler's), and `GraphTracerProtocol` has no method to receive usage at all.
+
+So usage is deliberately assembler-only. That gap is *asserted* rather than normalized away: `tests/unit/pipelex/tracing/test_assembler_equivalence.py` runs a usage-bearing scenario through both builders, asserts they stay structurally equivalent, and asserts the divergence explicitly (assembler populated, tracer `None`). If someone later plumbs usage into the tracer, that test fails and tells them to delete the assertion — which is what a test documenting an intentional gap should do.
+
+## Files reference
+
+| File | Purpose |
+|------|---------|
+| `pipelex/graph/graphspec.py` | `NodeUsageSpec`, `GraphUsageSpec` and the four invariants |
+| `pipelex/tracing/usage_attribution.py` | `UsageAccumulator`, `roll_up`, `attribute_usage` |
+| `pipelex/tracing/graphspec_assembler.py` | Folds `UsageReportEvent`, attributes in pass 2 |
+| `pipelex/tracing/usage_aggregator.py` | The other projection — the flat `tokens_usages` list |
+| `pipelex/cogt/usage/cost_registry.py` | `compute_tokens_usage_cost` — the one cost engine |
+
+## Related pages
+
+- [Execution Graph Tracing](execution-graph-tracing.md) — the GraphSpec model and the trace event streams.
+- [TokensUsage Wire Records](tokens-usage-wire-records.md) — the client-facing shape of the other projection.
+- [Cost Tracking & Reporting](../features/cost-tracking.md) — the console table and CSV export.
diff --git a/docs/under-the-hood/pipelex-kernel.md b/docs/under-the-hood/pipelex-kernel.md
index f17f1fd8d..6063dd1b5 100644
--- a/docs/under-the-hood/pipelex-kernel.md
+++ b/docs/under-the-hood/pipelex-kernel.md
@@ -67,7 +67,7 @@ Both façade calls take the concept and the output class the caller wants, defau
| Module | Entry points |
|---|---|
| `pipelex.kernel.pipelex_kernel` | `PipelexKernel.make`, `.llm_text`, `.llm_object`, `.make_step_metadata` |
-| `pipelex.kernel.llm_ops` | `resolve_llm_setting_for_text` / `_for_object`, `derive_templating_style`, `derive_structure_prompt`, `generate_object_content`, `run_llm_text`, `run_llm_object` |
+| `pipelex.kernel.llm_ops` | `resolve_llm_setting_for_text` / `_for_object`, `concrete_llm_model_handle`, `derive_templating_style`, `derive_structure_prompt`, `generate_object_content`, `run_llm_text`, `run_llm_object` |
| `pipelex.kernel.extract_ops` | `resolve_extract_setting`, `build_extract_job_params`, `run_extract` |
| `pipelex.kernel.img_gen_ops` | `resolve_img_gen_setting`, `resolve_default_size`, `build_img_gen_job_params`, `run_img_gen` |
| `pipelex.kernel.search_ops` | `resolve_search_setting`, `run_search` |
diff --git a/docs/under-the-hood/tokens-usage-wire-records.md b/docs/under-the-hood/tokens-usage-wire-records.md
index 86724bf11..bb2cfe72a 100644
--- a/docs/under-the-hood/tokens-usage-wire-records.md
+++ b/docs/under-the-hood/tokens-usage-wire-records.md
@@ -66,3 +66,4 @@ For this reason the trim must never be implemented as a `field_serializer`/`mode
- [Cost Tracking & Reporting](../features/cost-tracking.md) — the in-process cost table and CSV export (full internal records, unaffected by this wire contract).
- [Execution Graph Tracing](execution-graph-tracing.md) — the sibling `graph_spec` artifact and the trace event streams.
+- [Per-node Usage Attribution](per-node-usage-attribution.md) — the *other* projection of the same usage events, which keeps `node_id` and folds calls per graph node.
diff --git a/mkdocs.yml b/mkdocs.yml
index 889a2b8b9..fbf022e79 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -316,6 +316,7 @@ plugins:
- under-the-hood/dry-run-mock-generation.md: "Dry Run Mock Generation"
- under-the-hood/execution-graph-tracing.md: "Execution Graph Tracing"
- under-the-hood/tokens-usage-wire-records.md: "TokensUsage Wire Records"
+ - under-the-hood/per-node-usage-attribution.md: "Per-node Usage Attribution"
- under-the-hood/image-handling-in-llm-prompts.md: "Image Handling in LLM Prompts"
- under-the-hood/reasoning-controls.md: "Reasoning Controls"
- under-the-hood/stuffartefact-and-image-rendering.md: "StuffArtefact & Image Rendering"
@@ -557,6 +558,7 @@ nav:
- Dry Run Mock Generation: under-the-hood/dry-run-mock-generation.md
- Execution Graph Tracing: under-the-hood/execution-graph-tracing.md
- TokensUsage Wire Records: under-the-hood/tokens-usage-wire-records.md
+ - Per-node Usage Attribution: under-the-hood/per-node-usage-attribution.md
- Image Handling in LLM Prompts: under-the-hood/image-handling-in-llm-prompts.md
- Reasoning Controls: under-the-hood/reasoning-controls.md
- StuffArtefact & Image Rendering: under-the-hood/stuffartefact-and-image-rendering.md
diff --git a/pipelex/graph/graphspec.py b/pipelex/graph/graphspec.py
index 159bfa6df..820e79a91 100644
--- a/pipelex/graph/graphspec.py
+++ b/pipelex/graph/graphspec.py
@@ -257,6 +257,131 @@ class ErrorSpec(BaseModel):
return _truncate_string(value, max_length=MAX_STACK_LENGTH)
+class ModelUsageSpec(BaseModel):
+ """What one inference model actually did for a node.
+
+ This is the only place the graph records the model that **ran**. Everything else
+ in a GraphSpec that names a model records what was *asked for*:
+ ``execution_data.resolved_model`` holds the handle the pipe resolved to — which may
+ still be an alias (``@default-premium``) — and the pipe blueprint holds the authored
+ choice (``$writing-factual``). Those are three rungs of one ladder, and only this
+ rung is the outcome: it survives alias resolution, deck defaults, and any fallback
+ or retry that landed somewhere other than what was requested.
+
+ A node genuinely uses more than one model in ordinary cases — a ``PipeLLM``'s text
+ pass and its object-structuring pass resolve separately — so a node's models are a
+ LIST. Collapsing them to "the model" would be wrong in exactly the way a single
+ ``cost`` for mixed rated/unrated calls is wrong.
+
+ ``cost`` follows ``NodeUsageSpec`` invariant 2: ``None`` iff no call to this model
+ carried a rate table.
+ """
+
+ model_config = ConfigDict(extra="forbid", strict=True)
+
+ inference_model_name: str
+ inference_model_id: str
+ # Kind of inference: "llm", "img_gen", "extract", "search". The discriminator a
+ # consumer needs before displaying token counts: extract/search/img_gen are billed
+ # PER REQUEST, and that price is encoded by putting 1_000_000 in each token
+ # category (rates are per-million), so their "tokens" are a scaled request counter.
+ model_type: str
+ inference_calls: int = 0
+ rated_inference_calls: int = 0
+ cost: float | None = None
+
+
+class NodeUsageSpec(BaseModel):
+ """Inference usage attributed to one graph node.
+
+ Field names mirror the already-shipped client-facing ``TokensUsageRecord``
+ (``reporting/usage_records.py``) so the graph does not introduce a fifth vocabulary
+ for numbers this codebase already names four ways (``TokenCategory``,
+ ``LLMTokenCostReportField``, ``GenAISpanAttr``, ``PostHogAttr``).
+
+ INVARIANTS — the UI and every other consumer branch on these, not on guesses about
+ why a number is missing:
+
+ 1. ``NodeSpec.usage is None`` <=> no usage was reported anywhere in the run —
+ either usage collection was off, or the run made zero inference calls. As soon
+ as ONE usage event was seen, EVERY node carries a spec, zeroed where nothing
+ ran. A controller, a lifted pipe, and a PipeFunc all get ``inference_calls=0``,
+ never ``usage=None``. So the field is all-or-nothing across a graph: it never
+ distinguishes "this node was not measured" from "that node was".
+
+ 2. ``cost is None`` <=> ``rated_inference_calls == 0``. Nothing else. "Made no
+ call" and "made only unrated calls" both land here and are told apart by
+ ``inference_calls``.
+
+ 3. ``inference_calls > rated_inference_calls > 0`` => ``cost`` is a LOWER BOUND,
+ not a total: some of this node's calls carried no rate table. The UI must mark
+ it (a leading "≥").
+
+ 4. ``total_tokens`` is input_joined + output — the same definition as
+ ``AggregatedCosts.total_nb_tokens``. It is NOT the sum of
+ ``nb_tokens_by_category``: ``input_cached`` is a SUBSET of ``input``, not
+ additive (see ``usage_records.py``), so summing double-counts. Never sum the
+ dict; read this field.
+
+ 5. ``cost_input`` + ``cost_output`` == ``cost`` (to float precision). They are
+ the same number split by direction, not extra charges, and they are None on
+ exactly the same condition.
+
+ 6. ``by_model`` names the models that actually RAN, and its ``inference_calls``
+ sum to this spec's own. It is a list because one node routinely uses more
+ than one model (a PipeLLM's text pass and its object pass resolve
+ separately). Ordered by descending calls, then by name, so a consumer can
+ take the first entry as the dominant model without sorting.
+
+ The same invariants hold for the ``subtree_*`` half, which covers this node plus
+ every descendant (rolled up in the assembler, once, so no consumer re-derives it
+ and disagrees).
+ """
+
+ model_config = ConfigDict(extra="forbid", strict=True)
+
+ # This node's own inference.
+ inference_calls: int = 0
+ rated_inference_calls: int = 0
+ nb_tokens_by_category: dict[str, int] = Field(default_factory=dict)
+ total_tokens: int = 0
+ cost: float | None = None
+ # Components of ``cost``: input is the joined input cost (non-cached + cached).
+ # Both follow invariant 2 alongside ``cost`` — None iff nothing was priced.
+ cost_input: float | None = None
+ cost_output: float | None = None
+ by_model: list[ModelUsageSpec] = Field(default_factory=empty_list_factory_of(ModelUsageSpec))
+
+ # This node plus every descendant.
+ subtree_inference_calls: int = 0
+ subtree_rated_inference_calls: int = 0
+ subtree_nb_tokens_by_category: dict[str, int] = Field(default_factory=dict)
+ subtree_total_tokens: int = 0
+ subtree_cost: float | None = None
+ subtree_cost_input: float | None = None
+ subtree_cost_output: float | None = None
+ subtree_by_model: list[ModelUsageSpec] = Field(default_factory=empty_list_factory_of(ModelUsageSpec))
+
+
+class GraphUsageSpec(BaseModel):
+ """Run-level inference usage for a whole GraphSpec.
+
+ ``total`` covers every usage the run reported, attributed or not — it is the graph's
+ comparand for the cost report's own total. ``unattributed`` is the part that named no
+ live node (the ``UNATTRIBUTED_NODE_ID`` fallback, or a node that never emitted a
+ start event): surfaced as its own bucket rather than dropped, so the graph's total
+ can never silently disagree with the cost report's.
+
+ Both reuse ``NodeUsageSpec`` — one usage shape in the contract, not two. Neither has
+ a subtree distinct from itself, so their ``subtree_*`` fields repeat their own.
+ """
+
+ model_config = ConfigDict(extra="forbid", strict=True)
+
+ total: NodeUsageSpec
+ unattributed: NodeUsageSpec
+
+
class NodeSpec(BaseModel):
"""Specification for a node in the execution graph.
@@ -283,6 +408,8 @@ class NodeSpec(BaseModel):
error: ErrorSpec | None = None
tags: dict[str, str] = Field(default_factory=dict)
metrics: dict[str, float] = Field(default_factory=dict)
+ # Inference usage attributed to this node; None under NodeUsageSpec invariant 1.
+ usage: NodeUsageSpec | None = None
execution_data: dict[str, Any] = Field(default_factory=dict)
@@ -318,6 +445,8 @@ class GraphSpec(BaseModel):
pipeline_ref: PipelineRef = Field(default_factory=PipelineRef)
nodes: list[NodeSpec] = Field(default_factory=empty_list_factory_of(NodeSpec))
edges: list[EdgeSpec] = Field(default_factory=empty_list_factory_of(EdgeSpec))
+ # Run-level usage rollup; None under NodeUsageSpec invariant 1, alongside every node's.
+ usage: GraphUsageSpec | None = None
meta: dict[str, Any] = Field(default_factory=dict)
pipe_registry: dict[str, dict[str, Any]] = Field(default_factory=dict)
concept_registry: dict[str, dict[str, Any]] = Field(default_factory=dict)
diff --git a/pipelex/kernel/llm_ops.py b/pipelex/kernel/llm_ops.py
index 36a2e2f3a..02bec7488 100644
--- a/pipelex/kernel/llm_ops.py
+++ b/pipelex/kernel/llm_ops.py
@@ -20,9 +20,11 @@ from typing import Any
from pipelex import log
from pipelex.cogt.content_generation.cogt_run_params import CogtRunParams
+from pipelex.cogt.exceptions import ModelChoiceNotFoundError
from pipelex.cogt.llm.llm_prompt import LLMPrompt
from pipelex.cogt.llm.llm_setting import LLMModelChoice, LLMSetting
from pipelex.cogt.model_backends.model_type import ModelType
+from pipelex.cogt.models.model_reference import ModelReferenceKind, ensure_model_reference
from pipelex.cogt.templating.template_rendering import render_template
from pipelex.config import get_config
from pipelex.core.concepts.concept import Concept
@@ -59,6 +61,53 @@ def resolve_llm_setting_for_object(*, llm_choice: LLMModelChoice | None = None,
return model_deck.get_llm_setting(llm_choice=resolved_choice)
+# A resolution chain is at most a handful of hops (preset -> alias -> handle); this only
+# has to stop a malformed deck that points an alias back at itself.
+_MAX_MODEL_RESOLUTION_HOPS = 8
+
+
+def concrete_llm_model_handle(model: str) -> str:
+ """Resolve an LLM model reference down to a concrete handle, for display.
+
+ ``ModelDeck.get_llm_setting`` advances exactly ONE hop, and a preset's own model
+ field is frequently another alias — `$writing-factual` yields `@default-premium`,
+ not `claude-4.7-opus`. That is invisible during a real run, where the remaining hops
+ happen inside the call, but it means a DRY run can only ever report a half-resolved
+ handle. Following the chain here makes the answer to "which model does this pipe
+ use" identical whether or not the pipe ran.
+
+ A WATERFALL is returned as-is and deliberately so: which of its models serves a call
+ depends on availability at that moment, so there is no answer to give in advance —
+ and inventing one would be worse than naming the waterfall.
+
+ Never raises: an unknown reference is returned unchanged. This feeds an
+ observability field, and a graph must not fail to assemble because a deck entry is
+ missing.
+ """
+ model_deck = get_model_deck()
+ seen: set[str] = set()
+ current = model
+ for _hop in range(_MAX_MODEL_RESOLUTION_HOPS):
+ reference = ensure_model_reference(current)
+ match reference.kind:
+ case ModelReferenceKind.PRESET | ModelReferenceKind.ALIAS:
+ if current in seen:
+ log.warning(f"Cycle resolving model reference '{model}' at '{current}'; reporting it unresolved")
+ return current
+ seen.add(current)
+ try:
+ setting = model_deck.get_llm_setting(llm_choice=current)
+ except ModelChoiceNotFoundError:
+ return current
+ if setting.model == current:
+ return current
+ current = setting.model
+ case ModelReferenceKind.WATERFALL | ModelReferenceKind.HANDLE:
+ return current
+ log.warning(f"Model reference '{model}' did not resolve within {_MAX_MODEL_RESOLUTION_HOPS} hops")
+ return current
+
+
def derive_templating_style(*, llm_setting: LLMSetting) -> TemplatingStyle | None:
"""The configured prompting style for a resolved setting, via its prompting target.
diff --git a/pipelex/pipe_controllers/condition/pipe_condition.py b/pipelex/pipe_controllers/condition/pipe_condition.py
index 54f83aeee..001e5dfb9 100644
--- a/pipelex/pipe_controllers/condition/pipe_condition.py
+++ b/pipelex/pipe_controllers/condition/pipe_condition.py
@@ -393,8 +393,14 @@ class PipeCondition(PipeController):
)
raise PipeRunError(message=msg, run_mode=pipe_run_params.run_mode, pipe_code=self.code)
- # Here, it should launch the dry run of all the pipes in the outcomes map
- for pipe_code in self.pipe_dependencies():
+ # Here, it should launch the dry run of all the pipes in the outcomes map.
+ # pipe_dependencies() is a set, and every branch dry-runs into the SAME
+ # working memory under the SAME output_name — so the branch iterated last
+ # is the one whose stuff the caller sees as this condition's output. Left
+ # unsorted, that is string-hash order, which varies per process: the same
+ # bundle dry-run twice yields two different graphs. Sort so the last writer
+ # is a property of the method, not of PYTHONHASHSEED.
+ for pipe_code in sorted(self.pipe_dependencies()):
pipe = get_required_pipe(pipe_code=pipe_code)
await pipe.run_pipe(
job_metadata=job_metadata,
diff --git a/pipelex/pipe_operators/llm/pipe_llm.py b/pipelex/pipe_operators/llm/pipe_llm.py
index 2ec6f601f..85940e543 100644
--- a/pipelex/pipe_operators/llm/pipe_llm.py
+++ b/pipelex/pipe_operators/llm/pipe_llm.py
@@ -21,6 +21,7 @@ from pipelex.core.pipes.variable_multiplicity import VariableMultiplicity
from pipelex.core.stuffs.stuff_content import StuffContent
from pipelex.interpreter_hub import get_concept_library, get_native_concept, get_required_concept
from pipelex.kernel.llm_ops import (
+ concrete_llm_model_handle,
derive_templating_style,
resolve_llm_setting_for_object,
resolve_llm_setting_for_text,
@@ -308,8 +309,12 @@ class PipeLLM(PipeOperator[PipeLLMOutput]):
# Capture execution data for the graph tracer
execution_data_dict: dict[str, Any] = {
- "resolved_model": llm_setting_main.model,
- "resolved_model_for_object": llm_setting_for_object.model,
+ # Fully resolved, not `llm_setting_*.model`: the deck advances one hop, so a
+ # preset lands on another alias and a DRY run could only ever report a
+ # half-resolved handle. Resolving here makes the reported model identical
+ # whether or not the pipe ran.
+ "resolved_model": concrete_llm_model_handle(llm_setting_main.model),
+ "resolved_model_for_object": concrete_llm_model_handle(llm_setting_for_object.model),
"is_multiple_output": is_multiple_output,
"rendered_system_prompt": rendered_llm_prompt.system_text,
"rendered_user_prompt": rendered_llm_prompt.user_text,
diff --git a/pipelex/pipe_operators/structure/pipe_structure.py b/pipelex/pipe_operators/structure/pipe_structure.py
index 25efd740e..3876262e7 100644
--- a/pipelex/pipe_operators/structure/pipe_structure.py
+++ b/pipelex/pipe_operators/structure/pipe_structure.py
@@ -18,7 +18,7 @@ from pipelex.core.pipes.pipe_output import PipeOutput
from pipelex.core.pipes.variable_multiplicity import VariableMultiplicity
from pipelex.core.stuffs.stuff_content import StuffContent
from pipelex.interpreter_hub import get_concept_library, get_native_concept
-from pipelex.kernel.llm_ops import derive_structure_prompt, generate_object_content, resolve_llm_setting_for_object
+from pipelex.kernel.llm_ops import concrete_llm_model_handle, derive_structure_prompt, generate_object_content, resolve_llm_setting_for_object
from pipelex.kernel.memory_ops import store_result
from pipelex.pipe_operators.pipe_operator import PipeOperator
from pipelex.pipe_run.pipe_run_params import PipeRunParams, output_multiplicity_to_apply
@@ -167,7 +167,7 @@ class PipeStructure(PipeOperator[PipeStructureOutput]):
)
execution_data_dict: dict[str, Any] = {
- "resolved_model": llm_setting_for_object.model,
+ "resolved_model": concrete_llm_model_handle(llm_setting_for_object.model),
"is_multiple_output": is_multiple_output,
"rendered_user_prompt": rendered_user_prompt,
"structuring_path": "structure",
diff --git a/pipelex/pipe_run/delivery_executor.py b/pipelex/pipe_run/delivery_executor.py
index 7a9231f9d..534e8c876 100644
--- a/pipelex/pipe_run/delivery_executor.py
+++ b/pipelex/pipe_run/delivery_executor.py
@@ -152,6 +152,18 @@ class DeliveryExecutor:
client wire shape (``TokensUsageRecord``) — the same shape the ``/execute``
response carries on ``pipe_output.tokens_usages`` — never the internal
full-fidelity usage models.
+
+ TODO: the envelope carries no RUN TOTAL, so every consumer that wants "what did
+ this run cost" sums the list itself. The number already exists as
+ ``GraphSpec.usage.total`` — but only on the graph, an optional and often large
+ RENDERING artifact, which is the wrong contract to read a price from (turn off
+ ``is_generate_graph`` and the total disappears while this file is still written).
+ Store it here instead, from the same ``compute_tokens_usage_cost`` engine the
+ graph rollup and the cost report both go through, so two artifacts of one run can
+ never quote different prices. Keep the three-valued semantics rather than
+ flattening them: ``None`` means unrated and must never render as zero, and a
+ partially rated run's total is a lower bound. Tracked as T-5 in
+ ``pipelex-server/TODOS.md``.
"""
usage_doc: dict[str, Any] = {
"tokens_usages": dump_tokens_usage_records(pipe_output.tokens_usages),
diff --git a/pipelex/reporting/reporting_manager.py b/pipelex/reporting/reporting_manager.py
index 5b6a60d12..b45cc11a2 100644
--- a/pipelex/reporting/reporting_manager.py
+++ b/pipelex/reporting/reporting_manager.py
@@ -18,7 +18,7 @@ from pipelex.system.exceptions import MissingDependencyError
from pipelex.system.trace_context import TraceContext
from pipelex.tracing.activity_event_log import ActivityEventLogCache
from pipelex.tracing.event_log_protocol import EventLogProtocol
-from pipelex.tracing.trace_events import UsageReportEvent
+from pipelex.tracing.trace_events import UNATTRIBUTED_NODE_ID, UsageReportEvent
# DynamoDB PutItem failures come in two sibling botocore base classes (neither subclasses the other):
# ClientError (service-side throttle / auth) and BotoCoreError (transport / credential / timeout, e.g.
@@ -216,7 +216,7 @@ class ReportingManager(ReportingProtocol):
tokens_usage: AnyTokensUsage,
) -> None:
"""Fast-path emit through a context registered via set_event_log."""
- node_id = trace_context.parent_node_id or "unknown"
+ node_id = trace_context.parent_node_id or UNATTRIBUTED_NODE_ID
seq = context.event_log.next_sequence()
event = UsageReportEvent(
@@ -290,7 +290,7 @@ class ReportingManager(ReportingProtocol):
return
workflow_id = trace_context.tracer_key or trace_context.graph_id
- node_id = trace_context.parent_node_id or "unknown"
+ node_id = trace_context.parent_node_id or UNATTRIBUTED_NODE_ID
ActivityEventLogCache.log_once_runner_fallback_engaged(workflow_id=workflow_id, writer_id=process_event_log.writer_id)
diff --git a/pipelex/tracing/graphspec_assembler.py b/pipelex/tracing/graphspec_assembler.py
index c75fe3412..7feab95ba 100644
--- a/pipelex/tracing/graphspec_assembler.py
+++ b/pipelex/tracing/graphspec_assembler.py
@@ -17,17 +17,20 @@ from pipelex.graph.graphspec import (
ErrorSpec,
GraphSpec,
GraphSpecMode,
+ GraphUsageSpec,
IOSpec,
NodeIOSpec,
NodeKind,
NodeSpec,
NodeStatus,
+ NodeUsageSpec,
PipelineRef,
TimingSpec,
make_graphspec_meta,
output_digest_is_optional,
)
from pipelex.tracing.trace_events import (
+ UNATTRIBUTED_NODE_ID,
BatchAggregateEvent,
BatchItemEvent,
ControllerOutputEvent,
@@ -41,6 +44,7 @@ from pipelex.tracing.trace_events import (
TraceEvent,
UsageReportEvent,
)
+from pipelex.tracing.usage_attribution import NodeUsages, attribute_usage, make_self_contained_spec
class _AssemblerNodeData:
@@ -73,6 +77,7 @@ class _AssemblerNodeData:
self.status: NodeStatus = NodeStatus.RUNNING
self.skip_reason: str | None = None
self.metrics: dict[str, float] = {}
+ self.usage_spec: NodeUsageSpec | None = None
self.error: ErrorSpec | None = None
self.input_specs: list[IOSpec] = input_specs or []
self.output_specs: list[IOSpec] = []
@@ -105,6 +110,7 @@ class _AssemblerNodeData:
node_io=node_io,
error=self.error,
metrics=self.metrics,
+ usage=self.usage_spec,
execution_data=self.execution_data,
)
@@ -113,9 +119,17 @@ class GraphSpecAssembler:
"""Reconstructs a GraphSpec from trace events.
Two-pass algorithm:
- - Pass 1: Build nodes, producer map, and collect metadata from all events.
+ - Pass 1: Build nodes, producer map, per-node usage totals, and collect metadata
+ from all events.
- Pass 2: Generate DATA, BATCH_ITEM, BATCH_AGGREGATE, PARALLEL_COMBINE edges
- using the complete producer map.
+ using the complete producer map, then attribute usage — resolve each usage
+ total against the now-complete node set, roll subtrees up the parentage chain,
+ and compute the run total.
+
+ Usage attribution deliberately waits for pass 2: a ``UsageReportEvent`` can be read
+ before the ``PipeStartEvent`` of the node it names (they can come from different
+ workers, and the stream is ordered by (workflow_id, sequence)), so "is this node
+ real?" is only answerable once every event has been seen.
"""
@staticmethod
@@ -169,6 +183,12 @@ class _AssemblerState:
# Edges generated in Pass 2 (DATA, BATCH_ITEM, etc.)
self._generated_edges: list[EdgeSpec] = []
+ # Usage records, keyed by the node_id the event named — resolved against the
+ # real node set in pass 2 (see the class docstring). Raw records, not totals:
+ # the arithmetic is the cost engine's, applied once in usage_attribution.
+ self._usage_by_reported_node: dict[str, NodeUsages] = {}
+ self._graph_usage: GraphUsageSpec | None = None
+
def pass_one(self, events: Sequence[TraceEvent]) -> None:
"""Pass 1: Build nodes, producer map, and collect metadata."""
for event in events:
@@ -195,7 +215,7 @@ class _AssemblerState:
elif isinstance(event, ExecutionDataEvent):
self._handle_execution_data(event)
elif isinstance(event, UsageReportEvent):
- pass # Handled by UsageAggregator
+ self._handle_usage_report(event)
else:
log.warning(f"Unknown event type: {type(event).__name__}")
@@ -203,11 +223,12 @@ class _AssemblerState:
self._mark_canceled_nodes()
def pass_two(self) -> None:
- """Pass 2: Generate edges using the complete producer map."""
+ """Pass 2: Generate edges using the complete producer map, then attribute usage."""
self._generate_data_edges()
self._generate_batch_item_edges()
self._generate_batch_aggregate_edges()
self._generate_parallel_combine_edges()
+ self._attribute_usage()
def build_graph_spec(self) -> GraphSpec:
"""Build the final GraphSpec from accumulated state."""
@@ -220,6 +241,7 @@ class _AssemblerState:
pipeline_ref=self._pipeline_ref,
nodes=nodes,
edges=all_edges,
+ usage=self._graph_usage,
meta=make_graphspec_meta(mode=self._mode),
pipe_registry=dict(self._pipe_registry),
concept_registry=dict(self._concept_registry),
@@ -347,6 +369,15 @@ class _AssemblerState:
if event.output_spec.digest:
self._stuff_producer_map[event.output_spec.digest] = event.node_id
+ def _handle_usage_report(self, event: UsageReportEvent) -> None:
+ """Fold one inference call into the total of the node it named.
+
+ No node lookup here on purpose — see the class docstring: the node may not have
+ started yet as far as this stream is concerned. Resolution happens in
+ ``_attribute_usage``.
+ """
+ self._usage_by_reported_node.setdefault(event.node_id, []).append(event.tokens_usage)
+
def _handle_execution_data(self, event: ExecutionDataEvent) -> None:
node_data = self._nodes.get(event.node_id)
if node_data is None:
@@ -522,3 +553,46 @@ class _AssemblerState:
source_stuff_digest=branch_stuff_code,
target_stuff_digest=combined_stuff_code,
)
+
+ # ------------------------------------------------------------------
+ # Pass 2 usage attribution
+ # ------------------------------------------------------------------
+
+ def _attribute_usage(self) -> None:
+ """Resolve the accumulated usage against the node set, roll it up, and total it.
+
+ Nothing is emitted at all when the stream reported no usage: every node keeps
+ ``usage=None`` and so does the graph (``NodeUsageSpec`` invariant 1, the
+ all-or-nothing half). As soon as one usage event was seen, EVERY node gets a
+ spec — zeroed for controllers, PipeFunc, skipped and lifted pipes, which run no
+ inference of their own and must not read as unmeasured.
+ """
+ if not self._usage_by_reported_node:
+ return
+
+ own_usages_by_node: dict[str, NodeUsages] = {node_id: [] for node_id in self._nodes}
+ unattributed_usages: NodeUsages = []
+ all_usages: NodeUsages = []
+ for reported_node_id, usages in self._usage_by_reported_node.items():
+ all_usages.extend(usages)
+ own_usages = own_usages_by_node.get(reported_node_id)
+ if own_usages is None:
+ if reported_node_id != UNATTRIBUTED_NODE_ID:
+ log.warning(f"UsageReportEvent for unknown node: {reported_node_id}")
+ unattributed_usages.extend(usages)
+ continue
+ own_usages.extend(usages)
+
+ usage_specs = attribute_usage(
+ own_usages_by_node=own_usages_by_node,
+ parent_by_node={node_id: node_data.parent_node_id for node_id, node_data in self._nodes.items()},
+ )
+ for node_id, usage_spec in usage_specs.items():
+ self._nodes[node_id].usage_spec = usage_spec
+
+ # The run total counts every usage the stream reported, attributed or not — it is
+ # what the cost report's own total must agree with.
+ self._graph_usage = GraphUsageSpec(
+ total=make_self_contained_spec(all_usages),
+ unattributed=make_self_contained_spec(unattributed_usages),
+ )
diff --git a/pipelex/tracing/trace_events.py b/pipelex/tracing/trace_events.py
index 521f24700..2ae77943d 100644
--- a/pipelex/tracing/trace_events.py
+++ b/pipelex/tracing/trace_events.py
@@ -179,6 +179,12 @@ class ExecutionDataEvent(TraceEvent):
execution_data: dict[str, Any] = Field(default_factory=dict)
+# node_id stamped on a UsageReportEvent emitted outside any pipe context (no
+# parent_node_id on the trace context). The assembler routes it to
+# GraphUsageSpec.unattributed rather than dropping it.
+UNATTRIBUTED_NODE_ID = "unknown"
+
+
class UsageReportEvent(TraceEvent):
"""Emitted when an inference job reports token usage."""
diff --git a/pipelex/tracing/usage_attribution.py b/pipelex/tracing/usage_attribution.py
new file mode 100644
index 000000000..f39b8b498
--- /dev/null
+++ b/pipelex/tracing/usage_attribution.py
@@ -0,0 +1,253 @@
+"""Attribution of inference usage to graph nodes.
+
+One event stream, two projections. ``UsageAggregator`` projects ``UsageReportEvent``
+into the flat, client-facing ``tokens_usages`` list (``node_id`` dropped, one record per
+call). This module projects the SAME events into per-node groups for the GraphSpec
+(``node_id`` kept), and hands each group to ``CostRegistry.aggregate_costs`` — the
+shipped cost engine that already produces the run's totals, its per-model grouping and
+its per-category token and cost columns.
+
+**No arithmetic lives here.** This module groups usage records by node, walks the
+parentage chain to build each node's subtree group, and maps ``AggregatedCosts`` onto
+the wire shape. Every number is the cost report's own, so the graph and the cost table
+cannot disagree by construction rather than by discipline.
+
+The one thing the engine cannot express is *unrated*: it treats a usage with no rate
+table as costing ``0.0``, while the graph must say ``None`` (a dry run and an unpriced
+model are not free, they are unpriced). That distinction is a count of the records whose
+``unit_costs`` is empty — see ``_rated_count`` — not a reimplementation of costing.
+"""
+
+from collections.abc import Mapping, Sequence
+
+from pipelex import log
+from pipelex.cogt.llm.llm_report import LLMTokenCostReportField
+from pipelex.cogt.usage.cost_registry import AggregatedCosts, CostRegistry
+from pipelex.graph.graphspec import ModelUsageSpec, NodeUsageSpec
+from pipelex.reporting.reporting_types import AnyTokensUsage
+
+# A node's usage is just the records attributed to it, kept in arrival order.
+NodeUsages = list[AnyTokensUsage]
+
+
+def _rated_count(usages: Sequence[AnyTokensUsage]) -> int:
+ """How many of these calls carried a rate table.
+
+ The single fact ``AggregatedCosts`` cannot carry: it prices an unrated call at
+ ``0.0``, which is indistinguishable from a genuinely free one. Comparing this to
+ ``len(usages)`` is what makes ``cost=None`` (nothing priced) and a partial cost
+ (some priced) expressible.
+ """
+ return sum(1 for tokens_usage in usages if tokens_usage.unit_costs)
+
+
+def _merge_token_categories(usages: Sequence[AnyTokensUsage]) -> dict[str, int]:
+ """Raw provider-reported counts, summed per category, exactly as reported.
+
+ Deliberately the untouched vocabulary rather than the cost report's derived
+ ``input_joined`` / ``input_non_cached`` columns: this mirrors the client-facing
+ ``TokensUsageRecord.nb_tokens_by_category``, so a consumer sees one token vocabulary
+ across the graph and the API.
+ """
+ merged: dict[str, int] = {}
+ for tokens_usage in usages:
+ for token_category, nb_tokens in tokens_usage.nb_tokens_by_category.items():
+ merged[token_category] = merged.get(token_category, 0) + nb_tokens
+ return merged
+
+
+def _model_specs(*, usages: Sequence[AnyTokensUsage], aggregated: AggregatedCosts) -> list[ModelUsageSpec]:
+ """Per-model breakdown, taken from the cost report's own grouping.
+
+ ``AggregatedCosts.grouped_by_model`` already splits tokens and costs per model name;
+ all this adds is the model id, the call counts, and the unrated ``None`` the engine
+ cannot express. Ordered most-used first (ties by name) so a consumer can read
+ ``by_model[0]`` as the dominant model without sorting.
+ """
+ calls_by_model: dict[str, int] = {}
+ rated_by_model: dict[str, int] = {}
+ model_ids: dict[str, str] = {}
+ for tokens_usage in usages:
+ model_name = tokens_usage.inference_model_name
+ calls_by_model[model_name] = calls_by_model.get(model_name, 0) + 1
+ model_ids.setdefault(model_name, tokens_usage.inference_model_id)
+ if tokens_usage.unit_costs:
+ rated_by_model[model_name] = rated_by_model.get(model_name, 0) + 1
+
+ specs: list[ModelUsageSpec] = []
+ for model_name, model_data in aggregated.grouped_by_model.items():
+ rated_calls = rated_by_model.get(model_name, 0)
+ model_cost = CostRegistry.compute_total_cost(
+ input_non_cached_cost=model_data[LLMTokenCostReportField.COST_INPUT_NON_CACHED],
+ input_cached_cost=model_data[LLMTokenCostReportField.COST_INPUT_CACHED],
+ output_cost=model_data[LLMTokenCostReportField.COST_OUTPUT],
+ )
+ specs.append(
+ ModelUsageSpec(
+ inference_model_name=model_name,
+ inference_model_id=model_ids.get(model_name, ""),
+ model_type=aggregated.model_types.get(model_name, "llm"),
+ inference_calls=calls_by_model.get(model_name, 0),
+ rated_inference_calls=rated_calls,
+ cost=model_cost if rated_calls else None,
+ )
+ )
+ specs.sort(key=lambda spec: (-spec.inference_calls, spec.inference_model_name, spec.inference_model_id))
+ return specs
+
+
+class _ScopedUsage:
+ """One scope's numbers — a node's own, or its whole subtree's.
+
+ Every field is read off ``AggregatedCosts``; the class exists only so the own half
+ and the subtree half are computed by identical code.
+ """
+
+ def __init__(self, usages: Sequence[AnyTokensUsage]) -> None:
+ aggregated = CostRegistry.aggregate_costs(tokens_usages=usages)
+ rated_calls = _rated_count(usages)
+
+ self.inference_calls: int = len(usages)
+ self.rated_inference_calls: int = rated_calls
+ self.nb_tokens_by_category: dict[str, int] = _merge_token_categories(usages)
+ self.total_tokens: int = aggregated.total_nb_tokens
+ # None, not 0.0, when nothing was priced: unpriced is not free.
+ self.cost: float | None = aggregated.total_cost if rated_calls else None
+ self.cost_input: float | None = None
+ self.cost_output: float | None = None
+ if rated_calls:
+ records = aggregated.records
+ self.cost_input = sum(record.get(LLMTokenCostReportField.COST_INPUT_JOINED, 0.0) for record in records)
+ self.cost_output = sum(record.get(LLMTokenCostReportField.COST_OUTPUT, 0.0) for record in records)
+ self.by_model: list[ModelUsageSpec] = _model_specs(usages=usages, aggregated=aggregated)
+
+
+def make_node_usage_spec(*, own_usages: Sequence[AnyTokensUsage], subtree_usages: Sequence[AnyTokensUsage]) -> NodeUsageSpec:
+ """Map a node's own usage records and its subtree's onto the wire shape."""
+ own = _ScopedUsage(own_usages)
+ subtree = _ScopedUsage(subtree_usages)
+ return NodeUsageSpec(
+ inference_calls=own.inference_calls,
+ rated_inference_calls=own.rated_inference_calls,
+ nb_tokens_by_category=own.nb_tokens_by_category,
+ total_tokens=own.total_tokens,
+ cost=own.cost,
+ cost_input=own.cost_input,
+ cost_output=own.cost_output,
+ by_model=own.by_model,
+ subtree_inference_calls=subtree.inference_calls,
+ subtree_rated_inference_calls=subtree.rated_inference_calls,
+ subtree_nb_tokens_by_category=subtree.nb_tokens_by_category,
+ subtree_total_tokens=subtree.total_tokens,
+ subtree_cost=subtree.cost,
+ subtree_cost_input=subtree.cost_input,
+ subtree_cost_output=subtree.cost_output,
+ subtree_by_model=subtree.by_model,
+ )
+
+
+def make_self_contained_spec(usages: Sequence[AnyTokensUsage]) -> NodeUsageSpec:
+ """Wire shape for a total with no subtree distinct from itself.
+
+ Used for ``GraphUsageSpec.total`` and ``GraphUsageSpec.unattributed``, whose
+ ``subtree_*`` fields repeat their own by definition.
+ """
+ return make_node_usage_spec(own_usages=usages, subtree_usages=usages)
+
+
+def roll_up(
+ *,
+ own_usages_by_node: Mapping[str, NodeUsages],
+ parent_by_node: Mapping[str, str | None],
+) -> dict[str, NodeUsages]:
+ """Collect each node's own usage records together with every descendant's.
+
+ Memoized post-order over the parentage chain: O(nodes), not O(nodes x depth). Two
+ malformed-parentage cases are tolerated rather than allowed to blow up, because both
+ are reachable from a partial cross-worker event read:
+
+ - a ``parent_node_id`` naming a node absent from ``own_usages_by_node`` — the child
+ is treated as a root (it contributes to no subtree but keeps its own records)
+ instead of raising ``KeyError``;
+ - a cycle in the parentage chain — broken by an in-progress guard and logged, so
+ the walk terminates instead of hanging. The rolled-up records for the nodes on
+ the cycle are then incomplete, which is why it is a WARNING and not silent.
+ """
+ children_by_node: dict[str, list[str]] = {}
+ for node_id in own_usages_by_node:
+ parent_node_id = parent_by_node.get(node_id)
+ if parent_node_id is None:
+ continue
+ if parent_node_id not in own_usages_by_node:
+ # Dangling parent: treat this node as a root rather than dropping it.
+ continue
+ children_by_node.setdefault(parent_node_id, []).append(node_id)
+
+ subtree_by_node: dict[str, NodeUsages] = {}
+ for start_node_id in own_usages_by_node:
+ if start_node_id in subtree_by_node:
+ continue
+ _walk_subtree(
+ start_node_id=start_node_id,
+ own_usages_by_node=own_usages_by_node,
+ children_by_node=children_by_node,
+ subtree_by_node=subtree_by_node,
+ )
+ return subtree_by_node
+
+
+def _walk_subtree(
+ *,
+ start_node_id: str,
+ own_usages_by_node: Mapping[str, NodeUsages],
+ children_by_node: Mapping[str, list[str]],
+ subtree_by_node: dict[str, NodeUsages],
+) -> None:
+ """Explicit-stack post-order walk filling ``subtree_by_node`` from ``start_node_id`` down.
+
+ An explicit stack rather than recursion: a deep pipeline must not depend on the
+ interpreter's recursion limit.
+ """
+ in_progress: set[str] = set()
+ stack: list[tuple[str, bool]] = [(start_node_id, False)]
+ while stack:
+ node_id, is_expanded = stack.pop()
+ if is_expanded:
+ collected: NodeUsages = list(own_usages_by_node[node_id])
+ for child_node_id in children_by_node.get(node_id, []):
+ child_subtree = subtree_by_node.get(child_node_id)
+ if child_subtree is not None:
+ collected.extend(child_subtree)
+ subtree_by_node[node_id] = collected
+ in_progress.discard(node_id)
+ continue
+
+ if node_id in subtree_by_node:
+ continue
+ if node_id in in_progress:
+ log.warning(f"Cycle in graph node parentage at '{node_id}'; subtree usage rollup will be incomplete")
+ continue
+
+ in_progress.add(node_id)
+ stack.append((node_id, True))
+ for child_node_id in children_by_node.get(node_id, []):
+ if child_node_id not in subtree_by_node:
+ stack.append((child_node_id, False))
+
+
+def attribute_usage(
+ *,
+ own_usages_by_node: Mapping[str, NodeUsages],
+ parent_by_node: Mapping[str, str | None],
+) -> dict[str, NodeUsageSpec]:
+ """Turn per-node usage records into per-node wire specs, subtree fields included.
+
+ Every node in ``own_usages_by_node`` gets a spec — zeroed where nothing ran. That is
+ ``NodeUsageSpec`` invariant 1: once a run reports any usage at all, no node is left
+ reading as "not collected".
+ """
+ subtree_by_node = roll_up(own_usages_by_node=own_usages_by_node, parent_by_node=parent_by_node)
+ return {
+ node_id: make_node_usage_spec(own_usages=own_usages, subtree_usages=subtree_by_node.get(node_id, own_usages))
+ for node_id, own_usages in own_usages_by_node.items()
+ }
diff --git a/subject_grants.toml b/subject_grants.toml
index 65172f3e6..6f33ecf7a 100644
--- a/subject_grants.toml
+++ b/subject_grants.toml
@@ -3468,6 +3468,10 @@ rationale = "Plugin entrypoint: registers into the registrar; single self-labell
param = "config"
rationale = "PipeFuncExecutorFactoryFn implementation: the registry invokes factories positionally via the Callable type."
+["pipelex/kernel/llm_ops.py::concrete_llm_model_handle"]
+param = "model"
+rationale = "Verb-object: resolve one model reference to a concrete handle; the single obvious operand is the model."
+
["pipelex/kit/cursor_rules.py::_is_pipelex_managed"]
param = "mdc_path"
rationale = "Predicate on the .mdc file path; single operand."
@@ -7064,6 +7068,10 @@ rationale = "Dispatch handler: handles the event; single operand."
param = "event"
rationale = "Dispatch handler: handles the event; single operand."
+["pipelex/tracing/graphspec_assembler.py::_AssemblerState._handle_usage_report"]
+param = "event"
+rationale = "Dispatch handler: handles the event; single operand."
+
["pipelex/tracing/graphspec_assembler.py::_AssemblerState._track_earliest_timestamp"]
param = "event"
rationale = "Verb-object: reads the event to track its timestamp; single operand."
@@ -7079,3 +7087,15 @@ rationale = "Derivation: encodes the workflow id into a file name; writer_id sta
["pipelex/tracing/usage_aggregator.py::UsageAggregator.aggregate"]
param = "events"
rationale = "Verb-object: aggregates the events; single operand."
+
+["pipelex/tracing/usage_attribution.py::_merge_token_categories"]
+param = "usages"
+rationale = "Verb-object: operates on one list of usage records; the single obvious operand is the usages."
+
+["pipelex/tracing/usage_attribution.py::_rated_count"]
+param = "usages"
+rationale = "Verb-object: operates on one list of usage records; the single obvious operand is the usages."
+
+["pipelex/tracing/usage_attribution.py::make_self_contained_spec"]
+param = "usages"
+rationale = "Verb-object: operates on one list of usage records; the single obvious operand is the usages."
diff --git a/tests/integration/pipelex/pipeline/test_dry_run_condition_branch_order.py b/tests/integration/pipelex/pipeline/test_dry_run_condition_branch_order.py
new file mode 100644
index 000000000..9def277e7
--- /dev/null
+++ b/tests/integration/pipelex/pipeline/test_dry_run_condition_branch_order.py
@@ -0,0 +1,145 @@
+"""Regression test: a dry run of a ``PipeCondition`` must not depend on set iteration order.
+
+``PipeCondition.pipe_dependencies()`` returns a ``set[str]``, and ``_dry_run_controller_pipe``
+dry-runs **every** branch into the same working memory under the same ``output_name`` — so the
+branch iterated last is the one whose stuff the caller sees as the condition's output. Iterating
+the raw set made that "last" a function of string-hash order, which varies per process: the same
+bundle dry-run in two processes produced two different graphs, and any consumer diffing graph
+specs (the mthds-ui fixture corpus, a static-vs-dry parity check) saw phantom changes.
+
+The loop sorts now. This test pins the property that makes the sort load-bearing: forcing the
+dependency set to iterate in either order must produce the same graph.
+"""
+
+from collections.abc import Iterator
+
+import pytest
+from pytest_mock import MockerFixture
+from typing_extensions import override
+
+from pipelex.pipe_controllers.condition.pipe_condition import PipeCondition
+from pipelex.pipeline.dry_run_pipeline import dry_run_pipeline
+
+_CONDITION_DOMAIN = "dry_run_condition_branch_order"
+_CONDITION_MTHDS = f"""
+domain = "{_CONDITION_DOMAIN}"
+description = "Sequence whose second step is a condition over two named branches"
+main_pipe = "respond"
+
+[pipe.respond]
+type = "PipeSequence"
+description = "Classify then route"
+inputs = {{ request = "Text" }}
+output = "Text"
+steps = [
+ {{ pipe = "classify", result = "classified" }},
+ {{ pipe = "route", result = "response" }},
+]
+
+[pipe.classify]
+type = "PipeLLM"
+description = "Classify the request"
+inputs = {{ request = "Text" }}
+output = "Text"
+prompt = "Classify $request"
+
+[pipe.route]
+type = "PipeCondition"
+description = "Route to one of two branches"
+inputs = {{ classified = "Text" }}
+output = "Text"
+expression = "classified"
+outcomes = {{ alpha = "alpha_branch", beta = "beta_branch" }}
+default_outcome = "alpha_branch"
+
+[pipe.alpha_branch]
+type = "PipeSequence"
+description = "Alpha branch, whose tail step names its own result"
+inputs = {{ classified = "Text" }}
+output = "Text"
+steps = [
+ {{ pipe = "write_alpha", result = "alpha_result" }},
+]
+
+[pipe.beta_branch]
+type = "PipeSequence"
+description = "Beta branch, whose tail step names its own result"
+inputs = {{ classified = "Text" }}
+output = "Text"
+steps = [
+ {{ pipe = "write_beta", result = "beta_result" }},
+]
+
+[pipe.write_alpha]
+type = "PipeLLM"
+description = "Alpha branch body"
+inputs = {{ classified = "Text" }}
+output = "Text"
+prompt = "Alpha $classified"
+
+[pipe.write_beta]
+type = "PipeLLM"
+description = "Beta branch body"
+inputs = {{ classified = "Text" }}
+output = "Text"
+prompt = "Beta $classified"
+"""
+
+
+class _OrderedDependencySet(set[str]):
+ """A dependency set that iterates in a caller-chosen order.
+
+ Subclasses ``set`` so every other ``pipe_dependencies()`` call site — membership,
+ emptiness, set arithmetic — keeps behaving exactly as before; only ``__iter__`` is
+ pinned, which is the single thing the dry-run loop is being tested against.
+ """
+
+ def __init__(self, *, values: list[str]) -> None:
+ super().__init__(values)
+ self._ordered = values
+
+ @override
+ def __iter__(self) -> Iterator[str]:
+ return iter(self._ordered)
+
+
[email protected](loop_scope="class")
+class TestDryRunConditionBranchOrder:
+ async def _graph_output_names(self, *, mocker: MockerFixture, reverse: bool) -> dict[str, list[str]]:
+ """Dry-run the bundle with the condition's dependency set iterating in a pinned order.
+
+ Two things this must NOT do, both learned the hard way:
+
+ - Do not replace the method unconditionally. Patching a CLASS attribute reaches
+ every ``PipeCondition`` alive in the process, and ``pipe_dependencies()`` is not
+ read only by the dry-run loop — library loading calls it to validate that each
+ outcome names a real pipe. An unconditional replacement tells every condition in
+ every bundle this worker loaded that it depends on this test's branches.
+ - Do not hard-code the dependency names. They come back domain-qualified
+ (``<domain>.alpha_branch``), and a bare code fails that same validation lookup.
+
+ So: delegate to the real implementation for every other pipe, and derive the order
+ from what it actually returns rather than restating it.
+ """
+ original = PipeCondition.pipe_dependencies
+
+ def ordered_dependencies(condition: PipeCondition) -> set[str]:
+ real = original(condition)
+ if condition.code != "route":
+ return real
+ return _OrderedDependencySet(values=sorted(real, reverse=reverse))
+
+ mocker.patch.object(PipeCondition, "pipe_dependencies", ordered_dependencies)
+ graph_spec, _ = await dry_run_pipeline(mthds_contents=[_CONDITION_MTHDS])
+ return {node.pipe_code: [output.name for output in node.node_io.outputs] for node in graph_spec.nodes if node.pipe_code}
+
+ async def test_branch_iteration_order_does_not_change_the_graph(self, mocker: MockerFixture) -> None:
+ """Both iteration orders must yield the same graph — that is what sorting buys."""
+ forward = await self._graph_output_names(mocker=mocker, reverse=False)
+ reverse = await self._graph_output_names(mocker=mocker, reverse=True)
+
+ assert forward == reverse
+ # Pin the value too, not just the agreement: the sorted loop ends on the branch whose
+ # pipe code sorts last, so the enclosing sequence's output is always beta's stuff.
+ # Without the sort these two runs disagree — alpha_result one way, beta_result the other.
+ assert forward["respond"] == ["beta_result"]
diff --git a/tests/unit/pipelex/graph/test_graphspec_roundtrip.py b/tests/unit/pipelex/graph/test_graphspec_roundtrip.py
index f4434d552..44ac2ecf9 100644
--- a/tests/unit/pipelex/graph/test_graphspec_roundtrip.py
+++ b/tests/unit/pipelex/graph/test_graphspec_roundtrip.py
@@ -3,17 +3,22 @@
from datetime import UTC, datetime
from pathlib import Path
+import pytest
+from pydantic import ValidationError
+
from pipelex.graph.graphspec import (
EdgeKind,
EdgeSpec,
ErrorSpec,
GraphSpec,
GraphSpecMode,
+ GraphUsageSpec,
IOSpec,
NodeIOSpec,
NodeKind,
NodeSpec,
NodeStatus,
+ NodeUsageSpec,
PipelineRef,
TimingSpec,
)
@@ -287,6 +292,107 @@ class TestGraphSpecRoundtrip:
if error_spec.stack and len(long_stack) > PreviewTruncationData.MAX_STACK_LENGTH:
assert error_spec.stack.endswith("...")
+ def test_roundtrip_with_usage_preserves_none_cost(self) -> None:
+ """An unrated node round-trips as unrated — never as $0.00 (NodeUsageSpec invariant 2)."""
+ node = NodeSpec(
+ node_id="node_unrated",
+ kind=NodeKind.OPERATOR,
+ pipe_code="generate_text",
+ pipe_type="PipeLLM",
+ status=NodeStatus.SUCCEEDED,
+ usage=NodeUsageSpec(
+ inference_calls=2,
+ rated_inference_calls=0,
+ nb_tokens_by_category={"input": 100, "input_cached": 40, "output": 50},
+ total_tokens=150,
+ cost=None,
+ subtree_inference_calls=2,
+ subtree_rated_inference_calls=0,
+ subtree_nb_tokens_by_category={"input": 100, "input_cached": 40, "output": 50},
+ subtree_total_tokens=150,
+ subtree_cost=None,
+ ),
+ )
+ graph = GraphSpec(
+ graph_id="usage_roundtrip",
+ created_at=ValidGraphData.CREATED_AT,
+ pipeline_ref=PipelineRef(),
+ nodes=[node],
+ edges=[],
+ usage=GraphUsageSpec(
+ total=NodeUsageSpec(inference_calls=2, total_tokens=150),
+ unattributed=NodeUsageSpec(),
+ ),
+ )
+
+ restored = GraphSpec.model_validate_json(graph.to_json())
+
+ restored_usage = restored.nodes[0].usage
+ assert restored_usage is not None
+ assert restored_usage.cost is None
+ assert restored_usage.subtree_cost is None
+ assert restored_usage.inference_calls == 2
+ assert restored_usage.rated_inference_calls == 0
+ assert restored_usage.total_tokens == 150
+ assert restored_usage.nb_tokens_by_category == {"input": 100, "input_cached": 40, "output": 50}
+ assert restored.usage is not None
+ assert restored.usage.total.total_tokens == 150
+ assert restored.usage.unattributed.inference_calls == 0
+
+ def test_roundtrip_with_rated_usage(self) -> None:
+ """A rated node keeps its dollar figure and its rated-call count through the wire."""
+ node = NodeSpec(
+ node_id="node_rated",
+ kind=NodeKind.OPERATOR,
+ pipe_code="generate_text",
+ pipe_type="PipeLLM",
+ status=NodeStatus.SUCCEEDED,
+ usage=NodeUsageSpec(inference_calls=1, rated_inference_calls=1, total_tokens=150, cost=0.0043),
+ )
+ graph = GraphSpec(
+ graph_id="usage_rated",
+ created_at=ValidGraphData.CREATED_AT,
+ pipeline_ref=PipelineRef(),
+ nodes=[node],
+ edges=[],
+ )
+
+ restored = GraphSpec.model_validate_json(graph.to_json())
+
+ restored_usage = restored.nodes[0].usage
+ assert restored_usage is not None
+ assert restored_usage.cost == 0.0043
+ assert restored_usage.rated_inference_calls == 1
+ assert restored.usage is None
+
+ def test_usage_forbids_unknown_fields(self) -> None:
+ """extra="forbid" on NodeUsageSpec: a stale or invented field fails loudly."""
+ with pytest.raises(ValidationError):
+ NodeUsageSpec.model_validate({"inference_calls": 1, "cost_usd": 0.01})
+
+ def test_roundtrip_without_usage_stays_none(self) -> None:
+ """A graph assembled with usage collection off keeps None everywhere (invariant 1)."""
+ graph = GraphSpec(
+ graph_id="no_usage",
+ created_at=ValidGraphData.CREATED_AT,
+ pipeline_ref=PipelineRef(),
+ nodes=[
+ NodeSpec(
+ node_id="node_001",
+ kind=NodeKind.OPERATOR,
+ pipe_code="test_pipe",
+ pipe_type="PipeLLM",
+ status=NodeStatus.SUCCEEDED,
+ )
+ ],
+ edges=[],
+ )
+
+ restored = GraphSpec.model_validate_json(graph.to_json())
+
+ assert restored.usage is None
+ assert restored.nodes[0].usage is None
+
def test_roundtrip_with_all_node_kinds(self) -> None:
"""Test round-trip preserves all NodeKind enum values."""
nodes = [
diff --git a/tests/unit/pipelex/kernel/test_llm_ops_model_resolution.py b/tests/unit/pipelex/kernel/test_llm_ops_model_resolution.py
new file mode 100644
index 000000000..931c084c4
--- /dev/null
+++ b/tests/unit/pipelex/kernel/test_llm_ops_model_resolution.py
@@ -0,0 +1,25 @@
+"""Tests for resolving a model reference down to a concrete handle."""
+
+import pytest
+
+from pipelex.kernel.llm_ops import concrete_llm_model_handle
+
+
+class TestConcreteLlmModelHandle:
+ """The reported model must be the same whether or not the pipe actually ran."""
+
+ def test_an_alias_resolves_in_one_hop(self) -> None:
+ assert concrete_llm_model_handle("@default-general") == "claude-4.6-sonnet"
+
+ def test_a_preset_follows_the_chain_past_its_alias(self) -> None:
+ """The case that made DRY and LIVE disagree: a preset's model is itself an alias."""
+ assert concrete_llm_model_handle("$writing-factual") == concrete_llm_model_handle("@default-premium")
+ assert not concrete_llm_model_handle("$writing-factual").startswith(("@", "$"))
+
+ def test_a_concrete_handle_is_returned_unchanged(self) -> None:
+ assert concrete_llm_model_handle("claude-4.6-sonnet") == "claude-4.6-sonnet"
+
+ def test_an_unknown_reference_is_returned_unchanged_rather_than_raising(self) -> None:
+ """This feeds an observability field; a missing deck entry must not fail assembly."""
+ assert concrete_llm_model_handle("@no-such-alias") == "@no-such-alias"
diff --git a/tests/unit/pipelex/tracing/test_assembler_equivalence.py b/tests/unit/pipelex/tracing/test_assembler_equivalence.py
index 6875ae24a..acfe51d5d 100644
--- a/tests/unit/pipelex/tracing/test_assembler_equivalence.py
+++ b/tests/unit/pipelex/tracing/test_assembler_equivalence.py
@@ -14,11 +14,16 @@ from typing import Any, Callable
import pytest
+from pipelex.cogt.llm.llm_report import LLMTokensUsage
+from pipelex.cogt.usage.cost_category import CostCategory
+from pipelex.cogt.usage.token_category import TokenCategory
from pipelex.graph.graph_tracer import GraphTracer
from pipelex.graph.graphspec import EdgeKind, EdgeSpec, GraphSpec, IOSpec, NodeKind, NodeSpec
+from pipelex.system.job_metadata import JobCategory, JobMetadata, UnitJobId
from pipelex.system.trace_context import TraceContext
from pipelex.tracing.graphspec_assembler import GraphSpecAssembler
from pipelex.tracing.in_memory_event_log import InMemoryEventLog
+from pipelex.tracing.trace_events import PipeStartEvent, UsageReportEvent
from tests.unit.pipelex.graph.conftest import make_defaulted_data_inclusion_config
# ---------------------------------------------------------------------------
@@ -422,6 +427,67 @@ def _scenario_pass_through(tracer: GraphTracer, context: TraceContext) -> None:
)
+def _scenario_with_usage(tracer: GraphTracer, context: TraceContext) -> None:
+ """A sequence whose child reports inference usage through the event log.
+
+ Feeds ``test_usage_is_the_one_intentional_divergence``: the assembler folds the
+ usage event, the in-process tracer has no channel to receive it at all.
+ """
+ started_at = _T0
+
+ parent_id, child_ctx = tracer.on_pipe_start(
+ trace_context=context,
+ pipe_code="sequence",
+ pipe_type="PipeSequence",
+ node_kind=NodeKind.CONTROLLER,
+ started_at=started_at,
+ )
+
+ child_id, _ = tracer.on_pipe_start(
+ trace_context=child_ctx,
+ pipe_code="gen_text",
+ pipe_type="PipeLLM",
+ node_kind=NodeKind.OPERATOR,
+ started_at=started_at + timedelta(seconds=1),
+ )
+ tracer.on_pipe_end_success(
+ node_id=child_id,
+ ended_at=started_at + timedelta(seconds=2),
+ output_spec=IOSpec(name="output_text", digest="digest_a"),
+ )
+ tracer.on_pipe_end_success(
+ node_id=parent_id,
+ ended_at=started_at + timedelta(seconds=3),
+ )
+
+
+def _usage_events_for(*, assembled_child_node_id: str, event_log: InMemoryEventLog) -> list[UsageReportEvent]:
+ """One rated LLM call attributed to ``assembled_child_node_id``."""
+ return [
+ UsageReportEvent(
+ pipeline_run_id=_PIPELINE_RUN_ID,
+ workflow_id=_WORKFLOW_ID,
+ writer_id=event_log.writer_id,
+ timestamp=_T0 + timedelta(seconds=2),
+ sequence=event_log.next_sequence(),
+ node_id=assembled_child_node_id,
+ tokens_usage=LLMTokensUsage(
+ job_metadata=JobMetadata(
+ user_id="user_test",
+ pipeline_run_id=_PIPELINE_RUN_ID,
+ pipe_code="gen_text",
+ unit_job_id=UnitJobId.LLM_GEN_TEXT,
+ job_category=JobCategory.LLM_JOB,
+ ),
+ inference_model_name="test-model",
+ inference_model_id="test-model-id",
+ unit_costs={CostCategory.INPUT: 3.0, CostCategory.OUTPUT: 15.0},
+ nb_tokens_by_category={TokenCategory.INPUT: 100, TokenCategory.OUTPUT: 50},
+ ),
+ )
+ ]
+
+
def _scenario_condition_selected_outcome(tracer: GraphTracer, context: TraceContext) -> None:
"""Condition pipe with SELECTED_OUTCOME edge."""
started_at = _T0
@@ -482,3 +548,54 @@ class TestAssemblerEquivalence:
"""GraphTracer teardown and GraphSpecAssembler produce structurally identical GraphSpecs."""
direct_spec, assembled_spec = _run_both_paths(scenario_fn)
_assert_graphs_equivalent(direct_spec, assembled_spec)
+
+ def test_usage_is_the_one_intentional_divergence(self) -> None:
+ """The two builders stay structurally equivalent, and diverge on `usage` on purpose.
+
+ The in-process GraphTracer has no channel to receive a UsageReportEvent —
+ ``GraphTracerProtocol`` has no method for it — and its GraphSpec is discarded at
+ every call site anyway (runner.py, pipe_run.py, dry_run_in_process.py all read
+ ``pipe_output.graph_spec``, i.e. the assembler's). So usage is deliberately
+ assembler-only. That gap is asserted here rather than normalized away: if someone
+ later plumbs usage into the tracer, this test fails and tells them to delete the
+ divergence assertion instead of leaving a silently rotting comparison.
+ """
+ data_inclusion = make_defaulted_data_inclusion_config()
+
+ tracer_direct = GraphTracer()
+ ctx_direct = tracer_direct.setup(graph_id=_GRAPH_ID, data_inclusion=data_inclusion)
+ _scenario_with_usage(tracer_direct, ctx_direct)
+ direct_spec = tracer_direct.teardown()
+ assert direct_spec is not None
+
+ event_log = InMemoryEventLog()
+ tracer_event = GraphTracer()
+ ctx_event = tracer_event.setup(
+ graph_id=_GRAPH_ID,
+ data_inclusion=data_inclusion,
+ event_log=event_log,
+ workflow_id=_WORKFLOW_ID,
+ pipeline_run_id=_PIPELINE_RUN_ID,
+ )
+ _scenario_with_usage(tracer_event, ctx_event)
+ tracer_event.teardown()
+
+ # The reporting manager emits usage against the node the inference ran under;
+ # replicate that here against the operator node the scenario produced.
+ operator_node_id = next(
+ event.node_id for event in event_log.read_events(_PIPELINE_RUN_ID) if isinstance(event, PipeStartEvent) and event.pipe_code == "gen_text"
+ )
+ for usage_event in _usage_events_for(assembled_child_node_id=operator_node_id, event_log=event_log):
+ event_log.emit(usage_event)
+
+ assembled_spec = GraphSpecAssembler.assemble(events=event_log.read_events(_PIPELINE_RUN_ID), graph_id=_GRAPH_ID)
+
+ # Everything the two builders both model still matches.
+ _assert_graphs_equivalent(direct_spec, assembled_spec)
+
+ # The intentional divergence.
+ assert direct_spec.usage is None
+ assert all(node.usage is None for node in direct_spec.nodes)
+ assert assembled_spec.usage is not None
+ assert assembled_spec.usage.total.inference_calls == 1
+ assert all(node.usage is not None for node in assembled_spec.nodes)
diff --git a/tests/unit/pipelex/tracing/test_assembler_usage_attribution.py b/tests/unit/pipelex/tracing/test_assembler_usage_attribution.py
new file mode 100644
index 000000000..402951445
--- /dev/null
+++ b/tests/unit/pipelex/tracing/test_assembler_usage_attribution.py
@@ -0,0 +1,294 @@
+"""Tests for usage attribution inside GraphSpecAssembler (NodeUsageSpec invariant 1, the run total)."""
+
+import math
+from datetime import UTC, datetime, timedelta
+from typing import Any
+
+from pipelex.cogt.llm.llm_report import LLMTokensUsage
+from pipelex.cogt.usage.cost_category import CostCategory, CostsByCategoryDict
+from pipelex.cogt.usage.token_category import TokenCategory
+from pipelex.graph.graphspec import GraphSpec, NodeKind, NodeSpec
+from pipelex.reporting.reporting_types import AnyTokensUsage
+from pipelex.system.job_metadata import JobCategory, JobMetadata, UnitJobId
+from pipelex.tracing.graphspec_assembler import GraphSpecAssembler
+from pipelex.tracing.trace_events import (
+ UNATTRIBUTED_NODE_ID,
+ PipeEndSuccessEvent,
+ PipeStartEvent,
+ TraceEvent,
+ UsageReportEvent,
+)
+
+_GRAPH_ID = "usage_graph"
+_PIPELINE_RUN_ID = "run_001"
+_WORKFLOW_ID = "wf_a"
+_T0 = datetime(2025, 6, 15, 10, 0, 0, tzinfo=UTC)
+
+_RATES: CostsByCategoryDict = {CostCategory.INPUT: 3.0, CostCategory.OUTPUT: 15.0}
+_UNRATED: CostsByCategoryDict = {}
+_COST_OF_ONE_RATED_CALL = 100 * 3.0 / 1_000_000 + 50 * 15.0 / 1_000_000
+
+_CONTROLLER_NODE_ID = f"{_GRAPH_ID}:{_WORKFLOW_ID}:node_ctrl"
+_LLM_NODE_ID = f"{_GRAPH_ID}:{_WORKFLOW_ID}:node_llm"
+_FUNC_NODE_ID = f"{_GRAPH_ID}:{_WORKFLOW_ID}:node_func"
+
+
+def _base(sequence: int) -> dict[str, Any]:
+ return {
+ "pipeline_run_id": _PIPELINE_RUN_ID,
+ "workflow_id": _WORKFLOW_ID,
+ "timestamp": _T0 + timedelta(seconds=sequence),
+ "sequence": sequence,
+ }
+
+
+def _make_usage(*, unit_costs: CostsByCategoryDict, model_name: str = "test-model") -> AnyTokensUsage:
+ return LLMTokensUsage(
+ job_metadata=JobMetadata(
+ user_id="user_test",
+ pipeline_run_id=_PIPELINE_RUN_ID,
+ pipe_code="test_pipe",
+ unit_job_id=UnitJobId.LLM_GEN_TEXT,
+ job_category=JobCategory.LLM_JOB,
+ ),
+ inference_model_name=model_name,
+ inference_model_id=f"{model_name}-id",
+ unit_costs=unit_costs,
+ nb_tokens_by_category={TokenCategory.INPUT: 100, TokenCategory.OUTPUT: 50},
+ )
+
+
+def _pipe_start(*, sequence: int, node_id: str, parent_node_id: str | None, node_kind: NodeKind, pipe_type: str) -> PipeStartEvent:
+ return PipeStartEvent(
+ **_base(sequence),
+ node_id=node_id,
+ parent_node_id=parent_node_id,
+ pipe_code="test_pipe",
+ pipe_type=pipe_type,
+ node_kind=node_kind,
+ )
+
+
+def _pipe_end(*, sequence: int, node_id: str) -> PipeEndSuccessEvent:
+ return PipeEndSuccessEvent(**_base(sequence), node_id=node_id, ended_at=_T0 + timedelta(seconds=sequence))
+
+
+def _usage_event(*, sequence: int, node_id: str, unit_costs: CostsByCategoryDict, model_name: str = "test-model") -> UsageReportEvent:
+ return UsageReportEvent(
+ **_base(sequence),
+ node_id=node_id,
+ tokens_usage=_make_usage(unit_costs=unit_costs, model_name=model_name),
+ )
+
+
+def _skeleton_events() -> list[TraceEvent]:
+ """A controller with one LLM child and one inference-free child, all completed."""
+ return [
+ _pipe_start(
+ sequence=0,
+ node_id=_CONTROLLER_NODE_ID,
+ parent_node_id=None,
+ node_kind=NodeKind.CONTROLLER,
+ pipe_type="PipeSequence",
+ ),
+ _pipe_start(
+ sequence=1,
+ node_id=_LLM_NODE_ID,
+ parent_node_id=_CONTROLLER_NODE_ID,
+ node_kind=NodeKind.OPERATOR,
+ pipe_type="PipeLLM",
+ ),
+ _pipe_end(sequence=2, node_id=_LLM_NODE_ID),
+ _pipe_start(
+ sequence=3,
+ node_id=_FUNC_NODE_ID,
+ parent_node_id=_CONTROLLER_NODE_ID,
+ node_kind=NodeKind.OPERATOR,
+ pipe_type="PipeFunc",
+ ),
+ _pipe_end(sequence=4, node_id=_FUNC_NODE_ID),
+ _pipe_end(sequence=5, node_id=_CONTROLLER_NODE_ID),
+ ]
+
+
+def _node_by_id(graph_spec: GraphSpec, *, node_id: str) -> NodeSpec:
+ return next(node for node in graph_spec.nodes if node.node_id == node_id)
+
+
+def _is_close(actual: float | None, *, expected: float) -> bool:
+ return actual is not None and math.isclose(actual, expected, rel_tol=1e-9)
+
+
+class TestAssemblerUsageAttribution:
+ """Tests for GraphSpecAssembler usage folding, rollup and totals."""
+
+ def test_invariant_1_no_usage_event_leaves_every_node_none(self) -> None:
+ """No usage in the stream: the graph and every node read as "not collected"."""
+ graph_spec = GraphSpecAssembler.assemble(events=_skeleton_events(), graph_id=_GRAPH_ID)
+
+ assert graph_spec.usage is None
+ assert all(node.usage is None for node in graph_spec.nodes)
+
+ def test_invariant_1_one_usage_event_gives_every_node_a_spec(self) -> None:
+ """One usage event: the controller and the inference-free child are zeroed, not None."""
+ events = [*_skeleton_events(), _usage_event(sequence=6, node_id=_LLM_NODE_ID, unit_costs=_RATES)]
+
+ graph_spec = GraphSpecAssembler.assemble(events=events, graph_id=_GRAPH_ID)
+
+ assert all(node.usage is not None for node in graph_spec.nodes)
+ func_usage = _node_by_id(graph_spec, node_id=_FUNC_NODE_ID).usage
+ assert func_usage is not None
+ assert func_usage.inference_calls == 0
+ assert func_usage.subtree_inference_calls == 0
+ assert func_usage.cost is None
+
+ def test_own_usage_lands_on_the_node_that_reported_it(self) -> None:
+ """The LLM node carries its own call; the controller carries none of its own."""
+ events = [*_skeleton_events(), _usage_event(sequence=6, node_id=_LLM_NODE_ID, unit_costs=_RATES)]
+
+ graph_spec = GraphSpecAssembler.assemble(events=events, graph_id=_GRAPH_ID)
+
+ llm_usage = _node_by_id(graph_spec, node_id=_LLM_NODE_ID).usage
+ assert llm_usage is not None
+ assert llm_usage.inference_calls == 1
+ assert llm_usage.rated_inference_calls == 1
+ assert llm_usage.total_tokens == 150
+ assert _is_close(llm_usage.cost, expected=_COST_OF_ONE_RATED_CALL)
+
+ controller_usage = _node_by_id(graph_spec, node_id=_CONTROLLER_NODE_ID).usage
+ assert controller_usage is not None
+ assert controller_usage.inference_calls == 0
+ assert controller_usage.cost is None
+
+ def test_subtree_rolls_up_to_the_controller(self) -> None:
+ """The controller reports its children's tokens and cost as its subtree."""
+ events = [
+ *_skeleton_events(),
+ _usage_event(sequence=6, node_id=_LLM_NODE_ID, unit_costs=_RATES),
+ _usage_event(sequence=7, node_id=_LLM_NODE_ID, unit_costs=_RATES),
+ ]
+
+ graph_spec = GraphSpecAssembler.assemble(events=events, graph_id=_GRAPH_ID)
+
+ controller_usage = _node_by_id(graph_spec, node_id=_CONTROLLER_NODE_ID).usage
+ assert controller_usage is not None
+ assert controller_usage.subtree_inference_calls == 2
+ assert controller_usage.subtree_total_tokens == 300
+ assert _is_close(controller_usage.subtree_cost, expected=2 * _COST_OF_ONE_RATED_CALL)
+
+ def test_unrated_usage_keeps_cost_none_all_the_way_up(self) -> None:
+ """A dry/mock run has tokens and no dollar — at the node and at the controller."""
+ events = [*_skeleton_events(), _usage_event(sequence=6, node_id=_LLM_NODE_ID, unit_costs=_UNRATED)]
+
+ graph_spec = GraphSpecAssembler.assemble(events=events, graph_id=_GRAPH_ID)
+
+ llm_usage = _node_by_id(graph_spec, node_id=_LLM_NODE_ID).usage
+ controller_usage = _node_by_id(graph_spec, node_id=_CONTROLLER_NODE_ID).usage
+ assert llm_usage is not None
+ assert controller_usage is not None
+ assert llm_usage.cost is None
+ assert llm_usage.total_tokens == 150
+ assert controller_usage.subtree_cost is None
+ assert controller_usage.subtree_total_tokens == 150
+
+ def test_unattributed_node_id_goes_to_the_graph_bucket(self) -> None:
+ """Usage emitted outside any pipe context is surfaced, not dropped."""
+ events = [
+ *_skeleton_events(),
+ _usage_event(sequence=6, node_id=UNATTRIBUTED_NODE_ID, unit_costs=_RATES),
+ ]
+
+ graph_spec = GraphSpecAssembler.assemble(events=events, graph_id=_GRAPH_ID)
+
+ assert graph_spec.usage is not None
+ assert graph_spec.usage.unattributed.inference_calls == 1
+ assert _is_close(graph_spec.usage.unattributed.cost, expected=_COST_OF_ONE_RATED_CALL)
+ assert all(node.usage is not None and node.usage.inference_calls == 0 for node in graph_spec.nodes)
+
+ def test_usage_for_a_node_that_never_started_goes_to_the_graph_bucket(self) -> None:
+ """A node_id with no PipeStartEvent still counts toward the run total."""
+ events = [
+ *_skeleton_events(),
+ _usage_event(sequence=6, node_id="graph:wf_other:node_ghost", unit_costs=_RATES),
+ ]
+
+ graph_spec = GraphSpecAssembler.assemble(events=events, graph_id=_GRAPH_ID)
+
+ assert graph_spec.usage is not None
+ assert graph_spec.usage.unattributed.inference_calls == 1
+ assert graph_spec.usage.total.inference_calls == 1
+
+ def test_run_total_covers_attributed_and_unattributed_usage(self) -> None:
+ """graph.usage.total is what the cost report's own total must agree with."""
+ events = [
+ *_skeleton_events(),
+ _usage_event(sequence=6, node_id=_LLM_NODE_ID, unit_costs=_RATES),
+ _usage_event(sequence=7, node_id=UNATTRIBUTED_NODE_ID, unit_costs=_RATES),
+ ]
+
+ graph_spec = GraphSpecAssembler.assemble(events=events, graph_id=_GRAPH_ID)
+
+ assert graph_spec.usage is not None
+ assert graph_spec.usage.total.inference_calls == 2
+ assert graph_spec.usage.total.total_tokens == 300
+ assert _is_close(graph_spec.usage.total.cost, expected=2 * _COST_OF_ONE_RATED_CALL)
+
+ own_calls = sum(node.usage.inference_calls for node in graph_spec.nodes if node.usage is not None)
+ assert graph_spec.usage.total.inference_calls == own_calls + graph_spec.usage.unattributed.inference_calls
+
+ def test_usage_event_read_before_the_node_started_still_attributes(self) -> None:
+ """Cross-worker ordering can put usage first; attribution waits for pass 2."""
+ events: list[TraceEvent] = [
+ _usage_event(sequence=0, node_id=_LLM_NODE_ID, unit_costs=_RATES),
+ *_skeleton_events(),
+ ]
+
+ graph_spec = GraphSpecAssembler.assemble(events=events, graph_id=_GRAPH_ID)
+
+ assert graph_spec.usage is not None
+ assert graph_spec.usage.unattributed.inference_calls == 0
+ llm_usage = _node_by_id(graph_spec, node_id=_LLM_NODE_ID).usage
+ assert llm_usage is not None
+ assert llm_usage.inference_calls == 1
+
+ def test_usage_survives_the_graphspec_json_roundtrip(self) -> None:
+ """The assembled usage is wire-stable, cost=None included."""
+ events = [
+ *_skeleton_events(),
+ _usage_event(sequence=6, node_id=_LLM_NODE_ID, unit_costs=_UNRATED),
+ ]
+ graph_spec = GraphSpecAssembler.assemble(events=events, graph_id=_GRAPH_ID)
+
+ restored = GraphSpec.model_validate_json(graph_spec.to_json())
+
+ assert restored.usage is not None
+ assert restored.usage.total.cost is None
+ assert restored.usage.total.inference_calls == 1
+ llm_usage = _node_by_id(restored, node_id=_LLM_NODE_ID).usage
+ assert llm_usage is not None
+ assert llm_usage.cost is None
+ assert llm_usage.total_tokens == 150
+
+ def test_by_model_records_the_models_that_actually_ran(self) -> None:
+ """The graph's only record of the model that RAN, as opposed to the one requested."""
+ events = [
+ *_skeleton_events(),
+ _usage_event(sequence=6, node_id=_LLM_NODE_ID, unit_costs=_RATES, model_name="claude-4.6-sonnet"),
+ _usage_event(sequence=7, node_id=_LLM_NODE_ID, unit_costs=_RATES, model_name="structurer"),
+ ]
+
+ graph_spec = GraphSpecAssembler.assemble(events=events, graph_id=_GRAPH_ID)
+
+ llm_usage = _node_by_id(graph_spec, node_id=_LLM_NODE_ID).usage
+ assert llm_usage is not None
+ assert [entry.inference_model_name for entry in llm_usage.by_model] == ["claude-4.6-sonnet", "structurer"]
+ assert [entry.inference_model_id for entry in llm_usage.by_model] == ["claude-4.6-sonnet-id", "structurer-id"]
+
+ # The controller carries none of its own but reports the branch's models.
+ controller_usage = _node_by_id(graph_spec, node_id=_CONTROLLER_NODE_ID).usage
+ assert controller_usage is not None
+ assert controller_usage.by_model == []
+ assert {entry.inference_model_name for entry in controller_usage.subtree_by_model} == {
+ "claude-4.6-sonnet",
+ "structurer",
+ }
diff --git a/tests/unit/pipelex/tracing/test_usage_attribution.py b/tests/unit/pipelex/tracing/test_usage_attribution.py
new file mode 100644
index 000000000..2377dd77f
--- /dev/null
+++ b/tests/unit/pipelex/tracing/test_usage_attribution.py
@@ -0,0 +1,326 @@
+"""Tests for per-node usage attribution: the wire mapping and the subtree rollup."""
+
+import math
+
+from pipelex.cogt.llm.llm_report import LLMTokensUsage
+from pipelex.cogt.usage.cost_category import CostCategory, CostsByCategoryDict
+from pipelex.cogt.usage.token_category import NbTokensByCategoryDict, TokenCategory
+from pipelex.graph.graphspec import NodeUsageSpec
+from pipelex.reporting.reporting_types import AnyTokensUsage
+from pipelex.system.job_metadata import JobCategory, JobMetadata, UnitJobId
+from pipelex.tracing.usage_attribution import attribute_usage, make_node_usage_spec, roll_up
+
+# Per-million-token USD, so 100 input + 50 output tokens cost 0.0003 + 0.00075.
+_RATES: CostsByCategoryDict = {CostCategory.INPUT: 3.0, CostCategory.OUTPUT: 15.0}
+_UNRATED: CostsByCategoryDict = {}
+_COST_OF_ONE_RATED_CALL = 100 * 3.0 / 1_000_000 + 50 * 15.0 / 1_000_000
+
+
+def _is_close(actual: float | None, *, expected: float) -> bool:
+ """Cost comparison that also pins down "not None" — a None cost is never "close"."""
+ return actual is not None and math.isclose(actual, expected, rel_tol=1e-9)
+
+
+def _make_usage(
+ *,
+ unit_costs: CostsByCategoryDict,
+ nb_tokens_by_category: NbTokensByCategoryDict | None = None,
+ model_name: str = "test-model",
+) -> AnyTokensUsage:
+ return LLMTokensUsage(
+ job_metadata=JobMetadata(
+ user_id="user_test",
+ pipeline_run_id="run_001",
+ pipe_code="test_pipe",
+ unit_job_id=UnitJobId.LLM_GEN_TEXT,
+ job_category=JobCategory.LLM_JOB,
+ ),
+ inference_model_name=model_name,
+ inference_model_id=f"{model_name}-id",
+ unit_costs=unit_costs,
+ nb_tokens_by_category=nb_tokens_by_category or {TokenCategory.INPUT: 100, TokenCategory.OUTPUT: 50},
+ )
+
+
+def _usages_with(*, unit_costs_per_call: list[CostsByCategoryDict]) -> list[AnyTokensUsage]:
+ return [_make_usage(unit_costs=unit_costs) for unit_costs in unit_costs_per_call]
+
+
+def _leaf_spec(*, unit_costs_per_call: list[CostsByCategoryDict]) -> NodeUsageSpec:
+ """The wire spec for a leaf node making these calls — a leaf's subtree is itself."""
+ usages = _usages_with(unit_costs_per_call=unit_costs_per_call)
+ return make_node_usage_spec(own_usages=usages, subtree_usages=usages)
+
+
+class TestUsageAttribution:
+ """Tests for make_node_usage_spec(), roll_up() and attribute_usage()."""
+
+ def test_one_rated_call(self) -> None:
+ """Counts, tokens and cost all land, and come from the shipped cost engine."""
+ spec = _leaf_spec(unit_costs_per_call=[_RATES])
+
+ assert spec.inference_calls == 1
+ assert spec.rated_inference_calls == 1
+ assert spec.nb_tokens_by_category == {TokenCategory.INPUT: 100, TokenCategory.OUTPUT: 50}
+ assert spec.total_tokens == 150
+ assert _is_close(spec.cost, expected=_COST_OF_ONE_RATED_CALL)
+
+ def test_several_calls_on_one_node_add_up(self) -> None:
+ """Two calls on one node add up, per category and in total."""
+ spec = _leaf_spec(unit_costs_per_call=[_RATES, _RATES])
+
+ assert spec.inference_calls == 2
+ assert spec.nb_tokens_by_category == {TokenCategory.INPUT: 200, TokenCategory.OUTPUT: 100}
+ assert spec.total_tokens == 300
+ assert _is_close(spec.cost, expected=2 * _COST_OF_ONE_RATED_CALL)
+
+ def test_invariant_2_unrated_keeps_cost_none(self) -> None:
+ """A call with no rate table counts as a call, and cost stays None — never 0.0.
+
+ This is the one thing ``AggregatedCosts`` cannot express: it prices an unrated
+ call at 0.0, which would render as "free" rather than "unpriced".
+ """
+ spec = _leaf_spec(unit_costs_per_call=[_UNRATED, _UNRATED])
+
+ assert spec.inference_calls == 2
+ assert spec.rated_inference_calls == 0
+ assert spec.cost is None
+ assert spec.total_tokens == 300
+
+ def test_invariant_2_no_call_is_distinguishable_from_only_unrated_calls(self) -> None:
+ """Both have cost None; inference_calls is what tells them apart."""
+ ran_nothing = _leaf_spec(unit_costs_per_call=[])
+ only_unrated = _leaf_spec(unit_costs_per_call=[_UNRATED])
+
+ assert ran_nothing.cost is None
+ assert only_unrated.cost is None
+ assert ran_nothing.inference_calls == 0
+ assert only_unrated.inference_calls == 1
+
+ def test_invariant_3_mixed_rated_and_unrated_is_a_lower_bound(self) -> None:
+ """Cost covers only the rated calls, and rated < calls says so."""
+ spec = _leaf_spec(unit_costs_per_call=[_RATES, _UNRATED])
+
+ assert spec.inference_calls == 2
+ assert spec.rated_inference_calls == 1
+ assert _is_close(spec.cost, expected=_COST_OF_ONE_RATED_CALL)
+
+ def test_invariant_4_total_tokens_is_not_the_sum_of_the_categories(self) -> None:
+ """input_cached is a subset of input, so summing the dict double-counts it."""
+ usages = [
+ _make_usage(
+ unit_costs=_RATES,
+ nb_tokens_by_category={
+ TokenCategory.INPUT: 100,
+ TokenCategory.INPUT_CACHED: 40,
+ TokenCategory.OUTPUT: 50,
+ },
+ )
+ ]
+ spec = make_node_usage_spec(own_usages=usages, subtree_usages=usages)
+
+ assert spec.total_tokens == 150
+ assert sum(spec.nb_tokens_by_category.values()) == 190
+
+ def test_invariant_5_input_and_output_costs_split_the_total(self) -> None:
+ """cost_input + cost_output == cost: one number by direction, not extra charges."""
+ spec = _leaf_spec(unit_costs_per_call=[_RATES, _RATES])
+
+ assert _is_close(spec.cost_input, expected=2 * 100 * 3.0 / 1_000_000)
+ assert _is_close(spec.cost_output, expected=2 * 50 * 15.0 / 1_000_000)
+ assert spec.cost_input is not None
+ assert spec.cost_output is not None
+ assert _is_close(spec.cost_input + spec.cost_output, expected=spec.cost or 0.0)
+
+ def test_component_costs_are_none_on_exactly_the_same_condition_as_cost(self) -> None:
+ """Invariant 2 covers the components too — unrated has no input or output price."""
+ for spec in (_leaf_spec(unit_costs_per_call=[_UNRATED]), _leaf_spec(unit_costs_per_call=[])):
+ assert spec.cost is None
+ assert spec.cost_input is None
+ assert spec.cost_output is None
+
+ def test_by_model_keeps_the_two_models_of_one_node_apart(self) -> None:
+ """A PipeLLM's text pass and object pass resolve separately — both must survive."""
+ usages = [
+ _make_usage(unit_costs=_RATES, model_name="sonnet"),
+ _make_usage(unit_costs=_RATES, model_name="sonnet"),
+ _make_usage(unit_costs=_RATES, model_name="structurer"),
+ ]
+ spec = make_node_usage_spec(own_usages=usages, subtree_usages=usages)
+
+ assert [entry.inference_model_name for entry in spec.by_model] == ["sonnet", "structurer"]
+ assert [entry.inference_calls for entry in spec.by_model] == [2, 1]
+ assert [entry.inference_model_id for entry in spec.by_model] == ["sonnet-id", "structurer-id"]
+ # The per-model calls account for every call the node made.
+ assert sum(entry.inference_calls for entry in spec.by_model) == spec.inference_calls
+
+ def test_by_model_is_ordered_by_calls_then_name(self) -> None:
+ """Most-used model first, so a consumer can take the first entry as dominant."""
+ usages = [_make_usage(unit_costs=_RATES, model_name=name) for name in ["alpha", "beta", "beta", "gamma", "gamma", "gamma"]]
+ spec = make_node_usage_spec(own_usages=usages, subtree_usages=usages)
+
+ assert [entry.inference_model_name for entry in spec.by_model] == ["gamma", "beta", "alpha"]
+
+ def test_by_model_cost_is_none_for_an_unrated_model(self) -> None:
+ """Invariant 2 holds per model: a model with no rate table reports no price."""
+ usages = [
+ _make_usage(unit_costs=_RATES, model_name="priced"),
+ _make_usage(unit_costs=_UNRATED, model_name="own-gpu"),
+ ]
+ spec = make_node_usage_spec(own_usages=usages, subtree_usages=usages)
+ by_name = {entry.inference_model_name: entry for entry in spec.by_model}
+
+ assert _is_close(by_name["priced"].cost, expected=_COST_OF_ONE_RATED_CALL)
+ assert by_name["priced"].rated_inference_calls == 1
+ assert by_name["own-gpu"].cost is None
+ assert by_name["own-gpu"].rated_inference_calls == 0
+ assert by_name["own-gpu"].inference_calls == 1
+
+ def test_by_model_carries_the_model_type_discriminator(self) -> None:
+ """model_type is what tells a consumer whether that model's tokens are real."""
+ spec = _leaf_spec(unit_costs_per_call=[_RATES])
+
+ assert spec.by_model[0].model_type == "llm"
+
+ def test_leaf_subtree_equals_own(self) -> None:
+ """A childless node's subtree is exactly itself."""
+ own_usages_by_node = {"leaf": _usages_with(unit_costs_per_call=[_RATES])}
+
+ spec = attribute_usage(own_usages_by_node=own_usages_by_node, parent_by_node={"leaf": None})["leaf"]
+
+ assert spec.subtree_inference_calls == 1
+ assert spec.subtree_total_tokens == 150
+ assert _is_close(spec.subtree_cost, expected=_COST_OF_ONE_RATED_CALL)
+
+ def test_controller_subtree_sums_its_children(self) -> None:
+ """A controller runs no inference of its own but reports its children's."""
+ own_usages_by_node: dict[str, list[AnyTokensUsage]] = {
+ "ctrl": [],
+ "llm_a": _usages_with(unit_costs_per_call=[_RATES]),
+ "llm_b": _usages_with(unit_costs_per_call=[_RATES]),
+ "func_c": [],
+ }
+ parent_by_node: dict[str, str | None] = {"ctrl": None, "llm_a": "ctrl", "llm_b": "ctrl", "func_c": "ctrl"}
+
+ usage_specs = attribute_usage(own_usages_by_node=own_usages_by_node, parent_by_node=parent_by_node)
+
+ controller_spec = usage_specs["ctrl"]
+ assert controller_spec.inference_calls == 0
+ assert controller_spec.cost is None
+ assert controller_spec.subtree_inference_calls == 2
+ assert controller_spec.subtree_total_tokens == 300
+ assert _is_close(controller_spec.subtree_cost, expected=2 * _COST_OF_ONE_RATED_CALL)
+ assert usage_specs["func_c"].subtree_inference_calls == 0
+ assert usage_specs["func_c"].subtree_cost is None
+
+ def test_nested_three_deep_counts_each_call_once(self) -> None:
+ """Every ancestor sees the whole subtree below it, once each."""
+ own_usages_by_node: dict[str, list[AnyTokensUsage]] = {
+ "root": [],
+ "mid": _usages_with(unit_costs_per_call=[_RATES]),
+ "leaf_a": _usages_with(unit_costs_per_call=[_RATES]),
+ "leaf_b": _usages_with(unit_costs_per_call=[_RATES]),
+ }
+ parent_by_node: dict[str, str | None] = {"root": None, "mid": "root", "leaf_a": "mid", "leaf_b": "mid"}
+
+ usage_specs = attribute_usage(own_usages_by_node=own_usages_by_node, parent_by_node=parent_by_node)
+
+ assert usage_specs["leaf_a"].subtree_inference_calls == 1
+ assert usage_specs["mid"].subtree_inference_calls == 3
+ assert usage_specs["root"].subtree_inference_calls == 3
+ assert _is_close(usage_specs["root"].subtree_cost, expected=3 * _COST_OF_ONE_RATED_CALL)
+
+ def test_all_unrated_subtree_keeps_subtree_cost_none(self) -> None:
+ """An entirely unrated subtree must not roll up into a $0.00 controller."""
+ own_usages_by_node: dict[str, list[AnyTokensUsage]] = {
+ "ctrl": [],
+ "llm_a": _usages_with(unit_costs_per_call=[_UNRATED]),
+ }
+ parent_by_node: dict[str, str | None] = {"ctrl": None, "llm_a": "ctrl"}
+
+ controller_spec = attribute_usage(own_usages_by_node=own_usages_by_node, parent_by_node=parent_by_node)["ctrl"]
+
+ assert controller_spec.subtree_cost is None
+ assert controller_spec.subtree_inference_calls == 1
+ assert controller_spec.subtree_total_tokens == 150
+
+ def test_subtree_by_model_merges_the_models_of_a_whole_branch(self) -> None:
+ """A controller reports every model its branch used, merged across children."""
+ own_usages_by_node: dict[str, list[AnyTokensUsage]] = {
+ "ctrl": [],
+ "left": [_make_usage(unit_costs=_RATES, model_name="sonnet")],
+ "right": [
+ _make_usage(unit_costs=_RATES, model_name="sonnet"),
+ _make_usage(unit_costs=_RATES, model_name="haiku"),
+ ],
+ }
+ parent_by_node: dict[str, str | None] = {"ctrl": None, "left": "ctrl", "right": "ctrl"}
+
+ controller_spec = attribute_usage(own_usages_by_node=own_usages_by_node, parent_by_node=parent_by_node)["ctrl"]
+
+ assert controller_spec.by_model == []
+ by_name = {entry.inference_model_name: entry for entry in controller_spec.subtree_by_model}
+ assert by_name["sonnet"].inference_calls == 2
+ assert by_name["haiku"].inference_calls == 1
+ assert _is_close(by_name["sonnet"].cost, expected=2 * _COST_OF_ONE_RATED_CALL)
+
+ def test_component_costs_roll_up_through_a_subtree(self) -> None:
+ """A controller's branch cost splits by direction the same way a leaf's does."""
+ own_usages_by_node: dict[str, list[AnyTokensUsage]] = {
+ "ctrl": [],
+ "llm_a": _usages_with(unit_costs_per_call=[_RATES]),
+ "llm_b": _usages_with(unit_costs_per_call=[_RATES]),
+ }
+ parent_by_node: dict[str, str | None] = {"ctrl": None, "llm_a": "ctrl", "llm_b": "ctrl"}
+
+ controller_spec = attribute_usage(own_usages_by_node=own_usages_by_node, parent_by_node=parent_by_node)["ctrl"]
+
+ assert controller_spec.cost_input is None
+ assert controller_spec.subtree_cost_input is not None
+ assert controller_spec.subtree_cost_output is not None
+ assert _is_close(
+ controller_spec.subtree_cost_input + controller_spec.subtree_cost_output,
+ expected=controller_spec.subtree_cost or 0.0,
+ )
+
+ def test_roll_up_survives_a_cycle_in_the_parent_chain(self) -> None:
+ """A parentage cycle must terminate, not hang, and still produce every node."""
+ own_usages_by_node = {
+ "node_a": _usages_with(unit_costs_per_call=[_RATES]),
+ "node_b": _usages_with(unit_costs_per_call=[_RATES]),
+ }
+ parent_by_node: dict[str, str | None] = {"node_a": "node_b", "node_b": "node_a"}
+
+ subtree_by_node = roll_up(own_usages_by_node=own_usages_by_node, parent_by_node=parent_by_node)
+
+ assert set(subtree_by_node.keys()) == {"node_a", "node_b"}
+ for usages in subtree_by_node.values():
+ assert len(usages) >= 1
+
+ def test_roll_up_treats_a_dangling_parent_as_a_root(self) -> None:
+ """A parent absent from the node set must not raise, and must not lose the child."""
+ own_usages_by_node = {
+ "orphan": _usages_with(unit_costs_per_call=[_RATES]),
+ "other": _usages_with(unit_costs_per_call=[_RATES]),
+ }
+ parent_by_node: dict[str, str | None] = {"orphan": "node_from_another_worker", "other": None}
+
+ usage_specs = attribute_usage(own_usages_by_node=own_usages_by_node, parent_by_node=parent_by_node)
+
+ assert usage_specs["orphan"].subtree_inference_calls == 1
+ assert _is_close(usage_specs["orphan"].subtree_cost, expected=_COST_OF_ONE_RATED_CALL)
+ assert usage_specs["other"].subtree_inference_calls == 1
+
+ def test_attribute_usage_gives_every_node_a_spec(self) -> None:
+ """Invariant 1: a node that ran no inference is zeroed, never left specless."""
+ own_usages_by_node: dict[str, list[AnyTokensUsage]] = {
+ "ctrl": [],
+ "llm_a": _usages_with(unit_costs_per_call=[_RATES]),
+ }
+ parent_by_node: dict[str, str | None] = {"ctrl": None, "llm_a": "ctrl"}
+
+ usage_specs = attribute_usage(own_usages_by_node=own_usages_by_node, parent_by_node=parent_by_node)
+
+ assert set(usage_specs.keys()) == {"ctrl", "llm_a"}
+ assert usage_specs["llm_a"].inference_calls == 1
+ assert usage_specs["ctrl"].inference_calls == 0
diff --git a/wip/costs-in-graph-spec.md b/wip/costs-in-graph-spec.md
new file mode 100644
index 000000000..c96f7416b
--- /dev/null
+++ b/wip/costs-in-graph-spec.md
@@ -0,0 +1,521 @@
+# Per-node cost and token usage in the GraphSpec
+
+Status: **implemented and verified, all six phases.** Not yet shipped: nothing reaches users until `@pipelex/mthds-ui` is published and `make refresh-graph-ui-sri` bumps the pinned bundle.
+
+Revision 2 — absorbs the Codex outside-voice pass. See "Outside voice absorbed" at the end for what changed and why.
+
+## Implementation status
+
+| Phase | State |
+|---|---|
+| 1 — pipelex model + assembler | done (T1-T5, T7) |
+| 2 — pipelex tests | done (T6 + `test_usage_attribution.py`, `test_assembler_usage_attribution.py`, roundtrip tests) |
+| 3 — pipelex docs + changelog | done (T8, new `docs/under-the-hood/per-node-usage-attribution.md`) |
+| 4 — DRY fixtures with `--mock-usage` | done (T10, T11 — full corpus regenerated, all green) |
+| 5 — mthds-ui | done (T9, T12 + `usageFormat.ts`, `UsageSection.tsx`, `docs/usage-attribution.md`) |
+| 6 — LIVE regeneration | done — all 32 pipelines, $2.85, zero failures |
+
+**CHECKPOINT 1 findings.** The rollup is correct on the deepest fixture: `pipeline_24` (DEEP_NESTING, 12 nodes, containment depth 3) matches an independent recomputation from the CONTAINS edges on every node, and its root subtree equals `graph.usage.total`. Nothing in the corpus is deep or wide enough to *measure* the memoization — the walk is memoized and uses an explicit stack, so the O(n) claim and the recursion-limit independence are structural, not benchmarked. `GraphSpec.usage.unattributed.inference_calls` is **0 on every one of the 32 DRY fixtures** — nothing escapes attribution on a local in-process run.
+
+**CHECKPOINT 2.** pipelex is shippable as-is: Phases 1-3 are self-contained and the API, the MCP and `vscode-pipelex` get per-node usage whether or not a card renders it.
+
+**CHECKPOINT 3.** Reached and reviewed in Storybook against the real LIVE corpus. The treatment changed substantially from the plan under review — see "What review changed" below. Remaining: commit both repos, publish `@pipelex/mthds-ui`, and `make refresh-graph-ui-sri` to bump the pinned bundle. Until that publish, the standalone `reactflow.html` pipelex emits still loads the old bundle and shows no cost.
+
+### What review changed (post-implementation)
+
+The plan's UI design did not survive contact with the screen. Four reversals, all in the same direction — showing less, and only what was actually measured:
+
+1. **No cost on the cards at all.** The plan put a chip in the card's annotation row. Removed entirely — the card is for structure; a price on every node turns the graph into a spreadsheet. Cost lives only in the side panel.
+2. **Cost sits on the status line**, formatted exactly like the duration beside it (`● Succeeded 8.37s $0.0268 ⌄`), not as a labelled block. They are three facts of the same kind about one run.
+3. **No token counts anywhere.** The decisive finding: extract, search and image generation are billed per request, and pipelex encodes that price by putting exactly `1_000_000` in each token category (rates are per-million, so the arithmetic reproduces the per-request price — `linkup_extract_worker.py:89`, `linkup_search_worker.py:92`, `gateway_extract_worker.py:182`). A one-page extract reports 2,000,000 "tokens". Worse, a controller's `subtree_total_tokens` sums those sentinels with real LLM tokens, so **no token figure is trustworthy at any level of a graph**. Cost is the only number that survives the encoding.
+4. **`--mock-usage` reverted out of the DRY fixtures.** It makes a dry run report invented token counts; a dry run executes nothing, so those were fabrications rendered as measurements. DRY specs now carry zero tokens and a null cost, which is the truth, and the cost surface is gated on a real run.
+
+### New defect found, not fixed (out of scope)
+
+The `1_000_000` sentinel is not confined to the graph. It rides the `/execute` API's `tokens_usages` records and the CLI cost table's token columns, and `AggregatedCosts.total_nb_tokens` sums it — so any client computing token usage from the API gets nonsense as soon as a method touches extract or search. The fix is a `billing_unit` discriminator on the usage record so per-request prices stop masquerading as token counts. That is an API-contract change and needs its own plan.
+
+### Where the plan was refined during implementation
+
+1. **Invariant 1 is weaker than stated, and the docstring now says so honestly.** The plan wrote `usage is None <=> collection was OFF`. The event stream carries no "collection is on" signal, so a run with collection on that makes *zero* inference calls is indistinguishable from collection off. The shipped invariant is: `usage is None` ⟺ **no usage was reported anywhere in the run** (off, or zero calls) — and it stays all-or-nothing across a graph, which is the property consumers actually branch on. Both states render as nothing, so nothing downstream cares.
+2. **Attribution resolves in pass 2, not in the handler.** A `UsageReportEvent` can be read *before* the `PipeStartEvent` of the node it names (different workers, stream ordered by `(workflow_id, sequence)`), so "is this node real?" is only answerable once every event is in. `_handle_usage_report` folds by raw `node_id`; `_attribute_usage` resolves. This also gets a test.
+3. **`UNATTRIBUTED_NODE_ID` is now a named constant** in `trace_events.py`, replacing the bare `"unknown"` literal duplicated at `reporting_manager.py:219` and `:293`.
+4. **The Phase 4 fixture gate is split by freshness.** `assertValid` holds only *freshly generated* specs to the usage gate; specs reused from disk are exempt. Otherwise a partial `--only` LIVE run against the existing (pre-usage) LIVE corpus would die on every reused spec.
+5. **The DRY token gate is conditional.** "at least one node has `total_tokens > 0`" fires only when the run made any inference call, so an inference-free pipeline does not fail the gate. Note only `PipeLLM` reports mock usage — `PipeExtract` / `PipeSearch` / `PipeImgGen` nodes show zero calls in DRY.
+6. **Fixture regeneration turned out not to be bit-stable, for reasons unrelated to usage.** `pipeline_11`'s `PipeCondition` branches on mock-generated text, so a dry run routes to either branch; `snapshots.test.ts` already documents per-regeneration fingerprint drift and prescribes `vitest -u`. Regenerating the *whole* corpus at once produced a coherent, fully green tree (1835 tests). Regenerating a subset can leave the corpus internally inconsistent.
+
+## The problem in one paragraph
+
+Pipelex already records what every inference call cost, and already records the shape of the run as a graph. Both are built from **one** trace-event stream, and every usage event already carries the graph node it belongs to. At assembly the link is thrown away: `GraphSpecAssembler` skips `UsageReportEvent` entirely, and `UsageAggregator` returns a flat list with `node_id` stripped. So `PipeOutput.graph_spec` and `PipeOutput.tokens_usages` are two disjoint artifacts describing the same run, and no consumer can answer "which pipe spent the money".
+
+```
+ inference call completes
+ │
+ ▼
+ ReportingManager._emit_usage_event trace_context.parent_node_id
+ reporting_manager.py:161 │
+ │ │
+ ▼ │
+ UsageReportEvent{ node_id ◄──────────────────────────────────┘ , tokens_usage }
+ trace_events.py:182 the link ALREADY EXISTS here
+ │
+ ▼
+ ┌──────────────── event log — ONE stream, one read ────────────────┐
+ │ PipeStartEvent PipeEndSuccessEvent EdgeEvent UsageReport │
+ └───────────────────────────────┬──────────────────────────────────┘
+ │ tracing_assembly.py:101-136
+ ┌───────────────┴───────────────┐
+ ▼ ▼
+ GraphSpecAssembler UsageAggregator
+ :197 elif UsageReportEvent: :22 [e.tokens_usage for e in ...]
+ pass ◄── LINK DROPPED ◄── node_id DROPPED
+ │ │
+ ▼ ▼
+ PipeOutput.graph_spec PipeOutput.tokens_usages
+ (nodes, edges, no cost) (costs, no graph position)
+```
+
+The fix is to stop dropping it, model the result unambiguously, and render it.
+
+## What already exists
+
+Reuse these; do not rebuild them.
+
+| Thing | Where | How this plan uses it |
+|---|---|---|
+| `UsageReportEvent.node_id` | `tracing/trace_events.py:182-187` | The correlation key. Already emitted on both paths (`reporting_manager.py:219`, `:293`). |
+| `compute_tokens_usage_cost()` | `cogt/usage/cost_registry.py:27` | The single canonical per-call USD total, and the authority on rated-vs-unrated (returns `None` iff `unit_costs` is empty). **Call it; never re-derive cost math.** |
+| `AggregatedCosts.total_nb_tokens` | `cogt/usage/cost_registry.py:~70` | Defines the canonical token total as input-joined + output. Our `total_tokens` uses the same definition so the graph and the cost report cannot disagree. |
+| `TokenCategory` / `nb_tokens_by_category` | `cogt/usage/token_category.py`, `reporting/usage_records.py:49` | The already-shipped token vocabulary. Reused verbatim — no new token names. |
+| `TokensUsageRecord.cost: float \| None` | `reporting/usage_records.py:51` | The already-shipped encoding of "unrated". Mirrored, not reinvented. |
+| Node parentage | `_AssemblerNodeData.parent_node_id` (`graphspec_assembler.py:69`) | Subtree rollup walks this; no need to re-derive from `CONTAINS` edges. |
+| `mthds-ui` GraphSpec plumbing | `types.ts`, `validateGraphSpec.ts`, `pipeCardPayload.ts`, `PipeCardBase.tsx` | The payload pipeline already exists; we add one field through all four. |
+| `generate-fixtures.mjs` | `mthds-ui/scripts/` | Already shells out to `../pipelex/.venv/bin/pipelex`. A local Python change is picked up with **zero wiring**. |
+| `is_generate_usage = true` | `pipelex/pipelex.toml:307`, not overridden in `mthds-ui/.pipelex/pipelex.toml` | Usage events are **already** emitted on every fixture run. No CLI flag change needed. |
+| `--mock-usage` | `_run_core.py:55-76`, `dry_mock.py:150+` | Hidden dry-run sub-flag producing non-zero synthetic tokens. Used in Phase 4 to get a meaningful free token chip. |
+
+## Decisions taken
+
+Decided on best-practice / most-future-proof grounds rather than asked.
+
+### D1 — A typed `NodeSpec.usage` with no overloaded nulls
+
+`NodeSpec.metrics: dict[str, float]` exists and is already rendered by `PipeDetailPanel.tsx:190`, which makes it the tempting zero-schema-change home. **Rejected.** `dict[str, float]` cannot express *unrated*: `compute_tokens_usage_cost` returns `None` whenever the model carries no rate table, and `dry_mock.py:133` hardcodes `unit_costs={}` for every synthetic dry-run call — so **every one of the 32 committed DRY fixtures is unrated**. In a float dict the only encoding for "unrated" is to omit the key, indistinguishable from "never collected".
+
+The first draft of this plan then made the same mistake one level up, by letting a single `cost: None` mean four different things. It does not now. Three states, three orthogonal encodings, stated as invariants:
+
+```python
+# pipelex/graph/graphspec.py
+
+class NodeUsageSpec(BaseModel):
+ """Inference usage attributed to one graph node.
+
+ Field names mirror the already-shipped client-facing ``TokensUsageRecord``
+ (reporting/usage_records.py) so the graph does not introduce a fifth
+ vocabulary for numbers this codebase already names four ways
+ (TokenCategory, LLMTokenCostReportField, GenAISpanAttr, PostHogAttr).
+
+ INVARIANTS — the UI and every other consumer branch on these, not on
+ guesses about why a number is missing:
+
+ 1. ``NodeSpec.usage is None`` <=> usage collection was OFF for the run.
+ When it was ON, EVERY node carries a spec, zeroed if it ran no
+ inference. A controller, a lifted pipe, and a PipeFunc all get
+ ``inference_calls=0``, never ``usage=None``.
+
+ 2. ``cost is None`` <=> ``rated_inference_calls == 0``.
+ Nothing else. "Made no call" and "made only unrated calls" both land
+ here and are told apart by ``inference_calls``.
+
+ 3. ``inference_calls > rated_inference_calls > 0`` => ``cost`` is a
+ LOWER BOUND, not a total. The UI must mark it (a leading "≥").
+
+ 4. ``total_tokens`` is input_joined + output — the same definition as
+ ``AggregatedCosts.total_nb_tokens``. It is NOT the sum of
+ ``nb_tokens_by_category``: ``input_cached`` is a SUBSET of ``input``,
+ not additive (usage_records.py:47-48), so summing double-counts.
+ Never sum the dict; read this field.
+ """
+
+ model_config = ConfigDict(extra="forbid", strict=True)
+
+ # This node's own inference.
+ inference_calls: int = 0
+ rated_inference_calls: int = 0
+ nb_tokens_by_category: dict[str, int] = Field(default_factory=dict)
+ total_tokens: int = 0
+ cost: float | None = None
+
+ # This node plus every descendant.
+ subtree_inference_calls: int = 0
+ subtree_rated_inference_calls: int = 0
+ subtree_nb_tokens_by_category: dict[str, int] = Field(default_factory=dict)
+ subtree_total_tokens: int = 0
+ subtree_cost: float | None = None
+```
+
+`NodeSpec` gains `usage: NodeUsageSpec | None = None`. `metrics` is left exactly as it is — still free-form, still unused — so nothing that reads it today changes.
+
+**Why not OTel semconv keys** (`gen_ai.usage.input_tokens`, already imported at `otel_constants.py:50`): semconv names *span attributes*, a different layer with different consumers (Langfuse, PostHog), and pipelex already uses it correctly there (`llm_worker_abstract.py:256`). Borrowing that dotted namespace into a graph contract read by UIs buys nothing and, with `PipeDetailPanel.tsx:195` rendering `label={key}`, would leak `gen_ai.usage.input_tokens` onto the screen. A typed model has field names, not attribute keys, so the question dissolves.
+
+### D2 — Own and subtree, for tokens as well as cost
+
+A controller (`PipeSequence`, `PipeBatch`, `PipeParallel`) runs no inference, so its own numbers are always zero. Shipping only own values makes every controller card read empty, which reads as a bug.
+
+Subtree tokens matter as much as subtree cost, and more in practice: the default Storybook view is the DRY corpus, which is unrated, so **tokens are the only number a controller can show there**. Both rollups ship.
+
+Rollup is computed **in the assembler**, not the UI, for the same reason `AggregatedCosts` computes its totals once: three consumers (`pipelex-app`, `vscode-pipelex`, MCP) must not each re-derive it and disagree.
+
+```
+ seq_node calls 0 (0 rated) subtree 3 calls (2 rated) subtree_cost 0.0071 subtree_tokens 4210
+ ├── llm_a calls 1 (1 rated) cost 0.0043 tokens 2130
+ ├── llm_b calls 1 (1 rated) cost 0.0028 tokens 2080
+ └── func_c calls 0 (0 rated) cost None tokens 0 ← ran, made no inference
+```
+
+`func_c` has `usage` present with `cost=None` and `inference_calls=0` — invariant 1 and 2 together say "collected, ran nothing", never "unknown".
+
+### D3 — A typed `GraphSpec.usage`, not a `meta` bag
+
+Both emit paths fall back to `node_id = trace_context.parent_node_id or "unknown"` (`reporting_manager.py:219`, `:293`). Usage landing on `"unknown"` has no node to attach to, and dropping it would make the graph's total silently disagree with the cost report.
+
+The first draft put the run total and the unattributed bucket in `GraphSpec.meta`. That was inconsistent — arguing for typing at the node level and then reaching for an untyped bag at the graph level. `meta` is `[key: string]: unknown` in `mthds-ui`, so anything put there arrives untyped at every consumer.
+
+```python
+class GraphUsageSpec(BaseModel):
+ model_config = ConfigDict(extra="forbid", strict=True)
+
+ total: NodeUsageSpec # whole-run rollup
+ unattributed: NodeUsageSpec # usage whose node_id was "unknown"
+```
+
+`GraphSpec` gains `usage: GraphUsageSpec | None = None`, `None` under the same invariant 1 as the node field. Reusing `NodeUsageSpec` for both keeps exactly one usage shape in the contract.
+
+### D4 — Assert the builder divergence; do not normalize it away
+
+There are two GraphSpec builders. `GraphSpecAssembler` produces every consumed spec. The in-process `GraphTracer`'s `teardown()` output is **discarded at every call site** — `runner.py:281`, `pipe_run.py:60`, and `dry_run_in_process.py:151` (which then reads `pipe_output.graph_spec`, i.e. the assembler's). `GraphTracerProtocol` has no method to receive usage at all; adding one, plus its no-op implementations, would be plumbing into output nobody reads.
+
+But `test_assembler_equivalence.py:57` puts `"metrics": node.metrics` in its structural comparison, and **none of its six scenarios emit a usage event**. Left alone, the two builders diverge and CI stays green — a rotting test, worse than an absent one.
+
+Add `_scenario_with_usage`, and rather than deleting `usage` from `_normalize_node` (which would weaken the test permanently), **assert the divergence explicitly**: the assembled spec has `usage` populated, the tracer spec has `usage is None`, and that asymmetry is the assertion. If someone later plumbs usage into the tracer, the test fails and tells them to delete the assertion — which is what a test documenting an intentional gap should do.
+
+### D5 — Free runs prove the unrated path; only paid runs show a dollar
+
+`_report_synthetic_llm_job` sets `unit_costs={}` unconditionally (`dry_mock.py:133`), shared by both `report_dry_llm_job` and `report_mock_usage_llm_job`. The hidden `--mock-usage` flag buys non-zero *tokens* and never a non-zero *cost*.
+
+So the free loop is worth more than the first draft credited, provided it uses `--mock-usage`. Plain `--dry-run` emits zero-token usage, which exercises `inference_calls` and nothing else. With `--mock-usage` the DRY corpus produces a genuinely non-zero **unrated** graph — the exact default Storybook state — and the token chip becomes reviewable for free. Only the dollar itself needs Phase 6 and real money.
+
+### D6 — One attribution module, not logic buried in `_AssemblerState`
+
+Codex's strongest structural point: the plan risks a second aggregation model with subtly different semantics from `CostRegistry`. Mitigated by extracting accumulation and rollup into `pipelex/tracing/usage_attribution.py` — independently testable, one place where the invariants live, and reusable if the tracer ever needs it. `_AssemblerState` calls into it and holds no usage arithmetic of its own.
+
+Not accepted: Codex's fuller version, preserving `node_id` on the records `UsageAggregator` returns. That would change `PipeOutput.tokens_usages`, whose trimmed wire shape (`TokensUsageRecord`, `extra="forbid"`) is a shipped client contract. Two projections of one stream is the design `tracing_assembly.py` already documents; the DRY fix belongs in the shared helper, not in merging the projections.
+
+---
+
+## Phases
+
+### Phase 1 — pipelex: model and assemble
+
+Files: `pipelex/graph/graphspec.py`, new `pipelex/tracing/usage_attribution.py`, `pipelex/tracing/graphspec_assembler.py`.
+
+1. Add `NodeUsageSpec`, `GraphUsageSpec`, `NodeSpec.usage`, `GraphSpec.usage` (D1, D3).
+2. New `usage_attribution.py` (D6): a `UsageAccumulator` that folds one `AnyTokensUsage`, tracking `inference_calls`, `rated_inference_calls`, merged `nb_tokens_by_category`, canonical `total_tokens`, and summed cost via `compute_tokens_usage_cost`. Plus `roll_up(nodes)` producing the subtree fields.
+3. Replace `graphspec_assembler.py:197-198`:
+ ```python
+ elif isinstance(event, UsageReportEvent):
+ self._handle_usage_report(event)
+ ```
+4. Attach a spec to **every** node when any usage event was seen, zeroed where nothing ran (invariant 1). Emit `GraphSpec.usage` with the run total and the `"unknown"` bucket.
+5. Rollup in `pass_two()`: memoized post-order walk of `parent_node_id`. Memoize (O(n), not O(n·depth)), guard cycles with a visited set, and treat a dangling `parent_node_id` as a root rather than a `KeyError`.
+6. Update the class docstring at `graphspec_assembler.py:112-119` — it enumerates pass 1 and pass 2 and goes stale the moment step 3 lands.
+
+All new functions keyword-only per `docs/contribute/keyword-only-arguments.md`. Run `make agent-check`.
+
+> **CHECKPOINT 1.** `pipe_output.graph_spec` now carries per-node usage, and the API, the MCP, and `vscode-pipelex` get it whether or not a card renders it. The independently valuable half — worth landing alone if the UI work stalls. Record here: whether rollup memoization held on the deepest fixture, and whether `GraphSpec.usage.unattributed` was non-zero on any real run (it should not be).
+
+### Phase 2 — pipelex: tests
+
+Files: new `tests/unit/pipelex/tracing/test_usage_attribution.py`, `test_graphspec_assembler.py`, `test_assembler_equivalence.py`, `tests/unit/pipelex/graph/test_graphspec_roundtrip.py`.
+
+Full coverage per the diagram below, including the D4 equivalence assertion. Every invariant in the `NodeUsageSpec` docstring gets a named test — they are the contract.
+
+### Phase 3 — pipelex: docs and changelog
+
+- New page under `docs/under-the-hood/` on the two projections of one event stream and the three usage states. Sibling to `tokens-usage-wire-records.md`.
+- Update `docs/features/cost-tracking.md` for per-node attribution.
+- `CHANGELOG.md` under `## [Unreleased]`: breaking — `GraphSpec` nodes gained `usage`, and `NodeSpec` is `extra="forbid"`, so an older pipelex rejects the new JSON. Per workspace policy there is no transition window; the note is the deliverable.
+
+> **CHECKPOINT 2.** pipelex is shippable here. Everything after crosses into `mthds-ui` and eventually needs an npm publish + SRI bump to reach users. Decide whether to land Phase 1-3 as its own PR.
+
+### Phase 4 — regenerate DRY fixtures with `--mock-usage` (free)
+
+`generate-fixtures.mjs:124-131` passes `--dry-run --mock-inputs`. Add `--mock-usage` to that DRY arg list (D5) so the corpus carries non-zero unrated tokens instead of zeros.
+
+```bash
+cd ../mthds-ui && make fixtures
+```
+
+Picks up the local `.venv` pipelex automatically (`generate-fixtures.mjs:48-50`).
+
+**Gate — stated correctly this time.** Not "every node has `inference_calls >= 1`"; controllers, `PipeFunc`, skipped and lifted pipes legitimately have zero. The gate is:
+
+- every node has `usage != null` (invariant 1),
+- `cost == null` on every node (invariant 2 — DRY is always unrated),
+- at least one LLM-bearing node has `total_tokens > 0` (proves `--mock-usage` took effect),
+- every controller has `subtree_total_tokens >= ` the max of its children's,
+- `graph.usage.unattributed.inference_calls == 0`.
+
+`assertValid` (`generate-fixtures.mjs:163-177`) checks format/mode/nodes/description/domain_code only and will pass an all-null regeneration. Extend it — that is task T10, not an optional grep.
+
+### Phase 5 — mthds-ui: validator, type, payload, card
+
+Files: `src/graph/types.ts`, `src/graph/validateGraphSpec.ts`, `src/graph/pipeCardPayload.ts`, `src/graph/react/nodes/pipe/pipeCardTypes.ts`, `PipeCardBase.tsx`, `graph-core.css`, `react/detail/PipeDetailPanel.tsx`.
+
+1. `types.ts` — `usage?: NodeUsage` on the node type, `usage?: GraphUsage` on the spec, mirroring the Python models.
+2. `validateGraphSpec.ts` — **the boundary gate**. Types alone are a compile-time fiction; malformed `usage` from a stale or hostile spec otherwise flows through as trusted data. Validate shape and the `cost === null || typeof cost === "number"` invariant.
+3. `pipeCardPayload.ts:22-23` — one line, mirroring the existing `tags` passthrough:
+ ```ts
+ if (node.usage !== undefined) payload.usage = node.usage;
+ ```
+4. `PipeCardBase.tsx` — a usage chip in the existing `pipe-card-annotations` row (`:~130`), already the established pattern for `outcome` / `batch_multiplicity`. Four render states, visually distinct:
+
+ | State | Condition | Card shows |
+ |---|---|---|
+ | not collected | `usage === undefined` | nothing |
+ | ran nothing | `inference_calls === 0 && subtree_inference_calls === 0` | nothing |
+ | unrated | `cost === null` with calls > 0 | token count, no `$` |
+ | partial | `inference_calls > rated_inference_calls > 0` | `≥ $0.0043` |
+ | rated | `rated === inference_calls > 0` | `$0.0043` |
+
+ Controllers read the `subtree_*` fields; operators read own (D2).
+5. `PipeDetailPanel.tsx` — a Usage section beside the existing Metrics section, with the per-category breakdown and both own and subtree columns.
+
+### Phase 6 — LIVE regeneration, real money
+
+```bash
+cd ../mthds-ui && make fixtures-live ONLY=pipeline_01,pipeline_09
+```
+
+Two or three pipelines, never the corpus. The script's own header (`:20-23`) warns a full sweep has no skip path and leaves a half-swept mixed-version tree on any failure; see `mthds-ui/wip/fixtures-live-corpus-regeneration.md`. `--only` reuses every other pipeline's on-disk spec so the emitted fixtures stay complete.
+
+**Cross-check semantics, stated so the check is meaningful.** `CostRegistry.aggregate_costs` treats an unrated usage as zero-cost; `NodeUsageSpec.cost` is `None`. Comparing them naively produces a false disagreement. The check is: `graph.usage.total.cost` must equal the CSV total **when `total.rated_inference_calls == total.inference_calls`**. When they differ, the graph total is a lower bound and the CSV is the wrong comparand — compare `rated_inference_calls` against the CSV row count instead.
+
+**Do not commit the LIVE regeneration** unless you intend to. Every regenerated live spec is real inference against whatever models are current; the diff churns far beyond the `usage` field.
+
+> **CHECKPOINT 3.** You have looked at it. Decide whether the card treatment is right before the npm publish + `make refresh-graph-ui-sri` bump that makes it reach users.
+
+---
+
+## Test coverage
+
+```
+CODE PATHS USER FLOWS
+[+] tracing/usage_attribution.py (new) [+] Storybook, DRY corpus (default view)
+ ├── UsageAccumulator.fold() ├── [GAP] Unrated card: token count,
+ │ ├── [GAP] first usage for a node │ no "$", not "$0.00"
+ │ ├── [GAP] second usage, same node (accumulate) ├── [GAP] Controller shows subtree
+ │ ├── [GAP] INVARIANT 2: unrated -> cost stays None │ tokens, not blank
+ │ ├── [GAP] INVARIANT 3: mixed rated/unrated -> cost ├── [GAP] Zero-call node (PipeFunc,
+ │ │ is a lower bound, rated < calls │ lifted) shows nothing, and
+ │ └── [GAP] INVARIANT 4: total_tokens is joined+output, │ is distinguishable from
+ │ NOT sum(nb_tokens_by_category) │ "not collected"
+ └── roll_up() └── [GAP] Detail panel Usage section,
+ ├── [GAP] leaf: subtree == own own + subtree columns
+ ├── [GAP] controller: sum of children
+ ├── [GAP] nested 3 deep, memoization correct [+] Storybook, LIVE fixtures
+ ├── [GAP] all-unrated subtree -> subtree_cost None ├── [GAP] Real dollars on a card
+ ├── [GAP] CRITICAL cycle in parent chain -> guard ├── [GAP] Partial "≥ $" state
+ └── [GAP] CRITICAL dangling parent -> treated as root └── [GAP] Sequence subtree_cost ==
+ sum of children's cost
+[+] tracing/graphspec_assembler.py
+ ├── _handle_usage_report() [+] Contract consumers
+ │ ├── [GAP] node_id="unknown" -> graph.usage.unattributed ├── [GAP] [→E2E] graphspec.json
+ │ └── [GAP] node_id for a node that never started │ round-trips through
+ └── build_graph_spec() │ validateGraphSpec
+ ├── [GAP] INVARIANT 1: usage=None on ALL nodes when └── [GAP] vscode-pipelex adapter
+ │ no usage event was seen renders a usage-bearing spec
+ ├── [GAP] INVARIANT 1: EVERY node gets a spec when
+ │ any usage event was seen (controllers,
+ │ PipeFunc, skipped, lifted all zeroed)
+ └── [GAP] graph.usage.total == sum over nodes' own
+
+[+] graph/graphspec.py
+ ├── [GAP] NodeUsageSpec JSON round-trip preserves cost=None
+ ├── [GAP] extra="forbid" rejects an unknown usage field
+ └── [★★ TESTED] NodeSpec round-trip — test_graphspec_roundtrip.py
+ └── [GAP] REGRESSION: round-trip with usage populated
+
+[+] tests/.../test_assembler_equivalence.py
+ └── [GAP] CRITICAL: _normalize_node compares "metrics" (:57) but no
+ scenario emits usage. Add _scenario_with_usage AND assert the
+ divergence explicitly (assembler has usage, tracer has None) —
+ do NOT normalize usage away (D4).
+
+[+] mthds-ui
+ ├── validateGraphSpec.ts
+ │ ├── [GAP] accepts a well-formed usage object
+ │ ├── [GAP] rejects cost of the wrong type
+ │ └── [GAP] accepts a spec with no usage at all
+ ├── pipeCardPayload.ts
+ │ ├── [GAP] usage passed through when present
+ │ └── [GAP] usage absent -> key omitted (mirrors tags at :23)
+ └── PipeCardBase.tsx
+ ├── [GAP] all five render states distinct
+ └── [GAP] controller reads subtree_*, operator reads own
+
+COVERAGE: 1/35 paths tested (3%) | Code: 1/22 (5%) | Flows: 0/13 (0%)
+QUALITY: ★★:1 | GAPS: 34 (1 E2E, 0 eval, 3 CRITICAL)
+```
+
+Legend: ★★★ behavior + edge + error | ★★ happy path | ★ smoke | [→E2E] integration
+
+## Failure modes
+
+| Codepath | Realistic production failure | Test? | Error handling? | Silent? |
+|---|---|---|---|---|
+| `roll_up` | Cycle in `parent_node_id` (not enforced anywhere today) | planned | visited-set guard, else infinite loop | **hang, not error** |
+| `roll_up` | `parent_node_id` names a node absent from `_nodes` (cross-worker partial read) | planned | treat as root, never `KeyError` | **would be silently wrong** |
+| `UsageAccumulator.fold` | Mixed rated + unrated on one node renders a partial total as if complete | planned | `rated_inference_calls` + the UI "≥" marker | **would be silently wrong money** |
+| UI token chip | Consumer sums `nb_tokens_by_category`, double-counting `input_cached` | planned | `total_tokens` is the only sanctioned total; docstring says so | **would be silently wrong** |
+| `_handle_usage_report` | Event for a `node_id` that never emitted `PipeStartEvent` (worker died between) | planned | mirror the existing `log.warning(f"...unknown node: {event.node_id}")` used by every sibling handler | no, once handled |
+| `graph.usage.unattributed` | Usage emitted outside any pipe context | planned | typed bucket, surfaced not dropped | no |
+| Temporal retry | Retried activity re-emits usage at a new sequence — documented over-count R2 (`reporting_manager.py:265-270`) | not fixable here | none; inherited | yes, and now visible per-node |
+| Phase 4 gate | `usage` null everywhere because Phase 1 mis-wired | T10 | none in `generate-fixtures.mjs` today | **yes without T10** |
+
+**Critical gaps: 3.** The cycle guard (hang, no error), the dangling-parent rollup (silently wrong), and mixed rated/unrated (silently wrong *money*, the worst kind). All three are in Phase 1 and must land with it.
+
+## NOT in scope
+
+| Deferred | Why |
+|---|---|
+| Plumbing usage into the in-process `GraphTracer` | Its `GraphSpec` is discarded at all three call sites (D4). Divergence asserted instead. |
+| Preserving `node_id` on `PipeOutput.tokens_usages` | Would change `TokensUsageRecord`, a shipped `extra="forbid"` client contract. The DRY win is taken via the shared module (D6) instead. |
+| npm publish of `@pipelex/mthds-ui` + `make refresh-graph-ui-sri` | Local loop first. Nothing reaches users until this happens; deliberate later step. |
+| Committing LIVE fixture regeneration | Real inference against current models; diff churns far past `usage`. |
+| Per-model cost breakdown on a node | `NodeUsageSpec` is extensible; add `by_model` when something asks. |
+| A real `GraphSpec` version field | Docs call GraphSpec "versioned" and it has no version field. Real gap, pre-existing, own change. TODO below. |
+| Fixing the Temporal retry over-count (R2) | Pre-existing, documented, separate. |
+| `mermaidflow` renderer | Text renderer, no cost surface planned. |
+| Attributing cost to an org / debiting a balance | The deferred billing connection (`monetization-metering-already-exists`). Different consumer of the same stream. |
+| `_generate_data_edges` nested-loop complexity (`graphspec_assembler.py:474-486`) | Pre-existing, flagged below, not touched. |
+
+## Performance
+
+Single pass over events; accumulation is O(1) per event. Rollup is O(n) memoized, O(n·depth) without — memoize. Payload grows by two small objects per node; negligible.
+
+**Pre-existing, flagged not fixed:** `_generate_batch_item_edges` (`:474-486`) and `_generate_batch_aggregate_edges` nest a full node scan inside a per-item loop, so batch fan-out is roughly O(items × nodes × inputs). Invisible on 32 small fixtures; it will bite on a wide production batch.
+
+## Parallelization
+
+| Step | Modules touched | Depends on |
+|---|---|---|
+| Phase 1 model + attribute | `pipelex/graph/`, `pipelex/tracing/` | — |
+| Phase 2 tests | `tests/unit/pipelex/` | Phase 1 |
+| Phase 3 docs | `pipelex/docs/`, `CHANGELOG.md` | Phase 1 |
+| Phase 4 DRY fixtures | `mthds-ui/scripts/`, `mthds-ui/data/`, `.../specs/` | Phase 1 |
+| Phase 5 UI | `mthds-ui/src/graph/` | Phase 4 |
+| Phase 6 LIVE fixtures | `mthds-ui/data/` | Phase 5 |
+
+```
+Lane A: Phase 1 → Phase 2 (sequential, shared pipelex/)
+Lane B: Phase 3 (independent once Phase 1 lands — docs only)
+Lane C: Phase 4 → Phase 5 → Phase 6 (sequential, shared mthds-ui/)
+
+Launch: Phase 1 alone. Then B and C in parallel with A's remainder.
+```
+
+**Conflict flag:** Lanes B and C are clean against each other. Lane C reads the pipelex `.venv`, so do not start it until Phase 1 is committed.
+
+## Implementation Tasks
+
+- [x] **T1 (P1, human: ~3h / CC: ~30min)** — graphspec — `NodeUsageSpec`, `GraphUsageSpec`, and the four invariants as docstring contract
+ - Surfaced by: D1/D3 — first draft overloaded `cost: None` four ways; `meta` bag was untyped
+ - Files: `pipelex/graph/graphspec.py`
+ - Verify: `.venv/bin/pytest tests/unit/pipelex/graph/test_graphspec_roundtrip.py`
+- [x] **T2 (P1, human: ~4h / CC: ~40min)** — tracing — New `usage_attribution.py`: accumulator + rollup, one home for the invariants
+ - Surfaced by: D6 — keeps a second aggregation model out of `_AssemblerState`
+ - Files: `pipelex/tracing/usage_attribution.py`
+ - Verify: new `tests/unit/pipelex/tracing/test_usage_attribution.py`
+- [x] **T3 (P1, human: ~2h / CC: ~20min)** — assembler — Fold `UsageReportEvent`; attach a spec to EVERY node when collection was on
+ - Surfaced by: `graphspec_assembler.py:197` `pass`; invariant 1 (controllers/PipeFunc/lifted nodes must not read as "not collected")
+ - Files: `pipelex/tracing/graphspec_assembler.py`
+ - Verify: `.venv/bin/pytest tests/unit/pipelex/tracing/test_graphspec_assembler.py`
+- [x] **T4 (P1, human: ~2h / CC: ~20min)** — tracing — CRITICAL: cycle guard, dangling-parent tolerance, memoized rollup
+ - Surfaced by: Failure modes — cycle hangs with no error; dangling parent is silently wrong
+ - Files: `pipelex/tracing/usage_attribution.py`
+ - Verify: `test_usage_attribution.py`
+- [x] **T5 (P1, human: ~1h / CC: ~10min)** — assembler — `"unknown"` usage into `GraphSpec.usage.unattributed`; emit run total
+ - Surfaced by: D3 — `reporting_manager.py:219` fallback would otherwise drop it
+ - Files: `pipelex/tracing/graphspec_assembler.py`
+ - Verify: `test_usage_attribution.py`
+- [x] **T6 (P1, human: ~1h / CC: ~15min)** — tests — CRITICAL: assert the builder divergence in `test_assembler_equivalence`
+ - Surfaced by: D4 — `:57` compares `metrics`, no scenario emits usage, divergence would be silent
+ - Files: `tests/unit/pipelex/tracing/test_assembler_equivalence.py`
+ - Verify: `.venv/bin/pytest tests/unit/pipelex/tracing/test_assembler_equivalence.py`
+- [x] **T7 (P2, human: ~30min / CC: ~5min)** — assembler — Refresh the pass-1/pass-2 docstring at `:112-119`
+ - Surfaced by: Code quality — docstring/diagram maintenance is part of the change
+ - Files: `pipelex/tracing/graphspec_assembler.py`
+ - Verify: read it
+- [x] **T8 (P2, human: ~2h / CC: ~20min)** — docs — Under-the-hood page on the two projections and the three usage states
+ - Surfaced by: workspace rule — document at every iteration
+ - Files: `docs/under-the-hood/`, `docs/features/cost-tracking.md`, `CHANGELOG.md`
+ - Verify: `make agent-check`
+- [x] **T9 (P1, human: ~1h / CC: ~10min)** — mthds-ui — `usage` through `types.ts` and `pipeCardPayload.ts`
+ - Surfaced by: Phase 5 — mirrors the existing `tags` passthrough at `pipeCardPayload.ts:23`
+ - Files: `mthds-ui/src/graph/types.ts`, `pipeCardPayload.ts`, `pipeCardTypes.ts`
+ - Verify: `npm run typecheck && npx vitest run pipeCardPayload`
+- [x] **T10 (P1, human: ~1h / CC: ~15min)** — mthds-ui — Validate `usage` in `validateGraphSpec.ts`; extend `assertValid` in `generate-fixtures.mjs`
+ - Surfaced by: Codex — the validator is the boundary gate; `assertValid` would pass an all-null regeneration
+ - Files: `mthds-ui/src/graph/validateGraphSpec.ts`, `mthds-ui/scripts/generate-fixtures.mjs`
+ - Verify: `make fixtures` fails loudly against a pre-Phase-1 pipelex
+- [x] **T11 (P1, human: ~30min / CC: ~5min)** — mthds-ui — Add `--mock-usage` to the DRY arg list
+ - Surfaced by: D5 — plain `--dry-run` yields zero-token usage, so the token chip is untestable for free
+ - Files: `mthds-ui/scripts/generate-fixtures.mjs:124-131`
+ - Verify: regenerated DRY fixtures show `total_tokens > 0` on LLM nodes
+- [x] **T12 (P1, human: ~5h / CC: ~50min)** — mthds-ui — Usage chip, five distinct render states, subtree-vs-own by node kind
+ - Surfaced by: D1/D2/D5 — unrated is the default view and must not look like `$0.00`; partial must not look complete
+ - Files: `PipeCardBase.tsx`, `graph-core.css`, `PipeDetailPanel.tsx`
+ - Verify: `make storybook`, inspect DRY and LIVE stories
+
+## TODOs to capture
+
+1. **`GraphSpec` has no version field** despite `graphspec.py:3` calling it "versioned" and `pipelex-hosted-envelope.md:96` calling it a strict contract. Adding `usage` under `extra="forbid"` is a hard break for any older reader with no way to detect it. Pre-existing; own change.
+2. **Revive-or-delete the in-process `GraphTracer` GraphSpec.** Discarded at all three call sites. Either something consumes it or `close_tracer` stops returning it.
+3. **`_generate_batch_item_edges` nested scan.** O(items × nodes × inputs) on batch fan-out. Needs a wide-batch benchmark to justify.
+
+## Outside voice absorbed
+
+Codex reviewed revision 1. Ten findings accepted, one partially, one noted as policy-exempt.
+
+| Finding | Verdict |
+|---|---|
+| `cost: None` overloaded four ways | **Accepted** — invariants 1-3, `rated_inference_calls` |
+| No subtree token rollup; controllers blank in DRY | **Accepted** — `subtree_nb_tokens_by_category`, `subtree_total_tokens` |
+| Mixed rated/unrated renders a partial as complete | **Accepted** — invariant 3 + UI "≥" state |
+| Summing `nb_tokens_by_category` double-counts `input_cached` | **Accepted** — canonical `total_tokens`, invariant 4 |
+| `usage is None` ≠ "not collected" for zero-call nodes | **Accepted** — every node gets a spec; Phase 4 gate rewritten |
+| DRY fixtures need `--mock-usage` to be meaningful | **Accepted** — T11 |
+| `meta` bag is untyped; use a typed `GraphSpec.usage` | **Accepted** — D3 rewritten, `GraphUsageSpec` |
+| `validateGraphSpec.ts` missing from the task list | **Accepted** — T10 |
+| CSV cross-check semantics undefined (`null` vs `0`) | **Accepted** — Phase 6 states the comparison |
+| Normalizing `usage` away weakens equivalence permanently | **Accepted** — D4 asserts the divergence instead |
+| Preserve `node_id` on usage records, one shared rollup | **Partially** — shared module (D6) yes; changing `tokens_usages` no, it is a shipped `extra="forbid"` wire contract |
+| `extra="forbid"` breaks older readers | **Noted, not acted** — workspace policy is explicitly no backward compatibility. The missing version field is a fair catch and is TODO 1. |
+
+**No cross-model tension.** Codex did not contradict any review finding; it found gaps the review missed. The one strategic challenge (merge the two projections) was answered by taking its DRY intent without breaking a shipped contract.
+
+## GSTACK REVIEW REPORT
+
+| Review | Trigger | Why | Runs | Status | Findings |
+|--------|---------|-----|------|--------|----------|
+| CEO Review | `/plan-ceo-review` | Scope & strategy | 0 | — | — |
+| Codex Review | `/codex review` | Independent 2nd opinion | 1 | ISSUES_FOUND | 12 findings, 10 accepted, 1 partial, 1 policy-exempt |
+| Eng Review | `/plan-eng-review` | Architecture & tests (required) | 1 | ISSUES_OPEN | 44 issues, 3 critical gaps |
+| Design Review | `/plan-design-review` | UI/UX gaps | 0 | — | — |
+| DX Review | `/plan-devex-review` | Developer experience gaps | 0 | — | — |
+
+- **CODEX:** Outside voice ran on revision 1 and materially changed the design — the overloaded `cost: None`, the missing subtree token rollup, the double-counted `input_cached`, the untyped `meta` bag, and the missing `validateGraphSpec` gate all came from it.
+- **CROSS-MODEL:** No tension. Codex found gaps rather than contradicting findings; its one strategic challenge (merge the two event-stream projections) was absorbed as a shared attribution module without changing the shipped `TokensUsageRecord` wire contract.
+- **VERDICT:** ENG REVIEW COMPLETE — plan revised and ready to implement. Status is ISSUES_OPEN rather than CLEAR because 3 critical gaps (rollup cycle guard, dangling-parent tolerance, mixed rated/unrated partial cost) are planned but not yet built; they clear when T4 and T2 land.
+
+NO UNRESOLVED DECISIONS
diff --git a/wip/drift-contracts/dogfood-log.md b/wip/drift-contracts/dogfood-log.md
index ecbf62c2e..8837f0f86 100644
--- a/wip/drift-contracts/dogfood-log.md
+++ b/wip/drift-contracts/dogfood-log.md
@@ -79,3 +79,7 @@ The surviving finding is real and small: the table under "What a programmatic ca
- **2026-08-13 · cli-docs · clean-pass** — opened by the PR #1099 review fixes to `validate pipe` / `show pipe` (ambiguity now a clean exit instead of a traceback) and `which` (markup escaping). Reviewed both targets: `docs/tools/cli/` documents no exit codes or error rendering for these commands and has no `which` page, and the agent-CLI contract doc covers only the agent CLI, untouched here. The one user-visible behavior (ambiguous codes refused, naming candidates) was already documented in `docs/building-methods/libraries.md` by this branch — the fixes make the code honor the doc, the pleasant inverse of drift. Second consecutive clean-pass on cli-docs from the same underlying fact (the targets simply do not document the surfaces this branch keeps touching); not yet friction — the contract opened on genuine CLI-surface edits and the review was cheap.
- **2026-08-13 · pipelex-kernel-docs · clean-pass** — Opened on the `search_domain_codes`→`search_scope` rename threading through `shape_inputs` and the boot-contract mock swapping to `get_required_entry_concept`; the kernel doc names neither the parameter nor the resolver method, so nothing was stale.
+- **2026-08-14 · pipelex-kernel-docs · friction** — Reopened with **zero trigger-file changes**. Every blob under `pipelex/kernel/**` and `test_kernel_boot_contract.py` was byte-identical to the previous ack (verified by diffing the ack's recorded `trigger_files` map against `git ls-files -s`); what moved was the contract-definition half of the digest, across the `dev` merge on `feature/usage-graph-spec`. So the obligation fired on a change that could not, even in principle, have made the review target stale — and it fired in CI, as the one red check on PR #1100, on a branch about graph usage attribution that never touched the kernel.
+ No **trigger** narrowing prevents this one: the triggers matched nothing. It is the definition-change reopening rule, and the case for that rule is real (widen a glob or add a review target and every prior ack genuinely under-covers). But it does not distinguish a *semantic* definition change from bookkeeping. Two candidate refinements, both cheap: digest the definition and the trigger content **separately**, so `drift plan` can say "definition-only reopening, trigger content unchanged" and a reviewer knows up front that no code moved; or exempt a definition change that only **narrows** (removing triggers, removing review targets) since the prior review still covers a superset. The first is the honest one — it costs a field in the ack file and turns this entry's ten minutes of forensics into a printed line.
+ The review itself: ran anyway, on both halves the contract names. Module table diffed against every public function AST-parsed out of `pipelex/kernel/*.py` — complete, including `concrete_llm_model_handle` added by this branch; `kernel.exceptions` is correctly absent (error classes, no entry point), which the check flagged and I dismissed by opening it, the same lead-generator-not-verdict lesson the 2026-08-03 entry draws. Gates table's claim that the boot-contract test covers every entry point except the three `resolve_*_setting` deck-reading helpers still matches that test's own stated exception. Nothing stale.
+ **Third consecutive clean review on this contract** (2026-08-03, 2026-08-13, today), and today's opened on nothing at all. The pattern is not yet "drop" — the doc is a real specification and a renamed entry point would genuinely falsify it — but it is worth asking at verdict time whether the module table, the one part that keeps being checked and keeps being fine, is mechanizable outright: the AST enumeration I wrote here is ~20 lines and would have answered "is the table complete" as a derived check, leaving only the prose claims to human review.