Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

feat(reporting): one-arg render_cost_report_for_output submitter helper (#978)

Louis Choquel942579ed8dbc
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c5f6e67ab..97527b819 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,11 @@
# Changelog
+## [Unreleased]
+
+### Added
+
+- **`render_cost_report_for_output(pipe_output)` submitter helper.** A one-arg convenience over `render_run_cost_report` that unpacks `pipeline_run_id` and `tokens_usages` from a finished `PipeOutput` and derives the `--costs` gate from the output itself — `pipe_output.tokens_usages is None` is exactly the signal that cost reporting was off for the run, the decision the runner already resolved (with all `--costs/--no-costs` overrides applied) and recorded on the output. Embedders and the `pipelex run` CLI no longer re-derive the three primitive arguments by hand or re-read global config to reconstruct the gate. `render_run_cost_report` is unchanged as the low-level primitive for paths where the three values come from different places (the agent CLI's `build_cost_summary` JSON envelope, distributed reassembly).
+
## [v0.32.0] - 2026-06-09
This cycle reworks **cost reporting** so it survives distributed execution and stops leaking. Cost now rides on the run result instead of a side buffer: a multi-worker Temporal run aggregates usage from every worker into a single end-of-run report, the success-path registry leak is gone by removal, and a new `--costs` switch decouples cost collection from `--graph`. It also extracts the **framework-agnostic runtime bridge** so any host runtime — not just Mistral Workflows — can embed Pipelex through one boundary.
diff --git a/docs/features/cost-tracking.md b/docs/features/cost-tracking.md
index 228bebf40..ce29aa182 100644
--- a/docs/features/cost-tracking.md
+++ b/docs/features/cost-tracking.md
@@ -19,6 +19,18 @@ Real-time cost information displayed in the console during pipeline execution. E
Export detailed cost reports as CSV files for analysis, billing, and audit trails. Each row captures the pipe, model, token counts (input/output), and cost for a single operation.
+## Rendering Reports When Embedding
+
+Cost is part of the run result: a finished run carries its assembled usage on `pipe_output.tokens_usages`. If you embed Pipelex and want the same end-of-run console table / CSV that the CLI prints, render it in one call from the output:
+
+```python
+from pipelex.reporting.cost_report_renderer import render_cost_report_for_output
+
+render_cost_report_for_output(pipe_output)
+```
+
+The `--costs` gate is read off the output itself — `tokens_usages` is `None` exactly when cost reporting was off for the run — so you never re-read global config to reconstruct whether costs were on.
+
## Custom Reporting
Implement custom reporting backends via the [Reporting Delegate](advanced-customizations.md) injection point. The reporting protocol receives structured cost events during execution, allowing you to push data to your own analytics, dashboards, or billing systems.
diff --git a/pipelex/cli/commands/run/_run_core.py b/pipelex/cli/commands/run/_run_core.py
index e0feb6620..97d7553d2 100644
--- a/pipelex/cli/commands/run/_run_core.py
+++ b/pipelex/cli/commands/run/_run_core.py
@@ -32,7 +32,7 @@ from pipelex.pipe_run.pipe_run_mode import PipeRunMode
from pipelex.pipelex import Pipelex
from pipelex.pipeline.exceptions import PipelineExecutionError
from pipelex.pipeline.runner import PipelexRunner
-from pipelex.reporting.cost_report_renderer import render_run_cost_report
+from pipelex.reporting.cost_report_renderer import render_cost_report_for_output
from pipelex.system.runtime import IntegrationMode
from pipelex.system.telemetry.events import EventProperty
from pipelex.tools.misc.exceptions import JsonTypeError
@@ -312,16 +312,11 @@ async def _execute_run(
raise typer.Exit(1) from csv_exc
log.verbose(f"Main stuff CSV saved to: {save_csv}")
- # Render the end-of-run cost report from the usage assembled onto pipe_output (event-sourced),
- # gated by the resolved --costs. Channels (console / CSV) follow the reporting config (D6).
- # The renderer keeps its cost-domain name (`is_generate_costs`) on purpose; it is fed the usage
- # gate (`is_generate_usage`) — "if usage was generated, render the cost view". Do not "align" the
- # keyword to the field: the cost report is a view over usage, not the same switch.
- render_run_cost_report(
- pipeline_run_id=response.pipeline_run_id,
- tokens_usages=pipe_output.tokens_usages,
- is_generate_costs=execution_config.is_generate_usage,
- )
+ # Render the end-of-run cost report from the usage assembled onto pipe_output (event-sourced).
+ # The gate is read off the output itself — pipe_output.tokens_usages is None exactly when cost
+ # reporting was off for this run — so the submitter no longer re-derives it from config. Channels
+ # (console / CSV) follow the reporting config (D6).
+ render_cost_report_for_output(pipe_output)
# Print completion recap
console = get_console()
diff --git a/pipelex/reporting/cost_report_renderer.py b/pipelex/reporting/cost_report_renderer.py
index 3269f776c..cb34d84f1 100644
--- a/pipelex/reporting/cost_report_renderer.py
+++ b/pipelex/reporting/cost_report_renderer.py
@@ -14,6 +14,7 @@ from pipelex import log
from pipelex.base_exceptions import PipelexError
from pipelex.cogt.usage.cost_registry import CostRegistry
from pipelex.config import get_config
+from pipelex.core.pipes.pipe_output import PipeOutput
from pipelex.reporting.reporting_types import AnyTokensUsage
from pipelex.tools.misc.file_utils import ensure_path, get_incremental_file_path
@@ -80,3 +81,31 @@ def render_run_cost_report(
)
except (OSError, UnicodeEncodeError, PipelexError) as cost_report_error:
log.warning(f"Cost report generation failed (run succeeded): {cost_report_error}")
+
+
+def render_cost_report_for_output(pipe_output: PipeOutput) -> None:
+ """Render the end-of-run cost report from a finished output (self-contained, one-arg submitter helper).
+
+ The standard submitter convenience over :func:`render_run_cost_report`: a finished
+ :class:`~pipelex.core.pipes.pipe_output.PipeOutput` already carries the three values the primitive
+ needs, so callers (the ``pipelex run`` CLI, external embedders) no longer re-derive them by hand.
+
+ Crucially, the ``--costs`` gate is read **off the output, not config**:
+ :attr:`~pipelex.core.pipes.pipe_output.PipeOutput.tokens_usages` is ``None`` exactly when cost
+ reporting was off for this run. The runner already resolved that decision at run time (with all
+ ``--costs/--no-costs`` overrides applied) and recorded it on the output. Re-reading global config to
+ reconstruct it would only be correct when config wasn't overridden or mutated between run and render —
+ the output is authoritative, config is not.
+
+ Outcome-identical to the historical CLI gate (``execution_config.is_generate_usage``): the primitive
+ ANDs both guards (``not is_generate_costs or not tokens_usages``), so the cases where the two sources
+ disagree (usage on but tracing off → ``tokens_usages is None``) no-op either way.
+
+ Args:
+ pipe_output: The finished run output carrying ``pipeline_run_id`` and ``tokens_usages``.
+ """
+ render_run_cost_report(
+ pipeline_run_id=pipe_output.pipeline_run_id,
+ tokens_usages=pipe_output.tokens_usages,
+ is_generate_costs=pipe_output.tokens_usages is not None,
+ )
diff --git a/tests/unit/pipelex/reporting/test_render_cost_report_for_output.py b/tests/unit/pipelex/reporting/test_render_cost_report_for_output.py
new file mode 100644
index 000000000..5ddec0b03
--- /dev/null
+++ b/tests/unit/pipelex/reporting/test_render_cost_report_for_output.py
@@ -0,0 +1,66 @@
+"""Unit tests for the one-arg submitter helper ``render_cost_report_for_output``.
+
+The helper is a thin convenience over ``render_run_cost_report``: it unpacks the three primitive
+arguments from a finished ``PipeOutput`` and — crucially — derives the ``--costs`` gate from the
+output itself (``tokens_usages is None`` exactly when cost reporting was off), not from global config.
+These tests pin that derivation and the no-op/render outcomes it produces.
+"""
+
+from pytest_mock import MockerFixture
+
+from pipelex.cogt.llm.llm_report import LLMTokensUsage
+from pipelex.cogt.usage.cost_category import CostCategory
+from pipelex.cogt.usage.token_category import TokenCategory
+from pipelex.config import get_config
+from pipelex.core.pipes.pipe_output import PipeOutput
+from pipelex.pipeline.job_metadata import JobMetadata
+from pipelex.reporting.cost_report_renderer import render_cost_report_for_output
+
+_RUN_ID = "output-helper-run"
+
+
+def _usage(job_metadata: JobMetadata) -> LLMTokensUsage:
+ return LLMTokensUsage(
+ job_metadata=job_metadata,
+ inference_model_name="test-model",
+ inference_model_id="test-model-id",
+ nb_tokens_by_category={TokenCategory.INPUT: 100, TokenCategory.OUTPUT: 50},
+ unit_costs={CostCategory.INPUT: 1000, CostCategory.OUTPUT: 2000},
+ )
+
+
+class TestRenderCostReportForOutput:
+ def _enable_channels(self, mocker: MockerFixture) -> None:
+ reporting_config = get_config().pipelex.reporting_config
+ mocker.patch.object(reporting_config, "is_log_costs_to_console", True)
+ mocker.patch.object(reporting_config, "is_generate_cost_report_file_enabled", False)
+
+ def test_no_op_when_tokens_usages_none(self, mocker: MockerFixture) -> None:
+ """tokens_usages is None encodes 'cost reporting was off' -> the derived gate is False -> no render."""
+ spy = mocker.patch("pipelex.reporting.cost_report_renderer.CostRegistry.render_report")
+ self._enable_channels(mocker)
+
+ render_cost_report_for_output(PipeOutput(pipeline_run_id=_RUN_ID, tokens_usages=None))
+
+ spy.assert_not_called()
+
+ def test_no_op_when_tokens_usages_empty(self, mocker: MockerFixture) -> None:
+ """An empty list (costs on, no inference) is not None, but the primitive's `not tokens_usages` guard no-ops."""
+ spy = mocker.patch("pipelex.reporting.cost_report_renderer.CostRegistry.render_report")
+ self._enable_channels(mocker)
+
+ render_cost_report_for_output(PipeOutput(pipeline_run_id=_RUN_ID, tokens_usages=[]))
+
+ spy.assert_not_called()
+
+ def test_renders_with_output_run_id_when_usage_present(self, mocker: MockerFixture, job_metadata: JobMetadata) -> None:
+ """A non-empty tokens_usages derives the gate True and renders, titled with the output's own run id."""
+ spy = mocker.patch("pipelex.reporting.cost_report_renderer.CostRegistry.render_report")
+ self._enable_channels(mocker)
+
+ render_cost_report_for_output(PipeOutput(pipeline_run_id=_RUN_ID, tokens_usages=[_usage(job_metadata)]))
+
+ spy.assert_called_once()
+ kwargs = spy.call_args.kwargs
+ assert kwargs["pipeline_run_id"] == _RUN_ID
+ assert kwargs["cost_report_file_path"] is None
diff --git a/wip/registry/README.md b/wip/registry/README.md
index 9a06f5a93..1ee07a0f8 100644
--- a/wip/registry/README.md
+++ b/wip/registry/README.md
@@ -10,6 +10,10 @@ Shipped in **PR #967** on `fix/For-API-update`, with the companion rename **#968
- [`cost-reporting-overview.html`](cost-reporting-overview.html) — **the illustrated overview. Start here.** TL;DR, the two-bugs-one-lifecycle framing, before/after architecture, the shared-transport `--costs` switch, the cross-worker flow, the critical diffs, and the deferred items.
+## Shipped follow-ups
+
+- [`cost-report-submitter-helper-handoff.md`](cost-report-submitter-helper-handoff.md) — ✅ the one-arg `render_cost_report_for_output(pipe_output)` submitter helper (the embedder-ergonomics facet): submitters render the standard cost report from a finished output in one call, with the `--costs` gate read off the output (`tokens_usages is None`) instead of re-read from config. Landed on `feature/Cost-report-helper`; the downstream cocode delegate collapse is still pending. Composes with — does not resolve — decisions #3/#6 below.
+
## Open follow-ups
- [`deferred-followups.md`](deferred-followups.md) — the deliberate non-goals: the costs-only in-memory tracer skip (E1 deep half), cost-per-node correlation, the A1 `TraceContext.from_execution_config` factory, T5, T3 request-scoped tracing state, the costs-only event-log explicit-close cleanup (PR #977 re-review), `--mock-inference` coverage, and the leftover `graph_id → run_id` rename + shelved `TraceContext` split.
diff --git a/wip/registry/cost-report-submitter-helper-handoff.md b/wip/registry/cost-report-submitter-helper-handoff.md
new file mode 100644
index 000000000..2a894f5c2
--- /dev/null
+++ b/wip/registry/cost-report-submitter-helper-handoff.md
@@ -0,0 +1,86 @@
+# Handoff — one-arg `render_cost_report_for_output(pipe_output)` submitter helper
+
+**Status: ✅ Shipped** (`feature/Cost-report-helper`). `render_cost_report_for_output` is in `pipelex/reporting/cost_report_renderer.py`, the `pipelex run` CLI (`_run_core.py`) uses it, `render_run_cost_report` is unchanged as the primitive, unit tests pin the gate-from-output derivation (`tests/unit/pipelex/reporting/test_render_cost_report_for_output.py`), and the as-built overview + `docs/features/cost-tracking.md` document the surface. Downstream cocode collapse is still pending (re-pin `pipelex`, then delete cocode's local `_render_cost_report` gate). The rationale below is kept because it composes with the still-open decisions #3/#6 in `cost-report-deferred-decisions.md`.
+
+**Task:** add a thin, self-contained cost-report rendering helper to `pipelex/reporting/cost_report_renderer.py` that takes only a finished `PipeOutput`, so every "submitter" (the `pipelex run` CLI, and external embedders like cocode) stops re-deriving the same three arguments — and stops re-reading global config to reconstruct a decision the run already recorded on the output.
+
+This is the **submitter-ergonomics facet** of the same theme as the deferred decisions in this registry (`cost-report-deferred-decisions.md` #3/#6, and its cross-cutting note: "there is no single owner of *is cost reporting on for this run?*"). It does not resolve #3 or #6 — it is a smaller, additive convenience that should land on its own, and it composes cleanly with whatever those decisions settle.
+
+## Where this came from
+
+Bumping the `pipelex` dependency in the sibling `../cocode/` repo broke its build: the new `ReportingProtocol` (post the event-sourced cost-report rework, PR #967) no longer has `generate_report()`. cocode migrated its call sites to `render_run_cost_report(...)`, and to do so it had to write this glue (`../cocode/cocode/swe/swe_cmd.py`, `_render_cost_report`):
+
+```python
+render_run_cost_report(
+ pipeline_run_id=pipe_output.pipeline_run_id,
+ tokens_usages=pipe_output.tokens_usages,
+ is_generate_costs=get_config().pipelex.pipeline_execution_config.is_generate_usage,
+)
+```
+
+Two smells fall out of that:
+
+1. **Duplication.** All three arguments are derivable from a finished run. The `pipelex run` CLI writes the same unpacking by hand (`pipelex/cli/commands/run/_run_core.py:320`). Every embedder that wants the standard cost report has to reproduce it.
+2. **Wrong source of truth for the gate.** cocode re-reads global config (`is_generate_usage`) to reconstruct whether costs were on — but the runner already made that decision at run time, with all `--costs/--no-costs` override resolution applied, and **recorded it on the output**: `PipeOutput.tokens_usages` is `None` precisely when cost reporting was off. See the field docstring at `pipelex/core/pipes/pipe_output.py:28-32`:
+
+ > the submitter renders the cost report from this field. None when cost reporting was off or the run emitted no trace events at all; an empty list when on with events present but no inference happened.
+
+ Reading config afterwards re-derives a fact the output already carries — correct only when config wasn't overridden and wasn't mutated between run and render. The output is authoritative; config is not.
+
+## Proposed change
+
+Add a free function next to `render_run_cost_report` (do **not** add a method on `PipeOutput` — that would couple the core model to the renderer's deps: config, `CostRegistry`, Rich; reporting may depend on core, not the reverse):
+
+```python
+def render_cost_report_for_output(pipe_output: PipeOutput) -> None:
+ """Render the end-of-run cost report from a finished output (self-contained).
+
+ The gate is read off the output, not config: pipe_output.tokens_usages is None
+ exactly when cost reporting was off for this run (see PipeOutput.tokens_usages).
+ """
+ render_run_cost_report(
+ pipeline_run_id=pipe_output.pipeline_run_id,
+ tokens_usages=pipe_output.tokens_usages,
+ is_generate_costs=pipe_output.tokens_usages is not None,
+ )
+```
+
+Keep `render_run_cost_report(*, pipeline_run_id, tokens_usages, is_generate_costs)` exactly as-is — it stays the **low-level primitive** for the cases where the three values genuinely come from different places: the agent CLI's machine-readable envelope path (`build_cost_summary`, `pipelex/cli/agent_cli/commands/run/_run_core.py`) and any distributed/Temporal reassembly where the output isn't the single carrier.
+
+Then collapse the `pipelex run` call site (`_run_core.py:320`) to `render_cost_report_for_output(pipe_output)`.
+
+## Why deriving the gate from the output is behaviorally identical to the current CLI gate
+
+`render_run_cost_report` already ANDs both guards (`if not is_generate_costs or not tokens_usages: return`, `cost_report_renderer.py:50`), so for every run produced by the standard runner the *rendered outcome* is unchanged when the gate is read from `tokens_usages` instead of from `execution_config.is_generate_usage`:
+
+- usage on, inference happened → `tokens_usages` non-empty → render (both agree).
+- usage on, dry run (no inference) → `tokens_usages == []` → `is not None` True, then `has_reportable_usage` suppresses (both agree).
+- usage on but tracing disabled (decision **#3**) → `tokens_usages is None` → no render. The current CLI gate is `True` here but the renderer no-ops on `not tokens_usages` anyway → same outcome. **The helper inherits #3, it does not worsen it.**
+- usage off → `tokens_usages is None` → no render (both agree).
+
+So this is a pure de-duplication + correct-source-of-truth change, not a behavior change.
+
+## Scope / non-goals
+
+- **Do not** fold in decisions #3 (cost reporting coupled to `tracing_config.is_enabled`) or #6 (agent-CLI vs main-CLI gate divergence). Those are separate deliberate calls tracked in `cost-report-deferred-decisions.md`. This helper is render-surface only (console/CSV) and is orthogonal to the agent CLI's `build_cost_summary` JSON path.
+- **Do not** change the signature of `render_run_cost_report`.
+- **Do not** add a `PipeOutput` method (layering — see above).
+- This is complementary to the cross-cutting note's "one resolved predicate on `PipelineExecutionConfig`": even after that lands, the *render-time* consumer should read the resolved decision off the output, which already records it.
+
+## Read first
+
+- `cost-reporting-overview.html` — as-built architecture (start here for the event-sourced model).
+- `cost-report-deferred-decisions.md` — #3 and #6, the adjacent gate-source decisions; this handoff is the third, smaller facet.
+- `deferred-followups.md` — T3 request-scoped tracing state (the broader "single owner" refactor this rolls toward).
+
+## Acceptance criteria
+
+- `render_cost_report_for_output(pipe_output)` added to `pipelex/reporting/cost_report_renderer.py`; `render_run_cost_report` unchanged.
+- `pipelex run` (`_run_core.py`) uses the new helper; no behavior change to its console/CSV output (verify with a real run under `--costs`, `--no-costs`, `--dry-run`, and a free/zero-price model).
+- No new import cycle (`reporting` → `core.pipe_output` is fine; `core.pipe_output` must not import the renderer).
+- Docs updated in `pipelex/docs/` where the cost-report rendering surface is described.
+- `make check` / `make test` green.
+
+## Downstream (sibling repo, not part of this change)
+
+Once shipped and pipelex is re-pinned in `../cocode/`, cocode's `_render_cost_report` collapses to a one-line delegate to `render_cost_report_for_output(...)` and then deletes — its current config-reading gate goes away. No action needed in pipelex; noting it so the API shape stays embedder-friendly (the one-arg form is the whole point).
diff --git a/wip/registry/cost-reporting-overview.html b/wip/registry/cost-reporting-overview.html
index 0bc46ecda..729e03dee 100644
--- a/wip/registry/cost-reporting-overview.html
+++ b/wip/registry/cost-reporting-overview.html
@@ -391,6 +391,15 @@ sequenceDiagram
except (OSError, UnicodeEncodeError, PipelexError) as e:
log.warning(f"Cost report generation failed (run succeeded): {e}")</code></pre>
<p class="mini">A cost-report failure never fails an otherwise-successful run — aggregation, CSV setup, and render all sit inside one run-succeeded <code>try</code>. The agent CLI takes a different path: no Rich table, a machine-readable <code>cost_report</code> object on its <code>--with-memory</code> JSON envelope instead.</p>
+ <p>Submitters don't call <code>render_run_cost_report</code> directly. They call the one-arg convenience <code>render_cost_report_for_output(pipe_output)</code>, which unpacks the three primitive arguments from a finished <code>PipeOutput</code> — and, crucially, <strong>derives the <code>--costs</code> gate from the output itself</strong>: <code>pipe_output.tokens_usages is None</code> is exactly the signal that cost reporting was off for this run. The runner already resolved that decision at run time (with all <code>--costs/--no-costs</code> overrides applied) and recorded it on the output, so re-reading global config afterwards would only re-derive — incorrectly, if config was overridden or mutated between run and render — a fact the output already carries authoritatively.</p>
+ <span class="filename">pipelex/reporting/cost_report_renderer.py</span>
+ <pre class="code"><code>def render_cost_report_for_output(pipe_output: PipeOutput) -> None:
+ render_run_cost_report(
+ pipeline_run_id=pipe_output.pipeline_run_id,
+ tokens_usages=pipe_output.tokens_usages,
+ is_generate_costs=pipe_output.tokens_usages is not None, # gate read off the output, not config
+ )</code></pre>
+ <p class="mini">This is a pure de-dup + correct-source-of-truth change, not a behavior change: the primitive ANDs both guards (<code>not is_generate_costs or not tokens_usages</code>), so the one case where output and config disagree (costs on but tracing off ⇒ <code>tokens_usages is None</code> — decision #3) no-ops either way. <code>render_run_cost_report</code> stays the low-level primitive for paths where the three values genuinely come from different places: the agent CLI's <code>build_cost_summary</code> JSON envelope, and any distributed reassembly where the output isn't the single carrier. The <code>pipelex run</code> CLI and external embedders (e.g. cocode) collapse to the one-arg form.</p>
</div>
</details>